chadtmiller.com/pds.js

Switch repository

Loading…

What changes

Integers of 2^32 or more are written. encodeHead in packages/core/src/repo.js had no case past four bytes, and wrote nothing for a larger value, so cborEncode and cborEncodeDagCbor left a map key with no value. It now writes an 8-byte head (CBOR's info 27) up to Number.MAX_SAFE_INTEGER, and throws a RangeError past it.

What an encoder cannot write is refused. A number that is not a safe integer, a BigInt, or undefined in an array threw nothing and wrote nothing. Each now throws a TypeError naming it.

The decoder refuses bytes that do not hold an item. It read past the end of its input, where a missing byte comes back undefined and reads as 0, so a record cut short decoded with plausible values. It now reads 8-byte integers, and throws for an item that runs past the end, for floats, and for indefinite or reserved heads.

Two git-host tests pass CBOR, as putRecord documents. They passed JSON text, which decoded only because the decoder read anything.

Why

knotless starts a new ref's generation at the time in milliseconds (keys::first_generation), about 1.79e12. On this PDS every new ref in a space was stored without that value, under the CID of the shortened bytes: refs/heads/main of chadtmiller.com/group-host is 196 bytes where its encoding is 197, and its stored CID, bafyreihr4u…, is the hash of the 196. knotless-serve's sync of a private repository's space then failed on it, every time (a record that does not come to its CID, then a record that is not a DRISL map), so no pull, issue or check of that repository reached its index. Reading a ref back, the decoder made the missing value 0, and the next push wrote generation 1: a ref's generations started again, which knotless relies on not doing across a ref's lives.

Found from that repository: the record's bytes over space.getRepo, space.getRecord's CID for it, and the encoding of the same record by this encoder as it was (197 bytes, in Node and on workerd under wrangler dev).

How it was checked

npm run ci: 216 files, 3292 tests, biome, typecheck, coverage. packages/core/test/repo.test.js adds a knotless ref with a millisecond generation, which both encoders write and the decoder reads back exactly; the 8-byte heads at 2^32 and at 1793736300000; each value the encoders refuse; and the decoder refusing bytes cut short, a float, an indefinite head and an integer past MAX_SAFE_INTEGER.

Shortcomings

Records already stored cut short stay so until written again: on this PDS, any space record created with an integer of 2^32 or more. A ref is mended by its next push. Reading one now throws where it decoded to a guess, which makes a getRecord of one fail rather than answer wrong.

The pull adds an 8-byte CBOR head (info 27) to encodeHead, so integers from 2^32 up to MAX_SAFE_INTEGER are written. Both encoders now throw on values they cannot write. The decoder reads 8-byte integers and throws on truncated input, floats, and indefinite or reserved heads. I read the encoder and decoder changes and the new tests, and found nothing wrong. I couldn't read the base branch's AGENTS.md (the read failed), and I didn't read the checks.

The 8-byte head arithmetic is correct: the high and low words come from Math.floor and %, and the tests pin the exact bytes for 2^32 and 1793736300000. take() bounds every read, and a huge array or map length ends in a thrown truncation error rather than a runaway loop.

One behavior change to be aware of: plain cborEncode doesn't filter undefined object values the way cborEncodeDagCbor does. Such a value used to produce a key with nothing after it and now throws. That is the right outcome, but any caller passing optional fields through cborEncode will now fail loudly. I saw no such caller in the diff.