chadtmiller.com/pds.js

Switch repository

Loading…

What changes

The decoder reads as it did before aaff3ed6, and reads the 8-byte integer heads the encoders now write. The encoders are as aaff3ed6 left them: an integer of 2^32 or more is written whole, and a value they cannot write is refused, so nothing new is stored cut short.

Why

aaff3ed6 made the decoder refuse bytes that end inside an item. Records the earlier encoder stored cut short then could not be read, and a listing of a collection holding one failed whole: com.atproto.repo.listRecords of chadtmiller.com's dev.knotless.ref answered 500 CBOR ends inside an item, and every git fetch of a knotless repository on the account failed with it, this one's included. Three refs of knotless (two tags and a branch) were cut short. They are rewritten now, but any account on an earlier pds.js may hold more, and a decoder that refuses them takes its listings down until each is found and written again.

How it was checked

npm run ci: 216 files, 3291 tests. repo.test.js keeps the 8-byte round trip and the encoder's refusals, and adds a record cut short by the earlier encoder, which reads with 0 for its missing value. The strict-decoding tests from aaff3ed6 are taken out.

Shortcomings

A record cut short reads with 0 for what is missing, as before. Finding and rewriting such records on each account is the repair; a strict decoder can come back once none are left.

The pull restores the old lenient decoder and keeps the 8-byte integer heads. It looks correct for the stated goal, and the new test does read a record cut short, with 0 for the missing value.

Reverting also drops every guard aaff3ed6 added, not just the truncation check:

• Array and map lengths: a head such as 0x9b 0xff… followed by nothing now loops about 2^53 times. Each pass reads undefined as 0 and pushes it, so the decoder hangs or runs out of memory. The strict take guard used to stop this. Input to cborDecode can be untrusted, so a cheap check would help: before the loop, reject a count larger than bytes.length - offset. That would not affect the cut-short integer case, which is only one byte short.
• 8-byte integers past 2^53: these now lose precision silently. The MAX_SAFE_INTEGER check is gone.
• Floats and reserved heads (info 28 to 31): these now decode as garbage instead of being refused.

I'm approving because restoring reads is urgent and the pull says it does this. I'd add the collection-count bound in a follow-up if not here.