getRepo lets other requests run while it reads the repo

What changes

com.atproto.sync.getRepo, and the backup snapshot that shares buildFullRepoCar, wait on a timer after every 500 blocks the walk reads. Other requests run in those gaps.

Why

On Cloudflare one Durable Object answers every request for the account. buildFullRepoCar awaited only promises already settled, so it never let anything else in until the whole repo was read.

microcosm's lightrail fetches the whole repo after any commit that deletes a collection's last record. That's a bug in lightrail: it can never prove such a death, so it resyncs. In prod on 2026-10-29, wrangler tail showed requests to chadtmiller.com's PDS waiting about 3.6 s behind lightrail's getRepo, then taking 14 ms once they ran. Organization clicks on knotless that came straight after a write took 8.6 s and 11.8 s instead of about 4.5 s.

How it was checked

A new test in test/xrpc-sync.test.js runs getRepo over a 601-record repo, with a timer set to fire while it runs. The timer runs before getRepo ends. Without the change, it runs after.
npm run ci passes: 3242 tests, within the coverage floors.

Benchmarked locally against better-sqlite3:

Repo	getRepo before	getRepo after	A request arriving mid-read waits
20,000 records, 7 MB	about 200 ms	about 250 ms	before: until getRepo ends; after: 1–23 ms
60,000 records, 22 MB	about 650 ms	about 790 ms	before: until getRepo ends; after: 15–18 ms

I also tried reading the blocks a tree level at a time, 100 to a query. It was no faster locally, and slower overall, so the walk keeps getBlock.

Shortcomings

That a timer lets a Durable Object deliver another request is checked in Node, not in workerd. The check in workerd is prod: wrangler tail during an organization click after this deploys.
getRepo still builds the whole CAR in memory before sending it.

🤖 Generated with Claude Code

chadtmiller.com/pds.js Switch repository Loading…