chadtmiller.com/knotless

Switch repository

Loading…

What changes

The harness PDS commits to SQLite without waiting on the disk. harness/pds.mjs patches the npm PDS's Database.sqlite (the way it already has the alpha migrate once) so every database it opens, an account's, the account manager's and the sequencer's, runs with synchronous = OFF. The PDS code is otherwise the npm one. KNOTLESS_PDS_FSYNC=1 keeps it as it is. docs/development.md says so.

Why

The harness PDS's writes cost 150 to 350 ms at the median in CI against a few milliseconds for a read, and createAccount 0.8 to 1.7 s. Measured on the runner, an fsync of 4 KiB in /tmp takes 20 to 23 ms. A PDS write is a commit to several SQLite databases, each waiting on its own fsync, and every test binary shares two PDSes, so those waits stack up under load. A harness has no data to keep through a crash.

How it was checked

CI's test check, which runs every test against the patched PDS. Before and after times are in a comment.

Shortcomings

It does not help the share of time the PDS spends hashing passwords (createSession is about 400 ms under load) or signing.

Changeset: none

This patches Database.sqlite in the PDS's db/db.js before the network starts, so each SQLite database opens with synchronous = OFF. KNOTLESS_PDS_FSYNC turns it off, and docs/development.md says so. It follows the existing migrateOnce approach.

• pdsDist resolves @atproto/pds from the dev-env's own location, so the patch reaches the same module instance the PDS loads, including when the dev-env is hoisted or nested.
• The wrapper merges opts.pragmas and then overrides only synchronous, so the PDS's other pragmas stay.
• Losing data in a crash is fine for a test harness.

I could not run it, and I did not check that the npm PDS's Database.sqlite takes pragmas in opts. The test check on this pull will show a mismatch, because the patch would then silently do nothing or fail at startup.

The doc sentence is a little long, but it is accurate.

Before and after, CI, same run (one harness PDS with fsync as on main, one without), server side p50/avg ms per call: applyWrites 175/197-361 now 13/19; putRecord 175/294 now 11/20; createAccount 1009/1230-1810 now 120-150/137-168; uploadBlob 250/410 now 15/28; getRecord and listRecords p50 7-10 now 3-5; node event loop p99 lag 120-780 ms now 10 ms. Whole test check, coverage in: 1046 s, 1095 s, 1258 s on main and an earlier pull, 437 s here. spaces run 270-754 s now 223 s, forge 182-377 s now 131 s, review 357 s now 159 s, mcp 208-276 s now 64 s, gateway 267-358 s now 143 s. No flaky test this run.