chadtmiller.com/knotless

Switch repository

Loading…

What changes

A change syncs the private mirrors viewers are reading, in the background. When the space's sync finds something written there (a push, a runner's check, an issue), or the owner's account changes, the server starts a sync of every viewer's mirror of that repository whose viewer was proven within the minute. There is one task per mirror at a time: a change that comes while one is waiting does not start another.

A page's read is served the mirror as last synced while that sync runs. The reads of a repository's code that 744499d2 let public repositories answer without waiting (getRepo, getTree, getLastCommits, listBranches, listTags) now do the same for private ones. Each viewer's mirror as last synced is kept apart from the lock its sync holds (App::viewed_last). It is served for ten seconds from the first change, under the same rule (serves, which now takes whether the mirror is quiet rather than an event stamp). It is served only while the space is followed and was synced lately, and only to a viewer proven within REPROVE, the same minute that already let a read skip asking the owner's PDS. A read served this way still makes its reader the space's syncer (keep), as every read did. A viewer whose proof has lapsed is proven again first, then served the same way, and that read starts the sync itself. Concurrent reads of one repository do not even queue on its lock now, and a sync of one mirror is shared by every read that comes during it.

Read-your-writes is kept. A write from the pages bumps the owner's writes, so the reads after it wait for its sync, as before. A read that comes with a delegation token always waits too: that is how the command line shows the appview what it just pushed (show_appview), and how clients read.

located_for_reader takes the Arc<App> it spawns with, so Repo::find_for_reader and found_for_reader do too, and web/repo.rs parses the record URI in a shared repo_uri. The rest of as_viewer is the same, split into fresh_viewed and resync, which the background sync reuses.

Why

From admin_logs --message "mirror sync" (the last 500 lines, 06:48 to 19:14 UTC on Nov 10), with each line's trace read back:

• 124 were spec-pds (did:plc:bcgltzqazw5tb6k2g3ttenbj, private), 118 of them inside a request: 100 from the browser and 17 from knotless/0.12.0, its show_appview after a push.
• Of the 100 browser syncs, 79 fetched nothing past the records (duration_ms - records_ms < 50 ms): nothing in the repository's git had changed.
• Browser requests that synced: median 385 ms, max 2.2 s. CLI ones: median 1.7 s, max 4.9 s. The earlier findings had getRepo for spec-pds at a 2.6 s median and up to 7.1 s on Nov 9, with all five of a page's requests waiting the same sync. Example: trace 25a2dc4925e8257054ffa42f73c84c6f (sync 2341 ms, getRepo 2884 ms).

What triggers them: the space's moves. The runners file their checks into the space (knotless-run writes dev.knotless.check with com.atproto.space.putRecord). Each write is announced through notifyWrite, spaces::follow syncs the space, finds a write, and calls App::moved, so the private mirror counts as stale. The page's /_live/repo sees the index change and draws the page again, and its five reads synced inline. The browser syncs line up with spec-pds's runs: syncs at 18:47:26 and 18:47:48 around a run at 18:47:44; at 18:59:13, 18:59:21, 18:59:42, 19:00:23 and 19:00:27 around runs at 18:59:09, 18:59:38 and 19:00:21; at 18:02:32 to 18:02:54 around runs at 18:02:31 and 18:02:50. Changes in the owner's account (heard_of) mark it stale the same way. TTL and the viewer's proof are not triggers: the space is followed, and a lapsed proof only asks for a credential again. No space not synced lines name spec-pds. Public repositories avoided this since 744499d2, but private reads went through as_viewer, which always synced inline, and nothing refreshed a private mirror in the background: heard_of spawns refreshes only for App::repos, the public entries.

Making the space's moves finer, so that a check written there did not count as a change to the mirror, would cut the syncs themselves. But moves is counted per space, not per writer or collection, and that is a larger change than this one. It does not rule it out.

How it was checked

a_stale_mirror_is_served_while_it_syncs is updated for the new serves signature. private_repositories_indexed_here already reads a private repository as a member, then pushes and waits for the push to show, which now goes through the background sync and the served-stale read. It gains a second push with reprove at 0, so the read that proves the viewer again is served the mirror as it was and starts the sync, and the push shows once that sync is done. Nothing was built or run here (cargo fmt only); the rest is left to CI.

After this deploys, knotless admin logs --message "mirror sync" should show spec-pds's syncs without a trace_id, outside page requests, except right after a write from the pages, a read whose viewer must be proven again more than ten seconds after a change, and the command line's own reads.

Shortcomings

• A push is now seen on a page once its background sync finishes, not by the first read after the space's sync finds it. A page drawn again within that time shows the mirror as it was, and the next look shows the push. For a public repository the owner is exempt from this. For a private one the owner is nearly every reader, so they are not: their pushes from the command line are synced by its own delegated read.
• A viewer not proven within the minute when the change came gets no background sync. Their next read proves them again and, within ten seconds of the change, is served stale while it syncs. After that, it waits as before.
• Each viewer has their own mirror, so a change syncs one mirror for each viewer proven within the minute, not one for the repository.
• A background sync that fails leaves the stale mirror served until its ten seconds are up, then reads wait again, as with public repositories.