~ knotless Search/
↑↓ move↵ open⇧↵ all results

chadtmiller.com/lex-space-notes.md

Notes on @atproto/lex-space

0
0
1 file · 0 comments
History
Code
lex-space-notes.md 84 lines · 6.1 KB Raw

Notes on @atproto/lex-space

@atproto/lex-space everything worked pretty well! only major thing that came up was around the scalability of allRecords but I don’t think that’s a big deal right now.

Here's some notes from my agent.

Claude Opus 5.5

From two pieces of work on @atproto/lex-space:

  • building a Svelte book club app against a local dev-env network (PLC, two PDSes), with OAuth and space: scopes. Members share reviews only with each other.
  • measuring how an app on a group host scales when the browser reads every member's repo in a space itself (allRecords), with no index service in between.

References:

Overall: the core works, and it's well designed. Most of the friction is ergonomics, and the cost of reading a whole space.

What's good

  • Typed by collection. SpaceClient's typed helpers only accept the collections the space type declares, and that's checked at compile time. allRecords({ collections }) narrows each record by type.
  • The hard parts are built in. Delegation tokens, credential exchange and renewal, routing each request to the right host, MAC verification, and retries after a key rotation or a 401. None of it surfaced in app code.
  • The README is excellent. It covers management, building a sync service, and its own limitations, honestly.

Reading a whole space (allRecords)

Measurements

Against a local group host world, debug builds, on a loaded machine:

RoomCold openDownloadedClient holdsViews in the client8 readers at once
30 records0.25–0.6 s15 KB0.3 MBunder 0.1 msunder 1 s each
2,065 records0.5–0.8 s0.7 MB8 MB2 ms or less1.6–2.2 s each
19,175 records3.3–3.6 s, 3.2 s of it client CPU checking records5.3 MB72 MB3–20 ms15–17 s each

Working out views in the client (counts, threads, topics by latest reply) is never the cost. Getting and checking the records is.

Issues

  1. getRepoSnapshot hashes every record twice. Reading a repo, it hashes each record once to check the records, and again to compute heads. At 19k records, nearly all of a cold open is that client CPU. Checking without computing heads would roughly halve it (an estimate, not measured).
  2. allRecords keeps nothing between calls.
    • Every reader pays the full read on every open, and each repo is held in memory whole (README limitation).
    • Reading on is cheap at any size: about 6 requests, 6 KB and 150 ms after new writes, and one request when nothing changed. lex-space already has what it needs (writers({ cursor }), and pullRepo from kept heads), but allRecords doesn't use it.
    • The fix is to keep each repo's head and records, in memory while the page is open and in IndexedDB between visits, and read on from there. Then a member pays the full read once per device. This matters more than #1 at scale.
  3. One host's failure fails the whole read.
    • Repos are fetched 4 at a time with Promise.all, so each group waits for its slowest host.
    • Only RepoNotFound, RepoTakendown, RepoSuspended and RepoDeactivated are skipped. A network error or 5xx from any one member's host fails the whole iteration.
    • There's no option for partial results, or for per-repo error reporting.

Where that leaves an app

The browser alone serves groups up to a few thousand records in a space. Past that, an app should read from an index service: one that registers as a syncer of the space (authorizeSpaceSync with notify), keeping counts, links, orders and hiding.

Other library issues

  1. The README's main example doesn't type-check against generated code. For a space def, lex build only emits $nsid and main on the namespace. So SpaceClient.openSpace(com.example.forum, …) as documented fails to compile, and you have to pass com.example.forum.main. Record namespaces work as documented, since they expose $type, $build and so on.
  2. Argument styles vary between constructors.
    • SpaceClient.openSpace(schema, ref, { agent, ...options }) takes the agent in its options.
    • SpaceClient.createSpace(agent, schema, { policy, skey }, options) takes the agent first, and splits input and options into separate arguments. It's easy to put didResolver in the input object; TypeScript does catch that (TS2353).
    • SpaceManager.open(agent, uri, options) accepts an Agent, and checks at runtime that it's signed in.
    • new SpaceManager(agent, uri) requires an AccountAgent, so passing a plain client.agent fails to compile (TS2345). The tests work around it with an agentOf() helper.
  3. Credentials live in memory only, per SpaceClient. Every page load pays getDelegationToken plus authorizeSpaceSync (two round trips) per space before the first read. Apps also need their own one-client-per-space map, because a new instance exchanges again. @atproto/oauth-client lets you plug in session and state stores; a credential store hook here would match.
  4. No client-wide DID resolver default. It defaults to plc.directory. In development, every openSpace and createSpace needs didResolver passed explicitly.

Checked and fine

  • Removing a member with MemberspacePolicyManager.removeMembers() drops their records from allRecords() immediately. No separate removeWriter() is needed for reads.

Suggested follow-ups

ItemKind
allRecords: keep heads and records, and read on with writers({ cursor }) and pullRepoAPI
getRepoSnapshot: hash each record oncePerformance
allRecords: per-repo errors, a partial-results option, a sliding concurrency windowAPI
Generate schema accessors on space namespaces, or fix the READMECodegen / docs
One argument convention across openSpace, createSpace and SpaceManager; accept Agent everywhereAPI
Credential store hook, and a shared client-level SpaceClient cacheAPI