chadtmiller.com/knotless

Switch repository

Loading…

What changes

A space type for organizations. An organization keeps its invitations and hidden members in at://<org>/space/dev.knotless.orgSpace/self, and lexicons/dev/knotless/orgSpace.json is that type's space lexicon: key literal:self, the name "Organization", and its two collections, orgMember and orgAcceptance. harness/lexicons.mjs checks it as it checks space.json, and npm run lex in web/ leaves it out as it does that one. publish-lexicons publishes it when this lands.

The move. A space's type is part of its URI and no call changes it, so each organization's records are moved from its old space (dev.knotless.org) into a new one. knotless::org::recorded, which every change through the gateway opens with, moves first (moved):

1. makes the new space, keeping one that is there;
2. puts each account the old space holds in the new one, with its read and write, unless the new one has it;
3. writes each orgMember record from the old space into the new one, unless the new one has one for that account (written since, so it counts), and only then deletes them from the old.

An old space with no orgMember record left has nothing to move, so a second run does nothing, and a run cut off part way is finished by the next. The old space is never deleted by the move: org delete deletes both. As it starts, knotless-serve also moves every organization whose public record names it as gateway (App::move_org_spaces), one at a time under the organization's lock, then reindexes each.

Hidden members' join records. A hidden member's orgAcceptance is in their own repo in the old space, which only they can write. It stays there and counts until they join again: the gateway's admit_as reads a join record from either space, and the index counts a record of joining in the old space for a record in the new one (Index::org_roster_with, orgs_of). org join --hidden now writes into the new space and deletes the one in any other place, the old space included; org leave deletes from all three.

Readers of both. The index's org standings read both spaces, the new one winning for an account in both. keep_org keeps both. resync syncs the old space while it is followed, which an organization made since never had. A space's authority may tell of writes in either (from_authority).

How an operator knows it finished. Each start logs org spaces moved (count moved or already moved, failed not moved, each also logged as org space not moved with why) and org joins in old spaces (count, the hidden members' records still there). The organizations' part is done when failed is 0. Documented in docs/design/gateway.md (Moving out of the old space), with access.md and serve.md pointing there.

Why

The next pull narrows both OAuth scopes from space:* to knotless's own space types. The spaces alpha (11c5d98c) resolves the lexicon of every space type a login's scope names on the authorization page, and refuses the login (invalid_scope) for one that is not a space lexicon. dev.knotless.org is the organization's record lexicon, so a scope naming it would refuse every login.

What I checked in the alpha:

• createSpace checks only the caller's scope (assertSpaceRef(ref, {manage: 'create'})). It resolves no lexicon, so the move runs at once on deploy, with the gateway's grant as it is (space:*?authority=self&…&manage=create&manage=update). Only naming the type in a login's scope needs it published.
• No call changes a space's type (updateSpace replaces only the policy), so moving means a new space.
• A member's PDS takes writes to their repo in any space without asking its authority, so a hidden join into the new space always lands.

So the order is: this pull, then publish-lexicons on main, then the narrowed scopes. This pull asks for no new scope, so no login breaks in between.

How it was checked

organizations_move_out_of_old_spaces (spaces alpha only) builds an organization as the old code left one: its old space with a hidden member, an invitee and the member's own join record there. Before the move both stand as they did. After move_org_spaces the new space holds both records and the old one none, both stand as before, and the join record counts from the old space. A second run moves nothing. An organization naming the gateway that never delegated fails and is logged. Joining again moves the hidden member's record (the old space holds none after), an invitee joins hidden in the new space, and org delete ends it. The index unit test covers a join record in the old space counting. node harness/lexicons.mjs passes. The rest is left to CI.

Shortcomings

• A hidden member who never joins again keeps their join record only in the old space. It counts while the index has it. The index keeps an old space it followed, but one built anew finds it only by keep_org. Their count is in the log.
• An older client running org delete as the organization deletes only the old space, and leaves the new one with its records. Nothing reads those, as the organization's public record is gone.
• The scopes still say space:*. Narrowing them, and loading both lexicons into the test alpha's lexicon store, is the next pull, once this has landed and publish-lexicons has passed. That pull also has to keep org delete working when a narrowed login cannot delete a dev.knotless.org space, and keep the old spaces synced as long as join records are left there.

Review

Start at moved in crates/knotless/src/org.rs, then org_standings and move_org_spaces in crates/knotless-serve/src/orgs.rs.

🤖 Generated with Claude Code

This pull gives organizations their own space type, dev.knotless.orgSpace, and moves each organization's records out of the old dev.knotless.org space. The move runs from moved in org.rs and at serve start. I read moved, recorded and move_org_spaces. I did not read the whole 1,000-line diff, because the tool saved it to a file I couldn't open. I did not review the index changes, the CLI changes, the lexicon, the docs or the tests.

What I read looks correct:

• Order of writes: moved writes each record into the new space before it deletes it from the old one, so a reader of both never misses a record.
• Rerun and resume: a second run does nothing, and a run cut off part way is finished by the next.
• Newer new-space record wins: a new-space record written since the move began is kept and not overwritten.
• Concurrency and failure: move_org_spaces takes the per-org lock, and a failure is logged and counted without stopping the others.

The CI checks (test, check, lexicons, docs, web, e2e) were still queued or running when I looked, so I can't confirm they pass. Please wait for them before merging. The pull's own note that the narrowing PR must keep org delete and old-space syncing working is a real follow-up, so keep it on the list.

This round's only change is the new commit 0bfc496, "Clippy: a stale join record is one written now". The earlier head 177baf1 failed check and test, which is why the merge queue ejected the pull. I take this commit to be the fix for that. I did not re-read the whole diff, so my verdict from round 1 stands: moved writes before it deletes, a rerun does nothing, and move_org_spaces runs under the organization's lock.

The checks on 0bfc496 were still queued or running when I looked, so none has passed yet. Wait for check, test, lexicons, docs and e2e to pass before queueing the merge again. The test run matters most, because it covers organizations_move_out_of_old_spaces. If test fails again, the failure is in more than clippy.

The only change since my last round is the new commit f057c0a, "Covered: a move cut off part way, and asking an organization with no gateway". From its message it adds tests for two cases: a move cut off part way, and an organization with no gateway. I did not re-read the full diff. My earlier points still stand: moved writes to the new space before it deletes from the old one, a rerun does nothing, and move_org_spaces runs under the organization's lock. Nothing I've seen needs changing.

None of the checks on f057c0a has finished. check is running and test, lexicons, docs, web, shots and e2e are queued. test matters most, because it runs organizations_move_out_of_old_spaces and the new cases. The earlier head failed check and test, so wait for both to pass before the merge goes back in the queue.