~ knotless Search/
↑↓ move↵ open⇧↵ all results

chadtmiller.com/pds.js

An atproto PDS in JavaScript, for Cloudflare Workers and Node

0
0
main
357 lines · 16.7 KB Raw History

Permissioned data

An implementation of proposal 0016: private per-user repos, shared through spaces.

A space has one authority and a set of members. Each member writes to their own private repo on their own PDS. The authority holds the writer set, and it issues the credential that lets a reader pull every member repo. The PDS stores and serves those repos. It does not assemble them. That job belongs to the application.

A space's policy decides who may hold a credential and whose writes the authority records. @pdsjs/spaces provides LtHash commits, MACed reads, space credentials, space: OAuth scopes, and the com.atproto.space, com.atproto.memberspace and com.atproto.managedspace methods. Node, Deno, and Cloudflare Workers all support it.

Telling a client spaces are here

com.atproto.server.describeServer answers spacesAlpha: true when PDS_ENABLE_SPACES is set, as the reference PDS does. community.lexicon.service.describe also answers with this server's roles and every XRPC method it routes, so the space methods appear in that list exactly when the feature is on.

Policies

A space holds exactly one policy, set by createSpace and replaced whole by updateSpace. The authority always reads and writes its own space, whatever the policy says.

PolicyWho readsWhose writes count
com.atproto.personalspace.policythe authority alonethe authority alone
com.atproto.memberspace.policyanyone under publicRead, else members with readanyone under publicWrite, else members with write
com.atproto.managedspace.policywhoever managingApp admitswhoever managingApp admits

A memberspace may also name allowedApps, the client_ids that may receive a credential. Absent, any client may. A managedspace asks its managing app on each credential request, through com.atproto.managedspace.checkUserAccess with service auth for that method. A managing app that does not answer, or answers no, admits nobody.

com.atproto.memberspace.putMember, removeMember and listMembers work only on a memberspace. On any other policy they answer PolicyMismatch. A member added without write, removed, or changed to lose write leaves the writer set at once, and com.atproto.space.removeWriter does the same for any writer. A change of policy type drops every member.

com.atproto.space.getSpace answers the authority with the space's URI and its policy. Nobody else may read the policy this way.

Published policy and members

A space may publish its policy to its readers. With publishPolicy set, the authority writes the policy as com.atproto.space.policy/self into its own repo in the space. A memberspace with publishMembers set also writes one com.atproto.memberspace.member record per member, keyed by the member's DID. The server keeps these records in step with the policy and the member list. Nobody writes those two collections directly: createRecord, putRecord, deleteRecord and applyWrites refuse them with InvalidRequest.

Clearing a repo

com.atproto.space.clearRecords deletes every record the caller holds in a space, in commits of at most 200 writes. It needs a delete scope for each collection it touches. In the authority's own repo it leaves the published policy and member records in place.

Spaces made before policies

A database from the October 22 alpha holds each space's read and write rules in separate columns. On first start the server converts each space it is the authority of. A space with a managing app on either side becomes a managedspace, and its members are dropped, since its app decides access. Any other becomes a memberspace: public read or write carries over, an app allow list becomes allowedApps, and members keep their rights. Neither kind publishes anything until its authority asks. The old columns stay in the table and nothing reads them.

Not implemented

The reference also moves spaces between servers with an account: importRepo restores space records, and exportSpaceMeta and importSpaceMeta carry a space's policy and members. pds.js implements none of the three.

scripts/space-reference/ tracks the atproto PR; ngerakines.me/atproto-crates is the other implementation in use.

Getting a credential

An application asks the space's authority for a credential with com.atproto.space.getSpaceCredential. The request is not signed. Its body names the space and credentialKid, a fresh P-256 did:key the application holds the private half of. The authority puts that did:key in the credential's cnf.kid. The key needs no registration.

{
  "space": "at://did:plc:.../space/com.example.forum/general",
  "credentialKid": "did:key:zDn...",
  "delegationToken": "<from the user's PDS>",
  "clientAttestation": "<optional>"
}

The delegation token comes from the user's own PDS through getDelegationToken, addressed to <authority DID>#atproto_pds and signed with the user's #atproto key. It says which user the application acts for, and a space whose policy admits only some users needs one. A memberspace with publicRead issues a credential without one. The authority checks the user before the application. A user the policy does not admit, or a missing token on a space that is not public, answers SpaceNotFound, the same as a space that does not exist, so the request cannot learn that it does. An admitted user through an application the space does not allow answers AppNotAuthorized. A deleted space answers SpaceDeleted either way. A token that does not verify, or one already spent, is InvalidDelegationToken.

A credential lives ten minutes, and never more than an hour. A delegation token buys one credential. The authority spends its jti and refuses a second presentation. A client attestation is spent the same way.

Presenting a credential

Every request presenting a credential uses the Atproto-Space-Credential scheme and names the account it is addressed to in Atproto-Space-Audience. For a repo read that is the repo's owner, and for listRepos, registerNotify and unregisterNotify it is the space's authority. An RFC 9421 HTTP message signature by the cnf.kid key covers both headers, in that order, under the label and tag atproto-space-credential.

Authorization: Atproto-Space-Credential <credential>
Atproto-Space-Audience: did:plc:...
Signature-Input: atproto-space-credential=("authorization" "atproto-space-audience");tag="atproto-space-credential"
Signature: atproto-space-credential=:<64-byte r||s, padded base64>:

A host refuses a signature made for another account, so a host given a credential to serve one repo cannot present it to another. A signature covers nothing else about the request, so the application may reuse it for the same credential and audience until the credential expires.

Reading with a MAC

A space commit carries no signature. It names the repo, the space, the rev, and data, the SHA-256 of the repo's LtHash state. The repo host authenticates what it serves to each syncer with a MAC instead, under a secret only the two of them share.

Every account has a P-256 key published as #atproto_kem in its DID document. A syncer reading a repo makes a fresh P-256 key for the request, derives a secret against the author's KEM key (DHKEM(P-256, HKDF-SHA256), RFC 9180 §4.1), and sends its public half:

Atproto-Space-Want-Mac: type="atproto-space-repo", ver=1, key_share="did:key:zDn..."

The host derives the same secret from its KEM private key and answers with the MAC of the commit, or of the record for getRecord (type="atproto-space-record"):

Atproto-Space-Mac: type="atproto-space-repo", ver=1, mac=:<32 bytes, padded base64>:

A request presenting a credential must ask for a MAC on getRecord, getRepo, getLatestCommit and listRepoOps; without one, or with a header that does not parse, the host answers BadSpaceMac. The account's own session may ask and need not. listRecords carries no MAC. Each such response has Vary: Atproto-Space-Want-Mac, and exposes Atproto-Space-Mac to browser apps across origins.

getRepo answers with the repo as a STAR archive (application/octet-stream): the commit as the archive's metadata, then each record's path and DAG-CBOR bytes in byte order of the path, a control byte before each record and one to end the archive. A syncer verifies the commit against the MAC, folds each record into a set hash, and trusts the records only when that hash matches the commit's data. A record's DAG-CBOR encoding may be at most 1 MiB.

A space record's JSON {$link} is stored as a CID link and {$bytes} as a byte string, so a record's CID is the one any implementation computes from the JSON getRecord serves.

KEM keys

An account created on pds.js gets its KEM key with its first DID document, in the genesis PLC operation. An account that predates the key gets one at startup and hourly after: the server publishes it with a PLC update when it holds a rotation key for the account and the account's PDS is this server, and otherwise stores it and logs a warning for the owner to publish. com.atproto.identity.getRecommendedDidCredentials includes it, and submitPlcOperation refuses an operation that names another one. A did:web account publishes the key from getRecommendedDidCredentials itself.

Revoking a credential

A space's authority may tell a repo host that credentials it issued are revoked before they expire, with com.atproto.space.notifyCredentialRevoked. The body names the space and up to 100 credential ids (jti). The call takes service auth issued by the authority and addressed to an account the host serves. The host refuses a revoked credential with CredentialRevoked, and keeps each revocation for an hour and ten seconds, which outlives every credential it can match.

Which application is asking

A delegation token says the user delegated to somebody. A client attestation says who: a JWT the client signs with its own authentication key. The authority resolves the iss to its client-metadata.json, fetches the key set that document publishes, and checks the signature against the key its kid names.

A memberspace's allowedApps list rests on that check. Without it the client_id would be a claim the caller writes for itself, and the list would state a rule the authority could not enforce.

Bearer is refused. One credential reads a whole space and its holder presents it to every repo host in that space, so as a bearer token it would be a shared secret: a host given one to serve its own repo could replay it against every other host.

Write notifications

A space's writes never reach the firehose. A service that follows a space registers with the authority through registerNotify, naming itself by a service identifier whose DID document publishes the endpoint to deliver to. The authority then forwards a notifyWrite to it for every write it learns of, including writes into its own repo, where no notification arrives over HTTP for it to forward.

A notice carries the repo rev and commit data a repo reached, and no records. The syncer reads with its own member's session, which is the only thing that can. A registration lasts a day, and a lapsed one stops receiving notices, so a syncer renews before it expires.

The authority puts every update it accepts on one line for the whole space. A forwarded notice carries the spaceRev the authority gave the update and the prevSpaceRev of the update before it, absent on the space's first. A notice at or behind the writer's last accepted repo rev is not forwarded again, and a repo rev more than five minutes ahead of the authority's clock is refused with FutureRev. listRepos lists writers in spaceRev order. Its cursor is a space revision, every page with entries ends on one, and only an empty page has none. A syncer that sees a gap in prevSpaceRev catches up from its last spaceRev through listRepos.

The authority applies the space's policy to each notice it receives. A writer the space does not admit stays out of the writer set, so listRepos never names them and no syncer pulls their repo.

A repo host owes the authority a notice for every write into its space. It stores the notice before it sends it. A notice that fails for a reason that may pass, a network failure or a 5xx, 408, 425 or 429 answer, is retried with backoff from ten seconds up to an hour. Retries stop when the notice lands, when a newer write to the same space replaces it, or a day after that write. Any other answer stops them at once. On Node and Deno a ten-second interval retries what is due. On Cloudflare the Durable Object's alarm comes forward to the next retry.

Forwarding a write from the authority to a syncer is best effort. Sync correctness rests on comparing commit data, so a missed notice is recovered by the next one or by the syncer's own catch-up through listRepos.

Deleting a space is different, because no later notice follows it. The authority owes every registered syncer a notifySpaceDeleted, stored and retried with the same backoff until it lands or the syncer's registration expires. A syncer that still misses it learns from SpaceDeleted on its next credential request.

A write by the authority into its own space needs the space to exist. Into one it never created, or one it deleted, the write is refused with SpaceNotFound.

Blobs

A space record's blobs are ordinary account blobs. They are uploaded through com.atproto.repo.uploadBlob and referenced from the record, so a client writing blob-bearing records needs a blob permission alongside its space permission. com.atproto.space.listBlobs and com.atproto.space.getBlob read them back within the space.

Two things guard them from orphan cleanup, which deletes any blob no record points at. A space write records its references in the account's one blob link table, beside those of public records, so the reaper's primary query already counts them. And collectReferencedBlobs, the cross-check that runs before anything is deleted, reads space records as well as public ones, so a link that was never written cannot cost a blob either.

getBlob serves only a blob the named space references. A credential admits its holder to that space, not to the account's whole blob store. The reference implementation is looser here.

The public endpoints hold the same line from the other side: com.atproto.sync.getBlob refuses a blob that only space records reference, and sync.listBlobs leaves it out. One window stays open by design: a blob between its upload and the write that references it has no links yet, and the public write flow needs such blobs served. The gap closes at the first reference.

Enabling it

Set PDS_ENABLE_SPACES=true. On Cloudflare that is a [vars] entry in wrangler.toml.

Without the flag the space endpoints do not exist, and token issuance refuses space: scopes. A deployment that leaves it off never installs @pdsjs/spaces.

Stability

The proposal and its reference implementation (atproto, published as @atproto/space@0.0.0-spaces-alpha-*; pds.js tracks 0.0.0-spaces-alpha-20261105160032) are both drafts. Wire formats may change, and a change may need a repo rebuild.

scripts/space-reference/ tests both directions against the reference. pds.js produces artifacts and the reference verifies them, then the reference produces artifacts and pds.js verifies them. A pass means the KEM, both MACs, BLAKE3, LtHash, the STAR layout, record CIDs, the tokens and the credential signature all agree. Both directions carry negative cases, so a pass cannot be vacuous.

npm run test:reference

The reverse direction also runs in the plain test suite, against committed fixtures. The full two-directional run needs a worktree of the pinned reference. scripts/space-reference/README.md has the setup.

The run club example

Track Tuesdays demonstrates the whole flow over plain HTTP. Two accounts on two PDS instances share a private space. A feed page assembles the week, which is the role the PDS itself does not play.

docker compose up -d     # local PLC directory
npm run dev:runclub      # feed at http://localhost:2800

The page verifies every commit MAC, set hash, and record CID before it renders, and shows the receipts in its footer.

Git repositories

@pdsjs/git stores a git repository in a space when the remote URL names one: atproto://<authority>/space/<space-type>/<skey>/<repo>. The package README covers the auth flows and limits.

Scopes

space: OAuth scopes gate access to space data. scope-comparison.md compares the full scope set against the reference implementation.