chadtmiller.com/pds.js
An atproto PDS in JavaScript, for Cloudflare Workers and Node
0
0
# Permissioned data
An implementation of
[proposal 0016](https://github.com/bluesky-social/proposals/tree/main/0016-permissioned-data):
private per-user repos, shared through spaces.
A space has one authority and a set of members. Each member writes to their own
private repo on their own PDS. The authority holds the writer set, and it issues
the credential that lets a reader pull every member repo. The PDS stores and
serves those repos. It does not assemble them. That job belongs to the
application.
A space's policy decides who may hold a credential and whose writes the
authority records. `@pdsjs/spaces` provides LtHash commits, MACed reads, space
credentials, `space:` OAuth scopes, and the `com.atproto.space`,
`com.atproto.memberspace` and `com.atproto.managedspace` methods. Node, Deno,
and Cloudflare Workers all support it.
## Telling a client spaces are here
`com.atproto.server.describeServer` answers `spacesAlpha: true` when
`PDS_ENABLE_SPACES` is set, as the reference PDS does.
`community.lexicon.service.describe` also answers with this server's roles and
every XRPC method it routes, so the space methods appear in that list exactly
when the feature is on.
## Policies
A space holds exactly one policy, set by `createSpace` and replaced whole by
`updateSpace`. The authority always reads and writes its own space, whatever
the policy says.
| Policy | Who reads | Whose writes count |
| --- | --- | --- |
| `com.atproto.personalspace.policy` | the authority alone | the authority alone |
| `com.atproto.memberspace.policy` | anyone under `publicRead`, else members with `read` | anyone under `publicWrite`, else members with `write` |
| `com.atproto.managedspace.policy` | whoever `managingApp` admits | whoever `managingApp` admits |
A memberspace may also name `allowedApps`, the client_ids that may receive a
credential. Absent, any client may. A managedspace asks its managing app on
each credential request, through `com.atproto.managedspace.checkUserAccess`
with service auth for that method. A managing app that does not answer, or
answers no, admits nobody.
`com.atproto.memberspace.putMember`, `removeMember` and `listMembers` work only
on a memberspace. On any other policy they answer `PolicyMismatch`. A member
added without `write`, removed, or changed to lose `write` leaves the writer
set at once, and `com.atproto.space.removeWriter` does the same for any writer.
A change of policy type drops every member.
`com.atproto.space.getSpace` answers the authority with the space's URI and its
policy. Nobody else may read the policy this way.
### Published policy and members
A space may publish its policy to its readers. With `publishPolicy` set, the
authority writes the policy as `com.atproto.space.policy/self` into its own
repo in the space. A memberspace with `publishMembers` set also writes one
`com.atproto.memberspace.member` record per member, keyed by the member's DID.
The server keeps these records in step with the policy and the member list.
Nobody writes those two collections directly: `createRecord`, `putRecord`,
`deleteRecord` and `applyWrites` refuse them with `InvalidRequest`.
### Clearing a repo
`com.atproto.space.clearRecords` deletes every record the caller holds in a
space, in commits of at most 200 writes. It needs a delete scope for each
collection it touches. In the authority's own repo it leaves the published
policy and member records in place.
### Spaces made before policies
A database from the October 22 alpha holds each space's read and write rules
in separate columns. On first start the server converts each space it is the
authority of. A space with a managing app on either side becomes a
managedspace, and its members are dropped, since its app decides access. Any
other becomes a memberspace: public read or write carries over, an app allow
list becomes `allowedApps`, and members keep their rights. Neither kind
publishes anything until its authority asks. The old columns stay in the table
and nothing reads them.
### Not implemented
The reference also moves spaces between servers with an account:
`importRepo` restores space records, and `exportSpaceMeta` and
`importSpaceMeta` carry a space's policy and members. pds.js implements none of
the three.
`scripts/space-reference/` tracks the atproto PR; `ngerakines.me/atproto-crates`
is the other implementation in use.
## Getting a credential
An application asks the space's authority for a credential with
`com.atproto.space.getSpaceCredential`. The request is not signed. Its body
names the space and `credentialKid`, a fresh P-256 did:key the application
holds the private half of. The authority puts that did:key in the credential's
`cnf.kid`. The key needs no registration.
```json
{
"space": "at://did:plc:.../space/com.example.forum/general",
"credentialKid": "did:key:zDn...",
"delegationToken": "<from the user's PDS>",
"clientAttestation": "<optional>"
}
```
The delegation token comes from the user's own PDS through
`getDelegationToken`, addressed to `<authority DID>#atproto_pds` and signed
with the user's `#atproto` key. It says which user the application acts for,
and a space whose policy admits only some users needs one. A memberspace with
`publicRead` issues a credential without one. The authority checks the user
before the application. A user the policy does not admit, or a missing token on
a space that is not public, answers `SpaceNotFound`, the same as a space that
does not exist, so the request cannot learn that it does. An admitted user
through an application the space does not allow answers `AppNotAuthorized`. A deleted space answers
`SpaceDeleted` either way. A token that does not verify, or one already spent,
is `InvalidDelegationToken`.
A credential lives ten minutes, and never more than an hour. A delegation token
buys one credential. The authority spends its `jti` and refuses a second
presentation. A client attestation is spent the same way.
## Presenting a credential
Every request presenting a credential uses the `Atproto-Space-Credential` scheme
and names the account it is addressed to in `Atproto-Space-Audience`. For a repo
read that is the repo's owner, and for `listRepos`, `registerNotify` and
`unregisterNotify` it is the space's authority. An
[RFC 9421](https://www.rfc-editor.org/rfc/rfc9421) HTTP message signature by the
`cnf.kid` key covers both headers, in that order, under the label and tag
`atproto-space-credential`.
```http
Authorization: Atproto-Space-Credential <credential>
Atproto-Space-Audience: did:plc:...
Signature-Input: atproto-space-credential=("authorization" "atproto-space-audience");tag="atproto-space-credential"
Signature: atproto-space-credential=:<64-byte r||s, padded base64>:
```
A host refuses a signature made for another account, so a host given a
credential to serve one repo cannot present it to another. A signature covers
nothing else about the request, so the application may reuse it for the same
credential and audience until the credential expires.
## Reading with a MAC
A space commit carries no signature. It names the repo, the space, the rev, and
`data`, the SHA-256 of the repo's LtHash state. The repo host authenticates
what it serves to each syncer with a MAC instead, under a secret only the two
of them share.
Every account has a P-256 key published as `#atproto_kem` in its DID document.
A syncer reading a repo makes a fresh P-256 key for the request, derives a
secret against the author's KEM key (DHKEM(P-256, HKDF-SHA256), RFC 9180
§4.1), and sends its public half:
```http
Atproto-Space-Want-Mac: type="atproto-space-repo", ver=1, key_share="did:key:zDn..."
```
The host derives the same secret from its KEM private key and answers with the
MAC of the commit, or of the record for `getRecord`
(`type="atproto-space-record"`):
```http
Atproto-Space-Mac: type="atproto-space-repo", ver=1, mac=:<32 bytes, padded base64>:
```
A request presenting a credential must ask for a MAC on `getRecord`, `getRepo`,
`getLatestCommit` and `listRepoOps`; without one, or with a header that does
not parse, the host answers `BadSpaceMac`. The account's own session may ask
and need not. `listRecords` carries no MAC. Each such response has
`Vary: Atproto-Space-Want-Mac`, and exposes `Atproto-Space-Mac` to browser
apps across origins.
`getRepo` answers with the repo as a STAR archive
(`application/octet-stream`): the commit as the archive's metadata, then each
record's path and DAG-CBOR bytes in byte order of the path, a control byte
before each record and one to end the archive. A syncer verifies
the commit against the MAC, folds each record into a set hash, and trusts the
records only when that hash matches the commit's `data`. A record's DAG-CBOR
encoding may be at most 1 MiB.
A space record's JSON `{$link}` is stored as a CID link and `{$bytes}` as a
byte string, so a record's CID is the one any implementation computes from the
JSON `getRecord` serves.
### KEM keys
An account created on pds.js gets its KEM key with its first DID document, in
the genesis PLC operation. An account that predates the key gets one at startup
and hourly after: the server publishes it with a PLC update when it holds a
rotation key for the account and the account's PDS is this server, and
otherwise stores it and logs a warning for the owner to publish.
`com.atproto.identity.getRecommendedDidCredentials` includes it, and
`submitPlcOperation` refuses an operation that names another one. A did:web
account publishes the key from `getRecommendedDidCredentials` itself.
## Revoking a credential
A space's authority may tell a repo host that credentials it issued are revoked
before they expire, with `com.atproto.space.notifyCredentialRevoked`. The body
names the space and up to 100 credential ids (`jti`). The call takes service
auth issued by the authority and addressed to an account the host serves. The
host refuses a revoked credential with `CredentialRevoked`, and keeps each
revocation for an hour and ten seconds, which outlives every credential it can
match.
## Which application is asking
A delegation token says the user delegated to somebody. A client attestation
says who: a JWT the client signs with its own authentication key. The authority
resolves the `iss` to its `client-metadata.json`, fetches the key set that
document publishes, and checks the signature against the key its `kid` names.
A memberspace's `allowedApps` list rests on that check. Without it the client_id would be
a claim the caller writes for itself, and the list would state a rule the
authority could not enforce.
`Bearer` is refused. One credential reads a whole space and its holder presents
it to every repo host in that space, so as a bearer token it would be a shared
secret: a host given one to serve its own repo could replay it against every
other host.
## Write notifications
A space's writes never reach the firehose. A service that follows a space
registers with the authority through `registerNotify`, naming itself by a service
identifier whose DID document publishes the endpoint to deliver to. The authority
then forwards a `notifyWrite` to it for every write it learns of, including writes
into its own repo, where no notification arrives over HTTP for it to forward.
A notice carries the repo rev and commit `data` a repo reached, and no records. The
syncer reads with its own member's session, which is the only thing that can. A
registration lasts a day, and a lapsed one stops receiving notices, so a syncer
renews before it expires.
The authority puts every update it accepts on one line for the whole space. A
forwarded notice carries the `spaceRev` the authority gave the update and the
`prevSpaceRev` of the update before it, absent on the space's first. A notice at
or behind the writer's last accepted repo rev is not forwarded again, and a repo
rev more than five minutes ahead of the authority's clock is refused with
`FutureRev`. `listRepos` lists writers in `spaceRev` order. Its `cursor` is a
space revision, every page with entries ends on one, and only an empty page has
none. A syncer that sees a gap in `prevSpaceRev` catches up from its last
`spaceRev` through `listRepos`.
The authority applies the space's policy to each notice it receives. A
writer the space does not admit stays out of the writer set, so `listRepos`
never names them and no syncer pulls their repo.
A repo host owes the authority a notice for every write into its space. It
stores the notice before it sends it. A notice that fails for a reason that may
pass, a network failure or a 5xx, 408, 425 or 429 answer, is retried with
backoff from ten seconds up to an hour. Retries stop when the notice lands,
when a newer write to the same space replaces it, or a day after that write.
Any other answer stops them at once. On Node and Deno a ten-second interval
retries what is due. On Cloudflare the Durable Object's alarm comes forward to
the next retry.
Forwarding a write from the authority to a syncer is best effort. Sync
correctness rests on comparing commit data, so a missed notice is recovered by
the next one or by the syncer's own catch-up through `listRepos`.
Deleting a space is different, because no later notice follows it. The
authority owes every registered syncer a `notifySpaceDeleted`, stored and
retried with the same backoff until it lands or the syncer's registration
expires. A syncer that still misses it learns from `SpaceDeleted` on its next
credential request.
A write by the authority into its own space needs the space to exist. Into one
it never created, or one it deleted, the write is refused with
`SpaceNotFound`.
## Blobs
A space record's blobs are ordinary account blobs. They are uploaded through
`com.atproto.repo.uploadBlob` and referenced from the record, so a client writing
blob-bearing records needs a blob permission alongside its space permission.
`com.atproto.space.listBlobs` and `com.atproto.space.getBlob` read them back
within the space.
Two things guard them from orphan cleanup, which deletes any blob no record
points at. A space write records its references in the account's one blob link
table, beside those of public records, so the reaper's primary query already
counts them. And `collectReferencedBlobs`, the cross-check that runs before
anything is deleted, reads space records as well as public ones, so a link that
was never written cannot cost a blob either.
`getBlob` serves only a blob the named space references. A credential admits its
holder to that space, not to the account's whole blob store. The reference
implementation is looser here.
The public endpoints hold the same line from the other side:
`com.atproto.sync.getBlob` refuses a blob that only space records reference,
and `sync.listBlobs` leaves it out. One window stays open by design: a blob
between its upload and the write that references it has no links yet, and the
public write flow needs such blobs served. The gap closes at the first
reference.
## Enabling it
Set `PDS_ENABLE_SPACES=true`. On Cloudflare that is a `[vars]` entry in
`wrangler.toml`.
Without the flag the space endpoints do not exist, and token issuance refuses
`space:` scopes. A deployment that leaves it off never installs `@pdsjs/spaces`.
## Stability
The proposal and its reference implementation (atproto, published as
`@atproto/space@0.0.0-spaces-alpha-*`; pds.js tracks
`0.0.0-spaces-alpha-20261105160032`) are both drafts. Wire formats may change,
and a change may need a repo rebuild.
`scripts/space-reference/` tests both directions against the reference. pds.js
produces artifacts and the reference verifies them, then the reference produces
artifacts and pds.js verifies them. A pass means the KEM, both MACs, BLAKE3,
LtHash, the STAR layout, record CIDs, the tokens and the credential signature
all agree. Both directions carry negative cases, so a pass cannot be
vacuous.
```bash
npm run test:reference
```
The reverse direction also runs in the plain test suite, against committed
fixtures. The full two-directional run needs a worktree of the pinned reference.
`scripts/space-reference/README.md` has the setup.
## The run club example
[Track Tuesdays](../examples/runclub/README.md) demonstrates the whole flow over
plain HTTP. Two accounts on two PDS instances share a private space. A feed page
assembles the week, which is the role the PDS itself does not play.
```bash
docker compose up -d # local PLC directory
npm run dev:runclub # feed at http://localhost:2800
```
The page verifies every commit MAC, set hash, and record CID before it renders,
and shows the receipts in its footer.
## Git repositories
`@pdsjs/git` stores a git repository in a space when the remote URL names one:
`atproto://<authority>/space/<space-type>/<skey>/<repo>`. The
[package README](../packages/git/README.md#private-repositories) covers the
auth flows and limits.
## Scopes
`space:` OAuth scopes gate access to space data. [scope-comparison.md](scope-comparison.md)
compares the full scope set against the reference implementation.