~ knotless Search/
↑↓ move↵ open⇧↵ all results

chadtmiller.com/pds.js

An atproto PDS in JavaScript, for Cloudflare Workers and Node

0
0
main
142 lines · 7.3 KB Raw History

Reference-parity: firehose events, rpc scopes, DPoP nonces, migration endpoints

Date: 2026-07-30 Goal: close the four interop gaps against the reference bluesky-social/atproto PDS, adapted to pds.js's single-actor, port-based architecture. Reference behavior is the design authority; where pds.js has no equivalent subsystem (email, multi-account), the adaptation is documented below.

1. Firehose completeness

1a. #commit gains since and prevData

  • since = the rev of the previous commit (null on the first commit). Already available in createCommit via getLatestCommit().
  • prevData = the data (MST root) CID of the previous commit, read by fetching the previous commit block and decoding it (same DAG-CBOR parsing the CAR loader uses). Omitted on the first commit.
  • Both are stored in the event payload at commit time so replayed events carry them too. Old stored events without the fields replay as before (fields absent), which is valid per the lexicon (both optional/nullable).

1b. Typed events: #identity, #account, #sync

Stored event payloads are opaque CBOR decoded by formatFirehoseEvent, so a type discriminator is added inside the payload — no storage schema or port changes. putEvent's commitCid column gets '' for non-commit events.

Wire shapes (from com.atproto.sync.subscribeRepos lexicon):

  • #identity: {seq, did, time, handle}
  • #account: {seq, did, time, active, status?}
  • #sync: {seq, did, blocks, rev, time} — blocks is a CAR containing the current commit block; announces non-inductive state (v1.1 semantics).

Emission points (reference order):

  • /init on a fresh PDS: #identity → #account (active); if a commit already exists (re-init), also #sync.
  • /init re-run that changes the handle: #identity.
  • com.atproto.repo.importRepo (new, §4): #sync after the import commit.
  • activateAccount / deactivateAccount (new, §4): #account.

Each typed event consumes a sequence number from the same per-actor sequence as commits. The events table becomes the sequence authority (every commit already writes an event row): ActorStoragePort gains getLatestSeq(): Promise<number> (max seq in the events table, 0 when empty) and createCommit switches from prevCommit.seq + 1 to getLatestSeq() + 1.

2. rpc: OAuth scopes

Extend packages/core/src/scope.js:

  • Syntax (reference @atproto/oauth-scopes): rpc:<lxm>?aud=<aud>, rpc:*?aud=<aud>, rpc:<lxm> with aud in query only, plus rpc?lxm=X&lxm=Y&aud=Z repeatable-param form. aud is a DID, a did#serviceId ref, or *.
  • ScopePermissions.allowsRpc(lxm, aud); matching follows the May 2026 reference change: the scope check uses the combined did#serviceId form, while outbound service JWTs keep the bare DID aud (proposal 0014 phase 1). Lxm comparison is case-insensitive (Feb 2026 reference change).
  • Enforcement: the app.bsky.* AppView proxy and atproto-proxy header path require full access OR allowsRpc(lxm, ${serviceDid}#${serviceId}). getServiceAuth (§4) enforces the same on its aud+lxm params.

3. DPoP nonces

Per RFC 9449 §8 / atproto OAuth spec (AS must supply nonces):

  • Nonce = base64url(HMAC-SHA256(jwtSecret, "dpop-nonce:" + floor(now/300s))) truncated to 32 bytes of input material; current and previous 5-minute windows are accepted (no storage).
  • /oauth/par and /oauth/token: DPoP proofs must carry a valid nonce claim. Missing/stale → HTTP 400 {"error":"use_dpop_nonce"} with a fresh DPoP-Nonce response header. Successful responses also carry DPoP-Nonce.
  • Server metadata unchanged (dpop_signing_alg_values_supported already advertised). Resource-server nonces stay unimplemented (optional in spec).
  • The e2e DPoP client helper learns the standard retry-on-use_dpop_nonce dance, which is itself the compliance test.

4. Account migration endpoints

New XRPC routes (all require full-access auth unless noted):

  • com.atproto.server.getServiceAuth (GET): params aud (DID or did#serviceId), lxm (required here — pds.js always enforces a method), exp (optional, capped at now+1h). Mints an ES256 service JWT via the existing createServiceJwt, aud = bare DID. Granular tokens need a matching rpc: scope.
  • com.atproto.server.checkAccountStatus (GET): returns {activated, validDid, repoCommit, repoRev, repoBlocks, indexedRecords, privateStateValues: 0, expectedBlobs, importedBlobs}. Counts come from the storage ports (listAllRecords, listBlobs, block walk like getRepo). validDid = stored DID parses and the signing key exists.
  • com.atproto.repo.importRepo (POST, CAR body): loads via the existing loadRepositoryFromCar into actor storage. The CAR's commit DID must match the authed DID. Afterwards emits a #sync event. Allowed while deactivated (that's the migration flow).
  • com.atproto.repo.listMissingBlobs (GET): walks all records with findBlobRefs, diffs against listBlobs, returns {blobs: [{cid, recordUri}], cursor} (cursor = last record URI, limit default 500).
  • com.atproto.server.activateAccount / deactivateAccount (POST): flips a new accountStatus metadata value (active default / deactivated). Deactivated blocks record/blob writes (409 AccountDeactivated), is reflected by getRepoStatus/listRepos/checkAccountStatus, and emits #account. deleteAfter is accepted and ignored (no scheduler).
  • PLC operations (com.atproto.identity.*), adapted for a PDS with no email:
    • getRecommendedDidCredentials (GET): {alsoKnownAs: [at://handle], verificationMethods: {atproto: did:key}, rotationKeys: [did:key], services: {atproto_pds: {type: 'AtprotoPersonalDataServer', endpoint: https://hostname}}}. did:key encoding lifted from scripts/setup.js into packages/core/src/plc.js (shared with the script).
    • requestPlcOperationSignature (POST): succeeds as a no-op — there is no email channel; no token is issued and none is required.
    • signPlcOperation (POST): merges the request body over the recommended credentials, sets prev from GET {plcUrl}/{did}/log/last, signs with the repo signing key, returns {operation}. The token field is accepted and ignored (documented).
    • submitPlcOperation (POST): POSTs {operation} to {plcUrl}/{did}.
    • New constructor option plcUrl (default https://plc.directory).

Storage port additions

ActorStoragePort gains getAccountStatus(): Promise<string|null> and setAccountStatus(status: string): Promise<void>, implemented in the metadata KV table both sqlite and the Cloudflare DO already have; the readonly routing shim returns 'active'; test mocks updated.

Non-goals

Multi-account, email/2FA, app passwords, admin/moderation endpoints, getBlocks/listReposByCollection, resource-server DPoP nonces, incremental (since) CAR diffs in getRepo, and rate limiting stay out of scope.

Testing

TDD throughout. Unit: scope parsing (scope.test.js), nonce generation/windowing and PLC helpers (new test files), event formatting. E2E: firehose event assertions over WebSocket (identity/account/sync + since/prevData on commits), nonce retry flow in the DPoP helper, and a full migration round-trip (deactivate → export via getRepo → import into the same PDS → listMissingBlobs → checkAccountStatus → activate) on all three platforms via the existing suite.