chadtmiller.com/pds.js

Switch repository

Loading…

# Reference-parity: firehose events, rpc scopes, DPoP nonces, migration endpoints

Date: 2026-07-30
Goal: close the four interop gaps against the reference `bluesky-social/atproto`
PDS, adapted to pds.js's single-actor, port-based architecture. Reference
behavior is the design authority; where pds.js has no equivalent subsystem
(email, multi-account), the adaptation is documented below.

## 1. Firehose completeness

### 1a. `#commit` gains `since` and `prevData`

- `since` = the `rev` of the previous commit (`null` on the first commit).
  Already available in `createCommit` via `getLatestCommit()`.
- `prevData` = the `data` (MST root) CID of the previous commit, read by
  fetching the previous commit block and decoding it (same DAG-CBOR parsing
  the CAR loader uses). Omitted on the first commit.
- Both are stored in the event payload at commit time so replayed events carry
  them too. Old stored events without the fields replay as before (fields
  absent), which is valid per the lexicon (both optional/nullable).

### 1b. Typed events: `#identity`, `#account`, `#sync`

Stored event payloads are opaque CBOR decoded by `formatFirehoseEvent`, so a
`type` discriminator is added inside the payload — **no storage schema or port
changes**. `putEvent`'s `commitCid` column gets `''` for non-commit events.

Wire shapes (from `com.atproto.sync.subscribeRepos` lexicon):

- `#identity`: `{seq, did, time, handle}`
- `#account`: `{seq, did, time, active, status?}`
- `#sync`: `{seq, did, blocks, rev, time}` — `blocks` is a CAR containing the
  current commit block; announces non-inductive state (v1.1 semantics).

Emission points (reference order):

- `/init` on a fresh PDS: `#identity` → `#account (active)`; if a commit
  already exists (re-init), also `#sync`.
- `/init` re-run that changes the handle: `#identity`.
- `com.atproto.repo.importRepo` (new, §4): `#sync` after the import commit.
- `activateAccount` / `deactivateAccount` (new, §4): `#account`.

Each typed event consumes a sequence number from the same per-actor sequence
as commits. The events table becomes the sequence authority (every commit
already writes an event row): `ActorStoragePort` gains
`getLatestSeq(): Promise<number>` (max seq in the events table, 0 when empty)
and `createCommit` switches from `prevCommit.seq + 1` to `getLatestSeq() + 1`.

## 2. `rpc:` OAuth scopes

Extend `packages/core/src/scope.js`:

- Syntax (reference `@atproto/oauth-scopes`): `rpc:<lxm>?aud=<aud>`,
  `rpc:*?aud=<aud>`, `rpc:<lxm>` with `aud` in query only, plus
  `rpc?lxm=X&lxm=Y&aud=Z` repeatable-param form. `aud` is a DID, a
  `did#serviceId` ref, or `*`.
- `ScopePermissions.allowsRpc(lxm, aud)`; matching follows the May 2026
  reference change: the *scope check* uses the combined `did#serviceId` form,
  while outbound service JWTs keep the bare DID `aud` (proposal 0014 phase 1).
  Lxm comparison is case-insensitive (Feb 2026 reference change).
- Enforcement: the `app.bsky.*` AppView proxy and `atproto-proxy` header path
  require full access OR `allowsRpc(lxm, `${serviceDid}#${serviceId}`)`.
  `getServiceAuth` (§4) enforces the same on its `aud`+`lxm` params.

## 3. DPoP nonces

Per RFC 9449 §8 / atproto OAuth spec (AS must supply nonces):

- Nonce = base64url(HMAC-SHA256(jwtSecret, "dpop-nonce:" + floor(now/300s)))
  truncated to 32 bytes of input material; current and previous 5-minute
  windows are accepted (no storage).
- `/oauth/par` and `/oauth/token`: DPoP proofs must carry a valid `nonce`
  claim. Missing/stale → HTTP 400 `{"error":"use_dpop_nonce"}` with a fresh
  `DPoP-Nonce` response header. Successful responses also carry `DPoP-Nonce`.
- Server metadata unchanged (`dpop_signing_alg_values_supported` already
  advertised). Resource-server nonces stay unimplemented (optional in spec).
- The e2e DPoP client helper learns the standard retry-on-`use_dpop_nonce`
  dance, which is itself the compliance test.

## 4. Account migration endpoints

New XRPC routes (all require full-access auth unless noted):

- `com.atproto.server.getServiceAuth` (GET): params `aud` (DID or
  `did#serviceId`), `lxm` (required here — pds.js always enforces a method),
  `exp` (optional, capped at now+1h). Mints an ES256 service JWT via the
  existing `createServiceJwt`, `aud` = bare DID. Granular tokens need a
  matching `rpc:` scope.
- `com.atproto.server.checkAccountStatus` (GET): returns `{activated,
  validDid, repoCommit, repoRev, repoBlocks, indexedRecords,
  privateStateValues: 0, expectedBlobs, importedBlobs}`. Counts come from the
  storage ports (`listAllRecords`, `listBlobs`, block walk like `getRepo`).
  `validDid` = stored DID parses and the signing key exists.
- `com.atproto.repo.importRepo` (POST, CAR body): loads via the existing
  `loadRepositoryFromCar` into actor storage. The CAR's commit DID must match
  the authed DID. Afterwards emits a `#sync` event. Allowed while
  deactivated (that's the migration flow).
- `com.atproto.repo.listMissingBlobs` (GET): walks all records with
  `findBlobRefs`, diffs against `listBlobs`, returns `{blobs: [{cid,
  recordUri}], cursor}` (cursor = last record URI, limit default 500).
- `com.atproto.server.activateAccount` / `deactivateAccount` (POST): flips a
  new `accountStatus` metadata value (`active` default / `deactivated`).
  Deactivated blocks record/blob writes (409 `AccountDeactivated`), is
  reflected by `getRepoStatus`/`listRepos`/`checkAccountStatus`, and emits
  `#account`. `deleteAfter` is accepted and ignored (no scheduler).
- PLC operations (`com.atproto.identity.*`), adapted for a PDS with no email:
  - `getRecommendedDidCredentials` (GET): `{alsoKnownAs: [at://handle],
    verificationMethods: {atproto: did:key}, rotationKeys: [did:key],
    services: {atproto_pds: {type: 'AtprotoPersonalDataServer', endpoint:
    https://hostname}}}`. did:key encoding lifted from `scripts/setup.js`
    into `packages/core/src/plc.js` (shared with the script).
  - `requestPlcOperationSignature` (POST): succeeds as a no-op — there is no
    email channel; no token is issued and none is required.
  - `signPlcOperation` (POST): merges the request body over the recommended
    credentials, sets `prev` from `GET {plcUrl}/{did}/log/last`, signs with
    the repo signing key, returns `{operation}`. The `token` field is
    accepted and ignored (documented).
  - `submitPlcOperation` (POST): POSTs `{operation}` to `{plcUrl}/{did}`.
  - New constructor option `plcUrl` (default `https://plc.directory`).

### Storage port additions

`ActorStoragePort` gains `getAccountStatus(): Promise<string|null>` and
`setAccountStatus(status: string): Promise<void>`, implemented in the
`metadata` KV table both sqlite and the Cloudflare DO already have; the
readonly routing shim returns `'active'`; test mocks updated.

## Non-goals

Multi-account, email/2FA, app passwords, admin/moderation endpoints,
`getBlocks`/`listReposByCollection`, resource-server DPoP nonces, incremental
(`since`) CAR diffs in `getRepo`, and rate limiting stay out of scope.

## Testing

TDD throughout. Unit: scope parsing (`scope.test.js`), nonce
generation/windowing and PLC helpers (new test files), event formatting.
E2E: firehose event assertions over WebSocket (identity/account/sync +
since/prevData on commits), nonce retry flow in the DPoP helper, and a full
migration round-trip (deactivate → export via getRepo → import into the same
PDS → listMissingBlobs → checkAccountStatus → activate) on all three
platforms via the existing suite.