chadtmiller.com/pds.js
An atproto PDS in JavaScript, for Cloudflare Workers and Node
0
0
# App platform: repo-hosted static sites (the Serve primitive)
Date: 2026-08-06
Goal: serve static sites and apps out of the account's atproto repo — files as
blobs, a manifest as a record — on the deployment's own hostnames (apex +
one-label subdomains), in a new `@pdsjs/sites` package behind a single
hostname-dispatch seam in core. First of five declarative app-platform
primitives (Serve, Grant, Query, React, Schedule); this spec designs Serve
fully and sketches the rest so nothing in Serve forecloses them.
## 0. Why this shape
- **The repo is the deploy artifact.** A site is ordinary repo data: blobs
uploaded with `com.atproto.repo.uploadBlob`, one `dev.pdsjs.site.deploy` record
written with `putRecord`. Deploys ride the firehose, survive migration, are
covered by backup and CAR export, and need zero new write endpoints or auth
surface. The server side of Serve is read-only.
- **Origin isolation is the security model.** Site content serves only on
hostnames that are not the PDS hostname. The PDS origin keeps its invariant:
`com.atproto.sync.getBlob` forces `Content-Disposition: attachment` and a
sandbox CSP precisely so repo content cannot execute next to the account
session cookie and OAuth endpoints. Sites do not weaken that; they move
execution to origins that hold no credentials.
- **The server side is declarative.** An app is a configuration of fixed
primitives, never third-party code executing in the PDS process. That line
is what keeps the eventual "install an app" story reviewable.
## 1. Packaging and module boundaries
New workspace package `@pdsjs/sites`, workspace-only dependency on
`@pdsjs/core` (for `repo` utilities and port typedefs), no external
dependencies.
```
packages/sites/src/
index.js re-exports
hostname.js hostnameToSite(hostname, siteDomain, apexName) → string|null
lexicons.js dev.pdsjs.site.deploy / dev.pdsjs.app.manifest / dev.pdsjs.app.install
manifest.js loadSite(): record read → decode → base merge → path map (LRU)
serve.js path normalization, lookup order, header assembly, MIME table
handler.js createSiteHandler(ctx) → (request, url) => Response|null
browser.js createSiteBrowser(ctx) → SiteBrowserPort (account UI listing)
deploy.js pure walk/diff/record-build helpers for the CLI
cli.js pdsjs-site deploy CLI (the only file importing node:fs)
```
### Dependency direction
`@pdsjs/core` must not depend on `@pdsjs/sites`. Core exposes a hook; sites
plugs into it:
- The constructor gains `config.siteHandler` and `config.siteBrowser`.
- `fetch()` gains one hostname check at its top (§4).
- The platform packages construct the handler from ports + env and pass it in.
A deployment that does not set `PDS_SITE_DOMAIN` never installs or loads the
package, and core's behavior is byte-identical to today.
### Handler convention
`createSiteHandler(ctx)` receives ports and config, never the
`PersonalDataServer` instance:
```js
{ actorStorage, blobs, getDid, siteDomain, apexName }
```
Handlers are closures over that ctx — unit-testable without standing up a PDS,
and structurally unable to reach PDS internals. This is the same convention
space routes established.
## 2. Lexicons
Defined as plain JSON in `packages/sites/src/lexicons.js`. Record validation
fails open for unresolvable collections, so the schemas gate nothing at write
time; they exist for the CLI, tooling, and eventual publication.
### `dev.pdsjs.site.deploy` — one record per site, rkey = site name
The rkey is a lowercase DNS label; it binds the site to
`<rkey>.<siteDomain>`, and the configured apex name (default `home`) binds to
the apex.
- `files` (required): array of `{path, blob, contentType?, cacheControl?}`.
An array of objects rather than a map because Lexicon has no open-keyed map
type — and each entry embeds a real `{$type: "blob", ref, mimeType, size}`
value, which is exactly what `findBlobRefs` walks, so every deployed asset
is protected from orphan GC by the record that serves it. Files dropped by a
redeploy become orphans and the hourly reaper collects them after 24h.
- `fallback`: path served with status 200 for any unmatched request path (SPA
client-side routing).
- `base`: `com.atproto.repo.strongRef` to another `dev.pdsjs.site.deploy` record
in the same repo whose files merge beneath this record's (this record wins
on conflicts). One hop only; a base may not itself have a base. This is the
headroom valve for the 1 MB record write cap: one record holds roughly
5–7k files (~130–180 bytes of DAG-CBOR per entry), a base doubles it.
- `createdAt`.
`contentType` overrides the stored blob mime at serve time. Upload-time mime
sniffing only recognizes image formats, so text types carry whatever
Content-Type the deploy client sent; the manifest field is the deliberate
signal and the recovery path for blobs uploaded with generic tools.
`cacheControl` is a single string, not an open header map — an open map is a
header-injection surface, and every other header is either dedicated
(`contentType`) or the server's job.
### `dev.pdsjs.app.manifest` — published in the developer's repo
`name`, `description`, `icon` (image blob), `site` (strongRef to the
`dev.pdsjs.site.deploy` record holding the app's frontend), `scopes` (strings in
the host's OAuth scope grammar), `primitives` (enum array:
serve/grant/query/react/schedule), `external` (boolean: the app talks to
services beyond the user's own PDS — surfaced prominently at install),
`createdAt`.
### `dev.pdsjs.app.install` — written to the user's repo at install (rkey tid)
`manifest` (strongRef — the CID pin freezes the reviewed version; a changed
manifest requires re-consent), `grantedScopes`, `site` (local site name
holding this install's copied frontend), `config` (unknown), `createdAt`.
## 3. Serving semantics
Per request, in order:
1. **Hostname → site name** (`hostname.js`). Exact `siteDomain` match →
`apexName`. `<label>.<siteDomain>` where label matches
`/^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$/` (case-folded) → that label as
rkey. Deeper subdomains of `siteDomain` → the built-in 404 page (they are
claimed hostnames with nothing behind them, and Universal SSL only covers
one wildcard level anyway). Anything else → `null`: not ours, fall through
to normal PDS dispatch so health checks, tunnels, and bare-IP requests
behave exactly as today.
2. **`/.well-known/atproto-did`** on any claimed hostname → the account DID as
`text/plain`. Three deliberate lines re-implementing the core route, which
the site handler shadows; this is what lets the apex domain double as the
account handle.
3. **Methods**: GET and HEAD only. Everything else, OPTIONS included, → 405
with `Allow: GET, HEAD`. A static site has no preflightable API, and
answering OPTIONS with the PDS's permissive CORS headers would advertise
PDS-ness on a site origin.
4. **Load the site** (`manifest.js`): `getRecord` on
`at://<did>/dev.pdsjs.site.deploy/<name>`; miss → built-in 404 page. Decode
the DAG-CBOR, stringify blob CIDs, merge a `base` record if present
(refusing a second hop), and build `Map<path, {cid, mimeType, size,
contentType?, cacheControl?}>`. The record is re-read every request — it is
DO-local SQLite and it is the only correct redeploy signal — but the
decoded, merged path map is cached in a module-level LRU (≤32 entries)
keyed by record CID. Redeploy → new CID → miss: correct by construction,
no TTLs, no invalidation protocol. The DO is single-threaded, so a plain
Map suffices.
5. **Path resolution** (`serve.js`): `decodeURIComponent` in try/catch (→
400); reject any `..` segment, backslash, or NUL (→ 400); strip the
leading slash; empty path or trailing slash appends `index.html`. Lookup:
exact match → else if `path + "/index.html"` exists, 301 to `path + "/"`
(the correct relative-URL base beats saving a round trip) → else
`fallback` with status 200 → else the built-in minimal 404
(`text/html`, `no-store`).
6. **Conditional requests before any blob read**: strong `ETag` = the blob
CID string. `If-None-Match` (list parse, `W/` tolerated, `*` honored) →
304 with ETag + Cache-Control and no blob fetch — the CID is already in
the manifest.
7. **HEAD** answers entirely from manifest metadata (`Content-Length` from
the blob ref's `size`), no blob fetch.
8. **GET**: `blobs.get(did, cid)` — fully buffered, bounded by the 5 MB
upload cap; null → 404 (manifest/blob skew).
9. **Headers**: `Content-Type` = manifest `contentType` ‖ stored blob mime ‖
extension map ‖ `application/octet-stream`; always
`X-Content-Type-Options: nosniff`; `Cache-Control` = per-file override ‖
`public, max-age=60, must-revalidate`. Not `immutable` by default —
paths are mutable across deploys, and the ETag turns revalidation into a
header-only 304; builds emitting hashed filenames opt in via
`cacheControl`. No sandbox CSP, no `Content-Disposition`: executing is the
point, and this origin is not the PDS.
Errors inside the handler are caught and returned as a plain-text 500; site
origins never emit the PDS's JSON error shape.
## 4. The core seam
Two constructor options, stored and otherwise untouched by core:
```js
/** @property {(request: Request, url: URL) => Promise<Response|null>} [siteHandler] */
/** @property {SiteBrowserPort} [siteBrowser] */
```
`this.sitesEnabled = Boolean(siteHandler)`. `SiteBrowserPort` joins
`SpaceBrowserPort` in `ports.js`:
```js
/**
* @typedef {Object} SiteBrowserPort
* @property {string} domain
* @property {string} apexName
* @property {() => Promise<Array<{name: string, url: string,
* fileCount: number, totalSize: number, deployedAt: string|null,
* cid: string}>>} listSites
*/
```
The dispatch hook sits at the very top of `fetch()`:
```js
if (this.siteHandler) {
const url = new URL(request.url);
if (url.hostname !== this.hostname) {
const siteResponse = await this.siteHandler(request, url);
if (siteResponse) return siteResponse;
}
}
```
Every position in that placement is load-bearing, and each is pinned by a
seam test:
- **Before the OPTIONS short-circuit**, or every site hostname answers
preflights with the PDS's permissive CORS headers.
- **Outside `withDpopNonce`**, or site responses grow `DPoP-Nonce` headers;
sites are not OAuth resources.
- **No `addCorsHeaders`** — the handler owns its headers completely.
- **Before the account-app prefix intercept**, or
`https://blog.example.com/account` serves the account UI and
`readAccountSession` reads cookies on a site origin.
- A `null` return falls through to normal dispatch, so hostnames that are
neither the PDS hostname nor under `siteDomain` behave exactly as today —
including `.well-known/atproto-did` answering host-agnostically.
- `dispatch()` is untouched; its error handler never sees site traffic.
Edge documented rather than engineered around: a deployment whose PDS
hostname *is* the site apex serves the PDS there (exact match wins) and the
apex site is unreachable. The README recommends a `pds.` subdomain; the Sites
account page warns when it detects the collision.
## 5. Feature flag
The flag is the presence of `PDS_SITE_DOMAIN` (with optional
`PDS_SITE_APEX_NAME`, default `home`) — a boolean without a domain is
meaningless. It gates exactly once, at platform registration: the platform
package either builds `siteHandler`/`siteBrowser` and passes them, or passes
nothing and core has no site code path at all. There are no flag checks
inside handlers.
The flag-off path is load-bearing: with `PDS_SITE_DOMAIN` unset, a request
carrying a foreign Host header must behave byte-identically to today (404
through normal dispatch, OPTIONS answered with CORS). That is an explicit
test, not an assumption.
## 6. Platform wiring
- **Cloudflare**: static imports (the Worker bundles the graph regardless);
gate in the DO constructor next to the spaces gate; ctx built from the DO's
`actorStorage`/`blobs` adapters with `getDid` as a late-bound closure over
the constructed PDS. No Worker-entry change: the Worker already forwards
every request, hostname intact, to the single DO. Zone routing
(`example.com/*` + `*.example.com/*` — wildcards require zone routes, not
Custom Domains — plus proxied apex/wildcard DNS records) is deployment
configuration, documented in `examples/cloudflare/wrangler.toml`.
- **Node/Deno**: dynamic import behind the env var, optional peer dependency,
mirroring spaces. Multi-hostname TLS is the reverse proxy's job (wildcard
cert via DNS-01); README note.
- **readonly**: once the core seam exists, the read-only server can compose
the same handler over its own ports — sites keep serving from a mirror
during an outage. Not wired in v1.
### Deferred: Worker edge cache
A designed-but-deferred fast path for Cloudflare: the Worker checks
`caches.default` for non-PDS hostnames, calls the DO on miss, and
`waitUntil`-caches GET 200s for 60 s. Bounded staleness after redeploy
instead of a purge protocol; the DO still owns correctness via ETag. Ship v1
without it — it is the pressure valve if the single-DO hot path (site traffic
sharing one Durable Object with the firehose and writes) ever shows strain.
## 7. Deploy CLI
`pdsjs-site deploy <dir> --site <name> --pds <url> --handle <handle>`, app
password via `PDS_APP_PASSWORD` (never argv). Walk the directory (skip
dotfiles); compute each file's CID locally with `createBlobCid`; fetch the
existing deploy record and upload only changed files, with `Content-Type`
from the extension map (this is what makes stored blob mimes correct);
unchanged files reuse their existing blob refs verbatim and stay
GC-referenced through the new record. Abort before any upload if a file
exceeds the blob cap (list offenders, point at `PDS_BLOB_UPLOAD_LIMIT`), or
if the assembled record exceeds ~900 KB of DAG-CBOR (point at `base`). Then
one `putRecord` and a changed/unchanged/deleted summary. Deletes need no
calls: dropped blobs orphan and the existing GC reaps them.
## 8. Account UI
`GET /account/api/sites` (via the `accountApi` wrapper, backed entirely by
`siteBrowser` so site knowledge stays out of core) and
`POST /account/api/sites/delete`. `features.sites = Boolean(siteBrowser)`
gates the SPA page. The page is a read-and-delete view: card per site with
live link, file count, total size (summed from blob-ref sizes, no blob
reads), deploy time, record CID; empty state teaches the CLI invocation.
Deploys stay records-only — no upload UI.
## 9. The other primitives (sketches)
Nothing in Serve forecloses these; each attaches at its own seam.
- **Grant.** A repo-hosted app already has a natural OAuth `client_id`: the
URL of a client-metadata document the app's own site serves as a deployed
file. Grant is therefore pre-consent plus a ceiling, not a new credential:
the install sheet renders the manifest's requested scopes; the authorize
path, for a client_id whose origin is an installed site, skips the consent
screen when requested ⊆ granted and rejects when not. Known gap: grants
live only as scope strings on token rows, so revoking an install means
revoking all matching-client tokens; per-app toggles eventually want a
grants table keyed by client_id.
- **Query.** `dev.pdsjs.query.def` records (collection, filter, sort)
materialized into DO SQLite by replay plus incremental updates from a new
optional `onCommit(evt)` core callback; served at
`GET /xrpc/dev.pdsjs.query.run` via a `queryRoutes` route-table option (these
are path-routed, so the spaceRoutes shape fits).
- **React.** `dev.pdsjs.react.rule` records: collection match → signed webhook
POST or template-derived record write. Consumes the same `onCommit` tap;
retries ride platform alarms. Strictly URL + static headers + payload —
never code.
- **Schedule.** `dev.pdsjs.schedule.task` records (cron, fetch URL, target
collection); the platform clock (DO alarm / node interval, the hourly-GC
precedent) calls an exported `runDueTasks(ctx)`. Core untouched.
## 10. Testing
Unit tests gate everything (e2e is excluded from CI): hostname resolution
bounds; manifest decode/merge/LRU; the full serving matrix (traversal,
encodings, 301/fallback/404, ETag/304 and HEAD asserted to make zero blob
reads via mock call counts, MIME precedence, cacheControl); and the seam
itself driven through `pds.fetch` with foreign Host headers — flag-off
identical to today, flag-on serving with no CORS and no DPoP-Nonce, PDS
hostname unaffected, `/account` on a site host a site 404, OPTIONS a 405.
CLI diffing logic is pure and tested (CID-stable skip, oversize and
record-size aborts, MIME assignment).
## 11. What shipped past phase 1 (2026-08-11)
Built against a real deployment: photos.chadtmiller.com, a one-file SPA that
renders 89 Grain galleries from live queries. That site drove each addition
below and is the reference composition for the platform.
- **Query, v1.** `dev.pdsjs.query.def` and `dev.pdsjs.query.run` shipped in
`@pdsjs/sites/query`, wired by `PDS_ENABLE_QUERIES`. One deviation from the
section 9 sketch: materialization is an in-memory decoded copy of each
queried collection, replayed from the records table on first use, not rows
in DO SQLite. At single-account scale replay is milliseconds, so
persistence bought nothing; the sketch's architecture (replay plus
incremental maintenance from the commit tap) is otherwise as designed. A
CAR import bypasses the tap and is picked up on the restart that follows.
- **The commit tap.** Core takes `config.onCommit` and calls it after each
successful commit with `{did, seq, rev, ops}`. Errors are logged, never
failing the write. Query consumes it today; React's rules are the next
consumer.
- **Repo blobs on the site origin.** `/.blobs/<cid>` under a reserved
dot-prefix serves any repo blob, immutable with an ETag of the CID, safe
mimes only, CSP-sandboxed otherwise. This is what lets a query-rendered
site show images without a manifest entry per photo, and it composes with
Cloudflare image transformations for resized variants.
- **Serving conveniences.** Per-file Cache-Control from the deploy
(`--cache-control "[glob:]value"`), an edge cache for site hostnames in
the Worker entry, HTTPS-only with HSTS, and a www-to-apex redirect when no
www site exists.
- **The live-versus-snapshot trade, resolved in practice.** A snapshot site
(data baked at deploy) is fastest and pins content by CID; a query-rendered
site is always current and needs no rebuild on new records. The photos site
chose live: queries for data, `/.blobs` for images, one HTML file in the
manifest. The snapshot path stays right for sites that want pinning, and a
React rule firing a rebuild webhook is its freshness story.
## 12. Install (2026-08-11)
The app-store seam shipped as `pdsjs-site install <at-uri>`: no new server
surface, because an install is stock atproto traffic exactly as a deploy is.
The CLI resolves the developer's DID to their PDS, reads the manifest, and
refuses a site record that drifted from the manifest's CID pin. Files copy
blob by blob, each upload checked against the CID the record named, which is
the integrity property content addressing gives for free. The install writes
the local site record, the query defs the manifest declares, and a
dev.pdsjs.app.install record pinning the manifest version.
Apps are generic by construction: the reference app (Photos) discovers its
owner from its own origin's /.well-known/atproto-did, resolves the PDS from
the DID document, and reads its settings from the host's public install
record. Settings are declared data on the manifest ({key, label, type,
default, options}), snapshotted into the install record with chosen values,
edited on the account page's Sites section through a form generated from
the snapshot, and validated server-side against it. Grant stayed future:
an app that only reads public records installs with no consent step at all.