chadtmiller.com/pds.js
An atproto PDS in JavaScript, for Cloudflare Workers and Node
0
0
# @pdsjs/lexicon-resolver
## 2.5.2
## 2.5.1
## 2.5.0
### Minor Changes
- bda4794: A record write validates against a preloaded lexicon instead of resolving one
over the network. Validating a single `app.bsky.feed.post` walked 16 schemas,
and each one ran the full chain — a DNS-over-HTTPS lookup for the authority,
a PLC read for its DID document, then a `getRecord` on its PDS. Measured
against the live network that cost 48 requests and 5.3 seconds, and the caches
holding the answer are in memory, so a Cloudflare Durable Object paid it again
on every eviction. It dominated `createRecord`: the same server answers a read
in well under a tenth of that.
`schemas/published.js` carries the lexicons for every record type in
`knownRecords`, and every schema they reference, generated by
`scripts/gen-bundled-lexicons.mjs`. The resolver preloads them under the same
TTL as a fetched schema, so the published document still replaces each one as
it ages. Enforcement is unchanged: it comes from `knownRecords`, and a
caller-supplied schema still wins. A post now validates in 10 ms with no
request at all.
Three fixes cover the collections that are still resolved:
- One DNS lookup per authority domain and one DID resolution per DID, rather
than one of each per NSID. A dozen schemas from a single authority repeated
both.
- A schema's references resolve together instead of one after another.
- A resolution that produced nothing is cached on the negative TTL. Without
that entry every write to a collection with no published lexicon repeated
the whole chain. A second write to a `dev.pdsjs.*` collection cost two DNS
lookups; it now costs none.
The refresh that runs once the preloaded schemas age out takes 31 requests and
1.6 seconds, against 48 and 5.3.
- ec85961: A write resolves a lexicon only for the collections this server enforces.
`com.atproto.repo.createRecord` defines its default as "validate only for known
Lexicons", and the enforced set is that list, but the check sat after the
resolution rather than before it. Every write resolved its collection over the
network first and consulted the list second, so a collection the server does not
enforce paid the full chain — a DNS-over-HTTPS lookup for the authority, a PLC
read, a `getRecord` on its PDS, and the same again for every reference.
The chain could not change the answer. A collection outside the enforced set
cannot fail a write: when its lexicon resolves and the record conforms the
status is 'valid', and in every other case, resolved or not, it is 'unknown'.
The one thing the network bought was turning 'unknown' into 'valid' for a
lexicon this server does not hold, which is not what the reference PDS reports.
`assertValid` now answers 'unknown' for an unenforced collection without
resolving anything. Every enforced collection is preloaded, so the default write
path makes no network request at all. Measured cold, per write:
`dev.pdsjs.drive.file` 653 ms and 2 requests to 0 and none,
`com.whtwnd.blog.entry` 336 ms and 2 to none, `pub.leaflet.document` 1177 ms and
36 to none.
`validate: true` is unchanged and is now the only path that reaches the network:
it resolves whatever the collection is and enforces the result. Enforcement of
the known record types is unchanged.
A third-party record whose lexicon resolves and conforms reports
`validationStatus: 'unknown'` where it reported 'valid'. That is what the
reference PDS answers for a lexicon it does not hold.
## 2.4.3
## 2.4.2
## 2.4.1
## 2.4.0
## 2.3.0
### Patch Changes
- d8cae21: Hold resolved DID documents, so a space request stops reading the directory.
Every space credential names its issuer, and verifying one reads that issuer's
key. plc.directory sends no `Cache-Control`, so nothing below the resolver held
the answer and every space request paid a round trip to it.
`createCachedDidResolver` holds an answer for an hour, an unresolvable DID for a
minute, and serves what it holds for up to a day when the directory stops
answering. The platform packages build one at startup and share it between the
PDS and the space routes.
A held document can name a key its issuer has rotated away from. Verification
now reads past the held answer and tries once more when a signature fails,
which is how `getSigningKey` gained its `forceRefresh` argument. The service
token that authorizes `com.atproto.server.createAccount` always reads past it:
adopting a DID is worth a fresh read, and it happens once per account.
## 2.2.0
### Minor Changes
- 09e110d: Show a check on the Records list for collections whose lexicon is published via
the protocol.
A lexicon is "published" when its NSID resolves through the protocol — DNS
authority (`_lexicon.<domain>`) → DID → the `com.atproto.lexicon.schema` record.
The resolver already walked exactly that chain for record validation, so it gains
an `isPublished(nsid)` method (negatively cached, so an unpublished NSID doesn't
re-hit DNS on every check) and core exposes it at
`GET /account/api/lexicon-status?collections=…`. The account UI loads that
separately from the overview, so the collection list still renders instantly and
a green check fills in beside each NSID whose lexicon resolves — on both the app's
Records list and the header of a collection's records page. Clicking the check
opens a popover that explains, in plain language, what a published record type
means (a click popover rather than a hover tooltip so it works on touch and can
hold actions). From there a "View schema" button opens a dialog that lays out the
record's fields in plain terms — each field's name, a friendly type, whether it's
required, and its description — rather than raw JSON, since this is for people
reading their own data. The schema is served by
`GET /account/api/lexicon-schema?collection=…` (backed by a new `getLexicon(nsid)`
on the resolver).
- 62dc25a: Stop rejecting writes over a third-party lexicon fetched off the network.
`com.atproto.repo.createRecord` defines its `validate` flag as "false to skip
Lexicon schema validation, true to require it, or leave unset to validate only
for known Lexicons". The reference PDS reads "known" as a hardcoded map of ~20
record types it bundles, with a standing `@TODO` for resolving schemas
dynamically — so it never fails a write because of a lexicon it had to go fetch.
pds.js does resolve them, and was enforcing every one it found, which made it the
only PDS in the ecosystem to reject records the rest of the network accepts.
That is a real failure mode rather than a theoretical one: a publisher's lexicon
routinely lags the client writing the records, so grain.social galleries were
rejected because the published `social.grain.photo` still required `alt` while
grain's client omits it when blank.
A resolved lexicon is now advisory by default — it reports `valid` when the
record conforms and `unknown` when it doesn't, instead of a 400. Rejection is
reserved for collections on the `enforce` list, which defaults to the same set
the reference PDS validates strictly, plus any schema handed to the resolver up
front. Those are still resolved from the network rather than bundled, so there's
no vendored copy to fall out of date; all 36 lexicons in that set's ref closure
currently match the reference's own byte for byte. `assertValid` takes a `strict`
option, wired to the caller's `validate: true`, that requires validation to
happen and enforces whatever resolves.
Two resolution bugs fell out of this. A ref that failed to resolve left the
schema set incomplete, and the resulting "Lexicon 'x' not found" surfaced as an
invalid record — so a DNS or directory hiccup turned into a write the client
could never land; that now reports `unknown` too, while a record broken
independently of the ref is still rejected. And authority lookup hardcoded a
two-segment domain, which cannot reach an NSID whose authority is deeper:
`com.atproto.lexicon.schema` publishes only at `_lexicon.lexicon.atproto.com`.
Lookup now tries the spec form — every segment but the last, reversed — before
falling back to the two-segment domain, which authorities serving a wildcard
still answer.
### Patch Changes
- 91e52aa: Give concurrent published-lexicon checks the real answer instead of a race.
When several requests asked about the same NSID at once, only one performed the
resolution (DNS → DID → schema record); the others waited a fixed 100ms and
then answered "not published", since a real resolution takes longer than that.
That false negative was cached for the full one-hour TTL, so a collection's
published check could show on its records page but stay missing from the app's
Records list for the rest of the session. Concurrent callers now share the one
in-flight resolution and all receive its result, and a cached negative expires
on its own short TTL (`negativeTtl`, default one minute) so a transient
resolution failure isn't pinned for an hour.
The app page also stampeded itself: until its batch lexicon-status query
resolved, every row's check component saw no answer and fired its own
single-NSID lookup alongside the batch. Rows now always pass the check a
boolean, so the batch request is the only one in flight.
## 2.1.0
## 2.0.0
## 1.0.2
## 1.0.1
## 1.0.0