chadtmiller.com/monolith
An atproto PDS
Records against their lexicons
A write checks each record it makes against the lexicon of its type, when
the server knows one, and answers which it did. What every method shares,
and how the server is checked, is in PDS.md; the writes
themselves are as REPO.md says, and blob refs as
BLOBS.md says. atproto/lexicon.md defines the lexicon
language; this page says exactly what the server holds a record to, which
is in places more or less than that.
Which records
The server knows the record types whose lexicons are in lexicons/, and no
others: the 36 the reference knows.
app.bsky.actor:profile,status,link,contentVisibilityDeclarationapp.bsky.feed:post,like,repost,generator,threadgate,postgateapp.bsky.graph:follow,block,list,listitem,listblock,starterpack,verification,referencelistoptoutapp.bsky.labeler.service,app.bsky.notification.declaration,chat.bsky.actor.declarationcom.atproto.label.record,com.atproto.lexicon.schemacom.germnetwork.declaration;place.stream.livestream,place.stream.chat.profile,place.stream.chat.pinnedRecord;site.standard.document,site.standard.publication,site.standard.graph.subscription,site.standard.graph.recommend,site.standard.theme.basic;social.grain.gallery,social.grain.gallery.item,social.grain.photo,social.grain.favorite
A lexicon there of anything but a record makes no type known: a record of
an object's type (app.bsky.embed.images), of a lexicon of definitions
(app.bsky.feed.defs) or of a method's (app.bsky.feed.getTimeline) is
of another type, as is one of an NSID in a known namespace that names
nothing (app.bsky.feed.unknownthing). Nor do the spaces' lexicons
(com.atproto.space, memberspace, personalspace, managedspace,
SPACES.md).
Note: their records (com.atproto.space.policy,
com.atproto.memberspace.member) are a space's, which the reference
knows nothing of as a repository's record types.
A record's type is its $type, which is its collection (REPO.md: one
without $type is given the collection's, and one with another is refused
before anything here).
validate
createRecord, putRecord and applyWrites take validate, a boolean;
anything else, null too, is 400 InvalidRequest with the input.
validate | A known type | Another type |
|---|---|---|
| left out | checked; validationStatus: "valid" | not checked; validationStatus: "unknown" |
true | checked; "valid" | 400 InvalidRequest |
false | not checked; no validationStatus | not checked; no validationStatus |
A record that fails the check is 400 InvalidRequest, and the write makes
nothing. validationStatus is in createRecord's and putRecord's answer,
and in each #createResult and #updateResult of applyWrites;
#deleteResults have none, and deletes are not checked. validate applies
to every write of an applyWrites, which checks them in order, and answers
the first that fails; its key and record are as below, whatever the others.
The record is stored exactly as given: the check fills in nothing, not a
lexicon's default, so its CID is the CID of what was sent (with $type
added where it was left out).
A record not checked, with validate: false or of another type, is held
to nothing here, only to the data model (REPO.md): an object in it with a
$type that is not a string ({"$type": {"a": 1}}, {"$type": 5}), or a
union value without one, is stored as it is.
Where in a write
The order PDS.md gives goes on: a write checks its token, its input (the
method's own lexicon, validate among it), its repo, its record's
$type, the record key's syntax (REPO.md), and then, for a known type
unless validate is false, the record key against the type's lexicon and
then the record against it. Only after these come the legacy blob shape
(BLOBS.md), a swap, a key that exists already (createRecord's 500),
putRecord's write of exactly the record stored (REPO.md: still checked,
so an invalid record is refused even then), and the other blob refs.
validate itself is input, but what it asks for is not: the refusal of
validate: true on a type the server knows no lexicon for comes where the
record is checked, after the repo. So such a write to another account's
repository, by createRecord, putRecord or applyWrites, is 401 AuthenticationRequired, as is one of an invalid record or record key
there; validate: null there is 400 InvalidRequest, as input.
The record key
A type's lexicon names its key:
key | The record key must be | Types |
|---|---|---|
tid | a TID: 13 characters of 234567abcdefghijklmnopqrstuvwxyz, the first one of 234567abcdefghij (atproto/tid.md) | most |
literal:self | self, in lower case | profile, status and the declarations |
any | any record key | app.bsky.feed.generator |
nsid | an NSID (atproto/nsid.md) | com.atproto.lexicon.schema |
A key that is not is 400 InvalidRequest. createRecord without rkey, or
an applyWrites create without one, makes a TID, which is no self: such a
write of a literal:self or nsid type is 400 InvalidRequest too, unless
validate is false.
The record
The record is checked against its lexicon's record object, as
atproto/lexicon.md defines each type, with these rules. Any one broken is
400 InvalidRequest; which is found first is not compared.
- Objects. A
requiredfield must be there; one that is not, and a field that is there, must be of its type. A field set tonullis there, and no field of a known type may benull. Fields the lexicon does not name are taken as they are, at any depth. $typeof a nested object. An object arefnames (replyof a post,app.bsky.feed.post#replyRef; itsroot, acom.atproto.repo.strongRef) may leave out$type. If it has one, it must be that definition's: the NSID for a lexicon'smain,<NSID>#<name>for another.- Unions. Every union of a known type is open. Its value must be an
object with a
$typethat is a string. A$typethat is one of the union'srefs(written as above) is checked as that definition; any other string is taken without a look inside, a definition the union does not name (app.bsky.embed.images#viewfor a post'sembed), or a string that names nothing (not a type!). - Strings.
maxLengthandminLengthcount UTF-8 bytes;maxGraphemescounts extended grapheme clusters (Unicode UAX #29), so a family emoji of five code points joined by ZWJs is one, and so are a flag and a letter with a combining accent.knownValuesare not enforced: any string is taken. Aformatis as below. - Integers are held to
minimumandmaximum. (Floats are outside the data model, refused before,REPO.md.) - Booleans must be
trueorfalse. - Arrays: each item as its lexicon says, and
maxLengthandminLengthcount items. - Bytes must be bytes,
{"$bytes": …}in JSON, of any length, empty too; a string or a link is not. - Blobs must be a blob ref of exactly the keys
$type(blob),ref(a link to a CIDv1 of therawcodec),mimeType(a string with a/) andsize(an integer, 0 or more;-1is no blob here, thoughBLOBS.mdtakes it in other places), and nothing else: the legacy shape is no blob here either. ItsmimeTypemust matchaccept: be one of its entries, or start with<type>/for an entry<type>/*, case and all (image/*takesimage/gifandimage/, notIMAGE/PNGorimagex/png); and itssizemust be at mostmaxSize. These are the ref's ownmimeTypeandsize, as the record says them;BLOBS.mdthen holds them to the uploaded blob.
No known type uses enum, const, nullable, minGraphemes, a closed
union, cid-link or unknown.
String formats
interop/syntax/ has atproto's test vectors for each format: a string in a
*_syntax_valid.txt file is valid, and one in a *_syntax_invalid.txt or
*_parse_invalid.txt file is not. The server is held to them all, but
where the reference differs:
cid: a string that, parsed as a CID, is written the same way again: a CIDv0 in base58btc (Qm…, which the vectors call invalid), or a CIDv1 in base32 in lower case (bafy…,bafk…). A CIDv1 in any other base (z…,m…,f…), upper case, or of digits, which the vectors call valid, is refused.uri: exactly the strings^\w+:(?:\/\/)?[^\s/][^\s]*$matches,\wbeing ASCII letters, digits and_, of any length. Socontent-type:text/planandmicrosoft.windows.camera:thing, which the vectors call valid, are refused, and the very long URI they call invalid is taken.
The formats records use, and what each is:
datetime: at most 64 characters, matching^[0-9]{4}-(0[1-9]|1[012])-([0-2][0-9]|3[01])T([0-1][0-9]|2[0-3]):[0-5][0-9]:([0-5][0-9]|60)(\.[0-9]+)?(Z|[+-]([0-1][0-9]|2[0-3]):[0-5][0-9])$, not ending-00:00, with seconds other than60(the regular expression lets60through, but no leap second is taken), and whose instant, in UTC, is in the years 0000 to 9999 (0000-01-01T00:00:00+01:00is not). The day is not held to its month:2023-02-30T00:00:00Zand2023-04-31T00:00:00Zare taken. Upper-caseTandZonly.did,handle,nsid,at-uri: asatproto/did.md,handle.md,nsid.mdandat-uri-scheme.mddefine them, and the vectors agree: adidis any DID by its syntax (did:example:xanddid:plc:abcare valid DIDs), and anat-urimay stop after its authority or collection (at://bob.test).language: aLanguage-Tagof RFC 5646 §2.1's grammar, its letters in any case but where these say otherwise, with no variant or extension singleton repeated, compared in any case (§4.1):de-DE-1901-1901anden-u-co-phonebk-U-ca-buddhistare refused.- A tag with a primary language subtag starts with 2 or 3 lower-case
letters:
ja,ban,ja-jpn, but notJA,JaorZH-HAKKA, and not the 4 and 5 to 8 letter subtags the grammar allows (jaja,abcde). The subtags after it are in any case:en-latn-us,zh-YUE-hk,en-X-A. - A tag of private use alone starts
xorX:x-fr-ch,X-fr-CH(andxalone is no tag). - Of the grandfathered tags, the irregular ones are taken only exactly
as RFC 5646 writes them:
en-GB-oed,i-ami,i-bnn,i-default,i-enochian,i-hak,i-klingon,i-lux,i-mingo,i-navajo,i-pwn,i-tao,i-tay,i-tsu,sgn-BE-FR,sgn-BE-NL,sgn-CH-DE. SoI-default,i-Klingon,en-gb-oed,sgn-be-nl, and any otheri-tag (i-foo) are refused. The regular ones (zh-hakka,art-lojban,no-bok, ...) are tags of the grammar too, and are taken as any such tag is:zh-Hakka, notZh-hakka.
- A tag with a primary language subtag starts with 2 or 3 lower-case
letters:
Duplicates
createRecord, unless validate is false, replaces what it duplicates.
Its record links to a subject:
- an
app.bsky.graph.followorapp.bsky.graph.block, itssubjectwhen that is a string that is a DID; - an
app.bsky.feed.likeorapp.bsky.feed.repost, itssubject.uriwhen that is a string that is an AT URI.
The same commit then deletes every record of the account in the same
collection whose link is that same subject, compared as strings (so a like
of at://bob.test/… and one of at://did:plc:…/… are not the same), before
it creates the new one; createRecord answers as for any create, and the
deletes are in the commit, on the firehose and in the repository as any
delete is: the commit's ops are the deletes, in any order, then the create.
A record links to its subject whichever write made it, with validate: false too, by putRecord or applyWrites; only createRecord deletes.
So two follows of one DID, both kept by validate: false or putRecord,
are both deleted by the next createRecord of a follow of that DID. The
other writes delete nothing: putRecord and applyWrites of a second
follow keep both.
The record already at the key the write creates is one of those it deletes
when it links to the same subject: that createRecord is no 500 (as it is
for a key that exists, REPO.md), but the record there deleted and the new
one created, in one commit, even when the two are the same. Its ops are a
delete of the key and a create of it, whose prev is the deleted record's
CID (SYNC.md). A record at the key that links to another subject, or to
none, is kept, and the write is the 500; and so it is, whatever the
record there, with validate: false.
Data
The record lexicons, and the definitions they use, are in lexicons/:
app.bsky.*andchat.bsky.*fromlexicons/of https://github.com/bluesky-social/bsky atbe148a09fb(2026-11-01), andcom.atproto.*fromlexicons/of https://github.com/bluesky-social/atproto at0849c24d49(2026-11-09), both MIT or Apache-2.0, unchanged. They mean the same as the copies the reference is built with, at@atproto/pds@0.5.55(31eb9a09d3): those leave out"lexicon": 1and the descriptions, and nothing else differs.com.germnetwork.*,place.stream.*,site.standard.*andsocial.grain.*as the reference is built with them,packages/pds/lexicons/(andlexicons-overrides/, which some of those link to) of https://github.com/bluesky-social/atproto at31eb9a09d3, MIT or Apache-2.0, unchanged. They are other publishers' lexicons, which that repository holds a copy of.
interop/syntax/ is the syntax/ directory of
https://github.com/bluesky-social/atproto-interop-tests at 056e5741bb,
CC0 as the rest of interop/.
How it is checked
harness/checks/validation.mjs writes records of the known types, mostly
app.bsky, and some of other types (in no known namespace, an app.bsky
NSID that names nothing, and lexicons that are no record's), on both sides,
and compares the answers as PDS.md says (validationStatus among them,
and the record's CID): by createRecord, putRecord and applyWrites (of
one to four creates, updates and deletes), with validate left out,
true, false or not a boolean (null, "true", 1, 0, {}). Each
record is drawn valid from its lexicon, then sometimes broken in one way:
- a field left out where it is
required, or any fieldnull, or of another type; - a string too long in bytes, or in graphemes, with ASCII, accented letters precomposed and with combining accents, CJK, ZWJ emoji, flags and skin tones (or as long as it may be, valid); or too short;
- a string of each format drawn from
interop/syntax/, valid and not, the reference's differences above among them, and datetimes and language tags of each rule above; - an integer below its
minimumor above itsmaximum, an array with more items than itsmaxLengthor fewer than itsminLength; - a union value without
$type, with a$typethat is not a string, of a member broken inside, of a definition it does not name, or naming nothing; a nested object with a$typeof another definition; - a blob of a type
acceptrefuses, larger thanmaxSize, withsize-1, without$type, or in the legacy shape; - bytes that are a string or a link;
- a record key against its type's
key: atidtype withself, a TID of the wrong length, alphabet or case; aliteral:selftype with another key; annsidtype with a non-NSID; or no key at all, whichcreateRecordanswers with a TID of each side's own, taken out of the answers before they are compared, and the record, made, deleted again.
Fields the lexicon does not name are added at random, and knownValues
fields are given other strings. A valid blob ref names a blob the account
holds, as BLOBS.md's check knows them, or one no account has uploaded.
Each record also carries a field of a number no other has, so that no two
writes send the same record, but where one sends again exactly the record
written last at its key. Follows, blocks, likes and reposts are drawn from
few subjects at few keys, so that duplicates are often made, by every
write and validate, at another key and at the same one. After such a
write the collection is listed on both sides, compared, and the
reference's listing held to the keys the harness knows to be there, by
this page's rules; the account's repository then holds the same records
on both sides (PDS.md's repository check), deletes and all. With a
--firehose run, the deletes are on both firehoses. Writes of each kind
go to another account's repository too, with validate: true on another
type among them.
Not compared: which error a record broken in more than one way answers
first, and the error messages, as everywhere; the order of a commit's
deletes of duplicates (SYNC.md's check compares a commit's ops as a
set). The harness does not draw grapheme clusters whose segmentation
changed in recent Unicode versions.