chadtmiller.com/monolith
An atproto PDS
Identity
The com.atproto.identity methods the server serves, resolveHandle,
updateHandle and getRecommendedDidCredentials; how it resolves a
handle outside its own domains; the handle resolution it serves over HTTP,
GET /.well-known/atproto-did; and what describeRepo says of an
account's DID document. What every method shares, and how the server is
checked, is in PDS.md; accounts, their handles and their DIDs
are as ACCOUNTS.md says. submitPlcOperation,
requestPlcOperationSignature and signPlcOperation, which move an
account's DID to another server, are MIGRATION.md's.
Environment
| Variable | Is | Default |
|---|---|---|
PDS_HANDLE_DNS_SERVER | the DNS server asked for a handle's _atproto TXT records: an IPv4 or IPv6 address, with an optional port, as 127.0.0.1:5353 or [::1]:5353 | unset: the system's resolver |
PDS_HANDLE_FETCH | for development and testing only: a base URL, http:// or https://, a host, an optional port and path, with no trailing /, from which every handle's /.well-known/atproto-did is fetched instead (below) | unset |
A value that does not parse makes pds serve exit 1, with why on standard
error.
Resolving a handle
A handle outside PDS_HANDLE_DOMAINS is an external one, which the server
takes for an account only if it resolves to that account's DID: in
createAccount with a did (MIGRATION.md) and in updateHandle
(below). It resolves the handle, lowercased, as atproto/handle.md
describes, both ways at once:
- DNS. The TXT records of
_atproto.<handle>, each record's strings joined, asked once and given 3 seconds. Of those startingdid=, exactly one must be there; what followsdid=is the DID. None, two or more, or a lookup that fails or takes longer, find none. - HTTPS.
GET https://<handle>/.well-known/atproto-did, following up to 10 redirects, each through the same guard, given 3 seconds in all. A200whose body's first line, trimmed of white space, starts withdid:finds that line; anything else, no answer in time included, finds none. Only the first 8192 bytes of the body are read, and a longer body is read no further, not refused; bytes that are not UTF-8 are read as U+FFFD.
DNS's DID wins when it finds one; otherwise the HTTPS one. The handle resolves to the account's DID only if the DID found is that DID exactly. Nothing is kept: each check resolves the handle again, and an account's handle is not checked again once it is the account's.
With PDS_HANDLE_FETCH set, the HTTPS fetch goes to <base>/<handle>/.well-known/atproto-did
instead, as PDS_OAUTH_CLIENT_FETCH maps client documents
(OAUTH-CLIENTS.md): through the same guard, so that a base on this
machine or a private network is reached only with
PDS_DISABLE_SSRF_PROTECTION=true as well. A server open to the internet
must not set it. Each fetch is an outbound request of target handle
(OPERATIONS.md); a DNS lookup is none.
resolveHandle
resolveHandle takes no token, and ignores one, even one the server did
not give out. A handle that is missing, or is not a handle by
atproto/handle.md, is 400 InvalidRequest. Otherwise the handle is
lowercased, and:
- an active account's handle, in any case, answers
{did}, the account's DID; one whose account is not active is as no account's (LIFECYCLE.md); - a handle ending in one of
PDS_HANDLE_DOMAINSthat is no account's is400 HandleNotFound: one no account has had, one an account had before it changed it, and one no account could have, such asa.b.testoradmin.test; - any other handle is not the server's, unless it is an account's
(above). The reference asks its AppView (
PROXY.md), and answers400 InvalidRequestwhen that finds no DID. The server does not look it up, and answers400 InvalidRequestfor every such handle, whatever its top-level domain.
updateHandle
updateHandle changes the handle of the token's account. It checks, in
this order, and answers the first error:
- The token: none is
401 AuthMissing, and one the server did not give out400 InvalidToken, however wrong the input. A session made with an app password, privileged or not, may change the handle, as the reference's may (SESSIONS.md). Then the rate limits ofLIMITS.mdcount it, after its input is checked. - The input against the lexicon: a
handlethat is missing, not a string, or not a handle byatproto/handle.mdis400 InvalidRequest. Fields the lexicon does not name are ignored. - The handle, lowercased, as step 4 of
createAccountchecks it (ACCOUNTS.md), but for a handle outside the server's domains:- a top-level domain that
atproto/handle.mddisallows, such as.exampleor.invalid, is400 InvalidHandle; - a handle ending in one of
PDS_HANDLE_DOMAINSmust have one label of 3 to 18 characters before it: a dot in it, then a length outside that range, is400 InvalidHandle, and then a label indata/reserved-handles.txtis400 HandleNotAvailable; - a handle not ending in one of them is an external one: it must
resolve to the account's DID ("Resolving a handle", above), else it
is
400 InvalidRequest. Its labels are not checked further.
- a top-level domain that
- A handle another account has, compared in lower case, is
400 HandleNotAvailable(wherecreateAccountanswersInvalidRequest).
A handle that passes is the account's, even when it is the handle the account has already: that is a change like any other, with its operation and its event. The server then:
- for a
did:plc, posts an update operation for the account's DID toPDS_PLC_URL, asatproto/did-plc.mddescribes. Its fields are those of the DID's last operation in the directory (asGET <PLC>/<did>/log/lastanswers it), which may be one the server did not post, withalsoKnownAschanged and nothing else: its firstat://entry becomesat://<handle>, or, with none,at://<handle>goes first.previs the CID of that last operation, and it is signed with the server's rotation key (PDS_PLC_ROTATION_KEY_HEX). Adid:webaccount posts none: its document must name the handle already, asMIGRATION.md("Adid:webaccount") says; - gives the account the handle, which frees the one it had: another account, or a new one, may take that at once;
- sends
#identityon the firehose, with the account'sdidand the newhandle, numbered after every event before it.
It answers 200 with no body. After it, every method that names an
account by handle knows it by the new one (describeRepo, getRecord,
listRecords, writes, createSession, resolveHandle, and
/.well-known/atproto-did), and by the old one knows no account:
describeRepo of it is 400 RepoNotFound, a write naming it 403 AccountNotFound. getSession and createSession answer the new handle.
The account's sessions still hold.
A refused change posts no operation, changes nothing, and sends no event.
If the directory refuses the operation, the handle does not change, there
is no event, and the answer is 400 InvalidRequest with the directory's
message; if the directory cannot be reached, 502 UpstreamFailure.
At once
- Two
updateHandles of different accounts to the same handle at once, or anupdateHandleand acreateAccountof it, give it to one account at most. The server answers the other as if the handle were already taken:400 HandleNotAvailablefor anupdateHandle. The reference sometimes answers the loser500 InternalServerErrorinstead, having already posted its operation, so that the loser's DID document names a handle another account has. That is a fault, and the server does not copy it: it posts no operation for a change it then refuses, and the document the directory serves for each of its DIDs names the account's handle. - Of two
updateHandles of one account at once, the reference refuses one when its directory refuses the second operation, built on the same last operation as the first:400 InvalidRequest. The server may instead make them one after the other. Whichever it does, the handle it ends with is the one the directory's document names.
Crashes
A handle change is kept as a write is (DURABILITY.md): one the server has
answered survives it being killed at any moment after, both in the server
and in the directory, which holds its operation. One it is killed during
lands whole or not at all. Whole is the account with the new handle, the
directory's log with the operation, and the #identity event; not at all
is the account with its old handle and the directory's log without the
operation.
As the directory and the server are apart, the server records the operation, with the handle, before it posts it, and keeps the handle taken while it is in flight. An operation it recorded and had not finished when it stopped does not hold up its start: it serves at once, and settles the operation in the background, asking the directory for the DID's log until the directory answers.
- Until it is settled, the account's handle is the old one everywhere:
getSession,describeRepo,resolveHandleand/.well-known/atproto-didall answer it. The new handle stays taken, so no other account can have it. AnotherupdateHandleof the account waits until it is settled, then goes on as any change does: while the directory cannot be reached, that is502 UpstreamFailure. - If the directory's log holds an operation with its CID, the directory
accepted it, and the change lands: the account takes the handle, the old
one is free, and
#identitygoes on the firehose, numbered after every event before it. - If not, the change is dropped: the new handle is free again, and nothing else changes.
With the directory up, as the harness runs it, that is settled as the server starts, before or soon after its first answer.
/.well-known/atproto-did
GET /.well-known/atproto-did resolves the handle the request's Host
header names, less any :port, as atproto/handle.md's HTTPS method
asks. The host is matched as it is given, not lowercased: as every handle
here is lower case, one with a capital letter names no account here. An
account's handle answers 200, Content-Type: text/plain; charset=utf-8,
with the account's DID as the whole body, with no newline. Any other host,
including a handle an account had before, the server's own hostname, and
an IP address, is 404.
getRecommendedDidCredentials
getRecommendedDidCredentials needs the account's token: none is 401 AuthMissing, and one the server did not give out 400 InvalidToken. It
answers what a DID document of the account on this server holds, as
createAccount's genesis operation would hold it now:
alsoKnownAs,["at://<handle>"], the account's handle now, whatever the directory's document says;verificationMethods,{atproto: <the signing key's did:key>, atproto_kem: <the KEM key's did:key>}(SPACES.md);rotationKeys,PDS_RECOVERY_DID_KEYif set, then the rotation key'sdid:key;services,{atproto_pds: {type: "AtprotoPersonalDataServer", endpoint: PDS_PUBLIC_URL}}.
describeRepo's didDoc and handleIsCorrect
describeRepo's didDoc is the document the directory serves for the DID
now, or for a did:web, the document its host serves (MIGRATION.md, "A
did:web account"): after a handle change, the one that names the new
handle, and after an operation the server did not post, the one that
operation made. handleIsCorrect is whether the first at:// entry of
the document's alsoKnownAs is at://<handle>, the account's handle,
which for a did:plc account here is true, and for a did:web one
whatever its owner's document says. A did:web document that cannot be
read makes describeRepo 400 InvalidRequest, as the reference answers.
The reference keeps the DID documents it resolves for an hour, and its
describeRepo answers the one it kept, so after a change it answers the
document as it was, with handleIsCorrect: false. The server keeps them
too, for PDS_DID_CACHE_SECS, so that describeRepo does not call the
directory each time; but after an operation it posts, a handle change, it
serves the document that operation made at once (LIMITS.md). An
operation it did not post shows once what it kept is older than
PDS_DID_CACHE_SECS; the harness's runs set it to 0, so that every
describeRepo asks the directory, and the run of LIMITS.md checks the
rest.
How it is checked
Every run serves handles to both sides from harness/lib/handles.mjs: a
DNS server the server is given as PDS_HANDLE_DNS_SERVER, and an HTTP
web it is given as PDS_HANDLE_FETCH, with
PDS_DISABLE_SSRF_PROTECTION=true to reach it. The reference, which runs
in the harness's process, asks the same two for every handle it resolves.
Each answers only the handles a check gave it; every other handle
resolves to nothing, both ways.
harness/checks/identity.mjs holds the server to the reference, as
PDS.md says, with each account's DID named alike on both sides:
resolveHandleof an account's handle in mixed case; of a handle under.testthat no account has, that an account had before, or that none could have; of an external handle, underexample.com,example.net,example.orgor.invalid, which resolve to nothing; and of handles the lexicon refuses. Compared whole.updateHandleto a new handle, to the account's own in mixed case, or to one another account had before; and with each fault of its checks, alone or with a missing or bad token, compared by status anderror. External handles here are only those underexample.com,example.netandexample.org, which resolve to nothing; the migration run (MIGRATION.md) changes handles to external ones that resolve, by DNS or HTTPS, to the account's DID or another's. After each, on each side, the directory's log must have grown by one operation, or by none for a refused change, and that operation must be the update above:prevthe CID of the one before, every field butalsoKnownAsthe same, and its signature the side's rotation key's. ThengetSession,describeRepoby the new handle and the old,resolveHandleof both, and/.well-known/atproto-didof both are compared.- Sometimes, before a change, the harness posts an operation of its own for
the DID on each side, signed with that side's rotation key, that adds an
https://entry toalsoKnownAs: the change must keep it, and build on that operation. /.well-known/atproto-didwith theHostof an account's handle, with a port, in mixed case, of a handle no account has, or of a host that is no handle. The status is compared, and for a200, the body, with DIDs named alike, and that the content type istext/plain.getRecommendedDidCredentialswith the account's token, none, or one never given out, compared with keys and URLs taken out; on each siderotationKeysmust be the side's recovery and rotation keys, andverificationMethods.atprotothe account's signing key. On the server,verificationMethods.atproto_kemmust be the KEM key of the account's genesis operation; thelatestreference has no KEM key, so that entry is held to this rule and taken out before the sides are compared (PDS.md).- Two accounts changing to the same new handle at once: as many changes
land on each side, one, and on the server the other is
400 HandleNotAvailable, and each account's handle is the one its DID document names. Which account wins may differ between the sides, so the harness then gives both accounts new handles, one after the other, on both. As that makes events the other side does not have, this is not sent with--firehose.
With --firehose, each #identity event must carry the handle the
account had been given by then, in order, and the server's events other
than commits are compared with the reference's (SYNC.md). With
--crash-every, the server is sometimes killed while a handle change is
on its way. Once it serves again, the directory's log must be as it was,
and then the server's handle must stay the old one; or have the update
operation, and then the server must answer the new handle within 3
seconds. A change that landed is then made on the reference too, and its
events compared. That the server starts and answers while the directory
is down is not checked: the harness's directory is always up.
As the reference keeps DID documents for an hour, the harness, which runs
it in its own process, has it resolve a DID's document again after every
change it makes to it, so that its describeRepo answers as the server's
must. handleIsCorrect is then compared.
Not compared: the body and content type of a 404 from
/.well-known/atproto-did; the reference also answers that path with a
trailing slash, in any case, and to HEAD, and takes the host from
X-Forwarded-Host on a request from the loopback address, none of which
a client resolving a handle sends. A directory that refuses an operation
or cannot be reached, as the harness's directory takes every operation it
is sent. Two changes of one account at once, which the reference answers
by a race. An updateHandle and a createAccount of one handle at once,
where which wins decides which accounts exist. The reference may refuse
handles it finds offensive, and so may the server; the harness sends none.