~ knotless Search/
↑↓ move↵ open⇧↵ all results
0
0
164 lines · 8.5 KB Raw History

Federation: what a relay reads

The com.atproto.sync methods a relay uses besides getRepo, getLatestCommit and the firehose (SYNC.md): listRepos, getRepoStatus, getBlocks and getRecord; and the requestCrawl the server sends to the relays it is configured with. What every method shares, and how the server is checked, is in PDS.md; the repository they serve is as REPO.md says.

Each names an account by did, which its lexicon requires to be a DID: a handle there is 400 InvalidRequest, as is any other argument its lexicon refuses, before anything else is checked. Then a DID no account here has is 400 RepoNotFound. An account that is not active is answered as LIFECYCLE.md says; what follows is of an active one.

Environment

Besides those PDS.md and ACCOUNTS.md list:

VariableIs
PDS_CRAWLERSoptional: the base URLs of the crawlers (relays) to notify, comma-separated, such as http://127.0.0.1:2470

listRepos

listRepos answers {repos, cursor}: every account whose repository has its first commit, oldest first, in the order createAccount made them. Each is {did, head, rev, active: true}, with no status: head the CID of the repository's latest commit and rev its rev, as getLatestCommit answers them.

  • limit is 1 to 1000, by default 500; outside that, or not an integer, it is 400 InvalidRequest.
  • A page with repos has a cursor, whether or not it is full; the next page, from that cursor, starts with the account after the page's last. A page with none has no cursor, so the one after the last account is {repos: []}. A cursor stays good: the same cursor asked again later, after a restart too, lists the accounts made since. An empty cursor is the same as none.
  • The server may choose what a cursor holds. The reference's is <ms>::<did>, the account's creation time and DID, and one it cannot read is 400 InvalidRequest. The harness sends each side only the cursors it gave out, or none.

One thing the reference does is a fault, and the server does not copy it. The reference keeps each account's head and rev for listRepos apart from its repository, and updates them after each commit; killed between the two, it lists the commit before, until the account's next commit. The server's listRepos names the latest commit, as getLatestCommit does, after a restart too.

getRepoStatus

getRepoStatus answers {did, active: true, rev}, rev the repository's latest commit's.

getBlocks

getBlocks answers the blocks of the account's repository whose CIDs cids lists, as a CAR file (application/vnd.ipld.car, atproto/repository.md) whose header has no roots, {version: 1, roots: []}. Each block is in it once, however often cids names it, and in any order.

  • cids is required, and each must be a CID: none, or one that does not parse, is 400 InvalidRequest, before the did is looked up.
  • The repository holds its latest commit and every MST node and record of its current tree. If cids names any CID it does not hold, the answer is 400 InvalidRequest, with no blocks, not the BlockNotFound the lexicon names. So is a block only another account's repository holds, though the server has it.
  • Its cost is that of the CIDs it names, not of the repository: a CID is looked up in an index, and one the repository does not hold is found missing without reading the tree (LIMITS.md).
  • A block the repository held before and no longer does (an earlier commit, or a node or record a later commit took out of the tree) is answered as one it does not hold, 400 InvalidRequest, as the reference answers it: the reference deletes such blocks, and the server need not keep them (STORE.md), and answers them so even while it still has them.

getRecord

getRecord (com.atproto.sync.getRecord, not the com.atproto.repo method of the same name) answers a CAR file whose root is the account's latest commit, with the blocks that prove the record at collection/rkey in it, or prove that there is none:

  • the commit;
  • every MST node a search for the key collection/rkey passes through, from the root: in each node, the search ends at the entry whose key it is, or else goes on into the subtree just before the first entry whose key sorts after it (the node's l if that is its first entry, the last entry's t if no entry sorts after it), and ends where there is no such subtree;
  • the record, if the tree holds the key.

Nothing else, in any order. A record the tree does not hold is not an error: the answer is 200 with the proof of its absence, though the lexicon names RecordNotFound. A collection that is not an NSID, or an rkey that is not a record key (atproto/record-key.md), is 400 InvalidRequest.

requestCrawl

With PDS_CRAWLERS set, the server tells each crawler that it has something new: POST <url>/xrpc/com.atproto.sync.requestCrawl, with content-type: application/json and the body {"hostname": "<PDS_HOSTNAME>"}.

  • It does so at the first firehose event after it starts, and then at the first event at least 20 minutes after it last did; at no other time. So not when it starts, and not while there are no events: a write that is refused, or that makes no commit, makes no event. Each event is a notification due, not a request on its own, so a burst of events is one request to each crawler.
  • The requests go in the background: no answer of the server waits for one, and an answer that is an error, or a crawler that cannot be reached, is logged at most, not retried. Each crawler is told independently of the others.
  • When it last told them need not survive a restart: started again, the server may tell them again at its first event.

How it is checked

The harness starts both sides with the same three crawlers, each its own: one that answers 200, one that answers 500, and a URL where nothing listens. Both crawlers that listen record every request.

  • listRepos. Each side is paged through with its own cursors, at a random limit, and every page is compared, with each account's DID named as PDS.md says and head and rev taken out; on each side, each repo's head and rev must be what that side's getLatestCommit answers. The last page's cursor is kept, and at the next such step, asked again on each side: it must list the accounts made since, and only those. An empty cursor, and a limit out of range, are compared too.
  • getRepoStatus of an account, of a DID no one has, of a handle, and of something that is no DID, compared with rev taken out; rev must be the side's latest.
  • getBlocks of a random set of blocks of an account's repository, read from the reference's getRepo: MST nodes and records, the same on both sides since their trees are, and sometimes each side's own latest commit, sometimes with a CID named twice. Sometimes it adds a CID the repository never held (another account's latest commit, or a block of no repository, as dag-cbor or raw), and sometimes it is a DID no one has, a CID that does not parse, or no cids. Errors are compared by status and name; a CAR must have no roots, and hold exactly the blocks asked, each the bytes its CID names.
  • getBlocks of blocks the repository no longer holds: MST nodes and records the reference's tree had at an earlier such step and has not now, and each side's own earlier commit, sometimes with a block it holds now. Both sides must answer 400 InvalidRequest.
  • getRecord of a key of an account's repository, there or not, and of a bad collection or record key or a DID no one has. Errors are compared; a CAR's root must be the side's latest commit, the reference library's verifyProofs must verify it against the record's CID, or its absence, and the account's signing key, and its blocks besides the commit must be exactly the reference's.
  • requestCrawl. Before the first account is made, neither side has sent one. Once the first accounts are made, and at the end of the run, each crawler that listens must have had from each side the request above, compared by method, path, content-type and body, whose hostname is localhost on both sides: the first within 5 seconds, and no more than one, plus one for every whole 20 minutes the run has lasted, and from the server one more for each restart.

With --crash-every, after each restart, once the repositories are checked, listRepos and each account's getRepoStatus must list every account in the order made, with the head and rev the repository check found.