# Federation: what a relay reads The `com.atproto.sync` methods a relay uses besides `getRepo`, `getLatestCommit` and the firehose (`SYNC.md`): `listRepos`, `getRepoStatus`, `getBlocks` and `getRecord`; and the `requestCrawl` the server sends to the relays it is configured with. What every method shares, and how the server is checked, is in [`PDS.md`](PDS.md); the repository they serve is as [`REPO.md`](REPO.md) says. Each names an account by `did`, which its lexicon requires to be a DID: a handle there is `400 InvalidRequest`, as is any other argument its lexicon refuses, before anything else is checked. Then a DID no account here has is `400 RepoNotFound`. An account that is not active is answered as `LIFECYCLE.md` says; what follows is of an active one. ## Environment Besides those `PDS.md` and `ACCOUNTS.md` list: | Variable | Is | | --- | --- | | `PDS_CRAWLERS` | optional: the base URLs of the crawlers (relays) to notify, comma-separated, such as `http://127.0.0.1:2470` | ## listRepos `listRepos` answers `{repos, cursor}`: every account whose repository has its first commit, oldest first, in the order `createAccount` made them. Each is `{did, head, rev, active: true}`, with no `status`: `head` the CID of the repository's latest commit and `rev` its rev, as `getLatestCommit` answers them. - `limit` is 1 to 1000, by default 500; outside that, or not an integer, it is `400 InvalidRequest`. - A page with repos has a `cursor`, whether or not it is full; the next page, from that cursor, starts with the account after the page's last. A page with none has no `cursor`, so the one after the last account is `{repos: []}`. A cursor stays good: the same cursor asked again later, after a restart too, lists the accounts made since. An empty `cursor` is the same as none. - The server may choose what a cursor holds. The reference's is `::`, the account's creation time and DID, and one it cannot read is `400 InvalidRequest`. The harness sends each side only the cursors it gave out, or none. One thing the reference does is a fault, and the server does not copy it. The reference keeps each account's `head` and `rev` for `listRepos` apart from its repository, and updates them after each commit; killed between the two, it lists the commit before, until the account's next commit. The server's `listRepos` names the latest commit, as `getLatestCommit` does, after a restart too. ## getRepoStatus `getRepoStatus` answers `{did, active: true, rev}`, `rev` the repository's latest commit's. ## getBlocks `getBlocks` answers the blocks of the account's repository whose CIDs `cids` lists, as a CAR file (`application/vnd.ipld.car`, `atproto/repository.md`) whose header has no roots, `{version: 1, roots: []}`. Each block is in it once, however often `cids` names it, and in any order. - `cids` is required, and each must be a CID: none, or one that does not parse, is `400 InvalidRequest`, before the `did` is looked up. - The repository holds its latest commit and every MST node and record of its current tree. If `cids` names any CID it does not hold, the answer is `400 InvalidRequest`, with no blocks, not the `BlockNotFound` the lexicon names. So is a block only another account's repository holds, though the server has it. - Its cost is that of the CIDs it names, not of the repository: a CID is looked up in an index, and one the repository does not hold is found missing without reading the tree (`LIMITS.md`). - A block the repository held before and no longer does (an earlier commit, or a node or record a later commit took out of the tree) is answered as one it does not hold, `400 InvalidRequest`, as the reference answers it: the reference deletes such blocks, and the server need not keep them (`STORE.md`), and answers them so even while it still has them. ## getRecord `getRecord` (`com.atproto.sync.getRecord`, not the `com.atproto.repo` method of the same name) answers a CAR file whose root is the account's latest commit, with the blocks that prove the record at `collection/rkey` in it, or prove that there is none: - the commit; - every MST node a search for the key `collection/rkey` passes through, from the root: in each node, the search ends at the entry whose key it is, or else goes on into the subtree just before the first entry whose key sorts after it (the node's `l` if that is its first entry, the last entry's `t` if no entry sorts after it), and ends where there is no such subtree; - the record, if the tree holds the key. Nothing else, in any order. A record the tree does not hold is not an error: the answer is `200` with the proof of its absence, though the lexicon names `RecordNotFound`. A `collection` that is not an NSID, or an `rkey` that is not a record key (`atproto/record-key.md`), is `400 InvalidRequest`. ## requestCrawl With `PDS_CRAWLERS` set, the server tells each crawler that it has something new: `POST /xrpc/com.atproto.sync.requestCrawl`, with `content-type: application/json` and the body `{"hostname": ""}`. - It does so at the first firehose event after it starts, and then at the first event at least 20 minutes after it last did; at no other time. So not when it starts, and not while there are no events: a write that is refused, or that makes no commit, makes no event. Each event is a notification due, not a request on its own, so a burst of events is one request to each crawler. - The requests go in the background: no answer of the server waits for one, and an answer that is an error, or a crawler that cannot be reached, is logged at most, not retried. Each crawler is told independently of the others. - When it last told them need not survive a restart: started again, the server may tell them again at its first event. ## How it is checked The harness starts both sides with the same three crawlers, each its own: one that answers `200`, one that answers `500`, and a URL where nothing listens. Both crawlers that listen record every request. - **listRepos.** Each side is paged through with its own cursors, at a random `limit`, and every page is compared, with each account's DID named as `PDS.md` says and `head` and `rev` taken out; on each side, each repo's `head` and `rev` must be what that side's `getLatestCommit` answers. The last page's cursor is kept, and at the next such step, asked again on each side: it must list the accounts made since, and only those. An empty cursor, and a `limit` out of range, are compared too. - **getRepoStatus** of an account, of a DID no one has, of a handle, and of something that is no DID, compared with `rev` taken out; `rev` must be the side's latest. - **getBlocks** of a random set of blocks of an account's repository, read from the reference's `getRepo`: MST nodes and records, the same on both sides since their trees are, and sometimes each side's own latest commit, sometimes with a CID named twice. Sometimes it adds a CID the repository never held (another account's latest commit, or a block of no repository, as `dag-cbor` or `raw`), and sometimes it is a DID no one has, a CID that does not parse, or no `cids`. Errors are compared by status and name; a CAR must have no roots, and hold exactly the blocks asked, each the bytes its CID names. - **getBlocks** of blocks the repository no longer holds: MST nodes and records the reference's tree had at an earlier such step and has not now, and each side's own earlier commit, sometimes with a block it holds now. Both sides must answer `400 InvalidRequest`. - **getRecord** of a key of an account's repository, there or not, and of a bad collection or record key or a DID no one has. Errors are compared; a CAR's root must be the side's latest commit, the reference library's `verifyProofs` must verify it against the record's CID, or its absence, and the account's signing key, and its blocks besides the commit must be exactly the reference's. - **requestCrawl.** Before the first account is made, neither side has sent one. Once the first accounts are made, and at the end of the run, each crawler that listens must have had from each side the request above, compared by method, path, `content-type` and body, whose `hostname` is `localhost` on both sides: the first within 5 seconds, and no more than one, plus one for every whole 20 minutes the run has lasted, and from the server one more for each restart. With `--crash-every`, after each restart, once the repositories are checked, `listRepos` and each account's `getRepoStatus` must list every account in the order made, with the `head` and `rev` the repository check found.