# Spaces atproto spaces are permissioned data: records with an access perimeter, kept beside the public repository rather than in it, and read only with a credential the space's authority issues. This page is where spaces start: who takes part, the keys, what this server is in each part, and the building blocks every later part rests on, each specified exactly enough to build from and held to test vectors. What every method shares, and how the server is checked, is in [`PDS.md`](PDS.md). Spaces are an alpha. The protocol is `spaces/proposal.md`, the atproto proposal 0016 as vendored (`spaces/README.md` says which revision, and the other sources and their licences). Where this page and the proposal say the same thing, this page is the one to build from; where this page is silent, the proposal is. The reference for spaces is the pinned spaces alpha of the reference PDS, which the spaces run starts (below), not the `latest` reference the other pages compare with. ## Addressing A space is the triple of its **authority**, a DID; its **type**, an NSID; and its **key** (`skey`), a string with the syntax of a record key (`atproto/record-key.md`) and at most 512 bytes long. It is written as a URI, and a record in it as a longer one: ``` at://{authority}/space/{type}/{skey} at://{authority}/space/{type}/{skey}/{author}/{collection}/{rkey} ``` `space` is a literal segment, which no collection can be, as an NSID always has dots. A string in the format **`space-ref`** is a space's URI and no more: exactly the first form, with each part valid as said. ## Who takes part | Role | Is | This server | | --- | --- | --- | | author | an account writing records into a space, each into its own **space repo**: one per account and space | each of its accounts | | repo host | holds an author's space repos, serves their writes and reads, and tells the space host of each write | the repo host of its accounts' space repos | | space authority | the DID at the root of a space: its `#atproto` key signs the space's credentials, and it decides who may read the space and whose writes it tracks | an account of its own, for the spaces that account creates | | space host | the authority's PDS (its `#atproto_pds`): authorizes syncers, issuing their credentials, keeps the **writer set**, sequences the space's updates and notifies the syncers registered for them | the space host of every space whose authority is one of its accounts | | user's PDS | mints the delegation token an app trades for a credential | the PDS of its accounts | | syncer | an app keeping a copy of a space: pulls repos from their hosts, is told of the space's updates by the space host at an endpoint of its own | none: the harness plays it | The parts built on the building blocks below each have a page of their own: - the space repo: the `com.atproto.space` record writes and reads, its commit and operation log ([`SPACES-REPO.md`](SPACES-REPO.md)); - credentials: delegation tokens, space credentials, revocation, in [`SPACES-CREDENTIALS.md`](SPACES-CREDENTIALS.md), which verifies credentials for the methods that take them; - policies and the space lifecycle: creating, updating and deleting spaces, and member lists: [`SPACES-POLICIES.md`](SPACES-POLICIES.md); - full-state sync: a repo as a STAR archive (`getRepo`), blobs, and importing a repo: [`SPACES-EXPORT.md`](SPACES-EXPORT.md); - write notifications, the writer set, syncers' registrations and the space updates sent them: [`SPACES-SYNC.md`](SPACES-SYNC.md). Their lexicons join `lexicons/` when they are served; `spaces/lexicons/` has those the server calls on others and does not serve. `com.atproto.space.defs` (the commit, and the updates a writer set holds) is in `lexicons/`, with the space repo's methods. ## Keys Each account has two keys, both kept by the server for the account alone: - **`#atproto`**, the signing key `ACCOUNTS.md` gives it. For spaces, it is also the key that signs the delegation tokens the account's PDS mints, the credentials of the spaces the account is the authority of, and the service auth it sends as their authority (later parts). - **`#atproto_kem`**, a P-256 key of its own, below. It signs nothing: a syncer and the repo host agree a shared secret with it, under which the repo host authenticates what it serves with MACs. ### The KEM key `createAccount` makes the account a new P-256 keypair, its **KEM key**, with the signing key (`ACCOUNTS.md`), and keeps the private key as it keeps the signing key: it survives restarts (`DURABILITY.md`), and no other account's data holds it. Its public key, as a `did:key` (below), is: - `verificationMethods.atproto_kem` in the DID's genesis operation, beside `verificationMethods.atproto`; - `verificationMethods.atproto_kem` in what `getRecommendedDidCredentials` answers (`IDENTITY.md`); - in the DID document the directory serves, as the directory renders a verification method named `atproto_kem`. The proposal's form is a `Multikey` with id `#atproto_kem`. The harness's directory renders it in the legacy form `atproto/did.md` describes: id `#atproto_kem`, type `EcdsaSecp256r1VerificationKey2019`, the uncompressed point, and `https://w3id.org/security/suites/ecdsa-2019/v1` added to `@context`. The server serves what the directory serves, as it does for the signing key. The KEM key does not change: an update operation the server posts, as for a handle change, keeps it, as it keeps every field but the one it changes. ### describeServer `describeServer` answers `spacesAlpha: true` besides what `ACCOUNTS.md` and `BLOBS.md` list: the reference's flag that it serves the spaces alpha. *Note:* `describeServer`'s lexicon does not name it; the reference answers it all the same, and calls it temporary. ## The building blocks Each is held to a file of test vectors in `interop/spaces/` (below), and STAR to the published fixtures in `interop/star/` as well. Hex in a vector is lower case; a vector's `comment` says what the case is about. ### P-256 did:keys A P-256 public key as a string is a `did:key`, as `atproto/cryptography.md` defines it: `did:key:z`, then base58btc of the bytes `0x80 0x24` (the multicodec `p256-pub`) followed by the 33-byte compressed point (`0x02` or `0x03`, then `x`). Every P-256 `did:key` starts `did:key:zDn`. A key share or credential key that is not a `did:key`, does not parse, is of another curve (such as secp256k1's `0xE7 0x01`), is not 33 bytes of compressed point, or names an `x` with no point on P-256, is no P-256 key. The uncompressed form of a point is 65 bytes: `0x04`, `x`, `y`, each 32 bytes big-endian. ### The shared secret A syncer that wants a MAC sends a **key share**: a fresh P-256 public key, as a `did:key`. The repo host makes the shared secret from it and the author's KEM private key `d`, as HPKE's secret export (RFC 9180 §5.3, `spaces/rfc9180.txt`) in base mode, with the ciphersuite DHKEM(P-256, HKDF-SHA256), HKDF-SHA256 and the Export-only AEAD, the key share as `enc`, `info` the ASCII bytes `atproto-mac`, an empty exporter context and a length of 32. Spelled out, `‖` joining bytes, quoted parts ASCII, and `u16(n)` the two bytes of `n` big-endian: 1. `P` is the key share's point (an error if it is no P-256 key, as above). 2. `dh` is the 32-byte big-endian `x` of the point `d·P` (ECDH). 3. `enc` is `P` uncompressed, and `author` is the author's KEM public key uncompressed, 65 bytes each. 4. `Extract(salt, ikm)` is HMAC-SHA256 of `ikm` keyed with `salt` (an empty `salt` an empty key), and `Expand(prk, info)` is HMAC-SHA256 of `info ‖ 0x01` keyed with `prk`: HKDF's extract, and its expand to 32 bytes. 5. With `kem` the bytes `"KEM" ‖ u16(0x0010)` and `suite` the bytes `"HPKE" ‖ u16(0x0010) ‖ u16(0x0001) ‖ u16(0xFFFF)`, `LabeledExtract(id, salt, label, ikm)` is `Extract(salt, "HPKE-v1" ‖ id ‖ label ‖ ikm)`, and `LabeledExpand(id, prk, label, info)` is `Expand(prk, u16(32) ‖ "HPKE-v1" ‖ id ‖ label ‖ info)`. 6. `kem_secret` is `LabeledExpand(kem, LabeledExtract(kem, "", "eae_prk", dh), "shared_secret", enc ‖ author)`: DHKEM's shared secret (RFC 9180 §4.1). 7. `context` is `0x00 ‖ LabeledExtract(suite, "", "psk_id_hash", "") ‖ LabeledExtract(suite, "", "info_hash", "atproto-mac")`, and `exporter_secret` is `LabeledExpand(suite, LabeledExtract(suite, kem_secret, "secret", ""), "exp", context)`: the key schedule, with no pre-shared key. 8. The **shared secret** is `LabeledExpand(suite, exporter_secret, "sec", "")`. The syncer makes the same secret from its private key and the author's public one. `interop/spaces/kem.json`: each valid case gives `dh`, `kem_secret` and `shared_secret` for its author key, key share and `info`; the first is RFC 9180's test vector A.3.1, whose `kem_secret` is the RFC's `shared_secret`, and the second is HPKE's own test vector for this ciphersuite, with its `info`. Each invalid key share must be refused. ### The set hash, records and commits A space repo's records are summed by **LtHash**: - The state is 2048 bytes, read as 1024 unsigned 16-bit lanes, little endian; it starts all zeros. - An element is a string. To add it, expand its UTF-8 bytes to 2048 bytes with BLAKE3 in its extendable-output mode (the plain hash, unkeyed, read to 2048 bytes), read those as 1024 little-endian lanes, and add each to the state's lane of the same index, modulo 65536. To remove it, subtract instead, modulo 65536. - The digest is SHA-256 of the 2048 bytes of the state. So the state depends only on which elements are in it, not the order they came in, and the empty digest is `e5a00aa9991ac8a5ee3109844d84a55583bd20572ad3ffcd42792f3c36b183ad`. `interop/spaces/lthash.json` gives digests for elements added and removed, and the whole state after adding `a`. A record in a space repo is the same DRISL (DAG-CBOR) value a public record is (`REPO.md`), with the same CID: version 1, codec `0x71`, a SHA-256 multihash of its bytes, in base32. Its **element** is `{collection}/{rkey}/{cid}`, the CID as a string. A space repo's **commit** (`com.atproto.space.defs#commit`) is `{ver: 1, repo, space, rev, data}`: `repo` the author's DID, `space` the space's URI, `rev` a TID, and `data` the LtHash digest of the elements of every record the repo holds, 32 bytes. It has no signature. ### HTTP Message Signatures Both a request made with a space credential and a repo host's MAC are HTTP Message Signatures, as RFC 9421 (`spaces/rfc9421.txt`) defines them, with the choices below. What they share: - `Signature-Input`, `Signature` and `Accept-Signature` are RFC 9651 Dictionaries. A signature's **label** is its member's name, and carries no meaning: a signature is found by its `tag` parameter. The members **tagged** `T` are those whose value is an Inner List with a parameter `tag` that is the String `T`; any other member (another tag, no tag, a `tag` that is a Token, a value that is no Inner List) is passed over. A member, or a parameter, given more than once counts as its last, as RFC 9651 reads a Dictionary. - The **signature base** is one line per covered component, in the Inner List's order, then the line `"@signature-params": ` and the Inner List with its parameters, serialized as RFC 9651 §4.1.1.1 serializes one, so `;sf=?1` is written `;sf`; the lines are joined by a line feed, with none after the last, in UTF-8. A component's line is the component as serialized there, `: `, and its value. - `alg`, if a signature has it, must be the String the signature's kind names below; `keyid`, if it has it, must be a String, the key it is checked against. **A credential's signature.** A request made with a space credential carries it in `Authorization`, and signs it with the key the credential is bound to (its `cnf.kid`, a P-256 `did:key`), as in: ``` Authorization: Atproto-Space-Credential Atproto-Space-Audience: Signature-Input: sig=("authorization" "atproto-space-audience");tag="atproto-space-credential" Signature: sig=:<64 bytes, base64>: ``` To verify one against the credential's key: 1. `Signature-Input` must have exactly one member tagged `atproto-space-credential`, and `Signature` a member of the same label. In `Signature-Input` it must be an Inner List of exactly the Strings `"authorization"` and `"atproto-space-audience"`, in that order, with no parameters on them. `alg`, if given, must be `ecdsa-p256-sha256`, and `keyid`, if given, the credential's key; other parameters are allowed (they are signed, in the base). In `Signature` the member must be a Byte Sequence with no parameters, of 64 bytes; base64 padding is optional. 2. The base's two component lines are ``` "authorization": "atproto-space-audience": ``` each value as received, less leading and trailing whitespace: `Atproto-Space-Credential` keeps the case it was sent in. 3. The 64 bytes are `r ‖ s`, each 32 bytes big-endian. The signature must verify as ECDSA over P-256 with SHA-256 of the base, against the credential's key, which must be a P-256 `did:key`. Both low-S and high-S signatures verify; a DER signature is refused. `interop/spaces/http-signature.json`: requests' headers, each `valid` or not against `key_id`, with the exact `signature_base` of each valid one. These hold the signature alone; the credential itself, and that the audience is the one the request needs, are the credentials' part (later). ### MACs A repo host authenticates a commit or a record to a syncer with a MAC: an HTTP Message Signature with the algorithm `hmac-sha256` (RFC 9421 §3.3.3), keyed with their shared secret, over a response header that describes what it answers. There are two **profiles**, each a tag and the header it covers: | Profile | For | Covers | | --- | --- | --- | | `atproto-space-commit` | a commit | `Atproto-Space-Commit` | | `atproto-space-record` | a record | `Atproto-Space-Record` | **Asking.** A syncer asks for a MAC with the request header `Accept-Signature` (RFC 9421 §5.1), as in: ``` Accept-Signature: mac=("atproto-space-commit";sf);keyid="did:key:zDn...";tag="atproto-space-commit" ``` The repo host reads it for the one profile the method answers: 1. No header, or one with no member tagged with the profile, asks for no MAC; members tagged otherwise are passed over. 2. A header that is not a Dictionary is refused, and so is one with more than one member tagged with the profile. 3. That member's Inner List must be exactly one item, the String naming the profile's header in lower case (`"atproto-space-commit"` or `"atproto-space-record"`), with exactly the one parameter `sf`, the Boolean true. Its parameters must have `keyid`, a String, the **key share**, and may have `alg`, which must be the String `hmac-sha256`; any other parameter is refused, `created` and `nonce` among them. 4. The key share must be a P-256 key (above), or the request is refused. **Answering.** The response has three headers more, and `Vary` naming `Accept-Signature`: ``` Atproto-Space-Commit: ver=1, repo="did:plc:...", space="at://...", rev="3mwcj3b6uak2f", data=:: Signature-Input: mac=("atproto-space-commit";sf);keyid="did:key:zDn...";tag="atproto-space-commit" Signature: mac=:: ``` - The covered header is a Dictionary, serialized as RFC 9651 serializes one: for a commit, its fields in the order `ver` (an Integer), `repo`, `space` and `rev` (Strings) and `data` (a Byte Sequence of its 32 bytes); for a record, `uri`, its full space URI, and `cid`, its CID as a string, both Strings, as in `uri="at://...", cid="bafyrei..."`. - `Signature-Input`'s one member has the request's label, and its value is the request's Inner List, serialized as above. - `Signature`'s one member has the same label, and its value is the MAC as a Byte Sequence: HMAC-SHA256, keyed with the shared secret of the key share and the author's KEM key, of the signature base, whose one component line is the profile's, as in ``` "atproto-space-commit";sf: ver=1, repo="did:plc:...", space="at://...", rev="3mwcj3b6uak2f", data=:...: "@signature-params": ("atproto-space-commit";sf);keyid="did:key:zDn...";tag="atproto-space-commit" ``` **Checking**, as a syncer does: `Signature-Input` has exactly one member tagged with the profile, an Inner List of exactly the one item above, with `keyid`, if given, the key share asked with and `alg`, if given, `hmac-sha256` (other parameters are allowed: they are in the base); `Signature` has a member of its label, a Byte Sequence of 32 bytes with no parameters; the covered header, read as a Dictionary and serialized again (the `sf` flag), gives the base; the MAC matches it; and the header names what the answer holds: exactly the profile's fields, none with parameters, each of its type and equal to the answer's. Which methods take `Accept-Signature`, and the error they answer when it is refused, are the space repo's (a later part). `interop/spaces/mac.json`: for each commit and record, the shared secret, the request, the covered header, the signature `base`, the `mac` and the response headers; and `accept_signature`, request headers each with the profile it is read for and what it yields: the label, key share and serialized Inner List, `null` where no MAC is asked, or `"refused"`. ### STAR A space repo travels as a STAR archive, as `spaces/star.md` defines STAR: the magic bytes `0x2A 0x6C 0x01 0x00`; the metadata, a DRISL map with a String `$type`, after its length; then each pair as `0x01`, the key's length, the key, the value's length and the value; then `0x00`, and nothing after. Lengths are unsigned varints, minimally encoded, at most 9 bytes. Any fault refuses the whole archive. `interop/star/` has STAR's published fixtures: each valid one must parse to its metadata and pairs, and each invalid one must be refused. The profile `atproto-space-repo` is a space repo: - **The metadata** is the repo's commit, with `$type`: `{$type: "atproto-space-repo", ver: 1, repo, space, rev, data}`, `data` as DRISL bytes. A writer writes these six fields; a reader ignores any others STAR allows. - **A pair** is a record: its key the UTF-8 bytes of `{collection}/{rkey}`, its value the record's DRISL bytes, those its CID names. Pairs are in strict ascending order of the key bytes, so `com.example.x.y/1` comes before `com.example.x/10`, as `.` is `0x2E` and `/` is `0x2F`; an empty repo has none. So the archive of a commit and its records is one string of bytes. - **Limits**: the metadata is at most 65536 bytes, a key 4096, a value 1048576. A reader, given the space and repo it asked for, the key share it asked with, the shared secret and the response's headers, refuses the whole archive on any of these, and otherwise takes it as the repo: - a fault of STAR's, a limit passed, or metadata that is not canonical DRISL; - a `$type` other than `atproto-space-repo`, a `ver` other than `1`, or a commit field missing or of another type; - a `space` or `repo` other than those asked for; - a commit MAC that does not check (above) against the response's headers, or an `Atproto-Space-Commit` header that names another commit than the metadata; - a key that is not a record path (an NSID, `/`, a record key), or one not after the key before it, duplicates included; - an LtHash digest of the records' elements, each CID computed from the value's bytes, other than `data`. `interop/spaces/repo-star.json`: valid archives, each exactly the archive of its commit and records, to serialize and to read; one a reader must take though no writer here makes it; and archives to refuse, with their space, repo, key share, secret and response headers. ## Test vectors | File | Holds | | --- | --- | | `interop/spaces/kem.json` | the shared secret | | `interop/spaces/lthash.json` | LtHash | | `interop/spaces/mac.json` | commit and record MACs, and `Accept-Signature` read | | `interop/spaces/http-signature.json` | a credential's HTTP Message Signature | | `interop/spaces/repo-star.json` | space repos as STAR archives | | `interop/star/*.json` | STAR's own fixtures, published with STAR | Each file says in its `description` how to read its cases. The server's own tests (`cargo test`) hold each building block to every case of its file. The spaces run holds every case to the reference's own library and to the harness's code written from this page (below), so each vector is one the reference and this page agree on; the one case the reference's library cannot take, `kem.json`'s second, whose `info` is not `atproto-mac`, is held to the harness's code alone. ## How it is checked `harness/spaces/compare.mjs` is the spaces run. It starts the pinned spaces alpha (`harness/spaces/package.json`; `pin-alpha.sh` pins another): its PLC directory and its PDS, the reference here. It starts the server beside it on that directory, in the environment `PDS.md` lists, with the alpha's recovery key as `PDS_RECOVERY_DID_KEY`, and as `PDS_LEXICON_AUTHORITY_DID` the DID of the run's lexicon authority, a PDS of its own on that directory publishing space type declarations, which the alpha resolves from too (`OAUTH-SCOPES.md`, "How it is checked"): ```sh cd harness/spaces && npm ci # once node compare.mjs --seed 1 [--accounts 3] -- ``` First it checks every vector, as above, and stops at the first that either side disagrees with. Then, on both sides, with each account's DID named alike as `PDS.md` says: - **createAccount** of a few accounts: the answers are compared, with keys, tokens and URLs taken out. Each genesis operation in the directory must be what `ACCOUNTS.md` says, with `verificationMethods.atproto_kem` a P-256 `did:key` of a point on the curve, other than the signing key; the answer's `didDoc` must be the directory's document; and that document must have the `#atproto_kem` verification method, its key the operation's. - **getRecommendedDidCredentials** of each: compared with keys and URLs taken out; on each side `verificationMethods.atproto_kem` must be the genesis operation's. - **describeServer**: `did`, `availableUserDomains`, `inviteCodeRequired` and `spacesAlpha` compared; `spacesAlpha` must be `true` on each side. - **updateHandle** of most of them: the operation it posts must keep every `verificationMethods` entry, and `getRecommendedDidCredentials` still name the same KEM key. - **A restart**: the server is killed and started again on its data, and every account's `getRecommendedDidCredentials` must name the KEM key it had, with no new operation in the directory. The reference is held to each check too, which checks the check. The other runs, against the `latest` reference, hold what the KEM key changes to this page instead (`PDS.md`). That the server makes the shared secret, MACs, signatures and archives the vectors hold, over HTTP, against a syncer, each later part checks through its own methods, in this run.