# Design ## Constraints knotless has to run against the reference PDS (`@atproto/pds`) as shipped, so public repositories use only `com.atproto.repo.*`, `com.atproto.sync.getBlob` and `com.atproto.server.*`. Private repositories add `com.atproto.space.*` and `com.atproto.simplespace.*` from the permissioned-data alpha of the same PDS. What that PDS allows shapes every choice below. | Limit (reference PDS defaults) | Consequence | | --- | --- | | Blob upload ≤ 5 MB (`PDS_BLOB_UPLOAD_LIMIT`) | Large packs are split into 4 MB parts; a part refused as too large is retried at half the size. | | JSON request body ≈ 1 MB; the firehose carries every record | Packs ride inline only up to 192 KiB (`KNOTLESS_INLINE_MAX`). | | `applyWrites` ≤ 200 writes | A push of more than ~99 refs spans batches. Each ref's claim travels in the same batch as the ref. | | Rate limits, when enabled (as on bsky.social): 5000 points/hour per account (create 3, update 2, delete 1); `uploadBlob` 1000/day | A small push costs 8 points (pack 3, claim 3, ref 2), so about 600 pushes an hour per account. Inline packs keep small pushes off the blob quota. | | `applyWrites` has no per-write swap, only `swapCommit` | Compare-and-swap comes from create-only claim keys (below), not from `swapCommit`, which would conflict with every other write to the account. | | A create on an existing key fails the whole batch, with a 500 | A refused batch is followed by a reread of the refs, to tell a lost race from any other failure. | | Only an account writes to its own repo | Collaborators push through a gateway the owner has delegated to (below). | | `listRecords` filters by collection, not by prefix | Keys share the repository's prefix, and scans start from the cursor `/`. The reference PDS orders by rkey and pages with `rkey < cursor`, so the scan stops at the first key outside the prefix. | ## Records ``` dev.knotless.repo key, name, description?, defaultBranch?, collaborators?, gateway?, forkOf?, transfer?, transferredFrom? dev.knotless.ref . name, target?, generation dev.knotless.claim .. name, generation, from?, to?, pack? dev.knotless.pack . tips, bases, size, objects, data | parts, supersedes? dev.knotless.redirect to, space? dev.knotless.pull repo, base, head {repo, branch}, title, body? dev.knotless.issue repo, title, body? dev.knotless.comment repo, subject, body, verdict?, commit? dev.knotless.state repo, subject, state?, commit?, base? dev.knotless.hide repo, subject dev.knotless.block subject, repo? ``` `h(ref)` is the first 26 characters of the lowercase base32 SHA-256 of the ref name, so any ref git allows becomes a valid record key. `key` is a TID minted when the repository is created. Every record carries it, so a rename rewrites only the `repo` record. The first five live in the owner's repo. The pull, issue, comment and state records live in each author's repo, and hides and blocks in the owner's; they are described under [Pulls and issues](#pulls-and-issues). ### Identity A repository is its owner's DID and its `key`. The name is for people: it can change, and once a repository is deleted or renamed away, another can take it. So a record that names a repository (a pull, issue, comment or state, a pull's head, a fork's `forkOf`) holds a `repoRef`: the `at://` URI of the `repo` record as it was named when written, and the key. Readers match on the URI's owner and the key, never the name, so those records follow a rename and never attach to a later repository of the same name. A rename, in one `applyWrites`, creates the `repo` record under the new name, deletes the old one, and leaves a `redirect` under the old name. It also repoints every redirect already leading to the repository, so an old name is always one hop from the current one, and deletes a redirect under the new name if there was one. A lookup by name that finds no `repo` record reads the `redirect` under that name and follows it, so clone URLs, remotes and links made with the old name keep working; `git` is told the new name. A redirect is followed only within the account and only to a record with the key it names. Creating a repository under a name that holds a redirect replaces it; deleting a repository deletes its redirects. A private repository's records stay in the space named for its first name, its home. A rename creates a space under the new name, with the same members, holding only a redirect whose `space` names the home; a member who opens the repository by its new name finds the redirect there and joins the home. Sharing and unsharing change every such space, and deleting the repository deletes them all. Members' cached credentials are bound to a space's address, so a space's name is never taken by another repository: creating one where a renamed repository's space is refuses. A repository moves to another account in two steps, since neither account can write to the other. The owner offers it, setting `transfer` on the `repo` record to a `repoRef` in the new account. The recipient accepts: the repository is copied into their account with the same key and `transferredFrom` naming the old place. A move holds only when both sides say so: a reader following the old name goes to the new place only if the new `repo` record names the old one, and records about the old repository count for the new one only if the old owner's offer, or the redirect they leave when they delete the old copy, leads there. State changes the old owner made before the new `repo` record was created still count; later ones do not. When the old copy is deleted, its name and redirects become redirects into the new account. A `knotless://handle/name` remote resolves the handle on every use. The helper records the DID the remote first resolved to (`remote..knotlessdid`) and refuses to push or fetch if the handle later names another account. ### Push 1. Read the `repo` record, the repository's refs and its newest page of packs, in parallel. 2. For each ref: refuse a non-fast-forward unless forced, and refuse an old target that is not available locally ("fetch first"). 3. `git pack-objects --revs` of the new tips, excluding every remote ref target the local repository has. The pack is self-contained: no thin deltas, so it can be indexed alone. 4. One `applyWrites`: create the `repo` record if this is the first push, create the pack, and for each ref create `claim(..)` and put `ref(., generation + 1)`. Two writers that read generation *n* both create `claim(...n)`, and the PDS accepts only the first. The ref update travels in the same batch, so it lands only with its claim. A deleted ref keeps its record, without a target, so its generation keeps counting and an old claim can never be replayed. ### Fetch Every wanted object present already: done. Otherwise, walk the pack pages newest first. Download, in parallel, each pack with a tip that is missing locally, and index it. Add the pack's `bases` to the wanted set, and stop once everything wanted is present. If a walk ends with something still missing, walk once more from the top: a compaction may have replaced packs between two pages. If it is still missing after that, report it. ### Compaction Repack every object the live refs reach into one pack, written with `supersedes` listing the packs it replaces, and delete those in the same `applyWrites`. Claims older than a day go in the same writes: a claim guards only the generation it names while a writer may still hold a read of it, and a push refuses to write from a read older than half that long. Only packs older than the grace period (ten minutes) are replaced. A reader that listed them before the delete either finishes inside that window or restarts its walk once, as above. Objects that no ref reaches are dropped here, and their blobs are dereferenced and removed by the PDS. ### Commits without a checkout `knotless commit` syncs a bare mirror in the cache directory. It builds the commit there through a temporary index (`read-tree`, `hash-object`, `update-index --index-info`, `write-tree`, `commit-tree`) and pushes it. Put and delete name paths, so replaying them onto a tip that moved cannot conflict. A lost race backs off with exponential jitter and replays. The claim pins the parent's generation, so these pushes skip git's fast-forward check. A commit to a branch that does not exist yet starts it from the commit the changes were made against, or else from the default branch. Commits are authored by whoever makes them, by handle. `knotless branch` makes and removes branches the same way, without a commit. Mirrors are shared by every process using the cache. Packs are indexed without a lock, because each lands under its own name. The ref transaction runs under a lock file, and a lock older than two minutes is treated as abandoned. ## Logging in `knotless login ` logs in with OAuth as atproto profiles it. knotless is a loopback client: its client ID is `http://localhost` with its scope and redirect in the query, so there is no client metadata to host. It finds the account's authorization server from its PDS (`/.well-known/oauth-protected-resource`), pushes the request with PKCE, opens the authorization page in a browser, and takes the code back on a listener on `127.0.0.1`. Tokens are bound to a P-256 DPoP key kept with the session; every request to the PDS carries a proof, and one refused for want of the server's nonce is sent again with it. Refresh tokens are used once, so processes refresh under a lock file, and one that finds the session already refreshed takes it. Logging out revokes the tokens. The scope is `transition:generic`, what an app password allows, plus a `space` scope for private repositories, which `transition:generic` does not reach. It names every space type: naming `dev.knotless.space` alone makes the authorization server resolve its lexicon, which is not published. A PDS without spaces drops the scope it does not know. An app password (`--password-stdin`, or `KNOTLESS_PASSWORD`) still logs in. A loopback client's session lasts two weeks at most, after which `knotless login` again. ## Private repositories A private repository is the space `at:///space/dev.knotless.space/`, created with open app access and member-list read and write policies. Its records are the public ones, written with the space's `applyWrites` into the owner's repo inside the space. A create on an existing key fails there too, so the claim compare-and-swap is unchanged. The owner reads their own repo in the space with their session. Anyone else, the owner included when reading another member's repo, exchanges a delegation token from their own PDS for a space credential from the owner's, signing both requests with a fresh P-256 key (RFC 9421, as the alpha specifies). Reads then carry the credential and an audience naming the repo being read. App passwords cannot mint delegation tokens, so reading someone else's private repository takes an OAuth login. The credential and its key are cached, readable only by the user, and reused until a minute before they expire. A repository is looked for publicly first, then in a space as whoever is logged in; which one it was is cached with its key. A PDS without spaces answers the space methods as unknown, and the repository is public. Every member is a space writer: the alpha lists a member's repo in the space only if their writes are allowed, and members write their issues and comments there. Pushing is a separate grant, the `collaborators` of the `repo` record. Deleting a private repository deletes the space. `knotless-serve` reads public records only, so what it shows anyone may see. With `--private` it looks in spaces too, as whoever is logged in. It reads a private repository's pulls and issues from its members' repos in the space, and lists the account's own private repositories on its profile (`com.atproto.space.listSpaces`, which names only spaces the account has written to, so not others' it may read). It then refuses to listen anywhere but a loopback address, since its pages would show that person's private repositories to whoever reached them. ## Gateway Only an account can write to its own repo, so a collaborator cannot push directly. A gateway is a `knotless-serve` with `--public-url`, identified as `did:web` of its host and serving its own DID document with a `#knotless_gateway` service. - **Delegation.** The gateway is an OAuth client of its own, a confidential one: its client ID is `/oauth-client-metadata.json` on its public URL, and it authenticates to authorization servers with a P-256 key kept in its config directory (`oauth-client-key`), published in that metadata. `knotless delegate` calls `dev.knotless.putDelegation` with a service token; the gateway pushes an authorization request for the owner and returns its URL, which the owner approves in a browser. The browser comes back to the gateway's `/oauth/callback`, and the gateway keeps the session, refreshing it as that client. `knotless delegate` waits on `dev.knotless.getDelegation`. The grant is narrow: the four knotless record types, blobs, and those collections in the owner's own spaces. A confidential client's session lasts up to two years if it is used every three months; the owner revokes it from their PDS's account settings. Sharing a repository for writing records the gateway in its `repo` record. - **Authentication.** Every call carries a service token from `com.atproto.server.getServiceAuth`, with the gateway as audience and the method as `lxm`. The gateway checks it against the signing key in the issuer's DID document (Multikey, or the legacy secp256k1 type PLC still returns), then checks the issuer is the owner or a collaborator. - **Push.** `dev.knotless.pushRefs` takes a JSON header line (the repository and ref updates) and then the pack. The gateway indexes it into its mirror and pushes as the owner, with the same compare-and-swap as any push. - **Commit.** `dev.knotless.createCommit` takes path changes and an optional `base`. Commits to one branch queue; whichever request takes the branch turn drains the queue, builds the commits as a chain and lands them in one push. If the branch moved, the chain is rebuilt on the new tip, up to eight times. A change to a path that also changed since its `base` is a conflict (409) for that commit alone. The gateway holds an OAuth session for the owner, so it is trusted with writing knotless records to the owner's account: run your own, or use one you trust. ## Pulls and issues Items and everything said about them are records in their authors' accounts, naming the repository as under [Identity](#identity). A pull names a target branch and a head repository and branch, which may be a fork. A comment with a verdict is a review and records the head commit it saw. An item's state is the latest `state` record, ordered by key, from someone allowed to set it: the item's author, the owner, or a collaborator. Only the owner and collaborators can set `merged`. A `state` with a `base` and no `state` moves a pull to another target branch, as after a branch is renamed, and leaves its state alone, so it cannot undo a merge it raced; the pull's author changes its head branch by rewriting the pull record. To list them, a client needs every author's records about one repository. For public repositories an appview indexes them from the firehose (`dev.knotless.getForge`, a page at a time); for private ones the client enumerates the space's member repos (`com.atproto.space.listRepos`) and reads each. A comment reads the item it is about straight from its author's repo, so it does not wait for an index. `knotless merge` syncs the target's mirror, fetches the head from the source's packs, and fast-forwards when it can. Otherwise it builds a merge commit with `git merge-tree --write-tree`, and stops on conflicts. It pushes directly as the owner or through the gateway as a collaborator, then writes a `merged` state naming the commit. Comparing a pull does the same up to the push: the commits it brings in, and whether it is merged already, fast-forwards, merges cleanly or conflicts, and on which paths. The diff of a pull is its head against where it left its target, wherever the head lives. Renaming a branch makes the new one, retargets the open pulls on the old one, then deletes it, so a rename cut short loses nothing. A pull or issue just opened may not be indexed yet, so showing one the appview lacks reads it from its author's repo if it is under a minute old, and neither the appview's takedowns (asked of its `queryLabels`) nor the owner's own hides and blocks keep it out. A fork copies every ref into a new repository whose `repo` record names its source; a fork of a private repository is private, and shared for reading with the source's owner so they can take pulls from it. ### Moderation Moderation is atproto's. A repository's owner keeps out what accounts they block write about it, and hides pulls, issues and comments with a `hide` record for each, as a postgate detaches quotes of one post: one record each, so there is no list to outgrow. What was said about a hidden item goes with it. A block is the forge's own record, shaped as `app.bsky.graph.block`: blocking someone on Bluesky says nothing about their pulls, and a block can name one repository, keeping its subject out of that one alone. A block naming a private repository is kept in the space with it. Readers apply both: the appview indexes owners' blocks and hides; in a space the client reads the owner's blocks from their repo and the space. Anyone can report an account, a repository or a record with `com.atproto.moderation.createReport`, sent with a service token to the moderation service the DID document of `KNOTLESS_MODERATOR` names (`#atproto_labeler`), or by default to the appview's. The service's operator reads reports with `dev.knotless.listReports` and takes subjects down with `com.atproto.admin.updateSubjectStatus`, both with the admin password as a PDS takes it. ## knotless-serve A stateless cache in front of PDSes. Per repository it keeps a bare mirror, synced on demand when older than `--ttl`. With `--firehose` it also follows `subscribeRepos`: a commit touching a `dev.knotless.*` collection marks that account's mirrors stale and refreshes them in the background. While the subscription is up, a mirror with no new events for its account is served without asking the PDS; while it is down, the TTL applies. With `--jetstream` the same watching runs over `network.bsky.jetstream.subscribeEvents`, asking only for `dev.knotless.*` commits (identity, account and sync events arrive for every account, since Jetstream filters collections on commits alone). Jetstream re-encodes the firehose as JSON and includes each record decoded, so the index takes the record from the event and writes it in event order, with no `getRecord`; it resumes from the sequence after the last one seen, as the cursor is inclusive. Jetstream frames are unsigned: the record is what the Jetstream operator says the commit held. That is the trust a server already gives the relay it follows, and the index stays a cache, rebuildable by `--backfill`. A cursor means something only on the host that issued it. Clones run through `git upload-pack --stateless-rpc`, so protocol v2, `--filter` and `--depth` work as they do anywhere. The API answers from the same mirror. Revisions are rejected if git could read them as options, and responses named by a full hash are marked immutable. With `--index` it is also an appview. Firehose events for knotless collections are indexed into SQLite by fetching the record as it now stands, or removing it; `--backfill` walks `com.atproto.sync.listRepos` on a PDS or relay once at start. The index answers `dev.knotless.listRepos` (newest first, matched on name and description, paged by cursor) and `dev.knotless.getForge`, which takes the repository's URI and key and returns the records naming that owner and key under any name, paged by URI. The index is a cache: one built for another version of its tables is dropped and rebuilt from the firehose and `--backfill`. An appview with `--public-url` is also a moderation service. It keeps the reports it is sent and its takedowns in the same file, outside the cache, so a rebuild keeps them. A takedown of an account or record leaves the index, the pages, the API and clones at once. Each goes out as a `!takedown` label (or its negation) on `com.atproto.label.subscribeLabels` and `queryLabels`, and with `--labeler ` an appview follows another labeler's stream and honours its `!takedown` and `!hide` labels the same way. Labels go out unsigned; a follower trusts the endpoint the labeler's DID document names. ### Web interface Read-only HTML over the same mirrors and index. Pages read objects in process through `gix`, opened once per mirror, because starting a git process costs more than the reads themselves; only a commit's diff runs `git show`. Each page is one document with its styles inline and two fixed scripts, allowed by hash in a `default-src 'none'` content security policy: - **Highlighting.** Code is sent as escaped text. Tree-sitter grammars for the common languages produce ranges, which travel in a `data-hl` attribute as `name:gap,len,…` in UTF-16 units; the script turns them into `Range`s for the CSS Custom Highlight API, and marks the line a `#L12` fragment names. Files over 1 MB, and languages without a grammar, are shown plain. - **Diffs.** A commit's patch is read into files and rows, each row a line with its old and new numbers, shown as a table. The ranges for a diff run over the rows' text as if joined by newlines, and the script cuts each range at row boundaries. Removed lines are highlighted as the old side of the hunks reads, the rest as the new side reads, so a comment or string that spans lines is still recognized. A removed line paired with an added one also gets the part between their common prefix and suffix marked, when that shared part is at least half the shorter line. A file with more than 2000 rows is folded away. - **Navigation.** Speculation rules prerender a same-site link as the pointer settles on it, and cross-document view transitions keep the header still between pages. - **Caching.** Pages named by a commit hash are immutable; pages named by a branch are revalidated. HTML is compressed; git responses are not. - **Social cards.** The front page, each repository's pages, and each pull and issue name a 1200×630 PNG in Open Graph tags, at the page's path plus `/card.png`. A card is an SVG written like the pages, drawn by `resvg` with fonts compiled in, so it is the same on any host: two Inter Display faces subset to Latin, and Noto Color Emoji (COLRv1, 5 MB) for emoji; other scripts show as boxes. Each run of emoji is placed on its own, at an x measured here: shaped together with text in another font, resvg drops emoji sequences, or panics on them. A card is written with a palette when it has fewer than 256 colors, as one without emoji does: exact, a third of the bytes to compress, and smaller with a fast compressor than RGB with a thorough one. The URL carries `?v=` and a hash of the SVG, so under that version it is immutable, and a page names a new URL when what its card shows changes, which is also what makes chat apps that cache previews by URL show the new one. Under any other version, it is the current card, revalidated by ETag. Cards are kept in memory by version, the oldest of 512 dropped, and drawn on blocking threads, one per core at most. The origin comes from `Host`, and `X-Forwarded-Proto: https` from a proxy in front. - **Layout.** One stylesheet of custom properties, redefined for dark mode, with a single column under 640 px. Nothing is rendered lazily with `content-visibility`, so a full-page screenshot is the same every time. An account's page, at `/`, lists its public repositories from its own PDS rather than the index, so it is right without an appview, and shows the display name and description of its Bluesky profile record if it has one. Private repositories are in spaces and never listed there. A page asked for by a repository's old name redirects to the same page under its new name. The redirect is temporary, as the old name can be given to another repository. Clones over HTTP follow the old name without a redirect. README markdown is rendered with raw HTML shown as text, relative links pointed into the repository, and links with schemes other than http(s) and mailto dropped. Raw files are served sandboxed, typed only as text, an image, or bytes. A handle is shown for a DID only if it resolves back to that DID. ## knotless-mcp An MCP server over stdio, acting as the logged-in account (or `KNOTLESS_IDENTIFIER` and `KNOTLESS_PASSWORD`). Its tools are the CLI's operations with structured results: reads answer from the cache mirror, writes go through the same code as `knotless commit`, `fork`, `pull` and `merge`, and errors come back as tool errors for the agent to read. ## Integrity Every git object is named by its hash, and `index-pack` verifies every pack it reads. A PDS, or a `knotless-serve` in front of one, can withhold data but cannot substitute it. Ref targets come from the owner's signed atproto repo. `knotless` reads them with plain XRPC and does not check the commit signature itself; `com.atproto.sync.getRecord` returns the proof a client that wants to check it needs. A gateway can write the knotless records its OAuth grant allows to an owner's account; what it writes to a repository is still checked by every reader's `index-pack`. Paths in commits are refused if they are empty, contain `.`, `..` or `.git` segments, or contain NUL or newline characters. ## Prior art - **pds.js `@pdsjs/git`** stored a repository as one record holding every ref plus a chain of git bundles. Any two pushes to the repository contended on that one record. knotless splits refs into records with their own compare-and-swap, and replaces bundles with self-contained packs that arrive inline with the listing. - **Tangled** keeps git on separate servers ("knots") and the social records in the PDS. knotless keeps git in the PDS, and adds a server (`knotless-serve`) as an optional cache that anyone can run and nobody has to trust, and as a gateway that only the owners who delegate to it trust. - **Anthropic's Scion** ("Redesigning source control at Anthropic", October 2026) replaced git internally once agents pushed merges past 30,000 a day. Its lessons that carry over: clients are online, so fetch by immutable hash through tiers of caches (`knotless-serve`, the object API). Shared mutable state should be a compare-and-swap on one row (one record per ref). Refs are the scaling problem, so agents should work on their own branches. A working copy is optional (`knotless commit`). - **Cursor Origin** keeps git's wire protocol and builds the agent-facing forge around it. `knotless-serve` makes the same bet for reads: plain git clients, with no special client required.