chadtmiller.com/knotless
Git hosted in your atproto account, on any stock PDS, fast enough for agents.
git clone knotless://chadtmiller.com/knotlessneeds the knotless git helper. set it up
# knotless
Git hosted in your atproto account, on any stock PDS, fast enough for agents.
A repository is a handful of records in your own repo: no git server, no
knot, nothing installed on the PDS. It moves with your account, rides in
your CAR backups, and anyone can clone it from your PDS directly. Private
repositories live in a permissioned space; collaborators push through a
gateway you delegate to; forks, pulls, issues and reviews are records in
each participant's own account. Agents work through MCP, people read through
a small web interface, and both are served by one binary that is cheap to
run.
```sh
cargo install --path crates/knotless # git-remote-knotless + knotless
knotless login alice.bsky.social # OAuth, in your browser
git push knotless://alice.bsky.social/my-project main
git clone knotless://alice.bsky.social/my-project
```
## Why it is fast
Every operation is planned around round trips to the PDS, because on a real
network those are what an agent waits for.
| Operation | Round trips (warm) |
| --- | --- |
| `git push` | 2: read the refs and newest packs (three reads in parallel), then one `applyWrites` |
| `git fetch` | 1 for pushes under 192 KiB, whose packs arrive inside the listing; plus parallel blob reads for larger ones |
| `knotless commit` (no checkout) | 2, plus 3 per lost race |
| Ref compare-and-swap | 0 extra: a claim record in the same `applyWrites` |
The identity, session and repository key are cached on disk after the first
call, and every request in a process shares one connection.
Measured with `cargo run --release --example bench -- --rtt 50`: the npm
reference PDS (`@atproto/pds` 0.5.43, SQLite) on the same machine, behind a
proxy adding 50 ms of round-trip time, on an Apple-silicon laptop.
| | p50 | p95 |
| --- | --- | --- |
| `git push`, one small commit | 223 ms | 276 ms |
| `git fetch`, one new commit | 177 ms | 211 ms |
| `knotless commit`, no checkout | 366 ms | 596 ms |
| `git clone`, 21 packs | 240 ms | 253 ms |
| `git clone`, compacted | 158 ms | 172 ms |
| `git clone --depth 1` via `knotless-serve` | 208 ms | 299 ms |
| One file via the `knotless-serve` API, warm | 45 ms | 57 ms |
| 16 agents pushing 16 branches at once | 443 ms total | |
| 16 agents committing to one branch at once | 10.9 s total, 97 pushes | |
A branch is one compare-and-swap, so direct writers to one branch land one
per round, and the reference PDS serializes an account's writes. Give each
agent its own branch and merge, or send commits through a gateway, which
queues them per branch and lands each queue as one push.
What the web pages show is read from a mirror's objects in process, in
about a millisecond warm; a highlighted 40 KB source file takes a few more.
The pages themselves take a few milliseconds more to draw. A page's social
card is drawn in about 5 ms the first time, about 30 KB, and served from
memory after that (`just bench-cards`).
## How it works
Four record types hold a repository (schemas in
[`lexicons/`](lexicons/dev/knotless)):
- **`dev.knotless.repo`**, keyed by name: the repository's `key` (a TID),
description, default branch, collaborators, the gateway they push
through, and the repository it was forked from.
- **`dev.knotless.ref`**, keyed `<key>.<hash of ref name>`: one per ref,
with its target and a generation counter.
- **`dev.knotless.claim`**, keyed `<key>.<hash>.<generation>`: the lock.
A push creates the claim for the generation it read, in the same
`applyWrites` as the ref update. Two pushes from the same generation both
create the same key, and the PDS accepts only the first, so every ref has
its own compare-and-swap, with no `swapCommit` contention against the rest
of the account. The claims double as the reflog.
- **`dev.knotless.pack`**, keyed `<key>.<tid>`: an immutable,
self-contained git pack, inline as bytes when small and split over blobs
when not.
Because the keys share a prefix, a `listRecords` from the cursor `<key>/`
walks one repository's records newest first and stops at the first key
outside it. A fetch reads the newest page of packs alongside the refs and
downloads only the packs whose tips it lacks.
Compaction folds a repository's packs into one once it holds more than 32,
so a clone stays a few reads however long the history. It runs detached
after a push, and keeps packs younger than ten minutes so a reader halfway
through a walk still finds them.
Four more hold the conversation around it, each in its author's account and
naming the repository by its `at://` URI: **`dev.knotless.pull`**,
**`dev.knotless.issue`**, **`dev.knotless.comment`** (a review, when it
carries a verdict and the head it saw) and **`dev.knotless.state`** (open,
closed or merged). An item's state is the latest one set by someone allowed
to: its author, the owner, or a collaborator, and only the latter two merge.
[`docs/design.md`](docs/design.md) has the details: the failure cases, the
limits of a stock PDS, and the prior art.
## Private repositories
```sh
knotless create knotless://alice.test/secret --private
git push knotless://alice.test/secret main
knotless share knotless://alice.test/secret bob.test # bob may clone
knotless members knotless://alice.test/secret
```
A private repository is a permissioned space,
`at://<did>/space/dev.knotless.space/<name>`, holding the same records. Its
members read with a space credential, which needs an OAuth login rather
than an app password, and which is cached until shortly before it expires. Each
member writes their issues and comments into their own repo in the space,
so the conversation stays as private as the code. Spaces are in the
reference PDS's permissioned-data alpha; on a PDS without them, every
repository is public.
A public `knotless-serve` never shows a private repository. To browse one,
run your own as whoever is logged in; it listens on this machine only, and
your own profile lists your private repositories:
```sh
knotless-serve --private # then http://127.0.0.1:2590/alice.test/secret
```
## Collaborating
Only an account can write to its own repo, so collaborators push through a
gateway: a `knotless-serve` the owner has delegated to.
```sh
knotless delegate alice.test did:web:git.example.com # once, as the owner
knotless share knotless://alice.test/proj bob.test --write
# then, as bob, plain git:
git push knotless://alice.test/proj feature
```
A collaborator's helper signs each request with a service token from their
own PDS, and the gateway checks it against the repository's collaborators
before writing as the owner. Commits sent to one branch at once are queued
and landed together as one push, rebuilt on the new tip if the branch moved
underneath.
Anyone can fork and offer changes back:
```sh
knotless fork knotless://alice.test/proj
git push knotless://bob.test/proj topic
knotless pull knotless://alice.test/proj main knotless://bob.test/proj topic "Fix the parser" -m "Details."
knotless comment knotless://alice.test/proj <pull at-uri> "Looks right" --approve
knotless merge knotless://alice.test/proj <pull at-uri> # fast-forward, or a merge commit
```
Pulls and issues on public repositories are listed by an appview
(`KNOTLESS_APPVIEW`), which indexes them from Jetstream or the firehose; on private ones,
from the members' repos in the space. A review reads the pull from its
author's repo directly, so it never waits for an index.
Moderation is atproto's own. An owner hides what is said about their
repository and blocks accounts from it or from all of theirs; anyone reports to the
appview's moderation service, whose operator takes accounts and records
down, and other appviews can follow its takedowns as labels:
```sh
knotless hide knotless://alice.test/proj <issue at-uri>
knotless block troll.test knotless://alice.test/proj
knotless report <issue at-uri> spam -m "the same link in every issue"
KNOTLESS_ADMIN_PASSWORD=… knotless admin reports
KNOTLESS_ADMIN_PASSWORD=… knotless admin takedown troll.test
```
## For agents
`knotless-mcp` is an MCP server over stdio. Logged in as the account it acts
for, an agent reads and changes repositories with no clone and no checkout.
To connect it:
```sh
cargo install --path crates/knotless-mcp
knotless login you.bsky.social # the account the agent acts as
claude mcp add knotless -- knotless-mcp # Claude Code
```
Other clients, such as Claude Desktop (`claude_desktop_config.json`) and
Cursor (`.cursor/mcp.json`), take the same command in their MCP settings:
```json
{ "mcpServers": { "knotless": { "command": "knotless-mcp" } } }
```
It acts as the account `knotless login` last stored. To pin another, set
`KNOTLESS_IDENTIFIER` in the server's `env`; set `KNOTLESS_APPVIEW` to a
`knotless-serve` index for pulls and issues on public repositories.
| Tools | |
| --- | --- |
| `list_repos`, `refs`, `list_tree`, `read_file`, `search`, `log`, `diff` | read, by `handle/name` |
| `commit` | change files on a branch, optionally from a base and through the gateway |
| `create_branch`, `delete_branch`, `rename_branch` | make, remove or rename a branch; a rename takes the open pulls on it along |
| `create_repo`, `rename_repo`, `share`, `members`, `delegate`, `transfer_repo`, `accept_transfer` | manage repositories and access; `delegate` hands a gateway (the appview, by default) your writes, so collaborators can push |
| `fork`, `open_pull`, `open_issue`, `list_items`, `show_item`, `pull_diff`, `comment`, `set_state`, `merge`, `retarget_pull` | collaborate; `show_item` on a pull says what it brings in and whether it merges cleanly |
| `hide_item`, `block_account`, `report` | moderate your repositories, report to the appview |
| `takedown`, `moderation_status`, `list_reports` | as the appview's operator (`KNOTLESS_ADMIN_PASSWORD`) |
The same is on the command line:
```sh
# Change files on a branch without a checkout. If the branch moved, the same
# changes are replayed onto the new tip and pushed again.
knotless commit knotless://alice.test/proj -b main -m "fix typo" \
--put README.md ./README.md --put-exec run.sh - --delete old.txt --json < run.sh
knotless refs knotless://alice.test/proj # one round trip
```
## The server
`knotless-serve` is one binary with four jobs, each optional:
```sh
knotless-serve --listen 0.0.0.0:2590 \
--jetstream wss://jetstream.us-east.bsky.network \ # keep mirrors warm; feed the index
--index /var/lib/knotless/index.db \ # appview: repositories, pulls, issues
--backfill https://bsky.network \ # index everything already there
--public-url https://git.example.com \ # be a gateway, as did:web:git.example.com
--labeler did:plc:… # honour a labeler's takedowns
--web http://127.0.0.1:2591 # the web pages, from web/
```
Events come from a Jetstream v2 host or, with `--firehose wss://…`, from a
relay or PDS directly. Jetstream is the cheaper of the two: it sends only
`dev.knotless.*` commits, plus identity and account changes, as small JSON
frames carrying each record, so the server neither downloads and decodes
every commit on the network nor fetches each record from its author's PDS.
Its frames are unsigned, so the index trusts the Jetstream operator for
what a record says; mirrors still sync from the PDS, and `--backfill`
rebuilds the index.
- **A read accelerator**, a cache and never a source of truth. It speaks git
smart HTTP (protocol v0 and v2, partial and shallow clones), read only,
and an object API:
```sh
git clone --filter=blob:none http://localhost:2590/alice.test/proj
curl 'localhost:2590/api/alice.test/proj/blob?rev=main&path=src/lib.rs'
curl 'localhost:2590/api/alice.test/proj/tree?rev=main&path=src'
curl 'localhost:2590/api/alice.test/proj/log?rev=main&path=src&limit=20'
curl 'localhost:2590/api/alice.test/proj/diff?base=main&head=feature'
curl 'localhost:2590/api/alice.test/proj/object/<sha>' # immutable, cacheable
```
- **A web interface**, read only: what knotless is and how to connect,
repositories and search, each account's profile and public repositories,
trees, files, history, commits, pulls and issues, laid out for a phone as well as a desktop. A repository's page is
at its clone URL. Commits show each file's diff with old and new line
numbers, syntax highlighting, and the changed part of each edited line.
Code is sent as plain text and highlighted with tree-sitter ranges applied
by the CSS Custom Highlight API; links load on hover; pages move with
view transitions; anything named by a commit hash is cached as immutable.
The front page, repositories, pulls and issues unfurl into a social card,
drawn in process. Writing is for git, the CLI and agents.
The pages are a SvelteKit app in [`web/`](web), run with Node, that
`--web` names: the server hands it every request it does not answer
itself, and it draws each page from the server's JSON under `/_data` (its
TypeScript types are written by `cargo test`) and, for the front page,
`dev.knotless.getFeed`. Without `--web`, the server has no pages.
```sh
just web # build web/build
just pages # serve it on :2591
just serve --index index.db --web http://127.0.0.1:2591
just web-dev # or redraw as the source changes
```
- **An appview**: `dev.knotless.listRepos`, `dev.knotless.getForge` and
`dev.knotless.getFeed`, from a SQLite index of what Jetstream or the firehose reports.
- **A gateway**: `dev.knotless.pushRefs`, `dev.knotless.createCommit` and
`dev.knotless.putDelegation` (the gateway is an OAuth client the owner
approves in a browser), for collaborators.
- **A moderation service**, as an appview with a public URL:
`com.atproto.moderation.createReport` for anyone,
`com.atproto.admin.updateSubjectStatus` and `dev.knotless.listReports`
for its operator (`KNOTLESS_ADMIN_PASSWORD`), and its takedowns as
labels on `com.atproto.label.subscribeLabels`.
State is a directory of bare mirrors and one SQLite file, so all four fit
on one small VPS.
## Commands
| | |
| --- | --- |
| `knotless login <handle> [--password-stdin]` / `logout <handle>` | store or drop a session |
| `knotless create <url> [--private] [--description] [--default-branch]` | create an empty repository |
| `knotless ls <handle>` | list an account's repositories |
| `knotless refs <url>` | print refs |
| `knotless commit <url> -b <branch> -m <msg> ...` | commit without a checkout |
| `knotless branch <url> <name> [<from>]` / `--delete` | make or remove a branch without a checkout |
| `knotless delegate <handle> <gateway did>` | let a gateway write for you |
| `knotless share <url> <handle> [--write]` / `unshare` / `members` | access |
| `knotless fork <url> [<name>]` | copy a repository into your account |
| `knotless pull <url> <base> <head url> <branch> <title> [-m <body>]` | open a pull |
| `knotless issue <url> <title> [-m <body>]` | open an issue |
| `knotless list <url> [--pulls \| --issues]` / `show <url> <at-uri>` | read them |
| `knotless comment <url> <at-uri> <text> [--approve \| --changes]` | comment or review |
| `knotless close` / `reopen` / `merge <url> <at-uri>` | change state |
| `knotless retarget <url> <at-uri> [--base <b>] [--head <b>]` | point a pull at other branches |
| `knotless hide` / `unhide <url> <at-uri>` | as owner, hide a pull, issue or comment |
| `knotless block` / `unblock <handle> [<url>]` | keep an account out of your repositories, or out of one |
| `knotless report <handle, url or at-uri> <reason> [-m <text>]` | report to the moderation service |
| `knotless admin takedown <subject> [--undo]` / `status` / `reports` | as the appview's operator |
| `knotless rename <url> <name>` | rename; the old name redirects |
| `knotless transfer <url> <handle or url>` / `--cancel`, `accept <url>` | move to another account |
| `knotless delete <url> --yes` | delete a repository and its history |
| `knotless compact <url>` | fold packs into one |
| `knotless mirror <url>` | sync the cache mirror and print its path |
Environment: `KNOTLESS_PASSWORD` (an app password, instead of OAuth),
`KNOTLESS_IDENTIFIER`, `KNOTLESS_BROWSER` (the command an OAuth login opens
its page with), `KNOTLESS_SERVICE`
(skip DID resolution), `KNOTLESS_PLC_URL`, `KNOTLESS_HANDLE_RESOLVER`,
`KNOTLESS_APPVIEW`, `KNOTLESS_MODERATOR` (the appview's),
`KNOTLESS_ADMIN_PASSWORD`, `KNOTLESS_DID_WEB_SCHEME` (`https`),
`KNOTLESS_INLINE_MAX` (192 KiB), `KNOTLESS_CHUNK_SIZE` (4 MB),
`KNOTLESS_COMPACT` (`background`, `inline`, `off`),
`KNOTLESS_COMPACT_THRESHOLD` (32), `KNOTLESS_COMPACT_GRACE_SECS` (600),
`KNOTLESS_CLAIM_GRACE_SECS` (86400),
`KNOTLESS_IDENTITY_TTL_SECS` (3600), `KNOTLESS_TIMEOUT_SECS` (60),
`KNOTLESS_CONFIG_DIR`, `KNOTLESS_CACHE_DIR`.
## Development
Tests run against the reference PDS from npm, started by
[`harness/pds.mjs`](harness/pds.mjs), and again against the permissioned-data
alpha build of it for private repositories. Every line and function is
covered across the two runs, and CI fails below that.
```sh
just setup # npm install in harness/ and web/, llvm-tools for coverage
just test # every test
just test-spaces # against the alpha: KNOTLESS_DEV_ENV names its dev-env
just cov # both runs side by side, failing under 100% lines or functions
just lint # rustfmt, clippy and svelte-check
just bench 50 # the table above, at 50 ms RTT
just bench-cards # what a social card costs to draw, with the PNGs
```
While working, run only what a change needs:
```sh
just pds # leave a reference PDS up for test binaries to share
just changed # files changed since tests last passed, and their tests
just t-changed # run those, and mark them passed
just t -p knotless-serve --test web # or any cargo test arguments
```
Each test binary otherwise starts a PDS of its own, which takes most of a
short run. On a shared one, tests make accounts as they need them.
The screenshots ([`harness/shots`](harness/shots)) take every page in
light and dark, at desktop and phone widths, with Playwright's Chromium.
The pages are drawn by the app in `web/` from what knotless-serve answered
for them, recorded in `harness/shots/fixtures.json`, so a run needs no PDS
or server. The social cards the pages name, drawn by knotless-serve and the
same in any browser, are taken once, from a live world (`just shots-live`):
```sh
just shots # retake the pages whose HTML changed, as the baselines
just shots profile --project=desktop-light # or only some
just shots-check # take every page, failing on any pixel that differs
```
The recording comes from a world: a reference PDS filled with the same
repositories, pulls and issues each time, with fixed commit dates and
record times. Record again after a change to what the server answers:
```sh
just shots-up # seed once, and keep it up for the runs below
just shots-record # record each page's answers from it
just shots-live # every page, through knotless-serve over the world
```
Each baseline keeps a digest of the HTML it was taken from, and a page
whose HTML still matches is skipped as unchanged before a browser opens
it. Without `shots-up`, a recording or live run seeds a world of its own.
Baselines are kept per platform, under `harness/shots/baseline/<platform>`;
DIDs, which the PLC directory makes new for each world, are masked.
Scenarios ([`harness/scenarios`](harness/scenarios)) put agents to a day's
git work through the MCP server. Each track is a few steps by different
people, such as a contributor's fix going through review. Each step is a
fresh Claude session that has only the knotless tools, logged in as that
person, on a fresh PDS and appview. Each agent reports what tripped it up,
and the run collects those reports with every failed call:
```sh
just scenarios # every track; uses the `claude` CLI and costs model usage
just scenarios contribution # or only some
```
Transcripts and `report.md` land in `target/scenarios/<time>/`.
`SCENARIO_MODEL` picks the model.
## Not yet
- Verifying refs against signed repository commits.
- The `dev.knotless` NSIDs assume control of `knotless.dev`.