chadtmiller.com/knotless
git that lives in your Atmosphere account
1
10
A README that starts from where a repository lives
README.md+170−125
| @@ -1,155 +1,200 @@ | ||
# knotless | ||
| ||
Git that lives in your atproto account. | ||
knotless keeps a git repository in an atproto account, the kind of account | ||
Bluesky runs on. You push and clone with plain git, and pulls, issues and | ||
reviews are records in the accounts of the people who wrote them. | ||
| ||
It works on any stock PDS, with nothing to install. People push with plain | ||
git through a remote helper, and agents do the same over MCP. | ||
On a forge, a repository lives on the forge's servers. So do its pulls and | ||
issues, and the name you sign them with. They leave only as exports. An | ||
[atproto](https://atproto.com) account works the other way round: its data | ||
sits on a server its owner chooses, a PDS, under a handle such as | ||
`alice.bsky.social` or a domain they own, and the account can move to | ||
another host with everything in it. knotless puts the repository there. It | ||
adds no git server, and installs nothing on the PDS. | ||
| ||
A knotless repository is a handful of records in your own atproto repo. | ||
There is no git server and no knot. The repository moves with your account | ||
and is in your CAR backups, and anyone can clone it from your PDS directly. | ||
Tangled, an earlier git forge on atproto, keeps git on separate servers it | ||
calls knots. knotless keeps git in the account itself, so there is no knot. | ||
| ||
Forks, pulls, issues and reviews are records too, each in its author's | ||
account. One optional server, `knotless-serve`, adds a read cache, an index | ||
of pulls and issues, a gateway for collaborators, and web pages. It is cheap | ||
to run. | ||
## Try it | ||
| ||
```sh | ||
curl -fsSL https://knotless.dev/install.sh | sh # knotless, git-remote-knotless, knotless-mcp, and the runner, labeller and reviewer | ||
knotless login alice.bsky.social # OAuth, in your browser | ||
You need git and an atproto account. A Bluesky account is one. | ||
| ||
```sh | ||
curl -fsSL https://knotless.dev/install.sh | sh | ||
knotless login alice.bsky.social | ||
git push knotless://alice.bsky.social/my-project main | ||
git clone knotless://alice.bsky.social/my-project | ||
``` | ||
| ||
[knotless.dev](https://knotless.dev) runs `knotless-serve`, and hosts | ||
knotless's own repository: | ||
[chadtmiller.com/knotless](https://knotless.dev/chadtmiller.com/knotless). | ||
| ||
## What it does | ||
| ||
- `git push`, `fetch` and `clone` over `knotless://` URLs. `knotless-serve` | ||
also serves reads over git smart HTTP, partial and shallow clones | ||
included. | ||
- Commits, branches, reviews and merges from the CLI or MCP, with no clone. | ||
- Forks, pulls, reviews, issues, protected branches and a merge queue. | ||
Collaborators push through a gateway the owner chooses. | ||
- Private repositories in permissioned spaces, where the pulls, issues and | ||
comments are as private as the code. | ||
- An MCP server for agents: over stdio on your machine, or over HTTP from | ||
`knotless-serve`. | ||
- CI: `knotless-run` runs workflows in containers on pushes and pulls, and | ||
writes checks that gate merges. | ||
- Read-only web pages for code, history, diffs, pulls, issues and checks, | ||
with syntax highlighting and Mermaid diagrams, on a phone or a desktop. | ||
- atproto's moderation: hides, blocks, reports, and takedowns published as | ||
labels. | ||
| ||
## How it works | ||
The script installs the client on macOS and Linux, arm64 and x86_64. | ||
`knotless login` opens your PDS in a browser, where you approve access to | ||
the `dev.knotless.*` records and blobs knotless writes. The first push to a | ||
`knotless://` URL creates the repository. | ||
| ||
Anyone with the client can clone a public repository straight from your | ||
PDS, without logging in. Plain git with nothing installed can clone it too, | ||
read only, from a knotless server: | ||
`git clone https://knotless.dev/alice.bsky.social/my-project`. | ||
[Getting started](docs/guide/getting-started.md) goes on from here. | ||
| ||
## The rest of a forge | ||
| ||
Collaborators cannot write to your account, so they push through a | ||
[gateway](docs/design/gateway.md): a server you allow, once, to write your | ||
repository's records for the accounts you name. You approve it with | ||
`knotless account delegate` and name them with `knotless repo share`. The | ||
gateway then holds a session for your account, so use one you trust or run | ||
your own. [Collaborating](docs/guide/collaborating.md) has the steps. | ||
| ||
Pulls, issues, comments and reviews live in their authors' accounts, so | ||
something has to gather them. That is an appview, a service that indexes | ||
records across accounts. `knotless-serve` is one, and it draws the web | ||
pages too, where people log in through their own PDS to comment, review and | ||
merge, on a desktop or a phone. An [iOS app](ios/README.md) reads the same | ||
appview. Branches can require reviews, code owners and a | ||
[merge queue](docs/design/merge-queue.md), and owners hide comments and | ||
block accounts on their own repositories. | ||
| ||
No forge runs your CI either, so a repository names its runners. | ||
`knotless-run` runs [workflows](docs/guide/workflows.md), YAML files under | ||
`.knotless/workflows/`, on pushes and pulls, in containers by default, and | ||
files each result as a check record in the runner's own account. Pages and | ||
merges count checks only from the runners a repository names. | ||
| ||
A [private repository](docs/guide/private.md) lives in a space, a record | ||
store only its members read, with its pulls and issues beside it. Spaces | ||
are in the reference PDS's permissioned-data alpha; on a PDS without them, | ||
every repository is public. `knotless release publish` publishes | ||
[package versions](docs/design/packages.md), and `knotless repo site` | ||
deploys a [static site](docs/guide/sites.md). | ||
| ||
## For agents | ||
| ||
An agent works as your account, and needs no checkout. `knotless repo | ||
commit` commits files to a branch directly, making the branch if it has to, | ||
and `knotless-mcp` offers about 130 MCP tools, one per CLI command. | ||
| ||
```sh | ||
knotless repo commit knotless://alice.bsky.social/my-project \ | ||
-b fix-typo -m "Fix a typo" --put README.md ./README.md | ||
knotless pull create knotless://alice.bsky.social/my-project main \ | ||
knotless://alice.bsky.social/my-project fix-typo "Fix a typo" | ||
claude mcp add knotless -- knotless-mcp | ||
``` | ||
| ||
An assistant that connects to a server instead, such as claude.ai, takes | ||
`https://knotless.dev/mcp` as a connector, and you sign in at your own PDS | ||
with nothing installed. [Agents](docs/guide/agents.md) covers Claude Code, | ||
Claude Desktop, Cursor and claude.ai. | ||
| ||
Pushes to separate branches never contend, so many agents can work in one | ||
repository at once, a branch each. A repository can name agent accounts, | ||
whose pulls its runners check with no secrets. `knotless-agent` | ||
[reviews](docs/guide/reviews.md) the pulls its account is asked to. | ||
| ||
## What has to run | ||
| ||
```mermaid | ||
flowchart LR | ||
you[git, knotless CLI,<br/>or an agent over MCP] -->|XRPC| pds[(Your PDS:<br/>dev.knotless.* records<br/>and pack blobs)] | ||
others[Collaborators] -->|service token| gw[Gateway] | ||
you[You: git, the CLI,<br/>or an agent] -->|write| pds[(Your PDS:<br/>the repository)] | ||
them[Collaborators] -->|push| gw[Gateway] | ||
gw -->|writes as you| pds | ||
pds --> feed[Jetstream] | ||
feed --> serve[knotless-serve:<br/>cache, index, web pages] | ||
feed --> run[knotless-run:<br/>workflows] | ||
run -->|checks| ci[(Runner's PDS)] | ||
readers[Anyone] -->|clone, browse| serve | ||
readers -->|clone| pds | ||
them -->|pulls, issues, reviews| theirs[(Their PDSes)] | ||
run[knotless-run] -->|checks| runs[(Runner's PDS)] | ||
pds --> serve[knotless-serve:<br/>index, cache, pages] | ||
theirs --> serve | ||
runs --> serve | ||
anyone[Anyone] -->|browse, clone| serve | ||
anyone -->|clone| pds | ||
``` | ||
| ||
Four record types in the owner's account hold a repository: | ||
| ||
| Record | Key | Holds | | ||
| Piece | What it adds | Who runs it | | ||
| --- | --- | --- | | ||
| `dev.knotless.repo` | name | the repository's `key` (a TID), description, default branch, collaborators, gateway and branch rules | | ||
| `dev.knotless.ref` | `<key>.<hash of ref name>` | one ref's target, and a generation counter | | ||
| `dev.knotless.claim` | `<key>.<hash>.<generation>` | the lock on one generation of a ref | | ||
| `dev.knotless.pack` | `<key>.<tid>` | an immutable, self-contained git pack, inline when small and in blobs when not | | ||
| A PDS | Holds the repository. It is the authority on where a ref points. | Your account's host, already | | ||
| [`knotless`](crates/knotless) | The `knotless` CLI and `git-remote-knotless`, which git runs for `knotless://` URLs | Each person who pushes | | ||
| [`knotless-mcp`](crates/knotless-mcp) | The CLI's commands as MCP tools | Whoever runs an agent | | ||
| [`knotless-serve`](crates/knotless-serve) | Appview, read cache, HTTPS clones, gateway, and the pages in [`web`](web) | knotless.dev, or you | | ||
| [`knotless-run`](crates/knotless-run) | CI and the merge queue | Whoever a repository names as a runner | | ||
| [`knotless-triage`](crates/knotless-triage) | Labels on issues, chosen by a classifier | Whoever a repository names as a triager | | ||
| [`knotless-agent`](crates/knotless-agent) | Reviews of pulls, written by a command such as Claude Code | Whoever owns the account asked to review | | ||
| ||
The first two rows are enough to push, fetch and clone. The rest is for | ||
working with other people, and anyone can run it. | ||
[knotless.dev](https://knotless.dev) runs `knotless-serve` for anyone to | ||
use, and hosts | ||
[knotless's own repository](https://knotless.dev/chadtmiller.com/knotless). | ||
Your own is one binary whose state is a directory of git mirrors and one | ||
SQLite file, most of it cache that can be thrown away and read again. See | ||
[Running knotless-serve](docs/guide/server.md) and | ||
[how knotless.dev runs it](deploy/README.md). | ||
| ||
## How a PDS holds a repository | ||
| ||
A PDS stores records, which are JSON documents addressed by a key, and | ||
blobs. A repository is a few kinds of record: one for the repository, one | ||
per ref with a generation counter, and git packs, inline when small and in | ||
blobs when not. | ||
| ||
A PDS has no locks, but it refuses to create a key that exists. A push | ||
writes its pack, a claim record keyed by the ref's name and the generation | ||
it read, and the moved ref, in one all-or-nothing call. Of two pushes that | ||
read the same generation, the second is refused and git says `fetch first`. | ||
Each ref has that compare-and-swap to itself, which is why separate | ||
branches never contend. | ||
| ||
Every push adds a pack. Once a repository holds 32, the owner's next | ||
`git push` folds the older ones into one in the background, so a clone | ||
stays a few reads. [Storage](docs/design/storage.md) and the | ||
[design overview](docs/design/README.md) have the rest. | ||
| ||
A push creates the claim for the ref generation it read, in the same | ||
`applyWrites` as the ref update. The PDS accepts only the first create of a | ||
key, so each ref has its own compare-and-swap, and agents pushing to | ||
separate branches never contend. | ||
## Performance | ||
| ||
The keys share the repository's prefix, so one `listRecords` reads a | ||
repository's refs, or its newest packs. Once a repository holds more than 32 | ||
packs, they are folded into one, so a clone stays a few reads however long | ||
the history is. | ||
What an operation costs is mostly the round trips it waits for: two to | ||
connect to the PDS, then one for each request it makes in turn. Medians | ||
against the reference PDS on the same machine, reached over HTTPS through a | ||
proxy adding 50 ms to every round trip: | ||
| ||
The [design pages](docs/design/README.md) cover the rest: the limits of a | ||
stock PDS, the failure cases, the trust model and the prior art. | ||
| Operation | Requests | Time | | ||
| --- | --- | --- | | ||
| `git push`, one small commit | 2 | 296 ms | | ||
| `git fetch`, one new commit | 1 | 259 ms | | ||
| `git clone`, compacted repository | 1 | 263 ms | | ||
| `knotless repo commit`, no checkout | 2 | 435 ms | | ||
| 16 agents pushing 16 branches at once | 2 each | 561 ms in total | | ||
| ||
## Performance | ||
With an appview, as the installer sets one, a push of a branch also asks it | ||
which pulls the push landed, and takes 507 ms. | ||
| ||
A warm push is two round trips to the PDS, and a fetch of a small push is | ||
one. Against the reference PDS on the same machine, with 50 ms of | ||
round-trip time added: | ||
| ||
| | p50 | | ||
| --- | --- | | ||
| `git push`, one small commit | 223 ms | | ||
| `git fetch`, one new commit | 177 ms | | ||
| `knotless repo commit`, no checkout | 366 ms | | ||
| `git clone`, compacted | 158 ms | | ||
| 16 agents pushing 16 branches at once | 443 ms total | | ||
| ||
[Performance](docs/reference/performance.md) has every measurement and how | ||
it was taken. | ||
| ||
## Documentation | ||
| ||
- [Getting started](docs/guide/getting-started.md): install, log in, push, | ||
commit without a checkout. | ||
- [Collaborating](docs/guide/collaborating.md): sharing, forks, pulls, | ||
reviews, merging, moderation. | ||
- [Private repositories](docs/guide/private.md). | ||
- [Agents](docs/guide/agents.md): connect Claude Code, Claude Desktop, | ||
Cursor or claude.ai. | ||
- [Running knotless-serve](docs/guide/server.md): the cache, appview, | ||
gateway and web pages. | ||
- [Workflows](docs/guide/workflows.md): CI with `knotless-run`. | ||
- [Command reference](docs/reference/cli.md): every command, flag and | ||
environment variable. | ||
| ||
The [docs index](docs/README.md) lists every page. | ||
| ||
## Components | ||
| ||
- [`knotless`](crates/knotless): the CLI and `git-remote-knotless`. They | ||
talk to PDSes directly. | ||
- [`knotless-mcp`](crates/knotless-mcp): the MCP server. | ||
- [`knotless-serve`](crates/knotless-serve): the optional read cache, | ||
appview, gateway, moderation service and web pages, in one binary that | ||
fits a small VPS. | ||
- [`knotless-run`](crates/knotless-run): the workflow runner. | ||
- [`knotless-triage`](crates/knotless-triage): the triage bot, which labels | ||
issues with a classifier's choices. | ||
- [`web`](web): the SvelteKit pages that `knotless-serve` hands requests to. | ||
| ||
## Development | ||
A repository's size does not change these. With no latency added, | ||
nodejs/node, 49,240 commits and 1.53 GB on the PDS, pushes one small commit | ||
in 0.2 s, the same as a new repository. Its cold clone takes 51 s. | ||
[Performance](docs/reference/performance.md) says how each was measured, | ||
what the measurement leaves out, and the times at other latencies. | ||
| ||
```sh | ||
just setup # npm install in harness/ and web/, llvm-tools for coverage | ||
just test # every test, against the npm reference PDS | ||
just fmt # rustfmt and oxfmt | ||
just lint # rustfmt, oxfmt, clippy and svelte-check | ||
``` | ||
## Status | ||
| ||
Tests cover every line and function, and CI fails below that. | ||
[Development](docs/development.md) covers the test harness, screenshots and | ||
agent scenarios. [AGENTS.md](AGENTS.md) says how changes are made and | ||
landed. | ||
knotless is not yet at 1.0, and hosts its own development. Public | ||
repositories use only standard PDS methods. Private ones depend on the | ||
permissioned-data alpha. Git objects are checked by hash, but where a ref | ||
points is read from the owner's PDS and is not yet verified against the | ||
account's signed commits. [Known gaps](docs/design/security.md#known-gaps) | ||
lists the rest. | ||
| ||
## Status | ||
## Documentation and development | ||
| ||
The [docs index](docs/README.md) lists the guides, the reference and the | ||
design pages, and [knotless.dev/docs](https://knotless.dev/docs) publishes | ||
them. [Actions](docs/reference/actions.md) lines up everything you can do | ||
with its command, MCP tool and page. The record schemas are in | ||
[`lexicons/dev/knotless`](lexicons/dev/knotless). | ||
| ||
knotless runs [knotless.dev](https://knotless.dev), which hosts knotless's | ||
own development. Not done yet: verifying refs against signed repository | ||
commits ([Known gaps](docs/design/security.md#known-gaps)). | ||
`just setup` installs what the tests need, `just test` runs them against | ||
the npm reference PDS, and `just lint` checks formatting and runs clippy | ||
and the pages' checks. Tests cover every line and function of the Rust | ||
code, and CI fails below that. [Development](docs/development.md) and | ||
[`AGENTS.md`](AGENTS.md) say how changes are made and checked. | ||
| ||
## License | ||
| ||
MIT or Apache-2.0, at your option: [LICENSE-MIT](LICENSE-MIT), | ||
[LICENSE-APACHE](LICENSE-APACHE). | ||
MIT or Apache-2.0, at your option: [`LICENSE-MIT`](LICENSE-MIT), | ||
[`LICENSE-APACHE`](LICENSE-APACHE). | ||