Pages

A repository can have a website. The site is a set of static files, such as HTML, CSS, scripts and images, that knotless-serve answers over HTTP at an address of its own.

The difficulty is that anyone may publish a site, and the files live in the publisher's own atproto account on a stock PDS. A PDS is the server that holds an account's records and blobs. Serving what strangers publish raises four questions. How do you keep their pages away from knotless.dev's login and away from each other? How do you fit a site into the limits of a PDS? How do you make sure a visitor gets what the owner published, however many caches sit between? How do you let a site change without a visitor ever seeing half of one deploy and half of another?

Storage explains how a repository itself is kept, and serve the server that answers for it.

What a site is

A repository has at most one live site. Its files are blobs in the owner's account, and one record, dev.knotless.site, lists them. A record is a JSON document in a collection, addressed by a key. A blob is binary data stored beside records. A record references a blob by its CID, a content hash, and the PDS frees any blob that no record references.

knotless-serve only serves a site. It holds nothing that a rebuild from the PDS would lose.

A site is built the way a release is (see packages). Each file is a blob, and the signed record carries its size and sha256. A file larger than the PDS accepts as one blob is split into parts by the same code that splits packs. A part is at most KNOTLESS_CHUNK_SIZE bytes, 4,000,000 by default and never above the PDS's blobUploadLimit. Every blob is stored as application/octet-stream.

What shapes the design

A site must not run next to knotless.dev's login. A page that runs on a knotless.dev origin could act as the signed-in visitor.

Owners' sites must not reach each other. One owner's script must not read or set another owner's cookies or storage.

A PDS limits the size of a blob, and some PDSes limit how many blobs an account uploads in a day. A site has many files, and it is deployed again and again.

A visitor reads through caches, such as the browser's, a proxy's and serve's own. What arrives must be what the owner published.

A site changes, and a deploy must not leave a mix of old and new files.

The record

One dev.knotless.site record exists per repository, in the owner's account. Its record key is the repository's key, a TID, which is a time-ordered key. Each deploy replaces the record with putRecord or applyWrites.

Releases do the opposite. A release writes a record for every version, because someone may install an old version again. A site is meant to change, and nobody asks for last week's. Replacing the one record has these effects.

• The PDS frees the old deploy's blobs once nothing references them, so old deploys cost no storage.
• Serve reads the live site with one getRecord by key. It never lists.
• The swap is one write. A visitor sees the old site or the new one, never a mix.

The cost is that a site has no deploy history. To roll back, deploy an older commit's build again. The record names the commit it was built from and the check run that built it, so what is live is always known. Deleting the record takes the site down.

Details

• files lists each file with path, size, sha256, an optional contentType, and parts, the blobs that hold it. An empty file has no parts.
• notFound is optional. It is the path of a page served with status 404 for a path no file has.
• fallback is optional. It is the path of a page served with status 200 for a page no file has, for single-page apps. For a page, it is taken before notFound.
• commit and check are optional.
• createdAt is the time of the deploy.
• A site has at most 2,000 files.
• A path is relative and /-separated. It has no leading /, no . or .. segment, and at most 512 bytes. Paths are distinct within a site.
• contentType is what serve sends as Content-Type. Publishing picks it from the file's extension, and serve does the same for a file without one. Because the blob is application/octet-stream, the PDS's own guess at its type never matters, and the PDS never serves the file as a page from its own origin.

Publishing a site

The knotless repo site command deploys a directory as a repository's site. It runs as the owner, logged in as that account. In CI it runs as releases do, with an app password that the repository's secrets give the step. The flags are in the CLI reference.

The command works in this order.

1. It walks the directory and checks every path and size.
2. It opens the repository. A private one's site is kept in its space (see private sites).
3. It reads the current dev.knotless.site record.
4. It uploads only the files whose sha256 the current record does not already list. For the files it does list, it reuses their blob references.
5. It writes the new record. For a private repository, it then shows the appview the space, as any write there does, since nothing written in a space reaches the firehose.

Reuse matters because uploads are scarce. bsky.social, for one, allows 1,000 blob uploads a day. A site uploaded whole on every deploy would run out. A build that names its assets by hash changes few files from one deploy to the next, so most deploys upload little.

Inside a runner step, commit defaults to KNOTLESS_RUN_SHA, as it does for releases.

Deploying on a push

A site deploys on every push to a branch by a workflow step that runs knotless repo site on push to that branch. The site shared step makes this one line (see reusable workflows). The sites guide shows the workflow, and the workflow reference lists the keys.

There is no mode that serves a branch's tree directly, so there is one way to store and serve a site. The reasons are these.

• Most sites are built. Serving the tree would commit the build's output to the repository's history for good.
• The record names exactly which bytes are live and what built them.
• Serving a blob by its hash is simpler than reading a tree out of packs on every page view. Nothing has to watch a ref to rebuild a map of files.

Where a site is served

Sites are served on knotless.page, a registrable domain of its own, and never under knotless.dev. Browsers treat every *.knotless.dev name as the same site as knotless.dev. A page published there could set cookies that knotless.dev then receives. It could also make form posts that SameSite cookies do not stop, because those cookies block only requests from other sites.

knotless.page is to be on the Public Suffix List. This is the list browsers use to decide where one site ends and the next begins, for cookies and storage. With the domain on it, each <name>.knotless.page is a site of its own, and one owner's pages cannot read or set another's cookies. Until the list takes it, a page can set a cookie for all of knotless.page, which another owner's pages then receive. Nothing there signs anyone in, so such a cookie reaches only what the sites themselves keep in cookies.

Each owner has a subdomain, and each repository's site is a path under it.

https://<owner>.knotless.page/<repo>/

All of one owner's sites share an origin, which is acceptable because they share an owner. Different owners never share one.

Names

From a handle to a label

An account has a DID, its permanent ID such as did:plc:…, and a handle, a changeable name such as alice.example.com that resolves to the DID. The owner's subdomain is their handle written as one DNS label. Each - becomes --, and then each . becomes -.

Handle	Label
chadtmiller.com	chadtmiller-com
alice.bsky.social	alice-bsky-social
a-b.c	a--b-c
a.b-c	a-b--c

No two handles give the same label. Serve reads the handle back from the label, where a pair of - is a - and a single - is a .. It then resolves the handle to a DID as it resolves any handle, and checks that the DID's document names the handle back.

The label is a single one for a reason. A handle has any number of dots, and a dot in a hostname starts a new level. The Public Suffix List can only mark a boundary a fixed number of labels deep. With chadtmiller.com.knotless.page, the boundary would make com.knotless.page the site that every .com handle shared. A wildcard certificate, *.knotless.page, also covers one level only.

The address uses the handle, not a name people claim, so nothing has to be handed out or arbitrated, and no one must claim a name before a site works. The cost is that the address follows the handle and moves when the handle does. After a change, the old label resolves to whoever holds that handle now, if anyone. A site's address is always its owner's current handle.

When the handle does not fit

A handle has no subdomain if its label would be over 63 bytes, or if it would begin xn--, which browsers read as an internationalized name. Every did:plc account also has a subdomain named by its DID, so did:plc:<id> is served at <id>.knotless.page. A plc id has no -, and every handle label has one because a handle has a dot, so the two forms never collide. The DID name never changes.

Renames

A repository rename moves the site to the new path, because the record follows the repository's key. The old path does not redirect yet. It is to redirect temporarily, as repository pages do (see identity), once the index keeps a rename's redirect record, which a site's lookup reads rather than the owner's PDS.

The address https://<owner>.knotless.page/, with no repository, redirects to the owner's page on knotless.dev.

How paths are answered

Serve answers a request from the record's file map.

• /<repo> redirects to /<repo>/.
• /<repo>/x/ serves the file x/index.html.
• /<repo>/x, where no file x exists but x/index.html does, redirects to /<repo>/x/.
• A page no file has gets the fallback page with status 200 if the record names one. A request is for a page when it asks for text/html in its Accept header, or its path's last segment has no extension.
• Anything else no file has gets the notFound page with 404, or else a plain 404.

A single-page app uses fallback, so that its router can answer paths the server has no file for. Only pages fall back. A script or image that is missing still gets a 404, so a broken build shows as one rather than as the app's HTML where a script should be.

Because the site lives under /<repo>/, its build must use that as its base path. In Vite that is base, and in SvelteKit it is paths.base.

Responses carry X-Content-Type-Options: nosniff, so a browser trusts the record's contentType. They carry an ETag of the file's sha256. They also carry a short Cache-Control max-age. The max-age is short because a site changes and its paths are not content-addressed. A revisit revalidates by ETag.

How serve keeps a site

Serve indexes records from Jetstream, the public stream of every account's record changes, as it does for releases. A new deploy is live within seconds. The index row holds the record.

From the record, serve builds a map of path to file. It keeps the map until the indexed record changes, so a redeploy needs no TTL and no invalidation.

Serve reads each file from the owner's PDS the first time someone asks for it. It joins the parts, checks the result against the record's size and sha256, and keeps it by its hash for good, as it does for release files. A file whose bytes differ from the record fails rather than being served. What a visitor gets is therefore what the owner's account signed, whichever cache it came through.

Details

• Files are fetched, checked and kept as release files are, by the code behind Records::fetch in crates/knotless-serve/src/releases.rs.
• The PDS signs the account's repo, so the record's hashes carry the owner's signature.

What is not served

A private repository's site is not served on its owner's subdomain. Private sites says where it is served instead.

A taken-down account or repository is not served, in the same way its pages are not. Moderation is described in security.

Only GET and HEAD are answered. Anything else gets 405. Serve sets no cookies on knotless.page, and it answers nothing else there, so there is no XRPC and no login on that domain.

Custom domains are not served yet.

Private sites

A private repository's site is read only by its members, as the repository is (see access). A shared password would name no one, and could not be taken back from one person, so membership is the gate.

The site's record and blobs are kept in the repository's space, as its packs are. Serve's index of the space keeps the record, as it keeps the space's other records, and serve reads the blobs as the visitor, with that member's credential, as it reads a private repository for its pages.

A private site cannot share its owner's origin. Any script there, such as one on the owner's public site, could fetch the private site with the visitor's cookie, since a cookie the page's scripts cannot read is still sent with their requests. So each private site has a host of its own under private.knotless.page, named by the repository's key and its owner's plc id, so that a rename does not move it:

https://<repo key>-<plc id>.private.knotless.page/

The site is served at the host's root. An owner whose DID is not a did:plc has no such host, and publishing says so.

knotless.page has no login, so the visitor proves who they are to knotless.dev instead.

sequenceDiagram
  participant B as Browser
  participant S as Private site's host
  participant D as knotless.dev
  participant P as Owner's PDS
  B->>S: a page, with no pass
  S-->>B: redirect to knotless.dev for a ticket
  B->>D: ask for a ticket, logged in
  D->>P: a credential to the space, as the visitor
  P-->>D: given only to a member
  D-->>B: redirect back with a ticket
  B->>S: the ticket
  S-->>B: a pass, as a cookie of the host's own
  B->>S: the page, with the pass

1. A request with no pass for the host redirects to knotless.dev's /oauth/site, naming the host and the page asked for.
2. There, a visitor who is not logged in is sent to log in, and back.
3. knotless.dev proves the visitor a member, as it does to show them the repository, and redirects back to the host's /.knotless/enter with a ticket. The ticket is signed by serve, names the visitor's DID and that one host, lasts a minute, and is taken once.
4. The host trades the ticket for a pass, a cookie of its own, and goes on to the page first asked for.

Each request after is checked against the pass and against membership. Membership is asked again as the pages ask it, a minute after it was last proven, so someone removed from the repository stops reading its site within that minute. A pass signed before its visitor logged out of knotless.dev no longer holds.

Details

• The cookie is __Host-knotless_site, HttpOnly, Secure, signed, and lasts eight hours.
• Responses are Cache-Control: private, no-store, so no shared cache keeps them.
• Serve keeps a private site's files apart from every other site's, so a record elsewhere that names the same hash is never served them.
• Paths are answered as a public site's are, with the host's root in place of /<repo>/.

Certificates and DNS

knotless.page and *.knotless.page point at the host that serve runs on. One wildcard certificate, issued by a DNS challenge, covers every owner's subdomain. The operations detail is in deploy/README.md.