~ knotless Search/
↑↓ move↵ open⇧↵ all results

chadtmiller.com/hud

A remote MCP server that shows atproto records as interactive UI inside MCP Apps hosts

0
0
153 lines · 16.1 KB Raw History
---
title: How it works
section: Reference
order: 301
summary: How HUD resolves identities and lexicons, serves blobs, joins, counts, queries and keeps sign-ins and canvases.
---

HUD reads records from each account's PDS and resolves identities and lexicons from the network. It keeps no database of its own, only caches and the files below. With `HUD_LOGS=on` it also keeps its log lines in a SQLite file for its operators ([observability](observability.md#kept-lines)).

## Identity

HUD finds a handle's DID through DNS (`_atproto.<handle>`, asked of a public resolver) or at `https://<handle>/.well-known/atproto-did`. It reads the DID's document from PLC or `did:web`, and the PDS from its `#atproto_pds` service. A handle counts only if the document names it back.

HUD keeps an identity for 1 hour. It keeps answers that there's none (no document, no DID) for 5 minutes. It doesn't keep a missing answer (none in time, or a server that can't answer now), so the next call asks again.

## Lexicons

HUD bundles the records, defs and queries of `com.atproto.*`, `app.bsky.*`, `social.grain.*`, `xyz.statusphere.*` and `fm.plyr.*` in `lexicons/`. `just lexicons` fetches them from their repositories. HUD's own lexicons, `place.hud.*`, are in `schemas/`. An operator's app may ship its own beside its views, and HUD reads those before the bundled ones ([an app's lexicons](views.md#an-apps-lexicons)).

HUD resolves other lexicons from the network, first the `_lexicon.<authority>` TXT record, then the authority's `com.atproto.lexicon.schema` record. It resolves their refs one level deep. It keeps them for 24 hours and answers misses for 1 hour, in memory and on disk. It doesn't keep a missing answer on the way.

## Blobs

Every image, audio and video blob in a record gets a sibling `url`, `<public origin>/blob/<did>/<cid>`. The proxy there fetches the blob from the account's PDS. It serves only these:

- images (not SVG) up to 20 MB
- audio up to 100 MB
- MP4, QuickTime and WebM video up to 100 MB

When a PDS sends a blob untyped (`application/octet-stream`), the proxy knows it by its first bytes. It never serves a blob as HTML or SVG. Browsers cache it forever, since a CID never changes.

Images stream straight through. The proxy keeps audio and video on disk (`<HUD_CACHE_DIR>/blobs/<cid>`), and only if their bytes hash to their CID. That lets a player seek, because PDSes answer whole blobs only and the proxy answers `Range` from the cache.

The proxy downloads each audio or video blob once, however many requests ask for it while it comes. It answers each request as the bytes arrive, against the length the PDS said, so a player starts on the first bytes, not the last. It sends a blob's last byte only once the whole hashes to its CID. A blob whose PDS gave no length is answered once all of it is here.

Like every other outbound request, the proxy's requests refuse plain `http` and private, loopback and link-local addresses. A DID document can name any host.

## Image CDN

The image CDN (`HUD_IMAGE_CDN`) serves an image of a public record instead of the proxy. That covers images read from the record's repository with `com.atproto.repo.*` or found in the Bluesky AppView's answer. GIFs are the exception, since the CDN would make them still.

The CDN serves a JPEG at the size HUD draws it:

- `url` is the CDN's `feed_fullsize`
- `thumb` is its `feed_thumbnail` (`avatar_thumbnail` for an `avatar` or `icon` field)
- `proxy` is the blob proxy's URL, which the renderer loads once instead when the CDN fails

A 695 KB avatar is 6 KB as its thumbnail.

Every caller of `blobs::rewrite` says whether what it read is public. A record of a space keeps every blob on the proxy alone and never names it to a public CDN. So does an answer of any service but the AppView's to the signed-in account.

## Joins

HUD keeps no index of its own. It lists a repository's records of a backlinks collection whole and keeps them a minute. It reads each record a ref names from its own repository's PDS, eight fetches at a time per call. If a record can't be read, its result is null or skipped, and the call still answers.

Query joins call their services as `query` does, once per answer, all at once, each within 3 s. So a slow one costs a tile at most 3 s, and a failed one gives a null. A query view's joins that don't read its answer start with its call ([views](views.md#when-its-called)).

## Counts

A network-wide backlink index (Constellation, by default) counts the records pointing at a record, by collection and path. HUD asks it once per record and keeps its answers and failures a minute. It shows null counts when the index is off (`HUD_BACKLINKS=none`) or failing.

Views name what to count and never name the index, so another index can replace it. `describe_collection` lists the counts of each sample, so a model writing a view can see which there are.

## Queries

The service's DID document gives the endpoint of the service that its fragment names. HUD calls it through the same outbound client, within 10 s and 5 MiB.

Blob refs in an answer get URLs where HUD knows the repository holding them. That's the case for a record beside the `uri` naming it. Images go through the image CDN when HUD asks as no one or asks the AppView. Otherwise they go through the proxy alone.

Every video the answer shows (`app.bsky.embed.video#view`) gets a `url` of its blob from the proxy, in the repository of the post around it. Its own playlist, HLS on Bluesky's video service (`video.bsky.app`, its segments on `video.cdn.bsky.app`), is adaptive and served from a CDN, so the resolved tree gives it too (`hls`). Where the platform plays HLS natively (iOS, Safari, Chrome), the video plays from there. Elsewhere (Firefox) it plays the blob from the proxy, and the view never loads hls.js. Where the HLS fails, the web falls back to the proxy as a second source, and iOS goes on from the proxy where the HLS stopped.

HUD makes calls as no one. When signed in, it makes them through the account's PDS, which proxies them to the service (`atproto-proxy`), for the services its sign-in asks to call ([sign-in](sign-in.md#what-hud-asks-for)).

### Kept answers

HUD keeps each answer a short while for whoever asked it, and no one else: 20 s when it asked as no one, 8 s for a signed-in account. An answer is kept by the method, the service, the params (by name, so their order doesn't matter) and the account. So one person's answer never reaches another. This holds for `query`, query joins and the Bluesky posts a document embeds. Grain's Explore, opened again within that time, answers in ~6 ms, where its three calls to grain.social took ~700 ms (measured, signed out).

When an account writes through HUD (`create_record`, `apply_writes`, `delete_record` or `call_procedure`), HUD forgets that account's answers as the write returns, landed or not. So its next read shows its own like or comment. An answer still on its way then isn't kept.

Calls of the same query by the same person at the same moment share one call to the service, and its answer or error. HUD never keeps an error, so the next ask calls again. If the call that leads is dropped (what asked stopped waiting), the next caller makes it again. A view or query join that needs what's there now says `fresh` ([views](views.md#when-its-called)). HUD then calls it each time and keeps none of its answers. Kept answers share the cache memory as [memory](memory.md#server) says.

## MCP

HUD serves Streamable HTTP at `POST /mcp`, stateless. It serves 2026-07-28 and `initialize`-era clients (2025-11-25, 2025-06-18) too, with a bearer token in either. A modern request's client capabilities may name HUD's kit, the extension `place.hud/kit`, and each tool's tree is fitted to the kit it names (`hud_core::kit`, [old builds and new views](views.md#old-builds-and-new-views)).

## Compression

HUD compresses its answers itself, since no proxy in front of it does. An answer of JSON, HTML, SVG, JavaScript or CSS of 1 KB or more goes out as Brotli or gzip, whichever the request's `Accept-Encoding` prefers, and as it is when that names neither. Brotli runs at quality 4, quick enough for each answer. An answer that comes in parts (a page the web app streams) is compressed part by part, and each part goes out as soon as it's compressed. An event stream is never compressed, so its events reach the client as they're written. Blobs and icons aren't either, since they come compressed, and neither is what the web app compressed itself.

## The web app's pages

With `HUD_WEB` set, HUD is the front door of your own web app, such as a SvelteKit app that draws pages on the server. HUD hands the app `/`, `/favicon.svg`, `/favicon.ico` and every request that none of its own routes take. HUD's routes always win, and so does any path under `/oauth/`, `/.well-known/`, `/xrpc/`, `/blob/`, `/icon/` or `/workspace/`, even with no route there. That way a mistyped method never comes back as a page.

| HUD's | The app's |
| --- | --- |
| `/mcp`, `/_health`, `/playground`, `/terms`, `/privacy`, `/guidelines`, `/workspace` and the prefixes above | `/`, `/favicon.svg`, `/favicon.ico`, and every other path, such as `/profile/<handle>` and `/_app/immutable/<file>.js` |

The app gets the request as the browser sent it, with its method, cookies, body and `Host`. HUD adds `X-Forwarded-Host`, `X-Forwarded-Proto` and `X-Forwarded-For` (the client's address after any a proxy in front of HUD named). It keeps a forwarded host or scheme that a proxy in front of it set, and otherwise sends the request's `Host` and the scheme of `HUD_PUBLIC_URL`. The answer goes back with its status and headers, `Set-Cookie` and `Location` too. HUD follows no redirect, so the browser does. Bodies go through as they come both ways, so a page the app streams reaches the browser part by part.

The app is the one server HUD reaches past its filter of public addresses, through a client of its own (`hud_core::net::Web`). Its origin comes from you, so a private address (a container's network) is fine. HUD looks its host up as each request goes out, so it starts before the app is up and answers a page it can't reach with a 502. Each request is logged as the route `web`, and the call to the app as any outbound call is ([observability](observability.md)).

## Sign-ins

HUD keeps sign-ins under `<HUD_DATA_DIR>/oauth/`:

- the client key
- each session, an MCP client's or the workspace's, rewritten whole on every change
- the clients registered

A session holds its atproto tokens and DPoP key, and the SHA-256 of HUD's tokens or cookie for it. It never holds those tokens or that cookie themselves.

HUD sweeps expired sessions every hour. It never logs tokens.

## Canvases

Canvases sit behind a store trait (`hud_core::canvas::store`). By default it's the server's own files (`FileStore`). Each canvas is `<HUD_DATA_DIR>/canvas/<id>.json`, with its info and tiles. Each person's index is `canvas/by-did/<sha256 of the DID>.json`. It holds their canvases, their tabs in order and their current one, and HUD rebuilds it from the canvases if it's missing. HUD writes these files whole after each change.

A canvas from before a person could have many is kept as `canvas/<sha256 of the DID>.json.moved` once moved.

HUD turns tiles from before columns (`x`, `y`, `w`, `h`) into columns as it reads them at start, and writes them back. It orders them by rows, top to bottom, and each row left to right, as 1, 2, 3…. It takes their width from their pixels: under 520 a third, under 760 a half, else two thirds. A person from before tabs gets every canvas they've joined open, ordered by when it was made.

With `HUD_SPACES=on`, `SpaceStore` (`hud_core::canvas::space`) keeps a canvas in its owner's own space when no one else is in it, as [the workspace](workspace.md) describes. It goes through their OAuth session (`hud_core::space`, the `com.atproto.space.*` methods, DPoP as on every PDS call). It checks each record against its lexicon first. It turns tile records from before columns into columns as it reads them and writes them that way. It holds those canvases in memory as last written, and the person's index stays a file.

Every change, and who's on each canvas, goes to the members over one broadcast channel per person. HUD makes the channel when it's first needed and lets it go when no stream listens. Each stream passes on what concerns the person's open tabs.

## Themes

HUD keeps each person's chosen theme as `<HUD_DATA_DIR>/theme/<sha256 of the DID>.json`, written whole. One of HUD's is kept by its key, and a person's theme is copied whole. With none kept, the workspace follows the system's light or dark.

## Icons

HUD asks `HUD_FAVICONS` for an app's favicon at `https://<domain>`, within 5 s. If the answer is a 200 `image/png` of at most 100,000 bytes, HUD keeps it as `<HUD_CACHE_DIR>/icons/<domain>.png` for a week. Otherwise it keeps `<domain>.miss` for a day.

A domain is a hostname, never an address. It's never a private name unless `HUD_ALLOW_PRIVATE` is set.

## Checks

`just check` runs rustfmt, clippy, the tests under `just cov`, svelte-check, the renderer's server test, `just lint` (the workspace's layers and contrast, and the docs) and `just lex`. `just cov` runs every crate's tests under cargo-llvm-cov and fails if they run less than 100% of the lines or functions outside the tests themselves, naming each line they miss. It counts the `hud` binary's lines too, as tests run it. `just lex` runs two scripts:

- `harness/lexicons.mjs` applies goat's lint and the Lexicon Style Guide's rules to `schemas/`, and checks that they only grow from main's.
- `harness/answers.mjs` checks every XRPC answer that hud-server's tests receive against its lexicon.

The server test (`cd web && npm run ssr`, `web/renderer/test/ssr.mjs`) draws Grain's screens with the renderer compiled for a server, in Node with no browser globals. Then it hydrates each one in a happy-dom document with Svelte's development build. It fails if a screen throws, draws nothing, has no link for a call it opens or nests a link in a button. It also fails if hydrating warns or changes what the server drew. The screens are trees in `web/renderer/test/trees/`, which `just ssr-data` makes from the gallery's recorded answers and a few made of them (`harness/ssr/data.mjs`).

The runner (runner.knotless.dev) runs the same checks on every push to main and every pull into it, from `.knotless/workflows/`, the docs as `docs`. It also runs the e2e flows, `e2e` on a stock PDS and `e2e-spaces` on the spaces alpha. Main takes pulls only through the merge queue, each once review.knotless.dev has reviewed its head.

## The view and the workspace

Each one is a self-contained HTML file built from `web/` (Svelte 5, Vite) as `web/dist/index.html` and `web/dist/workspace.html`, and embedded in the binary. Both draw with `@hud/renderer` (`web/renderer`), which knows nothing of MCP.

The renderer draws on a server too, so an app can send its pages as HTML and hydrate them. `Tree` takes `now`, the time the page is drawn at, and `hrefOf`, which gives the address of the page a call draws. The app passes both the same on the server and in the browser. Each call the tree opens is then a link (`<a href>`) that works before the page's code arrives and opens in a new tab with a modifier key. A plain click still goes to `oncall`. Until the page is mounted, dates and counts are written in `en-US` and UTC and times are read against `now`, so the browser's first drawing matches the server's. After that they're in the browser's own locale, time zone and clock. What a person did (a like, a text opened in full) is kept in the browser alone, so a server holds nothing of one request in the next.

A browser that translates the page (Chrome's, Safari's, Google Translate) leaves alone what the renderer and the shell mark `translate="no"`: names and identifiers, never words. That's a picture's ALT badge and its ⓘ lines (camera, lens, settings), a short time ("3h"), accounts' names, @handles and DIDs (a `Link` with `style: "name"`, a `Text`, title or label that is an @handle or a DID whole, a mention in text, a profile's names, a face's or an account's row) and the app's wordmark. Captions, titles, comments and the app's own words ("Show more") are translated with the page. `translateOf` (`web/renderer/src/translate.ts`) says whether a text is an identifier.