# SYSTEM: socialweb-computer, the whole system - one page map of every component, repo, host and flow - deep dives: [KCP.md](KCP.md) (kcp, workspaces, API catalog), [protocol/README.md](protocol/README.md) (every lexicon, its use and its step in the sequence), [ARCHITECTURE.md](ARCHITECTURE.md) (decisions, invariants; open by heading) - rules for agents: [../AGENTS.md](../AGENTS.md) - no history here. what was done when: git log, `docs/plans/*.yaml` ## What It Is socialweb-computer is a compute market on atproto. A requester posts a signed RFP record to its own atproto repo; bidders (hosts that run firecracker VMs) see it on the firehose, bid with records in their own repos, the requester accepts one bid, and the winner boots a VM from cloud-init `user_data`. The requester reaches the guest over iroh (plus opt-in opentunnel https and webrtc). On top of the market sit products: ssh into a fresh VM as yourself (`ssh @socialweb.computer`), tenant Kubernetes clusters that scale nodes from zero by renting market VMs, a browser page and an iPhone app that run a requester in process, and atirc agents that join a chat channel from inside a VM. Every application is a kcp of its own (or a workspace of one shared kcp in a market network); state is kcp objects; secrets live only in the application's own OpenBao. ## Repos | repo | upstream | role | where it runs | |---|---|---|---| | socialweb-computer-kcp | https://github.com/publicdomainrelay/socialweb-computer-kcp | the monorepo: Go core `socialweb-computer`, guest agent, vmexec, apigen, lexicons, manifests, policy actions, socialweb-meshd (rust) | gateway host, every bidder, CLI users, guests (guest tools), wasm page and iOS app (as a library) | | kcp fork | https://github.com/johnandersen777/kcp | kcp built for js/wasm with memory storage; builds the browser page (`socialweb-wasm/`, `page-rcN` tags) | shell.socialweb.computer (browser) | | kcp (native) | https://github.com/johnandersen777/kcp at `d71f020c` (tag `swc-apibinding-requeue-1`: kcp-dev/kcp main `74e9fb9c` plus one commit) | native kcp asset, built from that fork and pinned in `internal/versions/versions.json` | gateway, bidders, CLI requesters, market networks | | indigo | https://github.com/bluesky-social/indigo | atproto Go libs (repo, MST, xrpc, identity), Go module dependency; local checkout for reading | linked into socialweb-computer | | atirc | `git@tangled.org:did:plc:vtad4w7ujsvz7t6o6iuvmhjq` | atproto IRC: channels, subhandle tokens (mint, revoke, lease), agents, web chat, send/recv pipe | irc.fedfork.com server; `atirc` CLI in guests and on operator hosts | | AT-IRC iOS app | local only (`atirc-client-ios`) | iPhone chat client; embeds `pkg/embedded` (in-process kcp + requester) to launch VM agents | iPhone | | did-key-associator | local only (frozen TS, copy also in monorepo history) | qr.fedfork.com: OAuth QR session-transfer page for headless identities (`oauthQr`) | qr.fedfork.com | | atproto-reverse-proxy | https://github.com/publicdomainrelay/atproto-reverse-proxy | fedproxy.com ingress: legacy deno relay, ssh relay, caddy; the Go ingress relay from this monorepo runs as a drop-in | fedproxy.com | | typescript-helpers | this repo, `typescript-helpers/` (the GitHub repo is private since 2026-11-20) | frozen TS helpers (event streams, abc, cli-args-env); reference only | nowhere live | ## Hosts And Roles | host | role | runs | public names | |---|---|---|---| | socialweb.computer | gateway host | Gateway app (kcp 1): GatewayAPI web app, cluster provider, tenant k3s control planes, k8s requester; SshGateway app (kcp 2): Go ssh server on :22, per-user Requesters; caddy 443; host sshd on 1997 | k8s.socialweb.computer, kcp.socialweb.computer, ssh.socialweb.computer | | hat-8 | "local" bidder + operator/lead host | MarketBidder (account `swc-bidder`), dev market networks, builds, deploy scripts | none (requestCrawl to relay) | | hat-7 | bidder | MarketBidder (bob, `oauthQr` identity) | none | | hat-9 | bidder | MarketBidder | none | | irc.fedfork.com | atirc | atirc server, `atirc-iroh.service`, `/chat` web client | irc.fedfork.com, `did:web:irc.fedfork.com#atirc` | | fedproxy.com | did:key xrpc ingress | Go `internal ingress-relay` (caddy drop-in `30-go.conf` of `xrpc-relay-server.service`), ssh relay :2222 | xrpc.fedproxy.com, `*.fedproxy.com` | | relay.mini-cloud-0002.chadig.com | atproto relay | Go `internal atproto-relay` (127.0.0.1:2584 behind caddy) | firehose source, bidders requestCrawl here | | qr.fedfork.com | OAuth QR | did-key-associator backend | qr.fedfork.com | | shell.socialweb.computer | static page | wasm kcp + requester in the browser tab | shell.socialweb.computer | | iPhone | client | AT-IRC app, embedded requester | none | - every listener behind the edge binds port 0 and reports its address into kcp status - only stable public entry points keep fixed ports: caddy 443, gateway kcp front 127.0.0.1:6443, GatewayAPI web-app front 127.0.0.1:8788, ssh 22 - prod releases come from CI `promote`: it moves channel `rc`, deploys the `socialweb.computer` droplets, and the rc bidders upgrade themselves by following the channel ([deploy.md](deploy.md), [protocol/release.md](protocol/release.md)) ## System Context ```mermaid flowchart LR subgraph USERS["Users"] SSHC["ssh client"] CLI["socialweb-computer run request"] PAGE["browser page shell.socialweb.computer"] IOS["AT-IRC iOS app"] KUBE["kubectl / XRPC k8s client"] end subgraph GWH["socialweb.computer"] GW["Gateway app: GatewayAPI, cluster provider, tenant k3s"] SG["SshGateway app: SshServer :22, per-user Requesters"] BAO["OpenBao: OAuth sessions, keys"] end subgraph BID["Bidders: hat-8, hat-7, hat-9"] MB["MarketBidder: Bidder, GuestControlPlane, Pds, PolicyEngine"] VM["firecracker guests"] end subgraph AT["atproto network"] PDS["PDSes: user repos"] RELAY["relay.mini-cloud-0002.chadig.com"] JS["Jetstream us-east, us-west"] PLC["plc.directory"] ING["xrpc.fedproxy.com ingress"] QR["qr.fedfork.com"] end ATIRC["irc.fedfork.com atirc"] SSHC -->|"ssh handle@"| SG KUBE -->|"service auth XRPC"| GW CLI -->|"records + submitRfp"| PDS PAGE -->|"records"| PDS IOS -->|"records"| PDS SG --> BAO GW --> BAO SG -->|"rfp, accept records"| PDS GW -->|"rfp, accept records"| PDS MB -->|"bid, receipt, events"| PDS PDS --> RELAY RELAY -->|"firehose"| MB RELAY -->|"firehose"| SG JS -->|"jetstream"| MB ING <-->|"submitBid, submitEvent"| MB ING <-->|"did:key callbacks"| SG MB --> VM SG -->|"ssh over iroh"| VM VM -->|"atirc agent"| ATIRC SG -->|"mint, revoke"| ATIRC IOS -->|"chat"| ATIRC MB -->|"oauthQr sign-in"| QR PDS --- PLC ``` ## Applications - one root object per application, in its own kcp (prod) or its own workspace `root:` (market network) - the core (`socialweb-computer` process) runs the root's composition controller (`internal/app/`), every controller, the component runtime and the HostProcess supervisor - components run in process; third-party binaries (kcp, kine, OpenBao, dumbpipe, firecracker via vmexec, deno for policy) are HostProcess units | root kind | CLI | composes (main children) | role | |---|---|---|---| | MarketBidder | `run bidder`, `install bidder` | Bidder, GuestControlPlane, Pds (localRepo mode) or RemotePDS (OAuth), PolicyEngine, OpenBaoServer, IngressProxy, MarketIdentity, EventStreams, ComputeVM/FirecrackerVM (per contract) | sells VMs | | MarketRequester | `run request` | Requester, PolicyEngine, OpenBaoServer, Pds (localRepo) or OAuth identity, IngressProxy, MarketIdentity, EventStreams, ComputeContract | rents VMs; one fresh kcp per invocation, state dir persists | | Gateway | `serve gateway kcp`, `install gateway` | GatewayAPI, Requester, PolicyEngine, OpenBaoServer, cluster provider, MarketIdentity (alice requester), Infra per cluster | hosted tenant k8s clusters | | SshGateway | `run gateway ssh` | SshServer, per-DID Requesters, PolicyEngine, OpenBaoServer `ssh-gateway`, iroh dialer HostProcess, session-owner MarketIdentities, Exposures | ssh as yourself into a fresh VM | | MarketNetwork | `dev up --profile market-network` | Plc, Relay, Pds alice/bob, IngressProxy dispatcher, DispatcherTLS, PolicyEngine, OpenBaoServer `network` (PKI), child roots (bidders, gateway, requesters, ssh gateways) | dev/CI/soak: a whole local market in one kcp | ### Components (in-process kinds) | component | package | does | |---|---|---| | Bidder | `internal/services/bidder`, `internal/market/bidder` | reads RFPs, runs policy, bids (`bids.free`), takes accepts, writes receipts, creates ComputeVMs, publishes vm events | | GuestControlPlane | `internal/services/guestcontrolplane` | guest-facing endpoints at `http://192.0.2.1:`: OpenBao allow-list proxy (workload identity), `POST /v1/events`, `GET /v1/downloads//`, `compute.guest.subscribeEvents` | | Requester | `internal/services/requester`, `internal/market/requester` | runs each ComputeContract: RFP, bid window, accept, receipt, network, secrets, atirc mint/revoke, release | | Pds | `internal/services/pds`, `internal/pds/local` | a local PDS for an identity without a real account (localRepo); `spec.accountCreation` Closed (default): only the application's own code makes accounts on it, XRPC `createAccount` answers 400 `InvalidInviteCode`; Open on a market network's Pds | | Plc, Relay | `internal/services/plc`, `internal/services/relay` | network-local PLC and relay (market network) | | IngressProxy | `internal/services/ingressproxy` | did:key ingress to the app's XRPC (`xrpc-subscription`, iroh or opentunnel provider); default host `xrpc.fedproxy.com` | | DispatcherTLS | `internal/services/dispatchertls` | TLS front of the market network dispatcher | | GatewayAPI | `internal/services/gatewayapi`, `internal/gatewayhttp` | k8s.socialweb.computer: `/oauth/*`, `/session`, `/api/*`, `/xrpc/computer.socialweb.k8s.*`, `/downloads/*` | | SshServer | `internal/services/sshserver` | ssh on :22, web sign-in at `/`, `/keys`, session owners, per-session contracts | | PolicyEngine | `internal/policyengine` | gha-lite workflows in embedded deno; bundled actions `bundled-actions/tangy/` | ## Market Protocol - lexicons: `lexicons/` (NSID prefix `com.publicdomainrelay.temp.`) - every lexicon, field by field, with its sequence step: [protocol/README.md](protocol/README.md) - applications talk ONLY over this protocol. a shared kcp is never a data channel ### Records | record | written by | in repo of | purpose | |---|---|---|---| | `compute.vm` | requester | requester | the VM asked for (size, role) | | `market.rfp` | requester | requester | signed request for proposals; strongRefs vm payload + policy; callbacks submitBid/submitEvent | | `market.offering`, `market.heartbeat` | bidder | bidder | what the bidder sells; liveness. offering at rkey `self` | | `market.bid` + `market.bids.free` / `bids.x402` | bidder | bidder | bid envelope + payload. rkey `market.BidRKey` (one bid per RFP) | | `market.accept` | requester | requester | the chosen bid (no payload record) | | `market.receipt` | bidder | bidder | bidder took the accept; payload = the RFP compute.vm ref | | `market.event` + `compute.events.vm.*`, `compute.events.guest.*` | both | sender | lifecycle: vm.hostKeys, vm.onNetwork, vm.delete, vm.terminated, guest.* | | `market.association`, `link.ident.endorsement` | principal (operator, renter, R2) | principal | the grant + the ident.link endorsement of an agent, rkey = agent DID ([design/ident-link-associations.md](design/ident-link-associations.md)) | | `market.associationRequest` | guest requester R2 | R2 | {principal, contract, nonce}, rkey = principal DID | | `badgeBlueKeys` | signer / user | signer / user | key bindings: signer key binding, ssh key claims | | `sh.tangled.graph.vouch` | operators, users | voucher | trust edge read by tangled-vouch policies | | `computer.socialweb.infra`, `computer.socialweb.ephemeralInfra` | the owner's kcp (infra controller: CLI, gateway, page, iOS) | owner | the record of an Infra / EphemeralInfra object (two-way sync); per-account index at rkey `self`; never deleted ([protocol/infra.md](protocol/infra.md)) | ### XRPC | method | served by | caller | |---|---|---| | `market.submitRfp` | bidder ingress | requester (push path); 409 `RFPClosed` after the window | | `market.submitBid` | requester ingress | bidder | | `market.submitAccept` | bidder ingress | requester | | `market.submitEvent` | both ingresses | either side | | `compute.secrets.getSecrets/` | requester ingress | guest `setup-secrets.service` (workload identity token) | | `compute.guest.subscribeEvents` | GuestControlPlane | guest agent (WebSocket, cursor, 25 s keepalive) | | `computer.socialweb.k8s.*` | GatewayAPI | cluster owners (service auth, aud `did:web:k8s.socialweb.computer`) | | `com.fedfork.exec.ops.listVms`, `getVm`, `getBidder`, `getHealth`, `getLog`, `getVmMemory` | bidder ingress (its market endpoint: https where its offering has one, else iroh) | the operator's checks (the prod gate in CI): `Authorization: Bearer ` | | `com.fedfork.exec.ops.getRequesterLog` | SshServer web origin | the same | Read-only operations routes (owner, 2026-11-18; lexicons `lexicons/com/fedfork/exec/ops/`, contract `internal/contract/opsread`, routes `internal/services/opsread`, host reads `internal/opsread/hostread`, kcp reads `internal/controllers/opsread`). GET only; each answers one fixed shape, takes no selector, path or shell text, and writes nothing. They exist only when the host holds an ops read secret: a file of mode 0600, `/ops-read-secret`, or for a bidder the path of `run bidder --ops-read-secret-file` (`install bidder` writes it to config.json); no file or an empty one: 404. The secret is compared in constant time (sha256 of both sides), never logged, never in a kcp object; a wrong or missing bearer is 401, a POST 405. A bidder's routes are on its market mux, so they are reached over its market endpoint, the one its `market.offering/self` record names: its https endpoint where the offering has one (staging), else its iroh endpoint (prod bidders are behind NAT and offer iroh and webrtc only). Over iroh the test host (`tests/host/src/ops.rs`) sends the same GET and `Authorization: Bearer` as HTTP/1.1 through atlibs-iroh's raw fetch (`iroh:///`: `dumbpipe connect `, the stream a requester's market calls take to that listener), end to end encrypted to the iroh node the ticket names. What binds the ticket to the bidder: the offering record in the bidder's own repo on its PDS, the PDS resolved from the bidder DID's PLC document, the DID from `deploy/release/rc-operators.json` (`scripts/ci-prod-target.sh`). A URL is never `http://` (the secret goes to https origins or an iroh endpoint ticket only). The gateway's `getRequesterLog` is on its https web origin. | route | params | answer | |---|---|---| | `listVms` | none | `{items: [{metadata {name, deletionTimestamp}, spec {vm.cpus, requester.did, contract.receipt.uri}, status {phase, lostReason}}]}`: the bidder's ComputeVMs, every tenant's | | `getVm` | `name` (a DNS label) | `{computeVm (as in listVms), firecrackerVm {metadata {name, deletionTimestamp}, status {phase, lostReason, image {sha256, fetched}}}}`, each null when absent; no work dir, socket, pid, tap, MAC or address | | `getBidder` | none | `{identity {metadata, status {phase, pendingAction {kind, reason}, conditions [{type, status, reason}]}}, bidder {metadata, spec {policy}}, marketBidder {metadata, status {phase, did}}}`: never the pending action's URL, a session ref, associations, declines, listeners or disk usage | | `getHealth` | none | `{unit, started, errors, startupErrors, restarts, vmProcesses, errorSample[<=3] {time, msg, err<=160}, guestImageReadyAt, guestImageUnready}` of the core since its process started (startup: its first 10 min; restarts: the unit's `NRestarts`) | | `getLog` | `msg` (one of the names in `opsread.Messages`), `subject` (optional: a ComputeVM name, `vm-` or an at:// URI) | `{log, count, lines[<=50]}`: lines since the process started holding that message's texts and the subject; a line is `time`, `msg` and the fixed keys of `opsread.LineKeys` | | `getVmMemory` | `name` | `{unit, current, high, max, swapMax, events}` of the VM unit's memory cgroup | | `getRequesterLog` | `did`, `since` (RFC 3339) | `{lines}`: the ssh gateway core's lines of that DID since then whose msg is one of `opsread.RequesterMessages`, as `time`, `msg`, `subhandle`, `contract`, `receipt`, `deleted`, `of`, `reason`, `accept` | The log is the core's own: its stderr file when it has one (a dev instance, the ssh gateway: `logs/core.log`, every logger of the process writes there). Else, once the secret is configured, the process's one log handler (`internal/kit/logx` `Tee`, under every logger the CLI hands out: the components', the controllers', the compute stack's) also writes each line to `/logs/core.log` from `devcore.Run` on, and the routes read that (an installed account cannot read its own journal entries); logcap rotates it to `core.log.1` at 64 MiB and the read takes `.1` then the file. Else the journal of its uid. A failed read answers 502 with a fixed message; the detail is in the host's log. ### Comms modes - `spec.comms.mode`: `XRPC` (ingress only), `Firehose` (no inbound endpoint), `Both` (requester default) - event sources (Firehose/Both): network relay or `relay.mini-cloud-0002.chadig.com`, Jetstream v2 us-east/us-west - server-side filter: `wantedCollections`, `wantedDids`; client filter always runs too - requester stream carries `market.bid`, `market.receipt`, `market.event`; bidder stream `market.rfp`, `market.accept`, `market.event` - payload records are read through the strongRef (getRecord), never taken as stream input - dedup: one sink per role drops a `uri|cid` seen in the last 15 s - addressee index: requester keeps only records about its own ComputeContracts; bidder only RFPs, accepts of its bids, events of its receipts. rest counted as `market_foreign_dropped` ### Selection - the bidder's bid window: 10 min from RFP `createdAt` (`market.RFPBidWindow`); the requester's: `bidWindowSeconds` (30 s) - bidder bids once per RFP (Bid object name = rkey) - requester picks, recorded in `status.chosen.reason`: - `firstFree`: first scope-policy-allowed free bid ends the window (the normal case; fastest bidder wins) - `policy`: else the lowest cost after the window, confirmed by the RFP's attached policy - `bestWithinWindow` and `resumed` are in the enum; no code path leaves them written - `status.bids` keeps at most 20 `{did, uri, cost, verdict, arrivedAt}` (`offer` is in the type, never set) - the whole flow with its failure branches: [protocol/flow.md](protocol/flow.md) ### Policies - a market policy ALWAYS runs, both sides, both lanes (scope + full), every decision. no skip paths - defaults: `bidder-tangled-vouch` (bidder), `requester-tangled-vouch` (requester). `open` only when named, never prod - tangled-vouch: counterparty must be vouched (`sh.tangled.graph.vouch`) by the operator; both directions needed for a deal - bidder full lane = own workflow + every policy the RFP attaches; allow only if all allow - caching lives inside the engine run (`cache-key` over workflow, mode, perspective, DIDs, records, args). never across parties, never a skip - missing engine, evaluator or non-boolean `allow` = deny ### Market sequence ```mermaid sequenceDiagram autonumber participant R as Requester participant RP as requester PDS participant FH as relay / jetstream participant B as Bidder participant BP as bidder PDS participant VM as guest VM R->>RP: putRecord policy record, compute.vm, market.rfp R->>B: submitRfp RP->>FH: commit FH->>B: market.rfp B->>RP: getRecord rfp payload + policy B->>B: bidder-tangled-vouch (scope + full) B->>BP: market.bid + bids.free B->>R: submitBid (ingress) R->>R: requester-tangled-vouch, firstFree R->>RP: market.accept R->>B: submitAccept B->>B: capacity check + reservation B->>BP: market.receipt B->>VM: ComputeVM to FirecrackerVM, boot user_data VM->>B: workload login binds its ssh host key VM->>B: POST /v1/events vm.onNetwork (iroh ticket) B->>BP: market.event vm.hostKeys, vm.onNetwork B->>R: submitEvent R->>R: ComputeContract Held R->>VM: ssh / tunnels over iroh R->>RP: market.event vm.delete R->>B: submitEvent vm.delete B->>VM: stop, remove B->>BP: market.event vm.terminated R->>RP: delete own contract records R->>R: ComputeContract Released ``` ### ComputeContract phases - `Pending` (only while admission holds it back) -> `Requesting` -> `Held` -> `Releasing` -> `Released`; or `Failed` (a Failed contract is still released); a `vm.terminated` can settle Requesting or Held straight to Released - `status.contract.phase`: requested, rfp_submitted, winner_selected, accepted, receipt, network, released - `status.contract`: strongRefs rfp/bid/accept/receipt, winner, vmAddress (iroh), hostKeys, networks, release, termination, timestamps - every write lands before the flow moves on; a restarted runner resumes from the recorded stage - release: `spec.releaseRequestedAt` (SSA, release manager), outcome in `status.release.requestedAt`; the vm.delete submitEvent is retried 5 s doubling to 5 min until the release deadline (an ssh session's release: 6 tries, 1 s doubling to 30 s) - a release whose vm.delete keeps failing settles Released once the bidder's vm.terminated appears ## Guests ### Boot - the RFP flow is the ONLY way a guest comes up. no hand-started VM, no agent install outside cloud-init - `user_data` built by `BuildUserData` + module registry (`internal/atproto/cloudinit`) - requester modules carry config only; bidder adds `setup-guest-binaries.service` (`internal/guestseed`) - binaries come from the bidder's GuestControlPlane `GET /v1/downloads//`, sha256 checked, installed to `/usr/local/bin`. never GitHub/GHCR from a guest - downloads = release asset `socialweb-guest-downloads`: opentunnel, wootty, atirc, socialweb-guest-agent - opt-in `spec.guestTools: [socialweb-computer]`: bidder serves its own binary (linux/amd64 only) - guest image: `firecracker-x86_64` (or `firecracker-aarch64`), pinned by sha256 in `internal/versions/versions.json`, from `guest-images//` in the public bucket ```mermaid flowchart TD A["Bidder: accept taken, receipt written"] --> B["ComputeVM created, owned by Bid"] B --> C["FirecrackerVM: vmruntime Prepare"] C --> D["socialweb-vmexec loads nft table swcguest"] D --> E{"fence loaded?"} E -->|"no"| X["VM does not start"] E -->|"yes"| F["tap + pasta, firecracker boots"] F --> G["cloud-init runs user_data"] G --> H["setup-guest-binaries.service: fetch by sha256 from 192.0.2.1"] G --> I["iroh module: endpoint up"] I --> J["POST /v1/events vm.onNetwork: iroh ticket, host keys"] J --> K["Bidder signs + publishes market.event"] K --> L["Requester: contract Held"] G -->|"comms XRPC or Both"| M["setup-secrets.service: getSecrets"] G -->|"comms Firehose + Sealed"| N["guest agent: guest.secretsRequested, sealed bundle"] M --> O["secrets.ready"] N --> O O --> P["units After= secrets: wootty, atirc-agent env"] ``` ### Transports (three, never add one) | transport | default | how | |---|---|---| | iroh | required, every guest | requester ssh + `status.contract.vmAddress`; contract Held on iroh address + host keys | | opentunnel | opt-in per `spec.tunnels[]` | https URL per tag to guest `127.0.0.1:`; `vm.onNetwork` kind `https`; optional wootty shell | | webrtc | opt-in per `spec.tunnels[]` | guest agent bridges datachannel tag to `127.0.0.1:`; STUN `stun.cloudflare.com:3478`, no TURN; `run request proxy`, `ssh --transport webrtc` | - signaling sealed: age-x25519 `guest.publicKey` per tag, offer/answer sealed. no key, SDP, ICE or IP in any record, object or log - browser shell: ssh over webrtc first, wootty fallback ### Fence - nf_tables bridge table `swcguest` loaded before the tap exists (`internal/guestnet`) - allowed from guest: TCP to `192.0.2.1` only on ComputeVM `spec.hostPorts`; DNS to pasta gateway; ICMPv6 RD/ND - dropped: host's own addresses, guest link, RFC1918, CGNAT, loopback, link-local, multicast, ULA; every ethertype but IPv4/IPv6/ARP - `lanUDP` (default on): UDP 1024+ to LAN (not 1900/5353/5355) for same-LAN webrtc - `hostUDP` (default only with policy `bidder-only-me`): UDP to host's own addresses - forwarded webrtc UDP port per ComputeVM from `webrtcUDPPorts` (default 31000-31999) - every guest (also with `lanUDP` off): UDP to host's own addresses on destination ports in `webrtcUDPPorts` (default 31000-31999): any host UDP socket bound there (other guests' forwards, the bidder's `bidder-mesh`) - TCP to LAN or to host addresses: never ### Secrets - `spec.secretsRef`: bundle `[{path, value, mode}]` in the requester's OpenBao - primary: getSecrets on requester ingress, auth = guest workload identity token (bidder OpenBao identity engine via GuestControlPlane proxy, GuestIdentityGrant) - sealed `secretsFallback: Sealed`: only in comms mode Firehose (no inbound endpoint), on demand, after `guest.secretsRequested`; sealed to the guest's key as ContractEvent `secrets-`; with an inbound endpoint never - never send a sealed bundle unasked - condition `SecretsApplied` reason `GetSecrets` or `SealedFallback` - guest events: guest POSTs to GuestControlPlane, bidder signs + publishes; bidder never opens a sealed body ## Identity And Auth ### Where identity lives - OAuth sessions, relay keypairs, OIDC signing key: ONLY in the app's own OpenBao - signing keys are 0600 files in the app's state dir: the attestation keys (`attestation.jwk`, `attestation-key.json`), `repo-key.json`, the ingress `market-relay.jwk`, the Pds `repo-keys/` and the PLC `plc-rotation.key` ([protocol/identity.md](protocol/identity.md#keys-who-signs-what)) - kcp objects hold references (SecretRef), never values - the callee holds credentials; a bus handle never exposes a token - atproto OAuth sessions held by the OpenBao atproto auth plugin (`auth/atproto session/`); refresh in the plugin - plugin keeps a session on transient refresh errors (`invalid_client*`, `use_dpop_nonce`, 429, 5xx, unreachable); expires on `invalid_grant` etc. ### MarketIdentity modes | mode | repo lives in | sign-in | used by | |---|---|---|---| | `localRepo` | app's own Pds component | none (account created at install) | default bidder, dev requester | | `pdsAccount` | a PDS account (password in vault) | app password | market network accounts | | `oauthQr` | user's PDS | QR session-transfer at qr.fedfork.com (`status.pendingAction.url`, 5 min) | headless bidder with a real account (hat-7) | | `oauthBrowser` | user's PDS | authorize URL in `status.pendingAction` | CLI requester, browser page | | `oauthDelegated` | user's PDS | session handed in by another app | delegated flows | | `oauthSessionProxy` | user's PDS | reads a session another component stored in the vault | ssh gateway session owners, gateway cluster owners | - scope upgrade: optional new scope (e.g. `rpc:chat.atirc.revoke`) -> `pendingAction {kind: signInAgain, reason: ScopeUpgrade}`, identity stays Ready - a required new scope = SessionExpired for OAuth identities on rollout: plan the re-sign-in first - granted scopes are read from the token response, never the access token's `scope` claim - sign-ins are scripted (`scripts/oauth-signin.ts`, headless Chrome). no sign-in link handed to a person ### Associations - ident.link pair + grant: agent DID doc `https://ident.link/agent-of/`; principal repo `link.ident.endorsement` + `market.association` at rkey = agent DID ([design/ident-link-associations.md](design/ident-link-associations.md)) - the identlink walk (`internal/identlink`, `sync.getRecord`, CAR verified) decides; MarketIdentity `status.associations` reports the roots (plus `spec.approvedAssociations`, unwalked). no manual patch - standing (bidder hosts, gateway requester): written by the app with the operator's session, or `scripts/associate-bidder.ts --agent-did ` - contract-scoped (R2, co-located accounts, VM agents): grant `{agent, contract, expiresAt}`, after the R2 nonce check ### Service auth - atproto service auth JWT (`lxm` + `aud`) for every cross-service call: from `getServiceAuth` of the identity's PDS session, or self-signed with `repo-key.json` by a `localRepo` requester; the ingress relay registration is a self-signed did:key token: - k8s API: aud `did:web:k8s.socialweb.computer`, lxm `computer.socialweb.k8s.*` - atirc: aud `did:web:irc.fedfork.com#atirc`, lxm `chat.atirc.mint` / `chat.atirc.revoke` - did:key ingress (IngressProxy) carries submitBid/submitEvent callbacks for apps without a public host - ingress registration is bound to the key: getRegistrationNonce answers 403 `IssuerMismatch` unless the token's `iss` is the `key` asked for; subscribe answers 403 `IssuerMismatch` before the upgrade unless the token's `iss` is the `did` param; only a signature whose `key` is the registration `key` counts (`no signature by the registered key`); `#renew` keeps its issuer check. A new subscription of the same key replaces the route and closes the old socket with 4009 (`replaced by a newer subscription of the same key`) ### ssh key claims - `badgeBlueKeys` at a stable `keyRkey` in the user's own repo, written via ssh.socialweb.computer `/keys` - ssh authorizer: one getRecord per offered key. no listing ## SSH Gateway - `ssh @socialweb.computer ` -> a VM rented AS that user, from that user's repo - sign in once at https://ssh.socialweb.computer (OAuth, session in OpenBao `ssh-gateway`) - admission: policy `bidder-tangled-vouch` scope lane, self = `spec.ssh.operatorDid`. refused = signed out, "this account was not admitted" - admitted -> session owner MarketIdentity `owner-` (oauthSessionProxy) -> Requester `requester-` - cap `spec.ssh.maxAccounts` (default 64) - per-DID requester has `spec.contractOwnerDid`: runs only that DID's ComputeContracts - every session = a ComputeContract: transient `ssh-run-` (TTL 1 h, released at command end unless `LC_KEEP_VM`) or held `LC_SESSION` `ssh-` (idle 5 min) - disconnect (client close, 3 missed 15 s keepalives) -> cancel command -> revoke atirc token -> release contract - restart: first sync releases stale `ssh-run-` contracts, adopts held sessions still alive ```mermaid sequenceDiagram autonumber participant U as ssh client participant W as ssh.socialweb.computer participant V as OpenBao ssh-gateway participant S as SshServer :22 participant P as user PDS participant R as Requester for DID participant B as Bidders participant VM as guest participant A as atirc U->>W: OAuth sign-in (once) W->>W: policy bidder-tangled-vouch (operator) W->>V: store session at auth/atproto session/DID W->>R: EnsureSessionOwner, compose Requester U->>S: ssh HANDLE@socialweb.computer CMD S->>P: getRecord badgeBlueKeys at keyRkey P-->>S: key bound to DID S->>R: session as ComputeContract R->>V: session via vault (oauthSessionProxy) R->>P: compute.vm + market.rfp B-->>R: bids R->>R: requester-tangled-vouch, firstFree R->>B: accept B->>VM: boot, vm.onNetwork R->>A: mint child token (if atirc cmd), leased S->>VM: ssh hop over iroh dialer, run as agent user U--xS: close / drop S->>A: revoke token S->>R: release contract B->>VM: terminate, vm.terminated ``` ## atirc Integration - atirc = atproto IRC at irc.fedfork.com; identities are subhandles `.` - tokens: `chat.atirc.mint` with explicit exp (max 24 h), `chat.atirc.revoke`; revoking a subhandle revokes everything below it - lease: a token minted for a contract is leased on the receipt + the bidder's `vm.terminated` at-uri. dies at vm.terminated even if revoke missed ### In a contract (`spec.agent`) - requester mints after the receipt is verified, writes `ATIRC_TOKEN` into the bundle at `/etc/atirc/atirc.env` - user_data installs `atirc-agent.service`, never starts it; `atirc-agent-env.path` try-restarts on env change - `guest.agentStart` (requester event) -> guest agent starts the unit - `run request --agent`, `run request agent ` ### In an ssh session - command starting `atirc` (or `LC_ATIRC=1`) -> agent module in user_data; requester mints child `ssh-.` - token reaches the command only via ssh env `LC_ATIRC_TOKEN` -> process env. never argv, never a file - `atirc agent run ` -> agent joins as `.`; mint failure fails the command - runs as the agent user (named by `atirc nick`, wheel, NOPASSWD), not root (claude refuses root) ### Revoke before release - order: revoke leased token BEFORE vm.delete, while the lease is alive; 401 "lease not alive" counts as revoked - every mint recorded under `atircMinted` in the bundle secret; background revoke with backoff (cap 5 min) until revoked or exp - `status.atircRevokes`: Done / Pending / Abandoned (24 h) - `run request release` waits up to 15 min (`RequestRevokeWait`) for revokes, exits non-zero on timeout and keeps state ### Guest requester (R2) - `spec.association {fromGuest}`: guest runs its own requester (`run request --guest-requester`) as the agent user - requester mints the R2 child token `r2-.` into `/etc/socialweb-computer/r2-atirc.env` - R2 mints its own agents below that subhandle (`spec.atircParentRef`) - release cascades: `guest.r2Release` -> guest requester releases its contracts -> `guest.r2Released` ### Clients - `atirc sub` / `atirc pub` (agents, operators), `atirc send` / `recv` pipe (`chat.atirc.pipe`) - web chat `/chat`, AT-IRC iOS app (also launches VM agents through its embedded requester) ## Tenant Kubernetes Clusters (Gateway) - the one traditional kcp use: tenant k3s `--disable-agent` control planes behind kcp - per cluster: k3s over kine over its own postgres, cluster OpenBao (identity bundle), External Secrets pair, two ClusterLinks (dumbpipe iroh in front of apiserver and vault) - blue/green control-plane slots a/b behind a per-cluster front (`controlplane.json`) - scale from zero: a pod -> NodeClaim -> one entry of the cluster's Infra `nodes-` -> ComputeContract `infra-nodes--node-` -> RFP flow -> VM boots as k3s node, joins over iroh - delete the pod -> claim deleted -> contract released -> VM gone - node network: `--flannel-backend=none`, bridge CNI, `socialweb-meshd` (rust) cross-node, CoreDNS per node - nodes published to the owner's repo: `computer.socialweb.infra/nodes-` (no secrets, no user data) - API: `computer.socialweb.k8s.kubectlApplyServerSide`, `getJob`, `subscribeClusters` with service auth ```mermaid flowchart LR POD["pod applied"] --> NC["NodeClaim (cluster home workspace)"] NC --> INF["Infra nodes-hex entry"] INF --> CC["ComputeContract"] CC --> RQ["gateway Requester: RFP flow"] RQ --> BD["Bidder: VM with k3s node user_data"] BD --> JOIN["node joins via ClusterLink iroh"] JOIN --> RUN["pod runs"] INF --> REC["owner repo: computer.socialweb.infra"] ``` ## Records Policy - never delete user records. no sweep, retention or GC of people's repo records - release deletes only the app's own contract-scoped records it wrote (e.g. its association) - record growth pain = fix the read, never the data - no `listRecords` on a collection that grows without bound; one `getRecord` at a deterministic rkey - stable rkeys: - bid: `market.BidRKey` per RFP - offering: `self` - infra index: `computer.socialweb.infra/self` - grant, endorsement: the agent DID - ssh key claim: `keyRkey` - `listRecords` remains only as a legacy fallback (e.g. a ComputeVM accepted before stable rkeys) - progress fields write at most once a minute (no churn commits) ## Deploy And Release | artifact | how built | where installed | |---|---|---| | release binary | `scripts/build-socialweb-computer.sh` (clean checkout, every pinned asset, `-tags "release embed_deno"`, `CGO_ENABLED=0`) -> `~/.tmp/socialweb-computer-build/release//` + `SHA256SUMS` | gateway, bidders | | dev binary | `--dev --publish /var/tmp/` | own market network only | | guest image | CI or `scripts/build-guest-image.sh`, content-addressed in the bucket | bidders fetch + verify | | guest downloads | `scripts/build-guest-downloads.sh` | embedded, served by GuestControlPlane | | browser page | kcp fork `scripts/browser-dev/build-page.sh`, tag `page-rcN` | shell.socialweb.computer | - releases: CI `promote` writes the rc's `software.release.*` records and moves channel `rc`, then `stable` after the prod gate; no git tag is built ([deploy.md](deploy.md), [protocol/release.md](protocol/release.md)) - gateway: `scripts/deploy.sh` (ship, `upgrade --binary --sha256`, `install gateway` again, caddy sites), run by `scripts/promote.sh` for the droplets. host `socialweb.computer`, ssh port 1997, account `socialweb` - bidder: follows its channel and upgrades itself (`internal/releasefollow`); by hand: stage binary, `upgrade --binary --sha256 ` as `swc-bidder` - upgrade: switches `bin/current`, keeps `bin/previous`, core restarts and adopts every unit by name + InvocationID. VMs, state, OpenBao, DID, association survive - rollback: `socialweb-computer rollback` swaps the two binary slots - fresh install keeps nothing: new DID, new association - ingress relay: `scripts/deploy-xrpc-relay.sh`; atproto relay: `scripts/deploy-atproto-relay.sh` (both health-check + auto rollback). never run `atproto-reverse-proxy/scripts/deploy.sh` (prints a token) - before a rollout restarts bidders: tell testers, wait for a gap - a new required OAuth scope forces re-sign-in of OAuth identities: line it up first ### Burned schema revisions - APIResourceSchema spec is immutable in kcp; a type change needs a Revision bump in `api/*/v1alpha1/resources.go` - `deploy/apiresourceschema-revisions.txt` locks name -> sha256 forever - a revision once applied to prod with one spec is burned: reusing its number with another spec (e.g. a reverted branch's build) makes kcp refuse the schema -> core crash loop - fix: roll back, bump to a fresh revision, roll forward. details: [KCP.md](KCP.md#schema-revisions) ## Code Map | path | holds | |---|---| | `cmd/socialweb-computer` | the one binary: `run`, `install`, `upgrade`, `rollback`, `dev`, `units`, `serve`, `internal ...` | | `cmd/socialweb-guest-agent`, `cmd/socialweb-vmexec`, `cmd/socialweb-node` | guest agent, VM exec + fence, node tools | | `cmd/socialweb-apigen` | schema/export generator | | `api/` | kcp API types (see [KCP.md](KCP.md#api-resource-catalog)) | | `internal/app/` | five composition roots + `compose` | | `internal/controllers/` | one reconciler per kind | | `internal/services/` | component XRPC/HTTP adapters | | `internal/market/{bidder,requester,participant}` | market logic | | `internal/atproto/{cloudinit,ingress,relay,market}` | user_data, ingress relay, relay, market records | | `internal/bus`, `internal/component` | contract handles + ComponentLinks; component runtime | | `internal/stack/{componentcore,gatewaykcp,gatewayssh,bidder,request}` | core wiring per profile | | `internal/kcpserver`, `internal/kcpdatastore`, `internal/apiserverfront` | kcp slots, kine, fronts | | `internal/policyengine` | policy engine + bundled actions | | `internal/guestnet`, `internal/guestseed`, `internal/guestagent` | fence, guest binaries, guest agent | | `pkg/embedded` | in-process requester for other modules (iOS) | | `deploy/` | generated schemas/exports, sets, app manifest sets, market network | | `lexicons/` | atproto lexicons | | `rust/` | socialweb-meshd | ## See Also - [KCP.md](KCP.md), [protocol/README.md](protocol/README.md), [ARCHITECTURE.md](ARCHITECTURE.md), [bidder.md](bidder.md), [deploy.md](deploy.md), [gateway.md](gateway.md) - [../deploy/market-network/README.md](../deploy/market-network/README.md) - [acceptance/](acceptance/), [ATIRC-WORKFLOW.md](ATIRC-WORKFLOW.md)