~ knotless Search/
↑↓ move↵ open⇧↵ all results

socialweb.computer/socialweb-computer-kcp

socialweb.computer: a compute market on atproto (kcp)

1 collaborator
0
0
581 lines · 39.4 KB Raw History

SYSTEM: socialweb-computer, the whole system

  • one page map of every component, repo, host and flow
  • deep dives: KCP.md (kcp, workspaces, API catalog), protocol/README.md (every lexicon, its use and its step in the sequence), ARCHITECTURE.md (decisions, invariants; open by heading)
  • rules for agents: ../AGENTS.md
  • no history here. what was done when: git log, docs/plans/*.yaml

What It Is

socialweb-computer is a compute market on atproto. A requester posts a signed RFP record to its own atproto repo; bidders (hosts that run firecracker VMs) see it on the firehose, bid with records in their own repos, the requester accepts one bid, and the winner boots a VM from cloud-init user_data. The requester reaches the guest over iroh (plus opt-in opentunnel https and webrtc). On top of the market sit products: ssh into a fresh VM as yourself (ssh <handle>@socialweb.computer), tenant Kubernetes clusters that scale nodes from zero by renting market VMs, a browser page and an iPhone app that run a requester in process, and atirc agents that join a chat channel from inside a VM. Every application is a kcp of its own (or a workspace of one shared kcp in a market network); state is kcp objects; secrets live only in the application's own OpenBao.

Repos

repoupstreamrolewhere it runs
socialweb-computer-kcphttps://github.com/publicdomainrelay/socialweb-computer-kcpthe monorepo: Go core socialweb-computer, guest agent, vmexec, apigen, lexicons, manifests, policy actions, socialweb-meshd (rust)gateway host, every bidder, CLI users, guests (guest tools), wasm page and iOS app (as a library)
kcp forkhttps://github.com/johnandersen777/kcpkcp built for js/wasm with memory storage; builds the browser page (socialweb-wasm/, page-rcN tags)shell.socialweb.computer (browser)
kcp (native)https://github.com/johnandersen777/kcp at d71f020c (tag swc-apibinding-requeue-1: kcp-dev/kcp main 74e9fb9c plus one commit)native kcp asset, built from that fork and pinned in internal/versions/versions.jsongateway, bidders, CLI requesters, market networks
indigohttps://github.com/bluesky-social/indigoatproto Go libs (repo, MST, xrpc, identity), Go module dependency; local checkout for readinglinked into socialweb-computer
atircgit@tangled.org:did:plc:vtad4w7ujsvz7t6o6iuvmhjqatproto IRC: channels, subhandle tokens (mint, revoke, lease), agents, web chat, send/recv pipeirc.fedfork.com server; atirc CLI in guests and on operator hosts
AT-IRC iOS applocal only (atirc-client-ios)iPhone chat client; embeds pkg/embedded (in-process kcp + requester) to launch VM agentsiPhone
did-key-associatorlocal only (frozen TS, copy also in monorepo history)qr.fedfork.com: OAuth QR session-transfer page for headless identities (oauthQr)qr.fedfork.com
atproto-reverse-proxyhttps://github.com/publicdomainrelay/atproto-reverse-proxyfedproxy.com ingress: legacy deno relay, ssh relay, caddy; the Go ingress relay from this monorepo runs as a drop-infedproxy.com
typescript-helpersthis repo, typescript-helpers/ (the GitHub repo is private since 2026-11-20)frozen TS helpers (event streams, abc, cli-args-env); reference onlynowhere live

Hosts And Roles

hostrolerunspublic names
socialweb.computergateway hostGateway app (kcp 1): GatewayAPI web app, cluster provider, tenant k3s control planes, k8s requester; SshGateway app (kcp 2): Go ssh server on :22, per-user Requesters; caddy 443; host sshd on 1997k8s.socialweb.computer, kcp.socialweb.computer, ssh.socialweb.computer
hat-8"local" bidder + operator/lead hostMarketBidder (account swc-bidder), dev market networks, builds, deploy scriptsnone (requestCrawl to relay)
hat-7bidderMarketBidder (bob, oauthQr identity)none
hat-9bidderMarketBiddernone
irc.fedfork.comatircatirc server, atirc-iroh.service, /chat web clientirc.fedfork.com, did:web:irc.fedfork.com#atirc
fedproxy.comdid:key xrpc ingressGo internal ingress-relay (caddy drop-in 30-go.conf of xrpc-relay-server.service), ssh relay :2222xrpc.fedproxy.com, *.fedproxy.com
relay.mini-cloud-0002.chadig.comatproto relayGo internal atproto-relay (127.0.0.1:2584 behind caddy)firehose source, bidders requestCrawl here
qr.fedfork.comOAuth QRdid-key-associator backendqr.fedfork.com
shell.socialweb.computerstatic pagewasm kcp + requester in the browser tabshell.socialweb.computer
iPhoneclientAT-IRC app, embedded requesternone
  • every listener behind the edge binds port 0 and reports its address into kcp status
  • only stable public entry points keep fixed ports: caddy 443, gateway kcp front 127.0.0.1:6443, GatewayAPI web-app front 127.0.0.1:8788, ssh 22
  • prod releases come from CI promote: it moves channel rc, deploys the socialweb.computer droplets, and the rc bidders upgrade themselves by following the channel (deploy.md, protocol/release.md)

System Context

flowchart LR
  subgraph USERS["Users"]
    SSHC["ssh client"]
    CLI["socialweb-computer run request"]
    PAGE["browser page shell.socialweb.computer"]
    IOS["AT-IRC iOS app"]
    KUBE["kubectl / XRPC k8s client"]
  end

  subgraph GWH["socialweb.computer"]
    GW["Gateway app: GatewayAPI, cluster provider, tenant k3s"]
    SG["SshGateway app: SshServer :22, per-user Requesters"]
    BAO["OpenBao: OAuth sessions, keys"]
  end

  subgraph BID["Bidders: hat-8, hat-7, hat-9"]
    MB["MarketBidder: Bidder, GuestControlPlane, Pds, PolicyEngine"]
    VM["firecracker guests"]
  end

  subgraph AT["atproto network"]
    PDS["PDSes: user repos"]
    RELAY["relay.mini-cloud-0002.chadig.com"]
    JS["Jetstream us-east, us-west"]
    PLC["plc.directory"]
    ING["xrpc.fedproxy.com ingress"]
    QR["qr.fedfork.com"]
  end

  ATIRC["irc.fedfork.com atirc"]

  SSHC -->|"ssh handle@"| SG
  KUBE -->|"service auth XRPC"| GW
  CLI -->|"records + submitRfp"| PDS
  PAGE -->|"records"| PDS
  IOS -->|"records"| PDS
  SG --> BAO
  GW --> BAO
  SG -->|"rfp, accept records"| PDS
  GW -->|"rfp, accept records"| PDS
  MB -->|"bid, receipt, events"| PDS
  PDS --> RELAY
  RELAY -->|"firehose"| MB
  RELAY -->|"firehose"| SG
  JS -->|"jetstream"| MB
  ING <-->|"submitBid, submitEvent"| MB
  ING <-->|"did:key callbacks"| SG
  MB --> VM
  SG -->|"ssh over iroh"| VM
  VM -->|"atirc agent"| ATIRC
  SG -->|"mint, revoke"| ATIRC
  IOS -->|"chat"| ATIRC
  MB -->|"oauthQr sign-in"| QR
  PDS --- PLC

Applications

  • one root object per application, in its own kcp (prod) or its own workspace root:<app> (market network)
  • the core (socialweb-computer process) runs the root's composition controller (internal/app/<root>), every controller, the component runtime and the HostProcess supervisor
  • components run in process; third-party binaries (kcp, kine, OpenBao, dumbpipe, firecracker via vmexec, deno for policy) are HostProcess units
root kindCLIcomposes (main children)role
MarketBidderrun bidder, install bidderBidder, GuestControlPlane, Pds (localRepo mode) or RemotePDS (OAuth), PolicyEngine, OpenBaoServer, IngressProxy, MarketIdentity, EventStreams, ComputeVM/FirecrackerVM (per contract)sells VMs
MarketRequesterrun requestRequester, PolicyEngine, OpenBaoServer, Pds (localRepo) or OAuth identity, IngressProxy, MarketIdentity, EventStreams, ComputeContractrents VMs; one fresh kcp per invocation, state dir persists
Gatewayserve gateway kcp, install gatewayGatewayAPI, Requester, PolicyEngine, OpenBaoServer, cluster provider, MarketIdentity (alice requester), Infra per clusterhosted tenant k8s clusters
SshGatewayrun gateway sshSshServer, per-DID Requesters, PolicyEngine, OpenBaoServer ssh-gateway, iroh dialer HostProcess, session-owner MarketIdentities, Exposuresssh as yourself into a fresh VM
MarketNetworkdev up --profile market-networkPlc, Relay, Pds alice/bob, IngressProxy dispatcher, DispatcherTLS, PolicyEngine, OpenBaoServer network (PKI), child roots (bidders, gateway, requesters, ssh gateways)dev/CI/soak: a whole local market in one kcp

Components (in-process kinds)

componentpackagedoes
Bidderinternal/services/bidder, internal/market/bidderreads RFPs, runs policy, bids (bids.free), takes accepts, writes receipts, creates ComputeVMs, publishes vm events
GuestControlPlaneinternal/services/guestcontrolplaneguest-facing endpoints at http://192.0.2.1:<port>: OpenBao allow-list proxy (workload identity), POST /v1/events, GET /v1/downloads/<sha256>/<name>, compute.guest.subscribeEvents
Requesterinternal/services/requester, internal/market/requesterruns each ComputeContract: RFP, bid window, accept, receipt, network, secrets, atirc mint/revoke, release
Pdsinternal/services/pds, internal/pds/locala local PDS for an identity without a real account (localRepo); spec.accountCreation Closed (default): only the application's own code makes accounts on it, XRPC createAccount answers 400 InvalidInviteCode; Open on a market network's Pds
Plc, Relayinternal/services/plc, internal/services/relaynetwork-local PLC and relay (market network)
IngressProxyinternal/services/ingressproxydid:key ingress to the app's XRPC (xrpc-subscription, iroh or opentunnel provider); default host xrpc.fedproxy.com
DispatcherTLSinternal/services/dispatchertlsTLS front of the market network dispatcher
GatewayAPIinternal/services/gatewayapi, internal/gatewayhttpk8s.socialweb.computer: /oauth/*, /session, /api/*, /xrpc/computer.socialweb.k8s.*, /downloads/*
SshServerinternal/services/sshserverssh on :22, web sign-in at /, /keys, session owners, per-session contracts
PolicyEngineinternal/policyenginegha-lite workflows in embedded deno; bundled actions bundled-actions/tangy/

Market Protocol

  • lexicons: lexicons/ (NSID prefix com.publicdomainrelay.temp.)
  • every lexicon, field by field, with its sequence step: protocol/README.md
  • applications talk ONLY over this protocol. a shared kcp is never a data channel

Records

recordwritten byin repo ofpurpose
compute.vmrequesterrequesterthe VM asked for (size, role)
market.rfprequesterrequestersigned request for proposals; strongRefs vm payload + policy; callbacks submitBid/submitEvent
market.offering, market.heartbeatbidderbidderwhat the bidder sells; liveness. offering at rkey self
market.bid + market.bids.free / bids.x402bidderbidderbid envelope + payload. rkey market.BidRKey (one bid per RFP)
market.acceptrequesterrequesterthe chosen bid (no payload record)
market.receiptbidderbidderbidder took the accept; payload = the RFP compute.vm ref
market.event + compute.events.vm.*, compute.events.guest.*bothsenderlifecycle: vm.hostKeys, vm.onNetwork, vm.delete, vm.terminated, guest.*
market.association, link.ident.endorsementprincipal (operator, renter, R2)principalthe grant + the ident.link endorsement of an agent, rkey = agent DID (design/ident-link-associations.md)
market.associationRequestguest requester R2R2{principal, contract, nonce}, rkey = principal DID
badgeBlueKeyssigner / usersigner / userkey bindings: signer key binding, ssh key claims
sh.tangled.graph.vouchoperators, usersvouchertrust edge read by tangled-vouch policies
computer.socialweb.infra, computer.socialweb.ephemeralInfrathe owner's kcp (infra controller: CLI, gateway, page, iOS)ownerthe record of an Infra / EphemeralInfra object (two-way sync); per-account index at rkey self; never deleted (protocol/infra.md)

XRPC

methodserved bycaller
market.submitRfpbidder ingressrequester (push path); 409 RFPClosed after the window
market.submitBidrequester ingressbidder
market.submitAcceptbidder ingressrequester
market.submitEventboth ingresseseither side
compute.secrets.getSecrets/<contract>requester ingressguest setup-secrets.service (workload identity token)
compute.guest.subscribeEventsGuestControlPlaneguest agent (WebSocket, cursor, 25 s keepalive)
computer.socialweb.k8s.*GatewayAPIcluster owners (service auth, aud did:web:k8s.socialweb.computer)
com.fedfork.exec.ops.listVms, getVm, getBidder, getHealth, getLog, getVmMemorybidder ingress (its market endpoint: https where its offering has one, else iroh)the operator's checks (the prod gate in CI): Authorization: Bearer <ops read secret>
com.fedfork.exec.ops.getRequesterLogSshServer web originthe same

Read-only operations routes (owner, 2026-11-18; lexicons lexicons/com/fedfork/exec/ops/, contract internal/contract/opsread, routes internal/services/opsread, host reads internal/opsread/hostread, kcp reads internal/controllers/opsread). GET only; each answers one fixed shape, takes no selector, path or shell text, and writes nothing. They exist only when the host holds an ops read secret: a file of mode 0600, <instance state dir>/ops-read-secret, or for a bidder the path of run bidder --ops-read-secret-file (install bidder writes it to config.json); no file or an empty one: 404. The secret is compared in constant time (sha256 of both sides), never logged, never in a kcp object; a wrong or missing bearer is 401, a POST 405.

A bidder's routes are on its market mux, so they are reached over its market endpoint, the one its market.offering/self record names: its https endpoint where the offering has one (staging), else its iroh endpoint (prod bidders are behind NAT and offer iroh and webrtc only). Over iroh the test host (tests/host/src/ops.rs) sends the same GET and Authorization: Bearer as HTTP/1.1 through atlibs-iroh's raw fetch (iroh://<endpoint ticket>/<path>: dumbpipe connect <ticket>, the stream a requester's market calls take to that listener), end to end encrypted to the iroh node the ticket names. What binds the ticket to the bidder: the offering record in the bidder's own repo on its PDS, the PDS resolved from the bidder DID's PLC document, the DID from deploy/release/rc-operators.json (scripts/ci-prod-target.sh). A URL is never http:// (the secret goes to https origins or an iroh endpoint ticket only). The gateway's getRequesterLog is on its https web origin.

routeparamsanswer
listVmsnone{items: [{metadata {name, deletionTimestamp}, spec {vm.cpus, requester.did, contract.receipt.uri}, status {phase, lostReason}}]}: the bidder's ComputeVMs, every tenant's
getVmname (a DNS label){computeVm (as in listVms), firecrackerVm {metadata {name, deletionTimestamp}, status {phase, lostReason, image {sha256, fetched}}}}, each null when absent; no work dir, socket, pid, tap, MAC or address
getBiddernone{identity {metadata, status {phase, pendingAction {kind, reason}, conditions [{type, status, reason}]}}, bidder {metadata, spec {policy}}, marketBidder {metadata, status {phase, did}}}: never the pending action's URL, a session ref, associations, declines, listeners or disk usage
getHealthnone{unit, started, errors, startupErrors, restarts, vmProcesses, errorSample[<=3] {time, msg, err<=160}, guestImageReadyAt, guestImageUnready} of the core since its process started (startup: its first 10 min; restarts: the unit's NRestarts)
getLogmsg (one of the names in opsread.Messages), subject (optional: a ComputeVM name, vm-<name> or an at:// URI){log, count, lines[<=50]}: lines since the process started holding that message's texts and the subject; a line is time, msg and the fixed keys of opsread.LineKeys
getVmMemoryname{unit, current, high, max, swapMax, events} of the VM unit's memory cgroup
getRequesterLogdid, since (RFC 3339){lines}: the ssh gateway core's lines of that DID since then whose msg is one of opsread.RequesterMessages, as time, msg, subhandle, contract, receipt, deleted, of, reason, accept

The log is the core's own: its stderr file when it has one (a dev instance, the ssh gateway: logs/core.log, every logger of the process writes there). Else, once the secret is configured, the process's one log handler (internal/kit/logx Tee, under every logger the CLI hands out: the components', the controllers', the compute stack's) also writes each line to <instance>/logs/core.log from devcore.Run on, and the routes read that (an installed account cannot read its own journal entries); logcap rotates it to core.log.1 at 64 MiB and the read takes .1 then the file. Else the journal of its uid. A failed read answers 502 with a fixed message; the detail is in the host's log.

Comms modes

  • spec.comms.mode: XRPC (ingress only), Firehose (no inbound endpoint), Both (requester default)
  • event sources (Firehose/Both): network relay or relay.mini-cloud-0002.chadig.com, Jetstream v2 us-east/us-west
  • server-side filter: wantedCollections, wantedDids; client filter always runs too
  • requester stream carries market.bid, market.receipt, market.event; bidder stream market.rfp, market.accept, market.event
  • payload records are read through the strongRef (getRecord), never taken as stream input
  • dedup: one sink per role drops a uri|cid seen in the last 15 s
  • addressee index: requester keeps only records about its own ComputeContracts; bidder only RFPs, accepts of its bids, events of its receipts. rest counted as market_foreign_dropped

Selection

  • the bidder's bid window: 10 min from RFP createdAt (market.RFPBidWindow); the requester's: bidWindowSeconds (30 s)
  • bidder bids once per RFP (Bid object name = rkey)
  • requester picks, recorded in status.chosen.reason:
    • firstFree: first scope-policy-allowed free bid ends the window (the normal case; fastest bidder wins)
    • policy: else the lowest cost after the window, confirmed by the RFP's attached policy
    • bestWithinWindow and resumed are in the enum; no code path leaves them written
  • status.bids keeps at most 20 {did, uri, cost, verdict, arrivedAt} (offer is in the type, never set)
  • the whole flow with its failure branches: protocol/flow.md

Policies

  • a market policy ALWAYS runs, both sides, both lanes (scope + full), every decision. no skip paths
  • defaults: bidder-tangled-vouch (bidder), requester-tangled-vouch (requester). open only when named, never prod
  • tangled-vouch: counterparty must be vouched (sh.tangled.graph.vouch) by the operator; both directions needed for a deal
  • bidder full lane = own workflow + every policy the RFP attaches; allow only if all allow
  • caching lives inside the engine run (cache-key over workflow, mode, perspective, DIDs, records, args). never across parties, never a skip
  • missing engine, evaluator or non-boolean allow = deny

Market sequence

sequenceDiagram
  autonumber
  participant R as Requester
  participant RP as requester PDS
  participant FH as relay / jetstream
  participant B as Bidder
  participant BP as bidder PDS
  participant VM as guest VM

  R->>RP: putRecord policy record, compute.vm, market.rfp
  R->>B: submitRfp
  RP->>FH: commit
  FH->>B: market.rfp
  B->>RP: getRecord rfp payload + policy
  B->>B: bidder-tangled-vouch (scope + full)
  B->>BP: market.bid + bids.free
  B->>R: submitBid (ingress)
  R->>R: requester-tangled-vouch, firstFree
  R->>RP: market.accept
  R->>B: submitAccept
  B->>B: capacity check + reservation
  B->>BP: market.receipt
  B->>VM: ComputeVM to FirecrackerVM, boot user_data
  VM->>B: workload login binds its ssh host key
  VM->>B: POST /v1/events vm.onNetwork (iroh ticket)
  B->>BP: market.event vm.hostKeys, vm.onNetwork
  B->>R: submitEvent
  R->>R: ComputeContract Held
  R->>VM: ssh / tunnels over iroh
  R->>RP: market.event vm.delete
  R->>B: submitEvent vm.delete
  B->>VM: stop, remove
  B->>BP: market.event vm.terminated
  R->>RP: delete own contract records
  R->>R: ComputeContract Released

ComputeContract phases

  • Pending (only while admission holds it back) -> Requesting -> Held -> Releasing -> Released; or Failed (a Failed contract is still released); a vm.terminated can settle Requesting or Held straight to Released
  • status.contract.phase: requested, rfp_submitted, winner_selected, accepted, receipt, network, released
  • status.contract: strongRefs rfp/bid/accept/receipt, winner, vmAddress (iroh), hostKeys, networks, release, termination, timestamps
  • every write lands before the flow moves on; a restarted runner resumes from the recorded stage
  • release: spec.releaseRequestedAt (SSA, release manager), outcome in status.release.requestedAt; the vm.delete submitEvent is retried 5 s doubling to 5 min until the release deadline (an ssh session's release: 6 tries, 1 s doubling to 30 s)
  • a release whose vm.delete keeps failing settles Released once the bidder's vm.terminated appears

Guests

Boot

  • the RFP flow is the ONLY way a guest comes up. no hand-started VM, no agent install outside cloud-init
  • user_data built by BuildUserData + module registry (internal/atproto/cloudinit)
  • requester modules carry config only; bidder adds setup-guest-binaries.service (internal/guestseed)
  • binaries come from the bidder's GuestControlPlane GET /v1/downloads/<sha256>/<name>, sha256 checked, installed to /usr/local/bin. never GitHub/GHCR from a guest
  • downloads = release asset socialweb-guest-downloads: opentunnel, wootty, atirc, socialweb-guest-agent
  • opt-in spec.guestTools: [socialweb-computer]: bidder serves its own binary (linux/amd64 only)
  • guest image: firecracker-x86_64 (or firecracker-aarch64), pinned by sha256 in internal/versions/versions.json, from guest-images/<target>/<sha256> in the public bucket
flowchart TD
  A["Bidder: accept taken, receipt written"] --> B["ComputeVM created, owned by Bid"]
  B --> C["FirecrackerVM: vmruntime Prepare"]
  C --> D["socialweb-vmexec loads nft table swcguest"]
  D --> E{"fence loaded?"}
  E -->|"no"| X["VM does not start"]
  E -->|"yes"| F["tap + pasta, firecracker boots"]
  F --> G["cloud-init runs user_data"]
  G --> H["setup-guest-binaries.service: fetch by sha256 from 192.0.2.1"]
  G --> I["iroh module: endpoint up"]
  I --> J["POST /v1/events vm.onNetwork: iroh ticket, host keys"]
  J --> K["Bidder signs + publishes market.event"]
  K --> L["Requester: contract Held"]
  G -->|"comms XRPC or Both"| M["setup-secrets.service: getSecrets"]
  G -->|"comms Firehose + Sealed"| N["guest agent: guest.secretsRequested, sealed bundle"]
  M --> O["secrets.ready"]
  N --> O
  O --> P["units After= secrets: wootty, atirc-agent env"]

Transports (three, never add one)

transportdefaulthow
irohrequired, every guestrequester ssh + status.contract.vmAddress; contract Held on iroh address + host keys
opentunnelopt-in per spec.tunnels[]https URL per tag to guest 127.0.0.1:<port>; vm.onNetwork kind https; optional wootty shell
webrtcopt-in per spec.tunnels[]guest agent bridges datachannel tag to 127.0.0.1:<port>; STUN stun.cloudflare.com:3478, no TURN; run request proxy, ssh --transport webrtc
  • signaling sealed: age-x25519 guest.publicKey per tag, offer/answer sealed. no key, SDP, ICE or IP in any record, object or log
  • browser shell: ssh over webrtc first, wootty fallback

Fence

  • nf_tables bridge table swcguest loaded before the tap exists (internal/guestnet)
  • allowed from guest: TCP to 192.0.2.1 only on ComputeVM spec.hostPorts; DNS to pasta gateway; ICMPv6 RD/ND
  • dropped: host's own addresses, guest link, RFC1918, CGNAT, loopback, link-local, multicast, ULA; every ethertype but IPv4/IPv6/ARP
  • lanUDP (default on): UDP 1024+ to LAN (not 1900/5353/5355) for same-LAN webrtc
  • hostUDP (default only with policy bidder-only-me): UDP to host's own addresses
  • forwarded webrtc UDP port per ComputeVM from webrtcUDPPorts (default 31000-31999)
  • every guest (also with lanUDP off): UDP to host's own addresses on destination ports in webrtcUDPPorts (default 31000-31999): any host UDP socket bound there (other guests' forwards, the bidder's bidder-mesh)
  • TCP to LAN or to host addresses: never

Secrets

  • spec.secretsRef: bundle [{path, value, mode}] in the requester's OpenBao
  • primary: getSecrets on requester ingress, auth = guest workload identity token (bidder OpenBao identity engine via GuestControlPlane proxy, GuestIdentityGrant)
  • sealed secretsFallback: Sealed: only in comms mode Firehose (no inbound endpoint), on demand, after guest.secretsRequested; sealed to the guest's key as ContractEvent secrets-<contract>; with an inbound endpoint never
  • never send a sealed bundle unasked
  • condition SecretsApplied reason GetSecrets or SealedFallback
  • guest events: guest POSTs to GuestControlPlane, bidder signs + publishes; bidder never opens a sealed body

Identity And Auth

Where identity lives

  • OAuth sessions, relay keypairs, OIDC signing key: ONLY in the app's own OpenBao
  • signing keys are 0600 files in the app's state dir: the attestation keys (attestation.jwk, attestation-key.json), repo-key.json, the ingress market-relay.jwk, the Pds repo-keys/ and the PLC plc-rotation.key (protocol/identity.md)
  • kcp objects hold references (SecretRef), never values
  • the callee holds credentials; a bus handle never exposes a token
  • atproto OAuth sessions held by the OpenBao atproto auth plugin (auth/atproto session/<did>); refresh in the plugin
  • plugin keeps a session on transient refresh errors (invalid_client*, use_dpop_nonce, 429, 5xx, unreachable); expires on invalid_grant etc.

MarketIdentity modes

moderepo lives insign-inused by
localRepoapp's own Pds componentnone (account created at install)default bidder, dev requester
pdsAccounta PDS account (password in vault)app passwordmarket network accounts
oauthQruser's PDSQR session-transfer at qr.fedfork.com (status.pendingAction.url, 5 min)headless bidder with a real account (hat-7)
oauthBrowseruser's PDSauthorize URL in status.pendingActionCLI requester, browser page
oauthDelegateduser's PDSsession handed in by another appdelegated flows
oauthSessionProxyuser's PDSreads a session another component stored in the vaultssh gateway session owners, gateway cluster owners
  • scope upgrade: optional new scope (e.g. rpc:chat.atirc.revoke) -> pendingAction {kind: signInAgain, reason: ScopeUpgrade}, identity stays Ready
  • a required new scope = SessionExpired for OAuth identities on rollout: plan the re-sign-in first
  • granted scopes are read from the token response, never the access token's scope claim
  • sign-ins are scripted (scripts/oauth-signin.ts, headless Chrome). no sign-in link handed to a person

Associations

  • ident.link pair + grant: agent DID doc https://ident.link/agent-of/<principal DID>; principal repo link.ident.endorsement + market.association at rkey = agent DID (design/ident-link-associations.md)
  • the identlink walk (internal/identlink, sync.getRecord, CAR verified) decides; MarketIdentity status.associations reports the roots (plus spec.approvedAssociations, unwalked). no manual patch
  • standing (bidder hosts, gateway requester): written by the app with the operator's session, or scripts/associate-bidder.ts --agent-did <did>
  • contract-scoped (R2, co-located accounts, VM agents): grant {agent, contract, expiresAt}, after the R2 nonce check

Service auth

  • atproto service auth JWT (lxm + aud) for every cross-service call: from getServiceAuth of the identity's PDS session, or self-signed with repo-key.json by a localRepo requester; the ingress relay registration is a self-signed did:key token:
    • k8s API: aud did:web:k8s.socialweb.computer, lxm computer.socialweb.k8s.*
    • atirc: aud did:web:irc.fedfork.com#atirc, lxm chat.atirc.mint / chat.atirc.revoke
  • did:key ingress (IngressProxy) carries submitBid/submitEvent callbacks for apps without a public host
  • ingress registration is bound to the key: getRegistrationNonce answers 403 IssuerMismatch unless the token's iss is the key asked for; subscribe answers 403 IssuerMismatch before the upgrade unless the token's iss is the did param; only a signature whose key is the registration key counts (no signature by the registered key); #renew keeps its issuer check. A new subscription of the same key replaces the route and closes the old socket with 4009 (replaced by a newer subscription of the same key)

ssh key claims

  • badgeBlueKeys at a stable keyRkey in the user's own repo, written via ssh.socialweb.computer /keys
  • ssh authorizer: one getRecord per offered key. no listing

SSH Gateway

  • ssh <handle>@socialweb.computer <cmd> -> a VM rented AS that user, from that user's repo
  • sign in once at https://ssh.socialweb.computer (OAuth, session in OpenBao ssh-gateway)
  • admission: policy bidder-tangled-vouch scope lane, self = spec.ssh.operatorDid. refused = signed out, "this account was not admitted"
  • admitted -> session owner MarketIdentity owner-<stamp(did)> (oauthSessionProxy) -> Requester requester-<stamp(did)>
  • cap spec.ssh.maxAccounts (default 64)
  • per-DID requester has spec.contractOwnerDid: runs only that DID's ComputeContracts
  • every session = a ComputeContract: transient ssh-run-<hash> (TTL 1 h, released at command end unless LC_KEEP_VM) or held LC_SESSION ssh-<hash> (idle 5 min)
  • disconnect (client close, 3 missed 15 s keepalives) -> cancel command -> revoke atirc token -> release contract
  • restart: first sync releases stale ssh-run- contracts, adopts held sessions still alive
sequenceDiagram
  autonumber
  participant U as ssh client
  participant W as ssh.socialweb.computer
  participant V as OpenBao ssh-gateway
  participant S as SshServer :22
  participant P as user PDS
  participant R as Requester for DID
  participant B as Bidders
  participant VM as guest
  participant A as atirc

  U->>W: OAuth sign-in (once)
  W->>W: policy bidder-tangled-vouch (operator)
  W->>V: store session at auth/atproto session/DID
  W->>R: EnsureSessionOwner, compose Requester
  U->>S: ssh HANDLE@socialweb.computer CMD
  S->>P: getRecord badgeBlueKeys at keyRkey
  P-->>S: key bound to DID
  S->>R: session as ComputeContract
  R->>V: session via vault (oauthSessionProxy)
  R->>P: compute.vm + market.rfp
  B-->>R: bids
  R->>R: requester-tangled-vouch, firstFree
  R->>B: accept
  B->>VM: boot, vm.onNetwork
  R->>A: mint child token (if atirc cmd), leased
  S->>VM: ssh hop over iroh dialer, run as agent user
  U--xS: close / drop
  S->>A: revoke token
  S->>R: release contract
  B->>VM: terminate, vm.terminated

atirc Integration

  • atirc = atproto IRC at irc.fedfork.com; identities are subhandles <sub>.<handle>
  • tokens: chat.atirc.mint with explicit exp (max 24 h), chat.atirc.revoke; revoking a subhandle revokes everything below it
  • lease: a token minted for a contract is leased on the receipt + the bidder's vm.terminated at-uri. dies at vm.terminated even if revoke missed

In a contract (spec.agent)

  • requester mints after the receipt is verified, writes ATIRC_TOKEN into the bundle at /etc/atirc/atirc.env
  • user_data installs atirc-agent.service, never starts it; atirc-agent-env.path try-restarts on env change
  • guest.agentStart (requester event) -> guest agent starts the unit
  • run request --agent, run request agent <name>

In an ssh session

  • command starting atirc (or LC_ATIRC=1) -> agent module in user_data; requester mints child ssh-<rfp rkey>.<handle>
  • token reaches the command only via ssh env LC_ATIRC_TOKEN -> process env. never argv, never a file
  • atirc agent run <sub> -> agent joins as <sub>.<handle>; mint failure fails the command
  • runs as the agent user (named by atirc nick, wheel, NOPASSWD), not root (claude refuses root)

Revoke before release

  • order: revoke leased token BEFORE vm.delete, while the lease is alive; 401 "lease not alive" counts as revoked
  • every mint recorded under atircMinted in the bundle secret; background revoke with backoff (cap 5 min) until revoked or exp
  • status.atircRevokes: Done / Pending / Abandoned (24 h)
  • run request release waits up to 15 min (RequestRevokeWait) for revokes, exits non-zero on timeout and keeps state

Guest requester (R2)

  • spec.association {fromGuest}: guest runs its own requester (run request --guest-requester) as the agent user
  • requester mints the R2 child token r2-<accept rkey>.<handle> into /etc/socialweb-computer/r2-atirc.env
  • R2 mints its own agents below that subhandle (spec.atircParentRef)
  • release cascades: guest.r2Release -> guest requester releases its contracts -> guest.r2Released

Clients

  • atirc sub / atirc pub (agents, operators), atirc send / recv pipe (chat.atirc.pipe)
  • web chat /chat, AT-IRC iOS app (also launches VM agents through its embedded requester)

Tenant Kubernetes Clusters (Gateway)

  • the one traditional kcp use: tenant k3s --disable-agent control planes behind kcp
  • per cluster: k3s over kine over its own postgres, cluster OpenBao (identity bundle), External Secrets pair, two ClusterLinks (dumbpipe iroh in front of apiserver and vault)
  • blue/green control-plane slots a/b behind a per-cluster front (controlplane.json)
  • scale from zero: a pod -> NodeClaim -> one entry of the cluster's Infra nodes-<hex> -> ComputeContract infra-nodes-<hex>-node-<hex> -> RFP flow -> VM boots as k3s node, joins over iroh
  • delete the pod -> claim deleted -> contract released -> VM gone
  • node network: --flannel-backend=none, bridge CNI, socialweb-meshd (rust) cross-node, CoreDNS per node
  • nodes published to the owner's repo: computer.socialweb.infra/nodes-<hex> (no secrets, no user data)
  • API: computer.socialweb.k8s.kubectlApplyServerSide, getJob, subscribeClusters with service auth
flowchart LR
  POD["pod applied"] --> NC["NodeClaim (cluster home workspace)"]
  NC --> INF["Infra nodes-hex entry"]
  INF --> CC["ComputeContract"]
  CC --> RQ["gateway Requester: RFP flow"]
  RQ --> BD["Bidder: VM with k3s node user_data"]
  BD --> JOIN["node joins via ClusterLink iroh"]
  JOIN --> RUN["pod runs"]
  INF --> REC["owner repo: computer.socialweb.infra"]

Records Policy

  • never delete user records. no sweep, retention or GC of people's repo records
  • release deletes only the app's own contract-scoped records it wrote (e.g. its association)
  • record growth pain = fix the read, never the data
  • no listRecords on a collection that grows without bound; one getRecord at a deterministic rkey
  • stable rkeys:
    • bid: market.BidRKey per RFP
    • offering: self
    • infra index: computer.socialweb.infra/self
    • grant, endorsement: the agent DID
    • ssh key claim: keyRkey
  • listRecords remains only as a legacy fallback (e.g. a ComputeVM accepted before stable rkeys)
  • progress fields write at most once a minute (no churn commits)

Deploy And Release

artifacthow builtwhere installed
release binaryscripts/build-socialweb-computer.sh (clean checkout, every pinned asset, -tags "release embed_deno", CGO_ENABLED=0) -> ~/.tmp/socialweb-computer-build/release/<commit>/ + SHA256SUMSgateway, bidders
dev binary--dev --publish /var/tmp/<work>own market network only
guest imageCI or scripts/build-guest-image.sh, content-addressed in the bucketbidders fetch + verify
guest downloadsscripts/build-guest-downloads.shembedded, served by GuestControlPlane
browser pagekcp fork scripts/browser-dev/build-page.sh, tag page-rcNshell.socialweb.computer
  • releases: CI promote writes the rc's software.release.* records and moves channel rc, then stable after the prod gate; no git tag is built (deploy.md, protocol/release.md)
  • gateway: scripts/deploy.sh (ship, upgrade --binary --sha256, install gateway again, caddy sites), run by scripts/promote.sh for the droplets. host socialweb.computer, ssh port 1997, account socialweb
  • bidder: follows its channel and upgrades itself (internal/releasefollow); by hand: stage binary, upgrade --binary <path> --sha256 <sha> as swc-bidder
  • upgrade: switches bin/current, keeps bin/previous, core restarts and adopts every unit by name + InvocationID. VMs, state, OpenBao, DID, association survive
  • rollback: socialweb-computer rollback swaps the two binary slots
  • fresh install keeps nothing: new DID, new association
  • ingress relay: scripts/deploy-xrpc-relay.sh; atproto relay: scripts/deploy-atproto-relay.sh (both health-check + auto rollback). never run atproto-reverse-proxy/scripts/deploy.sh (prints a token)
  • before a rollout restarts bidders: tell testers, wait for a gap
  • a new required OAuth scope forces re-sign-in of OAuth identities: line it up first

Burned schema revisions

  • APIResourceSchema spec is immutable in kcp; a type change needs a Revision bump in api/*/v1alpha1/resources.go
  • deploy/apiresourceschema-revisions.txt locks name -> sha256 forever
  • a revision once applied to prod with one spec is burned: reusing its number with another spec (e.g. a reverted branch's build) makes kcp refuse the schema -> core crash loop
  • fix: roll back, bump to a fresh revision, roll forward. details: KCP.md

Code Map

pathholds
cmd/socialweb-computerthe one binary: run, install, upgrade, rollback, dev, units, serve, internal ...
cmd/socialweb-guest-agent, cmd/socialweb-vmexec, cmd/socialweb-nodeguest agent, VM exec + fence, node tools
cmd/socialweb-apigenschema/export generator
api/kcp API types (see KCP.md)
internal/app/<root>five composition roots + compose
internal/controllers/<kind>one reconciler per kind
internal/services/<svc>component XRPC/HTTP adapters
internal/market/{bidder,requester,participant}market logic
internal/atproto/{cloudinit,ingress,relay,market}user_data, ingress relay, relay, market records
internal/bus, internal/componentcontract handles + ComponentLinks; component runtime
internal/stack/{componentcore,gatewaykcp,gatewayssh,bidder,request}core wiring per profile
internal/kcpserver, internal/kcpdatastore, internal/apiserverfrontkcp slots, kine, fronts
internal/policyenginepolicy engine + bundled actions
internal/guestnet, internal/guestseed, internal/guestagentfence, guest binaries, guest agent
pkg/embeddedin-process requester for other modules (iOS)
deploy/generated schemas/exports, sets, app manifest sets, market network
lexicons/atproto lexicons
rust/socialweb-meshd

See Also