~ knotless Search/
↑↓ move↵ open⇧↵ all results

socialweb.computer/socialweb-computer-kcp

socialweb.computer: a compute market on atproto (kcp)

1 collaborator
0
0
307 lines · 32.7 KB Raw History

Acceptance 0013: Mixed prod (VM, Deno sandbox, agent VM on a user-configured bidder)

Proves ../../examples/infra/mixed-prod.yaml end to end against prod services: a long-lived requester (run requester, then install requester) with a real atproto account, kubectl apply of one EphemeralInfra pinned to a bidder the user configured, use each contract, restart the requester mid-run, delete. The default policy runs on both sides; no file and no command names open (../quickstart-prod.md sections 2, 5 and 7).

  • Paths: A run requester in the foreground (OAuth QR); B install requester (systemd user unit of its own account); C the gateway in the requester kcp (run requester --with-gateway, a tenant cluster)
  • Plan: 0039 items F-MIXED-PROD (this runbook) and ACC39-MIXED-PROD (its live pass)
  • Dev network, C (0013:c, composition, own 2-bidder network on hat-8, main aefab467 dev build + the OIDC discovery fix), 2026-11-15 00:57-01:08Z, alice: 3 passes, pod_ready_s 95.3 (the discovery race hit: the first fetch was rejected, the next one after the issuer was configured), 50.3, 45.1; nodes_zero_s 155.5, 155.5, 155.4; seconds 296.8, 235.4, 235.3. Before the fix, same net and binary: pod_ready_s 275.7 (5 min discovery cache), seconds 479.0
  • Last pass A, B, C: 2026-11-13 03:11-04:44Z on prod rollout 20 (v0.0.0-rc165-pre1 038bd32e, binary 24a8b530 for the CLI and install requester), bob, no pin (the kit's mixed-prod-nopin.yaml; winner hat-8 every time); consent pages qr.fedfork.com for @bobvmbuilder.bsky.social (screenshots read). A: ready 42 s after start, oauthQr; Deployed 31 s (vm1, sbx1, agent1 Held); sbx1 mixedprod-deno-ok sum=10; agent1 (bundle by A4b) association +32 s after agentStart, online and listening +141 s, uname -m answered x86_64 in 3.6 s; Ctrl-C and restart: ready 12 s, same DID, all Held, hostname answered 3 s; delete 10.8 s, R2 released first, requester_association_deleted, records 6/6, 10/10, 14/14, the 14 requester-repo URIs RecordNotFound x3 (the bidder's own market.receipt stays), 0 market.event / vm.delete / rfp / accept left (fix 13), R2 and agent DIDs plc_tombstone, vouches kept, hat-8 empty. A6 re-checked on a vm1-only EphemeralInfra (the first apply carried an old key from the kit file: Permission denied (publickey), a harness fault): iroh 4.4 s, webrtc 1.4 s, mixedprod-vm-ok x86_64, release 10/10, RecordNotFound x3. The restart logged 10 x Reconciler error the OpenBaoServer requester reports no address yet (phase Starting) at one instant (first start 0 ERROR). B (swc-requester, vm1 + sbx1): install exit 0 after the sign-in; ssh iroh 3.8 s, webrtc 1.3 s; systemctl --user restart: kcp, kine, front and bao units keep their InvocationID, same DID, ssh 0.98 s; delete 3 s, 8/8 RecordNotFound x3; uninstall --purge: no account, state, linger or install record (the installed requester's logs/ stayed empty and the journal had none of its lines, so B's ERROR count was not read). C: ready with the gateway 44 s after start, cluster c1 + kubeconfig 17 s, pod Ready 50 s after the apply (node *-c1-node1 on hat-8), quickstart-k8s-ok; delete Deployment: nodes 0 in 156 s, node records 6/6, self and nodes-<hex> mirrors gone, 0 ERROR
  • C also reads the index: after nodes 0 computer.socialweb.infra/self is the account's, not the run's (it holds every deployment of the DID, live or in recentEnded, and is deleted only when nothing is left in it), so C checks it apart from the run's records: it is gone, or neither its live nor its recentEnded names this run's nodes-<hex> and it is not empty; the run's own records, the nodes-<hex> mirror among them, must be RecordNotFound (2026-11-18, alice: self held 32 recentEnded entries of other deployments, so the old "self gone" read failed, as on 2026-11-14); not yet run with that check; the controller reconciles the index (infra_index_reconciled: entries whose record ended or is missing are dropped from live at start and on each index write) (a 2-bidder dev net 2026-11-14 never got a Ready quickstart-k8s pod in 15 min: the k8s node did not join there)
  • Previous pass C: 2026-11-12 21:12-21:19Z, kcp e190a3b1 (rollout 19, rc165 candidate): 0 ERROR, the infra self and nodes-<hex> mirror gone 20 s after nodes 0 (8e849b42). Before: 2026-11-12 20:02-20:10Z, kcp ac51b91e (rollout 18): node Ready, quickstart-k8s-ok, the requester logs 0 ERROR (no ComponentLink errors: fixed in 363dd814); nodes 0 in 156 s; but the emptied computer.socialweb.infra/nodes-<hex> mirror stayed in the repo (infra_record_replaced with 0 nodes; deleted by hand), self went. Last pass: 2026-11-12 18:46-19:00Z on prod as bob, kcp cdd93727 (release 6380eee4, rollout 16), A and C, no pin (winner hat-8): agent1 online, uname -m answered in 2.9 s, delete in 12 s (r2Released 8.8 s), 45/45 RecordNotFound, core.log 0 ERROR; C: node Ready, quickstart-k8s-ok, nodes 0 in 156 s, computer.socialweb.infra/{self,nodes-...} gone with the cluster (none left after the stop). Open: C's requester logs Reconciler error bus: no accepted ComponentLink allows root:EventStream.../watch-infra-nodes-<hex> (and watch-infraindex-<repo>) events.subscribe x18 while the node lives. Before: 2026-11-12 17:47-18:08Z, kcp 60d0d3b7 (release 0b68ceeb, rollout 15), path A, NO bidderDids pin (requester discovery; winners hat-8, then hat-9): vm1 iroh + webrtc, sbx1, agent1 with its harness from the pre-written bundle (step A4b): association written 36 s after agent-start, online +136 s, messages both ways, uname -m answered in 2.9 s; delete: every contract record and agent1's grant + endorsement RecordNotFound x3, both DIDs plc_tombstone, hosts empty. 2026-11-11 (kcp f482083c, no pin): A, B (hat-7) and C (hat-8) pass except agent1 (no association on a declarative contract: fixed in 60d0d3b7). Earlier: 2026-11-10 as bob (bidder hat-9): A, B and C; 2026-11-01: A and C on rc159 (ec0f4849), B on the rc160 CLI (840f0437). Prod readings below
sequenceDiagram
  participant U as user (kubectl, atirc)
  participant K as requester kcp (run requester)
  participant R as Requester app
  participant B as user-configured bidder
  participant I as irc.fedfork.com
  U->>K: apply vouch.yaml, mixed-prod.yaml
  K->>R: ComputeContract vm1, sbx1, agent1 (bidderDids pin)
  R->>B: RFP (requester-tangled-vouch attached)
  B->>B: bidder-tangled-vouch + attached policy: operator vouched requester?
  B->>R: bid
  R->>R: requester-tangled-vouch: requester vouched bidder?
  R->>B: accept
  B->>R: receipt, signed guest events at computed keys (onNetwork, hostKeys, vm.sandbox)
  R->>I: chat.atirc.mint (agent1, leased on receipt + vm.terminated)
  R->>K: ComputeContracts Held, EphemeralInfra Deployed
  U->>B: ssh vm1 (iroh, webrtc); read sbx1 stdout
  U->>K: agent-start ContractEvent -> guest.agentStart
  U->>I: JOIN of the agent nick, prompt, answer
  U->>K: Ctrl-C, restart: Held stays Held
  U->>K: delete EphemeralInfra -> revoke -> vm.delete -> vm.terminated

Preconditions

ItemNeed
Biddera bidder the user configured: install bidder with the default policy (no --bidder-policy: bidder-tangled-vouch), ../bidder.md; for sbx1 it must serve deno: run bidder --compute-kinds vm,deno, or install bidder --compute-kinds vm,deno (written to its config.json; on an already installed bidder, run the install again with the flag, which restarts the bidder unit); optional --bidder-operator-signin <operator handle> for the operator vouch of step A4. A prod-qr install bidder lists https in its offering endpoints only with the bidder option market-https=true (config.json "bidder-options": "--market-https=true"); without it the offering lists iroh and webrtc
Bidder DIDMarketBidder app status.did in the bidder kcp (the operator reads it); it replaces REPLACE-BIDDER-DID in every contract of mixed-prod.yaml
Bidder associationthe bidder DID is its operator's agent: agent-of/<operator DID> in its DID doc (written by the bidder) and, in the operator repo, market.association/<bidder DID> then link.ident.endorsement/<bidder DID> (written by the bidder with --bidder-operator-signin, needs the optional scope repo:link.ident.endorsement?action=create&action=delete, or by deno run -A --no-lock scripts/associate-bidder.ts --creds <file> --account <ACCOUNT> --agent-did <bidder DID>); check: socialweb-computer internal association <bidder DID> roots [operator DID]
Requester accounta real atproto account (handle resolves to its DID, PDS public) signed in by OAuth QR; the creds file owner-only (mode 0600), read only by scripts/oauth-signin.ts
Vouchesboth directions: requester -> bidder by ../../examples/infra/vouch.yaml in the requester kcp (needs the optional scope repo:sh.tangled.graph.vouch) or goat; bidder operator -> requester by ../../examples/infra/vouch-bidder-operator.yaml in the bidder kcp (operator sign-in) or goat from the operator account, unless the requester account is that bidder's own user or one of the user's verified agents. Owner, 2026-11-18: "the vouch policy mode must also trust the own user and their associated DIDs/agents to N levels down". bidder-tangled-vouch (and requester-tangled-vouch) build the trust snapshot from the identlink walk of each party (trust.BuildSnapshot): each link verified both ways (the agent's DID doc agent-of/<principal>, the principal's link.ident.endorsement/<agent> and com.publicdomainrelay.temp.market.association/<agent>), N = 100 levels (identlink.DefaultMaxDepth), 256 lookups; a requester that shares a root with the bidder, or whose operator is on the bidder's side, is trusted (policytest cases operated-by-counterparty, same-verified-operator, two-levels-below-counterparty, counterparty-two-levels-below-self(-operator) and their forged and half-verified twins). The test runs, per bidder, the same walks of the account and of the bidder (com.fedfork.exec.identlink.walk) and reads the operator's vouch of the account; a bidder admits the account when that vouch exists or the walks say so
atirca token of your own (atirc token <nick>) to atirc sub the owner channel; never print it
HostLinux, kubectl, jq, ssh, dumbpipe on PATH (the requester tries iroh first; a bidder without https, such as hat-9, is reached only over iroh or webrtc), the release or dev binary of the commit under test as $SWC; B: root on the requester host
RulesREADME.md shared rules: announce on atirc first, one driver per account, release what you rent

Steps

The composition starts its run requester with --bidder-dids <the target's bidders> (plan 0040 step 5): the list filters discovery for every rental, a --with-gateway node's too; on a market network the network sets them.

ST is a NEW owner-only state dir for each run (ST=~/.tmp/acc13-$(date +%s); mkdir "$ST"; chmod 700 "$ST"): a run requester --name mp on an existing dir resumes that dir's requester with its old account and session (it logs run requester resumes an existing requester), never a new sign-in. Write each vouch at least 5 minutes before the first apply when a contract was denied before (verdicts are cached, denials too).

A. run requester (foreground, OAuth QR)

  1. Start: $SWC run requester --name mp --state-dir "$ST" --atproto-oauth-qr > "$ST/mp.log" 2>&1 &. Sign in from the pendingAction.url in the log with scripts/oauth-signin.ts --creds <file> --account <ACCT> --url "<url>" (read the url right before, never print or paste it).

  2. Wait for $ST/requesters/mp/ready.json (written atomically once the root is Ready); eval "$($SWC run requester env mp --state-dir "$ST")" sets KUBECONFIG to the requester workspace; kubectl get marketrequesters.socialweb.computer app -o jsonpath='{.status.phase} {.status.did}' is Ready <did>; the ready file's identityMode is oauthQr.

  3. Requester -> bidder vouch: sed -e "s/REPLACE-REQUESTER-REPO/repo-$(jq -r .did "$ST/requesters/mp/ready.json" | tr : -)/" -e "s/REPLACE-BIDDER-DID/<bidder did>/g" -e "s/2026-11-01T00:00:00Z/$(date -u +%Y-%m-%dT%H:%M:%SZ)/" examples/infra/vouch.yaml | kubectl apply -f -; AtprotoRecord vouch-bidder Applied with at://<requester did>/sh.tangled.graph.vouch/<bidder did>. When the session lacks the vouch scope, write the same record with goat record create from the requester account instead.

  4. Bidder operator -> requester vouch: in the bidder kcp, vouch-bidder-operator.yaml with REPLACE-REQUESTER-DID (the operator sign-in of ../bidder.md), or goat from the operator account; check with goat get "at://<operator did>/sh.tangled.graph.vouch/<requester did>" | jq -r .kind -> vouch. Skip it for a bidder whose user is the requester account or one of the account's operators (Preconditions, Vouches). When the write needs the owner's account, it is owner-only: ask the owner to write it; an agent does not. 4b. agent1's harness (optional; without it the agent joins the mesh but nothing answers): write agent1's bundle into the requester's own OpenBao before the apply, with the placeholder /etc/atirc/atirc.env and an atirc agent config (as 0004 --agent-config; mode 0600 file, never printed) at /etc/atirc/config.toml; a bundle that is there is kept, and the requester adds the association entries itself:

    D=$ST/requesters/mp/dev/mp/openbaoservers/requester/openbao
    BAO=$(find ~/.cache/socialweb-computer/runtime -maxdepth 2 -name bao -type f | head -1)
    export BAO_ADDR=$(kubectl get openbaoservers.socialweb.computer requester -o jsonpath='{.status.address}')
    export BAO_CACERT=$D/tls/ca.crt BAO_CLIENT_CERT=$D/clients/requester/client.crt BAO_CLIENT_KEY=$D/clients/requester/client.key
    (umask 077; jq -n --rawfile c ~/.config/atirc/config.toml '[{path: "/etc/atirc/atirc.env", value: "", mode: "0600"},
      {path: "/etc/atirc/config.toml", value: $c, mode: "0600"}]' > "$ST/agent1-bundle.json")
    BAO_TOKEN=$($BAO write -field=token auth/cert/login name=requester) \
      $BAO kv put -mount=requester contracts/einfra-mixedprod-agent1/secrets bundle=@"$ST/agent1-bundle.json" >/dev/null
    rm -f "$ST/agent1-bundle.json"
    
  5. Apply: sed -e "s|ssh-ed25519 AAAAREPLACE-WITH-YOUR-PUBLIC-KEY you@laptop|$(cat "$(jq -r .sshPublicKey "$ST/requesters/mp/ready.json")")|" -e "s/REPLACE-BIDDER-DID/<bidder did>/" examples/infra/mixed-prod.yaml | kubectl apply -f -; kubectl wait ephemeralinfras.socialweb.computer/mixedprod --for=jsonpath='{.status.phase}'=Deployed --timeout=5m.

  6. vm1: $SWC run requester ssh mp --state-dir "$ST" --contract einfra-mixedprod-vm1 --exec -- cat /root/mixedprod prints mixedprod-vm-ok; the same with --transport webrtc --exec -- uname -m prints x86_64.

  7. sbx1: kubectl get computecontracts.socialweb.computer einfra-mixedprod-sbx1 -o jsonpath='{.status.contract.sandbox}' has stdout mixedprod-deno-ok sum=10 on linux/x86_64 and exitCode 0.

  8. agent1: the requester log has requester_agent_bundle_placeholder (no bao step) and, after the receipt, the token-less mesh agent's requester_agent_mesh_key_written and requester_association_written (grant + endorsement at rkey = the in-VM R2 DID; the agent's peer record is R2's co-located account's, no record in the requester repo names it, 0004); then sed -e 's/quickstart-agent-agent1-start/mixedprod-agent1-start/' -e 's/einfra-quickstart-agent-agent1/einfra-mixedprod-agent1/' examples/infra/agent-start.yaml | kubectl apply -f - (ContractEvent Submitted). The owner mesh node (atirc mesh run --peer <requester_agent_identity peer> as the requester account) lists the agent and sees online: <nick> (the agent posts it by itself; without it the entry FAILs); pub/sub both ways and a prompt as 0004 (agent1 has sshAuthorizedKeys for the guest side). Without step A4b the agent has no harness and never answers.

  9. Restart mid-run: Ctrl-C (kill -INT the run requester): the ready file is removed, the kcp stops, the state stays; start the same command again (the same CLI options, such as --network; no application options are needed): same DID, ready file rewritten, every contract still Held, step 6 ssh works again, the agent still answers.

  10. Delete: kubectl delete ephemeralinfras.socialweb.computer mixedprod: each contract releases first; agent1 status.atircRevokes is Done before the bidder's vm.terminated; the EphemeralInfra and its ComputeContracts are gone; the bidder host is empty (0008 watcher: no VM unit, no sandbox); the requester log has requester_contract_records_deleted, and a getRecord of each contract record (accept, rfp, compute.vm, com.fedproxy.rbac; for agent1 also market.association and link.ident.endorsement at rkey = its R2 DID) at its key answers RecordNotFound (read 3 times); the two vouch records are still in the repos (deletionPolicy: Retain; goat get both).

  11. Stop: Ctrl-C the run requester.

B. install requester (systemd user unit)

  1. sudo $SWC install requester --name mp --dry-run, then sudo $SWC install requester --name mp (account swc-requester, identity --atproto-oauth-qr by default); the sign-in URL: $SWC run requester signin mp --state-dir /var/lib/socialweb-computer/swc-requester (as root or the account); sign in with scripts/oauth-signin.ts.
  2. R=/var/lib/socialweb-computer/swc-requester/requesters/mp/ready.json; as the account, kubectl --kubeconfig "$(jq -r .kubeconfig "$R")" reads MarketRequester app Ready.
  3. Steps A3 to A8 as the account, $SWC run requester ssh mp --state-dir /var/lib/socialweb-computer/swc-requester.
  4. Restart mid-run: systemctl --user restart socialweb-computer-requester@mp.service as the account (and upgrade to the same binary): same DID, kcp and OpenBao units adopted (same InvocationID), contracts still Held, ssh again.
  5. Step A10, then sudo $SWC uninstall requester --account swc-requester (keeps the state dir; --purge removes it).

C. Gateway in the requester kcp (Cluster and k8s-cluster.yaml)

The gateway runs inside the requester's core (Gateway spec.requesterRef): its clusterprovider, k3s and kine are HostProcesses of that core, and every node is a contract of the same requester. Needs a subuid/subgid range for the user (check-host reports it).

  1. $SWC run requester --name mk --state-dir "$ST" --with-gateway --atproto-oauth-qr > "$ST/mk.log" 2>&1 &; sign in as in A1; wait until the ready file names the gateway: jq -r .gateway "$ST/requesters/mk/ready.json" is not null.
  2. $SWC run requester cluster mk --cluster c1 --state-dir "$ST"; then TK=$($SWC run requester kubeconfig mk --cluster c1 --state-dir "$ST") (retry until it prints a path: the Cluster is Ready).
  3. kubectl --kubeconfig "$TK" apply -f examples/infra/k8s-cluster.yaml: a NodeClaim becomes an Infra in the requester kcp, its contract is bid and Held, the node VM fetches meshd by sha256 from the public bucket and joins k3s; poll until the quickstart-k8s pod is Ready (a kubectl wait before the pod exists returns no match); kubectl logs deploy/quickstart-k8s prints quickstart-k8s-ok.
  4. Pin the node to a remote bidder by leaving only that bidder able to win (the run below suspended the local bidder: kubectl patch marketbidders.socialweb.computer app --type merge -p '{"spec":{"suspend":true}}' in its kcp, resumed after).
  5. kubectl --kubeconfig "$TK" delete deployment quickstart-k8s: the node is reaped, its contract released, kubectl get nodes is empty (scale to zero); the bidder host is empty; computer.socialweb.infra/nodes-<hex> answers RecordNotFound; computer.socialweb.infra/self is gone, or names no nodes-<hex> of this run in live or recentEnded and is not empty.
  6. Ctrl-C the run requester.

Expected Observations

EventWhereTypical
ready file written, identityMode oauthQr, DID of the accountrequesters/mp/ready.json11 s after the sign-in on an own network (quickstart-prod section 7)
vouch-bidder Applied, at://<requester did>/sh.tangled.graph.vouch/<bidder did>; operator record read back vouchAtprotoRecord status, goat get1-4 s
RFP of each contract carries requester-tangled-vouch; the bidder's run allows (operator -> requester present)bidder PolicyWorkflowRun, requester logfirst run after the records
vm1 Deployed, ssh prints mixedprod-vm-ok, x86_64EphemeralInfra status.contracts[vm1], ssh~20 s after the apply (0012: 19 s)
sbx1 Held with ComputeVM backend DenoSandbox, sandbox.stdout mixedprod-deno-ok sum=10 on linux/x86_64contract status~6 s (0012)
agent1: requester_agent_bundle_placeholder, requester_service_auth atircSigning oauth, requester_atirc_minted, nick JOIN after agent-startrequester log, atirc channelHeld ~20 s, JOIN within a minute of agent-start (0012 prod readings)
restart: same DID, ready file rewritten, contracts Heldready file, ComputeContractsready again in seconds (quickstart-prod section 7: 1.4 s)
delete: requester_atirc_revoked, status.atircRevokes Done, then vm.delete, vm.terminated, contracts gonerequester log, bidder repo, kcpunder a minute
contract records deleted: requester_contract_records_deleted, requester_association_deleted (agent1); accept, rfp, compute.vm, rbac, agent1's grant + endorsement RecordNotFound at their keysrequester log, getRecord in the requester reposeconds after the release
bidder host empty, vouch records present0008 watcher, goat getright after the delete

Prod Readings (2026-11-01, rc159 ec0f4849, account alice)

time (Z)path, stepreading
10:37:20A2ready file, identityMode oauthQr, alice's DID
10:37:55A5mixed-prod Deployed in 35 s on the local bidder (install bidder --compute-kinds vm,deno): vm1, sbx1, agent1 Held
10:38A6, A7vm1 ssh iroh and webrtc exit 0; sbx1 stdout mixedprod-deno-ok sum=10 on linux/x86_64, exit 0
10:38:31A8placeholder bundle, mint, agent JOIN, answered x86_64
10:39A9restart: same DID, all Held, ssh again
10:40:16.805A10agent1 revoked, then the terminations at 10:40:17; 0 contracts in 5 s; host empty
11:19:31B3 (rc159)run requester ssh as the account: did not answer within 1m (the ProxyCommand ran under the account's nologin shell)
11:46:44B1, B2 (rc160)install requester (account swc-requester, socialweb.slice) printed the journal and run requester signin hints; the URL from run requester signin mp; Ready
11:47:14B3vm.yaml pinned to hat-7; ssh as the account: quickstart-vm-ok, x86_64, exit 0
11:47:15B4systemctl --user restart: same DID, contract Held, ssh quickstart-vm-ok again
11:47:35B5delete: 0 contracts; uninstall --purge removed the account; hat-7 empty
11:31:52C1--with-gateway, ready file names the gateway
11:32:03C2cluster c1 kubeconfig
11:33:04C3, C4node 2wjeqtgx-c1-node1 on hat-7 (remote, local bidder suspended), k3s v1.36.4+k3s1, pod Ready, quickstart-k8s-ok
11:35:41C5node reaped (hat-7 terminate 11:35:36, released 11:35:38), 0 nodes, hat-7 empty

No plaintext token, key or sign-in URL in any log (check 0 on every file).

Prod Readings (2026-11-10, bob, bidder hat-9 on kcp 2d367fe8, hat-8 requester host)

stepresult
A1-A2run requester --name mp with --atproto-oauth-qr, signed in by scripts/oauth-signin.ts (creds 0600); ready file identityMode oauthQr
A3vouch-bidder Applied (bob -> hat-9)
A4not written: hat-9's operator is bob, the requester itself
A5-A7mixedprod Deployed; vm1 mixedprod-vm-ok over iroh, x86_64 over webrtc; sbx1 mixedprod-deno-ok sum=10
A8mesh key, lease and peer record written; agentStart Submitted
A9Ctrl-C removes the ready file; the restart keeps the DID; 3 contracts Held; ssh again
A10 (first run)FAIL: records kept, requester_record_delete_skipped (the QR session lacked every optional delete scope: qr.fedfork.com's client metadata was behind scope.go)
A10 (after the QR metadata fix, fresh sign-in, main ea6935f7 CLI)requester_contract_records_deleted 6/6, 6/6, 10/10 and requester_agent_peer_deleted; no market record of the run left (the requester's badgeBlueKeys signer key stays by rule)
Binstall requester --name mp as swc-requester, signed in; Ready, contracts Held, ssh and deno; systemctl --user restart keeps the kcp and kine InvocationIDs and the DID; delete releases (on the 2d367fe8 binary the policy and vm.delete records stayed: fixed in ea6935f7); uninstall requester --purge leaves no account, linger or state
Crun requester --name mk --with-gateway, Cluster c1, the node *-c1-node1 from hat-9 Ready in 50 s, quickstart-k8s-ok; delete Deployment: nodes 0 in 160 s; the node contract's records deleted 6/6; the Infra mirror computer.socialweb.infra/nodes-<hex> and self are kept by design and were deleted by hand

Local Readings (2026-11-01, own market network mxp, main 6497fe61 dev build)

dev up --bidders 1 --compute-kinds vm,deno (default bidder-tangled-vouch, network-composed vouches), then run requester --name mp --network mxp (localRepo, handle root-mp.requester.local), ready in 11.5 s.

time (Z)stepreading
08:38kubectl apply --dry-run=server, the file as committed and filled inephemeralinfra.socialweb.computer/mixedprod created (server dry run), exit 0; no policy in any contract
08:38:38apply, bidderDids = the network bidderagent1 Failed at once, refused: ... root-mp.requester.local is a .local handle ... --atproto-handle ..., placeholder bundle written; vm1 and sbx1 Held on the pinned bidder (winner did) by 08:38:58
08:38:43sbx1sandbox {"exitCode":0,"stdout":"mixedprod-deno-ok sum=10 on linux/x86_64\n"}
08:39:05run requester ssh mp --contract einfra-mixedprod-vm1iroh: mixedprod-vm-ok, x86_64 (0.87 s); webrtc: x86_64 (0.68 s)
08:39:20Ctrl-Crequester_contract_left_for_restart vm1, sbx1; kcp stopped, state kept
08:39:30restart without --networkrefused: "created with other options (network "mxp" ...)" (the network is part of the same command)
08:39:39restart with --network mxpready in 1.6 s, same DID; 08:39:44 vm1 and sbx1 Failed the held contract is not resumed: the policy run for the recorded winner ... the policy engine is not running, then infra_ephemeral_complete, both released, the EphemeralInfra deleted

On today's main a run request release of an invocation that Failed before a VM was held, or is Released, only removes it; no second release is needed.

Pass Criteria

  • kubectl apply --dry-run=server -f examples/infra/mixed-prod.yaml passes against the requester (A and B)
  • no file and no command names a policy: requester-tangled-vouch and bidder-tangled-vouch run, and the contracts are bid only after both vouch records exist
  • every contract lands on the pinned bidder (status.contract.winner.did is the DID put in for REPLACE-BIDDER-DID)
  • vm1 ssh over iroh and webrtc, sbx1 stdout, agent1 JOIN, its own <nick> online line (seen by the owner node within 60 s of subscribing; none is a FAIL, 0004 pass criteria) and answer, all on the same requester
  • a Ctrl-C (A) or unit restart (B) mid-run keeps the DID and every Held contract Held
  • delete: every agent token revoked before vm.terminated (0004 pass criteria), the bidder host empty, every contract record of the requester RecordNotFound at its key, both vouch records still present
  • no tokens, passwords or session secrets in any log (plaintext check 0 on every log file); the short-lived QR sign-in URL is printed by design

Cleanup

  • kubectl delete ephemeralinfras.socialweb.computer mixedprod (it ends itself at endWhen: Complete or the 1 h ttl)
  • A: Ctrl-C the run requester; remove $ST only when the DID is not needed again
  • B: sudo $SWC uninstall requester --account swc-requester (--purge for a throwaway account)
  • the vouch records stay: remove a vouch only from the account itself (goat), never by a kcp delete
  • post done on atirc; stop your atirc sub

Known Pitfalls

SymptomCause / fix
A: FAIL the agent never posted <nick> onlinebefore atirc 58f46d0 the agent's first line was the model's choice; since, agent run posts it itself: 0004 Known pitfalls
C: the tenant node joins 4 to 5 min after the apply (pod_ready_s 275.7; was 45 to 50): the node's socialweb-node-prep logs HTTP 400 could not load configuration from auth/jwt-<hash>/login, the cluster OpenBao error checking oidc discovery URL: issuer did not match ... got https://127.0.0.1:0/v1/identity/oidcbefore the fix: the bidder's guest control plane fetched the OIDC discovery document 3 s before it configured the issuer and cached the OpenBao default for 5 min (oidcDocumentRefresh); a document that does not name the configured issuer is no longer cached (internal/guestidentity/service.go, bidder log refreshing the guest OIDC document failed; no document is cached yet ... does not name the configured issuer yet)
C on a dev network: no node VM, ComputeContract Requesting / rfp_submitted, requester_submit_rfp_failed 401 from prod biddersbefore 22b49fb2: the test host started run requester without --network, so it discovered the public relays and the prod bidders; requester_bidder_discovery must read relay:http://127.0.0.1:<port> and the network's bidders
after a restart every Held contract is Failed the held contract is not resumed: ... the policy engine is not runningmain 6497fe61 and rc157: fixed in rc158 (resume waits for the policy engine)
B: no sign-in URL in journalctl --user of the accountthe user units of a system account log to the system journal, which the account cannot read: run requester signin mp --state-dir /var/lib/socialweb-computer/swc-requester (rc160), or as root journalctl _UID=<uid> _SYSTEMD_USER_UNIT=socialweb-computer-requester@mp.service
B: run requester ssh as the account does not answer within 1mrc159: ssh ran its ProxyCommand with the account's nologin shell; rc160 runs it under /bin/sh
C: run requester cluster says the requester runs no gatewayrc159: the ready file was first written before the gateway was Ready; rc160 writes it once, with .gateway
sbx1 no bids received within 30s, stays without a bidthe pinned bidder does not serve deno: --compute-kinds vm,deno on run bidder, or install bidder --compute-kinds vm,deno again for an installed bidder
every contract no bids; bidder policy rejected ... not vouchedthe operator -> requester record is missing (step A4); denials are cached 5 minutes
no bids though both vouches exist; internal association <bidder DID> has no rootsthe bidder is not its operator's agent: no agent-of, or the operator's endorsement is missing (operator_endorsement_skipped: sign the operator in again with the optional scope)
agent1 Failed before its RFP, refused: the atirc agent needs a requester handle that resolves to <did> ... --atproto-handle ...the requester handle is .local (a localRepo identity, the default without an identity option) or does not resolve; use the OAuth account, or a local PDS on plc.directory with --atproto-handle <a domain you control> (../quickstart-prod.md section 5)
vouch-bidder (or the operator record) not Applied, the MarketIdentity has pendingAction signInAgain ScopeUpgradethe QR page (qr.fedfork.com) granted a session without repo:sh.tangled.graph.vouch; the scope is optional, so the identity stays Ready: write the record with goat, or sign in again once the QR page asks for it
agent1 Failed secrets need the requester ingress (getSecrets) or secretsFallback: Sealed--comms-mode Firehose has no inbound endpoint: add --secrets-fallback Sealed, or keep the default comms Both
a restart is refusedthe application options differ from requester.json of the first start: give the same ones or none, or start a new name
contracts land on another bidderREPLACE-BIDDER-DID left in one contract or a wrong DID: every contract of the file carries its own bidderDids
Held never reached after the acceptthe winner's offering endpoints hold nothing the requester can use: no dumbpipe on PATH for iroh, and no https endpoint
HTTP 400 on market.event at the bidderthe event is unsigned: the requester is an old build