# Acceptance 0007: bidder-side VM termination (ops) Proves a bidder operator can end a VM whose requester cannot release it (tab or app gone, cascade gap, stray debug VM) and that the delete leaves nothing behind: the contract is released, `vm.terminated` is pinned in the bidder repo, and every atirc token leased to the contract dies. - Procedure: [bidder.md, Terminate a VM from the bidder](../bidder.md#terminate-a-vm-from-the-bidder) - Used on prod: R2 orphan (hat-7, 2026-10-30), iOS cross-restart retry VM (hat-7, 2026-10-31 02:07Z), stray debug VM (hat-7, 2026-10-31 18:11Z) - Last pass: 2026-10-31 (hat-7) ```mermaid sequenceDiagram participant Op as operator (bidder host) participant K as bidder kcp participant P as bidder repo participant Rq as requester (maybe gone) participant A as atirc Op->>K: delete ComputeVM fc-- --wait=false K->>P: termination StateSync + vm.terminated (reason deleted_on_request) K->>K: stop VM process, unit, work dir; release contract K-->>Rq: vm.terminated event (404 = requester gone, fine) A->>P: lease check sees vm.terminated -> token dead (latched) K->>K: drop finalizer, ComputeVM gone ``` ## Preconditions - Authority: the owner asked for it, or the VM is matched to a contract you are ending (stray test / orphan). Never an owner-held VM unless the owner says so - Shell on the bidder host with sudo; announce on atirc: `: deleting on out of band (contract )` - Know the requester DID and the contract (`kb get computevms ... -o wide`, watcher timeline) ## Steps ```bash K=$(sudo -n -u swc-bidder sh -c 'ls /var/lib/socialweb-computer/swc-bidder/handles/*/kcp/admin.kubeconfig | head -1') kb() { sudo -n -u swc-bidder env HOME=/var/lib/socialweb-computer/swc-bidder KUBECONFIG="$K" kubectl "$@"; } kb get computevms.socialweb.computer -o custom-columns=NAME:.metadata.name,PHASE:.status.phase,CREATED:.metadata.creationTimestamp # name = fc--; match DID prefix AND creation time kb delete computevms.socialweb.computer --wait=false kb get computevms.socialweb.computer -w # until it is gone ``` Then pin `vm.terminated` (getRecord on the bidder repo, see [0008](0008-host-observer.md)) and ask atirc-dev to confirm the leased token is dead. ## Expected observations Bidder journal, in order (hat-7 example, 2026-10-31 22:51Z): | Line | Time | | --- | --- | | termination StateSync `termination-` | 22:51:38.49 | | `Running -> Releasing` | | | `released` | 22:51:39.75 (+1.3 s) | | `released: the backend is gone and the finalizer is dropped` | 22:51:39.81 | | `a released bid is marked terminated` | | | `vm.terminated` record pinned, reason `deleted_on_request` | | - Requester side: `vm.terminated` delivery may 404 (requester gone): record in the bidder repo still stands - atirc: next use of any token leased on the contract -> 401 `lease is not alive`, latched (no revival if the record later disappears) ## Pass criteria - ComputeVM gone within seconds; contract released - `vm.terminated` exists in the bidder repo with reason `deleted_on_request` - Host clean: `kb get computevms,firecrackervms` "No resources found", `pgrep -c -x firecracker` 0, `sudo ls -A /vms | wc -l` 0 - Leased tokens dead (atirc-dev check, or a probe with the token gets 401) - Unleased (plain) tokens of the contract revoked manually (atirc-dev, by subhandle) ## Cleanup - Post the timeline and the pinned record on atirc (split lines < 300 chars) - Note the root cause of the orphan (why the requester did not release) as an open item; out-of-band delete is a mitigation, not the fix ## Known pitfalls | Symptom | Cause / fix | | --- | --- | | deleted the wrong VM | matched by DID prefix only; always match DID prefix + creation time + contract | | `vms/` count 0 while a VM runs | dir is 0700; read with sudo | | token still answers | token was plain (pre-lease or test mint); revoke it by subhandle | | ComputeVM `Lost` after `uninstall --force` | expected: forced uninstall does not run the finalizer path; do not use it to end one VM | | requester resumes later and re-releases | harmless: release is idempotent; the contract is already Released |