~ knotless Search/
↑↓ move↵ open⇧↵ all results

socialweb.computer/socialweb-computer-kcp

socialweb.computer: a compute market on atproto (kcp)

1 collaborator
0
0

ci-debug: mistakes 14 to 18 from roll three

Merged johnandersen777.bsky.social asked to merge into main from socialweb.computer/socialweb-computer-kcp:phase-2
docs/ci-debug.md+6−0
@@ -497,6 +497,11 @@ The lead's own, on plan 0040 (2026-11-20). Each has the rule it left; plan 0040
| 11 | Accepted a test entry whose precondition was "the previous release lacks the feature" (`0007:fence` took its from binary from the previous rc) | The entry failed for ever the moment its own roll went green; caught before the next roll, one more fix | Ask of every new entry: does it still pass after it lands, on the roll after, and after a rollback |
| 12 | Added a pinned-binary bump (dumbpipe) to a fix stack late, proven by pins and a file diff, without switching a running bidder between the two builds | The full hand cycle found both bidders deaf over iroh after the switch, in both directions: on prod it would have been an outage at the roll and at the rollback | A change of a pinned binary or of anything a core stores across restarts is proven by `0007:upgrade` alone first, both directions, before it joins any stack |
| 13 | Kept adding fixes to the stack as I found intermittents | Five hand cycles and no second roll in a day | Freeze the stack once it holds what failed; anything new that is not a regression of the stack goes to its own roll |
| 14 | Rolled after checking the four bidder hosts but not the runner | `test` red in 80 s: the runner container held 173 zombie ssh, 199 of its 256 processes | Before a roll read the runner container's `pids.current` on the gateway host; over 100: `systemctl restart knotless-fork-runner.service` (no run active) |
| 15 | Asked for a rerun of a red `test` before pulling its artifacts | The rerun pruned them; the bidder's side of the failure was never read | Pull a failed run's artifacts (`knotless run artifacts`, a fork-built CLI) before any rerun |
| 16 | Waited on a CI `test` by its state word only | Three hours on a run that was 6 PASS of 20 | At the first doubt read the run from inside: `knotless run attach`, then `swc-test-host attach --ci` (state, the gate dir's RESULT lines); a pass ends near 2 h |
| 17 | Took seven hand cycles as a rehearsal of CI | CI's placement (its own VM on the hosts) and its runner were never exercised by hand | Before the roll of a batch, one hand network in CI's layout with the overlay measured (pair matrix); save a meshd log of a healthy network |
| 18 | Named a cause (VM placement) from one run's layout | Two green rolls had the same layout | Compare with the runs that passed before naming a cause |
Before a pull is queued, each of these is a yes:
@@ -506,6 +511,7 @@ Before a pull is queued, each of these is a yes:
- Anything that differs on prod (hosts, routers, repo sizes, a requester inside a guest, guests booted by the old
bidder) has a rehearsal that has that difference, or is written in the pull as unproven with the reason.
- The tree in the pull is the tree that passed.
- The runner container's process count was read (mistake 14); a healthy network's overlay (meshd) log is saved.
## Do not waste time