chadtmiller.com/knotless

Switch repository

Loading…

What changes

A run waiting after a deploy is dropped as the runner stops. A stopping runner keeps each run not yet started at the same commit and base as a running superseded: finish run, a deploy, to start itself, since the deploy may wait for its commit's other checks. Runner::dropping now leaves out a run whose workflow names that one in after: it is dropped, as any other not started, and the runner handed over to runs it once the deploy has passed. finishing notes each such run's workflow beside where it is, for that.

release.yml runs after: [deploy]. So on a release commit it starts on the new runner, not the one the deploy is replacing.

docs/design/runner.md (A stop) says so.

Why

The 0.8.0 release was cancelled at bb49d4bc, 16 minutes into its build, "as the runner stopped, past what a stop waits for". A version bump changes every binary, so the deploy at the same commit rebuilds the runner and hands it over; the old runner waits --stop-wait (15 minutes) for runs going, then cancels them. The release's build for four targets takes longer than that. So every release commit races its own deploy, and loses whenever the build is slow.

after: [deploy] alone would not do: waiting, the release is a run not started at the deploy's commit, which the stop keeps, to start on the old runner once the deploy ends, where it is cut off the same way. A longer --stop-wait would hold the old runner, and its jobs, for half an hour on every such deploy.

How it was checked

a_stop_drops_what_has_not_started gains a release workflow after: [deploy] at main's commit: queued behind the deploy, it is cancelled as the runner stops while the push's other run is still kept, and the runner started next runs it, once. The rest is left to CI.

Shortcomings

A deploy that fails now files the release failure without running; it is run again by hand once a deploy passes (knotless rerun … --ref refs/heads/main --workflow release). A deploy that passes without deploying, as main moved past it, still lets the release run, whose own step waits for the seven checks as before.

This makes a stopping runner drop a not-yet-started run whose workflow names the running superseded: finish workflow in after, so the runner handed over to starts it. It also gives release.yml after: [deploy].

I read the dropping change and found no bug. finishing now holds (place, workflow) pairs, and a run is kept only if some finishing run at its place is a workflow it doesn't wait on, which matches the stated intent. The release run is dropped when only the deploy is going, and other runs at that commit are still kept. The drop(finishing) before the .await is fine because the iterator isn't used afterwards. The test extends the existing stop scenario to cover cancel-then-run-once on the next runner. The changeset and the design doc are updated.

The failed-deploy case is named in the body as a known shortcoming, so I didn't weigh it further.

The new commit, 9fb2a7e, only wraps the finishing borrow in dropping in a block so the future stays Send. That is what the failed pre-flight check at 02d3303 most likely hit. The logic I approved before is unchanged: a not-yet-started run is kept only if some finishing run at its place is a workflow it doesn't wait on. The borrow is released before the .await. I did not re-read the checks for the new head, so confirm they pass before merging. The tests, docs and changeset are as before.

Closed for a drain long enough for the runs a stop waits for: a longer --stop-wait on both hosts, in its own pull. That keeps a release independent of the deploy.