chadtmiller.com/knotless
git that lives in your Atmosphere account
1
10
A runner handing over lets its runs finish, a release's build among them
deploy/runner-host/docker-compose.yml+4−2
| @@ -18,8 +18,10 @@ services: | ||
# Root, as on the main host: it holds the docker socket. What it runs | ||
# is held apart by gVisor, not by this. | ||
user: root | ||
# As on the main host: a stop waits for runs whose steps started, up | ||
# to --stop-wait (15 minutes). | ||
# A stop waits for runs whose steps started, up to --stop-wait (15 | ||
# minutes, its default). Not the main host's hour: a deploy here | ||
# recreates the runner rather than handing over, so it, and every | ||
# pull's checks, wait for the stop to end. | ||
stop_grace_period: 20m | ||
command: | ||
- sh | ||
deploy/docker-compose.yml+12−10
| @@ -192,16 +192,17 @@ services: | ||
# Stopped, it drops each run still waiting its turn, for the next | ||
# start to run, and finishes only the runs whose steps have started, | ||
# a batch's and those at a going deploy's commit: most finish in a few | ||
# minutes, the longest, test, in about 9. A deploy does not wait for | ||
# that: it starts the next runner beside it (handover.sh), and only a | ||
# stop by hand does. Past --stop-wait (15 minutes) it cancels | ||
# those still going itself, which a stop nothing comes to kill needs. | ||
# 20 minutes is that with room to spare; past it, docker kills the | ||
# runner, and the next ends what was cut off as an error and removes | ||
# its containers. Each | ||
# step may take --step-timeout (30 minutes), but none does in practice, | ||
# and a restart should not wait on one that hangs. | ||
stop_grace_period: 20m | ||
# minutes, test in about 9, a release's build of the client in about | ||
# 17. A deploy does not wait for that: it starts the next runner beside | ||
# it (handover.sh), which lends the old one the jobs its runs hold, so | ||
# waiting costs nothing, and only a stop by hand waits. --stop-wait is | ||
# an hour, so that no run is cut off that its own limits let go on: a | ||
# step that hangs ends at --step-timeout (30 minutes). Past the hour it | ||
# cancels those still going itself, which a stop nothing comes to kill | ||
# needs. 65 minutes is that with room to spare; past it, docker kills | ||
# the runner, and the next ends what was cut off as an error and | ||
# removes its containers. | ||
stop_grace_period: 65m | ||
command: | ||
- sh | ||
- -c | ||
| @@ -213,6 +214,7 @@ services: | ||
--state ${KNOTLESS_RUN_DATA:-/var/lib/knotless-run}/state.json | ||
--work ${KNOTLESS_RUN_DATA:-/var/lib/knotless-run}/work | ||
--caches ${KNOTLESS_RUN_DATA:-/var/lib/knotless-run}/caches | ||
--stop-wait 3600 | ||
${KNOTLESS_RUN_ARGS:-} | ||
${KNOTLESS_RUN_REPOS:-} | ||
environment: | ||
deploy/README.md+6−5
| @@ -454,9 +454,10 @@ coverage `test` uncapped takes about 9 GiB and every core, and two at | ||
once stalled the host. | ||
| ||
A restart, a deploy's among them, waits only for the runs whose steps have | ||
started, a deploy's included, up to `--stop-wait` (15 minutes), when it | ||
cancels those still going, and within `stop_grace_period` (20 minutes) | ||
in any case: | ||
started, a deploy's included, up to `--stop-wait` (an hour, so that no | ||
run is cut off before its own limits: a release's build of the client | ||
takes about 17 minutes), when it cancels those still going, and within | ||
`stop_grace_period` (65 minutes) in any case: | ||
each run still waiting its turn is dropped, its check filed as cancelled | ||
(not deleted: see `drop_run` in knotless-run), and the | ||
runner started next runs it, at once, as it was. The checks of a deploy | ||
| @@ -604,8 +605,8 @@ deploy's commit. Once the note is there, `handover.sh` starts the new | ||
runner beside it (`docker compose up --scale`). The new one reads the | ||
state file, leaves the old one's runs, checks and caches alone, and | ||
takes no job the old one's runs hold. The old one exits once its runs | ||
end, `--stop-wait` (15 minutes) after its stop at most, and is removed | ||
then. | ||
end, `--stop-wait` (an hour) after its stop at most, and is removed | ||
then. It holds only the jobs the new one lends it meanwhile. | ||
| ||
The deploy log says how it went: | ||
| ||