chadtmiller.com/knotless
git that lives in your Atmosphere account
1
10
Deploy, release and publish-lexicons wait for main's checks holding no job
.knotless/workflows/deploy.yml+12−39
| @@ -6,18 +6,17 @@ | ||
# restarted last, once this run is filed, and only if what it runs | ||
# changed (deploy.sh). | ||
# | ||
# The image is built first, while the checks run, as a candidate, | ||
# knotless:build-<commit>, which changes nothing that runs; once they | ||
# pass, it goes up. Until then a deploy gives way to a newer commit: when | ||
# main moves past it, before or during the build or while it waits for | ||
# the checks, it stops there, removes its candidate and passes, leaving | ||
# the deploy to the newer one, as it does when main has moved by the time | ||
# the checks pass. A check that fails removes the candidate too, and the | ||
# next build removes any other left behind. Once it goes up, it runs to | ||
# its end: a newer commit's deploy waits for this one rather than cutting | ||
# it off halfway (superseded: finish). Its checks can wait for a job | ||
# behind other runs, so it waits for them over two steps, each under the | ||
# runner's 30 minutes a step. | ||
# It starts only once the seven checks have passed at its commit | ||
# (`after`), and holds no job while it waits for them, so the checks it | ||
# waits for are never kept from one; a check that fails files it failed | ||
# without running. The image is then built as a candidate, | ||
# knotless:build-<commit>, which changes nothing that runs, and goes up. | ||
# Until then a deploy gives way to a newer commit: when main moves past | ||
# it, before or during the build, it stops there, removes its candidate | ||
# and passes, leaving the deploy to the newer one. The next build removes | ||
# any candidate left behind. Once it goes up, it runs to its end: a newer | ||
# commit's deploy waits for this one rather than cutting it off halfway | ||
# (superseded: finish). | ||
# | ||
# Once up, the image is published as the release of the `knotless-server` | ||
# package at main's version (docs/design/packages.md#images), if that | ||
| @@ -36,6 +35,7 @@ | ||
on: | ||
push: | ||
refs: [refs/heads/main] | ||
after: [check, test, web, lexicons, shots, docs, e2e] | ||
superseded: finish | ||
image: docker:28-cli | ||
secrets: [DEPLOY_SSH_KEY, RELEASE_PASSWORD, RUNNER_HOST_SSH_KEY] | ||
| @@ -157,33 +157,6 @@ steps: | ||
moved && exit 0 | ||
unless-moved deploy/deploy.sh build | ||
[ ! -e ~/.moved ] || deploy/deploy.sh drop | ||
- name: checks done | ||
run: | | ||
[ ! -e ~/.moved ] || exit 0 | ||
# 25 minutes over all seven, and never a failure: the next step judges. | ||
unless-moved on-host ' | ||
end=$(($(date +%s) + 1500)) | ||
for wf in check test web lexicons shots docs e2e; do | ||
left=$((end - $(date +%s))) | ||
[ "$left" -gt 0 ] || break | ||
knotless checks "$REPO" --ref "$REF" --commit "$SHA" --workflow "$wf" --wait "$left" || true | ||
done' || true | ||
[ ! -e ~/.moved ] || deploy/deploy.sh drop | ||
- name: checks pass | ||
run: | | ||
[ ! -e ~/.moved ] || exit 0 | ||
if ! on-host ' | ||
end=$(($(date +%s) + 1500)) | ||
for wf in check test web lexicons shots docs e2e; do | ||
left=$((end - $(date +%s))) | ||
[ "$left" -gt 0 ] || left=1 | ||
out=$(knotless checks "$REPO" --ref "$REF" --commit "$SHA" --workflow "$wf" --wait "$left") | ||
echo "$out" | ||
case "$out" in success*) ;; *) echo "$wf has not passed: not deploying"; exit 1 ;; esac | ||
done'; then | ||
deploy/deploy.sh drop | ||
exit 1 | ||
fi | ||
- name: deploy | ||
run: | | ||
[ ! -e ~/.moved ] || exit 0 | ||
.knotless/workflows/publish-lexicons.yml+7−7
| @@ -18,12 +18,16 @@ | ||
# granted now, so such a run writes the schemas, then fails, saying which | ||
# record to make. | ||
# | ||
# Asked for on any other branch, `knotless rerun <repo> --ref | ||
# refs/heads/<branch> --workflow publish-lexicons`, it plans, and the wait | ||
# for checks and the publish, which run at main alone, are skipped. | ||
# It starts only once main's seven checks have passed at its commit | ||
# (`after`), holding no job while it waits, so the checks are never kept | ||
# from one. Asked for on any other branch, `knotless rerun <repo> --ref | ||
# refs/heads/<branch> --workflow publish-lexicons`, it waits for those of | ||
# them that ran there, plans, and skips the publish, which runs at main | ||
# alone. | ||
on: | ||
push: {refs: [refs/heads/main], paths: [lexicons/**, .knotless/workflows/publish-lexicons.yml]} | ||
asked: {refs: [refs/heads/*]} | ||
after: [check, test, web, lexicons, shots, docs, e2e] | ||
image: node:24.22.1 | ||
secrets: [LEX_PASSWORD, CLOUDFLARE_API_TOKEN] | ||
env: | ||
| @@ -35,10 +39,6 @@ steps: | ||
- uses: chadtmiller.com/steps/publish-lexicons@1 | ||
name: plan | ||
with: {did: "did:plc:yqsg7znbbhfmj3a2nrpnaerk", zone: knotless.dev} | ||
- uses: chadtmiller.com/steps/wait-for-checks@2 | ||
name: checks pass | ||
on: {refs: [refs/heads/main]} | ||
with: {workflows: check test web lexicons shots docs e2e} | ||
- uses: chadtmiller.com/steps/publish-lexicons@1 | ||
name: publish | ||
on: {refs: [refs/heads/main]} | ||
.knotless/workflows/release.yml+1−8
| @@ -14,6 +14,7 @@ | ||
on: | ||
push: {refs: [refs/heads/main], paths: [Cargo.toml]} | ||
asked: {refs: [refs/heads/*]} | ||
after: [check, test, web, lexicons, shots, docs, e2e] | ||
superseded: finish | ||
image: rust:1.98 | ||
cache: [target, /usr/local/cargo/registry, /usr/local/rustup, /opt/tools] | ||
| @@ -36,14 +37,6 @@ steps: | ||
elif [ "$KNOTLESS_RUN_REF" != refs/heads/main ]; then | ||
echo "on $KNOTLESS_RUN_REF: building $version, publishing nothing" | ||
fi | ||
# All seven, over 25 minutes, with the knotless just built. At main | ||
# alone, as publish is, and even when the version is published already, | ||
# as the step cannot see ~/.done: it then waits for checks it does not | ||
# need. | ||
- uses: chadtmiller.com/steps/wait-for-checks@2 | ||
name: checks pass | ||
on: {refs: [refs/heads/main]} | ||
with: {workflows: check test web lexicons shots docs e2e} | ||
- name: tools | ||
run: | | ||
[ ! -e ~/.done ] || exit 0 | ||
.knotless/uses.lock+0−4
| @@ -19,7 +19,3 @@ chadtmiller.com/steps/rust-lint@1: | ||
version: 1.0.0 | ||
commit: 686cdbb38978730bb1ba5d1c999967e48497ce15 | ||
release: at://did:plc:bcgltzqazw5tb6k2g3ttenbj/dev.knotless.release/3mza7iq7tundv.zvxvlxa6fe7s4pnwsnmgmm63aj.sjjb7q6l3fsl3spvqsuzdoe73w | ||
chadtmiller.com/steps/wait-for-checks@2: | ||
version: 2.0.0 | ||
commit: d5b057a2314833fdee9f600c7872db2916b0d0aa | ||
release: at://did:plc:bcgltzqazw5tb6k2g3ttenbj/dev.knotless.release/3mza7iq7tundv.nglt7rswdf5dzx2tmob2gquhpu.6ivl2z3tvmrsq2jsdlkldzd2yd | ||
deploy/README.md+10−10
| @@ -29,16 +29,16 @@ its `_atproto` TXT. | ||
| ||
### From a push to main to the site running it | ||
| ||
`.knotless/workflows/deploy.yml` deploys each push to main, once its six | ||
checks pass, and builds the image while they run: | ||
`.knotless/workflows/deploy.yml` deploys each push to main once its | ||
seven checks (check, test, web, lexicons, shots, docs and e2e) pass at | ||
it. It names them in `after`, so it waits for them holding none of the | ||
runner's jobs, and the checks it waits for are never kept from one. One | ||
that fails files the deploy failed without running. Then: | ||
| ||
1. **build**: `deploy.sh build` builds `knotless:build-<commit>`, a | ||
candidate, which nothing runs: `knotless:current` and the containers | ||
stay as they are. It removes any candidate an earlier deploy left. | ||
2. **checks done**, **checks pass**: waits for check, test, web, lexicons, | ||
shots and docs at this commit, in one container on the host for all six. | ||
One that fails removes the candidate, and the deploy fails. | ||
3. **deploy**: if main is still at this commit, `deploy.sh up` tags the | ||
2. **deploy**: if main is still at this commit, `deploy.sh up` tags the | ||
candidate `knotless:current`, removes the candidate's tag, and rolls | ||
out as below, restarting `run` only if what it runs changed | ||
([When a deploy restarts the runner](#when-a-deploy-restarts-the-runner)), | ||
| @@ -46,14 +46,14 @@ checks pass, and builds the image while they run: | ||
deploys replaced. | ||
| ||
Until it goes up, a deploy gives way: when main moves past its commit, | ||
asked every 30s through the build and the wait, it stops (a build is | ||
asked every 30s through the build, it stops (a build is | ||
cancelled with its client, its cargo cache kept for the next), removes | ||
its candidate and passes, and the newer commit's deploy, which waits for | ||
it (`superseded: finish`), starts. Once it goes up it runs to its end. | ||
| ||
So a push goes live about a minute after the slower of its build and its | ||
checks: the build, two minutes or so for a change to our crates in the | ||
`deploy` profile, hides behind test, and the rollout takes about 30s. | ||
So a push goes live a few minutes after its checks pass: the build, two | ||
minutes or so for a change to our crates in the `deploy` profile, then | ||
the rollout, about 30s. | ||
| ||
The site comes first on the host for CPU, then a deploy and main's | ||
checks, then pulls' runs. serve and pages have `cpu_shares` of 4096 (a | ||