chadtmiller.com/knotless
git that lives in your Atmosphere account
1
10
A pull's test run goes against both nets at once, as a batch's does
.knotless/workflows/test.yml+7−7
| @@ -1,8 +1,7 @@ | ||
# The tests, under `just cov`'s gate: every test binary against the npm | ||
# reference PDS and the spaces alpha, failing under 100% of lines or | ||
# functions; on a pull, against the stock PDS first, and the alpha too | ||
# only when the gate falls short without it. Then that every XRPC answer | ||
# they received is as its lexicon says. | ||
# functions, a pull's run as the merge queue's batch. Then that every | ||
# XRPC answer they received is as its lexicon says. | ||
# | ||
# Node, the harness's packages, the alpha's build, cargo-llvm-cov and the | ||
# build are kept between runs; each is made again only when what it is | ||
| @@ -70,10 +69,11 @@ steps: | ||
# never run, and nothing need be cleaned first. | ||
# Half the host's cores, as two test runs go side by side, over two | ||
# PDSes of each net, as one PDS is one node process. | ||
# A pull's run, its pre-flight, goes against the stock PDS alone, to | ||
# be quick, and against the alpha too only when the gate falls short | ||
# without it; the merge queue's batch runs both. | ||
COV_NETS=${KNOTLESS_RUN_PULL:+stock} COV_STORE=/opt/testcache COV_JOBS=$(( $(nproc) / 2 )) COV_HARNESSES=2 COV_TOOLCHAIN=$(rustup show active-toolchain | cut -d' ' -f1) \ | ||
# A pull's run, its pre-flight, goes against both nets at once, as | ||
# the batch's does: one against the stock PDS first fell short of the | ||
# gate in every pull that changed a test binary, then ran the alpha | ||
# after it, which cost a minute more than running both side by side. | ||
COV_STORE=/opt/testcache COV_JOBS=$(( $(nproc) / 2 )) COV_HARNESSES=2 COV_TOOLCHAIN=$(rustup show active-toolchain | cut -d' ' -f1) \ | ||
node harness/cov.mjs /opt/atproto/packages/dev-env | ||
- name: answers | ||
run: | | ||
docs/design/merge-queue.md+5−4
| @@ -229,10 +229,11 @@ failure ejects only on the current tip: what failed against an older one | ||
the branch may have fixed since. | ||
| ||
A pre-flight may also check less than the batch does, for speed. | ||
knotless's own `test` does: a pull's run tests against the stock PDS | ||
alone and only reports a coverage shortfall, since lines only the spaces | ||
alpha reaches go unrun, and the batch's run tests against both and holds | ||
the 100% gate. | ||
knotless's own `test` checks the same: a pull's run tests against the | ||
stock PDS and the spaces alpha and holds the 100% gate, as the batch's | ||
does. Testing against the stock PDS first and the alpha only when the | ||
gate fell short was slower: nearly every change that reached a test | ||
binary fell short, and then ran the two one after the other. | ||
| ||
An ejected pull never entered a batch. Its reason names the head, the | ||
branch and its tip, and the failed workflows, and `checks` points at | ||
harness/cov.mjs+5−7
| @@ -37,13 +37,11 @@ | ||
// a busy runner is slow: spaces forge takes 70s on a quiet one and near | ||
// 300s when many runs share it. | ||
// | ||
// COV_NETS=stock runs against the stock PDS alone, as a pull's check | ||
// does to be quick: the spaces alpha's runs count only as stored. Should | ||
// the gate then fall short, the runs it left are made too, against the | ||
// alpha, and the gate holds: a line only the alpha reaches is then | ||
// reached, and one nothing reaches fails the pull, not a batch after it. | ||
// Most pulls change nothing the stored runs do not already reach, and | ||
// never start the alpha. | ||
// COV_NETS=stock runs against the stock PDS alone: the spaces alpha's | ||
// runs count only as stored. Should the gate then fall short, the runs it | ||
// left are made too, against the alpha, and the gate holds. CI runs both | ||
// at once instead: a change that reaches a test binary almost always falls | ||
// short without the alpha, and the two one after the other take longer. | ||
// | ||
// COV_JOBS sets how many run at once; COV_CLEAN=1 builds every workspace | ||
// crate again first; COV_ONLY=a,b runs only the test binaries named, to | ||
AGENTS.md+3−5
| @@ -197,11 +197,9 @@ on every pull. `main` deploys only once they all pass: | ||
passed before is not run again: when it, the workspace binaries it ran | ||
and the harness are the same, its stored profile counts instead | ||
(`COV_STORE` in harness/cov.mjs), so a change runs only the binaries it | ||
changes. A pull's run, its pre-flight, goes against the stock PDS | ||
alone (`COV_NETS=stock`) and, should that fall short of the gate, makes | ||
the runs against the spaces alpha it left too, then holds the gate: | ||
a line only the alpha reaches costs a pull a few minutes, and a line | ||
nothing reaches fails the pull rather than a batch. A binary that fails is run once more; one that then passes is | ||
changes. A pull's run, its pre-flight, goes against both nets and | ||
holds the gate, as a batch's does, so a line nothing reaches fails | ||
the pull rather than a batch. A binary that fails is run once more; one that then passes is | ||
logged `FLAKY: <net> <binary> <test>` with its first failure: fix it. | ||
One that fails again only in tests `harness/quarantine.json` names, | ||
until the day it gives, passes too, logged `QUARANTINED: <net> <binary> | ||