chadtmiller.com/knotless

Switch repository

Loading…

What changes

harness/cov.mjs runs more of a test binary's tests at once when that binary took long last time. Each binary gets one test thread for every two minutes of thread time its last run took. Thread time is how long the run took multiplied by the threads it had, so the count stays steady as runs get faster. It is never fewer than the CPUs the container has, and at most COV_TEST_THREADS (8 by default). The log names each binary that got more threads. RUST_TEST_THREADS, when set, still applies to every binary.

Why

A test run on the runner host takes about 10 minutes: 3.6 minutes to build, then about 6.4 minutes until the slowest binary, spaces run, finishes. Most other binaries finish in under 2 minutes. libtest runs as many tests at once as the cgroup's CPUs, which is 4 here. The integration tests mostly wait on their PDS, their runner and git rather than compute: one test job averaged about 0.9 of its 4 CPUs over its run, and its CPU limit rarely held it back. So run spent its time running 38 tests 4 at a time, and the whole run waited for it. Every pull and every batch pays this, usually twice for each pull.

More threads only for the slow binaries leaves the total number of tests running at once nearly where it was, so the job's memory stays near what it uses now.

How it was checked

Measured on this pull's own test run. A change to the harness runs every binary again, so its run compares directly with a pull's full run of about 10 minutes.

Shortcomings

The tests run on a developer's machine with as many threads as it has cores, so running them in parallel is not new. But the runner's shared PDS harnesses now take more tests at once, so a test that only fails under load may show up as a FLAKY: line.

This gives each test binary RUST_TEST_THREADS from its last run's thread time. The count is one thread per two minutes of thread time. It is at least the cgroup's CPUs and at most COV_TEST_THREADS. The last run's thread count is stored under a new <net> <name> threads key in cov-times.json.

I read the change against the rest of harness/cov.mjs and found nothing that must change.

• Nothing iterates times, so the extra keys are safe.
• A missing threads entry falls back to the CPU count, so an old times file still works.
• A set RUST_TEST_THREADS is passed through unchanged.
• The "run alone" retries go through one(), so they get the same thread count.
• times is read only when threadsOf is called, which is after its declaration, so there is no TDZ problem.

I did not check the test run's results, and the body gives no measured time from the pull's own run. Compare that run's time with the roughly 10-minute baseline before you rely on the speedup.

Two small notes, neither blocking:

• The COV_ list in the header comment (COV_JOBS, COV_CLEAN, and so on) doesn't name COV_TEST_THREADS. It is only described in the new paragraph above it.
• A non-numeric RUST_TEST_THREADS would become the string "NaN". The old code passed it through to libtest unchanged. This is an edge case.