chadtmiller.com/knotless

Switch repository

Loading…

What changes

A spike, not for landing yet: the test check measures itself. It prints cargo's build timings unit by unit, and samples host CPU, this container's CPU and its throttling every 5s. Knobs in test.yml's env force a cold build of the workspace's crates, no stored runs, both nets, and set the jobs and harnesses, so each push measures one configuration.

Why

To make the test check faster with numbers before and after: whether the runs are CPU-bound or waiting, and where the 154s build goes.

Shortcomings

Not to be queued: it turns off the stored runs.

Changeset: none

🤖 Generated with Claude Code

This spike adds measurement to the test check. It passes --timings to the build and prints the 25 slowest units. A new harness/exp/sample.sh samples host CPU, cgroup CPU and throttling every 5s. The knobs in test.yml's env force a cold build, drop the stored runs and run both nets.

I read the diff and found nothing that would break the measurement. readFileSync, join and target are already defined in cov.mjs, and a failure to read the timings report is caught and logged. The touch ... || true and the kill $sampler paths are fine under a stopping shell.

You said it isn't for landing, so keep it off the queue. EXP_NOSTORE turns off the stored runs, and EXP_COLD, EXP_NETS and the sampler would slow every pull's test check. To land any of it later, revert the knobs in test.yml and keep only what you want, such as the --timings printing. Two things I did not check: that target/cpu.log exists when the sampler is killed early, and that the --timings report lands under target/llvm-cov-target.

This round adds a second spike commit. harness/exp/replay.sh takes the checkout back before main's last five commits to crates/, lexicons/ and web/src/, then replays them one at a time through cov.mjs with COV_DIFF=1. That mode reuses a stored run when the change missed what the run exercised, using harness/reuse.mjs. The replay data is five patch files, which is why the diff is so large. The first commit's measurement knobs are unchanged.

I found nothing that must change for a spike. replay.sh is not run by any workflow, so it doesn't touch the test check. It does git apply -R on a working tree, so run it only in a throwaway checkout. The test and check runs for this head were still queued when I looked, so I haven't seen the measurements.

My earlier ask still applies: keep this off the merge queue. EXP_NOSTORE, EXP_COLD and EXP_NETS slow every pull's test check. If you land the reuse work (reuse.mjs and COV_DIFF), split it into its own pull without the knobs and the patch fixtures. It also needs its own tests, since a wrong reuse would hide failing runs.

This round adds EXP_SKIP_TEST, set to "1" in test.yml. It skips the test step and the answers and plans steps that read its report, so the replay can be timed alone. It also sets CARGO_BUILD_JOBS=4 in replay.sh, filters the replay's log output harder, and prints a count of uncovered lines.

I found nothing that must change. The [ -n "$EXP_SKIP_TEST" ] && exit 0 lines are not the last command in their scripts, so an empty variable doesn't fail the step under set -e.

My earlier asks are unchanged, and nothing here answers them. Keep this off the merge queue. With EXP_SKIP_TEST=1 the test check runs no tests, so a green check means nothing. Pair that with EXP_NOSTORE and the cold-build knobs. If the reuse work lands, split it into its own pull without the knobs or the patch fixtures, and give it its own tests.

This round changes how replay.sh base builds the starting tree. It used to reverse-apply the five patches. Now it deletes crates, lexicons and web/src, except the fonts and assets, and unpacks a new base.tar.gz. That makes the replay independent of what main is now.

I found nothing that must change for a spike. The find ... -delete is destructive, and so was the earlier git apply -R. Run it only in a throwaway checkout. The deletion also removes files outside the excluded paths, so the tarball has to carry every file the base tree needs. I couldn't inspect the binary tarball, so I haven't checked that it does.

My earlier asks still stand and this round doesn't address them. Keep this off the merge queue, because EXP_NOSTORE, EXP_COLD, EXP_NETS and EXP_SKIP_TEST=1 make the test check meaningless or slow. If the reuse work lands, split it into its own pull without the knobs or the fixtures, and give it its own tests.

This round adds harness/exp/proc.sh, which samples every process's CPU time every 2s. The test step runs it next to the CPU sampler and prints CPU seconds by command. The round also removes the six replay steps and sets EXP_SKIP_TEST back to empty, so the tests run again.

I found nothing that must change for a spike. The field offsets are right: after splitting at ") ", f[12] and f[13] are utime and stime, fields 14 and 15. The sleep 3 after kill is long enough for the script's TERM trap to run, because the trap waits out the script's current sleep 2. The TERM trap writes proc.log before the cat.

Processes that live under 2s are missed, as the script's own comment says. Each pid's last sample is counted, so a reused pid would be overwritten, which is minor.

My earlier asks are unchanged. Keep this off the merge queue, because EXP_NOSTORE, EXP_COLD, EXP_NETS and the two samplers slow the test check. If the reuse work (reuse.mjs, COV_DIFF) lands, split it into its own pull without the knobs or the patch fixtures, and give it its own tests.

Spike done; its findings and numbers are in A pull's test run goes against both nets at once, as a batch's does, which lands the one change they support. Closed unmerged: reuse across builds and the SQLite pragma measured as no help. Branch ci/test-speed kept for its code.