chadtmiller.com/knotless

Switch repository

Loading…

What changes

/_status watches the disk. knotless-serve's /_status now reports the disk its cache, and so its mirrors, is on (KNOTLESS_CACHE_DIR). The answer gets a disk object with the path, a state, free and total bytes, and an error if the disk could not be asked.

• ok is 15% free or more.
• low is under 15% free. It is listed among the problems, but the answer stays 200.
• full is under 5% free, and the answer is 503.
• unknown means the disk could not be asked, and the answer is 503 too.

The thresholds are a pure function, disk_state, with constants DISK_LOW and DISK_FULL. Only the running server watches a disk: App::build leaves it unset, so tests do not depend on how full their machine is.

knotless status shows it on a disk line, for example disk /data/cache ok (low), 18G free of 150G. The field is optional, so the client still reads older servers' answers.

Why

On 2026-10-31 prod's disk filled, and knotless.dev answered 502 with git index-pack: fatal: write error: No space left on device. Nothing warned before it happened. With this, the external monitor deploy/README.md describes fails at 5% free. Its keyword monitor on "ok":true and knotless status show the low state from 15%. On prod's 150 GB that is about 22 GB, a day or more of deploys and runs, which leaves time to clear space.

How it was checked

Unit tests cover the thresholds at their edges and the problem text for each state. Another test asks a real disk through a directory not made yet, and one more asks a path with no existing directory at all, which gives unknown and failing. The render test covers the disk line in each state. Everything else is left to CI. Nothing was built locally.

Shortcomings

Only the disk serve's cache is on is watched. On prod that is the root disk, the same one the runner's caches and Docker use, so it covers them, but a host with separate volumes would only be warned about the cache's. The 15% warning reaches a monitor only through the keyword check, because the status code stays 200 until 5%.

🤖 Generated with Claude Code

This adds a disk object to /_status. It reports ok, low, full or unknown for the disk under KNOTLESS_CACHE_DIR, using statvfs on the nearest existing ancestor of the path. full and unknown make the answer 503, and low only adds a problem line. The client's Disk field is optional, so older servers' answers still parse. App::build leaves disk unset, so tests don't depend on how full the machine is.

I read the diff but didn't build it or run the tests; the PR says nothing was built locally either. I found no bugs:

• rustix is already a workspace dependency with the fs feature, so the new use in knotless-serve needs no feature change.
• The thresholds, the total.max(1) guard against a zero total, and the use of f_bavail (space available to unprivileged writes) are right.
• The docs, README, CLI reference and changeset are updated with the change.

CI is the remaining check.