chadtmiller.com/knotless

Switch repository

Loading…

What changes

The firehose backs off. When a relay refuses a connection, knotless-serve now waits twice as long before each new attempt, up to 60 s, the same way Jetstream does. Before, it tried again every 500 ms for as long as the relay was down. After a connection, the wait goes back to 500 ms.

Batches are skipped by the pull trigger's paths. A merge-queue batch is pulls landing, so a workflow whose pull trigger lists paths is skipped when the batch changes none of them, as it is for each pull. Before, a batch was filtered by the push trigger's paths, so workflows with paths only under pull (web, lexicons, shots) ran on every batch.

The registry forgets staged blobs one at a time. Before, once 1024 blobs were staged across all accounts, the registry emptied the whole map, which broke every push still under way. Now an account past 256 staged blobs loses only its own oldest, and past 1024 in all only the oldest of everyone's goes. A push under way stages its blobs last, so it keeps them.

Descriptions match the runner. dev.knotless.check describes how a batch's run is keyed, says baseCommit is set for batches too, and lists every reason a run is cancelled. A step's exitCode is described as absent when the step is cancelled as well as when it runs out of time. The comment at the top of queue.rs says a head that moves mid-run is caught when the batch lands. Only descriptions change in the lexicon.

Why

Fixes Firehose reconnects every 500 ms with no backoff while a relay is down Fixes knotless-run: merge queue batches skip workflows by the push trigger's paths Fixes Registry forgets every pending pushed blob once 1024 are staged Fixes Stale descriptions: queue.rs module comment and check.json

All four were small and in one place each. They were found in an audit of the open issues against main.

How it was checked

There are new unit tests for the firehose's wait (a_refused_relay_is_asked_less_often) and for staging (staging_forgets_only_the_oldest). The batch change is one condition, and its existing runs cover it. Nothing was built here: CI runs the tests.

Shortcomings

A staged blob is still kept after its release is made, until it ages out. It is not needed then, because a release's blobs are found by digest.

🤖 Generated with Claude Code

This pull makes four changes, and I found nothing that must change.

• Firehose backoff: next_wait doubles the wait after each refused connection, up to RETRY_MAX, and resets it to RETRY after a connection. The first refusal waits 1 s, not 500 ms. That is harmless.
• Batch skipping: pull is now true for a batch, so the pull trigger's paths decide whether a batch is skipped. The docs match this.
• Staged blobs: a per-account cap and a global cap each drop only the oldest entry, and the new test covers both. bounded stays because other code still uses it.
• Descriptions: the lexicon and queue.rs changes are comments and descriptions only.

I did not run the tests or check the CI results. I also did not check that a batch's changed set is computed against the branch tip, as the new docs say.