chadtmiller.com/knotless
git that lives in your Atmosphere account
A test server's OAuth client ID is never one an earlier server had
MergedThis gives each test TLS front its own hostname, <tid>.gateway.knotless.internal. A reused port then no longer produces a client ID an earlier server had. The harness certificate now covers *.gateway.knotless.internal, and the PDS's DNS override resolves those names to 127.0.0.1.
I read the whole diff and found nothing that makes it wrong. knotless::keys::tid() exists and is public. A TID label is valid in DNS and in a certificate wildcard. The endsWith('.' + GATEWAY_HOST) check is anchored on the dot, so it can't match unrelated names. The other places that use the bare name (the e2e Playwright config and the video camera) still work, because the bare name still resolves and is still in the certificate.
I did not run it. The cause you describe (the PDS keeps client metadata for ten minutes by client ID) is consistent with the code change, and the reproduction you give with two gateways forced onto one port is convincing.
Since my approval at 877930d, the only change is that harness/quarantine.json drops the entry for racing_gateways_rebuild_on_the_moved_branch. That entry's until date (2026-11-14) has already passed, so removing it is consistent. If that test still fails on the spaces net, the test check will show it. What I approved still stands.
8 passed, 3 skipped
The gateway binary's tests failed now and then with
invalid_client: Validation of "client_assertion" failed: signature verification failed, or a pages login answering 502, a different test each time (on knotless-serve answers Maven's repository layout for a repository's artifacts,orgs::organizations_with_invitations_only, thenreviewed::protected_branches_take_reviewed_pullsandsubscriptions::watching_and_subscribing_from_the_pages). The PDS took a test server for an earlier one that had had its port. Only the harness and the tests' helpers change; no reader's attention needed beyond a look.Claude Opus 5.5
What changes
Each TLS front has a hostname of its own.
tls_proxy(crates/knotless/tests/common/mod.rs), which every gateway, pages, forge and MCP server the tests start sits behind, now puts its front at<tid>.gateway.knotless.internal:<port>rather thangateway.knotless.internal:<port>, so each server's OAuth client ID is one no other server had.The harness covers those names. harness/pds.mjs makes its certificate for
gateway.knotless.internaland*.gateway.knotless.internal, and resolves both to 127.0.0.1. The bare name still works, for e2e, the scenarios and the video camera, which use it as before. authorize.mjs already resolves every https name to 127.0.0.1, and tls-proxy.mjs serves one certificate whatever the name, so neither changes.racing_gateways_rebuild_on_the_moved_branchleaves quarantine. Its entry in harness/quarantine.json, past its day since 2026-11-14, was for this failure.Why
A gateway signs its
client_assertionwith the key in its own config directory, a new one for each test server, and publishes it in its client metadata, at its client ID,https://gateway.knotless.internal:<front port>/oauth-client-metadata.json(or/oauth/web-client-metadata.jsonfor the pages). The reference PDS's OAuth provider keeps a client's metadata, its JWKS among it, for ten minutes by client ID (clientMetadataCache,ttl: 600e3). The fronts listen on port 0, and every test binary on a harness shares its PDS: once a front comes to a port an earlier server's front had within those ten minutes, the PDS checks the new server's assertion against the old server's key. Both keys have the samekid, so jose finds the key and the signature fails, which is the error the checks showed; the gateway answers the PDS's refusal with a 502.On the runner it is likelier than it looks. Linux gives a socket with SO_REUSEADDR, as node's and tokio's listeners are, a port from the lower half of
ip_local_port_rangefirst, at a random odd offset (inet_csk_find_open_port): about 7,000 ports. The gateway binary alone starts about 50 servers, and the pages, forge and MCP binaries more on the same harnesses, and a binary that fails is run again on them, by then with every earlier server's metadata kept.A key the tests share would have hidden the signature failures, but not the rest of what the PDS keeps of a client (its redirect URIs and scope, which differ between the gateway's, the pages' and the MCP server's metadata at a reused port), and would no longer make a key for each server. Turning the PDS's cache off would change the PDS under test. A client ID no earlier server had is the fault itself fixed: the label is a TID, made from the time and the process, so it never repeats.
Fixes Test racing_gateways_rebuild_on_the_moved_branch fails a client_assertion signature, where
racing_gateways_rebuild_on_the_moved_branchfailed the same signature.How it was checked
A spy loaded into the harness's PDS (
node --import, not committed) logged each confidential client it loaded, with its key, and each assertion it refused, with the key it held and the one the client serves at that moment.The gateway binary, built once from main and once from this branch, run here against harness PDSes with that spy:
oauth_logins_reach_a_gatewaytwice, one PDS)knotless: UpstreamFailure (502): invalid_client: Validation of "client_assertion" failed: signature verification failed; the spy:REFUSED https://gateway.knotless.internal:47123/oauth-client-metadata.json held x=vHBhKN7L0R serves x=3C37247khd3n2vjcxwwvqur.gateway.knotless.internal:47123and3n2vjd5v77sfs.gateway.knotless.internal:47123testcheck runs it, with the fronts taking ports as Linux does (a preload into their node: a random odd port in the lower half of 32768–60999, then the next free one up): 30 fresh PDSes, the binary twice on each, as a failed binary is run again, two PDSes at a time; 60 runssessions::logging_out_one_login_of_several,private_flow::delegating_writes_from_the_pagesandthemes_from_the_pagesat gateway.rs:1598 (502 for 200),org_index::the_appview_counts_organization_membersonaccount delegate(invalid_client). One more failed apart from this, at orgs.rs:371private_flow::private_repositories_made_and_shared_from_the_pagesandprivate_repositories_on_the_pagesgot 502 at gateway.rs:1598Two runs side by side on one PDS also fail
themes_from_the_pagesat gateway.rs:2717, on both builds: it lists everyone's themes the latest first and finds the other run's. The check never runs two gateway binaries on one PDS at once, so that is left out of the counts above (the MCP tests publish themes too, on the spaces net, so it could still meet one there).The
testcheck ran every binary again on both nets, as the harness changed, and all passed.Shortcomings
A
just pdsleft running from before this change has the old certificate and resolves only the bare name: restart it.The rates above are from this Mac, with Linux's choice of port imitated in the fronts alone. The runner shares each PDS among more binaries than these runs did (pages, forge and MCP servers have fronts too), so it likely met the fault more often than 6.7% of runs.