grain.social/grain

Switch repository

Loading…

What changes

Stacked on the flags pull (grain-serve is set by flags, as knotless-serve is), whose compose passes flags: it lands after that one.

An image of grain-serve and its pages. deploy/Dockerfile builds grain-serve, the grain CLI, grain-mcp and grain-import with --profile deploy --locked (a new [profile.deploy] in Cargo.toml: release's, without LTO across crates and in 16 units), stamped with the commit it was built from (GRAIN_COMMIT), beside the SvelteKit app built with adapter-node, under tini as the node user. Its own Dockerfile.dockerignore keeps target/ and node_modules out of the context. The root Dockerfile is the old server's and is left as it is: main and hetzner-infra build it until cutover, so the two coexist by name, and at cutover this one takes the root's place.

/healthz and /_status, as knotless-serve answers them. /healthz is ok while it listens. /_status is 200 or 503 with JSON: Jetstream connected and bringing an event in the last two minutes (the follower now tells the index's feeds as it connects and hears), the index answering a read, the disk under it with room (15% low, 5% full), and the version and commit running; Cache-Control: no-store.

--check-index <file>: says whether the index there is of this build's tables, without changing it, and serves nothing. grain's index migrates rather than rebuilds, so the answer is "keeps it", "would migrate it" (rollout.sh then stops the old one first) or "past this build's" (not started).

deploy/, in knotless's shape: docker-compose.yml (project grain-serve: serve on 2600 and pages on 2601, journald logging, healthchecks with start_interval, every setting a flag from ${GRAIN_*:-}, secrets from /etc/grain-serve/serve.env on the host), deploy.sh (build / up / drop, over SSH to the host's docker, its own build cgroup, the .env written on the host), rollout.sh (no-gap rollover, with the index check), logs.sh (the journal through jq), local.env.example, README.md; just deploy and just logs.

Why

The audit against knotless: grain-serve had no image, health or status route, index check or rollout; knotless has all of them, and grain's move off the old server deploys this way.

Mirrors knotless's Dockerfile, [profile.deploy] in Cargo.toml, /healthz and /_status (crates/knotless-serve/src/lib.rs routes, crates/knotless-serve/src/status.rs, crates/knotless/src/status.rs), --check-index (check_index in crates/knotless-serve/src/lib.rs), and deploy/{docker-compose.yml,rollout.sh,deploy.sh,logs.sh,local.env.example,README.md}. Kept grain-specific: the Dockerfile is deploy/Dockerfile, beside the old server's root one; the pages are built from the root package.json, as the web still is, so the image carries its pruned node_modules (knotless's web is bundled whole); the index check speaks of migrating, not rebuilding; /_status has no mirror probe and adds the version and commit; the feed state lives on the Index rather than App, so no test or pull in flight that builds an App changes; deploy.sh has no runner, triager or reviewer.

The hand-off to hetzner-infra

Production compose stays in ~/code/hetzner-infra (stacks/grain), which this does not change. grain-serve is a second compose project on grain's host, in /etc/grain-serve, as knotless's is in /etc/knotless, reached on an external network, grain-serve, that deploy.sh makes. At cutover hetzner-infra:

• puts Caddy on the grain-serve network, as it is on knotless, and proxies grain's domain to grain-serve:2600 for grain-serve's paths and grain-pages:2601 for the rest (the Caddyfile is in deploy/README.md), or all of it to grain-serve:2600 once grain-serve hands the pages on (--web, in flight elsewhere);
• drops the grain service and its build from grain-deploy;
• writes /etc/grain-serve/serve.env from its secrets (the CDN key and salt, the push key file paths, Clef's token) and moves the push key files to /mnt/data/grain-serve/certs;
• runs grain-import once into /mnt/data/grain-serve/index.db before serve first starts.

deploy.yml then runs deploy/deploy.sh build and up, as knotless's deploy workflow does; until cutover it deploys the old server.

How it was checked

Unit tests: status.rs (feeds, thresholds, the disk, both routes and their no-store), check_index against an index at, behind and past this build's tables, and tests/main.rs runs grain-serve --check-index. Clippy here on grain-serve; docker compose config on the compose file; bash -n/sh -n on the scripts. The checks run the rest.

Shortcomings

• The image has not been built: nothing in CI builds it, and a release build is not run on this machine. The first deploy.sh build on the host is its test.
• deploy.sh has not been run against a host.
• Until grain-serve hands the pages every request it does not answer, the proxy has to know grain-serve's paths.

Review

The status module (crates/grain-serve/src/status.rs) and deploy/docker-compose.yml first; deploy.sh and rollout.sh follow knotless's line by line.

🤖 Generated with Claude Code

This pull adds /healthz and /_status, --check-index, a deploy/Dockerfile, and a compose project with deploy.sh, rollout.sh and logs.sh. It mirrors knotless's deploy shape.

I didn't read the whole diff. It's about 2,500 lines and the tool output was truncated, so I read the files at the head commit instead:

• status.rs
• deploy/docker-compose.yml
• deploy/Dockerfile
• deploy/rollout.sh
• the feeds call sites

I did not read deploy.sh, logs.sh, check_index or its tests, tests/main.rs, the Cargo profile or the README.

I found nothing that must change.

• Status logic: the disk thresholds and the 200/503 mapping are consistent with their tests.
• Rollout: the serve path stops the old container first when the index would migrate, and refuses to start when the index is past the build or the check can't run.
• Env var: GRAIN_SERVE_URL is the name the web code reads.
• Feeds: only Jetstream is registered as a feed, so fed being true when any feed is ok matches the single-feed case. If you add a second feed, one dead feed would give 200 with a problem listed.

All four checks (test, rust, harness, check) were still queued when I looked, so none had passed. As you say, the image has never been built and deploy.sh has never run against a host. The first host build is the real test.

Since my last round the pull has one new commit: grain-serve now shuts down gracefully on SIGTERM as well as ctrl-c. That fits the rollout, because docker stops the old container with SIGTERM through tini. I read the change in lib.rs. The select! over the two signals is correct, and the handler is installed inside the shutdown future. A new test in tests/main.rs sends kill -TERM and checks that the process exits successfully. The tokio::signal::unix call is Unix-only, which is fine for a server image. Nothing from my earlier review needed a follow-up. All four checks were still queued when I looked, so none has passed yet. The image build and deploy.sh are still untested against a host, as the pull says.

Since my last round the pull has one new commit. It reformats two healthcheck.test arrays in deploy/docker-compose.yml across several lines. The commands are character-for-character the same, so nothing changes at runtime. My earlier rounds asked for nothing, so there is nothing to follow up. The image build and deploy.sh are still untested against a host, as the pull says.

Now stacked on the telemetry pull too (grain-serve logs JSON lines, a wide line a request, and slow SQL), merged in, since both changed main.rs, FLAGS and serve(); and on origin/batch/one. So it lands after flags, the batch and telemetry. The merge keeps --check-index ahead of telemetry::init, SIGTERM in serve() around telemetry::observed, and the compose file passes --metrics 127.0.0.1:9464 as knotless's does. Also: deploy/docker-compose.yml formatted as vp fmt checks (the check failure at 5b41fdd).

Since my last round the pull has merged in the batch and telemetry branches, plus one commit that adds --metrics 127.0.0.1:9464 to the serve command in deploy/docker-compose.yml. I read the compose file at the head. The flag binds to loopback inside the container, so it isn't published and can only be read with docker exec, as the comment says. The healthchecks, ports and networks are unchanged. My earlier rounds asked for nothing, so there is nothing to follow up. I did not review the merged batch and telemetry code, since those are separate pulls. All four checks (test, rust, harness, check) were still queued when I looked, so none has passed. The image build and deploy.sh are still untested against a host, as the pull says.

Landed on feature/rust at 164794a, fast-forwarded with infra/justfile (The Justfile runs grain-serve and the web, as knotless's does), which contains this head and passed every check against fc5f686. What the review skipped was read before merging: deploy.sh works only in /etc/grain-serve and /mnt/data/grain-serve under its own compose project, so it cannot touch the running /etc/grain stack; no workflow runs it; logs.sh only reads; deploy.yml now waits on more checks, not fewer.