knotless-runner-k8s

Proof of concept for a Kubernetes Job executor for knotless-run.

knotless-run normally runs one docker container per workflow run. This crate proves the same shape on a socialweb-computer tenant k3s cluster: one Job per run, one pod that stays alive, one kubectl exec per step. The runner daemon stays outside the cluster; the Job pod is what makes the tenant scale a node from zero.

Status: proof of concept. It does not patch knotless-run yet. PLAN.md maps this code to crates/knotless-run/src/executor.rs.

Working rules

• Do not open pull requests, merge requests, or send patches to any upstream repository. All work stays here and in our own forks.
• Target repos are local forks, not the original checkouts: ~/src/knotless-fork and ~/src/socialweb-computer-kcp-fork. Their upstream remotes are read-only; push to fork as work lands, over knotless://.
• Never print or commit credential values. scripts/oauth-signin.ts reads the credential file itself.

See AGENTS.md for the full rules.

Build and test

cargo test
cargo run -- --help

Tests use a fake kubectl, so no cluster is needed.

Dry run

cargo run --quiet -- run \
  --name demo \
  --image busybox:1.36 \
  --workdir examples \
  --steps-file examples/steps.tsv \
  --outputs "$HOME/.tmp/knotless-runner-k8s/outputs" \
  --dry-run

This prints the Job manifest and every kubectl call without touching a cluster.

Live run on a hosted tenant cluster

The gateway is API-only, so sign-in is driven by socialweb-computer-kcp/scripts/oauth-signin.ts. The script below reads ~/.aliceoa_bsky_social_source through that script, never itself, and writes the session cookie and kubeconfig 0600 under ~/.tmp/knotless-runner-k8s/tenant.

scripts/tenant-kubeconfig.sh
export KUBECONFIG=$HOME/.tmp/knotless-runner-k8s/tenant/knotless-poc.kubeconfig.yaml
kubectl get nodes

Then run a workflow:

cargo run --quiet -- run \
  --name knotless-poc \
  --runner did:plc:runner \
  --image busybox:1.36 \
  --workdir examples \
  --steps-file examples/steps.tsv \
  --outputs "$HOME/.tmp/knotless-runner-k8s/outputs" \
  --result "$HOME/.tmp/knotless-runner-k8s/result.json" \
  --log "$HOME/.tmp/knotless-runner-k8s/run.log"

The Job is deleted unless --keep is given. A deleted Job lets the tenant return to zero nodes after its scale hold.

Scoped kubeconfig

scripts/tenant-kubeconfig.sh returns the gateway's admin kubeconfig. Make a runner-scoped one from it:

ADMIN_KUBECONFIG=$HOME/.tmp/knotless-runner-k8s/tenant/knotless-poc.kubeconfig.yaml \
  scripts/scoped-kubeconfig.sh

That creates a ServiceAccount, a namespaced Role for Jobs, pods, exec, logs, events and NetworkPolicies, and a 24-hour token kubeconfig at ~/.tmp/knotless-runner-k8s/tenant/knotless-runner.kubeconfig.yaml, mode 0600. The script checks that the token can create Jobs but cannot read nodes.

To delete the tenant cluster:

DELETE=1 scripts/tenant-kubeconfig.sh

How a run works

1. kubectl apply a batch/v1 Job whose pod runs sleep infinity.
2. Wait for the pod to be Ready.
3. Stream the checkout to /work with tar over kubectl exec -i.
4. Stream shared steps to /steps.
5. Run each step with kubectl exec <pod> -- sh -ec <script>, appending stdout and stderr to one log.
6. Stream /outputs back, if asked.
7. Delete the Job.

A step timeout kills the local kubectl client and then the Job cleanup deletes the pod, which stops the remote process. The proof does not yet use a Kubernetes client crate; it shells out to kubectl, as the docker executor shells out to docker.

Result

{
  "name": "knotless-poc",
  "job": "knotless-poc",
  "namespace": "default",
  "pod": "knotless-poc-abcde",
  "status": "success",
  "steps": [
    {"name": "date", "status": "success", "exitCode": 0, "durationMs": 42, "timedOut": false}
  ],
  "log": "/home/.../run.log",
  "outputs": "/home/.../outputs",
  "jobDeleted": true,
  "dryRun": false,
  "error": null
}

The step fields mirror dev.knotless.check's steps array.

Not in this proof

• workflow YAML parsing; steps are name<TAB>script
• caches and persistent volumes
• secret and variable injection
• XRPC cancellation
• queue landing
• a scoped runner identity for the public tenant route; getKubeconfig is cluster-admin by construction, and internal tenantcredential is for the loopback apiserver

See PLAN.md for the integration plan and the remaining risks.