1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
# knotless-runner-k8s
Proof of concept for a Kubernetes Job executor for `knotless-run`.
`knotless-run` normally runs one docker container per workflow run. This
crate proves the same shape on a socialweb-computer tenant k3s cluster:
one Job per run, one pod that stays alive, one `kubectl exec` per step.
The runner daemon stays outside the cluster; the Job pod is what makes
the tenant scale a node from zero.
Status: proof of concept. It does not patch `knotless-run` yet. `PLAN.md`
maps this code to `crates/knotless-run/src/executor.rs`.
## Working rules
- Do not open pull requests, merge requests, or send patches to any
upstream repository. All work stays here and in our own forks.
- Target repos are local forks, not the original checkouts:
`~/src/knotless-fork` and `~/src/socialweb-computer-kcp-fork`.
Their `upstream` remotes are read-only; push to `fork` as work lands,
over `knotless://`.
- Never print or commit credential values. `scripts/oauth-signin.ts`
reads the credential file itself.
See `AGENTS.md` for the full rules.
## Build and test
```bash
cargo test
cargo run -- --help
```
Tests use a fake `kubectl`, so no cluster is needed.
## Dry run
```bash
cargo run --quiet -- run \
--name demo \
--image busybox:1.36 \
--workdir examples \
--steps-file examples/steps.tsv \
--outputs "$HOME/.tmp/knotless-runner-k8s/outputs" \
--dry-run
```
This prints the Job manifest and every `kubectl` call without touching a
cluster.
## Live run on a hosted tenant cluster
The gateway is API-only, so sign-in is driven by
`socialweb-computer-kcp/scripts/oauth-signin.ts`. The script below reads
`~/.aliceoa_bsky_social_source` through that script, never itself, and
writes the session cookie and kubeconfig 0600 under
`~/.tmp/knotless-runner-k8s/tenant`.
```bash
scripts/tenant-kubeconfig.sh
export KUBECONFIG=$HOME/.tmp/knotless-runner-k8s/tenant/knotless-poc.kubeconfig.yaml
kubectl get nodes
```
Then run a workflow:
```bash
cargo run --quiet -- run \
--name knotless-poc \
--runner did:plc:runner \
--image busybox:1.36 \
--workdir examples \
--steps-file examples/steps.tsv \
--outputs "$HOME/.tmp/knotless-runner-k8s/outputs" \
--result "$HOME/.tmp/knotless-runner-k8s/result.json" \
--log "$HOME/.tmp/knotless-runner-k8s/run.log"
```
The Job is deleted unless `--keep` is given. A deleted Job lets the
tenant return to zero nodes after its scale hold.
### Scoped kubeconfig
`scripts/tenant-kubeconfig.sh` returns the gateway's admin kubeconfig.
Make a runner-scoped one from it:
```bash
ADMIN_KUBECONFIG=$HOME/.tmp/knotless-runner-k8s/tenant/knotless-poc.kubeconfig.yaml \
scripts/scoped-kubeconfig.sh
```
That creates a ServiceAccount, a namespaced Role for Jobs, pods, exec,
logs, events and NetworkPolicies, and a 24-hour token kubeconfig at
`~/.tmp/knotless-runner-k8s/tenant/knotless-runner.kubeconfig.yaml`, mode
0600. The script checks that the token can create Jobs but cannot read
nodes.
To delete the tenant cluster:
```bash
DELETE=1 scripts/tenant-kubeconfig.sh
```
## How a run works
1. `kubectl apply` a `batch/v1 Job` whose pod runs `sleep infinity`.
2. Wait for the pod to be Ready.
3. Stream the checkout to `/work` with tar over `kubectl exec -i`.
4. Stream shared steps to `/steps`.
5. Run each step with `kubectl exec <pod> -- sh -ec <script>`, appending
stdout and stderr to one log.
6. Stream `/outputs` back, if asked.
7. Delete the Job.
A step timeout kills the local `kubectl` client and then the Job cleanup
deletes the pod, which stops the remote process. The proof does not yet
use a Kubernetes client crate; it shells out to `kubectl`, as the docker
executor shells out to `docker`.
## Result
```json
{
"name": "knotless-poc",
"job": "knotless-poc",
"namespace": "default",
"pod": "knotless-poc-abcde",
"status": "success",
"steps": [
{"name": "date", "status": "success", "exitCode": 0, "durationMs": 42, "timedOut": false}
],
"log": "/home/.../run.log",
"outputs": "/home/.../outputs",
"jobDeleted": true,
"dryRun": false,
"error": null
}
```
The step fields mirror `dev.knotless.check`'s `steps` array.
## Not in this proof
- workflow YAML parsing; steps are `name<TAB>script`
- caches and persistent volumes
- secret and variable injection
- XRPC cancellation
- queue landing
- a scoped runner identity for the public tenant route; `getKubeconfig`
is cluster-admin by construction, and `internal tenantcredential` is
for the loopback apiserver
See `PLAN.md` for the integration plan and the remaining risks.
