0003 — socialweb-computer kcp MCP servers

Status: accepted

Date: 2026-11-02

Context

asyncai already runs stdio MCP servers through rmcp. The first server, asyncai-mcp-simple, exposes one HTTP GET tool. The next tool surface is the set of kcp instances that socialweb-computer starts beside each application instance.

A socialweb-computer installation is not one cluster. Every application owns a kcp. A bidder keeps its kcp under /var/lib/socialweb-computer/swc-bidder/handles/<hash>/kcp; a production gateway keeps one under /var/lib/socialweb-computer/socialweb/kcp; a dev network keeps one under its state root. Each kcp exposes a Kubernetes API with a root logical cluster and child workspaces. The state root also holds the admin kubeconfig.

Discovery against hat-7 and the production gateway found the same shape:

Fact	hat-7	gateway (socialweb.computer)
kcp front	127.0.0.1:33059 to slot 127.0.0.1:40359	kcp API on 127.0.0.1:6443
admin kubeconfig	/var/lib/socialweb-computer/swc-bidder/handles/<hash>/kcp/admin.kubeconfig	/var/lib/socialweb-computer/socialweb/kcp/slot-a/admin.kubeconfig
identity	shard-admin, system:masters	shard-admin
workspaces	root, root:app, root:socialweb-provider	root, root:app, root:socialweb-provider, account workspaces
access	loopback only; root-owned files; agents used ssh + sudo -n kubectl	loopback only; root-owned files
existing read-only role	none	none

Two facts constrain the design.

First, the admin kubeconfig is a bearer token in system:masters. Every agent session inspected reached for that file directly, so every read ran with full write authority. A read tool that accepts the admin kubeconfig is therefore not read-only in practice.

Second, a kcp service account token is scoped to one logical cluster. A token minted in root cannot read root:app; each workspace needs its own token and its own binding. "Read-only credentials" are plural by construction.

The kcp API is reachable only on the host's loopback interface. Tests must not require a live cluster, a fixed port, or a real state path, so all transport and file access goes through a command runner that tests replace with a fake kubectl.

Decision

Add two stdio MCP binaries:

Binary	Tools	Credential
socialweb-computer-kcp-readonly	discovery, workspaces, api-resources, list/get, logs, events, credential status	a freshly minted, workspace-scoped service account token
socialweb-computer-kcp-readwrite	the read tools plus server-side apply, delete, patch, scale	the admin kubeconfig

Shared implementation lives in src/mcp/servers/socialweb_kcp/.

src/mcp/servers/socialweb_kcp/
  config.rs      clap flags, no baked-in endpoint
  runner.rs      local or `ssh <host> sudo -n` command execution
  discovery.rs   find kcp admin kubeconfigs, read context, walk workspaces
  auth.rs        ServiceAccount/RBAC bootstrap and TokenRequest
  service.rs     kcp resolution, credential cache, protected-host guard
  tools.rs       tool implementations
  readonly.rs    rmcp router for the read-only binary
  readwrite.rs   rmcp router for the read-write binary
src/mcp/servers/socialweb_kcp_readonly/main.rs
src/mcp/servers/socialweb_kcp_readwrite/main.rs

The read-only server never uses the admin kubeconfig for a read. On the first tool call for a workspace it:

1. resolves the kcp from discovery;
2. applies one manifest containing a socialweb-computer-kcp-readonly ServiceAccount, a ClusterRole with get, list, and watch, and a ClusterRoleBinding in that workspace;
3. calls kubectl create token ... --duration=900s;
4. writes a token-only kubeconfig with the workspace URL and the admin kubeconfig's CA data, mode 0600, no client certificate and no client key;
5. runs every read with --kubeconfig <token-kubeconfig>.

Tokens are cached per (kcp, workspace) and refreshed when less than three minutes remain. The token itself is never returned by a tool; the credential tool reports only the private kubeconfig path and lifetime.

Discovery searches state roots for admin.kubeconfig under a kcp directory, excludes shared, groups slot directories under one kcp root, and prefers the kcp-root file over a slot file. --root and --kubeconfig are repeatable. Nothing is searched unless configured; the default roots are the documented state locations.

--ssh <host> makes the runner execute every command on that host. The admin kubeconfig and minted token files never leave the host. The kubeconfig's server URL is read from the file at runtime; no host or port is compiled in.

Protected hosts (socialweb.computer, hat-7, hat-9) refuse every operation until --allow-prod is passed. This includes the read-only server, because minting its credential creates RBAC objects.

--mcp-stdio wires either binary into asyncai:

./target/debug/asyncai \
  --mcp-stdio "./target/debug/socialweb-computer-kcp-readonly --ssh hat-7 --sudo --allow-prod"

Invariants

1. No endpoint or port is compiled in. A server URL comes from a kubeconfig read at runtime or from --ssh plus that kubeconfig.
2. The read-only binary never passes the admin kubeconfig to get, logs, api-resources, or any other read command.
3. A read-only token is minted per workspace. Tokens are never reused across workspaces.
4. A minted token has a 900 second lifetime and is refreshed before it expires.
5. A token kubeconfig contains a bearer token, CA data, and an optional TLS server name. It contains no client certificate and no client key, and is mode 0600.
6. Tool output and command logs never contain the admin token, the minted token, or kubeconfig contents.
7. The read-only tool set contains no mutating verb. Its only writes are the RBAC bootstrap and the TokenRequest.
8. The read-write server uses the admin kubeconfig and no read-only token.
9. apply_object validates apiVersion, kind, and metadata.name, and rejects a List and generateName.
10. Discovery excludes shared admin kubeconfigs and collapses slot files under their kcp root.
11. Protected hosts require --allow-prod, in both binaries.
12. Tests use a fake kubectl, a temporary state directory, and an ephemeral loopback port. No test reaches a real kcp.
13. apply_object with dry_run performs server-side admission and stores nothing.
14. Object-returning kubectl calls use -o json, so code mode (ADR 0004) parses them into Python dicts and the model filters them with the stdlib. get_api_resources converts kubectl's wide table into a JSON array of {name, short_names, api_version, namespaced, kind, verbs, categories}. delete_object uses -o name; get_logs is plain text. Output is never truncated. A failed tool returns an MCP result with is_error: true and an error: ... text, so code mode raises it as RuntimeError.
15. JSON tools declare an MCP outputSchema and return structuredContent (plus the same JSON as text). Because structuredContent must be an object, array results are wrapped: list_workspaces returns {"workspaces": [...]} and get_api_resources returns {"resources": [...]}. Kubernetes objects and lists use a shared KubeObject / KubeList schema. Each description ends with a one-line Python example for code mode.
16. Tools carry MCP annotations. Read tools are read_only_hint: true. apply_object, patch_object, and scale_object are read_only_hint: false, destructive_hint: false. delete_object is destructive_hint: true. Code mode logs every non-read-only call at warn.
17. The kcp argument is optional. When omitted, the server uses the only discovered kcp, or fails with the list of kcp ids if there are several.
18. Watchable resources (ADR 0005): kcp-events://watch/{workspace}/{resource}{?namespace,labelSelector,fieldSelector,kcp} is an event-log resource. One watcher per URI seeds with a list, streams kubectl get -w --output-watch-events -o json --request-timeout=0 with the read credential (minted token), dedupes on uid + resourceVersion, strips managedFields, keeps a 256-event ring with seq, pushes notifications/resources/updated via subscriptions/listen, and reconciles by re-listing after a stream ends. It gives up after 5 consecutive failures with an ERROR event. No polling. MODIFIED events (streamed or reconciled) carry old, the previous known object.
19. kcp-api://{workspace}/resources{?kcp} is a readable, non-watchable resource with the same JSON as get_api_resources.
20. list_objects / get_object with an unknown resource type append did you mean: ... (up to 3 names from api-resources: plural, short name, kind, name.group; a -/_ group typo maps to .).

Test plan

Hermetic tests in tests/socialweb_kcp_mcp_stdio.rs drive the real binaries through McpToolExecutor:

• the read-only server lists list_kcps, list_workspaces, get_api_resources, list_objects, get_object, get_logs, get_events, and readonly_credentials, and does not list apply_object or delete_object;
• discovery reads a context from the fake kubeconfig and reports the state path;
• a recursive workspace walk visits root and root:app and returns both;
• list_objects triggers the RBAC bootstrap and TokenRequest, then reads with --kubeconfig <temp token file>;
• the generated token kubeconfig contains the minted token and CA data, and contains no client certificate, client key, or admin token;
• the token kubeconfig is mode 0600;
• the credential tool reports a lifetime and never echoes the token;
• the read-write server lists the write tools and apply_object runs kubectl apply --server-side with the admin kubeconfig and sends the manifest on stdin;
• object-returning calls pass -o json; get_api_resources returns a JSON array;
• apply_object rejects a manifest without metadata.name before running kubectl.

Unit tests cover kubeconfig context parsing, workspace URL derivation, remote shell quoting, token kubeconfig construction, and API-resource table to JSON conversion.

Verification

$ cargo fmt --check
$ cargo clippy --all-targets -- -D warnings
$ cargo test
running 3 tests
test apply_rejects_a_manifest_without_a_name ... ok
test readwrite_server_applies_with_admin_credentials ... ok
test readonly_server_discovers_and_reads_with_a_minted_token ... ok

All Rust tests pass without a kcp, a state directory, an SSH host, or a known port.

Live verification is separate from cargo test. The read-only binary ran over SSH with --ssh hat-7 --sudo --allow-prod:

Call	Result
list_kcps	one kcp, discovery path from the host, running: true
list_workspaces recursive	root:app and root:socialweb-provider, both Ready
get_api_resources in root:app	served resource list
list_objects computevms in root:app	empty List
readonly_credentials in root:app	token kubeconfig written mode 0600, lifetime 900 s
list_objects workspaces in root	two workspaces, read with the root token

The read-write binary ran with --ssh hat-7 --sudo --allow-prod:

Call	Result
list_objects computevms in root:app	empty List with the admin kubeconfig
apply_object dry run of a ConfigMap	object returned by server-side admission, not stored

The dry-run object was checked afterwards and was absent:

Error from server (NotFound): configmaps "mcp-dry-run-probe" not found

The minted identity on the host:

Username  system:serviceaccount:default:socialweb-computer-kcp-readonly
Extra     authentication.kcp.io/scopes [cluster:1mrhuj6frnoznfpq]

Permission probe with that token, in root:app:

create configmaps  no - access denied
list computevms    yes
list workspaces in root  no - access denied

The last line is the workspace scoping invariant in action: the root:app token cannot read the root workspace. A later read in root minted a separate root token and succeeded.

The read-only ServiceAccount, ClusterRole, and ClusterRoleBinding were present in the workspace after the call.

Consequences

Reads from the read-only server are genuinely read-only: the admin token is used only to create three RBAC objects and a TokenRequest, and every read after that carries a workspace-scoped token that the server did not possess before the first call.

The bootstrap leaves one ServiceAccount, one ClusterRole, and one ClusterRoleBinding per workspace. They are named deterministically and applying them again is a no-op. Deleting them revokes future tokens but does not revoke tokens already minted; those expire within fifteen minutes.

The server shells out to kubectl rather than embedding a Kubernetes client. This keeps the implementation small and uses the kubeconfig and TLS behavior operators already rely on. It requires kubectl on the target host, which hat-7 has and the production gateway obtains from its embedded k3s. A future change can replace the runner with a native client behind the same tool surface.

The read-only role grants get, list, and watch on every resource, including Secrets. It is read-only, not least-privilege. A caller that needs a narrower surface should replace the generated ClusterRole; the tool surface does not change.

--allow-prod is a deliberate speed bump, not an authorization boundary. The admin kubeconfig remains the real authority in the read-write binary.