~ knotless Search/
↑↓ move↵ open⇧↵ all results
0
0
main
247 lines · 12.1 KB Raw History

Gateway architecture

Host readings on 2026-10-22 on the installed production gateway socialweb.computer (release binary sha256 30ab2e465a080bbe47ea16e085cdd99d10c640f4c90d438484363a443b31bd92); the source claims were rechecked against commit 1cad64b0 on 2026-10-26.

Shape

One socialweb-computer core owns the gateway and its kcp. The gateway is an application root in that kcp; its components run in the core and its infrastructure runs as HostProcess units. There is no web-app DenoPod and no per-component systemd application unit.

public
  k8s.socialweb.computer ──caddy──> 127.0.0.1:8788  (web-app front)
  kcp.socialweb.computer ──caddy──> 127.0.0.1:6443  (kcp, no client cert)
                              └───> 127.0.0.1:8444  (client-certificate front, with cert)

systemd --user (account socialweb, uid 982), state /var/lib/socialweb-computer/socialweb
  socialweb.slice
  ├─ socialweb-computer-gateway.service                 core: serve gateway kcp
  ├─ socialweb-kcp-gw@a.service / @b.service            blue/green kcp slots
  ├─ socialweb-kcp-gw-kine@a / @b.service               kine on the shared Postgres
  ├─ socialweb-kcp-gw-pg.service                        Postgres (unix socket)
  ├─ socialweb-kcp-gw-front.service                     L4 front for kcp
  └─ socialweb-hp-gw-bao-root-vault.service             OpenBao HostProcess

core goroutines
  ├─ devcore: kcp slot, bootstrap, HostProcess supervisor
  ├─ appctl: Gateway root, PDS, Requester, GatewayAPI, PolicyEngine,
  │          MarketIdentity, Service, Exposure, ComponentLink
  ├─ componentcore: in-process handlers for every component
  ├─ cluster provider: Cluster, NodeClaim, tenant bootstrap, scale driver
  └─ PolicyEngine and StateSync workers

process-local listeners
  127.0.0.1:6443   kcp API
  127.0.0.1:8443   TLS front for the gateway XRPC listener
  127.0.0.1:8444   client-certificate front for kcp
  127.0.0.1:8788   web-app front; its backend is the GatewayAPI local exposure
  127.0.0.1:20000  Pds local Exposure
  127.0.0.1:20001  Requester local Exposure
  127.0.0.1:20002  GatewayAPI local Exposure

The local port numbers are leases and may move between component restarts. The stable entry points are the fixed fronts 6443, 8443, 8444 and 8788. gatewaykcp resolves the 8788 backend from the GatewayAPI local Exposure when the API-only gateway has no socialweb-computer-k8s component.

Public front

/etc/caddy/sites/k8s.socialweb.computer.caddy sends the whole public origin to http://127.0.0.1:8788. That front proxies to the GatewayAPI exposure, which serves:

  • POST /oauth/start, GET /oauth/callback, GET|DELETE /session
  • GET /api/config, POST /api/clusters/{list,create,up,down,delete,kubeconfig}
  • /xrpc/* for the gateway server
  • /downloads/*

/etc/caddy/sites/kcp.socialweb.computer.caddy keeps client-certificate-aware routing: a request without a client certificate goes to kcp 6443; a request with one goes to the 8444 certificate front with X-Socialweb-Client-Certificate, and Caddy authenticates to that front with the gateway's edge-client certificate. Caddy trusts the gateway front certificate at /etc/caddy/socialweb-computer/gateway.crt, the kcp serving certificate at /etc/caddy/socialweb-computer/kcp.crt, and holds the edge client certificate at /etc/caddy/socialweb-computer/edge-client.{crt,key}.

scripts/deploy.sh installs the binary, runs install gateway, copies those four Caddy files from the new state, writes the sites, validates the Caddyfile and reloads Caddy. A fresh install that regenerates state must run this certificate handoff; otherwise Caddy keeps the previous gateway and edge-client certificates and the kcp front returns bad certificate.

The web root returns 404 because this tree is API-only. There is no SPA at /. OAuth is started by POST /oauth/start with {"handle":"..."}; the response is an authorization URL. The browser flow ends at /oauth/callback, which sets the swc_session cookie on k8s.socialweb.computer.

kcp layout

Observed root workspaces:

workspacetyperolephase
socialweb-provideruniversalschemas, exports, provider service accountsReady
appsocialweb-applicationgateway application objects and componentsReady
2wjeqtgxaccountAlice's account workspace (did:plc:lpfuqerea3deuoyrn7ojser4)Ready

The root workspace namespaces are default and kcp-system; the app logical cluster also has default and kube-system. The account workspace is the tenant boundary for Alice's clusters and contains the clusters child mount pointing at her cluster workspace.

Provider exports in root:socialweb-provider: applications, atprotoidentities, clusters, computecontracts, computevms, events, exposures, hostprocesses, marketidentities, pds and policy. The root bindings include computecontracts and computevms in addition to the default application set. socialweb-application binds computevms, so the gateway set applies the computevms resource schema, export and root binding before it creates root:app.

Main objects:

kind/namephasenote
Gateway/gatewayReadyprofile prod, DID did:plc:5ipkeflamx7tz2venxxgjupl
OpenBaoServer/root-vaultReadyroot vault
GatewayAPI/gateway-apiReady/api, /session, /oauth, /xrpc, downloads
Pds/pdsReadyapp-owned PDS
Requester/requesterReadymarket requester
PolicyEngine/policy-engineRunningin-process engine

Services and exposures:

ServicetargetExposure
svc-gatewayapi-gateway-apiGatewayAPI/gateway-apilocal 127.0.0.1:20002; ingress through IngressProxy/ingress
svc-pds-pdsPds/pdslocal 127.0.0.1:20000; ingress through IngressProxy/ingress
svc-requester-requesterRequester/requesterlocal 127.0.0.1:20001; ingress through IngressProxy/ingress

The prod profile defaults spec.ingress to xrpc-subscription with ingressProxyHost: xrpc.fedproxy.com and the app IngressProxy/ingress, so the app PDS owns a public URL. A gateway spec that sets spec.ingress.opentunnel selects opentunnel instead; a spec that sets spec.ingress names its own provider. The market-network profile inherits the dispatcher provider from MarketNetworkValue.ingress. Public k8s traffic still uses the local 8788 front, not an ingress Exposure.

Gateway and requester identity

Gateway/gateway has spec.profile: prod, spec.webAppFront: 127.0.0.1:8788, and options:

network:
  plcDirectoryUrl: https://plc.directory
  ingressProxyHost: xrpc.fedproxy.com
  qrOrigin: ""
ingress:
  provider: xrpc-subscription
  ingressRef: {kind: IngressProxy, name: ingress}
  xrpc:
    ingressProxyHost: xrpc.fedproxy.com
options:
  - {name: public-origin, value: https://k8s.socialweb.computer}
  - {name: gateway-audience, value: did:web:k8s.socialweb.computer}
  - {name: kcp-public-host, value: https://kcp.socialweb.computer}
  - {name: node-policy, value: tangled-vouch}
  - {name: node-policy-args, value: '{"firstFree":true}'}

The Gateway's app-owned PDS is created by appctl, its account session is stored in the root vault at identity/requester/pds-session, and the requester reports its repo DID. The gateway DID above is that requester repo DID, not a browser account DID.

Web sign-in uses the PDS as the OAuth authorization server, with the GatewayAPI as the OAuth client and callback server. The browser session is a cookie held by the gateway; the PDS session is in the root vault, not in the gateway's memory. Alice's observed session:

  • handle aliceoa.bsky.social
  • DID did:plc:lpfuqerea3deuoyrn7ojser4
  • GET /session returns the DID after the OAuth callback
  • POST /api/clusters/list returns the account's clusters

Cluster API and kubeconfig

/api/clusters/* is the HTTP surface used by the browser and by scripts. The same operations are also available as gateway XRPC (com.publicdomainrelay.temp.cluster.*) on the same handler.

For a Ready running cluster, /api/clusters/kubeconfig returns a client-certificate kubeconfig whose server is:

https://kcp.socialweb.computer/clusters/<account>/<cluster>

On the prod profile the kubeconfig has no certificate-authority-data; it uses the system trust store and therefore trusts Caddy's public certificate. A gateway on a loopback or self-signed front keeps the gateway's tls/tls.crt as the CA unless the kcp-public-ca-file option names another file. The client certificate is issued by the cluster's client CA and is presented through Caddy's 8444 certificate front.

A live check before the smoke cluster was deleted:

curl -sS -X POST -H "Cookie: $COOKIE" -H 'Origin: https://k8s.socialweb.computer' \
  -H 'Content-Type: application/json' -d '{"name":"prod-smoke","ttlSeconds":3600}' \
  https://k8s.socialweb.computer/api/clusters/kubeconfig
jq -r .kubeconfig response.json > prod-smoke.kubeconfig.yaml
kubectl --kubeconfig prod-smoke.kubeconfig.yaml get namespaces
# default, kube-node-lease, kube-public, kube-system, users

Deploy and upgrade

The installed layout is:

  • account socialweb (uid 982), state /var/lib/socialweb-computer/socialweb
  • binary slot bin/current -> bin/30ab2e...
  • config /var/lib/socialweb-computer/socialweb/config.json
  • environment /var/lib/socialweb-computer/socialweb/gateway.env (mode 0600, owned by socialweb)
  • spec /var/lib/socialweb-computer/gateway-spec-socialweb.yaml
  • Caddy sites under /etc/caddy/sites, certificates under /etc/caddy/socialweb-computer

A routine upgrade is:

# as socialweb
CONFIG_PATH_SOCIALWEB_COMPUTER=/var/lib/socialweb-computer/socialweb/config.json \
  /var/lib/socialweb-computer/socialweb/bin/current/socialweb-computer \
  upgrade --binary /var/tmp/swc-release/socialweb-computer-<sha> --sha256 <sha>
sudo /var/lib/socialweb-computer/socialweb/bin/current/socialweb-computer install gateway \
  --account socialweb --state-dir /var/lib/socialweb-computer/socialweb \
  --gateway-spec-file /var/lib/socialweb-computer/gateway-spec-socialweb.yaml \
  --cert-front-address 127.0.0.1:8444 --tls-front-address 127.0.0.1:8443 \
  --kcp-front-address 127.0.0.1:6443 --host-backend systemd-user

Then run the Caddy certificate/site handoff from scripts/deploy.sh (or the script itself). An upgrade preserves the kcp store, root vault, requester PDS, DID, account workspaces and clusters.

A fresh install is required when the existing app workspace type is the old denoruntime: the WorkspaceType name is immutable. A fresh install removes the old state and requires the Caddy certificate handoff and a new Alice sign-in.

Live state

  • Gateway Ready: root vault, GatewayAPI, Pds, Requester and PolicyEngine all Ready.
  • Alice sign-in: did:plc:lpfuqerea3deuoyrn7ojser4, swc_session cookie.
  • Account workspace 2wjeqtgx Ready; the smoke cluster was created Ready and then deleted, leaving no clusters.
  • kcp.socialweb.computer/healthz 200; k8s.socialweb.computer/api/config 200; / 404 (API-only).
  • kcp-public-host is https://kcp.socialweb.computer; kubeconfigs use the system CA on prod.

Known gaps

  • No static web UI. A browser sees 404 at /; sign-in is the JSON /oauth/start call.
  • Ingress Exposures are Pending without an IngressProxy; the public path is the Caddy local front.
  • A fresh install must refresh Caddy's gateway, edge-client and kcp CA files. scripts/deploy.sh does this; a bare install gateway does not.
  • The app workspace type is immutable; an old denoruntime install cannot be upgraded in place.

Inspect

K=$HOME/.tmp/prod-kubeconfig-new   # fetched from the host admin kubeconfig
kubectl --kubeconfig "$K" get workspaces,gateway,openbaoserver
kubectl --kubeconfig "$K" get gatewayapi,pds,requester,policyengine,service,exposure -o wide
kubectl --kubeconfig "$K" get apibinding -A
kubectl --kubeconfig "$K" get gateway gateway -o jsonpath='{.spec}{"\n"}{.status}{"\n"}'

On the host, systemctl --user -M socialweb@ list-units shows the core, kcp slots, Postgres, kine, front and OpenBao units.