~ knotless Search/
↑↓ move↵ open⇧↵ all results

johnandersen777.bsky.social/socialweb-computer-console

console.socialweb.computer: kcp, kubectl and ssh compiled to wasm in a browser tab with a terminal; a kcp fork

0
0
main
229 lines · 14.1 KB Raw History

AGENTS.md: browser kcp + socialweb (iPhone) test handoff

This fork (origin https://github.com/johnandersen777/kcp, branch kcp-wasm-memory-storage, tag iphone-working-v0) runs real kcp and the socialweb control plane compiled to js/wasm in a browser tab. The page requests a VM from the socialweb market (bidders run by the lead), waits for it to be Held, and opens a shell on it: ssh over WebRTC from the page's terminal (primary), or the wootty web shell over opentunnel (fallback).

State at iphone-working-v0 (2026-10-25): on an iPhone in Firehose mode (no ingress), request -> Held 40-72 s, ssh datachannel open 4-5 s after the offer (host/udp on the home LAN), release through the tab's EphemeralInfra. Idle allocation ~100-150 MiB/min, 3-4 GCs/min. Open: 1-3 GB/min while a contract runs, from socialweb re-applying unchanged ComponentLinks/EventSources/ EventStreams (~900 no-op applies/min); the lead is fixing it.

Layout

PathWhat
~/src/kcp/kcp-wasmmain checkout on the lead's host (hat-8, Fedora, user johnandersen777), branch kcp-wasm-memory-storage; the dev server serves this tree (page JS from scripts/browser/, wasm from bin/wasm/)
~/src/kcp/kcp-wasm/.claude/worktrees/<name>worktrees (agents get one each): edit and build here, then fast-forward the main checkout
bin/wasm/built wasm, not in git; a fresh worktree copies it from the main checkout (cp -a ~/src/kcp/kcp-wasm/bin/wasm/. bin/wasm/)
~/src/kcp/socialweb-computer-kcpsocialweb source (the lead's; read only for agents; git -c safe.directory='*')
~/.tmp/claude-kcpwasmdev-server debug files (debug.log, control.ndjson, profiles) and the lead's helpers: takeover.sh, tunnel-watchdog.sh, page-hold.ts, boot-smoke.ts
scripts/browser/the page: app.js, kcp_worker.js, oauth_refresh_worker.js, vmshell.js, webrtc_guest.js, ssh_cmd.js, identity_ui.js, debug_link.js (debug log + control stream)
socialweb-wasm/Go glue: cmd/kcp-wasm (wasm main, refresher bridge, memstats), internal/socialweb/orchestrator.go (MarketRequester app, comms mode, events.Register)
contrib/wasm/storagekcp's in-memory storage for wasm (per-resource update counters live here)
scripts/prepare-kcp-patched-modules.shpatches k8s modules for wasm (incl. the JSON fast path for apply bodies)
scripts/kubectl_wasm_web.tsdev server (static files + /debug/* hooks)
scripts/browser-demo.tsdrives a phone through the whole VM test and records it to demos/
scripts/browser-dev/{build-page.sh,phone.sh}build/deploy and one-shot page commands
contrib/wasm/FUTURE_WORK.mdopen items (build-time snapshot secrets, Firehose on a phone)

Build and deploy

export TMPDIR=~/.tmp                                                        # never /tmp
cd ~/src/kcp/kcp-wasm/.claude/worktrees/<name>
REV=<socialweb commit, e.g. main> DEPLOY=1 scripts/browser-dev/build-page.sh   # ~1 min incremental
git add ...; git commit                                                     # commit in the worktree
git -C ~/src/kcp/kcp-wasm merge --ff-only <worktree branch>                 # page JS is live now
git -C ~/src/kcp/kcp-wasm push origin kcp-wasm-memory-storage
# helper modules (kubectl, kubectl-slim, goat, yq, ssh) into bin/wasm, rarely needed:
make build-kubectl-wasm build-goat-wasm build-yq-wasm build-sshclient-wasm  # goat/yq: GO_CLI_TOOLCHAIN=go1.27.3 (the default)

Notes:

  • build-page.sh defaults: SOCIALWEB_SRC=~/src/kcp/socialweb-computer-kcp, MAIN_CHECKOUT=~/src/kcp/kcp-wasm (DEPLOY target), TMPDIR=~/.tmp.
  • The build mirrors socialweb at REV into .work/socialweb-mirror, 3-way merges the compose.go overlay onto it and syncs socialweb-wasm/manifests (commit those). A new socialweb schema needs that sync or the page logs field not declared in schema.
  • Merging into the main checkout deploys page JS immediately; bin/wasm/kcp.wasm is only replaced by DEPLOY=1. Do not merge half-finished page changes while a phone test runs.
  • The page re-applies the socialweb manifests when the kcp.wasm ETag changes (recorded only after a full boot).
  • Commit the user's own uncommitted edits in the main checkout before merging (small commit of their diff), never discard them.
  • Browser unit tests: cd scripts/browser && deno test --allow-read --allow-net.
  • Policy suite, page half (always-run/page/..., names shared with socialweb's policy tests): scripts/browser-dev/policy-suite.sh (deno tests, the headless Chrome fence check, the js/wasm browsersandbox Go tests; needs .work). Storage tests: cd contrib/wasm/storage && GOTOOLCHAIN=go1.26.9 go test ./.

Dev server, public URL, phone

The lead runs the :8090 server and the tunnel; agents do not touch them and start their own server on a free port for tests (no --origin, page at http://127.0.0.1:<port>/?debuglog=1, debug files under ~/.tmp/<own dir>).

D=~/.tmp/claude-kcpwasm; mkdir -p $D
cd ~/src/kcp/kcp-wasm && deno run --allow-read --allow-net --allow-write=$D \
  scripts/kubectl_wasm_web.ts --port=8090 \
  --origin=https://kcpwasm--johnandersen777-bsky-social.fedproxy.com \
  --debug-log=$D/debug.log --debug-control=$D/control.ndjson \
  2>&1 | tee $D/server8090.log                              # run in the background

bash $D/takeover.sh 2>&1 | tee -a $D/tunnel.log             # public tunnel, background
bash $D/tunnel-watchdog.sh 2>&1 | tee -a $D/watchdog.log    # background
  • takeover.sh waits until the public URL fails twice (60 s apart), then loops ssh -NnT -p 2222 ... -R kcpwasm:80:127.0.0.1:8090 johnandersen777.bsky.social@fedproxy.com.
  • The fedproxy legacy ssh relay drops the whole forward after one failed forwarded-tcpip channel open while ssh stays connected, so tunnel-watchdog.sh kills that ssh when the public URL fails twice (checked every 60 s) and the takeover loop reconnects it.
  • Phone/desktop URL: https://kcpwasm--johnandersen777-bsky-social.fedproxy.com ("no route configured for host", a 404, from fedproxy = the tunnel is down).
  • Production is static files; the /debug/* hooks exist only on this dev server. Profiles turn on only when HEAD /debug/control answers 204. scripts/deploy.sh builds public/ from bin/wasm, writes the OAuth client metadata for https://console.socialweb.computer and ships it to the socialweb.computer host (root@socialweb.computer -p 1997, Caddy site /etc/caddy/sites/console.socialweb.computer.caddy). Another origin is another OAuth client and browser storage: sign in again there.
  • iOS gets low-memory mode automatically (no snapshot, lazy kubectl, GOGC 25, watch cache off). iOS kills the tab near ~630 MiB of wasm memory.

Debug log and remote control of a page

  • Every page served by the dev server streams its console, worker logs, Go logs (kcp: go: lines), terminal output (term) and command results to ~/.tmp/claude-kcpwasm/debug.log, one line each: <time> [<session> <ua>] <level> <message>. The session id is per tab (sessionStorage); the iPhone tab was l5ti1s.
  • Commands: scripts/browser-dev/phone.sh <session> '<js body using d>' [timeout] writes one command to control.ndjson; the page runs it (async function body, debug handle d) and logs result <id> ok|error ....
  • d: run(line) (terminal line + Enter), type(text), screen(), text(sel), click(sel), set(sel, value), state() (status feed), rest(method, path, body, contentType) (kcp REST), call(fn, payload), release(name), reload(), reset() (stop kcp, clear IndexedDB/OPFS/ localStorage except debug switches, reload: a fresh kcp, needs sign-in).
  • Reload or reset the phone yourself; do not ask the user. After sending d.reload()/d.reset() remove that line from control.ndjson (grep -v 'd.reload()' ... > tmp && cat tmp > control.ndjson) so a replay cannot loop (the server replays the last 2 minutes on reconnect).
  • Outages (scripts/browser/debug_link.js): while the dev server or the tunnel is unreachable (fedproxy's "no route" is a 404; only a 404/405 before the first delivery means a static host) the page keeps up to 2000 log lines (oldest dropped), retries with backoff up to 30 s and logs debug log reconnected after <s> s; <n> lines dropped. The control stream reopens with backoff after errors, an open stalled 30 s, or 45 s without the server's 15 s ping event, and right after the log reconnects.
  • localStorage switches (set with d, take effect on reload): kcp.heapProfile=<s> (Go heap+goroutine profiles uploaded to ~/.tmp/claude-kcpwasm/{heap,goroutine}-<session>-<t>.pb.gz; adds memory, iOS may kill the tab), kcp.gogcReady=<n>, socialweb.commsMode=Firehose|Both (also the Compute panel's Comms select), kcp.debugGo=0 (Go logs off).
  • Memory/churn: with debugGo on, Go logs every 60 s a memstats: line (heap, totalAlloc, numGC, gcCPU, goroutines) and an updates (written/noop per resource) line. Allocation per minute = delta of totalAlloc.
  • Profiles: GOTOOLCHAIN=go1.26.9 go tool pprof -sample_index=alloc_space -base <older> -top -cum <newer>.

Running the VM test

D=~/.tmp/claude-kcpwasm
cd ~/src/kcp/kcp-wasm && deno run --allow-read=$D --allow-write=$D,$PWD/demos scripts/browser-demo.ts \
  --log=$D/debug.log --control=$D/control.ndjson \
  --out=demos --session=<id>              # or --ua=iPhone / --ua=X11 to wait for the next page load

Headless (no phone): start browser-demo.ts --ua=X11 first, then deno run -A $D/page-hold.ts ~/.aliceoa_bsky_social_source <page URL>: a puppeteer-core headless Chrome (/usr/bin/google-chrome --no-sandbox) that loads the page, signs in alice with the two-tap flow (first tap on #identity-signin prepares the link, the second, "Continue sign-in", opens the PDS popup; it fills the handle and accepts consent) and holds the tab until $D/page-hold.stop exists. Its UA tag in the debug log is X11. $D/boot-smoke.ts [url] [secs] loads a page and reports kcp ready in and the first memstats line.

It waits for kcp and a signed-in MarketIdentity, releases leftovers, taps Request VM (the tab's EphemeralInfra gets a contract; its ComputeContract is einfra-<infra>-vmshell-...), waits for Held, runs ssh <vm> over WebRTC, types PS1="$ ", ip a, uname -a, exits, releases and polls kubectl until Released. A failed run releases its VM. Output: demos/<t>-<session>.{timeline.txt,term.txt,log}. Start it after the page connected (--session) or before a reload (--ua); starting it a second after a reload misses the connect line.

Release real VMs after every test. Do not create contracts against the bidders without a reason.

Market calls and trust (since 2026-11-18)

  • socialweb is pinned at c525057a (rc142; public repo github.com/publicdomainrelay/socialweb-computer-kcp). What moved in the market since then is carried as Go overlays (socialweb-wasm/overlays/, listed in scripts/prepare-socialweb-wasm-overlay.sh): requester/market.go (a bidder is its offering's webrtc endpoint, service auth aud <did>#pdr_temp_market), requester/mesh_js.go (webrtc:// calls go to the page), oauth/scope.go (computer.socialweb.ingress.offer), identlink/ + trust/trust.go (a bidder's operator from the ident.link walk).
  • The calls: Go -> __socialwebMarketFetch (kcp_worker.js) -> page (market_webrtc.js, @publicdomainrelay/atlibs-webrtc: one RTCPeerConnection per bidder, the offer record computer.socialweb.ingress.offer/<bidder>.<tid> written by Go in the user's repo, the answer read from the bidder's repo, HTTP/1.1 over market datachannels). Activity log (VERBOSE): market POST /xrpc/... over WebRTC, market call over WebRTC failed: ....
  • Debugging a failed rent: kubectl get computecontracts -o yaml in the page terminal names each bid's verdict (policyRejected = the requester's own policy: check the bidder's agent-of chain and your vouch for its operator) and the drop reason of every discovered bidder.
  • A new scope means sign in again (the PDS shows the consent screen).

Sign-in (OAuth)

  • The page is a public atproto OAuth client: client_id <origin>/oauth-client-metadata.json, redirect <origin>/oauth/callback/requester; the tab is paused while on the PDS and finishes the sign-in on return.
  • The account is aliceoa.bsky.social (did:plc:lpfuqerea3deuoyrn7ojser4) on bsky.social. The user stated its credential is in ~/.aliceoa_bsky_social_source; never print, log or commit it.
  • After d.reset() the identity shows "names neither pdsUrl nor a resolvable did" until Sign in is tapped (enter the handle first, or the default bsky.social server is used).
  • Refresh tokens are single use; the page refreshes them in oauth_refresh_worker.js (socialweb oauthclient TokenRefresher seam), which stores each rotated set in IndexedDB before answering. SessionExpired / invalid_grant means sign in again.
  • A new scope in socialweb (requester-scope.txt) needs a fresh sign-in.

The lead (and other agents)

The peer Mac is gone. The lead runs everything on its Fedora host (hat-8, 192.168.0.20, user johnandersen777): the dev server and tunnel, the bidders (local = hat-8, plus hat-9, hat-7), the relay and the socialweb repo, and decides socialweb design; the user directs it. Agents work in worktrees and report back to the lead. Rules agreed with the user:

  • Networking decisions (TURN, candidates, NAT, relays, which streams) are the user's: report observations, do not pick fixes.
  • One owner per file area; coordinate conflicts through the lead.
  • Agents commit in their worktree; the lead merges and pushes. Ask the lead before relying on an unreleased commit.
  • Test peers never get a real shell; never type into a terminal you did not start.
  • Use ~/.tmp (set TMPDIR), never /tmp.

Monitors worth keeping

  • page milestones: tail -n0 -F ~/.tmp/claude-kcpwasm/debug.log | command grep --line-buffered <session> | command grep --line-buffered -E "debug log (on|reconnected)|kcp ready in|MarketIdentity|ComputeContract .* (Held|Failed)|over WebRTC|uncaught" (command grep: the tool grep ignores --line-buffered)
  • per-minute allocation: memstats totalAlloc deltas (see above).