~ knotless Search/
↑↓ move↵ open⇧↵ all results

socialweb.computer/socialweb-computer-kcp

socialweb.computer: a compute market on atproto (kcp)

1 collaborator
0
0
268 lines · 22.5 KB Raw History

0043 - CLI arguments: every option before the command, --exec -- <command> [args...]

Status: in flight on branch cli-args (agent cli-args), landed by the lead as one cycle after review. Facts: ~/.tmp/cli-args/FACTS.md (the long form; the short form is below).

Goal

Owner decision 1 (2026-11-19, plan 0040 decision 49), looking at the README's socialweb-computer run request --name vm1 --exec id -- --atproto-oauth-qr: "why is the --atproto-oauth-qr flag behind exec and the double hyphen? that doesn't seem like how CLIs that exec other things usually work, usually it would be socialweb-computer run request --name vm1 --atproto-oauth-qr --exec -- some-command --with-that-commands-args".

Facts

  • Parsing: main.go parses a verb's own options (internal/clicfg/options.json, the names in builtVerbs) and hands the rest on; splitWords cuts it at --; the tail is parsed by compose.ParseAppArgs against the verb's application in deploy/apps/option-map.yaml (market-requester, market-bidder, ssh-gateway) into the root object: the identity and sign-in options (--atproto-oauth-qr, --atproto-oauth, --atproto-handle, --atproto-identity-mode, --atproto-pds-url, --expect-handle, --skip-qr, --associate-with), network, ingress, comms, bidder DIDs, contract (--vm-name, --allow-bidder-dids, --user-data, --policy) and the options passthrough.
  • Why: run request, run bidder and run gateway ssh are the Go ports (2026-10-12/13) of the TypeScript programs request-vm-ssh, hono-bidder and hono-socialweb-computer-ssh. The CLI used to exec them with everything after --; the port kept that argv as the tail and mapped every old flag in option-map.yaml. There is no embedded program any more, so the -- separates nothing.
  • Verbs with that tail: run request, run requester, install requester (market-requester), run bidder (market-bidder), run gateway ssh (ssh-gateway). Installed units (run bidder --handle %i --fg, run requester --name %i --fg) pass no tail. internal ns-exec and internal deno-fence already take a command after --.
  • Ten names were defined in both tables for run request: exec, hold, skip-ssh, secrets, tunnel, agent, guest-tools, associate, association-ref, state-dir.
  • --exec took one string; ssh root@guest <string> hands it to root's login shell on the guest (bash -c): a shell line, never an exact argv. stdin is the ssh's stdin; a non-zero remote exit makes the verb exit 1; run request ssh and run requester ssh share the code.
  • Consumers outside this repo pin no CLI binary. The requester repo's guest user-data (modules/user-data/src/lib.rs) passes own options only and keeps working. The atlibs app page (atlibs-socialweb-computer-app/index.html:130) shows the old tail as a snippet.
  • Correction (review, 2026-11-19): kcp's guest requester script (internal/atproto/cloudinit/agent.go guestRequesterRunScript) is NOT own-options-only: it appends -- --plc-directory-url <p> --ingress-proxy-host <h> whenever /etc/socialweb-guest-agent/r2-network.env names them, which the requester writes when its network's PLC is not plc.directory (internal/market/requester/secrets.go guestNetworkOf: staging and local market networks; never prod). The script is the requester's (user_data), the socialweb-computer it runs is the bidder's own executable (internal/guestseed/binaries.go), so the two can be of different releases.
  • The prod hosts (read 2026-11-19, read only): the four bidders (hat-7, 8, 9, 10) run run bidder --handle bobvmbuilder.bsky.social --fg from the install bidder template, with no tail and no ExecStart drop-in (hat-7 holds an inactive socialweb-computer-bidder@johnandersen777.bsky.social.service.d/10-oauth-qr.conf.bak with a -- --atproto-identity-mode pdsAccount ... tail: systemd reads only *.conf, and that instance does not run); their config.json has no bidder-options and no bidder-spec-file. The gateway host's hand-written /etc/systemd/system/swc-ssh-gateway.service runs run gateway ssh --host-backend proc --fg --name ssh-gateway --instance sg --state-dir /var/lib/socialweb-computer/socialweb/sg --network-timeout 10m -- --ssh-public-listen 0.0.0.0:22 --ssh-web-listen 127.0.0.1:8789 --public-origin https://ssh.socialweb.computer --ssh-operator-did did:plc:5svqtrhheairglgiiyvutzik --ssh-max-accounts 64, and scripts/promote.sh restarts it on every promote: the new binary refuses that line (exit 2), so it must move to a spec file before the promote (decision 12).
  • Conventions: kubectl exec pod [opts] -- cmd args, sudo [opts] -- cmd args, env [opts] -- cmd args, docker run [opts] image cmd args: the tool's options first, -- ends them, the command's argv is passed exactly. Plain ssh host cmd args joins words for the remote shell, the trap this plan avoids.

Decisions

  1. The layout: socialweb-computer <verb> [options] [--exec -- <command> [args...]]. Every option of the verb, its own (options.json) and its application's (option-map.yaml), comes before --, in any order.

  2. --exec is a flag (no value). The command is every word after --, passed to the guest as that exact argv. The old forms are removed, not a second mode. They fail with an error that names the new form: --exec id and --exec=id ("--exec takes no value: --exec -- id"), --exec with no --, --exec -- with no command, -- without --exec, and options after -- ("-- ends the options: put --atproto-oauth-qr before --exec --"). Without --exec the behaviour is the same as before: an interactive session when stdin is a terminal, otherwise "no session opened".

  3. Argv-safe exec: each word is single-quoted (POSIX: ' written as '\'') and the words are joined into the one line that ssh hands to the guest's login shell, which unquotes it back to the same argv. No word is split, globbed or expanded. To use a shell, write --exec -- sh -c '...'. stdin, the exit status rule and the --hold reconnect stay as they were.

  4. The same rule for every verb that execs: run request, run request resume, run request ssh <name>, run requester ssh <name> --contract <cc>, and so --hold followed by a later ssh -- cmd.

  5. -- means only "end of options; the rest is the command". A verb that takes no command refuses anything after it.

  6. Two tables, each name defined once per verb: options.json holds the CLI's own options, option-map.yaml an application's. A verb names its application in builtVerbs; the parser takes both kinds on one command line. A name defined in both for one verb is an error of the tables that the CLI reports before it parses anything. The duplicates are removed from option-map.yaml (the options.json ones read the same values). --help lists the application's options.

  7. run requester, install requester, run bidder and run gateway ssh get the same treatment: the same parser and the same reason (no embedded program). dev up, install bidder|gateway and upgrade take no tail: nothing to change.

  8. Migration: every caller in this repo changes in the same change (Go, the test host, testkit and system tests, modules, scripts, docs, site). Records of past runs (ARCHITECTURE findings and measurements, plan logs, research, 0040-findings.md) stay as they ran. Statements of current behaviour are updated.

  9. The test host: a module's args are the CLI's new form (--exec -- true). The host moves --exec -- ... to the end of the command line it builds. Its own exec line inputs (request.ssh, request.put, logs) run as --exec -- bash -c <line>, which is what root's login shell did with them before. A new argv input passes an exact argv. The compat binary (0006:compat, the previous rc) gets the old layout while its run request --help lacks the new usage line. Step 7 removes that once the previous rc carries this change.

  10. Composition coverage: --exec already runs in 0003 (run, ssh2), 0006 (run, compat), 0017, the staging stand-up (--exec -- true) and every request.ssh/request.put user (0005, 0013, 0015, ...). 0003's ssh2 now runs sh -c 'uname -a; id; printf "%s|" "$@"' x 'a b' -c "d'e" and also asserts a b|-c|d'e|. Its Linux and uid= checks stay.

  11. The guest requester script across releases. The script now asks the guest's binary: when run request --help prints --exec -- <command> [args...] (cloudinit.RequestExecUsage, the same text the CLI's usage line and the test host's probe use) it puts the PLC and ingress options with the others, else after --. Without a PLC or ingress there is no tail in any release, so prod is never affected. The four cases (~/.tmp/cli-args/r2matrix.py: each release's script run against each release's binary, v0.0.0 = 688c2105 as the old side):

    script (requester) \ binary (bidder)old (v0.0.0)new
    oldparsesrefused (exit 2) on a non-public PLC; parses on prod
    newparses (probe: old form)parses (probe: new form)

    The old-requester/new-bidder case on a staging or local network has no fix this side of a second parsing mode in the new binary (the old script is fixed in the requester's user_data): it needs a requester older than its network's bidders and a fromGuest guest with guest tools, which neither gate runs (staging runs one build; 0006:compat starts no guest requester) and prod cannot produce. By the owner's no-backcompat rule it is not built; the lead or the owner may decide otherwise (the only form that closes it: the new binary accepting -- --plc-directory-url <p> --ingress-proxy-host <h> after --guest-requester).

  12. Hand-written units take nothing a later or an earlier binary may refuse. An option that only some releases take on the command line goes in the spec file both releases read (readSpec is the same in v0.0.0 and this branch): the ssh gateway's --ssh-gateway-spec-file (ssh {publicListen, webListen, operatorDid, maxAccounts}, options [{name: public-origin}]), a bidder's --bidder-spec-file (identity: {mode: oauthQr} is what --atproto-oauth-qr set). CONTRIBUTING's oauthQr recipe now writes the spec file.

  13. internal parse-args <subcommand> ... (hidden) parses a command line as that subcommand would, runs nothing, and prints {verb, words, app, command}; scripts/check-cli-args.sh (in deploy/check.sh --fast) builds the CLI and runs --help of every verb of verbApps (read from main.go; fewer than five read fails) and of the four command verbs, each of which must show --exec -- <command> [args...] (a name defined in both tables fails it); parses every unit template's ExecStart and fails when it parsed none; and fails on an old form in tracked text: an option after -- in a unit, a drop-in or a cloud-init text, --exec with a value in tests/, scripts/, rust/, infra/ and deploy/ (the second review found tests/vmm/tests/vmm.rs:282 this way), and -- --<option> after a socialweb-computer command in the Markdown and YAML that describe today (plans, acceptance, research, ARCHITECTURE, design and NEXT.yaml are records; cargo ... -- -- and set -- -- are not the CLI); a git grep that errors fails it. Each kind proven to fail on a planted old form.

  14. What the de-duplication changed, each decided: --tunnel and --guest-tools repeat and accumulate (options.json repeat), as the old tail did; --guest-host-udp on the command line beats the spec file's guestNetwork.hostUDP again (it did as a tail; config.json still loses to the spec file); secrets, agent, associate, association-ref, exec, hold, skip-ssh took one value either way; state-dir was never reachable as a tail option.

  15. An application option takes the next word as its value only when that word starts with no - (--skip-qr -hold is two options). run request --exec id -- <x> is refused (run request takes no argument).

  16. Known limit: the test host speaks the old layout to an old binary only where 0006:compat runs one (request.start and testkit RunRequest::start); request.ssh, sshTry, the VM logs and Requester::* always speak the new one. 0006:compat's previous-rc CLI only starts and releases, so this holds while that stays so; step 7 removes the old builder.

Steps

  1. Facts and this plan (done).
  2. (done) Parser and tables: main.go splits own options, application options and the command; --exec a flag; request.SSH.Run takes an argv; the duplicates leave option-map.yaml; help and error texts.
  3. (done) Test host, testkit, system tests, modules (0003 assertion), staging module.
  4. (done) Docs, site, scripts, options and capabilities texts, AGENTS.md.
  5. (done, Built) Proof on hat-8, a local market as test-0001: the README example in the new form, an argv with spaces and quotes, --hold then ssh -- cmd, an old form and its error; cargo check -p swc-test-host, cargo wasm, deploy/check.sh --fast, scripts/check-docs.sh.
  6. Lead review, then landed as one cycle.
  7. Once the previous rc carries this change: remove the test host's old-layout builder for the compat binary.
  8. atlibs-socialweb-computer-app index.html:130: the snippet in the new form (that repo, its own change), with the promote that moves stable onto this change; site/deploy.sh only after that promote too (the README and the site would otherwise show commands the stable binary refuses).
  9. (DONE 2026-11-20 02:58Z, the lead, on v0.0.0's binary) Before the promote of this change (the procedure in Built): move swc-ssh-gateway.service to --ssh-gateway-spec-file. The bidders need nothing (no tail on any of them). Seen: the unit's ExecStart now ends --network-timeout 10m --ssh-gateway-spec-file /var/lib/socialweb-computer/socialweb/ssh-gateway-spec.yaml (socialweb:socialweb 0644); active since 02:58:12Z, NRestarts 0; *:22 and 127.0.0.1:8789 listened by the gateway process; journal "the SshGateway root object is applied into its own kcp"; https://ssh.socialweb.computer/ 200. The applied SshGateway app (read with the instance kubeconfig sg/dev/sg/kcp/admin.kubeconfig): generation 2, phase Ready, spec {profile default, ssh {publicListen 0.0.0.0:22, webListen 127.0.0.1:8789, operatorDid did:plc:5svqtrhheairglgiiyvutzik, maxAccounts 64}, options [public-origin https://ssh.socialweb.computer], network, bidderDids, comms, requesterIngress null}: the old tail's five options one for one. Rollback file: /root/swc-ssh-gateway.service.pre-0043 (cp -p it to /etc/systemd/system/swc-ssh-gateway.service, daemon-reload, restart; only needed on a binary from before this change). The entry through the ssh gateway, 0002:run, by hand on prod as alice after the migration (2026-11-20 03:02:28Z to 03:16:51Z, the stable CLI v0.0.0-rc99...688c2105 as the requester, this branch's test host, tests/targets/prod.json): PASS in 861.5 s, no cleanup error: web sign-in at https://ssh.socialweb.computer/, a temporary key on /keys, a transient ssh command (90.6 s), a transient and a held atirc agent run session each answering the observer (34.1 s and 202.9 s after the first prompt), every token revoked before its contract's release (gateway.log), no record left (assert.noneSince). Run dir ~/.tmp/swc-system-tests/runs/0002-composition-1795143748039, log ~/.tmp/cli-args/gate0002/run.log.

Merge note (the lead merges this branch into the batch)

  • With branch secrets: in tests/host/src/swc.rs request_start, secrets' let mut args = self.request_args(input)?; becomes let (mut args, command) = self.request_args(input)?; (this branch's request_args returns the options and the command apart, and request_line puts the command last). Secrets' AGENTS.md sentence run request -- --comms-mode Firehose --secrets-fallback Sealed becomes run request --comms-mode Firehose --secrets-fallback Sealed: scripts/check-cli-args.sh fails the merge until it does (proven: that sentence appended to AGENTS.md fails the check).

Built (branch cli-args)

  • cmd/socialweb-computer/args.go: the split (own options, application options, the command after --), the checks and messages, the help lines; main.go runs it for the five application verbs and every verb with --exec. --exec (options.json, now a boolean) is an option of run request, run request resume, run request ssh and run requester ssh only (it did nothing on release, proxy, agent, associate, r2 and requester proxy). request.RemoteCommand single-quotes the argv for ssh.

  • Removed from option-map.yaml, the 13 names options.json also defined: market-requester state-dir, secrets, agent, guest-tools, associate, association-ref, tunnel, exec, hold, skip-ssh; market-bidder state-dir, guest-host-udp; ssh-gateway state-dir.

  • Proof on hat-8, local market cliargs as test-0001, dev build of 98ae3ac2 (2026-11-19 17:50-17:56Z; logs ~/.tmp/cli-args/proof-*.log): run request --network cliargs --name vm1 --exec -- id printed uid=0(root) gid=0(root) groups=0(root), Released; --vm-name cli-args --exec -- sh -c 'printf "%s|" "$@"' x 'a b' -c "d'e" printed a b|-c|d'e|; --hold --exec -- true, then run request ssh vm3 --exec -- uname -m x86_64, --exec -- printf '%s\n' '$HOME' '*' 'a;b' printed them literally, stdin reached --exec -- cat, --exec -- sh -c 'exit 3' exit 1, release Released; each old form exit 2 with the new form in its message; dev down left nothing.

  • Checks on hat-8 under swc-heavy: deploy/check.sh --fast exit 0 at 07440bc0 (the docs checker in it: 3 run=gate blocks ran and passed); cargo check --workspace --tests in tests/ and cargo wasm in tests/modules exit 0.

  • After the review fixes (rebased on swc/main 888df03f): deploy/check.sh --fast exit 0 at the plan commit after f6c8ab8e (the docs checker 3 run=gate blocks; check-cli-args: ok (8 verbs, 3 unit templates, no option after --)); cargo check --workspace --tests and cargo wasm exit 0.

  • Same root on both releases (hat-8 as test-0001, a scratch state dir each, stopped and removed after; logs ~/.tmp/cli-args/bm-*.log, gm-*.log; v0.0.0 = 688c2105 from scripts/install.sh stable): run bidder --profile prod-qr --handle cliargs-probe.bsky.social applied MarketBidder app with spec {profile prod-qr, identity {handle cliargs-probe.bsky.social, mode oauthQr}, options null, policy null, guestNetwork {lanUDP true}} in all four runs: v0.0.0 with -- --atproto-oauth-qr, v0.0.0 with --bidder-spec-file (identity: {mode: oauthQr}), this branch with that spec file, this branch with --atproto-oauth-qr. run gateway ssh --instance gm... --network-timeout 3m applied SshGateway app with {ssh {publicListen, webListen, operatorDid, maxAccounts 64}, options [public-origin], network null} identically in all four: v0.0.0 with the prod unit's -- --ssh-public-listen ... --ssh-max-accounts 64 tail (ports moved to 42222/48789), v0.0.0 with --ssh-gateway-spec-file, this branch with that spec file, this branch with the options on its line.

  • The guest side of decision 11 in real guests (2026-11-20 02:48-02:50Z, two local markets on hat-8 as test-0001, ~/.tmp/cli-args/proof2.sh, logs p2-*.log): a rent with --guest-tools socialweb-computer from a market whose bidder is this branch's dev build found /usr/local/bin/socialweb-computer = that build, and its run request --help holds the usage line the script looks for (count 2); from a market whose bidder is v0.0.0, the guest's binary is v0.0.0 and the count is 0 (the script takes the old form). Both Released, both dev down left nothing.

  • The script half (~/.tmp/cli-args/r2matrix.py, r2matrix.out): each release's guestRequesterRunScript, extracted from internal/atproto/cloudinit/agent.go at 688c2105 and at this branch, run with an r2-network.env naming a PLC and an ingress against each release's binary: the argv each script built and whether the binary's parser took it are the table of decision 11 (the parsed cases stop later, at the missing association request file, which only a guest has).

  • The sweep (~/.tmp/cli-args/sweep.py, sweep-code.txt): 52 command-line shapes that the test host, testkit, the modules, the staging scripts, scripts/, infra/, the unit templates and the guest scripts emit (from a git grep of every invocation site), and the 82 command lines in the fenced blocks of every Markdown file outside plans, research and ARCHITECTURE, each through internal parse-args of this branch: all parse, apart from two extraction artifacts read by hand (the site's --secrets <(jq ...) and a folded verb).

  • The ssh gateway unit, before the promote of this change (the lead; on the gateway host, ssh -p 1997 root@socialweb.computer; the spec file is read the same way by v0.0.0 and this branch, so the unit works on either binary, and on a rollback):

    S=/var/lib/socialweb-computer/socialweb
    install -m 0644 -o socialweb -g socialweb /dev/stdin $S/ssh-gateway-spec.yaml <<'EOF'
    ssh:
      publicListen: 0.0.0.0:22
      webListen: 127.0.0.1:8789
      operatorDid: did:plc:5svqtrhheairglgiiyvutzik
      maxAccounts: 64
    options:
      - name: public-origin
        value: https://ssh.socialweb.computer
    EOF
    U=/etc/systemd/system/swc-ssh-gateway.service
    cp -p $U /root/swc-ssh-gateway.service.pre-0043
    sed -i "s| -- --ssh-public-listen .*\$| --ssh-gateway-spec-file $S/ssh-gateway-spec.yaml|" $U
    grep '^ExecStart=' $U
    systemctl daemon-reload && systemctl restart swc-ssh-gateway.service
    systemctl is-active swc-ssh-gateway.service && ss -ltn | grep -E ':22 |:8789 '
    

    Health: active, ports 22 and 8789 listening, https://ssh.socialweb.computer/ serves the sign-in page, and the runbook 0002 probe (ssh aliceoa.bsky.social@socialweb.computer 'uname -a'). Rollback: cp -p /root/swc-ssh-gateway.service.pre-0043 $U && systemctl daemon-reload && systemctl restart swc-ssh-gateway.service (only on a binary from before this change; the spec-file unit runs on both). A restart ends the running ssh sessions (a transient session's VM is released with it); held LC_SESSION sessions are kept and adopted by the next start (AGENTS.md, the SshServer). The bidders need nothing: none has a tail.

Done when

  • The README example reads as he wrote it (run request --name vm1 --atproto-oauth-qr --exec -- id) and runs.
  • deploy/check.sh --fast and scripts/check-docs.sh are green.
  • The staging and prod gates pass with no assertion changed, except 0003's added argv check.