johnandersen777.bsky.social/zkwid

Switch repository

Loading…

# zkwid.at (N): measured-workload keys and OAuth proxy, atproto-native

## Context

A hosted service H (e.g. a knotless CI runner) runs instances of a requester workload R: a shim plus stage2..N WASM. Each instance must act on behalf of user U with exactly the OAuth scopes U approved for that workload.

N (zkwid.at) is the service built here. It does four things:

1. For each workload record in U's repo, it hosts a dedicated OAuth client. U consents once per workload, and N stores the resulting session.
2. It creates `at.zkwid.key` (K) in U's repo only after a successful ZK exchange. H proves in RISC Zero that the measured shim (chain M) produced K's public key for N's challenge. U's workload record must list M and list H as an allowed host. H holds K's private key.
   - Only N can write K, using N's own OAuth client and its `repo:at.zkwid.key` scope.
   - Workload clients never get any `at.zkwid.*` scope, so workloads cannot mint keys.
3. It reverse-proxies XRPC calls from instances that sign service-auth JWTs with a K key, forwarding them through U's session within the workload's scopes.
4. Revocation is a record deletion. Deleting K revokes one instance; deleting the workload record revokes all of them.

Scope:

- H is a stub. U_ATRepo is a real `atproto-pds` from ../atproto-crates.
- Changes to knotless and socialweb-computer-kcp are documented as future work only, not built.
- ZK gates key creation only.
  - Backend: RISC Zero 3.0.x, using `RISC0_DEV_MODE` receipts in the devnet and real succinct receipts in production.
  - M is an ordered list of blob CIDs ("DICE-lite"). Full DICE is deferred.
- Code stays minimal: no comments, small crates, `at.zkwid.*` NSIDs only.
- Auth, OAuth, CIDs and keys all come from atproto-crates.

## Actors and identifiers

| Actor | Identifier | Dev host (via CONNECT proxy) |
|---|---|---|
| N | `did:web:zkwid.test`, service `#zkwid` | `zkwid.test` |
| H stub | `did:web:h.zkwid.test`, `#atproto` key | `h.zkwid.test` |
| U | did:plc from a fake PLC directory | account on `pds.zkwid.test` |
| Instance | `did:key:<K pub>`, self-certifying `iss` | none |

- Client ID: `https://zkwid.test/<handle>/<rkey>/oauth-client-metadata.json`.
- We use the path form. The spec allows any path. The subdomain form can exceed the 63-byte DNS label limit, and handles that already contain dashes make it ambiguous.
- If U's handle changes, the client ID changes. This is documented.

## Lexicons (`lexicons/at/zkwid/`)

- **`at.zkwid.workload`** (record, in U's repo, rkey `any`, used in the URL): `{ measurements: [{ name, wasm: blob }], scope: string, hosts: [did], createdAt }`. The PDS hosts the stage blobs.
- **`at.zkwid.key`** (record, in U's repo, written by N, rkey tid): `{ key: did:key, workload: strongRef, host: did, instance: string, createdAt, expiresAt }`.
- **`at.zkwid.createChallenge`** (procedure on N, caller H): input `{ repo, workload: strongRef }`, output `{ challenge }`.
  - The challenge is a stateless JWT signed by N's `#atproto` key: ES256, `typ: zkwid-challenge+jwt`, claims `{ iss: N, sub: H, repo: U, workload: { uri, cid }, exp: now + 300 s, jti }`.
  - N stores nothing for it.
- **`at.zkwid.createKey`** (procedure on N, caller H): input `{ challenge, instance, receipt: bytes }`, output `{ uri, cid, key }`.
  - The key and the measurements come from the receipt journal, never from caller fields.
  - K's rkey is `base32(sha256(challenge))`, so a reused challenge collides with the existing record. That makes it single-use, with U's repo as the only state.
  - Errors: `WorkloadNotFound`, `MeasurementMismatch`, `HostNotAllowed`, `InvalidReceipt`, `ChallengeInvalid`, `ChallengeUsed`, `NotAuthorized`.
- **Journal**, committed by the guest: `{ measurements: [cid], key: did:key, nonce: sha256(challenge), aud: did }`.
  - The guest measures every stage blob and runs the shim in wasmi.
  - The shim calls the guest import `keygen(seed)`, which derives a K256 key natively in the guest with the accelerated `k256` crate. The seed is a private witness held by H.
- **`at.zkwid.deleteKey`** (procedure on N, caller H): `{ uri }`.
- **`at.zkwid.demo.log`** (record): the demo workload writes this through the proxy.

## N behaviour

**Client metadata** at `GET /<handle>/<rkey>/oauth-client-metadata.json`:

- N resolves the handle to the DID and PDS, then fetches `at.zkwid.workload/<rkey>`.
- The document is a confidential web client with:
  - `scope = "atproto " + workload.scope`
  - N refuses to serve the document (404) if `workload.scope` names any `at.zkwid.*` resource
  - `private_key_jwt` with ES256 and an inline `jwks` holding a client key per workload, stored in the DB
  - `redirect_uris = [https://zkwid.test/<handle>/<rkey>/callback]`
  - `dpop_bound_access_tokens: true`
- It is served with status 200 and no redirect.

**OAuth flow:**

- `GET /<handle>/<rkey>/login` runs `atproto_oauth::workflow::oauth_init` (PAR, PKCE, DPoP).
- `GET /<handle>/<rkey>/callback` runs `oauth_complete_with_binding` and stores the session keyed by (U DID, rkey).
- Refresh goes through `atproto_oauth::refresh::RefreshCoordinator`, with an atomic swap in the store.

**N's own client** (`https://zkwid.test/oauth-client-metadata.json`):

- Scope is `atproto repo:at.zkwid.key`. U consents to it once, during onboarding.
- N stores this session per U. It is the only credential that ever writes or deletes K.

**`at.zkwid.createChallenge`:**

- Verify H's service auth (`lxm` = createChallenge).
- Fetch the workload by strongRef and check that `iss ∈ workload.hosts`.
- Mint the challenge JWT with `atproto_oauth::jwt::mint`. N stores nothing.

**`at.zkwid.createKey`:**

1. Run `atproto_xrpcs::verify_service_auth` with `iss` = H, `aud` = `did:web:zkwid.test#zkwid` and `lxm` = createKey. No jti cache is needed, because a replayed request hits the fixed rkey.
2. Verify the challenge JWT:
   - the signature is N's key;
   - `typ` is `zkwid-challenge+jwt`;
   - it has not expired;
   - `sub` equals the caller.
3. Fetch the workload record from the challenge's strongRef, requiring the CID to match. Check that `iss ∈ hosts` and that both N's own session and the workload session exist for U.
4. Run `receipt.verify(ZKWID_GUEST_ID)` and decode the journal. Require:
   - `journal.measurements == workload.measurements[*].wasm.ref`
   - `journal.nonce == sha256(challenge)`
   - `journal.aud == did:web:zkwid.test`
5. `createRecord` K at rkey `base32(sha256(challenge))` using N's own session. If the record already exists, return `ChallengeUsed`. Otherwise index `did:key` → K uri.
6. The verifier is built with `disable-dev-mode` unless the devnet flag `ZKWID_DEV_RECEIPTS=1` is set.

**Proxy** at `ANY /xrpc/<nsid>`, for anything that is not an `at.zkwid.*` method:

1. Verify the JWT with `verify_service_auth` through a resolver that synthesizes a document for `did:key` issuers and delegates everything else to `DidResolver`. Check `aud` = N, `lxm` = path NSID, jti not replayed.
2. Look up K from the `did:key` index, then `getRecord` both K and the workload live (30 s cache) to confirm neither was revoked.
3. Check scope with `atproto_oauth::scopes::ScopesSet`:
   - `allows_repo(collection, action)` for repo writes, reading the collection from the body.
   - `allows_rpc(lxm, aud)` for other calls, using `aud` from the `atproto-proxy` header.
   - Always deny `at.zkwid.key` and `at.zkwid.workload`.
4. Forward the call to U's PDS with `atproto_client` DPoP using the stored session.
   - Buffer the body so DPoP-nonce and refresh retries can resend it.
   - Return the status and body verbatim and strip forwarded headers. This follows the atproto-reverse-proxy pattern, but nothing is ported from its Go code.

**Store:** SQLite holding:

- per-workload client keys and sessions
- N's own sessions
- the `did:key` → K index
- the proxy jti cache

It holds no challenge state.

## Repo layout

```
Cargo.toml              workspace; path deps ../atproto-crates/crates/{atproto-oauth,atproto-xrpcs,atproto-identity,atproto-client,atproto-dasl,atproto-record,atproto-testkit}
lexicons/at/zkwid/      workload.json key.json createKey.json deleteKey.json demo/log.json
crates/zkwid/           lib: lexicon types, measure (compute_raw_cid), client-metadata builder, scope check, did:key resolver
crates/zkwid-n/         bin N: axum routes above + store.rs
crates/zkwid-zkvm/      risc0 methods crate (build.rs embed_methods) + guest/: measure stages, run shim in wasmi, keygen import, commit Journal
crates/zkwid-h/         stub H: did:web doc, seed, prove via zkwid-zkvm, createChallenge/createKey/deleteKey, instance runs stage2 in wasmi, proxy calls
crates/zkwid-dev/       devnet: FakePlcDirectory, spawns `pds`, local CA (rcgen) + CONNECT proxy, U account, seeds U_ATRepo, headless consent
stages/shim stages/stage2   wasm32 no_std; stage2 emits the demo.log record body
tests/e2e.rs            in crates/zkwid-dev: full flow + negatives
docs/future-work.md     knotless, socialweb-computer-kcp, ZK/DICE
```

## Dev network (mirrors atproto-crates/tools/conformance)

1. Start `FakePlcDirectory` and the `pds` binary (`--features clap`) with:
   - `PDS_HOSTNAME=pds.zkwid.test`
   - `PDS_TRUSTED_PROXY_HOPS=1`
   - `PDS_ALLOW_DEV_DEFAULTS`
   - non-production egress
2. Generate a CA and leaves with rcgen. Run a CONNECT proxy that maps `{zkwid,h.zkwid,pds.zkwid}.test:443` to loopback TLS terminators.
3. Every process gets `HTTPS_PROXY` and `SSL_CERT_FILE`. This requires Linux, as in the conformance harness.
4. Create U, upload the stage blobs, and write `at.zkwid.workload/demo`.
5. Approve consent headlessly through the PDS one-step JSON authorize POST.

## Waves of parallel subagents

Every wave ends with a gate: `cargo build --workspace && cargo test --workspace`, run by the coordinator (me). Agents work in separate crates, so they never edit the same files. The coordinator owns `Cargo.toml` and the lexicons.

- **Wave 0 (coordinator, serial).** Workspace skeleton, lexicon JSON, frozen interfaces in `crates/zkwid` (type and fn signatures, `todo!()` bodies), the `Devnet` struct API, N's route table, and env vars. Spike: build `pds` and boot it once.
- **Wave 1 (5 agents in parallel):**
  - A: `crates/zkwid` bodies and unit tests.
  - B: `crates/zkwid-dev` devnet: PLC, PDS, CA, proxy, account, seeding, both consents.
  - C: `stages/*` and the H stub core (did:web doc, seed, the native wasmi runner for stage2).
  - D: N OAuth (N's own client plus per-workload clients, metadata routes, login, callback, session store, refresh).
  - Z: `crates/zkwid-zkvm` guest and host prove fn, `Journal`, and a dev-mode round-trip test.
- **Wave 2 (3 agents in parallel):**
  - E: N `createChallenge`, `createKey` (receipt verify), and `deleteKey`.
  - F: N proxy (did:key JWT verify, live K check, scope check, forward).
  - G: H stub client calls (challenge, prove, createKey as H, instance JWT, proxy writes).
- **Wave 3 (2 agents in parallel):**
  - H: `tests/e2e.rs`, the full flow plus negatives.
  - I: `docs/future-work.md` and the README.
- **Wave 4 (coordinator):** `/code-review` and `/simplify` passes, then fixes.

Each agent brief contains its crate path, the frozen interfaces, the atproto-crates functions to reuse (listed above), the no-comments/minimal style rule, and its gate command.

## Verification

`cargo test -p zkwid-dev --test e2e` boots the devnet and asserts:

**Happy path:**

1. The metadata scope equals `atproto <workload.scope> repo:at.zkwid.key`.
2. After consent, N holds a session.
3. createKey from H puts K in U's repo.
4. The instance JWT is accepted, and the proxied `createRecord at.zkwid.demo.log` lands in U's repo.

**Negatives, each refused:**

- workload client metadata never contains `at.zkwid`, and a workload whose scope names `at.zkwid.key` gets a 404 for its metadata
- a proxied workload call that tries to create `at.zkwid.key` → 403, and none is created
- createKey with no receipt, a tampered journal, or a wrong ImageID → `InvalidReceipt`
- an expired, forged, or re-signed challenge, or one whose `sub` is another H → `ChallengeInvalid`
- a reused challenge, even with a fresh receipt → `ChallengeUsed`, leaving exactly one K for that challenge
- a journal key K created without any receipt cannot exist: every K maps to a verified receipt in N's log
- measurement mismatch → `MeasurementMismatch` and no K
- H not in hosts → `HostNotAllowed`
- K deleted → proxy 401
- workload deleted → 401
- out-of-scope collection → 403
- proxy write to `at.zkwid.key` → 403
- replayed jti → 401
- expired JWT → 401
- wrong `aud` → 401

## Future work (documented only)

- **knotless (H):**
  - The runner measures the workload stages, generates a per-run key, calls `at.zkwid.createKey` with its did:web service auth, passes N's URL and the key to the instance, and calls `deleteKey` when the run finishes.
  - The run identity is the `dev.knotless.check` at-uri, used as `instance`.
- **socialweb-computer-kcp:**
  - Accept `at.zkwid.key` as a signer-binding kind, in place of `badgeBlueKeys market_requester`. The RFP is then signed by the instance key.
  - Add a policy workflow that checks K → workload → measurements and hosts.
- **ZK / DICE:**
  - Real succinct receipts on a GPU or Boundless prover. Measure prove time.
  - A chain-head CDI.
  - Publish the receipt as a blob referenced from K, so third parties can audit it.
  - Note the limit: without a TEE, H knows the seed. The proof binds K to an execution of the measured shim and to N's challenge. It does not protect the key from H, which U already trusts.
- **Lexicon publishing:** a `_lexicon.zkwid.at` TXT record and `com.atproto.lexicon.schema` records.