~ knotless Search/
↑↓ move↵ open⇧↵ all results

socialweb.computer/socialweb-computer-kcp

socialweb.computer: a compute market on atproto (kcp)

1 collaborator
0
0
581 lines · 32.3 KB Raw History

lexicons-market: one row per lexicon

  • the fields of every lexicon come from the lexicon files: each section below includes its generated field table (the lexicon: link; on https://socialweb.computer/docs/protocol/lexicons-market.html the table is rendered in place) and adds what the code does with it: the rkey it writes, who writes and reads it, and the step
  • every lexicon, generated: https://socialweb.computer/docs/lexicons/ (XRPC methods: https://socialweb.computer/docs/xrpc/)
  • covers every lexicon under lexicons/com/publicdomainrelay/temp/ except cluster.* (infra.md) and badgeBlueKeys (identity.md), with compute.secrets.getSecrets and compute.config.wif.simple at the end
  • the flows: market.md (rfp to release), guest-events.md (events under a receipt); steps S3 to S9: README.md
  • headings drop the prefix com.publicdomainrelay.temp.; file paths are under lexicons/com/publicdomainrelay/temp/
  • the Go types are generated from these files into internal/atproto/market/zz_generated.lexicons.go (go generate in internal/atproto/market/nsid.go)
  • "defined, not written": the lexicon and its NSID constant exist, but no Go code in this repo writes the record

Rows

rowmeaning
rkeywhat the code writes, not only the lexicon key
written by / read byrole and the Go function
code noteswhat the code does with a field, where it differs from what the lexicon allows
stepthe README step it belongs to

Summary

NSID tailrkey writtenwritten bystep
market.rfpTIDrequesterS3
market.bidmarket.BidRKeybidderS4
market.bids.free<bid rkey>-payloadbidderS4
market.bids.x402nonedefined, not writtenS4
market.acceptSessionRKey("accept", ...)requesterS5
market.accepts.free, market.accepts.x402nonedefined, not writtenS5
market.receiptmarket.ReceiptRKeybidderS6
market.receipts.free, market.receipts.x402nonedefined, not writtenS6
market.eventper kindbothS7-S9
market.offeringselfbidderS3 (discovery)
market.heartbeatself (literal)bidderS3 (discovery)
market.relaysselfno Go writer hereS3 (discovery)
market.associationthe agent DIDprincipalS5, S8
market.associationRequestthe principal DIDguest requesterS8
computer.socialweb.peerthe subhandleguest requester (its own and each co-located account's); requester (each spec.agent contract's atirc agent)S8
market.submitRfp-served by bidderS3
market.submitBid-served by requesterS4
market.submitAccept-served by bidderS5
market.submitEvent-served by bothS7-S9
compute.vmTIDrequesterS3
compute.events.vm.*per kindbidder (vm.delete: requester)S7, S9
compute.events.guest.*per kindrequester or bidder for the guestS8
compute.guest.subscribeEvents-served by bidder GuestControlPlaneS8
ingress.webRTCwebrtc (listen), TID (offer, answer)requester, bidderS3-S6
compute.secrets.getSecrets-served by requesterS8
compute.config.wif.simple<bid rkey>-configbidderS4

market.rfp

lexicon: com.publicdomainrelay.temp.market.rfp

itemvalue
rkeyTID (putRFP puts the signed record at a TID it makes)
written byrequester: contractRunner in internal/market/requester/runner.go putRFP (market.BuildRFP, market.SignRecord, putRecord)
read bybidder: stream or submitRfp, then getRecord (internal/market/bidder/submit.go SubmitRFP, acceptContract in internal/market/bidder/adapters.go); bid window from createdAt (internal/atproto/market/rfpwindow.go)
stepS3

market.bid

lexicon: com.publicdomainrelay.temp.market.bid

itemvalue
rkeymarket.BidRKey(bidderDID, rfpURI, rfpCID) = bid-<20 hex>; also the Bid object name
written bybidder: writeBidRecord in internal/market/bidder/bids.go (market.BuildBid, market.SignRecord)
read byrequester: submitBid (Requester.SubmitBid in internal/market/requester/callbacks.go), stream, catch-up getRecord at BidRKey (internal/market/requester/bidcatchup.go); bidder at accept time
stepS4

market.bids.free

lexicon: com.publicdomainrelay.temp.market.bids.free

itemvalue
rkey<bid rkey>-payload
written bybidder: writeBidRecord
read byrequester: free detection for firstFree (internal/market/requester/accept.go, internal/market/requester/inbox.go)
stepS4

market.bids.x402

lexicon: com.publicdomainrelay.temp.market.bids.x402

itemvalue
rkeynone: defined, not written
written byno Go writer
read byrequester reads cost when a bid names one (internal/market/requester/accept.go, internal/market/requester/inbox.go)
stepS4

market.accept

lexicon: com.publicdomainrelay.temp.market.accept

itemvalue
code notespayload is never set by the code
rkeyacceptRKeyFor -> market.SessionRKey("accept", repoDID, contract, rfpURI, rfpCID, bidURI, bidCID) (internal/market/requester/secrets.go)
written byrequester: putAccept in runner.go (market.SignRecord, putRecord)
read bybidder: submitAccept or stream -> provisioning StateSync (SubmitAccept in internal/market/bidder/submit.go, acceptContract); the receipt cid binds it
stepS5

market.accepts.free

lexicon: com.publicdomainrelay.temp.market.accepts.free

itemvalue
rkeynone: defined, not written
written byno Go writer (AcceptsFreeNSID constant only)
read bynone
stepS5

market.accepts.x402

lexicon: com.publicdomainrelay.temp.market.accepts.x402

itemvalue
rkeynone: defined, not written
written byno Go writer (AcceptsX402NSID constant only)
read bynone
stepS5

market.receipt

lexicon: com.publicdomainrelay.temp.market.receipt

itemvalue
code notessubmitEvent is copied from the accept when the bidder serves inbound XRPC; payload is the RFP's compute.vm (null when none)
rkeymarket.ReceiptRKey(acceptURI, acceptCID) = receipt-<32 hex>
written bybidder: createReceipt in internal/market/bidder/submit.go -> market.CreateReceiptRecordAt (internal/atproto/market/attest.go)
read byrequester: FindReceipt (one getRecord, internal/market/requester/market.go), market.VerifyReceipt in acceptReceipt; policy trust (internal/policyengine/trust/association.go)
stepS6

market.receipts.free

lexicon: com.publicdomainrelay.temp.market.receipts.free

itemvalue
rkeynone: defined, not written
written byno Go writer (ReceiptsFreeNSID constant; named in OAuth scopes, internal/atproto/oauth/scope.go)
read bynone
stepS6

market.receipts.x402

lexicon: com.publicdomainrelay.temp.market.receipts.x402

itemvalue
rkeynone: defined, not written
written byno Go writer (ReceiptsX402NSID constant; named in OAuth scopes)
read bynone
stepS6

market.event

lexicon: com.publicdomainrelay.temp.market.event

itemvalue
rkeybidder: market.EventRKeys(key) second value <prefix>-event-<32 hex>; requester: SessionRKey("event", receiptURI) for vm.delete, SessionRKey("contract-event-envelope", uid) for ContractEvents
written bybidder event adapter (writeMarketEvent in internal/market/bidder/adapters.go); requester destroyEnvelope (runner.go), publishContractEvent (internal/market/requester/contractevents.go); both via market.PutEvent
read bythe other party: submitEvent or stream; requester FindTermination
stepS7, S8, S9

market.offering

lexicon: com.publicdomainrelay.temp.market.offering

itemvalue
rkeyself (market.OfferingRKey)
written bybidder: putOffering, withdraw in internal/market/bidder/offering.go
read byrequester: market.ReadOffering (self, else the rkey the heartbeat names), market.OfferingApplies; relays list its repos by listReposByCollection
stepS3 (discovery)

market.heartbeat

lexicon: com.publicdomainrelay.temp.market.heartbeat

itemvalue
rkeyself
written bybidder: beat in internal/market/bidder/offering.go, every HeartbeatInterval (60 s)
read byrequester: market.HeartbeatCache.FreshDIDs (fresh = at most HeartbeatTTL 300 s old); internal/marketpresence
stepS3 (discovery)

market.relays

lexicon: com.publicdomainrelay.temp.market.relays

itemvalue
rkeyself (market.RelaysRKey)
written byno Go writer in this repo
read byrequester discovery: market.RelayURLsOf from relaySources in internal/market/requester/discovery.go
stepS3 (discovery)

market.association

lexicon: com.publicdomainrelay.temp.market.association

itemvalue
rkeythe agent DID
meaningthe grant beside the link.ident.endorsement at the same rkey: identity.md
written bythe operator (standing, no contract: compose.ApplyOperatorPair, scripts/associate-bidder.ts); the requester for its guest requester (contract = the accept {uri, cid}, expiresAt = accept + 24 h: internal/market/requester/association.go); the guest requester for its co-located accounts (R2's contract and expiresAt: internal/app/compose/peer.go)
read byinternal/identlink walk: policy trust, bidder lifetime cap (internal/market/bidder/association.go, ComputeVM spec.associationExpiresAt = the walk expiresAt; spec.associationContracts = the contracts (accept + receipt) of its contract-scoped grants at accept, and the VM ends association_ended once each of them has its vm.terminated or vm.delete), requester discovery, MarketIdentity
chainfrom the agent's DID document (alsoKnownAs agent-of), up to maxDepth (AssociationPolicy maxDepth, 1 to 100, default 100), 256 getRecords per walk. socialweb-computer internal association <did> prints the walk
stepS5, S8; release deletes only its own

market.associationRequest

lexicon: com.publicdomainrelay.temp.market.associationRequest

itemvalue
rkeythe principal DID (compose.ApplyAssociationRequestRecord)
written bythe guest requester: an AtprotoRecord object from internal/app/compose/records.go
read bythe authority requester: one getRecord in the DID the guest reported (internal/market/requester/association.go); cmd/socialweb-computer/requestassociation.go
stepS8; flow: identity.md

computer.socialweb.peer

lexicon: computer.socialweb.peer

itemvalue
code notesonly createdAt and iroh are written; no writer sets webrtc or channels
rkeythe subhandle: the guest requester's atirc subhandle (r2-<accept rkey>.<owner handle>, its handle without one); <name>.<that> for a co-located account; <label>-<rfp rkey>.<handle> for a contract's atirc agent (label: the handle's first label, at most 14 characters)
irohthe short ticket of the node that is that identity: for an account a PDS hosts, the PDS's XRPC endpoint (ingress-iroh); for an atirc agent identity, its atirc mesh run daemon (ALPN atirc/mesh/1). One identity, one node; the dialer picks the protocol by ALPN
written bythe guest requester (run request --guest-requester [--colocated a,b]): AtprotoRecord objects of internal/app/compose/peer.go in its own repo and each co-located account's, present only while its association ref verifies. The requester of a spec.agent contract writes no peer record: after the verified receipt it puts the agent's iroh key (a 32-byte ed25519 seed, /etc/atirc/iroh_key.hex, 0600) and, for a guest requester, R2_COLOCATED=<nick>=<ticket> (/etc/socialweb-computer/r2-agent.env) into the bundle (internal/market/requester/agentpeer.go publishAgentPeer), and R2 writes the agent's peer record
read bymesh peers: one getRecord(DID, computer.socialweb.peer, subhandle), never listRecords; trusted only while the walk of the repo DID reaches the owner
releasethe guest requester's run request release sets spec.peer.retire: the peer, association and endorsement records are deleted, each local account deactivated and its did:plc tombstoned (best effort; expiry covers a skipped cleanup)

market.submitRfp

lexicon: com.publicdomainrelay.temp.market.submitRfp

itemvalue
input{rfpUri, rfpCid, ingress?} (ingress = ingress.webRTC offers)
output{ok, name, phase, ingress?, declined?: {kind, reason}} (declined: the bidder will not bid; kind capacity = full now, unservable = nothing but another bidder will do); 409 RFPClosed once the bid window is over; every error is in the lexicon
served bybidder: handleSubmitRFP in internal/services/bidder/routes.go -> Bidder.SubmitRFP
called byrequester: solicitBids -> HTTPMarket.SubmitRFP (internal/market/requester/runner.go, market.go)
stepS3

market.submitBid

lexicon: com.publicdomainrelay.temp.market.submitBid

itemvalue
input{uri, cid, record} of the market.bid, ingress?
output{ok, ingress?}; errors InvalidRequest, Unauthorized, Forbidden, Unavailable
served byrequester ingress: internal/services/requester/routes.go -> Requester.SubmitBid; the bidder route only parses it
called bybidder: submitBid in internal/market/bidder/submit.go to the RFP's submitBid
stepS4

market.submitAccept

lexicon: com.publicdomainrelay.temp.market.submitAccept

itemvalue
input{acceptUri, acceptCid, ingress?}
output202 {ok, id, acceptUri, acceptCid}; 409 CapacityExceeded when the accept would overcommit the host; every error is in the lexicon
served bybidder: handleSubmitAccept -> Bidder.SubmitAccept (provisioning StateSync)
called byrequester: submitAcceptAndWait (internal/market/requester/accept.go)
stepS5

market.submitEvent

lexicon: com.publicdomainrelay.temp.market.submitEvent

itemvalue
input{uri, cid, record} of the market.event, ingress?
output{ok, ingress?}; 403 Forbidden "not the accept author"
served bybidder handleSubmitEvent -> Bidder.SubmitEvent; requester Requester.SubmitEvent (callbacks.go)
called byrequester releaseContract, publishContractEvent; bidder event adapter (internal/market/bidder/adapters.go)
stepS7, S8, S9

compute.vm

lexicon: com.publicdomainrelay.temp.compute.vm

itemvalue
rkeyTID (putSetupRecord: putRecord at a TID; unsigned)
written byrequester: runner.go with DefaultComputeCPUs, DefaultComputeMem, DefaultComputeDisk (internal/market/requester/requester.go); user_data from cloudinit.BuildUserData (internal/atproto/cloudinit)
read bybidder through rfp.payload: kind check (rfpKind in submit.go), boot (acceptContract, internal/guestseed adds the binaries unit)
stepS3

compute.events.vm.delete

lexicon: com.publicdomainrelay.temp.compute.events.vm.delete

itemvalue
rkeymarket.ReleasePayloadRKey(receiptURI) = SessionRKey("delete", receiptURI)
written byrequester: destroyEnvelope in internal/market/requester/runner.go
read bybidder: verifyInboundDelete, applyRelease (internal/market/bidder/adapters.go); policy trust
stepS9

compute.events.vm.hostKeys

lexicon: com.publicdomainrelay.temp.compute.events.vm.hostKeys

itemvalue
rkeyEventRKeys("hostKeys:<receipt>") -> hostkeys-payload-<32 hex>
written bybidder: report in internal/market/bidder/report.go from the host key bound at login (RecordBoundHostKey in internal/guestidentity/service.go)
read byrequester: waitGuest -> status.contract.hostKeys
stepS7

compute.events.vm.onNetwork

lexicon: com.publicdomainrelay.temp.compute.events.vm.onNetwork

itemvalue
rkeyEventRKeys("onNetwork:<receipt>") or EventRKeys("onNetwork:<tag>:<receipt>")
written bybidder: report (internal/market/bidder/report.go) from the guest POST (Guest.onNetwork in internal/guestidentity/service.go); sandbox: internal/market/bidder/sandbox.go
read byrequester: waitGuest -> status.contract.vmAddress, status.contract.networks
stepS7

compute.events.vm.sandbox

lexicon: com.publicdomainrelay.temp.compute.events.vm.sandbox

itemvalue
rkeyEventRKeys("<nsid>:<receipt>")
written bybidder: internal/market/bidder/sandbox.go (compute.vm.kind: deno)
read byrequester: status.contract.sandbox (internal/market/requester/events.go, runner.go)
stepS7

compute.events.vm.memoryPressure

lexicon: com.publicdomainrelay.temp.compute.events.vm.memoryPressure

itemvalue
rkeyfrom the report StateSync key <nsid>:<kind>-<n>:<receipt>
written bybidder: internal/market/bidder/memory.go (edge-triggered from the VM cgroup's memory.events: one per crossing, re-armed after 5 min below 90% of the threshold, at most 1 per kind per 10 min and 10 memory events per contract)
read bynone in Go (requesters and the watch page)
stepS7

compute.events.vm.oomKilled

lexicon: com.publicdomainrelay.temp.compute.events.vm.oomKilled

itemvalue
rkeyfrom the report StateSync key <nsid>:oom-<n>:<receipt>
written bybidder: internal/market/bidder/memory.go (one per new oom_kill count, inside the same 10-per-contract cap)
read bynone in Go
stepS7

compute.events.vm.terminated

lexicon: com.publicdomainrelay.temp.compute.events.vm.terminated

itemvalue
rkeyEventRKeys(TerminationKey(receiptURI)) -> termination-payload-<32 hex>
written bybidder: termination StateSync (internal/computevm/termination.go, internal/controllers/computevm/controller.go), adapter atproto.market.event
read byrequester FindTermination (market.go); atirc via the token lease (contractLease in internal/market/requester/atirc.go); policy trust
stepS9

compute.events.guest.publicKey

lexicon: com.publicdomainrelay.temp.compute.events.guest.publicKey

itemvalue
rkeyrequester: SessionRKey("contract-event", uid); bidder: EventRKeys("guest:...")
written byrequester (outbound ContractEvent, contractevents.go); bidder for the guest (PublishGuestEvent in internal/market/bidder/guestevents.go)
read byguest agent (internal/guestagent/subscribe.go); requester (inbound ContractEvent)
stepS8

compute.events.guest.signalWebRTC

lexicon: com.publicdomainrelay.temp.compute.events.guest.signalWebRTC

itemvalue
rkeyas guest.publicKey
written byrequester (CLI / browser ContractEvent, internal/stack/request/webrtc.go); bidder for the guest
read byguest agent (internal/guestagent/session.go, subscribe.go); requester (inbound ContractEvent)
stepS8

compute.events.guest.secrets

lexicon: com.publicdomainrelay.temp.compute.events.guest.secrets

itemvalue
rkeySessionRKey("contract-event", uid) of ContractEvent secrets-<contract>
written byrequester only, on demand: sealGuestSecrets in internal/market/requester/guestsecrets.go
read byguest agent (internal/guestagent/secrets.go); the bidder forwards it unopened
stepS8

compute.events.guest.secretsRequested

lexicon: com.publicdomainrelay.temp.compute.events.guest.secretsRequested

itemvalue
rkeybidder EventRKeys("guest:...")
written bybidder for the guest agent (comms mode Firehose with secretsFallback: Sealed only)
read byrequester: status.contract.secretsRequested, then sealGuestSecrets
stepS8

compute.events.guest.secretsApplied

lexicon: com.publicdomainrelay.temp.compute.events.guest.secretsApplied

itemvalue
rkeybidder EventRKeys("guest:...")
written bybidder for the guest agent
read byrequester: status.contract.secretsApplied, condition SecretsApplied
stepS8

compute.events.guest.secretsUpdated

lexicon: com.publicdomainrelay.temp.compute.events.guest.secretsUpdated

itemvalue
rkeySessionRKey("contract-event", uid) of ContractEvent secrets-updated-<SessionRKey("secretsUpdated", contract, ref)>
written byrequester only: notifySecretsUpdated (internal/market/requester/association.go), every fresh atirc mint
read byguest agent: fetch the bundle again
stepS8

compute.events.guest.agentStart

lexicon: com.publicdomainrelay.temp.compute.events.guest.agentStart

itemvalue
rkeySessionRKey("contract-event", uid)
written byrequester only (run request agent <name> -> ContractEvent; needs spec.agent)
read byguest agent: systemctl start --no-block atirc-agent.service (internal/guestagent/agentstart.go)
stepS8

compute.events.guest.requesterDid

lexicon: com.publicdomainrelay.temp.compute.events.guest.requesterDid

itemvalue
rkeybidder EventRKeys("guest:...")
written bybidder for the guest agent (internal/guestagent/association.go)
read byrequester: status.contract.guestRequester (internal/market/requester/association.go, market.ValidGuestRequesterDID)
stepS8

compute.events.guest.r2Start

lexicon: com.publicdomainrelay.temp.compute.events.guest.r2Start

itemvalue
rkeySessionRKey("contract-event", uid)
written byrequester only (run request r2 <name>; needs spec.association.fromGuest)
read byguest agent: starts socialweb-r2.service (internal/guestagent/association.go)
stepS8

compute.events.guest.r2Release

lexicon: com.publicdomainrelay.temp.compute.events.guest.r2Release

itemvalue
rkeySessionRKey("contract-event", uid)
written byrequester only: releaseGuestRequester (internal/market/requester/guestrequester.go)
read byguest agent: socialweb-r2-release.service
stepS9

compute.events.guest.r2Released

lexicon: com.publicdomainrelay.temp.compute.events.guest.r2Released

itemvalue
rkeybidder EventRKeys("guest:...")
written bybidder for the guest agent
read byrequester: status.contract.r2Release, condition GuestRequesterReleased
stepS9

compute.events.guest.memoryPressure

lexicon: com.publicdomainrelay.temp.compute.events.guest.memoryPressure

itemvalue
rkeybidder EventRKeys("guest:...")
written bybidder for the guest agent (internal/guestagent/memory.go), capped by memoryEventCapped
read byrequester (requester_market_event), the test modules (readings.memory), the watch page

compute.events.guest.oomKill

lexicon: com.publicdomainrelay.temp.compute.events.guest.oomKill

itemvalue
rkeybidder EventRKeys("guest:...")
written bybidder for the guest agent (internal/guestagent/memory.go), capped by memoryEventCapped
read byrequester (requester_market_event), the test modules (fail the run), the watch page

compute.events.guest.image

lexicon: com.publicdomainrelay.temp.compute.events.guest.image

itemvalue
rkeybidder EventRKeys("guest:...")
written bybidder for the guest agent (internal/guestagent/image.go), one per contract
read byrequester: requester_guest_image {marker, expected, ok}; a contract with an image releases on a mismatch

compute.guest.subscribeEvents

lexicon: com.publicdomainrelay.temp.compute.guest.subscribeEvents

itemvalue
served bybidder GuestControlPlane: handleSubscribeEvents in internal/services/guestcontrolplane/subscribe.go, queue internal/guestcp/queue
called byguest agent: internal/guestagent/subscribe.go (bearer = workload identity token)
carriesrequester-authored compute.events.guest.* payloads, in order; keepalive every 25 s
stepS8

ingress.webRTC

lexicon: com.publicdomainrelay.temp.ingress.webRTC

itemvalue
rkeyrequester listen: webrtc (ingressDescriptor in internal/market/requester/requester.go); offer: TID (createIngressOffer in internal/market/requester/market.go); bidder answer: TID (AnswerIngressRecords in internal/market/bidder/submit.go)
written byrequester (listen, offer), bidder (answer)
read bythe other side through ingress[] on rfp, bid, accept, receipt, offering and the submit calls; only refs to this NSID are allowed (validateIngressList in internal/atproto/market/records.go)
stepS3 to S6

compute.secrets.getSecrets

lexicon: com.publicdomainrelay.temp.compute.secrets.getSecrets

itemvalue
filecompute/secrets/getSecrets.json; SecretsNSID = com.publicdomainrelay.temp.compute.secrets, SecretsRoute in internal/market/requester/secrets.go
typequery-shaped HTTP route: GET /xrpc/com.publicdomainrelay.temp.compute.secrets.getSecrets/<contract>
authAuthorization: Bearer <workload identity token>; audience api://ATProto?actx=<requester DID>; subject claim must match the contract's grant
outputJSON array [{path, value, mode?}]; 401 AuthenticationRequired, InvalidToken; 404 NotFound; 503 NotReady; 500 InternalError
served byrequester ingress: internal/services/requester/routes.go -> Requester.Secrets
called byguest setup-secrets.service (internal/atproto/cloudinit)
stepS8; fallback: guest-events.md

compute.config.wif.simple

lexicon: com.publicdomainrelay.temp.compute.config.wif.simple

itemvalue
filecompute/config/wif/simple.json; WIFSimpleNSID in internal/guestidentity/bidconfig.go
typerecord (bid config)
key fieldsaccept_path, issuer_uri, to_issue (openbao-identity-oidc), actx, token_path, url_path, url_route (/v1/identity/oidc/token), subject (actx:{actx}:plc:{did-plc-key}:role:{role}), createdAt
rkey<bid rkey>-config
written bybidder: writeBidRecord via guests.BidConfig (internal/services/bidder/guests.go, guestidentity.BidConfigFor)
read byrequester: resolveBidConfig and prepareWorkloadGrant (internal/market/requester/secrets.go) to build the workload identity grant; bidder at accept time
stepS4