~ knotless Search/
↑↓ move↵ open⇧↵ all results

johnandersen777.bsky.social/socialweb-computer-watch

The socialweb.computer market's live topology from Jetstreams and relays, deduplicated; capture and replay

0
0
main
Go to filet
Code
README.md

socialweb-computer-watch

The socialweb.computer market, live: https://watch.socialweb.computer/

  • Sources: Jetstreams and relays at once, editable (default: Bluesky's and the socialweb.computer relay on mini-cloud-0002, which carries the bidders' and guests' repos). Each op counts once (repo/collection/rkey@rev); the table shows per source its ops, how many it delivered first, and the duplicates dropped.
  • Topology: CI runs (dev.knotless.check) → requesters → rented VMs (rfp → bid → accept → receipt → events) → guest requesters (R2, named by guest.requesterDid) → their VMs; every VM → its bidder; principal → agent (market.association, link.ident.endorsement, alsoKnownAs agent-of). A run's VMs are its repo owner's made while it ran (dashed: inferred).
  • Filter: a DID, handle, repo, commit or workflow keeps what hangs off it.
  • Replay: a capture file (below), with speed and scrub.
  • Nothing persists: a reload starts over. Built on atlibs (subscribeXrpc, decodeCbor, resolveDid, pdsOf).
git clone https://knotless.dev/johnandersen777.bsky.social/socialweb-computer-watch
cd socialweb-computer-watch
npm ci
python3 -m http.server 8799 --bind 127.0.0.1   # http://127.0.0.1:8799/

URL parameters: ?filter=<text>&rewind=<minutes of Jetstream history, default 30>&sources=<url>,<url>.

Test (the page test needs Chrome; CHROME names another binary):

node --test test/model.test.mjs
node test/page.mjs http://127.0.0.1:8799/ --wait 30

Capture

Node 22+, the default sources, deduplicated, until --seconds or Ctrl-C (--all keeps duplicate frames too):

node capture.mjs --out capture.ndjson --seconds 600 --rewind-minutes 30

In CI, in the background (Linux; fetches Node v26.13.0, sha256 checked, when there is no Node 22+):

git clone https://knotless.dev/johnandersen777.bsky.social/socialweb-computer-watch /opt/swc-watch
sh /opt/swc-watch/scripts/ci-capture.sh /work/runs/watch.ndjson 7200

With websocat 1.14, goat 0.2.7 and jq 1.7 instead, one file per source, then one deduplicated file:

mkdir -p capture && cd capture
W='wantedCollections=com.publicdomainrelay.temp.*&wantedCollections=computer.socialweb.*&wantedCollections=link.ident.*&wantedCollections=dev.knotless.*'
cursor=$(( ($(date +%s) - 1800) * 1000000 ))
for js in jetstream1.us-east jetstream2.us-west; do
  websocat -n -U -B 4194304 "wss://$js.bsky.network/subscribe?$W&cursor=$cursor" \
    | jq -c --unbuffered --arg src "wss://$js.bsky.network/subscribe" '{src: $src, recv: (now * 1000 | floor), frame: .}' > $js.ndjson &
done
websocat -n -U -B 4194304 wss://relay.mini-cloud-0002.chadig.com/xrpc/com.atproto.sync.subscribeRepos \
  | jq -c --unbuffered --arg re '^(com\.publicdomainrelay\.temp|computer\.socialweb|link\.ident|dev\.knotless)\.' \
    'select(any(.ops[]?; .path | test($re))) | {src: "wss://relay.mini-cloud-0002.chadig.com", recv: (now * 1000 | floor), frame: .}' > mini-cloud-0002.ndjson &
c="-c link.ident.endorsement -c dev.knotless.check"
for n in market.rfp market.bid market.accept market.receipt market.event market.heartbeat market.offering \
  market.association market.associationRequest compute.vm compute.events.guest.requesterDid; do c="$c -c com.publicdomainrelay.temp.$n"; done
goat firehose --relay-host wss://relay1.us-east.bsky.network --blocks $c \
  | jq -c --unbuffered 'select(.type == "commit") | {src: "wss://relay1.us-east.bsky.network", recv: (now * 1000 | floor), frame: .}' > relay1.us-east.ndjson &
sleep 600; kill $(jobs -p)
cd .. && jq -sc 'sort_by(.recv)[]' capture/*.ndjson | jq -nc -f dedup.jq > capture.ndjson

A repo's records as they are now, with curl (one collection; the page takes these lines too):

did=did:plc:7q5lnitxv4meizdon5fmhzrq c=com.publicdomainrelay.temp.market.receipt
pds=$(curl -s https://plc.directory/$did | jq -r '.service[] | select(.id == "#atproto_pds").serviceEndpoint')
curl -s "$pds/xrpc/com.atproto.repo.listRecords?repo=$did&collection=$c&limit=100" | jq -c --arg did $did '.records[]
  | {src: "listRecords", recv: (.value.createdAt | sub("\\.[0-9]+Z$"; "Z") | fromdate * 1000), frame: {did: $did, time_us: 0,
  kind: "commit", commit: {rev: "", operation: "create", collection: (.uri | split("/")[3]), rkey: (.uri | split("/")[4]), cid, record: .value}}}' > records.ndjson

Deploy

Stage dist/, back up the live directory and the Caddy config on the socialweb.computer gateway, serve https://watch.socialweb.computer/:

TILE_AS=you.bsky.social scripts/deploy.sh

It then publishes the same files as a web tile (ing.dasl.masl, written to records/tiles/; a tile frame has no network, so there it replays a capture), as TILE_AS's atlibs login session, or its app password in ATPROTO_APP_PASSWORD. NO_TILE=1 skips the tile.

whatfrom
@publicdomainrelay/atlibs 0.7.1, d3 7.9.0npm (package-lock.json), served from node_modules/