johnandersen777.bsky.social/socialweb-computer-watch
The socialweb.computer market's live topology from Jetstreams and relays, deduplicated; capture and replay
0
0
# socialweb-computer-watch
The socialweb.computer market, live: https://watch.socialweb.computer/

- **Sources**: Jetstreams and relays at once, editable (default: Bluesky's and the socialweb.computer relay on
mini-cloud-0002, which carries the bidders' and guests' repos). Each op counts once (repo/collection/rkey@rev);
the table shows per source its ops, how many it delivered first, and the duplicates dropped.
- **Topology**: CI runs (`dev.knotless.check`) → requesters → rented VMs (rfp → bid → accept → receipt → events)
→ guest requesters (R2, named by `guest.requesterDid`) → their VMs; every VM → its bidder; principal → agent
(`market.association`, `link.ident.endorsement`, `alsoKnownAs` agent-of). A run's VMs are its repo owner's made
while it ran (dashed: inferred).
- **Filter**: a DID, handle, repo, commit or workflow keeps what hangs off it.
- **Replay**: a capture file (below), with speed and scrub.
- Nothing persists: a reload starts over. Built on [atlibs](https://knotless.dev/johnandersen777.bsky.social/atlibs)
(`subscribeXrpc`, `decodeCbor`, `resolveDid`, `pdsOf`).
```sh
git clone https://knotless.dev/johnandersen777.bsky.social/socialweb-computer-watch
cd socialweb-computer-watch
npm ci
python3 -m http.server 8799 --bind 127.0.0.1 # http://127.0.0.1:8799/
```
URL parameters: `?filter=<text>&rewind=<minutes of Jetstream history, default 30>&sources=<url>,<url>`.
Test (the page test needs Chrome; `CHROME` names another binary):
```sh
node --test test/model.test.mjs
node test/page.mjs http://127.0.0.1:8799/ --wait 30
```
## Capture
Node 22+, the default sources, deduplicated, until `--seconds` or Ctrl-C (`--all` keeps duplicate frames too):
```sh
node capture.mjs --out capture.ndjson --seconds 600 --rewind-minutes 30
```
In CI, in the background (Linux; fetches Node v26.13.0, sha256 checked, when there is no Node 22+):
```sh
git clone https://knotless.dev/johnandersen777.bsky.social/socialweb-computer-watch /opt/swc-watch
sh /opt/swc-watch/scripts/ci-capture.sh /work/runs/watch.ndjson 7200
```
With websocat 1.14, goat 0.2.7 and jq 1.7 instead, one file per source, then one deduplicated file:
```sh
mkdir -p capture && cd capture
W='wantedCollections=com.publicdomainrelay.temp.*&wantedCollections=computer.socialweb.*&wantedCollections=link.ident.*&wantedCollections=dev.knotless.*'
cursor=$(( ($(date +%s) - 1800) * 1000000 ))
for js in jetstream1.us-east jetstream2.us-west; do
websocat -n -U -B 4194304 "wss://$js.bsky.network/subscribe?$W&cursor=$cursor" \
| jq -c --unbuffered --arg src "wss://$js.bsky.network/subscribe" '{src: $src, recv: (now * 1000 | floor), frame: .}' > $js.ndjson &
done
websocat -n -U -B 4194304 wss://relay.mini-cloud-0002.chadig.com/xrpc/com.atproto.sync.subscribeRepos \
| jq -c --unbuffered --arg re '^(com\.publicdomainrelay\.temp|computer\.socialweb|link\.ident|dev\.knotless)\.' \
'select(any(.ops[]?; .path | test($re))) | {src: "wss://relay.mini-cloud-0002.chadig.com", recv: (now * 1000 | floor), frame: .}' > mini-cloud-0002.ndjson &
c="-c link.ident.endorsement -c dev.knotless.check"
for n in market.rfp market.bid market.accept market.receipt market.event market.heartbeat market.offering \
market.association market.associationRequest compute.vm compute.events.guest.requesterDid; do c="$c -c com.publicdomainrelay.temp.$n"; done
goat firehose --relay-host wss://relay1.us-east.bsky.network --blocks $c \
| jq -c --unbuffered 'select(.type == "commit") | {src: "wss://relay1.us-east.bsky.network", recv: (now * 1000 | floor), frame: .}' > relay1.us-east.ndjson &
sleep 600; kill $(jobs -p)
cd .. && jq -sc 'sort_by(.recv)[]' capture/*.ndjson | jq -nc -f dedup.jq > capture.ndjson
```
A repo's records as they are now, with curl (one collection; the page takes these lines too):
```sh
did=did:plc:7q5lnitxv4meizdon5fmhzrq c=com.publicdomainrelay.temp.market.receipt
pds=$(curl -s https://plc.directory/$did | jq -r '.service[] | select(.id == "#atproto_pds").serviceEndpoint')
curl -s "$pds/xrpc/com.atproto.repo.listRecords?repo=$did&collection=$c&limit=100" | jq -c --arg did $did '.records[]
| {src: "listRecords", recv: (.value.createdAt | sub("\\.[0-9]+Z$"; "Z") | fromdate * 1000), frame: {did: $did, time_us: 0,
kind: "commit", commit: {rev: "", operation: "create", collection: (.uri | split("/")[3]), rkey: (.uri | split("/")[4]), cid, record: .value}}}' > records.ndjson
```
## Deploy
Stage `dist/`, back up the live directory and the Caddy config on the socialweb.computer gateway, serve
https://watch.socialweb.computer/:
```sh
TILE_AS=you.bsky.social scripts/deploy.sh
```
It then publishes the same files as a [web tile](https://dasl.ing/tiles.html) (`ing.dasl.masl`, written to
`records/tiles/`; a tile frame has no network, so there it replays a capture), as `TILE_AS`'s `atlibs login`
session, or its app password in `ATPROTO_APP_PASSWORD`. `NO_TILE=1` skips the tile.
| what | from |
|---|---|
| `@publicdomainrelay/atlibs` 0.7.1, `d3` 7.9.0 | npm (`package-lock.json`), served from `node_modules/` |