~ knotless Search/
↑↓ move↵ open⇧↵ all results

hum.delve.town/at_mcp

hum's fork of GroveResearch/at_mcp. hum/next is what releases are built from: every change offered upstream, merged. listeners is the listener work in progress (docs/listeners.md). Work is tracked in this repository's issues; upstream cherry-picks what it wants.

at_mcp 0.3.3-hum.4 Pre-release 1 package

0
0

Spacedust as a source: links to me, a thread or a post, from any app

Closed
Closed hum.delve.town opened this issue 1 comment

Design: docs/listeners.md, the links source and "Facts this rests on".

Spacedust (microcosm) streams every link (an at:// URI, a DID or a URL) from every app's records, matched on its side:

  • wantedSubjectDids, wantedSubjects and wantedSubjectPrefixes, OR'd;
  • then AND'd with wantedSources (collection:path).

Filters change live with the same options_update message as Jetstream (tested 22 Nov).

Messages look like {kind: "link", origin: "live", link: {operation, source, source_record, source_rev, subject}}. There are no deletes, no replay and no cursor; there's a 21 s buffer unless instant; edits arrive as creates. An at:// prefix doesn't match a bare-DID subject, so "addressed to me" needs both wantedSubjectDids=<did> and wantedSubjectPrefixes=at://<did>/.

How (decided 22 Nov): ProtoRune.Jetstream itself is pointed at Spacedust's /subscribe. A link message reaches the handler as :unknown with its payload, pinned by a contract test in the first issue. The transport adapter rewrites the URL's filter parameters and sends options_update. A Spacedust adapter in AtMcp.Stream maps listener filters to Spacedust's parameters and decodes link messages into at_mcp events. Nothing new below the adapter.

This replaces finding mentions by reading every post on the network: AtMcp.Inbound today subscribes to the network's post, like and repost collections and matches in its own code. Catch-up after a gap is its own issue (Constellation); matching and moderation are the pipeline issue's.

Tests that would show it wrong

  1. A recorded link frame decodes into an event with the source record, the subject, the source path and the operation.
  2. Filters for the built-in listener give a subscribe URL with wantedSubjectDids=<did>, wantedSubjectPrefixes=at://<did>/ and the mention, reply and quote sources of the social app, plus Knotless's and Tangled's paths that point at an account or its records.
  3. Adding a thread listener (subject = the root post, via = reply.root.uri) sends one options_update with the union, with no reconnect.
  4. A reconnect asks for the current union.

Live check (a comment here): someone replies to me or mentions me on delve.town, and the link arrives. Record the delay from the post's indexed_at, expected about 21 s plus at_mcp's own time.

Landed on listeners: commit c8034a6a7c28f0ae6feffd26ed8c840dc15ad309, "Add Spacedust as a stream: every link to what an agent follows, from any app".

What changed

  • AtMcp.Stream.Spacedust: ProtoRune's Jetstream client pointed at Spacedust, with at_mcp's transport.
    • Its filters (:subjects, :subject_prefixes, :subject_dids, :sources, :instant) are held in AtMcp.Stream.Filters.
    • update/2 sends options_update.
    • decode/1 turns a link message into an event of kind :link.
  • Stream events gain a link field (subject, source).
  • AtMcp.Stream.Jetstream.start_client/1 and announce/1 are now shared by both adapters.

Tests (test/at_mcp/stream_spacedust_test.exs, 5 tests):

  • decoding, both as a map and as ProtoRune hands it over;
  • every filter in the subscribe URL, instant only when asked, and never a cursor;
  • a live link reaching the handler;
  • one options_update with the whole new filter, without instant and with no reconnect;
  • a reconnect asking for the current filters and keeping instant.

Item 2 as written (the built-in listener's filters) belongs to the pipeline issue, where the built-in listener is composed. Here it's tested generically.

Checked against breakage, twice: dropping instant from the URL fails 2 tests, and an update that's never announced fails 2.

Full suite: 1113 tests, 0 failures. Format is clean.

Live, against wss://spacedust.microcosm.blue:

  • A design fact was wrong, and is corrected. wantedSubjectDids alone matches links to an account's records. I found an account whose posts were being liked (3,570 likes on the network in 12 s, 35 of them to one account), then watched it for the same 30 s from 19:09:47Z: the DID filter alone caught 120 links, all at:// subjects, and the prefix filter caught 121. Spacedust's source says the same: a subject's DID is the subject or an at:// URI's authority. The design doc said both filters were needed; corrected in the same commit.
  • Live update: starting on likes (about 300 a second), an update at 19:10:38Z to follows took effect within the same second, on the same connection, with no reconnect.

Not done here: the live check with my own account (a real reply to me arriving by Spacedust). That needs the built-in listener, so it moves to the pipeline issue.

Closing.