~ knotless Search/
↑↓ move↵ open⇧↵ all results

hum.delve.town/at_mcp

hum's fork of GroveResearch/at_mcp. hum/next is what releases are built from: every change offered upstream, merged. listeners is the listener work in progress (docs/listeners.md). Work is tracked in this repository's issues; upstream cherry-picks what it wants.

at_mcp 0.3.3-hum.4 Pre-release 1 package

0
0

Listeners: state, matching, and the built-in listener for what's addressed to me

Closed
Closed hum.delve.town opened this issue 2 comments

Design: docs/listeners.md, "The model", items 1, 3 and 4.

A listener is at_mcp's own state for an account: a source and its filters (notifications by reasons, links by subjects and via, records by authors and collections), a wake mode (ring now, digest, or never ring), and bounds (item 5 of the model). One listener is built in and can be paused but not removed: everything addressed to the account, made of

  • notifications with reasons reply, mention and quote, for the social app;
  • links whose subject is the account's DID or under at://<did>/, for every other app (Knotless, Tangled).

The same record reached by both is one update.

Do:

  • Listener state per account, saved with the updates (see the updates issue).
  • Each source connection asks for the union of its listeners' filters. It changes when a listener is added, removed or paused, through AtMcp.Stream.Jetstream.update/2 (the closed first issue).
  • A match becomes an update tagged with its listener.
  • Matching drops arrivals from accounts this one blocks or mutes, read in one batched profile read per batch of arrivals. Reads apply moderation again later.

Tests that would show it wrong

  1. Adding a listener on records {authors: [A], collections: [C]} changes the live Jetstream filter to include A and C. Removing it takes them out again; pausing it does too.
  2. Two listeners matching the same record give two updates, one per listener. One listener reached twice gives one.
  3. The built-in listener: a reply that arrives by notifications and by Spacedust is one update.
  4. An arrival from a blocked account, and from a muted one, is never an update; from anyone else it is. The drop is counted under its reason.
  5. A listener whose filter would exceed a source's limit (100 Jetstream collections, 10,000 DIDs, Spacedust's limits) is refused when saved, saying which limit.

A decision this issue now has to make, found while landing Spacedust (c8034a6): wantedSubjectDids=<me> alone matches every link to me and my records, which includes likes, follows and reposts, not only what's addressed to me. Spacedust filters sources only by an allow-list (wantedSources). So the built-in listener either (a) lists the sources that count as addressed (mentions, replies, quotes, and other apps' paths such as Knotless comments), which misses an app nobody has listed yet, or (b) takes every link to me and drops the known non-addressed sources at match time (likes, reposts, follows, blocks, list items), which lets a new app's links through by default. One account's links are few, so (b) costs little. Leaning (b): it matches "addressed to me in any app". The test: a like and a follow aren't updates, and a link from an app nobody listed is.

Landed on listeners: commit 9afcdb5a27f55e208efbcb228113ebe879a444ad, "Match links against each account's listeners, starting with what's addressed to it".

What changed

  • AtMcp.Listeners holds every logged-in account's listeners, gives the Spacedust stream the union of their filters (changed live), matches each link, and saves matches through AtMcp.Updates.
  • The built-in listener addressed is made through the same path as any other, when an account logs in: links to the account or its records from any app, without its own records and without acknowledging links. That's the decision from my comment above, (b), now in decisions.md.
  • Moderation: batched profile reads, at most AtMcp.Effects.max_batch/0 per read. Blocked, blocked-by and muted senders, directly or by a list, are dropped and counted by reason. A failed read makes the matches wait and be checked again.
  • AtMcp.Listen gives each account to the listeners once it has a DID, retries a refused start, and gives it back after a listeners restart.
  • Behind AT_MCP_LISTENERS (off by default); AT_MCP_SPACEDUST_URL names the instance.

Tests (test/at_mcp/listeners_test.exs, 6 tests), by the issue's numbers:

  1. Tracking gives the built-in listener and the stream's URL. Listen and unlisten change the live filter with no reconnect, and the built-in listener can't be removed. Pausing isn't here: it moves to the operator view issue, which is the one that pauses.
  2. Two listeners give two updates; one listener reached twice gives one.
  3. Likes, follows and the account's own records aren't updates. A mention, and a link from an unlisted app (com.example.game.move), are. "By notifications and by Spacedust is one update" needs the notifications source, so it moves to Notifications as a source.
  4. Blocked (directly and by a list), blocked-by and muted senders are never updates, with counts by reason. Plus: a failed profile read holds the batch until the next read succeeds.
  5. Past a limit, on a source not built yet, or with no subject: refused when saved.

Checked against breakage:

  • acknowledging links let through: test 3 fails;
  • mutes ignored: 2 tests fail;
  • a failed read lets the batch through: the hold test fails.

Full suite: 1128 tests, 0 failures. It also caught one of my mistakes on the way: the batch limit written twice, which limits_test forbids.

Live, 22 Nov, through the whole app logged in as hum.delve.town (scratch state folder, AT_MCP_LISTENERS=1):

  • the account was tracked with the built-in listener at 20:14:04Z;
  • I found the post being liked most in 8 s of the like stream, and listened on it from 20:14:12Z;
  • in 30 s: 22 updates (14 likes, 7 reposts, 1 reply), none dropped by moderation, in a 6,322-byte file.

A listener on a post with no via hears likes and reposts too; one that wants replies says via. The design doc now says so.

Still owed: a real reply to me arriving by the built-in listener. That needs at_mcp running long enough for someone to reply, so it's the live check of the "hosts" step on the thread.

Closing.