grain.social/grain

Switch repository

Loading…

Stacked on grain-serve keeps when an account last saw its notifications (notifications/state). This pull's own change is its last commit. Merge that one first.

What changes

social.grain.notification.listNotifications and getUnreadCount (read/notification.rs), for the caller's service token only (401 without one). Notifications are not stored. They are found when asked for, as a UNION ALL of nine arms, one for each of hatk's sources:

• favorites of the viewer's gallery, story or comment;
• follows of the viewer;
• replies to the viewer's comments;
• top-level comments on the viewer's gallery or story;
• comments and galleries that mention the viewer.

Each arm leaves out the viewer's own activity. It also leaves out what the read side leaves out (moderation.rs): authors that are taken down or gone, authors who block the viewer or whom the viewer blocks, accounts the viewer muted, and records with a hiding label from grain or from their author. Each arm applies the viewer's notification_pref in SQL: a kind turned off, or a kind limited to accounts the viewer follows. Reasons, hydration (author profileView, gallery title and cover, story thumb, comment and reply text) and the comment-favorite-through-to-its-gallery lookup are hatk's.

Bluesky's read model. The list says seenAt, and each item says indexedAt and isRead. indexedAt is when the index first held the record, or the record's own createdAt (in UTC) if that is earlier. Items are sorted by it, and it is what is compared with seenAt: an item is read when indexedAt <= seenAt. getUnreadCount counts items with indexedAt > seenAt, or all of them if the viewer has never called updateSeen. The cursor is indexedAt::uri, as the other lists' are. The lexicon changes accordingly: unseenCount is gone from the list, seenAt is added, and indexedAt and isRead are added to notificationItem and required there. The embedded copies in hatk.generated.ts are updated to match, and lexicons/README.md notes how this differs from hatk.

mention (migration V6): one row for each account a comment or gallery mentions, by app.bsky.richtext.facet#mention features. Triggers on comment and gallery keep it up to date through inserts, edits and deletes, and the migration fills it from what the index already holds. It is derived, so a rebuild rebuilds it.

The push seam. notification::arising(db, uri) runs the same arms the other way: for one record, it returns whom that record notifies and why. It applies the push preference instead of in_app. unread_count(db, did) is the badge number. The push step calls these for each record the stream creates. Neither push nor registerPush is built here.

Why

The second half of the notifications step, after updateSeen and the state tables (#1 of this stack). The user chose Bluesky's model: seenAt and isRead on the list, and the count as a method of its own.

Why derive at read time rather than keep a notification table. hatk derived them, and every arm reads by index from the recipient (gallery_did then favorite_subject, follow_subject, comment_did then comment_reply_to, mention's key, and so on). Deriving means nothing to backfill, and nothing goes stale when a record is deleted or an account taken down. A table would mean writing a row for every favorite and keeping it in step with deletes, takedowns, blocks and preferences. The plan check covers all three queries in both directions.

Why a mention table. hatk found mentions with facets LIKE '%' || did || '%' over every comment and every gallery, which reads both tables whole on every call. The plan check refuses that.

Where this differs from hatk, and why:

• Preferences filter in SQL, the count too. hatk filtered after its LIMIT, so a page could come back short or empty while a cursor was still offered. Its count ignored preferences, so the badge counted notifications the list hid.
• Each record is one notification at most. hatk listed a top-level comment that mentions the viewer on the viewer's own story twice (story-comment and gallery-comment-mention).
• A reply on the viewer's gallery to someone else's comment, mentioning the viewer, notifies them as a mention. hatk's comment arm excluded replies and its mention arm excluded the viewer's galleries, so this case was never notified.
• Only real mention facets count. hatk's LIKE matched the DID anywhere in the facets, such as inside a link.
• Sorting and read state use indexedAt, as Bluesky's do. hatk sorted on the raw createdAt string, so a record dated in the future stayed at the top and stayed unread. A timezone offset also compared wrongly against the seen time.
• The cursor is indexedAt::uri. hatk's created_at < cursor skipped notifications that shared a timestamp.
• Moderation is the read side's: host status as well as admin takedowns, and labels that hide. hatk checked takedowns, blocks and mutes only.

How it was checked

tests/notifications.rs builds hatk's notificationSources fixture, plus the cases above, against the reference PDS's account as viewer. Every answer goes through common for answers.mjs. The tests cover:

• every reason in order, and each item's hydration: cover from position 0, story thumb, reply text, the gallery a comment-favorite resolves through to, the gallery mention, an author with no profile;
• unread before and after updateSeen, and isRead at, before and after seenAt;
• a record dated in the future, and one with an earlier offset createdAt;
• a hiding label, a takedown, a mute, and a block in each direction, in both the list and the count;
• inApp off, push off (still listed), and from: follows before and after following;
• paging through to the end with no gaps or repeats, and a limit out of bounds or a malformed cursor refused;
• 401 without a token;
• arising: a reply that is also a mention notifies two accounts, and the push preference applies where the in-app one does not;
• mention following an edit that drops a mention, and a delete.

read::tests::the_reads_read_by_index gains the list, count and arising queries and the hydration queries. I ran it, cargo check and clippy here. CI runs the integration tests, the coverage gate and the answers check.

Shortcomings

• Each call reads all of an account's notifications through the indexes, then sorts them, as hatk's did. For an account with many thousands, that is a large read per page and per count. If it gets slow, the fix is a stored notification table filled as records arrive, which the push step would want anyway.
• Notifications on a gallery or story whose author is gone still list, without galleryUri/storyUri, as hatk's did.
• arising is not called yet: push is a later step.
• The web app still calls hatk's getNotifications. Moving it is the web step's.

Review

The design. Start at ARMS and union in read/notification.rs, then V6 in index/schema.rs.

🤖 Generated with Claude Code

This pull adds listNotifications and getUnreadCount. Both are derived at read time from a nine-arm UNION ALL in read/notification.rs. It also adds a trigger-maintained mention table (V6) and the arising and unread_count seams for push.

I read read/notification.rs and the V6 migration in full. I did not read the rest of the diff, the tests or the lexicon changes: the 67k-character diff was saved to a file that my tools could not open. I did not check the CI results.

• Arms: each record comes out as at most one notification. The mention arms exclude the cases the reply and comment arms already cover. Preferences, moderation and the viewer's own activity are all filtered in SQL, so a page can't come back short while a cursor is still offered.
• Read state: seen_at is stored in UTC with milliseconds (set_seen, V5). sort_at is built with the same strftime format, so the text comparisons in isRead and the count line up. A never-seen viewer gets '', so every item counts as unread, as the pull describes.
• V6 triggers: insert, update and delete are all covered, and the backfill uses the same extraction as the triggers. Only real #mention facets with a text did count.

I found nothing that has to change. The pull's own shortcoming about reading all of an account's notifications per page is a fair one to revisit if it gets slow.