grain.social/grain

Switch repository

Loading…

Stacked on grain-serve answers profiles, galleries, comments, follows and stories (read/views): its commits are this branch's first, and this pull's own change is its last commit. Merge that one first.

What changes

social.grain.feed.getFeed and describeFeeds (read/feed.rs). A feed is named by at://<the server's DID>/social.grain.feed.generator/<name>, as app.bsky.feed.getFeed names a generator. describeFeeds lists the nine with their labels. A URI naming no feed, or another server's, is 400 UnknownFeed; a server with no DID is 501. Each page is hydrated by views::galleries_at, from the previous pull.

The nine feeds, ported query for query from server/feeds:

• Recent, Following, Hashtag, Camera, Location and Groups list galleries the viewer may see. feed::seen covers that: an account shown, no hiding label from grain's labeler or the author, at least one photo, and no block either way or mute by the viewer. They sort by sort_at, the earlier of createdAt and when the index first held the record, as hatk's galleryFeedTable does.
• Actor and Group leave out blocks and mutes, as hatk did. Actor sorts by createdAt.
• Camera (read/camera.rs) reads the camera through a port of cleanCameraName, so the raw EXIF make and model and the cleaned name find the same galleries. The port is hand-written; the workspace has no regex crate, and the seven rules need none.
• Location (read/location.rs) reads a place's label every way hatk did (three, two or one parts, with expandCountryAliases as place::spellings), plus the place's own name. Where the label finds nothing it falls back to the location cell. A city cell (resolution 5) takes in every cell within it.
• For You (read/foryou.rs) keeps hatk's steps and weights: seeds, co-favoriters who got there first or within a day after, path scores, smoothing, a six-hour half-life and the popularity penalty. With nothing to go on it falls back to the month's most favorited. The time sums are done with SQLite's julianday.

Lexicon. getFeed declares name, the place label the web already sends to the Location feed.

The plan check reads what runs. The hand-written feed and hydration queries in index::tests::the_feeds_read_by_index are gone. read::tests::the_reads_read_by_index now checks the SQL each feed and each view builds, taken from the functions that build it. The views' SQL is lifted into named consts and functions for that. What is left in index::tests (renamed the_queries_still_to_port_read_by_index) is the queries of methods not yet ported.

Why

The second read step of the plan, after views (#1) and before search (#3). Where hatk and Bluesky part, this follows Bluesky:

• Following and For You are the token's caller's (401 AuthenticationRequired without one), as a timeline is. hatk took the account from the actor parameter, so anyone could ask for anyone's.
• The chronological feeds page by a <sort key>::<uri> cursor, not hatk's base64 of the same. For You keeps an offset, since a ranking has no key to resume from. A malformed cursor is 400.
• A city cell is read as ranges of the H3 index. hatk read every gallery with a place and called cellToParent on each in JavaScript, then paged by finding the cursor's URI in the result. H3 numbers a cell's descendants at one resolution in one run, its own digits then the finer ones from 0 to 6. So the galleries within a city are ten ranges of gallery.h3 (resolutions 6 to 15) plus the cell itself, read on gallery_h3. The ranges were checked against h3-js's cellToChildren.
• A gallery its author self-labelled with a hiding value is left out of the feeds, as stories are. hatk's gallery feeds checked grain's labels only.
• For You's cold start fetches one past the page to know whether there is more; hatk guessed from a full page.

How it was checked

tests/read.rs gains a test a feed. Each puts fixtures into the index and calls getFeed through tests/common, so answers.mjs holds every page to galleryView. The tests assert:

• order, including a future-dated gallery sorting as when it was first held;
• pages and cursors;
• what is left out (no photos, labelled, self-labelled, negated label kept, taken down) and what the viewer's blocks and mutes remove, with a token and without;
• Following and For You refused without a token;
• Camera by raw and by cleaned name;
• Hashtag in a description and in a comment;
• Location by every reading of a label, by venue and by city cell, and the fallback from label to cell;
• a group's pool newest first;
• For You's ranking: the two-path gallery above the one-path one, seeds and the viewer's own left out, a co-favoriter two days late not counted, its offset pages, and its cold start.

Unit tests cover cleanCameraName's documented examples, the H3 ranges, the label readings and country spellings. The plan check covers every feed's SQL.

Nothing was built or run on this machine, except the H3 and camera-name expectations, worked out with h3-js and the TypeScript cleanCameraName. CI runs fmt, clippy, the tests, the coverage gate and the answers check.

Shortcomings

• The feed URI's collection, social.grain.feed.generator, is a choice: no lexicon defines a record by that name. It is only ever a name here.
• Location by label still reads json_extract over the galleries in sort_at order, as hatk's did. It walks an index, so the plan check passes it, but a rare label walks far.
• For You still caps a page at 50, as hatk did, below the lexicon's 100.
• The web app still calls dev.hatk.getFeed; moving it is cutover's work.

Review

Start at read/feed.rs, then read/location.rs for the H3 ranges, which are the one new idea here.

🤖 Generated with Claude Code

This adds social.grain.feed.getFeed and describeFeeds, with the nine feeds ported from hatk. It also turns the plan check into a check of the SQL the code actually builds.

I read read/feed.rs, location.rs, foryou.rs and camera.rs in full. I did not read the tests or the rest of the diff. CI had not finished when I looked (test, rust, check and harness were still queued or running), so I can't say it passes.

I found nothing that must change.

• H3 ranges: I checked the bit arithmetic in cells(). The child digits run 0 to 6, the unused digits stay 7, and the same-length lowercase hex strings make BETWEEN compare correctly.
• Cursors and sorting: The (key, tie) < (?2, ?3) cursor and its tie-breaker are sound.
• Auth: Following and For You require a signed-in viewer.
• Camera name port: The byte-based strip only slices at ASCII match boundaries, so it won't panic on non-ASCII input.

Non-blocking: the Camera feed runs SELECT DISTINCT make || ' ' || model FROM photo_exif and cleans every row on each request. That is a full scan that grows with the index. If it shows up in profiling, cache the cleaned-name map or store the cleaned name at index time.