grain.social/grain-support-agent
grain's support agent: triages mentions and reports, investigates bugs, proposes fixes
Keep anything about a child-safety report from models
OpenThis pull keeps child-safety reports away from models. The test is in child-safety.ts. It's checked at intake, in the workers, in investigate and its resume, in conversation turns, in every support-mcp tool and in the dashboard routes. Model-initiated moderation is refused, and SQL access to report and classification is closed. Briefs now get an account's earlier reports from getReviewSubject. The guards are thorough. Unknown or failed lookups count as child safety, the migration's LIKE patterns match what the poller writes for both classifier and person reports, and the tests cover the paths the body describes.
One gap stops it delivering its main promise. The SQL tool can still read the label table. Any investigation or conversation session, including one on a clean item, can run SELECT uri, did FROM label WHERE val = 'csae' and get the account DIDs and record URIs of every csae-labeled subject in grain. moderationPrompt already leaves labels out for a child-safety account because "they name its records", and the README says a model never sees "its subject's record". The SQL path undoes both.
The authorizer only allows or denies whole tables and columns, so it can't filter csae rows. Close label along with report and classification, since the brief already gets labels from getReviewSubject and shows them only when it's safe to. Then update the "tables that matter here" bullet and the tool descriptions to match, and add a test to grain-db.test.ts. If takedown stores free-text reasons, check it the same way.
apps/server/src/grain-db.tsline 30
]); | |
| |
/** Reports and what the classifiers said: a model reads them only as the brief shows them. */ | |
export const MODERATION_TABLES = new Set(["report", "classification"]); |
label should be here too. Its rows include every csae label with the uri of the labeled record or account. Today any session's sql_query can read SELECT uri FROM label WHERE val = 'csae' and get exactly the subjects this pull withholds from briefs. The authorizer can't filter rows, so closing the table is the reliable fix. Briefs already get labels from getReviewSubject, via the account object in moderationPrompt, which leaves them out for child-safety accounts.
apps/server/src/investigate.tsline 108
| |
You can also query grain-serve's index, the appview's database, read-only with the grain-db tools. grain/crates/grain-serve/src/index/schema.rs describes every table. The tables that matter here: | |
| |
- label: labels grain gave out on the account or its records (uri is the bare DID for an account, an at:// URI for a record; neg = 1 is a retraction). |
This bullet still sends the agent to the label table, which leaks csae subjects (see the comment on grain-db.ts). Once label is closed, remove this bullet and rely on the labels in the getReviewSubject block above.
1 review asking for changes
The support agent now follows grain-mcp's child-safety rule: what a child-safety report is about never reaches a model, and only a person acts on it.
The test
childSafety(label, reasonType)inapps/server/src/child-safety.tsis grain'slabels::child_safety: the label iscsae, or the reason type is one thatlabels::label_forfiles under csae. That means everytools.ozone.report.defs#reasonChildSafety*and no other reason type. A report is also in scope when grain-serve'ssocial.grain.admin.getReviewSubjectgives its accountchildSafetyanything butfalse. That flag is set by a child-safety report on the account or its records, in any state, or acsaelabel, negated or not. grain-mcp'sclear()makes the same call. If the lookup fails or the item has no account, the report is treated as child safety. Accounts found child safety are cached for the life of the process. Clear accounts are cached for 30s. Lookup errors are never cached.Guards
reports.ts): the report is stored whole for the dashboard and goes straight toneeds_review.report(the brief) holds only its ids, its date and grain-mcp's HUMAN line.main.ts): the triage worker checks before Clef, and the investigate worker checks before a session starts. Resumed investigations, turns and fixes are checked as well.investigate(),resumeInvestigation(), conversation turns and every support-mcp tool (itemFor) check the item again at the moment of use. They refuse with HUMAN. When an account has become child safety since intake, the item is marked from then on.modelText(). Briefs now get an account's earlier reports fromgetReviewSubject, run throughshownReport().reportandclassificationare denied by the authorizer and hidden fromlist_tablesanddescribe_table. Tool descriptions and prompts were updated to match.act(item, m, by). The"model"path refusescsae, any child-safety item or account, and any open child-safety report on the subject. The dashboard's confirm button uses"person"and still works.messagesandinvestigatereturn 409 with HUMAN. The thread shows HUMAN as an event.Tests
test/child-safety.test.tsuses fixtures only: a csae-labeled classifier report on a photo, a child-safety reason type, innocuous reports on accounts with a prior csae report or a csae label, a failed lookup, a clean account, and an account that becomes child safety later. They check that no prompt, brief ormodelTextcontains the reason, subject URI, image link, handle or gallery title. They also check that nothing reached a model host, thatinvestigatethrows before opening a session, thatactas model refuses whileactas person works, and thatlabelNamesleaves out csae.grain-db.test.tscovers the SQL denial.npm test -w server: 37/37 pass;tsc --noEmitis clean.🤖 Generated with Claude Code