Identity migration runbook
The Rust service stores one canonical software.release.project per product.
Legacy caches and repositories instead hold one project per raw
{part}:{vendor}:{product} observation, so Red Hat Enterprise Linux is split
across red-hat-red-hat-enterprise-linux-8, redhat-enterprise-linux,
redhat-rhel-els, and other variants.
identity-migrate is the one-time, out-of-band admin tool that rewrites those
references and deletes the legacy release records. There is no runtime
migration phase, no startup ALTER TABLE, and no implicit write from
vuln-notif: the service never runs this migration itself. A human runs
plan, reviews the map, then runs apply and verify.
Locations:
- Binary:
target/release/identity-migrate(src/bin/identity-migrate.rs). - Plan implementation:
src/migrate/. - Curated aliases:
data/identity/aliases.json. - Local e2e:
scripts/identity-migrate-e2e.sh. - Fixtures:
tests/fixtures/identity-migration/.
Identity rules
identity = (class, canonical_vendor, canonical_family)
slug = truncate64(join(canonical_vendor, canonical_family, '-'))
classisos,app,hardware, orfirmware. Curated OS families (RHEL, Fedora, CentOS, Debian, Ubuntu, SUSE, Oracle Linux, Amazon Linux, Windows, macOS) are alwaysos. Otherwise CPE part mapso→os,a→app,h→hardware. The rawpartis not part of the identity, so thea/oRHEL duplicates collapse.canonical_vendoris CPE-unescaped, lowercased, non-alphanumerics collapsed to-, then looked up in the curated vendor aliases (redhat,red_hat,Red Hat→red-hat).canonical_familystrips a redundant vendor prefix, then applies the curated product aliases.enterprise_linux*,rhel_els,enterprise_linux_for_*, andcodeready_linux_builder*all becomered-hat-enterprise-linux.- Trailing stream/version text is split only for families listed in
versioned_families.Xand single-digitX.0are NVD major sentinels: they produce a channel and no version.X.Yproduces channelXand versionX.Y(9.4,8.10, multi-digit10.0preserved). Edition/arch tokens (eus,els,aus,sap,rt,aarch64, ...) become channel suffixes; defaultx86_64is omitted. - Channels never contain dots (
Ubuntu 20.04→ channel20-04); versions keep dots. Version and manifest records are immutable; only channels move. - Placeholder vendors/products (
n/a,any,unknown,*,-) produce no identity. The oldn-a-n-aproject is never created again. - Slug collisions between different identities are resolved deterministically:
a curated family owns the base slug, otherwise the lexicographically smaller
identity key owns it and the other gets
-<sha256(identity_key)[:6]>. - Every write is same-DID with the repository. Cross-DID strongRefs are
rejected by the tool and by
verify.
RHEL example:
| Observation | Legacy raw slug | Canonical project | Channel |
|---|---|---|---|
NVD Red Hat + Red Hat Enterprise Linux 8 | red-hat-red-hat-enterprise-linux-8 | red-hat-enterprise-linux | 8 |
CPE cpe:2.3:o:redhat:enterprise_linux:9.4 | redhat-enterprise-linux | red-hat-enterprise-linux | 9 (version 9.4) |
NVD redhat + rhel_els | redhat-rhel-els | red-hat-enterprise-linux | els |
Legacy com.nahdig.software records are only encountered when migrating old
snapshots; the current service does not publish them. Raw vendor, product,
and cpes live in the local cache and in CVE software[] entries.
Alias table
data/identity/aliases.json is embedded with include_str!, validated at
startup and by tests. Editing this file and rebuilding is the only supported
way to merge identities; there is no fuzzy matching, stemming, or
batch-order dependence.
{
"vendors": {"redhat": "red-hat", "red-hat": "red-hat"},
"products": {"enterprise-linux": "red-hat-enterprise-linux",
"rhel-els": "red-hat-enterprise-linux"},
"families": {"red-hat-enterprise-linux": "red-hat"},
"versioned_families": ["red-hat-enterprise-linux"],
"class_overrides": {"red-hat-enterprise-linux": "os"}
}
vendors: normalized vendor → canonical vendor.products: normalized product alias → canonical family.families: canonical family → canonical vendor (used for slug ownership).versioned_families: families that may split trailing version/edition text.class_overrides: family → identity class.
Every legacy → canonical pair appears in the plan reason and in the plan
entries before apply, so the diff is reviewable per record.
Migration phases
snapshot → plan → review diff → apply → verify
- Snapshot. Write every record in the migration collections to JSONL. Read-only on the cache and PDS.
- Plan. Inventory the local cache (
cves,software,software_purls, and purl2cpe when available) plus the remotelistRecordsinventory, then write a reviewable JSON map.planperforms no writes. - Review diff. Inspect every
delete_release_record,repoint_software,repoint_purl,repoint_cve,create_project, andnormalize_purl_slugentry. Do not continue if a pair is wrong; project rkeys are permanent. - Apply. Execute the map in dependency order with a persisted cursor.
Already-
doneentries are skipped, so re-running is safe. - Verify. Re-read the repository, assert canonical projects exist, legacy
projects have no unmanaged backlinks, CVE refs are deduped and canonical,
and strongRef CIDs are current.
verifyexits1if any error is reported.
Local
scripts/local_plc.sh start
scripts/local_atproto_pds.sh start
eval "$(scripts/create_local_account.sh 2>/dev/null)" # exports ATP_PDS_URL/HANDLE/APP_PASSWORD
target/release/identity-migrate snapshot \
--cache-db .state/local.db --snapshot-dir migration/snapshot-local
target/release/identity-migrate plan \
--cache-db .state/local.db --purldb .state/purl2cpe.db \
--out migration/plan-local.json
target/release/identity-migrate apply \
--cache-db .state/local.db --map migration/plan-local.json \
--batch 200
target/release/identity-migrate verify \
--cache-db .state/local.db --map migration/plan-local.json
scripts/identity-migrate-e2e.sh runs this whole flow against a temporary PLC
and PDS, seeds the legacy RHEL split, and asserts the plan, apply report,
zero-write second apply, and verify report.
Production
Stop the service first (freeze writers) and keep it stopped until verify
passes. Never run apply concurrently with a service cycle.
export ATP_HANDLE=did:plc:... # repository DID/handle, not a secret
export ATP_APP_PASSWORD=... # privileged app password
unset ATP_PDS_URL # resolve the PDS from PLC, or set it explicitly
DB=.state/prod-week.db
MAP=migration/identity-$(date +%Y%m%dT%H%M%S)
cp "$DB" "$DB.pre-migration" # cache/rate-budget/cursor backup
target/release/identity-migrate snapshot \
--cache-db "$DB" --snapshot-dir "$MAP.snapshot"
target/release/identity-migrate plan \
--cache-db "$DB" --purldb .state/purl2cpe.db \
--out "$MAP.plan.json"
# Review before applying.
jq -r '.entries[] | .action' "$MAP.plan.json" | sort | uniq -c
jq -r '.entries[] | select(.action == "create_project") | .canonical_slug' "$MAP.plan.json"
jq -r '.entries[] | select(.action == "delete_release_record") |
[.legacy_slug, .canonical_slug, .legacy_uri] | @tsv' "$MAP.plan.json" | sort -u
jq -r '.entries[] | select(.action == "repoint_cve") | .legacy_slug' "$MAP.plan.json" | wc -l
target/release/identity-migrate apply \
--cache-db "$DB" --map "$MAP.plan.json" \
--batch 200 --snapshot-dir "$MAP.snapshot"
target/release/identity-migrate verify \
--cache-db "$DB" --map "$MAP.plan.json"
apply prints one identity_apply event with a report:
applied, skipped_done, created_projects, software_repoints,
purl_repoints, cve_repoints, deleted, and normalized. verify prints
identity_verify with canonical_projects, software_repoints, purl_repoints,
cve_repoints, deletions, stale_refs, and errors.
If the cache holds no repository DID (or several), pass
--handle did:plc:... explicitly to normalize-purl-slugs.
Dependency order
apply sorts and executes in this order; never delete a release record before
its referrers move, because strongRef CIDs cascade.
| Order | Action | Effect |
|---|---|---|
| 1 | create_project | software.release.project/<canonical> if missing |
| 2 | repoint_software | com.nahdig.software.project strongRef |
| 3 | repoint_purl | com.nahdig.purl.software[].cid |
| 4 | repoint_cve | com.nahdig.cve.atprotoV1.software[].project; dedupe to one canonical ref; drop refs to records being deleted |
| 5 | normalize_purl_slug | cache-only: software_purls.software_slug AT-URI → slug |
| 6 | delete_release_record | applyWrites batches of legacy project/channel/version/manifest/status records |
Local link rows are not rebuilt from listRecords. The service indexes links
as it writes, repairs failed indexes from link_repairs, and re-indexes
migrated URIs on its next pass; verify checks the local links table only
for unmanaged backlinks to legacy projects.
Rate budget
PDS write costs:
| Operation | Points |
|---|---|
| create / put (new record) | 3 |
update (putRecord on existing record) | 2 |
| delete | 1 |
applyWrites | each record in the batch at its own operation cost (deletes cost 1) |
The migration's referrer moves are putRecord updates (2 points each); legacy
record deletions are applyWrites deletes (1 point per record). Budget:
5,000 points/hour and 35,000 points/day per repository DID, with a 0.9 safety
margin. The governor reserves points before each write and persists state in
the cache DB's rate_budget table.
| Work | Records | Approx. points |
|---|---|---|
| CVE refs | ~14.3k updates | ~28.6k |
| software/purl refs + legacy deletes | ~100 | <1k |
The CVE pass is close to one daily budget, so plan it as a day-long run. Use
--limit to cap entries per invocation and --batch (default 200) for deletes.
Stopping and resuming does not lose budget: it is persisted with the cursor.
Rollback via snapshot
snapshot writes one <collection>.jsonl file per migration collection; each
line is {"uri", "cid", "value"} for a live record. Snapshots are full
collection supersets, so they are also a cache-independent backup of the
release/software/purl/CVE records.
A snapshot taken before apply, plus the cache DB copy, is the rollback
material. Restoring is a manual putRecord replay (the tool intentionally has
no destructive rollback command):
SNAP=migration/identity-YYYYmmddTHHMMSS.snapshot
DID=$ATP_HANDLE
ACCESS_JWT=$(curl -fsS -X POST "$ATP_PDS_URL/xrpc/com.atproto.server.createSession" \
-H 'Content-Type: application/json' \
-d "$(jq -nc --arg identifier "$ATP_HANDLE" --arg password "$ATP_APP_PASSWORD" \
'{identifier:$identifier,password:$password}')" | jq -r .accessJwt)
for file in "$SNAP"/*.jsonl; do
collection=$(basename "$file" .jsonl)
while IFS= read -r line; do
rkey=$(jq -r '.uri | split("/") | last' <<<"$line")
value=$(jq -c '.value' <<<"$line")
jq -nc --arg repo "$DID" --arg collection "$collection" \
--arg rkey "$rkey" --argjson record "$value" \
'{repo:$repo,collection:$collection,rkey:$rkey,record:$record}' |
curl -fsS -X POST \
-H "Authorization: Bearer $ACCESS_JWT" \
-H 'Content-Type: application/json' \
-d @- "$ATP_PDS_URL/xrpc/com.atproto.repo.putRecord"
done <"$file"
done
Restore all collections, then re-run verify with the original plan: it
should now report the pre-migration errors (legacy refs), which confirms the
restore is complete. Restoring the cache DB copy rolls back the cursor and
rate-budget rows as well. Do not restore only the project records: software,
purl, and CVE strongRefs would still point at deleted records.
Failure recovery and resume
apply persists one cursor row per (repo_did, phase, raw_slug) in
identity_migration with status pending, done, or error and an
attempts counter. On a failed write it marks the entry error, commits, and
exits nonzero. Completed entries are never repeated.
# Retry pending/error entries; done entries are skipped.
target/release/identity-migrate apply \
--cache-db "$DB" --map "$MAP.plan.json" --resume
# Inspect progress.
sqlite3 "$DB" "SELECT action, status, count(*) FROM identity_migration GROUP BY 1,2;"
sqlite3 "$DB" "SELECT phase, raw_slug, attempts, error FROM identity_migration WHERE status='error';"
- Without
--resume, anerror/pendingcursor row aborts with "pass --resume to retry". This prevents accidental repeated writes. - Partial deletes are safe: the batch's cursor rows stay
error, and--resumere-sends only entries that are notdone. - If
verifyreportsstale cid for <uri>, the referenced record changed after its referrer was written. Re-runplanto pick up current CIDs, or re-runapply --resumeif the entry is still pending. - If
verifyreportslegacy project ... still has a backlink, an unmanaged record (not part of the plan) references a deleted legacy project. Inspect it with the locallinkstable orlistRecords; add the missing alias or repoint it, then regenerate the plan. - After a successful
apply, runverify. Only when it exits0restart the service.
Local e2e harness
scripts/identity-migrate-e2e.sh
The script builds vuln-notif and identity-migrate, starts a temporary
hono-plc and atproto-pds through the existing scripts, creates an account,
initializes the cache schema, seeds tests/fixtures/identity-migration/
(cache_seed.sql plus legacy_records.json), and then asserts:
- the plan maps all three RHEL legacy projects and software records to
red-hat-enterprise-linux, creates that project, repoints the purl and CVE, normalizes the legacy purl slug, and plans 7 legacy deletions; applyreports the expected counters and writes a 12-record snapshot;- the canonical project exists, the CVE has one canonical ref plus its
preserved
nvdstrongRef, software/purl refs carry current CIDs, and every legacy release record returnsRecordNotFound; - a second
applyreportsapplied == 0,deleted == 0, and skips every entry; verifyreports no errors and no stale refs.
Requirements: cargo, curl, jq, sqlite3, deno, a hono-plc checkout
(HONO_PLC_DIR), and the pinned atproto-crates checkout
(ATPROTO_CRATES_DIR). Network access is not used; only localhost ports
4800/2587 (override with PDS_PORT/PLC_PORT). BUILD_BINS=0 skips the
cargo build, and KEEP_TMP=1 keeps the temp directory on failure.