~ knotless Search/
↑↓ move↵ open⇧↵ all results

johnandersen777.bsky.social/vuln-notif

0
0
main
330 lines · 14.7 KB Raw History

Identity migration runbook

The Rust service stores one canonical software.release.project per product. Legacy caches and repositories instead hold one project per raw {part}:{vendor}:{product} observation, so Red Hat Enterprise Linux is split across red-hat-red-hat-enterprise-linux-8, redhat-enterprise-linux, redhat-rhel-els, and other variants.

identity-migrate is the one-time, out-of-band admin tool that rewrites those references and deletes the legacy release records. There is no runtime migration phase, no startup ALTER TABLE, and no implicit write from vuln-notif: the service never runs this migration itself. A human runs plan, reviews the map, then runs apply and verify.

Locations:

  • Binary: target/release/identity-migrate (src/bin/identity-migrate.rs).
  • Plan implementation: src/migrate/.
  • Curated aliases: data/identity/aliases.json.
  • Local e2e: scripts/identity-migrate-e2e.sh.
  • Fixtures: tests/fixtures/identity-migration/.

Identity rules

identity = (class, canonical_vendor, canonical_family)
slug     = truncate64(join(canonical_vendor, canonical_family, '-'))
  • class is os, app, hardware, or firmware. Curated OS families (RHEL, Fedora, CentOS, Debian, Ubuntu, SUSE, Oracle Linux, Amazon Linux, Windows, macOS) are always os. Otherwise CPE part maps o → os, a → app, h → hardware. The raw part is not part of the identity, so the a/o RHEL duplicates collapse.
  • canonical_vendor is CPE-unescaped, lowercased, non-alphanumerics collapsed to -, then looked up in the curated vendor aliases (redhat, red_hat, Red Hat → red-hat).
  • canonical_family strips a redundant vendor prefix, then applies the curated product aliases. enterprise_linux*, rhel_els, enterprise_linux_for_*, and codeready_linux_builder* all become red-hat-enterprise-linux.
  • Trailing stream/version text is split only for families listed in versioned_families. X and single-digit X.0 are NVD major sentinels: they produce a channel and no version. X.Y produces channel X and version X.Y (9.4, 8.10, multi-digit 10.0 preserved). Edition/arch tokens (eus, els, aus, sap, rt, aarch64, ...) become channel suffixes; default x86_64 is omitted.
  • Channels never contain dots (Ubuntu 20.04 → channel 20-04); versions keep dots. Version and manifest records are immutable; only channels move.
  • Placeholder vendors/products (n/a, any, unknown, *, -) produce no identity. The old n-a-n-a project is never created again.
  • Slug collisions between different identities are resolved deterministically: a curated family owns the base slug, otherwise the lexicographically smaller identity key owns it and the other gets -<sha256(identity_key)[:6]>.
  • Every write is same-DID with the repository. Cross-DID strongRefs are rejected by the tool and by verify.

RHEL example:

ObservationLegacy raw slugCanonical projectChannel
NVD Red Hat + Red Hat Enterprise Linux 8red-hat-red-hat-enterprise-linux-8red-hat-enterprise-linux8
CPE cpe:2.3:o:redhat:enterprise_linux:9.4redhat-enterprise-linuxred-hat-enterprise-linux9 (version 9.4)
NVD redhat + rhel_elsredhat-rhel-elsred-hat-enterprise-linuxels

Legacy com.nahdig.software records are only encountered when migrating old snapshots; the current service does not publish them. Raw vendor, product, and cpes live in the local cache and in CVE software[] entries.

Alias table

data/identity/aliases.json is embedded with include_str!, validated at startup and by tests. Editing this file and rebuilding is the only supported way to merge identities; there is no fuzzy matching, stemming, or batch-order dependence.

{
  "vendors": {"redhat": "red-hat", "red-hat": "red-hat"},
  "products": {"enterprise-linux": "red-hat-enterprise-linux",
               "rhel-els": "red-hat-enterprise-linux"},
  "families": {"red-hat-enterprise-linux": "red-hat"},
  "versioned_families": ["red-hat-enterprise-linux"],
  "class_overrides": {"red-hat-enterprise-linux": "os"}
}
  • vendors: normalized vendor → canonical vendor.
  • products: normalized product alias → canonical family.
  • families: canonical family → canonical vendor (used for slug ownership).
  • versioned_families: families that may split trailing version/edition text.
  • class_overrides: family → identity class.

Every legacy → canonical pair appears in the plan reason and in the plan entries before apply, so the diff is reviewable per record.

Migration phases

snapshot → plan → review diff → apply → verify

  1. Snapshot. Write every record in the migration collections to JSONL. Read-only on the cache and PDS.
  2. Plan. Inventory the local cache (cves, software, software_purls, and purl2cpe when available) plus the remote listRecords inventory, then write a reviewable JSON map. plan performs no writes.
  3. Review diff. Inspect every delete_release_record, repoint_software, repoint_purl, repoint_cve, create_project, and normalize_purl_slug entry. Do not continue if a pair is wrong; project rkeys are permanent.
  4. Apply. Execute the map in dependency order with a persisted cursor. Already-done entries are skipped, so re-running is safe.
  5. Verify. Re-read the repository, assert canonical projects exist, legacy projects have no unmanaged backlinks, CVE refs are deduped and canonical, and strongRef CIDs are current. verify exits 1 if any error is reported.

Local

scripts/local_plc.sh start
scripts/local_atproto_pds.sh start
eval "$(scripts/create_local_account.sh 2>/dev/null)"   # exports ATP_PDS_URL/HANDLE/APP_PASSWORD

target/release/identity-migrate snapshot \
  --cache-db .state/local.db --snapshot-dir migration/snapshot-local

target/release/identity-migrate plan \
  --cache-db .state/local.db --purldb .state/purl2cpe.db \
  --out migration/plan-local.json

target/release/identity-migrate apply \
  --cache-db .state/local.db --map migration/plan-local.json \
  --batch 200

target/release/identity-migrate verify \
  --cache-db .state/local.db --map migration/plan-local.json

scripts/identity-migrate-e2e.sh runs this whole flow against a temporary PLC and PDS, seeds the legacy RHEL split, and asserts the plan, apply report, zero-write second apply, and verify report.

Production

Stop the service first (freeze writers) and keep it stopped until verify passes. Never run apply concurrently with a service cycle.

export ATP_HANDLE=did:plc:...          # repository DID/handle, not a secret
export ATP_APP_PASSWORD=...            # privileged app password
unset ATP_PDS_URL                      # resolve the PDS from PLC, or set it explicitly

DB=.state/prod-week.db
MAP=migration/identity-$(date +%Y%m%dT%H%M%S)

cp "$DB" "$DB.pre-migration"           # cache/rate-budget/cursor backup

target/release/identity-migrate snapshot \
  --cache-db "$DB" --snapshot-dir "$MAP.snapshot"

target/release/identity-migrate plan \
  --cache-db "$DB" --purldb .state/purl2cpe.db \
  --out "$MAP.plan.json"

# Review before applying.
jq -r '.entries[] | .action' "$MAP.plan.json" | sort | uniq -c
jq -r '.entries[] | select(.action == "create_project") | .canonical_slug' "$MAP.plan.json"
jq -r '.entries[] | select(.action == "delete_release_record") |
       [.legacy_slug, .canonical_slug, .legacy_uri] | @tsv' "$MAP.plan.json" | sort -u
jq -r '.entries[] | select(.action == "repoint_cve") | .legacy_slug' "$MAP.plan.json" | wc -l

target/release/identity-migrate apply \
  --cache-db "$DB" --map "$MAP.plan.json" \
  --batch 200 --snapshot-dir "$MAP.snapshot"

target/release/identity-migrate verify \
  --cache-db "$DB" --map "$MAP.plan.json"

apply prints one identity_apply event with a report: applied, skipped_done, created_projects, software_repoints, purl_repoints, cve_repoints, deleted, and normalized. verify prints identity_verify with canonical_projects, software_repoints, purl_repoints, cve_repoints, deletions, stale_refs, and errors.

If the cache holds no repository DID (or several), pass --handle did:plc:... explicitly to normalize-purl-slugs.

Dependency order

apply sorts and executes in this order; never delete a release record before its referrers move, because strongRef CIDs cascade.

OrderActionEffect
1create_projectsoftware.release.project/<canonical> if missing
2repoint_softwarecom.nahdig.software.project strongRef
3repoint_purlcom.nahdig.purl.software[].cid
4repoint_cvecom.nahdig.cve.atprotoV1.software[].project; dedupe to one canonical ref; drop refs to records being deleted
5normalize_purl_slugcache-only: software_purls.software_slug AT-URI → slug
6delete_release_recordapplyWrites batches of legacy project/channel/version/manifest/status records

Local link rows are not rebuilt from listRecords. The service indexes links as it writes, repairs failed indexes from link_repairs, and re-indexes migrated URIs on its next pass; verify checks the local links table only for unmanaged backlinks to legacy projects.

Rate budget

PDS write costs:

OperationPoints
create / put (new record)3
update (putRecord on existing record)2
delete1
applyWriteseach record in the batch at its own operation cost (deletes cost 1)

The migration's referrer moves are putRecord updates (2 points each); legacy record deletions are applyWrites deletes (1 point per record). Budget: 5,000 points/hour and 35,000 points/day per repository DID, with a 0.9 safety margin. The governor reserves points before each write and persists state in the cache DB's rate_budget table.

WorkRecordsApprox. points
CVE refs~14.3k updates~28.6k
software/purl refs + legacy deletes~100<1k

The CVE pass is close to one daily budget, so plan it as a day-long run. Use --limit to cap entries per invocation and --batch (default 200) for deletes. Stopping and resuming does not lose budget: it is persisted with the cursor.

Rollback via snapshot

snapshot writes one <collection>.jsonl file per migration collection; each line is {"uri", "cid", "value"} for a live record. Snapshots are full collection supersets, so they are also a cache-independent backup of the release/software/purl/CVE records.

A snapshot taken before apply, plus the cache DB copy, is the rollback material. Restoring is a manual putRecord replay (the tool intentionally has no destructive rollback command):

SNAP=migration/identity-YYYYmmddTHHMMSS.snapshot
DID=$ATP_HANDLE
ACCESS_JWT=$(curl -fsS -X POST "$ATP_PDS_URL/xrpc/com.atproto.server.createSession" \
  -H 'Content-Type: application/json' \
  -d "$(jq -nc --arg identifier "$ATP_HANDLE" --arg password "$ATP_APP_PASSWORD" \
        '{identifier:$identifier,password:$password}')" | jq -r .accessJwt)

for file in "$SNAP"/*.jsonl; do
  collection=$(basename "$file" .jsonl)
  while IFS= read -r line; do
    rkey=$(jq -r '.uri | split("/") | last' <<<"$line")
    value=$(jq -c '.value' <<<"$line")
    jq -nc --arg repo "$DID" --arg collection "$collection" \
      --arg rkey "$rkey" --argjson record "$value" \
      '{repo:$repo,collection:$collection,rkey:$rkey,record:$record}' |
      curl -fsS -X POST \
        -H "Authorization: Bearer $ACCESS_JWT" \
        -H 'Content-Type: application/json' \
        -d @- "$ATP_PDS_URL/xrpc/com.atproto.repo.putRecord"
  done <"$file"
done

Restore all collections, then re-run verify with the original plan: it should now report the pre-migration errors (legacy refs), which confirms the restore is complete. Restoring the cache DB copy rolls back the cursor and rate-budget rows as well. Do not restore only the project records: software, purl, and CVE strongRefs would still point at deleted records.

Failure recovery and resume

apply persists one cursor row per (repo_did, phase, raw_slug) in identity_migration with status pending, done, or error and an attempts counter. On a failed write it marks the entry error, commits, and exits nonzero. Completed entries are never repeated.

# Retry pending/error entries; done entries are skipped.
target/release/identity-migrate apply \
  --cache-db "$DB" --map "$MAP.plan.json" --resume

# Inspect progress.
sqlite3 "$DB" "SELECT action, status, count(*) FROM identity_migration GROUP BY 1,2;"
sqlite3 "$DB" "SELECT phase, raw_slug, attempts, error FROM identity_migration WHERE status='error';"
  • Without --resume, an error/pending cursor row aborts with "pass --resume to retry". This prevents accidental repeated writes.
  • Partial deletes are safe: the batch's cursor rows stay error, and --resume re-sends only entries that are not done.
  • If verify reports stale cid for <uri>, the referenced record changed after its referrer was written. Re-run plan to pick up current CIDs, or re-run apply --resume if the entry is still pending.
  • If verify reports legacy project ... still has a backlink, an unmanaged record (not part of the plan) references a deleted legacy project. Inspect it with the local links table or listRecords; add the missing alias or repoint it, then regenerate the plan.
  • After a successful apply, run verify. Only when it exits 0 restart the service.

Local e2e harness

scripts/identity-migrate-e2e.sh

The script builds vuln-notif and identity-migrate, starts a temporary hono-plc and atproto-pds through the existing scripts, creates an account, initializes the cache schema, seeds tests/fixtures/identity-migration/ (cache_seed.sql plus legacy_records.json), and then asserts:

  • the plan maps all three RHEL legacy projects and software records to red-hat-enterprise-linux, creates that project, repoints the purl and CVE, normalizes the legacy purl slug, and plans 7 legacy deletions;
  • apply reports the expected counters and writes a 12-record snapshot;
  • the canonical project exists, the CVE has one canonical ref plus its preserved nvd strongRef, software/purl refs carry current CIDs, and every legacy release record returns RecordNotFound;
  • a second apply reports applied == 0, deleted == 0, and skips every entry;
  • verify reports no errors and no stale refs.

Requirements: cargo, curl, jq, sqlite3, deno, a hono-plc checkout (HONO_PLC_DIR), and the pinned atproto-crates checkout (ATPROTO_CRATES_DIR). Network access is not used; only localhost ports 4800/2587 (override with PDS_PORT/PLC_PORT). BUILD_BINS=0 skips the cargo build, and KEEP_TMP=1 keeps the temp directory on failure.