# vuln-notif NVD CVEs → atproto. A Rust service that mirrors NVD v2 records and publishes custom atproto v1 records linking to canonical `software.release.*` identity. One product is one project; channels and versions hang off it. - [Record formats](#record-formats) - [Software identity](#software-identity) - [Run and release modes](#run) - [Read records](#read-records) - [Local end-to-end (Rust PDS + hono-plc)](#local-end-to-end-rust-pds--hono-plc) - [Lexicons](#lexicons) and [Tests](#tests) - [Source matrix](docs/release-sources.md) and [architecture](docs/release-architecture.md) for operators. ## Record formats ### `com.nahdig.cve.nvdV2` — raw NVD mirror Full NVD API v2 CVE object, normalized for atproto. rkey = CVE ID. ```json { "$type": "com.nahdig.cve.nvdV2", "id": "CVE-2026-48859", "published": "2026-06-10T...Z", "lastModified": "2026-06-10T...Z", "vulnStatus": "Analyzed", "descriptions": [{"lang": "en", "value": "..."}], "metrics": {"cvssMetricV31": [{"cvssData": {"baseScore": "5.3"}}]}, "configurations": ["..."] } ``` Floats become strings; naive dates get `Z`. Oversized CVEs use `dataBlob` (gzip full JSON) + `dataSha256`/`dataBytes`/`dataEncoding`. ### `com.nahdig.cve.atprotoV1` — link/index format ```json { "$type": "com.nahdig.cve.atprotoV1", "createdAt": "2026-11-11T21:26:02.724Z", "cve": "CVE-2026-48859", "nvd": { "uri": "at://did:plc:example/com.nahdig.cve.nvdV2/CVE-2026-48859", "cid": "bafyreib6jj..." }, "software": [ { "project": { "uri": "at://did:plc:example/software.release.project/red-hat-enterprise-linux", "cid": "bafyreihi3bq..." }, "part": "o", "vendor": "Red Hat", "product": "Red Hat Enterprise Linux 9", "cpes": ["cpe:2.3:o:redhat:enterprise_linux:9.4:*:*:*:*:*:*:*"], "purls": [], "channels": [ { "uri": "at://did:plc:example/software.release.channel/red-hat-enterprise-linux:9", "cid": "bafyreibcha..." } ], "versions": [ { "uri": "at://did:plc:example/software.release.version/red-hat-enterprise-linux:9.4", "cid": "bafyreibver..." } ], "manifests": [ { "uri": "at://did:plc:example/software.release.manifest/3labcdefghijk", "cid": "bafyreibman..." } ], "versionLabels": ["9.4.z"], "affectedRanges": ["< 9.4.1"] } ] } ``` - `nvd` = strongRef to `com.nahdig.cve.nvdV2`. - `software[].project` = strongRef to the canonical `software.release.project`. - `channels`, `versions`, and `manifests` are optional arrays of strongRefs emitted only for records that already exist. A version without a record becomes a `versionLabels` entry; range-only observations keep `affectedRanges`. - Constellation backlink paths: `software[].project.uri`, `software[].channels[].uri`, `software[].versions[].uri`, and `software[].manifests[].uri` under `com.nahdig.cve.atprotoV1`. ### Canonical `software.release.*` records One project per canonical identity. Project rkeys are permanent; versions and manifests are immutable; channels are mutable pointers. All refs are same-DID. `software.release.project/red-hat-enterprise-linux` (rkey = `name`): ```json {"$type": "software.release.project", "name": "red-hat-enterprise-linux"} ``` `software.release.channel/red-hat-enterprise-linux:9` (mutable pointer): ```json { "$type": "software.release.channel", "project": "at://did:plc:example/software.release.project/red-hat-enterprise-linux", "name": "9", "version": {"uri": "at://did:plc:example/software.release.version/red-hat-enterprise-linux:9.4", "cid": "bafy..."} } ``` `software.release.version/red-hat-enterprise-linux:9.4` (immutable): ```json { "$type": "software.release.version", "project": "at://did:plc:example/software.release.project/red-hat-enterprise-linux", "version": "9.4", "manifest": {"uri": "at://did:plc:example/software.release.manifest/3labcdefghijk", "cid": "bafy..."} } ``` `software.release.manifest/3labcdefghijk` (TID rkey; immutable): ```json { "$type": "software.release.manifest", "project": "at://did:plc:example/software.release.project/red-hat-enterprise-linux", "artifacts": [ { "$type": "software.release.defs#httpArtifact", "fileName": "ubi-9.4.tar", "url": "https://example.invalid/ubi-9.4.tar", "cid": "bafkreiexample...", "size": 123456789, "mimeType": "application/x-tar" } ] } ``` `software.release.status/3labcdefghijl` (append-only): ```json { "$type": "software.release.status", "subject": "at://did:plc:example/software.release.version/red-hat-enterprise-linux:9.4", "status": "deprecated", "kind": "superseded", "reason": "superseded by 9.5" } ``` Record-key constraints: project `1`–`64` lowercase alphanumerics and interior hyphens, no dots or colons; channel and version rkeys have exactly one colon and the suffix must equal `name`/`version` (channels cannot contain dots, versions can); manifest and status use TID-shaped rkeys. `status` is append-only: current state is the latest live same-DID status for the subject. `com.nahdig.release.projectMeta/` (rkey is the project slug): ```json { "$type": "com.nahdig.release.projectMeta", "project": "at://did:plc:example/software.release.project/red-hat-enterprise-linux", "name": "red-hat-enterprise-linux", "description": "Red Hat Enterprise Linux", "homepage": "https://example.invalid/rhel", "license": "GPL-2.0-or-later", "repository": "https://example.invalid/rhel.git", "versioningScheme": "rpm" } ``` `versioningScheme` comes from the source when it declares one, otherwise from the purl ecosystem identity (`semver`, `pep440`, `maven`, `nuget`, `go`, `rubygems`, `debian`, `rpm`, `lexical`). The record is mutable but keyed by project and only `putRecord` when the content changes. `com.nahdig.release.gitMeta` is an embedded metadata fragment (not a top-level record). It appears in an artifact's `metadata` array: ```json { "$type": "com.nahdig.release.gitMeta", "repository": "https://example.invalid/rhel.git", "revision": "refs/tags/v9.4", "commit": "0123456789abcdef0123456789abcdef01234567", "tag": "v9.4" } ``` Manifest artifacts may carry two optional fragments: a `metadata` array entry with git facts (`repository`, `revision`, `commit`, `tag`) and a `target` object (`os`, `architectures`). `release_version_meta` (local index) stores the same version's `publishedAt`, `rawVersion`, and artifact cid/size. For example, an RHEL UBI artifact carries `"target": {"os": "linux"}` and a gem built for `x86_64-linux` carries both `os` and the architecture. `software.release.defs` is the shared definitions schema (`target`, `blobArtifact`, `gitRepository`, `gitRevision`, `gitCommit`); the manifest artifact union is open, and the service currently writes `software.release.defs#httpArtifact`. ### Raw aliases and PURLs Raw NVD `part:vendor:product` aliases (CPEs, PURLs, display spelling) are kept in the local SQLite cache and inside each `com.nahdig.cve.atprotoV1` `software[]` entry. They are **not** published as standalone `com.nahdig.software` records. PURLs stay as strings in CVE `software[].purls[]` and in the local SQLite mapping tables. No `com.nahdig.purl` or `com.nahdig.software` records are published. See [`docs/identity-model.md`](docs/identity-model.md) for the full identity rules and [`docs/identity-migration.md`](docs/identity-migration.md) for the one-time rewrite. ## Software identity Identity is deterministic: ```text identity = (class, canonical_vendor, canonical_family) slug = truncate64(join(canonical_vendor, canonical_family, "-")) ``` Curated aliases live in `data/identity/aliases.json`; editing that table is the only supported merge mechanism. There is no fuzzy matching. | Rule | Behavior | |---|---| | Class | `os`, `app`, `hardware`, or `firmware`; curated OS families override the CPE part, so `a`/`o` duplicates collapse | | Vendor | CPE-unescaped, lowercased, non-alphanumerics → `-`, then vendor aliases (`Red Hat`/`redhat` → `red-hat`) | | Vendor prefix | Redundant prefix stripped (`Red Hat` + `Red Hat Enterprise Linux 9` → `enterprise-linux`) | | Product aliases | `enterprise_linux*`, `rhel_els`, `enterprise_linux_for_*`, `codeready_linux_builder*` → `red-hat-enterprise-linux` | | Versioned families | Only curated families split trailing stream/version text; unlisted apps and hardware never split | | Channel/version | `X`/single-digit `X.0` → channel only; `X.Y` → channel `X` + version `X.Y`; multi-digit `10.0` stays a version | | Editions/arch | `eus`, `aus`, `tus`, `sap`, `els`, `rt`, `nfv`, `av`, `crb`, `desktop`, `workstation`, `hpc` become channel suffixes; `aarch64`/`ppc64le`/`ppc64`/`s390x` append; `x86_64` is omitted | | Collisions | Curated family owns the base slug; otherwise the lexicographically smaller identity key owns it and the other gets `-` | | Length | Slug truncated at a hyphen boundary to 64 characters; project `name` must equal the rkey | | Placeholders | `n/a`, `any`, `unknown`, `*`, and `-` produce no identity; `n-a-n-a` is never created | RHEL worked example: every legacy spelling becomes one project, `red-hat-enterprise-linux`: | Legacy observation | Legacy raw slug | Channel | Version | |---|---|---|---| | CPE `enterprise_linux:6.0` | `redhat-enterprise-linux` | `6` | none | | CPE `enterprise_linux:7.0` | `redhat-enterprise-linux` | `7` | none | | NVD `Red Hat` + `Red Hat Enterprise Linux 8` | `red-hat-red-hat-enterprise-linux-8` | `8` | none | | CPE `enterprise_linux:9.4` | `redhat-enterprise-linux` | `9` | `9.4` | | CPE `enterprise_linux:10.0` | `redhat-enterprise-linux` | `10` | `10.0` | | CPE `enterprise_linux_eus:8` | `redhat-enterprise-linux-eus` | `8-eus` | none | | CPE `rhel_els:6` | `redhat-rhel-els` | `6-els` | none | | CPE `enterprise_linux:8.10` | `redhat-enterprise-linux` | `8` | `8.10` | Channels `6`, `7`, `8`, `9`, `10`, `8-eus`, and `6-els` all hang off the one project; `9.4`, `8.10`, and `10.0` are versions under their channels. PURLs do not create projects: purl2cpe bridges a PURL to a CPE product, and `rpm`, `deb`, `apk`, and `docker` are packaging namespaces, not vendors. ## Build Rust 1.97 (pinned by `rust-toolchain.toml`), edition 2024. The release binary is `target/release/vuln-notif`. ```bash cargo build --release --locked ./target/release/vuln-notif --help ``` Docker image (multi-stage; runs as a non-root user; includes `ca-certificates`): ```bash docker build -t vuln-notif . docker run --rm -v "$PWD/.state:/data/.state" vuln-notif --once --dry-run ``` ## Run ```bash export ATP_PDS_URL=https://your-pds.example export ATP_HANDLE=you.example.com export ATP_APP_PASSWORD=... export NVD_API_KEY=... # optional, strongly recommended export S3_BUCKET=your-bucket # optional export S3_ENDPOINT_URL= # blank for AWS S3 export S3_REGION=us-east-1 export S3_ACCESS_KEY_ID=... export S3_SECRET_ACCESS_KEY=... vuln-notif --once --dry-run # fetch + cache, publish nothing vuln-notif --once # one cycle, then exit vuln-notif # poll every --poll-interval seconds ``` `ATP_PDS_URL` may be omitted; it resolves from the handle's DID document. `--dry-run` does not require handle/app-password. Exit codes: `0` success, `1` fatal, `2` CLI error, `130` interrupted. `SIGINT`/`SIGTERM` drain the in-flight phase, commit, and exit cleanly. ### Flags and environment Boolean environment variables accept `1`, `true`, `yes`, or `on`. The table matches `src/cli.rs`; `vuln-notif` has no subcommands. | Flag | Env | Default | Notes | |---|---|---|---| | `--once` | | off | run one cycle, then exit | | `--dry-run` | | off | fetch/cache only; publish nothing | | `--limit` | | `0` (all) | max cache rows published per cycle; cursor still advances | | `--backfill` | | off | start from 1999 regardless of the stored cursor; the cursor is replaced as pages arrive | | `--initial-lookback-days` | | `7` | cursor initialization window | | `--order` | `ORDER` | `cache` | pending publish order: `cache`, `newest`, `oldest` | | `--publish-atproto` | | off | create software/purl/atprotoV1 records | | `--software-limit` | | `0` (all) | max CVEs enriched per run | | `--publish-releases` | | off | run the standalone `latest` mode switch: publish latest `software.release.*` records for canonical projects | | `--release-mode` | `RELEASE_MODE` | `latest` | `latest`, `affected`, `history`, `extras`, or `all`; `latest` runs only with `--publish-releases`, while `all` always runs latest | | `--release-limit` | | `0` (all) | max software projects inspected per cycle | | `--release-recheck-seconds` | `RELEASE_RECHECK_SECONDS` | `86400` | skip projects checked more recently; `0` forces a check | | `--release-ecosystems` | | empty (all) | release purl type filter (comma-separated) | | `--release-sources` | | empty (all) | release source filter (comma-separated); matches purl types, adapter names such as `github` or `docker`, and `osv` for affected work | | `--release-history-limit` | | `0` (all) | max versions processed per project per history cycle | | `--release-concurrency` | | `4` | projects enumerated concurrently in history mode; affected runs in order | | `--release-source-url` | `RELEASE_SOURCE_URL` | unset | base-URL override for history/extras adapters with configurable endpoints: `composer`, `cpan`, `apache`, `gitee`, `gnu` (history and extras) and `sourceforge` (extras); local e2e only | | `--artifact-max-bytes` | | `0` (no cap) | reject artifacts larger than this while streaming | | `--artifact-s3-bucket` | `ARTIFACT_S3_BUCKET` | unset | mirror verified artifact bytes to this S3 bucket | | `--artifact-s3-path` | | `artifacts` | S3 key prefix for mirrored artifacts | | `--github-token` | `GITHUB_TOKEN` | unset | GitHub release-source token; never logged | | `--gitlab-token` | `GITLAB_TOKEN` | unset | GitLab release-source token; never logged | | `--gitee-token` | `GITEE_TOKEN` | unset | Gitee release-source token; never logged | | `--osv-online` | | on | query the OSV API (accepts `--osv-online=false`) | | `--osv-offline` | | off | never query OSV; requires `--osv-bulk-dir` | | `--osv-bulk-dir` | | unset | load OSV advisory JSON files from this directory; implies offline | | `--purldb` | | `.state/purl2cpe.db` | purl2cpe SQLite path; downloaded if missing | | `--cache-db` | | `.state/vuln-notif.db` | SQLite cache path | | `--poll-interval` | `POLL_INTERVAL` | `300` | seconds between cycles | | `--page-size` | | `2000` | NVD results per page | | `--concurrency` | `CONCURRENCY` | `8` | concurrent PDS checks; clamped to ≥1 | | `--pds-timeout` | `PDS_TIMEOUT` | `60` | PDS HTTP timeout in seconds | | `--nvd-api-key` | `NVD_API_KEY` | unset | NVD API key | | `--require-nvd-key` | `REQUIRE_NVD_KEY` | off | refuse `--backfill` without a key | | `--nvd-url` | `NVD_API_URL` | NVD 2.0 endpoint | NVD base URL; test seam | | `--plc-url` | `PLC_URL` | `plc.directory` | PLC directory base for handle resolution; test seam | | `--pds-hourly-points` | `PDS_WRITE_HOURLY` | `5000` | PDS write-point budget per hour | | `--pds-daily-points` | `PDS_WRITE_DAILY` | `35000` | PDS write-point budget per day | | `--pds-requests-per-5min` | `PDS_REQUESTS_PER_5MIN` | `3000` | PDS request cap per 5 minutes | | `--pds-url` | `ATP_PDS_URL` | resolved from the handle | PDS base URL | | `--handle` | `ATP_HANDLE` | required unless dry-run | account handle; a leading `@` is stripped | | `--app-password` | `ATP_APP_PASSWORD` | required unless dry-run | app password | | `--collection` | `ATP_COLLECTION` | `com.nahdig.cve.nvdV2` | NVD record collection | | `--s3-bucket` | `S3_BUCKET` | unset | enables S3 cache snapshots | | `--s3-path` | `S3_PATH` | `vuln-notif/cve-cache.db` | S3 object key | | `--s3-endpoint-url` | `S3_ENDPOINT_URL` | AWS S3 | custom S3-compatible endpoint | | `--s3-access-key-id` | `S3_ACCESS_KEY_ID`, `AWS_ACCESS_KEY_ID` | default chain | static credentials | | `--s3-secret-access-key` | `S3_SECRET_ACCESS_KEY`, `AWS_SECRET_ACCESS_KEY` | default chain | static credentials | | `--s3-region` | `S3_REGION` | `us-east-1` | S3 region | | `--debug` | `DEBUG` | off | debug-level NDJSON logging | ## Recent + backfill ```bash # last 7 days, newest first, poll every 15 min vuln-notif \ --initial-lookback-days 7 --poll-interval 900 --order newest \ --publish-atproto --software-limit 100 \ --cache-db .state/recent.db --s3-path vuln-notif/recent.db # one full-history pass, newest first vuln-notif \ --once --backfill --order newest \ --publish-atproto --software-limit 100 \ --cache-db .state/backfill.db --s3-path vuln-notif/backfill.db ``` `--publish-atproto` creates software projects, PURL lookups, and `com.nahdig.cve.atprotoV1`. `--software-limit N` caps CVEs enriched per run. Separate cache DBs + S3 paths. ### Release modes `--release-mode` selects release ingestion; `latest` is the default value but the standalone mode only runs when `--publish-releases` is also set. `all` runs latest, affected, history (when `--release-history-limit > 0`), and extras in that order, sharing one candidate rotation cursor and artifact resolver. | Mode | Command | What it publishes | |---|---|---| | `latest` | `--publish-releases --release-mode latest` | deps.dev default/latest version for the 7 deps.dev ecosystems: project, manifest, version, `latest` channel, optional status/projectMeta | | `affected` | `--release-mode affected` | OSV advisory matches mapped onto the project catalog; only affected versions with a fetchable artifact become records, otherwise `versionLabels` | | `history` | `--release-mode history` | full source catalog as immutable manifest + version records, bounded by `--release-history-limit` and `--release-concurrency` | | `extras` | `--release-mode extras` | one newest stable version for every registered extras adapter (pypi, cargo, gem, GitHub, GitLab, Bitbucket, distro, composer, cpan, apache, gitee, gnu, docker/oci, sourceforge) | | `all` | `--release-mode all` | one pass of every mode (history only with `--release-history-limit > 0`) | Standalone latest: ```bash vuln-notif --once \ --publish-atproto --publish-releases --release-mode latest --release-limit 50 ``` `--publish-releases` resolves each candidate through `software_identity`, so project records and rkeys use canonical identity (for example `red-hat-enterprise-linux`, not `red-hat-red-hat-enterprise-linux-8`). Raw aliases stay local and inside CVE records. For the selected default/latest version it publishes `software.release.project`, an immutable `manifest`, an immutable `version`, a mutable `channel` (`latest` for this mode), and optional `status` records. When the purl type has a source adapter it also publishes `com.nahdig.release.projectMeta` (only when the content changed) and attaches git/target facts to the manifest artifacts. Statuses are append-only: a deps.dev deprecation without a registry signal is routed through the same status module, so a reason change appends a new event instead of rewriting. `--release-limit` caps projects inspected per cycle; `--release-recheck-seconds` (default 86400) skips projects checked recently. Affected versions come from OSV advisories and are mapped onto the release version catalog: ```bash vuln-notif --once \ --release-mode affected --release-limit 50 --release-recheck-seconds 86400 \ --release-ecosystems pypi,npm \ --cache-db .state/affected.db ``` OSV is queried online by default. For a network-free or reproducible run, put the OSV advisory JSON files in a directory and use `--osv-bulk-dir` (it implies `--osv-offline`): ```bash vuln-notif --once \ --release-mode affected --release-limit 50 \ --osv-bulk-dir .state/osv-bulk \ --cache-db .state/affected.db ``` For each canonical project the run queries OSV by versionless purl, evaluates the advisory ranges against `release_versions` rows plus deps.dev versions, publishes an immutable `manifest`/`version` for affected versions that have a fetchable artifact, and points the major channel at the highest known version. New versions also record `release_version_meta` (`publishedAt`, `rawVersion`) and carry the source's git/target facts on their manifest artifacts. Statuses come only from registry signals; a CVE advisory never creates or yanks one. Versions without an artifact stay in `versionLabels`; nothing is fabricated. CVEs linked to the project through `software[].project.uri` are updated only when their refs change. Per-project failures are isolated and counted in the cycle stats. `--release-mode extras` and `all` are documented below. ### History mode `--release-mode history` walks every candidate project's source catalog and publishes each version as an immutable `manifest` + `version` record: ```bash vuln-notif --once \ --release-mode history --release-limit 25 \ --release-history-limit 100 --release-concurrency 4 \ --release-ecosystems pypi,cargo,gem,composer \ --artifact-max-bytes 104857600 \ --cache-db .state/history.db ``` - Versions are enqueued in `release_history_jobs` keyed by `(repo, project, raw version)`. Versions that already have a version row are skipped, so re-enumeration is idempotent. - Status signals are evaluated for versions materialized in the cycle and for deprecated catalog entries, so a registry reason change appends a new append-only status event on a later pass. - `--release-history-limit` bounds jobs processed per project per cycle; the next cycle resumes pending/retry jobs deterministically. Failed jobs retry up to three times and then remain `error` until inspected. - Artifacts are resolved through the shared resolver: declared digests are verified against downloaded bytes, missing digests are computed, and a verified artifact is cached in `release_artifacts` so it is never downloaded twice. `--artifact-max-bytes` caps the streaming download; `0` means no cap. - With `--artifact-s3-bucket`, verified bytes are also mirrored to S3 under `--artifact-s3-path` (mirror key `artifacts/`), and the mirror is checked before the registry on later resolves. Without a bucket the mirror is a no-op and only the local digest cache is used. Mirror failures fall back to the registry and do not fail the resolve. - History adapters: PyPI, crates.io, RubyGems, GitHub, GitLab, Bitbucket, distro (rpm/deb/apk), Packagist, MetaCPAN, Apache, Gitee, and GNU. npm, Maven, NuGet, Go, Docker/OCI, and SourceForge have full history adapters in this build; they are typed skips, recorded in `release_project_state.unsupported_reason`, and the cycle continues. Docker/OCI and SourceForge are available in `extras`; npm/Maven/NuGet/Go are available in `latest`. - Cycle counters: `release_history_enqueued`, `release_history_processed`, `release_history_created`, `release_history_skipped`, `release_history_failed`, `release_history_reused`, plus `artifact_downloads`, `artifact_bytes`, `artifact_cache_hits`, `artifact_mirror_puts`, and `artifact_errors`. ### Extras mode `--release-mode extras` covers the non-deps.dev ecosystems. For each candidate project it builds the registered adapter, takes the newest non-prerelease / non-draft version, resolves the source-declared artifact through the shared artifact resolver, and publishes `software.release.project`, an immutable `manifest` + `version`, the `latest` channel, the source statuses, and `com.nahdig.release.projectMeta`. Each new version records `release_version_meta` and carries the source's git/target facts in the manifest artifact metadata: ```bash vuln-notif --once \ --release-mode extras --release-limit 25 --release-recheck-seconds 86400 \ --release-ecosystems pypi,github,rpm,docker \ --release-sources github,docker \ --artifact-max-bytes 104857600 \ --cache-db .state/extras.db ``` - Adapters: PyPI, crates.io, RubyGems, GitHub, GitLab, Bitbucket, distro (rpm/deb/apk), Packagist, MetaCPAN, Apache, Gitee, GNU, Docker Hub, and SourceForge. npm, Maven, NuGet, and Go have full extras/history adapters and are typed `unsupported`; they are still covered by `latest` through deps.dev. An unknown purl type is recorded in `release_project_state.unsupported_reason` and never silently dropped. - `--release-ecosystems` filters purl types; `--release-sources` also matches adapter names (`docker` matches `docker`/`oci`, `distro` matches `rpm`/`deb`/`apk`). Empty filters mean all. - `--dry-run` fetches and caches NVD data only; release phases are skipped (standalone release modes require a live publisher), so no version, artifact, or metadata is fetched and no PDS record is written. - Tokens: `GITHUB_TOKEN`, `GITLAB_TOKEN`, `GITEE_TOKEN` (or the matching flags). Gitee sends `Authorization: Bearer `; tokens never appear in URLs or logs. - Cycle counters: `release_extras_projects_checked`, `release_extras_projects_created`, `release_extras_versions_created`, `release_extras_manifests_created`, `release_extras_statuses_created`, `release_extras_unchanged`, `release_extras_skipped`, `release_extras_unsupported`, and `release_extras_failures`. - Per-ecosystem source, auth, artifact, digest, status, and rate-limit matrix: [`docs/release-sources.md`](docs/release-sources.md), with the operational summary in [`MAINTAINERS.md#extras-mode`](MAINTAINERS.md#extras-mode). ### All mode `--release-mode all` runs one cycle of every release mode in order: latest, affected, history (only when `--release-history-limit > 0`), then extras. The modes share the candidate rotation cursor and the artifact resolver, and keep separate counters in `cycle_stats`: ```bash vuln-notif --once \ --release-mode all --release-limit 100 --release-recheck-seconds 86400 \ --release-history-limit 20 --release-concurrency 4 \ --release-ecosystems pypi,github,composer \ --artifact-max-bytes 104857600 \ --cache-db .state/all.db ``` Latest writes `release_*`, affected writes `release_affected_*`, history writes `release_history_*`, and extras writes `release_extras_*`; `artifact_*` counters aggregate every resolve in the cycle. ### Artifact cache and S3 mirror Every mode that materializes a version resolves one artifact through the same resolver: ```bash vuln-notif --once \ --release-mode history --release-limit 25 --release-history-limit 100 \ --artifact-max-bytes 104857600 \ --artifact-s3-bucket "$ARTIFACT_S3_BUCKET" --artifact-s3-path artifacts \ --cache-db .state/history.db ``` - Verified metadata lives in `release_artifacts`, keyed by `(repo, source, package, raw version, url)`. A second resolve is a `artifact_cache_hits` increment, not a download. - `--artifact-max-bytes` is a streaming cap; `0` means no cap. Size-cap and digest-mismatch failures are typed and never retried as artifact errors. - `--artifact-s3-bucket` enables the byte mirror. The resolver key is `artifacts/`; `--artifact-s3-path` (default `artifacts`) is the object-key prefix, so the default object key is `artifacts/artifacts/`. The mirror is read before the registry and written best-effort after a verified download. It uses the same S3 endpoint/credentials as the cache snapshot (`S3_ENDPOINT_URL`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY`, `S3_REGION`). - Without a bucket the mirror is a no-op: only the local digest cache is used. The resulting `com.nahdig.cve.atprotoV1` ref carries channel and version strongRefs (generic example, not production credentials): ```json { "$type": "com.nahdig.cve.atprotoV1", "cve": "CVE-2024-3094", "nvd": { "uri": "at://did:plc:example/com.nahdig.cve.nvdV2/CVE-2024-3094", "cid": "bafy..." }, "software": [{ "project": { "uri": "at://did:plc:example/software.release.project/red-hat-enterprise-linux", "cid": "bafy..." }, "part": "o", "vendor": "Red Hat", "product": "enterprise_linux", "purls": ["pkg:rpm/redhat/enterprise_linux"], "channels": [{ "uri": "at://did:plc:example/software.release.channel/red-hat-enterprise-linux:9", "cid": "bafy..." }], "versions": [{ "uri": "at://did:plc:example/software.release.version/red-hat-enterprise-linux:9.2", "cid": "bafy..." }] }] } ``` The one-time rewrite of existing raw-slug projects is the separate `identity-migrate` admin binary (`target/release/identity-migrate`), not a `vuln-notif` flag. See [`docs/identity-migration.md`](docs/identity-migration.md) and the migration runbook in `MAINTAINERS.md`. Extra-ecosystem ingestion is wired through `--release-mode extras` and `--release-mode all`. ## Read records ```bash PDS=$ATP_PDS_URL DID=$(goat account check-auth | sed -n 's/^DID: //p') ``` List a collection: ```bash COL=com.nahdig.cve.atprotoV1 cursor= while :; do url="$PDS/xrpc/com.atproto.repo.listRecords?repo=$DID&collection=$COL&limit=100${cursor:+&cursor=$cursor}" body=$(curl -sS "$url") jq -r '.records[].uri' <<<"$body" cursor=$(jq -r '.cursor // empty' <<<"$body") [ -z "$cursor" ] && break done ``` ```yaml # jq -r output, one URI per line - at://did:plc:example/com.nahdig.cve.atprotoV1/CVE-2026-48859 - at://did:plc:example/com.nahdig.cve.atprotoV1/CVE-2026-48858 ``` One CVE in both formats: ```bash curl -sS "$PDS/xrpc/com.atproto.repo.getRecord?repo=$DID&collection=com.nahdig.cve.nvdV2&rkey=CVE-2024-3094" | jq .value curl -sS "$PDS/xrpc/com.atproto.repo.getRecord?repo=$DID&collection=com.nahdig.cve.atprotoV1&rkey=CVE-2024-3094" | jq .value ``` ```yaml # nvdV2 value $type: com.nahdig.cve.nvdV2 id: CVE-2024-3094 published: 2024-03-29T17:15:21.150Z lastModified: 2024-03-29T17:15:21.150Z vulnStatus: Analyzed descriptions: - lang: en value: backdoor in xz/liblzma metrics: cvssMetricV31: - cvssData: baseScore: "10.0" vectorString: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H ``` ```yaml # atprotoV1 value $type: com.nahdig.cve.atprotoV1 createdAt: 2026-11-11T21:26:02.724Z cve: CVE-2026-48859 nvd: uri: at://did:plc:example/com.nahdig.cve.nvdV2/CVE-2026-48859 cid: bafyreib6jjbqprgu5fvy3pbk4inrefzjfnrkfwvelu77i7ljhd5h4jy5ee software: - project: uri: at://did:plc:example/software.release.project/red-hat-enterprise-linux cid: bafyreihi3bqgqdsom5l6gay53aenpqnvypccdwszhl2kdeilh5o6ffwuy4 part: o vendor: Red Hat product: Red Hat Enterprise Linux 9 cpes: - cpe:2.3:o:redhat:enterprise_linux:9.4:*:*:*:*:*:*:* channels: - uri: at://did:plc:example/software.release.channel/red-hat-enterprise-linux:9 cid: bafyreibcha... versions: - uri: at://did:plc:example/software.release.version/red-hat-enterprise-linux:9.4 cid: bafyreibver... manifests: - uri: at://did:plc:example/software.release.manifest/3labcdefghijk cid: bafyreibman... ``` Constellation backlinks for a canonical project: ```bash PROJ=red-hat-enterprise-linux PROJECT="at://$DID/software.release.project/$PROJ" CHANNEL="at://$DID/software.release.channel/$PROJ:9" VERSION="at://$DID/software.release.version/$PROJ:9.4" curl -sS -G 'https://constellation.microcosm.blue/xrpc/blue.microcosm.links.getBacklinks' \ --data-urlencode "subject=$PROJECT" \ --data-urlencode 'source=com.nahdig.cve.atprotoV1:software[].project.uri' \ --data-urlencode 'limit=100' | jq . ``` ```yaml total: 3 records: - did: did:plc:example collection: com.nahdig.cve.atprotoV1 rkey: CVE-2026-48859 - did: did:plc:example collection: com.nahdig.cve.atprotoV1 rkey: CVE-2026-42791 cursor: null ``` Channel and version backlinks use the same query with their own subject and source path: ```bash for PAIR in "channels:$CHANNEL" "versions:$VERSION"; do KIND=${PAIR%%:*} SUBJECT=${PAIR#*:} curl -sS -G 'https://constellation.microcosm.blue/xrpc/blue.microcosm.links.getBacklinks' \ --data-urlencode "subject=$SUBJECT" \ --data-urlencode "source=com.nahdig.cve.atprotoV1:software[].$KIND[].uri" \ --data-urlencode 'limit=100' | jq . done ``` ```yaml # channel subject .../software.release.channel/red-hat-enterprise-linux:9 total: 1 records: - did: did:plc:example collection: com.nahdig.cve.atprotoV1 rkey: CVE-2026-48859 cursor: null # version subject .../software.release.version/red-hat-enterprise-linux:9.4 total: 1 records: - did: did:plc:example collection: com.nahdig.cve.atprotoV1 rkey: CVE-2026-48859 cursor: null ``` Browser-style `/links`, JSON + jq: ```bash curl -sS -G 'https://constellation.microcosm.blue/links' \ --data-urlencode "target=$PROJECT" \ --data-urlencode 'collection=com.nahdig.cve.atprotoV1' \ --data-urlencode 'path=.software[].project.uri' \ | jq -r '"total=\(.total)", (.linking_records[].rkey)' ``` ```yaml total: 86 linking_records: - did: did:plc:example collection: com.nahdig.cve.atprotoV1 rkey: CVE-2026-23780 - did: did:plc:example collection: com.nahdig.cve.atprotoV1 rkey: CVE-2026-23781 ``` Count canonical project backlinks: ```bash for PROJ in red-hat-enterprise-linux trustedfirmware-mbedtls erlang-erlang-otp gleam-lang-gleam; do SUBJECT="at://$DID/software.release.project/$PROJ" curl -sS -G 'https://constellation.microcosm.blue/links' \ --data-urlencode "target=$SUBJECT" \ --data-urlencode 'collection=com.nahdig.cve.atprotoV1' \ --data-urlencode 'path=.software[].project.uri' \ | jq -r --arg p "$PROJ" '"\($p): \(.total) CVEs"' done ``` PURL lookup: ```bash PURL='pkg:deb/debian/openssl' HASH=$(printf %s "$PURL" | sha256sum | cut -d' ' -f1) Oversized NVD v2 blob: ```bash REC=$(curl -sS "$PDS/xrpc/com.atproto.repo.getRecord?repo=$DID&collection=com.nahdig.cve.nvdV2&rkey=CVE-2021-3942") CID=$(jq -r '.value.dataBlob.ref["$link"]' <<<"$REC") curl -sS "$PDS/xrpc/com.atproto.sync.getBlob?did=$DID&cid=$CID" | gunzip | jq . ``` ```yaml $type: com.nahdig.cve.nvdV2 id: CVE-2021-3942 dataBlob: $type: blob ref: $link: bafkreih2tdaajuwo3k2a2extvn4haz34o5dez3zbay3klo3o4fj4lhbysi mimeType: application/gzip size: 161053 dataSha256: 5dd2befa352bd20cd0f95fff1c970d3a8eaf890e02bfe9d56c17f1c6cbc2324e dataBytes: 1265981 dataEncoding: gzip ``` ```yaml # gunzip | jq . (blob contents) $type: com.nahdig.cve.nvdV2 id: CVE-2021-3942 configurations: - nodes: [] # full CPE tree in blob ``` Watch new records: ```bash websocat -n "wss://jetstream2.us-east.bsky.network/subscribe?wantedCollections=com.nahdig.cve.atprotoV1&wantedDids=$DID" goat firehose --collection com.nahdig.cve.atprotoV1 --ops ``` ## Local backlinks index Every successful write also indexes its outgoing links in the cache DB (`links` table). Query locally, no Constellation: ```bash sqlite3 .state/prod.db " SELECT src_collection, path, src_uri, subject_cid FROM links WHERE repo_did='did:plc:example' AND subject_uri IN ( 'at://did:plc:example/software.release.project/red-hat-enterprise-linux', 'at://did:plc:example/software.release.channel/red-hat-enterprise-linux:9', 'at://did:plc:example/software.release.version/red-hat-enterprise-linux:9.4') ORDER BY indexed_at DESC LIMIT 10;" ``` ```yaml - - com.nahdig.cve.atprotoV1 - software[].project.uri - at://did:plc:example/com.nahdig.cve.atprotoV1/CVE-2026-48859 - bafyreihi3bqgqdsom5l6gay53aenpqnvypccdwszhl2kdeilh5o6ffwuy4 - - com.nahdig.cve.atprotoV1 - software[].channels[].uri - at://did:plc:example/com.nahdig.cve.atprotoV1/CVE-2026-48859 - bafyreibcha... - - com.nahdig.cve.atprotoV1 - software[].versions[].uri - at://did:plc:example/com.nahdig.cve.atprotoV1/CVE-2026-48859 - bafyreibver... ``` Outgoing links from a record: ```bash sqlite3 .state/prod.db " SELECT path, subject_uri, subject_cid FROM links WHERE repo_did='did:plc:example' AND src_uri='at://did:plc:example/com.nahdig.cve.atprotoV1/CVE-2024-3094' ORDER BY path;" ``` The index is written as records are published and repaired from `link_repairs` after a failed write. A `listRecords`-based rebuild is deliberately deferred; the one-time identity migration repoints known URIs out of band and `verify` checks for unmanaged legacy backlinks. ## Local end-to-end (Rust PDS + hono-plc) | Service | Port | Start script | |---|---|---| | `hono-plc` (Deno did:plc directory, in-memory) | 2587 | `scripts/local_plc.sh` | | `atproto-pds` (Rust, from pinned atproto-crates rev) | 4800 | `scripts/local_atproto_pds.sh` | | RustFS (S3-compatible) | 9000 | `scripts/local_rustfs.sh` | Startup order: **PLC → PDS → account**. Start the services yourself in this order (below), or let `scripts/local-e2e.sh` start its own temporary PLC/PDS. The PDS needs `PDS_LEXICON_DIR` pointing at this repo's `lexicons/`, `PDS_DID_PLC_URL=http://127.0.0.1:2587`, `PDS_PORT=4800`, and `PDS_ALLOW_DEV_DEFAULTS=true`; the script sets these, builds `atproto-pds` (`cargo +stable build --release --bin pds --features clap`), and waits on `/xrpc/_health`. It also verifies the atproto-crates checkout is at the pinned revision. RustFS is only needed for the S3 cache test. ```bash scripts/local_plc.sh start # hono-plc :2587 scripts/local_atproto_pds.sh start # atproto-pds :4800, PDS_LEXICON_DIR=$PWD/lexicons scripts/create_local_account.sh # creates account + app password, prints ATP_* env export ATP_PDS_URL=http://127.0.0.1:4800 export ATP_HANDLE=cve.test export ATP_APP_PASSWORD=test-password vuln-notif --once --limit 5 ``` End-to-end scripts, in order: ```bash scripts/local-e2e.sh # PLC + PDS + account, write/read a custom record, optional --once run scripts/blob-e2e.sh # oversized CVE -> compact record + gzip blob round-trip scripts/local_rustfs.sh start # :9000 scripts/s3-cache-e2e.sh # upload snapshot, delete DB, restore, publish next batch scripts/identity-migrate-e2e.sh # seed legacy RHEL split, plan, apply, verify, idempotent re-run scripts/release-affected-e2e.sh # offline OSV bulk dir, affected version + CVE ref scripts/release-history-e2e.sh # fake composer source, history limit/resume, artifact counters scripts/release-extras-e2e.sh # fake composer source, extras publish + artifact scripts/check-lexicons.sh # jq-validate every lexicon under lexicons/ ``` The blob and contract tests run against a live local PDS: set `ATP_PDS_URL`, `ATP_HANDLE`, and `ATP_APP_PASSWORD` (the e2e scripts do this), or the tests skip with a clear message. ## Lexicons The repository vendors ten lexicon JSON files: four `com.nahdig.*` documents (two record types plus the embedded `gitMeta` and `projectMeta` schemas) and six `software.release.*` documents (the shared `defs` document plus the five record types): ```bash goat lex parse \ lexicons/com/nahdig/cve.nvdV2.json \ lexicons/com/nahdig/cve.atprotoV1.json \ lexicons/com/nahdig/release.projectMeta.json \ lexicons/com/nahdig/release.gitMeta.json \ lexicons/software.release.defs.json \ lexicons/software.release.project.json \ lexicons/software.release.channel.json \ lexicons/software.release.version.json \ lexicons/software.release.manifest.json \ lexicons/software.release.status.json ``` `scripts/check-lexicons.sh` does not keep its own list: it recursively walks `lexicons/` and `jq`-validates every `*.json` it finds. The same file list is duplicated in `MAINTAINERS.md` for the publish loop. Publish/verify: `MAINTAINERS.md`. ## Tests ```bash cargo fmt --check cargo clippy --all-targets -- -D warnings cargo test cargo test --test pds_contract scripts/local-e2e.sh scripts/blob-e2e.sh scripts/s3-cache-e2e.sh scripts/identity-migrate-e2e.sh scripts/release-affected-e2e.sh scripts/release-history-e2e.sh scripts/release-extras-e2e.sh scripts/check-lexicons.sh ``` Implementation and operator detail for the release lane: [`docs/release-architecture.md`](docs/release-architecture.md) and [`docs/release-sources.md`](docs/release-sources.md). ## License GPL-3.0-or-later.