# Release architecture How the Rust service ingests releases, publishes `software.release.*` records, and keeps local state consistent. Source capabilities are in [`release-sources.md`](release-sources.md); identity rules are in [`identity-model.md`](identity-model.md). ## Process and module map The binary is two reactors: - `src/runtime/orchestrator.rs` — signals, scheduling, event output, run totals. - `src/runtime/mod.rs` — `MirrorRuntime`: cache, NVD client, publisher, cursor, phase state machine. - `src/runtime/phases.rs` — the only bridge from the reactor to lane modules. - `src/runtime/events.rs` — typed `ProgressEvent` (lanes) and `RuntimeEvent` (output); progress is forwarded on a bounded channel. - `src/db/mod.rs` — every SQLite operation runs on one dedicated OS thread behind `DbHandle`; callers await command replies. Release modules: | Module | Contract | |---|---| | `src/release/mod.rs` | `ReleaseConfig`, `AffectedConfig`, `HistoryModeConfig`, `ExtrasConfig`, `ReleaseCursor`, and per-mode counters | | `src/release/source.rs` | `ReleaseSource` trait, `SourceCapabilities`, `UnsupportedReason`, `SOURCE_REGISTRY`, `build_source*` | | `src/release/sources/` | one adapter per source; OSV is a lookup client, not a `ReleaseSource` | | `src/release/service.rs` | mode orchestration (`sync_releases*`, `sync_affected*`, `sync_history*`, `sync_extras*`, `sync_all*`), record prep, project state | | `src/release/planner.rs` | deps.dev default selection (`select_release`) and OSV range matching (`plan_versions`) | | `src/release/versioning.rs` | version schemes, normalization, comparison | | `src/release/records.rs` | wire shapes and rkeys for `software.release.*` | | `src/release/artifacts.rs` | artifact resolver: cap, verify/compute digest, CID, cache, S3 mirror, counters | | `src/release/artifact.rs` | purl selection and per-ecosystem artifact fetchers (npm, PyPI, cargo, RubyGems, Maven, NuGet, Go) | | `src/release/http.rs` | `RegistryHttp` seam and production reqwest transport with retries/caps | | `src/release/history.rs` | resumable `release_history_jobs` queue | | `src/release/cve_links.rs` | atprotoV1 `channels`/`versions`/`manifests`/`versionLabels`/`affectedRanges` refs | | `src/release/status.rs` | append-only status transition planning | | `src/release/metadata.rs` | project metadata, git/target facts, versioning scheme | | `src/identity/` | deterministic canonical identity and CVE ref building | | `src/software.rs` | local raw aliases and atprotoV1 records | | `src/links.rs`, `src/links/repair.rs` | local link index, durability queue, repair drain | | `src/pds/` | publisher, same-DID assertions, governs writes and blobs | | `src/migrate/`, `src/bin/identity-migrate.rs` | out-of-band one-time identity migration | ## Data flow ### Common cycle ```text S3 snapshot download (optional) -> open cache DB; load cursor; --backfill resets to 1999; otherwise initialize --initial-lookback-days when no cursor -> build RateGovernor; resolve PDS URL; login (unless --dry-run) -> NVD fetch page -> upsert cves -> set cursor -> commit -> publish phase: drain due publish_errors, publish pending rows, drain again -> software phase when --publish-atproto: raw aliases, purls, atprotoV1 -> release mode dispatch (below) -> drain link_repairs for release work -> VACUUM INTO + S3 snapshot upload (optional) ``` Per-project failures inside a mode are isolated and counted; the next candidate proceeds. Release work commits local rows every 10 projects (history commits per concurrency chunk and every 50 jobs). ### `latest` ```text release_candidates_after(limit, recheck_seconds, after_slug) -> parse_purl + choose_purl (leaf match, then ecosystem priority) -> depsdev_versions(system, name) -> select_release (default, else newest non-deprecated representable) -> build_source for pypi/cargo/gem only; npm/maven/nuget/golang have no adapter -> resolve artifact through ArtifactResolver (cap, digest, cache, mirror) -> put project -> manifest -> version -> latest channel -> plan/apply statuses (registry signal + deps.dev deprecation) -> projectMeta when the source exposes it and the PDS supports listRecords -> mark release_* rows and release_project_state ``` The standalone mode runs only when `--publish-releases` is set. `all` runs the same pass unconditionally. ### `affected` ```text release_candidates_after(...) -> choose_affected_purl (ecosystem/source filters) -> resolve existing project strongRef -> local backlinks(software[].project.uri) -> CVE ids; no CVEs = skip -> OSV: online query/querybatch by versionless purl, or bulk-dir advisories -> affected_catalog = release_versions rows + deps.dev versions (7 ecosystems) -> plan_versions per CVE: introduced/fixed/last_affected ranges -> affected versions with an artifact -> immutable manifest + version affected versions without an artifact -> versionLabels range-only observations -> affectedRanges -> ensure channel points at the highest known version row -> update atprotoV1 software[] refs only when content changed -> write release_cve_links and local links ``` Artifacts in affected mode use the seven-ecosystem `artifact_for` path; a distro or other purl whose artifact cannot be resolved stays label-only. OSV never creates or yanks a version; only registry signals drive statuses. ### `history` ```text release_candidates_after(...) -> build_history_source (pypi, cargo, gem, github, gitlab, bitbucket, distro, composer, cpan, apache, gitee, gnu) -> source.versions() -> enqueue_versions (skip versions that already have a release_versions row; never reset an existing pending/retry/done/error job) -> process_jobs(limit = --release-history-limit): resolve one artifact through HistoryArtifacts -> reuse an existing manifest/version when recorded -> else publish immutable manifest + version -> plan statuses for materialized/deprecated entries ``` Jobs run `pending` -> `retry` -> `done`, or terminal `error` after three failed attempts. `skipped` jobs carry a typed reason and are not retried automatically. Projects are enumerated concurrently (`--release-concurrency`); writes still serialize through one PDS governor. ### `extras` ```text release_candidates_after(...) -> choose_affected_purl filters -> build_source (every registered adapter) -> source.versions() -> newest stable (prerelease/draft markers excluded; fall back to newest prerelease when every version is prerelease) -> source.artifacts() -> resolve_declared through the shared resolver -> project -> projectMeta -> manifest -> version -> latest channel -> source.statuses() only when capabilities.statuses -> release_version_meta and release_project_state ``` An unchanged project (same newest version, existing version row and channel) performs no PDS writes. A source with no artifact is `skipped` with reason `no_artifact`; an unknown/unsupported purl type is `unsupported` with the rendered `UnsupportedReason` in `release_project_state.unsupported_reason`. ### `all` `latest -> affected -> history (only when --release-history-limit > 0) -> extras`, sharing one candidate rotation cursor and one artifact resolver. Each mode keeps its own counters; `artifact_*` aggregates every resolve. The runtime guards every release arm with `!dry_run`, so `--dry-run` skips all release modes; `sync_all_with_http_resolver`'s library-level dry-run extras branch is not reachable from the CLI. ## Database tables Existing NVD/publish state: | Table | Purpose | |---|---| | `meta` | cursor and small key/value state | | `cves` | raw NVD JSON keyed by CVE id | | `publications` | per-repo CVE publish status, URI/CID, blob metadata | | `software` | raw CPE alias rows and canonical project refs | | `software_purls` | PURL lookup rows; `software_slug` stores the canonical slug | | `cve_atproto` | atprotoV1 publish status | | `links` | local outgoing-link index (source URI/CID, path, subject URI/CID) | Release current-state indexes: | Table | Purpose | |---|---| | `release_projects` | canonical project row, selected purl/system, latest version, record refs | | `release_versions` | immutable version row and manifest refs | | `release_channels` | mutable channel pointer per project | | `release_status` | current/latest status rows by subject | Release caches, queues, and identity state: | Table | Purpose | |---|---| | `software_identity` | raw slug -> canonical slug/identity key | | `project_aliases` | declared alias-key mapping; no current caller outside the DB layer | | `project_channels` | declared channel/latest/EOL state; no current caller outside the DB layer | | `release_artifacts` | verified artifact cache, PK `(repo, source, package, version, url)` | | `release_manifests` | manifest key -> URI/CID/artifacts JSON | | `release_history_jobs` | durable version jobs with status/attempts/error | | `release_osv` | cached OSV advisories by purl/id | | `release_cve_links` | CVE -> project/version/channel/manifest refs and labels | | `release_source_state` | declared per-source cursor/etag; no current caller outside the DB layer | | `release_status_events` | append-only status transitions | | `release_project_state` | canonical purl, source, raw versions, checked_at, unsupported reason | | `release_version_meta` | published_at, raw version, artifact cid/size | | `identity_migration` | migration cursor per `(repo, phase, raw_slug)` | | `link_repairs` | serialized records awaiting reindex | | `publish_errors` | CVE publish retry state | | `source_host_state` | per-host token-bucket window and `Retry-After` backoff; read/written by `GovernedHttp` | | `rate_budget` | persistent PDS hour/day points per repo DID | Schema is additive and created with `CREATE TABLE IF NOT EXISTS`; no `ALTER TABLE` or app-startup migration runs. ## Event and cache contracts - Lanes report `ProgressEvent`; the runtime forwards it as a `RuntimeEvent`. `RuntimeEvent::payload` serializes the full stats struct plus `error` for terminal phase events; `CycleCompleted` includes the complete `CycleStats`. - stdout is NDJSON `{"event": name, ...payload}`; stderr is the log stream. The full event/field table is in [`../MAINTAINERS.md#runtime-events`](../MAINTAINERS.md#runtime-events). - Artifact cache key: `(repo_did, source, package, raw_version, url)`. A cache hit returns the stored digest/CID/size/mime and increments `artifact_cache_hits` without downloading. - Manifest reuse: `release_manifests` is keyed by `(repo, project_slug, manifest_key)` and holds the published URI/CID and artifacts JSON. An existing version row is never overwritten. - S3 mirror key: the resolver key is `artifacts/`; `--artifact-s3-path` (default `artifacts`) is the object-key prefix, so the default object key is `artifacts/artifacts/`. The mirror is read first; a miss falls back to the registry; a failed put is best-effort and does not fail the resolve. - OSV cache: `release_osv` stores aliases/affected/modified per `(repo, purl, osv_id)`; `summary` and `severity` are advisory-only and are not persisted. - Local links: `links` is replaced per source URI on every successful write, so reindexing is idempotent. ## Idempotency and immutability - `software.release.project` rkeys are permanent. `name` equals the rkey; renames are expressed with `renamedTo`, never an in-place edit. - `software.release.version` and `software.release.manifest` are immutable: re-observing one reuses the row/record; it never rewrites bytes or artifacts. - `software.release.channel` is a mutable pointer; a new record is written only when the pointer changes. - `software.release.status` is append-only. The rkey is a digest over `(subject, status, reason)`; current state is the latest live same-DID event. - `com.nahdig.release.projectMeta` is compared before `putRecord`; an unchanged second run makes zero PDS writes. Without PDS `listRecords`, metadata writes are skipped because "only when changed" cannot be verified. - `com.nahdig.cve.atprotoV1` is `putRecord` only when `$type`, `cve`, `nvd`, or `software` content changes. - History jobs are never reset: re-enumeration skips existing versions and preserves pending/retry/error attempts. - Artifact resolution reuses the digest cache; declared digests must match the downloaded bytes. Manifest/version writes reuse recorded rows. - Cross-DID strongRefs are rejected before every write: project, channel, version, manifest, status, software, purl, and atprotoV1. ## Failure and retry semantics | Failure | Behavior | |---|---| | NVD 403/5xx/timeout | bounded outer retry at 30/60/120/300s + jitter; non-200 loops are bounded | | NVD rate limit | client token bucket; 5 req/30s without a key, 50/30s with a key | | PDS 429 | honor `Retry-After`/`RateLimit-Reset`, else 60s -> 1800s jittered backoff, bounded attempts | | PDS write budget | governor sleeps until the hour/day window resets; state persists in `rate_budget` | | Publish failure | recorded in `publish_errors`; retried before/after the next publish phase with capped exponential backoff; success clears the row | | Link index failure | record serialized to `link_repairs` in the same transaction; drained at startup and after phases; no PDS calls | | Registry 429/5xx | shared HTTP transport: 4 attempts, 1/2/4s backoff; artifact size-cap and digest-mismatch errors are not retried | | Artifact over cap | typed `TooLarge`; affected falls back to `versionLabels`, latest/extras count a project failure, history fails the job (retried, then terminal `error`); never retried as a transient source error | | Artifact digest mismatch | typed `DigestMismatch`; never accepted or cached | | Source error | per-project failure counter; cycle continues; unsupported types are typed and logged | | History job failure | attempts incremented; `pending`/`retry` until 3 attempts, then terminal `error`; `--resume` is not needed because the next cycle resumes pending/retry jobs | | S3 cache upload | 4 attempts, 1/2/4s + jitter, post-upload size check; final failure is logged and returned | | S3 artifact mirror | best-effort; miss re-downloads; failure does not fail the resolve | | Fatal runtime event | orchestrator sends shutdown; process exits nonzero | `--dry-run` keeps credentials optional, fetches/caches NVD data only, skips every release mode, and writes no PDS records. `identity-migrate` is the only tool that rewrites canonical refs; the service never runs it. ## Related - [`release-sources.md`](release-sources.md) — adapter matrix, auth, digests, status signals, unsupported reasons, RHEL/UBI mapping. - [`../MAINTAINERS.md`](../MAINTAINERS.md) — prod commands, rate tables, events, write-budget schedules, migration runbook.