# Release sources Per-adapter reference for the release lane. The source contract and registry live in `src/release/source.rs`; implementations live in `src/release/sources/`. Module and data-flow detail is in [`release-architecture.md`](release-architecture.md). ## Mode support | Mode | Source path | |---|---| | `latest` | deps.dev default/latest for `npm`, `pypi`, `maven`, `nuget`, `gem`, `cargo`, `golang`; every registered type also has a full-history adapter | | `affected` | OSV advisories matched by purl; version catalogs come from local release rows plus deps.dev when the purl type is one of the seven; artifacts only through the seven-ecosystem artifact fetchers | | `history` | registered history adapters: `pypi`, `cargo`, `gem`, `github`, `gitlab`, `bitbucket`, `rpm`/`deb`/`apk` (distro), `composer`, `cpan`, `apache`, `gitee`, `gnu` | | `extras` | every adapter in `SOURCE_REGISTRY`, including npm, Maven, NuGet, Go, Docker/OCI, and SourceForge | `SOURCE_REGISTRY` has 21 purl types mapping to 18 adapter names: | Purl type | Adapter | deps.dev system | |---|---|---| | `npm` | `npm` | `NPM` | | `pypi` | `pypi` | `PYPI` | | `maven` | `maven` | `MAVEN` | | `nuget` | `nuget` | `NUGET` | | `gem` | `gem` | `RUBYGEMS` | | `cargo` | `cargo` | `CARGO` | | `golang` | `golang` | `GO` | | `rpm` | `distro` | — | | `deb` | `distro` | — | | `apk` | `distro` | — | | `github` | `github` | — | | `gitlab` | `gitlab` | — | | `bitbucket` | `bitbucket` | — | | `composer` | `composer` | — | | `cpan` | `cpan` | — | | `apache` | `apache` | — | | `gitee` | `gitee` | — | | `gnu` | `gnu` | — | | `docker` | `docker` | — | | `oci` | `docker` | — | | `sourceforge` | `sourceforge` | — | Every registered purl type now has a full-history/extras adapter. `latest` still uses deps.dev for default selection; `history`/`extras` use the registry adapters directly. ## Capability matrix: auth and versions | Purl type(s) | Adapter | Auth / token | Version source | |---|---|---|---| | `npm` | deps.dev | none (public registry tarball fetch) | deps.dev `/v3/systems/NPM/packages/{name}`; default/deprecated ordering | | `pypi` | PyPI | none | `https://pypi.org/pypi/{pep503-name}/json` release keys; latest uses deps.dev | | `maven` | Maven Central `maven-metadata.xml` | `maven` | full versions; `.jar`/`-sources.jar`/`.pom` artifacts; `.sha1` never relabeled | | `nuget` | NuGet registration | `nuget` | full versions; `.nupkg` via `packageContent`; unlisted/deprecated statuses | | `gem` | RubyGems | none | `https://rubygems.org/api/v1/versions/{name}.json`; latest uses deps.dev | | `cargo` | crates.io | none | `https://crates.io/api/v1/crates/{name}` versions array; latest uses deps.dev | | `golang` | `proxy.golang.org` `@v/list` | `golang` | full versions with Go path escaping; pseudo-versions opt-in; `.zip`/`.mod`; retractions | | `rpm`, `deb`, `apk` | distro | none for public lifecycle/UBI data; authenticated registries are typed `Unavailable` | curated RHEL table + Red Hat Product Life Cycle API + endoflife.date v1 | | `github` | GitHub | optional `GITHUB_TOKEN` (`Authorization: Bearer`) | `api.github.com/repos/{owner}/{repo}/releases` then `/tags` fallback | | `gitlab` | GitLab | optional `GITLAB_TOKEN` (`PRIVATE-TOKEN`) | `gitlab.com/api/v4/projects/{id}/releases` then `/repository/tags` | | `bitbucket` | Bitbucket | none | `api.bitbucket.org/2.0/repositories/{workspace}/{repo}/refs/tags` | | `composer` | Packagist | none | `repo.packagist.org/p2/{vendor}/{package}.json` | | `cpan` | MetaCPAN | none | MetaCPAN `_search` release documents for the distribution | | `apache` | Apache | none | `downloads.apache.org/{project}/` and `archive.apache.org/dist/{project}/` directory indexes | | `gitee` | Gitee | optional `GITEE_TOKEN` (`Authorization: Bearer`) | `gitee.com/api/v5/repos/{owner}/{repo}/releases` | | `gnu` | GNU FTP | none | `ftp.gnu.org/gnu/{project}/` autoindex | | `docker`, `oci` | Docker Hub | anonymous pull token per operation from `auth.docker.io` | `registry-1.docker.io/v2/{repo}/tags/list` (RFC 8288 pagination) | | `sourceforge` | SourceForge | none | project RSS feed `/projects/{project}/rss?path=/` plus `/projects/{project}/files/` listing | Tokens are sent only in request headers, never in URLs, logs, cycle stats, or events. `GITHUB_TOKEN`, `GITLAB_TOKEN`, and `GITEE_TOKEN` are the only token names; every other adapter is anonymous. ## Capability matrix: artifacts, digests, statuses | Purl type(s) | Artifact and digest path | Status signals | |---|---|---| | `npm` | `registry.npmjs.org` full version map; tarball URL; SHA-1/SHA-512 never relabeled; SHA-256 computed on download | `deprecated` (`kind: registry`) with the npm reason; dist-tags exposed as channels | | `pypi` | release files; wheel preferred; declared `digests.sha256` verified against downloaded bytes | `yanked` when every file is yanked; reason from `yanked_reason` | | `maven` | Maven Central `maven-metadata.xml` full versions; `.jar`/`-sources.jar`/`.pom`; SHA-256 computed | none (`NoStatuses`) | | `nuget` | NuGet registration full versions/pages; `.nupkg` via `packageContent`; SHA-256 computed | `deprecated` for unlisted/deprecated entries | | `gem` | `gem_uri`; declared `sha` and optional size, HEAD size fallback | `yanked` with `kind: registry` and `yanked_reason` | | `cargo` | `static.crates.io` `.crate`; declared `checksum` and `crate_size` | `yanked` with the registry reason kind | | `golang` | `proxy.golang.org` `@v/list` full versions; `.zip`/`.mod`; pseudo-versions opt-in; SHA-256 computed | `retracted` from per-version `.info` | | `rpm`, `deb`, `apk` | RHEL/UBI only: public OCI index manifest digest/URL; other distro builds are typed `NoArtifacts` | `deprecated` (`kind: eol`) after lifecycle EOL; `archived` (`kind: eol`) for retired products | | `github` | release assets; asset `digest` when present, otherwise download and SHA-256; `tarball_url`/`zipball_url` fallback | `deprecated` for draft/prerelease; `archived` for archived repositories | | `gitlab` | release asset links; SHA-256 computed; tag archive fallback | `deprecated` for `upcoming_release` (`kind: draft`); `archived` for archived projects | | `bitbucket` | repository downloads matched by file name; no declared digest, SHA-256 computed | none (`NoStatuses`) | | `composer` | p2 `dist.url` zip; `dist.shasum` used as SHA-256 only when 64 hex | `deprecated` for `abandoned` packages | | `cpan` | release tarball URL with declared SHA-256 | none (`NoStatuses`); backpan entries are marked deprecated in the catalog but emit no status record | | `apache` | archive files from the project index; `.sha256` sidecars are disabled by default in this build, so SHA-256 is computed from the bytes | none (`NoStatuses`) | | `gitee` | release assets; no declared digest, SHA-256 computed | `prerelease` for prerelease releases | | `gnu` | archive files from the FTP index; signatures ignored; SHA-256 computed | none (`NoStatuses`) | | `docker`, `oci` | immutable manifest URL; `Docker-Content-Digest` is authoritative, size from `Content-Length`/manifest entries | `deprecated` with `kind: mutable_tag` when a tag resolves to a new digest | | `sourceforge` | direct download host `downloads.sourceforge.net`; size from listing; digest via downloader | none (`NoStatuses`) | Digest semantics are uniform: a declared digest is verified against the downloaded bytes; a missing digest is computed; the artifact CID is the raw CIDv1 over the SHA-256. A digest mismatch or an over-cap download is a typed artifact error and is never retried as a transient source failure. ## Rate limits and retry | Source | Behavior | |---|---| | Shared registry HTTP | 4 attempts, 1s/2s/4s backoff, retry on 429 and 500/502/503/504; 60s timeout; redirects limited to 10 | | `cargo` | every JSON request spaced at least 1s per source instance | | `github` | surfaces `x-ratelimit-remaining`, `x-ratelimit-reset`, and `retry-after`; fails typed when exhausted | | `gitlab` | follows response pagination links; uses `PRIVATE-TOKEN` | | `docker`, `oci` | anonymous bearer token fetched per operation; 401/403 is typed `Unavailable` (private vs missing is indistinguishable) | | `sourceforge` | HTML error pages and missing projects are typed `Unavailable`, never parsed as records | | OSV | online `POST /v1/query` and `/v1/querybatch` (split at 1,000 queries); offline bulk directory reads every `*.json` | | PDS | separate governor: 5,000 points/hour, 35,000/day, 3,000 requests/5 min; create/put 3, update 2, delete 1 | There is no adaptive per-host bucket in this build. `source_host_state` exists is written by `GovernedHttp` on every rate-limit block and loaded at startup. ## Unsupported reasons `UnsupportedReason` is typed and logged; a project is never silently dropped: | Reason | Meaning | |---|---| | `UnknownPurlType(String)` | purl type absent from `SOURCE_REGISTRY` | | `NoVersions` | source has no version catalog for this project | | `NoArtifacts` | source has versions but no fetchable artifact (for example a distro build not published as UBI) | | `NoStatuses` | source has no typed lifecycle signal | | `Unavailable(&'static str)` | source cannot answer, for example missing transport, authenticated Red Hat registry, SourceForge error page, or an unknown/unsupported registry type | Extras records the rendered reason in `release_project_state.unsupported_reason` and emits `release_extras_skipped`. History records `unsupported` counters and keeps the project row; latest marks `unsupported_no_purl` when the project has no release-ecosystem purl. ## RHEL / UBI artifact mapping The distro adapter treats every Red Hat Enterprise Linux variant as the canonical project `red-hat-enterprise-linux` (slug and rkey). Versions and channels come from: 1. the curated `RHEL_CURATED_CHANNELS` table (`6`, `6-els`, `7`, `7-els`, `8`, `8-eus`, `9`, `9-eus`, `10`); 2. the public Red Hat Product Life Cycle API (`access.redhat.com/product-life-cycles/api/v1/products?name=Red%20Hat%20Enterprise%20Linux`); 3. endoflife.date v1 (`endoflife.date/api/v1`) when the product is published there. Point releases are only emitted when lifecycle data supplies them; they are never invented. Channels replace dots with hyphens because channel rkeys cannot contain dots; versions keep dots. For RHEL/UBI, the artifact is the public OCI index at: ```text https://registry.access.redhat.com/v2/ubi{major}/ubi/manifests/{stream} ``` - `{major}` is the leading numeric component of the version (`9` from `9.4`). - `{stream}` is the version itself when it names a point release (`9.4`), otherwise the major (`9`). - Architecture tokens (`aarch64`, `ppc64le`, `ppc64`, `s390x`) are stripped from the stream and select a manifest from the index; the default is `amd64`/`x86_64`. - The response `Docker-Content-Digest` (or a selected index entry digest) is the SHA-256; the immutable manifest URL is the artifact URL; the file name is `ubi{major}-ubi-{stream}.oci.json`. - A 401/403 means the registry requires authentication and produces `Unavailable("redhat registry requires authentication")`, so callers emit version labels only. A non-UBI distro build produces `NoArtifacts`. No subscription artifact is fabricated. The canonical channel/version mapping is documented in [`identity-model.md`](identity-model.md) and [`identity-migration.md`](identity-migration.md). ## Source URL overrides `--release-source-url` (or `RELEASE_SOURCE_URL`) is a local e2e/test seam for adapters with configurable base URLs: | Mode | Adapters | |---|---| | `history` | `composer`, `cpan`, `apache`, `gitee`, `gnu` | | `extras` | `composer`, `cpan`, `apache`, `gitee`, `gnu`, `sourceforge` | It does not affect deps.dev, registry artifact URLs, GitHub/GitLab/Bitbucket, Docker Hub, distro lifecycle URLs, or OSV. ## Related - [`release-architecture.md`](release-architecture.md) — module map, DB tables, event/cache contracts, retries. - [`identity-model.md`](identity-model.md) — canonical project, channel, and version rules. - `src/release/source.rs` — `SOURCE_REGISTRY`, capabilities, unsupported enum.