Release sources
Per-adapter reference for the release lane. The source contract and registry
live in src/release/source.rs; implementations live in
src/release/sources/. Module and data-flow detail is in
release-architecture.md.
Mode support
| Mode | Source path |
|---|---|
latest | deps.dev default/latest for npm, pypi, maven, nuget, gem, cargo, golang; every registered type also has a full-history adapter |
affected | OSV advisories matched by purl; version catalogs come from local release rows plus deps.dev when the purl type is one of the seven; artifacts only through the seven-ecosystem artifact fetchers |
history | registered history adapters: pypi, cargo, gem, github, gitlab, bitbucket, rpm/deb/apk (distro), composer, cpan, apache, gitee, gnu |
extras | every adapter in SOURCE_REGISTRY, including npm, Maven, NuGet, Go, Docker/OCI, and SourceForge |
SOURCE_REGISTRY has 21 purl types mapping to 18 adapter names:
| Purl type | Adapter | deps.dev system |
|---|---|---|
npm | npm | NPM |
pypi | pypi | PYPI |
maven | maven | MAVEN |
nuget | nuget | NUGET |
gem | gem | RUBYGEMS |
cargo | cargo | CARGO |
golang | golang | GO |
rpm | distro | — |
deb | distro | — |
apk | distro | — |
github | github | — |
gitlab | gitlab | — |
bitbucket | bitbucket | — |
composer | composer | — |
cpan | cpan | — |
apache | apache | — |
gitee | gitee | — |
gnu | gnu | — |
docker | docker | — |
oci | docker | — |
sourceforge | sourceforge | — |
Every registered purl type now has a full-history/extras adapter. latest
still uses deps.dev for default selection; history/extras use the registry
adapters directly.
Capability matrix: auth and versions
| Purl type(s) | Adapter | Auth / token | Version source |
|---|---|---|---|
npm | deps.dev | none (public registry tarball fetch) | deps.dev /v3/systems/NPM/packages/{name}; default/deprecated ordering |
pypi | PyPI | none | https://pypi.org/pypi/{pep503-name}/json release keys; latest uses deps.dev |
maven | Maven Central maven-metadata.xml | maven | full versions; .jar/-sources.jar/.pom artifacts; .sha1 never relabeled |
nuget | NuGet registration | nuget | full versions; .nupkg via packageContent; unlisted/deprecated statuses |
gem | RubyGems | none | https://rubygems.org/api/v1/versions/{name}.json; latest uses deps.dev |
cargo | crates.io | none | https://crates.io/api/v1/crates/{name} versions array; latest uses deps.dev |
golang | proxy.golang.org @v/list | golang | full versions with Go path escaping; pseudo-versions opt-in; .zip/.mod; retractions |
rpm, deb, apk | distro | none for public lifecycle/UBI data; authenticated registries are typed Unavailable | curated RHEL table + Red Hat Product Life Cycle API + endoflife.date v1 |
github | GitHub | optional GITHUB_TOKEN (Authorization: Bearer) | api.github.com/repos/{owner}/{repo}/releases then /tags fallback |
gitlab | GitLab | optional GITLAB_TOKEN (PRIVATE-TOKEN) | gitlab.com/api/v4/projects/{id}/releases then /repository/tags |
bitbucket | Bitbucket | none | api.bitbucket.org/2.0/repositories/{workspace}/{repo}/refs/tags |
composer | Packagist | none | repo.packagist.org/p2/{vendor}/{package}.json |
cpan | MetaCPAN | none | MetaCPAN _search release documents for the distribution |
apache | Apache | none | downloads.apache.org/{project}/ and archive.apache.org/dist/{project}/ directory indexes |
gitee | Gitee | optional GITEE_TOKEN (Authorization: Bearer) | gitee.com/api/v5/repos/{owner}/{repo}/releases |
gnu | GNU FTP | none | ftp.gnu.org/gnu/{project}/ autoindex |
docker, oci | Docker Hub | anonymous pull token per operation from auth.docker.io | registry-1.docker.io/v2/{repo}/tags/list (RFC 8288 pagination) |
sourceforge | SourceForge | none | project RSS feed /projects/{project}/rss?path=/ plus /projects/{project}/files/ listing |
Tokens are sent only in request headers, never in URLs, logs, cycle stats, or
events. GITHUB_TOKEN, GITLAB_TOKEN, and GITEE_TOKEN are the only token
names; every other adapter is anonymous.
Capability matrix: artifacts, digests, statuses
| Purl type(s) | Artifact and digest path | Status signals |
|---|---|---|
npm | registry.npmjs.org full version map; tarball URL; SHA-1/SHA-512 never relabeled; SHA-256 computed on download | deprecated (kind: registry) with the npm reason; dist-tags exposed as channels |
pypi | release files; wheel preferred; declared digests.sha256 verified against downloaded bytes | yanked when every file is yanked; reason from yanked_reason |
maven | Maven Central maven-metadata.xml full versions; .jar/-sources.jar/.pom; SHA-256 computed | none (NoStatuses) |
nuget | NuGet registration full versions/pages; .nupkg via packageContent; SHA-256 computed | deprecated for unlisted/deprecated entries |
gem | gem_uri; declared sha and optional size, HEAD size fallback | yanked with kind: registry and yanked_reason |
cargo | static.crates.io .crate; declared checksum and crate_size | yanked with the registry reason kind |
golang | proxy.golang.org @v/list full versions; .zip/.mod; pseudo-versions opt-in; SHA-256 computed | retracted from per-version .info |
rpm, deb, apk | RHEL/UBI only: public OCI index manifest digest/URL; other distro builds are typed NoArtifacts | deprecated (kind: eol) after lifecycle EOL; archived (kind: eol) for retired products |
github | release assets; asset digest when present, otherwise download and SHA-256; tarball_url/zipball_url fallback | deprecated for draft/prerelease; archived for archived repositories |
gitlab | release asset links; SHA-256 computed; tag archive fallback | deprecated for upcoming_release (kind: draft); archived for archived projects |
bitbucket | repository downloads matched by file name; no declared digest, SHA-256 computed | none (NoStatuses) |
composer | p2 dist.url zip; dist.shasum used as SHA-256 only when 64 hex | deprecated for abandoned packages |
cpan | release tarball URL with declared SHA-256 | none (NoStatuses); backpan entries are marked deprecated in the catalog but emit no status record |
apache | archive files from the project index; .sha256 sidecars are disabled by default in this build, so SHA-256 is computed from the bytes | none (NoStatuses) |
gitee | release assets; no declared digest, SHA-256 computed | prerelease for prerelease releases |
gnu | archive files from the FTP index; signatures ignored; SHA-256 computed | none (NoStatuses) |
docker, oci | immutable manifest URL; Docker-Content-Digest is authoritative, size from Content-Length/manifest entries | deprecated with kind: mutable_tag when a tag resolves to a new digest |
sourceforge | direct download host downloads.sourceforge.net; size from listing; digest via downloader | none (NoStatuses) |
Digest semantics are uniform: a declared digest is verified against the downloaded bytes; a missing digest is computed; the artifact CID is the raw CIDv1 over the SHA-256. A digest mismatch or an over-cap download is a typed artifact error and is never retried as a transient source failure.
Rate limits and retry
| Source | Behavior |
|---|---|
| Shared registry HTTP | 4 attempts, 1s/2s/4s backoff, retry on 429 and 500/502/503/504; 60s timeout; redirects limited to 10 |
cargo | every JSON request spaced at least 1s per source instance |
github | surfaces x-ratelimit-remaining, x-ratelimit-reset, and retry-after; fails typed when exhausted |
gitlab | follows response pagination links; uses PRIVATE-TOKEN |
docker, oci | anonymous bearer token fetched per operation; 401/403 is typed Unavailable (private vs missing is indistinguishable) |
sourceforge | HTML error pages and missing projects are typed Unavailable, never parsed as records |
| OSV | online POST /v1/query and /v1/querybatch (split at 1,000 queries); offline bulk directory reads every *.json |
| PDS | separate governor: 5,000 points/hour, 35,000/day, 3,000 requests/5 min; create/put 3, update 2, delete 1 |
There is no adaptive per-host bucket in this build. source_host_state exists
is written by GovernedHttp on every rate-limit block and loaded at startup.
Unsupported reasons
UnsupportedReason is typed and logged; a project is never silently dropped:
| Reason | Meaning |
|---|---|
UnknownPurlType(String) | purl type absent from SOURCE_REGISTRY |
NoVersions | source has no version catalog for this project |
NoArtifacts | source has versions but no fetchable artifact (for example a distro build not published as UBI) |
NoStatuses | source has no typed lifecycle signal |
Unavailable(&'static str) | source cannot answer, for example missing transport, authenticated Red Hat registry, SourceForge error page, or an unknown/unsupported registry type |
Extras records the rendered reason in
release_project_state.unsupported_reason and emits
release_extras_skipped. History records unsupported counters and keeps the
project row; latest marks unsupported_no_purl when the project has no
release-ecosystem purl.
RHEL / UBI artifact mapping
The distro adapter treats every Red Hat Enterprise Linux variant as the
canonical project red-hat-enterprise-linux (slug and rkey). Versions and
channels come from:
- the curated
RHEL_CURATED_CHANNELStable (6,6-els,7,7-els,8,8-eus,9,9-eus,10); - the public Red Hat Product Life Cycle API
(
access.redhat.com/product-life-cycles/api/v1/products?name=Red%20Hat%20Enterprise%20Linux); - endoflife.date v1 (
endoflife.date/api/v1) when the product is published there.
Point releases are only emitted when lifecycle data supplies them; they are never invented. Channels replace dots with hyphens because channel rkeys cannot contain dots; versions keep dots.
For RHEL/UBI, the artifact is the public OCI index at:
https://registry.access.redhat.com/v2/ubi{major}/ubi/manifests/{stream}
{major}is the leading numeric component of the version (9from9.4).{stream}is the version itself when it names a point release (9.4), otherwise the major (9).- Architecture tokens (
aarch64,ppc64le,ppc64,s390x) are stripped from the stream and select a manifest from the index; the default isamd64/x86_64. - The response
Docker-Content-Digest(or a selected index entry digest) is the SHA-256; the immutable manifest URL is the artifact URL; the file name isubi{major}-ubi-{stream}.oci.json. - A 401/403 means the registry requires authentication and produces
Unavailable("redhat registry requires authentication"), so callers emit version labels only. A non-UBI distro build producesNoArtifacts. No subscription artifact is fabricated.
The canonical channel/version mapping is documented in
identity-model.md and
identity-migration.md.
Source URL overrides
--release-source-url (or RELEASE_SOURCE_URL) is a local e2e/test seam for
adapters with configurable base URLs:
| Mode | Adapters |
|---|---|
history | composer, cpan, apache, gitee, gnu |
extras | composer, cpan, apache, gitee, gnu, sourceforge |
It does not affect deps.dev, registry artifact URLs, GitHub/GitLab/Bitbucket, Docker Hub, distro lifecycle URLs, or OSV.
Related
release-architecture.md— module map, DB tables, event/cache contracts, retries.identity-model.md— canonical project, channel, and version rules.src/release/source.rs—SOURCE_REGISTRY, capabilities, unsupported enum.