1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
# Release sources
Per-adapter reference for the release lane. The source contract and registry
live in `src/release/source.rs`; implementations live in
`src/release/sources/`. Module and data-flow detail is in
[`release-architecture.md`](release-architecture.md).
## Mode support
| Mode | Source path |
|---|---|
| `latest` | deps.dev default/latest for `npm`, `pypi`, `maven`, `nuget`, `gem`, `cargo`, `golang`; every registered type also has a full-history adapter |
| `affected` | OSV advisories matched by purl; version catalogs come from local release rows plus deps.dev when the purl type is one of the seven; artifacts only through the seven-ecosystem artifact fetchers |
| `history` | registered history adapters: `pypi`, `cargo`, `gem`, `github`, `gitlab`, `bitbucket`, `rpm`/`deb`/`apk` (distro), `composer`, `cpan`, `apache`, `gitee`, `gnu` |
| `extras` | every adapter in `SOURCE_REGISTRY`, including npm, Maven, NuGet, Go, Docker/OCI, and SourceForge |
`SOURCE_REGISTRY` has 21 purl types mapping to 18 adapter names:
| Purl type | Adapter | deps.dev system |
|---|---|---|
| `npm` | `npm` | `NPM` |
| `pypi` | `pypi` | `PYPI` |
| `maven` | `maven` | `MAVEN` |
| `nuget` | `nuget` | `NUGET` |
| `gem` | `gem` | `RUBYGEMS` |
| `cargo` | `cargo` | `CARGO` |
| `golang` | `golang` | `GO` |
| `rpm` | `distro` | — |
| `deb` | `distro` | — |
| `apk` | `distro` | — |
| `github` | `github` | — |
| `gitlab` | `gitlab` | — |
| `bitbucket` | `bitbucket` | — |
| `composer` | `composer` | — |
| `cpan` | `cpan` | — |
| `apache` | `apache` | — |
| `gitee` | `gitee` | — |
| `gnu` | `gnu` | — |
| `docker` | `docker` | — |
| `oci` | `docker` | — |
| `sourceforge` | `sourceforge` | — |
Every registered purl type now has a full-history/extras adapter. `latest`
still uses deps.dev for default selection; `history`/`extras` use the registry
adapters directly.
## Capability matrix: auth and versions
| Purl type(s) | Adapter | Auth / token | Version source |
|---|---|---|---|
| `npm` | deps.dev | none (public registry tarball fetch) | deps.dev `/v3/systems/NPM/packages/{name}`; default/deprecated ordering |
| `pypi` | PyPI | none | `https://pypi.org/pypi/{pep503-name}/json` release keys; latest uses deps.dev |
| `maven` | Maven Central `maven-metadata.xml` | `maven` | full versions; `.jar`/`-sources.jar`/`.pom` artifacts; `.sha1` never relabeled |
| `nuget` | NuGet registration | `nuget` | full versions; `.nupkg` via `packageContent`; unlisted/deprecated statuses |
| `gem` | RubyGems | none | `https://rubygems.org/api/v1/versions/{name}.json`; latest uses deps.dev |
| `cargo` | crates.io | none | `https://crates.io/api/v1/crates/{name}` versions array; latest uses deps.dev |
| `golang` | `proxy.golang.org` `@v/list` | `golang` | full versions with Go path escaping; pseudo-versions opt-in; `.zip`/`.mod`; retractions |
| `rpm`, `deb`, `apk` | distro | none for public lifecycle/UBI data; authenticated registries are typed `Unavailable` | curated RHEL table + Red Hat Product Life Cycle API + endoflife.date v1 |
| `github` | GitHub | optional `GITHUB_TOKEN` (`Authorization: Bearer`) | `api.github.com/repos/{owner}/{repo}/releases` then `/tags` fallback |
| `gitlab` | GitLab | optional `GITLAB_TOKEN` (`PRIVATE-TOKEN`) | `gitlab.com/api/v4/projects/{id}/releases` then `/repository/tags` |
| `bitbucket` | Bitbucket | none | `api.bitbucket.org/2.0/repositories/{workspace}/{repo}/refs/tags` |
| `composer` | Packagist | none | `repo.packagist.org/p2/{vendor}/{package}.json` |
| `cpan` | MetaCPAN | none | MetaCPAN `_search` release documents for the distribution |
| `apache` | Apache | none | `downloads.apache.org/{project}/` and `archive.apache.org/dist/{project}/` directory indexes |
| `gitee` | Gitee | optional `GITEE_TOKEN` (`Authorization: Bearer`) | `gitee.com/api/v5/repos/{owner}/{repo}/releases` |
| `gnu` | GNU FTP | none | `ftp.gnu.org/gnu/{project}/` autoindex |
| `docker`, `oci` | Docker Hub | anonymous pull token per operation from `auth.docker.io` | `registry-1.docker.io/v2/{repo}/tags/list` (RFC 8288 pagination) |
| `sourceforge` | SourceForge | none | project RSS feed `/projects/{project}/rss?path=/` plus `/projects/{project}/files/` listing |
Tokens are sent only in request headers, never in URLs, logs, cycle stats, or
events. `GITHUB_TOKEN`, `GITLAB_TOKEN`, and `GITEE_TOKEN` are the only token
names; every other adapter is anonymous.
## Capability matrix: artifacts, digests, statuses
| Purl type(s) | Artifact and digest path | Status signals |
|---|---|---|
| `npm` | `registry.npmjs.org` full version map; tarball URL; SHA-1/SHA-512 never relabeled; SHA-256 computed on download | `deprecated` (`kind: registry`) with the npm reason; dist-tags exposed as channels |
| `pypi` | release files; wheel preferred; declared `digests.sha256` verified against downloaded bytes | `yanked` when every file is yanked; reason from `yanked_reason` |
| `maven` | Maven Central `maven-metadata.xml` full versions; `.jar`/`-sources.jar`/`.pom`; SHA-256 computed | none (`NoStatuses`) |
| `nuget` | NuGet registration full versions/pages; `.nupkg` via `packageContent`; SHA-256 computed | `deprecated` for unlisted/deprecated entries |
| `gem` | `gem_uri`; declared `sha` and optional size, HEAD size fallback | `yanked` with `kind: registry` and `yanked_reason` |
| `cargo` | `static.crates.io` `.crate`; declared `checksum` and `crate_size` | `yanked` with the registry reason kind |
| `golang` | `proxy.golang.org` `@v/list` full versions; `.zip`/`.mod`; pseudo-versions opt-in; SHA-256 computed | `retracted` from per-version `.info` |
| `rpm`, `deb`, `apk` | RHEL/UBI only: public OCI index manifest digest/URL; other distro builds are typed `NoArtifacts` | `deprecated` (`kind: eol`) after lifecycle EOL; `archived` (`kind: eol`) for retired products |
| `github` | release assets; asset `digest` when present, otherwise download and SHA-256; `tarball_url`/`zipball_url` fallback | `deprecated` for draft/prerelease; `archived` for archived repositories |
| `gitlab` | release asset links; SHA-256 computed; tag archive fallback | `deprecated` for `upcoming_release` (`kind: draft`); `archived` for archived projects |
| `bitbucket` | repository downloads matched by file name; no declared digest, SHA-256 computed | none (`NoStatuses`) |
| `composer` | p2 `dist.url` zip; `dist.shasum` used as SHA-256 only when 64 hex | `deprecated` for `abandoned` packages |
| `cpan` | release tarball URL with declared SHA-256 | none (`NoStatuses`); backpan entries are marked deprecated in the catalog but emit no status record |
| `apache` | archive files from the project index; `.sha256` sidecars are disabled by default in this build, so SHA-256 is computed from the bytes | none (`NoStatuses`) |
| `gitee` | release assets; no declared digest, SHA-256 computed | `prerelease` for prerelease releases |
| `gnu` | archive files from the FTP index; signatures ignored; SHA-256 computed | none (`NoStatuses`) |
| `docker`, `oci` | immutable manifest URL; `Docker-Content-Digest` is authoritative, size from `Content-Length`/manifest entries | `deprecated` with `kind: mutable_tag` when a tag resolves to a new digest |
| `sourceforge` | direct download host `downloads.sourceforge.net`; size from listing; digest via downloader | none (`NoStatuses`) |
Digest semantics are uniform: a declared digest is verified against the
downloaded bytes; a missing digest is computed; the artifact CID is the raw
CIDv1 over the SHA-256. A digest mismatch or an over-cap download is a typed
artifact error and is never retried as a transient source failure.
## Rate limits and retry
| Source | Behavior |
|---|---|
| Shared registry HTTP | 4 attempts, 1s/2s/4s backoff, retry on 429 and 500/502/503/504; 60s timeout; redirects limited to 10 |
| `cargo` | every JSON request spaced at least 1s per source instance |
| `github` | surfaces `x-ratelimit-remaining`, `x-ratelimit-reset`, and `retry-after`; fails typed when exhausted |
| `gitlab` | follows response pagination links; uses `PRIVATE-TOKEN` |
| `docker`, `oci` | anonymous bearer token fetched per operation; 401/403 is typed `Unavailable` (private vs missing is indistinguishable) |
| `sourceforge` | HTML error pages and missing projects are typed `Unavailable`, never parsed as records |
| OSV | online `POST /v1/query` and `/v1/querybatch` (split at 1,000 queries); offline bulk directory reads every `*.json` |
| PDS | separate governor: 5,000 points/hour, 35,000/day, 3,000 requests/5 min; create/put 3, update 2, delete 1 |
There is no adaptive per-host bucket in this build. `source_host_state` exists
is written by `GovernedHttp` on every rate-limit block and loaded at startup.
## Unsupported reasons
`UnsupportedReason` is typed and logged; a project is never silently dropped:
| Reason | Meaning |
|---|---|
| `UnknownPurlType(String)` | purl type absent from `SOURCE_REGISTRY` |
| `NoVersions` | source has no version catalog for this project |
| `NoArtifacts` | source has versions but no fetchable artifact (for example a distro build not published as UBI) |
| `NoStatuses` | source has no typed lifecycle signal |
| `Unavailable(&'static str)` | source cannot answer, for example missing transport, authenticated Red Hat registry, SourceForge error page, or an unknown/unsupported registry type |
Extras records the rendered reason in
`release_project_state.unsupported_reason` and emits
`release_extras_skipped`. History records `unsupported` counters and keeps the
project row; latest marks `unsupported_no_purl` when the project has no
release-ecosystem purl.
## RHEL / UBI artifact mapping
The distro adapter treats every Red Hat Enterprise Linux variant as the
canonical project `red-hat-enterprise-linux` (slug and rkey). Versions and
channels come from:
1. the curated `RHEL_CURATED_CHANNELS` table (`6`, `6-els`, `7`, `7-els`,
`8`, `8-eus`, `9`, `9-eus`, `10`);
2. the public Red Hat Product Life Cycle API
(`access.redhat.com/product-life-cycles/api/v1/products?name=Red%20Hat%20Enterprise%20Linux`);
3. endoflife.date v1 (`endoflife.date/api/v1`) when the product is published
there.
Point releases are only emitted when lifecycle data supplies them; they are
never invented. Channels replace dots with hyphens because channel rkeys
cannot contain dots; versions keep dots.
For RHEL/UBI, the artifact is the public OCI index at:
```text
https://registry.access.redhat.com/v2/ubi{major}/ubi/manifests/{stream}
```
- `{major}` is the leading numeric component of the version (`9` from `9.4`).
- `{stream}` is the version itself when it names a point release (`9.4`),
otherwise the major (`9`).
- Architecture tokens (`aarch64`, `ppc64le`, `ppc64`, `s390x`) are stripped
from the stream and select a manifest from the index; the default is
`amd64`/`x86_64`.
- The response `Docker-Content-Digest` (or a selected index entry digest) is
the SHA-256; the immutable manifest URL is the artifact URL; the file name is
`ubi{major}-ubi-{stream}.oci.json`.
- A 401/403 means the registry requires authentication and produces
`Unavailable("redhat registry requires authentication")`, so callers emit
version labels only. A non-UBI distro build produces `NoArtifacts`. No
subscription artifact is fabricated.
The canonical channel/version mapping is documented in
[`identity-model.md`](identity-model.md) and
[`identity-migration.md`](identity-migration.md).
## Source URL overrides
`--release-source-url` (or `RELEASE_SOURCE_URL`) is a local e2e/test seam for
adapters with configurable base URLs:
| Mode | Adapters |
|---|---|
| `history` | `composer`, `cpan`, `apache`, `gitee`, `gnu` |
| `extras` | `composer`, `cpan`, `apache`, `gitee`, `gnu`, `sourceforge` |
It does not affect deps.dev, registry artifact URLs, GitHub/GitLab/Bitbucket,
Docker Hub, distro lifecycle URLs, or OSV.
## Related
- [`release-architecture.md`](release-architecture.md) — module map, DB tables,
event/cache contracts, retries.
- [`identity-model.md`](identity-model.md) — canonical project, channel, and
version rules.
- `src/release/source.rs` — `SOURCE_REGISTRY`, capabilities, unsupported enum.