~ knotless Search/
↑↓ move↵ open⇧↵ all results

johnandersen777.bsky.social/vuln-notif

0
0
main
196 lines · 11.9 KB Raw History

Release sources

Per-adapter reference for the release lane. The source contract and registry live in src/release/source.rs; implementations live in src/release/sources/. Module and data-flow detail is in release-architecture.md.

Mode support

ModeSource path
latestdeps.dev default/latest for npm, pypi, maven, nuget, gem, cargo, golang; every registered type also has a full-history adapter
affectedOSV advisories matched by purl; version catalogs come from local release rows plus deps.dev when the purl type is one of the seven; artifacts only through the seven-ecosystem artifact fetchers
historyregistered history adapters: pypi, cargo, gem, github, gitlab, bitbucket, rpm/deb/apk (distro), composer, cpan, apache, gitee, gnu
extrasevery adapter in SOURCE_REGISTRY, including npm, Maven, NuGet, Go, Docker/OCI, and SourceForge

SOURCE_REGISTRY has 21 purl types mapping to 18 adapter names:

Purl typeAdapterdeps.dev system
npmnpmNPM
pypipypiPYPI
mavenmavenMAVEN
nugetnugetNUGET
gemgemRUBYGEMS
cargocargoCARGO
golanggolangGO
rpmdistro—
debdistro—
apkdistro—
githubgithub—
gitlabgitlab—
bitbucketbitbucket—
composercomposer—
cpancpan—
apacheapache—
giteegitee—
gnugnu—
dockerdocker—
ocidocker—
sourceforgesourceforge—

Every registered purl type now has a full-history/extras adapter. latest still uses deps.dev for default selection; history/extras use the registry adapters directly.

Capability matrix: auth and versions

Purl type(s)AdapterAuth / tokenVersion source
npmdeps.devnone (public registry tarball fetch)deps.dev /v3/systems/NPM/packages/{name}; default/deprecated ordering
pypiPyPInonehttps://pypi.org/pypi/{pep503-name}/json release keys; latest uses deps.dev
mavenMaven Central maven-metadata.xmlmavenfull versions; .jar/-sources.jar/.pom artifacts; .sha1 never relabeled
nugetNuGet registrationnugetfull versions; .nupkg via packageContent; unlisted/deprecated statuses
gemRubyGemsnonehttps://rubygems.org/api/v1/versions/{name}.json; latest uses deps.dev
cargocrates.iononehttps://crates.io/api/v1/crates/{name} versions array; latest uses deps.dev
golangproxy.golang.org @v/listgolangfull versions with Go path escaping; pseudo-versions opt-in; .zip/.mod; retractions
rpm, deb, apkdistronone for public lifecycle/UBI data; authenticated registries are typed Unavailablecurated RHEL table + Red Hat Product Life Cycle API + endoflife.date v1
githubGitHuboptional GITHUB_TOKEN (Authorization: Bearer)api.github.com/repos/{owner}/{repo}/releases then /tags fallback
gitlabGitLaboptional GITLAB_TOKEN (PRIVATE-TOKEN)gitlab.com/api/v4/projects/{id}/releases then /repository/tags
bitbucketBitbucketnoneapi.bitbucket.org/2.0/repositories/{workspace}/{repo}/refs/tags
composerPackagistnonerepo.packagist.org/p2/{vendor}/{package}.json
cpanMetaCPANnoneMetaCPAN _search release documents for the distribution
apacheApachenonedownloads.apache.org/{project}/ and archive.apache.org/dist/{project}/ directory indexes
giteeGiteeoptional GITEE_TOKEN (Authorization: Bearer)gitee.com/api/v5/repos/{owner}/{repo}/releases
gnuGNU FTPnoneftp.gnu.org/gnu/{project}/ autoindex
docker, ociDocker Hubanonymous pull token per operation from auth.docker.ioregistry-1.docker.io/v2/{repo}/tags/list (RFC 8288 pagination)
sourceforgeSourceForgenoneproject RSS feed /projects/{project}/rss?path=/ plus /projects/{project}/files/ listing

Tokens are sent only in request headers, never in URLs, logs, cycle stats, or events. GITHUB_TOKEN, GITLAB_TOKEN, and GITEE_TOKEN are the only token names; every other adapter is anonymous.

Capability matrix: artifacts, digests, statuses

Purl type(s)Artifact and digest pathStatus signals
npmregistry.npmjs.org full version map; tarball URL; SHA-1/SHA-512 never relabeled; SHA-256 computed on downloaddeprecated (kind: registry) with the npm reason; dist-tags exposed as channels
pypirelease files; wheel preferred; declared digests.sha256 verified against downloaded bytesyanked when every file is yanked; reason from yanked_reason
mavenMaven Central maven-metadata.xml full versions; .jar/-sources.jar/.pom; SHA-256 computednone (NoStatuses)
nugetNuGet registration full versions/pages; .nupkg via packageContent; SHA-256 computeddeprecated for unlisted/deprecated entries
gemgem_uri; declared sha and optional size, HEAD size fallbackyanked with kind: registry and yanked_reason
cargostatic.crates.io .crate; declared checksum and crate_sizeyanked with the registry reason kind
golangproxy.golang.org @v/list full versions; .zip/.mod; pseudo-versions opt-in; SHA-256 computedretracted from per-version .info
rpm, deb, apkRHEL/UBI only: public OCI index manifest digest/URL; other distro builds are typed NoArtifactsdeprecated (kind: eol) after lifecycle EOL; archived (kind: eol) for retired products
githubrelease assets; asset digest when present, otherwise download and SHA-256; tarball_url/zipball_url fallbackdeprecated for draft/prerelease; archived for archived repositories
gitlabrelease asset links; SHA-256 computed; tag archive fallbackdeprecated for upcoming_release (kind: draft); archived for archived projects
bitbucketrepository downloads matched by file name; no declared digest, SHA-256 computednone (NoStatuses)
composerp2 dist.url zip; dist.shasum used as SHA-256 only when 64 hexdeprecated for abandoned packages
cpanrelease tarball URL with declared SHA-256none (NoStatuses); backpan entries are marked deprecated in the catalog but emit no status record
apachearchive files from the project index; .sha256 sidecars are disabled by default in this build, so SHA-256 is computed from the bytesnone (NoStatuses)
giteerelease assets; no declared digest, SHA-256 computedprerelease for prerelease releases
gnuarchive files from the FTP index; signatures ignored; SHA-256 computednone (NoStatuses)
docker, ociimmutable manifest URL; Docker-Content-Digest is authoritative, size from Content-Length/manifest entriesdeprecated with kind: mutable_tag when a tag resolves to a new digest
sourceforgedirect download host downloads.sourceforge.net; size from listing; digest via downloadernone (NoStatuses)

Digest semantics are uniform: a declared digest is verified against the downloaded bytes; a missing digest is computed; the artifact CID is the raw CIDv1 over the SHA-256. A digest mismatch or an over-cap download is a typed artifact error and is never retried as a transient source failure.

Rate limits and retry

SourceBehavior
Shared registry HTTP4 attempts, 1s/2s/4s backoff, retry on 429 and 500/502/503/504; 60s timeout; redirects limited to 10
cargoevery JSON request spaced at least 1s per source instance
githubsurfaces x-ratelimit-remaining, x-ratelimit-reset, and retry-after; fails typed when exhausted
gitlabfollows response pagination links; uses PRIVATE-TOKEN
docker, ocianonymous bearer token fetched per operation; 401/403 is typed Unavailable (private vs missing is indistinguishable)
sourceforgeHTML error pages and missing projects are typed Unavailable, never parsed as records
OSVonline POST /v1/query and /v1/querybatch (split at 1,000 queries); offline bulk directory reads every *.json
PDSseparate governor: 5,000 points/hour, 35,000/day, 3,000 requests/5 min; create/put 3, update 2, delete 1

There is no adaptive per-host bucket in this build. source_host_state exists is written by GovernedHttp on every rate-limit block and loaded at startup.

Unsupported reasons

UnsupportedReason is typed and logged; a project is never silently dropped:

ReasonMeaning
UnknownPurlType(String)purl type absent from SOURCE_REGISTRY
NoVersionssource has no version catalog for this project
NoArtifactssource has versions but no fetchable artifact (for example a distro build not published as UBI)
NoStatusessource has no typed lifecycle signal
Unavailable(&'static str)source cannot answer, for example missing transport, authenticated Red Hat registry, SourceForge error page, or an unknown/unsupported registry type

Extras records the rendered reason in release_project_state.unsupported_reason and emits release_extras_skipped. History records unsupported counters and keeps the project row; latest marks unsupported_no_purl when the project has no release-ecosystem purl.

RHEL / UBI artifact mapping

The distro adapter treats every Red Hat Enterprise Linux variant as the canonical project red-hat-enterprise-linux (slug and rkey). Versions and channels come from:

  1. the curated RHEL_CURATED_CHANNELS table (6, 6-els, 7, 7-els, 8, 8-eus, 9, 9-eus, 10);
  2. the public Red Hat Product Life Cycle API (access.redhat.com/product-life-cycles/api/v1/products?name=Red%20Hat%20Enterprise%20Linux);
  3. endoflife.date v1 (endoflife.date/api/v1) when the product is published there.

Point releases are only emitted when lifecycle data supplies them; they are never invented. Channels replace dots with hyphens because channel rkeys cannot contain dots; versions keep dots.

For RHEL/UBI, the artifact is the public OCI index at:

https://registry.access.redhat.com/v2/ubi{major}/ubi/manifests/{stream}
  • {major} is the leading numeric component of the version (9 from 9.4).
  • {stream} is the version itself when it names a point release (9.4), otherwise the major (9).
  • Architecture tokens (aarch64, ppc64le, ppc64, s390x) are stripped from the stream and select a manifest from the index; the default is amd64/x86_64.
  • The response Docker-Content-Digest (or a selected index entry digest) is the SHA-256; the immutable manifest URL is the artifact URL; the file name is ubi{major}-ubi-{stream}.oci.json.
  • A 401/403 means the registry requires authentication and produces Unavailable("redhat registry requires authentication"), so callers emit version labels only. A non-UBI distro build produces NoArtifacts. No subscription artifact is fabricated.

The canonical channel/version mapping is documented in identity-model.md and identity-migration.md.

Source URL overrides

--release-source-url (or RELEASE_SOURCE_URL) is a local e2e/test seam for adapters with configurable base URLs:

ModeAdapters
historycomposer, cpan, apache, gitee, gnu
extrascomposer, cpan, apache, gitee, gnu, sourceforge

It does not affect deps.dev, registry artifact URLs, GitHub/GitLab/Bitbucket, Docker Hub, distro lifecycle URLs, or OSV.

Related

  • release-architecture.md — module map, DB tables, event/cache contracts, retries.
  • identity-model.md — canonical project, channel, and version rules.
  • src/release/source.rs — SOURCE_REGISTRY, capabilities, unsupported enum.