Hummingbird OSV Generator
A CLI tool that converts Red Hat vulnerability data into OSV records for Hummingbird RPMs, so that vulnerability scanners can match them against the packages installed in an image.
For how a run works end to end, see the workflow.
Features
- Two sources: shipped fixes from the advisories repository and all known affects, including unfixed CVEs, from OSIDB
- Binary RPM names: scanners match installed binary packages, so source RPMs are expanded to their binary RPMs
- Fixed and unfixed states: records carry
fixed,affected,will_not_fixorunder_investigationindatabase_specific.status - RPM epochs: fixed versions are written as
epoch:version-release - Incremental sync: content hashing means only changed inputs are parsed and only changed records are written
- Embargo safe: embargoed or non-public OSIDB data is never published
Output format
The output follows OSV schema 1.9.0 with one record per source RPM per CVE and
one affected entry per binary RPM.
| Field | Content |
|---|---|
id |
Opaque, stable identifier: x_HBA prefix, CVE year and a hash |
upstream |
The CVE the record covers |
affected[].package.ecosystem |
Ecosystem scoped to one product stream by its CPE |
affected[].package.purl |
Versionless package URL |
affected[].ranges |
introduced: 0 and, once shipped, fixed: <epoch:version-release> |
affected[].ecosystem_specific.advisory_id |
Advisory that supplied the fixed version, absent on unfixed entries |
database_specific.status |
fixed, affected, under_investigation or will_not_fix |
modified |
Time of the last content change |
The x_ in the identifier prefix marks a local database. It is dropped once
HBA is registered with the OSV schema project. The ecosystem name is also a
placeholder until it is registered.
Prerequisites
- Go: the version in
hummingbird-osv-generator/go.mod - Advisories checkout: a clone of the advisories repository, passed as its
data/directory - OSIDB access: the OSIDB base URL, reachable from where the tool runs
- Git: only when using
-gitor-git-remote
Installation
git clone https://gitlab.com/redhat/hummingbird/tools.git
cd tools/hummingbird-osv-generator
go build -o rhcve-osv-sync ./cmd/rhcve-osv-sync/
Usage
Full sync for one product stream, committing the output to git:
./rhcve-osv-sync \
-advisories /path/to/advisories/data \
-osidb-url https://osidb.example.com \
-streams hummingbird-1 \
-products hummingbird \
-output /path/to/osv-output \
-state /path/to/state.db \
-full -git
Later runs without -full use an OSIDB delta query. Run a full
reconciliation daily so that missed changes are corrected.
Binary RPM expansion
OSIDB tracks CVEs against source RPM names, while scanners need binary RPM
names. -expansion selects how a source RPM is expanded:
| Mode | Source |
|---|---|
osidb |
OSIDB subpackage_purls |
advisory |
Binary RPM names derived from advisory purls |
chain |
OSIDB first, then the advisory mapping |
The default is chain.
Incremental sync
State lives in a SQLite database. Each run goes through these phases: advisory diff, OSIDB diff, convert, write output and commit state. The workflow describes each phase.
- Advisory files are content-hashed and only changed files are parsed
- OSIDB uses a delta query, plus a full reconciliation on demand
- Output files are compared with a manifest and only changed files are written
- Output is written before state is committed, so a crash is safe
- If the manifest is lost, the next full run rebuilds it from the output and withdraws stale records
- If OSIDB fails and the convert phase does not need it, advisory output is still published and the process exits non-zero so a scheduler can retry or alert
Unchanged input produces unchanged output, so duplicate scheduling wastes work but does not corrupt the output.
Configuration
| Flag | Required | Description |
|---|---|---|
-advisories |
Yes | Path to the advisories repository data/ directory |
-osidb-url |
Yes | OSIDB base URL |
-streams |
Yes | Comma-separated ps_update_stream values, also filters advisories by product_stream |
-output |
Yes | Output directory for the OSV JSON files |
-state |
Yes | Path to the SQLite state database |
-products |
No | Filter advisory conversion by CPE product name |
-full |
No | Force a full OSIDB reconciliation |
-expansion |
No | Binary RPM expansion mode: osidb, advisory or chain |
-git |
No | Run git init, add and commit on the output after the sync |
-git-remote |
No | Git remote to push to after the commit (implies -git) |
-log-json |
No | Write structured JSON logs |
Development
cd hummingbird-osv-generator
# Vet and test, including the OSV schema validation test
go vet ./...
go test ./...
The hummingbird-osv-generator CI job runs the same commands on merge
requests.
License
This project is licensed under the GNU General Public License v3.0 or later - see the LICENSE file for details.