Hummingbird OSV Generator

A CLI tool that converts Red Hat vulnerability data into OSV records for Hummingbird RPMs, so that vulnerability scanners can match them against the packages installed in an image.

For how a run works end to end, see the workflow.

Features

  • Two sources: shipped fixes from the advisories repository and all known affects, including unfixed CVEs, from OSIDB
  • Binary RPM names: scanners match installed binary packages, so source RPMs are expanded to their binary RPMs
  • Fixed and unfixed states: records carry fixed, affected, will_not_fix or under_investigation in database_specific.status
  • RPM epochs: fixed versions are written as epoch:version-release
  • Incremental sync: content hashing means only changed inputs are parsed and only changed records are written
  • Embargo safe: embargoed or non-public OSIDB data is never published

Output format

The output follows OSV schema 1.9.0 with one record per source RPM per CVE and one affected entry per binary RPM.

Field Content
id Opaque, stable identifier: x_HBA prefix, CVE year and a hash
upstream The CVE the record covers
affected[].package.ecosystem Ecosystem scoped to one product stream by its CPE
affected[].package.purl Versionless package URL
affected[].ranges introduced: 0 and, once shipped, fixed: <epoch:version-release>
affected[].ecosystem_specific.advisory_id Advisory that supplied the fixed version, absent on unfixed entries
database_specific.status fixed, affected, under_investigation or will_not_fix
modified Time of the last content change

The x_ in the identifier prefix marks a local database. It is dropped once HBA is registered with the OSV schema project. The ecosystem name is also a placeholder until it is registered.

Prerequisites

  • Go: the version in hummingbird-osv-generator/go.mod
  • Advisories checkout: a clone of the advisories repository, passed as its data/ directory
  • OSIDB access: the OSIDB base URL, reachable from where the tool runs
  • Git: only when using -git or -git-remote

Installation

git clone https://gitlab.com/redhat/hummingbird/tools.git
cd tools/hummingbird-osv-generator
go build -o rhcve-osv-sync ./cmd/rhcve-osv-sync/

Usage

Full sync for one product stream, committing the output to git:

./rhcve-osv-sync \
  -advisories /path/to/advisories/data \
  -osidb-url https://osidb.example.com \
  -streams hummingbird-1 \
  -products hummingbird \
  -output /path/to/osv-output \
  -state /path/to/state.db \
  -full -git

Later runs without -full use an OSIDB delta query. Run a full reconciliation daily so that missed changes are corrected.

Binary RPM expansion

OSIDB tracks CVEs against source RPM names, while scanners need binary RPM names. -expansion selects how a source RPM is expanded:

Mode Source
osidb OSIDB subpackage_purls
advisory Binary RPM names derived from advisory purls
chain OSIDB first, then the advisory mapping

The default is chain.

Incremental sync

State lives in a SQLite database. Each run goes through these phases: advisory diff, OSIDB diff, convert, write output and commit state. The workflow describes each phase.

  • Advisory files are content-hashed and only changed files are parsed
  • OSIDB uses a delta query, plus a full reconciliation on demand
  • Output files are compared with a manifest and only changed files are written
  • Output is written before state is committed, so a crash is safe
  • If the manifest is lost, the next full run rebuilds it from the output and withdraws stale records
  • If OSIDB fails and the convert phase does not need it, advisory output is still published and the process exits non-zero so a scheduler can retry or alert

Unchanged input produces unchanged output, so duplicate scheduling wastes work but does not corrupt the output.

Configuration

Flag Required Description
-advisories Yes Path to the advisories repository data/ directory
-osidb-url Yes OSIDB base URL
-streams Yes Comma-separated ps_update_stream values, also filters advisories by product_stream
-output Yes Output directory for the OSV JSON files
-state Yes Path to the SQLite state database
-products No Filter advisory conversion by CPE product name
-full No Force a full OSIDB reconciliation
-expansion No Binary RPM expansion mode: osidb, advisory or chain
-git No Run git init, add and commit on the output after the sync
-git-remote No Git remote to push to after the commit (implies -git)
-log-json No Write structured JSON logs

Development

cd hummingbird-osv-generator

# Vet and test, including the OSV schema validation test
go vet ./...
go test ./...

The hummingbird-osv-generator CI job runs the same commands on merge requests.

License

This project is licensed under the GNU General Public License v3.0 or later - see the LICENSE file for details.