ProdSec RPM Catalog

Builds an RPM catalog from each Hummingbird Pulp domain and publishes it to that domain’s rpm-catalog file repository. The public catalog reads RPM metadata through DNF. Private catalogs read repository versions through the authenticated Pulp API so content guards on downloads do not block catalog generation. A monitoring companion prints the last public publication time.

Features

  • DNF-Based Public Query: Reads package metadata from all 5 public RPM repos (source, x86_64, aarch64, ppc64le, s390x)
  • Authenticated Private Query: Reads the same repos from a pinned Pulp repository version, requesting only the fields used in the catalog
  • Pulp REST API Upload: Creates file content in the selected repository; autopublish updates its distribution
  • Artifact Deduplication: Skips upload when content SHA-256 matches an existing artifact
  • Async Task Polling: Handles Pulp’s asynchronous tasks with configurable timeout (10 minutes)
  • Sentry Error Reporting: Errors are reported to Sentry when SENTRY_DSN is set
  • Monitoring Check: Companion module prints the publication timestamp for automated freshness monitoring

Prerequisites

  • Python 3.11 or later
  • dnf system library for the public catalog (available on Fedora/RHEL)
  • Network access to the Pulp API and public repo metadata
  • Pulp API access with client certificate or username/password auth
  • An autopublishing file repository and distribution named rpm-catalog in each target domain
  • Writer access to view and modify the file repository in each target domain

Usage

Build and Publish Catalog

python3 -m hummingbird_tools.prodsec_catalog rpm-catalog

For a private domain, use its restricted-reader config for package listing and its writer config for catalog publication:

PULP_READER_CONFIG=./reader-cli.toml PULP_CONFIG=./writer-cli.toml \
  python3 -m hummingbird_tools.prodsec_catalog rpm-catalog \
    --domain private-hummingbird-example --source pulp-api

The private CronJob mounts a ConfigMap with one domain per line and processes each domain with the same reader and writer credentials:

PULP_READER_CONFIG=./reader-cli.toml PULP_CONFIG=./writer-cli.toml \
  python3 -m hummingbird_tools.prodsec_catalog rpm-catalog \
    --domains-file ./domains.txt --source pulp-api

Blank lines and lines beginning with # are ignored. Only private-* domains are accepted. If one domain fails, the updater continues with the others and exits unsuccessfully after processing the full list.

The updater refuses to publish an empty catalog. A missing RPM repository or an incomplete Pulp API page also fails the run instead of publishing partial data.

Check Last Publication Time

python3 -m hummingbird_tools.prodsec_catalog_check

Prints the timestamp of the most recent catalog publication in local timezone. Uses the same authentication as the catalog builder.

Configuration

Authentication

Client certificate auth (when configured):

Variable Description
HUMMINGBIRD_PULP_BOT_CERTIFICATE Path to client certificate PEM file
HUMMINGBIRD_PULP_BOT_KEY Path to private key PEM file (optional)
HUMMINGBIRD_PULP_BOT_PASSWORD Optional passphrase for the key

Basic auth (alternative for local use):

Variable Description
PULP_USERNAME Pulp API username
PULP_PASSWORD Pulp API password

Credentials can also be configured in ~/.config/pulp/cli.toml (same format as the pulp CLI). Set PULP_CONFIG to override the config file path (useful in containers where the home directory may vary).

Pulp API

Variable Description
PULP_BASE_URL Pulp API base URL (or read from cli.toml)
PULP_CONFIG Path to pulp CLI config file (default ~/.config/pulp/cli.toml)
PULP_READER_CONFIG Optional separate read-only CLI config for API enumeration

--domain selects one Pulp domain (default public-hummingbird), while --domains-file reads private domains from a mounted ConfigMap. --source selects dnf (public default) or pulp-api (private domains). PULP_BASE_URL, when set, must already include the selected domain’s /api/v3 path. When PULP_READER_CONFIG is set, its credentials are used only for package listing; PULP_CONFIG remains the writer used to upload the catalog. Without a separate reader config, the writer performs both operations.

Error Reporting

Variable Description
SENTRY_DSN Optional Sentry DSN for error tracking

CronJob Schedule

Deployed via Kubernetes CronJobs in the infrastructure repository’s kubernetes/hummingbird-tools/ templates:

CronJob Schedule Package source
hummingbird-tools-prodsec-catalog @daily Public DNF metadata
hummingbird-tools-private-catalog @daily Pulp API for configured private domains

The private CronJob has one container and mounts a ConfigMap containing the private domain list. To onboard a domain, provision its RPM repositories and an autopublishing rpm-catalog file repository and distribution, grant the shared restricted-reader and writer accounts access to that domain, then add its name to the ConfigMap in the infrastructure repository. Regenerate and commit the infrastructure manifests. The CronJob mounts generic reader and writer Kubernetes Secrets; one domain’s failure does not stop updates for the remaining domains, but the Job still fails overall.

Catalog Format

The catalog is a tab-separated file with one RPM per line:

{name}-{version}-{release}.{arch}.rpm\t{repo}\t{build_timestamp}

Example:

nginx-1.28.0-1.hum1.x86_64.rpm  x86_64  2026-01-15 14:30:00
kernel-6.12.5-1.hum1.src.rpm    source  2026-01-10 08:00:00

For example, a catalog can be served at https://packages.example.com/api/pulp-content/private-hummingbird-example/rpm-catalog/hummingbird-rpm-catalog.txt. The distribution’s content guard still applies when downloading it.

Development

See the main README for development workflows.

make hummingbird-tools/setup  # Install dependencies
make check                     # Lint code (ruff)
make test                      # Run unit tests

License

This project is licensed under the GNU General Public License v3.0 or later - see the LICENSE file for details.