ProdSec RPM Catalog
Builds an RPM catalog from each Hummingbird Pulp domain and publishes it to
that domain’s rpm-catalog file repository. The public catalog reads RPM
metadata through DNF. Private catalogs read repository versions through the
authenticated Pulp API so content guards on downloads do not block catalog
generation. A monitoring companion prints the last public publication time.
Features
- DNF-Based Public Query: Reads package metadata from all 5 public RPM repos (source, x86_64, aarch64, ppc64le, s390x)
- Authenticated Private Query: Reads the same repos from a pinned Pulp repository version, requesting only the fields used in the catalog
- Pulp REST API Upload: Creates file content in the selected repository; autopublish updates its distribution
- Artifact Deduplication: Skips upload when content SHA-256 matches an existing artifact
- Async Task Polling: Handles Pulp’s asynchronous tasks with configurable timeout (10 minutes)
- Sentry Error Reporting: Errors are reported to Sentry when
SENTRY_DSNis set - Monitoring Check: Companion module prints the publication timestamp for automated freshness monitoring
Prerequisites
- Python 3.11 or later
dnfsystem library for the public catalog (available on Fedora/RHEL)- Network access to the Pulp API and public repo metadata
- Pulp API access with client certificate or username/password auth
- An autopublishing file repository and distribution named
rpm-catalogin each target domain - Writer access to view and modify the file repository in each target domain
Usage
Build and Publish Catalog
python3 -m hummingbird_tools.prodsec_catalog rpm-catalog
For a private domain, use its restricted-reader config for package listing and its writer config for catalog publication:
PULP_READER_CONFIG=./reader-cli.toml PULP_CONFIG=./writer-cli.toml \
python3 -m hummingbird_tools.prodsec_catalog rpm-catalog \
--domain private-hummingbird-example --source pulp-api
The private CronJob mounts a ConfigMap with one domain per line and processes each domain with the same reader and writer credentials:
PULP_READER_CONFIG=./reader-cli.toml PULP_CONFIG=./writer-cli.toml \
python3 -m hummingbird_tools.prodsec_catalog rpm-catalog \
--domains-file ./domains.txt --source pulp-api
Blank lines and lines beginning with # are ignored. Only private-*
domains are accepted. If one domain fails, the updater continues with the
others and exits unsuccessfully after processing the full list.
The updater refuses to publish an empty catalog. A missing RPM repository or an incomplete Pulp API page also fails the run instead of publishing partial data.
Check Last Publication Time
python3 -m hummingbird_tools.prodsec_catalog_check
Prints the timestamp of the most recent catalog publication in local timezone. Uses the same authentication as the catalog builder.
Configuration
Authentication
Client certificate auth (when configured):
| Variable | Description |
|---|---|
HUMMINGBIRD_PULP_BOT_CERTIFICATE |
Path to client certificate PEM file |
HUMMINGBIRD_PULP_BOT_KEY |
Path to private key PEM file (optional) |
HUMMINGBIRD_PULP_BOT_PASSWORD |
Optional passphrase for the key |
Basic auth (alternative for local use):
| Variable | Description |
|---|---|
PULP_USERNAME |
Pulp API username |
PULP_PASSWORD |
Pulp API password |
Credentials can also be configured in ~/.config/pulp/cli.toml (same format
as the pulp CLI). Set PULP_CONFIG to override the config file path (useful
in containers where the home directory may vary).
Pulp API
| Variable | Description |
|---|---|
PULP_BASE_URL |
Pulp API base URL (or read from cli.toml) |
PULP_CONFIG |
Path to pulp CLI config file (default ~/.config/pulp/cli.toml) |
PULP_READER_CONFIG |
Optional separate read-only CLI config for API enumeration |
--domain selects one Pulp domain (default public-hummingbird), while
--domains-file reads private domains from a mounted ConfigMap. --source
selects dnf (public default) or pulp-api (private domains). PULP_BASE_URL,
when set, must already include the selected domain’s /api/v3 path.
When PULP_READER_CONFIG is set, its credentials are used only for package
listing; PULP_CONFIG remains the writer used to upload the catalog. Without
a separate reader config, the writer performs both operations.
Error Reporting
| Variable | Description |
|---|---|
SENTRY_DSN |
Optional Sentry DSN for error tracking |
CronJob Schedule
Deployed via Kubernetes CronJobs in the infrastructure repository’s
kubernetes/hummingbird-tools/ templates:
| CronJob | Schedule | Package source |
|---|---|---|
hummingbird-tools-prodsec-catalog |
@daily |
Public DNF metadata |
hummingbird-tools-private-catalog |
@daily |
Pulp API for configured private domains |
The private CronJob has one container and mounts a ConfigMap containing the
private domain list. To onboard a domain, provision its RPM repositories and
an autopublishing rpm-catalog file repository and distribution, grant the
shared restricted-reader and writer accounts access to that domain, then add
its name to the ConfigMap in the infrastructure repository. Regenerate and
commit the infrastructure manifests. The CronJob mounts generic reader and
writer Kubernetes Secrets; one domain’s failure does not stop updates for
the remaining domains, but the Job still fails overall.
Catalog Format
The catalog is a tab-separated file with one RPM per line:
{name}-{version}-{release}.{arch}.rpm\t{repo}\t{build_timestamp}
Example:
nginx-1.28.0-1.hum1.x86_64.rpm x86_64 2026-01-15 14:30:00
kernel-6.12.5-1.hum1.src.rpm source 2026-01-10 08:00:00
For example, a catalog can be served at
https://packages.example.com/api/pulp-content/private-hummingbird-example/rpm-catalog/hummingbird-rpm-catalog.txt.
The distribution’s content guard still applies when downloading it.
Development
See the main README for development workflows.
make hummingbird-tools/setup # Install dependencies
make check # Lint code (ruff)
make test # Run unit tests
License
This project is licensed under the GNU General Public License v3.0 or later - see the LICENSE file for details.