# RPM pipeline timing design

LLMS index: [llms.txt](https://hummingbird-project.io/llms.txt) | Full content: [llms-full.txt](https://hummingbird-project.io/llms-full.txt)

---

## Problem

We lack a clear way to see where time is spent in the RPM build pipeline.

## Background

> References:
> [HUM-5819](https://redhat.atlassian.net/browse/HUM-5819) - Create a diagram that describes all
> phases and steps of various pipelines from start to finish. Then review with team.
> [HUM-5820](https://redhat.atlassian.net/browse/HUM-5820) - Time breakdown from MR creation to
> completion of Release pipeline.

During the F44 rebuild, RPM builds had problems whose causes were not immediately clear. The sprint
retrospective and a later F44-specific retrospective created HUM-5819 and HUM-5820 to improve
visibility into pipeline behavior and time spent in each phase.

An initial pass through HUM-5819 identified key areas for time tracking. HUM-5820 records those areas.

## Findings

The team initially considered a small addition to the `tools/hummingbird-status` database. A [draft
MR](https://gitlab.com/redhat/hummingbird/tools/-/merge_requests/1165) explored that approach.

Findings from the draft MR:

1. The existing Status events database is a suitable home because the system forwards GitLab and
   Konflux events through SNS, archives them in S3, and updates PostgreSQL. The timing worker runs
   as an independent service with its own image and deployment lifecycle.
2. Some `hummingbird-dashboard` functions do directly query the `raw` JSON.
3. The original purpose of `hummingbird-status` was to diagnose Konflux event issues, not long-term
   system diagnosis.
4. The `raw` event JSON for Konflux PipelineRuns consumes significant space, about 97% of the 47 GB
   database, despite redundant event storage in S3.
5. `hummingbird-status` PVC monitoring is now in place; the draft MR highlighted its absence.
6. The solution to our problem statement is not to mimic the existing approach.

## Proposal

Track required event data in new tables in the existing database and update event processing to
populate them without changing existing tables or Dashboard behavior.

### Events Already Agreed Upon for Tracking

HUM-5820 outlines the events already agreed upon for tracking.

1. GitLab MRs and commits
2. GitLab pipelines
3. GitLab pipeline jobs
4. Konflux PipelineRuns
5. Konflux Tekton TaskRuns

The model must support reporting of:

1. Status
2. Relevant start and end times, including MR creation and merge times
3. Event timestamps
4. Event lineage across GitLab and Konflux, sufficient to reconstruct a single pipeline timeline

This is not a database design. It identifies the reporting requirements that detailed design must meet.

### Approach

1. Preserve existing event ingestion, database tables, and Dashboard behavior. Run timing ingestion
   in a separate `pipeline-timing` service and image while retaining the existing publishers,
   archiver, and events database.
2. Create a new schema in the existing PostgreSQL database.
   - The schema provides logical isolation so existing consumers do not accidentally use the new tables.
3. Create tables that hold the data needed to meet the agreed reporting requirements without
   duplicating raw event JSON.
4. Treat the S3 archive as the authoritative raw-event record for the new schema.
   - Store a reference that lets consumers retrieve the archived event for detail, display, or later
     data-model improvements.
5. Configure GitLab webhooks and the Kubernetes event forwarder to include the agreed event types,
   then route them through the existing SNS event pipeline.
6. Ensure the S3 archiver can archive and identify every agreed event type.

## Acceptance Criteria

1. Existing Status database tables and Dashboard behavior remain unchanged.
2. The new PostgreSQL schema provides the agreed event coverage.
3. Representative data demonstrates an RPM pipeline timeline from MR creation through completion of
   the Release pipeline.
4. Consumers can calculate time spent between tracked pipeline phases without querying raw event JSON.
5. Consumers can trace lineage across all agreed event types and retrieve the corresponding
   authoritative S3 event when needed.
6. Tests verify the new data model, lineage, and S3 references.

## Out of Scope

1. Creating a UI for visualization is out of scope.
   - The data may later support a new application, Dashboard changes, or Grafana. UX should be
     involved in that decision.
2. Tracking the containers repository pipeline is out of scope.
   - That pipeline builds OCI images from RPMs produced by the RPM pipeline and is a potential
     follow-up effort.
