Message Bus Architecture
The Hummingbird message bus is an event-driven architecture built on AWS SNS. Events from multiple sources flow through a central topic, enabling subscribers to filter and process only the events they need.
Architecture
flowchart TD
subgraph Publishers
CVE[CVE Analysis]
GL[GitLab Webhooks]
JIRA[Jira Automation]
K8S[Kubernetes Clusters]
end
subgraph MessageBus [Message Bus]
SNS[(SNS Topic)]
end
subgraph Archiver
ARCH[SNS S3 Archiver]
S3[(S3 Bucket)]
end
subgraph StatusDB [Status Database]
SQS1[SQS Queue]
WORKER[hummingbird-status]
PG[(PostgreSQL)]
end
subgraph ConsumerQueue [Consumer Queue]
SQS2[SQS Queue]
end
subgraph Catalog [Container Catalog]
SYNC[SyncFunction]
DDB[(DynamoDB)]
end
subgraph Consumers
CONSUMER[Event Consumer]
end
CVE -->|CVE-agent sweep request| SNS
GL -->|gitlab-event-forwarder| SNS
JIRA -->|jira-event-forwarder| SNS
K8S -->|kubernetes-event-forwarder| SNS
SNS --> ARCH
ARCH --> S3
SNS --> SQS1
SQS1 --> WORKER
WORKER --> PG
SNS -->|"FilterPolicy\nkind=Release"| SYNC
SYNC -->|fetch manifests| Registry[(OCI Registry)]
SYNC -->|read/write| DDB
SNS --> SQS2
SQS2 -->|new events| CONSUMER
S3 -.->|historic events| CONSUMER
PG -.->|aggregated status| CONSUMER
Components
| Component | Role | Description | Staging |
|---|---|---|---|
| hummingbird-events-topic | Infrastructure | Central SNS topic for all events | Yes |
| hummingbird-cve-analysis | Publisher | Publishes requests for CVE-agent sweeps | No |
| hummingbird-cve-agent | Subscriber | Consumes sweep requests and investigates eligible Jira tickets | No |
| gitlab-event-forwarder | Publisher | Receives GitLab webhooks, publishes to SNS | Yes |
| jira-event-forwarder | Publisher | Receives Jira Automation webhooks, publishes to SNS | Yes |
| kubernetes-event-forwarder | Publisher | Watches K8s resources, publishes changes to SNS | Yes |
| sns-s3-archiver | Subscriber | Archives all events to S3 for querying/replay | Yes |
| hummingbird-status | Subscriber | Ingests events to PostgreSQL for structured queries | Yes |
| pipeline-timing | Subscriber | Stores pipeline timing in its own schema and queue | Yes |
| hummingbird-agent | Subscriber | Processes events to drive automated workflows | Yes |
| workqueue-service | Subscriber | Queues and dispatches work items from events | Yes |
| container-catalog | Subscriber | Incrementally syncs image metadata to DynamoDB on Release events | No |
For subscriber rows, the Staging column describes running consumers. Pipeline Timing queues and subscriptions can be provisioned in both environments before the workers run; their presence does not mark the service as deployed.
Message Format
All messages include standard attributes for filtering:
| Attribute | Description | Examples |
|---|---|---|
source |
Event origin | cve-analysis, gitlab, jira, kubernetes |
event_type |
Type of event | cve_agent_sweep_requested, push, merge_request, pipeline, … |
Additional attributes vary by source - see individual publisher docs for details.
GitLab Events
Published by gitlab-event-forwarder:
| Attribute | Description | Example |
|---|---|---|
project_path |
Full project path | redhat/hummingbird/containers |
group_path |
Full group path | redhat/hummingbird |
When configured, Pipeline Hooks use event_type: pipeline; Job Hooks use
event_type: build.
CVE-agent Sweep Requests
Published by hummingbird-cve-analysis:
| Attribute | Value | Description |
|---|---|---|
source |
cve-analysis |
Identifies the publisher |
event_type |
cve_agent_sweep_requested |
Requests a CVE-agent sweep (wake-up request) |
The SNS Message is plain JSON, with no content_encoding attribute.
The subscription uses SNS envelope delivery, so the SQS body contains the SNS
notification envelope and its Message field contains the JSON above. Publish
errors are logged but do not fail the completed analysis cycle; a later enabled
cycle can publish another request.
Jira Events
Published by jira-event-forwarder:
| Attribute | Description | Example |
|---|---|---|
project_key |
Jira project key | HUM |
issue_type |
Issue type name | Story, Bug |
The event_type attribute is set from the X-Jira-Event-Type request header,
which each Jira Automation rule hardcodes for its trigger type
(e.g., issue_created, issue_updated).
Kubernetes Events
Published by kubernetes-event-forwarder:
| Attribute | Description | Example |
|---|---|---|
cluster |
Logical kubeconfig cluster entry name | konflux-rh03 |
namespace |
Object namespace | production |
kind |
Resource kind | PipelineRun, TaskRun, Snapshot, Release |
api_version |
API version | tekton.dev/v1, appstudio.redhat.com/v1alpha1 |
object_name |
Resource name | nginx-7d8c4c9d6f |
Published kinds depend on the deployed forwarder configuration. TaskRun source
activation is separate rollout work. The forwarder obtains cluster from the
selected kubeconfig context’s context.cluster value, not the API server URL.
ExternalSyncQueue
SQS queue fed by ExternalReleaseFunction (triggered by EventBridge schedule
every 6 hours) and consumed by ExternalSyncFunction (one message per
invocation, BatchSize: 1, ReportBatchItemFailures) for registry tag
resolution and metadata sync.
Message body:
{ "registry": "docker.io/library/nginx" }
Same-account SQS with no resource policy — the sender’s Lambda execution role
carries the sqs:SendMessage policy. Failed messages redrive to
ExternalSyncDLQ after 3 receives.
ExternalIndexQueue
SQS queue fed by ExternalSyncFunction and consumed by
ExternalIndexFunction (one message per invocation, BatchSize: 1,
ReportBatchItemFailures) for Syft SBOM generation. One message per digest,
emitted after each TAGS write for every non-superseded digest of the registry
(registries with no changes at all emit nothing).
Message body:
{
"registry": "docker.io/library/python",
"canonicals": ["3.15"],
"digest": "sha256:…",
"architectures": ["amd64", "arm64"]
}
Same-account SQS with no resource policy — the sender’s Lambda execution role
carries the sqs:SendMessage policy. Failed messages redrive to
ExternalIndexDLQ after 3 receives; DLQ entries are not retried automatically
(see the container-catalog documentation for recovery).
Subscription Filtering
SNS filter policies enable subscribers to receive only relevant events:
{
"source": ["gitlab"],
"event_type": ["push", "merge_request"]
}
{
"source": ["jira"],
"event_type": ["issue_created", "issue_updated"]
}
{
"source": ["kubernetes"],
"kind": ["Deployment"],
"event_type": ["MODIFIED"]
}
Kubernetes Release events (used by container-catalog sync Lambda):
{
"kind": ["Release"]
}
See hummingbird-events-topic for subscription setup instructions.
Event Flow Example
- Developer pushes to GitLab repository
- GitLab sends webhook to gitlab-event-forwarder Lambda
- Lambda validates token, extracts metadata, publishes to SNS
- SNS delivers to all matching subscribers:
- sns-s3-archiver stores event in S3
- Event consumers process new events in real-time
Archive keys
The S3 archiver writes one deterministic key for each SNS message. Duplicate
delivery rewrites the same object. GitLab pipeline and job identities include
the project and their numeric IDs; all Kubernetes identities include logical
cluster, namespace, and name. URL-valued cluster attributes from older
forwarders retain their name-only archive identities.
See SNS S3 Archiver for the exact key format and the machine-readable archive-key contract used by the archiver and GitLab forwarder tests.
Consuming Events
Consumers can receive events from multiple sources:
- New events: Subscribe to SNS topic for real-time processing
- Historic events: Download from S3 archive for replay or catch-up
- Structured queries: Query PostgreSQL via hummingbird-status for pipeline status, component state, and cross-referenced data
The S3 archive enables consumers to bootstrap state, then switch to live SNS
events. Hummingbird Status provides a queryable view of its current pipeline
status data. Pipeline Timing consumes its own SQS queue and stores selected
reporting fields in the pipeline_timing schema. Follow a domain row’s
execution identity and parentage to query creation/start/completion times and
overall status. After historical creation-timestamp recovery, the legacy event
catalogue and pointer columns are removed. Timing ingestion maintains only reporting
state and ordering metadata; the independent S3 archiver continues unchanged.
Archived notifications can be searched using publication time and resource identity,
subject to archive retention, without a permanent database row for every message.
Replaying Events
The sns-s3-archiver stores complete SNS records with decoded payloads, enabling easy replay:
# Download archived events
aws s3 sync s3://bucket-name/2025/12/13/ ./local-events/
# Browse events
zcat ./local-events/2025/12/13/12/34/*.json.gz | jq
import gzip
import json
from pathlib import Path
# Replay archived events to handler
for path in sorted(Path("./local-events").rglob("*.json.gz")):
with gzip.open(path, "rt") as f:
sns_record = json.load(f)
event = {"Records": [{"Sns": sns_record}]}
my_handler.lambda_handler(event, None)
See sns-s3-archiver documentation for details on storage format and
the _decode_message pattern for handlers.
Staging Environment
Production and staging use independent SNS topics. Each environment has its own publishers, archiver, and consumer queues so that changes can be tested in staging before production deployment.
flowchart TD
GL["GitLab Projects\n(dual webhook delivery)"]
JIRA["Jira Automation\n(dual webhook delivery)"]
CVE["CVE Analysis\n(CronJob)"]
K8S["Konflux Cluster\n(K8s resources)"]
subgraph production ["Production"]
CVE_P["hummingbird-cve-analysis\n(CronJob)"]
CVE_Q_P["CVE-agent SQS queue"]
CVE_A_P["hummingbird-cve-agent"]
GEF_P["gitlab-event-forwarder\n(Lambda)"]
JEF_P["jira-event-forwarder\n(Lambda)"]
KEF_P["kubernetes-event-forwarder\n(K8s pod)"]
SNS_P["SNS: arr-hummingbird-prod-events"]
prodConsumers["consumers\n(status, agent, workqueue,\narchiver, etc.)"]
CVE_P --> SNS_P
SNS_P -->|sweep request filter| CVE_Q_P
CVE_Q_P --> CVE_A_P
GEF_P --> SNS_P
JEF_P --> SNS_P
KEF_P --> SNS_P
SNS_P --> prodConsumers
end
subgraph staging ["Staging"]
CVE_S["hummingbird-cve-analysis\n(CronJob)"]
CVE_Q_S["CVE-agent SQS queue"]
CVE_A_S["hummingbird-cve-agent"]
GEF_S["gitlab-event-forwarder\n(Lambda)"]
JEF_S["jira-event-forwarder\n(Lambda)"]
KEF_S["kubernetes-event-forwarder\n(K8s pod)"]
SNS_S["SNS: arr-hummingbird-staging-events"]
stagConsumers["consumers\n(status, agent, workqueue,\narchiver)"]
CVE_S --> SNS_S
SNS_S -->|sweep request filter| CVE_Q_S
CVE_Q_S --> CVE_A_S
GEF_S --> SNS_S
JEF_S --> SNS_S
KEF_S --> SNS_S
SNS_S --> stagConsumers
end
GL --> GEF_P
GL --> GEF_S
CVE --> CVE_P
CVE --> CVE_S
JIRA --> JEF_P
JIRA --> JEF_S
K8S --> KEF_P
K8S --> KEF_S
How it works
- Duplicate webhook delivery: GitLab sends every event to both
gitlab-event-forwarder.hummingbird-project.io(prod) andgitlab-event-forwarder.staging.hummingbird-project.io(staging). Each forwarder publishes to its own SNS topic. - Duplicate webhook delivery: Jira Automation sends every event to both
jira-event-forwarder.hummingbird-project.io(prod) andjira-event-forwarder.staging.hummingbird-project.io(staging). Each forwarder publishes to its own SNS topic. - Dual Kubernetes watches: Both prod and staging kubernetes-event-forwarder pods watch the same Konflux cluster resources and publish to their respective topics.
- CVE-agent sweep requests: Each analysis CronJob publishes to its environment’s topic only when its sweep switch is enabled. Attribute filters route the request to that environment’s dedicated agent queue.
- Consumer isolation: Staging consumers (hummingbird-status,
pipeline-timing, hummingbird-agent, workqueue-service) have separate SQS queues
subscribed to the staging topic. Pipeline Timing shares Hummingbird Status’s
staging
eventsdatabase and owns only thepipeline_timingschema. - Staging archiver: A separate sns-s3-archiver subscribes to the staging topic and archives to its own S3 bucket with a shorter retention period than production.