Hummingbird Events Topic

An SNS topic for all Hummingbird project events. This topic serves as a central hub for distributing events from multiple sources (GitLab webhooks, Kubernetes, etc.) to multiple subscribers, enabling event-driven architectures and integrations.

Features

Central Event Hub: Dedicated SNS topic per environment (production and staging) for all project events from multiple sources
Multiple Subscribers: Supports Lambda, SQS, HTTP endpoints, email, SMS, and more
Event Filtering: Subscribers can filter events using SNS subscription filter policies

Prerequisites

AWS CLI configured with appropriate credentials (IAM permissions for SNS, CloudFormation)
Podman or Docker (for containerized SAM build/deploy)

Deployment

Build and deploy using containerized AWS SAM CLI:

cd hummingbird-events-topic
make build     # Build SAM application
make deploy    # First deployment (interactive/guided)
make redeploy  # Subsequent deployments (non-interactive)

Deployment outputs:

TopicArn - SNS topic ARN (for event publishers)
TopicName - SNS topic name

Parameters

Parameter	Description	Default
`TopicName`	SNS topic name	`myapp-prod-events`

Resource naming: The SNS topic uses the provided TopicName parameter.

Usage

Publishing Events

Event publishers need the topic ARN to publish messages:

# Get topic ARN from CloudFormation stack
aws cloudformation describe-stacks \
  --stack-name <stack-name> \
  --query 'Stacks[0].Outputs[?OutputKey==`TopicArn`].OutputValue' \
  --output text

Event publishers:

gitlab-event-forwarder - Publishes GitLab webhook events
kubernetes-event-forwarder - Publishes Kubernetes resource changes
Custom event sources

Subscribing to Events

Subscribe services to receive events:

Via AWS Console:

Open SNS console → Topics
Select the topic
Create subscription (choose protocol: Lambda, SQS, HTTP, Email, etc.)
Add subscription filter policy (optional)

Via AWS CLI:

aws sns subscribe \
  --topic-arn <topic-arn> \
  --protocol lambda \
  --notification-endpoint <lambda-arn>

Add filter policy:

aws sns set-subscription-attributes \
  --subscription-arn <subscription-arn> \
  --attribute-name FilterPolicy \
  --attribute-value '{"source": ["gitlab"], "event_type": ["push"]}'

Subscription Filter Examples

GitLab push events from specific project:

{
  "source": ["gitlab"],
  "event_type": ["push"],
  "project_path": ["redhat/hummingbird/containers"]
}

All merge request events:

{
  "source": ["gitlab"],
  "event_type": ["merge_request"]
}

All events from GitLab:

{
  "source": ["gitlab"]
}

Event metadata: See publisher documentation for available metadata:

gitlab-event-forwarder - GitLab webhook event metadata
kubernetes-event-forwarder - Kubernetes resource event metadata

Pipeline Timing queue and recovery

Pipeline Timing uses a separate standard SQS queue and dead-letter queue in each environment. The source queue subscribes to its environment’s SNS topic through pipeline-timing/template.yaml; the DLQ receives messages from the source queue after the receive limit. The Status queue and worker remain separate and unchanged. Provision queues and subscriptions in both environments before deploying the workers. Messages received before worker deployment may expire after four days; this pre-launch loss is accepted. Without a consumer, messages are not received and do not automatically move to the DLQ. After launch, monitor queue age during worker pauses to avoid losing events. Validate the staging worker before deploying the production worker or enabling new production event sources.

Setting	Value
Queue	`arr-hummingbird-<env>-pipeline-timing-sqs`
Dead-letter queue	`arr-hummingbird-<env>-pipeline-timing-sqs-dlq`
Message retention	4 days on the source queue; 14 days on the DLQ
Redrive limit	3 receives
Visibility timeout	300 seconds, minimum 240 seconds
SNS raw message delivery	`false`, preserving the SNS envelope, message ID, and timestamp
Subscription filtering	None, the worker receives each topic event

When SQS automatically moves a standard-queue message to its DLQ, it preserves the original enqueue timestamp. The DLQ’s 14-day retention is measured from that timestamp, not added after the source queue’s four days. An operator redrive from the DLQ back to the source queue resets the enqueue timestamp.

The worker’s 120-second per-message processing budget must be enforced before it starts polling. The 300-second visibility timeout is at least twice that budget. The queue names include sqs-dlq so the existing CloudWatch rules classify the DLQ separately. Grafana Alloy is Hummingbird’s monitoring collector. SAM stack tags let its CloudWatch exporter find the source queue and DLQ and forward their queue-depth and oldest-message-age metrics to Mimir. Alloy does not read or consume SQS messages.

The dedicated worker IAM user can call sqs:ReceiveMessage and sqs:DeleteMessage on the timing queue only. It has no access to the Status queue or the DLQ. The queue policy grants SNS delivery only from the matching environment topic and denies non-HTTPS requests. The DLQ redrive policy accepts only its matching source queue. The template uses the Status stack’s UserName and UserArn output names. Operator redrive uses separate AWS credentials.

Fix malformed events before redriving them. An operator can move messages from the DLQ back to the timing queue and check the move task. Set ENV_PREFIX to staging or prod, then read the queue ARNs from the stack outputs:

STACK_NAME="arr-hummingbird-${ENV_PREFIX}-pipeline-timing"
TIMING_QUEUE_ARN=$(aws cloudformation describe-stacks \
  --stack-name "$STACK_NAME" \
  --query "Stacks[0].Outputs[?OutputKey=='QueueArn'].OutputValue" \
  --output text)
TIMING_DLQ_ARN=$(aws cloudformation describe-stacks \
  --stack-name "$STACK_NAME" \
  --query "Stacks[0].Outputs[?OutputKey=='DeadLetterQueueArn'].OutputValue" \
  --output text)

aws sqs start-message-move-task \
  --source-arn "$TIMING_DLQ_ARN" \
  --destination-arn "$TIMING_QUEUE_ARN"

aws sqs list-message-move-tasks --source-arn "$TIMING_DLQ_ARN"

Verify that the message returns to the timing queue and that the worker processes it. During the initial infrastructure test, if the worker is not deployed yet, remove only the identifiable test message after confirming the move. Do not purge the queue. SNS message IDs make worker redelivery idempotent.

Development

This is a pure infrastructure project (no application code). See the main README for SAM build/deploy commands.

Security & Limitations

Security:

SNS topic follows least privilege principle
Access controlled via IAM policies
Supports server-side encryption (optional)
Publishers require sns:Publish permission
Subscribers require appropriate protocol permissions

Limitations:

Message size: Up to 256 KB
Maximum subscriptions: 12,500,000 per topic
Message retention: Not supported (use SQS for durable queuing)
Delivery retries: Protocol-dependent

License

This project is licensed under the GNU General Public License v3.0 or later - see the LICENSE file for details.