S3 Lookaside Cache

Infrastructure for caching dist-git lookaside content, providing access to source artifacts for the Hummingbird build pipeline.

Overview

This stack provides S3-based infrastructure for dist-git artifacts with CloudFront CDN. It caches content from upstream sources like Git tarballs.

A companion stack (s3-lookaside-cache-upload-role) creates a GitLab OIDC identity provider and an IAM role that GitLab CI jobs can assume using short-lived JWT tokens, eliminating the need for long-lived access keys.

Architecture

Cache Infrastructure

flowchart LR
    clients["Build Clients"]
    cf["CloudFront\nDistribution"]
    s3["S3 Bucket\n(dist-git-cache)"]
    logs["S3 Logs\nBucket"]
    backup["AWS Backup\nVault"]
    headers["Security\nHeaders"]

    clients --> cf --> s3
    cf --> headers
    s3 --> logs --> backup

GitLab CI Upload Role

flowchart LR
    gitlab["GitLab CI\n(.gitlab-ci.yml)"]
    sts["AWS STS\n(validates JWT\nvia OIDC/JWKS)"]
    role["IAM Role\n(scoped to\nS3 PutObject)"]
    s3["S3 Bucket\n(dist-git cache)"]

    gitlab -- "id_token" --> sts
    sts -- "AssumeRoleWithWebIdentity" --> role
    role -- "temporary credentials" --> sts
    sts -- "credentials" --> gitlab
    gitlab -- "s3:PutObject" --> s3

Components

Cache Stack (s3-lookaside-cache)

Upload Role Stack (s3-lookaside-cache-upload-role)

Parameters

Cache Stack Parameters

Upload Role Parameters

S3 Key Structure

Files are stored using the dist-git lookaside path convention:

{namespace}/{package}/{filename}/{hashType}/{hash}/{filename}

Example:

rpms/tar/tar-1.35.tar.xz/sha512/abc123.../tar-1.35.tar.xz

Trust Policy

The upload role’s trust policy restricts access using three conditions (all StringEquals):

• Audience (gitlab.com:aud): Must be https://gitlab.com
• Subject (gitlab.com:sub): Must match project_path:<GitLabProjectPath>:ref_type:branch:ref:main
• Protected ref (gitlab.com:ref_protected): Must be "true"

Only the main branch of the configured project can assume the role.

GitLab CI Usage

Configure your .gitlab-ci.yml to assume the role using id_tokens. The AWS CLI automatically calls AssumeRoleWithWebIdentity when AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN are set:

upload to cache:
  image:
    name: amazon/aws-cli:latest
    entrypoint: [""]
  id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.com
  script:
    - set +x
    - printenv GITLAB_OIDC_TOKEN > /tmp/oidc-token
    - export AWS_WEB_IDENTITY_TOKEN_FILE=/tmp/oidc-token
    - export AWS_ROLE_ARN="arn:aws:iam::${AWS_ACCOUNT_ID}:role/${ROLE_NAME}"
    - export AWS_ROLE_SESSION_NAME="gitlab-ci-${CI_JOB_ID}"
    - aws s3 cp "$FILE" "s3://${BUCKET}/${S3_KEY}"
    - rm -f /tmp/oidc-token

The aud value in id_tokens must match the audience configured in the OIDC identity provider (https://gitlab.com). The token is written to a file because the AWS SDK reads it from AWS_WEB_IDENTITY_TOKEN_FILE rather than accepting it inline.

Security Features

• Encryption: AES256 server-side encryption with S3 bucket keys
• Public Access: All public access blocked on both buckets
• Transport Security: HTTPS enforced via bucket policy
• Object Lock: GOVERNANCE mode with 1-day default retention (will be increased in the future)
• CloudFront OAC: Modern Origin Access Control (not legacy OAI)
• Security Headers: HSTS, X-Frame-Options (DENY), X-Content-Type-Options, X-XSS-Protection, strict referrer policy
• TLS 1.2 Minimum: For all CloudFront connections
• OIDC Federation: Short-lived tokens instead of long-lived access keys

Backup Strategy

Continuous backup is enabled for point-in-time recovery.

Log Lifecycle

Access logs transition through storage tiers:

Outputs

Cache Stack Outputs

Upload Role Outputs

References

• GitLab: Configure OIDC in AWS
• AWS: Setting up OIDC with GitLab CI/CD
• AWS: Obtain OIDC provider thumbprint