build_images.sh

Build container images using buildah with support for multiple architectures and container engines

Purpose

Build container images using buildah with support for multiple architectures and container engines.

Usage

Usage: ci/build_images.sh [OPTIONS] [GROUP_NAMES...] [-- BUILDAH_ARGS...]

OPTIONS:
    --verbose, -v        Enable verbose output during build
    --arch ARCHITECTURE  Specify target architecture (e.g., amd64, arm64, arm/v7)
    --engine ENGINE      Specify runtime engine for testing/export (podman or docker)
    --setup              Set up Docker-in-Docker environment before exporting
    --component-name NAME
                         Parse component name (format: group--distro--variant)
                         Example: curl--rawhide--default → curl/rawhide/default
    --local-rpms-dir DIR Directory containing custom RPMs to use during build.
                         These have a higher priority and thus override the
                         standard repositories.
    --build-deps         Build all dependencies of specified groups, but NOT the
                         specified groups themselves. Collects forward, reverse,
                         and transitive dependencies automatically.
    --pull               Pull images from registry instead of building locally.
                         Fast alternative for local development. Uses published
                         images from quay.io.
    --dryrun             Show what would be built (or pulled with --pull) without
                         actually building/pulling.
    --help, -h           Show this help message

Examples

# Build single image group (all distro/variants)
ci/build_images.sh nginx
ci/build_images.sh nodejs-20

# Build specific distro/variant only
ci/build_images.sh nginx/rawhide/builder
ci/build_images.sh nodejs-20/hummingbird/default

# Build multiple image groups at once
ci/build_images.sh nginx curl git
ci/build_images.sh nginx/rawhide/default curl/hummingbird/builder

# Build with options
ci/build_images.sh --verbose curl
ci/build_images.sh --arch arm64 nginx
ci/build_images.sh --engine docker nginx
ci/build_images.sh --engine podman --verbose nginx

# Build multiple image groups with options
ci/build_images.sh --verbose nginx curl git
ci/build_images.sh --arch arm64 nginx postgresql

# Build all dependencies of dotnet-runtime-10-0 (forward + reverse + transitive)
ci/build_images.sh --build-deps dotnet-runtime-10-0

# Pull dependencies from registry instead of building (faster for local dev)
ci/build_images.sh --build-deps --pull dotnet-runtime-10-0

# Pull a specific image from registry
ci/build_images.sh --pull nginx/rawhide/default

# Build from CI component name format (used in CI environments)
ci/build_images.sh --component-name curl--rawhide--default

# Build with custom RPMs (for testing modified packages)
ci/build_images.sh --local-rpms-dir ../rpms/builds/hostname/RPMS git/rawhide/builder

Note: Some images require git submodules initialized (use git init --recurse-submodules when cloning or git submodule update --init if already cloned). When building for foreign architectures, make sure qemu-user-static is available.

Dependency Building

The --build-deps flag builds all dependencies needed for testing an image, but not the image itself. This is used in CI to ensure all required images are available before running tests.

Pulling vs Building Dependencies

For local test development, use --pull with --build-deps to pull pre-built images from the registry instead of building them locally. This is much faster and uses the same dependency collection logic:

# Slow: Build all dependencies locally
ci/build_images.sh --build-deps nginx

# Fast: Pull all dependencies from registry
ci/build_images.sh --build-deps --pull nginx

The --pull flag works without --build-deps too:

# Pull a specific image instead of building it
ci/build_images.sh --pull caddy/rawhide/default

How Dependency Collection Works

When you run ci/build_images.sh --build-deps <image>, it performs a 2-level expansion:

Level 1: Collects forward and reverse dependencies of the specified image
- Forward dependencies: Images that the specified image’s tests depend on (detected by TEST_IMAGES[...] references in test files)
- Reverse dependencies: Images that depend on the specified image (filtered by reverse_dependency_tests: true in properties.yml)
Level 2: Collects forward dependencies of the reverse dependencies
Stops: No further expansion (avoids infinite graph traversal)

All dependencies are automatically deduplicated to ensure each image is built exactly once.

Relationship to Testing

The reverse_dependency_tests property in properties.yml affects both building and testing:

Build phase (--build-deps): Filters which reverse dependencies to build
Test phase (ci/run_tests_container.sh and ci/run_tests_k8s.sh with --include-reverse-deps): Filters which reverse dependencies to test

Set reverse_dependency_tests: false for images like curl that are used pervasively but don’t need reverse dependency workflows.

Dependency Building Examples

# Build dependencies for dotnet-runtime-10-0
# Builds: dotnet-sdk-10-0 (forward dependency)
ci/build_images.sh --build-deps dotnet-runtime-10-0

# Pull dependencies for dotnet-runtime-10-0 (faster alternative)
# Pulls: dotnet-sdk-10-0 from quay.io/hummingbird-rawhide
ci/build_images.sh --build-deps --pull dotnet-runtime-10-0

# Build dependencies for core-runtime
# Builds: xcaddy, go, rust (reverse deps with reverse_dependency_tests: true)
# Plus their forward dependencies
ci/build_images.sh --build-deps core-runtime

# Use --dryrun to see what would be built/pulled without actually doing it
ci/build_images.sh --dryrun --build-deps core-runtime
ci/build_images.sh --dryrun --build-deps --pull core-runtime

Building with custom RPMs

The --local-rpms-dir option enables testing container images with custom-built RPM packages, for iterating on package changes before committing to the RPM repository.

Workflow

Build custom RPMs in the rpms repository (see its documentation for the complete workflow):

cd ../rpms
# modify a package
ci/build_rpms.sh packagename
# Built RPMs will be in builds/packagename/RPMS/

Build container image using the custom RPMs:

cd ../containers
ci/build_images.sh --local-rpms-dir ../rpms/builds/packagename/RPMS imagename/builder

Verify the custom package was installed:

podman run --rm --entrypoint '' quay.io/hummingbird/imagename:latest-builder rpm -qa