run_tests_k8s.sh
Purpose
Run K8s tests for image groups using kubectl. Tests execute in a real Kubernetes environment, validating that images work correctly in Kubernetes.
Usage
Run ci/run_tests_k8s.sh --help for full usage information.
Usage: ci/run_tests_k8s.sh [OPTIONS] [GROUP_NAMES...]
OPTIONS:
--context CONTEXT K8s context to use (REQUIRED unless --kubeconfig set)
--kubeconfig FILE Path to kubeconfig file (REQUIRED unless --context set)
--push-image IMAGE Push local image to OpenShift internal registry before testing
Sets TEST_IMAGE to the internal registry reference
--verbose, -v Show full output for passing tests
--pause, -p Pause on test failure before cleanup to allow debugging
--component-name NAME
Parse component name (format: group--distro--variant)
Example: curl--rawhide--default → curl/rawhide/default
--group-component-name NAME
Parse component name and add /group variant
Example: curl--rawhide--default → curl/rawhide/group
--include-reverse-deps
Also run tests that use the specified image variants
--output FILE Write JSON test results to FILE (for CI integration)
When set, exits 0 after writing results regardless of test outcome
--help, -h Show this help message
Safety: The script requires explicit --context or --kubeconfig to prevent
accidental operations on production clusters.
Note: If no distro/variant is specified, all combinations will be tested. To test
only a specific variant, use <group_name>/<distro>/<variant>. To test only a
specific test, use <group_name>/<distro>/<variant>/<test>.
Local Development Workflow
For local development, build images and push them to the internal registry:
# Build the image locally
ci/build_images.sh nginx/hummingbird/default
podman tag registry.access.redhat.com/hi/nginx:latest my-nginx:dev
# Push to internal registry and test
ci/run_tests_k8s.sh --context mpp-preprod --push-image my-nginx:dev nginx/hummingbird/default
The --push-image flag:
- Compares local image digest with remote
- Skips push if image already exists with same digest
- Uses port-forward to registry-proxy for pushing
- Sets
TEST_IMAGEto the internal registry reference
Prerequisites:
oc loginto the target clusteroc project <namespace>to set the target namespace- Port-forward access to
registry-proxyinhummingbird--internal
Testing with Published Images
Test published images without building locally:
# Test using published images (resolved via IMAGE_URL/IMAGE_NAME)
IMAGE_URL=registry.access.redhat.com/hi/nginx:latest \
IMAGE_NAME=nginx--hummingbird--default \
ci/run_tests_k8s.sh --context mpp-preprod nginx/hummingbird/default
Reverse dependency selection
--include-reverse-deps uses ci/internal/image_test_plan.py, shared with the
container runner, to select dependent tests individually. It reads Kubernetes
commands from tests-k8s.yml and follows shell scripts and sourced fixtures for
TEST_IMAGES[group/variant] references. Container-only references do not select
Kubernetes tests.
ci/run_tests_k8s.sh --context kind-kind --include-reverse-deps nginx/hummingbird/default
This selects all tests for the requested nginx variant and only dependent tests
that use nginx/default in the same distro. For example, Python custom CA tests
run, but standalone Python version checks do not. Explicit test selectors remain
restricted to the requested test. reverse_dependency_tests: false disables
selection of consumers of that image.
--include-reverse-deps runs the planner in single-component mode: reverse
dependency tests that need a same-group sibling variant are excluded. Such a
test references TEST_IMAGES[<same-group>/variant] for a variant other than the
one under test (or is itself defined for a same-group variant), but a per-component
run provides only the image under test. The sibling would fall back to a released
image — silently testing the changed component against an older sibling — or fail to
pull when no release exists. Those combinations run in the group pipelines (pr group), which provide every variant of the group at the exact commit. Cross-group
reverse dependencies are still selected, and tests of the requested variant itself
are unaffected.
Kubernetes variants lists match exact variant names. Tests without a list apply
to all image variants except group. Tests with variants: [group] run as group
tests. The planner does not apply container engine or host FIPS filters to
Kubernetes definitions, or infer the cluster architecture from the local host.
Environment Variables
Tests have access to these environment variables:
| Variable | Description |
|---|---|
TEST_IMAGE |
Container image under test (with digest) |
TEST_IMAGES |
Associative array with group/variant image URLs (uses TEST_DISTRO context) |
TEST_IMAGES_PATH |
Path to file containing serialized TEST_IMAGES array |
TEST_GROUP |
The image group being tested |
TEST_DISTRO |
The distro being tested (e.g., rawhide) |
TEST_VARIANT |
The variant being tested (e.g., default) |
TEST_VERBOSE |
Show test command output (true or false) |
TEST_RUN_ID |
Unique ID for this test run (for resource naming) |
TEST_RUN_LABEL |
Label selector for cleanup (hum-k8s-test=<id>) |
Cluster Access
The kubectl command is pre-configured with the context/kubeconfig from CLI args,
so tests can use it directly without additional configuration.
Helper Function
| Function | Description |
|---|---|
test_fail |
Fail the test with a custom error message |
Examples
# Test single group with explicit context (all distro/variants)
ci/run_tests_k8s.sh --context mpp-preprod nginx
# Test specific distro/variant
ci/run_tests_k8s.sh --context mpp-preprod nginx/hummingbird/default
# Test specific test
ci/run_tests_k8s.sh --context mpp-preprod nginx/hummingbird/default/readiness-probe
# Test with verbose output for debugging
ci/run_tests_k8s.sh --verbose --context mpp-preprod nginx
# Test with kubeconfig file (CI environments)
ci/run_tests_k8s.sh --kubeconfig /workspace/kubeconfig nginx/hummingbird/default
# Build locally and push to internal registry
ci/build_images.sh nginx/hummingbird/default
podman tag registry.access.redhat.com/hi/nginx:latest my-nginx:dev
ci/run_tests_k8s.sh --context mpp-preprod --push-image my-nginx:dev nginx/hummingbird/default
# Test from CI component name format
ci/run_tests_k8s.sh --kubeconfig /workspace/kubeconfig --component-name nginx--hummingbird--default
# Write JSON results for CI integration
ci/run_tests_k8s.sh --output /tmp/results.json --kubeconfig /workspace/kubeconfig nginx/hummingbird/default
Resource Cleanup
Tests should label resources with TEST_RUN_LABEL for automatic cleanup:
kubectl create configmap my-config --from-literal=key=value
kubectl label configmap my-config "${TEST_RUN_LABEL}"
The test runner automatically cleans up all resources with the test run label after each test and at script exit.
CI Integration
In CI environments (Konflux), the script receives:
--kubeconfigpointing to the ephemeral namespace kubeconfig--component-nameor--group-component-namefor component identification--outputfor structured JSON results
The --output flag writes results in Konflux-compatible format and exits 0,
allowing the pipeline to read results without relying on exit codes.