stable URL: /l/tools-k8s-testing

K8s Integration Tests

Smoke tests that verify tools container images work in a real Kubernetes cluster. Each test creates short-lived pods, checks expected behavior, and cleans up. Tests run automatically on every MR via the Konflux K8s test pipeline.

Component Inventory

Component Test file What it tests
cloudwatch-log-forwarder tests-k8s.yml Python import
container-catalog tests-k8s.yml Bootstrap --help, grype/syft present
gitlab-ci tests-k8s.yml 13 tools present (shellcheck, go, kubectl, …)
hummingbird-agent tests-k8s.yml CLI --help, Python imports, kubectl present
hummingbird-cve-analysis tests-k8s.yml Python imports, git/rpm present
hummingbird-dashboard tests-k8s.yml Starts and serves on port 8080
hummingbird-status tests-k8s.yml Python import
hummingbird-tools tests-k8s.yml Python imports, psql/dnf present
kubernetes-event-forwarder tests-k8s.yml Python import
playwright-test tests-k8s.yml Node/npm/xvfb-run present

Components without tests: hummingbird-agent-vm-sandbox (bootc VM image, not a container workload), hummingbird-agent-vm-sandbox-disk (qcow2 extraction build, not a runtime image).

Test Runner

Tests are executed by ci/run_tests_k8s.sh. For each component it:

  1. Reads {component}/tests-k8s.yml
  2. Parses each test entry (YAML -> JSON via Python)
  3. Runs the command block with kubectl wired to the target cluster
  4. Cleans up labeled resources (hum-k8s-test=<run-id>) after each test

Running Locally

Test against a local cluster (kind, minikube, or remote):

# Single component with a published image
IMAGE_URL=quay.io/hummingbird-ci/gitlab-ci:latest \
IMAGE_NAME=gitlab-ci--tools \
  ci/run_tests_k8s.sh --context kind-kind gitlab-ci--tools

# Single component with a locally-built image
podman build -t localhost/hummingbird-dashboard:dev hummingbird-dashboard/
kind load docker-image localhost/hummingbird-dashboard:dev
IMAGE_URL=localhost/hummingbird-dashboard:dev \
IMAGE_NAME=hummingbird-dashboard--tools \
  ci/run_tests_k8s.sh --context kind-kind hummingbird-dashboard--tools

CI Integration

In Konflux, the tools-k8s-test IntegrationTestScenario triggers on every MR. The pipeline:

  1. Checks whether tests-k8s.yml exists for the changed component
  2. Provisions an ephemeral EaaS namespace
  3. Runs ci/run_tests_k8s.sh with the built image

Components without tests-k8s.yml skip the test (the pipeline exits early after the check step).

Adding Tests for a New Component

  1. Create {component}/tests-k8s.yml with one or more named tests:

    ---
smoke-test:
  command: |
    name="${TEST_GROUP}-smoke-${TEST_RUN_ID}"
    kubectl run "${name}" --image="${TEST_IMAGE:?}" --restart=Never \
      --labels="${TEST_RUN_LABEL}" \
      --command -- echo "hello"
    kubectl wait --for=jsonpath='{.status.phase}'=Succeeded \
      "pod/${name}" --timeout=120s || test_fail "Pod did not succeed"
    kubectl logs "${name}"

  2. Use ${TEST_IMAGE} for the image under test, ${TEST_RUN_ID} and ${TEST_RUN_LABEL} for unique naming and cleanup.

  3. Call test_fail "message" to fail a test with a clear message.

  4. Keep tests fast (under 2 minutes) — these are smoke tests, not integration suites.

Common pitfalls

  • All lines in command: | block scalars must be indented relative to the command key. Unindented lines break YAML parsing.
  • Use command -v instead of which to check for commands — which is not available in all container images.
  • For multi-statement Python one-liners, use semicolons on a single line: python3 -c 'import foo; import bar; print("ok")'

Debugging Failures

See the EaaS and Debugging guide for accessing ephemeral namespaces and using Kubearchive for historical PipelineRun data.