# Hummingbird Agent Design

LLMS index: [llms.txt](https://hummingbird-project.io/llms.txt) | Full content: [llms-full.txt](https://hummingbird-project.io/llms-full.txt)

---

Architectural design document for the hummingbird-agent. Covers the reasoning
behind every major design choice so that future changes can be made safely,
without accidentally violating invariants that hold the system together.

For operational usage (CLI, config reference, deployment), see
[Hummingbird Agent](https://hummingbird-project.io/l/hummingbird-agent). For agent turn flow and tool
handling, see [Agent Model Loop](https://hummingbird-project.io/docs/background/tools/hummingbird-agent/model-loop/). Provider
adapters and wire formats are documented in [Hummingbird AI](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/).

## 1. Design Philosophy

Five principles shaped the agent's architecture. Every component traces back
to at least one of these.

**Security by default.** The agent processes untrusted merge requests. All
LLM-driven commands run inside isolated sandbox containers with no network
and no credentials. Orchestrator write tokens and model read tokens live in
separate code paths that never cross. No cluster-admin is required.

**Config over code.** Investigation logic lives in markdown workflow files
that become the LLM system prompt verbatim -- changing the analysis strategy
requires no code change and no redeployment. Operational settings (data
sources, project allowlists, iteration limits) live in a YAML config file
that is auditable and committable. Secrets are the only thing in environment
variables.

**Bounded cost.** Every output path has a size cap. Large tool outputs are
auto-spilled to sandbox files with only a preview returned to the model. Each
session has both an iteration limit and a per-call context token limit with
two-tier escalation (soft warning, then hard stop with tool disabling). The
relationship between these constants is documented and centralized.

**Partial over nothing.** When some data is unavailable (expired pod logs,
unreachable Konflux cluster, Testing Farm 404), the agent continues with
whatever data it has and notes the gap in its output. A partial report is
more valuable than a crash.

**Model-agnostic agent loop.** The agent loop (`agent.py`) does not inspect
the internal structure of the `contents` list. It appends `raw_content` from
model responses and `make_tool_responses()` output without looking inside.
Each model backend owns its wire format. This makes it possible to add new
model backends (Claude, GPT) without touching the agent loop.

## 2. System Architecture

### 2.1. Component overview

```mermaid
flowchart TB
    subgraph input [Event Sources]
        SQS["SQS Queue<br/>(gitlab::pipeline, gitlab::note)"]
        CLI["CLI<br/>(--event / --event-file)"]
    end

    subgraph orchestrator [Orchestrator Process]
        Events["events.py<br/>SQS consumer"]
        Runner["runner.py<br/>Event routing"]
        Agent["agent.py<br/>Model loop"]
        Tools["tools.py<br/>Tool registry"]
        Actions["actions.py<br/>GitLab notes"]
        Sessions["sessions.py<br/>Persistence"]
        WfConfig["workflow_config.py<br/>YAML config"]
    end

    subgraph ai [Shared AI Library]
        Models["hummingbird-ai<br/>Gemini / Anthropic adapters"]
    end

    subgraph sandbox [Sandbox Container]
        SB["PodmanSandbox / K8sSandbox / KubeVirtSandbox<br/>jq, python3, yq"]
        SpillFiles["/tmp/data/_out/ spill files"]
        DataFiles["/tmp/data/ working files"]
    end

    subgraph dataSources [Data Source Modules]
        GL["gitlab.py"]
        KX["konflux.py"]
        TF["testing_farm.py"]
    end

    subgraph external [External APIs]
        GitLabAPI["GitLab API"]
        KonfluxAPI["K8s + Kubearchive"]
        TFAPI["Testing Farm API"]
    end

    subgraph storage [Storage]
        S3["S3 Sessions"]
        MRNote["GitLab MR Notes"]
    end

    SQS --> Events --> Runner
    CLI --> Runner
    Runner --> Agent
    Agent --> Tools
    Tools --> SB
    Tools --> dataSources
    dataSources --> external
    Agent --> Models
    Runner --> Actions --> MRNote
    Runner --> Sessions --> S3
    Runner --> WfConfig
```

### 2.2. Request lifecycle

A complete run proceeds through these stages:

1. **Event arrival.** An SQS message (production) or CLI invocation (dev)
   provides a GitLab project path and MR IID.

2. **Config lookup.** `workflow_config.py` maps the project to one or more
   workflows via the `project_index`. Each match yields a `WorkflowConfig`
   and `ProjectEntry` with data source declarations and per-project settings.

3. **Rate-limit check and SHA dedup.** `actions.scan_agent_threads()` scans
   MR discussions in a single pass, parsing JSON session markers to determine
   the per-workflow thread count and whether the current SHA has already been
   reviewed. If the SHA was already reviewed, the workflow is skipped. If the
   thread count meets or exceeds `max_runs_per_mr`, the workflow is skipped.
   Slash commands and replies bypass this check.

4. **Placeholder note.** `actions.create_placeholder_note()` posts a
   placeholder so the reviewer knows analysis is in progress. The note
   contains a JSON session marker
   (`<!-- hummingbird-session: {"id":"UUID","wf":"name","sha":"abc"} -->`)
   for rate limiting, SHA dedup, and session resumption.

5. **Sandbox start.** `sandbox.create_sandbox()` starts a Podman container
   or K8s pod. `/tmp/data/` is pre-created for the model's use.

6. **Data source registration.** `data_sources.register_selected()` resolves
   tokens and URLs from the config and registers tool functions on the
   `ToolRegistry`. Only data sources declared in the workflow config are
   registered.

7. **Agent loop.** `agent.run_agent_loop()` runs the model loop: the
   workflow markdown becomes the system prompt, tool definitions are provided,
   and the model iterates calling tools and producing text until it emits a
   final response or hits a budget limit.

8. **Note update.** The final text, wrapped with a session marker and reply
   prompt, replaces the placeholder note.

9. **Session save.** Conversation history (`contents`), transcript, and
   sandbox archive are saved to S3 (production) or a local directory (dev).

10. **Sandbox cleanup.** The container/pod is deleted. K8s pods also have a
    configurable `activeDeadlineSeconds` backstop (default 1800s) in case the
    orchestrator dies.

### 2.3. Architectural boundaries

The codebase is organized around four strict boundaries:

**Runner** (`runner.py`) is the orchestration layer. It owns event routing,
the placeholder/update note lifecycle, session save/load, and sandbox
lifecycle. It calls `agent.run_agent_loop()` but never reaches into the
agent's internals.

**Agent** (`agent.py`) is the model loop. It knows about the model interface,
tool definitions, and the `contents` list, but nothing about GitLab, SQS,
sessions, or actions. It returns `(text, usage, transcript, contents)` and
is completely unaware of what happens with those values.

**Tools** (`tools.py`) bridge the agent and the sandbox/data-sources. The
agent calls `tool_registry.execute(tool_call)` and gets a dict back. It
never calls sandbox methods directly. This indirection is what enables
auto-spill: the tool registry can transparently save large outputs to
sandbox files and return previews.

**Model providers** are supplied by the shared
[`hummingbird-ai` library](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/),
which owns adapter construction,
provider formats, retries, and AI usage accounting. The agent loop uses the
library's normalized adapter interface and treats conversation content as
opaque.

## 3. Module Architecture

### 3.1. Dependency graph

```mermaid
flowchart TD
    main["__main__.py"]
    config_mod["config.py"]
    wf_config["workflow_config.py"]
    events_mod["events.py"]
    runner_mod["runner.py"]
    agent_mod["agent.py"]
    tools_mod["tools.py"]
    sandbox_mod["sandbox.py"]
    actions_mod["actions.py"]
    sessions_mod["sessions.py"]
    transcript_mod["transcript.py"]
    ds_init["data_sources/__init__.py"]
    ds_gitlab["data_sources/gitlab.py"]
    ds_konflux["data_sources/konflux.py"]
    ds_tf["data_sources/testing_farm.py"]
    config_watch_mod["config_watch.py"]
    ai_pkg["hummingbird-ai library"]

    main --> config_mod
    main --> config_watch_mod
    main --> wf_config
    main --> events_mod
    main --> runner_mod
    main --> sessions_mod
    main --> transcript_mod
    main --> actions_mod

    config_watch_mod --> config_mod
    config_watch_mod --> wf_config

    runner_mod --> actions_mod
    runner_mod --> agent_mod
    runner_mod --> config_mod
    runner_mod --> ds_init
    runner_mod --> sandbox_mod
    runner_mod --> sessions_mod
    runner_mod --> tools_mod
    runner_mod --> ai_pkg
    runner_mod --> wf_config
    runner_mod --> transcript_mod

    agent_mod --> config_mod
    agent_mod --> ai_pkg
    agent_mod --> tools_mod

    tools_mod --> config_mod
    tools_mod --> ai_pkg
    tools_mod --> sandbox_mod

    ds_init --> tools_mod
    ds_init --> wf_config
    ds_init --> ds_gitlab
    ds_init --> ds_konflux
    ds_init --> ds_tf

    ds_gitlab --> ai_pkg
    ds_konflux --> ai_pkg
    ds_tf --> ai_pkg

    sessions_mod --> sandbox_mod

```

### 3.2. Module responsibilities

Each module has a single, well-defined responsibility. The boundary rules
below are invariants -- violating them breaks the security model or the
separation of concerns.

#### `config.py` -- Runtime configuration and constants

- **Owns:** `Config` dataclass and agent tuning constants
  (`DEFAULT_MAX_ITERATIONS`, `CONTEXT_LIMIT`, `OUTPUT_PREVIEW_BYTES`, etc.)
- **Boundary:** Pure data. No I/O except reading env vars in `Config.load()`.
  No imports from other Hummingbird agent modules; model setup and pricing are
  owned by [`hummingbird-ai`](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/).
- **Invariant:** All interdependent budget/spill constants must be defined
  here with their relationship documented in the comment block.

#### `workflow_config.py` -- YAML config loader and token resolution

- **Owns:** `AgentConfig`, `WorkflowConfig` (with `trigger`, `description`,
  `trigger_rules`, `ignore_users`, `ignore_branches` fields), `TriggerRule`,
  `ProjectEntry`, `DataSourceConfig` dataclasses; `load()`, `validate_env()`,
  `evaluate_trigger_rules()`, `resolve_tool_token()`, `resolve_cluster_url()`,
  `get_prompt()`.
- **Boundary:** Reads YAML from disk and env vars. Never instantiates
  network clients or model objects.
- **Invariant:** `resolve_tool_token()` resolves model tool tokens only.
  It must never return orchestrator tokens. The resolution chain is:
  `project_entry.tokens[ds] -> workflow_cfg.data_sources[ds].token_env ->
"" (empty)`.

#### `events.py` -- SQS consumer

- **Owns:** `decode_sns_message()` (SNS envelope decoding),
  `poll_loop()` (blocking SQS consumer with concurrency control).
- **Boundary:** Knows about SQS/SNS wire formats. Calls a generic
  `EventHandler` callback. Does not know about GitLab, workflows, or the
  agent.
- **Invariant:** Failed messages are not deleted from SQS (they go to
  the DLQ after visibility timeout expires). Successful messages are
  deleted after handler returns.

#### `config_watch.py` -- Background config hot-reload

- **Owns:** `ConfigHolder` (thread-safe config pair with atomic swap),
  `start_watcher()` (daemon thread that polls config file mtime),
  `_watch_loop()`.
- **Boundary:** Knows about `config.Config.load()` and
  `workflow_config.load()`. Does not know about events, the agent, or
  any runtime state.
- **Invariant:** On reload failure (parse error, missing env var), the
  previous config is kept and a warning is logged. The watcher never
  crashes the serve loop.

#### `runner.py` -- Event routing and workflow execution

- **Owns:** `handle_event()`, `handle_pipeline()`, `handle_merge_request()`,
  `handle_note()`, `_execute_workflow()`, `_handle_reply()`, `run_workflow()`,
  `_acquire_sandbox()`, `_resolve_slash_workflow()`, `_format_help()`,
  `_is_ignored_user()`, `WorkflowRequest`, `WorkflowResult`, `SandboxOpts`.
- **Boundary:** Orchestrates everything: config lookup, rate limiting,
  placeholder notes, sandbox lifecycle, agent invocation, session save.
  This is the only module that touches both `actions` and `agent`.
- **Invariant:** `run_workflow()` either lingers the sandbox (success)
  or cleans it up (exception).

#### `agent.py` -- Model-agnostic tool-calling loop

- **Owns:** `run_agent_loop()`, system prompt constants (`BASE_SYSTEM_PROMPT`,
  `CONTINUATION_PROMPT`, warning/nudge strings), budget logic.
- **Boundary:** Uses the `ModelAdapter` interface from
  [`hummingbird-ai`](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/) and
  `tools` (for `execute()`). Does not
  know about GitLab, SQS, sessions, or actions, or which backend is in use.
- **Invariant:** The `contents` list is treated as opaque. Items are
  appended via `response.raw_content` and `model.make_tool_responses()`.
  The agent never inspects, modifies, or deletes items in the list (except
  for empty response retries, which `pop()` the last appended item before
  the model has seen any tool results for it).

#### `tools.py` -- Tool registry

- **Owns:** `ToolRegistry` class with `sandbox_exec`, `fetch_to_sandbox`,
  `fetch_batch_to_sandbox` built-in tools; data source registration and
  dispatch; auto-spill logic.
- **Boundary:** Owns the sandbox reference and all tool execution. The
  agent never touches the sandbox directly.
- **Invariant:** `_spill_counter` is shared across all spill paths
  (sandbox exec stdout, sandbox exec stderr, data source auto-spill) to
  prevent filename collisions in `/tmp/data/_out/`.

#### `sandbox.py` -- Sandbox backends

- **Owns:** `Sandbox` protocol, `PodmanSandbox`, `K8sSandbox`,
  `K8sPoolSandbox`, `SandboxPool`, `KubeVirtSandbox`, `VmiPool`,
  `KubeVirtPoolSandbox`, `create_sandbox()`, `ExecResult`.
- **Boundary:** Knows about container/pod/VM lifecycle and command execution.
  Does not know about tools, models, or the agent.
- **Invariant:** All backends must implement the same 8-method protocol
  (`start`, `exec`, `write_file`, `stream_to_file`, `stream_exec`,
  `read_file_iter`, `cleanup`, `linger`). `exec()` accepts optional
  `stdin_data` for piping raw bytes. All must pre-create `/tmp/data/` in
  `start()`. All must use `stdin` piping for `write_file()` (never host
  volume mounts).

#### `actions.py` -- GitLab note lifecycle

- **Owns:** Orchestrator token resolution (`resolve_orchestrator_token()`),
  workflow action token client creation (`_make_client()`), bot user ID
  collection (`collect_workflow_bot_ids()`), note CRUD, JSON marker parsing
  (`marker_tag`, `parse_marker`), `scan_agent_threads` (per-workflow rate
  limiting + SHA dedup), `post_simple_reply`, member access checks,
  session-for-reply lookup, `AgentThreadInfo`, `SessionRef`.
- **Boundary:** Functions are split by token tier:
  - _Orchestrator token_ (operational reads + infrastructure notes, use
    `_get_client`): `check_member_access`, `get_latest_push_author`,
    `get_head_pipeline_id`, `is_first_note_in_discussion`,
    `reply_access_denied`, `notify_rate_limit`, `post_simple_reply`.
  - _Workflow action token_ (per-workflow-attributable writes, take explicit
    `token: str`): `create_placeholder_discussion`, `post_discussion_note`,
    `resolve_agent_threads`, `scan_agent_threads`.
    Never touches model tool tokens, the tool registry, or the agent.
- **Invariant:** Orchestrator tokens are resolved from `ORCHESTRATOR_*` env
  vars only. Workflow action tokens are resolved from `action_tokens` in YAML
  and passed as explicit parameters. Neither tier is exposed to the model.

#### `sessions.py` -- Session persistence

- **Owns:** `archive_sandbox()`, `save_local()`, `save_s3()`, `load_s3()`,
  `load_local()`, `restore_sandbox()`, `SessionData`.
- **Boundary:** Knows about sandbox (for archiving) and S3/filesystem (for
  storage). Does not know about the agent, tools, or models. Load/save uses
  canonical envelopes with `format_version: 1`; unversioned legacy sessions
  are not supported.
- **Invariant:** The sandbox archive is never extracted on the orchestrator.
  It is created inside the sandbox (`tar czf`), streamed out via
  `read_file_iter()`, and restored inside a new sandbox via
  `stream_exec("tar xzf -")`.

#### `transcript.py` -- Transcript rendering

- **Owns:** `render_markdown()`, per-tool rendering functions, truncation.
- **Boundary:** Pure transformation from `TranscriptEntry` list to markdown.
  No I/O, no side effects.

#### `data_sources/__init__.py` -- Data source registration

- **Owns:** `register_selected()` -- resolves tokens/URLs from config and
  calls each data source's `register()` function.
- **Boundary:** Bridges `workflow_config` (for token/URL resolution) and
  `tools` (for registration). Only registers data sources declared in the
  workflow config.

#### `data_sources/gitlab.py`, `konflux.py`, `testing_farm.py`

- **Own:** `register()` function that creates tool definitions and closures
  over credentials, and registers them on the `ToolRegistry`.
- **Boundary:** Each module talks to one external API. Credentials are
  captured at registration time via closure, never stored globally.

## 4. Security Model

The agent runs in a shared OpenShift cluster without cluster-admin access,
processing merge requests from repositories where external contributors can
submit code. The security model addresses two threat vectors: (1) the LLM
executing arbitrary commands chosen by an attacker-controlled MR, and (2)
credential leakage between the model, the sandbox, and orchestrator actions.

### 4.1. Sandbox isolation

All commands generated by the LLM run inside an ephemeral container, never
on the orchestrator host. The sandbox has no credentials, no network, and no
visibility into the orchestrator.

```mermaid
flowchart LR
    subgraph orchestrator [Orchestrator]
        Agent["Agent Loop"]
        Creds["Secrets<br/>(tokens, kubeconfig)"]
    end
    subgraph sandbox [Sandbox Container]
        Shell["sh -c commands"]
        Files["/tmp/data/<br/>(incl. _out/ spill dir)"]
    end
    Agent -->|"stdin pipe<br/>(write_file)"| sandbox
    Agent -->|"exec command"| sandbox
    sandbox -->|"stdout/stderr"| Agent
    sandbox -.-x|"NO network"| Internet["Internet / K8s API"]
    sandbox -.-x|"NO access"| Creds
```

**Podman (local development):**

- `--network=none` -- complete network isolation; curl, wget, pip install all
  fail
- `--user 65532` -- fixed non-root UID; no privilege escalation
- No host volume mounts -- data enters only via `stdin` piping through
  `write_file()`

**Kubernetes (production):**

Every field in the pod manifest is set explicitly for portability to vanilla
Kubernetes with Pod Security Admission (restricted level), not just reliance
on OpenShift's `restricted-v2` SCC admission:

- `automountServiceAccountToken: false` -- no K8s API access from sandbox
- `runAsNonRoot: true` -- enforced at pod level
- `seccompProfile: RuntimeDefault` -- required by restricted level
- `allowPrivilegeEscalation: false` -- container level
- `capabilities.drop: ["ALL"]` -- container level
- `activeDeadlineSeconds: 1800` -- pod self-terminates after 30 minutes even
  if the orchestrator crashes; prevents orphaned pods
- `restartPolicy: Never` -- pod does not restart on failure
- `NetworkPolicy` on the sandbox namespace blocks all egress from all pods
  in the namespace (`podSelector: {}`, `egress: []`)

The pod does not set `runAsUser` explicitly. On OpenShift, the SCC assigns a
UID from the namespace range. On vanilla K8s, the image's USER directive is
used.

### 4.2. Credential separation (four-tier token model)

Tokens are split into four tiers with strict code-path separation:

```mermaid
flowchart TB
    subgraph tier1 [Tier 1: Orchestrator Tokens]
        OT["ORCHESTRATOR_GITLAB_TOKEN_*<br/>Operational reads<br/>Env vars only, never in YAML"]
    end
    subgraph tier2 [Tier 2: Workflow Action Tokens]
        AT["action_tokens in YAML<br/>Per-workflow bot identity<br/>Write scope (api)"]
    end
    subgraph tier3 [Tier 3: Model Tool Tokens]
        MT["GITLAB_TOKEN_RO, etc.<br/>Read scope (read_api)<br/>Env var NAMES in YAML"]
    end
    subgraph tier4 [Tier 4: Sandbox]
        SB["Zero credentials<br/>Zero network<br/>Zero SA token"]
    end

    OT -->|"used by"| OpsReads["actions.py<br/>(member access, push author,<br/>rate-limit notices)"]
    AT -->|"used by"| WfWrites["actions.py<br/>(notes, thread resolution)"]
    MT -->|"used by"| DataSources["data_sources/<br/>(GitLab, Konflux, TF)"]
    SB -->|"used by"| SandboxExec["sandbox_exec<br/>(sh -c commands)"]

    OpsReads -.-x|"NEVER"| DataSources
    WfWrites -.-x|"NEVER"| DataSources
    WfWrites -.-x|"NEVER"| SandboxExec
    DataSources -.-x|"NEVER"| OpsReads
    DataSources -.-x|"NEVER"| WfWrites
```

**Tier 1 -- Orchestrator tokens.** Tokens used by `actions.py` for
operational reads (member access checks, push author lookup, head pipeline
queries) and infrastructure notes (access-denied replies, rate-limit
notices). Resolved by convention from env vars:
`ORCHESTRATOR_GITLAB_TOKEN_<MANGLED_PROJECT>` (per-project) or
`ORCHESTRATOR_GITLAB_TOKEN` (global fallback). The `ORCHESTRATOR_` prefix is
a structural safeguard -- these env var names can never appear in the YAML
config's `data_sources`, `tokens`, or `action_tokens` sections. These tokens
can be scoped to `read_api` since all per-workflow writes moved to Tier 2.

**Tier 2 -- Workflow action tokens.** Per-workflow bot identity tokens
(`api` scope) used by `actions.py` for all workflow-attributable GitLab
writes: placeholder notes, discussion notes, and thread resolution. Each
workflow gets a dedicated bot user per project, providing clear audit trails
for which agent produced each note. Declared in the YAML config as env var
_names_ at the project level:

```yaml
projects:
  redhat/hummingbird/containers:
    action_tokens:
      gitlab: HUMMINGBIRD_AGENT_ACTION_CODE_REVIEW_GITLAB_TOKEN_CONTAINERS
```

Resolved by `resolve_action_token(project_entry, ds_name)` which reads the
env var name from `action_tokens` and resolves it from `os.environ`. These
tokens are passed as explicit `token: str` parameters to write functions
in `actions.py`. They NEVER enter the `ToolRegistry` or model tool calls.

At startup, `collect_workflow_bot_ids()` authenticates each unique workflow
token to build a set of all workflow bot user IDs. This set is used by
`handle_note` for self-filtering (skipping notes from any workflow bot)
and by `find_session_for_reply` for session marker scanning.

The naming convention groups tokens alphabetically:
`HUMMINGBIRD_AGENT_ACTION_<WORKFLOW>_GITLAB_TOKEN_<PROJECT>`.

**Tier 3 -- Model tool tokens.** Read-only tokens (`read_api` scope for
GitLab) used by data source modules during tool execution. Declared in the
YAML config as env var _names_ (not values):

```yaml
data_sources:
  gitlab:
    token_env: GITLAB_TOKEN_RO # name of the env var
```

Per-project overrides are possible:

```yaml
projects:
  redhat/hummingbird/containers:
    tokens:
      gitlab: GITLAB_TOKEN_CONTAINERS_RO # overrides token_env for this project
```

The resolution chain in `resolve_tool_token()` is:
`project_entry.tokens[ds] -> workflow_cfg.data_sources[ds].token_env -> ""`

Storing env var _names_ (not values) in YAML means the config file is safe
to commit and audit. Actual secret values live in environment variables,
injected via K8s Secrets at deployment time.

**Tier 4 -- Sandbox.** The sandbox container has zero credentials, zero
network access, and `automountServiceAccountToken: false` (no K8s API
access). Data enters the sandbox only via `write_file()` (stdin piping).
The model cannot instruct the sandbox to reach external APIs -- it must use
the orchestrator's data source tools.

### 4.3. Namespace separation

Sandbox pods run in a dedicated namespace (`hummingbird--agent-sandbox`),
separate from the orchestrator namespace (`hummingbird--internal`). This
limits blast radius: even if a sandbox pod is compromised, it has no
visibility into the orchestrator's Secrets, Pods, or ServiceAccount tokens.

**RBAC setup:**

The orchestrator's ServiceAccount gets a Role in the _sandbox_ namespace
(not its own namespace) granting only:

- `pods: create, get, list, delete, patch` -- sandbox pod lifecycle and pool claims
- `pods/exec: create` -- command execution via `kubectl exec`

No CRDs, no custom runtimes, no cluster-scoped resources. The orchestrator
needs only namespace-scoped permissions, so it works with standard
OpenShift RBAC without requesting cluster-admin.

Konflux data is fetched via bearer token from kubeconfig credentials, not
from inside the cluster. The orchestrator's ServiceAccount does not need
access to Konflux namespaces.

**NetworkPolicy:**

A deny-all-egress NetworkPolicy in the sandbox namespace uses `podSelector: {}`
to match all pods and sets `egress: []`. The sandbox cannot reach the internet,
the K8s API, or other pods in the cluster.

### 4.4. Security invariants

These must hold for the security model to be effective. Any change that
violates one of these is a security regression:

1. **Orchestrator tokens must NEVER flow into `ToolRegistry`,
   `data_sources`, or model `contents`.** They are resolved in `actions.py`
   only.

2. **Workflow action tokens must NEVER flow into `ToolRegistry`,
   `data_sources`, or model `contents`.** They are passed as explicit
   `token: str` parameters within `actions.py` and `runner.py` only.

3. **Model tool tokens must NEVER flow into `actions.py`.** They are
   resolved in `workflow_config.py` and consumed in `data_sources/`.

4. **The sandbox must NEVER have network access or credentials.** No host
   mounts, no SA token, no env vars with secrets.

5. **The sandbox archive is NEVER extracted on the orchestrator.** It is
   created inside one sandbox and restored inside another. The orchestrator
   only transports the bytes.

6. **No cluster-admin required.** Only namespace-scoped resources (Role,
   RoleBinding, Pod, NetworkPolicy) are used.

7. **Agent-generated notes are skipped on re-processing.** Notes containing
   `SESSION_MARKER_PREFIX` are filtered out in `handle_note()` to prevent
   infinite loops. Self-filtering checks against all workflow bot user IDs
   (collected at startup) plus the orchestrator bot ID.

8. **All auto-triggers require Developer access.** `handle_pipeline()`,
   `handle_merge_request()`, and `handle_note()` each check
   `check_member_access()` on the event's user before executing a workflow.
   Events from non-developers are silently skipped (or replied to with an
   access-denied message for slash commands). This ensures that the _target
   MR's_ work products -- its diff, description, CI logs, and commit
   messages -- originate from a trusted author. Fork MRs from external
   contributors are blocked because the MR author lacks Developer+ on the
   target project.

   **Scope limitation:** this gate only covers the target MR. Once the
   agent is running, the model controls tool arguments and can direct tools
   at content beyond the target MR -- other MRs, other refs, other job
   IDs, even other projects reachable by the read-only token. The current
   mitigations are: (a) workflow prompts instruct the model to use the
   event's `{project, iid, sha}`, (b) the model would need to be
   manipulated via prompt injection from already-trusted content to deviate,
   (c) the sandbox prevents the model from acting on manipulated reasoning
   beyond producing text output, and (d) the token is read-only with
   minimal scope. However, if a data source tool is added that returns
   user-authored prose from arbitrary resources (e.g. issue bodies, wiki
   pages), it should apply per-author trust filtering (invariant #10).

9. **Data source tools that accept URLs must validate them against an
   allowlist.** `tf_get_test_log` restricts URLs to the Testing Farm
   artifacts prefix to prevent the LLM from directing the orchestrator to
   fetch arbitrary URLs (SSRF).

10. **Third-party commentary entering the model prompt must be
    trust-filtered.** Data sources that feed text from users _other than the
    event trigger_ into `contents` (discussion comments, issue bodies) must
    gate on project membership at Developer+ level per author. Content from
    untrusted authors must be redacted to a fixed placeholder, never
    sanitized or escaped -- there is no reliable way to escape adversarial
    text for an LLM prompt. Discussions where _all_ notes are untrusted must
    be dropped entirely. See
    [§9.1](https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-runtime/#91-gitlab) for the reference
    implementation in `gitlab_get_mr_discussions`.

This invariant covers _commentary_ (what other people said about the
MR), not _work products_ (the diff, CI logs, commit messages). Work
products are inherently the input to the agent's analysis and cannot be
content-filtered without defeating the agent's purpose. For the target
MR, their trust comes from invariant #8 (the event trigger is
Developer+). For content the model fetches beyond the target MR, trust
depends on the tool: discussion tools must filter per-author (#10), while
code/log/metadata tools rely on the mitigations described in #7.

## 5. Configuration System

### 5.1. Two sources, strict separation

Configuration comes from exactly two sources with no overlap:

- **YAML config file** (`CONFIG_PATH`): operational settings, workflow
  definitions, project allowlists, data source declarations, and token env
  var _names_. This file is safe to commit, review, and audit.
- **Environment variables**: secrets only (API keys, GitLab tokens,
  kubeconfig paths). These are injected at deployment time via K8s Secrets.

This split exists because YAML provides structure, validation, and audit
trails, while secrets must stay out of version control.

### 5.2. Config loading

Two config objects are built at startup:

**`Config`** (from `config.py`): loaded by `Config.load(settings)`, where
`settings` is the `settings:` section from the YAML file. Auth/bootstrap
fields come from env vars (`GOOGLE_API_KEY`, `GOOGLE_CLOUD_PROJECT`, etc.).
Operational fields come from the settings dict with sensible defaults.

**`AgentConfig`** (from `workflow_config.py`): loaded by
`workflow_config.load(path)`. Contains all workflow definitions, project
entries, and a pre-built `project_index`.

The two are kept separate because `Config` is needed everywhere (model
construction, sandbox creation, session storage), while `AgentConfig` is
only needed for event routing and data source registration.

### 5.3. YAML config structure

```yaml
settings:
  gitlab_url: https://gitlab.com # operational, not a secret
  sandbox:
    image: quay.io/.../image:tag
    namespace: my-namespace # K8s only
  model: gemini-3.1-pro-preview
  # model: claude-sonnet-4-5@20250929   # Anthropic via Vertex AI (alternative)
  max_iterations: 30
  max_runs_per_mr: 5
  internal_notes: true
  max_concurrent_agents: 4
  sqs_queue_url: "" # empty = no SQS (local dev)
  s3_session_bucket: "" # empty = no S3 (local dev)
  trigger_prefix: /hummingbird # slash command prefix
  session_marker_prefix: hummingbird-session # HTML comment marker ID
  auto_trigger: true # auto-run on failed pipelines

workflows:
  analyze-failures:
    prompt: workflows/analyze-failures.md # relative to config file dir
    action: post_gitlab_note
    model: gemini-3.1-pro-preview # per-workflow override
    max_iterations: 50 # per-workflow override

    data_sources:
      gitlab:
        token_env: GITLAB_TOKEN_RO # env var name, not the value
      konflux:
        cluster_url: https://example.com:6443/ns/my-tenant
        kubeconfig_env: KUBECONFIG
        kubearchive_url: https://kubearchive-api-server-product-kubearchive.apps.example.com
      testing_farm: {}

    projects:
      redhat/hummingbird/containers:
        tokens: # per-project token overrides
          gitlab: GITLAB_TOKEN_CONTAINERS_RO
```

Design choices in this structure:

**Workflow-first organization.** Each workflow owns its project list, not the
other way around. This scopes data source permissions per workflow-project
pair. A future code-review workflow can have different GitLab tokens (with
different scopes) than the analyze-failures workflow, with no ambiguity.

**Token env var names in YAML (not values).** The YAML file is committed to
the repo. It contains `token_env: GITLAB_TOKEN_RO` (a name), not the actual
token. The actual secret value is resolved at runtime via
`os.environ.get(env_var)`. This allows the config to be reviewed and
audited without exposing secrets.

**Inline `cluster_url` only.** The Konflux cluster URL is operational
configuration (it identifies which cluster to talk to), not a secret.
Putting it inline in YAML makes it visible and auditable. The URL must
include the namespace path (e.g. `/ns/my-tenant`).

### 5.4. Project index

At load time, `_build_project_index()` constructs a reverse lookup:

```python
project_index: dict[str, list[tuple[str, WorkflowConfig, ProjectEntry]]]
# e.g. {"redhat/hummingbird/containers": [("analyze-failures", wf_cfg, proj_entry)]}
```

This provides O(1) lookup when a pipeline event arrives with a project
path. A single project can appear in multiple workflows (e.g. both
`analyze-failures` and a future `code-review`), and each matching workflow
will be triggered independently.

### 5.5. Token resolution chains

**Model tool tokens** (for data source API calls):

```text
resolve_tool_token(project_entry, ds_name, workflow_cfg):
  1. project_entry.tokens[ds_name]           -> per-project override
  2. workflow_cfg.data_sources[ds_name].token_env  -> workflow default
  3. "" (empty string)                        -> no token
  Each step resolves the env var NAME, then reads os.environ[name].
```

**Workflow action tokens** (for per-workflow GitLab writes):

```text
resolve_action_token(project_entry, ds_name):
  1. project_entry.action_tokens[ds_name]  -> env var NAME
  2. os.environ[name]                      -> resolved token value
  3. "" (empty)                            -> not configured
  No fallback chain. Missing tokens cause validate_env to fail at startup.
```

**Orchestrator tokens** (for operational reads and infrastructure notes):

```text
resolve_orchestrator_token(project_path):
  1. ORCHESTRATOR_GITLAB_TOKEN_<MANGLED_PROJECT>  -> per-project
  2. ORCHESTRATOR_GITLAB_TOKEN                     -> global fallback
  Mangling: "/" -> "_", "-" -> "_", uppercase.
  e.g. "redhat/hummingbird/containers" -> ORCHESTRATOR_GITLAB_TOKEN_REDHAT_HUMMINGBIRD_CONTAINERS
```

**Cluster URL** (for Konflux):

```text
resolve_cluster_url(ds_cfg):
  -> ds_cfg.cluster_url (inline value in YAML)
```

### 5.6. Environment validation

`validate_env(agent_cfg)` checks at startup that all referenced env vars
exist. It walks every workflow's data sources, project token overrides, and
project action token entries, collecting missing vars into a single error
message. This catches configuration errors early instead of failing mid-run
when a specific data source or workflow action is first used. Missing
`action_tokens` env vars are treated the same as missing model tool tokens:
startup fails unconditionally.

## See Also

- [Hummingbird Agent Design — Engine](https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-engine/)
  — Agent loop (§6), tool system (§7), sandbox backends (§8)
- [Hummingbird Agent Design — Runtime](https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-runtime/)
  — Data sources (§9), event pipeline (§10), sessions (§11), workflows
  (§12), deployment (§13)
- [Hummingbird Agent Design — Decisions](https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-decisions/)
  — Design Decision Registry
