# Hummingbird Agent Model Loop

LLMS index: [llms.txt](https://hummingbird-project.io/llms.txt) | Full content: [llms-full.txt](https://hummingbird-project.io/llms-full.txt)

---

How the Hummingbird agent runs conversation turns, calls tools, and integrates
user replies.

For the architectural design rationale behind the loop (why full history
replay, why iteration-based budgets, why nudge via system prompt), see
[Agent Design -- Section 6](https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-engine/#6-agent-loop-design).
For operational usage documentation, see
[Hummingbird Agent](https://hummingbird-project.io/l/hummingbird-agent).
Provider adapters, wire formats, canonical message conversion, and caching are
documented in [Hummingbird AI](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/).

## What gets sent to the model

Every call to the model API sends three things:

### 1. System instruction

A single text string, rebuilt every iteration. The adapter maps it to the
active provider's request format. It is composed of:

```text
BASE_SYSTEM_PROMPT            # agent.py: sandbox rules, tool usage tips
+ tool_notes                  # per-data-source notes from ToolRegistry
+ workflow_prompt             # full content of e.g. workflows/analyze-failures.md
```

Near the iteration or context limit, a warning suffix is appended to the
system prompt for that call. The suffix escalates through three levels:

- `ITERATION_WARNING` / `CONTEXT_WARNING` -- soft "start wrapping up" at 80%
  of the iteration or context limit. Tools remain available.
- `FINAL_TURN_WARNING` -- hard stop on the absolute last iteration or after
  the context limit is exceeded. Combined with tool-calling disabled for that
  request to mechanically prevent further tool calls.
- `EMPTY_RESPONSE_NUDGE` -- appended when the model returns an empty response
  (no text, no tool calls). Tools remain available.

### 2. contents

A list of messages representing the **full conversation history**. It grows
every iteration. Provider-specific content conversion is handled by the
adapter; see [Hummingbird AI](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/#provider-wire-formats).

### 3. tools

Tool definitions, static across all iterations. Adapters translate the shared
tool definition into the provider's schema.

### 4. Tool availability (conditional)

On the final turn (last iteration or after context limit exceeded), the
request disables tools to mechanically prevent function calls. The active
adapter applies the provider-specific setting.

This is only sent when `FINAL_TURN_WARNING` is active. All other iterations
omit this constraint, allowing the model to choose freely between text and tools.

## The agent loop, turn by turn

The loop in `run_agent_loop` runs up to `max_iterations` times. Each
iteration is one round-trip to the model.

### Initialization

```text
system_prompt = BASE_SYSTEM_PROMPT + tool_notes + workflow_prompt
tool_defs     = [sandbox_exec, fetch_to_sandbox, fetch_batch_to_sandbox, <data sources...>]
contents      = [user: {"project": "org/repo", "iid": 42, "sha": "abc...", "session_id": "uuid"}]
```

### Iteration 1

```text
-> Send: system_instruction + contents (1 item) + tool_defs
<- Response: tool_call(gitlab_get_mr_details, {project: "org/repo", iid: 42})

contents.append(model: response.raw_content)  # opaque provider content
  execute tool -> result = {"result": "{\"title\": \"Fix auth\",...}"}
contents.append(tool: model.make_tool_responses([(name, result)]))
```

### Iteration 2

```text
-> Send: system_instruction + contents (3 items) + tool_defs
<- Response: tool_call(fetch_batch_to_sandbox, {requests: [...]})

contents.append(model: ...)
  execute tool -> result = {"results": [{"saved_to": "/tmp/data/pipelineruns.json", "bytes": 85432}]}
contents.append(tool: ...)
```

### Iteration 3

```text
-> Send: system_instruction + contents (5 items) + tool_defs
<- Response: tool_call(sandbox_exec, {command: "jq '[...]' /tmp/data/pipelineruns.json"})

contents.append(model: ...)
  execute tool -> result = {"exit_code": 0, "stdout": "...(preview)...", "stdout_file": "/tmp/_out/0.txt"}
contents.append(tool: ...)
```

### Iteration N (final)

```text
-> Send: system_instruction + contents (2N-1 items) + tool_defs
<- Response: text("## Konflux Failure Analysis\n\n...")    // text + no tool calls = DONE

contents.append(model: {text: "## Konflux Failure Analysis..."})
BREAK -- return (text, usage, transcript, contents)
```

## Response handling and termination

Each iteration, the model's response can contain text, tool calls, both, or
neither. The agent handles each case:

| Response              | Action                                                                                                                                                                                                                                                            |
| --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Text only**         | Final response. Save text, break. (Happy path.)                                                                                                                                                                                                                   |
| **Text + tool calls** | Save text as `last_text` fallback, execute the tool calls, continue loop. The model is thinking aloud while also acting.                                                                                                                                          |
| **Tool calls only**   | Execute the tool calls, continue loop.                                                                                                                                                                                                                            |
| **Neither**           | Empty response -- retry up to `MAX_EMPTY_RETRIES` (2) times. Each retry nudges the model via `EMPTY_RESPONSE_NUDGE` appended to the system prompt (not injected as a user message, to keep the contents list clean). Tools remain available during nudge retries. |

The text + tool calls case is worth noting: the model's text is **not**
returned to the user immediately. It is stored as `last_text` (a fallback in
case the loop terminates later without a clean final text, e.g. by hitting max
iterations). The provider response is appended to `contents` unchanged; its
wire format is documented in [Hummingbird AI](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/).

The loop terminates on:

1. **Text only** -- the model produced a final response.
2. **Max iterations reached** -- the final iteration uses
   `FINAL_TURN_WARNING` and disables tools through the adapter to force text output. Falls back
   to whatever `last_text` was seen, or `"[Agent did not produce a final
   response]"`.
3. **Context limit exceeded** -- `input_tokens >= CONTEXT_LIMIT` sets
   `context_exceeded`, making the next iteration final (same as #2). A soft
   warning (`CONTEXT_WARNING`) fires earlier at 80% of the limit.
4. **Fatal model error** -- a provider error after shared retry handling is
   exhausted. Error classification and retries are described in
   [Hummingbird AI](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/#calls-retries-and-accounting).
5. **Empty responses exhausted** -- `MAX_EMPTY_RETRIES` (2) nudge retries
   failed.
6. **Unexpected error** -- any exception not handled by the retry mechanism
   (e.g. malformed API response) breaks the loop and returns partial results.
   Accumulated contents, transcript, and sandbox are preserved and the
   session is saved normally.

### Budget escalation

Both iterations and context size use the same two-tier pattern:

|                | Soft warning (80%)                             | Hard stop (100%)                                                       |
| -------------- | ---------------------------------------------- | ---------------------------------------------------------------------- |
| **Iterations** | `ITERATION_WARNING` at 80% of `max_iterations` | `FINAL_TURN_WARNING` on last iteration                                 |
| **Context**    | `CONTEXT_WARNING` at 80% of `CONTEXT_LIMIT`    | `FINAL_TURN_WARNING` on next iteration after exceeding `CONTEXT_LIMIT` |

Soft warnings are advisory ("start wrapping up") and keep tools available.
The hard stop disables tools through the active adapter to mechanically
prevent further tool calls, ensuring the model produces text.

## The contents list in detail

The agent treats each entry in `contents` as opaque adapter content during
execution. Provider representations and canonical conversions are documented
in [Hummingbird AI](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/#canonical-sessions).

`model.make_user_content()` constructs the initial event turn and
`model.make_tool_responses()` appends results for dispatched calls. The agent
stores the provider response as `raw_content` and appends it unchanged.

### Typical 8-iteration conversation shape

```text
contents[0]  = user:  {"project":"org/repo","iid":42,...}          # initial event
contents[1]  = model: tool_call(gitlab_get_mr_details)             # iter 1
contents[2]  = tool:  result(gitlab_get_mr_details)                # iter 1
contents[3]  = model: tool_call(fetch_batch_to_sandbox)            # iter 2
contents[4]  = tool:  result(fetch_batch_to_sandbox)               # iter 2
contents[5]  = model: tool_call(sandbox_exec)                      # iter 3
contents[6]  = tool:  result(sandbox_exec)                         # iter 3
...
contents[13] = model: tool_call(sandbox_exec)                      # iter 7
contents[14] = tool:  result(sandbox_exec)                         # iter 7
contents[15] = model: text("## Konflux Failure Analysis...")       # iter 8 (final)
```

This entire list is persisted as canonical messages in `context.json`; the
adapter converts to and from its provider format at the persistence boundary.
See [Hummingbird
AI](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/#canonical-sessions) for
details.

## How tools are called

When the model's response contains tool calls, `_execute_tool_calls`
iterates them sequentially:

```text
tool_registry.execute(ToolCall(name, args))
  match name:
     "sandbox_exec"           -> runs sh -c in container -> {exit_code, stdout, stderr}
     "fetch_to_sandbox"       -> calls data source func -> writes to sandbox file -> {saved_to, bytes}
     "fetch_batch_to_sandbox" -> multiple fetch_to_sandbox in one call
     any data source name     -> calls func directly -> returns inline or auto-spills large output
```

### Large output handling

When stdout or a data source response exceeds 4KB, it is automatically saved
to a sandbox file (`/tmp/_out/N.txt`) and only a preview (head + tail) is
returned to the model. The model gets the file path and can use `sandbox_exec`
with `jq`/`grep`/`head` to process it. This keeps the context window
manageable.

## How a user reply integrates (session resumption)

When a user replies to an agent note on GitLab, the orchestrator resumes the
conversation by restoring the previous state and appending the reply.

### What gets restored from S3

- **`context.json`** -- the persisted conversation in canonical form; the
  active adapter converts it to the in-memory representation before the loop
  runs
- **`sandbox.tar.gz`** -- `/tmp/_out/` files (PipelineRun JSONs, test logs,
  jq output, etc.) restored into the new sandbox

Canonical storage allows sessions to resume with another provider; conversion
details are in [Hummingbird AI](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/#canonical-sessions).

### The resumed contents list

```text
// Restored from context.json (previous run)
contents[0]  = user:  {"project":"org/repo","iid":42,...}           # original event
contents[1]  = model: tool_call(gitlab_get_mr_details)              # iter 1
contents[2]  = tool:  result(gitlab_get_mr_details)                 # iter 1
...                                                                  # all prior turns
contents[15] = model: text("## Konflux Failure Analysis...")        # previous final text

// NEW: user reply appended
contents[16] = user:  "Can you look at the clair-scan timeout more closely?"
```

### The agent loop continues

```text
ITERATION 1 (resumed):
  -> Send: system_instruction (+ CONTINUATION_PROMPT) + contents[0..16] + tool_defs
  <- Response: tool_call(sandbox_exec, {command: "grep -i timeout /tmp/data/..."})
                                                       ^ using restored sandbox files
  contents[17] = model: tool_call(sandbox_exec)
  contents[18] = tool:  result(sandbox_exec)

ITERATION 2 (resumed):
  <- Response: text("The clair-scan timeout is caused by...")
  contents[19] = model: text("The clair-scan timeout is caused by...")
  BREAK
```

### Cold start vs resumed -- key differences

| Aspect               | Cold start                   | Resumed                                                |
| -------------------- | ---------------------------- | ------------------------------------------------------ |
| System prompt        | BASE + tool_notes + workflow | BASE + tool_notes + workflow + **CONTINUATION_PROMPT** |
| First user message   | `{"project":..., "iid":...}` | `{"project":..., "iid":...}` (restored)                |
| Conversation history | Empty (just the event)       | Full prior conversation                                |
| Latest user message  | --                           | `"Can you look at the clair-scan timeout?"`            |
| Sandbox files        | Empty                        | Restored from archive                                  |
| Tool definitions     | Same                         | Same                                                   |
| Session format       | Adapter content in memory    | Canonical on disk; adapter content after load          |

### The CONTINUATION_PROMPT

Without this, the workflow prompt (e.g. `analyze-failures.md`) tells the model
to follow a rigid workflow: Data.1, Data.2, Data.3, Analysis.1... The model
might try to re-run the entire analysis. The continuation prompt overrides
this:

```text
## Continuation

This is a follow-up to a previous conversation. The conversation history
contains your prior analysis and tool calls. The user is replying with a
question or request about your previous work.

**Do NOT re-run the full workflow from scratch.** Instead:
- Respond directly to the user's question
- Use your tools to investigate further if needed (logs, data are still
  available in the sandbox)
- Reference your previous findings where relevant
- Keep your response focused on what the user asked
```

The model now understands: "I already did the analysis (it's all in the
conversation history). The user has a specific question. Let me answer it."

## Iteration and token budget for resumed sessions

The resumed session gets a **full fresh iteration budget**. `run_workflow`
sets `max_iterations` from the workflow config (e.g. 50 for
analyze-failures), and the loop counter starts at 0 regardless of how many
iterations the previous session used. The model can make as many tool calls
as it needs to answer the follow-up.

The practical constraint is the **context window**, not iterations. The
restored contents can be large -- a typical 8-iteration cold start uses
~50-100K input tokens. Since the model re-counts the full history every
iteration, the resumed session starts at roughly the token count where the
previous session ended, plus the new user message. Each new tool call adds
more. Token counts logged by the agent are **billed** tokens (cumulative
across API calls; each call re-sends the full history, so these overlap).

The existing `CONTEXT_LIMIT` check still applies: at 80% of the limit, a
soft `CONTEXT_WARNING` tells the model to start wrapping up. If the limit is
exceeded, the next iteration becomes final (`FINAL_TURN_WARNING` with provider
tools disabled by the adapter).

For the typical case (user asks one focused follow-up, agent does 1-3 tool
calls), this is fine. Multi-turn deep dives will eventually hit the context
limit, at which point the agent wraps up -- and the user can start a fresh
conversation if needed.

## Provider behavior

Provider wire formats, canonical message conversion, prompt caching, server-side
tools, and adapter continuations are owned by the
[`hummingbird-ai` library](https://hummingbird-project.io/docs/background/tools/hummingbird-ai/).
The Hummingbird agent loop
remains responsible for tool dispatch, iteration/context budgets, and session
resumption.
