<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
	<channel>
		<title>Hummingbird Agent on Project Hummingbird</title>
		<link>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/</link>
		<description>Recent content in Hummingbird Agent on Project Hummingbird</description>
		<generator>Hugo</generator>
		<language>en</language>
		
		
		
		
			<atom:link href="https://hummingbird-project.io/docs/background/tools/hummingbird-agent/index.xml" rel="self" type="application/rss+xml" />
			<item>
				<title>Hummingbird Agent Model Loop</title>
				<link>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/model-loop/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/model-loop/</guid>
				<description>&lt;p&gt;How the Hummingbird agent runs conversation turns, calls tools, and integrates&#xA;user replies.&lt;/p&gt;&#xA;&lt;p&gt;For the architectural design rationale behind the loop (why full history&#xA;replay, why iteration-based budgets, why nudge via system prompt), see&#xA;&lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-engine/#6-agent-loop-design&#34;&gt;Agent Design &amp;ndash; Section 6&lt;/a&gt;.&#xA;For operational usage documentation, see&#xA;&lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/&#34;&gt;Hummingbird Agent&lt;/a&gt;.&#xA;Provider adapters, wire formats, canonical message conversion, and caching are&#xA;documented in &lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-ai/&#34;&gt;Hummingbird AI&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2 id=&#34;what-gets-sent-to-the-model&#34;&gt;What gets sent to the model&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#what-gets-sent-to-the-model&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;p&gt;Every call to the model API sends three things:&lt;/p&gt;</description>
			</item>
			<item>
				<title>Hummingbird Agent Design</title>
				<link>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design/</guid>
				<description>&lt;p&gt;Architectural design document for the hummingbird-agent. Covers the reasoning&#xA;behind every major design choice so that future changes can be made safely,&#xA;without accidentally violating invariants that hold the system together.&lt;/p&gt;&#xA;&lt;p&gt;For operational usage (CLI, config reference, deployment), see&#xA;&lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/&#34;&gt;Hummingbird Agent&lt;/a&gt;. For agent turn flow and tool&#xA;handling, see &lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/model-loop/&#34;&gt;Agent Model Loop&lt;/a&gt;. Provider&#xA;adapters and wire formats are documented in &lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-ai/&#34;&gt;Hummingbird AI&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2 id=&#34;1-design-philosophy&#34;&gt;1. Design Philosophy&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#1-design-philosophy&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;p&gt;Five principles shaped the agent&amp;rsquo;s architecture. Every component traces back&#xA;to at least one of these.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Hummingbird Agent Design — Engine</title>
				<link>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-engine/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-engine/</guid>
				<description>&lt;p&gt;Agent loop, tool system, and sandbox backends for the Hummingbird Agent.&#xA;This page covers the execution engine: how the agent iterates, dispatches tools,&#xA;and isolates execution. For foundations (design philosophy, system architecture,&#xA;modules, security, config), see &lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design/&#34;&gt;Hummingbird Agent Design&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2 id=&#34;6-agent-loop-design&#34;&gt;6. Agent Loop Design&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#6-agent-loop-design&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;p&gt;For the wire-format walkthrough (what bytes go to the model API, what comes back),&#xA;see &lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/model-loop/&#34;&gt;Agent Model Loop&lt;/a&gt;. This section&#xA;covers the &lt;em&gt;design rationale&lt;/em&gt; behind the loop.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Hummingbird Agent Evals</title>
				<link>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/evals/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/evals/</guid>
				<description>&lt;p&gt;Evaluation framework for scientific measurement of LLM output quality.&#xA;For the agent itself, see &lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/&#34;&gt;Hummingbird Agent&lt;/a&gt;.&#xA;For architecture details, see&#xA;&lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design/&#34;&gt;Hummingbird Agent Design&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2 id=&#34;what-evals-are&#34;&gt;What evals are&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#what-evals-are&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;p&gt;Evals measure the quality of LLM outputs under controlled conditions.&#xA;They answer questions like &amp;ldquo;does prompt v5 produce better code reviews&#xA;than v4?&amp;rdquo; or &amp;ldquo;can an LLM answer this question correctly using only&#xA;public documentation?&amp;rdquo;&lt;/p&gt;&#xA;&lt;p&gt;Evals are &lt;strong&gt;not unit tests&lt;/strong&gt;. Unit tests verify deterministic behavior;&#xA;evals quantify stochastic output quality through structured scoring.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Documentation Quality Evaluation</title>
				<link>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/docs-eval/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/docs-eval/</guid>
				<description>&lt;p&gt;Reference for the documentation quality evaluation, which measures whether&#xA;public documentation is complete enough for LLMs with web search to answer&#xA;questions correctly. For the general evaluation framework, see&#xA;&lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/evals/&#34;&gt;Hummingbird Agent Evals&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2 id=&#34;config-file&#34;&gt;Config file&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#config-file&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;p&gt;A single YAML file combines evaluation settings and the golden dataset.&#xA;It is passed to the run script via &lt;code&gt;-c&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;python -m hummingbird_agent.evals.docs_eval.run_eval -c path/to/docs_eval.yml&#xA;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The config file lives in the documentation repo alongside the content&#xA;being measured.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Hummingbird Agent Design — Runtime</title>
				<link>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-runtime/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-runtime/</guid>
				<description>&lt;p&gt;Data sources, event pipeline, sessions, workflows, and deployment for the Hummingbird Agent.&#xA;This page covers the runtime: how the agent connects to external systems, processes events,&#xA;manages sessions, and is deployed. For the execution engine (agent loop, tool system, sandbox),&#xA;see &lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-engine/&#34;&gt;Hummingbird Agent Design — Engine&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2 id=&#34;9-data-sources&#34;&gt;9. Data Sources&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#9-data-sources&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;p&gt;Data sources are external APIs wrapped as tool-calling functions. The model&#xA;invokes them by name; the orchestrator executes them and returns results&#xA;(inline or auto-spilled). Each data source module follows the same pattern:&lt;/p&gt;</description>
			</item>
			<item>
				<title>Hummingbird Agent Design — Decisions</title>
				<link>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-decisions/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design-decisions/</guid>
				<description>&lt;p&gt;Design Decision Registry (DDR) for the Hummingbird Agent — every major&#xA;architectural choice, the alternatives considered, why the chosen approach won,&#xA;and what would break if reversed. For the full design context, see&#xA;&lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-agent/design/&#34;&gt;Hummingbird Agent Design&lt;/a&gt;. Provider adapter&#xA;decisions are in &lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-ai/&#34;&gt;Hummingbird AI Design&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2 id=&#34;design-decisions&#34;&gt;Design Decisions&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#design-decisions&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;h3 id=&#34;kubectl-exec-over-kubernetes-python-exec-api&#34;&gt;kubectl exec over kubernetes Python exec API&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#kubectl-exec-over-kubernetes-python-exec-api&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;kubectl exec&lt;/code&gt; as subprocess for sandbox command execution.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; kubernetes Python client &lt;code&gt;stream()&lt;/code&gt; API.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Three showstopper bugs in the Python client: (1) WebSocket&#xA;v1-v4 has no stdin EOF signal, so &lt;code&gt;cat &amp;gt; /file&lt;/code&gt; hangs forever; (2)&#xA;&lt;code&gt;BrokenPipeError&lt;/code&gt; on stdin larger than ~1MB; (3) &lt;code&gt;stream()&lt;/code&gt; accumulates&#xA;all stdout in memory with no control. &lt;code&gt;kubectl&lt;/code&gt; avoids all three and&#xA;provides the same subprocess interface as Podman.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; &lt;code&gt;write_file()&lt;/code&gt; would hang or fail on large data. Pod&#xA;log retrieval with large outputs would OOM. Sandbox reliability would&#xA;drop significantly.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;yaml-config-over-env-vars-for-operational-settings&#34;&gt;YAML config over env vars for operational settings&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#yaml-config-over-env-vars-for-operational-settings&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Single YAML file for workflows, projects, limits, data&#xA;source declarations.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Everything in env vars (original design).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Env vars cannot express structured data (workflow-project&#xA;mappings, per-project token overrides, data source config with multiple&#xA;fields). YAML provides structure, validation, and audit trails.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Token scoping would be lost (no per-workflow-project&#xA;token overrides). Project allowlists would be impossible. The config&#xA;would be unauditable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;token-env-var-names-in-yaml-not-values&#34;&gt;Token env var names in YAML, not values&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#token-env-var-names-in-yaml-not-values&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; YAML contains &lt;code&gt;token_env: GITLAB_TOKEN_RO&lt;/code&gt; (the env var&#xA;name), not the actual token value.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Inline secrets in YAML, or env vars for everything.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; The YAML file can be committed, reviewed, and audited. Actual&#xA;secrets stay in env vars (injected via K8s Secrets). Inline secrets&#xA;would make the config file a secret itself.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; The config file would become a secret, breaking audit&#xA;trails and code review workflows.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;orchestrator-token-prefix-convention-orchestrator_&#34;&gt;Orchestrator token prefix convention (ORCHESTRATOR_*)&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#orchestrator-token-prefix-convention-orchestrator_&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Orchestrator tokens use &lt;code&gt;ORCHESTRATOR_GITLAB_TOKEN_*&lt;/code&gt; env&#xA;var names by convention.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Same env var namespace as model tokens, distinguished&#xA;by context.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Structural separation makes it impossible to accidentally pass&#xA;an orchestrator token to a model tool (or vice versa). The&#xA;&lt;code&gt;ORCHESTRATOR_&lt;/code&gt; prefix is never used in YAML &lt;code&gt;token_env&lt;/code&gt; fields.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; A misconfiguration could leak write-capable tokens to&#xA;the model, which could then expose them via tool calls.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;auto-spill-over-conversation-compaction&#34;&gt;Auto-spill over conversation compaction&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#auto-spill-over-conversation-compaction&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Large outputs are saved to sandbox files with previews&#xA;returned to the model.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Replace old tool results in &lt;code&gt;contents&lt;/code&gt; with compact&#xA;summaries (conversation compaction).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Compaction requires modifying the &lt;code&gt;contents&lt;/code&gt; list, which risks&#xA;violating provider conversation constraints. Auto-spill achieves ~90% of&#xA;the token reduction without modifying model history. See&#xA;&lt;a href=&#34;https://hummingbird-project.io/docs/background/tools/hummingbird-ai/&#34;&gt;Hummingbird AI&lt;/a&gt; for adapter&#xA;format requirements.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Token usage would increase ~15-30%. Per-call context&#xA;would grow unbounded. Sessions would hit the context limit much sooner.&#xA;Conversation compaction could be added &lt;em&gt;on top of&lt;/em&gt; auto-spill in the&#xA;future, but is not needed with current workloads.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;iteration-count-over-cumulative-token-budget&#34;&gt;Iteration count over cumulative token budget&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#iteration-count-over-cumulative-token-budget&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;max_iterations&lt;/code&gt; as the primary cost control lever.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Cumulative token budget (stop when total billed tokens&#xA;exceed a threshold).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; With full history replay, each API call re-sends everything.&#xA;Cumulative billed tokens double-count: call 1 = 5K, call 2 = 10K&#xA;(including 5K again), total = 15K billed but only 10K new content. With&#xA;auto-spill keeping per-call size bounded, iteration count is a much&#xA;simpler and more predictable proxy for actual cost.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; The budget model would be confusing and inaccurate.&#xA;Cost estimates would be wrong. The cumulative metric is still logged&#xA;for observability, but it does not drive termination.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;tooldef-notes-in-system-prompt-not-tool-schema&#34;&gt;ToolDef notes in system prompt, not tool schema&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#tooldef-notes-in-system-prompt-not-tool-schema&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Domain knowledge (Konflux dual-fetch, TF XML structure) goes&#xA;in &lt;code&gt;ToolDef.notes&lt;/code&gt;, injected into the system prompt.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Put everything in the tool schema &lt;code&gt;description&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Tool schemas have character limits and are sent in the &lt;code&gt;tools&lt;/code&gt;&#xA;field of every API call. Long descriptions waste tokens on tool defs.&#xA;System prompt notes are sent once and can be arbitrarily detailed.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Tool descriptions would be bloated. Domain knowledge&#xA;would need to be duplicated in every workflow &lt;code&gt;.md&lt;/code&gt; file.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;full-history-replay-no-compaction&#34;&gt;Full history replay (no compaction)&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#full-history-replay-no-compaction&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; The &lt;code&gt;contents&lt;/code&gt; list grows monotonically. Items are never&#xA;removed or modified (except empty response retries).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Compact old turns to reduce context size.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Provider APIs impose strict conversation structure. Modifying or&#xA;removing items risks invalid history. Auto-spill handles the growth problem&#xA;at the source (preventing large items from entering history).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Risk of provider API errors from malformed conversation&#xA;structure and of confusing the model with summarized results.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;fetch_to_sandbox-kept-alongside-auto-spill&#34;&gt;fetch_to_sandbox kept alongside auto-spill&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#fetch_to_sandbox-kept-alongside-auto-spill&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Both &lt;code&gt;fetch_to_sandbox&lt;/code&gt; and auto-spill coexist.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Remove &lt;code&gt;fetch_to_sandbox&lt;/code&gt; since auto-spill handles&#xA;large outputs automatically.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; &lt;code&gt;fetch_to_sandbox&lt;/code&gt; provides explicit path control (model can&#xA;choose meaningful filenames), batch fetching (one tool call for multiple&#xA;fetches), and no-preview responses (useful when the model will process&#xA;with jq anyway).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; The model would lose path control and batch fetching&#xA;would require multiple auto-spilled calls. The workflow would need more&#xA;iterations to achieve the same result.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;k8s-library-for-lifecycle--kubectl-for-exec-hybrid&#34;&gt;K8s library for lifecycle + kubectl for exec (hybrid)&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#k8s-library-for-lifecycle--kubectl-for-exec-hybrid&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Use the &lt;code&gt;kubernetes&lt;/code&gt; Python library for pod&#xA;create/read/delete, &lt;code&gt;kubectl&lt;/code&gt; subprocess for exec.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; All-kubectl (subprocess for everything) or all-library.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; The library provides typed pod status polling and clean error&#xA;handling for lifecycle. &lt;code&gt;kubectl&lt;/code&gt; provides reliable exec (see &amp;ldquo;kubectl&#xA;exec over kubernetes Python exec API&amp;rdquo; above). Using the library for exec&#xA;would require solving the three WebSocket bugs.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Either lifecycle management would be fragile (parsing&#xA;kubectl JSON output for pod status) or exec would be unreliable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;single-file-yaml-config-over-multiple-config-files&#34;&gt;Single-file YAML config over multiple config files&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#single-file-yaml-config-over-multiple-config-files&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Everything in one &lt;code&gt;config.yaml&lt;/code&gt; with &lt;code&gt;settings&lt;/code&gt; and&#xA;&lt;code&gt;workflows&lt;/code&gt; sections.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Separate files per workflow, or settings.yaml +&#xA;workflows.yaml.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Single source of truth. All project-workflow mappings visible in&#xA;one place. Settings-level defaults flow down to all projects.&#xA;No file discovery logic needed.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Token resolution would need cross-file lookups. Project&#xA;index would need multi-file aggregation. Config validation would be more&#xA;complex.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;workflow-md-as-pure-prompt-metadata-in-yaml&#34;&gt;Workflow .md as pure prompt, metadata in YAML&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#workflow-md-as-pure-prompt-metadata-in-yaml&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; The workflow &lt;code&gt;.md&lt;/code&gt; file is pure system prompt text. Metadata&#xA;(&lt;code&gt;action&lt;/code&gt;, &lt;code&gt;model&lt;/code&gt;, &lt;code&gt;max_iterations&lt;/code&gt;, &lt;code&gt;data_sources&lt;/code&gt;, &lt;code&gt;projects&lt;/code&gt;) lives&#xA;in the YAML config.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; YAML frontmatter in the &lt;code&gt;.md&lt;/code&gt; file (original design).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Separation of concerns. The &lt;code&gt;.md&lt;/code&gt; file is the model&amp;rsquo;s&#xA;instructions &amp;ndash; it should be readable and editable by anyone writing&#xA;prompts. The YAML config is the orchestrator&amp;rsquo;s instructions &amp;ndash; it&#xA;controls routing, limits, and credentials. Mixing them in one file&#xA;conflates two audiences.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Prompt authors would need to understand YAML config&#xA;structure. Token/project config would be scattered across &lt;code&gt;.md&lt;/code&gt; files&#xA;instead of centralized.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;nudge-via-system-prompt-suffix-not-user-message&#34;&gt;Nudge via system prompt suffix, not user message&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#nudge-via-system-prompt-suffix-not-user-message&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Empty response nudge and budget warnings are appended to the&#xA;system prompt as suffixes.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Inject synthetic user messages into &lt;code&gt;contents&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; User messages in &lt;code&gt;contents&lt;/code&gt; must come from the actual user (the&#xA;initial event, or a reply). Injecting synthetic messages pollutes the&#xA;conversation history that is saved in sessions. System prompt suffixes&#xA;are transient &amp;ndash; they affect one API call without permanently modifying&#xA;the conversation state.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Saved sessions would contain synthetic user messages.&#xA;Resumed conversations would be confusing. The model might respond to the&#xA;synthetic messages instead of the user&amp;rsquo;s actual input.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;session-id-reuse-over-new-id-per-resume&#34;&gt;Session ID reuse over new-ID-per-resume&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#session-id-reuse-over-new-id-per-resume&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Resumed sessions keep the original session ID. S3 state is&#xA;overwritten in place.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Each resume generates a new UUID, saving alongside the&#xA;original (append-only tree of sessions).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; GitLab discussions are linear, not branching. There is no&#xA;scenario where two different sessions from the same thread are both&#xA;valid. A new UUID per resume creates orphaned S3 snapshots that are&#xA;never referenced again, since &lt;code&gt;find_session_for_reply()&lt;/code&gt; always picks&#xA;the latest marker. Reusing the ID is simpler, uses less storage, and&#xA;matches the linear conversation model.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; S3 would accumulate orphaned session snapshots. Each&#xA;resume would need a new note marker, but the old markers would still be&#xA;in the thread, creating confusion about which session is current.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;reply-to-unauthorized-agent-directed-notes-instead-of-silent-skip&#34;&gt;Reply to unauthorized agent-directed notes instead of silent skip&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#reply-to-unauthorized-agent-directed-notes-instead-of-silent-skip&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; When an unauthorized user sends a slash command or replies to&#xA;an agent thread, the agent replies in the same discussion with a short&#xA;access-denied message. Non-agent-directed notes are silently ignored&#xA;(debug log, no access check).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Log a warning and skip silently for all unauthorized&#xA;notes (the original behavior).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Silent skip gives no feedback to someone who intentionally tried&#xA;to engage the agent, which is confusing. On the other hand, checking&#xA;access and logging a warning for every random comment on a public project&#xA;is noisy and pointless. Moving the access check after intent detection&#xA;cleanly separates the two cases. The reply uses the discussions API so it&#xA;automatically inherits confidentiality from the parent note/thread.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Users without sufficient access who try &lt;code&gt;/hummingbird&lt;/code&gt;&#xA;would get no feedback. The agent log would be noisy with warnings for&#xA;every comment on public MRs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;deployment-backed-pod-pool-over-python-thread-pool&#34;&gt;Deployment-backed pod pool over Python-thread pool&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#deployment-backed-pod-pool-over-python-thread-pool&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; A Kubernetes Deployment maintains pre-warmed standby pods.&#xA;The agent claims a pod by relabeling it; the Deployment replaces it.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; A Python-side thread pool that pre-creates pods and&#xA;queues them for use.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; The Deployment provides self-healing (restart crashed pods),&#xA;native scaling (&lt;code&gt;kubectl scale&lt;/code&gt;), rolling updates (image changes), and&#xA;monitoring via standard K8s tooling. A Python pool would need to&#xA;reimplement all of these.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Pod replenishment, crash recovery, and image updates&#xA;would all need custom code. Scaling would require agent redeployment.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;pod-isolation-never-reuse-sandbox-pods&#34;&gt;Pod isolation: never reuse sandbox pods&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#pod-isolation-never-reuse-sandbox-pods&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Each workflow run gets its own pod. After cleanup, the pod is&#xA;deleted. Claimed pods are detached from the ReplicaSet.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Return used pods to the pool and reset them.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Residual state from a previous run (files, environment, running&#xA;processes) could leak between MR investigations, creating security and&#xA;correctness risks. Deletion is simple and foolproof.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Would need a reliable pod-reset mechanism and auditing&#xA;that no state survives between runs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;absolute-reap-by-deadline-over-relative-claimed-at-age&#34;&gt;Absolute reap-by deadline over relative claimed-at age&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#absolute-reap-by-deadline-over-relative-claimed-at-age&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Active pods are annotated with &lt;code&gt;hummingbird/reap-by&lt;/code&gt; (an&#xA;absolute UTC timestamp). The reaper deletes pods where &lt;code&gt;now &amp;gt; reap-by&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Annotate with &lt;code&gt;claimed-at&lt;/code&gt; and compute age relative to&#xA;&lt;code&gt;max_active_seconds&lt;/code&gt;; or use &lt;code&gt;activeDeadlineSeconds&lt;/code&gt; on the pod spec.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; An absolute deadline simplifies the reaper to a single&#xA;comparison. It also supports varying deadlines: claim sets&#xA;&lt;code&gt;reap-by = now + max_active_seconds&lt;/code&gt;, while linger sets&#xA;&lt;code&gt;reap-by = now + linger_seconds&lt;/code&gt;. With a relative timestamp, the reaper&#xA;would need to know which mode the pod is in.&#xA;&lt;code&gt;activeDeadlineSeconds&lt;/code&gt; applies from pod creation, not from claim &amp;ndash;&#xA;standby pods would expire before being used.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; The reaper would need mode-aware age calculations.&#xA;Lingering pods would require a separate annotation or reaper path.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;indefinite-claim-wait-over-timeout&#34;&gt;Indefinite claim wait over timeout&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#indefinite-claim-wait-over-timeout&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;SandboxPool.claim()&lt;/code&gt; polls indefinitely (governed by&#xA;shutdown event), logging at DEBUG then WARNING.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Timeout after N seconds and raise an error.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; The pool is typically smaller than &lt;code&gt;max_concurrent_agents&lt;/code&gt; for&#xA;cost reasons. Waiting for a replacement pod is normal operational&#xA;behavior, not an error. A timeout would cause spurious failures during&#xA;burst traffic. The SQS semaphore already bounds concurrency.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Burst traffic would cause avoidable failures instead of&#xA;brief delays.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;pool-config-in-deployment-only-not-in-agent-configmap&#34;&gt;Pool config in Deployment only, not in agent configmap&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#pool-config-in-deployment-only-not-in-agent-configmap&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; In pool mode (&lt;code&gt;k8spool&lt;/code&gt;), the pod image, resources, labels,&#xA;and security context are defined solely in the Deployment template. The&#xA;agent config only needs &lt;code&gt;namespace&lt;/code&gt; and &lt;code&gt;active_deadline_seconds&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Keep image/resources/metadata in the agent configmap too&#xA;(as in &lt;code&gt;k8s&lt;/code&gt; mode).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Eliminates config duplication. The Deployment template is the&#xA;single source of truth. Changes to pod resources or image only require&#xA;updating one place and re-rolling the Deployment.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Config drift between the Deployment and the agent&#xA;configmap would be a constant risk.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;pod-lingering-over-immediate-cleanup-for-reply-latency&#34;&gt;Pod lingering over immediate cleanup for reply latency&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#pod-lingering-over-immediate-cleanup-for-reply-latency&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; After a successful workflow, pool sandbox pods linger for&#xA;&lt;code&gt;linger_seconds&lt;/code&gt; (default 300) with a &lt;code&gt;session-id&lt;/code&gt; annotation. Replies&#xA;within the window reclaim the pod via &lt;code&gt;try_reclaim()&lt;/code&gt;, skipping pod&#xA;creation and S3 archive restoration.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Always delete the pod immediately and restore from S3 on&#xA;every reply.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Reply latency drops from seconds (pod claim + S3 restore) to&#xA;near-zero. The S3 archive is still saved as a fallback if the pod is&#xA;gone. The &lt;code&gt;reap-by&lt;/code&gt; annotation ensures lingering pods are cleaned up if&#xA;no reply arrives. The &lt;code&gt;session-id&lt;/code&gt; annotation doubles as a concurrency&#xA;guard: it is only present while lingering, preventing active pods from&#xA;being reclaimed.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Every reply would pay full pod + restore latency, even&#xA;for immediate follow-ups. User experience for conversational interactions&#xA;would degrade noticeably.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;linger-on-sandbox-protocol-over-isinstance-checks-in-runner&#34;&gt;linger() on Sandbox protocol over isinstance checks in runner&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#linger-on-sandbox-protocol-over-isinstance-checks-in-runner&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; All sandbox backends implement &lt;code&gt;linger(session_id)&lt;/code&gt;. Non-pool&#xA;backends fall back to &lt;code&gt;cleanup()&lt;/code&gt;. The runner calls &lt;code&gt;sb.linger()&lt;/code&gt; without&#xA;type checks.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; &lt;code&gt;isinstance(sb, K8sPoolSandbox)&lt;/code&gt; in &lt;code&gt;run_workflow()&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Keeps the runner backend-agnostic. Adding a new backend requires&#xA;only implementing the protocol, not touching the runner. The fallback&#xA;behavior is co-located with each backend.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; The runner would need to know about every backend type&#xA;and their linger capabilities.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;gcp-sa-key-over-workload-identity-federation&#34;&gt;GCP SA key over Workload Identity Federation&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#gcp-sa-key-over-workload-identity-federation&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; GCP service account key stored in Vault, rotated via&#xA;cki-tools credential manager (prepare/switch/clean cycle).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Workload Identity Federation (WIF) with projected SA&#xA;token. Two sub-options: (a) automatic OIDC discovery if the cluster&#xA;issuer is public, (b) manual JWKS upload for internal issuers.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; mpp-prod&amp;rsquo;s OIDC issuer is &lt;code&gt;https://kubernetes.default.svc&lt;/code&gt;&#xA;(internal, not publicly reachable), so GCP STS cannot discover it&#xA;automatically. Manual JWKS upload works but requires re-upload after&#xA;SRE-triggered signing key rotations. SA key integrates with the&#xA;existing credential manager rotation infrastructure (same pattern as&#xA;AWS keys and GitLab tokens), needs no OIDC reachability, and enables&#xA;automated validate/update via &lt;code&gt;google-auth&lt;/code&gt; in CI. The application code&#xA;(&lt;code&gt;hummingbird_ai.http.VertexAuth&lt;/code&gt; / &lt;code&gt;google.auth.default()&lt;/code&gt;) works identically&#xA;with both approaches &amp;ndash; switching to WIF later requires only infrastructure&#xA;changes.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Replace SA key with WIF projected token + credential&#xA;config ConfigMap. The &lt;code&gt;gcp_service_account_key&lt;/code&gt; token type in&#xA;cki-tools would no longer be needed for this use case.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;developer-trust-filtering-over-unfiltered-or-sanitized-discussion-comments&#34;&gt;Developer+ trust filtering over unfiltered or sanitized discussion comments&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#developer-trust-filtering-over-unfiltered-or-sanitized-discussion-comments&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;gitlab_get_mr_discussions&lt;/code&gt; checks each note author&amp;rsquo;s project&#xA;access level (Developer+ / &amp;gt;= 30). Untrusted notes are replaced with a fixed&#xA;placeholder in mixed-trust discussions, or the entire discussion is dropped&#xA;if all notes are untrusted. Agent notes are always trusted (detected by&#xA;session marker) but have transcripts and metadata stripped.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternatives considered:&lt;/strong&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(a) Include all comments unfiltered. Simplest, but any external&#xA;contributor can craft comments that manipulate the model&amp;rsquo;s review output&#xA;(prompt injection).&lt;/li&gt;&#xA;&lt;li&gt;(b) Sanitize/escape untrusted content (strip markdown, quote as code&#xA;blocks, prefix with &amp;ldquo;[external]&amp;rdquo;). There is no reliable escaping mechanism&#xA;for LLM prompts &amp;ndash; the model interprets natural language regardless of&#xA;formatting. Escaping gives a false sense of security.&lt;/li&gt;&#xA;&lt;li&gt;(c) Only include agent-authored notes (skip all human comments). Safe, but&#xA;defeats the purpose: the model would never see developer responses to its&#xA;own findings.&lt;/li&gt;&#xA;&lt;li&gt;(d) Include only discussions where the agent participated. Better, but&#xA;still misses developer-initiated review threads that provide relevant&#xA;context.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Developer+ is the same threshold used for pipeline trigger&#xA;authorization (invariant #8) and slash command access. It matches the trust&#xA;boundary already established: people who can push code and approve MRs are&#xA;trusted to provide review context. The placeholder approach preserves&#xA;discussion structure (the model sees that &lt;em&gt;someone&lt;/em&gt; replied) without&#xA;exposing the content. Full-drop for all-untrusted discussions avoids&#xA;noise from discussions that contain zero useful context.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; (a) opens a prompt injection vector on any project that&#xA;accepts external MRs. (b) provides no actual protection. (c) makes&#xA;follow-up reviews unable to see developer explanations, causing repeated&#xA;false positives. (d) misses developer-initiated context.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;bot-author-filtering-for-session-marker-parsing-over-unfiltered-note-scanning&#34;&gt;Bot-author filtering for session marker parsing over unfiltered note scanning&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#bot-author-filtering-for-session-marker-parsing-over-unfiltered-note-scanning&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;find_session_for_reply()&lt;/code&gt;, &lt;code&gt;scan_agent_threads()&lt;/code&gt;, and the&#xA;&lt;code&gt;handle_note()&lt;/code&gt; self-filter all resolve the bot user ID via&#xA;&lt;code&gt;get_bot_user_id()&lt;/code&gt; (which calls &lt;code&gt;gl.auth()&lt;/code&gt; on the orchestrator token)&#xA;and only consider notes where &lt;code&gt;author.id&lt;/code&gt; matches. In&#xA;&lt;code&gt;find_session_for_reply()&lt;/code&gt;, the first matching marker wins.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Parse markers from any note and keep the last match;&#xA;substring self-filter in &lt;code&gt;handle_note&lt;/code&gt; (previous behavior).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; The previous unfiltered approach allowed session hijacking: a&#xA;user reply or agent review prose containing an example marker&#xA;(&lt;code&gt;&amp;lt;!-- hummingbird-session: ... --&amp;gt;&lt;/code&gt;) was parsed as a real session,&#xA;causing &amp;ldquo;session expired&amp;rdquo; errors in production. For &lt;code&gt;handle_note&lt;/code&gt;, the&#xA;substring check caused the reverse problem: a user embedding the marker&#xA;prefix in a reply silently suppressed the event (denial of service). The&#xA;&lt;code&gt;gl.auth()&lt;/code&gt; call is one &lt;code&gt;GET /user&lt;/code&gt; per invocation &amp;ndash; negligible cost.&#xA;&lt;code&gt;_get_client_with_bot_id()&lt;/code&gt; returns both the authenticated client and&#xA;the bot user ID, avoiding redundant client construction.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Any MR participant can inject a marker to hijack the&#xA;session ID or inflate rate-limit counts. The agent&amp;rsquo;s own review prose&#xA;containing example markers causes spurious &amp;ldquo;session expired&amp;rdquo; messages.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;ijson-streaming-over-buffered-json-for-k8s-list-responses&#34;&gt;ijson streaming over buffered JSON for K8s list responses&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#ijson-streaming-over-buffered-json-for-k8s-list-responses&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;session.get(stream=True)&lt;/code&gt; + &lt;code&gt;ijson.parse(resp.raw)&lt;/code&gt; for&#xA;&lt;code&gt;iter_paginated()&lt;/code&gt;. Items are yielded one at a time via &lt;code&gt;ObjectBuilder&lt;/code&gt;;&#xA;the &lt;code&gt;metadata.continue&lt;/code&gt; pagination token is captured in the same parse&#xA;pass. &lt;code&gt;resp.raw.decode_content = True&lt;/code&gt; is required because Kubearchive&#xA;returns gzip Content-Encoding; without it &lt;code&gt;ijson&lt;/code&gt; sees compressed bytes.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; &lt;code&gt;session.get().json()&lt;/code&gt; loads the full page into memory&#xA;(the original implementation).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; A single K8s list page can contain 500 PipelineRuns (80+ MB&#xA;JSON). Parsing this with &lt;code&gt;.json()&lt;/code&gt; creates a +247 MB RSS spike (raw&#xA;bytes + decoded string + parsed dict coexist). With 4 concurrent&#xA;workflows, this exceeds any reasonable pod memory limit. &lt;code&gt;ijson&lt;/code&gt;&#xA;stream-parsing yields items one at a time, reducing the peak to +12 MB&#xA;for the same data &amp;ndash; a 95% reduction. The continue token appears in&#xA;the &lt;code&gt;metadata&lt;/code&gt; object (before or after &lt;code&gt;items&lt;/code&gt; depending on the server);&#xA;the event-driven parser captures it regardless of order.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Large MRs (100+ components) would OOM the agent pod.&#xA;Concurrent workflows would multiply the problem. The pod memory limit&#xA;would need to scale with the largest possible page size, which is&#xA;unbounded.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;action-token-tier-for-per-workflow-identity&#34;&gt;Action token tier for per-workflow identity&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#action-token-tier-for-per-workflow-identity&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; A dedicated Tier 2 of workflow action tokens, separate&#xA;from both orchestrator tokens (Tier 1) and model tool tokens (Tier 3). Each workflow gets&#xA;a dedicated GitLab bot user per project for all write operations (notes,&#xA;thread resolution).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Extend orchestrator tokens with a per-workflow fallback&#xA;chain (&lt;code&gt;ORCHESTRATOR_&amp;lt;WORKFLOW&amp;gt;_GITLAB_TOKEN_&amp;lt;PROJECT&amp;gt;&lt;/code&gt; →&#xA;&lt;code&gt;ORCHESTRATOR_GITLAB_TOKEN_&amp;lt;PROJECT&amp;gt;&lt;/code&gt; → &lt;code&gt;ORCHESTRATOR_GITLAB_TOKEN&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Orchestrator tokens serve a fundamentally different purpose&#xA;(operational reads + infrastructure notes) than workflow output (analysis&#xA;results, review comments). Mixing them in one fallback chain risks a&#xA;single bot identity posting notes on behalf of multiple workflows,&#xA;eliminating the audit trail. A separate tier with explicit &lt;code&gt;token: str&lt;/code&gt;&#xA;parameters makes the code path unambiguous: callers must choose which&#xA;tier they use. The naming convention&#xA;(&lt;code&gt;HUMMINGBIRD_AGENT_ACTION_&amp;lt;WORKFLOW&amp;gt;_GITLAB_TOKEN_&amp;lt;PROJECT&amp;gt;&lt;/code&gt;) groups&#xA;tokens alphabetically by workflow, making env var auditing&#xA;straightforward.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; All workflow notes would appear from the same bot user,&#xA;making it impossible to distinguish which agent (code-review vs&#xA;analyze-failures vs renovate-babysit) produced a given note in the MR&#xA;timeline. GitLab&amp;rsquo;s audit log would attribute all actions to one identity.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;per-project-only-action-token-config-no-workflow-level-default&#34;&gt;Per-project-only action token config (no workflow-level default)&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#per-project-only-action-token-config-no-workflow-level-default&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;action_tokens&lt;/code&gt; is defined at the project level only in YAML&#xA;config. No workflow-level &lt;code&gt;action_tokens&lt;/code&gt; default is supported.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Allow workflow-level &lt;code&gt;action_tokens&lt;/code&gt; that projects&#xA;inherit by default, with per-project overrides.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; GitLab project access tokens are project-scoped. A&#xA;workflow-level default would be misleading &amp;ndash; a single token cannot&#xA;write to multiple projects. Forcing per-project declaration makes the&#xA;mapping explicit and prevents configuration errors where a cross-project&#xA;default silently fails.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Operators could set a workflow-level token that only&#xA;works for one project, causing silent 403 errors for other projects in&#xA;the same workflow.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;ssh-over-kubectl-exec-for-kubevirt-vms&#34;&gt;SSH over kubectl exec for KubeVirt VMs&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#ssh-over-kubectl-exec-for-kubevirt-vms&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; SSH (&lt;code&gt;ssh -T sandbox@ip command&lt;/code&gt;) for VM command execution.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; &lt;code&gt;virtctl console&lt;/code&gt; (serial console), KubeVirt websocket&#xA;API, or &lt;code&gt;qemu-guest-agent exec&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; VMs have no &lt;code&gt;kubectl exec&lt;/code&gt; equivalent. &lt;code&gt;virtctl console&lt;/code&gt; provides&#xA;a serial terminal, not a programmable exec channel. The websocket API&#xA;requires a VNC or serial connection. &lt;code&gt;qemu-guest-agent&lt;/code&gt; is not installed&#xA;in the VM image. SSH provides clean stdin/stdout/stderr separation,&#xA;timeout control, and is already well-tested in subprocess pipelines.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Would need &lt;code&gt;qemu-guest-agent&lt;/code&gt; in the image and the&#xA;&lt;code&gt;exec&lt;/code&gt; endpoint, which has poorer error reporting and no streaming stdin.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;secret-volume-for-ssh-key-injection&#34;&gt;Secret volume for SSH key injection&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#secret-volume-for-ssh-key-injection&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; Kubernetes Secret mounted as a virtio disk (serial&#xA;&lt;code&gt;ssh-pubkeys&lt;/code&gt;) read by the VM&amp;rsquo;s &lt;code&gt;inject-ssh-keys.service&lt;/code&gt; at boot.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; &lt;code&gt;cloudInitNoCloud&lt;/code&gt; or &lt;code&gt;accessCredentials&lt;/code&gt; with&#xA;&lt;code&gt;qemu-guest-agent&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; The VM image already has &lt;code&gt;inject-ssh-keys.service&lt;/code&gt; which reads&#xA;from the virtio disk. &lt;code&gt;cloudInitNoCloud&lt;/code&gt; requires cloud-init in the&#xA;image. &lt;code&gt;accessCredentials&lt;/code&gt; requires &lt;code&gt;qemu-guest-agent&lt;/code&gt;. The Secret&#xA;volume approach requires no additional software in the VM image.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Would need cloud-init or guest agent in the bootc&#xA;image, adding complexity and attack surface.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;vmirs-for-vm-pool-replenishment&#34;&gt;VMIRS for VM pool replenishment&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#vmirs-for-vm-pool-replenishment&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;VirtualMachineInstanceReplicaSet&lt;/code&gt; (VMIRS) as the&#xA;replenishment controller for the VM pool.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Custom controller, or VirtualMachine with RunStrategy.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; VMIRS is KubeVirt&amp;rsquo;s native equivalent of a Deployment&amp;rsquo;s&#xA;ReplicaSet &amp;ndash; it maintains a desired count of identical VMIs. When a&#xA;VMI is claimed (ownerReferences cleared), the VMIRS creates a&#xA;replacement automatically. Same pattern as the pod pool with Deployment.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; Would need a custom controller to maintain the VMI pool,&#xA;adding operational complexity.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3 id=&#34;per-workflow-sandbox-backend&#34;&gt;Per-workflow sandbox backend&lt;a class=&#34;td-heading-self-link&#34; href=&#34;#per-workflow-sandbox-backend&#34; aria-label=&#34;Heading self-link&#34;&gt;&lt;/a&gt;&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Chosen:&lt;/strong&gt; &lt;code&gt;sandbox:&lt;/code&gt; field in workflow config, resolved per-workflow&#xA;with fallback to CLI &lt;code&gt;--sandbox&lt;/code&gt; flag.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Alternative:&lt;/strong&gt; Global-only sandbox selection via CLI.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Why:&lt;/strong&gt; Different workflows have different isolation requirements.&#xA;Code review needs only a lightweight container, while failure analysis&#xA;may need a full VM with root access. Per-workflow selection avoids&#xA;running all workflows in VMs (wasteful) or all in containers&#xA;(insufficient isolation).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;If reversed:&lt;/strong&gt; All workflows would share the same sandbox backend,&#xA;requiring either over-provisioning (VMs for everything) or accepting&#xA;weaker isolation for some workflows.&lt;/li&gt;&#xA;&lt;/ul&gt;</description>
			</item>
	</channel>
</rss>
