Container Catalog Design Decisions

Design Decision Registry (DDR) for container-catalog — every major design choice, the alternatives considered, and why the chosen approach won. For operational usage, see Container Catalog.

DDR-1: Historical RELEASE_SBOM writer survives the SBOM cutover as a write-if-absent seed

Decision: When the index function takes over SBOM-item writes, sync’s build_release_sbom_item keeps its RELEASE_SBOM#{hex} construction, written via an atomic write-if-absent conditional write (put_content_versioned with attribute_not_exists(ver) — every post-cutover writer stamps ver, so the condition blocks any rewrite of normalized content); the historical branch of merge_release writes only digest-keyed items; SBOM#{canonical} becomes index-exclusive.

Context: The SBOM cutover (W1 MR2) moves SBOM-item writes from sync to index. Index only writes for digests current in TAGS; the SBOM backfill enumerates scan blobs. Digests that were never scanned while current therefore lose every writer. Production census (2026-11-11, 10 images, 10,697 history digests, vuln-item presence as the scan/blob proxy): 228 digests (2.1%) have an SBOM item but were never scanned — 225 from 2026-04, before the scan pipeline wrote blobs, plus ~1/month of new misses (events raced out of TAGS, dropped events). The registry tagging model (unique sha256-{hex}/timestamp tags per build; ~a dozen moving plain version tags) means every rebuild supersedes the previous digest out of TAGS, so this population accrues slowly but indefinitely. The only code path that can serve these digests is the catalog sync CLI’s historical sweep — the production sync Lambda processes only each event’s release and never sweeps. The sweep is an on-demand repair/repopulation tool (“Full DynamoDB population”, --purge).

Alternatives: (a) Accept the gap — delete the RELEASE_SBOM construction outright; SBOM items become exclusively the index/backfill domain; the frozen 225 keep their existing items, new misses get history and details but no SBOM, documented as a known gap. (b) Keep an unconditional retained write — preserve today’s behavior for the historical branch unchanged. (c) Extend the backfill to synthesize RELEASE_SBOM from attestations for no-blob digests.

Rationale: (a) rejected because a repair tool with a measured blind spot is an operational footgun: after an incident or data loss, a catalog sync run would leave SBOM holes nothing else can fill (no blob → the backfill enumerates blobs and skips them; no event → index never sees them). (b) rejected because the sweep population is dominated by ex-current, blob-backed digests that the SBOM backfill normalizes to scan-derived content; sync’s writes are unconditional (batch_write; put_content_versioned has no content comparison), so every CLI run would overwrite blob-derived items back to attestation-derived content, silently undoing the backfill. (c) rejected because the backfill is one-off by design — it cannot cover future misses, only the current cohort. The write-if-absent seed keeps the repair tool complete for exactly the population only it can reach, while the conditional write makes it converge with the backfill instead of fighting it: sync seeds once (attestation-derived, today’s field spellings — consistent with the accepted mixed-vocabulary story for no-blob digests), index and the backfill own the final content for blob-backed digests, and a CLI run never rewrites an item that already exists. Zero new IAM (one DDB read); ~20 lines retained plus the guard, confined to a CLI-only code path.

DDR-2: Index invocation converges on the queue-decoupled pattern

Decision: The internal index function drops its SNS Release subscription and consumes a queue of tag-release events; sync performs its DynamoDB writes and then enqueues one event per release canonical, using the shared TagReleaseEvent shape (the promoted ExternalTagReleaseEvent). The function is shaped as the future unified index — the same per-event processing structure as the external index, with the event shape and blob-key prefixes already shared — so the external plane’s unification (retiring external-index-lambda into this function) remains a small, well-bounded change.

Context: The internal index was SNS-triggered like sync, which forced it to read TAGS to learn a release’s canonicals — introducing the sync-fast/index-slow race (TAGS-absent → retry → DLQ) and requiring per-image error aggregation inside one invocation. The external plane had already solved invocation decoupling: its sync resolves tags, writes TAGS, then enqueues per-digest events to an index queue whose batch-item-failure semantics provide per-record retry for free. Downstream of sync’s writes the two planes are the same pattern; only the chain’s head differs (the internal producer is the near-realtime bus event, the external one a 6-hour poll).

Alternatives: (a) Keep the SNS trigger and heal the TAGS race with retries — the race class stays, one slow image can block the record, and the two index functions keep divergent shapes. (b) Unify into a single shared deployment rather than two deployments of one function — rejected for blast-radius isolation: a bad deployment would take out both planes at once. (c) Defer the convergence to the external-plane MR — rejected: every week of divergence adds code that both index functions must carry, and the queue convergence was already decided for the internal plane on its own merits.

Rationale: Queue decoupling deletes the TAGS-race class by construction: the event is emitted after the writes it depends on and carries everything the index needs (canonicals, digest, architectures), so the index never reads TAGS and canonical identity is re-resolved against the registry. SQS batch-item failures replace hand-rolled per-image aggregation with the platform’s own per-record retry. Sync redelivery is harmless — a duplicate event hits the index’s dedup skip and flows into the recovery path, which is the recovery’s designed purpose. The canonical re-verification stays regardless of ordering: the moved-digest mid-scan race is real in any invocation shape. Shaping the internal index as the unified function now (shared event type, external’s processing structure, env-configured prefixes) keeps MR4 small: point the external queue at the function and delete external-index-lambda, plus four parameterizations the planes still differ on — the DDB partition key (IMAGE# vs EXTERNAL#), the pull-reference source (config.pull_registry vs the raw registry reference), the arch source (manifest list via fetch_arch_attestations vs the event’s architectures), and the Hermeto-merge gating (classify_by_hermeto from DISTRO vs none). The event shape and the blob-key prefixes are already zero-code.