@tyroneross/build-loop 0.30.3

package/skills/mcp-builder/references/mcp-security.md

@@ -0,0 +1,106 @@
+<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
+
+# MCP Security Model — stage-aware controls
+
+Derived from the NSA paper *"MCP: Security Design Considerations"* (May 2026) and the OWASP LLM / Agentic Top 10. This reference is the **rubric** the mcp-builder skill runs as a preflight and the `security-reviewer` agent grades against.
+
+> Source note: the control split and decision-table cells were specified by the build-loop plan that introduced this file, then cross-checked against the NSA paper directly (2026-05-22). The attack-class table and Tier 1/Tier 2 controls are faithful to the paper's documented concerns.
+
+## What the NSA paper documents (attack classes)
+
+| Attack class | What it is |
+|---|---|
+| Parameter injection | Untrusted/ambiguous input forwarded into a tool's typed parameters, escaping the intended instruction/data boundary |
+| Toolchain pivot / naming collision | A malicious or look-alike tool registers a colliding name, so calls intended for a trusted tool route to a parasitic one |
+| CVE-2025-49596 (RCE) | A remote-code-execution vulnerability in **MCP-Inspector**, the MCP-server testing toolchain: it accepted unverified inputs, letting a crafted message trigger RCE. Fixed in version 0.14.1. The paper cites it as proof that well-known weaknesses resurface when AI toolchains skip security hygiene. |
+| Missing RBAC | No access-control layer; any caller gets any tool at full privilege |
+| Opaque approval drift | The set of capabilities a server exposes changes over time without a re-approval workflow, so the user's original consent silently no longer matches reality |
+
+## Output contract
+
+mcp-builder emits a checklist split into two lists:
+
+- **"Mandatory for this build"** — all of Tier 1, plus every Tier 2 cell that resolved to `mandatory-now`.
+- **"Plan-now, deferred"** — every Tier 2 cell that resolved to `design-now-implement-later` (interface stubbed, trust boundary documented, implementation deferred).
+
+The checklist is **advisory**. It routes to the build-loop run report (Notes from judges / `state.json`). It is **never** an `AskUserQuestion` and **never** a hard gate — per build-loop's "advisory checks are automated" rule.
+
+---
+
+## Tier 1 — always mandatory
+
+These apply to **every** MCP server regardless of app type or dev stage — including a throwaway consumer prototype. They are cheap, and skipping them is the difference between "small bug" and "RCE".
+
+1. **JSON-schema parameter validation.** Every tool's `inputSchema` is a strict, specific JSON Schema. Validate arguments against it before the tool body runs. Reject on mismatch.
+2. **Block parameter forwarding from ambiguous/user-supplied sources.** Never pass a raw user string, prompt fragment, or unvalidated LLM output straight into a tool parameter that reaches a filesystem path, shell, URL, query, or another tool. Re-derive the value from a trusted, typed source.
+3. **Insecure-deserialization hygiene.** No `pickle`, no `eval`-backed parsers, no `Function(...)` over inbound data. Deserialize only with schema-bound, format-restricted parsers. (Part of the untrusted-input-reaches-executor class — the failure mode behind CVE-2025-49596.)
+4. **Unique / pinned tool identifiers.** Tool names are stable, namespaced, and collision-resistant. Pin the server identity. This is the naming-collision + parasitic-toolchain defense.
+5. **Tool-execution sandboxing.** Run tool bodies under least privilege — seccomp / AppArmor / a restricted user / a container — so a compromised tool cannot reach the host.
+6. **stderr-only logging.** `stdout` is the JSON-RPC channel. Anything else corrupts the protocol *and* can leak data into the transport.
+7. **No network at startup.** Do not validate credentials or call out in the constructor / `initialize`. Lazy-init on first tool call. (Also a load-time-latency fix.)
+8. **Treat every tool output as untrusted input to the next stage.** A tool's return value is attacker-influenceable. Validate it before it feeds a prompt, another tool, or a renderer. (Cascading-failure defense.)
+9. **Task / data isolation across trust zones — no shared mutable context.** Tools and connected servers must not blend context. A server only sees the data its current task needs; do not pass the whole client context to every server, and do not let one server's output silently become another's task input without origin + scope verification. This is the paper's *unverified task propagation* / cross-server context-bleed risk — demonstrated by the WhatsApp-MCP exploit, where a malicious server connected alongside a trusted one coerced the client into leaking message data. Per-task scoping also limits blast radius when a single server is compromised.
+
+---
+
+## Tier 2 — stage / type-gated decision table
+
+Each control resolves to one of:
+
+- `mandatory-now` — implement in this build.
+- `design-now-implement-later` — document the trust boundary and stub the interface now; implement before the next stage.
+- `optional` — defense-in-depth; note it, do not require it.
+
+`app_type` ∈ {`consumer`, `enterprise`}. `stage` ∈ {`prototype`, `MVP`, `growth`, `production`}.
+
+| Control | consumer / prototype | consumer / MVP | consumer / growth+prod | enterprise / prototype | enterprise / MVP | enterprise / growth | enterprise / production |
+|---|---|---|---|---|---|---|---|
+| Access control + RBAC | optional | optional | design-now-implement-later | design-now-implement-later | design-now-implement-later | mandatory-now | mandatory-now |
+| Token lifecycle (refresh / revoke / rotate) | optional | design-now-implement-later | mandatory-now | design-now-implement-later | mandatory-now | mandatory-now | mandatory-now |
+| Capability-change approval workflow | optional | optional | design-now-implement-later | optional | design-now-implement-later | mandatory-now | mandatory-now |
+| Signed + replay-protected messages | optional | optional | optional | optional | design-now-implement-later | design-now-implement-later | mandatory-now |
+| Audit-log depth (→ SIEM) | optional | optional | design-now-implement-later | design-now-implement-later | design-now-implement-later | mandatory-now | mandatory-now |
+| Egress filtering proxy / DLP | optional | optional | optional | optional | design-now-implement-later | design-now-implement-later | mandatory-now |
+| Local-only MCP instance for sensitive data | optional | optional | design-now-implement-later (if `data_sensitivity = high`) | design-now-implement-later | mandatory-now (if `data_sensitivity = high`) | mandatory-now (if `data_sensitivity = high`) | mandatory-now (if `data_sensitivity = high`) |
+| Rate limiting / DoS protection | optional | design-now-implement-later | mandatory-now | design-now-implement-later | mandatory-now | mandatory-now | mandatory-now |
+
+**Worked example — RBAC (the user's case).** A consumer-type server resolves RBAC to `optional`. An enterprise server still at MVP resolves it to `design-now-implement-later`: document the trust boundary, stub the RBAC interface, but do not block the MVP on a full implementation. An enterprise server in production resolves it to `mandatory-now`.
+
+When `data_sensitivity = high`, escalate any `optional` cell in that control's row by one step (`optional` → `design-now-implement-later`, `design-now-implement-later` → `mandatory-now`).
+
+---
+
+## Signal inference rules (no config file)
+
+`app_type`, `stage`, and `data_sensitivity` are **inferred from repo signals** — there is no security config file to author or maintain. Run these against the target repo:
+
+### app_type — `enterprise` vs `consumer`
+
+`enterprise` if **either** holds; otherwise `consumer`:
+
+- An auth library is present (`better-auth`, `@clerk/*`, `next-auth`, `@auth/*`, `passport`, `@supabase/auth-*`, `lucia`, …) **AND** multi-tenant signals exist: an `org` / `organization` / `team` / `workspace` / `tenant` table or model, or RBAC scaffolding (a `roles` / `permissions` / `policies` table, a `role`-typed column, a `casl` / `oso` / `permit` dependency).
+- Regulated-data keywords appear in the repo: `HIPAA`, `SOC2`, `PCI`, `EHR`, `PHI`, `GDPR` with a DPA, or domain models clearly holding medical / financial / government records.
+
+### stage — `prototype` → `MVP` → `growth` → `production`
+
+Derive from the *highest* tier whose signals are present:
+
+| Stage | Signals |
+|---|---|
+| `prototype` | No tests, no CI config, no deploy manifest. Often a single dev. |
+| `MVP` | A test directory with real tests, OR a CI workflow file (`.github/workflows/*`). |
+| `growth` | CI **and** a deploy manifest (`vercel.json`, `railway.json`, `fly.toml`, `Dockerfile` + a deploy step), OR a staging environment. |
+| `production` | All of the above **and** a monitoring SDK (`@sentry/*`, `posthog`, `datadog`, OpenTelemetry) **or** a live custom domain referenced in config. |
+
+### data_sensitivity — `high` vs `normal`
+
+`high` if the repo handles PII, EHR/PHI, financial records, or secrets beyond its own service credentials — detected via table/model names (`patients`, `ssn`, `card_number`, `payment_methods`, `medical_*`), regulated-data keywords, or a secrets-management dependency wired to user data. Otherwise `normal`.
+
+---
+
+## How security-reviewer consumes this
+
+When `triggers.riskSurfaceChange` is set **and** the diff introduces a new MCP server, the `security-reviewer` agent grades the diff against:
+
+- **Tier 1** — always, every control. A missing Tier 1 control is at least `HIGH`.
+- **Tier 2** — only the cells that the inferred `app_type` / `stage` / `data_sensitivity` resolved to `mandatory-now`. A missing `mandatory-now` control is a finding; a missing `design-now-implement-later` control is `LOW` (note it, do not block) provided the interface stub + boundary doc exist.

package/skills/model-tiering/SKILL.md

@@ -0,0 +1,216 @@
+---
+name: model-tiering
+description: Use when choosing a model tier for a subagent, deciding code-tier vs thinking-tier in frontmatter, or escalating mid-flow. Covers the multi-model abstraction — Opus/Sonnet/Haiku are Anthropic-default mappings; the tier abstraction is provider-portable.
+user-invocable: false
+---
+
+<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
+
+# Model Tiering — Authoritative Reference for Build-Loop
+
+Governs model selection across all build-loop phases. Build-loop is **multi-model** — the role assignments below are tier-based (thinking / code / pattern), not provider-specific. Anthropic's Opus/Sonnet/Haiku is the default mapping; equivalents from other providers map to the same tiers.
+
+Use `references/agent-role-taxonomy.md` for responsibility boundaries. This
+skill answers "which tier should run the role?", not "who owns the work?".
+
+## Tier abstraction (canonical)
+
+| Tier | Anthropic default | Role | Equivalents (advisory — verify benchmarks before swapping) |
+|---|---|---|---|
+| **Thinking** | Opus 4.7 | Synthesis, planning, ambiguity resolution, severity ranking, audit/learnings, cross-file judgment | GPT-5 Thinking, Gemini 2.5 Pro, future Claude tier; any model >= Opus 4.6 on SWE-bench Verified + Frontier-class on ARC-AGI / MMLU-Pro |
+| **Code** | Sonnet 4.6 | Application — apply rule to bounded input, scoped implementation, adversarial critic, mechanical refactor | Sonnet 4.7+, GPT-5 Codex, qwen2.5-coder-32B (local); any model with SWE-bench Verified within ~5pt of Sonnet 4.6 (currently ~79.6%) |
+| **Pattern** | Haiku 4.5 | Recognition — regex/syntactic match, classification into known buckets, log scan, deterministic checklist | Haiku 4.6, GPT-5 Mini, llama3.2-3b (local); any small/fast model that handles structured pattern matching |
+
+**Rule of substitution:** tier A's swap target must score within tolerance of the default on the benchmark relevant to its role. For Code tier that's SWE-bench Verified ≥75% AND tool-use accuracy ≥85%; for Thinking tier that's SWE-bench ≥78% AND ARC-AGI / GPQA Diamond competitive; for Pattern tier no benchmark — just "fast and cheap, doesn't hallucinate on bounded structured tasks."
+
+## Provider-swap recipe
+
+Build-loop's agent frontmatter uses Anthropic model aliases (`opus`, `sonnet`, `haiku`) because Claude Code is the primary host. To run on a different provider:
+
+1. **One-time edit per agent:** open each `agents/*.md` and change the `model:` field to your provider's equivalent. The tier (Thinking/Code/Pattern) determines the substitution target.
+2. **Runtime override:** `.build-loop/config.json.modelOverrides` accepts `{ thinking: "<id>", code: "<id>", pattern: "<id>" }`. The orchestrator resolves this through `scripts/model_overrides.py` before dispatching subagents (see `references/model-tier-mapping.md` for full schema).
+3. **Per-dispatch override:** any orchestrator dispatch may pass `model: <id>` in the subagent prompt to force that call.
+
+The role-and-task table below uses tier names. The Anthropic-default mapping in the right column is illustrative; substitute your equivalents at swap time.
+
+## When to use this skill
+
+- Choosing `model:` field in an agent frontmatter
+- Orchestrator deciding which subagent tier to spawn for a task
+- Estimating cost tradeoffs before starting a build
+- Deciding whether to escalate mid-flow after failures
+- Evaluating whether to swap providers (use the Tier abstraction table above as the contract)
+
+## Evidence base (2026 Q1)
+
+| Claim | Source | Certainty |
+|-------|--------|-----------|
+| Sonnet 4.6: 79.6% SWE-bench Verified | Anthropic announcement + SWE-bench leaderboard | ⚠️ T2, single-source |
+| Opus 4.6: 80.8% SWE-bench Verified (1.2pt gap — smallest in Claude history) | Same | ⚠️ T2, single-source |
+| Sonnet 4.6 uses 70% fewer tokens than 4.5 on complex file ops with +38% accuracy | Anthropic Sonnet 4.6 announcement | ⚠️ T2, single-source |
+| Pricing: Sonnet $3/$15 per MTok input/output | Anthropic pricing page | ⚠️ verify before billing |
+| Pricing: Opus $15/$75 per MTok input/output (5x gap) | Anthropic pricing page | ⚠️ verify before billing |
+
+## MECE primitive: cognitive type of the task
+
+Before consulting the role table, classify the task by reasoning shape. The MECE cut is the kind of thinking the task requires; lifecycle stage (plan/execute/review) is a second-order cut that often mixes types.
+
+| Reasoning shape | Model | What it means | Example tasks |
+|---|---|---|---|
+| **Synthesis** — combine N inputs into a novel decision; cross-file/cross-system reasoning; ambiguity resolution; severity ranking | **Opus** | The "what" and "why" calls. No single rule produces the answer. | Frame goal, draft spec/ADRs, trace call-paths across files, rank finding severity, escalate stuck iteration with causal-tree, write audit/learnings |
+| **Application** — apply a known rule, spec, or pattern to bounded input; produce an artifact that matches a contract | **Sonnet** | The "how" call when "what" is decided. Single-correct-answer derivable from a rule. | Implement a commit's owned files per spec, write tests for given F-criteria, adversarial critic vs rubric, mechanical simplify, fact-check with named source |
+| **Recognition** — pure regex/syntactic match; classify into known buckets; no judgment | **Haiku** | No gradient — matches or doesn't. | Mock-data scan, log pattern detection, file inventory, cross-run pattern detection, deterministic checklist verification |
+
+**Decision tree:** "Does this task have a single-correct answer derivable from a rule applied to bounded input?" → Yes = Application/Sonnet. Else "Is the answer pure pattern-match?" → Yes = Recognition/Haiku. Else = Synthesis/Opus.
+
+## Default assignments
+
+| Task | Reasoning shape | Model | effort | Why |
+|------|------|-------|--------|-----|
+| Frame & plan: goal, ADRs, scope, F-criteria, MECE partition | Synthesis | Opus | medium | Ambiguity resolution; wrong plan compounds |
+| Plan-verify deterministic checklist | Recognition | (script) | — | No model; runs `plan_verify.py` |
+| Plan-critic adversarial review against rubric+checklist | Application | Sonnet | high | Bounded — rubric is the rule. Separation drives quality |
+| **Scope auditor (NEW — Plan→Execute boundary)**: trace callers of every modified-API symbol; annotate `caller_audit:` per commit | Synthesis | Opus | medium | Cross-file call-path tracing that fanned-out implementers can't do (round-2 lesson) |
+| Code execution — bounded chunk, spec clear | Application | Sonnet | medium | Default. High accuracy, 5× cheaper |
+| Code execution — ambiguous spec | Synthesis | Opus | medium | Interpretation cost cheaper than rework |
+| Adversarial critic pass (read-only diff vs rubric) | Application | Sonnet | high | Bounded — rubric vs diff is rule-application; separation effect |
+| Code review — severity ranking + recommendation order (given findings) | Synthesis | Opus | medium | Cross-finding judgment about what matters most |
+| Mock data scanning | Recognition | Haiku | low | Regex only |
+| Fact-checking — trace metric → source, judge accuracy | Synthesis | Opus | medium | Cross-system; "is this number real?" requires cross-context judgment |
+| Fact-checking — trace metric → named-source pattern (rule-bound) | Application | Sonnet | medium | When the source-pattern rule is explicit |
+| Simplify — apply known simplifications | Application | Sonnet | medium | Inline single-use helper, delete dead branch — bounded |
+| Debugging — symptom-to-known-pattern match | Application | Sonnet | high | Memory-first gate's "Application until the rule runs out" |
+| Debugging — causal-tree after 2 consecutive failures | Synthesis | Opus | high | Synthesis takes over when rule-match exhausts |
+| Novel architecture decision | Synthesis | Opus | medium | Cross-file impact; wrong call is expensive |
+| Writing user-facing prose (copy, microcopy, errors) | Synthesis | Opus | medium | Tone, restraint, and nuance matter |
+| Audit / learnings / Phase 6 promotion-decision | Synthesis | Opus | medium | Cross-run synthesis |
+| Recurring-pattern detection across runs[] | Recognition | Haiku | low | Pattern-match across structured logs |
+
+## Round 2 evidence (2026-05-07, example-app news-podcast iteration 2)
+
+n=2 dispatch-pattern A/B comparison on a 6-commit feature reversed the round-1 belief that Skill-path (Sonnet fan-out) is materially cheaper across the board:
+
+| Dimension | A — Skill-path / Sonnet fan-out | B — Agent-tool / inline-Opus |
+|---|---|---|
+| Wall-clock | ~70-80 min | ~30 min |
+| Token mix | ~50% Opus orchestrator + ~50% Sonnet implementer | ~100% Opus single-context |
+| News-podcast jest | 52/0 | 59/0 |
+| Iterations needed | 1 (cross-file prop wiring missed by Sonnet) | 0 |
+| LLM judge speaker-flow | not run live (proxy from B) | 4/4 × 3 samples |
+
+Findings that updated the model tiering:
+
+1. **Sonnet implementers are scoped to `files_owned`** and miss cross-file integration gaps (the AIBriefPage→PodcastGenerator props case). This motivates the new Scope Auditor role above.
+2. **Orchestrator overhead (research dispatches, plan-critic, iterate coordination, audit) burns ~50% of total tokens at Opus rate** on small/medium features. Sonnet's lower per-token rate doesn't dominate at 6-commit scale.
+3. **Inline-Opus is faster wall-clock** when there's no real parallelism to exploit. Fan-out parallelism is only a win when ≥3 chunks are truly independent.
+4. **Plan-critic on Sonnet caught 17 substantive findings** on a written spec — confirms "rubric-application = Sonnet" is robust.
+
+These findings inform the role assignments, especially the rubric-application=Sonnet vs severity-assessment=Opus split for code review.
+
+## Escalation triggers (stay on Sonnet UNLESS)
+
+- 2 consecutive failures on the same chunk after a retry at effort=high
+- Spec is ambiguous and interpretation will materially change implementation
+- A cross-file architectural decision surfaces mid-execution that wasn't in the plan
+- Critic flags a "strong-checkpoint" finding that requires judgment, not just a fix
+- Novel error pattern not found in `.build-loop/issues/` or debugging memory
+- Task produces user-visible prose where tone and restraint are load-bearing
+
+## Techniques that work
+
+- **Self-refine with external verification** (tests, lint, type-check). External oracle is non-negotiable — without it, self-refine is circular.
+- **Adversarial critic loop** (writer agent + read-only reviewer agent). Separation is what makes it work. Same model reviewing its own output doesn't catch errors.
+- **Best-of-N sampling with self-certainty voting** on HARD chunks only (flagged by plan or first-pass failure). N=3. Cost = ~3x Sonnet, still under 1.5x single-pass Opus.
+- **Test-time compute** (effort=medium default, effort=high on retry). Easier problems benefit from revisions; harder problems need parallel sampling — not just more thinking on one path.
+- **Plan-then-execute split** (Opus plans once, Sonnet executes many). Established pattern. Amortizes Opus cost across N subagent calls.
+
+## Techniques to avoid
+
+- **Multi-agent debate** (3+ agents argue toward consensus). ⚠️ ICLR 2025 MAD analysis shows majority voting captures most gains; debate adds cost without consistent wins. Use simple voting instead.
+- **Self-critique without adversarial separation**. A model editing its own output won't reliably catch its own errors. Use a separate read-only reviewer or rely on external tests.
+- **Chain of Density**. Summarization-specific technique; not applicable to code work.
+- **Best-of-N by default**. Only on hard chunks. Blanket best-of-N wastes tokens on easy tasks where effort=high is sufficient and cheaper.
+
+## Cost math quick reference
+
+> ⚠️ **Advisory only.** The numbers below are directional heuristics based on single-source token-profile estimates and public pricing as of the skill's last update. They are **not** verified against real usage telemetry and should not be used as hard routing logic. Treat them as "this tier costs roughly this much more than that tier," not as commitments. Pricing, token profiles, and model output lengths all drift over time. Before using these ratios in any cost-minimization decision, pull actual usage data from the last 30 days of builds and re-derive the numbers for your workload.
+
+| Configuration | Relative cost (advisory) |
+|---------------|--------------|
+| Single-pass Opus | ~5x baseline |
+| Single-pass Sonnet 4.6 | ~0.3x (70% fewer tokens in observed samples) |
+| Sonnet 4.6, effort=high | ~0.6x |
+| Sonnet 4.6, best-of-3 + critic | ~1.2x |
+| Sonnet 4.6 best-of-3 + critic vs single-pass Opus | ~4x cheaper |
+
+❓ Best-of-N + critic vs single-pass Opus on SWE-bench has not been directly benchmarked.
+
+**How to convert these into routing decisions**: don't. Use the numbers to sanity-check a tier choice after the fact ("was this worth the 5x?"), not to justify forcing a model swap. When real telemetry disagrees with this table, trust telemetry and file an issue to update the table.
+
+## How the build-loop uses this
+
+Orchestrator (Opus 4.7) spawns implementer subagent (Sonnet, effort=medium) → external verification gate (tests/lint/types) → Sonnet critic agent (read-only, effort=high) → if strong-checkpoint flagged, escalate to Opus for judgment call. See `agents/build-orchestrator.md §Escalation Triggers`. The **tier mapping** is the policy; the cost numbers above are advisory context, not the basis for overrides.
+
+Haiku is only used for Phase 7B mock scanning. Never for reasoning tasks.
+
+## Pin vs inherit in agent frontmatter
+
+Not every agent should hard-pin its model. Use this rule:
+
+- **Pin** (`model: opus | sonnet | haiku`) when the task has a clear right tier and cost/quality drift from user's session choice would be a bug. Examples: `independent-auditor` (Sonnet advisory judge across chunk + build scope, consolidated 2026-05-23 — replaces retired `commit-auditor` and earlier `sonnet-critic`), `mock-scanner` (pattern matching only), `build-orchestrator` (Opus judgment at plan/review boundaries).
+- **Inherit** (`model: inherit`) when user intent should flow through. The user's main-session choice is itself a cost/speed preference; respect it. Pair with a "recommended: X" note in this skill rather than forcing via frontmatter. Example: `fact-checker` — recommended Sonnet, but inherit honors whatever tier the user picked upstream.
+- **Override mechanism**: users can override any pin by passing `model:` when spawning the agent or by editing the frontmatter. Pins are defaults, not locks.
+
+Forward-compat note: pinned family aliases (`sonnet`, `opus`) auto-track latest versions (e.g., 4.6 → 4.7). `inherit` additionally picks up brand-new tiers (e.g., a future Flash-class model) without frontmatter edits.
+
+## Limitations of this guidance
+
+- ⚠️ Sonnet 4.6 token-efficiency claim is single-source (Anthropic announcement). Treat as directionally correct, not proven.
+- ❓ Best-of-N + critic hasn't been tested against single-pass Opus on SWE-bench specifically.
+- ⚠️ Escalation triggers are heuristics, not proven thresholds. Revise after observing 5+ real builds and logging outcomes to `.build-loop/memory/`.
+
+## When to consult `model-router`
+
+For Phase 3 (Execute) sub-agent dispatch, prefer the standalone router over inline tier reasoning when it is available:
+
+```bash
+python3 ~/.claude/scripts/model-router.py \
+  --task "<one-line task summary>" \
+  --complexity auto \
+  --phase execute \
+  --task-id "<task-id-for-cost-ledger>" \
+  --json
+```
+
+The router returns:
+
+```json
+{
+  "provider": "ollama-mcp" | "codex" | "claude",
+  "model": "<model-id>",
+  "tool_call": {"name": "<mcp-tool-name>", "args": {...}},
+  "reason": "...",
+  "evidence_refs": ["<paths to docs that justify this decision>"]
+}
+```
+
+Why prefer the router:
+- **Evidence-cited**: every decision references the doc that supports it (DOE results, model-tiering policy, cost-ledger design)
+- **Deterministic**: same input → same output, auditable across builds
+- **task_id propagates** to MCP tool args, so `cost-ledger-reader.py --by-task` shows per-build-phase economics
+- **Free**: heuristic-only, no LLM call to decide
+
+Fallback when router is unavailable: use the inline tier rules above. The router's policy mirrors them, so behavior is consistent either way.
+
+Always resolve the final tier to a concrete model before writing the dispatch
+cost-ledger row:
+
+```bash
+python3 ${CLAUDE_PLUGIN_ROOT}/scripts/model_overrides.py \
+  --workdir "$PWD" \
+  --tier code \
+  --fallback sonnet \
+  --json
+```
+
+Full contract and routing matrix: `~/dev/research/topics/llm/llm.build-loop-router-integration-2026-04.md`

package/skills/native-ax-driver/SKILL.md

@@ -0,0 +1,220 @@
+---
+name: build-loop:native-ax-driver
+description: Use when the build needs to automate a macOS .app without touching the hardware cursor, or the user asks to "click through the app" or "test the UI headlessly". Drives running apps via Accessibility API; self-contained Swift binary — no IBR, Playwright, or Appium required.
+version: 1.1.0
+user-invocable: false
+---
+
+<!-- SPDX-FileCopyrightText: 2025-2026 Tyrone Ross, Jr <46267523+tyroneross@users.noreply.github.com> | SPDX-License-Identifier: Apache-2.0 -->
+
+# Native AX Driver
+
+**Built-in capability**, not a bridge. Build-loop ships its own Swift binary and Python launcher for navigating macOS apps via the Accessibility API. When this skill loads, the orchestrator can read the AX tree of any running `.app` and dispatch actions on individual elements **while the user keeps using their cursor for something else**.
+
+## Why this exists
+
+Build-loop's Iterate phase needs to verify native macOS UI fixes the same way it verifies web routes. Previously, build-loop deferred to IBR's MCP for native automation, which made native verification IBR-dependent. This skill lifts the needed capability into build-loop directly: same Swift code lineage, same AX actions, no MCP hop, no plugin requirement.
+
+Build-loop still routes to IBR as the primary verifier when `skills/ibr-bridge/SKILL.md` detects it, because IBR can add session management, baselines, and screenshots as auxiliary evidence. This skill is the native fallback path when IBR is absent.
+
+## What "cursor-free" means
+
+The Swift binary calls `AXUIElementPerformAction` and `AXUIElementSetAttributeValue` on the resolved element directly. It does **not** invoke `CGEventCreateMouseEvent`, `CGEventPost`, or `IOHIDPostEvent`. Concretely:
+
+| Path | API | Cursor moves? |
+|---|---|---|
+| **This skill** | macOS Accessibility (`AXUIElement*`) | ❌ Never |
+| Virtual HID fallback (NOT bundled) | Quartz Event Services / IOKit HID | ✅ Yes |
+| AppleScript "click at {x,y}" | CGEvent under the hood | ✅ Yes |
+
+When an element has no AX action handler (rare in SwiftUI/AppKit; common in custom Metal/canvas surfaces), the driver returns `Element not found at path` or an action-specific error rather than secretly falling through to mouse synthesis. This is intentional — a missing AX handler is usually a real accessibility bug in the app being tested, and papering over it hides the defect.
+
+## What the driver supports
+
+Actions exposed by `python3 scripts/native_driver.py action`:
+
+| Action | What it does | macOS AX constant |
+|---|---|---|
+| `press` | Click a button, activate a control | `kAXPressAction` |
+| `setValue` | Set a text-field / slider / picker value (requires `--value`) | `AXSetAttributeValue(kAXValueAttribute, …)` |
+| `increment` | Step a stepper / slider up | `kAXIncrementAction` |
+| `decrement` | Step a stepper / slider down | `kAXDecrementAction` |
+| `showMenu` | Open a contextual / popup menu | `kAXShowMenuAction` |
+| `confirm` | Default button activation in a dialog | `kAXConfirmAction` |
+| `cancel` | Cancel-button activation in a dialog | `kAXCancelAction` |
+| `focus` | Move keyboard focus to the element | `AXSetAttributeValue(kAXFocusedAttribute, true)` |
+| `scrollToVisible` | Scroll an element into the visible viewport | `AXScrollToVisible` |
+
+Element targeting uses an integer index path from the main window root (e.g. `0,2,1` = first child → third child → second child). The path is returned by every `scan` element under the `path` key, so the typical loop is `scan` → match by `identifier` / `title` → use that element's `path` for `action`.
+
+## Files in this skill
+
+```
+skills/native-ax-driver/
+├─ SKILL.md                                 (this file)
+├─ scripts/
+│  ├─ layout_fill.py                        (layout-fill / gap analyzer; stdlib only)
+│  └─ native_driver.py                      (Python launcher; stdlib only)
+└─ swift/bl-ax-driver/
+   ├─ Package.swift                         (Swift 5.9, macOS 13+)
+   └─ Sources/main.swift                    (~535 LOC, AX implementation)
+```
+
+The Swift binary compiles on first use to the **consumer project's** `.build-loop/bin/bl-ax-driver` — never inside the plugin tree. Subsequent runs reuse the cached binary; rebuild fires only if `Sources/main.swift` or `Package.swift` is newer than the cached binary.
+
+## Prerequisites
+
+| Prereq | How to check | Failure mode |
+|---|---|---|
+| `swift` on PATH | `command -v swift` | `RuntimeError: \`swift\` not found on PATH` from `ensure_binary()` |
+| Xcode CLT installed | `xcode-select -p` | First `swift build` fails with missing-SDK error |
+| AX permission for parent process | `python3 native_driver.py preflight` | All AX calls return `kAXErrorAPIDisabled`; binary exits with the canonical "Accessibility permission required" message |
+| Target app running | `python3 native_driver.py apps` | `findMainWindow` returns nil; binary exits with "No windows found for pid …" |
+
+The parent process needing AX permission is whichever process invoked Claude Code (Terminal, iTerm, VS Code, etc.). The driver binary itself does not need to be in the AX list — permission inherits from the parent.
+
+## Operational protocol
+
+### Pre-flight (run once per build-loop run with a native target)
+
+```bash
+python3 ${CLAUDE_PLUGIN_ROOT}/skills/native-ax-driver/scripts/native_driver.py preflight
+```
+
+Exit codes: `0` AX granted · `2` AX missing · `1` osascript missing.
+
+If `2`, surface to Iterate as a blocker rather than retrying — the user has to grant permission once in System Settings; build-loop cannot do that itself.
+
+### Scan a running app
+
+```bash
+python3 .../native_driver.py scan --app "Secrets Vault"      # by name (substring, case-insensitive)
+python3 .../native_driver.py scan --pid 44330                # by pid
+```
+
+Stdout: `WINDOW:<id>:<WxH>:<title>` header followed by the window root's children as a JSON array of `AXExtractedElement` (`role`, `subrole`, `title`, `identifier`, `value`, `enabled`, `focused`, `actions[]`, `position`, `size`, `children[]`, `path[]`).
+
+### Analyze layout-fill / gap findings
+
+```bash
+# Analyze a saved Swift scan JSON array
+python3 .../native_driver.py analyze-layout --from-file /tmp/ax-tree.json
+
+# Accept raw scan stdout with the leading WINDOW:<id>:<WxH>:<title> header
+python3 .../native_driver.py scan --app "Easy Terminal" \
+  | python3 .../native_driver.py analyze-layout --stdin
+
+# Re-scan a running app or pid and analyze the result
+python3 .../native_driver.py analyze-layout --app "Easy Terminal"
+python3 .../native_driver.py analyze-layout --pid 44330
+```
+
+`analyze-layout` catches the layout bug class where a content element renders narrow and centered inside a larger container, leaving large empty gutters that can be invisible in screenshots. It reads the existing Swift scan JSON only; no Swift change or rebuild is required.
+
+Inputs:
+- `--from-file`, `--stdin`, `--pid`, or `--app` are mutually exclusive.
+- `--threshold` defaults to `0.12`.
+- `--min-container-px` defaults to `50`.
+
+The analyzer returns the bridge envelope shape from `skills/ibr-bridge/SKILL.md`:
+
+```json
+{
+  "status": "ran",
+  "route": "native",
+  "verifier": "native-ax-driver",
+  "artifacts": ["stdin"],
+  "verification": "native-ax-driver analyze-layout ran; found 1 layout-fill finding.",
+  "findings": [
+    {
+      "severity": "warning",
+      "category": "structure",
+      "message": "layout-fill: AXSplitGroup [Main]: leading empty band 317px = 30% of container width 1074px (horizontal)",
+      "finding": {
+        "containerRole": "AXSplitGroup",
+        "containerLabel": "Main",
+        "axis": "horizontal",
+        "emptyPx": 317.0,
+        "emptyPct": 0.2951582867783985,
+        "position": "leading",
+        "containerWidth": 1074.0,
+        "containerHeight": 700.0,
+        "detail": "AXSplitGroup [Main]: leading empty band 317px = 30% of container width 1074px (horizontal)"
+      }
+    }
+  ]
+}
+```
+
+The Swift extractor emits absolute screen coordinates, which are correct for this analysis. Each computed value is an intra-container delta, so a constant origin offset cancels out: `firstChild.min - container.min`, `container.max - lastChild.max`, and `band / container.extent`.
+
+### Drive an element
+
+```bash
+# Press a button at AX index path 0,2,1 in pid 44330
+python3 .../native_driver.py action --pid 44330 --element-path 0,2,1 --action press
+
+# Type into a text field
+python3 .../native_driver.py action --pid 44330 --element-path 0,4,0 \
+    --action setValue --value "BUILD_LOOP_TEST"
+
+# Open the popup menu of a NSPopUpButton
+python3 .../native_driver.py action --pid 44330 --element-path 0,1,3 --action showMenu
+```
+
+Stdout JSON shape: `{"success": bool, "action": "press", "error": "AXPress failed" | null}`. Exit 0 on success, 1 on AX failure, 2 on bad arguments.
+
+### Resolve a name without AX permission
+
+`resolve` and `apps` work without AX permission — useful for the orchestrator to confirm a freshly-launched app has actually started before the AX-gated operations.
+
+```bash
+python3 .../native_driver.py resolve --app "Secrets Vault"
+# {"pid": 44330, "name": "Secrets Vault", "bundleIdentifier": "com.secretsvault.app"}
+
+python3 .../native_driver.py apps
+# [{"name": "Finder", "pid": 612, "bundleIdentifier": "com.apple.finder"}, ...]
+```
+
+## Integration with build-loop phases
+
+| Phase | Use |
+|---|---|
+| **Sub-step B Validate** (Review) — when uiTarget kind = `native-macos` | Run: `preflight` → `scan` → drive critical-path actions → re-`scan` → diff. |
+| **Sub-step D Coverage gaps** | Enumerate `scan` results; for any element with `actions != []` and no corresponding repo-native render/interaction coverage, write a `.build-loop/ux-queue/` entry with a proposed test step pinned by `identifier` (preferred) or `title`. |
+| **Phase 5 Iterate** — `.swift` files under a macOS target | Re-launch the rebuilt `.app` (`open -b <bundleId>`); replay the failing element-path + action; if two consecutive iterations fail on the same element, escalate to root-cause-investigator with the AX path — common cause is a missing `.accessibilityIdentifier(...)` modifier. |
+
+## When *not* to use this skill
+
+- **Web targets** — use `ui-validator` and the host browser/screenshot tooling; this skill won't help.
+- **iOS simulator** — the simulator runs on macOS, but interaction goes through `idb ui tap`, not direct AX (the simulator's AX surface is too noisy for path stability). See `reference_idb_sim_tap.md`.
+- **Drag-and-drop, hover-only effects, NSTrackingArea-driven UI** — these need real `CGEvent` mouse events. Out of scope. If the feature is critical, fix the AX surface in the app under test (add `.accessibilityAction { … }`) rather than synthesizing mouse events.
+- **App not yet running** — the driver does not launch apps. The orchestrator's pre-step must `open -b <bundleId>` (or `open <path/to/.app>`) and verify with `resolve` before driving.
+
+## Failure modes & recovery
+
+| Symptom | Likely cause | Fix |
+|---|---|---|
+| Binary missing after first install | `swift build` failed silently (sandbox rejection) | `ensure_binary()` retries with `--disable-sandbox` automatically; if both fail, install Xcode Command Line Tools |
+| All AX calls return `kAXErrorAPIDisabled` | Parent process not in System Settings → Privacy & Security → Accessibility | Add the process; macOS 13+ requires a fresh launch after granting |
+| `Element not found at path` after a UI change | Path indices shifted | Re-`scan` and look up the element by `identifier` again — paths are not stable across UI changes |
+| `AXPress failed` on a clearly-clickable button | SwiftUI view missing `.accessibilityAction` | Treat as a real bug in the app under test, not a driver bug |
+| iOS simulator AX tree is empty | Simulator was scanned in macOS mode | Pass `--device-name` to scope to the simulator window |
+
+## Self-test
+
+```bash
+python3 ${CLAUDE_PLUGIN_ROOT}/skills/native-ax-driver/scripts/native_driver.py preflight
+python3 ${CLAUDE_PLUGIN_ROOT}/skills/native-ax-driver/scripts/native_driver.py apps | head
+# pick a running app, then:
+python3 ${CLAUDE_PLUGIN_ROOT}/skills/native-ax-driver/scripts/native_driver.py resolve --app "Finder"
+python3 ${CLAUDE_PLUGIN_ROOT}/skills/native-ax-driver/scripts/native_driver.py scan --app "Finder" | head -40
+```
+
+If all four print sensible JSON, the skill is healthy.
+
+## Provenance
+
+Swift extractor ported from `interface-built-right/src/native/swift/ibr-ax-extract/Sources/main.swift`. The two copies started identical; build-loop's copy can drift independently and is not auto-synced. If a future bug is fixed in IBR's copy, port it manually and bump this skill's `version`.
+
+Layout-fill / gap analysis in `scripts/layout_fill.py` is ported from `interface-built-right/src/native/layout-fill.ts` v1.4.0. Its fixtures mirror `interface-built-right/src/native/layout-fill.test.ts`, especially the Easy Terminal regression case: a 440px terminal centered in a 1074px container yields `emptyPx == 317`, `emptyPct ~= 0.2952`, and `position == "leading"`. The analyzer is pure Python and has no live AX dependency.