@tyroneross/build-loop 0.36.0 → 0.43.0

package/skills/model-tiering/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: model-tiering
-description: Use when choosing a model tier or segment for a subagent, deciding a role descriptor (segment + tier) in frontmatter, or escalating mid-flow. Covers the two-axis taxonomy (work-role segment × 7-rung capability ladder) — Opus/Sonnet/Haiku are Anthropic-default mappings; selection is provider-portable and data-driven. NOT for benchmarking models head-to-head on a task — use `model-bakeoff`.
+description: Recommend which model tier (e.g. Opus/Sonnet/Haiku) to use for a subagent or task. Use when the user asks "which model should I use", "is Sonnet enough or do I need Opus", or sets a segment+tier in agent frontmatter. Not for running multiple models head-to-head and scoring them (use `model-bakeoff`).
 user-invocable: false
 ---
 
@@ -26,25 +26,57 @@ Agents declare a `(segment, tier)` ROLE; the resolver (`scripts/model_resolver.p
 
 | Tier | Anthropic default | Role | Equivalents (advisory — verify benchmarks before swapping) |
 |---|---|---|---|
-| **Frontier** | Fable 5 | **Phase 2 Plan synthesis (frame goal, draft spec/ADRs, F-criteria, MECE partition) via the Advisor dispatch ladder when stakes-gated** — `advisor` agent / peer host / already-Fable session; honestly-labeled inline-Opus fallback otherwise (`references/advisor-dispatch-ladder.md`). (Advisor v1 = Phase 2 only; Phase 1 Assess synthesis runs inline as today until v2.) AND verification judgment (plan-critic, scope-auditor, independent-auditor, fix-critique, fact-checker, security-reviewer, overfitting-reviewer, promotion-reviewer) | GPT-5.5 Thinking (or whichever tier scores above the prior Thinking-tier ceiling), future Claude tier above Opus; any model that benchmarks above the Thinking-tier contract on SWE-bench Verified AND ARC-AGI / GPQA Diamond |
-| **Thinking** | Opus 4.8 | Coordination — build-orchestrator, assessment-orchestrator — and the escalation target for execution (ambiguous spec, 2 consecutive failures, cross-file surprise) and audit/learnings synthesis when Frontier is unavailable | GPT-5 Thinking, Gemini 2.5 Pro; any model >= Opus 4.6 on SWE-bench Verified + Frontier-class on ARC-AGI / MMLU-Pro |
-| **Code** | Sonnet 5 | Application — apply rule to bounded input, scoped implementation, mechanical refactor, bounded domain assessment | GPT-5 Codex, qwen2.5-coder-32B (local); any model with SWE-bench Verified within ~5pt of the Code-tier default (last published Anthropic Sonnet figure: 4.6 ~79.6%; Sonnet 5 reaches prior Opus-tier coding/agentic quality per claude-api T1) |
-| **Pattern** | Haiku 4.5 | Recognition — regex/syntactic match, classification into known buckets, log scan, deterministic checklist | Haiku 4.6, GPT-5 Mini, llama3.2-3b (local); any small/fast model that handles structured pattern matching |
+| **Frontier** | Opus 5 | **Phase 2 Plan synthesis via the Advisor dispatch ladder when stakes-gated**, plus every gating verification agent (`plan-critic`, `independent-auditor`, `security-reviewer`). Fable is the SECOND T1 entry, reached only via `scripts/frontier_gate.py` for planning on tightly-coupled repos | GPT-5.6 Sol; future models that clear the Thinking-tier contract and prior ceiling |
+| **Thinking** | Opus 5 | Coordination and escalation, plus medium-risk verification (`scope-auditor`, `fact-checker`, `fix-critique`, `overfitting-reviewer`, `promotion-reviewer`) | GPT-5.6 Terra for routine orchestration; GPT-5.6 Sol for the Governance/Evaluation T2 cell and genuinely ambiguous escalation |
+| **Code** | Sonnet 5 | Application — apply rule to bounded input, scoped implementation, mechanical refactor, bounded domain assessment | GPT-5.6 Terra, qwen2.5-coder-32B (local); any model within the Code-tier benchmark tolerance |
+| **Pattern** | Haiku 4.5 | Recognition — regex/syntactic match, classification into known buckets, log scan, deterministic checklist | GPT-5.6 Luna, Haiku-class or local small/fast models that handle bounded structured work |
 
 **Rule of substitution:** tier A's swap target must score within tolerance of the default on the benchmark relevant to its role. For Code tier that's SWE-bench Verified ≥75% AND tool-use accuracy ≥85%; for Thinking tier that's SWE-bench ≥78% AND ARC-AGI / GPQA Diamond competitive; for Frontier tier that's clearing the Thinking-tier contract AND scoring above the prior-generation Thinking-tier ceiling on at least one of SWE-bench Verified / ARC-AGI / GPQA Diamond; for Pattern tier no benchmark — just "fast and cheap, doesn't hallucinate on bounded structured tasks."
 
 **Why Frontier sits above Thinking for plan + verification (and not for execution):** wrong plans and wrong verdicts compound — a bad plan dispatches N implementers into the wrong work, and a bad verdict ships a regression. The user's standing priority is Accuracy > Speed > Cost (`feedback_accuracy_speed_cost_priority.md`), so the planning and verification surfaces — where one miscall poisons everything downstream — pay the Frontier premium. Execution and coordination stay on Sonnet/Opus because they're either bounded application (Sonnet implementer applies a settled plan) or routing (Opus orchestrator chooses which subagent runs next, with deterministic gates as the safety net).
 
+## Harness amplifies, does not replace, model capability
+
+A better harness (scaffold, tool-set, context budget, verify loop) lifts a weak model *up to* its capability ceiling — it does not push the model *past* it. Treat the harness as a large, routinely-undervalued multiplier, not a substitute for tier. The evidence:
+
+- **Variance decomposition** over 25k+ agent runs / 8 domains attributes ~41.4% of outcome variance to base-model reasoning vs ~1.5% to scaffold (a ~28:1 ratio) — the ceiling is a model property.
+- **SWE-agent** raised the *same* GPT-4 Turbo from 3.8% (RAG pipeline) to 12.5% (ACI scaffold) — a 3.3× harness lift, yet still only 12.5% *absolute*. A weak base stays absolutely unreliable on hard tasks no matter the harness.
+- Internal dogfood corroboration: after a transport/parser fix unblocked a small local model's tool calls (0→8 parsed), the *model* ceiling remained — it still could not produce compiling Rust.
+
+**Routing consequence (keep as-is):** route hard, open-ended reasoning UP to the frontier tier; route bounded, oracle-checkable work DOWN to local/small models. Do NOT invest harness effort expecting a small/local model to clear reasoning-heavy tasks — that is a ceiling failure, not a scaffold failure, and no amount of harness engineering fixes it. When a reliability failure appears (agent lies about success, doesn't follow through, unsafe action), route to harness/verify/guardrail fixes, not to a bigger prompt or a smaller-model-plus-better-scaffold gamble.
+
+Source: `build-loop-memory/research/2026-07-06-ai-coding-fundamentals-and-harness-claims.md` (Claim 4 — "MIXED, leaning CHALLENGE on the strong form"; variance-decomposition + SWE-agent + internal ledger).
+
 ## Provider-swap recipe
 
 Build-loop's agent frontmatter uses Anthropic model aliases (`fable`, `opus`, `sonnet`, `haiku`) because Claude Code is the primary host. To run on a different provider:
 
 1. **Edit the INDEX, not each agent.** `model:` frontmatter is index-DERIVED (generated by `scripts/sync_agent_model_defaults.py`), so do not hand-edit it. To swap providers, reorder the preferred list / change the default for the `(segment, tier)` cell in `references/model-taxonomy.json` (or classify a new model once via `scripts/classify_model_tier.py`), then run `python3 scripts/sync_agent_model_defaults.py --apply` to regenerate every agent's `model:`. The role's `(segment, tier)` is the durable key; the tier determines the substitution target.
-2. **Runtime override:** `.build-loop/config.json.modelOverrides` accepts `{ frontier: "<id>", thinking: "<id>", code: "<id>", pattern: "<id>" }`. The orchestrator resolves this through `scripts/model_overrides.py` before dispatching subagents (see `references/model-tier-mapping.md` for full schema). Configs without `frontier` resolve frontier → `fable` by default.
+2. **Runtime override:** `.build-loop/config.json.modelOverrides` accepts `{ frontier: "<id>", thinking: "<id>", code: "<id>", pattern: "<id>" }`. The orchestrator resolves this through `scripts/model_overrides.py` before dispatching subagents (see `references/model-tier-mapping.md` for full schema). Configs without `frontier` resolve frontier → the T1 default (`opus`).
 3. **Per-dispatch override:** any orchestrator dispatch may pass `model: <id>` in the subagent prompt to force that call.
 
 The role-and-task table below uses tier names. The Anthropic-default mapping in the right column is illustrative; substitute your equivalents at swap time.
 
+### Groq workload catalog (on demand)
+
+When the task asks which Groq API model or system should run a downstream workload, read both `references/provider-catalogs/groq.md` and `references/provider-catalogs/groq-models.json`. The first contains dated decision guidance; the second is the machine-readable snapshot and freshness contract.
+
+Keep this separate from host-agent reachability. A Groq catalog entry does not become an available Build Loop subagent model until a host adapter works and the model clears the role-specific benchmark. Reverify the catalog after its `review_after` date, on any deprecation change, and before production billing or capacity decisions because availability, pricing, rate limits, advertised throughput, and capability compatibility are dynamic.
+
+### GPT-5.6 Codex policy
+
+- **Sol** is approved for planning synthesis and gating verification agents: `advisor`, `plan-critic`, `scope-auditor`, `independent-auditor`, `fix-critique`, `fact-checker`, `security-reviewer`, `overfitting-reviewer`, and `promotion-reviewer`.
+- **Terra** is approved for ordinary orchestration, bounded implementation, domain assessment, advisory review, and Learn drafting. It is the Codex default for `build-orchestrator`, `assessment-orchestrator`, and `implementer`.
+- **Luna** is approved for bounded recognition agents: `mock-scanner`, `recurring-pattern-detector`, and `transcript-pattern-miner`. Prefer a deterministic script when the rule is fully expressible.
+- Start bounded coding and execution at Terra `high`; this is an advisory default and explicit per-dispatch effort remains valid. Start Luna at `low|medium` and Sol at `medium` (`high` for adversarial/security verdicts). Use `xhigh` only after evidence of a miss; reserve `max` and Ultra for rare cases with justified consequence or meaningful independent streams.
+- The live resolver returns `preferred_effort` and `effort_guidance` with every agent resolution. Dispatchers must record requested versus actual effort and the verifier result before promoting a workload to `xhigh` or `max`.
+- **Claude:** Sonnet, Opus, and Fable start at `high`; step to `xhigh` only after a verifier records a quality miss, then evaluate `max` only when the added quality warrants it. Haiku exposes no effort control, so keep it on bounded recognition work and escalate the model rather than inventing an effort retry.
+- Escalate Luna→Terra when rule application appears; Terra→Sol when ambiguity, high consequence, or repeated verified failure appears. Model size never weakens least privilege, confirmation, sandboxing, or independent verification.
+
+Source: OpenAI GPT-5.6 System Card (2026-07-09), retained in build-loop-memory with the full PDF and routing extract.
+
+**Claude exception:** `scope-auditor`, `fact-checker`, `fix-critique`, `overfitting-reviewer`, and `promotion-reviewer` use Opus at medium effort. `plan-critic`, `independent-auditor`, and `security-reviewer` moved to Opus on 2026-07-28 (commit `5e0122a`) — Fable is no longer the default for any verification role. The Codex mapping remains Sol for every gating verification agent.
+
 ## Chat-triggered index maintenance (host-LLM-driven)
 
 The model index (`references/model-taxonomy.json`) is **user-editable and chat-maintainable**. When the user expresses model intent in conversation, recognize it and act on the index directly — this is host-LLM-driven per the repo's "host coding agent is the LLM" rule: you recognize the intent and run deterministic scripts; there is NO vendor API call and NO hard hook. (A `UserPromptSubmit` hook that pre-detects these phrasings is an OPTIONAL future hardening, not required — the LLM recognizing intent is the mechanism.)
@@ -83,7 +115,7 @@ Dispatch always resolves the role LIVE through `resolve_agent_model.py`, so an i
 | (prior-gen baseline) Sonnet 4.6 uses 70% fewer tokens than 4.5 on complex file ops with +38% accuracy | Anthropic Sonnet 4.6 announcement | ⚠️ T2, single-source; superseded by Sonnet 5's new tokenizer |
 | Pricing: Sonnet 5 $3/$15 per MTok ($2/$10 introductory through 2026-08-31) | claude-api skill cache 2026-06-24 (T1 — Anthropic) | ⚠️ verify before billing |
 | Pricing: Opus 4.8 $5/$25 per MTok input/output | Anthropic pricing page | ⚠️ verify before billing |
-| Pricing: Fable 5 $10/$50 per MTok input/output (1M context, capability tier above Opus 4.8) | claude-api skill cache 2026-05-26 (T1 — Anthropic) | ✅ T1 source, advisory until re-confirmed at next billing audit |
+| Pricing: Fable 5 $10/$50 per MTok input/output (1M context; superseded as the T1 default by Opus 5 on 2026-07-28) | claude-api skill cache 2026-05-26 (T1 — Anthropic) | ✅ T1 source, advisory until re-confirmed at next billing audit |
 
 ## MECE primitive: cognitive type of the task
 
@@ -91,48 +123,58 @@ Before consulting the role table, classify the task by reasoning shape. The MECE
 
 | Reasoning shape | Model | What it means | Example tasks |
 |---|---|---|---|
-| **Planning + Verification synthesis** — frame the goal, draft the spec/ADRs, define F-criteria, MECE-partition the work, then later judge whether a plan, a commit, a fix, a claim, or a security/scope boundary actually holds | **Fable (Frontier)** | The "what to do" and "did it actually work" calls. Wrong calls poison every downstream dispatch. | Phase 2 Plan drafting (reaches Fable via the stakes-gated Advisor ladder; Phase 1 Assess synthesis stays inline until v2), plan-critic, scope-auditor, independent-auditor, fix-critique, fact-checker, security-reviewer, overfitting-reviewer, promotion-reviewer |
-| **Coordination + escalation synthesis** — route work between subagents, ladder severity, run causal-tree on stuck iterations, write audit/learnings | **Opus (Thinking)** | The "who runs next" + "why did the rule run out" calls. Deterministic gates backstop the routing. | build-orchestrator, assessment-orchestrator, severity ranking after critic findings, causal-tree after 2 consecutive failures, Phase 6 Learn audit synthesis (when no Frontier escalation needed) |
+| **Planning + highest-consequence verification synthesis** — frame the goal and judge verdicts where a miss has the largest blast radius | **Opus (Frontier)** | Wrong calls poison every downstream dispatch. | Phase 2 Plan drafting, plan-critic, independent-auditor, security-reviewer |
+| **Coordination + bounded verification synthesis** — route work, run causal trees, or judge a bounded surface with strong checks | **Opus (Thinking)** | Strong judgment backed by deterministic gates; also the T1 default since 2026-07-28. | build-orchestrator, assessment-orchestrator, scope-auditor, fact-checker, fix-critique, overfitting-reviewer, promotion-reviewer |
 | **Application** — apply a known rule, spec, or pattern to bounded input; produce an artifact that matches a contract | **Sonnet (Code)** | The "how" call when "what" is decided. Single-correct-answer derivable from a rule. | Implement a commit's owned files per spec, write tests for given F-criteria, mechanical simplify, bounded domain assessment (api/db/frontend/perf), design-contract reconciliation, ui-validator, retrospective-synthesizer, self-improvement-architect drafting |
 | **Recognition** — pure regex/syntactic match; classify into known buckets; no judgment | **Haiku (Pattern)** | No gradient — matches or doesn't. | Mock-data scan, log pattern detection, file inventory, cross-run pattern detection, deterministic checklist verification |
 
-**Decision tree:** "Does this task have a single-correct answer derivable from a rule applied to bounded input?" → Yes = Application/Sonnet. Else "Is the answer pure pattern-match?" → Yes = Recognition/Haiku. Else, Synthesis. Then ask: "Is this a planning decision (what to build) or a verification verdict (did it hold)?" → Yes = Frontier/Fable. Else (routing, escalation, audit-synthesis when no verdict is being rendered) = Thinking/Opus.
+**Decision tree:** "Does this task have a single-correct answer derivable from a rule applied to bounded input?" → Yes = Application/Sonnet. Else "Is the answer pure pattern-match?" → Yes = Recognition/Haiku. Else, Synthesis. Then ask: "Is this a planning decision (what to build) or a verification verdict (did it hold)?" → Yes = Frontier (currently Opus; Fable only through `frontier_gate.py` on tightly-coupled planning). Else (routing, escalation, audit-synthesis when no verdict is being rendered) = Thinking/Opus.
 
 ## Default assignments
 
 | Task | Reasoning shape | Model | effort | Why |
 |------|------|-------|--------|-----|
-| Frame & plan: goal, ADRs, scope, F-criteria, MECE partition | Planning synthesis | Fable | medium | A wrong plan dispatches N implementers into the wrong work; user's standing priority Accuracy > Speed > Cost |
+| Frame & plan: goal, ADRs, scope, F-criteria, MECE partition | Planning synthesis | Opus | medium | A wrong plan dispatches N implementers into the wrong work; user's standing priority Accuracy > Speed > Cost |
 | Plan-verify deterministic checklist | Recognition | (script) | — | No model; runs `plan_verify.py` |
-| Plan-critic adversarial review against rubric+checklist | Verification synthesis | Fable | high | Verification verdict — separation drives quality; verdict gates Phase 3 dispatch |
-| Scope auditor (Plan→Execute boundary): trace callers of every modified-API symbol; annotate `caller_audit:` per commit | Verification synthesis | Fable | medium | Cross-file call-path tracing AND a gating verdict on whether a commit is `internal_only`; verification compound risk |
-| Code execution — bounded chunk, spec clear | Application | Sonnet | medium | Default workhorse. Spec is settled; apply the rule |
+| Plan-critic adversarial review against rubric+checklist | Verification synthesis | Opus | high | Verification verdict — separation drives quality; verdict gates Phase 3 dispatch |
+| Scope auditor (Plan→Execute boundary): trace callers of every modified-API symbol; annotate `caller_audit:` per commit | Bounded verification synthesis | Opus | medium | Read-only caller tracing has a strong grep/plan oracle; Codex resolves Sol through Governance/Evaluation T2 |
+| Code execution — bounded chunk, spec clear | Application | Sonnet | high | Default workhorse. First-pass correctness reduces rework; explicit effort may override this preference. |
 | Code execution — ambiguous spec or cross-file surprise mid-execution | Coordination synthesis | Opus | medium | Escalation target; interpretation cost cheaper than rework |
-| Independent-auditor adversarial pass (read-only diff vs rubric at chunk + build scope) | Verification synthesis | Fable | high | Verdict gates the build's outcome line; a missed regression in production-impacting work is the most expensive miss in the loop |
+| Independent-auditor adversarial pass (read-only diff vs rubric at chunk + build scope) | Verification synthesis | Opus | high | Verdict gates the build's outcome line; a missed regression in production-impacting work is the most expensive miss in the loop |
 | Severity ranking + recommendation order (given findings) | Coordination synthesis | Opus | medium | Cross-finding routing; no per-finding verdict being rendered, the verdicts are upstream |
 | Mock data scanning | Recognition | Haiku | low | Regex only |
-| Fact-checking — trace metric → source, judge accuracy | Verification synthesis | Fable | medium | Final read on "is this number real" before report ships; user-trust verdict |
-| Fix-critique — pressure-test a proposed fix before "resolved" | Verification synthesis | Fable | medium | Verdict on whether the fix addresses root cause vs symptom; wrong verdict reopens the bug downstream |
-| Security-reviewer — adversarial OWASP/ATLAS pass | Verification synthesis | Fable | high | Verdict gates riskSurfaceChange dispatch; missed exposure is the most expensive verification miss |
-| Overfitting-reviewer — Goodhart / test-gaming verdict on optimize runs | Verification synthesis | Fable | medium | Verdict on whether optimization is genuine; cheap to wrong-call into a regression |
-| Promotion-reviewer — Phase 6 Learn experimental promotion verdict | Verification synthesis | Fable | medium | Gates the move from `experimental/` to `active/`; durable surface |
+| Fact-checking — trace metric → source, judge accuracy | Bounded verification synthesis | Opus | medium | Source trace supplies an external oracle; Codex resolves Sol |
+| Fix-critique — pressure-test a proposed fix before "resolved" | Bounded verification synthesis | Opus | medium | Counterfactual and regression evidence backstop the verdict; Codex resolves Sol |
+| Security-reviewer — adversarial OWASP/ATLAS pass | Verification synthesis | Opus | high | Verdict gates riskSurfaceChange dispatch; missed exposure is the most expensive verification miss |
+| Overfitting-reviewer — Goodhart / test-gaming verdict on optimize runs | Bounded verification synthesis | Opus | medium | Read-only experiment history and guard metrics provide the oracle; Codex resolves Sol |
+| Promotion-reviewer — Phase 6 Learn experimental promotion verdict | Bounded advisory verification | Opus | medium | Advisory verdict remains subject to explicit promotion confirmation; Codex resolves Sol |
 | Simplify — apply known simplifications | Application | Sonnet | medium | Inline single-use helper, delete dead branch — bounded |
 | Debugging — symptom-to-known-pattern match | Application | Sonnet | high | Memory-first gate's "Application until the rule runs out" |
 | Debugging — causal-tree after 2 consecutive failures | Coordination synthesis | Opus | high | Synthesis takes over routing when rule-match exhausts |
-| Novel architecture decision | Planning synthesis | Fable | medium | Cross-file impact; wrong call compounds |
+| Novel architecture decision | Planning synthesis | Opus | medium | Cross-file impact; wrong call compounds |
 | Writing user-facing prose (copy, microcopy, errors) | Coordination synthesis | Opus | medium | Tone, restraint, and nuance matter; no verification verdict being rendered |
 | Audit / learnings / Phase 6 audit synthesis | Coordination synthesis | Opus | medium | Cross-run routing; promotion-reviewer carries the gating verdict separately |
 | Recurring-pattern detection across runs[] | Recognition | Haiku | low | Pattern-match across structured logs |
 
 ### Deliberate exceptions (Sonnet retained for cost where the surface is high-frequency advisory)
 
-Two verification-shaped agents stay on Sonnet rather than escalating to Fable. The tension with round-2 evidence ("rubric-application = Sonnet is robust") is real; the user chose Fable for the rest of the verification surface anyway because the compound risk of a wrong verification verdict outweighs the per-call premium. Pins are defaults, not locks — these can be overridden per dispatch or re-tiered after telemetry.
+Two high-frequency advisory agents stay on Sonnet. Every verification agent — bounded and gating alike — now resolves to Opus. Pins are defaults, not locks and should be re-tiered when telemetry disagrees.
 
 | Agent | Pin | Why retained on Sonnet |
 |---|---|---|
 | `alignment-checker` | Sonnet | Called once per queue item during autonomous iterate (up to 25× per run). Advisory only — flags drift, doesn't gate. Cost dominates value at this fan-out frequency. |
 | `synthesis-critic` | Sonnet | Per-UI-commit WARN-only check. Advisory only — never gates. Frequency × non-gating shape means a cheaper tier is the right tradeoff. |
 
+### Database actions pin Frontier — standing user rule (2026-07-22)
+
+**All database actions run at Frontier tier (currently Opus on the Anthropic mapping), overriding the Code-tier execution default.** DB mutations — migrations, schema changes, constraint/index DDL, data-plane rehearsals, repair/quarantine — are irreversible-leaning and high blast-radius, exactly the compounding-error class Frontier owns. Scope:
+
+- **Assessment/diagnosis:** `database-assessor` role is `(generative_reasoning, frontier)` → resolves to Opus. Do not re-tier it down.
+- **Execution:** any Phase 2 plan chunk whose owned files include migrations, `schema.prisma`/`schema.sql`, or DDL scripts is assigned Frontier in plan model-assignment, not the Sonnet implementer default. Label the assignment in the plan.
+- **DB skills:** `data-plane-worktrees` and any database skill dispatch their executing subagent at Frontier tier (Agent `model` override / workflow `opts.model`).
+
+This is a provider-portable *tier* pin (Frontier), not a hardcoded id; on hosts where a higher-ranked Frontier id is available the resolver may pick it, honoring Accuracy > Speed > Cost. Source: user standing rule, example-app-ai migration rehearsal (run-593729).
+
 ## Round 2 evidence (2026-05-07, example-app news-podcast iteration 2)
 
 n=2 dispatch-pattern A/B comparison on a 6-commit feature reversed the round-1 belief that Skill-path (Sonnet fan-out) is materially cheaper across the board:
@@ -152,11 +194,11 @@ Findings that updated the model tiering:
 3. **Inline-Opus is faster wall-clock** when there's no real parallelism to exploit. Fan-out parallelism is only a win when ≥3 chunks are truly independent.
 4. **Plan-critic on Sonnet caught 17 substantive findings** on a written spec — confirms "rubric-application = Sonnet" is robust.
 
-These findings informed the earlier rubric-application=Sonnet split for code review. The current org overrides that for the verification surface specifically — the user chose Fable for verification because a missed verdict at this stage compounds, even though round-2 showed Sonnet rubric-application was substantively robust on a 17-finding plan-critic pass. The exceptions table above (alignment-checker, synthesis-critic) preserves the Sonnet split where the surface is high-frequency advisory and non-gating.
+These findings supported a three-level verification split. The 2026-07-28 audit bakeoff collapsed the top two levels: Opus 5 was a strict superset of Fable across 3/3 adversarial-audit rounds, so gating and bounded verdicts both sit on Opus, with Sonnet keeping high-frequency advisory checks. The exceptions table preserves Sonnet where the surface is non-gating.
 
-## Escalation triggers (Sonnet execution → Opus, NOT to Fable)
+## Escalation triggers (Sonnet execution → Opus)
 
-Execution escalates to **Opus**, not Fable. Fable is reserved for planning and verification; execution under genuine ambiguity is a coordination call (interpret the spec, route to a new chunk, decide whether to re-plan) that the orchestrator owns.
+Execution escalates to **Opus** — a coordination call the orchestrator owns. Fable is not an execution escalation target: it is reserved for super-complex multi-step work and for judging it, and is reached through `scripts/frontier_gate.py`, not by climbing the execution ladder. Several issues open at once is the signal that hands work to Fable; one hard task is not.
 
 - 2 consecutive failures on the same chunk after a retry at effort=high → respawn implementer at Opus
 - Spec is ambiguous and interpretation will materially change implementation → Opus
@@ -165,7 +207,7 @@ Execution escalates to **Opus**, not Fable. Fable is reserved for planning and v
 - Novel error pattern not found in `.build-loop/issues/` or debugging memory → Opus
 - Task produces user-visible prose where tone and restraint are load-bearing → Opus
 
-If the ambiguity surfaces a **planning** problem (the original plan no longer fits) rather than an execution problem, route back to Phase 2 Plan — Fable re-plans, then execution resumes on Sonnet/Opus.
+If the ambiguity surfaces a **planning** problem (the original plan no longer fits) rather than an execution problem, route back to Phase 2 Plan — the Advisor re-plans at Frontier (Fable if `frontier_gate.py` says the repo is tightly coupled, else Opus), then execution resumes on Sonnet.
 
 ## Techniques that work
 
@@ -182,11 +224,30 @@ If the ambiguity surfaces a **planning** problem (the original plan no longer fi
 - **Chain of Density**. Summarization-specific technique; not applicable to code work.
 - **Best-of-N by default**. Only on hard chunks. Blanket best-of-N wastes tokens on easy tasks where effort=high is sufficient and cheaper.
 
+## Prompting profiles — how to prompt the rung you resolved
+
+Model-tiering answers *which* model runs an agent. The `prompting_profiles` block in `references/model-taxonomy.json` answers *how to prompt it* — a separate axis, keyed by capability rung, carried on the same resolution path.
+
+**Lookup:** `python3 scripts/resolve_agent_model.py <agent> --json` returns `prompting_profile` alongside `model` — no second call. From code, `model_taxonomy.prompting_profile(tier)` reads the same block directly.
+
+| Rung | examples | constraint_posture | edge_case_handling | rationale | prompt_budget |
+|---|---|---|---|---|---|
+| T0 | omit | contextual | delegate | required | compressed |
+| T1 | omit | contextual | delegate | required | compressed |
+| T2 | minimal | contextual | delegate | required | standard |
+| T3 | worked | mixed | enumerate-known | recommended | full |
+| T4 | worked | directive-ok | enumerate-known | optional | full |
+| T5 | worked | directive-ok | enumerate-known | optional | full |
+
+Profile edits happen in `references/model-taxonomy.json`, not here — the same maintenance path as the model index itself; see §"Chat-triggered index maintenance" above.
+
+The T0–T2 rows are source-verified against the Anthropic Claude Code team's 2026-07-21 published guidance; T3's `examples`/`edge_case_handling` are repo-measured and the rest of T3 is inferred; T4/T5 are weakly-evidenced placeholders (`confidence: weak`, `status_quo: true`) encoding current behavior, not a chosen posture.
+
 ## Cost math quick reference
 
 > ⚠️ **Advisory only.** The numbers below are directional heuristics based on single-source token-profile estimates and public pricing as of the skill's last update. They are **not** verified against real usage telemetry and should not be used as hard routing logic. Treat them as "this tier costs roughly this much more than that tier," not as commitments. Pricing, token profiles, and model output lengths all drift over time. Before using these ratios in any cost-minimization decision, pull actual usage data from the last 30 days of builds and re-derive the numbers for your workload.
 
-**Price ratios only** (T1, claude-api — per-MTok, in/out): Fable 5 $10/$50 · Opus 4.8 $5/$25 · Sonnet 5 $3/$15 ($2/$10 intro through 2026-08-31) · Haiku 4.5 $1/$5.
+**Price ratios only** (T1, claude-api — per-MTok, in/out): Fable 5 $10/$50 · Opus 5 $5/$25 · Sonnet 5 $3/$15 ($2/$10 intro through 2026-08-31) · Haiku 4.5 $1/$5.
 
 | Pair (same task) | Price ratio | Basis |
 |---|---|---|
@@ -204,9 +265,9 @@ If the ambiguity surfaces a **planning** problem (the original plan no longer fi
 
 ## How the build-loop uses this
 
-**Fable plans (when stakes-gated) and verifies. Opus coordinates. Sonnet executes. Haiku recognizes.**
+**Fable plans and handles the highest-consequence verification. Opus coordinates and handles bounded verification. Sonnet executes. Haiku recognizes.**
 
-Phase 2 Plan synthesis reaches **Fable** through the **Advisor dispatch ladder** when stakes-gating trips (`synthesisDensity > 5`, `riskSurfaceChange`, `stakes >= medium`, or `dispatch_tier: frontier`): the orchestrator dispatches the `advisor` agent (Rung 1), routes to a peer host (Rung 2), or — if its own session is already Fable — synthesizes inline at Frontier (Rung 0). When no trigger fires or no dispatch path is reachable, the orchestrator synthesizes the plan **inline on its own model (Opus)** and labels it honestly (Rung 3 = today's behavior; the floor equals current state). So "Fable plans" is the *guarantee on high-stakes plans*, with an honestly-labeled inline fallback otherwise — not unconditional. Full protocol: `references/advisor-dispatch-ladder.md`. The Advisor frames the goal, drafts the spec/ADRs, sets F-criteria, and MECE-partitions the work. The orchestrator (**Opus**, `build-orchestrator`, `assessment-orchestrator`) coordinates: it routes dispatches, runs deterministic gates, manages parallel fan-out, walks the Advisor ladder, and handles the escalation ladder. Phase 3 implementer subagents run on **Sonnet** at effort=high (default workhorse; xhigh on hard/code-heavy chunks) → external verification gate (tests/lint/types) → adversarial **Fable** verification surface (`plan-critic`, `scope-auditor`, `independent-auditor`, `fix-critique`, `fact-checker`, `security-reviewer`, `overfitting-reviewer`, `promotion-reviewer`). On a first execution-problem chunk failure, retry the same Sonnet 5 implementer at **effort=xhigh** (intermediate rung — near-Opus coding at lower cost) before paying for Opus; a strong-checkpoint finding or a 2nd consecutive failure after the xhigh retry escalates to **Opus** for judgment; if the failure traces back to a planning miss, route back to Fable to re-plan. See `agents/build-orchestrator.md §Escalation Triggers`. The **tier mapping** is the policy; the cost numbers above are advisory context, not the basis for overrides.
+Phase 2 Plan synthesis reaches **Fable** through the **Advisor dispatch ladder** when stakes-gating trips. The orchestrator (**Opus**) coordinates; Phase 3 implementers run on **Sonnet** at high effort. Verification then splits by consequence: **Fable** for `plan-critic`, `independent-auditor`, and `security-reviewer`; **Opus** for `scope-auditor`, `fix-critique`, `fact-checker`, `overfitting-reviewer`, and `promotion-reviewer`; **Sonnet** for the advisory `alignment-checker` and `synthesis-critic`. On Codex, both governance verification cells resolve to GPT-5.6 Sol. Execution retries Sonnet at `xhigh` before escalating to Opus; planning misses route back to Fable.
 
 Haiku is only used for Phase 7B mock scanning and recurring-pattern detection across `runs[]`. Never for reasoning tasks.
 
@@ -214,7 +275,7 @@ Haiku is only used for Phase 7B mock scanning and recurring-pattern detection ac
 
 Not every agent should hard-pin its model. Use this rule:
 
-- **Pin** (`model: fable | opus | sonnet | haiku`) when the task has a clear right tier and cost/quality drift from user's session choice would be a bug. Examples: `plan-critic` / `independent-auditor` / `scope-auditor` / `fact-checker` / `fix-critique` / `security-reviewer` / `overfitting-reviewer` / `promotion-reviewer` (Fable — verification verdicts gate downstream work), `mock-scanner` (Haiku, pattern matching only), `build-orchestrator` and `assessment-orchestrator` (Opus, coordination at plan/review boundaries), `implementer` (Sonnet, default execution workhorse).
+- **Pin** (`model: fable | opus | sonnet | haiku`) when the task has a clear right tier and cost/quality drift from user's session choice would be a bug. `plan-critic`, `independent-auditor`, and `security-reviewer` pin Fable. `scope-auditor`, `fact-checker`, `fix-critique`, `overfitting-reviewer`, and `promotion-reviewer` pin Opus. `mock-scanner` pins Haiku; orchestrators pin Opus; `implementer` pins Sonnet.
 - **Inherit** (`model: inherit`) when user intent should flow through. The user's main-session choice is itself a cost/speed preference; respect it. Pair with a "recommended: X" note in this skill rather than forcing via frontmatter. Example: `root-cause-investigator` — recommended Opus on causal-tree work, but inherit honors whatever tier the user picked upstream.
 - **Override mechanism**: users can override any pin by passing `model:` when spawning the agent or by editing the frontmatter. Pins are defaults, not locks. The deliberate exceptions documented above (`alignment-checker`, `synthesis-critic` on Sonnet despite being verification-shaped) are exactly this kind of cost-vs-judgment pin and can be lifted if telemetry says so.
 
@@ -273,4 +334,4 @@ python3 ${CLAUDE_PLUGIN_ROOT}/scripts/model_overrides.py \
 
 Accepted tiers: `frontier` (default `fable`), `thinking` (default `opus`), `code` (default `sonnet`), `pattern` (default `haiku`). Configs without `frontier` resolve frontier → `fable` so older repos keep working without edits.
 
-Full contract and routing matrix: `~/dev/research/topics/llm/llm.build-loop-router-integration-2026-04.md`
+Full contract and routing matrix: "LLM Build-Loop Router Integration" (private research note, 2026-04 — substance summarized here; the live routing matrix ships in-repo at `references/model-taxonomy.json`).

package/skills/native-ax-driver/SKILL.md

@@ -47,6 +47,8 @@ Actions exposed by `python3 scripts/native_driver.py action`:
 
 Element targeting uses an integer index path from the main window root (e.g. `0,2,1` = first child → third child → second child). The path is returned by every `scan` element under the `path` key, so the typical loop is `scan` → match by `identifier` / `title` → use that element's `path` for `action`.
 
+`python3 scripts/native_driver.py launch` is the deterministic launch + pid-capture step: it starts an isolated instance of a `.app`/bundle id and returns the new pid to scope every subsequent `scan`/`action` call to. See "Single-instance PID-scoped verification mode" below.
+
 ## Files in this skill
 
 ```
@@ -54,7 +56,8 @@ skills/native-ax-driver/
 ├─ SKILL.md                                 (this file)
 ├─ scripts/
 │  ├─ layout_fill.py                        (layout-fill / gap analyzer; stdlib only)
-│  └─ native_driver.py                      (Python launcher; stdlib only)
+│  ├─ native_driver.py                      (Python launcher; stdlib only)
+│  └─ test_native_driver.py                 (pure-helper tests; run with pytest)
 └─ swift/bl-ax-driver/
    ├─ Package.swift                         (Swift 5.9, macOS 13+)
    └─ Sources/main.swift                    (~535 LOC, AX implementation)
@@ -85,10 +88,40 @@ Exit codes: `0` AX granted · `2` AX missing · `1` osascript missing.
 
 If `2`, surface to Iterate as a blocker rather than retrying — the user has to grant permission once in System Settings; build-loop cannot do that itself.
 
+### Single-instance PID-scoped verification mode
+
+**This is the documented standard for native UI verification whenever other instances of the target app may be running — including the user's own.** Do not skip verification in that situation, and do not target by `--app <name>` — `--app` matches by substring against every running process's name, so it can resolve to (and drive) the user's window instead of the isolated one being verified.
+
+The Swift driver already scopes every AX call to one process: it resolves the target via `AXUIElementCreateApplication(pid)` (`swift/bl-ax-driver/Sources/main.swift:264`) and only ever walks that process's AX tree. `scan --pid <pid>` and `action --pid <pid>` were always safe to run alongside ambient instances — the missing piece was a deterministic way to launch a fresh, isolated instance and know for certain which pid is it. `launch` closes that gap:
+
+1. **Launch** an isolated instance under a private state dir, forcing a brand-new process (`open -n`) and (optionally) ignoring any saved window/scene state (`-F` via `--fresh`) so it never resumes into the user's prior session:
+
+   ```bash
+   python3 .../native_driver.py launch --app-path /Applications/MyApp.app \
+       --state-env-var ET_STATE_DIR --state-dir /tmp/myapp-verify-$$  --fresh
+   ```
+
+2. **Read the captured pid** from the JSON result — `launch` snapshots the running GUI pid set before launch and diffs it against the pid set after, so the returned pid is provably the new instance, never a guess. If the diff is ambiguous (zero or more than one new pid appeared), `success` is `false` and no pid is reported — treat that as a hard stop, not a fallback to name-matching:
+
+   ```json
+   {"success": true, "pid": 55210, "bundle_id": "com.example.myapp", "state_dir": "/tmp/myapp-verify-1234", "fresh": true, "error": null, "next": "drive AX scoped to this pid: native_driver.py scan --pid 55210 / action --pid 55210 ..."}
+   ```
+
+3. **Scan and drive using ONLY `--pid <captured>`** for the remainder of the verification pass — never `--app`:
+
+   ```bash
+   python3 .../native_driver.py scan --pid 55210
+   python3 .../native_driver.py action --pid 55210 --element-path 0,2,1 --action press
+   ```
+
+Because AX targeting is pid-scoped end-to-end, every action in step 3 is confined to the launched instance's AX tree — the user's other windows, and any other ambient instances of the same app, are never touched. This is safe to run with ambient instances live.
+
+`launch` flags: `--app-path` (path to a `.app`) or `--bundle-id` (mutually exclusive, one required); `--state-dir` + `--state-env-var` (set an env var to a private state directory before launch, e.g. an app-specific `ET_STATE_DIR`); `--fresh` (pass `-F` to `open`); `--arg` (repeatable, extra argv passed to the launched app after `--args`); `--timeout` (seconds to wait for the new pid to appear, default `10.0`). Exit codes: `0` success · `1` launch or pid-capture failed · `2` bad arguments.
+
 ### Scan a running app
 
 ```bash
-python3 .../native_driver.py scan --app "Secrets Vault"      # by name (substring, case-insensitive)
+python3 .../native_driver.py scan --app "MyApp"      # by name (substring, case-insensitive)
 python3 .../native_driver.py scan --pid 44330                # by pid
 ```
 
@@ -169,8 +202,8 @@ Stdout JSON shape: `{"success": bool, "action": "press", "error": "AXPress faile
 `resolve` and `apps` work without AX permission — useful for the orchestrator to confirm a freshly-launched app has actually started before the AX-gated operations.
 
 ```bash
-python3 .../native_driver.py resolve --app "Secrets Vault"
-# {"pid": 44330, "name": "Secrets Vault", "bundleIdentifier": "com.secretsvault.app"}
+python3 .../native_driver.py resolve --app "MyApp"
+# {"pid": 44330, "name": "MyApp", "bundleIdentifier": "com.example.myapp"}
 
 python3 .../native_driver.py apps
 # [{"name": "Finder", "pid": 612, "bundleIdentifier": "com.apple.finder"}, ...]
@@ -189,7 +222,7 @@ python3 .../native_driver.py apps
 - **Web targets** — use `ui-validator` and the host browser/screenshot tooling; this skill won't help.
 - **iOS simulator** — the simulator runs on macOS, but interaction goes through `idb ui tap`, not direct AX (the simulator's AX surface is too noisy for path stability). See `reference_idb_sim_tap.md`.
 - **Drag-and-drop, hover-only effects, NSTrackingArea-driven UI** — these need real `CGEvent` mouse events. Out of scope. If the feature is critical, fix the AX surface in the app under test (add `.accessibilityAction { … }`) rather than synthesizing mouse events.
-- **App not yet running** — the driver does not launch apps. The orchestrator's pre-step must `open -b <bundleId>` (or `open <path/to/.app>`) and verify with `resolve` before driving.
+- **App not yet running** — use `launch` (see "Single-instance PID-scoped verification mode" above) to start an isolated instance and capture its pid before driving, especially when other instances of the app may already be running. For a single-instance context where no ambient collision risk exists, a plain `open -b <bundleId>` (or `open <path/to/.app>`) followed by `resolve` also works.
 
 ## Failure modes & recovery