aiki-cli 0.2.2 → 0.3.0

package/CHANGELOG.md

@@ -1,23 +1,114 @@
 # Changelog
 
-## Unreleased
+## 0.3.0 — 2026-07-15 — evidence-grounded decision council
+
+This release turns idea refinement into a bounded, evidence-linked decision workflow with explicit modes,
+selective rebuttal, a reader-first dossier, and stronger recovery from malformed provider output. The frozen
+code-review result remains unchanged: 100% vs 77% seeded-bug recall at equal precision on the pre-registered
+10-case holdout. The idea-v3 evaluation harness ships here, but its paid benchmark is still pending and no
+idea-quality lift is claimed yet.
 
 ### Added
-- **Contextual intent preflight** — idea-refinement now starts with an S0 run brief: the analyst generates
-  3-4 context-specific questions, the TUI asks them before the main council work, and the answers are
-  persisted in `00b-run-brief.json` and included in downstream prompts.
+- **Explicit idea modes and adaptive budgets (R6)** — `aiki run idea-refinement --mode
+  quick|council|research` defaults to council without a learned router. Quick is one structured analyst and
+  never presents itself as a council; council uses a 6-call base with at most two graph-triggered extras;
+  research permits up to four extras and enables the verified `codex --search exec ...` capability only on
+  Codex scout calls while retaining `-s read-only`. Mode-aware defaults are 4/10/12 calls respectively.
+- **Two-view decision preflight (R6)** — two readings run in parallel and deterministically merge into one
+  user-confirmed or visibly headless/defaulted decision contract. The old S1/S2/S3 model calls are gone;
+  analyst prompts are filled deterministically. Receipts split calls into discovery, verification, repair,
+  and planning, and resume preserves the original mode.
+- **Evidence and calculation integrity (R4)** — idea runs accept `--evidence <file|directory>` and persist
+  only source paths + SHA-256 hashes; evidence cards enforce source/freshness rules; derived numeric claims
+  can carry a pure arithmetic ledger whose values/units are recomputed; S8 emits typed claim verification;
+  and invalid evidence references are rejected before the chair call.
+- **Selective rebuttal and evidence-linked chair (R5)** — only verdict-flipping graph nodes enter one
+  anonymous rebuttal round; council coverage-fill and rebuttal share a two-call cap while chair/planner
+  budget stays reserved; responses append as immutable `08b-rebuttals.json` events; and the chair must emit
+  graph-linked rulings, recommendation reasons, conditions, pivots, and strongest counter-cases. A judge-
+  authored node is excluded before the chair prompt and remains unresolved under degradation.
+- **Startup preflight** — typing bare `aiki` now runs the full doctor before the home screen: per-provider
+  progress rows checking CLI presence, version, and auth/quota (smoke, cached 6h). Fewer than 2 providers
+  ready shows a failure screen with the exact fix per provider; a single degraded provider shows a warning
+  on the home screen and the council continues with the remaining quorum.
+- **Idea-lane bench resume** — `aiki bench idea-refinement --set build --resume [--yes]` continues the
+  latest campaign file in `bench/results/`: completed case×rotation pairs are kept (never re-paid), missing
+  or failed pairs re-run, and new observations append to the same file. The dry-run estimate reflects only
+  what is left to run. `--case <id>` restricts the metered run to one build case (an unknown id fails loud
+  with the list of valid cases), and a run that completes `low_diversity` (a scout seat died mid-run) is
+  rejected instead of being recorded — it is not a valid rotation sample.
+- **Adjudication import** — `aiki bench idea-refinement --import <file>` (offline, no provider calls)
+  imports blind adjudications: each `{ case_id, rotation, adjudication }` entry flows through the frozen R0
+  scorer and fills that pair's null recall/precision in the campaign file. Unknown pairs fail loud;
+  re-scoring an already-scored pair is refused (blind adjudication is one pass); when every pair is scored
+  it prints the lane default selection.
 - **Idea report v3** — idea-refinement reports now emit an explicit BLUF recommendation
   (`PROCEED`, `PROCEED_WITH_CONDITIONS`, `PIVOT`, `STOP`), conditions when needed, a best-effort
   12-dimension scorecard, assumption audit table in HTML, deterministic debate narrative, anchored
   validation plan with kill signals (`09b-action-plan.json`), open questions, red-team note, and a
   call/provider receipt. The Markdown copy button includes the expanded brief.
+- **R7 decision dossier** — `10-decision-report.json` now persists the graph-anchored recommendation chain,
+  evidence source/date/freshness/verification table, genuine disagreement and append-only position-change
+  events, coverage and sensitivity ledgers, executable experiments, strongest counter-case, strictly
+  verified unique-provider contributions, categorized receipt, and technical graph fold. Final Markdown,
+  HTML, and Copy-Markdown render from that same dossier; R6-era and older runs retain their legacy HTML. It
+  provides the canonical report foundation refined by the reader-first snapshot below.
+- **Reader-first decision snapshot (report v3.1)** — terminal, Markdown, and HTML lead with the council
+  recommendation, independently verified evidence coverage, decisive facts, first action, strongest
+  counter-case, and three critical unknowns. Financial and threshold-heavy chairs can emit strict
+  graph-anchored decisive numbers, explicit payback, option commitments marked `KNOWN`, `TARGET_CAP`, or
+  `UNKNOWN`, and one go/no-go tripwire. Invalid claim anchors remove the snapshot instead of presenting
+  unsupported numbers.
+- **Idea-v3 benchmark harness** — the frozen B/C/D2/R protocol now has an eight-case build set, a
+  12-case/tag/provenance holdout contract that remains closed until freeze, checkpoint/resume for successes
+  and failures, isolated baseline-provider campaigns, three independently shuffled blind-rating packets,
+  one-pass scoring, and hash-locked freeze and holdout guards. Build tuning, blind ratings, and the paid
+  holdout evaluation remain pending.
 
 ### Changed
-- Idea-refinement run estimate is now ~12 provider calls / ~4 Claude-Opus calls because S0 writes the
-  intent preflight and the judge seat also writes the validation plan. The default budget is now 13 so
-  a normal run still has room for one repair without skipping the validation plan.
+- **Three-level decision report** — idea-refinement's final report is restructured: (1) a one-screen
+  terminal summary (recommendation, decision state, verified evidence coverage, decisive result, dissent,
+  next action, and optional tripwire) printed after `aiki run`; (2) a 10-section reader-first graph-backed
+  Decision Report markdown (`final-report.md`) ordered as decision, action plan, reasoning, what could change
+  the decision, evidence, risks/gaps, dissent, council value, run details, and technical audit; (3)
+  machine-readable `10-decision-report.json` that Markdown and HTML render from, so the
+  surfaces cannot disagree. Statuses are ACCEPTED / ACCEPTED_WITH_CONDITIONS / INCONCLUSIVE / REJECTED
+  (mapped from the judge's recommendation). The structural score is moved behind technical detail and never
+  presented as decision accuracy: 40% verification coverage + 25%
+  independent convergence + 20% evidence quality + 15% stability − critical-risk penalty; model
+  self-confidence never enters it and consensus alone can never reach the High band. Labeled a heuristic in
+  the report until benchmark-calibrated.
+- Idea-refinement estimates are mode-aware: quick ≈3 calls / 1 Claude-Opus; council 6–8 / ~2 Opus; research
+  8–10 / ~2 Opus. Chair and planner calls are reserved before optional graph work.
+- **Mode-aware time limits** — quick and council retain a 20-minute run deadline; research uses 45 minutes.
+  The per-provider-call ceiling is 15 minutes, so legitimate deep scout work can finish while the overall
+  deadline still bounds the run.
 
 ### Fixed
+- Idea analyst outputs now canonicalize observed Gemini evidence-enum variants before strict validation:
+  freshness accepts case-insensitive canonical words, while evidence support accepts a leading known token
+  (`SUPPORT`, `OPPOSE`, or `OPPOSES`) and maps it to the canonical enum. Arbitrary prose and unknown values
+  still fail the schema boundary.
+- A failed S4 repair no longer kills the run for recoverable shape damage: deterministic salvage strips
+  unknown keys, drops invalid evidence/calculation/coverage/question entries, removes only individually
+  invalid positions, and scrubs their references. It never invents content and an empty claim set remains
+  fatal. The same fallback applies when the repair call itself dies (for example, quota).
+- All schema-validated model stages now attempt lossless coercion before a paid repair (for example, wrapping
+  a lone array item) and bounded lossy coercion only after repair failure (for example, truncating beyond a
+  declared maximum). The full zod schema still decides whether data may cross the stage boundary.
+- Provider timeouts now resolve immediately after killing the process group and unref a surviving detached
+  child, preventing one escaped subprocess from blocking the CLI long after the configured timeout.
+- Idea prose containing ordinary words such as `class`, `export`, or `import` no longer misroutes to code
+  review; routing now requires actual code structure such as a diff, fence, code file path, declaration, or
+  import/export syntax.
+- Decisive warnings no longer echo an unresolved affirmative claim as reassurance, and the calculation
+  checker now canonicalizes ordinary plurals, dimensionless ratios, and currency-margin units without
+  weakening arithmetic validation.
+- Benchmark safeguards now redact receipt costs and degradation tokens from blind packets, refuse accidental
+  same-day campaign overwrites, and honor the frozen baseline provider when a holdout run does not pass an
+  explicit override.
+- Release builds now clean `dist/` before compiling, so removed stages cannot survive as stale JavaScript in
+  the npm tarball.
 - Codex provider smoke no longer crashes in non-git folders; Aiki now passes Codex's verified
   `--skip-git-repo-check` flag while keeping `-s read-only`.
 - `aiki --version` now reads from `package.json`, preventing CLI/package version drift.

package/README.md

@@ -7,18 +7,20 @@
 <p align="center"><em>A local <strong>model council</strong> for code review and idea stress-testing — driven by the AI CLIs you already have.</em></p>
 
 <p align="center">
+  <a href="https://www.npmjs.com/package/aiki-cli"><img alt="npm: aiki-cli" src="https://img.shields.io/npm/v/aiki-cli.svg"></a>
   <img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-blue.svg">
   <img alt="Node ≥ 20" src="https://img.shields.io/badge/node-%E2%89%A5%2020-brightgreen.svg">
   <img alt="Local-first, no API keys" src="https://img.shields.io/badge/local--first-no%20API%20keys-informational.svg">
   <img alt="Read-only orchestration" src="https://img.shields.io/badge/orchestration-read--only-success.svg">
-  <img alt="Tests" src="https://img.shields.io/badge/tests-261%20passing-success.svg">
+  <img alt="Tests" src="https://img.shields.io/badge/tests-381%20passing-success.svg">
 </p>
 
 ---
 
-**aiki** runs the AI coding CLIs you already have installed and logged in (Claude Code, Codex, Antigravity/Gemini)
-as a **panel that can genuinely disagree** — they review independently, cross-examine each other, a judge
-adjudicates the disputes, and you get a clear decision brief.
+**aiki** is a local-first AI model council. It does **not** use hosted Aiki APIs, ask for API keys, or ship
+model weights. Instead, it runs the AI coding CLIs you already have installed and logged in — Claude Code,
+Codex, and Antigravity/Gemini — as a **panel that can genuinely disagree**. They review independently,
+cross-examine each other, a judge adjudicates the disputes, and you get a clear decision brief.
 
 It does two jobs, well:
 
@@ -28,7 +30,7 @@ It does two jobs, well:
 aiki is **not** a general assistant. Trivia and chat are routed away, not answered — a council adds cost, not
 accuracy, when there's one right answer.
 
-**Jump to:** [Why](#why) · [Benchmark](#benchmark) · [Requirements](#requirements) · [Install](#install) · [Quickstart](#quickstart) · [The two workflows](#the-two-workflows) · [Example](#example-a-real-idea-run) · [Configuration](#configuration) · [Sessions & resume](#sessions--resume) · [Safety](#safety-model) · [Costs & limits](#costs--limits) · [How it works](#how-it-works)
+**Jump to:** [Why](#why) · [What's new](#whats-new-in-030) · [How it works](#how-aiki-works-no-apis) · [Benchmarks](#benchmarks) · [Requirements](#requirements) · [Install](#install) · [Quickstart](#quickstart) · [The two workflows](#the-two-workflows) · [Example](#example-a-real-idea-run) · [Configuration](#configuration) · [Sessions & resume](#sessions--resume) · [Safety](#safety-model) · [Costs & limits](#costs--limits)
 
 ---
 
@@ -43,7 +45,41 @@ stop copy-pasting between them by hand.
   <img src="docs/Three.png" alt="One model's field of view lets bugs slip past; three overlapping fields catch them all" width="820">
 </p>
 
-## Benchmark
+## What's new in 0.3.0
+
+- **Evidence-grounded decisions.** Supply local sources with `--evidence`; Aiki records their paths and
+  hashes, checks freshness and citations, independently verifies selected load-bearing claims, and shows the
+  remaining coverage gaps before the chair.
+- **Three explicit modes.** Use `quick` for one structured analyst, `council` for bounded multi-model
+  deliberation, or `research` for current-fact work with Codex search. Aiki never hides this choice behind a
+  learned router.
+- **A decision dossier, not an essay.** Reports lead with the recommendation, verified evidence coverage,
+  decisive facts, first action, strongest counter-case, and critical unknowns. Financial and threshold-heavy
+  decisions can include graph-anchored numbers, payback, option commitments, and a go/no-go tripwire.
+- **More resilient long runs.** Bounded rebuttal, schema-safe output recovery, mode-aware deadlines, startup
+  provider checks, and call replay make malformed model output, quotas, and timeouts less likely to waste a
+  whole run.
+
+The published code-review benchmark result below is unchanged. The new idea-decision benchmark machinery is
+included in 0.3.0, but its paid evaluation is still pending, so no idea-quality improvement claim is made yet.
+
+## How Aiki works (no APIs)
+
+Aiki is orchestration, not a hosted AI service:
+
+1. You install and sign in to three provider CLIs: Claude Code (`claude`), Codex (`codex`), and
+   Antigravity/Gemini (`agy`).
+2. Aiki starts those CLIs as local child processes with read-only/sandbox flags.
+3. Each model gets the same task independently, returns structured JSON, and never sees another model's
+   answer until the cross-exam stage.
+4. Aiki validates every stage with zod, stores the audit trail under `.aiki/` or `~/.aiki/`, and renders the
+   final decision brief.
+5. Your existing provider logins/subscriptions handle model access. Aiki never asks for API keys and never
+   reads credential folders.
+
+## Benchmarks
+
+### Code-review benchmark — completed
 
 On a **pre-registered, 10-case held-out** code-review benchmark (frozen before the run so it couldn't be
 tuned post-hoc — see [BENCHMARK.md](BENCHMARK.md) and [RESULTS.md](RESULTS.md)):
@@ -85,20 +121,39 @@ set. It is **not** a claim of beating cheap self-consistency (that comparison is
 n = 10 cases, single run per arm — directional, not a p-value. Full method and every number in
 [RESULTS.md §7](RESULTS.md).
 
+### Idea-decision benchmark — evaluation pending
+
+Version 0.3.0 includes the frozen B/C/D2/R protocol, eight build cases, a 12-case/tag/provenance holdout
+contract that stays closed until the protocol freeze, checkpoint/resume, independently shuffled blind-rating
+packets, a frozen scorer, and post-freeze hash guards. Paid build tuning, blind ratings, and the holdout run
+are not complete, so there is **no published idea-decision performance claim yet**. See
+[BENCHMARK-IDEA-V3.md](BENCHMARK-IDEA-V3.md) for the pre-registration and
+[docs/IDEA_V3_BENCH.md](docs/IDEA_V3_BENCH.md) for the operator workflow.
+
 ## Requirements
 
-> ⚠️ **aiki drives your existing CLIs — it does not ship or host any model.** You must have the provider CLIs
-> **installed and already logged in.** aiki never sees, stores, or transmits your credentials.
+> ⚠️ **aiki drives your existing CLIs — it does not ship or host any model.** You should install and log in to
+> all three provider CLIs before judging results. aiki never sees, stores, or transmits your credentials.
 
 - **Node ≥ 20.** (Node 16/18 will crash at startup — this is a hard requirement.)
 - **macOS or Linux** (WSL2 works).
 - The provider CLIs on your `PATH`, **each already authenticated**:
-  | CLI | Command | Shown in aiki as |
-  |---|---|---|
-  | Claude Code | `claude` | Claude |
-  | Codex | `codex` | Codex |
-  | Antigravity | `agy` | Gemini |
-- **At least 2 of the 3** must be ready (a council needs a panel). Check anytime with `aiki doctor`.
+
+  | Provider CLI | Command Aiki runs | Shown in Aiki as | Check |
+  |---|---|---|---|
+  | Claude Code | `claude` | Claude | `claude --version` |
+  | Codex | `codex` | Codex | `codex --version` |
+  | Antigravity | `agy` | Gemini | `agy --version` |
+
+- **Recommended: 3/3 providers ready.** Aiki can start with 2/3, but full council behavior and benchmarked
+  code-review quality expect Claude Code, Codex, and Antigravity/Gemini together.
+
+You don't have to remember to check: launching `aiki` runs this preflight automatically — per-provider
+progress rows for CLI presence, version, and auth/quota — and only opens the menu when at least 2 providers
+pass. A failing provider shows its exact fix (install command, `run <binary> once to log in`, or retry
+later). The auth/quota smoke result is cached for 6 hours, so repeat launches are instant and free.
+
+Check manually anytime with `aiki doctor`:
 
 ```bash
 aiki doctor          # lists each provider: version, ready/not, read-only mode
@@ -111,6 +166,9 @@ keeping the read-only sandbox enabled, so non-git folders do not crash the provi
 aiki doctor --fresh
 ```
 
+`--fresh` bypasses the six-hour cache and can make up to three tiny provider calls; use it only when you need
+to recheck authentication or quota immediately.
+
 ## Install
 
 ```bash
@@ -119,6 +177,13 @@ npm install -g aiki-cli
 
 The package is `aiki-cli`; it installs the `aiki` command.
 
+Upgrade an existing installation:
+
+```bash
+npm install -g aiki-cli@latest
+aiki --version                 # 0.3.0
+```
+
 From source:
 
 ```bash
@@ -169,7 +234,9 @@ Plain text is never charged silently — you get a confirm step before any run s
 
 ```bash
 aiki run idea-refinement "a fridge-photo-to-recipe app for busy parents"
+aiki run idea-refinement "an early idea" --mode quick             # one structured analyst; no council claim
 aiki run idea-refinement ./idea.md
+aiki run idea-refinement ./idea.md --mode research --evidence ./research/  # grounded, source-verifying council
 aiki run code-review --base main             # review this branch vs main
 aiki run code-review --diff ./changes.patch  # review a patch file
 aiki run code-review --cheap                 # Gemini+Codex review, Claude judges only disputes (~⅓ the Opus)
@@ -178,6 +245,10 @@ aiki show <run-id> --html --open             # open the shareable decision brief
 
 An idea run **auto-opens** its report in your browser when it finishes.
 
+`--evidence` accepts one local file or directory already in your scope. Aiki records absolute paths and
+SHA-256 hashes in the run, gives those paths to the read-only scouts, and never copies the source files or
+reads provider credential directories.
+
 ## The two workflows
 
 <p align="center">
@@ -188,17 +259,23 @@ An idea run **auto-opens** its report in your browser when it finishes.
 real line in the diff) → mutual adversarial cross-examination → consensus/dispute map → the judge adjudicates
 only the disputes → report.
 
-**Idea refinement** — a contextual preflight (a few sharp questions to pin down what you actually mean) →
-intent contract → misunderstanding guard → parallel adversarial analysis → disagreement map → verifier →
-judge → a validation planner. The report is a **decision brief**, not an essay:
-
-- a **BLUF recommendation** — `PROCEED` / `PROCEED WITH CONDITIONS` / `PIVOT` / `STOP`
-- the **chairman's reasoning** (what decided it, where the models split, whose side the judge took)
-- a **dimension scorecard** (which of 12 angles were examined, contested, or missed)
-- an **assumption audit** (held / failed / unverified, with confidence)
-- **the debate** (who argued what, who won)
-- an **anchored validation plan** — concrete next actions, each with an effort estimate and a *kill signal*
-- a **cost receipt** (calls per provider)
+**Idea refinement** — two independent preflight readings → one confirmed/defaulted decision contract →
+complementary analyst lanes → deterministic claim/evidence graph audit → only decision-critical verification
+or rebuttal → evidence-linked chair → validation planner. The report is a graph-backed **decision dossier**,
+not an essay.
+Choose `--mode quick` for one structured analyst, `--mode council` (default) for the full decision council,
+or `--mode research` for source-grounded current-fact work. Aiki never chooses a mode with a learned router:
+
+- a **reader-first decision card** — recommendation, verified evidence coverage, decisive facts, first action,
+  strongest counter-case, and critical unknowns before audit detail
+- a **numeric decision snapshot when relevant** — graph-anchored decisive numbers, explicit payback, option
+  commitments labeled `KNOWN` / `TARGET_CAP` / `UNKNOWN`, and a go/no-go tripwire
+- a **graph-anchored recommendation and claim chain** — every decisive statement links to stored claim IDs
+- an **evidence and coverage ledger** — source, date, freshness, verification, `NOT_APPLICABLE`, and missing evidence
+- **genuine disagreements and position changes** — explicit `CONCEDE` / `COUNTER` / `NARROW` events
+- **decision sensitivity and an executable experiment plan** — anchored tests with effort and kill signals
+- a **verified contribution ledger** — unique provider claims count only after independent verification
+- an **orchestration receipt and technical fold** — calls, degradation, submissions, edges, and graph events
 
 ## Example: a real idea run
 
@@ -218,7 +295,8 @@ The council's verdict on that one:
 
 …followed by **7 anchored validation actions**, e.g. *"Pull the last 90 days of churned users and tag each
 with their primary churn reason"* (effort S, kill signal: churn isn't feature-driven). The full brief opens in
-your browser and has a **Copy report (Markdown)** button so you can paste it straight into your coding assistant.
+your browser and has a **Copy report (Markdown)** button backed by the same persisted dossier as the HTML,
+so the copied claims, evidence statuses, experiments, and receipt cannot drift from the page.
 
 ## Configuration
 
@@ -271,9 +349,10 @@ This is the part that makes aiki trustworthy to point at a real repo:
 
 ## Costs & limits
 
-- **Runs cost real model calls** against your existing CLI subscriptions/quota. Idea refinement is about
-  **12 provider calls** (~4 on Claude/Opus); code review is about **5**. `aiki run` shows an estimate and asks
-  to confirm (skip with `--yes`).
+- **Runs cost real model calls** against your existing CLI subscriptions/quota. Idea refinement is nominally
+  **3 calls in quick**, **6–8 in council**, or **8–10 in research** (schema repairs can add calls within the
+  mode-aware budget); code review is about **5**. `aiki run` shows the mode, range, budget, and reserved
+  chair/planner calls before asking to confirm (skip with `--yes`).
 - **Not a general assistant.** Questions and "explore my whole codebase" requests are redirected, not answered
   — aiki reviews a *diff* and vets a *stated idea*.
 - **Analysis, not advice.** Every report is a decision aid. Verify before acting.
@@ -287,7 +366,7 @@ This is the part that makes aiki trustworthy to point at a real repo:
 - Anywhere else → `~/.aiki/`. `$AIKI_HOME` overrides the global home.
 - The global session registry always lives in `~/.aiki/`.
 
-## How it works
+## Implementation notes
 
 Every stage is a small, independently-testable unit with a zod-validated output contract. Model text lives in
 bounded, capped fields slotted into deterministic report structure — so the output is a briefing, not a chat

package/dist/bench/arms.js

@@ -12,7 +12,7 @@ import { sameFinding } from '../orchestration/stages/cr-map.js';
 import { s9ReviewJudge } from '../orchestration/stages/cr-s9-judge.js';
 import { scoreFindings } from '../orchestration/stages/cr-report.js';
 import { runCodeReview } from '../workflows/code-review.js';
-export const ARM_IDS = ['A', 'B', 'C', 'D', 'E'];
+export const ARM_IDS = ['A', 'B', 'C', 'D', 'E', 'L'];
 const SCHEMA_LINE = 'findings: [{id "F1"..., file, line_start, line_end, severity P0|P1|P2|P3, category CORRECTNESS|SECURITY|CONCURRENCY|ERROR_HANDLING|PERF|MAINTAINABILITY, claim, evidence, suggested_fix, self_confidence 0-1}]';
 const A_PROMPT = `Review this code diff and report the bugs you find. The diff is at {{DIFF_PATH}} (repo root = your cwd).
 Output ONLY JSON: {task_echo (≤2 sentences), ${SCHEMA_LINE}}. JSON only.`;
@@ -88,17 +88,18 @@ export const armC = async (ctx, diff) => {
     const judge = await s9ReviewJudge(ctx, map); // ctx.roles.judge = claude for arm C (harness sets it)
     return scoreFindings(map, judge).filter((s) => s.disposition === 'kept').map((s) => s.finding);
 };
-/** Arms D and E — the full product pipeline (`runCodeReview`) → read back its kept findings for scoring.
+/** Arms D, E, and L — the product pipeline (`runCodeReview`) → read back kept findings for scoring.
  *  SAME pipeline; the harness injects their differing roles: D = claude+codex reviewers + agy judge;
- *  E = agy+codex reviewers + claude judge (the Opus-thrift variant — claude fires only on disputes). */
-const productPipeline = async (ctx, diff) => {
-    await runCodeReview(ctx, diff);
+ *  E = agy+codex reviewers + claude judge; L = E plus targeted Claude coverage-hole hunts. */
+async function runProductPipeline(ctx, diff, ladder = false) {
+    await runCodeReview(ctx, diff, { ladder });
     const [map, judge] = await Promise.all([
         readFile(resolve(ctx.writer.dir, '07-review-map.json'), 'utf8').then(JSON.parse),
         readFile(resolve(ctx.writer.dir, '09-judge-report.json'), 'utf8').then(JSON.parse),
     ]);
     return scoreFindings(map, judge).filter((s) => s.disposition === 'kept').map((s) => s.finding);
-};
-export const armD = productPipeline;
-export const armE = productPipeline;
-export const ARMS = { A: armA, B: armB, C: armC, D: armD, E: armE };
+}
+export const armD = (ctx, diff) => runProductPipeline(ctx, diff);
+export const armE = (ctx, diff) => runProductPipeline(ctx, diff);
+export const armL = (ctx, diff) => runProductPipeline(ctx, diff, true);
+export const ARMS = { A: armA, B: armB, C: armC, D: armD, E: armE, L: armL };

package/dist/bench/harness.js

@@ -29,12 +29,12 @@ export async function loadCases(suite, set, root = process.cwd()) {
     }
     return cases;
 }
-/** A/B/C need claude (the fixed single model); D needs claude+codex reviewers; E needs agy+codex
- *  reviewers + claude judge (the Opus-thrift variant). */
+/** A/B/C need claude (the fixed single model); D needs claude+codex reviewers; E/L need agy+codex
+ *  reviewers + claude judge/hole-hunter. */
 function armAvailable(arm, available) {
     if (arm === 'D')
         return available.includes('claude') && available.includes('codex');
-    if (arm === 'E')
+    if (arm === 'E' || arm === 'L')
         return available.includes('agy') && available.includes('codex') && available.includes('claude');
     return available.includes('claude');
 }
@@ -56,10 +56,10 @@ async function readPrecision(runId, reported, root = process.cwd()) {
 }
 async function runArmOnCase(arm, c, handles, available, root) {
     const runId = makeRunId('code-review');
-    // Arm C's synthesis judge stays same-model; Arm E swaps the product pipeline's roles to the
-    // Opus-thrift config (agy+codex hunt, claude judge). D uses code-review defaults.
+    // Arm C's synthesis judge stays same-model; E/L use the Opus-thrift roles
+    // (agy+codex tier-1 hunt, claude judge/hole-hunter). D uses code-review defaults.
     const overrides = arm === 'C' ? { judge: 'claude' }
-        : arm === 'E' ? { s4: ['agy', 'codex'], judge: 'claude' }
+        : arm === 'E' || arm === 'L' ? { s4: ['agy', 'codex'], judge: 'claude' }
             : undefined;
     const roles = resolveRoles('code-review', available, overrides);
     const ctx = new RunCtx({ runId, workflow: 'code-review', handles, roles, writer: new RunWriter(runId, join(root, '.aiki')), cwd: c.dir });
@@ -91,6 +91,8 @@ async function runArmOnCase(arm, c, handles, available, root) {
         seeded: s.seeded,
         matched: s.matched,
         recall: s.recall,
+        matchedRelaxed: s.matchedRelaxed,
+        recallRelaxed: s.recallRelaxed,
         reported: s.reported,
         unmatched: s.unmatched,
         precision: await readPrecision(runId, s.reported, root),
@@ -100,7 +102,7 @@ async function runArmOnCase(arm, c, handles, available, root) {
 }
 /** Approx claude/Opus calls each arm makes per case — for the pre-run quota estimate (§19). Not exact:
  *  §14 JSON repairs can add a few, and D's cross-exam/judge vary; deliberately a round upper-ish figure. */
-export const CLAUDE_CALLS_PER_CASE = { A: 1, B: 1, C: 4, D: 2, E: 1 };
+export const CLAUDE_CALLS_PER_CASE = { A: 1, B: 1, C: 4, D: 2, E: 1, L: 5 };
 /** ≈ claude/Opus calls for a list of case×arm pairs (the quota-sensitive cost the user cares about). */
 export function estimateClaudeCalls(pairs) {
     return pairs.reduce((n, p) => n + (CLAUDE_CALLS_PER_CASE[p.arm] ?? 0), 0);
@@ -238,12 +240,13 @@ export function renderTable(result) {
     const L = [];
     L.push(`bench ${result.suite} — set ${result.set} — ${result.cases.length} case(s)`);
     L.push('');
-    L.push('| Arm | Recall (micro) | Recall (macro) | Matched/Seeded | Reported | Unmatched(FP?) | Precision | Calls | Wall(s) |');
-    L.push('|---|---|---|---|---|---|---|---|---|');
+    L.push('| Arm | Recall (strict) | Recall (category-relaxed) | Recall (macro) | Matched/Seeded | Reported | Unmatched(FP?) | Precision | Calls | Wall(s) |');
+    L.push('|---|---|---|---|---|---|---|---|---|---|');
     for (const r of result.summary) {
         const pct = (n) => `${(n * 100).toFixed(0)}%`;
         const prec = r.precision === null ? '—' : pct(r.precision);
-        L.push(`| ${r.arm} | ${pct(r.recall)} | ${pct(r.recallMacro)} | ${r.matched}/${r.seeded} | ${r.reported} | ${r.unmatched} | ${prec} | ${r.calls} | ${(r.wallMs / 1000).toFixed(1)} |`);
+        const relaxed = r.recallRelaxed === undefined ? '—' : pct(r.recallRelaxed);
+        L.push(`| ${r.arm} | ${pct(r.recall)} | ${relaxed} | ${pct(r.recallMacro)} | ${r.matched}/${r.seeded} | ${r.reported} | ${r.unmatched} | ${prec} | ${r.calls} | ${(r.wallMs / 1000).toFixed(1)} |`);
     }
     L.push('');
     L.push('Precision "—" = not yet adjudicated (label false positives with `aiki resolve <run>`).');