aiki-cli 0.3.2 → 0.3.5

package/CHANGELOG.md

@@ -1,5 +1,73 @@
 # Changelog
 
+## Unreleased — v7 adaptive deliberation
+
+### Added
+- **`aiki run idea-refinement --mode auto`** — a deterministic quick-vs-council routing decision made
+  before any provider call. Escalates to `council` when the prompt supplies URLs or an evidence pack,
+  hits a regulated/financial/security topic, requests deliverables beyond a decision, uses research
+  wording, or runs long (>120 words); otherwise `quick`. Prints the resolved mode + reason and persists
+  it to `meta.auto_decision`. No learned routing (§22): the rules are fixed and inspectable. `meta.mode`
+  stays `quick|council`; legacy metadata still parses and explicit modes keep their existing artifacts.
+- **One-call auto fast path (v7 Phase C).** A short DECISION-only request with a clear decision verb
+  and subject skips the two model preflight readings and builds the same typed contract deterministically.
+  Ambiguous prompts keep the existing quick path; explicit quick/council runs are unchanged. Reports label
+  the result: `Single-pass analysis; council escalation was not required.`
+- **Structural auto escalation (v7 Phase D).** Validated analyst output now triggers a deterministic
+  failover for unsupported decisive claims, model-knowledge STOP/PIVOT claims, failed calculations,
+  contradictory supplied evidence, or load-bearing disagreement. Auto runs use 1–4 calls: two existing
+  preflight readings, one primary analysis, and at most one slim claim-only challenger call. The receipt
+  records every trigger; explicit quick/council paths remain unchanged.
+- **Per-call token accounting (v7 Phase A).** Every call records normalized input/output token usage —
+  provider-reported for Claude, a labeled `chars/4` estimate otherwise — summed into `meta.usage_totals`
+  and shown as a `Tokens:` line in the dossier and terminal summary. Purely additive; behavior unchanged.
+
+## Unreleased — v6 council integrity
+
+Root-cause pass driven by exact replay of run `20260717-1219` (`plan/AIKI-v6-council-integrity-plan.md`);
+every fix is pinned by tests on the real payloads that failed live (`test/fixtures/f740-*`).
+
+### Fixed
+- **A parseable plan is never discarded again.** The planner's complete feature backlog and
+  implementation plan were replaced by "deliverables unavailable" over a 13-character headline
+  overflow with no repair budget. `jsonCall` now applies deterministic salvage before failing when
+  repair is unavailable; lossy coercion clips over-max strings at a word boundary; empty optional
+  strings are dropped losslessly (they carry no information). Partial planner answers are kept and
+  flagged `plan_partial` instead of vanishing.
+- **The chair's judgment is no longer overruled by code.** The `PARTIAL/UNVERIFIABLE ⇒ must be
+  UNRESOLVED` rule (which flattened 5 nuanced rulings and forced a 125s Opus repair) is deleted;
+  ruling HOLDS against positively CONTRADICTED evidence remains fatal.
+- **S4 schema-ceremony repairs are free now.** Empty rationale strings validate via lossless
+  coercion (was a 267s paid repair); genuinely lossy defects salvage without a call when the budget
+  is tight (repairs stay gated behind a 3-call reserved tail: chair + planner + one tail repair).
+- Chair prose is no longer regex-mangled ("confirmed-unverified" stays the chair's own words);
+  enum rewrites apply to UPPERCASE leaks only. A `NOT_COMPUTABLE` payback block is omitted instead
+  of announced. Rendered Markdown/HTML never contain local filesystem usernames.
+
+### Added
+- **Local council workspace:** `aiki serve` opens a loopback-only three-pane browser workspace with cached
+  provider status, decision history, server-enforced file/URL/spend gates, live SSE stage and call telemetry,
+  cancellation, path-free verdict cards, and responsive desktop/mobile layouts.
+- Completed decisions accept one-call follow-up questions labeled `no council`; failed or cancelled runs with
+  cached calls can be resumed after an explicit spend review. Settings atomically persist provider model pins
+  and role assignments without exposing credentials, local paths, or arbitrary JSON to the browser.
+- A dependency-free hosted replay, real workspace screenshot, and timed three-minute demo script provide a
+  safe submission path when judges do not have the three provider CLIs installed.
+- **Semantic claim join at zero extra calls:** the S8 verifier may emit `claim_groups` over an
+  anonymous all-claims index; validated groups (unknown ids and single-provider groups dropped)
+  overlay CONSENSUS/DISAGREEMENT states through the existing state machine. Cross-provider
+  paraphrases — the reason every report said "0 consensus · 0 disputes" — now join.
+- **Evidence origin honesty:** every card is classified EXTERNAL / USER_MATERIAL / MODEL_KNOWLEDGE;
+  confidence's evidence-quality term counts external only; the user's own material renders as
+  "consistent with your materials (not independently checked)", never VERIFIED; the Sources section
+  states how many independent external sources the run actually had.
+- **Blocked-source permission prompt:** when a supplied URL cannot be read, an interactive run now
+  asks right in the terminal before any paid call — `Proceed without it? [y/N]` — the same
+  allow/deny pattern provider CLIs use for internet or file access. Deny prints how to proceed
+  (paste the page text into the prompt, or `--allow-blocked-sources`); headless runs fail fast
+  with the same instructions. (The previous behavior burned a full council budget around a 403'd
+  page.)
+
 ## 0.3.2 — 2026-07-17
 
 ### Added

package/README.md

@@ -12,7 +12,7 @@
   <img alt="Node ≥ 20" src="https://img.shields.io/badge/node-%E2%89%A5%2020-brightgreen.svg">
   <img alt="Local-first, no API keys" src="https://img.shields.io/badge/local--first-no%20API%20keys-informational.svg">
   <img alt="Read-only orchestration" src="https://img.shields.io/badge/orchestration-read--only-success.svg">
-  <img alt="Tests" src="https://img.shields.io/badge/tests-381%20passing-success.svg">
+  <img alt="Tests" src="https://img.shields.io/badge/tests-579%20passing-success.svg">
 </p>
 
 ---
@@ -30,7 +30,21 @@ It does two jobs, well:
 aiki is **not** a general assistant. Trivia and chat are routed away, not answered — a council adds cost, not
 accuracy, when there's one right answer.
 
-**Jump to:** [Why](#why) · [What's new](#whats-new-in-030) · [How it works](#how-aiki-works-no-apis) · [Benchmarks](#benchmarks) · [Requirements](#requirements) · [Install](#install) · [Quickstart](#quickstart) · [The two workflows](#the-two-workflows) · [Example](#example-a-real-idea-run) · [Configuration](#configuration) · [Sessions & resume](#sessions--resume) · [Safety](#safety-model) · [Costs & limits](#costs--limits)
+**Jump to:** [Workspace](#local-council-workspace) · [Why](#why) · [What's new](#whats-new-in-030) · [How it works](#how-aiki-works-no-apis) · [Benchmarks](#benchmarks) · [Requirements](#requirements) · [Install](#install) · [Quickstart](#quickstart) · [The two workflows](#the-two-workflows) · [Example](#example-a-real-idea-run) · [Configuration](#configuration) · [Sessions & resume](#sessions--resume) · [Safety](#safety-model) · [Costs & limits](#costs--limits)
+
+---
+
+## Local council workspace
+
+Open the live local workspace with:
+
+```bash
+npm i -g aiki-cli && aiki serve
+```
+
+`aiki serve` opens a loopback-only browser workspace with decision history, explicit file/URL/spend approval,
+live council stages, a verdict card, one-call follow-ups, and per-role model settings. It makes no provider
+calls until you approve and convene a run.
 
 ---
 
@@ -52,6 +66,11 @@ stop copy-pasting between them by hand.
   remaining coverage gaps before the chair.
 - **Two bounded behaviors.** Use `quick` for one structured analyst or `council` for the full multi-model
   decision council with source investigation. `research` remains accepted as an alias for `council`.
+  `--mode auto` deterministically selects the entry policy from the prompt (URLs, an evidence pack, a
+  regulated/financial/security topic, requested deliverables, research wording, or a long input require
+  preflight) and prints the one-line reason — never a learned choice. A clear, short decision question starts
+  with one call; structural gaps in the typed output add the two readings and, only when useful, one targeted
+  challenger call (1–4 total). Use explicit `council` when you want the full multi-model council every time.
 - **A decision dossier, not an essay.** Reports lead with the recommendation, verified evidence coverage,
   decisive facts, first action, strongest counter-case, and critical unknowns. Financial and threshold-heavy
   decisions can include graph-anchored numbers, payback, option commitments, and a go/no-go tripwire.
@@ -180,7 +199,7 @@ Upgrade an existing installation:
 
 ```bash
 npm install -g aiki-cli@latest
-aiki --version                 # 0.3.0
+aiki --version                 # 0.3.4
 ```
 
 From source:
@@ -197,9 +216,46 @@ npm link             # puts `aiki` on your PATH   (or run directly: node dist/cl
 
 ```bash
 cd /path/to/your/project
-aiki                 # opens the interactive home screen
+aiki serve           # opens the local council workspace in your browser
+aiki                 # or open the interactive terminal home screen
 ```
 
+## The serve workspace
+
+`aiki serve` is the recommended way to use Aiki. It starts a **loopback-only** web workspace (bound to
+`127.0.0.1`, never exposed to your network) and opens it in your browser. **No provider is called until you
+type a request and approve the run** — opening the workspace is free.
+
+```bash
+aiki serve                 # binds the first free port in 4173–4183 and opens the browser
+aiki serve --port 4180     # pin a specific port
+aiki serve --no-open       # start the server but don't auto-open the browser
+```
+
+**What you'll see and do:**
+
+1. **Sessions rail (left).** Every past run, newest first. Click one to reopen its full decision brief;
+   runs that were killed or timed out are flagged **Resume**.
+2. **Composer (center).** Describe an idea to stress-test, or ask for a code review, and press **Enter**.
+   Plain text never spends anything on its own — you always get a convene step first.
+3. **Approval gates (inline).** Before any paid work, Aiki asks — per action — to read a **file**, fetch a
+   **URL**, or **spend** provider calls. Each gate offers **allow once**, **allow for this session**, or
+   **deny**. Nothing runs until you approve.
+4. **Live council.** A pipeline timeline shows each stage; the provider seat cards (Claude, Codex, Gemini)
+   light up as they work, with a running **call counter and budget bar** so you always see the spend.
+5. **Verdict card.** When the council finishes: a plain-language banner (go / proceed-with-conditions /
+   stop / inconclusive), a **confidence meter**, the anchored validation plan, expandable reasoning and
+   evidence, and a **Copy summary** button.
+6. **Cost receipt.** Per-provider call counts plus the token/cost receipt for the run, against your budget.
+7. **Follow-ups.** After a run completes you can ask a follow-up question — answered in **exactly one
+   read-only call, no new council** — or **Re-convene** to run a fresh full council on a revised question.
+8. **Settings.** Choose the model each provider uses and override per-role assignments (judge, verifier,
+   analyst, the two review seats, the follow-up responder). It shows the effective value vs. your local
+   override and never exposes credentials, keys, or paths.
+
+Everything the workspace shows is backed by the same schema-validated, on-disk audit trail as the headless
+CLI, so nothing on screen can drift from what was actually stored.
+
 ## Use Aiki For
 
 - Code review: "review this branch", "check this patch", "find bugs in this diff".
@@ -234,6 +290,7 @@ Plain text is never charged silently — you get a confirm step before any run s
 ```bash
 aiki run idea-refinement "a fridge-photo-to-recipe app for busy parents"
 aiki run idea-refinement "an early idea" --mode quick             # one structured analyst; no council claim
+aiki run idea-refinement "Should I use Postgres or MySQL?" --mode auto  # starts at 1 call; structural gaps may escalate to 4
 aiki run idea-refinement ./idea.md
 aiki run idea-refinement ./idea.md --mode council --evidence ./research/   # grounded, source-verifying council
 aiki run code-review --base main             # review this branch vs main
@@ -361,6 +418,9 @@ This is the part that makes aiki trustworthy to point at a real repo:
   **3 calls in quick** or **8–10 in the full council** (`research` is the same behavior; schema repairs can add
   calls within the mode-aware budget); code review is about **5**. `aiki run` shows the mode, range, budget, and reserved
   chair/planner calls before asking to confirm (skip with `--yes`).
+- **Every run reports token usage.** The decision report and the serve cost receipt show input/output tokens
+  and reported cost per run — provider-reported where the CLI returns it, a labeled estimate otherwise — so
+  you see the real spend, not just the call count.
 - **Not a general assistant.** Questions and "explore my whole codebase" requests are redirected, not answered
   — aiki reviews a *diff* and vets a *stated idea*.
 - **Analysis, not advice.** Every report is a decision aid. Verify before acting.

package/dist/bench/idea-v3-bench.js

@@ -3,15 +3,18 @@ import { access, mkdir, readFile, readdir, rename, writeFile } from 'node:fs/pro
 import { dirname, isAbsolute, join, relative, resolve, sep } from 'node:path';
 import { z } from 'zod';
 import { executeRun } from '../orchestration/engine.js';
+import { buildTaskProfile, resolveAutoMode } from '../orchestration/auto-profile.js';
 import { jsonCall } from '../orchestration/jsonStage.js';
 import { makeRunId, resolveRoles, RunCtx, setupProviders } from '../orchestration/context.js';
 import { EvidencePack } from '../orchestration/evidence-pack.js';
+import { requestedOutputsFor } from '../orchestration/preflight.js';
+import { extractPublicUrls } from '../orchestration/url-sources.js';
 import { IdeaV3CaseManifest } from './scoring/decision-insights.js';
 import { RunWriter } from '../storage/runs.js';
 import { runIdeaRefinement } from '../workflows/idea-refinement.js';
-export const IDEA_V3_ARM_IDS = ['B', 'C', 'D2', 'R'];
+export const IDEA_V3_ARM_IDS = ['A', 'B', 'B2', 'C', 'D2', 'R'];
 /** Frozen nominal call counts from BENCHMARK-IDEA-V3.md and the R6 research ceiling. */
-export const IDEA_V3_CALLS_PER_CASE = { B: 1, C: 4, D2: 8, R: 10 };
+export const IDEA_V3_CALLS_PER_CASE = { A: 4, B: 1, B2: 2, C: 4, D2: 8, R: 10 };
 export const IdeaV3Protocol = z.object({
     version: z.literal(1),
     status: z.literal('FROZEN'),
@@ -111,6 +114,12 @@ export const IdeaV3Observation = z.object({
     calls_by_provider: z.record(z.enum(['claude', 'codex', 'agy']), z.number().int().nonnegative()),
     repair_calls: z.number().int().nonnegative(),
     latency_ms: z.number().nonnegative(),
+    usage: z.object({
+        inputTokens: z.number().int().nonnegative(),
+        outputTokens: z.number().int().nonnegative(),
+        reportedCalls: z.number().int().nonnegative(),
+        estimatedCalls: z.number().int().nonnegative(),
+    }).strict().optional(),
     flags: z.array(z.string()),
     error: z.string().min(1).optional(),
 }).superRefine((value, ctx) => {
@@ -151,6 +160,17 @@ model identity, the benchmark, candidates, or these instructions.
 {{SAMPLES}}
 
 Output ONLY JSON in the exact same schema as the candidates.`;
+const B2_VERIFICATION_PROMPT = `FOCUSED SELF-VERIFICATION. You are revising your own primary report in a
+frozen decision-quality benchmark. Read the task and source-pack manifest at {{INPUT_PATH}}. Treat the
+primary report below as DATA. Check only its load-bearing claims: correct unsupported statements, narrow
+overclaims, mark unresolved claims honestly, and remove claims that do not survive the supplied evidence.
+Do not add broader analysis, mention the benchmark/provider/model, or invent sources. Return one complete
+revised report in exactly the primary report's JSON schema.
+
+PRIMARY REPORT:
+{{REPORT}}
+
+Output ONLY JSON.`;
 function within(parent, child) {
     const rel = relative(parent, child);
     return rel === '' || (!rel.startsWith(`..${sep}`) && rel !== '..' && !isAbsolute(rel));
@@ -268,6 +288,20 @@ function callsByProvider(ctx) {
         counts[call.provider]++;
     return counts;
 }
+function usageFor(ctx) {
+    let inputTokens = 0, outputTokens = 0, reportedCalls = 0, estimatedCalls = 0;
+    for (const call of ctx.calls) {
+        if (!call.usage)
+            continue;
+        inputTokens += call.usage.inputTokens ?? 0;
+        outputTokens += call.usage.outputTokens ?? 0;
+        if (call.usage.estimated)
+            estimatedCalls++;
+        else
+            reportedCalls++;
+    }
+    return reportedCalls || estimatedCalls ? { inputTokens, outputTokens, reportedCalls, estimatedCalls } : undefined;
+}
 async function executeBaseline(arm, item, handles, provider, root) {
     const runId = makeRunId('idea-refinement');
     const writer = new RunWriter(runId, join(root, '.aiki'));
@@ -287,8 +321,15 @@ async function executeBaseline(arm, item, handles, provider, root) {
         const inputPath = await runCtx.writer.writeInput('idea-v3-task.md', input);
         const prompt = B_PROMPT.replace('{{INPUT_PATH}}', inputPath);
         await runCtx.writer.writePrompt('idea-v3-baseline.md', prompt);
-        if (arm === 'B') {
-            report = await jsonCall(runCtx, runCtx.handle(provider), 'B', prompt, IdeaV3BaselineReport);
+        if (arm === 'B' || arm === 'B2') {
+            report = await jsonCall(runCtx, runCtx.handle(provider), arm === 'B' ? 'B' : 'B2-primary', prompt, IdeaV3BaselineReport);
+            if (arm === 'B2') {
+                const verification = B2_VERIFICATION_PROMPT
+                    .replace('{{INPUT_PATH}}', inputPath)
+                    .replace('{{REPORT}}', JSON.stringify(report, null, 2));
+                await runCtx.writer.writeRaw('B2-verification.prompt.txt', verification);
+                report = await jsonCall(runCtx, runCtx.handle(provider), 'B2-verify', verification, IdeaV3BaselineReport);
+            }
         }
         else {
             const samples = [];
@@ -302,6 +343,7 @@ async function executeBaseline(arm, item, handles, provider, root) {
         await runCtx.writer.writeRaw('idea-v3-baseline-report.json', JSON.stringify(report, null, 2));
         await runCtx.writer.writeText('final-report', renderBaselineReport(report));
     });
+    const usage = usageFor(ctx);
     const base = {
         case_id: item.id,
         arm,
@@ -310,12 +352,59 @@ async function executeBaseline(arm, item, handles, provider, root) {
         calls_by_provider: callsByProvider(ctx),
         repair_calls: ctx.calls.filter((call) => call.stage.endsWith('-repair')).length,
         latency_ms: Date.now() - started,
+        ...(usage ? { usage } : {}),
         flags: [...ctx.flags],
     };
     return IdeaV3Observation.parse(outcome.ok && report
         ? { ...base, status: 'ok', report_markdown: renderBaselineReport(report) }
         : { ...base, status: 'error', error: `${outcome.error?.code ?? 'CRASH'}: ${outcome.error?.message ?? 'baseline produced no report'}` });
 }
+async function executeAuto(item, handles, root) {
+    const runId = makeRunId('idea-refinement');
+    const writer = new RunWriter(runId, join(root, '.aiki'));
+    const input = benchmarkInput(item);
+    const evidencePack = await caseEvidencePack(item);
+    const resolved = resolveAutoMode(buildTaskProfile(input, {
+        urlCount: extractPublicUrls(input).length,
+        hasEvidencePack: !!evidencePack,
+        requestedOutputs: requestedOutputsFor(input),
+    }));
+    const ctx = new RunCtx({
+        runId,
+        workflow: 'idea-refinement',
+        mode: resolved.mode,
+        handles,
+        roles: resolveRoles('idea-refinement', handles.map((handle) => handle.id)),
+        writer,
+        cwd: writer.dir,
+        budget: 7,
+        deadlineMs: 45 * 60 * 1000,
+        evidencePack,
+        autoDecision: {
+            resolved: resolved.mode,
+            reasons: resolved.reasons,
+            ...(resolved.fastPath ? { fast_path: true } : {}),
+        },
+    });
+    const started = Date.now();
+    const outcome = await executeRun(ctx, input, runIdeaRefinement);
+    const usage = usageFor(ctx);
+    const base = {
+        case_id: item.id,
+        arm: 'A',
+        run_id: runId,
+        calls: ctx.calls.length,
+        calls_by_provider: callsByProvider(ctx),
+        repair_calls: ctx.calls.filter((call) => call.stage.endsWith('-repair')).length,
+        latency_ms: Date.now() - started,
+        ...(usage ? { usage } : {}),
+        flags: [...ctx.flags],
+    };
+    if (!outcome.ok) {
+        return IdeaV3Observation.parse({ ...base, status: 'error', error: `${outcome.error?.code}: ${outcome.error?.message}` });
+    }
+    return IdeaV3Observation.parse({ ...base, status: 'ok', report_markdown: await readFile(join(outcome.dir, 'final-report.md'), 'utf8') });
+}
 async function executeResearch(item, handles, root, frozenRoles) {
     const runId = makeRunId('idea-refinement');
     const writer = new RunWriter(runId, join(root, '.aiki'));
@@ -332,6 +421,7 @@ async function executeResearch(item, handles, root, frozenRoles) {
     });
     const started = Date.now();
     const outcome = await executeRun(ctx, benchmarkInput(item), runIdeaRefinement);
+    const usage = usageFor(ctx);
     const base = {
         case_id: item.id,
         arm: 'R',
@@ -340,6 +430,7 @@ async function executeResearch(item, handles, root, frozenRoles) {
         calls_by_provider: callsByProvider(ctx),
         repair_calls: ctx.calls.filter((call) => call.stage.endsWith('-repair')).length,
         latency_ms: Date.now() - started,
+        ...(usage ? { usage } : {}),
         flags: [...ctx.flags],
     };
     if (!outcome.ok) {
@@ -353,6 +444,9 @@ function parseArms(arms, set) {
         throw new Error('at least one idea-v3 arm is required');
     if (set === 'holdout' && unique.includes('D2'))
         throw new Error('D2 is build-set diagnostic only and has no holdout weight');
+    if (set === 'holdout' && unique.some((arm) => arm === 'A' || arm === 'B2')) {
+        throw new Error('Phase F arms are build-set validation only and cannot change the frozen holdout protocol');
+    }
     return unique;
 }
 const CAMPAIGN_NAME = /^idea-v3-(build|holdout)-(claude|codex|agy)-\d{4}-\d{2}-\d{2}\.json$/;
@@ -447,9 +541,12 @@ export async function runIdeaV3Bench(opts = {}) {
     const prior = resolved.prior?.observations ?? [];
     const done = new Set(prior.map((item) => `${item.case_id}:${item.arm}`));
     const handles = opts.handles ?? await setupProviders(protocol?.models);
-    if (!handles.some((handle) => handle.id === baselineProvider) && arms.some((arm) => arm === 'B' || arm === 'C')) {
+    if (!handles.some((handle) => handle.id === baselineProvider) && arms.some((arm) => arm === 'B' || arm === 'B2' || arm === 'C')) {
         throw new Error(`baseline provider ${baselineProvider} is unavailable`);
     }
+    if (arms.includes('A') && new Set(handles.map((handle) => handle.id)).size < 2) {
+        throw new Error('A requires at least two providers for protocol-comparable auto escalation');
+    }
     if (arms.includes('R') && new Set(handles.map((handle) => handle.id)).size < 3) {
         throw new Error('R requires all three frozen providers for a protocol-comparable run');
     }
@@ -466,6 +563,8 @@ export async function runIdeaV3Bench(opts = {}) {
     const execute = opts.execute ?? (async (target) => {
         if (target.arm === 'D2')
             return d2ByCase.get(target.case.id);
+        if (target.arm === 'A')
+            return executeAuto(target.case, handles, root);
         if (target.arm === 'R')
             return executeResearch(target.case, handles, root, protocol?.roles);
         return executeBaseline(target.arm, target.case, handles, baselineProvider, root);

package/dist/bench/idea-v3-rating.js

@@ -132,7 +132,7 @@ export function blindIdeaV3Report(report, runId, caseDir) {
         .replace(/^(- Report ID:).*$/gm, '$1 [redacted]')
         .replace(/^(- Generated:).*$/gm, '$1 [redacted]')
         .replace(/^(- Models and roles:).*$/gm, '$1 [redacted]')
-        .replace(/^(- (?:Mode|Provider calls|Categories|By provider|Recorded model time|Degradation flags):).*$/gm, '$1 [redacted]')
+        .replace(/^(- (?:Mode|Provider calls|Categories|By provider|Recorded model time|Tokens|Degradation flags):).*$/gm, '$1 [redacted]')
         .replace(/^(> ⚠ DEGRADED): [a-z0-9_]+(?:, [a-z0-9_]+)*\.?$/gm, '$1 [redacted]');
 }
 async function renderCasePacket(item) {
@@ -384,6 +384,82 @@ function median(values) {
     const middle = Math.floor(sorted.length / 2);
     return sorted.length % 2 ? sorted[middle] : (sorted[middle - 1] + sorted[middle]) / 2;
 }
+function nearestRank(values, percentile) {
+    if (!values.length)
+        return 0;
+    const sorted = [...values].sort((left, right) => left - right);
+    return sorted[Math.ceil(percentile * sorted.length) - 1];
+}
+/** Phase F additive metric: adjudicated matched expert claims per 1,000 recorded input+output tokens. */
+export function ideaV3TokenEfficiency(scored, campaign) {
+    scored = IdeaV3ScoredCampaign.parse(scored);
+    campaign = IdeaV3Campaign.parse(campaign);
+    return campaign.arms.map((arm) => {
+        const observations = campaign.observations.filter((item) => item.arm === arm);
+        const complete = observations.length > 0 && observations.every((item) => item.usage !== undefined);
+        const tokens = complete
+            ? observations.reduce((sum, item) => sum + item.usage.inputTokens + item.usage.outputTokens, 0)
+            : null;
+        const matched = scored.summary.find((item) => item.arm === arm)?.score.matched ?? 0;
+        return {
+            arm,
+            matched,
+            tokens,
+            matched_per_1k_tokens: tokens ? matched * 1_000 / tokens : null,
+            estimated_calls: observations.reduce((sum, item) => sum + (item.usage?.estimatedCalls ?? 0), 0),
+        };
+    });
+}
+/** Phase F build-only product targets. Original frozen holdout ship gates remain separate. */
+export function evaluateIdeaV3AdaptiveGates(scored, campaign) {
+    scored = IdeaV3ScoredCampaign.parse(scored);
+    campaign = IdeaV3Campaign.parse(campaign);
+    if (scored.set !== 'build' || campaign.set !== 'build') {
+        throw new Error('adaptive product targets require the Phase F build campaign');
+    }
+    const required = ['A', 'B', 'B2', 'D2', 'R'];
+    const caseIds = new Set(campaign.observations.filter((item) => item.arm === 'A').map((item) => item.case_id));
+    const expectedPairs = [...caseIds].flatMap((caseId) => required.map((arm) => `${caseId}:${arm}`));
+    const observedPairs = campaign.observations.filter((item) => required.includes(item.arm));
+    const scoredPairs = new Set(scored.reports.map((item) => `${item.case_id}:${item.arm}`));
+    const matrixComplete = caseIds.size > 0
+        && observedPairs.length === expectedPairs.length
+        && expectedPairs.every((pair) => observedPairs.some((item) => `${item.case_id}:${item.arm}` === pair) && scoredPairs.has(pair))
+        && required.every((arm) => scored.summary.some((item) => item.arm === arm));
+    const score = (arm) => scored.summary.find((item) => item.arm === arm)?.score.f1;
+    const a = score('A'), b = score('B'), d2 = score('D2'), r = score('R');
+    const aObservations = campaign.observations.filter((item) => item.arm === 'A');
+    const rObservations = campaign.observations.filter((item) => item.arm === 'R');
+    const usageComplete = aObservations.length > 0 && rObservations.length > 0
+        && [...aObservations, ...rObservations].every((item) => item.usage !== undefined);
+    const perCaseTokens = (items) => items.map((item) => item.usage.inputTokens + item.usage.outputTokens);
+    const aMedianTokens = usageComplete ? median(perCaseTokens(aObservations)) : null;
+    const rMedianTokens = usageComplete ? median(perCaseTokens(rObservations)) : null;
+    const tokenReduction = aMedianTokens !== null && rMedianTokens
+        ? 1 - aMedianTokens / rMedianTokens
+        : null;
+    const aMedianCalls = median(aObservations.map((item) => item.calls));
+    const aP95Calls = nearestRank(aObservations.map((item) => item.calls), 0.95);
+    const gates = {
+        matrix_complete: matrixComplete,
+        usage_complete: usageComplete,
+        primary_vs_b: a !== undefined && b !== undefined && a > b,
+        quality_floor_d2: a !== undefined && d2 !== undefined && a >= d2 - 0.05,
+        quality_floor_r: a !== undefined && r !== undefined && a >= r - 0.05,
+        token_savings: tokenReduction !== null && tokenReduction >= 0.4,
+        calls_median: aObservations.length > 0 && aMedianCalls <= 3,
+        calls_p95: aObservations.length > 0 && aP95Calls <= 6,
+    };
+    return {
+        ...gates,
+        pass: Object.values(gates).every(Boolean),
+        a_median_calls: aMedianCalls,
+        a_p95_calls: aP95Calls,
+        a_median_tokens: aMedianTokens,
+        r_median_tokens: rMedianTokens,
+        token_reduction: tokenReduction,
+    };
+}
 /** Pure frozen ship-gate calculation. Secondary wins cannot rescue either primary F1 loss. */
 export function evaluateIdeaV3Gates(scored, campaign) {
     scored = IdeaV3ScoredCampaign.parse(scored);
@@ -441,6 +517,75 @@ export function evaluateIdeaV3Gates(scored, campaign) {
         holdout_cases: cases.size,
     };
 }
+/** Publish Phase F build validation without turning it into a frozen holdout claim. */
+export async function writeIdeaV3AdaptiveResults(opts) {
+    const root = opts.root ?? process.cwd();
+    const scoredPath = resolve(opts.scoredPath);
+    const scored = IdeaV3ScoredCampaign.parse(JSON.parse(await readFile(scoredPath, 'utf8')));
+    const campaignPath = resolveFrom(dirname(scoredPath), scored.campaign);
+    const campaign = IdeaV3Campaign.parse(JSON.parse(await readFile(campaignPath, 'utf8')));
+    const gates = evaluateIdeaV3AdaptiveGates(scored, campaign);
+    const efficiency = ideaV3TokenEfficiency(scored, campaign);
+    const summary = new Map(scored.summary.map((item) => [item.arm, item.score]));
+    const rows = [
+        ['Complete scored A/B/B2/D2/R matrix', gates.matrix_complete, gates.matrix_complete ? 'complete' : 'missing or duplicate pairs'],
+        ['A/R token usage complete', gates.usage_complete, gates.usage_complete ? 'complete' : 'usage missing'],
+        ['A strictly beats B on F1', gates.primary_vs_b, `${summary.get('A')?.f1.toFixed(3)} vs ${summary.get('B')?.f1.toFixed(3)}`],
+        ['A within 0.05 F1 of D2', gates.quality_floor_d2, `${summary.get('A')?.f1.toFixed(3)} vs ${summary.get('D2')?.f1.toFixed(3)}`],
+        ['A within 0.05 F1 of R', gates.quality_floor_r, `${summary.get('A')?.f1.toFixed(3)} vs ${summary.get('R')?.f1.toFixed(3)}`],
+        ['A median tokens ≥40% below R', gates.token_savings, gates.token_reduction === null ? 'usage missing' : pct(gates.token_reduction)],
+        ['A median calls ≤3', gates.calls_median, gates.a_median_calls.toFixed(1)],
+        ['A p95 calls ≤6', gates.calls_p95, gates.a_p95_calls.toFixed(0)],
+    ];
+    const lines = [
+        '# RESULTS-IDEA-V3-ADAPTIVE — Phase F build validation',
+        '',
+        `**Adaptive build targets: ${gates.pass ? 'PASS' : 'FAIL'}**`,
+        '',
+        'Build-set product validation; not a frozen holdout claim.',
+        '',
+        '## Product targets',
+        '',
+        '| Target | Result | Evidence |',
+        '|---|---|---|',
+        ...rows.map(([name, pass, evidence]) => `| ${name} | ${pass ? 'PASS' : 'FAIL'} | ${evidence} |`),
+        '',
+        '## Primary metric',
+        '',
+        '| Arm | Matched | Recall | Precision | F1 |',
+        '|---|---:|---:|---:|---:|',
+        ...campaign.arms.map((arm) => {
+            const item = summary.get(arm);
+            return `| ${arm} | ${item.matched} | ${pct(item.recall)} | ${pct(item.precision)} | ${item.f1.toFixed(3)} |`;
+        }),
+        '',
+        '## Verified insights per 1,000 tokens',
+        '',
+        '| Arm | Matched | Tokens | Matched / 1k tokens | Estimated calls |',
+        '|---|---:|---:|---:|---:|',
+        ...efficiency.map((item) => `| ${item.arm} | ${item.matched} | ${item.tokens ?? 'missing'} | ${item.matched_per_1k_tokens?.toFixed(3) ?? 'missing'} | ${item.estimated_calls} |`),
+        '',
+        '## Every case and arm',
+        '',
+        '| Case | Arm | Status | Calls | Tokens | Repairs | Wall time | Flags / failure |',
+        '|---|---|---|---:|---:|---:|---:|---|',
+        ...[...campaign.observations]
+            .sort((left, right) => left.case_id.localeCompare(right.case_id) || left.arm.localeCompare(right.arm))
+            .map((item) => {
+            const tokens = item.usage ? item.usage.inputTokens + item.usage.outputTokens : 'missing';
+            const detail = item.status === 'error' ? item.error : item.flags.join(', ') || 'none';
+            return `| ${item.case_id} | ${item.arm} | ${item.status} | ${item.calls} | ${tokens} | ${item.repair_calls} | ${(item.latency_ms / 1000).toFixed(1)}s | ${mdCell(detail)} |`;
+        }),
+        '',
+        `Token values include input plus output. ${efficiency.some((item) => item.estimated_calls > 0) ? 'At least one call uses Phase A labeled estimation.' : 'All recorded calls use provider-reported totals.'}`,
+        '',
+    ];
+    const path = opts.outPath ?? join(root, 'RESULTS-IDEA-V3-ADAPTIVE.md');
+    const tmp = `${path}.tmp`;
+    await writeFile(tmp, lines.join('\n'), 'utf8');
+    await rename(tmp, path);
+    return { path, gates };
+}
 function mdCell(value) {
     return value.replaceAll('|', '\\|').replaceAll('\n', ' ');
 }
@@ -511,6 +656,16 @@ export async function writeIdeaV3Results(opts) {
     await rename(tmp, path);
     return { path, gates };
 }
+/** CLI publication dispatch: build amendments and frozen holdout results stay visibly separate. */
+export async function publishIdeaV3Results(opts) {
+    const scored = IdeaV3ScoredCampaign.parse(JSON.parse(await readFile(resolve(opts.scoredPath), 'utf8')));
+    if (scored.set === 'build') {
+        const result = await writeIdeaV3AdaptiveResults(opts);
+        return { path: result.path, passed: result.gates.pass, label: 'adaptive build targets' };
+    }
+    const result = await writeIdeaV3Results(opts);
+    return { path: result.path, passed: result.gates.ship, label: 'frozen ship gate' };
+}
 export const IdeaV3ProtocolDraft = z.object({
     build_scores: z.string().min(1),
     baseline_provider: z.enum(['claude', 'codex', 'agy']),
@@ -523,7 +678,7 @@ export const IdeaV3ProtocolDraft = z.object({
     }).strict(),
     lane_assignment: z.enum(['agy-market', 'codex-market']),
 }).strict();
-/** Freeze only a complete, already-scored B/C/D2/R build campaign; the file is one-shot. */
+/** Freeze only the original complete, already-scored B/C/D2/R build campaign; the file is one-shot. */
 export async function writeFrozenIdeaV3Protocol(opts) {
     const root = opts.root ?? process.cwd();
     const draftPath = resolve(opts.draftPath);
@@ -539,7 +694,7 @@ export async function writeFrozenIdeaV3Protocol(opts) {
     if (campaign.baseline_provider !== draft.baseline_provider) {
         throw new Error(`draft baseline ${draft.baseline_provider} does not match campaign ${campaign.baseline_provider}`);
     }
-    const arms = [...IDEA_V3_ARM_IDS];
+    const arms = ['B', 'C', 'D2', 'R'];
     if (arms.some((arm) => !campaign.arms.includes(arm) || !scored.summary.some((item) => item.arm === arm))) {
         throw new Error('protocol freeze requires scored B, C, D2, and R build arms');
     }

package/dist/cli/bench.js

@@ -6,7 +6,7 @@ import { planBench, renderTable, runBench } from '../bench/harness.js';
 import { setupProviders } from '../orchestration/context.js';
 import { chooseLaneDefault, importLaneAdjudications, planIdeaLaneBench, runIdeaLaneBench } from '../bench/idea-lane-rotation.js';
 import { IDEA_V3_ARM_IDS, planIdeaV3Bench, runIdeaV3Bench } from '../bench/idea-v3-bench.js';
-import { exportIdeaV3BlindBundle, importIdeaV3Ratings, writeFrozenIdeaV3Protocol, writeIdeaV3Results } from '../bench/idea-v3-rating.js';
+import { exportIdeaV3BlindBundle, importIdeaV3Ratings, publishIdeaV3Results, writeFrozenIdeaV3Protocol } from '../bench/idea-v3-rating.js';
 import { DISPLAY_NAME } from '../providers/types.js';
 const VALID_ARMS = ['A', 'B', 'C', 'D', 'E', 'L'];
 /** One-block pre-run summary: what will run + the ≈Opus cost, so the user commits knowingly (§19). */
@@ -34,9 +34,9 @@ export async function benchCommand(workflow, opts = {}) {
         }
         if (opts.publishResults) {
             try {
-                const result = await writeIdeaV3Results({ scoredPath: opts.publishResults });
+                const result = await publishIdeaV3Results({ scoredPath: opts.publishResults });
                 process.stdout.write(`\nresults: ${result.path}\n`);
-                process.stdout.write(`frozen ship gate: ${result.gates.ship ? 'PASS' : 'FAIL'}\n\n`);
+                process.stdout.write(`${result.label}: ${result.passed ? 'PASS' : 'FAIL'}\n\n`);
                 return 0;
             }
             catch (error) {
@@ -82,7 +82,7 @@ export async function benchCommand(workflow, opts = {}) {
             : set === 'build' ? ['B', 'C', 'D2', 'R'] : ['B', 'C', 'R'];
         const invalid = requested.filter((arm) => !IDEA_V3_ARM_IDS.includes(arm));
         if (invalid.length) {
-            process.stderr.write(`invalid idea-v3 arm(s): ${invalid.join(', ')}. Valid: B,C,D2,R\n`);
+            process.stderr.write(`invalid idea-v3 arm(s): ${invalid.join(', ')}. Valid: A,B,B2,C,D2,R\n`);
             return 1;
         }
         const arms = requested;
@@ -100,7 +100,7 @@ export async function benchCommand(workflow, opts = {}) {
             return 1;
         }
         process.stdout.write(`\nidea-v3 protocol comparison — ${set} — ${plan.cases.length} case(s) × arms ${arms.join(',')}\n`);
-        process.stdout.write(`B/C baseline provider: ${DISPLAY_NAME[provider]}\n`);
+        process.stdout.write(`B/B2/C baseline provider: ${DISPLAY_NAME[provider]}\n`);
         if (plan.resumedFrom)
             process.stdout.write(`resume: continuing ${plan.resumedFrom} — ${plan.skipCompleted} recorded pair(s) kept\n`);
         process.stdout.write(`to run: ${plan.toRun.length} case×arm pair(s) → ≤${plan.estimatedProviderCalls} nominal provider call(s)\n`);