amicus 1.1.0 → 1.2.0

package/skills/second-opinion/MODEL-NOTES.md

@@ -33,7 +33,9 @@ _Last updated: 2026-06-10 (v3 migration: engine workarounds pruned — see chang
   names — so rankings are directly comparable (one fanout wave distributes it).
 - **Require a `FINAL RANKING:` block** at the end of the response (e.g. `1. Review C / 2. Review
   A …`), plus a per-finding `agree | dispute | neutral` verdict with a one-line reason for each
-  finding referenced by label+id (e.g. `A2`).
+  finding referenced by run-global label id (e.g. `A2` = Review A's 2nd finding).
+- After de-anonymizing, assemble the tally input (see SKILL.md Stage 2 assembly recipe) and run
+  `amicus council tally <input.json> --json` — do not hand-tally tiers or street-cred numbers.
 
 ## Per-model notes
 
@@ -59,28 +61,32 @@ _Last updated: 2026-06-10 (v3 migration: engine workarounds pruned — see chang
 - Opus / o-series etc. are reachable via amicus **if their API keys are configured**. Add notes
   here the first time each is used.
 
-## Reviewer-reliability table
+## Reviewer-reliability
 
-Consulted in Stage 0 (council selection) and updated with approval in Stage 6.
+Quantitative reliability data (runs, avg peers-only street-cred, confirm-rate, fact-error rate, conformance distribution) is now generated from the append-only `council-ledger.jsonl` via:
 
-- **avg street-cred** — rolling average of this model's per-run street-cred (mean rank position
-  across judges' `FINAL RANKING:` blocks; lower = better).
-- **confirm-rate** — share of this model's findings that reached the **Confirmed** tier (agrees
-  outweigh disputes, ≥ 2 judges engaged).
+```
+amicus council stats [--json]
+```
 
-| model | runs | avg street-cred | confirm-rate | notes |
-| --- | --- | --- | --- | --- |
-| deepseek | 1 | 2.33 | 100% (12/12) | strong synthesis, resilient; chaired well |
-| gpt | 1 | 2.67 | 92% (23/25) | thorough but verbose; self-ranked #1 → discount; OpenRouter |
-| gemini | 1 | 3.67 | 89% (8/9) | fast, large-context; more absolute/adversarial ("blocker" inflation); ranked lowest |
+**Do not hand-edit reliability numbers here.** The ledger is the authoritative source; `amicus council stats` aggregates it and flags low-N models (`runs < 3`). Consult `amicus council stats` in Stage 0 for bench recommendations. In Stage 6 the ledger row is appended automatically — no manual update needed.
 
-_Scale note: the 2026-06-04 run used a 4-review pool (Claude in-council), so street-cred is on a
-1–4 scale rather than 1–3 — treat these as run-1 baselines, not directly comparable to future
-3-model runs. Merge/prune rather than append._
+This section keeps only per-model **qualitative quirks** and **structural-conformance notes** (`clean` / `repaired` / `unstructured`), which cannot be captured by the ledger:
+
+### Qualitative notes (hand-curated)
+
+- **deepseek** — strong synthesis, resilient; occasional transient 502 → re-run the leg. Proven chair. Conforms cleanly.
+- **gpt** — thorough but verbose; peers have dinged it for volume-over-judgment. Self-ranked its own review #1 in the 2026-06-04 run → the peers-only street-cred rule (now enforced by `tally`) mitigates this. Conforms cleanly. Accessible via OpenRouter.
+- **gemini** — fast, very large context; tends toward absolute severity labels ("blocker" inflation vs peers). Conforms cleanly; watch for preamble narration — instruct it to emit the JSON block verbatim after the prose.
 
 ## Cost guardrail
-- **Never** use `o3` / `o3-pro` unless the user explicitly asks for it by name — these cost
-  roughly $10–60+ per request. Warn about cost before proceeding even when asked.
+- The budget gate enforces this in code: a per-$/Mtok threshold (ON by default)
+  refuses o3/o3-pro-class models before a wave launches. This replaces the old
+  "remember not to" rule — it can no longer be forgotten.
+- To run `o3`/`o3-pro` (≈ $10–60+/request) the user must ask by name AND you
+  pass `--no-cost-gate` (disables both guards) for that run. Still warn first.
+- `--max-cost <$>` raises only the soft total ceiling; it does not unblock an
+  over-threshold model.
 
 ## General
 - Model citations are usually real but **verify any load-bearing reference before publishing**;

package/skills/second-opinion/SKILL.md

@@ -54,11 +54,11 @@ in this run is written here. Use its absolute path in all `--prompt-file` argume
   `--prompt-file` — never inline a briefing as a CLI argument. All `_tmp-*` files are cleaned up
   after the run.
 
-**Pick the council.** Default: **3 models from different families (non-Claude)**. Recommend them ranked by fit, consulting the reviewer-reliability table in `MODEL-NOTES.md`. State the estimated cost. **Disclose the run shape up front** before asking for confirmation — e.g.:
+**Pick the council.** Default: **3 models from different families (non-Claude)**. Recommend them ranked by fit, consulting both the reviewer-reliability data from `amicus council stats` (the authoritative quantitative source — runs, avg peers-only street-cred, confirm-rate, fact-error rate) and the qualitative quirks in `MODEL-NOTES.md`. State the estimated cost. The estimate is the budget gate's pre-flight figure (per-$/Mtok pricing from the cached catalog; direct-provider legs without catalog pricing are disclosed as "cost unknown"). State it as an estimate, not a guarantee. **Disclose the run shape up front** before asking for confirmation — e.g.:
 
 > This run uses 3 council models across 2 fanout waves + 1 chair call (~7 model runs), ~10 min.
 
-Then **wait for confirmation**. Never launch without it. Honor the cost guardrail in `MODEL-NOTES.md` (no `o3`/`o3-pro` without explicit ask-by-name).
+Then **wait for confirmation**. Never launch without it. The budget gate enforces the cost guardrail in code: by default it refuses any leg whose price exceeds the per-$/Mtok threshold (the o3/o3-pro guard). To run an intentionally expensive model the user explicitly asked for by name, pass `--no-cost-gate`; to raise only the total ceiling, pass `--max-cost <$>`.
 
 **Scale-down is explicit — state which mode applies:**
 - **1 model** → thorough single pass; Stage 2 (cross-review) and Stage 3 (chair synthesis) are skipped entirely; Claude synthesizes directly. Transport: a single solo `amicus start --no-ui --json` (no fanout).
@@ -110,18 +110,32 @@ equivalent.
 
 **Required structured output from every model.** Instruct each council model to produce:
 
-1. A **findings list** — every finding contains:
-   - `id` — sequential integer within this review (1, 2, 3…)
-   - `claim` — the specific issue or observation
-   - `severity` — one of: `blocker | major | minor | nit`
-   - `location` — section heading or verbatim quote identifying where in the artifact
-   - `rationale` — why this is a problem or worth noting
+1. A **prose review** — the reviewer's full narrative assessment of the artifact.
 
-2. A **short overall take** — 2–4 sentences summarizing the reviewer's overall assessment.
+2. A **trailing fenced ` ```json ` block** immediately after the prose, containing:
+   ```json
+   {
+     "overall": "one-paragraph take",
+     "findings": [
+       { "id": 1, "severity": "blocker",
+         "claim": "…", "location": "…", "rationale": "…" }
+     ]
+   }
+   ```
+   - `id` — sequential integer within this review (`1..n`); at Stage-2 assembly Claude rewrites each into a **run-global label id** (`A1`, `B1`, …) by prefixing the review's anonymized label.
+   - `severity ∈ {blocker, major, minor, nit}`
+   - `claim`, `location`, `rationale` — non-empty strings.
 
-Instruct models to emit the structured output verbatim, without preamble, so it reads cleanly.
+Instruct models to emit the structured JSON verbatim after the prose, without preamble, so it parses cleanly.
 
-When the wave returns, save each leg's `summary` to the run folder as `review-<model>.md`
+**After the wave returns, validate each leg's findings block** using `validateFindings` (Unit A — `src/council/findings.js`). If a leg's JSON fails validation:
+1. Issue a **solo `start --json`** re-prompt to that one model: "re-emit only the findings JSON, fixing: \<errors\>." Keep the first-pass prose. (Solo `start` is **not** subject to the WS-2 fanout cost gate, so a repair cannot be refused mid-council.)
+2. If still malformed, retry **once more** (cap = **2** re-prompts total).
+3. If still malformed after 2 retries, mark the review `unstructured` and hand-parse its prose into the schema. The review proceeds — never dropped for a formatting miss.
+
+Record per-model **conformance** (`clean` | `repaired` | `unstructured`) for inclusion in the tally input's `runStats` and the Stage-6 MODEL-NOTES note.
+
+Save each leg's full output (prose + findings block) to the run folder as `review-<model>.md`
 (one file per reviewer) before moving on.
 
 **"Claude in the council" (when toggled on):** Claude also produces a **fresh** Stage-1 review on the artifact in the identical findings format — a new structured pass on the artifact, not a formalization of anything said upstream. This review is added to the bundle as one more anonymous entry. Claude does not rank or adjudicate in Stage 2 (it holds the label map), and does not chair in Stage 3. Save it as `review-claude.md`.
@@ -172,7 +186,21 @@ FINAL RANKING:
 
 **Task B — Adjudicate findings.** For every finding in the bundle, state: `agree | dispute | neutral` plus one-line reason. Reference each finding as **review-label + finding-id** — for example, `A2` means Review A's 2nd finding, `B1` means Review B's 1st finding. An "I missed this — it's valid" counts as `agree`.
 
-As each judge's ranking + adjudication response returns, collect it (the raw per-judge responses are working intermediates, not separate run-folder artifacts). Once all are in, de-anonymize and tally them into the single `crossreview-matrix.md` — the adjudication grid plus the street-cred table (see *Output & naming*). This de-anonymized data feeds Stage 3 (chair briefing), the scoring/street-cred table, and the cross-review matrix artifact — but is never re-anonymized or forwarded to any council model.
+As each judge's ranking + adjudication response returns, collect it (the raw per-judge responses are working intermediates, not separate run-folder artifacts). Once all are in, **assemble the de-anonymized tally input** and then call `amicus council tally`:
+
+**Stage-2 → tally assembly recipe (Claude's work before calling `tally`):**
+1. **Rewrite finding ids to run-global label ids.** Each Stage-1 review's local integer ids (`1`, `2`, `3`…) become `A1`, `A2`, `A3`… (where `A` is that review's anonymized label). The label↔model map (`Review A → deepseek`, etc.) is the key.
+2. **Build `adjudications`** — for every judge across all findings: `findingId` = run-global label id; `judge` = the model id (de-anonymized via the map); `verdict ∈ {agree, dispute, neutral}`. Include every judge's verdict on every finding. The raiser's own adjudication of its own finding is **included in the input** (the tally engine excludes it when computing peers-only tiers — do not pre-filter it).
+3. **Translate each judge's `FINAL RANKING:` block** — convert the label order (`1. Review C / 2. Review A / 3. Review B`) into a model `order` array via the same map (e.g. `{C→mistral, A→deepseek, B→gpt}` ⇒ `order: ["mistral","deepseek","gpt"]`). This is each entry in `rankings[]`.
+4. **Populate `runStats`** from the per-leg run documents emitted by `fanout --json` (and any solo red-team/chair `start --json` docs): copy `model`, `status`, `durationMs`, `usage` verbatim. Any leg with no run doc gets `durationMs: null` and `usage: null` — never invent a value. Attach `role` (`council` | `redteam` | `claude`), `wasChair`, and `conformance` (`clean` | `repaired` | `unstructured`) as council-domain labels.
+
+Then call:
+
+```
+amicus council tally <run-folder>/tally-input.json --json
+```
+
+The output `record` carries the deterministic tiers (Disputed / Confirmed / Contested / Singleton), `confidence` (`solid` | `thin`), both street-cred numbers (`withSelf` and `peersOnly`), the validated `runStats`, and `tierCounts`. **Claude may override a `thin`-confidence tier at the margins** before Stage 4 — record the override in `tierOverride: {from, to, reason}`; the matrix and `verdict.json` surface it. De-anonymize and write the tally results to `crossreview-matrix.md` — the adjudication grid plus the street-cred table. This data feeds Stage 3 (chair briefing) and is never re-anonymized or forwarded to any council model.
 
 ---
 
@@ -180,7 +208,7 @@ As each judge's ranking + adjudication response returns, collect it (the raw per
 
 A designated **non-Claude** chair synthesizes the verdict across all reviews, rankings, and adjudications. The chair produces an independent verdict that Claude then presents — Claude does not paraphrase, edit, or re-synthesize it.
 
-**Chair selection (confirmed in Stage 0).** Default: Claude recommends the strongest reasoner in the council (guided by the reviewer-reliability table in `MODEL-NOTES.md`) and the user confirms before the run launches. The chair may be a council member who already participated in Stages 1 and 2 — it receives the de-anonymized full bundle, all ranking outputs, and all adjudications so it has the complete picture.
+**Chair selection (confirmed in Stage 0).** Default: Claude recommends the strongest reasoner in the council (guided by `amicus council stats` (peers-only street-cred) and the qualitative quirks in `MODEL-NOTES.md`) and the user confirms before the run launches. The chair may be a council member who already participated in Stages 1 and 2 — it receives the de-anonymized full bundle, all ranking outputs, and all adjudications so it has the complete picture.
 
 **Fallback order if the chair fails:**
 1. Re-run the chair call (transient failure — `MODEL-NOTES.md` mitigations apply).
@@ -212,11 +240,11 @@ Save the chair's output to the run folder as `verdict.md`.
 
 ### Stage 4 — Tiered decisions (peer-validated)
 
-All findings from the bundle are sorted into two tiers based on the **peer-confidence tier derived from the Stage 2 adjudication data**. These tiers are **derived from the Stage 2 adjudications** — a judgment call, not a rigid formula (see *Key mechanics → §5.2 Scoring* for the full rule): a finding is **Confirmed** when agrees clearly outweigh disputes (≥ 2 judges engaged), **Contested** when there is a meaningful split or explicit disputes, and **Singleton** when only its original raiser stands behind it. Present the tiers in this order.
+All findings from the bundle are sorted into tiers based on the **peer-confidence tier assigned by `amicus council tally`** (see *Key mechanics → §5.2 Scoring* in COUNCIL-DESIGN.md for the full cascade): **Disputed** (strong peer pushback — `d ≥ 2` and `d > a`), **Confirmed** (≥ 2 peer agreements, agrees dominate), **Contested** (at least one live dispute), **Singleton** (at most one endorsement, no pushback). `confidence: thin` cells `(0,0)/(1,0)/(0,1)` are override-eligible (Claude records any override in `tierOverride`). Present the tiers in this order: Confirmed first (bulk decision), then Disputed and Contested and Singleton individually in the judgment tier.
 
 **Scale-down:** In a 1-model run, Stage 2 was skipped — there is no peer-confidence data, so present every finding individually for decision (no tiers). In a 2-model run, the Confirmed tier rests on thin cross-review (one ranker per review, per Stage 0) — say so when presenting it.
 
-**Consensus tier — Confirmed findings** (peers agree clearly outweigh disputes, with ≥ 2 judges engaged)
+**Consensus tier — Confirmed findings** (≥ 2 peer agreements, agrees dominate)
 
 - Present the full list in one block: id, claim, severity, and which models raised / endorsed it.
 - Offer one **bulk accept/deny decision** over the whole tier:
@@ -225,12 +253,13 @@ All findings from the bundle are sorted into two tiers based on the **peer-confi
 
 - The user may accept the block, deny the block, or enumerate exceptions. Handle exceptions individually before moving on.
 
-**Judgment tier — Contested and Singleton findings**
+**Judgment tier — Disputed, Contested, and Singleton findings**
 
-This is one tier with two sub-types presented separately. Present each finding individually. Handle the two sub-types distinctly:
+This is one tier with three sub-types presented separately. Present each finding individually. Handle the sub-types distinctly:
 
-- **Contested** (meaningful split or explicit disputes): For each finding show the claim and severity, which model raised it, who agreed, who disputed, and the one-line reasons from the adjudications. Ask for a decision before proceeding to the next: **accept / deny / modify**.
-- **Singleton** (only the original raiser; no other judge engaged — neutral or silent): For each finding show the claim and severity and that no other judge engaged with it. Name the sole raiser. Ask for a decision before proceeding to the next: **accept / deny / modify**.
+- **Disputed** (`d ≥ 2` and `d > a` — strong peer pushback, the finding itself may be wrong): For each finding show the claim, severity, which model raised it, which peers dispute it and why. Ask for a decision before proceeding to the next: **accept / deny / modify**.
+- **Contested** (`d ≥ 1` with a meaningful split): For each finding show the claim and severity, which model raised it, who agreed, who disputed, and the one-line reasons from the adjudications. Ask for a decision before proceeding to the next: **accept / deny / modify**.
+- **Singleton** (only the original raiser; all other judges were neutral or silent — `d = 0` and `a < 2`): For each finding show the claim and severity and that no other judge engaged with it. Name the sole raiser. Ask for a decision before proceeding to the next: **accept / deny / modify**.
 
 **Recording decisions.** Keep a running decision log throughout this stage — every finding's outcome (accepted / denied / modified, with any modification noted). This log feeds Stage 5 (only accepted changes go into the reviewed copy) and Stage 6 (the run-folder report).
 
@@ -249,15 +278,19 @@ Do not advance to Stage 5 until every finding in both tiers has a recorded decis
 - Do not attempt to produce a modified copy.
 - Write a **standalone reviewed report** instead: the full decision log, the chair's verdict, and clear callouts of what should be changed and where — formatted so the user can apply the changes manually.
 
-**Run-folder artifacts — always write these** regardless of source type. The full artifact set and naming conventions are defined in the *Output & naming* section of this skill; write every artifact specified there. The four canonical run-folder files are:
+**Run-folder artifacts — always write these** regardless of source type. The full artifact set and naming conventions are defined in the *Output & naming* section of this skill; write every artifact specified there. The canonical run-folder files are:
 - `review-<model>.md` × N (already saved in Stage 1)
 - `crossreview-matrix.md` — the de-anonymized adjudication grid and street-cred table
 - `verdict.md` (already saved in Stage 3)
+- `verdict.json` — write via `buildVerdict(record, decisions)` (`src/council/verdict.js`): pass the tally `record` from Stage 2 and the Stage-4 decision map (accepted / denied / modified / deferred per finding, plus any `duplicateOf` links Claude identified). This is the schema-stamped machine-readable record of the full run. Write it with an atomic tmp+rename to the run folder.
 - `report.md` — the chair's synthesis + the full Stage-4 decision log + a summary of what was
   applied (+ the "How Claude's review fared" readout when "Claude in the council" is on) + a
   **run-stats table**: one row per model call — **stage** (which stage you launched the call for)
-  plus **model, status, durationMs** read from the wave/run JSON documents. The schema carries no
-  cost data — do not invent cost figures.
+  plus **model, status, durationMs, and cost** read from the wave/run JSON `usage`
+  block. Cost is `usage.cost.amount` (USD); mark it with its `usage.cost.source`
+  — exact for `reported`, `~` for `estimated`, `?` for `unknown` — and never
+  invent a figure. Add a wave **total cost** row from the wave document's
+  `usage.cost` (`source: reported|estimated|mixed|unknown`). Any leg with no run doc → `durationMs: null`, `usage: null`; never invent a value.
 
 Tell the user exactly which files were written and where.
 
@@ -274,10 +307,7 @@ This stage updates `MODEL-NOTES.md` to make future runs better. **Nothing is wri
 
 Draft new or updated entries for the per-model sections of `MODEL-NOTES.md` that capture what was learned.
 
-**Update the reviewer-reliability table.** After every completed council run, update the rolling table in `MODEL-NOTES.md` (the "Reviewer reliability" table) for each council model that participated:
-- **avg street-cred** — incorporate this run's rank position into each model's running average.
-- **confirm-rate** — incorporate this run's share of each model's findings that ended up Confirmed.
-- Merge into the existing row for that model; prune the notes column to stay tight.
+**Ledger auto-append (automatic — no approval required).** After `verdict.json` is written, the ledger row is appended automatically when the tally record is finalized — `ledger.appendRun(record)` writes one row per (run × model) to the append-only `council-ledger.jsonl` under `getConfigDir()`. The run summary shows the appended row. This is a deterministic, content-free model-level record (no finding text, no claim strings, no artifact body content). The quantitative reviewer-reliability data in `MODEL-NOTES.md` is now sourced entirely from `amicus council stats` (which aggregates the ledger) — **do not hand-edit reliability numbers in MODEL-NOTES**.
 
 **Compose the proposed MODEL-NOTES diff.** Combine the run-lessons updates and the reviewer-reliability table updates into a single proposed diff (old → new for every changed section). **Write the full diff to a file in the run folder** — `_tmp-proposed-model-notes-update.md` — so the user can open and review it before deciding. Presenting the diff as chat text alone is **not sufficient**: an approval dialog can hide the chat transcript, so the user may be asked to decide on a diff they never saw.
 
@@ -309,23 +339,21 @@ Claude **de-anonymizes only** at two points: when computing scores and when writ
 
 ### §5.2 Scoring
 
-Claude tallies two scoring signals from the Stage-2 outputs. No code is required; Claude works through the structured output directly.
-
-**Street-cred** = each model's **average rank position** across all judges' `FINAL RANKING:` blocks (lower is better). For example, if three judges rank DeepSeek 1st, 2nd, and 1st, its street-cred score is 1.33. Surface this as a compact table in the cross-review matrix and report. Street-cred drives the chair's weighting of reviewer findings in Stage 3 and feeds the reviewer-reliability table updated in Stage 6.
+`amicus council tally` computes the two scoring signals from the assembled tally input. Claude's role is to assemble the input (Stage-2 assembly recipe in Stage 2 above) and to exercise judgment on `thin`-confidence overrides.
 
-**Per-finding peer-confidence tier** = a qualitative label derived from the Stage-2 adjudications for each finding:
+**Street-cred** — computed two ways:
+- **withSelf** = each model's mean rank position across **all** judges' `FINAL RANKING:` blocks (lower is better).
+- **peersOnly** = mean rank excluding the model's own ranking of itself.
 
-- **Confirmed** — agrees clearly outweigh disputes, with at least 2 judges having engaged with the finding.
-- **Contested** — a meaningful split exists or explicit disputes were recorded.
-- **Singleton** — only the original raiser stands behind it; all other judges were neutral or silent.
+Both are surfaced in `crossreview-matrix.md` and `report.md`. The ledger and Stage-0 bench recommendations use **peersOnly** only.
 
-These three tiers drive the Stage-4 decision flow. Assigning a tier is a **judgment call, not a rigid formula** — Claude reads the adjudication signals and makes the call at the margins, especially when engagement is sparse or agreements and disputes are close in number. When in doubt, downgrade toward Contested or Singleton rather than overstate confidence.
+**Per-finding peer-confidence tier** — assigned by the peers-only cascade in `amicus council tally` (see COUNCIL-DESIGN.md §5.2 for the full table): **Disputed** → **Confirmed** → **Contested** → **Singleton**. The raiser's own adjudication is excluded from the cascade. `confidence: thin` when total engaged peers `a + d ≤ 1` — cells `(0,0)`, `(1,0)`, `(0,1)`. **Claude may override a `thin` tier at the margins** before presenting Stage 4 — the override is recorded in `tierOverride: {from, to, reason}` and surfaced in the matrix and `verdict.json`.
 
 ---
 
 ### §5.3 Chair selection & fallback
 
-The default is for Claude to **recommend a non-Claude chair** from the council — typically the model with the strongest reasoning capability or the best reviewer-reliability score in `MODEL-NOTES.md` — and the user confirms this recommendation before the run launches (Stage 0). The chair **may** be a council member who already participated in Stages 1 and 2; it receives the full de-anonymized picture (all reviews with model attribution, all rankings, all adjudications) so it can synthesize from a complete view.
+The default is for Claude to **recommend a non-Claude chair** from the council — typically the model with the strongest reasoning capability or the best peers-only street-cred from `amicus council stats` — and the user confirms this recommendation before the run launches (Stage 0). The chair **may** be a council member who already participated in Stages 1 and 2; it receives the full de-anonymized picture (all reviews with model attribution, all rankings, all adjudications) so it can synthesize from a complete view.
 
 **Fallback chain if the chair call fails:**
 
@@ -346,8 +374,8 @@ Enabling this toggle lets the bench judge Claude's own take, so you can see how
 **Always fresh.** When the toggle is on, Claude performs a new structured Stage-1 review on the artifact — a fresh pass in the required findings format, not a formalization or summary of anything said earlier in the main conversation. Upstream feedback does not seed or constrain this review.
 
 **"How Claude's review fared" readout.** Included in both `crossreview-matrix.md` and `report.md` when the toggle is on:
-- Claude's street-cred rank among peers (its average rank position in the judges' `FINAL RANKING:` blocks).
-- The Confirmed / Contested / Singleton split of Claude's findings — how many of its claims the bench endorsed, contested, or ignored.
+- Claude's street-cred rank among peers (its `peersOnly` average rank position in the judges' `FINAL RANKING:` blocks — `withSelf == peersOnly` for Claude since it never casts rankings).
+- The Disputed / Confirmed / Contested / Singleton split of Claude's findings — how many of its claims the bench pushed back on, endorsed, disputed, or ignored.
 
 **Integrity.** When Claude presents results — including the bench's assessment of its own review — it reports the verdict at face value. Claude does not defend, contextualize away, or re-litigate findings the bench disputed or ranked poorly. The point of the toggle is an honest external read on Claude's review; undermining that defeats the purpose.
 
@@ -355,16 +383,16 @@ Enabling this toggle lets the bench judge Claude's own take, so you can see how
 
 ## Model-recommendation heuristics
 
-Use these together with the reviewer-reliability table in `MODEL-NOTES.md`, which holds live performance data from prior runs:
+Use these together with `amicus council stats` (the ledger — authoritative quantitative reliability data: runs, avg peers-only street-cred, confirm-rate, fact-error rate) and the qualitative quirks in `MODEL-NOTES.md`:
 
 - **Large or long material, broad coverage sweep** → favor a large-context model (e.g., Gemini) that won't truncate or degrade on the full source.
 - **Reasoning-heavy critique, structured argument evaluation, citations** → favor a strong reasoner (e.g., DeepSeek, GPT, Opus) that will interrogate claims rather than accept them.
 - **Code review** → favor a code-strong model (e.g., DeepSeek, GPT, Opus); general-purpose models often miss implementation-level issues.
 - **Independence matters** → pick models from **different families**; two models from the same family produce correlated opinions and reduce the value of the cross-review.
 - **Contrarian / red-team value** → when material is persuasive, consensus-prone, or high-stakes, assign one model an explicit red-team brief: argue against the others, hunt for what they will miss. This is especially valuable when the default council is likely to agree.
-- **Consult the reviewer-reliability table** in `MODEL-NOTES.md` — a model's historical confirm-rate and avg street-cred are the best predictors of council value for a given run type.
+- **Consult `amicus council stats`** — a model's historical confirm-rate and avg peers-only street-cred (from the ledger) are the best predictors of council value for a given run type.
 
-Always **rank recommendations by fit**, state the trade-off for each option, and surface the estimated cost. Never present a single option without explanation.
+Always **rank recommendations by fit**, state the trade-off for each option, and surface the estimated cost (an estimate, not a guarantee; unpriced legs disclosed as "cost unknown"). Never present a single option without explanation.
 
 ---
 
@@ -373,9 +401,14 @@ Always **rank recommendations by fit**, state the trade-off for each option, and
 - Run folder: `output/<stem>-council/` (or `./second-opinion/<stem>-council/` if no `output/` exists), containing:
   - `review-<model>.md` ×N — raw Stage 1 reviews (plus `review-claude.md` when "Claude in the council" is on)
   - `crossreview-matrix.md` — adjudication grid + de-anonymized street-cred table
-  - `verdict.md` — the chair's synthesis
+  - `verdict.md` — the chair's synthesis (prose)
+  - `verdict.json` — schema-stamped machine-readable record: tally output + Stage-4 decisions, written via `buildVerdict(record, decisions)` at Stage 5
   - `report.md` — synthesis + decision log + what was applied (+ the "How Claude's review fared" readout when the toggle is on) + a
-    **run-stats table**: one row per model call — **stage** (which stage you launched the call for) plus **model, status, durationMs** read from the wave/run JSON documents. The schema carries no cost data — do not invent cost figures.
+    **run-stats table**: one row per model call — **stage** (which stage you launched the call for) plus **model, status, durationMs, and cost** read from the wave/run JSON `usage`
+    block. Cost is `usage.cost.amount` (USD); mark it with its `usage.cost.source`
+    — exact for `reported`, `~` for `estimated`, `?` for `unknown` — and never
+    invent a figure. Add a wave **total cost** row from the wave document's
+    `usage.cost` (`source: reported|estimated|mixed|unknown`). Any leg with no run doc → `durationMs: null`, `usage: null`.
 - Reviewed copy: `<stem>-reviewed.<ext>`, next to the source.
 - Temp working files (`_tmp-*.md`: extracts, stage briefings, red-team brief, bundle, chair packet, proposed
   MODEL-NOTES diff) live in the run folder and are cleaned up at the end of the run — the proposed-diff file
@@ -385,6 +418,5 @@ Always **rank recommendations by fit**, state the trade-off for each option, and
 
 ## Files
 
-- `MODEL-NOTES.md` — operating rules, per-model quirks, cost guardrail, and the reviewer-reliability rolling table. **Read it before Stage 0 (council selection and launch); update it (with approval) in Stage 6.**
-- `COUNCIL-DESIGN.md` — the design spec this skill implements (v3). Consult it if a mechanics
-  question arises that the skill prose does not resolve.
+- `MODEL-NOTES.md` — operating rules, per-model qualitative quirks, cost guardrail, and structural-conformance notes. **Read it before Stage 0 (council selection and launch); update qualitative notes (with approval) in Stage 6.** Quantitative reliability data (runs, avg street-cred, confirm-rate, fact-error rate) comes from `amicus council stats`, not this file.
+- `COUNCIL-DESIGN.md` — the design spec this skill implements (v3 + WS-3). Consult it if a mechanics question arises that the skill prose does not resolve.

package/src/cli-handlers-council.js

@@ -0,0 +1,59 @@
+// src/cli-handlers-council.js
+'use strict';
+const fs = require('fs');
+const { tally } = require('./council/tally');
+const { deriveReliability } = require('./council/ledger');
+const { failJson, ERROR_CODES } = require('./utils/error-doc');
+
+function runTally(inputPath, useJson) {
+  if (!inputPath) {
+    return failJson(useJson, { code: ERROR_CODES.BAD_ARGS, message: 'council tally needs an <input.json> path',
+      hint: 'amicus council tally <input.json> [--json]' });
+  }
+  let input;
+  try { input = JSON.parse(fs.readFileSync(inputPath, 'utf-8')); }
+  catch (e) {
+    return failJson(useJson, { code: ERROR_CODES.BAD_ARGS, message: `cannot read ${inputPath}: ${e.message}`,
+      hint: 'pass a valid tally input JSON file' });
+  }
+  let record;
+  try { record = tally(input); }
+  catch (e) {
+    return failJson(useJson, { code: ERROR_CODES.BAD_ARGS, message: `malformed tally input: ${e.message}`,
+      hint: 'input needs meta.models, findings[], adjudications[], rankings[]' });
+  }
+  process.stdout.write(useJson ? JSON.stringify(record, null, 2) + '\n' : renderRecord(record));
+  return 0;
+}
+
+function runStats(useJson) {
+  const agg = deriveReliability();
+  process.stdout.write(useJson ? JSON.stringify(agg, null, 2) + '\n' : renderStats(agg));
+  return 0;
+}
+
+function renderRecord(r) {
+  const t = r.tierCounts;
+  return `Council tally (${r.meta.runId})\n` +
+    `  Confirmed ${t.Confirmed}  Contested ${t.Contested}  Singleton ${t.Singleton}  Disputed ${t.Disputed}\n`;
+}
+function renderStats(agg) {
+  if (!agg.length) { return 'No council runs recorded yet.\n'; }
+  return 'model            runs  avg-cred  confirm  fact-err  notes\n' +
+    agg.map(a => `${a.model.padEnd(16)} ${String(a.runs).padStart(4)}  ` +
+      `${fmt(a.avgStreetCredPeersOnly)}     ${fmt(a.lifetimeConfirmRate)}    ${fmt(a.lifetimeFactErrorRate)}` +
+      `${a.lowN ? '   low-N' : ''}`).join('\n') + '\n';
+}
+function fmt(v) { return (v === null || v === undefined) ? '  —  ' : v.toFixed(2); }
+
+/** @param {{_:string[], json?:boolean}} args @returns {Promise<number>} */
+async function handleCouncil(args) {
+  const sub = args._[1];
+  const useJson = !!args.json;
+  if (sub === 'tally') { return runTally(args._[2], useJson); }
+  if (sub === 'stats') { return runStats(useJson); }
+  return failJson(useJson, { code: ERROR_CODES.BAD_ARGS,
+    message: `unknown council subcommand '${sub || ''}'`, hint: 'amicus council tally|stats' });
+}
+
+module.exports = { handleCouncil };

package/src/cli-handlers-doctor.js

@@ -0,0 +1,173 @@
+// src/cli-handlers-doctor.js
+'use strict';
+
+const MAX_CATALOG_AGE_MS = 24 * 60 * 60 * 1000; // 24h (mirrors model-catalog DEFAULT_MAX_AGE_MS)
+
+/** Default real helpers; tests override via deps. */
+function realDeps() {
+  const fs = require('fs');
+  const os = require('os');
+  const path = require('path');
+  return {
+    nodeVersion: process.version,
+    readApiKeys: () => require('./utils/api-key-store').readApiKeys(),
+    getConfigDir: () => require('./utils/config').getConfigDir(),
+    resolveModel: () => require('./utils/config').resolveModel(),
+    readCache: () => require('./utils/model-catalog').readCache(),
+    collectAliasSources: () => require('./utils/alias-audit').collectAliasSources(),
+    findStaleAliases: (s, c) => require('./utils/alias-audit').findStaleAliases(s, c),
+    hasOpencodeBinary: () => {
+      const { ensureNodeModulesBinInPath } = require('./utils/path-setup');
+      ensureNodeModulesBinInPath();
+      const root = path.join(__dirname, '..', 'node_modules');
+      const candidates = process.platform === 'win32'
+        ? [path.join(root, `opencode-windows-${os.arch() === 'arm64' ? 'arm64' : 'x64'}`, 'bin', 'opencode.exe'),
+           path.join(root, `opencode-windows-${os.arch() === 'arm64' ? 'arm64' : 'x64'}-baseline`, 'bin', 'opencode.exe')]
+        : [path.join(root, '.bin', 'opencode')];
+      return candidates.some(p => fs.existsSync(p));
+    },
+    getElectronPath: () => require('./sidecar/interactive').getElectronPath(),
+    discoverClaudeCodeMcps: () => require('./utils/mcp-discovery').discoverClaudeCodeMcps(),
+    discoverCoworkMcps: () => require('./utils/mcp-discovery').discoverCoworkMcps(),
+    skillInstalled: () => {
+      const dir = path.join(os.homedir(), '.claude', 'skills');
+      return fs.existsSync(path.join(dir, 'sidecar', 'SKILL.md'))
+        && fs.existsSync(path.join(dir, 'second-opinion', 'SKILL.md'));
+    },
+  };
+}
+
+/** Run one guarded check; a thrown fn becomes an error line. */
+function guard(id, name, fn) {
+  try { return fn(); }
+  catch (e) { return { id, name, status: 'error', message: e.message, hint: null }; }
+}
+
+/**
+ * Compose the health checks. Never throws.
+ * @param {object} [depsOverride]
+ * @returns {Array<{id,name,status,message,hint}>}
+ */
+function runDoctorChecks(depsOverride = {}) {
+  const d = { ...realDeps(), ...depsOverride };
+  const checks = [];
+
+  checks.push(guard('node', 'Node.js', () => {
+    const major = parseInt(String(d.nodeVersion).replace(/^v/, '').split('.')[0], 10);
+    return major >= 18
+      ? { id: 'node', name: 'Node.js', status: 'ok', message: d.nodeVersion, hint: null }
+      : { id: 'node', name: 'Node.js', status: 'error', message: `${d.nodeVersion} (need >=18)`, hint: 'Install Node 18 or newer from https://nodejs.org' };
+  }));
+
+  checks.push(guard('config-dir', 'Config directory', () => (
+    { id: 'config-dir', name: 'Config directory', status: 'ok', message: d.getConfigDir(), hint: null }
+  )));
+
+  checks.push(guard('keys', 'API keys', () => {
+    const keys = d.readApiKeys();
+    const set = Object.keys(keys).filter(k => keys[k]);
+    return set.length > 0
+      ? { id: 'keys', name: 'API keys', status: 'ok', message: `configured: ${set.join(', ')}`, hint: null }
+      : { id: 'keys', name: 'API keys', status: 'error', message: 'no provider keys configured', hint: 'amicus key <provider> <key>  (or run: amicus setup)' };
+  }));
+
+  checks.push((() => {
+    try {
+      const model = d.resolveModel();
+      return { id: 'default-model', name: 'Default model', status: 'ok', message: model, hint: null };
+    } catch (e) {
+      return { id: 'default-model', name: 'Default model', status: 'error', message: e.message || 'no default model', hint: 'amicus setup' };
+    }
+  })());
+
+  checks.push(guard('catalog', 'Model catalog', () => {
+    const cache = d.readCache();
+    if (!cache || !cache.fetchedAt) {
+      return { id: 'catalog', name: 'Model catalog', status: 'warn', message: 'no cache yet', hint: 'amicus models --refresh' };
+    }
+    const ageMs = Date.now() - cache.fetchedAt;
+    const fresh = ageMs <= MAX_CATALOG_AGE_MS;
+    const hrs = Math.round(ageMs / 3600000);
+    return fresh
+      ? { id: 'catalog', name: 'Model catalog', status: 'ok', message: `${cache.models.length} models, ${hrs}h old`, hint: null }
+      : { id: 'catalog', name: 'Model catalog', status: 'warn', message: `stale (${hrs}h old)`, hint: 'amicus models --refresh' };
+  }));
+
+  checks.push(guard('aliases', 'Model aliases', () => {
+    const cache = d.readCache();
+    const catalog = (cache && cache.models) || [];
+    const stale = d.findStaleAliases(d.collectAliasSources(), catalog);
+    return stale.length === 0
+      ? { id: 'aliases', name: 'Model aliases', status: 'ok', message: catalog.length ? 'all resolve' : 'catalog empty — not checked', hint: null }
+      : { id: 'aliases', name: 'Model aliases', status: 'warn', message: `${stale.length} stale: ${stale.map(s => s.alias).join(', ')}`, hint: 'amicus models --check' };
+  }));
+
+  checks.push(guard('opencode-bin', 'OpenCode binary', () => (
+    d.hasOpencodeBinary()
+      ? { id: 'opencode-bin', name: 'OpenCode binary', status: 'ok', message: 'found', hint: null }
+      : { id: 'opencode-bin', name: 'OpenCode binary', status: 'error', message: 'not found', hint: 'npm install -g amicus' }
+  )));
+
+  checks.push(guard('electron', 'Electron (interactive GUI)', () => (
+    d.getElectronPath()
+      ? { id: 'electron', name: 'Electron (interactive GUI)', status: 'ok', message: 'installed', hint: null }
+      : { id: 'electron', name: 'Electron (interactive GUI)', status: 'warn', message: 'not installed — headless still works', hint: 'npm install -g amicus  (reinstall to add Electron)' }
+  )));
+
+  checks.push(guard('skills', 'Skills installed', () => (
+    d.skillInstalled()
+      ? { id: 'skills', name: 'Skills installed', status: 'ok', message: '~/.claude/skills/{sidecar,second-opinion}', hint: null }
+      : { id: 'skills', name: 'Skills installed', status: 'warn', message: 'one or both skills missing', hint: 'npm install -g amicus  (re-runs the skill install)' }
+  )));
+
+  checks.push(guard('mcp', 'MCP registration', () => {
+    const code = d.discoverClaudeCodeMcps();
+    const cowork = d.discoverCoworkMcps();
+    const inCode = !!(code && code.amicus);
+    const inCowork = !!(cowork && cowork.amicus);
+    // Primary signal: Claude Code MCP registration. Cowork/Desktop is reported as bonus only.
+    if (!inCode) {
+      return { id: 'mcp', name: 'MCP registration', status: 'warn', message: 'not registered in Claude Code', hint: 'npm install -g amicus  (or install the amicus plugin)' };
+    }
+    const extra = inCowork ? ', Cowork/Desktop' : '';
+    return { id: 'mcp', name: 'MCP registration', status: 'ok', message: `registered: Claude Code${extra}`, hint: null };
+  }));
+
+  return checks;
+}
+
+const MARK = { ok: '✓', warn: '⚠', error: '✗' }; // ✓ ⚠ ✗
+
+function renderHuman(checks) {
+  let out = 'amicus doctor\n\n';
+  for (const c of checks) {
+    out += `${MARK[c.status] || '?'} ${c.name}: ${c.message}\n`;
+    if (c.hint && c.status !== 'ok') { out += `    → ${c.hint}\n`; }
+  }
+  const errors = checks.filter(c => c.status === 'error').length;
+  const warns = checks.filter(c => c.status === 'warn').length;
+  out += `\n${errors} error(s), ${warns} warning(s).\n`;
+  return out;
+}
+
+/**
+ * `amicus doctor [--json]`. Injectable `runChecks` for tests.
+ * @param {{_:string[], json?:boolean}} args
+ * @param {(deps?:object)=>Array} [runChecks]
+ * @returns {Promise<number>} exit code
+ */
+async function handleDoctor(args, runChecks = runDoctorChecks) {
+  const useJson = !!args.json;
+  const checks = runChecks();
+  if (useJson) {
+    const { buildDoctorDoc } = require('./utils/result-schema');
+    const VERSION = require('../package.json').version;
+    const doc = buildDoctorDoc({ version: VERSION, timestamp: new Date().toISOString(), checks });
+    process.stdout.write(JSON.stringify(doc, null, 2) + '\n');
+    return doc.ok ? 0 : 1;
+  }
+  process.stdout.write(renderHuman(checks));
+  return checks.some(c => c.status === 'error') ? 1 : 0;
+}
+
+module.exports = { runDoctorChecks, handleDoctor, MAX_CATALOG_AGE_MS };