amicus 4.4.0 → 4.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +32 -0
- package/README.md +3 -1
- package/docs/DISTRIBUTION.md +234 -0
- package/docs/ROADMAP.md +200 -0
- package/docs/SHIMS.md +62 -0
- package/docs/architecture.md +104 -0
- package/docs/configuration.md +371 -0
- package/docs/council.md +911 -0
- package/docs/doc-system.md +92 -0
- package/docs/electron-testing.md +471 -0
- package/docs/jsdoc-setup.md +75 -0
- package/docs/opencode-integration.md +114 -0
- package/docs/publishing.md +60 -0
- package/docs/schemas.md +55 -0
- package/docs/testing.md +589 -0
- package/docs/troubleshooting.md +298 -0
- package/docs/usage.md +699 -0
- package/electron/fold.js +1 -1
- package/electron/main.js +4 -1
- package/electron/setup-ui-aliases.js +6 -6
- package/electron/workspace-ui/live-model.js +12 -1
- package/electron/workspace-ui/md-lite.js +52 -8
- package/electron/workspace-ui/workspace-matrix.js +46 -9
- package/electron/workspace-ui/workspace-panels.js +14 -3
- package/electron/workspace-ui/workspace-render.js +7 -1
- package/electron/workspace-ui/workspace-verbs.js +48 -2
- package/package.json +8 -3
- package/schemas/council-run.schema.json +20 -0
- package/schemas/progress.schema.json +12 -0
- package/schemas/spend.schema.json +52 -4
- package/src/cli-handlers-spend.js +20 -2
- package/src/cli-handlers-watch.js +11 -0
- package/src/cli.js +4 -2
- package/src/council/briefings-debate.js +27 -7
- package/src/council/briefings-stage2.js +155 -25
- package/src/council/briefings.js +24 -1
- package/src/council/findings.js +236 -9
- package/src/council/parse-stage2.js +10 -2
- package/src/council/report.js +19 -8
- package/src/council/run-assemble.js +42 -1
- package/src/council/run-budget.js +64 -11
- package/src/council/run-chair.js +4 -1
- package/src/council/run-debate.js +4 -2
- package/src/council/run-finalize.js +102 -0
- package/src/council/run-launch.js +29 -1
- package/src/council/run-server.js +248 -0
- package/src/council/run-stage2.js +118 -0
- package/src/council/run-stages.js +132 -111
- package/src/council/run-state.js +23 -1
- package/src/council/run.js +44 -46
- package/src/council/tally.js +10 -0
- package/src/headless.js +175 -6
- package/src/observe/council-legs.js +60 -3
- package/src/observe/live-doc.js +18 -1
- package/src/observe/watch-render.js +4 -1
- package/src/sidecar/child-sessions.js +1 -2
- package/src/sidecar/fanout-leg-fallback.js +69 -21
- package/src/sidecar/fanout-leg.js +6 -0
- package/src/sidecar/fanout-signals.js +61 -0
- package/src/sidecar/fanout-wave-io.js +75 -0
- package/src/sidecar/fanout.js +61 -70
- package/src/sidecar/progress-fields.js +26 -4
- package/src/sidecar/progress.js +8 -1
- package/src/sidecar/session-utils.js +23 -14
- package/src/spend-query.js +17 -5
- package/src/utils/lifecycle.js +37 -1
- package/src/utils/path-fence.js +39 -1
- package/src/utils/pricing.js +26 -10
- package/src/utils/server-setup.js +79 -1
- package/src/utils/spend-ledger.js +24 -3
- package/src/workspace/artifact-guard.js +22 -1
- package/src/workspace/fold-format.js +33 -4
- package/src/workspace/live-normalize.js +28 -15
- package/src/workspace/run-detail.js +7 -1
package/docs/council.md
ADDED
|
@@ -0,0 +1,911 @@
|
|
|
1
|
+
# Council Reference — Pipeline, Schemas, and a Worked Example
|
|
2
|
+
|
|
3
|
+
> Quick summary is in the [README "The Council" section](../README.md#the-council). This document
|
|
4
|
+
> is the field-by-field reference for the `amicus council` subcommands: what each one reads, what
|
|
5
|
+
> it writes, and how the pipeline chains together. It is self-contained — every command and JSON
|
|
6
|
+
> example below is real and was run against the CLI while writing this doc.
|
|
7
|
+
|
|
8
|
+
This page exists because `amicus council tally|stats|report|validate|verdict|save|list|show` are
|
|
9
|
+
**deterministic local math and file I/O** — none of them call a model. The one exception is
|
|
10
|
+
[`amicus council run`](#amicus-council-run) (v4.0), the **headless engine**: it drives the whole
|
|
11
|
+
pipeline below — Stage-1 reviews → anonymized cross-review → tally → chair verdict — in one
|
|
12
|
+
command, and it does call models. In the interactive path, the models run in the
|
|
13
|
+
`second-opinion` skill's Stage 1/2/3 waves (`amicus fanout` / `amicus start`); these subcommands
|
|
14
|
+
consume and produce the JSON that glues those stages together. If you're driving a live council
|
|
15
|
+
run, follow **[skills/second-opinion/SKILL.md](../skills/second-opinion/SKILL.md)** — it's the
|
|
16
|
+
orchestration recipe. This page is the reference for the artifacts that recipe produces.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Table of contents
|
|
21
|
+
|
|
22
|
+
- [The pipeline, end to end](#the-pipeline-end-to-end)
|
|
23
|
+
- [`amicus council run`](#amicus-council-run)
|
|
24
|
+
- [Debate mode](#debate-mode)
|
|
25
|
+
- [Council Workspace (GUI)](#council-workspace-gui)
|
|
26
|
+
- [`amicus council validate`](#amicus-council-validate)
|
|
27
|
+
- [`amicus council tally`](#amicus-council-tally)
|
|
28
|
+
- [`amicus council verdict`](#amicus-council-verdict)
|
|
29
|
+
- [`amicus council report`](#amicus-council-report)
|
|
30
|
+
- [`amicus council stats`](#amicus-council-stats)
|
|
31
|
+
- [Council presets: `save` / `list` / `show`](#council-presets-save--list--show)
|
|
32
|
+
- [Worked example](#worked-example)
|
|
33
|
+
- [Where artifacts live](#where-artifacts-live)
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## The pipeline, end to end
|
|
38
|
+
|
|
39
|
+
```
|
|
40
|
+
Stage 1 reviews (models, via `amicus fanout`)
|
|
41
|
+
│ each leg saved as review-<model>.md (prose + trailing ```json findings block)
|
|
42
|
+
▼
|
|
43
|
+
amicus council validate <leg-file> ← per-leg findings-block check (tri-state exit)
|
|
44
|
+
│ ok:true → findings[] usable
|
|
45
|
+
▼
|
|
46
|
+
Stage 2 cross-review (models rank + adjudicate, via `amicus fanout`)
|
|
47
|
+
│ Claude assembles the de-anonymized tally-input JSON (see SKILL.md's
|
|
48
|
+
│ "Stage-2 → tally assembly recipe")
|
|
49
|
+
▼
|
|
50
|
+
amicus council tally <tally-input.json> ← deterministic tiers + street-cred + ledger append
|
|
51
|
+
│ writes/prints the tally RECORD (save it as tally.json)
|
|
52
|
+
▼
|
|
53
|
+
Stage 4 decisions (Claude presents tiers, user accepts/denies/modifies)
|
|
54
|
+
│ Claude writes decisions.json — one {id, decision, ...} object per finding
|
|
55
|
+
▼
|
|
56
|
+
amicus council verdict <tally.json> --decisions <decisions.json> -o verdict.json
|
|
57
|
+
│ merges the tally record + decisions into the schema-stamped verdict.json
|
|
58
|
+
▼
|
|
59
|
+
amicus council report <verdict.json> [--md|--html]
|
|
60
|
+
│ renders the adjudication matrix, street-cred table, findings-by-tier, cost
|
|
61
|
+
▼
|
|
62
|
+
report.md (Claude-authored synthesis) + report.html (deterministic renderer output)
|
|
63
|
+
|
|
64
|
+
Independently, at any time:
|
|
65
|
+
amicus council stats ← ledger-derived reviewer reliability (all past runs)
|
|
66
|
+
amicus council save|list|show ← named --models lists for `--council <name>`
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Three things to hold onto:
|
|
70
|
+
|
|
71
|
+
1. **`tally` is the finalize step.** It both computes the deterministic record *and* appends one
|
|
72
|
+
row per (run × model) to the append-only reliability ledger (`~/.config/amicus/council-ledger.jsonl`,
|
|
73
|
+
or `$AMICUS_CONFIG_DIR/council-ledger.jsonl` when that env var is set) — unless you pass
|
|
74
|
+
`--no-ledger`. `stats` reads that ledger back.
|
|
75
|
+
2. **`verdict` doesn't recompute anything.** It's a pure merge of the tally record (deterministic
|
|
76
|
+
tiers/street-cred, already computed) with your Stage-4 decisions (accept/deny/modify, made by a
|
|
77
|
+
human via Claude). See [provenance](#amicus-council-verdict) below.
|
|
78
|
+
3. **`report` never touches a model.** It reads `verdict.json` (+ optional `wave.json` for the cost
|
|
79
|
+
total) and renders it — Markdown or a self-contained HTML page. All prose synthesis (the actual
|
|
80
|
+
"what does this mean" writeup) is `report.md`, written by Claude in Stage 5 of the skill — a
|
|
81
|
+
*different* artifact from what this renderer produces. See
|
|
82
|
+
[Where artifacts live](#where-artifacts-live).
|
|
83
|
+
4. **Optional council elements (v2.2.0) are orchestration-level — nothing in these subcommands
|
|
84
|
+
changed for them.** The skill's opt-in elements (critic seat, expert lenses, debate mode, chair
|
|
85
|
+
verdict scale — see [SKILL.md](../skills/second-opinion/SKILL.md) and
|
|
86
|
+
[SEAT-BRIEFS.md](../skills/second-opinion/SEAT-BRIEFS.md)) ride on existing engine surfaces:
|
|
87
|
+
seat roles travel as free-form `runStats[].role` labels (`"critic"`, `"lens:<slug>"`), debate
|
|
88
|
+
mode runs the Stage-2 tally with `--no-ledger` (provisional) and re-tallies after the rebuttal
|
|
89
|
+
round (that second, post-rebuttal tally is the ledger-recorded one), and lens runs always pass
|
|
90
|
+
`--no-ledger` so non-comparable reviews never feed `stats`.
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## `amicus council run`
|
|
95
|
+
|
|
96
|
+
```
|
|
97
|
+
amicus council run --prompt-file <briefing.md>
|
|
98
|
+
(--council <preset> | --models a,b,c) # >=2 seats, XOR (fanout semantics)
|
|
99
|
+
[--chair <model>] # default deepseek; must NOT be a bench seat
|
|
100
|
+
[--critic <model>] # must BE a bench seat; adversarial brief
|
|
101
|
+
[--lenses s1,s2,...] # count == seat count; forces no-ledger;
|
|
102
|
+
# mutually exclusive with --critic in v4.0
|
|
103
|
+
[--out-dir <dir>] # default ./council-<runId>/
|
|
104
|
+
[--json] [--max-cost <usd>] [--timeout <min>]
|
|
105
|
+
[--gateway auto|direct|openrouter] [--no-validate-model]
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
**The headless engine (v4.0).** Everything the `second-opinion` skill orchestrates by hand in
|
|
109
|
+
Stages 1–3+5 — seat briefings (anti-sycophancy clause included), the Stage-1 review wave,
|
|
110
|
+
anonymization and run-global finding-id rewriting, the identical judge bundle, the Stage-2
|
|
111
|
+
cross-review wave with bounded repair re-prompts, the tally, chair synthesis with the verdict
|
|
112
|
+
scale, and the deterministic report — runs as **one command with no Claude runtime**. Stage 4
|
|
113
|
+
stays human: the engine is report-only and never fabricates accept/deny decisions
|
|
114
|
+
(`amicus council verdict --decisions` remains the post-hoc path).
|
|
115
|
+
|
|
116
|
+
Key semantics:
|
|
117
|
+
|
|
118
|
+
- `--prompt-file` is **required** — councils always have real briefings (no inline `--prompt`).
|
|
119
|
+
- Seat/chair/critic/lens validation happens **pre-flight** and fails through the error envelope
|
|
120
|
+
(exit 1) before any spend. The chair must not be a bench seat.
|
|
121
|
+
- `--timeout` is the **per-leg** timeout (existing fanout semantics); there is no run-level
|
|
122
|
+
watchdog in v4.0 — bound the aggregate with your CI job timeout.
|
|
123
|
+
- `--max-cost` is a **whole-run** ceiling checked before each paid stage launch (Stage-1 wave,
|
|
124
|
+
repair solos, Stage-2 wave, chair). Hitting it mid-run stops launching and finalizes what
|
|
125
|
+
exists; in-flight legs are never aborted for cost.
|
|
126
|
+
- **It bounds KNOWN spend, and it never blocks a run.** A leg whose cost could not be
|
|
127
|
+
determined contributes nothing to the figure the ceiling is checked against, is never guessed
|
|
128
|
+
at, and never halts anything — the standing ruling is fail *loud*, not fail *closed*. The
|
|
129
|
+
measured consequence of the older, quieter version: `council-wsgate02` really spent
|
|
130
|
+
**$0.9859 against a $0.75 ceiling (131%)** while amicus believed $0.3720 — and exited `0`.
|
|
131
|
+
So when a ceiling is set **and** the run's total is inexact (`usage.costExact: false` — any
|
|
132
|
+
`unknownLegs` or `subtreeUnknownLegs`), the run **exits `2`**, through the same degrade
|
|
133
|
+
channel as a bench the ceiling shrank — the stages, verdict and usage block are untouched;
|
|
134
|
+
`exitCode` becomes `2` and `status` becomes `partial`, as for every other degradation, because
|
|
135
|
+
`0`/`complete` reads as "clean, and inside your ceiling" and a run publishing a floor has not
|
|
136
|
+
earned that. With **no** ceiling there is nothing to be inexact against and an unpriced leg
|
|
137
|
+
leaves the exit code (and status) alone.
|
|
138
|
+
- Each launch is measured against the **remaining** allowance (ceiling − known spend −
|
|
139
|
+
allowances already claimed by a wave that is launching right now). Stage 1 launches its seat
|
|
140
|
+
wave and its critic/lens waves concurrently, so the claim is atomic — two waves can never
|
|
141
|
+
both spend the same remaining dollars. **If the ceiling refuses one of them, the run
|
|
142
|
+
continues with a partial bench**: launched waves are never rolled back and the run is never
|
|
143
|
+
aborted for cost. The refusal is printed as a `Notice:` naming the wave and its models,
|
|
144
|
+
recorded on `run.json` as `budgetRefusals[]`, and degrades the exit code to `2`. If it takes
|
|
145
|
+
the bench below two reviews, the usual `COUNCIL_QUORUM` failure (exit 1) applies.
|
|
146
|
+
- A run starts **one** OpenCode server and threads it through every wave (two concurrent
|
|
147
|
+
starts race on OpenCode's SQLite). Whether that worked is always on the record, in the
|
|
148
|
+
affirmative as well as the negative — exactly one of these two keys is present:
|
|
149
|
+
- `sharedServer` `{acquired: true, at, goPid, models}` — the run got its shared server and
|
|
150
|
+
every wave rode it. `goPid` is the server's pid; no wave writes a `goPid` into its own
|
|
151
|
+
`metadata.json` while riding an injected server, so this is direct evidence rather than
|
|
152
|
+
an inference drawn from a `goPid` appearing where it should not.
|
|
153
|
+
- `sharedServerUnavailable` `{error, at}` — the server could not start and the run **still
|
|
154
|
+
proceeded** on one server per wave, the configuration that races. Also printed as a
|
|
155
|
+
`Notice:`. It does not change the exit code; treat its presence as "expect degraded
|
|
156
|
+
results".
|
|
157
|
+
- Chair failure recovery: one retry of the same chair → promote the highest peers-only
|
|
158
|
+
street-cred model (from `amicus council stats`) that is not a bench seat → give up and write
|
|
159
|
+
the verdict with `overallVerdict: null`.
|
|
160
|
+
- SIGINT/SIGTERM abort the active wave/solo, finalize `run.json` as `aborted`, exit 130/143.
|
|
161
|
+
`amicus abort <councilRunId>` (and the MCP tools via the sessions-dir pointer file
|
|
162
|
+
`council-<runId>.json`) work on council runs; `status`/`list` resolve them the same way.
|
|
163
|
+
There is no CLI `wait` — the MCP `amicus_wait` tool is the blocking primitive and resolves
|
|
164
|
+
council runs the same way.
|
|
165
|
+
|
|
166
|
+
**Exit codes and degradation:**
|
|
167
|
+
|
|
168
|
+
| Condition | Behavior | Exit |
|
|
169
|
+
|---|---|---|
|
|
170
|
+
| All stages complete, chair verdict parsed | full run | 0 |
|
|
171
|
+
| Fewer than 2 completed Stage-1 reviews | stop; error doc `COUNCIL_QUORUM` | 1 |
|
|
172
|
+
| At least 2 reviews but fewer than 2 completed judges | proceed; tally `judged:false` | 2 |
|
|
173
|
+
| Chair fails (1 retry + 1 fallback promotion) | `verdict.json` written, `overallVerdict:null` | 2 |
|
|
174
|
+
| Chair output missing `VERDICT:` line after 1 repair | chair prose kept, `overallVerdict:null` | 2 |
|
|
175
|
+
| Cost ceiling hit after the tally exists | verdict written (no chair), `overallVerdict:null` | 2 |
|
|
176
|
+
| Cost ceiling hit before the tally | stop; error doc `COST_EXCEEDED` | 1 |
|
|
177
|
+
| Cost ceiling refused a wave at pre-flight | partial bench; `Notice:` + `run.json` `budgetRefusals[]` | 2 |
|
|
178
|
+
| `--max-cost` set and `usage.costExact:false` | stages/verdict/usage untouched; `run.json` `status` becomes `partial` — the total is a floor, so it is not reported as clean | 2 |
|
|
179
|
+
| Aborted | `run.json` status `aborted` | 130/143 |
|
|
180
|
+
|
|
181
|
+
**The run directory** (durable state; skill-compatible layout):
|
|
182
|
+
|
|
183
|
+
```
|
|
184
|
+
council-<runId>/
|
|
185
|
+
briefing-stage1.md # composed seat briefing (user briefing + templates)
|
|
186
|
+
review-<model>.md x N # Stage-1 outputs
|
|
187
|
+
bundle-stage2.md # anonymized judge bundle (identical for all judges)
|
|
188
|
+
judge-<model>.md x N # Stage-2 raw outputs
|
|
189
|
+
chair-packet.md # de-anonymized chair packet (+ verdict-scale addendum)
|
|
190
|
+
chair-output.md # chair raw output
|
|
191
|
+
tally-input.json # the assembled five-keys object (auditability)
|
|
192
|
+
tally.json # engine tally record (council family v2)
|
|
193
|
+
verdict.json # undecided verdict (tiers + overallVerdict)
|
|
194
|
+
report.html # deterministic renderer output
|
|
195
|
+
run.json # manifest: schemaVersion 2, type council-run, stages, usage
|
|
196
|
+
_scratch/ # cwd for judge legs (isolation)
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
Two more files appear when the run was started through **`amicus_council_run`** rather than the
|
|
200
|
+
CLI, both written by the MCP handler before it spawns the engine:
|
|
201
|
+
|
|
202
|
+
```
|
|
203
|
+
briefing.md # the briefing the tool copied in (the child briefs off this copy)
|
|
204
|
+
spawn.pid # the spawned child's pid
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
`spawn.pid` exists so a child that dies *before* the engine checkpoints its own pid into
|
|
208
|
+
`run.json` is still detectable: `amicus status` crash detection and `amicus abort`'s process
|
|
209
|
+
fallback both read `run.json`'s pid first and fall back to this file. It is deliberately a
|
|
210
|
+
separate single-write file rather than a field patched into `run.json` — the spawning process and
|
|
211
|
+
the engine child both write `run.json`, and its checkpoint is a read-merge-write with no
|
|
212
|
+
cross-process lock.
|
|
213
|
+
|
|
214
|
+
`verdict.json` here is the **undecided** verdict — same schema as [`amicus council
|
|
215
|
+
verdict`](#amicus-council-verdict)'s output (council family v2) plus **`overallVerdict`**
|
|
216
|
+
(`"Ship it" | "Fix these first" | "Fundamental rethink" | null`), parsed from the chair's final
|
|
217
|
+
`VERDICT:` line. Example excerpt:
|
|
218
|
+
|
|
219
|
+
```json
|
|
220
|
+
{
|
|
221
|
+
"schemaVersion": 2,
|
|
222
|
+
"runId": "pr-142-council",
|
|
223
|
+
"chair": "deepseek",
|
|
224
|
+
"overallVerdict": "Fix these first",
|
|
225
|
+
"tierCounts": { "Confirmed": 2, "Contested": 1, "Singleton": 1, "Disputed": 0 }
|
|
226
|
+
}
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
One-shot CI-shaped example (this is exactly what the Council Review GitHub Action runs):
|
|
230
|
+
|
|
231
|
+
```bash
|
|
232
|
+
$ amicus council run --models gemini,glm --chair deepseek \
|
|
233
|
+
--prompt-file briefing.md --out-dir council-run \
|
|
234
|
+
--json --max-cost 2.00 --timeout 10 --no-validate-model
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
Consumers gate on **tiers + the chair verdict line** (`overallVerdict`), per the engine's
|
|
238
|
+
report-only Stage-4 policy. Headless runs pin `meta.claudeInCouncil: false`,
|
|
239
|
+
`meta.runType: "headless"`, and the chair is excluded from the street-cred universe.
|
|
240
|
+
|
|
241
|
+
### Debate mode
|
|
242
|
+
|
|
243
|
+
`--debate` adds a **Stage-2.5 rebuttal round** between cross-review and the final tally
|
|
244
|
+
(COUNCIL-DESIGN.md §12.3): **provisional tally → defense → re-vote → final tally**, exactly
|
|
245
|
+
one round.
|
|
246
|
+
|
|
247
|
+
1. **Provisional tally.** Immediately after Stage-2 cross-review, the engine tallies with
|
|
248
|
+
`--no-ledger` and writes `tally-provisional.json`. If nothing landed Contested or Disputed,
|
|
249
|
+
there is nothing to debate — the engine skips straight to the final tally
|
|
250
|
+
(`debate.outcome: "nothing-to-debate"`).
|
|
251
|
+
2. **Defense.** Every Contested/Disputed finding goes back to its raiser as one concurrent solo
|
|
252
|
+
run — `rebuttal-<model>.md` per raiser — asking for exactly one of `DEFEND` / `AMEND` /
|
|
253
|
+
`WITHDRAW` per finding. A dead or unparseable defense leg means the original claim stands
|
|
254
|
+
undefended.
|
|
255
|
+
3. **Re-vote.** Defended/amended findings go back to the judges who disputed them, as ONE
|
|
256
|
+
shared fanout wave — `revote-bundle.md` (the shared prompt, written to the run dir like
|
|
257
|
+
Stage 2's `bundle-stage2.md`) + `revote-<model>.md` per judge. A missing/unparseable re-vote
|
|
258
|
+
line leaves that judge's original verdict standing.
|
|
259
|
+
4. **Final tally.** The engine reassembles the tally input with the defense/re-vote outcomes
|
|
260
|
+
folded in and re-tallies — this final, post-rebuttal tally is the one that appends to the
|
|
261
|
+
reliability ledger (a lens run is the only thing that suppresses the append — `council run` has no `--no-ledger` escape hatch; that flag is parsed only by `council tally`). Withdrawn findings stay
|
|
262
|
+
in `findings[]` and are auto-recorded `denied` at Stage 4 — never presented for a user
|
|
263
|
+
decision.
|
|
264
|
+
|
|
265
|
+
**Exactly one round** — there is no second defense/re-vote cycle; whatever remains unsettled
|
|
266
|
+
after the re-vote keeps its final tier.
|
|
267
|
+
|
|
268
|
+
**Where it shows up:**
|
|
269
|
+
- `run.json`'s `debate` object summarizes the round: `{enabled, outcome, contested, disputed,
|
|
270
|
+
defended, amended, withdrawn, noResponse, revoteJudges, revoteApplied, verdictChanges}`.
|
|
271
|
+
`outcome` is `"nothing-to-debate"`, `"ran"`, or `"skipped-cost-ceiling"` (the whole-run cost
|
|
272
|
+
ceiling was hit before a warranted re-vote).
|
|
273
|
+
- Every debated finding in `tally.json`/`verdict.json` carries a `findings[].debate` object —
|
|
274
|
+
`{action: "defended"|"amended"|"withdrawn"|"no-response", previousTier}` — decorating the
|
|
275
|
+
finding with what happened in the round and the tier it held before the re-vote.
|
|
276
|
+
- Extra run-dir artifacts, written only when a defense/re-vote actually ran:
|
|
277
|
+
`tally-provisional.json`, `revote-bundle.md`, `debate.json` (the round's structured record),
|
|
278
|
+
`rebuttal-<model>.md` × (raisers), `revote-<model>.md` × (disputing judges).
|
|
279
|
+
- `--claude-review <file>` enters Claude's own review (from a file, no leg launched) as a judged
|
|
280
|
+
entry; per the reserved-seat rule, it is never asked to defend in the debate round — its
|
|
281
|
+
Contested/Disputed findings simply stand, the same "originals stand" outcome as a dead defense
|
|
282
|
+
leg.
|
|
283
|
+
|
|
284
|
+
---
|
|
285
|
+
|
|
286
|
+
## Council Workspace (GUI)
|
|
287
|
+
|
|
288
|
+
Watch a council think — not just tail a log:
|
|
289
|
+
|
|
290
|
+
```bash
|
|
291
|
+
amicus watch <councilRunId> --ui # open one run
|
|
292
|
+
amicus watch --ui # open the run list for this project
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
This is a third Electron mode (`AMICUS_MODE=council-workspace`), opened by `amicus watch --ui`
|
|
296
|
+
itself — not a separate command or a separate launch surface (that's v4.5). One window per
|
|
297
|
+
launch:
|
|
298
|
+
|
|
299
|
+
- **Run list** (bare `--ui`, no id) — every council run in the current project, newest first,
|
|
300
|
+
discovered from the sessions-dir pointer files (`council-<runId>.json`); each row shows status,
|
|
301
|
+
the chair's `overallVerdict` chip once one exists, and cost.
|
|
302
|
+
- **Run detail** (`--ui <runId>`) — header + status/verdict chips, a stage rail, a live **Seats**
|
|
303
|
+
table (model, role, status, stage, messages, tokens, cost, last activity, a stall flag), the
|
|
304
|
+
Stage-1 reviews, the **verbatim** anonymized Stage-2 packet (`bundle-stage2.md`, shown as-is —
|
|
305
|
+
never re-rendered), judge prose, the **adjudication matrix** (finding × judge, tier-colored
|
|
306
|
+
rows, `a/d/n` basis counts, a `thin` badge when `a+d<=1`, an override badge when a Stage-4
|
|
307
|
+
decision changed a finding's tier, capped at 500 rows with a "showing N of M" note past that),
|
|
308
|
+
dissent drill-in (click a ✗ cell to open that judge's prose with the finding id highlighted —
|
|
309
|
+
the Stage-2 contract carries no structured reason field, so rationale lives in prose; on a
|
|
310
|
+
`--debate` run, a re-voted cell instead opens `revote-<model>.md` and also surfaces the
|
|
311
|
+
structured `reason` `debate.json` records for that re-vote), chair verdict + street-cred +
|
|
312
|
+
Stage-4 decisions, and a cost-by-seat table with a `--max-cost` ceiling gauge.
|
|
313
|
+
|
|
314
|
+
**Historical runs** render entirely from the run directory — open any old `council-<runId>` at any
|
|
315
|
+
time; nothing here requires the run to still be live.
|
|
316
|
+
|
|
317
|
+
**Live updates.** While a run is in progress, the window polls the same v4.3 data layer `amicus
|
|
318
|
+
watch` reads from a terminal — every 1.5s while the window is visible and focused, every 5s
|
|
319
|
+
otherwise, stopping once the run reaches a terminal status. A stall (no leg activity for a while)
|
|
320
|
+
surfaces as a banner with an Abort shortcut next to it; if a live-data read itself fails, a
|
|
321
|
+
separate "live data unavailable" banner appears while the last-known panels stay on screen — the
|
|
322
|
+
poll keeps retrying rather than blanking anything.
|
|
323
|
+
|
|
324
|
+
**Blind mode** (toggle, top right): labels (`Review A`, `Review B`…) instead of model names —
|
|
325
|
+
**ON by default while a run is live, OFF once it reaches a terminal status**, flippable either way
|
|
326
|
+
at any time. This is a **reading aid against anchoring bias, not a security control** — the label
|
|
327
|
+
map is `run.json`'s own `labelMap` field, sitting in plaintext in the run directory like every
|
|
328
|
+
other artifact; nothing stops you opening it in a text editor.
|
|
329
|
+
|
|
330
|
+
**Masks the roster, not just the seat table** (amended 2026-07-25 — see §6 resolved-Q2 amendment
|
|
331
|
+
in the design spec). Blind mode covers *every* place a model id co-occurs with review authorship:
|
|
332
|
+
the header's bench/critic/chair chips, the currently-open run's own row in the run-list rail
|
|
333
|
+
(its chair chip), seat rows, cost rows, and revote titles — the same `display(pair, blind)`
|
|
334
|
+
formatter backs all of them, so there is one place to get this right instead of five. Best-effort
|
|
335
|
+
only for other rows in the run list: each row's chair can only be masked when a label happens to
|
|
336
|
+
be known for it (in practice, only the currently-open run's own row), so an unopened run's row
|
|
337
|
+
still shows its raw chair id — consistent with "reading aid," not a hard guarantee.
|
|
338
|
+
|
|
339
|
+
**Two verbs, nothing else:**
|
|
340
|
+
|
|
341
|
+
- **Abort** — confirm-gated, hidden once the run is terminal. It calls the same council-aware
|
|
342
|
+
abort path `amicus abort` uses. This is the one place the workspace changes anything on disk,
|
|
343
|
+
and it does so by delegating to the engine's own abort handling — not a direct write from the
|
|
344
|
+
workspace code itself.
|
|
345
|
+
- **Fold** — writes the nonced `[SIDECAR_FOLD:…]` block plus the chair's verdict to the launching
|
|
346
|
+
terminal's stdout (no model call — it reformats what's already on disk). Folding again after a
|
|
347
|
+
successful fold just reports "already folded"; it doesn't write a second time.
|
|
348
|
+
|
|
349
|
+
Apart from Abort, the workspace is **read-only against the run directory**. `--ui` is
|
|
350
|
+
interactive-only — there is no `--json` for it, and passing both fails fast rather than silently
|
|
351
|
+
falling back to the terminal renderer. Closing the window never auto-folds — everything is
|
|
352
|
+
already on disk, so nothing is lost; reopen with `amicus watch <runId> --ui` and fold whenever
|
|
353
|
+
you're ready.
|
|
354
|
+
|
|
355
|
+
**Degraded states are rendered honestly, never hidden:** a run whose `run.json` can't be parsed at
|
|
356
|
+
all shows an "unreadable" banner with the error and the run directory path; a run written by a
|
|
357
|
+
different amicus schema version shows a schema-mismatch banner instead of guessing at a rendering;
|
|
358
|
+
a tally with fewer than 2 completed judges shows an explicit "tally is peers-reduced" note **above**
|
|
359
|
+
the adjudication matrix — the matrix still renders over the surviving judges, and the note is what
|
|
360
|
+
stops it being read as more authoritative than the underlying data supports; and a
|
|
361
|
+
chair-less verdict (retry + fallback promotion both failed, or the cost ceiling was hit before the
|
|
362
|
+
chair ran) shows "no chair verdict" plus the engine's own reason, never a blank panel.
|
|
363
|
+
|
|
364
|
+
**Posture, briefly** — this page renders another model's prose, so it's the most locked-down page
|
|
365
|
+
in the app: full `sandbox`/`contextIsolation`, a minimal preload exposing exactly one `invoke()`
|
|
366
|
+
gated by a 7-channel allowlist, a CSP with **no network directive at all** (`default-src 'none'`),
|
|
367
|
+
and every model-derived string reaches the DOM through `textContent`/`createTextNode` only —
|
|
368
|
+
never `innerHTML`, enforced by a static source scan in the test suite.
|
|
369
|
+
|
|
370
|
+
---
|
|
371
|
+
|
|
372
|
+
## `amicus council validate`
|
|
373
|
+
|
|
374
|
+
```
|
|
375
|
+
amicus council validate <file> [--json]
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
Validates a Stage-1 reviewer's saved output (`review-<model>.md`: prose + a trailing ` ```json `
|
|
379
|
+
fenced block) against the findings-block contract, without calling a model. Thin CLI wrapper over
|
|
380
|
+
`validateFindings` (`src/council/findings.js`).
|
|
381
|
+
|
|
382
|
+
**What it checks**, in order:
|
|
383
|
+
- A ` ```json ` fenced block exists (last one in the file wins if there are several) —
|
|
384
|
+
`NO_FENCED_BLOCK` if not.
|
|
385
|
+
- It parses as JSON — `NOT_PARSEABLE` if not.
|
|
386
|
+
- `findings` is present and is an array — `EMPTY_FINDINGS` if it is missing or is some other
|
|
387
|
+
type. **An array that is present and empty is valid**, provided `overall` is a non-empty
|
|
388
|
+
string; `EMPTY_FINDINGS` if the array is empty *and* `overall` is missing, blank, or not a
|
|
389
|
+
string. See "A clean review is a valid review" below.
|
|
390
|
+
- Every finding has a **sequential integer `id`** starting at 1 (`NON_SEQUENTIAL_ID` /
|
|
391
|
+
`DUPLICATE_ID` otherwise), a `severity` in `{blocker, major, minor, nit}` (`BAD_SEVERITY`
|
|
392
|
+
otherwise), and non-empty string `claim`, `location`, `rationale` (`MISSING_FIELD` otherwise).
|
|
393
|
+
|
|
394
|
+
**A clean review is a valid review.** A reviewer that read the material, found nothing wrong,
|
|
395
|
+
and said so passes validation with `"findings": []` — it is not sent to a repair re-prompt and
|
|
396
|
+
its seat is recorded `conformance: clean`, exactly like any other well-formed review. Three
|
|
397
|
+
things make that safe:
|
|
398
|
+
|
|
399
|
+
- A **broken** emit is a different outcome with its own code: no fenced block at all is
|
|
400
|
+
`NO_FENCED_BLOCK`, and a block that does not parse is `NOT_PARSEABLE`. Both return before the
|
|
401
|
+
empty-set rule is ever reached, so "my output broke" is never mistaken for "I found nothing".
|
|
402
|
+
- **`overall` is what carries the claim.** An empty findings array with a blank, missing, or
|
|
403
|
+
non-string `overall` is a hollow shell, not a judgement, and stays `EMPTY_FINDINGS`. The
|
|
404
|
+
Stage-1 briefing states the same rule to the model: `overall` is always required, `findings`
|
|
405
|
+
may be `[]`, and a finding is never to be invented to fill it.
|
|
406
|
+
- A **missing** `findings` key is not a declaration of zero and stays an error. Only an array
|
|
407
|
+
that is present and empty counts as "I found nothing" — the same line `countAttemptedFindings`
|
|
408
|
+
draws when it checks a repair against the count the original declared.
|
|
409
|
+
|
|
410
|
+
This closes a contradiction that used to be shipped in every run: the anti-sycophancy clause in
|
|
411
|
+
each Stage-1 briefing says "An empty severity category is a valid result", while the validator
|
|
412
|
+
rejected exactly that answer — so the only way for a reviewer to satisfy the schema was to
|
|
413
|
+
produce a finding. Downstream, an all-clean bench degrades cleanly rather than silently: Stage 2
|
|
414
|
+
still runs (the peer **ranking** — and therefore street-cred — is unaffected by an empty findings
|
|
415
|
+
pool, only the adjudication half is vacuous), the judge bundle and chair packet state the empty
|
|
416
|
+
findings index explicitly instead of rendering a heading over nothing, `tierCounts` comes out
|
|
417
|
+
all-zeros, per-model `confirmRate`/`factErrorRate` are `null` (no denominator to divide by), and
|
|
418
|
+
a `--debate` run records `debate.outcome: "nothing-to-debate"` on `run.json`.
|
|
419
|
+
|
|
420
|
+
**Exit codes are a tri-state contract, not the usual 0/1:**
|
|
421
|
+
|
|
422
|
+
| Exit | Meaning |
|
|
423
|
+
|---|---|
|
|
424
|
+
| `0` | `ok:true` — the findings block is well-formed. |
|
|
425
|
+
| `2` | `ok:false` — parsed as a *result*, but validation failed. A distinct, scriptable outcome — not a crash. |
|
|
426
|
+
| `1` | `BAD_ARGS` envelope — the file path is missing or unreadable. |
|
|
427
|
+
|
|
428
|
+
**Output** (`--json`): `{ok, findings, errors}`. `errors[]` is `{code, detail}[]` using the codes
|
|
429
|
+
above. Verified against the real binary:
|
|
430
|
+
|
|
431
|
+
```bash
|
|
432
|
+
$ amicus council validate review-deepseek.md --json
|
|
433
|
+
```
|
|
434
|
+
```json
|
|
435
|
+
{
|
|
436
|
+
"ok": true,
|
|
437
|
+
"findings": [
|
|
438
|
+
{ "id": 1, "severity": "major",
|
|
439
|
+
"claim": "The three-tier pricing table omits a monthly/annual toggle.",
|
|
440
|
+
"location": "pricing-page.html, .pricing-table section",
|
|
441
|
+
"rationale": "Users can't compare annual savings without it, and competitors all show one." }
|
|
442
|
+
],
|
|
443
|
+
"errors": []
|
|
444
|
+
}
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
A malformed file (no fenced block) returns exit `2`:
|
|
448
|
+
|
|
449
|
+
```json
|
|
450
|
+
{ "ok": false, "findings": [], "errors": [ { "code": "NO_FENCED_BLOCK", "detail": "no ```json block found" } ] }
|
|
451
|
+
```
|
|
452
|
+
|
|
453
|
+
---
|
|
454
|
+
|
|
455
|
+
## `amicus council tally`
|
|
456
|
+
|
|
457
|
+
```
|
|
458
|
+
amicus council tally <input.json> [--json] [--no-ledger]
|
|
459
|
+
```
|
|
460
|
+
|
|
461
|
+
Reads a **tally-input** JSON file, computes the deterministic tally **record**, prints it (human
|
|
462
|
+
summary, or the full record with `--json`), and — unless `--no-ledger` — appends one row per
|
|
463
|
+
model to the reliability ledger. Thin CLI wrapper over `tally()` (`src/council/tally.js`).
|
|
464
|
+
|
|
465
|
+
### Tally-input schema
|
|
466
|
+
|
|
467
|
+
Claude assembles this file at the end of Stage 2 (the full assembly recipe is in
|
|
468
|
+
[SKILL.md](../skills/second-opinion/SKILL.md#stage-2--cross-review), under "Stage-2 → tally
|
|
469
|
+
assembly recipe"). It needs **all five top-level keys** — `tally()` throws
|
|
470
|
+
`Cannot read properties of undefined (reading 'map')` if `meta` or `findings` is missing:
|
|
471
|
+
|
|
472
|
+
| Field | Type | Meaning |
|
|
473
|
+
|---|---|---|
|
|
474
|
+
| `meta.runId` | string | Run identifier (conventionally the run-folder stem). |
|
|
475
|
+
| `meta.models` | string[] | Every reviewed model id, including `"claude"` when "Claude in the council" is on — this is the street-cred universe. |
|
|
476
|
+
| `meta.chair` | string | The confirmed chair model id. |
|
|
477
|
+
| `meta.claudeInCouncil` | boolean | The Stage-0 toggle. |
|
|
478
|
+
| `meta.runType`, `meta.date` | string (optional) | Free-form labels carried through to `verdict.json`. |
|
|
479
|
+
| `findings[]` | array | One entry per finding across all reviews: `{id, raiser, severity}` (`claim` may ride along but isn't required by the tally engine). `id` is the run-global label (e.g. `A1`, `B2`) assigned during Stage-2 assembly, not the reviewer's local integer id. |
|
|
480
|
+
| `adjudications[]` | array | One entry per (judge × finding): `{findingId, judge, verdict}`, `verdict ∈ {agree, dispute, neutral}`. Include every judge's verdict on every finding, **including the raiser's own adjudication of its own finding** — the engine excludes it automatically when scoring (don't pre-filter it). |
|
|
481
|
+
| `rankings[]` | array | One entry per judge: `{judge, order}`. `order` is that judge's `FINAL RANKING:` block translated to model ids, e.g. `["gpt", "deepseek"]` (ties may use a nested array, e.g. `[["gpt","deepseek"], "mistral"]`). |
|
|
482
|
+
| `runStats[]` | array | One entry per model call: `{model, role, wasChair, conformance, status, durationMs, usage}`. May be `[]`. Any leg with no run document gets `durationMs: null, usage: null` — never invent a value. |
|
|
483
|
+
|
|
484
|
+
### Tally-record schema (what `tally()` returns / prints)
|
|
485
|
+
|
|
486
|
+
```json
|
|
487
|
+
{
|
|
488
|
+
"schemaVersion": 2,
|
|
489
|
+
"type": "council-tally",
|
|
490
|
+
"meta": { "...": "echoed from input" },
|
|
491
|
+
"judged": true,
|
|
492
|
+
"streetCred": [ { "model": "gpt", "withSelf": 1, "peersOnly": 1, "perJudgeRank": { "...": "..." } } ],
|
|
493
|
+
"findings": [
|
|
494
|
+
{ "id": "A1", "raiser": "deepseek", "severity": "major", "tier": "Confirmed",
|
|
495
|
+
"basis": { "a": 1, "d": 0, "n": 0 }, "confidence": "thin",
|
|
496
|
+
"tierOverride": null, "adjudications": [ { "judge": "deepseek", "verdict": "agree" } ] }
|
|
497
|
+
],
|
|
498
|
+
"runStats": [ { "...": "validated, echoed from input" } ],
|
|
499
|
+
"tierCounts": { "Confirmed": 1, "Contested": 1, "Singleton": 1, "Disputed": 0 }
|
|
500
|
+
}
|
|
501
|
+
```
|
|
502
|
+
|
|
503
|
+
| Field | Notes |
|
|
504
|
+
|---|---|
|
|
505
|
+
| `schemaVersion` | Tally-record schema version (currently `2`, council family v2 — see `type` below). This is a *separate* version line from the `--json` **error-envelope** schema version used by `BAD_ARGS` failures (also currently `2`) — the two happen to share a value right now but evolve independently; don't conflate them when scripting against output. |
|
|
506
|
+
| `type` | Document-type discriminator; always `"council-tally"` (council family v2 envelope). |
|
|
507
|
+
| `judged` | `true` only when `rankings.length >= 2`. `false` (1 or 0 rankings) means street-cred numbers exist but rest on thin cross-review. |
|
|
508
|
+
| `streetCred[].withSelf` | Mean rank position across **all** judges' rankings (lower = better). |
|
|
509
|
+
| `streetCred[].peersOnly` | Mean rank position **excluding** the model's own ranking of itself. This is the number used everywhere else (ledger, `stats`, bench recommendations). |
|
|
510
|
+
| `findings[].tier` | One of `Confirmed \| Contested \| Singleton \| Disputed` — see the cascade below. |
|
|
511
|
+
| `findings[].basis` | `{a, d, n}` = peer agree/dispute/neutral counts (raiser's own vote excluded when a raiser is known). |
|
|
512
|
+
| `findings[].confidence` | `"thin"` when `a + d <= 1` (only one peer engaged), else `"solid"`. Thin-confidence findings are the ones Claude may override before Stage 4. |
|
|
513
|
+
| `findings[].tierOverride` | `null` unless Claude recorded an override; shape `{from, to, reason}`. |
|
|
514
|
+
| `tierCounts` | Convenience totals across all findings — this is what `renderRecord`'s human-readable summary prints. |
|
|
515
|
+
|
|
516
|
+
**The peers-only tier cascade** (`assignTier(a, d)` — exhaustive over all `(a, d)`):
|
|
517
|
+
|
|
518
|
+
| Condition | Tier | Confidence |
|
|
519
|
+
|---|---|---|
|
|
520
|
+
| `d >= 2 && d > a` | Disputed | solid |
|
|
521
|
+
| `a >= 2 && a > d`, or `a === 1 && d === 0` | Confirmed | solid (≥2 agree) / thin (lone peer) |
|
|
522
|
+
| `d >= 1` (and not Disputed) | Contested | thin if `a+d<=1`, else solid |
|
|
523
|
+
| else (`a === 0 && d === 0`) | Singleton | thin |
|
|
524
|
+
|
|
525
|
+
A lone corroborating peer (`a=1, d=0`) ranks as **Confirmed (thin)** — it must not rank weaker than
|
|
526
|
+
a lone disputing peer (`a=0, d=1`, which is **Contested (thin)**). A 2-vs-2 split is **Contested**
|
|
527
|
+
(large-bench tie), not Disputed — `d > a` is required for Disputed, not just `d >= 2`.
|
|
528
|
+
|
|
529
|
+
**Ledger append.** Unless `--no-ledger`, `tally` writes one row per `meta.models` entry to
|
|
530
|
+
`council-ledger.jsonl` (append-only, JSON Lines). Use `--no-ledger` for a re-tally that shouldn't
|
|
531
|
+
double-count (e.g. re-running after fixing a malformed input). Two standing uses from the skill's
|
|
532
|
+
optional elements (v2.2.0): **debate mode** tallies provisionally with `--no-ledger` after Stage 2
|
|
533
|
+
and records only the final post-rebuttal tally, and **expert-lens runs** always pass `--no-ledger`
|
|
534
|
+
(lens reviews aren't comparable to standard reviews, so they must not feed `stats`). This is
|
|
535
|
+
best-effort: a ledger write failure prints a notice to stderr but does not fail the tally.
|
|
536
|
+
|
|
537
|
+
---
|
|
538
|
+
|
|
539
|
+
## `amicus council verdict`
|
|
540
|
+
|
|
541
|
+
```
|
|
542
|
+
amicus council verdict <tally.json> [--decisions <decisions.json>] [-o|--out <out.json>]
|
|
543
|
+
```
|
|
544
|
+
|
|
545
|
+
**Provenance — this is the answer to "where does verdict.json come from":** `verdict.json` is a
|
|
546
|
+
**pure merge** of two inputs, computed by `buildVerdict(record, decisions)`
|
|
547
|
+
(`src/council/verdict.js`) with **no recomputation of tiers or street-cred**:
|
|
548
|
+
|
|
549
|
+
1. `<tally.json>` — the tally **record** exactly as printed by `amicus council tally --json`
|
|
550
|
+
(Claude saves it to disk after Stage 2; see the worked example below).
|
|
551
|
+
2. `--decisions <decisions.json>` — a **JSON array**, one object per finding, produced by Claude
|
|
552
|
+
during Stage 4 (the accept/deny/modify pass): `{id, decision, applied?, duplicateOf?, tierOverride?}`.
|
|
553
|
+
Optional — defaults to `[]`, which produces a verdict with every finding's `decision: null`.
|
|
554
|
+
|
|
555
|
+
For each finding, `buildVerdict` looks up the matching decision by `id` and folds in `decision`,
|
|
556
|
+
`applied` (default `false`), `duplicateOf` (default `null`), and `tierOverride` (decision's
|
|
557
|
+
override wins over the tally record's, if both are present — the effective `tier` becomes
|
|
558
|
+
`tierOverride.to` when an override exists). Everything else (`basis`, `confidence`,
|
|
559
|
+
`adjudications`, `streetCred`, `runStats`, `tierCounts`) passes through from the tally record
|
|
560
|
+
unchanged.
|
|
561
|
+
|
|
562
|
+
**Output schema** (`verdict.json`, schema v2 — independent of the tally record's own
|
|
563
|
+
`schemaVersion`):
|
|
564
|
+
|
|
565
|
+
```json
|
|
566
|
+
{
|
|
567
|
+
"schemaVersion": 2,
|
|
568
|
+
"type": "council-verdict",
|
|
569
|
+
"overallVerdict": null,
|
|
570
|
+
"runId": "...", "runType": "...", "date": "...", "chair": "...",
|
|
571
|
+
"council": ["deepseek", "gpt"],
|
|
572
|
+
"claudeInCouncil": false,
|
|
573
|
+
"findings": [
|
|
574
|
+
{ "id": "A1", "raiser": "deepseek", "severity": "major", "tier": "Confirmed",
|
|
575
|
+
"basis": { "a": 1, "d": 0, "n": 0 }, "confidence": "thin", "tierOverride": null,
|
|
576
|
+
"duplicateOf": null, "adjudications": [ "..." ],
|
|
577
|
+
"decision": "accepted", "applied": true }
|
|
578
|
+
],
|
|
579
|
+
"streetCred": [ { "model": "gpt", "withSelf": 1, "peersOnly": 1 } ],
|
|
580
|
+
"runStats": [ "..." ],
|
|
581
|
+
"tierCounts": { "Confirmed": 1, "Contested": 1, "Singleton": 1, "Disputed": 0 }
|
|
582
|
+
}
|
|
583
|
+
```
|
|
584
|
+
|
|
585
|
+
**Key notes:**
|
|
586
|
+
- `schemaVersion` — verdict-document schema version (currently `2`).
|
|
587
|
+
- `type` — document-type discriminator; always `"council-verdict"` (council family v2 envelope).
|
|
588
|
+
- `overallVerdict` — the chair's verdict-scale outcome: one of `"Ship it"`, `"Fix these first"`, `"Fundamental rethink"`, or `null` when no chair verdict was produced (populated by the headless engine during Stage 3; `null` for a plain `council verdict` merge without engine integration).
|
|
589
|
+
- All other keys (`runId`, `council`, `findings`, `streetCred`, `runStats`, `tierCounts`) are passed through unchanged from the tally record.
|
|
590
|
+
|
|
591
|
+
**Write path:** always atomic — a `<out>.tmp-<pid>` file is written first, then renamed over the
|
|
592
|
+
target (`writeVerdictAtomic`), matching the repo's `wave.json` convention. Default output path is
|
|
593
|
+
`./verdict.json`; override with `-o`/`--out`.
|
|
594
|
+
|
|
595
|
+
**Re-rendering after Stage 4 (`--render`).** `council verdict`'s `--render` flag refreshes
|
|
596
|
+
`report.html` next to the freshly-decided verdict in the same call — without it, `report.html`
|
|
597
|
+
stays the engine's undecided pre-Stage-4 render, and a user opening it would see decisions that
|
|
598
|
+
were never actually made. It calls the same renderer `amicus council report` uses
|
|
599
|
+
(`buildReport(..., {format: 'html'})`) and writes the HTML into the output path's directory; a
|
|
600
|
+
render failure after a successful verdict write reports the error but leaves `verdict.json` on
|
|
601
|
+
disk (re-run `amicus council report <verdict.json> --html` manually to recover — the verdict
|
|
602
|
+
itself is not lost). The MCP equivalent is `amicus_verdict`'s `render: true` + `outDir:
|
|
603
|
+
<run-folder>`. Because that tool is pure/stateless and writes nothing unless both are given, this
|
|
604
|
+
is a **second** call after the first: call once with `record`/`decisions` and no `render` to get
|
|
605
|
+
the decided verdict back as JSON and write it to `<run-folder>/verdict.json` yourself, then call
|
|
606
|
+
again with `render: true` and `outDir: <run-folder>` — this refreshes `<outDir>/report.html` on
|
|
607
|
+
disk and also returns the verdict's Markdown rendering (for `report.md`); it still does not write
|
|
608
|
+
`verdict.json` itself.
|
|
609
|
+
|
|
610
|
+
**Windows PowerShell 5.1 caveat** (also called out in SKILL.md): redirecting `council tally`'s
|
|
611
|
+
`--json` output with a bare `>` writes UTF-16 under legacy PowerShell 5.1, which then makes
|
|
612
|
+
`council verdict` fail to parse `tally.json` with a confusing `BAD_ARGS` — pipe through
|
|
613
|
+
`| Out-File -Encoding utf8` on 5.1, or run under pwsh 7+/bash.
|
|
614
|
+
|
|
615
|
+
---
|
|
616
|
+
|
|
617
|
+
## `amicus council report`
|
|
618
|
+
|
|
619
|
+
```
|
|
620
|
+
amicus council report <verdict.json> [--wave <wave.json>] [--md|--html]
|
|
621
|
+
```
|
|
622
|
+
|
|
623
|
+
Pure renderer — reads `verdict.json` (+ optional `wave.json`, used only to source the wave's total
|
|
624
|
+
cost) and produces **one self-contained string**: Markdown (default) or a self-contained HTML page
|
|
625
|
+
(`--html`). No scoring, no anonymization, no synthesis — those already happened upstream. Thin CLI
|
|
626
|
+
wrapper over `buildReport()` (`src/council/report.js` / `report-html.js`).
|
|
627
|
+
|
|
628
|
+
**What it renders**, in this order: a header (run type, id, date, chair, council members), a
|
|
629
|
+
verdict-summary tier-count table, the **adjudication matrix** (finding × judge, `✓`/`✗`/`–` with
|
|
630
|
+
`*` marking the raiser's own vote), the **peers-only street-cred table**, **findings grouped by
|
|
631
|
+
tier** (Disputed first), and a **cost table** (per-model status/duration/cost + wave total,
|
|
632
|
+
sourced from `runStats[].usage`).
|
|
633
|
+
|
|
634
|
+
This is the same renderer the `second-opinion` skill calls in Stage 5 to produce `report.html`.
|
|
635
|
+
**`report.md` and this renderer's output are two different files** — `report.md` is Claude-authored
|
|
636
|
+
prose that includes a copy of this renderer's Markdown as one section, not this renderer's own
|
|
637
|
+
output; see [Where artifacts live](#where-artifacts-live) for the exact contract.
|
|
638
|
+
|
|
639
|
+
**`--html` output** is a self-contained page (one file, inlined styles) — the default artifact to
|
|
640
|
+
hand a user, per the skill.
|
|
641
|
+
|
|
642
|
+
---
|
|
643
|
+
|
|
644
|
+
## `amicus council stats`
|
|
645
|
+
|
|
646
|
+
```
|
|
647
|
+
amicus council stats [--json]
|
|
648
|
+
```
|
|
649
|
+
|
|
650
|
+
Reads the **append-only ledger** (`council-ledger.jsonl`, written by every non-`--no-ledger`
|
|
651
|
+
`council tally` call) and aggregates per-model reliability across **all past council runs on this
|
|
652
|
+
machine** — this is historical, cross-run data, not anything from a single tally/verdict. Thin CLI
|
|
653
|
+
wrapper over `deriveReliability()` (`src/council/ledger.js`).
|
|
654
|
+
|
|
655
|
+
Since v4.0 (council schema v2), `--json` wraps the rows in the family envelope —
|
|
656
|
+
`{ "schemaVersion": 2, "type": "council-stats", "models": [ … ] }` — the per-model row
|
|
657
|
+
shape below is unchanged. (Pre-4.0 emitted the bare array.)
|
|
658
|
+
|
|
659
|
+
**Output**, one row per model that has ever appeared in `meta.models`:
|
|
660
|
+
|
|
661
|
+
| Field | Meaning |
|
|
662
|
+
|---|---|
|
|
663
|
+
| `runs` | Number of ledger rows for this model (one per council run it participated in). |
|
|
664
|
+
| `lowN` | `true` when `runs < 3` — treat the numbers as noisy. |
|
|
665
|
+
| `avgStreetCredPeersOnly` | Mean of `streetCredPeersOnly` across all runs (`null` if the model was never judged). |
|
|
666
|
+
| `lifetimeConfirmRate` | Mean, across runs, of `(findings this model raised that landed Confirmed) / (findings this model raised)`. `null` when `judged` was false for every run or the model raised nothing. |
|
|
667
|
+
| `lifetimeFactErrorRate` | Same shape, but for the `Disputed` tier — a proxy for how often the bench caught this model asserting something wrong. |
|
|
668
|
+
| `conformance` | Tally of `{clean, repaired, unstructured}` counts — how often this model's Stage-1 findings JSON needed a repair re-prompt. |
|
|
669
|
+
|
|
670
|
+
This is the data source the `second-opinion` skill's Stage 0 model recommendations and the
|
|
671
|
+
`MODEL-NOTES.md` quantitative table both read — **never hand-edit reliability numbers there**;
|
|
672
|
+
they come from here.
|
|
673
|
+
|
|
674
|
+
---
|
|
675
|
+
|
|
676
|
+
## Council presets: `save` / `list` / `show`
|
|
677
|
+
|
|
678
|
+
```
|
|
679
|
+
amicus council save <name> --models a,b,c
|
|
680
|
+
amicus council list [--json]
|
|
681
|
+
amicus council show <name> [--json]
|
|
682
|
+
```
|
|
683
|
+
|
|
684
|
+
A named preset is just a saved `--models`-style list that `--council <name>` (on `fanout` and the
|
|
685
|
+
`amicus_fanout` MCP tool) can run in one shot, instead of spelling out `--models` every time.
|
|
686
|
+
|
|
687
|
+
- **`save`** validates ≥2 members (each must resolve via the same alias/catalog logic
|
|
688
|
+
`resolveCouncilMembers` uses — a known alias, or a raw `provider/model` id containing `/`) and
|
|
689
|
+
writes them to `~/.config/amicus/config.json` under `councils.<name>`. Overwrites silently
|
|
690
|
+
report `overwritten: true` in `--json` mode — this is also how you shadow a built-in bench.
|
|
691
|
+
- **`list`** shows your saved councils **plus** the three built-in benches (`free`, `budget`,
|
|
692
|
+
`frontier`), each marked `builtin: true`. If a saved council shares a name with a built-in,
|
|
693
|
+
**both** entries are listed — the saved one (`builtin: false`) is the one `--council <name>`
|
|
694
|
+
actually resolves to; the built-in entry gets `shadowed: true`.
|
|
695
|
+
- **`show <name>`** resolves a name exactly like `--council` does (user config first, built-in
|
|
696
|
+
fallback) and reports the raw members plus a `resolved`/`dropped` split — diagnostic-only, so it
|
|
697
|
+
still reports even for a council currently below the 2-member usable minimum (unlike an actual
|
|
698
|
+
run, which refuses below 2).
|
|
699
|
+
|
|
700
|
+
**Built-in benches** (work with zero setup): `free` (zero-cost `:free`-suffixed OpenRouter models,
|
|
701
|
+
resolved dynamically from the live catalog), `budget` (cheap workhorses, one per vendor family),
|
|
702
|
+
`frontier` (premium flagships, one per vendor family).
|
|
703
|
+
|
|
704
|
+
---
|
|
705
|
+
|
|
706
|
+
## Worked example
|
|
707
|
+
|
|
708
|
+
Everything below was run against the real `amicus` binary while writing this page — no mocked
|
|
709
|
+
output. Two reviewers (`deepseek`, `gpt`) reviewed a pricing page; deepseek raised one finding,
|
|
710
|
+
gpt raised two.
|
|
711
|
+
|
|
712
|
+
**1. `tally-input.json`** — assembled by Claude at the end of Stage 2:
|
|
713
|
+
|
|
714
|
+
```json
|
|
715
|
+
{
|
|
716
|
+
"meta": {
|
|
717
|
+
"runId": "pricing-page-council",
|
|
718
|
+
"runType": "design-review",
|
|
719
|
+
"date": "2026-07-02T18:00:00Z",
|
|
720
|
+
"models": ["deepseek", "gpt"],
|
|
721
|
+
"chair": "deepseek",
|
|
722
|
+
"claudeInCouncil": false
|
|
723
|
+
},
|
|
724
|
+
"findings": [
|
|
725
|
+
{ "id": "A1", "raiser": "deepseek", "severity": "major",
|
|
726
|
+
"claim": "The three-tier pricing table omits a monthly/annual toggle." },
|
|
727
|
+
{ "id": "B1", "raiser": "gpt", "severity": "minor",
|
|
728
|
+
"claim": "The 'Enterprise' tier has no visible CTA button." },
|
|
729
|
+
{ "id": "B2", "raiser": "gpt", "severity": "blocker",
|
|
730
|
+
"claim": "Listed prices contradict the numbers in the FAQ section." }
|
|
731
|
+
],
|
|
732
|
+
"adjudications": [
|
|
733
|
+
{ "findingId": "A1", "judge": "deepseek", "verdict": "agree" },
|
|
734
|
+
{ "findingId": "A1", "judge": "gpt", "verdict": "agree" },
|
|
735
|
+
{ "findingId": "B1", "judge": "deepseek", "verdict": "neutral" },
|
|
736
|
+
{ "findingId": "B1", "judge": "gpt", "verdict": "agree" },
|
|
737
|
+
{ "findingId": "B2", "judge": "deepseek", "verdict": "dispute" },
|
|
738
|
+
{ "findingId": "B2", "judge": "gpt", "verdict": "agree" }
|
|
739
|
+
],
|
|
740
|
+
"rankings": [
|
|
741
|
+
{ "judge": "deepseek", "order": ["gpt", "deepseek"] },
|
|
742
|
+
{ "judge": "gpt", "order": ["gpt", "deepseek"] }
|
|
743
|
+
],
|
|
744
|
+
"runStats": [
|
|
745
|
+
{ "model": "deepseek", "role": "council", "wasChair": true, "conformance": "clean",
|
|
746
|
+
"status": "complete", "durationMs": 41230,
|
|
747
|
+
"usage": { "cost": { "amount": 0.038, "source": "reported" } } },
|
|
748
|
+
{ "model": "gpt", "role": "council", "wasChair": false, "conformance": "clean",
|
|
749
|
+
"status": "complete", "durationMs": 37810,
|
|
750
|
+
"usage": { "cost": { "amount": 0.052, "source": "reported" } } }
|
|
751
|
+
]
|
|
752
|
+
}
|
|
753
|
+
```
|
|
754
|
+
|
|
755
|
+
**2. Tally** — note deepseek's `B2` dispute makes it Contested even though gpt agrees; `A1` gets
|
|
756
|
+
one uncontested peer agreement (Confirmed, thin); `B1` draws no agree/dispute at all (Singleton):
|
|
757
|
+
|
|
758
|
+
```bash
|
|
759
|
+
$ amicus council tally tally-input.json --json > tally.json
|
|
760
|
+
$ amicus council tally tally-input.json
|
|
761
|
+
Council tally (pricing-page-council)
|
|
762
|
+
Confirmed 1 Contested 1 Singleton 1 Disputed 0
|
|
763
|
+
Cost: $0.0900
|
|
764
|
+
```
|
|
765
|
+
|
|
766
|
+
**3. Stage 4 decisions** — Claude presents the three tiers, the user decides:
|
|
767
|
+
|
|
768
|
+
```json
|
|
769
|
+
[
|
|
770
|
+
{ "id": "A1", "decision": "accepted", "applied": true },
|
|
771
|
+
{ "id": "B1", "decision": "deferred" },
|
|
772
|
+
{ "id": "B2", "decision": "accepted", "applied": true }
|
|
773
|
+
]
|
|
774
|
+
```
|
|
775
|
+
|
|
776
|
+
**4. Verdict** — merges the tally record with the decisions above:
|
|
777
|
+
|
|
778
|
+
```bash
|
|
779
|
+
$ amicus council verdict tally.json --decisions decisions.json -o verdict.json
|
|
780
|
+
Verdict (schema v2, pricing-page-council) → verdict.json
|
|
781
|
+
accepted 2 deferred 1
|
|
782
|
+
```
|
|
783
|
+
|
|
784
|
+
**5. Report** — deterministic rendering of `verdict.json` (`--md` shown; `--html` produces the
|
|
785
|
+
same content as a self-contained page):
|
|
786
|
+
|
|
787
|
+
```bash
|
|
788
|
+
$ amicus council report verdict.json --md
|
|
789
|
+
```
|
|
790
|
+
```
|
|
791
|
+
# Council Report — design-review (pricing-page-council)
|
|
792
|
+
|
|
793
|
+
_2026-07-02T18:00:00Z · chair: deepseek · council: deepseek, gpt_
|
|
794
|
+
|
|
795
|
+
## Verdict summary
|
|
796
|
+
|
|
797
|
+
| Tier | Count |
|
|
798
|
+
|---|---|
|
|
799
|
+
| Disputed | 0 |
|
|
800
|
+
| Contested | 1 |
|
|
801
|
+
| Confirmed | 1 |
|
|
802
|
+
| Singleton | 1 |
|
|
803
|
+
|
|
804
|
+
## Adjudication matrix
|
|
805
|
+
|
|
806
|
+
| Finding | Sev | Raiser | deepseek | gpt | Tier | Decision |
|
|
807
|
+
|---|---|---|---|---|---|---|
|
|
808
|
+
| A1 | major | deepseek | ✓* | ✓ | Confirmed | accepted |
|
|
809
|
+
| B1 | minor | gpt | – | ✓* | Singleton | deferred |
|
|
810
|
+
| B2 | blocker | gpt | ✗ | ✓* | Contested | accepted |
|
|
811
|
+
|
|
812
|
+
_Legend: ✓ agree · ✗ dispute · – neutral · `*` raiser's own vote_
|
|
813
|
+
|
|
814
|
+
## Street-cred (peers-only; lower = better)
|
|
815
|
+
|
|
816
|
+
| Model | peers-only | with-self |
|
|
817
|
+
|---|---|---|
|
|
818
|
+
| deepseek | 2.00 | 2.00 |
|
|
819
|
+
| gpt | 1.00 | 1.00 |
|
|
820
|
+
|
|
821
|
+
## Findings by tier
|
|
822
|
+
|
|
823
|
+
### Contested
|
|
824
|
+
- **B2** (blocker, raiser gpt) — a0/d1/n0 — accepted (applied)
|
|
825
|
+
|
|
826
|
+
### Confirmed
|
|
827
|
+
- **A1** (major, raiser deepseek) — a1/d0/n0 — accepted (applied)
|
|
828
|
+
|
|
829
|
+
### Singleton
|
|
830
|
+
- **B1** (minor, raiser gpt) — a0/d0/n1 — deferred
|
|
831
|
+
|
|
832
|
+
## Cost
|
|
833
|
+
|
|
834
|
+
| Model | Status | Duration | Cost |
|
|
835
|
+
|---|---|---|---|
|
|
836
|
+
| deepseek | complete | 41s | $0.0380 |
|
|
837
|
+
| gpt | complete | 38s | $0.0520 |
|
|
838
|
+
| **Wave total** | | | $0.0900 |
|
|
839
|
+
```
|
|
840
|
+
|
|
841
|
+
**6. Stats** — after this run's `tally` call appended to the ledger:
|
|
842
|
+
|
|
843
|
+
```bash
|
|
844
|
+
$ amicus council stats
|
|
845
|
+
model runs avg-cred confirm fact-err notes
|
|
846
|
+
deepseek 1 2.00 1.00 0.00 low-N
|
|
847
|
+
gpt 1 1.00 0.00 0.00 low-N
|
|
848
|
+
```
|
|
849
|
+
|
|
850
|
+
(`low-N` because each model has only 1 recorded run — `runs < 3`.)
|
|
851
|
+
|
|
852
|
+
**7. Presets**, for reference (independent of the run above):
|
|
853
|
+
|
|
854
|
+
```bash
|
|
855
|
+
$ amicus council save my-bench --models deepseek,gpt,gemini
|
|
856
|
+
Saved council 'my-bench': deepseek, gpt, gemini
|
|
857
|
+
|
|
858
|
+
$ amicus council list
|
|
859
|
+
Councils:
|
|
860
|
+
my-bench deepseek, gpt, gemini
|
|
861
|
+
free [built-in]
|
|
862
|
+
budget [built-in]
|
|
863
|
+
frontier [built-in]
|
|
864
|
+
|
|
865
|
+
$ amicus council show my-bench
|
|
866
|
+
Council 'my-bench'
|
|
867
|
+
members: deepseek, gpt, gemini
|
|
868
|
+
resolved: deepseek, gpt, gemini
|
|
869
|
+
```
|
|
870
|
+
|
|
871
|
+
---
|
|
872
|
+
|
|
873
|
+
## Where artifacts live
|
|
874
|
+
|
|
875
|
+
Every run writes to a run folder — `output/<stem>-council/` (or `./second-opinion/<stem>-council/`
|
|
876
|
+
if no `output/` directory exists), per the skill's Stage 0. This section cross-checks against
|
|
877
|
+
[SKILL.md's "Output & naming"](../skills/second-opinion/SKILL.md#output--naming) — treat that
|
|
878
|
+
section as authoritative if the two ever drift; file an issue if they do.
|
|
879
|
+
|
|
880
|
+
| File | Written by | Contains |
|
|
881
|
+
|---|---|---|
|
|
882
|
+
| `review-<model>.md` × N | Claude, from each Stage-1 leg's output | Prose review + trailing findings JSON block (this is what `council validate` checks). |
|
|
883
|
+
| `crossreview-matrix.md` | Claude, after Stage 2 | De-anonymized adjudication grid + street-cred table (hand-assembled from the tally record, not this renderer). |
|
|
884
|
+
| `tally.json` | `amicus council tally --json`, redirected to disk | The tally record — input to `council verdict`. |
|
|
885
|
+
| `verdict.md` | Claude, saved from the Stage-3 chair call | The chair's raw synthesized verdict (prose, unedited by Claude). |
|
|
886
|
+
| `verdict.json` | `amicus council verdict` | Schema-stamped merge of the tally record + Stage-4 decisions. See [provenance](#amicus-council-verdict). |
|
|
887
|
+
| `decisions.json` | Claude, during Stage 4 | The array passed to `council verdict --decisions`. |
|
|
888
|
+
| `report.md` | **Claude**, in Stage 5 | The chair's synthesis + the full Stage-4 decision log + a run-stats table (stage/model/status/duration/cost per call). **This is Claude-authored prose, not this page's renderer output.** |
|
|
889
|
+
| `report.html` | `amicus council report verdict.json --html` | The **deterministic** rendering shown in the [worked example](#worked-example) above — adjudication matrix, street-cred table, findings-by-tier, cost. No chair prose, no decision-log narrative. This is the default artifact handed to the user. |
|
|
890
|
+
| `<stem>-reviewed.<ext>` | Claude, in Stage 5 | The source artifact with accepted findings applied (editable-source runs only). |
|
|
891
|
+
|
|
892
|
+
**The one thing worth over-stating:** `report.md` and `report.html` are **not** the same content in
|
|
893
|
+
two formats. `report.html` is `amicus council report`'s pure render of `verdict.json` — deterministic,
|
|
894
|
+
no model involved. `report.md` is Claude's own synthesis document, written by hand in Stage 5,
|
|
895
|
+
which *includes* a copy of the same renderer's Markdown output as one section but also carries the
|
|
896
|
+
chair's prose verdict and the full decision log that the renderer never sees. If you only need the
|
|
897
|
+
deterministic data, run `amicus council report` yourself against any `verdict.json` — you don't
|
|
898
|
+
need Claude or a live council run to regenerate it.
|
|
899
|
+
|
|
900
|
+
---
|
|
901
|
+
|
|
902
|
+
## See also
|
|
903
|
+
|
|
904
|
+
- **[skills/second-opinion/SKILL.md](../skills/second-opinion/SKILL.md)** — the orchestration
|
|
905
|
+
recipe that drives an actual council run (model selection, briefings, anonymization, the Stage-2
|
|
906
|
+
tally assembly recipe, Stage 4 decision presentation).
|
|
907
|
+
- **[skills/second-opinion/COUNCIL-DESIGN.md](../skills/second-opinion/COUNCIL-DESIGN.md)** — the
|
|
908
|
+
design spec behind the tier cascade and scoring model.
|
|
909
|
+
- **[docs/usage.md](./usage.md)** — full CLI/MCP flag reference for every command, including
|
|
910
|
+
`council`.
|
|
911
|
+
- **[README "The Council"](../README.md#the-council)** — the narrative overview and cost framing.
|