amicus 1.8.0 → 1.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +31 -0
- package/README.md +9 -4
- package/package.json +1 -1
- package/skills/second-opinion/MODEL-NOTES.md +47 -1
- package/skills/second-opinion/SKILL.md +36 -8
- package/skills/sidecar/SKILL.md +36 -38
- package/src/headless.js +2 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "amicus",
|
|
3
|
-
"version": "1.8.
|
|
3
|
+
"version": "1.8.1",
|
|
4
4
|
"description": "Multi-model LLM Council + parallel AI window for Claude Code. Run structured council reviews across Gemini, GPT, DeepSeek and more — or fork a conversation to any model and fold the results back.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Christian Wagner"
|
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,37 @@ All notable changes to Amicus are documented here. Format follows
|
|
|
5
5
|
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
## [1.8.1] - 2026-07-02
|
|
9
|
+
|
|
10
|
+
Docs & skills accuracy sprint from the Phase-8 whole-branch review — no engine changes. Every item fixed a claim
|
|
11
|
+
that actively misdirected Claude or users, plus one headless completion-state bugfix.
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
- **`report.html` is now the default final council artifact**, and an inline verdict summary in chat is
|
|
15
|
+
MANDATORY at Stage 5 of the second-opinion skill.
|
|
16
|
+
- **The `sidecar` skill's frontmatter dropped the "second opinion from another model" trigger** — those requests
|
|
17
|
+
now route to the `second-opinion` skill instead.
|
|
18
|
+
- **MODEL-NOTES seed updated** with durable lessons from council runs 4-7 (new Grok/Kimi/Mistral/Claude-in-council
|
|
19
|
+
sections; shipped/local split defined). Existing installs: the machine-local copy is installed only-if-missing —
|
|
20
|
+
merge/refresh manually by pointing at the shipped file.
|
|
21
|
+
- **Council mechanics hardened:** mandatory no-tools preamble for judges and chair (plus scratch-cwd advice);
|
|
22
|
+
`--max-cost` / `--no-cost-gate` pass-through documented for repair and chair calls (the false solo-start
|
|
23
|
+
cost-gate exemption was removed); `--models` lists quoted in every example; current-date injection rule for
|
|
24
|
+
time-sensitive artifacts.
|
|
25
|
+
|
|
26
|
+
### Fixed
|
|
27
|
+
- **Plugin quick-start now states the truth:** plugin installs do not put `amicus` on `PATH`; use
|
|
28
|
+
`npx -y amicus@latest <cmd>`. Both skills gained an npx-fallback/transport rule.
|
|
29
|
+
- **README/usage now document `doctor`, `key`, and `council`;** troubleshooting leads with `amicus doctor`; the
|
|
30
|
+
false "`amicus list` shows active servers" claim is replaced with real `netstat`/`lsof` guidance.
|
|
31
|
+
- **Headless runs that finish via idle detection no longer write `status:"error"` / `reason:"Incomplete"` to
|
|
32
|
+
`metadata.json`.** The poll loop's two genuine idle-completion exits — the SDK-authoritative `session.status`
|
|
33
|
+
idle signal and the stable-poll activity heuristic (both gated on real output, F1 #16) — broke out of the loop
|
|
34
|
+
without setting `completed`, so `resolveTerminalState` fell through to error and poisoned `amicus_list` /
|
|
35
|
+
`amicus_status` / wave rollups for successful runs, while the stdout `--json` doc correctly said
|
|
36
|
+
`status:"complete"`. Both exits now mark the run completed, matching the fold-marker branch. Dead-server
|
|
37
|
+
classification is unchanged: the consecutive-poll-failure fast-exit (F4) and crash paths still report an error.
|
|
38
|
+
|
|
8
39
|
## [1.8.0] - 2026-07-02
|
|
9
40
|
|
|
10
41
|
### Added
|
package/README.md
CHANGED
|
@@ -57,7 +57,7 @@ Claude is the orchestrator. The council and chat skills run *on top of* the engi
|
|
|
57
57
|
|
|
58
58
|
## Quick start
|
|
59
59
|
|
|
60
|
-
**Install** — pick whichever fits
|
|
60
|
+
**Install** — pick whichever fits. Every path delivers the MCP server and both skills; the `amicus`/`am` CLI lands on your PATH with the **npm and install-script paths** (the plugin path runs the CLI on demand via `npx -y amicus@latest <command>`):
|
|
61
61
|
|
|
62
62
|
**As a Claude Code plugin** — the most native path if you use Claude Code:
|
|
63
63
|
|
|
@@ -67,7 +67,7 @@ Claude is the orchestrator. The council and chat skills run *on top of* the engi
|
|
|
67
67
|
/reload-plugins
|
|
68
68
|
```
|
|
69
69
|
|
|
70
|
-
Claude Code registers the MCP server and both skills for you — nothing to configure. (The standalone Electron window is npm-only
|
|
70
|
+
Claude Code registers the MCP server and both skills for you — nothing to configure. (The plugin does not put `amicus` on your PATH — CLI calls go through `npx -y amicus@latest <command>`; the standalone Electron window is npm-only; and the first council/sidecar call downloads the OpenCode engine.)
|
|
71
71
|
|
|
72
72
|
**With the install script** — macOS, Linux, or Windows (needs [Node.js](https://nodejs.org) ≥ 18):
|
|
73
73
|
|
|
@@ -96,6 +96,8 @@ For the **npm** and **install-script** paths, a postinstall auto-configures ever
|
|
|
96
96
|
|
|
97
97
|
```bash
|
|
98
98
|
amicus setup
|
|
99
|
+
# plugin-only install (no CLI on PATH):
|
|
100
|
+
npx -y amicus@latest setup
|
|
99
101
|
```
|
|
100
102
|
|
|
101
103
|
This opens a graphical wizard:
|
|
@@ -113,7 +115,7 @@ This opens a graphical wizard:
|
|
|
113
115
|
|
|
114
116
|
> *council review this*
|
|
115
117
|
|
|
116
|
-
Claude prepares the material, recommends a bench of models, discloses the run shape and cost, and orchestrates the rest. You make the accept/deny calls at the end. (The `second-opinion` skill
|
|
118
|
+
Claude prepares the material, recommends a bench of models, discloses the run shape and cost, and orchestrates the rest. You make the accept/deny calls at the end. (The `second-opinion` skill is what teaches Claude to recognize this — if nothing happens, run `amicus doctor` (or `npx -y amicus@latest doctor`). npm/install-script installs place the skill at `~/.claude/skills/second-opinion/`; plugin installs keep it inside the plugin itself — check `/plugin` in Claude Code to confirm amicus is enabled.)
|
|
117
119
|
|
|
118
120
|
**Your first sidecar.** The sidecar is the lower-level path — you can invoke it by phrase through Claude too, but the CLI gives you the flags directly:
|
|
119
121
|
|
|
@@ -261,6 +263,9 @@ amicus update
|
|
|
261
263
|
| `amicus read` | Output a session's summary / conversation / metadata. |
|
|
262
264
|
| `amicus status <id>` | One-shot status for a session or fan-out wave (human or `--json`; `--wave <id>` alternative spelling). |
|
|
263
265
|
| `amicus models` | List, search, refresh the catalog, or audit aliases. |
|
|
266
|
+
| `amicus doctor` | Diagnose your setup — keys, default model, catalog, aliases, OpenCode binary, Electron, skills, MCP registration, OpenRouter credit (`--json`; `--fix` self-heals what it can). |
|
|
267
|
+
| `amicus key` | Manage API keys non-interactively: `amicus key <provider> <key>` saves after live validation; `--remove`; bare `amicus key` lists providers. |
|
|
268
|
+
| `amicus council` | Council math: `tally <input.json>` (deterministic tiers + ledger append), `stats` (reviewer reliability), `report <verdict.json> [--md\|--html]`. |
|
|
264
269
|
| `amicus abort` | Abort a running session (or `--all`). |
|
|
265
270
|
| `amicus setup` | Configure default model, API keys, and aliases. |
|
|
266
271
|
| `amicus update` | Update to the latest version. |
|
|
@@ -304,7 +309,7 @@ The `am` alias is interchangeable with `amicus` everywhere.
|
|
|
304
309
|
### `amicus fanout` — same prompt, many models
|
|
305
310
|
|
|
306
311
|
```bash
|
|
307
|
-
amicus fanout --models gemini,deepseek,gpt --prompt "Review this design" --json
|
|
312
|
+
amicus fanout --models "gemini,deepseek,gpt" --prompt "Review this design" --json
|
|
308
313
|
```
|
|
309
314
|
|
|
310
315
|
Fanout runs one **headless wave**: every leg gets the **same** prompt (this is the shared-prompt model the council's review stages are built on). When all legs are terminal it prints **one** JSON wave document on stdout.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "amicus",
|
|
3
|
-
"version": "1.8.
|
|
3
|
+
"version": "1.8.1",
|
|
4
4
|
"description": "Multi-model LLM Council + parallel AI window for Claude Code. Run structured council reviews across Gemini, GPT, DeepSeek and more — or fork a conversation to any model and fold the results back.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"claude",
|
|
@@ -4,7 +4,8 @@ This file is the `second-opinion` skill's evolving memory of **how to actually d
|
|
|
4
4
|
well**. Read it before Stage 0 (council selection and launch); update it, with the user's
|
|
5
5
|
approval, at the end of each run (Stage 6). Keep it tight — merge and prune rather than append.
|
|
6
6
|
|
|
7
|
-
_Last updated: 2026-
|
|
7
|
+
_Last updated: 2026-07-02 (runs 4-7 folded back from the field ledger: PowerShell quoting, date
|
|
8
|
+
injection, long-read failures, judge tool-wander; see changelog)._
|
|
8
9
|
|
|
9
10
|
## Global operating rules (all models)
|
|
10
11
|
- **Council runs are headless by design** (autonomous batch work): `fanout` is headless by
|
|
@@ -26,6 +27,18 @@ _Last updated: 2026-06-10 (v3 migration: engine workarounds pruned — see chang
|
|
|
26
27
|
model-specific signals. Never present a half-finished run as an answer.
|
|
27
28
|
- **Credentials:** keys live in `~/.config/amicus/.env` (legacy `~/.config/sidecar/.env` still
|
|
28
29
|
read). Configure with `amicus setup`.
|
|
30
|
+
- **PowerShell `--models` quoting (Windows):** always quote comma-separated model lists —
|
|
31
|
+
`--models "gemini,gpt,deepseek"`. Unquoted, PowerShell splits on commas and amicus receives one
|
|
32
|
+
mangled alias → instant arg-parse failure. (Now baked into every SKILL.md example.)
|
|
33
|
+
- **Inject the current date into briefings for time-sensitive artifacts** (resumes, dated plans).
|
|
34
|
+
Headless models don't reliably know "today": one run produced a false "future-dated" blocker two
|
|
35
|
+
judges then confirmed. (Now a Stage-0 rule in SKILL.md.)
|
|
36
|
+
- **Very long artifacts (80k+ words) break the agent-reads-the-file transport for some models.**
|
|
37
|
+
gpt/deepseek/grok have handled 82k-word agentic reads; gemini(-flash) and kimi stalled (narrate-
|
|
38
|
+
stall / 25-min timeout / poller "Incomplete"). Pre-select proven long-read models or inline the
|
|
39
|
+
text for large-context models.
|
|
40
|
+
- **Stage-6 approvals:** write the proposed MODEL-NOTES diff to a run-folder file and put that path
|
|
41
|
+
in the approval prompt — chat-text diffs can be hidden behind the approval dialog.
|
|
29
42
|
|
|
30
43
|
## Stage-2 cross-review briefing tips
|
|
31
44
|
|
|
@@ -36,6 +49,11 @@ _Last updated: 2026-06-10 (v3 migration: engine workarounds pruned — see chang
|
|
|
36
49
|
finding referenced by run-global label id (e.g. `A2` = Review A's 2nd finding).
|
|
37
50
|
- After de-anonymizing, assemble the tally input (see SKILL.md Stage 2 assembly recipe) and run
|
|
38
51
|
`amicus council tally <input.json> --json` — do not hand-tally tiers or street-cred numbers.
|
|
52
|
+
- Telling judges that **material severity inflation can justify a `dispute`** sharpens adjudications.
|
|
53
|
+
- **Plan-agent judges can wander to tools** (reading run-folder files = anonymization leak). The
|
|
54
|
+
no-tools preamble is now mandatory in SKILL.md Stage 2/3 — keep it verbatim.
|
|
55
|
+
- The tally input needs **all five keys** (`meta` incl. `meta.models`, `findings`, `adjudications`,
|
|
56
|
+
`rankings`, `runStats`) — see the SKILL.md Stage-2 recipe step 0.
|
|
39
57
|
|
|
40
58
|
## Per-model notes
|
|
41
59
|
|
|
@@ -45,17 +63,44 @@ _Last updated: 2026-06-10 (v3 migration: engine workarounds pruned — see chang
|
|
|
45
63
|
structured output verbatim, without preamble. (Historical: its narrate-then-glob habit used to
|
|
46
64
|
trip the old headless poller; the engine handles tool-call gaps now — F1.)
|
|
47
65
|
- Don't trust its self-reported version string ("I am gemini-X") as ground truth.
|
|
66
|
+
- **Red-team:** takes an adversarial brief well — high variance by design; use when consensus risk is high.
|
|
67
|
+
- **Blind self-votes are inconsistent** (self-#1 in some runs, self-last in others) → discount self-votes either way.
|
|
68
|
+
- Alias has resolved to **flash** tiers: fast, shallowest coverage, yet a recurring sharp fact/consistency checker (it alone refuted a bench-wide date error). Cheap cross-check value.
|
|
69
|
+
- **Unreliable on long agentic reads** (see global rule) — inline the text or swap models for book-length material.
|
|
48
70
|
|
|
49
71
|
### DeepSeek (`--model deepseek` → via OpenRouter)
|
|
50
72
|
- **Strengths:** resilient; produces strong, well-structured, well-cited critical analysis. A good
|
|
51
73
|
default reviewer and a proven chair.
|
|
52
74
|
- **Quirk:** occasional transient 502 mid-run → re-run the leg.
|
|
75
|
+
- Proven chair (5 clean chairings) — decisive, well-structured synthesis.
|
|
76
|
+
- As a Stage-1 reviewer of human-facing documents it **over-escalates severity** (typos/tenure → "blocker"); discount its blocker labels against peers. Prune its self-retractions when tallying.
|
|
77
|
+
- **"Agree-with-the-adversary" lean:** it has been the lone endorser of a red-team's harshest claims, turning them Contested — cross-check before treating its lone agreements as consensus.
|
|
53
78
|
|
|
54
79
|
### GPT (`--model gpt` → via OpenRouter)
|
|
55
80
|
- **Strengths:** reachable via the OpenRouter key; resilient; very thorough structured critique
|
|
56
81
|
(25 findings on a 1-page framework). Cleanly separates the review criteria.
|
|
57
82
|
- **Quirks:** verbose — peers dinged it for volume-over-judgment (good coverage, lower
|
|
58
83
|
discrimination); **self-ranked its own review #1** in cross-review → discount self-votes.
|
|
84
|
+
- Ranked genuine #1 by all judges (incl. non-self) in one run — thoroughness is real, not a self-vote artifact. Handled an 82k-word agentic read cleanly.
|
|
85
|
+
- **Asserts context-dependent facts (dates, "is this future?") without verifying** — and self-confirms them in adjudication. Cross-check any time-dependent claim it raises.
|
|
86
|
+
- A good calibration anchor in cross-review: confirms observational findings, disputes interpretive overreach.
|
|
87
|
+
|
|
88
|
+
### Grok (`--model grok` → via OpenRouter)
|
|
89
|
+
- Very fast legs; credible judge and chair (rejected its own weak findings as chair; honest blind self-rank).
|
|
90
|
+
- Strong red-team fit; handled an 82k-word agentic read. Weight its **observational** catches heavily and its **interpretive** verdicts cautiously (bench pattern: the former confirmed, the latter disputed).
|
|
91
|
+
- Stage-1 non-red-team reviews skew to scope-inflated "missing content" majors.
|
|
92
|
+
|
|
93
|
+
### Kimi (`--model kimi` → via OpenRouter)
|
|
94
|
+
- The bench's sharpest adjudicator (caught strawmen and misreads other judges waved through).
|
|
95
|
+
- **Very slow legs (5-7 min)** — it gates wave wall-clock; budget timeouts around it.
|
|
96
|
+
- Stalls on long agentic reads (poller "Incomplete" with only a preamble). Reserve for short-artifact adjudication.
|
|
97
|
+
|
|
98
|
+
### Mistral (`--model mistral` → via OpenRouter)
|
|
99
|
+
- Fast, broad coverage, catches real issues.
|
|
100
|
+
- **Hallucination risk is real:** has invented non-existent product models/specs, disputed independently by two judges. Cross-check every specific model number or product claim it introduces.
|
|
101
|
+
|
|
102
|
+
### Claude (in-council, when toggle on)
|
|
103
|
+
- Consistently the most *calibrated* reviewer (no severity inflation; findings overwhelmingly Confirmed; bench-best street-cred in recent runs) but sometimes the least *original* — it can miss the boldest single catch. Treat as a reliability floor, not a discovery engine.
|
|
59
104
|
|
|
60
105
|
### (others — add as used)
|
|
61
106
|
- Opus / o-series etc. are reachable via amicus **if their API keys are configured**. Add notes
|
|
@@ -114,3 +159,4 @@ This section keeps only per-model **qualitative quirks** and **structural-confor
|
|
|
114
159
|
cap (superseded by `--prompt-file`), the absolute-path/cwd trap (fixed by F2), and the
|
|
115
160
|
GUI-hangs-on-this-machine rule (resolved 2026-06-10; headless stays the council default by
|
|
116
161
|
design). Config path updated to `~/.config/amicus/.env`.
|
|
162
|
+
- **2026-07-02** — Folded back field lessons from runs 4-7 (AV-receiver, pork-shoulder, resume, novel ×2 councils): PowerShell `--models` quoting; current-date injection; long-read model selection; judge no-tools preamble; severity-inflation-justifies-dispute; five-keys tally schema; new Grok/Kimi/Mistral/Claude-in-council sections. Quantitative history stays in the ledger (`amicus council stats`).
|
|
@@ -18,6 +18,8 @@ Operating lessons from each run fold back into `MODEL-NOTES.md` (with approval),
|
|
|
18
18
|
|
|
19
19
|
**Before launching any model, READ `MODEL-NOTES.md`** (next to this file). It holds the operating rules and per-model quirks that decide whether a run succeeds or silently fails. These were learned the hard way; skipping them wastes runs and produces empty results that look like answers.
|
|
20
20
|
|
|
21
|
+
**Transport rule — CLI not on PATH:** every command below assumes the `amicus` CLI. If `amicus` is not on PATH (typical for **plugin-only installs**), run the identical commands as `npx -y amicus@latest <args>` (e.g. `npx -y amicus@latest fanout --models "m1,m2,m3" --prompt-file <path> --json`), or use the equivalent MCP tools (`amicus_fanout`, `amicus_start`, `amicus_status`, `amicus_read`, `amicus_council_tally`, `amicus_council_stats`, `amicus_verdict`) — council briefings are always self-contained (`--no-context`), so MCP transport is equivalent.
|
|
22
|
+
|
|
21
23
|
## When to use
|
|
22
24
|
|
|
23
25
|
- The user provides documents, artifacts, or links **and** an analysis request **and** criteria, and wants other models to weigh in independently.
|
|
@@ -53,6 +55,10 @@ in this run is written here. Use its absolute path in all `--prompt-file` argume
|
|
|
53
55
|
- Write every briefing to a temp file (`_tmp-*.md` in the run folder) and pass it with
|
|
54
56
|
`--prompt-file` — never inline a briefing as a CLI argument. All `_tmp-*` files are cleaned up
|
|
55
57
|
after the run.
|
|
58
|
+
- **Inject the current date into every briefing when the artifact is time-sensitive** (resumes, dated
|
|
59
|
+
plans, anything with start/end dates or 'present' ranges). Headless council models do not reliably
|
|
60
|
+
know "today" and have raised false "future-dated" blockers; state the date explicitly, e.g.
|
|
61
|
+
"Today's date is YYYY-MM-DD."
|
|
56
62
|
|
|
57
63
|
**Pick the council.** Default: **3 models from different families (non-Claude)**. Recommend them ranked by fit, consulting both the reviewer-reliability data from `amicus council stats` (the authoritative quantitative source — runs, avg peers-only street-cred, confirm-rate, fact-error rate) and the qualitative quirks in `MODEL-NOTES.md`. State the estimated cost. The estimate is the budget gate's pre-flight figure (per-$/Mtok pricing from the cached catalog; direct-provider legs without catalog pricing are disclosed as "cost unknown"). State it as an estimate, not a guarantee. **Disclose the run shape up front** before asking for confirmation — e.g.:
|
|
58
64
|
|
|
@@ -97,10 +103,12 @@ Each council model reviews **the artifact** independently. Write one Stage-1 bri
|
|
|
97
103
|
(`_tmp-briefing-stage1.md` in the run folder) and launch the whole wave as ONE background call:
|
|
98
104
|
|
|
99
105
|
```
|
|
100
|
-
amicus fanout --models <m1,m2,m3> --prompt-file <run-folder>/_tmp-briefing-stage1.md --json \
|
|
106
|
+
amicus fanout --models "<m1,m2,m3>" --prompt-file <run-folder>/_tmp-briefing-stage1.md --json \
|
|
101
107
|
--agent Plan --no-context --summary-length verbose --timeout <minutes>
|
|
102
108
|
```
|
|
103
109
|
|
|
110
|
+
Always quote the `--models` list — unquoted, PowerShell splits on commas and the CLI receives one mangled alias (instant arg-parse failure).
|
|
111
|
+
|
|
104
112
|
Run it in the background (`run_in_background: true`); you are notified on completion — do not
|
|
105
113
|
poll. `fanout` is headless by definition. The command exits when every leg is terminal and prints
|
|
106
114
|
ONE JSON wave document on stdout (`schemaVersion: 1`; the wave's id field is `waveId`, each leg's id is `taskId`): check `status` (`complete` | `partial` |
|
|
@@ -145,7 +153,7 @@ equivalent.
|
|
|
145
153
|
Instruct models to emit the structured JSON verbatim after the prose, without preamble, so it parses cleanly.
|
|
146
154
|
|
|
147
155
|
**After the wave returns, validate each leg's findings block** using `validateFindings` (Unit A — `src/council/findings.js`). If a leg's JSON fails validation:
|
|
148
|
-
1. Issue a **solo `start --json`** re-prompt to that one model: "re-emit only the findings JSON, fixing: \<errors\>." Keep the first-pass prose. (Solo `start`
|
|
156
|
+
1. Issue a **solo `start --json`** re-prompt to that one model: "re-emit only the findings JSON, fixing: \<errors\>." Keep the first-pass prose. (Solo `start` passes through the **same budget gate** as `fanout`. If launching the wave required `--max-cost <$>` or `--no-cost-gate`, pass the **same flag on every repair re-prompt and on the chair call** — otherwise the gate can refuse a repair or the chair mid-council.)
|
|
149
157
|
2. If still malformed, retry **once more** (cap = **2** re-prompts total).
|
|
150
158
|
3. If still malformed after 2 retries, mark the review `unstructured` and hand-parse its prose into the schema. The review proceeds — never dropped for a formatting miss.
|
|
151
159
|
|
|
@@ -182,7 +190,7 @@ Each model **unknowingly ranks and adjudicates its own review** — this is the
|
|
|
182
190
|
model. Write the bundle + judging instructions to `_tmp-bundle-stage2.md` and launch one wave:
|
|
183
191
|
|
|
184
192
|
```
|
|
185
|
-
amicus fanout --models <m1,m2,m3> --prompt-file <run-folder>/_tmp-bundle-stage2.md --json \
|
|
193
|
+
amicus fanout --models "<m1,m2,m3>" --prompt-file <run-folder>/_tmp-bundle-stage2.md --json \
|
|
186
194
|
--agent Plan --no-context --summary-length verbose --timeout <minutes>
|
|
187
195
|
```
|
|
188
196
|
|
|
@@ -191,6 +199,12 @@ adjudication response. **Stage-2 degrade:** a judge leg dies → tally over the
|
|
|
191
199
|
(≥ 1) and disclose the reduced bench in `crossreview-matrix.md`; tier definitions are unchanged
|
|
192
200
|
(they already count "judges engaged"). Each judge is asked to do two things on the bundle:
|
|
193
201
|
|
|
202
|
+
**Judge-briefing hardening (required).** Open `_tmp-bundle-stage2.md` with this preamble, verbatim, as its first line:
|
|
203
|
+
|
|
204
|
+
> Do NOT use any tools or read any files; everything is in this message; begin immediately with A1:
|
|
205
|
+
|
|
206
|
+
Plan-agent judges have wandered to tools mid-adjudication (reading files instead of judging and returning only narration), and a tool-capable judge can read the de-anonymized `review-<model>.md` files in the run folder — an anonymization leak. The preamble closes both. **Scratch-cwd (optional second layer):** launch the Stage-2 wave (and the Stage-3 chair call) with `--cwd <run-folder>/_scratch/` — create the empty directory first — so even a wandering agent finds nothing to read. Caveat: those legs' session records then live under `_scratch/.claude/amicus_sessions/`, so any later `amicus read <taskId>` for them needs the same `--cwd`.
|
|
207
|
+
|
|
194
208
|
**Task A — Rank.** Order the reviews from most to least accurate and insightful. End the response with a parseable block in exactly this format (no other text on those lines):
|
|
195
209
|
|
|
196
210
|
```
|
|
@@ -205,11 +219,16 @@ FINAL RANKING:
|
|
|
205
219
|
As each judge's ranking + adjudication response returns, collect it (the raw per-judge responses are working intermediates, not separate run-folder artifacts). Once all are in, **assemble the de-anonymized tally input** and then call `amicus council tally`:
|
|
206
220
|
|
|
207
221
|
**Stage-2 → tally assembly recipe (Claude's work before calling `tally`):**
|
|
222
|
+
0. **Build `meta` and `findings[]` first — `tally` requires both** (missing either fails with `BAD_ARGS: Cannot read properties of undefined (reading 'map')`):
|
|
223
|
+
- `meta` = `{ "runId": "<run-folder stem>", "models": [<every reviewed model id, including "claude" when the toggle is on — this is the street-cred universe>], "chair": "<confirmed chair model id>", "claudeInCouncil": <Stage-0 toggle> }`. Optional extras: `runType`, `date`.
|
|
224
|
+
- `findings[]` = one entry per finding across ALL reviews: `{ "id": "<run-global label id from step 1, e.g. A1>", "raiser": "<de-anonymized model that raised it>", "severity": "<from the review JSON>" }` (`claim` may be carried along but is not required).
|
|
208
225
|
1. **Rewrite finding ids to run-global label ids.** Each Stage-1 review's local integer ids (`1`, `2`, `3`…) become `A1`, `A2`, `A3`… (where `A` is that review's anonymized label). The label↔model map (`Review A → deepseek`, etc.) is the key.
|
|
209
226
|
2. **Build `adjudications`** — for every judge across all findings: `findingId` = run-global label id; `judge` = the model id (de-anonymized via the map); `verdict ∈ {agree, dispute, neutral}`. Include every judge's verdict on every finding. The raiser's own adjudication of its own finding is **included in the input** (the tally engine excludes it when computing peers-only tiers — do not pre-filter it).
|
|
210
227
|
3. **Translate each judge's `FINAL RANKING:` block** — convert the label order (`1. Review C / 2. Review A / 3. Review B`) into a model `order` array via the same map (e.g. `{C→mistral, A→deepseek, B→gpt}` ⇒ `order: ["mistral","deepseek","gpt"]`). This is each entry in `rankings[]`.
|
|
211
228
|
4. **Populate `runStats`** from the per-leg run documents emitted by `fanout --json` (and any solo red-team/chair `start --json` docs): copy `model`, `status`, `durationMs`, `usage` verbatim. Any leg with no run doc gets `durationMs: null` and `usage: null` — never invent a value. Attach `role` (`council` | `redteam` | `claude`), `wasChair`, and `conformance` (`clean` | `repaired` | `unstructured`) as council-domain labels.
|
|
212
229
|
|
|
230
|
+
**Five-keys checklist — verify `tally-input.json` has ALL of:** `meta` (with `meta.models`), `findings`, `adjudications`, `rankings`, `runStats` (`runStats` may be `[]`; the other four are required). Do not call `tally` until all five are present.
|
|
231
|
+
|
|
213
232
|
Then call:
|
|
214
233
|
|
|
215
234
|
```
|
|
@@ -240,11 +259,15 @@ amicus start --model <chair> --no-ui --json \
|
|
|
240
259
|
--agent Plan --no-context --summary-length verbose --timeout <minutes>
|
|
241
260
|
```
|
|
242
261
|
|
|
262
|
+
(The budget gate applies to this solo call too — if Stage 0 needed `--max-cost <$>` or `--no-cost-gate` to launch the wave, the chair call needs the same flag.)
|
|
263
|
+
|
|
243
264
|
The run document's `summary` is the verdict. The packet contains:
|
|
244
265
|
- All Stage-1 reviews (de-anonymized — model attribution restored)
|
|
245
266
|
- All cross-review ranking outputs (with model attribution)
|
|
246
267
|
- All adjudication outputs (with model attribution and `agree | dispute | neutral` verdicts per finding)
|
|
247
268
|
|
|
269
|
+
Open `_tmp-chair-packet.md` with the no-tools preamble, adjusted for the chair: *'Do NOT use any tools or read any files; everything is in this message; begin immediately with the verdict.'* The packet is complete by construction — the chair must never go looking for files.
|
|
270
|
+
|
|
248
271
|
Instruct the chair to write a **synthesized verdict** that:
|
|
249
272
|
- Weighs each reviewer's findings by their peer-validated standing (street-cred rank and adjudication pattern)
|
|
250
273
|
- Distinguishes findings the bench broadly endorsed from contested or singleton claims
|
|
@@ -307,15 +330,17 @@ Do not advance to Stage 5 until every finding in both tiers has a recorded decis
|
|
|
307
330
|
— exact for `reported`, `~` for `estimated`, `?` for `unknown` — and never
|
|
308
331
|
invent a figure. Add a wave **total cost** row from the wave document's
|
|
309
332
|
`usage.cost` (`source: reported|estimated|mixed|unknown`). Any leg with no run doc → `durationMs: null`, `usage: null`; never invent a value.
|
|
310
|
-
- **Renderer:** once `verdict.json` is written, generate
|
|
311
|
-
`amicus council report <run-folder>/verdict.json --md > <run-folder>/report.md`
|
|
312
|
-
|
|
333
|
+
- **Renderer:** once `verdict.json` is written, generate BOTH renderings:
|
|
334
|
+
`amicus council report <run-folder>/verdict.json --md > <run-folder>/report.md` and
|
|
335
|
+
`amicus council report <run-folder>/verdict.json --html > <run-folder>/report.html`.
|
|
336
|
+
**`report.html` is the default final artifact to hand the user** — a self-contained,
|
|
337
|
+
shareable page. This emits the
|
|
313
338
|
adjudication matrix (finding × judge), the peers-only street-cred table, the
|
|
314
339
|
findings-by-tier groupings (Disputed-first), and the per-model + wave cost —
|
|
315
340
|
deterministic data only. Prefer it over hand-assembling the matrix; reserve
|
|
316
341
|
prose for the chair's synthesis and the decision log.
|
|
317
342
|
|
|
318
|
-
Tell the user exactly which files were written and where.
|
|
343
|
+
Tell the user exactly which files were written and where, leading with `report.html`, **and present the verdict inline in chat** — the chair's overall assessment (verbatim or lightly trimmed) plus the tier counts (Confirmed/Disputed/Contested/Singleton) and what was applied. Never hand over only file paths.
|
|
319
344
|
|
|
320
345
|
---
|
|
321
346
|
|
|
@@ -323,6 +348,8 @@ Tell the user exactly which files were written and where.
|
|
|
323
348
|
|
|
324
349
|
This stage updates `MODEL-NOTES.md` to make future runs better. **Nothing is written until the user approves a specific diff.**
|
|
325
350
|
|
|
351
|
+
The `MODEL-NOTES.md` **next to this file** is your machine-local run ledger: npm updates never overwrite it (it is installed only if missing), so lessons accumulate per machine. Durable, machine-independent lessons get folded back into the version-controlled copy in the amicus repo at release time (see the release checklist in `docs/publishing.md`).
|
|
352
|
+
|
|
326
353
|
**Reflect on this run.** Review the run for:
|
|
327
354
|
- Failures, near-misses, and mitigations that worked (poller traps, empty responses, timeout issues, briefing problems)
|
|
328
355
|
- Briefing wording that produced **richer or poorer** structured output than expected
|
|
@@ -432,6 +459,7 @@ Always **rank recommendations by fit**, state the trade-off for each option, and
|
|
|
432
459
|
— exact for `reported`, `~` for `estimated`, `?` for `unknown` — and never
|
|
433
460
|
invent a figure. Add a wave **total cost** row from the wave document's
|
|
434
461
|
`usage.cost` (`source: reported|estimated|mixed|unknown`). Any leg with no run doc → `durationMs: null`, `usage: null`.
|
|
462
|
+
- `report.html` — the same report rendered as a self-contained page (`amicus council report <verdict.json> --html`); the default artifact to share.
|
|
435
463
|
- Reviewed copy: `<stem>-reviewed.<ext>`, next to the source.
|
|
436
464
|
- Temp working files (`_tmp-*.md`: extracts, stage briefings, red-team brief, bundle, chair packet, proposed
|
|
437
465
|
MODEL-NOTES diff) live in the run folder and are cleaned up at the end of the run — the proposed-diff file
|
|
@@ -441,5 +469,5 @@ Always **rank recommendations by fit**, state the trade-off for each option, and
|
|
|
441
469
|
|
|
442
470
|
## Files
|
|
443
471
|
|
|
444
|
-
- `MODEL-NOTES.md` — operating rules, per-model qualitative quirks, cost guardrail, and structural-conformance notes. **Read it before Stage 0 (council selection and launch); update qualitative notes (with approval) in Stage 6.** Quantitative reliability data (runs, avg street-cred, confirm-rate, fact-error rate) comes from `amicus council stats`, not this file.
|
|
472
|
+
- `MODEL-NOTES.md` — operating rules, per-model qualitative quirks, cost guardrail, and structural-conformance notes. **Read it before Stage 0 (council selection and launch); update qualitative notes (with approval) in Stage 6.** Quantitative reliability data (runs, avg street-cred, confirm-rate, fact-error rate) comes from `amicus council stats`, not this file. This copy is machine-local (never overwritten on update); the shipped seed lives in the amicus repo and absorbs durable lessons at release time.
|
|
445
473
|
- `COUNCIL-DESIGN.md` — the design spec this skill implements (v3 + WS-3). Consult it if a mechanics question arises that the skill prose does not resolve.
|
package/skills/sidecar/SKILL.md
CHANGED
|
@@ -1,39 +1,37 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: sidecar
|
|
3
3
|
description: >
|
|
4
|
-
Spawn
|
|
5
|
-
Grok, Mistral,
|
|
6
|
-
talk to, chat with,
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
and argument-size caps). (4) NEVER use o3 or o3-pro unless the user explicitly asks for it by
|
|
17
|
-
name. These models are extremely expensive ($10-60+ per request). If the user asks for
|
|
18
|
-
o3, warn them about the cost before proceeding. Default to gemini for most tasks.
|
|
19
|
-
(5) When the user asks to query MULTIPLE LLMs simultaneously (e.g., "ask Gemini AND
|
|
20
|
-
ChatGPT", "compare Gemini vs GPT"), ALWAYS use --no-ui (headless) for all of them
|
|
21
|
-
unless the user explicitly requests interactive. Opening multiple Electron windows at
|
|
22
|
-
once is disruptive. Launch them all in parallel with run_in_background: true.
|
|
23
|
-
(6) When the SAME prompt should go to N models, use `amicus fanout --models a,b,c
|
|
24
|
-
--prompt-file <path> --json` (one headless wave, one JSON result) instead of N
|
|
25
|
-
separate start calls. Different prompts per model → separate parallel
|
|
26
|
-
`amicus start --no-ui` calls.
|
|
27
|
-
(7) For a SINGLE-model sidecar, DEFAULT to interactive — omit --no-ui so the
|
|
28
|
-
Electron UI opens and the user can watch, converse, and click Fold. Use --no-ui
|
|
29
|
-
for a single model only when the user asks for headless/autonomous, or for
|
|
30
|
-
unattended bulk automation. Interactive launches still use run_in_background: true.
|
|
4
|
+
Spawn a conversation with another LLM (Gemini, GPT, ChatGPT, Codex, o3, DeepSeek,
|
|
5
|
+
Qwen, Grok, Mistral, or Claude as a target) and fold the results back into your
|
|
6
|
+
context. TRIGGER when: the user asks to talk to, chat with, call, use, or spawn
|
|
7
|
+
another LLM or model; names any non-current model; wants parallel exploration or
|
|
8
|
+
a quick take from a different model; or says "sidecar", "fork", or "fold". This
|
|
9
|
+
is NOT the skill for structured multi-model review of provided material — requests
|
|
10
|
+
like "second opinion", "council review", or red-team/stress-test against criteria
|
|
11
|
+
belong to the second-opinion skill. Before running any amicus command, read the
|
|
12
|
+
Operating Rules section at the top of this document: background launches,
|
|
13
|
+
--prompt-file briefings, interactive vs headless defaults, fanout for same-prompt
|
|
14
|
+
multi-model runs, the o3/o3-pro cost warning, and the npx fallback when amicus is
|
|
15
|
+
not on PATH.
|
|
31
16
|
---
|
|
32
17
|
|
|
33
18
|
# Amicus: Multi-Model Sidecar Tool
|
|
34
19
|
|
|
35
20
|
Spawn parallel conversations with different LLMs (Gemini, GPT, ChatGPT, Codex, o3, etc.) and fold results back into your context.
|
|
36
21
|
|
|
22
|
+
## Operating Rules
|
|
23
|
+
|
|
24
|
+
These rules are mandatory for every amicus invocation in this skill:
|
|
25
|
+
|
|
26
|
+
1. **ALWAYS launch amicus CLI commands with the Bash tool's `run_in_background: true`.** Never run `amicus start/resume/continue` in the foreground.
|
|
27
|
+
2. **The fold summary returns on stdout** when the user clicks Fold in the GUI or the headless agent finishes. Use TaskOutput to read it when the background task completes.
|
|
28
|
+
3. **For long or multi-line briefings, write them to a temp file and pass `--prompt-file <path>`** (mutually exclusive with `--prompt`; avoids shell-quoting hazards and argument-size caps).
|
|
29
|
+
4. **NEVER use o3 or o3-pro unless the user explicitly asks for it by name.** These models are extremely expensive (\$10-60+ per request). If the user asks for o3, warn them about the cost before proceeding. Default to gemini for most tasks. The CLI enforces this in code: a built-in budget gate refuses any model above a per-$/Mtok threshold (o3-pro class) before launch unless you pass `--no-cost-gate`; `--max-cost <$>` sets a soft estimated-total ceiling. When a run is refused with `BUDGET_EXCEEDED`, relay the gate's message — don't silently retry with the flag.
|
|
30
|
+
5. **When the user asks to query MULTIPLE LLMs simultaneously** (e.g., "ask Gemini AND ChatGPT", "compare Gemini vs GPT"), ALWAYS use `--no-ui` (headless) for all of them unless the user explicitly requests interactive. Opening multiple Electron windows at once is disruptive. Launch them all in parallel with `run_in_background: true`.
|
|
31
|
+
6. **When the SAME prompt should go to N models, use `amicus fanout --models "a,b,c" --prompt-file <path> --json`** (one headless wave, one JSON result) instead of N separate start calls. Different prompts per model → separate parallel `amicus start --no-ui` calls.
|
|
32
|
+
7. **For a SINGLE-model sidecar, DEFAULT to interactive** — omit `--no-ui` so the Electron UI opens and the user can watch, converse, and click Fold. Use `--no-ui` for a single model only when the user asks for headless/autonomous, or for unattended bulk automation. Interactive launches still use `run_in_background: true`.
|
|
33
|
+
8. **If `amicus` is not on PATH** (typical for plugin-only installs), run every command in this skill as `npx -y amicus@latest <args>` (e.g. `npx -y amicus@latest start --model gemini --prompt "..."`), or use the MCP tools (`amicus_start`, `amicus_status`, `amicus_read`, …) instead. Do not conclude the tool is broken because `amicus` is not found.
|
|
34
|
+
|
|
37
35
|
## Installation
|
|
38
36
|
|
|
39
37
|
```bash
|
|
@@ -204,11 +202,11 @@ amicus start \
|
|
|
204
202
|
```
|
|
205
203
|
|
|
206
204
|
**Required:**
|
|
207
|
-
- `--
|
|
208
|
-
- `--prompt`: Detailed task description you generate
|
|
205
|
+
- `--prompt` (or `--prompt-file`): Detailed task description you generate
|
|
209
206
|
|
|
210
207
|
**Recommended:**
|
|
211
|
-
- `--
|
|
208
|
+
- `--model`: The model to use (see Models below). Omit it to use your configured default (`amicus setup`); the CLI errors only when neither an explicit model nor a configured default exists.
|
|
209
|
+
- `--session-id`: Your Claude Code session ID for accurate context passing
|
|
212
210
|
|
|
213
211
|
**Optional:**
|
|
214
212
|
- `--no-ui`: Run autonomously without GUI (for bulk tasks)
|
|
@@ -257,7 +255,7 @@ The CLI validates all inputs **before** launching the sidecar. Invalid inputs fa
|
|
|
257
255
|
|
|
258
256
|
| Input | Validation | Error Message |
|
|
259
257
|
|-------|------------|---------------|
|
|
260
|
-
| `--model` |
|
|
258
|
+
| `--model` | Optional — falls back to the config default. An explicit value must resolve to a known alias or `provider/model` | `Error: Unknown model alias '<x>' …` or `No model specified and no default configured. Run 'amicus setup' to set a default model.` |
|
|
261
259
|
| `--prompt` | Must be present and non-empty | `Error: --prompt is required` or `Error: --prompt cannot be empty or whitespace-only` |
|
|
262
260
|
| `--cwd` | If provided, directory must exist | `Error: --cwd path does not exist: <path>` |
|
|
263
261
|
| `--session-id` | If explicit ID provided (not 'current'), must exist | `Error: --session-id '<id>' not found. Use 'amicus list' to see available sessions or omit --session-id for most recent.` |
|
|
@@ -283,7 +281,7 @@ If you receive a validation error, fix the input and retry:
|
|
|
283
281
|
|
|
284
282
|
```bash
|
|
285
283
|
# Error: --session-id 'abc123' not found
|
|
286
|
-
# Fix: Use 'current' or omit --session
|
|
284
|
+
# Fix: Use 'current' or omit --session-id
|
|
287
285
|
amicus start --model gemini --prompt "Task" --session-id current
|
|
288
286
|
|
|
289
287
|
# Error: --agent cannot be empty
|
|
@@ -303,7 +301,7 @@ amicus start --model gemini --prompt "Task"
|
|
|
303
301
|
### Fan Out One Prompt to N Models
|
|
304
302
|
|
|
305
303
|
```bash
|
|
306
|
-
amicus fanout --models gemini,gpt,deepseek --prompt-file ./briefing.md --json
|
|
304
|
+
amicus fanout --models "gemini,gpt,deepseek" --prompt-file ./briefing.md --json
|
|
307
305
|
```
|
|
308
306
|
|
|
309
307
|
Runs the same prompt on every listed model in parallel (one shared engine server, headless),
|
|
@@ -404,7 +402,7 @@ amicus abort --all # stop every running session in this project
|
|
|
404
402
|
|
|
405
403
|
### Model Selection
|
|
406
404
|
|
|
407
|
-
Use short aliases (run `amicus
|
|
405
|
+
Use short aliases (run `amicus models` to see the live catalog, and `amicus models --check` to audit your aliases):
|
|
408
406
|
- `--model gemini` -- Google Gemini (fast, large context)
|
|
409
407
|
- `--model opus` -- Claude Opus (deep analysis)
|
|
410
408
|
- `--model gpt` -- OpenAI GPT
|
|
@@ -454,13 +452,13 @@ ls -lt ~/.claude/projects/-Users-john-myproject/*.jsonl | head -5
|
|
|
454
452
|
The most recently modified file is likely your current session. Extract the UUID from the filename.
|
|
455
453
|
|
|
456
454
|
**Session ID behavior:**
|
|
457
|
-
- **Omit `--session`** or
|
|
455
|
+
- **Omit `--session-id`** or pass `--session-id current`: Uses the most recently modified session file (less reliable if multiple sessions are active)
|
|
458
456
|
- **Explicit session ID** (`--session-id abc123-def456`): Must exist or the command fails immediately with: `Error: --session-id 'abc123-def456' not found`
|
|
459
457
|
|
|
460
458
|
**If you get a session not found error:**
|
|
461
459
|
1. List available sessions: `amicus list`
|
|
462
460
|
2. Use one of the listed session IDs, OR
|
|
463
|
-
3. Omit `--session` to use the most recent session
|
|
461
|
+
3. Omit `--session-id` to use the most recent session
|
|
464
462
|
|
|
465
463
|
---
|
|
466
464
|
|
|
@@ -849,7 +847,7 @@ Find the correct encoded path for your project. Remember that `/`, `\`, and `_`
|
|
|
849
847
|
|
|
850
848
|
### "Multiple active sessions detected"
|
|
851
849
|
|
|
852
|
-
You have multiple Claude Code windows. Pass `--session` explicitly:
|
|
850
|
+
You have multiple Claude Code windows. Pass `--session-id` explicitly:
|
|
853
851
|
```bash
|
|
854
852
|
ls -lt ~/.claude/projects/[your-path]/*.jsonl | head -3
|
|
855
853
|
# Pick the correct session UUID
|
|
@@ -912,7 +910,7 @@ amicus start --model gemini --prompt "Debug the auth issue in TokenManager.ts"
|
|
|
912
910
|
|
|
913
911
|
The explicit session ID doesn't exist. Either:
|
|
914
912
|
1. Use `amicus list` to find valid session IDs
|
|
915
|
-
2. Omit `--session` to use the most recent session
|
|
913
|
+
2. Omit `--session-id` to use the most recent session
|
|
916
914
|
3. Use `--session-id current` for automatic resolution
|
|
917
915
|
|
|
918
916
|
```bash
|
package/src/headless.js
CHANGED
|
@@ -463,6 +463,7 @@ async function runHeadless(model, systemPrompt, userMessage, taskId, project, ti
|
|
|
463
463
|
const s = (statusData && statusData.type) ? statusData : (statusData && statusData[sessionId]);
|
|
464
464
|
if (s && s.type === 'idle') {
|
|
465
465
|
logger.debug('Session reported idle by SDK — completing', { sessionId });
|
|
466
|
+
completed = true;
|
|
466
467
|
break;
|
|
467
468
|
}
|
|
468
469
|
} catch (statusErr) {
|
|
@@ -493,6 +494,7 @@ async function runHeadless(model, systemPrompt, userMessage, taskId, project, ti
|
|
|
493
494
|
const threshold = assistantFinished ? stableFinishedPolls : stableIdlePolls;
|
|
494
495
|
if (stablePolls >= threshold) {
|
|
495
496
|
logger.debug('Session appears complete (idle)', { stablePolls, assistantFinished });
|
|
497
|
+
completed = true;
|
|
496
498
|
break;
|
|
497
499
|
}
|
|
498
500
|
} else {
|