@simonren/quorum 0.9.1 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +91 -48
- package/commands/multi-consult.md +47 -72
- package/commands/multi-review.md +68 -99
- package/dist/adapters/base.d.ts +53 -70
- package/dist/adapters/base.js +21 -79
- package/dist/adapters/claude.d.ts +38 -17
- package/dist/adapters/claude.js +93 -185
- package/dist/adapters/cli-adapter.d.ts +78 -0
- package/dist/adapters/cli-adapter.js +217 -0
- package/dist/adapters/codex.d.ts +18 -13
- package/dist/adapters/codex.js +64 -200
- package/dist/adapters/gemini.d.ts +30 -21
- package/dist/adapters/gemini.js +55 -149
- package/dist/adapters/index.d.ts +8 -5
- package/dist/adapters/index.js +7 -8
- package/dist/config.d.ts +98 -28
- package/dist/config.js +44 -25
- package/dist/consult-prompt.d.ts +6 -5
- package/dist/consult-prompt.js +48 -23
- package/dist/decoders/claude.d.ts +6 -0
- package/dist/decoders/claude.js +10 -1
- package/dist/doctor.d.ts +20 -0
- package/dist/doctor.js +91 -0
- package/dist/executor.d.ts +58 -46
- package/dist/executor.js +218 -129
- package/dist/guards.d.ts +29 -0
- package/dist/guards.js +72 -0
- package/dist/handoff.d.ts +79 -229
- package/dist/handoff.js +411 -402
- package/dist/index.d.ts +8 -12
- package/dist/index.js +81 -106
- package/dist/integrity.d.ts +16 -0
- package/dist/integrity.js +49 -0
- package/dist/sections.d.ts +22 -0
- package/dist/sections.js +31 -0
- package/dist/server.d.ts +9 -0
- package/dist/server.js +36 -0
- package/dist/tools/consult.d.ts +17 -76
- package/dist/tools/consult.js +72 -188
- package/dist/tools/format.d.ts +35 -0
- package/dist/tools/format.js +85 -0
- package/dist/tools/review.d.ts +55 -0
- package/dist/tools/review.js +120 -0
- package/dist/types.d.ts +20 -103
- package/dist/types.js +18 -29
- package/package.json +7 -4
- package/dist/cli/check.d.ts +0 -20
- package/dist/cli/check.js +0 -78
- package/dist/cli/codex.d.ts +0 -11
- package/dist/cli/codex.js +0 -255
- package/dist/cli/gemini.d.ts +0 -12
- package/dist/cli/gemini.js +0 -253
- package/dist/context.d.ts +0 -1538
- package/dist/context.js +0 -383
- package/dist/decoders/gemini.d.ts +0 -33
- package/dist/decoders/gemini.js +0 -58
- package/dist/errors.d.ts +0 -46
- package/dist/errors.js +0 -198
- package/dist/pipeline.d.ts +0 -135
- package/dist/pipeline.js +0 -462
- package/dist/prompt-v2.d.ts +0 -38
- package/dist/prompt-v2.js +0 -391
- package/dist/prompt.d.ts +0 -71
- package/dist/prompt.js +0 -309
- package/dist/schema.d.ts +0 -660
- package/dist/schema.js +0 -536
- package/dist/tools/feedback.d.ts +0 -91
- package/dist/tools/feedback.js +0 -117
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# Quorum
|
|
1
|
+
# Quorum — Multi-Model AI Review & Consultation for Claude Code
|
|
2
2
|
|
|
3
|
-
Convene a quorum of AI models
|
|
3
|
+
Convene a quorum of AI models — OpenAI Codex, Google Gemini, and a fresh-context Claude (Fable) — to review Claude Code's work or answer its questions. Claude Code verifies every finding against the code and delivers one verdict.
|
|
4
4
|
|
|
5
5
|
## Quick Install
|
|
6
6
|
|
|
@@ -11,7 +11,12 @@ claude mcp add -s user quorum -- npx -y @simonren/quorum
|
|
|
11
11
|
|
|
12
12
|
**Step 2: Restart Claude Code**
|
|
13
13
|
|
|
14
|
-
The MCP tools and slash commands (`/multi-review`, `/multi-consult`) are automatically
|
|
14
|
+
The MCP tools and slash commands (`/multi-review`, `/multi-consult`) are installed automatically on first launch.
|
|
15
|
+
|
|
16
|
+
**Check what quorum detects:**
|
|
17
|
+
```bash
|
|
18
|
+
npx -y @simonren/quorum doctor
|
|
19
|
+
```
|
|
15
20
|
|
|
16
21
|
**Manual command install** (if needed):
|
|
17
22
|
```bash
|
|
@@ -35,50 +40,100 @@ claude mcp add -s user quorum -- node /path/to/quorum/mcp-server/dist/index.js
|
|
|
35
40
|
|
|
36
41
|
## Prerequisites
|
|
37
42
|
|
|
38
|
-
Install at least one
|
|
43
|
+
Install at least one reviewer CLI. Claude Code itself is the third reviewer — it is spawned as a separate `claude -p` instance with read-only tools, no MCP servers, and no settings-file hooks or plugins, so it shares none of your conversation. It does still see the project's `CLAUDE.md` and Claude Code's auto-memory for that directory; only `--bare` would remove those, and `--bare` disables OAuth.
|
|
39
44
|
|
|
40
45
|
```bash
|
|
41
46
|
# OpenAI Codex CLI
|
|
42
47
|
npm install -g @openai/codex-cli
|
|
43
48
|
codex login
|
|
44
49
|
|
|
45
|
-
# Google Gemini — via Antigravity CLI (`agy`). `gemini-cli` is sunset on 2026-06-18.
|
|
50
|
+
# Google Gemini — via the Antigravity CLI (`agy`). `gemini-cli` is sunset on 2026-06-18.
|
|
46
51
|
curl -fsSL https://antigravity.google/cli/install.sh | bash
|
|
47
52
|
agy # follow the Google OAuth flow
|
|
48
53
|
```
|
|
49
54
|
|
|
55
|
+
Node.js 20 or newer.
|
|
56
|
+
|
|
50
57
|
## Usage
|
|
51
58
|
|
|
52
|
-
|
|
59
|
+
```bash
|
|
60
|
+
/multi-review # review the work from this conversation
|
|
61
|
+
/multi-review focus the challenge on rollback safety # steer both passes
|
|
62
|
+
/multi-review security [xhigh] # focus area + Codex xhigh reasoning
|
|
63
|
+
/multi-consult <question> # ask the panel, get one synthesized answer
|
|
64
|
+
/multi-consult <question> [flex] # Codex flex tier (cheaper/slower)
|
|
65
|
+
```
|
|
53
66
|
|
|
54
|
-
**
|
|
55
|
-
- `/multi-
|
|
56
|
-
- `/multi-consult` - Ask all CLIs the same question and synthesize their answers. For consultation/Q&A — what's the best approach, how to solve X.
|
|
67
|
+
- `/multi-review` — for work Claude Code has already produced (code changes, a plan, findings, a proposal). Every model runs a **standard pass** (bugs, gaps, does it clear the stated bar) and an **adversarial pass** (assumptions, design decisions, failure paths) in parallel. Claude Code verifies each finding in the code and reports a verdict with a mandatory "Panel said:" footer.
|
|
68
|
+
- `/multi-consult` — for a question with no prior work to review: which approach, which technology, how to structure X. Each model answers in a fixed structure (Recommendation / Reasoning / Tradeoffs / Risks / Open questions); Claude Code synthesizes one recommendation with a "Models said:" footer so dissent stays visible.
|
|
57
69
|
|
|
58
|
-
|
|
70
|
+
Reserved tokens count only when bracketed at the end of the arguments: `[flex]` / `[cheap]`, `[default tier]`, `[xhigh]` (review) or `[high reasoning]` (consult). Both commands refuse to run from `~`, `/`, `/etc`, or other sensitive directories.
|
|
59
71
|
|
|
60
|
-
|
|
72
|
+
**For quick reviews:** just say "review" and Claude Code uses its native abilities. The panel is only convened when you ask for it — a call runs six reviewer processes and takes minutes.
|
|
61
73
|
|
|
62
|
-
|
|
74
|
+
## How It Works
|
|
63
75
|
|
|
64
|
-
```bash
|
|
65
|
-
/multi-review # Parallel standard + adversarial reviews from all CLIs
|
|
66
|
-
/multi-review focus on race conditions # Steer the adversarial focus
|
|
67
|
-
/multi-consult <question> # Ask all CLIs and synthesize
|
|
68
|
-
/multi-consult <question> [flex] # Use Codex flex tier (cheaper/slower)
|
|
69
76
|
```
|
|
77
|
+
Claude Code does work → /multi-review → brief + cwd → Codex, Gemini, Claude review (standard + adversarial, read-only)
|
|
78
|
+
→ Claude Code verifies findings against the code → verdict + "Panel said:"
|
|
70
79
|
|
|
71
|
-
|
|
72
|
-
|
|
80
|
+
You have a question → /multi-consult → question + criteria → Codex, Gemini, Claude answer
|
|
81
|
+
→ Claude Code synthesizes → recommendation + "Models said:"
|
|
73
82
|
```
|
|
74
|
-
|
|
75
|
-
|
|
83
|
+
|
|
84
|
+
**Key principles:**
|
|
85
|
+
- **Claude Code is primary.** External models assist; Claude Code verifies and decides.
|
|
86
|
+
- **Working-directory strategy.** Claude Code passes the cwd and a small brief; reviewers read the files themselves. Read-only by contract: Codex runs with `--sandbox read-only`; Claude runs in default permission mode with prompts disabled, write and agent tools removed, and only read-only `git` allowed in Bash; agy runs with `--sandbox`. agy cannot make its file tools read-only, so quorum fingerprints the working tree (`git status`) before and after every run and flags any change at the top of the result.
|
|
87
|
+
- **Structured, verifiable answers.** Reviewers respond with `Verdict` / `Findings` (severity, location, evidence, impact, fix, confidence) / `Answers` / `Decisions`; drift from the contract is flagged so Claude Code can discount it.
|
|
88
|
+
- **Synthesis, not passthrough.** Raw model output never reaches you unless you ask for it; the provenance footer always does.
|
|
89
|
+
|
|
90
|
+
## Configuration
|
|
91
|
+
|
|
92
|
+
`~/.config/quorum/config.json` is created with defaults on first launch and hot-reloaded when edited. Every key is optional.
|
|
93
|
+
|
|
94
|
+
```json
|
|
95
|
+
{
|
|
96
|
+
"codex": {
|
|
97
|
+
"enabled": true,
|
|
98
|
+
"model": "gpt-5.5",
|
|
99
|
+
"reasoningEffort": "high",
|
|
100
|
+
"serviceTier": "fast",
|
|
101
|
+
"consultReasoningEffort": "xhigh",
|
|
102
|
+
"consultServiceTier": "fast",
|
|
103
|
+
"inactivityTimeoutMs": { "high": 300000, "xhigh": 600000 },
|
|
104
|
+
"maxTimeoutMs": 3600000
|
|
105
|
+
},
|
|
106
|
+
"claude": {
|
|
107
|
+
"enabled": true,
|
|
108
|
+
"model": "fable",
|
|
109
|
+
"fallbackModel": "opus",
|
|
110
|
+
"effort": "high",
|
|
111
|
+
"consultEffort": "xhigh",
|
|
112
|
+
"inactivityTimeoutMs": 300000,
|
|
113
|
+
"maxTimeoutMs": 3600000
|
|
114
|
+
},
|
|
115
|
+
"gemini": {
|
|
116
|
+
"enabled": true,
|
|
117
|
+
"model": "gemini-3-pro-high",
|
|
118
|
+
"inactivityTimeoutMs": 900000,
|
|
119
|
+
"maxTimeoutMs": 3600000
|
|
120
|
+
}
|
|
121
|
+
}
|
|
76
122
|
```
|
|
77
123
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
124
|
+
| Key | Notes |
|
|
125
|
+
|-----|-------|
|
|
126
|
+
| `<adapter>.enabled` | `false` leaves that model out of the panel even when its CLI is installed. |
|
|
127
|
+
| `claude.model` | Passed as `claude --model`: an alias (`fable`, `opus`, `sonnet`) or a full model id. |
|
|
128
|
+
| `claude.fallbackModel` | Passed as `--fallback-model`, used when the primary is overloaded or not available; `null` disables it. |
|
|
129
|
+
| `claude.effort` / `claude.consultEffort` | `claude --effort` (`low`…`max`) for review / consult; `null` omits the flag. |
|
|
130
|
+
| `codex.reasoningEffort` / `codex.serviceTier` | Review defaults; the `consult*` keys are the consult defaults. Tool arguments override both. |
|
|
131
|
+
| `gemini.model` | Passed as `agy --model`; omit to use agy's own setting (`agy models` lists ids). |
|
|
132
|
+
| `*.inactivityTimeoutMs` | How long a reviewer may stay silent. Codex liveness is also proven via its rollout file; agy never streams, so its value is effectively a total cap and is also passed as agy's `--print-timeout`. |
|
|
133
|
+
|
|
134
|
+
`npx -y @simonren/quorum doctor` prints the detected CLIs, their versions, and the effective settings.
|
|
135
|
+
|
|
136
|
+
**Upgrading from 0.9.x:** your existing config file is kept as is and new keys get defaults. A config written by 0.9.x already contains `"model": "opus"`, so the Fable default only applies to fresh installs — set `claude.model` to `fable` yourself to switch. `doctor` warns when the fallback equals the model.
|
|
82
137
|
|
|
83
138
|
## Focus Areas
|
|
84
139
|
|
|
@@ -95,21 +150,12 @@ User has a question → User: /multi-consult → External CLIs answer → CC syn
|
|
|
95
150
|
|
|
96
151
|
## MCP Tools
|
|
97
152
|
|
|
98
|
-
The plugin exposes two MCP tools:
|
|
99
|
-
|
|
100
153
|
| Tool | Description |
|
|
101
154
|
|------|-------------|
|
|
102
|
-
| `multi_review` |
|
|
103
|
-
| `multi_consult` | Ask
|
|
104
|
-
|
|
105
|
-
## Output Format
|
|
155
|
+
| `multi_review` | Standard + adversarial review from every available CLI. Requires `ccOutput` (Claude Code's brief) and `outputType` (`implementation`, `plan`, `findings`, `analysis`, `proposal`). Returns one `<review model=… pass=…>` block per model and pass. |
|
|
156
|
+
| `multi_consult` | Ask every available CLI the same question. Requires `question`; optional `relevantFiles`. Returns one `<answer model=…>` block per model with a 5-section structured response. |
|
|
106
157
|
|
|
107
|
-
|
|
108
|
-
- **Findings**: Issues with severity, confidence, location, and suggestions
|
|
109
|
-
- **Agreements**: Validations of CC's correct assessments
|
|
110
|
-
- **Disagreements**: Challenges to CC's claims with corrections
|
|
111
|
-
- **Alternatives**: Different approaches with tradeoffs
|
|
112
|
-
- **Risk Assessment**: Overall risk level with top concerns
|
|
158
|
+
Both tools are read-only and take minutes; failures in one model never hide the others' results.
|
|
113
159
|
|
|
114
160
|
## Development
|
|
115
161
|
|
|
@@ -118,25 +164,22 @@ cd mcp-server
|
|
|
118
164
|
npm install
|
|
119
165
|
npm run build # Build once
|
|
120
166
|
npm run dev # Watch mode
|
|
121
|
-
npm test # Run tests
|
|
167
|
+
npm test # Run tests (no network, no CLIs needed)
|
|
122
168
|
npm run test:watch # Watch mode tests
|
|
123
169
|
npm start # Run server
|
|
170
|
+
node dist/index.js doctor
|
|
124
171
|
```
|
|
125
172
|
|
|
126
173
|
## Publishing
|
|
127
174
|
|
|
128
|
-
|
|
129
|
-
CI triggers on GitHub Release, validates the tag matches `package.json`.
|
|
175
|
+
Local, Makefile-driven, with npm's web 2FA (run from the repo root):
|
|
130
176
|
|
|
131
177
|
```bash
|
|
132
|
-
# 1. Bump version in package.json
|
|
133
|
-
#
|
|
134
|
-
|
|
135
|
-
#
|
|
136
|
-
git
|
|
137
|
-
git tag v1.x.x
|
|
138
|
-
git push && git push --tags
|
|
139
|
-
gh release create v1.x.x --title "v1.x.x" --generate-notes
|
|
178
|
+
# 1. Bump "version" in mcp-server/package.json and .claude-plugin/plugin.json
|
|
179
|
+
make check # build + tests
|
|
180
|
+
git add -A && git commit -m "vX.Y.Z — <summary>"
|
|
181
|
+
make release # npm publish (browser 2FA) → verify → tag + push tag
|
|
182
|
+
git push
|
|
140
183
|
```
|
|
141
184
|
|
|
142
185
|
## License
|
|
@@ -1,109 +1,84 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
## Arguments
|
|
6
|
-
- `$ARGUMENTS` — the question itself, optional steering, or both
|
|
7
|
-
|
|
8
|
-
## When to Use
|
|
9
|
-
|
|
10
|
-
Use `/multi-consult` when you have a question or problem and want a synthesized panel opinion. The panel responds in a fixed 5-section structure (Recommendation / Reasoning / Tradeoffs / Risks / Open questions). CC reads all three responses and presents one consolidated answer with a "Models said:" provenance footer.
|
|
11
|
-
|
|
12
|
-
## Examples
|
|
13
|
-
|
|
14
|
-
```
|
|
15
|
-
/multi-consult Should we use Postgres or DynamoDB for a write-heavy timeseries workload?
|
|
16
|
-
/multi-consult How should I refactor the auth middleware? Focus on rollback safety.
|
|
17
|
-
/multi-consult What's the cleanest way to memoize this expensive selector? [flex]
|
|
18
|
-
```
|
|
1
|
+
---
|
|
2
|
+
description: Ask Codex, Gemini, and Claude (fresh context) the same question in parallel and synthesize one recommendation
|
|
3
|
+
argument-hint: <question or steering> [flex|high reasoning]
|
|
4
|
+
---
|
|
19
5
|
|
|
20
|
-
|
|
6
|
+
# Multi Consult
|
|
21
7
|
|
|
22
|
-
|
|
8
|
+
Ask the panel — Codex, Gemini, and Claude with a fresh context — the same question in parallel and synthesize their answers into one recommendation. Use this for **consultation**: choosing an approach, weighing alternatives, getting a second opinion on a design. Not for reviewing work you (Claude Code, "CC") already did — that is `/multi-review`.
|
|
23
9
|
|
|
24
|
-
|
|
10
|
+
Each model answers in a fixed structure (Recommendation with a confidence line / Reasoning / Tradeoffs / Risks / Open questions). You read all of them and present one consolidated answer with a "Models said:" footer so dissent stays visible.
|
|
25
11
|
|
|
26
|
-
|
|
27
|
-
- **Otherwise — `$ARGUMENTS` IS the literal question.** Set `customPrompt` to empty. Reserved tokens are extracted *only* when they appear at the *end* of `$ARGUMENTS` inside brackets or parens — e.g., `... [flex]`, `... (high reasoning)`. A bare occurrence of `flex` / `cheap` / `default tier` inside the prose is treated as part of the question, NOT a flag, to avoid corrupting questions like *"Should we offer a flex tier or default tier for customers?"*.
|
|
12
|
+
## Arguments
|
|
28
13
|
|
|
29
|
-
|
|
14
|
+
- If this conversation already contains the question you have been working on, compose `question` from that context and treat `$ARGUMENTS` as pure steering → `customPrompt`.
|
|
15
|
+
- Otherwise `$ARGUMENTS` **is** the literal question. Leave `customPrompt` empty.
|
|
30
16
|
|
|
31
|
-
|
|
17
|
+
Reserved tokens count only when bracketed at the very end of `$ARGUMENTS`:
|
|
32
18
|
|
|
33
|
-
|
|
19
|
+
- `[flex]` / `[cheap]` / `[budget]` → `serviceTier: "flex"` (Codex, cheaper and slower)
|
|
20
|
+
- `[default tier]` → `serviceTier: "default"`
|
|
21
|
+
- `[high reasoning]` → `reasoningEffort: "high"` (Codex; the consult default is `xhigh`)
|
|
34
22
|
|
|
35
|
-
|
|
23
|
+
A bare `flex` or `default tier` inside the prose is part of the question — *"Should we offer a flex tier or a default tier?"* must not be corrupted.
|
|
36
24
|
|
|
37
|
-
|
|
25
|
+
## Step 1 — Compose the question
|
|
38
26
|
|
|
39
|
-
|
|
27
|
+
Write a self-contained question. The panel has no conversation context: include the constraints, the scale, the stack, and what a good answer must decide. Do not paste code; reference files instead.
|
|
40
28
|
|
|
41
|
-
**
|
|
29
|
+
**Append a CRITERIA block**, priority-ordered, each tagged `[stated]` (explicit in the conversation) or `[assumed]` (you had to fix it to make the question answerable):
|
|
42
30
|
|
|
43
31
|
```
|
|
44
32
|
CRITERIA (priority order):
|
|
45
|
-
1. [stated] cost
|
|
46
|
-
2. [stated] team writes Go; minimize
|
|
33
|
+
1. [stated] cost per request under $X per 1M ops
|
|
34
|
+
2. [stated] team writes Go; minimize operational complexity
|
|
47
35
|
3. [assumed] sustained ~10k QPS write rate
|
|
48
36
|
4. [assumed] eventual consistency acceptable for analytics
|
|
49
37
|
```
|
|
50
38
|
|
|
51
|
-
|
|
52
|
-
- `[assumed]` = you needed to fix it to recommend; the user did NOT say.
|
|
53
|
-
- Cap `[assumed]` at 3. If the top 3 don't fit, the question is too vague — bounce back to the user before calling.
|
|
39
|
+
Cap `[assumed]` at three. If the top three do not fit, the question is too vague — ask the user before calling.
|
|
54
40
|
|
|
55
|
-
**
|
|
41
|
+
**Clarification gate.** If any `[assumed]` criterion is load-bearing (the recommendation would flip if it is wrong), stop and ask the user before calling the tool:
|
|
56
42
|
|
|
57
|
-
>
|
|
43
|
+
> Before I consult the panel: I'm assuming <assumption>. Right, or should I go with <plausible alternative>?
|
|
58
44
|
|
|
59
|
-
A burned panel call on a wrong
|
|
45
|
+
A burned panel call on a wrong assumption costs more than the round-trip. Skip the gate when the stated criteria pin the answer, when the user told you to proceed without questions, or when the remaining assumptions are clearly incidental.
|
|
60
46
|
|
|
61
|
-
|
|
62
|
-
- `[stated]` criteria fully pin the answer space (no assumptions needed).
|
|
63
|
-
- The user told you to proceed without clarification.
|
|
64
|
-
- Remaining assumptions are clearly incidental (would not flip the rec).
|
|
47
|
+
## Step 2 — Triage files
|
|
65
48
|
|
|
66
|
-
|
|
49
|
+
If the question is about this codebase, set `relevantFiles` to the minimal subset (typically 3–10) the panel must read. For general questions (*"Postgres or DynamoDB for a write-heavy timeseries workload?"*) omit it — the panel answers from expertise without touching the filesystem.
|
|
67
50
|
|
|
68
|
-
Call
|
|
51
|
+
## Step 3 — Call the tool
|
|
69
52
|
|
|
70
|
-
|
|
71
|
-
{
|
|
72
|
-
"workingDir": "<current directory>",
|
|
73
|
-
"question": "<CC-composed question OR literal $ARGUMENTS minus end-bracket reserved tokens>",
|
|
74
|
-
"relevantFiles": ["<file1>", "<file2>"],
|
|
75
|
-
"customPrompt": "<steering text or empty>"
|
|
76
|
-
}
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
### Reserved-token mappings (only when bracketed at end of $ARGUMENTS)
|
|
80
|
-
|
|
81
|
-
- `[flex]` / `[cheap]` / `[budget]` → `serviceTier: "flex"`
|
|
82
|
-
- `[default tier]` / `[standard tier]` → `serviceTier: "default"`
|
|
83
|
-
- `[high reasoning]` → `reasoningEffort: "high"` (overrides default `xhigh`)
|
|
53
|
+
Call `multi_consult` with:
|
|
84
54
|
|
|
85
|
-
If the
|
|
55
|
+
- `workingDir`: absolute path of the project root. If the current directory is `~`, `/`, `/etc`, or another sensitive system path, **refuse** and ask the user to run from a project root.
|
|
56
|
+
- `question`: the composed question including its CRITERIA block.
|
|
57
|
+
- `relevantFiles`: from Step 2, or omitted.
|
|
58
|
+
- `customPrompt`: steering, or omitted.
|
|
59
|
+
- `serviceTier` / `reasoningEffort`: only from bracketed reserved tokens.
|
|
86
60
|
|
|
87
|
-
|
|
61
|
+
The call takes minutes. Do not call it twice for the same question.
|
|
88
62
|
|
|
89
|
-
|
|
63
|
+
## Step 4 — Synthesize
|
|
90
64
|
|
|
91
|
-
|
|
65
|
+
The result is a `# Multi-Consult` header followed by one `<answer model=… status=…>` block per model. A block may carry `status="failed"` or a `⚠️ Format drift` line — discount a drifted answer and name the failure in your report.
|
|
92
66
|
|
|
93
|
-
1. **
|
|
94
|
-
2. **Mine Tradeoffs and Risks.** Even when
|
|
95
|
-
3. **Forward Open questions**
|
|
96
|
-
4. **Apply your own judgment.** You have
|
|
97
|
-
5. **
|
|
98
|
-
6. **Append
|
|
67
|
+
1. **Compare Recommendations** (and their confidence lines). All agree → high confidence. 2-vs-1 → take a side and surface the dissent explicitly; do not flatten it. All differ → present the tradeoff space honestly and pick.
|
|
68
|
+
2. **Mine Tradeoffs and Risks.** Even when the recommendations agree, the reasons and risks diverge. Surface the union. A risk raised by a single model is reported as "1 model raised: …", never dropped.
|
|
69
|
+
3. **Forward Open questions** only if material. Do not relay every "what's your scale?" clarifier.
|
|
70
|
+
4. **Apply your own judgment.** You have context the panel lacks. You may override a panel view that misses the user's real constraint — say so when you do.
|
|
71
|
+
5. **Answer in this shape:** **Recommendation** (what to do) → **Why** (synthesis of the reasoning, judged against the criteria) → **Watch out for** (consolidated risks, including single-model risks) → optional **Open question for you** if a real ambiguity blocks the answer.
|
|
72
|
+
6. **Append the footer** — one line per model, recommendation in ≤80 chars, failures included:
|
|
99
73
|
|
|
100
74
|
```
|
|
101
75
|
---
|
|
102
|
-
**Models said:**
|
|
76
|
+
**Models said:** Codex → Postgres + read replicas · Gemini → Postgres + Citus · Claude → DynamoDB, with a cost caveat at scale
|
|
103
77
|
```
|
|
104
78
|
|
|
105
|
-
This is
|
|
106
|
-
7.
|
|
107
|
-
|
|
79
|
+
This is non-negotiable; without it the synthesis is opaque.
|
|
80
|
+
7. Do not paste raw model output unless the user asks ("show me what each model said").
|
|
81
|
+
|
|
82
|
+
**All failed:** report the failure types and suggestions, then ask *"Panel unavailable — want my solo answer instead?"*. Do not silently substitute your own answer for the panel's.
|
|
108
83
|
|
|
109
84
|
$ARGUMENTS
|
package/commands/multi-review.md
CHANGED
|
@@ -1,139 +1,108 @@
|
|
|
1
|
-
|
|
1
|
+
---
|
|
2
|
+
description: Standard + adversarial review of your recent work by Codex, Gemini, and Claude (fresh context), synthesized into one verdict
|
|
3
|
+
argument-hint: [steering, e.g. "focus the challenge on rollback safety"] [flex|xhigh]
|
|
4
|
+
---
|
|
2
5
|
|
|
3
|
-
|
|
6
|
+
# Multi Review
|
|
4
7
|
|
|
5
|
-
|
|
8
|
+
Convene the panel on work you (Claude Code, "CC") produced in this conversation: code you changed, a plan you wrote, findings, an analysis, or a proposal. Every available model reviews it twice, in parallel — a **standard pass** (bugs, gaps, does it clear the bar) and an **adversarial pass** (assumptions, design decisions, failure paths). The reviewers have the filesystem but not this conversation. They return raw findings; **you verify each one against the code and deliver one verdict.** You are the judge, not a relay.
|
|
6
9
|
|
|
7
|
-
|
|
10
|
+
If there is no CC-produced work in this conversation to review, say so and suggest `/multi-consult` instead of inventing a brief.
|
|
8
11
|
|
|
9
12
|
## Arguments
|
|
10
|
-
- `$ARGUMENTS` - Optional: focus area, custom instructions, or adversarial steering
|
|
11
13
|
|
|
12
|
-
|
|
14
|
+
`$ARGUMENTS` is optional steering, passed to both passes: a focus (`security`, `performance`, …) or free text such as `focus the challenge on race conditions and rollback safety`.
|
|
13
15
|
|
|
14
|
-
|
|
16
|
+
Reserved tokens count only when bracketed at the very end of `$ARGUMENTS`:
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
/multi-review
|
|
20
|
-
/multi-review focus the challenge on race conditions and rollback safety
|
|
21
|
-
/multi-review challenge whether this was the right caching and retry design
|
|
22
|
-
```
|
|
18
|
+
- `[flex]` / `[cheap]` / `[budget]` → `serviceTier: "flex"` (Codex, cheaper and slower)
|
|
19
|
+
- `[default tier]` → `serviceTier: "default"`
|
|
20
|
+
- `[xhigh]` → `reasoningEffort: "xhigh"` (Codex, deeper than the default `high`)
|
|
23
21
|
|
|
24
|
-
|
|
22
|
+
A bare `flex` or `xhigh` inside the prose is part of the steering, not a flag.
|
|
25
23
|
|
|
26
|
-
|
|
24
|
+
## Step 1 — Write the brief (`ccOutput`)
|
|
27
25
|
|
|
28
|
-
|
|
26
|
+
The brief carries what only you know. Be concrete and short; skip any section that would be empty (SUMMARY is required). Never leave a placeholder.
|
|
29
27
|
|
|
30
28
|
```
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
29
|
+
SUMMARY:
|
|
30
|
+
<what you did and why, 1–3 sentences>
|
|
31
|
+
Bar: <what shipping-ready means here — concrete, testable criteria. Reviewers judge "material" against this line.>
|
|
34
32
|
|
|
35
|
-
|
|
33
|
+
UNCERTAINTIES (verify these):
|
|
34
|
+
1. [load-bearing] <if this assumption is wrong, the work is not shippable>
|
|
35
|
+
2. [incidental] <worth checking; would not block>
|
|
36
36
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
- `[incidental]` = nice to verify but won't block ship
|
|
37
|
+
QUESTIONS:
|
|
38
|
+
1. <a specific question you want answered>
|
|
40
39
|
|
|
41
|
-
|
|
40
|
+
DECISIONS:
|
|
41
|
+
1. <choice>. Rationale: <why this over the alternatives you considered>
|
|
42
42
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
- [load-bearing] "Is the cache invalidation race-free under concurrent updates?"
|
|
46
|
-
- [incidental] "Is the TTL value optimal — could it be 60s instead of 30s?"
|
|
47
|
-
```
|
|
43
|
+
KNOWN GAPS:
|
|
44
|
+
- <things you deliberately left out or deferred, so reviewers do not re-report them>
|
|
48
45
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
QUESTIONS:
|
|
52
|
-
- "Should I use write-through or write-behind caching?"
|
|
53
|
-
- "Is there a race condition in the invalidation logic?"
|
|
46
|
+
PRIORITY FILES:
|
|
47
|
+
- <file the reviewer must read first>
|
|
54
48
|
```
|
|
55
49
|
|
|
56
|
-
|
|
50
|
+
Rules for a good brief:
|
|
57
51
|
|
|
58
|
-
|
|
52
|
+
- State the **bar** even when it seems obvious. Without it reviewers default to code-quality vibes and drift between runs.
|
|
53
|
+
- Tag every uncertainty. Load-bearing items decide the verdict; incidental items are cheap to check.
|
|
54
|
+
- List **decisions** whenever you chose between alternatives (caching strategy, retry policy, schema shape). They give the adversarial pass something concrete to attack. Skip for a plain bug fix.
|
|
55
|
+
- Do not paste diffs or file contents. Reviewers read the code themselves.
|
|
59
56
|
|
|
60
|
-
|
|
61
|
-
DECISIONS:
|
|
62
|
-
1. Chose write-through cache over write-behind. Rationale: stronger read-after-write consistency at the cost of slightly slower writes; we prioritize correctness for catalog data.
|
|
63
|
-
2. Chose 30s TTL with explicit invalidation on update. Rationale: TTL bounds staleness if invalidation misses; explicit invalidation catches the common path immediately.
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
## Tool Invocation
|
|
57
|
+
## Step 2 — Call the tool
|
|
67
58
|
|
|
68
59
|
Call `multi_review` with:
|
|
69
60
|
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
```
|
|
61
|
+
- `workingDir`: absolute path of the project root. If the current directory is `~`, `/`, `/etc`, or another sensitive system path, **refuse** and ask the user to run from a project root.
|
|
62
|
+
- `ccOutput`: the brief.
|
|
63
|
+
- `outputType`: `implementation` for code you changed; `plan` for a plan not yet executed; `findings` for an investigation or audit result; `analysis`; `proposal` for a design.
|
|
64
|
+
- `analyzedFiles`: the files you created or modified, relative to `workingDir`.
|
|
65
|
+
- `baseRef`: only when the work is already committed (a branch or PR checked out with a clean tree): the ref to diff against, e.g. `main` or the merge-base SHA. Omit for uncommitted work.
|
|
66
|
+
- `focusAreas`: only if the user named one, and only values from `security`, `performance`, `architecture`, `correctness`, `maintainability`, `scalability`, `testing`, `documentation`. Omit otherwise.
|
|
67
|
+
- `customPrompt`: the steering text from `$ARGUMENTS` minus reserved tokens. Omit if empty.
|
|
68
|
+
- `serviceTier` / `reasoningEffort`: only from bracketed reserved tokens.
|
|
79
69
|
|
|
80
|
-
|
|
81
|
-
- If user says "flex", "cheap", or "budget" → set `serviceTier: "flex"`
|
|
82
|
-
- If user says "default tier" or "standard tier" → set `serviceTier: "default"`
|
|
83
|
-
- Otherwise → omit `serviceTier` (defaults to `"fast"` — priority processing, ~2x cost)
|
|
70
|
+
The call takes minutes. Do not call it twice for the same work.
|
|
84
71
|
|
|
85
|
-
|
|
72
|
+
## Step 3 — Verify, then judge
|
|
86
73
|
|
|
87
|
-
|
|
88
|
-
SUMMARY:
|
|
89
|
-
<what you did, 1-3 sentences>
|
|
90
|
-
Bar: <what counts as shipping-ready — concrete acceptance criteria>
|
|
74
|
+
The result is a `# Multi-Model Review` header followed by `## Standard review` and `## Adversarial review`, each holding one `<review model=… pass=… status=…>` block per model. Each block is a reviewer's raw response in the shape `## Verdict` / `## Findings` (`[S1|S2|S3]` with Location, Evidence, Impact, Fix, Confidence) / `## Answers` / `## Decisions`. A block may carry `status="failed"` or a `⚠️ Format drift` line.
|
|
91
75
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
76
|
+
1. **Read every block.** Note which model/pass failed and why; that goes in your report.
|
|
77
|
+
2. **Merge findings** into one deduplicated list across models and passes. Record who raised each. Agreement raises priority but does not replace verification.
|
|
78
|
+
3. **Verify each finding yourself.** Open the file at the cited location, trace the failure path, and mark it:
|
|
79
|
+
- ✓✓ verified — you confirmed the failure path in the code
|
|
80
|
+
- ✓ plausible — consistent with the code, not fully traced
|
|
81
|
+
- ✗ rejected — say why in one line
|
|
82
|
+
Adversarial-only findings are speculative by design: verify before acting. A finding with no location or evidence is unverified, not accepted.
|
|
83
|
+
4. **Resolve your own uncertainties** from the Answers sections, load-bearing first. Do the same for your questions and decisions; note where reviewers disagree with each other.
|
|
84
|
+
5. **Decide.** Your verdict, judged against your bar, not a tally of the panel's.
|
|
95
85
|
|
|
96
|
-
|
|
97
|
-
1. <question>
|
|
86
|
+
## Step 4 — Report
|
|
98
87
|
|
|
99
|
-
|
|
100
|
-
1. <choice>. Rationale: <why this over alternatives>
|
|
101
|
-
2. <choice>. Rationale: <why this over alternatives>
|
|
88
|
+
Use this shape:
|
|
102
89
|
|
|
103
|
-
PRIORITY FILES:
|
|
104
|
-
- <file>
|
|
105
90
|
```
|
|
91
|
+
**Verdict:** SHIP | SHIP WITH FIXES | DO NOT SHIP — <one sentence against the bar>
|
|
106
92
|
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
93
|
+
**Must fix** (S1/S2, ✓✓ or ✓): `path:line` — what is wrong → what you will do
|
|
94
|
+
**Worth fixing** (S3): …
|
|
95
|
+
**Rejected:** <finding> — <why>, one line each
|
|
96
|
+
**Uncertainties resolved:** <load-bearing items: verified / refuted / still open>
|
|
110
97
|
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
- Find agreements across models (higher confidence)
|
|
115
|
-
- Identify conflicts (YOU decide who's right)
|
|
116
|
-
|
|
117
|
-
2. **Challenge findings** — adversarial challenges from each model
|
|
118
|
-
- These target assumptions and design decisions, not just bugs
|
|
119
|
-
- Evaluate on merit — some challenges are speculative by design
|
|
120
|
-
- Strong challenges with evidence deserve serious consideration
|
|
98
|
+
---
|
|
99
|
+
**Panel said:** Codex → <verdict, ≤80 chars> · Gemini → … · Claude → … (standard / adversarial)
|
|
100
|
+
```
|
|
121
101
|
|
|
122
|
-
|
|
123
|
-
- Standard + challenge agreement = high confidence issue
|
|
124
|
-
- Challenge-only finding = investigate further before acting
|
|
102
|
+
The footer is mandatory — it is the audit trail. Include failed models as `Codex → failed (timeout)`. Do not paste raw model output unless the user asks for it.
|
|
125
103
|
|
|
126
|
-
|
|
127
|
-
- Check file/line references exist
|
|
128
|
-
- Read actual code
|
|
129
|
-
- Mark your confidence:
|
|
130
|
-
- ✓✓ Verified
|
|
131
|
-
- ✓ Plausible
|
|
132
|
-
- ? Investigate
|
|
133
|
-
- ✗ Rejected
|
|
104
|
+
Then act: if the user asked you to ship or the work is still in flight, apply the must-fix items now and re-verify; otherwise offer to apply them. Do not silently drop rejected findings.
|
|
134
105
|
|
|
135
|
-
|
|
136
|
-
- Don't just relay findings
|
|
137
|
-
- Apply your judgment
|
|
106
|
+
**All failed:** report the failure types and suggestions from the blocks. Do not present your own review as if it were the panel's; offer to review solo instead.
|
|
138
107
|
|
|
139
108
|
$ARGUMENTS
|