ai-consultants 2.25.1 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/.claude/commands/ai-consultants:consult.md +13 -19
  2. package/.claude/commands/ai-consultants:help.md +0 -2
  3. package/.env.example +1 -46
  4. package/CLAUDE.md +25 -37
  5. package/README.md +56 -89
  6. package/SKILL.md +17 -26
  7. package/docs/COST_RATES.md +1 -1
  8. package/docs/RECIPES.md +15 -97
  9. package/docs/SETUP.md +9 -37
  10. package/docs/SMART_ROUTING.md +0 -114
  11. package/docs/cost_rates.json +1 -1
  12. package/docs/releases/v2.25.2.md +53 -0
  13. package/docs/releases/v3.0.0.md +71 -0
  14. package/package.json +2 -1
  15. package/references/affinity.json +110 -56
  16. package/references/configuration.md +1 -97
  17. package/references/details.md +11 -5
  18. package/scripts/config.sh +4 -107
  19. package/scripts/configure.sh +1 -1
  20. package/scripts/consult_all.sh +2 -165
  21. package/scripts/doctor.sh +31 -26
  22. package/scripts/lib/api_query.sh +0 -9
  23. package/scripts/lib/common.sh +6 -122
  24. package/scripts/lib/personas.sh +1 -8
  25. package/scripts/lib/routing.sh +3 -75
  26. package/scripts/synthesize.sh +7 -10
  27. package/.claude/commands/ai-consultants:debate.md +0 -102
  28. package/.claude/commands/ai-consultants:roster-audit.md +0 -38
  29. package/references/calibration_benchmark.json +0 -65
  30. package/scripts/debate_round.sh +0 -409
  31. package/scripts/lib/orchestration.sh +0 -402
  32. package/scripts/lib/stance.sh +0 -70
  33. package/scripts/lib/voting.sh +0 -647
  34. package/scripts/peer_review.sh +0 -471
  35. package/scripts/roster_audit.sh +0 -157
  36. package/scripts/roster_calibrate.sh +0 -238
  37. package/scripts/run_calibration.sh +0 -76
  38. package/scripts/taste_elo.sh +0 -171
@@ -42,34 +42,29 @@ Query multiple AI models for expert opinions on a coding question.
42
42
  - Read `<output_dir>/synthesis.json` using the **Read** tool (may not exist if synthesis was skipped or failed)
43
43
  - If neither exists, use the **Glob** tool with pattern `<output_dir>/*.json` to discover individual consultant response files, then read each one
44
44
 
45
- 5. **Present the results** using this template:
45
+ 5. **Present the results** using this template. The panel's value is *coverage* — the
46
+ union of distinct points across diverse models, including what only one raised — not a
47
+ single voted winner:
46
48
 
47
- ### Consensus
48
- - **Score**: <consensus_percentage>% (<consensus_level: strong/medium/low/none>)
49
- - **Category**: <question_category>
49
+ ### Category
50
+ - <question_category>
50
51
 
51
52
  ### Consultant Summary
52
53
  | Consultant | Confidence | Approach | Key Insight |
53
54
  |------------|-----------|----------|-------------|
54
55
  | <name> | <score>/10 | <approach> | <one-line from summary> |
55
56
 
56
- ### Synthesized Recommendation
57
- - **Recommended approach**: <approach>
58
- - **Summary**: <synthesized summary>
57
+ ### Coverage (the union of distinct considerations)
58
+ - <every distinct point, recommendation, risk, and edge case raised by ANY consultant,
59
+ deduplicated but preserving points only one model raised; note who raised each>
59
60
 
60
- ### Key Agreements
61
- - <points most consultants agree on>
62
-
63
- ### Key Disagreements
64
- - <points where consultants diverge>
65
-
66
- ### Risk Assessment
67
- - <caveats and uncertainty factors raised>
61
+ ### Risks & Caveats
62
+ - <distinct caveats and uncertainty factors raised across the panel>
68
63
 
69
64
  ### Suggested Next Steps
70
- - <actionable items based on the recommendation>
65
+ - <actionable items drawn from the coverage above>
71
66
 
72
- If `synthesis.json` is missing, build the summary table and recommendation directly from the individual consultant JSON files in the output directory.
67
+ If `synthesis.json` is missing, build the summary table and coverage directly from the individual consultant JSON files in the output directory (take the union of their distinct points).
73
68
 
74
69
  6. **Error recovery**:
75
70
  - **Exit code 1** (general failure): Tell the user and suggest running diagnostics:
@@ -84,9 +79,8 @@ Query multiple AI models for expert opinions on a coding question.
84
79
 
85
80
  | Variable | Effect |
86
81
  |----------|--------|
87
- | `ENABLE_DEBATE=true` | Enable multi-round debate for deeper analysis |
88
82
  | `ENABLE_SMART_ROUTING=true` | Auto-select best consultants for the question type |
89
- | `DEBATE_ROUNDS=2` | Number of debate rounds (1-3) |
83
+ | `SYNTHESIS_STRATEGY=compare_only` | Present each consultant side-by-side instead of the coverage union |
90
84
  | `FORCE_PROJECT_TREE=true` | Include the project tree even for pointed categories (SECURITY, QUICK_SYNTAX, etc.) |
91
85
 
92
86
  ## Follow-up
@@ -12,8 +12,6 @@ All skill details (consultants, personas, presets, strategies, features, configu
12
12
  | Command | Description |
13
13
  |---------|-------------|
14
14
  | `/ai-consultants:consult` | Main consultation — ask AI consultants a coding question |
15
- | `/ai-consultants:debate` | Run consultation with multi-round debate |
16
- | `/ai-consultants:roster-audit` | Audit which consultants add distinct value vs echo the panel |
17
15
  | `/ai-consultants:help` | Show this help |
18
16
 
19
17
  ## Quick Examples
package/.env.example CHANGED
@@ -203,7 +203,7 @@ MINIMAX_FORMAT=openai
203
203
 
204
204
  # Defaults used when --preset / --strategy are omitted
205
205
  DEFAULT_PRESET=balanced
206
- DEFAULT_STRATEGY=majority
206
+ DEFAULT_STRATEGY=coverage
207
207
 
208
208
  # The invoking integration normally sets this per run for self-exclusion.
209
209
  INVOKING_AGENT=unknown
@@ -219,34 +219,6 @@ SYNTHESIS_MODEL=
219
219
  SYNTHESIS_EXTRACT_FIELDS=true
220
220
  SYNTH_MAX=10
221
221
 
222
- # Multi-Agent Debate (MAD): cross-critique rounds
223
- ENABLE_DEBATE=false
224
- DEBATE_ROUNDS=1 # 1 = no debate, 2-3 = with cross-critique
225
-
226
- # Peer Review: anonymous evaluation
227
- ENABLE_PEER_REVIEW=false
228
- PEER_REVIEW_MIN_RESPONSES=3
229
- REVIEWS_PER_CONSULTANT=3
230
-
231
- # Panic Mode: auto-rigor when uncertainty detected
232
- ENABLE_PANIC_MODE=auto
233
- PANIC_CONFIDENCE_THRESHOLD=5
234
- PANIC_EXTRA_DEBATE_ROUNDS=1
235
- PANIC_KEYWORDS="uncertain|maybe|not sure|possibly|unclear|depends|hard to say|difficult to determine"
236
-
237
- # Dynamic orchestration: choose how the panel deliberates
238
- # auto | fixed | quick | converge | adversarial | tournament | exhaustive
239
- ORCHESTRATION_MODE=auto
240
- CONVERGENCE_MAX_ROUNDS=4
241
- CONVERGENCE_TARGET_CONSENSUS=75
242
- CONVERGENCE_STALL_EPSILON=5
243
- ENABLE_ADVERSARIAL_VERIFY=true
244
-
245
- # Debate optimization: avoid extra rounds when the panel already agrees
246
- ENABLE_DEBATE_OPTIMIZATION=true
247
- DEBATE_CONFIDENCE_SPREAD_THRESHOLD=2
248
- DEBATE_USE_SUMMARIES=true
249
-
250
222
  # Health gate and quorum: validate the live panel before spending on a full run
251
223
  ENABLE_HEALTH_GATE=false # Adds one small ping per selected consultant
252
224
  HEALTH_GATE_TIMEOUT=30
@@ -297,23 +269,6 @@ COMPLEXITY_THRESHOLD_SIMPLE=3
297
269
  COMPLEXITY_THRESHOLD_MEDIUM=6
298
270
  FALLBACK_CONFIDENCE_THRESHOLD=7
299
271
 
300
- # Capability-aware routing & voting (v2.20, opt-in): weight votes and rank the
301
- # panel by each consultant's capability on the quality axis a question stresses
302
- # (intelligence or taste, per references/affinity.json 'category_axis'). cost is
303
- # a composition/budget axis only, never a vote weight.
304
- ENABLE_CAPABILITY_WEIGHTING=false # capability-weighted voting
305
- ENABLE_CAPABILITY_ROUTING=false # capability-aware panel composition
306
- CAPABILITY_WEIGHT_STRENGTH=10 # higher = gentler nudge: weight = conf*(S+cap)/S
307
- CAPABILITY_DEFAULT=5 # fallback capability for a missing consultant/axis
308
-
309
- # Stance-based semantic consensus (v2.21, opt-in): the panel picks from a shared
310
- # enumerated stance set so agreement is scored by exact match instead of lexical
311
- # similarity. Adds ~1 LLM call per run; degrades to the lexical cluster on any
312
- # failure, so it is always safe to enable.
313
- ENABLE_STANCE_CONSENSUS=false
314
- STANCE_MAX_OPTIONS=5 # max stance options generated per question
315
- STANCE_TIMEOUT=60 # seconds for the stance-generation call (guards a hang)
316
-
317
272
  # =============================================================================
318
273
  # COST MANAGEMENT
319
274
  # =============================================================================
package/CLAUDE.md CHANGED
@@ -6,7 +6,7 @@ AI Consultants is a multi-model AI deliberation system that queries up to 11 AI
6
6
 
7
7
  **Self-Exclusion**: The invoking agent is automatically excluded from the panel to prevent self-consultation. Claude Code won't query Claude, Codex CLI won't query Codex, etc.
8
8
 
9
- **Version**: 2.25.1
9
+ **Version**: 3.0.0
10
10
 
11
11
  ## Distribution
12
12
 
@@ -39,27 +39,18 @@ ai-consultants/
39
39
  │ ├── config.sh # Centralized configuration
40
40
  │ ├── doctor.sh # Diagnostic and auto-fix tool (v2.2)
41
41
  │ ├── update_clis.sh # Check/update installed consultant CLIs (v2.21)
42
- │ ├── peer_review.sh # Anonymous peer review (v2.2)
43
- │ ├── roster_audit.sh # Roster uncorrelated-value audit (v2.20)
44
- │ ├── roster_calibrate.sh # Measured capability calibration, Tier A (v2.20)
45
- │ ├── run_calibration.sh # Calibration data-collection harness (v2.20)
46
- │ ├── taste_elo.sh # Pairwise-judge taste Elo, Tier B (v2.20)
47
42
  │ ├── install.sh # One-liner installer (v2.2)
48
43
  │ ├── query_*.sh # Wrapper for each consultant
49
44
  │ ├── query_claude.sh # Claude consultant (v2.2)
50
- │ ├── synthesize.sh # Auto-synthesis of responses
51
- │ ├── debate_round.sh # Multi-Agent Debate
45
+ │ ├── synthesize.sh # Coverage-union synthesis of responses
52
46
  │ ├── classify_question.sh # Question classifier
53
47
  │ ├── followup.sh # Follow-up queries
54
48
  │ ├── preflight_check.sh # DEPRECATED v2.10.9 (thin wrapper -> doctor.sh)
55
49
  │ └── lib/
56
- │ ├── common.sh # Shared utilities (logging, panic mode)
50
+ │ ├── common.sh # Shared utilities (logging, quorum grading)
57
51
  │ ├── personas.sh # Consultant persona definitions
58
52
  │ ├── schema.json # JSON output schema
59
- │ ├── voting.sh # Voting/consensus + confidence intervals
60
- │ ├── routing.sh # Smart routing + cost-aware routing
61
- │ ├── orchestration.sh # Dynamic orchestration planner + shapes (v2.16)
62
- │ ├── stance.sh # Stance-based semantic consensus (v2.21, opt-in)
53
+ │ ├── routing.sh # Smart routing (category affinity) + cost-aware routing
63
54
  │ ├── session.sh # Session management
64
55
  │ ├── costs.sh # Cost tracking + response limits
65
56
  │ ├── cache.sh # Semantic caching (v2.3)
@@ -158,17 +149,13 @@ Each consultant must produce JSON with this minimum structure:
158
149
  ## Main Flow
159
150
 
160
151
  1. `consult_all.sh` receives query, optional files, and flags (`--preset`, `--strategy`)
161
- 2. Applies preset if specified (`apply_preset()` in config.sh)
152
+ 2. Applies preset if specified (`apply_preset()` in config.sh) — sets the consultant set + model tier
162
153
  3. Classifies the question (`classify_question.sh`)
163
- 4. Plans orchestration: complexity + intent → shape (`select_orchestration_shape()` in `lib/orchestration.sh`); `ORCHESTRATION_MODE=fixed` bypasses to the legacy path
164
- 5. Selects consultants (smart routing or all)
165
- 6. Launches parallel queries (`query_*.sh`)
166
- 7. Deliberates per the chosen shape: convergence loop / adversarial gate / tournament / exhaustive sweep (`run_orchestration()`), or fixed debate rounds in `fixed` mode
167
- 8. Calculates voting/consensus with confidence intervals (`lib/voting.sh`)
168
- 9. Checks for panic mode triggers (`lib/common.sh`)
169
- 10. Generates synthesis with selected strategy (`synthesize.sh`)
170
- 11. Optionally runs peer review (`peer_review.sh`)
171
- 12. Produces final report
154
+ 4. Selects consultants (smart routing by category affinity, or all)
155
+ 5. Launches parallel queries (`query_*.sh`) — one shot per consultant, no serial rounds
156
+ 6. Grades quorum (`grade_quorum()` in `lib/common.sh`) — surfaces a degraded panel
157
+ 7. Generates synthesis with the selected strategy (`synthesize.sh`); the default `coverage` produces the deduplicated UNION of every distinct point across the panel
158
+ 8. Produces final report
172
159
 
173
160
  ## v2.2 Features
174
161
 
@@ -613,32 +600,21 @@ for f in scripts/*.sh scripts/lib/*.sh; do bash -n "$f" && echo "OK: $f"; done
613
600
  |----------|---------|-------------|
614
601
  | `INVOKING_AGENT` | unknown | Agent invoking the skill (for self-exclusion) |
615
602
  | `ENABLE_CLAUDE` | true | Enable Claude consultant (auto-excluded under Claude Code) |
616
- | `ENABLE_DEBATE` | false | Enable Multi-Agent Debate |
617
- | `DEBATE_ROUNDS` | 1 | Number of debate rounds |
618
- | `ENABLE_SYNTHESIS` | true | Auto-synthesis of responses |
619
- | `ENABLE_SMART_ROUTING` | false | Intelligent routing |
603
+ | `ENABLE_SYNTHESIS` | true | Coverage-union synthesis of responses |
604
+ | `ENABLE_SMART_ROUTING` | false | Intelligent routing by category |
620
605
  | `ENABLE_COST_TRACKING` | true | Track costs |
621
606
  | `MAX_SESSION_COST` | 1.00 | Max budget ($) |
622
- | `ENABLE_PANIC_MODE` | auto | Panic mode trigger (v2.2) |
623
- | `PANIC_CONFIDENCE_THRESHOLD` | 5 | Panic threshold (v2.2) |
624
607
  | `CLAUDE_MODEL` | claude-opus-4-8 | Claude model (v2.5) |
625
608
  | `GEMINI_MODEL` | Gemini 3.1 Pro (High) | Gemini agy CLI model (v2.15) |
626
609
  | `GEMINI_API_MODEL` | gemini-3.1-pro-preview | Gemini API-mode model ID (v2.15) |
627
610
  | `CODEX_MODEL` | gpt-5.5 | Codex model (v2.5) |
628
611
  | `MISTRAL_MODEL` | mistral-large-3 | Mistral model (v2.5) |
629
- | `SYNTHESIS_STRATEGY` | majority | Synthesis strategy (v2.2) |
612
+ | `SYNTHESIS_STRATEGY` | coverage | Synthesis strategy (coverage=union of distinct points) |
630
613
  | `ENABLE_SEMANTIC_CACHE` | true | Semantic response caching (v2.3) |
631
614
  | `CACHE_TTL_HOURS` | 24 | Cache expiration in hours (v2.3) |
632
615
  | `ENABLE_RESPONSE_LIMITS` | false | Response token limits (v2.3, opt-in) |
633
616
  | `ENABLE_COST_AWARE_ROUTING` | false | Cost-based model routing (v2.3) |
634
- | `ENABLE_DEBATE_OPTIMIZATION` | true | Skip debate if all agree; SECURITY and ARCHITECTURE remain mandatory (default since v2.13) |
635
617
  | `ENABLE_COMPACT_REPORT` | true | Compact report format (v2.3) |
636
- | `ENABLE_CAPABILITY_WEIGHTING` | false | Capability-weighted voting (v2.20, opt-in) |
637
- | `ENABLE_CAPABILITY_ROUTING` | false | Capability-aware panel composition (v2.20, opt-in) |
638
- | `CAPABILITY_WEIGHT_STRENGTH` | 10 | Vote-weight modulation: conf*(S+cap)/S (v2.20) |
639
- | `CAPABILITY_DEFAULT` | 5 | Fallback capability for a missing consultant/axis (v2.20) |
640
- | `ENABLE_STANCE_CONSENSUS` | false | Enumerated-stance exact-match consensus (v2.21, opt-in; +1 LLM call/run) |
641
- | `STANCE_MAX_OPTIONS` | 5 | Max stance options generated per question (v2.21) |
642
618
  | `ENABLE_BUDGET_LIMIT` | false | Budget enforcement (v2.4, opt-in) |
643
619
  | `BUDGET_ACTION` | warn | Action on budget exceeded: warn/stop (v2.4) |
644
620
  | `QWEN3_USE_API` | false | Use DashScope API instead of qwen CLI (v2.7) |
@@ -801,6 +777,18 @@ curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/s
801
777
 
802
778
  ## Changelog
803
779
 
780
+ ### v3.0.0
781
+ - **Phase 2 of the panel-premise investigation: the deliberation/consensus machinery is removed and the default is now the coverage union.** Two independent reviews (Codex `gpt-5.6-sol`, then Fable-5) had judged the panel premise weak — voting/consensus measure *agreement*, not correctness, and no held-out comparison against one strong model existed. A held-out A/W/C coverage experiment (maintainer instrumentation on the `experiment/coverage-metric` branch, not shipped) resolved it: **task-dependent.** On convergent single-answer defect-finding a single strong model (Gemini 3.1 Pro) caught **19/19** hard bugs (repo-internal + external CVEs/concurrency incl. the Rust `Arc` bug found by RustBelt formal verification) → the panel has no coverage headroom. On **breadth/enumeration** (deep 60-point rubrics across JWT/webhook/cache), rubric coverage was **A 51% · self-consistency C 70% · panel W 93%, W>C>A on every item** → diversity, not volume, and specifically the raw **union** (deliberation OFF). The durable value is the diverse fan-out + union; the averaging machinery adds nothing measurable. Directional (n=3 breadth, hand-validated; n=19 defect).
782
+ - **Removed end to end** (six test-green commits on `refactor/remove-deliberation`, merged via PR #6): `lib/voting.sh` (voted recommendation, 1-10 score, consensus score, dissenters), `lib/orchestration.sh` (shape planner + convergence loop) + `debate_round.sh`, `lib/stance.sh`, `peer_review.sh`, and the capability-calibration cluster `roster_audit.sh`/`roster_calibrate.sh`/`run_calibration.sh`/`taste_elo.sh` + `references/calibration_benchmark.json` + the `capabilities`/`category_axis` blocks in `affinity.json`. Panic functions cut from `common.sh`; capability functions (`get_capability`/`get_category_axis`) and the `ENABLE_CAPABILITY_ROUTING` branch cut from `routing.sh`. Config: `ENABLE_DEBATE`/`DEBATE_ROUNDS`, `ORCHESTRATION_MODE`/`CONVERGENCE_*`, `ENABLE_DEBATE_OPTIMIZATION`/`DEBATE_*`, `ENABLE_PANIC_MODE`/`PANIC_*`, `ENABLE_ADVERSARIAL_VERIFY`, `ENABLE_PEER_REVIEW`/`PEER_REVIEW_MIN`, `ENABLE_CAPABILITY_*`/`CAPABILITY_*`, `ENABLE_STANCE_CONSENSUS`/`STANCE_*`. Presets no longer force debate/peer-review. The `debate` and `roster-audit` slash commands are gone (installers prune them). 19 files deleted, ~-4600 lines. 6 test suites removed.
783
+ - **`consult_all.sh` pipeline** is now classify → route → parallel fan-out → synthesis (the orchestration-planning, panic, Round-2+ deliberation, and voting/consensus blocks are gone; `optimization_metrics.json` and the report no longer emit consensus/orchestration/capability fields). `QUERY_COMPLEXITY` is retained (from `costs.sh`) for opt-in cost-aware routing. `KNOWN_FEATURE_FLAGS` synced (27 → 19).
784
+ - **`synthesize.sh`** default strategy is now `coverage` (alias `union`): the deduplicated union of every distinct point, crediting what only one model raised; it does NOT collapse to a winner. `majority` reworded (no consensus language); `compare_only`/`risk_averse`/`security_first`/`cost_capped` kept. `DEFAULT_STRATEGY` default `majority` → `coverage`. `doctor --suggest-preset` now recommends `coverage`.
785
+ - **`npm test` = 15 suites** (was 22), hermetic (`AI_CONSULTANTS_CONFIG_DIR=<empty>`). **Release-gate note (recurring):** `scripts/release.sh` runs `npm test` against the maintainer's real `~/.config` (it sources `config.sh` → `load_user_config` first), and under the real config `test_context_optimization.sh` hung the gate; the tree is green under the documented hermetic condition (`AI_CONSULTANTS_CONFIG_DIR=<empty> npm test </dev/null`). Same hermeticity class as the v2.23.0/v2.25.1 gate incidents.
786
+ - **Docs swept** (README/SKILL/CLAUDE.md/consult.md/references/docs) to the coverage-first framing; the reference-doc sweep was delegated and verified. CLAUDE.md's historical `## v2.X Features` sections are left as archive — this entry recontextualizes them. **Not statistically binding**: breadth is n=3; a binding claim needs n≥15–20 deep-rubric items (the harness exists on `experiment/coverage-metric`).
787
+
788
+ ### v2.25.2
789
+ - **`doctor`'s static ✓ was a false all-clear.** `check_cli_consultant` verifies `command -v` (installed) or key-present; neither confirms the consultant answers. So `✓ All systems healthy!` and JSON `status: "healthy"` were printed on installation state alone — a consultant with an expired key or spent quota passes every static check and fails the first real query. Surfaced concretely: static doctor called all 11 healthy; `doctor --live` showed 8/11 (Cursor usage limit, Qwen3 401, Grok bad xAI key returned as HTTP 400). Fix gates the word "healthy" on `LIVE_MODE`: a static-only clean run now prints "Static checks passed: CLIs installed, keys present … verify with doctor --live", and JSON reports `status: "static_ok"` + a new `verified: live|static` field; "healthy" now implies a live check backed it. `test_doctor.sh` asserts the invariant directly (a static-only run must never report `healthy`) rather than pinning a specific status, so it holds regardless of the test host's own consultant health. v2.18.0 added `--live` but left the default over-claiming; this closes that. `npm test` = 22 suites.
790
+ - **Note (not shipped, local only)**: the 3 down consultants were all credentials, not code — Grok/Qwen3 keys refreshed and Cursor disabled in the maintainer's own `~/.config`, taking the local live panel to 10/10 (Cursor off by choice). Qwen locally points at Token Plan `qwen3.8-max-preview`; the shipped default stays `qwen3.7-max`.
791
+
804
792
  ### v2.25.1
805
793
  - **Six defects found by an independent review of the released v2.25.0** (Codex `gpt-5.6-sol`, effort high, read-only via the `sol-reviewer` profile). Five were reproduced locally by executing the shipped functions before accepting them; the bash 3.2 crash was verified by the reviewer against `/bin/bash 3.2.57`.
806
794
  - **`calculate_final_score` scored every consultant as a dissenter.** `calculate_weighted_recommendation` emitted `_map_sanitize_key`'s output (spaces and punctuation stripped) as the winner, while `calculate_final_score` compared the raw `.response.approach` against it. Unanimous multiword panels scored 2/10, single-word panels 10/10. Voting now keeps original strings in bash 3.2-compatible indexed arrays. **The shipped voting tests used single-word approaches, which is exactly why this survived every prior review** — when writing a voting fixture, use a realistic multiword approach.
package/README.md CHANGED
@@ -1,8 +1,8 @@
1
- # AI Consultants v2.25.1
1
+ # AI Consultants v3.0.0
2
2
 
3
- > **A harness for every question.** A panel of up to 11 frontier models that writes its own playbook per question — fan out, debate to convergence, cross-examine under adversarial review, or run a tournament — and checks its work before it reaches you.
3
+ > **Coverage, not a single guess.** A panel of up to 11 frontier models from different vendors fans out on your question in parallel and hands you the *union* of what they collectively see — the risks, edge cases, and approaches a single model misses.
4
4
 
5
- [![Version](https://img.shields.io/badge/version-2.25.1-blue.svg)](https://github.com/matteoscurati/ai-consultants)
5
+ [![Version](https://img.shields.io/badge/version-3.0.0-blue.svg)](https://github.com/matteoscurati/ai-consultants)
6
6
  [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
7
7
  [![Claude Code Skill](https://img.shields.io/badge/Claude%20Code-Skill-orange.svg)](https://docs.anthropic.com/en/docs/claude-code/skills)
8
8
  [![GitHub stars](https://img.shields.io/github/stars/matteoscurati/ai-consultants?style=social)](https://github.com/matteoscurati/ai-consultants)
@@ -35,15 +35,13 @@
35
35
 
36
36
  ## Why AI Consultants?
37
37
 
38
- A single model gives you a single guess. AI Consultants gives you a **panel that deliberates — and adapts how it deliberates to the question in front of it.**
38
+ A single model gives you a single guess — and it misses whatever falls in its blind spots. AI Consultants fans your question out to a **panel of models from different vendors** and returns the **union of their distinct answers**: the point one model raised and the others didn't is exactly the value.
39
39
 
40
- Instead of running a fixed script, it classifies your question, picks an orchestration **shape**, and iterates until the answers hold up — the way a workflow builds a harness for the task at hand:
41
-
42
- - **Dynamic orchestration** — the engine chooses the strategy per question: a quick read, a convergence loop, an adversarial refutation gate, a tournament of approaches, or an exhaustive sweep
43
- - **Convergence, not fixed rounds** — debate iterates until the panel actually agrees (or provably won't), instead of a hardcoded count
44
- - **Self-checking** — security answers are stress-tested by consultants trying to refute them before anything reaches you
45
- - **11 supported consultants** with distinct personas (Architect, Pragmatist, Devil's Advocate, …)
46
- - **Confidence-weighted synthesis** — one recommendation, with the dissent and the path it took surfaced so you know how much to trust it
40
+ - **Cross-vendor diversity → coverage** — different model families have different blind spots, so the union covers what any one misses. On open-ended questions ("what could go wrong with this design?", "enumerate the risks") a diverse panel covers materially more of the answer space than one strong model — or than sampling one model repeatedly.
41
+ - **Parallel fan-out** — every consultant runs at once; no serial deliberation rounds.
42
+ - **Coverage synthesis** — the default synthesis is the deduplicated *union* of every distinct point, not a single voted winner (use `--strategy compare_only` for a side-by-side, or `majority` for a blended recommendation).
43
+ - **11 supported consultants** with distinct personas (Architect, Pragmatist, Devil's Advocate, …) — the personas deliberately decorrelate the panel.
44
+ - **Best for breadth** — threat-modeling, design review, "what am I missing?", exhaustive enumeration. For a single-answer factual or defect-finding question, one strong model is usually enough.
47
45
 
48
46
  ---
49
47
 
@@ -216,7 +214,6 @@ curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/s
216
214
  | Command | Description |
217
215
  |---------|-------------|
218
216
  | `/ai-consultants:consult` | Main consultation - ask AI consultants a coding question |
219
- | `/ai-consultants:debate` | Run consultation with multi-round debate |
220
217
  | `/ai-consultants:help` | Show all commands and usage |
221
218
 
222
219
  Configuration (presets, strategies, features, personas, API keys) can be managed via natural language — just ask.
@@ -342,8 +339,8 @@ cd ai-consultants
342
339
  # With preset
343
340
  ./scripts/consult_all.sh --preset balanced "Redis or Memcached?"
344
341
 
345
- # With debate
346
- ENABLE_DEBATE=true DEBATE_ROUNDS=2 ./scripts/consult_all.sh "Microservices vs monolith?"
342
+ # Side-by-side instead of the coverage union
343
+ ./scripts/consult_all.sh --strategy compare_only "Microservices vs monolith?"
347
344
 
348
345
  # With smart routing
349
346
  ENABLE_SMART_ROUTING=true ./scripts/consult_all.sh "Bug in auth code"
@@ -434,13 +431,8 @@ Choose the right balance of quality, speed, and cost with model quality tiers.
434
431
  | `medium` | Standard | 4 | General questions |
435
432
  | `fast` | Economy | 2 | Quick checks |
436
433
 
437
- **Deliberation depth is not fixed by the preset.** Under the default
438
- `ORCHESTRATION_MODE=auto` the planner picks the shape per question, so the
439
- `DEBATE_ROUNDS` a preset pins applies only under `ORCHESTRATION_MODE=fixed`,
440
- and a `SECURITY`-classified question runs the adversarial shape — including a
441
- peer-review round — whatever preset you chose. `max_quality` additionally
442
- enables peer review outright; `medium` and `fast` do not *clear* it, so an
443
- enabled peer review carries into them.
434
+ A preset only chooses the **consultant set and model tier** — every run then fans out in
435
+ parallel and returns the coverage union.
444
436
 
445
437
  ### Models by Tier
446
438
 
@@ -487,14 +479,14 @@ Choose how many consultants to use:
487
479
 
488
480
  | Preset | Consultants | Tier | Use Case |
489
481
  |--------|-------------|------|----------|
490
- | `max_quality` | 8 of 11 (+ peer review) | Premium | Critical decisions |
482
+ | `max_quality` | 8 of 11 | Premium | Critical decisions |
491
483
  | `medium` | 4 | Standard | General questions |
492
484
  | `fast` | 2 | Economy | Quick checks |
493
485
  | `minimal` | 2 (Gemini + Codex) | Default | Quick questions, low cost |
494
486
  | `balanced` | 4 (+ Mistral + Cursor) | Default | Standard consultations |
495
487
  | `thorough` | 4 | Default | Comprehensive analysis |
496
- | `high-stakes` | Expanded panel + debate | Default | Critical decisions |
497
- | `security` | Security-focused + debate | Default | Security reviews |
488
+ | `high-stakes` | Expanded panel (5 of 11) | Default | Critical decisions |
489
+ | `security` | Security-focused (4) | Default | Security reviews |
498
490
  | `cost-capped` | Budget-conscious | Default | Minimal API costs |
499
491
 
500
492
  **Bash:**
@@ -533,8 +525,8 @@ ai-consultants configure --interactive
533
525
  # Set any persistent parameter without opening an editor
534
526
  ai-consultants configure \
535
527
  --set DEFAULT_PRESET=balanced \
536
- --set ENABLE_DEBATE=true \
537
- --set ORCHESTRATION_MODE=converge
528
+ --set DEFAULT_STRATEGY=coverage \
529
+ --set ENABLE_SMART_ROUTING=true
538
530
 
539
531
  # Inspect the complete machine-readable parameter surface
540
532
  ai-consultants configure --show-parameters
@@ -561,8 +553,8 @@ For ad-hoc overrides without persisting, the most common knobs:
561
553
 
562
554
  ```bash
563
555
  DEFAULT_PRESET=balanced # minimal | balanced | thorough | high-stakes | fast | security
564
- DEFAULT_STRATEGY=majority # majority | risk_averse | security_first | cost_capped | compare_only
565
- ENABLE_DEBATE=true # Multi-agent debate (auto-skipped when consensus is high since v2.13)
556
+ DEFAULT_STRATEGY=coverage # coverage | compare_only | majority | risk_averse | security_first | cost_capped
557
+ ENABLE_SMART_ROUTING=true # Auto-select consultants by question category
566
558
  MAX_SESSION_COST=1.00 # USD budget cap (paired with ENABLE_BUDGET_LIMIT=true to enforce)
567
559
  KIMI_MODEL=kimi-code/k3 # Pin the Kimi consultant to K3
568
560
  ```
@@ -571,35 +563,25 @@ Full reference: [`references/configuration.md`](references/configuration.md). Co
571
563
 
572
564
  ### Configuration Recipes
573
565
 
574
- **Dynamic debate until convergence:**
566
+ **Breadth review — the coverage union (default):**
575
567
 
576
568
  ```bash
577
- ENABLE_DEBATE=true \
578
- ORCHESTRATION_MODE=converge \
579
- CONVERGENCE_MAX_ROUNDS=4 \
580
- CONVERGENCE_TARGET_CONSENSUS=75 \
581
- ai-consultants --strategy risk_averse \
582
- "Event log or mutable relational state for this service?"
569
+ ai-consultants --preset high-stakes \
570
+ "What could go wrong with this webhook-delivery design?" src/webhooks.ts@PRIMARY
583
571
  ```
584
572
 
585
- **Repeatable two-round review:**
573
+ **Security review, security-first framing:**
586
574
 
587
575
  ```bash
588
- ENABLE_DEBATE=true \
589
- ORCHESTRATION_MODE=fixed \
590
- DEBATE_ROUNDS=2 \
591
- ENABLE_DEBATE_OPTIMIZATION=false \
592
- ai-consultants "Review this migration plan" docs/migration.md@PRIMARY
576
+ ai-consultants --preset security --strategy security_first \
577
+ "Find authentication bypasses" src/auth.ts@PRIMARY
593
578
  ```
594
579
 
595
- **Security review with adversarial verification:**
580
+ **Side-by-side comparison (no synthesized union):**
596
581
 
597
582
  ```bash
598
- ENABLE_DEBATE=true \
599
- ORCHESTRATION_MODE=adversarial \
600
- ENABLE_PEER_REVIEW=true \
601
- ai-consultants --preset security --strategy security_first \
602
- "Find authentication bypasses" src/auth.ts@PRIMARY
583
+ ai-consultants --strategy compare_only \
584
+ "Event log or mutable relational state for this service?"
603
585
  ```
604
586
 
605
587
  **Live health gate plus hard quorum:**
@@ -611,9 +593,8 @@ QUORUM_ACTION=stop \
611
593
  ai-consultants "Make a release recommendation"
612
594
  ```
613
595
 
614
- See [13 complete recipes](docs/RECIPES.md) for fast, high-stakes, tournament,
615
- exhaustive audit, budget-capped, CLI-only, hybrid API, semantic-consensus, and
616
- large-context workflows.
596
+ See the [recipes](docs/RECIPES.md) for balanced, fast, security, compare-only,
597
+ budget-capped, CLI-only, hybrid API, and large-context workflows.
617
598
 
618
599
  ### Doctor Command
619
600
 
@@ -637,35 +618,18 @@ ai-consultants update-clis --dry-run # Preview: each C
637
618
  ## How It Works
638
619
 
639
620
  ```
640
- Classify -> Plan shape -> Fan out -> Deliberate & converge -> Synthesize
641
- | | | |
642
- category orchestration Gemini (8) convergence loop /
643
- complexity shape Codex (7) adversarial gate /
644
- intent Mistral (6) tournament / exhaustive
645
- Cursor (9)
621
+ Classify -> Route -> Fan out (parallel) -> Coverage synthesis
622
+ | | | |
623
+ category smart Gemini (8) union of every
624
+ routing Codex (7) distinct point,
625
+ Mistral (6) deduplicated
626
+ Cursor (9)
646
627
  ```
647
628
 
648
- The **shape** is chosen per question (or pinned via `ORCHESTRATION_MODE`):
649
-
650
- | Shape | When | What happens |
651
- |-------|------|--------------|
652
- | `quick` | simple questions | one fan-out, no debate |
653
- | `converge` | most questions | debate rounds until consensus is reached (not a fixed count) |
654
- | `adversarial` | security | a forced critique round + peer-review refutation gate |
655
- | `tournament` | "compare X vs Y" | converge, then declare a single winning approach |
656
- | `exhaustive` | "find all / audit" | loop until a round surfaces no new angle |
657
-
658
- Set `ORCHESTRATION_MODE=fixed` for the classic fixed-round pipeline.
659
-
660
- Consensus is measured lexically by default; set `ENABLE_STANCE_CONSENSUS=true`
661
- (opt-in, v2.21) to have the panel pick from a shared set of enumerated stance
662
- options and score agreement by exact match instead — see
663
- [references/configuration.md](references/configuration.md#semantic-consensus-v221-opt-in).
664
-
665
- With peer review:
666
- ```
667
- Responses -> Anonymize -> Peer Ranking -> De-anonymize -> Peer Scores
668
- ```
629
+ 1. **Classify** the question into a category.
630
+ 2. **Route** (optional, `ENABLE_SMART_ROUTING`) to the consultants with the best category affinity (`references/affinity.json`).
631
+ 3. **Fan out** to every selected consultant in parallel — one shot each, no serial rounds.
632
+ 4. **Synthesize** the **coverage union**: the deduplicated set of every distinct point, recommendation, risk, and edge case raised by any consultant. Override with `--strategy compare_only` (side-by-side) or `majority` (a single blended recommendation).
669
633
 
670
634
  ### Output
671
635
 
@@ -677,31 +641,34 @@ Each consultation generates:
677
641
  ├── codex.json # with confidence scores
678
642
  ├── mistral.json
679
643
  ├── cursor.json
680
- ├── voting.json # Consensus calculation
681
- ├── synthesis.json # Weighted recommendation
682
- ├── report.md # Human-readable report
683
- └── round_2/ # (if debate enabled)
644
+ ├── synthesis.json # Coverage union
645
+ └── report.md # Human-readable report
684
646
  ```
685
647
 
686
648
  ---
687
649
 
688
650
  ## Best Practices
689
651
 
690
- ### When to Use High-Stakes Mode
652
+ ### When the Panel Helps Most
653
+
654
+ The panel's edge is **coverage** — surfacing what a single model misses. It pays off on
655
+ open-ended, breadth questions:
691
656
 
692
- - Architectural decisions affecting system design
693
- - Security-critical code changes
694
- - Performance-critical optimizations
657
+ - Threat-modeling and design review ("what could go wrong with this?")
658
+ - Enumerating risks, edge cases, or failure modes
659
+ - "What am I missing?" / exhaustive audits
695
660
  - Decisions that are difficult to reverse
696
661
 
662
+ For a single-answer factual or defect-finding question, a single strong model is usually
663
+ enough — the panel adds little.
664
+
697
665
  ### Interpreting Results
698
666
 
699
667
  | Scenario | Recommendation |
700
668
  |----------|----------------|
701
- | High confidence + High consensus | Proceed with confidence |
702
- | Low confidence OR Low consensus | Consider more options |
703
- | Mistral (Devil's Advocate) disagrees | Investigate the risks |
704
- | Panic mode triggered | Add more consultants or debate rounds |
669
+ | A point only one consultant raised | Weigh it — the diversity is the point |
670
+ | Mistral (Devil's Advocate) flags a risk | Investigate it |
671
+ | Consultants diverge on approach | Use `--strategy compare_only` to see each side-by-side |
705
672
 
706
673
  ### Security
707
674
 
package/SKILL.md CHANGED
@@ -5,12 +5,12 @@ license: MIT
5
5
  compatibility: Requires bash, jq, and at least 2 AI CLI tools (agy, codex, vibe, etc.). macOS and Linux.
6
6
  metadata:
7
7
  author: matteoscurati
8
- version: 2.25.1
8
+ version: 3.0.0
9
9
  ---
10
10
 
11
- # AI Consultants v2.25.1 - AI Expert Panel
11
+ # AI Consultants v3.0.0 - AI Expert Panel
12
12
 
13
- **A harness for every question.** Convene a panel of AI "consultants" for coding questions — and let the engine pick *how* they deliberate: a quick read, a convergence loop, an adversarial refutation gate, a tournament, or an exhaustive sweep. Each consultant has a **configurable persona** that shapes its analysis.
13
+ **Coverage, not a single guess.** Convene a panel of AI "consultants" from different vendors for coding questions: they fan out in parallel and you get the *union* of what they collectively see — the risks, edge cases, and approaches a single model misses. Each consultant has a **configurable persona** that decorrelates its analysis.
14
14
 
15
15
  ## Quick Start
16
16
 
@@ -23,7 +23,6 @@ metadata:
23
23
  | Command | Description |
24
24
  |---------|-------------|
25
25
  | `/ai-consultants:consult` | Main consultation - ask AI consultants a coding question |
26
- | `/ai-consultants:debate` | Run consultation with multi-round debate |
27
26
  | `/ai-consultants:help` | Show all commands and usage |
28
27
 
29
28
  Configuration (presets, strategies, features, personas, API keys) can be managed via natural language — just ask.
@@ -71,29 +70,24 @@ For detailed CLI installation instructions, see [Setup Guide](docs/SETUP.md).
71
70
  ```
72
71
  /ai-consultants:consult "How to optimize this SQL query?"
73
72
  /ai-consultants:consult "Review this authentication flow" src/auth.ts
74
- /ai-consultants:debate "Microservices or monolith for our new service?"
73
+ /ai-consultants:consult "What could go wrong with this design?" src/service.ts
75
74
  ```
76
75
 
77
- Presets: `minimal`, `balanced`, `thorough`, `high-stakes`, `local`, `security`, `cost-capped`. Strategies: `majority`, `risk_averse`, `security_first`, `cost_capped`, `compare_only`. See [Reference Details](references/details.md) for full tables, bash usage, and best practices.
76
+ Presets: `minimal`, `balanced`, `thorough`, `high-stakes`, `security`, `cost-capped`, `max_quality`, `medium`, `fast`. Strategies: `coverage` (default; union of distinct points), `compare_only`, `majority`, `risk_averse`, `security_first`, `cost_capped`. See [Reference Details](references/details.md) for full tables, bash usage, and best practices.
78
77
 
79
78
  ## Workflow
80
79
 
81
- Query is classified, then sent to consultants in parallel. Responses are scored, voted on, and synthesized into a recommendation.
82
-
83
- **Dynamic orchestration (v2.16+):** a planner picks an orchestration *shape* per question — `quick` (simple), `converge` (debate until consensus, not a fixed round count), `adversarial` (SECURITY: forced critique + peer-review refutation gate), `tournament` (compare approaches → pick a winner), `exhaustive` (find-all: loop until no new angle). Set `ORCHESTRATION_MODE=fixed` for the legacy pipeline. See [configuration reference](references/configuration.md#dynamic-orchestration-v216).
80
+ The query is classified, optionally routed by category, then sent to all consultants in parallel. Their responses are synthesized into the **coverage union** — the deduplicated set of every distinct point, recommendation, and risk raised across the panel (not a single voted winner). Use `--strategy compare_only` for a side-by-side, or `majority` for a blended recommendation.
84
81
 
85
82
  ## Features
86
83
 
87
84
  | Feature | Description | Toggle |
88
85
  |---------|-------------|--------|
89
- | **Personas** | Each consultant has a role that shapes responses | `ENABLE_PERSONA` |
90
- | **Synthesis** | Auto-combine responses into recommendation | `ENABLE_SYNTHESIS` |
91
- | **Debate** | Consultants critique each other's answers | `ENABLE_DEBATE` |
92
- | **Peer Review** | Consultants anonymously rank each other | `ENABLE_PEER_REVIEW` |
93
- | **Smart Routing** | Auto-select best consultants per question type | `ENABLE_SMART_ROUTING` |
86
+ | **Personas** | Each consultant has a role that decorrelates responses | `ENABLE_PERSONA` |
87
+ | **Coverage synthesis** | Union of every distinct point across the panel (default) | `ENABLE_SYNTHESIS` |
88
+ | **Smart Routing** | Auto-select best consultants per question category | `ENABLE_SMART_ROUTING` |
94
89
  | **Cost Tracking** | Track API usage costs | `ENABLE_COST_TRACKING` |
95
- | **Panic Mode** | Auto-add rigor when uncertainty detected | `ENABLE_PANIC_MODE` |
96
- | **Stance Consensus** | Panel picks from enumerated stances; agreement by exact match (opt-in, v2.21) | `ENABLE_STANCE_CONSENSUS` |
90
+ | **Health Gate** | Ping and prune dead consultants before the run (opt-in) | `ENABLE_HEALTH_GATE` |
97
91
 
98
92
  ## Configuration
99
93
 
@@ -101,9 +95,8 @@ All settings use environment variables. Key toggles:
101
95
 
102
96
  ```bash
103
97
  ai-consultants configure # Auto-detect CLIs/API keys and persist all settings
104
- ENABLE_DEBATE=true # Multi-agent debate
105
- ENABLE_SYNTHESIS=true # Automatic synthesis
106
- ENABLE_PANIC_MODE=auto # Auto-rigor for uncertainty
98
+ ENABLE_SYNTHESIS=true # Coverage-union synthesis
99
+ ENABLE_SMART_ROUTING=true # Route by question category
107
100
  ENABLE_BUDGET_LIMIT=false # Budget enforcement (v2.4)
108
101
  ```
109
102
 
@@ -121,8 +114,7 @@ See [Full Configuration Reference](references/configuration.md) for all variable
121
114
  /tmp/ai_consultations/TIMESTAMP/
122
115
  ├── gemini.json # Individual responses
123
116
  ├── codex.json
124
- ├── voting.json # Consensus
125
- ├── synthesis.json # Recommendation
117
+ ├── synthesis.json # Coverage union
126
118
  └── report.md # Human-readable
127
119
  ```
128
120
 
@@ -147,10 +139,9 @@ When a consultant fails during a consultation, the run prints the captured reaso
147
139
 
148
140
  | Scenario | Recommendation |
149
141
  |----------|----------------|
150
- | High confidence + High consensus | Proceed confidently |
151
- | Low confidence OR Low consensus | Consider more options |
152
- | Mistral disagrees | Investigate risks |
153
- | Panic mode triggered | Add debate rounds |
142
+ | A point only one consultant raised | Weigh it — the diversity is the point |
143
+ | Mistral (Devil's Advocate) flags a risk | Investigate it |
144
+ | Consultants diverge on approach | Use `--strategy compare_only` to see each side-by-side |
154
145
 
155
146
  ## Troubleshooting
156
147
 
@@ -160,7 +151,7 @@ Run `./scripts/doctor.sh` to diagnose issues, or `./scripts/doctor.sh --fix` to
160
151
 
161
152
  - [Reference Details](references/details.md) - Presets, strategies, best practices, limitations
162
153
  - [Full Configuration](references/configuration.md) - All environment variables, models, tiers, timeouts
163
- - [Configuration Recipes](docs/RECIPES.md) - Copy-paste workflows for debate, routing, budgets, and transport
154
+ - [Configuration Recipes](docs/RECIPES.md) - Copy-paste workflows for routing, budgets, and transport
164
155
  - [Setup Guide](docs/SETUP.md) - Installation, authentication
165
156
  - [Cost Rates](docs/COST_RATES.md) - Model pricing
166
157
  - [Smart Routing](docs/SMART_ROUTING.md) - Category routing