ai-consultants 2.25.2 → 3.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/ai-consultants:consult.md +13 -19
- package/.claude/commands/ai-consultants:help.md +0 -2
- package/.env.example +14 -52
- package/CLAUDE.md +41 -38
- package/README.md +65 -92
- package/SKILL.md +20 -28
- package/docs/COST_RATES.md +1 -1
- package/docs/RECIPES.md +21 -100
- package/docs/SETUP.md +15 -39
- package/docs/SMART_ROUTING.md +0 -114
- package/docs/cost_rates.json +1 -1
- package/docs/releases/v3.0.0.md +71 -0
- package/docs/releases/v3.1.0.md +95 -0
- package/docs/releases/v3.1.1.md +54 -0
- package/package.json +2 -1
- package/references/affinity.json +110 -56
- package/references/configuration.md +13 -100
- package/references/details.md +11 -5
- package/scripts/config.sh +31 -114
- package/scripts/configure.sh +4 -4
- package/scripts/consult_all.sh +2 -165
- package/scripts/doctor.sh +26 -37
- package/scripts/lib/api_query.sh +0 -9
- package/scripts/lib/common.sh +14 -130
- package/scripts/lib/personas.sh +1 -8
- package/scripts/lib/routing.sh +3 -75
- package/scripts/query_grok.sh +214 -5
- package/scripts/synthesize.sh +7 -10
- package/scripts/update_clis.sh +3 -1
- package/.claude/commands/ai-consultants:debate.md +0 -102
- package/.claude/commands/ai-consultants:roster-audit.md +0 -38
- package/references/calibration_benchmark.json +0 -65
- package/scripts/debate_round.sh +0 -409
- package/scripts/lib/orchestration.sh +0 -402
- package/scripts/lib/stance.sh +0 -70
- package/scripts/lib/voting.sh +0 -647
- package/scripts/peer_review.sh +0 -471
- package/scripts/roster_audit.sh +0 -157
- package/scripts/roster_calibrate.sh +0 -238
- package/scripts/run_calibration.sh +0 -76
- package/scripts/taste_elo.sh +0 -171
|
@@ -42,34 +42,29 @@ Query multiple AI models for expert opinions on a coding question.
|
|
|
42
42
|
- Read `<output_dir>/synthesis.json` using the **Read** tool (may not exist if synthesis was skipped or failed)
|
|
43
43
|
- If neither exists, use the **Glob** tool with pattern `<output_dir>/*.json` to discover individual consultant response files, then read each one
|
|
44
44
|
|
|
45
|
-
5. **Present the results** using this template
|
|
45
|
+
5. **Present the results** using this template. The panel's value is *coverage* — the
|
|
46
|
+
union of distinct points across diverse models, including what only one raised — not a
|
|
47
|
+
single voted winner:
|
|
46
48
|
|
|
47
|
-
###
|
|
48
|
-
-
|
|
49
|
-
- **Category**: <question_category>
|
|
49
|
+
### Category
|
|
50
|
+
- <question_category>
|
|
50
51
|
|
|
51
52
|
### Consultant Summary
|
|
52
53
|
| Consultant | Confidence | Approach | Key Insight |
|
|
53
54
|
|------------|-----------|----------|-------------|
|
|
54
55
|
| <name> | <score>/10 | <approach> | <one-line from summary> |
|
|
55
56
|
|
|
56
|
-
###
|
|
57
|
-
-
|
|
58
|
-
|
|
57
|
+
### Coverage (the union of distinct considerations)
|
|
58
|
+
- <every distinct point, recommendation, risk, and edge case raised by ANY consultant,
|
|
59
|
+
deduplicated but preserving points only one model raised; note who raised each>
|
|
59
60
|
|
|
60
|
-
###
|
|
61
|
-
- <
|
|
62
|
-
|
|
63
|
-
### Key Disagreements
|
|
64
|
-
- <points where consultants diverge>
|
|
65
|
-
|
|
66
|
-
### Risk Assessment
|
|
67
|
-
- <caveats and uncertainty factors raised>
|
|
61
|
+
### Risks & Caveats
|
|
62
|
+
- <distinct caveats and uncertainty factors raised across the panel>
|
|
68
63
|
|
|
69
64
|
### Suggested Next Steps
|
|
70
|
-
- <actionable items
|
|
65
|
+
- <actionable items drawn from the coverage above>
|
|
71
66
|
|
|
72
|
-
If `synthesis.json` is missing, build the summary table and
|
|
67
|
+
If `synthesis.json` is missing, build the summary table and coverage directly from the individual consultant JSON files in the output directory (take the union of their distinct points).
|
|
73
68
|
|
|
74
69
|
6. **Error recovery**:
|
|
75
70
|
- **Exit code 1** (general failure): Tell the user and suggest running diagnostics:
|
|
@@ -84,9 +79,8 @@ Query multiple AI models for expert opinions on a coding question.
|
|
|
84
79
|
|
|
85
80
|
| Variable | Effect |
|
|
86
81
|
|----------|--------|
|
|
87
|
-
| `ENABLE_DEBATE=true` | Enable multi-round debate for deeper analysis |
|
|
88
82
|
| `ENABLE_SMART_ROUTING=true` | Auto-select best consultants for the question type |
|
|
89
|
-
| `
|
|
83
|
+
| `SYNTHESIS_STRATEGY=compare_only` | Present each consultant side-by-side instead of the coverage union |
|
|
90
84
|
| `FORCE_PROJECT_TREE=true` | Include the project tree even for pointed categories (SECURITY, QUICK_SYNTAX, etc.) |
|
|
91
85
|
|
|
92
86
|
## Follow-up
|
|
@@ -12,8 +12,6 @@ All skill details (consultants, personas, presets, strategies, features, configu
|
|
|
12
12
|
| Command | Description |
|
|
13
13
|
|---------|-------------|
|
|
14
14
|
| `/ai-consultants:consult` | Main consultation — ask AI consultants a coding question |
|
|
15
|
-
| `/ai-consultants:debate` | Run consultation with multi-round debate |
|
|
16
|
-
| `/ai-consultants:roster-audit` | Audit which consultants add distinct value vs echo the panel |
|
|
17
15
|
| `/ai-consultants:help` | Show this help |
|
|
18
16
|
|
|
19
17
|
## Quick Examples
|
package/.env.example
CHANGED
|
@@ -38,7 +38,7 @@
|
|
|
38
38
|
# =============================================================================
|
|
39
39
|
# API-BASED CONSULTANT KEYS
|
|
40
40
|
# =============================================================================
|
|
41
|
-
# These are required for
|
|
41
|
+
# These are required for API transports and API-only consultants.
|
|
42
42
|
|
|
43
43
|
# Qwen3 (Alibaba DashScope) - The Analyst
|
|
44
44
|
# Get yours at: https://dashscope.console.aliyun.com/
|
|
@@ -48,7 +48,8 @@
|
|
|
48
48
|
# Get yours at: https://open.z.ai/
|
|
49
49
|
# GLM_API_KEY=your_zhipu_api_key
|
|
50
50
|
|
|
51
|
-
# Grok (
|
|
51
|
+
# Grok API fallback (the Grok Build CLI normally uses its own login)
|
|
52
|
+
# Official docs call this XAI_API_KEY; GROK_API_KEY remains the project setting.
|
|
52
53
|
# Get yours at: https://console.x.ai/
|
|
53
54
|
# GROK_API_KEY=your_xai_api_key
|
|
54
55
|
|
|
@@ -73,18 +74,18 @@ ENABLE_MISTRAL=true
|
|
|
73
74
|
ENABLE_CURSOR=true
|
|
74
75
|
ENABLE_KIMI=true
|
|
75
76
|
ENABLE_QWEN3=true
|
|
77
|
+
ENABLE_GROK=true
|
|
76
78
|
ENABLE_MINIMAX=true
|
|
77
79
|
ENABLE_CLAUDE=true
|
|
78
80
|
|
|
79
81
|
# API-only consultants (default: disabled - require API keys)
|
|
80
82
|
ENABLE_GLM=false
|
|
81
|
-
ENABLE_GROK=false
|
|
82
83
|
ENABLE_DEEPSEEK=false
|
|
83
84
|
|
|
84
85
|
# =============================================================================
|
|
85
86
|
# CLI/API MODE SWITCHING (v2.6+)
|
|
86
87
|
# =============================================================================
|
|
87
|
-
#
|
|
88
|
+
# 7 agents support switching: Gemini, Codex, Claude, Mistral, Qwen3, Grok, MiniMax
|
|
88
89
|
# When API mode is enabled, CLI mode is automatically disabled.
|
|
89
90
|
|
|
90
91
|
# GEMINI_USE_API is auto-resolved when left UNSET (v2.15.1): API mode if
|
|
@@ -160,8 +161,13 @@ QWEN3_CMD=qwen
|
|
|
160
161
|
# rejected by the provider with a 400, not silently ignored.
|
|
161
162
|
# QWEN3_REASONING_EFFORT=
|
|
162
163
|
|
|
164
|
+
# GROK_USE_API is auto-resolved CLI-first: Grok Build is used whenever `grok`
|
|
165
|
+
# is installed; the xAI API is selected when the CLI is unavailable and a key
|
|
166
|
+
# exists. A failed CLI launch also falls back to the API at runtime.
|
|
167
|
+
# GROK_USE_API=false
|
|
168
|
+
|
|
163
169
|
# =============================================================================
|
|
164
|
-
# API-
|
|
170
|
+
# API-ONLY AND FALLBACK CONFIGURATION
|
|
165
171
|
# =============================================================================
|
|
166
172
|
|
|
167
173
|
# GLM - The Methodologist
|
|
@@ -169,7 +175,8 @@ GLM_MODEL=glm-5.2
|
|
|
169
175
|
GLM_TIMEOUT=180
|
|
170
176
|
GLM_API_URL=https://api.z.ai/api/coding/paas/v4/chat/completions
|
|
171
177
|
|
|
172
|
-
# Grok - The Provocateur
|
|
178
|
+
# Grok Build CLI + xAI API fallback - The Provocateur
|
|
179
|
+
GROK_CMD=grok
|
|
173
180
|
GROK_MODEL=grok-4.5
|
|
174
181
|
GROK_TIMEOUT=180
|
|
175
182
|
GROK_API_URL=https://api.x.ai/v1/chat/completions
|
|
@@ -203,7 +210,7 @@ MINIMAX_FORMAT=openai
|
|
|
203
210
|
|
|
204
211
|
# Defaults used when --preset / --strategy are omitted
|
|
205
212
|
DEFAULT_PRESET=balanced
|
|
206
|
-
DEFAULT_STRATEGY=
|
|
213
|
+
DEFAULT_STRATEGY=coverage
|
|
207
214
|
|
|
208
215
|
# The invoking integration normally sets this per run for self-exclusion.
|
|
209
216
|
INVOKING_AGENT=unknown
|
|
@@ -219,34 +226,6 @@ SYNTHESIS_MODEL=
|
|
|
219
226
|
SYNTHESIS_EXTRACT_FIELDS=true
|
|
220
227
|
SYNTH_MAX=10
|
|
221
228
|
|
|
222
|
-
# Multi-Agent Debate (MAD): cross-critique rounds
|
|
223
|
-
ENABLE_DEBATE=false
|
|
224
|
-
DEBATE_ROUNDS=1 # 1 = no debate, 2-3 = with cross-critique
|
|
225
|
-
|
|
226
|
-
# Peer Review: anonymous evaluation
|
|
227
|
-
ENABLE_PEER_REVIEW=false
|
|
228
|
-
PEER_REVIEW_MIN_RESPONSES=3
|
|
229
|
-
REVIEWS_PER_CONSULTANT=3
|
|
230
|
-
|
|
231
|
-
# Panic Mode: auto-rigor when uncertainty detected
|
|
232
|
-
ENABLE_PANIC_MODE=auto
|
|
233
|
-
PANIC_CONFIDENCE_THRESHOLD=5
|
|
234
|
-
PANIC_EXTRA_DEBATE_ROUNDS=1
|
|
235
|
-
PANIC_KEYWORDS="uncertain|maybe|not sure|possibly|unclear|depends|hard to say|difficult to determine"
|
|
236
|
-
|
|
237
|
-
# Dynamic orchestration: choose how the panel deliberates
|
|
238
|
-
# auto | fixed | quick | converge | adversarial | tournament | exhaustive
|
|
239
|
-
ORCHESTRATION_MODE=auto
|
|
240
|
-
CONVERGENCE_MAX_ROUNDS=4
|
|
241
|
-
CONVERGENCE_TARGET_CONSENSUS=75
|
|
242
|
-
CONVERGENCE_STALL_EPSILON=5
|
|
243
|
-
ENABLE_ADVERSARIAL_VERIFY=true
|
|
244
|
-
|
|
245
|
-
# Debate optimization: avoid extra rounds when the panel already agrees
|
|
246
|
-
ENABLE_DEBATE_OPTIMIZATION=true
|
|
247
|
-
DEBATE_CONFIDENCE_SPREAD_THRESHOLD=2
|
|
248
|
-
DEBATE_USE_SUMMARIES=true
|
|
249
|
-
|
|
250
229
|
# Health gate and quorum: validate the live panel before spending on a full run
|
|
251
230
|
ENABLE_HEALTH_GATE=false # Adds one small ping per selected consultant
|
|
252
231
|
HEALTH_GATE_TIMEOUT=30
|
|
@@ -297,23 +276,6 @@ COMPLEXITY_THRESHOLD_SIMPLE=3
|
|
|
297
276
|
COMPLEXITY_THRESHOLD_MEDIUM=6
|
|
298
277
|
FALLBACK_CONFIDENCE_THRESHOLD=7
|
|
299
278
|
|
|
300
|
-
# Capability-aware routing & voting (v2.20, opt-in): weight votes and rank the
|
|
301
|
-
# panel by each consultant's capability on the quality axis a question stresses
|
|
302
|
-
# (intelligence or taste, per references/affinity.json 'category_axis'). cost is
|
|
303
|
-
# a composition/budget axis only, never a vote weight.
|
|
304
|
-
ENABLE_CAPABILITY_WEIGHTING=false # capability-weighted voting
|
|
305
|
-
ENABLE_CAPABILITY_ROUTING=false # capability-aware panel composition
|
|
306
|
-
CAPABILITY_WEIGHT_STRENGTH=10 # higher = gentler nudge: weight = conf*(S+cap)/S
|
|
307
|
-
CAPABILITY_DEFAULT=5 # fallback capability for a missing consultant/axis
|
|
308
|
-
|
|
309
|
-
# Stance-based semantic consensus (v2.21, opt-in): the panel picks from a shared
|
|
310
|
-
# enumerated stance set so agreement is scored by exact match instead of lexical
|
|
311
|
-
# similarity. Adds ~1 LLM call per run; degrades to the lexical cluster on any
|
|
312
|
-
# failure, so it is always safe to enable.
|
|
313
|
-
ENABLE_STANCE_CONSENSUS=false
|
|
314
|
-
STANCE_MAX_OPTIONS=5 # max stance options generated per question
|
|
315
|
-
STANCE_TIMEOUT=60 # seconds for the stance-generation call (guards a hang)
|
|
316
|
-
|
|
317
279
|
# =============================================================================
|
|
318
280
|
# COST MANAGEMENT
|
|
319
281
|
# =============================================================================
|
package/CLAUDE.md
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
## Project Overview
|
|
4
4
|
|
|
5
|
-
AI Consultants is a multi-model
|
|
5
|
+
AI Consultants is a multi-model coverage system that queries up to 11 AI consultants (Gemini, Codex, Mistral, Cursor, Kimi, Claude, Qwen3, GLM, Grok, DeepSeek, MiniMax) to obtain diverse perspectives on coding problems.
|
|
6
6
|
|
|
7
7
|
**Self-Exclusion**: The invoking agent is automatically excluded from the panel to prevent self-consultation. Claude Code won't query Claude, Codex CLI won't query Codex, etc.
|
|
8
8
|
|
|
9
|
-
**Version**:
|
|
9
|
+
**Version**: 3.1.1
|
|
10
10
|
|
|
11
11
|
## Distribution
|
|
12
12
|
|
|
@@ -39,27 +39,18 @@ ai-consultants/
|
|
|
39
39
|
│ ├── config.sh # Centralized configuration
|
|
40
40
|
│ ├── doctor.sh # Diagnostic and auto-fix tool (v2.2)
|
|
41
41
|
│ ├── update_clis.sh # Check/update installed consultant CLIs (v2.21)
|
|
42
|
-
│ ├── peer_review.sh # Anonymous peer review (v2.2)
|
|
43
|
-
│ ├── roster_audit.sh # Roster uncorrelated-value audit (v2.20)
|
|
44
|
-
│ ├── roster_calibrate.sh # Measured capability calibration, Tier A (v2.20)
|
|
45
|
-
│ ├── run_calibration.sh # Calibration data-collection harness (v2.20)
|
|
46
|
-
│ ├── taste_elo.sh # Pairwise-judge taste Elo, Tier B (v2.20)
|
|
47
42
|
│ ├── install.sh # One-liner installer (v2.2)
|
|
48
43
|
│ ├── query_*.sh # Wrapper for each consultant
|
|
49
44
|
│ ├── query_claude.sh # Claude consultant (v2.2)
|
|
50
|
-
│ ├── synthesize.sh #
|
|
51
|
-
│ ├── debate_round.sh # Multi-Agent Debate
|
|
45
|
+
│ ├── synthesize.sh # Coverage-union synthesis of responses
|
|
52
46
|
│ ├── classify_question.sh # Question classifier
|
|
53
47
|
│ ├── followup.sh # Follow-up queries
|
|
54
48
|
│ ├── preflight_check.sh # DEPRECATED v2.10.9 (thin wrapper -> doctor.sh)
|
|
55
49
|
│ └── lib/
|
|
56
|
-
│ ├── common.sh # Shared utilities (logging,
|
|
50
|
+
│ ├── common.sh # Shared utilities (logging, quorum grading)
|
|
57
51
|
│ ├── personas.sh # Consultant persona definitions
|
|
58
52
|
│ ├── schema.json # JSON output schema
|
|
59
|
-
│ ├──
|
|
60
|
-
│ ├── routing.sh # Smart routing + cost-aware routing
|
|
61
|
-
│ ├── orchestration.sh # Dynamic orchestration planner + shapes (v2.16)
|
|
62
|
-
│ ├── stance.sh # Stance-based semantic consensus (v2.21, opt-in)
|
|
53
|
+
│ ├── routing.sh # Smart routing (category affinity) + cost-aware routing
|
|
63
54
|
│ ├── session.sh # Session management
|
|
64
55
|
│ ├── costs.sh # Cost tracking + response limits
|
|
65
56
|
│ ├── cache.sh # Semantic caching (v2.3)
|
|
@@ -158,17 +149,13 @@ Each consultant must produce JSON with this minimum structure:
|
|
|
158
149
|
## Main Flow
|
|
159
150
|
|
|
160
151
|
1. `consult_all.sh` receives query, optional files, and flags (`--preset`, `--strategy`)
|
|
161
|
-
2. Applies preset if specified (`apply_preset()` in config.sh)
|
|
152
|
+
2. Applies preset if specified (`apply_preset()` in config.sh) — sets the consultant set + model tier
|
|
162
153
|
3. Classifies the question (`classify_question.sh`)
|
|
163
|
-
4.
|
|
164
|
-
5.
|
|
165
|
-
6.
|
|
166
|
-
7.
|
|
167
|
-
8.
|
|
168
|
-
9. Checks for panic mode triggers (`lib/common.sh`)
|
|
169
|
-
10. Generates synthesis with selected strategy (`synthesize.sh`)
|
|
170
|
-
11. Optionally runs peer review (`peer_review.sh`)
|
|
171
|
-
12. Produces final report
|
|
154
|
+
4. Selects consultants (smart routing by category affinity, or all)
|
|
155
|
+
5. Launches parallel queries (`query_*.sh`) — one shot per consultant, no serial rounds
|
|
156
|
+
6. Grades quorum (`grade_quorum()` in `lib/common.sh`) — surfaces a degraded panel
|
|
157
|
+
7. Generates synthesis with the selected strategy (`synthesize.sh`); the default `coverage` produces the deduplicated UNION of every distinct point across the panel
|
|
158
|
+
8. Produces final report
|
|
172
159
|
|
|
173
160
|
## v2.2 Features
|
|
174
161
|
|
|
@@ -432,6 +419,15 @@ All consultants now use premium models by default:
|
|
|
432
419
|
|
|
433
420
|
Override with environment variables: `CLAUDE_MODEL`, `GEMINI_MODEL`, `CODEX_MODEL`, `KIMI_MODEL`, etc.
|
|
434
421
|
|
|
422
|
+
Grok is CLI-first through the official Grok Build `grok` command, with
|
|
423
|
+
`grok-4.5` passed explicitly in headless mode. The prompt is delivered with
|
|
424
|
+
`--prompt-file`; HOME and CWD are isolated, built-in/MCP tools are denied, and
|
|
425
|
+
the strict sandbox prevents user or project extensions from joining the
|
|
426
|
+
consultation. The xAI Chat Completions path is retained only when the CLI is
|
|
427
|
+
missing, cannot launch, or lacks usable authentication and `GROK_API_KEY` is
|
|
428
|
+
available. A timeout, model error, empty response, or other post-launch failure
|
|
429
|
+
is surfaced and never silently rerouted to the API.
|
|
430
|
+
|
|
435
431
|
## v2.4 Features
|
|
436
432
|
|
|
437
433
|
### Budget Enforcement (Opt-in)
|
|
@@ -613,32 +609,21 @@ for f in scripts/*.sh scripts/lib/*.sh; do bash -n "$f" && echo "OK: $f"; done
|
|
|
613
609
|
|----------|---------|-------------|
|
|
614
610
|
| `INVOKING_AGENT` | unknown | Agent invoking the skill (for self-exclusion) |
|
|
615
611
|
| `ENABLE_CLAUDE` | true | Enable Claude consultant (auto-excluded under Claude Code) |
|
|
616
|
-
| `
|
|
617
|
-
| `
|
|
618
|
-
| `ENABLE_SYNTHESIS` | true | Auto-synthesis of responses |
|
|
619
|
-
| `ENABLE_SMART_ROUTING` | false | Intelligent routing |
|
|
612
|
+
| `ENABLE_SYNTHESIS` | true | Coverage-union synthesis of responses |
|
|
613
|
+
| `ENABLE_SMART_ROUTING` | false | Intelligent routing by category |
|
|
620
614
|
| `ENABLE_COST_TRACKING` | true | Track costs |
|
|
621
615
|
| `MAX_SESSION_COST` | 1.00 | Max budget ($) |
|
|
622
|
-
| `ENABLE_PANIC_MODE` | auto | Panic mode trigger (v2.2) |
|
|
623
|
-
| `PANIC_CONFIDENCE_THRESHOLD` | 5 | Panic threshold (v2.2) |
|
|
624
616
|
| `CLAUDE_MODEL` | claude-opus-4-8 | Claude model (v2.5) |
|
|
625
617
|
| `GEMINI_MODEL` | Gemini 3.1 Pro (High) | Gemini agy CLI model (v2.15) |
|
|
626
618
|
| `GEMINI_API_MODEL` | gemini-3.1-pro-preview | Gemini API-mode model ID (v2.15) |
|
|
627
619
|
| `CODEX_MODEL` | gpt-5.5 | Codex model (v2.5) |
|
|
628
620
|
| `MISTRAL_MODEL` | mistral-large-3 | Mistral model (v2.5) |
|
|
629
|
-
| `SYNTHESIS_STRATEGY` |
|
|
621
|
+
| `SYNTHESIS_STRATEGY` | coverage | Synthesis strategy (coverage=union of distinct points) |
|
|
630
622
|
| `ENABLE_SEMANTIC_CACHE` | true | Semantic response caching (v2.3) |
|
|
631
623
|
| `CACHE_TTL_HOURS` | 24 | Cache expiration in hours (v2.3) |
|
|
632
624
|
| `ENABLE_RESPONSE_LIMITS` | false | Response token limits (v2.3, opt-in) |
|
|
633
625
|
| `ENABLE_COST_AWARE_ROUTING` | false | Cost-based model routing (v2.3) |
|
|
634
|
-
| `ENABLE_DEBATE_OPTIMIZATION` | true | Skip debate if all agree; SECURITY and ARCHITECTURE remain mandatory (default since v2.13) |
|
|
635
626
|
| `ENABLE_COMPACT_REPORT` | true | Compact report format (v2.3) |
|
|
636
|
-
| `ENABLE_CAPABILITY_WEIGHTING` | false | Capability-weighted voting (v2.20, opt-in) |
|
|
637
|
-
| `ENABLE_CAPABILITY_ROUTING` | false | Capability-aware panel composition (v2.20, opt-in) |
|
|
638
|
-
| `CAPABILITY_WEIGHT_STRENGTH` | 10 | Vote-weight modulation: conf*(S+cap)/S (v2.20) |
|
|
639
|
-
| `CAPABILITY_DEFAULT` | 5 | Fallback capability for a missing consultant/axis (v2.20) |
|
|
640
|
-
| `ENABLE_STANCE_CONSENSUS` | false | Enumerated-stance exact-match consensus (v2.21, opt-in; +1 LLM call/run) |
|
|
641
|
-
| `STANCE_MAX_OPTIONS` | 5 | Max stance options generated per question (v2.21) |
|
|
642
627
|
| `ENABLE_BUDGET_LIMIT` | false | Budget enforcement (v2.4, opt-in) |
|
|
643
628
|
| `BUDGET_ACTION` | warn | Action on budget exceeded: warn/stop (v2.4) |
|
|
644
629
|
| `QWEN3_USE_API` | false | Use DashScope API instead of qwen CLI (v2.7) |
|
|
@@ -801,6 +786,24 @@ curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/s
|
|
|
801
786
|
|
|
802
787
|
## Changelog
|
|
803
788
|
|
|
789
|
+
### v3.1.1
|
|
790
|
+
- **`doctor` no longer aborts at the first missing enabled consultant.** The v3.1.0 tag workflow exposed the pre-existing `set -e` interaction when Grok became CLI-first and the clean Ubuntu runner had no `grok` binary: the main JSON diagnostic exited before emitting output. Missing CLI/API configuration checks now add their issue and continue to the final summary, which still exits unhealthy. `test_doctor.sh` stubs the complete CLI roster, rejects empty JSON output, and includes a dedicated missing-Grok regression. Full gate: 16/16 suites; shellcheck green.
|
|
791
|
+
|
|
792
|
+
### v3.1.0
|
|
793
|
+
- **Grok is now a first-class CLI/API-switchable consultant on `grok-4.5`.** `scripts/query_grok.sh` uses the official Grok Build headless interface, `scripts/config.sh` resolves CLI-first while preserving API-only installations, and configure/doctor/update-clis/docs all classify Grok with the other switchable consultants. `XAI_API_KEY` is accepted as the official alias for the historical `GROK_API_KEY` contract. The route that actually answered is recorded as `metadata.transport = cli|api|api_fallback`.
|
|
794
|
+
- **The CLI boundary is enforced rather than documented.** The first implementation disabled plan/subagents/memory/web but still inherited the current checkout and the maintainer's Grok environment; `grok inspect --json` showed 1 MCP server, 13 plugins, 11 hooks, 79 skills, and 3 project instructions available to an advisory consultation. Each run now creates an ephemeral mode-700 HOME/CWD, copies only a regular non-symlink `auth.json` at mode 600, supplies `--tools ""`, `--permission-mode dontAsk`, `--sandbox strict`, and explicit deny rules, then validates the temp-prefix before recursive cleanup. A live Grok 4.5 smoke succeeded through this isolated path.
|
|
795
|
+
- **Prompts use `--prompt-file`, not argv.** `FULL_QUERY` includes context files; passing it through `-p "$FULL_QUERY"` exposed the prompt in the process list and failed before Grok launched once the combined context crossed `ARG_MAX`. Reproduced locally with a 1.1 MiB context (`timeout: Argument list too long`). The private prompt file removes both failure modes, and the regression suite drives the same 1.1 MiB case.
|
|
796
|
+
- **API fallback means transport unavailable, not "any non-zero".** The initial adapter sent every CLI error to the API when a key existed — including timeouts, model errors, empty responses, and a generic tested `exit 42` — which hid CLI regressions and could add an unexpected API charge. `grok_cli_is_unavailable` now recognizes missing/unexecutable binaries and authentication/startup failures only; post-launch failures remain `transport=cli` and surface as errors. Missing-command and authentication fallback paths are tested separately.
|
|
797
|
+
- **Regression and release coverage.** New `test_query_grok.sh` carries 26 assertions across headless flags/model pinning, HOME/CWD/auth isolation, tool denial, transport metadata, API fallback boundaries, and large contexts. The master runner discovers it automatically, taking `npm test` to 16 suites; the v3.1.0 release gate and shellcheck pass. README, SKILL, setup/recipes/configuration references, changelog, and the showcase roster are synchronized.
|
|
798
|
+
|
|
799
|
+
### v3.0.0
|
|
800
|
+
- **Phase 2 of the panel-premise investigation: the deliberation/consensus machinery is removed and the default is now the coverage union.** Two independent reviews (Codex `gpt-5.6-sol`, then Fable-5) had judged the panel premise weak — voting/consensus measure *agreement*, not correctness, and no held-out comparison against one strong model existed. A held-out A/W/C coverage experiment (maintainer instrumentation on the `experiment/coverage-metric` branch, not shipped) resolved it: **task-dependent.** On convergent single-answer defect-finding a single strong model (Gemini 3.1 Pro) caught **19/19** hard bugs (repo-internal + external CVEs/concurrency incl. the Rust `Arc` bug found by RustBelt formal verification) → the panel has no coverage headroom. On **breadth/enumeration** (deep 60-point rubrics across JWT/webhook/cache), rubric coverage was **A 51% · self-consistency C 70% · panel W 93%, W>C>A on every item** → diversity, not volume, and specifically the raw **union** (deliberation OFF). The durable value is the diverse fan-out + union; the averaging machinery adds nothing measurable. Directional (n=3 breadth, hand-validated; n=19 defect).
|
|
801
|
+
- **Removed end to end** (six test-green commits on `refactor/remove-deliberation`, merged via PR #6): `lib/voting.sh` (voted recommendation, 1-10 score, consensus score, dissenters), `lib/orchestration.sh` (shape planner + convergence loop) + `debate_round.sh`, `lib/stance.sh`, `peer_review.sh`, and the capability-calibration cluster `roster_audit.sh`/`roster_calibrate.sh`/`run_calibration.sh`/`taste_elo.sh` + `references/calibration_benchmark.json` + the `capabilities`/`category_axis` blocks in `affinity.json`. Panic functions cut from `common.sh`; capability functions (`get_capability`/`get_category_axis`) and the `ENABLE_CAPABILITY_ROUTING` branch cut from `routing.sh`. Config: `ENABLE_DEBATE`/`DEBATE_ROUNDS`, `ORCHESTRATION_MODE`/`CONVERGENCE_*`, `ENABLE_DEBATE_OPTIMIZATION`/`DEBATE_*`, `ENABLE_PANIC_MODE`/`PANIC_*`, `ENABLE_ADVERSARIAL_VERIFY`, `ENABLE_PEER_REVIEW`/`PEER_REVIEW_MIN`, `ENABLE_CAPABILITY_*`/`CAPABILITY_*`, `ENABLE_STANCE_CONSENSUS`/`STANCE_*`. Presets no longer force debate/peer-review. The `debate` and `roster-audit` slash commands are gone (installers prune them). 19 files deleted, ~-4600 lines. 6 test suites removed.
|
|
802
|
+
- **`consult_all.sh` pipeline** is now classify → route → parallel fan-out → synthesis (the orchestration-planning, panic, Round-2+ deliberation, and voting/consensus blocks are gone; `optimization_metrics.json` and the report no longer emit consensus/orchestration/capability fields). `QUERY_COMPLEXITY` is retained (from `costs.sh`) for opt-in cost-aware routing. `KNOWN_FEATURE_FLAGS` synced (27 → 19).
|
|
803
|
+
- **`synthesize.sh`** default strategy is now `coverage` (alias `union`): the deduplicated union of every distinct point, crediting what only one model raised; it does NOT collapse to a winner. `majority` reworded (no consensus language); `compare_only`/`risk_averse`/`security_first`/`cost_capped` kept. `DEFAULT_STRATEGY` default `majority` → `coverage`. `doctor --suggest-preset` now recommends `coverage`.
|
|
804
|
+
- **`npm test` = 15 suites** (was 22), hermetic (`AI_CONSULTANTS_CONFIG_DIR=<empty>`). **Release-gate note (recurring):** `scripts/release.sh` runs `npm test` against the maintainer's real `~/.config` (it sources `config.sh` → `load_user_config` first), and under the real config `test_context_optimization.sh` hung the gate; the tree is green under the documented hermetic condition (`AI_CONSULTANTS_CONFIG_DIR=<empty> npm test </dev/null`). Same hermeticity class as the v2.23.0/v2.25.1 gate incidents.
|
|
805
|
+
- **Docs swept** (README/SKILL/CLAUDE.md/consult.md/references/docs) to the coverage-first framing; the reference-doc sweep was delegated and verified. CLAUDE.md's historical `## v2.X Features` sections are left as archive — this entry recontextualizes them. **Not statistically binding**: breadth is n=3; a binding claim needs n≥15–20 deep-rubric items (the harness exists on `experiment/coverage-metric`).
|
|
806
|
+
|
|
804
807
|
### v2.25.2
|
|
805
808
|
- **`doctor`'s static ✓ was a false all-clear.** `check_cli_consultant` verifies `command -v` (installed) or key-present; neither confirms the consultant answers. So `✓ All systems healthy!` and JSON `status: "healthy"` were printed on installation state alone — a consultant with an expired key or spent quota passes every static check and fails the first real query. Surfaced concretely: static doctor called all 11 healthy; `doctor --live` showed 8/11 (Cursor usage limit, Qwen3 401, Grok bad xAI key returned as HTTP 400). Fix gates the word "healthy" on `LIVE_MODE`: a static-only clean run now prints "Static checks passed: CLIs installed, keys present … verify with doctor --live", and JSON reports `status: "static_ok"` + a new `verified: live|static` field; "healthy" now implies a live check backed it. `test_doctor.sh` asserts the invariant directly (a static-only run must never report `healthy`) rather than pinning a specific status, so it holds regardless of the test host's own consultant health. v2.18.0 added `--live` but left the default over-claiming; this closes that. `npm test` = 22 suites.
|
|
806
809
|
- **Note (not shipped, local only)**: the 3 down consultants were all credentials, not code — Grok/Qwen3 keys refreshed and Cursor disabled in the maintainer's own `~/.config`, taking the local live panel to 10/10 (Cursor off by choice). Qwen locally points at Token Plan `qwen3.8-max-preview`; the shipped default stays `qwen3.7-max`.
|