ai-consultants 3.5.0 → 4.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/ai-consultants:consult.md +29 -9
- package/.claude/commands/ai-consultants:help.md +5 -1
- package/.env.example +26 -14
- package/CLAUDE.md +64 -75
- package/README.md +49 -29
- package/SKILL.md +64 -10
- package/bin/ai-consultants +4 -5
- package/docs/COST_RATES.md +17 -23
- package/docs/JSON_SCHEMA.md +7 -47
- package/docs/SETUP.md +18 -23
- package/docs/cost_rates.json +14 -18
- package/docs/releases/v3.5.0.md +2 -2
- package/docs/releases/v4.0.0.md +107 -0
- package/docs/releases/v4.0.1.md +57 -0
- package/package.json +5 -5
- package/references/affinity.json +0 -10
- package/references/configuration.md +54 -12
- package/references/details.md +6 -6
- package/scripts/build_context.sh +22 -9
- package/scripts/config.sh +207 -51
- package/scripts/configure.sh +10 -42
- package/scripts/consult_all.sh +161 -52
- package/scripts/doctor.sh +6 -25
- package/scripts/followup.sh +80 -15
- package/scripts/install.sh +3 -3
- package/scripts/lib/api_query.sh +63 -24
- package/scripts/lib/common.sh +144 -31
- package/scripts/lib/costs.sh +0 -6
- package/scripts/lib/personas.sh +7 -7
- package/scripts/lib/progress.sh +2 -2
- package/scripts/lib/routing.sh +1 -1
- package/scripts/lib/schema.json +7 -39
- package/scripts/query_claude.sh +57 -6
- package/scripts/query_codex.sh +8 -3
- package/scripts/query_gemini.sh +9 -4
- package/scripts/query_grok.sh +72 -20
- package/scripts/query_kimi.sh +8 -3
- package/scripts/query_minimax.sh +32 -8
- package/scripts/query_mistral.sh +25 -6
- package/scripts/query_qwen3.sh +8 -3
- package/scripts/synthesize.sh +112 -83
- package/scripts/update_clis.sh +0 -2
- package/templates/consultation_report.md +27 -265
- package/templates/synthesis_prompt.md +47 -87
- package/scripts/query_cursor.sh +0 -131
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
description: Consult AI experts (Gemini, Codex, Mistral,
|
|
2
|
+
description: Consult AI experts (Gemini, Codex, Mistral, Kimi, Qwen, GLM, Grok, DeepSeek, MiniMax) for coding questions. Use when weighing trade-offs, comparing approaches, or wanting multiple expert perspectives on non-trivial decisions.
|
|
3
3
|
argument-hint: <question> [file1] [file2] ...
|
|
4
4
|
allowed-tools: Bash Read Glob Grep
|
|
5
5
|
---
|
|
@@ -14,6 +14,11 @@ Query multiple AI models for expert opinions on a coding question.
|
|
|
14
14
|
|
|
15
15
|
1. If no question provided in $ARGUMENTS, ask the user what they want to consult about.
|
|
16
16
|
|
|
17
|
+
Separate supported consultation options from the question before running the
|
|
18
|
+
panel. In particular, preserve `--preset <name>` and `--strategy <name>` as
|
|
19
|
+
CLI arguments placed before the question; they are not part of the question
|
|
20
|
+
text. Reject an incomplete option instead of guessing its value.
|
|
21
|
+
|
|
17
22
|
2. **File context handling**: Identify which strings in $ARGUMENTS refer to files the user wants the consultants to look at. Use your judgment — verify existence via `Glob` or `Bash ls` when uncertain. This replaces fragile regex detection: a `Makefile`, `Dockerfile`, dotfile, or path without a common extension is still a file; an URL or regex pattern in the question text is not.
|
|
18
23
|
|
|
19
24
|
For each file you identify:
|
|
@@ -25,22 +30,35 @@ Query multiple AI models for expert opinions on a coding question.
|
|
|
25
30
|
|
|
26
31
|
**Note (Claude only):** Do not use your `Read` tool to dump file contents into the query. `build_context.sh` handles file reading itself, without the `N\t` line-number prefix that `Read` adds.
|
|
27
32
|
|
|
28
|
-
3. **
|
|
33
|
+
3. **Write the question safely and run the consultation**:
|
|
34
|
+
- With Bash, create a query file using `mktemp`, set mode 600, and write the
|
|
35
|
+
exact question text (with file paths and consultation options removed)
|
|
36
|
+
through a **single-quoted heredoc delimiter that does not occur as its own
|
|
37
|
+
line in the question**. Never pass raw user text as a shell argument or
|
|
38
|
+
through an unquoted heredoc.
|
|
39
|
+
- Shell-quote every context-file path as one argument.
|
|
40
|
+
- Install a shell trap that removes only the exact query file when the Bash
|
|
41
|
+
invocation exits; question text must not persist in the temp directory.
|
|
42
|
+
- Capture `caller_root="$PWD"` before changing directories. Pass it through
|
|
43
|
+
`--context-root` so regular files from the user's project are staged under
|
|
44
|
+
a private temp directory without widening the runtime filesystem allowlist.
|
|
45
|
+
|
|
29
46
|
```bash
|
|
30
|
-
cd "${AI_CONSULTANTS_DIR:-$HOME/.claude/skills/ai-consultants}" && INVOKING_AGENT=claude ./scripts/consult_all.sh
|
|
47
|
+
caller_root="$PWD"; cd "${AI_CONSULTANTS_DIR:-$HOME/.claude/skills/ai-consultants}" && INVOKING_AGENT=claude ./scripts/consult_all.sh [--preset <name>] [--strategy <name>] --context-root "$caller_root" --query-file "<private-question-file>" "<file1[@TAG]>" "<file2[@TAG]>" ...
|
|
31
48
|
```
|
|
32
|
-
- `<question>` is the question text with file paths stripped. Use single quotes to prevent shell expansion.
|
|
33
49
|
- Each `<fileN>` is a file path, optionally suffixed with `@PRIMARY` or `@CONTEXT`.
|
|
34
|
-
- **If the question text exceeds ~8KB or contains awkward quoting**, write it to a tmpfile and use `--query-file <path>` instead:
|
|
35
|
-
```bash
|
|
36
|
-
cd "${AI_CONSULTANTS_DIR:-$HOME/.claude/skills/ai-consultants}" && INVOKING_AGENT=claude ./scripts/consult_all.sh --query-file /tmp/question.txt <file1[@TAG]> ...
|
|
37
|
-
```
|
|
38
50
|
- **Capture the last line of stdout** — it contains the output directory path (e.g., `/tmp/ai_consultations/20260315_143022_12345/`).
|
|
51
|
+
- Keep `INVOKING_AGENT=claude` exactly as shown. Never invoke
|
|
52
|
+
`query_claude.sh` directly and never remove self-exclusion to increase the
|
|
53
|
+
panel count.
|
|
39
54
|
|
|
40
55
|
4. **Read the results** from the output directory captured in step 3:
|
|
41
56
|
- Read `<output_dir>/report.md` using the **Read** tool
|
|
42
57
|
- Read `<output_dir>/synthesis.json` using the **Read** tool (may not exist if synthesis was skipped or failed)
|
|
43
58
|
- If neither exists, use the **Glob** tool with pattern `<output_dir>/*.json` to discover individual consultant response files, then read each one
|
|
59
|
+
- `claude.json` must not exist and `.synthesis_provider` must not be
|
|
60
|
+
`claude`. If either appears, report a self-exclusion failure instead of
|
|
61
|
+
presenting the run as valid.
|
|
44
62
|
|
|
45
63
|
5. **Present the results** using this template. The panel's value is *coverage* — the
|
|
46
64
|
union of distinct points across diverse models, including what only one raised — not a
|
|
@@ -87,5 +105,7 @@ Query multiple AI models for expert opinions on a coding question.
|
|
|
87
105
|
|
|
88
106
|
For follow-up questions on the same consultation:
|
|
89
107
|
```bash
|
|
90
|
-
cd "${AI_CONSULTANTS_DIR:-$HOME/.claude/skills/ai-consultants}" && INVOKING_AGENT=claude ./scripts/followup.sh
|
|
108
|
+
cd "${AI_CONSULTANTS_DIR:-$HOME/.claude/skills/ai-consultants}" && INVOKING_AGENT=claude ./scripts/followup.sh --query-file "<private-follow-up-file>"
|
|
91
109
|
```
|
|
110
|
+
Create and clean the follow-up file with the same private heredoc/trap contract.
|
|
111
|
+
Never target Claude from a Claude-hosted follow-up.
|
|
@@ -19,9 +19,13 @@ All skill details (consultants, personas, presets, strategies, features, configu
|
|
|
19
19
|
```
|
|
20
20
|
/ai-consultants:consult "How to optimize this SQL query?"
|
|
21
21
|
/ai-consultants:consult "Review this code" src/utils.ts
|
|
22
|
-
/ai-consultants:
|
|
22
|
+
/ai-consultants:consult --preset max_quality "Microservices or monolith?"
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
+
When Claude invokes the panel, the command sets `INVOKING_AGENT=claude` so
|
|
26
|
+
Claude is excluded from both consultation and synthesis while Codex and the
|
|
27
|
+
other enabled providers remain available.
|
|
28
|
+
|
|
25
29
|
Configuration (presets, strategies, features, personas, API keys) can be managed via natural language — just ask.
|
|
26
30
|
|
|
27
31
|
Troubleshooting: run `./scripts/doctor.sh` or `./scripts/doctor.sh --fix`.
|
package/.env.example
CHANGED
|
@@ -65,13 +65,12 @@
|
|
|
65
65
|
# ENABLED CONSULTANTS
|
|
66
66
|
# =============================================================================
|
|
67
67
|
# Set to "false" to disable a consultant you haven't configured.
|
|
68
|
-
# MINIMUM 2 consultants must be enabled for
|
|
68
|
+
# MINIMUM 2 consultants must be enabled for a useful coverage panel.
|
|
69
69
|
|
|
70
70
|
# CLI-based consultants (default: enabled)
|
|
71
71
|
ENABLE_GEMINI=true
|
|
72
72
|
ENABLE_CODEX=true
|
|
73
73
|
ENABLE_MISTRAL=true
|
|
74
|
-
ENABLE_CURSOR=true
|
|
75
74
|
ENABLE_KIMI=true
|
|
76
75
|
ENABLE_QWEN3=true
|
|
77
76
|
ENABLE_GROK=true
|
|
@@ -113,17 +112,18 @@ ENABLE_DEEPSEEK=false
|
|
|
113
112
|
# https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions
|
|
114
113
|
|
|
115
114
|
# =============================================================================
|
|
116
|
-
# MODEL CONFIGURATION (
|
|
115
|
+
# MODEL CONFIGURATION (August 2026 premium defaults)
|
|
117
116
|
# =============================================================================
|
|
118
117
|
|
|
119
118
|
# Gemini - The Architect (CLI mode via Antigravity CLI: agy)
|
|
120
|
-
GEMINI_MODEL="Gemini 3.
|
|
119
|
+
GEMINI_MODEL="Gemini 3.7 Flash (High)"
|
|
121
120
|
GEMINI_TIMEOUT=180
|
|
122
121
|
GEMINI_CMD=agy
|
|
123
122
|
# API-mode model ID (only used when GEMINI_USE_API=true)
|
|
124
123
|
GEMINI_API_MODEL=gemini-3.1-pro-preview
|
|
125
|
-
#
|
|
126
|
-
# gemini-3.7-flash
|
|
124
|
+
# Gemini 3.1 Pro and 3.6 Flash remain valid CLI pins. API mode stays on the
|
|
125
|
+
# separately verified 3.1 Pro ID; gemini-3.7-flash is an API-only opt-in until
|
|
126
|
+
# that transport completes its own authenticated smoke.
|
|
127
127
|
# Reasoning effort for the agy CLI. Passed as --effort <value>. Accepts only
|
|
128
128
|
# low|medium|high; anything else is rejected by the provider and surfaces as a
|
|
129
129
|
# diagnosed error (provider owns the enum — same stance as QWEN3_REASONING_EFFORT).
|
|
@@ -131,6 +131,7 @@ GEMINI_API_MODEL=gemini-3.1-pro-preview
|
|
|
131
131
|
|
|
132
132
|
# Codex - The Pragmatist
|
|
133
133
|
CODEX_MODEL=gpt-5.6-sol
|
|
134
|
+
CODEX_API_MAX_TOKENS=4096
|
|
134
135
|
CODEX_TIMEOUT=180
|
|
135
136
|
CODEX_CMD=codex
|
|
136
137
|
# Reasoning effort for the codex CLI (and API mode). Passed as
|
|
@@ -143,15 +144,12 @@ CODEX_CMD=codex
|
|
|
143
144
|
# Large 3, and Small 4 API IDs are documented opt-ins until an API smoke passes.
|
|
144
145
|
MISTRAL_MODEL=mistral-large-3
|
|
145
146
|
MISTRAL_CLI_MODEL=mistral-medium-3.5
|
|
147
|
+
MISTRAL_API_MAX_TOKENS=4096
|
|
146
148
|
# Official API opt-ins: use mistral-medium-3-5, mistral-large-2512, or
|
|
147
149
|
# mistral-small-2603 for MISTRAL_MODEL after an authenticated API smoke.
|
|
148
150
|
MISTRAL_TIMEOUT=180
|
|
149
151
|
MISTRAL_CMD=vibe
|
|
150
|
-
|
|
151
|
-
# Cursor - The Integrator
|
|
152
|
-
CURSOR_MODEL=composer-2.5
|
|
153
|
-
CURSOR_TIMEOUT=180
|
|
154
|
-
CURSOR_CMD=cursor-agent
|
|
152
|
+
MISTRAL_MAX_TURNS=4
|
|
155
153
|
|
|
156
154
|
# Kimi - The Eastern Sage (K3 via Kimi Code CLI)
|
|
157
155
|
KIMI_MODEL=kimi-code/k3
|
|
@@ -174,6 +172,8 @@ CLAUDE_API_MAX_TOKENS=16384
|
|
|
174
172
|
# Qwen3 - The Analyst (CLI/API switchable v2.7)
|
|
175
173
|
QWEN3_MODEL=qwen3.7-max
|
|
176
174
|
QWEN3_TIMEOUT=180
|
|
175
|
+
QWEN3_MAX_QUALITY_TIMEOUT=600
|
|
176
|
+
QWEN3_API_MAX_TOKENS=16384
|
|
177
177
|
QWEN3_CMD=qwen
|
|
178
178
|
# Reasoning effort. API mode ONLY - the qwen CLI has no effort flag, so in CLI
|
|
179
179
|
# mode this is ignored (with a warning); set model.reasoningEffort in
|
|
@@ -196,24 +196,36 @@ QWEN3_CMD=qwen
|
|
|
196
196
|
# GLM - The Methodologist
|
|
197
197
|
GLM_MODEL=glm-5.3
|
|
198
198
|
GLM_TIMEOUT=180
|
|
199
|
+
GLM_API_MAX_TOKENS=16384
|
|
199
200
|
GLM_API_URL=https://api.z.ai/api/coding/paas/v4/chat/completions
|
|
201
|
+
# GLM_REASONING_EFFORT= # max_quality sets max
|
|
200
202
|
|
|
201
203
|
# Grok Build CLI + xAI API fallback - The Provocateur
|
|
202
204
|
GROK_CMD=grok
|
|
203
205
|
GROK_MODEL=grok-4.6
|
|
204
206
|
GROK_TIMEOUT=180
|
|
207
|
+
GROK_MAX_TURNS=4
|
|
208
|
+
GROK_API_MAX_TOKENS=4096
|
|
205
209
|
GROK_API_URL=https://api.x.ai/v1/chat/completions
|
|
210
|
+
# GROK_REASONING_EFFORT= # max_quality sets xhigh (Grok Build maximum)
|
|
206
211
|
|
|
207
212
|
# DeepSeek - The Code Specialist
|
|
208
213
|
DEEPSEEK_MODEL=deepseek-v4-pro
|
|
209
214
|
DEEPSEEK_TIMEOUT=180
|
|
215
|
+
DEEPSEEK_MAX_QUALITY_TIMEOUT=600
|
|
216
|
+
DEEPSEEK_API_MAX_TOKENS=16384
|
|
210
217
|
DEEPSEEK_API_URL=https://api.deepseek.com/v1/chat/completions
|
|
218
|
+
# DEEPSEEK_REASONING_EFFORT= # max_quality sets max
|
|
211
219
|
|
|
212
220
|
# MiniMax - The Pragmatic Optimizer (CLI via mmx by default, v2.21; API opt-in)
|
|
213
221
|
MINIMAX_CMD=mmx
|
|
214
222
|
MINIMAX_MODEL=MiniMax-M2.7
|
|
215
223
|
# max_quality selects the smoke-tested MiniMax-M3 target.
|
|
216
224
|
MINIMAX_TIMEOUT=180
|
|
225
|
+
MINIMAX_API_MAX_TOKENS=4096
|
|
226
|
+
MINIMAX_MAX_TOKENS=4096
|
|
227
|
+
MINIMAX_MAX_QUALITY_TOKENS=16384
|
|
228
|
+
MAX_QUALITY_API_MAX_TOKENS=16384
|
|
217
229
|
MINIMAX_API_URL=https://api.minimax.io/v1/chat/completions
|
|
218
230
|
|
|
219
231
|
# Wire format for a consultant's API mode. Honored since v2.25 (previously
|
|
@@ -249,6 +261,9 @@ SYNTHESIS_CMD=claude
|
|
|
249
261
|
SYNTHESIS_MODEL=
|
|
250
262
|
SYNTHESIS_EXTRACT_FIELDS=true
|
|
251
263
|
SYNTH_MAX=10
|
|
264
|
+
SYNTH_DETAIL_MAX_CHARS=4000
|
|
265
|
+
SYNTHESIS_TIMEOUT=240
|
|
266
|
+
SYNTHESIS_TOTAL_TIMEOUT=480
|
|
252
267
|
|
|
253
268
|
# Health gate and quorum: validate the live panel before spending on a full run
|
|
254
269
|
ENABLE_HEALTH_GATE=false # Adds one small ping per selected consultant
|
|
@@ -388,7 +403,6 @@ REPORT_MAX_JSON_LINES=50
|
|
|
388
403
|
# GEMINI_PERSONA_ID=1
|
|
389
404
|
# CODEX_PERSONA_ID=2
|
|
390
405
|
# MISTRAL_PERSONA_ID=3
|
|
391
|
-
# CURSOR_PERSONA_ID=5
|
|
392
406
|
# KIMI_PERSONA_ID=20
|
|
393
407
|
# CLAUDE_PERSONA_ID=18
|
|
394
408
|
# QWEN3_PERSONA_ID=6
|
|
@@ -402,8 +416,6 @@ REPORT_MAX_JSON_LINES=50
|
|
|
402
416
|
# CODEX_PERSONA_NAME=Custom persona name
|
|
403
417
|
# MISTRAL_PERSONA=Custom persona instructions
|
|
404
418
|
# MISTRAL_PERSONA_NAME=Custom persona name
|
|
405
|
-
# CURSOR_PERSONA=Custom persona instructions
|
|
406
|
-
# CURSOR_PERSONA_NAME=Custom persona name
|
|
407
419
|
# KIMI_PERSONA=Custom persona instructions
|
|
408
420
|
# KIMI_PERSONA_NAME=Custom persona name
|
|
409
421
|
# CLAUDE_PERSONA=Custom persona instructions
|
package/CLAUDE.md
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
## Project Overview
|
|
4
4
|
|
|
5
|
-
AI Consultants is a multi-model coverage system that queries up to
|
|
5
|
+
AI Consultants is a multi-model coverage system that queries up to 10 AI consultants (Gemini, Codex, Mistral, Kimi, Claude, Qwen3, GLM, Grok, DeepSeek, MiniMax) to obtain diverse perspectives on coding problems.
|
|
6
6
|
|
|
7
|
-
**Self-Exclusion**: The invoking agent is automatically excluded from the panel
|
|
7
|
+
**Self-Exclusion**: The invoking agent is automatically excluded from both the panel and synthesis. Claude Code won't query or synthesize with Claude, Codex CLI won't query or synthesize with Codex, etc.
|
|
8
8
|
|
|
9
|
-
**Version**:
|
|
9
|
+
**Version**: 4.0.1
|
|
10
10
|
|
|
11
11
|
## Distribution
|
|
12
12
|
|
|
@@ -176,25 +176,33 @@ Functions in `lib/common.sh`:
|
|
|
176
176
|
- `get_self_consultant_name()` - Maps invoking agent to consultant name
|
|
177
177
|
- `should_skip_consultant()` - Returns true if consultant should be excluded
|
|
178
178
|
- `log_self_exclusion_status()` - Debug logging for exclusion
|
|
179
|
+
- `resolve_synthesis_cli()` - Refuses the invoking agent as synthesis provider
|
|
180
|
+
|
|
181
|
+
Agent hosts must set `INVOKING_AGENT` explicitly and run `consult_all.sh`
|
|
182
|
+
instead of individual adapters. A Claude-hosted run must produce no
|
|
183
|
+
`claude.json`, and its `synthesis_provider` must not be `claude`; the equivalent
|
|
184
|
+
invariant applies to Codex and Gemini. See `SKILL.md` for the full execution
|
|
185
|
+
contract.
|
|
179
186
|
|
|
180
187
|
### Configuration Presets
|
|
181
188
|
```bash
|
|
182
189
|
# Quality Tier Presets (v2.5)
|
|
183
|
-
./scripts/consult_all.sh --preset max_quality "question" #
|
|
184
|
-
./scripts/consult_all.sh --preset medium "question" #
|
|
190
|
+
./scripts/consult_all.sh --preset max_quality "question" # all 10 consultants + maximum tier/max effort
|
|
191
|
+
./scripts/consult_all.sh --preset medium "question" # 3 consultants + standard models
|
|
185
192
|
./scripts/consult_all.sh --preset fast "question" # 2 consultants + economy models
|
|
186
193
|
|
|
187
194
|
# Use Case Presets
|
|
188
195
|
./scripts/consult_all.sh --preset minimal "question" # Gemini + Codex
|
|
189
|
-
./scripts/consult_all.sh --preset balanced "question" # + Mistral
|
|
190
|
-
./scripts/consult_all.sh --preset high-stakes "question" #
|
|
196
|
+
./scripts/consult_all.sh --preset balanced "question" # + Mistral
|
|
197
|
+
./scripts/consult_all.sh --preset high-stakes "question" # Broad premium panel
|
|
191
198
|
```
|
|
192
199
|
|
|
193
200
|
Presets are defined in `config.sh` via `apply_preset()` function.
|
|
194
201
|
|
|
195
202
|
### Synthesis Strategies
|
|
196
203
|
```bash
|
|
197
|
-
./scripts/consult_all.sh --strategy
|
|
204
|
+
./scripts/consult_all.sh --strategy coverage "question" # Default: union of distinct points
|
|
205
|
+
./scripts/consult_all.sh --strategy majority "question" # Blended recommendation
|
|
198
206
|
./scripts/consult_all.sh --strategy risk_averse "question" # Conservative
|
|
199
207
|
./scripts/consult_all.sh --strategy security_first "question" # Security focus
|
|
200
208
|
./scripts/consult_all.sh --strategy compare_only "question" # No recommendation
|
|
@@ -210,34 +218,6 @@ Strategies are implemented in `synthesize.sh` via `get_strategy_instructions()`.
|
|
|
210
218
|
./scripts/doctor.sh --verbose # Detailed output
|
|
211
219
|
```
|
|
212
220
|
|
|
213
|
-
### Panic Button Mode
|
|
214
|
-
Automatically adds rigor when uncertainty detected:
|
|
215
|
-
- Average confidence below `PANIC_CONFIDENCE_THRESHOLD` (default: 5)
|
|
216
|
-
- Uncertainty keywords detected in responses
|
|
217
|
-
|
|
218
|
-
Configuration in `config.sh`:
|
|
219
|
-
- `ENABLE_PANIC_MODE` - "auto", "always", or "never"
|
|
220
|
-
- `PANIC_CONFIDENCE_THRESHOLD` - Trigger threshold
|
|
221
|
-
- `PANIC_EXTRA_DEBATE_ROUNDS` - Additional debate rounds
|
|
222
|
-
|
|
223
|
-
### Confidence Intervals
|
|
224
|
-
Functions in `lib/voting.sh`:
|
|
225
|
-
- `calculate_confidence_stddev()` - Returns stddev scaled by 10
|
|
226
|
-
- `calculate_confidence_interval()` - Returns JSON with mean, stddev, interval
|
|
227
|
-
- `has_high_confidence_variance()` - Returns 0 if variance > 2.0
|
|
228
|
-
- `format_confidence_range()` - Returns "7 ± 1.5" format
|
|
229
|
-
|
|
230
|
-
### Anonymous Peer Review
|
|
231
|
-
```bash
|
|
232
|
-
./scripts/peer_review.sh <responses_dir> <output_dir>
|
|
233
|
-
```
|
|
234
|
-
|
|
235
|
-
Process:
|
|
236
|
-
1. Anonymize all responses (remove consultant names)
|
|
237
|
-
2. Each consultant ranks and critiques responses
|
|
238
|
-
3. Aggregate peer scores to identify strongest arguments
|
|
239
|
-
4. De-anonymize in final report
|
|
240
|
-
|
|
241
221
|
## v2.9 Features
|
|
242
222
|
|
|
243
223
|
### Kimi CLI Support
|
|
@@ -288,7 +268,7 @@ npm install -g @qwen-code/qwen-code@latest
|
|
|
288
268
|
## v2.6 Features
|
|
289
269
|
|
|
290
270
|
### CLI/API Mode Switching
|
|
291
|
-
|
|
271
|
+
Seven consultants can switch between CLI and API mode: **Gemini, Codex, Claude, Mistral, Qwen3, Grok, MiniMax**.
|
|
292
272
|
|
|
293
273
|
When API mode is enabled for an agent, CLI mode is disabled (mutual exclusivity).
|
|
294
274
|
|
|
@@ -325,6 +305,7 @@ New environment variables in `config.sh`:
|
|
|
325
305
|
| `CLAUDE_USE_API` | false | Use Anthropic API instead of claude CLI |
|
|
326
306
|
| `MISTRAL_USE_API` | false | Use Mistral API instead of vibe CLI |
|
|
327
307
|
| `QWEN3_USE_API` | false | Use DashScope API instead of qwen CLI (v2.7) |
|
|
308
|
+
| `GROK_USE_API` | auto | Use xAI API when Grok Build is unavailable |
|
|
328
309
|
| `MINIMAX_USE_API` | false | Use MiniMax API instead of the mmx CLI (v2.21) |
|
|
329
310
|
| `GEMINI_API_URL` | https://generativelanguage.googleapis.com/v1beta/models | Google AI endpoint |
|
|
330
311
|
| `CODEX_API_URL` | https://api.openai.com/v1/chat/completions | OpenAI endpoint |
|
|
@@ -368,10 +349,10 @@ Three normal tiers plus a `maximum` tier are configurable via `apply_model_tier(
|
|
|
368
349
|
|
|
369
350
|
| Tier | Description | Example Models |
|
|
370
351
|
|------|-------------|----------------|
|
|
371
|
-
| **premium** | Latest flagship models | claude-opus-5, Gemini 3.
|
|
372
|
-
| **maximum** |
|
|
373
|
-
| **standard** | Good quality at reasonable cost | claude-sonnet-5, Gemini 3.
|
|
374
|
-
| **economy** | Optimized for speed and low cost | claude-haiku-4-5, Gemini 3.
|
|
352
|
+
| **premium** | Latest flagship models | claude-opus-5, Gemini 3.7 Flash (High), gpt-5.6-sol |
|
|
353
|
+
| **maximum** | All 10 consultants; maximum targets and highest provider effort | K3-256k, Qwen3.8-Max, MiniMax M3; Grok xhigh, GLM/DeepSeek max |
|
|
354
|
+
| **standard** | Good quality at reasonable cost | claude-sonnet-5, Gemini 3.7 Flash (High), gpt-5.6-terra |
|
|
355
|
+
| **economy** | Optimized for speed and low cost | claude-haiku-4-5, Gemini 3.7 Flash (Low), gpt-5.6-luna |
|
|
375
356
|
|
|
376
357
|
**Default models are now premium tier** for maximum quality.
|
|
377
358
|
|
|
@@ -384,7 +365,7 @@ apply_model_tier "standard" # Set all consultants to standard models
|
|
|
384
365
|
apply_model_tier "economy" # Set all consultants to economy models
|
|
385
366
|
|
|
386
367
|
# Get model for a specific consultant and tier (v2.8.1)
|
|
387
|
-
get_model_for_tier "gemini" "premium" # → Gemini 3.
|
|
368
|
+
get_model_for_tier "gemini" "premium" # → Gemini 3.7 Flash (High) on agy
|
|
388
369
|
get_model_for_tier "claude" "economy" # → claude-haiku-4-5
|
|
389
370
|
```
|
|
390
371
|
|
|
@@ -392,26 +373,25 @@ get_model_for_tier "claude" "economy" # → claude-haiku-4-5
|
|
|
392
373
|
Three new presets leverage the model tiers:
|
|
393
374
|
|
|
394
375
|
```bash
|
|
395
|
-
# Maximum quality -
|
|
376
|
+
# Maximum quality - all consultants + maximum models and provider effort
|
|
396
377
|
./scripts/consult_all.sh --preset max_quality "critical architecture decision"
|
|
397
378
|
|
|
398
|
-
# Balanced quality - standard models,
|
|
379
|
+
# Balanced quality - standard models, 3 consultants
|
|
399
380
|
./scripts/consult_all.sh --preset medium "general coding question"
|
|
400
381
|
|
|
401
|
-
# Super fast - economy models, 2 consultants
|
|
382
|
+
# Super fast - economy models, 2 consultants
|
|
402
383
|
./scripts/consult_all.sh --preset fast "quick syntax question"
|
|
403
384
|
```
|
|
404
385
|
|
|
405
|
-
### Premium Model Defaults (
|
|
386
|
+
### Premium Model Defaults (August 2026)
|
|
406
387
|
All consultants now use premium models by default:
|
|
407
388
|
|
|
408
389
|
| Consultant | Default Model |
|
|
409
390
|
|------------|---------------|
|
|
410
391
|
| Claude | claude-opus-5 |
|
|
411
|
-
| Gemini | Gemini 3.
|
|
392
|
+
| Gemini | Gemini 3.7 Flash (High) (via agy CLI); API: gemini-3.1-pro-preview |
|
|
412
393
|
| Codex | gpt-5.6-sol |
|
|
413
394
|
| Mistral | CLI: mistral-medium-3.5; API: mistral-large-3 |
|
|
414
|
-
| Cursor | composer-2.5 |
|
|
415
395
|
| DeepSeek | deepseek-v4-pro |
|
|
416
396
|
| GLM | glm-5.3 |
|
|
417
397
|
| Grok | grok-4.6 |
|
|
@@ -430,6 +410,20 @@ missing, cannot launch, or lacks usable authentication and `GROK_API_KEY` is
|
|
|
430
410
|
available. A timeout, model error, empty response, or other post-launch failure
|
|
431
411
|
is surfaced and never silently rerouted to the API.
|
|
432
412
|
|
|
413
|
+
The live full-panel smoke defines the `max_quality` runtime envelope, not just
|
|
414
|
+
the model names. Mistral and Grok receive four bounded advisory turns plus an
|
|
415
|
+
explicit no-tools/workspace instruction. Qwen3.8-Max gets a 600-second Token
|
|
416
|
+
Plan timeout; DeepSeek V4 Pro/max gets 600 seconds and 16,384 completion tokens;
|
|
417
|
+
GLM gets 16,384; MiniMax M3 gets 16,384 plus a compact Markdown contract in
|
|
418
|
+
mmx's native system channel. OpenAI-compatible
|
|
419
|
+
`finish_reason=length` is an error. Every
|
|
420
|
+
response records `metadata.response_quality=structured|fallback|error`:
|
|
421
|
+
malformed JSON-looking output fails closed, while real markdown/prose is kept
|
|
422
|
+
as an explicit fallback. Synthesis filters error envelopes and includes bounded
|
|
423
|
+
detail from structured and fallback responses. Claude CLI consultations are
|
|
424
|
+
preflighted, stateless (`--no-session-persistence`), setting-source-free,
|
|
425
|
+
tool-free, and MCP-free; a hung/help/auth probe fails before a paid dispatch.
|
|
426
|
+
|
|
433
427
|
## v2.4 Features
|
|
434
428
|
|
|
435
429
|
### Budget Enforcement (Opt-in)
|
|
@@ -449,8 +443,7 @@ BUDGET_ACTION=warn # or "stop"
|
|
|
449
443
|
**Enforcement Points:**
|
|
450
444
|
1. Before Round 1 - Check estimated cost vs budget
|
|
451
445
|
2. After Round 1 - Check actual cost vs warning threshold
|
|
452
|
-
3. Before
|
|
453
|
-
4. Before Synthesis - Check cumulative + synthesis estimate
|
|
446
|
+
3. Before Synthesis - Check cumulative + synthesis estimate
|
|
454
447
|
|
|
455
448
|
Functions in `lib/costs.sh`:
|
|
456
449
|
- `is_budget_enabled()` - Check if budget enforcement is enabled
|
|
@@ -523,22 +516,6 @@ Functions in `lib/routing.sh`:
|
|
|
523
516
|
- `get_premium_model()` - Returns premium model for consultant
|
|
524
517
|
- `get_escalation_summary()` - Returns escalation info as JSON
|
|
525
518
|
|
|
526
|
-
### Debate Optimization (Opt-in)
|
|
527
|
-
Skips debate if confidence spread is low (all consultants agree).
|
|
528
|
-
|
|
529
|
-
```bash
|
|
530
|
-
ENABLE_DEBATE_OPTIMIZATION=false # Default: false (opt-in per quality review)
|
|
531
|
-
DEBATE_CONFIDENCE_SPREAD_THRESHOLD=3 # Min spread to trigger debate
|
|
532
|
-
DEBATE_USE_SUMMARIES=true # Use summaries in debate rounds
|
|
533
|
-
```
|
|
534
|
-
|
|
535
|
-
**Category Exceptions**: SECURITY and ARCHITECTURE always trigger debate regardless of confidence spread.
|
|
536
|
-
|
|
537
|
-
Functions in `debate_round.sh`:
|
|
538
|
-
- `is_mandatory_debate_category()` - Returns true for SECURITY/ARCHITECTURE
|
|
539
|
-
- `should_skip_debate()` - Returns true if debate can be skipped
|
|
540
|
-
- `extract_compact_summary()` - Token-optimized summary for debates
|
|
541
|
-
|
|
542
519
|
### Quality Monitoring
|
|
543
520
|
Logs optimization metrics and saves to output directory.
|
|
544
521
|
|
|
@@ -555,12 +532,9 @@ Output file `optimization_metrics.json`:
|
|
|
555
532
|
"cache_hits": 2,
|
|
556
533
|
"response_limits_enabled": false,
|
|
557
534
|
"cost_aware_routing": false,
|
|
558
|
-
"debate_optimization": false,
|
|
559
535
|
"compact_report": true
|
|
560
536
|
},
|
|
561
537
|
"quality_metrics": {
|
|
562
|
-
"consensus_score": 75,
|
|
563
|
-
"consensus_level": "medium",
|
|
564
538
|
"successful_responses": 4,
|
|
565
539
|
"total_consultants": 4,
|
|
566
540
|
"category": "ARCHITECTURE"
|
|
@@ -595,9 +569,6 @@ npm run lint
|
|
|
595
569
|
# Test with preset
|
|
596
570
|
./scripts/consult_all.sh --preset minimal "Quick question"
|
|
597
571
|
|
|
598
|
-
# Test with debate
|
|
599
|
-
ENABLE_DEBATE=true ./scripts/consult_all.sh "Microservices or monolith?"
|
|
600
|
-
|
|
601
572
|
# Test with strategy
|
|
602
573
|
./scripts/consult_all.sh --strategy risk_averse "Security question"
|
|
603
574
|
|
|
@@ -617,7 +588,7 @@ for f in scripts/*.sh scripts/lib/*.sh; do bash -n "$f" && echo "OK: $f"; done
|
|
|
617
588
|
| `MAX_SESSION_COST` | 1.00 | Max budget ($) |
|
|
618
589
|
| `CLAUDE_MODEL` | claude-opus-5 | Claude model (v2.5) |
|
|
619
590
|
| `CLAUDE_API_MAX_TOKENS` | 16384 | Claude API thinking + visible-output budget |
|
|
620
|
-
| `GEMINI_MODEL` | Gemini 3.
|
|
591
|
+
| `GEMINI_MODEL` | Gemini 3.7 Flash (High) | Gemini agy CLI model |
|
|
621
592
|
| `GEMINI_API_MODEL` | gemini-3.1-pro-preview | Gemini API-mode model ID (v2.15) |
|
|
622
593
|
| `CODEX_MODEL` | gpt-5.6-sol | Codex model (v2.5) |
|
|
623
594
|
| `MISTRAL_MODEL` | mistral-large-3 | Mistral model (v2.5) |
|
|
@@ -646,7 +617,6 @@ for f in scripts/*.sh scripts/lib/*.sh; do bash -n "$f" && echo "OK: $f"; done
|
|
|
646
617
|
- `agy` CLI - Antigravity CLI (Gemini consultant; successor to the deprecated Gemini CLI, v2.15)
|
|
647
618
|
- `codex` CLI - OpenAI Codex
|
|
648
619
|
- `vibe` CLI - Mistral Vibe
|
|
649
|
-
- `cursor-agent` CLI - Cursor (validated by capability, read-only ask mode)
|
|
650
620
|
- `kimi` CLI - Kimi Code (v2.9)
|
|
651
621
|
- `claude` CLI - Claude (v2.2, also used for synthesis)
|
|
652
622
|
- `qwen` CLI - Qwen via qwen-code (v2.7, optional)
|
|
@@ -782,7 +752,7 @@ curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/s
|
|
|
782
752
|
### Guidelines
|
|
783
753
|
|
|
784
754
|
- **Audience**: Developers who use the skill. Write for someone who hasn't seen the commits.
|
|
785
|
-
- **Highlights first**: Lead with impact, not implementation. "
|
|
755
|
+
- **Highlights first**: Lead with impact, not implementation. "Provider failures are now diagnosed" > "Added an error branch".
|
|
786
756
|
- **Breaking changes prominent**: If any exist, they go in a dedicated section — never buried in a bullet list.
|
|
787
757
|
- **Quantify when possible**: Token savings percentages, issue counts, file counts.
|
|
788
758
|
- **Upgrade guide always present**: Even if the answer is "just pull", make it explicit.
|
|
@@ -790,6 +760,25 @@ curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/s
|
|
|
790
760
|
|
|
791
761
|
## Changelog
|
|
792
762
|
|
|
763
|
+
### v4.0.1
|
|
764
|
+
|
|
765
|
+
- **The installer's first line of user-visible output was six major/minor generations stale.** `scripts/install.sh:53` printed `AI Consultants v2.10 - Installation` even while the public binary, package metadata, npm registry, active checkout, and doctor all reported v4.0.0. That did not change installed bytes or runtime behavior, but it made a successful v4 install look wrong at the exact moment users were deciding whether to trust it.
|
|
766
|
+
- **The banner is now deliberately version-neutral, not dynamically coupled to a tenth version surface.** The curl-piped installer prints before it has cloned or updated the checkout, so sourcing `scripts/config.sh` there would either be impossible on a fresh install or would report the old local version during an update. A static `AI Consultants - Installation` label stays truthful throughout both paths and does not need another release-time rewrite.
|
|
767
|
+
- **The failure is regression-tested through rendered output.** `scripts/test_install.sh` sources the existing define-only path, calls `print_header`, requires the installer label, and rejects `AI Consultants v<major>.<minor>` in the banner. The focused installer suite grows to 47 checks; the full 23-suite gate and ShellCheck pass on the 4.0.1 surfaces.
|
|
768
|
+
|
|
769
|
+
Deliberately unchanged: no runtime, model, preset, credential, host command, or minisite copy changes. The showcase site already reflects the ten-consultant v4 panel; after npm serves 4.0.1 it needs only its two mechanical version surfaces bumped.
|
|
770
|
+
|
|
771
|
+
### v4.0.0
|
|
772
|
+
|
|
773
|
+
- **`max_quality` now means the full ten-consultant panel at each exact transport's strongest supported effort and response budget.** GLM and DeepSeek run at `max`; Grok uses its highest supported setting. K3-256k, MiniMax M3, and Qwen3.8-Max remain confined to the maximum tier, with Qwen promoted only when the authenticated Token Plan transport is already configured.
|
|
774
|
+
- **Gemini 3.7 Flash High completed an authenticated exact-adapter `agy` smoke and is promoted only on that CLI transport.** It serves maximum, premium, and standard; Low serves economy. The unverified Google API transport remains on 3.1 Pro, while Fable 5 and the new Mistral API IDs remain opt-in.
|
|
775
|
+
- **Cursor is no longer a consultant.** The ambiguous `agent`/`cursor-agent` runtime, model tiers, presets, doctor/configuration fields, personas, costs, tests, and website card were removed. Cursor remains a supported host through SkillPort; the panel itself has 10 consultants.
|
|
776
|
+
- **Managed migrations are exact and pins survive.** `configure` upgrades only the historical managed Gemini 3.1 Pro CLI, GLM 5.2, and Grok 4.5 values, while removing obsolete Cursor settings. Environment/`--set` model values and any `# ai-consultants:pin` entry remain user-owned. The catalog-parity gate checks all 40 consultant/tier cells and all 16 Gemini/Mistral transport cells against `docs/cost_rates.json`, requiring every automatic target to be priced or explicitly unpriced.
|
|
777
|
+
- **Claude can launch the skill as a real host, not merely describe it.** `SKILL.md` carries an imperative execution contract; all host commands use a private query-file handoff, preserve preset/strategy options, pass the caller project through `--context-root`, and set `INVOKING_AGENT`; self-exclusion covers consultation, full-roster targeted follow-ups, and synthesis for Claude/Codex/Gemini. Offline E2E proves a Claude host dispatches Gemini/Codex/Mistral with no Claude artifacts. A real Claude Code/Sonnet run started from an external temp project, used `mktemp`, mode 600, a quoted heredoc, trap cleanup, `--context-root`, and `src/context.txt@CONTEXT`; Gemini 3.7 High and Codex 5.6 Sol returned (2/2), the external marker/tag reached both, no Claude artifact appeared, and the private staging directory was removed. The live run disabled synthesis; symmetric regressions prove each host is excluded there. Current runtime prompts, schemas, templates, help, and instructions are coverage-only; stale Codex/Gemini debate and roster-audit commands are removed.
|
|
778
|
+
- **Cross-family host review went `NO_SHIP` → `NO_SHIP` → `SHIP`.** The review found the Bash 3.2 empty-array abort, query-file/context parser conflict, caller-project path loss, unsafe/host-self follow-ups, stale live consensus output, broad uninstall glob, and active v3.0 residue. Each was fixed with executable coverage. After the `SHIP` verdict, its remaining low-severity findings were also resolved: staged files retain their original project-relative identity, fallback synthesis reports the requested strategy, follow-up context no longer rides in argv and respects disabled consultants, and remaining contributor/cost/test residue was removed.
|
|
779
|
+
|
|
780
|
+
Deliberate residuals: Vibe isolates the workspace but retains ambient HOME for its authenticated CLI state; unpriced credit/subscription models are disclosed but cannot be enforced by a dollar budget; provider/requested model substitutions are available in raw JSON but not yet summarized in the human report.
|
|
781
|
+
|
|
793
782
|
### v3.5.0
|
|
794
783
|
|
|
795
784
|
- **The catalog now distinguishes an advisory panel's normal premium tier from subscription-only maximum targets.** `get_model_for_tier()` accepts an explicit transport, while `apply_model_tier maximum` selects K3-256k and MiniMax M3 and promotes Qwen3.8-Max only when `QWEN3_USE_API=true`, the wire is OpenAI-compatible, the endpoint is a Token Plan `/chat/completions` URL, and a key is present (`scripts/config.sh:581-737`). The Qwen effort is marked tier-managed, cleared when leaving maximum, and never overwrites a user pin. Gemini 3.7, Fable 5, and the new Mistral API IDs remain catalogued opt-ins after their exact transports failed authentication or timed out; normal runs do not inherit delegation-kit's routing gates.
|