ai-consultants 2.14.1 → 2.17.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +115 -26
- package/README.md +52 -37
- package/SKILL.md +9 -7
- package/docs/COST_RATES.md +55 -51
- package/docs/SETUP.md +23 -13
- package/docs/cost_rates.json +364 -100
- package/docs/releases/v2.14.2.md +77 -0
- package/docs/releases/v2.15.0.md +72 -0
- package/docs/releases/v2.15.1.md +80 -0
- package/docs/releases/v2.16.0.md +71 -0
- package/docs/releases/v2.17.0.md +62 -0
- package/docs/releases/v2.17.1.md +46 -0
- package/docs/releases/v2.17.2.md +51 -0
- package/package.json +5 -2
- package/references/configuration.md +48 -12
- package/scripts/build_context.sh +39 -6
- package/scripts/config.sh +85 -32
- package/scripts/configure.sh +2 -2
- package/scripts/consult_all.sh +56 -36
- package/scripts/doctor.sh +17 -3
- package/scripts/lib/api.sh +1 -1
- package/scripts/lib/common.sh +32 -3
- package/scripts/lib/costs.sh +17 -1
- package/scripts/lib/orchestration.sh +374 -0
- package/scripts/lib/reflection.sh +5 -2
- package/scripts/peer_review.sh +1 -1
- package/scripts/query_claude.sh +2 -2
- package/scripts/query_gemini.sh +26 -9
- package/scripts/setup_wizard.sh +5 -5
- package/scripts/synthesize.sh +7 -0
- package/scripts/test_context_optimization.sh +35 -2
- package/scripts/test_orchestration.sh +203 -0
- package/scripts/test_suite.sh +59 -6
- package/scripts/test_user_config.sh +43 -0
package/CLAUDE.md
CHANGED
|
@@ -6,7 +6,7 @@ AI Consultants is a multi-model AI deliberation system that queries up to 15 AI
|
|
|
6
6
|
|
|
7
7
|
**Self-Exclusion**: The invoking agent is automatically excluded from the panel to prevent self-consultation. Claude Code won't query Claude, Codex CLI won't query Codex, etc.
|
|
8
8
|
|
|
9
|
-
**Version**: 2.
|
|
9
|
+
**Version**: 2.17.2
|
|
10
10
|
|
|
11
11
|
## Distribution
|
|
12
12
|
|
|
@@ -54,6 +54,7 @@ ai-consultants/
|
|
|
54
54
|
│ ├── schema.json # JSON output schema
|
|
55
55
|
│ ├── voting.sh # Voting/consensus + confidence intervals
|
|
56
56
|
│ ├── routing.sh # Smart routing + cost-aware routing
|
|
57
|
+
│ ├── orchestration.sh # Dynamic orchestration planner + shapes (v2.16)
|
|
57
58
|
│ ├── session.sh # Session management
|
|
58
59
|
│ ├── costs.sh # Cost tracking + response limits
|
|
59
60
|
│ ├── cache.sh # Semantic caching (v2.3)
|
|
@@ -154,13 +155,15 @@ Each consultant must produce JSON with this minimum structure:
|
|
|
154
155
|
1. `consult_all.sh` receives query, optional files, and flags (`--preset`, `--strategy`)
|
|
155
156
|
2. Applies preset if specified (`apply_preset()` in config.sh)
|
|
156
157
|
3. Classifies the question (`classify_question.sh`)
|
|
157
|
-
4.
|
|
158
|
-
5.
|
|
159
|
-
6.
|
|
160
|
-
7.
|
|
161
|
-
8.
|
|
162
|
-
9.
|
|
163
|
-
10.
|
|
158
|
+
4. Plans orchestration: complexity + intent → shape (`select_orchestration_shape()` in `lib/orchestration.sh`); `ORCHESTRATION_MODE=fixed` bypasses to the legacy path
|
|
159
|
+
5. Selects consultants (smart routing or all)
|
|
160
|
+
6. Launches parallel queries (`query_*.sh`)
|
|
161
|
+
7. Deliberates per the chosen shape: convergence loop / adversarial gate / tournament / exhaustive sweep (`run_orchestration()`), or fixed debate rounds in `fixed` mode
|
|
162
|
+
8. Calculates voting/consensus with confidence intervals (`lib/voting.sh`)
|
|
163
|
+
9. Checks for panic mode triggers (`lib/common.sh`)
|
|
164
|
+
10. Generates synthesis with selected strategy (`synthesize.sh`)
|
|
165
|
+
11. Optionally runs peer review (`peer_review.sh`)
|
|
166
|
+
12. Produces final report
|
|
164
167
|
|
|
165
168
|
## v2.2 Features
|
|
166
169
|
|
|
@@ -223,7 +226,7 @@ OLLAMA_MODEL=codellama ./scripts/consult_all.sh "question"
|
|
|
223
226
|
```
|
|
224
227
|
|
|
225
228
|
Configuration in `config.sh`:
|
|
226
|
-
- `OLLAMA_MODEL` - Model to use (default:
|
|
229
|
+
- `OLLAMA_MODEL` - Model to use (default: hf.co/prithivMLmods/VibeThinker-3B-GGUF)
|
|
227
230
|
- `OLLAMA_HOST` - Server URL (default: http://localhost:11434)
|
|
228
231
|
- `OLLAMA_TIMEOUT` - Timeout in seconds (default: 300)
|
|
229
232
|
|
|
@@ -367,7 +370,7 @@ New environment variables in `config.sh`:
|
|
|
367
370
|
|
|
368
371
|
| Variable | Default | Description |
|
|
369
372
|
|----------|---------|-------------|
|
|
370
|
-
| `GEMINI_USE_API` | false | Use Google AI API instead of
|
|
373
|
+
| `GEMINI_USE_API` | false | Use Google AI API instead of the agy CLI |
|
|
371
374
|
| `CODEX_USE_API` | false | Use OpenAI API instead of codex CLI |
|
|
372
375
|
| `CLAUDE_USE_API` | false | Use Anthropic API instead of claude CLI |
|
|
373
376
|
| `MISTRAL_USE_API` | false | Use Mistral API instead of vibe CLI |
|
|
@@ -414,9 +417,9 @@ Three tiers of models are available for each consultant, configurable via `apply
|
|
|
414
417
|
|
|
415
418
|
| Tier | Description | Example Models |
|
|
416
419
|
|------|-------------|----------------|
|
|
417
|
-
| **premium** | Latest flagship models | claude-opus-4-
|
|
418
|
-
| **standard** | Good quality at reasonable cost | claude-sonnet-4-6,
|
|
419
|
-
| **economy** | Optimized for speed and low cost | claude-haiku-4-5,
|
|
420
|
+
| **premium** | Latest flagship models | claude-opus-4-8, Gemini 3.1 Pro (High), gpt-5.5 |
|
|
421
|
+
| **standard** | Good quality at reasonable cost | claude-sonnet-4-6, Gemini 3.5 Flash (High), gpt-5.4 |
|
|
422
|
+
| **economy** | Optimized for speed and low cost | claude-haiku-4-5, Gemini 3.5 Flash (Low), gpt-5.4-nano |
|
|
420
423
|
|
|
421
424
|
**Default models are now premium tier** for maximum quality.
|
|
422
425
|
|
|
@@ -428,7 +431,7 @@ apply_model_tier "standard" # Set all consultants to standard models
|
|
|
428
431
|
apply_model_tier "economy" # Set all consultants to economy models
|
|
429
432
|
|
|
430
433
|
# Get model for a specific consultant and tier (v2.8.1)
|
|
431
|
-
get_model_for_tier "gemini" "premium" # →
|
|
434
|
+
get_model_for_tier "gemini" "premium" # → Gemini 3.1 Pro (High)
|
|
432
435
|
get_model_for_tier "claude" "economy" # → claude-haiku-4-5
|
|
433
436
|
```
|
|
434
437
|
|
|
@@ -446,25 +449,25 @@ Three new presets leverage the model tiers:
|
|
|
446
449
|
./scripts/consult_all.sh --preset fast "quick syntax question"
|
|
447
450
|
```
|
|
448
451
|
|
|
449
|
-
### Premium Model Defaults (
|
|
452
|
+
### Premium Model Defaults (June 2026)
|
|
450
453
|
All consultants now use premium models by default:
|
|
451
454
|
|
|
452
455
|
| Consultant | Default Model |
|
|
453
456
|
|------------|---------------|
|
|
454
|
-
| Claude | claude-opus-4-
|
|
455
|
-
| Gemini |
|
|
457
|
+
| Claude | claude-opus-4-8 |
|
|
458
|
+
| Gemini | Gemini 3.1 Pro (High) (via agy CLI) |
|
|
456
459
|
| Codex | gpt-5.5 |
|
|
457
460
|
| Mistral | mistral-large-3 |
|
|
458
|
-
| Cursor | composer-2 |
|
|
461
|
+
| Cursor | composer-2.5 |
|
|
459
462
|
| DeepSeek | deepseek-v4-pro |
|
|
460
|
-
| GLM | glm-5.
|
|
463
|
+
| GLM | glm-5.2 |
|
|
461
464
|
| Grok | grok-4.3 |
|
|
462
|
-
| Qwen3 | qwen3.
|
|
465
|
+
| Qwen3 | qwen3.7-max |
|
|
463
466
|
| Kimi | kimi-code/kimi-for-coding |
|
|
464
|
-
| Aider |
|
|
467
|
+
| Aider | qwen3-coder:free |
|
|
465
468
|
| MiniMax | MiniMax-M2.7 |
|
|
466
469
|
| Kilo | auto |
|
|
467
|
-
| Ollama |
|
|
470
|
+
| Ollama | hf.co/prithivMLmods/VibeThinker-3B-GGUF |
|
|
468
471
|
|
|
469
472
|
Override with environment variables: `CLAUDE_MODEL`, `GEMINI_MODEL`, `CODEX_MODEL`, `KIMI_MODEL`, etc.
|
|
470
473
|
|
|
@@ -652,9 +655,10 @@ for f in scripts/*.sh scripts/lib/*.sh; do bash -n "$f" && echo "OK: $f"; done
|
|
|
652
655
|
| `ENABLE_PANIC_MODE` | auto | Panic mode trigger (v2.2) |
|
|
653
656
|
| `PANIC_CONFIDENCE_THRESHOLD` | 5 | Panic threshold (v2.2) |
|
|
654
657
|
| `ENABLE_OLLAMA` | false | Local model support (v2.2) |
|
|
655
|
-
| `OLLAMA_MODEL` |
|
|
656
|
-
| `CLAUDE_MODEL` | claude-opus-4-
|
|
657
|
-
| `GEMINI_MODEL` |
|
|
658
|
+
| `OLLAMA_MODEL` | hf.co/prithivMLmods/VibeThinker-3B-GGUF | Ollama model (v2.17 default) |
|
|
659
|
+
| `CLAUDE_MODEL` | claude-opus-4-8 | Claude model (v2.5) |
|
|
660
|
+
| `GEMINI_MODEL` | Gemini 3.1 Pro (High) | Gemini agy CLI model (v2.15) |
|
|
661
|
+
| `GEMINI_API_MODEL` | gemini-3.1-pro-preview | Gemini API-mode model ID (v2.15) |
|
|
658
662
|
| `CODEX_MODEL` | gpt-5.5 | Codex model (v2.5) |
|
|
659
663
|
| `MISTRAL_MODEL` | mistral-large-3 | Mistral model (v2.5) |
|
|
660
664
|
| `SYNTHESIS_STRATEGY` | majority | Synthesis strategy (v2.2) |
|
|
@@ -677,7 +681,7 @@ for f in scripts/*.sh scripts/lib/*.sh; do bash -n "$f" && echo "OK: $f"; done
|
|
|
677
681
|
|
|
678
682
|
## External Dependencies
|
|
679
683
|
|
|
680
|
-
- `
|
|
684
|
+
- `agy` CLI - Antigravity CLI (Gemini consultant; successor to the deprecated Gemini CLI, v2.15)
|
|
681
685
|
- `codex` CLI - OpenAI Codex
|
|
682
686
|
- `vibe` CLI - Mistral Vibe
|
|
683
687
|
- `kilocode` CLI - Kilo Code
|
|
@@ -822,6 +826,91 @@ curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/s
|
|
|
822
826
|
|
|
823
827
|
## Changelog
|
|
824
828
|
|
|
829
|
+
### v2.17.2
|
|
830
|
+
- **SECURITY: revert v2.17.1's `allow_absolute=true` for context files — it opened a secret-exfiltration surface (caught in `/code-review max`).** v2.17.1 fixed the macOS `/private/tmp` drop by letting `build_context.sh` accept *any* absolute context path behind `validate_file_path`'s prefix-only blocklist (`/etc /root /var/log /proc /sys /dev`). That blocklist covers no home secrets, so a context arg of `~/.ssh/id_rsa`, `~/.aws/credentials`, `~/.netrc`, or `~/.config/gh/hosts.yml` was read verbatim and **sent to the external AI providers**. It also matched literal non-canonical prefixes, so `/private/etc/master.passwd` (the real `/etc` on macOS, via the same symlink aliasing) bypassed the `/etc` rule.
|
|
831
|
+
- **Fix (correct altitude — allowlist, not blocklist)**: context files are now accepted only when (a) relative and in-tree (`validate_file_path ... false` — rejects absolute and `..`), or (b) under a recognized temp root via the new `_is_temp_path` helper (`/tmp`, `/private/tmp`, `$TMPDIR`, and the `/private`-prefixed macOS alias of `$TMPDIR`; `..` rejected so a temp prefix can't traverse out). Still fixes the original macOS scratch-file regression (Claude Code writes `/private/tmp/...`) **without** widening to the whole filesystem. Verified: `/tmp`, `/private/tmp`, `$TMPDIR/...`, relative in-tree → accepted; `~/.ssh/id_rsa`, `/etc/passwd`, `/private/etc/master.passwd`, `/tmp/../etc/passwd` → rejected.
|
|
832
|
+
- **`build_context.sh` OUTPUT_FILE**: removed the `/tmp/*` short-circuit that jumped past `validate_file_path` (so `/tmp/../etc/x` bypassed the `..` check). Output paths now always run through `validate_file_path "$OUTPUT_FILE" "true"` — absolute allowed (output lives under the XDG cache or `/tmp`), traversal + sensitive-path guards enforced. The two validation sites are intentionally different now (OUTPUT is tool-chosen; context files are untrusted input) and the comments say so.
|
|
833
|
+
- **Test**: rewrote `test_context_optimization.sh` Test 15 as a *boundary* test — temp-root accepted, `~/.ssh/id_rsa` rejected (exfiltration blocked), `/etc/passwd` rejected. Uses the auto-cleaned `$_TMPDIR` (no `$HOME` litter) and `QUESTION_CATEGORY=SECURITY` to skip the project-tree `find` the assertions don't need. Fixed the stale suite-header comment that contradicted the new behavior. 8 suites pass; shellcheck clean.
|
|
834
|
+
- **Note**: v2.17.1 was tagged and GitHub-released but **never published to npm** (npm stayed at 2.17.0), so no npm user received the vulnerable version. Publish **2.17.2**, not 2.17.1.
|
|
835
|
+
|
|
836
|
+
### v2.17.1
|
|
837
|
+
- **Fix: context files at absolute paths outside `/tmp` were silently dropped.** `build_context.sh`'s context-file gate accepted only relative paths or a literal `/tmp/*` prefix (`if [[ "$_PARSED_PATH" == /tmp/* ]] || validate_file_path "$_PARSED_PATH" "false"`). On macOS `/tmp` is a symlink to `/private/tmp`, so Claude Code scratch files arrive as `/private/tmp/...` and matched neither branch → `log_warn "Skipping invalid file path"` and the file was excluded, with `build_context.sh` falling back to a generic repo auto-context. Net effect: a consultation looked like it ran with the user's context but the consultants never received it. (Reported from a real session where the passed context was dropped and an auto-context substituted.)
|
|
838
|
+
- **Fix**: the gate now uses `validate_file_path "$_PARSED_PATH" "true"` (allow absolute), mirroring the OUTPUT_FILE handling a few lines above. Context files are explicitly user/agent-provided, so absolute paths are legitimate; the sensitive-path blocklist (`/etc /root /var/log /proc /sys /dev`), path-traversal (`..`), and null-byte guards in `validate_file_path` still apply. Verified: `/private/tmp/...`, `/tmp/...`, relative, and `$HOME/...` paths accepted; `/etc/passwd`, `/var/log/...`, `../../etc/...` still rejected.
|
|
839
|
+
- **Test**: `test_context_optimization.sh` Test 15 — an absolute context path outside `/tmp` (a `$HOME` file) is now included in the built context, and `/etc/passwd` is still skipped. 8 suites pass; shellcheck clean.
|
|
840
|
+
- **Note (not a code change)**: the same session also showed Gemini/Codex failing immediately — because the *installed* skill at `~/.claude/skills/ai-consultants` was **v2.10.0** (pre-agy migration), so Gemini called the deprecated `gemini` binary. Updating the installed skill to ≥v2.15 (now v2.17.1) resolves that; it's an install-staleness issue, not a current-code bug.
|
|
841
|
+
|
|
842
|
+
### v2.17.0
|
|
843
|
+
- **Model catalog refresh (June 2026)** across all three tiers + cost rates, for every agent. Source of truth stays `config.sh::get_model_for_tier` (+ default `*_MODEL` vars) mirrored by `docs/cost_rates.json` (`model_tiers` + `consultant_fallbacks` + per-1K `models` rates). CLI-addressed models verified by querying the installed binaries (`agy models`, `agent --list-models`, kimi config); API/provider models + pricing researched against official sources. Superseded IDs kept in `cost_rates.json` for historical/pinned lookups.
|
|
844
|
+
- **Changed models** (premium / standard / economy):
|
|
845
|
+
- **codex**: gpt-5.5 / **gpt-5.4** (was gpt-5.3) / **gpt-5.4-nano** (was gpt-4o-mini).
|
|
846
|
+
- **cursor**: **composer-2.5** / **composer-2** / **gemini-3-flash** (was composer-2 / composer-1.5 / gemini-2.0-flash).
|
|
847
|
+
- **deepseek**: deepseek-v4-pro / **deepseek-v4-flash** ×2 (was deepseek-v3.2 / deepseek-chat; chat+reasoner deprecate 2026-07-24).
|
|
848
|
+
- **glm**: **glm-5.2** premium+standard (was glm-5.1) / glm-4-flash.
|
|
849
|
+
- **grok**: grok-4.3 / **grok-4.1-fast** ×2 (was grok-3 / grok-3-mini).
|
|
850
|
+
- **qwen3**: **qwen3.7-max** (was qwen3.6-plus) / qwen3.6-35b-a3b / qwen3-32b.
|
|
851
|
+
- **aider**: **qwen3-coder:free** (was nvidia/nemotron…:free) / **gpt-5.4** / **gpt-5.4-nano**.
|
|
852
|
+
- **ollama**: **hf.co/prithivMLmods/VibeThinker-3B-GGUF** default (was qwen2.5-coder:32b); standard/economy keep llama3.3/llama3.2.
|
|
853
|
+
- **Unchanged (verified current)**: claude, gemini (agy display names — no newer Gemini), mistral, minimax, kimi/amp/kilo.
|
|
854
|
+
- **Cost-rate corrections (per-1K)**: `gpt-5.5` → 0.005/0.030 (was understated 0.003/0.012), `deepseek-v4-pro` → 0.000435/0.00087, `minimax-m2.7` → 0.00025/0.001, `composer-2` → 0.0005/0.0025, Gemini Flash → 0.0015/0.009, Gemini 3.1 Pro → 0.002/0.012, mistral-medium → 0.001/0.003, devstral-small-2 → 0 (free). New IDs added with researched per-1K rates. `COST_RATES.md` regenerated to match the JSON.
|
|
855
|
+
- **Estimates flagged**: `glm-5.2` (mirrors glm-5.1) and `qwen3.7-max` (mirrors qwen3-max) pending official pricing.
|
|
856
|
+
- **Docs synced**: README, configuration.md, .env.example (+ fixed pre-existing GROK/DEEPSEEK default drift), SETUP.md, CLAUDE.md tables. Tests: tier assertions updated +10 new; 8 suites pass; shellcheck clean.
|
|
857
|
+
- **Found in `/code-review max` (workflow-backed, 61 agents): case-insensitive cost lookup.** `lib/costs.sh::get_rate_from_file` did an exact-case jq key match, but callers lowercase the model name first — so every **mixed-case** rate key silently missed and fell to `default_rate` ($0.005/$0.015 per 1K). This diff newly tripped it with the Ollama default `hf.co/prithivMLmods/VibeThinker-3B-GGUF` (a free local model billed at $0.02/query → wrong session cost reports, pre-run estimates, and budget halts under `ENABLE_BUDGET_LIMIT`), and it had silently rendered the **Gemini** agy display-name rates inert since v2.15 (every Gemini consultation cost-reported at default, not its real rate). Fixed by matching keys case-insensitively (`ascii_downcase` both sides via `first(.models|to_entries[]|select(...))`) — lowercase keys (minimax/gpt-5.5) still match; mixed-case (Gemini, VibeThinker) now resolve. Verified: VibeThinker → $0, Gemini 3.1 Pro → $0.014 for 1k+1k. Regression tests added (VibeThinker=0, Gemini resolves, local model free).
|
|
858
|
+
- **Also from review**: `estimate_query_cost`/`format_cost` now restore the leading zero bc drops on sub-1 values (".014000"/".03¢" → "0.014000"/"0.03¢") — surfaced by the diff's many sub-cent rates. README roster + `references/configuration.md` Grok/DeepSeek default cells synced to grok-4.3 / deepseek-v4-pro (the partial sync had left them stale). `config.sh` codex + ollama inline comments corrected to the new defaults. `cost_rates.json` premium-block date comment → Jun 2026. (Skipped as deliberate: re-ordering legacy entries under `_comment_legacy`, and the `gemini-3-flash-preview` vs `gemini-3-flash` coexistence — the former is the historical Gemini-API id, the latter Cursor's economy model, kept separately on purpose.)
|
|
859
|
+
|
|
860
|
+
### v2.16.0
|
|
861
|
+
- **Dynamic orchestration engine** — the fixed `classify → query → debate(N fixed rounds) → synth` pipeline becomes adaptive, inspired by Claude Code's dynamic workflows but implemented entirely in standalone bash (no Workflow tool / Claude-Code dependency). A planner picks an orchestration **shape** per question; debate becomes a **convergence loop**.
|
|
862
|
+
- **New module `lib/orchestration.sh`** (sourced by `consult_all.sh` after voting/costs). Public surface:
|
|
863
|
+
- `detect_intent <query>` → `advise|compare|exhaustive` (heuristic regex over the query; zero-dep).
|
|
864
|
+
- `select_orchestration_shape <category> <complexity> <intent>` → `quick|converge|adversarial|tournament|exhaustive|fixed`. Auto resolution priority: explicit `ORCHESTRATION_MODE` override → intent (`exhaustive`/`compare`) → category (`SECURITY`→adversarial) → complexity (≤`COMPLEXITY_THRESHOLD_SIMPLE`→quick, else converge).
|
|
865
|
+
- `run_orchestration <shape> <dir> <category>` dispatcher; skips multi-round shapes when `SUCCESS_COUNT ≤ 1`.
|
|
866
|
+
- Pure decision helpers, unit-tested without live CLIs: `_convergence_should_stop <score> <prev> <target> <epsilon>` (→`converged|stalled|continue`) and `_approach_signature <dir>` (sorted-unique approach set, the loop-until-dry stop signal).
|
|
867
|
+
- **Convergence loop (`run_convergence_loop`)** replaces the fixed `for round=2..DEBATE_ROUNDS`. Stops on: consensus ≥ `CONVERGENCE_TARGET_CONSENSUS` (converged), per-round gain < `CONVERGENCE_STALL_EPSILON` (stalled), `CONVERGENCE_MAX_ROUNDS` reached, or budget. `min_rounds` param forces ≥N critique rounds even on early consensus (adversarial uses 2). Reuses `calculate_consensus_score` (voting.sh) for the stop signal and the extracted `_apply_debate_round` helper (the legacy loop body, now shared) for execution+merge. Trajectory + stop reason written to `orchestration.json`.
|
|
868
|
+
- **Shapes**: `quick` (no debate), `converge` (loop to consensus), `adversarial` (forced critique round + peer review as refutation gate — `SECURITY`), `tournament` (converge, then synthesis declares one winner via `ORCHESTRATION_SELECT_WINNER` directive in `synthesize.sh`), `exhaustive` (`run_exhaustive_loop`: iterate until a round adds no new `.response.approach`).
|
|
869
|
+
- **`consult_all.sh` integration**: after classification, computes `QUERY_COMPLEXITY` (`calculate_query_complexity`, costs.sh) + `QUERY_INTENT` + `ORCH_SHAPE`; the debate block dispatches on `ORCHESTRATION_MODE` (`fixed` → byte-equivalent legacy loop via `_apply_debate_round`; else → `run_orchestration`). Adversarial shape force-enables `ENABLE_PEER_REVIEW`. Shape/complexity/intent recorded in `optimization_metrics.json`.
|
|
870
|
+
- **Config (all back-compat)**: `ORCHESTRATION_MODE` (default `auto`), `CONVERGENCE_MAX_ROUNDS` (4), `CONVERGENCE_TARGET_CONSENSUS` (75), `CONVERGENCE_STALL_EPSILON` (5), `ENABLE_ADVERSARIAL_VERIFY` (true). `ORCHESTRATION_MODE=fixed` restores the exact pre-2.16 pipeline.
|
|
871
|
+
- **Behavioral change (default)**: with `auto` the panel may run more or fewer rounds than the old fixed default, driven by consensus — complex/contested questions iterate further, simple ones short-circuit to `quick`. Every round still passes `enforce_budget`, so `MAX_SESSION_COST` continues to cap spend. Set `ORCHESTRATION_MODE=fixed` to opt out.
|
|
872
|
+
- **Minor robustness**: `_apply_debate_round` swallows a failed `debate_round.sh` (`|| true`) instead of aborting the whole consultation under `set -e` (the legacy inline `$()` capture would abort). Applies to both `fixed` and dynamic paths.
|
|
873
|
+
- **Convergence actually converges (fixed in `/code-review max`).** The round file carries the consultant's *updated* top-level `.response.approach` (`build_structured_response` uses `$inner.response`), but the legacy merge grafts only `.debate` (which `build_structured_response` doesn't even preserve → null). Since `calculate_consensus_score` reads `.response.approach`, the consensus signal was invariant under debate → the loop stalled after exactly one round every time. Fix: `_apply_debate_round` gained a `promote` flag; the dynamic loops (`promote=true`) adopt the round file's post-debate `.response`/`.confidence` so consensus reflects evolved positions. The legacy `fixed` path (`promote=false`) keeps the original `.debate`-only graft. Verified: a stubbed converging panel now moves 0 → 100 (`converged`), where before it logged `stalled` at round 1. Regression tests assert promote adopts the updated approach and legacy preserves the original.
|
|
874
|
+
- **ARCHITECTURE keeps its mandatory debate (fixed in `/code-review max`).** The legacy pipeline always debated `SECURITY` *and* `ARCHITECTURE`; the first planner cut only special-cased `SECURITY`→adversarial, letting `ARCHITECTURE` fall through to `converge` (min_rounds=1), which early-exits with zero debate rounds when the fan-out already agrees. Now `ARCHITECTURE` pins to `converge` (never `quick`) and `run_orchestration` forces `min_rounds=2` for both mandatory categories, so they always run ≥1 critique round — restoring pre-2.16 behavior.
|
|
875
|
+
- **Tests**: new `scripts/test_orchestration.sh` (32 assertions, 13 tests) — intent detection, shape selection (intent/category/complexity priority + overrides + fixed bypass + threshold boundary + ARCHITECTURE mandatory-debate), convergence stop decision (converged/stalled incl. negative-gain/continue), `_apply_debate_round` promote-vs-legacy merge, `_approach_signature` over fixtures, `ORCHESTRATION_MODE=auto` default. Auto-discovered by `test_all.sh` → now **8 suites**. Convergence loop smoke-tested end-to-end with a stubbed `debate_round.sh` (0 → 100 converged).
|
|
876
|
+
|
|
877
|
+
### v2.15.1
|
|
878
|
+
- **Markdown-fence parsing fix — Gemini's default model produced degraded (fallback) responses under v2.15.0.** The v2.15.0 migration note claimed "agy prints the model JSON directly (top-level `.response`)"; verified against agy 1.0.10, that is only true for some models (e.g. `Gemini 3.5 Flash (Low)` returns bare JSON). The **default** `Gemini 3.1 Pro (High)` wraps the envelope in a ```` ```json … ``` ```` markdown fence, so `process_consultant_response` failed the `.response.summary` jq probe and fell through to `build_fallback_response` — emitting `summary: "Unstructured response - see detailed"`, `confidence: 5`, and empty pros/cons for every Gemini reply, with the real structured envelope buried as a fenced string inside `.detailed`. Fix: new shared helper `lib/common.sh::strip_json_fence` — returns the text unchanged when it already parses as JSON (a real fence makes the text invalid JSON, so the gate reliably detects it), otherwise drops pure fence-marker lines (`/^[[:space:]]*```[[:alnum:]]*[[:space:]]*$/d`). Consultant-agnostic; the bare-JSON path (Flash, every other consultant) is untouched. Verified end-to-end against live agy: `Gemini 3.1 Pro (High)` now yields a populated `build_structured_response` (summary/approach/pros/cons/confidence=10). Regression test in `test_functions.sh::test_process_consultant_response_fence`.
|
|
879
|
+
- **Fence fix applied to ALL agy output paths (caught in `/code-review max`).** The fence isn't only seen by `process_consultant_response` — two other paths consume agy output directly and were still corrupting it:
|
|
880
|
+
- **`lib/reflection.sh::run_reflection_cycle`**: `_exec_consultant` returns raw (fenced) agy output. The critique's `jq -r '.needs_refinement'` returned `""` on fenced text (so the early-stop never fired and every cycle ran), and the refined response was written back as fenced non-JSON, making downstream voting/synthesis/report `jq` reads silently fall back — discarding the refined Gemini answer. Now de-fences `critique` and `refined` via `strip_json_fence` at the consumption points.
|
|
881
|
+
- (`peer_review.sh` already had its own `extract_json_from_response` fence handler applied before aggregation — verified safe, left as-is.)
|
|
882
|
+
- **Synthesis via agy was broken (caught in `/code-review max`).** `lib/common.sh::build_synthesis_args` gemini branch produced a bare `agy` invocation (`SYNTHESIS_ARGS=("${GEMINI_CMD:-agy}")`), invoked as `echo "$prompt" | agy`. Unlike codex (`--full-auto`) and claude (`--print`), agy with no `--print`/`-p` launches its **interactive** session and never reads the piped prompt, so synthesis hung/produced nothing whenever agy was the chosen synthesizer (reachable e.g. when Claude Code is the invoking agent and agy is on PATH). Fixed to `("${GEMINI_CMD:-agy}" "-p" "-")` + `--model`, mirroring `query_gemini.sh`.
|
|
883
|
+
- **Comment rot (caught in `/code-review max`)**: `query_gemini.sh` comments claimed agy emits raw JSON "no envelope to unwrap"; corrected to note the default model fences its JSON and that the fence is stripped centrally.
|
|
884
|
+
- **Gemini transport auto-resolution — makes the Gemini consultant work out-of-the-box for npm/npx users.** v2.15.0 left Gemini enabled by default (`ENABLE_GEMINI=true`) in CLI mode (`GEMINI_USE_API=false`), but the agy (Antigravity) CLI cannot be installed via npm (it's a `curl|bash` binary into `~/.local/bin/agy`) and is OAuth-only (no headless/API-key auth). Net effect: every fresh `npx ai-consultants` run silently dropped Gemini from the panel (`consult_all.sh` marks it FAILED and continues) — *even when the user had `GEMINI_API_KEY` exported*, because API mode was opt-in with no auto-detection. The API path (pure `curl` + key, no binary, no browser) is the npm-friendly one but was off by default.
|
|
885
|
+
- **`config.sh`**: removed the hardcoded `GEMINI_USE_API="${GEMINI_USE_API:-false}"` at the mode-switching block (`scripts/config.sh:69-72`). The mode is now auto-resolved in the Gemini config section (`scripts/config.sh:113-131`), *after* `GEMINI_CMD`/`GEMINI_API_KEY` are known: when `GEMINI_USE_API` is unset, pick `true` if `GEMINI_API_KEY` is present, else `false` (agy CLI). An explicit `GEMINI_USE_API=true/false` is always honored (back-compat, via the `${GEMINI_USE_API+x}` set-vs-unset guard). Idempotent across the 15-30 `config.sh` re-sources per consultation: once resolved and `export`ed, subsequent sources see it as user-set. Only Gemini auto-resolves; the other four switchable agents (Codex, Claude, Mistral, Qwen3) keep their hardcoded `:-false` defaults since their CLIs are npm/pip-installable.
|
|
886
|
+
- **`doctor.sh`**: `check_cli_consultant` now skips the CLI install check when the consultant is in API mode (`${env_var}_USE_API == true`) — previously it would flag a missing CLI as a hard `✗ NOT INSTALLED` failure regardless of mode, which after auto-resolution would false-fail for every key-only Gemini user. This also fixes the same pre-existing latent false-positive for Codex/Claude/Mistral/Qwen3 when those run in API mode. Additionally, when Gemini *is* in CLI mode and `agy` is missing, the failure now prints a `tip: set GEMINI_API_KEY to use API mode (no CLI install needed)` remediation pointing at the npm-friendly path.
|
|
887
|
+
- **Tests**: `test_user_config.sh` +4 assertions (Tests 19-22): auto-API with key, auto-CLI without key, explicit `false` wins over a present key, explicit `true` honored without a key. Suite now 42 checks; all 7 suites pass.
|
|
888
|
+
- **No breaking change**: agy/OAuth users without a key keep CLI mode; anyone who pinned `GEMINI_USE_API` keeps their value. The only behavioral change is that a key-only environment now reaches Gemini over the API instead of silently dropping it.
|
|
889
|
+
|
|
890
|
+
### v2.15.0
|
|
891
|
+
- **Gemini consultant migrated from the Gemini CLI to the Antigravity CLI (`agy`)**. Google deprecated the Gemini CLI on 2026-06-18 (transitioning all individual/Pro/Ultra users to Antigravity CLI); the `gemini` binary stops serving requests for non-enterprise accounts. The consultant stays "Gemini" / "The Architect" (ID 1) — only the transport binary, model addressing, and auth change. Verified against `agy` 1.0.10.
|
|
892
|
+
- **`config.sh`**: `GEMINI_CMD` default `gemini` → `agy` (`scripts/config.sh:106`); `GEMINI_MODEL` default `gemini-3.1-pro-preview` → `Gemini 3.1 Pro (High)` (agy addresses models by display name, not API ID); new `GEMINI_API_MODEL` (default `gemini-3.1-pro-preview`) decouples the API-mode model ID from the CLI display name. `get_model_for_tier "gemini"` now returns agy display names: premium `Gemini 3.1 Pro (High)`, standard `Gemini 3.5 Flash (High)`, economy `Gemini 3.5 Flash (Low)`.
|
|
893
|
+
- **`query_gemini.sh`** — three verified flag/parse changes:
|
|
894
|
+
- `-m` → `--model` (agy has no `-m` short alias).
|
|
895
|
+
- **Dropped `--output-format json`** — agy has no such flag and prints the model's response as plain text. Because the persona instruction already forces the model to emit our JSON schema, that plain text *is* the JSON envelope we need.
|
|
896
|
+
- **Dropped the `native_json_field="response"` argument** to `process_consultant_response`. The old Gemini CLI wrapped output as `{"response":"<text>","stats":{…}}` and we extracted `.response`; agy's output is the model JSON directly (top-level `.response`), so extracting `.response` would strip a level and force every structured reply into the fallback path. Confirmed end-to-end: a real `agy` call now yields a correct `build_structured_response` (consultant=Gemini, model="Gemini 3.1 Pro (High)", persona=The Architect, populated pros/cons/confidence).
|
|
897
|
+
- API-mode branch now passes `$GEMINI_API_MODEL` (not `$GEMINI_MODEL`) to `run_api_mode_query`, since the Google AI endpoint builds `…/${model}:generateContent` and needs an API ID, not a display name.
|
|
898
|
+
- **CLI invocation parity**: same `-m`→`--model` change applied to the two other call sites — `peer_review.sh:150` and `lib/reflection.sh:71`.
|
|
899
|
+
- **Install/auth surfaces** repointed to agy: `doctor.sh` (install hint + `${GEMINI_CMD:-gemini}`→`:-agy` in the two config-dump fallbacks), `setup_wizard.sh`, `configure.sh` (`CLI_AGENT_CMDS`/`CLI_AGENT_HINTS`), `lib/common.sh` synthesis fallbacks (`:-agy`), `.env.example`, `docs/SETUP.md`, `references/configuration.md`. Install is now `curl -fsSL https://antigravity.google/cli/install.sh | bash` (binary lands in `~/.local/bin/agy`). Auth is **OAuth-only** (`agy` with no args → browser sign-in; creds cached); `agy` does **not** honor an API-key env var for headless use — that path remains exclusive to API mode.
|
|
900
|
+
- **Cost catalog**: added `Gemini 3.1 Pro (High)` / `Gemini 3.5 Flash (High)` / `Gemini 3.5 Flash (Low)` keys to `cost_rates.json` (per-1K, mirroring the matching Gemini API tier; agy itself bills via OAuth/subscription) and repointed `consultant_fallbacks.gemini` + all three `model_tiers.*.gemini` to the display names. The old `gemini-3.1-pro-preview`/`gemini-3-flash-preview`/`gemini-2.0-flash` entries are kept for API mode and historical lookups. `COST_RATES.md` synced.
|
|
901
|
+
- **Tests**: `test_suite.sh` tier assertions updated (`get_model_for_tier "gemini" premium/economy` and `get_economic_model "gemini"` now expect the agy display names) — same pattern as the v2.14.2 claude-opus-4-8 bump. All 7 suites pass.
|
|
902
|
+
- **Known follow-up (out of scope, flagged in the release note)**: the *host-side* integration — running ai-consultants **from** Gemini CLI as a slash-command host (`~/.gemini/skills/`, `INVOKING_AGENT=gemini`) — is affected by the same deprecation (Antigravity rebrands Extensions as "plugins", `agy plugin …`). That migration is **not** done here; this release covers only the Gemini *consultant* (the model we query). The relevant README/SETUP host sections were left intact rather than rewritten on unverified plugin mechanics.
|
|
903
|
+
|
|
904
|
+
### v2.14.2
|
|
905
|
+
- **Claude premium tier upgraded `claude-opus-4-7` → `claude-opus-4-8`** (Opus 4.8 released 2026-05-29). Single source of truth is the `premium` case in `config.sh::get_model_for_tier` (`scripts/config.sh:583`) and the `CLAUDE_MODEL` default (`scripts/config.sh:223`); both now resolve to `claude-opus-4-8`. Standard (`claude-sonnet-4-6`) and economy (`claude-haiku-4-5`) tiers are unchanged — Sonnet 4.6 and Haiku 4.5 remain the latest in their classes.
|
|
906
|
+
- **`docs/cost_rates.json`**: added a `claude-opus-4-8` entry; repointed `consultant_fallbacks.claude` and `model_tiers.premium.claude` to it; moved `claude-opus-4-7` into the `_comment_legacy` block so cost lookups for cached/historical responses and pinned overrides still resolve. Value entered in per-1K (see units fix below).
|
|
907
|
+
- **CRITICAL — cost-catalog unit normalization (fixes ~1000× cost overstatement)**: `lib/costs.sh` computes cost as `(token_count / 1000) * rate`, i.e. every value in `cost_rates.json` MUST be USD **per-1K** tokens (the hardcoded fallback table in `costs.sh`, e.g. `claude-3-haiku → 0.00025`, confirms this contract). But the premium/standard blocks had been populated with **per-MTok** dollar figures (`claude-opus-4-7: 5.00`, `claude-sonnet-4-6: 3.00`, `gpt-5.5: 3.00`, …) — so `estimate_query_cost` reported ~1000× the true cost for almost every premium/standard model (e.g. a ~1k-in/1k-out Opus query was reported as **$30** instead of **$0.03**). The economy block (incl. `claude-haiku-4-5: 0.001`) was already correct per-1K. Normalized the **entire** catalog (premium, standard, economy, legacy, `default_rate`) to per-1K by dividing the per-MTok entries by 1000; left the already-correct per-1K entries untouched. Added a `_comment_units` field documenting the contract to prevent regression. Verified: Opus 1k+1k = $0.03, 1M+1M = $30; Haiku 1k+1k = $0.006; gpt-5.5 1M+1M = $15; gemini-pro 1M+1M = $6.25 — all match provider pricing.
|
|
908
|
+
- **`docs/COST_RATES.md`**: Claude rows synced to the corrected per-1K values (`claude-opus-4-8` $0.005/$0.025, `claude-sonnet-4-6` $0.003/$0.015, `claude-haiku-4-5` $0.001/$0.005). Note: non-Claude rows in this human-facing doc may still lag the JSON and should be re-synced in a follow-up — `cost_rates.json` is the runtime source of truth, not this file.
|
|
909
|
+
- **Stale-alias cleanup (pre-existing, fixed in passing)**: five spots still carried pre-v2.10.6 short aliases and now use canonical IDs — `scripts/query_claude.sh` (header comment + `MODEL_USED` fallback), `.env.example` (`CLAUDE_MODEL=opus-4.6`), `references/configuration.md`, and the README "Models by Tier" table all → `claude-opus-4-8`; plus `lib/api.sh::build_anthropic_request` default `sonnet-4.6` → `claude-sonnet-4-6`. All were latent (callers always pass an explicit model / `config.sh` always exports `CLAUDE_MODEL`), but would have surfaced if those scripts were invoked standalone or the example `.env` copied verbatim.
|
|
910
|
+
- **Test**: `scripts/test_suite.sh::test_model_for_tier` premium assertion updated to expect `claude-opus-4-8`. No other test references the Claude premium ID.
|
|
911
|
+
- **Correction of an earlier mis-diagnosis (for the record)**: during development this was first flagged as "Haiku is 1000× *understated*". That was wrong — under the per-1K contract `claude-haiku-4-5: 0.001` is correct ($1/$5 per MTok). The real bug was the *opposite*: premium/standard entries were 1000× *overstated*. The unit normalization above is the actual fix.
|
|
912
|
+
- **Behavioral change**: cost *reporting* now drops ~1000× for premium/standard models (it was massively overstating). Orchestration, routing, and synthesis are unaffected. Note: `ENABLE_COST_AWARE_ROUTING` / budget thresholds compare against these figures, so anyone who tuned `MAX_SESSION_COST` against the old inflated numbers should revisit their threshold. Remaining out-of-scope item: a few legacy per-model rates (e.g. Mistral/DeepSeek/GLM tier orderings) reflect pre-existing catalog figures I couldn't verify against authoritative pricing — only the unit bug was fixed, not per-model price accuracy.
|
|
913
|
+
|
|
825
914
|
### v2.14.1
|
|
826
915
|
- **Pre-commit hook**: `scripts/hooks/pre-commit` runs `shellcheck` on staged `.sh` files under `scripts/` using the exact CI invocation (`-S warning -x -e SC1091,SC1090,SC2034,SC2155`). Mirrors `.github/workflows/ci.yml:32` so the same warnings that fail CI also fail the local commit. Filtered with the regex `^scripts/(lib/)?[^/]+\.sh$` to match the CI glob exactly (test fixtures under `scripts/test_fixtures/` are correctly excluded). Bypass: `git commit --no-verify`.
|
|
827
916
|
- **`scripts/install-hooks.sh`**: idempotent installer wired to `npm run install-hooks`. Backs up any existing different hook to `.git/hooks/pre-commit.backup.<timestamp>` to avoid clobbering contributor customizations (`FORCE=1` skips backup). Silent no-op outside a git checkout, so it's safe to wire to npm `prepare` or similar lifecycle hooks if ever needed.
|
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# AI Consultants v2.
|
|
1
|
+
# AI Consultants v2.17.2
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> **A harness for every question.** A panel of up to 15 frontier models that writes its own playbook per question — fan out, debate to convergence, cross-examine under adversarial review, or run a tournament — and checks its work before it reaches you.
|
|
4
4
|
|
|
5
|
-
[](https://github.com/matteoscurati/ai-consultants)
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
[](https://docs.anthropic.com/en/docs/claude-code/skills)
|
|
8
8
|
[](https://github.com/matteoscurati/ai-consultants)
|
|
@@ -35,14 +35,16 @@
|
|
|
35
35
|
|
|
36
36
|
## Why AI Consultants?
|
|
37
37
|
|
|
38
|
-
|
|
38
|
+
A single model gives you a single guess. AI Consultants gives you a **panel that deliberates — and adapts how it deliberates to the question in front of it.**
|
|
39
39
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
- **
|
|
43
|
-
- **
|
|
44
|
-
- **
|
|
45
|
-
- **
|
|
40
|
+
Instead of running a fixed script, it classifies your question, picks an orchestration **shape**, and iterates until the answers hold up — the way a workflow builds a harness for the task at hand:
|
|
41
|
+
|
|
42
|
+
- **Dynamic orchestration** — the engine chooses the strategy per question: a quick read, a convergence loop, an adversarial refutation gate, a tournament of approaches, or an exhaustive sweep
|
|
43
|
+
- **Convergence, not fixed rounds** — debate iterates until the panel actually agrees (or provably won't), instead of a hardcoded count
|
|
44
|
+
- **Self-checking** — security answers are stress-tested by consultants trying to refute them before anything reaches you
|
|
45
|
+
- **15 frontier models** with distinct personas (Architect, Pragmatist, Devil's Advocate, …)
|
|
46
|
+
- **Confidence-weighted synthesis** — one recommendation, with the dissent and the path it took surfaced so you know how much to trust it
|
|
47
|
+
- **Local + private** — runs fully offline via Ollama when you need it
|
|
46
48
|
|
|
47
49
|
---
|
|
48
50
|
|
|
@@ -405,7 +407,7 @@ INVOKING_AGENT=codex ./scripts/consult_all.sh "Question" # Codex excluded
|
|
|
405
407
|
|
|
406
408
|
| Consultant | CLI | Persona | Focus |
|
|
407
409
|
|------------|-----|---------|-------|
|
|
408
|
-
| **Google Gemini** | `
|
|
410
|
+
| **Google Gemini** | `agy` | The Architect | Design patterns, scalability, enterprise |
|
|
409
411
|
| **OpenAI Codex** | `codex` | The Pragmatist | Simplicity, quick wins, proven solutions |
|
|
410
412
|
| **Mistral Vibe** | `vibe` | The Devil's Advocate | Problems, edge cases, vulnerabilities |
|
|
411
413
|
| **Kilo Code** | `kilocode` | The Innovator | Creativity, unconventional approaches |
|
|
@@ -419,24 +421,24 @@ INVOKING_AGENT=codex ./scripts/consult_all.sh "Question" # Codex excluded
|
|
|
419
421
|
|
|
420
422
|
| Consultant | Default Model | Persona | Focus |
|
|
421
423
|
|------------|---------------|---------|-------|
|
|
422
|
-
| **Qwen3** | qwen3.
|
|
423
|
-
| **GLM** | glm-5.
|
|
424
|
-
| **Grok** | grok-4.
|
|
425
|
-
| **DeepSeek** | deepseek-
|
|
424
|
+
| **Qwen3** | qwen3.7-max | The Analyst | Data-driven analysis |
|
|
425
|
+
| **GLM** | glm-5.2 | The Methodologist | Structured approaches |
|
|
426
|
+
| **Grok** | grok-4.3 | The Provocateur | Challenge conventions |
|
|
427
|
+
| **DeepSeek** | deepseek-v4-pro | The Code Specialist | Algorithms, code generation |
|
|
426
428
|
| **MiniMax** | MiniMax-M2.7 | The Pragmatic Optimizer | Performance, efficiency, pragmatism |
|
|
427
429
|
|
|
428
430
|
### Local Consultants
|
|
429
431
|
|
|
430
432
|
| Consultant | Default Model | Persona | Focus |
|
|
431
433
|
|------------|---------------|---------|-------|
|
|
432
|
-
| **Ollama** |
|
|
434
|
+
| **Ollama** | hf.co/prithivMLmods/VibeThinker-3B-GGUF | The Local Expert | Privacy-first, zero API cost |
|
|
433
435
|
|
|
434
436
|
### Installing Consultant CLIs
|
|
435
437
|
|
|
436
438
|
At least 2 consultant CLIs are required:
|
|
437
439
|
|
|
438
440
|
```bash
|
|
439
|
-
|
|
441
|
+
curl -fsSL https://antigravity.google/cli/install.sh | bash # Gemini (Antigravity CLI: agy)
|
|
440
442
|
npm install -g @openai/codex # Codex
|
|
441
443
|
pip install mistral-vibe # Mistral
|
|
442
444
|
npm install -g @kilocode/cli # Kilo
|
|
@@ -471,18 +473,18 @@ Choose the right balance of quality, speed, and cost with model quality tiers.
|
|
|
471
473
|
|
|
472
474
|
| Consultant | Premium | Standard | Economy |
|
|
473
475
|
|------------|---------|----------|---------|
|
|
474
|
-
| Claude | opus-4
|
|
475
|
-
| Gemini |
|
|
476
|
-
| Codex | gpt-5.
|
|
476
|
+
| Claude | claude-opus-4-8 | claude-sonnet-4-6 | claude-haiku-4-5 |
|
|
477
|
+
| Gemini | Gemini 3.1 Pro (High) | Gemini 3.5 Flash (High) | Gemini 3.5 Flash (Low) |
|
|
478
|
+
| Codex | gpt-5.5 | gpt-5.4 | gpt-5.4-nano |
|
|
477
479
|
| Mistral | mistral-large-3 | mistral-medium-latest | devstral-small-2 |
|
|
478
|
-
| Cursor | composer-
|
|
479
|
-
| DeepSeek | deepseek-
|
|
480
|
-
| GLM | glm-5.
|
|
481
|
-
| Grok | grok-4.
|
|
482
|
-
| Qwen3 | qwen3.
|
|
483
|
-
| Aider |
|
|
480
|
+
| Cursor | composer-2.5 | composer-2 | gemini-3-flash |
|
|
481
|
+
| DeepSeek | deepseek-v4-pro | deepseek-v4-flash | deepseek-v4-flash |
|
|
482
|
+
| GLM | glm-5.2 | glm-5.2 | glm-4-flash |
|
|
483
|
+
| Grok | grok-4.3 | grok-4.1-fast | grok-4.1-fast |
|
|
484
|
+
| Qwen3 | qwen3.7-max | qwen3.6-35b-a3b | qwen3-32b |
|
|
485
|
+
| Aider | qwen3-coder:free | gpt-5.4 | gpt-5.4-nano |
|
|
484
486
|
| MiniMax | MiniMax-M2.7 | MiniMax-M2.7 | MiniMax-M2.5 |
|
|
485
|
-
| Ollama |
|
|
487
|
+
| Ollama | hf.co/prithivMLmods/VibeThinker-3B-GGUF | llama3.3 | llama3.2 |
|
|
486
488
|
|
|
487
489
|
### Usage
|
|
488
490
|
|
|
@@ -584,18 +586,25 @@ ai-consultants doctor --suggest-preset --question "..." # Recommend prese
|
|
|
584
586
|
## How It Works
|
|
585
587
|
|
|
586
588
|
```
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
589
|
+
Classify -> Plan shape -> Fan out -> Deliberate & converge -> Synthesize
|
|
590
|
+
| | | |
|
|
591
|
+
category orchestration Gemini (8) convergence loop /
|
|
592
|
+
complexity shape Codex (7) adversarial gate /
|
|
593
|
+
intent Mistral (6) tournament / exhaustive
|
|
594
|
+
Kilo (9)
|
|
593
595
|
```
|
|
594
596
|
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
597
|
+
The **shape** is chosen per question (or pinned via `ORCHESTRATION_MODE`):
|
|
598
|
+
|
|
599
|
+
| Shape | When | What happens |
|
|
600
|
+
|-------|------|--------------|
|
|
601
|
+
| `quick` | simple questions | one fan-out, no debate |
|
|
602
|
+
| `converge` | most questions | debate rounds until consensus is reached (not a fixed count) |
|
|
603
|
+
| `adversarial` | security | a forced critique round + peer-review refutation gate |
|
|
604
|
+
| `tournament` | "compare X vs Y" | converge, then declare a single winning approach |
|
|
605
|
+
| `exhaustive` | "find all / audit" | loop until a round surfaces no new angle |
|
|
606
|
+
|
|
607
|
+
Set `ORCHESTRATION_MODE=fixed` for the classic fixed-round pipeline.
|
|
599
608
|
|
|
600
609
|
With peer review:
|
|
601
610
|
```
|
|
@@ -659,6 +668,12 @@ Each consultation generates:
|
|
|
659
668
|
|
|
660
669
|
## Changelog
|
|
661
670
|
|
|
671
|
+
### v2.15.1
|
|
672
|
+
|
|
673
|
+
- **Gemini fixes (Antigravity CLI)**: the default model (`Gemini 3.1 Pro (High)`) wraps its JSON in a ```` ```json ```` fence; v2.15.0 didn't strip it, so every Gemini reply degraded to a generic fallback. Now de-fenced on all paths (response processing, self-reflection, synthesis).
|
|
674
|
+
- **Works out-of-the-box for `npx`**: Gemini auto-selects API mode when `GEMINI_API_KEY` is set (the `agy` CLI can't be npm-installed and is OAuth-only), so a key-only environment no longer silently drops Gemini.
|
|
675
|
+
- **Synthesis no longer hangs** when `agy` is the synthesizer (it was launched interactively instead of in print mode).
|
|
676
|
+
|
|
662
677
|
### v2.10.0
|
|
663
678
|
|
|
664
679
|
- **MiniMax M2.5 support**: New API-based consultant with "The Pragmatic Optimizer" persona
|
package/SKILL.md
CHANGED
|
@@ -2,15 +2,15 @@
|
|
|
2
2
|
name: ai-consultants
|
|
3
3
|
description: Consult Gemini CLI, Codex CLI, Mistral Vibe, Kilo CLI, Cursor, Claude, Amp, Kimi, Qwen, MiniMax, and Ollama as external experts for coding questions. Automatically excludes the invoking agent from the panel to avoid self-consultation. Use when you have doubts about implementations, want a second opinion, need to choose between different approaches, or when explicitly requested with phrases like "ask the consultants", "what do the other models think", "compare solutions", "get expert opinions", "I'm not sure about this approach", "what would other models say". Make sure to consult this skill whenever the user is weighing trade-offs, comparing architectures, validating complex solutions, or wants multiple perspectives on any non-trivial coding decision. Do NOT use for simple questions that only need one model's answer or when you already have high confidence in a solution.
|
|
4
4
|
license: MIT
|
|
5
|
-
compatibility: Requires bash, jq, and at least 2 AI CLI tools (
|
|
5
|
+
compatibility: Requires bash, jq, and at least 2 AI CLI tools (agy, codex, vibe, etc.). macOS and Linux.
|
|
6
6
|
metadata:
|
|
7
7
|
author: matteoscurati
|
|
8
|
-
version: 2.
|
|
8
|
+
version: 2.17.2
|
|
9
9
|
---
|
|
10
10
|
|
|
11
|
-
# AI Consultants v2.
|
|
11
|
+
# AI Consultants v2.17.2 - AI Expert Panel
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
**A harness for every question.** Convene a panel of AI "consultants" for coding questions — and let the engine pick *how* they deliberate: a quick read, a convergence loop, an adversarial refutation gate, a tournament, or an exhaustive sweep. Each consultant has a **configurable persona** that shapes its analysis.
|
|
14
14
|
|
|
15
15
|
## Quick Start
|
|
16
16
|
|
|
@@ -32,7 +32,7 @@ Configuration (presets, strategies, features, personas, API keys) can be managed
|
|
|
32
32
|
|
|
33
33
|
| Consultant | CLI | Persona | Focus |
|
|
34
34
|
|------------|-----|---------|-------|
|
|
35
|
-
| **Google Gemini** | `
|
|
35
|
+
| **Google Gemini** | `agy` | The Architect | Design patterns, scalability |
|
|
36
36
|
| **OpenAI Codex** | `codex` | The Pragmatist | Simplicity, proven solutions |
|
|
37
37
|
| **Mistral Vibe** | `vibe` | The Devil's Advocate | Edge cases, vulnerabilities |
|
|
38
38
|
| **Kilo Code** | `kilocode` | The Innovator | Creativity, unconventional |
|
|
@@ -46,7 +46,7 @@ Configuration (presets, strategies, features, personas, API keys) can be managed
|
|
|
46
46
|
|
|
47
47
|
**API-only consultants**: GLM (The Methodologist), Grok (The Provocateur), DeepSeek (The Code Specialist), MiniMax (The Pragmatic Optimizer)
|
|
48
48
|
|
|
49
|
-
**CLI/API Mode**: Gemini, Codex, Claude, Mistral, and Qwen can switch between CLI and API mode via `*_USE_API` environment variables.
|
|
49
|
+
**CLI/API Mode**: Gemini, Codex, Claude, Mistral, and Qwen can switch between CLI and API mode via `*_USE_API` environment variables. Gemini auto-selects API mode when `GEMINI_API_KEY` is set (no `agy` install needed) and the CLI otherwise.
|
|
50
50
|
|
|
51
51
|
**Self-Exclusion**: The invoking agent is automatically excluded from the panel. When invoked from Claude Code, Claude is excluded; when invoked from Codex CLI, Codex is excluded, etc.
|
|
52
52
|
|
|
@@ -81,7 +81,9 @@ Presets: `minimal`, `balanced`, `thorough`, `high-stakes`, `local`, `security`,
|
|
|
81
81
|
|
|
82
82
|
## Workflow
|
|
83
83
|
|
|
84
|
-
Query is classified, then sent to consultants in parallel. Responses are scored, voted on, and synthesized into a recommendation.
|
|
84
|
+
Query is classified, then sent to consultants in parallel. Responses are scored, voted on, and synthesized into a recommendation.
|
|
85
|
+
|
|
86
|
+
**Dynamic orchestration (v2.16+):** a planner picks an orchestration *shape* per question — `quick` (simple), `converge` (debate until consensus, not a fixed round count), `adversarial` (SECURITY: forced critique + peer-review refutation gate), `tournament` (compare approaches → pick a winner), `exhaustive` (find-all: loop until no new angle). Set `ORCHESTRATION_MODE=fixed` for the legacy pipeline. See [configuration reference](references/configuration.md#dynamic-orchestration-v216).
|
|
85
87
|
|
|
86
88
|
## Features
|
|
87
89
|
|
package/docs/COST_RATES.md
CHANGED
|
@@ -1,7 +1,9 @@
|
|
|
1
|
-
# Cost Rates - AI Consultants v2.
|
|
1
|
+
# Cost Rates - AI Consultants v2.17.2
|
|
2
2
|
|
|
3
3
|
This page documents the per-token rates used by the cost tracking system.
|
|
4
4
|
|
|
5
|
+
> **Units:** all rates below are **USD per 1,000 tokens (per-1K)** and mirror `docs/cost_rates.json`, which is the runtime source of truth. `lib/costs.sh` computes cost as `(token_count / 1000) * rate`, so a per-1K rate equals the provider's `$/1M` price divided by 1000.
|
|
6
|
+
|
|
5
7
|
## Model Quality Tiers (v2.5+)
|
|
6
8
|
|
|
7
9
|
Since v2.5, models are organized into three tiers. Use `apply_model_tier()` or `get_model_for_tier()` to select models programmatically.
|
|
@@ -18,34 +20,35 @@ Since v2.5, models are organized into three tiers. Use `apply_model_tier()` or `
|
|
|
18
20
|
|
|
19
21
|
| Model | Tier | Cost/1K | Consultant |
|
|
20
22
|
|-------|------|---------|------------|
|
|
21
|
-
|
|
|
22
|
-
|
|
|
23
|
-
|
|
|
24
|
-
| gpt-5.5 | Premium | $0.
|
|
25
|
-
| gpt-5.
|
|
26
|
-
| gpt-
|
|
27
|
-
|
|
|
28
|
-
| claude-opus-4-
|
|
23
|
+
| Gemini 3.1 Pro (High) | Premium | $0.002 | Gemini (agy CLI) |
|
|
24
|
+
| Gemini 3.5 Flash (High) | Standard | $0.0015 | Gemini (agy CLI) |
|
|
25
|
+
| Gemini 3.5 Flash (Low) | Economy | $0.0015 | Gemini (agy CLI) |
|
|
26
|
+
| gpt-5.5 | Premium | $0.005 | Codex |
|
|
27
|
+
| gpt-5.4 | Standard | $0.0025 | Codex/Aider |
|
|
28
|
+
| gpt-5.4-nano | Economy | $0.0002 | Codex/Aider |
|
|
29
|
+
| qwen3-coder:free | Premium | $0.00 | Aider |
|
|
30
|
+
| claude-opus-4-8 | Premium | $0.005 | Claude |
|
|
29
31
|
| claude-sonnet-4-6 | Standard | $0.003 | Claude |
|
|
30
|
-
| claude-haiku-4-5 | Economy | $0.
|
|
31
|
-
| mistral-large-3 | Premium | $0.
|
|
32
|
-
| mistral-medium-latest | Standard | $0.
|
|
33
|
-
| devstral-small-2 | Economy | $0.
|
|
34
|
-
| composer-2 | Premium
|
|
35
|
-
|
|
|
36
|
-
|
|
|
37
|
-
| deepseek-
|
|
38
|
-
|
|
|
39
|
-
| glm-
|
|
32
|
+
| claude-haiku-4-5 | Economy | $0.001 | Claude |
|
|
33
|
+
| mistral-large-3 | Premium | $0.002 | Mistral |
|
|
34
|
+
| mistral-medium-latest | Standard | $0.001 | Mistral |
|
|
35
|
+
| devstral-small-2 | Economy | $0.00 | Mistral |
|
|
36
|
+
| composer-2.5 | Premium | $0.0005 | Cursor |
|
|
37
|
+
| composer-2 | Standard | $0.0005 | Cursor |
|
|
38
|
+
| gemini-3-flash | Economy | $0.0005 | Cursor |
|
|
39
|
+
| deepseek-v4-pro | Premium | $0.000435 | DeepSeek |
|
|
40
|
+
| deepseek-v4-flash | Standard/Economy | $0.00014 | DeepSeek |
|
|
41
|
+
| glm-5.2 | Premium/Standard | $0.00098 | GLM |
|
|
42
|
+
| glm-4-flash | Economy | $0.001 | GLM |
|
|
40
43
|
| grok-4.3 | Premium | $0.00125 | Grok |
|
|
41
|
-
| grok-
|
|
42
|
-
|
|
|
43
|
-
| qwen3.6-plus | Premium | $0.000325 | Qwen3 |
|
|
44
|
+
| grok-4.1-fast | Standard/Economy | $0.0002 | Grok |
|
|
45
|
+
| qwen3.7-max | Premium | $0.0012 | Qwen3 |
|
|
44
46
|
| qwen3.6-35b-a3b | Standard | $0.000163 | Qwen3 |
|
|
45
|
-
| qwen3-32b | Economy | $0.
|
|
46
|
-
| MiniMax-M2.7 | Premium/Standard | $0.
|
|
47
|
-
| MiniMax-M2.5 | Economy | $0.
|
|
48
|
-
| kimi-code/kimi-for-coding | Premium/Standard | $0.
|
|
47
|
+
| qwen3-32b | Economy | $0.0004 | Qwen3 |
|
|
48
|
+
| MiniMax-M2.7 | Premium/Standard | $0.00025 | MiniMax |
|
|
49
|
+
| MiniMax-M2.5 | Economy | $0.000255 | MiniMax |
|
|
50
|
+
| kimi-code/kimi-for-coding | Premium/Standard | $0.0005 | Kimi |
|
|
51
|
+
| hf.co/prithivMLmods/VibeThinker-3B-GGUF | All | $0.00 | Ollama (local) |
|
|
49
52
|
| auto | All | $0.002 | Kilo (internal routing) |
|
|
50
53
|
| default | - | $0.005 | Fallback |
|
|
51
54
|
|
|
@@ -53,34 +56,35 @@ Since v2.5, models are organized into three tiers. Use `apply_model_tier()` or `
|
|
|
53
56
|
|
|
54
57
|
| Model | Tier | Cost/1K | Consultant |
|
|
55
58
|
|-------|------|---------|------------|
|
|
56
|
-
|
|
|
57
|
-
|
|
|
58
|
-
|
|
|
59
|
-
| gpt-5.5 | Premium | $0.
|
|
60
|
-
| gpt-5.
|
|
61
|
-
| gpt-
|
|
62
|
-
|
|
|
63
|
-
| claude-opus-4-
|
|
59
|
+
| Gemini 3.1 Pro (High) | Premium | $0.012 | Gemini (agy CLI) |
|
|
60
|
+
| Gemini 3.5 Flash (High) | Standard | $0.009 | Gemini (agy CLI) |
|
|
61
|
+
| Gemini 3.5 Flash (Low) | Economy | $0.009 | Gemini (agy CLI) |
|
|
62
|
+
| gpt-5.5 | Premium | $0.030 | Codex |
|
|
63
|
+
| gpt-5.4 | Standard | $0.015 | Codex/Aider |
|
|
64
|
+
| gpt-5.4-nano | Economy | $0.00125 | Codex/Aider |
|
|
65
|
+
| qwen3-coder:free | Premium | $0.00 | Aider |
|
|
66
|
+
| claude-opus-4-8 | Premium | $0.025 | Claude |
|
|
64
67
|
| claude-sonnet-4-6 | Standard | $0.015 | Claude |
|
|
65
|
-
| claude-haiku-4-5 | Economy | $0.
|
|
66
|
-
| mistral-large-3 | Premium | $0.
|
|
67
|
-
| mistral-medium-latest | Standard | $0.
|
|
68
|
-
| devstral-small-2 | Economy | $0.
|
|
69
|
-
| composer-2 | Premium
|
|
70
|
-
|
|
|
71
|
-
|
|
|
72
|
-
| deepseek-
|
|
73
|
-
|
|
|
74
|
-
| glm-
|
|
68
|
+
| claude-haiku-4-5 | Economy | $0.005 | Claude |
|
|
69
|
+
| mistral-large-3 | Premium | $0.006 | Mistral |
|
|
70
|
+
| mistral-medium-latest | Standard | $0.003 | Mistral |
|
|
71
|
+
| devstral-small-2 | Economy | $0.00 | Mistral |
|
|
72
|
+
| composer-2.5 | Premium | $0.0025 | Cursor |
|
|
73
|
+
| composer-2 | Standard | $0.0025 | Cursor |
|
|
74
|
+
| gemini-3-flash | Economy | $0.003 | Cursor |
|
|
75
|
+
| deepseek-v4-pro | Premium | $0.00087 | DeepSeek |
|
|
76
|
+
| deepseek-v4-flash | Standard/Economy | $0.00028 | DeepSeek |
|
|
77
|
+
| glm-5.2 | Premium/Standard | $0.00308 | GLM |
|
|
78
|
+
| glm-4-flash | Economy | $0.003 | GLM |
|
|
75
79
|
| grok-4.3 | Premium | $0.0025 | Grok |
|
|
76
|
-
| grok-
|
|
77
|
-
|
|
|
78
|
-
| qwen3.6-plus | Premium | $0.00195 | Qwen3 |
|
|
80
|
+
| grok-4.1-fast | Standard/Economy | $0.0005 | Grok |
|
|
81
|
+
| qwen3.7-max | Premium | $0.006 | Qwen3 |
|
|
79
82
|
| qwen3.6-35b-a3b | Standard | $0.0009 | Qwen3 |
|
|
80
|
-
| qwen3-32b | Economy | $0.
|
|
81
|
-
| MiniMax-M2.7 | Premium/Standard | $0.
|
|
82
|
-
| MiniMax-M2.5 | Economy | $0.
|
|
83
|
-
| kimi-code/kimi-for-coding | Premium/Standard | $0.
|
|
83
|
+
| qwen3-32b | Economy | $0.0016 | Qwen3 |
|
|
84
|
+
| MiniMax-M2.7 | Premium/Standard | $0.001 | MiniMax |
|
|
85
|
+
| MiniMax-M2.5 | Economy | $0.001 | MiniMax |
|
|
86
|
+
| kimi-code/kimi-for-coding | Premium/Standard | $0.002 | Kimi |
|
|
87
|
+
| hf.co/prithivMLmods/VibeThinker-3B-GGUF | All | $0.00 | Ollama (local) |
|
|
84
88
|
| auto | All | $0.006 | Kilo |
|
|
85
89
|
| default | - | $0.015 | Fallback |
|
|
86
90
|
|