ai-consultants 4.0.4 → 5.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +1 -1
- package/CLAUDE.md +12 -1
- package/README.md +51 -32
- package/SKILL.md +50 -5
- package/bin/ai-consultants +16 -4
- package/docs/COST_RATES.md +1 -1
- package/docs/JSON_SCHEMA.md +63 -1
- package/docs/cost_rates.json +1 -1
- package/package.json +1 -1
- package/references/details.md +31 -20
- package/scripts/config.sh +83 -46
- package/scripts/consult_all.sh +148 -69
- package/scripts/doctor.sh +11 -0
- package/scripts/generate_public_docs.sh +112 -0
- package/scripts/lib/common.sh +90 -0
- package/scripts/lib/coverage_integrity.sh +150 -0
- package/scripts/lib/schema.json +36 -0
- package/scripts/public_registry.sh +190 -0
- package/scripts/synthesize.sh +116 -8
package/.env.example
CHANGED
|
@@ -250,7 +250,7 @@ MINIMAX_FORMAT=openai
|
|
|
250
250
|
# =============================================================================
|
|
251
251
|
|
|
252
252
|
# Defaults used when --preset / --strategy are omitted
|
|
253
|
-
DEFAULT_PRESET=balanced
|
|
253
|
+
# DEFAULT_PRESET=balanced
|
|
254
254
|
DEFAULT_STRATEGY=coverage
|
|
255
255
|
|
|
256
256
|
# The invoking integration normally sets this per run for self-exclusion.
|
package/CLAUDE.md
CHANGED
|
@@ -6,7 +6,7 @@ AI Consultants is a multi-model coverage system that queries up to 10 AI consult
|
|
|
6
6
|
|
|
7
7
|
**Self-Exclusion**: The invoking agent is automatically excluded from both the panel and synthesis. Claude Code won't query or synthesize with Claude, Codex CLI won't query or synthesize with Codex, etc.
|
|
8
8
|
|
|
9
|
-
**Version**:
|
|
9
|
+
**Version**: 5.0.0
|
|
10
10
|
|
|
11
11
|
## Distribution
|
|
12
12
|
|
|
@@ -760,6 +760,17 @@ curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/s
|
|
|
760
760
|
|
|
761
761
|
## Changelog
|
|
762
762
|
|
|
763
|
+
### v5.0.0
|
|
764
|
+
|
|
765
|
+
- **Coverage claims are now locally auditable instead of resting on synthesis prose.** `scripts/lib/coverage_integrity.sh` normalizes successful structured envelopes into deterministic `<consultant-slug>:<index>` findings, overwriting any provider-supplied IDs. `scripts/synthesize.sh` asks the synthesizer to cite those IDs, then independently compares expected and represented IDs. Missing/duplicate/non-normalizable sources produce `DEGRADED`; unknown IDs and structural failures produce `FAILED`; invented IDs are removed from published coverage. The legacy `coverage`, `weighted_recommendation`, risk, action, and individual-response fields remain present.
|
|
766
|
+
- **Truncation can no longer hide evidence loss.** Coverage/union prompts receive complete atomic findings. Only non-normalizable fallback context is capped, at Unicode code-point boundaries, and local `coverage_input_truncated` / `truncated_consultants` values overwrite model output in normal, local-fallback, and failed-closed artifacts. The report renders status, audited fields, truncation, and the affected consultants. The first Ubuntu CI run exposed an jq-expression compatibility gap; the final form uses jq 1.6-compatible string slicing and passed the release runner.
|
|
767
|
+
- **Presets are host-aware and fail closed.** `get_effective_preset_panel_size` and `select_preset_consultants` preserve the advertised panel after self-exclusion by filling from the canonical roster using static transport evidence only. `max_quality` is ten in the catalog and nine for a canonical host. Insufficient capacity now aborts with promised/selected/missing counts, and health-gate pruning is rechecked before Round 1. Custom agents can satisfy capacity; explicit per-consultant false values remain fallback opt-outs.
|
|
768
|
+
- **Public modes and their documentation have one executable source of truth.** `scripts/public_registry.sh` owns `fast-check -> fast/coverage`, `coverage-review -> balanced/coverage`, `max-coverage -> max_quality/coverage`, every existing preset/alias, and every strategy/default. `scripts/generate_public_docs.sh --check` renders and verifies README, SKILL, and reference tables. CLI conflict/override/unknown diagnostics run before user config, output directories, health checks, or adapters. `doctor --suggest-preset` downgrades to runnable `minimal` when only two effective transports remain.
|
|
769
|
+
- **The v5 release path was exercised end to end within the preregistered cap.** One no-retry `coverage-review` smoke dispatched Gemini, Mistral, and Kimi once each, then synthesized once with Claude: four dispatches total, 3/3 consultant responses, `coverage_integrity=MET` with 12/12 source IDs, and no truncation. The release gate passed 27/27 suites, ShellCheck, Bash 3.2 syntax, generated-doc drift, packaging, and sentinel isolation.
|
|
770
|
+
- **Development safety record.** An early P0.5 mode-registry lookup returned only the mode name, left the preset empty, and caused a local test to invoke ambient adapter auth/capability gates. Gemini/Codex were fixtures; the other adapters returned missing-key/auth/capability diagnostics. The retained transcript does not establish a network/provider request and no successful provider response was observed. The lookup was fixed, all later local release gates used provider/curl sentinels, and the final stub-only mode smoke recorded zero non-fixture adapter calls.
|
|
771
|
+
|
|
772
|
+
Deliberately retained: every v4 preset/alias, advanced configuration key, smart routing, custom agents, self-exclusion, and legacy JSON fields. Not claimed: semantic verification that a synthesized sentence faithfully paraphrases the cited atom; v5.0 verifies attribution-set integrity, not natural-language entailment.
|
|
773
|
+
|
|
763
774
|
### v4.0.4
|
|
764
775
|
|
|
765
776
|
- **Housekeeping now removes dead implementation without removing public compatibility.** The cleanup deletes uncalled API, cache, optimizer, common, cost, persona, reliability, routing-summary, and session-display helpers, plus the stale README changelog and redundant npm ignore file. The documented `get_routing_mode`, `get_recommended_count`, and `get_category_timeout` helpers stay available and are regression-tested. Six historic config keys that had no runtime reader remain accepted as explicitly deprecated no-ops through v4.x: `SESSION_CLEANUP_DAYS`, `ENABLE_PROGRESS_BARS`, `ENABLE_EARLY_TERMINATION`, `USE_COMPACT_PROMPTS`, `ENABLE_SELECTIVE_CONTEXT`, and `MAX_FILES_PER_CONSULTANT`.
|
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# AI Consultants
|
|
1
|
+
# AI Consultants v5.0.0
|
|
2
2
|
|
|
3
3
|
> **Coverage, not a single guess.** A panel of up to 10 frontier models from different vendors fans out on your question in parallel and hands you the *union* of what they collectively see — the risks, edge cases, and approaches a single model misses.
|
|
4
4
|
|
|
5
|
-
[](https://github.com/matteoscurati/ai-consultants)
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
[](https://docs.anthropic.com/en/docs/claude-code/skills)
|
|
8
8
|
[](https://github.com/matteoscurati/ai-consultants)
|
|
@@ -455,6 +455,7 @@ Choose the right balance of quality, speed, and cost with model quality tiers.
|
|
|
455
455
|
|
|
456
456
|
A preset only chooses the **consultant set and model tier** — every run then fans out in
|
|
457
457
|
parallel and returns the coverage union.
|
|
458
|
+
Preset panels use statically configured transports only (a CLI on `PATH`, or selected API mode with its key). Host self-exclusion is fail-closed; lost canonical slots are filled from `ALL_CONSULTANTS` in canonical order. If the effective target cannot be met, including after an enabled health gate prunes the panel, the run stops with promised/selected/missing-capacity guidance. Configured custom API agents are appended and may satisfy capacity, so a full preset can exceed its advertised count. `max_quality` still advertises 10 consultants, with an effective target of 9 for a canonical invoking host.
|
|
458
459
|
For `max_quality`, Grok, GLM, and DeepSeek are also pinned to their highest
|
|
459
460
|
accepted reasoning effort: Grok Build uses `xhigh`, while GLM and DeepSeek use
|
|
460
461
|
`max`. Unsupported transport capabilities fail explicitly instead of reducing
|
|
@@ -468,9 +469,15 @@ likewise receives a 16,384-token completion budget in the maximum tier and a
|
|
|
468
469
|
provider-specific compact Markdown contract through mmx's native system channel.
|
|
469
470
|
Provider output is tagged `structured`, `fallback`, or `error`;
|
|
470
471
|
malformed/truncated JSON fails
|
|
471
|
-
closed
|
|
472
|
-
|
|
473
|
-
|
|
472
|
+
closed. In coverage/union mode, only locally atomized `summary`, `pros`,
|
|
473
|
+
`cons`, `alternatives`, `caveats`, and `references` can be source-attributed;
|
|
474
|
+
fallback prose remains context-only for manual review. `synthesis.json` records
|
|
475
|
+
`coverage_integrity`: `MET` applies only to those audited fields, while
|
|
476
|
+
`DEGRADED`/`FAILED` must not be read as comprehensive coverage. It also records
|
|
477
|
+
locally authoritative `coverage_input_truncated` and `truncated_consultants`:
|
|
478
|
+
only non-normalizable fallback context is capped (at Unicode code-point
|
|
479
|
+
boundaries), never atomic findings; a truncated coverage input is disclosed
|
|
480
|
+
and cannot support a comprehensive coverage claim.
|
|
474
481
|
|
|
475
482
|
### Models by Tier
|
|
476
483
|
|
|
@@ -525,39 +532,47 @@ apply_model_tier "economy" # Set all to economy models
|
|
|
525
532
|
|
|
526
533
|
## Configuration
|
|
527
534
|
|
|
528
|
-
|
|
535
|
+
<!-- ai-consultants:public-registry:start -->
|
|
536
|
+
<!-- ai-consultants:default-synthesis-strategy=coverage -->
|
|
537
|
+
### Public Modes
|
|
538
|
+
|
|
539
|
+
| Mode | Preset | Default strategy | Use case |
|
|
540
|
+
|---|---|---|---|
|
|
541
|
+
| `fast-check` | `fast` | `coverage` | Fast economy panel for a quick coverage check |
|
|
542
|
+
| `coverage-review` | `balanced` | `coverage` | Balanced panel for a general coverage review |
|
|
543
|
+
| `max-coverage` | `max_quality` | `coverage` | Maximum-quality panel for the broadest coverage |
|
|
544
|
+
|
|
545
|
+
### Configuration Presets
|
|
546
|
+
|
|
547
|
+
| Preset | Alias | Target | Tier | Use case |
|
|
548
|
+
|---|---|---:|---|---|
|
|
549
|
+
| `minimal` | — | 2 | base | Quick questions |
|
|
550
|
+
| `balanced` | — | 3 | base | Standard coverage |
|
|
551
|
+
| `thorough` | — | 3 | base | Comprehensive analysis |
|
|
552
|
+
| `high-stakes` | — | 4 | premium | Critical decisions |
|
|
553
|
+
| `security` | — | 3 | base | Security reviews |
|
|
554
|
+
| `cost-capped` | — | 3 | economy | Budget-conscious options |
|
|
555
|
+
| `max_quality` | `max-quality` | 10 | maximum | Maximum coverage for critical decisions |
|
|
556
|
+
| `medium` | — | 3 | standard | General questions |
|
|
557
|
+
| `fast` | — | 2 | economy | Quick checks |
|
|
529
558
|
|
|
530
|
-
|
|
559
|
+
### Synthesis Strategies
|
|
531
560
|
|
|
532
|
-
|
|
|
533
|
-
|
|
534
|
-
| `
|
|
535
|
-
| `
|
|
536
|
-
| `
|
|
537
|
-
| `
|
|
538
|
-
| `
|
|
539
|
-
| `
|
|
540
|
-
|
|
541
|
-
| `security` | Security-focused (3) | Default | Security reviews |
|
|
542
|
-
| `cost-capped` | Budget-conscious | Default | Minimal API costs |
|
|
561
|
+
| Strategy | Description |
|
|
562
|
+
|---|---|
|
|
563
|
+
| `coverage` | Union of every distinct point across the panel **Default** |
|
|
564
|
+
| `compare_only` | Present each consultant side-by-side, without a synthesized union |
|
|
565
|
+
| `majority` | Produce one blended recommendation, weighting all equally |
|
|
566
|
+
| `risk_averse` | Weight conservative responses higher |
|
|
567
|
+
| `security_first` | Prioritize security-focused insights |
|
|
568
|
+
| `cost_capped` | Prefer cheaper consultant opinions within budget |
|
|
569
|
+
<!-- ai-consultants:public-registry:end -->
|
|
543
570
|
|
|
544
571
|
**Bash:**
|
|
545
572
|
```bash
|
|
546
573
|
./scripts/consult_all.sh --preset balanced "Question"
|
|
547
574
|
```
|
|
548
575
|
|
|
549
|
-
### Synthesis Strategies
|
|
550
|
-
|
|
551
|
-
Control how responses are combined:
|
|
552
|
-
|
|
553
|
-
| Strategy | Description |
|
|
554
|
-
|----------|-------------|
|
|
555
|
-
| `majority` | Most common answer wins (default) |
|
|
556
|
-
| `risk_averse` | Weight conservative responses higher |
|
|
557
|
-
| `security_first` | Prioritize security considerations |
|
|
558
|
-
| `cost_capped` | Prefer simpler, cheaper solutions |
|
|
559
|
-
| `compare_only` | No recommendation, just comparison |
|
|
560
|
-
|
|
561
576
|
**Bash:**
|
|
562
577
|
```bash
|
|
563
578
|
./scripts/consult_all.sh --strategy risk_averse "Question"
|
|
@@ -691,7 +706,11 @@ Classify -> Route -> Fan out (parallel) -> Coverage synthesis
|
|
|
691
706
|
1. **Classify** the question into a category.
|
|
692
707
|
2. **Route** (optional, `ENABLE_SMART_ROUTING`) to the consultants with the best category affinity (`references/affinity.json`).
|
|
693
708
|
3. **Fan out** to every selected consultant in parallel — one shot each, no serial rounds.
|
|
694
|
-
4. **Synthesize** the **coverage union
|
|
709
|
+
4. **Synthesize** the **coverage union** over auditable normalized fields. Read
|
|
710
|
+
`coverage_integrity` before treating it as complete: `MET` means every local
|
|
711
|
+
atomic source ID is represented once; `DEGRADED` and `FAILED` are explicitly
|
|
712
|
+
non-comprehensive. Override with `--strategy compare_only` (side-by-side) or
|
|
713
|
+
`majority` (a single blended recommendation).
|
|
695
714
|
|
|
696
715
|
### Output
|
|
697
716
|
|
|
@@ -704,7 +723,7 @@ ${XDG_CACHE_HOME:-$HOME/.cache}/ai-consultants/consultations/TIMESTAMP/
|
|
|
704
723
|
├── codex.json # with confidence scores
|
|
705
724
|
├── mistral.json
|
|
706
725
|
├── grok.json
|
|
707
|
-
├── synthesis.json # Coverage union
|
|
726
|
+
├── synthesis.json # Coverage union plus coverage_integrity/audited_fields
|
|
708
727
|
├── optimization_metrics.json
|
|
709
728
|
└── report.md # Human-readable report
|
|
710
729
|
```
|
package/SKILL.md
CHANGED
|
@@ -5,10 +5,10 @@ license: MIT
|
|
|
5
5
|
compatibility: Requires bash, jq, and at least 2 AI CLI tools (agy, codex, vibe, etc.). macOS and Linux.
|
|
6
6
|
metadata:
|
|
7
7
|
author: matteoscurati
|
|
8
|
-
version:
|
|
8
|
+
version: 5.0.0
|
|
9
9
|
---
|
|
10
10
|
|
|
11
|
-
# AI Consultants
|
|
11
|
+
# AI Consultants v5.0.0 - AI Expert Panel
|
|
12
12
|
|
|
13
13
|
**Coverage, not a single guess.** Convene a panel of AI "consultants" from different vendors for coding questions: they fan out in parallel and you get the *union* of what they collectively see — the risks, edge cases, and approaches a single model misses. Each consultant has a **configurable persona** that decorrelates its analysis.
|
|
14
14
|
|
|
@@ -115,8 +115,14 @@ Maximum-quality transports use smoke-tested runtime budgets: four advisory
|
|
|
115
115
|
turns for Mistral/Grok, extended Qwen/DeepSeek timeouts, and larger response
|
|
116
116
|
budgets for reasoning APIs and a compact native-system Markdown contract for
|
|
117
117
|
MiniMax M3. Responses carry an explicit `structured`,
|
|
118
|
-
`fallback`, or `error` quality. Truncated JSON fails closed
|
|
119
|
-
|
|
118
|
+
`fallback`, or `error` quality. Truncated JSON fails closed. Coverage/union
|
|
119
|
+
synthesis source-attributed claims use only locally normalized atomic fields;
|
|
120
|
+
fallback prose remains visible as context-only manual-review material. Inspect
|
|
121
|
+
`coverage_integrity`: `MET` applies only to its listed `audited_fields`, while
|
|
122
|
+
`DEGRADED` and `FAILED` are never comprehensive coverage claims.
|
|
123
|
+
`synthesis.json` locally records `coverage_input_truncated` and
|
|
124
|
+
`truncated_consultants`; only non-normalizable fallback context can be capped
|
|
125
|
+
(on Unicode code-point boundaries), never normalized atomic findings.
|
|
120
126
|
|
|
121
127
|
**Self-Exclusion**: The invoking agent is automatically excluded from the panel. When invoked from Claude Code, Claude is excluded; when invoked from Codex CLI, Codex is excluded, etc.
|
|
122
128
|
|
|
@@ -147,7 +153,46 @@ For detailed CLI installation instructions, see [Setup Guide](docs/SETUP.md).
|
|
|
147
153
|
/ai-consultants:consult "What could go wrong with this design?" src/service.ts
|
|
148
154
|
```
|
|
149
155
|
|
|
150
|
-
|
|
156
|
+
<!-- ai-consultants:public-registry:start -->
|
|
157
|
+
<!-- ai-consultants:default-synthesis-strategy=coverage -->
|
|
158
|
+
## Public Modes
|
|
159
|
+
|
|
160
|
+
| Mode | Preset | Default strategy | Use case |
|
|
161
|
+
|---|---|---|---|
|
|
162
|
+
| `fast-check` | `fast` | `coverage` | Fast economy panel for a quick coverage check |
|
|
163
|
+
| `coverage-review` | `balanced` | `coverage` | Balanced panel for a general coverage review |
|
|
164
|
+
| `max-coverage` | `max_quality` | `coverage` | Maximum-quality panel for the broadest coverage |
|
|
165
|
+
|
|
166
|
+
## Configuration Presets
|
|
167
|
+
|
|
168
|
+
| Preset | Alias | Target | Tier | Use case |
|
|
169
|
+
|---|---|---:|---|---|
|
|
170
|
+
| `minimal` | — | 2 | base | Quick questions |
|
|
171
|
+
| `balanced` | — | 3 | base | Standard coverage |
|
|
172
|
+
| `thorough` | — | 3 | base | Comprehensive analysis |
|
|
173
|
+
| `high-stakes` | — | 4 | premium | Critical decisions |
|
|
174
|
+
| `security` | — | 3 | base | Security reviews |
|
|
175
|
+
| `cost-capped` | — | 3 | economy | Budget-conscious options |
|
|
176
|
+
| `max_quality` | `max-quality` | 10 | maximum | Maximum coverage for critical decisions |
|
|
177
|
+
| `medium` | — | 3 | standard | General questions |
|
|
178
|
+
| `fast` | — | 2 | economy | Quick checks |
|
|
179
|
+
|
|
180
|
+
## Synthesis Strategies
|
|
181
|
+
|
|
182
|
+
| Strategy | Description |
|
|
183
|
+
|---|---|
|
|
184
|
+
| `coverage` | Union of every distinct point across the panel **Default** |
|
|
185
|
+
| `compare_only` | Present each consultant side-by-side, without a synthesized union |
|
|
186
|
+
| `majority` | Produce one blended recommendation, weighting all equally |
|
|
187
|
+
| `risk_averse` | Weight conservative responses higher |
|
|
188
|
+
| `security_first` | Prioritize security-focused insights |
|
|
189
|
+
| `cost_capped` | Prefer cheaper consultant opinions within budget |
|
|
190
|
+
<!-- ai-consultants:public-registry:end -->
|
|
191
|
+
|
|
192
|
+
Agents may use `--mode` instead of `--preset`; the mode chooses its mapped
|
|
193
|
+
preset and coverage strategy unless an explicit `--strategy` is supplied. See
|
|
194
|
+
[Reference Details](references/details.md) for capacity, fallback, and Bash
|
|
195
|
+
usage details.
|
|
151
196
|
|
|
152
197
|
## Workflow
|
|
153
198
|
|
package/bin/ai-consultants
CHANGED
|
@@ -15,6 +15,17 @@ BIN_DIR="$(cd -P "$(dirname "$SOURCE")" && pwd)"
|
|
|
15
15
|
PROJECT_ROOT="$(cd -P "$BIN_DIR/.." && pwd)"
|
|
16
16
|
SCRIPTS_DIR="$PROJECT_ROOT/scripts"
|
|
17
17
|
|
|
18
|
+
load_public_registry() {
|
|
19
|
+
# Keep `version` and the lightweight malformed-install diagnostic usable even
|
|
20
|
+
# when a partial installation lacks the optional public registry file.
|
|
21
|
+
if [ ! -f "$SCRIPTS_DIR/public_registry.sh" ]; then
|
|
22
|
+
echo "Error: public registry missing at $SCRIPTS_DIR/public_registry.sh" >&2
|
|
23
|
+
return 1
|
|
24
|
+
fi
|
|
25
|
+
# shellcheck source=../scripts/public_registry.sh
|
|
26
|
+
source "$SCRIPTS_DIR/public_registry.sh"
|
|
27
|
+
}
|
|
28
|
+
|
|
18
29
|
# --- Ensure execute permissions (npm can strip them) ---
|
|
19
30
|
ensure_permissions() {
|
|
20
31
|
local needs_fix=false
|
|
@@ -47,11 +58,13 @@ case "${1:-}" in
|
|
|
47
58
|
exit 0
|
|
48
59
|
;;
|
|
49
60
|
help|--help|-h)
|
|
61
|
+
load_public_registry || exit 1
|
|
50
62
|
cat <<EOF
|
|
51
63
|
ai-consultants v$VERSION - Cross-vendor AI coverage panel
|
|
52
64
|
|
|
53
65
|
Usage:
|
|
54
66
|
ai-consultants "question" [files...] Consult AI experts
|
|
67
|
+
ai-consultants --mode <name> "question" Use a public mode
|
|
55
68
|
ai-consultants --preset <name> "question" Use a configuration preset
|
|
56
69
|
ai-consultants --strategy <name> "question" Use a synthesis strategy
|
|
57
70
|
ai-consultants doctor [--fix] [--json] Run diagnostics
|
|
@@ -62,10 +75,9 @@ Usage:
|
|
|
62
75
|
ai-consultants version Show version
|
|
63
76
|
ai-consultants help Show this help
|
|
64
77
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
Strategies: coverage, compare_only, majority, risk_averse, security_first, cost_capped
|
|
78
|
+
Modes: $(registry_mode_rows | cut -d'|' -f1 | tr '\n' ' ' | sed 's/ $//')
|
|
79
|
+
Presets: $(registry_preset_rows | cut -d'|' -f1 | tr '\n' ' ' | sed 's/ $//')
|
|
80
|
+
Strategies: $(registry_strategy_rows | cut -d'|' -f1 | tr '\n' ' ' | sed 's/ $//')
|
|
69
81
|
|
|
70
82
|
Examples:
|
|
71
83
|
npx ai-consultants "How to optimize this SQL query?"
|
package/docs/COST_RATES.md
CHANGED
package/docs/JSON_SCHEMA.md
CHANGED
|
@@ -22,7 +22,10 @@ The schema is defined in `scripts/lib/schema.json` following JSON Schema Draft-0
|
|
|
22
22
|
"cons": [...],
|
|
23
23
|
"alternatives": [...],
|
|
24
24
|
"caveats": [...],
|
|
25
|
-
"references": [...]
|
|
25
|
+
"references": [...],
|
|
26
|
+
"findings": [
|
|
27
|
+
{"id": "gemini:1", "kind": "summary", "field": "summary", "text": "..."}
|
|
28
|
+
]
|
|
26
29
|
},
|
|
27
30
|
"confidence": {
|
|
28
31
|
"score": 8,
|
|
@@ -61,6 +64,65 @@ The schema is defined in `scripts/lib/schema.json` following JSON Schema Draft-0
|
|
|
61
64
|
|
|
62
65
|
## Optional Fields
|
|
63
66
|
|
|
67
|
+
### response.findings
|
|
68
|
+
|
|
69
|
+
`findings` is added locally by the synthesis stage to successful consultant
|
|
70
|
+
responses. It preserves all legacy fields and never trusts provider-supplied
|
|
71
|
+
IDs. Each item has a deterministic local `id` of
|
|
72
|
+
`<consultant-slug>:<one-based-index>`, plus `kind`, source `field`, and the
|
|
73
|
+
atomically attributable `text`. The shared slug rule ASCII-lowercases the
|
|
74
|
+
consultant name, retains `[a-z0-9_]`, replaces other runs with `_`, trims edge
|
|
75
|
+
underscores, and uses `unknown` if empty. Thus the public custom name
|
|
76
|
+
`My_agent` has IDs such as `my_agent:1`.
|
|
77
|
+
|
|
78
|
+
The conservative normalization contract derives findings only from `summary`,
|
|
79
|
+
`pros`, `cons`, `alternatives`, `caveats`, and `references`; alternatives may
|
|
80
|
+
be meaningful strings or objects with meaningful name/reason text. Empty,
|
|
81
|
+
null, or whitespace alternatives create no finding. Free-form fallback prose
|
|
82
|
+
(`response_quality: fallback`) and successful envelopes with zero atomic
|
|
83
|
+
findings are retained for human review but are non-normalizable; neither can
|
|
84
|
+
be presented as audited coverage. Error envelopes remain excluded from
|
|
85
|
+
synthesis.
|
|
86
|
+
|
|
87
|
+
## Synthesis Coverage Attribution
|
|
88
|
+
|
|
89
|
+
Each synthesized `coverage` item requires `source_ids`, containing only local
|
|
90
|
+
IDs from normalized findings. `scripts/lib/schema.json` defines this as
|
|
91
|
+
`definitions.synthesis_coverage_item`; the model prompt requests it, but the
|
|
92
|
+
local post-check is authoritative.
|
|
93
|
+
|
|
94
|
+
Every synthesis includes `coverage_integrity`:
|
|
95
|
+
|
|
96
|
+
| Status | Meaning |
|
|
97
|
+
|-------|---------|
|
|
98
|
+
| `MET` | Every expected normalized source ID appears exactly once, with at least one expected ID and no non-normalizable success. It applies only to the audited fields. |
|
|
99
|
+
| `DEGRADED` | A coverage-union source is missing or duplicated, no atomic source exists, or a successful response is non-normalizable. It is never comprehensive. |
|
|
100
|
+
| `FAILED` | Coverage/source attribution is structurally unusable or includes an invented source ID; do not rely on coverage. |
|
|
101
|
+
| `NOT_APPLICABLE` | A non-coverage strategy was selected and no attribution failure was found; it makes no coverage-union claim. |
|
|
102
|
+
|
|
103
|
+
The object contains `expected_count`, `represented_count`, `missing_ids`,
|
|
104
|
+
`unknown_ids`, `duplicate_ids`, `non_normalizable_consultants`,
|
|
105
|
+
`structural_errors`, `audited_fields`, and `normalization_version`.
|
|
106
|
+
`represented_count` is the count of unique expected IDs actually represented;
|
|
107
|
+
unknown and duplicate IDs cannot inflate it. In coverage/union mode only
|
|
108
|
+
`audited_fields` reach the attributable prompt. Structured `detailed` text and
|
|
109
|
+
fallback prose are not atomized; fallback text is context-only and must never
|
|
110
|
+
produce coverage items or source IDs.
|
|
111
|
+
|
|
112
|
+
Every synthesis artifact also carries locally authoritative
|
|
113
|
+
`coverage_input_truncated` (boolean) and `truncated_consultants` (a stable,
|
|
114
|
+
first-occurrence-unique array of consultant names discovered in lexically
|
|
115
|
+
ordered response-file order). These fields overwrite any values supplied by a
|
|
116
|
+
model and are present even on local fallback or failed-closed artifacts.
|
|
117
|
+
`SYNTH_DETAIL_MAX_CHARS` applies only to usable, non-normalizable fallback
|
|
118
|
+
context, never to normalized finding text. Its unit is Unicode code points:
|
|
119
|
+
input with exactly the limit is retained and does not set the flag; only input
|
|
120
|
+
with more code points is shortened to that limit, preserving complete
|
|
121
|
+
multibyte UTF-8 characters. When such context is truncated, coverage/union is
|
|
122
|
+
at least `DEGRADED` and its disclosure names the affected consultants; it is
|
|
123
|
+
never a comprehensive claim. A non-coverage strategy can remain
|
|
124
|
+
`NOT_APPLICABLE`, but still exposes this metadata and disclosure.
|
|
125
|
+
|
|
64
126
|
### response.code_snippets
|
|
65
127
|
|
|
66
128
|
Array of code snippets:
|
package/docs/cost_rates.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"version": "
|
|
2
|
+
"version": "5.0.0",
|
|
3
3
|
"updated": "2026-08-23",
|
|
4
4
|
"models": {
|
|
5
5
|
"_comment_units": "All rates are USD per 1,000 tokens (per-1K). lib/costs.sh computes cost as (token_count / 1000) * rate, so every value here MUST be a per-1K rate = (provider $/1M price) / 1000. Do NOT paste $/MTok figures directly.",
|
package/package.json
CHANGED
package/references/details.md
CHANGED
|
@@ -2,32 +2,43 @@
|
|
|
2
2
|
|
|
3
3
|
Read this file when the user asks about presets, strategies, bash usage, best practices, or known limitations.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
<!-- ai-consultants:public-registry:start -->
|
|
6
|
+
<!-- ai-consultants:default-synthesis-strategy=coverage -->
|
|
7
|
+
## Public Modes
|
|
8
|
+
|
|
9
|
+
| Mode | Preset | Default strategy | Use case |
|
|
10
|
+
|---|---|---|---|
|
|
11
|
+
| `fast-check` | `fast` | `coverage` | Fast economy panel for a quick coverage check |
|
|
12
|
+
| `coverage-review` | `balanced` | `coverage` | Balanced panel for a general coverage review |
|
|
13
|
+
| `max-coverage` | `max_quality` | `coverage` | Maximum-quality panel for the broadest coverage |
|
|
6
14
|
|
|
7
|
-
|
|
8
|
-
|--------|-------------|----------|
|
|
9
|
-
| `minimal` | 2 (Gemini + Codex) | Quick questions |
|
|
10
|
-
| `balanced` | 3 (+Mistral) | Standard use |
|
|
11
|
-
| `thorough` | 3 | Comprehensive |
|
|
12
|
-
| `high-stakes` | Expanded panel (4 of 10) | Critical decisions |
|
|
13
|
-
| `security` | Security-focused (3) | Security reviews |
|
|
14
|
-
| `cost-capped` | Budget-friendly | Low cost |
|
|
15
|
-
| `max_quality` | All 10, maximum tier and max effort | Critical decisions |
|
|
16
|
-
| `medium` | 3, standard models | General questions |
|
|
17
|
-
| `fast` | 2, economy models | Quick checks |
|
|
15
|
+
## Configuration Presets
|
|
18
16
|
|
|
19
|
-
|
|
17
|
+
| Preset | Alias | Target | Tier | Use case |
|
|
18
|
+
|---|---|---:|---|---|
|
|
19
|
+
| `minimal` | — | 2 | base | Quick questions |
|
|
20
|
+
| `balanced` | — | 3 | base | Standard coverage |
|
|
21
|
+
| `thorough` | — | 3 | base | Comprehensive analysis |
|
|
22
|
+
| `high-stakes` | — | 4 | premium | Critical decisions |
|
|
23
|
+
| `security` | — | 3 | base | Security reviews |
|
|
24
|
+
| `cost-capped` | — | 3 | economy | Budget-conscious options |
|
|
25
|
+
| `max_quality` | `max-quality` | 10 | maximum | Maximum coverage for critical decisions |
|
|
26
|
+
| `medium` | — | 3 | standard | General questions |
|
|
27
|
+
| `fast` | — | 2 | economy | Quick checks |
|
|
20
28
|
|
|
21
29
|
## Synthesis Strategies
|
|
22
30
|
|
|
23
31
|
| Strategy | Description |
|
|
24
|
-
|
|
25
|
-
| `coverage` | Union of every distinct point across the panel
|
|
26
|
-
| `compare_only` | Present each consultant side-by-side,
|
|
27
|
-
| `majority` |
|
|
28
|
-
| `risk_averse` | Weight conservative responses |
|
|
29
|
-
| `security_first` | Prioritize security |
|
|
30
|
-
| `cost_capped` | Prefer cheaper
|
|
32
|
+
|---|---|
|
|
33
|
+
| `coverage` | Union of every distinct point across the panel **Default** |
|
|
34
|
+
| `compare_only` | Present each consultant side-by-side, without a synthesized union |
|
|
35
|
+
| `majority` | Produce one blended recommendation, weighting all equally |
|
|
36
|
+
| `risk_averse` | Weight conservative responses higher |
|
|
37
|
+
| `security_first` | Prioritize security-focused insights |
|
|
38
|
+
| `cost_capped` | Prefer cheaper consultant opinions within budget |
|
|
39
|
+
<!-- ai-consultants:public-registry:end -->
|
|
40
|
+
|
|
41
|
+
A preset chooses the consultant set + model tier, then uses only statically configured transports (a CLI on `PATH`, or selected API mode with its key). After host self-exclusion it fills canonical slots from `ALL_CONSULTANTS` in canonical order. Explicit `ENABLE_<CONSULTANT>=false` choices remain opt-outs, including during fallback, so a broad preset never silently enrolls a disabled billed API transport. If the effective target cannot be met, including after an enabled health gate prunes the panel, it stops with promised/selected/missing-capacity guidance rather than silently running fewer consultants. Configured custom API agents are appended and can satisfy capacity; a full canonical preset may therefore exceed its advertised count. `max_quality` remains advertised as 10, but has an effective target of 9 under a canonical invoking host because self-exclusion is fail-closed.
|
|
31
42
|
|
|
32
43
|
## Bash Usage
|
|
33
44
|
|