ai-consultants 2.22.0 → 2.23.0

package/.env.example

@@ -207,10 +207,6 @@ SYNTH_MAX=10
 ENABLE_DEBATE=false
 DEBATE_ROUNDS=1              # 1 = no debate, 2-3 = with cross-critique
 
-# Self-Reflection (experimental): generate-critique-refine loop
-ENABLE_REFLECTION=false
-REFLECTION_CYCLES=1
-
 # Peer Review: anonymous evaluation
 ENABLE_PEER_REVIEW=false
 PEER_REVIEW_MIN_RESPONSES=3

package/CLAUDE.md

@@ -6,7 +6,7 @@ AI Consultants is a multi-model AI deliberation system that queries up to 11 AI
 
 **Self-Exclusion**: The invoking agent is automatically excluded from the panel to prevent self-consultation. Claude Code won't query Claude, Codex CLI won't query Codex, etc.
 
-**Version**: 2.22.0
+**Version**: 2.23.0
 
 ## Distribution
 
@@ -64,8 +64,7 @@ ai-consultants/
 │       ├── costs.sh            # Cost tracking + response limits
 │       ├── cache.sh            # Semantic caching (v2.3)
 │       ├── api_query.sh        # API mode query execution (v2.6)
-│       ├── progress.sh         # Progress bars
-│       └── reflection.sh       # Self-reflection + judge step
+│       └── progress.sh         # Progress bars
 ├── references/
 │   ├── configuration.md      # Full configuration reference
 │   └── details.md            # Presets, strategies, best practices
@@ -194,7 +193,7 @@ Functions in `lib/common.sh`:
 ### Configuration Presets
 ```bash
 # Quality Tier Presets (v2.5)
-./scripts/consult_all.sh --preset max_quality "question"  # All + premium models + debate
+./scripts/consult_all.sh --preset max_quality "question"  # 8 of 11 consultants + premium models + peer review
 ./scripts/consult_all.sh --preset medium "question"       # 4 consultants + standard models
 ./scripts/consult_all.sh --preset fast "question"         # 2 consultants + economy models
 
@@ -252,12 +251,6 @@ Process:
 3. Aggregate peer scores to identify strongest arguments
 4. De-anonymize in final report
 
-### Judge Step (Overconfidence Detection)
-Functions in `lib/reflection.sh`:
-- `judge_response()` - Evaluates single response for overconfidence
-- `heuristic_overconfidence_check()` - Fast fallback without LLM
-- `judge_all_responses()` - Batch evaluation
-
 ## v2.9 Features
 
 ### Kimi CLI Support
@@ -410,7 +403,7 @@ get_model_for_tier "claude" "economy"   # → claude-haiku-4-5
 Three new presets leverage the model tiers:
 
 ```bash
-# Maximum quality - all premium models + debate + reflection
+# Maximum quality - all premium models + debate + peer review
 ./scripts/consult_all.sh --preset max_quality "critical architecture decision"
 
 # Balanced quality - standard models, 4 consultants, light debate
@@ -802,6 +795,15 @@ curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/s
 
 ## Changelog
 
+### v2.23.0
+- **`lib/reflection.sh` deleted (452 lines) and `ENABLE_REFLECTION` / `REFLECTION_CYCLES` removed.** The module was never sourced — `grep -rn 'reflection\.sh' scripts/` matched only the file's own header — so the generate-critique-refine loop had never executed. The two flags were declared at `config.sh` and exported by four presets (`high-stakes`, `max_quality`, `fast`, `_disable_all_consultants`) with no consumer anywhere. It had also aged past the codebase: `_exec_consultant` hardcoded a Gemini/Codex/Mistral case (3 of 11 consultants; the rest hit `*) return 1`) and its Gemini branch still used the pre-`agy` CLI syntax retired in v2.15.0 — so wiring it up as written would have produced partial reflection with a broken Gemini path. Removed rather than implemented because v2.16's `converge`/`adversarial` shapes already run critique-refine off measured consensus rather than a fixed cycle count. Worth noting for archaeology: the module was *maintained* in at least three review passes (v2.15.1 de-fencing, the `--model` parity change, the v2.10.9 `((var++))` sweep) — effort spent on code that never ran.
+- **The release gate had been red since 2.22.0 and nobody could see it.** `scripts/release.sh:26` sources `lib/common.sh` → `config.sh` → `load_user_config` *before* `npm test` at line 194, so `_apply_env_file`'s `export "$key=$value"` put the maintainer's entire `~/.config/ai-consultants/.env` into the test environment. Two suites were not hermetic against that, and the first aborted the run before the second could surface:
+  - **`test_user_config.sh`** — `test_xdg_fallback` (Test 6) unsets `AI_CONSULTANTS_CONFIG_DIR` in the suite's own shell (required, to assert the XDG branch) and `_reset_state` never restored it, so the unset leaked into all 22 later tests, which then read the *real* user config. Fixed by re-exporting an empty temp dir in `_reset_state`, making isolation order-independent; tests that legitimately override the dir per-invocation are unaffected. Plus the whole `*_USE_API` set unset at suite top (the guard `test_configure.sh` already carried), and `ENABLE_KIMI`/`GROK_API_URL` (Test 27) and `GEMINI_MODEL`/`PANIC_KEYWORDS` (Test 28) unset inside their subshells — those assert on *real* var names and `load_user_config` lets the environment win over the file by design, so with `config.sh` pre-sourced the file under test was never consulted.
+  - **`test_configure.sh`** — `run_clean_configure` scrubbed the nine API keys but no settings, while `configure`'s `has_explicit_value` reads any ambient value as a deliberate pin. An exported `DEFAULT_PRESET` therefore overrode the value Tests 6 and 10 had just written into the file they assert on. The scrub set is now derived from `configure --show-parameters` — configure's own contract — with a hard failure if that list comes back empty, so the scrub can never silently degrade to a no-op. **Verified pre-existing**: the same two assertions fail at `2ac18d4` (the v2.22.0 release commit) under the same conditions, via a throwaway worktree.
+- **`test_functions.sh` was under-reporting failures across the whole file.** `run_tests` scores a test by its exit status — that of its LAST command — so a failing assertion followed by a passing one printed `FAIL`, incremented nothing, and let the suite exit 0. 8 of the 13 test functions run more than one assertion and none accumulated a result. The five `assert_*` helpers now set `_ASSERT_FAILED` on every failure path and `run_tests` fails a test when either that flag or the exit status is non-zero. **Caught by trying to make the new pin fail**: the first version of `test_known_feature_flags_in_sync` printed `FAIL: ... Actual: PHANTOM_FEATURE` while the suite still reported "All tests passed". Verified against a pre-existing test by breaking the first of three assertions in `test_case_conversion` — previously green, now correctly red.
+- **`KNOWN_FEATURE_FLAGS` synced (9 → 27) and pinned.** `_discover_custom_api_agents` (`consult_all.sh:48`) walks `env` and enrolls any `ENABLE_X=true` as a consultant when `X_API_URL` is set and `X` is absent from the registry, so the 18 missing flags — `PEER_REVIEW`, `HEALTH_GATE`, `SEMANTIC_CACHE`, … — were latent phantom consultants. Measured drift: 18 missing, 0 stale. `test_known_feature_flags_in_sync` derives the expected set from `config.sh`'s top-level `ENABLE_*` declarations minus `KNOWN_CLI_AGENTS`/`KNOWN_API_AGENTS` and asserts both directions, so the next added flag fails the build. (Checked that the `^ENABLE_` grep misses nothing declared only inside functions — the sole extra hit was `AGENTNAME`, from the convention's doc comment.)
+- **Preset tables were documenting a mode that stopped being the default in v2.16.** Found by the `/code-review max` pass on the removal branch: renaming the README column from "Reflection" to "Peer Review" turned a code-enforced "No" into a false claim, because `_disable_all_consultants` exported `ENABLE_REFLECTION=false` — the line this release deletes — whereas nothing ever clears `ENABLE_PEER_REVIEW`. Combined with `consult_all.sh:285` force-enabling peer review for the adversarial shape, `--preset medium` on a SECURITY question runs a full billed peer-review round the table promised it would not. The Debate column was stale for the same reason (`DEBATE_ROUNDS` is only read under `ORCHESTRATION_MODE=fixed`). Both tables now state only what the preset pins, with the planner's role called out. Also corrected `max_quality`'s "All consultants" → 8 of 11, which contradicted the README table in the same commit.
+
 ### v2.22.0
 - Added the public `ai-consultants configure` / `config` subcommand. Automatic mode detects every current consultant, uses CLI-first transport selection with API fallback when credentials are available, persists to the XDG user config directory, preserves custom values and secrets while refreshing availability-derived `ENABLE_*` flags, backs up rewrites, and keeps files at mode 600.
 - Replaced the stale 1,092-line v2.0 configurator with a compact implementation whose accepted parameter surface is derived from `.env.example`. `--set KEY=VALUE` supports repeatable automation and participates in detection, `--interactive` reviews credentials/transports/panel selection, `--advanced` reviews every parameter, `--show-parameters` exposes the contract, and side-effect-free `--dry-run` redacts secrets.

package/README.md

@@ -1,8 +1,8 @@
-# AI Consultants v2.22.0
+# AI Consultants v2.23.0
 
 > **A harness for every question.** A panel of up to 11 frontier models that writes its own playbook per question — fan out, debate to convergence, cross-examine under adversarial review, or run a tournament — and checks its work before it reaches you.
 
-[![Version](https://img.shields.io/badge/version-2.22.0-blue.svg)](https://github.com/matteoscurati/ai-consultants)
+[![Version](https://img.shields.io/badge/version-2.23.0-blue.svg)](https://github.com/matteoscurati/ai-consultants)
 [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
 [![Claude Code Skill](https://img.shields.io/badge/Claude%20Code-Skill-orange.svg)](https://docs.anthropic.com/en/docs/claude-code/skills)
 [![GitHub stars](https://img.shields.io/github/stars/matteoscurati/ai-consultants?style=social)](https://github.com/matteoscurati/ai-consultants)
@@ -428,11 +428,19 @@ Choose the right balance of quality, speed, and cost with model quality tiers.
 
 ### Tier Presets
 
-| Preset | Tier | Agents | Debate | Reflection | Use Case |
-|--------|------|--------|--------|------------|----------|
-| `max_quality` | Premium | 8 | 3 rounds | 2 cycles + peer review | Critical decisions |
-| `medium` | Standard | 4 | 1 round | No | General questions |
-| `fast` | Economy | 2 | No | No | Quick checks |
+| Preset | Tier | Agents | Use Case |
+|--------|------|--------|----------|
+| `max_quality` | Premium | 8 of 11 | Critical decisions |
+| `medium` | Standard | 4 | General questions |
+| `fast` | Economy | 2 | Quick checks |
+
+**Deliberation depth is not fixed by the preset.** Under the default
+`ORCHESTRATION_MODE=auto` the planner picks the shape per question, so the
+`DEBATE_ROUNDS` a preset pins applies only under `ORCHESTRATION_MODE=fixed`,
+and a `SECURITY`-classified question runs the adversarial shape — including a
+peer-review round — whatever preset you chose. `max_quality` additionally
+enables peer review outright; `medium` and `fast` do not *clear* it, so an
+enabled peer review carries into them.
 
 ### Models by Tier
 
@@ -479,8 +487,8 @@ Choose how many consultants to use:
 
 | Preset | Consultants | Tier | Use Case |
 |--------|-------------|------|----------|
-| `max_quality` | 8 + debate + reflection | Premium | Critical decisions |
-| `medium` | 4 + light debate | Standard | General questions |
+| `max_quality` | 8 of 11 (+ peer review) | Premium | Critical decisions |
+| `medium` | 4 | Standard | General questions |
 | `fast` | 2 | Economy | Quick checks |
 | `minimal` | 2 (Gemini + Codex) | Default | Quick questions, low cost |
 | `balanced` | 4 (+ Mistral + Cursor) | Default | Standard consultations |
@@ -717,6 +725,14 @@ Each consultation generates:
 
 ## Changelog
 
+### v2.23.0
+
+- **`ENABLE_REFLECTION` and `REFLECTION_CYCLES` removed.** The self-reflection module they controlled was never called by anything, so the settings had never affected a consultation — v2.16's `converge` and `adversarial` shapes already run critique-refine off measured consensus. **If you set either key**, remove it: `configure --set ENABLE_REFLECTION=...` now exits non-zero, and both keys are dropped when an existing `.env` is rewritten.
+- **The release gate works again.** Two test suites failed deterministically against any real user configuration, so no release since 2.22.0 could pass its own `npm test`.
+- **`npm test` no longer swallows failures.** A failing assertion followed by a passing one inside the same test function was counted as a pass, across 8 of 13 test functions.
+- **Feature flags can no longer be enrolled as consultants.** A registry that told feature flags apart from custom API agents was 18 entries out of date; it is now complete and pinned by a test.
+- **Preset tables corrected** — they described a debate and peer-review depth the code has not enforced since v2.16, and called `high-stakes`/`max_quality` "all consultants" when they enable 5 and 8 of 11. See the [v2.23.0 release note](docs/releases/v2.23.0.md).
+
 ### v2.22.0
 
 - **`ai-consultants configure`**: a public automatic configurator. It detects your installed CLIs and API keys, picks CLI-first transports, and writes a private XDG config while preserving your existing settings and secrets. Drive it with repeatable `--set KEY=VALUE`, review it with `--interactive` / `--advanced`, or preview it with `--dry-run`. See the [v2.22.0 release note](docs/releases/v2.22.0.md).

package/SKILL.md

@@ -5,10 +5,10 @@ license: MIT
 compatibility: Requires bash, jq, and at least 2 AI CLI tools (agy, codex, vibe, etc.). macOS and Linux.
 metadata:
   author: matteoscurati
-  version: 2.22.0
+  version: 2.23.0
 ---
 
-# AI Consultants v2.22.0 - AI Expert Panel
+# AI Consultants v2.23.0 - AI Expert Panel
 
 **A harness for every question.** Convene a panel of AI "consultants" for coding questions — and let the engine pick *how* they deliberate: a quick read, a convergence loop, an adversarial refutation gate, a tournament, or an exhaustive sweep. Each consultant has a **configurable persona** that shapes its analysis.
 

package/docs/COST_RATES.md

@@ -1,4 +1,4 @@
-# Cost Rates - AI Consultants v2.22.0
+# Cost Rates - AI Consultants v2.23.0
 
 This page documents the per-token rates used by the cost tracking system.
 
@@ -102,14 +102,6 @@ Since v2.5, models are organized into three tiers. Use `apply_model_tier()` or `
 
 **Estimated cost:** $0.08 - $0.15
 
-### With Self-Reflection
-
-**Additional parameters:**
-- Critique + refinement cycle per consultant
-- ~500 additional tokens per cycle
-
-**Estimated cost:** $0.10 - $0.20
-
 ## Budget Management
 
 ### Configuration Variables

package/docs/RECIPES.md

@@ -102,8 +102,6 @@ ORCHESTRATION_MODE=adversarial
 ENABLE_ADVERSARIAL_VERIFY=true
 ENABLE_PEER_REVIEW=true
 PEER_REVIEW_MIN_RESPONSES=3
-ENABLE_REFLECTION=true
-REFLECTION_CYCLES=1
 ```
 
 ```bash

package/docs/cost_rates.json

@@ -1,5 +1,5 @@
 {
-  "version": "2.22.0",
+  "version": "2.23.0",
   "updated": "2026-07-16",
   "models": {
     "_comment_units": "All rates are USD per 1,000 tokens (per-1K). lib/costs.sh computes cost as (token_count / 1000) * rate, so every value here MUST be a per-1K rate = (provider $/1M price) / 1000. Do NOT paste $/MTok figures directly.",

package/docs/releases/v2.23.0.md

@@ -0,0 +1,75 @@
+# Release v2.23.0
+
+**Date:** 2026-07-20
+**Type:** Minor — removes a dead feature flag and repairs the release gate
+**Previous:** v2.22.0
+
+## Highlights
+
+- **`ENABLE_REFLECTION` / `REFLECTION_CYCLES` removed.** The self-reflection module they controlled was never called by anything, so the settings had never affected a consultation.
+- **The release gate works again.** `scripts/release.sh` could not pass its own `npm test` since 2.22.0; two test suites failed deterministically against any real user configuration.
+- **`npm test` now reports failures it used to swallow.** A failing assertion followed by a passing one inside the same test function was counted as a pass.
+- **A registry that could enroll feature flags as consultants is fixed**, and pinned by a test so it cannot drift again.
+- **Preset documentation corrected** — the tables described a deliberation depth the code has not enforced since v2.16.
+
+## What's New
+
+### Self-reflection is gone
+
+`lib/reflection.sh` implemented a generate-critique-refine loop across 452 lines. Nothing ever sourced it. `ENABLE_REFLECTION` and `REFLECTION_CYCLES` were declared in `config.sh` and set by four presets, but no consumer read them — the flags were configuration that promised behavior the tool never performed.
+
+The module had also aged past the codebase. Its `_exec_consultant` hardcoded a three-branch dispatch (Gemini/Codex/Mistral) predating both the 11-consultant roster and the CLI/API transport layer, and its Gemini branch still used the CLI syntax retired in v2.15.0 by the `agy` migration. Wiring it up as written would have produced reflection for 3 of 11 consultants with a broken Gemini path.
+
+It was removed rather than implemented because v2.16 already covers the ground: the `converge` and `adversarial` orchestration shapes run critique-refine driven by measured consensus instead of a fixed cycle count.
+
+### The release gate
+
+`scripts/release.sh` sources `lib/common.sh` — and so `config.sh`, and so `load_user_config` — before running `npm test`. Every setting in the maintainer's own `~/.config/ai-consultants/.env` was therefore exported into the test process, and two suites were not isolated against it:
+
+- **`test_user_config.sh`** — `test_xdg_fallback` unsets `AI_CONSULTANTS_CONFIG_DIR` in the suite's own shell, which it must in order to assert the XDG branch, but nothing restored it. The unset leaked into all 22 following tests, which then read the real user config. Two further tests asserted on real configuration variable names that an ambient export silently replaced.
+- **`test_configure.sh`** — its `run_clean_configure` helper scrubbed the nine API keys but no settings, while `configure`'s `has_explicit_value` reads *any* ambient value as a deliberate user pin. An exported `DEFAULT_PRESET` overrode the value the preservation tests had just written into the file they were asserting on.
+
+Both suites now derive their isolation from the contract they test rather than from a hand-listed subset: `test_user_config.sh` re-establishes an empty config directory before every test, and `test_configure.sh` builds its scrub set from `configure --show-parameters`.
+
+This is why the failure was invisible: `configure` shipped in 2.22.0, so only a maintainer who had actually run it hit the condition, and the first failing suite aborted the run before the second could be seen.
+
+### Test failures that were being swallowed
+
+`test_functions.sh` scores a test function by its exit status, which in bash is that of its last command. A failing assertion followed by a passing one printed `FAIL`, counted as a pass, left the failure total at zero and exited 0. Eight of the thirteen test functions run more than one assertion and were exposed.
+
+The `assert_*` helpers now record failures in a flag the runner consults, so a test fails if either its status or any assertion inside it did.
+
+### Phantom consultants from feature flags
+
+`_discover_custom_api_agents` walks the environment and enrolls any `ENABLE_X=true` as a consultant when an `X_API_URL` is also present and `X` is absent from `KNOWN_FEATURE_FLAGS`. That registry listed 9 entries against 27 live non-consultant flags, so 18 — `ENABLE_PEER_REVIEW` and `ENABLE_HEALTH_GATE` among them — were latent phantom consultants. All 27 are registered, and a test derives the expected set from `config.sh` so the next added flag fails the build instead of drifting silently.
+
+### Preset documentation
+
+The tier-preset tables stated a per-preset debate and peer-review depth that the code does not enforce. Under the default `ORCHESTRATION_MODE=auto` the planner chooses the shape per question: a `DEBATE_ROUNDS` value a preset pins applies only under `ORCHESTRATION_MODE=fixed`, and a `SECURITY`-classified question runs the adversarial shape — peer review included — whatever preset was chosen. `max_quality` was also described as "all consultants" while enabling 8 of the 11-consultant roster.
+
+## Breaking Changes
+
+**`ENABLE_REFLECTION` and `REFLECTION_CYCLES` are no longer accepted parameters.**
+
+No runtime behavior changes — the flags controlled nothing. What changes is the configuration surface:
+
+- `ai-consultants configure --set ENABLE_REFLECTION=...` now exits non-zero instead of accepting the value. A provisioning script that pins either key will abort after upgrading.
+- Both keys are dropped, without warning, the next time `configure` rewrites an existing user `.env`.
+
+## Upgrade Guide
+
+```bash
+# If installed via git clone
+cd ~/.claude/skills/ai-consultants && git pull
+
+# If installed via curl | bash
+curl -fsSL https://raw.githubusercontent.com/matteoscurati/ai-consultants/main/scripts/install.sh | bash
+
+# If installed via npx — nothing to do, npx resolves the published version
+```
+
+Remove `ENABLE_REFLECTION` and `REFLECTION_CYCLES` from any `.env` file or provisioning script that sets them. No other configuration changes are required.
+
+## Contributors
+
+- matteoscurati

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ai-consultants",
-  "version": "2.22.0",
+  "version": "2.23.0",
   "description": "A harness for every question - a panel of up to 11 AI models that picks its own deliberation strategy per question (convergence, adversarial verification, tournament) and checks its work before answering",
   "bin": {
     "ai-consultants": "bin/ai-consultants"

package/references/configuration.md

@@ -164,8 +164,6 @@ ENABLE_DEBATE=false          # Multi-agent debate
 DEBATE_ROUNDS=1              # Used by ORCHESTRATION_MODE=fixed
 ENABLE_PEER_REVIEW=false     # Anonymous peer review
 PEER_REVIEW_MIN_RESPONSES=3  # Minimum panel size for peer review
-ENABLE_REFLECTION=false      # Generate -> critique -> refine
-REFLECTION_CYCLES=1
 ENABLE_PANIC_MODE=auto       # auto | always | never
 ENABLE_SMART_ROUTING=false   # Category-based consultant selection
 ENABLE_COST_TRACKING=true    # Track API usage costs

package/scripts/config.sh

@@ -374,16 +374,6 @@ PANIC_EXTRA_DEBATE_ROUNDS="${PANIC_EXTRA_DEBATE_ROUNDS:-1}"
 # Keywords that trigger panic mode when found in responses
 PANIC_KEYWORDS="${PANIC_KEYWORDS:-uncertain|maybe|not sure|possibly|unclear|depends|hard to say|difficult to determine}"
 
-# =============================================================================
-# SELF-REFLECTION (v2.0)
-# =============================================================================
-
-# Enable auto-reflection (generate-critique-refine)
-ENABLE_REFLECTION="${ENABLE_REFLECTION:-false}"
-
-# Number of reflection cycles per response
-REFLECTION_CYCLES="${REFLECTION_CYCLES:-1}"
-
 # =============================================================================
 # SMART ROUTING (v2.0)
 # =============================================================================
@@ -761,7 +751,7 @@ apply_model_tier() {
 #   high-stakes  - Broad premium panel + debate (maximum rigor)
 #
 # Quality Tiers (v2.5):
-#   max_quality  - All consultants + premium models + debate + reflection
+#   max_quality  - 8 of 11 consultants + premium models + peer review
 #   medium       - 4 consultants + standard models + light debate
 #   fast         - 2 consultants + economy models, no debate
 #
@@ -773,7 +763,7 @@ _disable_all_consultants() {
     export ENABLE_CURSOR=false ENABLE_KIMI=false ENABLE_CLAUDE=false
     export ENABLE_QWEN3=false ENABLE_GLM=false ENABLE_GROK=false
     export ENABLE_DEEPSEEK=false ENABLE_MINIMAX=false
-    export ENABLE_DEBATE=false ENABLE_REFLECTION=false
+    export ENABLE_DEBATE=false
 }
 
 # Apply a preset configuration
@@ -800,7 +790,6 @@ apply_preset() {
             export ENABLE_GEMINI=true ENABLE_CODEX=true ENABLE_MISTRAL=true
             export ENABLE_CURSOR=true ENABLE_CLAUDE=true
             export ENABLE_DEBATE=true DEBATE_ROUNDS=2
-            export ENABLE_REFLECTION=true REFLECTION_CYCLES=1
             ;;
         security)
             export ENABLE_GEMINI=true ENABLE_CODEX=true
@@ -820,7 +809,6 @@ apply_preset() {
             export ENABLE_CURSOR=true ENABLE_KIMI=true
             export ENABLE_CLAUDE=true ENABLE_QWEN3=true ENABLE_MINIMAX=true
             export ENABLE_DEBATE=true DEBATE_ROUNDS=3
-            export ENABLE_REFLECTION=true REFLECTION_CYCLES=2
             export ENABLE_PEER_REVIEW=true
             ;;
         medium)
@@ -834,7 +822,7 @@ apply_preset() {
             # Super fast - economy models, minimal consultants
             apply_model_tier "economy"
             export ENABLE_GEMINI=true ENABLE_CODEX=true
-            export ENABLE_DEBATE=false ENABLE_REFLECTION=false
+            export ENABLE_DEBATE=false
             export ENABLE_COMPACT_REPORT=true
             ;;
         *)
@@ -853,7 +841,7 @@ list_presets() {
 Available presets:
 
 Quality Tiers (v2.5):
-  max_quality  All consultants + premium models + debate + reflection
+  max_quality  8 of 11 consultants + premium models + peer review
   medium       4 consultants + standard models + light debate
   fast         2 consultants + economy models, no debate
 
@@ -873,4 +861,4 @@ EOF
 # VERSION
 # =============================================================================
 
-AI_CONSULTANTS_VERSION="2.22.0"
+AI_CONSULTANTS_VERSION="2.23.0"

package/scripts/consult_all.sh

@@ -82,7 +82,7 @@ Options:
                          balanced     - 4 models, good coverage
                          thorough     - 5 models, comprehensive
                          high-stakes  - All models + debate
-                         max_quality  - All + premium models + debate + reflection
+                         max_quality  - 8 of 11 consultants + premium models + peer review
                          medium       - 4 models + standard models + light debate
                          fast         - 2 models + economy models, no debate
                          security     - Security-focused + debate

package/scripts/lib/common.sh

@@ -276,7 +276,21 @@ to_title() {
 # This list is used by discovery functions to identify custom agents
 KNOWN_CLI_AGENTS="GEMINI CODEX MISTRAL CURSOR KIMI CLAUDE QWEN3 MINIMAX"
 KNOWN_API_AGENTS="GLM GROK DEEPSEEK"
-KNOWN_FEATURE_FLAGS="PERSONA SYNTHESIS DEBATE REFLECTION CLASSIFICATION SMART_ROUTING COST_TRACKING PROGRESS_BARS EARLY_TERMINATION PREFLIGHT"
+
+# Every non-consultant ENABLE_* flag declared in config.sh. This is a denylist
+# guarding _discover_custom_api_agents (consult_all.sh): that function walks
+# `env`, and any ENABLE_X=true whose X is absent here is enrolled as a custom
+# consultant once an X_API_URL is also present. A flag missing from this list is
+# therefore a latent phantom-consultant bug, not a cosmetic omission -- it had
+# drifted 18 flags out of date before the pin below existed.
+# KEEP IN SYNC with the top-level ENABLE_* declarations in config.sh;
+# test_functions.sh::test_known_feature_flags_in_sync fails the build otherwise.
+KNOWN_FEATURE_FLAGS="PERSONA SYNTHESIS DEBATE CLASSIFICATION SMART_ROUTING \
+COST_TRACKING PROGRESS_BARS EARLY_TERMINATION PREFLIGHT ADVERSARIAL_VERIFY \
+AST_EXTRACTION BUDGET_LIMIT CAPABILITY_ROUTING CAPABILITY_WEIGHTING \
+COMPACT_REPORT COST_AWARE_ROUTING DEBATE_OPTIMIZATION HEALTH_GATE PANIC_MODE \
+PEER_REVIEW RELIABILITY_TRACKING RESPONSE_LIMITS SELECTIVE_CONTEXT \
+SEMANTIC_CACHE SEMANTIC_CHUNKING STANCE_CONSENSUS SYMBOL_COMPRESSION"
 
 # Check if an agent name is a known predefined agent
 # Usage: is_known_agent "AGENTNAME"

package/scripts/lib/reflection.sh

@@ -1,452 +0,0 @@
-#!/bin/bash
-# reflection.sh - Self-reflection (Reflexion pattern) for AI Consultants
-#
-# Implements the generate-critique-refine pattern to improve
-# response quality through self-criticism.
-
-# =============================================================================
-# REFLECTION PROMPTS
-# =============================================================================
-
-# Prompt for self-critique
-CRITIQUE_PROMPT_TEMPLATE='You provided this response to a programming question:
-
----
-%RESPONSE%
----
-
-Now critique your response. Consider:
-1. **Correctness**: Are there logical or technical errors?
-2. **Completeness**: Is anything important missing?
-3. **Clarity**: Is it well explained?
-4. **Edge cases**: Have you considered all edge cases?
-5. **Alternatives**: Are there better approaches not mentioned?
-6. **Risks**: Are there hidden potential problems?
-
-Respond in JSON:
-{
-  "critique": {
-    "strengths": ["<strengths of the response>"],
-    "weaknesses": ["<weak points>"],
-    "missing_aspects": ["<what is missing>"],
-    "errors_found": ["<identified errors>"],
-    "improvement_suggestions": ["<how to improve>"]
-  },
-  "overall_quality": <1-10>,
-  "needs_refinement": <true if significant improvement is needed>
-}'
-
-# Prompt for refinement
-REFINE_PROMPT_TEMPLATE='Your original response was:
-
----
-%ORIGINAL%
----
-
-Your self-critique identified:
-
----
-%CRITIQUE%
----
-
-Now produce an IMPROVED response that:
-1. Corrects the identified errors
-2. Adds the missing aspects
-3. Strengthens the weak points
-4. Maintains the strengths
-
-Respond with the same JSON schema as the original response, but improved.'
-
-# =============================================================================
-# REFLECTION FUNCTIONS
-# =============================================================================
-
-# Helper: Execute prompt with a specific consultant
-# Usage: _exec_consultant <consultant> <prompt>
-_exec_consultant() {
-    local consultant="$1"
-    local prompt="$2"
-
-    case "$consultant" in
-        Gemini)  echo "$prompt" | "$GEMINI_CMD" -p - --model "$GEMINI_MODEL" 2>/dev/null ;;
-        Codex)   "$CODEX_CMD" exec --skip-git-repo-check "$prompt" 2>/dev/null ;;
-        Mistral) "$MISTRAL_CMD" --prompt "$prompt" --auto-approve 2>/dev/null ;;
-        *)       return 1 ;;
-    esac
-}
-
-# Generate critique of a response
-# Usage: generate_critique <consultant> <response_content>
-# Output: JSON with the critique
-generate_critique() {
-    local consultant="$1"
-    local response="$2"
-    local prompt="${CRITIQUE_PROMPT_TEMPLATE//%RESPONSE%/$response}"
-
-    _exec_consultant "$consultant" "$prompt" || \
-        echo '{"critique": {"error": "Unknown consultant"}, "needs_refinement": false}'
-}
-
-# Refine a response based on critique
-# Usage: refine_response <consultant> <original_response> <critique>
-# Output: refined response
-refine_response() {
-    local consultant="$1"
-    local original="$2"
-    local critique="$3"
-
-    local prompt="${REFINE_PROMPT_TEMPLATE//%ORIGINAL%/$original}"
-    prompt="${prompt//%CRITIQUE%/$critique}"
-
-    _exec_consultant "$consultant" "$prompt"
-}
-
-# Execute complete reflection cycle
-# Usage: run_reflection_cycle <consultant> <query> <output_file> [cycles]
-run_reflection_cycle() {
-    local consultant="$1"
-    local query="$2"
-    local output_file="$3"
-    local cycles="${4:-1}"
-
-    local current_response=""
-    local reflection_history=()
-
-    # Step 1: Get initial response (already done externally)
-    if [[ -f "$output_file" && -s "$output_file" ]]; then
-        current_response=$(cat "$output_file")
-    else
-        log_warn "No initial response found for reflection"
-        return 1
-    fi
-
-    # Reflection cycles
-    for ((cycle=1; cycle<=cycles; cycle++)); do
-        log_debug "[$consultant] Reflection cycle $cycle/$cycles"
-
-        # Step 2: Self-critique
-        local critique=$(generate_critique "$consultant" "$current_response")
-        # CLIs like agy wrap JSON in a ```json fence; strip it so the jq reads
-        # below (and the final response write) see valid JSON.
-        critique=$(strip_json_fence "$critique")
-
-        # Check if refinement is needed
-        local needs_refinement=$(echo "$critique" | jq -r '.needs_refinement // false' 2>/dev/null)
-
-        if [[ "$needs_refinement" == "false" && $cycle -gt 1 ]]; then
-            log_debug "[$consultant] No refinement needed, stopping at cycle $cycle"
-            break
-        fi
-
-        # Save to history
-        reflection_history+=("$critique")
-
-        # Step 3: Refine
-        local refined=$(refine_response "$consultant" "$current_response" "$critique")
-
-        if [[ -n "$refined" ]]; then
-            current_response=$(strip_json_fence "$refined")
-        else
-            log_warn "[$consultant] Refinement failed, keeping previous response"
-            break
-        fi
-    done
-
-    # Add reflection metadata to final result
-    if echo "$current_response" | jq -e '.' > /dev/null 2>&1; then
-        echo "$current_response" | jq \
-            --argjson cycles "$cycles" \
-            --argjson history "$(printf '%s\n' ${reflection_history[@]+"${reflection_history[@]}"} | jq -s '.')" \
-            '. + {reflection: {cycles_completed: $cycles, history: $history}}' \
-            > "$output_file"
-    else
-        echo "$current_response" > "$output_file"
-    fi
-}
-
-# =============================================================================
-# QUALITY ASSESSMENT
-# =============================================================================
-
-# Evaluate if a response would benefit from reflection
-# Usage: should_reflect <response_json>
-# Returns: 0 if yes, 1 if no
-should_reflect() {
-    local response="$1"
-
-    # Check confidence score
-    local confidence=$(echo "$response" | jq -r '.confidence.score // 5' 2>/dev/null)
-
-    # If confidence is low, reflect
-    if [[ $confidence -lt 6 ]]; then
-        return 0
-    fi
-
-    # Check if there are many caveats
-    local caveats_count=$(echo "$response" | jq -r '.response.caveats | length // 0' 2>/dev/null)
-    if [[ $caveats_count -gt 3 ]]; then
-        return 0
-    fi
-
-    # Check uncertainty factors
-    local uncertainty_count=$(echo "$response" | jq -r '.confidence.uncertainty_factors | length // 0' 2>/dev/null)
-    if [[ $uncertainty_count -gt 2 ]]; then
-        return 0
-    fi
-
-    return 1
-}
-
-# Calculate quality score of a response
-# Usage: calculate_quality_score <response_json>
-calculate_quality_score() {
-    local response="$1"
-
-    local confidence=$(echo "$response" | jq -r '.confidence.score // 5' 2>/dev/null)
-    local has_code=$(echo "$response" | jq -r '.response.code_snippets | length > 0' 2>/dev/null)
-    local has_pros=$(echo "$response" | jq -r '.response.pros | length > 0' 2>/dev/null)
-    local has_cons=$(echo "$response" | jq -r '.response.cons | length > 0' 2>/dev/null)
-
-    local score=$confidence
-
-    # Bonus for completeness
-    [[ "$has_code" == "true" ]] && score=$((score + 1))
-    [[ "$has_pros" == "true" ]] && score=$((score + 1))
-    [[ "$has_cons" == "true" ]] && score=$((score + 1))
-
-    # Cap at 10
-    if [[ $score -gt 10 ]]; then
-        score=10
-    fi
-
-    echo $score
-}
-
-# =============================================================================
-# JUDGE STEP - OVERCONFIDENCE DETECTION (v2.2)
-# =============================================================================
-
-# Prompt template for the judge evaluation
-JUDGE_PROMPT_TEMPLATE='You are an expert meta-evaluator. Your job is to detect OVERCONFIDENCE in AI responses.
-
-Analyze this response for overconfidence signals:
-
----
-CONSULTANT: %CONSULTANT%
-CONFIDENCE SCORE: %CONFIDENCE%/10
-
-SUMMARY:
-%SUMMARY%
-
-DETAILED RESPONSE:
-%DETAILED%
-
-REASONING FOR CONFIDENCE:
-%REASONING%
----
-
-Evaluate for overconfidence by checking:
-
-1. **Hedging vs Certainty Mismatch**: Does the text use hedging language ("might", "could", "possibly") but claim high confidence?
-2. **Evidence Quality**: Is the confidence score justified by the evidence/reasoning provided?
-3. **Complexity Acknowledgment**: Does a high confidence score acknowledge the complexity of the problem?
-4. **Edge Cases**: Are edge cases mentioned? High confidence without edge case consideration is suspicious.
-5. **Alternative Awareness**: Does the response acknowledge alternative approaches exist?
-
-Respond ONLY with valid JSON:
-{
-  "consultant": "%CONSULTANT%",
-  "original_confidence": %CONFIDENCE%,
-  "overconfidence_detected": <true/false>,
-  "adjusted_confidence": <1-10>,
-  "analysis": {
-    "hedging_language_count": <number of hedging phrases found>,
-    "certainty_claims_count": <number of certainty claims>,
-    "evidence_quality": "<weak|moderate|strong>",
-    "complexity_acknowledged": <true/false>,
-    "edge_cases_mentioned": <true/false>,
-    "alternatives_acknowledged": <true/false>
-  },
-  "red_flags": ["<specific overconfidence indicators found>"],
-  "recommendation": "<keep|adjust_down|flag_for_review>"
-}'
-
-# Run judge evaluation on a single response
-# Usage: judge_response <response_file>
-# Output: JSON with judge evaluation
-judge_response() {
-    local response_file="$1"
-
-    if [[ ! -f "$response_file" || ! -s "$response_file" ]]; then
-        echo '{"error": "Invalid response file"}'
-        return 1
-    fi
-
-    local consultant confidence summary detailed reasoning
-    consultant=$(jq -r '.consultant // "unknown"' "$response_file" 2>/dev/null)
-    confidence=$(jq -r '.confidence.score // 5' "$response_file" 2>/dev/null)
-    summary=$(jq -r '.response.summary // "No summary"' "$response_file" 2>/dev/null)
-    detailed=$(jq -r '.response.detailed // "No details"' "$response_file" 2>/dev/null | head -c 2000)
-    reasoning=$(jq -r '.confidence.reasoning // "No reasoning provided"' "$response_file" 2>/dev/null)
-
-    local prompt="${JUDGE_PROMPT_TEMPLATE}"
-    prompt="${prompt//%CONSULTANT%/$consultant}"
-    prompt="${prompt//%CONFIDENCE%/$confidence}"
-    prompt="${prompt//%SUMMARY%/$summary}"
-    prompt="${prompt//%DETAILED%/$detailed}"
-    prompt="${prompt//%REASONING%/$reasoning}"
-
-    # Use Claude for judging (most reliable for meta-analysis)
-    if command -v claude &> /dev/null; then
-        echo "$prompt" | claude --print 2>/dev/null
-    else
-        # Fallback: heuristic-based detection
-        heuristic_overconfidence_check "$response_file"
-    fi
-}
-
-# Heuristic-based overconfidence detection (no LLM required)
-# Usage: heuristic_overconfidence_check <response_file>
-heuristic_overconfidence_check() {
-    local response_file="$1"
-
-    local consultant confidence text
-    consultant=$(jq -r '.consultant // "unknown"' "$response_file" 2>/dev/null)
-    confidence=$(jq -r '.confidence.score // 5' "$response_file" 2>/dev/null)
-    text=$(jq -r '(.response.summary // "") + " " + (.response.detailed // "") + " " + (.confidence.reasoning // "")' "$response_file" 2>/dev/null | tr '[:upper:]' '[:lower:]')
-
-    # Count hedging language
-    local hedging_words="might|could|possibly|perhaps|maybe|likely|probably|seems|appears|suggest"
-    local hedging_count
-    hedging_count=$(echo "$text" | grep -oiE "$hedging_words" | wc -l | tr -d ' ')
-
-    # Count certainty language
-    local certainty_words="definitely|certainly|absolutely|always|never|must|clearly|obviously|undoubtedly"
-    local certainty_count
-    certainty_count=$(echo "$text" | grep -oiE "$certainty_words" | wc -l | tr -d ' ')
-
-    # Check for edge cases mention
-    local edge_case_mentioned=false
-    echo "$text" | grep -qiE "edge case|corner case|exception|special case" && edge_case_mentioned=true
-
-    # Check for alternatives mention
-    local alternatives_mentioned=false
-    echo "$text" | grep -qiE "alternative|another approach|other option|could also|alternatively" && alternatives_mentioned=true
-
-    # Calculate overconfidence indicators
-    local red_flags=()
-    local overconfidence=false
-    local adjusted=$confidence
-
-    # High confidence + lots of hedging = suspicious
-    if [[ $confidence -ge 8 && $hedging_count -ge 3 ]]; then
-        red_flags+=("High confidence with excessive hedging language")
-        overconfidence=true
-        adjusted=$((confidence - 2))
-    fi
-
-    # High confidence + no edge cases = suspicious
-    if [[ $confidence -ge 8 && "$edge_case_mentioned" == "false" ]]; then
-        red_flags+=("High confidence without edge case consideration")
-        overconfidence=true
-        adjusted=$((adjusted - 1))
-    fi
-
-    # Very high confidence is almost always overconfident
-    if [[ $confidence -ge 9 ]]; then
-        red_flags+=("Extremely high confidence (9-10) is rarely justified")
-        overconfidence=true
-        adjusted=$((adjusted - 1))
-    fi
-
-    # Clamp adjusted confidence
-    [[ $adjusted -lt 1 ]] && adjusted=1
-    [[ $adjusted -gt 10 ]] && adjusted=10
-
-    local recommendation="keep"
-    if [[ "$overconfidence" == "true" ]]; then
-        if [[ $((confidence - adjusted)) -ge 2 ]]; then
-            recommendation="adjust_down"
-        else
-            recommendation="flag_for_review"
-        fi
-    fi
-
-    jq -n \
-        --arg consultant "$consultant" \
-        --argjson original "$confidence" \
-        --argjson overconfidence "$overconfidence" \
-        --argjson adjusted "$adjusted" \
-        --argjson hedging "$hedging_count" \
-        --argjson certainty "$certainty_count" \
-        --argjson edge_cases "$edge_case_mentioned" \
-        --argjson alternatives "$alternatives_mentioned" \
-        --arg red_flags "$(IFS='|'; echo "${red_flags[*]:-}")" \
-        --arg recommendation "$recommendation" \
-        '{
-            consultant: $consultant,
-            original_confidence: $original,
-            overconfidence_detected: $overconfidence,
-            adjusted_confidence: $adjusted,
-            analysis: {
-                hedging_language_count: $hedging,
-                certainty_claims_count: $certainty,
-                evidence_quality: (if $hedging > $certainty then "weak" elif $hedging == $certainty then "moderate" else "strong" end),
-                complexity_acknowledged: ($hedging > 0),
-                edge_cases_mentioned: $edge_cases,
-                alternatives_acknowledged: $alternatives
-            },
-            red_flags: ($red_flags | split("|") | map(select(. != ""))),
-            recommendation: $recommendation
-        }'
-}
-
-# Run judge evaluation on all responses in a directory
-# Usage: judge_all_responses <responses_dir> <output_file>
-judge_all_responses() {
-    local responses_dir="$1"
-    local output_file="$2"
-
-    local results='[]'
-    local overconfidence_count=0
-    local total_count=0
-
-    for f in "$responses_dir"/*.json; do
-        # Shared metadata filter (common.sh): also excludes orchestration.json,
-        # panic_diagnosis.json, optimization_metrics.json, stance_options.json,
-        # which this loop previously judged as if they were consultant answers.
-        if _is_consultant_response_file "$f"; then
-            local evaluation
-            evaluation=$(judge_response "$f")
-
-            if echo "$evaluation" | jq -e '.' > /dev/null 2>&1; then
-                results=$(echo "$results" | jq --argjson eval "$evaluation" '. + [$eval]')
-
-                local is_overconfident
-                is_overconfident=$(echo "$evaluation" | jq -r '.overconfidence_detected // false')
-                [[ "$is_overconfident" == "true" ]] && { ((overconfidence_count++)) || true; }
-                ((total_count++)) || true
-            fi
-        fi
-    done
-
-    # Generate summary
-    jq -n \
-        --argjson evaluations "$results" \
-        --argjson overconfident "$overconfidence_count" \
-        --argjson total "$total_count" \
-        '{
-            judge_report: {
-                timestamp: (now | todate),
-                total_evaluated: $total,
-                overconfidence_detected: $overconfident,
-                evaluations: $evaluations,
-                summary: {
-                    reliability: (if $overconfident == 0 then "high" elif ($overconfident / $total) < 0.3 then "medium" else "low" end),
-                    action_required: ($overconfident > 0)
-                }
-            }
-        }' > "$output_file"
-
-    cat "$output_file"
-}