coderouter-cli 2.6.0__tar.gz → 2.7.0__tar.gz

{coderouter_cli-2.6.0 → coderouter_cli-2.7.0}/.gitignore

@@ -36,6 +36,7 @@ ENV/
 # _OUTPUTS
 # ============================================================
 _OUTPUTS/
+_run/
 
 # uv
 # Note: uv.lock SHOULD be committed (per plan.md §5.4)

{coderouter_cli-2.6.0 → coderouter_cli-2.7.0}/CHANGELOG.md

@@ -6,6 +6,188 @@ versioning follows [SemVer](https://semver.org/).
 
 ---
 
+## [v2.7.0] — 2026-07-02 (Reliability & security: full-source review fixes)
+
+22 reliability and security fixes from a full-source review (26,600 lines),
+landed as two sequential PRs: high-priority **PR #34** (H1–H8) and
+medium-priority **PR #35** (M1–M14), both merged to `main`. No API or
+config-schema breaking changes. Backward compatibility highlights: the new
+launcher token auth is **opt-in** (`CODEROUTER_LAUNCHER_TOKEN` unset ⇒
+unchanged behavior + a startup warning), and the Host-validation middleware
+allows loopback by default with `CODEROUTER_ALLOWED_HOSTS` to extend it.
+
+### Fixed
+
+- **`/metrics` 500 when drift counters are non-zero** — three malformed
+  Prometheus label tuples (`(((),), v)` → `((), v)`) (`coderouter/metrics/prometheus.py`, PR #34/H1).
+- **`tool_repair` deleted unrelated fenced code blocks** while stripping
+  tool-call-shaped ones, losing user content
+  (`coderouter/translation/tool_repair.py`, PR #34/H2).
+- **Probe URL mismatch (`/v1/v1` duplication)** between the continuous probe
+  and the adapters caused healthy backends to be wrongly demoted; URL
+  normalization is now shared
+  (`coderouter/adapters/anthropic_native.py`, `coderouter/guards/continuous_probe.py`, PR #34/H4).
+- **`context_budget` dropped entire history at once** on threshold breach;
+  trimming is now sequential per tool-pair with re-estimation after each
+  step, and the new head turn is renormalized to a plain user turn
+  (`coderouter/guards/context_budget.py`, PR #34/H5).
+- **Empty-stream termination violated the SSE protocol** by sending a
+  terminator with no preceding `message_start`; one is now synthesized
+  first (`coderouter/translation/convert.py`, PR #34/H6).
+- **Excluded providers never recovered after a restart** — restored
+  self-healing state didn't re-arm recovery probes (only the live
+  UNHEALTHY-transition path did); probes are now re-spawned after state
+  restore, queued if the event loop isn't running yet
+  (`coderouter/routing/fallback.py`, `coderouter/guards/self_healing.py`, PR #34/H7).
+- **Malformed upstream responses bypassed retry/fallback** — an uncaught
+  pydantic `ValidationError` on missing/misshaped fields now converts to a
+  retryable `AdapterError` on non-stream paths, and stream paths skip the
+  malformed chunk with a once-per-stream warning
+  (`coderouter/adapters/anthropic_native.py`, `coderouter/adapters/openai_compat.py`, PR #35/M6).
+- **`"oom"` substring matched inside `"room"` / `"zoom"` / `"bloom"`** in
+  upstream error bodies, cooling down healthy providers; the memory-pressure
+  guard now matches the token on word boundaries
+  (`coderouter/guards/memory_pressure.py`, PR #35/M10).
+- **Interleaved text/tool-call deltas sent `input_json_delta` to
+  already-closed block indices** — blocks are now reopened at a fresh index
+  with their original `id`/`name` preserved
+  (`coderouter/translation/convert.py`, PR #35/M7).
+- **`tool_result.is_error` was dropped in both translation directions** — an
+  `"Error: "` content marker now round-trips the flag without
+  double-prefixing (`coderouter/translation/convert.py`, PR #35/M8).
+- **Streams ending without `message_stop` sent no terminal chunk**, hanging
+  OpenAI-compatible clients; a `finish_reason` + usage chunk is now
+  synthesized on abnormal termination (mirrors the H6 fix in the other
+  direction) (`coderouter/translation/convert.py`, PR #35/M9).
+- **`configure_logging` permanently detached the metrics collector** on a
+  second `create_app()` call by removing every root handler;
+  `install_collector` now re-attaches when missing, and
+  `configure_logging` removes only its own marked handlers
+  (`coderouter/logging.py`, PR #35/M4).
+- **Metrics persistence save/load asymmetry** zeroed language-tax
+  accounting on every restart — `save_state` was missing
+  `language_tax_usd` / `language_tax_usd_aggregate`, which `load_state`
+  expects (`coderouter/metrics/collector.py`, PR #35/M5).
+
+### Changed
+
+- **Config validation now fails fast at load time.** Previously-silent
+  misconfiguration is now rejected: profile chains referencing unknown
+  provider names, duplicate provider/profile names, inverted thresholds
+  (e.g. `context_budget` warn > trim, `trim_target` ≥ `trim_threshold`,
+  `recovery_probe_initial_s` > `recovery_probe_max_s`), and
+  `has_tools: false` / `has_image: false` matchers (which could never
+  match) (`coderouter/config/schemas.py`, PR #35/M13).
+- **Drift-based chain demotion now applies regardless of the `adaptive`
+  flag.** A dedicated cooldown-based demotion map is applied in
+  `_resolve_anthropic_chain` unconditionally; previously demotion only
+  affected ordering when `profile.adaptive` was true (and did nothing
+  under five samples) (`coderouter/routing/fallback.py`, PR #35/M3).
+- **Adaptive routing now observes streaming paths.** `record_attempt` was
+  only called from `generate_anthropic`; all four entry points
+  (`stream_anthropic` / `stream` / `generate` and non-stream) now record
+  first-event latency and outcomes, including Claude Code's default
+  streaming path (`coderouter/routing/fallback.py`, PR #35/M2).
+- **Drift verdicts are now request-scoped (`ContextVar`)** instead of
+  stored on the shared engine, eliminating a race where concurrent
+  requests could read another request's drift header or a stale verdict
+  during cooldown; the engine attribute remains as a deprecated mirror
+  (`coderouter/routing/fallback.py`, PR #35/M1).
+- **Chain resolution and context estimation no longer run twice per
+  request** — the resolved dispatch is cached per request and reused when
+  the request object is unchanged (`coderouter/routing/fallback.py`, PR #35/M11).
+- **Request/audit logs are now buffered** (20 records or 2 seconds,
+  flushed on close/`atexit`; `flush_every_n=1` restores write-through)
+  instead of synchronous per-request writes
+  (`coderouter/state/audit_log.py`, `coderouter/state/request_log.py`, PR #35/M12).
+- **The metrics collector hot path exits early** for unrecognized events
+  before acquiring the lock, and known events are dispatched via a dict
+  table instead of a ~30-branch `if`/`elif` chain
+  (`coderouter/metrics/collector.py`, PR #35/M12).
+- **Adapters now share a single `httpx.AsyncClient`** (lazily created,
+  per-call timeout override, closed on shutdown) instead of creating one
+  per request, enabling keep-alive/TLS session reuse
+  (`coderouter/adapters/base.py`, `coderouter/adapters/openai_compat.py`,
+  `coderouter/adapters/anthropic_native.py`, `coderouter/ingress/app.py`, PR #34/H3).
+
+### Security
+
+- **Host-validation middleware** on all routes rejects non-loopback `Host`
+  headers as a DNS-rebinding defense; loopback
+  (`localhost` / `127.0.0.1` / `[::1]`) is allowed by default and
+  `CODEROUTER_ALLOWED_HOSTS` (comma-separated) extends the allowlist
+  (`coderouter/ingress/app.py`, PR #34/H8).
+- **Opt-in launcher token authentication.** `launcher start/stop/delete`
+  require the `X-CodeRouter-Token` header (constant-time compare) when
+  `CODEROUTER_LAUNCHER_TOKEN` is set; unset behaves exactly as before, with
+  a one-time startup warning (`coderouter/ingress/launcher_routes.py`, PR #34/H8).
+- **`-m`/`--model` re-specification rejected** in launcher `extra_args`/
+  `options` to block arbitrary model-path injection
+  (`coderouter/ingress/launcher_routes.py`, PR #34/H8).
+- **Request body size cap** (64 MB default, `CODEROUTER_MAX_BODY_BYTES`
+  override) returns 413; SSE responses are unaffected
+  (`coderouter/ingress/app.py`, PR #35/M14).
+- **`/api/launcher/suggest` path confinement** — resolved paths must now
+  fall inside the configured `model_dirs`, closing an arbitrary-path
+  existence/size probe (`coderouter/ingress/launcher_routes.py`, PR #35/M14).
+
+### Tests
+
+- Full suite grew from **1263 → 1401 passed** (1 skipped, environment-only)
+  across the two PRs: 139 new regression tests in 12 new test files
+  (`tests/test_fix_h*.py` ×6, `tests/test_fix_m*.py` ×6). One existing test,
+  `tests/test_auto_router.py::test_has_tools_false_rejected_at_load`, was
+  flipped to expect `ValidationError` — its own docstring had anticipated
+  this once the new config validator (M13) landed; it is the only existing
+  test changed by either PR.
+
+---
+
+## [v2.6.1] — 2026-06-28 (Token-savings accounting)
+
+Patch release: surfaces **token-savings accounting** in the metrics layer
+and dashboard. The figure is owned by core, so it appears even when no
+plugin is installed (trim savings from the context-budget guard), and the
+optional `compress` plugin adds to the same total via a neutral
+`tokens-saved` log event. **No new core dependency**, **no behavioral
+change** to existing paths, and **fully backward compatible** — existing
+counters and events are untouched; the new buckets are additive.
+
+### Added
+
+- **Token-savings buckets in `MetricsCollector`**
+  (`coderouter/metrics/collector.py`). `tokens_saved_total` plus a
+  per-mechanism breakdown (`tokens_saved_by_mechanism`). Two feeds
+  aggregate under one schema: `trim` (derived from the existing
+  `context-budget-trimmed` event's before/after token estimate) and
+  `compress` (the neutral `tokens-saved` event emitted by the compress
+  plugin — no core import). Wired through `snapshot()`, `save_state()`,
+  `load_state()`, and `reset()`.
+- **Dashboard Token Savings tiles**
+  (`coderouter/ingress/dashboard_routes.py`). Three tiles — trim /
+  compress / total — in the "Cost & Language Tax" panel, zero-filled so a
+  fresh or local-only deployment renders cleanly.
+
+### Docs / examples
+
+- Reorganized provider samples under `examples/` with a category index.
+- Added the language-tax guide (JA/EN) and a Claude Code +
+  llama.cpp/vllm backend guide.
+
+### Tests
+
+- `tests/test_tokens_saved_metric.py` — trim/compress accounting,
+  negative-delta clamping, combined aggregate, persistence round-trip,
+  and reset. Full suite: **1263 passed, 1 skipped**.
+
+### Companion
+
+- `coderouter-plugin-compress` emits the `tokens-saved` event (plugin
+  branch `feat/tokens-saved-emit`, v0.2.0). CodeRouter core works
+  standalone — trim savings show without the plugin.
+
+---
+
 ## [v2.6.0] — 2026-06-20 (Language Tax: measure, route, visualize)
 
 Minor release: makes the CJK **"language tax"** — cloud tokenizers bill

{coderouter_cli-2.6.0 → coderouter_cli-2.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: coderouter-cli
-Version: 2.6.0
+Version: 2.7.0
 Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
 Project-URL: Homepage, https://github.com/zephel01/CodeRouter
 Project-URL: Repository, https://github.com/zephel01/CodeRouter
@@ -161,6 +161,36 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 | **`coderouter replay`** | provider 切替の効果を統計比較 (A/B 分析) / `--suggest-rules` でルール最適化提案 |
 | **Continuous Probe** | idle 時も定期的に backend を監視 |
 
+### 言語税トラッキング — v2.6.0
+
+日本語などの CJK テキストは、クラウドのトークナイザだと「同じ意味の英語」より多くのトークンを消費します（**実測: GPT-4o 系 o200k で平均 1.6 倍、GPT-4 系 cl100k で平均 2.0 倍**）。ローカル LLM は課金されないので、この「言語税」はクラウド利用時だけ効いてきます。CodeRouter v2.6.0 はこれを **計測・ルーティング回避・可視化** します。
+
+| 機能 | 何をしてくれるか |
+|---|---|
+| **言語税の計測** | プロバイダに `tokenizer_path`（ローカルの `tokenizer.json`）を指定すると、char/4 ヒューリスティック比の実トークン倍率と割増 USD を算出（ネットワーク不要・未設定なら無効） |
+| **`cjk_ratio_min` ルーティング** | CJK 比率が高いリクエストを自動でローカル LLM（課金ゼロ）へ。コードや英語はクラウドへ |
+| **ダッシュボード可視化** | `/dashboard` の「Cost & Language Tax」パネルで総支出・キャッシュ節約・言語税をリアルタイム表示 |
+
+```yaml
+# providers.yaml — CJK 多めのターンはローカルへ自動回避
+auto_router:
+  rules:
+    - match: { cjk_ratio_min: 0.3 }   # 日本語が3割以上 → ローカル
+      profile: local
+    - match: { has_tools: true }      # ツール使用 → クラウド
+      profile: cloud
+  default_rule_profile: cloud
+
+providers:
+  - name: cloud-sonnet
+    kind: anthropic
+    base_url: https://api.anthropic.com
+    model: claude-sonnet-4-6
+    tokenizer_path: ~/.coderouter/tokenizers/sonnet.json   # 言語税の正確計測（任意）
+```
+
+詳細 → [言語税ガイド](./docs/guides/language-tax.md)
+
 ### Launcher — llama.cpp / vllm 起動 UI
 
 `http://localhost:8088/launcher` で開けるブラウザ UI。llama.cpp や vllm を GUI で起動・管理できます。
@@ -229,6 +259,7 @@ providers:
 | 使いこなす | [利用ガイド](./docs/guides/usage-guide.md) |
 | 無料で回す | [無料枠ガイド](./docs/guides/free-tier-guide.md) |
 | llama.cpp / vllm を GUI で起動 | [Launcher ガイド](./docs/backends/launcher.md) |
+| 言語税を計測・回避する | [言語税ガイド](./docs/guides/language-tax.md) |
 | 詰まった | [トラブルシューティング](./docs/guides/troubleshooting.md) |
 | 設計を知りたい | [アーキテクチャ詳細](./docs/concepts/architecture.md) |
 | 全リリース履歴 | [CHANGELOG](./CHANGELOG.md) |

{coderouter_cli-2.6.0 → coderouter_cli-2.7.0}/README.en.md

@@ -116,6 +116,36 @@ Full decision matrix → [Do I need CodeRouter?](./docs/start/when-do-i-need-cod
 | **`coderouter replay`** | Compare providers statistically (A/B analysis) / `--suggest-rules` for automated rule suggestions |
 | **Continuous Probe** | Background health monitoring even during idle |
 
+### Language Tax tracking — v2.6.0
+
+CJK text (Japanese / Chinese / Korean) costs more tokens on cloud tokenizers than the same meaning in English (**measured: ~1.6× on average with GPT-4o-era o200k, ~2.0× with GPT-4-era cl100k**). Local models bill nothing per token, so this "language tax" only bites on the cloud leg. CodeRouter v2.6.0 makes it **measurable, routable, and visible**.
+
+| Feature | What it does |
+|---|---|
+| **Language-tax measurement** | Set a provider's `tokenizer_path` (a local `tokenizer.json`) to compute the real token multiplier vs the char/4 heuristic and the extra USD. No network; inert when unset. |
+| **`cjk_ratio_min` routing** | Auto-route CJK-heavy turns to a local (tax-free) model; code/English falls through to the cloud chain. |
+| **Dashboard panel** | The `/dashboard` "Cost & Language Tax" panel shows total spend, cache savings, and language-tax spend live. |
+
+```yaml
+# providers.yaml — steer CJK-heavy turns to local
+auto_router:
+  rules:
+    - match: { cjk_ratio_min: 0.3 }   # >=30% CJK chars -> local
+      profile: local
+    - match: { has_tools: true }      # tool use -> cloud
+      profile: cloud
+  default_rule_profile: cloud
+
+providers:
+  - name: cloud-sonnet
+    kind: anthropic
+    base_url: https://api.anthropic.com
+    model: claude-sonnet-4-6
+    tokenizer_path: ~/.coderouter/tokenizers/sonnet.json   # accurate language-tax (optional)
+```
+
+Details → [Language Tax guide](./docs/guides/language-tax.en.md)
+
 ### Launcher — llama.cpp / vllm GUI
 
 Browser UI at `http://localhost:8088/launcher` for starting and managing local inference backends.
@@ -184,6 +214,7 @@ More detail → [Usage guide](./docs/guides/usage-guide.en.md) · [Architecture]
 | Use it well | [Usage guide](./docs/guides/usage-guide.en.md) |
 | Run for free | [Free-tier guide](./docs/guides/free-tier-guide.en.md) |
 | Launch llama.cpp / vllm via GUI | [Launcher guide](./docs/backends/launcher.md) |
+| Measure & avoid the language tax | [Language Tax guide](./docs/guides/language-tax.en.md) |
 | Stuck? | [Troubleshooting](./docs/guides/troubleshooting.en.md) |
 | Understand the design | [Architecture](./docs/concepts/architecture.md) |
 | Full release history | [CHANGELOG](./CHANGELOG.md) |

{coderouter_cli-2.6.0 → coderouter_cli-2.7.0}/README.md

@@ -116,6 +116,36 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 | **`coderouter replay`** | provider 切替の効果を統計比較 (A/B 分析) / `--suggest-rules` でルール最適化提案 |
 | **Continuous Probe** | idle 時も定期的に backend を監視 |
 
+### 言語税トラッキング — v2.6.0
+
+日本語などの CJK テキストは、クラウドのトークナイザだと「同じ意味の英語」より多くのトークンを消費します（**実測: GPT-4o 系 o200k で平均 1.6 倍、GPT-4 系 cl100k で平均 2.0 倍**）。ローカル LLM は課金されないので、この「言語税」はクラウド利用時だけ効いてきます。CodeRouter v2.6.0 はこれを **計測・ルーティング回避・可視化** します。
+
+| 機能 | 何をしてくれるか |
+|---|---|
+| **言語税の計測** | プロバイダに `tokenizer_path`（ローカルの `tokenizer.json`）を指定すると、char/4 ヒューリスティック比の実トークン倍率と割増 USD を算出（ネットワーク不要・未設定なら無効） |
+| **`cjk_ratio_min` ルーティング** | CJK 比率が高いリクエストを自動でローカル LLM（課金ゼロ）へ。コードや英語はクラウドへ |
+| **ダッシュボード可視化** | `/dashboard` の「Cost & Language Tax」パネルで総支出・キャッシュ節約・言語税をリアルタイム表示 |
+
+```yaml
+# providers.yaml — CJK 多めのターンはローカルへ自動回避
+auto_router:
+  rules:
+    - match: { cjk_ratio_min: 0.3 }   # 日本語が3割以上 → ローカル
+      profile: local
+    - match: { has_tools: true }      # ツール使用 → クラウド
+      profile: cloud
+  default_rule_profile: cloud
+
+providers:
+  - name: cloud-sonnet
+    kind: anthropic
+    base_url: https://api.anthropic.com
+    model: claude-sonnet-4-6
+    tokenizer_path: ~/.coderouter/tokenizers/sonnet.json   # 言語税の正確計測（任意）
+```
+
+詳細 → [言語税ガイド](./docs/guides/language-tax.md)
+
 ### Launcher — llama.cpp / vllm 起動 UI
 
 `http://localhost:8088/launcher` で開けるブラウザ UI。llama.cpp や vllm を GUI で起動・管理できます。
@@ -184,6 +214,7 @@ providers:
 | 使いこなす | [利用ガイド](./docs/guides/usage-guide.md) |
 | 無料で回す | [無料枠ガイド](./docs/guides/free-tier-guide.md) |
 | llama.cpp / vllm を GUI で起動 | [Launcher ガイド](./docs/backends/launcher.md) |
+| 言語税を計測・回避する | [言語税ガイド](./docs/guides/language-tax.md) |
 | 詰まった | [トラブルシューティング](./docs/guides/troubleshooting.md) |
 | 設計を知りたい | [アーキテクチャ詳細](./docs/concepts/architecture.md) |
 | 全リリース履歴 | [CHANGELOG](./CHANGELOG.md) |

{coderouter_cli-2.6.0 → coderouter_cli-2.7.0}/coderouter/adapters/anthropic_native.py

@@ -35,6 +35,7 @@ from collections.abc import AsyncIterator
 from typing import Any
 
 import httpx
+from pydantic import ValidationError
 
 from coderouter.adapters.base import (
     AdapterError,
@@ -68,6 +69,26 @@ _RETRYABLE_STATUSES = {404, 408, 425, 429, 500, 502, 503, 504}
 _DEFAULT_ANTHROPIC_VERSION = "2023-06-01"
 
 
+def anthropic_messages_url(base_url: str) -> str:
+    """Resolve the ``/v1/messages`` endpoint URL from a ``base_url``.
+
+    ``base_url`` may be given with or without a trailing ``/v1`` — users
+    may point it at either ``https://api.anthropic.com`` or
+    ``https://api.anthropic.com/v1`` (LM Studio and similar servers
+    commonly advertise the latter). We normalize by stripping a trailing
+    ``/v1`` so appending ``/v1/messages`` always yields a valid,
+    non-duplicated URL.
+
+    Shared by :class:`AnthropicAdapter` and the continuous-probe guard so
+    both hit the exact same endpoint (see bug H4: a mismatch made the
+    probe target ``/v1/v1/messages`` and 404 on healthy backends).
+    """
+    base = base_url.rstrip("/")
+    if base.endswith("/v1"):
+        base = base[: -len("/v1")]
+    return f"{base}/v1/messages"
+
+
 class AnthropicAdapter(BaseAdapter):
     """Native Anthropic Messages API adapter (passthrough).
 
@@ -87,15 +108,10 @@ class AnthropicAdapter(BaseAdapter):
 
         ``base_url`` may be given with or without a trailing ``/v1`` —
         we normalize by stripping it so appending ``/v1/messages``
-        always yields a valid URL.
+        always yields a valid URL. Delegates to the shared
+        :func:`anthropic_messages_url` helper.
         """
-        base = str(self.config.base_url).rstrip("/")
-        # Users may point base_url at either `https://api.anthropic.com`
-        # or `https://api.anthropic.com/v1`. We normalize to the former so
-        # we can always append /v1/messages.
-        if base.endswith("/v1"):
-            base = base[: -len("/v1")]
-        return f"{base}/v1/messages"
+        return anthropic_messages_url(str(self.config.base_url))
 
     def _headers(self, request: AnthropicRequest | None = None) -> dict[str, str]:
         """Build Anthropic-native HTTP headers, including beta-header forwarding.
@@ -161,21 +177,21 @@ class AnthropicAdapter(BaseAdapter):
         that auth works and the endpoint is reachable.
         """
         try:
-            async with httpx.AsyncClient(timeout=5.0) as client:
-                resp = await client.post(
-                    self._url(),
-                    headers=self._headers(),
-                    json={
-                        "model": self.config.model,
-                        "max_tokens": 1,
-                        "messages": [{"role": "user", "content": "ping"}],
-                    },
-                )
-                # 200 is clearly healthy. 4xx auth errors indicate the
-                # endpoint is reachable even if the key is bad — still a
-                # "the server answered" signal for healthcheck purposes.
-                # 5xx is upstream trouble.
-                return resp.status_code < 500
+            resp = await self.client().post(
+                self._url(),
+                headers=self._headers(),
+                json={
+                    "model": self.config.model,
+                    "max_tokens": 1,
+                    "messages": [{"role": "user", "content": "ping"}],
+                },
+                timeout=5.0,
+            )
+            # 200 is clearly healthy. 4xx auth errors indicate the
+            # endpoint is reachable even if the key is bad — still a
+            # "the server answered" signal for healthcheck purposes.
+            # 5xx is upstream trouble.
+            return resp.status_code < 500
         except httpx.HTTPError:
             return False
 
@@ -331,8 +347,9 @@ class AnthropicAdapter(BaseAdapter):
         timeout = self.effective_timeout(overrides)
 
         try:
-            async with httpx.AsyncClient(timeout=timeout) as client:
-                resp = await client.post(url, json=payload, headers=self._headers(request))
+            resp = await self.client().post(
+                url, json=payload, headers=self._headers(request), timeout=timeout
+            )
         except httpx.TimeoutException as exc:
             raise AdapterError(
                 f"timeout contacting {url}", provider=self.name, retryable=True
@@ -394,7 +411,22 @@ class AnthropicAdapter(BaseAdapter):
         # (future additions like thinking blocks) pass through via
         # extra="allow" on AnthropicResponse.
         data["coderouter_provider"] = self.name
-        return AnthropicResponse.model_validate(data)
+        # M6: guard the model validation. A 200 body that isn't a valid
+        # Anthropic Messages response (missing ``content``/``id``, an
+        # error envelope returned with a 200, etc.) would otherwise raise
+        # a bare pydantic ValidationError that escapes the engine's
+        # AdapterError-based retry/fallback path. Convert to a retryable
+        # AdapterError so the chain can fall through (mirrors the
+        # invalid-JSON branch above, but retryable — a malformed shape is
+        # usually transient upstream noise rather than a config fault).
+        try:
+            return AnthropicResponse.model_validate(data)
+        except ValidationError as exc:
+            raise AdapterError(
+                f"malformed response shape from upstream: {exc}",
+                provider=self.name,
+                retryable=True,
+            ) from exc
 
     async def stream_anthropic(
         self,
@@ -424,10 +456,14 @@ class AnthropicAdapter(BaseAdapter):
         logged_flag: list[bool] = [False]
 
         try:
-            async with (
-                httpx.AsyncClient(timeout=timeout) as client,
-                client.stream("POST", url, json=payload, headers=self._headers(request)) as resp,
-            ):
+            # H3: stream over the shared client (pool / keep-alive / TLS
+            # reuse). Only the ``stream(...)`` context is scoped here; the
+            # client persists and is closed via ``aclose`` on shutdown. The
+            # ``async with`` releases the response's borrowed connection even
+            # if the caller abandons this generator mid-stream — no GC leak.
+            async with self.client().stream(
+                "POST", url, json=payload, headers=self._headers(request), timeout=timeout
+            ) as resp:
                 if resp.status_code >= 400:
                     body = await resp.aread()
                     raise AdapterError(

{coderouter_cli-2.6.0 → coderouter_cli-2.7.0}/coderouter/adapters/base.py

@@ -11,6 +11,7 @@ from abc import ABC, abstractmethod
 from collections.abc import AsyncIterator
 from typing import Any, Literal
 
+import httpx
 from pydantic import BaseModel, ConfigDict, Field
 
 from coderouter.config.schemas import ProviderConfig
@@ -164,17 +165,59 @@ class BaseAdapter(ABC):
     def __init__(self, config: ProviderConfig) -> None:
         """Bind the adapter to a :class:`ProviderConfig`.
 
-        Subclasses do not need to override this; HTTP clients are
-        constructed lazily inside :meth:`generate` / :meth:`stream` so
-        each call can honor a per-call timeout override.
+        Subclasses do not need to override this. A single shared
+        :class:`httpx.AsyncClient` is created lazily on first use (see
+        :meth:`client`) and reused across every call on this adapter so
+        the connection pool, HTTP keep-alive, and TLS session are all
+        reused. Per-call timeouts are still honored by passing
+        ``timeout=`` to ``client.post`` / ``client.stream`` — they do
+        NOT require a fresh client.
         """
         self.config = config
+        # Shared client, created lazily inside the running event loop the
+        # first time an adapter method needs it (see ``client``). Kept as
+        # ``None`` until then so constructing an adapter has no I/O cost and
+        # is safe outside an event loop (e.g. at import / config time).
+        self._client: httpx.AsyncClient | None = None
 
     @property
     def name(self) -> str:
         """Shortcut for ``self.config.name`` — used in log trails and errors."""
         return self.config.name
 
+    # ---- H3: shared HTTP client (connection-pool / keep-alive reuse) ----
+    def client(self) -> httpx.AsyncClient:
+        """Return the shared :class:`httpx.AsyncClient`, creating it lazily.
+
+        The client is created on first use so construction happens inside
+        the running event loop and carries no I/O cost at adapter-build
+        time. Subsequent calls return the same instance, which is what
+        lets httpx reuse pooled connections, keep-alive, and the TLS
+        session across requests.
+
+        No default timeout is baked in here: every call site passes an
+        explicit per-call ``timeout=`` (resolved from the active profile
+        via :meth:`effective_timeout`), so leaving the client timeout
+        unset avoids a surprising default clamping long-running calls.
+        """
+        if self._client is None:
+            self._client = httpx.AsyncClient(timeout=None)
+        return self._client
+
+    async def aclose(self) -> None:
+        """Close the shared HTTP client and drop the reference.
+
+        Idempotent: safe to call when no client was ever created, and
+        safe to call more than once. Invoked from the app lifespan
+        shutdown path so pooled connections are released cleanly rather
+        than left to garbage collection. After ``aclose`` a later call
+        re-creates the client on demand via :meth:`client`.
+        """
+        client = self._client
+        self._client = None
+        if client is not None:
+            await client.aclose()
+
     # ---- v0.6-B override resolution helpers -----------------------------
     def effective_timeout(self, overrides: ProviderCallOverrides | None) -> float:
         """Profile override wins when set; else provider default."""