coderouter-cli 2.6.1__tar.gz → 2.7.1__tar.gz

{coderouter_cli-2.6.1 → coderouter_cli-2.7.1}/CHANGELOG.md

@@ -6,6 +6,194 @@ versioning follows [SemVer](https://semver.org/).
 
 ---
 
+## [v2.7.1] — 2026-07-04 (Native-endpoint shims + benchmarked tool-repair upgrade)
+
+Two feature PRs driven by the 2026-07-04 competitive re-survey and the new
+tool-call repair benchmark: **PR #39** (native `/v1/messages` endpoint gap
+shims) and **PR #40** (lenient tool-call repair). All new behavior is
+**opt-in with `off` defaults** (except the previously-404 `count_tokens`
+endpoint, which is additive), no config-schema breaking changes, and the
+Core runtime stays at **5 dependencies**.
+
+### Added
+
+- **`POST /v1/messages/count_tokens`** (was 404). Anthropic-shape
+  `{"input_tokens": N}`; accurate when the provider declares
+  `tokenizer_path`, char/4 heuristic otherwise. Fills a gap native backends
+  leave open (Ollama's Anthropic-compatible API ships without
+  `count_tokens` / `tool_choice` / prompt caching). (PR #39)
+- **`tool_choice_action: off | warn | emulate`** (profile option) +
+  `Capabilities.tool_choice` + registry field + `provider_supports_tool_choice()`.
+  `emulate` rewrites forced tool_choice (`any` / `tool`) into a system-prompt
+  instruction for non-supporting backends; the original request is preserved
+  for fallback to capable providers. New capability-degraded reason:
+  `unsupported-backend`. (PR #39)
+- **`cache_control_action: off | strip`** (profile option). `strip` removes
+  `cache_control` markers before sending to non-supporting providers
+  (logged as `cache-control-stripped`; deliberately no tokens-saved
+  accounting). (PR #39)
+- **Lenient tool-call repair** (`translation/tool_repair.py`): second-pass
+  parsing for malformed JSON (double braces, trailing commas, single quotes,
+  unquoted keys), key aliases (`tool`/`tool_name`, `parameters`/`input`/`args`),
+  and XML-flavoured forms — all gated on `allowed_tool_names`. Residue
+  cleanup (empty fences, `[,]`). Benchmark: recall 80.6% → **100%** on a
+  55-case corpus; live fail-rate at default temperature 17% → **1%**
+  (qwen2.5-coder:7b × 100 requests). (PR #40)
+
+### Fixed
+
+- **Two real false positives in the previous repairer** (python-fence dict
+  literals and documentation JSON with allowed tool names were converted
+  into executable calls — same class as the v2.7.0 code-eating regression).
+  Now fixed and gated by an expanded 12-case negative corpus. (PR #40)
+
+### Docs
+
+- README repositioned for the native `/v1/messages` era: repair + guards
+  lead, wire translation demoted to last (passthrough note); new section
+  "why a router when Ollama connects directly?".
+
+Tests: **1481 passed, 0 failed** (1401 → 1481; +37 shim, +43 lenient-repair).
+
+---
+
+## [v2.7.0] — 2026-07-02 (Reliability & security: full-source review fixes)
+
+22 reliability and security fixes from a full-source review (26,600 lines),
+landed as two sequential PRs: high-priority **PR #34** (H1–H8) and
+medium-priority **PR #35** (M1–M14), both merged to `main`. No API or
+config-schema breaking changes. Backward compatibility highlights: the new
+launcher token auth is **opt-in** (`CODEROUTER_LAUNCHER_TOKEN` unset ⇒
+unchanged behavior + a startup warning), and the Host-validation middleware
+allows loopback by default with `CODEROUTER_ALLOWED_HOSTS` to extend it.
+
+### Fixed
+
+- **`/metrics` 500 when drift counters are non-zero** — three malformed
+  Prometheus label tuples (`(((),), v)` → `((), v)`) (`coderouter/metrics/prometheus.py`, PR #34/H1).
+- **`tool_repair` deleted unrelated fenced code blocks** while stripping
+  tool-call-shaped ones, losing user content
+  (`coderouter/translation/tool_repair.py`, PR #34/H2).
+- **Probe URL mismatch (`/v1/v1` duplication)** between the continuous probe
+  and the adapters caused healthy backends to be wrongly demoted; URL
+  normalization is now shared
+  (`coderouter/adapters/anthropic_native.py`, `coderouter/guards/continuous_probe.py`, PR #34/H4).
+- **`context_budget` dropped entire history at once** on threshold breach;
+  trimming is now sequential per tool-pair with re-estimation after each
+  step, and the new head turn is renormalized to a plain user turn
+  (`coderouter/guards/context_budget.py`, PR #34/H5).
+- **Empty-stream termination violated the SSE protocol** by sending a
+  terminator with no preceding `message_start`; one is now synthesized
+  first (`coderouter/translation/convert.py`, PR #34/H6).
+- **Excluded providers never recovered after a restart** — restored
+  self-healing state didn't re-arm recovery probes (only the live
+  UNHEALTHY-transition path did); probes are now re-spawned after state
+  restore, queued if the event loop isn't running yet
+  (`coderouter/routing/fallback.py`, `coderouter/guards/self_healing.py`, PR #34/H7).
+- **Malformed upstream responses bypassed retry/fallback** — an uncaught
+  pydantic `ValidationError` on missing/misshaped fields now converts to a
+  retryable `AdapterError` on non-stream paths, and stream paths skip the
+  malformed chunk with a once-per-stream warning
+  (`coderouter/adapters/anthropic_native.py`, `coderouter/adapters/openai_compat.py`, PR #35/M6).
+- **`"oom"` substring matched inside `"room"` / `"zoom"` / `"bloom"`** in
+  upstream error bodies, cooling down healthy providers; the memory-pressure
+  guard now matches the token on word boundaries
+  (`coderouter/guards/memory_pressure.py`, PR #35/M10).
+- **Interleaved text/tool-call deltas sent `input_json_delta` to
+  already-closed block indices** — blocks are now reopened at a fresh index
+  with their original `id`/`name` preserved
+  (`coderouter/translation/convert.py`, PR #35/M7).
+- **`tool_result.is_error` was dropped in both translation directions** — an
+  `"Error: "` content marker now round-trips the flag without
+  double-prefixing (`coderouter/translation/convert.py`, PR #35/M8).
+- **Streams ending without `message_stop` sent no terminal chunk**, hanging
+  OpenAI-compatible clients; a `finish_reason` + usage chunk is now
+  synthesized on abnormal termination (mirrors the H6 fix in the other
+  direction) (`coderouter/translation/convert.py`, PR #35/M9).
+- **`configure_logging` permanently detached the metrics collector** on a
+  second `create_app()` call by removing every root handler;
+  `install_collector` now re-attaches when missing, and
+  `configure_logging` removes only its own marked handlers
+  (`coderouter/logging.py`, PR #35/M4).
+- **Metrics persistence save/load asymmetry** zeroed language-tax
+  accounting on every restart — `save_state` was missing
+  `language_tax_usd` / `language_tax_usd_aggregate`, which `load_state`
+  expects (`coderouter/metrics/collector.py`, PR #35/M5).
+
+### Changed
+
+- **Config validation now fails fast at load time.** Previously-silent
+  misconfiguration is now rejected: profile chains referencing unknown
+  provider names, duplicate provider/profile names, inverted thresholds
+  (e.g. `context_budget` warn > trim, `trim_target` ≥ `trim_threshold`,
+  `recovery_probe_initial_s` > `recovery_probe_max_s`), and
+  `has_tools: false` / `has_image: false` matchers (which could never
+  match) (`coderouter/config/schemas.py`, PR #35/M13).
+- **Drift-based chain demotion now applies regardless of the `adaptive`
+  flag.** A dedicated cooldown-based demotion map is applied in
+  `_resolve_anthropic_chain` unconditionally; previously demotion only
+  affected ordering when `profile.adaptive` was true (and did nothing
+  under five samples) (`coderouter/routing/fallback.py`, PR #35/M3).
+- **Adaptive routing now observes streaming paths.** `record_attempt` was
+  only called from `generate_anthropic`; all four entry points
+  (`stream_anthropic` / `stream` / `generate` and non-stream) now record
+  first-event latency and outcomes, including Claude Code's default
+  streaming path (`coderouter/routing/fallback.py`, PR #35/M2).
+- **Drift verdicts are now request-scoped (`ContextVar`)** instead of
+  stored on the shared engine, eliminating a race where concurrent
+  requests could read another request's drift header or a stale verdict
+  during cooldown; the engine attribute remains as a deprecated mirror
+  (`coderouter/routing/fallback.py`, PR #35/M1).
+- **Chain resolution and context estimation no longer run twice per
+  request** — the resolved dispatch is cached per request and reused when
+  the request object is unchanged (`coderouter/routing/fallback.py`, PR #35/M11).
+- **Request/audit logs are now buffered** (20 records or 2 seconds,
+  flushed on close/`atexit`; `flush_every_n=1` restores write-through)
+  instead of synchronous per-request writes
+  (`coderouter/state/audit_log.py`, `coderouter/state/request_log.py`, PR #35/M12).
+- **The metrics collector hot path exits early** for unrecognized events
+  before acquiring the lock, and known events are dispatched via a dict
+  table instead of a ~30-branch `if`/`elif` chain
+  (`coderouter/metrics/collector.py`, PR #35/M12).
+- **Adapters now share a single `httpx.AsyncClient`** (lazily created,
+  per-call timeout override, closed on shutdown) instead of creating one
+  per request, enabling keep-alive/TLS session reuse
+  (`coderouter/adapters/base.py`, `coderouter/adapters/openai_compat.py`,
+  `coderouter/adapters/anthropic_native.py`, `coderouter/ingress/app.py`, PR #34/H3).
+
+### Security
+
+- **Host-validation middleware** on all routes rejects non-loopback `Host`
+  headers as a DNS-rebinding defense; loopback
+  (`localhost` / `127.0.0.1` / `[::1]`) is allowed by default and
+  `CODEROUTER_ALLOWED_HOSTS` (comma-separated) extends the allowlist
+  (`coderouter/ingress/app.py`, PR #34/H8).
+- **Opt-in launcher token authentication.** `launcher start/stop/delete`
+  require the `X-CodeRouter-Token` header (constant-time compare) when
+  `CODEROUTER_LAUNCHER_TOKEN` is set; unset behaves exactly as before, with
+  a one-time startup warning (`coderouter/ingress/launcher_routes.py`, PR #34/H8).
+- **`-m`/`--model` re-specification rejected** in launcher `extra_args`/
+  `options` to block arbitrary model-path injection
+  (`coderouter/ingress/launcher_routes.py`, PR #34/H8).
+- **Request body size cap** (64 MB default, `CODEROUTER_MAX_BODY_BYTES`
+  override) returns 413; SSE responses are unaffected
+  (`coderouter/ingress/app.py`, PR #35/M14).
+- **`/api/launcher/suggest` path confinement** — resolved paths must now
+  fall inside the configured `model_dirs`, closing an arbitrary-path
+  existence/size probe (`coderouter/ingress/launcher_routes.py`, PR #35/M14).
+
+### Tests
+
+- Full suite grew from **1263 → 1401 passed** (1 skipped, environment-only)
+  across the two PRs: 139 new regression tests in 12 new test files
+  (`tests/test_fix_h*.py` ×6, `tests/test_fix_m*.py` ×6). One existing test,
+  `tests/test_auto_router.py::test_has_tools_false_rejected_at_load`, was
+  flipped to expect `ValidationError` — its own docstring had anticipated
+  this once the new config validator (M13) landed; it is the only existing
+  test changed by either PR.
+
+---
+
 ## [v2.6.1] — 2026-06-28 (Token-savings accounting)
 
 Patch release: surfaces **token-savings accounting** in the metrics layer

{coderouter_cli-2.6.1 → coderouter_cli-2.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: coderouter-cli
-Version: 2.6.1
+Version: 2.7.1
 Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
 Project-URL: Homepage, https://github.com/zephel01/CodeRouter
 Project-URL: Repository, https://github.com/zephel01/CodeRouter
@@ -70,8 +70,8 @@ Description-Content-Type: text/markdown
         │
         ▼
   ┌─ CodeRouter ─┐
-  │  翻訳 + 修復  │──→  ① ローカル (Ollama — 無料・最速)
-  │  ガード + 監視 │──→  ② 無料クラウド (OpenRouter / NIM)
+  │  修復 + ガード │──→  ① ローカル (Ollama — 無料・最速)
+  │  監視 + 診断  │──→  ② 無料クラウド (OpenRouter / NIM)
   │  自動フォールバック │──→  ③ 有料 (Claude — opt-in 時のみ)
   └──────────────┘
 ```
@@ -79,13 +79,31 @@ Description-Content-Type: text/markdown
 **やってくれること:**
 
 - ローカルモデルが壊した tool calling を Claude Code に届く前に修復する
+- 8 時間回しても止まらないように 6 種類のガードで守る
 - 1 つ目が落ちたら自動で次のプロバイダに切り替える
 - 有料 API は明示的に許可したときだけ使う (デフォルトは無料のみ)
-- 8 時間回しても止まらないように 6 種類のガードで守る
 - 何がおかしいか `coderouter doctor` コマンド一発で診断する
 
 ---
 
+## 「Ollama に直結できるのに、なぜルーター?」
+
+2026 年、Ollama (v0.14+) / LM Studio (0.4.1+) / llama.cpp / vLLM は Anthropic 互換 `/v1/messages` を標準装備しました。`ANTHROPIC_BASE_URL` を直接向ければ Claude Code は一応動きます。
+
+**でも直結では、こうなります:**
+
+| 直結の現実 | CodeRouter 経由 |
+|---|---|
+| 壊れた tool call は壊れたまま届く | 届く前に修復 |
+| backend が落ちたらセッション終了 | ローカル → 無料 → 有料へ自動フォールバック |
+| 長時間で context 溢れ・drift・ループ | 6 系統ガード + self-healing |
+| モデル名がハードコード (リタイアで即エラー) | プロファイルで抽象化、差し替え 1 行 |
+| 何が悪いか分からない | `doctor` 6 プローブ + `/dashboard` + audit/replay |
+
+直結で困っていないなら CodeRouter は不要です。**長時間・無人・弱いモデル**のどれかに当てはまったら、戻ってきてください。
+
+---
+
 ## インストール (3 行)
 
 ```bash
@@ -120,8 +138,9 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 
 | あなたの状況 | CodeRouter は？ |
 |---|---|
-| Claude Code + ローカル Ollama で tool calling が壊れる | **必須** — wire 変換 + tool 修復 |
-| Claude Code + ローカルで長時間回すと止まる | **便利** — 6 系統ガード + self-healing |
+| Claude Code + ローカル Ollama で tool calling が壊れる | **必須** — tool 修復 (+ 必要なら wire 変換) |
+| Claude Code + ローカルで長時間回すと止まる | **必須級** — 6 系統ガード + self-healing |
+| Ollama v0.14+ / LM Studio にネイティブ直結で動いてる | **便利** — 直結に無い fallback / ガード / 診断を追加 (passthrough で翻訳ゼロ) |
 | codex / gemini-cli + Ollama 直繋ぎで動いてる | オプション — フォールバックが欲しいなら |
 | Claude API を直接叩いてて問題ない | 不要 |
 
@@ -131,14 +150,14 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 
 ## 主な機能
 
-### 接続と修復
+### 修復と接続
 
 | 機能 | 何をしてくれるか |
 |---|---|
-| **Wire 翻訳** | Claude Code (Anthropic形式) ↔ Ollama (OpenAI形式) を自動変換 |
 | **Tool-call 修復** | ローカルモデルがテキストで吐いた JSON を正しい tool_use ブロックに復元 |
 | **3 層フォールバック** | ローカル → 無料クラウド → 有料の順に自動切替 |
-| **出力フィルタ** | `<think>` タグ漏れ、stop marker 漏れを自動除去 |
+| **出力フィルタ** | `<think>` タグ漏れ、stop marker 漏れ、byte-fallback (`<0xNN>`) を自動除去/修復 |
+| **Wire 翻訳** | Anthropic 形式 ↔ OpenAI 形式を自動変換 (ネイティブ `/v1/messages` 対応 backend は passthrough で翻訳ゼロ) |
 
 ### 長時間運用ガード
 

{coderouter_cli-2.6.1 → coderouter_cli-2.7.1}/README.md

@@ -25,8 +25,8 @@
         │
         ▼
   ┌─ CodeRouter ─┐
-  │  翻訳 + 修復  │──→  ① ローカル (Ollama — 無料・最速)
-  │  ガード + 監視 │──→  ② 無料クラウド (OpenRouter / NIM)
+  │  修復 + ガード │──→  ① ローカル (Ollama — 無料・最速)
+  │  監視 + 診断  │──→  ② 無料クラウド (OpenRouter / NIM)
   │  自動フォールバック │──→  ③ 有料 (Claude — opt-in 時のみ)
   └──────────────┘
 ```
@@ -34,13 +34,31 @@
 **やってくれること:**
 
 - ローカルモデルが壊した tool calling を Claude Code に届く前に修復する
+- 8 時間回しても止まらないように 6 種類のガードで守る
 - 1 つ目が落ちたら自動で次のプロバイダに切り替える
 - 有料 API は明示的に許可したときだけ使う (デフォルトは無料のみ)
-- 8 時間回しても止まらないように 6 種類のガードで守る
 - 何がおかしいか `coderouter doctor` コマンド一発で診断する
 
 ---
 
+## 「Ollama に直結できるのに、なぜルーター?」
+
+2026 年、Ollama (v0.14+) / LM Studio (0.4.1+) / llama.cpp / vLLM は Anthropic 互換 `/v1/messages` を標準装備しました。`ANTHROPIC_BASE_URL` を直接向ければ Claude Code は一応動きます。
+
+**でも直結では、こうなります:**
+
+| 直結の現実 | CodeRouter 経由 |
+|---|---|
+| 壊れた tool call は壊れたまま届く | 届く前に修復 |
+| backend が落ちたらセッション終了 | ローカル → 無料 → 有料へ自動フォールバック |
+| 長時間で context 溢れ・drift・ループ | 6 系統ガード + self-healing |
+| モデル名がハードコード (リタイアで即エラー) | プロファイルで抽象化、差し替え 1 行 |
+| 何が悪いか分からない | `doctor` 6 プローブ + `/dashboard` + audit/replay |
+
+直結で困っていないなら CodeRouter は不要です。**長時間・無人・弱いモデル**のどれかに当てはまったら、戻ってきてください。
+
+---
+
 ## インストール (3 行)
 
 ```bash
@@ -75,8 +93,9 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 
 | あなたの状況 | CodeRouter は？ |
 |---|---|
-| Claude Code + ローカル Ollama で tool calling が壊れる | **必須** — wire 変換 + tool 修復 |
-| Claude Code + ローカルで長時間回すと止まる | **便利** — 6 系統ガード + self-healing |
+| Claude Code + ローカル Ollama で tool calling が壊れる | **必須** — tool 修復 (+ 必要なら wire 変換) |
+| Claude Code + ローカルで長時間回すと止まる | **必須級** — 6 系統ガード + self-healing |
+| Ollama v0.14+ / LM Studio にネイティブ直結で動いてる | **便利** — 直結に無い fallback / ガード / 診断を追加 (passthrough で翻訳ゼロ) |
 | codex / gemini-cli + Ollama 直繋ぎで動いてる | オプション — フォールバックが欲しいなら |
 | Claude API を直接叩いてて問題ない | 不要 |
 
@@ -86,14 +105,14 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 
 ## 主な機能
 
-### 接続と修復
+### 修復と接続
 
 | 機能 | 何をしてくれるか |
 |---|---|
-| **Wire 翻訳** | Claude Code (Anthropic形式) ↔ Ollama (OpenAI形式) を自動変換 |
 | **Tool-call 修復** | ローカルモデルがテキストで吐いた JSON を正しい tool_use ブロックに復元 |
 | **3 層フォールバック** | ローカル → 無料クラウド → 有料の順に自動切替 |
-| **出力フィルタ** | `<think>` タグ漏れ、stop marker 漏れを自動除去 |
+| **出力フィルタ** | `<think>` タグ漏れ、stop marker 漏れ、byte-fallback (`<0xNN>`) を自動除去/修復 |
+| **Wire 翻訳** | Anthropic 形式 ↔ OpenAI 形式を自動変換 (ネイティブ `/v1/messages` 対応 backend は passthrough で翻訳ゼロ) |
 
 ### 長時間運用ガード
 

{coderouter_cli-2.6.1 → coderouter_cli-2.7.1}/coderouter/adapters/anthropic_native.py

@@ -35,6 +35,7 @@ from collections.abc import AsyncIterator
 from typing import Any
 
 import httpx
+from pydantic import ValidationError
 
 from coderouter.adapters.base import (
     AdapterError,
@@ -68,6 +69,26 @@ _RETRYABLE_STATUSES = {404, 408, 425, 429, 500, 502, 503, 504}
 _DEFAULT_ANTHROPIC_VERSION = "2023-06-01"
 
 
+def anthropic_messages_url(base_url: str) -> str:
+    """Resolve the ``/v1/messages`` endpoint URL from a ``base_url``.
+
+    ``base_url`` may be given with or without a trailing ``/v1`` — users
+    may point it at either ``https://api.anthropic.com`` or
+    ``https://api.anthropic.com/v1`` (LM Studio and similar servers
+    commonly advertise the latter). We normalize by stripping a trailing
+    ``/v1`` so appending ``/v1/messages`` always yields a valid,
+    non-duplicated URL.
+
+    Shared by :class:`AnthropicAdapter` and the continuous-probe guard so
+    both hit the exact same endpoint (see bug H4: a mismatch made the
+    probe target ``/v1/v1/messages`` and 404 on healthy backends).
+    """
+    base = base_url.rstrip("/")
+    if base.endswith("/v1"):
+        base = base[: -len("/v1")]
+    return f"{base}/v1/messages"
+
+
 class AnthropicAdapter(BaseAdapter):
     """Native Anthropic Messages API adapter (passthrough).
 
@@ -87,15 +108,10 @@ class AnthropicAdapter(BaseAdapter):
 
         ``base_url`` may be given with or without a trailing ``/v1`` —
         we normalize by stripping it so appending ``/v1/messages``
-        always yields a valid URL.
+        always yields a valid URL. Delegates to the shared
+        :func:`anthropic_messages_url` helper.
         """
-        base = str(self.config.base_url).rstrip("/")
-        # Users may point base_url at either `https://api.anthropic.com`
-        # or `https://api.anthropic.com/v1`. We normalize to the former so
-        # we can always append /v1/messages.
-        if base.endswith("/v1"):
-            base = base[: -len("/v1")]
-        return f"{base}/v1/messages"
+        return anthropic_messages_url(str(self.config.base_url))
 
     def _headers(self, request: AnthropicRequest | None = None) -> dict[str, str]:
         """Build Anthropic-native HTTP headers, including beta-header forwarding.
@@ -161,21 +177,21 @@ class AnthropicAdapter(BaseAdapter):
         that auth works and the endpoint is reachable.
         """
         try:
-            async with httpx.AsyncClient(timeout=5.0) as client:
-                resp = await client.post(
-                    self._url(),
-                    headers=self._headers(),
-                    json={
-                        "model": self.config.model,
-                        "max_tokens": 1,
-                        "messages": [{"role": "user", "content": "ping"}],
-                    },
-                )
-                # 200 is clearly healthy. 4xx auth errors indicate the
-                # endpoint is reachable even if the key is bad — still a
-                # "the server answered" signal for healthcheck purposes.
-                # 5xx is upstream trouble.
-                return resp.status_code < 500
+            resp = await self.client().post(
+                self._url(),
+                headers=self._headers(),
+                json={
+                    "model": self.config.model,
+                    "max_tokens": 1,
+                    "messages": [{"role": "user", "content": "ping"}],
+                },
+                timeout=5.0,
+            )
+            # 200 is clearly healthy. 4xx auth errors indicate the
+            # endpoint is reachable even if the key is bad — still a
+            # "the server answered" signal for healthcheck purposes.
+            # 5xx is upstream trouble.
+            return resp.status_code < 500
         except httpx.HTTPError:
             return False
 
@@ -331,8 +347,9 @@ class AnthropicAdapter(BaseAdapter):
         timeout = self.effective_timeout(overrides)
 
         try:
-            async with httpx.AsyncClient(timeout=timeout) as client:
-                resp = await client.post(url, json=payload, headers=self._headers(request))
+            resp = await self.client().post(
+                url, json=payload, headers=self._headers(request), timeout=timeout
+            )
         except httpx.TimeoutException as exc:
             raise AdapterError(
                 f"timeout contacting {url}", provider=self.name, retryable=True
@@ -394,7 +411,22 @@ class AnthropicAdapter(BaseAdapter):
         # (future additions like thinking blocks) pass through via
         # extra="allow" on AnthropicResponse.
         data["coderouter_provider"] = self.name
-        return AnthropicResponse.model_validate(data)
+        # M6: guard the model validation. A 200 body that isn't a valid
+        # Anthropic Messages response (missing ``content``/``id``, an
+        # error envelope returned with a 200, etc.) would otherwise raise
+        # a bare pydantic ValidationError that escapes the engine's
+        # AdapterError-based retry/fallback path. Convert to a retryable
+        # AdapterError so the chain can fall through (mirrors the
+        # invalid-JSON branch above, but retryable — a malformed shape is
+        # usually transient upstream noise rather than a config fault).
+        try:
+            return AnthropicResponse.model_validate(data)
+        except ValidationError as exc:
+            raise AdapterError(
+                f"malformed response shape from upstream: {exc}",
+                provider=self.name,
+                retryable=True,
+            ) from exc
 
     async def stream_anthropic(
         self,
@@ -424,10 +456,14 @@ class AnthropicAdapter(BaseAdapter):
         logged_flag: list[bool] = [False]
 
         try:
-            async with (
-                httpx.AsyncClient(timeout=timeout) as client,
-                client.stream("POST", url, json=payload, headers=self._headers(request)) as resp,
-            ):
+            # H3: stream over the shared client (pool / keep-alive / TLS
+            # reuse). Only the ``stream(...)`` context is scoped here; the
+            # client persists and is closed via ``aclose`` on shutdown. The
+            # ``async with`` releases the response's borrowed connection even
+            # if the caller abandons this generator mid-stream — no GC leak.
+            async with self.client().stream(
+                "POST", url, json=payload, headers=self._headers(request), timeout=timeout
+            ) as resp:
                 if resp.status_code >= 400:
                     body = await resp.aread()
                     raise AdapterError(

{coderouter_cli-2.6.1 → coderouter_cli-2.7.1}/coderouter/adapters/base.py

@@ -11,6 +11,7 @@ from abc import ABC, abstractmethod
 from collections.abc import AsyncIterator
 from typing import Any, Literal
 
+import httpx
 from pydantic import BaseModel, ConfigDict, Field
 
 from coderouter.config.schemas import ProviderConfig
@@ -164,17 +165,59 @@ class BaseAdapter(ABC):
     def __init__(self, config: ProviderConfig) -> None:
         """Bind the adapter to a :class:`ProviderConfig`.
 
-        Subclasses do not need to override this; HTTP clients are
-        constructed lazily inside :meth:`generate` / :meth:`stream` so
-        each call can honor a per-call timeout override.
+        Subclasses do not need to override this. A single shared
+        :class:`httpx.AsyncClient` is created lazily on first use (see
+        :meth:`client`) and reused across every call on this adapter so
+        the connection pool, HTTP keep-alive, and TLS session are all
+        reused. Per-call timeouts are still honored by passing
+        ``timeout=`` to ``client.post`` / ``client.stream`` — they do
+        NOT require a fresh client.
         """
         self.config = config
+        # Shared client, created lazily inside the running event loop the
+        # first time an adapter method needs it (see ``client``). Kept as
+        # ``None`` until then so constructing an adapter has no I/O cost and
+        # is safe outside an event loop (e.g. at import / config time).
+        self._client: httpx.AsyncClient | None = None
 
     @property
     def name(self) -> str:
         """Shortcut for ``self.config.name`` — used in log trails and errors."""
         return self.config.name
 
+    # ---- H3: shared HTTP client (connection-pool / keep-alive reuse) ----
+    def client(self) -> httpx.AsyncClient:
+        """Return the shared :class:`httpx.AsyncClient`, creating it lazily.
+
+        The client is created on first use so construction happens inside
+        the running event loop and carries no I/O cost at adapter-build
+        time. Subsequent calls return the same instance, which is what
+        lets httpx reuse pooled connections, keep-alive, and the TLS
+        session across requests.
+
+        No default timeout is baked in here: every call site passes an
+        explicit per-call ``timeout=`` (resolved from the active profile
+        via :meth:`effective_timeout`), so leaving the client timeout
+        unset avoids a surprising default clamping long-running calls.
+        """
+        if self._client is None:
+            self._client = httpx.AsyncClient(timeout=None)
+        return self._client
+
+    async def aclose(self) -> None:
+        """Close the shared HTTP client and drop the reference.
+
+        Idempotent: safe to call when no client was ever created, and
+        safe to call more than once. Invoked from the app lifespan
+        shutdown path so pooled connections are released cleanly rather
+        than left to garbage collection. After ``aclose`` a later call
+        re-creates the client on demand via :meth:`client`.
+        """
+        client = self._client
+        self._client = None
+        if client is not None:
+            await client.aclose()
+
     # ---- v0.6-B override resolution helpers -----------------------------
     def effective_timeout(self, overrides: ProviderCallOverrides | None) -> float:
         """Profile override wins when set; else provider default."""