coderouter-cli 2.7.2__tar.gz → 2.7.4__tar.gz

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/CHANGELOG.md

@@ -6,6 +6,89 @@ versioning follows [SemVer](https://semver.org/).
 
 ---
 
+## [v2.7.4] — 2026-07-06 (Launcher provider auto-sync + /v1/models passthrough)
+
+Two PRs closing the "launcher started it, but nothing can route to it /
+nothing can *name* it" gap, driven by a real field failure (launcher on
+port 8085, providers.yaml pointing at 8080; external benchmark reports
+indistinguishable across loaded GGUFs). **PR #48** (models passthrough)
+and **PR #49** (launcher auto-sync). Fully backward compatible, no new
+dependencies. Verified end-to-end on real hardware (NucBox EVO-X2,
+llama.cpp + omnicoder-9b): zero-config launch → route via
+`X-CodeRouter-Profile: launcher` → response carries the real GGUF name.
+
+### Added
+
+- **`/v1/models` upstream passthrough** for empty-model providers.
+  A provider with `model: ""` (the launcher/llama-server pattern — the
+  upstream decides what is loaded) used to be listed by its static
+  provider name only. Now `/v1/models` queries the upstream's own
+  `/models` (2s timeout, 30s TTL cache) and surfaces the real loaded
+  model id(s) as `owned_by: coderouter/<provider>`. Any failure falls
+  back to the historic provider-name entry; providers with a configured
+  model keep the exact previous shape. External benchmarks (e.g.
+  SWE-bench harnesses with `model: auto`) can now tell GGUF swaps apart
+  without config edits. (PR #48)
+- **Launcher provider auto-sync.** A backend started from the embedded
+  launcher (`/api/launcher/start`) is auto-registered as a routable
+  provider named `launcher-<backend>-<port>` (restart on the same port
+  replaces, never duplicates) and placed at the FRONT of an on-demand
+  `launcher` profile. `default_profile` is never touched — routing is an
+  explicit opt-in via `X-CodeRouter-Profile: launcher` or body
+  `profile`. Registration is deliberately **in-memory only** (a YAML
+  rewrite would destroy operator comments; provider and process share
+  the server's lifetime). Sync failures never fail the start; the
+  result rides on the start response as `provider_sync`. New engine
+  API: `FallbackEngine.register_provider()`. (PR #49)
+
+### Docs
+
+- README (JA/EN) launcher feature table + `docs/backends/launcher.md`:
+  auto-sync semantics, verification curl recipes, and the desktop-GUI
+  caveat (launcher_gui.py runs out-of-process and is not synced).
+
+---
+
+## [v2.7.3] — 2026-07-05 (Per-request empty-response fallback + L3 bench evidence)
+
+Two PRs: **PR #45** (`empty_response_action` — the fallback leg of the
+three-tier tool-call reliability story) and **PR #46** (L3 benchmark
+refresh: model matrix, repairer-version A/B archives, results). Fully
+backward compatible: new knob defaults to `off`, no breaking config
+changes, no new dependencies.
+
+### Added
+
+- **`FallbackChain.empty_response_action: off | warn | fallback`**
+  (default `off`). With `fallback`, a 200 response that is *content-empty*
+  (no `tool_use`, no non-whitespace text; thinking-only counts as empty) is
+  re-dispatched in-flight to the next provider in the chain. Non-streaming
+  responses are judged on the finalized object; streaming buffers events
+  until the first real content token and discards an unobserved stream.
+  Emptiness is judged on content, not `usage.output_tokens` (unreliable on
+  some backends). Motivation: gemma4:26b returns blank 200s on
+  `no_tool_temptation`-type prompts (20/20 at temperature 0, direct AND via
+  router) — no text exists for the repair layer to fix, and the windowed
+  drift guard `empty_response_rate` cannot rescue a single blank turn
+  in-flight. 21 new tests. (PR #45)
+- **Benchmark refresh**: full L3 model matrix in
+  `benchmarks/tool-repair/providers.bench.yaml` (llama3.2/3.1, mistral,
+  qwen2.5-coder:1.5b, phi4-mini + originals, `bench-gemma-chain` enabled),
+  one-shot matrix runner `bench_l3_p1.sh`, latest live results plus
+  repairer-version A/B snapshots under `results/archive-v2.7.{0,1}/`.
+  (PR #46)
+
+### Results
+
+- Live chain validation (M3 Max, temperature 0, 100 requests, zero
+  errors): gemma4:26b → qwen3-coder:30b chain **80% → 100%** native —
+  all 20 blank turns rescued.
+- Three-tier story now fully measured: repair rescues broken-but-present
+  calls (qwen2.5×2 / mistral 0→100, phi4 +60pt), healthy models pass
+  through undegraded, and fallback closes what repair cannot touch.
+
+---
+
 ## [v2.7.2] — 2026-07-05 (R4 tool-call repair: nested-XML / JSON-envelope / call-syntax forms)
 
 Single feature PR **#43**, driven by the 2026-07-05 L3 live benchmark

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: coderouter-cli
-Version: 2.7.2
+Version: 2.7.4
 Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
 Project-URL: Homepage, https://github.com/zephel01/CodeRouter
 Project-URL: Repository, https://github.com/zephel01/CodeRouter
@@ -220,6 +220,8 @@ providers:
 | **オプションプロファイル** | `providers.yaml` に名前付きプリセットを定義 → ドロップダウンで選択するだけ |
 | **複数プロセス管理** | llama.cpp と vllm を同時に起動し、ポートごとに独立管理 |
 | **ログビューア** | 各プロセスの stdout/stderr をブラウザ内でリアルタイム確認 |
+| **provider 自動同期** (v2.7.4) | 起動したバックエンドを provider として自動登録(`launcher-llamacpp-8085` 等)。providers.yaml 無編集で `X-CodeRouter-Profile: launcher` からルーティング可能。メモリ内のみ・serve と同寿命 |
+| **モデル名パススルー** (v2.7.4) | `model: ""` の provider は `/v1/models` が上流のロード中モデル ID(gguf 名)をそのまま返す。gguf を差し替えても config 編集不要 — 外部ベンチからモデルを識別できる |
 
 ```yaml
 # providers.yaml に追記するだけで有効になる

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/README.en.md

@@ -156,6 +156,8 @@ Browser UI at `http://localhost:8088/launcher` for starting and managing local i
 | **Option profiles** | Name your flag presets in `providers.yaml` — select from a dropdown, no CLI needed |
 | **Multi-process** | Run llama.cpp and vllm side by side on different ports |
 | **Log viewer** | stdout/stderr of each process shown live in the browser |
+| **Provider auto-sync** (v2.7.4) | A started backend is auto-registered as a routable provider (`launcher-llamacpp-8085`); route to it with `X-CodeRouter-Profile: launcher` — zero providers.yaml edits. In-memory, shares the server's lifetime |
+| **Model-id passthrough** (v2.7.4) | Providers with `model: ""` make `/v1/models` surface the upstream's actually-loaded model id (the GGUF name). Swap models without touching config — external benchmarks can tell runs apart |
 
 ```yaml
 # Add to providers.yaml — no code changes needed

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/README.md

@@ -175,6 +175,8 @@ providers:
 | **オプションプロファイル** | `providers.yaml` に名前付きプリセットを定義 → ドロップダウンで選択するだけ |
 | **複数プロセス管理** | llama.cpp と vllm を同時に起動し、ポートごとに独立管理 |
 | **ログビューア** | 各プロセスの stdout/stderr をブラウザ内でリアルタイム確認 |
+| **provider 自動同期** (v2.7.4) | 起動したバックエンドを provider として自動登録(`launcher-llamacpp-8085` 等)。providers.yaml 無編集で `X-CodeRouter-Profile: launcher` からルーティング可能。メモリ内のみ・serve と同寿命 |
+| **モデル名パススルー** (v2.7.4) | `model: ""` の provider は `/v1/models` が上流のロード中モデル ID(gguf 名)をそのまま返す。gguf を差し替えても config 編集不要 — 外部ベンチからモデルを識別できる |
 
 ```yaml
 # providers.yaml に追記するだけで有効になる

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/coderouter/config/schemas.py

@@ -718,6 +718,57 @@ class FallbackChain(BaseModel):
         ),
     )
 
+    # --- ⑧ (empty-response): per-request empty-response fallback ------------
+    #
+    # Some local backends (observed: gemma4:26b on ``no_tool_temptation``
+    # prompts) return a 200 with a structurally-valid but *content-empty*
+    # Anthropic response — no tool_use and no non-whitespace text — for a
+    # fraction of requests. The drift guard's ``empty_response_rate`` is a
+    # windowed aggregate (default threshold 0.3, ``min_window_fill`` 6): it
+    # promotes/reloads a backend once the *rate* is bad, but cannot rescue a
+    # single blank turn in-flight. This knob adds a per-request in-flight
+    # fallback that re-dispatches the *same* request to the next provider in
+    # the chain the moment an empty response is detected.
+    #
+    # Design: "empty" is judged on *content*, not usage.output_tokens (which
+    # some backends report unreliably). A response is empty when its content
+    # list is empty, or every block is either a whitespace-only ``text`` block
+    # or a ``thinking`` block — i.e. nothing the client can act on. A single
+    # ``tool_use`` block or one non-whitespace ``text`` block makes it non-empty.
+    #
+    #   * ``off``      — no detection, no fallback, no log. Backward-compatible
+    #                    default (byte-for-byte identical to pre-⑧ behavior).
+    #   * ``warn``     — detect + emit an ``empty-response-detected`` log line
+    #                    only; the empty response is returned unchanged.
+    #   * ``fallback`` — on an empty response, log ``empty-response-detected``
+    #                    and continue to the next provider (the empty response
+    #                    is *not* recorded as an error). If every provider in
+    #                    the chain returns empty, the last empty response is
+    #                    returned as-is (a 200 blank is a legitimate answer)
+    #                    with ``chain_exhausted=True`` on the log line. On the
+    #                    streaming path, ``fallback`` buffers events until real
+    #                    content is observed, so an empty stream can be swapped
+    #                    to the next provider without the client seeing bytes.
+    #
+    # Default ``off`` preserves complete backward compatibility; the original
+    # request object is never mutated, so a later provider always receives the
+    # untouched request.
+    empty_response_action: Literal["off", "warn", "fallback"] = Field(
+        default="off",
+        description=(
+            "⑧ (empty-response): action when a provider returns a 200 with "
+            "empty content (no tool_use and no non-whitespace text; a "
+            "thinking-only response counts as empty). ``off`` (default) does "
+            "nothing. ``warn`` emits an ``empty-response-detected`` log only "
+            "and returns the empty response. ``fallback`` re-dispatches the "
+            "same request to the next provider (empty is not counted as an "
+            "error); if the whole chain returns empty, the last empty response "
+            "is returned unchanged. Streaming buffers events until real "
+            "content appears so empty streams can be swapped provider-side "
+            "without the client seeing any bytes."
+        ),
+    )
+
     # --- S2 (shim): tool_choice capability gate + emulation -----------------
     #
     # Anthropic clients (Claude Code, SDKs) can pin the model to a specific

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/coderouter/ingress/launcher_routes.py

@@ -655,6 +655,28 @@ async def api_backends(request: Request) -> dict[str, Any]:
     return {"backends": result}
 
 
+def _launcher_provider_config(backend: str, port: int) -> Any:
+    """Build the provider entry for a launcher-started backend.
+
+    ``model`` is left EMPTY on purpose: llama-server / vllm / mlx decide
+    which model is actually loaded, and the empty-model /v1/models
+    passthrough then surfaces the upstream's real model id to benchmark
+    clients — swap the GGUF, no config edit needed.
+
+    Name embeds backend + port (``launcher-llamacpp-8085``) so restarting
+    on the same port REPLACES the entry instead of piling up duplicates.
+    """
+    from coderouter.config.schemas import ProviderConfig
+
+    safe_backend = backend.replace(".", "")
+    return ProviderConfig(
+        name=f"launcher-{safe_backend}-{port}",
+        base_url=f"http://localhost:{port}/v1",
+        model="",
+        timeout_s=120.0,
+    )
+
+
 @router.post("/api/launcher/start")
 async def api_start(req: StartRequest, request: Request) -> dict[str, Any]:
     """Start a new backend process."""
@@ -717,7 +739,35 @@ async def api_start(req: StartRequest, request: Request) -> dict[str, Any]:
     _background_tasks.add(_task)
     _task.add_done_callback(_background_tasks.discard)
 
-    return {"id": proc_id, "pid": p.pid, "command": cmd}
+    # Provider auto-sync: register the just-started backend as a routable
+    # provider so requests can reach it without hand-editing providers.yaml
+    # (in-memory only — see FallbackEngine.register_provider docstring).
+    # Sync failure must never fail the start itself.
+    provider_sync: dict[str, Any] | None = None
+    engine = getattr(request.app.state, "engine", None)
+    if engine is not None and hasattr(engine, "register_provider"):
+        try:
+            provider_sync = engine.register_provider(
+                _launcher_provider_config(req.backend, req.port)
+            )
+            proc.log_tail.append(
+                "[launcher] provider sync: "
+                f"{provider_sync['provider']} -> profile "
+                f"'{provider_sync['profile']}' (in-memory)"
+            )
+        except Exception as exc:
+            logger.warning(
+                "launcher provider sync failed",
+                extra={"backend": req.backend, "port": req.port, "error": str(exc)},
+            )
+            proc.log_tail.append(f"[launcher] provider sync failed: {exc}")
+
+    return {
+        "id": proc_id,
+        "pid": p.pid,
+        "command": cmd,
+        "provider_sync": provider_sync,
+    }
 
 
 @router.post("/api/launcher/stop/{proc_id}")

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/coderouter/ingress/openai_routes.py

@@ -78,22 +78,99 @@ def _resolve_stream_timeout_s(engine: FallbackEngine, profile: str | None) -> fl
     return max(_STREAM_TIMEOUT_MIN_S, float(per_call) * _STREAM_TIMEOUT_MULTIPLIER)
 
 
+# --- /v1/models upstream passthrough -------------------------------------
+#
+# Providers with an *empty* ``model`` field are passthrough providers: the
+# upstream (llama-server, LM Studio, ...) decides which model is loaded and
+# CodeRouter never sends a model name. For those providers the historic
+# behaviour — returning the provider *name* — makes external benchmarks
+# indistinguishable across loaded GGUFs (the launcher_gui setup always
+# reported ``llama-cpp-local`` regardless of the selected model). We now ask
+# the upstream's own ``/models`` endpoint for its real model id(s) and
+# surface those instead.
+#
+# Guards: only for ``kind == "openai_compat"`` with ``model == ""``; a short
+# per-upstream TTL cache keeps repeated SDK probes cheap; ANY failure (refused
+# connection, timeout, unexpected shape) falls back to the historic
+# provider-name entry, so this is strictly additive. Providers with a
+# configured model keep the exact previous behaviour.
+
+_UPSTREAM_MODELS_TTL_S = 30.0
+_UPSTREAM_MODELS_TIMEOUT_S = 2.0
+# base_url -> (expires_monotonic, ids)
+_upstream_models_cache: dict[str, tuple[float, list[str]]] = {}
+
+
+async def _fetch_upstream_model_ids(base_url: str) -> list[str]:
+    """GET {base_url}/models and return the upstream model ids.
+
+    Raises on any transport / shape problem — the caller treats every
+    exception as "fall back to the provider-name entry".
+    """
+    import httpx  # local import keeps module import time flat
+
+    url = base_url.rstrip("/") + "/models"
+    async with httpx.AsyncClient(timeout=_UPSTREAM_MODELS_TIMEOUT_S) as client:
+        resp = await client.get(url)
+        resp.raise_for_status()
+        payload = resp.json()
+    ids = [
+        str(item["id"])
+        for item in payload.get("data", [])
+        if isinstance(item, dict) and item.get("id")
+    ]
+    if not ids:
+        raise ValueError(f"upstream {url} returned no model ids")
+    return ids
+
+
+async def _upstream_model_ids_cached(base_url: str) -> list[str]:
+    """TTL-cached wrapper around :func:`_fetch_upstream_model_ids`."""
+    now = time.monotonic()
+    hit = _upstream_models_cache.get(base_url)
+    if hit is not None and hit[0] > now:
+        return hit[1]
+    ids = await _fetch_upstream_model_ids(base_url)
+    _upstream_models_cache[base_url] = (now + _UPSTREAM_MODELS_TTL_S, ids)
+    return ids
+
+
 @router.get("/models")
 async def list_models(request: Request) -> dict[str, object]:
-    """Minimal /v1/models so OpenAI SDKs that probe it don't choke."""
+    """/v1/models: provider names, with upstream passthrough for empty-model
+    providers (see the passthrough note above)."""
     config = request.app.state.config
-    return {
-        "object": "list",
-        "data": [
+    created = int(time.time())
+    data: list[dict[str, object]] = []
+    for p in config.providers:
+        if p.model == "" and p.kind == "openai_compat":
+            try:
+                ids = await _upstream_model_ids_cached(str(p.base_url))
+            except Exception:  # any failure means "no passthrough"
+                logger.debug(
+                    "models passthrough failed; falling back to provider name",
+                    extra={"provider": p.name},
+                )
+            else:
+                data.extend(
+                    {
+                        "id": model_id,
+                        "object": "model",
+                        "created": created,
+                        "owned_by": f"coderouter/{p.name}",
+                    }
+                    for model_id in ids
+                )
+                continue
+        data.append(
             {
                 "id": p.name,
                 "object": "model",
-                "created": int(time.time()),
+                "created": created,
                 "owned_by": "coderouter",
             }
-            for p in config.providers
-        ],
-    }
+        )
+    return {"object": "list", "data": data}
 
 
 @router.post("/chat/completions", response_model=None)

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/coderouter/logging.py

@@ -1372,6 +1372,46 @@ def log_partial_stitch_surfaced(
     )
 
 
+# ---------------------------------------------------------------------------
+# ⑧ (empty-response): per-request empty-response fallback log shape
+#
+# Fired when a provider returns a 200 with content-empty output (no
+# tool_use and no non-whitespace text). The single event lane carries the
+# ``action`` (warn|fallback) that triggered it, whether the empty response
+# came from the streaming or non-streaming path, and ``chain_exhausted``
+# — True only when every provider in the chain returned empty and the last
+# empty response is being returned to the client as-is.
+# ---------------------------------------------------------------------------
+
+
+def log_empty_response_detected(
+    logger: logging.Logger,
+    provider: str,
+    *,
+    action: str,
+    stream: bool,
+    chain_exhausted: bool = False,
+) -> None:
+    """Emit an ``empty-response-detected`` warn line.
+
+    Fired when ``empty_response_action`` is ``warn`` or ``fallback`` and a
+    provider returns a structurally-valid but content-empty response. The
+    ``action`` field records which knob value produced the line; ``stream``
+    distinguishes the SSE path from the non-streaming path; ``chain_exhausted``
+    is True only when the whole chain returned empty and the last empty
+    response is being returned unchanged.
+    """
+    logger.warning(
+        "empty-response-detected",
+        extra={
+            "provider": provider,
+            "action": action,
+            "stream": stream,
+            "chain_exhausted": chain_exhausted,
+        },
+    )
+
+
 # ---------------------------------------------------------------------------
 # v2.0-I: Continuous probe log shapes
 #

{coderouter_cli-2.7.2 → coderouter_cli-2.7.4}/coderouter/metrics/collector.py

@@ -52,6 +52,7 @@ Event inventory (dispatch table in :meth:`MetricsCollector._dispatch`)
     ``drift-promoted`` (v2.0-G)  → ``drift_promoted_total``
     ``drift-reload-attempted``   → ``drift_reload_total`` / success
     ``partial-stitch-surfaced``  → ``partial_stitch_surfaced_total`` (v2.0-H)
+    ``empty-response-detected``  → ``empty_responses_total`` + per-provider (⑧)
     ``probe-completed`` (v2.0-I) → ``probe_total`` / ``probe_success`` / ``probe_failure``
                                    + per-provider latency gauge
     ``probe-round-completed``    → ``probe_rounds_total`` (v2.0-I)
@@ -109,6 +110,7 @@ _KNOWN_EVENTS: Final[frozenset[str]] = frozenset(
         "drift-promoted",
         "drift-reload-attempted",
         "partial-stitch-surfaced",
+        "empty-response-detected",
         "probe-completed",
         "probe-round-completed",
         "probe-capabilities-drift",
@@ -272,6 +274,13 @@ class MetricsCollector(logging.Handler):
         # client instead of a generic error event.
         self._partial_stitch_surfaced_total: int = 0
 
+        # ⑧ (empty-response): count of per-request empty-response detections
+        # (both ``warn`` and ``fallback`` actions). Aggregate + per-provider
+        # so a dashboard can spot which backend blanks out. Counts every
+        # detection event, including the terminal chain-exhausted one.
+        self._empty_responses_total: int = 0
+        self._empty_responses_by_provider: Counter[str] = Counter()
+
         # v2.0-I: continuous probe counters. Per-provider probe attempts
         # and outcomes, plus a round counter for the dashboard's
         # "probes/min" panel.
@@ -329,6 +338,7 @@ class MetricsCollector(logging.Handler):
             "drift-promoted": self._on_drift_promoted,
             "drift-reload-attempted": self._on_drift_reload_attempted,
             "partial-stitch-surfaced": self._on_partial_stitch_surfaced,
+            "empty-response-detected": self._on_empty_response_detected,
             "probe-completed": self._on_probe_completed,
             "probe-round-completed": self._on_probe_round_completed,
             "probe-capabilities-drift": self._on_probe_capabilities_drift,
@@ -670,6 +680,17 @@ class MetricsCollector(logging.Handler):
         self._partial_stitch_surfaced_total += 1
         self._push_recent("partial-stitch-surfaced", extras, record)
 
+    def _on_empty_response_detected(
+        self, extras: dict[str, Any], record: logging.LogRecord
+    ) -> None:
+        # ⑧ (empty-response): a content-empty 200 was detected under a
+        # ``warn`` / ``fallback`` action.
+        self._empty_responses_total += 1
+        provider = _str(extras.get("provider"))
+        if provider:
+            self._empty_responses_by_provider[provider] += 1
+        self._push_recent("empty-response-detected", extras, record)
+
     def _on_probe_completed(
         self, extras: dict[str, Any], record: logging.LogRecord
     ) -> None:
@@ -888,6 +909,11 @@ class MetricsCollector(logging.Handler):
                     "drift_reload_success_total": self._drift_reload_success_total,
                     # v2.0-H (L6): partial stitch surfaced.
                     "partial_stitch_surfaced_total": self._partial_stitch_surfaced_total,
+                    # ⑧ (empty-response): per-request empty-response fallback.
+                    "empty_responses_total": self._empty_responses_total,
+                    "empty_responses_by_provider": dict(
+                        self._empty_responses_by_provider
+                    ),
                     # v2.0-I: continuous probe counters.
                     "probe_total": dict(self._probe_total),
                     "probe_success": dict(self._probe_success),
@@ -1042,6 +1068,9 @@ class MetricsCollector(logging.Handler):
             self._tokens_saved_by_mechanism.clear()
             # v2.0-H (L6)
             self._partial_stitch_surfaced_total = 0
+            # ⑧ (empty-response)
+            self._empty_responses_total = 0
+            self._empty_responses_by_provider.clear()
             # v2.0-I
             self._probe_total.clear()
             self._probe_success.clear()