coderouter-cli 2.9.2__tar.gz → 2.9.4__tar.gz

{coderouter_cli-2.9.2 → coderouter_cli-2.9.4}/.gitignore

@@ -7,6 +7,7 @@
 !.env.example
 secrets.env
 ~/.coderouter/
+CLAUDE.md
 
 # ============================================================
 # Python

{coderouter_cli-2.9.2 → coderouter_cli-2.9.4}/CHANGELOG.md

@@ -9,6 +9,187 @@ are kept verbatim where the Japanese text itself is the subject).
 
 ---
 
+## [v2.9.4] — 2026-07-19 (launcher device selection & bench sweep, cache-stable system prompt)
+
+### Added
+
+- **llama.cpp device selection (`--device` / `--tensor-split`) in the
+  Launcher.** Both the desktop GUI and the Web edition can now detect
+  available `llama-server` devices (via `--list-devices`), display their
+  VRAM, and let the operator pick which one(s) to offload to instead of
+  always relying on llama.cpp's own default placement. New shared module
+  `coderouter/launcher_devices.py` (dataclasses + pure functions only, no
+  pydantic, so the standalone GUI can import it without the `coderouter`
+  package) owns detection/caching (`detect_llama_devices`,
+  `parse_list_devices`, a 60s TTL cache keyed by binary path),
+  CLI-fragment building (`DeviceSelection.to_cli_args` — empty selection
+  emits nothing, preserving today's launch command byte-for-byte),
+  VRAM-ratio tensor-split suggestion (`suggest_tensor_split`), and backend
+  grouping (`backend_of` / `group_by_backend` / `selectable_devices`) so a
+  GPU listed twice under different backends (e.g. `CUDA0` and `Vulkan1`
+  for the same physical card) is never double-counted and cross-backend
+  configs are never auto-generated. Devices reporting `0 MiB` (e.g. macOS
+  `BLAS: Accelerate`) are still shown for information but excluded from
+  selection/suggestion. `StartRequest` gained `device_ids` / `tensor_split`
+  (both default empty — existing Web clients are unaffected), and
+  `GET /api/launcher/devices` (`?backend=`, `?refresh=1`) returns the
+  probe plus a per-backend `suggested_tensor_split` and `auto_configs`.
+  `launcher_gui.py`'s LAUNCH form gained a device checklist (🔍 Detect,
+  falling back to a manual comma-separated entry on detection failure)
+  and a tensor-split field that auto-hides when one or fewer devices are
+  detected. See `tests/test_launcher_devices.py`,
+  `tests/test_launcher_devices_detect.py`,
+  `tests/test_launcher_devices_routes.py`,
+  `tests/test_launcher_gui_devices.py`, and the new "Device selection"
+  section in `docs/backends/launcher.md` / `launcher.en.md`.
+- **Bench sweep: automated device-configuration benchmarking against an
+  external `llmbench`.** Both Launcher editions can now drive a list of
+  device configurations (e.g. `CUDA0` alone / `CUDA1` alone / a
+  multi-GPU split) through start → wait for readiness → run an external
+  bench command → stop → advance to the next configuration, then compare
+  the results. `coderouter/launcher_devices.py` supplies the shared
+  `SweepPlan` / `SweepStep` / `SweepState` state machine,
+  `build_auto_sweep_configs` (one config per selectable device plus one
+  per multi-device backend group), `render_bench_command` (template
+  placeholders `{port}` `{config}` `{base_url}` `{results_dir}` `{runs}`,
+  expanded via plain string substitution rather than `str.format` so JSON
+  braces in the command aren't misinterpreted, then `shlex`-split with
+  `posix=False` on Windows to keep backslash paths intact), and
+  `load_latest_results` / `summarize_results` (best-effort, alias-tolerant
+  extraction of `tokens_per_sec` / `ttft_ms` / `latency_ms` from the
+  newest `llmbench` results JSON). New `LauncherBenchConfig`
+  (`command_template` / `runs` / `results_dir` / `readiness_timeout_s`,
+  read from an optional `launcher.bench:` block — omitted entirely by
+  default, so existing `providers.yaml` files are unaffected) supplies
+  the sweep's defaults. The Web edition gained
+  `POST /api/launcher/sweep/start` (token-gated; 409 if a sweep is
+  already running, 400 for empty `configs` or a busy/unavailable port),
+  `GET /api/launcher/sweep/status`, `POST /api/launcher/sweep/abort`
+  (token-gated), and `GET /api/launcher/sweep/logs`, backed by a new
+  `_SweepRunner` that reuses the existing `spawn_process` /
+  `stop_process` / `proc.ready` readiness primitives. `launcher_gui.py`
+  gained a "📊 Bench sweep" button opening a separate `SweepWindow` with
+  its own `_SweepWorker` thread, reusing `poll_until_ready` for readiness
+  and driving the same external bench subprocess model. See
+  `tests/test_launcher_config_bench.py` and the new "Bench sweep" section
+  in `docs/backends/launcher.md` / `launcher.en.md`.
+
+### Fixed
+
+- **Mid-conversation `role:"system"` messages are no longer hoisted into
+  the top-level `system` field.** (#75, contributed by @firelzrd.)
+  Claude Code CLI ≥ 2.1.154 emits volatile system reminders as
+  `role:"system"` messages *inside* `messages[]`; hoisting them meant the
+  serialized prompt's head changed every turn, so llama.cpp / LM Studio
+  style prefix-matching KV caches were invalidated wholesale (a captured
+  real session measured only 11.8 % reusable prefix between consecutive
+  requests). `normalize_message_roles()` now splits by position: leading
+  system messages (before any emitted turn) are hoisted as before — the
+  legitimate OpenAI-style system prompt — while mid-conversation ones are
+  coerced to `user` in place, keeping `messages` append-only and the
+  prompt head stable as the conversation grows. See
+  `tests/test_role_normalization.py`
+  (`test_leading_system_is_hoisted_but_mid_conversation_is_not`,
+  `test_system_prompt_is_stable_as_conversation_grows`).
+
+### Security
+
+- **Dependency bumps for two fresh advisories** (lockfile-only; both
+  verified with a clean `uv sync --frozen` + full test suite, 1905
+  passed): `click` 8.3.2 → 8.4.2 (PYSEC-2026-2132, fix ≥ 8.3.3;
+  transitive via uvicorn) and `json-repair` 0.59.10 → 0.61.5
+  (GHSA-xf7x-x43h-rpqh, CVSS 7.5, fix ≥ 0.60.1). The `json-repair`
+  floor in `pyproject.toml` was raised from `>=0.30` to `>=0.60.1` so
+  fresh installs can never resolve a vulnerable version; `uv.lock` also
+  records the new lockfile revision 3 format (upload-time metadata)
+  emitted by current uv.
+
+---
+
+## [v2.9.3] — 2026-07-12 (GUI/Web parity, swap polish, launcher UI fixes)
+
+### Added
+
+- **Per-model TTL override for `launcher.swap`.** `SwapModelSpec.ttl_seconds`
+  (new, optional, `None` by default) lets a catalog entry override the
+  global `launcher.swap.ttl_seconds` for just that one model — `None`
+  keeps following the global value, `0` unloads the model as soon as
+  its last in-flight lease releases (same meaning as the global field's
+  `0`, just scoped to one entry). `SwapManager.sweep_once` now resolves
+  each model's effective TTL individually, and the sweeper starts
+  whenever either the global TTL or at least one catalog entry's
+  override is set (previously gated solely on the global value). See
+  `tests/test_launcher_swap.py` (`test_ttl_override_spec_wins_over_global`,
+  `test_ttl_override_unset_falls_back_to_global`,
+  `test_ttl_override_zero_unloads_even_when_global_ttl_disabled`) and the
+  updated `launcher.swap` field table in `docs/backends/launcher.md` /
+  `launcher.en.md`.
+- **Configurable ephemeral-port retry count for `launcher.swap`.**
+  `LauncherSwapConfig.port_retry_attempts` (new, default `2`, range
+  0–5) replaces the previously hard-coded single retry for catalog
+  entries that leave `port` unset — the swap manager now attempts up
+  to `1 + port_retry_attempts` spawns (each on a freshly picked
+  ephemeral port) before giving up, versus the previous fixed 2 total
+  attempts. Fixed-port entries are unaffected (still exactly one
+  attempt — a second try on the same port would just collide again).
+  This narrows the practical impact of the known pick-then-bind TOCTOU
+  window documented on `SwapModelSpec.port` / `_pick_ephemeral_port`
+  but does not close it outright; a fixed `port` remains the only way
+  to eliminate the race entirely. See
+  `tests/test_launcher_swap.py`
+  (`test_port_none_retries_default_port_retry_attempts`,
+  `test_port_retry_attempts_exhausted_raises`,
+  `test_port_retry_attempts_configurable_to_zero`).
+- **`/launcher` UI now shows which processes are swap-managed.**
+  `GET /api/launcher/processes` includes `swap_managed` (bool) and
+  `swap_model` (the swap catalog model name, or `null`) for every
+  process; `spawn_process` gained a `swap_model` kwarg that
+  `SwapManager._spawn` now passes through. The process table renders a
+  small "swap" badge next to the name of any swap-managed process
+  (title shows the backing catalog model when known). Manually-started
+  processes are unaffected (`swap_managed: false`, `swap_model: null`).
+  See `tests/test_launcher_swap.py::test_i1_on_demand_spawn_reaches_200`.
+
+### Fixed
+
+- **Swap-managed "stopped" processes no longer accumulate in the launcher
+  registry.** `stop_process` only sets `status="stopped"` and never
+  removes the entry — deliberate for manually started processes (the
+  stopped row is visible history in the /launcher UI, with logs and an
+  explicit ✕ delete button), but swap-managed processes went through the
+  same path, so every failed readiness attempt left one permanent
+  "stopped" row (`1 + port_retry_attempts` rows per failed load — 3 with
+  the defaults) and every TTL unload left another, all growing
+  `GET /api/launcher/processes` and the UI without bound. `SwapManager`
+  now removes its own processes from the registry after stopping them,
+  in both the failed-readiness cleanup (`_spawn_with_retry`) and the TTL
+  unload (`_unload_locked`), guarded on `ManagedProcess.swap_managed` so
+  a manual process is never swept up. Crash leftovers (a swap process
+  that died on its own) are deliberately kept — their log tail is the
+  only crash forensics an operator has. See `tests/test_launcher_swap.py`
+  (`test_failed_load_leaves_no_registry_litter`,
+  `test_ttl_unload_removes_registry_entry`,
+  `test_registry_removal_skips_non_swap_processes`; adapted from the
+  review repro with its assertions inverted).
+- **/launcher UI no longer 404-polls a removed process's logs forever.**
+  The log panel's poller (`poll()` → `refreshLogs()`, every 3s) never
+  checked the response status: once the polled id left the registry —
+  a `coderouter serve` restart under a still-open browser tab, a delete
+  issued from another client, or (with the fix above) a swap TTL unload
+  removing the entry mid-view — the 404 body has no `.logs`, the
+  resulting TypeError landed in the generic catch, and `selectedLogId`
+  was never cleared, so the tab kept hitting
+  `GET /api/launcher/logs/<stale-id>` and spamming `404 Not Found` into
+  the serve log indefinitely. The UI now stops polling as soon as the
+  selected id disappears from the periodic `/api/launcher/processes`
+  refresh (usually before a single 404 is even issued), treats a logs
+  404 as terminal (shows "(process removed)" instead of retrying), and
+  the server keeps answering 404 for unknown ids — that behavior is
+  correct and now pinned by
+  `tests/test_launcher_swap.py::test_logs_unknown_proc_id_is_404`.
+
+---
+
 ## [v2.9.2] — 2026-07-12 (config: no more dummy provider for swap-only setups)
 
 ### Changed

{coderouter_cli-2.9.2 → coderouter_cli-2.9.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: coderouter-cli
-Version: 2.9.2
+Version: 2.9.4
 Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
 Project-URL: Homepage, https://github.com/zephel01/CodeRouter
 Project-URL: Repository, https://github.com/zephel01/CodeRouter
@@ -40,7 +40,7 @@ Requires-Dist: types-pyyaml>=6.0.12; extra == 'dev'
 Provides-Extra: doctor
 Requires-Dist: ruamel-yaml>=0.18.6; extra == 'doctor'
 Provides-Extra: repair
-Requires-Dist: json-repair>=0.30; extra == 'repair'
+Requires-Dist: json-repair>=0.60.1; extra == 'repair'
 Description-Content-Type: text/markdown
 
 <h1 align="center">CodeRouter</h1>

{coderouter_cli-2.9.2 → coderouter_cli-2.9.4}/coderouter/config/schemas.py

@@ -1460,6 +1460,49 @@ class LauncherOptionProfile(BaseModel):
     )
 
 
+class LauncherBenchConfig(BaseModel):
+    """The ``launcher.bench:`` block — defaults for the bench sweep feature.
+
+    設計 §4.1。デバイス構成スイープ(起動→readiness→外部ベンチ→停止→次)の
+    既定値を providers.yaml から供給する。未設定(``LauncherConfig.bench`` が
+    None)ならスイープ UI はハードコード既定を使う(完全後方互換)。すべての
+    フィールドに ``default`` があるので、既存 YAML は無改変で通過する。
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    command_template: str = Field(
+        default="llmbench run --model local-openai --runs {runs}",
+        description=(
+            "外部ベンチコマンドのテンプレ。``{port}`` ``{config}`` "
+            "``{base_url}`` ``{results_dir}`` ``{runs}`` を単純置換で展開して "
+            "argv 化する(``str.format`` ではないので JSON 波括弧で誤爆しない)。"
+        ),
+    )
+    runs: int = Field(
+        default=5,
+        ge=1,
+        le=1000,
+        description="1 構成あたりのベンチ実行回数。``{runs}`` に展開される。",
+    )
+    results_dir: str | None = Field(
+        default=None,
+        description=(
+            "llmbench の results/ ディレクトリ。相対はサーバ CWD 基準。"
+            "設定時のみスイープ結果 JSON を読み比較サマリを付与する。"
+        ),
+    )
+    readiness_timeout_s: float = Field(
+        default=300.0,
+        ge=5.0,
+        le=3600.0,
+        description=(
+            "スイープの各構成でサーバが ready になるのを待つ最大秒数。"
+            "大きな GGUF ロードを考慮した既定 5 分。"
+        ),
+    )
+
+
 class SwapModelSpec(BaseModel):
     """One entry in ``launcher.swap.models`` (docs/designs/launcher-model-swap.md).
 
@@ -1518,10 +1561,25 @@ class SwapModelSpec(BaseModel):
         le=65535,
         description=(
             "Fixed port (recommended — §10 Q2). When unset, the swap "
-            "manager picks an OS-assigned ephemeral port and retries "
-            "once on a different port if the backend fails to become "
-            "ready. Best-effort only — no strong TOCTOU guarantee "
-            "(§6.6 known-trap #4)."
+            "manager picks an OS-assigned ephemeral port and retries, "
+            "each time on a freshly picked port, up to "
+            "``LauncherSwapConfig.port_retry_attempts`` additional "
+            "times if the backend fails to become ready. Best-effort "
+            "only — no strong TOCTOU guarantee (§6.6 known-trap #4)."
+        ),
+    )
+    ttl_seconds: float | None = Field(
+        default=None,
+        ge=0.0,
+        description=(
+            "[Unreleased] Per-model override of "
+            "``LauncherSwapConfig.ttl_seconds``. ``None`` (default) = "
+            "use the global value. ``0`` = unload as soon as the last "
+            "in-flight lease for THIS model releases — same meaning as "
+            "the global field's ``0``, just scoped to one catalog "
+            "entry. Lets a large/expensive model unload sooner (or "
+            "stay resident longer) than the rest of the catalog "
+            "without changing the global default."
         ),
     )
     option_profile: str | None = Field(
@@ -1611,8 +1669,10 @@ class LauncherSwapConfig(BaseModel):
             "Seconds of no in-flight requests after which an idle swap "
             "process is auto-stopped. None = TTL disabled (runs until "
             "explicitly stopped). 0 = unload as soon as the last "
-            "in-flight lease releases. §10 Q1: one GLOBAL value for "
-            "Phase 1 — no per-model override yet."
+            "in-flight lease releases. §10 Q1: this is the GLOBAL "
+            "default; a catalog entry's ``SwapModelSpec.ttl_seconds`` "
+            "overrides it for that one model when set "
+            "([Unreleased] per-model TTL override)."
         ),
     )
     readiness_timeout_s: float = Field(
@@ -1631,6 +1691,24 @@ class LauncherSwapConfig(BaseModel):
         le=600.0,
         description="How often the TTL sweeper background task scans for idle processes.",
     )
+    port_retry_attempts: int = Field(
+        default=2,
+        ge=0,
+        le=5,
+        description=(
+            "[Unreleased] Number of ADDITIONAL spawn attempts (each on a "
+            "freshly picked ephemeral port) after the first, used only "
+            "when a catalog entry's ``port`` is unset and the previous "
+            "attempt fails to become ready. 0 = no retry. Ignored when "
+            "``port`` is set — a fixed port never retries (a second "
+            "attempt would just collide on the same port again). This "
+            "does not close the pick-then-bind TOCTOU window (see "
+            "``coderouter.launcher_swap._pick_ephemeral_port``); it only "
+            "bounds how many times the swap manager re-rolls the dice. "
+            "A fixed ``port`` remains the recommended way to eliminate "
+            "the race entirely (§10 Q2)."
+        ),
+    )
     inject_auto_router_rules: bool = Field(
         default=True,
         description=(
@@ -1828,6 +1906,13 @@ class LauncherConfig(BaseModel):
             "behavior."
         ),
     )
+    bench: LauncherBenchConfig | None = Field(
+        default=None,
+        description=(
+            "デバイス構成ベンチスイープ(設計 §4)の既定値。None(既定)なら "
+            "スイープ UI はハードコード既定を使う=完全後方互換。"
+        ),
+    )
 
     @model_validator(mode="after")
     def _check_auto_restart_backoff_ordered(self) -> LauncherConfig: