coderouter-cli 2.9.3__tar.gz → 2.9.4__tar.gz

{coderouter_cli-2.9.3 → coderouter_cli-2.9.4}/.gitignore

@@ -7,6 +7,7 @@
 !.env.example
 secrets.env
 ~/.coderouter/
+CLAUDE.md
 
 # ============================================================
 # Python

{coderouter_cli-2.9.3 → coderouter_cli-2.9.4}/CHANGELOG.md

@@ -9,6 +9,103 @@ are kept verbatim where the Japanese text itself is the subject).
 
 ---
 
+## [v2.9.4] — 2026-07-19 (launcher device selection & bench sweep, cache-stable system prompt)
+
+### Added
+
+- **llama.cpp device selection (`--device` / `--tensor-split`) in the
+  Launcher.** Both the desktop GUI and the Web edition can now detect
+  available `llama-server` devices (via `--list-devices`), display their
+  VRAM, and let the operator pick which one(s) to offload to instead of
+  always relying on llama.cpp's own default placement. New shared module
+  `coderouter/launcher_devices.py` (dataclasses + pure functions only, no
+  pydantic, so the standalone GUI can import it without the `coderouter`
+  package) owns detection/caching (`detect_llama_devices`,
+  `parse_list_devices`, a 60s TTL cache keyed by binary path),
+  CLI-fragment building (`DeviceSelection.to_cli_args` — empty selection
+  emits nothing, preserving today's launch command byte-for-byte),
+  VRAM-ratio tensor-split suggestion (`suggest_tensor_split`), and backend
+  grouping (`backend_of` / `group_by_backend` / `selectable_devices`) so a
+  GPU listed twice under different backends (e.g. `CUDA0` and `Vulkan1`
+  for the same physical card) is never double-counted and cross-backend
+  configs are never auto-generated. Devices reporting `0 MiB` (e.g. macOS
+  `BLAS: Accelerate`) are still shown for information but excluded from
+  selection/suggestion. `StartRequest` gained `device_ids` / `tensor_split`
+  (both default empty — existing Web clients are unaffected), and
+  `GET /api/launcher/devices` (`?backend=`, `?refresh=1`) returns the
+  probe plus a per-backend `suggested_tensor_split` and `auto_configs`.
+  `launcher_gui.py`'s LAUNCH form gained a device checklist (🔍 Detect,
+  falling back to a manual comma-separated entry on detection failure)
+  and a tensor-split field that auto-hides when one or fewer devices are
+  detected. See `tests/test_launcher_devices.py`,
+  `tests/test_launcher_devices_detect.py`,
+  `tests/test_launcher_devices_routes.py`,
+  `tests/test_launcher_gui_devices.py`, and the new "Device selection"
+  section in `docs/backends/launcher.md` / `launcher.en.md`.
+- **Bench sweep: automated device-configuration benchmarking against an
+  external `llmbench`.** Both Launcher editions can now drive a list of
+  device configurations (e.g. `CUDA0` alone / `CUDA1` alone / a
+  multi-GPU split) through start → wait for readiness → run an external
+  bench command → stop → advance to the next configuration, then compare
+  the results. `coderouter/launcher_devices.py` supplies the shared
+  `SweepPlan` / `SweepStep` / `SweepState` state machine,
+  `build_auto_sweep_configs` (one config per selectable device plus one
+  per multi-device backend group), `render_bench_command` (template
+  placeholders `{port}` `{config}` `{base_url}` `{results_dir}` `{runs}`,
+  expanded via plain string substitution rather than `str.format` so JSON
+  braces in the command aren't misinterpreted, then `shlex`-split with
+  `posix=False` on Windows to keep backslash paths intact), and
+  `load_latest_results` / `summarize_results` (best-effort, alias-tolerant
+  extraction of `tokens_per_sec` / `ttft_ms` / `latency_ms` from the
+  newest `llmbench` results JSON). New `LauncherBenchConfig`
+  (`command_template` / `runs` / `results_dir` / `readiness_timeout_s`,
+  read from an optional `launcher.bench:` block — omitted entirely by
+  default, so existing `providers.yaml` files are unaffected) supplies
+  the sweep's defaults. The Web edition gained
+  `POST /api/launcher/sweep/start` (token-gated; 409 if a sweep is
+  already running, 400 for empty `configs` or a busy/unavailable port),
+  `GET /api/launcher/sweep/status`, `POST /api/launcher/sweep/abort`
+  (token-gated), and `GET /api/launcher/sweep/logs`, backed by a new
+  `_SweepRunner` that reuses the existing `spawn_process` /
+  `stop_process` / `proc.ready` readiness primitives. `launcher_gui.py`
+  gained a "📊 Bench sweep" button opening a separate `SweepWindow` with
+  its own `_SweepWorker` thread, reusing `poll_until_ready` for readiness
+  and driving the same external bench subprocess model. See
+  `tests/test_launcher_config_bench.py` and the new "Bench sweep" section
+  in `docs/backends/launcher.md` / `launcher.en.md`.
+
+### Fixed
+
+- **Mid-conversation `role:"system"` messages are no longer hoisted into
+  the top-level `system` field.** (#75, contributed by @firelzrd.)
+  Claude Code CLI ≥ 2.1.154 emits volatile system reminders as
+  `role:"system"` messages *inside* `messages[]`; hoisting them meant the
+  serialized prompt's head changed every turn, so llama.cpp / LM Studio
+  style prefix-matching KV caches were invalidated wholesale (a captured
+  real session measured only 11.8 % reusable prefix between consecutive
+  requests). `normalize_message_roles()` now splits by position: leading
+  system messages (before any emitted turn) are hoisted as before — the
+  legitimate OpenAI-style system prompt — while mid-conversation ones are
+  coerced to `user` in place, keeping `messages` append-only and the
+  prompt head stable as the conversation grows. See
+  `tests/test_role_normalization.py`
+  (`test_leading_system_is_hoisted_but_mid_conversation_is_not`,
+  `test_system_prompt_is_stable_as_conversation_grows`).
+
+### Security
+
+- **Dependency bumps for two fresh advisories** (lockfile-only; both
+  verified with a clean `uv sync --frozen` + full test suite, 1905
+  passed): `click` 8.3.2 → 8.4.2 (PYSEC-2026-2132, fix ≥ 8.3.3;
+  transitive via uvicorn) and `json-repair` 0.59.10 → 0.61.5
+  (GHSA-xf7x-x43h-rpqh, CVSS 7.5, fix ≥ 0.60.1). The `json-repair`
+  floor in `pyproject.toml` was raised from `>=0.30` to `>=0.60.1` so
+  fresh installs can never resolve a vulnerable version; `uv.lock` also
+  records the new lockfile revision 3 format (upload-time metadata)
+  emitted by current uv.
+
+---
+
 ## [v2.9.3] — 2026-07-12 (GUI/Web parity, swap polish, launcher UI fixes)
 
 ### Added

{coderouter_cli-2.9.3 → coderouter_cli-2.9.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: coderouter-cli
-Version: 2.9.3
+Version: 2.9.4
 Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
 Project-URL: Homepage, https://github.com/zephel01/CodeRouter
 Project-URL: Repository, https://github.com/zephel01/CodeRouter
@@ -40,7 +40,7 @@ Requires-Dist: types-pyyaml>=6.0.12; extra == 'dev'
 Provides-Extra: doctor
 Requires-Dist: ruamel-yaml>=0.18.6; extra == 'doctor'
 Provides-Extra: repair
-Requires-Dist: json-repair>=0.30; extra == 'repair'
+Requires-Dist: json-repair>=0.60.1; extra == 'repair'
 Description-Content-Type: text/markdown
 
 <h1 align="center">CodeRouter</h1>

{coderouter_cli-2.9.3 → coderouter_cli-2.9.4}/coderouter/config/schemas.py

@@ -1460,6 +1460,49 @@ class LauncherOptionProfile(BaseModel):
     )
 
 
+class LauncherBenchConfig(BaseModel):
+    """The ``launcher.bench:`` block — defaults for the bench sweep feature.
+
+    設計 §4.1。デバイス構成スイープ(起動→readiness→外部ベンチ→停止→次)の
+    既定値を providers.yaml から供給する。未設定(``LauncherConfig.bench`` が
+    None)ならスイープ UI はハードコード既定を使う(完全後方互換)。すべての
+    フィールドに ``default`` があるので、既存 YAML は無改変で通過する。
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    command_template: str = Field(
+        default="llmbench run --model local-openai --runs {runs}",
+        description=(
+            "外部ベンチコマンドのテンプレ。``{port}`` ``{config}`` "
+            "``{base_url}`` ``{results_dir}`` ``{runs}`` を単純置換で展開して "
+            "argv 化する(``str.format`` ではないので JSON 波括弧で誤爆しない)。"
+        ),
+    )
+    runs: int = Field(
+        default=5,
+        ge=1,
+        le=1000,
+        description="1 構成あたりのベンチ実行回数。``{runs}`` に展開される。",
+    )
+    results_dir: str | None = Field(
+        default=None,
+        description=(
+            "llmbench の results/ ディレクトリ。相対はサーバ CWD 基準。"
+            "設定時のみスイープ結果 JSON を読み比較サマリを付与する。"
+        ),
+    )
+    readiness_timeout_s: float = Field(
+        default=300.0,
+        ge=5.0,
+        le=3600.0,
+        description=(
+            "スイープの各構成でサーバが ready になるのを待つ最大秒数。"
+            "大きな GGUF ロードを考慮した既定 5 分。"
+        ),
+    )
+
+
 class SwapModelSpec(BaseModel):
     """One entry in ``launcher.swap.models`` (docs/designs/launcher-model-swap.md).
 
@@ -1863,6 +1906,13 @@ class LauncherConfig(BaseModel):
             "behavior."
         ),
     )
+    bench: LauncherBenchConfig | None = Field(
+        default=None,
+        description=(
+            "デバイス構成ベンチスイープ(設計 §4)の既定値。None(既定)なら "
+            "スイープ UI はハードコード既定を使う=完全後方互換。"
+        ),
+    )
 
     @model_validator(mode="after")
     def _check_auto_restart_backoff_ordered(self) -> LauncherConfig: