coderouter-cli 2.4.0__tar.gz → 2.5.0__tar.gz

{coderouter_cli-2.4.0 → coderouter_cli-2.5.0}/CHANGELOG.md

@@ -6,6 +6,54 @@ versioning follows [SemVer](https://semver.org/).
 
 ---
 
+## [v2.5.0] — 2026-05-22 (Launcher — llama.cpp / vllm GUI)
+
+Browser-based process manager for local inference backends, integrated
+into the existing CodeRouter web UI at `/launcher`.
+
+### Added
+
+- **`coderouter/ingress/launcher_routes.py`**: New route module providing
+  the Launcher UI and its backing API.
+
+  - `GET /launcher` — Single-page HTML UI (Tailwind CDN + inline JS,
+    same dark-theme aesthetic as `/dashboard`).
+  - `GET /api/launcher/models` — Recursively scans `launcher.model_dirs`
+    and returns discovered model files with name, path, size (GB), and
+    extension.
+  - `GET /api/launcher/option-profiles` — Returns named option presets
+    from `providers.yaml` keyed by backend (`llama.cpp`, `vllm`).
+  - `GET /api/launcher/processes` — Lists all managed processes.
+  - `POST /api/launcher/start` — Starts a backend process. Accepts name,
+    backend, model_path, port, options dict, and extra_args free-text.
+  - `POST /api/launcher/stop/{id}` — SIGTERM → SIGKILL (5 s timeout).
+  - `DELETE /api/launcher/processes/{id}` — Removes a stopped process.
+  - `GET /api/launcher/logs/{id}` — Returns last N lines from the
+    process's 200-line stdout/stderr ring buffer.
+
+- **`LauncherOptionProfile` / `LauncherConfig`** (`config/schemas.py`):
+  New Pydantic models for the `launcher:` block in `providers.yaml`.
+  Adding new CLI flags requires only a YAML edit — no code change.
+
+- **`launcher_profiles.yaml.example`**: Template with 7 llama.cpp
+  presets and 7 vllm presets. For GitHub distribution and community
+  profile contributions.
+
+- **`/dashboard` header**: Added a "Launcher" navigation link.
+
+### Design notes
+
+- **YAML-driven**: option profiles live entirely in `providers.yaml`.
+  No code changes needed to add new backend flags.
+- **Multi-process**: each launched process gets a UUID-based ID and is
+  tracked independently. llama.cpp and vllm can run side by side.
+- **Zero new dependencies**: uses `asyncio.create_subprocess_exec`
+  (stdlib only). The 5-dep invariant is maintained.
+- **In-memory registry**: does not persist across CodeRouter restarts
+  (intentional — avoids zombie GPU allocations on restart).
+
+---
+
 ## [v2.4.0] — 2026-05-15 (Goal-session awareness — P1-4/5/6)
 
 Stable release following v2.3.0a4. Promotes the Plugin SDK to stable,

{coderouter_cli-2.4.0 → coderouter_cli-2.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: coderouter-cli
-Version: 2.4.0
+Version: 2.5.0
 Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
 Project-URL: Homepage, https://github.com/zephel01/CodeRouter
 Project-URL: Repository, https://github.com/zephel01/CodeRouter
@@ -141,7 +141,7 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 | ガード | 何から守るか |
 |---|---|
 | **Context Budget** | メッセージが溜まりすぎて context window 溢れ → 自動 trim |
-| **Drift Detection** | モデルの応答品質が徐々に劣化 → 別 provider に切替 or KV cache flush |
+| **Drift Detection** | モデルの応答品質が徐々に劣化 → 別 provider に切替 or KV cache flush (6 シグナル、`goal_mode` で目標達成停滞も検知) |
 | **Self-healing** | backend が落ちた → 自動除外 + restart + 回復 probe で自動復帰 |
 | **Tool Loop Guard** | 同じツールを無限に呼び続ける → 検知して停止 |
 | **Memory Pressure** | GPU メモリ不足を検知 → 軽量モデルに切替 |
@@ -154,9 +154,40 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 | **`coderouter doctor`** | プロバイダの問題を 6 プローブで即診断 + 修正パッチ出力 |
 | **`/dashboard`** | ブラウザで今何が起きてるかリアルタイム確認 |
 | **`coderouter audit`** | guard 発火履歴を検索 |
-| **`coderouter replay`** | provider 切替の効果を統計比較 (A/B 分析) |
+| **`coderouter replay`** | provider 切替の効果を統計比較 (A/B 分析) / `--suggest-rules` でルール最適化提案 |
 | **Continuous Probe** | idle 時も定期的に backend を監視 |
 
+### Launcher — llama.cpp / vllm 起動 UI
+
+`http://localhost:8088/launcher` で開けるブラウザ UI。llama.cpp や vllm を GUI で起動・管理できます。
+
+| 機能 | 詳細 |
+|---|---|
+| **モデルスキャン** | `model_dirs` に指定したフォルダを再帰スキャンして `.gguf` / `.safetensors` をリスト化 |
+| **オプションプロファイル** | `providers.yaml` に名前付きプリセットを定義 → ドロップダウンで選択するだけ |
+| **複数プロセス管理** | llama.cpp と vllm を同時に起動し、ポートごとに独立管理 |
+| **ログビューア** | 各プロセスの stdout/stderr をブラウザ内でリアルタイム確認 |
+
+```yaml
+# providers.yaml に追記するだけで有効になる
+launcher:
+  model_dirs:
+    - ~/models
+  option_profiles:
+    llama.cpp:
+      - name: "GPU フル活用"
+        args:
+          "-ngl": 99
+          "--ctx-size": 4096
+    vllm:
+      - name: "標準"
+        args:
+          "--dtype": "auto"
+          "--max-model-len": 4096
+```
+
+詳細 → [Launcher ガイド](./docs/launcher.md)
+
 ---
 
 ## 設定例 (最小)
@@ -193,6 +224,7 @@ providers:
 | すぐ動かす | [Quickstart](./docs/quickstart.md) |
 | 使いこなす | [利用ガイド](./docs/usage-guide.md) |
 | 無料で回す | [無料枠ガイド](./docs/free-tier-guide.md) |
+| llama.cpp / vllm を GUI で起動 | [Launcher ガイド](./docs/launcher.md) |
 | 詰まった | [トラブルシューティング](./docs/troubleshooting.md) |
 | 設計を知りたい | [アーキテクチャ詳細](./docs/architecture.md) |
 | 全リリース履歴 | [CHANGELOG](./CHANGELOG.md) |

{coderouter_cli-2.4.0 → coderouter_cli-2.5.0}/README.en.md

@@ -100,7 +100,7 @@ Full decision matrix → [Do I need CodeRouter?](./docs/when-do-i-need-coderoute
 | Guard | What it protects against |
 |---|---|
 | **Context Budget** | Messages piling up → context window overflow. Auto-trim at 90% |
-| **Drift Detection** | Model quality degrading over time → switch provider or flush KV cache |
+| **Drift Detection** | Model quality degrading over time → switch provider or flush KV cache (6 signals incl. `goal_progress_stall`; `goal_mode` for tighter thresholds) |
 | **Self-healing** | Backend crashes → auto-exclude + restart + recovery probe → auto-restore |
 | **Tool Loop Guard** | Agent calling the same tool forever → detect and break |
 | **Memory Pressure** | GPU running out of VRAM → switch to lighter model |
@@ -113,9 +113,40 @@ Full decision matrix → [Do I need CodeRouter?](./docs/when-do-i-need-coderoute
 | **`coderouter doctor`** | 6-probe diagnosis of provider issues + copy-paste YAML patches |
 | **`/dashboard`** | Real-time browser view of what's happening |
 | **`coderouter audit`** | Search guard activation history |
-| **`coderouter replay`** | Compare providers statistically (A/B analysis) |
+| **`coderouter replay`** | Compare providers statistically (A/B analysis) / `--suggest-rules` for automated rule suggestions |
 | **Continuous Probe** | Background health monitoring even during idle |
 
+### Launcher — llama.cpp / vllm GUI
+
+Browser UI at `http://localhost:8088/launcher` for starting and managing local inference backends.
+
+| Feature | Detail |
+|---|---|
+| **Model scanner** | Recursively scans `model_dirs` for `.gguf` / `.safetensors` and lists them |
+| **Option profiles** | Name your flag presets in `providers.yaml` — select from a dropdown, no CLI needed |
+| **Multi-process** | Run llama.cpp and vllm side by side on different ports |
+| **Log viewer** | stdout/stderr of each process shown live in the browser |
+
+```yaml
+# Add to providers.yaml — no code changes needed
+launcher:
+  model_dirs:
+    - ~/models
+  option_profiles:
+    llama.cpp:
+      - name: "Full GPU"
+        args:
+          "-ngl": 99
+          "--ctx-size": 4096
+    vllm:
+      - name: "Standard"
+        args:
+          "--dtype": "auto"
+          "--max-model-len": 4096
+```
+
+Details → [Launcher guide](./docs/launcher.md)
+
 ---
 
 ## Minimal Config
@@ -152,6 +183,7 @@ More detail → [Usage guide](./docs/usage-guide.en.md) · [Architecture](./docs
 | Get running fast | [Quickstart](./docs/quickstart.en.md) |
 | Use it well | [Usage guide](./docs/usage-guide.en.md) |
 | Run for free | [Free-tier guide](./docs/free-tier-guide.en.md) |
+| Launch llama.cpp / vllm via GUI | [Launcher guide](./docs/launcher.md) |
 | Stuck? | [Troubleshooting](./docs/troubleshooting.en.md) |
 | Understand the design | [Architecture](./docs/architecture.md) |
 | Full release history | [CHANGELOG](./CHANGELOG.md) |

{coderouter_cli-2.4.0 → coderouter_cli-2.5.0}/README.md

@@ -100,7 +100,7 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 | ガード | 何から守るか |
 |---|---|
 | **Context Budget** | メッセージが溜まりすぎて context window 溢れ → 自動 trim |
-| **Drift Detection** | モデルの応答品質が徐々に劣化 → 別 provider に切替 or KV cache flush |
+| **Drift Detection** | モデルの応答品質が徐々に劣化 → 別 provider に切替 or KV cache flush (6 シグナル、`goal_mode` で目標達成停滞も検知) |
 | **Self-healing** | backend が落ちた → 自動除外 + restart + 回復 probe で自動復帰 |
 | **Tool Loop Guard** | 同じツールを無限に呼び続ける → 検知して停止 |
 | **Memory Pressure** | GPU メモリ不足を検知 → 軽量モデルに切替 |
@@ -113,9 +113,40 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
 | **`coderouter doctor`** | プロバイダの問題を 6 プローブで即診断 + 修正パッチ出力 |
 | **`/dashboard`** | ブラウザで今何が起きてるかリアルタイム確認 |
 | **`coderouter audit`** | guard 発火履歴を検索 |
-| **`coderouter replay`** | provider 切替の効果を統計比較 (A/B 分析) |
+| **`coderouter replay`** | provider 切替の効果を統計比較 (A/B 分析) / `--suggest-rules` でルール最適化提案 |
 | **Continuous Probe** | idle 時も定期的に backend を監視 |
 
+### Launcher — llama.cpp / vllm 起動 UI
+
+`http://localhost:8088/launcher` で開けるブラウザ UI。llama.cpp や vllm を GUI で起動・管理できます。
+
+| 機能 | 詳細 |
+|---|---|
+| **モデルスキャン** | `model_dirs` に指定したフォルダを再帰スキャンして `.gguf` / `.safetensors` をリスト化 |
+| **オプションプロファイル** | `providers.yaml` に名前付きプリセットを定義 → ドロップダウンで選択するだけ |
+| **複数プロセス管理** | llama.cpp と vllm を同時に起動し、ポートごとに独立管理 |
+| **ログビューア** | 各プロセスの stdout/stderr をブラウザ内でリアルタイム確認 |
+
+```yaml
+# providers.yaml に追記するだけで有効になる
+launcher:
+  model_dirs:
+    - ~/models
+  option_profiles:
+    llama.cpp:
+      - name: "GPU フル活用"
+        args:
+          "-ngl": 99
+          "--ctx-size": 4096
+    vllm:
+      - name: "標準"
+        args:
+          "--dtype": "auto"
+          "--max-model-len": 4096
+```
+
+詳細 → [Launcher ガイド](./docs/launcher.md)
+
 ---
 
 ## 設定例 (最小)
@@ -152,6 +183,7 @@ providers:
 | すぐ動かす | [Quickstart](./docs/quickstart.md) |
 | 使いこなす | [利用ガイド](./docs/usage-guide.md) |
 | 無料で回す | [無料枠ガイド](./docs/free-tier-guide.md) |
+| llama.cpp / vllm を GUI で起動 | [Launcher ガイド](./docs/launcher.md) |
 | 詰まった | [トラブルシューティング](./docs/troubleshooting.md) |
 | 設計を知りたい | [アーキテクチャ詳細](./docs/architecture.md) |
 | 全リリース履歴 | [CHANGELOG](./CHANGELOG.md) |

{coderouter_cli-2.4.0 → coderouter_cli-2.5.0}/coderouter/config/schemas.py

@@ -874,6 +874,130 @@ class AutoRouterConfig(BaseModel):
     )
 
 
+class LauncherBackendConfig(BaseModel):
+    """Per-backend binary path configuration for the Launcher.
+
+    When ``binary`` is unset, the Launcher falls back to the default
+    executable name (``llama-server`` for llama.cpp, ``python`` for vllm)
+    and relies on ``$PATH`` resolution — which works when the tool is
+    globally installed.  Set ``binary`` when:
+
+    - llama.cpp was built from source (e.g. ``~/llama.cpp/build/bin/llama-server``)
+    - vllm lives in a virtualenv (e.g. ``~/.venv/bin/python``)
+    - Multiple builds coexist and you want to pin a specific one
+
+    Tilde (``~``) and environment variables are expanded at launch time.
+
+    Example::
+
+        backends:
+          llama.cpp:
+            binary: ~/llama.cpp/build/bin/llama-server
+          vllm:
+            binary: ~/.venv/bin/python
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    binary: str | None = Field(
+        default=None,
+        description=(
+            "Absolute or ``~``-relative path to the backend executable. "
+            "llama.cpp default: ``llama-server`` (PATH). "
+            "vllm default: ``python`` (PATH). "
+            "Expanded at launch time."
+        ),
+    )
+
+
+class LauncherOptionProfile(BaseModel):
+    """One named option preset for a launcher backend (e.g. llama.cpp / vllm).
+
+    ``args`` maps CLI flag strings to their values.  A bool value of
+    ``True`` means "include the flag without a value" (e.g. ``--no-mmap``);
+    ``False`` means "omit the flag entirely".  All other value types are
+    converted to strings and appended as ``--flag value`` pairs.
+
+    Example::
+
+        name: "GPU速度重視"
+        args:
+          "-ngl": 99
+          "--ctx-size": 4096
+          "--no-mmap": false
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    name: str = Field(..., description="Display name shown in the Launcher UI dropdown.")
+    args: dict[str, str | int | float | bool] = Field(
+        default_factory=dict,
+        description=(
+            "CLI flag → value mapping. "
+            "bool True = flag only (no value). "
+            "bool False = omit flag. "
+            "All other types are stringified and passed as '--flag value'."
+        ),
+    )
+
+
+class LauncherConfig(BaseModel):
+    """The ``launcher:`` block in providers.yaml.
+
+    Controls the Launcher UI available at ``/launcher``.
+
+    Example::
+
+        launcher:
+          model_dirs:
+            - ~/models
+            - /data/gguf
+          option_profiles:
+            llama.cpp:
+              - name: "GPU速度重視"
+                args:
+                  "-ngl": 99
+                  "--ctx-size": 4096
+            vllm:
+              - name: "標準"
+                args:
+                  "--dtype": "auto"
+                  "--max-model-len": 4096
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    model_dirs: list[str] = Field(
+        default_factory=list,
+        description=(
+            "Directories to scan for model files "
+            "(.gguf, .safetensors, .bin, .pt, .ggml). "
+            "Paths are expanded (~ and env vars) at scan time, not at load. "
+            "Non-existent paths are silently skipped."
+        ),
+    )
+    backends: dict[str, LauncherBackendConfig] = Field(
+        default_factory=dict,
+        description=(
+            "Per-backend binary path overrides. "
+            "Keys are backend names ('llama.cpp', 'vllm'). "
+            "When a key is absent, the default executable is used "
+            "('llama-server' / 'python') and resolved via PATH. "
+            "Useful when running a from-source build or a venv-specific binary."
+        ),
+    )
+    option_profiles: dict[str, list[LauncherOptionProfile]] = Field(
+        default_factory=dict,
+        description=(
+            "Named option presets per backend. "
+            "Keys should be backend names: 'llama.cpp', 'vllm'. "
+            "Each key maps to an ordered list of named presets. "
+            "A free-form 'extra args' field is always available in the UI "
+            "for one-off overrides without touching this config."
+        ),
+    )
+
+
 class PluginsConfig(BaseModel):
     """The ``plugins:`` block in providers.yaml (v2.3.0).
 
@@ -1082,6 +1206,17 @@ class CodeRouterConfig(BaseModel):
             "plugins (zero-cost, backward-compatible default)."
         ),
     )
+    launcher: LauncherConfig | None = Field(
+        default=None,
+        description=(
+            "Launcher configuration for the /launcher UI. "
+            "Defines model_dirs to scan and option_profiles per backend "
+            "('llama.cpp', 'vllm'). "
+            "Unset (None) = Launcher UI shows empty model list and no profiles. "
+            "The Launcher UI itself is always available at /launcher "
+            "regardless of this setting."
+        ),
+    )
 
     @model_validator(mode="after")
     def _check_default_profile_exists(self) -> CodeRouterConfig:

{coderouter_cli-2.4.0 → coderouter_cli-2.5.0}/coderouter/ingress/app.py

@@ -13,6 +13,7 @@ from coderouter import __version__
 from coderouter.config import load_config
 from coderouter.ingress.anthropic_routes import router as anthropic_router
 from coderouter.ingress.dashboard_routes import router as dashboard_router
+from coderouter.ingress.launcher_routes import router as launcher_router
 from coderouter.ingress.metrics_routes import router as metrics_router
 from coderouter.ingress.openai_routes import router as openai_router
 from coderouter.logging import configure_logging, get_logger
@@ -178,6 +179,12 @@ def create_app(config_path: str | None = None) -> FastAPI:
             with contextlib.suppress(Exception):
                 await probe_task
 
+        # Launcher: stop child llama.cpp / vllm processes so they don't orphan.
+        from coderouter.ingress.launcher_routes import shutdown_launcher
+
+        with contextlib.suppress(Exception):
+            await shutdown_launcher(app)
+
         # v2.0-J: graceful shutdown of recovery probe tasks.
         with contextlib.suppress(Exception):
             await engine.shutdown_recovery_probes()
@@ -259,6 +266,10 @@ def create_app(config_path: str | None = None) -> FastAPI:
     # Same root-level mount as /metrics.json — the dashboard is a UI
     # concern and doesn't belong under the /v1 API surface.
     app.include_router(dashboard_router, tags=["dashboard"])
+    # Launcher UI + process management API.
+    # /launcher       → single-page HTML UI
+    # /api/launcher/* → model scan, process start/stop/logs
+    app.include_router(launcher_router, tags=["launcher"])
 
     return app
 

{coderouter_cli-2.4.0 → coderouter_cli-2.5.0}/coderouter/ingress/dashboard_routes.py

@@ -76,6 +76,7 @@ _DASHBOARD_HTML = r"""<!doctype html>
   <header class="border-b border-slate-800 px-6 py-3">
     <div class="max-w-7xl mx-auto flex flex-wrap items-center gap-x-6 gap-y-2 text-sm">
       <span class="text-lg font-semibold tracking-tight">CodeRouter</span>
+      <a href="/launcher" class="text-slate-400 hover:text-slate-200 transition-colors text-sm">Launcher</a>
       <span class="text-slate-400">profile: <span data-bind="profile" class="text-slate-100 font-mono">—</span></span>
       <span class="text-slate-400">uptime: <span data-bind="uptime" class="text-slate-100 font-mono tabnum">—</span></span>
       <span class="text-slate-400">requests: <span data-bind="requests_total" class="text-slate-100 font-mono tabnum">0</span></span>