llm-switcher 1.2.10 → 1.2.12

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog — LLM Switcher
 
+## Release 1.2.12
+
+- **Update now:** The update notice on the dashboard has an **Update now** button. The gateway installs the release, and a new gateway process starts on the new code. Then the dashboard reloads. `switch update` does the same from a terminal.
+- **Safe for a git checkout:** A git checkout updates with `git pull --ff-only`. If the checkout has local changes, or commits that are not on the upstream branch, the update stops and changes nothing.
+- **Open requests:** Open requests on the gateway finish on the old code. The Codex interceptor stops and the new gateway starts it again, so a Codex request that is open at that moment fails one time. The old process stays as the parent of the new process, so a service manager keeps the process that it started.
+- **Update at logon:** `switch service install` adds `--autoupdate` to the service. The gateway then installs the newest release before it starts. `--no-autoupdate` installs the service without the flag. Run `switch service install` again to add the flag to a service that is already installed. `switch port` keeps the choice of the installed service.
+- **macOS and Linux:** launchd and `systemd --user` start the gateway with a short `PATH`, and an npm from Homebrew or nvm is not on it. The update now runs npm with the directory of the running `node` first on `PATH`.
+
+## Release 1.2.11
+
+- **Test connection on a Bifrost model:** The test sent a plain `ping` request, and Anthropic answers that with a fake 429 for a Claude Code account. Now, when intact names `bifrost_ua` for the model, the test reports that the key works and the model is served, and it sends no message.
+
 ## Release 1.2.10
 
 - **Bifrost in the log:** The log line and the request inspector showed `anthropic -> openai-chat` for a Bifrost request, because the gateway logged the profile's `outFormat` before it checked Bifrost. Now both show `bifrost`.

package/README.md

@@ -424,12 +424,13 @@ switch ui                      # Open the Web UI dashboard in your browser
 switch status                  # Display status for all active CLI targets
 switch version                 # Show the version and tell you when npm has a newer one
 switch doctor                  # Audit environment, settings & routing
+switch update                  # Install the newest release and restart the gateway on it
 switch on [profile]            # Start the gateway and activate a profile
 switch <profile>               # Activate a profile for both tools
 switch claude <profile>        # Set the active profile for Claude Code only
 switch codex <profile>         # Set the active profile for Codex only
 switch port <number>           # Change the gateway port (restarts it if running)
-switch service install         # Install OS background autostart service (Windows / macOS / Linux)
+switch service install         # Install OS background autostart service (Windows / macOS / Linux); updates at each logon
 switch service uninstall       # Remove background autostart service
 switch shim install            # Route new Claude and Codex sessions through the gateway
 switch shim status             # Verify shims + detect running sessions that bypass the gateway
@@ -453,6 +454,19 @@ The service runs without your shell. `switch service install` therefore copies `
 
 On Linux the unit is a systemd *user* service: it starts when you log in. To start it at boot without a login (a server reached over SSH), run `loginctl enable-linger $USER` once. The unit records the absolute path of the current `node`; if you manage Node with nvm and remove that version, run `switch service install` again.
 
+### Updates
+
+When npm has a newer release, the dashboard shows a notice. Click **Update now** in this notice. The gateway installs the release and starts the new code. Then the dashboard reloads. `switch update` does the same from a terminal.
+
+The gateway installs a release in one of two ways:
+
+- A git checkout runs `git pull --ff-only`. If the checkout has local changes, or commits that are not on the upstream branch, the update stops and changes nothing.
+- An npm install runs `npm install -g llm-switcher@<latest>`.
+
+Requests that are open on the gateway when the update starts finish on the old code. New requests go to the new code. The Codex interceptor stops, and the new gateway starts it again. Thus a Codex request that is open at that moment fails one time. The old process stays as the parent of the new process until the gateway stops. Thus a service manager keeps the process that it started.
+
+`switch service install` adds `--autoupdate` to the service command line. With this flag, the gateway installs the newest release before it starts, at each logon. To install the service without this flag, run `switch service install --no-autoupdate`. If a service from an older version is installed, run `switch service install` again to add the flag.
+
 ### Resumed sessions & the shim (important)
 
 `switch on` writes `env-claude.*` and `env-codex.*` and writes nothing into

package/README.vi.md

@@ -421,12 +421,13 @@ switch ui                      # Mở giao diện Web UI trên trình duyệt
 switch status                  # Xem trạng thái kích hoạt của tất cả các CLI
 switch version                 # Xem version đang chạy và báo khi npm có bản mới
 switch doctor                  # Quét & thanh tra toàn bộ môi trường, settings và định tuyến
+switch update                  # Cài bản mới nhất và khởi động lại gateway trên bản đó
 switch on [profile]            # Khởi động gateway và kích hoạt một profile
 switch <profile>               # Kích hoạt một profile cho cả hai công cụ
 switch claude <profile>        # Đặt profile kích hoạt riêng cho Claude Code
 switch codex <profile>         # Đặt profile kích hoạt riêng cho Codex
 switch port <number>           # Đổi cổng gateway (tự restart nếu đang chạy)
-switch service install         # Cài đặt gateway thành service chạy ngầm tự bật cùng máy
+switch service install         # Cài đặt gateway thành service chạy ngầm tự bật cùng máy; tự update mỗi lần đăng nhập
 switch service uninstall       # Gỡ bỏ service chạy ngầm
 switch shim install            # Route phiên Claude và Codex mới qua gateway
 switch shim status             # Kiểm tra shim + phát hiện phiên đang chạy ngoài gateway
@@ -450,6 +451,20 @@ các route đầu vào mà chúng đại diện đã bị gỡ.
 Service không chạy trong shell của bạn. Vì vậy `switch service install` chép `LLM_SWITCHER_HOME`, `CLAUDE_CONFIG_DIR`, `LLM_SWITCHER_CONFIG`, `LLM_SWITCHER_STATE_DIR` và `LLM_SWITCHER_BLINDFOLD_CERTS` vào unit systemd hoặc plist launchd khi các biến này có giá trị. Task Windows không mang được các biến này; hãy đặt chúng thành biến môi trường User. Nếu file định nghĩa đã cài khác file mới (ví dụ đã sửa tay), file cũ được giữ lại thành `<file>.bak`. Trên Windows, task được tạo từ file XML, nên đường dẫn có dấu cách không cần quote thêm và task không bị giới hạn thời gian chạy. Đường Windows này chưa được test trên Windows.
 
 Trên Linux, unit là service systemd *của user*: nó khởi động khi bạn đăng nhập. Muốn nó chạy ngay khi máy boot mà không cần đăng nhập (ví dụ server truy cập qua SSH), chạy một lần `loginctl enable-linger $USER`. Unit ghi đường dẫn tuyệt đối của `node` hiện tại; nếu bạn quản lý Node bằng nvm và gỡ phiên bản đó, hãy chạy lại `switch service install`.
+
+### Cập nhật
+
+Khi npm có bản mới, dashboard hiện một thông báo. Bấm **Update now** trong thông báo này. Gateway cài bản mới và chạy code mới. Sau đó dashboard tự tải lại. `switch update` làm việc tương tự từ terminal.
+
+Gateway cài bản mới theo một trong hai cách:
+
+- Bản git checkout chạy `git pull --ff-only`. Nếu checkout có thay đổi cục bộ, hoặc có commit không nằm trên nhánh upstream, việc cập nhật dừng lại và không đổi gì.
+- Bản cài bằng npm chạy `npm install -g llm-switcher@<latest>`.
+
+Các request đang chạy trên gateway lúc cập nhật sẽ chạy xong trên code cũ. Request mới đi vào code mới. Interceptor của Codex dừng lại, và gateway mới khởi động nó lại. Vì vậy một request Codex đang chạy đúng lúc đó sẽ lỗi một lần. Tiến trình cũ ở lại làm tiến trình cha của tiến trình mới cho tới khi gateway dừng. Nhờ vậy service manager vẫn giữ đúng tiến trình mà nó đã khởi động.
+
+`switch service install` thêm `--autoupdate` vào dòng lệnh của service. Với cờ này, gateway cài bản mới nhất trước khi khởi động, ở mỗi lần đăng nhập. Muốn cài service không có cờ này, chạy `switch service install --no-autoupdate`. Nếu service đã cài từ phiên bản cũ, chạy lại `switch service install` để thêm cờ.
+
 ### Phiên mở lại (`--resume`) và cơ chế shim — quan trọng
 
 `switch on` ghi `env-claude.*` và `env-codex.*`, và **không ghi gì** vào

package/docs/TOKEN-OPTIMIZER-INTEROP.md

@@ -1,110 +1,110 @@
-# Token Optimizer Interoperability & Failure Mode Report
-
-**How LLM Switcher acts as the protective outermost edge gateway for aggressive prompt/token optimizers (Headroom, RTK, Ponytail).**
-
----
-
-## Executive Summary
-
-Third-party prompt optimizers and token compressors — such as **Headroom**, **RTK (Rust Token Killer)**, and **Ponytail** — attempt to reduce LLM input tokens by aggressively pruning message history, truncating command stdout, or forcing extreme prompt brevity.
-
-While these tools can reduce raw token counts in simple scenarios, **they frequently break complex agentic coding workflows** by corrupting message graphs, orphaning tool calls, and stripping reasoning parameters. When these pruned payloads hit upstream APIs directly (such as Anthropic, OpenAI, or 9Router), the provider immediately throws fatal `HTTP 400 Bad Request` errors or severely degrades reasoning depth.
-
-**LLM Switcher solves this by acting as the outermost edge gatekeeper (`127.0.0.1:3456`).** It intercepts the pruned payload before it leaves your machine, runs its built-in **Healer Engine** to repair message graphs and restore reasoning parameters, unlocks 1M context windows, and safely converts the protocol to your upstream provider.
-
----
-
-## Tool Breakdown: What They Do & How They Break Payloads
-
-### 1. Headroom (`headroomlabs-ai/headroom`)
-- **Mechanism:** Runs as a local proxy on `:8787` (or wraps CLI agents). Compresses conversation history, RAG chunks, and tool outputs using SmartCrusher (JSON), CodeCompressor (AST), and Kompress-v2-base. Also attempts "effort routing" to dial down thinking budgets.
-- **Critical Failure Points:**
-  - **Orphaned `tool_result` blocks:** When pruning historical turns, Headroom often discards the `assistant` turn containing a `tool_use`, while retaining the subsequent `user` turn containing the `tool_result`. Anthropic's API strictly validates tool use IDs and crashes with:
-    ```
-    HTTP 400 invalid_request_error: "tool_use_id 'xxx' does not correspond to any tool_use"
-    ```
-  - **Consecutive `user` turns:** Dropping intermediary assistant turns causes multiple user messages to sit adjacent to each other. Anthropic strictly throws:
-    ```
-    HTTP 400 invalid_request_error: "roles must alternate between 'user' and 'assistant'"
-    ```
-  - **Reasoning Suppression:** Its "effort routing" dials down `thinking.budget_tokens` on routine tool turns. On complex models (Claude Opus, Gemini Flash), this prevents the model from formulating multi-step reasoning before acting.
-
-### 2. RTK (`rtk-ai/rtk` - Rust Token Killer)
-- **Mechanism:** A single Rust binary that hooks into shell tool execution (e.g. `PreToolUse` in Claude Code / Cursor) and rewrites CLI commands (`git`, `ls`, `cat`, `grep`, `pytest`) to filter out noise, truncate lines, and inject recall tokens (`[full output: rtk recall xxx]`).
-- **Critical Failure Points:**
-  - **Corrupted Structural Data:** When an agent invokes a tool expecting machine-readable JSON or exact AST formatting, RTK's heuristic summaries can alter structural delimiters, causing downstream tool call parsing errors.
-  - **Custom Tracking Headers:** RTK and associated tracing proxies inject headers (`x-rtk-*`, `traceparent`, `x-optimizer-id`) that some strict upstream endpoints reject if not cleanly forwarded.
-
-### 3. Ponytail (`DietrichGebert/ponytail`)
-- **Mechanism:** A behavioral prompt engineering plugin/ruleset that injects extreme conciseness instructions ("write one line, it works, YAGNI") into agent system prompts across 20+ coding tools.
-- **Critical Failure Points:**
-  - **Premature Execution without Reasoning:** By commanding the model to be maximally brief and avoid planning, reasoning models are discouraged from spending thinking tokens. The model outputs untested single-liners that often fail type checks and test suites.
-  - **System Prompt Prefix Invalidation:** Injected rules alter the leading system prompt bytes, busting provider prompt caches unless carefully aligned.
-
----
-
-## The Healer Engine: How LLM Switcher Protects the Workflow
-
-LLM Switcher sits between the optimizer tool and the upstream LLM:
-
-```
-[CLI Agent] ──> [Optimizer: Headroom / RTK] ──> [LLM Switcher :3456] ──> [Upstream / 9Router]
-                                                 │
-                                                 ├── 1. Heal Orphaned tool_results
-                                                 ├── 2. Merge Consecutive Turns
-                                                 ├── 3. Restore Stripped Thinking
-                                                 └── 4. Enforce 1M Context
-```
-
-### Protection Matrix: Before vs. After
-
-| Scenario | Direct to Upstream (Without Switcher) | Through LLM Switcher (Healer Engine) |
-|---|---|---|
-| **Orphaned `tool_result` turn** | ❌ **HTTP 400 Crash**: `tool_use_id does not correspond to any tool_use` | ✅ **HTTP 200 OK**: Heals orphaned result into contextual text block `[Tool Result (id)]: ...` |
-| **Consecutive `user` turns** | ❌ **HTTP 400 Crash**: `roles must alternate` | ✅ **HTTP 200 OK**: Merges consecutive turns into a single valid turn seamlessly |
-| **Stripped `thinking` parameters** | ⚠️ **Degraded AI**: Reasoning disabled, model outputs shallow single-liners | ✅ **HTTP 200 OK**: Detects reasoning models and automatically restores safe thinking budget |
-| **Orphaned `tool` role in Chat API** | ❌ **HTTP 400 Crash**: `tool role must respond to tool_calls` | ✅ **HTTP 200 OK**: Converts orphaned tool message into user context |
-| **Custom tracking headers** | ⚠️ Connection dropped / unrecognized header warnings | ✅ **HTTP 200 OK**: Cleanly passes through `traceparent`, `x-request-id`, `x-rtk-*` |
-
----
-
-## Test Methodology & Verification Suite
-
-We created an automated verification suite in [`tests/live-optimizer-interop.mjs`](../tests/live-optimizer-interop.mjs) that systematically replicates the failure modes of each tool:
-
-### Running the Test Suite
-```bash
-node tests/live-optimizer-interop.mjs
-```
-
-### Live Test Results
-```text
-================================================================
-  LLM SWITCHER — TOKEN OPTIMIZER INTEROPERABILITY TEST SUITE   
-  Simulating failure modes from Headroom, RTK, and Ponytail    
-================================================================
-
-[TEST] Headroom Simulation: Orphaned tool_result turn... PASS (8840ms)
-       ↳ Healed orphaned tool_result. Response HTTP 200: "It looks like you've shared a fragment of context ..."
-[TEST] Headroom Simulation: Consecutive User turns (Role alternation violation)... PASS (2683ms)
-       ↳ Merged consecutive turns seamlessly. Stop reason: end_turn
-[TEST] Thinking Guard: Restoring stripped thinking parameter on reasoning models... PASS (5372ms)
-       ↳ Automatically restored thinking: thinking_delta=true, signature_delta=true, text_delta=true
-[TEST] RTK Intermediary: Custom headers and traceparent passthrough... PASS (2453ms)
-       ↳ Headers accepted cleanly with HTTP 200 OK
-[TEST] OpenAI Chat Healer: Orphaned role tool without preceding assistant tool_calls... PASS (1561ms)
-       ↳ Chat Healer rescued orphaned tool role. Stop reason: length
-
-================================================================
-  TEST RESULTS: 5 PASSED / 0 FAILED               
-================================================================
-```
-
----
-
-## Recommended User Setup
-
-For developers using prompt optimization tools:
-1. Keep the optimizer installed in your CLI tool as usual.
-2. In the optimizer's configuration (e.g. `headroom.yaml` or RTK upstream settings), set the upstream target URL to **LLM Switcher** (`http://127.0.0.1:3456`).
-3. Enjoy prompt compression savings without worrying about broken conversation graphs, HTTP 400 crashes, or lost reasoning depth.
+# Token Optimizer Interoperability & Failure Mode Report
+
+**How LLM Switcher acts as the protective outermost edge gateway for aggressive prompt/token optimizers (Headroom, RTK, Ponytail).**
+
+---
+
+## Executive Summary
+
+Third-party prompt optimizers and token compressors — such as **Headroom**, **RTK (Rust Token Killer)**, and **Ponytail** — attempt to reduce LLM input tokens by aggressively pruning message history, truncating command stdout, or forcing extreme prompt brevity.
+
+While these tools can reduce raw token counts in simple scenarios, **they frequently break complex agentic coding workflows** by corrupting message graphs, orphaning tool calls, and stripping reasoning parameters. When these pruned payloads hit upstream APIs directly (such as Anthropic, OpenAI, or 9Router), the provider immediately throws fatal `HTTP 400 Bad Request` errors or severely degrades reasoning depth.
+
+**LLM Switcher solves this by acting as the outermost edge gatekeeper (`127.0.0.1:3456`).** It intercepts the pruned payload before it leaves your machine, runs its built-in **Healer Engine** to repair message graphs and restore reasoning parameters, unlocks 1M context windows, and safely converts the protocol to your upstream provider.
+
+---
+
+## Tool Breakdown: What They Do & How They Break Payloads
+
+### 1. Headroom (`headroomlabs-ai/headroom`)
+- **Mechanism:** Runs as a local proxy on `:8787` (or wraps CLI agents). Compresses conversation history, RAG chunks, and tool outputs using SmartCrusher (JSON), CodeCompressor (AST), and Kompress-v2-base. Also attempts "effort routing" to dial down thinking budgets.
+- **Critical Failure Points:**
+  - **Orphaned `tool_result` blocks:** When pruning historical turns, Headroom often discards the `assistant` turn containing a `tool_use`, while retaining the subsequent `user` turn containing the `tool_result`. Anthropic's API strictly validates tool use IDs and crashes with:
+    ```
+    HTTP 400 invalid_request_error: "tool_use_id 'xxx' does not correspond to any tool_use"
+    ```
+  - **Consecutive `user` turns:** Dropping intermediary assistant turns causes multiple user messages to sit adjacent to each other. Anthropic strictly throws:
+    ```
+    HTTP 400 invalid_request_error: "roles must alternate between 'user' and 'assistant'"
+    ```
+  - **Reasoning Suppression:** Its "effort routing" dials down `thinking.budget_tokens` on routine tool turns. On complex models (Claude Opus, Gemini Flash), this prevents the model from formulating multi-step reasoning before acting.
+
+### 2. RTK (`rtk-ai/rtk` - Rust Token Killer)
+- **Mechanism:** A single Rust binary that hooks into shell tool execution (e.g. `PreToolUse` in Claude Code / Cursor) and rewrites CLI commands (`git`, `ls`, `cat`, `grep`, `pytest`) to filter out noise, truncate lines, and inject recall tokens (`[full output: rtk recall xxx]`).
+- **Critical Failure Points:**
+  - **Corrupted Structural Data:** When an agent invokes a tool expecting machine-readable JSON or exact AST formatting, RTK's heuristic summaries can alter structural delimiters, causing downstream tool call parsing errors.
+  - **Custom Tracking Headers:** RTK and associated tracing proxies inject headers (`x-rtk-*`, `traceparent`, `x-optimizer-id`) that some strict upstream endpoints reject if not cleanly forwarded.
+
+### 3. Ponytail (`DietrichGebert/ponytail`)
+- **Mechanism:** A behavioral prompt engineering plugin/ruleset that injects extreme conciseness instructions ("write one line, it works, YAGNI") into agent system prompts across 20+ coding tools.
+- **Critical Failure Points:**
+  - **Premature Execution without Reasoning:** By commanding the model to be maximally brief and avoid planning, reasoning models are discouraged from spending thinking tokens. The model outputs untested single-liners that often fail type checks and test suites.
+  - **System Prompt Prefix Invalidation:** Injected rules alter the leading system prompt bytes, busting provider prompt caches unless carefully aligned.
+
+---
+
+## The Healer Engine: How LLM Switcher Protects the Workflow
+
+LLM Switcher sits between the optimizer tool and the upstream LLM:
+
+```
+[CLI Agent] ──> [Optimizer: Headroom / RTK] ──> [LLM Switcher :3456] ──> [Upstream / 9Router]
+                                                 │
+                                                 ├── 1. Heal Orphaned tool_results
+                                                 ├── 2. Merge Consecutive Turns
+                                                 ├── 3. Restore Stripped Thinking
+                                                 └── 4. Enforce 1M Context
+```
+
+### Protection Matrix: Before vs. After
+
+| Scenario | Direct to Upstream (Without Switcher) | Through LLM Switcher (Healer Engine) |
+|---|---|---|
+| **Orphaned `tool_result` turn** | ❌ **HTTP 400 Crash**: `tool_use_id does not correspond to any tool_use` | ✅ **HTTP 200 OK**: Heals orphaned result into contextual text block `[Tool Result (id)]: ...` |
+| **Consecutive `user` turns** | ❌ **HTTP 400 Crash**: `roles must alternate` | ✅ **HTTP 200 OK**: Merges consecutive turns into a single valid turn seamlessly |
+| **Stripped `thinking` parameters** | ⚠️ **Degraded AI**: Reasoning disabled, model outputs shallow single-liners | ✅ **HTTP 200 OK**: Detects reasoning models and automatically restores safe thinking budget |
+| **Orphaned `tool` role in Chat API** | ❌ **HTTP 400 Crash**: `tool role must respond to tool_calls` | ✅ **HTTP 200 OK**: Converts orphaned tool message into user context |
+| **Custom tracking headers** | ⚠️ Connection dropped / unrecognized header warnings | ✅ **HTTP 200 OK**: Cleanly passes through `traceparent`, `x-request-id`, `x-rtk-*` |
+
+---
+
+## Test Methodology & Verification Suite
+
+We created an automated verification suite in [`tests/live-optimizer-interop.mjs`](../tests/live-optimizer-interop.mjs) that systematically replicates the failure modes of each tool:
+
+### Running the Test Suite
+```bash
+node tests/live-optimizer-interop.mjs
+```
+
+### Live Test Results
+```text
+================================================================
+  LLM SWITCHER — TOKEN OPTIMIZER INTEROPERABILITY TEST SUITE   
+  Simulating failure modes from Headroom, RTK, and Ponytail    
+================================================================
+
+[TEST] Headroom Simulation: Orphaned tool_result turn... PASS (8840ms)
+       ↳ Healed orphaned tool_result. Response HTTP 200: "It looks like you've shared a fragment of context ..."
+[TEST] Headroom Simulation: Consecutive User turns (Role alternation violation)... PASS (2683ms)
+       ↳ Merged consecutive turns seamlessly. Stop reason: end_turn
+[TEST] Thinking Guard: Restoring stripped thinking parameter on reasoning models... PASS (5372ms)
+       ↳ Automatically restored thinking: thinking_delta=true, signature_delta=true, text_delta=true
+[TEST] RTK Intermediary: Custom headers and traceparent passthrough... PASS (2453ms)
+       ↳ Headers accepted cleanly with HTTP 200 OK
+[TEST] OpenAI Chat Healer: Orphaned role tool without preceding assistant tool_calls... PASS (1561ms)
+       ↳ Chat Healer rescued orphaned tool role. Stop reason: length
+
+================================================================
+  TEST RESULTS: 5 PASSED / 0 FAILED               
+================================================================
+```
+
+---
+
+## Recommended User Setup
+
+For developers using prompt optimization tools:
+1. Keep the optimizer installed in your CLI tool as usual.
+2. In the optimizer's configuration (e.g. `headroom.yaml` or RTK upstream settings), set the upstream target URL to **LLM Switcher** (`http://127.0.0.1:3456`).
+3. Enjoy prompt compression savings without worrying about broken conversation graphs, HTTP 400 crashes, or lost reasoning depth.