coderouter-cli 2.17.0__tar.gz → 2.18.0__tar.gz

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/CHANGELOG.md

@@ -9,6 +9,43 @@ are kept verbatim where the Japanese text itself is the subject).
 
 ---
 
+## [v2.18.0] — 2026-10-24 (route on Claude Code's own request hints)
+
+### Added
+
+- **auto_router matchers on Claude Code's gateway hint headers.** With
+  `CLAUDE_CODE_GATEWAY_HINT_HEADERS=1` (Claude Code 2.1.273+), every request
+  says what it is. Three new `RuleMatcher` fields read those headers:
+  - `request_class` — `x-claude-code-request-class` (`main` / `subagent` /
+    `workflow` / `compaction` / `auxiliary`), case-insensitive;
+  - `agent_type` — `x-claude-code-agent-type` (e.g. `Explore`),
+    case-insensitive;
+  - `compaction: true` — `x-claude-code-compaction` is truthy (`false` is
+    rejected at load, like the other boolean matchers).
+  Typical use: auto-mode classifier / title calls (`auxiliary`) to the
+  smallest local model, compaction to a long-context profile, `Explore`
+  sub-agents to a fast one. Without the headers these matchers never match,
+  so existing rule sets behave exactly as before. `classify()` gained an
+  optional `hints` argument; both `/v1/messages` and `/v1/chat/completions`
+  pass the headers through. Docs: `docs/guides/subagent-routing.md` / `.en.md`.
+- **`slot_affinity_slots` (provider; llama.cpp only).** For
+  `llama-server --parallel N`: requests carrying a conversation key
+  (`x-claude-code-session-id`) get `id_slot = sha256(key) % N`, so every
+  turn of a conversation reuses the same slot's prompt cache instead of
+  re-prefilling on whichever slot is free. Stable across restarts; an
+  explicit `id_slot` in `extra_body` wins; off by default. Works on both the
+  native (`kind: anthropic`) and translated (`kind: openai_compat`) paths,
+  and survives every internal request rebuild (thinking strip for backends
+  without thinking, `tool_choice` emulation, `cache_control` strip,
+  OpenAI→Anthropic translation). The main conversation and its compaction
+  share a key; sub-agents and auxiliary calls get `<session>:<class>:<agent>`
+  so they do not queue on — or evict — the main conversation's slot.
+  Docs: `docs/backends/llamacpp-direct.md` / `.en.md`.
+- **`vscode-init` also writes `CLAUDE_CODE_GATEWAY_HINT_HEADERS=1`**, and
+  `doctor --check-claude-code` reports (as info) when it is missing.
+
+---
+
 ## [v2.17.0] — 2026-10-24 (time-to-first-token cut-off, retry signals)
 
 ### Added

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: coderouter-cli
-Version: 2.17.0
+Version: 2.18.0
 Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
 Project-URL: Homepage, https://github.com/zephel01/CodeRouter
 Project-URL: Repository, https://github.com/zephel01/CodeRouter

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/adapters/anthropic_native.py

@@ -46,6 +46,7 @@ from coderouter.adapters.base import (
     StreamChunk,
     StreamTruncatedError,
 )
+from coderouter.adapters.slot_affinity import apply_slot_affinity
 from coderouter.credentials import resolve_provider_credential
 from coderouter.logging import (
     get_logger,
@@ -243,6 +244,9 @@ class AnthropicAdapter(BaseAdapter):
         body.update(dumped)
         body["model"] = self.config.model
         body["stream"] = stream
+        apply_slot_affinity(
+            body, slots=self.config.slot_affinity_slots, key=req.session_key
+        )
         return body
 
     # ------------------------------------------------------------------

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/adapters/base.py

@@ -65,6 +65,9 @@ class ChatRequest(BaseModel):
 
     # CodeRouter-specific extension (not sent upstream)
     profile: str | None = Field(default=None, exclude=True)
+    # v2.18.0: conversation key for KV-slot affinity (see
+    # ``ProviderConfig.slot_affinity_slots``). Never serialized.
+    session_key: str | None = Field(default=None, exclude=True)
 
 
 class ChatResponse(BaseModel):

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/adapters/openai_compat.py

@@ -30,6 +30,7 @@ from coderouter.adapters.base import (
     StreamChunk,
     StreamTruncatedError,
 )
+from coderouter.adapters.slot_affinity import apply_slot_affinity
 from coderouter.credentials import resolve_provider_credential
 from coderouter.logging import (
     get_logger,
@@ -182,6 +183,11 @@ class OpenAICompatAdapter(BaseAdapter):
             value = getattr(request, field, None)
             if value is not None:
                 body[field] = value
+        apply_slot_affinity(
+            body,
+            slots=self.config.slot_affinity_slots,
+            key=getattr(request, "session_key", None),
+        )
         if stream:
             # Request a terminal usage chunk. Providers that honor this
             # (OpenAI, OpenRouter, Ollama >=0.x) will send one extra chunk

coderouter_cli-2.18.0/coderouter/adapters/slot_affinity.py

@@ -0,0 +1,54 @@
+"""v2.18.0: KV-slot affinity for llama-server (``id_slot``).
+
+llama-server started with ``--parallel N`` serves N slots, each holding its
+own prompt cache. Without a hint it picks a free slot, so consecutive turns
+of one conversation can land on different slots and re-prefill the whole
+prompt. Sending ``id_slot`` pins a conversation to one slot.
+
+The slot is ``sha256(key) % N`` — stable across processes and restarts
+(Python's ``hash`` is salted per process).
+"""
+
+from __future__ import annotations
+
+import hashlib
+from collections.abc import Mapping
+from typing import Any
+
+# Request classes whose prompt shares the main conversation's prefix: they
+# belong on the main conversation's slot. Everything else (sub-agents,
+# auxiliary calls such as the auto-mode classifier or titles, workflows) has
+# its own prompt and gets its own key, so it neither waits on the main slot
+# nor evicts its cache.
+_MAIN_CLASSES = frozenset({"", "main", "compaction"})
+
+
+def affinity_key(hints: Mapping[str, str]) -> str | None:
+    """Conversation key for slot affinity, from Claude Code's hint headers.
+
+    ``None`` when there is no session id. ``main`` / ``compaction`` (and a
+    missing class) share ``<session>``; other classes get
+    ``<session>:<class>:<agent_type>``.
+    """
+    session = hints.get("session_id")
+    if not session:
+        return None
+    request_class = hints.get("request_class", "").lower()
+    if request_class in _MAIN_CLASSES:
+        return session
+    return f"{session}:{request_class}:{hints.get('agent_type', '').lower()}"
+
+
+def slot_for(key: str, slots: int) -> int:
+    digest = hashlib.sha256(key.encode("utf-8")).digest()
+    return int.from_bytes(digest[:8], "big") % slots
+
+
+def apply_slot_affinity(body: dict[str, Any], *, slots: int | None, key: str | None) -> None:
+    """Set ``body["id_slot"]`` when affinity is configured and a key exists.
+
+    An ``id_slot`` already present (e.g. from ``extra_body``) wins.
+    """
+    if not slots or not key or "id_slot" in body:
+        return
+    body["id_slot"] = slot_for(key, slots)

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/cli.py

@@ -483,7 +483,8 @@ def _build_parser() -> argparse.ArgumentParser:
         help=(
             "v2.16.0: do not write the Claude Code gateway settings "
             "(ENABLE_TOOL_SEARCH=false, CLAUDE_CODE_AUTO_MODE_SERVER=0, "
-            "and --max-context-tokens). Use when the chain is "
+            "CLAUDE_CODE_GATEWAY_HINT_HEADERS=1 and --max-context-tokens). "
+            "Use when the chain is "
             "api.anthropic.com only."
         ),
     )

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/config/schemas.py

@@ -562,6 +562,23 @@ class ProviderConfig(BaseModel):
     # withholds those opening events from the client. ``None`` (default)
     # disables it. Typical use: a local llama-server still loading a model
     # or chewing through a long prompt while a cloud fallback could answer.
+    # v2.18.0: KV-slot affinity for llama-server started with
+    # ``--parallel N``. When set to N and the request carries a conversation
+    # key (Claude Code's ``x-claude-code-session-id`` hint header), the body
+    # gets ``id_slot = stable_hash(key) % N`` so every turn of a conversation
+    # lands on the same slot and reuses its prompt cache instead of
+    # re-prefilling on whichever slot is free. llama.cpp only (other servers
+    # ignore or reject ``id_slot``); ``None`` (default) sends nothing.
+    slot_affinity_slots: int | None = Field(
+        default=None,
+        ge=1,
+        le=256,
+        description=(
+            "v2.18.0: llama-server --parallel N. Pins each conversation to "
+            "slot hash(session) % N via id_slot so its KV cache is reused. "
+            "llama.cpp only. None disables."
+        ),
+    )
     ttft_timeout_s: float | None = Field(
         default=None,
         gt=0.0,
@@ -1639,6 +1656,21 @@ class RuleMatcher(BaseModel):
     # declares any tools); ``False`` is rejected by ``_exactly_one``
     # since a "no-tools" rule would shadow the default fall-through.
     has_tools: bool | None = None
+    # v2.18.0: Claude Code gateway hint headers. Claude Code 2.1.273+ sends
+    # these to a custom ANTHROPIC_BASE_URL when
+    # ``CLAUDE_CODE_GATEWAY_HINT_HEADERS=1`` is set (``coderouter
+    # vscode-init`` sets it). They say what a request *is* — something the
+    # body heuristics above can only guess:
+    #   * ``request_class`` — ``x-claude-code-request-class``: ``main`` /
+    #     ``subagent`` / ``workflow`` / ``compaction`` / ``auxiliary``
+    #     (auto-mode classifier, titles, …). Case-insensitive exact match.
+    #   * ``agent_type`` — ``x-claude-code-agent-type`` (e.g. ``Explore``),
+    #     case-insensitive exact match.
+    #   * ``compaction: True`` — ``x-claude-code-compaction`` is truthy.
+    # Without the headers these matchers simply never match.
+    request_class: str | None = None
+    agent_type: str | None = None
+    compaction: bool | None = None
 
     _MATCHER_FIELDS: tuple[str, ...] = (
         "has_image",
@@ -1649,12 +1681,15 @@ class RuleMatcher(BaseModel):
         "content_token_count_min",
         "has_tools",
         "cjk_ratio_min",
+        "request_class",
+        "agent_type",
+        "compaction",
     )
 
     # Boolean matcher fields carry meaning only at ``True`` — the runtime
     # evaluator uses ``is True`` — so a ``False`` value is a dead rule that
     # never matches. ``_exactly_one`` rejects it explicitly (see below).
-    _BOOL_MATCHER_FIELDS: tuple[str, ...] = ("has_image", "has_tools")
+    _BOOL_MATCHER_FIELDS: tuple[str, ...] = ("has_image", "has_tools", "compaction")
 
     @model_validator(mode="after")
     def _exactly_one(self) -> Self:

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/doctor_claude_code.py

@@ -125,6 +125,18 @@ def check(version: Version | None, env: Mapping[str, str]) -> list[ClaudeCodeFin
             )
         )
 
+    if env.get("CLAUDE_CODE_GATEWAY_HINT_HEADERS", "").lower() in ("1", "true"):
+        out.append(ClaudeCodeFinding("ok", "CLAUDE_CODE_GATEWAY_HINT_HEADERS=1"))
+    else:
+        out.append(
+            ClaudeCodeFinding(
+                "info",
+                "CLAUDE_CODE_GATEWAY_HINT_HEADERS is not set: auto_router rules "
+                "on request_class / agent_type / compaction (v2.18.0) never "
+                "match without it.",
+            )
+        )
+
     if env.get("CLAUDE_CODE_AUTO_MODE_SERVER") == "0":
         out.append(ClaudeCodeFinding("ok", "CLAUDE_CODE_AUTO_MODE_SERVER=0"))
     else:

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/ingress/anthropic_routes.py

@@ -31,6 +31,7 @@ from fastapi import APIRouter, Header, HTTPException, Request
 from fastapi.responses import JSONResponse, Response, StreamingResponse
 
 from coderouter.adapters.base import StreamTruncatedError
+from coderouter.adapters.slot_affinity import affinity_key
 from coderouter.guards.context_budget import ContextBudgetRejectError
 from coderouter.guards.tool_loop import ToolCountExceededError, ToolLoopBreakError
 from coderouter.ingress.anthropic_errors import (
@@ -49,7 +50,7 @@ from coderouter.routing import (
     MidStreamError,
     NoProvidersAvailableError,
 )
-from coderouter.routing.auto_router import RESERVED_PROFILE_NAME, classify
+from coderouter.routing.auto_router import RESERVED_PROFILE_NAME, classify, extract_gateway_hints
 from coderouter.routing.fallback_trace import (
     SSE_FALLBACK_EVENT,
     current_fallback_trace,
@@ -226,6 +227,11 @@ async def messages(
     if anthropic_beta:
         anth_req.anthropic_beta = anthropic_beta
 
+    # v2.18.0: Claude Code gateway hint headers — routing matchers below,
+    # and the conversation key for llama-server KV-slot affinity.
+    gateway_hints = extract_gateway_hints(request.headers)
+    anth_req.session_key = affinity_key(gateway_hints)
+
     # Profile selection — body field wins over header (same policy as OpenAI route).
     if anth_req.profile is None and x_coderouter_profile:
         anth_req.profile = x_coderouter_profile
@@ -253,7 +259,7 @@ async def messages(
     # AnthropicRequest), so both OpenAI and Anthropic ingress use the same
     # classifier without a shared request shim.
     if anth_req.profile is None and config.default_profile == RESERVED_PROFILE_NAME:
-        anth_req.profile = classify(payload, config)
+        anth_req.profile = classify(payload, config, gateway_hints)
 
     if anth_req.profile is not None:
         try:

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/ingress/openai_routes.py

@@ -32,9 +32,10 @@ from fastapi import APIRouter, Header, HTTPException, Request
 from fastapi.responses import JSONResponse, StreamingResponse
 
 from coderouter.adapters.base import ChatRequest
+from coderouter.adapters.slot_affinity import affinity_key
 from coderouter.logging import get_logger
 from coderouter.routing import FallbackEngine, NoProvidersAvailableError
-from coderouter.routing.auto_router import RESERVED_PROFILE_NAME, classify
+from coderouter.routing.auto_router import RESERVED_PROFILE_NAME, classify, extract_gateway_hints
 from coderouter.routing.fallback_trace import current_fallback_trace
 
 router = APIRouter()
@@ -199,6 +200,10 @@ async def chat_completions(
     except Exception as exc:  # pydantic.ValidationError, etc.
         raise HTTPException(status_code=422, detail=str(exc)) from exc
 
+    # v2.18.0: gateway hint headers (routing matchers + KV-slot affinity key).
+    gateway_hints = extract_gateway_hints(request.headers)
+    chat_req.session_key = affinity_key(gateway_hints)
+
     # Header-based override (body wins if both are set — see module docstring)
     if chat_req.profile is None and x_coderouter_profile:
         chat_req.profile = x_coderouter_profile
@@ -226,7 +231,7 @@ async def chat_completions(
     # When inactive, the engine still falls through to
     # ``config.default_profile`` on its own — same semantics as pre-v1.6.
     if chat_req.profile is None and config.default_profile == RESERVED_PROFILE_NAME:
-        chat_req.profile = classify(payload, config)
+        chat_req.profile = classify(payload, config, gateway_hints)
 
     # Validate profile exists before we kick off any upstream call
     if chat_req.profile is not None:

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/routing/auto_router.py

@@ -36,6 +36,7 @@ from __future__ import annotations
 
 import logging
 import re
+from collections.abc import Mapping
 from typing import TYPE_CHECKING, Any
 
 from coderouter.config.schemas import AutoRouterConfig, AutoRouteRule, RuleMatcher
@@ -169,6 +170,32 @@ def _has_tools_in_body(body: dict[str, Any]) -> bool:
     return isinstance(functions, list) and len(functions) > 0
 
 
+# v2.18.0: Claude Code gateway hint headers → hint keys.
+_HINT_HEADERS: dict[str, str] = {
+    "x-claude-code-request-class": "request_class",
+    "x-claude-code-agent-type": "agent_type",
+    "x-claude-code-compaction": "compaction",
+    "x-claude-code-session-id": "session_id",
+    "x-claude-code-prompt-id": "prompt_id",
+}
+_TRUTHY = frozenset({"1", "true", "yes", "on"})
+
+
+def extract_gateway_hints(headers: Mapping[str, str]) -> dict[str, str]:
+    """Pull Claude Code's gateway hint headers out of a request.
+
+    Returns only the hints that are present (lower-cased keys from
+    :data:`_HINT_HEADERS`). Header lookup is case-insensitive whenever the
+    mapping is (Starlette's ``Headers`` is).
+    """
+    out: dict[str, str] = {}
+    for header, key in _HINT_HEADERS.items():
+        value = headers.get(header)
+        if value:
+            out[key] = value.strip()
+    return out
+
+
 def _match_rule(
     rule: AutoRouteRule,
     message: dict[str, Any] | None,
@@ -176,8 +203,17 @@ def _match_rule(
     model: str | None,
     estimated_tokens: int,
     has_tools: bool,
+    hints: Mapping[str, str] | None = None,
 ) -> bool:
     m = rule.match
+    if m.request_class is not None:
+        value = (hints or {}).get("request_class")
+        return value is not None and value.lower() == m.request_class.lower()
+    if m.agent_type is not None:
+        value = (hints or {}).get("agent_type")
+        return value is not None and value.lower() == m.agent_type.lower()
+    if m.compaction is True:
+        return (hints or {}).get("compaction", "").lower() in _TRUTHY
     if m.has_image is True:
         return message is not None and _has_image(message)
     if m.code_fence_ratio_min is not None:
@@ -237,7 +273,11 @@ def _extract_model(body: dict[str, Any]) -> str | None:
     return None
 
 
-def classify(body: dict[str, Any], config: CodeRouterConfig) -> str:
+def classify(
+    body: dict[str, Any],
+    config: CodeRouterConfig,
+    hints: Mapping[str, str] | None = None,
+) -> str:
     """Resolve an incoming request body to a profile name.
 
     Rule order (first match wins) is determined by:
@@ -252,6 +292,10 @@ def classify(body: dict[str, Any], config: CodeRouterConfig) -> str:
 
     Emits one of two log events: ``auto-router-resolved`` on match, or
     ``auto-router-fallthrough`` on default-rule fall.
+
+    v2.18.0: ``hints`` carries Claude Code's gateway hint headers (see
+    :func:`extract_gateway_hints`) for the ``request_class`` /
+    ``agent_type`` / ``compaction`` matchers.
     """
     user_msg = _latest_user_message(body)
     text = _extract_text(user_msg) if user_msg is not None else ""
@@ -294,7 +338,7 @@ def classify(body: dict[str, Any], config: CodeRouterConfig) -> str:
     # present — they short out via ``_match_rule``'s message-None
     # handling.
     for rule in rules:
-        if _match_rule(rule, user_msg, text, model, estimated_tokens, has_tools):
+        if _match_rule(rule, user_msg, text, model, estimated_tokens, has_tools, hints):
             _emit_resolved(rule, user_msg, text, model, estimated_tokens, has_tools)
             return rule.profile
 

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/routing/capability.py

@@ -194,12 +194,13 @@ def strip_thinking(request: AnthropicRequest) -> AnthropicRequest:
 
     # model_dump() serializes extras; roundtripping via validate rebuilds a
     # clean instance without the dropped key. exclude=True fields (profile,
-    # anthropic_beta) are omitted by model_dump, so we reassign them.
+    # anthropic_beta, session_key) are omitted by model_dump, so we reassign them.
     dumped = request.model_dump()
     dumped.pop("thinking", None)
     stripped = AnthropicRequest.model_validate(dumped)
     stripped.profile = request.profile
     stripped.anthropic_beta = request.anthropic_beta
+    stripped.session_key = request.session_key
     return stripped
 
 
@@ -450,6 +451,7 @@ def emulate_tool_choice(request: AnthropicRequest) -> AnthropicRequest:
     emulated = AnthropicRequest.model_validate(dumped)
     emulated.profile = request.profile
     emulated.anthropic_beta = request.anthropic_beta
+    emulated.session_key = request.session_key
     return emulated
 
 
@@ -522,6 +524,7 @@ def strip_cache_control(request: AnthropicRequest) -> tuple[AnthropicRequest, in
     stripped = AnthropicRequest.model_validate(dumped)
     stripped.profile = request.profile
     stripped.anthropic_beta = request.anthropic_beta
+    stripped.session_key = request.session_key
     return stripped, removed
 
 

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/translation/anthropic.py

@@ -277,6 +277,9 @@ class AnthropicRequest(BaseModel):
     # beta-gated body fields like `context_management`, `cache_control`,
     # `thinking` beyond what the default minor version accepts.
     anthropic_beta: str | None = Field(default=None, exclude=True)
+    # v2.18.0: conversation key for KV-slot affinity (Claude Code's
+    # ``x-claude-code-session-id``). Never serialized.
+    session_key: str | None = Field(default=None, exclude=True)
 
     @model_validator(mode="before")
     @classmethod

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/translation/convert.py

@@ -285,6 +285,8 @@ def to_chat_request(req: AnthropicRequest) -> ChatRequest:
     )
     # Propagate CodeRouter routing hint.
     chat_req.profile = req.profile
+    # v2.18.0: carry the KV-slot affinity key across the translation.
+    chat_req.session_key = req.session_key
     return chat_req
 
 
@@ -1114,6 +1116,8 @@ def to_anthropic_request(chat_req: ChatRequest) -> AnthropicRequest:
     )
     # Propagate CodeRouter routing hint.
     req.profile = chat_req.profile
+    # v2.18.0: KV-slot affinity key (exclude=True, so set by hand).
+    req.session_key = chat_req.session_key
     return req
 
 

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/coderouter/vscode_init.py

@@ -127,16 +127,25 @@ _MANAGED_ENV_KEYS: tuple[str, ...] = (
 #   is given. Since 2.1.285 Claude Code assumes a 1M window behind a
 #   gateway for Opus 4.7+ / Sonnet 5+ / Fable ids, so auto-compaction would
 #   wait far past what a local model holds.
+# * ``CLAUDE_CODE_GATEWAY_HINT_HEADERS=1`` (v2.18.0) — makes Claude Code send
+#   ``x-claude-code-request-class`` / ``-agent-type`` / ``-compaction`` …, which
+#   the auto router's ``request_class`` / ``agent_type`` / ``compaction``
+#   matchers read. Harmless when no rule uses them.
 CLAUDE_CODE_ENV_KEYS: tuple[str, ...] = (
     "ENABLE_TOOL_SEARCH",
     "CLAUDE_CODE_AUTO_MODE_SERVER",
+    "CLAUDE_CODE_GATEWAY_HINT_HEADERS",
     "CLAUDE_CODE_MAX_CONTEXT_TOKENS",
 )
 
 
 def claude_code_env(*, max_context_tokens: int | None = None) -> dict[str, str]:
     """v2.16.0: the Claude Code env vars ``vscode-init`` writes by default."""
-    env = {"ENABLE_TOOL_SEARCH": "false", "CLAUDE_CODE_AUTO_MODE_SERVER": "0"}
+    env = {
+        "ENABLE_TOOL_SEARCH": "false",
+        "CLAUDE_CODE_AUTO_MODE_SERVER": "0",
+        "CLAUDE_CODE_GATEWAY_HINT_HEADERS": "1",
+    }
     if max_context_tokens is not None:
         if max_context_tokens <= 0:
             raise ValueError("max_context_tokens must be positive")

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/docs/backends/llamacpp-direct.en.md

@@ -264,3 +264,24 @@ Naming the provider `llamacpp-<model>` keeps things consistent with the conventi
 Known issue: combining a nextn-embedded model (or active speculative decoding) with `--split-mode tensor` can crash llama.cpp ([issue #24309](https://github.com/ggml-org/llama.cpp/issues/24309)). Use `--split-mode layer` if this applies to you.
 
 **The [CodeRouter Launcher](./launcher.en.md#mtp--speculative-decoding-llamacpp) automates this whole dance** — if the main gguf embeds nextn, or a companion draft/MTP gguf sits in the same folder, the Launcher auto-detects it and adds `--spec-type` / `--model-draft` to the launch command for you (set the **MTP** field to `off` to disable).
+
+## Reusing the KV cache per conversation across slots (v2.18.0)
+
+`llama-server --parallel N` serves N slots, each with its own prompt cache. Without a hint the server picks a free slot, so the next turn of the same conversation can land on a different slot and prefill the whole prompt again.
+
+Set `slot_affinity_slots: N` (the same N as `--parallel`) on the provider and CodeRouter sends `id_slot = hash(session) % N`, so a conversation always lands on the same slot.
+
+```yaml
+providers:
+  - name: local-llamacpp
+    kind: openai_compat
+    base_url: http://127.0.0.1:8080/v1
+    model: ""
+    slot_affinity_slots: 4     # match llama-server --parallel 4
+```
+
+- The main conversation and its compaction share a slot. Sub-agents and auxiliary calls (auto-mode classifier, titles, …) get their own key per kind, so they neither wait on the main slot nor evict its cache.
+- The conversation key is `x-claude-code-session-id`, one of Claude Code's hint headers; it needs `CLAUDE_CODE_GATEWAY_HINT_HEADERS=1` (set by `coderouter vscode-init`). Requests without it get nothing added.
+- llama.cpp only — do not set it for other servers (they ignore `id_slot` or reject it as unknown).
+- An `id_slot` set explicitly in `extra_body` wins.
+- With more conversations than slots, conversations share slots (and evict each other's cache).

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/docs/backends/llamacpp-direct.md

@@ -316,3 +316,24 @@ provider name は `llamacpp-<モデル>` の命名で揃えると `examples/prov
 既知の問題として、nextn 埋め込みモデル(または speculative decoding 有効時)と `--split-mode tensor` の組み合わせは llama.cpp がクラッシュすることがあります([issue #24309](https://github.com/ggml-org/llama.cpp/issues/24309))。該当する場合は `--split-mode layer` を使ってください。
 
 **この一連の手組みは [CodeRouter Launcher](./launcher.md#mtp--speculative-decoding-llamacpp) が自動化しています** — メインの gguf に nextn が埋め込まれているか、同じフォルダに companion の draft/MTP gguf があれば、Launcher が `--spec-type` / `--model-draft` を自動検出して起動コマンドに付与します(`MTP` 欄で `off` にすれば無効化も可能)。
+
+## 複数スロットで会話ごとに KV キャッシュを使い回す（v2.18.0）
+
+`llama-server --parallel N` で起動すると、サーバは N 個のスロットを持ち、それぞれがプロンプトのキャッシュを抱えます。何も指定しないと空いているスロットが選ばれるため、同じ会話の次のターンが別のスロットに行き、プロンプト全体を処理し直すことがあります。
+
+provider に `slot_affinity_slots: N`（`--parallel` と同じ数）を書くと、CodeRouter は会話ごとに `id_slot = hash(セッション) % N` を付けて送り、同じ会話は毎回同じスロットに入ります。
+
+```yaml
+providers:
+  - name: local-llamacpp
+    kind: openai_compat
+    base_url: http://127.0.0.1:8080/v1
+    model: ""
+    slot_affinity_slots: 4     # llama-server --parallel 4 と合わせる
+```
+
+- 本会話と、その要約（compaction）は同じスロットを使います。サブエージェントや補助呼び出し（auto mode の判定、タイトル生成など）は、種類ごとに別のキーになり、本会話のスロットを待たず、そのキャッシュも上書きしません。
+- 会話の識別には、Claude Code がヒントヘッダで送る `x-claude-code-session-id` を使います。`CLAUDE_CODE_GATEWAY_HINT_HEADERS=1` が必要です（`coderouter vscode-init` が設定します）。ヘッダが無い要求には何も付けません。
+- llama.cpp 専用の設定です。ほかのサーバには付けないでください（無視されるか、未知のパラメータとして拒否されます）。
+- `extra_body` で `id_slot` を明示している場合は、そちらが優先されます。
+- 会話の数がスロット数より多いと、別の会話が同じスロットを共有します（そのたびにキャッシュは入れ替わります）。

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/docs/guides/subagent-routing.en.md

@@ -124,7 +124,7 @@ auto_router:
 - `model_pattern` uses **`re.fullmatch`**, not `re.search`. Your regex must match the *entire* string of whatever model name Claude Code actually sends (whether it stays `opus` or expands to `claude-opus-4-8` is environment-dependent). Forgetting the trailing `.*` is a common way for a rule to silently miss.
 - The exact model-name string that arrives is environment- and version-dependent, and cannot be assumed. **Before relying on a rule in production, confirm the real value from the auto-router log** (the `signals.model` field of the `auto-router-resolved` event). See [§6](#6-verifying-it-works) for the procedure.
 
-### The full list of auto_router matchers (8 total)
+### The full list of auto_router matchers (11 total)
 
 A `RuleMatcher` allows **exactly one matcher per rule** — a load-time validator enforces this, so AND-composing multiple conditions is not possible (see gap G1 in [§7](#7-limitations-and-known-gaps)). Rules are evaluated top-to-bottom, first match wins.
 
@@ -139,7 +139,24 @@ A `RuleMatcher` allows **exactly one matcher per rule** — a load-time validato
 | `content_token_count_min` | `int` (≥1) | Matches if the estimated token count (char/4 heuristic) over system + all messages is ≥ threshold | Entire request |
 | `has_tools` | `bool` (only `true` is valid) | Matches if the body declares 1+ entries in `tools[]` (OpenAI/Anthropic common) or legacy OpenAI `functions[]` | Entire body |
 
-`model_pattern`, `content_token_count_min`, and `has_tools` can fire even without a user message (e.g. a system-only request, or a body carrying just `model` + `tools`). Every other matcher requires a user message to be present. Boolean matchers (`has_image` / `has_tools`) reject an explicit `false` at load time (dead-rule prevention — omit the field instead if unused).
+**The hint-header matchers (v2.18.0)** — `request_class` (`x-claude-code-request-class`: `main` / `subagent` / `workflow` / `compaction` / `auxiliary`), `agent_type` (`x-claude-code-agent-type`, e.g. `Explore`) and `compaction: true` (`x-claude-code-compaction`) — work only when Claude Code 2.1.273+ runs with `CLAUDE_CODE_GATEWAY_HINT_HEADERS=1` (set by `coderouter vscode-init`). Unlike the body heuristics, Claude Code itself declares what the request is, so there is nothing to guess. Matching is case-insensitive.
+
+```yaml
+auto_router:
+  default_rule_profile: main
+  rules:
+    - id: aux-to-tiny            # auto-mode classifier, titles → smallest model
+      match: { request_class: auxiliary }
+      profile: tiny-local
+    - id: compaction-long-ctx    # conversation summaries → long-context model
+      match: { compaction: true }
+      profile: long-context
+    - id: explore-fast           # Explore sub-agents → fast model
+      match: { agent_type: Explore }
+      profile: fast-local
+```
+
+`model_pattern`, `content_token_count_min`, and `has_tools` can fire even without a user message (e.g. a system-only request, or a body carrying just `model` + `tools`). Every other matcher requires a user message to be present. Boolean matchers (`has_image` / `has_tools` / `compaction`) reject an explicit `false` at load time (dead-rule prevention — omit the field instead if unused).
 
 > **`content_token_count_min` means something very different as of v2.12.0.** The estimator now counts `tool_result`, `tool_use` and `thinking` blocks, which v2.11.x treated as zero characters, so the same conversation estimates an order of magnitude higher. Measured on a tool-driven session: v2.11.x reported 1,083 tokens after 200 turns; v2.12.0 reports 93,183. **An existing threshold will fire far earlier than you tuned it for** — including the `32000` used in the examples below. Retune it, or set `token_estimation_include_tool_content: false` to restore the v2.11.x estimate. See [context-budget.md](../concepts/context-budget.en.md).
 

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/docs/guides/subagent-routing.md

@@ -124,7 +124,7 @@ auto_router:
 - `model_pattern` は `re.search` ではなく **`re.fullmatch`** である。Claude Code が実際に送るモデル名文字列(`opus` のままか `claude-opus-4-8` に展開されるかは環境依存)の**全体**にマッチする正規表現を書く必要がある。末尾の `.*` を忘れると一致しない。
 - 実際に届くモデル名は環境・バージョン依存で断定できない。**運用前に必ず auto-router のログ(`auto-router-resolved` イベントの `signals.model`)で実値を確認する**こと。確認手順は [§6 動作確認](#6-動作確認) を参照。
 
-### auto_router matcher 一覧(全 8 種)
+### auto_router matcher 一覧(全 11 種)
 
 `RuleMatcher` は **1 ルール = ちょうど 1 matcher**。読み込み時のバリデータが強制するため、複数条件の AND 合成はできない(→ [§7](#7-制限と既知の課題) の G1)。ルール順は「先着一致」で、上から順に評価される。
 
@@ -138,8 +138,28 @@ auto_router:
 | `model_pattern` | `str` | body の `model` に対する `re.fullmatch`(読み込み時にコンパイル検証) | body の `model` |
 | `content_token_count_min` | `int`(≥1) | 全メッセージ(system + messages)の推定トークン数(char/4)が閾値以上 | リクエスト全体 |
 | `has_tools` | `bool`(`true` のみ有効) | `tools[]`(OpenAI/Anthropic 共通)または legacy `functions[]` を1個以上宣言 | body 全体 |
+| `request_class`(v2.18.0) | `str` | Claude Code のヒントヘッダ `x-claude-code-request-class`(`main` / `subagent` / `workflow` / `compaction` / `auxiliary`)と一致(大小区別なし) | ヘッダ |
+| `agent_type`(v2.18.0) | `str` | `x-claude-code-agent-type`(例: `Explore`)と一致(大小区別なし) | ヘッダ |
+| `compaction`(v2.18.0) | `bool`(`true` のみ有効) | `x-claude-code-compaction` が真 | ヘッダ |
 
-`model_pattern` / `content_token_count_min` / `has_tools` は user メッセージが無くても発火できる(system-only なリクエストや model+tools のみの body)。それ以外の matcher は user メッセージが必須。ブール matcher(`has_image` / `has_tools`)に `false` を明示すると読み込みエラーになる(死にルール防止のため、未使用は省略する)。
+**ヒントヘッダの matcher(v2.18.0)** は、Claude Code 2.1.273 以降に `CLAUDE_CODE_GATEWAY_HINT_HEADERS=1` を設定したときだけ働きます(`coderouter vscode-init` が設定します)。本文から推測する他の matcher と違い、リクエストが何であるか(本会話か、サブエージェントか、要約か、権限判定などの補助呼び出しか)を Claude Code 自身が申告するので、取り違えがありません。たとえば次のように使います。
+
+```yaml
+auto_router:
+  default_rule_profile: main
+  rules:
+    - id: aux-to-tiny            # auto mode の判定やタイトル生成は最小モデルへ
+      match: { request_class: auxiliary }
+      profile: tiny-local
+    - id: compaction-long-ctx    # 会話の要約は文脈の長いモデルへ
+      match: { compaction: true }
+      profile: long-context
+    - id: explore-fast           # Explore サブエージェントは速いモデルへ
+      match: { agent_type: Explore }
+      profile: fast-local
+```
+
+`model_pattern` / `content_token_count_min` / `has_tools` は user メッセージが無くても発火できる(system-only なリクエストや model+tools のみの body)。それ以外の matcher は user メッセージが必須。ブール matcher(`has_image` / `has_tools` / `compaction`)に `false` を明示すると読み込みエラーになる(死にルール防止のため、未使用は省略する)。
 
 > **v2.12.0 で `content_token_count_min` の実効値が大きく変わりました。** 推定器が `tool_result` / `tool_use` / `thinking` を数えるようになったため(v2.11.x は 0 文字扱い)、同じ会話でも推定値が桁で増えます。実測では tool 主体のセッションで、v2.11.x なら 200 ターンでも 1,083 tokens だったものが v2.12.0 では 93,183 tokens です。**既存の閾値を書いている場合、想定よりずっと早くルールが発火します。** 本ページ以下の例で使っている `32000` も同様です。閾値を見直すか、`token_estimation_include_tool_content: false` で v2.11.x の推定に戻してください。詳細は [context-budget.md](../concepts/context-budget.md) を参照。
 

{coderouter_cli-2.17.0 → coderouter_cli-2.18.0}/pyproject.toml

@@ -11,7 +11,7 @@
 # in plan.md §11.B; once granted, this name will become an alias and
 # `coderouter` will become the canonical distribution name.
 name = "coderouter-cli"
-version = "2.17.0"
+version = "2.18.0"
 description = "Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible."
 readme = "README.md"
 requires-python = ">=3.12"