coderouter-cli 2.16.0__tar.gz → 2.18.0__tar.gz

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/CHANGELOG.md

@@ -9,6 +9,90 @@ are kept verbatim where the Japanese text itself is the subject).
 
 ---
 
+## [v2.18.0] — 2026-10-24 (route on Claude Code's own request hints)
+
+### Added
+
+- **auto_router matchers on Claude Code's gateway hint headers.** With
+  `CLAUDE_CODE_GATEWAY_HINT_HEADERS=1` (Claude Code 2.1.273+), every request
+  says what it is. Three new `RuleMatcher` fields read those headers:
+  - `request_class` — `x-claude-code-request-class` (`main` / `subagent` /
+    `workflow` / `compaction` / `auxiliary`), case-insensitive;
+  - `agent_type` — `x-claude-code-agent-type` (e.g. `Explore`),
+    case-insensitive;
+  - `compaction: true` — `x-claude-code-compaction` is truthy (`false` is
+    rejected at load, like the other boolean matchers).
+  Typical use: auto-mode classifier / title calls (`auxiliary`) to the
+  smallest local model, compaction to a long-context profile, `Explore`
+  sub-agents to a fast one. Without the headers these matchers never match,
+  so existing rule sets behave exactly as before. `classify()` gained an
+  optional `hints` argument; both `/v1/messages` and `/v1/chat/completions`
+  pass the headers through. Docs: `docs/guides/subagent-routing.md` / `.en.md`.
+- **`slot_affinity_slots` (provider; llama.cpp only).** For
+  `llama-server --parallel N`: requests carrying a conversation key
+  (`x-claude-code-session-id`) get `id_slot = sha256(key) % N`, so every
+  turn of a conversation reuses the same slot's prompt cache instead of
+  re-prefilling on whichever slot is free. Stable across restarts; an
+  explicit `id_slot` in `extra_body` wins; off by default. Works on both the
+  native (`kind: anthropic`) and translated (`kind: openai_compat`) paths,
+  and survives every internal request rebuild (thinking strip for backends
+  without thinking, `tool_choice` emulation, `cache_control` strip,
+  OpenAI→Anthropic translation). The main conversation and its compaction
+  share a key; sub-agents and auxiliary calls get `<session>:<class>:<agent>`
+  so they do not queue on — or evict — the main conversation's slot.
+  Docs: `docs/backends/llamacpp-direct.md` / `.en.md`.
+- **`vscode-init` also writes `CLAUDE_CODE_GATEWAY_HINT_HEADERS=1`**, and
+  `doctor --check-claude-code` reports (as info) when it is missing.
+
+---
+
+## [v2.17.0] — 2026-10-24 (time-to-first-token cut-off, retry signals)
+
+### Added
+
+- **`ttft_timeout_s` (provider).** A streaming attempt that has not
+  produced its first real output (text / tool_use / thinking delta) within
+  this many seconds is abandoned before any byte reached the client, and
+  the chain moves on — e.g. a local llama-server still loading a model or
+  prefilling a long prompt while a cloud provider could answer now. Never
+  applied to the last provider of a chain (nothing to fall back to), nor to
+  non-streaming requests (`timeout_s` covers those), nor to the
+  `/v1/chat/completions` route. It is also skipped for the tool-turn
+  downgrade (`kind: openai_compat` with tools, which calls the upstream
+  non-streaming so tool calls can be repaired): there the "first output" is
+  the whole answer, and a deadline would discard slow but healthy replies —
+  use `kind: anthropic` (native `/v1/messages`) to get the cut-off for a
+  local server. It always covers the
+  wait for the first event; covering the gap between the opening events and
+  the first real output needs `empty_response_action: fallback`, which is
+  what withholds those events from the client. Abandoned attempts are
+  ordinary retryable failures: new fallback reason `ttft-timeout` (detail
+  `no-first-token`) and the self-healing guards learn from them. A cut that
+  happens after the opening events is handled like a truncated stream: the
+  withheld, unterminated preamble is never flushed to the client if the
+  rest of the chain also fails, and the abandoned upstream stream is closed
+  immediately. Default
+  `None` (off). Docs: `docs/concepts/commit-point.md` / `.en.md`.
+- **`x-should-retry` on `/v1/messages` errors.** Every Anthropic-shaped
+  error CodeRouter produces now says whether retrying can help — the header
+  Claude Code reads, with `retry-after`, to decide whether to spend its
+  retry budget: `false` for 400 / 413 (including tool-search and
+  `context_budget_action: reject` refusals), `true` for 429, and for 502
+  `true` only when at least one upstream failure was retryable (a chain that
+  failed purely on 401/403 fails the same way on every retry).
+- **Docs: the commit point** (`docs/concepts/commit-point.md` / `.en.md`) —
+  which failures before the body ships CodeRouter can still move to another
+  provider, which it cannot, and the error each chain failure becomes.
+  Recommends `empty_response_action: fallback` for Claude Code profiles.
+
+### Known limits
+
+- No keep-alive `ping` is sent while opening events are withheld; keep
+  `ttft_timeout_s` below the client's idle timeout
+  (`CLAUDE_STREAM_IDLE_TIMEOUT_MS` for Claude Code).
+
+---
+
 ## [v2.16.0] — 2026-10-24 (Claude Code 2.1.3xx behind a gateway)
 
 Claude Code 2.1.285–2.1.309 changed several defaults for sessions behind a

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: coderouter-cli
-Version: 2.16.0
+Version: 2.18.0
 Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
 Project-URL: Homepage, https://github.com/zephel01/CodeRouter
 Project-URL: Repository, https://github.com/zephel01/CodeRouter

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/adapters/anthropic_native.py

@@ -46,6 +46,7 @@ from coderouter.adapters.base import (
     StreamChunk,
     StreamTruncatedError,
 )
+from coderouter.adapters.slot_affinity import apply_slot_affinity
 from coderouter.credentials import resolve_provider_credential
 from coderouter.logging import (
     get_logger,
@@ -243,6 +244,9 @@ class AnthropicAdapter(BaseAdapter):
         body.update(dumped)
         body["model"] = self.config.model
         body["stream"] = stream
+        apply_slot_affinity(
+            body, slots=self.config.slot_affinity_slots, key=req.session_key
+        )
         return body
 
     # ------------------------------------------------------------------

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/adapters/base.py

@@ -65,6 +65,9 @@ class ChatRequest(BaseModel):
 
     # CodeRouter-specific extension (not sent upstream)
     profile: str | None = Field(default=None, exclude=True)
+    # v2.18.0: conversation key for KV-slot affinity (see
+    # ``ProviderConfig.slot_affinity_slots``). Never serialized.
+    session_key: str | None = Field(default=None, exclude=True)
 
 
 class ChatResponse(BaseModel):
@@ -200,6 +203,23 @@ class StreamTruncatedError(AdapterError):
         self.tool_call_in_flight = tool_call_in_flight
 
 
+# v2.17.0: the provider did not produce its first real streamed output within
+# ``ProviderConfig.ttft_timeout_s``. Raised by the engine (not the adapter),
+# before any byte reached the client, so the chain falls back normally and the
+# self-healing guards learn from it like any other retryable failure.
+class TTFTTimeoutError(AdapterError):
+    """No first token within ``ttft_timeout_s``."""
+
+    def __init__(self, *, provider: str, ttft_timeout_s: float) -> None:
+        super().__init__(
+            f"timeout: no first token within {ttft_timeout_s:g}s (ttft_timeout_s)",
+            provider=provider,
+            status_code=None,
+            retryable=True,
+        )
+        self.ttft_timeout_s = ttft_timeout_s
+
+
 # v0.6-B: per-call overrides resolved from the active profile. The engine
 # builds one instance per request (since a profile is invariant across its
 # chain) and threads it through every adapter call on that chain. Adapters

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/adapters/openai_compat.py

@@ -30,6 +30,7 @@ from coderouter.adapters.base import (
     StreamChunk,
     StreamTruncatedError,
 )
+from coderouter.adapters.slot_affinity import apply_slot_affinity
 from coderouter.credentials import resolve_provider_credential
 from coderouter.logging import (
     get_logger,
@@ -182,6 +183,11 @@ class OpenAICompatAdapter(BaseAdapter):
             value = getattr(request, field, None)
             if value is not None:
                 body[field] = value
+        apply_slot_affinity(
+            body,
+            slots=self.config.slot_affinity_slots,
+            key=getattr(request, "session_key", None),
+        )
         if stream:
             # Request a terminal usage chunk. Providers that honor this
             # (OpenAI, OpenRouter, Ollama >=0.x) will send one extra chunk

coderouter_cli-2.18.0/coderouter/adapters/slot_affinity.py

@@ -0,0 +1,54 @@
+"""v2.18.0: KV-slot affinity for llama-server (``id_slot``).
+
+llama-server started with ``--parallel N`` serves N slots, each holding its
+own prompt cache. Without a hint it picks a free slot, so consecutive turns
+of one conversation can land on different slots and re-prefill the whole
+prompt. Sending ``id_slot`` pins a conversation to one slot.
+
+The slot is ``sha256(key) % N`` — stable across processes and restarts
+(Python's ``hash`` is salted per process).
+"""
+
+from __future__ import annotations
+
+import hashlib
+from collections.abc import Mapping
+from typing import Any
+
+# Request classes whose prompt shares the main conversation's prefix: they
+# belong on the main conversation's slot. Everything else (sub-agents,
+# auxiliary calls such as the auto-mode classifier or titles, workflows) has
+# its own prompt and gets its own key, so it neither waits on the main slot
+# nor evicts its cache.
+_MAIN_CLASSES = frozenset({"", "main", "compaction"})
+
+
+def affinity_key(hints: Mapping[str, str]) -> str | None:
+    """Conversation key for slot affinity, from Claude Code's hint headers.
+
+    ``None`` when there is no session id. ``main`` / ``compaction`` (and a
+    missing class) share ``<session>``; other classes get
+    ``<session>:<class>:<agent_type>``.
+    """
+    session = hints.get("session_id")
+    if not session:
+        return None
+    request_class = hints.get("request_class", "").lower()
+    if request_class in _MAIN_CLASSES:
+        return session
+    return f"{session}:{request_class}:{hints.get('agent_type', '').lower()}"
+
+
+def slot_for(key: str, slots: int) -> int:
+    digest = hashlib.sha256(key.encode("utf-8")).digest()
+    return int.from_bytes(digest[:8], "big") % slots
+
+
+def apply_slot_affinity(body: dict[str, Any], *, slots: int | None, key: str | None) -> None:
+    """Set ``body["id_slot"]`` when affinity is configured and a key exists.
+
+    An ``id_slot`` already present (e.g. from ``extra_body``) wins.
+    """
+    if not slots or not key or "id_slot" in body:
+        return
+    body["id_slot"] = slot_for(key, slots)

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/cli.py

@@ -483,7 +483,8 @@ def _build_parser() -> argparse.ArgumentParser:
         help=(
             "v2.16.0: do not write the Claude Code gateway settings "
             "(ENABLE_TOOL_SEARCH=false, CLAUDE_CODE_AUTO_MODE_SERVER=0, "
-            "and --max-context-tokens). Use when the chain is "
+            "CLAUDE_CODE_GATEWAY_HINT_HEADERS=1 and --max-context-tokens). "
+            "Use when the chain is "
             "api.anthropic.com only."
         ),
     )

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/config/schemas.py

@@ -550,6 +550,45 @@ class ProviderConfig(BaseModel):
         description="If true, only used when ALLOW_PAID=true (plan.md §2.3).",
     )
     timeout_s: float = Field(default=30.0, ge=1.0, le=86400.0)
+    # v2.17.0: time-to-first-token cut-off for streaming requests. When the
+    # provider has not produced its first real output (text / tool_use /
+    # thinking delta) within this many seconds of the attempt starting, the
+    # attempt is abandoned and the chain moves on — before any byte reached
+    # the client. Never applied to the last provider of a chain (nothing to
+    # move on to), nor to non-streaming requests (``timeout_s`` covers
+    # those). Before the first *event* it always applies; between the first
+    # event and the first real output it needs
+    # ``empty_response_action: fallback`` on the profile, which is what
+    # withholds those opening events from the client. ``None`` (default)
+    # disables it. Typical use: a local llama-server still loading a model
+    # or chewing through a long prompt while a cloud fallback could answer.
+    # v2.18.0: KV-slot affinity for llama-server started with
+    # ``--parallel N``. When set to N and the request carries a conversation
+    # key (Claude Code's ``x-claude-code-session-id`` hint header), the body
+    # gets ``id_slot = stable_hash(key) % N`` so every turn of a conversation
+    # lands on the same slot and reuses its prompt cache instead of
+    # re-prefilling on whichever slot is free. llama.cpp only (other servers
+    # ignore or reject ``id_slot``); ``None`` (default) sends nothing.
+    slot_affinity_slots: int | None = Field(
+        default=None,
+        ge=1,
+        le=256,
+        description=(
+            "v2.18.0: llama-server --parallel N. Pins each conversation to "
+            "slot hash(session) % N via id_slot so its KV cache is reused. "
+            "llama.cpp only. None disables."
+        ),
+    )
+    ttft_timeout_s: float | None = Field(
+        default=None,
+        gt=0.0,
+        le=3600.0,
+        description=(
+            "v2.17.0: seconds to wait for the first real streamed output "
+            "before falling back to the next provider (not applied to the "
+            "last provider in a chain). None disables."
+        ),
+    )
 
     # v2.6 language-tax track: path to a LOCAL ``tokenizer.json`` for this
     # provider's model, used to measure the CJK over-count vs the char/4
@@ -1617,6 +1656,21 @@ class RuleMatcher(BaseModel):
     # declares any tools); ``False`` is rejected by ``_exactly_one``
     # since a "no-tools" rule would shadow the default fall-through.
     has_tools: bool | None = None
+    # v2.18.0: Claude Code gateway hint headers. Claude Code 2.1.273+ sends
+    # these to a custom ANTHROPIC_BASE_URL when
+    # ``CLAUDE_CODE_GATEWAY_HINT_HEADERS=1`` is set (``coderouter
+    # vscode-init`` sets it). They say what a request *is* — something the
+    # body heuristics above can only guess:
+    #   * ``request_class`` — ``x-claude-code-request-class``: ``main`` /
+    #     ``subagent`` / ``workflow`` / ``compaction`` / ``auxiliary``
+    #     (auto-mode classifier, titles, …). Case-insensitive exact match.
+    #   * ``agent_type`` — ``x-claude-code-agent-type`` (e.g. ``Explore``),
+    #     case-insensitive exact match.
+    #   * ``compaction: True`` — ``x-claude-code-compaction`` is truthy.
+    # Without the headers these matchers simply never match.
+    request_class: str | None = None
+    agent_type: str | None = None
+    compaction: bool | None = None
 
     _MATCHER_FIELDS: tuple[str, ...] = (
         "has_image",
@@ -1627,12 +1681,15 @@ class RuleMatcher(BaseModel):
         "content_token_count_min",
         "has_tools",
         "cjk_ratio_min",
+        "request_class",
+        "agent_type",
+        "compaction",
     )
 
     # Boolean matcher fields carry meaning only at ``True`` — the runtime
     # evaluator uses ``is True`` — so a ``False`` value is a dead rule that
     # never matches. ``_exactly_one`` rejects it explicitly (see below).
-    _BOOL_MATCHER_FIELDS: tuple[str, ...] = ("has_image", "has_tools")
+    _BOOL_MATCHER_FIELDS: tuple[str, ...] = ("has_image", "has_tools", "compaction")
 
     @model_validator(mode="after")
     def _exactly_one(self) -> Self:

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/doctor_claude_code.py

@@ -125,6 +125,18 @@ def check(version: Version | None, env: Mapping[str, str]) -> list[ClaudeCodeFin
             )
         )
 
+    if env.get("CLAUDE_CODE_GATEWAY_HINT_HEADERS", "").lower() in ("1", "true"):
+        out.append(ClaudeCodeFinding("ok", "CLAUDE_CODE_GATEWAY_HINT_HEADERS=1"))
+    else:
+        out.append(
+            ClaudeCodeFinding(
+                "info",
+                "CLAUDE_CODE_GATEWAY_HINT_HEADERS is not set: auto_router rules "
+                "on request_class / agent_type / compaction (v2.18.0) never "
+                "match without it.",
+            )
+        )
+
     if env.get("CLAUDE_CODE_AUTO_MODE_SERVER") == "0":
         out.append(ClaudeCodeFinding("ok", "CLAUDE_CODE_AUTO_MODE_SERVER=0"))
     else:

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/ingress/anthropic_errors.py

@@ -33,6 +33,13 @@ terminal error 429                    429     ``rate_limit_error``
 anything else                         502     ``api_error``
 ====================================  ======  ========================
 
+v2.17.0: every mapped response also carries ``x-should-retry`` — the header
+Claude Code reads (with ``retry-after``) to decide whether to spend its retry
+budget. ``false`` for the 4xx rows (resending the same request cannot
+help), ``true`` for 429, and for 502 ``true`` only when at least one
+upstream failure was retryable — a chain that failed purely on, say, 401s
+will fail the same way on every retry.
+
 Upstream 401 / 403 deliberately fall into the last row: they describe
 CodeRouter's *own* upstream credentials, not the client's, and handing a
 401 to Claude Code would send the user to ``/login`` for nothing.
@@ -164,17 +171,26 @@ def map_no_providers_error(exc: NoProvidersAvailableError) -> AnthropicErrorResp
             msg = upstream_message(err)
             if not msg.lower().startswith(PROMPT_TOO_LONG_PREFIX):
                 msg = f"{PROMPT_TOO_LONG_PREFIX}: {msg}"
-            return AnthropicErrorResponse(400, "invalid_request_error", msg)
+            return AnthropicErrorResponse(
+                400, "invalid_request_error", msg, {"x-should-retry": "false"}
+            )
 
     terminal = errors[-1] if errors else None
     if terminal is not None and terminal.status_code is not None:
         status = terminal.status_code
         if status in (400, 422) and not terminal.retryable:
-            return AnthropicErrorResponse(400, "invalid_request_error", upstream_message(terminal))
+            return AnthropicErrorResponse(
+                400,
+                "invalid_request_error",
+                upstream_message(terminal),
+                {"x-should-retry": "false"},
+            )
         if status == 413:
-            return AnthropicErrorResponse(413, "request_too_large", upstream_message(terminal))
+            return AnthropicErrorResponse(
+                413, "request_too_large", upstream_message(terminal), {"x-should-retry": "false"}
+            )
         if status == 429:
-            headers: dict[str, str] = {}
+            headers: dict[str, str] = {"x-should-retry": "true"}
             retry_after = getattr(terminal, "retry_after", None)
             if retry_after:
                 headers["retry-after"] = retry_after
@@ -182,4 +198,7 @@ def map_no_providers_error(exc: NoProvidersAvailableError) -> AnthropicErrorResp
                 429, "rate_limit_error", upstream_message(terminal), headers
             )
 
-    return AnthropicErrorResponse(502, "api_error", redact(str(exc)))
+    retry = any(e.retryable for e in errors)
+    return AnthropicErrorResponse(
+        502, "api_error", redact(str(exc)), {"x-should-retry": "true" if retry else "false"}
+    )

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/ingress/anthropic_routes.py

@@ -31,6 +31,7 @@ from fastapi import APIRouter, Header, HTTPException, Request
 from fastapi.responses import JSONResponse, Response, StreamingResponse
 
 from coderouter.adapters.base import StreamTruncatedError
+from coderouter.adapters.slot_affinity import affinity_key
 from coderouter.guards.context_budget import ContextBudgetRejectError
 from coderouter.guards.tool_loop import ToolCountExceededError, ToolLoopBreakError
 from coderouter.ingress.anthropic_errors import (
@@ -49,7 +50,7 @@ from coderouter.routing import (
     MidStreamError,
     NoProvidersAvailableError,
 )
-from coderouter.routing.auto_router import RESERVED_PROFILE_NAME, classify
+from coderouter.routing.auto_router import RESERVED_PROFILE_NAME, classify, extract_gateway_hints
 from coderouter.routing.fallback_trace import (
     SSE_FALLBACK_EVENT,
     current_fallback_trace,
@@ -226,6 +227,11 @@ async def messages(
     if anthropic_beta:
         anth_req.anthropic_beta = anthropic_beta
 
+    # v2.18.0: Claude Code gateway hint headers — routing matchers below,
+    # and the conversation key for llama-server KV-slot affinity.
+    gateway_hints = extract_gateway_hints(request.headers)
+    anth_req.session_key = affinity_key(gateway_hints)
+
     # Profile selection — body field wins over header (same policy as OpenAI route).
     if anth_req.profile is None and x_coderouter_profile:
         anth_req.profile = x_coderouter_profile
@@ -253,7 +259,7 @@ async def messages(
     # AnthropicRequest), so both OpenAI and Anthropic ingress use the same
     # classifier without a shared request shim.
     if anth_req.profile is None and config.default_profile == RESERVED_PROFILE_NAME:
-        anth_req.profile = classify(payload, config)
+        anth_req.profile = classify(payload, config, gateway_hints)
 
     if anth_req.profile is not None:
         try:
@@ -276,7 +282,9 @@ async def messages(
         )
         if ts_action == "reject":
             return _anthropic_error_response(
-                AnthropicErrorResponse(400, "invalid_request_error", REJECT_MESSAGE)
+                AnthropicErrorResponse(
+                    400, "invalid_request_error", REJECT_MESSAGE, {"x-should-retry": "false"}
+                )
             )
         if ts_action == "expand":
             anth_req = expand_tool_search(anth_req)
@@ -290,7 +298,9 @@ async def messages(
         # v2.16.0: ``context_budget_action: reject`` — Anthropic's own
         # overflow answer, so Claude Code compacts the conversation.
         return _anthropic_error_response(
-            AnthropicErrorResponse(400, "invalid_request_error", str(exc))
+            AnthropicErrorResponse(
+                400, "invalid_request_error", str(exc), {"x-should-retry": "false"}
+            )
         )
 
     if anth_req.stream:
@@ -362,7 +372,9 @@ async def messages(
             # swap) — same answer as the pre-dispatch check above.
             await _aclose_quietly(source)
             return _anthropic_error_response(
-                AnthropicErrorResponse(400, "invalid_request_error", str(exc))
+                AnthropicErrorResponse(
+                400, "invalid_request_error", str(exc), {"x-should-retry": "false"}
+            )
             )
         except (ToolLoopBreakError, ToolCountExceededError, MidStreamError) as exc:
             # Policy errors and mid-stream failures keep their existing
@@ -415,7 +427,9 @@ async def messages(
     except ContextBudgetRejectError as exc:
         # See the streaming branch: the engine can re-run the guard.
         return _anthropic_error_response(
-            AnthropicErrorResponse(400, "invalid_request_error", str(exc))
+            AnthropicErrorResponse(
+                400, "invalid_request_error", str(exc), {"x-should-retry": "false"}
+            )
         )
     except NoProvidersAvailableError as exc:
         # v2.16.0: Anthropic-shaped error with the upstream's own wording

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/ingress/openai_routes.py

@@ -32,9 +32,10 @@ from fastapi import APIRouter, Header, HTTPException, Request
 from fastapi.responses import JSONResponse, StreamingResponse
 
 from coderouter.adapters.base import ChatRequest
+from coderouter.adapters.slot_affinity import affinity_key
 from coderouter.logging import get_logger
 from coderouter.routing import FallbackEngine, NoProvidersAvailableError
-from coderouter.routing.auto_router import RESERVED_PROFILE_NAME, classify
+from coderouter.routing.auto_router import RESERVED_PROFILE_NAME, classify, extract_gateway_hints
 from coderouter.routing.fallback_trace import current_fallback_trace
 
 router = APIRouter()
@@ -199,6 +200,10 @@ async def chat_completions(
     except Exception as exc:  # pydantic.ValidationError, etc.
         raise HTTPException(status_code=422, detail=str(exc)) from exc
 
+    # v2.18.0: gateway hint headers (routing matchers + KV-slot affinity key).
+    gateway_hints = extract_gateway_hints(request.headers)
+    chat_req.session_key = affinity_key(gateway_hints)
+
     # Header-based override (body wins if both are set — see module docstring)
     if chat_req.profile is None and x_coderouter_profile:
         chat_req.profile = x_coderouter_profile
@@ -226,7 +231,7 @@ async def chat_completions(
     # When inactive, the engine still falls through to
     # ``config.default_profile`` on its own — same semantics as pre-v1.6.
     if chat_req.profile is None and config.default_profile == RESERVED_PROFILE_NAME:
-        chat_req.profile = classify(payload, config)
+        chat_req.profile = classify(payload, config, gateway_hints)
 
     # Validate profile exists before we kick off any upstream call
     if chat_req.profile is not None:

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/routing/auto_router.py

@@ -36,6 +36,7 @@ from __future__ import annotations
 
 import logging
 import re
+from collections.abc import Mapping
 from typing import TYPE_CHECKING, Any
 
 from coderouter.config.schemas import AutoRouterConfig, AutoRouteRule, RuleMatcher
@@ -169,6 +170,32 @@ def _has_tools_in_body(body: dict[str, Any]) -> bool:
     return isinstance(functions, list) and len(functions) > 0
 
 
+# v2.18.0: Claude Code gateway hint headers → hint keys.
+_HINT_HEADERS: dict[str, str] = {
+    "x-claude-code-request-class": "request_class",
+    "x-claude-code-agent-type": "agent_type",
+    "x-claude-code-compaction": "compaction",
+    "x-claude-code-session-id": "session_id",
+    "x-claude-code-prompt-id": "prompt_id",
+}
+_TRUTHY = frozenset({"1", "true", "yes", "on"})
+
+
+def extract_gateway_hints(headers: Mapping[str, str]) -> dict[str, str]:
+    """Pull Claude Code's gateway hint headers out of a request.
+
+    Returns only the hints that are present (lower-cased keys from
+    :data:`_HINT_HEADERS`). Header lookup is case-insensitive whenever the
+    mapping is (Starlette's ``Headers`` is).
+    """
+    out: dict[str, str] = {}
+    for header, key in _HINT_HEADERS.items():
+        value = headers.get(header)
+        if value:
+            out[key] = value.strip()
+    return out
+
+
 def _match_rule(
     rule: AutoRouteRule,
     message: dict[str, Any] | None,
@@ -176,8 +203,17 @@ def _match_rule(
     model: str | None,
     estimated_tokens: int,
     has_tools: bool,
+    hints: Mapping[str, str] | None = None,
 ) -> bool:
     m = rule.match
+    if m.request_class is not None:
+        value = (hints or {}).get("request_class")
+        return value is not None and value.lower() == m.request_class.lower()
+    if m.agent_type is not None:
+        value = (hints or {}).get("agent_type")
+        return value is not None and value.lower() == m.agent_type.lower()
+    if m.compaction is True:
+        return (hints or {}).get("compaction", "").lower() in _TRUTHY
     if m.has_image is True:
         return message is not None and _has_image(message)
     if m.code_fence_ratio_min is not None:
@@ -237,7 +273,11 @@ def _extract_model(body: dict[str, Any]) -> str | None:
     return None
 
 
-def classify(body: dict[str, Any], config: CodeRouterConfig) -> str:
+def classify(
+    body: dict[str, Any],
+    config: CodeRouterConfig,
+    hints: Mapping[str, str] | None = None,
+) -> str:
     """Resolve an incoming request body to a profile name.
 
     Rule order (first match wins) is determined by:
@@ -252,6 +292,10 @@ def classify(body: dict[str, Any], config: CodeRouterConfig) -> str:
 
     Emits one of two log events: ``auto-router-resolved`` on match, or
     ``auto-router-fallthrough`` on default-rule fall.
+
+    v2.18.0: ``hints`` carries Claude Code's gateway hint headers (see
+    :func:`extract_gateway_hints`) for the ``request_class`` /
+    ``agent_type`` / ``compaction`` matchers.
     """
     user_msg = _latest_user_message(body)
     text = _extract_text(user_msg) if user_msg is not None else ""
@@ -294,7 +338,7 @@ def classify(body: dict[str, Any], config: CodeRouterConfig) -> str:
     # present — they short out via ``_match_rule``'s message-None
     # handling.
     for rule in rules:
-        if _match_rule(rule, user_msg, text, model, estimated_tokens, has_tools):
+        if _match_rule(rule, user_msg, text, model, estimated_tokens, has_tools, hints):
             _emit_resolved(rule, user_msg, text, model, estimated_tokens, has_tools)
             return rule.profile
 

{coderouter_cli-2.16.0 → coderouter_cli-2.18.0}/coderouter/routing/capability.py

@@ -194,12 +194,13 @@ def strip_thinking(request: AnthropicRequest) -> AnthropicRequest:
 
     # model_dump() serializes extras; roundtripping via validate rebuilds a
     # clean instance without the dropped key. exclude=True fields (profile,
-    # anthropic_beta) are omitted by model_dump, so we reassign them.
+    # anthropic_beta, session_key) are omitted by model_dump, so we reassign them.
     dumped = request.model_dump()
     dumped.pop("thinking", None)
     stripped = AnthropicRequest.model_validate(dumped)
     stripped.profile = request.profile
     stripped.anthropic_beta = request.anthropic_beta
+    stripped.session_key = request.session_key
     return stripped
 
 
@@ -450,6 +451,7 @@ def emulate_tool_choice(request: AnthropicRequest) -> AnthropicRequest:
     emulated = AnthropicRequest.model_validate(dumped)
     emulated.profile = request.profile
     emulated.anthropic_beta = request.anthropic_beta
+    emulated.session_key = request.session_key
     return emulated
 
 
@@ -522,6 +524,7 @@ def strip_cache_control(request: AnthropicRequest) -> tuple[AnthropicRequest, in
     stripped = AnthropicRequest.model_validate(dumped)
     stripped.profile = request.profile
     stripped.anthropic_beta = request.anthropic_beta
+    stripped.session_key = request.session_key
     return stripped, removed