llm-switcher 1.6.11 → 1.6.13

package/CHANGELOG.md

@@ -1,5 +1,32 @@
 # Changelog — LLM Switcher
 
+## Release 1.6.13 — 2026-10-12
+
+- **`switch on` with no profile.** `switch on` gives each tool the profile that it had before `switch off`. A tool that was off stays off. A tool that is on keeps its profile.
+- **System turns on the translated route.** A `system` turn inside the `messages` of an Anthropic request now goes into the system prompt when the proxy translates the request.
+- **System messages on the direct Anthropic route.** A folded `[system] ` text keeps its place between the turns around it. It never moves past an assistant turn. A system message with no text is dropped, and the healer note now says so. A system message after a final assistant prefill becomes the last user turn, so the request no longer ends on the prefill.
+- **Gemini `_placeholder` in `anyOf`.** If one branch of an `anyOf` declares a real `_placeholder` property, the proxy keeps the key in the arguments.
+- **Gemini `_placeholder` and tool names.** If two padded tools map back to one client tool name, the proxy removes nothing for that name.
+- **Retry to another upstream format.** A request that the proxy builds again for another upstream format gets a new tool vocabulary and a new placeholder rule.
+- **Reasoning before a compaction item.** In a Codex request, reasoning that waits for an assistant turn is dropped at a `compaction` item.
+- **Model list cache time.** A cache entry with a time in the future is stale. It no longer stays fresh forever.
+- **More token count forms.** A context overflow message with `128k`, `1m`, `131 072` (no-break space), `131'072` or `1,31,072` gives the full number to Claude Code. A decimal number after `limit` is not read as a token count. A plain `limit N` counts as the maximum only when `tokens` follows the number. A count of zero, or a count too large to be exact, is not read as a token count.
+- **Output limit for every profile.** If a converted request finds no fresh model list for its profile, the proxy reads the list in the background. The request does not wait. The next request gets the limit. Each profile keeps its own limit, also on a shared base URL. The proxy reads `<baseURL>/models` with the API key of the profile, except for a profile with an `endpoints` override. That profile gets no output limit. The `/v1/models` list also does not send the key of that profile to `<baseURL>/models`.
+- **Goal check when the evaluator call fails.** If the native evaluator call fails, the goal check does not let the goal through. A test now holds this behavior.
+- **Test cleanup.** Every test file that makes a temp directory deletes it, also when a test fails. `npm test` deletes `node-compile-cache` only when npm made it for this run. A failed delete no longer changes the exit status of the tests.
+
+## Release 1.6.12 — 2026-10-11
+
+- **Reasoning from an aborted turn.** In a Codex request, reasoning that waits for an assistant turn is dropped at the next user message. Separate reasoning items are joined with a newline.
+- **Token counts with separators.** A context overflow message with `131,072`, `131.072`, `131 072` or `131_072` gives the full number to Claude Code. A message without numbers keeps the plain form.
+- **Context overflow over WebSocket.** A Codex turn on WebSocket fails with the code `context_length_exceeded`, the same as on HTTP.
+- **Goal check and images.** The goal-check request to a chat upstream uses the same image retry as other requests.
+- **Output limit cache.** The output limit comes only from the cache entry of the same profile, and only when the entry is less than one hour old. The cache key ignores the case of scheme and host and a trailing slash. A new model list is merged per model, so a list with only output limits keeps the cached windows. A cached output limit is kept only while the newest list still states it.
+- **System messages on the direct Anthropic route.** A `system` message inside `messages` is no longer lost. Its text goes into the next user turn, or into the previous one if no user turn follows, with the prefix `[system] `.
+- **Gemini `_placeholder` in tool calls.** If the model fills the `_placeholder` property that the proxy added, the proxy removes it from the arguments. This applies to stream and non-stream answers for every client format, Code Assist (agy) included, and only where the proxy added the property.
+- **Test temp files.** The tests delete all their temp directories, also when a test fails. `npm test` deletes the `node-compile-cache` directory that npm made at the start of the run.
+- **`replayReasoning` documentation.** The README files now say that `reasoning_content` goes out only on a request with thinking on.
+
 ## Release 1.6.11 — 2026-10-11
 
 - **Output limit.** The proxy keeps the output limit of each model from the `/models` list of the profile (`max_output_tokens` or `top_provider.max_completion_tokens`) in `model-windows.json`. If the client asks for more tokens than this limit, the proxy sends the limit. This applies to `max_tokens`, `max_completion_tokens` and `maxOutputTokens`. If the limit is not known, the value does not change. The limit does not apply to the anthropic-to-anthropic passthrough (direct mode, outFormat `anthropic`, and the native goal-check request). These requests go to the upstream unchanged.

package/README.md

@@ -519,7 +519,8 @@ switch status                  # Display status for all active CLI targets
 switch version                 # Show the version and tell you when npm has a newer one
 switch doctor                  # Audit environment, settings & routing
 switch update                  # Install the newest release and restart the gateway on it
-switch on [profile]            # Start the gateway and activate a profile
+switch on                      # Start the gateway; each tool gets back the profile it had before `switch off`
+switch on <profile>            # Start the gateway and activate a profile
 switch <profile>               # Activate a profile for both tools
 switch claude <profile>        # Set the active profile for Claude Code only
 switch codex <profile>         # Set the active profile for Codex only
@@ -869,7 +870,7 @@ Because of this split, a provider fingerprint is never a rule in this gateway. I
 | `LLM_SWITCHER_BIFROST_RETRY_MS` | Time before the gateway asks intact again after a failed Bifrost lookup (default 30000 ms). |
 | `x-llm-profile: <key>` header (alias `x-profile`) or `?profile=<key>` | Route a single request through a specific profile. An unknown key returns HTTP 400 instead of silently falling back. |
 | `profile.thinkingMode` | `auto` (default, for gateways like intact or 9Router): restore stripped thinking, inject a `<think>` guide for non-reasoning models, send `thinking` + `reasoning_effort`, and send the earlier thinking as `reasoning_content` on assistant tool-call turns. `native` (strict OpenAI APIs): send only `reasoning_effort` when the client asks, never touch the prompt, use `max_completion_tokens`. Native mode never sends `reasoning_content` unless the profile sets `replayReasoning: true`. `off`: never send reasoning parameters. |
-| `profile.replayReasoning` | `true` sends the earlier thinking as `reasoning_content` on assistant tool-call turns in `native` mode. DeepSeek and Kimi thinking mode need it. The default is `false`. `auto` mode always sends the field, and `off` never sends it. The model name and the client request do not change this rule. |
+| `profile.replayReasoning` | `true` sends the earlier thinking as `reasoning_content` on assistant tool-call turns in `native` mode. DeepSeek and Kimi thinking mode need it. The default is `false`. The field goes out only on a request with thinking on. In `native` mode, thinking is on when the client asks for it. In `auto` mode, the flag is not needed: thinking is on when the client asks for it, or when the model reasons natively and the client did not turn thinking off. `off` never sends the field. |
 | `profile.thinkingEffort` | `auto` (default, caller/client controls effort), or an explicit override level: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`, `ultra`. Overrides client effort before routing upstream. Automatically maps to the nearest equivalent level accepted by the target upstream (e.g. OpenAI ceiling `high`, Anthropic `max`, Gemini budget bounds). In the Web UI, this selector dynamically hides with a guidance hint when Bifrost passthrough is active. |
 | `profile.endpoints.countTokens` | Override the Anthropic `count_tokens` URL. |
 | `profile.endpoints` | Override upstream URLs per format: `{ "openai-chat": "...", "anthropic": "...", "vertex": "https://.../models/{model}:{action}" }`. |

package/README.vi.md

@@ -516,7 +516,8 @@ switch status                  # Xem trạng thái kích hoạt của tất cả
 switch version                 # Xem version đang chạy và báo khi npm có bản mới
 switch doctor                  # Quét & thanh tra toàn bộ môi trường, settings và định tuyến
 switch update                  # Cài bản mới nhất và khởi động lại gateway trên bản đó
-switch on [profile]            # Khởi động gateway và kích hoạt một profile
+switch on                      # Khởi động gateway; mỗi công cụ lấy lại profile nó có trước `switch off`
+switch on <profile>            # Khởi động gateway và kích hoạt một profile
 switch <profile>               # Kích hoạt một profile cho cả hai công cụ
 switch claude <profile>        # Đặt profile kích hoạt riêng cho Claude Code
 switch codex <profile>         # Đặt profile kích hoạt riêng cho Codex
@@ -864,7 +865,7 @@ Nhờ cách chia này, fingerprint của provider không bao giờ là rule tron
 | `LLM_SWITCHER_BIFROST_RETRY_MS` | Thời gian trước khi gateway hỏi lại intact sau một lần tra cứu Bifrost thất bại (mặc định 30000 ms). |
 | Header `x-llm-profile: <key>` (tên khác `x-profile`) hoặc `?profile=<key>` | Định tuyến riêng 1 request qua profile chỉ định. Key không tồn tại trả HTTP 400 thay vì âm thầm dùng profile khác. |
 | `profile.thinkingMode` | `auto` (mặc định, cho gateway như intact hoặc 9Router): phục hồi thinking bị xoá, inject hướng dẫn `<think>` cho model không có reasoning, gửi `thinking` + `reasoning_effort`, và gửi lại thinking trước đó thành `reasoning_content` ở lượt assistant có tool call. `native` (API OpenAI nghiêm ngặt): chỉ gửi `reasoning_effort` khi client yêu cầu, không sửa prompt, dùng `max_completion_tokens`; không gửi `reasoning_content` trừ khi profile đặt `replayReasoning: true`. `off`: không bao giờ gửi tham số reasoning. |
-| `profile.replayReasoning` | `true`: ở chế độ `native`, gửi lại thinking trước đó thành `reasoning_content` ở lượt assistant có tool call. Thinking mode của DeepSeek và Kimi cần field này. Mặc định là `false`. `auto` luôn gửi, `off` không bao giờ gửi. Tên model và request của client không đổi quy tắc này. |
+| `profile.replayReasoning` | `true`: ở chế độ `native`, gửi lại thinking trước đó thành `reasoning_content` ở lượt assistant có tool call. Thinking mode của DeepSeek và Kimi cần field này. Mặc định là `false`. Field chỉ được gửi khi request bật thinking. Ở `native`, thinking bật khi client yêu cầu. Ở `auto`, không cần cờ này: thinking bật khi client yêu cầu, hoặc khi model có reasoning sẵn và client không tắt thinking. `off` không bao giờ gửi field này. |
 | `profile.thinkingEffort` | `auto` (mặc định, client tự quyết định), hoặc đặt cố định mức override: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`, `ultra`. Ghi đè mức reasoning effort từ công cụ trước khi gửi lên upstream. Tự động ánh xạ về mức tương đương gần nhất mà upstream đích hỗ trợ (OpenAI trần `high`, Anthropic trần `max`, Gemini điều chỉnh `thinkingBudget`). Trên Web UI, tùy chọn này tự động ẩn cùng thông báo gợi ý khi profile bật chế độ Bifrost passthrough. |
 | `profile.endpoints.countTokens` | Ghi đè URL `count_tokens` của Anthropic. |
 | `profile.endpoints` | Ghi đè URL upstream theo từng format: `{ "openai-chat": "...", "anthropic": "...", "vertex": "https://.../models/{model}:{action}" }`. |

package/formats.mjs

@@ -24,7 +24,7 @@
 // }
 // ============================================================
 
-import { sanitizeToolName, toolNameMapForIR, rewriteToolNameMentions, rewriteToolSearchArgs } from './toolvocab.mjs';
+import { sanitizeToolName, toolNameMapForIR, rewriteToolNameMentions, rewriteToolSearchArgs, createToolVocab, NULL_TOOL_VOCAB } from './toolvocab.mjs';
 import { decodeCompaction, COMPACTION_INSTRUCTION } from './compaction.mjs';
 
 export const OUT_FORMATS = ['openai-chat', 'anthropic', 'vertex'];
@@ -455,9 +455,22 @@ function anthropicToIR(payload) {
     }
   }
 
+  // Messages has no system role. Left in place, the emitters send such a turn as user text, so fold
+  // it into the system prompt, as responsesToIR and the Chat reader already do.
+  const foldSystemTurn = (msg) => {
+    if (msg.role !== 'system' && msg.role !== 'developer') return false;
+    const text =
+      typeof msg.content === 'string'
+        ? msg.content
+        : (msg.content || []).filter(p => p && p.type === 'text').map(p => p.text).join('\n');
+    if (text.trim()) ir.system = ir.system ? `${ir.system}\n\n${text}` : text;
+    return true;
+  };
+
   if (Array.isArray(payload.messages)) {
     for (const msg of payload.messages) {
       if (!msg) continue;
+      if (foldSystemTurn(msg)) continue;
       if (typeof msg.content === 'string') {
         ir.messages.push({ role: msg.role, content: msg.content });
         continue;
@@ -693,10 +706,12 @@ function responsesToIR(payload, { allowOpaqueCompaction = false } = {}) {
     if (text) ir.system = ir.system ? `${ir.system}\n\n${text}` : text;
   };
 
-  // A reasoning item precedes the assistant output it belongs to.
+  // A reasoning item precedes the assistant output it belongs to. A user message or a compaction item
+  // ends that turn, so reasoning still pending there came from an aborted turn and is dropped.
   let reasoning = '';
+  const joinLines = (a, b) => (a ? `${a}\n${b}` : b);
   const takeReasoning = (m) => {
-    if (reasoning && m?.role === 'assistant') { m.reasoning = (m.reasoning || '') + reasoning; reasoning = ''; }
+    if (reasoning && m?.role === 'assistant') { m.reasoning = joinLines(m.reasoning || '', reasoning); reasoning = ''; }
   };
 
   // Parallel tool calls arrive as consecutive items -> merge into 1 assistant turn,
@@ -715,7 +730,7 @@ function responsesToIR(payload, { allowOpaqueCompaction = false } = {}) {
     ir.messages.push({ role: 'user', content: input });
   } else if (Array.isArray(input)) {
     for (const item of input) {
-      if (typeof item === 'string') { pushText('user', item); continue; }
+      if (typeof item === 'string') { reasoning = ''; pushText('user', item); continue; }
       if (!item || typeof item !== 'object') continue;
       // EasyInputMessage ({role, content}) doesn't require `type`.
       if (item.type === 'message' || (!item.type && item.role)) {
@@ -739,6 +754,7 @@ function responsesToIR(payload, { allowOpaqueCompaction = false } = {}) {
           continue;
         }
         const role = item.role === 'assistant' ? 'assistant' : 'user';
+        if (role === 'user') reasoning = '';
         if (parts.every(p => p.type === 'text')) {
           pushText(role, parts.map(p => p.text).join('\n'));
         } else {
@@ -747,7 +763,7 @@ function responsesToIR(payload, { allowOpaqueCompaction = false } = {}) {
         if (role === 'assistant') takeReasoning(ir.messages[ir.messages.length - 1]);
       } else if (item.type === 'reasoning') {
         const texts = (Array.isArray(item.summary) ? item.summary : []).map(x => x?.text).filter(t => typeof t === 'string' && t);
-        reasoning += texts.join('\n');
+        if (texts.length) reasoning = joinLines(reasoning, texts.join('\n'));
       } else if (item.type === 'function_call') {
         pushCall({ id: item.call_id || item.id || null, name: responsesToolName(item.namespace, item.name), args: parseArgs(item.arguments) });
       } else if (item.type === 'custom_tool_call') {
@@ -760,6 +776,7 @@ function responsesToIR(payload, { allowOpaqueCompaction = false } = {}) {
       } else if (item.type === 'additional_tools' && Array.isArray(item.tools)) {
         toolList.push(...item.tools);
       } else if (item.type === 'compaction') {
+        reasoning = '';
         if (allowOpaqueCompaction && !item.encrypted_content?.startsWith('llm-gateway-compact-v1:')) { ir.opaqueCompaction = true; continue; }
         pushText('user', `Conversation summary from an earlier turn:\n${decodeCompaction(item.encrypted_content)}`);
       } else if (item.type === 'compaction_trigger') {
@@ -1560,6 +1577,112 @@ function collapseNullableUnion(branches) {
 // key keeps the type an object; the root of a tool with no arguments may stay empty.
 const GEMINI_PLACEHOLDER_PROPERTIES = { _placeholder: { type: 'boolean', description: 'Unused. Leave unset.' } };
 
+// A node the padding produced: its only property is the placeholder. Matched by shape, not by
+// reference, because the schema is deep-copied again before it is sent.
+function isGeminiPadded(schema) {
+  const props = schema?.properties;
+  const p = props?._placeholder;
+  return Boolean(p && Object.keys(props).length === 1 && p.type === GEMINI_PLACEHOLDER_PROPERTIES._placeholder.type
+    && p.description === GEMINI_PLACEHOLDER_PROPERTIES._placeholder.description);
+}
+
+// Removes `_placeholder` from the arguments wherever the schema sent upstream was padded, and only there.
+export function stripGeminiPlaceholders(value, schema) {
+  if (!value || typeof value !== 'object' || !schema || typeof schema !== 'object') return value;
+  if (Array.isArray(value)) {
+    if (!schema.items) return value;
+    const out = value.map(v => stripGeminiPlaceholders(v, schema.items));
+    return out.some((v, i) => v !== value[i]) ? out : value;
+  }
+  if (isGeminiPadded(schema)) {
+    if (!Object.hasOwn(value, '_placeholder')) return value;
+    const { _placeholder, ...rest } = value;
+    return rest;
+  }
+  let out = value;
+  const props = schema.properties && typeof schema.properties === 'object' ? schema.properties : {};
+  for (const [k, v] of Object.entries(value)) {
+    if (!Object.hasOwn(props, k)) continue;
+    const next = stripGeminiPlaceholders(v, props[k]);
+    if (next !== v) out = { ...out, [k]: next };
+  }
+  // A branch that declares a real `_placeholder` makes the key the client's own, so no padded branch strips it.
+  const branches = Array.isArray(schema.anyOf) ? schema.anyOf : [];
+  const declared = branches.some(b => !isGeminiPadded(b) && b?.properties && Object.hasOwn(b.properties, '_placeholder'));
+  for (const branch of branches) if (!(declared && isGeminiPadded(branch))) out = stripGeminiPlaceholders(out, branch);
+  return out;
+}
+
+function hasGeminiPadding(schema) {
+  if (!schema || typeof schema !== 'object') return false;
+  if (isGeminiPadded(schema)) return true;
+  return Object.values(schema.properties || {}).some(hasGeminiPadding) || hasGeminiPadding(schema.items)
+    || (Array.isArray(schema.anyOf) && schema.anyOf.some(hasGeminiPadding));
+}
+
+// The upstream tool name -> the schema sent, for each tool whose schema was padded. Reads both the
+// Vertex and the chat body shape; an empty map means no argument needs the strip.
+export function geminiPaddedTools(upBody) {
+  const padded = new Map();
+  for (const t of Array.isArray(upBody?.tools) ? upBody.tools : []) {
+    const decls = Array.isArray(t?.functionDeclarations) ? t.functionDeclarations : t?.function ? [t.function] : [];
+    for (const d of decls) if (d?.name && hasGeminiPadding(d.parameters)) padded.set(d.name, d.parameters);
+  }
+  return padded;
+}
+
+// The tool vocabulary, plus the strip for padded tools. A renderer holds the arguments of a tool
+// whose needsArgsRewrite is true until the JSON is whole, so the strip also works on a stream.
+export function withGeminiPlaceholderStrip(vocab, padded) {
+  if (!padded?.size) return vocab;
+  // By client name too. Two padded tools that one client name maps back to make that name
+  // ambiguous, so it gets no strip, as the vocabulary refuses a tie.
+  const byClient = new Map();
+  for (const [upstream, schema] of padded) {
+    const client = vocab.name(upstream);
+    byClient.set(client, byClient.has(client) ? null : schema);
+  }
+  const schemaOf = (name) => padded.get(name) || byClient.get(vocab.name(name)) || byClient.get(name);
+  const args = (name, argsJson) => {
+    const out = vocab.args(name, argsJson);
+    const schema = schemaOf(name);
+    if (!schema) return out;
+    let parsed;
+    try { parsed = JSON.parse(out || '{}'); } catch { return out; }
+    const stripped = stripGeminiPlaceholders(parsed, schema);
+    return stripped === parsed ? out : JSON.stringify(stripped);
+  };
+  return {
+    ...vocab,
+    args,
+    needsArgsRewrite: (name) => Boolean(schemaOf(name)) || Boolean(vocab.needsArgsRewrite?.(name)),
+    repair(upstream, argsJson) {
+      const fixedName = vocab.name(upstream);
+      return { name: fixedName, args: args(upstream, argsJson), renamed: fixedName !== upstream };
+    }
+  };
+}
+
+// The tool vocabulary the caller declared, so a model that answers with a name from its own
+// training is written back in the caller's words instead of ending the turn with "No such tool
+// available". Built once per request from the tools the caller actually sent, so a request that
+// declared none pays nothing and a turn where the model agreed with the list changes nothing.
+export function toolVocabForIR(ir, outFormat) {
+  if (!ir || typeof ir !== 'object') return NULL_TOOL_VOCAB;
+  if (ir._vocab?.outFormat !== outFormat) {
+    const vocab = createToolVocab(ir.tools, { allowHyphens: outFormat !== 'vertex', names: [...toolNameMapForIR(ir).forward.keys()] });
+    ir._vocab = { outFormat, vocab: withGeminiPlaceholderStrip(vocab, ir._geminiPadded) };
+  }
+  return ir._vocab.vocab;
+}
+
+// Read by toolVocabForIR when the answer is rendered: the arguments lose the keys padding added.
+// Called on every emit, so a rebuild of the same IR never keeps the strip of an earlier body.
+export function recordGeminiPadding(ir, upBody, gemini) {
+  ir._geminiPadded = gemini && Array.isArray(upBody?.tools) ? geminiPaddedTools(upBody) : null;
+  ir._vocab = null;
+}
+
 function toGeminiSchema(schema, root, seen) {
   const nested = Boolean(root);
   const emptyProperties = () => (nested ? { ...GEMINI_PLACEHOLDER_PROPERTIES } : {});
@@ -1744,8 +1867,28 @@ function endsInUnsignedToolLoop(messages) {
 function healAnthropicPayload(payload) {
   const notes = [];
   if (!payload || !Array.isArray(payload.messages)) return { payload, changed: false, notes };
-  const src = payload.messages.filter(m => m && (m.role === 'user' || m.role === 'assistant'));
-  let changed = src.length !== payload.messages.length;
+  // Anthropic has no system role inside messages. Its text becomes a user turn in the same place, so it
+  // never moves past an assistant turn; Step 0 merges it into a user turn beside it, Steps 1-3 keep a tool_result first.
+  const src = [];
+  let folded = false;
+  let droppedSystem = false;
+  for (const m of payload.messages) {
+    if (m?.role === 'system') {
+      const blocks = toAnthropicBlocks(m.content);
+      const kept = blocks.filter(b => b.type === 'text' && !isBlankText(b));
+      if (!kept.length || kept.length !== blocks.length) droppedSystem = true;
+      if (kept.length) {
+        src.push({ role: 'user', content: [{ type: 'text', text: `[system] ${kept.map(b => b.text).join('\n')}` }] });
+        folded = true;
+      }
+      continue;
+    }
+    if (!m || (m.role !== 'user' && m.role !== 'assistant')) continue;
+    src.push(m);
+  }
+  if (folded) notes.push('folded system messages into user turns');
+  if (droppedSystem) notes.push('dropped system content with no text');
+  let changed = folded || droppedSystem || src.length !== payload.messages.length;
 
   let system = payload.system;
   if (Array.isArray(system) && system.some(isBlankText)) {
@@ -2753,7 +2896,7 @@ function vertexUsage(prompt, completion, cached, reasoning) {
   return u;
 }
 
-function buildVertexMessage({ model, think, text, tools, finish, prompt, completion, cached, reasoning, sig, ids = false }) {
+function buildVertexMessage({ model, think, text, tools, finish, prompt, completion, cached, reasoning, sig, ids = false, vocab = null }) {
   const parts = [];
   const thinking = (think || []).join('');
   if (thinking) {
@@ -2764,7 +2907,8 @@ function buildVertexMessage({ model, think, text, tools, finish, prompt, complet
   const body = (text || []).join('');
   if (body) parts.push({ text: body });
   for (const tc of (tools || [])) {
-    const part = { functionCall: { ...(ids && tc.id ? { id: tc.id } : {}), name: tc.name || 'tool', args: parseArgs(tc.args) } };
+    const args = vocab && typeof tc.args === 'string' ? vocab.args(tc.name, tc.args) : tc.args;
+    const part = { functionCall: { ...(ids && tc.id ? { id: tc.id } : {}), name: tc.name || 'tool', args: parseArgs(args) } };
     const tsig = tc.sig || lookupToolSignature(tc.id);
     if (tsig) part.thoughtSignature = tsig;
     parts.push(part);
@@ -2778,7 +2922,7 @@ function buildVertexMessage({ model, think, text, tools, finish, prompt, complet
 }
 
 // ids: name each functionCall, as Code Assist does. Plain Vertex has no call id.
-function createVertexStream(emit, model, { ids = false } = {}) {
+function createVertexStream(emit, model, { ids = false, vocab = null } = {}) {
   const cand = (parts) => emit(null, { candidates: [{ content: { role: 'model', parts }, index: 0 }], modelVersion: model });
   // Tool args arrive from OpenAI/Anthropic as deltas; Vertex needs complete functionCalls -> buffer them, emit at the end.
   const tools = new Map();
@@ -2805,7 +2949,8 @@ function createVertexStream(emit, model, { ids = false } = {}) {
       const parts = [...tools.values()]
         .filter(t => t.name)
         .map(t => {
-          const part = { functionCall: { ...(ids && t.id ? { id: t.id } : {}), name: t.name, args: parseArgs(t.args || '{}') } };
+          const args = vocab ? vocab.args(t.name, t.args || '{}') : (t.args || '{}');
+          const part = { functionCall: { ...(ids && t.id ? { id: t.id } : {}), name: t.name, args: parseArgs(args) } };
           const tsig = t.sig || lookupToolSignature(t.id);
           if (tsig) part.thoughtSignature = tsig;
           return part;
@@ -2830,10 +2975,10 @@ function codeAssistTrace() {
   return Array.from({ length: 16 }, () => Math.floor(Math.random() * 16).toString(16)).join('');
 }
 
-function createCodeAssistStream(emit, model) {
+function createCodeAssistStream(emit, model, { vocab = null } = {}) {
   const traceId = codeAssistTrace();
   const responseId = `agy-${traceId}`;
-  return createVertexStream((e, d) => emit(e, d?.error ? d : { response: { ...d, responseId }, traceId }), model, { ids: true });
+  return createVertexStream((e, d) => emit(e, d?.error ? d : { response: { ...d, responseId }, traceId }), model, { ids: true, vocab });
 }
 
 function buildCodeAssistMessage(args) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "llm-switcher",
-  "version": "1.6.11",
+  "version": "1.6.13",
   "description": "Zero-dependency multi-protocol edge gateway & provider switcher for Claude Code and Codex, with OpenAI-compatible, Anthropic and Vertex upstreams",
   "keywords": [
     "llm",

package/proxy.mjs

@@ -11,6 +11,7 @@ import { createFrameReader } from './blindfold/wsframe.mjs';
 import {
   OUT_FORMATS, parseToIR, emitUpstreamBody, irToAnthropicBody, createUpstreamNormalizer, createCollector,
   createThinkTagSplitter, splitThinkTags, healAnthropicPayload, estimateTokens, THINKING_MODES, VALID_EFFORTS,
+  toolVocabForIR, recordGeminiPadding,
   toGeminiSchema, isAntigravityModel, refusesImageInput, omitChatImages,
   createAnthropicStream, createChatStream, createResponsesStream, createVertexStream,
   buildAnthropicMessage, buildChatMessage, buildResponsesMessage, buildVertexMessage,
@@ -26,8 +27,7 @@ import {
   codexModelEntry,
   contractLabSettings, STATE_DIR, stopRecordedBlindfold
 } from './state.mjs';
-import { parseModelWindows, resolveProfileWindows, readLocalCodexWindows, readWindowsCache } from './windows.mjs';
-import { createToolVocab, NULL_TOOL_VOCAB, toolNameMapForIR } from './toolvocab.mjs';
+import { parseModelWindows, resolveProfileWindows, readLocalCodexWindows, windowsCacheKey, cachedProfileWindows, createWindowsRefresher } from './windows.mjs';
 import { readUpstreamPayloads } from './upstream-stream.mjs';
 import { compactionResponse, emitCompaction, expandGatewayCompactions, collectCompaction } from './compaction.mjs';
 import { classifyCodexRole, classifyClaudeTier, syncLocalCatalog, refreshCatalog, checkVersionAndRefresh } from './catalog.mjs';
@@ -386,10 +386,26 @@ function isContextOverflow(status, message) {
 }
 
 // Claude Code compacts on "prompt is too long: N tokens > M maximum", so the numbers lead when the upstream names them.
+// A count may group its digits (131,072 / 131.072 / 131 072 / 131'072 / 1,31,072, also with U+00A0 or U+202F) or
+// carry a k or m suffix. A dot before exactly three digits reads as a group, so 3.141 reads as 3141: the cost of
+// reading 131.072. Any other decimal part (0.125, 3.14) with no k or m after it is no token count.
+const TOKEN_COUNT = String.raw`(\d+(?:\.\d+)?[km](?![a-z])|[1-9]\d?(?:,\d\d)+,\d{3}(?![.,]?\d)|[1-9]\d{0,2}(?:[,. _'\u00a0\u202f]\d{3})+(?![.,]?\d)|\d+(?![.,]?\d))`;
+// A bare "limit" may be a rate or a time, so it counts only when "tokens" follows the number.
+const MAX_TOKENS_RE = new RegExp(String.raw`(?:maximum context length|context window|tokens? limit) (?:is |of )?${TOKEN_COUNT}|limit (?:is |of )?${TOKEN_COUNT}(?= ?tokens?\b)`, 'i');
+const ASKED_TOKENS_RE = new RegExp(String.raw`(?:requested|resulted in|prompt (?:is|has|contains)|input (?:is|has)) (?:about )?` + TOKEN_COUNT, 'i');
+
+function tokenCount(text) {
+  if (!text) return '';
+  const short = text.match(/^(\d+(?:\.\d+)?)([km])$/i);
+  const n = short ? Math.round(Number(short[1]) * (short[2].toLowerCase() === 'k' ? 1e3 : 1e6)) : Number(text.replace(/\D/g, ''));
+  return Number.isSafeInteger(n) && n > 0 ? String(n) : '';
+}
+
 function promptTooLongMessage(message) {
   if (/^prompt is too long/i.test(message)) return message;
-  const max = message.match(/(?:maximum context length|context window|limit) (?:is |of )?(\d+)/i)?.[1];
-  const asked = message.match(/(?:requested|resulted in|prompt (?:is|has|contains)|input (?:is|has)) (?:about )?(\d+)/i)?.[1];
+  const m = message.match(MAX_TOKENS_RE);
+  const max = tokenCount(m?.[1] ?? m?.[2]);
+  const asked = tokenCount(message.match(ASKED_TOKENS_RE)?.[1]);
   return max && asked ? `prompt is too long: ${asked} tokens > ${max} maximum (${message})` : `prompt is too long: ${message}`;
 }
 
@@ -488,21 +504,11 @@ function upstreamEndpoint(profile, outFormat, model, stream, req) {
   return { url: ov['openai-chat'] || `${base}/chat/completions`, headers };
 }
 
-// The tool vocabulary the caller declared, so a model that answers with a name from its own
-// training is written back in the caller's words instead of ending the turn with "No such tool
-// available". Built once per request from the tools the caller actually sent, so a request that
-// declared none pays nothing and a turn where the model agreed with the list changes nothing.
-function toolVocabFor(ir, outFormat) {
-  if (!ir || typeof ir !== 'object') return NULL_TOOL_VOCAB;
-  if (!ir._vocab) ir._vocab = createToolVocab(ir.tools, { allowHyphens: outFormat !== 'vertex', names: [...toolNameMapForIR(ir).forward.keys()] });
-  return ir._vocab;
-}
-
 function clientRenderer(clientFormat, res, model, opts = {}) {
   if (clientFormat === 'anthropic') return createAnthropicStream((e, d) => sendSSE(res, e, d), model, opts);
   if (clientFormat === 'responses') return createResponsesStream((e, d) => sendSSE(res, e, d), model, opts);
-  if (clientFormat === 'vertex') return createVertexStream((e, d) => sendSSE(res, null, d), model);
-  if (clientFormat === 'codeassist') return createCodeAssistStream((e, d) => sendSSE(res, null, d), model);
+  if (clientFormat === 'vertex') return createVertexStream((e, d) => sendSSE(res, null, d), model, opts);
+  if (clientFormat === 'codeassist') return createCodeAssistStream((e, d) => sendSSE(res, null, d), model, opts);
   return createChatStream((e, d) => sendSSE(res, null, d), model, opts);
 }
 
@@ -781,7 +787,7 @@ async function maybeCompactIdle(req, clientFormat, payload, ir, profile, profile
     const summaryModel = mapModel(policy.model, profile, clientFormat);
     const summaryFormat = resolveOutFormat(profile, summaryModel);
     askSummary({ ir: prefixIR, policy, profile, model: summaryModel,
-      build: (prof, model, summaryIR) => buildUpstreamRequest(prof, summaryFormat, summaryIR, model, req),
+      build: (prof, model, summaryIR) => buildUpstreamRequest(prof, summaryFormat, summaryIR, model, req, null, profileKey),
       onDiagnostic: ({ code, status }) => console.warn(`[llm-switcher] idle compact ${key.slice(0, 12)}: ${code}${status ? ` (${status})` : ''}`),
     }).then(summary => {
       if (!summary) return;
@@ -828,24 +834,30 @@ function resolveThinkingEffort(profile, slot, mappedModel) {
   return profile?.thinkingEffort || 'auto';
 }
 
-// The output limit the profile's own model list names for this model, or 0 when no cached list
-// names one. The cache is filled by /v1/models; this only reads it.
-function modelOutputLimit(profile, mappedModel) {
+const refreshProfileWindows = createWindowsRefresher(STATE_DIR, { debug: (...a) => debugLog(...a) });
+
+// A profile with an endpoints override sends its requests past baseURL, so its key must not go to baseURL/models.
+const routesThroughOverride = (profile) => Object.values(profile?.endpoints || {}).some(Boolean);
+
+// The output limit this profile's own model list names for this model, or 0 when no fresh cached
+// list names one. With no fresh entry the list is read in the background; this request never waits.
+function modelOutputLimit(profileKey, profile, mappedModel) {
   if (!profile?.baseURL || !mappedModel) return 0;
-  for (const entry of Object.values(readWindowsCache(STATE_DIR))) {
-    const output = entry?.baseURL === profile.baseURL ? Number(entry.models?.[mappedModel]?.output) : 0;
-    if (output > 0) return output;
-  }
-  return 0;
+  if (routesThroughOverride(profile)) return 0;
+  const key = windowsCacheKey(profileKey, profile.baseURL);
+  const models = cachedProfileWindows(STATE_DIR, key);
+  if (!Object.keys(models).length) refreshProfileWindows(key, profile.baseURL, profile.apiKey);
+  const output = Number(models[mappedModel]?.output);
+  return output > 0 ? output : 0;
 }
 
-function buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot = null) {
+function buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot = null, profileKey = null) {
   const effort = resolveThinkingEffort(profile, slot, mappedModel);
   const upBody = emitUpstreamBody(outFormat, ir, mappedModel, {
     thinkingMode: profile.thinkingMode,
     thinkingEffort: effort,
     replayReasoning: profile.replayReasoning === true,
-    maxOutputTokens: modelOutputLimit(profile, mappedModel),
+    maxOutputTokens: modelOutputLimit(profileKey, profile, mappedModel),
   });
   if (upBody?.tools && Array.isArray(upBody.tools)) {
     upBody.tools = cleanSchemaDeep(upBody.tools);
@@ -853,6 +865,7 @@ function buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot = n
       upBody.tools = geminiSafeTools(upBody.tools);
     }
   }
+  recordGeminiPadding(ir, upBody, outFormat === 'vertex' || (outFormat === 'openai-chat' && isAntigravityModel(mappedModel)));
   const { url, headers } = upstreamEndpoint(profile, outFormat, mappedModel, ir.stream, req);
   return { url, headers, upBody };
 }
@@ -1001,7 +1014,7 @@ async function executeGoalCheckGate({ req, res, clientFormat, payload, ir, profi
   try {
     const nativePromise = (async () => {
       try {
-        let url, headers, body;
+        let url, headers, body, upBody = null;
         if (bifrost) {
           url = bifrostURL(profile, clientFormat, req, opts);
           headers = bifrostHeaders(req, profile);
@@ -1019,16 +1032,18 @@ async function executeGoalCheckGate({ req, res, clientFormat, payload, ir, profi
           body = Buffer.from(JSON.stringify(json), 'utf8');
         } else {
           const slot = resolveSlot(requestedModel, mappedModel, profile, clientFormat);
-          const reqData = buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot);
+          const reqData = buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot, profileKey);
           url = reqData.url;
           headers = reqData.headers;
-          body = JSON.stringify(reqData.upBody);
+          upBody = reqData.upBody;
         }
 
         const traceId = clientFormat === 'codeassist' ? null : (probeTraceId(req) || contractLab.traceFor(mappedModel));
         if (traceId) headers['x-intact-trace'] = traceId;
 
-        const upRes = await fetch(url, { method: 'POST', headers, body, signal: gateAC.signal });
+        const upRes = upBody
+          ? await fetchUpstream(url, headers, upBody, gateAC.signal, outFormat, profileKey, mappedModel)
+          : await fetch(url, { method: 'POST', headers, body, signal: gateAC.signal });
         if (!upRes.ok) {
           const errText = await upRes.text().catch(() => '');
           return { malformed: true, error: `Upstream HTTP ${upRes.status}: ${errText.slice(0, 150)}` };
@@ -1302,7 +1317,7 @@ async function handleConvert(clientFormat, req, res, bodyBuffer, opts = {}) {
       return log({ status: 400, error: 'unsupported compaction state' });
     }
     const slot = resolveSlot(requestedModel, mappedModel, profile, clientFormat);
-    const { url, headers, upBody } = buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot);
+    const { url, headers, upBody } = buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot, profileKey);
     if (traceId) headers['x-intact-trace'] = traceId;
     debugLog(`[${profileKey}] ${clientFormat} -> ${outFormat} ${url} ::`, JSON.stringify(upBody).slice(0, 500));
 
@@ -1372,7 +1387,7 @@ async function handleConvert(clientFormat, req, res, bodyBuffer, opts = {}) {
       const out = clientMessage(clientFormat, {
         model: requestedModel || mappedModel, think, text: [split.text], tools, toolMeta: ir.toolMeta,
         finish: col.finish, prompt: col.prompt, completion, cached: col.cached,
-        reasoning: col.reasoning, sig: col.sig, vocab: toolVocabFor(ir, outFormat)
+        reasoning: col.reasoning, sig: col.sig, vocab: toolVocabForIR(ir, outFormat)
       });
       sendJson(res, 200, out);
       answered = true;
@@ -1389,7 +1404,7 @@ async function handleConvert(clientFormat, req, res, bodyBuffer, opts = {}) {
       'Connection': 'keep-alive',
       'X-Accel-Buffering': 'no'
     });
-    const renderer = clientRenderer(clientFormat, res, requestedModel || mappedModel, { toolMeta: ir.toolMeta, vocab: toolVocabFor(ir, outFormat) });
+    const renderer = clientRenderer(clientFormat, res, requestedModel || mappedModel, { toolMeta: ir.toolMeta, vocab: toolVocabForIR(ir, outFormat) });
     renderer.start();
     const splitter = createThinkTagSplitter(t => renderer.think(t), t => renderer.text(t));
     const streamError = await pumpStream(upstreamRes, { normalize, col, renderer, splitter, signal: ac.signal, sink: res, tag: profileKey });
@@ -1962,7 +1977,8 @@ async function servedModels(req) {
 // still gets the cached numbers, because an entry with the template window is better than none.
 async function modelWindowsFor(profileKey, profile) {
   if (!profile?.baseURL) return {};
-  const listed = await resolveProfileWindows(STATE_DIR, `${profileKey || 'profile'}@${profile.baseURL}`, profile.baseURL, profile.apiKey);
+  if (routesThroughOverride(profile)) return readLocalCodexWindows();
+  const listed = await resolveProfileWindows(STATE_DIR, windowsCacheKey(profileKey, profile.baseURL), profile.baseURL, profile.apiKey);
   if (Object.keys(listed).length) return listed;
   return readLocalCodexWindows();
 }
@@ -2385,7 +2401,7 @@ function sendWsFailed(socket, model, message, status = 500, tap = null, code = n
     if (socket.writable) socket.write(encodeWsFrame(JSON.stringify(d)));
   }, model || 'main');
   renderer.start();
-  renderer.error(message, code || responsesErrorCode(status));
+  renderer.error(message, code || (isContextOverflow(status, message) ? 'context_length_exceeded' : responsesErrorCode(status)));
 }
 
 // Codex on WebSocket sends only the new items of a turn and names the turn before in
@@ -2573,7 +2589,7 @@ async function handleWsResponseCreate(socket, payload, req, ac, history = null)
   let answered = false; // set only after a complete answer; the half upload depends on it
   try {
     const slot = resolveSlot(requestedModel, mappedModel, profile, clientFormat);
-    const { url, headers, upBody } = buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot);
+    const { url, headers, upBody } = buildUpstreamRequest(profile, outFormat, ir, mappedModel, req, slot, profileKey);
     if (traceId) headers['x-intact-trace'] = traceId;
     debugLog(`[${profileKey}:ws] ${clientFormat} -> ${outFormat} ${url} ::`, JSON.stringify(upBody).slice(0, 300));
 
@@ -2619,7 +2635,7 @@ async function handleWsResponseCreate(socket, payload, req, ac, history = null)
       if (socket.writable) {
         socket.write(encodeWsFrame(JSON.stringify(d)));
       }
-    }, requestedModel || mappedModel, { toolMeta: ir.toolMeta, vocab: toolVocabFor(ir, outFormat) });
+    }, requestedModel || mappedModel, { toolMeta: ir.toolMeta, vocab: toolVocabForIR(ir, outFormat) });
 
     renderer.start();
     const splitter = createThinkTagSplitter(t => renderer.think(t), t => renderer.text(t));