pi-cache-optimizer 2.4.7 → 2.4.8

package/README.md

@@ -339,10 +339,40 @@ Edit ~/.pi/agent/models.json -> providers["otokapi"] -> compat (same level as ba
 
 ### `/cache-optimizer compat`
 
-Shows only the compat suggestion for the active model, including file path,
-provider path, and copyable JSON snippet. When no flags are missing, it shows
-`✅ Compat fully configured.` if the model is an applicable third-party proxy,
-or `ℹ️ Compat check not applicable for this model.` otherwise.
+Shows the compat suggestion for the active model, including file path,
+provider path, and copyable JSON snippet. When no compat flags are missing,
+it shows `✅ Compat fully configured.` if the model is an applicable
+third-party proxy, or `ℹ️ Compat check not applicable for this model.`
+otherwise.
+
+When the model is routed through a known router/channel proxy (OpenRouter,
+Vercel AI Gateway, LiteLLM, OneAPI/NewAPI/VoAPI, or a generic third-party
+OpenAI-compatible proxy), both `doctor` and `compat` subcommands append
+router/channel diagnostics with targeted recommendations.
+
+### Router/channel diagnostics
+
+For models using OpenAI-compatible APIs (`openai-completions` or
+`openai-responses`) through a non-official base URL, the extension detects
+common router/channel proxy patterns from `provider`, `baseUrl`, and `compat`
+metadata:
+
+| Profile | Detection | Recommendation |
+|---------|-----------|----------------|
+| **OpenRouter** | baseUrl or provider contains `openrouter`/`openrouter.ai` | Fix the upstream provider with `openRouterRouting.only` or `.order` in compat |
+| **Vercel AI Gateway** | baseUrl contains `ai-gateway.vercel.sh` or provider contains `vercel` | Fix the upstream with `vercelGatewayRouting.only` or `.order` in compat |
+| **LiteLLM / OneAPI / NewAPI / VoAPI** | baseUrl or provider contains `litellm`, `oneapi`/`one-api`, `newapi`/`new-api`, `voapi`/`vo-api` | Ensure sticky session routing, forward `prompt_cache_key` + session-affinity headers, return cache usage fields |
+| **Generic third-party proxy** | Any `openai-completions` model with non-official base URL not matching above | General guidance: verify single-upstream routing, forward `prompt_cache_key` + session-affinity headers, return cache usage |
+
+These diagnostics are **advisory only**. They do not participate in adapter
+selection (still id/name-only), prompt_cache_key injection, footer stats, or
+any automated configuration changes. Detection uses only metadata exposed by
+Pi (`provider`, `api`, `baseUrl`, `compat`) — no API keys, prompts, payloads,
+headers, or model outputs are read or exposed.
+
+Official OpenAI (`api.openai.com`) and custom transports (`kiro-api`,
+`anthropic-messages`, `bedrock-converse-stream`) are excluded from router/
+channel diagnostics.
 
 ### Security
 

package/README.zh-CN.md

@@ -325,7 +325,24 @@ Edit ~/.pi/agent/models.json -> providers["otokapi"] -> compat (same level as ba
 
 ### `/cache-optimizer compat`
 
-仅显示当前模型的 compat 建议，包括文件路径、provider 路径和可复制 JSON 片段。当没有缺失标志时，如果模型是适用的第三方代理则显示 `✅ Compat fully configured.`，否则显示 `ℹ️ Compat check not applicable for this model.`。
+显示当前模型的 compat 建议，包括文件路径、provider 路径和可复制 JSON 片段。当没有缺失的 compat 标志时，如果模型是适用的第三方代理则显示 `✅ Compat fully configured.`，否则显示 `ℹ️ Compat check not applicable for this model.`。
+
+当模型通过已知的路由器/通道代理（OpenRouter、Vercel AI Gateway、LiteLLM、OneAPI/NewAPI/VoAPI 或通用第三方 OpenAI-compatible 代理）时，`doctor` 和 `compat` 子命令都会附加路由/通道诊断信息和建议。
+
+### 路由/通道诊断
+
+对于通过非官方 base URL 使用 OpenAI-compatible API（`openai-completions` 或 `openai-responses`）的模型，扩展会从 `provider`、`baseUrl` 和 `compat` 元数据中检测常见的路由/通道代理模式：
+
+| 类型 | 检测方式 | 建议 |
+|------|----------|------|
+| **OpenRouter** | baseUrl 或 provider 包含 `openrouter`/`openrouter.ai` | 在 compat 中用 `openRouterRouting.only` 或 `.order` 固定上游 provider |
+| **Vercel AI Gateway** | baseUrl 包含 `ai-gateway.vercel.sh` 或 provider 包含 `vercel` | 在 compat 中用 `vercelGatewayRouting.only` 或 `.order` 固定上游 |
+| **LiteLLM / OneAPI / NewAPI / VoAPI** | baseUrl 或 provider 包含 `litellm`、`oneapi`/`one-api`、`newapi`/`new-api`、`voapi`/`vo-api` | 确保每 session 固定路由，转发 `prompt_cache_key` + session-affinity headers，返回缓存用量字段 |
+| **通用第三方代理** | 任何非官方 base URL 的 `openai-completions` 模型，且不匹配以上类型 | 通用建议：验证单上游路由、转发 `prompt_cache_key` + session-affinity headers、返回缓存用量 |
+
+这些诊断**仅用于建议**。它们不参与 adapter selection（仍基于 id/name）、不参与 `prompt_cache_key` 注入、不参与 footer 统计、也不做任何自动化配置修改。检测仅使用 Pi 暴露的元数据（`provider`、`api`、`baseUrl`、`compat`），不会读取或暴露 API key、prompt、payload、headers 或模型输出。
+
+官方 OpenAI（`api.openai.com`）和 custom transport（`kiro-api`、`anthropic-messages`、`bedrock-converse-stream`）不会触发路由/通道诊断。
 
 ### 安全说明
 

package/index.ts

@@ -2621,6 +2621,171 @@ function isCompatCheckApplicable(model: PiModel): boolean {
   return lower(model.api) === "openai-completions" && !isOfficialOpenAIBaseUrl(model);
 }
 
+/**
+ * Detect router / channel profiles from a PiModel and return diagnostic notes.
+ *
+ * This function is advisory only — it does NOT participate in adapter selection,
+ * prompt_cache_key injection, or footer stats. It inspects provider, api, baseUrl,
+ * and compat to identify common proxy/router patterns where cache performance may
+ * be degraded due to multi-backend routing.
+ *
+ * Known profiles (checked in order):
+ *   1. OpenRouter — baseUrl or provider id matching openrouter.ai / openrouter
+ *   2. Vercel AI Gateway — baseUrl matching ai-gateway.vercel.sh, or provider
+ *      matching vercel / vercel-ai-gateway
+ *   3. LiteLLM / OneAPI / NewAPI / VoAPI — baseUrl or provider matching litellm,
+ *      oneapi, one-api, newapi, new-api, voapi, vo-api (self-hosted aggregation)
+ *   4. Generic third-party OpenAI-compatible proxy — any openai-completions model
+ *      with a non-official base URL that does not match a higher-profile above.
+ *
+ * Official OpenAI (api.openai.com) and custom transports (kiro-api, anthropic-messages,
+ * bedrock-converse-stream) do NOT produce notes.
+ */
+function describeRouterChannelDiagnostics(model: PiModel): string[] {
+  const notes: string[] = [];
+  const api = lower(model.api);
+  const baseUrl = lower(model.baseUrl || "");
+  const provider = lower(model.provider);
+
+  // Only OpenAI-compatible APIs are applicable for router/channel diagnostics.
+  // Custom transports like kiro-api, anthropic-messages, bedrock-converse-stream
+  // or non-OpenAI APIs are excluded.
+  if (api !== "openai-completions" && api !== "openai-responses") {
+    return notes;
+  }
+
+  // Official OpenAI bypass — no notes needed.
+  if (isOfficialOpenAIBaseUrl(model)) {
+    return notes;
+  }
+
+  // ── 1. OpenRouter ────────────────────────────────────────────────
+  if (
+    baseUrl.includes("openrouter.ai") ||
+    baseUrl.includes("openrouter") ||
+    provider.includes("openrouter")
+  ) {
+    const compat = getCompat(model);
+    const hasOnly = !!(compat as Record<string, unknown>)["openRouterRouting"]?.only;
+    const hasOrder = !!(compat as Record<string, unknown>)["openRouterRouting"]?.order;
+
+    notes.push(
+      "🔀 Router/channel: OpenRouter detected. OpenRouter is a multi-provider router; " +
+      "low cache hit rates are common when each turn lands on a different upstream provider.",
+    );
+
+    if (!hasOnly && !hasOrder) {
+      notes.push(
+        "   Suggestion: Add an openRouterRouting config to fix the upstream provider. " +
+        "Example for models.json -> providers[\"<providerId>\"] -> compat:",
+      );
+      notes.push(
+        `   { "sendSessionAffinityHeaders": true, "supportsLongCacheRetention": true, ` +
+        `"openRouterRouting": { "only": ["<provider-slug>"] } }`,
+      );
+      notes.push(
+        '   Replace <provider-slug> with the actual OpenRouter provider slug (e.g. "openai", "anthropic").',
+      );
+      notes.push(
+        "   Alternatively, use openRouterRouting.order: [\"<provider-slug>\", \"...\"] for fallback order. " +
+        "Only set supportsLongCacheRetention if your upstream supports long cache retention.",
+      );
+    }
+
+    return notes;
+  }
+
+  // ── 2. Vercel AI Gateway ─────────────────────────────────────────
+  if (
+    baseUrl.includes("ai-gateway.vercel.sh") ||
+    provider.includes("vercel") ||
+    provider.includes("vercel-ai-gateway")
+  ) {
+    const compat = getCompat(model);
+    const hasOnly = !!(compat as Record<string, unknown>)["vercelGatewayRouting"]?.only;
+    const hasOrder = !!(compat as Record<string, unknown>)["vercelGatewayRouting"]?.order;
+
+    notes.push(
+      "🔀 Router/channel: Vercel AI Gateway detected. The gateway may route to different " +
+      "provider endpoints per request, reducing cache locality.",
+    );
+
+    if (!hasOnly && !hasOrder) {
+      notes.push(
+        "   Suggestion: Add a vercelGatewayRouting config to fix the upstream. " +
+        "Example for models.json -> providers[\"<providerId>\"] -> compat:",
+      );
+      notes.push(
+        `   { "sendSessionAffinityHeaders": true, "supportsLongCacheRetention": true, ` +
+        `"vercelGatewayRouting": { "only": ["<provider-id>"] } }`,
+      );
+      notes.push(
+        "   Replace <provider-id> with the actual Vercel provider ID (e.g. \"openai\").",
+      );
+      notes.push(
+        "   Only set supportsLongCacheRetention if your upstream supports it.",
+      );
+    }
+
+    return notes;
+  }
+
+  // ── 3. LiteLLM / OneAPI / NewAPI / VoAPI (self-hosted aggregation) ──
+  const aggregationPatterns = ["litellm", "oneapi", "one-api", "newapi", "new-api", "voapi", "vo-api"];
+  if (
+    aggregationPatterns.some((p) => baseUrl.includes(p)) ||
+    aggregationPatterns.some((p) => provider.includes(p))
+  ) {
+    notes.push(
+      "🔀 Router/channel: Self-hosted aggregation proxy detected (LiteLLM / OneAPI / NewAPI / VoAPI). " +
+      "These proxies route to multiple upstream accounts or instances, which can split the cache.",
+    );
+    notes.push(
+      "   Suggestions:",
+    );
+    notes.push(
+      "   • Ensure the proxy can fix to a single upstream per session (session_id affinity).",
+    );
+    notes.push(
+      "   • Forward prompt_cache_key and session-affinity headers to the upstream.",
+    );
+    notes.push(
+      "   • Return cache usage fields (prompt_cache_hit_tokens, etc.) in the response.",
+    );
+    notes.push(
+      `   Example compat: { "sendSessionAffinityHeaders": true, "supportsLongCacheRetention": true }`,
+    );
+
+    return notes;
+  }
+
+  // ── 4. Generic third-party OpenAI-compatible proxy ─────────────────
+  if (api === "openai-completions" && baseUrl) {
+    const missing = describeMissingOpenAICompatibleProxyCompat(model);
+    notes.push(
+      "🔀 Router/channel: Third-party OpenAI-compatible proxy. If cache hit rates are low:",
+    );
+    notes.push(
+      "   • Verify the proxy routes to the same upstream account/instance per session.",
+    );
+    notes.push(
+      "   • Ensure the proxy forwards prompt_cache_key and sends session-affinity headers.",
+    );
+    notes.push(
+      "   • Check that the proxy returns cache usage fields (prompt_cache_hit_tokens etc.).",
+    );
+    if (missing.length > 0) {
+      notes.push(
+        `   • The compat flags above (${missing.join(", ")}) are recommended for cache stability.`,
+      );
+    }
+
+    return notes;
+  }
+
+  return notes;
+}
+
 function buildDoctorDiagnosis(model: PiModel): string {
   const lines: string[] = [];
   lines.push(`Provider: ${model.provider}`);
@@ -2648,6 +2813,15 @@ function buildDoctorDiagnosis(model: PiModel): string {
     lines.push("ℹ️ Compat check not applicable for this model.");
   }
 
+  // ── Router/channel diagnostics ──
+  const routerNotes = describeRouterChannelDiagnostics(model);
+  if (routerNotes.length > 0) {
+    lines.push("");
+    for (const note of routerNotes) {
+      lines.push(note);
+    }
+  }
+
   // ── Integrity diagnostics ──
   if (lastPromptIntegrityWarningAt > 0) {
     const ago = Date.now() - lastPromptIntegrityWarningAt;
@@ -2670,21 +2844,46 @@ function buildDoctorDiagnosis(model: PiModel): string {
 
 function buildCompatDiagnosis(model: PiModel): string | undefined {
   const missing = describeMissingOpenAICompatibleProxyCompat(model);
-  if (missing.length === 0) return undefined;
+  const routerNotes = describeRouterChannelDiagnostics(model);
+
+  if (missing.length === 0 && routerNotes.length === 0) return undefined;
 
   const key = modelKey(model);
-  const slashIdx = key.indexOf("/");
-  const providerLabel = slashIdx > 0 ? key.slice(0, slashIdx) : key;
-  const suggestion = Object.fromEntries(missing.map((f) => [f, true]));
-  const modelsJsonPath = getModelsJsonDisplayPath();
-  return (
-    `Active model: ${key}\n` +
-    `Missing: ${missing.join(", ")}\n\n` +
-    `Edit ${modelsJsonPath} -> providers["${providerLabel}"] -> compat` +
-    ` (at the same level as baseUrl/api/apiKey/models) and add:\n` +
-    `${JSON.stringify(suggestion, null, 2)}\n\n` +
-    `Only enable if your endpoint supports them.`
-  );
+  const lines: string[] = [];
+
+  if (missing.length > 0) {
+    const slashIdx = key.indexOf("/");
+    const providerLabel = slashIdx > 0 ? key.slice(0, slashIdx) : key;
+    const suggestion = Object.fromEntries(missing.map((f) => [f, true]));
+    const modelsJsonPath = getModelsJsonDisplayPath();
+    lines.push(`Active model: ${key}`);
+    lines.push(`Missing: ${missing.join(", ")}`);
+    lines.push("");
+    lines.push(`Edit ${modelsJsonPath} -> providers["${providerLabel}"] -> compat`);
+    lines.push(`(at the same level as baseUrl/api/apiKey/models) and add:`);
+    lines.push(JSON.stringify(suggestion, null, 2));
+    lines.push("");
+    lines.push(`Only enable if your endpoint supports them.`);
+  }
+
+  // When compat is fully configured but router notes exist, prefix the status.
+  if (routerNotes.length > 0 && missing.length === 0) {
+    if (isCompatCheckApplicable(model)) {
+      lines.push("✅ Compat fully configured.");
+    } else {
+      lines.push("ℹ️ Compat check not applicable for this model.");
+    }
+    lines.push("");
+  }
+
+  if (routerNotes.length > 0) {
+    if (missing.length > 0) lines.push("");
+    for (const note of routerNotes) {
+      lines.push(note);
+    }
+  }
+
+  return lines.join("\n");
 }
 
 // Internal helpers exported only so the task verification script
@@ -2835,6 +3034,7 @@ export const __internals_for_tests = {
   isCompatCheckApplicable,
   buildDoctorDiagnosis,
   buildCompatDiagnosis,
+  describeRouterChannelDiagnostics,
   // Cache stats helpers (module-level, usable from verify script)
   addUsageToCacheStats,
   formatCacheStats,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-cache-optimizer",
-  "version": "2.4.7",
+  "version": "2.4.8",
   "description": "Pi extension that improves provider-side KV/prompt cache hit rates (DeepSeek, OpenAI, Claude, Gemini) by reordering the system prompt, requesting long retention, and showing footer cache stats. Renamed from pi-deepseek-cache-optimizer.",
   "keywords": [
     "pi-package",