pi-cache-optimizer 2.8.4 → 2.8.6

package/README.md

@@ -114,7 +114,7 @@ The explicit setting is stored in `pi-cache-optimizer-config.json` under Pi's ag
 
 Third-party `openai-completions` proxies (LiteLLM / OneAPI / NewAPI / OpenRouter-like channels) often route one session across multiple upstream backends. That splits provider-side prompt caches.
 
-Pi 0.84.1 also fixes built-in Fireworks compatibility for models that reject `prompt_cache_retention`; keep the extension's diagnostics focused on the effective model compat exposed by Pi rather than adding a provider-name special case. Pi 0.81+ also has a built-in `llama.cpp` provider using an OpenAI-shaped transport. Pi 0.82+ core generates a session `prompt_cache_key` for it when cache retention is enabled, so this extension preserves that key and may add the same conservative fallback when missing. The built-in provider's explicit compat fingerprint is excluded from generic proxy routing/session-affinity advice, but a custom or overridden provider that merely reuses the id `llama.cpp` is treated like any other OpenAI-compatible channel. `prompt_cache_retention` remains subject to the normal safety rule: keep it only for official OpenAI or an explicit effective `supportsLongCacheRetention: true` opt-in in `models.json`; otherwise strip it before sending. Effective values follow Pi's precedence: `modelOverrides[modelId].compat` first, then the matching `models[].compat`, then provider-level `compat`. An explicit `false` at a higher layer overrides `true` below it.
+Pi 0.84.1 also fixes built-in Fireworks compatibility for models that reject `prompt_cache_retention`; the extension avoids provider-name special cases and resolves exact provider/model compat from `models.json` plus the runtime model. Pi 0.81+ also has a built-in `llama.cpp` provider using an OpenAI-shaped transport. Pi 0.82+ core generates a session `prompt_cache_key` for it when cache retention is enabled, so this extension preserves that key and may add the same conservative fallback when missing. The built-in provider's explicit compat fingerprint is excluded from generic proxy routing/session-affinity advice, but a custom or overridden provider that merely reuses the id `llama.cpp` is treated like any other OpenAI-compatible channel. `prompt_cache_retention` remains subject to the normal safety rule: keep it only for official OpenAI or an explicit effective `supportsLongCacheRetention: true` opt-in in `models.json`; otherwise strip it before sending. Effective values follow Pi's precedence: `modelOverrides[modelId].compat` first, then the matching `models[].compat`, then provider-level `compat`. An explicit `false` at a higher layer overrides `true` below it.
 
 For real proxies, start with session affinity:
 
@@ -262,6 +262,8 @@ Provider-level minimal override:
 }
 ```
 
+Pi Cache Optimizer resolves effective compat using Pi's precedence (`provider.compat` → matching `models[].compat` → runtime model compat → `modelOverrides[modelId].compat`). This also covers extension providers that replace their model list and accidentally omit lower-level compat from the runtime model object. For non-official `openai-completions` channels, when effective `sendSessionAffinityHeaders` is `true` but Pi's runtime model lost it, the extension restores Pi-compatible affinity headers at request time without overwriting existing headers. An explicit `false` remains a respected opt-out.
+
 If only one model should change, use `modelOverrides`:
 
 ```json

package/README.zh-CN.md

@@ -114,7 +114,7 @@ Footer 默认使用 `session`，避免另一个并行 Pi 终端使用相同 prov
 
 LiteLLM / OneAPI / NewAPI / 类 OpenRouter 渠道等第三方 `openai-completions` 代理，常会把同一个 session 分散到多个上游后端，导致 provider 侧 prompt cache 被拆散。
 
-Pi 0.84.1 还修复了内置 Fireworks 渠道对拒绝 `prompt_cache_retention` 的模型兼容性；本扩展继续依据 Pi 暴露的有效 model compat 做诊断，不按 provider 名称增加特殊分支。Pi 0.81+ 也内置了使用 OpenAI-shaped transport 的 `llama.cpp` provider。Pi 0.82+ core 在启用 cache retention 时会为它生成 session `prompt_cache_key`，因此本扩展会保留该 key，并在缺失时使用同样的保守 fallback。只有符合 Pi 内置 provider 明确 compat 指纹的模型会跳过通用 proxy 路由 / session-affinity 建议；仅复用 `llama.cpp` id 的自定义或覆盖 provider 仍按普通 OpenAI-compatible 渠道处理。`prompt_cache_retention` 继续遵循统一安全规则：仅官方 OpenAI 或 `models.json` 中有效配置为 `supportsLongCacheRetention: true` 时保留，否则发送前移除。有效值遵循 Pi 的优先级：先看 `modelOverrides[modelId].compat`，再看匹配的 `models[].compat`，最后看 provider 级 `compat`；高层显式 `false` 会覆盖低层的 `true`。
+Pi 0.84.1 还修复了内置 Fireworks 渠道对拒绝 `prompt_cache_retention` 的模型兼容性；本扩展不按 provider 名称增加特殊分支，而是结合 `models.json` 与 runtime model，按精确 provider/model 解析有效 compat。Pi 0.81+ 也内置了使用 OpenAI-shaped transport 的 `llama.cpp` provider。Pi 0.82+ core 在启用 cache retention 时会为它生成 session `prompt_cache_key`，因此本扩展会保留该 key，并在缺失时使用同样的保守 fallback。只有符合 Pi 内置 provider 明确 compat 指纹的模型会跳过通用 proxy 路由 / session-affinity 建议；仅复用 `llama.cpp` id 的自定义或覆盖 provider 仍按普通 OpenAI-compatible 渠道处理。`prompt_cache_retention` 继续遵循统一安全规则：仅官方 OpenAI 或 `models.json` 中有效配置为 `supportsLongCacheRetention: true` 时保留，否则发送前移除。有效值遵循 Pi 的优先级：先看 `modelOverrides[modelId].compat`，再看匹配的 `models[].compat`，最后看 provider 级 `compat`；高层显式 `false` 会覆盖低层的 `true`。
 
 对真正的代理，建议先启用 session affinity：
 
@@ -262,6 +262,8 @@ Provider 级最小 override：
 }
 ```
 
+Pi Cache Optimizer 按 Pi 的优先级解析有效 compat（`provider.compat` → 匹配的 `models[].compat` → runtime model compat → `modelOverrides[modelId].compat`）。这也覆盖了某些 extension provider 替换模型列表后，runtime model 意外丢失低层 compat 的情况。对于非官方 `openai-completions` 渠道，如果有效 `sendSessionAffinityHeaders` 为 `true`、但 Pi runtime model 丢失了该值，本扩展会在请求阶段恢复 Pi-compatible affinity headers，且不会覆盖已有 header；显式 `false` 仍作为有效 opt-out 尊重。
+
 如果只想影响单个模型，用 `modelOverrides`：
 
 ```json

package/index.ts

@@ -1,6 +1,6 @@
 import { createHash, randomUUID } from "node:crypto";
 import { chmod, copyFile, lstat, mkdir, readFile, readdir, rename, rm, stat, unlink, writeFile } from "node:fs/promises";
-import { constants as fsConstants, readFileSync, watch } from "node:fs";
+import { constants as fsConstants, readFileSync, statSync, watch } from "node:fs";
 import { homedir } from "node:os";
 import { dirname, join } from "node:path";
 import {
@@ -21,6 +21,7 @@ type CacheRetentionEnvSnapshot = {
 const PI_CACHE_RETENTION_ENV = "PI_CACHE_RETENTION";
 const LONG_CACHE_RETENTION_VALUE = "long";
 const PI_CACHE_RETENTION_BASELINE_SYMBOL = Symbol.for("pi.cache.optimizer.retention-baseline.v1");
+const ROUTED_FALLBACK_MODEL_SYMBOL = Symbol("pi-cache-optimizer.routed-fallback-model");
 
 type CacheRetentionBaselineV1 = {
   version: 1;
@@ -236,6 +237,10 @@ type CacheCompat = {
   cacheControlFormat?: string;
   forceAdaptiveThinking?: boolean;
   allowEmptySignature?: boolean;
+  openRouterRouting?: UnknownRecord;
+  vercelGatewayRouting?: UnknownRecord;
+  chatTemplateKwargs?: UnknownRecord;
+  chatTemplateArgs?: UnknownRecord;
 };
 
 type CacheStats = {
@@ -1013,19 +1018,23 @@ function resolveActiveRouteSnapshot(
 }
 
 function routeSnapshotToPiModel(snapshot: PiRouteSnapshot, fallback?: PiModel): PiModel {
+  const sameIdentity = fallback?.provider === snapshot.provider && fallback?.id === snapshot.modelId;
   return {
-    ...(fallback ?? {}),
+    ...(sameIdentity ? fallback ?? {} : {}),
     id: snapshot.modelId,
     name: snapshot.canonicalModelId ?? snapshot.modelId,
     provider: snapshot.provider,
-    api: snapshot.api ?? fallback?.api ?? "",
-    baseUrl: fallback?.baseUrl ?? "",
-    reasoning: fallback?.reasoning ?? false,
-    input: fallback?.input ?? ["text"],
-    cost: fallback?.cost ?? { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
-    contextWindow: fallback?.contextWindow ?? 0,
-    maxTokens: fallback?.maxTokens ?? 0,
-    compat: fallback?.compat,
+    api: snapshot.api ?? (sameIdentity ? fallback?.api : undefined) ?? "",
+    baseUrl: sameIdentity ? fallback?.baseUrl ?? "" : "",
+    reasoning: sameIdentity ? fallback?.reasoning ?? false : false,
+    input: sameIdentity ? fallback?.input ?? ["text"] : ["text"],
+    cost: sameIdentity
+      ? fallback?.cost ?? { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }
+      : { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+    contextWindow: sameIdentity ? fallback?.contextWindow ?? 0 : 0,
+    maxTokens: sameIdentity ? fallback?.maxTokens ?? 0 : 0,
+    compat: sameIdentity ? fallback?.compat : undefined,
+    [ROUTED_FALLBACK_MODEL_SYMBOL]: true,
   } as PiModel;
 }
 
@@ -1041,6 +1050,28 @@ function findModelInRegistry(registry: ModelRegistryLike | undefined, provider:
   return all.find((candidate) => candidate.provider === provider && candidate.id === id);
 }
 
+function isRoutedFallbackModel(model: PiModel | undefined): boolean {
+  return !!model && (model as PiModel & Record<symbol, unknown>)[ROUTED_FALLBACK_MODEL_SYMBOL] === true;
+}
+
+function applyConfiguredTransportToModel(model: PiModel, config: unknown): PiModel {
+  const providers = asRecord(asRecord(config)?.providers);
+  const provider = asRecord(providers?.[model.provider]);
+  const customModel = findLastExactModelDefinition(
+    Array.isArray(provider?.models) ? provider.models : undefined,
+    model.id,
+  );
+  const configuredApi = customModel?.api ?? provider?.api;
+  const configuredBaseUrl = customModel?.baseUrl ?? provider?.baseUrl;
+  const api = isNonEmptyString(model.api)
+    ? model.api
+    : (isNonEmptyString(configuredApi) ? configuredApi : model.api);
+  const baseUrl = isNonEmptyString(model.baseUrl)
+    ? model.baseUrl
+    : (isNonEmptyString(configuredBaseUrl) ? configuredBaseUrl : model.baseUrl);
+  return api === model.api && baseUrl === model.baseUrl ? model : { ...model, api, baseUrl };
+}
+
 function resolveRouteModel(
   model: PiModel | undefined,
   ctx?: ContextWithOptionalModelRegistry,
@@ -1048,8 +1079,9 @@ function resolveRouteModel(
   const snapshot = resolveActiveRouteSnapshot(model, ctx);
   if (!snapshot) return undefined;
 
-  return findModelInRegistry(ctx?.modelRegistry, snapshot.provider, snapshot.modelId)
+  const resolved = findModelInRegistry(ctx?.modelRegistry, snapshot.provider, snapshot.modelId)
     ?? routeSnapshotToPiModel(snapshot, model);
+  return applyConfiguredTransportToModel(resolved, readEffectiveCompatConfig());
 }
 
 function isVirtualRoutingModel(model: PiModel | undefined, ctx?: Pick<ExtensionContext, "sessionManager">): boolean {
@@ -1134,17 +1166,335 @@ function getNonNegativeNumber(record: UnknownRecord, key: string): number | unde
  * Model-level compat takes precedence over provider-level compat for overlapping keys.
  * This matches Pi's model-registry.js mergeCompat behavior.
  */
+const NESTED_COMPAT_KEYS = [
+  "openRouterRouting",
+  "vercelGatewayRouting",
+  "chatTemplateKwargs",
+  "chatTemplateArgs",
+] as const;
+
+function findLastExactModelDefinition(
+  models: unknown[] | undefined,
+  modelId: string,
+): UnknownRecord | undefined {
+  if (!models) return undefined;
+  // Pi's provider composer processes definitions in array order. If a malformed
+  // or hand-edited config repeats an id, the last definition replaces earlier
+  // definitions, so effective resolution must use the same deterministic rule.
+  for (let index = models.length - 1; index >= 0; index--) {
+    const candidate = asRecord(models[index]);
+    if (candidate?.id === modelId) return candidate;
+  }
+  return undefined;
+}
+
+function mergeCacheCompat(...sources: Array<UnknownRecord | undefined>): CacheCompat {
+  const merged: CacheCompat = {};
+  for (const source of sources) {
+    if (!source) continue;
+    const previousNested = Object.fromEntries(
+      NESTED_COMPAT_KEYS.map((key) => [key, asRecord(merged[key])]),
+    ) as Partial<Record<(typeof NESTED_COMPAT_KEYS)[number], UnknownRecord | undefined>>;
+    Object.assign(merged, source);
+    for (const key of NESTED_COMPAT_KEYS) {
+      const baseValue = previousNested[key];
+      const overrideValue = asRecord(source[key]);
+      if (baseValue || overrideValue) {
+        merged[key] = { ...baseValue, ...overrideValue };
+      }
+    }
+  }
+  return merged;
+}
+
+function getEffectiveCompatSources(model: PiModel, config: unknown): Array<{ source: "provider" | "model" | "runtime" | "modelOverride"; compat: UnknownRecord }> {
+  const root = asRecord(config);
+  const providers = asRecord(root?.providers);
+  const provider = asRecord(providers?.[model.provider]);
+  const providerCompat = asRecord(provider?.compat);
+  const customModel = findLastExactModelDefinition(
+    Array.isArray(provider?.models) ? provider.models : undefined,
+    model.id,
+  );
+  const customModelCompat = asRecord(asRecord(customModel)?.compat);
+  const runtimeCompat = asRecord(model.compat);
+  const modelOverride = asRecord(asRecord(provider?.modelOverrides)?.[model.id]);
+  const modelOverrideCompat = asRecord(modelOverride?.compat);
+
+  return [
+    ...(providerCompat ? [{ source: "provider" as const, compat: providerCompat }] : []),
+    ...(customModelCompat ? [{ source: "model" as const, compat: customModelCompat }] : []),
+    ...(runtimeCompat ? [{ source: "runtime" as const, compat: runtimeCompat }] : []),
+    ...(modelOverrideCompat ? [{ source: "modelOverride" as const, compat: modelOverrideCompat }] : []),
+  ];
+}
+
+function resolveEffectiveCompatFromConfig(model: PiModel, config: unknown): CacheCompat {
+  // Pi's effective precedence is provider → models[] → runtime model →
+  // modelOverrides. The runtime layer is intentionally included between the
+  // config model and modelOverride: extension providers can replace the model
+  // object after lower config layers were applied, while the override remains
+  // Pi's highest-precedence user layer.
+  return mergeCacheCompat(...getEffectiveCompatSources(model, config).map(({ compat }) => compat));
+}
+
+function getEffectiveCompatValueSource(
+  model: PiModel,
+  config: unknown,
+  key: keyof CacheCompat,
+): "provider" | "model" | "runtime" | "modelOverride" | undefined {
+  let source: "provider" | "model" | "runtime" | "modelOverride" | undefined;
+  for (const candidate of getEffectiveCompatSources(model, config)) {
+    if (Object.prototype.hasOwnProperty.call(candidate.compat, key)) source = candidate.source;
+  }
+  return source;
+}
+
+function isOptionalString(value: unknown): boolean {
+  return value === undefined || (typeof value === "string" && value.length > 0);
+}
+
+function isOptionalBoolean(value: unknown): boolean {
+  return value === undefined || typeof value === "boolean";
+}
+
+function isFiniteNumber(value: unknown): value is number {
+  return typeof value === "number" && Number.isFinite(value);
+}
+
+function isRecordOfStrings(value: unknown): boolean {
+  const record = asRecord(value);
+  return !!record && Object.values(record).every((entry) => typeof entry === "string");
+}
+
+function isThinkingLevelMap(value: unknown): boolean {
+  const map = asRecord(value);
+  if (!map) return false;
+  return ["off", "minimal", "low", "medium", "high", "xhigh", "max"].every((key) =>
+    map[key] === undefined || map[key] === null || typeof map[key] === "string"
+  );
+}
+
+function isValidModelCostTier(value: unknown): boolean {
+  const tier = asRecord(value);
+  return !!tier
+    && isFiniteNumber(tier.inputTokensAbove)
+    && isFiniteNumber(tier.input)
+    && isFiniteNumber(tier.output)
+    && isFiniteNumber(tier.cacheRead)
+    && isFiniteNumber(tier.cacheWrite);
+}
+
+function isValidModelCost(value: unknown, partial: boolean): boolean {
+  const cost = asRecord(value);
+  if (!cost) return false;
+  for (const key of ["input", "output", "cacheRead", "cacheWrite"]) {
+    if ((!partial || cost[key] !== undefined) && !isFiniteNumber(cost[key])) return false;
+  }
+  return cost.tiers === undefined || (Array.isArray(cost.tiers) && cost.tiers.every(isValidModelCostTier));
+}
+
+function isValidModelDefinition(value: unknown): boolean {
+  const model = asRecord(value);
+  if (!model || !isNonEmptyString(model.id)) return false;
+  if (!isOptionalString(model.name) || !isOptionalString(model.api) || !isOptionalString(model.baseUrl)) return false;
+  if (!isOptionalBoolean(model.reasoning) || !isValidCompatRecord(model.compat)) return false;
+  if (model.thinkingLevelMap !== undefined && !isThinkingLevelMap(model.thinkingLevelMap)) return false;
+  if (model.input !== undefined && (!Array.isArray(model.input) || !model.input.every((entry) => entry === "text" || entry === "image"))) return false;
+  if (model.cost !== undefined && !isValidModelCost(model.cost, false)) return false;
+  if (model.contextWindow !== undefined && !isFiniteNumber(model.contextWindow)) return false;
+  if (model.maxTokens !== undefined && !isFiniteNumber(model.maxTokens)) return false;
+  if (model.samplingParams !== undefined && !asRecord(model.samplingParams)) return false;
+  if (model.headers !== undefined && !isRecordOfStrings(model.headers)) return false;
+  return true;
+}
+
+function isValidModelOverride(value: unknown): boolean {
+  const override = asRecord(value);
+  if (!override || !isValidCompatRecord(override.compat)) return false;
+  if (!isOptionalString(override.name) || !isOptionalBoolean(override.reasoning)) return false;
+  if (override.thinkingLevelMap !== undefined && !isThinkingLevelMap(override.thinkingLevelMap)) return false;
+  if (override.input !== undefined && (!Array.isArray(override.input) || !override.input.every((entry) => entry === "text" || entry === "image"))) return false;
+  if (override.cost !== undefined && !isValidModelCost(override.cost, true)) return false;
+  if (override.contextWindow !== undefined && !isFiniteNumber(override.contextWindow)) return false;
+  if (override.maxTokens !== undefined && !isFiniteNumber(override.maxTokens)) return false;
+  if (override.samplingParams !== undefined && !asRecord(override.samplingParams)) return false;
+  if (override.headers !== undefined && !isRecordOfStrings(override.headers)) return false;
+  return true;
+}
+
+function isValidOpenAICompletionsCompat(compat: UnknownRecord): boolean {
+  const booleanKeys = [
+    "supportsStore", "supportsDeveloperRole", "supportsReasoningEffort",
+    "supportsUsageInStreaming", "requiresToolResultName", "requiresAssistantAfterToolResult",
+    "requiresThinkingAsText", "requiresReasoningContentOnAssistantMessages",
+    "supportsOpenAIGrammarTools", "supportsStrictMode", "sendSessionAffinityHeaders",
+    "supportsLongCacheRetention",
+  ];
+  if (booleanKeys.some((key) => !isOptionalBoolean(compat[key]))) return false;
+  if (compat.maxTokensField !== undefined && compat.maxTokensField !== "max_completion_tokens" && compat.maxTokensField !== "max_tokens") return false;
+  if (compat.thinkingFormat !== undefined && ![
+    "openai", "openrouter", "together", "baseten", "deepseek", "zai", "qwen",
+    "chat-template", "qwen-chat-template", "string-thinking", "ant-ling",
+  ].includes(String(compat.thinkingFormat))) return false;
+  if (compat.cacheControlFormat !== undefined && compat.cacheControlFormat !== "anthropic") return false;
+  if (compat.deferredToolsMode !== undefined && compat.deferredToolsMode !== "kimi") return false;
+  if (compat.sessionAffinityFormat !== undefined && !["openai", "openai-nosession", "openrouter"].includes(String(compat.sessionAffinityFormat))) return false;
+  for (const key of NESTED_COMPAT_KEYS) {
+    if (compat[key] !== undefined && !asRecord(compat[key])) return false;
+  }
+  return true;
+}
+
+function isValidOpenAIResponsesCompat(compat: UnknownRecord): boolean {
+  const booleanKeys = [
+    "supportsDeveloperRole", "supportsLongCacheRetention", "supportsStrictMode",
+    "supportsOpenAIGrammarTools", "supportsAdditionalTools", "supportsToolSearch",
+  ];
+  if (booleanKeys.some((key) => !isOptionalBoolean(compat[key]))) return false;
+  return compat.sessionAffinityFormat === undefined
+    || ["openai", "openai-nosession", "openrouter"].includes(String(compat.sessionAffinityFormat));
+}
+
+function isValidAnthropicMessagesCompat(compat: UnknownRecord): boolean {
+  return [
+    "supportsEagerToolInputStreaming", "supportsLongCacheRetention",
+    "sendSessionAffinityHeaders", "supportsCacheControlOnTools", "supportsTemperature",
+    "forceAdaptiveThinking", "allowEmptySignature", "supportsStrictTools", "supportsToolReferences",
+  ].every((key) => isOptionalBoolean(compat[key]));
+}
+
+function isValidCompatRecord(value: unknown): boolean {
+  if (value === undefined) return true;
+  const compat = asRecord(value);
+  return !!compat && (
+    isValidOpenAICompletionsCompat(compat)
+    || isValidOpenAIResponsesCompat(compat)
+    || isValidAnthropicMessagesCompat(compat)
+  );
+}
+
+// Credential-blind fail-closed subset of Pi's models.json validation. This
+// intentionally covers only fields consumed by compat diagnostics, transport
+// recovery, and exact route fallback; it is not a replacement for Pi's full
+// schema and must stay conservative when the file is malformed.
+function isValidModelsConfigForEffectiveCompat(value: unknown): boolean {
+  const root = asRecord(value);
+  const providers = asRecord(root?.providers);
+  if (!root || !providers) return false;
+
+  for (const providerValue of Object.values(providers)) {
+    const provider = asRecord(providerValue);
+    if (!provider) return false;
+    if (!isOptionalString(provider.name) || !isOptionalString(provider.baseUrl) || !isOptionalString(provider.apiKey) || !isOptionalString(provider.api)) return false;
+    if (!isOptionalBoolean(provider.authHeader) || !isValidCompatRecord(provider.compat)) return false;
+    if (provider.oauth !== undefined && provider.oauth !== "radius") return false;
+    if (provider.headers !== undefined && !isRecordOfStrings(provider.headers)) return false;
+    if (provider.models !== undefined && (!Array.isArray(provider.models) || !provider.models.every(isValidModelDefinition))) return false;
+    const overrides = provider.modelOverrides === undefined ? undefined : asRecord(provider.modelOverrides);
+    if (provider.modelOverrides !== undefined && (!overrides || !Object.values(overrides).every(isValidModelOverride))) return false;
+  }
+  return true;
+}
+
+type ModelsConfigCache = {
+  signature: string;
+  value: unknown | undefined;
+};
+
+let modelsConfigCache: ModelsConfigCache | undefined;
+
+function invalidateModelsConfigCache(): void {
+  modelsConfigCache = undefined;
+}
+
+function getModelsConfigSignature(): string {
+  try {
+    const info = statSync(MODELS_JSON_PATH);
+    return `${info.dev}:${info.ino}:${info.size}:${info.mtimeMs}:${info.ctimeMs}`;
+  } catch {
+    return "missing";
+  }
+}
+
+function readEffectiveCompatConfig(): unknown | undefined {
+  const signature = getModelsConfigSignature();
+  if (modelsConfigCache?.signature === signature) return modelsConfigCache.value;
+
+  let value: unknown | undefined;
+  if (signature !== "missing") {
+    try {
+      const parsed = parseJsonc(readFileSync(MODELS_JSON_PATH, "utf8"));
+      value = isValidModelsConfigForEffectiveCompat(parsed) ? parsed : undefined;
+    } catch {
+      value = undefined;
+    }
+  }
+
+  modelsConfigCache = { signature, value };
+  return value;
+}
+
 function getCompat(model: PiModel | undefined): CacheCompat {
   if (!model) return {} as CacheCompat;
+  return resolveEffectiveCompatFromConfig(model, readEffectiveCompatConfig());
+}
 
-  // Pi merges provider.compat with model.compat (model wins on conflicts)
-  // We approximate this by reading from ctx.model which should already have merged compat
-  // However, for safety, we check both levels if available
-  const modelCompat = (model.compat ?? {}) as CacheCompat;
+function hasProviderHeader(headers: Record<string, unknown>, name: string): boolean {
+  const normalized = name.toLowerCase();
+  return Object.keys(headers).some((headerName) => headerName.toLowerCase() === normalized);
+}
 
-  // Note: ctx.model from Pi should already contain merged compat,
-  // but we document the two-level structure for clarity
-  return modelCompat;
+function setProviderHeaderIfMissing(
+  headers: Record<string, string | null | undefined>,
+  name: string,
+  value: string,
+): boolean {
+  if (hasProviderHeader(headers, name)) return false;
+  headers[name] = value;
+  return true;
+}
+
+function addEffectiveSessionAffinityHeaders(
+  headers: Record<string, string | null | undefined>,
+  model: PiModel | undefined,
+  sessionId: string | undefined,
+  effectiveCompat?: CacheCompat,
+  optimizerEnabled: boolean = runtimeOptimizerEnabled,
+  effectiveSource?: "provider" | "model" | "runtime" | "modelOverride",
+): boolean {
+  if (!optimizerEnabled || !model || !isNonEmptyString(sessionId)) return false;
+  if (!isOpenAICompatibleProxyApi(model.api) || !isNonEmptyString(model.baseUrl) || isOfficialOpenAIBaseUrl(model)) return false;
+  const config = effectiveCompat === undefined || effectiveSource === undefined
+    ? readEffectiveCompatConfig()
+    : undefined;
+  const compat = effectiveCompat ?? resolveEffectiveCompatFromConfig(model, config);
+  const source = effectiveSource
+    ?? getEffectiveCompatValueSource(model, config, "sendSessionAffinityHeaders")
+    ?? (Object.prototype.hasOwnProperty.call(asRecord(model.compat) ?? {}, "sendSessionAffinityHeaders") ? "runtime" : undefined);
+  if (compat.sendSessionAffinityHeaders !== true) return false;
+
+  // Pi already handles an effective runtime-model true. The bridge is only for
+  // effective values that live in models.json layers and were lost when an
+  // extension provider rebuilt/replaced the runtime model object.
+  if (source === "runtime") return false;
+
+  const format = compat.sessionAffinityFormat ?? (
+    lower(model.provider).includes("openrouter") || lower(model.baseUrl).includes("openrouter.ai")
+      ? "openrouter"
+      : "openai"
+  );
+  const value = sessionId.trim();
+  let changed = false;
+  if (format === "openrouter") {
+    return setProviderHeaderIfMissing(headers, "x-session-id", value);
+  }
+  if (format === "openai") {
+    changed = setProviderHeaderIfMissing(headers, "session_id", value) || changed;
+  }
+  changed = setProviderHeaderIfMissing(headers, "x-client-request-id", value) || changed;
+  changed = setProviderHeaderIfMissing(headers, "x-session-affinity", value) || changed;
+  return changed;
 }
 
 /** Join display-only path fragments without resolving them for I/O. */
@@ -1626,6 +1976,14 @@ function isKimiCodingEmptySignatureModel(model: PiModel | undefined): boolean {
 }
 
 function isAdaptiveThinkingCompatApplicable(model: PiModel): boolean {
+  // A routed registry miss may preserve the upstream API/model identity while
+  // lacking a verified endpoint. Do not diagnose or suggest compat fixes for
+  // that incomplete transport metadata; ordinary direct Anthropic models still
+  // use their normal native adaptive-thinking check.
+  if (isRoutedFallbackModel(model) && !isNonEmptyString(model.baseUrl)) {
+    return false;
+  }
+
   return lower(model.api) === "anthropic-messages"
     && (isAdaptiveGenerationModel(model) || isKimiCodingAdaptiveModel(model));
 }
@@ -2451,13 +2809,17 @@ function isOfficialOpenAIBaseUrl(model: PiModel): boolean {
   }
 }
 
+function isKnownThirdPartyOpenAIEndpoint(model: PiModel): boolean {
+  return isNonEmptyString(model.baseUrl) && !isOfficialOpenAIBaseUrl(model);
+}
+
 function describeMissingOpenAIFamilyProxyCompat(model: PiModel): string[] {
   const compat = getCompat(model);
   const missing: string[] = [];
 
   if (!isOpenAIFamilyModel(model)) return missing;
   if (!isOpenAICompatibleProxyApi(model.api)) return missing;
-  if (isOfficialOpenAIBaseUrl(model)) return missing;
+  if (!isKnownThirdPartyOpenAIEndpoint(model)) return missing;
 
   if (compat.sendSessionAffinityHeaders !== true) {
     missing.push("sendSessionAffinityHeaders");
@@ -2477,7 +2839,7 @@ function describeMissingOpenAICompatibleProxyCompat(model: PiModel): string[] {
   const missing: string[] = [];
 
   if (!isOpenAICompatibleProxyApi(model.api)) return missing;
-  if (isOfficialOpenAIBaseUrl(model)) return missing;
+  if (!isKnownThirdPartyOpenAIEndpoint(model)) return missing;
   if (isPiBuiltInLlamaCppModel(model)) return missing;
 
   if (compat.sendSessionAffinityHeaders === undefined) {
@@ -2505,7 +2867,7 @@ function describeOptionalOpenAICompatibleProxyCompat(model: PiModel): string[] {
   const optional: string[] = [];
 
   if (!isOpenAICompatibleProxyApi(model.api)) return optional;
-  if (isOfficialOpenAIBaseUrl(model)) return optional;
+  if (!isKnownThirdPartyOpenAIEndpoint(model)) return optional;
   if (isPiBuiltInLlamaCppModel(model)) return optional;
 
   if (compat.supportsLongCacheRetention !== true) {
@@ -2690,7 +3052,10 @@ function describeMissingDeepSeekCompat(model: PiModel): string[] {
 }
 
 function isDeepSeekCompatCheckApplicable(model: PiModel): boolean {
-  return isDeepSeekLikeModel(model) && isOpenAICompatibleApi(model.api) && !isPiBuiltInLlamaCppModel(model);
+  return isDeepSeekLikeModel(model)
+    && isOpenAICompatibleApi(model.api)
+    && isKnownThirdPartyOpenAIEndpoint(model)
+    && !isPiBuiltInLlamaCppModel(model);
 }
 
 function describeMissingCacheCompatForModel(model: PiModel): string[] {
@@ -4852,12 +5217,12 @@ async function maybeCleanupStatsShardsV7(now = Date.now()): Promise<void> {
 }
 
 function isCompatCheckApplicable(model: PiModel): boolean {
-  return isOpenAICompatibleProxyApi(model.api) && !isOfficialOpenAIBaseUrl(model) && !isPiBuiltInLlamaCppModel(model);
+  return isOpenAICompatibleProxyApi(model.api) && isKnownThirdPartyOpenAIEndpoint(model) && !isPiBuiltInLlamaCppModel(model);
 }
 
 function isPromptCacheRetention400Applicable(model: PiModel): boolean {
   return isOpenAICompatibleApi(model.api) &&
-    !isOfficialOpenAIBaseUrl(model) &&
+    isKnownThirdPartyOpenAIEndpoint(model) &&
     !isPiBuiltInLlamaCppModel(model) &&
     getCompat(model).supportsLongCacheRetention === true;
 }
@@ -4885,6 +5250,7 @@ function isExplicitPromptCacheRetentionUnsupportedApplicable(model: PiModel): bo
  */
 function isSessionAffinity403Applicable(model: PiModel): boolean {
   if (!isOpenAICompatibleProxyApi(model.api)) return false;
+  if (!isKnownThirdPartyOpenAIEndpoint(model)) return false;
   if (isPiBuiltInLlamaCppModel(model)) return false;
   return getCompat(model).sendSessionAffinityHeaders === true;
 }
@@ -4901,7 +5267,7 @@ function isSessionAffinity403Applicable(model: PiModel): boolean {
  */
 function isOpenAISdkHeader403Applicable(model: PiModel): boolean {
   if (!isOpenAICompatibleProxyApi(model.api)) return false;
-  if (isOfficialOpenAIBaseUrl(model)) return false;
+  if (!isKnownThirdPartyOpenAIEndpoint(model)) return false;
   if (isPiBuiltInLlamaCppModel(model)) return false;
   return getCompat(model).sendSessionAffinityHeaders !== true;
 }
@@ -4939,8 +5305,9 @@ function describeRouterChannelDiagnostics(model: PiModel): string[] {
     return notes;
   }
 
-  // Official OpenAI bypass — no notes needed.
-  if (isOfficialOpenAIBaseUrl(model)) {
+  // Unknown/default endpoints and official OpenAI are not diagnosable as
+  // third-party router channels. Request-header bridging also fails closed.
+  if (!isKnownThirdPartyOpenAIEndpoint(model)) {
     return notes;
   }
 
@@ -5107,6 +5474,13 @@ function getCompatCheckNotApplicableLines(model: PiModel): string[] {
     ];
   }
 
+  if (isOpenAICompatibleApi(api) && !isNonEmptyString(model.baseUrl)) {
+    return [
+      "ℹ️ Compat check not applicable for this model.",
+      "   Upstream endpoint metadata is unavailable; session-affinity header injection is disabled.",
+    ];
+  }
+
   return ["ℹ️ Compat check not applicable for this model."];
 }
 
@@ -5759,12 +6133,13 @@ function resolveExplicitCompatValue(
     return { source: "modelOverride", value: overrideCompat[compatKey] };
   }
 
-  if (Array.isArray(provider.models)) {
-    const model = provider.models.find((entry: unknown) => asRecord(entry)?.id === modelId);
-    const modelCompat = asRecord(asRecord(model)?.compat);
-    if (modelCompat && Object.prototype.hasOwnProperty.call(modelCompat, compatKey)) {
-      return { source: "model", value: modelCompat[compatKey] };
-    }
+  const model = findLastExactModelDefinition(
+    Array.isArray(provider.models) ? provider.models : undefined,
+    modelId,
+  );
+  const modelCompat = asRecord(model?.compat);
+  if (modelCompat && Object.prototype.hasOwnProperty.call(modelCompat, compatKey)) {
+    return { source: "model", value: modelCompat[compatKey] };
   }
 
   const providerCompat = asRecord(provider.compat);
@@ -6008,9 +6383,14 @@ function locateModelInJsonc(
       allModelIds.push(elementId);
     }
 
-    if (elementId === modelId && modelBrace < 0) {
+    if (elementId === modelId) {
+      // Match Pi's provider composer: later duplicate definitions replace
+      // earlier ones. The effective/fix target is therefore the last exact id.
       modelBrace = elementBrace;
       modelEndBrace = elementEnd;
+      compatKeyStartClean = -1;
+      compatBrace = -1;
+      compatEndBrace = -1;
 
       const compatKey = findJsonObjectKey(clean, modelBrace, "compat");
       if (compatKey && compatKey.keyStart < modelEndBrace) {
@@ -7158,6 +7538,7 @@ export const __internals_for_tests = {
   buildSafeOpenAIProxyCompatSuggestion,
   getPromptCacheRetentionUnsupportedHint,
   isOfficialOpenAIBaseUrl,
+  isKnownThirdPartyOpenAIEndpoint,
   isCompatCheckApplicable,
   isPromptCacheRetention400Applicable,
   isSessionAffinity403Applicable,
@@ -7278,6 +7659,12 @@ export const __internals_for_tests = {
   buildOpenAIProxyCompatWarningText,
   getModelIdNameTokenValues,
   getAssistantMessageModelTokenValues,
+  mergeCacheCompat,
+  resolveEffectiveCompatFromConfig,
+  getEffectiveCompatValueSource,
+  findLastExactModelDefinition,
+  isValidModelsConfigForEffectiveCompat,
+  addEffectiveSessionAffinityHeaders,
   getCompat,
   modelKey,
   modelFromAssistantMessage,
@@ -7352,6 +7739,8 @@ export const __internals_for_tests = {
   parseRouteSnapshot,
   resolveActiveRouteSnapshot,
   routeSnapshotToPiModel,
+  isRoutedFallbackModel,
+  applyConfiguredTransportToModel,
   resolveRouteModel,
   isVirtualRoutingModel,
   installCacheHintsService,
@@ -7951,13 +8340,11 @@ export default function (pi: ExtensionAPI) {
    * The caller strips prompt_cache_retention when this returns false.
    */
   function hasExplicitLongRetentionOptIn(model: PiModel): boolean {
-    try {
-      const text = readFileSync(MODELS_JSON_PATH, "utf8");
-      const parsed = parseJsonc(text);
-      return hasExplicitLongRetentionOptInFromConfig(parsed, model.provider, model.id);
-    } catch {
-      return false;
-    }
+    return hasExplicitLongRetentionOptInFromConfig(
+      readEffectiveCompatConfig(),
+      model.provider,
+      model.id,
+    );
   }
 
   pi.on("session_start", async (event, ctx) => {
@@ -8125,6 +8512,15 @@ export default function (pi: ExtensionAPI) {
     return {};
   });
 
+  pi.on("before_provider_headers", (event, ctx) => {
+    const requestModel = resolveRouteModel(ctx.model, ctx) ?? ctx.model;
+    addEffectiveSessionAffinityHeaders(
+      event.headers,
+      requestModel,
+      ctx.sessionManager.getSessionId(),
+    );
+  });
+
   pi.on("before_provider_request", (event, ctx) => {
     const requestModel = resolveRouteModel(ctx.model, ctx) ?? ctx.model;
 
@@ -8742,6 +9138,20 @@ export default function (pi: ExtensionAPI) {
           return;
         }
 
+        const duplicateTargetDefinitions = location.allModelIds.filter(
+          (configuredModelId) => configuredModelId === suggestion.modelId,
+        ).length;
+        if (duplicateTargetDefinitions > 1) {
+          cmdCtx.ui.notify(
+            `❌ ${getModelsJsonDisplayPath()} contains ${duplicateTargetDefinitions} custom model definitions ` +
+            `with the exact id "${suggestion.modelId}" under providers["${suggestion.providerLabel}"].\n` +
+            `Pi uses the last definition, but /cache-optimizer fix refuses an ambiguous duplicate-id edit. ` +
+            `Remove or consolidate the duplicates, then run the command again.`,
+            "error",
+          );
+          return;
+        }
+
         // Compose the modified text — observed runtime failures are always
         // model-scoped; ordinary compat fixes use the safety-based placement.
         const decision = chooseFixPlacement(
@@ -8840,6 +9250,7 @@ export default function (pi: ExtensionAPI) {
             return;
           }
 
+          invalidateModelsConfigCache();
           cmdCtx.ui.notify(
             `✅ Fix applied to ${getModelsJsonDisplayPath()}.\n` +
             `Backup saved to: ${backupPath}\n` +

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-cache-optimizer",
-  "version": "2.8.4",
+  "version": "2.8.6",
   "description": "Improve Pi prompt/KV cache hit rates with stable prompts, OpenAI-compatible cache keys, proxy compat warnings, and footer cache stats.",
   "keywords": [
     "pi-package",