pi-cache-optimizer 2.8.10 → 2.8.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -8
- package/README.zh-CN.md +20 -8
- package/index.ts +475 -31
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -25,6 +25,7 @@ Pi extension for improving provider-side KV / prompt cache hit rates. It keeps s
|
|
|
25
25
|
- [Auto-repair with `/cache-optimizer fix`](#auto-repair-with-cache-optimizer-fix)
|
|
26
26
|
- [DeepSeek protocol safety and rollback](#deepseek-protocol-safety-and-rollback)
|
|
27
27
|
- [Footer stats](#footer-stats)
|
|
28
|
+
- [Native virtual models (Pi 0.99+)](#native-virtual-models-pi-099)
|
|
28
29
|
- [For router / virtual-channel extension authors](#for-router--virtual-channel-extension-authors)
|
|
29
30
|
- [Uninstall](#uninstall)
|
|
30
31
|
- [Verify effect](#verify-effect)
|
|
@@ -42,6 +43,7 @@ Pi extension for improving provider-side KV / prompt cache hit rates. It keeps s
|
|
|
42
43
|
- Shows current conversation-session provider/model footer stats by default; `total` aggregates all valid local shards for the exact provider/model.
|
|
43
44
|
- Supports optional router-extension integration through versioned global protocols (`Symbol.for("pi.routing.registry.v1")` and `Symbol.for("pi.cache.hints.v1")`) without importing router packages.
|
|
44
45
|
- Includes disabled-by-default deterministic ordering for verified built-in tool payloads.
|
|
46
|
+
- Supports Pi 0.99+ native virtual models (`pi.registerVirtualModel()`): request hooks, footer stats, and diagnostics act on the physical model each request is routed to.
|
|
45
47
|
|
|
46
48
|
Caching is provider-side and best-effort. Third-party proxies and router extensions can still hide cache usage, reject unsupported parameters, or route requests across multiple upstreams.
|
|
47
49
|
|
|
@@ -61,7 +63,7 @@ Run `/reload` in Pi after install/update/remove so extension hooks refresh.
|
|
|
61
63
|
|
|
62
64
|
On Pi 0.79.7 and newer, `pi update` updates Pi itself only. To update installed Pi packages such as this extension, run `pi update --extensions` (packages only) or `pi update --all` (Pi + packages).
|
|
63
65
|
|
|
64
|
-
This extension requires Pi 0.82+ and is validated against Pi 0.
|
|
66
|
+
This extension requires Pi 0.82+ and is validated against Pi 0.99.2. It uses the official Pi package types directly for type-checking, along with extension hooks, `getAgentDir()`, and prompt options shared by those versions; it does not depend on Pi 0.83+ APIs such as `ctx.scopedModels` or the bundled TypeBox 1.3 aliases. Native virtual model support and codemode nested-call coalescing activate only on Pi 0.99+ hosts that produce them; older hosts keep the previous behavior.
|
|
65
67
|
|
|
66
68
|
## Commands
|
|
67
69
|
|
|
@@ -134,7 +136,7 @@ The explicit setting is stored in `pi-cache-optimizer-config.json` under Pi's ag
|
|
|
134
136
|
|
|
135
137
|
## Per-model `prompt_cache_key` opt-out
|
|
136
138
|
|
|
137
|
-
Some OpenAI-compatible endpoints reject `prompt_cache_key` with HTTP 400 even though the same field is valid for other providers. Pi 0.
|
|
139
|
+
Some OpenAI-compatible endpoints reject `prompt_cache_key` with HTTP 400 even though the same field is valid for other providers. Pi 0.99.2 has no native `supportsPromptCacheKey` compat field; do **not** add that unknown field to `models.json`. `supportsLongCacheRetention` is not an equivalent switch and should not be used for this purpose.
|
|
138
140
|
|
|
139
141
|
When the extension observes an explicit field-level `prompt_cache_key` unsupported error for the exact provider/model, ordinary `/cache-optimizer fix` offers a confirmed model-scoped repair. Value-validation failures and conditional restrictions such as “not allowed when temperature is set” do not qualify. If concurrent responses from different models cannot be correlated because Pi provides no request ID, header-only evidence is ignored unless the finalized assistant message supplies exact provider/model identity. If you already know that the endpoint rejects the field, use the explicit command:
|
|
140
142
|
|
|
@@ -148,7 +150,7 @@ The preview explains that the setting is stored in the extension-owned `pi-cache
|
|
|
148
150
|
|
|
149
151
|
Third-party `openai-completions` proxies (LiteLLM / OneAPI / NewAPI / OpenRouter-like channels) often route one session across multiple upstream backends. That splits provider-side prompt caches.
|
|
150
152
|
|
|
151
|
-
Pi 0.84.1 also fixes built-in Fireworks compatibility for models that reject `prompt_cache_retention`; the extension avoids provider-name special cases and resolves exact provider/model compat from `models.json` plus the runtime model. Pi 0.81+ also has a built-in `llama.cpp` provider using an OpenAI-shaped transport. Pi 0.82+ core generates a session `prompt_cache_key` for it when cache retention is enabled, so this extension preserves that key and may add the same conservative fallback when missing. The built-in provider's explicit compat fingerprint is excluded from generic proxy routing/session-affinity advice, but a custom or overridden provider that merely reuses the id `llama.cpp` is treated like any other OpenAI-compatible channel. `prompt_cache_retention` remains subject to the normal safety rule: keep it only for official OpenAI or an explicit effective `supportsLongCacheRetention: true` opt-in in `models.json`; otherwise strip it before sending. Pi 0.
|
|
153
|
+
Pi 0.84.1 also fixes built-in Fireworks compatibility for models that reject `prompt_cache_retention`; the extension avoids provider-name special cases and resolves exact provider/model compat from `models.json` plus the runtime model. Pi 0.81+ also has a built-in `llama.cpp` provider using an OpenAI-shaped transport. Pi 0.82+ core generates a session `prompt_cache_key` for it when cache retention is enabled, so this extension preserves that key and may add the same conservative fallback when missing. The built-in provider's explicit compat fingerprint is excluded from generic proxy routing/session-affinity advice, but a custom or overridden provider that merely reuses the id `llama.cpp` is treated like any other OpenAI-compatible channel. `prompt_cache_retention` remains subject to the normal safety rule: keep it only for official OpenAI or an explicit effective `supportsLongCacheRetention: true` opt-in in `models.json`; otherwise strip it before sending. Pi 0.99.2 has no native `supportsPromptCacheKey` compat field, so the per-model key opt-out is stored in this extension's `pi-cache-optimizer-config.json` instead of `models.json`. The extension config is independent of Pi's compat precedence and only affects the exact provider/model listed there.
|
|
152
154
|
|
|
153
155
|
For real proxies, start with session affinity:
|
|
154
156
|
|
|
@@ -174,7 +176,7 @@ Notes:
|
|
|
174
176
|
|
|
175
177
|
- `sendSessionAffinityHeaders: true` is the safe default when your proxy supports sticky routing.
|
|
176
178
|
- `supportsLongCacheRetention: true` is optional. Add it only when the endpoint explicitly supports OpenAI long prompt cache retention.
|
|
177
|
-
- Do not add `supportsPromptCacheKey` to `models.json`: Pi 0.
|
|
179
|
+
- Do not add `supportsPromptCacheKey` to `models.json`: Pi 0.99.2 does not define that compat field. Use `/cache-optimizer fix prompt-cache-key` to store an exact provider/model omit rule in the extension-owned config; it removes both key spellings, including a key supplied by Pi.
|
|
178
180
|
- If you see `400 Unsupported parameter: prompt_cache_retention`, remove/avoid `supportsLongCacheRetention` for that channel. Keep `sendSessionAffinityHeaders` if supported. The extension detects the explicit error from response headers or the finalized assistant error message and strips the parameter from subsequent requests in the current process.
|
|
179
181
|
- Use `/cache-optimizer compat` or `/cache-optimizer doctor` to see model-specific advice.
|
|
180
182
|
- DeepSeek model names select the `DS cache` adapter only; they do not prove a reasoning wire protocol. Generic cache/routing advice remains active for absent or non-DeepSeek formats. DeepSeek replay advice is shown only when effective `compat.thinkingFormat: "deepseek"` is explicitly configured; it never treats `thinkingFormat` as a missing fix key.
|
|
@@ -262,7 +264,7 @@ Pi 0.80.9+ already includes Kimi K3 in built-in Kimi Coding, Moonshot AI / China
|
|
|
262
264
|
- DeepSeek Pi Mono replay compat (`requiresReasoningContentOnAssistantMessages: true` only when `thinkingFormat: "deepseek"` is already explicit; `/fix` never invents that format)
|
|
263
265
|
- OpenAI-compatible proxy session affinity (`sendSessionAffinityHeaders: true` for `openai-completions`). Pi 0.80.7+ controls `openai-responses` header shape with `sessionAffinityFormat` and auto-detects its default; this extension no longer writes the removed `sendSessionIdHeader` field.
|
|
264
266
|
|
|
265
|
-
**Scope:**
|
|
267
|
+
**Scope:** the fix starts from the currently active model. An affinity-only repair can write provider-level compat once for all applicable models on that provider, including models absent from `models[]` when the provider entry already exists and no higher-priority setting shadows it. Explicit per-model `false` remains an opt-out. Other or ambiguous repairs stay model-scoped.
|
|
266
268
|
|
|
267
269
|
**Safety:**
|
|
268
270
|
|
|
@@ -275,15 +277,15 @@ Pi 0.80.9+ already includes Kimi K3 in built-in Kimi Coding, Moonshot AI / China
|
|
|
275
277
|
7. Writes a privacy-safe, versioned receipt atomically only after a successful write; the receipt contains transaction/model identity, placement, scalar compat before/after values, file hashes, backup filename, timestamps/status, and no credentials or request data
|
|
276
278
|
8. Uses unique, non-overwriting backup names and falls back to manual guidance if the JSONC scanner cannot confidently locate the target
|
|
277
279
|
|
|
278
|
-
Existing `modelOverrides[modelId]` entries have Pi's highest precedence, so `fix` repairs them directly. For built-in or API-login models without a custom `models[]` entry, `fix` creates a compat-only `modelOverrides` entry instead of inventing a custom model definition. Runtime-observed provider failures are always written to that highest-precedence model override so extension-provided runtime compat cannot shadow the repair. Self-validation checks the full provider → custom model → runtime model → modelOverride result and rejects an ineffective lower-layer edit.
|
|
280
|
+
Existing `modelOverrides[modelId]` entries have Pi's highest precedence, so `fix` repairs them directly. For built-in or API-login models without a custom `models[]` entry, an affinity-only fix prefers provider-level compat when the existing provider is safely locatable and no higher-priority explicit/runtime value shadows it; otherwise `fix` creates a compat-only `modelOverrides` entry instead of inventing a custom model definition. Runtime-observed provider failures are always written to that highest-precedence model override so extension-provided runtime compat cannot shadow the repair. Self-validation checks the full provider → custom model → runtime model → modelOverride result and rejects an ineffective lower-layer edit.
|
|
279
281
|
|
|
280
282
|
**Non-interactive mode:** refuses to write; shows manual edit guidance instead.
|
|
281
283
|
|
|
282
|
-
**Run:** `/cache-optimizer fix` when the active model has detected compat issues. The command shows "nothing to fix" when compat is already complete.
|
|
284
|
+
**Run:** `/cache-optimizer fix` when the active model has detected compat issues. Affinity-only missing-compat notifications are shown once per provider per extension instance and point to this confirmed command; model-specific warnings remain separate. The command shows "nothing to fix" when compat is already complete.
|
|
283
285
|
|
|
284
286
|
## `/cache-optimizer rollback`
|
|
285
287
|
|
|
286
|
-
Rollback is available through completion, direct execution, and the interactive menu. It always requires UI confirmation; without an interactive UI it gives manual-review guidance and does not write. The command selects the latest unapplied receipt for the active provider/model, validates the recorded backup and current file hashes, creates a new access-mode-preserving rollback backup, and uses temp-file + atomic rename. Fix and rollback transactions are serialized across extension instances; rollback also binds the receipt transaction id/hash from preview through commit and refuses if another transaction replaces it.
|
|
288
|
+
Rollback is available through completion, direct execution, and the interactive menu. It always requires UI confirmation; without an interactive UI it gives manual-review guidance and does not write. The command selects the latest unapplied receipt for the active provider/model (or any active model on that provider for a provider-level receipt), validates the recorded backup and current file hashes, creates a new access-mode-preserving rollback backup, and uses temp-file + atomic rename. Fix and rollback transactions are serialized across extension instances; rollback also binds the receipt transaction id/hash from preview through commit and refuses if another transaction replaces it.
|
|
287
289
|
|
|
288
290
|
If `models.json` is unchanged since the fix, rollback can restore the exact pre-fix JSONC. If the file changed, it never blindly replaces it: it may restore only receipt-owned scalar compat keys whose recorded post-fix values are still present, preserving later user changes. If a receipt-owned key changed, the target was removed/moved, or the fix created a new target entry, it refuses and points to the recorded backup for manual review. Successful rollback marks the receipt and requires `/reload` or a restart.
|
|
289
291
|
|
|
@@ -358,6 +360,16 @@ Supported footer labels include: DS, Claude, OpenAI, Gemini, Kimi, Qwen, GLM, Mi
|
|
|
358
360
|
|
|
359
361
|
Adapter selection uses only model id/name (plus assistant message model/name on message end). Generic OpenAI-shaped APIs are not treated as OpenAI-family unless the model id/name matches a supported family.
|
|
360
362
|
|
|
363
|
+
## Native virtual models (Pi 0.99+)
|
|
364
|
+
|
|
365
|
+
Pi 0.99 lets extensions register virtual models with `pi.registerVirtualModel()`. While one is selected, `ctx.model` stays the virtual model (`api: "pi-virtual"`) and Pi routes every request to a physical model. This extension follows the physical model automatically; router authors do not need the protocol below.
|
|
366
|
+
|
|
367
|
+
- Request hooks read the dispatched model id from the provider payload and match it against physical models with configured credentials. The `prompt_cache_key` fallback, `prompt_cache_retention` safety, Anthropic TTL repair, and per-model `prompt_cache_key` omit rules then apply to that physical model. If several credentialed providers share the id and would be treated differently, the extension does not guess; identity-dependent request changes are skipped.
|
|
368
|
+
- Footer stats and `/cache-optimizer doctor`, `compat`, `stats`, `reset`, and `fix` use the physical model that answered last on the current session branch, matching Pi's own routed-model display and context limits. Doctor and compat name both the virtual selection and that physical model.
|
|
369
|
+
- Before Pi routes the first request, the footer stays empty and diagnostics ask you to send a prompt first.
|
|
370
|
+
- Prompt rewriting is skipped for virtual selections: Pi picks the physical model after the system prompt is built, and a reordered prompt must not reach a safety-filtered Codex route.
|
|
371
|
+
- The session-affinity header bridge is skipped as well, because Pi builds request headers before the payload exists. Pi still sends the physical model's own configured affinity headers.
|
|
372
|
+
|
|
361
373
|
## For router / virtual-channel extension authors
|
|
362
374
|
|
|
363
375
|
If your Pi extension provides a virtual routing provider (for example `router/auto`, `router/smart`, or a profile/channel that forwards to a real upstream), this extension can show cache stats for the real upstream provider/model instead of the virtual shell. Integration is optional, versioned, and does **not** require importing this package.
|
package/README.zh-CN.md
CHANGED
|
@@ -25,6 +25,7 @@
|
|
|
25
25
|
- [使用 `/cache-optimizer fix` 自动修复](#使用-cache-optimizer-fix-自动修复)
|
|
26
26
|
- [DeepSeek 协议安全与回滚](#deepseek-协议安全与回滚)
|
|
27
27
|
- [Footer 统计](#footer-统计)
|
|
28
|
+
- [原生虚拟模型(Pi 0.99+)](#原生虚拟模型pi-099)
|
|
28
29
|
- [Router / Virtual-channel 扩展作者指南](#router--virtual-channel-扩展作者指南)
|
|
29
30
|
- [卸载](#卸载)
|
|
30
31
|
- [验证效果](#验证效果)
|
|
@@ -42,6 +43,7 @@
|
|
|
42
43
|
- Footer 默认显示当前 conversation session 的 provider/model 统计;`total` 可聚合同一精确 provider/model 的所有有效本地 shard。
|
|
43
44
|
- 通过版本化全局协议(`Symbol.for("pi.routing.registry.v1")` 与 `Symbol.for("pi.cache.hints.v1")`)支持可选的 router extension 集成,而不导入任何 router 包。
|
|
44
45
|
- 提供默认关闭的确定性排序,用于已验证的 Pi 内置工具 payload。
|
|
46
|
+
- 支持 Pi 0.99+ 原生虚拟模型(`pi.registerVirtualModel()`):请求 hook、footer 统计和诊断都作用于每次请求实际路由到的物理模型。
|
|
45
47
|
|
|
46
48
|
缓存是 provider 侧的 best-effort 行为。第三方代理和 router extension 仍可能隐藏缓存 usage、拒绝不支持的参数,或把请求路由到多个上游。
|
|
47
49
|
|
|
@@ -61,7 +63,7 @@ pi remove npm:pi-deepseek-cache-optimizer && pi install npm:pi-cache-optimizer
|
|
|
61
63
|
|
|
62
64
|
Pi 0.79.7 及之后,`pi update` 默认只更新 Pi 本体。若要更新已安装的 Pi package(包括本扩展),请运行 `pi update --extensions`(只更新 packages)或 `pi update --all`(Pi 与 packages 一起更新)。
|
|
63
65
|
|
|
64
|
-
本扩展要求 Pi 0.82+,并已使用 Pi 0.
|
|
66
|
+
本扩展要求 Pi 0.82+,并已使用 Pi 0.99.2 验证。TypeScript 校验直接使用官方 Pi package 类型,同时只使用这些版本共有的 extension hooks、`getAgentDir()` 和 prompt options;不依赖 Pi 0.83+ 专有 API(例如 `ctx.scopedModels` 或 bundled TypeBox 1.3 aliases)。原生虚拟模型支持与 codemode 嵌套工具调用合并只在会产生它们的 Pi 0.99+ 上生效,较早版本保持原有行为。
|
|
65
67
|
|
|
66
68
|
## 命令
|
|
67
69
|
|
|
@@ -134,7 +136,7 @@ Footer 默认使用 `session`,避免另一个并行 Pi 终端使用相同 prov
|
|
|
134
136
|
|
|
135
137
|
## 按模型关闭 `prompt_cache_key`
|
|
136
138
|
|
|
137
|
-
某些 OpenAI-compatible endpoint 会因 `prompt_cache_key` 返回 HTTP 400,但同一个字段对其他 provider 可能有效。Pi 0.
|
|
139
|
+
某些 OpenAI-compatible endpoint 会因 `prompt_cache_key` 返回 HTTP 400,但同一个字段对其他 provider 可能有效。Pi 0.99.2 没有原生的 `supportsPromptCacheKey` compat 字段,**不要**把这个未知字段加入 `models.json`。`supportsLongCacheRetention` 也不是等价开关,不应借用来实现此目的。
|
|
138
140
|
|
|
139
141
|
当扩展观察到精确 provider/model 对 `prompt_cache_key` 的明确字段级拒绝后,普通 `/cache-optimizer fix` 会提供经过确认的模型级修复。参数值校验失败,以及“设置 temperature 时不允许”这类条件限制都不构成证据。若不同模型的响应并发交错,而 Pi 又没有提供 request ID,扩展会忽略无法安全关联的 response-header 证据;只有最终 assistant message 提供精确 provider/model 身份时才恢复归因。如果你已经确定 endpoint 不支持该字段,可以主动执行:
|
|
140
142
|
|
|
@@ -148,7 +150,7 @@ Footer 默认使用 `session`,避免另一个并行 Pi 终端使用相同 prov
|
|
|
148
150
|
|
|
149
151
|
LiteLLM / OneAPI / NewAPI / 类 OpenRouter 渠道等第三方 `openai-completions` 代理,常会把同一个 session 分散到多个上游后端,导致 provider 侧 prompt cache 被拆散。
|
|
150
152
|
|
|
151
|
-
Pi 0.84.1 还修复了内置 Fireworks 渠道对拒绝 `prompt_cache_retention` 的模型兼容性;本扩展不按 provider 名称增加特殊分支,而是结合 `models.json` 与 runtime model,按精确 provider/model 解析有效 compat。Pi 0.81+ 也内置了使用 OpenAI-shaped transport 的 `llama.cpp` provider。Pi 0.82+ core 在启用 cache retention 时会为它生成 session `prompt_cache_key`,因此本扩展会保留该 key,并在缺失时使用同样的保守 fallback。只有符合 Pi 内置 provider 明确 compat 指纹的模型会跳过通用 proxy 路由 / session-affinity 建议;仅复用 `llama.cpp` id 的自定义或覆盖 provider 仍按普通 OpenAI-compatible 渠道处理。`prompt_cache_retention` 继续遵循统一安全规则:仅官方 OpenAI 或 `models.json` 中有效配置为 `supportsLongCacheRetention: true` 时保留,否则发送前移除。Pi 0.
|
|
153
|
+
Pi 0.84.1 还修复了内置 Fireworks 渠道对拒绝 `prompt_cache_retention` 的模型兼容性;本扩展不按 provider 名称增加特殊分支,而是结合 `models.json` 与 runtime model,按精确 provider/model 解析有效 compat。Pi 0.81+ 也内置了使用 OpenAI-shaped transport 的 `llama.cpp` provider。Pi 0.82+ core 在启用 cache retention 时会为它生成 session `prompt_cache_key`,因此本扩展会保留该 key,并在缺失时使用同样的保守 fallback。只有符合 Pi 内置 provider 明确 compat 指纹的模型会跳过通用 proxy 路由 / session-affinity 建议;仅复用 `llama.cpp` id 的自定义或覆盖 provider 仍按普通 OpenAI-compatible 渠道处理。`prompt_cache_retention` 继续遵循统一安全规则:仅官方 OpenAI 或 `models.json` 中有效配置为 `supportsLongCacheRetention: true` 时保留,否则发送前移除。Pi 0.99.2 没有原生的 `supportsPromptCacheKey` compat 字段,因此按模型关闭 key 的策略保存在本扩展的 `pi-cache-optimizer-config.json` 中,而不是 `models.json`。扩展配置独立于 Pi 的 compat 优先级,只影响其中列出的精确 provider/model。
|
|
152
154
|
|
|
153
155
|
对真正的代理,建议先启用 session affinity:
|
|
154
156
|
|
|
@@ -174,7 +176,7 @@ Pi 0.84.1 还修复了内置 Fireworks 渠道对拒绝 `prompt_cache_retention`
|
|
|
174
176
|
|
|
175
177
|
- `sendSessionAffinityHeaders: true` 是安全默认项,前提是你的代理支持 sticky routing。
|
|
176
178
|
- `supportsLongCacheRetention: true` 是可选项。只有 endpoint 明确支持 OpenAI long prompt cache retention 时才添加。
|
|
177
|
-
- 不要把 `supportsPromptCacheKey` 加入 `models.json`:Pi 0.
|
|
179
|
+
- 不要把 `supportsPromptCacheKey` 加入 `models.json`:Pi 0.99.2 没有定义这个 compat 字段。请使用 `/cache-optimizer fix prompt-cache-key` 在扩展自有配置中保存精确 provider/model 的 omit 规则;它会移除两种 key 写法,包括 Pi 提供的 key。
|
|
178
180
|
- 如果出现 `400 Unsupported parameter: prompt_cache_retention`,请为该渠道移除 / 避免 `supportsLongCacheRetention`;如支持,可保留 `sendSessionAffinityHeaders`。扩展会从响应头或最终 assistant error message 中识别这条明确错误,并在当前进程的后续请求中移除该参数。
|
|
179
181
|
- 使用 `/cache-optimizer compat` 或 `/cache-optimizer doctor` 查看当前模型的具体建议。
|
|
180
182
|
- DeepSeek 模型名只用于选择 `DS cache` adapter,不能证明 reasoning wire protocol。缺少或使用非 DeepSeek format 时仍保留通用缓存 / 路由建议;只有 effective `compat.thinkingFormat: "deepseek"` 被明确配置时,才显示 DeepSeek replay 建议,且不会把 `thinkingFormat` 列为缺失修复项。
|
|
@@ -262,7 +264,7 @@ Pi 0.80.9+ 已在内置 Kimi Coding、Moonshot AI / 中国区、OpenRouter 和 V
|
|
|
262
264
|
- DeepSeek Pi Mono replay compat(仅当已明确配置 `thinkingFormat: "deepseek"` 时使用 `requiresReasoningContentOnAssistantMessages: true`;`/fix` 不会自行推断该 format)
|
|
263
265
|
- OpenAI-compatible proxy session affinity(`openai-completions` 使用 `sendSessionAffinityHeaders: true`)。Pi 0.80.7+ 使用 `sessionAffinityFormat` 控制 `openai-responses` header 形式并自动检测默认值;本扩展不再写入已移除的 `sendSessionIdHeader`。
|
|
264
266
|
|
|
265
|
-
**范围:**
|
|
267
|
+
**范围:** 从当前 active model 发起修复。仅缺少 affinity 时,如 provider 已有配置项且无更高优先级配置遮挡,可一次写入 provider 级 compat,覆盖其适用模型(包括不在 `models[]` 中的模型);模型级显式 `false` 仍为 opt-out。其他或有歧义的修复仍按模型处理。
|
|
266
268
|
|
|
267
269
|
**安全机制:**
|
|
268
270
|
|
|
@@ -274,15 +276,15 @@ Pi 0.80.9+ 已在内置 Kimi Coding、Moonshot AI / 中国区、OpenRouter 和 V
|
|
|
274
276
|
6. 完全保留 `models.json` 原有访问权限,不主动收紧或放宽(例如 `0600` 保持 `0600`,`0644` 保持 `0644`)
|
|
275
277
|
7. 备份名唯一且不会覆盖已有备份;如果 JSONC 扫描器无法置信定位目标,则回退到手动修改指引
|
|
276
278
|
|
|
277
|
-
已有的 `modelOverrides[modelId]` 具有 Pi 的最高优先级,因此 `fix` 会直接修复该 entry。对于没有自定义 `models[]` entry 的内置模型或 API-login
|
|
279
|
+
已有的 `modelOverrides[modelId]` 具有 Pi 的最高优先级,因此 `fix` 会直接修复该 entry。对于没有自定义 `models[]` entry 的内置模型或 API-login 模型,若现有 provider 可安全使用 provider 级 affinity 修复,则优先写入 provider compat;否则 `fix` 会创建仅含 compat 的 `modelOverrides` entry,而不会凭空添加自定义模型定义。运行时观察到的 provider 失败也始终写入这一最高优先级 model override,避免 extension-provided runtime compat 遮挡修复。自检会验证完整的 provider → custom model → runtime model → modelOverride 结果;无效的低层写入会被拒绝。
|
|
278
280
|
|
|
279
281
|
**非交互模式:** 拒绝写入,显示手动编辑指引。
|
|
280
282
|
|
|
281
|
-
**运行:** 当 active model 检测到 compat 问题时执行 `/cache-optimizer fix
|
|
283
|
+
**运行:** 当 active model 检测到 compat 问题时执行 `/cache-optimizer fix`。仅缺少 affinity 的提醒按 provider 在当前扩展实例内去重,并指向这一确认式命令;模型专属提醒仍单独显示。compat 已完整时,命令显示"无需修复"。
|
|
282
284
|
|
|
283
285
|
## `/cache-optimizer rollback`
|
|
284
286
|
|
|
285
|
-
Rollback 可通过补全、直接命令和交互菜单使用,并始终需要 UI 确认。命令会选择匹配当前 provider/model
|
|
287
|
+
Rollback 可通过补全、直接命令和交互菜单使用,并始终需要 UI 确认。命令会选择匹配当前 provider/model(provider 级 receipt 可从同 provider 任一模型操作)的最新未回滚 receipt,验证备份与文件 hash,创建新的保留访问权限的 rollback backup,并使用临时文件 + 原子替换。Fix 与 rollback 在多个 extension instance 间串行执行;rollback 还会把预览时的 receipt transaction id/hash 绑定到提交阶段,若另一事务替换 receipt 就安全拒绝。没有交互式 UI 时只提供手动恢复指引,不会写入文件。
|
|
286
288
|
|
|
287
289
|
如果 `models.json` 自 fix 后没有变化,rollback 可以恢复完整的 pre-fix JSONC;如果文件发生变化,则绝不会盲目替换整个文件,只会在 receipt-owned scalar key 仍等于记录的 post-fix 值时撤销该 key,并保留之后的用户修改。如果 receipt-owned key 已变化、目标被删除/移动,或 fix 新建了目标项,命令会安全拒绝并指向记录的 backup 供手动检查。成功后 receipt 会标记为已回滚,并需要 `/reload` 或重启。
|
|
288
290
|
|
|
@@ -357,6 +359,16 @@ Pi 0.79+ 已内置 footer `CH` 标记,用于显示最近一次 prompt cache hi
|
|
|
357
359
|
|
|
358
360
|
Adapter 选择只看模型 id/name(以及 message_end 时 assistant message 的 model/name)。仅使用 OpenAI-shaped API 不会被当作 OpenAI-family,除非模型 id/name 匹配受支持的家族。
|
|
359
361
|
|
|
362
|
+
## 原生虚拟模型(Pi 0.99+)
|
|
363
|
+
|
|
364
|
+
Pi 0.99 允许扩展通过 `pi.registerVirtualModel()` 注册虚拟模型。选中虚拟模型时,`ctx.model` 保持为虚拟模型(`api: "pi-virtual"`),而 Pi 会把每次请求路由到某个物理模型。本扩展会自动跟随该物理模型,router 作者无需接入下文的协议。
|
|
365
|
+
|
|
366
|
+
- 请求 hook 从 provider payload 中读取实际派发的模型 id,并在已配置凭证的物理模型中匹配。`prompt_cache_key` fallback、`prompt_cache_retention` 安全规则、Anthropic TTL 修复以及按模型关闭 `prompt_cache_key` 的规则都按该物理模型生效。如果多个已配置凭证的 provider 共用同一个 id 且处理方式不同,扩展不会猜测,而是跳过依赖模型身份的请求修改。
|
|
367
|
+
- Footer 统计以及 `/cache-optimizer doctor`、`compat`、`stats`、`reset`、`fix` 使用当前会话分支上最近一次应答的物理模型,与 Pi 自身显示的路由模型和 context 上限一致。doctor 与 compat 会同时标出虚拟选择和该物理模型。
|
|
368
|
+
- 在 Pi 路由第一个请求之前,footer 保持为空,诊断会提示先发送一个 prompt。
|
|
369
|
+
- 虚拟选择不做 prompt 改写:Pi 在构建 system prompt 之后才决定物理模型,重排后的 prompt 不应被送到有安全过滤的 Codex 路由。
|
|
370
|
+
- Session-affinity header 桥接同样跳过,因为 Pi 在 payload 生成之前就构造请求 header。Pi 仍会发送物理模型自身配置的 affinity header。
|
|
371
|
+
|
|
360
372
|
## Router / Virtual-channel 扩展作者指南
|
|
361
373
|
|
|
362
374
|
如果你的 Pi 扩展提供虚拟 routing provider(例如 `router/auto`、`router/smart`,或会转发到真实上游的 profile/channel),本扩展可以为真实上游 provider/model 显示缓存统计,而不是把统计记到虚拟外壳上。集成是可选、版本化的,并且**不需要导入本包**。
|
package/index.ts
CHANGED
|
@@ -212,6 +212,8 @@ type PromptCacheKeyConfigReceiptSnapshot = {
|
|
|
212
212
|
};
|
|
213
213
|
const PI_ROUTING_REGISTRY_SYMBOL = Symbol.for("pi.routing.registry.v1");
|
|
214
214
|
const PI_CACHE_HINTS_SYMBOL = Symbol.for("pi.cache.hints.v1");
|
|
215
|
+
// Model API that Pi 0.99+ assigns to native virtual models (VIRTUAL_MODEL_API).
|
|
216
|
+
const PI_VIRTUAL_MODEL_API = "pi-virtual";
|
|
215
217
|
const PI_CACHE_HINTS_OWNER_SYMBOL = Symbol.for("pi.cache.optimizer.hints-owner.v1");
|
|
216
218
|
const ANTHROPIC_TTL_FALLBACK_SYMBOL = Symbol.for("pi.cache.optimizer.anthropic-ttl-fallback.v1");
|
|
217
219
|
const REASONING_PROTOCOL_FALLBACK_SYMBOL = Symbol.for("pi.cache.optimizer.reasoning-protocol-fallback.v1");
|
|
@@ -1397,6 +1399,9 @@ function resolveRouteModel(
|
|
|
1397
1399
|
model: PiModel | undefined,
|
|
1398
1400
|
ctx?: ContextWithOptionalModelRegistry,
|
|
1399
1401
|
): PiModel | undefined {
|
|
1402
|
+
const nativeVirtualRoute = resolveNativeVirtualRouteModel(model, ctx);
|
|
1403
|
+
if (nativeVirtualRoute) return nativeVirtualRoute;
|
|
1404
|
+
|
|
1400
1405
|
const snapshot = resolveActiveRouteSnapshot(model, ctx);
|
|
1401
1406
|
if (!snapshot) return undefined;
|
|
1402
1407
|
|
|
@@ -1407,7 +1412,171 @@ function resolveRouteModel(
|
|
|
1407
1412
|
|
|
1408
1413
|
function isVirtualRoutingModel(model: PiModel | undefined, ctx?: Pick<ExtensionContext, "sessionManager">): boolean {
|
|
1409
1414
|
if (!model) return false;
|
|
1410
|
-
return isRouterModel(model) || !!getRoutingRegistry()?.getRouter(model.provider) || !!resolveActiveRouteSnapshot(model, ctx);
|
|
1415
|
+
return isNativeVirtualModel(model) || isRouterModel(model) || !!getRoutingRegistry()?.getRouter(model.provider) || !!resolveActiveRouteSnapshot(model, ctx);
|
|
1416
|
+
}
|
|
1417
|
+
|
|
1418
|
+
// Pi 0.99+ native virtual models (`pi.registerVirtualModel()`). `ctx.model`
|
|
1419
|
+
// stays the virtual selection while Pi dispatches every request to a physical
|
|
1420
|
+
// model; only assistant messages and the provider payload name that model.
|
|
1421
|
+
// Older Pi hosts never produce this API, so every native-virtual path is inert.
|
|
1422
|
+
type NativeVirtualDispatch = { provider: string; id: string; api: string };
|
|
1423
|
+
|
|
1424
|
+
function isNativeVirtualModel(model: { api?: unknown } | undefined): boolean {
|
|
1425
|
+
return model?.api === PI_VIRTUAL_MODEL_API;
|
|
1426
|
+
}
|
|
1427
|
+
|
|
1428
|
+
function readSessionBranch(ctx: Pick<ExtensionContext, "sessionManager"> | undefined): unknown[] {
|
|
1429
|
+
try {
|
|
1430
|
+
const manager = ctx?.sessionManager as { getBranch?: () => unknown } | undefined;
|
|
1431
|
+
if (typeof manager?.getBranch !== "function") return [];
|
|
1432
|
+
const branch = manager.getBranch();
|
|
1433
|
+
return Array.isArray(branch) ? branch : [];
|
|
1434
|
+
} catch {
|
|
1435
|
+
return [];
|
|
1436
|
+
}
|
|
1437
|
+
}
|
|
1438
|
+
|
|
1439
|
+
/**
|
|
1440
|
+
* Physical models that answered on the current session branch, newest first:
|
|
1441
|
+
* the latest successful response (Pi's `request.previous`) and the latest
|
|
1442
|
+
* response of any outcome (a failed attempt that a retry may reuse). The
|
|
1443
|
+
* catalog id in `message.model` is preferred over the echoed `responseModel`
|
|
1444
|
+
* because it is the id Pi dispatched and the registry knows.
|
|
1445
|
+
*/
|
|
1446
|
+
function findNativeVirtualDispatches(ctx: Pick<ExtensionContext, "sessionManager"> | undefined): {
|
|
1447
|
+
latestSuccessful?: NativeVirtualDispatch;
|
|
1448
|
+
latestAny?: NativeVirtualDispatch;
|
|
1449
|
+
} {
|
|
1450
|
+
const branch = readSessionBranch(ctx);
|
|
1451
|
+
let latestSuccessful: NativeVirtualDispatch | undefined;
|
|
1452
|
+
let latestAny: NativeVirtualDispatch | undefined;
|
|
1453
|
+
for (let index = branch.length - 1; index >= 0 && !latestSuccessful; index--) {
|
|
1454
|
+
const entry = asRecord(branch[index]);
|
|
1455
|
+
if (entry?.type !== "message") continue;
|
|
1456
|
+
const message = getAssistantRecord(entry.message);
|
|
1457
|
+
if (!message || isNativeVirtualModel(message)) continue;
|
|
1458
|
+
const dispatch = nativeVirtualDispatchFromMessage(message);
|
|
1459
|
+
if (!dispatch) continue;
|
|
1460
|
+
latestAny ??= dispatch;
|
|
1461
|
+
if (message.stopReason !== "error" && message.stopReason !== "aborted") latestSuccessful = dispatch;
|
|
1462
|
+
}
|
|
1463
|
+
return { latestSuccessful, latestAny };
|
|
1464
|
+
}
|
|
1465
|
+
|
|
1466
|
+
function nativeVirtualDispatchFromMessage(message: unknown): NativeVirtualDispatch | undefined {
|
|
1467
|
+
const record = getAssistantRecord(message);
|
|
1468
|
+
if (!record || isNativeVirtualModel(record)) return undefined;
|
|
1469
|
+
const provider = firstNonEmptyString(record.provider);
|
|
1470
|
+
const id = firstNonEmptyString(record.model, record.responseModel);
|
|
1471
|
+
return provider && id ? { provider, id, api: firstNonEmptyString(record.api) ?? "" } : undefined;
|
|
1472
|
+
}
|
|
1473
|
+
|
|
1474
|
+
function nativeVirtualDispatchToModel(
|
|
1475
|
+
dispatch: NativeVirtualDispatch,
|
|
1476
|
+
ctx?: ContextWithOptionalModelRegistry,
|
|
1477
|
+
): PiModel {
|
|
1478
|
+
// A virtual model hides a same-id physical model in the catalog, so a
|
|
1479
|
+
// registry hit that is itself virtual is not physical metadata.
|
|
1480
|
+
const registered = findModelInRegistry(ctx?.modelRegistry, dispatch.provider, dispatch.id);
|
|
1481
|
+
const model = registered && !isNativeVirtualModel(registered)
|
|
1482
|
+
? registered
|
|
1483
|
+
: ({
|
|
1484
|
+
provider: dispatch.provider,
|
|
1485
|
+
id: dispatch.id,
|
|
1486
|
+
name: dispatch.id,
|
|
1487
|
+
api: dispatch.api,
|
|
1488
|
+
baseUrl: "",
|
|
1489
|
+
reasoning: false,
|
|
1490
|
+
input: ["text"],
|
|
1491
|
+
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
|
1492
|
+
contextWindow: 0,
|
|
1493
|
+
maxTokens: 0,
|
|
1494
|
+
[ROUTED_FALLBACK_MODEL_SYMBOL]: true,
|
|
1495
|
+
} as PiModel);
|
|
1496
|
+
return applyConfiguredTransportToModel(model, readEffectiveCompatConfig());
|
|
1497
|
+
}
|
|
1498
|
+
|
|
1499
|
+
/**
|
|
1500
|
+
* Physical model behind a native virtual selection for pre-request UX (footer,
|
|
1501
|
+
* doctor, compat, stats, reset, fix): the model that answered last on this
|
|
1502
|
+
* session branch, matching Pi's own routed-model display and context limits.
|
|
1503
|
+
*/
|
|
1504
|
+
function resolveNativeVirtualRouteModel(
|
|
1505
|
+
model: PiModel | undefined,
|
|
1506
|
+
ctx?: ContextWithOptionalModelRegistry,
|
|
1507
|
+
): PiModel | undefined {
|
|
1508
|
+
if (!isNativeVirtualModel(model)) return undefined;
|
|
1509
|
+
const { latestSuccessful, latestAny } = findNativeVirtualDispatches(ctx);
|
|
1510
|
+
const dispatch = latestSuccessful ?? latestAny;
|
|
1511
|
+
return dispatch ? nativeVirtualDispatchToModel(dispatch, ctx) : undefined;
|
|
1512
|
+
}
|
|
1513
|
+
|
|
1514
|
+
function getProviderPayloadModelId(payload: unknown): string | undefined {
|
|
1515
|
+
const record = asRecord(payload);
|
|
1516
|
+
// Pi's built-in transports put the dispatched catalog id in `model`
|
|
1517
|
+
// (OpenAI, Anthropic, Google, Mistral) or `modelId` (Bedrock Converse).
|
|
1518
|
+
return firstNonEmptyString(record?.model, record?.modelId);
|
|
1519
|
+
}
|
|
1520
|
+
|
|
1521
|
+
function physicalModelsWithId(candidates: readonly PiModel[] | undefined, id: string): PiModel[] {
|
|
1522
|
+
return (candidates ?? []).filter((candidate) => candidate.id === id && !isNativeVirtualModel(candidate));
|
|
1523
|
+
}
|
|
1524
|
+
|
|
1525
|
+
/**
|
|
1526
|
+
* Physical model of one native-virtual request. `before_provider_request`
|
|
1527
|
+
* carries only the payload and `ctx.model` stays virtual, so the payload's
|
|
1528
|
+
* dispatched model id is matched against credentialed physical models (a
|
|
1529
|
+
* router may only route to those). A unique provider match wins. An id shared
|
|
1530
|
+
* by several credentialed providers is used only when every candidate gets the
|
|
1531
|
+
* same request treatment (`requestPolicyKey`); the identity then stays
|
|
1532
|
+
* ambiguous so response evidence is never pinned to a guessed provider.
|
|
1533
|
+
* Differing policies return undefined so identity-dependent request mutations
|
|
1534
|
+
* fail closed. Without a credentialed match, the sticky branch candidates and
|
|
1535
|
+
* then the full catalog are consulted.
|
|
1536
|
+
*/
|
|
1537
|
+
function resolveNativeVirtualRequestModel(
|
|
1538
|
+
model: PiModel | undefined,
|
|
1539
|
+
payload: unknown,
|
|
1540
|
+
ctx?: ContextWithOptionalModelRegistry,
|
|
1541
|
+
requestPolicyKey?: (candidate: PiModel) => string,
|
|
1542
|
+
): { model: PiModel; identityAmbiguous: boolean } | undefined {
|
|
1543
|
+
if (!isNativeVirtualModel(model)) return undefined;
|
|
1544
|
+
const payloadModelId = getProviderPayloadModelId(payload);
|
|
1545
|
+
if (!payloadModelId) return undefined;
|
|
1546
|
+
const config = readEffectiveCompatConfig();
|
|
1547
|
+
// null: no candidate; undefined: ambiguous with differing request policies.
|
|
1548
|
+
const pick = (matches: PiModel[]): { model: PiModel; identityAmbiguous: boolean } | undefined | null => {
|
|
1549
|
+
if (matches.length === 0) return null;
|
|
1550
|
+
const configured = matches.map((candidate) => applyConfiguredTransportToModel(candidate, config));
|
|
1551
|
+
if (new Set(configured.map((candidate) => candidate.provider)).size === 1) {
|
|
1552
|
+
return { model: configured[0], identityAmbiguous: false };
|
|
1553
|
+
}
|
|
1554
|
+
const policies = requestPolicyKey ? new Set(configured.map(requestPolicyKey)) : undefined;
|
|
1555
|
+
return policies?.size === 1 ? { model: configured[0], identityAmbiguous: true } : undefined;
|
|
1556
|
+
};
|
|
1557
|
+
try {
|
|
1558
|
+
const credentialed = pick(physicalModelsWithId(ctx?.modelRegistry?.getAvailable?.(), payloadModelId));
|
|
1559
|
+
if (credentialed !== null) return credentialed;
|
|
1560
|
+
|
|
1561
|
+
const { latestSuccessful, latestAny } = findNativeVirtualDispatches(ctx);
|
|
1562
|
+
for (const dispatch of [latestSuccessful, latestAny]) {
|
|
1563
|
+
if (dispatch?.id === payloadModelId) {
|
|
1564
|
+
return { model: nativeVirtualDispatchToModel(dispatch, ctx), identityAmbiguous: false };
|
|
1565
|
+
}
|
|
1566
|
+
}
|
|
1567
|
+
|
|
1568
|
+
return pick(physicalModelsWithId(ctx?.modelRegistry?.getAll?.(), payloadModelId)) ?? undefined;
|
|
1569
|
+
} catch {
|
|
1570
|
+
// Registry access is optional input; never fail the provider request hook.
|
|
1571
|
+
return undefined;
|
|
1572
|
+
}
|
|
1573
|
+
}
|
|
1574
|
+
|
|
1575
|
+
function describeNativeVirtualRouteNote(selected: PiModel | undefined, effective: PiModel | undefined): string | undefined {
|
|
1576
|
+
if (!selected || !isNativeVirtualModel(selected)) return undefined;
|
|
1577
|
+
// Without a routed response, the not-applicable text explains the state.
|
|
1578
|
+
if (!effective || isNativeVirtualModel(effective)) return undefined;
|
|
1579
|
+
return `🔀 Native virtual model ${modelKey(selected)} → latest routed physical model ${modelKey(effective)}. The diagnostics below apply to that physical model; Pi may route later requests elsewhere.`;
|
|
1411
1580
|
}
|
|
1412
1581
|
|
|
1413
1582
|
function isCacheHintsServiceV1(value: unknown): value is PiCacheHintsV1 {
|
|
@@ -2589,11 +2758,13 @@ function isPiBuiltInLlamaCppModel(model: PiModel | undefined): boolean {
|
|
|
2589
2758
|
// Pi's built-in llama.cpp provider supplies this exact explicit compat
|
|
2590
2759
|
// fingerprint. Provider ids are extension-overridable and models.json can add
|
|
2591
2760
|
// cache/routing overrides, so provider id alone must never imply exemption.
|
|
2761
|
+
// supportsUsageInStreaming is deliberately not part of the fingerprint: Pi
|
|
2762
|
+
// 0.82.x sets it false and Pi 0.83+ sets it true after fixing streamed usage,
|
|
2763
|
+
// and it carries no routing or cache configuration.
|
|
2592
2764
|
const compat = getCompat(model);
|
|
2593
2765
|
return compat.supportsStore === false
|
|
2594
2766
|
&& compat.supportsDeveloperRole === false
|
|
2595
2767
|
&& compat.supportsReasoningEffort === false
|
|
2596
|
-
&& compat.supportsUsageInStreaming === false
|
|
2597
2768
|
&& compat.supportsStrictMode === false
|
|
2598
2769
|
&& compat.maxTokensField === "max_tokens"
|
|
2599
2770
|
&& compat.sendSessionAffinityHeaders === undefined
|
|
@@ -2602,7 +2773,7 @@ function isPiBuiltInLlamaCppModel(model: PiModel | undefined): boolean {
|
|
|
2602
2773
|
}
|
|
2603
2774
|
|
|
2604
2775
|
function shouldInjectOpenAIPromptCacheKeyForModel(model: PiModel | undefined): boolean {
|
|
2605
|
-
// Pi 0.
|
|
2776
|
+
// Pi 0.99.2 has no native supportsPromptCacheKey compat field. Per-model
|
|
2606
2777
|
// opt-out is owned by this extension's promptCacheKey.omit configuration;
|
|
2607
2778
|
// this helper only exposes the transport API gate for fixture consumers.
|
|
2608
2779
|
return isOpenAICompatibleApi(model?.api);
|
|
@@ -3445,6 +3616,22 @@ function snapshotProviderRequestModel(model: PiModel | undefined): PiModel | und
|
|
|
3445
3616
|
} as PiModel;
|
|
3446
3617
|
}
|
|
3447
3618
|
|
|
3619
|
+
// Some provider requests never produce a finalized assistant message: Pi's
|
|
3620
|
+
// prompt-cache warming (0.86+) replays a request with maxTokens: 1 outside the
|
|
3621
|
+
// agent loop, so message_end never consumes its lifecycle record. Cap the
|
|
3622
|
+
// process-local list and drop the oldest completed record first.
|
|
3623
|
+
const MAX_PROVIDER_REQUEST_STATES = 32;
|
|
3624
|
+
|
|
3625
|
+
function pruneProviderRequestStates<T extends { responseReceived: boolean }>(
|
|
3626
|
+
states: T[],
|
|
3627
|
+
max = MAX_PROVIDER_REQUEST_STATES,
|
|
3628
|
+
): void {
|
|
3629
|
+
while (states.length > max) {
|
|
3630
|
+
const completed = states.findIndex((state) => state.responseReceived);
|
|
3631
|
+
states.splice(completed >= 0 ? completed : 0, 1);
|
|
3632
|
+
}
|
|
3633
|
+
}
|
|
3634
|
+
|
|
3448
3635
|
function keyForModelExt(model: { provider: string; id: string }): string {
|
|
3449
3636
|
return `${model.provider}/${model.id}`;
|
|
3450
3637
|
}
|
|
@@ -4200,7 +4387,7 @@ function buildOpenAIProxyCompatWarningText(key: string, missing: string[]): stri
|
|
|
4200
4387
|
const modelsJsonPath = getModelsJsonDisplayPath();
|
|
4201
4388
|
const lines: string[] = [
|
|
4202
4389
|
`💡 pi-cache-optimizer: ${key} is a third-party OpenAI-compatible proxy but merged compat lacks ${missing.join(" and ")}.`,
|
|
4203
|
-
`
|
|
4390
|
+
`Run /cache-optimizer fix to preview a confirmed repair (or edit ${modelsJsonPath} -> providers["${providerLabel}"] -> compat manually).`,
|
|
4204
4391
|
``,
|
|
4205
4392
|
];
|
|
4206
4393
|
|
|
@@ -5271,6 +5458,9 @@ const CACHE_PROVIDER_ADAPTERS: CacheProviderAdapter[] = [
|
|
|
5271
5458
|
];
|
|
5272
5459
|
|
|
5273
5460
|
function selectAdapterForModel(model: PiModel | undefined): CacheProviderAdapter | undefined {
|
|
5461
|
+
// A native virtual selection has no cache identity of its own; stats always
|
|
5462
|
+
// belong to the physical model it routed to.
|
|
5463
|
+
if (isNativeVirtualModel(model)) return undefined;
|
|
5274
5464
|
return CACHE_PROVIDER_ADAPTERS.find((adapter) => adapter.matchesModel(model));
|
|
5275
5465
|
}
|
|
5276
5466
|
|
|
@@ -5312,7 +5502,12 @@ function notifyCacheCompatIfNeeded(
|
|
|
5312
5502
|
const text = adapter?.warningText?.(model);
|
|
5313
5503
|
if (!adapter || !text) return;
|
|
5314
5504
|
|
|
5315
|
-
const
|
|
5505
|
+
const affinityOnly = adapter.warningText !== undefined &&
|
|
5506
|
+
describeMissingOpenAICompatibleProxyCompat(model).length === 1 &&
|
|
5507
|
+
describeMissingCacheCompatForModel(model).length === 1;
|
|
5508
|
+
const key = affinityOnly
|
|
5509
|
+
? `proxy-affinity:${model.provider}`
|
|
5510
|
+
: `${adapter.id}:${modelKey(model)}`;
|
|
5316
5511
|
if (warnedModels.has(key)) return;
|
|
5317
5512
|
warnedModels.add(key);
|
|
5318
5513
|
|
|
@@ -6637,6 +6832,13 @@ function describeRouterChannelDiagnostics(model: PiModel): string[] {
|
|
|
6637
6832
|
function getCompatCheckNotApplicableLines(model: PiModel): string[] {
|
|
6638
6833
|
const api = lower(model.api);
|
|
6639
6834
|
|
|
6835
|
+
if (isNativeVirtualModel(model)) {
|
|
6836
|
+
return [
|
|
6837
|
+
"ℹ️ Compat check not applicable for this model.",
|
|
6838
|
+
" Native virtual model: Pi routes each request to a physical model and none has answered on this session branch yet. Send a prompt, then rerun to diagnose the physical model that answered.",
|
|
6839
|
+
];
|
|
6840
|
+
}
|
|
6841
|
+
|
|
6640
6842
|
if (isMistralConversationsApi(api)) {
|
|
6641
6843
|
return [
|
|
6642
6844
|
"ℹ️ Compat check not applicable for this model.",
|
|
@@ -7425,6 +7627,50 @@ interface ModelOverrideNodeLocation {
|
|
|
7425
7627
|
modelOverrideCompatEnd: number;
|
|
7426
7628
|
}
|
|
7427
7629
|
|
|
7630
|
+
interface ProviderCompatNodeLocation {
|
|
7631
|
+
providerObjectBrace: number;
|
|
7632
|
+
providerObjectEnd: number;
|
|
7633
|
+
providerCompatBrace: number;
|
|
7634
|
+
providerCompatEnd: number;
|
|
7635
|
+
}
|
|
7636
|
+
|
|
7637
|
+
/** Locate a unique provider and its direct compat object without requiring models[]. */
|
|
7638
|
+
function locateProviderCompatInJsonc(
|
|
7639
|
+
text: string,
|
|
7640
|
+
providerLabel: string,
|
|
7641
|
+
): ProviderCompatNodeLocation | undefined {
|
|
7642
|
+
const clean = stripJsoncComments(text);
|
|
7643
|
+
const rootBrace = skipJsonWhitespace(clean, 0);
|
|
7644
|
+
if (clean[rootBrace] !== "{") return undefined;
|
|
7645
|
+
const providersKey = findJsonObjectKey(clean, rootBrace, "providers");
|
|
7646
|
+
if (!providersKey || providersKey.count !== 1) return undefined;
|
|
7647
|
+
const providersBrace = skipJsonWhitespace(clean, providersKey.valueStart);
|
|
7648
|
+
if (clean[providersBrace] !== "{") return undefined;
|
|
7649
|
+
const providersEnd = findMatchingBracket(clean, providersBrace);
|
|
7650
|
+
if (providersEnd === undefined) return undefined;
|
|
7651
|
+
const providerKey = findJsonObjectKey(clean, providersBrace, providerLabel);
|
|
7652
|
+
if (!providerKey || providerKey.count !== 1 || providerKey.keyStart > providersEnd) return undefined;
|
|
7653
|
+
const providerObjectBrace = skipJsonWhitespace(clean, providerKey.valueStart);
|
|
7654
|
+
if (clean[providerObjectBrace] !== "{") return undefined;
|
|
7655
|
+
const providerObjectEnd = findMatchingBracket(clean, providerObjectBrace);
|
|
7656
|
+
if (providerObjectEnd === undefined || providerObjectEnd > providersEnd) return undefined;
|
|
7657
|
+
|
|
7658
|
+
for (const key of ["models", "modelOverrides"]) {
|
|
7659
|
+
const child = findJsonObjectKey(clean, providerObjectBrace, key);
|
|
7660
|
+
if (child?.count && child.count > 1) return undefined;
|
|
7661
|
+
}
|
|
7662
|
+
const compatKey = findJsonObjectKey(clean, providerObjectBrace, "compat");
|
|
7663
|
+
if (compatKey?.count && compatKey.count > 1) return undefined;
|
|
7664
|
+
if (!compatKey) {
|
|
7665
|
+
return { providerObjectBrace, providerObjectEnd, providerCompatBrace: -1, providerCompatEnd: -1 };
|
|
7666
|
+
}
|
|
7667
|
+
const providerCompatBrace = skipJsonWhitespace(clean, compatKey.valueStart);
|
|
7668
|
+
if (clean[providerCompatBrace] !== "{") return undefined;
|
|
7669
|
+
const providerCompatEnd = findMatchingBracket(clean, providerCompatBrace);
|
|
7670
|
+
if (providerCompatEnd === undefined || providerCompatEnd > providerObjectEnd) return undefined;
|
|
7671
|
+
return { providerObjectBrace, providerObjectEnd, providerCompatBrace, providerCompatEnd };
|
|
7672
|
+
}
|
|
7673
|
+
|
|
7428
7674
|
/** Locate a provider and optional modelOverrides entry without requiring models[]. */
|
|
7429
7675
|
function locateModelOverrideInJsonc(
|
|
7430
7676
|
text: string,
|
|
@@ -8383,9 +8629,44 @@ function chooseFixPlacement(
|
|
|
8383
8629
|
return decision;
|
|
8384
8630
|
}
|
|
8385
8631
|
|
|
8632
|
+
type CompatInsertionLocation = Pick<ModelNodeLocation,
|
|
8633
|
+
| "modelObjectBrace"
|
|
8634
|
+
| "compatObjectBrace"
|
|
8635
|
+
| "compatObjectEnd"
|
|
8636
|
+
| "providerObjectBrace"
|
|
8637
|
+
| "providerCompatBrace"
|
|
8638
|
+
| "providerCompatEnd"
|
|
8639
|
+
| "modelOverrideObjectBrace"
|
|
8640
|
+
| "modelOverrideCompatBrace"
|
|
8641
|
+
| "modelOverrideCompatEnd"
|
|
8642
|
+
>;
|
|
8643
|
+
|
|
8644
|
+
function composeProviderAffinityInsertion(
|
|
8645
|
+
original: string,
|
|
8646
|
+
providerLabel: string,
|
|
8647
|
+
): { modifiedText: string; placementLabel: string } | undefined {
|
|
8648
|
+
const location = locateProviderCompatInJsonc(original, providerLabel);
|
|
8649
|
+
if (!location) return undefined;
|
|
8650
|
+
const node: CompatInsertionLocation = {
|
|
8651
|
+
modelObjectBrace: -1,
|
|
8652
|
+
compatObjectBrace: -1,
|
|
8653
|
+
compatObjectEnd: -1,
|
|
8654
|
+
providerObjectBrace: location.providerObjectBrace,
|
|
8655
|
+
providerCompatBrace: location.providerCompatBrace,
|
|
8656
|
+
providerCompatEnd: location.providerCompatEnd,
|
|
8657
|
+
modelOverrideObjectBrace: -1,
|
|
8658
|
+
modelOverrideCompatBrace: -1,
|
|
8659
|
+
modelOverrideCompatEnd: -1,
|
|
8660
|
+
};
|
|
8661
|
+
return {
|
|
8662
|
+
modifiedText: composeFixInsertion(original, node, { sendSessionAffinityHeaders: true }, "provider"),
|
|
8663
|
+
placementLabel: `providers["${providerLabel}"] -> compat (provider level)`,
|
|
8664
|
+
};
|
|
8665
|
+
}
|
|
8666
|
+
|
|
8386
8667
|
function composeFixInsertion(
|
|
8387
8668
|
original: string,
|
|
8388
|
-
location:
|
|
8669
|
+
location: CompatInsertionLocation,
|
|
8389
8670
|
compatKeys: Record<string, unknown>,
|
|
8390
8671
|
placement: "provider" | "model" | "modelOverride" = "model",
|
|
8391
8672
|
): string {
|
|
@@ -9464,20 +9745,27 @@ function locateReceiptCompatTarget(
|
|
|
9464
9745
|
};
|
|
9465
9746
|
}
|
|
9466
9747
|
|
|
9748
|
+
if (placement === "provider") {
|
|
9749
|
+
const location = locateProviderCompatInJsonc(text, provider);
|
|
9750
|
+
if (!location) return undefined;
|
|
9751
|
+
return {
|
|
9752
|
+
targetExists: true,
|
|
9753
|
+
compatBrace: location.providerCompatBrace,
|
|
9754
|
+
compatEnd: location.providerCompatEnd,
|
|
9755
|
+
};
|
|
9756
|
+
}
|
|
9467
9757
|
const location = locateModelInJsonc(text, provider, modelId);
|
|
9468
9758
|
if (!location || location.providerKeyCount !== 1) return undefined;
|
|
9469
|
-
// A changed-file
|
|
9470
|
-
// was the receipt's target. Refuse that ambiguity instead of touching
|
|
9471
|
-
// later user-added definition. Provider-level
|
|
9472
|
-
//
|
|
9473
|
-
// ambiguous and are rejected for both placements.
|
|
9759
|
+
// A changed-file model-scoped rollback cannot tell which duplicate model
|
|
9760
|
+
// object was the receipt's target. Refuse that ambiguity instead of touching
|
|
9761
|
+
// a later user-added definition. Provider-level receipts returned above do
|
|
9762
|
+
// not depend on model-array identity.
|
|
9474
9763
|
if (location.allModelIds.filter((id) => id === modelId).length !== 1) return undefined;
|
|
9475
|
-
if (
|
|
9476
|
-
if (placement === "model" && location.modelCompatKeyCount > 1) return undefined;
|
|
9764
|
+
if (location.modelCompatKeyCount > 1) return undefined;
|
|
9477
9765
|
return {
|
|
9478
9766
|
targetExists: location.modelObjectBrace >= 0,
|
|
9479
|
-
compatBrace:
|
|
9480
|
-
compatEnd:
|
|
9767
|
+
compatBrace: location.compatObjectBrace,
|
|
9768
|
+
compatEnd: location.compatObjectEnd,
|
|
9481
9769
|
};
|
|
9482
9770
|
}
|
|
9483
9771
|
|
|
@@ -10121,6 +10409,16 @@ export const __internals_for_tests = {
|
|
|
10121
10409
|
applyConfiguredTransportToModel,
|
|
10122
10410
|
resolveRouteModel,
|
|
10123
10411
|
isVirtualRoutingModel,
|
|
10412
|
+
PI_VIRTUAL_MODEL_API,
|
|
10413
|
+
isNativeVirtualModel,
|
|
10414
|
+
findNativeVirtualDispatches,
|
|
10415
|
+
resolveNativeVirtualRouteModel,
|
|
10416
|
+
resolveNativeVirtualRequestModel,
|
|
10417
|
+
getProviderPayloadModelId,
|
|
10418
|
+
getCompatCheckNotApplicableLines,
|
|
10419
|
+
describeNativeVirtualRouteNote,
|
|
10420
|
+
pruneProviderRequestStates,
|
|
10421
|
+
MAX_PROVIDER_REQUEST_STATES,
|
|
10124
10422
|
installCacheHintsService,
|
|
10125
10423
|
getCacheHintsService,
|
|
10126
10424
|
markOptimizerOwnedCacheHintsService,
|
|
@@ -10163,11 +10461,13 @@ export const __internals_for_tests = {
|
|
|
10163
10461
|
hasExplicitLongRetentionOptInFromConfig,
|
|
10164
10462
|
locateModelInJsonc,
|
|
10165
10463
|
locateModelOverrideInJsonc,
|
|
10464
|
+
locateProviderCompatInJsonc,
|
|
10166
10465
|
composeFixInsertion,
|
|
10167
10466
|
selfCheckFix,
|
|
10168
10467
|
analyzeModelsJsonForMissingEntry,
|
|
10169
10468
|
composeMissingEntryInsertion,
|
|
10170
10469
|
composeModelOverrideInsertion,
|
|
10470
|
+
composeProviderAffinityInsertion,
|
|
10171
10471
|
selfCheckMissingEntryInsertion,
|
|
10172
10472
|
decideFixPlacement,
|
|
10173
10473
|
chooseFixPlacement,
|
|
@@ -10254,6 +10554,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
10254
10554
|
model: PiModel;
|
|
10255
10555
|
responseReceived: boolean;
|
|
10256
10556
|
correlationAmbiguous: boolean;
|
|
10557
|
+
// A native virtual request whose dispatched id matched several providers.
|
|
10558
|
+
identityAmbiguous?: boolean;
|
|
10257
10559
|
};
|
|
10258
10560
|
const providerRequestStates: ProviderRequestState[] = [];
|
|
10259
10561
|
let shardCreatedAt = Date.now();
|
|
@@ -10277,7 +10579,12 @@ export default function (pi: ExtensionAPI) {
|
|
|
10277
10579
|
};
|
|
10278
10580
|
}
|
|
10279
10581
|
if (isSessionAffinity403Applicable(model) && sendSessionAffinityHeaders403Models.has(key)) {
|
|
10280
|
-
return {
|
|
10582
|
+
return {
|
|
10583
|
+
providerLabel,
|
|
10584
|
+
modelId: model.id,
|
|
10585
|
+
compatKeys: { sendSessionAffinityHeaders: false },
|
|
10586
|
+
forceModelLevel: true,
|
|
10587
|
+
};
|
|
10281
10588
|
}
|
|
10282
10589
|
return undefined;
|
|
10283
10590
|
}
|
|
@@ -10780,6 +11087,24 @@ export default function (pi: ExtensionAPI) {
|
|
|
10780
11087
|
);
|
|
10781
11088
|
}
|
|
10782
11089
|
|
|
11090
|
+
/**
|
|
11091
|
+
* Request treatment that before_provider_request derives from a model's
|
|
11092
|
+
* identity. A native virtual request whose dispatched id matches several
|
|
11093
|
+
* credentialed providers is resolved only when this key agrees for all of
|
|
11094
|
+
* them, so no provider-specific decision is applied on a guess.
|
|
11095
|
+
*/
|
|
11096
|
+
function requestPolicyKey(model: PiModel): string {
|
|
11097
|
+
const key = modelKey(model);
|
|
11098
|
+
return JSON.stringify([
|
|
11099
|
+
model.api ?? "",
|
|
11100
|
+
isOfficialOpenAIBaseUrl(model),
|
|
11101
|
+
hasExplicitLongRetentionOptIn(model),
|
|
11102
|
+
promptCacheRetention400Models.has(key),
|
|
11103
|
+
anthropicTtlOrderErrorModels.has(key),
|
|
11104
|
+
isPromptCacheKeyOmittedForModel(model),
|
|
11105
|
+
]);
|
|
11106
|
+
}
|
|
11107
|
+
|
|
10783
11108
|
pi.on("session_start", async (event, ctx) => {
|
|
10784
11109
|
if (runtimeOptimizerEnabled) requestLongCacheRetention();
|
|
10785
11110
|
await restoreCacheStats(event.reason, ctx);
|
|
@@ -10794,7 +11119,10 @@ export default function (pi: ExtensionAPI) {
|
|
|
10794
11119
|
await publishStatus(ctx);
|
|
10795
11120
|
});
|
|
10796
11121
|
|
|
10797
|
-
pi.on("tool_execution_end", async (
|
|
11122
|
+
pi.on("tool_execution_end", async (event, ctx) => {
|
|
11123
|
+
// Pi 0.99+ codemode scripts run nested tool calls in parallel; the calling
|
|
11124
|
+
// tool's own end event follows them, so one refresh covers the batch.
|
|
11125
|
+
if (isNonEmptyString((event as { parentToolCallId?: unknown } | undefined)?.parentToolCallId)) return;
|
|
10798
11126
|
await refreshShardAggregate();
|
|
10799
11127
|
await publishStatus(ctx);
|
|
10800
11128
|
});
|
|
@@ -10867,6 +11195,13 @@ export default function (pi: ExtensionAPI) {
|
|
|
10867
11195
|
if (model && isResponsesPromptRewriteBypassApi(model.api)) {
|
|
10868
11196
|
return {};
|
|
10869
11197
|
}
|
|
11198
|
+
// Pi 0.99+ native virtual selections pick the physical model per request,
|
|
11199
|
+
// after this system prompt is built, so the bypass above cannot be decided
|
|
11200
|
+
// here. A route may reach the safety-filtered Codex backend; keep Pi's
|
|
11201
|
+
// prompt byte-for-byte instead of reordering it.
|
|
11202
|
+
if (isNativeVirtualModel(_ctx.model)) {
|
|
11203
|
+
return {};
|
|
11204
|
+
}
|
|
10870
11205
|
|
|
10871
11206
|
if (!runtimeOptimizerEnabled) return {};
|
|
10872
11207
|
|
|
@@ -10945,6 +11280,10 @@ export default function (pi: ExtensionAPI) {
|
|
|
10945
11280
|
});
|
|
10946
11281
|
|
|
10947
11282
|
pi.on("before_provider_headers", (event, ctx) => {
|
|
11283
|
+
// Pi transforms headers before the payload exists and passes no model, so a
|
|
11284
|
+
// native virtual request's physical model is unknown here. Fail closed; Pi
|
|
11285
|
+
// core still sends the physical model's own configured affinity headers.
|
|
11286
|
+
if (isNativeVirtualModel(ctx.model)) return;
|
|
10948
11287
|
const requestModel = resolveRouteModel(ctx.model, ctx) ?? ctx.model;
|
|
10949
11288
|
addEffectiveSessionAffinityHeaders(
|
|
10950
11289
|
event.headers,
|
|
@@ -10954,12 +11293,24 @@ export default function (pi: ExtensionAPI) {
|
|
|
10954
11293
|
});
|
|
10955
11294
|
|
|
10956
11295
|
pi.on("before_provider_request", (event, ctx) => {
|
|
10957
|
-
|
|
11296
|
+
// A native virtual selection keeps ctx.model virtual; resolve the physical
|
|
11297
|
+
// model from the dispatched payload. Unresolved requests keep the virtual
|
|
11298
|
+
// model, which fails every identity-dependent mutation closed.
|
|
11299
|
+
const nativeVirtualRequest = resolveNativeVirtualRequestModel(ctx.model, event.payload, ctx, requestPolicyKey);
|
|
11300
|
+
const requestModel = nativeVirtualRequest?.model ?? resolveRouteModel(ctx.model, ctx) ?? ctx.model;
|
|
10958
11301
|
// Request-local identity is also needed by the always-on Anthropic TTL
|
|
10959
11302
|
// validity repair, so retain the credential-blind snapshot even while the
|
|
10960
11303
|
// optional runtime optimizer features are disabled.
|
|
10961
11304
|
const snapshot = snapshotProviderRequestModel(requestModel);
|
|
10962
|
-
if (snapshot)
|
|
11305
|
+
if (snapshot) {
|
|
11306
|
+
providerRequestStates.push({
|
|
11307
|
+
model: snapshot,
|
|
11308
|
+
responseReceived: false,
|
|
11309
|
+
correlationAmbiguous: false,
|
|
11310
|
+
identityAmbiguous: isNativeVirtualModel(ctx.model) && (!nativeVirtualRequest || nativeVirtualRequest.identityAmbiguous),
|
|
11311
|
+
});
|
|
11312
|
+
pruneProviderRequestStates(providerRequestStates);
|
|
11313
|
+
}
|
|
10963
11314
|
let requestPayload: unknown = event.payload;
|
|
10964
11315
|
let toolOrderChanged = false;
|
|
10965
11316
|
|
|
@@ -11045,7 +11396,12 @@ export default function (pi: ExtensionAPI) {
|
|
|
11045
11396
|
}
|
|
11046
11397
|
}
|
|
11047
11398
|
if (responseState) responseState.responseReceived = true;
|
|
11048
|
-
|
|
11399
|
+
// A native virtual request whose physical provider could not be pinned
|
|
11400
|
+
// records no model-scoped header evidence; finalized assistant messages
|
|
11401
|
+
// still carry exact provider/model identity.
|
|
11402
|
+
const model = responseState
|
|
11403
|
+
? (responseState.identityAmbiguous ? undefined : responseState.model)
|
|
11404
|
+
: (pendingStates.length === 0 && !isNativeVirtualModel(ctx.model) ? (resolveRouteModel(ctx.model, ctx) ?? ctx.model) : undefined);
|
|
11049
11405
|
if (!runtimeOptimizerEnabled || !model) return;
|
|
11050
11406
|
|
|
11051
11407
|
// Keep only the category, never the provider's complete error text. This
|
|
@@ -11154,9 +11510,10 @@ export default function (pi: ExtensionAPI) {
|
|
|
11154
11510
|
: (completedIndex >= 0 ? completedIndex : (contextIndex >= 0 ? contextIndex : 0));
|
|
11155
11511
|
const state = providerRequestStates.splice(index, 1)[0];
|
|
11156
11512
|
if (!state) return { model: undefined, ambiguous: false };
|
|
11513
|
+
const ambiguous = (state.correlationAmbiguous || state.identityAmbiguous === true) && explicitIndex < 0;
|
|
11157
11514
|
return {
|
|
11158
|
-
model:
|
|
11159
|
-
ambiguous
|
|
11515
|
+
model: ambiguous ? undefined : state.model,
|
|
11516
|
+
ambiguous,
|
|
11160
11517
|
};
|
|
11161
11518
|
})()
|
|
11162
11519
|
: { model: undefined, ambiguous: false };
|
|
@@ -11244,7 +11601,18 @@ export default function (pi: ExtensionAPI) {
|
|
|
11244
11601
|
}
|
|
11245
11602
|
}
|
|
11246
11603
|
|
|
11247
|
-
|
|
11604
|
+
// A native virtual selection keeps ctx.model virtual, while the finalized
|
|
11605
|
+
// message names the physical catalog model Pi dispatched. Resolve that
|
|
11606
|
+
// model from the registry so adapter tokens, the stats key, and the footer
|
|
11607
|
+
// compat marker use the same physical identity as the pre-request UX.
|
|
11608
|
+
const nativeVirtualDispatch = isNativeVirtualModel(ctx.model)
|
|
11609
|
+
? nativeVirtualDispatchFromMessage(event.message)
|
|
11610
|
+
: undefined;
|
|
11611
|
+
const nativeVirtualMessageModel = nativeVirtualDispatch
|
|
11612
|
+
? nativeVirtualDispatchToModel(nativeVirtualDispatch, ctx)
|
|
11613
|
+
: undefined;
|
|
11614
|
+
|
|
11615
|
+
const adapter = selectAdapterForAssistantMessage(event.message, nativeVirtualMessageModel ?? ctx.model);
|
|
11248
11616
|
if (!adapter) return;
|
|
11249
11617
|
|
|
11250
11618
|
// Skip stats for error/aborted messages (network retries, user aborts).
|
|
@@ -11274,6 +11642,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
11274
11642
|
// the active model id. Virtual routing providers keep message-local
|
|
11275
11643
|
// identity (router correctness).
|
|
11276
11644
|
statsModel = consolidateDirectProviderStatsModel(statsModel, ctx.model, ctx);
|
|
11645
|
+
if (nativeVirtualMessageModel) statsModel = nativeVirtualMessageModel;
|
|
11277
11646
|
let routedModelChanged = false;
|
|
11278
11647
|
if (isVirtualRoutingModel(ctx.model, ctx) && statsModel && !isVirtualRoutingModel(statsModel, ctx)) {
|
|
11279
11648
|
const nextRoutedModel: PersistedRoutedModelRef = {
|
|
@@ -11388,9 +11757,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
11388
11757
|
const statsState = model ? cacheStatsTotalsByModel[modelKey(model)] : undefined;
|
|
11389
11758
|
const samples = sk ? getRecentSamples(sk) : [];
|
|
11390
11759
|
const lowHitLines = buildLowHitDiagnosis(model, adapter, statsState, samples);
|
|
11391
|
-
const
|
|
11392
|
-
|
|
11393
|
-
: diagnosis;
|
|
11760
|
+
const routeNote = describeNativeVirtualRouteNote(selectedModel, model);
|
|
11761
|
+
const fullDiagnosis = [routeNote, diagnosis, ...lowHitLines].filter((line) => line !== undefined).join("\n");
|
|
11394
11762
|
cmdCtx.ui.notify(fullDiagnosis, "info");
|
|
11395
11763
|
} else if (subcommand === "stats") {
|
|
11396
11764
|
const aggregate = await refreshShardAggregate();
|
|
@@ -11441,13 +11809,15 @@ export default function (pi: ExtensionAPI) {
|
|
|
11441
11809
|
return;
|
|
11442
11810
|
}
|
|
11443
11811
|
const compatResult = buildCompatDiagnosis(model);
|
|
11812
|
+
const compatRouteNote = describeNativeVirtualRouteNote(selectedModel, model);
|
|
11813
|
+
const withRouteNote = (text: string): string => compatRouteNote ? `${compatRouteNote}\n${text}` : text;
|
|
11444
11814
|
if (compatResult) {
|
|
11445
|
-
cmdCtx.ui.notify(compatResult, "warning");
|
|
11815
|
+
cmdCtx.ui.notify(withRouteNote(compatResult), "warning");
|
|
11446
11816
|
} else {
|
|
11447
11817
|
cmdCtx.ui.notify(
|
|
11448
|
-
isAdaptiveThinkingCompatApplicable(model) || isDeepSeekCompatCheckApplicable(model) || isCompatCheckApplicable(model)
|
|
11818
|
+
withRouteNote(isAdaptiveThinkingCompatApplicable(model) || isDeepSeekCompatCheckApplicable(model) || isCompatCheckApplicable(model)
|
|
11449
11819
|
? "✅ Compat fully configured."
|
|
11450
|
-
: getCompatCheckNotApplicableLines(model).join("\n"),
|
|
11820
|
+
: getCompatCheckNotApplicableLines(model).join("\n")),
|
|
11451
11821
|
"info",
|
|
11452
11822
|
);
|
|
11453
11823
|
}
|
|
@@ -11462,7 +11832,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
11462
11832
|
const useConfigReceipt = isActionablePromptCacheKeyConfigReceipt(configReceipt) &&
|
|
11463
11833
|
configReceipt.provider === model.provider &&
|
|
11464
11834
|
configReceipt.modelId === model.id &&
|
|
11465
|
-
(!isActionableModelsJsonFixReceipt(modelsReceipt) || modelsReceipt.provider !== model.provider ||
|
|
11835
|
+
(!isActionableModelsJsonFixReceipt(modelsReceipt) || modelsReceipt.provider !== model.provider ||
|
|
11836
|
+
(modelsReceipt.placement !== "provider" && modelsReceipt.modelId !== model.id) || configReceipt.appliedAt >= modelsReceipt.appliedAt);
|
|
11466
11837
|
if (useConfigReceipt) {
|
|
11467
11838
|
if (!cmdCtx.hasUI) {
|
|
11468
11839
|
cmdCtx.ui.notify("❌ Rollback requires interactive confirmation. No changes were made.\nRun /cache-optimizer rollback in Pi's interactive UI to restore the prompt-cache-key setting.", "warning");
|
|
@@ -11505,7 +11876,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
11505
11876
|
cmdCtx.ui.notify("ℹ️ No unapplied /cache-optimizer fix receipt was found.", "info");
|
|
11506
11877
|
return;
|
|
11507
11878
|
}
|
|
11508
|
-
if (receipt.provider !== model.provider || receipt.modelId !== model.id) {
|
|
11879
|
+
if (receipt.provider !== model.provider || (receipt.placement !== "provider" && receipt.modelId !== model.id)) {
|
|
11509
11880
|
cmdCtx.ui.notify(
|
|
11510
11881
|
`ℹ️ The latest fix receipt is for ${receipt.provider}/${receipt.modelId}, not the active model ${model.provider}/${model.id}. ` +
|
|
11511
11882
|
"Switch to the matching model before running rollback. No changes were made.",
|
|
@@ -11743,6 +12114,79 @@ export default function (pi: ExtensionAPI) {
|
|
|
11743
12114
|
const location = locateModelInJsonc(originalText, suggestion.providerLabel, suggestion.modelId);
|
|
11744
12115
|
if (!location) {
|
|
11745
12116
|
const diagnosis = analyzeModelsJsonForMissingEntry(originalText, suggestion.providerLabel, suggestion.modelId);
|
|
12117
|
+
const parsedOriginal = (() => { try { return parseJsonc(originalText); } catch { return undefined; } })();
|
|
12118
|
+
const provider = asRecord(asRecord(parsedOriginal)?.providers)?.[suggestion.providerLabel];
|
|
12119
|
+
const explicit = resolveExplicitCompatValue(parsedOriginal, suggestion.providerLabel, suggestion.modelId, "sendSessionAffinityHeaders");
|
|
12120
|
+
const targetOverrideLocation = locateModelOverrideInJsonc(
|
|
12121
|
+
originalText, suggestion.providerLabel, suggestion.modelId,
|
|
12122
|
+
);
|
|
12123
|
+
const hasTargetOverride = (targetOverrideLocation?.modelOverrideObjectBrace ?? -1) >= 0;
|
|
12124
|
+
const providerPlan = diagnosis && diagnosis.scenario !== "provider_missing" &&
|
|
12125
|
+
isValidModelsConfigForEffectiveCompat(parsedOriginal) && asRecord(provider) &&
|
|
12126
|
+
Object.keys(suggestion.compatKeys).length === 1 && suggestion.compatKeys.sendSessionAffinityHeaders === true &&
|
|
12127
|
+
!suggestion.forceModelLevel && !hasTargetOverride && explicit === undefined &&
|
|
12128
|
+
getEffectiveCompatValueSource(model, parsedOriginal, "sendSessionAffinityHeaders") === undefined
|
|
12129
|
+
? composeProviderAffinityInsertion(originalText, suggestion.providerLabel)
|
|
12130
|
+
: undefined;
|
|
12131
|
+
if (providerPlan) {
|
|
12132
|
+
const checkProvider = (writtenText: string): string | null => {
|
|
12133
|
+
try {
|
|
12134
|
+
const changed = parseJsonc(writtenText);
|
|
12135
|
+
if (!isValidModelsConfigForEffectiveCompat(changed)) return "provider config is invalid";
|
|
12136
|
+
if (resolveEffectiveCompatFromConfig(model, changed).sendSessionAffinityHeaders !== true) return "affinity flag is not effective";
|
|
12137
|
+
const reverted = JSON.parse(JSON.stringify(changed)) as Record<string, unknown>;
|
|
12138
|
+
const target = asRecord(asRecord(reverted.providers)?.[suggestion.providerLabel]);
|
|
12139
|
+
const compat = asRecord(target?.compat);
|
|
12140
|
+
if (!target || !compat || compat.sendSessionAffinityHeaders !== true) return "provider affinity key is missing";
|
|
12141
|
+
delete compat.sendSessionAffinityHeaders;
|
|
12142
|
+
const originalCompat = asRecord(asRecord(provider)?.compat);
|
|
12143
|
+
if (!originalCompat) delete target.compat;
|
|
12144
|
+
return JSON.stringify(reverted) === JSON.stringify(parsedOriginal) ? null : "unrelated configuration was altered";
|
|
12145
|
+
} catch { return "invalid provider JSONC"; }
|
|
12146
|
+
};
|
|
12147
|
+
const checkError = checkProvider(providerPlan.modifiedText);
|
|
12148
|
+
if (checkError) {
|
|
12149
|
+
cmdCtx.ui.notify(`❌ Provider-level self-check failed: ${checkError}. No changes were made.`, "error");
|
|
12150
|
+
return;
|
|
12151
|
+
}
|
|
12152
|
+
const backupPath = `${MODELS_JSON_PATH}.backup-cache-optimizer-${backupTimestamp()}`;
|
|
12153
|
+
const confirmed = await cmdCtx.ui.confirm("Cache Optimizer — Fix provider affinity", [
|
|
12154
|
+
`📝 Preview of changes to ${getModelsJsonDisplayPath()}:`,
|
|
12155
|
+
`Location: ${providerPlan.placementLabel}`,
|
|
12156
|
+
`Compat JSON to write: ${JSON.stringify(suggestion.compatKeys)}`,
|
|
12157
|
+
`⚠️ This affects all models using this provider across all sessions; model-level overrides remain authoritative.`,
|
|
12158
|
+
`A timestamped backup will be written to: ${backupPath}`,
|
|
12159
|
+
"Run /reload or restart Pi for the change to take effect.",
|
|
12160
|
+
"Apply these changes?",
|
|
12161
|
+
].join("\n"));
|
|
12162
|
+
if (!confirmed) {
|
|
12163
|
+
cmdCtx.ui.notify("No changes were made. Canceled by user.", "info");
|
|
12164
|
+
return;
|
|
12165
|
+
}
|
|
12166
|
+
const receipt = createModelsJsonFixReceipt(
|
|
12167
|
+
originalText, providerPlan.modifiedText, suggestion.providerLabel, suggestion.modelId,
|
|
12168
|
+
"provider", suggestion.compatKeys, true, backupPath,
|
|
12169
|
+
);
|
|
12170
|
+
if (!receipt) {
|
|
12171
|
+
cmdCtx.ui.notify("❌ Could not create a privacy-safe fix receipt. No changes were made.", "error");
|
|
12172
|
+
return;
|
|
12173
|
+
}
|
|
12174
|
+
try {
|
|
12175
|
+
const result = await applyModelsJsonFixTransaction(providerPlan.modifiedText, backupPath, checkProvider, {
|
|
12176
|
+
expectedCurrentHash: hashText(originalText), purpose: "fix",
|
|
12177
|
+
onCommitted: async () => writeModelsJsonFixReceipt(receipt),
|
|
12178
|
+
});
|
|
12179
|
+
if ("postCheckError" in result) {
|
|
12180
|
+
cmdCtx.ui.notify(`❌ Post-write self-check failed: ${result.postCheckError}. Backup restored.`, "error");
|
|
12181
|
+
return;
|
|
12182
|
+
}
|
|
12183
|
+
invalidateModelsConfigCache();
|
|
12184
|
+
cmdCtx.ui.notify(`✅ Fix applied to ${getModelsJsonDisplayPath()}.\nBackup saved to: ${backupPath}\nRun /reload or restart Pi.`, "info");
|
|
12185
|
+
} catch (error) {
|
|
12186
|
+
cmdCtx.ui.notify(`❌ Write failed: ${error instanceof Error ? error.message : String(error)}. Backup may be at: ${backupPath}`, "error");
|
|
12187
|
+
}
|
|
12188
|
+
return;
|
|
12189
|
+
}
|
|
11746
12190
|
if (diagnosis && cmdCtx.hasUI) {
|
|
11747
12191
|
const overrideLocation = locateModelOverrideInJsonc(
|
|
11748
12192
|
originalText, suggestion.providerLabel, suggestion.modelId,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-cache-optimizer",
|
|
3
|
-
"version": "2.8.
|
|
3
|
+
"version": "2.8.12",
|
|
4
4
|
"description": "Improve Pi prompt/KV cache hit rates with stable prompts, OpenAI-compatible cache keys, proxy compat warnings, and footer cache stats.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package",
|
|
@@ -35,8 +35,8 @@
|
|
|
35
35
|
"check": "npm run typecheck && npm test && npm run check:diff && npm run check:pack"
|
|
36
36
|
},
|
|
37
37
|
"devDependencies": {
|
|
38
|
-
"@earendil-works/pi-coding-agent": "^0.
|
|
39
|
-
"@earendil-works/pi-server": "0.
|
|
38
|
+
"@earendil-works/pi-coding-agent": "^0.99.2",
|
|
39
|
+
"@earendil-works/pi-server": "0.99.2",
|
|
40
40
|
"@types/node": "^22.0.0",
|
|
41
41
|
"jiti": "^2.7.0",
|
|
42
42
|
"typescript": "^5.8.3"
|