pi-cache-optimizer 2.0.2 → 2.1.1

package/README.md

@@ -65,7 +65,15 @@ Generic OpenAI-compatible proxies are **not** treated as OpenAI-family just beca
 pi install npm:pi-cache-optimizer
 ```
 
-After installation, `PI_CACHE_RETENTION=long` is applied automatically, the system prompt is reordered automatically, `~/.pi/agent/models.json` is auto-seeded with a DeepSeek block when no DeepSeek-like model is configured, and the footer shows cache stats after supported model-family responses with exposed usage.
+After installation, `PI_CACHE_RETENTION=long` is applied automatically, the system prompt is reordered and skills are compressed automatically, session-overview churn is stripped automatically, `~/.pi/agent/models.json` is auto-seeded with a DeepSeek block when no DeepSeek-like model is configured, and the footer shows cache stats after supported model-family responses with exposed usage.
+
+## Opt-out
+
+| Env var | Effect |
+|---------|--------|
+| `PI_CACHE_OPTIMIZER_NO_AUTO_CONFIG=1` | Skip DeepSeek `models.json` auto-seed |
+| `PI_CACHE_OPTIMIZER_NO_SKILL_COMPRESSION=1` | Keep pi's verbose `<available_skills>` XML (opt out of one-line index) |
+| `PI_CACHE_OPTIMIZER_OPENAI_CACHE_KEY=1` | Add `prompt_cache_key` to OpenAI-family requests (opt-in) |
 
 ## Uninstall
 

package/README.zh-CN.md

@@ -23,6 +23,9 @@
 | 功能 | 方式 | 是否需要手动操作 |
 |------|------|:---:|
 | 🔄 重组 system prompt | `before_agent_start` 钩子 — 稳定前缀在前、动态上下文在后 | ❌ 自动 |
+| 🗜️ 压缩 Skills XML | 将 pi 的每 skill 四行 XML 替换为按 skills-root 分组的紧凑单行索引（大小缩减约 93%） | ❌ 自动 |
+| 🧹 剥离 session-overview 动态尾字段 | 从 `<session-overview>` 中移除 `RECENT COMMITS`、`Working directory`、`Line count`——这些字段每轮都在变，破坏前缀缓存 | ❌ 自动 |
+| 🛡️ 完整性 guard | 检测 prompt 重排是否意外截断了 trellis 结构标记；如发生则回退到原始 prompt 并在 footer 显示 `⚠️ integrity` | ❌ 自动 |
 | ⏳ 长缓存保留 | 扩展加载时设置 `PI_CACHE_RETENTION=long`；Pi/provider compat 决定实际发送内容 | ❌ 自动 |
 | 🔗 保守 compat 提醒 | DeepSeek session-affinity 提醒，以及 Claude 兼容 endpoint 的明显 cache-control 提醒 | ⚠️ 见下 |
 | 📊 Provider-specific 底部统计 | 在 Pi footer/status 中显示受支持 provider family 的只读缓存统计 | ❌ 自动 |
@@ -65,7 +68,15 @@ Generic OpenAI-compatible 代理**不会**仅因为使用 OpenAI 形状 API 或
 pi install npm:pi-cache-optimizer
 ```
 
-安装后 `PI_CACHE_RETENTION=long` **自动生效**，system prompt **自动重组**；如果 `~/.pi/agent/models.json` 还没有 DeepSeek-like 模型，会自动 seed 一个 `deepseek` provider 块；受支持 model family 的响应完成且暴露 usage 后，底部状态栏会显示缓存统计。
+安装后 `PI_CACHE_RETENTION=long` **自动生效**，system prompt **自动重组**、skills 自动压缩、session-overview 动态尾字段自动剥离；如果 `~/.pi/agent/models.json` 还没有 DeepSeek-like 模型，会自动 seed 一个 `deepseek` provider 块；受支持 model family 的响应完成且暴露 usage 后，底部状态栏会显示缓存统计。
+
+## 退出（Opt-out）
+
+| 环境变量 | 作用 |
+|---------|------|
+| `PI_CACHE_OPTIMIZER_NO_AUTO_CONFIG=1` | 跳过 `models.json` DeepSeek 自动写入 |
+| `PI_CACHE_OPTIMIZER_NO_SKILL_COMPRESSION=1` | 保留 pi 的 verbose `<available_skills>` XML（退出一行索引模式） |
+| `PI_CACHE_OPTIMIZER_OPENAI_CACHE_KEY=1` | 对 OpenAI-family 请求添加 `prompt_cache_key`（需主动启用） |
 
 ## 卸载
 
@@ -196,7 +207,8 @@ Pi 本身还会根据模型 compat 和 `PI_CACHE_RETENTION` 决定是否发送
 
 本包现在有 provider-family stats adapter，但仍避免盲目泛化：
 
-- DeepSeek cache 是自动的 prefix/KV cache。命中是 best-effort，代理可能隐藏 DeepSeek usage 字段。
+- DeepSeek cache 是自动的 prefix/KV cache。命中是 best-effort，代理可能隐藏 DeepSeek usage 字段。DeepSeek 的 Anthropic API 兼容层**明确忽略 `cache_control` markers**（对所有 content 类型均忽略）——像 Claude Code 那样用显式缓存断点对 DeepSeek 无效。
+- **Kiro / kiro-api**：`pi-provider-kiro` 扩展使用 AWS CodeWhisperer / Q Developer 流式协议（不是 Anthropic Messages / OpenAI Chat Completions / Bedrock Converse）。该协议没有 `cache_control` marker 的注入位置，也不返回 `cache_read_input_tokens`。对 Kiro Claude 模型，底部会显示 **0%**——这是 `pi-provider-kiro` 的限制，不是本扩展的 bug。不要强行用特殊逻辑 bump 这些数字。
 - OpenAI-family prompt caching 只有在真实上游支持且 prompt 足够长时才会自动生效。adapter 基于模型名称且刻意保守；不会用 provider/API/base URL metadata 推断官方 OpenAI 支持。
 - Claude prompt caching 依赖显式 Anthropic cache-control breakpoints。本版本只报告 Pi/provider 暴露的统计；不会插入 breakpoint，也不会修改请求体。
 - Gemini/Vertex 可能暴露 implicit cached-content token count。本版本不会创建、保存、更新或删除 explicit Gemini cached-content resources。

package/index.ts

@@ -7,7 +7,7 @@ import {
 } from "node:fs";
 import { mkdir, readFile, rename, unlink, writeFile } from "node:fs/promises";
 import { homedir } from "node:os";
-import { join } from "node:path";
+import { dirname, join } from "node:path";
 import type { BuildSystemPromptOptions, ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
 
 /**
@@ -47,8 +47,24 @@ const CACHE_PROVIDER_IDS: CacheProviderId[] = ["deepseek", "openai", "claude", "
 const OPENAI_CACHE_KEY_ENV = "PI_CACHE_OPTIMIZER_OPENAI_CACHE_KEY";
 const OPENAI_PROMPT_CACHE_KEY_PREFIX = "pi-dsco-";
 const NO_AUTO_CONFIG_ENV = "PI_CACHE_OPTIMIZER_NO_AUTO_CONFIG";
+const NO_SKILL_COMPRESSION_ENV = "PI_CACHE_OPTIMIZER_NO_SKILL_COMPRESSION";
 const DEEPSEEK_API_KEY_ENV = "DEEPSEEK_API_KEY";
 
+// WORM-flag: if optimizeSystemPrompt ever detects that its blind-replace
+// logic has accidentally truncated the trellis `<workflow-state>` block
+// (or any structural marker from an upstream extension), we flip this.
+// publishStatus reads it once, appends a footer warning, then resets it.
+// The flag surface is kept separate from the regular cache-stats counter
+// so that a one-turn glitch doesn't poison the persisted metrics.
+let promptTruncationDetected = false;
+
+// Minimum count of skills before compression is worth applying.
+// Below this, pi's verbose XML block is small enough that the overhead of
+// an additional one-line index isn't worth the loss of per-skill
+// description hints. The 31-skill snapshot in this repo was 13.3 KB; one
+// or two skills is well under 1 KB and not worth touching.
+const SKILL_COMPRESSION_MIN_COUNT = 4;
+
 // Minimum trimmed length for a candidate to qualify as a stable-prefix "part".
 //
 // `optimizeSystemPrompt` removes each accepted candidate from the dynamic
@@ -166,6 +182,121 @@ function formatSkillsForPrompt(skills: NonNullable<BuildSystemPromptOptions["ski
   return lines.join("\n");
 }
 
+/**
+ * Compressed alternative to `formatSkillsForPrompt`.
+ *
+ * Pi emits a four-line XML block per skill (`<name>`, `<description>`,
+ * `<location>`) plus a three-sentence preamble. With 31 skills active in
+ * this repo that block measured 13.3 KB — 61.5 % of the total system
+ * prompt. The full description text matters when the model has to decide
+ * which skill to load, but the model can read SKILL.md on demand: the
+ * names alone plus a known location pattern is enough to identify
+ * candidates.
+ *
+ * This compressed form preserves:
+ *   1. The instruction to read SKILL.md when a task matches a skill name.
+ *   2. The relative-path resolution rule (parent of SKILL.md is the
+ *      skill directory).
+ *   3. Discoverability of every skill: name + location prefix per skill.
+ *
+ * It drops:
+ *   - Per-skill description text (model loads it via `read` when a name
+ *     matches a task).
+ *   - The `<available_skills>` XML envelope and per-skill XML overhead
+ *     (~110 bytes per skill of pure structure, plus the location path).
+ *
+ * Output shape is a single text block grouped by skill-root directory so
+ * the model can compute each skill's full path by name. Names are sorted
+ * alphabetically within each group for determinism (cache stability).
+ */
+function formatSkillsForPromptCompressed(
+  skills: NonNullable<BuildSystemPromptOptions["skills"]>,
+): string {
+  const visibleSkills = skills.filter((skill) => !skill.disableModelInvocation);
+  if (visibleSkills.length === 0) return "";
+
+  const groups = new Map<string, string[]>();
+  for (const skill of visibleSkills) {
+    // skill.filePath = .../<skill-name>/SKILL.md, so dirname is the
+    // skill directory and dirname-of-dirname is the skills root.
+    const skillDir = dirname(skill.filePath);
+    const root = dirname(skillDir);
+    const list = groups.get(root) ?? [];
+    list.push(skill.name);
+    groups.set(root, list);
+  }
+
+  // Sort group entries by root for determinism: same skill set under the
+  // same roots must always produce the same string, otherwise the
+  // provider prompt-prefix cache loses on prompt builder runs that
+  // happened to iterate the underlying Map in different orders.
+  const sortedGroups = [...groups.entries()].sort(([a], [b]) =>
+    a < b ? -1 : a > b ? 1 : 0,
+  );
+
+  const lines: string[] = [
+    "",
+    "",
+    "The following skills provide specialized instructions for specific tasks. When a skill name matches the task you are doing, read the SKILL.md at the listed location to load the full instructions. When a SKILL.md references a relative path, resolve it against the skill directory (parent of SKILL.md / dirname of the path) and use that absolute path in tool commands.",
+  ];
+
+  for (const [root, names] of sortedGroups) {
+    names.sort();
+    lines.push("");
+    lines.push(`Skills under ${root}/<name>/SKILL.md:`);
+    // Wrap the name list at ~80 columns for readability without
+    // affecting determinism. Each line is `  name1, name2, name3,`.
+    let buf = "  ";
+    for (let i = 0; i < names.length; i++) {
+      const name = names[i];
+      const piece = (buf === "  " ? "" : ", ") + name;
+      if (buf.length > 2 && buf.length + piece.length > 80) {
+        lines.push(`${buf},`);
+        buf = `  ${name}`;
+      } else {
+        buf += piece;
+      }
+    }
+    if (buf.length > 2) lines.push(buf);
+  }
+
+  return lines.join("\n");
+}
+
+/**
+ * Replace pi's verbose `<available_skills>` block in `prompt` with the
+ * compressed one-index form. Idempotent: if the verbose form is not
+ * present (compression already applied, or skill count below threshold),
+ * the prompt is returned unchanged.
+ *
+ * Opt-out: set `PI_CACHE_OPTIMIZER_NO_SKILL_COMPRESSION=1`.
+ *
+ * Pre-conditions for compression to fire:
+ *   - opts.skills present and visible-skill count >= SKILL_COMPRESSION_MIN_COUNT
+ *   - Verbose block (built from the same `opts.skills`) is found in
+ *     `prompt` (substring match, no regex). This anchors the substitution
+ *     to pi's own emitter; if pi changes the format, we no-op rather
+ *     than mangle.
+ */
+function compressSkillsInSystemPrompt(
+  prompt: string,
+  opts: BuildSystemPromptOptions,
+): string {
+  if (isEnabledEnv(process.env[NO_SKILL_COMPRESSION_ENV])) return prompt;
+  if (!opts.skills || opts.skills.length === 0) return prompt;
+
+  const visible = opts.skills.filter((skill) => !skill.disableModelInvocation);
+  if (visible.length < SKILL_COMPRESSION_MIN_COUNT) return prompt;
+
+  const verbose = formatSkillsForPrompt(opts.skills);
+  if (!verbose || !prompt.includes(verbose)) return prompt;
+
+  const compressed = formatSkillsForPromptCompressed(opts.skills);
+  if (!compressed || compressed.length >= verbose.length) return prompt;
+
+  return prompt.replace(verbose, compressed);
+}
+
 function buildStableCandidates(opts: BuildSystemPromptOptions): string[] {
   const candidates: string[] = [];
 
@@ -195,12 +326,67 @@ function buildStableCandidates(opts: BuildSystemPromptOptions): string[] {
   }
 
   if (opts.skills && opts.skills.length > 0) {
+    // Push BOTH forms so `optimizeSystemPrompt` finds whichever is
+    // actually present in the prompt. The `rest.includes(part)`
+    // short-circuit skips the form that isn't there. The two strings
+    // are mutually distinguishable (the verbose form contains the
+    // literal `<available_skills>` envelope; the compressed form
+    // contains `Skills under ` and no XML tags) so they cannot
+    // accidentally match each other.
     candidates.push(formatSkillsForPrompt(opts.skills));
+    candidates.push(formatSkillsForPromptCompressed(opts.skills));
   }
 
   return candidates;
 }
 
+/**
+ * Strip per-turn churn from trellis `<session-overview>` block.
+ *
+ * Trellis injects a session-overview that includes `RECENT COMMITS`
+ * (shifts on every git commit), `Working directory: Clean/N uncommitted`
+ * (shifts on every edit/commit), and `Line count: N / 2000` (shifts on
+ * every journal append). These fields are at the tail of the
+ * session-overview and poison the prompt-prefix cache for everything
+ * that follows.
+ *
+ * This function surgically removes those three churn fields from the
+ * `<session-overview>...</session-overview>` block. The remaining
+ * fields (DEVELOPER, GIT STATUS branch-only, CURRENT TASK, ACTIVE
+ * TASKS, MY TASKS, JOURNAL FILE active-file-only, PACKAGES, PATHS)
+ * are stable within a session and become cache-friendlier.
+ *
+ * No-op when the `<session-overview>` tag is not present (e.g.
+ * trellis hook chose not to inject it, or a different extension
+ * owns the prompt).
+ */
+function stripSessionOverviewChurn(prompt: string): string {
+  const startTag = "<session-overview>";
+  const endTag = "</session-overview>";
+
+  const startIdx = prompt.indexOf(startTag);
+  if (startIdx === -1) return prompt;
+
+  const endIdx = prompt.indexOf(endTag, startIdx + startTag.length);
+  if (endIdx === -1) return prompt;
+
+  const before = prompt.slice(0, startIdx + startTag.length);
+  const inner = prompt.slice(startIdx + startTag.length, endIdx);
+  const after = prompt.slice(endIdx);
+
+  let cleaned = inner
+    // Drop the RECENT COMMITS section (from the heading through the
+    // next heading or end of inner). The model sees commit history
+    // via `git log`; carrying it in every system prompt is redundant.
+    .replace(/\n## RECENT COMMITS\n[\s\S]*?(?=\n## |$)/, "")
+    // Drop "Working directory: ..." (Git status tail churn).
+    .replace(/\nWorking directory:[^\n]*/g, "")
+    // Drop "Line count: N / NNNN" (Journal tail churn).
+    .replace(/\nLine count:[^\n]*/g, "");
+
+  return before + cleaned + after;
+}
+
 function optimizeSystemPrompt(
   original: string,
   opts: BuildSystemPromptOptions,
@@ -230,10 +416,27 @@ function optimizeSystemPrompt(
     return { systemPrompt: original, stablePrefix: "", changed: false };
   }
 
+  const systemPrompt =
+    stablePrefix +
+    (dynamicRemainder.length > 0 ? "\n\n---\n\n" + dynamicRemainder : "");
+
+  // Sanity check: if trellis (or another extension) injected structural
+  // markers into the prompt that happen to share a substring with one of
+  // our stable candidates, the blind `rest.replace(part, "")` could
+  // silently eat part of the dynamic layer. We anchor on
+  // `<workflow-state>` because it is the most stable structural marker
+  // trellis emits and is never a stable candidate itself.
+  //
+  // When the marker was present in the original but is missing in the
+  // result, the reorder is unsafe — fall back to the original prompt
+  // so the model gets a complete prompt, and flag the footer warning.
+  if (original.includes("<workflow-state>") && !systemPrompt.includes("<workflow-state>")) {
+    promptTruncationDetected = true;
+    return { systemPrompt: original, stablePrefix: "", changed: false };
+  }
+
   return {
-    systemPrompt:
-      stablePrefix +
-      (dynamicRemainder.length > 0 ? "\n\n---\n\n" + dynamicRemainder : ""),
+    systemPrompt,
     stablePrefix,
     changed: true,
   };
@@ -1036,7 +1239,12 @@ function emitDeepseekApiKeyHintIfNeeded(
 export const __internals_for_tests = {
   buildStableCandidates,
   optimizeSystemPrompt,
+  stripSessionOverviewChurn,
+  formatSkillsForPrompt,
+  formatSkillsForPromptCompressed,
+  compressSkillsInSystemPrompt,
   MIN_STABLE_CANDIDATE_LENGTH,
+  SKILL_COMPRESSION_MIN_COUNT,
 };
 
 export default function (pi: ExtensionAPI) {
@@ -1120,7 +1328,17 @@ export default function (pi: ExtensionAPI) {
     await rollOverStatsIfNeeded(ctx);
 
     const adapter = selectAdapterForModel(model);
-    const statusText = adapter ? formatCacheStats(adapter, getStatsForAdapter(adapter)) : undefined;
+    let statusText: string | undefined = adapter ? formatCacheStats(adapter, getStatsForAdapter(adapter)) : undefined;
+
+    // If optimizeSystemPrompt detected structural truncation on this or
+    // a recent turn, flag it once in the footer so the user knows to
+    // /reload before continuing. The flag resets after emission so a
+    // single-turn glitch does not permanently taint the footer.
+    if (promptTruncationDetected && statusText !== undefined) {
+      statusText = statusText + " ⚠️ integrity";
+      promptTruncationDetected = false;
+    }
+
     if (statusText === lastStatusText) return;
 
     lastStatusText = statusText;
@@ -1145,13 +1363,45 @@ export default function (pi: ExtensionAPI) {
   });
 
   pi.on("before_agent_start", async (event, _ctx) => {
-    const optimized = optimizeSystemPrompt(event.systemPrompt, event.systemPromptOptions);
+    // Step 1: strip per-turn churn from <session-overview>.
+    // Removing RECENT COMMITS, Working directory status, and
+    // Journal line count makes more of the session-overview stable
+    // across turns, which DeepSeek's prefix cache can then retain.
+    const strippedPrompt = stripSessionOverviewChurn(event.systemPrompt);
+
+    // Step 2: compress skills XML → one-line index.
+    // The compressed form is identical-string-equivalent to the
+    // verbose one as far as cache-stability is concerned because both
+    // are deterministic from the same `event.systemPromptOptions.skills`.
+    // No-op if opted out, below SKILL_COMPRESSION_MIN_COUNT, or if pi
+    // emitted a format we don't recognize.
+    const compressedPrompt = compressSkillsInSystemPrompt(
+      strippedPrompt,
+      event.systemPromptOptions,
+    );
+
+    // Step 3: lift stable content above dynamic content for cache
+    // stability. Operates on the (stripped + compressed) prompt so the
+    // cache key derived from `stablePrefix` reflects what actually
+    // ships to the provider.
+    const optimized = optimizeSystemPrompt(compressedPrompt, event.systemPromptOptions);
     latestPromptCacheKey = buildPromptCacheKey(optimized.stablePrefix);
 
     if (optimized.changed && optimized.systemPrompt.trim().length > 0) {
       return { systemPrompt: optimized.systemPrompt };
     }
 
+    // Reorder didn't apply but compression might have. Return the
+    // compressed (or stripped) prompt directly so we still benefit from
+    // the volume cut even when reorder is a no-op (e.g., short sessions
+    // where no stable candidate is long enough).
+    if (compressedPrompt !== strippedPrompt && compressedPrompt.trim().length > 0) {
+      return { systemPrompt: compressedPrompt };
+    }
+    if (strippedPrompt !== event.systemPrompt && strippedPrompt.trim().length > 0) {
+      return { systemPrompt: strippedPrompt };
+    }
+
     return {};
   });
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-cache-optimizer",
-  "version": "2.0.2",
+  "version": "2.1.1",
   "description": "Pi extension that improves provider-side KV/prompt cache hit rates (DeepSeek, OpenAI, Claude, Gemini) by reordering the system prompt, requesting long retention, and showing footer cache stats. Renamed from pi-deepseek-cache-optimizer.",
   "keywords": [
     "pi-package",