pi-kimi-keepalive 0.2.1 → 0.3.0

package/README.md

@@ -2,7 +2,7 @@
 
 Prompt-cache keepalive for [Kimi](https://www.kimi.com/) (`kimi-coding` provider) sessions in the [Pi coding agent](https://github.com/earendil-works/pi-coding-agent).
 
-Kimi's automatic prompt cache has an observed TTL of ~5 minutes. Once it expires, the next request re-reads the full context at full input price. This extension captures the last real provider request and replays it on a fixed interval while the session is idle, so the cached prefix stays warm and subsequent requests are billed at cache-read rates.
+Kimi's automatic prompt cache nominally expires after ~5 minutes of idle, but real-world testing shows the live TTL runs longer — probes as far apart as 8 minutes still hit reliably. Once the cache does expire, the next request re-reads the full context at full input price. This extension captures the last real provider request and replays it on a fixed interval while the session is idle, so the cached prefix stays warm and subsequent requests are billed at cache-read rates.
 
 Replayed requests are sent directly to the provider endpoint. They do not pass through the Pi session pipeline: no synthetic messages, no model turns, no changes to conversation history. Only aggregate statistics are surfaced.
 
@@ -46,17 +46,19 @@ The wizard ends with a prompt to enable keepalive. Probing starts after the next
 
 Two modes share the same guardrails; they differ in how the cadence is chosen.
 
-**`default` (the starting mode)** — the cadence is fixed at whatever `interval=` says, **8 minutes** by default — deliberately past the ~5-minute nominal cache TTL. An 8-minute probe rarely hits the cache: it runs once at full input price, confirms the cache has expired, and the default `miss=1` stops probing immediately, so the worst case is a single cold read per idle period. For an always-warm session instead, set a cadence inside the observed TTL (`/keepalive interval=4m45s`…`7m`): every probe is then billed at cache-read rates (~1/10 of full price) and the loop keeps running until `maxidle` cuts it off.
+**`default` (the starting mode)** — the cadence is fixed at whatever `interval=` says, **8 minutes** by default. Real-world testing shows probes at this cadence reliably hit the prefix cache (the effective TTL runs longer than the ~5-minute nominal one), so the default cadence already works as the always-warm heartbeat: every probe renews the cache at cache-read rates (~1/10 of full input price) until `maxidle` cuts the loop off.
+
+If the cache does expire underneath it (server-side eviction, TTL change), the mode self-heals instead of giving up: the missed probe itself rebuilds the cache entry with the same prefix, so the cadence drops to the **5-minute safe floor** (inside the nominal TTL), probing continues, and the next probe renews the entry. Only a miss **at** the 5-minute floor counts toward the `miss` pause threshold — a cache that cannot even be rebuilt at 5m is an environment problem, and the default `miss=1` stops probing there. The back-off is persisted; `/keepalive interval=8m` returns to the original cadence.
 
 **`smart` (`/keepalive mode=smart`)** — self-tunes the cadence toward the real cache TTL instead of guessing it:
 
 1. Starts at **8 minutes**.
-2. After **5 consecutive hits** the cadence is confirmed and the value grows by **+30s**. Only confirmed values are persisted, so `~/.pi/cache-keepalive/state.json` always holds the largest cadence with observed consecutive hits.
+2. After **3 consecutive hits** the cadence is confirmed and the value grows by **+30s**. Only confirmed values are persisted, so `~/.pi/cache-keepalive/state.json` always holds the largest cadence with observed consecutive hits.
 3. A **miss** immediately parks probing: the cadence steps back **30s** to the last confirmed value (never below the 8m floor), the value is persisted, and probing stays **stopped**. A fresh real turn does **not** resume it — only re-selecting smart mode (`/keepalive mode=smart`) continues from the parked cadence.
 4. **Context guard:** growth only happens while the last probe saw ≤ **200k prompt tokens**; the first probe above that immediately reverts the cadence to the 8m floor and keeps it there (a missed probe on a 200k+ context is too expensive to risk).
 5. Guardrails (`maxidle`, `cap`, `errors`, HTTP 401/403) still apply while smart is probing.
 
-Per learn cycle the spend is minimal: 5 cache-read probes confirm a step, and one full-price probe ends the cycle — the parked cadence keeps every future session at the highest value the cache has proven to hold.
+Per learn cycle the spend is minimal: 3 cache-read probes confirm a step, and one full-price probe ends the cycle — the parked cadence keeps every future session at the highest value the cache has proven to hold.
 
 ## Commands
 
@@ -66,9 +68,9 @@ Per learn cycle the spend is minimal: 5 cache-read probes confirm a step, and on
 /keepalive on|off           enable / disable (persisted)
 /keepalive now              one manual probe (bypasses pauses)
 /keepalive resume           clear a sticky pause
-/keepalive mode=smart       adaptive cadence (8m floor; +30s per 5-hit confirmation; a miss parks probing, mode=smart resumes)
+/keepalive mode=smart       adaptive cadence (8m floor; +30s per 3-hit confirmation; a miss parks probing, mode=smart resumes)
 /keepalive mode=default     fixed cadence (the interval= value)
-/keepalive interval=4m45s   probe cadence in default mode (≥ 30s; ≤ 5m to stay inside the cache TTL)
+/keepalive interval=4m45s   probe cadence in default mode (≥ 30s; default 8m reliably hits the cache in practice)
 /keepalive maxidle=30m      idle cutoff (0 = disabled)
 /keepalive miss=1           pause after N consecutive cache misses
 /keepalive errors=3         pause after N consecutive probe failures
@@ -120,7 +122,7 @@ On a Kimi subscription, billing is quota-based and USD figures are indicative on
 
 ## Limitations
 
-- The ~5 minute TTL and the pricing above are observed behavior, not an API contract. The `saved` estimate is informational, not a guaranteed saving.
+- The nominal ~5-minute TTL, the longer effective TTL observed in practice, and the pricing above are observed behavior, not an API contract. The `saved` estimate is informational, not a guaranteed saving.
 - Only the `kimi-coding` provider (`kimi-openai-completions` API, with an `anthropic-messages` fallback) is supported. Other providers have different cache-key semantics and are out of scope.
 - Captured payloads and headers live only in process memory; probes are sent only to `https://` endpoints.
 

package/README.zh-CN.md

@@ -4,7 +4,7 @@ English | [中文文档](README.zh-CN.md)
 
 Prompt-cache keepalive for [Kimi](https://www.kimi.com/) (`kimi-coding` provider) sessions in the [Pi coding agent](https://github.com/earendil-works/pi-coding-agent).
 
-Kimi 的自动 prompt 缓存 TTL 实测约 5 分钟。TTL 过期后恢复会话时，整个上下文以全价 input（$3 / 1M）重新计算。本扩展捕获最后一条真实 provider 请求，在会话空闲期间以固定间隔将其重放至同一端点，使缓存前缀在 TTL 内保持有效，后续请求按 cache-read 价格（$0.3 / 1M）计费。
+Kimi 的自动 prompt 缓存名义上在空闲 ~5 分钟后过期，但实测实际 TTL 更长——间隔 8 分钟的探测仍基本稳定命中。缓存真正过期后，整个上下文以全价 input（$3 / 1M）重新计算。本扩展捕获最后一条真实 provider 请求，在会话空闲期间以固定间隔将其重放至同一端点，使缓存前缀在 TTL 内保持有效，后续请求按 cache-read 价格（$0.3 / 1M）计费。
 
 重放请求直接发送到 provider 端点，不经过 Pi 会话管道：不产生合成消息、不产生模型回合、不改动对话历史，仅在界面上展示聚合统计。
 
@@ -46,17 +46,19 @@ npm 发布后可使用 `pi install npm:pi-kimi-keepalive`。
 
 两种模式共用一套护栏，区别仅在间隔的确定方式。
 
-**`default`（默认模式）**——间隔固定为 `interval=` 所设值，默认 **8 分钟**——刻意超过 ~5 分钟的名义缓存 TTL。8 分钟的探测通常无法命中：以全价输入运行一次、确认缓存已过期，默认 `miss=1` 随即停止探测，因此最坏情况是每个空闲期仅一次全价冷读。若想保持会话常热，可把间隔设在实测 TTL 之内（`/keepalive interval=4m45s`…`7m`）：每次探测按缓存读价计费（约为全价的 1/10），循环持续到 `maxidle` 截断。
+**`default`（默认模式）**——间隔固定为 `interval=` 所设值，默认 **8 分钟**。实测该间隔的探测基本稳定命中前缀缓存（实际 TTL 长于名义的 ~5 分钟），因此默认间隔本身就是常热心跳：每次探测以缓存读价（约为全价 input 的 1/10）续期缓存，循环持续到 `maxidle` 截断。
+
+若缓存确实在下方过期（服务端逐出、TTL 变更），该模式会自愈而非直接失效：miss 的那次探测本身会以相同前缀重建缓存条目，因此档位自动回退至 **5 分钟安全档**（名义 TTL 之内）、继续探测，下一次探测即续期成功。只有**在 5 分钟档**再 miss 才计入 `miss` 暂停阈值——连 5m 都无法重建缓存的属于环境异常，默认 `miss=1` 在此暂停。降档会持久化；`/keepalive interval=8m` 可调回原档位。
 
 **`smart`（`/keepalive mode=smart`）**——不猜测 TTL，而是自适应逼近真实值：
 
 1. 从 **8 分钟**起跳。
-2. **连续 5 次命中**后该间隔即被确认，档位 **+30s**。只有确认过的值才会持久化，因此 `~/.pi/cache-keepalive/state.json` 中始终保存的是实测可连续命中的最大档位。
+2. **连续 3 次命中**后该间隔即被确认，档位 **+30s**。只有确认过的值才会持久化，因此 `~/.pi/cache-keepalive/state.json` 中始终保存的是实测可连续命中的最大档位。
 3. 一次 **miss** 立即停靠探测：档位回退 **30s** 到最近确认值（至多退到 8m 下限）并持久化，随后探测**停止**。下一次真实轮次**不会**自动恢复——只有重新选择 smart 模式（`/keepalive mode=smart`）才会从停靠档位继续探测。
 4. **上下文保护：** 只有最近一次探测的 prompt tokens ≤ **200k** 才允许升档；一旦超过，档位立即回退至 8m 下限并冻结升档（200k+ 上下文一次全价 miss 代价太高，不冒这个险）。
 5. smart 探测运行期间通用护栏（`maxidle`、`cap`、`errors`、HTTP 401/403）仍然生效。
 
-每个学习周期的花费极小：5 次缓存读价探测确认一档，一次全价探测结束本轮——停靠后的档位让之后的每个会话都直接运行在实测可行的最高值上。
+每个学习周期的花费极小：3 次缓存读价探测确认一档，一次全价探测结束本轮——停靠后的档位让之后的每个会话都直接运行在实测可行的最高值上。
 
 ## 命令
 
@@ -66,9 +68,9 @@ npm 发布后可使用 `pi install npm:pi-kimi-keepalive`。
 /keepalive on|off           启用 / 停用（持久化）
 /keepalive now              手动探测一次（绕过暂停）
 /keepalive resume           清除 sticky 暂停
-/keepalive mode=smart       自适应间隔（下限 8m；每 5 连中 +30s；一次 miss 即停靠，mode=smart 恢复探测）
+/keepalive mode=smart       自适应间隔（下限 8m；每 3 连中 +30s；一次 miss 即停靠，mode=smart 恢复探测）
 /keepalive mode=default     固定间隔（即 interval= 的值）
-/keepalive interval=4m45s   default 模式下的探测间隔（≥ 30s；应 ≤ 5m 以保持缓存命中）
+/keepalive interval=4m45s   default 模式下的探测间隔（≥ 30s；默认 8m 实测基本稳定命中）
 /keepalive maxidle=30m      空闲上限（0 = 不设限）
 /keepalive miss=1           连续 N 次缓存 miss 后暂停
 /keepalive errors=3         连续 N 次探测失败后熔断
@@ -118,7 +120,7 @@ Kimi 订阅按 quota 计费，USD 数值仅供参考。
 
 ## 限制
 
-- ~5 分钟 TTL 与上述定价为观测行为，非 API 契约；`saved` 仅为估算。
+- 名义 ~5 分钟 TTL、实测更长的有效 TTL 与上述定价均为观测行为，非 API 契约；`saved` 仅为估算。
 - 仅支持 `kimi-coding`（`kimi-openai-completions` API，含 `anthropic-messages` 回退）；其他 provider 缓存键语义不同，不在范围内。
 - 捕获内容仅存内存；探测仅发往 `https://` 端点。
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-kimi-keepalive",
-  "version": "0.2.1",
+  "version": "0.3.0",
   "description": "Prompt-cache keepalive for Kimi (kimi-coding) sessions in the Pi coding agent. Replays the last real provider request while idle so the automatic prefix cache never expires.",
   "type": "module",
   "license": "MIT",

package/src/index.ts

@@ -76,11 +76,12 @@ interface PersistedConfig {
 
 const DEFAULT_CONFIG: Readonly<PersistedConfig> = Object.freeze({
   enabled: false,
-  // 7 min: past the ~5 min cache TTL by design. A 7-min probe never hits the
-  // cache, so it acts as a single full-price confirmation that the cache is
-  // dead, and the default miss=1 stops probing right after — worst-case total
-  // spend is one cold read per session. Use /keepalive interval=4m45s for a
-  // hit-mode heartbeat (every probe is a cache read, ~10x cheaper).
+  // 8 min: real-world testing shows probes at this cadence still hit the
+  // prefix cache reliably (the effective TTL runs longer than the ~5 min
+  // nominal TTL), so the default cadence IS the hit-mode heartbeat — every
+  // probe is billed at cache-read rates (~10x cheaper than a cold read).
+  // If the cache does expire (e.g. server-side eviction), the probe misses
+  // once at full price and the default miss=1 stops probing immediately.
   intervalMs: 8 * 60_000,
   maxIdleMs: 30 * 60_000,
   minPromptTokens: 512,
@@ -94,9 +95,16 @@ const DEFAULT_CONFIG: Readonly<PersistedConfig> = Object.freeze({
 
 // smart-mode constants
 const SMART_BASE_MS = 8 * 60_000; // smart starting/floor cadence (matches the default interval)
-const SMART_STEP_MS = 30_000; // +30s per 5-hit confirmation
-const SMART_CONFIRM_HITS = 5;
+const SMART_STEP_MS = 30_000; // +30s per 3-hit confirmation
+const SMART_CONFIRM_HITS = 3;
 const SMART_MAX_CONTEXT_TOKENS = 200_000; // context cap: grow only below this
+/**
+ * default-mode safe floor: the nominal cache TTL. A miss while probing above
+ * this cadence drops the cadence here and keeps probing (the miss probe
+ * itself rebuilds the cache entry, so the next ≤5m probe renews it); only a
+ * miss AT this floor counts toward the miss-pause threshold.
+ */
+const DEFAULT_FALLBACK_MS = 5 * 60_000;
 
 const MIN_INTERVAL_MS = 30_000;
 const PROBE_TIMEOUT_MS = 30_000;
@@ -108,9 +116,9 @@ const HELP_TEXT = [
   "  /keepalive on|off         enable / disable (persisted)",
   "  /keepalive now            one manual probe (bypasses pauses)",
   "  /keepalive resume         clear a sticky pause",
-  "  /keepalive mode=smart     adaptive cadence (8m floor; +30s per 5-hit confirmation; a miss pauses probing, mode=smart resumes)",
+  "  /keepalive mode=smart     adaptive cadence (8m floor; +30s per 3-hit confirmation; a miss parks probing, mode=smart resumes)",
   "  /keepalive mode=default   fixed cadence (the interval= value)",
-  "  /keepalive interval=4m45s probe cadence (default mode; >= 30s; <= 5m stays inside the cache TTL)",
+  "  /keepalive interval=4m45s probe cadence (default mode; >= 30s; default 8m reliably hits the cache in practice)",
   "  /keepalive maxidle=30m    stop probing after this idle time (0 = never stop)",
   "  /keepalive miss=1         pause after N consecutive cache misses",
   "  /keepalive errors=3       pause after N consecutive probe failures",
@@ -319,7 +327,7 @@ export default function (pi: ExtensionAPI) {
     const modeRaw = await wizardCtx.ui.input(
       "Step 5/5 — Probing mode (now " + config.mode + ")\n" +
         "default: probes run at the fixed interval above.\n" +
-        "smart: starts at 8m; after 5 consecutive hits (context ≤ 200k) the cadence grows by 30s; " +
+        "smart: starts at 8m; after 3 consecutive hits (context ≤ 200k) the cadence grows by 30s; " +
         "one miss steps back to the last confirmed value and parks probing until you re-select smart mode — " +
         "it self-tunes toward the real cache TTL to minimize probe spend.\n" +
         "Type smart to enable; leave empty / press Esc for default.",
@@ -564,6 +572,20 @@ export default function (pi: ExtensionAPI) {
       debug(`probe miss: cache_read=0 input=${usage.inputTokens}`);
       if (config.mode === "smart") {
         smartAdaptAfterMiss();
+      } else if (config.intervalMs > DEFAULT_FALLBACK_MS) {
+        // Miss while probing above the safe floor: the miss probe itself
+        // rebuilds the cache entry with the same prefix, so drop to the 5m
+        // safe cadence (inside the nominal TTL) and keep probing — the next
+        // probe renews it. Reset the streak so the new cadence gets a fresh
+        // chance before a pause is considered.
+        config.intervalMs = DEFAULT_FALLBACK_MS;
+        missStreak = 0;
+        persistConfig();
+        notify(
+          `cache miss — cadence backed off to ${formatDuration(config.intervalMs)} (safe TTL window); probing continues`,
+          "info",
+        );
+        updateUi();
       } else {
         missStreak += 1;
         debug(`probe miss #${missStreak}: cache_read=0 input=${usage.inputTokens}`);
@@ -617,7 +639,7 @@ export default function (pi: ExtensionAPI) {
     const roomUnderMaxIdle =
       config.maxIdleMs === 0 || config.intervalMs + SMART_STEP_MS < config.maxIdleMs;
     if (roomUnderMaxIdle) {
-      config.intervalMs += SMART_STEP_MS; // 5-hit-confirmed value stays on disk
+      config.intervalMs += SMART_STEP_MS; // 3-hit-confirmed value stays on disk
       persistConfig();
       debug(
         `smart: ${smartHitStreak} consecutive hits — cadence grows to ${formatDuration(config.intervalMs)}`,