pi-kimi-keepalive 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -6
- package/README.zh-CN.md +26 -5
- package/package.json +5 -3
- package/src/index.ts +197 -24
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Prompt-cache keepalive for [Kimi](https://www.kimi.com/) (`kimi-coding` provider) sessions in the [Pi coding agent](https://github.com/earendil-works/pi-coding-agent).
|
|
4
4
|
|
|
5
|
-
Kimi's automatic prompt cache
|
|
5
|
+
Kimi's automatic prompt cache nominally expires after ~5 minutes of idle, but real-world testing shows the live TTL runs longer — probes as far apart as 8 minutes still hit reliably. Once the cache does expire, the next request re-reads the full context at full input price. This extension captures the last real provider request and replays it on a fixed interval while the session is idle, so the cached prefix stays warm and subsequent requests are billed at cache-read rates.
|
|
6
6
|
|
|
7
7
|
Replayed requests are sent directly to the provider endpoint. They do not pass through the Pi session pipeline: no synthetic messages, no model turns, no changes to conversation history. Only aggregate statistics are surfaced.
|
|
8
8
|
|
|
@@ -30,18 +30,35 @@ After the package is published to npm: `pi install npm:pi-kimi-keepalive`.
|
|
|
30
30
|
|
|
31
31
|
## Setup
|
|
32
32
|
|
|
33
|
-
On the first start with no `~/.pi/cache-keepalive/state.json` present and an interactive UI, the extension runs a setup wizard that configures the
|
|
33
|
+
On the first start with no `~/.pi/cache-keepalive/state.json` present and an interactive UI, the extension runs a setup wizard that configures the five settings below. Leaving a prompt empty or pressing Esc keeps the default. The wizard can be rerun with `/keepalive setup`; in headless sessions it is skipped and defaults are kept.
|
|
34
34
|
|
|
35
35
|
| Step | Setting | Command | Default | Description |
|
|
36
36
|
| --- | --- | --- | --- | --- |
|
|
37
37
|
| 1 | Max idle cutoff | `maxidle` | `30m` | Probing stops after this much idle time; `0` disables the cutoff. |
|
|
38
|
-
| 2 | Miss pause threshold | `miss` | `1` | Pause after N consecutive probes that do not hit the prompt cache. A hit resets the count. |
|
|
38
|
+
| 2 | Miss pause threshold | `miss` | `1` | Pause after N consecutive probes that do not hit the prompt cache. A hit resets the count. Applies in `default` mode. |
|
|
39
39
|
| 3 | Error circuit breaker | `errors` | `3` | Pause after N consecutive probe failures (network errors, HTTP 5xx). HTTP 401/403 always pauses immediately. |
|
|
40
40
|
| 4 | Session spend cap | `cap` | `$1.00` | Ceiling on estimated USD probe spend per session; `0` removes the cap. |
|
|
41
|
+
| 5 | Probing mode | `mode` | `default` | `default` keeps the fixed cadence; `smart` self-tunes it (below). |
|
|
41
42
|
|
|
42
43
|
The wizard ends with a prompt to enable keepalive. Probing starts after the next real turn, which provides the captured request.
|
|
43
44
|
|
|
44
|
-
|
|
45
|
+
## Probing modes
|
|
46
|
+
|
|
47
|
+
Two modes share the same guardrails; they differ in how the cadence is chosen.
|
|
48
|
+
|
|
49
|
+
**`default` (the starting mode)** — the cadence is fixed at whatever `interval=` says, **8 minutes** by default. Real-world testing shows probes at this cadence reliably hit the prefix cache (the effective TTL runs longer than the ~5-minute nominal one), so the default cadence already works as the always-warm heartbeat: every probe renews the cache at cache-read rates (~1/10 of full input price) until `maxidle` cuts the loop off.
|
|
50
|
+
|
|
51
|
+
If the cache does expire underneath it (server-side eviction, TTL change), the mode self-heals instead of giving up: the missed probe itself rebuilds the cache entry with the same prefix, so the cadence drops to the **5-minute safe floor** (inside the nominal TTL), probing continues, and the next probe renews the entry. Only a miss **at** the 5-minute floor counts toward the `miss` pause threshold — a cache that cannot even be rebuilt at 5m is an environment problem, and the default `miss=1` stops probing there. The back-off is persisted; `/keepalive interval=8m` returns to the original cadence.
|
|
52
|
+
|
|
53
|
+
**`smart` (`/keepalive mode=smart`)** — self-tunes the cadence toward the real cache TTL instead of guessing it:
|
|
54
|
+
|
|
55
|
+
1. Starts at **8 minutes**.
|
|
56
|
+
2. After **3 consecutive hits** the cadence is confirmed and the value grows by **+30s**. Only confirmed values are persisted, so `~/.pi/cache-keepalive/state.json` always holds the largest cadence with observed consecutive hits.
|
|
57
|
+
3. A **miss** immediately parks probing: the cadence steps back **30s** to the last confirmed value (never below the 8m floor), the value is persisted, and probing stays **stopped**. A fresh real turn does **not** resume it — only re-selecting smart mode (`/keepalive mode=smart`) continues from the parked cadence.
|
|
58
|
+
4. **Context guard:** growth only happens while the last probe saw ≤ **200k prompt tokens**; the first probe above that immediately reverts the cadence to the 8m floor and keeps it there (a missed probe on a 200k+ context is too expensive to risk).
|
|
59
|
+
5. Guardrails (`maxidle`, `cap`, `errors`, HTTP 401/403) still apply while smart is probing.
|
|
60
|
+
|
|
61
|
+
Per learn cycle the spend is minimal: 3 cache-read probes confirm a step, and one full-price probe ends the cycle — the parked cadence keeps every future session at the highest value the cache has proven to hold.
|
|
45
62
|
|
|
46
63
|
## Commands
|
|
47
64
|
|
|
@@ -51,7 +68,9 @@ Two operating points are worth knowing. The default cadence is **7 minutes** —
|
|
|
51
68
|
/keepalive on|off enable / disable (persisted)
|
|
52
69
|
/keepalive now one manual probe (bypasses pauses)
|
|
53
70
|
/keepalive resume clear a sticky pause
|
|
54
|
-
/keepalive
|
|
71
|
+
/keepalive mode=smart adaptive cadence (8m floor; +30s per 3-hit confirmation; a miss parks probing, mode=smart resumes)
|
|
72
|
+
/keepalive mode=default fixed cadence (the interval= value)
|
|
73
|
+
/keepalive interval=4m45s probe cadence in default mode (≥ 30s; default 8m reliably hits the cache in practice)
|
|
55
74
|
/keepalive maxidle=30m idle cutoff (0 = disabled)
|
|
56
75
|
/keepalive miss=1 pause after N consecutive cache misses
|
|
57
76
|
/keepalive errors=3 pause after N consecutive probe failures
|
|
@@ -103,7 +122,7 @@ On a Kimi subscription, billing is quota-based and USD figures are indicative on
|
|
|
103
122
|
|
|
104
123
|
## Limitations
|
|
105
124
|
|
|
106
|
-
- The ~5
|
|
125
|
+
- The nominal ~5-minute TTL, the longer effective TTL observed in practice, and the pricing above are observed behavior, not an API contract. The `saved` estimate is informational, not a guaranteed saving.
|
|
107
126
|
- Only the `kimi-coding` provider (`kimi-openai-completions` API, with an `anthropic-messages` fallback) is supported. Other providers have different cache-key semantics and are out of scope.
|
|
108
127
|
- Captured payloads and headers live only in process memory; probes are sent only to `https://` endpoints.
|
|
109
128
|
|
package/README.zh-CN.md
CHANGED
|
@@ -4,7 +4,7 @@ English | [中文文档](README.zh-CN.md)
|
|
|
4
4
|
|
|
5
5
|
Prompt-cache keepalive for [Kimi](https://www.kimi.com/) (`kimi-coding` provider) sessions in the [Pi coding agent](https://github.com/earendil-works/pi-coding-agent).
|
|
6
6
|
|
|
7
|
-
Kimi 的自动 prompt
|
|
7
|
+
Kimi 的自动 prompt 缓存名义上在空闲 ~5 分钟后过期,但实测实际 TTL 更长——间隔 8 分钟的探测仍基本稳定命中。缓存真正过期后,整个上下文以全价 input($3 / 1M)重新计算。本扩展捕获最后一条真实 provider 请求,在会话空闲期间以固定间隔将其重放至同一端点,使缓存前缀在 TTL 内保持有效,后续请求按 cache-read 价格($0.3 / 1M)计费。
|
|
8
8
|
|
|
9
9
|
重放请求直接发送到 provider 端点,不经过 Pi 会话管道:不产生合成消息、不产生模型回合、不改动对话历史,仅在界面上展示聚合统计。
|
|
10
10
|
|
|
@@ -30,17 +30,36 @@ npm 发布后可使用 `pi install npm:pi-kimi-keepalive`。
|
|
|
30
30
|
|
|
31
31
|
## 初始化
|
|
32
32
|
|
|
33
|
-
首次启动(`~/.pi/cache-keepalive/state.json` 不存在)且有交互 UI
|
|
33
|
+
首次启动(`~/.pi/cache-keepalive/state.json` 不存在)且有交互 UI 时,扩展运行初始化向导配置五项设置。提示符留空或按 Esc 保留默认值;之后可用 `/keepalive setup` 重新运行;headless 会话自动跳过并保留默认值。
|
|
34
34
|
|
|
35
35
|
| 步骤 | 设置项 | 命令 | 默认值 | 说明 |
|
|
36
36
|
| --- | --- | --- | --- | --- |
|
|
37
37
|
| 1 | Max idle cutoff | `maxidle` | `30m` | 空闲超过该时长后停止探测;`0` 表示不设限 |
|
|
38
|
-
| 2 | Miss pause threshold | `miss` | `
|
|
38
|
+
| 2 | Miss pause threshold | `miss` | `1` | 连续 N 次探测未命中前缀缓存后暂停;命中会重置计数。仅在 `default` 模式下生效 |
|
|
39
39
|
| 3 | Error circuit breaker | `errors` | `3` | 连续 N 次探测失败(网络错误、HTTP 5xx)后暂停;HTTP 401/403 不受此值约束,直接暂停 |
|
|
40
40
|
| 4 | Session spend cap | `cap` | `$1.00` | 会话探测花费(估算 USD)上限;`0` 表示不设上限 |
|
|
41
|
+
| 5 | Probing mode | `mode` | `default` | `default` 固定间隔;`smart` 自适应(见下文) |
|
|
41
42
|
|
|
42
43
|
向导结束时询问是否立即启用 keepalive。探测在下一次真实请求(完成捕获)之后开始。全部配置持久化到 `~/.pi/cache-keepalive/state.json`。
|
|
43
44
|
|
|
45
|
+
## 探测模式
|
|
46
|
+
|
|
47
|
+
两种模式共用一套护栏,区别仅在间隔的确定方式。
|
|
48
|
+
|
|
49
|
+
**`default`(默认模式)**——间隔固定为 `interval=` 所设值,默认 **8 分钟**。实测该间隔的探测基本稳定命中前缀缓存(实际 TTL 长于名义的 ~5 分钟),因此默认间隔本身就是常热心跳:每次探测以缓存读价(约为全价 input 的 1/10)续期缓存,循环持续到 `maxidle` 截断。
|
|
50
|
+
|
|
51
|
+
若缓存确实在下方过期(服务端逐出、TTL 变更),该模式会自愈而非直接失效:miss 的那次探测本身会以相同前缀重建缓存条目,因此档位自动回退至 **5 分钟安全档**(名义 TTL 之内)、继续探测,下一次探测即续期成功。只有**在 5 分钟档**再 miss 才计入 `miss` 暂停阈值——连 5m 都无法重建缓存的属于环境异常,默认 `miss=1` 在此暂停。降档会持久化;`/keepalive interval=8m` 可调回原档位。
|
|
52
|
+
|
|
53
|
+
**`smart`(`/keepalive mode=smart`)**——不猜测 TTL,而是自适应逼近真实值:
|
|
54
|
+
|
|
55
|
+
1. 从 **8 分钟**起跳。
|
|
56
|
+
2. **连续 3 次命中**后该间隔即被确认,档位 **+30s**。只有确认过的值才会持久化,因此 `~/.pi/cache-keepalive/state.json` 中始终保存的是实测可连续命中的最大档位。
|
|
57
|
+
3. 一次 **miss** 立即停靠探测:档位回退 **30s** 到最近确认值(至多退到 8m 下限)并持久化,随后探测**停止**。下一次真实轮次**不会**自动恢复——只有重新选择 smart 模式(`/keepalive mode=smart`)才会从停靠档位继续探测。
|
|
58
|
+
4. **上下文保护:** 只有最近一次探测的 prompt tokens ≤ **200k** 才允许升档;一旦超过,档位立即回退至 8m 下限并冻结升档(200k+ 上下文一次全价 miss 代价太高,不冒这个险)。
|
|
59
|
+
5. smart 探测运行期间通用护栏(`maxidle`、`cap`、`errors`、HTTP 401/403)仍然生效。
|
|
60
|
+
|
|
61
|
+
每个学习周期的花费极小:3 次缓存读价探测确认一档,一次全价探测结束本轮——停靠后的档位让之后的每个会话都直接运行在实测可行的最高值上。
|
|
62
|
+
|
|
44
63
|
## 命令
|
|
45
64
|
|
|
46
65
|
```
|
|
@@ -49,7 +68,9 @@ npm 发布后可使用 `pi install npm:pi-kimi-keepalive`。
|
|
|
49
68
|
/keepalive on|off 启用 / 停用(持久化)
|
|
50
69
|
/keepalive now 手动探测一次(绕过暂停)
|
|
51
70
|
/keepalive resume 清除 sticky 暂停
|
|
52
|
-
/keepalive
|
|
71
|
+
/keepalive mode=smart 自适应间隔(下限 8m;每 3 连中 +30s;一次 miss 即停靠,mode=smart 恢复探测)
|
|
72
|
+
/keepalive mode=default 固定间隔(即 interval= 的值)
|
|
73
|
+
/keepalive interval=4m45s default 模式下的探测间隔(≥ 30s;默认 8m 实测基本稳定命中)
|
|
53
74
|
/keepalive maxidle=30m 空闲上限(0 = 不设限)
|
|
54
75
|
/keepalive miss=1 连续 N 次缓存 miss 后暂停
|
|
55
76
|
/keepalive errors=3 连续 N 次探测失败后熔断
|
|
@@ -99,7 +120,7 @@ Kimi 订阅按 quota 计费,USD 数值仅供参考。
|
|
|
99
120
|
|
|
100
121
|
## 限制
|
|
101
122
|
|
|
102
|
-
- ~5 分钟 TTL
|
|
123
|
+
- 名义 ~5 分钟 TTL、实测更长的有效 TTL 与上述定价均为观测行为,非 API 契约;`saved` 仅为估算。
|
|
103
124
|
- 仅支持 `kimi-coding`(`kimi-openai-completions` API,含 `anthropic-messages` 回退);其他 provider 缓存键语义不同,不在范围内。
|
|
104
125
|
- 捕获内容仅存内存;探测仅发往 `https://` 端点。
|
|
105
126
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-kimi-keepalive",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "Prompt-cache keepalive for Kimi (kimi-coding) sessions in the Pi coding agent. Replays the last real provider request while idle so the automatic prefix cache never expires.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -23,7 +23,9 @@
|
|
|
23
23
|
"keepalive"
|
|
24
24
|
],
|
|
25
25
|
"pi": {
|
|
26
|
-
"extensions": [
|
|
26
|
+
"extensions": [
|
|
27
|
+
"./src/index.ts"
|
|
28
|
+
]
|
|
27
29
|
},
|
|
28
30
|
"files": [
|
|
29
31
|
"src",
|
|
@@ -43,4 +45,4 @@
|
|
|
43
45
|
"@types/node": "^24",
|
|
44
46
|
"typescript": "^5"
|
|
45
47
|
}
|
|
46
|
-
}
|
|
48
|
+
}
|
package/src/index.ts
CHANGED
|
@@ -68,27 +68,44 @@ interface PersistedConfig {
|
|
|
68
68
|
maxMissStreak: number;
|
|
69
69
|
/** Consecutive failed probes (network/server errors) before probing pauses. */
|
|
70
70
|
maxErrorStreak: number;
|
|
71
|
+
/** "default" = fixed interval; "smart" = adaptive cadence that grows while probes keep hitting. */
|
|
72
|
+
mode: "default" | "smart";
|
|
71
73
|
/** True once the first-run setup wizard has completed (or been skipped). */
|
|
72
74
|
initialized: boolean;
|
|
73
75
|
}
|
|
74
76
|
|
|
75
77
|
const DEFAULT_CONFIG: Readonly<PersistedConfig> = Object.freeze({
|
|
76
78
|
enabled: false,
|
|
77
|
-
//
|
|
78
|
-
// cache
|
|
79
|
-
//
|
|
80
|
-
//
|
|
81
|
-
//
|
|
82
|
-
|
|
79
|
+
// 8 min: real-world testing shows probes at this cadence still hit the
|
|
80
|
+
// prefix cache reliably (the effective TTL runs longer than the ~5 min
|
|
81
|
+
// nominal TTL), so the default cadence IS the hit-mode heartbeat — every
|
|
82
|
+
// probe is billed at cache-read rates (~10x cheaper than a cold read).
|
|
83
|
+
// If the cache does expire (e.g. server-side eviction), the probe misses
|
|
84
|
+
// once at full price and the default miss=1 stops probing immediately.
|
|
85
|
+
intervalMs: 8 * 60_000,
|
|
83
86
|
maxIdleMs: 30 * 60_000,
|
|
84
87
|
minPromptTokens: 512,
|
|
85
88
|
maxOutputTokens: 16,
|
|
86
89
|
spendCapUsd: 1.0,
|
|
87
90
|
maxMissStreak: 1,
|
|
88
91
|
maxErrorStreak: 3,
|
|
92
|
+
mode: "default",
|
|
89
93
|
initialized: false,
|
|
90
94
|
});
|
|
91
95
|
|
|
96
|
+
// smart-mode constants
|
|
97
|
+
const SMART_BASE_MS = 8 * 60_000; // smart starting/floor cadence (matches the default interval)
|
|
98
|
+
const SMART_STEP_MS = 30_000; // +30s per 3-hit confirmation
|
|
99
|
+
const SMART_CONFIRM_HITS = 3;
|
|
100
|
+
const SMART_MAX_CONTEXT_TOKENS = 200_000; // context cap: grow only below this
|
|
101
|
+
/**
|
|
102
|
+
* default-mode safe floor: the nominal cache TTL. A miss while probing above
|
|
103
|
+
* this cadence drops the cadence here and keeps probing (the miss probe
|
|
104
|
+
* itself rebuilds the cache entry, so the next ≤5m probe renews it); only a
|
|
105
|
+
* miss AT this floor counts toward the miss-pause threshold.
|
|
106
|
+
*/
|
|
107
|
+
const DEFAULT_FALLBACK_MS = 5 * 60_000;
|
|
108
|
+
|
|
92
109
|
const MIN_INTERVAL_MS = 30_000;
|
|
93
110
|
const PROBE_TIMEOUT_MS = 30_000;
|
|
94
111
|
|
|
@@ -99,7 +116,9 @@ const HELP_TEXT = [
|
|
|
99
116
|
" /keepalive on|off enable / disable (persisted)",
|
|
100
117
|
" /keepalive now one manual probe (bypasses pauses)",
|
|
101
118
|
" /keepalive resume clear a sticky pause",
|
|
102
|
-
" /keepalive
|
|
119
|
+
" /keepalive mode=smart adaptive cadence (8m floor; +30s per 3-hit confirmation; a miss parks probing, mode=smart resumes)",
|
|
120
|
+
" /keepalive mode=default fixed cadence (the interval= value)",
|
|
121
|
+
" /keepalive interval=4m45s probe cadence (default mode; >= 30s; default 8m reliably hits the cache in practice)",
|
|
103
122
|
" /keepalive maxidle=30m stop probing after this idle time (0 = never stop)",
|
|
104
123
|
" /keepalive miss=1 pause after N consecutive cache misses",
|
|
105
124
|
" /keepalive errors=3 pause after N consecutive probe failures",
|
|
@@ -144,7 +163,12 @@ export default function (pi: ExtensionAPI) {
|
|
|
144
163
|
let pausedReason: string | null = null;
|
|
145
164
|
let missStreak = 0;
|
|
146
165
|
let errorStreak = 0;
|
|
147
|
-
|
|
166
|
+
// smart-mode runtime state (cadence itself lives in config.intervalMs, persisted)
|
|
167
|
+
let smartHitStreak = 0;
|
|
168
|
+
// Set when a smart-mode miss pauses probing; only re-selecting smart mode
|
|
169
|
+
// clears it (a fresh real turn does NOT resume probing in this case).
|
|
170
|
+
let smartPaused = false;
|
|
171
|
+
let lastProbeInputTokens = 0;
|
|
148
172
|
const stats: Stats = { probes: 0, hits: 0, misses: 0, errors: 0, savedUsd: 0, spendUsd: 0 };
|
|
149
173
|
|
|
150
174
|
// ---------- persistence ----------
|
|
@@ -177,6 +201,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
177
201
|
: DEFAULT_CONFIG.spendCapUsd,
|
|
178
202
|
maxMissStreak: int(raw.maxMissStreak, DEFAULT_CONFIG.maxMissStreak, 1),
|
|
179
203
|
maxErrorStreak: int(raw.maxErrorStreak, DEFAULT_CONFIG.maxErrorStreak, 1),
|
|
204
|
+
mode: raw.mode === "smart" ? "smart" : "default",
|
|
180
205
|
initialized: raw.initialized === true,
|
|
181
206
|
};
|
|
182
207
|
} catch {
|
|
@@ -217,7 +242,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
217
242
|
config.initialized = true;
|
|
218
243
|
persistConfig();
|
|
219
244
|
notify(
|
|
220
|
-
"Setup skipped — defaults kept (
|
|
245
|
+
"Setup skipped — defaults kept (mode default/8m, maxidle 30m, miss 1, errors 3, cap $1.00). " +
|
|
221
246
|
"Run /keepalive setup to configure later, /keepalive on to enable.",
|
|
222
247
|
"info",
|
|
223
248
|
);
|
|
@@ -225,9 +250,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
225
250
|
return;
|
|
226
251
|
}
|
|
227
252
|
|
|
228
|
-
// 1/
|
|
253
|
+
// 1/5 — max idle cutoff
|
|
229
254
|
const maxIdleRaw = await wizardCtx.ui.input(
|
|
230
|
-
"Step 1/
|
|
255
|
+
"Step 1/5 — Max idle cutoff (now " + formatDuration(config.maxIdleMs) + ")\n" +
|
|
231
256
|
"Stop probing once you have been idle longer than this, so a session left overnight " +
|
|
232
257
|
"does not keep spending quota. Examples: 30m, 1h, 2h — or 0 to never stop.\n" +
|
|
233
258
|
"Leave empty / press Esc to keep the default (30m).",
|
|
@@ -246,9 +271,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
246
271
|
}
|
|
247
272
|
}
|
|
248
273
|
|
|
249
|
-
// 2/
|
|
274
|
+
// 2/5 — miss pause threshold
|
|
250
275
|
const missRaw = await wizardCtx.ui.input(
|
|
251
|
-
"Step 2/
|
|
276
|
+
"Step 2/5 — Miss pause threshold (now " + config.maxMissStreak + ")\n" +
|
|
252
277
|
"Pause probing after this many consecutive probes that did NOT hit the prompt cache " +
|
|
253
278
|
"(a cache hit resets the count). A high cache-read price with no hits means the " +
|
|
254
279
|
"provider's caching behaviour changed; pausing keeps you from burning quota blindly.",
|
|
@@ -263,9 +288,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
263
288
|
}
|
|
264
289
|
}
|
|
265
290
|
|
|
266
|
-
// 3/
|
|
291
|
+
// 3/5 — error circuit breaker
|
|
267
292
|
const errorRaw = await wizardCtx.ui.input(
|
|
268
|
-
"Step 3/
|
|
293
|
+
"Step 3/5 — Error circuit breaker (default " + config.maxErrorStreak + ")\n" +
|
|
269
294
|
"Pause probing after this many consecutive failed probes (network errors, HTTP 5xx). " +
|
|
270
295
|
"Auth failures (HTTP 401/403) always pause immediately regardless of this value.",
|
|
271
296
|
String(config.maxErrorStreak),
|
|
@@ -279,9 +304,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
279
304
|
}
|
|
280
305
|
}
|
|
281
306
|
|
|
282
|
-
// 4/
|
|
307
|
+
// 4/5 — spend cap
|
|
283
308
|
const spendRaw = await wizardCtx.ui.input(
|
|
284
|
-
"Step 4/
|
|
309
|
+
"Step 4/5 — Session spend cap in USD (default " +
|
|
285
310
|
formatUsd(DEFAULT_CONFIG.spendCapUsd ?? 1.0) +
|
|
286
311
|
")\n" +
|
|
287
312
|
"Ceiling on the estimated USD cost of probes in this session. A probe costs roughly " +
|
|
@@ -298,6 +323,27 @@ export default function (pi: ExtensionAPI) {
|
|
|
298
323
|
}
|
|
299
324
|
}
|
|
300
325
|
|
|
326
|
+
// 5/5 — probing mode
|
|
327
|
+
const modeRaw = await wizardCtx.ui.input(
|
|
328
|
+
"Step 5/5 — Probing mode (now " + config.mode + ")\n" +
|
|
329
|
+
"default: probes run at the fixed interval above.\n" +
|
|
330
|
+
"smart: starts at 8m; after 3 consecutive hits (context ≤ 200k) the cadence grows by 30s; " +
|
|
331
|
+
"one miss steps back to the last confirmed value and parks probing until you re-select smart mode — " +
|
|
332
|
+
"it self-tunes toward the real cache TTL to minimize probe spend.\n" +
|
|
333
|
+
"Type smart to enable; leave empty / press Esc for default.",
|
|
334
|
+
"",
|
|
335
|
+
);
|
|
336
|
+
if (modeRaw !== undefined) {
|
|
337
|
+
const v = modeRaw.trim().toLowerCase();
|
|
338
|
+
if (v === "smart" || v === "default") {
|
|
339
|
+
config.mode = v;
|
|
340
|
+
smartPaused = false;
|
|
341
|
+
if (v === "smart" && config.intervalMs < SMART_BASE_MS) config.intervalMs = SMART_BASE_MS;
|
|
342
|
+
} else if (v !== "") {
|
|
343
|
+
notify(`Unknown mode "${modeRaw}" — keeping ${config.mode}`, "error");
|
|
344
|
+
}
|
|
345
|
+
}
|
|
346
|
+
|
|
301
347
|
config.initialized = true;
|
|
302
348
|
persistConfig();
|
|
303
349
|
|
|
@@ -314,6 +360,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
314
360
|
persistConfig();
|
|
315
361
|
notify(
|
|
316
362
|
"pi-kimi-keepalive configured:\n" +
|
|
363
|
+
` mode: ${config.mode}${config.mode === "smart" ? ` (starting at ${formatDuration(config.intervalMs)})` : ` (fixed ${formatDuration(config.intervalMs)})`}\n` +
|
|
317
364
|
` maxidle: ${config.maxIdleMs === 0 ? "never stop" : formatDuration(config.maxIdleMs)}\n` +
|
|
318
365
|
` miss pause: after ${config.maxMissStreak} consecutive cache misses\n` +
|
|
319
366
|
` error breaker: ${config.maxErrorStreak} consecutive failures\n` +
|
|
@@ -507,6 +554,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
507
554
|
if (!capture) return;
|
|
508
555
|
stats.probes += 1;
|
|
509
556
|
stats.spendUsd += estimateProbeSpendUsd(usage, capture.cost);
|
|
557
|
+
lastProbeInputTokens = usage.inputTokens;
|
|
510
558
|
|
|
511
559
|
if (!isCacheMiss(usage.inputTokens, usage.cacheReadTokens, config.minPromptTokens)) {
|
|
512
560
|
stats.hits += 1;
|
|
@@ -516,12 +564,34 @@ export default function (pi: ExtensionAPI) {
|
|
|
516
564
|
debug(
|
|
517
565
|
`probe hit: cache_read=${usage.cacheReadTokens} input=${usage.inputTokens} saved=${stats.savedUsd.toFixed(4)}`,
|
|
518
566
|
);
|
|
567
|
+
if (config.mode === "smart") {
|
|
568
|
+
smartAdaptAfterHit(usage.inputTokens);
|
|
569
|
+
}
|
|
519
570
|
} else {
|
|
520
571
|
stats.misses += 1;
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
572
|
+
debug(`probe miss: cache_read=0 input=${usage.inputTokens}`);
|
|
573
|
+
if (config.mode === "smart") {
|
|
574
|
+
smartAdaptAfterMiss();
|
|
575
|
+
} else if (config.intervalMs > DEFAULT_FALLBACK_MS) {
|
|
576
|
+
// Miss while probing above the safe floor: the miss probe itself
|
|
577
|
+
// rebuilds the cache entry with the same prefix, so drop to the 5m
|
|
578
|
+
// safe cadence (inside the nominal TTL) and keep probing — the next
|
|
579
|
+
// probe renews it. Reset the streak so the new cadence gets a fresh
|
|
580
|
+
// chance before a pause is considered.
|
|
581
|
+
config.intervalMs = DEFAULT_FALLBACK_MS;
|
|
582
|
+
missStreak = 0;
|
|
583
|
+
persistConfig();
|
|
584
|
+
notify(
|
|
585
|
+
`cache miss — cadence backed off to ${formatDuration(config.intervalMs)} (safe TTL window); probing continues`,
|
|
586
|
+
"info",
|
|
587
|
+
);
|
|
588
|
+
updateUi();
|
|
589
|
+
} else {
|
|
590
|
+
missStreak += 1;
|
|
591
|
+
debug(`probe miss #${missStreak}: cache_read=0 input=${usage.inputTokens}`);
|
|
592
|
+
if (missStreak >= config.maxMissStreak) {
|
|
593
|
+
pause("probes stopped hitting the prefix cache; waiting for your next real turn");
|
|
594
|
+
}
|
|
525
595
|
}
|
|
526
596
|
}
|
|
527
597
|
|
|
@@ -536,6 +606,65 @@ export default function (pi: ExtensionAPI) {
|
|
|
536
606
|
}
|
|
537
607
|
}
|
|
538
608
|
|
|
609
|
+
/**
|
|
610
|
+
* smart-mode cadence adaptation.
|
|
611
|
+
*
|
|
612
|
+
* config.intervalMs doubles as the persisted "last confirmed cadence":
|
|
613
|
+
* promotion only ever happens after SMART_CONFIRM_HITS consecutive hits, so
|
|
614
|
+
* the value on disk is always one the cache has actually held for. A miss
|
|
615
|
+
* steps back 30s to that last confirmed value (never below the 8m floor),
|
|
616
|
+
* persists it, and pauses probing — a fresh real turn does NOT resume;
|
|
617
|
+
* only re-selecting smart mode (`/keepalive mode=smart`) continues.
|
|
618
|
+
* Contexts above 200k tokens are never pushed upward and immediately revert
|
|
619
|
+
* to the floor cadence, because a miss there costs too much full-price
|
|
620
|
+
* input to risk.
|
|
621
|
+
*/
|
|
622
|
+
function smartAdaptAfterHit(inputTokens: number): void {
|
|
623
|
+
if (inputTokens > SMART_MAX_CONTEXT_TOKENS) {
|
|
624
|
+
// Too expensive to experiment; drop to floor and stop growing.
|
|
625
|
+
if (config.intervalMs > SMART_BASE_MS) {
|
|
626
|
+
config.intervalMs = SMART_BASE_MS;
|
|
627
|
+
persistConfig();
|
|
628
|
+
notify(
|
|
629
|
+
`smart: context ${inputTokens} tokens exceeds ${SMART_MAX_CONTEXT_TOKENS / 1000}k — cadence back to the 8m floor`,
|
|
630
|
+
"info",
|
|
631
|
+
);
|
|
632
|
+
}
|
|
633
|
+
smartHitStreak = 0;
|
|
634
|
+
updateUi();
|
|
635
|
+
return;
|
|
636
|
+
}
|
|
637
|
+
smartHitStreak += 1;
|
|
638
|
+
if (smartHitStreak < SMART_CONFIRM_HITS) return;
|
|
639
|
+
const roomUnderMaxIdle =
|
|
640
|
+
config.maxIdleMs === 0 || config.intervalMs + SMART_STEP_MS < config.maxIdleMs;
|
|
641
|
+
if (roomUnderMaxIdle) {
|
|
642
|
+
config.intervalMs += SMART_STEP_MS; // 3-hit-confirmed value stays on disk
|
|
643
|
+
persistConfig();
|
|
644
|
+
debug(
|
|
645
|
+
`smart: ${smartHitStreak} consecutive hits — cadence grows to ${formatDuration(config.intervalMs)}`,
|
|
646
|
+
);
|
|
647
|
+
}
|
|
648
|
+
smartHitStreak = 0;
|
|
649
|
+
updateUi();
|
|
650
|
+
}
|
|
651
|
+
|
|
652
|
+
function smartAdaptAfterMiss(): void {
|
|
653
|
+
smartHitStreak = 0;
|
|
654
|
+
const fellBack = config.intervalMs > SMART_BASE_MS;
|
|
655
|
+
if (fellBack) {
|
|
656
|
+
// Step back to the last confirmed cadence before pausing.
|
|
657
|
+
config.intervalMs = Math.max(SMART_BASE_MS, config.intervalMs - SMART_STEP_MS);
|
|
658
|
+
persistConfig();
|
|
659
|
+
}
|
|
660
|
+
smartPaused = true;
|
|
661
|
+
pause(
|
|
662
|
+
`cache miss in smart mode — cadence ${fellBack ? `back to ${formatDuration(config.intervalMs)}` : "already at the floor"}; ` +
|
|
663
|
+
`probing stopped, run /keepalive mode=smart to resume`,
|
|
664
|
+
);
|
|
665
|
+
updateUi();
|
|
666
|
+
}
|
|
667
|
+
|
|
539
668
|
function recordFailure(message: string): void {
|
|
540
669
|
stats.errors += 1;
|
|
541
670
|
errorStreak += 1;
|
|
@@ -600,6 +729,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
600
729
|
"pi-kimi-keepalive",
|
|
601
730
|
` state: ${config.enabled ? "on" : "off"}${pausedReason ? ` (paused: ${pausedReason})` : ""}`,
|
|
602
731
|
` capture: ${route}`,
|
|
732
|
+
` mode: ${config.mode}${config.mode === "smart" ? ` · cadence ${formatDuration(config.intervalMs)}${lastProbeInputTokens > SMART_MAX_CONTEXT_TOKENS ? ` · context ${lastProbeInputTokens.toLocaleString()} > cap, frozen at floor` : ""}` : " (fixed via interval=)"}`,
|
|
603
733
|
` interval: ${formatDuration(config.intervalMs)} · maxidle ${config.maxIdleMs === 0 ? "off" : formatDuration(config.maxIdleMs)} · minPromptTokens ${config.minPromptTokens} · maxOutput ${config.maxOutputTokens}`,
|
|
604
734
|
` spend cap: ${config.spendCapUsd === null ? "none" : formatUsd(config.spendCapUsd)} · est. probe spend ${formatUsd(stats.spendUsd)}`,
|
|
605
735
|
` probes: ${stats.probes} (hits ${stats.hits}, misses ${stats.misses}, errors ${stats.errors})`,
|
|
@@ -653,12 +783,44 @@ export default function (pi: ExtensionAPI) {
|
|
|
653
783
|
}
|
|
654
784
|
break;
|
|
655
785
|
case "resume":
|
|
786
|
+
if (smartPaused) {
|
|
787
|
+
notify("smart-mode pause is intentional — run /keepalive mode=smart to resume probing", "warning");
|
|
788
|
+
break;
|
|
789
|
+
}
|
|
656
790
|
pausedReason = null;
|
|
657
791
|
missStreak = 0;
|
|
658
792
|
errorStreak = 0;
|
|
793
|
+
smartHitStreak = 0;
|
|
659
794
|
notify("keepalive resumed", "info");
|
|
660
795
|
break;
|
|
796
|
+
case "mode": {
|
|
797
|
+
const next = value === "smart" || value === "default" ? value : null;
|
|
798
|
+
if (next === null) {
|
|
799
|
+
notify("usage: /keepalive mode=smart (adaptive) or mode=default (fixed)", "error");
|
|
800
|
+
break;
|
|
801
|
+
}
|
|
802
|
+
config.mode = next;
|
|
803
|
+
smartHitStreak = 0;
|
|
804
|
+
smartPaused = false; // re-selecting the mode is the documented way out of a smart miss-pause
|
|
805
|
+
if (next === "smart") {
|
|
806
|
+
// smart manages the cadence itself; snap back to its floor.
|
|
807
|
+
if (config.intervalMs < SMART_BASE_MS) config.intervalMs = SMART_BASE_MS;
|
|
808
|
+
notify(
|
|
809
|
+
`mode=smart — probing resumes at ${formatDuration(config.intervalMs)}; +30s per ${SMART_CONFIRM_HITS} hits (context ≤ ${SMART_MAX_CONTEXT_TOKENS / 1000}k), one miss parks probing`,
|
|
810
|
+
"info",
|
|
811
|
+
);
|
|
812
|
+
} else {
|
|
813
|
+
notify("mode=default — fixed cadence via /keepalive interval=<duration>", "info");
|
|
814
|
+
}
|
|
815
|
+
persistConfig();
|
|
816
|
+
updateUi();
|
|
817
|
+
break;
|
|
818
|
+
}
|
|
661
819
|
case "interval": {
|
|
820
|
+
if (config.mode === "smart") {
|
|
821
|
+
notify("smart mode manages the cadence itself — use /keepalive mode=default to set it manually", "warning");
|
|
822
|
+
break;
|
|
823
|
+
}
|
|
662
824
|
const ms = value !== undefined ? parseDurationMs(value) : null;
|
|
663
825
|
if (ms === null || ms < MIN_INTERVAL_MS) {
|
|
664
826
|
notify(`/keepalive interval=${value ?? "?"} rejected — minimum 30s, e.g. interval=4m`, "error");
|
|
@@ -791,12 +953,23 @@ export default function (pi: ExtensionAPI) {
|
|
|
791
953
|
cost: (model.cost ?? undefined) as Partial<CostPerM> | undefined,
|
|
792
954
|
};
|
|
793
955
|
// A fresh real request means fresh credentials and a warm prefix cache;
|
|
794
|
-
// automatically recover from any sticky pause.
|
|
956
|
+
// automatically recover from any sticky pause. Exception: a smart-mode
|
|
957
|
+
// miss intentionally parks probing until the user re-selects smart mode.
|
|
795
958
|
if (pausedReason !== null || missStreak > 0 || errorStreak > 0) {
|
|
796
|
-
|
|
959
|
+
if (smartPaused) {
|
|
960
|
+
if (pausedReason !== null) {
|
|
961
|
+
debug("fresh real request observed — smart-mode miss pause kept; /keepalive mode=smart to resume");
|
|
962
|
+
missStreak = 0;
|
|
963
|
+
errorStreak = 0;
|
|
964
|
+
updateUi();
|
|
965
|
+
return;
|
|
966
|
+
}
|
|
967
|
+
} else {
|
|
968
|
+
pausedReason = null;
|
|
969
|
+
debug("fresh real request observed — keepalive unpaused");
|
|
970
|
+
}
|
|
797
971
|
missStreak = 0;
|
|
798
972
|
errorStreak = 0;
|
|
799
|
-
debug("fresh real request observed — keepalive unpaused");
|
|
800
973
|
}
|
|
801
974
|
updateUi();
|
|
802
975
|
});
|