pi-ptc-meter 1.0.31 → 1.0.33

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +53 -46
  2. package/index.ts +0 -37
  3. package/package.json +1 -1
package/README.md CHANGED
@@ -9,7 +9,7 @@
9
9
 
10
10
  > 适用 pi · Node ≥ 22.19 · MIT · 无运行时依赖(仅 peer: @earendil-works/pi-coding-agent)
11
11
 
12
- 每轮工具调用计量:记录调用次数与耗时,与开工前声明的估算对比后,把实际值与偏差回灌给模型。
12
+ 每轮工具调用计量:记录调用次数与耗时,回灌实际值与会话中位。不做预估、不算偏差(1.0.32 起两处预估均已移除)。
13
13
 
14
14
  ## 快速事实
15
15
 
@@ -17,7 +17,7 @@
17
17
  | --- | --- |
18
18
  | 配置文件 | ~/.pi/ptc-meter.json(可省略) |
19
19
  | 命令行 | /ptc [n](默认 20) |
20
- | 调用方式 | **全自动**:计数与回灌无需触发;估算由模型在回复中自行声明(如「预计 40 次」),扩展只负责解析与对比 |
20
+ | 调用方式 | **全自动**:每轮计数与回灌无需触发;只回灌实际值与中位,不收集预估(1.0.32 起调用数/耗时两处预估均已删除——实测中位误差 94% / 199%) |
21
21
 
22
22
  完整配置(全部键与默认值):
23
23
 
@@ -52,60 +52,63 @@ pi install npm:pi-ptc-meter
52
52
 
53
53
  | 命令 | 作用 |
54
54
  | --- | --- |
55
- | `/ptc [n]` | 最近 n 轮:次数、时长、预计、中位、超标 |
55
+ | `/ptc [n]` | 最近 n 轮:次数、时长、中位、超标 |
56
56
 
57
+ ## 模拟测试
58
+
59
+ pi-ptc-meter 的改动在**隔离副本**上做回放验证后才发版;当前版本通过 **10,011 条断言**。
60
+
61
+ | 测试面 | 规模 |
62
+ | --- | --- |
63
+ | 真实日志回放 | 4,349 轮 |
64
+ | 随机状态 | 10,000 条 |
65
+ | 统计函数边界 | 11 项 |
66
+
67
+ 改动:删调用数预估 + 耗时预估。依据:预估在声明点不可预测:412 个声明点上 6 种预测器全部 71–94% 误差;耗时中位误差 199%。
57
68
  ## 作用(源码事实)
58
69
 
59
70
  - `tool_call`:顶层调用计数 +1(codemode 内层调用不另计;不拦截、不改调用)。
60
- - `turn_end`:记录本轮次数与耗时(不落盘)。
61
- - `agent_end`:每 run 落盘一行日志;goal 自动续轮共享同一 run 时,`turn_end` 超阈值即提示一次(每 run 一次),避免回路整段失效。
62
- - `before_agent_start`:结算上一轮并以 `nextTurn` 尾部消息回灌——计数、耗时、偏差、基线中位、**缓存命中率**(仅当该轮有 usage 数据时出现;< 90% 标「低」)——(`display: false`)。
71
+ - `turn_end`:记录本轮次数与耗时(不落盘);超阈值时每 run 提示一次(goal 自动续轮共享同一 run 时 `agent_end` 可能整段不触发,避免回路失效)。
72
+ - `agent_end`:每 run 落盘一行日志。
73
+ - `before_agent_start`:结算上一轮并以 `nextTurn` 尾部消息回灌——计数、耗时、**缓存命中率**(仅当该轮有 usage 数据时出现;< 90% 标「低」)、会话中位、阈值判定(`display: false`)。
74
+ - **1.0.32 起不再回灌「预计 / 实际 / 偏差」**:声明率虽升到 65%,中位误差仍为 94%(调用数)与 199%(耗时),且 412 个声明点上 6 种预测器(常量 12/16、1.1× 上轮、2.0× 上轮、全局中位、区间 3~40)全部落在 71–94% 误差。
63
75
 
64
76
  **注入样例(示意,字段可能随版本增删)**
65
77
 
66
78
  ```text
67
- PTC 计量:上一轮 9 次工具调用 · 实际 34s · 命中 92% · 预计 8 次 / 实际 9 次(偏差 +13%)
68
- · 会话中位 25 · 阈值 50 · 判据 = 本轮计数不超阈值且无同类调用可并
79
+ PTC 计量:上一轮 9 次工具调用 · 实际 34s · 命中 92% · 会话中位 25 · 阈值 50 · 判据 = 本轮计数不超阈值且无同类调用可并
69
80
  ```
70
81
  - `session_start`:模块级计量归零。
71
82
  - system 内 section `ptc_meter` 只含逐字不变的骨架。
72
-
73
83
  ## 原理(专业)
74
84
 
75
85
  ```text
76
- 本轮开工前 ─ 声明「预计 N 次 / 预计 Ns」
77
- ↓
78
86
  tool_call ─ 仅顶层调用计数 +1
79
87
  ↓
80
- turn_end ─ 记录实际次数与耗时 + 解析预计值
88
+ turn_end ─ 记录实际次数与耗时
81
89
  ↓
82
- agent_end ─ 落盘一行日志(含预计)
90
+ agent_end ─ 落盘一行日志
83
91
  ↓
84
92
  before_agent_start ─ 结算上一轮
85
93
  ↓
86
- 对比:实际 vs 预计 → 偏差 %
94
+ 回灌:实际次数 / 耗时 / 命中率 / 会话中位 / 阈值判定
87
95
  ↓ nextTurn 尾部消息
88
- (回到本轮开工前)
89
96
 
90
97
  system 只留静态骨架(ptc_meter section)
91
98
  ```
92
-
93
99
  ## 原理(小白)
94
100
 
95
- **像记账对账**:开工前先说「这趟大概 5 步」,收工后对账「实际 7 步,比预计多 40%」,下次开工前把账报给你。
101
+ **像仪表盘**:不看你预测,只报「刚跑了几步、这个会话通常几步」。计划由你定,仪表只给事实。
96
102
 
97
103
  ```text
98
- 开工前先估:预计几步
99
- ↓
100
104
  干活:实际跑几步
101
105
  ↓
102
- 收工对比:实际 vs 预计
106
+ 收工记账
103
107
  ↓
104
- 下次开工时报出偏差
108
+ 下次开工报:上一轮 7 步,会话中位 25
105
109
  ↓
106
- (回到开工前先估)
110
+ 你自己决定要不要合并调用
107
111
  ```
108
-
109
112
  ## License
110
113
 
111
114
  MIT
@@ -116,7 +119,7 @@ MIT
116
119
 
117
120
  <h1 align="center">🌐 English Version &nbsp;⬇️</h1>
118
121
 
119
- Per-turn tool-call metering: records calls and duration, compares them with the estimate declared before the turn, and feeds back the actuals and deviation.
122
+ Per-turn tool-call metering: records calls and duration and feeds back the actuals plus the session median. No estimates, no deviation (1.0.32 removed both).
120
123
 
121
124
  ## At a glance
122
125
 
@@ -124,7 +127,7 @@ Per-turn tool-call metering: records calls and duration, compares them with the
124
127
  | --- | --- |
125
128
  | Config file | ~/.pi/ptc-meter.json (optional) |
126
129
  | Commands | /ptc [n] (default 20) |
127
- | Automation | **Fully automatic**: metering and feedback need no trigger. The estimate is declared by the model in its reply (e.g. "about 40 calls"); the extension only parses and compares |
130
+ | Automation | **Fully automatic**: metering and feedback need no trigger. Only the actual values and the session median are fed back; no estimates are collected (1.0.32 removed both the call-count and duration estimates - measured median errors were 94% and 199%) |
128
131
 
129
132
  Full config (all keys and defaults):
130
133
 
@@ -159,58 +162,62 @@ Also: log `~/.pi/ptc-meter.jsonl` (1 MiB truncation, tail-preserving); no enviro
159
162
 
160
163
  | Command | Purpose |
161
164
  | --- | --- |
162
- | `/ptc [n]` | Last n turns: counts, duration, estimates, median, over-threshold |
165
+ | `/ptc [n]` | Last n turns: counts, duration, median, over-threshold |
163
166
 
167
+ ## Verification
168
+
169
+ Changes to pi-ptc-meter were verified on an isolated copy before release; the current version passes **10,011 assertions**.
170
+
171
+ | Surface | Scale |
172
+ | --- | --- |
173
+ | Real log replay | 4,349 turns |
174
+ | Random states | 10,000 |
175
+ | Statistic edges | 11 |
176
+
177
+ Change: both estimates removed. Why: estimates are unpredictable at declaration points: six predictors all landed at 71-94% error; duration median error 199%.
164
178
  ## What it does (source facts)
165
179
 
166
180
  - `tool_call`: increments the counter for top-level calls only (nested codemode calls are not counted separately; never intercepts or rewrites).
167
- - `turn_end`: records calls and duration (no disk write).
168
- - `agent_end`: writes one log line per run.; when goal auto-continuation shares one run, `turn_end` warns once above threshold (once per run) so the loop never goes dark.
169
- - `before_agent_start`: settles the previous turn and feeds back counts, duration, deviation, the baseline median and the **cache hit rate** (only when that turn reported usage; flagged low below 90%) as a `nextTurn` trailing message (`display: false`).
181
+ - `turn_end`: records calls and duration (no disk write); when goal auto-continuation shares one run and `agent_end` never fires, it warns once above threshold (once per run).
182
+ - `agent_end`: writes one log line per run.
183
+ - `before_agent_start`: settles the previous turn and feeds back counts, duration, the **cache hit rate** (only when that turn reported usage; flagged low below 90%), the session median and the threshold verdict as a `nextTurn` trailing message (`display: false`).
184
+ - **1.0.32 no longer feeds back estimate / actual / deviation**: declaration rate rose to 65% but the median error stayed at 94% (call count) and 199% (duration); on 412 declaration points all six predictors (constant 12/16, 1.1x previous, 2.0x previous, global median, interval 3-40) landed between 71% and 94% error.
170
185
 
171
186
  **Sample injection (illustrative; fields may change between versions)**
172
187
 
173
188
  ```text
174
- PTC metering: last turn 9 tool calls · 34s actual · 92% cache hit · estimated 8 / actual 9 (+13%)
175
- · session median 25 · threshold 50 · verdict: under threshold, no obvious merge candidates
189
+ PTC metering: last turn 9 tool calls · 34s actual · 92% cache hit · session median 25 · threshold 50 · verdict: under threshold, no obvious merge candidates
176
190
  ```
177
191
  - `session_start`: resets module-level counters.
178
192
  - The `ptc_meter` system section holds only a byte-stable skeleton.
179
-
180
193
  ## How it works (technical)
181
194
 
182
195
  ```text
183
- before the turn ─ declare: about N calls / Ns
184
- ↓
185
196
  tool_call ─ top-level call: counter +1
186
197
  ↓
187
- turn_end ─ record actual calls and duration + parse estimate
198
+ turn_end ─ record actual calls and duration
188
199
  ↓
189
- agent_end ─ write one log line (includes estimate)
200
+ agent_end ─ write one log line
190
201
  ↓
191
202
  before_agent_start ─ settle the previous turn
192
203
  ↓
193
- compare: actual vs estimate -> deviation %
204
+ feed back: calls / duration / cache hit / session median / threshold verdict
194
205
  ↓ nextTurn message
195
- (back to before the turn)
196
206
 
197
207
  the system prompt keeps only a static skeleton (ptc_meter section)
198
208
  ```
199
-
200
209
  ## How it works (plain words)
201
210
 
202
- **Like a ledger**: you estimate "about 5 steps" before starting, the ledger compares it with the actual 7 (40% over), and reports the gap before the next job.
211
+ **Like a dashboard**: it does not care what you predicted; it just reports how many steps the last turn took and what this session usually takes. You decide the plan; the dashboard gives facts.
203
212
 
204
213
  ```text
205
- before starting: guess N steps
206
- ↓
207
- work: actually N steps
214
+ work: N actual steps
208
215
  ↓
209
- at the end: actual vs guess
216
+ log it
210
217
  ↓
211
- report the gap next time
218
+ next turn reports: last turn 7 calls, session median 25
212
219
  ↓
213
- (back to before starting)
220
+ you decide whether to merge calls
214
221
  ```
215
222
  ## License
216
223
 
package/index.ts CHANGED
@@ -200,46 +200,9 @@ function statsLine(th: number, med: number, rows: Row[]): string {
200
200
  if (secs !== null) {
201
201
  parts.push(`实际 ${secs}s`);
202
202
  if (lastTurnHit !== null && lastTurnHit < 90) parts.push(`命中 ${lastTurnHit}%(低)`);
203
- if (lastTurnEstCalls !== null && lastTurnEstCalls > 0) {
204
- const dc = Math.round(((last - lastTurnEstCalls) / lastTurnEstCalls) * 100);
205
- if (Math.abs(dc) <= 10) parts.push(`预计 ${lastTurnEstCalls} / 实际 ${last} 次(基本吻合)`);
206
- else parts.push(`预计 ${lastTurnEstCalls} 次 / 实际 ${last} 次(偏差 ${dc >= 0 ? "+" : ""}${dc}%)`);
207
- }
208
- if (lastTurnEst !== null) {
209
- const dev = Math.round(((lastTurnMs / 1000 - lastTurnEst) / lastTurnEst) * 100);
210
- parts.push(`预计 ${lastTurnEst}s / 实际 ${secs}s(偏差 ${dev >= 0 ? "+" : ""}${dev}%)`);
211
- } else {
212
- const base = Math.round(median(rows.map((r) => r.ms)) / 1000);
213
- // 借鉴 think-anchor 的自适应:声明率已高时不再每轮催(实测 88.9% 声明率下这句话是纯噪音)
214
- const declRate = rows.length ? rows.filter((r) => typeof r.estCalls === "number" && (r.estCalls as number) > 0).length / rows.length : 0;
215
- const nudge = declRate < 0.6 ? " → 本轮开工前先报「预计 N 次」" : "";
216
- parts.push((base > 0 ? "预计未声明(同类中位 " + base + "s)" : "预计未声明") + nudge);
217
- }
218
203
  }
219
204
  // 校正指引(回路自动纠偏):近若干轮系统性偏离时,直接给出本轮该报的倍数——不靠自觉。
220
- // 调用数校正:次数由任务类型决定,方差远小于时长——优先用它给指引。
221
205
  try {
222
- const ecRows = rows.filter((r) => typeof r.estCalls === "number" && (r.estCalls as number) > 0);
223
- if (ecRows.length >= 3) {
224
- const dcs = ecRows.map((r) => ((r.calls - (r.estCalls as number)) / (r.estCalls as number)) * 100);
225
- const mdc = median(dcs);
226
- if (mdc <= -20 || mdc >= 20) {
227
- const f = Math.min(3, Math.max(0.2, Math.round((1 + mdc / 100) * 100) / 100)); // 夹在 0.2–3:极端中位(声明失真)不放大成荒谬倍数
228
- const lastEst = ecRows[ecRows.length - 1].estCalls as number;
229
- parts.push(`近 ${ecRows.length} 轮调用数系统性${mdc < 0 ? "高估" : "低估"}(中位 ${mdc > 0 ? "+" : ""}${Math.round(mdc)}%)→ 本轮按约 ${f} 倍报(≈ ${Math.max(1, Math.round(lastEst * f))} 次)`);
230
- }
231
- }
232
- } catch { /* 静默 */ }
233
- try {
234
- const estRows = rows.filter((r) => typeof r.est === "number" && (r.est as number) > 0);
235
- if (estRows.length >= 3) {
236
- const devs = estRows.map((r) => ((r.ms / 1000) - (r.est as number)) / (r.est as number) * 100);
237
- const md = median(devs);
238
- if (md <= -20 || md >= 20) {
239
- const factor = Math.min(3, Math.max(0.2, Math.round((1 + md / 100) * 100) / 100));
240
- parts.push(`近 ${estRows.length} 轮耗时系统性${md < 0 ? "高估" : "低估"}(中位 ${md > 0 ? "+" : ""}${Math.round(md)}%)→ 本轮按约 ${factor} 倍报`);
241
- }
242
- }
243
206
  } catch { /* 指引失败不影响统计行 */ }
244
207
  const verdict = last >= th
245
208
  ? `超阈值 ${th} · 判据 = 本轮计数不超阈值且无同类调用可并(可并性人工复核)`
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-ptc-meter",
3
- "version": "1.0.31",
3
+ "version": "1.0.33",
4
4
  "description": "Measures tool-call batching per turn and feeds the numbers back to the model.",
5
5
  "type": "module",
6
6
  "license": "MIT",