pi-ptc-meter 1.0.32 → 1.0.34

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +58 -46
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -9,7 +9,7 @@
9
9
 
10
10
  > 适用 pi · Node ≥ 22.19 · MIT · 无运行时依赖(仅 peer: @earendil-works/pi-coding-agent)
11
11
 
12
- 每轮工具调用计量:记录调用次数与耗时,与开工前声明的估算对比后,把实际值与偏差回灌给模型。
12
+ 每轮工具调用计量:记录调用次数与耗时,回灌实际值与会话中位。不做预估、不算偏差(1.0.32 起两处预估均已移除)。
13
13
 
14
14
  ## 快速事实
15
15
 
@@ -17,7 +17,7 @@
17
17
  | --- | --- |
18
18
  | 配置文件 | ~/.pi/ptc-meter.json(可省略) |
19
19
  | 命令行 | /ptc [n](默认 20) |
20
- | 调用方式 | **全自动**:计数与回灌无需触发;估算由模型在回复中自行声明(如「预计 40 次」),扩展只负责解析与对比 |
20
+ | 调用方式 | **全自动**:每轮计数与回灌无需触发;只回灌实际值与中位,不收集预估(1.0.32 起调用数/耗时两处预估均已删除——实测中位误差 94% / 199%) |
21
21
 
22
22
  完整配置(全部键与默认值):
23
23
 
@@ -52,60 +52,66 @@ pi install npm:pi-ptc-meter
52
52
 
53
53
  | 命令 | 作用 |
54
54
  | --- | --- |
55
- | `/ptc [n]` | 最近 n 轮:次数、时长、预计、中位、超标 |
55
+ | `/ptc [n]` | 最近 n 轮:次数、时长、中位、超标 |
56
+
57
+ ## 模拟测试
58
+
59
+ pi-ptc-meter 的改动在**隔离副本**上做回放验证后才发版;当前版本通过 **10,011 条断言**。
60
+
61
+ | 测试面 | 规模 | 结果 |
62
+ | --- | --- | --- |
63
+ | 真实日志回放 | 4,349 轮 | 🟢 |
64
+ | 随机状态 | 10,000 条 | 🟢 |
65
+ | 统计函数边界 | 11 项 | 🟢 |
66
+
67
+ 🟢 **结论**:预估功能已删除——412 个声明点上 6 种预测器全部落在 71–94% 误差,预估恰好在它被需要的那一刻不可预测。
68
+
69
+ 改动:删调用数预估 + 耗时预估。依据:预估在声明点不可预测:412 个声明点上 6 种预测器全部 71–94% 误差;耗时中位误差 199%。
56
70
 
57
71
  ## 作用(源码事实)
58
72
 
59
73
  - `tool_call`:顶层调用计数 +1(codemode 内层调用不另计;不拦截、不改调用)。
60
- - `turn_end`:记录本轮次数与耗时(不落盘)。
61
- - `agent_end`:每 run 落盘一行日志;goal 自动续轮共享同一 run 时,`turn_end` 超阈值即提示一次(每 run 一次),避免回路整段失效。
62
- - `before_agent_start`:结算上一轮并以 `nextTurn` 尾部消息回灌——计数、耗时、偏差、基线中位、**缓存命中率**(仅当该轮有 usage 数据时出现;< 90% 标「低」)——(`display: false`)。
74
+ - `turn_end`:记录本轮次数与耗时(不落盘);超阈值时每 run 提示一次(goal 自动续轮共享同一 run 时 `agent_end` 可能整段不触发,避免回路失效)。
75
+ - `agent_end`:每 run 落盘一行日志。
76
+ - `before_agent_start`:结算上一轮并以 `nextTurn` 尾部消息回灌——计数、耗时、**缓存命中率**(仅当该轮有 usage 数据时出现;< 90% 标「低」)、会话中位、阈值判定(`display: false`)。
77
+ - **1.0.32 起不再回灌「预计 / 实际 / 偏差」**:声明率虽升到 65%,中位误差仍为 94%(调用数)与 199%(耗时),且 412 个声明点上 6 种预测器(常量 12/16、1.1× 上轮、2.0× 上轮、全局中位、区间 3~40)全部落在 71–94% 误差。
63
78
 
64
79
  **注入样例(示意,字段可能随版本增删)**
65
80
 
66
81
  ```text
67
- PTC 计量:上一轮 9 次工具调用 · 实际 34s · 命中 92% · 预计 8 次 / 实际 9 次(偏差 +13%)
68
- · 会话中位 25 · 阈值 50 · 判据 = 本轮计数不超阈值且无同类调用可并
82
+ PTC 计量:上一轮 9 次工具调用 · 实际 34s · 命中 92% · 会话中位 25 · 阈值 50 · 判据 = 本轮计数不超阈值且无同类调用可并
69
83
  ```
70
84
  - `session_start`:模块级计量归零。
71
85
  - system 内 section `ptc_meter` 只含逐字不变的骨架。
72
-
73
86
  ## 原理(专业)
74
87
 
75
88
  ```text
76
- 本轮开工前 ─ 声明「预计 N 次 / 预计 Ns」
77
- ↓
78
89
  tool_call ─ 仅顶层调用计数 +1
79
90
  ↓
80
- turn_end ─ 记录实际次数与耗时 + 解析预计值
91
+ turn_end ─ 记录实际次数与耗时
81
92
  ↓
82
- agent_end ─ 落盘一行日志(含预计)
93
+ agent_end ─ 落盘一行日志
83
94
  ↓
84
95
  before_agent_start ─ 结算上一轮
85
96
  ↓
86
- 对比:实际 vs 预计 → 偏差 %
97
+ 回灌:实际次数 / 耗时 / 命中率 / 会话中位 / 阈值判定
87
98
  ↓ nextTurn 尾部消息
88
- (回到本轮开工前)
89
99
 
90
100
  system 只留静态骨架(ptc_meter section)
91
101
  ```
92
-
93
102
  ## 原理(小白)
94
103
 
95
- **像记账对账**:开工前先说「这趟大概 5 步」,收工后对账「实际 7 步,比预计多 40%」,下次开工前把账报给你。
104
+ **像仪表盘**:不看你预测,只报「刚跑了几步、这个会话通常几步」。计划由你定,仪表只给事实。
96
105
 
97
106
  ```text
98
- 开工前先估:预计几步
99
- ↓
100
107
  干活:实际跑几步
101
108
  ↓
102
- 收工对比:实际 vs 预计
109
+ 收工记账
103
110
  ↓
104
- 下次开工时报出偏差
111
+ 下次开工报:上一轮 7 步,会话中位 25
105
112
  ↓
106
- (回到开工前先估)
113
+ 你自己决定要不要合并调用
107
114
  ```
108
-
109
115
  ## License
110
116
 
111
117
  MIT
@@ -116,7 +122,7 @@ MIT
116
122
 
117
123
  <h1 align="center">🌐 English Version &nbsp;⬇️</h1>
118
124
 
119
- Per-turn tool-call metering: records calls and duration, compares them with the estimate declared before the turn, and feeds back the actuals and deviation.
125
+ Per-turn tool-call metering: records calls and duration and feeds back the actuals plus the session median. No estimates, no deviation (1.0.32 removed both).
120
126
 
121
127
  ## At a glance
122
128
 
@@ -124,7 +130,7 @@ Per-turn tool-call metering: records calls and duration, compares them with the
124
130
  | --- | --- |
125
131
  | Config file | ~/.pi/ptc-meter.json (optional) |
126
132
  | Commands | /ptc [n] (default 20) |
127
- | Automation | **Fully automatic**: metering and feedback need no trigger. The estimate is declared by the model in its reply (e.g. "about 40 calls"); the extension only parses and compares |
133
+ | Automation | **Fully automatic**: metering and feedback need no trigger. Only the actual values and the session median are fed back; no estimates are collected (1.0.32 removed both the call-count and duration estimates - measured median errors were 94% and 199%) |
128
134
 
129
135
  Full config (all keys and defaults):
130
136
 
@@ -159,58 +165,64 @@ Also: log `~/.pi/ptc-meter.jsonl` (1 MiB truncation, tail-preserving); no enviro
159
165
 
160
166
  | Command | Purpose |
161
167
  | --- | --- |
162
- | `/ptc [n]` | Last n turns: counts, duration, estimates, median, over-threshold |
168
+ | `/ptc [n]` | Last n turns: counts, duration, median, over-threshold |
169
+
170
+ ## Verification
171
+
172
+ Changes to pi-ptc-meter were verified on an isolated copy before release; the current version passes **10,011 assertions**.
173
+
174
+ | Surface | Scale | Result |
175
+ | --- | --- | --- |
176
+ | Real log replay | 4,349 turns | 🟢 |
177
+ | Random states | 10,000 | 🟢 |
178
+ | Statistic edges | 11 | 🟢 |
179
+
180
+ 🟢 **Result**: both estimates removed - on 412 declaration points all six predictors landed at 71-94% error, so estimating is unpredictable exactly when it is needed.
181
+ Change: both estimates removed. Why: estimates are unpredictable at declaration points: six predictors all landed at 71-94% error; duration median error 199%.
163
182
 
164
183
  ## What it does (source facts)
165
184
 
166
185
  - `tool_call`: increments the counter for top-level calls only (nested codemode calls are not counted separately; never intercepts or rewrites).
167
- - `turn_end`: records calls and duration (no disk write).
168
- - `agent_end`: writes one log line per run.; when goal auto-continuation shares one run, `turn_end` warns once above threshold (once per run) so the loop never goes dark.
169
- - `before_agent_start`: settles the previous turn and feeds back counts, duration, deviation, the baseline median and the **cache hit rate** (only when that turn reported usage; flagged low below 90%) as a `nextTurn` trailing message (`display: false`).
186
+ - `turn_end`: records calls and duration (no disk write); when goal auto-continuation shares one run and `agent_end` never fires, it warns once above threshold (once per run).
187
+ - `agent_end`: writes one log line per run.
188
+ - `before_agent_start`: settles the previous turn and feeds back counts, duration, the **cache hit rate** (only when that turn reported usage; flagged low below 90%), the session median and the threshold verdict as a `nextTurn` trailing message (`display: false`).
189
+ - **1.0.32 no longer feeds back estimate / actual / deviation**: declaration rate rose to 65% but the median error stayed at 94% (call count) and 199% (duration); on 412 declaration points all six predictors (constant 12/16, 1.1x previous, 2.0x previous, global median, interval 3-40) landed between 71% and 94% error.
170
190
 
171
191
  **Sample injection (illustrative; fields may change between versions)**
172
192
 
173
193
  ```text
174
- PTC metering: last turn 9 tool calls · 34s actual · 92% cache hit · estimated 8 / actual 9 (+13%)
175
- · session median 25 · threshold 50 · verdict: under threshold, no obvious merge candidates
194
+ PTC metering: last turn 9 tool calls · 34s actual · 92% cache hit · session median 25 · threshold 50 · verdict: under threshold, no obvious merge candidates
176
195
  ```
177
196
  - `session_start`: resets module-level counters.
178
197
  - The `ptc_meter` system section holds only a byte-stable skeleton.
179
-
180
198
  ## How it works (technical)
181
199
 
182
200
  ```text
183
- before the turn ─ declare: about N calls / Ns
184
- ↓
185
201
  tool_call ─ top-level call: counter +1
186
202
  ↓
187
- turn_end ─ record actual calls and duration + parse estimate
203
+ turn_end ─ record actual calls and duration
188
204
  ↓
189
- agent_end ─ write one log line (includes estimate)
205
+ agent_end ─ write one log line
190
206
  ↓
191
207
  before_agent_start ─ settle the previous turn
192
208
  ↓
193
- compare: actual vs estimate -> deviation %
209
+ feed back: calls / duration / cache hit / session median / threshold verdict
194
210
  ↓ nextTurn message
195
- (back to before the turn)
196
211
 
197
212
  the system prompt keeps only a static skeleton (ptc_meter section)
198
213
  ```
199
-
200
214
  ## How it works (plain words)
201
215
 
202
- **Like a ledger**: you estimate "about 5 steps" before starting, the ledger compares it with the actual 7 (40% over), and reports the gap before the next job.
216
+ **Like a dashboard**: it does not care what you predicted; it just reports how many steps the last turn took and what this session usually takes. You decide the plan; the dashboard gives facts.
203
217
 
204
218
  ```text
205
- before starting: guess N steps
206
- ↓
207
- work: actually N steps
219
+ work: N actual steps
208
220
  ↓
209
- at the end: actual vs guess
221
+ log it
210
222
  ↓
211
- report the gap next time
223
+ next turn reports: last turn 7 calls, session median 25
212
224
  ↓
213
- (back to before starting)
225
+ you decide whether to merge calls
214
226
  ```
215
227
  ## License
216
228
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-ptc-meter",
3
- "version": "1.0.32",
3
+ "version": "1.0.34",
4
4
  "description": "Measures tool-call batching per turn and feeds the numbers back to the model.",
5
5
  "type": "module",
6
6
  "license": "MIT",