pi-ptc-meter 1.0.32 → 1.0.34
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +58 -46
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
|
|
10
10
|
> 适用 pi · Node ≥ 22.19 · MIT · 无运行时依赖(仅 peer: @earendil-works/pi-coding-agent)
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
每轮工具调用计量:记录调用次数与耗时,回灌实际值与会话中位。不做预估、不算偏差(1.0.32 起两处预估均已移除)。
|
|
13
13
|
|
|
14
14
|
## 快速事实
|
|
15
15
|
|
|
@@ -17,7 +17,7 @@
|
|
|
17
17
|
| --- | --- |
|
|
18
18
|
| 配置文件 | ~/.pi/ptc-meter.json(可省略) |
|
|
19
19
|
| 命令行 | /ptc [n](默认 20) |
|
|
20
|
-
| 调用方式 |
|
|
20
|
+
| 调用方式 | **全自动**:每轮计数与回灌无需触发;只回灌实际值与中位,不收集预估(1.0.32 起调用数/耗时两处预估均已删除——实测中位误差 94% / 199%) |
|
|
21
21
|
|
|
22
22
|
完整配置(全部键与默认值):
|
|
23
23
|
|
|
@@ -52,60 +52,66 @@ pi install npm:pi-ptc-meter
|
|
|
52
52
|
|
|
53
53
|
| 命令 | 作用 |
|
|
54
54
|
| --- | --- |
|
|
55
|
-
| `/ptc [n]` | 最近 n
|
|
55
|
+
| `/ptc [n]` | 最近 n 轮:次数、时长、中位、超标 |
|
|
56
|
+
|
|
57
|
+
## 模拟测试
|
|
58
|
+
|
|
59
|
+
pi-ptc-meter 的改动在**隔离副本**上做回放验证后才发版;当前版本通过 **10,011 条断言**。
|
|
60
|
+
|
|
61
|
+
| 测试面 | 规模 | 结果 |
|
|
62
|
+
| --- | --- | --- |
|
|
63
|
+
| 真实日志回放 | 4,349 轮 | 🟢 |
|
|
64
|
+
| 随机状态 | 10,000 条 | 🟢 |
|
|
65
|
+
| 统计函数边界 | 11 项 | 🟢 |
|
|
66
|
+
|
|
67
|
+
🟢 **结论**:预估功能已删除——412 个声明点上 6 种预测器全部落在 71–94% 误差,预估恰好在它被需要的那一刻不可预测。
|
|
68
|
+
|
|
69
|
+
改动:删调用数预估 + 耗时预估。依据:预估在声明点不可预测:412 个声明点上 6 种预测器全部 71–94% 误差;耗时中位误差 199%。
|
|
56
70
|
|
|
57
71
|
## 作用(源码事实)
|
|
58
72
|
|
|
59
73
|
- `tool_call`:顶层调用计数 +1(codemode 内层调用不另计;不拦截、不改调用)。
|
|
60
|
-
- `turn_end
|
|
61
|
-
- `agent_end`:每 run
|
|
62
|
-
- `before_agent_start`:结算上一轮并以 `nextTurn`
|
|
74
|
+
- `turn_end`:记录本轮次数与耗时(不落盘);超阈值时每 run 提示一次(goal 自动续轮共享同一 run 时 `agent_end` 可能整段不触发,避免回路失效)。
|
|
75
|
+
- `agent_end`:每 run 落盘一行日志。
|
|
76
|
+
- `before_agent_start`:结算上一轮并以 `nextTurn` 尾部消息回灌——计数、耗时、**缓存命中率**(仅当该轮有 usage 数据时出现;< 90% 标「低」)、会话中位、阈值判定(`display: false`)。
|
|
77
|
+
- **1.0.32 起不再回灌「预计 / 实际 / 偏差」**:声明率虽升到 65%,中位误差仍为 94%(调用数)与 199%(耗时),且 412 个声明点上 6 种预测器(常量 12/16、1.1× 上轮、2.0× 上轮、全局中位、区间 3~40)全部落在 71–94% 误差。
|
|
63
78
|
|
|
64
79
|
**注入样例(示意,字段可能随版本增删)**
|
|
65
80
|
|
|
66
81
|
```text
|
|
67
|
-
PTC 计量:上一轮 9 次工具调用 · 实际 34s · 命中 92% ·
|
|
68
|
-
· 会话中位 25 · 阈值 50 · 判据 = 本轮计数不超阈值且无同类调用可并
|
|
82
|
+
PTC 计量:上一轮 9 次工具调用 · 实际 34s · 命中 92% · 会话中位 25 · 阈值 50 · 判据 = 本轮计数不超阈值且无同类调用可并
|
|
69
83
|
```
|
|
70
84
|
- `session_start`:模块级计量归零。
|
|
71
85
|
- system 内 section `ptc_meter` 只含逐字不变的骨架。
|
|
72
|
-
|
|
73
86
|
## 原理(专业)
|
|
74
87
|
|
|
75
88
|
```text
|
|
76
|
-
本轮开工前 ─ 声明「预计 N 次 / 预计 Ns」
|
|
77
|
-
↓
|
|
78
89
|
tool_call ─ 仅顶层调用计数 +1
|
|
79
90
|
↓
|
|
80
|
-
turn_end ─ 记录实际次数与耗时
|
|
91
|
+
turn_end ─ 记录实际次数与耗时
|
|
81
92
|
↓
|
|
82
|
-
agent_end ─
|
|
93
|
+
agent_end ─ 落盘一行日志
|
|
83
94
|
↓
|
|
84
95
|
before_agent_start ─ 结算上一轮
|
|
85
96
|
↓
|
|
86
|
-
|
|
97
|
+
回灌:实际次数 / 耗时 / 命中率 / 会话中位 / 阈值判定
|
|
87
98
|
↓ nextTurn 尾部消息
|
|
88
|
-
(回到本轮开工前)
|
|
89
99
|
|
|
90
100
|
system 只留静态骨架(ptc_meter section)
|
|
91
101
|
```
|
|
92
|
-
|
|
93
102
|
## 原理(小白)
|
|
94
103
|
|
|
95
|
-
|
|
104
|
+
**像仪表盘**:不看你预测,只报「刚跑了几步、这个会话通常几步」。计划由你定,仪表只给事实。
|
|
96
105
|
|
|
97
106
|
```text
|
|
98
|
-
开工前先估:预计几步
|
|
99
|
-
↓
|
|
100
107
|
干活:实际跑几步
|
|
101
108
|
↓
|
|
102
|
-
|
|
109
|
+
收工记账
|
|
103
110
|
↓
|
|
104
|
-
|
|
111
|
+
下次开工报:上一轮 7 步,会话中位 25
|
|
105
112
|
↓
|
|
106
|
-
|
|
113
|
+
你自己决定要不要合并调用
|
|
107
114
|
```
|
|
108
|
-
|
|
109
115
|
## License
|
|
110
116
|
|
|
111
117
|
MIT
|
|
@@ -116,7 +122,7 @@ MIT
|
|
|
116
122
|
|
|
117
123
|
<h1 align="center">🌐 English Version ⬇️</h1>
|
|
118
124
|
|
|
119
|
-
Per-turn tool-call metering: records calls and duration
|
|
125
|
+
Per-turn tool-call metering: records calls and duration and feeds back the actuals plus the session median. No estimates, no deviation (1.0.32 removed both).
|
|
120
126
|
|
|
121
127
|
## At a glance
|
|
122
128
|
|
|
@@ -124,7 +130,7 @@ Per-turn tool-call metering: records calls and duration, compares them with the
|
|
|
124
130
|
| --- | --- |
|
|
125
131
|
| Config file | ~/.pi/ptc-meter.json (optional) |
|
|
126
132
|
| Commands | /ptc [n] (default 20) |
|
|
127
|
-
| Automation | **Fully automatic**: metering and feedback need no trigger.
|
|
133
|
+
| Automation | **Fully automatic**: metering and feedback need no trigger. Only the actual values and the session median are fed back; no estimates are collected (1.0.32 removed both the call-count and duration estimates - measured median errors were 94% and 199%) |
|
|
128
134
|
|
|
129
135
|
Full config (all keys and defaults):
|
|
130
136
|
|
|
@@ -159,58 +165,64 @@ Also: log `~/.pi/ptc-meter.jsonl` (1 MiB truncation, tail-preserving); no enviro
|
|
|
159
165
|
|
|
160
166
|
| Command | Purpose |
|
|
161
167
|
| --- | --- |
|
|
162
|
-
| `/ptc [n]` | Last n turns: counts, duration,
|
|
168
|
+
| `/ptc [n]` | Last n turns: counts, duration, median, over-threshold |
|
|
169
|
+
|
|
170
|
+
## Verification
|
|
171
|
+
|
|
172
|
+
Changes to pi-ptc-meter were verified on an isolated copy before release; the current version passes **10,011 assertions**.
|
|
173
|
+
|
|
174
|
+
| Surface | Scale | Result |
|
|
175
|
+
| --- | --- | --- |
|
|
176
|
+
| Real log replay | 4,349 turns | 🟢 |
|
|
177
|
+
| Random states | 10,000 | 🟢 |
|
|
178
|
+
| Statistic edges | 11 | 🟢 |
|
|
179
|
+
|
|
180
|
+
🟢 **Result**: both estimates removed - on 412 declaration points all six predictors landed at 71-94% error, so estimating is unpredictable exactly when it is needed.
|
|
181
|
+
Change: both estimates removed. Why: estimates are unpredictable at declaration points: six predictors all landed at 71-94% error; duration median error 199%.
|
|
163
182
|
|
|
164
183
|
## What it does (source facts)
|
|
165
184
|
|
|
166
185
|
- `tool_call`: increments the counter for top-level calls only (nested codemode calls are not counted separately; never intercepts or rewrites).
|
|
167
|
-
- `turn_end`: records calls and duration (no disk write).
|
|
168
|
-
- `agent_end`: writes one log line per run
|
|
169
|
-
- `before_agent_start`: settles the previous turn and feeds back counts, duration,
|
|
186
|
+
- `turn_end`: records calls and duration (no disk write); when goal auto-continuation shares one run and `agent_end` never fires, it warns once above threshold (once per run).
|
|
187
|
+
- `agent_end`: writes one log line per run.
|
|
188
|
+
- `before_agent_start`: settles the previous turn and feeds back counts, duration, the **cache hit rate** (only when that turn reported usage; flagged low below 90%), the session median and the threshold verdict as a `nextTurn` trailing message (`display: false`).
|
|
189
|
+
- **1.0.32 no longer feeds back estimate / actual / deviation**: declaration rate rose to 65% but the median error stayed at 94% (call count) and 199% (duration); on 412 declaration points all six predictors (constant 12/16, 1.1x previous, 2.0x previous, global median, interval 3-40) landed between 71% and 94% error.
|
|
170
190
|
|
|
171
191
|
**Sample injection (illustrative; fields may change between versions)**
|
|
172
192
|
|
|
173
193
|
```text
|
|
174
|
-
PTC metering: last turn 9 tool calls · 34s actual · 92% cache hit ·
|
|
175
|
-
· session median 25 · threshold 50 · verdict: under threshold, no obvious merge candidates
|
|
194
|
+
PTC metering: last turn 9 tool calls · 34s actual · 92% cache hit · session median 25 · threshold 50 · verdict: under threshold, no obvious merge candidates
|
|
176
195
|
```
|
|
177
196
|
- `session_start`: resets module-level counters.
|
|
178
197
|
- The `ptc_meter` system section holds only a byte-stable skeleton.
|
|
179
|
-
|
|
180
198
|
## How it works (technical)
|
|
181
199
|
|
|
182
200
|
```text
|
|
183
|
-
before the turn ─ declare: about N calls / Ns
|
|
184
|
-
↓
|
|
185
201
|
tool_call ─ top-level call: counter +1
|
|
186
202
|
↓
|
|
187
|
-
turn_end ─ record actual calls and duration
|
|
203
|
+
turn_end ─ record actual calls and duration
|
|
188
204
|
↓
|
|
189
|
-
agent_end ─ write one log line
|
|
205
|
+
agent_end ─ write one log line
|
|
190
206
|
↓
|
|
191
207
|
before_agent_start ─ settle the previous turn
|
|
192
208
|
↓
|
|
193
|
-
|
|
209
|
+
feed back: calls / duration / cache hit / session median / threshold verdict
|
|
194
210
|
↓ nextTurn message
|
|
195
|
-
(back to before the turn)
|
|
196
211
|
|
|
197
212
|
the system prompt keeps only a static skeleton (ptc_meter section)
|
|
198
213
|
```
|
|
199
|
-
|
|
200
214
|
## How it works (plain words)
|
|
201
215
|
|
|
202
|
-
**Like a
|
|
216
|
+
**Like a dashboard**: it does not care what you predicted; it just reports how many steps the last turn took and what this session usually takes. You decide the plan; the dashboard gives facts.
|
|
203
217
|
|
|
204
218
|
```text
|
|
205
|
-
|
|
206
|
-
↓
|
|
207
|
-
work: actually N steps
|
|
219
|
+
work: N actual steps
|
|
208
220
|
↓
|
|
209
|
-
|
|
221
|
+
log it
|
|
210
222
|
↓
|
|
211
|
-
|
|
223
|
+
next turn reports: last turn 7 calls, session median 25
|
|
212
224
|
↓
|
|
213
|
-
|
|
225
|
+
you decide whether to merge calls
|
|
214
226
|
```
|
|
215
227
|
## License
|
|
216
228
|
|