@tmustier/pi-usage-extension 0.9.0 → 0.9.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +39 -0
- package/README.md +23 -9
- package/data.ts +699 -61
- package/export.ts +34 -0
- package/index.ts +29 -7
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,44 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.9.4] - 2026-07-22
|
|
4
|
+
|
|
5
|
+
### Changed
|
|
6
|
+
- Radically simplified nested-agent reconciliation. Child sessions on disk are now trusted as the record: when every child session file behind a tool report is part of the scan, the parent's aggregate is skipped; otherwise the whole report (or each unresolved legacy child) is counted under `Tools / summaries`. This replaces 0.9.3's exact contiguous token-and-cost vector matching and residual arithmetic (~250 lines) with a file-existence check — validated byte-identical against 0.9.3 on a 10 GB / 6,166-session corpus.
|
|
7
|
+
- Resolved child identities are unioned across copied parent history before emission, because branch copies change runId-derived child paths. Same outcome as 0.9.3's global reconciliation, without vector identity keys.
|
|
8
|
+
- Known trade-offs: a crash-truncated child session slightly undercounts, and a partially-missing child set counts the full aggregate (slight overcount). Neither case appeared in the audit corpus.
|
|
9
|
+
|
|
10
|
+
### Internal
|
|
11
|
+
- Cache format stays **v5** — no rebuild on upgrade.
|
|
12
|
+
|
|
13
|
+
## [0.9.3] - 2026-07-22
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
- Reconcile nested-agent tool usage against recursively scanned child sessions using exact contiguous token-and-cost vectors. Canonical tool aggregates now exclude child usage already counted from session files, and mixed aggregates add only the unmatched residual under `Tools / summaries`.
|
|
17
|
+
- Reconcile globally across copied parent history, so a stale copy with a missing child path cannot win deduplication and reintroduce usage represented by another copy's linked child session.
|
|
18
|
+
- Backfill recognised legacy `subagent` and `subagent_wait` child usage only when no scanned child-session span represents it.
|
|
19
|
+
|
|
20
|
+
### Performance
|
|
21
|
+
- Extract accounting metadata from large tool results without decoding or JSON-parsing multi-megabyte output bodies.
|
|
22
|
+
|
|
23
|
+
### Internal
|
|
24
|
+
- Cache format bumped to **v5** to retain tool reports and child-session linkage. The first `/usage` open after upgrading performs a one-off rebuild.
|
|
25
|
+
- Document the irrecoverable historical boundary: compaction and branch-summary entries written without usage metadata remain unmetered. The compatibility audit found 2,753 such compactions and 20 branch summaries.
|
|
26
|
+
|
|
27
|
+
## [0.9.2] - 2026-07-21
|
|
28
|
+
|
|
29
|
+
### Fixed
|
|
30
|
+
- Include usage persisted by Pi 0.81.0 on tool results, compactions, and branch summaries. Auxiliary cost and token fields now participate in `/usage` accounting under the same synthetic `Tools / summaries` bucket as Pi's footer and `/session`, while `Msgs` remains an assistant-message count.
|
|
31
|
+
- Keep auxiliary usage out of conversation context/cache-miss classification, attribute it separately in Insights, and deduplicate copied auxiliary entries by stable session entry id.
|
|
32
|
+
|
|
33
|
+
### Internal
|
|
34
|
+
- Cache format bumped to **v4**. The first `/usage` open after upgrading performs a one-off rebuild so previously ignored auxiliary entries are ingested.
|
|
35
|
+
|
|
36
|
+
## [0.9.1] - 2026-07-17
|
|
37
|
+
|
|
38
|
+
### Changed
|
|
39
|
+
- Exports now save to `/tmp` (or the OS temp dir where `/tmp` doesn't exist) instead of the pi session's cwd, so they never litter a repo or home directory. The `✓ Saved` note shows the full path.
|
|
40
|
+
- The export directory is configurable: `{ "usage-extension": { "exportDir": "~/Downloads" } }` in `~/.pi/agent/settings.json` (`~` expands; the directory is created if missing).
|
|
41
|
+
|
|
3
42
|
## [0.9.0] - 2026-07-17
|
|
4
43
|
|
|
5
44
|
### Added
|
package/README.md
CHANGED
|
@@ -7,7 +7,9 @@ A Pi extension that displays aggregated usage statistics across all sessions.
|
|
|
7
7
|
## Compatibility
|
|
8
8
|
|
|
9
9
|
- **Pi version:** 0.42.4+
|
|
10
|
-
- **Last updated:** 2026-07-
|
|
10
|
+
- **Last updated:** 2026-07-22 (0.9.3)
|
|
11
|
+
|
|
12
|
+
Pi 0.81.0+ can persist tool-result, compaction, and branch-summary usage. `/usage` includes that auxiliary usage in totals under `Tools / summaries`. Nested-agent reports are reconciled against recursively scanned child sessions, so a child call is counted once: when every child session file behind a report is part of the scan, the children are the record and the parent's aggregate is skipped; otherwise the report is counted. Older Pi versions remain supported.
|
|
11
13
|
|
|
12
14
|
## Installation
|
|
13
15
|
|
|
@@ -83,6 +85,7 @@ The **Insights** view has two sections, facts first:
|
|
|
83
85
|
|
|
84
86
|
| Insight | Detail |
|
|
85
87
|
|---|---|
|
|
88
|
+
| Tool and summary usage | share of cost reported by nested tool calls, compaction, and branch summarization (Pi 0.81.0+) |
|
|
86
89
|
| Context tax | share of cost spent at ≥ 150k context, with the average per-message cost vs messages under 100k |
|
|
87
90
|
| Project mix | top 3 project directories by cost, derived from each session's working directory (worktrees collapse into their repository; hidden when one project is ≥ 90% — you already know) |
|
|
88
91
|
| Reasoning share | share of output tokens that were hidden reasoning (recorded by pi 0.80.3+ only; hidden below 5%) |
|
|
@@ -107,7 +110,7 @@ A note on reading them: cache-miss and context numbers are **observed cost, not
|
|
|
107
110
|
|
|
108
111
|
### Export
|
|
109
112
|
|
|
110
|
-
Press `e` in any view to write the **current slice** to the
|
|
113
|
+
Press `e` in any view to write the **current slice** to `/tmp` (or the OS temp dir where `/tmp` doesn't exist) — exports never litter your repo or home directory, and the `✓ Saved` note shows the full path so you can grab the file if you want to keep it:
|
|
111
114
|
|
|
112
115
|
| View | File | Contents |
|
|
113
116
|
|---|---|---|
|
|
@@ -117,6 +120,14 @@ Press `e` in any view to write the **current slice** to the current working dire
|
|
|
117
120
|
|
|
118
121
|
A `✓ Saved …` note confirms the write (or reports the error) above the help line.
|
|
119
122
|
|
|
123
|
+
To export somewhere permanent instead, set an export directory in `~/.pi/agent/settings.json` (created if missing, `~` expands):
|
|
124
|
+
|
|
125
|
+
```json
|
|
126
|
+
{
|
|
127
|
+
"usage-extension": { "exportDir": "~/Downloads" }
|
|
128
|
+
}
|
|
129
|
+
```
|
|
130
|
+
|
|
120
131
|
### Graph explorer
|
|
121
132
|
|
|
122
133
|
The **Graphs** view plots usage over time for the active period as a braille line chart, with a legend showing per-series totals and shares.
|
|
@@ -128,7 +139,7 @@ The **Graphs** view plots usage over time for the active period as a braille lin
|
|
|
128
139
|
- **Buckets**: hourly for Today / This Week / Last Week, daily for Last 30 Days / All Time.
|
|
129
140
|
- **Line clipping**: every series (provider, model, thinking level, `other`, Total) is drawn only between its first and last bucket with usage, so late-starting or retired series don't drag a flat zero/flat tail across the whole period.
|
|
130
141
|
|
|
131
|
-
Thinking levels are replayed from `thinking_level_change` entries in each session file; messages before the first recorded change appear as `unknown`. Reasoning token counts come from `usage.reasoning` where providers report them; pi only records this field since **pi 0.80.3 (30 June 2026)**, so earlier sessions show zero reasoning tokens even though thinking models were in use.
|
|
142
|
+
Thinking levels are replayed from `thinking_level_change` entries in each session file; messages before the first recorded change appear as `unknown`. Auxiliary usage has no reliable thinking-level attribution and appears as `Tools/summaries` in that grouping. Reasoning token counts come from `usage.reasoning` where providers report them; pi only records this field since **pi 0.80.3 (30 June 2026)**, so earlier sessions show zero reasoning tokens even though thinking models were in use.
|
|
132
143
|
|
|
133
144
|
### Time Periods
|
|
134
145
|
|
|
@@ -152,8 +163,8 @@ Time periods are calculated in the local timezone where Pi runs. If you want to
|
|
|
152
163
|
|--------|-------------|
|
|
153
164
|
| **Provider / Model** | Provider name, expandable to show models |
|
|
154
165
|
| **Sessions** | Number of unique sessions |
|
|
155
|
-
| **Msgs** | Number of assistant messages |
|
|
156
|
-
| **Cost** | Total cost in USD (from API response) |
|
|
166
|
+
| **Msgs** | Number of assistant messages; auxiliary `Tools / summaries` usage does not inflate this count |
|
|
167
|
+
| **Cost** | Total cost in USD (from API response), including usage reported by tools and summaries |
|
|
157
168
|
| **Tokens** | Fresh tokens for the turn: input + output + cache write |
|
|
158
169
|
| **↑In** | Fresh input tokens: input + cache write *(dimmed)* |
|
|
159
170
|
| **↓Out** | Output tokens *(dimmed)* |
|
|
@@ -187,13 +198,16 @@ On narrow terminals, `/usage` automatically switches to a compact table instead
|
|
|
187
198
|
- **On-disk cache.** Per-file extraction results are cached in `<agentDir>/usage-extension-cache.json` (respects `PI_CODING_AGENT_DIR`), keyed by file size + mtime. Warm opens only re-parse session files that changed since the last run — on a 5.2 GB / 3,310-file corpus that takes the open from ~17 s to ~0.3 s.
|
|
188
199
|
- **First open** after install (or after deleting the cache) does a one-off full build, showing the usual cancellable loader. Cancelling saves partial progress, so the next open resumes where it left off.
|
|
189
200
|
- The cache is safe to delete at any time; it is rebuilt automatically. Corrupt or version-mismatched caches are ignored and rebuilt rather than trusted.
|
|
190
|
-
- **0.
|
|
201
|
+
- **0.9.3 bumps the cache format to v5** to retain child-session linkage for tool-usage reconciliation (v4 added Pi 0.81.0 tool and summary usage; v3 added session working directory and compaction markers; v2 added thinking level and reasoning tokens). The first open after upgrading does a one-off full rebuild (with a progress message and live file counter), then warm opens are fast again.
|
|
202
|
+
- Large nested-agent tool results use an allocation-safe metadata parser: multi-megabyte output bodies are scanned as bytes rather than decoded and JSON-parsed in full.
|
|
191
203
|
|
|
192
204
|
## Provider Notes
|
|
193
205
|
|
|
194
206
|
### Cost Tracking
|
|
195
207
|
|
|
196
|
-
Cost data comes directly from
|
|
208
|
+
Cost data comes directly from persisted usage values. For assistant messages it is grouped by provider/model. Pi 0.81.0+ can also persist usage reported by tools, compaction, and branch summarization; because those entries do not carry reliable provider/model attribution, `/usage` groups them under `Tools / summaries`, matching Pi's `/session` breakdown. Recognised legacy `subagent` and `subagent_wait` details are used as a fallback when their child session is no longer available. Accuracy depends on the provider or tool reporting costs.
|
|
209
|
+
|
|
210
|
+
Only persisted usage can be counted. Pi did not add usage metadata retroactively, so historical compaction or branch-summary entries written without `usage` remain unmetered: their exact token and cost vectors cannot be reconstructed. The compatibility audit corpus contained 2,753 such compactions and 20 branch summaries.
|
|
197
211
|
|
|
198
212
|
### Cache Tokens
|
|
199
213
|
|
|
@@ -211,9 +225,9 @@ The "Cache" column combines both read and write tokens.
|
|
|
211
225
|
|
|
212
226
|
## Data Source
|
|
213
227
|
|
|
214
|
-
Statistics are parsed recursively from session files in `~/.pi/agent/sessions/`, including nested subagent runs such as `run-0/` directories. Each session is a JSONL file containing
|
|
228
|
+
Statistics are parsed recursively from session files in `~/.pi/agent/sessions/`, including nested subagent runs such as `run-0/` directories. Each session is a JSONL file containing assistant messages and, on Pi 0.81.0+, optional usage on tool-result, compaction, and branch-summary entries.
|
|
215
229
|
|
|
216
|
-
Assistant messages duplicated across branched session files are deduplicated by timestamp + total tokens,
|
|
230
|
+
Assistant messages duplicated across branched session files are deduplicated by timestamp + total tokens. Auxiliary usage is deduplicated by its stable session entry id, which avoids collapsing parallel tools that happen to report identical usage. Nested-agent reports are suppressed when their child session files are part of the scan (resolved across all copies of the parent entry, since branch copies change runId-derived paths); reports whose children are missing stay under `Tools / summaries`. Tool and summary usage contributes cost and tokens to totals, graphs, project/session mix, and burn trend, but does not count as an assistant message or distort conversation context/cache insights.
|
|
217
231
|
|
|
218
232
|
Respects the `PI_CODING_AGENT_DIR` environment variable if set.
|
|
219
233
|
|
package/data.ts
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
* Data collection, caching, and insights for the /usage dashboard.
|
|
3
3
|
*
|
|
4
4
|
* Performance model (see CHANGELOG 0.4.0):
|
|
5
|
-
* - Session JSONL files are scanned at the buffer level. Only lines
|
|
6
|
-
*
|
|
7
|
-
*
|
|
8
|
-
*
|
|
5
|
+
* - Session JSONL files are scanned at the buffer level. Only lines relevant
|
|
6
|
+
* to assistant or auxiliary accounting are decoded and JSON.parsed. Ordinary
|
|
7
|
+
* multi-megabyte tool results are skipped; accounting-bearing large results
|
|
8
|
+
* use an allocation-safe byte parser for their small metadata fields.
|
|
9
9
|
* - Per-file extraction results are persisted to an on-disk cache keyed by
|
|
10
10
|
* (size, mtimeMs). Session files are append-only, so a warm load only
|
|
11
11
|
* re-parses files that changed since the last run.
|
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
|
|
14
14
|
import { readdir, readFile, rename, stat, writeFile } from "node:fs/promises";
|
|
15
15
|
import { homedir } from "node:os";
|
|
16
|
-
import { dirname, join } from "node:path";
|
|
16
|
+
import { basename, dirname, isAbsolute, join, resolve } from "node:path";
|
|
17
17
|
|
|
18
18
|
// =============================================================================
|
|
19
19
|
// Types
|
|
@@ -66,8 +66,12 @@ interface CostCount {
|
|
|
66
66
|
}
|
|
67
67
|
|
|
68
68
|
interface PeriodRawData {
|
|
69
|
+
/** All recorded cost, including usage reported by tools and summaries. */
|
|
69
70
|
totalCost: number;
|
|
70
|
-
|
|
71
|
+
/** Cost attached to assistant messages, used as the turn-insight denominator. */
|
|
72
|
+
assistantCost: number;
|
|
73
|
+
/** Usage reported by tool results, compactions, and branch summaries. */
|
|
74
|
+
auxiliaryCost: number;
|
|
71
75
|
/** Messages at ≥ CTX_TAX_THRESHOLD context. */
|
|
72
76
|
ctxHigh: CostCount;
|
|
73
77
|
/** Messages below CTX_LOW_THRESHOLD context (comparison group). */
|
|
@@ -164,18 +168,31 @@ export type TabName = "today" | "thisWeek" | "lastWeek" | "last30Days" | "allTim
|
|
|
164
168
|
|
|
165
169
|
export const TAB_ORDER: TabName[] = ["today", "thisWeek", "lastWeek", "last30Days", "allTime"];
|
|
166
170
|
|
|
167
|
-
export
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
171
|
+
export type UsageSource = "assistant" | "auxiliary";
|
|
172
|
+
|
|
173
|
+
/** Pi's own label for usage that cannot be attributed to a provider/model. */
|
|
174
|
+
export const AUXILIARY_PROVIDER = "Tools";
|
|
175
|
+
export const AUXILIARY_MODEL = "summaries";
|
|
176
|
+
export const AUXILIARY_THINKING_LEVEL = "Tools/summaries";
|
|
177
|
+
|
|
178
|
+
export interface UsageAmount {
|
|
172
179
|
cost: number;
|
|
173
180
|
input: number;
|
|
174
181
|
output: number;
|
|
175
182
|
cacheRead: number;
|
|
176
183
|
cacheWrite: number;
|
|
177
|
-
/** Reasoning output tokens reported by the provider (0 when absent). */
|
|
178
184
|
reasoning: number;
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
export interface SessionMessage extends UsageAmount {
|
|
188
|
+
provider: string;
|
|
189
|
+
model: string;
|
|
190
|
+
/** Thinking level active when the message was produced; "" when unknown. */
|
|
191
|
+
thinkingLevel: string;
|
|
192
|
+
/** Assistant response, or usage reported by a tool/summary entry. */
|
|
193
|
+
source: UsageSource;
|
|
194
|
+
/** Session entry id used to dedupe copied auxiliary entries; empty for assistant messages. */
|
|
195
|
+
sourceId: string;
|
|
179
196
|
timestamp: number;
|
|
180
197
|
/**
|
|
181
198
|
* True when a compaction entry occurred between the previous assistant
|
|
@@ -185,20 +202,41 @@ export interface SessionMessage {
|
|
|
185
202
|
afterCompaction: boolean;
|
|
186
203
|
}
|
|
187
204
|
|
|
205
|
+
export interface ChildToolUsage {
|
|
206
|
+
resultIndex: number;
|
|
207
|
+
/** Persisted child session path when the tool supplied one; empty otherwise. */
|
|
208
|
+
sessionFile: string;
|
|
209
|
+
usage: UsageAmount;
|
|
210
|
+
}
|
|
211
|
+
|
|
212
|
+
export interface ToolUsageRecord {
|
|
213
|
+
/** Parent tool-result entry id, stable across copied branch history. */
|
|
214
|
+
sourceId: string;
|
|
215
|
+
timestamp: number;
|
|
216
|
+
/** Canonical Pi 0.81+ tool usage; null for legacy nested-agent results. */
|
|
217
|
+
reportedUsage: UsageAmount | null;
|
|
218
|
+
/** Run id used to derive the standard nested-session path when needed. */
|
|
219
|
+
runId: string;
|
|
220
|
+
/** Recognised per-child usage from nested-agent tool details. */
|
|
221
|
+
children: ChildToolUsage[];
|
|
222
|
+
}
|
|
223
|
+
|
|
188
224
|
export interface ParsedSessionFile {
|
|
189
225
|
/** Empty string when the file has no session header — such files are ignored. */
|
|
190
226
|
sessionId: string;
|
|
191
227
|
/** Working directory from the session header; "" when absent. */
|
|
192
228
|
cwd: string;
|
|
193
|
-
/** Extracted assistant
|
|
229
|
+
/** Extracted assistant and summary usage records, pre-dedupe. */
|
|
194
230
|
messages: SessionMessage[];
|
|
231
|
+
/** Tool usage is reconciled against recursively scanned child sessions later. */
|
|
232
|
+
toolUsages: ToolUsageRecord[];
|
|
195
233
|
}
|
|
196
234
|
|
|
197
235
|
// =============================================================================
|
|
198
236
|
// Paths
|
|
199
237
|
// =============================================================================
|
|
200
238
|
|
|
201
|
-
function getAgentDir(): string {
|
|
239
|
+
export function getAgentDir(): string {
|
|
202
240
|
// Replicate Pi's logic: respect PI_CODING_AGENT_DIR env var
|
|
203
241
|
return process.env.PI_CODING_AGENT_DIR || join(homedir(), ".pi", "agent");
|
|
204
242
|
}
|
|
@@ -252,35 +290,374 @@ const NEWLINE = 0x0a;
|
|
|
252
290
|
// only cost a wasted JSON.parse — the parsed entry is still shape-checked.
|
|
253
291
|
const PATTERN_ASSISTANT_COMPACT = Buffer.from('"role":"assistant"');
|
|
254
292
|
const PATTERN_ASSISTANT_SPACED = Buffer.from('"role": "assistant"');
|
|
293
|
+
const PATTERN_TOOL_RESULT_COMPACT = Buffer.from('"role":"toolResult"');
|
|
294
|
+
const PATTERN_TOOL_RESULT_SPACED = Buffer.from('"role": "toolResult"');
|
|
295
|
+
const PATTERN_USAGE_COMPACT = Buffer.from('"usage":{');
|
|
296
|
+
const PATTERN_USAGE_SPACED = Buffer.from('"usage": {');
|
|
255
297
|
const PATTERN_SESSION_COMPACT = Buffer.from('"type":"session"');
|
|
256
298
|
const PATTERN_SESSION_SPACED = Buffer.from('"type": "session"');
|
|
257
299
|
const PATTERN_THINKING_COMPACT = Buffer.from('"type":"thinking_level_change"');
|
|
258
300
|
const PATTERN_THINKING_SPACED = Buffer.from('"type": "thinking_level_change"');
|
|
259
301
|
const PATTERN_COMPACTION_COMPACT = Buffer.from('"type":"compaction"');
|
|
260
302
|
const PATTERN_COMPACTION_SPACED = Buffer.from('"type": "compaction"');
|
|
303
|
+
const PATTERN_BRANCH_SUMMARY_COMPACT = Buffer.from('"type":"branch_summary"');
|
|
304
|
+
const PATTERN_BRANCH_SUMMARY_SPACED = Buffer.from('"type": "branch_summary"');
|
|
305
|
+
// pi-subagents versions predating Pi 0.81 persisted child usage in details but
|
|
306
|
+
// could not put it on the canonical tool-result usage field. Their tool names
|
|
307
|
+
// are near the start of the line, so we can recover those records without
|
|
308
|
+
// scanning every multi-megabyte tool result.
|
|
309
|
+
const PATTERN_SUBAGENT_TOOL_COMPACT = Buffer.from('"toolName":"subagent"');
|
|
310
|
+
const PATTERN_SUBAGENT_TOOL_SPACED = Buffer.from('"toolName": "subagent"');
|
|
311
|
+
const PATTERN_SUBAGENT_WAIT_TOOL_COMPACT = Buffer.from('"toolName":"subagent_wait"');
|
|
312
|
+
const PATTERN_SUBAGENT_WAIT_TOOL_SPACED = Buffer.from('"toolName": "subagent_wait"');
|
|
261
313
|
|
|
262
314
|
const PARSE_YIELD_EVERY_LINES = 2000;
|
|
263
315
|
|
|
316
|
+
function finiteNumber(value: unknown): number {
|
|
317
|
+
return typeof value === "number" && Number.isFinite(value) ? value : 0;
|
|
318
|
+
}
|
|
319
|
+
|
|
320
|
+
function parseUsageAmount(value: unknown): UsageAmount | null {
|
|
321
|
+
if (!value || typeof value !== "object") return null;
|
|
322
|
+
const persisted = value as Record<string, unknown>;
|
|
323
|
+
const costValue = persisted.cost;
|
|
324
|
+
const cost =
|
|
325
|
+
typeof costValue === "number"
|
|
326
|
+
? finiteNumber(costValue)
|
|
327
|
+
: costValue && typeof costValue === "object"
|
|
328
|
+
? finiteNumber((costValue as Record<string, unknown>).total)
|
|
329
|
+
: 0;
|
|
330
|
+
const usage = {
|
|
331
|
+
cost,
|
|
332
|
+
input: finiteNumber(persisted.input),
|
|
333
|
+
output: finiteNumber(persisted.output),
|
|
334
|
+
cacheRead: finiteNumber(persisted.cacheRead),
|
|
335
|
+
cacheWrite: finiteNumber(persisted.cacheWrite),
|
|
336
|
+
reasoning: finiteNumber(persisted.reasoning),
|
|
337
|
+
};
|
|
338
|
+
return usage.cost === 0 && usage.input === 0 && usage.output === 0 && usage.cacheRead === 0 && usage.cacheWrite === 0
|
|
339
|
+
? null
|
|
340
|
+
: usage;
|
|
341
|
+
}
|
|
342
|
+
|
|
343
|
+
function parsedTimestamp(messageTimestamp: unknown, entryTimestamp: unknown): number {
|
|
344
|
+
const parsed =
|
|
345
|
+
typeof messageTimestamp === "number"
|
|
346
|
+
? messageTimestamp
|
|
347
|
+
: new Date(String(messageTimestamp ?? entryTimestamp ?? "")).getTime();
|
|
348
|
+
return Number.isFinite(parsed) ? parsed : 0;
|
|
349
|
+
}
|
|
350
|
+
|
|
351
|
+
function auxiliaryMessage(usage: UsageAmount, timestamp: number, sourceId: string): SessionMessage {
|
|
352
|
+
return {
|
|
353
|
+
provider: AUXILIARY_PROVIDER,
|
|
354
|
+
model: AUXILIARY_MODEL,
|
|
355
|
+
thinkingLevel: AUXILIARY_THINKING_LEVEL,
|
|
356
|
+
source: "auxiliary",
|
|
357
|
+
sourceId,
|
|
358
|
+
...usage,
|
|
359
|
+
timestamp,
|
|
360
|
+
afterCompaction: false,
|
|
361
|
+
};
|
|
362
|
+
}
|
|
363
|
+
|
|
364
|
+
function buildToolUsageRecord(
|
|
365
|
+
toolName: unknown,
|
|
366
|
+
detailsValue: unknown,
|
|
367
|
+
reportedValue: unknown,
|
|
368
|
+
sourceIdValue: unknown,
|
|
369
|
+
messageTimestamp: unknown,
|
|
370
|
+
entryTimestamp: unknown
|
|
371
|
+
): ToolUsageRecord | null {
|
|
372
|
+
const reportedUsage = parseUsageAmount(reportedValue);
|
|
373
|
+
const details = detailsValue && typeof detailsValue === "object" ? (detailsValue as Record<string, unknown>) : null;
|
|
374
|
+
const totalChildUsage = parseUsageAmount(details?.totalChildUsage);
|
|
375
|
+
const knownNestedTool = toolName === "subagent" || toolName === "subagent_wait";
|
|
376
|
+
const children: ChildToolUsage[] = [];
|
|
377
|
+
if (details && Array.isArray(details.results)) {
|
|
378
|
+
for (let resultIndex = 0; resultIndex < details.results.length; resultIndex++) {
|
|
379
|
+
const result = details.results[resultIndex];
|
|
380
|
+
if (!result || typeof result !== "object") continue;
|
|
381
|
+
const child = result as Record<string, unknown>;
|
|
382
|
+
const usage = parseUsageAmount(child.usage);
|
|
383
|
+
const sessionFile = typeof child.sessionFile === "string" ? child.sessionFile : "";
|
|
384
|
+
// Legacy fallback is deliberately restricted to recognised nested-agent
|
|
385
|
+
// records. Generic Pi 0.81 tools remain canonical through reportedUsage.
|
|
386
|
+
if (usage && (knownNestedTool || totalChildUsage || (reportedUsage && sessionFile))) {
|
|
387
|
+
children.push({ resultIndex, sessionFile, usage });
|
|
388
|
+
}
|
|
389
|
+
}
|
|
390
|
+
}
|
|
391
|
+
if (!reportedUsage && children.length === 0) return null;
|
|
392
|
+
return {
|
|
393
|
+
sourceId: typeof sourceIdValue === "string" ? sourceIdValue : "",
|
|
394
|
+
timestamp: parsedTimestamp(messageTimestamp, entryTimestamp),
|
|
395
|
+
reportedUsage,
|
|
396
|
+
runId: details && typeof details.runId === "string" ? details.runId : "",
|
|
397
|
+
children,
|
|
398
|
+
};
|
|
399
|
+
}
|
|
400
|
+
|
|
401
|
+
const LARGE_TOOL_RESULT_BYTES = 64 * 1024;
|
|
402
|
+
const PROPERTY_ID = Buffer.from('"id":');
|
|
403
|
+
const PROPERTY_TIMESTAMP = Buffer.from('"timestamp":');
|
|
404
|
+
const PROPERTY_MESSAGE = Buffer.from('"message":');
|
|
405
|
+
const PROPERTY_TOOL_NAME = Buffer.from('"toolName":');
|
|
406
|
+
const PROPERTY_DETAILS = Buffer.from('"details":');
|
|
407
|
+
const PROPERTY_USAGE = Buffer.from('"usage":');
|
|
408
|
+
const DIRECT_CHILD_PROPERTIES = new Set(["usage", "sessionFile"]);
|
|
409
|
+
|
|
410
|
+
function skipJsonWhitespace(buffer: Buffer, offset: number, limit: number): number {
|
|
411
|
+
while (offset < limit && (buffer[offset] === 0x20 || buffer[offset] === 0x09 || buffer[offset] === 0x0a || buffer[offset] === 0x0d)) offset++;
|
|
412
|
+
return offset;
|
|
413
|
+
}
|
|
414
|
+
|
|
415
|
+
/** Find one JSON value's end without decoding large strings or container bodies. */
|
|
416
|
+
function jsonValueEnd(buffer: Buffer, offset: number, limit: number): number {
|
|
417
|
+
offset = skipJsonWhitespace(buffer, offset, limit);
|
|
418
|
+
if (offset >= limit) return offset;
|
|
419
|
+
const first = buffer[offset];
|
|
420
|
+
if (first === 0x22) {
|
|
421
|
+
let escaped = false;
|
|
422
|
+
for (let i = offset + 1; i < limit; i++) {
|
|
423
|
+
const byte = buffer[i];
|
|
424
|
+
if (escaped) escaped = false;
|
|
425
|
+
else if (byte === 0x5c) escaped = true;
|
|
426
|
+
else if (byte === 0x22) return i + 1;
|
|
427
|
+
}
|
|
428
|
+
return limit;
|
|
429
|
+
}
|
|
430
|
+
if (first === 0x7b || first === 0x5b) {
|
|
431
|
+
let depth = 0;
|
|
432
|
+
let inString = false;
|
|
433
|
+
let escaped = false;
|
|
434
|
+
for (let i = offset; i < limit; i++) {
|
|
435
|
+
const byte = buffer[i];
|
|
436
|
+
if (inString) {
|
|
437
|
+
if (escaped) escaped = false;
|
|
438
|
+
else if (byte === 0x5c) escaped = true;
|
|
439
|
+
else if (byte === 0x22) inString = false;
|
|
440
|
+
continue;
|
|
441
|
+
}
|
|
442
|
+
if (byte === 0x22) inString = true;
|
|
443
|
+
else if (byte === 0x7b || byte === 0x5b) depth++;
|
|
444
|
+
else if (byte === 0x7d || byte === 0x5d) {
|
|
445
|
+
depth--;
|
|
446
|
+
if (depth === 0) return i + 1;
|
|
447
|
+
}
|
|
448
|
+
}
|
|
449
|
+
return limit;
|
|
450
|
+
}
|
|
451
|
+
let end = offset;
|
|
452
|
+
while (end < limit && buffer[end] !== 0x2c && buffer[end] !== 0x5d && buffer[end] !== 0x7d) end++;
|
|
453
|
+
return end;
|
|
454
|
+
}
|
|
455
|
+
|
|
456
|
+
function parseJsonValueAt(buffer: Buffer, offset: number, limit: number): unknown {
|
|
457
|
+
const start = skipJsonWhitespace(buffer, offset, limit);
|
|
458
|
+
const end = jsonValueEnd(buffer, start, limit);
|
|
459
|
+
if (end <= start) return undefined;
|
|
460
|
+
try {
|
|
461
|
+
return JSON.parse(buffer.toString("utf8", start, end));
|
|
462
|
+
} catch {
|
|
463
|
+
return undefined;
|
|
464
|
+
}
|
|
465
|
+
}
|
|
466
|
+
|
|
467
|
+
function parsePropertyValue(buffer: Buffer, property: Buffer, from: number, to: number): unknown {
|
|
468
|
+
const propertyOffset = buffer.indexOf(property, from);
|
|
469
|
+
if (propertyOffset < 0 || propertyOffset >= to) return undefined;
|
|
470
|
+
return parseJsonValueAt(buffer, propertyOffset + property.length, to);
|
|
471
|
+
}
|
|
472
|
+
|
|
473
|
+
function parseLastPropertyValue(buffer: Buffer, property: Buffer, from: number, to: number): unknown {
|
|
474
|
+
const propertyOffset = buffer.lastIndexOf(property, to - 1);
|
|
475
|
+
if (propertyOffset < from) return undefined;
|
|
476
|
+
return parseJsonValueAt(buffer, propertyOffset + property.length, to);
|
|
477
|
+
}
|
|
478
|
+
|
|
479
|
+
interface DirectObjectScan {
|
|
480
|
+
end: number;
|
|
481
|
+
values: Map<string, [start: number, end: number]>;
|
|
482
|
+
}
|
|
483
|
+
|
|
484
|
+
/** Scan direct object properties while skipping nested values allocation-free. */
|
|
485
|
+
function scanDirectObjectProperties(buffer: Buffer, objectStart: number, limit: number, wanted: Set<string>): DirectObjectScan {
|
|
486
|
+
const values = new Map<string, [number, number]>();
|
|
487
|
+
let cursor = objectStart + 1;
|
|
488
|
+
while (cursor < limit) {
|
|
489
|
+
cursor = skipJsonWhitespace(buffer, cursor, limit);
|
|
490
|
+
if (buffer[cursor] === 0x7d) return { end: cursor + 1, values };
|
|
491
|
+
if (buffer[cursor] === 0x2c) {
|
|
492
|
+
cursor++;
|
|
493
|
+
continue;
|
|
494
|
+
}
|
|
495
|
+
if (buffer[cursor] !== 0x22) return { end: jsonValueEnd(buffer, objectStart, limit), values };
|
|
496
|
+
const keyEnd = jsonValueEnd(buffer, cursor, limit);
|
|
497
|
+
let colon = skipJsonWhitespace(buffer, keyEnd, limit);
|
|
498
|
+
if (buffer[colon] !== 0x3a) return { end: jsonValueEnd(buffer, objectStart, limit), values };
|
|
499
|
+
const valueStart = skipJsonWhitespace(buffer, colon + 1, limit);
|
|
500
|
+
const valueEnd = jsonValueEnd(buffer, valueStart, limit);
|
|
501
|
+
const key = buffer.toString("utf8", cursor + 1, keyEnd - 1);
|
|
502
|
+
if (wanted.has(key)) values.set(key, [valueStart, valueEnd]);
|
|
503
|
+
if (valueEnd <= valueStart) return { end: limit, values };
|
|
504
|
+
cursor = valueEnd;
|
|
505
|
+
}
|
|
506
|
+
return { end: limit, values };
|
|
507
|
+
}
|
|
508
|
+
|
|
509
|
+
function parseJsonRange(buffer: Buffer, range: [number, number] | undefined): unknown {
|
|
510
|
+
if (!range) return undefined;
|
|
511
|
+
try {
|
|
512
|
+
return JSON.parse(buffer.toString("utf8", range[0], range[1]));
|
|
513
|
+
} catch {
|
|
514
|
+
return undefined;
|
|
515
|
+
}
|
|
516
|
+
}
|
|
517
|
+
|
|
518
|
+
interface ChildResultsScan {
|
|
519
|
+
end: number;
|
|
520
|
+
results: Array<Record<string, unknown>>;
|
|
521
|
+
}
|
|
522
|
+
|
|
523
|
+
function scanChildResults(buffer: Buffer, arrayStart: number, limit: number): ChildResultsScan {
|
|
524
|
+
const results: Array<Record<string, unknown>> = [];
|
|
525
|
+
let cursor = arrayStart + 1;
|
|
526
|
+
while (cursor < limit) {
|
|
527
|
+
cursor = skipJsonWhitespace(buffer, cursor, limit);
|
|
528
|
+
if (buffer[cursor] === 0x5d) return { end: cursor + 1, results };
|
|
529
|
+
if (buffer[cursor] === 0x2c) {
|
|
530
|
+
cursor++;
|
|
531
|
+
continue;
|
|
532
|
+
}
|
|
533
|
+
if (buffer[cursor] === 0x7b) {
|
|
534
|
+
const scanned = scanDirectObjectProperties(buffer, cursor, limit, DIRECT_CHILD_PROPERTIES);
|
|
535
|
+
results.push({
|
|
536
|
+
usage: parseJsonRange(buffer, scanned.values.get("usage")),
|
|
537
|
+
sessionFile: parseJsonRange(buffer, scanned.values.get("sessionFile")),
|
|
538
|
+
});
|
|
539
|
+
if (scanned.end <= cursor) return { end: limit, results };
|
|
540
|
+
cursor = scanned.end;
|
|
541
|
+
continue;
|
|
542
|
+
}
|
|
543
|
+
const valueEnd = jsonValueEnd(buffer, cursor, limit);
|
|
544
|
+
if (valueEnd <= cursor) return { end: limit, results };
|
|
545
|
+
cursor = valueEnd;
|
|
546
|
+
}
|
|
547
|
+
return { end: limit, results };
|
|
548
|
+
}
|
|
549
|
+
|
|
550
|
+
interface LargeDetailsScan {
|
|
551
|
+
end: number;
|
|
552
|
+
details: Record<string, unknown>;
|
|
553
|
+
}
|
|
554
|
+
|
|
555
|
+
/** Scan nested-agent details and its result metadata in a single byte pass. */
|
|
556
|
+
function scanLargeDetails(buffer: Buffer, objectStart: number, limit: number): LargeDetailsScan {
|
|
557
|
+
const details: Record<string, unknown> = { results: [] };
|
|
558
|
+
let cursor = objectStart + 1;
|
|
559
|
+
while (cursor < limit) {
|
|
560
|
+
cursor = skipJsonWhitespace(buffer, cursor, limit);
|
|
561
|
+
if (buffer[cursor] === 0x7d) return { end: cursor + 1, details };
|
|
562
|
+
if (buffer[cursor] === 0x2c) {
|
|
563
|
+
cursor++;
|
|
564
|
+
continue;
|
|
565
|
+
}
|
|
566
|
+
if (buffer[cursor] !== 0x22) return { end: jsonValueEnd(buffer, objectStart, limit), details };
|
|
567
|
+
const keyEnd = jsonValueEnd(buffer, cursor, limit);
|
|
568
|
+
let colon = skipJsonWhitespace(buffer, keyEnd, limit);
|
|
569
|
+
if (buffer[colon] !== 0x3a) return { end: jsonValueEnd(buffer, objectStart, limit), details };
|
|
570
|
+
const valueStart = skipJsonWhitespace(buffer, colon + 1, limit);
|
|
571
|
+
const key = buffer.toString("utf8", cursor + 1, keyEnd - 1);
|
|
572
|
+
let valueEnd: number;
|
|
573
|
+
if (key === "results" && buffer[valueStart] === 0x5b) {
|
|
574
|
+
const scanned = scanChildResults(buffer, valueStart, limit);
|
|
575
|
+
details.results = scanned.results;
|
|
576
|
+
valueEnd = scanned.end;
|
|
577
|
+
} else {
|
|
578
|
+
valueEnd = jsonValueEnd(buffer, valueStart, limit);
|
|
579
|
+
if (key === "runId" || key === "totalChildUsage") {
|
|
580
|
+
details[key] = parseJsonRange(buffer, [valueStart, valueEnd]);
|
|
581
|
+
}
|
|
582
|
+
}
|
|
583
|
+
if (valueEnd <= valueStart) return { end: limit, details };
|
|
584
|
+
cursor = valueEnd;
|
|
585
|
+
}
|
|
586
|
+
return { end: limit, details };
|
|
587
|
+
}
|
|
588
|
+
|
|
589
|
+
/**
|
|
590
|
+
* Parse only the small accounting fields from a large tool-result line. This
|
|
591
|
+
* avoids UTF-8 decoding and JSON.parse allocation for multi-megabyte content.
|
|
592
|
+
*/
|
|
593
|
+
function parseLargeToolResultLine(line: Buffer): ToolUsageRecord | null {
|
|
594
|
+
const messageOffset = line.indexOf(PROPERTY_MESSAGE);
|
|
595
|
+
if (messageOffset < 0) return null;
|
|
596
|
+
const detailsOffset = line.indexOf(PROPERTY_DETAILS, messageOffset);
|
|
597
|
+
let detailsEnd = -1;
|
|
598
|
+
let details: Record<string, unknown> | null = null;
|
|
599
|
+
if (detailsOffset >= 0) {
|
|
600
|
+
const detailsStart = skipJsonWhitespace(line, detailsOffset + PROPERTY_DETAILS.length, line.length);
|
|
601
|
+
if (line[detailsStart] === 0x7b) {
|
|
602
|
+
const scanned = scanLargeDetails(line, detailsStart, line.length);
|
|
603
|
+
detailsEnd = scanned.end;
|
|
604
|
+
details = scanned.details;
|
|
605
|
+
}
|
|
606
|
+
}
|
|
607
|
+
|
|
608
|
+
const reportedUsage = detailsEnd >= 0
|
|
609
|
+
? parsePropertyValue(line, PROPERTY_USAGE, detailsEnd, line.length)
|
|
610
|
+
: parseLastPropertyValue(line, PROPERTY_USAGE, messageOffset, line.length);
|
|
611
|
+
const sourceId = parsePropertyValue(line, PROPERTY_ID, 0, messageOffset);
|
|
612
|
+
const entryTimestamp = parsePropertyValue(line, PROPERTY_TIMESTAMP, 0, messageOffset);
|
|
613
|
+
const messageTimestamp = parseLastPropertyValue(line, PROPERTY_TIMESTAMP, messageOffset, line.length);
|
|
614
|
+
const toolName = parsePropertyValue(line, PROPERTY_TOOL_NAME, messageOffset, detailsOffset >= 0 ? detailsOffset : line.length);
|
|
615
|
+
return buildToolUsageRecord(toolName, details, reportedUsage, sourceId, messageTimestamp, entryTimestamp);
|
|
616
|
+
}
|
|
617
|
+
|
|
264
618
|
function lineMightBeRelevant(line: Buffer): boolean {
|
|
619
|
+
// Entry type/role fields are at the front of Pi's JSONL objects. Restrict
|
|
620
|
+
// those checks to a small prefix so multi-megabyte tool output is not scanned
|
|
621
|
+
// repeatedly for every possible entry shape.
|
|
622
|
+
const head = line.length > 1024 ? line.subarray(0, 1024) : line;
|
|
623
|
+
if (head.includes(PATTERN_ASSISTANT_COMPACT) || head.includes(PATTERN_ASSISTANT_SPACED)) return true;
|
|
624
|
+
|
|
625
|
+
if (head.includes(PATTERN_TOOL_RESULT_COMPACT) || head.includes(PATTERN_TOOL_RESULT_SPACED)) {
|
|
626
|
+
if (
|
|
627
|
+
head.includes(PATTERN_SUBAGENT_TOOL_COMPACT) ||
|
|
628
|
+
head.includes(PATTERN_SUBAGENT_TOOL_SPACED) ||
|
|
629
|
+
head.includes(PATTERN_SUBAGENT_WAIT_TOOL_COMPACT) ||
|
|
630
|
+
head.includes(PATTERN_SUBAGENT_WAIT_TOOL_SPACED)
|
|
631
|
+
) {
|
|
632
|
+
return true;
|
|
633
|
+
}
|
|
634
|
+
// Pi serializes optional tool usage after content/details and immediately
|
|
635
|
+
// before the small isError/timestamp suffix. Checking the tail preserves
|
|
636
|
+
// the fast path for ordinary tool results, which dominate session bytes.
|
|
637
|
+
const tail = line.length > 4096 ? line.subarray(line.length - 4096) : line;
|
|
638
|
+
return tail.includes(PATTERN_USAGE_COMPACT) || tail.includes(PATTERN_USAGE_SPACED);
|
|
639
|
+
}
|
|
640
|
+
|
|
265
641
|
return (
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
642
|
+
head.includes(PATTERN_SESSION_COMPACT) ||
|
|
643
|
+
head.includes(PATTERN_THINKING_COMPACT) ||
|
|
644
|
+
head.includes(PATTERN_COMPACTION_COMPACT) ||
|
|
645
|
+
head.includes(PATTERN_BRANCH_SUMMARY_COMPACT) ||
|
|
646
|
+
head.includes(PATTERN_SESSION_SPACED) ||
|
|
647
|
+
head.includes(PATTERN_THINKING_SPACED) ||
|
|
648
|
+
head.includes(PATTERN_COMPACTION_SPACED) ||
|
|
649
|
+
head.includes(PATTERN_BRANCH_SUMMARY_SPACED)
|
|
274
650
|
);
|
|
275
651
|
}
|
|
276
652
|
|
|
277
653
|
/**
|
|
278
|
-
* Extract the session id
|
|
654
|
+
* Extract the session id plus assistant/tool/summary usage from a JSONL buffer.
|
|
279
655
|
* Returns partial results when aborted — callers must check `signal.aborted`
|
|
280
656
|
* before caching or using the result.
|
|
281
657
|
*/
|
|
282
658
|
export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal): Promise<ParsedSessionFile> {
|
|
283
659
|
const messages: SessionMessage[] = [];
|
|
660
|
+
const toolUsages: ToolUsageRecord[] = [];
|
|
284
661
|
let sessionId = "";
|
|
285
662
|
let cwd = "";
|
|
286
663
|
// Assistant messages don't carry the thinking level; pi records it as separate
|
|
@@ -289,6 +666,7 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
289
666
|
// to the level active when it was produced.
|
|
290
667
|
let thinkingLevel = "";
|
|
291
668
|
let compactionPending = false;
|
|
669
|
+
|
|
292
670
|
let start = 0;
|
|
293
671
|
let lineNumber = 0;
|
|
294
672
|
|
|
@@ -299,10 +677,18 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
299
677
|
lineNumber++;
|
|
300
678
|
if (lineNumber % PARSE_YIELD_EVERY_LINES === 0) {
|
|
301
679
|
await new Promise<void>((resolve) => setImmediate(resolve));
|
|
302
|
-
if (signal?.aborted) return { sessionId, cwd, messages };
|
|
680
|
+
if (signal?.aborted) return { sessionId, cwd, messages, toolUsages };
|
|
303
681
|
}
|
|
304
682
|
|
|
305
|
-
|
|
683
|
+
const lineBuffer = buffer.subarray(start, end);
|
|
684
|
+
if (end > start && lineMightBeRelevant(lineBuffer)) {
|
|
685
|
+
const head = lineBuffer.subarray(0, Math.min(1024, lineBuffer.length));
|
|
686
|
+
if (lineBuffer.length > LARGE_TOOL_RESULT_BYTES && head.includes(PATTERN_TOOL_RESULT_COMPACT)) {
|
|
687
|
+
const toolUsage = parseLargeToolResultLine(lineBuffer);
|
|
688
|
+
if (toolUsage) toolUsages.push(toolUsage);
|
|
689
|
+
start = end + 1;
|
|
690
|
+
continue;
|
|
691
|
+
}
|
|
306
692
|
try {
|
|
307
693
|
const entry = JSON.parse(buffer.toString("utf8", start, end));
|
|
308
694
|
|
|
@@ -312,7 +698,12 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
312
698
|
} else if (entry.type === "thinking_level_change") {
|
|
313
699
|
if (typeof entry.thinkingLevel === "string") thinkingLevel = entry.thinkingLevel;
|
|
314
700
|
} else if (entry.type === "compaction") {
|
|
701
|
+
const usage = parseUsageAmount(entry.usage);
|
|
702
|
+
if (usage) messages.push(auxiliaryMessage(usage, parsedTimestamp(undefined, entry.timestamp), typeof entry.id === "string" ? entry.id : ""));
|
|
315
703
|
compactionPending = true;
|
|
704
|
+
} else if (entry.type === "branch_summary") {
|
|
705
|
+
const usage = parseUsageAmount(entry.usage);
|
|
706
|
+
if (usage) messages.push(auxiliaryMessage(usage, parsedTimestamp(undefined, entry.timestamp), typeof entry.id === "string" ? entry.id : ""));
|
|
316
707
|
} else if (entry.type === "message" && entry.message?.role === "assistant") {
|
|
317
708
|
const msg = entry.message;
|
|
318
709
|
if (msg.usage && msg.provider && msg.model) {
|
|
@@ -321,6 +712,8 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
321
712
|
provider: msg.provider,
|
|
322
713
|
model: msg.model,
|
|
323
714
|
thinkingLevel,
|
|
715
|
+
source: "assistant",
|
|
716
|
+
sourceId: "",
|
|
324
717
|
cost: msg.usage.cost?.total || 0,
|
|
325
718
|
input: msg.usage.input || 0,
|
|
326
719
|
output: msg.usage.output || 0,
|
|
@@ -332,6 +725,10 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
332
725
|
});
|
|
333
726
|
compactionPending = false;
|
|
334
727
|
}
|
|
728
|
+
} else if (entry.type === "message" && entry.message?.role === "toolResult") {
|
|
729
|
+
const msg = entry.message;
|
|
730
|
+
const toolUsage = buildToolUsageRecord(msg.toolName, msg.details, msg.usage, entry.id, msg.timestamp, entry.timestamp);
|
|
731
|
+
if (toolUsage) toolUsages.push(toolUsage);
|
|
335
732
|
}
|
|
336
733
|
} catch {
|
|
337
734
|
// Skip malformed lines
|
|
@@ -341,14 +738,14 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
341
738
|
start = end + 1;
|
|
342
739
|
}
|
|
343
740
|
|
|
344
|
-
return { sessionId, cwd, messages };
|
|
741
|
+
return { sessionId, cwd, messages, toolUsages };
|
|
345
742
|
}
|
|
346
743
|
|
|
347
744
|
// =============================================================================
|
|
348
745
|
// On-disk cache
|
|
349
746
|
// =============================================================================
|
|
350
747
|
|
|
351
|
-
const CACHE_VERSION =
|
|
748
|
+
const CACHE_VERSION = 5;
|
|
352
749
|
|
|
353
750
|
type CachedMessageTuple = [
|
|
354
751
|
providerIdx: number,
|
|
@@ -362,6 +759,31 @@ type CachedMessageTuple = [
|
|
|
362
759
|
thinkingLevelIdx: number,
|
|
363
760
|
reasoning: number,
|
|
364
761
|
afterCompaction: 0 | 1,
|
|
762
|
+
auxiliary: 0 | 1,
|
|
763
|
+
sourceIdIdx: number,
|
|
764
|
+
];
|
|
765
|
+
|
|
766
|
+
type CachedUsageTuple = [
|
|
767
|
+
cost: number,
|
|
768
|
+
input: number,
|
|
769
|
+
output: number,
|
|
770
|
+
cacheRead: number,
|
|
771
|
+
cacheWrite: number,
|
|
772
|
+
reasoning: number,
|
|
773
|
+
];
|
|
774
|
+
|
|
775
|
+
type CachedChildToolUsageTuple = [
|
|
776
|
+
resultIndex: number,
|
|
777
|
+
sessionFileIdx: number,
|
|
778
|
+
usage: CachedUsageTuple,
|
|
779
|
+
];
|
|
780
|
+
|
|
781
|
+
type CachedToolUsageTuple = [
|
|
782
|
+
sourceIdIdx: number,
|
|
783
|
+
timestamp: number,
|
|
784
|
+
reportedUsage: CachedUsageTuple | null,
|
|
785
|
+
runIdIdx: number,
|
|
786
|
+
children: CachedChildToolUsageTuple[],
|
|
365
787
|
];
|
|
366
788
|
|
|
367
789
|
interface CacheFileEntry {
|
|
@@ -370,6 +792,7 @@ interface CacheFileEntry {
|
|
|
370
792
|
sessionId: string;
|
|
371
793
|
cwd: string;
|
|
372
794
|
messages: CachedMessageTuple[];
|
|
795
|
+
toolUsages: CachedToolUsageTuple[];
|
|
373
796
|
}
|
|
374
797
|
|
|
375
798
|
export interface CachedFileState {
|
|
@@ -398,21 +821,29 @@ export async function loadUsageCache(cachePath: string): Promise<Map<string, Cac
|
|
|
398
821
|
typeof entry.mtimeMs !== "number" ||
|
|
399
822
|
typeof entry.sessionId !== "string" ||
|
|
400
823
|
typeof entry.cwd !== "string" ||
|
|
401
|
-
!Array.isArray(entry.messages)
|
|
824
|
+
!Array.isArray(entry.messages) ||
|
|
825
|
+
!Array.isArray(entry.toolUsages)
|
|
402
826
|
) {
|
|
403
827
|
continue;
|
|
404
828
|
}
|
|
405
829
|
const messages: SessionMessage[] = [];
|
|
406
830
|
let valid = true;
|
|
407
831
|
for (const tuple of entry.messages) {
|
|
408
|
-
if (!Array.isArray(tuple) || tuple.length !==
|
|
832
|
+
if (!Array.isArray(tuple) || tuple.length !== 13) {
|
|
409
833
|
valid = false;
|
|
410
834
|
break;
|
|
411
835
|
}
|
|
412
836
|
const provider = names[tuple[0]];
|
|
413
837
|
const model = names[tuple[1]];
|
|
414
838
|
const thinkingLevel = names[tuple[8]];
|
|
415
|
-
|
|
839
|
+
const sourceId = names[tuple[12]];
|
|
840
|
+
if (
|
|
841
|
+
typeof provider !== "string" ||
|
|
842
|
+
typeof model !== "string" ||
|
|
843
|
+
typeof thinkingLevel !== "string" ||
|
|
844
|
+
typeof sourceId !== "string" ||
|
|
845
|
+
(tuple[11] !== 0 && tuple[11] !== 1)
|
|
846
|
+
) {
|
|
416
847
|
valid = false;
|
|
417
848
|
break;
|
|
418
849
|
}
|
|
@@ -420,6 +851,8 @@ export async function loadUsageCache(cachePath: string): Promise<Map<string, Cac
|
|
|
420
851
|
provider,
|
|
421
852
|
model,
|
|
422
853
|
thinkingLevel,
|
|
854
|
+
source: tuple[11] === 1 ? "auxiliary" : "assistant",
|
|
855
|
+
sourceId,
|
|
423
856
|
cost: Number(tuple[2]) || 0,
|
|
424
857
|
input: Number(tuple[3]) || 0,
|
|
425
858
|
output: Number(tuple[4]) || 0,
|
|
@@ -431,15 +864,70 @@ export async function loadUsageCache(cachePath: string): Promise<Map<string, Cac
|
|
|
431
864
|
});
|
|
432
865
|
}
|
|
433
866
|
if (!valid) continue;
|
|
867
|
+
const toolUsages: ToolUsageRecord[] = [];
|
|
868
|
+
for (const tuple of entry.toolUsages) {
|
|
869
|
+
if (!Array.isArray(tuple) || tuple.length !== 5 || !Array.isArray(tuple[4])) {
|
|
870
|
+
valid = false;
|
|
871
|
+
break;
|
|
872
|
+
}
|
|
873
|
+
const sourceId = names[tuple[0]];
|
|
874
|
+
const runId = names[tuple[3]];
|
|
875
|
+
const reportedUsage = cachedUsageAmount(tuple[2]);
|
|
876
|
+
if (typeof sourceId !== "string" || typeof runId !== "string" || (tuple[2] !== null && !reportedUsage)) {
|
|
877
|
+
valid = false;
|
|
878
|
+
break;
|
|
879
|
+
}
|
|
880
|
+
const children: ChildToolUsage[] = [];
|
|
881
|
+
for (const childTuple of tuple[4]) {
|
|
882
|
+
if (!Array.isArray(childTuple) || childTuple.length !== 3) {
|
|
883
|
+
valid = false;
|
|
884
|
+
break;
|
|
885
|
+
}
|
|
886
|
+
const sessionFile = names[childTuple[1]];
|
|
887
|
+
const usage = cachedUsageAmount(childTuple[2]);
|
|
888
|
+
if (typeof childTuple[0] !== "number" || typeof sessionFile !== "string" || !usage) {
|
|
889
|
+
valid = false;
|
|
890
|
+
break;
|
|
891
|
+
}
|
|
892
|
+
children.push({ resultIndex: childTuple[0], sessionFile, usage });
|
|
893
|
+
}
|
|
894
|
+
if (!valid) break;
|
|
895
|
+
toolUsages.push({
|
|
896
|
+
sourceId,
|
|
897
|
+
timestamp: Number(tuple[1]) || 0,
|
|
898
|
+
reportedUsage,
|
|
899
|
+
runId,
|
|
900
|
+
children,
|
|
901
|
+
});
|
|
902
|
+
}
|
|
903
|
+
if (!valid) continue;
|
|
434
904
|
result.set(filePath, {
|
|
435
905
|
size: entry.size,
|
|
436
906
|
mtimeMs: entry.mtimeMs,
|
|
437
|
-
parsed: { sessionId: entry.sessionId, cwd: entry.cwd, messages },
|
|
907
|
+
parsed: { sessionId: entry.sessionId, cwd: entry.cwd, messages, toolUsages },
|
|
438
908
|
});
|
|
439
909
|
}
|
|
440
910
|
return result;
|
|
441
911
|
}
|
|
442
912
|
|
|
913
|
+
function cachedUsageAmount(value: unknown): UsageAmount | null {
|
|
914
|
+
if (!Array.isArray(value) || value.length !== 6 || value.some((part) => typeof part !== "number" || !Number.isFinite(part))) {
|
|
915
|
+
return null;
|
|
916
|
+
}
|
|
917
|
+
return {
|
|
918
|
+
cost: value[0],
|
|
919
|
+
input: value[1],
|
|
920
|
+
output: value[2],
|
|
921
|
+
cacheRead: value[3],
|
|
922
|
+
cacheWrite: value[4],
|
|
923
|
+
reasoning: value[5],
|
|
924
|
+
};
|
|
925
|
+
}
|
|
926
|
+
|
|
927
|
+
function cacheUsageAmount(usage: UsageAmount): CachedUsageTuple {
|
|
928
|
+
return [usage.cost, usage.input, usage.output, usage.cacheRead, usage.cacheWrite, usage.reasoning];
|
|
929
|
+
}
|
|
930
|
+
|
|
443
931
|
export async function saveUsageCache(cachePath: string, states: Map<string, CachedFileState>): Promise<void> {
|
|
444
932
|
const names: string[] = [];
|
|
445
933
|
const nameIndex = new Map<string, number>();
|
|
@@ -472,6 +960,19 @@ export async function saveUsageCache(cachePath: string, states: Map<string, Cach
|
|
|
472
960
|
intern(m.thinkingLevel),
|
|
473
961
|
m.reasoning,
|
|
474
962
|
m.afterCompaction ? 1 : 0,
|
|
963
|
+
m.source === "auxiliary" ? 1 : 0,
|
|
964
|
+
intern(m.source === "auxiliary" ? m.sourceId : ""),
|
|
965
|
+
]),
|
|
966
|
+
toolUsages: state.parsed.toolUsages.map((tool): CachedToolUsageTuple => [
|
|
967
|
+
intern(tool.sourceId),
|
|
968
|
+
tool.timestamp,
|
|
969
|
+
tool.reportedUsage ? cacheUsageAmount(tool.reportedUsage) : null,
|
|
970
|
+
intern(tool.runId),
|
|
971
|
+
tool.children.map((child): CachedChildToolUsageTuple => [
|
|
972
|
+
child.resultIndex,
|
|
973
|
+
intern(child.sessionFile),
|
|
974
|
+
cacheUsageAmount(child.usage),
|
|
975
|
+
]),
|
|
475
976
|
]),
|
|
476
977
|
};
|
|
477
978
|
}
|
|
@@ -511,7 +1012,8 @@ function emptyTimeFilteredStats(): TimeFilteredStats {
|
|
|
511
1012
|
function emptyPeriodRawData(): PeriodRawData {
|
|
512
1013
|
return {
|
|
513
1014
|
totalCost: 0,
|
|
514
|
-
|
|
1015
|
+
assistantCost: 0,
|
|
1016
|
+
auxiliaryCost: 0,
|
|
515
1017
|
ctxHigh: { cost: 0, messages: 0 },
|
|
516
1018
|
ctxLow: { cost: 0, messages: 0 },
|
|
517
1019
|
projectCosts: new Map(),
|
|
@@ -581,7 +1083,7 @@ function addToHourlyBuckets(hourly: Map<number, Map<HourlyKey, HourlyCell>>, msg
|
|
|
581
1083
|
cell = { messages: 0, cost: 0, input: 0, output: 0, cacheRead: 0, cacheWrite: 0, reasoning: 0 };
|
|
582
1084
|
bucket.set(key, cell);
|
|
583
1085
|
}
|
|
584
|
-
cell.messages++;
|
|
1086
|
+
if (msg.source === "assistant") cell.messages++;
|
|
585
1087
|
cell.cost += msg.cost;
|
|
586
1088
|
cell.input += msg.input;
|
|
587
1089
|
cell.output += msg.output;
|
|
@@ -594,9 +1096,10 @@ function addToHourlyBuckets(hourly: Map<number, Map<HourlyKey, HourlyCell>>, msg
|
|
|
594
1096
|
function accumulateStats(
|
|
595
1097
|
target: BaseStats,
|
|
596
1098
|
cost: number,
|
|
597
|
-
tokens: { total: number; input: number; output: number; cacheRead: number; cacheWrite: number }
|
|
1099
|
+
tokens: { total: number; input: number; output: number; cacheRead: number; cacheWrite: number },
|
|
1100
|
+
countMessage: boolean
|
|
598
1101
|
): void {
|
|
599
|
-
target.messages++;
|
|
1102
|
+
if (countMessage) target.messages++;
|
|
600
1103
|
target.cost += cost;
|
|
601
1104
|
target.tokens.total += tokens.total;
|
|
602
1105
|
target.tokens.input += tokens.input;
|
|
@@ -683,18 +1186,30 @@ function addMessagesToUsageData(
|
|
|
683
1186
|
providerStats.models.set(msg.model, modelStats);
|
|
684
1187
|
}
|
|
685
1188
|
|
|
1189
|
+
const isAssistant = msg.source === "assistant";
|
|
686
1190
|
modelStats.sessions.add(sessionId);
|
|
687
|
-
accumulateStats(modelStats, msg.cost, tokens);
|
|
1191
|
+
accumulateStats(modelStats, msg.cost, tokens, isAssistant);
|
|
688
1192
|
|
|
689
1193
|
providerStats.sessions.add(sessionId);
|
|
690
|
-
accumulateStats(providerStats, msg.cost, tokens);
|
|
1194
|
+
accumulateStats(providerStats, msg.cost, tokens, isAssistant);
|
|
691
1195
|
|
|
692
|
-
accumulateStats(stats.totals, msg.cost, tokens);
|
|
1196
|
+
accumulateStats(stats.totals, msg.cost, tokens, isAssistant);
|
|
693
1197
|
sessionContributed[period] = true;
|
|
694
1198
|
|
|
695
1199
|
const raw = rawByPeriod[period];
|
|
696
1200
|
raw.totalCost += msg.cost;
|
|
697
|
-
raw.
|
|
1201
|
+
raw.projectCosts.set(project, (raw.projectCosts.get(project) ?? 0) + msg.cost);
|
|
1202
|
+
raw.sessionCosts.set(sessionId, (raw.sessionCosts.get(sessionId) ?? 0) + msg.cost);
|
|
1203
|
+
|
|
1204
|
+
// Auxiliary calls belong in accounting totals, project/session mix, and
|
|
1205
|
+
// burn trend. They are not assistant turns, so do not let their synthetic
|
|
1206
|
+
// model identity or nested context distort turn/cache insights.
|
|
1207
|
+
if (!isAssistant) {
|
|
1208
|
+
raw.auxiliaryCost += msg.cost;
|
|
1209
|
+
continue;
|
|
1210
|
+
}
|
|
1211
|
+
raw.assistantCost += msg.cost;
|
|
1212
|
+
|
|
698
1213
|
const ctx = msg.input + msg.cacheRead + msg.cacheWrite;
|
|
699
1214
|
if (ctx >= CTX_TAX_THRESHOLD) {
|
|
700
1215
|
raw.ctxHigh.cost += msg.cost;
|
|
@@ -703,8 +1218,6 @@ function addMessagesToUsageData(
|
|
|
703
1218
|
raw.ctxLow.cost += msg.cost;
|
|
704
1219
|
raw.ctxLow.messages++;
|
|
705
1220
|
}
|
|
706
|
-
raw.projectCosts.set(project, (raw.projectCosts.get(project) ?? 0) + msg.cost);
|
|
707
|
-
raw.sessionCosts.set(sessionId, (raw.sessionCosts.get(sessionId) ?? 0) + msg.cost);
|
|
708
1221
|
if (mm.isSessionStart) raw.upfrontCost += msg.cost;
|
|
709
1222
|
if (
|
|
710
1223
|
!msg.afterCompaction &&
|
|
@@ -729,6 +1242,103 @@ function addMessagesToUsageData(
|
|
|
729
1242
|
if (sessionContributed.allTime) data.allTime.totals.sessions++;
|
|
730
1243
|
}
|
|
731
1244
|
|
|
1245
|
+
// =============================================================================
|
|
1246
|
+
// Nested tool-usage accounting
|
|
1247
|
+
// =============================================================================
|
|
1248
|
+
|
|
1249
|
+
// Recognised nested-agent tool results report usage for child runs that
|
|
1250
|
+
// pi-subagents also persists as ordinary session files, which this scan
|
|
1251
|
+
// already counts with full model attribution. When every child session file
|
|
1252
|
+
// behind a report is part of the scan, the children speak for themselves and
|
|
1253
|
+
// the parent's aggregate is skipped. Otherwise the aggregate (or, for pre-0.81
|
|
1254
|
+
// legacy entries, each unresolved child's reported usage) is counted under
|
|
1255
|
+
// Tools / summaries.
|
|
1256
|
+
//
|
|
1257
|
+
// Copied branch history gets a new parent filename, so a copy's runId-derived
|
|
1258
|
+
// child paths can dangle even though the original's resolve. Resolved child
|
|
1259
|
+
// identities are therefore unioned across all copies of an entry before any
|
|
1260
|
+
// emission, and identical emissions collapse in the sourceId dedupe.
|
|
1261
|
+
|
|
1262
|
+
interface ScannedSessionIndex {
|
|
1263
|
+
/** Resolved paths of scanned files that have a session header. */
|
|
1264
|
+
paths: Set<string>;
|
|
1265
|
+
/** Directory of each scanned file → number of scanned files inside it. */
|
|
1266
|
+
fileCountByDir: Map<string, number>;
|
|
1267
|
+
}
|
|
1268
|
+
|
|
1269
|
+
function buildScannedSessionIndex(states: Map<string, CachedFileState>): ScannedSessionIndex {
|
|
1270
|
+
const paths = new Set<string>();
|
|
1271
|
+
const fileCountByDir = new Map<string, number>();
|
|
1272
|
+
for (const [filePath, state] of states) {
|
|
1273
|
+
if (!state.parsed.sessionId) continue;
|
|
1274
|
+
const resolved = resolve(filePath);
|
|
1275
|
+
paths.add(resolved);
|
|
1276
|
+
const dir = dirname(resolved);
|
|
1277
|
+
fileCountByDir.set(dir, (fileCountByDir.get(dir) ?? 0) + 1);
|
|
1278
|
+
}
|
|
1279
|
+
return { paths, fileCountByDir };
|
|
1280
|
+
}
|
|
1281
|
+
|
|
1282
|
+
function childSessionScanned(
|
|
1283
|
+
parentFilePath: string,
|
|
1284
|
+
tool: ToolUsageRecord,
|
|
1285
|
+
child: ChildToolUsage,
|
|
1286
|
+
index: ScannedSessionIndex
|
|
1287
|
+
): boolean {
|
|
1288
|
+
if (child.sessionFile) {
|
|
1289
|
+
const explicit = isAbsolute(child.sessionFile)
|
|
1290
|
+
? resolve(child.sessionFile)
|
|
1291
|
+
: resolve(dirname(parentFilePath), child.sessionFile);
|
|
1292
|
+
if (index.paths.has(explicit)) return true;
|
|
1293
|
+
}
|
|
1294
|
+
if (tool.runId) {
|
|
1295
|
+
const runDir = resolve(dirname(parentFilePath), basename(parentFilePath, ".jsonl"), tool.runId, `run-${child.resultIndex}`);
|
|
1296
|
+
// The run directory holds exactly one session per child run, so either the
|
|
1297
|
+
// conventional name or a lone scanned file inside it identifies the child.
|
|
1298
|
+
if (index.paths.has(join(runDir, "session.jsonl"))) return true;
|
|
1299
|
+
if (index.fileCountByDir.get(runDir) === 1) return true;
|
|
1300
|
+
}
|
|
1301
|
+
return false;
|
|
1302
|
+
}
|
|
1303
|
+
|
|
1304
|
+
/** Identity of one child slot of one tool entry, stable across copied history. */
|
|
1305
|
+
function toolChildIdentity(tool: ToolUsageRecord, child: ChildToolUsage): string {
|
|
1306
|
+
const fingerprint = child.usage.input + child.usage.output + child.usage.cacheRead + child.usage.cacheWrite;
|
|
1307
|
+
return `${tool.sourceId}:${tool.timestamp}:${child.resultIndex}:${fingerprint}`;
|
|
1308
|
+
}
|
|
1309
|
+
|
|
1310
|
+
function resolvedToolChildIdentities(states: Map<string, CachedFileState>, index: ScannedSessionIndex): Set<string> {
|
|
1311
|
+
const resolved = new Set<string>();
|
|
1312
|
+
for (const [filePath, state] of states) {
|
|
1313
|
+
for (const tool of state.parsed.toolUsages) {
|
|
1314
|
+
if (!tool.sourceId) continue;
|
|
1315
|
+
for (const child of tool.children) {
|
|
1316
|
+
if (childSessionScanned(filePath, tool, child, index)) resolved.add(toolChildIdentity(tool, child));
|
|
1317
|
+
}
|
|
1318
|
+
}
|
|
1319
|
+
}
|
|
1320
|
+
return resolved;
|
|
1321
|
+
}
|
|
1322
|
+
|
|
1323
|
+
function toolUsageMessages(
|
|
1324
|
+
parentFilePath: string,
|
|
1325
|
+
tool: ToolUsageRecord,
|
|
1326
|
+
index: ScannedSessionIndex,
|
|
1327
|
+
resolvedChildren: Set<string>
|
|
1328
|
+
): SessionMessage[] {
|
|
1329
|
+
const scanned = (child: ChildToolUsage) =>
|
|
1330
|
+
childSessionScanned(parentFilePath, tool, child, index) ||
|
|
1331
|
+
(tool.sourceId !== "" && resolvedChildren.has(toolChildIdentity(tool, child)));
|
|
1332
|
+
if (tool.reportedUsage) {
|
|
1333
|
+
if (tool.children.length > 0 && tool.children.every(scanned)) return [];
|
|
1334
|
+
return [auxiliaryMessage(tool.reportedUsage, tool.timestamp, tool.sourceId)];
|
|
1335
|
+
}
|
|
1336
|
+
// Before Pi 0.81, recognised nested-agent tools persisted per-child usage in
|
|
1337
|
+
// details only. Count just the children whose sessions this scan cannot see.
|
|
1338
|
+
return tool.children
|
|
1339
|
+
.filter((child) => !scanned(child))
|
|
1340
|
+
.map((child) => auxiliaryMessage(child.usage, tool.timestamp, tool.sourceId ? `${tool.sourceId}:child:${child.resultIndex}` : ""));
|
|
1341
|
+
}
|
|
732
1342
|
// =============================================================================
|
|
733
1343
|
// Collection orchestration
|
|
734
1344
|
// =============================================================================
|
|
@@ -925,6 +1535,8 @@ export async function collectUsageData(options: CollectUsageOptions = {}): Promi
|
|
|
925
1535
|
const costByDayIdx = new Map<number, number>();
|
|
926
1536
|
const seenSessions = new Set<string>();
|
|
927
1537
|
const seenHashes = new Set<string>();
|
|
1538
|
+
const scannedSessions = buildScannedSessionIndex(current);
|
|
1539
|
+
const resolvedToolChildren = resolvedToolChildIdentities(current, scannedSessions);
|
|
928
1540
|
let processedFiles = 0;
|
|
929
1541
|
|
|
930
1542
|
for (const filePath of filePaths) {
|
|
@@ -939,15 +1551,26 @@ export async function collectUsageData(options: CollectUsageOptions = {}): Promi
|
|
|
939
1551
|
// Deduplicate copied history across branched session files, computing
|
|
940
1552
|
// adjacency metadata (idle gaps, previous context) on the raw file order
|
|
941
1553
|
// so branch copies do not distort miss classification.
|
|
942
|
-
const
|
|
1554
|
+
const toolMessages = state.parsed.toolUsages.flatMap((tool) => toolUsageMessages(filePath, tool, scannedSessions, resolvedToolChildren));
|
|
1555
|
+
const rawMsgs = toolMessages.length > 0 ? [...state.parsed.messages, ...toolMessages] : state.parsed.messages;
|
|
943
1556
|
const deduped: SessionMessage[] = [];
|
|
944
1557
|
const meta: MessageMeta[] = [];
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
1558
|
+
let previousAssistant: SessionMessage | null = null;
|
|
1559
|
+
for (const m of rawMsgs) {
|
|
1560
|
+
// Auxiliary usage is interleaved with conversation entries, but it must
|
|
1561
|
+
// not become the "previous message" for cache-miss classification.
|
|
1562
|
+
const prev = m.source === "assistant" ? previousAssistant : null;
|
|
1563
|
+
if (m.source === "assistant") previousAssistant = m;
|
|
1564
|
+
|
|
1565
|
+
// Pi entry ids survive copied branch history and distinguish parallel
|
|
1566
|
+
// tool results that happen to report identical usage in the same ms.
|
|
1567
|
+
const tokenFingerprint = m.input + m.output + m.cacheRead + m.cacheWrite;
|
|
1568
|
+
const hash =
|
|
1569
|
+
m.source === "auxiliary" && m.sourceId
|
|
1570
|
+
? `auxiliary:${m.sourceId}:${m.timestamp}:${tokenFingerprint}`
|
|
1571
|
+
: `${m.source}:${m.timestamp}:${tokenFingerprint}`;
|
|
948
1572
|
if (seenHashes.has(hash)) continue;
|
|
949
1573
|
seenHashes.add(hash);
|
|
950
|
-
const prev = i > 0 ? rawMsgs[i - 1]! : null;
|
|
951
1574
|
deduped.push(m);
|
|
952
1575
|
meta.push({
|
|
953
1576
|
gapMs: prev && prev.timestamp > 0 && m.timestamp > 0 ? m.timestamp - prev.timestamp : -1,
|
|
@@ -957,9 +1580,10 @@ export async function collectUsageData(options: CollectUsageOptions = {}): Promi
|
|
|
957
1580
|
});
|
|
958
1581
|
}
|
|
959
1582
|
if (deduped.length === 0) continue;
|
|
960
|
-
|
|
1583
|
+
const firstAssistantIndex = deduped.findIndex((m) => m.source === "assistant");
|
|
1584
|
+
if (firstAssistantIndex !== -1 && !seenSessions.has(state.parsed.sessionId)) {
|
|
961
1585
|
seenSessions.add(state.parsed.sessionId);
|
|
962
|
-
meta[
|
|
1586
|
+
meta[firstAssistantIndex]!.isSessionStart = true;
|
|
963
1587
|
}
|
|
964
1588
|
|
|
965
1589
|
addMessagesToUsageData(
|
|
@@ -1045,42 +1669,44 @@ function fmtPercent(p: number): string {
|
|
|
1045
1669
|
* distinct empty-state for that case.
|
|
1046
1670
|
*/
|
|
1047
1671
|
function computeInsights(raw: PeriodRawData, trend: TrendInfo | null): PeriodInsights {
|
|
1048
|
-
if (raw.
|
|
1672
|
+
if (raw.totalCost <= 0) {
|
|
1049
1673
|
return { insights: [] };
|
|
1050
1674
|
}
|
|
1051
1675
|
const total = raw.totalCost;
|
|
1676
|
+
const assistantTotal = raw.assistantCost;
|
|
1677
|
+
const assistantPctLabel = raw.auxiliaryCost > 0 ? "assistant-message cost" : "this period";
|
|
1052
1678
|
const insights: Insight[] = [];
|
|
1053
1679
|
|
|
1054
1680
|
// --- Alarms (listed first) ---
|
|
1055
1681
|
|
|
1056
|
-
const ttlPct = (raw.ttlMissCost /
|
|
1682
|
+
const ttlPct = assistantTotal > 0 ? (raw.ttlMissCost / assistantTotal) * 100 : 0;
|
|
1057
1683
|
if (ttlPct >= CACHE_MISS_ALARM_PERCENT && raw.ttlMissCost >= CACHE_MISS_ALARM_MIN_COST) {
|
|
1058
1684
|
insights.push({
|
|
1059
1685
|
kind: "alarm",
|
|
1060
1686
|
stat: fmtMoney(raw.ttlMissCost),
|
|
1061
|
-
headline: `spent resuming conversations after a break (${fmtPercent(ttlPct)} of
|
|
1687
|
+
headline: `spent resuming conversations after a break (${fmtPercent(ttlPct)} of ${assistantPctLabel})`,
|
|
1062
1688
|
advice:
|
|
1063
1689
|
"Sent context is only reusable for a few minutes. After a longer pause, the next message pays to send the whole conversation again. Replying while a session is fresh avoids this.",
|
|
1064
1690
|
});
|
|
1065
1691
|
}
|
|
1066
1692
|
|
|
1067
|
-
const switchPct = (raw.modelSwitchMissCost /
|
|
1693
|
+
const switchPct = assistantTotal > 0 ? (raw.modelSwitchMissCost / assistantTotal) * 100 : 0;
|
|
1068
1694
|
if (switchPct >= CACHE_MISS_ALARM_PERCENT && raw.modelSwitchMissCost >= CACHE_MISS_ALARM_MIN_COST) {
|
|
1069
1695
|
insights.push({
|
|
1070
1696
|
kind: "alarm",
|
|
1071
1697
|
stat: fmtMoney(raw.modelSwitchMissCost),
|
|
1072
|
-
headline: `spent switching models mid-conversation (${fmtPercent(switchPct)} of
|
|
1698
|
+
headline: `spent switching models mid-conversation (${fmtPercent(switchPct)} of ${assistantPctLabel})`,
|
|
1073
1699
|
advice:
|
|
1074
1700
|
"Changing model re-sends the whole conversation at full price — the previous model's saved context doesn't transfer. Switching between tasks instead of mid-conversation avoids this.",
|
|
1075
1701
|
});
|
|
1076
1702
|
}
|
|
1077
1703
|
|
|
1078
|
-
const prefixPct = (raw.prefixMissCost /
|
|
1704
|
+
const prefixPct = assistantTotal > 0 ? (raw.prefixMissCost / assistantTotal) * 100 : 0;
|
|
1079
1705
|
if (prefixPct >= CACHE_MISS_ALARM_PERCENT && raw.prefixMissCost >= CACHE_MISS_ALARM_MIN_COST) {
|
|
1080
1706
|
insights.push({
|
|
1081
1707
|
kind: "alarm",
|
|
1082
1708
|
stat: fmtMoney(raw.prefixMissCost),
|
|
1083
|
-
headline: `spent re-sending conversations mid-session (${fmtPercent(prefixPct)} of
|
|
1709
|
+
headline: `spent re-sending conversations mid-session (${fmtPercent(prefixPct)} of ${assistantPctLabel})`,
|
|
1084
1710
|
advice:
|
|
1085
1711
|
"These messages paid full price for context that had already been sent — with no break, compaction, or model switch to explain it. Usually a tool or workflow is restarting or rewriting conversations. Worth a look if it stays high.",
|
|
1086
1712
|
});
|
|
@@ -1100,17 +1726,17 @@ function computeInsights(raw: PeriodRawData, trend: TrendInfo | null): PeriodIns
|
|
|
1100
1726
|
}
|
|
1101
1727
|
}
|
|
1102
1728
|
|
|
1103
|
-
const upfrontPct = (raw.upfrontCost /
|
|
1729
|
+
const upfrontPct = assistantTotal > 0 ? (raw.upfrontCost / assistantTotal) * 100 : 0;
|
|
1104
1730
|
if (upfrontPct >= UPFRONT_ALARM_PERCENT) {
|
|
1105
1731
|
insights.push({
|
|
1106
1732
|
kind: "alarm",
|
|
1107
1733
|
stat: fmtMoney(raw.upfrontCost),
|
|
1108
|
-
headline: `spent on the opening message of new sessions (${fmtPercent(upfrontPct)} of
|
|
1734
|
+
headline: `spent on the opening message of new sessions (${fmtPercent(upfrontPct)} of ${assistantPctLabel})`,
|
|
1109
1735
|
advice: "A session's first message sends everything from scratch. Fewer, longer sessions cut this overhead.",
|
|
1110
1736
|
});
|
|
1111
1737
|
}
|
|
1112
1738
|
|
|
1113
|
-
if (
|
|
1739
|
+
if (assistantTotal >= LEVERAGE_MIN_COST && raw.freshTokens >= LEVERAGE_MIN_FRESH_TOKENS) {
|
|
1114
1740
|
const leverage = raw.cacheReadTokens / raw.freshTokens;
|
|
1115
1741
|
if (leverage < LEVERAGE_FLOOR) {
|
|
1116
1742
|
insights.push({
|
|
@@ -1125,8 +1751,20 @@ function computeInsights(raw: PeriodRawData, trend: TrendInfo | null): PeriodIns
|
|
|
1125
1751
|
|
|
1126
1752
|
// --- Structure (always-on) ---
|
|
1127
1753
|
|
|
1128
|
-
if (raw.
|
|
1129
|
-
const pct = (raw.
|
|
1754
|
+
if (raw.auxiliaryCost > 0) {
|
|
1755
|
+
const pct = (raw.auxiliaryCost / total) * 100;
|
|
1756
|
+
if (pct >= 1) {
|
|
1757
|
+
insights.push({
|
|
1758
|
+
kind: "structure",
|
|
1759
|
+
stat: fmtPercent(pct),
|
|
1760
|
+
headline: "of your cost came from usage reported by tools and conversation summaries",
|
|
1761
|
+
advice: "Pi records this separately because it cannot be attributed reliably to a specific provider and model.",
|
|
1762
|
+
});
|
|
1763
|
+
}
|
|
1764
|
+
}
|
|
1765
|
+
|
|
1766
|
+
if (raw.ctxHigh.messages > 0 && assistantTotal > 0) {
|
|
1767
|
+
const pct = (raw.ctxHigh.cost / assistantTotal) * 100;
|
|
1130
1768
|
if (pct >= 1) {
|
|
1131
1769
|
const avgHigh = raw.ctxHigh.cost / raw.ctxHigh.messages;
|
|
1132
1770
|
const avgLow = raw.ctxLow.messages > 0 ? raw.ctxLow.cost / raw.ctxLow.messages : 0;
|
|
@@ -1137,7 +1775,7 @@ function computeInsights(raw: PeriodRawData, trend: TrendInfo | null): PeriodIns
|
|
|
1137
1775
|
insights.push({
|
|
1138
1776
|
kind: "structure",
|
|
1139
1777
|
stat: fmtPercent(pct),
|
|
1140
|
-
headline: `of your cost came from messages with ≥${formatThresholdTokens(CTX_TAX_THRESHOLD)} tokens loaded${cmp}`,
|
|
1778
|
+
headline: `of your ${raw.auxiliaryCost > 0 ? "assistant-message cost" : "cost"} came from messages with ≥${formatThresholdTokens(CTX_TAX_THRESHOLD)} tokens loaded${cmp}`,
|
|
1141
1779
|
advice: "Long conversations cost more per message. /compact mid-task and /clear between tasks keep them lean.",
|
|
1142
1780
|
});
|
|
1143
1781
|
}
|
package/export.ts
CHANGED
|
@@ -7,9 +7,43 @@
|
|
|
7
7
|
* component owns file naming and disk writes.
|
|
8
8
|
*/
|
|
9
9
|
|
|
10
|
+
import { join } from "node:path";
|
|
10
11
|
import type { Insight, ProviderStats, TotalStats } from "./data.ts";
|
|
11
12
|
import type { GraphModel } from "./graph.ts";
|
|
12
13
|
|
|
14
|
+
/**
|
|
15
|
+
* Read the configured export directory from settings.json content, if any.
|
|
16
|
+
* Config shape: `{ "usage-extension": { "exportDir": "~/Downloads" } }`.
|
|
17
|
+
*/
|
|
18
|
+
export function parseExportDirSetting(settingsJson: string): string | null {
|
|
19
|
+
try {
|
|
20
|
+
const parsed = JSON.parse(settingsJson) as { "usage-extension"?: { exportDir?: unknown } };
|
|
21
|
+
const dir = parsed["usage-extension"]?.exportDir;
|
|
22
|
+
return typeof dir === "string" && dir.trim() !== "" ? dir.trim() : null;
|
|
23
|
+
} catch {
|
|
24
|
+
return null;
|
|
25
|
+
}
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
/**
|
|
29
|
+
* Pick the export directory. A configured dir wins (with `~` expanded);
|
|
30
|
+
* otherwise exports go to /tmp so they never litter a repo or home
|
|
31
|
+
* directory, falling back to the OS temp dir where /tmp doesn't exist.
|
|
32
|
+
*/
|
|
33
|
+
export function resolveExportDir(
|
|
34
|
+
configured: string | null,
|
|
35
|
+
home: string,
|
|
36
|
+
slashTmpExists: boolean,
|
|
37
|
+
fallbackTmp: string,
|
|
38
|
+
): string {
|
|
39
|
+
if (configured !== null) {
|
|
40
|
+
if (configured === "~") return home;
|
|
41
|
+
if (configured.startsWith("~/")) return join(home, configured.slice(2));
|
|
42
|
+
return configured;
|
|
43
|
+
}
|
|
44
|
+
return slashTmpExists ? "/tmp" : fallbackTmp;
|
|
45
|
+
}
|
|
46
|
+
|
|
13
47
|
/** Quote a CSV field only when it needs it (comma, quote, or newline). */
|
|
14
48
|
function csvField(value: string | number): string {
|
|
15
49
|
const s = String(value);
|
package/index.ts
CHANGED
|
@@ -13,7 +13,7 @@ import type { ExtensionAPI, ExtensionCommandContext, Theme } from "@earendil-wor
|
|
|
13
13
|
import { DynamicBorder } from "@earendil-works/pi-coding-agent";
|
|
14
14
|
import { CancellableLoader, Container, Spacer, matchesKey, visibleWidth, truncateToWidth, wrapTextWithAnsi } from "@earendil-works/pi-tui";
|
|
15
15
|
|
|
16
|
-
import { collectUsageData, TAB_ORDER } from "./data";
|
|
16
|
+
import { collectUsageData, getAgentDir, TAB_ORDER } from "./data";
|
|
17
17
|
import type { CollectProgress } from "./data";
|
|
18
18
|
import type { BaseStats, ProviderStats, TabName, TotalStats, UsageData } from "./data";
|
|
19
19
|
import {
|
|
@@ -26,8 +26,16 @@ import {
|
|
|
26
26
|
TOTAL_SERIES_KEY,
|
|
27
27
|
} from "./graph";
|
|
28
28
|
import type { GraphGroupBy, GraphMetric, GraphModel } from "./graph";
|
|
29
|
-
import {
|
|
30
|
-
|
|
29
|
+
import {
|
|
30
|
+
buildGraphCsv,
|
|
31
|
+
buildInsightsJson,
|
|
32
|
+
buildTableCsv,
|
|
33
|
+
exportFileName,
|
|
34
|
+
parseExportDirSetting,
|
|
35
|
+
resolveExportDir,
|
|
36
|
+
} from "./export";
|
|
37
|
+
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
|
38
|
+
import { homedir, tmpdir } from "node:os";
|
|
31
39
|
import { join } from "node:path";
|
|
32
40
|
|
|
33
41
|
type ViewMode = "table" | "insights" | "graph";
|
|
@@ -524,9 +532,19 @@ class UsageComponent {
|
|
|
524
532
|
content = buildTableCsv(visible.providers, visible.totals);
|
|
525
533
|
}
|
|
526
534
|
try {
|
|
527
|
-
|
|
535
|
+
let configured: string | null = null;
|
|
536
|
+
try {
|
|
537
|
+
configured = parseExportDirSetting(readFileSync(join(getAgentDir(), "settings.json"), "utf8"));
|
|
538
|
+
} catch {
|
|
539
|
+
// No settings file or unreadable: fall through to the default dir.
|
|
540
|
+
}
|
|
541
|
+
const home = homedir();
|
|
542
|
+
const dir = resolveExportDir(configured, home, existsSync("/tmp"), tmpdir());
|
|
543
|
+
mkdirSync(dir, { recursive: true });
|
|
544
|
+
const path = join(dir, name);
|
|
528
545
|
writeFileSync(path, content);
|
|
529
|
-
|
|
546
|
+
const shown = path.startsWith(home + "/") ? "~" + path.slice(home.length) : path;
|
|
547
|
+
this.exportNote = { text: `Saved ${shown}`, ok: true };
|
|
530
548
|
} catch (err) {
|
|
531
549
|
this.exportNote = { text: `Export failed: ${err instanceof Error ? err.message : String(err)}`, ok: false };
|
|
532
550
|
}
|
|
@@ -655,7 +673,11 @@ class UsageComponent {
|
|
|
655
673
|
const th = this.theme;
|
|
656
674
|
const stats = this.data[this.activeTab];
|
|
657
675
|
const { insights } = stats.insights;
|
|
658
|
-
const
|
|
676
|
+
const hasUsage =
|
|
677
|
+
stats.totals.messages > 0 ||
|
|
678
|
+
stats.totals.cost > 0 ||
|
|
679
|
+
stats.totals.tokens.total > 0 ||
|
|
680
|
+
stats.totals.tokens.cacheRead > 0;
|
|
659
681
|
const hasCost = stats.totals.cost > 0;
|
|
660
682
|
const lines: string[] = [];
|
|
661
683
|
|
|
@@ -669,7 +691,7 @@ class UsageComponent {
|
|
|
669
691
|
}
|
|
670
692
|
lines.push("");
|
|
671
693
|
|
|
672
|
-
if (!
|
|
694
|
+
if (!hasUsage) {
|
|
673
695
|
lines.push(th.fg("dim", " No usage recorded for this period."));
|
|
674
696
|
lines.push("");
|
|
675
697
|
return lines;
|