@tmustier/pi-usage-extension 0.9.1 → 0.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +41 -0
- package/README.md +15 -9
- package/data.ts +714 -74
- package/index.ts +6 -2
- package/package.json +3 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,46 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.9.5] - 2026-09-22
|
|
4
|
+
|
|
5
|
+
### Changed
|
|
6
|
+
- Treat Pi 0.87 `context_edit` entries as intentional request-prefix changes so they do not count as cache misses.
|
|
7
|
+
|
|
8
|
+
### Internal
|
|
9
|
+
- Cache format bumped to **v6**. The first `/usage` open after upgrading performs a one-off rebuild.
|
|
10
|
+
|
|
11
|
+
## [0.9.4] - 2026-07-22
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
- Radically simplified nested-agent reconciliation. Child sessions on disk are now trusted as the record: when every child session file behind a tool report is part of the scan, the parent's aggregate is skipped; otherwise the whole report (or each unresolved legacy child) is counted under `Tools / summaries`. This replaces 0.9.3's exact contiguous token-and-cost vector matching and residual arithmetic (~250 lines) with a file-existence check — validated byte-identical against 0.9.3 on a 10 GB / 6,166-session corpus.
|
|
15
|
+
- Resolved child identities are unioned across copied parent history before emission, because branch copies change runId-derived child paths. Same outcome as 0.9.3's global reconciliation, without vector identity keys.
|
|
16
|
+
- Known trade-offs: a crash-truncated child session slightly undercounts, and a partially-missing child set counts the full aggregate (slight overcount). Neither case appeared in the audit corpus.
|
|
17
|
+
|
|
18
|
+
### Internal
|
|
19
|
+
- Cache format stays **v5** — no rebuild on upgrade.
|
|
20
|
+
|
|
21
|
+
## [0.9.3] - 2026-07-22
|
|
22
|
+
|
|
23
|
+
### Fixed
|
|
24
|
+
- Reconcile nested-agent tool usage against recursively scanned child sessions using exact contiguous token-and-cost vectors. Canonical tool aggregates now exclude child usage already counted from session files, and mixed aggregates add only the unmatched residual under `Tools / summaries`.
|
|
25
|
+
- Reconcile globally across copied parent history, so a stale copy with a missing child path cannot win deduplication and reintroduce usage represented by another copy's linked child session.
|
|
26
|
+
- Backfill recognised legacy `subagent` and `subagent_wait` child usage only when no scanned child-session span represents it.
|
|
27
|
+
|
|
28
|
+
### Performance
|
|
29
|
+
- Extract accounting metadata from large tool results without decoding or JSON-parsing multi-megabyte output bodies.
|
|
30
|
+
|
|
31
|
+
### Internal
|
|
32
|
+
- Cache format bumped to **v5** to retain tool reports and child-session linkage. The first `/usage` open after upgrading performs a one-off rebuild.
|
|
33
|
+
- Document the irrecoverable historical boundary: compaction and branch-summary entries written without usage metadata remain unmetered. The compatibility audit found 2,753 such compactions and 20 branch summaries.
|
|
34
|
+
|
|
35
|
+
## [0.9.2] - 2026-07-21
|
|
36
|
+
|
|
37
|
+
### Fixed
|
|
38
|
+
- Include usage persisted by Pi 0.81.0 on tool results, compactions, and branch summaries. Auxiliary cost and token fields now participate in `/usage` accounting under the same synthetic `Tools / summaries` bucket as Pi's footer and `/session`, while `Msgs` remains an assistant-message count.
|
|
39
|
+
- Keep auxiliary usage out of conversation context/cache-miss classification, attribute it separately in Insights, and deduplicate copied auxiliary entries by stable session entry id.
|
|
40
|
+
|
|
41
|
+
### Internal
|
|
42
|
+
- Cache format bumped to **v4**. The first `/usage` open after upgrading performs a one-off rebuild so previously ignored auxiliary entries are ingested.
|
|
43
|
+
|
|
3
44
|
## [0.9.1] - 2026-07-17
|
|
4
45
|
|
|
5
46
|
### Changed
|
package/README.md
CHANGED
|
@@ -7,7 +7,9 @@ A Pi extension that displays aggregated usage statistics across all sessions.
|
|
|
7
7
|
## Compatibility
|
|
8
8
|
|
|
9
9
|
- **Pi version:** 0.42.4+
|
|
10
|
-
- **Last updated:** 2026-07-
|
|
10
|
+
- **Last updated:** 2026-07-22 (0.9.3)
|
|
11
|
+
|
|
12
|
+
Pi 0.81.0+ can persist tool-result, compaction, and branch-summary usage. `/usage` includes that auxiliary usage in totals under `Tools / summaries`. Nested-agent reports are reconciled against recursively scanned child sessions, so a child call is counted once: when every child session file behind a report is part of the scan, the children are the record and the parent's aggregate is skipped; otherwise the report is counted. Older Pi versions remain supported.
|
|
11
13
|
|
|
12
14
|
## Installation
|
|
13
15
|
|
|
@@ -83,6 +85,7 @@ The **Insights** view has two sections, facts first:
|
|
|
83
85
|
|
|
84
86
|
| Insight | Detail |
|
|
85
87
|
|---|---|
|
|
88
|
+
| Tool and summary usage | share of cost reported by nested tool calls, compaction, and branch summarization (Pi 0.81.0+) |
|
|
86
89
|
| Context tax | share of cost spent at ≥ 150k context, with the average per-message cost vs messages under 100k |
|
|
87
90
|
| Project mix | top 3 project directories by cost, derived from each session's working directory (worktrees collapse into their repository; hidden when one project is ≥ 90% — you already know) |
|
|
88
91
|
| Reasoning share | share of output tokens that were hidden reasoning (recorded by pi 0.80.3+ only; hidden below 5%) |
|
|
@@ -94,7 +97,7 @@ The **Insights** view has two sections, facts first:
|
|
|
94
97
|
|---|---|
|
|
95
98
|
| Resuming after a break | ≥ 2% of the period's cost (and ≥ $1) went to messages that re-sent a large conversation from scratch after a > 5 min idle gap — provider caches expire after a few minutes idle |
|
|
96
99
|
| Switching models mid-conversation | ≥ 2% of cost (and ≥ $1) went to large-context misses right after the provider/model changed mid-session — the previous model's cache doesn't transfer |
|
|
97
|
-
| Mid-session re-sends (prefix change) | ≥ 2% of cost (and ≥ $1) went to large-context misses with **no** idle gap, compaction, or model switch to explain them — something rewrote the request prefix |
|
|
100
|
+
| Mid-session re-sends (prefix change) | ≥ 2% of cost (and ≥ $1) went to large-context misses with **no** idle gap, compaction, context edit, or model switch to explain them — something unexpectedly rewrote the request prefix |
|
|
98
101
|
| Session concentration | the top 5 sessions account for ≥ 35% of the period's cost |
|
|
99
102
|
| Upfront tax | ≥ 8% of cost was the first message of a session (session starts pay for their whole prompt uncached) |
|
|
100
103
|
| Cache leverage floor | fewer than 5 cached tokens served per fresh token paid (shown only above $5 / 1M fresh tokens, to avoid noise) |
|
|
@@ -136,7 +139,7 @@ The **Graphs** view plots usage over time for the active period as a braille lin
|
|
|
136
139
|
- **Buckets**: hourly for Today / This Week / Last Week, daily for Last 30 Days / All Time.
|
|
137
140
|
- **Line clipping**: every series (provider, model, thinking level, `other`, Total) is drawn only between its first and last bucket with usage, so late-starting or retired series don't drag a flat zero/flat tail across the whole period.
|
|
138
141
|
|
|
139
|
-
Thinking levels are replayed from `thinking_level_change` entries in each session file; messages before the first recorded change appear as `unknown`. Reasoning token counts come from `usage.reasoning` where providers report them; pi only records this field since **pi 0.80.3 (30 June 2026)**, so earlier sessions show zero reasoning tokens even though thinking models were in use.
|
|
142
|
+
Thinking levels are replayed from `thinking_level_change` entries in each session file; messages before the first recorded change appear as `unknown`. Auxiliary usage has no reliable thinking-level attribution and appears as `Tools/summaries` in that grouping. Reasoning token counts come from `usage.reasoning` where providers report them; pi only records this field since **pi 0.80.3 (30 June 2026)**, so earlier sessions show zero reasoning tokens even though thinking models were in use.
|
|
140
143
|
|
|
141
144
|
### Time Periods
|
|
142
145
|
|
|
@@ -160,8 +163,8 @@ Time periods are calculated in the local timezone where Pi runs. If you want to
|
|
|
160
163
|
|--------|-------------|
|
|
161
164
|
| **Provider / Model** | Provider name, expandable to show models |
|
|
162
165
|
| **Sessions** | Number of unique sessions |
|
|
163
|
-
| **Msgs** | Number of assistant messages |
|
|
164
|
-
| **Cost** | Total cost in USD (from API response) |
|
|
166
|
+
| **Msgs** | Number of assistant messages; auxiliary `Tools / summaries` usage does not inflate this count |
|
|
167
|
+
| **Cost** | Total cost in USD (from API response), including usage reported by tools and summaries |
|
|
165
168
|
| **Tokens** | Fresh tokens for the turn: input + output + cache write |
|
|
166
169
|
| **↑In** | Fresh input tokens: input + cache write *(dimmed)* |
|
|
167
170
|
| **↓Out** | Output tokens *(dimmed)* |
|
|
@@ -195,13 +198,16 @@ On narrow terminals, `/usage` automatically switches to a compact table instead
|
|
|
195
198
|
- **On-disk cache.** Per-file extraction results are cached in `<agentDir>/usage-extension-cache.json` (respects `PI_CODING_AGENT_DIR`), keyed by file size + mtime. Warm opens only re-parse session files that changed since the last run — on a 5.2 GB / 3,310-file corpus that takes the open from ~17 s to ~0.3 s.
|
|
196
199
|
- **First open** after install (or after deleting the cache) does a one-off full build, showing the usual cancellable loader. Cancelling saves partial progress, so the next open resumes where it left off.
|
|
197
200
|
- The cache is safe to delete at any time; it is rebuilt automatically. Corrupt or version-mismatched caches are ignored and rebuilt rather than trusted.
|
|
198
|
-
- **0.
|
|
201
|
+
- **0.9.3 bumps the cache format to v5** to retain child-session linkage for tool-usage reconciliation (v4 added Pi 0.81.0 tool and summary usage; v3 added session working directory and compaction markers; v2 added thinking level and reasoning tokens). The first open after upgrading does a one-off full rebuild (with a progress message and live file counter), then warm opens are fast again.
|
|
202
|
+
- Large nested-agent tool results use an allocation-safe metadata parser: multi-megabyte output bodies are scanned as bytes rather than decoded and JSON-parsed in full.
|
|
199
203
|
|
|
200
204
|
## Provider Notes
|
|
201
205
|
|
|
202
206
|
### Cost Tracking
|
|
203
207
|
|
|
204
|
-
Cost data comes directly from
|
|
208
|
+
Cost data comes directly from persisted usage values. For assistant messages it is grouped by provider/model. Pi 0.81.0+ can also persist usage reported by tools, compaction, and branch summarization; because those entries do not carry reliable provider/model attribution, `/usage` groups them under `Tools / summaries`, matching Pi's `/session` breakdown. Recognised legacy `subagent` and `subagent_wait` details are used as a fallback when their child session is no longer available. Accuracy depends on the provider or tool reporting costs.
|
|
209
|
+
|
|
210
|
+
Only persisted usage can be counted. Pi did not add usage metadata retroactively, so historical compaction or branch-summary entries written without `usage` remain unmetered: their exact token and cost vectors cannot be reconstructed. The compatibility audit corpus contained 2,753 such compactions and 20 branch summaries.
|
|
205
211
|
|
|
206
212
|
### Cache Tokens
|
|
207
213
|
|
|
@@ -219,9 +225,9 @@ The "Cache" column combines both read and write tokens.
|
|
|
219
225
|
|
|
220
226
|
## Data Source
|
|
221
227
|
|
|
222
|
-
Statistics are parsed recursively from session files in `~/.pi/agent/sessions/`, including nested subagent runs such as `run-0/` directories. Each session is a JSONL file containing
|
|
228
|
+
Statistics are parsed recursively from session files in `~/.pi/agent/sessions/`, including nested subagent runs such as `run-0/` directories. Each session is a JSONL file containing assistant messages and, on Pi 0.81.0+, optional usage on tool-result, compaction, and branch-summary entries.
|
|
223
229
|
|
|
224
|
-
Assistant messages duplicated across branched session files are deduplicated by timestamp + total tokens,
|
|
230
|
+
Assistant messages duplicated across branched session files are deduplicated by timestamp + total tokens. Auxiliary usage is deduplicated by its stable session entry id, which avoids collapsing parallel tools that happen to report identical usage. Nested-agent reports are suppressed when their child session files are part of the scan (resolved across all copies of the parent entry, since branch copies change runId-derived paths); reports whose children are missing stay under `Tools / summaries`. Tool and summary usage contributes cost and tokens to totals, graphs, project/session mix, and burn trend, but does not count as an assistant message or distort conversation context/cache insights.
|
|
225
231
|
|
|
226
232
|
Respects the `PI_CODING_AGENT_DIR` environment variable if set.
|
|
227
233
|
|
package/data.ts
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
* Data collection, caching, and insights for the /usage dashboard.
|
|
3
3
|
*
|
|
4
4
|
* Performance model (see CHANGELOG 0.4.0):
|
|
5
|
-
* - Session JSONL files are scanned at the buffer level. Only lines
|
|
6
|
-
*
|
|
7
|
-
*
|
|
8
|
-
*
|
|
5
|
+
* - Session JSONL files are scanned at the buffer level. Only lines relevant
|
|
6
|
+
* to assistant or auxiliary accounting are decoded and JSON.parsed. Ordinary
|
|
7
|
+
* multi-megabyte tool results are skipped; accounting-bearing large results
|
|
8
|
+
* use an allocation-safe byte parser for their small metadata fields.
|
|
9
9
|
* - Per-file extraction results are persisted to an on-disk cache keyed by
|
|
10
10
|
* (size, mtimeMs). Session files are append-only, so a warm load only
|
|
11
11
|
* re-parses files that changed since the last run.
|
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
|
|
14
14
|
import { readdir, readFile, rename, stat, writeFile } from "node:fs/promises";
|
|
15
15
|
import { homedir } from "node:os";
|
|
16
|
-
import { dirname, join } from "node:path";
|
|
16
|
+
import { basename, dirname, isAbsolute, join, resolve } from "node:path";
|
|
17
17
|
|
|
18
18
|
// =============================================================================
|
|
19
19
|
// Types
|
|
@@ -66,8 +66,12 @@ interface CostCount {
|
|
|
66
66
|
}
|
|
67
67
|
|
|
68
68
|
interface PeriodRawData {
|
|
69
|
+
/** All recorded cost, including usage reported by tools and summaries. */
|
|
69
70
|
totalCost: number;
|
|
70
|
-
|
|
71
|
+
/** Cost attached to assistant messages, used as the turn-insight denominator. */
|
|
72
|
+
assistantCost: number;
|
|
73
|
+
/** Usage reported by tool results, compactions, and branch summaries. */
|
|
74
|
+
auxiliaryCost: number;
|
|
71
75
|
/** Messages at ≥ CTX_TAX_THRESHOLD context. */
|
|
72
76
|
ctxHigh: CostCount;
|
|
73
77
|
/** Messages below CTX_LOW_THRESHOLD context (comparison group). */
|
|
@@ -164,25 +168,53 @@ export type TabName = "today" | "thisWeek" | "lastWeek" | "last30Days" | "allTim
|
|
|
164
168
|
|
|
165
169
|
export const TAB_ORDER: TabName[] = ["today", "thisWeek", "lastWeek", "last30Days", "allTime"];
|
|
166
170
|
|
|
167
|
-
export
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
171
|
+
export type UsageSource = "assistant" | "auxiliary";
|
|
172
|
+
|
|
173
|
+
/** Pi's own label for usage that cannot be attributed to a provider/model. */
|
|
174
|
+
export const AUXILIARY_PROVIDER = "Tools";
|
|
175
|
+
export const AUXILIARY_MODEL = "summaries";
|
|
176
|
+
export const AUXILIARY_THINKING_LEVEL = "Tools/summaries";
|
|
177
|
+
|
|
178
|
+
export interface UsageAmount {
|
|
172
179
|
cost: number;
|
|
173
180
|
input: number;
|
|
174
181
|
output: number;
|
|
175
182
|
cacheRead: number;
|
|
176
183
|
cacheWrite: number;
|
|
177
|
-
/** Reasoning output tokens reported by the provider (0 when absent). */
|
|
178
184
|
reasoning: number;
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
export interface SessionMessage extends UsageAmount {
|
|
188
|
+
provider: string;
|
|
189
|
+
model: string;
|
|
190
|
+
/** Thinking level active when the message was produced; "" when unknown. */
|
|
191
|
+
thinkingLevel: string;
|
|
192
|
+
/** Assistant response, or usage reported by a tool/summary entry. */
|
|
193
|
+
source: UsageSource;
|
|
194
|
+
/** Session entry id used to dedupe copied auxiliary entries; empty for assistant messages. */
|
|
195
|
+
sourceId: string;
|
|
196
|
+
timestamp: number;
|
|
197
|
+
/** The request followed a compaction or context edit that intentionally changed its prefix. */
|
|
198
|
+
afterContextChange: boolean;
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
export interface ChildToolUsage {
|
|
202
|
+
resultIndex: number;
|
|
203
|
+
/** Persisted child session path when the tool supplied one; empty otherwise. */
|
|
204
|
+
sessionFile: string;
|
|
205
|
+
usage: UsageAmount;
|
|
206
|
+
}
|
|
207
|
+
|
|
208
|
+
export interface ToolUsageRecord {
|
|
209
|
+
/** Parent tool-result entry id, stable across copied branch history. */
|
|
210
|
+
sourceId: string;
|
|
179
211
|
timestamp: number;
|
|
180
|
-
/**
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
212
|
+
/** Canonical Pi 0.81+ tool usage; null for legacy nested-agent results. */
|
|
213
|
+
reportedUsage: UsageAmount | null;
|
|
214
|
+
/** Run id used to derive the standard nested-session path when needed. */
|
|
215
|
+
runId: string;
|
|
216
|
+
/** Recognised per-child usage from nested-agent tool details. */
|
|
217
|
+
children: ChildToolUsage[];
|
|
186
218
|
}
|
|
187
219
|
|
|
188
220
|
export interface ParsedSessionFile {
|
|
@@ -190,8 +222,10 @@ export interface ParsedSessionFile {
|
|
|
190
222
|
sessionId: string;
|
|
191
223
|
/** Working directory from the session header; "" when absent. */
|
|
192
224
|
cwd: string;
|
|
193
|
-
/** Extracted assistant
|
|
225
|
+
/** Extracted assistant and summary usage records, pre-dedupe. */
|
|
194
226
|
messages: SessionMessage[];
|
|
227
|
+
/** Tool usage is reconciled against recursively scanned child sessions later. */
|
|
228
|
+
toolUsages: ToolUsageRecord[];
|
|
195
229
|
}
|
|
196
230
|
|
|
197
231
|
// =============================================================================
|
|
@@ -252,35 +286,378 @@ const NEWLINE = 0x0a;
|
|
|
252
286
|
// only cost a wasted JSON.parse — the parsed entry is still shape-checked.
|
|
253
287
|
const PATTERN_ASSISTANT_COMPACT = Buffer.from('"role":"assistant"');
|
|
254
288
|
const PATTERN_ASSISTANT_SPACED = Buffer.from('"role": "assistant"');
|
|
289
|
+
const PATTERN_TOOL_RESULT_COMPACT = Buffer.from('"role":"toolResult"');
|
|
290
|
+
const PATTERN_TOOL_RESULT_SPACED = Buffer.from('"role": "toolResult"');
|
|
291
|
+
const PATTERN_USAGE_COMPACT = Buffer.from('"usage":{');
|
|
292
|
+
const PATTERN_USAGE_SPACED = Buffer.from('"usage": {');
|
|
255
293
|
const PATTERN_SESSION_COMPACT = Buffer.from('"type":"session"');
|
|
256
294
|
const PATTERN_SESSION_SPACED = Buffer.from('"type": "session"');
|
|
257
295
|
const PATTERN_THINKING_COMPACT = Buffer.from('"type":"thinking_level_change"');
|
|
258
296
|
const PATTERN_THINKING_SPACED = Buffer.from('"type": "thinking_level_change"');
|
|
259
297
|
const PATTERN_COMPACTION_COMPACT = Buffer.from('"type":"compaction"');
|
|
260
298
|
const PATTERN_COMPACTION_SPACED = Buffer.from('"type": "compaction"');
|
|
299
|
+
const PATTERN_CONTEXT_EDIT_COMPACT = Buffer.from('"type":"context_edit"');
|
|
300
|
+
const PATTERN_CONTEXT_EDIT_SPACED = Buffer.from('"type": "context_edit"');
|
|
301
|
+
const PATTERN_BRANCH_SUMMARY_COMPACT = Buffer.from('"type":"branch_summary"');
|
|
302
|
+
const PATTERN_BRANCH_SUMMARY_SPACED = Buffer.from('"type": "branch_summary"');
|
|
303
|
+
// pi-subagents versions predating Pi 0.81 persisted child usage in details but
|
|
304
|
+
// could not put it on the canonical tool-result usage field. Their tool names
|
|
305
|
+
// are near the start of the line, so we can recover those records without
|
|
306
|
+
// scanning every multi-megabyte tool result.
|
|
307
|
+
const PATTERN_SUBAGENT_TOOL_COMPACT = Buffer.from('"toolName":"subagent"');
|
|
308
|
+
const PATTERN_SUBAGENT_TOOL_SPACED = Buffer.from('"toolName": "subagent"');
|
|
309
|
+
const PATTERN_SUBAGENT_WAIT_TOOL_COMPACT = Buffer.from('"toolName":"subagent_wait"');
|
|
310
|
+
const PATTERN_SUBAGENT_WAIT_TOOL_SPACED = Buffer.from('"toolName": "subagent_wait"');
|
|
261
311
|
|
|
262
312
|
const PARSE_YIELD_EVERY_LINES = 2000;
|
|
263
313
|
|
|
314
|
+
function finiteNumber(value: unknown): number {
|
|
315
|
+
return typeof value === "number" && Number.isFinite(value) ? value : 0;
|
|
316
|
+
}
|
|
317
|
+
|
|
318
|
+
function parseUsageAmount(value: unknown): UsageAmount | null {
|
|
319
|
+
if (!value || typeof value !== "object") return null;
|
|
320
|
+
const persisted = value as Record<string, unknown>;
|
|
321
|
+
const costValue = persisted.cost;
|
|
322
|
+
const cost =
|
|
323
|
+
typeof costValue === "number"
|
|
324
|
+
? finiteNumber(costValue)
|
|
325
|
+
: costValue && typeof costValue === "object"
|
|
326
|
+
? finiteNumber((costValue as Record<string, unknown>).total)
|
|
327
|
+
: 0;
|
|
328
|
+
const usage = {
|
|
329
|
+
cost,
|
|
330
|
+
input: finiteNumber(persisted.input),
|
|
331
|
+
output: finiteNumber(persisted.output),
|
|
332
|
+
cacheRead: finiteNumber(persisted.cacheRead),
|
|
333
|
+
cacheWrite: finiteNumber(persisted.cacheWrite),
|
|
334
|
+
reasoning: finiteNumber(persisted.reasoning),
|
|
335
|
+
};
|
|
336
|
+
return usage.cost === 0 && usage.input === 0 && usage.output === 0 && usage.cacheRead === 0 && usage.cacheWrite === 0
|
|
337
|
+
? null
|
|
338
|
+
: usage;
|
|
339
|
+
}
|
|
340
|
+
|
|
341
|
+
function parsedTimestamp(messageTimestamp: unknown, entryTimestamp: unknown): number {
|
|
342
|
+
const parsed =
|
|
343
|
+
typeof messageTimestamp === "number"
|
|
344
|
+
? messageTimestamp
|
|
345
|
+
: new Date(String(messageTimestamp ?? entryTimestamp ?? "")).getTime();
|
|
346
|
+
return Number.isFinite(parsed) ? parsed : 0;
|
|
347
|
+
}
|
|
348
|
+
|
|
349
|
+
function auxiliaryMessage(usage: UsageAmount, timestamp: number, sourceId: string): SessionMessage {
|
|
350
|
+
return {
|
|
351
|
+
provider: AUXILIARY_PROVIDER,
|
|
352
|
+
model: AUXILIARY_MODEL,
|
|
353
|
+
thinkingLevel: AUXILIARY_THINKING_LEVEL,
|
|
354
|
+
source: "auxiliary",
|
|
355
|
+
sourceId,
|
|
356
|
+
...usage,
|
|
357
|
+
timestamp,
|
|
358
|
+
afterContextChange: false,
|
|
359
|
+
};
|
|
360
|
+
}
|
|
361
|
+
|
|
362
|
+
function buildToolUsageRecord(
|
|
363
|
+
toolName: unknown,
|
|
364
|
+
detailsValue: unknown,
|
|
365
|
+
reportedValue: unknown,
|
|
366
|
+
sourceIdValue: unknown,
|
|
367
|
+
messageTimestamp: unknown,
|
|
368
|
+
entryTimestamp: unknown
|
|
369
|
+
): ToolUsageRecord | null {
|
|
370
|
+
const reportedUsage = parseUsageAmount(reportedValue);
|
|
371
|
+
const details = detailsValue && typeof detailsValue === "object" ? (detailsValue as Record<string, unknown>) : null;
|
|
372
|
+
const totalChildUsage = parseUsageAmount(details?.totalChildUsage);
|
|
373
|
+
const knownNestedTool = toolName === "subagent" || toolName === "subagent_wait";
|
|
374
|
+
const children: ChildToolUsage[] = [];
|
|
375
|
+
if (details && Array.isArray(details.results)) {
|
|
376
|
+
for (let resultIndex = 0; resultIndex < details.results.length; resultIndex++) {
|
|
377
|
+
const result = details.results[resultIndex];
|
|
378
|
+
if (!result || typeof result !== "object") continue;
|
|
379
|
+
const child = result as Record<string, unknown>;
|
|
380
|
+
const usage = parseUsageAmount(child.usage);
|
|
381
|
+
const sessionFile = typeof child.sessionFile === "string" ? child.sessionFile : "";
|
|
382
|
+
// Legacy fallback is deliberately restricted to recognised nested-agent
|
|
383
|
+
// records. Generic Pi 0.81 tools remain canonical through reportedUsage.
|
|
384
|
+
if (usage && (knownNestedTool || totalChildUsage || (reportedUsage && sessionFile))) {
|
|
385
|
+
children.push({ resultIndex, sessionFile, usage });
|
|
386
|
+
}
|
|
387
|
+
}
|
|
388
|
+
}
|
|
389
|
+
if (!reportedUsage && children.length === 0) return null;
|
|
390
|
+
return {
|
|
391
|
+
sourceId: typeof sourceIdValue === "string" ? sourceIdValue : "",
|
|
392
|
+
timestamp: parsedTimestamp(messageTimestamp, entryTimestamp),
|
|
393
|
+
reportedUsage,
|
|
394
|
+
runId: details && typeof details.runId === "string" ? details.runId : "",
|
|
395
|
+
children,
|
|
396
|
+
};
|
|
397
|
+
}
|
|
398
|
+
|
|
399
|
+
const LARGE_TOOL_RESULT_BYTES = 64 * 1024;
|
|
400
|
+
const PROPERTY_ID = Buffer.from('"id":');
|
|
401
|
+
const PROPERTY_TIMESTAMP = Buffer.from('"timestamp":');
|
|
402
|
+
const PROPERTY_MESSAGE = Buffer.from('"message":');
|
|
403
|
+
const PROPERTY_TOOL_NAME = Buffer.from('"toolName":');
|
|
404
|
+
const PROPERTY_DETAILS = Buffer.from('"details":');
|
|
405
|
+
const PROPERTY_USAGE = Buffer.from('"usage":');
|
|
406
|
+
const DIRECT_CHILD_PROPERTIES = new Set(["usage", "sessionFile"]);
|
|
407
|
+
|
|
408
|
+
function skipJsonWhitespace(buffer: Buffer, offset: number, limit: number): number {
|
|
409
|
+
while (offset < limit && (buffer[offset] === 0x20 || buffer[offset] === 0x09 || buffer[offset] === 0x0a || buffer[offset] === 0x0d)) offset++;
|
|
410
|
+
return offset;
|
|
411
|
+
}
|
|
412
|
+
|
|
413
|
+
/** Find one JSON value's end without decoding large strings or container bodies. */
|
|
414
|
+
function jsonValueEnd(buffer: Buffer, offset: number, limit: number): number {
|
|
415
|
+
offset = skipJsonWhitespace(buffer, offset, limit);
|
|
416
|
+
if (offset >= limit) return offset;
|
|
417
|
+
const first = buffer[offset];
|
|
418
|
+
if (first === 0x22) {
|
|
419
|
+
let escaped = false;
|
|
420
|
+
for (let i = offset + 1; i < limit; i++) {
|
|
421
|
+
const byte = buffer[i];
|
|
422
|
+
if (escaped) escaped = false;
|
|
423
|
+
else if (byte === 0x5c) escaped = true;
|
|
424
|
+
else if (byte === 0x22) return i + 1;
|
|
425
|
+
}
|
|
426
|
+
return limit;
|
|
427
|
+
}
|
|
428
|
+
if (first === 0x7b || first === 0x5b) {
|
|
429
|
+
let depth = 0;
|
|
430
|
+
let inString = false;
|
|
431
|
+
let escaped = false;
|
|
432
|
+
for (let i = offset; i < limit; i++) {
|
|
433
|
+
const byte = buffer[i];
|
|
434
|
+
if (inString) {
|
|
435
|
+
if (escaped) escaped = false;
|
|
436
|
+
else if (byte === 0x5c) escaped = true;
|
|
437
|
+
else if (byte === 0x22) inString = false;
|
|
438
|
+
continue;
|
|
439
|
+
}
|
|
440
|
+
if (byte === 0x22) inString = true;
|
|
441
|
+
else if (byte === 0x7b || byte === 0x5b) depth++;
|
|
442
|
+
else if (byte === 0x7d || byte === 0x5d) {
|
|
443
|
+
depth--;
|
|
444
|
+
if (depth === 0) return i + 1;
|
|
445
|
+
}
|
|
446
|
+
}
|
|
447
|
+
return limit;
|
|
448
|
+
}
|
|
449
|
+
let end = offset;
|
|
450
|
+
while (end < limit && buffer[end] !== 0x2c && buffer[end] !== 0x5d && buffer[end] !== 0x7d) end++;
|
|
451
|
+
return end;
|
|
452
|
+
}
|
|
453
|
+
|
|
454
|
+
function parseJsonValueAt(buffer: Buffer, offset: number, limit: number): unknown {
|
|
455
|
+
const start = skipJsonWhitespace(buffer, offset, limit);
|
|
456
|
+
const end = jsonValueEnd(buffer, start, limit);
|
|
457
|
+
if (end <= start) return undefined;
|
|
458
|
+
try {
|
|
459
|
+
return JSON.parse(buffer.toString("utf8", start, end));
|
|
460
|
+
} catch {
|
|
461
|
+
return undefined;
|
|
462
|
+
}
|
|
463
|
+
}
|
|
464
|
+
|
|
465
|
+
function parsePropertyValue(buffer: Buffer, property: Buffer, from: number, to: number): unknown {
|
|
466
|
+
const propertyOffset = buffer.indexOf(property, from);
|
|
467
|
+
if (propertyOffset < 0 || propertyOffset >= to) return undefined;
|
|
468
|
+
return parseJsonValueAt(buffer, propertyOffset + property.length, to);
|
|
469
|
+
}
|
|
470
|
+
|
|
471
|
+
function parseLastPropertyValue(buffer: Buffer, property: Buffer, from: number, to: number): unknown {
|
|
472
|
+
const propertyOffset = buffer.lastIndexOf(property, to - 1);
|
|
473
|
+
if (propertyOffset < from) return undefined;
|
|
474
|
+
return parseJsonValueAt(buffer, propertyOffset + property.length, to);
|
|
475
|
+
}
|
|
476
|
+
|
|
477
|
+
interface DirectObjectScan {
|
|
478
|
+
end: number;
|
|
479
|
+
values: Map<string, [start: number, end: number]>;
|
|
480
|
+
}
|
|
481
|
+
|
|
482
|
+
/** Scan direct object properties while skipping nested values allocation-free. */
|
|
483
|
+
function scanDirectObjectProperties(buffer: Buffer, objectStart: number, limit: number, wanted: Set<string>): DirectObjectScan {
|
|
484
|
+
const values = new Map<string, [number, number]>();
|
|
485
|
+
let cursor = objectStart + 1;
|
|
486
|
+
while (cursor < limit) {
|
|
487
|
+
cursor = skipJsonWhitespace(buffer, cursor, limit);
|
|
488
|
+
if (buffer[cursor] === 0x7d) return { end: cursor + 1, values };
|
|
489
|
+
if (buffer[cursor] === 0x2c) {
|
|
490
|
+
cursor++;
|
|
491
|
+
continue;
|
|
492
|
+
}
|
|
493
|
+
if (buffer[cursor] !== 0x22) return { end: jsonValueEnd(buffer, objectStart, limit), values };
|
|
494
|
+
const keyEnd = jsonValueEnd(buffer, cursor, limit);
|
|
495
|
+
let colon = skipJsonWhitespace(buffer, keyEnd, limit);
|
|
496
|
+
if (buffer[colon] !== 0x3a) return { end: jsonValueEnd(buffer, objectStart, limit), values };
|
|
497
|
+
const valueStart = skipJsonWhitespace(buffer, colon + 1, limit);
|
|
498
|
+
const valueEnd = jsonValueEnd(buffer, valueStart, limit);
|
|
499
|
+
const key = buffer.toString("utf8", cursor + 1, keyEnd - 1);
|
|
500
|
+
if (wanted.has(key)) values.set(key, [valueStart, valueEnd]);
|
|
501
|
+
if (valueEnd <= valueStart) return { end: limit, values };
|
|
502
|
+
cursor = valueEnd;
|
|
503
|
+
}
|
|
504
|
+
return { end: limit, values };
|
|
505
|
+
}
|
|
506
|
+
|
|
507
|
+
function parseJsonRange(buffer: Buffer, range: [number, number] | undefined): unknown {
|
|
508
|
+
if (!range) return undefined;
|
|
509
|
+
try {
|
|
510
|
+
return JSON.parse(buffer.toString("utf8", range[0], range[1]));
|
|
511
|
+
} catch {
|
|
512
|
+
return undefined;
|
|
513
|
+
}
|
|
514
|
+
}
|
|
515
|
+
|
|
516
|
+
interface ChildResultsScan {
|
|
517
|
+
end: number;
|
|
518
|
+
results: Array<Record<string, unknown>>;
|
|
519
|
+
}
|
|
520
|
+
|
|
521
|
+
function scanChildResults(buffer: Buffer, arrayStart: number, limit: number): ChildResultsScan {
|
|
522
|
+
const results: Array<Record<string, unknown>> = [];
|
|
523
|
+
let cursor = arrayStart + 1;
|
|
524
|
+
while (cursor < limit) {
|
|
525
|
+
cursor = skipJsonWhitespace(buffer, cursor, limit);
|
|
526
|
+
if (buffer[cursor] === 0x5d) return { end: cursor + 1, results };
|
|
527
|
+
if (buffer[cursor] === 0x2c) {
|
|
528
|
+
cursor++;
|
|
529
|
+
continue;
|
|
530
|
+
}
|
|
531
|
+
if (buffer[cursor] === 0x7b) {
|
|
532
|
+
const scanned = scanDirectObjectProperties(buffer, cursor, limit, DIRECT_CHILD_PROPERTIES);
|
|
533
|
+
results.push({
|
|
534
|
+
usage: parseJsonRange(buffer, scanned.values.get("usage")),
|
|
535
|
+
sessionFile: parseJsonRange(buffer, scanned.values.get("sessionFile")),
|
|
536
|
+
});
|
|
537
|
+
if (scanned.end <= cursor) return { end: limit, results };
|
|
538
|
+
cursor = scanned.end;
|
|
539
|
+
continue;
|
|
540
|
+
}
|
|
541
|
+
const valueEnd = jsonValueEnd(buffer, cursor, limit);
|
|
542
|
+
if (valueEnd <= cursor) return { end: limit, results };
|
|
543
|
+
cursor = valueEnd;
|
|
544
|
+
}
|
|
545
|
+
return { end: limit, results };
|
|
546
|
+
}
|
|
547
|
+
|
|
548
|
+
interface LargeDetailsScan {
|
|
549
|
+
end: number;
|
|
550
|
+
details: Record<string, unknown>;
|
|
551
|
+
}
|
|
552
|
+
|
|
553
|
+
/** Scan nested-agent details and its result metadata in a single byte pass. */
|
|
554
|
+
function scanLargeDetails(buffer: Buffer, objectStart: number, limit: number): LargeDetailsScan {
|
|
555
|
+
const details: Record<string, unknown> = { results: [] };
|
|
556
|
+
let cursor = objectStart + 1;
|
|
557
|
+
while (cursor < limit) {
|
|
558
|
+
cursor = skipJsonWhitespace(buffer, cursor, limit);
|
|
559
|
+
if (buffer[cursor] === 0x7d) return { end: cursor + 1, details };
|
|
560
|
+
if (buffer[cursor] === 0x2c) {
|
|
561
|
+
cursor++;
|
|
562
|
+
continue;
|
|
563
|
+
}
|
|
564
|
+
if (buffer[cursor] !== 0x22) return { end: jsonValueEnd(buffer, objectStart, limit), details };
|
|
565
|
+
const keyEnd = jsonValueEnd(buffer, cursor, limit);
|
|
566
|
+
let colon = skipJsonWhitespace(buffer, keyEnd, limit);
|
|
567
|
+
if (buffer[colon] !== 0x3a) return { end: jsonValueEnd(buffer, objectStart, limit), details };
|
|
568
|
+
const valueStart = skipJsonWhitespace(buffer, colon + 1, limit);
|
|
569
|
+
const key = buffer.toString("utf8", cursor + 1, keyEnd - 1);
|
|
570
|
+
let valueEnd: number;
|
|
571
|
+
if (key === "results" && buffer[valueStart] === 0x5b) {
|
|
572
|
+
const scanned = scanChildResults(buffer, valueStart, limit);
|
|
573
|
+
details.results = scanned.results;
|
|
574
|
+
valueEnd = scanned.end;
|
|
575
|
+
} else {
|
|
576
|
+
valueEnd = jsonValueEnd(buffer, valueStart, limit);
|
|
577
|
+
if (key === "runId" || key === "totalChildUsage") {
|
|
578
|
+
details[key] = parseJsonRange(buffer, [valueStart, valueEnd]);
|
|
579
|
+
}
|
|
580
|
+
}
|
|
581
|
+
if (valueEnd <= valueStart) return { end: limit, details };
|
|
582
|
+
cursor = valueEnd;
|
|
583
|
+
}
|
|
584
|
+
return { end: limit, details };
|
|
585
|
+
}
|
|
586
|
+
|
|
587
|
+
/**
|
|
588
|
+
* Parse only the small accounting fields from a large tool-result line. This
|
|
589
|
+
* avoids UTF-8 decoding and JSON.parse allocation for multi-megabyte content.
|
|
590
|
+
*/
|
|
591
|
+
function parseLargeToolResultLine(line: Buffer): ToolUsageRecord | null {
|
|
592
|
+
const messageOffset = line.indexOf(PROPERTY_MESSAGE);
|
|
593
|
+
if (messageOffset < 0) return null;
|
|
594
|
+
const detailsOffset = line.indexOf(PROPERTY_DETAILS, messageOffset);
|
|
595
|
+
let detailsEnd = -1;
|
|
596
|
+
let details: Record<string, unknown> | null = null;
|
|
597
|
+
if (detailsOffset >= 0) {
|
|
598
|
+
const detailsStart = skipJsonWhitespace(line, detailsOffset + PROPERTY_DETAILS.length, line.length);
|
|
599
|
+
if (line[detailsStart] === 0x7b) {
|
|
600
|
+
const scanned = scanLargeDetails(line, detailsStart, line.length);
|
|
601
|
+
detailsEnd = scanned.end;
|
|
602
|
+
details = scanned.details;
|
|
603
|
+
}
|
|
604
|
+
}
|
|
605
|
+
|
|
606
|
+
const reportedUsage = detailsEnd >= 0
|
|
607
|
+
? parsePropertyValue(line, PROPERTY_USAGE, detailsEnd, line.length)
|
|
608
|
+
: parseLastPropertyValue(line, PROPERTY_USAGE, messageOffset, line.length);
|
|
609
|
+
const sourceId = parsePropertyValue(line, PROPERTY_ID, 0, messageOffset);
|
|
610
|
+
const entryTimestamp = parsePropertyValue(line, PROPERTY_TIMESTAMP, 0, messageOffset);
|
|
611
|
+
const messageTimestamp = parseLastPropertyValue(line, PROPERTY_TIMESTAMP, messageOffset, line.length);
|
|
612
|
+
const toolName = parsePropertyValue(line, PROPERTY_TOOL_NAME, messageOffset, detailsOffset >= 0 ? detailsOffset : line.length);
|
|
613
|
+
return buildToolUsageRecord(toolName, details, reportedUsage, sourceId, messageTimestamp, entryTimestamp);
|
|
614
|
+
}
|
|
615
|
+
|
|
264
616
|
function lineMightBeRelevant(line: Buffer): boolean {
|
|
617
|
+
// Entry type/role fields are at the front of Pi's JSONL objects. Restrict
|
|
618
|
+
// those checks to a small prefix so multi-megabyte tool output is not scanned
|
|
619
|
+
// repeatedly for every possible entry shape.
|
|
620
|
+
const head = line.length > 1024 ? line.subarray(0, 1024) : line;
|
|
621
|
+
if (head.includes(PATTERN_ASSISTANT_COMPACT) || head.includes(PATTERN_ASSISTANT_SPACED)) return true;
|
|
622
|
+
|
|
623
|
+
if (head.includes(PATTERN_TOOL_RESULT_COMPACT) || head.includes(PATTERN_TOOL_RESULT_SPACED)) {
|
|
624
|
+
if (
|
|
625
|
+
head.includes(PATTERN_SUBAGENT_TOOL_COMPACT) ||
|
|
626
|
+
head.includes(PATTERN_SUBAGENT_TOOL_SPACED) ||
|
|
627
|
+
head.includes(PATTERN_SUBAGENT_WAIT_TOOL_COMPACT) ||
|
|
628
|
+
head.includes(PATTERN_SUBAGENT_WAIT_TOOL_SPACED)
|
|
629
|
+
) {
|
|
630
|
+
return true;
|
|
631
|
+
}
|
|
632
|
+
// Pi serializes optional tool usage after content/details and immediately
|
|
633
|
+
// before the small isError/timestamp suffix. Checking the tail preserves
|
|
634
|
+
// the fast path for ordinary tool results, which dominate session bytes.
|
|
635
|
+
const tail = line.length > 4096 ? line.subarray(line.length - 4096) : line;
|
|
636
|
+
return tail.includes(PATTERN_USAGE_COMPACT) || tail.includes(PATTERN_USAGE_SPACED);
|
|
637
|
+
}
|
|
638
|
+
|
|
265
639
|
return (
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
640
|
+
head.includes(PATTERN_SESSION_COMPACT) ||
|
|
641
|
+
head.includes(PATTERN_THINKING_COMPACT) ||
|
|
642
|
+
head.includes(PATTERN_COMPACTION_COMPACT) ||
|
|
643
|
+
head.includes(PATTERN_CONTEXT_EDIT_COMPACT) ||
|
|
644
|
+
head.includes(PATTERN_BRANCH_SUMMARY_COMPACT) ||
|
|
645
|
+
head.includes(PATTERN_SESSION_SPACED) ||
|
|
646
|
+
head.includes(PATTERN_THINKING_SPACED) ||
|
|
647
|
+
head.includes(PATTERN_COMPACTION_SPACED) ||
|
|
648
|
+
head.includes(PATTERN_CONTEXT_EDIT_SPACED) ||
|
|
649
|
+
head.includes(PATTERN_BRANCH_SUMMARY_SPACED)
|
|
274
650
|
);
|
|
275
651
|
}
|
|
276
652
|
|
|
277
653
|
/**
|
|
278
|
-
* Extract the session id
|
|
654
|
+
* Extract the session id plus assistant/tool/summary usage from a JSONL buffer.
|
|
279
655
|
* Returns partial results when aborted — callers must check `signal.aborted`
|
|
280
656
|
* before caching or using the result.
|
|
281
657
|
*/
|
|
282
658
|
export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal): Promise<ParsedSessionFile> {
|
|
283
659
|
const messages: SessionMessage[] = [];
|
|
660
|
+
const toolUsages: ToolUsageRecord[] = [];
|
|
284
661
|
let sessionId = "";
|
|
285
662
|
let cwd = "";
|
|
286
663
|
// Assistant messages don't carry the thinking level; pi records it as separate
|
|
@@ -288,7 +665,8 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
288
665
|
// message of a session. Replaying them in append order attributes each message
|
|
289
666
|
// to the level active when it was produced.
|
|
290
667
|
let thinkingLevel = "";
|
|
291
|
-
let
|
|
668
|
+
let contextChangePending = false;
|
|
669
|
+
|
|
292
670
|
let start = 0;
|
|
293
671
|
let lineNumber = 0;
|
|
294
672
|
|
|
@@ -299,10 +677,18 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
299
677
|
lineNumber++;
|
|
300
678
|
if (lineNumber % PARSE_YIELD_EVERY_LINES === 0) {
|
|
301
679
|
await new Promise<void>((resolve) => setImmediate(resolve));
|
|
302
|
-
if (signal?.aborted) return { sessionId, cwd, messages };
|
|
680
|
+
if (signal?.aborted) return { sessionId, cwd, messages, toolUsages };
|
|
303
681
|
}
|
|
304
682
|
|
|
305
|
-
|
|
683
|
+
const lineBuffer = buffer.subarray(start, end);
|
|
684
|
+
if (end > start && lineMightBeRelevant(lineBuffer)) {
|
|
685
|
+
const head = lineBuffer.subarray(0, Math.min(1024, lineBuffer.length));
|
|
686
|
+
if (lineBuffer.length > LARGE_TOOL_RESULT_BYTES && head.includes(PATTERN_TOOL_RESULT_COMPACT)) {
|
|
687
|
+
const toolUsage = parseLargeToolResultLine(lineBuffer);
|
|
688
|
+
if (toolUsage) toolUsages.push(toolUsage);
|
|
689
|
+
start = end + 1;
|
|
690
|
+
continue;
|
|
691
|
+
}
|
|
306
692
|
try {
|
|
307
693
|
const entry = JSON.parse(buffer.toString("utf8", start, end));
|
|
308
694
|
|
|
@@ -312,7 +698,14 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
312
698
|
} else if (entry.type === "thinking_level_change") {
|
|
313
699
|
if (typeof entry.thinkingLevel === "string") thinkingLevel = entry.thinkingLevel;
|
|
314
700
|
} else if (entry.type === "compaction") {
|
|
315
|
-
|
|
701
|
+
const usage = parseUsageAmount(entry.usage);
|
|
702
|
+
if (usage) messages.push(auxiliaryMessage(usage, parsedTimestamp(undefined, entry.timestamp), typeof entry.id === "string" ? entry.id : ""));
|
|
703
|
+
contextChangePending = true;
|
|
704
|
+
} else if (entry.type === "context_edit") {
|
|
705
|
+
contextChangePending = true;
|
|
706
|
+
} else if (entry.type === "branch_summary") {
|
|
707
|
+
const usage = parseUsageAmount(entry.usage);
|
|
708
|
+
if (usage) messages.push(auxiliaryMessage(usage, parsedTimestamp(undefined, entry.timestamp), typeof entry.id === "string" ? entry.id : ""));
|
|
316
709
|
} else if (entry.type === "message" && entry.message?.role === "assistant") {
|
|
317
710
|
const msg = entry.message;
|
|
318
711
|
if (msg.usage && msg.provider && msg.model) {
|
|
@@ -321,6 +714,8 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
321
714
|
provider: msg.provider,
|
|
322
715
|
model: msg.model,
|
|
323
716
|
thinkingLevel,
|
|
717
|
+
source: "assistant",
|
|
718
|
+
sourceId: "",
|
|
324
719
|
cost: msg.usage.cost?.total || 0,
|
|
325
720
|
input: msg.usage.input || 0,
|
|
326
721
|
output: msg.usage.output || 0,
|
|
@@ -328,10 +723,14 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
328
723
|
cacheWrite: msg.usage.cacheWrite || 0,
|
|
329
724
|
reasoning: msg.usage.reasoning || 0,
|
|
330
725
|
timestamp: msg.timestamp || (Number.isNaN(fallbackTs) ? 0 : fallbackTs),
|
|
331
|
-
|
|
726
|
+
afterContextChange: contextChangePending,
|
|
332
727
|
});
|
|
333
|
-
|
|
728
|
+
contextChangePending = false;
|
|
334
729
|
}
|
|
730
|
+
} else if (entry.type === "message" && entry.message?.role === "toolResult") {
|
|
731
|
+
const msg = entry.message;
|
|
732
|
+
const toolUsage = buildToolUsageRecord(msg.toolName, msg.details, msg.usage, entry.id, msg.timestamp, entry.timestamp);
|
|
733
|
+
if (toolUsage) toolUsages.push(toolUsage);
|
|
335
734
|
}
|
|
336
735
|
} catch {
|
|
337
736
|
// Skip malformed lines
|
|
@@ -341,14 +740,14 @@ export async function parseSessionBuffer(buffer: Buffer, signal?: AbortSignal):
|
|
|
341
740
|
start = end + 1;
|
|
342
741
|
}
|
|
343
742
|
|
|
344
|
-
return { sessionId, cwd, messages };
|
|
743
|
+
return { sessionId, cwd, messages, toolUsages };
|
|
345
744
|
}
|
|
346
745
|
|
|
347
746
|
// =============================================================================
|
|
348
747
|
// On-disk cache
|
|
349
748
|
// =============================================================================
|
|
350
749
|
|
|
351
|
-
const CACHE_VERSION =
|
|
750
|
+
const CACHE_VERSION = 6;
|
|
352
751
|
|
|
353
752
|
type CachedMessageTuple = [
|
|
354
753
|
providerIdx: number,
|
|
@@ -361,7 +760,32 @@ type CachedMessageTuple = [
|
|
|
361
760
|
timestamp: number,
|
|
362
761
|
thinkingLevelIdx: number,
|
|
363
762
|
reasoning: number,
|
|
364
|
-
|
|
763
|
+
afterContextChange: 0 | 1,
|
|
764
|
+
auxiliary: 0 | 1,
|
|
765
|
+
sourceIdIdx: number,
|
|
766
|
+
];
|
|
767
|
+
|
|
768
|
+
type CachedUsageTuple = [
|
|
769
|
+
cost: number,
|
|
770
|
+
input: number,
|
|
771
|
+
output: number,
|
|
772
|
+
cacheRead: number,
|
|
773
|
+
cacheWrite: number,
|
|
774
|
+
reasoning: number,
|
|
775
|
+
];
|
|
776
|
+
|
|
777
|
+
type CachedChildToolUsageTuple = [
|
|
778
|
+
resultIndex: number,
|
|
779
|
+
sessionFileIdx: number,
|
|
780
|
+
usage: CachedUsageTuple,
|
|
781
|
+
];
|
|
782
|
+
|
|
783
|
+
type CachedToolUsageTuple = [
|
|
784
|
+
sourceIdIdx: number,
|
|
785
|
+
timestamp: number,
|
|
786
|
+
reportedUsage: CachedUsageTuple | null,
|
|
787
|
+
runIdIdx: number,
|
|
788
|
+
children: CachedChildToolUsageTuple[],
|
|
365
789
|
];
|
|
366
790
|
|
|
367
791
|
interface CacheFileEntry {
|
|
@@ -370,6 +794,7 @@ interface CacheFileEntry {
|
|
|
370
794
|
sessionId: string;
|
|
371
795
|
cwd: string;
|
|
372
796
|
messages: CachedMessageTuple[];
|
|
797
|
+
toolUsages: CachedToolUsageTuple[];
|
|
373
798
|
}
|
|
374
799
|
|
|
375
800
|
export interface CachedFileState {
|
|
@@ -398,21 +823,29 @@ export async function loadUsageCache(cachePath: string): Promise<Map<string, Cac
|
|
|
398
823
|
typeof entry.mtimeMs !== "number" ||
|
|
399
824
|
typeof entry.sessionId !== "string" ||
|
|
400
825
|
typeof entry.cwd !== "string" ||
|
|
401
|
-
!Array.isArray(entry.messages)
|
|
826
|
+
!Array.isArray(entry.messages) ||
|
|
827
|
+
!Array.isArray(entry.toolUsages)
|
|
402
828
|
) {
|
|
403
829
|
continue;
|
|
404
830
|
}
|
|
405
831
|
const messages: SessionMessage[] = [];
|
|
406
832
|
let valid = true;
|
|
407
833
|
for (const tuple of entry.messages) {
|
|
408
|
-
if (!Array.isArray(tuple) || tuple.length !==
|
|
834
|
+
if (!Array.isArray(tuple) || tuple.length !== 13) {
|
|
409
835
|
valid = false;
|
|
410
836
|
break;
|
|
411
837
|
}
|
|
412
838
|
const provider = names[tuple[0]];
|
|
413
839
|
const model = names[tuple[1]];
|
|
414
840
|
const thinkingLevel = names[tuple[8]];
|
|
415
|
-
|
|
841
|
+
const sourceId = names[tuple[12]];
|
|
842
|
+
if (
|
|
843
|
+
typeof provider !== "string" ||
|
|
844
|
+
typeof model !== "string" ||
|
|
845
|
+
typeof thinkingLevel !== "string" ||
|
|
846
|
+
typeof sourceId !== "string" ||
|
|
847
|
+
(tuple[11] !== 0 && tuple[11] !== 1)
|
|
848
|
+
) {
|
|
416
849
|
valid = false;
|
|
417
850
|
break;
|
|
418
851
|
}
|
|
@@ -420,6 +853,8 @@ export async function loadUsageCache(cachePath: string): Promise<Map<string, Cac
|
|
|
420
853
|
provider,
|
|
421
854
|
model,
|
|
422
855
|
thinkingLevel,
|
|
856
|
+
source: tuple[11] === 1 ? "auxiliary" : "assistant",
|
|
857
|
+
sourceId,
|
|
423
858
|
cost: Number(tuple[2]) || 0,
|
|
424
859
|
input: Number(tuple[3]) || 0,
|
|
425
860
|
output: Number(tuple[4]) || 0,
|
|
@@ -427,19 +862,74 @@ export async function loadUsageCache(cachePath: string): Promise<Map<string, Cac
|
|
|
427
862
|
cacheWrite: Number(tuple[6]) || 0,
|
|
428
863
|
timestamp: Number(tuple[7]) || 0,
|
|
429
864
|
reasoning: Number(tuple[9]) || 0,
|
|
430
|
-
|
|
865
|
+
afterContextChange: tuple[10] === 1,
|
|
866
|
+
});
|
|
867
|
+
}
|
|
868
|
+
if (!valid) continue;
|
|
869
|
+
const toolUsages: ToolUsageRecord[] = [];
|
|
870
|
+
for (const tuple of entry.toolUsages) {
|
|
871
|
+
if (!Array.isArray(tuple) || tuple.length !== 5 || !Array.isArray(tuple[4])) {
|
|
872
|
+
valid = false;
|
|
873
|
+
break;
|
|
874
|
+
}
|
|
875
|
+
const sourceId = names[tuple[0]];
|
|
876
|
+
const runId = names[tuple[3]];
|
|
877
|
+
const reportedUsage = cachedUsageAmount(tuple[2]);
|
|
878
|
+
if (typeof sourceId !== "string" || typeof runId !== "string" || (tuple[2] !== null && !reportedUsage)) {
|
|
879
|
+
valid = false;
|
|
880
|
+
break;
|
|
881
|
+
}
|
|
882
|
+
const children: ChildToolUsage[] = [];
|
|
883
|
+
for (const childTuple of tuple[4]) {
|
|
884
|
+
if (!Array.isArray(childTuple) || childTuple.length !== 3) {
|
|
885
|
+
valid = false;
|
|
886
|
+
break;
|
|
887
|
+
}
|
|
888
|
+
const sessionFile = names[childTuple[1]];
|
|
889
|
+
const usage = cachedUsageAmount(childTuple[2]);
|
|
890
|
+
if (typeof childTuple[0] !== "number" || typeof sessionFile !== "string" || !usage) {
|
|
891
|
+
valid = false;
|
|
892
|
+
break;
|
|
893
|
+
}
|
|
894
|
+
children.push({ resultIndex: childTuple[0], sessionFile, usage });
|
|
895
|
+
}
|
|
896
|
+
if (!valid) break;
|
|
897
|
+
toolUsages.push({
|
|
898
|
+
sourceId,
|
|
899
|
+
timestamp: Number(tuple[1]) || 0,
|
|
900
|
+
reportedUsage,
|
|
901
|
+
runId,
|
|
902
|
+
children,
|
|
431
903
|
});
|
|
432
904
|
}
|
|
433
905
|
if (!valid) continue;
|
|
434
906
|
result.set(filePath, {
|
|
435
907
|
size: entry.size,
|
|
436
908
|
mtimeMs: entry.mtimeMs,
|
|
437
|
-
parsed: { sessionId: entry.sessionId, cwd: entry.cwd, messages },
|
|
909
|
+
parsed: { sessionId: entry.sessionId, cwd: entry.cwd, messages, toolUsages },
|
|
438
910
|
});
|
|
439
911
|
}
|
|
440
912
|
return result;
|
|
441
913
|
}
|
|
442
914
|
|
|
915
|
+
function cachedUsageAmount(value: unknown): UsageAmount | null {
|
|
916
|
+
if (!Array.isArray(value) || value.length !== 6 || value.some((part) => typeof part !== "number" || !Number.isFinite(part))) {
|
|
917
|
+
return null;
|
|
918
|
+
}
|
|
919
|
+
return {
|
|
920
|
+
cost: value[0],
|
|
921
|
+
input: value[1],
|
|
922
|
+
output: value[2],
|
|
923
|
+
cacheRead: value[3],
|
|
924
|
+
cacheWrite: value[4],
|
|
925
|
+
reasoning: value[5],
|
|
926
|
+
};
|
|
927
|
+
}
|
|
928
|
+
|
|
929
|
+
function cacheUsageAmount(usage: UsageAmount): CachedUsageTuple {
|
|
930
|
+
return [usage.cost, usage.input, usage.output, usage.cacheRead, usage.cacheWrite, usage.reasoning];
|
|
931
|
+
}
|
|
932
|
+
|
|
443
933
|
export async function saveUsageCache(cachePath: string, states: Map<string, CachedFileState>): Promise<void> {
|
|
444
934
|
const names: string[] = [];
|
|
445
935
|
const nameIndex = new Map<string, number>();
|
|
@@ -471,7 +961,20 @@ export async function saveUsageCache(cachePath: string, states: Map<string, Cach
|
|
|
471
961
|
m.timestamp,
|
|
472
962
|
intern(m.thinkingLevel),
|
|
473
963
|
m.reasoning,
|
|
474
|
-
m.
|
|
964
|
+
m.afterContextChange ? 1 : 0,
|
|
965
|
+
m.source === "auxiliary" ? 1 : 0,
|
|
966
|
+
intern(m.source === "auxiliary" ? m.sourceId : ""),
|
|
967
|
+
]),
|
|
968
|
+
toolUsages: state.parsed.toolUsages.map((tool): CachedToolUsageTuple => [
|
|
969
|
+
intern(tool.sourceId),
|
|
970
|
+
tool.timestamp,
|
|
971
|
+
tool.reportedUsage ? cacheUsageAmount(tool.reportedUsage) : null,
|
|
972
|
+
intern(tool.runId),
|
|
973
|
+
tool.children.map((child): CachedChildToolUsageTuple => [
|
|
974
|
+
child.resultIndex,
|
|
975
|
+
intern(child.sessionFile),
|
|
976
|
+
cacheUsageAmount(child.usage),
|
|
977
|
+
]),
|
|
475
978
|
]),
|
|
476
979
|
};
|
|
477
980
|
}
|
|
@@ -511,7 +1014,8 @@ function emptyTimeFilteredStats(): TimeFilteredStats {
|
|
|
511
1014
|
function emptyPeriodRawData(): PeriodRawData {
|
|
512
1015
|
return {
|
|
513
1016
|
totalCost: 0,
|
|
514
|
-
|
|
1017
|
+
assistantCost: 0,
|
|
1018
|
+
auxiliaryCost: 0,
|
|
515
1019
|
ctxHigh: { cost: 0, messages: 0 },
|
|
516
1020
|
ctxLow: { cost: 0, messages: 0 },
|
|
517
1021
|
projectCosts: new Map(),
|
|
@@ -581,7 +1085,7 @@ function addToHourlyBuckets(hourly: Map<number, Map<HourlyKey, HourlyCell>>, msg
|
|
|
581
1085
|
cell = { messages: 0, cost: 0, input: 0, output: 0, cacheRead: 0, cacheWrite: 0, reasoning: 0 };
|
|
582
1086
|
bucket.set(key, cell);
|
|
583
1087
|
}
|
|
584
|
-
cell.messages++;
|
|
1088
|
+
if (msg.source === "assistant") cell.messages++;
|
|
585
1089
|
cell.cost += msg.cost;
|
|
586
1090
|
cell.input += msg.input;
|
|
587
1091
|
cell.output += msg.output;
|
|
@@ -594,9 +1098,10 @@ function addToHourlyBuckets(hourly: Map<number, Map<HourlyKey, HourlyCell>>, msg
|
|
|
594
1098
|
function accumulateStats(
|
|
595
1099
|
target: BaseStats,
|
|
596
1100
|
cost: number,
|
|
597
|
-
tokens: { total: number; input: number; output: number; cacheRead: number; cacheWrite: number }
|
|
1101
|
+
tokens: { total: number; input: number; output: number; cacheRead: number; cacheWrite: number },
|
|
1102
|
+
countMessage: boolean
|
|
598
1103
|
): void {
|
|
599
|
-
target.messages++;
|
|
1104
|
+
if (countMessage) target.messages++;
|
|
600
1105
|
target.cost += cost;
|
|
601
1106
|
target.tokens.total += tokens.total;
|
|
602
1107
|
target.tokens.input += tokens.input;
|
|
@@ -683,18 +1188,30 @@ function addMessagesToUsageData(
|
|
|
683
1188
|
providerStats.models.set(msg.model, modelStats);
|
|
684
1189
|
}
|
|
685
1190
|
|
|
1191
|
+
const isAssistant = msg.source === "assistant";
|
|
686
1192
|
modelStats.sessions.add(sessionId);
|
|
687
|
-
accumulateStats(modelStats, msg.cost, tokens);
|
|
1193
|
+
accumulateStats(modelStats, msg.cost, tokens, isAssistant);
|
|
688
1194
|
|
|
689
1195
|
providerStats.sessions.add(sessionId);
|
|
690
|
-
accumulateStats(providerStats, msg.cost, tokens);
|
|
1196
|
+
accumulateStats(providerStats, msg.cost, tokens, isAssistant);
|
|
691
1197
|
|
|
692
|
-
accumulateStats(stats.totals, msg.cost, tokens);
|
|
1198
|
+
accumulateStats(stats.totals, msg.cost, tokens, isAssistant);
|
|
693
1199
|
sessionContributed[period] = true;
|
|
694
1200
|
|
|
695
1201
|
const raw = rawByPeriod[period];
|
|
696
1202
|
raw.totalCost += msg.cost;
|
|
697
|
-
raw.
|
|
1203
|
+
raw.projectCosts.set(project, (raw.projectCosts.get(project) ?? 0) + msg.cost);
|
|
1204
|
+
raw.sessionCosts.set(sessionId, (raw.sessionCosts.get(sessionId) ?? 0) + msg.cost);
|
|
1205
|
+
|
|
1206
|
+
// Auxiliary calls belong in accounting totals, project/session mix, and
|
|
1207
|
+
// burn trend. They are not assistant turns, so do not let their synthetic
|
|
1208
|
+
// model identity or nested context distort turn/cache insights.
|
|
1209
|
+
if (!isAssistant) {
|
|
1210
|
+
raw.auxiliaryCost += msg.cost;
|
|
1211
|
+
continue;
|
|
1212
|
+
}
|
|
1213
|
+
raw.assistantCost += msg.cost;
|
|
1214
|
+
|
|
698
1215
|
const ctx = msg.input + msg.cacheRead + msg.cacheWrite;
|
|
699
1216
|
if (ctx >= CTX_TAX_THRESHOLD) {
|
|
700
1217
|
raw.ctxHigh.cost += msg.cost;
|
|
@@ -703,11 +1220,9 @@ function addMessagesToUsageData(
|
|
|
703
1220
|
raw.ctxLow.cost += msg.cost;
|
|
704
1221
|
raw.ctxLow.messages++;
|
|
705
1222
|
}
|
|
706
|
-
raw.projectCosts.set(project, (raw.projectCosts.get(project) ?? 0) + msg.cost);
|
|
707
|
-
raw.sessionCosts.set(sessionId, (raw.sessionCosts.get(sessionId) ?? 0) + msg.cost);
|
|
708
1223
|
if (mm.isSessionStart) raw.upfrontCost += msg.cost;
|
|
709
1224
|
if (
|
|
710
|
-
!msg.
|
|
1225
|
+
!msg.afterContextChange &&
|
|
711
1226
|
mm.prevCtx >= MISS_MIN_PREV_CONTEXT &&
|
|
712
1227
|
msg.cacheRead < Math.min(MISS_MAX_CACHE_READ, 0.3 * mm.prevCtx)
|
|
713
1228
|
) {
|
|
@@ -729,6 +1244,103 @@ function addMessagesToUsageData(
|
|
|
729
1244
|
if (sessionContributed.allTime) data.allTime.totals.sessions++;
|
|
730
1245
|
}
|
|
731
1246
|
|
|
1247
|
+
// =============================================================================
|
|
1248
|
+
// Nested tool-usage accounting
|
|
1249
|
+
// =============================================================================
|
|
1250
|
+
|
|
1251
|
+
// Recognised nested-agent tool results report usage for child runs that
|
|
1252
|
+
// pi-subagents also persists as ordinary session files, which this scan
|
|
1253
|
+
// already counts with full model attribution. When every child session file
|
|
1254
|
+
// behind a report is part of the scan, the children speak for themselves and
|
|
1255
|
+
// the parent's aggregate is skipped. Otherwise the aggregate (or, for pre-0.81
|
|
1256
|
+
// legacy entries, each unresolved child's reported usage) is counted under
|
|
1257
|
+
// Tools / summaries.
|
|
1258
|
+
//
|
|
1259
|
+
// Copied branch history gets a new parent filename, so a copy's runId-derived
|
|
1260
|
+
// child paths can dangle even though the original's resolve. Resolved child
|
|
1261
|
+
// identities are therefore unioned across all copies of an entry before any
|
|
1262
|
+
// emission, and identical emissions collapse in the sourceId dedupe.
|
|
1263
|
+
|
|
1264
|
+
interface ScannedSessionIndex {
|
|
1265
|
+
/** Resolved paths of scanned files that have a session header. */
|
|
1266
|
+
paths: Set<string>;
|
|
1267
|
+
/** Directory of each scanned file → number of scanned files inside it. */
|
|
1268
|
+
fileCountByDir: Map<string, number>;
|
|
1269
|
+
}
|
|
1270
|
+
|
|
1271
|
+
function buildScannedSessionIndex(states: Map<string, CachedFileState>): ScannedSessionIndex {
|
|
1272
|
+
const paths = new Set<string>();
|
|
1273
|
+
const fileCountByDir = new Map<string, number>();
|
|
1274
|
+
for (const [filePath, state] of states) {
|
|
1275
|
+
if (!state.parsed.sessionId) continue;
|
|
1276
|
+
const resolved = resolve(filePath);
|
|
1277
|
+
paths.add(resolved);
|
|
1278
|
+
const dir = dirname(resolved);
|
|
1279
|
+
fileCountByDir.set(dir, (fileCountByDir.get(dir) ?? 0) + 1);
|
|
1280
|
+
}
|
|
1281
|
+
return { paths, fileCountByDir };
|
|
1282
|
+
}
|
|
1283
|
+
|
|
1284
|
+
function childSessionScanned(
|
|
1285
|
+
parentFilePath: string,
|
|
1286
|
+
tool: ToolUsageRecord,
|
|
1287
|
+
child: ChildToolUsage,
|
|
1288
|
+
index: ScannedSessionIndex
|
|
1289
|
+
): boolean {
|
|
1290
|
+
if (child.sessionFile) {
|
|
1291
|
+
const explicit = isAbsolute(child.sessionFile)
|
|
1292
|
+
? resolve(child.sessionFile)
|
|
1293
|
+
: resolve(dirname(parentFilePath), child.sessionFile);
|
|
1294
|
+
if (index.paths.has(explicit)) return true;
|
|
1295
|
+
}
|
|
1296
|
+
if (tool.runId) {
|
|
1297
|
+
const runDir = resolve(dirname(parentFilePath), basename(parentFilePath, ".jsonl"), tool.runId, `run-${child.resultIndex}`);
|
|
1298
|
+
// The run directory holds exactly one session per child run, so either the
|
|
1299
|
+
// conventional name or a lone scanned file inside it identifies the child.
|
|
1300
|
+
if (index.paths.has(join(runDir, "session.jsonl"))) return true;
|
|
1301
|
+
if (index.fileCountByDir.get(runDir) === 1) return true;
|
|
1302
|
+
}
|
|
1303
|
+
return false;
|
|
1304
|
+
}
|
|
1305
|
+
|
|
1306
|
+
/** Identity of one child slot of one tool entry, stable across copied history. */
|
|
1307
|
+
function toolChildIdentity(tool: ToolUsageRecord, child: ChildToolUsage): string {
|
|
1308
|
+
const fingerprint = child.usage.input + child.usage.output + child.usage.cacheRead + child.usage.cacheWrite;
|
|
1309
|
+
return `${tool.sourceId}:${tool.timestamp}:${child.resultIndex}:${fingerprint}`;
|
|
1310
|
+
}
|
|
1311
|
+
|
|
1312
|
+
function resolvedToolChildIdentities(states: Map<string, CachedFileState>, index: ScannedSessionIndex): Set<string> {
|
|
1313
|
+
const resolved = new Set<string>();
|
|
1314
|
+
for (const [filePath, state] of states) {
|
|
1315
|
+
for (const tool of state.parsed.toolUsages) {
|
|
1316
|
+
if (!tool.sourceId) continue;
|
|
1317
|
+
for (const child of tool.children) {
|
|
1318
|
+
if (childSessionScanned(filePath, tool, child, index)) resolved.add(toolChildIdentity(tool, child));
|
|
1319
|
+
}
|
|
1320
|
+
}
|
|
1321
|
+
}
|
|
1322
|
+
return resolved;
|
|
1323
|
+
}
|
|
1324
|
+
|
|
1325
|
+
function toolUsageMessages(
|
|
1326
|
+
parentFilePath: string,
|
|
1327
|
+
tool: ToolUsageRecord,
|
|
1328
|
+
index: ScannedSessionIndex,
|
|
1329
|
+
resolvedChildren: Set<string>
|
|
1330
|
+
): SessionMessage[] {
|
|
1331
|
+
const scanned = (child: ChildToolUsage) =>
|
|
1332
|
+
childSessionScanned(parentFilePath, tool, child, index) ||
|
|
1333
|
+
(tool.sourceId !== "" && resolvedChildren.has(toolChildIdentity(tool, child)));
|
|
1334
|
+
if (tool.reportedUsage) {
|
|
1335
|
+
if (tool.children.length > 0 && tool.children.every(scanned)) return [];
|
|
1336
|
+
return [auxiliaryMessage(tool.reportedUsage, tool.timestamp, tool.sourceId)];
|
|
1337
|
+
}
|
|
1338
|
+
// Before Pi 0.81, recognised nested-agent tools persisted per-child usage in
|
|
1339
|
+
// details only. Count just the children whose sessions this scan cannot see.
|
|
1340
|
+
return tool.children
|
|
1341
|
+
.filter((child) => !scanned(child))
|
|
1342
|
+
.map((child) => auxiliaryMessage(child.usage, tool.timestamp, tool.sourceId ? `${tool.sourceId}:child:${child.resultIndex}` : ""));
|
|
1343
|
+
}
|
|
732
1344
|
// =============================================================================
|
|
733
1345
|
// Collection orchestration
|
|
734
1346
|
// =============================================================================
|
|
@@ -925,6 +1537,8 @@ export async function collectUsageData(options: CollectUsageOptions = {}): Promi
|
|
|
925
1537
|
const costByDayIdx = new Map<number, number>();
|
|
926
1538
|
const seenSessions = new Set<string>();
|
|
927
1539
|
const seenHashes = new Set<string>();
|
|
1540
|
+
const scannedSessions = buildScannedSessionIndex(current);
|
|
1541
|
+
const resolvedToolChildren = resolvedToolChildIdentities(current, scannedSessions);
|
|
928
1542
|
let processedFiles = 0;
|
|
929
1543
|
|
|
930
1544
|
for (const filePath of filePaths) {
|
|
@@ -939,15 +1553,26 @@ export async function collectUsageData(options: CollectUsageOptions = {}): Promi
|
|
|
939
1553
|
// Deduplicate copied history across branched session files, computing
|
|
940
1554
|
// adjacency metadata (idle gaps, previous context) on the raw file order
|
|
941
1555
|
// so branch copies do not distort miss classification.
|
|
942
|
-
const
|
|
1556
|
+
const toolMessages = state.parsed.toolUsages.flatMap((tool) => toolUsageMessages(filePath, tool, scannedSessions, resolvedToolChildren));
|
|
1557
|
+
const rawMsgs = toolMessages.length > 0 ? [...state.parsed.messages, ...toolMessages] : state.parsed.messages;
|
|
943
1558
|
const deduped: SessionMessage[] = [];
|
|
944
1559
|
const meta: MessageMeta[] = [];
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
1560
|
+
let previousAssistant: SessionMessage | null = null;
|
|
1561
|
+
for (const m of rawMsgs) {
|
|
1562
|
+
// Auxiliary usage is interleaved with conversation entries, but it must
|
|
1563
|
+
// not become the "previous message" for cache-miss classification.
|
|
1564
|
+
const prev = m.source === "assistant" ? previousAssistant : null;
|
|
1565
|
+
if (m.source === "assistant") previousAssistant = m;
|
|
1566
|
+
|
|
1567
|
+
// Pi entry ids survive copied branch history and distinguish parallel
|
|
1568
|
+
// tool results that happen to report identical usage in the same ms.
|
|
1569
|
+
const tokenFingerprint = m.input + m.output + m.cacheRead + m.cacheWrite;
|
|
1570
|
+
const hash =
|
|
1571
|
+
m.source === "auxiliary" && m.sourceId
|
|
1572
|
+
? `auxiliary:${m.sourceId}:${m.timestamp}:${tokenFingerprint}`
|
|
1573
|
+
: `${m.source}:${m.timestamp}:${tokenFingerprint}`;
|
|
948
1574
|
if (seenHashes.has(hash)) continue;
|
|
949
1575
|
seenHashes.add(hash);
|
|
950
|
-
const prev = i > 0 ? rawMsgs[i - 1]! : null;
|
|
951
1576
|
deduped.push(m);
|
|
952
1577
|
meta.push({
|
|
953
1578
|
gapMs: prev && prev.timestamp > 0 && m.timestamp > 0 ? m.timestamp - prev.timestamp : -1,
|
|
@@ -957,9 +1582,10 @@ export async function collectUsageData(options: CollectUsageOptions = {}): Promi
|
|
|
957
1582
|
});
|
|
958
1583
|
}
|
|
959
1584
|
if (deduped.length === 0) continue;
|
|
960
|
-
|
|
1585
|
+
const firstAssistantIndex = deduped.findIndex((m) => m.source === "assistant");
|
|
1586
|
+
if (firstAssistantIndex !== -1 && !seenSessions.has(state.parsed.sessionId)) {
|
|
961
1587
|
seenSessions.add(state.parsed.sessionId);
|
|
962
|
-
meta[
|
|
1588
|
+
meta[firstAssistantIndex]!.isSessionStart = true;
|
|
963
1589
|
}
|
|
964
1590
|
|
|
965
1591
|
addMessagesToUsageData(
|
|
@@ -1045,42 +1671,44 @@ function fmtPercent(p: number): string {
|
|
|
1045
1671
|
* distinct empty-state for that case.
|
|
1046
1672
|
*/
|
|
1047
1673
|
function computeInsights(raw: PeriodRawData, trend: TrendInfo | null): PeriodInsights {
|
|
1048
|
-
if (raw.
|
|
1674
|
+
if (raw.totalCost <= 0) {
|
|
1049
1675
|
return { insights: [] };
|
|
1050
1676
|
}
|
|
1051
1677
|
const total = raw.totalCost;
|
|
1678
|
+
const assistantTotal = raw.assistantCost;
|
|
1679
|
+
const assistantPctLabel = raw.auxiliaryCost > 0 ? "assistant-message cost" : "this period";
|
|
1052
1680
|
const insights: Insight[] = [];
|
|
1053
1681
|
|
|
1054
1682
|
// --- Alarms (listed first) ---
|
|
1055
1683
|
|
|
1056
|
-
const ttlPct = (raw.ttlMissCost /
|
|
1684
|
+
const ttlPct = assistantTotal > 0 ? (raw.ttlMissCost / assistantTotal) * 100 : 0;
|
|
1057
1685
|
if (ttlPct >= CACHE_MISS_ALARM_PERCENT && raw.ttlMissCost >= CACHE_MISS_ALARM_MIN_COST) {
|
|
1058
1686
|
insights.push({
|
|
1059
1687
|
kind: "alarm",
|
|
1060
1688
|
stat: fmtMoney(raw.ttlMissCost),
|
|
1061
|
-
headline: `spent resuming conversations after a break (${fmtPercent(ttlPct)} of
|
|
1689
|
+
headline: `spent resuming conversations after a break (${fmtPercent(ttlPct)} of ${assistantPctLabel})`,
|
|
1062
1690
|
advice:
|
|
1063
1691
|
"Sent context is only reusable for a few minutes. After a longer pause, the next message pays to send the whole conversation again. Replying while a session is fresh avoids this.",
|
|
1064
1692
|
});
|
|
1065
1693
|
}
|
|
1066
1694
|
|
|
1067
|
-
const switchPct = (raw.modelSwitchMissCost /
|
|
1695
|
+
const switchPct = assistantTotal > 0 ? (raw.modelSwitchMissCost / assistantTotal) * 100 : 0;
|
|
1068
1696
|
if (switchPct >= CACHE_MISS_ALARM_PERCENT && raw.modelSwitchMissCost >= CACHE_MISS_ALARM_MIN_COST) {
|
|
1069
1697
|
insights.push({
|
|
1070
1698
|
kind: "alarm",
|
|
1071
1699
|
stat: fmtMoney(raw.modelSwitchMissCost),
|
|
1072
|
-
headline: `spent switching models mid-conversation (${fmtPercent(switchPct)} of
|
|
1700
|
+
headline: `spent switching models mid-conversation (${fmtPercent(switchPct)} of ${assistantPctLabel})`,
|
|
1073
1701
|
advice:
|
|
1074
1702
|
"Changing model re-sends the whole conversation at full price — the previous model's saved context doesn't transfer. Switching between tasks instead of mid-conversation avoids this.",
|
|
1075
1703
|
});
|
|
1076
1704
|
}
|
|
1077
1705
|
|
|
1078
|
-
const prefixPct = (raw.prefixMissCost /
|
|
1706
|
+
const prefixPct = assistantTotal > 0 ? (raw.prefixMissCost / assistantTotal) * 100 : 0;
|
|
1079
1707
|
if (prefixPct >= CACHE_MISS_ALARM_PERCENT && raw.prefixMissCost >= CACHE_MISS_ALARM_MIN_COST) {
|
|
1080
1708
|
insights.push({
|
|
1081
1709
|
kind: "alarm",
|
|
1082
1710
|
stat: fmtMoney(raw.prefixMissCost),
|
|
1083
|
-
headline: `spent re-sending conversations mid-session (${fmtPercent(prefixPct)} of
|
|
1711
|
+
headline: `spent re-sending conversations mid-session (${fmtPercent(prefixPct)} of ${assistantPctLabel})`,
|
|
1084
1712
|
advice:
|
|
1085
1713
|
"These messages paid full price for context that had already been sent — with no break, compaction, or model switch to explain it. Usually a tool or workflow is restarting or rewriting conversations. Worth a look if it stays high.",
|
|
1086
1714
|
});
|
|
@@ -1100,17 +1728,17 @@ function computeInsights(raw: PeriodRawData, trend: TrendInfo | null): PeriodIns
|
|
|
1100
1728
|
}
|
|
1101
1729
|
}
|
|
1102
1730
|
|
|
1103
|
-
const upfrontPct = (raw.upfrontCost /
|
|
1731
|
+
const upfrontPct = assistantTotal > 0 ? (raw.upfrontCost / assistantTotal) * 100 : 0;
|
|
1104
1732
|
if (upfrontPct >= UPFRONT_ALARM_PERCENT) {
|
|
1105
1733
|
insights.push({
|
|
1106
1734
|
kind: "alarm",
|
|
1107
1735
|
stat: fmtMoney(raw.upfrontCost),
|
|
1108
|
-
headline: `spent on the opening message of new sessions (${fmtPercent(upfrontPct)} of
|
|
1736
|
+
headline: `spent on the opening message of new sessions (${fmtPercent(upfrontPct)} of ${assistantPctLabel})`,
|
|
1109
1737
|
advice: "A session's first message sends everything from scratch. Fewer, longer sessions cut this overhead.",
|
|
1110
1738
|
});
|
|
1111
1739
|
}
|
|
1112
1740
|
|
|
1113
|
-
if (
|
|
1741
|
+
if (assistantTotal >= LEVERAGE_MIN_COST && raw.freshTokens >= LEVERAGE_MIN_FRESH_TOKENS) {
|
|
1114
1742
|
const leverage = raw.cacheReadTokens / raw.freshTokens;
|
|
1115
1743
|
if (leverage < LEVERAGE_FLOOR) {
|
|
1116
1744
|
insights.push({
|
|
@@ -1125,8 +1753,20 @@ function computeInsights(raw: PeriodRawData, trend: TrendInfo | null): PeriodIns
|
|
|
1125
1753
|
|
|
1126
1754
|
// --- Structure (always-on) ---
|
|
1127
1755
|
|
|
1128
|
-
if (raw.
|
|
1129
|
-
const pct = (raw.
|
|
1756
|
+
if (raw.auxiliaryCost > 0) {
|
|
1757
|
+
const pct = (raw.auxiliaryCost / total) * 100;
|
|
1758
|
+
if (pct >= 1) {
|
|
1759
|
+
insights.push({
|
|
1760
|
+
kind: "structure",
|
|
1761
|
+
stat: fmtPercent(pct),
|
|
1762
|
+
headline: "of your cost came from usage reported by tools and conversation summaries",
|
|
1763
|
+
advice: "Pi records this separately because it cannot be attributed reliably to a specific provider and model.",
|
|
1764
|
+
});
|
|
1765
|
+
}
|
|
1766
|
+
}
|
|
1767
|
+
|
|
1768
|
+
if (raw.ctxHigh.messages > 0 && assistantTotal > 0) {
|
|
1769
|
+
const pct = (raw.ctxHigh.cost / assistantTotal) * 100;
|
|
1130
1770
|
if (pct >= 1) {
|
|
1131
1771
|
const avgHigh = raw.ctxHigh.cost / raw.ctxHigh.messages;
|
|
1132
1772
|
const avgLow = raw.ctxLow.messages > 0 ? raw.ctxLow.cost / raw.ctxLow.messages : 0;
|
|
@@ -1137,7 +1777,7 @@ function computeInsights(raw: PeriodRawData, trend: TrendInfo | null): PeriodIns
|
|
|
1137
1777
|
insights.push({
|
|
1138
1778
|
kind: "structure",
|
|
1139
1779
|
stat: fmtPercent(pct),
|
|
1140
|
-
headline: `of your cost came from messages with ≥${formatThresholdTokens(CTX_TAX_THRESHOLD)} tokens loaded${cmp}`,
|
|
1780
|
+
headline: `of your ${raw.auxiliaryCost > 0 ? "assistant-message cost" : "cost"} came from messages with ≥${formatThresholdTokens(CTX_TAX_THRESHOLD)} tokens loaded${cmp}`,
|
|
1141
1781
|
advice: "Long conversations cost more per message. /compact mid-task and /clear between tasks keep them lean.",
|
|
1142
1782
|
});
|
|
1143
1783
|
}
|
package/index.ts
CHANGED
|
@@ -673,7 +673,11 @@ class UsageComponent {
|
|
|
673
673
|
const th = this.theme;
|
|
674
674
|
const stats = this.data[this.activeTab];
|
|
675
675
|
const { insights } = stats.insights;
|
|
676
|
-
const
|
|
676
|
+
const hasUsage =
|
|
677
|
+
stats.totals.messages > 0 ||
|
|
678
|
+
stats.totals.cost > 0 ||
|
|
679
|
+
stats.totals.tokens.total > 0 ||
|
|
680
|
+
stats.totals.tokens.cacheRead > 0;
|
|
677
681
|
const hasCost = stats.totals.cost > 0;
|
|
678
682
|
const lines: string[] = [];
|
|
679
683
|
|
|
@@ -687,7 +691,7 @@ class UsageComponent {
|
|
|
687
691
|
}
|
|
688
692
|
lines.push("");
|
|
689
693
|
|
|
690
|
-
if (!
|
|
694
|
+
if (!hasUsage) {
|
|
691
695
|
lines.push(th.fg("dim", " No usage recorded for this period."));
|
|
692
696
|
lines.push("");
|
|
693
697
|
return lines;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@tmustier/pi-usage-extension",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.5",
|
|
4
4
|
"description": "Usage statistics dashboard for Pi sessions.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Thomas Mustier",
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
"@earendil-works/pi-tui": "*"
|
|
26
26
|
},
|
|
27
27
|
"devDependencies": {
|
|
28
|
-
"@earendil-works/pi-coding-agent": "
|
|
29
|
-
"@earendil-works/pi-tui": "
|
|
28
|
+
"@earendil-works/pi-coding-agent": "0.87.0",
|
|
29
|
+
"@earendil-works/pi-tui": "0.87.0"
|
|
30
30
|
}
|
|
31
31
|
}
|