@narumitw/pi-analytics 0.46.0 → 0.49.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +43 -68
- package/package.json +3 -4
- package/src/analytics.ts +102 -109
- package/src/collector.ts +4 -0
- package/src/menu.ts +33 -16
- package/src/skills.ts +6 -7
- package/src/storage/files.ts +434 -0
- package/src/storage/format.ts +281 -0
- package/src/storage/queries.ts +99 -151
- package/src/storage/store.ts +32 -227
- package/src/storage/database.ts +0 -126
- package/src/storage/migrations.ts +0 -257
package/README.md
CHANGED
|
@@ -9,15 +9,15 @@
|
|
|
9
9
|
|
|
10
10
|
## ✨ Features
|
|
11
11
|
|
|
12
|
-
- Starts collecting settled Pi response cycles after installation with no configuration.
|
|
12
|
+
- Starts collecting settled Pi response cycles after installation with no configuration or startup I/O.
|
|
13
13
|
- Breaks skill activations down by explicit user invocation, model loading, provider, and model.
|
|
14
14
|
- Counts tool calls, failures, average duration, and model attribution.
|
|
15
15
|
- Reports logical LLM calls per response with average, median, P95, maximum, and distribution buckets.
|
|
16
16
|
- Separates HTTP 429/5xx responses, conservative connection-error categories, recovered errors, and terminal provider failures.
|
|
17
17
|
- Offers Today, rolling 7-day, rolling 30-day, and all-time views through one `/analytics` TUI/RPC dashboard.
|
|
18
|
-
- Stores only content-free metadata in
|
|
19
|
-
- Uses
|
|
20
|
-
- Never starts a server
|
|
18
|
+
- Stores only content-free metadata in private, versioned JSON Lines files.
|
|
19
|
+
- Uses one writer file per Pi runtime, so concurrent Pi processes never share a routine writer lock.
|
|
20
|
+
- Never starts a server or sends analytics anywhere.
|
|
21
21
|
|
|
22
22
|
## 📦 Install
|
|
23
23
|
|
|
@@ -39,15 +39,7 @@ Try a local checkout from the repository root:
|
|
|
39
39
|
pi -e ./experimental/pi-analytics
|
|
40
40
|
```
|
|
41
41
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
The embedded `@tursodatabase/database` dependency currently publishes native binaries for:
|
|
45
|
-
|
|
46
|
-
- Linux x64 and arm64 with glibc;
|
|
47
|
-
- macOS arm64; and
|
|
48
|
-
- Windows x64.
|
|
49
|
-
|
|
50
|
-
On another platform, Pi still loads the extension and `/analytics` remains available, but collection is disabled and the dashboard explains the supported platform boundary. The database engine is pre-1.0. Analytics are treated as non-critical derived metadata. If the history itself matters, stop every Pi process using the extension and back up both `pi-analytics.db` and `pi-analytics.db-wal`; copying only the main file is not a complete backup.
|
|
42
|
+
The storage implementation uses Node's built-in filesystem APIs and has no native database dependency.
|
|
51
43
|
|
|
52
44
|
## 🚀 Quick start
|
|
53
45
|
|
|
@@ -72,7 +64,7 @@ Provider errors 4
|
|
|
72
64
|
Recovered errors 3
|
|
73
65
|
```
|
|
74
66
|
|
|
75
|
-
Use the menu to change the time range or browse Skills, Tools, Provider reliability, Response cycles, and Data & privacy. Only fully settled cycles are included; active work is omitted
|
|
67
|
+
Use the menu to change the time range or browse Skills, Tools, Provider reliability, Response cycles, and Data & privacy. Only fully settled cycles are included; active work is omitted.
|
|
76
68
|
|
|
77
69
|
## 📐 Metric definitions
|
|
78
70
|
|
|
@@ -86,22 +78,15 @@ An **LLM call** is one logical provider generation. A provider may make several
|
|
|
86
78
|
|
|
87
79
|
An activation is **User initiated** when an observed interactive or RPC `/skill:<name>` input is associated with an active or subsequently started response cycle. This includes skill commands queued while Pi is streaming. It is **Model initiated** when the built-in `read` tool successfully loads the exact canonical `SKILL.md` path Pi discovered. A skill is counted at most once per response cycle, and explicit user use takes precedence.
|
|
88
80
|
|
|
89
|
-
Pi does not expose a first-class skill-invocation event or a post-chain acceptance event for input observers. Non-standard loading such as `bash` plus `cat SKILL.md`, unsuccessful reads, and provider behavior invisible to Pi are not counted
|
|
81
|
+
Pi does not expose a first-class skill-invocation event or a post-chain acceptance event for input observers. Non-standard loading such as `bash` plus `cat SKILL.md`, unsuccessful reads, and provider behavior invisible to Pi are not counted.
|
|
90
82
|
|
|
91
83
|
### Tools
|
|
92
84
|
|
|
93
|
-
A tool call starts at Pi's `tool_execution_start` event and finishes at `tool_execution_end`. The extension stores the tool name, model attribution, timing, completion state, and final error flag. It cannot reliably distinguish another extension blocking a call from every other tool error, so
|
|
85
|
+
A tool call starts at Pi's `tool_execution_start` event and finishes at `tool_execution_end`. The extension stores the tool name, model attribution, timing, completion state, and final error flag. It cannot reliably distinguish another extension blocking a call from every other tool error, so both appear as errors.
|
|
94
86
|
|
|
95
87
|
### Provider reliability
|
|
96
88
|
|
|
97
|
-
Pi exposes HTTP responses and final assistant failures, not every provider-SDK transport retry. The dashboard therefore labels these values as **observed provider errors**. It reports
|
|
98
|
-
|
|
99
|
-
- HTTP 429 and 5xx counts;
|
|
100
|
-
- DNS, timeout, connection-refused, connection-reset, TLS, other-network, and other-provider categories;
|
|
101
|
-
- recovered errors; and
|
|
102
|
-
- terminal failures.
|
|
103
|
-
|
|
104
|
-
Error messages are classified in memory and discarded. Raw error text is never stored.
|
|
89
|
+
Pi exposes HTTP responses and final assistant failures, not every provider-SDK transport retry. The dashboard therefore labels these values as **observed provider errors**. It reports HTTP 429 and 5xx counts; conservative DNS, timeout, connection, TLS, network, and provider categories; recovered errors; and terminal failures. Error messages are classified in memory and discarded.
|
|
105
90
|
|
|
106
91
|
## 💬 Command
|
|
107
92
|
|
|
@@ -111,66 +96,56 @@ Error messages are classified in memory and discarded. Raw error text is never s
|
|
|
111
96
|
|
|
112
97
|
The command accepts no arguments. TUI mode uses the full dashboard; RPC mode adapts the same standard screens to dialogs. Print and JSON modes reject the interactive command observably instead of writing ad hoc protocol output.
|
|
113
98
|
|
|
114
|
-
The root menu contains
|
|
115
|
-
|
|
116
|
-
```text
|
|
117
|
-
Change time range
|
|
118
|
-
Skills
|
|
119
|
-
Tools
|
|
120
|
-
Provider reliability
|
|
121
|
-
Response cycles
|
|
122
|
-
Data & privacy
|
|
123
|
-
Close
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
Skills and Tools are searchable browse views with details and model breakdowns. Escape goes Back from nested screens and closes the root. Ctrl+C closes the menu. Cancelling data deletion has no side effects.
|
|
99
|
+
The root menu contains Change time range, Skills, Tools, Provider reliability, Response cycles, Data & privacy, and Close. Skills and Tools are searchable browse views with details and model breakdowns. Escape goes Back from nested screens and closes the root. Ctrl+C closes the menu. Cancelling data deletion has no side effects.
|
|
127
100
|
|
|
128
101
|
## 🔐 Local data and privacy
|
|
129
102
|
|
|
130
|
-
|
|
103
|
+
Current analytics live under:
|
|
131
104
|
|
|
132
105
|
```text
|
|
133
|
-
<pi-agent-directory>/pi-analytics
|
|
106
|
+
<pi-agent-directory>/pi-analytics/
|
|
107
|
+
├── current
|
|
108
|
+
└── generations/
|
|
109
|
+
└── <opaque-generation-id>/
|
|
110
|
+
└── <opaque-writer-id>.jsonl
|
|
134
111
|
```
|
|
135
112
|
|
|
136
|
-
|
|
113
|
+
The opaque IDs are storage coordination identifiers generated by the extension; they are not Pi session IDs. On Unix, directories are restricted to mode `0700` and files to `0600`. Linked storage roots, markers, and writer files are rejected.
|
|
114
|
+
|
|
115
|
+
Stored fields are limited to timestamps and durations; extension-generated record IDs; provider/model IDs and thinking level; tool and skill names; user/model skill source; counts, outcomes, and completion states; HTTP status codes; and classified provider-error categories. Provider-supplied tool-call IDs are replaced with local ordinals before publication.
|
|
116
|
+
|
|
117
|
+
The extension does **not** store prompts, responses, thinking content, tool arguments or results, raw error messages, HTTP headers, cwd/project/file paths, session names or IDs, or credentials.
|
|
137
118
|
|
|
138
|
-
|
|
119
|
+
Each settled response is one versioned, newline-terminated frame. Frames larger than 1 MiB are dropped. Local writes receive a 500 ms cancellation deadline; Node filesystem cancellation is best-effort, so an operating-system request that has already begun may still finish. The extension reports the first failed or timed-out write and a later recovery without exposing filesystem errors.
|
|
139
120
|
|
|
140
|
-
-
|
|
141
|
-
- provider/model IDs and thinking level;
|
|
142
|
-
- tool and skill names;
|
|
143
|
-
- user/model skill source;
|
|
144
|
-
- counts, outcomes, and completion states;
|
|
145
|
-
- HTTP status codes; and
|
|
146
|
-
- classified provider-error categories.
|
|
121
|
+
`/analytics` streams and validates the active generation, checks cancellation between files and records, and periodically yields to the event loop. A crash-truncated final frame is ignored; completed malformed frames and unsupported format versions fail closed without replacing existing files.
|
|
147
122
|
|
|
148
|
-
|
|
123
|
+
### Clear analytics data
|
|
149
124
|
|
|
150
|
-
|
|
151
|
-
- tool arguments or results;
|
|
152
|
-
- raw error messages or HTTP headers;
|
|
153
|
-
- cwd, project names, file paths, session names, or Pi session IDs; or
|
|
154
|
-
- credentials.
|
|
125
|
+
Choose **Data & privacy → Clear analytics data…** to atomically publish a fresh active generation. Other Pi processes observe that generation before their next write. Records racing with Clear may land immediately before or after the generation switch.
|
|
155
126
|
|
|
156
|
-
|
|
127
|
+
The extension then removes the previous generation. If another process still has an obsolete file in use, Clear remains logically complete and reports that physical cleanup is incomplete; stop other Pi processes and clear again. Clearing files is not a secure-erasure guarantee for underlying storage media.
|
|
157
128
|
|
|
158
|
-
|
|
129
|
+
## 🧭 Legacy SQLite data
|
|
159
130
|
|
|
160
|
-
|
|
131
|
+
Versions that used Turso/SQLite stored data in:
|
|
161
132
|
|
|
162
|
-
|
|
133
|
+
```text
|
|
134
|
+
<pi-agent-directory>/pi-analytics.db
|
|
135
|
+
<pi-agent-directory>/pi-analytics.db-wal
|
|
136
|
+
```
|
|
163
137
|
|
|
164
|
-
The
|
|
138
|
+
The JSONL version deliberately does not open, import, migrate, delete, or rewrite those files, so startup cannot re-enter the old native database path. New analytics start empty.
|
|
165
139
|
|
|
166
|
-
|
|
140
|
+
If legacy history matters, stop every old Pi process first and preserve both files together. If it does not matter, stop every old Pi process before deleting both files manually. Never copy or remove only the main DB while an old process may still own its WAL.
|
|
167
141
|
|
|
168
|
-
## 🚧
|
|
142
|
+
## 🚧 Limitations
|
|
169
143
|
|
|
170
144
|
- There are no retention settings; records remain until explicitly cleared.
|
|
171
|
-
-
|
|
145
|
+
- Analytics are best-effort derived metadata. A failed or interrupted local write may be omitted.
|
|
146
|
+
- Large all-time histories require scanning the active JSONL generation when the dashboard opens.
|
|
147
|
+
- Prometheus, JSON/CSV export, cloud sync, browser dashboards, token/cost reporting, and project attribution are not included.
|
|
172
148
|
- Statistics cover only events visible through Pi's public extension API.
|
|
173
|
-
- Clearing rows is logical deletion, not a secure-erasure guarantee for underlying storage media.
|
|
174
149
|
|
|
175
150
|
## 🗂️ Package layout
|
|
176
151
|
|
|
@@ -185,10 +160,10 @@ experimental/pi-analytics/
|
|
|
185
160
|
│ ├── menu.ts # TUI/RPC analytics dashboard
|
|
186
161
|
│ ├── types.ts # Observation records
|
|
187
162
|
│ └── storage/
|
|
188
|
-
│ ├──
|
|
189
|
-
│ ├──
|
|
190
|
-
│ ├── queries.ts #
|
|
191
|
-
│ └── store.ts #
|
|
163
|
+
│ ├── files.ts # Private generations, writes, reads, and Clear
|
|
164
|
+
│ ├── format.ts # Versioned JSONL codec and validation
|
|
165
|
+
│ ├── queries.ts # Incremental aggregate projections
|
|
166
|
+
│ └── store.ts # Lifecycle-safe storage facade
|
|
192
167
|
├── test/
|
|
193
168
|
├── README.md
|
|
194
169
|
├── LICENSE
|
|
@@ -198,7 +173,7 @@ experimental/pi-analytics/
|
|
|
198
173
|
|
|
199
174
|
## 🔎 Keywords
|
|
200
175
|
|
|
201
|
-
Pi extension, Pi coding agent, local analytics, agent skills, tool usage, model calls, provider reliability,
|
|
176
|
+
Pi extension, Pi coding agent, local analytics, agent skills, tool usage, model calls, provider reliability, JSON Lines, content-free metrics.
|
|
202
177
|
|
|
203
178
|
## 📄 License
|
|
204
179
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@narumitw/pi-analytics",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.49.0",
|
|
4
4
|
"description": "Local-first usage analytics for Pi models, skills, tools, and reliability.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
"pi",
|
|
12
12
|
"analytics",
|
|
13
13
|
"metrics",
|
|
14
|
-
"
|
|
14
|
+
"jsonl"
|
|
15
15
|
],
|
|
16
16
|
"files": [
|
|
17
17
|
"src",
|
|
@@ -29,8 +29,7 @@
|
|
|
29
29
|
"typecheck": "tsc --noEmit"
|
|
30
30
|
},
|
|
31
31
|
"dependencies": {
|
|
32
|
-
"@narumitw/pi-tui-kit": "^0.
|
|
33
|
-
"@tursodatabase/database": "^0.7.2"
|
|
32
|
+
"@narumitw/pi-tui-kit": "^0.46.0"
|
|
34
33
|
},
|
|
35
34
|
"peerDependencies": {
|
|
36
35
|
"@earendil-works/pi-coding-agent": "*"
|
package/src/analytics.ts
CHANGED
|
@@ -8,46 +8,39 @@ import {
|
|
|
8
8
|
import { ResponseCollector } from "./collector.js";
|
|
9
9
|
import { type AnalyticsMenuDataSource, showAnalyticsMenu } from "./menu.js";
|
|
10
10
|
import { SkillTracker } from "./skills.js";
|
|
11
|
-
import {
|
|
12
|
-
import {
|
|
13
|
-
ChecksumMismatchError,
|
|
14
|
-
MigrationFailedError,
|
|
15
|
-
NewerSchemaError,
|
|
16
|
-
} from "./storage/migrations.js";
|
|
11
|
+
import type { ClearAnalyticsResult } from "./storage/files.js";
|
|
17
12
|
import type { AnalyticsSnapshot, TimeRange } from "./storage/queries.js";
|
|
18
13
|
import { AnalyticsStore } from "./storage/store.js";
|
|
19
14
|
import type { ModelIdentity, SettledRun, TriggerSource } from "./types.js";
|
|
20
15
|
|
|
21
16
|
const EXPERIMENTAL_WARNING = "pi-analytics is experimental; its metrics and dashboard may change.";
|
|
17
|
+
const STORAGE_DIRECTORY = "pi-analytics";
|
|
22
18
|
|
|
23
19
|
export interface AnalyticsStorePort {
|
|
24
20
|
readonly path: string;
|
|
25
|
-
recordRun(run: SettledRun): Promise<void>;
|
|
26
|
-
getSnapshot(range: TimeRange): Promise<AnalyticsSnapshot>;
|
|
27
|
-
clearAll(): Promise<
|
|
21
|
+
recordRun(run: SettledRun, signal?: AbortSignal): Promise<void>;
|
|
22
|
+
getSnapshot(range: TimeRange, signal?: AbortSignal): Promise<AnalyticsSnapshot>;
|
|
23
|
+
clearAll(signal?: AbortSignal): Promise<ClearAnalyticsResult>;
|
|
28
24
|
close(): Promise<void>;
|
|
29
25
|
}
|
|
30
26
|
|
|
31
27
|
interface AnalyticsDependencies {
|
|
32
|
-
|
|
28
|
+
createStore(path: string): AnalyticsStorePort;
|
|
29
|
+
createSkillTracker(cwd: string): SkillTracker;
|
|
33
30
|
getAgentDir(): string;
|
|
34
31
|
now(): number;
|
|
35
32
|
createId(): string;
|
|
36
|
-
platform(): string;
|
|
37
33
|
}
|
|
38
34
|
|
|
39
35
|
export function createAnalyticsExtension(
|
|
40
36
|
dependencies: Partial<AnalyticsDependencies> = {},
|
|
41
37
|
): (pi: ExtensionAPI) => void {
|
|
42
38
|
const deps: AnalyticsDependencies = {
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
(async (databasePath) =>
|
|
46
|
-
new AnalyticsStore(await openAnalyticsDatabase({ path: databasePath }))),
|
|
39
|
+
createStore: dependencies.createStore ?? ((rootPath) => new AnalyticsStore(rootPath)),
|
|
40
|
+
createSkillTracker: dependencies.createSkillTracker ?? ((cwd) => new SkillTracker(cwd)),
|
|
47
41
|
getAgentDir: dependencies.getAgentDir ?? getAgentDir,
|
|
48
42
|
now: dependencies.now ?? Date.now,
|
|
49
43
|
createId: dependencies.createId ?? randomUUID,
|
|
50
|
-
platform: dependencies.platform ?? runtimePlatform,
|
|
51
44
|
};
|
|
52
45
|
|
|
53
46
|
return function analyticsExtension(pi: ExtensionAPI): void {
|
|
@@ -57,7 +50,7 @@ export function createAnalyticsExtension(
|
|
|
57
50
|
let skillTracker: SkillTracker | undefined;
|
|
58
51
|
let store: AnalyticsStorePort | undefined;
|
|
59
52
|
let storageFailure: string | undefined;
|
|
60
|
-
|
|
53
|
+
const retiredCloseTasks = new Set<Promise<boolean>>();
|
|
61
54
|
let writeFailureActive = false;
|
|
62
55
|
let pendingTriggerSource: TriggerSource = "unknown";
|
|
63
56
|
let pendingAttemptWithoutRun = false;
|
|
@@ -85,36 +78,27 @@ export function createAnalyticsExtension(
|
|
|
85
78
|
},
|
|
86
79
|
});
|
|
87
80
|
|
|
88
|
-
pi.on("session_start",
|
|
89
|
-
|
|
81
|
+
pi.on("session_start", (_event, ctx) => {
|
|
82
|
+
++sessionGeneration;
|
|
90
83
|
if (ctx.hasUI) ctx.ui.notify(EXPERIMENTAL_WARNING, "warning");
|
|
84
|
+
const previousStore = store;
|
|
91
85
|
sessionController.abort(new DOMException("Analytics session replaced", "AbortError"));
|
|
86
|
+
if (previousStore) retire(previousStore);
|
|
92
87
|
sessionController = new AbortController();
|
|
93
88
|
collector = new ResponseCollector();
|
|
94
|
-
skillTracker =
|
|
89
|
+
skillTracker = deps.createSkillTracker(ctx.cwd);
|
|
95
90
|
store = undefined;
|
|
96
91
|
storageFailure = undefined;
|
|
97
92
|
writeFailureActive = false;
|
|
98
93
|
pendingTriggerSource = "unknown";
|
|
99
94
|
pendingAttemptWithoutRun = false;
|
|
100
|
-
const
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
}
|
|
108
|
-
store = opened;
|
|
109
|
-
} catch (error) {
|
|
110
|
-
if (generation !== sessionGeneration) return;
|
|
111
|
-
storageFailure = storageFailureMessage(error, deps.platform());
|
|
112
|
-
safeNotify(ctx, storageFailure, "warning");
|
|
113
|
-
}
|
|
114
|
-
})();
|
|
115
|
-
startupTask = task;
|
|
116
|
-
await task;
|
|
117
|
-
if (startupTask === task) startupTask = undefined;
|
|
95
|
+
const storageRoot = path.join(deps.getAgentDir(), STORAGE_DIRECTORY);
|
|
96
|
+
try {
|
|
97
|
+
store = deps.createStore(storageRoot);
|
|
98
|
+
} catch {
|
|
99
|
+
storageFailure = unavailableMessage();
|
|
100
|
+
safeNotify(ctx, storageFailure, "warning");
|
|
101
|
+
}
|
|
118
102
|
});
|
|
119
103
|
|
|
120
104
|
pi.on("input", (event, ctx) => {
|
|
@@ -136,21 +120,37 @@ export function createAnalyticsExtension(
|
|
|
136
120
|
pi.on("before_agent_start", async (event, ctx) => {
|
|
137
121
|
const generation = sessionGeneration;
|
|
138
122
|
const tracker = skillTracker;
|
|
123
|
+
const activeCollector = collector;
|
|
139
124
|
if (!tracker) return;
|
|
140
125
|
const skills = availableSkills(pi, event.systemPromptOptions.skills ?? []);
|
|
141
126
|
await tracker.setAvailableSkills(skills);
|
|
142
|
-
if (
|
|
127
|
+
if (
|
|
128
|
+
generation !== sessionGeneration ||
|
|
129
|
+
tracker !== skillTracker ||
|
|
130
|
+
activeCollector !== collector
|
|
131
|
+
) {
|
|
132
|
+
return;
|
|
133
|
+
}
|
|
143
134
|
const explicit = tracker.consumeExplicitSkill();
|
|
144
|
-
const interrupted =
|
|
135
|
+
const interrupted = activeCollector.begin({
|
|
145
136
|
id: deps.createId(),
|
|
146
137
|
now: deps.now(),
|
|
147
138
|
triggerSource: explicit?.source ?? pendingTriggerSource,
|
|
148
139
|
model: modelIdentity(ctx, pi),
|
|
149
140
|
});
|
|
150
141
|
pendingTriggerSource = "unknown";
|
|
151
|
-
if (interrupted)
|
|
142
|
+
if (interrupted) {
|
|
143
|
+
await persistRun(interrupted, ctx, generation, sessionController.signal);
|
|
144
|
+
if (
|
|
145
|
+
generation !== sessionGeneration ||
|
|
146
|
+
tracker !== skillTracker ||
|
|
147
|
+
activeCollector !== collector
|
|
148
|
+
) {
|
|
149
|
+
return;
|
|
150
|
+
}
|
|
151
|
+
}
|
|
152
152
|
if (explicit && skills.some(({ name }) => name === explicit.name)) {
|
|
153
|
-
|
|
153
|
+
activeCollector.activateSkill({
|
|
154
154
|
name: explicit.name,
|
|
155
155
|
initiatedBy: "user",
|
|
156
156
|
now: explicit.observedAtMs,
|
|
@@ -164,9 +164,7 @@ export function createAnalyticsExtension(
|
|
|
164
164
|
else pendingAttemptWithoutRun = true;
|
|
165
165
|
});
|
|
166
166
|
|
|
167
|
-
pi.on("turn_start", (_event, ctx) =>
|
|
168
|
-
ensureRun(ctx, "extension");
|
|
169
|
-
});
|
|
167
|
+
pi.on("turn_start", (_event, ctx) => ensureRun(ctx, "extension"));
|
|
170
168
|
|
|
171
169
|
pi.on("before_provider_request", (_event, ctx) => {
|
|
172
170
|
ensureRun(ctx, "extension");
|
|
@@ -204,14 +202,24 @@ export function createAnalyticsExtension(
|
|
|
204
202
|
if (event.toolName === "read" && !isBuiltinReadTool(pi)) return;
|
|
205
203
|
const generation = sessionGeneration;
|
|
206
204
|
const tracker = skillTracker;
|
|
207
|
-
|
|
205
|
+
const activeCollector = collector;
|
|
206
|
+
const runId = activeCollector.getActiveRunId();
|
|
207
|
+
if (!tracker || !runId) return;
|
|
208
208
|
const name = await tracker.matchSuccessfulRead({
|
|
209
209
|
toolName: event.toolName,
|
|
210
210
|
input: event.input,
|
|
211
211
|
isError: event.isError,
|
|
212
212
|
});
|
|
213
|
-
if (
|
|
214
|
-
|
|
213
|
+
if (
|
|
214
|
+
!name ||
|
|
215
|
+
generation !== sessionGeneration ||
|
|
216
|
+
tracker !== skillTracker ||
|
|
217
|
+
activeCollector !== collector ||
|
|
218
|
+
activeCollector.getActiveRunId() !== runId
|
|
219
|
+
) {
|
|
220
|
+
return;
|
|
221
|
+
}
|
|
222
|
+
activeCollector.activateSkill({
|
|
215
223
|
name,
|
|
216
224
|
initiatedBy: "model",
|
|
217
225
|
now: deps.now(),
|
|
@@ -225,37 +233,45 @@ export function createAnalyticsExtension(
|
|
|
225
233
|
|
|
226
234
|
pi.on("agent_settled", async (_event, ctx) => {
|
|
227
235
|
const generation = sessionGeneration;
|
|
236
|
+
const owner = sessionController;
|
|
228
237
|
const run = collector.settle(deps.now());
|
|
229
238
|
pendingAttemptWithoutRun = false;
|
|
230
239
|
pendingTriggerSource = "unknown";
|
|
231
240
|
skillTracker?.clearPending();
|
|
232
|
-
if (run) await persistRun(run, ctx, generation);
|
|
241
|
+
if (run) await persistRun(run, ctx, generation, owner.signal);
|
|
233
242
|
});
|
|
234
243
|
|
|
235
244
|
pi.on("session_shutdown", async (_event, ctx) => {
|
|
236
245
|
const activeStore = store;
|
|
237
|
-
const activeStartup = startupTask;
|
|
238
246
|
++sessionGeneration;
|
|
239
247
|
sessionController.abort(new DOMException("Analytics session shut down", "AbortError"));
|
|
240
248
|
skillTracker?.clearPending();
|
|
241
249
|
skillTracker = undefined;
|
|
242
250
|
store = undefined;
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
251
|
+
collector.interrupt(deps.now());
|
|
252
|
+
const closing = activeStore
|
|
253
|
+
? [closeResult(activeStore), ...retiredCloseTasks]
|
|
254
|
+
: [...retiredCloseTasks];
|
|
255
|
+
const results = await Promise.all(closing);
|
|
256
|
+
if (results.some((closed) => !closed)) {
|
|
257
|
+
safeNotify(ctx, "Analytics storage shutdown was incomplete.", "warning");
|
|
248
258
|
}
|
|
249
|
-
await activeStartup?.catch(() => undefined);
|
|
250
|
-
await activeStore?.close().catch(() => {
|
|
251
|
-
safeNotify(
|
|
252
|
-
ctx,
|
|
253
|
-
"Analytics storage shutdown was incomplete; some pending metrics may not have been saved.",
|
|
254
|
-
"warning",
|
|
255
|
-
);
|
|
256
|
-
});
|
|
257
259
|
});
|
|
258
260
|
|
|
261
|
+
function retire(retiredStore: AnalyticsStorePort): void {
|
|
262
|
+
const task = closeResult(retiredStore).finally(() => retiredCloseTasks.delete(task));
|
|
263
|
+
retiredCloseTasks.add(task);
|
|
264
|
+
}
|
|
265
|
+
|
|
266
|
+
async function closeResult(activeStore: AnalyticsStorePort): Promise<boolean> {
|
|
267
|
+
try {
|
|
268
|
+
await activeStore.close();
|
|
269
|
+
return true;
|
|
270
|
+
} catch {
|
|
271
|
+
return false;
|
|
272
|
+
}
|
|
273
|
+
}
|
|
274
|
+
|
|
259
275
|
function ensureRun(ctx: ExtensionContext, triggerSource: TriggerSource): void {
|
|
260
276
|
if (collector.hasActiveRun()) return;
|
|
261
277
|
collector.begin({
|
|
@@ -274,22 +290,30 @@ export function createAnalyticsExtension(
|
|
|
274
290
|
run: SettledRun,
|
|
275
291
|
ctx: ExtensionContext,
|
|
276
292
|
generation: number,
|
|
293
|
+
signal: AbortSignal,
|
|
277
294
|
): Promise<void> {
|
|
278
295
|
const activeStore = store;
|
|
279
|
-
if (!activeStore) return;
|
|
296
|
+
if (!activeStore || signal.aborted) return;
|
|
280
297
|
try {
|
|
281
|
-
await activeStore.recordRun(run);
|
|
282
|
-
if (generation !== sessionGeneration || activeStore !== store) return;
|
|
298
|
+
await activeStore.recordRun(run, signal);
|
|
299
|
+
if (generation !== sessionGeneration || activeStore !== store || signal.aborted) return;
|
|
283
300
|
if (writeFailureActive) {
|
|
284
301
|
writeFailureActive = false;
|
|
285
302
|
safeNotify(ctx, "Local analytics storage recovered.", "info");
|
|
286
303
|
}
|
|
287
304
|
} catch {
|
|
288
|
-
if (
|
|
305
|
+
if (
|
|
306
|
+
generation !== sessionGeneration ||
|
|
307
|
+
activeStore !== store ||
|
|
308
|
+
signal.aborted ||
|
|
309
|
+
writeFailureActive
|
|
310
|
+
) {
|
|
311
|
+
return;
|
|
312
|
+
}
|
|
289
313
|
writeFailureActive = true;
|
|
290
314
|
safeNotify(
|
|
291
315
|
ctx,
|
|
292
|
-
"Analytics could not save
|
|
316
|
+
"Analytics could not save this response cycle; its metrics were dropped.",
|
|
293
317
|
"warning",
|
|
294
318
|
);
|
|
295
319
|
}
|
|
@@ -297,27 +321,22 @@ export function createAnalyticsExtension(
|
|
|
297
321
|
|
|
298
322
|
function menuSource(generation: number, signal: AbortSignal): AnalyticsMenuDataSource {
|
|
299
323
|
return {
|
|
300
|
-
path: store?.path ?? path.join(deps.getAgentDir(),
|
|
301
|
-
async load(range) {
|
|
324
|
+
path: store?.path ?? path.join(deps.getAgentDir(), STORAGE_DIRECTORY),
|
|
325
|
+
async load(range, actionSignal) {
|
|
302
326
|
assertCurrent(generation, signal);
|
|
303
327
|
const activeStore = store;
|
|
304
328
|
if (!activeStore) {
|
|
305
|
-
return {
|
|
306
|
-
kind: "unavailable",
|
|
307
|
-
message: storageFailure ?? unavailableMessage(deps.platform()),
|
|
308
|
-
};
|
|
329
|
+
return { kind: "unavailable", message: storageFailure ?? unavailableMessage() };
|
|
309
330
|
}
|
|
310
|
-
const snapshot = await activeStore.getSnapshot(range);
|
|
331
|
+
const snapshot = await activeStore.getSnapshot(range, actionSignal);
|
|
311
332
|
assertCurrent(generation, signal);
|
|
312
333
|
return { kind: "ready", snapshot };
|
|
313
334
|
},
|
|
314
|
-
async clearAll() {
|
|
335
|
+
async clearAll(actionSignal) {
|
|
315
336
|
assertCurrent(generation, signal);
|
|
316
337
|
const activeStore = store;
|
|
317
|
-
if (!activeStore) return
|
|
318
|
-
|
|
319
|
-
assertCurrent(generation, signal);
|
|
320
|
-
return count;
|
|
338
|
+
if (!activeStore) return { cleanupIncomplete: false };
|
|
339
|
+
return activeStore.clearAll(actionSignal);
|
|
321
340
|
},
|
|
322
341
|
};
|
|
323
342
|
}
|
|
@@ -361,36 +380,10 @@ function availableSkills(
|
|
|
361
380
|
return result;
|
|
362
381
|
}
|
|
363
382
|
|
|
364
|
-
function
|
|
365
|
-
if (process.platform !== "linux") return `${process.platform}-${process.arch}`;
|
|
366
|
-
const report = process.report?.getReport() as
|
|
367
|
-
| { header?: { glibcVersionRuntime?: unknown } }
|
|
368
|
-
| undefined;
|
|
369
|
-
const glibc = report?.header?.glibcVersionRuntime;
|
|
370
|
-
return `linux-${process.arch}-${glibc ? "glibc" : "non-glibc"}`;
|
|
371
|
-
}
|
|
372
|
-
|
|
373
|
-
function storageFailureMessage(error: unknown, platform: string): string {
|
|
374
|
-
if (error instanceof AnalyticsDatabaseOpenError) {
|
|
375
|
-
return `${error.message}\nExisting files were not replaced. Repair or restore them before retrying.\nNo analytics are being collected.`;
|
|
376
|
-
}
|
|
377
|
-
if (error instanceof NewerSchemaError) {
|
|
378
|
-
return `${error.message}\nUpdate pi-analytics before using this database.\nNo analytics are being collected.`;
|
|
379
|
-
}
|
|
380
|
-
if (error instanceof ChecksumMismatchError) {
|
|
381
|
-
return `${error.message}\nRestore a known database backup or update pi-analytics.\nNo analytics are being collected.`;
|
|
382
|
-
}
|
|
383
|
-
if (error instanceof MigrationFailedError) {
|
|
384
|
-
return `${error.message}\nThe previous valid schema was preserved.\nNo analytics are being collected.`;
|
|
385
|
-
}
|
|
386
|
-
return unavailableMessage(platform);
|
|
387
|
-
}
|
|
388
|
-
|
|
389
|
-
function unavailableMessage(platform: string): string {
|
|
383
|
+
function unavailableMessage(): string {
|
|
390
384
|
return [
|
|
391
|
-
"
|
|
392
|
-
|
|
393
|
-
"Supported: Linux glibc x64/arm64, macOS arm64, Windows x64.",
|
|
385
|
+
"Local analytics storage could not be initialized safely.",
|
|
386
|
+
"Existing files were not replaced.",
|
|
394
387
|
"No analytics are being collected.",
|
|
395
388
|
].join("\n");
|
|
396
389
|
}
|