@ponbac/token-ledger 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -28,8 +28,9 @@ The default window is the current calendar month through today. `--since` and
28
28
  `--until` are inclusive local calendar days in the system time zone; select another
29
29
  IANA zone with `--time-zone`, such as `--time-zone Europe/Stockholm`. Days follow
30
30
  daylight saving time, so a day can last 23 or 25 hours. Read every
31
- supported source by omitting `--provider`. Configure Copilot collection before
32
- starting sessions as described below; installing the CLI does not enable telemetry.
31
+ supported source by omitting `--provider`. Copilot records nothing until its export
32
+ is switched on: run [`copilot enable`](#enable-copilots-usage-export) before starting
33
+ sessions; installing the CLI does not enable telemetry.
33
34
 
34
35
  ```sh
35
36
  npx @ponbac/token-ledger@latest report --since 2026-09-01 --until 2026-09-30 --format csv > usage.csv
@@ -93,18 +94,92 @@ node packages/cli/dist/main.js report
93
94
 
94
95
  ## Collection coverage
95
96
 
96
- | Provider | Default input | Coverage |
97
- | -------------------------- | ------------------------------------------------------ | ---------------------------------------------------------------------------- |
98
- | Codex | `~/.codex/sessions` and `archived_sessions` | Rollout token events, model switches, session and working-directory metadata |
99
- | Claude Code | `~/.claude/projects` | Assistant usage records; repeated message blocks counted once |
100
- | Grok Build | `~/.grok/sessions/**/updates.jsonl` | Saved completed turns and their model breakdowns |
101
- | Copilot CLI / VS Code Chat | `~/.copilot/otel` or `COPILOT_OTEL_FILE_EXPORTER_PATH` | JSONL request spans, VS Code inference events, and SQLite span exports |
97
+ | Provider | Default input | Coverage |
98
+ | -------------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------- |
99
+ | Codex | `~/.codex/sessions` and `archived_sessions` | Rollout token events, model switches, session and working-directory metadata |
100
+ | Claude Code | `~/.claude/projects` | Assistant usage records; repeated message blocks counted once |
101
+ | Grok Build | `~/.grok/sessions/**/updates.jsonl` | Saved completed turns and their model breakdowns |
102
+ | Copilot CLI / VS Code Chat | `~/.copilot/otel` or `COPILOT_OTEL_FILE_EXPORTER_PATH` | JSONL request spans, VS Code inference events, and SQLite span exports |
103
+ | Pi | `~/.pi/agent/sessions` or `PI_CODING_AGENT_SESSION_DIR` | Assistant messages on every branch, plus usage, compaction, and summary entries |
102
104
 
103
- `CODEX_HOME`, `CLAUDE_CONFIG_DIR`, `GROK_HOME`, and `COPILOT_HOME` override the usual
104
- homes. Configuration supports additional accounts, machines' exported histories,
105
+ `CODEX_HOME`, `CLAUDE_CONFIG_DIR`, `GROK_HOME`, `COPILOT_HOME`, and `PI_CODING_AGENT_DIR`
106
+ override the usual homes. Configuration supports additional accounts, machines' exported histories,
105
107
  and arbitrary source directories. Histories are opened read-only, including work
106
108
  performed outside T3 Code.
107
109
 
110
+ ### Enable Copilot's usage export
111
+
112
+ Copilot writes no usage log until a local OpenTelemetry file export is switched on,
113
+ and nothing is backfilled: sessions from before that are lost. One command switches
114
+ it on for Copilot CLI and VS Code Copilot Chat:
115
+
116
+ ```sh
117
+ npx @ponbac/token-ledger@latest copilot enable
118
+ ```
119
+
120
+ It lists the changes it is about to make and asks before making them:
121
+
122
+ - It creates `~/.copilot/otel` (`$COPILOT_HOME/otel` when `COPILOT_HOME` is set), the
123
+ folder reports and syncs read by default.
124
+ - **Copilot CLI** (`--cli`): it makes `COPILOT_OTEL_FILE_EXPORTER_PATH` name
125
+ `usage.jsonl` in that folder for every new terminal. On Windows that is a user
126
+ environment variable. On macOS and Linux it is two lines at the end of your login
127
+ shell's startup file: `~/.zshrc` for zsh (`$ZDOTDIR/.zshrc` when set); `~/.bashrc`
128
+ for bash on Linux, and on macOS the profile a login bash reads (`~/.bash_profile`,
129
+ `~/.bash_login`, or `~/.profile`); for fish, its own
130
+ `~/.config/fish/conf.d/token-ledger-copilot.fish`.
131
+ - **VS Code Copilot Chat** (`--vscode`): it sets the four `github.copilot.chat.otel.*`
132
+ settings shown [below](#vs-code-copilot-chat) in the user `settings.json`, with this
133
+ machine's path to `vscode-chat.jsonl` in the same folder. It also adds
134
+ `github.copilot.chat.otel.outfile` to the `settingsSync.ignoredSettings` list, because
135
+ Settings Sync would otherwise copy this machine's path to your other machines, where
136
+ it names nothing. Comments, trailing commas, and every other setting stay exactly as
137
+ they were.
138
+
139
+ | Platform | VS Code user settings |
140
+ | -------- | ----------------------------------------------------------------------------------- |
141
+ | Linux | `~/.config/Code/User/settings.json` (or `$XDG_CONFIG_HOME/Code/User/settings.json`) |
142
+ | macOS | `~/Library/Application Support/Code/User/settings.json` |
143
+ | Windows | `%APPDATA%\Code\User\settings.json` |
144
+
145
+ | Flag | Effect |
146
+ | -------------- | -------------------------------------------------------------------- |
147
+ | `--cli` | Only Copilot CLI; with neither `--cli` nor `--vscode`, both are done |
148
+ | `--vscode` | Only VS Code Copilot Chat |
149
+ | `--dry-run` | Print the changes, and change nothing |
150
+ | `--yes` | Make the changes without asking, as a script must |
151
+ | `--allow-root` | Run as root, where root is the account that uses Copilot |
152
+
153
+ Everything it touches is in your own profile, so it needs no admin rights. Run it as
154
+ yourself: under `sudo` the folder and files would belong to root, and Copilot and
155
+ VS Code could no longer write them, so it refuses to run as root.
156
+
157
+ Running it again is safe: what is already in place is left alone. Where it cannot
158
+ do a part, it finishes the others and prints the exact line or settings to add by
159
+ hand. That is the case for a shell other than bash, zsh, or fish, a startup file that
160
+ already sets the variable to something else or without `export`, a file it cannot
161
+ write, a `settings.json` with a syntax error, and a machine with no VS Code settings
162
+ folder. A file is replaced in one step, so a write that fails leaves it as it was, and
163
+ a symbolic link keeps pointing at the file it names.
164
+
165
+ Exit codes: `0` when everything asked for is in place, `2` when a part is left for
166
+ you, and `1` when nothing was changed: it was declined, had no terminal to ask on
167
+ without `--yes`, was run as root, or found that the export folder is a file or cannot
168
+ be written by your account, as after an earlier run under `sudo`.
169
+
170
+ The VS Code settings are application-wide, so the default profile's `settings.json`
171
+ covers every profile. VS Code Insiders is set up too when it is installed, with its
172
+ own `vscode-insiders-chat.jsonl`. Remote hosts (SSH, WSL, dev containers) are not:
173
+ run the command on the host where Copilot runs, or follow the steps below there.
174
+ VS Code started from an environment that has `COPILOT_OTEL_FILE_EXPORTER_PATH` writes
175
+ its Chat export to that file instead of `outfile`; both are in the folder that is
176
+ read. Afterwards, open a new terminal for Copilot CLI, and reload or restart VS Code
177
+ and start a new chat.
178
+
179
+ The rest of this section is the same setup by hand.
180
+
181
+ ### Copilot CLI
182
+
108
183
  Copilot CLI requires enabling its own file export **before** a session starts:
109
184
 
110
185
  ```sh
@@ -122,8 +197,10 @@ copilot
122
197
  ```
123
198
 
124
199
  Setting the file path automatically enables telemetry export to that local file;
125
- no separate toggle or telemetry server is needed. Prompt and response content
126
- capture is off by default.
200
+ no separate toggle or telemetry server is needed. Copilot CLI leaves prompts and
201
+ responses out of the export unless you turn its content capture on. Either way the
202
+ export is a local file: Token Ledger takes only accounting attributes from it and
203
+ never uploads content.
127
204
 
128
205
  The commands above set the variable for the current shell. To persist it for future
129
206
  Windows terminals, also run:
@@ -145,7 +222,11 @@ cache-token attributes are accepted. See the [Copilot CLI reference](https://doc
145
222
 
146
223
  ### VS Code Copilot Chat
147
224
 
148
- In your active VS Code profile's user settings, enable local file export:
225
+ In VS Code's user settings (**Preferences: Open User Settings (JSON)**), enable local
226
+ file export. The settings are application-wide: they are kept in the default profile's
227
+ `settings.json` and apply to every profile. In a profile other than the default, that
228
+ command opens the profile's own file, so run **Preferences: Open Application Settings
229
+ (JSON)** there instead:
149
230
 
150
231
  ```json
151
232
  {
@@ -161,6 +242,21 @@ directory, reload VS Code, and start a new chat. On Windows, use a path such as
161
242
  `C:/Users/you/.copilot/otel/vscode-chat.jsonl`. See the
162
243
  [VS Code monitoring documentation](https://code.visualstudio.com/docs/agents/guides/monitoring-agents).
163
244
 
245
+ If you use Settings Sync on machines whose home directories differ, also add
246
+ `"github.copilot.chat.otel.outfile"` to the `settingsSync.ignoredSettings` list: the
247
+ path is synced like any other user setting, and the last machine to set it would
248
+ otherwise send the others' export to a path that does not exist there. Where your
249
+ organization manages Copilot's telemetry settings by policy, the policy overrides these
250
+ user settings, and the export follows it.
251
+
252
+ `captureContent: false` asks VS Code to leave prompts and responses out of the export,
253
+ but do not count on the local file being free of them: an
254
+ [open VS Code issue](https://github.com/microsoft/vscode/issues/326254) reports spans
255
+ that carry prompts, system instructions, responses, and tool arguments whatever the
256
+ setting says. What the file holds stays on this machine. Token Ledger takes only
257
+ accounting attributes from it (token counts, models, timestamps, and repository
258
+ linkage) and never uploads content; see [what leaves the machine](#what-leaves-the-machine).
259
+
164
260
  Read that export, or combine CLI and Chat files from the same directory:
165
261
 
166
262
  ```sh
@@ -168,8 +264,14 @@ bun run dev report --provider copilot --source ~/.copilot/otel/vscode-chat.jsonl
168
264
  bun run dev report --provider copilot --source ~/.copilot/otel
169
265
  ```
170
266
 
171
- Both use the `copilot` provider. Default discovery reads `~/.copilot/otel`, unless
172
- `COPILOT_OTEL_FILE_EXPORTER_PATH` selects a specific file; `--source` overrides it.
267
+ Both use the `copilot` provider. Default discovery reads every `.jsonl` and `.db`
268
+ export in `~/.copilot/otel`. `COPILOT_OTEL_FILE_EXPORTER_PATH` naming a file in that
269
+ directory does not narrow this, so a VS Code export beside it is still read; a file
270
+ there with another extension is read as well, as a source of its own, so name exports
271
+ `.jsonl` to have them correlated with the rest. Naming a file outside the directory
272
+ selects that one file instead, and the directory is then not read: keep the exports
273
+ together, or list both under `sources` in a configuration. `--source` overrides
274
+ discovery.
173
275
  VS Code inference events require a response ID, timestamp, and input/output token
174
276
  counts. Missing or invalid fields are reported as skipped usage. Response IDs
175
277
  deduplicate copied events and matching request spans. Agent turn summaries,
@@ -202,8 +304,12 @@ and targets 1.140 in the [tracking issue](https://github.com/microsoft/vscode/is
202
304
  Token Ledger supports the corrected format, but cannot reconstruct spans already
203
305
  lost from an export.
204
306
 
205
- On affected builds, enable the **built-in SQLite span exporter** alongside the file
206
- settings above:
307
+ On affected builds, you can enable the **built-in SQLite span exporter** alongside the
308
+ file settings above. Decide first whether you want what it stores: the span database,
309
+ `agent-traces.db`, holds prompts and responses whatever `captureContent` says. In
310
+ VS Code 1.141.0 that setting filters the file export but is not applied to the
311
+ database. It is a local file, and Token Ledger selects only accounting attributes
312
+ from it and uploads no content, but the content is on disk.
207
313
 
208
314
  ```json
209
315
  {
@@ -422,7 +528,9 @@ does not keep a process running. Run it again to change the job.
422
528
  | Windows | Task Scheduler task | Runs at the next opportunity, on battery power as well |
423
529
 
424
530
  The job needs the server and token saved by `login`: it cannot read `--token` or
425
- variables set only in your shell, such as `TOKI_API_TOKEN` or `CLAUDE_CONFIG_DIR`.
531
+ variables set only in your shell, such as `TOKI_API_TOKEN`, `CLAUDE_CONFIG_DIR`, or a
532
+ `COPILOT_OTEL_FILE_EXPORTER_PATH` that names a file outside `~/.copilot/otel`; the job
533
+ then reads that directory while a sync from your terminal reads the file.
426
534
  The exception is `XDG_CONFIG_HOME`, which `schedule install` copies into the job so
427
535
  that it reads the same settings directory.
428
536
  It runs `npx --yes @ponbac/token-ledger@latest sync`, so it stays up to date. It
@@ -577,7 +685,7 @@ refused, `5` unsupported payload version (upgrade token-ledger), `6` payload rej
577
685
  - JSONL files are streamed, while Copilot buffers accounting records and span links
578
686
  per source for attribution; SQLite imports read the selected accounting rows. Each
579
687
  run rescans its sources, and memory grows with observed usage records.
580
- - Transcripts are append-only, so a Codex, Claude Code, or Grok file last modified more
688
+ - Transcripts are append-only, so a Codex, Claude Code, Grok, or Pi file last modified more
581
689
  than 36 hours before the window starts is not opened; coverage counts these as
582
690
  `filesBeforeWindow`. This trusts modification times: a history restored with times
583
691
  earlier than its last events is missed, so name such a file as the source itself,
@@ -590,6 +698,13 @@ refused, `5` unsupported payload version (upgrade token-ledger), `6` payload rej
590
698
  leading burst; it is not lost usage, so coverage stays `ok`. `partial` means real loss:
591
699
  malformed lines, usage events that could not be normalized, or, in a report, paths
592
700
  that could not be read.
701
+ - Pi `/fork` and `/clone` sessions copy their parent's entries unchanged. Each copied
702
+ request is counted once, for the session that made it; when that session's file is not
703
+ among the files read, it is counted for the fork. Pi names no model for nested tool
704
+ work or for compaction and branch summaries; those tokens are reported under the model
705
+ `unknown` and stay unpriced. Pi's own recorded costs are not used.
706
+ - A Pi `sessionDir` setting is not read; when sessions live outside the default
707
+ directory, set `PI_CODING_AGENT_SESSION_DIR` or configure a `pi` source.
593
708
  - A truncated or malformed JSONL line is skipped and counted. No prompts are printed
594
709
  in diagnostics. An active session can change during a scan; this is not a transactional snapshot.
595
710
  - Coverage diagnostics describe every opened file, including its events outside the
@@ -611,7 +726,7 @@ For manual publication:
611
726
  1. Log in with `npm login` using an account with publish access to `@ponbac/token-ledger`.
612
727
  2. Run `bun run check`. The CLI test packs the package, installs it offline outside
613
728
  the repository with lifecycle scripts disabled, and exercises the installed CLI.
614
- 3. Run `bun run pack` to create `dist/ponbac-token-ledger-0.2.0.tgz` (filename follows version).
729
+ 3. Run `bun run pack` to create `dist/ponbac-token-ledger-0.4.0.tgz` (filename follows version).
615
730
  4. Inspect `npm publish ./dist/npm --dry-run`.
616
731
  5. Publish with `npm publish ./dist/npm --access public`; complete npm's authentication
617
732
  prompt if requested.
@@ -237,6 +237,33 @@ The above copyright notice and this permission notice shall be included in all c
237
237
  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
238
238
 
239
239
 
240
+ ---
241
+
242
+ jsonc-parser@3.3.1
243
+
244
+ The MIT License (MIT)
245
+
246
+ Copyright (c) Microsoft
247
+
248
+ Permission is hereby granted, free of charge, to any person obtaining a copy
249
+ of this software and associated documentation files (the "Software"), to deal
250
+ in the Software without restriction, including without limitation the rights
251
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
252
+ copies of the Software, and to permit persons to whom the Software is
253
+ furnished to do so, subject to the following conditions:
254
+
255
+ The above copyright notice and this permission notice shall be included in all
256
+ copies or substantial portions of the Software.
257
+
258
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
259
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
260
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
261
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
262
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
263
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
264
+ SOFTWARE.
265
+
266
+
240
267
  ---
241
268
 
242
269
  string-width@4.2.3