@ponbac/token-ledger 0.1.2 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -24,7 +24,10 @@ These are equivalent alternatives; choose the package manager you already use.
24
24
  The npm package contains a bundled CLI with no runtime dependencies or install
25
25
  scripts. `bunx` uses the Node shebang, so Node 24+ is still required.
26
26
 
27
- The default window is the current calendar month through today, in UTC. Read every
27
+ The default window is the current calendar month through today. `--since` and
28
+ `--until` are inclusive local calendar days in the system time zone; select another
29
+ IANA zone with `--time-zone`, such as `--time-zone Europe/Stockholm`. Days follow
30
+ daylight saving time, so a day can last 23 or 25 hours. Read every
28
31
  supported source by omitting `--provider`. Configure Copilot collection before
29
32
  starting sessions as described below; installing the CLI does not enable telemetry.
30
33
 
@@ -90,15 +93,16 @@ node packages/cli/dist/main.js report
90
93
 
91
94
  ## Collection coverage
92
95
 
93
- | Provider | Default input | Coverage |
94
- | -------------------------- | ------------------------------------------------------ | ---------------------------------------------------------------------------- |
95
- | Codex | `~/.codex/sessions` and `archived_sessions` | Rollout token events, model switches, session and working-directory metadata |
96
- | Claude Code | `~/.claude/projects` | Assistant usage records; repeated message blocks counted once |
97
- | Grok Build | `~/.grok/sessions/**/updates.jsonl` | Saved completed turns and their model breakdowns |
98
- | Copilot CLI / VS Code Chat | `~/.copilot/otel` or `COPILOT_OTEL_FILE_EXPORTER_PATH` | JSONL request spans, VS Code inference events, and SQLite span exports |
96
+ | Provider | Default input | Coverage |
97
+ | -------------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------- |
98
+ | Codex | `~/.codex/sessions` and `archived_sessions` | Rollout token events, model switches, session and working-directory metadata |
99
+ | Claude Code | `~/.claude/projects` | Assistant usage records; repeated message blocks counted once |
100
+ | Grok Build | `~/.grok/sessions/**/updates.jsonl` | Saved completed turns and their model breakdowns |
101
+ | Copilot CLI / VS Code Chat | `~/.copilot/otel` or `COPILOT_OTEL_FILE_EXPORTER_PATH` | JSONL request spans, VS Code inference events, and SQLite span exports |
102
+ | Pi | `~/.pi/agent/sessions` or `PI_CODING_AGENT_SESSION_DIR` | Assistant messages on every branch, plus usage, compaction, and summary entries |
99
103
 
100
- `CODEX_HOME`, `CLAUDE_CONFIG_DIR`, `GROK_HOME`, and `COPILOT_HOME` override the usual
101
- homes. Configuration supports additional accounts, machines' exported histories,
104
+ `CODEX_HOME`, `CLAUDE_CONFIG_DIR`, `GROK_HOME`, `COPILOT_HOME`, and `PI_CODING_AGENT_DIR`
105
+ override the usual homes. Configuration supports additional accounts, machines' exported histories,
102
106
  and arbitrary source directories. Histories are opened read-only, including work
103
107
  performed outside T3 Code.
104
108
 
@@ -304,7 +308,8 @@ overlapping files: the first configured source owns their project attribution.
304
308
 
305
309
  The CLI fetches the public LiteLLM model-rate table and caches the decoded snapshot
306
310
  for 24 hours under `$XDG_CACHE_HOME/token-ledger` (default `~/.cache/token-ledger`).
307
- Only this public pricing request uses the network; histories are processed locally.
311
+ Histories are processed locally. `report` only uses the network for this public
312
+ pricing request; `sync` also uploads aggregated usage, as described below.
308
313
  `--offline` uses cached prices and config overrides. A failed refresh falls back to
309
314
  cached prices, with the snapshot timestamp retained in JSON.
310
315
 
@@ -323,8 +328,9 @@ applied to the whole selected history; historical price changes, service tiers,
323
328
  tool charges, taxes, negotiated discounts, and subscription allocations are not
324
329
  modeled. Provider-reported credit or dollar figures do not override this estimate.
325
330
 
326
- JSON is a versioned report with pricing provenance, applied unit rates, coverage diagnostics, and rows
327
- grouped by project/day/provider/model. CSV instead matches the terminal summary:
331
+ JSON is a versioned report (version 2; version 1 used UTC days) with pricing provenance, applied unit rates, coverage diagnostics, and rows
332
+ grouped by project/day/provider/model. Its `timeZone` field names the IANA zone of
333
+ `since`, `until`, and each row's `day`. CSV instead matches the terminal summary:
328
334
  one row per project, sorted by known API cost, plus a Total row. It includes token
329
335
  totals, API estimates rounded to two decimals, and cost percentages. Incomplete
330
336
  costs display `+ unknown`, and percentages use only the known subtotal. Neither
@@ -338,20 +344,264 @@ Exit codes: `0` for a generated report, `1` for invalid input or a fatal error,
338
344
  Ordinary mode still produces a partial report when some providers are unavailable.
339
345
  Keep developer identity alongside exported reports when combining them across a team.
340
346
 
347
+ ## Sync to toki2
348
+
349
+ `sync` uploads this machine's usage to a [toki2](https://github.com/ponbac/toki2)
350
+ server, which combines every developer's machines for team reporting and billing.
351
+ Run it on **every machine** where you use a coding agent, and schedule it daily.
352
+
353
+ ### Set up
354
+
355
+ 1. **Create an API token.** In toki2, open **Account** in the side navigation, then
356
+ **API tokens**, and create a token such as `token-ledger work-laptop`. The secret is
357
+ shown only once. A token per machine lets you revoke one machine alone.
358
+ 2. **Save the server and token:**
359
+
360
+ ```sh
361
+ npx @ponbac/token-ledger@latest login
362
+ ```
363
+
364
+ `login` asks for the server URL and the token, then writes them to `sync.json` in
365
+ your settings directory with mode 600. Windows ignores the mode; there the file is
366
+ as private as the directory, and `%APPDATA%` is yours alone:
367
+
368
+ | Platform | Settings directory |
369
+ | -------- | ------------------------------------------------------------- |
370
+ | Linux | `~/.config/token-ledger` (or `$XDG_CONFIG_HOME/token-ledger`) |
371
+ | macOS | `~/Library/Application Support/token-ledger` |
372
+ | Windows | `%APPDATA%\token-ledger` |
373
+
374
+ A set `XDG_CONFIG_HOME` takes precedence on every platform. Without a terminal,
375
+ pass `--server` and set `TOKI_API_TOKEN`; `login` saves those. `sync` also accepts
376
+ `TOKEN_LEDGER_SERVER` and `TOKI_API_TOKEN`, or `--server` and `--token`, without
377
+ saving them; flags win over environment variables, which win over `sync.json`.
378
+ The token is sent only over HTTPS, or plain HTTP to localhost, and never printed.
379
+
380
+ 3. **Check, then sync:**
381
+
382
+ ```sh
383
+ npx @ponbac/token-ledger@latest sync --dry-run # print the payload; upload nothing
384
+ npx @ponbac/token-ledger@latest sync
385
+ ```
386
+
387
+ The first run creates `machine.json` in the settings directory with a random
388
+ machine ID and a label taken from the host name. Edit the label freely, but keep
389
+ the ID: a new ID makes the same history count again as another machine.
390
+
391
+ If you map projects in a `token-ledger.json`, pass it with `--config`; scheduled jobs
392
+ do not run in your project directory. Days follow the system time zone; add
393
+ `--time-zone Europe/Stockholm` when a scheduler's environment may lack one.
394
+
395
+ ### Schedule a daily sync
396
+
397
+ Schedule a daily sync to upload history before providers prune it:
398
+
399
+ ```sh
400
+ npx @ponbac/token-ledger@latest schedule install
401
+ ```
402
+
403
+ This registers a daily job with the operating system's own scheduler; token-ledger
404
+ does not keep a process running. Run it again to change the job.
405
+
406
+ | Flag | Effect |
407
+ | ----------------------- | ------------------------------------------------------- |
408
+ | `--at 07:30` | Daily local time; the default is `12:30` |
409
+ | `--config <file>` | Project mappings for the job, saved as an absolute path |
410
+ | `--time-zone <zone>` | Pass a time zone when the scheduler may not have one |
411
+ | `--provider <provider>` | Sync one provider only |
412
+ | `--dry-run` | Print the files and commands, and change nothing |
413
+
414
+ `schedule status` shows the scheduler's own report, including the last run, and
415
+ `schedule uninstall` removes the job and keeps your settings. A run's output is in
416
+ `journalctl --user -u token-ledger-sync` on Linux and in
417
+ `~/Library/Logs/token-ledger-sync.log` on macOS.
418
+
419
+ | Platform | Scheduler | A missed run |
420
+ | -------- | ------------------- | ------------------------------------------------------ |
421
+ | Linux | systemd user timer | Runs within ten minutes of your next login |
422
+ | macOS | launchd agent | Runs on wake; also syncs each time you log in |
423
+ | Windows | Task Scheduler task | Runs at the next opportunity, on battery power as well |
424
+
425
+ The job needs the server and token saved by `login`: it cannot read `--token` or
426
+ variables set only in your shell, such as `TOKI_API_TOKEN` or `CLAUDE_CONFIG_DIR`.
427
+ The exception is `XDG_CONFIG_HOME`, which `schedule install` copies into the job so
428
+ that it reads the same settings directory.
429
+ It runs `npx --yes @ponbac/token-ledger@latest sync`, so it stays up to date. It
430
+ looks for `npx` beside the Node that ran `schedule install` and then on the
431
+ scheduler's `PATH`, so it survives removing that Node version when another is on the
432
+ `PATH`.
433
+
434
+ Linux needs a systemd user session, which Arch, Ubuntu, Debian, and Fedora provide.
435
+ Without one, as on Alpine or in WSL with systemd disabled, use the cron entry below.
436
+
437
+ <details>
438
+ <summary>Scheduling by hand</summary>
439
+
440
+ `schedule install --dry-run` prints the exact files and commands for your machine.
441
+ The examples below do the same with a login shell finding `npx`; if it is not on the
442
+ scheduler's `PATH`, use its absolute path or install the CLI globally with
443
+ `npm install -g @ponbac/token-ledger` and run `token-ledger sync`.
444
+
445
+ **Linux (systemd user timer).** Create `~/.config/systemd/user/token-ledger-sync.service`:
446
+
447
+ ```ini
448
+ [Unit]
449
+ Description=Upload AI usage to toki2
450
+
451
+ [Service]
452
+ Type=oneshot
453
+ ExecStart=/bin/sh -lc 'npx --yes @ponbac/token-ledger@latest sync'
454
+ ```
455
+
456
+ and `~/.config/systemd/user/token-ledger-sync.timer`:
457
+
458
+ ```ini
459
+ [Unit]
460
+ Description=Daily AI usage sync
461
+
462
+ [Timer]
463
+ OnCalendar=daily
464
+ Persistent=true
465
+ RandomizedDelaySec=1h
466
+
467
+ [Install]
468
+ WantedBy=timers.target
469
+ ```
470
+
471
+ Then run `systemctl --user enable --now token-ledger-sync.timer`. Check results with
472
+ `journalctl --user -u token-ledger-sync`. `Persistent=true` catches missed runs when
473
+ the user timer is next active.
474
+
475
+ **cron.** Create the log directory with `mkdir -p "$HOME/.cache"`, then run
476
+ `crontab -e` and add:
477
+
478
+ ```cron
479
+ 30 12 * * * /bin/sh -lc 'npx --yes @ponbac/token-ledger@latest sync' >> "$HOME/.cache/token-ledger-sync.log" 2>&1
480
+ ```
481
+
482
+ cron skips runs while the machine is asleep or off; prefer the systemd timer where
483
+ available.
484
+
485
+ **macOS (launchd).** Create `~/Library/LaunchAgents/com.github.ponbac.token-ledger.sync.plist`:
486
+
487
+ ```xml
488
+ <?xml version="1.0" encoding="UTF-8"?>
489
+ <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
490
+ <plist version="1.0">
491
+ <dict>
492
+ <key>Label</key>
493
+ <string>com.github.ponbac.token-ledger.sync</string>
494
+ <key>ProgramArguments</key>
495
+ <array>
496
+ <string>/bin/zsh</string>
497
+ <string>-lc</string>
498
+ <string>npx --yes @ponbac/token-ledger@latest sync</string>
499
+ </array>
500
+ <key>RunAtLoad</key>
501
+ <true/>
502
+ <key>StartCalendarInterval</key>
503
+ <dict>
504
+ <key>Hour</key>
505
+ <integer>12</integer>
506
+ <key>Minute</key>
507
+ <integer>30</integer>
508
+ </dict>
509
+ <key>StandardOutPath</key>
510
+ <string>/tmp/token-ledger-sync.log</string>
511
+ <key>StandardErrorPath</key>
512
+ <string>/tmp/token-ledger-sync.log</string>
513
+ </dict>
514
+ </plist>
515
+ ```
516
+
517
+ Then run `launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.github.ponbac.token-ledger.sync.plist`.
518
+ `RunAtLoad` also syncs when the agent loads at login. `StartCalendarInterval`
519
+ [catches missed intervals after sleep](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html),
520
+ but does not replay intervals missed while powered off.
521
+
522
+ **Windows (Task Scheduler).** In PowerShell:
523
+
524
+ ```powershell
525
+ $action = New-ScheduledTaskAction -Execute "cmd.exe" -Argument "/c npx --yes @ponbac/token-ledger@latest sync"
526
+ $trigger = New-ScheduledTaskTrigger -Daily -At 12:30
527
+ $settings = New-ScheduledTaskSettingsSet -StartWhenAvailable -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries
528
+ Register-ScheduledTask -TaskName "token-ledger sync" -Action $action -Trigger $trigger -Settings $settings
529
+ ```
530
+
531
+ `-StartWhenAvailable` runs a missed sync at the next opportunity. The battery
532
+ switches allow starts on battery and keep a running sync from being stopped when
533
+ AC power disconnects. See [Task Scheduler settings](https://learn.microsoft.com/en-us/powershell/module/scheduledtasks/new-scheduledtasksettingsset).
534
+
535
+ </details>
536
+
537
+ ### What leaves the machine
538
+
539
+ Provider histories stay on the machine. `sync` uploads only aggregates, and only when
540
+ you run it:
541
+
542
+ - **Uploaded:** hourly totals per session, project, provider, and model: input,
543
+ cache, and output tokens, request counts, and API-equivalent cost. Also the window,
544
+ your time zone name, the machine ID and label, the CLI version, per-provider
545
+ coverage counts, pricing provenance, and billing plans Codex reports.
546
+ - **Hashed:** session identities. Some providers use a transcript's file path as its
547
+ session, so no raw session ID is sent.
548
+ - **Never uploaded:** prompts, responses, individual requests, source code, file or
549
+ directory paths, and credentials. Usage attributed only to a local directory or a
550
+ local-path Git remote is sent as `unattributed`; configured project names and
551
+ network Git remotes (`host/owner/repo`) are sent as they are.
552
+
553
+ See [the payload contract](docs/sync-v1.md) for every field.
554
+
555
+ ### Windows and re-syncing
556
+
557
+ The default window is the last 14 local days. For each provider whose history was
558
+ fully read, the server replaces what this machine previously sent for that provider
559
+ in the window, so repeated syncs never double count, and machines add up. Fully read
560
+ means every file that could hold usage in the window: older transcripts are
561
+ [counted without being opened](#accuracy-and-limitations). A provider
562
+ with no history found, or with history that could not be read, keeps its stored data,
563
+ as do providers left out with `--provider`: missing data is never treated as zero.
564
+
565
+ Replacement also means an old window is risky: if a provider has since pruned its
566
+ history, syncing that window again replaces good server data with smaller totals.
567
+ Claude Code deletes transcripts after 30 days by default, so keep `--since` within
568
+ the last 30 days; `sync` warns when it is not. Once uploaded, the server keeps the
569
+ long-term record.
570
+
571
+ Exit codes: `0` synced, `1` invalid input or settings, `2` refused by `--strict`,
572
+ `3` server unreachable or temporarily failing (retry later), `4` token or machine
573
+ refused, `5` unsupported payload version (upgrade token-ledger), `6` payload rejected.
574
+
341
575
  ## Accuracy and limitations
342
576
 
343
577
  - Reports reflect saved, readable local history, not an authoritative provider invoice.
344
578
  - JSONL files are streamed, while Copilot buffers accounting records and span links
345
579
  per source for attribution; SQLite imports read the selected accounting rows. Each
346
580
  run rescans its sources, and memory grows with observed usage records.
581
+ - Transcripts are append-only, so a Codex, Claude Code, Grok, or Pi file last modified more
582
+ than 36 hours before the window starts is not opened; coverage counts these as
583
+ `filesBeforeWindow`. This trusts modification times: a history restored with times
584
+ earlier than its last events is missed, so name such a file as the source itself,
585
+ which is always read. Copilot exports are always read too, because requests are
586
+ correlated across files and a live SQLite database's modification time lags its rows.
347
587
  - Copied files, overlapping source paths, and stable provider record IDs are deduplicated.
348
588
  Missing stable IDs are called out where the provider cannot support that guarantee.
349
- - Codex fork histories use T3 Code's timing heuristic to exclude the leading copied
350
- burst. The report explicitly marks affected coverage as partial.
589
+ - Codex fork and subagent histories begin with a copy of their parent's history, which
590
+ the parent's own rollout already counts. T3 Code's timing heuristic excludes that
591
+ leading burst; it is not lost usage, so coverage stays `ok`. `partial` means real loss:
592
+ malformed lines, usage events that could not be normalized, or, in a report, paths
593
+ that could not be read.
594
+ - Pi `/fork` and `/clone` sessions copy their parent's entries unchanged. Each copied
595
+ request is counted once, for the session that made it; when that session's file is not
596
+ among the files read, it is counted for the fork. Pi names no model for nested tool
597
+ work or for compaction and branch summaries; those tokens are reported under the model
598
+ `unknown` and stay unpriced. Pi's own recorded costs are not used.
599
+ - A Pi `sessionDir` setting is not read; when sessions live outside the default
600
+ directory, set `PI_CODING_AGENT_SESSION_DIR` or configure a `pi` source.
351
601
  - A truncated or malformed JSONL line is skipped and counted. No prompts are printed
352
602
  in diagnostics. An active session can change during a scan; this is not a transactional snapshot.
353
- - Coverage diagnostics describe all scanned files, including events outside the selected
354
- date window. Codex updates with no usage payload are ignored, not counted as lost usage.
603
+ - Coverage diagnostics describe every opened file, including its events outside the
604
+ selected date window. Codex updates with no usage payload are ignored, not counted as lost usage.
355
605
  - Default project inference uses the current local Git metadata when the transcript
356
606
  lacks a remote. Explicit mappings make attribution more stable across machines.
357
607
  - A session that works across projects still needs an explicit allocation policy;
@@ -369,7 +619,7 @@ For manual publication:
369
619
  1. Log in with `npm login` using an account with publish access to `@ponbac/token-ledger`.
370
620
  2. Run `bun run check`. The CLI test packs the package, installs it offline outside
371
621
  the repository with lifecycle scripts disabled, and exercises the installed CLI.
372
- 3. Run `bun run pack` to create `dist/ponbac-token-ledger-0.1.2.tgz` (filename follows version).
622
+ 3. Run `bun run pack` to create `dist/ponbac-token-ledger-0.3.0.tgz` (filename follows version).
373
623
  4. Inspect `npm publish ./dist/npm --dry-run`.
374
624
  5. Publish with `npm publish ./dist/npm --access public`; complete npm's authentication
375
625
  prompt if requested.
@@ -395,7 +645,7 @@ bun run lint:fix
395
645
  `Ledger.report` returns data and typed failures, with no terminal rendering.
396
646
  - `packages/cli`: Effect CLI and process wiring. A later UI can run the same core
397
647
  through a local backend.
398
- - Effect and platform packages are pinned to `4.0.0-rc.117`.
648
+ - Effect and platform packages are pinned to stable `4.0.0`.
399
649
  - Oxlint `1.80.0` with type-aware checks, Oxfmt `0.66.0`, and all 18 generic plus
400
650
  five Effect anti-slop policies gate CI, including test code, without exemptions.
401
651
  TypeScript owns duplicate declaration checking for schema/type pairs.
@@ -412,14 +662,32 @@ application checks and editor indexing, and marked vendored for GitHub statistic
412
662
 
413
663
  ```sh
414
664
  # Start with a clean working tree. Change the version deliberately when upgrading.
415
- bun run reference:update effect 'effect@4.0.0-rc.117'
665
+ bun run reference:update effect 'effect@4.0.0'
416
666
  bun run reference:update t3code main
667
+ # Refresh only the Effect-source overlay within the T3 Code reference:
668
+ bun run reference:update effect-t3code 'effect@4.0.0'
417
669
  ```
418
670
 
419
- Update Effect dependencies together with their reference. Git subtree commit
671
+ Refresh the Effect reference after upgrading dependencies, in a separate PR stacked
672
+ above the dependency upgrade. The Effect reference is pinned to `effect@4.0.0`
673
+ (`67ba4e46a11ccda0b6761578bfd22c04ae00167d`) from
674
+ [`Effect-TS/effect`](https://github.com/Effect-TS/effect). Git subtree commit
420
675
  messages record the imported revisions. The initial T3 reference is upstream
421
676
  `aff9318bf46beaf05cc7155b428d3f0b8711efd2`.
422
677
 
678
+ The nested `.reference/t3code/.repos/effect` snapshot has a deliberate overlay
679
+ of the same stable Effect source. Related T3 Code docs and updater paths use this
680
+ canonical name; unrelated T3 Code reference files keep their imported revision.
681
+ Its nested subtree history was omitted by the outer squash, so the updater
682
+ replaces that snapshot and records its upstream URL, release tag, and commit in Git
683
+ commit trailers. Refreshing either outer reference reapplies this Effect overlay;
684
+ refreshing T3 Code uses the root manifest's installed Effect version.
685
+ The updater also removes a legacy nested snapshot reintroduced by upstream and
686
+ reapplies the canonical paths in T3 Code's reference docs and updater files.
687
+ Its own reference-sync command pins Effect to `effect@4.0.0`, independently of
688
+ the vendored T3 runtime catalog; `--latest` still explicitly selects `main`.
689
+ The parent updater reapplies this small source patch after outer T3 refreshes.
690
+
423
691
  ## License
424
692
 
425
693
  MIT. See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) and the licenses retained
@@ -30,7 +30,7 @@ THE SOFTWARE.
30
30
 
31
31
  ---
32
32
 
33
- @effect/platform-node-shared@4.0.0-rc.117
33
+ @effect/platform-node-shared@4.0.0
34
34
 
35
35
  MIT License
36
36
 
@@ -57,7 +57,7 @@ SOFTWARE.
57
57
 
58
58
  ---
59
59
 
60
- @effect/platform-node@4.0.0-rc.117
60
+ @effect/platform-node@4.0.0
61
61
 
62
62
  MIT License
63
63
 
@@ -156,7 +156,7 @@ SOFTWARE.
156
156
 
157
157
  ---
158
158
 
159
- effect@4.0.0-rc.117
159
+ effect@4.0.0
160
160
 
161
161
  MIT License
162
162