tokentrax 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +140 -16
- package/bin/tracker.js +49 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -13,9 +13,21 @@
|
|
|
13
13
|
### Via npm (recommended)
|
|
14
14
|
|
|
15
15
|
```bash
|
|
16
|
-
npm install -g tokentrax
|
|
16
|
+
npm install -g tokentrax --allow-scripts=better-sqlite3
|
|
17
17
|
```
|
|
18
18
|
|
|
19
|
+
> **Why the flag?** npm 12 and newer block dependency install scripts by
|
|
20
|
+
> default, and `better-sqlite3` is a native module — its install script is
|
|
21
|
+
> what fetches or builds the binding. Without the flag you get an install
|
|
22
|
+
> that *looks* fine and then fails the first time it opens a database.
|
|
23
|
+
> Older npm versions ignore the flag, so the command works everywhere.
|
|
24
|
+
>
|
|
25
|
+
> If you'd rather set it once:
|
|
26
|
+
>
|
|
27
|
+
> ```bash
|
|
28
|
+
> npm config set allow-scripts=better-sqlite3 --location=user
|
|
29
|
+
> ```
|
|
30
|
+
|
|
19
31
|
### Via git clone
|
|
20
32
|
|
|
21
33
|
```bash
|
|
@@ -38,21 +50,6 @@ This installs TokenTrax globally and automatically:
|
|
|
38
50
|
- Runs database migrations (schema updates are applied automatically)
|
|
39
51
|
- Backfills pricing for any existing data without costs
|
|
40
52
|
|
|
41
|
-
### Remote agent
|
|
42
|
-
|
|
43
|
-
To report usage from a *second* machine — a laptop, a work box, anywhere
|
|
44
|
-
with an AI harness — install the agent there:
|
|
45
|
-
|
|
46
|
-
```bash
|
|
47
|
-
npm install -g tokentrax-agent
|
|
48
|
-
tokentrax-agent setup --server http://your-dashboard:7680 --key <sync-key>
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
That writes the configuration, installs a systemd user timer and starts it.
|
|
52
|
-
The machine then reports its sessions, tokens and cost to the dashboard every
|
|
53
|
-
five minutes. See [`agent/README.md`](agent/README.md) for the details, and
|
|
54
|
-
[`docs/agent-configuration.md`](docs/agent-configuration.md) for every option.
|
|
55
|
-
|
|
56
53
|
### Requirements
|
|
57
54
|
|
|
58
55
|
- Node.js >= 20
|
|
@@ -74,6 +71,37 @@ npm update -g tokentrax
|
|
|
74
71
|
|
|
75
72
|
Updates are safe — the postinstall script automatically migrates your existing database schema and backfills any missing pricing data.
|
|
76
73
|
|
|
74
|
+
### Remote agent
|
|
75
|
+
|
|
76
|
+
To report usage from a *second* machine — a laptop, a work box, anywhere
|
|
77
|
+
with an AI harness — install the agent there. It ships as its own npm
|
|
78
|
+
package:
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
npm install -g tokentrax-agent --allow-scripts=better-sqlite3
|
|
82
|
+
tokentrax-agent setup --server http://your-dashboard:7680 --key <sync-key>
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The `--allow-scripts` flag is for npm 12+, which blocks dependency install
|
|
86
|
+
scripts by default — `better-sqlite3` needs its install script to build a
|
|
87
|
+
native module. Older npm ignores the flag. See the note in the
|
|
88
|
+
[Installation](#installation) section above.
|
|
89
|
+
|
|
90
|
+
`setup` writes the configuration, installs a systemd user timer, and starts it.
|
|
91
|
+
The machine then reports its sessions, tokens and cost to the dashboard every
|
|
92
|
+
five minutes.
|
|
93
|
+
|
|
94
|
+
No npm? Bootstrap straight from the repository:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
curl -fsSL https://raw.githubusercontent.com/jeshuwah/TokenTrax/main/agent/install.sh \
|
|
98
|
+
| bash -s -- --server http://your-dashboard:7680 --key <sync-key>
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Full details in [`agent/README.md`](agent/README.md) and
|
|
102
|
+
[`docs/agent-configuration.md`](docs/agent-configuration.md). See
|
|
103
|
+
[Remote agents](#remote-agents) for how it fits together.
|
|
104
|
+
|
|
77
105
|
---
|
|
78
106
|
|
|
79
107
|
## Features
|
|
@@ -85,6 +113,7 @@ Updates are safe — the postinstall script automatically migrates your existing
|
|
|
85
113
|
- **Bundled + live pricing** — 2200+ model prices cached locally, auto-refreshed
|
|
86
114
|
- **Normalization engine** — maps 100+ model aliases to canonical names
|
|
87
115
|
- **Privacy-first** — all data stays in `~/.tokentrax/`, no telemetry
|
|
116
|
+
- **Optional remote agents** — one dashboard, many machines, via `tokentrax-agent`
|
|
88
117
|
|
|
89
118
|
---
|
|
90
119
|
|
|
@@ -118,6 +147,12 @@ Theme toggle (dark/light), pricing freshness indicator, normalization rule edito
|
|
|
118
147
|
|
|
119
148
|
<img src="docs/screenshots/settings.png" alt="Settings" width="100%">
|
|
120
149
|
|
|
150
|
+
### Agents
|
|
151
|
+
|
|
152
|
+
Every machine reporting into this dashboard — sessions, tokens, cost and
|
|
153
|
+
context-reduction percentage — with a per-machine breakdown by model and
|
|
154
|
+
per-session detail. See [Remote agents](#remote-agents).
|
|
155
|
+
|
|
121
156
|
---
|
|
122
157
|
|
|
123
158
|
## Supported AI Tools
|
|
@@ -140,6 +175,78 @@ Theme toggle (dark/light), pricing freshness indicator, normalization rule edito
|
|
|
140
175
|
|
|
141
176
|
---
|
|
142
177
|
|
|
178
|
+
## Remote agents
|
|
179
|
+
|
|
180
|
+
TokenTrax is local-first: one machine, one SQLite file, no server. The
|
|
181
|
+
optional `tokentrax-agent` package adds the other half — reporting from
|
|
182
|
+
*other* machines into the same dashboard.
|
|
183
|
+
|
|
184
|
+
```
|
|
185
|
+
laptop (omarchy) server (olympus)
|
|
186
|
+
┌──────────────────────┐ ┌────────────────────────┐
|
|
187
|
+
│ pi / Claude / Codex │ POST │ tokentrax serve :7680 │
|
|
188
|
+
│ tokentrax-agent │────────▶│ agent_sessions table │
|
|
189
|
+
│ systemd timer 5min │ │ "Agents" dashboard tab │
|
|
190
|
+
└──────────────────────┘ └────────────────────────┘
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Each remote machine runs the agent on a timer. It reads that machine's
|
|
194
|
+
harness session data, condenses it, and POSTs it to the dashboard. The
|
|
195
|
+
dashboard shows every machine in one list with sessions, tokens, cost and
|
|
196
|
+
context-reduction percentage.
|
|
197
|
+
|
|
198
|
+
### Install on the remote machine
|
|
199
|
+
|
|
200
|
+
```bash
|
|
201
|
+
npm install -g tokentrax-agent --allow-scripts=better-sqlite3
|
|
202
|
+
tokentrax-agent setup --server http://your-dashboard:7680 --key <sync-key>
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
`setup` writes the configuration, verifies the native SQLite binding loads,
|
|
206
|
+
installs a systemd user service and timer, and enables it. It is safe to
|
|
207
|
+
re-run.
|
|
208
|
+
|
|
209
|
+
### Prepare the dashboard
|
|
210
|
+
|
|
211
|
+
The server needs a shared key before it will accept anything:
|
|
212
|
+
|
|
213
|
+
```bash
|
|
214
|
+
export TOKENTRAX_SYNC_KEY=$(openssl rand -base64 48)
|
|
215
|
+
npm start serve
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
The same value goes on every remote machine. With a key set, the server
|
|
219
|
+
refuses writes from anything that is not loopback, so it is safe to expose
|
|
220
|
+
on a LAN or over a VPN.
|
|
221
|
+
|
|
222
|
+
### What crosses the wire
|
|
223
|
+
|
|
224
|
+
Session metadata and usage numbers only — never prompts, file contents or
|
|
225
|
+
tool arguments. Project paths are hashed by default
|
|
226
|
+
(`TOKENTRAX_TRACK_PROJECT=hash`), so the dashboard never learns your
|
|
227
|
+
directory names. Set it to `full` only if you want the real paths.
|
|
228
|
+
|
|
229
|
+
### A note on hooks
|
|
230
|
+
|
|
231
|
+
The agent installs post-session hooks for harnesses that support them. The
|
|
232
|
+
harness with the most data — pi — has no hook mechanism at all, so a hook
|
|
233
|
+
would silently never fire there. The systemd timer is what actually keeps
|
|
234
|
+
the data flowing, and it works for every harness.
|
|
235
|
+
|
|
236
|
+
### Agent commands
|
|
237
|
+
|
|
238
|
+
| Command | Description |
|
|
239
|
+
| --------- | ------------- |
|
|
240
|
+
| `tokentrax-agent setup --server <url> --key <key>` | Configure the machine and start the timer |
|
|
241
|
+
| `tokentrax-agent sync [--auto]` | Send new sessions to the server |
|
|
242
|
+
| `tokentrax-agent status` | Show configuration, machine id and cursor |
|
|
243
|
+
| `tokentrax-agent init` | Install post-session hooks where supported |
|
|
244
|
+
| `tokentrax-agent uninstall` | Remove post-session hooks |
|
|
245
|
+
|
|
246
|
+
See [`agent/README.md`](agent/README.md) for the full package documentation.
|
|
247
|
+
|
|
248
|
+
---
|
|
249
|
+
|
|
143
250
|
## Commands
|
|
144
251
|
|
|
145
252
|
### Core Commands
|
|
@@ -203,6 +310,19 @@ All data stored in `~/.tokentrax/`:
|
|
|
203
310
|
- **Auditable** — open source, read the parser code in `src/lib/parsers/`
|
|
204
311
|
- **No telemetry** — no analytics, no crash reporting, no phone-home
|
|
205
312
|
|
|
313
|
+
### …unless you install the remote agent
|
|
314
|
+
|
|
315
|
+
`tokentrax-agent` is the one exception, and it is entirely opt-in: you
|
|
316
|
+
install it, on a machine you choose, and point it at a dashboard you
|
|
317
|
+
control. It sends session metadata and usage numbers to that one URL —
|
|
318
|
+
never prompts, file contents or tool arguments — and it sends nothing at
|
|
319
|
+
all until you run `tokentrax-agent setup`.
|
|
320
|
+
|
|
321
|
+
TokenTrax itself never phones home. The agent is a separate package, so
|
|
322
|
+
installing TokenTrax does not install it.
|
|
323
|
+
|
|
324
|
+
See [Remote agents](#remote-agents).
|
|
325
|
+
|
|
206
326
|
---
|
|
207
327
|
|
|
208
328
|
## Development
|
|
@@ -227,6 +347,10 @@ tokentrax/
|
|
|
227
347
|
│ ├── sqlite.js # DB schema + upsert logic
|
|
228
348
|
│ ├── pricing.js # Cost computation
|
|
229
349
|
│ └── parsers/ # One parser per AI tool (13 total)
|
|
350
|
+
├── agent/ # tokentrax-agent — separate npm package
|
|
351
|
+
│ ├── bin/agent.js # Remote agent entry point
|
|
352
|
+
│ ├── install.sh # Bootstrap script
|
|
353
|
+
│ └── src/ # setup, sync, session parsing, hooks
|
|
230
354
|
├── dashboard/
|
|
231
355
|
│ └── index.html # Single-file SPA dashboard
|
|
232
356
|
├── docs/
|
package/bin/tracker.js
CHANGED
|
@@ -1,10 +1,58 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
|
|
2
|
+
// Entry point for the TokenTrax CLI.
|
|
3
|
+
//
|
|
4
|
+
// One failure deserves a friendly explanation: a missing better-sqlite3
|
|
5
|
+
// native binding. npm 12+ blocks dependency install scripts by default, and
|
|
6
|
+
// better-sqlite3's binding is built by its install script. `require()` of
|
|
7
|
+
// the module succeeds without it, so the failure only surfaces when a
|
|
8
|
+
// database is opened — which for `tokentrax sync` is a runtime error deep
|
|
9
|
+
// inside a parser.
|
|
10
|
+
//
|
|
11
|
+
// Left alone, the user gets a wall of "Could not locate the bindings file.
|
|
12
|
+
// Tried: ..." listing eight paths, which tells them nothing about what to
|
|
13
|
+
// do about it.
|
|
14
|
+
|
|
15
|
+
/**
|
|
16
|
+
* Is this the missing-native-binding failure?
|
|
17
|
+
* @param {unknown} err
|
|
18
|
+
* @returns {boolean}
|
|
19
|
+
*/
|
|
20
|
+
function isMissingBinding(err) {
|
|
21
|
+
const message = String((err && err.message) || "");
|
|
22
|
+
return /bindings|Could not locate the bindings file/i.test(message);
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
function explainMissingBinding() {
|
|
26
|
+
console.error(
|
|
27
|
+
"\n✗ better-sqlite3's native binding is missing.\n\n" +
|
|
28
|
+
" This normally means the package's install script was blocked.\n" +
|
|
29
|
+
" Fix it with:\n\n" +
|
|
30
|
+
" npm install-scripts approve better-sqlite3\n" +
|
|
31
|
+
" npm rebuild better-sqlite3\n\n" +
|
|
32
|
+
" Or reinstall with:\n\n" +
|
|
33
|
+
" npm install -g tokentrax --allow-scripts=better-sqlite3\n",
|
|
34
|
+
);
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
let run;
|
|
38
|
+
try {
|
|
39
|
+
({ run } = require("../src/cli.js"));
|
|
40
|
+
} catch (err) {
|
|
41
|
+
if (isMissingBinding(err)) {
|
|
42
|
+
explainMissingBinding();
|
|
43
|
+
process.exit(1);
|
|
44
|
+
}
|
|
45
|
+
throw err;
|
|
46
|
+
}
|
|
3
47
|
|
|
4
48
|
(async () => {
|
|
5
49
|
try {
|
|
6
50
|
await run(process.argv.slice(2));
|
|
7
51
|
} catch (err) {
|
|
52
|
+
if (isMissingBinding(err)) {
|
|
53
|
+
explainMissingBinding();
|
|
54
|
+
process.exit(1);
|
|
55
|
+
}
|
|
8
56
|
console.error("tokentrax:", err.message);
|
|
9
57
|
process.exit(1);
|
|
10
58
|
}
|