tokensniff 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,809 @@
1
+ # tokensniff
2
+
3
+ **Local telemetry sidecar for AI coding CLIs.**
4
+
5
+ Track every token, every turn, every dollar — live in your terminal and on a calendar heatmap dashboard. Zero runtime dependencies.
6
+
7
+ [![npm version](https://img.shields.io/npm/v/tokensniff)](https://www.npmjs.com/package/tokensniff)
8
+ [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](./LICENSE)
9
+ [![Node.js](https://img.shields.io/badge/node-%3E%3D22.0.0-brightgreen)](https://nodejs.org)
10
+
11
+ ---
12
+
13
+ ## What Is tokensniff?
14
+
15
+ tokensniff is a **transparent HTTP reverse proxy and local telemetry sidecar** designed specifically for the [Claude Code](https://docs.anthropic.com/en/docs/claude-code) CLI harness running alongside the local [`antigravity-claude-proxy`](https://github.com/badri-s2001/antigravity-claude-proxy).
16
+
17
+ By default, tokensniff orchestrates and connects to `antigravity-claude-proxy` to route Claude Code API requests through Google Antigravity / Cloud Code, allowing you to utilize your Antigravity Gemini quota directly inside Claude Code.
18
+
19
+ ---
20
+
21
+ ### ⚠️ Important Notice: Use At Your Own Risk & Account Warning
22
+
23
+ > [!CAUTION]
24
+ > **Use at your own risk.** By default, tokensniff routes traffic through [`antigravity-claude-proxy`](https://github.com/badri-s2001/antigravity-claude-proxy), which accesses Google Antigravity / Cloud Code endpoints using unofficial proxying techniques.
25
+ >
26
+ > - **Risk of Account Suspension / Bans**: Google actively monitors and enforces Terms of Service (ToS) restrictions. Accounts connected to unofficial Cloud Code and Antigravity reverse proxies risk being **shadow-banned, quota-restricted, or permanently banned**.
27
+ > - **No Guarantees & Zero Liability**: The creators and maintainers of `tokensniff` provide **no guarantees or warranties** of any kind, express or implied. We are **not responsible or liable** for any account bans, suspensions, quota penalties, data loss, or any other consequences resulting from the use of this software.
28
+ > - **Safety Recommendation**: **Do not use your primary, personal, or corporate Google account.** If you choose to use this integration, use a dedicated burner/disposable Google account.
29
+ > - **Learn More**: Read the upstream [antigravity-claude-proxy GitHub Repository](https://github.com/badri-s2001/antigravity-claude-proxy) and its [Safety, Usage, and Risk Notices](https://github.com/badri-s2001/antigravity-claude-proxy#readme) to understand how the underlying authentication and proxying operate.
30
+
31
+ ---
32
+
33
+ It captures **every API turn** flowing through it and gives you:
34
+
35
+ - **Per-turn token counts** — input, output, cache-read, cache-write, thinking tokens
36
+ - **Real-time cost tracking** — per turn, per session, per day (in USD)
37
+ - **Performance metrics** — Time to First Token (TTFT), tokens per second (TPS), turn duration
38
+ - **Live terminal status line** — a single-line HUD rendered directly inside Claude Code's status bar
39
+ - **Calendar heatmap dashboard** — a beautiful, self-contained HTML page showing your daily token spend over the entire year
40
+ - **Multi-terminal session management** — run multiple terminals against the same proxy without conflicts
41
+ - **Tool call tracking** — which tools the model called, how many times, argument token sizes
42
+ - **Context window monitoring** — how full your context is, with color-coded warnings (green → yellow → red)
43
+
44
+ All of this happens **transparently**. Your AI tool doesn't know tokensniff exists. It just thinks it's talking to the normal API. tokensniff forwards every byte with zero latency overhead, while quietly recording the telemetry.
45
+
46
+ ---
47
+
48
+ ## How It Works
49
+
50
+ ```mermaid
51
+ flowchart TD
52
+ CLI["Claude Code CLI<br/><code>ANTHROPIC_BASE_URL:4000</code>"]
53
+ TS["tokensniff<br/><code>Reverse Proxy :4000</code>"]
54
+ Proxy["antigravity-claude-proxy<br/><code>Local Upstream :8085</code>"]
55
+ Google["Google Cloud Code API<br/><code>Gemini Backend / Quota</code>"]
56
+
57
+ subgraph Telemetry["Telemetry Engine"]
58
+ T1["latest.json"]
59
+ T2["totals.json"]
60
+ T3["history.ndjson"]
61
+ T4["/dashboard (HTML)"]
62
+ end
63
+
64
+ Statusline["Claude Code Statusline HUD"]
65
+ Dashboard["Live Calendar Heatmap<br/><code>http://localhost:4000/dashboard</code>"]
66
+
67
+ CLI <-->|"1. Anthropic API Requests / Streams"| TS
68
+ TS <-->|"2. Forward"| Proxy
69
+ TS -->|"3. Telemetry (Async)"| Telemetry
70
+ Proxy <-->|"4. Protocol Translation"| Google
71
+
72
+ T1 --> Statusline
73
+ T4 --> Dashboard
74
+ ```
75
+
76
+ 1. **Harness Redirection**: You point Claude Code to `http://localhost:4000` by setting `ANTHROPIC_BASE_URL` in `~/.claude/settings.json`. Claude Code continues sending standard Anthropic Messages API requests normally.
77
+ 2. **Upstream Proxy Startup**: When launched, tokensniff checks port 8085 and automatically starts the local upstream proxy command (`npx antigravity-claude-proxy@latest start`) if it is not already running.
78
+ 3. **Protocol & Quota Translation**: The local [`antigravity-claude-proxy`](https://github.com/badri-s2001/antigravity-claude-proxy) receives the Anthropic-formatted request, authenticates with your Google Antigravity / Cloud Code OAuth credentials, translates the schema to Google Generative AI format, and submits it to Google Cloud Code's backend to consume your Antigravity Gemini quota.
79
+ 4. **Streaming Response Pass-Through**: As Google's response streams back, `antigravity-claude-proxy` transforms it into Anthropic-compatible SSE events or buffered JSON. tokensniff instantly relays every raw byte back to Claude Code (`res.write(chunk)`) with zero latency overhead.
80
+ 5. **Telemetry Extraction**: Concurrently and non-blockingly, tokensniff's parser inspects the response payload. It extracts Google's `usageMetadata` (input, output, cache-read, cache-write, and thinking/reasoning tokens), measures Time to First Token (TTFT) and token generation speed (TPS), and tracks turn duration.
81
+ 6. **Live Pricing Calculation**: tokensniff queries live pricing from [OpenRouter's model catalog](https://openrouter.ai), computing turn cost, session cumulative spend, and daily totals.
82
+ 7. **Statusline HUD & Calendar Heatmap**: Telemetry snapshots are written atomically to disk (`latest.json`, `totals.json`, `history.ndjson`). Claude Code's status bar runs `tokensniff-status` to display the live single-line HUD, and tokensniff serves an interactive calendar heatmap dashboard at `http://localhost:4000/dashboard`.
83
+
84
+ ---
85
+
86
+ ## Quick Start
87
+
88
+ ### 1. Install
89
+
90
+ ```bash
91
+ npm install -g tokensniff
92
+ ```
93
+
94
+ Or with pnpm:
95
+
96
+ ```bash
97
+ pnpm add -g tokensniff
98
+ ```
99
+
100
+ ### 2. Initialize Configuration
101
+
102
+ ```bash
103
+ tokensniff init
104
+ ```
105
+
106
+ This creates a configuration file at `~/.tokensniff/config.json` with sensible defaults:
107
+
108
+ ```json
109
+ {
110
+ "upstreamCommand": "npx antigravity-claude-proxy@latest start",
111
+ "upstreamHost": "localhost",
112
+ "upstreamPort": 8085,
113
+ "listenPort": 4000,
114
+ "harnessCommand": "claude"
115
+ }
116
+ ```
117
+
118
+ > **Default Upstream:** Out of the box, tokensniff is pre-configured to launch [`antigravity-claude-proxy`](https://github.com/badri-s2001/antigravity-claude-proxy) on port `8085`, allowing you to route Claude Code prompts through your Antigravity Gemini quota. Please review the [Risk & Terms of Service Warning](#️-important-notice-use-at-your-own-risk--account-warning) before running with this configuration.
119
+ >
120
+
121
+
122
+ ### 3. Configure Claude Code
123
+
124
+ Add the following to your **`~/.claude/settings.json`** file:
125
+
126
+ ```json
127
+ {
128
+ "env": {
129
+ "ANTHROPIC_BASE_URL": "http://localhost:4000"
130
+ },
131
+ "statusLine": {
132
+ "type": "command",
133
+ "command": "tokensniff-status",
134
+ "padding": 0
135
+ }
136
+ }
137
+ ```
138
+
139
+ This does two things:
140
+
141
+ - **`ANTHROPIC_BASE_URL`** — Tells Claude Code to send all API requests through tokensniff's proxy on port 4000 instead of directly to Anthropic
142
+ - **`statusLine`** — Tells Claude Code to run `tokensniff-status` and display its output as a live status bar at the bottom of the terminal
143
+
144
+ ### 4. Run
145
+
146
+ ```bash
147
+ tokensniff
148
+ ```
149
+
150
+ That's it. tokensniff will:
151
+
152
+ 1. Start the upstream proxy (if configured)
153
+ 2. Start the telemetry proxy on port 4000
154
+ 3. Launch Claude Code (or whatever harness command you configured)
155
+ 4. Show you a live status line and dashboard URL
156
+
157
+ Open the dashboard in your browser:
158
+
159
+ ```
160
+ http://localhost:4000/dashboard
161
+ ```
162
+
163
+ ---
164
+
165
+ ## CLI Commands
166
+
167
+ ### `tokensniff`
168
+
169
+ Runs the full telemetry pipeline: starts the upstream proxy, starts the tokensniff proxy, and launches your AI coding tool.
170
+
171
+ ```bash
172
+ tokensniff
173
+ ```
174
+
175
+ ### `tokensniff init`
176
+
177
+ Generates the global configuration file at `~/.tokensniff/config.json` and prints the Claude Code settings.json instructions.
178
+
179
+ ```bash
180
+ tokensniff init
181
+ ```
182
+
183
+ ### `tokensniff dashboard`
184
+
185
+ Prints the URL of the live calendar heatmap dashboard.
186
+
187
+ ```bash
188
+ tokensniff dashboard
189
+ # Output: [tokensniff] live heatmap dashboard: http://127.0.0.1:4000/dashboard
190
+ ```
191
+
192
+ ### `tokensniff-status`
193
+
194
+ This is the statusline renderer. You don't run this directly — Claude Code runs it automatically via the `statusLine` setting. It reads the latest telemetry snapshot from disk and outputs a formatted single-line status bar.
195
+
196
+ If you want to test it manually:
197
+
198
+ ```bash
199
+ echo '{}' | tokensniff-status
200
+ ```
201
+
202
+ To get raw JSON output instead of the formatted status line:
203
+
204
+ ```bash
205
+ TOKENSNIFF_JSON=1 echo '{}' | tokensniff-status
206
+ ```
207
+
208
+ ---
209
+
210
+ ## The Status Line
211
+
212
+ When running inside Claude Code, you'll see a live status line at the bottom of your terminal that looks like this:
213
+
214
+ **Wide terminals (≥ 180 columns) — single line:**
215
+
216
+ ```
217
+ turn 3 [3.8-flash] | ctx: 38.5k/1M (3.9%) [cache: 25k (64.9%)] | input: +4.5k tok, output: 42 tok, think: 180 tok | ttft: 1.2s, 110 tok/s | cost: turn $0.0042, sess $0.058, today $0.245
218
+ ```
219
+
220
+ **Standard terminals (< 180 columns) — clean 2-line stack:**
221
+
222
+ ```
223
+ turn 3 [3.8-flash] | ctx: 38.5k/1M (3.9%) [cache: 25k (64.9%)]
224
+ input: +4.5k tok, output: 42 tok, think: 180 tok | ttft: 1.2s, 110 tok/s | cost: turn $0.0042, sess $0.058, today $0.245
225
+ ```
226
+
227
+ ### What Each Segment Means
228
+
229
+ | Segment | Example | Meaning |
230
+ |---------|---------|---------|
231
+ | **Turn** | `turn 3` | Which turn number this is in the current session |
232
+ | **Model** | `[3.8-flash]` | The AI model being used (vendor prefix stripped for readability) |
233
+ | **[bg]** | `[bg]` | Shown when this is a background agent turn (via `x-app: cli-bg` header) |
234
+ | **Context** | `ctx: 38.5k/1M (3.9%)` | Current context usage / max window size (percentage full) |
235
+ | **Cache** | `[cache: 25k (64.9%)]` | How many tokens were served from cache and the cache hit ratio |
236
+ | **Input** | `input: +4.5k tok` | New tokens added to context this turn (delta). Shows `freed` when context shrinks |
237
+ | **Output** | `output: 42 tok` | Tokens generated by the model this turn |
238
+ | **Thinking** | `think: 180 tok` | Reasoning/thinking tokens used (for models with extended thinking) |
239
+ | **Tool** | `tool: Read (45 tok)` | Which tool was called and how many argument tokens it used |
240
+ | **TTFT** | `ttft: 1.2s` | Time to First Token — how long before the model started generating |
241
+ | **Speed** | `110 tok/s` | Generation velocity in tokens per second |
242
+ | **Turn Cost** | `turn $0.0042` | How much this specific turn cost in USD |
243
+ | **Session Cost** | `sess $0.058` | Total spend for this entire session |
244
+ | **Today Cost** | `today $0.245` | Total spend across all sessions today (resets at local midnight) |
245
+ | **[idle]** | `[idle]` | Shown when the last turn was more than 120 seconds ago |
246
+
247
+ ### Color Coding
248
+
249
+ The context percentage is color-coded based on how full your context window is:
250
+
251
+ - 🟢 **Green** — Under 60% (plenty of room)
252
+ - 🟡 **Yellow** — 60-80% (getting full, consider starting a new session)
253
+ - 🔴 **Red** — Over 80% (context is nearly full, model may start forgetting earlier context)
254
+
255
+ These thresholds are configurable via `warnPct` and `critPct`.
256
+
257
+ ---
258
+
259
+ ## The Dashboard
260
+
261
+ tokensniff serves a **live calendar heatmap dashboard** directly on the proxy port. Open it in any browser:
262
+
263
+ ```
264
+ http://localhost:4000/dashboard
265
+ ```
266
+
267
+ The dashboard shows:
268
+
269
+ - **Total Spend** — cumulative USD spent across all sessions
270
+ - **Total Tokens** — cumulative token volume processed
271
+ - **Cache Ratio** — what percentage of tokens were served from cache
272
+ - **Total Turns** — how many API turns have been recorded
273
+ - **Calendar Heatmap** — a GitHub-contributions-style grid showing daily token volume across the year
274
+
275
+ ### Heatmap Tiers
276
+
277
+ The calendar tiles are colored using a 5-tier emerald luminosity scale:
278
+
279
+ | Tier | Daily Volume | Color |
280
+ |------|-------------|-------|
281
+ | 0 | No activity | Dark (nearly invisible) |
282
+ | 1 | < 5M tokens | Dark emerald |
283
+ | 2 | 5M – 25M tokens | Medium emerald |
284
+ | 3 | 25M – 100M tokens | Bright emerald |
285
+ | 4 | > 100M tokens | Vivid emerald (glowing) |
286
+
287
+ Hover over any tile to see a detailed tooltip with exact token counts, cost, cache leverage, and turns logged for that day.
288
+
289
+ ### Dashboard API
290
+
291
+ There's also a JSON API endpoint for programmatic access:
292
+
293
+ ```bash
294
+ curl http://localhost:4000/api/daily
295
+ ```
296
+
297
+ Returns the raw daily rollup data as JSON.
298
+
299
+ ---
300
+
301
+ ## Configuration
302
+
303
+ tokensniff uses a strict hierarchical configuration system:
304
+
305
+ ```
306
+ Environment Variables > Config File > Embedded Defaults
307
+ (highest) (medium) (lowest)
308
+ ```
309
+
310
+ ### Config File Location
311
+
312
+ ```
313
+ ~/.tokensniff/config.json
314
+ ```
315
+
316
+ You can override this path with the `TOKENSNIFF_CONFIG` environment variable:
317
+
318
+ ```bash
319
+ TOKENSNIFF_CONFIG=/path/to/custom/config.json tokensniff
320
+ ```
321
+
322
+ ### All Configuration Options
323
+
324
+ | Config Key | Env Variable | Default | Description |
325
+ |-----------|-------------|---------|-------------|
326
+ | `listenPort` | `TOKENSNIFF_PORT` | `4000` | Port the tokensniff proxy listens on |
327
+ | `listenHost` | `TOKENSNIFF_HOST` | `127.0.0.1` | Host/IP the proxy binds to |
328
+ | `upstreamHost` | `TOKENSNIFF_UPSTREAM_HOST` | `localhost` | Hostname of the upstream API server |
329
+ | `upstreamPort` | `TOKENSNIFF_UPSTREAM_PORT` | `8085` | Port of the upstream API server |
330
+ | `upstreamTimeoutMs` | `TOKENSNIFF_UPSTREAM_TIMEOUT_MS` | `300000` (5 min) | Timeout for upstream requests in milliseconds (range: 1,000 – 3,600,000) |
331
+ | `maxBodyBytes` | `TOKENSNIFF_MAX_BODY_BYTES` | `10485760` (10 MB) | Maximum request/response body size. Requests exceeding this get a 413 error (range: 1,024 – 104,857,600) |
332
+ | `statusDirs` | `TOKENSNIFF_STATUS_DIRS` | `~/.tokensniff/status` | Comma-separated list of directories where telemetry snapshots are written |
333
+ | `maxSessions` | `TOKENSNIFF_MAX_SESSIONS` | `50` | Maximum number of tracked sessions before oldest are evicted (range: 1 – 1,000) |
334
+ | `deadSessionMs` | `TOKENSNIFF_DEAD_SESSION_MS` | `259200000` (72 hrs) | How long to keep stale session files before pruning (range: 1 hour minimum) |
335
+ | `staleAfterS` | `TOKENSNIFF_STALE_AFTER_S` | `120` | Seconds of inactivity before a session shows `[idle]` in the status line (range: 5 – 86,400) |
336
+ | `labelMaxIn` | `TOKENSNIFF_LABEL_MAX_IN` | `2000` | Maximum input tokens for a turn to be classified as a micro-label |
337
+ | `labelMaxOut` | `TOKENSNIFF_LABEL_MAX_OUT` | `60` | Maximum output tokens for a turn to be classified as a micro-label |
338
+ | `warnPct` | `TOKENSNIFF_WARN_PCT` | `60` | Context utilization % threshold for yellow warning color (range: 1 – 99) |
339
+ | `critPct` | `TOKENSNIFF_CRIT_PCT` | `80` | Context utilization % threshold for red critical color (range: 2 – 100) |
340
+ | `color` | `TOKENSNIFF_COLOR` | `true` | Enable/disable ANSI color output. Automatically disabled when `NO_COLOR` env var is set (per [no-color.org](https://no-color.org/)) |
341
+ | `upstreamCommand` | `TOKENSNIFF_UPSTREAM_CMD` | `npx antigravity-claude-proxy@latest start` | Shell command to start the upstream proxy server |
342
+ | `upstreamStopCommand` | `TOKENSNIFF_UPSTREAM_STOP_CMD` | _(empty)_ | Shell command to stop the upstream proxy on shutdown |
343
+ | `harnessCommand` | `TOKENSNIFF_HARNESS_CMD` | `claude` | The AI coding tool to launch (e.g., `claude`, `codex`, or any executable) |
344
+
345
+ ### Example Config File
346
+
347
+ ```json
348
+ {
349
+ "listenPort": 4000,
350
+ "listenHost": "127.0.0.1",
351
+ "upstreamHost": "localhost",
352
+ "upstreamPort": 8085,
353
+ "upstreamCommand": "npx antigravity-claude-proxy@latest start",
354
+ "upstreamStopCommand": "",
355
+ "harnessCommand": "claude",
356
+ "maxSessions": 50,
357
+ "staleAfterS": 120,
358
+ "warnPct": 60,
359
+ "critPct": 80,
360
+ "color": true
361
+ }
362
+ ```
363
+
364
+ ### Environment Variable Examples
365
+
366
+ ```bash
367
+ # Change the proxy port
368
+ TOKENSNIFF_PORT=5000 tokensniff
369
+
370
+ # Point to a different upstream
371
+ TOKENSNIFF_UPSTREAM_HOST=api.anthropic.com TOKENSNIFF_UPSTREAM_PORT=443 tokensniff
372
+
373
+ # Disable colors
374
+ NO_COLOR=1 tokensniff
375
+
376
+ # Use a custom config file
377
+ TOKENSNIFF_CONFIG=./my-config.json tokensniff
378
+
379
+ # Write status to multiple directories
380
+ TOKENSNIFF_STATUS_DIRS="/path/a,/path/b" tokensniff
381
+ ```
382
+
383
+ ---
384
+
385
+ ## Multi-Terminal Support
386
+
387
+ tokensniff supports **multiple terminal sessions** running simultaneously against the same proxy. This is how it works:
388
+
389
+ 1. When you run `tokensniff`, it first checks if a proxy is **already running** on port 4000
390
+ 2. If yes, it **attaches** to the existing proxy (no duplicate servers) and just launches your harness
391
+ 3. Each terminal session registers itself in `~/.tokensniff/sessions/` with a PID lock file
392
+ 4. When you close a terminal (Ctrl+C or exit):
393
+ - If **other terminals are still active** → only the local harness is terminated; the proxy stays alive
394
+ - If **this was the last terminal** → the proxy and upstream are cleanly shut down
395
+ 5. Dead PID lock files (from crashed terminals) are automatically pruned
396
+
397
+ This means you can have 5 Claude Code windows all routing through the same tokensniff proxy, and the telemetry stays unified. Session costs are tracked independently, but `today` cost accumulates across all sessions.
398
+
399
+ ---
400
+
401
+ ## Supported Providers
402
+
403
+ tokensniff's parser understands multiple API response formats:
404
+
405
+ | Provider | Format | Detection |
406
+ |----------|--------|-----------|
407
+ | **Anthropic** (Claude) | SSE streams (`text/event-stream`) | `event: message_start` framing |
408
+ | **Anthropic** (Claude) | Buffered JSON (`application/json`) | `usage.input_tokens` / `usage.output_tokens` |
409
+ | **Google Gemini** (Antigravity) | Buffered JSON | `candidates[].content.parts[]` + `usageMetadata` |
410
+
411
+
412
+ The parser automatically detects the format from the response content-type and payload structure. You don't need to configure anything.
413
+
414
+ ### Pricing
415
+
416
+ tokensniff fetches live per-token pricing from [OpenRouter's public model catalog](https://openrouter.ai/api/v1/models). This means:
417
+
418
+ - Pricing is **always up to date** — no hardcoded rate tables to maintain
419
+ - **Any model** listed on OpenRouter is automatically priced correctly
420
+ - Pricing includes prompt, completion, cache-read, and cache-write tiers
421
+ - Context window sizes are also pulled dynamically from the catalog
422
+
423
+ If a model isn't found on OpenRouter, costs default to `$0.00` (zero-rate boundary) rather than guessing wrong. The context window defaults to 128K tokens for unknown models.
424
+
425
+ The pricing cache is populated on-demand (first turn using a new model triggers a background fetch) and persists in memory for the lifetime of the proxy process.
426
+
427
+ ---
428
+
429
+ ## Telemetry Data Files
430
+
431
+ tokensniff stores all telemetry in `~/.tokensniff/status/`:
432
+
433
+ | File | Format | Purpose |
434
+ |------|--------|---------|
435
+ | `latest.json` | JSON | Most recent telemetry snapshot (any session) |
436
+ | `latest-<session_id>.json` | JSON | Most recent snapshot for a specific session |
437
+ | `totals.json` | JSON | Cumulative spend per session |
438
+ | `history.ndjson` | Newline-delimited JSON | Append-only turn-by-turn ledger (feeds the heatmap) |
439
+
440
+ ### latest.json Schema (v1)
441
+
442
+ ```json
443
+ {
444
+ "v": 1,
445
+ "session_id": "abc-123",
446
+ "turn_index": 5,
447
+ "ts": 1726056000000,
448
+ "model": "gemini-3.8-flash-tiered",
449
+ "is_bg": false,
450
+ "ctx": 38500,
451
+ "window": 1000000,
452
+ "pct": 3.9,
453
+ "cache_read": 25000,
454
+ "cache_create": 0,
455
+ "delta_in": 4500,
456
+ "in_tokens": 13500,
457
+ "out_tokens": 42,
458
+ "thinking": true,
459
+ "thinking_tokens": 180,
460
+ "tool": "Read",
461
+ "tool_tk": 45,
462
+ "tools_summary": [{ "name": "Read", "count": 1, "arg_tk": 45 }],
463
+ "stop_reason": "end_turn",
464
+ "is_label": false,
465
+ "label": "",
466
+ "ttft_ms": 1200,
467
+ "tps": 110,
468
+ "dur_s": 1.6,
469
+ "cost_turn": 0.0042,
470
+ "cost_session": 0.058,
471
+ "cost_today": 0.245,
472
+ "error": null,
473
+ "status": 200,
474
+ "quota_pct": null,
475
+ "quota_reset": null,
476
+ "tier": null
477
+ }
478
+ ```
479
+
480
+ ### history.ndjson Record Format
481
+
482
+ Each line is a compact JSON object:
483
+
484
+ ```json
485
+ {"d":"2026-09-11","ts":1726056000000,"s":"abc-123","t":5,"m":"gemini-3.8-flash","tk":38542,"c":0.0042,"cr":25000,"th":180}
486
+ ```
487
+
488
+ | Field | Meaning |
489
+ |-------|---------|
490
+ | `d` | Local calendar date (YYYY-MM-DD) |
491
+ | `ts` | Unix timestamp in milliseconds |
492
+ | `s` | Session ID |
493
+ | `t` | Turn number |
494
+ | `m` | Model name |
495
+ | `tk` | Total tokens (context + output) |
496
+ | `c` | Cost in USD |
497
+ | `cr` | Cache-read tokens |
498
+ | `th` | Thinking tokens |
499
+
500
+ ---
501
+
502
+ ## Programmatic API
503
+
504
+ tokensniff exports its entire engine as a library. You can use it in your own Node.js projects:
505
+
506
+ ```bash
507
+ npm install tokensniff
508
+ ```
509
+
510
+ ### Start the Proxy Programmatically
511
+
512
+ ```js
513
+ import { startProxy } from 'tokensniff';
514
+
515
+ const server = startProxy({
516
+ listenPort: 4000,
517
+ listenHost: '127.0.0.1',
518
+ upstreamHost: 'localhost',
519
+ upstreamPort: 8085,
520
+ });
521
+
522
+ // server is a standard Node.js http.Server
523
+ server.on('listening', () => {
524
+ console.log('tokensniff proxy is running');
525
+ });
526
+ ```
527
+
528
+ ### Calculate Costs
529
+
530
+ ```js
531
+ import { costFor, ratesFor, resolveModelRates } from 'tokensniff/pricing';
532
+
533
+ // Synchronous (from cache, or zero if not yet fetched)
534
+ const rates = ratesFor('claude-3-7-sonnet');
535
+
536
+ // Async (fetches from OpenRouter if needed)
537
+ const rates2 = await resolveModelRates('gemini-2.5-pro');
538
+
539
+ // Calculate cost
540
+ const cost = costFor(rates, {
541
+ input: 10000,
542
+ output: 500,
543
+ cacheRead: 8000,
544
+ cacheCreate: 0,
545
+ });
546
+
547
+ console.log(`Turn cost: $${cost.toFixed(4)}`);
548
+ ```
549
+
550
+ ### Use the Schema
551
+
552
+ ```js
553
+ import { buildLatest, isTokenSniffLatest, SCHEMA_VERSION } from 'tokensniff/schema';
554
+
555
+ // Build a telemetry snapshot with safe defaults
556
+ const snapshot = buildLatest({
557
+ session_id: 'my-session',
558
+ model: 'gemini-3.8-flash',
559
+ ctx: 25000,
560
+ window: 1000000,
561
+ pct: 2.5,
562
+ });
563
+
564
+ // Validate unknown data
565
+ if (isTokenSniffLatest(someData)) {
566
+ console.log('Valid telemetry snapshot');
567
+ }
568
+ ```
569
+
570
+ ### Full API Exports
571
+
572
+ ```js
573
+ import {
574
+ // CLI Orchestrator
575
+ runCli,
576
+ countActiveSessions,
577
+ getSessionsDir,
578
+ isPortActive,
579
+ isProcessAlive,
580
+ registerSession,
581
+ waitForTcp,
582
+ writeInitFile,
583
+ printClaudeInstructions,
584
+
585
+ // Configuration
586
+ loadConfig,
587
+ getGlobalDir,
588
+ getGlobalConfigPath,
589
+
590
+ // Proxy Server
591
+ startProxy,
592
+
593
+ // Dashboard & Analytics
594
+ importCapturesDirectory,
595
+ loadDailyRollup,
596
+ renderHeatmapHtml,
597
+
598
+ // Statusline Renderer
599
+ formatStatus,
600
+ formatTokenCount,
601
+ formatToolSegment,
602
+ runStatusRenderer,
603
+ readStdin,
604
+ pickLatest,
605
+ extractModel,
606
+
607
+ // Pricing Engine
608
+ costFor,
609
+ ratesFor,
610
+ resolveModelRates,
611
+ normalizeModelName,
612
+ syncOpenRouterCatalog,
613
+
614
+ // Schema & Types
615
+ buildLatest,
616
+ isTokenSniffLatest,
617
+ extractCleanLabel,
618
+ SCHEMA_VERSION,
619
+ } from 'tokensniff';
620
+ ```
621
+
622
+ ### Sub-path Exports
623
+
624
+ ```js
625
+ // Just the schema types and guards
626
+ import { buildLatest, isTokenSniffLatest } from 'tokensniff/schema';
627
+
628
+ // Just the pricing engine
629
+ import { costFor, resolveModelRates } from 'tokensniff/pricing';
630
+ ```
631
+
632
+ ---
633
+
634
+ ## Architecture
635
+
636
+ ```
637
+ tokensniff/
638
+ ├── bin/
639
+ │ ├── tokensniff.js # CLI entrypoint → dist/cli.js
640
+ │ └── tokensniff-status.js # Statusline entrypoint → dist/status.js
641
+ ├── src/
642
+ │ ├── index.ts # Public API facade (re-exports everything)
643
+ │ ├── cli/
644
+ │ │ └── run.ts # Master CLI orchestrator & multi-terminal lifecycle
645
+ │ ├── collector/
646
+ │ │ ├── config.ts # Hierarchical config loader (env > file > defaults)
647
+ │ │ ├── index.ts # HTTP reverse proxy server & telemetry capture
648
+ │ │ ├── parse.ts # SSE stream & JSON payload parser (multi-provider)
649
+ │ │ └── store.ts # Atomic file persistence engine (Windows-safe)
650
+ │ ├── dashboard/
651
+ │ │ └── heatmap.ts # Calendar heatmap HTML renderer & capture importer
652
+ │ ├── renderer/
653
+ │ │ ├── format.ts # Responsive statusline formatter
654
+ │ │ └── index.ts # Statusline CLI renderer (stdin consumer)
655
+ │ └── shared/
656
+ │ ├── pricing.ts # Dynamic pricing engine (OpenRouter catalog sync)
657
+ │ └── schema.ts # Domain schemas, type guards, label extraction
658
+ └── test/
659
+ ├── fixtures.ts # Synthetic SSE/JSON payload generators
660
+ ├── collector.test.ts # Integration: proxy, streaming, 502, 413, dashboard
661
+ ├── format.test.ts # Statusline formatting & responsive layouts
662
+ ├── heatmap.test.ts # Daily rollups, capture import, HTML rendering
663
+ ├── parse.test.ts # SSE/JSON parsing, multi-provider, tool grouping
664
+ ├── pricing.test.ts # Cost math, OpenRouter sync, model normalization
665
+ ├── run.test.ts # CLI routing, TCP probing, session lifecycle
666
+ ├── schema.test.ts # Type guards, buildLatest, label extraction
667
+ └── store.test.ts # Atomic writes, totals, history, pruning, config
668
+ ```
669
+
670
+ ### Key Design Decisions
671
+
672
+ - **Zero runtime dependencies** — The entire package uses only Node.js built-in modules (`http`, `fs`, `net`, `path`, `os`, `child_process`). No Express, no Axios, no anything. This keeps the install tiny and avoids supply chain risk.
673
+
674
+ - **Atomic file writes** — All file persistence uses a temp-file + OS rename pattern. This means readers (the statusline renderer) never see a half-written JSON file. On Windows, retries with exponential backoff handle EPERM/EBUSY file lock contention.
675
+
676
+ - **Streaming-first proxy** — Response bytes are forwarded to the client as they arrive (`res.write(chunk)`). tokensniff never buffers the full response before forwarding. This means zero latency overhead. The telemetry parsing happens on the buffered copy.
677
+
678
+ - **LRU-bounded memory** — Internal maps (context history, turn indices) are capped at 500 entries using LRU eviction. The proxy can run for weeks without leaking memory.
679
+
680
+ - **Defensive parsing** — The parser never throws. Corrupt payloads, truncated SSE streams, binary noise — everything returns a safe fallback result. This is critical because the proxy sits in the hot path of your AI tool.
681
+
682
+ ---
683
+
684
+ ## Development
685
+
686
+ ### Prerequisites
687
+
688
+ - Node.js ≥ 22.0.0
689
+ - pnpm 11.x
690
+
691
+ ### Setup
692
+
693
+ ```bash
694
+ git clone https://github.com/neeraj0304/tokensniff.git
695
+ cd tokensniff
696
+ pnpm install
697
+ ```
698
+
699
+ ### Build
700
+
701
+ ```bash
702
+ pnpm build
703
+ ```
704
+
705
+ ### Run Tests
706
+
707
+ ```bash
708
+ pnpm test
709
+ ```
710
+
711
+ ### Type Check
712
+
713
+ ```bash
714
+ pnpm typecheck
715
+ ```
716
+
717
+ ### Lint
718
+
719
+ ```bash
720
+ pnpm lint
721
+ ```
722
+
723
+ ### Auto-fix Lint Issues
724
+
725
+ ```bash
726
+ pnpm lint:fix
727
+ ```
728
+
729
+ ### Watch Mode (Development)
730
+
731
+ ```bash
732
+ pnpm dev
733
+ ```
734
+
735
+ ### Validate Package Structure
736
+
737
+ ```bash
738
+ pnpm check:package
739
+ ```
740
+
741
+ ---
742
+
743
+ ## Troubleshooting
744
+
745
+ ### "waiting for first turn..."
746
+
747
+ The status line shows this message when tokensniff hasn't received any API requests yet. Make sure:
748
+
749
+ 1. Your `ANTHROPIC_BASE_URL` is set to `http://localhost:4000` in `~/.claude/settings.json`
750
+ 2. The tokensniff proxy is actually running (check terminal output)
751
+ 3. You've made at least one prompt in Claude Code
752
+
753
+ ### Port 4000 is already in use
754
+
755
+ Another tokensniff instance (or another program) is using port 4000. Either:
756
+
757
+ - Let tokensniff attach to it (it will do this automatically if the existing proxy is tokensniff)
758
+ - Change the port: `TOKENSNIFF_PORT=5000 tokensniff`
759
+ - Kill the existing process: find and terminate whatever is using port 4000
760
+
761
+ ### Upstream proxy failed health check
762
+
763
+ tokensniff couldn't connect to the upstream API server. Check that:
764
+
765
+ 1. Your `upstreamCommand` is valid and the upstream server starts correctly
766
+ 2. The `upstreamHost` and `upstreamPort` match where the upstream is listening
767
+ 3. The upstream server is not firewalled or blocked
768
+
769
+ tokensniff will continue running even if the upstream health check fails — downstream requests will get 502 errors until the upstream becomes available.
770
+
771
+ ### Costs showing $0.0000
772
+
773
+ This means tokensniff couldn't find pricing for the model you're using on OpenRouter. This can happen if:
774
+
775
+ - The model is brand new and not yet listed on OpenRouter
776
+ - The OpenRouter API was unreachable when tokensniff tried to fetch pricing
777
+ - You're using a custom/private model that isn't publicly listed
778
+
779
+ The proxy and telemetry still work perfectly — only the cost calculation defaults to zero.
780
+
781
+ ### Status line not appearing in Claude Code
782
+
783
+ Make sure your `~/.claude/settings.json` has the exact `statusLine` block:
784
+
785
+ ```json
786
+ {
787
+ "statusLine": {
788
+ "type": "command",
789
+ "command": "tokensniff-status",
790
+ "padding": 0
791
+ }
792
+ }
793
+ ```
794
+
795
+ Also verify that `tokensniff-status` is accessible in your PATH (it should be if you installed tokensniff globally).
796
+
797
+ ---
798
+
799
+ ## Requirements
800
+
801
+ - **Node.js** ≥ 22.0.0
802
+ - **OS**: Windows, macOS, or Linux
803
+ - **Terminal**: Any terminal that supports ANSI colors (for the status line color coding)
804
+
805
+ ---
806
+
807
+ ## License
808
+
809
+ [MIT](./LICENSE) © 2026 tokensniff contributors