tokentrax 0.0.0-stage → 0.1.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Josh Greene
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,413 @@
-# Temporary Holding Version
+<img src="assets/tokentrax.png" width=50%>
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**Lean, local-first AI token usage tracker — zero bloat**
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![Node](https://img.shields.io/badge/node-%3E%3D20-brightgreen)](https://nodejs.org) <img src="assets/built-with-ai-assistance.png" height=20>
+<img src="assets/phone-high.jpg" alt="TokenTrax Dashboard" width="100%">
+
+---
+
+## Installation
+
+### Via npm (recommended)
+
+```bash
+npm install -g tokentrax --allow-scripts=better-sqlite3
+```
+
+> **Why the flag?** npm 12 and newer block dependency install scripts by
+> default, and `better-sqlite3` is a native module — its install script is
+> what fetches or builds the binding. Without the flag you get an install
+> that *looks* fine and then fails the first time it opens a database.
+> Older npm versions ignore the flag, so the command works everywhere.
+>
+> If you'd rather set it once:
+>
+> ```bash
+> npm config set allow-scripts=better-sqlite3 --location=user
+> ```
+
+### Via git clone
+
+```bash
+git clone https://github.com/jeshuwah/TokenTrax.git
+cd tokentrax
+./install.sh
+```
+
+Or manually:
+
+```bash
+git clone https://github.com/jeshuwah/TokenTrax.git
+cd tokentrax
+npm install
+npm link
+```
+
+This installs TokenTrax globally and automatically:
+
+- Runs database migrations (schema updates are applied automatically)
+- Backfills pricing for any existing data without costs
+
+### Requirements
+
+- Node.js >= 20
+- `better-sqlite3` (installed automatically as a native module)
+
+### Setup
+
+```bash
+tokentrax init    # Install hooks for detected AI tools
+tokentrax sync    # Parse existing logs → SQLite
+tokentrax serve   # Open dashboard at localhost:7680
+```
+
+### Updating
+
+```bash
+npm update -g tokentrax
+```
+
+Updates are safe — the postinstall script automatically migrates your existing database schema and backfills any missing pricing data.
+
+### Remote agent
+
+To report usage from a *second* machine — a laptop, a work box, anywhere
+with an AI harness — install the agent there. It ships as its own npm
+package:
+
+```bash
+npm install -g tokentrax-agent --allow-scripts=better-sqlite3
+tokentrax-agent setup --server http://your-dashboard:7680 --key <sync-key>
+```
+
+The `--allow-scripts` flag is for npm 12+, which blocks dependency install
+scripts by default — `better-sqlite3` needs its install script to build a
+native module. Older npm ignores the flag. See the note in the
+[Installation](#installation) section above.
+
+`setup` writes the configuration, installs a systemd user timer, and starts it.
+The machine then reports its sessions, tokens and cost to the dashboard every
+five minutes.
+
+No npm? Bootstrap straight from the repository:
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/jeshuwah/TokenTrax/main/agent/install.sh \
+  | bash -s -- --server http://your-dashboard:7680 --key <sync-key>
+```
+
+Full details in [`agent/README.md`](agent/README.md) and
+[`docs/agent-configuration.md`](docs/agent-configuration.md). See
+[Remote agents](#remote-agents) for how it fits together.
+
+---
+
+## Features
+
+- **Zero runtime dependencies** — only `better-sqlite3` (native module)
+- **Single-file dashboard** — no bundler, no framework, no build step
+- **SQLite storage** — indexed upserts, bounded file size, WAL mode
+- **12 AI tool integrations** — Claude Code, Codex, Gemini, Cursor, Kiro, Hermes, and more
+- **Bundled + live pricing** — 2200+ model prices cached locally, auto-refreshed
+- **Normalization engine** — maps 100+ model aliases to canonical names
+- **Privacy-first** — all data stays in `~/.tokentrax/`, no telemetry
+- **Optional remote agents** — one dashboard, many machines, via `tokentrax-agent`
+
+---
+
+## Dashboard
+
+### Overview
+
+Real-time metrics, daily usage chart, and source breakdown — everything at a glance.
+
+### Activity Heatmap
+
+GitHub-style contribution grid showing weekly token usage across 52 weeks.
+
+<img src="docs/screenshots/heatmap.png" alt="Activity Heatmap" width="100%">
+
+### Models Breakdown
+
+Per-model and per-source token tracking with cost estimates, sorted by usage.
+
+<img src="docs/screenshots/models.png" alt="Models Breakdown" width="100%">
+
+### Import & Upload
+
+Drag-and-drop JSONL/CSV files from unsupported tools, or upload API usage logs.
+
+<img src="docs/screenshots/import.png" alt="Import & Upload" width="100%">
+
+### Settings
+
+Theme toggle (dark/light), pricing freshness indicator, normalization rule editor, and sync controls.
+
+<img src="docs/screenshots/settings.png" alt="Settings" width="100%">
+
+### Agents
+
+Every machine reporting into this dashboard — sessions, tokens, cost and
+context-reduction percentage — with a per-machine breakdown by model and
+per-session detail. See [Remote agents](#remote-agents).
+
+---
+
+## Supported AI Tools
+
+| Tool | Integration | Type |
+| ------ | ------------- | ------ |
+| Claude Code | SessionEnd hook | Hook |
+| Codex CLI | TOML notify | Hook |
+| Gemini CLI | SessionEnd hook | Hook |
+| CodeBuddy | SessionEnd hook | Hook |
+| Every Code | TOML notify | Hook |
+| OpenCode | Plugin + SQLite | Hook + Passive |
+| Cursor | API + SQLite | Passive |
+| Kiro | SQLite + JSONL | Passive |
+| Hermes | SQLite | Passive |
+| GitHub Copilot | OpenTelemetry | Passive |
+| Kimi Code | JSONL reader | Passive |
+| oh-my-pi | JSONL reader | Passive |
+| Pi Coding Agent | JSONL reader | Passive |
+
+---
+
+## Remote agents
+
+TokenTrax is local-first: one machine, one SQLite file, no server. The
+optional `tokentrax-agent` package adds the other half — reporting from
+*other* machines into the same dashboard.
+
+```
+  laptop (omarchy)                server (olympus)
+  ┌──────────────────────┐         ┌────────────────────────┐
+  │ pi / Claude / Codex  │  POST   │  tokentrax serve :7680 │
+  │      tokentrax-agent │────────▶│  agent_sessions table  │
+  │   systemd timer 5min │         │  "Agents" dashboard tab │
+  └──────────────────────┘         └────────────────────────┘
+```
+
+Each remote machine runs the agent on a timer. It reads that machine's
+harness session data, condenses it, and POSTs it to the dashboard. The
+dashboard shows every machine in one list with sessions, tokens, cost and
+context-reduction percentage.
+
+### Install on the remote machine
+
+```bash
+npm install -g tokentrax-agent --allow-scripts=better-sqlite3
+tokentrax-agent setup --server http://your-dashboard:7680 --key <sync-key>
+```
+
+`setup` writes the configuration, verifies the native SQLite binding loads,
+installs a systemd user service and timer, and enables it. It is safe to
+re-run.
+
+### Prepare the dashboard
+
+The server needs a shared key before it will accept anything:
+
+```bash
+export TOKENTRAX_SYNC_KEY=$(openssl rand -base64 48)
+npm start serve
+```
+
+The same value goes on every remote machine. With a key set, the server
+refuses writes from anything that is not loopback, so it is safe to expose
+on a LAN or over a VPN.
+
+### What crosses the wire
+
+Session metadata and usage numbers only — never prompts, file contents or
+tool arguments. Project paths are hashed by default
+(`TOKENTRAX_TRACK_PROJECT=hash`), so the dashboard never learns your
+directory names. Set it to `full` only if you want the real paths.
+
+### A note on hooks
+
+The agent installs post-session hooks for harnesses that support them. The
+harness with the most data — pi — has no hook mechanism at all, so a hook
+would silently never fire there. The systemd timer is what actually keeps
+the data flowing, and it works for every harness.
+
+### Agent commands
+
+| Command | Description |
+| --------- | ------------- |
+| `tokentrax-agent setup --server <url> --key <key>` | Configure the machine and start the timer |
+| `tokentrax-agent sync [--auto]` | Send new sessions to the server |
+| `tokentrax-agent status` | Show configuration, machine id and cursor |
+| `tokentrax-agent init` | Install post-session hooks where supported |
+| `tokentrax-agent uninstall` | Remove post-session hooks |
+
+See [`agent/README.md`](agent/README.md) for the full package documentation.
+
+---
+
+## Commands
+
+### Core Commands
+
+| Command | Description |
+| --------- | ------------- |
+| `tokentrax` | Open dashboard (alias for `serve`) |
+| `tokentrax serve [--port N] [--no-browser]` | Start HTTP server on :7680 |
+| `tokentrax init [--force] [--source NAME]` | Install hooks for detected AI tools |
+| `tokentrax sync [--auto] [--source NAME]` | Parse logs → SQLite |
+| `tokentrax sync-pricing` | Fetch latest model prices from OpenRouter |
+| `tokentrax migrate-pricing` | Lock costs at current prices (one-time) |
+| `tokentrax status [--verbose]` | Check hook configuration |
+| `tokentrax doctor [--fix]` | Health check (DB, paths, hooks) |
+| `tokentrax uninstall [--purge]` | Remove hooks, optionally delete all data |
+
+### Query Commands
+
+| Command | Description |
+| --------- | ------------- |
+| `tokentrax costs [--days N] [--source NAME] [--json]` | Show cost summary |
+| `tokentrax top [--limit N] [--days N] [--by model\|source]` | Top models/sources by usage |
+| `tokentrax price [model] [--json]` | Look up model prices |
+| `tokentrax export [--format csv\|json] [--days N] [--source NAME]` | Export usage data |
+
+---
+
+## Data & Privacy
+
+All data stored in `~/.tokentrax/`:
+
+| File | Purpose |
+| ------ | --------- |
+| `tokentrax.db` | SQLite database with token records |
+| `pricing-cache.json` | Cached model prices (auto-refreshed) |
+| `model-norm.json` | Custom model name normalization rules |
+| `config.json` | User preferences (theme, timezone) |
+
+### Environment Variables
+
+| Variable | Description | Default |
+| ---------- | ------------- | --------- |
+| `TOKENTRAX_DEBUG` | Enable debug output | — |
+| `TOKENTRAX_HOME` | Override TokenTrax data directory | `~/.tokentrax` |
+| `TOKENTRAX_CONFIG_PATH` | Override config file path | `~/.tokentrax/config.json` |
+| `GEMINI_HOME` | Override Gemini config dir | `~/.gemini` |
+| `CODEX_HOME` | Override Codex config dir | `~/.codex` |
+| `CLAUDE_HOME` | Override Claude config dir | `~/.claude` |
+| `CODEBUDDY_HOME` | Override Codebuddy config dir | `~/.codebuddy` |
+| `EVERYCODE_HOME` | Override Everycode config dir | `~/.code` |
+| `PI_CODING_AGENT_DIR` | Override Pi Coding Agent base directory | `~/.pi/agent` |
+| `PI_CODING_AGENT_SESSION_DIR` | Override Pi sessions directory | `~/.pi/agent/sessions` |
+| `COPILOT_OTEL_FILE_EXPORTER_PATH` | Override Copilot OTEL export directory | — |
+
+---
+
+## Privacy
+
+- **No content uploaded** — only token counts and timestamps are stored
+- **Local-only by default** — all data in `~/.tokentrax/tokentrax.db`
+- **Auditable** — open source, read the parser code in `src/lib/parsers/`
+- **No telemetry** — no analytics, no crash reporting, no phone-home
+
+### …unless you install the remote agent
+
+`tokentrax-agent` is the one exception, and it is entirely opt-in: you
+install it, on a machine you choose, and point it at a dashboard you
+control. It sends session metadata and usage numbers to that one URL —
+never prompts, file contents or tool arguments — and it sends nothing at
+all until you run `tokentrax-agent setup`.
+
+TokenTrax itself never phones home. The agent is a separate package, so
+installing TokenTrax does not install it.
+
+See [Remote agents](#remote-agents).
+
+---
+
+## Development
+
+```bash
+git clone https://github.com/jeshuwah/TokenTrax.git
+cd tokentrax
+npm install
+npm start              # Run CLI
+npm test               # Run tests
+```
+
+### Project Structure
+
+```
+tokentrax/
+├── bin/tracker.js         # npm entry point
+├── src/
+│   ├── cli.js             # Command dispatcher
+│   ├── commands/          # init, sync, serve, status, doctor, uninstall
+│   └── lib/
+│       ├── sqlite.js      # DB schema + upsert logic
+│       ├── pricing.js     # Cost computation
+│       └── parsers/       # One parser per AI tool (13 total)
+├── agent/                 # tokentrax-agent — separate npm package
+│   ├── bin/agent.js       # Remote agent entry point
+│   ├── install.sh         # Bootstrap script
+│   └── src/               # setup, sync, session parsing, hooks
+├── dashboard/
+│   └── index.html         # Single-file SPA dashboard
+├── docs/
+│   ├── screenshots/       # Dashboard screenshots
+│   └── IMPLEMENTATION.md  # Architecture & design notes
+├── data/                  # Bundled pricing seed data
+└── tests/                 # Test suite
+```
+
+---
+
+## Comparison: TokenTracker vs TokenTrax
+
+| Metric | TokenTracker | TokenTrax |
+| -------- | -------------- | ----------- |
+| CLI LOC | ~8000 | ~2500 |
+| Dashboard files | 146 | 1 |
+| Runtime deps | 413 packages | 1 (better-sqlite3) |
+| Storage | queue.jsonl | SQLite |
+| Pricing | Live API + cache | Cached + auto-refresh |
+| Build step | Vite + TypeScript | None |
+| Dashboard size | ~2MB | ~10KB |
+
+---
+
+## Troubleshooting
+
+### Database locked
+
+```bash
+pkill -f tokentrax           # Kill any running server
+rm ~/.tokentrax/tokentrax.db-wal
+rm ~/.tokentrax/tokentrax.db-shm
+```
+
+### Hook not triggering
+
+```bash
+tokentrax status             # Check configuration
+tokentrax uninstall
+tokentrax init               # Reinstall hooks
+```
+
+### Missing provider
+
+1. Verify the tool is installed (e.g., `claude --version`)
+2. Check the config file exists (e.g., `~/.claude/settings.json`)
+3. Run `tokentrax init --source <name>` to force installation
+
+---
+
+## License
+
+[MIT](LICENSE)
+
+---
+
+## Credits
+
+Inspired by [TokenTracker](https://github.com/mm7894215/TokenTracker). TokenTrax is a minimal reimplementation focused on local-only usage with zero bloat.

package/bin/postinstall.js

@@ -0,0 +1,61 @@
+#!/usr/bin/env node
+/**
+ * tokentrax postinstall
+ *
+ * Runs automatically after `npm install` or `npm update`.
+ * Handles database migrations silently to ensure schema is up-to-date.
+ */
+const { createDB } = require("../src/lib/sqlite");
+
+try {
+  // Initialize DB (triggers schema migrations automatically)
+  const db = createDB();
+
+  // Check if there are any token rows without pricing
+  const unpricedCount = db.db
+    .prepare("SELECT COUNT(*) as cnt FROM tokens WHERE total_cost_usd IS NULL")
+    .get();
+
+  if (unpricedCount.cnt > 0) {
+    // Silently migrate pricing for existing rows
+    // This is safe to run multiple times - it only updates NULL values
+    const { getPricing, computeCost } = require("../src/lib/pricing");
+
+    const CHUNK = 1000;
+    const updateStmt = db.db.prepare(
+      "UPDATE tokens SET total_cost_usd = ? WHERE id = ? AND total_cost_usd IS NULL",
+    );
+
+    const migrate = db.db.transaction(() => {
+      let rowsUpdated;
+
+      do {
+        const rows = db.db
+          .prepare(`
+          SELECT id, hour_start, source, model, input_tokens, output_tokens,
+                 cached_input_tokens, cache_creation_input_tokens,
+                 reasoning_output_tokens, conversation_count
+          FROM tokens WHERE total_cost_usd IS NULL
+          LIMIT ?
+        `)
+          .all(CHUNK);
+
+        for (const row of rows) {
+          const pricing = getPricing(row.model);
+          const cost = computeCost(row, pricing);
+          updateStmt.run(cost, row.id);
+        }
+
+        rowsUpdated = rows.length;
+      } while (rowsUpdated === CHUNK);
+    });
+
+    migrate();
+  }
+
+  db.close();
+} catch (err) {
+  // Non-fatal: log but don't fail install
+  console.error("tokentrax: postinstall warning:", err.message);
+  process.exit(0);
+}

package/bin/start-server.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+set -e
+
+PID_FILE="${TOKENTRAX_PID_FILE:-$HOME/.tokentrax/server.pid}"
+PORT="${TOKENTRAX_PORT:-7680}"
+# Default to localhost. Set TOKENTRAX_HOST=0.0.0.0 explicitly to expose on LAN.
+# WARNING: exposing 0.0.0.0 without setting TOKENTRAX_SYNC_KEY leaves mutating
+# endpoints (/api/upload, /api/settings, /api/agent-sync) unauthenticated.
+HOST="${TOKENTRAX_HOST:-localhost}"
+
+cd "$(dirname "$0")/.."
+
+if [ -f "$PID_FILE" ]; then
+  OLD_PID=$(cat "$PID_FILE")
+  if kill -0 "$OLD_PID" 2>/dev/null; then
+    echo "Stopping existing server (PID $OLD_PID)..."
+    kill "$OLD_PID" 2>/dev/null || true
+    sleep 1
+  fi
+  rm -f "$PID_FILE"
+fi
+
+echo "Starting TokenTrax dashboard at http://$HOST:$PORT ..."
+
+# Run node in the foreground so systemd (Type=simple) supervises the real
+# server process instead of the exiting wrapper. Disable JOBCTL errexit
+# just for the launch so a non-zero exit from the server itself still
+# propagates through `exec` rather than being masked by `set -e`.
+set +e
+echo "$$" > "$PID_FILE"
+echo "Server starting (wrapper PID $$, will exec node)"
+# Print the address the server is actually bound to. This used to be hardcoded
+# to localhost, which contradicted the line above whenever TOKENTRAX_HOST was
+# set to a LAN or container address — the log advertised a URL that refused
+# connections while the real one worked.
+echo "Dashboard: http://$HOST:$PORT"
+echo "PID saved to: $PID_FILE"
+exec node bin/tracker.js serve --host "$HOST" --port "$PORT" --no-open

package/bin/stop-server.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+
+PID_FILE="${TOKENTRAX_PID_FILE:-$HOME/.tokentrax/server.pid}"
+
+if [ -f "$PID_FILE" ]; then
+  OLD_PID=$(cat "$PID_FILE")
+  if kill -0 "$OLD_PID" 2>/dev/null; then
+    echo "Stopping server (PID $OLD_PID)..."
+    kill "$OLD_PID"
+    rm -f "$PID_FILE"
+    echo "Server stopped"
+  else
+    echo "Server not running (stale PID file)"
+    rm -f "$PID_FILE"
+  fi
+else
+  echo "No PID file found"
+fi

package/bin/tracker.js

@@ -0,0 +1,59 @@
+#!/usr/bin/env node
+// Entry point for the TokenTrax CLI.
+//
+// One failure deserves a friendly explanation: a missing better-sqlite3
+// native binding. npm 12+ blocks dependency install scripts by default, and
+// better-sqlite3's binding is built by its install script. `require()` of
+// the module succeeds without it, so the failure only surfaces when a
+// database is opened — which for `tokentrax sync` is a runtime error deep
+// inside a parser.
+//
+// Left alone, the user gets a wall of "Could not locate the bindings file.
+// Tried: ..." listing eight paths, which tells them nothing about what to
+// do about it.
+
+/**
+ * Is this the missing-native-binding failure?
+ * @param {unknown} err
+ * @returns {boolean}
+ */
+function isMissingBinding(err) {
+  const message = String((err && err.message) || "");
+  return /bindings|Could not locate the bindings file/i.test(message);
+}
+
+function explainMissingBinding() {
+  console.error(
+    "\n✗ better-sqlite3's native binding is missing.\n\n" +
+      "  This normally means the package's install script was blocked.\n" +
+      "  Fix it with:\n\n" +
+      "    npm install-scripts approve better-sqlite3\n" +
+      "    npm rebuild better-sqlite3\n\n" +
+      "  Or reinstall with:\n\n" +
+      "    npm install -g tokentrax --allow-scripts=better-sqlite3\n",
+  );
+}
+
+let run;
+try {
+  ({ run } = require("../src/cli.js"));
+} catch (err) {
+  if (isMissingBinding(err)) {
+    explainMissingBinding();
+    process.exit(1);
+  }
+  throw err;
+}
+
+(async () => {
+  try {
+    await run(process.argv.slice(2));
+  } catch (err) {
+    if (isMissingBinding(err)) {
+      explainMissingBinding();
+      process.exit(1);
+    }
+    console.error("tokentrax:", err.message);
+    process.exit(1);
+  }
+})();