@diogonzafe/tokenwatch 0.4.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +132 -6
- package/dist/adapters.d.cts +1 -1
- package/dist/adapters.d.ts +1 -1
- package/dist/cli.js +848 -8
- package/dist/cli.js.map +1 -1
- package/dist/exporters.cjs +76 -0
- package/dist/exporters.cjs.map +1 -0
- package/dist/exporters.d.cts +60 -0
- package/dist/exporters.d.ts +60 -0
- package/dist/exporters.js +56 -0
- package/dist/exporters.js.map +1 -0
- package/dist/{index-CJKk1hHw.d.cts → index-D9xq0RNg.d.cts} +19 -1
- package/dist/{index-CJKk1hHw.d.ts → index-D9xq0RNg.d.ts} +19 -1
- package/dist/index.cjs +63 -3
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +2 -2
- package/dist/index.d.ts +2 -2
- package/dist/index.js +63 -3
- package/dist/index.js.map +1 -1
- package/dist/langchain.d.cts +1 -1
- package/dist/langchain.d.ts +1 -1
- package/package.json +13 -3
- package/prices.json +1 -1
- package/dist/cli.d.ts +0 -1
package/README.md
CHANGED
|
@@ -542,18 +542,50 @@ Budget webhook payload:
|
|
|
542
542
|
{ "text": "[tokenwatch] Budget alert: user \"user_123\" reached $1.0031 USD (threshold: $1)" }
|
|
543
543
|
```
|
|
544
544
|
|
|
545
|
+
### Cost anomaly detection
|
|
546
|
+
|
|
547
|
+
Alert when a single call is anomalously expensive compared to the recent average — catches runaway agents, infinite loops, or abusive usage:
|
|
548
|
+
|
|
549
|
+
```ts
|
|
550
|
+
const tracker = createTracker({
|
|
551
|
+
anomalyDetection: {
|
|
552
|
+
multiplierThreshold: 3, // alert if call cost > 3× rolling average
|
|
553
|
+
webhookUrl: 'https://hooks.slack.com/...',
|
|
554
|
+
windowHours: 24, // lookback window (default: 24h)
|
|
555
|
+
mode: 'once', // 'once' (default) or 'always'
|
|
556
|
+
},
|
|
557
|
+
})
|
|
558
|
+
```
|
|
559
|
+
|
|
560
|
+
Checks two axes independently for each tracked call:
|
|
561
|
+
- **Per-model** — compares the call's cost to the average of all prior calls for that model within `windowHours`
|
|
562
|
+
- **Per-user** — same check scoped to a specific user (only fires when `userId` is present)
|
|
563
|
+
|
|
564
|
+
No alert fires on the first call for a given model or user (no history = no baseline). Alerts are cleared when `reset()` is called.
|
|
565
|
+
|
|
566
|
+
Anomaly webhook payload example:
|
|
567
|
+
```json
|
|
568
|
+
{ "text": "[tokenwatch] Anomaly: model \"gpt-4o\" call cost $0.5000 is 4.2x above 24h average ($0.0119)" }
|
|
569
|
+
```
|
|
570
|
+
|
|
571
|
+
`mode: 'always'` fires on every anomalous call instead of latching after the first.
|
|
572
|
+
|
|
545
573
|
---
|
|
546
574
|
|
|
547
575
|
## CLI
|
|
548
576
|
|
|
549
577
|
```bash
|
|
550
|
-
npx tokenwatch sync
|
|
551
|
-
npx tokenwatch prices
|
|
552
|
-
npx tokenwatch report
|
|
553
|
-
npx tokenwatch
|
|
578
|
+
npx tokenwatch sync # force update cached prices from remote
|
|
579
|
+
npx tokenwatch prices # list all models and current prices
|
|
580
|
+
npx tokenwatch report # show usage report from ~/.tokenwatch/usage.db
|
|
581
|
+
npx tokenwatch dashboard # open local web dashboard (default port: 4242)
|
|
582
|
+
npx tokenwatch dashboard --port 8080
|
|
583
|
+
npx tokenwatch help # show help
|
|
554
584
|
```
|
|
555
585
|
|
|
556
|
-
`tokenwatch report`
|
|
586
|
+
### `tokenwatch report`
|
|
587
|
+
|
|
588
|
+
Reads the local SQLite database and prints:
|
|
557
589
|
|
|
558
590
|
```
|
|
559
591
|
── tokenwatch report ──────────────────────────────
|
|
@@ -574,7 +606,101 @@ npx tokenwatch help # show help
|
|
|
574
606
|
───────────────────────────────────────────────────
|
|
575
607
|
```
|
|
576
608
|
|
|
577
|
-
|
|
609
|
+
### `tokenwatch dashboard`
|
|
610
|
+
|
|
611
|
+
Spins up a local web server and opens a dark-themed dashboard with real-time cost data:
|
|
612
|
+
|
|
613
|
+
- **Overview cards** — total cost, tokens, calls, burn rate per hour
|
|
614
|
+
- **Cost over time** — line chart bucketed by time (5min / 1h / 1day depending on filter)
|
|
615
|
+
- **Model breakdown** — doughnut chart + table with cost share per model
|
|
616
|
+
- **By user / feature** — collapsible tables, hidden when empty
|
|
617
|
+
- **Cost forecast** — projected daily and monthly spend based on recent burn rate
|
|
618
|
+
- **Time filter tabs** — 1h | 24h | 7d | 30d | All; updates chart and tables in real-time via SSE
|
|
619
|
+
|
|
620
|
+
Data updates automatically every 3 seconds without refreshing the page. Requires `storage: 'sqlite'` in your app and `better-sqlite3` installed. Zero external dependencies — pure Node.js HTTP server with Chart.js loaded from CDN.
|
|
621
|
+
|
|
622
|
+
---
|
|
623
|
+
|
|
624
|
+
## Production Usage
|
|
625
|
+
|
|
626
|
+
### Storage choice
|
|
627
|
+
|
|
628
|
+
| Setup | Recommended storage |
|
|
629
|
+
|---|---|
|
|
630
|
+
| Single process (monolith, lambda, single pod) | `'sqlite'` — zero config, persists across restarts |
|
|
631
|
+
| Multi-instance (Kubernetes, PaaS with ≥2 pods) | `PostgresStorage` / `MySQLStorage` / `MongoStorage` — shared, unified data |
|
|
632
|
+
| Ephemeral / testing | `'memory'` (default) — resets on restart |
|
|
633
|
+
|
|
634
|
+
### CI / test environments
|
|
635
|
+
|
|
636
|
+
Disable network calls and staleness warnings in CI:
|
|
637
|
+
|
|
638
|
+
```ts
|
|
639
|
+
const tracker = createTracker({
|
|
640
|
+
syncPrices: false, // skip remote price fetch — use bundled prices
|
|
641
|
+
warnIfStaleAfterHours: 0, // suppress staleness warning
|
|
642
|
+
})
|
|
643
|
+
```
|
|
644
|
+
|
|
645
|
+
### On-prem / air-gapped deployments
|
|
646
|
+
|
|
647
|
+
The daily GitHub Actions workflow updates `prices.json` and publishes a new npm package. Teams that cannot reach GitHub at runtime have two options:
|
|
648
|
+
|
|
649
|
+
1. **Pin and vendor** — copy `prices.json` from the installed package into your repo and commit it. Pass overrides via `customPrices` for any new models.
|
|
650
|
+
2. **Self-host the sync** — fork the `scripts/scrape-prices.mjs` script and run it on your own schedule, pointing to your internal registry.
|
|
651
|
+
|
|
652
|
+
Either way, set `syncPrices: false` so the library doesn't try to fetch from GitHub at runtime.
|
|
653
|
+
|
|
654
|
+
### Anomaly detection in production
|
|
655
|
+
|
|
656
|
+
Enable `anomalyDetection` to catch runaway agents or abuse early:
|
|
657
|
+
|
|
658
|
+
```ts
|
|
659
|
+
const tracker = createTracker({
|
|
660
|
+
storage: new PostgresStorage(pool),
|
|
661
|
+
anomalyDetection: {
|
|
662
|
+
multiplierThreshold: 3, // alert if a call costs 3x above the rolling average
|
|
663
|
+
webhookUrl: 'https://hooks.slack.com/...',
|
|
664
|
+
windowHours: 24, // baseline window (default: 24h)
|
|
665
|
+
},
|
|
666
|
+
})
|
|
667
|
+
```
|
|
668
|
+
|
|
669
|
+
---
|
|
670
|
+
|
|
671
|
+
## OpenTelemetry Exporter
|
|
672
|
+
|
|
673
|
+
Push tracked usage as metrics to any OTel-compatible backend (Datadog, Honeycomb, Grafana, New Relic, etc.) without changing your existing instrumentation:
|
|
674
|
+
|
|
675
|
+
```bash
|
|
676
|
+
npm install @opentelemetry/api
|
|
677
|
+
```
|
|
678
|
+
|
|
679
|
+
```ts
|
|
680
|
+
import { createTracker } from '@diogonzafe/tokenwatch'
|
|
681
|
+
import { OTelExporter } from '@diogonzafe/tokenwatch/exporters'
|
|
682
|
+
|
|
683
|
+
const tracker = createTracker({
|
|
684
|
+
exporter: new OTelExporter(), // uses the globally-registered MeterProvider
|
|
685
|
+
})
|
|
686
|
+
```
|
|
687
|
+
|
|
688
|
+
Four metrics are emitted per call, all with `model`, `session.id`, `user.id`, and `feature` attributes (optional fields omitted when absent):
|
|
689
|
+
|
|
690
|
+
| Metric | Type | Description |
|
|
691
|
+
|---|---|---|
|
|
692
|
+
| `tokenwatch.calls` | Counter | Number of LLM API calls |
|
|
693
|
+
| `tokenwatch.input_tokens` | Counter | Input tokens (includes cached + cache-creation) |
|
|
694
|
+
| `tokenwatch.output_tokens` | Counter | Output tokens |
|
|
695
|
+
| `tokenwatch.cost_usd` | Histogram | Cost per call in USD |
|
|
696
|
+
|
|
697
|
+
You must configure a `MeterProvider` before creating the exporter (e.g. using the OpenTelemetry SDK). `OTelExporter` has no compile-time dependency on `@opentelemetry/api` — it resolves it at runtime and throws a helpful error if the package is not installed.
|
|
698
|
+
|
|
699
|
+
Custom meter name:
|
|
700
|
+
|
|
701
|
+
```ts
|
|
702
|
+
new OTelExporter({ meterName: 'my-service' })
|
|
703
|
+
```
|
|
578
704
|
|
|
579
705
|
---
|
|
580
706
|
|
package/dist/adapters.d.cts
CHANGED
package/dist/adapters.d.ts
CHANGED