@diogonzafe/tokenwatch 0.5.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +137 -2
- package/dist/adapters.d.cts +1 -1
- package/dist/adapters.d.ts +1 -1
- package/dist/cli.js +467 -41
- package/dist/cli.js.map +1 -1
- package/dist/exporters.cjs +76 -0
- package/dist/exporters.cjs.map +1 -0
- package/dist/exporters.d.cts +60 -0
- package/dist/exporters.d.ts +60 -0
- package/dist/exporters.js +56 -0
- package/dist/exporters.js.map +1 -0
- package/dist/{index-CJKk1hHw.d.cts → index-D9xq0RNg.d.cts} +19 -1
- package/dist/{index-CJKk1hHw.d.ts → index-D9xq0RNg.d.ts} +19 -1
- package/dist/index.cjs +69 -3
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +2 -2
- package/dist/index.d.ts +2 -2
- package/dist/index.js +69 -3
- package/dist/index.js.map +1 -1
- package/dist/langchain.d.cts +1 -1
- package/dist/langchain.d.ts +1 -1
- package/package.json +13 -3
- package/prices.json +7 -1
- package/dist/cli.d.ts +0 -1
package/README.md
CHANGED
|
@@ -542,6 +542,34 @@ Budget webhook payload:
|
|
|
542
542
|
{ "text": "[tokenwatch] Budget alert: user \"user_123\" reached $1.0031 USD (threshold: $1)" }
|
|
543
543
|
```
|
|
544
544
|
|
|
545
|
+
### Cost anomaly detection
|
|
546
|
+
|
|
547
|
+
Alert when a single call is anomalously expensive compared to the recent average — catches runaway agents, infinite loops, or abusive usage:
|
|
548
|
+
|
|
549
|
+
```ts
|
|
550
|
+
const tracker = createTracker({
|
|
551
|
+
anomalyDetection: {
|
|
552
|
+
multiplierThreshold: 3, // alert if call cost > 3× rolling average
|
|
553
|
+
webhookUrl: 'https://hooks.slack.com/...',
|
|
554
|
+
windowHours: 24, // lookback window (default: 24h)
|
|
555
|
+
mode: 'once', // 'once' (default) or 'always'
|
|
556
|
+
},
|
|
557
|
+
})
|
|
558
|
+
```
|
|
559
|
+
|
|
560
|
+
Checks two axes independently for each tracked call:
|
|
561
|
+
- **Per-model** — compares the call's cost to the average of all prior calls for that model within `windowHours`
|
|
562
|
+
- **Per-user** — same check scoped to a specific user (only fires when `userId` is present)
|
|
563
|
+
|
|
564
|
+
No alert fires on the first call for a given model or user (no history = no baseline). Alerts are cleared when `reset()` is called.
|
|
565
|
+
|
|
566
|
+
Anomaly webhook payload example:
|
|
567
|
+
```json
|
|
568
|
+
{ "text": "[tokenwatch] Anomaly: model \"gpt-4o\" call cost $0.5000 is 4.2x above 24h average ($0.0119)" }
|
|
569
|
+
```
|
|
570
|
+
|
|
571
|
+
`mode: 'always'` fires on every anomalous call instead of latching after the first.
|
|
572
|
+
|
|
545
573
|
---
|
|
546
574
|
|
|
547
575
|
## CLI
|
|
@@ -549,12 +577,27 @@ Budget webhook payload:
|
|
|
549
577
|
```bash
|
|
550
578
|
npx tokenwatch sync # force update cached prices from remote
|
|
551
579
|
npx tokenwatch prices # list all models and current prices
|
|
552
|
-
npx tokenwatch report # show usage report
|
|
580
|
+
npx tokenwatch report # show usage report (SQLite default)
|
|
581
|
+
npx tokenwatch report --db postgres://user:pass@host:5432/db
|
|
553
582
|
npx tokenwatch dashboard # open local web dashboard (default port: 4242)
|
|
554
583
|
npx tokenwatch dashboard --port 8080
|
|
584
|
+
npx tokenwatch dashboard --db postgres://user:pass@host:5432/db
|
|
585
|
+
npx tokenwatch dashboard --db mysql://user:pass@host:3306/db
|
|
586
|
+
npx tokenwatch dashboard --db mongodb://user:pass@host:27017/db
|
|
555
587
|
npx tokenwatch help # show help
|
|
556
588
|
```
|
|
557
589
|
|
|
590
|
+
Both `report` and `dashboard` accept a `--db <url>` flag to connect directly to any database — no SQLite required. The URL protocol determines the adapter automatically:
|
|
591
|
+
|
|
592
|
+
| URL prefix | Adapter | Peer dep required |
|
|
593
|
+
|---|---|---|
|
|
594
|
+
| *(none)* | SQLite (`~/.tokenwatch/usage.db`) | `better-sqlite3` |
|
|
595
|
+
| `postgres://` or `postgresql://` | `PostgresStorage` | `pg` |
|
|
596
|
+
| `mysql://` | `MySQLStorage` | `mysql2` |
|
|
597
|
+
| `mongodb://` or `mongodb+srv://` | `MongoStorage` | `mongodb` |
|
|
598
|
+
|
|
599
|
+
If the required peer dep is not installed, a clear error message with the install command is shown.
|
|
600
|
+
|
|
558
601
|
### `tokenwatch report`
|
|
559
602
|
|
|
560
603
|
Reads the local SQLite database and prints:
|
|
@@ -589,7 +632,99 @@ Spins up a local web server and opens a dark-themed dashboard with real-time cos
|
|
|
589
632
|
- **Cost forecast** — projected daily and monthly spend based on recent burn rate
|
|
590
633
|
- **Time filter tabs** — 1h | 24h | 7d | 30d | All; updates chart and tables in real-time via SSE
|
|
591
634
|
|
|
592
|
-
Data updates automatically every 3 seconds without refreshing the page.
|
|
635
|
+
Data updates automatically every 3 seconds without refreshing the page. Zero external dependencies — pure Node.js HTTP server with Chart.js loaded from CDN.
|
|
636
|
+
|
|
637
|
+
Works with **any storage backend** via `--db <url>`:
|
|
638
|
+
|
|
639
|
+
```bash
|
|
640
|
+
tokenwatch dashboard # SQLite (default)
|
|
641
|
+
tokenwatch dashboard --db postgres://user:pass@host:5432/db # PostgreSQL
|
|
642
|
+
tokenwatch dashboard --db mysql://user:pass@host:3306/db # MySQL
|
|
643
|
+
tokenwatch dashboard --db mongodb://user:pass@host:27017/db # MongoDB
|
|
644
|
+
```
|
|
645
|
+
|
|
646
|
+
---
|
|
647
|
+
|
|
648
|
+
## Production Usage
|
|
649
|
+
|
|
650
|
+
### Storage choice
|
|
651
|
+
|
|
652
|
+
| Setup | Recommended storage |
|
|
653
|
+
|---|---|
|
|
654
|
+
| Single process (monolith, lambda, single pod) | `'sqlite'` — zero config, persists across restarts |
|
|
655
|
+
| Multi-instance (Kubernetes, PaaS with ≥2 pods) | `PostgresStorage` / `MySQLStorage` / `MongoStorage` — shared, unified data |
|
|
656
|
+
| Ephemeral / testing | `'memory'` (default) — resets on restart |
|
|
657
|
+
|
|
658
|
+
### CI / test environments
|
|
659
|
+
|
|
660
|
+
Disable network calls and staleness warnings in CI:
|
|
661
|
+
|
|
662
|
+
```ts
|
|
663
|
+
const tracker = createTracker({
|
|
664
|
+
syncPrices: false, // skip remote price fetch — use bundled prices
|
|
665
|
+
warnIfStaleAfterHours: 0, // suppress staleness warning
|
|
666
|
+
})
|
|
667
|
+
```
|
|
668
|
+
|
|
669
|
+
### On-prem / air-gapped deployments
|
|
670
|
+
|
|
671
|
+
The daily GitHub Actions workflow updates `prices.json` and publishes a new npm package. Teams that cannot reach GitHub at runtime have two options:
|
|
672
|
+
|
|
673
|
+
1. **Pin and vendor** — copy `prices.json` from the installed package into your repo and commit it. Pass overrides via `customPrices` for any new models.
|
|
674
|
+
2. **Self-host the sync** — fork the `scripts/scrape-prices.mjs` script and run it on your own schedule, pointing to your internal registry.
|
|
675
|
+
|
|
676
|
+
Either way, set `syncPrices: false` so the library doesn't try to fetch from GitHub at runtime.
|
|
677
|
+
|
|
678
|
+
### Anomaly detection in production
|
|
679
|
+
|
|
680
|
+
Enable `anomalyDetection` to catch runaway agents or abuse early:
|
|
681
|
+
|
|
682
|
+
```ts
|
|
683
|
+
const tracker = createTracker({
|
|
684
|
+
storage: new PostgresStorage(pool),
|
|
685
|
+
anomalyDetection: {
|
|
686
|
+
multiplierThreshold: 3, // alert if a call costs 3x above the rolling average
|
|
687
|
+
webhookUrl: 'https://hooks.slack.com/...',
|
|
688
|
+
windowHours: 24, // baseline window (default: 24h)
|
|
689
|
+
},
|
|
690
|
+
})
|
|
691
|
+
```
|
|
692
|
+
|
|
693
|
+
---
|
|
694
|
+
|
|
695
|
+
## OpenTelemetry Exporter
|
|
696
|
+
|
|
697
|
+
Push tracked usage as metrics to any OTel-compatible backend (Datadog, Honeycomb, Grafana, New Relic, etc.) without changing your existing instrumentation:
|
|
698
|
+
|
|
699
|
+
```bash
|
|
700
|
+
npm install @opentelemetry/api
|
|
701
|
+
```
|
|
702
|
+
|
|
703
|
+
```ts
|
|
704
|
+
import { createTracker } from '@diogonzafe/tokenwatch'
|
|
705
|
+
import { OTelExporter } from '@diogonzafe/tokenwatch/exporters'
|
|
706
|
+
|
|
707
|
+
const tracker = createTracker({
|
|
708
|
+
exporter: new OTelExporter(), // uses the globally-registered MeterProvider
|
|
709
|
+
})
|
|
710
|
+
```
|
|
711
|
+
|
|
712
|
+
Four metrics are emitted per call, all with `model`, `session.id`, `user.id`, and `feature` attributes (optional fields omitted when absent):
|
|
713
|
+
|
|
714
|
+
| Metric | Type | Description |
|
|
715
|
+
|---|---|---|
|
|
716
|
+
| `tokenwatch.calls` | Counter | Number of LLM API calls |
|
|
717
|
+
| `tokenwatch.input_tokens` | Counter | Input tokens (includes cached + cache-creation) |
|
|
718
|
+
| `tokenwatch.output_tokens` | Counter | Output tokens |
|
|
719
|
+
| `tokenwatch.cost_usd` | Histogram | Cost per call in USD |
|
|
720
|
+
|
|
721
|
+
You must configure a `MeterProvider` before creating the exporter (e.g. using the OpenTelemetry SDK). `OTelExporter` has no compile-time dependency on `@opentelemetry/api` — it resolves it at runtime and throws a helpful error if the package is not installed.
|
|
722
|
+
|
|
723
|
+
Custom meter name:
|
|
724
|
+
|
|
725
|
+
```ts
|
|
726
|
+
new OTelExporter({ meterName: 'my-service' })
|
|
727
|
+
```
|
|
593
728
|
|
|
594
729
|
---
|
|
595
730
|
|
package/dist/adapters.d.cts
CHANGED
package/dist/adapters.d.ts
CHANGED