tokenjam 0.5.3 → 0.5.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +51 -22
- package/bin/tj.js +76 -1
- package/package.json +22 -4
package/README.md
CHANGED
|
@@ -1,40 +1,69 @@
|
|
|
1
|
-
#
|
|
1
|
+
# tokenjam
|
|
2
|
+
|
|
3
|
+
<div align="center">
|
|
4
|
+
|
|
5
|
+
<img src="https://raw.githubusercontent.com/Metabuilder-Labs/tokenjam/main/docs/brand/tokenjam-repo-header.png" alt="TokenJam: token efficiency for AI agents. Reads your agent's telemetry, finds the waste, runs 100% local." width="760">
|
|
6
|
+
|
|
7
|
+
[](https://www.npmjs.com/package/tokenjam)
|
|
8
|
+
[](https://www.npmjs.com/package/tokenjam)
|
|
9
|
+
[](https://pypi.org/project/tokenjam/)
|
|
10
|
+
[](https://github.com/Metabuilder-Labs/tokenjam/blob/main/LICENSE)
|
|
11
|
+
|
|
12
|
+
</div>
|
|
13
|
+
|
|
14
|
+
TokenJam ingests telemetry data about your agents from a multitude of sources and provides you a quick and easy way to visualize and optimize cost so that you get the most out of the tokens you pay for. This package is the zero-install launcher: one command, no pip environment, no config.
|
|
2
15
|
|
|
3
16
|
```bash
|
|
4
17
|
npx tokenjam
|
|
5
18
|
```
|
|
6
19
|
|
|
7
|
-
|
|
8
|
-
and shows you **where your
|
|
9
|
-
Claude Code quota actually goes** — quota composition (re-reading context vs.
|
|
10
|
-
net-new work) plus a session timeline. No pip env, no daemon, no onboarding.
|
|
20
|
+
## What you get
|
|
11
21
|
|
|
12
|
-
|
|
13
|
-
package [`tokenjam`](https://pypi.org/project/tokenjam/) (command: `tj`). `npx tokenjam`
|
|
14
|
-
shells out to the first available Python runner:
|
|
22
|
+
Bare `npx tokenjam` reads the session logs you already have (Claude Code today; more sources land in the full CLI) and prints a 15-second, read-only report: quota composition (what share of your tokens went to re-reading history and context vs. net-new work) plus a session timeline. Nothing is installed, nothing is kept.
|
|
15
23
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
24
|
+
<div align="center">
|
|
25
|
+
<img src="https://raw.githubusercontent.com/Metabuilder-Labs/tokenjam/main/docs/assets/tj-quickstart-hero.png" alt="Sample npx tokenjam output: quota composition (95.9% re-reading context, 4.1% net-new work) and a session timeline table" width="620">
|
|
26
|
+
</div>
|
|
27
|
+
|
|
28
|
+
## Commands
|
|
19
29
|
|
|
20
|
-
All arguments pass straight through, so `
|
|
21
|
-
|
|
30
|
+
All arguments pass straight through to the Python CLI, so any `tj` subcommand and flag works here too.
|
|
31
|
+
|
|
32
|
+
| Command | What it does |
|
|
33
|
+
|---|---|
|
|
34
|
+
| `npx tokenjam` / `npx tokenjam quickstart` | Zero-install first run: quota composition and a session timeline from your existing session logs. |
|
|
35
|
+
| `npx tokenjam context` | Where your quota goes: re-read vs. net-new share, recurring inclusions, `/compact` candidates. |
|
|
36
|
+
| `npx tokenjam optimize` | Cost-saving candidates: model downsizing, cache opportunities, prompt trimming, workflow reuse, subagent right-sizing. |
|
|
37
|
+
| `npx tokenjam onboard` | Guided setup: writes a config, generates an ingest secret, and optionally installs the background daemon for live capture. |
|
|
22
38
|
|
|
23
39
|
## Go deeper
|
|
24
40
|
|
|
25
|
-
`npx tokenjam` is the no-setup front door.
|
|
26
|
-
dashboard, and the MCP server for Claude Code, install the full CLI and onboard:
|
|
41
|
+
`npx tokenjam` is the no-setup front door. One command sets up live capture, the local Lens dashboard, and the zero-token statusline:
|
|
27
42
|
|
|
28
43
|
```bash
|
|
29
|
-
pipx install tokenjam
|
|
30
|
-
tj onboard
|
|
44
|
+
npx tokenjam onboard # or: pipx install tokenjam && tj onboard
|
|
31
45
|
```
|
|
32
46
|
|
|
33
|
-
|
|
34
|
-
|
|
47
|
+
`tj onboard` asks how you use AI agents (Claude Code, Codex, or your own SDK/API agents) and wires the right path. For Claude Code and Codex that means backfilling recent history plus a statusline and hooks; restart and you're live. From there:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
tj optimize # cost-saving candidates from your actual usage
|
|
51
|
+
tj serve # open the Lens dashboard at http://127.0.0.1:7391/
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
- Full feature set, six analyzers, and Lens screenshots: [github.com/Metabuilder-Labs/tokenjam](https://github.com/Metabuilder-Labs/tokenjam)
|
|
55
|
+
- Product site and docs: [tokenjam.dev](https://tokenjam.dev)
|
|
56
|
+
|
|
57
|
+
## How the launcher works
|
|
58
|
+
|
|
59
|
+
This npm package is a thin launcher, not the real CLI. `npx tokenjam` shells out to the first available Python runner:
|
|
60
|
+
|
|
61
|
+
1. `uvx --from tokenjam tj …`
|
|
62
|
+
2. `pipx run --spec tokenjam tj …`
|
|
63
|
+
3. an already-installed `tj` on your `PATH`
|
|
64
|
+
|
|
65
|
+
The real CLI is the Python package [`tokenjam`](https://pypi.org/project/tokenjam/) (command: `tj`).
|
|
35
66
|
|
|
36
67
|
## Requirements
|
|
37
68
|
|
|
38
|
-
A Python runner
|
|
39
|
-
[`pipx`](https://pipx.pypa.io/). If neither is present, `npx tokenjam` prints install
|
|
40
|
-
guidance.
|
|
69
|
+
A Python runner: [`uv`](https://docs.astral.sh/uv/) (recommended) or [`pipx`](https://pipx.pypa.io/). If neither is present, `npx tokenjam` prints install guidance instead of failing silently.
|
package/bin/tj.js
CHANGED
|
@@ -16,10 +16,27 @@
|
|
|
16
16
|
* 3. `tj …` — an already-installed CLI on PATH
|
|
17
17
|
*
|
|
18
18
|
* If none are present we print actionable install guidance and exit non-zero.
|
|
19
|
+
*
|
|
20
|
+
* Note: tokenjam >=0.5.4 also ships a `tokenjam` console-script alias
|
|
21
|
+
* alongside `tj`, so bare `uvx tokenjam` / `pipx run tokenjam` work too. This
|
|
22
|
+
* wrapper keeps the explicit `--from tokenjam tj` / `--spec tokenjam tj` form
|
|
23
|
+
* below for back-compat with the 0.5.3 and earlier releases it also targets.
|
|
24
|
+
*
|
|
25
|
+
* Freshness (issue #111): `uv` reuses its cached tool environment and never
|
|
26
|
+
* re-resolves on its own, so a machine that first ran this wrapper on an old
|
|
27
|
+
* release keeps getting that release forever, even after newer ones hit
|
|
28
|
+
* PyPI. To avoid pinning stale versions indefinitely, the `uvx` branch passes
|
|
29
|
+
* `--refresh` at most once per 24h (tracked via a timestamp file — see
|
|
30
|
+
* `shouldRefresh`/`markRefreshed` below). `pipx run` isn't touched: its own
|
|
31
|
+
* cache already expires after ~14 days on its own. The installed-`tj` branch
|
|
32
|
+
* has no cache to go stale.
|
|
19
33
|
*/
|
|
20
34
|
"use strict";
|
|
21
35
|
|
|
22
36
|
const { spawnSync } = require("child_process");
|
|
37
|
+
const fs = require("fs");
|
|
38
|
+
const os = require("os");
|
|
39
|
+
const path = require("path");
|
|
23
40
|
|
|
24
41
|
// PyPI package name vs. command name differ (`tokenjam` ships the `tj` script),
|
|
25
42
|
// so ephemeral runners must be told the source package explicitly.
|
|
@@ -40,6 +57,60 @@ function runners() {
|
|
|
40
57
|
];
|
|
41
58
|
}
|
|
42
59
|
|
|
60
|
+
// --- uvx cache freshness (issue #111) -------------------------------------
|
|
61
|
+
//
|
|
62
|
+
// Only the `uvx` runner needs this: `uv` caches a resolved tool environment
|
|
63
|
+
// and reuses it forever unless told to `--refresh`, so a returning user
|
|
64
|
+
// silently keeps whatever version they first resolved. Tracked with a plain
|
|
65
|
+
// timestamp file rather than anything fancier — this wrapper is intentionally
|
|
66
|
+
// dependency-free (stdlib `fs`/`os`/`path` only).
|
|
67
|
+
|
|
68
|
+
const REFRESH_INTERVAL_MS = 24 * 60 * 60 * 1000; // 24h
|
|
69
|
+
|
|
70
|
+
function refreshCacheDir() {
|
|
71
|
+
const xdgCacheHome = process.env.XDG_CACHE_HOME;
|
|
72
|
+
const base =
|
|
73
|
+
xdgCacheHome && xdgCacheHome.trim()
|
|
74
|
+
? xdgCacheHome
|
|
75
|
+
: path.join(os.homedir(), ".cache");
|
|
76
|
+
return path.join(base, "tokenjam-npx");
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
function refreshTimestampPath() {
|
|
80
|
+
return path.join(refreshCacheDir(), "last-refresh");
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
// True if it's been >24h (or we've never refreshed / can't tell). Fails
|
|
84
|
+
// open on any fs error — an unwritable/unreadable cache dir must never
|
|
85
|
+
// break the wrapper, it just means we skip the freshness nudge this run.
|
|
86
|
+
function shouldRefresh() {
|
|
87
|
+
try {
|
|
88
|
+
const stat = fs.statSync(refreshTimestampPath());
|
|
89
|
+
return Date.now() - stat.mtimeMs > REFRESH_INTERVAL_MS;
|
|
90
|
+
} catch {
|
|
91
|
+
return true; // no timestamp yet (or unreadable) => treat as stale
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
// Called only AFTER `uvx --refresh` has actually returned with a zero exit
|
|
96
|
+
// status (not before we spawn it, and not on failure). If the refresh's
|
|
97
|
+
// download is interrupted partway through (network drop, Ctrl-C, OOM kill),
|
|
98
|
+
// spawnSync never returns normally, this never runs. If it returns but uv
|
|
99
|
+
// exits non-zero (PyPI unreachable, partial download), the caller also
|
|
100
|
+
// skips this call. Either way the *next* invocation still sees a
|
|
101
|
+
// stale/missing timestamp and retries `--refresh`. Writing the timestamp up
|
|
102
|
+
// front, or unconditionally on return, would mark the cache "fresh" even
|
|
103
|
+
// though that refresh never completed, silently pinning a broken/partial
|
|
104
|
+
// environment for a full 24h. Best-effort/fail-open: swallow fs errors.
|
|
105
|
+
function markRefreshed() {
|
|
106
|
+
try {
|
|
107
|
+
fs.mkdirSync(refreshCacheDir(), { recursive: true });
|
|
108
|
+
fs.writeFileSync(refreshTimestampPath(), String(Date.now()));
|
|
109
|
+
} catch {
|
|
110
|
+
// fail open — worst case we just try to refresh again next run
|
|
111
|
+
}
|
|
112
|
+
}
|
|
113
|
+
|
|
43
114
|
function main() {
|
|
44
115
|
// Bare `npx tokenjam` IS the zero-install first run — route it to
|
|
45
116
|
// `tj quickstart` (the quota report the docs promise). The branded home
|
|
@@ -51,10 +122,14 @@ function main() {
|
|
|
51
122
|
|
|
52
123
|
for (const { bin, prefix } of runners()) {
|
|
53
124
|
if (!has(bin)) continue;
|
|
54
|
-
const
|
|
125
|
+
const isUvx = bin === "uvx";
|
|
126
|
+
const doRefresh = isUvx && shouldRefresh();
|
|
127
|
+
const args = doRefresh ? ["--refresh", ...prefix] : prefix;
|
|
128
|
+
const result = spawnSync(bin, [...args, ...passthrough], {
|
|
55
129
|
stdio: "inherit",
|
|
56
130
|
});
|
|
57
131
|
if (result.error) continue; // try the next runner on spawn failure
|
|
132
|
+
if (doRefresh && result.status === 0) markRefreshed();
|
|
58
133
|
process.exit(result.status === null ? 1 : result.status);
|
|
59
134
|
}
|
|
60
135
|
|
package/package.json
CHANGED
|
@@ -1,8 +1,25 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tokenjam",
|
|
3
|
-
"version": "0.5.
|
|
4
|
-
"description": "Zero-install launcher for TokenJam (tj)
|
|
5
|
-
"keywords": [
|
|
3
|
+
"version": "0.5.6",
|
|
4
|
+
"description": "Zero-install launcher for TokenJam (tj): npx tokenjam runs the Python CLI via uvx/pipx and prints where your AI agent's token quota actually goes, no setup required.",
|
|
5
|
+
"keywords": [
|
|
6
|
+
"claude-code",
|
|
7
|
+
"ccusage",
|
|
8
|
+
"tokens",
|
|
9
|
+
"cost",
|
|
10
|
+
"llm",
|
|
11
|
+
"agents",
|
|
12
|
+
"observability",
|
|
13
|
+
"tokenjam",
|
|
14
|
+
"anthropic",
|
|
15
|
+
"claude",
|
|
16
|
+
"token-usage",
|
|
17
|
+
"cost-tracking",
|
|
18
|
+
"cli",
|
|
19
|
+
"statusline",
|
|
20
|
+
"opentelemetry",
|
|
21
|
+
"agent-observability"
|
|
22
|
+
],
|
|
6
23
|
"homepage": "https://tokenjam.dev",
|
|
7
24
|
"bugs": "https://github.com/Metabuilder-Labs/tokenjam/issues",
|
|
8
25
|
"license": "MIT",
|
|
@@ -12,7 +29,8 @@
|
|
|
12
29
|
"directory": "npm-wrapper"
|
|
13
30
|
},
|
|
14
31
|
"bin": {
|
|
15
|
-
"tj": "bin/tj.js"
|
|
32
|
+
"tj": "bin/tj.js",
|
|
33
|
+
"tokenjam": "bin/tj.js"
|
|
16
34
|
},
|
|
17
35
|
"files": [
|
|
18
36
|
"bin/tj.js",
|