@ramtinj95/opencode-tokenscope 1.4.0 → 1.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +65 -204
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,11 +1,53 @@
|
|
|
1
1
|
# OpenCode-Tokenscope, Token Analyzer Plugin
|
|
2
2
|
|
|
3
|
+
[](https://www.npmjs.com/package/@ramtinj95/opencode-tokenscope)
|
|
4
|
+
|
|
3
5
|
> Comprehensive token usage analysis and cost tracking for OpenCode AI sessions
|
|
4
6
|
|
|
5
7
|
Track and optimize your token usage across system prompts, user messages, tool outputs, and more. Get detailed breakdowns, accurate cost estimates, and visual insights for your AI development workflow.
|
|
6
8
|
|
|
7
9
|
## Installation
|
|
8
10
|
|
|
11
|
+
### Option 1: npm (Recommended)
|
|
12
|
+
|
|
13
|
+
1. **Install globally:**
|
|
14
|
+
```bash
|
|
15
|
+
npm install -g @ramtinj95/opencode-tokenscope
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
2. **Add to your `opencode.json`** (create one in your project root or `~/.config/opencode/opencode.json` for global config):
|
|
19
|
+
```json
|
|
20
|
+
{
|
|
21
|
+
"$schema": "https://opencode.ai/config.json",
|
|
22
|
+
"plugin": ["@ramtinj95/opencode-tokenscope"]
|
|
23
|
+
}
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
3. **Create the `/tokenscope` command** by creating `~/.config/opencode/command/tokenscope.md`:
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
mkdir -p ~/.config/opencode/command
|
|
30
|
+
cat > ~/.config/opencode/command/tokenscope.md << 'EOF'
|
|
31
|
+
---
|
|
32
|
+
description: Analyze token usage across the current session with detailed breakdowns by category
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
Call the tokenscope tool directly without delegating to other agents.
|
|
36
|
+
Then cat the token-usage-output.txt. DONT DO ANYTHING ELSE WITH THE OUTPUT.
|
|
37
|
+
EOF
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
4. **Restart OpenCode** and run `/tokenscope`
|
|
41
|
+
|
|
42
|
+
To always get the latest version automatically, use `@latest`:
|
|
43
|
+
```json
|
|
44
|
+
{
|
|
45
|
+
"plugin": ["@ramtinj95/opencode-tokenscope@latest"]
|
|
46
|
+
}
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
### Option 2: Install Script
|
|
50
|
+
|
|
9
51
|
```bash
|
|
10
52
|
curl -sSL https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/install.sh | bash
|
|
11
53
|
```
|
|
@@ -14,6 +56,23 @@ Then restart OpenCode and run `/tokenscope`
|
|
|
14
56
|
|
|
15
57
|
## Updating
|
|
16
58
|
|
|
59
|
+
### If installed via npm:
|
|
60
|
+
|
|
61
|
+
| Config in `opencode.json` | Behavior |
|
|
62
|
+
|---------------------------|----------|
|
|
63
|
+
| `"@ramtinj95/opencode-tokenscope"` | Uses the version installed at install time. **Never auto-updates.** |
|
|
64
|
+
| `"@ramtinj95/opencode-tokenscope@latest"` | Fetches latest version **every time OpenCode starts**. |
|
|
65
|
+
| `"@ramtinj95/opencode-tokenscope@1.4.0"` | Pins to exact version 1.4.0. Never updates. |
|
|
66
|
+
|
|
67
|
+
To manually update:
|
|
68
|
+
```bash
|
|
69
|
+
npm update -g @ramtinj95/opencode-tokenscope
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Or use `@latest` in your `opencode.json` to auto-update on OpenCode restart.
|
|
73
|
+
|
|
74
|
+
### If installed via script:
|
|
75
|
+
|
|
17
76
|
**Option 1: Local script** (if you have the plugin installed)
|
|
18
77
|
```bash
|
|
19
78
|
bash ~/.config/opencode/plugin/install.sh --update
|
|
@@ -273,32 +332,9 @@ SUMMARY
|
|
|
273
332
|
|
|
274
333
|
**Free/Open models** are marked with zero pricing.
|
|
275
334
|
|
|
276
|
-
##
|
|
277
|
-
|
|
278
|
-
### Add New Model Pricing
|
|
279
|
-
|
|
280
|
-
Edit `~/.config/opencode/plugin/models.json`:
|
|
281
|
-
|
|
282
|
-
```json
|
|
283
|
-
{
|
|
284
|
-
"your-model-name": {
|
|
285
|
-
"input": 1.50,
|
|
286
|
-
"output": 5.00,
|
|
287
|
-
"cacheWrite": 0.50,
|
|
288
|
-
"cacheRead": 0.10
|
|
289
|
-
}
|
|
290
|
-
}
|
|
291
|
-
```
|
|
292
|
-
|
|
293
|
-
Save the file and restart OpenCode. The plugin will automatically use the new pricing.
|
|
294
|
-
|
|
295
|
-
### Update Existing Model Pricing
|
|
296
|
-
|
|
297
|
-
Simply edit the values in `models.json` and restart OpenCode. No code changes needed!
|
|
298
|
-
|
|
299
|
-
### Configure Analysis Features
|
|
335
|
+
## Configuration
|
|
300
336
|
|
|
301
|
-
|
|
337
|
+
The plugin includes a `tokenscope-config.json` file with these defaults:
|
|
302
338
|
|
|
303
339
|
```json
|
|
304
340
|
{
|
|
@@ -309,205 +345,30 @@ Edit `~/.config/opencode/plugin/tokenscope-config.json` to enable/disable sectio
|
|
|
309
345
|
}
|
|
310
346
|
```
|
|
311
347
|
|
|
312
|
-
Set any option to `false` to hide that section from the output.
|
|
313
|
-
|
|
314
|
-
## How It Works
|
|
315
|
-
|
|
316
|
-
### System Prompt Inference
|
|
317
|
-
OpenCode doesn't expose system prompts in the session messages API. The plugin intelligently infers them using:
|
|
318
|
-
|
|
319
|
-
```
|
|
320
|
-
System Tokens = (API Input + Cache Read) - (User Tokens + Tool Tokens)
|
|
321
|
-
```
|
|
322
|
-
|
|
323
|
-
This works because the API input includes everything sent to the model.
|
|
324
|
-
|
|
325
|
-
### Dual Tracking
|
|
326
|
-
- **Current Context**: Uses the most recent API call with non-zero tokens (matches TUI)
|
|
327
|
-
- **Session Total**: Aggregates all API calls for accurate billing
|
|
328
|
-
|
|
329
|
-
### Subagent Analysis
|
|
330
|
-
The plugin uses OpenCode's session API to:
|
|
331
|
-
1. Fetch all child sessions spawned by the Task tool
|
|
332
|
-
2. Recursively analyze nested subagents (subagents can spawn their own subagents)
|
|
333
|
-
3. Aggregate tokens, costs, and API call counts
|
|
334
|
-
4. Calculate estimated costs using the same pricing as the main session
|
|
335
|
-
|
|
336
|
-
### Model Name Normalization
|
|
337
|
-
Automatically handles `provider/model` format (e.g., `qwen/qwen3-coder` → `qwen3-coder`)
|
|
338
|
-
|
|
339
|
-
## Understanding the Numbers
|
|
340
|
-
|
|
341
|
-
### Current Context vs Session Total
|
|
342
|
-
|
|
343
|
-
- **Current Context**: What's in your context window right now
|
|
344
|
-
- Based on most recent API call
|
|
345
|
-
- Used to understand current memory usage
|
|
346
|
-
|
|
347
|
-
- **Session Total**: All tokens processed in this session
|
|
348
|
-
- Sum of all API calls in the main session
|
|
349
|
-
- What you're billed for (main session only)
|
|
350
|
-
- Used for cost calculation
|
|
351
|
-
|
|
352
|
-
### Subagent Totals
|
|
353
|
-
|
|
354
|
-
When using the Task tool, OpenCode spawns subagent sessions. These are tracked separately:
|
|
355
|
-
|
|
356
|
-
- **Subagent Tokens**: Combined tokens from all child sessions
|
|
357
|
-
- **Subagent API Calls**: Total API calls made by all subagents
|
|
358
|
-
- **Grand Total**: Main session + all subagents combined
|
|
359
|
-
|
|
360
|
-
### Cache Tokens
|
|
361
|
-
|
|
362
|
-
- **Cache Read**: Tokens retrieved from cache (discounted rate ~90% off)
|
|
363
|
-
- **Cache Write**: Tokens written to cache (slight premium ~25% more)
|
|
364
|
-
- **Note**: Cache write is a billing charge, not additional context tokens
|
|
348
|
+
Set any option to `false` to hide that section from the output.
|
|
365
349
|
|
|
366
350
|
## Troubleshooting
|
|
367
351
|
|
|
368
|
-
###
|
|
369
|
-
|
|
370
|
-
Run the installer:
|
|
371
|
-
```bash
|
|
372
|
-
curl -sSL https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/install.sh | bash
|
|
373
|
-
```
|
|
374
|
-
|
|
375
|
-
### Command Not Appearing
|
|
352
|
+
### Command `/tokenscope` Not Appearing
|
|
376
353
|
|
|
377
354
|
1. Verify `tokenscope.md` exists:
|
|
378
355
|
```bash
|
|
379
356
|
ls ~/.config/opencode/command/tokenscope.md
|
|
380
357
|
```
|
|
381
|
-
2.
|
|
382
|
-
3.
|
|
358
|
+
2. If missing, create it (see Installation step 3)
|
|
359
|
+
3. Restart OpenCode completely
|
|
383
360
|
|
|
384
361
|
### Wrong Token Counts
|
|
385
362
|
|
|
386
363
|
The plugin uses API telemetry (ground truth). If counts seem off:
|
|
387
364
|
- **Expected ~2K difference from TUI**: Plugin analyzes before its own response is added
|
|
388
365
|
- **Model detection**: Check that the model name is recognized in the output
|
|
389
|
-
- **Tokenizer not installed**: Re-run the installer
|
|
390
|
-
|
|
391
|
-
### New Model Not Showing Correct Pricing
|
|
392
|
-
|
|
393
|
-
1. Check if model exists in `models.json`
|
|
394
|
-
2. Try exact match or prefix match (e.g., `claude-sonnet-4` matches `claude-sonnet-4-20250514`)
|
|
395
|
-
3. Add entry to `models.json` if missing
|
|
396
|
-
4. Restart OpenCode after editing `models.json`
|
|
397
|
-
|
|
398
|
-
### Plugin Fails to Load
|
|
399
|
-
|
|
400
|
-
1. Validate JSON syntax:
|
|
401
|
-
```bash
|
|
402
|
-
cd ~/.config/opencode/plugin
|
|
403
|
-
node -e "JSON.parse(require('fs').readFileSync('models.json', 'utf8'))"
|
|
404
|
-
```
|
|
405
|
-
2. Check for trailing commas or syntax errors
|
|
406
|
-
3. Plugin falls back to default pricing if file is invalid
|
|
407
|
-
|
|
408
|
-
## Architecture
|
|
409
|
-
|
|
410
|
-
### File Structure
|
|
411
|
-
|
|
412
|
-
```
|
|
413
|
-
plugin/
|
|
414
|
-
├── tokenscope.ts # Main entry point - Plugin export
|
|
415
|
-
├── tokenscope-lib/
|
|
416
|
-
│ ├── types.ts # All interfaces and type definitions
|
|
417
|
-
│ ├── config.ts # Constants, model maps, pricing loader
|
|
418
|
-
│ ├── tokenizer.ts # TokenizerManager class
|
|
419
|
-
│ ├── analyzer.ts # ModelResolver, ContentCollector, TokenAnalysisEngine
|
|
420
|
-
│ ├── cost.ts # CostCalculator class
|
|
421
|
-
│ ├── subagent.ts # SubagentAnalyzer class
|
|
422
|
-
│ ├── formatter.ts # OutputFormatter class
|
|
423
|
-
│ └── context.ts # ContextAnalyzer class (context breakdown, tool estimates, cache efficiency)
|
|
424
|
-
├── models.json # Pricing data for 41+ models
|
|
425
|
-
├── tokenscope-config.json # Feature toggles configuration
|
|
426
|
-
├── package.json # Plugin metadata
|
|
427
|
-
└── install.sh # Installation script
|
|
428
|
-
```
|
|
429
|
-
|
|
430
|
-
### Core Components
|
|
431
|
-
|
|
432
|
-
1. **TokenizerManager** (`tokenscope-lib/tokenizer.ts`): Loads and caches tokenizers (tiktoken, transformers)
|
|
433
|
-
2. **ModelResolver** (`tokenscope-lib/analyzer.ts`): Detects model and selects appropriate tokenizer
|
|
434
|
-
3. **ContentCollector** (`tokenscope-lib/analyzer.ts`): Extracts content from session messages, including tool call counts
|
|
435
|
-
4. **TokenAnalysisEngine** (`tokenscope-lib/analyzer.ts`): Counts tokens and applies API telemetry adjustments
|
|
436
|
-
5. **CostCalculator** (`tokenscope-lib/cost.ts`): Calculates costs from pricing database with cache-aware pricing
|
|
437
|
-
6. **SubagentAnalyzer** (`tokenscope-lib/subagent.ts`): Recursively fetches and analyzes child sessions from Task tool calls
|
|
438
|
-
7. **ContextAnalyzer** (`tokenscope-lib/context.ts`): Analyzes context breakdown, tool schema estimates, and cache efficiency
|
|
439
|
-
8. **OutputFormatter** (`tokenscope-lib/formatter.ts`): Generates visual reports with charts and summaries
|
|
440
366
|
|
|
441
367
|
## Privacy & Security
|
|
442
368
|
|
|
443
369
|
- **All processing is local**: No session data sent to external services
|
|
444
|
-
- **Tokenizers from official sources**:
|
|
445
|
-
- OpenAI tokenizers: npm registry
|
|
446
|
-
- Transformers: Hugging Face Hub
|
|
447
370
|
- **Open source**: Audit the code yourself
|
|
448
371
|
|
|
449
|
-
## Performance
|
|
450
|
-
|
|
451
|
-
- **Fast**: Tokenizers cached after first load
|
|
452
|
-
- **Parallel**: Categories processed concurrently
|
|
453
|
-
- **Efficient**: Only analyzes on demand
|
|
454
|
-
- **First-run download**: Transformers models download on demand (5-50MB per model)
|
|
455
|
-
- **Subsequent runs**: Instant (uses cache)
|
|
456
|
-
|
|
457
|
-
## Manual Installation
|
|
458
|
-
|
|
459
|
-
<details>
|
|
460
|
-
<summary>Click to expand manual installation steps</summary>
|
|
461
|
-
|
|
462
|
-
### Requirements
|
|
463
|
-
- OpenCode installed (`~/.config/opencode` directory exists)
|
|
464
|
-
- npm (for tokenizer dependencies)
|
|
465
|
-
- ~50MB disk space (for tokenizer models)
|
|
466
|
-
|
|
467
|
-
### Installation Steps
|
|
468
|
-
|
|
469
|
-
1. **Navigate to OpenCode config**:
|
|
470
|
-
```bash
|
|
471
|
-
cd ~/.config/opencode
|
|
472
|
-
```
|
|
473
|
-
|
|
474
|
-
2. **Download plugin files**:
|
|
475
|
-
```bash
|
|
476
|
-
mkdir -p plugin/tokenscope-lib
|
|
477
|
-
cd plugin
|
|
478
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope.ts
|
|
479
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/models.json
|
|
480
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/package.json
|
|
481
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-config.json
|
|
482
|
-
cd tokenscope-lib
|
|
483
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-lib/types.ts
|
|
484
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-lib/config.ts
|
|
485
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-lib/tokenizer.ts
|
|
486
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-lib/analyzer.ts
|
|
487
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-lib/cost.ts
|
|
488
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-lib/subagent.ts
|
|
489
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-lib/formatter.ts
|
|
490
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/plugin/tokenscope-lib/context.ts
|
|
491
|
-
```
|
|
492
|
-
|
|
493
|
-
3. **Download command file**:
|
|
494
|
-
```bash
|
|
495
|
-
cd ../../command
|
|
496
|
-
curl -O https://raw.githubusercontent.com/ramtinJ95/opencode-tokenscope/main/command/tokenscope.md
|
|
497
|
-
```
|
|
498
|
-
|
|
499
|
-
4. **Install dependencies**:
|
|
500
|
-
```bash
|
|
501
|
-
cd ../plugin
|
|
502
|
-
npm install js-tiktoken@1.0.15 @huggingface/transformers@3.1.2
|
|
503
|
-
```
|
|
504
|
-
|
|
505
|
-
5. **Restart OpenCode**
|
|
506
|
-
|
|
507
|
-
6. **Test**: Run `/tokenscope` in any session
|
|
508
|
-
|
|
509
|
-
</details>
|
|
510
|
-
|
|
511
372
|
## Contributing
|
|
512
373
|
|
|
513
374
|
Contributions welcome! Ideas for enhancement:
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://json.schemastore.org/package.json",
|
|
3
3
|
"name": "@ramtinj95/opencode-tokenscope",
|
|
4
|
-
"version": "1.4.
|
|
4
|
+
"version": "1.4.2",
|
|
5
5
|
"description": "OpenCode plugin for detailed token usage analysis with breakdowns by category, visual charts, and subagent cost tracking",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"main": "./dist/tokenscope.js",
|