tokenometer 0.0.3 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,6 +1,11 @@
1
1
  # tokenometer
2
2
 
3
- > Empirical token-cost benchmarking for LLM prompts. Tells you what your prompt actually costs across Claude, GPT-4o, and Gemini, in every format.
3
+ [![npm tokenometer](https://img.shields.io/npm/v/tokenometer.svg?label=tokenometer)](https://www.npmjs.com/package/tokenometer)
4
+ [![License: MIT](https://img.shields.io/github/license/faraa2m/tokenometer.svg)](https://github.com/faraa2m/tokenometer/blob/main/LICENSE)
5
+
6
+ > Empirical token-cost + latency benchmarking for LLM prompts. Tells you what your prompt actually costs and how fast each provider responds across Claude, GPT-4o, Gemini, Mistral, and Cohere — in every format.
7
+
8
+ See the [root README](https://github.com/faraa2m/tokenometer#readme) for findings, methodology, and the full project overview.
4
9
 
5
10
  [**Live playground: tokenometer.vercel.app**](https://tokenometer.vercel.app) · [Source](https://github.com/faraa2m/tokenometer) · MIT
6
11
 
@@ -20,36 +25,152 @@ Cheapest: gpt-4o as json ($0.000192)
20
25
  Priciest: claude-opus-4-7 as yaml ($0.001260, 6.74x more)
21
26
  ```
22
27
 
23
- A leading `~` marks an approximate count (offline mode for Claude / Gemini, since neither vendor publishes a public tokenizer).
28
+ A leading `~` marks an approximate count (offline mode for Claude / Gemini / Mistral-Tekken / Cohere, since none of those vendors publishes a public production tokenizer that ships in JS).
29
+
30
+ ## Flags
31
+
32
+ | Flag | Default | Notes |
33
+ |---|---|---|
34
+ | `--model <id[,id…]>` | `claude-opus-4-7` (or auto-detected) | Any registered model id (63 across 5 providers). |
35
+ | `--format <fmt[,fmt…]>` | `json,yaml,xml,markdown,text` | Subset of supported formats. |
36
+ | `--output <fmt>` | `table` | `table` \| `json` \| `sarif`. |
37
+ | `--by-file` | _off_ | Append a per-file token/USD table (multi-file only). |
38
+ | `--image <path>` | _none_ | Add vision-token cost for the image (repeatable). |
39
+ | `--config <path>` | _none_ | Load this exact config file (skips walk-up). |
40
+ | `--no-config` | _off_ | Skip `.tokenometer.yml` loading entirely. |
41
+ | `--empirical` | _off_ | Use provider `countTokens` APIs (free, exact). |
42
+ | `--latency` | _off_ | Measure real generation latency (TTFT, total ms, tokens/sec). Implies `--empirical`. |
43
+ | `--latency-trials <n>` | `3` | Trials per cell when `--latency` is set (1–10). |
44
+ | `--max-spend <usd>` | `0.05` (or `0.25` with `--latency`) | Hard ceiling for empirical / latency mode. |
45
+ | `--offline` | _off_ | Force offline path (overrides `--empirical`). |
46
+ | `-h`, `--help` | | Print help. |
47
+ | `-v`, `--version` | | Print version. |
48
+
49
+ ```
50
+ tokenometer <file> [options]
51
+ echo "prompt" | tokenometer - [options]
52
+ ```
53
+
54
+ ## Models supported
55
+
56
+ 63 models across 5 providers. Run `tokenometer --help` for the full list at runtime, or browse the [Cost Atlas](https://tokenometer.vercel.app/models) for sortable per-model pages.
57
+
58
+ | Provider | Examples | Offline tokenizer | Empirical |
59
+ |---|---|---|---|
60
+ | Anthropic | `claude-opus-4-7`, `claude-sonnet-4-6`, `claude-haiku-4-5`, Claude 3.x family | `gpt-tokenizer` `cl100k_base` (approximate) | `messages.countTokens` (free, exact) |
61
+ | OpenAI | `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`, `gpt-3.5-turbo`, `o1` family | `gpt-tokenizer` `o200k_base` (exact) | same `o200k_base` (matches production) |
62
+ | Google | `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-1.5-pro`, `gemini-1.5-flash` | `chars / 4` (approximate) | `model.countTokens` (free, exact) |
63
+ | Mistral (19 models) | `open-mistral-7b`, `open-mixtral-8x22b`, `mistral-large-latest`, `codestral-latest`, `mistral-nemo`, `pixtral-large-latest`, `mistral-medium-2505`, `magistral-small`, `ministral-3b-latest`, `devstral-small-2505` | `mistral-tokenizer-js` for SentencePiece V1/V2/V3 (exact); `chars/4` for Tekken (approximate) | unsupported (no public token-count API) |
64
+ | Cohere | `command-r`, `command-r-plus` | `chars / 4` (approximate) | `POST /v1/tokenize` (free, exact, requires `COHERE_API_KEY`) |
65
+
66
+ Pricing comes from the [`tokenlens`](https://www.npmjs.com/package/tokenlens) registry with a small set of local overrides for bleeding-edge models. Cohere pricing lives entirely in `LOCAL_OVERRIDES` because `@tokenlens/models` doesn't yet ship a Cohere catalog at v1.3.0.
24
67
 
25
68
  ## Empirical mode
26
69
 
27
- For exact, vendor-billed counts on Claude and Gemini, set the right env var and pass `--empirical`. The tool calls the providers' free `countTokens` endpoints — no charge.
70
+ For exact, vendor-billed counts on Claude, Gemini, and Cohere, set the right env var and pass `--empirical`. The tool calls each provider's free `countTokens`-equivalent endpoint — no charge.
28
71
 
29
72
  ```bash
30
- ANTHROPIC_API_KEY=… GOOGLE_API_KEY=… \
31
- npx tokenometer ./prompt.md --empirical
73
+ ANTHROPIC_API_KEY=… GOOGLE_API_KEY=… COHERE_API_KEY=… \
74
+ npx tokenometer ./prompt.md --empirical --model claude-opus-4-7,gemini-2.5-pro,command-r-plus
32
75
  ```
33
76
 
34
- ## Why not just `tiktoken`?
77
+ OpenAI's empirical path uses tiktoken `o200k_base` locally — that encoding matches OpenAI's production count exactly, so no API call is needed. Mistral has no public token-count endpoint; the offline `mistral-tokenizer-js` path is used regardless.
35
78
 
36
- `tiktoken`'s `cl100k_base` (the encoding most "Claude tokenizer" libraries fall back on) **under-counts Opus 4.7 by a median of +62%** across a 10-prompt benchmark. Sonnet 4.6 and Haiku 4.5 are closer (~17%). Format choice is a wash. Model choice swings cost by 12×. See [README](https://github.com/faraa2m/tokenometer#findings-anthropic-n150-cells-across-10-prompt-shapes) for the dataset findings.
79
+ ## Auto provider detection
37
80
 
38
- ## Flags
81
+ When `--model` is omitted, tokenometer picks a default based on which provider key is set in your environment:
82
+
83
+ - `ANTHROPIC_API_KEY` only → `claude-opus-4-7`
84
+ - `OPENAI_API_KEY` only → `gpt-4o`
85
+ - `GOOGLE_API_KEY` / `GEMINI_API_KEY` only → first known `gemini-*` model (falls back to `gemini-2.5-pro`)
86
+ - `MISTRAL_API_KEY` only → first known `mistral-*` model
87
+ - `COHERE_API_KEY` only → `command-r-plus`
88
+ - Multiple keys set → falls back to `claude-opus-4-7` and prints a stderr note. Pass `--model` to disambiguate.
89
+ - No keys set → existing default (`claude-opus-4-7`).
90
+
91
+ This means `npx tokenometer prompt.md` does the right thing in any of those environments without you having to remember model names.
92
+
93
+ ## `.tokenometer.yml` config
94
+
95
+ Drop a `.tokenometer.yml` (or `.yaml`) at the project root and tokenometer will pick it up automatically (walks up from the cwd, stopping at `.git`):
39
96
 
97
+ ```yaml
98
+ models: [claude-opus-4-7, gpt-4o, mistral-large-latest]
99
+ formats: [json, yaml, markdown]
100
+ paths: [prompts/**/*.md]
101
+ budgets:
102
+ total: 0.50
103
+ per-file: 0.10
40
104
  ```
41
- tokenometer <file> [options]
42
- echo "prompt" | tokenometer - [options]
43
105
 
44
- --model <id[,id…]> Default: claude-opus-4-7
45
- --format <fmt[,fmt…]> Default: all (json,yaml,xml,markdown,text)
46
- --empirical Use provider countTokens APIs (free, exact)
47
- --max-spend <usd> Hard ceiling for empirical mode (default 0.05)
48
- --offline Force offline (overrides --empirical)
49
- -h, --help
50
- -v, --version
106
+ User-passed CLI flags always win over config defaults. Use `--config <path>` to load an explicit file (skips the walk-up). Use `--no-config` to skip config loading entirely.
107
+
108
+ ## Output formats
109
+
110
+ The `--output` flag picks the *display* format (separate from `--format`, which controls how the prompt body is converted before tokenization):
111
+
112
+ - `--output table` (default) — the human-readable per-cell table you've been seeing.
113
+ - `--output json` — emits a `TokenometerResult` JSON shape: `{ files: [{ path, results: [...] }] }`. One entry per input file. Pipe to `jq` for filtering.
114
+ - `--output sarif` — emits SARIF 2.1.0 with one result per (file, model, format) cell. Drop the file into GitHub Code Scanning or any SARIF viewer.
115
+
116
+ ```bash
117
+ npx tokenometer ./prompt.md --output sarif > tokenometer.sarif
118
+ npx tokenometer ./prompt.md --output json | jq '.files[].results | map(.inputCost) | add'
51
119
  ```
52
120
 
121
+ ## Latency
122
+
123
+ `--latency` measures real generation latency in addition to token cost. For each `(model, format)` cell, tokenometer streams `n` real chat completions (default `n=3`, override with `--latency-trials 1..10`) capped at `max_tokens=200`, and reports:
124
+
125
+ - **TTFT** — time to first streamed token (ms)
126
+ - **Total** — wall-clock from request start to stream end (ms)
127
+ - **tokens/sec** — `output_tokens / (total - ttft)`
128
+
129
+ Numbers are reported as **p50 / p95 / mean** over the trials. Full per-trial data is included in `--output json`.
130
+
131
+ ```bash
132
+ ANTHROPIC_API_KEY=… OPENAI_API_KEY=… \
133
+ npx tokenometer ./prompt.md --latency --model claude-opus-4-7,gpt-4o
134
+ ```
135
+
136
+ `--latency` implies `--empirical` (offline mode can't measure real latency). The default `--max-spend` ceiling is bumped from `$0.05` to `$0.25` to cover the `n × 200-token` generations; pass `--max-spend` explicitly to override.
137
+
138
+ Supported providers: Anthropic (`messages.stream`), OpenAI (`/v1/chat/completions` SSE), Google (`generateContentStream`), Cohere (`/v1/chat` NDJSON), Mistral (`/v1/chat/completions` SSE). Each trial retries once on transient failures.
139
+
140
+ ## Per-file attribution
141
+
142
+ `--by-file` appends a per-file token + USD summary table when you pass multiple input files (single-file inputs are a no-op):
143
+
144
+ ```
145
+ By file:
146
+ File Tokens USD
147
+ ──────────────── ─────── ───────
148
+ prompts/agent.md 1,243 $0.0186
149
+ prompts/router.md 872 $0.0131
150
+ ```
151
+
152
+ Useful for figuring out which prompt files dominate the cost of a multi-file pipeline. The aggregator that produces this table is also what powers the GitHub Action's per-file Δ comment, and is unit-tested in [`packages/action`](https://github.com/faraa2m/tokenometer/tree/main/packages/action).
153
+
154
+ ## Vision tokens
155
+
156
+ Pass `--image <path>` (repeatable) to factor image-based vision tokens into the cost estimate alongside your prompt text:
157
+
158
+ ```bash
159
+ npx tokenometer ./prompt.md --image ./screenshot.png --image ./diagram.jpg
160
+ ```
161
+
162
+ Each image's dimensions are read with `image-size` (no native deps), then dispatched to the provider-specific vision-token estimator:
163
+
164
+ - Claude → Anthropic's `(width × height) / 750`, capped at 1600 tokens.
165
+ - GPT-4o → OpenAI's high-detail tiling: `85 + 170 × ceil(w/512) × ceil(h/512)` after the 2048/768 resize step.
166
+ - Gemini → Google's `258 × ceil(w/768) × ceil(h/768)` (with a flat 258 for ≤384×384 images).
167
+
168
+ Mistral and Cohere don't have published vision-token formulas, so vision images are skipped for those providers (with a stderr note). Vision-token cells are always marked `approximate: true` since they're formula-derived. Each image also gets its own row in the `--by-file` table as a virtual file `<image-path> [vision]`.
169
+
170
+ ## Why not just `tiktoken`?
171
+
172
+ `tiktoken`'s `cl100k_base` (the encoding most "Claude tokenizer" libraries fall back on) **under-counts Opus 4.7 by a median of +62%** across a 10-prompt benchmark. Sonnet 4.6 and Haiku 4.5 are closer (~17%). Format choice is a wash. Model choice swings cost by 12×. See [README](https://github.com/faraa2m/tokenometer#findings-anthropic-n150-cells-across-10-prompt-shapes) for the dataset findings.
173
+
53
174
  ## License
54
175
 
55
176
  MIT
package/dist/args.d.ts CHANGED
@@ -1,14 +1,29 @@
1
1
  import type { Format } from '@tokenometer/core';
2
+ export type OutputFormat = 'table' | 'json' | 'sarif';
2
3
  export interface ParsedArgs {
4
+ byFile: boolean;
5
+ configPath: string | null;
3
6
  empirical: boolean;
4
7
  formats: Format[];
8
+ formatsSet: boolean;
5
9
  help: boolean;
10
+ imagePaths: string[];
6
11
  inputPaths: string[];
12
+ inputPathsSet: boolean;
13
+ latency: boolean;
14
+ latencyTrials: number;
7
15
  maxSpend: number;
16
+ /** True iff the user passed `--max-spend` explicitly (not the default). */
17
+ maxSpendSet: boolean;
8
18
  modelIds: string[];
19
+ modelsSet: boolean;
20
+ noConfig: boolean;
9
21
  offline: boolean;
22
+ output: OutputFormat;
10
23
  version: boolean;
11
24
  }
12
25
  export declare const HELP_TEXT: string;
26
+ export declare const DEFAULT_LATENCY_MAX_SPEND_USD = 0.25;
27
+ export declare const DEFAULT_LATENCY_TRIALS = 3;
13
28
  export declare const parseArgs: (argv: readonly string[]) => ParsedArgs;
14
29
  //# sourceMappingURL=args.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"args.d.ts","sourceRoot":"","sources":["../src/args.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,mBAAmB,CAAC;AAEhD,MAAM,WAAW,UAAU;IACzB,SAAS,EAAE,OAAO,CAAC;IACnB,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,IAAI,EAAE,OAAO,CAAC;IACd,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,EAAE,CAAC;IACnB,OAAO,EAAE,OAAO,CAAC;IACjB,OAAO,EAAE,OAAO,CAAC;CAClB;AAED,eAAO,MAAM,SAAS,QAsBrB,CAAC;AAKF,eAAO,MAAM,SAAS,GAAI,MAAM,SAAS,MAAM,EAAE,KAAG,UAmFnD,CAAC"}
1
+ {"version":3,"file":"args.d.ts","sourceRoot":"","sources":["../src/args.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,mBAAmB,CAAC;AAEhD,MAAM,MAAM,YAAY,GAAG,OAAO,GAAG,MAAM,GAAG,OAAO,CAAC;AAEtD,MAAM,WAAW,UAAU;IACzB,MAAM,EAAE,OAAO,CAAC;IAChB,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IAC1B,SAAS,EAAE,OAAO,CAAC;IACnB,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,UAAU,EAAE,OAAO,CAAC;IACpB,IAAI,EAAE,OAAO,CAAC;IACd,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,aAAa,EAAE,OAAO,CAAC;IACvB,OAAO,EAAE,OAAO,CAAC;IACjB,aAAa,EAAE,MAAM,CAAC;IACtB,QAAQ,EAAE,MAAM,CAAC;IACjB,2EAA2E;IAC3E,WAAW,EAAE,OAAO,CAAC;IACrB,QAAQ,EAAE,MAAM,EAAE,CAAC;IACnB,SAAS,EAAE,OAAO,CAAC;IACnB,QAAQ,EAAE,OAAO,CAAC;IAClB,OAAO,EAAE,OAAO,CAAC;IACjB,MAAM,EAAE,YAAY,CAAC;IACrB,OAAO,EAAE,OAAO,CAAC;CAClB;AAED,eAAO,MAAM,SAAS,QA2CrB,CAAC;AAIF,eAAO,MAAM,6BAA6B,OAAO,CAAC;AAClD,eAAO,MAAM,sBAAsB,IAAI,CAAC;AAQxC,eAAO,MAAM,SAAS,GAAI,MAAM,SAAS,MAAM,EAAE,KAAG,UAkJnD,CAAC"}
package/dist/args.js CHANGED
@@ -6,37 +6,73 @@ USAGE
6
6
  echo "prompt" | tokenometer - [options]
7
7
 
8
8
  OPTIONS
9
- --model <id[,id...]> Comma-separated model ids (default: claude-opus-4-7).
9
+ --model <id[,id...]> Comma-separated model ids. Default: claude-opus-4-7,
10
+ or auto-detected from *_API_KEY env when omitted.
10
11
  Known: ${KNOWN_MODELS.join(', ')}
11
12
  --format <fmt[,fmt...]> Comma-separated formats (default: all).
12
13
  Known: ${allFormats().join(', ')}
14
+ --output <fmt> Output format: table (default), json, or sarif.
15
+ --by-file With multi-file input, append a per-file token/cost table.
16
+ --image <path> Path to an image to factor into vision-token cost.
17
+ Repeatable.
18
+ --config <path> Load this exact config file (skip walk-up).
19
+ --no-config Skip .tokenometer.yml loading entirely.
13
20
  --empirical Run sample API calls and report real charges.
14
21
  Requires the matching <PROVIDER>_API_KEY env var.
15
- --max-spend <usd> Hard ceiling for empirical mode (default: 0.05).
22
+ --latency Measure real generation latency (TTFT, total ms,
23
+ tokens/sec) per (model × format) cell. Implies
24
+ --empirical and bumps the default --max-spend
25
+ ceiling to $0.25 (each trial is a metered
26
+ ~200-token chat completion). Anthropic, OpenAI,
27
+ Google, Cohere, Mistral are supported.
28
+ --latency-trials <n> Trials per cell for --latency (1-10, default 3).
29
+ Each trial requests max_tokens=200 to keep cost
30
+ predictable while giving enough output to
31
+ stabilize tokens/sec.
32
+ --max-spend <usd> Hard ceiling for empirical mode (default: 0.05;
33
+ with --latency, default 0.25).
16
34
  --offline Force offline mode (overrides --empirical).
17
35
  -h, --help Show this help.
18
36
  -v, --version Show CLI version.
19
37
 
20
38
  EXAMPLES
21
39
  tokenometer ./prompt.md
22
- tokenometer ./prompt.md --model claude-opus-4-7,gpt-4o
40
+ tokenometer ./prompt.md --model claude-opus-4-7,gpt-4o --by-file
41
+ tokenometer ./prompt.md --output sarif > tokenometer.sarif
42
+ tokenometer ./prompt.md --image ./screenshot.png
23
43
  tokenometer ./prompt.md --format yaml,json --empirical --max-spend 0.01
44
+ tokenometer ./prompt.md --latency --model gpt-4o,claude-opus-4-7
24
45
  `;
25
46
  const DEFAULT_MODELS = ['claude-opus-4-7'];
26
47
  const DEFAULT_MAX_SPEND_USD = 0.05;
48
+ export const DEFAULT_LATENCY_MAX_SPEND_USD = 0.25;
49
+ export const DEFAULT_LATENCY_TRIALS = 3;
50
+ const MIN_LATENCY_TRIALS = 1;
51
+ const MAX_LATENCY_TRIALS = 10;
52
+ const OUTPUT_FORMATS = ['table', 'json', 'sarif'];
53
+ const isOutputFormat = (value) => OUTPUT_FORMATS.includes(value);
27
54
  export const parseArgs = (argv) => {
28
55
  const result = {
56
+ byFile: false,
57
+ configPath: null,
29
58
  empirical: false,
30
59
  formats: [...allFormats()],
60
+ formatsSet: false,
31
61
  help: false,
62
+ imagePaths: [],
32
63
  inputPaths: [],
64
+ inputPathsSet: false,
65
+ latency: false,
66
+ latencyTrials: DEFAULT_LATENCY_TRIALS,
33
67
  maxSpend: DEFAULT_MAX_SPEND_USD,
68
+ maxSpendSet: false,
34
69
  modelIds: [...DEFAULT_MODELS],
70
+ modelsSet: false,
71
+ noConfig: false,
35
72
  offline: false,
73
+ output: 'table',
36
74
  version: false,
37
75
  };
38
- let modelsSet = false;
39
- let formatsSet = false;
40
76
  for (let i = 0; i < argv.length; i++) {
41
77
  const arg = argv[i];
42
78
  if (!arg)
@@ -53,10 +89,59 @@ export const parseArgs = (argv) => {
53
89
  result.empirical = true;
54
90
  continue;
55
91
  }
92
+ if (arg === '--latency') {
93
+ result.latency = true;
94
+ // --latency implies --empirical (offline mode can't measure real latency).
95
+ result.empirical = true;
96
+ continue;
97
+ }
98
+ if (arg === '--latency-trials') {
99
+ const next = argv[++i];
100
+ if (!next)
101
+ throw new Error('--latency-trials requires a value');
102
+ const parsed = Number.parseInt(next, 10);
103
+ if (!Number.isFinite(parsed) || parsed < MIN_LATENCY_TRIALS || parsed > MAX_LATENCY_TRIALS) {
104
+ throw new Error(`--latency-trials must be an integer between ${MIN_LATENCY_TRIALS} and ${MAX_LATENCY_TRIALS}, got "${next}".`);
105
+ }
106
+ result.latencyTrials = parsed;
107
+ continue;
108
+ }
56
109
  if (arg === '--offline') {
57
110
  result.offline = true;
58
111
  continue;
59
112
  }
113
+ if (arg === '--by-file') {
114
+ result.byFile = true;
115
+ continue;
116
+ }
117
+ if (arg === '--no-config') {
118
+ result.noConfig = true;
119
+ continue;
120
+ }
121
+ if (arg === '--config') {
122
+ const next = argv[++i];
123
+ if (!next)
124
+ throw new Error('--config requires a value');
125
+ result.configPath = next;
126
+ continue;
127
+ }
128
+ if (arg === '--output') {
129
+ const next = argv[++i];
130
+ if (!next)
131
+ throw new Error('--output requires a value');
132
+ if (!isOutputFormat(next)) {
133
+ throw new Error(`Unknown --output "${next}". Known: ${OUTPUT_FORMATS.join(', ')}.`);
134
+ }
135
+ result.output = next;
136
+ continue;
137
+ }
138
+ if (arg === '--image') {
139
+ const next = argv[++i];
140
+ if (!next)
141
+ throw new Error('--image requires a value');
142
+ result.imagePaths.push(next);
143
+ continue;
144
+ }
60
145
  if (arg === '--model') {
61
146
  const next = argv[++i];
62
147
  if (!next)
@@ -65,7 +150,7 @@ export const parseArgs = (argv) => {
65
150
  .split(',')
66
151
  .map((s) => s.trim())
67
152
  .filter(Boolean);
68
- modelsSet = true;
153
+ result.modelsSet = true;
69
154
  continue;
70
155
  }
71
156
  if (arg === '--format') {
@@ -82,7 +167,7 @@ export const parseArgs = (argv) => {
82
167
  }
83
168
  }
84
169
  result.formats = formats;
85
- formatsSet = true;
170
+ result.formatsSet = true;
86
171
  continue;
87
172
  }
88
173
  if (arg === '--max-spend') {
@@ -94,19 +179,27 @@ export const parseArgs = (argv) => {
94
179
  throw new Error(`--max-spend must be a positive number, got "${next}".`);
95
180
  }
96
181
  result.maxSpend = parsed;
182
+ result.maxSpendSet = true;
97
183
  continue;
98
184
  }
99
185
  if (arg.startsWith('--')) {
100
186
  throw new Error(`Unknown flag: ${arg}`);
101
187
  }
102
188
  result.inputPaths.push(arg);
189
+ result.inputPathsSet = true;
103
190
  }
104
- if (!modelsSet && result.modelIds.length === 0) {
191
+ if (!result.modelsSet && result.modelIds.length === 0) {
105
192
  result.modelIds = [...DEFAULT_MODELS];
106
193
  }
107
- if (!formatsSet && result.formats.length === 0) {
194
+ if (!result.formatsSet && result.formats.length === 0) {
108
195
  result.formats = [...allFormats()];
109
196
  }
197
+ // `--latency` makes the default `--max-spend` ceiling more generous since
198
+ // each trial is a metered ~200-token chat completion. Only bump if the
199
+ // user did not explicitly set --max-spend (theirs always wins).
200
+ if (result.latency && !result.maxSpendSet) {
201
+ result.maxSpend = DEFAULT_LATENCY_MAX_SPEND_USD;
202
+ }
110
203
  return result;
111
204
  };
112
205
  //# sourceMappingURL=args.js.map
package/dist/args.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"args.js","sourceRoot":"","sources":["../src/args.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,QAAQ,EAAE,MAAM,mBAAmB,CAAC;AAcvE,MAAM,CAAC,MAAM,SAAS,GAAG;;;;;;;;oCAQW,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC;;oCAEvB,UAAU,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC;;;;;;;;;;;;CAY1D,CAAC;AAEF,MAAM,cAAc,GAAG,CAAC,iBAAiB,CAAC,CAAC;AAC3C,MAAM,qBAAqB,GAAG,IAAI,CAAC;AAEnC,MAAM,CAAC,MAAM,SAAS,GAAG,CAAC,IAAuB,EAAc,EAAE;IAC/D,MAAM,MAAM,GAAe;QACzB,SAAS,EAAE,KAAK;QAChB,OAAO,EAAE,CAAC,GAAG,UAAU,EAAE,CAAC;QAC1B,IAAI,EAAE,KAAK;QACX,UAAU,EAAE,EAAE;QACd,QAAQ,EAAE,qBAAqB;QAC/B,QAAQ,EAAE,CAAC,GAAG,cAAc,CAAC;QAC7B,OAAO,EAAE,KAAK;QACd,OAAO,EAAE,KAAK;KACf,CAAC;IAEF,IAAI,SAAS,GAAG,KAAK,CAAC;IACtB,IAAI,UAAU,GAAG,KAAK,CAAC;IAEvB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACpB,IAAI,CAAC,GAAG;YAAE,SAAS;QACnB,IAAI,GAAG,KAAK,IAAI,IAAI,GAAG,KAAK,QAAQ,EAAE,CAAC;YACrC,MAAM,CAAC,IAAI,GAAG,IAAI,CAAC;YACnB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,IAAI,IAAI,GAAG,KAAK,WAAW,EAAE,CAAC;YACxC,MAAM,CAAC,OAAO,GAAG,IAAI,CAAC;YACtB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,aAAa,EAAE,CAAC;YAC1B,MAAM,CAAC,SAAS,GAAG,IAAI,CAAC;YACxB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,WAAW,EAAE,CAAC;YACxB,MAAM,CAAC,OAAO,GAAG,IAAI,CAAC;YACtB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;YACtB,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,0BAA0B,CAAC,CAAC;YACvD,MAAM,CAAC,QAAQ,GAAG,IAAI;iBACnB,KAAK,CAAC,GAAG,CAAC;iBACV,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;iBACpB,MAAM,CAAC,OAAO,CAAC,CAAC;YACnB,SAAS,GAAG,IAAI,CAAC;YACjB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,UAAU,EAAE,CAAC;YACvB,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,2BAA2B,CAAC,CAAC;YACxD,MAAM,OAAO,GAAG,IAAI;iBACjB,KAAK,CAAC,GAAG,CAAC;iBACV,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;iBACpB,MAAM,CAAC,OAAO,CAAC,CAAC;YACnB,KAAK,MAAM,GAAG,IAAI,OAAO,EAAE,CAAC;gBAC1B,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;oBACnB,MAAM,IAAI,KAAK,CAAC,mBAAmB,GAAG,aAAa,UAAU,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;gBACjF,CAAC;YACH,CAAC;YACD,MAAM,CAAC,OAAO,GAAG,OAAmB,CAAC;YACrC,UAAU,GAAG,IAAI,CAAC;YAClB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,aAAa,EAAE,CAAC;YAC1B,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,8BAA8B,CAAC,CAAC;YAC3D,MAAM,MAAM,GAAG,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC;YACvC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,MAAM,IAAI,CAAC,EAAE,CAAC;gBAC5C,MAAM,IAAI,KAAK,CAAC,+CAA+C,IAAI,IAAI,CAAC,CAAC;YAC3E,CAAC;YACD,MAAM,CAAC,QAAQ,GAAG,MAAM,CAAC;YACzB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YACzB,MAAM,IAAI,KAAK,CAAC,iBAAiB,GAAG,EAAE,CAAC,CAAC;QAC1C,CAAC;QACD,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IAC9B,CAAC;IAED,IAAI,CAAC,SAAS,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC/C,MAAM,CAAC,QAAQ,GAAG,CAAC,GAAG,cAAc,CAAC,CAAC;IACxC,CAAC;IACD,IAAI,CAAC,UAAU,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC/C,MAAM,CAAC,OAAO,GAAG,CAAC,GAAG,UAAU,EAAE,CAAC,CAAC;IACrC,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC,CAAC"}
1
+ {"version":3,"file":"args.js","sourceRoot":"","sources":["../src/args.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,QAAQ,EAAE,MAAM,mBAAmB,CAAC;AA4BvE,MAAM,CAAC,MAAM,SAAS,GAAG;;;;;;;;;oCASW,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC;;oCAEvB,UAAU,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CAgC1D,CAAC;AAEF,MAAM,cAAc,GAAG,CAAC,iBAAiB,CAAC,CAAC;AAC3C,MAAM,qBAAqB,GAAG,IAAI,CAAC;AACnC,MAAM,CAAC,MAAM,6BAA6B,GAAG,IAAI,CAAC;AAClD,MAAM,CAAC,MAAM,sBAAsB,GAAG,CAAC,CAAC;AACxC,MAAM,kBAAkB,GAAG,CAAC,CAAC;AAC7B,MAAM,kBAAkB,GAAG,EAAE,CAAC;AAC9B,MAAM,cAAc,GAA4B,CAAC,OAAO,EAAE,MAAM,EAAE,OAAO,CAAC,CAAC;AAE3E,MAAM,cAAc,GAAG,CAAC,KAAa,EAAyB,EAAE,CAC7D,cAAoC,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC;AAExD,MAAM,CAAC,MAAM,SAAS,GAAG,CAAC,IAAuB,EAAc,EAAE;IAC/D,MAAM,MAAM,GAAe;QACzB,MAAM,EAAE,KAAK;QACb,UAAU,EAAE,IAAI;QAChB,SAAS,EAAE,KAAK;QAChB,OAAO,EAAE,CAAC,GAAG,UAAU,EAAE,CAAC;QAC1B,UAAU,EAAE,KAAK;QACjB,IAAI,EAAE,KAAK;QACX,UAAU,EAAE,EAAE;QACd,UAAU,EAAE,EAAE;QACd,aAAa,EAAE,KAAK;QACpB,OAAO,EAAE,KAAK;QACd,aAAa,EAAE,sBAAsB;QACrC,QAAQ,EAAE,qBAAqB;QAC/B,WAAW,EAAE,KAAK;QAClB,QAAQ,EAAE,CAAC,GAAG,cAAc,CAAC;QAC7B,SAAS,EAAE,KAAK;QAChB,QAAQ,EAAE,KAAK;QACf,OAAO,EAAE,KAAK;QACd,MAAM,EAAE,OAAO;QACf,OAAO,EAAE,KAAK;KACf,CAAC;IAEF,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACpB,IAAI,CAAC,GAAG;YAAE,SAAS;QACnB,IAAI,GAAG,KAAK,IAAI,IAAI,GAAG,KAAK,QAAQ,EAAE,CAAC;YACrC,MAAM,CAAC,IAAI,GAAG,IAAI,CAAC;YACnB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,IAAI,IAAI,GAAG,KAAK,WAAW,EAAE,CAAC;YACxC,MAAM,CAAC,OAAO,GAAG,IAAI,CAAC;YACtB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,aAAa,EAAE,CAAC;YAC1B,MAAM,CAAC,SAAS,GAAG,IAAI,CAAC;YACxB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,WAAW,EAAE,CAAC;YACxB,MAAM,CAAC,OAAO,GAAG,IAAI,CAAC;YACtB,2EAA2E;YAC3E,MAAM,CAAC,SAAS,GAAG,IAAI,CAAC;YACxB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,kBAAkB,EAAE,CAAC;YAC/B,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;YAChE,MAAM,MAAM,GAAG,MAAM,CAAC,QAAQ,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YACzC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,MAAM,GAAG,kBAAkB,IAAI,MAAM,GAAG,kBAAkB,EAAE,CAAC;gBAC3F,MAAM,IAAI,KAAK,CACb,+CAA+C,kBAAkB,QAAQ,kBAAkB,UAAU,IAAI,IAAI,CAC9G,CAAC;YACJ,CAAC;YACD,MAAM,CAAC,aAAa,GAAG,MAAM,CAAC;YAC9B,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,WAAW,EAAE,CAAC;YACxB,MAAM,CAAC,OAAO,GAAG,IAAI,CAAC;YACtB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,WAAW,EAAE,CAAC;YACxB,MAAM,CAAC,MAAM,GAAG,IAAI,CAAC;YACrB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,aAAa,EAAE,CAAC;YAC1B,MAAM,CAAC,QAAQ,GAAG,IAAI,CAAC;YACvB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,UAAU,EAAE,CAAC;YACvB,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,2BAA2B,CAAC,CAAC;YACxD,MAAM,CAAC,UAAU,GAAG,IAAI,CAAC;YACzB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,UAAU,EAAE,CAAC;YACvB,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,2BAA2B,CAAC,CAAC;YACxD,IAAI,CAAC,cAAc,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC1B,MAAM,IAAI,KAAK,CAAC,qBAAqB,IAAI,aAAa,cAAc,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YACtF,CAAC;YACD,MAAM,CAAC,MAAM,GAAG,IAAI,CAAC;YACrB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;YACtB,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,0BAA0B,CAAC,CAAC;YACvD,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YAC7B,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;YACtB,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,0BAA0B,CAAC,CAAC;YACvD,MAAM,CAAC,QAAQ,GAAG,IAAI;iBACnB,KAAK,CAAC,GAAG,CAAC;iBACV,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;iBACpB,MAAM,CAAC,OAAO,CAAC,CAAC;YACnB,MAAM,CAAC,SAAS,GAAG,IAAI,CAAC;YACxB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,UAAU,EAAE,CAAC;YACvB,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,2BAA2B,CAAC,CAAC;YACxD,MAAM,OAAO,GAAG,IAAI;iBACjB,KAAK,CAAC,GAAG,CAAC;iBACV,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;iBACpB,MAAM,CAAC,OAAO,CAAC,CAAC;YACnB,KAAK,MAAM,GAAG,IAAI,OAAO,EAAE,CAAC;gBAC1B,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;oBACnB,MAAM,IAAI,KAAK,CAAC,mBAAmB,GAAG,aAAa,UAAU,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;gBACjF,CAAC;YACH,CAAC;YACD,MAAM,CAAC,OAAO,GAAG,OAAmB,CAAC;YACrC,MAAM,CAAC,UAAU,GAAG,IAAI,CAAC;YACzB,SAAS;QACX,CAAC;QACD,IAAI,GAAG,KAAK,aAAa,EAAE,CAAC;YAC1B,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACvB,IAAI,CAAC,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,8BAA8B,CAAC,CAAC;YAC3D,MAAM,MAAM,GAAG,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC;YACvC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,MAAM,IAAI,CAAC,EAAE,CAAC;gBAC5C,MAAM,IAAI,KAAK,CAAC,+CAA+C,IAAI,IAAI,CAAC,CAAC;YAC3E,CAAC;YACD,MAAM,CAAC,QAAQ,GAAG,MAAM,CAAC;YACzB,MAAM,CAAC,WAAW,GAAG,IAAI,CAAC;YAC1B,SAAS;QACX,CAAC;QACD,IAAI,GAAG,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YACzB,MAAM,IAAI,KAAK,CAAC,iBAAiB,GAAG,EAAE,CAAC,CAAC;QAC1C,CAAC;QACD,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QAC5B,MAAM,CAAC,aAAa,GAAG,IAAI,CAAC;IAC9B,CAAC;IAED,IAAI,CAAC,MAAM,CAAC,SAAS,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACtD,MAAM,CAAC,QAAQ,GAAG,CAAC,GAAG,cAAc,CAAC,CAAC;IACxC,CAAC;IACD,IAAI,CAAC,MAAM,CAAC,UAAU,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACtD,MAAM,CAAC,OAAO,GAAG,CAAC,GAAG,UAAU,EAAE,CAAC,CAAC;IACrC,CAAC;IACD,0EAA0E;IAC1E,uEAAuE;IACvE,gEAAgE;IAChE,IAAI,MAAM,CAAC,OAAO,IAAI,CAAC,MAAM,CAAC,WAAW,EAAE,CAAC;QAC1C,MAAM,CAAC,QAAQ,GAAG,6BAA6B,CAAC;IAClD,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC,CAAC"}
@@ -0,0 +1,19 @@
1
+ export interface AutoDetectInput {
2
+ env?: NodeJS.ProcessEnv;
3
+ }
4
+ export interface AutoDetectResult {
5
+ /** The chosen default model id. */
6
+ modelId: string;
7
+ /** Optional human-readable note for stderr (e.g. multi-key conflict). */
8
+ note: string | null;
9
+ }
10
+ /**
11
+ * When the user has not passed `--model`, pick a default based on which
12
+ * provider API key is present in the environment.
13
+ *
14
+ * - Exactly one provider key set → that provider's canonical model.
15
+ * - Multiple keys set → fall back to the existing default with a stderr note.
16
+ * - No keys set → existing default behavior.
17
+ */
18
+ export declare const autoDetectDefaultModel: (input?: AutoDetectInput) => AutoDetectResult;
19
+ //# sourceMappingURL=auto-detect.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"auto-detect.d.ts","sourceRoot":"","sources":["../src/auto-detect.ts"],"names":[],"mappings":"AAaA,MAAM,WAAW,eAAe;IAC9B,GAAG,CAAC,EAAE,MAAM,CAAC,UAAU,CAAC;CACzB;AAED,MAAM,WAAW,gBAAgB;IAC/B,mCAAmC;IACnC,OAAO,EAAE,MAAM,CAAC;IAChB,yEAAyE;IACzE,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;CACrB;AAED;;;;;;;GAOG;AACH,eAAO,MAAM,sBAAsB,GAAI,QAAO,eAAoB,KAAG,gBAiBpE,CAAC"}
@@ -0,0 +1,41 @@
1
+ import { KNOWN_MODELS, MODELS } from '@tokenometer/core';
2
+ const DEFAULT_MODEL = 'claude-opus-4-7';
3
+ const firstGoogleGeminiModel = () => {
4
+ for (const id of KNOWN_MODELS) {
5
+ if (!id.startsWith('gemini-'))
6
+ continue;
7
+ const desc = MODELS[id];
8
+ if (desc?.provider === 'google')
9
+ return id;
10
+ }
11
+ return 'gemini-2.5-pro';
12
+ };
13
+ /**
14
+ * When the user has not passed `--model`, pick a default based on which
15
+ * provider API key is present in the environment.
16
+ *
17
+ * - Exactly one provider key set → that provider's canonical model.
18
+ * - Multiple keys set → fall back to the existing default with a stderr note.
19
+ * - No keys set → existing default behavior.
20
+ */
21
+ export const autoDetectDefaultModel = (input = {}) => {
22
+ const env = input.env ?? process.env;
23
+ const hasAnthropic = Boolean(env.ANTHROPIC_API_KEY);
24
+ const hasOpenAi = Boolean(env.OPENAI_API_KEY);
25
+ const hasGoogle = Boolean(env.GOOGLE_API_KEY ?? env.GEMINI_API_KEY);
26
+ const setCount = [hasAnthropic, hasOpenAi, hasGoogle].filter(Boolean).length;
27
+ if (setCount === 0)
28
+ return { modelId: DEFAULT_MODEL, note: null };
29
+ if (setCount > 1) {
30
+ return {
31
+ modelId: DEFAULT_MODEL,
32
+ note: `Multiple provider API keys detected; defaulting to ${DEFAULT_MODEL}. Pass --model to override.`,
33
+ };
34
+ }
35
+ if (hasAnthropic)
36
+ return { modelId: 'claude-opus-4-7', note: null };
37
+ if (hasOpenAi)
38
+ return { modelId: 'gpt-4o', note: null };
39
+ return { modelId: firstGoogleGeminiModel(), note: null };
40
+ };
41
+ //# sourceMappingURL=auto-detect.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"auto-detect.js","sourceRoot":"","sources":["../src/auto-detect.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,EAAE,MAAM,mBAAmB,CAAC;AAEzD,MAAM,aAAa,GAAG,iBAAiB,CAAC;AAExC,MAAM,sBAAsB,GAAG,GAAW,EAAE;IAC1C,KAAK,MAAM,EAAE,IAAI,YAAY,EAAE,CAAC;QAC9B,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,SAAS,CAAC;YAAE,SAAS;QACxC,MAAM,IAAI,GAAG,MAAM,CAAC,EAAE,CAAC,CAAC;QACxB,IAAI,IAAI,EAAE,QAAQ,KAAK,QAAQ;YAAE,OAAO,EAAE,CAAC;IAC7C,CAAC;IACD,OAAO,gBAAgB,CAAC;AAC1B,CAAC,CAAC;AAaF;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,sBAAsB,GAAG,CAAC,QAAyB,EAAE,EAAoB,EAAE;IACtF,MAAM,GAAG,GAAG,KAAK,CAAC,GAAG,IAAI,OAAO,CAAC,GAAG,CAAC;IACrC,MAAM,YAAY,GAAG,OAAO,CAAC,GAAG,CAAC,iBAAiB,CAAC,CAAC;IACpD,MAAM,SAAS,GAAG,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC,CAAC;IAC9C,MAAM,SAAS,GAAG,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,GAAG,CAAC,cAAc,CAAC,CAAC;IAEpE,MAAM,QAAQ,GAAG,CAAC,YAAY,EAAE,SAAS,EAAE,SAAS,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,MAAM,CAAC;IAC7E,IAAI,QAAQ,KAAK,CAAC;QAAE,OAAO,EAAE,OAAO,EAAE,aAAa,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IAClE,IAAI,QAAQ,GAAG,CAAC,EAAE,CAAC;QACjB,OAAO;YACL,OAAO,EAAE,aAAa;YACtB,IAAI,EAAE,sDAAsD,aAAa,6BAA6B;SACvG,CAAC;IACJ,CAAC;IACD,IAAI,YAAY;QAAE,OAAO,EAAE,OAAO,EAAE,iBAAiB,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IACpE,IAAI,SAAS;QAAE,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IACxD,OAAO,EAAE,OAAO,EAAE,sBAAsB,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;AAC3D,CAAC,CAAC"}
@@ -0,0 +1,18 @@
1
+ import { type TokenometerConfig } from '@tokenometer/core';
2
+ import type { ParsedArgs } from './args.js';
3
+ export interface ApplyConfigOptions {
4
+ /** Pre-loaded config (already parsed). When omitted, applyConfig is a no-op. */
5
+ config: TokenometerConfig | null;
6
+ }
7
+ /**
8
+ * Apply config defaults to ParsedArgs.
9
+ * User-passed flags ALWAYS win over config (we only fill in fields the user did not set).
10
+ * Returns a new ParsedArgs (does not mutate the input).
11
+ */
12
+ export declare const applyConfig: (args: ParsedArgs, opts: ApplyConfigOptions) => ParsedArgs;
13
+ /**
14
+ * Read a config from a user-specified path (used by `--config <path>`).
15
+ * Throws on parse / validation failure with the offending path included.
16
+ */
17
+ export declare const loadConfigFromPath: (path: string) => Promise<TokenometerConfig>;
18
+ //# sourceMappingURL=config-merge.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"config-merge.d.ts","sourceRoot":"","sources":["../src/config-merge.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,KAAK,iBAAiB,EAAe,MAAM,mBAAmB,CAAC;AACxE,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,WAAW,CAAC;AAE5C,MAAM,WAAW,kBAAkB;IACjC,gFAAgF;IAChF,MAAM,EAAE,iBAAiB,GAAG,IAAI,CAAC;CAClC;AAED;;;;GAIG;AACH,eAAO,MAAM,WAAW,GAAI,MAAM,UAAU,EAAE,MAAM,kBAAkB,KAAG,UAoBxE,CAAC;AAEF;;;GAGG;AACH,eAAO,MAAM,kBAAkB,GAAU,MAAM,MAAM,KAAG,OAAO,CAAC,iBAAiB,CAYhF,CAAC"}
@@ -0,0 +1,49 @@
1
+ import { readFile } from 'node:fs/promises';
2
+ import { parseConfig } from '@tokenometer/core';
3
+ /**
4
+ * Apply config defaults to ParsedArgs.
5
+ * User-passed flags ALWAYS win over config (we only fill in fields the user did not set).
6
+ * Returns a new ParsedArgs (does not mutate the input).
7
+ */
8
+ export const applyConfig = (args, opts) => {
9
+ const cfg = opts.config;
10
+ if (!cfg)
11
+ return args;
12
+ const next = { ...args, formats: [...args.formats], modelIds: [...args.modelIds] };
13
+ // When the config provides a value AND the user didn't pass it on the CLI,
14
+ // adopt the config value AND set the *Set flag so downstream auto-detect
15
+ // doesn't overwrite the config-provided default.
16
+ if (!args.modelsSet && cfg.models && cfg.models.length > 0) {
17
+ next.modelIds = [...cfg.models];
18
+ next.modelsSet = true;
19
+ }
20
+ if (!args.formatsSet && cfg.formats && cfg.formats.length > 0) {
21
+ next.formats = [...cfg.formats];
22
+ next.formatsSet = true;
23
+ }
24
+ if (!args.inputPathsSet && cfg.paths && cfg.paths.length > 0) {
25
+ next.inputPaths = [...cfg.paths];
26
+ next.inputPathsSet = true;
27
+ }
28
+ return next;
29
+ };
30
+ /**
31
+ * Read a config from a user-specified path (used by `--config <path>`).
32
+ * Throws on parse / validation failure with the offending path included.
33
+ */
34
+ export const loadConfigFromPath = async (path) => {
35
+ let text;
36
+ try {
37
+ text = await readFile(path, 'utf8');
38
+ }
39
+ catch (err) {
40
+ throw new Error(`Failed to read config "${path}": ${err.message}`);
41
+ }
42
+ try {
43
+ return parseConfig(text);
44
+ }
45
+ catch (err) {
46
+ throw new Error(`Invalid config at "${path}": ${err.message}`);
47
+ }
48
+ };
49
+ //# sourceMappingURL=config-merge.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"config-merge.js","sourceRoot":"","sources":["../src/config-merge.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAA0B,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAQxE;;;;GAIG;AACH,MAAM,CAAC,MAAM,WAAW,GAAG,CAAC,IAAgB,EAAE,IAAwB,EAAc,EAAE;IACpF,MAAM,GAAG,GAAG,IAAI,CAAC,MAAM,CAAC;IACxB,IAAI,CAAC,GAAG;QAAE,OAAO,IAAI,CAAC;IACtB,MAAM,IAAI,GAAe,EAAE,GAAG,IAAI,EAAE,OAAO,EAAE,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,EAAE,QAAQ,EAAE,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IAC/F,2EAA2E;IAC3E,yEAAyE;IACzE,iDAAiD;IACjD,IAAI,CAAC,IAAI,CAAC,SAAS,IAAI,GAAG,CAAC,MAAM,IAAI,GAAG,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC3D,IAAI,CAAC,QAAQ,GAAG,CAAC,GAAG,GAAG,CAAC,MAAM,CAAC,CAAC;QAChC,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC;IACxB,CAAC;IACD,IAAI,CAAC,IAAI,CAAC,UAAU,IAAI,GAAG,CAAC,OAAO,IAAI,GAAG,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC9D,IAAI,CAAC,OAAO,GAAG,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC,CAAC;QAChC,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC;IACzB,CAAC;IACD,IAAI,CAAC,IAAI,CAAC,aAAa,IAAI,GAAG,CAAC,KAAK,IAAI,GAAG,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC7D,IAAI,CAAC,UAAU,GAAG,CAAC,GAAG,GAAG,CAAC,KAAK,CAAC,CAAC;QACjC,IAAI,CAAC,aAAa,GAAG,IAAI,CAAC;IAC5B,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC,CAAC;AAEF;;;GAGG;AACH,MAAM,CAAC,MAAM,kBAAkB,GAAG,KAAK,EAAE,IAAY,EAA8B,EAAE;IACnF,IAAI,IAAY,CAAC;IACjB,IAAI,CAAC;QACH,IAAI,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC;IACtC,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,MAAM,IAAI,KAAK,CAAC,0BAA0B,IAAI,MAAO,GAAa,CAAC,OAAO,EAAE,CAAC,CAAC;IAChF,CAAC;IACD,IAAI,CAAC;QACH,OAAO,WAAW,CAAC,IAAI,CAAC,CAAC;IAC3B,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,MAAM,IAAI,KAAK,CAAC,sBAAsB,IAAI,MAAO,GAAa,CAAC,OAAO,EAAE,CAAC,CAAC;IAC5E,CAAC;AACH,CAAC,CAAC"}
package/dist/index.d.ts CHANGED
@@ -1,3 +1,14 @@
1
1
  #!/usr/bin/env node
2
- export declare const main: (argv: readonly string[]) => Promise<number>;
2
+ import type { LatencyResult, MeasureLatencyOptions } from '@tokenometer/core';
3
+ import { type ImageSizeReader } from './vision.js';
4
+ /** Test seam for `measureLatency`; production code uses the SDK-backed default. */
5
+ export type MeasureLatencyFn = (options: MeasureLatencyOptions) => Promise<LatencyResult>;
6
+ interface RunDeps {
7
+ imageSizeReader?: ImageSizeReader;
8
+ measureLatencyFn?: MeasureLatencyFn;
9
+ stderr?: NodeJS.WriteStream;
10
+ stdout?: NodeJS.WriteStream;
11
+ }
12
+ export declare const main: (argv: readonly string[], deps?: RunDeps) => Promise<number>;
13
+ export {};
3
14
  //# sourceMappingURL=index.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAoCA,eAAO,MAAM,IAAI,GAAU,MAAM,SAAS,MAAM,EAAE,KAAG,OAAO,CAAC,MAAM,CAwDlE,CAAC"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAeA,OAAO,KAAK,EAGV,aAAa,EACb,qBAAqB,EAEtB,MAAM,mBAAmB,CAAC;AAK3B,OAAO,EACL,KAAK,eAAe,EAIrB,MAAM,aAAa,CAAC;AAsDrB,mFAAmF;AACnF,MAAM,MAAM,gBAAgB,GAAG,CAAC,OAAO,EAAE,qBAAqB,KAAK,OAAO,CAAC,aAAa,CAAC,CAAC;AAE1F,UAAU,OAAO;IACf,eAAe,CAAC,EAAE,eAAe,CAAC;IAClC,gBAAgB,CAAC,EAAE,gBAAgB,CAAC;IACpC,MAAM,CAAC,EAAE,MAAM,CAAC,WAAW,CAAC;IAC5B,MAAM,CAAC,EAAE,MAAM,CAAC,WAAW,CAAC;CAC7B;AAuFD,eAAO,MAAM,IAAI,GAAU,MAAM,SAAS,MAAM,EAAE,EAAE,OAAM,OAAY,KAAG,OAAO,CAAC,MAAM,CAwJtF,CAAC"}
package/dist/index.js CHANGED
@@ -2,18 +2,27 @@
2
2
  import { realpathSync } from 'node:fs';
3
3
  import { readFile } from 'node:fs/promises';
4
4
  import { pathToFileURL } from 'node:url';
5
- import { tokenizeMatrix, tokenizeMatrixEmpirical } from '@tokenometer/core';
5
+ import { getModel, getRate, loadConfig, measureLatency, toSarif, tokenizeMatrix, tokenizeMatrixEmpirical, } from '@tokenometer/core';
6
6
  import { HELP_TEXT, parseArgs } from './args.js';
7
- import { renderModelLimits, renderSummary, renderTable } from './render.js';
7
+ import { autoDetectDefaultModel } from './auto-detect.js';
8
+ import { applyConfig, loadConfigFromPath } from './config-merge.js';
9
+ import { renderByFile, renderModelLimits, renderSummary, renderTable } from './render.js';
10
+ import { computeVisionTokens, defaultImageSizeReader, resolveImages, } from './vision.js';
8
11
  const VERSION = '0.0.2';
9
12
  const readEnv = () => {
10
13
  const env = {};
11
- const { ANTHROPIC_API_KEY, GEMINI_API_KEY, GOOGLE_API_KEY } = process.env;
14
+ const { ANTHROPIC_API_KEY, COHERE_API_KEY, GEMINI_API_KEY, GOOGLE_API_KEY, MISTRAL_API_KEY, OPENAI_API_KEY, } = process.env;
12
15
  if (ANTHROPIC_API_KEY)
13
16
  env.anthropicApiKey = ANTHROPIC_API_KEY;
17
+ if (COHERE_API_KEY)
18
+ env.cohereApiKey = COHERE_API_KEY;
14
19
  const googleKey = GOOGLE_API_KEY ?? GEMINI_API_KEY;
15
20
  if (googleKey)
16
21
  env.googleApiKey = googleKey;
22
+ if (MISTRAL_API_KEY)
23
+ env.mistralApiKey = MISTRAL_API_KEY;
24
+ if (OPENAI_API_KEY)
25
+ env.openaiApiKey = OPENAI_API_KEY;
17
26
  return env;
18
27
  };
19
28
  const readStdin = async () => {
@@ -23,64 +32,228 @@ const readStdin = async () => {
23
32
  }
24
33
  return Buffer.concat(chunks).toString('utf8');
25
34
  };
26
- const readPrompt = async (paths) => {
35
+ const readJoinedPrompt = async (paths) => {
27
36
  if (paths.length === 0 || paths[0] === '-') {
28
37
  return readStdin();
29
38
  }
30
39
  const contents = await Promise.all(paths.map((p) => readFile(p, 'utf8')));
31
40
  return contents.join('\n');
32
41
  };
33
- export const main = async (argv) => {
42
+ const readPerFilePrompts = async (paths) => {
43
+ if (paths.length === 0 || paths[0] === '-') {
44
+ const prompt = await readStdin();
45
+ return [{ path: '-', prompt }];
46
+ }
47
+ return Promise.all(paths.map(async (path) => {
48
+ const prompt = await readFile(path, 'utf8');
49
+ return { path, prompt };
50
+ }));
51
+ };
52
+ /**
53
+ * For each cell, run a real streaming generation and attach the resulting
54
+ * `LatencyResult` in-place. Skips cells whose provider doesn't yet support
55
+ * the latency path (currently: none — all 5 do).
56
+ */
57
+ const augmentWithLatency = async (cells, prompt, parsed, measure) => {
58
+ // Sequential (not parallel) so we don't slam a single provider with N
59
+ // concurrent metered requests; rate-limits are real.
60
+ for (const cell of cells) {
61
+ cell.latency = await measure({
62
+ env: readEnv(),
63
+ modelId: cell.model,
64
+ prompt,
65
+ trials: parsed.latencyTrials,
66
+ });
67
+ }
68
+ };
69
+ const buildPerFileResults = async (parsed, useEmpirical, measureLatencyFn) => {
70
+ const inputs = await readPerFilePrompts(parsed.inputPaths);
71
+ const out = [];
72
+ for (const { path, prompt } of inputs) {
73
+ if (!prompt.trim())
74
+ continue;
75
+ const cells = useEmpirical
76
+ ? await tokenizeMatrixEmpirical({
77
+ env: readEnv(),
78
+ formats: parsed.formats,
79
+ modelIds: parsed.modelIds,
80
+ prompt,
81
+ })
82
+ : tokenizeMatrix({
83
+ formats: parsed.formats,
84
+ modelIds: parsed.modelIds,
85
+ prompt,
86
+ });
87
+ if (parsed.latency) {
88
+ await augmentWithLatency(cells, prompt, parsed, measureLatencyFn ?? measureLatency);
89
+ }
90
+ out.push({ path, results: cells });
91
+ }
92
+ return out;
93
+ };
94
+ const buildImageResults = async (parsed, reader) => {
95
+ if (parsed.imagePaths.length === 0)
96
+ return [];
97
+ const resolved = await resolveImages(parsed.imagePaths, reader);
98
+ const results = [];
99
+ for (const img of resolved) {
100
+ const cells = parsed.modelIds.map((modelId) => {
101
+ const tokens = computeVisionTokens(modelId, img.dim, img.path);
102
+ return makeVisionCell(modelId, tokens);
103
+ });
104
+ results.push({ path: `${img.path} [vision]`, results: cells });
105
+ }
106
+ return results;
107
+ };
108
+ const makeVisionCell = (modelId, tokens) => {
109
+ // Vision tokens are formula-derived; we fold them under format='text' as a
110
+ // neutral synthetic axis (vision is format-agnostic).
111
+ const rate = getRate(modelId);
112
+ const provider = getModel(modelId).provider;
113
+ return {
114
+ approximate: true,
115
+ format: 'text',
116
+ inputCost: (tokens / 1000) * rate.inputPer1k,
117
+ inputTokens: tokens,
118
+ model: modelId,
119
+ provider,
120
+ tokenizer: 'heuristic',
121
+ };
122
+ };
123
+ export const main = async (argv, deps = {}) => {
124
+ const stdout = deps.stdout ?? process.stdout;
125
+ const stderr = deps.stderr ?? process.stderr;
126
+ const reader = deps.imageSizeReader ?? defaultImageSizeReader;
34
127
  let parsed;
35
128
  try {
36
129
  parsed = parseArgs(argv);
37
130
  }
38
131
  catch (err) {
39
- process.stderr.write(`${err.message}\n\n${HELP_TEXT}`);
132
+ stderr.write(`${err.message}\n\n${HELP_TEXT}`);
40
133
  return 2;
41
134
  }
42
135
  if (parsed.help) {
43
- process.stdout.write(HELP_TEXT);
136
+ stdout.write(HELP_TEXT);
44
137
  return 0;
45
138
  }
46
139
  if (parsed.version) {
47
- process.stdout.write(`tokenometer ${VERSION}\n`);
140
+ stdout.write(`tokenometer ${VERSION}\n`);
48
141
  return 0;
49
142
  }
50
- let prompt;
51
- try {
52
- prompt = await readPrompt(parsed.inputPaths);
143
+ // Apply config defaults (before auto-detect).
144
+ if (!parsed.noConfig) {
145
+ try {
146
+ if (parsed.configPath) {
147
+ const cfg = await loadConfigFromPath(parsed.configPath);
148
+ parsed = applyConfig(parsed, { config: cfg });
149
+ }
150
+ else {
151
+ const cfg = await loadConfig();
152
+ parsed = applyConfig(parsed, { config: cfg });
153
+ }
154
+ }
155
+ catch (err) {
156
+ stderr.write(`${err.message}\n`);
157
+ return 1;
158
+ }
53
159
  }
54
- catch (err) {
55
- process.stderr.write(`Failed to read input: ${err.message}\n`);
56
- return 1;
160
+ // Auto-detect default model when neither user nor config set --model.
161
+ if (!parsed.modelsSet) {
162
+ const detected = autoDetectDefaultModel();
163
+ parsed.modelIds = [detected.modelId];
164
+ if (detected.note)
165
+ stderr.write(`${detected.note}\n`);
57
166
  }
58
- if (!prompt.trim()) {
59
- process.stderr.write('Empty prompt nothing to measure.\n');
167
+ // Validate that we have at least one input source. (Prompt files may come
168
+ // from positional args or config.paths; images are optional but if they're
169
+ // the only input, that's a misuse.)
170
+ if (parsed.inputPaths.length === 0 && parsed.imagePaths.length === 0) {
171
+ stderr.write('No input files. Pass a path, "-" for stdin, or set paths in .tokenometer.yml.\n');
60
172
  return 1;
61
173
  }
62
174
  const useEmpirical = parsed.empirical && !parsed.offline;
63
- const results = useEmpirical
64
- ? await tokenizeMatrixEmpirical({
65
- env: readEnv(),
66
- formats: parsed.formats,
67
- modelIds: parsed.modelIds,
68
- prompt,
69
- })
70
- : tokenizeMatrix({
71
- formats: parsed.formats,
72
- modelIds: parsed.modelIds,
73
- prompt,
74
- });
75
- process.stdout.write(`${renderTable(results)}\n`);
76
- const limits = renderModelLimits(results);
175
+ // For json/sarif output, we MUST have per-file results.
176
+ // For table output, if --by-file or --image is on, we need per-file too.
177
+ // Otherwise the existing joined-prompt path is used.
178
+ if (parsed.output === 'json' || parsed.output === 'sarif') {
179
+ const fileResults = parsed.inputPaths.length > 0
180
+ ? await buildPerFileResults(parsed, useEmpirical, deps.measureLatencyFn)
181
+ : [];
182
+ const imageResults = await buildImageResults(parsed, reader);
183
+ const result = { files: [...fileResults, ...imageResults] };
184
+ if (result.files.length === 0) {
185
+ stderr.write('Empty prompt — nothing to measure.\n');
186
+ return 1;
187
+ }
188
+ if (parsed.output === 'json') {
189
+ stdout.write(`${JSON.stringify(result, null, 2)}\n`);
190
+ }
191
+ else {
192
+ const sarif = toSarif(result, { toolVersion: VERSION });
193
+ stdout.write(`${JSON.stringify(sarif, null, 2)}\n`);
194
+ }
195
+ return 0;
196
+ }
197
+ // Table output path.
198
+ let prompt = '';
199
+ if (parsed.inputPaths.length > 0) {
200
+ try {
201
+ prompt = await readJoinedPrompt(parsed.inputPaths);
202
+ }
203
+ catch (err) {
204
+ stderr.write(`Failed to read input: ${err.message}\n`);
205
+ return 1;
206
+ }
207
+ }
208
+ // Allow image-only invocation when there are no prompt files.
209
+ if (parsed.inputPaths.length > 0 && !prompt.trim()) {
210
+ stderr.write('Empty prompt — nothing to measure.\n');
211
+ return 1;
212
+ }
213
+ let mainResults = [];
214
+ if (parsed.inputPaths.length > 0) {
215
+ mainResults = useEmpirical
216
+ ? await tokenizeMatrixEmpirical({
217
+ env: readEnv(),
218
+ formats: parsed.formats,
219
+ modelIds: parsed.modelIds,
220
+ prompt,
221
+ })
222
+ : tokenizeMatrix({
223
+ formats: parsed.formats,
224
+ modelIds: parsed.modelIds,
225
+ prompt,
226
+ });
227
+ if (parsed.latency) {
228
+ await augmentWithLatency(mainResults, prompt, parsed, deps.measureLatencyFn ?? measureLatency);
229
+ }
230
+ }
231
+ // Compute image rows up front (needed for main table appendage and by-file).
232
+ const imageFileResults = await buildImageResults(parsed, reader);
233
+ const imageCells = imageFileResults.flatMap((f) => f.results);
234
+ const allMainCells = [...mainResults, ...imageCells];
235
+ stdout.write(`${renderTable(allMainCells)}\n`);
236
+ const limits = renderModelLimits(allMainCells);
77
237
  if (limits)
78
- process.stdout.write(`${limits}\n`);
79
- const summary = renderSummary(results);
238
+ stdout.write(`${limits}\n`);
239
+ const summary = renderSummary(allMainCells);
80
240
  if (summary)
81
- process.stdout.write(`${summary}\n`);
241
+ stdout.write(`${summary}\n`);
242
+ // by-file table requires per-file results from prompt files plus image virtual files.
243
+ if (parsed.byFile) {
244
+ const perFile = parsed.inputPaths.length > 0
245
+ ? await buildPerFileResults(parsed, useEmpirical, deps.measureLatencyFn)
246
+ : [];
247
+ const allFiles = [...perFile, ...imageFileResults];
248
+ const byFile = renderByFile(allFiles);
249
+ if (byFile)
250
+ stdout.write(`${byFile}\n`);
251
+ }
82
252
  if (useEmpirical) {
83
- process.stdout.write('\n(empirical: Anthropic / Google counts via provider countTokens API; OpenAI via tiktoken o200k_base)\n');
253
+ stdout.write('\n(empirical: Anthropic / Google counts via provider countTokens API; OpenAI via tiktoken o200k_base)\n');
254
+ }
255
+ if (parsed.latency) {
256
+ stdout.write(`(latency: ${parsed.latencyTrials} streaming generation${parsed.latencyTrials === 1 ? '' : 's'} per cell, max_tokens=200; p50/p95/mean over trials)\n`);
84
257
  }
85
258
  return 0;
86
259
  };
package/dist/index.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AACvC,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AACzC,OAAO,EAAE,cAAc,EAAE,uBAAuB,EAAE,MAAM,mBAAmB,CAAC;AAE5E,OAAO,EAAE,SAAS,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACjD,OAAO,EAAE,iBAAiB,EAAE,aAAa,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAE5E,MAAM,OAAO,GAAG,OAAO,CAAC;AAExB,MAAM,OAAO,GAAG,GAAiB,EAAE;IACjC,MAAM,GAAG,GAAiB,EAAE,CAAC;IAC7B,MAAM,EAAE,iBAAiB,EAAE,cAAc,EAAE,cAAc,EAAE,GAAG,OAAO,CAAC,GAAG,CAAC;IAC1E,IAAI,iBAAiB;QAAE,GAAG,CAAC,eAAe,GAAG,iBAAiB,CAAC;IAC/D,MAAM,SAAS,GAAG,cAAc,IAAI,cAAc,CAAC;IACnD,IAAI,SAAS;QAAE,GAAG,CAAC,YAAY,GAAG,SAAS,CAAC;IAC5C,OAAO,GAAG,CAAC;AACb,CAAC,CAAC;AAEF,MAAM,SAAS,GAAG,KAAK,IAAqB,EAAE;IAC5C,MAAM,MAAM,GAAa,EAAE,CAAC;IAC5B,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,OAAO,CAAC,KAAK,EAAE,CAAC;QACxC,MAAM,CAAC,IAAI,CAAC,KAAe,CAAC,CAAC;IAC/B,CAAC;IACD,OAAO,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;AAChD,CAAC,CAAC;AAEF,MAAM,UAAU,GAAG,KAAK,EAAE,KAAwB,EAAmB,EAAE;IACrE,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;QAC3C,OAAO,SAAS,EAAE,CAAC;IACrB,CAAC;IACD,MAAM,QAAQ,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC;IAC1E,OAAO,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC7B,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,IAAI,GAAG,KAAK,EAAE,IAAuB,EAAmB,EAAE;IACrE,IAAI,MAAoC,CAAC;IACzC,IAAI,CAAC;QACH,MAAM,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;IAC3B,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAI,GAAa,CAAC,OAAO,OAAO,SAAS,EAAE,CAAC,CAAC;QAClE,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,MAAM,CAAC,IAAI,EAAE,CAAC;QAChB,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC;QAChC,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;QACnB,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,eAAe,OAAO,IAAI,CAAC,CAAC;QACjD,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,MAAc,CAAC;IACnB,IAAI,CAAC;QACH,MAAM,GAAG,MAAM,UAAU,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;IAC/C,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,yBAA0B,GAAa,CAAC,OAAO,IAAI,CAAC,CAAC;QAC1E,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;QACnB,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,sCAAsC,CAAC,CAAC;QAC7D,OAAO,CAAC,CAAC;IACX,CAAC;IAED,MAAM,YAAY,GAAG,MAAM,CAAC,SAAS,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC;IACzD,MAAM,OAAO,GAAG,YAAY;QAC1B,CAAC,CAAC,MAAM,uBAAuB,CAAC;YAC5B,GAAG,EAAE,OAAO,EAAE;YACd,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,QAAQ,EAAE,MAAM,CAAC,QAAQ;YACzB,MAAM;SACP,CAAC;QACJ,CAAC,CAAC,cAAc,CAAC;YACb,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,QAAQ,EAAE,MAAM,CAAC,QAAQ;YACzB,MAAM;SACP,CAAC,CAAC;IAEP,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,WAAW,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;IAClD,MAAM,MAAM,GAAG,iBAAiB,CAAC,OAAO,CAAC,CAAC;IAC1C,IAAI,MAAM;QAAE,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,MAAM,IAAI,CAAC,CAAC;IAChD,MAAM,OAAO,GAAG,aAAa,CAAC,OAAO,CAAC,CAAC;IACvC,IAAI,OAAO;QAAE,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;IAClD,IAAI,YAAY,EAAE,CAAC;QACjB,OAAO,CAAC,MAAM,CAAC,KAAK,CAClB,yGAAyG,CAC1G,CAAC;IACJ,CAAC;IACD,OAAO,CAAC,CAAC;AACX,CAAC,CAAC;AAEF,MAAM,iBAAiB,GAAG,GAAY,EAAE;IACtC,MAAM,KAAK,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IAC9B,IAAI,CAAC,KAAK;QAAE,OAAO,KAAK,CAAC;IACzB,IAAI,CAAC;QACH,OAAO,aAAa,CAAC,YAAY,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC;IACrE,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC,CAAC;AAEF,IAAI,iBAAiB,EAAE,EAAE,CAAC;IACxB,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAC9B,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,EAC5B,CAAC,GAAY,EAAE,EAAE;QACf,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,qBAAsB,GAAa,CAAC,KAAK,IAAI,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACnF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IAClB,CAAC,CACF,CAAC;AACJ,CAAC"}
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AACvC,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AACzC,OAAO,EAGL,QAAQ,EACR,OAAO,EACP,UAAU,EACV,cAAc,EACd,OAAO,EACP,cAAc,EACd,uBAAuB,GACxB,MAAM,mBAAmB,CAAC;AAQ3B,OAAO,EAAE,SAAS,EAAmB,SAAS,EAAE,MAAM,WAAW,CAAC;AAClE,OAAO,EAAE,sBAAsB,EAAE,MAAM,kBAAkB,CAAC;AAC1D,OAAO,EAAE,WAAW,EAAE,kBAAkB,EAAE,MAAM,mBAAmB,CAAC;AACpE,OAAO,EAAE,YAAY,EAAE,iBAAiB,EAAE,aAAa,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1F,OAAO,EAEL,mBAAmB,EACnB,sBAAsB,EACtB,aAAa,GACd,MAAM,aAAa,CAAC;AAErB,MAAM,OAAO,GAAG,OAAO,CAAC;AAExB,MAAM,OAAO,GAAG,GAAiB,EAAE;IACjC,MAAM,GAAG,GAAiB,EAAE,CAAC;IAC7B,MAAM,EACJ,iBAAiB,EACjB,cAAc,EACd,cAAc,EACd,cAAc,EACd,eAAe,EACf,cAAc,GACf,GAAG,OAAO,CAAC,GAAG,CAAC;IAChB,IAAI,iBAAiB;QAAE,GAAG,CAAC,eAAe,GAAG,iBAAiB,CAAC;IAC/D,IAAI,cAAc;QAAE,GAAG,CAAC,YAAY,GAAG,cAAc,CAAC;IACtD,MAAM,SAAS,GAAG,cAAc,IAAI,cAAc,CAAC;IACnD,IAAI,SAAS;QAAE,GAAG,CAAC,YAAY,GAAG,SAAS,CAAC;IAC5C,IAAI,eAAe;QAAE,GAAG,CAAC,aAAa,GAAG,eAAe,CAAC;IACzD,IAAI,cAAc;QAAE,GAAG,CAAC,YAAY,GAAG,cAAc,CAAC;IACtD,OAAO,GAAG,CAAC;AACb,CAAC,CAAC;AAEF,MAAM,SAAS,GAAG,KAAK,IAAqB,EAAE;IAC5C,MAAM,MAAM,GAAa,EAAE,CAAC;IAC5B,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,OAAO,CAAC,KAAK,EAAE,CAAC;QACxC,MAAM,CAAC,IAAI,CAAC,KAAe,CAAC,CAAC;IAC/B,CAAC;IACD,OAAO,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;AAChD,CAAC,CAAC;AAEF,MAAM,gBAAgB,GAAG,KAAK,EAAE,KAAwB,EAAmB,EAAE;IAC3E,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;QAC3C,OAAO,SAAS,EAAE,CAAC;IACrB,CAAC;IACD,MAAM,QAAQ,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC;IAC1E,OAAO,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC7B,CAAC,CAAC;AAEF,MAAM,kBAAkB,GAAG,KAAK,EAC9B,KAAwB,EACqB,EAAE;IAC/C,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;QAC3C,MAAM,MAAM,GAAG,MAAM,SAAS,EAAE,CAAC;QACjC,OAAO,CAAC,EAAE,IAAI,EAAE,GAAG,EAAE,MAAM,EAAE,CAAC,CAAC;IACjC,CAAC;IACD,OAAO,OAAO,CAAC,GAAG,CAChB,KAAK,CAAC,GAAG,CAAC,KAAK,EAAE,IAAI,EAAE,EAAE;QACvB,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC;QAC5C,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC;IAC1B,CAAC,CAAC,CACH,CAAC;AACJ,CAAC,CAAC;AAYF;;;;GAIG;AACH,MAAM,kBAAkB,GAAG,KAAK,EAC9B,KAAuB,EACvB,MAAc,EACd,MAAkB,EAClB,OAAyB,EACV,EAAE;IACjB,sEAAsE;IACtE,qDAAqD;IACrD,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;QACzB,IAAI,CAAC,OAAO,GAAG,MAAM,OAAO,CAAC;YAC3B,GAAG,EAAE,OAAO,EAAE;YACd,OAAO,EAAE,IAAI,CAAC,KAAK;YACnB,MAAM;YACN,MAAM,EAAE,MAAM,CAAC,aAAa;SAC7B,CAAC,CAAC;IACL,CAAC;AACH,CAAC,CAAC;AAEF,MAAM,mBAAmB,GAAG,KAAK,EAC/B,MAAkB,EAClB,YAAqB,EACrB,gBAA8C,EACZ,EAAE;IACpC,MAAM,MAAM,GAAG,MAAM,kBAAkB,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;IAC3D,MAAM,GAAG,GAA4B,EAAE,CAAC;IACxC,KAAK,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,MAAM,EAAE,CAAC;QACtC,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE;YAAE,SAAS;QAC7B,MAAM,KAAK,GAAqB,YAAY;YAC1C,CAAC,CAAC,MAAM,uBAAuB,CAAC;gBAC5B,GAAG,EAAE,OAAO,EAAE;gBACd,OAAO,EAAE,MAAM,CAAC,OAAO;gBACvB,QAAQ,EAAE,MAAM,CAAC,QAAQ;gBACzB,MAAM;aACP,CAAC;YACJ,CAAC,CAAC,cAAc,CAAC;gBACb,OAAO,EAAE,MAAM,CAAC,OAAO;gBACvB,QAAQ,EAAE,MAAM,CAAC,QAAQ;gBACzB,MAAM;aACP,CAAC,CAAC;QACP,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;YACnB,MAAM,kBAAkB,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,gBAAgB,IAAI,cAAc,CAAC,CAAC;QACtF,CAAC;QACD,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC,CAAC;IACrC,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC,CAAC;AAEF,MAAM,iBAAiB,GAAG,KAAK,EAC7B,MAAkB,EAClB,MAAuB,EACW,EAAE;IACpC,IAAI,MAAM,CAAC,UAAU,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IAC9C,MAAM,QAAQ,GAAG,MAAM,aAAa,CAAC,MAAM,CAAC,UAAU,EAAE,MAAM,CAAC,CAAC;IAChE,MAAM,OAAO,GAA4B,EAAE,CAAC;IAC5C,KAAK,MAAM,GAAG,IAAI,QAAQ,EAAE,CAAC;QAC3B,MAAM,KAAK,GAAqB,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE;YAC9D,MAAM,MAAM,GAAG,mBAAmB,CAAC,OAAO,EAAE,GAAG,CAAC,GAAG,EAAE,GAAG,CAAC,IAAI,CAAC,CAAC;YAC/D,OAAO,cAAc,CAAC,OAAO,EAAE,MAAM,CAAC,CAAC;QACzC,CAAC,CAAC,CAAC;QACH,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,GAAG,GAAG,CAAC,IAAI,WAAW,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC,CAAC;IACjE,CAAC;IACD,OAAO,OAAO,CAAC;AACjB,CAAC,CAAC;AAEF,MAAM,cAAc,GAAG,CAAC,OAAe,EAAE,MAAc,EAAkB,EAAE;IACzE,2EAA2E;IAC3E,sDAAsD;IACtD,MAAM,IAAI,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;IAC9B,MAAM,QAAQ,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC,QAAQ,CAAC;IAC5C,OAAO;QACL,WAAW,EAAE,IAAI;QACjB,MAAM,EAAE,MAAgB;QACxB,SAAS,EAAE,CAAC,MAAM,GAAG,IAAI,CAAC,GAAG,IAAI,CAAC,UAAU;QAC5C,WAAW,EAAE,MAAM;QACnB,KAAK,EAAE,OAAO;QACd,QAAQ;QACR,SAAS,EAAE,WAAW;KACvB,CAAC;AACJ,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,IAAI,GAAG,KAAK,EAAE,IAAuB,EAAE,OAAgB,EAAE,EAAmB,EAAE;IACzF,MAAM,MAAM,GAAG,IAAI,CAAC,MAAM,IAAI,OAAO,CAAC,MAAM,CAAC;IAC7C,MAAM,MAAM,GAAG,IAAI,CAAC,MAAM,IAAI,OAAO,CAAC,MAAM,CAAC;IAC7C,MAAM,MAAM,GAAG,IAAI,CAAC,eAAe,IAAI,sBAAsB,CAAC;IAE9D,IAAI,MAAkB,CAAC;IACvB,IAAI,CAAC;QACH,MAAM,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;IAC3B,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,MAAM,CAAC,KAAK,CAAC,GAAI,GAAa,CAAC,OAAO,OAAO,SAAS,EAAE,CAAC,CAAC;QAC1D,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,MAAM,CAAC,IAAI,EAAE,CAAC;QAChB,MAAM,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC;QACxB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;QACnB,MAAM,CAAC,KAAK,CAAC,eAAe,OAAO,IAAI,CAAC,CAAC;QACzC,OAAO,CAAC,CAAC;IACX,CAAC;IAED,8CAA8C;IAC9C,IAAI,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC;QACrB,IAAI,CAAC;YACH,IAAI,MAAM,CAAC,UAAU,EAAE,CAAC;gBACtB,MAAM,GAAG,GAAG,MAAM,kBAAkB,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;gBACxD,MAAM,GAAG,WAAW,CAAC,MAAM,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,CAAC,CAAC;YAChD,CAAC;iBAAM,CAAC;gBACN,MAAM,GAAG,GAAG,MAAM,UAAU,EAAE,CAAC;gBAC/B,MAAM,GAAG,WAAW,CAAC,MAAM,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,CAAC,CAAC;YAChD,CAAC;QACH,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,CAAC,KAAK,CAAC,GAAI,GAAa,CAAC,OAAO,IAAI,CAAC,CAAC;YAC5C,OAAO,CAAC,CAAC;QACX,CAAC;IACH,CAAC;IAED,sEAAsE;IACtE,IAAI,CAAC,MAAM,CAAC,SAAS,EAAE,CAAC;QACtB,MAAM,QAAQ,GAAG,sBAAsB,EAAE,CAAC;QAC1C,MAAM,CAAC,QAAQ,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;QACrC,IAAI,QAAQ,CAAC,IAAI;YAAE,MAAM,CAAC,KAAK,CAAC,GAAG,QAAQ,CAAC,IAAI,IAAI,CAAC,CAAC;IACxD,CAAC;IAED,0EAA0E;IAC1E,2EAA2E;IAC3E,oCAAoC;IACpC,IAAI,MAAM,CAAC,UAAU,CAAC,MAAM,KAAK,CAAC,IAAI,MAAM,CAAC,UAAU,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACrE,MAAM,CAAC,KAAK,CAAC,iFAAiF,CAAC,CAAC;QAChG,OAAO,CAAC,CAAC;IACX,CAAC;IAED,MAAM,YAAY,GAAG,MAAM,CAAC,SAAS,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC;IAEzD,wDAAwD;IACxD,yEAAyE;IACzE,qDAAqD;IACrD,IAAI,MAAM,CAAC,MAAM,KAAK,MAAM,IAAI,MAAM,CAAC,MAAM,KAAK,OAAO,EAAE,CAAC;QAC1D,MAAM,WAAW,GACf,MAAM,CAAC,UAAU,CAAC,MAAM,GAAG,CAAC;YAC1B,CAAC,CAAC,MAAM,mBAAmB,CAAC,MAAM,EAAE,YAAY,EAAE,IAAI,CAAC,gBAAgB,CAAC;YACxE,CAAC,CAAC,EAAE,CAAC;QACT,MAAM,YAAY,GAAG,MAAM,iBAAiB,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;QAC7D,MAAM,MAAM,GAAsB,EAAE,KAAK,EAAE,CAAC,GAAG,WAAW,EAAE,GAAG,YAAY,CAAC,EAAE,CAAC;QAC/E,IAAI,MAAM,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC9B,MAAM,CAAC,KAAK,CAAC,sCAAsC,CAAC,CAAC;YACrD,OAAO,CAAC,CAAC;QACX,CAAC;QACD,IAAI,MAAM,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;YAC7B,MAAM,CAAC,KAAK,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;QACvD,CAAC;aAAM,CAAC;YACN,MAAM,KAAK,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,WAAW,EAAE,OAAO,EAAE,CAAC,CAAC;YACxD,MAAM,CAAC,KAAK,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;QACtD,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC;IAED,qBAAqB;IACrB,IAAI,MAAM,GAAG,EAAE,CAAC;IAChB,IAAI,MAAM,CAAC,UAAU,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACjC,IAAI,CAAC;YACH,MAAM,GAAG,MAAM,gBAAgB,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;QACrD,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,CAAC,KAAK,CAAC,yBAA0B,GAAa,CAAC,OAAO,IAAI,CAAC,CAAC;YAClE,OAAO,CAAC,CAAC;QACX,CAAC;IACH,CAAC;IAED,8DAA8D;IAC9D,IAAI,MAAM,CAAC,UAAU,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;QACnD,MAAM,CAAC,KAAK,CAAC,sCAAsC,CAAC,CAAC;QACrD,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,WAAW,GAAqB,EAAE,CAAC;IACvC,IAAI,MAAM,CAAC,UAAU,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACjC,WAAW,GAAG,YAAY;YACxB,CAAC,CAAC,MAAM,uBAAuB,CAAC;gBAC5B,GAAG,EAAE,OAAO,EAAE;gBACd,OAAO,EAAE,MAAM,CAAC,OAAO;gBACvB,QAAQ,EAAE,MAAM,CAAC,QAAQ;gBACzB,MAAM;aACP,CAAC;YACJ,CAAC,CAAC,cAAc,CAAC;gBACb,OAAO,EAAE,MAAM,CAAC,OAAO;gBACvB,QAAQ,EAAE,MAAM,CAAC,QAAQ;gBACzB,MAAM;aACP,CAAC,CAAC;QACP,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;YACnB,MAAM,kBAAkB,CACtB,WAAW,EACX,MAAM,EACN,MAAM,EACN,IAAI,CAAC,gBAAgB,IAAI,cAAc,CACxC,CAAC;QACJ,CAAC;IACH,CAAC;IAED,6EAA6E;IAC7E,MAAM,gBAAgB,GAAG,MAAM,iBAAiB,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACjE,MAAM,UAAU,GAAG,gBAAgB,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;IAC9D,MAAM,YAAY,GAAqB,CAAC,GAAG,WAAW,EAAE,GAAG,UAAU,CAAC,CAAC;IAEvE,MAAM,CAAC,KAAK,CAAC,GAAG,WAAW,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC;IAC/C,MAAM,MAAM,GAAG,iBAAiB,CAAC,YAAY,CAAC,CAAC;IAC/C,IAAI,MAAM;QAAE,MAAM,CAAC,KAAK,CAAC,GAAG,MAAM,IAAI,CAAC,CAAC;IACxC,MAAM,OAAO,GAAG,aAAa,CAAC,YAAY,CAAC,CAAC;IAC5C,IAAI,OAAO;QAAE,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;IAE1C,sFAAsF;IACtF,IAAI,MAAM,CAAC,MAAM,EAAE,CAAC;QAClB,MAAM,OAAO,GACX,MAAM,CAAC,UAAU,CAAC,MAAM,GAAG,CAAC;YAC1B,CAAC,CAAC,MAAM,mBAAmB,CAAC,MAAM,EAAE,YAAY,EAAE,IAAI,CAAC,gBAAgB,CAAC;YACxE,CAAC,CAAC,EAAE,CAAC;QACT,MAAM,QAAQ,GAAG,CAAC,GAAG,OAAO,EAAE,GAAG,gBAAgB,CAAC,CAAC;QACnD,MAAM,MAAM,GAAG,YAAY,CAAC,QAAQ,CAAC,CAAC;QACtC,IAAI,MAAM;YAAE,MAAM,CAAC,KAAK,CAAC,GAAG,MAAM,IAAI,CAAC,CAAC;IAC1C,CAAC;IAED,IAAI,YAAY,EAAE,CAAC;QACjB,MAAM,CAAC,KAAK,CACV,yGAAyG,CAC1G,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;QACnB,MAAM,CAAC,KAAK,CACV,aAAa,MAAM,CAAC,aAAa,wBAAwB,MAAM,CAAC,aAAa,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,wDAAwD,CACvJ,CAAC;IACJ,CAAC;IACD,OAAO,CAAC,CAAC;AACX,CAAC,CAAC;AAEF,MAAM,iBAAiB,GAAG,GAAY,EAAE;IACtC,MAAM,KAAK,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IAC9B,IAAI,CAAC,KAAK;QAAE,OAAO,KAAK,CAAC;IACzB,IAAI,CAAC;QACH,OAAO,aAAa,CAAC,YAAY,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,KAAK,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC;IACrE,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC,CAAC;AAEF,IAAI,iBAAiB,EAAE,EAAE,CAAC;IACxB,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAC9B,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,EAC5B,CAAC,GAAY,EAAE,EAAE;QACf,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,qBAAsB,GAAa,CAAC,KAAK,IAAI,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACnF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IAClB,CAAC,CACF,CAAC;AACJ,CAAC"}
package/dist/render.d.ts CHANGED
@@ -1,5 +1,10 @@
1
- import type { TokenizeResult } from '@tokenometer/core';
1
+ import type { TokenizeResult, TokenometerFileResult } from '@tokenometer/core';
2
2
  export declare const renderTable: (results: readonly TokenizeResult[]) => string;
3
3
  export declare const renderModelLimits: (results: readonly TokenizeResult[]) => string;
4
4
  export declare const renderSummary: (results: readonly TokenizeResult[]) => string;
5
+ /**
6
+ * Per-file token + cost summary table. One row per input file (or virtual
7
+ * file for `--image` entries). No-op when there's only one file.
8
+ */
9
+ export declare const renderByFile: (files: readonly TokenometerFileResult[]) => string;
5
10
  //# sourceMappingURL=render.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"render.d.ts","sourceRoot":"","sources":["../src/render.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,mBAAmB,CAAC;AAcxD,eAAO,MAAM,WAAW,GAAI,SAAS,SAAS,cAAc,EAAE,KAAG,MA8BhE,CAAC;AAQF,eAAO,MAAM,iBAAiB,GAAI,SAAS,SAAS,cAAc,EAAE,KAAG,MAgBtE,CAAC;AAEF,eAAO,MAAM,aAAa,GAAI,SAAS,SAAS,cAAc,EAAE,KAAG,MAOlE,CAAC"}
1
+ {"version":3,"file":"render.d.ts","sourceRoot":"","sources":["../src/render.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,cAAc,EAAE,qBAAqB,EAAE,MAAM,mBAAmB,CAAC;AAmB/E,eAAO,MAAM,WAAW,GAAI,SAAS,SAAS,cAAc,EAAE,KAAG,MA+ChE,CAAC;AAQF,eAAO,MAAM,iBAAiB,GAAI,SAAS,SAAS,cAAc,EAAE,KAAG,MAgBtE,CAAC;AAEF,eAAO,MAAM,aAAa,GAAI,SAAS,SAAS,cAAc,EAAE,KAAG,MAOlE,CAAC;AAEF;;;GAGG;AACH,eAAO,MAAM,YAAY,GAAI,OAAO,SAAS,qBAAqB,EAAE,KAAG,MA2BtE,CAAC"}
package/dist/render.js CHANGED
@@ -8,16 +8,35 @@ const formatCost = (usd) => {
8
8
  return `$${usd.toFixed(6)}`;
9
9
  return `$${usd.toExponential(2)}`;
10
10
  };
11
+ const formatMs = (ms) => `${Math.round(ms)}`;
12
+ const formatTps = (tps) => tps >= 100 ? Math.round(tps).toString() : tps.toFixed(1);
11
13
  export const renderTable = (results) => {
12
14
  if (results.length === 0)
13
15
  return '(no results)';
14
- const headers = ['model', 'format', 'tokens', 'est. cost'];
15
- const rows = results.map((r) => [
16
- r.model,
17
- r.format,
18
- `${r.approximate ? '~' : ' '}${r.inputTokens.toLocaleString()}`,
19
- formatCost(r.inputCost),
20
- ]);
16
+ // Latency columns are only added when at least one cell has latency data.
17
+ const hasLatency = results.some((r) => r.latency !== undefined);
18
+ const headers = hasLatency
19
+ ? ['model', 'format', 'tokens', 'est. cost', 'p50 ttft', 'p50 total', 'tokens/s']
20
+ : ['model', 'format', 'tokens', 'est. cost'];
21
+ const rows = results.map((r) => {
22
+ const base = [
23
+ r.model,
24
+ r.format,
25
+ `${r.approximate ? '~' : ' '}${r.inputTokens.toLocaleString()}`,
26
+ formatCost(r.inputCost),
27
+ ];
28
+ if (!hasLatency)
29
+ return base;
30
+ if (r.latency) {
31
+ return [
32
+ ...base,
33
+ `${formatMs(r.latency.p50.ttftMs)} ms`,
34
+ `${formatMs(r.latency.p50.totalMs)} ms`,
35
+ formatTps(r.latency.p50.tokensPerSec),
36
+ ];
37
+ }
38
+ return [...base, '-', '-', '-'];
39
+ });
21
40
  const widths = headers.map((h, colIdx) => {
22
41
  const maxRowWidth = rows.reduce((acc, row) => Math.max(acc, row[colIdx]?.length ?? 0), 0);
23
42
  return Math.max(h.length, maxRowWidth);
@@ -74,4 +93,33 @@ export const renderSummary = (results) => {
74
93
  const ratio = priciest.inputCost / Math.max(cheapest.inputCost, Number.EPSILON);
75
94
  return `\nCheapest: ${cheapest.model} as ${cheapest.format} (${formatCost(cheapest.inputCost)})\nPriciest: ${priciest.model} as ${priciest.format} (${formatCost(priciest.inputCost)}, ${ratio.toFixed(2)}x more)`;
76
95
  };
96
+ /**
97
+ * Per-file token + cost summary table. One row per input file (or virtual
98
+ * file for `--image` entries). No-op when there's only one file.
99
+ */
100
+ export const renderByFile = (files) => {
101
+ if (files.length <= 1)
102
+ return '';
103
+ const headers = ['File', 'Tokens', 'USD'];
104
+ const rows = files.map((f) => {
105
+ const tokens = f.results.reduce((acc, r) => acc + r.inputTokens, 0);
106
+ const cost = f.results.reduce((acc, r) => acc + r.inputCost, 0);
107
+ return [f.path, tokens.toLocaleString(), formatCost(cost)];
108
+ });
109
+ const widths = headers.map((h, colIdx) => {
110
+ const maxRowWidth = rows.reduce((acc, row) => Math.max(acc, row[colIdx]?.length ?? 0), 0);
111
+ return Math.max(h.length, maxRowWidth);
112
+ });
113
+ const sep = headers.map((_, i) => '─'.repeat(widths[i] ?? 0)).join(' ');
114
+ const headerLine = headers.map((h, i) => padRight(h, widths[i] ?? h.length)).join(' ');
115
+ const dataLines = rows.map((row) => row
116
+ .map((cell, i) => {
117
+ const isNumeric = i >= 1;
118
+ return isNumeric
119
+ ? padLeft(cell, widths[i] ?? cell.length)
120
+ : padRight(cell, widths[i] ?? cell.length);
121
+ })
122
+ .join(' '));
123
+ return ['\nBy file:', ` ${headerLine}`, ` ${sep}`, ...dataLines.map((l) => ` ${l}`)].join('\n');
124
+ };
77
125
  //# sourceMappingURL=render.js.map
@@ -1 +1 @@
1
- {"version":3,"file":"render.js","sourceRoot":"","sources":["../src/render.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,mBAAmB,CAAC;AAG7C,MAAM,QAAQ,GAAG,CAAC,KAAa,EAAE,KAAa,EAAU,EAAE,CACxD,KAAK,CAAC,MAAM,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,GAAG,CAAC,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC;AAE3E,MAAM,OAAO,GAAG,CAAC,KAAa,EAAE,KAAa,EAAU,EAAE,CACvD,KAAK,CAAC,MAAM,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC,GAAG,KAAK,CAAC;AAE3E,MAAM,UAAU,GAAG,CAAC,GAAW,EAAU,EAAE;IACzC,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;IAC7C,IAAI,GAAG,IAAI,QAAQ;QAAE,OAAO,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;IACjD,OAAO,IAAI,GAAG,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,CAAC;AACpC,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,WAAW,GAAG,CAAC,OAAkC,EAAU,EAAE;IACxE,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,cAAc,CAAC;IAEhD,MAAM,OAAO,GAAG,CAAC,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,WAAW,CAAU,CAAC;IACpE,MAAM,IAAI,GAAe,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC;QAC1C,CAAC,CAAC,KAAK;QACP,CAAC,CAAC,MAAM;QACR,GAAG,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,WAAW,CAAC,cAAc,EAAE,EAAE;QAC/D,UAAU,CAAC,CAAC,CAAC,SAAS,CAAC;KACxB,CAAC,CAAC;IAEH,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE;QACvC,MAAM,WAAW,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,GAAG,CAAC,MAAM,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;QAC1F,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,MAAM,EAAE,WAAW,CAAC,CAAC;IACzC,CAAC,CAAC,CAAC;IAEH,MAAM,UAAU,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACxF,MAAM,SAAS,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC/E,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CACjC,GAAG;SACA,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;QACf,MAAM,SAAS,GAAG,CAAC,IAAI,CAAC,CAAC;QACzB,OAAO,SAAS;YACd,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,MAAM,CAAC;YACzC,CAAC,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,MAAM,CAAC,CAAC;IAC/C,CAAC,CAAC;SACD,IAAI,CAAC,IAAI,CAAC,CACd,CAAC;IAEF,OAAO,CAAC,UAAU,EAAE,SAAS,EAAE,GAAG,SAAS,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1D,CAAC,CAAC;AAEF,MAAM,gBAAgB,GAAG,CAAC,CAAS,EAAU,EAAE;IAC7C,IAAI,CAAC,IAAI,SAAS;QAAE,OAAO,GAAG,CAAC,CAAC,GAAG,SAAS,CAAC,CAAC,OAAO,CAAC,CAAC,GAAG,SAAS,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC;IACtF,IAAI,CAAC,IAAI,IAAI;QAAE,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC;IACjD,OAAO,GAAG,CAAC,EAAE,CAAC;AAChB,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,iBAAiB,GAAG,CAAC,OAAkC,EAAU,EAAE;IAC9E,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IACpC,MAAM,IAAI,GAAG,IAAI,GAAG,EAAU,CAAC;IAC/B,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,IAAI,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC;YAAE,SAAS;QAChC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;QAClB,MAAM,CAAC,GAAG,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;QAC5B,IAAI,CAAC,CAAC,CAAC,aAAa,IAAI,CAAC,CAAC,CAAC,eAAe;YAAE,SAAS;QACrD,MAAM,KAAK,GAAa,EAAE,CAAC;QAC3B,IAAI,CAAC,CAAC,aAAa;YAAE,KAAK,CAAC,IAAI,CAAC,OAAO,gBAAgB,CAAC,CAAC,CAAC,aAAa,CAAC,EAAE,CAAC,CAAC;QAC5E,IAAI,CAAC,CAAC,eAAe;YAAE,KAAK,CAAC,IAAI,CAAC,OAAO,gBAAgB,CAAC,CAAC,CAAC,eAAe,CAAC,EAAE,CAAC,CAAC;QAChF,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;IAC7D,CAAC;IACD,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IAClC,OAAO,cAAc,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;AAC1C,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,aAAa,GAAG,CAAC,OAAkC,EAAU,EAAE;IAC1E,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IACpC,MAAM,QAAQ,GAAG,CAAC,GAAG,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,SAAS,GAAG,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3E,MAAM,QAAQ,GAAG,CAAC,GAAG,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,SAAS,GAAG,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3E,IAAI,CAAC,QAAQ,IAAI,CAAC,QAAQ,IAAI,QAAQ,KAAK,QAAQ;QAAE,OAAO,EAAE,CAAC;IAC/D,MAAM,KAAK,GAAG,QAAQ,CAAC,SAAS,GAAG,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC,OAAO,CAAC,CAAC;IAChF,OAAO,eAAe,QAAQ,CAAC,KAAK,OAAO,QAAQ,CAAC,MAAM,KAAK,UAAU,CAAC,QAAQ,CAAC,SAAS,CAAC,gBAAgB,QAAQ,CAAC,KAAK,OAAO,QAAQ,CAAC,MAAM,KAAK,UAAU,CAAC,QAAQ,CAAC,SAAS,CAAC,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC;AACrN,CAAC,CAAC"}
1
+ {"version":3,"file":"render.js","sourceRoot":"","sources":["../src/render.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,mBAAmB,CAAC;AAG7C,MAAM,QAAQ,GAAG,CAAC,KAAa,EAAE,KAAa,EAAU,EAAE,CACxD,KAAK,CAAC,MAAM,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,GAAG,CAAC,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC;AAE3E,MAAM,OAAO,GAAG,CAAC,KAAa,EAAE,KAAa,EAAU,EAAE,CACvD,KAAK,CAAC,MAAM,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC,GAAG,KAAK,CAAC;AAE3E,MAAM,UAAU,GAAG,CAAC,GAAW,EAAU,EAAE;IACzC,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;IAC7C,IAAI,GAAG,IAAI,QAAQ;QAAE,OAAO,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;IACjD,OAAO,IAAI,GAAG,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,CAAC;AACpC,CAAC,CAAC;AAEF,MAAM,QAAQ,GAAG,CAAC,EAAU,EAAU,EAAE,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,EAAE,CAAC;AAE7D,MAAM,SAAS,GAAG,CAAC,GAAW,EAAU,EAAE,CACxC,GAAG,IAAI,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;AAE3D,MAAM,CAAC,MAAM,WAAW,GAAG,CAAC,OAAkC,EAAU,EAAE;IACxE,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,cAAc,CAAC;IAEhD,0EAA0E;IAC1E,MAAM,UAAU,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC;IAEhE,MAAM,OAAO,GAAG,UAAU;QACxB,CAAC,CAAE,CAAC,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,WAAW,EAAE,UAAU,EAAE,WAAW,EAAE,UAAU,CAAW;QAC5F,CAAC,CAAE,CAAC,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,WAAW,CAAW,CAAC;IAC1D,MAAM,IAAI,GAAe,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;QACzC,MAAM,IAAI,GAAG;YACX,CAAC,CAAC,KAAK;YACP,CAAC,CAAC,MAAM;YACR,GAAG,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,WAAW,CAAC,cAAc,EAAE,EAAE;YAC/D,UAAU,CAAC,CAAC,CAAC,SAAS,CAAC;SACxB,CAAC;QACF,IAAI,CAAC,UAAU;YAAE,OAAO,IAAI,CAAC;QAC7B,IAAI,CAAC,CAAC,OAAO,EAAE,CAAC;YACd,OAAO;gBACL,GAAG,IAAI;gBACP,GAAG,QAAQ,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK;gBACtC,GAAG,QAAQ,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK;gBACvC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,YAAY,CAAC;aACtC,CAAC;QACJ,CAAC;QACD,OAAO,CAAC,GAAG,IAAI,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,CAAC;IAClC,CAAC,CAAC,CAAC;IAEH,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE;QACvC,MAAM,WAAW,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,GAAG,CAAC,MAAM,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;QAC1F,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,MAAM,EAAE,WAAW,CAAC,CAAC;IACzC,CAAC,CAAC,CAAC;IAEH,MAAM,UAAU,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACxF,MAAM,SAAS,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC/E,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CACjC,GAAG;SACA,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;QACf,MAAM,SAAS,GAAG,CAAC,IAAI,CAAC,CAAC;QACzB,OAAO,SAAS;YACd,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,MAAM,CAAC;YACzC,CAAC,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,MAAM,CAAC,CAAC;IAC/C,CAAC,CAAC;SACD,IAAI,CAAC,IAAI,CAAC,CACd,CAAC;IAEF,OAAO,CAAC,UAAU,EAAE,SAAS,EAAE,GAAG,SAAS,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1D,CAAC,CAAC;AAEF,MAAM,gBAAgB,GAAG,CAAC,CAAS,EAAU,EAAE;IAC7C,IAAI,CAAC,IAAI,SAAS;QAAE,OAAO,GAAG,CAAC,CAAC,GAAG,SAAS,CAAC,CAAC,OAAO,CAAC,CAAC,GAAG,SAAS,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC;IACtF,IAAI,CAAC,IAAI,IAAI;QAAE,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC;IACjD,OAAO,GAAG,CAAC,EAAE,CAAC;AAChB,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,iBAAiB,GAAG,CAAC,OAAkC,EAAU,EAAE;IAC9E,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IACpC,MAAM,IAAI,GAAG,IAAI,GAAG,EAAU,CAAC;IAC/B,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,IAAI,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC;YAAE,SAAS;QAChC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;QAClB,MAAM,CAAC,GAAG,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;QAC5B,IAAI,CAAC,CAAC,CAAC,aAAa,IAAI,CAAC,CAAC,CAAC,eAAe;YAAE,SAAS;QACrD,MAAM,KAAK,GAAa,EAAE,CAAC;QAC3B,IAAI,CAAC,CAAC,aAAa;YAAE,KAAK,CAAC,IAAI,CAAC,OAAO,gBAAgB,CAAC,CAAC,CAAC,aAAa,CAAC,EAAE,CAAC,CAAC;QAC5E,IAAI,CAAC,CAAC,eAAe;YAAE,KAAK,CAAC,IAAI,CAAC,OAAO,gBAAgB,CAAC,CAAC,CAAC,eAAe,CAAC,EAAE,CAAC,CAAC;QAChF,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;IAC7D,CAAC;IACD,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IAClC,OAAO,cAAc,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;AAC1C,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,aAAa,GAAG,CAAC,OAAkC,EAAU,EAAE;IAC1E,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IACpC,MAAM,QAAQ,GAAG,CAAC,GAAG,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,SAAS,GAAG,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3E,MAAM,QAAQ,GAAG,CAAC,GAAG,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,SAAS,GAAG,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3E,IAAI,CAAC,QAAQ,IAAI,CAAC,QAAQ,IAAI,QAAQ,KAAK,QAAQ;QAAE,OAAO,EAAE,CAAC;IAC/D,MAAM,KAAK,GAAG,QAAQ,CAAC,SAAS,GAAG,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC,OAAO,CAAC,CAAC;IAChF,OAAO,eAAe,QAAQ,CAAC,KAAK,OAAO,QAAQ,CAAC,MAAM,KAAK,UAAU,CAAC,QAAQ,CAAC,SAAS,CAAC,gBAAgB,QAAQ,CAAC,KAAK,OAAO,QAAQ,CAAC,MAAM,KAAK,UAAU,CAAC,QAAQ,CAAC,SAAS,CAAC,KAAK,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC;AACrN,CAAC,CAAC;AAEF;;;GAGG;AACH,MAAM,CAAC,MAAM,YAAY,GAAG,CAAC,KAAuC,EAAU,EAAE;IAC9E,IAAI,KAAK,CAAC,MAAM,IAAI,CAAC;QAAE,OAAO,EAAE,CAAC;IACjC,MAAM,OAAO,GAAG,CAAC,MAAM,EAAE,QAAQ,EAAE,KAAK,CAAU,CAAC;IACnD,MAAM,IAAI,GAAe,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;QACvC,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;QACpE,MAAM,IAAI,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,SAAS,EAAE,CAAC,CAAC,CAAC;QAChE,OAAO,CAAC,CAAC,CAAC,IAAI,EAAE,MAAM,CAAC,cAAc,EAAE,EAAE,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IAC7D,CAAC,CAAC,CAAC;IACH,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE;QACvC,MAAM,WAAW,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,GAAG,CAAC,MAAM,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;QAC1F,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,MAAM,EAAE,WAAW,CAAC,CAAC;IACzC,CAAC,CAAC,CAAC;IACH,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACzE,MAAM,UAAU,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACxF,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CACjC,GAAG;SACA,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;QACf,MAAM,SAAS,GAAG,CAAC,IAAI,CAAC,CAAC;QACzB,OAAO,SAAS;YACd,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,MAAM,CAAC;YACzC,CAAC,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,MAAM,CAAC,CAAC;IAC/C,CAAC,CAAC;SACD,IAAI,CAAC,IAAI,CAAC,CACd,CAAC;IACF,OAAO,CAAC,YAAY,EAAE,KAAK,UAAU,EAAE,EAAE,KAAK,GAAG,EAAE,EAAE,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAC1F,IAAI,CACL,CAAC;AACJ,CAAC,CAAC"}
@@ -0,0 +1,26 @@
1
+ /**
2
+ * Reads an image from disk and returns its width/height. Wrapped so tests
3
+ * can mock by reading a deterministic dimension out of a synthetic file.
4
+ */
5
+ export interface ImageDimensions {
6
+ width: number;
7
+ height: number;
8
+ }
9
+ export type ImageSizeReader = (path: string) => Promise<ImageDimensions>;
10
+ export declare const defaultImageSizeReader: ImageSizeReader;
11
+ /**
12
+ * Compute vision tokens for one image × one model. Dispatch by the model's
13
+ * registered provider. Throws a clear error when a provider doesn't support
14
+ * vision (we'll need this when Mistral / Cohere lands).
15
+ */
16
+ export declare const computeVisionTokens: (modelId: string, dim: ImageDimensions, imagePath: string) => number;
17
+ export interface ResolvedImage {
18
+ path: string;
19
+ dim: ImageDimensions;
20
+ }
21
+ /**
22
+ * Resolve all `--image` paths to dimensions in parallel. Reuses the file IO
23
+ * once per image (we still re-dispatch per model since the formula differs).
24
+ */
25
+ export declare const resolveImages: (paths: readonly string[], reader?: ImageSizeReader) => Promise<ResolvedImage[]>;
26
+ //# sourceMappingURL=vision.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"vision.d.ts","sourceRoot":"","sources":["../src/vision.ts"],"names":[],"mappings":"AAUA;;;GAGG;AACH,MAAM,WAAW,eAAe;IAC9B,KAAK,EAAE,MAAM,CAAC;IACd,MAAM,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,MAAM,eAAe,GAAG,CAAC,IAAI,EAAE,MAAM,KAAK,OAAO,CAAC,eAAe,CAAC,CAAC;AAEzE,eAAO,MAAM,sBAAsB,EAAE,eAOpC,CAAC;AAEF;;;;GAIG;AACH,eAAO,MAAM,mBAAmB,GAC9B,SAAS,MAAM,EACf,KAAK,eAAe,EACpB,WAAW,MAAM,KAChB,MAsBF,CAAC;AAEF,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,MAAM,CAAC;IACb,GAAG,EAAE,eAAe,CAAC;CACtB;AAED;;;GAGG;AACH,eAAO,MAAM,aAAa,GACxB,OAAO,SAAS,MAAM,EAAE,EACxB,SAAQ,eAAwC,KAC/C,OAAO,CAAC,aAAa,EAAE,CAMvB,CAAC"}
package/dist/vision.js ADDED
@@ -0,0 +1,44 @@
1
+ import { readFile } from 'node:fs/promises';
2
+ import { anthropicVisionTokens, getModel, googleVisionTokens, openaiVisionTokens, } from '@tokenometer/core';
3
+ import { imageSize } from 'image-size';
4
+ export const defaultImageSizeReader = async (path) => {
5
+ const bytes = await readFile(path);
6
+ const dim = imageSize(new Uint8Array(bytes.buffer, bytes.byteOffset, bytes.byteLength));
7
+ if (!dim.width || !dim.height) {
8
+ throw new Error(`Could not read image dimensions from "${path}".`);
9
+ }
10
+ return { height: dim.height, width: dim.width };
11
+ };
12
+ /**
13
+ * Compute vision tokens for one image × one model. Dispatch by the model's
14
+ * registered provider. Throws a clear error when a provider doesn't support
15
+ * vision (we'll need this when Mistral / Cohere lands).
16
+ */
17
+ export const computeVisionTokens = (modelId, dim, imagePath) => {
18
+ const model = getModel(modelId);
19
+ const provider = model.provider;
20
+ switch (provider) {
21
+ case 'anthropic':
22
+ return anthropicVisionTokens(dim);
23
+ case 'openai':
24
+ return openaiVisionTokens(dim);
25
+ case 'google':
26
+ return googleVisionTokens(dim);
27
+ case 'mistral':
28
+ case 'cohere':
29
+ throw new Error(`Vision tokens for provider "${provider}" are not yet supported (model "${modelId}", image "${imagePath}"). Use Claude, GPT-4o, or Gemini for image cost estimation.`);
30
+ default: {
31
+ const exhaustiveCheck = provider;
32
+ throw new Error(`Vision tokens are not supported for provider "${exhaustiveCheck}" (model "${modelId}", image "${imagePath}").`);
33
+ }
34
+ }
35
+ };
36
+ /**
37
+ * Resolve all `--image` paths to dimensions in parallel. Reuses the file IO
38
+ * once per image (we still re-dispatch per model since the formula differs).
39
+ */
40
+ export const resolveImages = async (paths, reader = defaultImageSizeReader) => Promise.all(paths.map(async (path) => {
41
+ const dim = await reader(path);
42
+ return { dim, path };
43
+ }));
44
+ //# sourceMappingURL=vision.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"vision.js","sourceRoot":"","sources":["../src/vision.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EACL,qBAAqB,EACrB,QAAQ,EACR,kBAAkB,EAClB,kBAAkB,GACnB,MAAM,mBAAmB,CAAC;AAE3B,OAAO,EAAE,SAAS,EAAE,MAAM,YAAY,CAAC;AAavC,MAAM,CAAC,MAAM,sBAAsB,GAAoB,KAAK,EAAE,IAAI,EAAE,EAAE;IACpE,MAAM,KAAK,GAAG,MAAM,QAAQ,CAAC,IAAI,CAAC,CAAC;IACnC,MAAM,GAAG,GAAG,SAAS,CAAC,IAAI,UAAU,CAAC,KAAK,CAAC,MAAM,EAAE,KAAK,CAAC,UAAU,EAAE,KAAK,CAAC,UAAU,CAAC,CAAC,CAAC;IACxF,IAAI,CAAC,GAAG,CAAC,KAAK,IAAI,CAAC,GAAG,CAAC,MAAM,EAAE,CAAC;QAC9B,MAAM,IAAI,KAAK,CAAC,yCAAyC,IAAI,IAAI,CAAC,CAAC;IACrE,CAAC;IACD,OAAO,EAAE,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,KAAK,EAAE,GAAG,CAAC,KAAK,EAAE,CAAC;AAClD,CAAC,CAAC;AAEF;;;;GAIG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,CACjC,OAAe,EACf,GAAoB,EACpB,SAAiB,EACT,EAAE;IACV,MAAM,KAAK,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC;IAChC,MAAM,QAAQ,GAAa,KAAK,CAAC,QAAQ,CAAC;IAC1C,QAAQ,QAAQ,EAAE,CAAC;QACjB,KAAK,WAAW;YACd,OAAO,qBAAqB,CAAC,GAAG,CAAC,CAAC;QACpC,KAAK,QAAQ;YACX,OAAO,kBAAkB,CAAC,GAAG,CAAC,CAAC;QACjC,KAAK,QAAQ;YACX,OAAO,kBAAkB,CAAC,GAAG,CAAC,CAAC;QACjC,KAAK,SAAS,CAAC;QACf,KAAK,QAAQ;YACX,MAAM,IAAI,KAAK,CACb,+BAA+B,QAAQ,mCAAmC,OAAO,aAAa,SAAS,8DAA8D,CACtK,CAAC;QACJ,OAAO,CAAC,CAAC,CAAC;YACR,MAAM,eAAe,GAAU,QAAQ,CAAC;YACxC,MAAM,IAAI,KAAK,CACb,iDAAiD,eAAe,aAAa,OAAO,aAAa,SAAS,KAAK,CAChH,CAAC;QACJ,CAAC;IACH,CAAC;AACH,CAAC,CAAC;AAOF;;;GAGG;AACH,MAAM,CAAC,MAAM,aAAa,GAAG,KAAK,EAChC,KAAwB,EACxB,SAA0B,sBAAsB,EACtB,EAAE,CAC5B,OAAO,CAAC,GAAG,CACT,KAAK,CAAC,GAAG,CAAC,KAAK,EAAE,IAAI,EAAE,EAAE;IACvB,MAAM,GAAG,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,CAAC;IAC/B,OAAO,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC;AACvB,CAAC,CAAC,CACH,CAAC"}
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "tokenometer",
3
- "version": "0.0.3",
4
- "description": "Empirical token-cost benchmarking CLI for LLM prompts. Tells you what your prompt actually costs across Claude, GPT-4o, and Gemini, in every format.",
3
+ "version": "0.1.0",
4
+ "description": "Tokenometer CLI LLM token cost + latency benchmarking across Claude, GPT-4o, Gemini, Mistral, and Cohere. Multi-format, empirical mode, vision tokens, SARIF output.",
5
5
  "license": "MIT",
6
6
  "author": "Faraazuddin Mohammed <mohdfaraaz1@gmail.com>",
7
7
  "homepage": "https://tokenometer.vercel.app",
@@ -16,16 +16,46 @@
16
16
  "keywords": [
17
17
  "ai",
18
18
  "anthropic",
19
+ "ci-cd",
19
20
  "claude",
21
+ "claude-code",
22
+ "claude-code-skill",
20
23
  "cli",
24
+ "code-scanning",
25
+ "codestral",
26
+ "cohere",
27
+ "command-r",
21
28
  "cost",
29
+ "cost-calculator",
30
+ "cursor",
22
31
  "gemini",
32
+ "github-action",
23
33
  "gpt",
34
+ "gpt-4o",
35
+ "latency",
24
36
  "llm",
37
+ "llm-cost",
38
+ "mistral",
39
+ "mistral-7b",
40
+ "mistral-large",
41
+ "mixtral",
42
+ "model-comparison",
43
+ "multimodal",
25
44
  "openai",
45
+ "pixtral",
26
46
  "prompt",
47
+ "prompt-cost",
48
+ "prompt-cost-regression",
49
+ "prompt-engineering",
50
+ "prompt-regression",
51
+ "sarif",
52
+ "tiktoken",
27
53
  "token",
28
- "tokenizer"
54
+ "token-budget",
55
+ "tokenizer",
56
+ "ttft",
57
+ "vision-tokens",
58
+ "vscode"
29
59
  ],
30
60
  "type": "module",
31
61
  "main": "./dist/index.js",
@@ -39,10 +69,7 @@
39
69
  "import": "./dist/index.js"
40
70
  }
41
71
  },
42
- "files": [
43
- "dist",
44
- "README.md"
45
- ],
72
+ "files": ["dist", "README.md"],
46
73
  "publishConfig": {
47
74
  "access": "public",
48
75
  "registry": "https://registry.npmjs.org/"
@@ -52,7 +79,8 @@
52
79
  "clean": "rm -rf dist"
53
80
  },
54
81
  "dependencies": {
55
- "@tokenometer/core": "0.0.3"
82
+ "@tokenometer/core": "0.1.0",
83
+ "image-size": "^2.0.2"
56
84
  },
57
85
  "devDependencies": {
58
86
  "@types/node": "^22.10.5",