@trazum/cli 1.10.0 → 1.26.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +120 -2
- package/dist/i18n/en.d.ts.map +1 -1
- package/dist/i18n/en.js +249 -4
- package/dist/i18n/en.js.map +1 -1
- package/dist/i18n/es.d.ts.map +1 -1
- package/dist/i18n/es.js +251 -4
- package/dist/i18n/es.js.map +1 -1
- package/dist/i18n/types.d.ts +499 -1
- package/dist/i18n/types.d.ts.map +1 -1
- package/dist/index.js +1586 -24
- package/dist/index.js.map +1 -1
- package/dist/markdown.d.ts +68 -0
- package/dist/markdown.d.ts.map +1 -1
- package/dist/markdown.js +329 -1
- package/dist/markdown.js.map +1 -1
- package/dist/time.d.ts +18 -0
- package/dist/time.d.ts.map +1 -0
- package/dist/time.js +32 -0
- package/dist/time.js.map +1 -0
- package/package.json +2 -2
- package/src/i18n/en.ts +370 -4
- package/src/i18n/es.ts +373 -4
- package/src/i18n/types.ts +522 -1
- package/src/index.ts +1857 -118
- package/src/markdown.ts +415 -1
- package/src/time.ts +32 -0
package/README.md
CHANGED
|
@@ -20,7 +20,8 @@ npm install -g @trazum/cli
|
|
|
20
20
|
| `trazum optimize <file>` | shorten it, and price what that is worth |
|
|
21
21
|
| `trazum check <file\|dir>` | does it fit its budget, and has the repo drifted past its baseline — exits 1 when either fails |
|
|
22
22
|
| `trazum baseline [dir]` | what the prompts cost now, recorded to a file you commit |
|
|
23
|
-
| `trazum profile <log
|
|
23
|
+
| `trazum profile <log\|dir>` | where the money actually went — reads a usage log (or a directory of rotated ones, `.gz` included), not a prompt. `--against` compares to the previous log; `--what-if <model>` prices these exact calls at another rate card; `--label`/`--since`/`--until` drill down; `--max-usd`/`--max-day-usd`/`--max-growth-usd`/`--max-cache-loss-usd` gate the spend; `--json`/`--csv-out`/`--markdown-out` for pipelines and CI |
|
|
24
|
+
| `trazum route <log.jsonl>` | is the cheaper model good enough? — measured, and it asks before spending |
|
|
24
25
|
| `trazum doctor [dir]` | the whole workspace: what nothing is watching, and what fixing would be worth |
|
|
25
26
|
| `trazum rank <dir>` | of these forty prompts, which is worth an afternoon |
|
|
26
27
|
| `trazum prune <file> --cases <file>` | which few-shot examples earn their tokens — measured, and it asks before spending |
|
|
@@ -96,7 +97,8 @@ returned. Recording it is three lines and no transformer:
|
|
|
96
97
|
```ts
|
|
97
98
|
appendFileSync('usage.jsonl', JSON.stringify({
|
|
98
99
|
model: response.model,
|
|
99
|
-
label: 'support-rag',
|
|
100
|
+
label: 'support-rag', // which workload — without it every call looks alike
|
|
101
|
+
session: conversationId, // which conversation — grouped by, never printed
|
|
100
102
|
...response.usage,
|
|
101
103
|
}) + '\n');
|
|
102
104
|
```
|
|
@@ -114,6 +116,122 @@ A model the pricing catalogue does not know is named and kept **out** of the
|
|
|
114
116
|
totals, because a total that silently omits calls is wrong in the flattering
|
|
115
117
|
direction.
|
|
116
118
|
|
|
119
|
+
### What would actually move this bill
|
|
120
|
+
|
|
121
|
+
The rules recover about **1%** — measured. Which model a call goes to moves 40% to
|
|
122
|
+
80%, and the Batch API moves 50% flat. `profile` prices those from your log:
|
|
123
|
+
|
|
124
|
+
```
|
|
125
|
+
→ support-rag on Claude Opus 5 — up to $16.80 of this bill (52.2%)
|
|
126
|
+
400 calls, $21.00 spent
|
|
127
|
+
· route it to Claude Sonnet 5, $12.60
|
|
128
|
+
· send it through the Batch API, $10.50
|
|
129
|
+
|
|
130
|
+
For comparison: shortening the prompt text can touch $18.00 at the very most.
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
The options are **combined, never summed** — batching a routed call discounts the
|
|
134
|
+
cheaper model, so the pair is $16.80 and not $23.10 against $21.00 spent. A route
|
|
135
|
+
prints the `eval` command rather than a recommendation: the arithmetic is exact and
|
|
136
|
+
says nothing about quality. Nothing crosses a vendor, and no figure is ever "per
|
|
137
|
+
month" — a log covers whatever period you recorded.
|
|
138
|
+
|
|
139
|
+
### What re-sending the conversation costs
|
|
140
|
+
|
|
141
|
+
A chat or agent workload replays the whole conversation every turn, so the input
|
|
142
|
+
grows with the turn count — routinely the largest line on an agent bill, and
|
|
143
|
+
invisible to everything else here. Add `session` (or `conversation_id`) to the log
|
|
144
|
+
and `profile` measures it:
|
|
145
|
+
|
|
146
|
+
```
|
|
147
|
+
agent on Claude Opus 5: input ranges from 600 tokens on the smallest turn
|
|
148
|
+
to 5,000 on the largest, over conversations of up to 12 turns.
|
|
149
|
+
If every turn had been the size of its smallest one, that input would have
|
|
150
|
+
cost $7.20 instead of $33.60 — so at most $26.40 of this bill is
|
|
151
|
+
conversation growth (57.9%).
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
**A ceiling, not a saving**: some of that growth is the user's own new messages, and
|
|
155
|
+
this reads counts rather than content. **The session key is never printed** — it
|
|
156
|
+
groups calls and counts turns, every figure comes out per label, and a test asserts
|
|
157
|
+
it appears nowhere in the report or in `--json`.
|
|
158
|
+
|
|
159
|
+
### Why a failing cache fails
|
|
160
|
+
|
|
161
|
+
`profile` can say *that* caching loses money on a label — the log carries counts,
|
|
162
|
+
not content. Map the label to its prompt file and it reads the file and says why:
|
|
163
|
+
|
|
164
|
+
```json
|
|
165
|
+
{ "labels": { "support-rag": "prompts/support.txt" } }
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
```
|
|
169
|
+
prompts/support.txt (as it is today — the log may predate it): the stable
|
|
170
|
+
prefix is 13 tokens and Claude Opus 5 caches nothing under 512. Setting
|
|
171
|
+
cache_control there does not error, it simply never caches.
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Three diagnoses: a prefix under the model's minimum, stable tokens stranded behind
|
|
175
|
+
the first placeholder (`--reorder` moves them), or a healthy file whose problem is
|
|
176
|
+
byte-identity between calls. Every sentence says the file is **today's** — the log
|
|
177
|
+
may predate it, and a fresh file presented as the history's explanation would be a
|
|
178
|
+
figure attributed to something it does not describe.
|
|
179
|
+
|
|
180
|
+
### Is the cheaper model good enough? — `trazum route`
|
|
181
|
+
|
|
182
|
+
The section above prices a route and can say nothing about whether it works. This
|
|
183
|
+
runs the measurement:
|
|
184
|
+
|
|
185
|
+
```bash
|
|
186
|
+
trazum route usage.jsonl --prompt-file prompts/support.txt --cases cases.txt --yes
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
```
|
|
190
|
+
support-rag on Claude Opus 5 → Claude Sonnet 5, worth $12.60 of this bill (60.0%).
|
|
191
|
+
|
|
192
|
+
The cheaper model agrees with the original 94% of the time. The original
|
|
193
|
+
agrees with itself 91% of the time — that is the yardstick, not 100%.
|
|
194
|
+
|
|
195
|
+
✓ HOLDS — the difference is inside the original model's own noise.
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
**The yardstick is the expensive model's own variance**, measured on the same cases
|
|
199
|
+
in the same run, so the verdict is not a threshold somebody picked. Three provider
|
|
200
|
+
calls per case — two on the original, one on the candidate — and it prints the
|
|
201
|
+
count and stops unless you pass `--yes`.
|
|
202
|
+
|
|
203
|
+
It says *agreement is not correctness* on every verdict, including the good one.
|
|
204
|
+
This measures whether the answers moved, not whether they were ever right.
|
|
205
|
+
|
|
206
|
+
### Did the caching pay for itself?
|
|
207
|
+
|
|
208
|
+
The rest of Trazum tells you to cache. This is the one report that can say the
|
|
209
|
+
advice was wrong here — and the cache hit rate will not.
|
|
210
|
+
|
|
211
|
+
A cache write costs 1.25x plain input on Anthropic and **2x** at the one-hour TTL,
|
|
212
|
+
so a prefix that changes faster than it is reused pays that premium for nothing:
|
|
213
|
+
|
|
214
|
+
```
|
|
215
|
+
Cache hit rate 97.8% of billable input.
|
|
216
|
+
Caching took $0.2675 off this bill, against the same tokens uncached.
|
|
217
|
+
! The total above hides a loss: caching costs $0.1250 across rag.
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
Computed per label as well as overall — ranked by what caching cost each one, not
|
|
221
|
+
by the size of its bill, because the worst cache in an estate usually sits on a
|
|
222
|
+
small workload. Each side is priced per model, so a provider whose writes cost the
|
|
223
|
+
same as input is never accused of a loss it cannot have; a model added through a
|
|
224
|
+
`pricing` overlay can declare its own `multipliers` for the same reason.
|
|
225
|
+
|
|
226
|
+
**When the log cannot settle it, neither does the report.** A cache write whose
|
|
227
|
+
TTL was not recorded is priced at the cheaper of the two rates, and that moves the
|
|
228
|
+
verdict rather than only the total — so instead of a figure you get both, and what
|
|
229
|
+
to record to remove the doubt.
|
|
230
|
+
|
|
231
|
+
This is the only counterfactual here, and it is arithmetic rather than a guess:
|
|
232
|
+
caching changes the multiplier on a token, never the token. `--json` carries it as
|
|
233
|
+
`cache` and `cacheByLabel` — **positive `deltaUsd` means worse.**
|
|
234
|
+
|
|
117
235
|
## The ceiling is not the problem
|
|
118
236
|
|
|
119
237
|
A budget says nothing while a prompt climbs from 800 tokens to 1,900 under a
|
package/dist/i18n/en.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"en.d.ts","sourceRoot":"","sources":["../../src/i18n/en.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;
|
|
1
|
+
{"version":3,"file":"en.d.ts","sourceRoot":"","sources":["../../src/i18n/en.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAqB9C;;;;;;GAMG;AACH,eAAO,MAAM,EAAE,EAAE,WA2qChB,CAAC"}
|
package/dist/i18n/en.js
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
/** Counts, grouped. A log with forty thousand torn lines should say so legibly. */
|
|
2
2
|
const count = (value) => value.toLocaleString('en-US');
|
|
3
|
+
/**
|
|
4
|
+
* A grouped count and its noun, agreeing.
|
|
5
|
+
*
|
|
6
|
+
* `1 calls` and `1 prompts` were reachable on ordinary input — a one-call log, a
|
|
7
|
+
* repository with a single prompt — and two messages in this file already did the
|
|
8
|
+
* agreement by hand while a dozen did not. A helper rather than a dozen ternaries,
|
|
9
|
+
* so the next message written gets it for free.
|
|
10
|
+
*/
|
|
11
|
+
const plural = (value, one, many = `${one}s`) => `${count(value)} ${value === 1 ? one : many}`;
|
|
3
12
|
/** "(46 days ago)", or nothing when the age is unknown. */
|
|
4
13
|
const ago = (days) => days === null ? '' : days === 0 ? ' (today)' : days === 1 ? ' (1 day ago)' : ` (${days} days ago)`;
|
|
5
14
|
/**
|
|
@@ -20,6 +29,7 @@ ${bold('USAGE')}
|
|
|
20
29
|
trazum baseline [dir] [options]
|
|
21
30
|
trazum eval <file> --cases <file> [options]
|
|
22
31
|
trazum eval <file> --cases <file> --export promptfoo -o suite.json
|
|
32
|
+
trazum route <log.jsonl> --prompt-file <file> --cases <file> --yes
|
|
23
33
|
trazum diff <before> <after> [options]
|
|
24
34
|
trazum diff --all <dir> <dir> [options]
|
|
25
35
|
trazum rank <dir> [options]
|
|
@@ -188,6 +198,85 @@ ${bold('OPTIONS FOR eval')}
|
|
|
188
198
|
optimised once. That baseline is the yardstick — without it, a divergence
|
|
189
199
|
figure means nothing. Exits with code 1 when the answers genuinely diverge.
|
|
190
200
|
|
|
201
|
+
${bold('OPTIONS FOR profile')}
|
|
202
|
+
--against <log.jsonl> Compare this bill to a previous log. Positive
|
|
203
|
+
means the bill grew; drivers are ranked by their
|
|
204
|
+
contribution to the change. No period is assumed —
|
|
205
|
+
judge the call counts before judging the money.
|
|
206
|
+
--label <name> Profile only the calls carrying this label — the
|
|
207
|
+
drill-down once the full report named a suspect.
|
|
208
|
+
A label that matches nothing is an error naming
|
|
209
|
+
the labels that exist. With --against, both logs
|
|
210
|
+
are filtered, so the comparison stays one workload.
|
|
211
|
+
--max-usd <n> Exit 1 when this log spent more than n dollars.
|
|
212
|
+
The budget applies to exactly the log handed in:
|
|
213
|
+
profile yesterday's log nightly and this is a
|
|
214
|
+
daily budget without Trazum guessing what a day is.
|
|
215
|
+
--max-growth-usd <n> With --against: exit 1 when the bill grew more
|
|
216
|
+
than n dollars over the previous log. Alone it is
|
|
217
|
+
an error, not a flag that silently gates nothing.
|
|
218
|
+
--max-cache-loss-usd <n> Exit 1 when caching added more than n dollars to
|
|
219
|
+
this bill. Reads the worst case when the log did
|
|
220
|
+
not record the write TTL — a gate reading the
|
|
221
|
+
flattering half would pass the bills it exists
|
|
222
|
+
to catch — and says which claim fired.
|
|
223
|
+
--max-day-usd <n> Fail when any single UTC day inside the log spent
|
|
224
|
+
more than this. A month under budget can hide the
|
|
225
|
+
afternoon a loop burned a quarter of it. A log
|
|
226
|
+
with no timestamps fails: not measured is not
|
|
227
|
+
under budget.
|
|
228
|
+
--since <when> Profile only calls at or after this moment. A UTC
|
|
229
|
+
--until <when> day (2026-08-14), a full ISO 8601 timestamp, a
|
|
230
|
+
relative window (7d, 24h) or "now";
|
|
231
|
+
--until with a bare date includes that whole day.
|
|
232
|
+
Calls with no "ts" cannot be placed and are left
|
|
233
|
+
out — counted out loud, never dropped silently.
|
|
234
|
+
With --against, both logs get the same window.
|
|
235
|
+
--what-if <model> Price these exact calls on another model. The
|
|
236
|
+
same token counts at a different rate card —
|
|
237
|
+
multiplication, not advice, and it says so.
|
|
238
|
+
Calls larger than that model's context window
|
|
239
|
+
are named as impossible, not priced as cheap.
|
|
240
|
+
--markdown-out <file> Also write the report as Markdown, for a CI job
|
|
241
|
+
summary or a pull request comment.
|
|
242
|
+
--csv-shape <shape> Which table --csv-out writes: slice (default),
|
|
243
|
+
day or hour. One row shape per file, so nothing
|
|
244
|
+
has to be filtered before it can be summed.
|
|
245
|
+
--csv-out <file> Also write the report as CSV, one row per label
|
|
246
|
+
and model. No total row, on purpose: a total
|
|
247
|
+
inside a data file gets summed with the data.
|
|
248
|
+
Unpriced models keep their tokens and get empty
|
|
249
|
+
dollar cells, never zeros.
|
|
250
|
+
--pricing <file> Local price overlay, as everywhere else.
|
|
251
|
+
--json The full report as data, including the levers.
|
|
252
|
+
|
|
253
|
+
Takes a log file or a directory of them — rotated daily logs are read in
|
|
254
|
+
name order as one bill, and how many were read is stated. Gzipped files
|
|
255
|
+
(.jsonl.gz and the rest) are read too, because that is what a rotated log
|
|
256
|
+
looks like a day later; one that will not decompress is an error naming the
|
|
257
|
+
file, never a bill quietly missing a day.
|
|
258
|
+
|
|
259
|
+
Reads what the provider actually charged. Optional fields unlock findings:
|
|
260
|
+
"label" (which workload), "session" (which conversation — grouped by, never
|
|
261
|
+
printed), "stop_reason"/"finish_reason" (answers cut off at max_tokens).
|
|
262
|
+
|
|
263
|
+
${bold('OPTIONS FOR route')}
|
|
264
|
+
--prompt-file <file> The prompt those calls send. Not --prompt, which
|
|
265
|
+
names a marked prompt inside a source file.
|
|
266
|
+
--cases <file> One input per line, or a JSON array. Required.
|
|
267
|
+
--label <name> Measure this workload instead of the costliest one.
|
|
268
|
+
--concurrency <n> Calls in flight at once. Default: 3.
|
|
269
|
+
--yes Actually spend the calls. Without it the count is
|
|
270
|
+
printed and nothing is called.
|
|
271
|
+
--json The slice and the measurement as data.
|
|
272
|
+
|
|
273
|
+
Reads a usage log, finds the slice where routing to a cheaper model is worth
|
|
274
|
+
the most, and measures whether that model still does the job. The same prompt
|
|
275
|
+
goes to both, and the original runs twice per case — so the verdict is judged
|
|
276
|
+
against that model's own variance rather than a threshold somebody picked.
|
|
277
|
+
|
|
278
|
+
Costs three provider calls per case and needs TRAZUM_LLM_* configured.
|
|
279
|
+
|
|
191
280
|
${bold('OPTIONS FOR diff')}
|
|
192
281
|
--max-growth <n> Fail if the prompt grew by more than n tokens.
|
|
193
282
|
--all Compare two directories of prompts, paired by relative
|
|
@@ -235,6 +324,12 @@ ${bold('CONFIG FILE')}
|
|
|
235
324
|
budgets { "prompts/**": 2000, "prompts/system.txt": 4000 }
|
|
236
325
|
baseline { "path": "trazum.baseline.json", "maxGrowthTokens": 0, "maxGrowthPct": 5 }
|
|
237
326
|
pricing "./prices.json" — local price corrections, see below
|
|
327
|
+
labels { "support-rag": "prompts/support.txt" } — which prompt file each
|
|
328
|
+
usage-log label sends, so "trazum profile" can read the file and
|
|
329
|
+
say why a failing cache fails
|
|
330
|
+
spend { "maxUsd": 200, "byLabel": { "chat": 40 } } — money budgets for
|
|
331
|
+
"trazum profile", in dollars. A budgeted label with no calls in
|
|
332
|
+
the log is reported as not measured, never as a pass
|
|
238
333
|
|
|
239
334
|
Flags beat the config; the config beats the defaults. Budgets resolve to the
|
|
240
335
|
most specific matching pattern — most literal characters wins. A boolean the
|
|
@@ -389,7 +484,12 @@ ${bold('EXAMPLES')}
|
|
|
389
484
|
// `tokens` arrives already formatted for the locale, so the singular is
|
|
390
485
|
// decided on the string rather than on a number that is no longer here.
|
|
391
486
|
tokensSaved: (tokens) => `${tokens} token${tokens === '1' ? '' : 's'} back, every call.`,
|
|
487
|
+
windowNegligible: (tokens, model, window) => `${tokens} tokens of ${model}'s ${window}-token window — under a tenth of a percent, so the window is not what constrains this prompt.`,
|
|
488
|
+
windowUnmoved: (share, model, window) => `${share} of ${model}'s ${window}-token window, before and after: this change is too small to move it.`,
|
|
489
|
+
beyondThisPromptTokensOnly: () => 'Shortening a prompt is the smallest lever there is: measured on an ordinary support prompt, the rules recover about 1% of a monthly bill. If any of your prompts go to a metered API, "trazum profile <usage.jsonl>" reads what the provider actually charged and prices the levers that are not the prompt. Recording that log is a few lines and it never contains prompt text.',
|
|
490
|
+
beyondThisPrompt: () => 'Shortening a prompt is the smallest lever there is: measured on an ordinary support prompt, the rules recover about 1% of a monthly bill. On a metered API the things that move 40% to 80% are which model the call goes to, the Batch API, prompt caching, and what re-sending the conversation costs — and "trazum profile <usage.jsonl>" prices all four from what the provider actually charged. Recording that log is a few lines and it never contains prompt text.',
|
|
392
491
|
windowUse: (before, after, model, window) => `Context window: ${before} → ${after} of ${model}'s ${window} tokens — room the conversation gets instead.`,
|
|
492
|
+
tokensOnlyAskedFor: () => 'You named a scenario, and it was not priced: Trazum is running somewhere that bills by subscription, so there is no bill here to reduce. Add --cost to price it anyway — the host says where Trazum runs, not where your prompt goes.',
|
|
393
493
|
tokensOnlyCost: () => 'Pass --cost if this prompt is bound for a metered API.',
|
|
394
494
|
pricingOverlaid: (models, lastReviewed) => `Prices for ${models} came from a local overlay reviewed ${lastReviewed}, not from the bundled catalogue.`,
|
|
395
495
|
reorderHeading: () => 'Reordered for caching',
|
|
@@ -675,9 +775,10 @@ ${bold('EXAMPLES')}
|
|
|
675
775
|
exactCountsCost: (files) => `Counting ${files} ${files === 1 ? 'file' : 'files'} through the API, one call each. This takes a moment.`,
|
|
676
776
|
},
|
|
677
777
|
profile: {
|
|
678
|
-
noTarget: () => 'Point this at a usage log: trazum profile usage.jsonl — one JSON object per line, each with a "model" and the "usage" object the API returned. Recording
|
|
778
|
+
noTarget: () => 'Point this at a usage log: trazum profile usage.jsonl — one JSON object per line, each with a "model" and the "usage" object the API returned. Add "label" (which workload), "session" (which conversation) and "ts" (when) while you are there: without them every call looks alike, and the largest findings this command makes — conversation growth, and whether the cache TTL fits how fast your turns come — cannot be made at all. Recording it is four lines in your own code, it never contains prompt text, and the session key is grouped by and never printed.',
|
|
679
779
|
heading: () => 'Where the money went',
|
|
680
|
-
|
|
780
|
+
calls: (n) => plural(n, 'call'),
|
|
781
|
+
spent: (calls, total) => `${calls} · ${total}`,
|
|
681
782
|
part: (name, usd, pct, tokens) => `${name.padEnd(13)}${usd.padStart(11)} ${pct.padStart(5)} ${tokens} tokens`,
|
|
682
783
|
partInput: () => 'Input',
|
|
683
784
|
partCacheRead: () => 'Cache reads',
|
|
@@ -685,17 +786,161 @@ ${bold('EXAMPLES')}
|
|
|
685
786
|
partOutput: () => 'Output',
|
|
686
787
|
byLabelHeading: () => 'By label',
|
|
687
788
|
byModelHeading: () => 'By model',
|
|
688
|
-
row: (name, usd, pct, calls) => `${usd.padStart(11)} ${pct.padStart(5)} ${name} (${calls}
|
|
689
|
-
unlabelled
|
|
789
|
+
row: (name, usd, pct, calls) => `${usd.padStart(11)} ${pct.padStart(5)} ${name} (${calls})`,
|
|
790
|
+
// Parenthesised so a real label named "unlabelled" cannot read identically
|
|
791
|
+
// beside it — the data already keeps them apart; the display should too.
|
|
792
|
+
unlabelled: () => '(no label)',
|
|
690
793
|
cacheHit: (pct) => `Cache hit rate ${pct} of billable input.`,
|
|
691
794
|
cacheNever: () => 'Caching was never used on these calls. If any prefix repeats, that is the largest saving available.',
|
|
795
|
+
cacheLost: (usd, writes, reads) => `Caching added ${usd} to this bill instead of taking it off. ${writes} tokens were written to the cache and ${reads} read back — and a write costs 1.25x plain input, or 2x at the 1-hour TTL. A prefix that changes faster than it is reused pays that premium for nothing. Either cache a prefix that holds still, or turn caching off here.`,
|
|
796
|
+
cachePaidOff: (usd) => `Caching took ${usd} off this bill, against the same tokens uncached.`,
|
|
797
|
+
cacheNoDifference: () => 'Caching came out level on this bill: what it charged for these tokens is what they would have cost as ordinary input. It is neither paying for itself nor costing you anything.',
|
|
798
|
+
cacheLostBy: (labels) => `The loss is in: ${labels}.`,
|
|
799
|
+
cacheLostHidden: (usd, labels) => `The total above hides a loss: caching costs ${usd} across ${labels}.`,
|
|
800
|
+
andMoreLabels: (n) => `and ${count(n)} more`,
|
|
801
|
+
cacheTtlUnsettled: (calls, asRecorded, atLongTtl) => `This log cannot say whether caching paid for itself. ${count(calls)} ${calls === 1 ? 'call' : 'calls'} did not record which cache-write TTL was used: at the 5-minute rate caching took ${asRecorded} off this bill, and at the 1-hour rate the same calls added ${atLongTtl} to it. Neither is reported as the answer. Record the "cache_creation" object the API returns and this settles itself.`,
|
|
802
|
+
cacheTtlBound: (calls, atLongTtl) => `That figure is a bound, not a measurement: ${count(calls)} ${calls === 1 ? 'call' : 'calls'} did not record a cache-write TTL, and at the 1-hour rate it is ${atLongTtl}.`,
|
|
803
|
+
cacheTtlUnsettledLabels: (labels) => `These would be losing money if their unrecorded writes used the 1-hour TTL: ${labels}.`,
|
|
692
804
|
biggestPart: (name, pct) => `${name} is ${pct} of this bill.`,
|
|
693
805
|
outputDominates: (pct) => `Output is ${pct} of this bill, so shortening prompts has a low ceiling here. What moves it is asking for shorter answers and capping max_tokens.`,
|
|
694
806
|
unpriced: (models, calls) => `${count(calls)} ${calls === 1 ? 'call is' : 'calls are'} not in these totals — the pricing catalogue does not know: ${models}. Add them with a pricing overlay (--pricing) to include them.`,
|
|
695
807
|
skipped: (lineCount, lines) => `${count(lineCount)} ${lineCount === 1 ? 'line' : 'lines'} could not be read and ${lineCount === 1 ? 'was' : 'were'} left out (${lineCount === 1 ? 'line' : 'lines'} ${lines}).`,
|
|
696
808
|
empty: () => 'No usage records in that file.',
|
|
697
809
|
nothingPriced: () => 'None of the models in that log are in the pricing catalogue, so there is no bill to report. Add them with a pricing overlay (--pricing) and run this again.',
|
|
810
|
+
leversHeading: () => 'What would actually move this bill',
|
|
811
|
+
leverSlice: (label, model, usd, pct) => `${label} on ${model} — up to ${usd} of this bill (${pct})`,
|
|
812
|
+
leverRoute: (candidate, usd) => `route it to ${candidate}, ${usd}`,
|
|
813
|
+
leverRouteVerify: (candidate) => `Whether that holds is an evaluation question, not an arithmetic one, and nothing here has seen a single answer. Measure it: trazum route <log> --prompt-file <prompt> --cases <cases> --yes`,
|
|
814
|
+
leverBatch: (usd) => `send it through the Batch API, ${usd}`,
|
|
815
|
+
leverCalls: (calls, spent) => `${calls}, ${spent} spent`,
|
|
816
|
+
leverPromptCeiling: (usd, pct) => `For comparison: shortening the prompt text can touch ${usd} at the very most — ${pct} of this bill, and only if you deleted every input token. The real figure is far below that, because most of those tokens are retrieved context, conversation history and tool results that no prompt file contains.`,
|
|
817
|
+
historyHeading: () => 'What re-sending the conversation costs',
|
|
818
|
+
historyGrowth: (label, model, first, last, turns) => `${label} on ${model}: input ranges from ${first} tokens on the smallest turn to ${last} on the largest, over conversations of up to ${turns} turns.`,
|
|
819
|
+
historyCeiling: (usd, pct, flat, spent) => `If every turn had been the size of its smallest one, that input would have cost ${flat} instead of ${spent} — so at most ${usd} of this bill is conversation growth (${pct}). It is a ceiling and not a saving: some of that is the user's own new messages, which nothing can truncate away, and this reads counts rather than content so it cannot tell the two apart. What moves it is capping the history you replay, or summarising it.`,
|
|
820
|
+
truncatedWaste: (calls, usd, pct) => `${calls} hit the max_tokens ceiling: ${usd} of the output spend (${pct}) bought answers that were cut off mid-generation — paid in full, frequently retried and billed again. Where the answer genuinely needs the room, raise max_tokens; where it does not, ask for less. Either way this is the one slice of a bill that is waste without a counterpart.`,
|
|
821
|
+
againstHeading: () => 'Against the previous log',
|
|
822
|
+
againstTotals: (before, after, delta, pct, callsBefore, callsAfter) => `${before} → ${after} ${delta} (${pct}) ${callsBefore} → ${callsAfter}. Positive means the bill grew. Both figures are exactly what each file holds — no period is assumed, so judge the call counts before judging the money.`,
|
|
823
|
+
againstDriver: (delta, label, before, after) => `${delta} ${label} (${before} → ${after})`,
|
|
824
|
+
againstDriverNew: (delta, label) => `${delta} ${label} (new since the previous log)`,
|
|
825
|
+
againstDriverGone: (delta, label) => `${delta} ${label} (gone since the previous log)`,
|
|
826
|
+
againstByModel: () => 'The same change, by model — where the mix moved:',
|
|
827
|
+
labelPrefixBelowMinimum: (file, prefix, minimum, model) => `${file} (as it is today — the log may predate it): the stable prefix is ${prefix} tokens and ${model} caches nothing under ${minimum}. Setting cache_control there does not error, it simply never caches — which is what a cache that only writes looks like from the bill.`,
|
|
828
|
+
labelPrefixMovable: (file, movable, prefix) => `${file} (as it is today — the log may predate it): ~${movable} stable tokens sit after the first placeholder, where caching cannot reach them; the cacheable prefix is ${prefix}. "trazum optimize ${file} --reorder" moves them in front and shows the diff.`,
|
|
829
|
+
labelPrefixHealthy: (file, prefix, minimum) => `${file} (as it is today — the log may predate it): the stable prefix is ${prefix} tokens, over the ${minimum} minimum. The prompt file is not the problem; look at whether the prefix is byte-identical between calls.`,
|
|
830
|
+
labelFileMissing: (label, file) => `labels["${label}"] points at ${file}, which does not exist — the mapping was skipped.`,
|
|
831
|
+
againstNothingPriced: () => 'The previous log has nothing the pricing catalogue knows, so there is no comparison to make.',
|
|
832
|
+
truncatedNotRecorded: () => 'Whether any answers were cut off could not be measured — no call in this log carries a stop reason. Add "stop_reason" (Anthropic) or "finish_reason" (OpenAI) to the record; the API already returns it beside "usage".',
|
|
833
|
+
historyNoSessions: () => 'No call in this log carried a session, so what re-sending the conversation costs could not be measured — usually the largest line on a chat or agent bill. Add "session" (or "conversation_id") to the record and run this again. Trazum groups by it and never prints it.',
|
|
834
|
+
leversUnlabelled: () => 'None of these calls carried a label, so this is every workload in one row — a classifier and a RAG pipeline merged into a single figure, with one route suggested for both. Add "label" to the record and the levers split by workload, which is the grouping a decision is actually made at.',
|
|
835
|
+
outputShapeHeading: () => 'Where the output spend concentrates',
|
|
836
|
+
outputTail: (label, model, callPct, spendPct, above, usd) => `${label} on ${model}: ${callPct} of calls hold ${spendPct} of the output spend — the ones answering with more than ${above} tokens, out of ${usd} of output on this slice.`,
|
|
837
|
+
outputTailAdvice: () => 'That is a tail, and a tail has a cause: a path through the prompt that invites an essay, a call with no max_tokens, a retrieval that returned a book. Finding it is a morning; it is not "make everything shorter".',
|
|
838
|
+
outputFlat: (label, model, callPct, spendPct, usd) => `${label} on ${model}: the output spend sits where the calls are — ${callPct} of them hold ${spendPct} of ${usd}. There is no tail to hunt.`,
|
|
839
|
+
outputFlatAdvice: () => 'The answer length is the task here, so the levers are the blunt ones: ask for shorter answers in the prompt, and cap max_tokens.',
|
|
840
|
+
outputPercentiles: (p50, p95) => `Half the measured answers fit within ${p50} output tokens, and 95% within ${p95} — the number a max_tokens cap actually wants. Measured on these calls, promised for nothing.`,
|
|
841
|
+
inputShapeHeading: () => 'How big these calls are',
|
|
842
|
+
inputSkewed: (label, model, p50, p95, ratio, usd) => `${label} on ${model} is uneven: half its calls fit within ${p50} input tokens and 95% within ${p95} — about ${ratio}x the ordinary call, over ${usd} of input spend.`,
|
|
843
|
+
inputSkewedAdvice: () => 'Past four times the median, the ordinary call is fine and something is growing on top of it: a conversation nobody truncates, a retrieval with no cap, a tool result pasted in whole. The fix is a limit on the large calls, not a rewrite of the prompt every call sends.',
|
|
844
|
+
inputEven: (label, model, p50, p95, usd) => `${label} on ${model} is even: half its calls fit within ${p50} input tokens and 95% within ${p95}, over ${usd} of input spend.`,
|
|
845
|
+
inputEvenAdvice: () => 'The large calls are not much larger than the ordinary one, so there is no tail to cap — the prompt is simply big. The levers are fewer retrieved documents, a shorter system block, and caching if the prefix repeats.',
|
|
846
|
+
inputHuge: (label, model, calls, usd) => `${label} on ${model}: every one of its ${calls} is larger than this tool measures precisely, over ${usd} of input spend. No ceiling is named because there is none to name honestly — that size is itself the finding.`,
|
|
847
|
+
repeatsHeading: () => 'The same request, sent again',
|
|
848
|
+
repeatsFound: (label, model, repeats, checked, seconds, usd) => `${label} on ${model}: ${repeats} of ${checked} calls re-sent the previous call's exact input size within ${seconds} seconds, in the same conversation, costing ${usd}.`,
|
|
849
|
+
repeatsAdvice: () => 'A conversation\'s input grows with every turn, so the same size twice in a row seconds apart is usually a retry after a timeout, an agent step repeating, or a loop — this reads counts and cannot see content, so it names the pattern and stops. Whatever it is, the money bought nothing the call before it had not already paid for.',
|
|
850
|
+
inputMostlyCached: (share) => `${share} of those tokens were cache reads, billed at a tenth of the input rate — the size is real and most of it is cheap.`,
|
|
851
|
+
inputFullRate: () => 'Almost none of that was a cache read, so every one of those tokens was billed at the full input rate. If any prefix repeats across these calls, caching is the lever with the largest ceiling here.',
|
|
852
|
+
leversNone: () => 'Nothing here clears 1% of the bill: these calls are already on the cheapest model of their family, or their provider has no batch API. That is a real answer, not an empty section.',
|
|
698
853
|
assumedWriteTtl: (calls) => `${count(calls)} ${calls === 1 ? 'call did' : 'calls did'} not say which cache-write TTL was used, so the cheaper 5-minute rate was assumed. A 1-hour entry costs 2x input rather than 1.25x, so this total is a floor for those calls. Record the "cache_creation" object the API returns to remove the assumption.`,
|
|
854
|
+
spanLine: (from, to, days) => `This log covers ${from} → ${to} (${days} days). The span is stated, never extrapolated — the monthly arithmetic is yours to do, and now it is valid.`,
|
|
855
|
+
spanPartial: (withTs, total) => `Only ${withTs} of ${total} calls carry a timestamp; the span describes those.`,
|
|
856
|
+
ttlFitExpires: (label, model, gap) => `${label} on ${model}: turns arrive a median of ${gap} apart and the 5-minute entry is gone by then — writes expire before the next turn reads them, which from the bill is a cache that only writes. The 1-hour TTL costs 2x input to write and would survive these gaps; the other honest option is caching switched off here.`,
|
|
857
|
+
ttlFitExpiresBoth: (label, model, gap) => `${label} on ${model}: turns arrive a median of ${gap} apart, and no cache entry lives that long — even the 1-hour TTL is gone by the next turn. Caching cannot work at this pace; turn it off here and stop paying the write premium.`,
|
|
858
|
+
ttlFitOverlong: (label, model, gap, usd) => `${label} on ${model}: turns arrive a median of ${gap} apart — comfortably inside the 5-minute window — and these writes pay the 1-hour rate, 2x input against 1.25x, for endurance the gaps never use. The same writes at the 5-minute TTL are ${usd} cheaper on this log, and that figure is exact: the same tokens at the other published rate.`,
|
|
859
|
+
ttlFitUnsettledGap: (label, model, gap) => `${label} on ${model}: turns arrive a median of ${gap} apart — a 5-minute entry is gone by then and a 1-hour one survives — and the log did not record which these writes were, so whether they ever get read back cannot be settled. Record the "cache_creation" object the API returns and it settles itself.`,
|
|
860
|
+
ttlFitFits: (label, model, gap) => `${label} on ${model}: turns arrive a median of ${gap} apart, inside the lifetime these writes use. The TTL is not the problem here.`,
|
|
861
|
+
ttlFitUnmeasured: () => 'Whether the cache TTL fits how fast the turns arrive could not be measured — it needs both "session" and "ts" on the record. A 5-minute entry on a workload whose turns come nine minutes apart expires unread on every write, and only the clock can see it. Trazum groups by the session and never shows it.',
|
|
862
|
+
dayPeak: (day, usd, xMedian) => `The most expensive day in this log was ${day}: ${usd}, ${xMedian}x the median day.`,
|
|
863
|
+
dayPeakLabel: (label, usd) => `Most of it was ${label} (${usd}).`,
|
|
864
|
+
maxUsdOk: (total, max) => `Within budget: ${total} spent against --max-usd ${max}.`,
|
|
865
|
+
maxUsdFailed: (total, max) => `FAILED — this log spent ${total} against a --max-usd of ${max}. The figures are the provider's own billed counts over exactly this log; no period was assumed.`,
|
|
866
|
+
maxGrowthUsdFailed: (delta, max) => `FAILED — the bill grew ${delta} against the previous log, over the --max-growth-usd limit of ${max}.`,
|
|
867
|
+
maxGrowthNeedsAgainst: () => '--max-growth-usd has nothing to compare without --against <previous.jsonl>. On its own it would have run silently and gated nothing, which is not an answer.',
|
|
868
|
+
maxCacheLossOk: (worst, max) => `Cache within budget: caching cost at most ${worst} against --max-cache-loss-usd ${max}, worst case included.`,
|
|
869
|
+
maxCacheLossFailed: (delta, max) => `FAILED — caching added ${delta} to this bill (the same tokens as plain input would have cost less), over the --max-cache-loss-usd limit of ${max}. The counterfactual is exact: the same tokens at the published input rate.`,
|
|
870
|
+
maxDayOk: (day, usd, max) => `No single day over budget: the worst was ${day} at ${usd}, against --max-day-usd ${max}.`,
|
|
871
|
+
maxDayFailed: (day, usd, max) => `FAILED — ${day} spent ${usd}, over the --max-day-usd limit of ${max}. A total under budget can hide a single runaway day, which is what this gate exists to catch.`,
|
|
872
|
+
maxDayNoClock: () => 'FAILED — --max-day-usd was asked for and no record in this log carries a timestamp, so there are no days to judge. That is not a pass: a bill nobody could measure by day is not a bill that stayed under a daily budget. Add "ts" to the record and the gate arms.',
|
|
873
|
+
maxDayUndated: (calls) => `${calls} calls carry no timestamp, so they are in the bill and in none of the days above — the worst day is a floor by whatever they held. A failure would stand regardless; this pass is over the part that could be dated.`,
|
|
874
|
+
maxCacheLossWorstCase: (calls, worst, max) => `FAILED — ${count(calls)} ${calls === 1 ? 'call' : 'calls'} did not record which cache-write TTL was paid, and at the 1-hour rate caching added up to ${worst}, over the --max-cache-loss-usd limit of ${max}. The gate reads the worst case on purpose: a gate reading the flattering half would pass exactly the bills it exists to catch. Record the "cache_creation" object the API returns and the ceiling becomes a figure.`,
|
|
875
|
+
pricesStale: (date, days) => `The price table behind every dollar here was last reviewed ${date} — ${count(days)} days ago, past the 45 this tool considers current. If the provider changed prices since, this report is wrong by exactly that change. --pricing-live fetches today's prices; --pricing overlays your own.`,
|
|
876
|
+
dayTableDay: () => 'Day (UTC)',
|
|
877
|
+
dayTableCalls: () => 'calls',
|
|
878
|
+
dayTableTop: () => 'biggest that day',
|
|
879
|
+
dayTableEarlier: (days) => `…and ${count(days)} earlier ${days === 1 ? 'day' : 'days'} not shown here. The full series rides --json as spendByDay.`,
|
|
880
|
+
gateOnFloor: (reasons) => `Note: the gated figure is a floor, not the bill — ${reasons}. Whatever those calls cost is not in the number the gate just judged, so a pass here means "the part I could read fits", never "the bill fits".`,
|
|
881
|
+
floorSkipped: (lines) => `${count(lines)} ${lines === 1 ? 'line was' : 'lines were'} unreadable and left out`,
|
|
882
|
+
floorUnpriced: (calls) => `${count(calls)} ${calls === 1 ? 'call is' : 'calls are'} on models the price table does not know`,
|
|
883
|
+
floorUndated: (calls) => `${count(calls)} ${calls === 1 ? 'call carries' : 'calls carry'} no timestamp and fell outside the window`,
|
|
884
|
+
sessionCost: (label, model, sessions, median, medianTurns, p95, max) => `${label} on ${model}: across ${sessions} conversations, the median one costs ${median} over ${medianTurns} turns, 95% come in under ${p95}, and the most expensive was ${max}. Exact billed counts, per conversation — the figure a per-seat price or a quota is set from. A conversation that started before this log or continues after it is counted only for the turns recorded here.`,
|
|
885
|
+
labelBudgetOk: (label, usd, max) => `Within budget: ${label} spent ${usd} against ${max}.`,
|
|
886
|
+
labelBudgetFailed: (label, usd, max) => `FAILED — ${label} spent ${usd} against its budget of ${max} in trazum.config.json.`,
|
|
887
|
+
labelBudgetMissing: (label) => `${label} has a budget in trazum.config.json and no calls in this log, so nothing was measured for it. Not a pass: a workload that did not appear is not a workload that came in under budget.`,
|
|
888
|
+
labelBudgetWindowed: () => 'Per-label budgets in trazum.config.json were not applied: --since/--until make "what this label spent" mean a slice, and a budget written for the whole period would be gating against something it does not describe.',
|
|
889
|
+
duplicateLines: (calls, usd) => `${plural(calls, 'line is an exact duplicate', `lines are exact duplicates`)} of an earlier line — same counts, same label and session, same millisecond — and that adds ${usd} to the total above. If a log was exported twice or two files in a directory overlap, this bill is overstated by that much. Two real calls colliding on all of that is possible; it is just unlikely.`,
|
|
890
|
+
budgetVsWire: (label, file, budget, perCall, share) => `The budget on ${file} is ${budget} tokens, and calls labelled ${label} carry about ${perCall} input tokens each — so that gate governs roughly ${share} of what actually goes up the wire. The rest is retrieved context, conversation history and tool results, which no prompt file contains and no budget on one can see. The budget is not wrong; it is just smaller than the bill.`,
|
|
891
|
+
badCsvShape: (value) => `--csv-shape does not know "${value}". It takes "slice" (one row per label and model, the default), "day" or "hour".`,
|
|
892
|
+
whatIfHeading: (model) => `These exact calls on ${model}`,
|
|
893
|
+
whatIfAssumption: () => 'This is multiplication, not advice: the same token counts at another rate card. It says nothing about whether that model could do the work, and a model that answers at greater length or gets retried would not send these counts at all.',
|
|
894
|
+
whatIfTotal: (current, target, delta) => `${current} of movable spend would have been ${target} — a difference of ${delta}.`,
|
|
895
|
+
whatIfCheaper: () => 'Verify before you move anything: trazum route measures one prompt against both models on your own examples.',
|
|
896
|
+
whatIfDearer: () => 'That direction costs more. The arithmetic is here so the number is not a guess.',
|
|
897
|
+
whatIfSlice: (label, model, current, target) => `${label} on ${model}: ${current} → ${target}`,
|
|
898
|
+
whatIfOverContext: (label, tokens, window, usd) => `${label} cannot move: its largest call carries ${tokens} input tokens and that model's window is ${window}. Those calls would fail, not cost less, so their ${usd} is excluded from the figures above.`,
|
|
899
|
+
whatIfAlreadyThere: (calls, usd) => `Already on that model: ${calls} worth ${usd}, left out of the figures above — money that cannot move would make the difference look smaller than it is.`,
|
|
900
|
+
whatIfUnpriced: (calls, models) => `Excluded: ${calls} whose model has no price here (${models}). Their cost on the target is knowable; the difference is not, because there is no current figure to subtract from.`,
|
|
901
|
+
whatIfNothingToMove: () => 'Nothing to compare: every priced call in this log is already on that model, or too large for its context window.',
|
|
902
|
+
whatIfUnknown: (value, available) => `--what-if does not know "${value}". Priced models: ${available}. Add it with --pricing if you have its rates.`,
|
|
903
|
+
badGzip: (file, detail) => `${file} is gzipped and would not decompress: ${detail}. Reading the rest and saying nothing would report a bill missing whatever that file held, so this stops instead. Check the file, or move it out of the directory.`,
|
|
904
|
+
coverageHeading: () => 'What this log cannot answer yet',
|
|
905
|
+
needsLabel: (seen) => `"label" on ${seen} records: without it every workload is one row, so no per-workload spend, no drill-down, and the levers describe a mixture rather than a decision.`,
|
|
906
|
+
needsSession: (seen) => `"session" on ${seen} records: without it there is no conversation growth, no per-conversation cost, and no cache-TTL fit. It is grouped by and never printed.`,
|
|
907
|
+
needsTs: (seen) => `"ts" on ${seen} records: without it the log has no period, no per-day or per-hour shape, and the cache-TTL question cannot be asked at all.`,
|
|
908
|
+
needsStopReason: (seen) => `"stop_reason" (Anthropic) or "finish_reason" (OpenAI) on ${seen} records: without it, answers cut off at max_tokens are invisible — and silence there is not the same as none.`,
|
|
909
|
+
needsCacheTtl: (seen) => `the "cache_creation" object on ${seen} of the records that wrote to the cache: without it the 5-minute rate is assumed, so those totals are a floor and some cache verdicts cannot be settled.`,
|
|
910
|
+
hoursConcentrated: (hours, list) => `80% of this spend lands in ${hours} hours of the UTC day (${list}) — interactive traffic somebody is waiting on, where the Batch API's 24-hour turnaround does not fit. Hours are UTC; shift them yourself if your traffic sits in one region.`,
|
|
911
|
+
hoursFlat: (hours) => `The spend is spread across the day: it takes ${hours} hours of the UTC day to cover 80% of it. That is the shape background work has, and background work is what the Batch API halves the price of — see the levers above for what it would be worth here. Whether these calls can wait is yours to say; the log only shows when they happened.`,
|
|
912
|
+
truncatedBy: (label, calls, measured, rate, usd) => `${label}: ${calls} of ${measured} calls that recorded a stop reason were cut off (${rate}), ${usd} of output. The denominator is the calls that measured, not every call — a workload logging the field half the time is not a workload whose other half completed.`,
|
|
913
|
+
truncatedCeiling: (p95) => `95% of the answers that finished fit within ${p95} output tokens, so a cap around there would stop cutting them off. Measured on these calls, promised for nothing.`,
|
|
914
|
+
readFiles: (files, directory) => `Read ${count(files)} log files from ${directory}, in name order, as one bill. Every figure below covers all of them.`,
|
|
915
|
+
noLogsInDirectory: (directory, extensions) => `No usage logs in "${directory}". Looked for files ending in ${extensions}. A directory with nothing readable in it is an error rather than an empty report, which would read as "you spent nothing".`,
|
|
916
|
+
sessionCostTail: (ratio) => `The 95th percentile is ${ratio}x the median there: most conversations are cheap and a few are not, which is a tail a quota can catch. Where median and p95 sit close together the workload is simply expensive and there is no tail to hunt.`,
|
|
917
|
+
againstOverlap: (from, to) => `These two logs both cover ${from} → ${to}, so some of the same calls sit on both sides of this subtraction and part of the change is the same money counted twice. Compare periods that do not overlap — or window both logs with --since/--until.`,
|
|
918
|
+
windowLine: (since, until) => `Filtered to --since ${since} --until ${until}. Everything below describes this window, not the whole log; a bare date means the whole of that UTC day.`,
|
|
919
|
+
windowUndated: (calls) => `${count(calls)} ${calls === 1 ? 'call carries' : 'calls carry'} no timestamp and cannot be placed inside or outside this window, so ${calls === 1 ? 'it was' : 'they were'} left out. Their spend is in the log and not in this report — the window's figures are a floor on the period.`,
|
|
920
|
+
windowRelative: () => 'That window is relative to this machine\'s clock, not to the log\'s last record — a log exported a month ago will answer "the last 7 days" with nothing.',
|
|
921
|
+
windowRelativeEmpty: () => 'A relative window is measured from this machine\'s clock: if this log was exported earlier, ask for the dates it covers instead.',
|
|
922
|
+
windowNeedsClock: () => 'No record in this log carries a timestamp, so --since/--until have nothing to filter by. A time window over a clockless log would gate nothing, which is not an answer. Add "ts" to the records — the recipe in the README shows where.',
|
|
923
|
+
windowMatchesNothing: (from, to) => `No record falls inside this window. The log covers ${from} → ${to}. A window matching nothing must not become a $0 report — under --max-usd it would pass a budget gate over a period the log does not cover.`,
|
|
924
|
+
sinceAfterUntil: () => '--since is at or after --until, so the window contains no time at all. Check the two dates.',
|
|
925
|
+
badWhen: (flag, value) => `--${flag} could not read "${value}". It takes a UTC day (2026-08-14) or a full ISO 8601 timestamp (2026-08-14T09:30:00Z).`,
|
|
926
|
+
singleTurnCeiling: (label, model, single, sessions, usd) => `${label} on ${model}: ${single} of ${sessions} conversations ended after their first turn, and their cache writes — ${usd} — paid for reuse their own conversation never made. Another conversation sharing the same prefix within the TTL could have read them; the log cannot see whose write a read hit, so that figure is a ceiling on the waste, not a bill.`,
|
|
927
|
+
singleTurnConfirmed: (label, model, single, sessions, usd) => `${label} on ${model}: ${single} of ${sessions} conversations ended after their first turn and spent ${usd} writing a cache that nothing in this log ever read. Within the conversation, across conversations — no read anywhere, so those writes bought nothing. Caching a one-shot call is pure write premium; stop marking these calls with cache_control.`,
|
|
928
|
+
},
|
|
929
|
+
route: {
|
|
930
|
+
noTarget: () => 'Point this at a usage log and a prompt: trazum route usage.jsonl --prompt-file prompts/support.txt --cases cases.txt --yes. It finds the slice worth the most, then measures whether the cheaper model still does the job. The flag is --prompt-file and not --prompt, because --prompt names a marked prompt inside a source file everywhere else in this tool.',
|
|
931
|
+
needsPrompt: () => '--prompt and --cases are both required. The log says which route is worth money; only the prompt and the cases can say whether it works.',
|
|
932
|
+
labelNotFound: (label, available) => `No call in this log carries the label "${label}". The labels here are: ${available}.`,
|
|
933
|
+
noRoute: () => 'No route on this log clears 1% of the bill. These calls are already on the cheapest model of their family, or the catalogue has nothing below them.',
|
|
934
|
+
picked: (label, model, candidate, usd, pct) => `${label} on ${model} → ${candidate}, worth ${usd} of this bill (${pct}).`,
|
|
935
|
+
willSpend: (calls, model, candidate) => `This will make ${count(calls)} provider calls: two per case on ${model} to measure its own variance, one per case on ${candidate}. Nothing has been spent yet — add --yes to run it.`,
|
|
936
|
+
dryRun: () => 'Nothing was called.',
|
|
937
|
+
running: (cases) => `Running ${count(cases)} cases...`,
|
|
938
|
+
agreement: (cross, self) => `The cheaper model agrees with the original ${cross} of the time. The original agrees with itself ${self} of the time — that is the yardstick, not 100%.`,
|
|
939
|
+
holds: (usd) => `HOLDS — the difference is inside the original model's own noise. On this bill that route is worth ${usd}.`,
|
|
940
|
+
diverges: (usd) => `DIVERGES — the cheaper model gives materially different answers. The ${usd} is real and so is the change in behaviour; this one is not free money.`,
|
|
941
|
+
inconclusive: () => 'INCONCLUSIVE — the original model was too inconsistent with itself on these cases to judge anything against. Add cases, or pick ones with less room for the model to wander.',
|
|
942
|
+
unlabelledSlice: () => 'These calls carry no label, so Trazum cannot tell whether they are all this prompt. If they are not, the figure above covers calls this measurement never touched — add "label" to the record and the slice becomes one workload, which is what makes the number attributable.',
|
|
943
|
+
yours: () => 'Agreement is not correctness. This measures whether the answers moved, not whether they were ever right — the decision is still yours.',
|
|
699
944
|
},
|
|
700
945
|
baseline: {
|
|
701
946
|
recorded: (path, files, tokens) => `Recorded ${files} prompts, ${tokens} tokens, to ${path}. Commit it — the gate compares the tree against what is committed.`,
|