mcp-context-cost 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -3,37 +3,12 @@
3
3
  [![npm](https://img.shields.io/npm/v/mcp-context-cost)](https://www.npmjs.com/package/mcp-context-cost)
4
4
  [![CI](https://github.com/athakur3/mcp-context-cost/actions/workflows/ci.yml/badge.svg)](https://github.com/athakur3/mcp-context-cost/actions/workflows/ci.yml)
5
5
 
6
- **Reproducible context-cost measurement for MCP servers for the one you publish, and for
7
- the stack you actually run.**
6
+ **What do the MCP servers in your config cost you before you type anything — and what did
7
+ that last config change add to every session you will ever run?**
8
8
 
9
- Every MCP server you wire into an agent injects its tool schemas into the model's context
10
- before any work happens. That cost is invisible and it varies by **1,700×** across
11
- popular servers:
12
-
13
- | server | context cost | tools |
14
- |---|---:|---:|
15
- | github (official) | **54,422 tokens** | 44 |
16
- | brave-search | 25,456 | 8 |
17
- | notion | 17,500 | 24 |
18
- | playwright *(4.8M installs/week)* | 4,024 | 24 |
19
- | filesystem (reference) | 2,823 | 14 |
20
- | markitdown | 64 | 1 |
21
-
22
- *(57 of 82 popular servers measured, 2026-08-16 sweep — full table in
23
- [results/leaderboard.md](results/leaderboard.md); every failure is listed with its reason.
24
- Each measured server also has a [detail page](https://athakur3.github.io/mcp-context-cost/servers/)
25
- showing which tools its tokens are in.)*
26
-
27
- This project makes that cost **legible and disputable**:
28
-
29
- ```
30
- [context cost | 12,430 tokens] ← shields.io badge, linked to the methodology
31
- ```
32
-
33
- ## What does *your* setup cost?
34
-
35
- The leaderboard measures one server at a time. You don't run one server — you run a stack.
36
- Point `audit` at your own MCP config and it measures every server you actually have installed:
9
+ Every MCP server you wire into an agent injects its tool schemas into the model's context on
10
+ every single request. You pay that whether or not the agent ends up using the tools, and no
11
+ client shows you the number. Point `audit` at your own MCP config:
37
12
 
38
13
  ```bash
39
14
  npx -y mcp-context-cost audit
@@ -55,6 +30,10 @@ claude-desktop ~/Library/Application Support/Claude/claude_desktop_config.json
55
30
  heaviest tools
56
31
  sequential-thinking · sequentialthinking 990
57
32
  memory · search_nodes 323
33
+
34
+ trim: disabling 3 tools (sequential-thinking·sequentialthinking, memory·search_nodes,
35
+ memory·open_nodes) would recover 1,635 tokens (20.7% of this config) — if your client
36
+ supports per-tool filtering.
58
37
  ```
59
38
 
60
39
  It finds configs for Claude Desktop, Claude Code (`~/.claude.json`, `.mcp.json`), Cursor,
@@ -67,6 +46,12 @@ Totals are reported per config file, never merged: a context window belongs to o
67
46
  session, so summing Cursor's servers into Claude Desktop's total would describe a session
68
47
  nobody runs.
69
48
 
49
+ One nuance: Claude Code's tool search (default-on in recent versions) defers full MCP
50
+ schemas until used, loading only tool names at session start. Audit totals are the weight
51
+ of the schema surface itself — what loads upfront in clients without deferral (Claude
52
+ Desktop, Cursor, VS Code, Windsurf today), and what Claude Code's documented fallback
53
+ modes still load. Deferral-aware reporting is on the roadmap.
54
+
70
55
  **In CI**, make it a gate — the bundlesize move for agents:
71
56
 
72
57
  ```bash
@@ -74,8 +59,107 @@ npx -y mcp-context-cost audit --config .mcp.json --budget 20000
74
59
  # exits 1 when the stack exceeds the budget, so a PR adding a 25K-token server fails
75
60
  ```
76
61
 
77
- Flags: `--json` (full report on stdout, progress on stderr), `--budget N`, `--context N`
78
- (default 200,000), `--timeout ms`, `--concurrency N`, `--docker`.
62
+ The budget is an absolute ceiling. What a reviewer actually wants to know is what *this pull
63
+ request* did, so record a baseline and diff against it:
64
+
65
+ ```bash
66
+ npx -y mcp-context-cost audit --config .mcp.json --json > baseline.json # on main
67
+ npx -y mcp-context-cost audit --config .mcp.json --baseline baseline.json --max-increase 2000
68
+ ```
69
+
70
+ ```
71
+ diff vs baseline measured 2026-08-18T01:51:49.555Z (methodology 1.0)
72
+
73
+ .mcp.json
74
+ 2,378 → 5,201 +2,823
75
+
76
+ added filesystem — → 2,823 +2,823
77
+ (1 server unchanged)
78
+
79
+ This change adds 2,823 tokens to every request in this client — 1.2% → 2.6% of a
80
+ 200,000-token context window.
81
+
82
+ INCREASE FAIL:
83
+ .mcp.json: +2,823 tokens per request, over the 2,000 allowed
84
+ ```
85
+
86
+ > **Version note.** `--baseline` and `--max-increase` are **not in the published 0.3.0** —
87
+ > they are on `main` and ship in the next release. This matters more than a normal
88
+ > unreleased-feature note: 0.3.0 ignores flags it does not recognise, so running the command
89
+ > above against it produces a plain audit and **exit 0** — a passing CI check on a gate that
90
+ > never ran. Builds after 0.3.0 reject unknown flags with exit 2 instead. Until the next
91
+ > release, pin the gate to a version that has it, or it is not gating anything.
92
+
93
+ A baseline is just a stored `audit --json` report, so any artifact store works. Without
94
+ `--max-increase` the diff is informational and the exit code is unchanged.
95
+
96
+ [`examples/github-actions.yml`](examples/github-actions.yml) is the whole thing as a workflow:
97
+ measure the base branch, measure the PR, fail on the difference.
98
+
99
+ `--max-increase` fails on more than the number — it also fails whenever the increase could
100
+ not be established. A server that measured yesterday and won't start today takes its tokens
101
+ out of the total in exactly the way uninstalling it would, and reporting that as a saving is
102
+ the one mistake this tool must not make. So a server that crossed the measured/unmeasured
103
+ line, a config with no baseline, or a baseline config this run never found each fail the
104
+ gate and name themselves:
105
+
106
+ ```
107
+ Not a clean comparison: a server changed measured-ness between the two runs.
108
+ The measured total moved −2,378, but that is not what your config did.
109
+
110
+ memory: measured 2,378 in the baseline and could not be measured now — its cost is
111
+ missing from the total, not gone from your config
112
+ → true cost is at least 2,378 higher than the 0 measured now.
113
+
114
+ INCREASE FAIL:
115
+ .mcp.json: a server changed measured-ness, so the change could not be established exactly
116
+ ```
117
+
118
+ Add `--claude` to annotate each server with its Anthropic-request cost from the published
119
+ [Claude divergence](docs/METHODOLOGY.md#claude-divergence) run — an exact number when the
120
+ published capture hash matches what you have installed, `—` (silence, not a stale guess)
121
+ when it doesn't (today the run covers the top 15 measured servers, so most installs will
122
+ show a mix):
123
+
124
+ ```
125
+ server tools tokens share claude
126
+ github 44 54,422 95.8% 18,406
127
+ memory 9 2,378 4.2% —
128
+ ```
129
+
130
+ Flags: `--json` (full report on stdout, progress on stderr), `--budget N`,
131
+ `--baseline <report.json>`, `--max-increase N`, `--context N` (default 200,000),
132
+ `--timeout ms`, `--concurrency N`, `--docker`, `--claude`.
133
+
134
+ ## Where the numbers come from
135
+
136
+ The number `audit` gives you is the same measurement, run across a curated set of public
137
+ servers — which is how you can tell it is a measurement and not this tool's opinion. It also
138
+ shows what you are choosing between: across the 65 servers measured, cost spans **1,700×**,
139
+ from the 32-token `postgres` reference server to github's 54,422. The table below starts at
140
+ markitdown's 64 tokens, an 850× spread; the full range is in
141
+ [results/leaderboard.md](results/leaderboard.md).
142
+
143
+ | server | context cost | tools |
144
+ |---|---:|---:|
145
+ | github (official) | **54,422 tokens** | 44 |
146
+ | brave-search | 25,456 | 8 |
147
+ | notion | 17,500 | 24 |
148
+ | playwright *(4.8M installs/week)* | 4,024 | 24 |
149
+ | filesystem (reference) | 2,823 | 14 |
150
+ | markitdown | 64 | 1 |
151
+
152
+ *(65 of 82 popular servers measured, 2026-08-18 sweep — full table in
153
+ [results/leaderboard.md](results/leaderboard.md); every failure is listed with its reason.
154
+ Each measured server also has a [detail page](https://athakur3.github.io/mcp-context-cost/servers/)
155
+ showing which tools its tokens are in.)*
156
+
157
+ If you publish a server, the same measurement is available as a badge, so your users can see
158
+ the cost before they install rather than after:
159
+
160
+ ```
161
+ [context cost | 12,430 tokens] ← shields.io badge, linked to the methodology
162
+ ```
79
163
 
80
164
  ## What it costs on Claude
81
165
 
@@ -94,9 +178,9 @@ the method is [Claude divergence](docs/METHODOLOGY.md#claude-divergence).
94
178
 
95
179
  ## Why trust the number?
96
180
 
97
- Every badge is backed by a `measurement.json` containing the raw `tools/list` capture, the
98
- SHA-256 of its canonical bytes, the pinned tokenizer (`o200k_base`), and the exact launch
99
- command. Disputes reduce to a byte-level diff:
181
+ Every published number is backed by a `measurement.json` containing the raw `tools/list`
182
+ capture, the SHA-256 of its canonical bytes, the pinned tokenizer (`o200k_base`), and the
183
+ exact launch command. Disputes reduce to a byte-level diff:
100
184
 
101
185
  ```bash
102
186
  npx -y mcp-context-cost verify results/github/measurement.json
@@ -110,6 +194,11 @@ Add `--json` for scripting (`{ ok, serverName, rederivedTokens, rederivedSha, pr
110
194
  on stdout, `badge` omitted on failure). Exit codes: `0` ok, `1` verification/measurement
111
195
  failed, `2` usage error.
112
196
 
197
+ `audit` runs that same code path on your own machine and reports each server's
198
+ `canonicalSha256` in `--json`, so you can check that the version you installed is byte-identical
199
+ to the one that was published — which is exactly what `--claude` uses to decide whether it is
200
+ allowed to show you a number.
201
+
113
202
  Full definition: [docs/METHODOLOGY.md](docs/METHODOLOGY.md) — what is counted, what the
114
203
  number is *not*, config policy, failure taxonomy, frozen color bands, known divergences.
115
204
 
@@ -119,7 +208,7 @@ number is *not*, config policy, failure taxonomy, frozen color bands, known dive
119
208
  |---|---|
120
209
  | `src/core/` | the measurement spec, executable — canonical form, tokenizer, bands, badge JSON |
121
210
  | `src/sweep/` | raw-wire MCP stdio client + Dockerized batch sweep + leaderboard/dashboard generators |
122
- | `src/audit/` | client-config discovery (5 clients, JSONC-tolerant) + the per-stack report |
211
+ | `src/audit/` | client-config discovery (5 clients, JSONC-tolerant), the per-stack report, and the baseline diff |
123
212
  | `src/cli.ts` | `audit` (measure your own stack), `verify` (re-derive any published number), `measure` |
124
213
  | `spec/fixtures/` | golden vectors shared by the TypeScript and bash implementations |
125
214
  | `tools/` | the one script that calls a network API (Claude divergence); kept out of the package so the library stays offline |
@@ -152,7 +241,7 @@ Or self-serve from CI via the (staged) mcp-tokens-action badge inputs — see
152
241
  ## Development
153
242
 
154
243
  ```bash
155
- npm test # 105 TS tests incl. golden fixtures + dispute drills
244
+ npm test # 158 TS tests incl. golden fixtures + dispute drills
156
245
  npx tsc --noEmit # typecheck
157
246
  ./upstream/tests/badge-test.sh # 21 bash tests — byte-identical to the TS reference
158
247
  npm run sweep:all -- --docker # full curated sweep (Docker isolation)
@@ -165,7 +254,11 @@ color bands are frozen against the observed distribution of the first full sweep
165
254
 
166
255
  ## Status
167
256
 
168
- Active. The leaderboard refreshes weekly; badge PRs are open across the ecosystem and
257
+ Active. 56 of the 65 numbers come from a single sweep on 2026-08-16, 3 from 2026-08-17, and
258
+ 6 from 2026-08-18 (an upstream `mcp` package bump broke the old low-level-`Server` API these
259
+ six relied on; pinning `mcp<2` in their launch commands fixed startup, not this project's code);
260
+ the weekly job currently re-measures one server (`memory`), so treat the leaderboard as a
261
+ dated snapshot rather than a live feed. Badge PRs are open across the ecosystem and
169
262
  [sd2k/mcp-tokens-action#5](https://github.com/sd2k/mcp-tokens-action/pull/5) proposes the
170
263
  self-serve badge path upstream. See [ROADMAP.md](ROADMAP.md) for what's next —
171
264
  contributions welcome, especially new `servers.yaml` entries.
@@ -1,5 +1,7 @@
1
+ import { type DivergenceRun } from '../core/divergence.js';
1
2
  import type { Measurement, MeasurementStatus, ToolMeasurement } from '../core/types.js';
2
3
  import type { ConfiguredServer, LoadedConfig } from './config.js';
4
+ import { type AuditDiff, type IncreaseGate } from './diff.js';
3
5
  export declare const DEFAULT_CONTEXT_WINDOW = 200000;
4
6
  export type AuditStatus = MeasurementStatus | 'remote-not-measurable';
5
7
  export interface AuditServerResult {
@@ -15,6 +17,13 @@ export interface AuditServerResult {
15
17
  /** Names only — a server's env values never enter a report. */
16
18
  envVarNames: string[];
17
19
  canonicalSha256?: string | null;
20
+ /**
21
+ * Anthropic-request cost from the published Claude divergence run, only when
22
+ * its captured hash matches this install (`--claude`). `null` means the
23
+ * install doesn't match what was published — silence, not a stale guess.
24
+ * `undefined` means `--claude` wasn't requested at all.
25
+ */
26
+ claudeTokens?: number | null;
18
27
  notes?: string;
19
28
  }
20
29
  export interface HeaviestTool {
@@ -22,6 +31,17 @@ export interface HeaviestTool {
22
31
  tool: string;
23
32
  tokens: number;
24
33
  }
34
+ /**
35
+ * What turning off the heaviest few tools would recover, for clients that let
36
+ * you disable individual tools rather than whole servers (Claude Code's
37
+ * per-tool permission rules, Cursor's per-tool toggles). `null` when there's
38
+ * nothing worth trimming (one tool total, or no measured tokens).
39
+ */
40
+ export interface TrimAdvice {
41
+ tools: HeaviestTool[];
42
+ recoverableTokens: number;
43
+ recoverableShare: number;
44
+ }
25
45
  export interface AuditConfigResult {
26
46
  client: string;
27
47
  source: string;
@@ -32,7 +52,37 @@ export interface AuditConfigResult {
32
52
  servers: AuditServerResult[];
33
53
  skipped: AuditServerResult[];
34
54
  heaviestTools: HeaviestTool[];
55
+ trimAdvice: TrimAdvice | null;
56
+ }
57
+ export interface BudgetFitStep {
58
+ name: string;
59
+ tokens: number;
60
+ /** What the config still costs after removing this one and everything above it. */
61
+ remaining: number;
62
+ }
63
+ export interface BudgetFit {
64
+ /** How far over the limit the worst config starts. */
65
+ overBy: number;
66
+ /** Heaviest-first removals until the remainder fits. Empty if nothing can be removed. */
67
+ drop: BudgetFitStep[];
68
+ keptCount: number;
69
+ keptTokens: number;
70
+ /** False when removing every measured server still would not fit — a limit set too low. */
71
+ feasible: boolean;
35
72
  }
73
+ /**
74
+ * The smallest heaviest-first set of servers that gets a config under its budget.
75
+ *
76
+ * `--budget` used to print "BUDGET FAIL: 84,455 > 20,000" and stop, which tells a reader
77
+ * they have a problem and nothing about the shape of it. The whole point of the audit
78
+ * surface is that the person running it is the person paying the tokens, and "you are over"
79
+ * is a measurement where "these two are why" is a decision.
80
+ *
81
+ * Heaviest-first is ONE ordering, not a recommendation: this cannot know which servers you
82
+ * need, and dropping by weight will sometimes name the one you cannot live without. That
83
+ * caveat is printed with the result rather than left implied.
84
+ */
85
+ export declare function planBudgetFit(config: AuditConfigResult, limit: number): BudgetFit;
36
86
  export interface AuditReport {
37
87
  methodologyVersion: string;
38
88
  encoding: 'o200k_base';
@@ -44,7 +94,18 @@ export interface AuditReport {
44
94
  worstTotal: number;
45
95
  worstSource: string;
46
96
  over: boolean;
97
+ /** Present only when over budget: the arithmetic of getting back under it. */
98
+ fit?: BudgetFit;
99
+ };
100
+ /** Present only when a divergence run was supplied (`--claude`). */
101
+ claudeDivergence?: {
102
+ model: string;
103
+ measuredAt: string;
47
104
  };
105
+ /** Present only when a baseline report was supplied (`--baseline`). */
106
+ diff?: AuditDiff;
107
+ /** Present only when `--max-increase` was supplied alongside a baseline. */
108
+ increaseGate?: IncreaseGate;
48
109
  problems: string[];
49
110
  }
50
111
  /** Cache key for measurement reuse: the exact argv two configs would spawn. */
@@ -58,6 +119,8 @@ export declare function buildReport(configs: LoadedConfig[], measured: Map<strin
58
119
  contextWindow?: number;
59
120
  budget?: number;
60
121
  generatedAt?: string;
122
+ /** Published `tools-delta/v1` run to join against (`--claude`); omit to skip the join. */
123
+ divergence?: DivergenceRun | null;
61
124
  }): AuditReport;
62
125
  /** Human output. JSON output is the report object itself. */
63
126
  export declare function formatReport(report: AuditReport): string;
@@ -13,7 +13,52 @@
13
13
  * commands shared by two configs are still only measured once.
14
14
  */
15
15
  import { METHODOLOGY_VERSION } from '../core/canonical.js';
16
+ import { isCurrent } from '../core/divergence.js';
17
+ import { formatDiff, formatGate } from './diff.js';
16
18
  export const DEFAULT_CONTEXT_WINDOW = 200_000;
19
+ const TRIM_TOOL_COUNT = 3;
20
+ function buildTrimAdvice(sortedTools, totalTokens) {
21
+ if (totalTokens <= 0 || sortedTools.length < 2)
22
+ return null;
23
+ const trimmed = sortedTools.slice(0, TRIM_TOOL_COUNT);
24
+ const recoverableTokens = trimmed.reduce((a, t) => a + t.tokens, 0);
25
+ return { tools: trimmed, recoverableTokens, recoverableShare: recoverableTokens / totalTokens };
26
+ }
27
+ /**
28
+ * The smallest heaviest-first set of servers that gets a config under its budget.
29
+ *
30
+ * `--budget` used to print "BUDGET FAIL: 84,455 > 20,000" and stop, which tells a reader
31
+ * they have a problem and nothing about the shape of it. The whole point of the audit
32
+ * surface is that the person running it is the person paying the tokens, and "you are over"
33
+ * is a measurement where "these two are why" is a decision.
34
+ *
35
+ * Heaviest-first is ONE ordering, not a recommendation: this cannot know which servers you
36
+ * need, and dropping by weight will sometimes name the one you cannot live without. That
37
+ * caveat is printed with the result rather than left implied.
38
+ */
39
+ export function planBudgetFit(config, limit) {
40
+ const measured = config.servers
41
+ .filter((srv) => typeof srv.tokens === 'number' && srv.tokens > 0)
42
+ .sort((a, b) => b.tokens - a.tokens);
43
+ const overBy = config.totalTokens - limit;
44
+ const drop = [];
45
+ let remaining = config.totalTokens;
46
+ for (const srv of measured) {
47
+ if (remaining <= limit)
48
+ break;
49
+ remaining -= srv.tokens;
50
+ drop.push({ name: srv.name, tokens: srv.tokens, remaining });
51
+ }
52
+ return {
53
+ overBy,
54
+ drop,
55
+ keptCount: measured.length - drop.length,
56
+ keptTokens: remaining,
57
+ // Removing everything measured still leaves unmeasured/base cost behind, so the
58
+ // honest test is whether the remainder actually landed under the limit.
59
+ feasible: remaining <= limit,
60
+ };
61
+ }
17
62
  /** Cache key for measurement reuse: the exact argv two configs would spawn. */
18
63
  export function serverKey(s) {
19
64
  return JSON.stringify(s.argv ?? [s.url ?? s.name]);
@@ -73,6 +118,7 @@ export function buildReport(configs, measured, opts = {}) {
73
118
  });
74
119
  continue;
75
120
  }
121
+ const divRow = opts.divergence?.servers[s.name];
76
122
  ok.push({
77
123
  ...base,
78
124
  status: m.status,
@@ -80,6 +126,7 @@ export function buildReport(configs, measured, opts = {}) {
80
126
  toolCount: m.toolCount,
81
127
  share: null, // filled once the total is known
82
128
  canonicalSha256: m.canonicalSha256,
129
+ claudeTokens: opts.divergence ? (isCurrent(divRow, m.canonicalSha256 ?? null) ? divRow.claudeDelta : null) : undefined,
83
130
  notes: m.status === 'dynamic' ? m.notes : undefined,
84
131
  });
85
132
  for (const t of m.tools)
@@ -105,6 +152,7 @@ export function buildReport(configs, measured, opts = {}) {
105
152
  servers: ok,
106
153
  skipped,
107
154
  heaviestTools: tools.slice(0, 5),
155
+ trimAdvice: buildTrimAdvice(tools, totalTokens),
108
156
  });
109
157
  }
110
158
  results.sort((a, b) => b.totalTokens - a.totalTokens);
@@ -116,15 +164,20 @@ export function buildReport(configs, measured, opts = {}) {
116
164
  configs: results,
117
165
  problems,
118
166
  };
167
+ if (opts.divergence) {
168
+ report.claudeDivergence = { model: opts.divergence.model, measuredAt: opts.divergence.measuredAt };
169
+ }
119
170
  if (typeof opts.budget === 'number') {
120
171
  // The worst config is the gate: passing because your *lightest* client fits
121
172
  // would be a green check on a session you don't run.
122
173
  const worst = results[0];
174
+ const over = (worst?.totalTokens ?? 0) > opts.budget;
123
175
  report.budget = {
124
176
  limit: opts.budget,
125
177
  worstTotal: worst?.totalTokens ?? 0,
126
178
  worstSource: worst?.source ?? '(none)',
127
- over: (worst?.totalTokens ?? 0) > opts.budget,
179
+ over,
180
+ fit: over && worst ? planBudgetFit(worst, opts.budget) : undefined,
128
181
  };
129
182
  }
130
183
  return report;
@@ -135,6 +188,7 @@ const pct = (x) => `${(x * 100).toFixed(1)}%`;
135
188
  export function formatReport(report) {
136
189
  const lines = [];
137
190
  lines.push(`mcp-context-cost audit · methodology ${report.methodologyVersion} · ${report.encoding} · context window ${n(report.contextWindow)}`);
191
+ const showClaude = !!report.claudeDivergence;
138
192
  for (const cfg of report.configs) {
139
193
  lines.push('');
140
194
  lines.push(`${cfg.client} ${cfg.source}`);
@@ -143,18 +197,21 @@ export function formatReport(report) {
143
197
  tools: s.toolCount === null ? '—' : String(s.toolCount),
144
198
  tokens: s.tokens === null ? '—' : n(s.tokens),
145
199
  share: s.share === null ? '—' : pct(s.share),
200
+ claude: s.claudeTokens == null ? '—' : n(s.claudeTokens),
146
201
  }));
147
202
  const w = {
148
203
  name: Math.max(6, ...rows.map((r) => r.name.length), 'total'.length),
149
204
  tools: Math.max(5, ...rows.map((r) => r.tools.length)),
150
205
  tokens: Math.max(6, ...rows.map((r) => r.tokens.length), n(cfg.totalTokens).length),
206
+ claude: Math.max(6, ...rows.map((r) => r.claude.length)),
151
207
  };
152
- const line = (name, tools, tokens, share) => ` ${name.padEnd(w.name)} ${tools.padStart(w.tools)} ${tokens.padStart(w.tokens)} ${share.padStart(6)}`;
153
- lines.push(line('server', 'tools', 'tokens', 'share'));
208
+ const line = (name, tools, tokens, share, claude) => ` ${name.padEnd(w.name)} ${tools.padStart(w.tools)} ${tokens.padStart(w.tokens)} ${share.padStart(6)}` +
209
+ (showClaude ? ` ${claude.padStart(w.claude)}` : '');
210
+ lines.push(line('server', 'tools', 'tokens', 'share', 'claude'));
154
211
  for (const r of rows)
155
- lines.push(line(r.name, r.tools, r.tokens, r.share));
156
- lines.push(` ${'─'.repeat(w.name + w.tools + w.tokens + 14)}`);
157
- lines.push(line('total', String(cfg.toolCount), n(cfg.totalTokens), ''));
212
+ lines.push(line(r.name, r.tools, r.tokens, r.share, r.claude));
213
+ lines.push(` ${'─'.repeat(w.name + w.tools + w.tokens + 14 + (showClaude ? w.claude + 2 : 0))}`);
214
+ lines.push(line('total', String(cfg.toolCount), n(cfg.totalTokens), '', ''));
158
215
  lines.push('');
159
216
  lines.push(` Every request in this client carries ${n(cfg.totalTokens)} tokens of tool schemas — ` +
160
217
  `${pct(cfg.contextShare)} of a ${n(report.contextWindow)}-token context window, before you type anything.`);
@@ -166,6 +223,13 @@ export function formatReport(report) {
166
223
  lines.push(` ${`${t.server} · ${t.tool}`.padEnd(tw)} ${n(t.tokens).padStart(7)}`);
167
224
  }
168
225
  }
226
+ if (cfg.trimAdvice) {
227
+ const names = cfg.trimAdvice.tools.map((t) => `${t.server}·${t.tool}`).join(', ');
228
+ lines.push('');
229
+ lines.push(` trim: disabling ${cfg.trimAdvice.tools.length} tool${cfg.trimAdvice.tools.length === 1 ? '' : 's'} ` +
230
+ `(${names}) would recover ${n(cfg.trimAdvice.recoverableTokens)} tokens ` +
231
+ `(${pct(cfg.trimAdvice.recoverableShare)} of this config) — if your client supports per-tool filtering.`);
232
+ }
169
233
  if (cfg.skipped.length) {
170
234
  lines.push('');
171
235
  lines.push(' not measured');
@@ -175,6 +239,11 @@ export function formatReport(report) {
175
239
  }
176
240
  }
177
241
  }
242
+ if (showClaude) {
243
+ lines.push('');
244
+ lines.push(` claude = Anthropic-request cost from the ${report.claudeDivergence.measuredAt} ${report.claudeDivergence.model} ` +
245
+ `divergence run, shown only where the published capture hash matches this install; '—' means no current match.`);
246
+ }
178
247
  if (report.problems.length) {
179
248
  lines.push('');
180
249
  lines.push('problems');
@@ -182,10 +251,52 @@ export function formatReport(report) {
182
251
  lines.push(` ${p}`);
183
252
  }
184
253
  if (report.budget) {
254
+ const b = report.budget;
255
+ lines.push('');
256
+ if (!b.over) {
257
+ const headroom = b.limit - b.worstTotal;
258
+ lines.push(`budget ok: ${n(b.worstTotal)} ≤ ${n(b.limit)} — ${n(headroom)} to spare`);
259
+ }
260
+ else {
261
+ lines.push(`BUDGET FAIL: ${n(b.worstTotal)} > ${n(b.limit)} (${b.worstSource})`);
262
+ const fit = b.fit;
263
+ if (fit && fit.drop.length) {
264
+ lines.push('');
265
+ lines.push(` over by ${n(fit.overBy)}. Heaviest-first, this is what gets you under:`);
266
+ for (const step of fit.drop) {
267
+ const verdict = step.remaining <= b.limit ? 'fits' : 'still over';
268
+ lines.push(` drop ${step.name.padEnd(22)} ${n(step.tokens).padStart(9)} → ${n(step.remaining).padStart(9)} ${verdict}`);
269
+ }
270
+ lines.push('');
271
+ if (fit.feasible && fit.keptCount === 0) {
272
+ // Arithmetically it fits, and the answer is useless: the only way under this
273
+ // limit is to run no servers at all. Saying "fits" here would be true and
274
+ // misleading, which is the pair this whole tool exists to keep apart.
275
+ lines.push(` no subset fits: every measured server would have to go. The limit is below`);
276
+ lines.push(` what any one of these servers costs.`);
277
+ }
278
+ else if (fit.feasible) {
279
+ const share = b.limit > 0 ? ` (${pct(fit.keptTokens / b.limit)} of budget)` : '';
280
+ lines.push(` keeps ${fit.keptCount} server(s) at ${n(fit.keptTokens)} tokens${share}`);
281
+ }
282
+ else {
283
+ lines.push(` even removing every measured server leaves ${n(fit.keptTokens)} — the limit is below this config's floor`);
284
+ }
285
+ lines.push('');
286
+ lines.push(' This is arithmetic, not advice: it cannot know which servers you need, and by');
287
+ lines.push(' weight alone it will sometimes name the one you cannot work without. Use --json');
288
+ lines.push(' for the full per-server list and pick your own order.');
289
+ }
290
+ else if (fit) {
291
+ lines.push(' nothing measured could be removed to get under the limit.');
292
+ }
293
+ }
294
+ }
295
+ if (report.diff)
296
+ lines.push(formatDiff(report.diff, report.contextWindow));
297
+ if (report.increaseGate) {
185
298
  lines.push('');
186
- lines.push(report.budget.over
187
- ? `BUDGET FAIL: ${n(report.budget.worstTotal)} > ${n(report.budget.limit)} (${report.budget.worstSource})`
188
- : `budget ok: ${n(report.budget.worstTotal)} ≤ ${n(report.budget.limit)}`);
299
+ lines.push(formatGate(report.increaseGate));
189
300
  }
190
301
  lines.push('');
191
302
  lines.push('These are wire tokens — what the server puts on the wire, counted with o200k_base. What your model is billed');
@@ -0,0 +1,124 @@
1
+ /**
2
+ * `audit --baseline <report.json>` — what a config change costs every future session.
3
+ *
4
+ * `audit` answers "what do my servers cost right now". That is a number a reader
5
+ * has to have an opinion about. A diff against a stored earlier report answers
6
+ * the question that needs no opinion at all: *this change adds 17,000 tokens to
7
+ * every request you will ever send from this client.* Same measurement path,
8
+ * same per-config discipline — a baseline is just an earlier `audit --json`.
9
+ *
10
+ * The trap this file exists to avoid: a server that measured fine before and
11
+ * fails to start now makes the total go DOWN. Subtracting two totals would
12
+ * report that as an improvement, which is the flattering reading and the true
13
+ * one having the same shape. So a server that changed measured-ness is never
14
+ * given a delta — it is named, its known side is printed, and the direction of
15
+ * the resulting error is stated ("understates by at least 9,246").
16
+ */
17
+ import type { AuditConfigResult, AuditReport } from './audit.js';
18
+ export type ServerDeltaKind = 'added' | 'removed' | 'changed' | 'unchanged'
19
+ /** Measured in the baseline, not measurable now — the total understates. */
20
+ | 'unmeasured-now'
21
+ /** Not measurable in the baseline, measured now — the increase overstates. */
22
+ | 'unmeasured-before'
23
+ /** Present and unmeasured in both runs — contributes 0 to both totals, but hides cost. */
24
+ | 'unmeasured-both';
25
+ export interface ServerDelta {
26
+ name: string;
27
+ kind: ServerDeltaKind;
28
+ /** Baseline tokens; `null` when absent from the baseline or unmeasured in it. */
29
+ before: number | null;
30
+ /** Current tokens; `null` when gone from the config or unmeasured now. */
31
+ after: number | null;
32
+ /** Signed change. `null` whenever the two sides are not the same kind of number. */
33
+ delta: number | null;
34
+ /** Why a delta is missing, in a sentence a reader can act on. */
35
+ note?: string;
36
+ }
37
+ export interface ConfigDiff {
38
+ client: string;
39
+ source: string;
40
+ /** How this config was paired with a baseline config. */
41
+ matchedBy: 'source' | 'sole-config' | 'unmatched';
42
+ beforeTotal: number | null;
43
+ afterTotal: number;
44
+ /** afterTotal - beforeTotal, or `null` when there is no baseline to subtract. */
45
+ delta: number | null;
46
+ beforeShare: number | null;
47
+ afterShare: number;
48
+ /**
49
+ * True when `delta` is the exact change in measured cost. False when a server
50
+ * crossed the measured/unmeasured line, which moves the total for a reason
51
+ * that is not a config change.
52
+ */
53
+ exact: boolean;
54
+ /** Tokens the diff is known to be missing, and which way it leans. */
55
+ understatedBy: number;
56
+ overstatedBy: number;
57
+ servers: ServerDelta[];
58
+ }
59
+ export interface AuditDiff {
60
+ baselineGeneratedAt: string;
61
+ baselineMethodologyVersion: string;
62
+ /** False when something makes the two reports incommensurable at all (methodology bump). */
63
+ comparable: boolean;
64
+ /** Baseline configs that no current config matched — never silently dropped. */
65
+ droppedConfigs: {
66
+ client: string;
67
+ source: string;
68
+ totalTokens: number;
69
+ }[];
70
+ warnings: string[];
71
+ configs: ConfigDiff[];
72
+ /**
73
+ * The largest per-config increase. Per config, never merged: a context window
74
+ * belongs to one client session, so a portfolio-wide "total delta" would
75
+ * describe a session nobody runs. `null` when nothing could be compared.
76
+ */
77
+ worstIncrease: {
78
+ source: string;
79
+ delta: number;
80
+ } | null;
81
+ }
82
+ /** Parse and shape-check a stored report. A baseline that cannot be read is never "no change". */
83
+ export declare function parseBaselineReport(text: string): {
84
+ report: AuditReport | null;
85
+ problem?: string;
86
+ };
87
+ export declare function diffConfig(before: AuditConfigResult | null, after: AuditConfigResult, matchedBy: ConfigDiff['matchedBy']): ConfigDiff;
88
+ /**
89
+ * Pair current configs with baseline configs.
90
+ *
91
+ * Exact source path first. Then one deliberate fallback: if each side has
92
+ * exactly one config, they are the same config seen from two machines — the CI
93
+ * case, where a baseline recorded at /Users/… meets a checkout at /home/runner/….
94
+ * Anything looser would pair two unrelated clients and call the difference a
95
+ * change, so everything else stays unmatched and says so.
96
+ */
97
+ export declare function pairConfigs(before: AuditConfigResult[], after: AuditConfigResult[]): {
98
+ pairs: {
99
+ before: AuditConfigResult | null;
100
+ after: AuditConfigResult;
101
+ matchedBy: ConfigDiff['matchedBy'];
102
+ }[];
103
+ dropped: AuditConfigResult[];
104
+ };
105
+ export declare function buildDiff(baseline: AuditReport, current: AuditReport): AuditDiff;
106
+ export declare function formatDiff(diff: AuditDiff, contextWindow: number): string;
107
+ export interface IncreaseGate {
108
+ limit: number;
109
+ pass: boolean;
110
+ /** The increase the gate measured, when it got far enough to measure one. */
111
+ increase: number | null;
112
+ reasons: string[];
113
+ }
114
+ /**
115
+ * `--max-increase N` — the CI gate. Fails on an increase over the limit, and
116
+ * equally on any reason the increase could not be established.
117
+ *
118
+ * That second half is the point. A gate that passes when a server failed to
119
+ * start, or when the baseline covered a config this run never found, is a green
120
+ * check on a question nobody asked. Everything this portfolio has learned says
121
+ * unchecked must not read as clean, so an inexact diff fails and names why.
122
+ */
123
+ export declare function evaluateIncreaseGate(diff: AuditDiff, limit: number): IncreaseGate;
124
+ export declare function formatGate(gate: IncreaseGate): string;