mcp-context-cost 0.9.0 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -193,9 +193,43 @@ still match today and silence for the rest. Most installs will show a mix:
193
193
  memory 9 2,378 4.2% —
194
194
  ```
195
195
 
196
+ Add `--suggest` to place each of your tools in the measured set's tool-shape distribution
197
+ ([method](docs/METHODOLOGY.md#tool-shape)) and get advice only where the data can point at
198
+ something. Only descriptions draw advice — schemas are functional surface; descriptions are
199
+ prose every request carries — and only descriptions at or above the 90th percentile of the
200
+ 1,150 measured tools:
201
+
202
+ ```
203
+ suggest — descriptions at or above the 90th percentile of measured tools
204
+ (baseline 2026-09-03: 1,150 tools across 81 measured servers):
205
+ stub · wordy — 345 tokens: description 321 (p92), schema 14
206
+ rewriting the description toward the measured median (27) would recover ≈294 tokens on every request
207
+ 1 of 2 tools sit inside the distribution — no advice where nothing is measurably unusual.
208
+ ```
209
+
210
+ A config where nothing is out of distribution is told that in those words, and a baseline
211
+ that cannot be fetched is a named problem, never a silently skipped check.
212
+
213
+ Add `--changed` to ask the other question — *did the servers I already have get heavier?*
214
+ Each installed server is identified against the published capture history by its canonical
215
+ hash, never by its name, because the name in your config is a label you chose and the bytes
216
+ are not:
217
+
218
+ ```
219
+ changed — published versions of your servers that have moved since
220
+ (index 2026-09-04, 2 published captures; matched by canonical hash, never by name):
221
+ notes (published as obsidian) — you have the capture published 2026-08-19 at 1,132 tokens;
222
+ the current one is 2,062 (+930, 2026-08-26)
223
+ updating all 1 would add 930 tokens to every request in this client.
224
+ ```
225
+
226
+ A server whose bytes match no published capture — a version never measured here, a fork, a
227
+ pin — is reported as unidentified with nothing claimed about it.
228
+ Method: [capture index](docs/METHODOLOGY.md#capture-index).
229
+
196
230
  Flags: `--json` (full report on stdout, progress on stderr), `--budget N`,
197
231
  `--baseline <report.json>`, `--max-increase N`, `--context N` (default 200,000),
198
- `--timeout ms`, `--concurrency N`, `--docker`, `--claude`.
232
+ `--timeout ms`, `--concurrency N`, `--docker`, `--claude`, `--suggest`, `--changed`.
199
233
 
200
234
  ## Where the numbers come from
201
235
 
@@ -221,6 +255,14 @@ sample of that range; the full range is in
221
255
  Each measured server also has a [detail page](https://athakur3.github.io/mcp-context-cost/servers/)
222
256
  showing which tools its tokens are in.)*
223
257
 
258
+ Because the set is re-measured on a rotation and most entries launch unpinned, the same data
259
+ answers a question no client asks: **what did this server cost last month?**
260
+ [results/regressions.md](results/regressions.md) reports each server's most recent movement —
261
+ dated to when it happened, separated into *shipped more tools* versus *same tools, rewritten*,
262
+ and compared only within one isolation. The ecosystem ratchets upward: of the servers whose
263
+ cost has moved at all, 9 moved up against 1 that moved down. Method:
264
+ [cost movement](docs/METHODOLOGY.md#cost-movement).
265
+
224
266
  If you publish a server, the same measurement is available as a badge, so your users can see
225
267
  the cost before they install rather than after:
226
268
 
@@ -304,6 +346,58 @@ Then in your README:
304
346
  [![context cost](https://img.shields.io/endpoint?url=<raw URL of badges/my-server.json>)](<link target>)
305
347
  ```
306
348
 
349
+ ### Defend the number, don't just display it
350
+
351
+ A badge says what your server costs today; it does nothing about the release
352
+ that adds 1,200 tokens to every user's context next month. Across the servers
353
+ measured here, that release is the norm rather than the exception — the
354
+ [movement report](results/regressions.md) has nine servers ratcheting upward
355
+ against one that got cheaper, and none of those maintainers had a check that
356
+ would have said so first. `measure` takes the same gate flags `audit` does, so
357
+ your own CI can be that check:
358
+
359
+ ```bash
360
+ # on your default branch, once — commit the result
361
+ npx -y mcp-context-cost measure --name my-server --command "node dist/index.js"
362
+ cp results/my-server/measurement.json .context-cost/baseline.json
363
+
364
+ # on every pull request
365
+ npx -y mcp-context-cost measure --name my-server --command "node dist/index.js" \
366
+ --baseline .context-cost/baseline.json --max-increase 500
367
+ ```
368
+
369
+ ```
370
+ diff vs baseline .context-cost/baseline.json
371
+ my-server: 61 → 182 +121 tokens (2 → 3 tools)
372
+ added: bulk_export (43)
373
+ grew: search 30 → 108 (+78)
374
+
375
+ INCREASE FAIL: +121 tokens, over the 100 allowed — this change adds that to every request of every install.
376
+ ```
377
+
378
+ Both sides are single measurements carrying per-tool counts, so an established
379
+ change is attributed exactly: which tools arrived, which grew, and by how much.
380
+ And `--max-increase` fails on more than the number — a server that stops
381
+ starting on the branch makes the total go *down*, and reporting that as an
382
+ improvement is the one mistake a gate like this must not make, so a change that
383
+ could not be established fails too.
384
+
385
+ As a GitHub Action, that whole workflow is five lines
386
+ ([full example](examples/server-author-ci.yml)):
387
+
388
+ ```yaml
389
+ - uses: athakur3/mcp-context-cost@v1
390
+ with:
391
+ name: my-server
392
+ command: node dist/index.js
393
+ baseline: .context-cost/baseline.json
394
+ max-increase: 500
395
+ ```
396
+
397
+ It exposes `tokens`, `tools`, `status`, `measurement` and `badge` as outputs —
398
+ available whether the gate passed or not — so a later step can comment the
399
+ number on the pull request or publish the badge.
400
+
307
401
  Point the link at the measurement behind the number — for servers in this sweep that is
308
402
  `https://athakur3.github.io/mcp-context-cost/servers/<name>.html`; otherwise the
309
403
  methodology page. A badge nobody can audit is decoration.
@@ -329,7 +423,10 @@ color bands are frozen against the observed distribution of the first full sweep
329
423
 
330
424
  ## Status
331
425
 
332
- Active. Every row carries the date of its own most recent measurement. Two
426
+ Active. Every row carries the date of its own most recent measurement, and what the data
427
+ says as a whole is written up, dated, in
428
+ [The State of MCP Context Cost](https://athakur3.github.io/mcp-context-cost/state-of-mcp-context-cost)
429
+ (September 2026). Two
333
430
  weekly jobs re-measure the set — the `memory` reference server on Mondays, and a rotating
334
431
  sixth of the list on Wednesdays, so every row comes round within six weeks. Read each row's
335
432
  date as the date it means, and don't take the cadence on trust — the build history is
@@ -1,4 +1,6 @@
1
1
  import { type DivergenceRun } from '../core/divergence.js';
2
+ import { type ToolShapeBaseline, type ToolSuggestion } from '../core/tool-shape.js';
3
+ import { type CaptureIndex, type CaptureVerdict } from '../core/capture-index.js';
2
4
  import type { Measurement, MeasurementStatus, ToolMeasurement } from '../core/types.js';
3
5
  import type { ConfiguredServer, LoadedConfig } from './config.js';
4
6
  import { type DeferralVerdict, type ToolSearchEnv, type ToolSearchSource } from './deferral.js';
@@ -43,6 +45,31 @@ export interface TrimAdvice {
43
45
  recoverableTokens: number;
44
46
  recoverableShare: number;
45
47
  }
48
+ /**
49
+ * `--suggest`: this config's tools placed in the measured set's tool-shape
50
+ * distribution. Only descriptions draw advice (schemas are functional surface;
51
+ * descriptions are prose every request carries), and only descriptions the
52
+ * baseline puts at or above the threshold percentile — a config where nothing
53
+ * is measurably unusual gets that said in those words, not advice invented to
54
+ * have some.
55
+ */
56
+ export interface ConfigSuggestions {
57
+ /** Heaviest-recovery first. */
58
+ outOfDistribution: ToolSuggestion[];
59
+ checkedTools: number;
60
+ }
61
+ /**
62
+ * `--changed`: one locally measured server placed against the published capture
63
+ * history, joined by canonical hash rather than by name. The local label is
64
+ * carried alongside the published server the bytes identify, because they need
65
+ * not agree — a config may call the official GitHub server anything at all, and
66
+ * a name that disagrees with the bytes is information rather than noise.
67
+ */
68
+ export interface ServerCaptureVerdict {
69
+ /** The name this config gave the server. */
70
+ name: string;
71
+ verdict: CaptureVerdict;
72
+ }
46
73
  export interface AuditConfigResult {
47
74
  client: string;
48
75
  source: string;
@@ -54,6 +81,10 @@ export interface AuditConfigResult {
54
81
  skipped: AuditServerResult[];
55
82
  heaviestTools: HeaviestTool[];
56
83
  trimAdvice: TrimAdvice | null;
84
+ /** Present only when `--suggest` ran with a usable baseline. */
85
+ suggestions?: ConfigSuggestions;
86
+ /** Present only when `--changed` ran with a usable capture index. */
87
+ captureVerdicts?: ServerCaptureVerdict[];
57
88
  /**
58
89
  * Whether this client loads the total up front or defers it, and — when the
59
90
  * client decides that by a threshold — which side of it this stack is on.
@@ -128,6 +159,17 @@ export interface AuditReport {
128
159
  model: string;
129
160
  measuredAt: string;
130
161
  };
162
+ /** Which published tool-shape baseline `--suggest` read its percentiles from. */
163
+ toolShape?: {
164
+ generatedAt: string;
165
+ toolCount: number;
166
+ serverCount: number;
167
+ };
168
+ /** Which published capture index `--changed` joined against. */
169
+ captureIndex?: {
170
+ generatedAt: string;
171
+ captureCount: number;
172
+ };
131
173
  /** Present only when a baseline report was supplied (`--baseline`). */
132
174
  diff?: AuditDiff;
133
175
  /** Present only when `--max-increase` was supplied alongside a baseline. */
@@ -160,6 +202,10 @@ export declare function buildReport(configs: LoadedConfig[], measured: Map<strin
160
202
  generatedAt?: string;
161
203
  /** Published `tools-delta/v1` run to join against (`--claude`); omit to skip the join. */
162
204
  divergence?: DivergenceRun | null;
205
+ /** Published `tool-shape/v1` baseline (`--suggest`); omit to skip suggestions. */
206
+ toolShape?: ToolShapeBaseline | null;
207
+ /** Published `capture-index/v1` (`--changed`); omit to skip the version join. */
208
+ captureIndex?: CaptureIndex | null;
163
209
  /**
164
210
  * The audited machine's SHELL tool-search variables. Passed in rather than
165
211
  * read here so this stays pure and a report is reproducible from its
@@ -14,10 +14,22 @@
14
14
  */
15
15
  import { METHODOLOGY_VERSION } from '../core/canonical.js';
16
16
  import { isCurrent } from '../core/divergence.js';
17
+ import { SUGGEST_DESCRIPTION_PERCENTILE, suggestFor, } from '../core/tool-shape.js';
18
+ import { identify } from '../core/capture-index.js';
17
19
  import { evaluateDeferral, PUBLISHED_WIRE_TO_CLIENT_RATIO, SHELL_SOURCE, } from './deferral.js';
18
20
  import { formatDiff, formatGate } from './diff.js';
19
21
  export const DEFAULT_CONTEXT_WINDOW = 200_000;
20
22
  const TRIM_TOOL_COUNT = 3;
23
+ function buildSuggestions(pool, baseline) {
24
+ const outOfDistribution = [];
25
+ for (const { server, t } of pool) {
26
+ const s = suggestFor(server, t, baseline);
27
+ if (s)
28
+ outOfDistribution.push(s);
29
+ }
30
+ outOfDistribution.sort((a, b) => b.approxRecoverableTokens - a.approxRecoverableTokens);
31
+ return { outOfDistribution, checkedTools: pool.length };
32
+ }
21
33
  function buildTrimAdvice(sortedTools, totalTokens) {
22
34
  if (totalTokens <= 0 || sortedTools.length < 2)
23
35
  return null;
@@ -189,6 +201,7 @@ export function buildReport(configs, measured, opts = {}) {
189
201
  const ok = [];
190
202
  const skipped = [];
191
203
  const tools = [];
204
+ const shapePool = [];
192
205
  // Counted only for servers that put a number into the total: a twin that
193
206
  // failed to launch is already a floor, and adds nothing to a sum.
194
207
  let sharedHere = 0;
@@ -240,8 +253,11 @@ export function buildReport(configs, measured, opts = {}) {
240
253
  claudeTokens: opts.divergence ? (isCurrent(divRow, m.canonicalSha256 ?? null) ? divRow.claudeDelta : null) : undefined,
241
254
  notes: m.status === 'dynamic' ? m.notes : undefined,
242
255
  });
243
- for (const t of m.tools)
256
+ for (const t of m.tools) {
244
257
  tools.push({ server: s.name, tool: t.name, tokens: t.tokens });
258
+ if (opts.toolShape)
259
+ shapePool.push({ server: s.name, t });
260
+ }
245
261
  }
246
262
  const totalTokens = ok.reduce((a, s) => a + (s.tokens ?? 0), 0);
247
263
  const toolCount = ok.reduce((a, s) => a + (s.toolCount ?? 0), 0);
@@ -264,6 +280,10 @@ export function buildReport(configs, measured, opts = {}) {
264
280
  skipped,
265
281
  heaviestTools: tools.slice(0, 5),
266
282
  trimAdvice: buildTrimAdvice(tools, totalTokens),
283
+ suggestions: opts.toolShape ? buildSuggestions(shapePool, opts.toolShape) : undefined,
284
+ captureVerdicts: opts.captureIndex
285
+ ? ok.map((s) => ({ name: s.name, verdict: identify(s.canonicalSha256, opts.captureIndex) }))
286
+ : undefined,
267
287
  };
268
288
  built.push(result);
269
289
  shared.set(result, sharedHere);
@@ -282,6 +302,19 @@ export function buildReport(configs, measured, opts = {}) {
282
302
  if (opts.divergence) {
283
303
  report.claudeDivergence = { model: opts.divergence.model, measuredAt: opts.divergence.measuredAt };
284
304
  }
305
+ if (opts.toolShape) {
306
+ report.toolShape = {
307
+ generatedAt: opts.toolShape.generatedAt,
308
+ toolCount: opts.toolShape.toolCount,
309
+ serverCount: opts.toolShape.serverCount,
310
+ };
311
+ }
312
+ if (opts.captureIndex) {
313
+ report.captureIndex = {
314
+ generatedAt: opts.captureIndex.generatedAt,
315
+ captureCount: Object.keys(opts.captureIndex.captures).length,
316
+ };
317
+ }
285
318
  if (typeof opts.budget === 'number') {
286
319
  // The worst config is the gate: passing because your *lightest* client fits
287
320
  // would be a green check on a session you don't run.
@@ -617,6 +650,68 @@ export function formatReport(report) {
617
650
  `(${names}) would recover ${n(cfg.trimAdvice.recoverableTokens)} tokens ` +
618
651
  `(${pct(cfg.trimAdvice.recoverableShare)} of this config) — if your client supports per-tool filtering.`);
619
652
  }
653
+ if (cfg.captureVerdicts) {
654
+ const behind = cfg.captureVerdicts.filter((v) => v.verdict.kind === 'behind');
655
+ const current = cfg.captureVerdicts.filter((v) => v.verdict.kind === 'current');
656
+ const unknown = cfg.captureVerdicts.length - behind.length - current.length;
657
+ const idx = report.captureIndex
658
+ ? `index ${report.captureIndex.generatedAt}, ${n(report.captureIndex.captureCount)} published captures`
659
+ : 'the published capture index';
660
+ lines.push('');
661
+ if (behind.length === 0) {
662
+ lines.push(` changed: no server here is running a published capture that has since moved ` +
663
+ `(${idx}; matched by canonical hash, never by name).`);
664
+ }
665
+ else {
666
+ const total = behind.reduce((a, v) => a + v.verdict.deltaTokens, 0);
667
+ lines.push(` changed — published versions of your servers that have moved since ` +
668
+ `(${idx}; matched by canonical hash, never by name):`);
669
+ for (const { name, verdict: v } of behind) {
670
+ // The local label and the published server are printed together: the
671
+ // bytes decide which server this is, and a name that disagrees is a
672
+ // fact worth seeing rather than one to smooth over.
673
+ const alias = name === v.server ? name : `${name} (published as ${v.server})`;
674
+ lines.push(` ${alias} — you have the capture published ${v.yourDate} at ${n(v.yourTokens)} tokens; ` +
675
+ `the current one is ${n(v.currentTokens)} (${v.deltaTokens >= 0 ? '+' : '−'}${n(Math.abs(v.deltaTokens))}, ${v.currentDate})`);
676
+ }
677
+ lines.push(` updating all ${behind.length} would ${total >= 0 ? 'add' : 'remove'} ${n(Math.abs(total))} tokens ` +
678
+ `${total >= 0 ? 'to' : 'from'} every request in this client.`);
679
+ }
680
+ if (unknown > 0) {
681
+ lines.push(` ${unknown} server${unknown === 1 ? '' : 's'} could not be identified: the installed bytes match no ` +
682
+ `published capture — a version never measured here, or one published before the index began. ` +
683
+ `Nothing is claimed about ${unknown === 1 ? 'it' : 'them'}.`);
684
+ }
685
+ }
686
+ if (cfg.suggestions) {
687
+ const sg = cfg.suggestions;
688
+ const base = report.toolShape
689
+ ? `baseline ${report.toolShape.generatedAt}: ${n(report.toolShape.toolCount)} tools across ` +
690
+ `${report.toolShape.serverCount} measured servers`
691
+ : 'published baseline';
692
+ lines.push('');
693
+ if (sg.outOfDistribution.length === 0) {
694
+ lines.push(` suggest: every description in this config sits inside the measured distribution — ` +
695
+ `nothing the data can point at (${sg.checkedTools} tools against ${base}).`);
696
+ }
697
+ else {
698
+ lines.push(` suggest — descriptions at or above the ${SUGGEST_DESCRIPTION_PERCENTILE}th percentile of ` +
699
+ `measured tools (${base}):`);
700
+ const shown = sg.outOfDistribution.slice(0, 8);
701
+ for (const s of shown) {
702
+ lines.push(` ${s.server} · ${s.tool} — ${n(s.tokens)} tokens: description ${n(s.descriptionTokens)} ` +
703
+ `(p${s.descriptionPercentile}), schema ${n(s.inputSchemaTokens)}`);
704
+ lines.push(` rewriting the description toward the measured median (${n(s.medianDescriptionTokens)}) ` +
705
+ `would recover ≈${n(s.approxRecoverableTokens)} tokens on every request`);
706
+ }
707
+ if (sg.outOfDistribution.length > shown.length) {
708
+ lines.push(` …and ${sg.outOfDistribution.length - shown.length} more above the threshold.`);
709
+ }
710
+ const within = sg.checkedTools - sg.outOfDistribution.length;
711
+ lines.push(` ${within} of ${sg.checkedTools} tools sit inside the distribution — ` +
712
+ `no advice where nothing is measurably unusual.`);
713
+ }
714
+ }
620
715
  if (cfg.skipped.length) {
621
716
  lines.push('');
622
717
  lines.push(' not measured');
@@ -1,10 +1,16 @@
1
1
  import type { Measurement } from '../core/types.js';
2
2
  import { type DivergenceRun } from '../core/divergence.js';
3
+ import { type ToolShapeBaseline } from '../core/tool-shape.js';
4
+ import { type CaptureIndex } from '../core/capture-index.js';
3
5
  import { type AuditReport } from './audit.js';
4
6
  import { type ToolSearchEnv, type ToolSearchSource } from './deferral.js';
5
7
  import { type LoadedConfig } from './config.js';
6
8
  /** Where the published `tools-delta/v1` run lives when `--claude` doesn't override it. */
7
9
  export declare const DEFAULT_DIVERGENCE_URL = "https://raw.githubusercontent.com/athakur3/mcp-context-cost/main/results/divergence.json";
10
+ /** Where the published `tool-shape/v1` baseline lives when `--suggest` doesn't override it. */
11
+ export declare const DEFAULT_TOOL_SHAPE_URL = "https://raw.githubusercontent.com/athakur3/mcp-context-cost/main/results/tool-shape.json";
12
+ /** Where the published `capture-index/v1` lives when `--changed` doesn't override it. */
13
+ export declare const DEFAULT_CAPTURE_INDEX_URL = "https://raw.githubusercontent.com/athakur3/mcp-context-cost/main/results/capture-index.json";
8
14
  export interface AuditOptions {
9
15
  /** Explicit config path(s); when empty, every known client location is tried. */
10
16
  configPaths?: string[];
@@ -19,6 +25,14 @@ export interface AuditOptions {
19
25
  claude?: boolean;
20
26
  /** Override the divergence.json source — mainly for tests and self-hosted mirrors. */
21
27
  divergenceUrl?: string;
28
+ /** Place this config's tools in the published tool-shape distribution and advise where the data can. */
29
+ suggest?: boolean;
30
+ /** Override the tool-shape.json source — mainly for tests and self-hosted mirrors. */
31
+ toolShapeUrl?: string;
32
+ /** Identify each server against the published capture history, by hash, and report what has moved. */
33
+ changed?: boolean;
34
+ /** Override the capture-index.json source — mainly for tests and self-hosted mirrors. */
35
+ captureIndexUrl?: string;
22
36
  /**
23
37
  * The tool-search variables as this process's SHELL has them. Defaults to
24
38
  * this process's environment. Overridable so a test can state a machine
@@ -33,6 +47,16 @@ export interface AuditOptions {
33
47
  settings?: ToolSearchSource[];
34
48
  onProgress?: (name: string, done: number, total: number) => void;
35
49
  }
50
+ /** Fetch and parse the published capture index. Never throws: a failure is a report problem, not a crash. */
51
+ export declare function fetchCaptureIndex(url: string): Promise<{
52
+ index: CaptureIndex | null;
53
+ problem?: string;
54
+ }>;
55
+ /** Fetch and parse the published tool-shape baseline. Never throws: a failure is a report problem, not a crash. */
56
+ export declare function fetchToolShape(url: string): Promise<{
57
+ baseline: ToolShapeBaseline | null;
58
+ problem?: string;
59
+ }>;
36
60
  /** Fetch and parse the published divergence run. Never throws: a failure is a report problem, not a crash. */
37
61
  export declare function fetchDivergence(url: string): Promise<{
38
62
  run: DivergenceRun | null;
package/dist/audit/run.js CHANGED
@@ -7,11 +7,43 @@
7
7
  import { homedir } from 'node:os';
8
8
  import { measureServer } from '../sweep/run.js';
9
9
  import { parseDivergence } from '../core/divergence.js';
10
+ import { parseToolShapeBaseline } from '../core/tool-shape.js';
11
+ import { parseCaptureIndex } from '../core/capture-index.js';
10
12
  import { buildReport, serverKey } from './audit.js';
11
13
  import { toolSearchEnv } from './deferral.js';
12
14
  import { configCandidates, loadConfigs, loadSettingsSources, settingsCandidates, } from './config.js';
13
15
  /** Where the published `tools-delta/v1` run lives when `--claude` doesn't override it. */
14
16
  export const DEFAULT_DIVERGENCE_URL = 'https://raw.githubusercontent.com/athakur3/mcp-context-cost/main/results/divergence.json';
17
+ /** Where the published `tool-shape/v1` baseline lives when `--suggest` doesn't override it. */
18
+ export const DEFAULT_TOOL_SHAPE_URL = 'https://raw.githubusercontent.com/athakur3/mcp-context-cost/main/results/tool-shape.json';
19
+ /** Where the published `capture-index/v1` lives when `--changed` doesn't override it. */
20
+ export const DEFAULT_CAPTURE_INDEX_URL = 'https://raw.githubusercontent.com/athakur3/mcp-context-cost/main/results/capture-index.json';
21
+ /** Fetch and parse the published capture index. Never throws: a failure is a report problem, not a crash. */
22
+ export async function fetchCaptureIndex(url) {
23
+ try {
24
+ const res = await fetch(url, { signal: AbortSignal.timeout(15_000) });
25
+ if (!res.ok)
26
+ return { index: null, problem: `capture index: HTTP ${res.status} fetching ${url}` };
27
+ const index = parseCaptureIndex(await res.text());
28
+ return index ? { index } : { index: null, problem: `capture index: malformed data at ${url}` };
29
+ }
30
+ catch (e) {
31
+ return { index: null, problem: `capture index: failed to fetch ${url}: ${e.message}` };
32
+ }
33
+ }
34
+ /** Fetch and parse the published tool-shape baseline. Never throws: a failure is a report problem, not a crash. */
35
+ export async function fetchToolShape(url) {
36
+ try {
37
+ const res = await fetch(url, { signal: AbortSignal.timeout(15_000) });
38
+ if (!res.ok)
39
+ return { baseline: null, problem: `tool shape: HTTP ${res.status} fetching ${url}` };
40
+ const baseline = parseToolShapeBaseline(await res.text());
41
+ return baseline ? { baseline } : { baseline: null, problem: `tool shape: malformed data at ${url}` };
42
+ }
43
+ catch (e) {
44
+ return { baseline: null, problem: `tool shape: failed to fetch ${url}: ${e.message}` };
45
+ }
46
+ }
15
47
  /** Fetch and parse the published divergence run. Never throws: a failure is a report problem, not a crash. */
16
48
  export async function fetchDivergence(url) {
17
49
  try {
@@ -89,14 +121,34 @@ export async function runAudit(opts = {}) {
89
121
  divergence = fetched.run;
90
122
  divergenceProblem = fetched.problem;
91
123
  }
124
+ let toolShape = null;
125
+ let toolShapeProblem;
126
+ if (opts.suggest) {
127
+ const fetched = await fetchToolShape(opts.toolShapeUrl ?? DEFAULT_TOOL_SHAPE_URL);
128
+ toolShape = fetched.baseline;
129
+ toolShapeProblem = fetched.problem;
130
+ }
131
+ let captureIndex = null;
132
+ let captureIndexProblem;
133
+ if (opts.changed) {
134
+ const fetched = await fetchCaptureIndex(opts.captureIndexUrl ?? DEFAULT_CAPTURE_INDEX_URL);
135
+ captureIndex = fetched.index;
136
+ captureIndexProblem = fetched.problem;
137
+ }
92
138
  const report = buildReport(configs, measured, {
93
139
  contextWindow: opts.contextWindow,
94
140
  budget: opts.budget,
95
141
  divergence,
142
+ toolShape,
143
+ captureIndex,
96
144
  env: opts.env ?? toolSearchEnv(process.env),
97
145
  settings: opts.settings ?? discoverSettings(opts),
98
146
  });
99
147
  if (divergenceProblem)
100
148
  report.problems.push(divergenceProblem);
149
+ if (toolShapeProblem)
150
+ report.problems.push(toolShapeProblem);
151
+ if (captureIndexProblem)
152
+ report.problems.push(captureIndexProblem);
101
153
  return report;
102
154
  }
package/dist/cli.js CHANGED
@@ -99,8 +99,19 @@ function rejectUnknownFlags(cmd, argv, spec) {
99
99
  const [, , cmd, ...rest] = process.argv;
100
100
  if (cmd === 'audit') {
101
101
  rejectUnknownFlags('audit', rest, {
102
- value: ['config', 'budget', 'baseline', 'max-increase', 'context', 'timeout', 'concurrency', 'divergence-url'],
103
- boolean: ['json', 'docker', 'claude'],
102
+ value: [
103
+ 'config',
104
+ 'budget',
105
+ 'baseline',
106
+ 'max-increase',
107
+ 'context',
108
+ 'timeout',
109
+ 'concurrency',
110
+ 'divergence-url',
111
+ 'tool-shape-url',
112
+ 'capture-index-url',
113
+ ],
114
+ boolean: ['json', 'docker', 'claude', 'suggest', 'changed'],
104
115
  });
105
116
  const argOf = (name) => {
106
117
  const i = rest.indexOf(`--${name}`);
@@ -171,6 +182,10 @@ if (cmd === 'audit') {
171
182
  docker: rest.includes('--docker'),
172
183
  claude: rest.includes('--claude'),
173
184
  divergenceUrl: argOf('divergence-url'),
185
+ suggest: rest.includes('--suggest'),
186
+ toolShapeUrl: argOf('tool-shape-url'),
187
+ changed: rest.includes('--changed'),
188
+ captureIndexUrl: argOf('capture-index-url'),
174
189
  // Progress goes to stderr so `--json` stdout stays a single parseable object.
175
190
  onProgress: json ? undefined : (name, done, total) => process.stderr.write(` [${done}/${total}] ${name}\n`),
176
191
  });
@@ -280,7 +295,7 @@ else if (cmd === 'verify') {
280
295
  }
281
296
  else if (cmd === 'measure') {
282
297
  rejectUnknownFlags('measure', rest, {
283
- value: ['name', 'command', 'remote', 'timeout', 'docker-image'],
298
+ value: ['name', 'command', 'remote', 'timeout', 'docker-image', 'baseline', 'max-increase', 'budget'],
284
299
  boolean: ['docker'],
285
300
  });
286
301
  const argOf = (name) => {
@@ -303,6 +318,45 @@ else if (cmd === 'measure') {
303
318
  console.error('usage: mcp-context-cost measure --name <slug> --command "npx -y <server>" [--timeout ms] [--docker]');
304
319
  process.exit(2);
305
320
  }
321
+ const { diffServer, evaluateServerGate, formatServerDiff, parseBaselineMeasurement, } = await import('./core/server-diff.js');
322
+ // Gate limits are read before anything is measured: an unusable number is a
323
+ // usage error, and finding that out after a two-minute container launch is
324
+ // the wrong time to find it out.
325
+ const numericFlag = (flag) => {
326
+ const raw = argOf(flag);
327
+ if (raw === undefined)
328
+ return undefined;
329
+ const v = Number(raw);
330
+ if (!Number.isFinite(v) || v < 0) {
331
+ console.error(`--${flag} must be a non-negative number, got '${raw}'`);
332
+ process.exit(2);
333
+ }
334
+ return v;
335
+ };
336
+ const maxIncrease = numericFlag('max-increase');
337
+ const budget = numericFlag('budget');
338
+ const baselinePath = argOf('baseline');
339
+ if (maxIncrease !== undefined && !baselinePath) {
340
+ console.error('--max-increase needs --baseline <measurement.json> to compare against');
341
+ process.exit(2);
342
+ }
343
+ let baseline = null;
344
+ if (baselinePath) {
345
+ let raw;
346
+ try {
347
+ raw = readFileSync(baselinePath, 'utf8');
348
+ }
349
+ catch (e) {
350
+ console.error(`cannot read baseline ${baselinePath}: ${e.message}`);
351
+ process.exit(2);
352
+ }
353
+ const parsed = parseBaselineMeasurement(raw);
354
+ if (!parsed.measurement) {
355
+ console.error(`${baselinePath}: ${parsed.problem}`);
356
+ process.exit(2);
357
+ }
358
+ baseline = parsed.measurement;
359
+ }
306
360
  const { measureServer } = await import('./sweep/run.js');
307
361
  const m = await measureServer(name, remoteUrl ? `npx -y mcp-remote ${remoteUrl}` : command, {
308
362
  timeoutMs: Number(argOf('timeout') ?? 60_000),
@@ -314,6 +368,20 @@ else if (cmd === 'measure') {
314
368
  console.log(ok
315
369
  ? `${name}: ${m.totalTokens} tokens across ${m.toolCount} tools (${m.status}) — results/${name}/measurement.json, badges/${name}.json`
316
370
  : `${name}: ${m.status} — ${m.notes ?? ''}`);
371
+ if (baseline || budget !== undefined || maxIncrease !== undefined) {
372
+ const diff = diffServer(name, baseline, m);
373
+ if (baseline) {
374
+ console.log('');
375
+ console.log(`diff vs baseline ${baselinePath}`);
376
+ console.log(formatServerDiff(diff));
377
+ }
378
+ const gate = evaluateServerGate(diff, { budget, maxIncrease });
379
+ if (!gate.pass) {
380
+ console.log('');
381
+ console.error(gate.failure);
382
+ process.exit(1);
383
+ }
384
+ }
317
385
  process.exit(ok ? 0 : 1);
318
386
  }
319
387
  else if (cmd !== undefined && cmd !== '--help' && cmd !== '-h') {
@@ -331,5 +399,8 @@ else {
331
399
  console.log(' verify --remote <url> [--json] same, fetched from a measurement URL');
332
400
  console.log(' measure --name x --command "npx -y <server>" run a one-off measurement');
333
401
  console.log(' measure --remote <url> [--name x] measure a remote server via mcp-remote');
402
+ console.log(' [--baseline <measurement.json>] [--max-increase N] [--budget N]');
403
+ console.log(' gate your own server in CI: fail the');
404
+ console.log(' build when a change adds too much');
334
405
  console.log('exit codes: 0 ok, 1 verification/measurement/budget failed, 2 usage error');
335
406
  }
@@ -0,0 +1,76 @@
1
+ /**
2
+ * Which published version is this, by its bytes?
3
+ *
4
+ * `audit --changed` answers "did the servers in my config get heavier?" — and
5
+ * the whole question turns on joining a machine's installed server to the
6
+ * published history. Matching by *name* would be a lie waiting to happen: a
7
+ * config's keys are arbitrary local labels, so a server a user calls `github`
8
+ * may be a fork, a pin, or something else entirely, and reporting the official
9
+ * server's movement against it would be a confident false statement.
10
+ *
11
+ * So the join is byte identity — the same discipline the Claude column already
12
+ * uses to decide whether it may print. `results/capture-index.json` maps the
13
+ * `canonicalSha256` of every capture the project has ever published to the
14
+ * server and date it belongs to. A local measurement either *is* one of those
15
+ * captures, exactly, or it is not in the published history at all, and there is
16
+ * no third state to be fuzzy about.
17
+ *
18
+ * The index is derivable from the per-server tool vectors, which is why it can
19
+ * only see as far back as those vectors go: a version published before the
20
+ * vectors existed is not in the index and reads as unknown, not as absent from
21
+ * history. Versioned independently, like every published artifact.
22
+ */
23
+ /** Method identifier, versioned independently of METHODOLOGY_VERSION. */
24
+ export declare const CAPTURE_INDEX_METHOD = "capture-index/v1";
25
+ export interface IndexedCapture {
26
+ server: string;
27
+ /** The day this capture was first measured. */
28
+ date: string;
29
+ totalTokens: number;
30
+ toolCount: number;
31
+ }
32
+ export interface CaptureIndex {
33
+ method: string;
34
+ /** UTC day the index was derived (YYYY-MM-DD). */
35
+ generatedAt: string;
36
+ /** canonicalSha256 → the capture it identifies. */
37
+ captures: Record<string, IndexedCapture>;
38
+ /** server → the newest published capture's hash. */
39
+ current: Record<string, string>;
40
+ }
41
+ export declare function parseCaptureIndex(text: string): CaptureIndex | null;
42
+ /**
43
+ * What the index can say about one locally measured server.
44
+ *
45
+ * `unknown` is the honest and common outcome: users install versions this
46
+ * project has never measured, and versions published before the capture index
47
+ * existed are not in it either. It is reported, not hidden — an absence of a
48
+ * record about that version, not a statement that nothing changed.
49
+ */
50
+ export type CaptureVerdict = {
51
+ kind: 'behind';
52
+ /** The published server this capture belongs to — established by bytes, not by name. */
53
+ server: string;
54
+ yourDate: string;
55
+ yourTokens: number;
56
+ currentDate: string;
57
+ currentTokens: number;
58
+ /** What moving to the current published version would add to every request. */
59
+ deltaTokens: number;
60
+ } | {
61
+ kind: 'current';
62
+ server: string;
63
+ date: string;
64
+ tokens: number;
65
+ } | {
66
+ kind: 'unknown';
67
+ };
68
+ /**
69
+ * Identify a local capture against the published index.
70
+ *
71
+ * A hash that matches the server's newest published capture is `current`; one
72
+ * that matches an older capture is `behind`, carrying the exact delta to
73
+ * current. Anything else — including a version newer than anything published —
74
+ * is `unknown`, because the index cannot describe what it has never measured.
75
+ */
76
+ export declare function identify(canonicalSha256: string | null | undefined, index: CaptureIndex): CaptureVerdict;