mcp-context-cost 0.11.3 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,6 +2,27 @@
2
2
 
3
3
  [![npm](https://img.shields.io/npm/v/mcp-context-cost)](https://www.npmjs.com/package/mcp-context-cost)
4
4
  [![CI](https://github.com/athakur3/mcp-context-cost/actions/workflows/ci.yml/badge.svg)](https://github.com/athakur3/mcp-context-cost/actions/workflows/ci.yml)
5
+ [![GitHub Marketplace](https://img.shields.io/badge/GitHub%20Marketplace-MCP%20context%20cost-blue?logo=github)](https://github.com/marketplace/actions/mcp-context-cost)
6
+
7
+ Two entry points: `audit`, for the config you run (below), and a GitHub Action, for the
8
+ server you publish — five lines in a workflow that fail a pull request adding more context
9
+ cost than you meant to ship:
10
+
11
+ ```yaml
12
+ - uses: athakur3/mcp-context-cost@v1
13
+ with:
14
+ name: my-server
15
+ command: node dist/index.js
16
+ baseline: .context-cost/baseline.json
17
+ max-increase: 500
18
+ ```
19
+
20
+ Inputs: `name`, `command` or `remote`, `baseline`, `max-increase`, `budget`, `timeout`,
21
+ `version`. Outputs: `tokens`, `tools`, `status`, `measurement`, `badge` — written whether the
22
+ gate passed or not, so a later step can comment the number on the PR or publish the badge.
23
+ [Full workflow](examples/server-author-ci.yml) ·
24
+ [how the gate decides](#defend-the-number-dont-just-display-it) ·
25
+ [what the number is](docs/METHODOLOGY.md).
5
26
 
6
27
  **What do the MCP servers in your config cost you before you type anything — and what did
7
28
  that last config change add to every session you will ever run?**
@@ -105,7 +126,7 @@ costs **more** than loading the definitions would.
105
126
 
106
127
  Three things the report will not do: it will not convert between units silently (in
107
128
  threshold mode the stack is compared as a range, because the audit counts wire bytes and the
108
- threshold is counted in what the client sends to the API — measured at 0.20×–1.92× across 20
129
+ threshold is counted in what the client sends to the API — measured at 0.20×–1.92× across 23
109
130
  servers); it will not claim a posture the machine did not state readably, which is four
110
131
  refusals and not one — when two places set the same variable to different values, when a
111
132
  settings file exists and cannot be read, when the place that would decide sets the variable
@@ -183,8 +204,8 @@ INCREASE FAIL:
183
204
  Add `--claude` to annotate each server with its Anthropic-request cost from the published
184
205
  [Claude divergence](docs/METHODOLOGY.md#claude-divergence) run — an exact number when the
185
206
  published capture hash matches what you have installed, `—` (silence, not a stale guess)
186
- when it doesn't. The run holds 20 rows — the top 20 measured servers by tokens when it ran —
187
- and [results/leaderboard.md](results/leaderboard.md) prints a claude number for the 16 that
207
+ when it doesn't. The run holds 23 rows — the measured servers it covered when it last ran —
208
+ and [results/leaderboard.md](results/leaderboard.md) prints a claude number for the 15 that
188
209
  still match today and silence for the rest. Most installs will show a mix:
189
210
 
190
211
  ```
@@ -235,7 +256,7 @@ Flags: `--json` (full report on stdout, progress on stderr), `--budget N`,
235
256
 
236
257
  The number `audit` gives you is the same measurement, run across a curated set of public
237
258
  servers — which is how you can tell it is a measurement and not this tool's opinion. It also
238
- shows what you are choosing between: across the 81 servers measured, cost spans **1,700×**,
259
+ shows what you are choosing between: across the 86 servers measured, cost spans **1,700×**,
239
260
  from `postgres` at 32 tokens to `github` at 54,622. The table below is a
240
261
  sample of that range; the full range is in
241
262
  [results/leaderboard.md](results/leaderboard.md).
@@ -244,13 +265,13 @@ sample of that range; the full range is in
244
265
  |---|---:|---:|
245
266
  | github (official) | **54,622 tokens** | 44 |
246
267
  | xcodebuildmcp | 26,594 | 24 |
247
- | brave-search | 25,456 | 8 |
268
+ | brave-search | 25,487 | 8 |
248
269
  | notion | 17,500 | 24 |
249
270
  | playwright *(4.8M installs/week)* | 4,024 | 24 |
250
271
  | filesystem (reference) | 2,823 | 14 |
251
272
  | markitdown | 64 | 1 |
252
273
 
253
- *(81 of 106 popular servers measured, each row dated by its own most recent sweep — full table in
274
+ *(86 of 106 popular servers measured, each row dated by its own most recent sweep — full table in
254
275
  [results/leaderboard.md](results/leaderboard.md); every failure is listed with its reason.
255
276
  Each measured server also has a [detail page](https://athakur3.github.io/mcp-context-cost/servers/)
256
277
  showing which tools its tokens are in.)*
@@ -260,7 +281,7 @@ answers a question no client asks: **what did this server cost last month?**
260
281
  [results/regressions.md](results/regressions.md) reports each server's most recent movement —
261
282
  dated to when it happened, separated into *shipped more tools* versus *same tools, rewritten*,
262
283
  and compared only within one isolation. The ecosystem ratchets upward: of the servers whose
263
- cost has moved at all, 9 moved up against 1 that moved down. Method:
284
+ cost has moved at all, 11 moved up against 6 that moved down. Method:
264
285
  [cost movement](docs/METHODOLOGY.md#cost-movement).
265
286
 
266
287
  If you publish a server, the same measurement is available as a badge, so your users can see
@@ -328,7 +349,7 @@ number is *not*, config policy, failure taxonomy, frozen color bands, known dive
328
349
  | `spec/fixtures/` | golden vectors shared by the TypeScript and bash implementations |
329
350
  | `tools/` | the one script that calls a network API (Claude divergence); kept out of the package so the library stays offline |
330
351
  | `upstream/` | `badge.sh` + composite-action patch + bash tests — the self-serve badge recipe, carried here |
331
- | `servers.yaml` | 82 curated candidates with live install metrics and provenance |
352
+ | `servers.yaml` | 106 curated candidates with live install metrics and provenance |
332
353
  | `results/` · `badges/` | measurements, leaderboard, history series, shields endpoint JSONs |
333
354
  | `docs/` | methodology, dashboard, and a generated page per measured server |
334
355
 
@@ -350,11 +371,11 @@ Then in your README:
350
371
 
351
372
  A badge says what your server costs today; it does nothing about the release
352
373
  that adds 1,200 tokens to every user's context next month. Across the servers
353
- measured here, that release is the norm rather than the exception the
354
- [movement report](results/regressions.md) has nine servers ratcheting upward
355
- against one that got cheaper, and none of those maintainers had a check that
356
- would have said so first. `measure` takes the same gate flags `audit` does, so
357
- your own CI can be that check:
374
+ measured here most costs hold steady from sweep to sweep, but when a cost does
375
+ move it usually moves up: the [movement report](results/regressions.md) has 11
376
+ servers ratcheting upward against 6 that got cheaper, and none of those
377
+ maintainers had a check that would have said so first. `measure` takes the same
378
+ gate flags `audit` does, so your own CI can be that check:
358
379
 
359
380
  ```bash
360
381
  # on your default branch, once — commit the result
@@ -229,7 +229,20 @@ export function resolveToolSearchSources(sources) {
229
229
  export const PUBLISHED_WIRE_TO_CLIENT_RATIO = {
230
230
  low: 0.2,
231
231
  high: 1.92,
232
- servers: 20,
232
+ // A snapshot of the run this package was cut against, which is what `source`
233
+ // below says it is — the installed package has no `results/` to read, so when
234
+ // a live run is supplied `wireToClientRatio` uses that instead and this is
235
+ // never consulted. It may therefore lag the run on trunk, and a test holds it
236
+ // to the two things that matter: the band must still be accurate to the
237
+ // precision it is published at, because a wrong band gives a wrong
238
+ // above/below verdict, and the count must never *exceed* the run, because
239
+ // that would be a claim about servers nobody measured.
240
+ //
241
+ // It read 20 from the day the run covered the top 20 until the run widened to
242
+ // every measured server on 2026-09-05, with nothing comparing the two. The
243
+ // band had not moved — the servers added sat inside it — which is exactly how
244
+ // a number like this goes wrong quietly.
245
+ servers: 23,
233
246
  source: 'the published claude-opus-5 divergence run',
234
247
  };
235
248
  /** Derive the band from a supplied divergence run, falling back to the published one. */
@@ -42,6 +42,8 @@ export interface DatedMeasurement {
42
42
  tokens: number;
43
43
  toolCount: number;
44
44
  status: string;
45
+ /** What the server called itself at `initialize`, or `''` when not recorded. */
46
+ version?: string;
45
47
  }
46
48
  /** Method identifier, versioned independently of METHODOLOGY_VERSION. */
47
49
  export declare const REGRESSION_METHOD = "cost-regression/v1";
@@ -67,6 +69,14 @@ export interface ToolVectorEntry {
67
69
  canonicalSha256: string;
68
70
  totalTokens: number;
69
71
  tools: ToolVector[];
72
+ /**
73
+ * The version the server reported at `initialize`, absent when it reported
74
+ * none or when the entry was stored before this field existed. Entries are
75
+ * deduped by capture and never rewritten, so an existing one does not gain a
76
+ * version retroactively — which is correct: identical bytes are the same
77
+ * definitions, and what upstream called them that day is not on record.
78
+ */
79
+ version?: string;
70
80
  }
71
81
  export interface ToolVectorFile {
72
82
  method: string;
@@ -150,6 +160,17 @@ export interface CostChange {
150
160
  fromToolCount: number;
151
161
  toToolCount: number;
152
162
  deltaTools: number;
163
+ /**
164
+ * What the server called itself on each side, or `''` where the row does not
165
+ * say. Never inferred: a movement whose earlier row predates the version
166
+ * column names no release, because nothing on disk records which one it was.
167
+ *
168
+ * The two being *equal* is a reading in its own right, and a common one —
169
+ * the cost moved while the version did not, which means the change came from
170
+ * a dependency the server does not pin rather than from its own release.
171
+ */
172
+ fromVersion: string;
173
+ toVersion: string;
153
174
  mechanism: Mechanism;
154
175
  /** True when the movement clears both thresholds. */
155
176
  significant: boolean;
@@ -162,11 +183,44 @@ export interface CostChange {
162
183
  /** Per-tool breakdown, or null when the two captures are not both on record. */
163
184
  attribution: ToolAttribution | null;
164
185
  }
186
+ /**
187
+ * A cost that has been measured more than once, under the same isolation, and
188
+ * has not moved.
189
+ *
190
+ * Published because it is a measured fact and not the absence of one. The
191
+ * report used to swallow it: `latestChange` returned null both for this and
192
+ * for "there is nothing to compare", and the page counted every null the same
193
+ * way — so a number four sweeps had confirmed read exactly like a number
194
+ * measured once.
195
+ */
196
+ export interface UnchangedSeries {
197
+ server: string;
198
+ tokens: number;
199
+ toolCount: number;
200
+ /** The first date this cost was recorded — when it became this, not when it was last looked at. */
201
+ since: string;
202
+ /** The newest measurement on record. */
203
+ measuredThrough: string;
204
+ /** How many comparable measurements carry this same cost, `since` included. */
205
+ sweeps: number;
206
+ }
207
+ /**
208
+ * What one server's series supports. Three readings, and they are three
209
+ * different claims: the cost moved, the cost was confirmed and held, or there
210
+ * is not yet a second comparable measurement to say either.
211
+ */
212
+ export type SeriesReading = {
213
+ kind: 'changed';
214
+ change: CostChange;
215
+ } | {
216
+ kind: 'unchanged';
217
+ held: UnchangedSeries;
218
+ } | {
219
+ kind: 'incomparable';
220
+ };
165
221
  export declare function isSignificant(deltaTokens: number, deltaPct: number): boolean;
166
222
  /**
167
- * The most recent comparable *movement* for one server, or null when there
168
- * isn't one: fewer than two comparable measurements, or a series that has never
169
- * changed.
223
+ * What one server's history supports, as one of three readings.
170
224
  *
171
225
  * Deliberately not "the newest pair". A server that grew 82% one week and held
172
226
  * that cost since has a newest pair of zero, and reporting only that would hide
@@ -176,10 +230,23 @@ export declare function isSignificant(deltaTokens: number, deltaPct: number): bo
176
230
  * actually happened — `2026-08-19 → 2026-08-26`, not a span up to today.
177
231
  * `measuredThrough` then carries how long the new cost has held.
178
232
  *
233
+ * Where that walk ends decides the reading. Reaching the start of the series
234
+ * means every comparable measurement agrees, which is `unchanged` — a fact
235
+ * about the server. Having fewer than two comparable rows to walk means there
236
+ * is nothing to compare, which is `incomparable` — an absence. The two used
237
+ * to share one `null` return, and the report counted them together.
238
+ *
179
239
  * `rows` is that server's history, already narrowed to the run a trend may be
180
240
  * drawn across (`plottableSeries`), so the isolation rule is applied once, in
181
241
  * the place that owns it.
182
242
  */
243
+ export declare function readSeries(server: string, rows: DatedMeasurement[], vectors?: ToolVectorFile | null): SeriesReading;
244
+ /**
245
+ * The most recent comparable movement, or null when there was none. Kept as
246
+ * the narrow question `readSeries` answers in full, for the callers that only
247
+ * ever wanted a movement — but a caller that publishes a count must ask
248
+ * `readSeries`, because null here is two different facts.
249
+ */
183
250
  export declare function latestChange(server: string, rows: DatedMeasurement[], vectors?: ToolVectorFile | null): CostChange | null;
184
251
  export interface RegressionSummary {
185
252
  changes: CostChange[];
@@ -188,7 +255,22 @@ export interface RegressionSummary {
188
255
  significant: number;
189
256
  /** Net tokens the measured set gained (or lost) across every listed movement. */
190
257
  netTokens: number;
191
- /** Servers with a measurement but no second comparable one to diff against. */
258
+ /**
259
+ * Servers measured more than once under the same isolation whose cost has
260
+ * not moved. Counted apart from `withoutComparison`, which they were folded
261
+ * into while both readings shared one `null`.
262
+ */
263
+ unchanged: UnchangedSeries[];
264
+ /**
265
+ * Servers with no second comparable measurement to diff against — a first
266
+ * measurement, or every earlier run taken under different isolation. Not
267
+ * "has not moved": that is `unchanged`.
268
+ */
192
269
  withoutComparison: number;
193
270
  }
194
- export declare function summarize(changes: CostChange[], withoutComparison: number): RegressionSummary;
271
+ /**
272
+ * `unchanged` is required rather than defaulted: every reading a caller
273
+ * classifies belongs to exactly one of the three, and a default would let a
274
+ * caller that forgot them publish a total that quietly does not sum.
275
+ */
276
+ export declare function summarize(changes: CostChange[], withoutComparison: number, unchanged: UnchangedSeries[]): RegressionSummary;
@@ -38,6 +38,7 @@ export function parseToolVectorFile(text) {
38
38
  date: e.date,
39
39
  canonicalSha256: e.canonicalSha256,
40
40
  totalTokens: e.totalTokens,
41
+ ...(typeof e.version === 'string' && e.version ? { version: e.version } : {}),
41
42
  tools: e.tools
42
43
  .filter((t) => !!t && typeof t.name === 'string' && typeof t.tokens === 'number')
43
44
  .map((t) => ({ name: t.name, tokens: t.tokens })),
@@ -62,6 +63,7 @@ export function vectorEntryOf(m) {
62
63
  date,
63
64
  canonicalSha256: m.canonicalSha256,
64
65
  totalTokens: m.totalTokens,
66
+ ...(m.serverVersion ? { version: m.serverVersion } : {}),
65
67
  tools: m.tools.map((t) => ({ name: t.name, tokens: t.tokens })),
66
68
  };
67
69
  }
@@ -139,9 +141,7 @@ export function isSignificant(deltaTokens, deltaPct) {
139
141
  return Math.abs(deltaPct) >= SIGNIFICANT_PCT && Math.abs(deltaTokens) >= SIGNIFICANT_TOKENS;
140
142
  }
141
143
  /**
142
- * The most recent comparable *movement* for one server, or null when there
143
- * isn't one: fewer than two comparable measurements, or a series that has never
144
- * changed.
144
+ * What one server's history supports, as one of three readings.
145
145
  *
146
146
  * Deliberately not "the newest pair". A server that grew 82% one week and held
147
147
  * that cost since has a newest pair of zero, and reporting only that would hide
@@ -151,14 +151,20 @@ export function isSignificant(deltaTokens, deltaPct) {
151
151
  * actually happened — `2026-08-19 → 2026-08-26`, not a span up to today.
152
152
  * `measuredThrough` then carries how long the new cost has held.
153
153
  *
154
+ * Where that walk ends decides the reading. Reaching the start of the series
155
+ * means every comparable measurement agrees, which is `unchanged` — a fact
156
+ * about the server. Having fewer than two comparable rows to walk means there
157
+ * is nothing to compare, which is `incomparable` — an absence. The two used
158
+ * to share one `null` return, and the report counted them together.
159
+ *
154
160
  * `rows` is that server's history, already narrowed to the run a trend may be
155
161
  * drawn across (`plottableSeries`), so the isolation rule is applied once, in
156
162
  * the place that owns it.
157
163
  */
158
- export function latestChange(server, rows, vectors) {
164
+ export function readSeries(server, rows, vectors) {
159
165
  const usable = rows.filter((r) => r.status === 'measured' || r.status === 'dynamic');
160
166
  if (usable.length < 2)
161
- return null;
167
+ return { kind: 'incomparable' };
162
168
  const newest = usable[usable.length - 1];
163
169
  // Walk back over measurements identical to the newest: the first of that run
164
170
  // is when the current cost arrived.
@@ -169,14 +175,26 @@ export function latestChange(server, rows, vectors) {
169
175
  break;
170
176
  toIdx--;
171
177
  }
178
+ /** The trailing run reaches back this far, and every row in it agrees. */
179
+ const held = () => ({
180
+ kind: 'unchanged',
181
+ held: {
182
+ server,
183
+ tokens: newest.tokens,
184
+ toolCount: newest.toolCount,
185
+ since: usable[toIdx].date,
186
+ measuredThrough: newest.date,
187
+ sweeps: usable.length - toIdx,
188
+ },
189
+ });
172
190
  if (toIdx === 0)
173
- return null; // the series has never changed
191
+ return held(); // every comparable measurement agrees
174
192
  const to = usable[toIdx];
175
193
  const from = usable[toIdx - 1];
176
194
  const deltaTokens = to.tokens - from.tokens;
177
195
  const deltaTools = to.toolCount - from.toolCount;
178
196
  if (deltaTokens === 0 && deltaTools === 0)
179
- return null;
197
+ return held();
180
198
  const deltaPct = from.tokens > 0 ? (deltaTokens / from.tokens) * 100 : 0;
181
199
  // Attribution needs both sides on record. Matched by *cost as of that date*,
182
200
  // not by date equality: vectors are deduped by capture and keep the first
@@ -209,23 +227,43 @@ export function latestChange(server, rows, vectors) {
209
227
  attribution = attribute(fromVec, toVec, deltaTokens);
210
228
  }
211
229
  return {
212
- server,
213
- fromDate: from.date,
214
- toDate: to.date,
215
- fromTokens: from.tokens,
216
- toTokens: to.tokens,
217
- deltaTokens,
218
- deltaPct,
219
- fromToolCount: from.toolCount,
220
- toToolCount: to.toolCount,
221
- deltaTools,
222
- mechanism: mechanismOf(deltaTokens, deltaTools),
223
- significant: isSignificant(deltaTokens, deltaPct),
224
- measuredThrough: newest.date,
225
- attribution,
230
+ kind: 'changed',
231
+ change: {
232
+ server,
233
+ fromDate: from.date,
234
+ toDate: to.date,
235
+ fromTokens: from.tokens,
236
+ toTokens: to.tokens,
237
+ deltaTokens,
238
+ deltaPct,
239
+ fromToolCount: from.toolCount,
240
+ toToolCount: to.toolCount,
241
+ deltaTools,
242
+ fromVersion: from.version ?? '',
243
+ toVersion: to.version ?? '',
244
+ mechanism: mechanismOf(deltaTokens, deltaTools),
245
+ significant: isSignificant(deltaTokens, deltaPct),
246
+ measuredThrough: newest.date,
247
+ attribution,
248
+ },
226
249
  };
227
250
  }
228
- export function summarize(changes, withoutComparison) {
251
+ /**
252
+ * The most recent comparable movement, or null when there was none. Kept as
253
+ * the narrow question `readSeries` answers in full, for the callers that only
254
+ * ever wanted a movement — but a caller that publishes a count must ask
255
+ * `readSeries`, because null here is two different facts.
256
+ */
257
+ export function latestChange(server, rows, vectors) {
258
+ const reading = readSeries(server, rows, vectors);
259
+ return reading.kind === 'changed' ? reading.change : null;
260
+ }
261
+ /**
262
+ * `unchanged` is required rather than defaulted: every reading a caller
263
+ * classifies belongs to exactly one of the three, and a default would let a
264
+ * caller that forgot them publish a total that quietly does not sum.
265
+ */
266
+ export function summarize(changes, withoutComparison, unchanged) {
229
267
  const sorted = [...changes].sort((a, b) => Math.abs(b.deltaPct) - Math.abs(a.deltaPct));
230
268
  return {
231
269
  changes: sorted,
@@ -233,6 +271,9 @@ export function summarize(changes, withoutComparison) {
233
271
  shrank: sorted.filter((c) => c.deltaTokens < 0).length,
234
272
  significant: sorted.filter((c) => c.significant).length,
235
273
  netTokens: sorted.reduce((a, c) => a + c.deltaTokens, 0),
274
+ // Longest-held first: the sentence this section exists to publish is "this
275
+ // number has been the same since <date>", so the oldest `since` leads.
276
+ unchanged: [...unchanged].sort((a, b) => a.since.localeCompare(b.since) || b.tokens - a.tokens),
236
277
  withoutComparison,
237
278
  };
238
279
  }
@@ -1,5 +1,15 @@
1
1
  /** Status taxonomy — every swept server gets exactly one; no silent drops. */
2
- export type MeasurementStatus = 'measured' | 'auth-required' | 'startup-failure' | 'timeout' | 'dynamic' | 'remote-auth-wall';
2
+ export type MeasurementStatus = 'measured' | 'auth-required' | 'startup-failure' | 'timeout'
3
+ /**
4
+ * This harness cannot run the server, for a reason that is a property of the
5
+ * harness rather than of the software: an OS or architecture the package does
6
+ * not ship for, or a backing service the isolation deliberately does not
7
+ * provide. Distinct from `startup-failure`, which asserts the server did not
8
+ * come up — a claim about someone else's code that these entries do not
9
+ * support. Only ever set when the entry declares the reason AND the failure's
10
+ * own text corroborates it (see `notApplicable` in report.ts).
11
+ */
12
+ | 'not-applicable' | 'dynamic' | 'remote-auth-wall';
3
13
  export interface ToolMeasurement {
4
14
  name: string;
5
15
  /** Tokens of the whole tool object, canonically serialized. */
@@ -43,6 +53,18 @@ export interface Measurement {
43
53
  image?: string;
44
54
  network?: string;
45
55
  note?: string;
56
+ /**
57
+ * The architecture the measurement ran on, as `<platform>/<arch>` (e.g.
58
+ * `linux/amd64`). Part of the isolation because some packages ship builds
59
+ * for only some of them: `local-mcp` was published as a startup failure on
60
+ * the strength of a run whose real finding was "this laptop is arm64 and
61
+ * the package has no arm64 runtime" — a fact about the machine that the
62
+ * record gave no way to see.
63
+ *
64
+ * Absent on records written before this was captured, which is why it is
65
+ * optional; absence means unknown, never "the same as yours".
66
+ */
67
+ arch?: string;
46
68
  };
47
69
  /** Request timeout in force during this measurement. */
48
70
  timeoutMs?: number;
@@ -13,6 +13,28 @@ export interface WireCapture {
13
13
  instructions: string | null;
14
14
  stderrTail: string;
15
15
  }
16
+ /**
17
+ * The part of a dead server's stderr worth keeping as evidence.
18
+ *
19
+ * A failure record is only useful if it contains the failure, and a plain tail
20
+ * reliably keeps the least useful part. `npx` prints a deprecation warning per
21
+ * transitive dependency and a version notice at the end, and a crashing process
22
+ * prints its message *before* the stack — so the last N characters of stderr
23
+ * are npm warnings and stack frames on exactly the servers whose failure needs
24
+ * explaining. Several published records ended up saying nothing about why the
25
+ * server did not start.
26
+ *
27
+ * This is not cosmetic. `run.ts` classifies a failure by reading these words:
28
+ * a record whose message was cut off is filed as `startup-failure` — the server
29
+ * is broken — when the surviving text would have said `auth-required`. Spending
30
+ * the budget on the message rather than the frames is what keeps the published
31
+ * taxonomy describing the server.
32
+ *
33
+ * Noise is only dropped while something else survives. A package that fails
34
+ * *inside* npm (EBADPLATFORM, a failed postinstall) has npm's own lines as its
35
+ * only evidence, and a server whose whole output is a stack keeps the stack.
36
+ */
37
+ export declare function evidenceTail(stderr: string, limit?: number): string;
16
38
  export declare class McpStdioClient {
17
39
  private child;
18
40
  private buffer;
@@ -6,6 +6,92 @@
6
6
  */
7
7
  import { spawn } from 'node:child_process';
8
8
  const PROTOCOL_VERSION = '2025-06-18';
9
+ /**
10
+ * The part of a dead server's stderr worth keeping as evidence.
11
+ *
12
+ * A failure record is only useful if it contains the failure, and a plain tail
13
+ * reliably keeps the least useful part. `npx` prints a deprecation warning per
14
+ * transitive dependency and a version notice at the end, and a crashing process
15
+ * prints its message *before* the stack — so the last N characters of stderr
16
+ * are npm warnings and stack frames on exactly the servers whose failure needs
17
+ * explaining. Several published records ended up saying nothing about why the
18
+ * server did not start.
19
+ *
20
+ * This is not cosmetic. `run.ts` classifies a failure by reading these words:
21
+ * a record whose message was cut off is filed as `startup-failure` — the server
22
+ * is broken — when the surviving text would have said `auth-required`. Spending
23
+ * the budget on the message rather than the frames is what keeps the published
24
+ * taxonomy describing the server.
25
+ *
26
+ * Noise is only dropped while something else survives. A package that fails
27
+ * *inside* npm (EBADPLATFORM, a failed postinstall) has npm's own lines as its
28
+ * only evidence, and a server whose whole output is a stack keeps the stack.
29
+ */
30
+ export function evidenceTail(stderr, limit = 600) {
31
+ const withoutNoise = drop(stderr, (l) => /^npm (warn|notice)\b/.test(l));
32
+ const withoutFrames = drop(withoutNoise, (l) => /^at\s/.test(l));
33
+ return bothEnds(withoutFrames, limit);
34
+ }
35
+ /**
36
+ * Keep the start and the end, eliding the middle.
37
+ *
38
+ * Dropping npm noise and stack frames is not enough on its own: a CLI that
39
+ * rejects its environment often prints one line saying why and then its entire
40
+ * usage screen, which is neither. kubernetes-mcp-server does exactly that, and
41
+ * a tail-only budget kept forty lines of flag documentation while discarding
42
+ * "no current-context is set and no contexts are defined in kubeconfig" — the
43
+ * only sentence in the output that explained anything.
44
+ *
45
+ * Failures put their explanation at one end or the other — a crash message
46
+ * above its aftermath, or an error at the end of a log — so both ends are kept
47
+ * and the middle is what goes. The split leans towards the head because a
48
+ * message that precedes its own noise is the more common shape here.
49
+ */
50
+ function bothEnds(text, limit) {
51
+ if (text.length <= limit)
52
+ return text;
53
+ const elision = '\n […] \n';
54
+ const budget = Math.max(0, limit - elision.length);
55
+ const lines = text.split('\n');
56
+ // Whole lines only: a boundary cut mid-word ("ool/prompt change") reads as
57
+ // corruption and loses the token an evidence string would match on.
58
+ const take = (from, to, cap, fromEnd) => {
59
+ const out = [];
60
+ let used = 0;
61
+ for (let i = fromEnd ? to : from; fromEnd ? i >= from : i <= to; i += fromEnd ? -1 : 1) {
62
+ const cost = lines[i].length + 1;
63
+ if (used + cost > cap)
64
+ break;
65
+ fromEnd ? out.unshift(lines[i]) : out.push(lines[i]);
66
+ used += cost;
67
+ }
68
+ return { out, used };
69
+ };
70
+ const headCap = Math.ceil(budget * 0.6);
71
+ const head = take(0, lines.length - 1, headCap, false);
72
+ // Whole lines, except when the first line alone overruns the budget. A server
73
+ // that logs structured JSON puts its entire message on one line, so that line
74
+ // is both the most informative thing in the output and the only one that can
75
+ // never fit — slack-mcp-server's `{"level":"fatal","message":"Authentication
76
+ // required: ..."}` was dropped in full, and the record it left behind said a
77
+ // child process exited. Truncated evidence beats none.
78
+ const headText = head.out.length > 0 ? head.out.join('\n') : lines[0].slice(0, headCap);
79
+ const headUsed = head.out.length > 0 ? head.used : headText.length;
80
+ const tailFrom = head.out.length > 0 ? head.out.length : 1;
81
+ const tail = take(tailFrom, lines.length - 1, budget - headUsed, true);
82
+ if (headText === '' && tail.out.length === 0)
83
+ return text.slice(0, budget) + elision;
84
+ return `${headText}${elision}${tail.out.join('\n')}`;
85
+ }
86
+ /** Drop matching lines, keeping the input whole if that would leave nothing. */
87
+ function drop(text, isNoise) {
88
+ const kept = text
89
+ .split('\n')
90
+ .filter((l) => !isNoise(l.trim()))
91
+ .join('\n')
92
+ .trim();
93
+ return kept || text.trim();
94
+ }
9
95
  export class McpStdioClient {
10
96
  child;
11
97
  buffer = '';
@@ -37,7 +123,7 @@ export class McpStdioClient {
37
123
  resolve();
38
124
  });
39
125
  this.child.on('exit', (code) => {
40
- const tail = this.stderrTail.slice(-600);
126
+ const tail = evidenceTail(this.stderrTail);
41
127
  this.deadReason = `server exited (code ${code})${tail ? `; stderr tail: ${tail}` : ''}`;
42
128
  for (const p of this.pending.values())
43
129
  p.reject(new Error(this.deadReason));
@@ -95,7 +181,12 @@ export class McpStdioClient {
95
181
  return new Promise((resolve, reject) => {
96
182
  const timer = setTimeout(() => {
97
183
  this.pending.delete(id);
98
- reject(new Error(`timeout after ${timeoutMs}ms waiting for ${method}`));
184
+ // A process that is killed mid-hang never reaches the exit handler, so
185
+ // without this a timed-out record carries no evidence at all — it says
186
+ // only that we waited. What the server managed to print before it
187
+ // stopped answering is usually the whole explanation.
188
+ const tail = evidenceTail(this.stderrTail);
189
+ reject(new Error(`timeout after ${timeoutMs}ms waiting for ${method}${tail ? `; stderr tail: ${tail}` : ''}`));
99
190
  }, timeoutMs);
100
191
  this.pending.set(id, {
101
192
  resolve: (v) => {
@@ -162,6 +162,7 @@ export function runCli(binPath, entry, opts) {
162
162
  dummyEnv: entry.env ?? [],
163
163
  dummyEnvValues: entry.envValues,
164
164
  needsGit: entry.needsGit,
165
+ aptPackages: entry.aptPackages,
165
166
  containerName,
166
167
  binds: [`${binPath}:/opt/mcp-tokens:ro`],
167
168
  });
@@ -287,6 +288,8 @@ if (isMain) {
287
288
  dummyEnv: e.env ?? [],
288
289
  dummyEnvValues: e.envValues,
289
290
  needsGit: e.needsGit,
291
+ aptPackages: e.aptPackages,
292
+ notApplicable: e.notApplicable,
290
293
  persist: false, // the measurements on disk are not this run's to rewrite
291
294
  });
292
295
  }
@@ -66,7 +66,22 @@ export function generateDashboard(root = process.cwd()) {
66
66
  const median = totals.length ? totals.slice().sort((a, b) => a - b)[Math.floor(totals.length / 2)] : 0;
67
67
  const max = totals.length ? Math.max(...totals) : 1;
68
68
  const fmt = (n) => n.toLocaleString('en-US');
69
- const now = new Date().toISOString().slice(0, 16).replace('T', ' ') + ' UTC';
69
+ /**
70
+ * The stamp is the newest measurement on the page, not the moment the page
71
+ * was written.
72
+ *
73
+ * A wall-clock stamp made `docs/dashboard.html` differ on every regeneration,
74
+ * so the scheduled jobs' "no change to publish" guard could never fire: one
75
+ * file that always diffs is enough to publish a commit that says nothing
76
+ * happened. Dated to the data instead, the page changes when the data does —
77
+ * and the line now answers the question a reader actually has, which is how
78
+ * fresh these numbers are, not when the HTML was rendered.
79
+ */
80
+ const dates = rows
81
+ .map((r) => String(r.m?.measuredAt ?? '').slice(0, 10))
82
+ .filter((d) => /^\d{4}-\d{2}-\d{2}$/.test(d))
83
+ .sort();
84
+ const newestMeasurement = dates[dates.length - 1] ?? null;
70
85
  const barRows = measured
71
86
  .map((r, i) => {
72
87
  const m = r.m;
@@ -225,7 +240,7 @@ export function generateDashboard(root = process.cwd()) {
225
240
  </style>
226
241
  <div class="wrap">
227
242
  <header>
228
- <p class="eyebrow">methodology v1.0 · o200k_base · generated ${now}</p>
243
+ <p class="eyebrow">methodology v1.0 · o200k_base · newest measurement ${newestMeasurement ?? 'not yet taken'}</p>
229
244
  <h1>mcp-context-cost</h1>
230
245
  <p class="sub">What popular MCP servers cost in context tokens before the agent does any work — measured from raw <code>tools/list</code> captures, every number re-derivable from its published measurement file.</p>
231
246
  </header>