mcp-context-cost 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +42 -8
- package/dist/cli.js +39 -6
- package/dist/core/bands.d.ts +5 -0
- package/dist/core/bands.js +8 -0
- package/dist/core/canonical.js +3 -1
- package/dist/core/divergence.d.ts +71 -0
- package/dist/core/divergence.js +103 -0
- package/dist/core/index.d.ts +1 -0
- package/dist/core/index.js +1 -0
- package/dist/sweep/dashboard.js +17 -14
- package/dist/sweep/history.d.ts +30 -0
- package/dist/sweep/history.js +120 -0
- package/dist/sweep/regen.js +8 -1
- package/dist/sweep/report.d.ts +5 -0
- package/dist/sweep/report.js +39 -6
- package/dist/sweep/run.js +4 -0
- package/dist/sweep/server-pages.d.ts +17 -0
- package/dist/sweep/server-pages.js +228 -0
- package/dist/sweep/sweep-all.js +4 -1
- package/package.json +5 -2
package/README.md
CHANGED
|
@@ -1,5 +1,8 @@
|
|
|
1
1
|
# mcp-context-cost
|
|
2
2
|
|
|
3
|
+
[](https://www.npmjs.com/package/mcp-context-cost)
|
|
4
|
+
[](https://github.com/athakur3/mcp-context-cost/actions/workflows/ci.yml)
|
|
5
|
+
|
|
3
6
|
**Reproducible context-cost badges for MCP servers.**
|
|
4
7
|
|
|
5
8
|
Every MCP server you wire into an agent injects its tool schemas into the model's context
|
|
@@ -16,7 +19,9 @@ popular servers:
|
|
|
16
19
|
| markitdown | 64 | 1 |
|
|
17
20
|
|
|
18
21
|
*(57 of 82 popular servers measured, 2026-08-16 sweep — full table in
|
|
19
|
-
[results/leaderboard.md](results/leaderboard.md); every failure is listed with its reason.
|
|
22
|
+
[results/leaderboard.md](results/leaderboard.md); every failure is listed with its reason.
|
|
23
|
+
Each measured server also has a [detail page](https://athakur3.github.io/mcp-context-cost/servers/)
|
|
24
|
+
showing which tools its tokens are in.)*
|
|
20
25
|
|
|
21
26
|
This project makes that cost **legible and disputable**:
|
|
22
27
|
|
|
@@ -24,6 +29,21 @@ This project makes that cost **legible and disputable**:
|
|
|
24
29
|
[context cost | 12,430 tokens] ← shields.io badge, linked to the methodology
|
|
25
30
|
```
|
|
26
31
|
|
|
32
|
+
## What it costs on Claude
|
|
33
|
+
|
|
34
|
+
The badge counts every byte a server returns. An Anthropic request carries only `name`,
|
|
35
|
+
`description`, and `input_schema` — and counts them with a denser tokenizer. Both effects are
|
|
36
|
+
now measured against a pinned model and published beside the badge, and they do not cancel:
|
|
37
|
+
|
|
38
|
+
| server | badge (o200k) | Claude (`claude-opus-5`) | |
|
|
39
|
+
|---|---:|---:|---|
|
|
40
|
+
| github | 54,422 | **18,406** | 81% of the capture is `annotations`/`outputSchema` metadata Claude never sees |
|
|
41
|
+
| notion | 17,500 | **33,560** | almost no metadata to drop, so the tokenizer difference dominates |
|
|
42
|
+
|
|
43
|
+
So the heaviest server on the badge is not the heaviest server on Claude. Per-server
|
|
44
|
+
breakdowns are on each [detail page](https://athakur3.github.io/mcp-context-cost/servers/);
|
|
45
|
+
the method is [Claude divergence](docs/METHODOLOGY.md#claude-divergence).
|
|
46
|
+
|
|
27
47
|
## Why trust the number?
|
|
28
48
|
|
|
29
49
|
Every badge is backed by a `measurement.json` containing the raw `tools/list` capture, the
|
|
@@ -31,10 +51,17 @@ SHA-256 of its canonical bytes, the pinned tokenizer (`o200k_base`), and the exa
|
|
|
31
51
|
command. Disputes reduce to a byte-level diff:
|
|
32
52
|
|
|
33
53
|
```bash
|
|
34
|
-
npx
|
|
54
|
+
npx -y mcp-context-cost verify results/github/measurement.json
|
|
35
55
|
# OK github-mcp-server: 54422 tokens (o200k_base, methodology 1.0) — capture, hash, and count all agree
|
|
56
|
+
|
|
57
|
+
# or point it at a published measurement.json directly, no clone required
|
|
58
|
+
npx -y mcp-context-cost verify --remote https://raw.githubusercontent.com/athakur3/mcp-context-cost/main/results/github/measurement.json
|
|
36
59
|
```
|
|
37
60
|
|
|
61
|
+
Add `--json` for scripting (`{ ok, serverName, rederivedTokens, rederivedSha, problems, badge }`
|
|
62
|
+
on stdout, `badge` omitted on failure). Exit codes: `0` ok, `1` verification/measurement
|
|
63
|
+
failed, `2` usage error.
|
|
64
|
+
|
|
38
65
|
Full definition: [docs/METHODOLOGY.md](docs/METHODOLOGY.md) — what is counted, what the
|
|
39
66
|
number is *not*, config policy, failure taxonomy, frozen color bands, known divergences.
|
|
40
67
|
|
|
@@ -46,9 +73,11 @@ number is *not*, config policy, failure taxonomy, frozen color bands, known dive
|
|
|
46
73
|
| `src/sweep/` | raw-wire MCP stdio client + Dockerized batch sweep + leaderboard/dashboard generators |
|
|
47
74
|
| `src/cli.ts` | `verify` (re-derive any published number) and `measure` |
|
|
48
75
|
| `spec/fixtures/` | golden vectors shared by the TypeScript and bash implementations |
|
|
76
|
+
| `tools/` | the one script that calls a network API (Claude divergence); kept out of the package so the library stays offline |
|
|
49
77
|
| `upstream/` | staged contribution to [sd2k/mcp-tokens-action](https://github.com/sd2k/mcp-tokens-action): `badge.sh` + action patch + tests |
|
|
50
78
|
| `servers.yaml` | 82 curated candidates with live install metrics and provenance |
|
|
51
|
-
| `results/` · `badges/` | measurements, leaderboard, shields endpoint JSONs |
|
|
79
|
+
| `results/` · `badges/` | measurements, leaderboard, history series, shields endpoint JSONs |
|
|
80
|
+
| `docs/` | methodology, dashboard, and a generated page per measured server |
|
|
52
81
|
|
|
53
82
|
## Measure your own server
|
|
54
83
|
|
|
@@ -61,16 +90,20 @@ cat badges/my-server.json # strict shields.io endpoint JSON
|
|
|
61
90
|
Then in your README:
|
|
62
91
|
|
|
63
92
|
```markdown
|
|
64
|
-
[](<
|
|
93
|
+
[](<link target>)
|
|
65
94
|
```
|
|
66
95
|
|
|
96
|
+
Point the link at the measurement behind the number — for servers in this sweep that is
|
|
97
|
+
`https://athakur3.github.io/mcp-context-cost/servers/<name>.html`; otherwise the
|
|
98
|
+
methodology page. A badge nobody can audit is decoration.
|
|
99
|
+
|
|
67
100
|
Or self-serve from CI via the (staged) mcp-tokens-action badge inputs — see
|
|
68
101
|
[upstream/action-patch.md](upstream/action-patch.md).
|
|
69
102
|
|
|
70
103
|
## Development
|
|
71
104
|
|
|
72
105
|
```bash
|
|
73
|
-
npm test #
|
|
106
|
+
npm test # 53 TS tests incl. golden fixtures + dispute drills
|
|
74
107
|
npx tsc --noEmit # typecheck
|
|
75
108
|
./upstream/tests/badge-test.sh # 21 bash tests — byte-identical to the TS reference
|
|
76
109
|
npm run sweep:all -- --docker # full curated sweep (Docker isolation)
|
|
@@ -83,8 +116,9 @@ color bands are frozen against the observed distribution of the first full sweep
|
|
|
83
116
|
|
|
84
117
|
## Status
|
|
85
118
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
119
|
+
Active. The leaderboard refreshes weekly; badge PRs are open across the ecosystem and
|
|
120
|
+
[sd2k/mcp-tokens-action#5](https://github.com/sd2k/mcp-tokens-action/pull/5) proposes the
|
|
121
|
+
self-serve badge path upstream. See [ROADMAP.md](ROADMAP.md) for what's next —
|
|
122
|
+
contributions welcome, especially new `servers.yaml` entries.
|
|
89
123
|
|
|
90
124
|
MIT © 2026
|
package/dist/cli.js
CHANGED
|
@@ -2,9 +2,12 @@
|
|
|
2
2
|
/**
|
|
3
3
|
* mcp-context-cost CLI — the dispute drill as a command.
|
|
4
4
|
*
|
|
5
|
-
* mcp-context-cost verify <measurement.json> re-derive the number from the
|
|
5
|
+
* mcp-context-cost verify <measurement.json> [--json] re-derive the number from the
|
|
6
6
|
* published capture; exit 1 on mismatch
|
|
7
|
+
* mcp-context-cost verify --remote <url> [--json] same, fetched from a measurement URL
|
|
7
8
|
* mcp-context-cost measure --name x --command "npx -y ..." one-off measurement
|
|
9
|
+
*
|
|
10
|
+
* Exit codes: 0 ok, 1 verification/measurement failed, 2 usage error.
|
|
8
11
|
*/
|
|
9
12
|
import { readFileSync } from 'node:fs';
|
|
10
13
|
import { canonicalString, countTokens, sha256Hex } from './core/canonical.js';
|
|
@@ -27,13 +30,41 @@ export function verifyMeasurement(m) {
|
|
|
27
30
|
}
|
|
28
31
|
const [, , cmd, ...rest] = process.argv;
|
|
29
32
|
if (cmd === 'verify') {
|
|
30
|
-
const
|
|
31
|
-
|
|
32
|
-
|
|
33
|
+
const json = rest.includes('--json');
|
|
34
|
+
const remoteIdx = rest.indexOf('--remote');
|
|
35
|
+
const remoteUrl = remoteIdx >= 0 ? rest[remoteIdx + 1] : undefined;
|
|
36
|
+
const path = rest.find((a) => !a.startsWith('--') && a !== remoteUrl);
|
|
37
|
+
if (!remoteUrl && !path) {
|
|
38
|
+
console.error('usage: mcp-context-cost verify <measurement.json> [--json]');
|
|
39
|
+
console.error(' mcp-context-cost verify --remote <url> [--json]');
|
|
33
40
|
process.exit(2);
|
|
34
41
|
}
|
|
35
|
-
|
|
42
|
+
let raw;
|
|
43
|
+
if (remoteUrl) {
|
|
44
|
+
try {
|
|
45
|
+
const res = await fetch(remoteUrl, { signal: AbortSignal.timeout(15_000) });
|
|
46
|
+
if (!res.ok)
|
|
47
|
+
throw new Error(`HTTP ${res.status}`);
|
|
48
|
+
raw = await res.text();
|
|
49
|
+
}
|
|
50
|
+
catch (e) {
|
|
51
|
+
const problem = `failed to fetch ${remoteUrl}: ${e.message}`;
|
|
52
|
+
if (json)
|
|
53
|
+
console.log(JSON.stringify({ ok: false, rederivedTokens: null, rederivedSha: null, problems: [problem] }));
|
|
54
|
+
else
|
|
55
|
+
console.error(problem);
|
|
56
|
+
process.exit(1);
|
|
57
|
+
}
|
|
58
|
+
}
|
|
59
|
+
else {
|
|
60
|
+
raw = readFileSync(path, 'utf8');
|
|
61
|
+
}
|
|
62
|
+
const m = JSON.parse(raw);
|
|
36
63
|
const r = verifyMeasurement(m);
|
|
64
|
+
if (json) {
|
|
65
|
+
console.log(JSON.stringify({ serverName: m.serverName, ...r, badge: r.ok ? toBadge(m) : undefined }));
|
|
66
|
+
process.exit(r.ok ? 0 : 1);
|
|
67
|
+
}
|
|
37
68
|
if (r.ok) {
|
|
38
69
|
console.log(`OK ${m.serverName}: ${r.rederivedTokens} tokens (${m.encoding}, methodology ${m.methodologyVersion}) — capture, hash, and count all agree`);
|
|
39
70
|
console.log(`badge: ${JSON.stringify(toBadge(m))}`);
|
|
@@ -73,6 +104,8 @@ else if (cmd !== undefined && cmd !== '--help' && cmd !== '-h') {
|
|
|
73
104
|
}
|
|
74
105
|
else {
|
|
75
106
|
console.log('mcp-context-cost — reproducible context-cost measurement for MCP servers');
|
|
76
|
-
console.log(' verify <measurement.json> re-derive tokens+sha from the published capture');
|
|
107
|
+
console.log(' verify <measurement.json> [--json] re-derive tokens+sha from the published capture');
|
|
108
|
+
console.log(' verify --remote <url> [--json] same, fetched from a measurement URL');
|
|
77
109
|
console.log(' measure --name x --command "npx -y <server>" run a one-off measurement');
|
|
110
|
+
console.log('exit codes: 0 ok, 1 verification/measurement failed, 2 usage error');
|
|
78
111
|
}
|
package/dist/core/bands.d.ts
CHANGED
|
@@ -5,3 +5,8 @@
|
|
|
5
5
|
*/
|
|
6
6
|
export declare function bandColor(totalTokens: number): string;
|
|
7
7
|
export declare const UNKNOWN_COLOR = "lightgrey";
|
|
8
|
+
/** Human-readable name + range for each band, shared by every generated view. */
|
|
9
|
+
export declare const BAND_META: Record<string, {
|
|
10
|
+
label: string;
|
|
11
|
+
range: string;
|
|
12
|
+
}>;
|
package/dist/core/bands.js
CHANGED
|
@@ -15,3 +15,11 @@ export function bandColor(totalTokens) {
|
|
|
15
15
|
return 'red';
|
|
16
16
|
}
|
|
17
17
|
export const UNKNOWN_COLOR = 'lightgrey';
|
|
18
|
+
/** Human-readable name + range for each band, shared by every generated view. */
|
|
19
|
+
export const BAND_META = {
|
|
20
|
+
brightgreen: { label: 'lean', range: '< 1K' },
|
|
21
|
+
green: { label: 'light', range: '1–5K' },
|
|
22
|
+
yellow: { label: 'moderate', range: '5–15K' },
|
|
23
|
+
orange: { label: 'heavy', range: '15–30K' },
|
|
24
|
+
red: { label: 'very heavy', range: '≥ 30K' },
|
|
25
|
+
};
|
package/dist/core/canonical.js
CHANGED
|
@@ -44,7 +44,9 @@ export function measureTools(tools, meta) {
|
|
|
44
44
|
toolCount: tools.length,
|
|
45
45
|
tools: perTool,
|
|
46
46
|
canonicalSha256: sha256Hex(canonical),
|
|
47
|
-
|
|
47
|
+
// Snapshot from the canonical string, not the live `tools` reference — the whole
|
|
48
|
+
// point of a capture is that it can't change out from under its own hash later.
|
|
49
|
+
rawToolsCapture: JSON.parse(canonical),
|
|
48
50
|
measuredAt: meta.measuredAt ?? new Date().toISOString(),
|
|
49
51
|
serverName: meta.serverName,
|
|
50
52
|
serverVersion: meta.serverVersion,
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
/** Method identifier, versioned independently of the o200k methodology. */
|
|
2
|
+
export declare const DIVERGENCE_METHOD = "tools-delta/v1";
|
|
3
|
+
/** The three fields an Anthropic tool definition carries — nothing else. */
|
|
4
|
+
export declare const ANTHROPIC_TOOL_FIELDS: readonly ['name', 'description', 'input_schema'];
|
|
5
|
+
export interface AnthropicTool {
|
|
6
|
+
name: string;
|
|
7
|
+
description: string;
|
|
8
|
+
input_schema: unknown;
|
|
9
|
+
}
|
|
10
|
+
export interface DivergenceRow {
|
|
11
|
+
/** o200k count of the full canonical capture — the published headline. */
|
|
12
|
+
o200kFull: number;
|
|
13
|
+
/** o200k count of the name/description/input_schema projection. */
|
|
14
|
+
o200kMapped: number;
|
|
15
|
+
/**
|
|
16
|
+
* Claude tokens the projection adds to a request: count_tokens(with tools)
|
|
17
|
+
* minus count_tokens(same request, no tools). Includes the one-time tool
|
|
18
|
+
* framework overhead, because attaching any server pays it once.
|
|
19
|
+
*/
|
|
20
|
+
claudeDelta: number;
|
|
21
|
+
toolCount: number;
|
|
22
|
+
/**
|
|
23
|
+
* canonicalSha256 of the measurement this row was computed from. A re-sweep
|
|
24
|
+
* changes the hash, which marks the row stale instead of silently mismatched.
|
|
25
|
+
*/
|
|
26
|
+
capturedSha256: string;
|
|
27
|
+
/** Set when count_tokens rejected the projection; no numbers are published. */
|
|
28
|
+
error?: string;
|
|
29
|
+
}
|
|
30
|
+
export interface DivergenceRun {
|
|
31
|
+
method: string;
|
|
32
|
+
/** Exact model id the counts are pinned to — Anthropic's tokenizer drifts. */
|
|
33
|
+
model: string;
|
|
34
|
+
/** UTC day the counts were taken (YYYY-MM-DD). */
|
|
35
|
+
measuredAt: string;
|
|
36
|
+
/** count_tokens for the probe request with no tools at all. */
|
|
37
|
+
baselineTokens: number;
|
|
38
|
+
/**
|
|
39
|
+
* A single minimal tool. Its delta is an upper bound on the fixed framework
|
|
40
|
+
* overhead included in every `claudeDelta` (the probe tool itself costs > 0,
|
|
41
|
+
* so the true overhead is strictly less).
|
|
42
|
+
*/
|
|
43
|
+
probeDelta: number;
|
|
44
|
+
servers: Record<string, DivergenceRow>;
|
|
45
|
+
}
|
|
46
|
+
/**
|
|
47
|
+
* Project a `tools/list` capture onto the Anthropic tool shape. Tools without a
|
|
48
|
+
* usable name are dropped rather than renamed — a fabricated name would change
|
|
49
|
+
* the token count being compared.
|
|
50
|
+
*/
|
|
51
|
+
export declare function toAnthropicTools(raw: unknown[]): AnthropicTool[];
|
|
52
|
+
/** o200k count of the projection — the same tokenizer as the headline number. */
|
|
53
|
+
export declare function mappedTokens(raw: unknown[]): number;
|
|
54
|
+
/**
|
|
55
|
+
* Share of the headline number that is MCP-only metadata (cause 1 above).
|
|
56
|
+
* Returns null when there is nothing to divide by.
|
|
57
|
+
*/
|
|
58
|
+
export declare function fieldSelectionShare(row: DivergenceRow): number | null;
|
|
59
|
+
/**
|
|
60
|
+
* Claude tokens per o200k token of the headline — the ratio a reader wants when
|
|
61
|
+
* asking "does the badge over- or under-state what this costs me?".
|
|
62
|
+
*/
|
|
63
|
+
export declare function claudeRatio(row: DivergenceRow): number | null;
|
|
64
|
+
/**
|
|
65
|
+
* A row is only reportable if it carries numbers and was computed from the
|
|
66
|
+
* capture currently on disk. Stale rows are hidden rather than shown with a
|
|
67
|
+
* caveat: a wrong number next to a fresh badge is worse than no number.
|
|
68
|
+
*/
|
|
69
|
+
export declare function isCurrent(row: DivergenceRow | undefined, canonicalSha256: string | null): row is DivergenceRow;
|
|
70
|
+
/** Parse results/divergence.json; anything malformed yields null, never throws. */
|
|
71
|
+
export declare function parseDivergence(text: string): DivergenceRun | null;
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Claude divergence — the gap between the published o200k index and what a
|
|
3
|
+
* server's tools actually cost in an Anthropic request.
|
|
4
|
+
*
|
|
5
|
+
* The gap has two independent causes, and reporting them as one number hides
|
|
6
|
+
* the larger of the two:
|
|
7
|
+
*
|
|
8
|
+
* 1. **Field selection.** An Anthropic tool definition carries exactly `name`,
|
|
9
|
+
* `description`, and `input_schema`. A `tools/list` result may additionally
|
|
10
|
+
* carry `title`, `annotations`, `outputSchema`, `execution`, `icons` — real
|
|
11
|
+
* bytes the server ships, counted by the canonical form, but never present
|
|
12
|
+
* in the `tools` array a client sends to the API. For some servers this is
|
|
13
|
+
* the majority of the payload.
|
|
14
|
+
* 2. **Tokenizer and framing.** o200k_base vs Anthropic's tokenizer, plus the
|
|
15
|
+
* fixed framework overhead the API adds once when any tool is present.
|
|
16
|
+
*
|
|
17
|
+
* So the divergence is published as a decomposition: the headline o200k count,
|
|
18
|
+
* the o200k count of the projection (isolating cause 1), and Claude's own count
|
|
19
|
+
* of that same projection (isolating cause 2).
|
|
20
|
+
*/
|
|
21
|
+
import { countTokens } from './canonical.js';
|
|
22
|
+
/** Method identifier, versioned independently of the o200k methodology. */
|
|
23
|
+
export const DIVERGENCE_METHOD = 'tools-delta/v1';
|
|
24
|
+
/** The three fields an Anthropic tool definition carries — nothing else. */
|
|
25
|
+
export const ANTHROPIC_TOOL_FIELDS = ['name', 'description', 'input_schema'];
|
|
26
|
+
/**
|
|
27
|
+
* Project a `tools/list` capture onto the Anthropic tool shape. Tools without a
|
|
28
|
+
* usable name are dropped rather than renamed — a fabricated name would change
|
|
29
|
+
* the token count being compared.
|
|
30
|
+
*/
|
|
31
|
+
export function toAnthropicTools(raw) {
|
|
32
|
+
const out = [];
|
|
33
|
+
for (const t of raw) {
|
|
34
|
+
const tool = (t ?? {});
|
|
35
|
+
if (typeof tool.name !== 'string' || tool.name === '')
|
|
36
|
+
continue;
|
|
37
|
+
out.push({
|
|
38
|
+
name: tool.name,
|
|
39
|
+
description: typeof tool.description === 'string' ? tool.description : '',
|
|
40
|
+
// An absent schema is sent as the empty object schema, which is what the
|
|
41
|
+
// API requires; every tool in the current sweep supplies one.
|
|
42
|
+
input_schema: tool.inputSchema ?? { type: 'object', properties: {} },
|
|
43
|
+
});
|
|
44
|
+
}
|
|
45
|
+
return out;
|
|
46
|
+
}
|
|
47
|
+
/** o200k count of the projection — the same tokenizer as the headline number. */
|
|
48
|
+
export function mappedTokens(raw) {
|
|
49
|
+
return countTokens(JSON.stringify(toAnthropicTools(raw)));
|
|
50
|
+
}
|
|
51
|
+
/**
|
|
52
|
+
* Share of the headline number that is MCP-only metadata (cause 1 above).
|
|
53
|
+
* Returns null when there is nothing to divide by.
|
|
54
|
+
*/
|
|
55
|
+
export function fieldSelectionShare(row) {
|
|
56
|
+
if (row.o200kFull <= 0)
|
|
57
|
+
return null;
|
|
58
|
+
return (row.o200kFull - row.o200kMapped) / row.o200kFull;
|
|
59
|
+
}
|
|
60
|
+
/**
|
|
61
|
+
* Claude tokens per o200k token of the headline — the ratio a reader wants when
|
|
62
|
+
* asking "does the badge over- or under-state what this costs me?".
|
|
63
|
+
*/
|
|
64
|
+
export function claudeRatio(row) {
|
|
65
|
+
if (row.o200kFull <= 0 || typeof row.claudeDelta !== 'number')
|
|
66
|
+
return null;
|
|
67
|
+
return row.claudeDelta / row.o200kFull;
|
|
68
|
+
}
|
|
69
|
+
/**
|
|
70
|
+
* A row is only reportable if it carries numbers and was computed from the
|
|
71
|
+
* capture currently on disk. Stale rows are hidden rather than shown with a
|
|
72
|
+
* caveat: a wrong number next to a fresh badge is worse than no number.
|
|
73
|
+
*/
|
|
74
|
+
export function isCurrent(row, canonicalSha256) {
|
|
75
|
+
if (!row || row.error)
|
|
76
|
+
return false;
|
|
77
|
+
if (typeof row.claudeDelta !== 'number')
|
|
78
|
+
return false;
|
|
79
|
+
return !!canonicalSha256 && row.capturedSha256 === canonicalSha256;
|
|
80
|
+
}
|
|
81
|
+
/** Parse results/divergence.json; anything malformed yields null, never throws. */
|
|
82
|
+
export function parseDivergence(text) {
|
|
83
|
+
let run;
|
|
84
|
+
try {
|
|
85
|
+
run = JSON.parse(text);
|
|
86
|
+
}
|
|
87
|
+
catch {
|
|
88
|
+
return null;
|
|
89
|
+
}
|
|
90
|
+
const r = run;
|
|
91
|
+
if (!r || typeof r.model !== 'string' || typeof r.measuredAt !== 'string')
|
|
92
|
+
return null;
|
|
93
|
+
if (!r.servers || typeof r.servers !== 'object')
|
|
94
|
+
return null;
|
|
95
|
+
return {
|
|
96
|
+
method: typeof r.method === 'string' ? r.method : DIVERGENCE_METHOD,
|
|
97
|
+
model: r.model,
|
|
98
|
+
measuredAt: r.measuredAt,
|
|
99
|
+
baselineTokens: typeof r.baselineTokens === 'number' ? r.baselineTokens : 0,
|
|
100
|
+
probeDelta: typeof r.probeDelta === 'number' ? r.probeDelta : 0,
|
|
101
|
+
servers: r.servers,
|
|
102
|
+
};
|
|
103
|
+
}
|
package/dist/core/index.d.ts
CHANGED
package/dist/core/index.js
CHANGED
package/dist/sweep/dashboard.js
CHANGED
|
@@ -6,17 +6,15 @@
|
|
|
6
6
|
import { existsSync, readFileSync, writeFileSync, mkdirSync } from 'node:fs';
|
|
7
7
|
import { dirname, join } from 'node:path';
|
|
8
8
|
import { parse } from 'yaml';
|
|
9
|
-
import { bandColor } from '../core/bands.js';
|
|
9
|
+
import { bandColor, BAND_META } from '../core/bands.js';
|
|
10
10
|
const esc = (s) => String(s ?? '').replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>').replace(/"/g, '"');
|
|
11
|
-
const BAND_META = {
|
|
12
|
-
brightgreen: { label: 'lean', range: '< 1K' },
|
|
13
|
-
green: { label: 'light', range: '1–5K' },
|
|
14
|
-
yellow: { label: 'moderate', range: '5–15K' },
|
|
15
|
-
orange: { label: 'heavy', range: '15–30K' },
|
|
16
|
-
red: { label: 'very heavy', range: '≥ 30K' },
|
|
17
|
-
};
|
|
18
11
|
export function generateDashboard(root = process.cwd()) {
|
|
19
12
|
const doc = parse(readFileSync(join(root, 'servers.yaml'), 'utf8'));
|
|
13
|
+
const divergencePath = join(root, 'results', 'divergence.json');
|
|
14
|
+
const divergence = existsSync(divergencePath)
|
|
15
|
+
? JSON.parse(readFileSync(divergencePath, 'utf8'))
|
|
16
|
+
: {};
|
|
17
|
+
const dSrv = divergence.servers ?? {};
|
|
20
18
|
const rows = doc.servers.map((entry) => {
|
|
21
19
|
const p = join(root, 'results', entry.name, 'measurement.json');
|
|
22
20
|
return { entry, m: existsSync(p) ? JSON.parse(readFileSync(p, 'utf8')) : null };
|
|
@@ -39,12 +37,15 @@ export function generateDashboard(root = process.cwd()) {
|
|
|
39
37
|
const meta = BAND_META[band];
|
|
40
38
|
const largest = [...m.tools].sort((a, b) => b.tokens - a.tokens)[0];
|
|
41
39
|
const pct = Math.max(1.2, (t / max) * 100);
|
|
42
|
-
|
|
40
|
+
const div = dSrv[r.entry.name];
|
|
41
|
+
const claudeTip = div ? ` · in a Claude request: ${fmt(div.claudeDelta)} tok` : '';
|
|
42
|
+
// The whole row is the link to the server's detail page (docs/servers/).
|
|
43
|
+
return `<a class="row" href="servers/${encodeURIComponent(r.entry.name)}.html" data-tip="${esc(m.toolCount)} tools · largest: ${esc(largest?.name)} (${fmt(largest?.tokens ?? 0)} tok)${claudeTip} · ${esc(r.entry.category)} · ${esc(m.status)}${m.serverVersion ? ' · v' + esc(String(m.serverVersion).replace(/^v/, '')) : ''}">
|
|
43
44
|
<span class="rank">${i + 1}</span>
|
|
44
45
|
<span class="name">${esc(r.entry.name)}</span>
|
|
45
46
|
<span class="track"><span class="bar" style="width:${pct.toFixed(1)}%"></span></span>
|
|
46
47
|
<span class="val"><span class="dot dot-${band}" aria-hidden="true"></span>${fmt(t)}<span class="bandname">${meta.label}</span></span>
|
|
47
|
-
</
|
|
48
|
+
</a>`;
|
|
48
49
|
})
|
|
49
50
|
.join('\n');
|
|
50
51
|
const failRows = failed
|
|
@@ -57,7 +58,8 @@ export function generateDashboard(root = process.cwd()) {
|
|
|
57
58
|
const tableRows = measured
|
|
58
59
|
.map((r, i) => {
|
|
59
60
|
const m = r.m;
|
|
60
|
-
|
|
61
|
+
const div = dSrv[r.entry.name];
|
|
62
|
+
return `<tr><td>${i + 1}</td><td>${esc(r.entry.name)}</td><td class="num">${fmt(m.totalTokens)}</td><td class="num">${div ? fmt(div.claudeDelta) : '—'}</td><td class="num">${esc(m.toolCount)}</td><td>${esc(BAND_META[bandColor(m.totalTokens)].label)}</td><td>${esc(r.entry.category)}</td></tr>`;
|
|
61
63
|
})
|
|
62
64
|
.join('\n');
|
|
63
65
|
const specimens = [measured[measured.length - 1], measured[0]]
|
|
@@ -115,8 +117,9 @@ export function generateDashboard(root = process.cwd()) {
|
|
|
115
117
|
.stat .l { font-size: 11px; letter-spacing: 0.08em; text-transform: uppercase; color: var(--muted); }
|
|
116
118
|
|
|
117
119
|
.board { background: var(--surface); border: 1px solid var(--line); border-radius: 8px; padding: 14px 16px; }
|
|
118
|
-
.row { display: grid; grid-template-columns: 2ch minmax(120px, 190px) 1fr max-content; gap: 10px; align-items: center; padding: 3px 4px; border-radius: 4px; outline: none; }
|
|
120
|
+
.row { display: grid; grid-template-columns: 2ch minmax(120px, 190px) 1fr max-content; gap: 10px; align-items: center; padding: 3px 4px; border-radius: 4px; outline: none; color: inherit; text-decoration: none; }
|
|
119
121
|
.row:hover, .row:focus-visible { background: var(--accent-soft); }
|
|
122
|
+
.row:hover .name, .row:focus-visible .name { text-decoration: underline; }
|
|
120
123
|
.rank { font-family: ui-monospace, Menlo, monospace; font-size: 11px; color: var(--muted); text-align: right; font-variant-numeric: tabular-nums; }
|
|
121
124
|
.name { font-size: 0.86rem; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
|
122
125
|
.track { background: var(--track); border-radius: 3px; height: 12px; overflow: hidden; }
|
|
@@ -169,7 +172,7 @@ export function generateDashboard(root = process.cwd()) {
|
|
|
169
172
|
</div>
|
|
170
173
|
|
|
171
174
|
<h2>Leaderboard</h2>
|
|
172
|
-
<p class="h2sub">Tokens = o200k_base count of the canonical <code>tools/list</code> bytes.
|
|
175
|
+
<p class="h2sub">Tokens = o200k_base count of the canonical <code>tools/list</code> bytes — the wire payload. What a <em>Claude request</em> actually carries can differ sharply (github: 54,422 on the wire, ${dSrv['github'] ? fmt(dSrv['github'].claudeDelta) : '…'} in a request — 80% of its schema bytes are fields no Anthropic request sends). Hover a row for both numbers; <a href="METHODOLOGY.html#claude-divergence">method</a>.</p>
|
|
173
176
|
<div class="board">
|
|
174
177
|
${barRows || '<p class="h2sub">Sweep in progress — first results land shortly.</p>'}
|
|
175
178
|
</div>
|
|
@@ -195,7 +198,7 @@ ${barRows || '<p class="h2sub">Sweep in progress — first results land shortly.
|
|
|
195
198
|
|
|
196
199
|
<details><summary>Full data table</summary>
|
|
197
200
|
<div class="tablewrap" style="margin-top:10px"><table>
|
|
198
|
-
<thead><tr><th>#</th><th>server</th><th>tokens</th><th>tools</th><th>band</th><th>category</th></tr></thead>
|
|
201
|
+
<thead><tr><th>#</th><th>server</th><th>tokens (o200k)</th><th>claude req</th><th>tools</th><th>band</th><th>category</th></tr></thead>
|
|
199
202
|
<tbody>${tableRows}</tbody>
|
|
200
203
|
</table></div>
|
|
201
204
|
</details>
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
import type { Measurement } from '../core/types.js';
|
|
2
|
+
export interface HistoryRow {
|
|
3
|
+
/** UTC calendar day of the measurement (YYYY-MM-DD). */
|
|
4
|
+
date: string;
|
|
5
|
+
server: string;
|
|
6
|
+
tokens: number;
|
|
7
|
+
toolCount: number;
|
|
8
|
+
/** 'measured' or 'dynamic' — dynamic means the tool set moved between captures. */
|
|
9
|
+
status: string;
|
|
10
|
+
}
|
|
11
|
+
export declare const HISTORY_HEADER = "date,server,tokens,toolCount,status";
|
|
12
|
+
/** Parse history.csv text; malformed or non-numeric rows are dropped, not thrown. */
|
|
13
|
+
export declare function parseHistory(text: string): HistoryRow[];
|
|
14
|
+
export declare function formatHistory(rows: HistoryRow[]): string;
|
|
15
|
+
/** Upsert one row by (date, server) — the newest write for a day wins. */
|
|
16
|
+
export declare function upsert(rows: HistoryRow[], row: HistoryRow): HistoryRow[];
|
|
17
|
+
/**
|
|
18
|
+
* A measurement contributes a row only if it produced a number: failures and
|
|
19
|
+
* auth walls are recorded in the leaderboard's "not measured" section, and
|
|
20
|
+
* writing them here as zeros would fabricate a drop to zero in the series.
|
|
21
|
+
*/
|
|
22
|
+
export declare function rowFor(server: string, m: Measurement): HistoryRow | null;
|
|
23
|
+
/**
|
|
24
|
+
* Fold every results/<server>/measurement.json into results/history.csv.
|
|
25
|
+
* Idempotent: running twice over the same results is a no-op.
|
|
26
|
+
*/
|
|
27
|
+
export declare function appendHistory(root?: string): {
|
|
28
|
+
rows: number;
|
|
29
|
+
added: number;
|
|
30
|
+
};
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* results/history.csv — the time series behind the leaderboard snapshot.
|
|
3
|
+
*
|
|
4
|
+
* One row per (date, server): a server's tokens on the day it was measured.
|
|
5
|
+
* Every sweep upserts by (date, server), so re-running a sweep on the same day
|
|
6
|
+
* corrects that day's row instead of appending a duplicate. Rows for earlier
|
|
7
|
+
* dates are never rewritten — history is append-only in practice.
|
|
8
|
+
*/
|
|
9
|
+
import { existsSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
|
|
10
|
+
import { join } from 'node:path';
|
|
11
|
+
export const HISTORY_HEADER = 'date,server,tokens,toolCount,status';
|
|
12
|
+
function csvCell(s) {
|
|
13
|
+
const v = String(s ?? '');
|
|
14
|
+
return /[",\n]/.test(v) ? `"${v.replace(/"/g, '""')}"` : v;
|
|
15
|
+
}
|
|
16
|
+
/** Split one CSV line, honouring double-quoted fields. */
|
|
17
|
+
function splitCsvLine(line) {
|
|
18
|
+
const out = [];
|
|
19
|
+
let cur = '';
|
|
20
|
+
let quoted = false;
|
|
21
|
+
for (let i = 0; i < line.length; i++) {
|
|
22
|
+
const c = line[i];
|
|
23
|
+
if (quoted) {
|
|
24
|
+
if (c === '"' && line[i + 1] === '"') {
|
|
25
|
+
cur += '"';
|
|
26
|
+
i++;
|
|
27
|
+
}
|
|
28
|
+
else if (c === '"')
|
|
29
|
+
quoted = false;
|
|
30
|
+
else
|
|
31
|
+
cur += c;
|
|
32
|
+
}
|
|
33
|
+
else if (c === '"')
|
|
34
|
+
quoted = true;
|
|
35
|
+
else if (c === ',') {
|
|
36
|
+
out.push(cur);
|
|
37
|
+
cur = '';
|
|
38
|
+
}
|
|
39
|
+
else
|
|
40
|
+
cur += c;
|
|
41
|
+
}
|
|
42
|
+
out.push(cur);
|
|
43
|
+
return out;
|
|
44
|
+
}
|
|
45
|
+
/** Parse history.csv text; malformed or non-numeric rows are dropped, not thrown. */
|
|
46
|
+
export function parseHistory(text) {
|
|
47
|
+
const rows = [];
|
|
48
|
+
for (const line of text.split('\n')) {
|
|
49
|
+
if (!line.trim() || line.startsWith('date,'))
|
|
50
|
+
continue;
|
|
51
|
+
const [date, server, tokens, toolCount, status] = splitCsvLine(line);
|
|
52
|
+
if (!/^\d{4}-\d{2}-\d{2}$/.test(date ?? '') || !server)
|
|
53
|
+
continue;
|
|
54
|
+
if (!/^\d+$/.test(tokens ?? '') || !/^\d+$/.test(toolCount ?? ''))
|
|
55
|
+
continue;
|
|
56
|
+
rows.push({ date, server, tokens: Number(tokens), toolCount: Number(toolCount), status: status ?? '' });
|
|
57
|
+
}
|
|
58
|
+
return rows;
|
|
59
|
+
}
|
|
60
|
+
export function formatHistory(rows) {
|
|
61
|
+
const sorted = [...rows].sort((a, b) => a.date.localeCompare(b.date) || a.server.localeCompare(b.server));
|
|
62
|
+
const lines = sorted.map((r) => [csvCell(r.date), csvCell(r.server), r.tokens, r.toolCount, csvCell(r.status)].join(','));
|
|
63
|
+
return [HISTORY_HEADER, ...lines].join('\n') + '\n';
|
|
64
|
+
}
|
|
65
|
+
/** Upsert one row by (date, server) — the newest write for a day wins. */
|
|
66
|
+
export function upsert(rows, row) {
|
|
67
|
+
const i = rows.findIndex((r) => r.date === row.date && r.server === row.server);
|
|
68
|
+
if (i < 0)
|
|
69
|
+
return [...rows, row];
|
|
70
|
+
const next = [...rows];
|
|
71
|
+
next[i] = row;
|
|
72
|
+
return next;
|
|
73
|
+
}
|
|
74
|
+
/**
|
|
75
|
+
* A measurement contributes a row only if it produced a number: failures and
|
|
76
|
+
* auth walls are recorded in the leaderboard's "not measured" section, and
|
|
77
|
+
* writing them here as zeros would fabricate a drop to zero in the series.
|
|
78
|
+
*/
|
|
79
|
+
export function rowFor(server, m) {
|
|
80
|
+
if (m.status !== 'measured' && m.status !== 'dynamic')
|
|
81
|
+
return null;
|
|
82
|
+
if (typeof m.totalTokens !== 'number' || typeof m.toolCount !== 'number')
|
|
83
|
+
return null;
|
|
84
|
+
const date = String(m.measuredAt ?? '').slice(0, 10);
|
|
85
|
+
if (!/^\d{4}-\d{2}-\d{2}$/.test(date))
|
|
86
|
+
return null;
|
|
87
|
+
return { date, server, tokens: m.totalTokens, toolCount: m.toolCount, status: m.status };
|
|
88
|
+
}
|
|
89
|
+
/**
|
|
90
|
+
* Fold every results/<server>/measurement.json into results/history.csv.
|
|
91
|
+
* Idempotent: running twice over the same results is a no-op.
|
|
92
|
+
*/
|
|
93
|
+
export function appendHistory(root = process.cwd()) {
|
|
94
|
+
const resultsDir = join(root, 'results');
|
|
95
|
+
const path = join(resultsDir, 'history.csv');
|
|
96
|
+
const existing = existsSync(path) ? parseHistory(readFileSync(path, 'utf8')) : [];
|
|
97
|
+
let rows = existing;
|
|
98
|
+
if (!existsSync(resultsDir))
|
|
99
|
+
return { rows: 0, added: 0 };
|
|
100
|
+
for (const server of readdirSync(resultsDir, { withFileTypes: true })
|
|
101
|
+
.filter((d) => d.isDirectory())
|
|
102
|
+
.map((d) => d.name)
|
|
103
|
+
.sort()) {
|
|
104
|
+
const file = join(resultsDir, server, 'measurement.json');
|
|
105
|
+
if (!existsSync(file))
|
|
106
|
+
continue;
|
|
107
|
+
let m;
|
|
108
|
+
try {
|
|
109
|
+
m = JSON.parse(readFileSync(file, 'utf8'));
|
|
110
|
+
}
|
|
111
|
+
catch {
|
|
112
|
+
continue; // a half-written measurement should not abort the whole fold
|
|
113
|
+
}
|
|
114
|
+
const row = rowFor(server, m);
|
|
115
|
+
if (row)
|
|
116
|
+
rows = upsert(rows, row);
|
|
117
|
+
}
|
|
118
|
+
writeFileSync(path, formatHistory(rows));
|
|
119
|
+
return { rows: rows.length, added: rows.length - existing.length };
|
|
120
|
+
}
|
package/dist/sweep/regen.js
CHANGED
|
@@ -1,7 +1,14 @@
|
|
|
1
|
-
/** Regenerate leaderboard +
|
|
1
|
+
/** Regenerate leaderboard + history + server pages from results/: npx tsx src/sweep/regen.ts */
|
|
2
2
|
import { readFileSync } from 'node:fs';
|
|
3
3
|
import { parse } from 'yaml';
|
|
4
4
|
import { writeLeaderboard, percentiles } from './report.js';
|
|
5
|
+
import { appendHistory } from './history.js';
|
|
6
|
+
import { writeServerPages } from './server-pages.js';
|
|
5
7
|
const doc = parse(readFileSync('servers.yaml', 'utf8'));
|
|
6
8
|
writeLeaderboard(doc.servers);
|
|
9
|
+
// History first: the server pages read history.csv for their over-time table.
|
|
10
|
+
const h = appendHistory();
|
|
11
|
+
const p = writeServerPages(doc.servers);
|
|
7
12
|
console.log('leaderboard:', JSON.stringify(percentiles(doc.servers)));
|
|
13
|
+
console.log(`history: ${h.rows} rows (${h.added >= 0 ? '+' : ''}${h.added})`);
|
|
14
|
+
console.log(`server pages: ${p.pages}`);
|
package/dist/sweep/report.d.ts
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { type DivergenceRun } from '../core/divergence.js';
|
|
1
2
|
export interface ServerEntry {
|
|
2
3
|
name: string;
|
|
3
4
|
command: string;
|
|
@@ -11,6 +12,10 @@ export interface ServerEntry {
|
|
|
11
12
|
dockerImage?: string;
|
|
12
13
|
timeoutSeconds?: number;
|
|
13
14
|
}
|
|
15
|
+
/** Neutralize markdown/table syntax in third-party strings (tool names, notes). */
|
|
16
|
+
export declare function mdCell(s: unknown): string;
|
|
17
|
+
/** results/divergence.json if a divergence run has been recorded, else null. */
|
|
18
|
+
export declare function loadDivergence(root?: string): DivergenceRun | null;
|
|
14
19
|
export declare function writeLeaderboard(entries: ServerEntry[], root?: string): void;
|
|
15
20
|
/** Percentile helper for freezing color bands against the observed distribution. */
|
|
16
21
|
export declare function percentiles(entries: ServerEntry[], root?: string): Record<string, number>;
|
package/dist/sweep/report.js
CHANGED
|
@@ -4,8 +4,9 @@
|
|
|
4
4
|
*/
|
|
5
5
|
import { existsSync, readFileSync, writeFileSync } from 'node:fs';
|
|
6
6
|
import { join } from 'node:path';
|
|
7
|
+
import { isCurrent, parseDivergence } from '../core/divergence.js';
|
|
7
8
|
/** Neutralize markdown/table syntax in third-party strings (tool names, notes). */
|
|
8
|
-
function mdCell(s) {
|
|
9
|
+
export function mdCell(s) {
|
|
9
10
|
return String(s ?? '')
|
|
10
11
|
.replace(/[|`[\]<>]/g, (c) => `\\${c}`)
|
|
11
12
|
.replace(/\r?\n/g, ' ')
|
|
@@ -21,8 +22,21 @@ function loadRows(entries, root = process.cwd()) {
|
|
|
21
22
|
return { entry, m: existsSync(p) ? JSON.parse(readFileSync(p, 'utf8')) : null };
|
|
22
23
|
});
|
|
23
24
|
}
|
|
25
|
+
/** results/divergence.json if a divergence run has been recorded, else null. */
|
|
26
|
+
export function loadDivergence(root = process.cwd()) {
|
|
27
|
+
const p = join(root, 'results', 'divergence.json');
|
|
28
|
+
return existsSync(p) ? parseDivergence(readFileSync(p, 'utf8')) : null;
|
|
29
|
+
}
|
|
24
30
|
export function writeLeaderboard(entries, root = process.cwd()) {
|
|
25
31
|
const rows = loadRows(entries, root);
|
|
32
|
+
const div = loadDivergence(root);
|
|
33
|
+
/** Claude tokens for a row, or null when not measured / stale / errored. */
|
|
34
|
+
const claude = (r) => {
|
|
35
|
+
if (!div || !r.m)
|
|
36
|
+
return null;
|
|
37
|
+
const d = div.servers[r.entry.name];
|
|
38
|
+
return isCurrent(d, r.m.canonicalSha256) ? d.claudeDelta : null;
|
|
39
|
+
};
|
|
26
40
|
const measured = rows
|
|
27
41
|
.filter((r) => r.m && (r.m.status === 'measured' || r.m.status === 'dynamic'))
|
|
28
42
|
.sort((a, b) => (b.m.totalTokens ?? 0) - (a.m.totalTokens ?? 0));
|
|
@@ -31,14 +45,28 @@ export function writeLeaderboard(entries, root = process.cwd()) {
|
|
|
31
45
|
md.push('# MCP server context-cost leaderboard');
|
|
32
46
|
md.push('');
|
|
33
47
|
md.push(`Tokens = o200k_base count of the canonical \`tools/list\` bytes ([methodology v1.0](../docs/METHODOLOGY.md)). ` +
|
|
34
|
-
`Measured ${measured.length}/${rows.length} candidates; every candidate is listed — failures are findings, not omissions
|
|
48
|
+
`Measured ${measured.length}/${rows.length} candidates; every candidate is listed — failures are findings, not omissions. ` +
|
|
49
|
+
`Server names link to their per-tool breakdown.`);
|
|
35
50
|
md.push('');
|
|
36
|
-
|
|
37
|
-
|
|
51
|
+
if (div) {
|
|
52
|
+
const n = measured.filter((r) => claude(r) !== null).length;
|
|
53
|
+
md.push(`The **claude** column is the same tools measured through Anthropic's \`count_tokens\` on ` +
|
|
54
|
+
`\`${mdCell(div.model)}\` (${mdCell(div.measuredAt)}, method \`${mdCell(div.method)}\`): the tokens the server's ` +
|
|
55
|
+
`tools add to a request, measured for the top ${n}. It is not a rescaling of the o200k column — ` +
|
|
56
|
+
`two effects pull in opposite directions, and the [per-server pages](../docs/servers/) break both out. ` +
|
|
57
|
+
`See [Claude divergence](../docs/METHODOLOGY.md#claude-divergence).`);
|
|
58
|
+
md.push('');
|
|
59
|
+
}
|
|
60
|
+
md.push(`| # | server | tokens |${div ? ' claude |' : ''} tools | largest tool | status | category |`);
|
|
61
|
+
md.push(`|---:|---|---:|${div ? '---:|' : ''}---:|---|---|---|`);
|
|
38
62
|
measured.forEach((r, i) => {
|
|
39
63
|
const m = r.m;
|
|
40
64
|
const largest = [...m.tools].sort((a, b) => b.tokens - a.tokens)[0];
|
|
41
|
-
|
|
65
|
+
const link = `[${mdCell(r.entry.name)}](../docs/servers/${encodeURIComponent(r.entry.name)}.md)`;
|
|
66
|
+
const c = claude(r);
|
|
67
|
+
md.push(`| ${i + 1} | ${link} | ${m.totalTokens.toLocaleString('en-US')} |` +
|
|
68
|
+
(div ? ` ${c === null ? '—' : c.toLocaleString('en-US')} |` : '') +
|
|
69
|
+
` ${m.toolCount} | ` +
|
|
42
70
|
`${largest ? `${mdCell(largest.name)} (${largest.tokens.toLocaleString('en-US')})` : '—'} | ${m.status} | ${mdCell(r.entry.category)} |`);
|
|
43
71
|
});
|
|
44
72
|
md.push('');
|
|
@@ -54,9 +82,12 @@ export function writeLeaderboard(entries, root = process.cwd()) {
|
|
|
54
82
|
md.push('');
|
|
55
83
|
}
|
|
56
84
|
writeFileSync(join(root, 'results', 'leaderboard.md'), md.join('\n') + '\n');
|
|
57
|
-
|
|
85
|
+
// Columns are append-only: consumers key off the header, so adding the Claude
|
|
86
|
+
// pair at the end leaves every existing parser working.
|
|
87
|
+
const csv = ['name,tokens,toolCount,status,category,metric,metricSource,claudeTokens,claudeModel'];
|
|
58
88
|
for (const r of rows) {
|
|
59
89
|
const m = r.m;
|
|
90
|
+
const c = claude(r);
|
|
60
91
|
csv.push([
|
|
61
92
|
csvCell(r.entry.name),
|
|
62
93
|
m?.totalTokens ?? '',
|
|
@@ -65,6 +96,8 @@ export function writeLeaderboard(entries, root = process.cwd()) {
|
|
|
65
96
|
csvCell(r.entry.category),
|
|
66
97
|
r.entry.metric ?? '',
|
|
67
98
|
csvCell(r.entry.metricSource),
|
|
99
|
+
c ?? '',
|
|
100
|
+
c === null ? '' : csvCell(div.model),
|
|
68
101
|
].join(','));
|
|
69
102
|
}
|
|
70
103
|
writeFileSync(join(root, 'results', 'leaderboard.csv'), csv.join('\n') + '\n');
|
package/dist/sweep/run.js
CHANGED
|
@@ -87,6 +87,10 @@ if (isMain) {
|
|
|
87
87
|
docker: process.argv.includes('--docker'),
|
|
88
88
|
dockerImage: arg('docker-image'),
|
|
89
89
|
});
|
|
90
|
+
// CLI path only — measureServer itself stays history-free so concurrent
|
|
91
|
+
// sweep-all workers never race on the same file.
|
|
92
|
+
const { appendHistory } = await import('./history.js');
|
|
93
|
+
appendHistory();
|
|
90
94
|
console.log(m.status === 'measured' || m.status === 'dynamic'
|
|
91
95
|
? `${name}: ${m.totalTokens} tokens across ${m.toolCount} tools (${m.status})`
|
|
92
96
|
: `${name}: ${m.status} — ${m.notes ?? ''}`);
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
import type { Measurement } from '../core/types.js';
|
|
2
|
+
import { type ServerEntry } from './report.js';
|
|
3
|
+
import { type HistoryRow } from './history.js';
|
|
4
|
+
import { type DivergenceRun } from '../core/divergence.js';
|
|
5
|
+
/** One server's page. `history` is that server's rows, oldest first. */
|
|
6
|
+
export declare function renderServerPage(entry: ServerEntry, m: Measurement, history?: HistoryRow[], divergence?: DivergenceRun | null): string;
|
|
7
|
+
/** The index that lists every candidate — measured ones link to their page. */
|
|
8
|
+
export declare function renderServerIndex(rows: {
|
|
9
|
+
entry: ServerEntry;
|
|
10
|
+
m: Measurement | null;
|
|
11
|
+
}[]): string;
|
|
12
|
+
/** Write docs/servers/*.md for every measured server, plus the index. */
|
|
13
|
+
export declare function writeServerPages(entries: ServerEntry[], root?: string): {
|
|
14
|
+
pages: number;
|
|
15
|
+
};
|
|
16
|
+
/** Public URL of a server's page — used by the leaderboard and the badge snippet. */
|
|
17
|
+
export declare function serverPageUrl(name: string): string;
|
|
@@ -0,0 +1,228 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Per-server detail pages: docs/servers/<name>.md plus an index.
|
|
3
|
+
*
|
|
4
|
+
* These are the badge's click-through target. A badge says "12,430 tokens";
|
|
5
|
+
* the page behind it says which tools those tokens are in, what launched the
|
|
6
|
+
* server, the hash of the bytes counted, and the one command that re-derives
|
|
7
|
+
* the number. Generated from results/ only — no network, no timestamps beyond
|
|
8
|
+
* the measurement's own, so regenerating without a new sweep is a no-op diff.
|
|
9
|
+
*/
|
|
10
|
+
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
|
|
11
|
+
import { join } from 'node:path';
|
|
12
|
+
import { bandColor, BAND_META } from '../core/bands.js';
|
|
13
|
+
import { loadDivergence, mdCell } from './report.js';
|
|
14
|
+
import { parseHistory } from './history.js';
|
|
15
|
+
import { claudeRatio, fieldSelectionShare, isCurrent } from '../core/divergence.js';
|
|
16
|
+
/**
|
|
17
|
+
* Pages are served from GitHub Pages (docs/), but results/ and badges/ are not
|
|
18
|
+
* published there — links into them must be absolute repo URLs.
|
|
19
|
+
*/
|
|
20
|
+
const REPO_URL = 'https://github.com/athakur3/mcp-context-cost';
|
|
21
|
+
const BLOB = `${REPO_URL}/blob/main`;
|
|
22
|
+
const PAGES_URL = 'https://athakur3.github.io/mcp-context-cost';
|
|
23
|
+
/** Longest per-tool table we print inline; the rest live in the raw capture. */
|
|
24
|
+
const MAX_TOOL_ROWS = 30;
|
|
25
|
+
const fmt = (n) => n.toLocaleString('en-US');
|
|
26
|
+
/** True when a measurement produced a number we can stand behind. */
|
|
27
|
+
function isMeasured(m) {
|
|
28
|
+
return !!m && (m.status === 'measured' || m.status === 'dynamic') && typeof m.totalTokens === 'number';
|
|
29
|
+
}
|
|
30
|
+
function isolationText(m) {
|
|
31
|
+
const iso = m.isolation;
|
|
32
|
+
if (!iso)
|
|
33
|
+
return 'not recorded';
|
|
34
|
+
if (!iso.docker)
|
|
35
|
+
return 'host process (no container)';
|
|
36
|
+
return ['docker', iso.image, iso.network ? `network ${iso.network}` : '', iso.note]
|
|
37
|
+
.filter(Boolean)
|
|
38
|
+
.join(' · ');
|
|
39
|
+
}
|
|
40
|
+
/**
|
|
41
|
+
* The Claude divergence section: the headline, the projection onto the three
|
|
42
|
+
* fields an Anthropic tool definition carries, and Claude's own count of that
|
|
43
|
+
* projection. Printed only when a current divergence row exists for this server.
|
|
44
|
+
*/
|
|
45
|
+
function divergenceSection(row, run) {
|
|
46
|
+
const share = fieldSelectionShare(row);
|
|
47
|
+
const ratio = claudeRatio(row);
|
|
48
|
+
const md = [];
|
|
49
|
+
md.push('## What this costs on Claude');
|
|
50
|
+
md.push('');
|
|
51
|
+
md.push(`Measured ${mdCell(run.measuredAt)} against \`${mdCell(run.model)}\` via Anthropic's \`count_tokens\` ` +
|
|
52
|
+
`(method \`${mdCell(run.method)}\`).`);
|
|
53
|
+
md.push('');
|
|
54
|
+
md.push('| | tokens | |');
|
|
55
|
+
md.push('|---|---:|---|');
|
|
56
|
+
md.push(`| o200k, full capture | ${fmt(row.o200kFull)} | the badge number — every byte \`tools/list\` returned |`);
|
|
57
|
+
md.push(`| o200k, Anthropic fields only | ${fmt(row.o200kMapped)} | ` +
|
|
58
|
+
`${share === null ? '—' : `${(share * 100).toFixed(1)}% of the capture is MCP-only metadata`} |`);
|
|
59
|
+
md.push(`| **Claude, same fields** | **${fmt(row.claudeDelta)}** | ` +
|
|
60
|
+
`${ratio === null ? '—' : `${ratio.toFixed(2)}× the badge number`} |`);
|
|
61
|
+
md.push('');
|
|
62
|
+
md.push(`An Anthropic tool definition carries \`name\`, \`description\`, and \`input_schema\` and nothing else, so ` +
|
|
63
|
+
`\`title\`, \`annotations\`, \`outputSchema\`, \`execution\`, and \`icons\` are dropped before the request — ` +
|
|
64
|
+
`that is the second row. The third row is the same tools counted by Anthropic, which is larger than the ` +
|
|
65
|
+
`second because Anthropic's tokenizer is denser on this content than o200k_base *and* the API adds its own ` +
|
|
66
|
+
`framing (at most ${fmt(run.probeDelta)} tokens of it fixed, measured against a single minimal tool). ` +
|
|
67
|
+
`The two effects run in opposite directions, which is why the Claude number is not a fixed multiple of the badge.`);
|
|
68
|
+
md.push('');
|
|
69
|
+
return md;
|
|
70
|
+
}
|
|
71
|
+
/** One server's page. `history` is that server's rows, oldest first. */
|
|
72
|
+
export function renderServerPage(entry, m, history = [], divergence = null) {
|
|
73
|
+
const total = m.totalTokens;
|
|
74
|
+
const band = BAND_META[bandColor(total)];
|
|
75
|
+
const tools = [...m.tools].sort((a, b) => b.tokens - a.tokens);
|
|
76
|
+
const shown = tools.slice(0, MAX_TOOL_ROWS);
|
|
77
|
+
const pct = (n) => (total > 0 ? `${((n / total) * 100).toFixed(1)}%` : '—');
|
|
78
|
+
const md = [];
|
|
79
|
+
md.push(`# ${mdCell(entry.name)} — context cost`);
|
|
80
|
+
md.push('');
|
|
81
|
+
md.push(`**${fmt(total)} tokens** across ${m.toolCount} tools — *${band.label}* (${band.range}). ` +
|
|
82
|
+
`Measured ${String(m.measuredAt).slice(0, 10)} under [methodology v${mdCell(m.methodologyVersion)}](../METHODOLOGY.html).`);
|
|
83
|
+
md.push('');
|
|
84
|
+
md.push('| | |');
|
|
85
|
+
md.push('|---|---|');
|
|
86
|
+
md.push(`| server (self-reported) | ${mdCell(m.serverName)}${m.serverVersion ? ` v${mdCell(String(m.serverVersion).replace(/^v/, ''))}` : ''} |`);
|
|
87
|
+
md.push(`| status | ${mdCell(m.status)} |`);
|
|
88
|
+
md.push(`| tokenizer | ${mdCell(m.provider)} / ${mdCell(m.encoding)} |`);
|
|
89
|
+
md.push(`| launch command | \`${mdCell(m.launchCommand ?? entry.command)}\` |`);
|
|
90
|
+
md.push(`| isolation | ${mdCell(isolationText(m))} |`);
|
|
91
|
+
md.push(`| env vars supplied | ${m.envVarNames?.length ? m.envVarNames.map(mdCell).join(', ') : 'none'} |`);
|
|
92
|
+
md.push(`| canonical SHA-256 | \`${mdCell(m.canonicalSha256)}\` |`);
|
|
93
|
+
if (entry.category)
|
|
94
|
+
md.push(`| category | ${mdCell(entry.category)} |`);
|
|
95
|
+
if (entry.repo)
|
|
96
|
+
md.push(`| source | ${mdCell(entry.repo)} |`);
|
|
97
|
+
md.push('');
|
|
98
|
+
if (m.status === 'dynamic') {
|
|
99
|
+
md.push('> This server\'s `tools/list` differed between two consecutive captures, so the number ' +
|
|
100
|
+
'is the first capture and moves between sweeps. Treat it as a range, not a constant.');
|
|
101
|
+
md.push('');
|
|
102
|
+
}
|
|
103
|
+
md.push('## Where the tokens are');
|
|
104
|
+
md.push('');
|
|
105
|
+
md.push('| tool | tokens | share | description | schema |');
|
|
106
|
+
md.push('|---|---:|---:|---:|---:|');
|
|
107
|
+
for (const t of shown) {
|
|
108
|
+
md.push(`| ${mdCell(t.name)} | ${fmt(t.tokens)} | ${pct(t.tokens)} | ${fmt(t.descriptionTokens)} | ${fmt(t.inputSchemaTokens)} |`);
|
|
109
|
+
}
|
|
110
|
+
md.push('');
|
|
111
|
+
if (tools.length > shown.length) {
|
|
112
|
+
const rest = tools.slice(shown.length).reduce((s, t) => s + t.tokens, 0);
|
|
113
|
+
md.push(`*${tools.length - shown.length} smaller tools omitted (${fmt(rest)} tokens combined) — ` +
|
|
114
|
+
`all of them are in the [raw capture](${BLOB}/results/${encodeURIComponent(entry.name)}/measurement.json).*`);
|
|
115
|
+
md.push('');
|
|
116
|
+
}
|
|
117
|
+
md.push('Each tool is tokenized on its own, so the parts do not sum exactly to the whole: the ' +
|
|
118
|
+
'array adds its own brackets and commas, and the tokenizer merges tokens across object ' +
|
|
119
|
+
'boundaries. The badge number is always the count of the whole array, never a sum of parts.');
|
|
120
|
+
md.push('');
|
|
121
|
+
const divRow = divergence?.servers[entry.name];
|
|
122
|
+
if (divergence && isCurrent(divRow, m.canonicalSha256)) {
|
|
123
|
+
md.push(...divergenceSection(divRow, divergence));
|
|
124
|
+
}
|
|
125
|
+
if (history.length > 1) {
|
|
126
|
+
md.push('## Over time');
|
|
127
|
+
md.push('');
|
|
128
|
+
md.push('| date | tokens | tools | change |');
|
|
129
|
+
md.push('|---|---:|---:|---:|');
|
|
130
|
+
history.forEach((h, i) => {
|
|
131
|
+
const prev = history[i - 1];
|
|
132
|
+
const delta = prev ? h.tokens - prev.tokens : null;
|
|
133
|
+
const change = delta === null ? '—' : delta === 0 ? 'no change' : `${delta > 0 ? '+' : ''}${fmt(delta)}`;
|
|
134
|
+
md.push(`| ${mdCell(h.date)} | ${fmt(h.tokens)} | ${h.toolCount} | ${change} |`);
|
|
135
|
+
});
|
|
136
|
+
md.push('');
|
|
137
|
+
md.push(`Full series: [results/history.csv](${BLOB}/results/history.csv).`);
|
|
138
|
+
md.push('');
|
|
139
|
+
}
|
|
140
|
+
md.push('## Re-derive it');
|
|
141
|
+
md.push('');
|
|
142
|
+
md.push('```bash');
|
|
143
|
+
md.push(`npx -y mcp-context-cost verify results/${entry.name}/measurement.json`);
|
|
144
|
+
md.push('```');
|
|
145
|
+
md.push('');
|
|
146
|
+
md.push(`That re-tokenizes the [published capture](${BLOB}/results/${encodeURIComponent(entry.name)}/measurement.json) ` +
|
|
147
|
+
`and checks the count and the hash. If it disagrees with the badge, the badge is wrong — ` +
|
|
148
|
+
`[open an issue](${REPO_URL}/issues) and it gets corrected.`);
|
|
149
|
+
md.push('');
|
|
150
|
+
md.push(`[Badge JSON](${BLOB}/badges/${encodeURIComponent(entry.name)}.json) · ` +
|
|
151
|
+
`[All servers](index.html) · [Leaderboard](${BLOB}/results/leaderboard.md) · ` +
|
|
152
|
+
`[Methodology](../METHODOLOGY.html)`);
|
|
153
|
+
md.push('');
|
|
154
|
+
return md.join('\n');
|
|
155
|
+
}
|
|
156
|
+
/** The index that lists every candidate — measured ones link to their page. */
|
|
157
|
+
export function renderServerIndex(rows) {
|
|
158
|
+
const measured = rows.filter((r) => isMeasured(r.m)).sort((a, b) => b.m.totalTokens - a.m.totalTokens);
|
|
159
|
+
const rest = rows.filter((r) => !isMeasured(r.m));
|
|
160
|
+
const md = [];
|
|
161
|
+
md.push('# Server pages');
|
|
162
|
+
md.push('');
|
|
163
|
+
md.push(`One page per measured server: the per-tool breakdown behind the badge, the exact launch ` +
|
|
164
|
+
`command, and the command that re-derives the number. ${measured.length} of ${rows.length} ` +
|
|
165
|
+
`candidates measured.`);
|
|
166
|
+
md.push('');
|
|
167
|
+
md.push('| # | server | tokens | tools | band |');
|
|
168
|
+
md.push('|---:|---|---:|---:|---|');
|
|
169
|
+
measured.forEach((r, i) => {
|
|
170
|
+
const t = r.m.totalTokens;
|
|
171
|
+
md.push(`| ${i + 1} | [${mdCell(r.entry.name)}](${encodeURIComponent(r.entry.name)}.html) | ${fmt(t)} | ` +
|
|
172
|
+
`${r.m.toolCount} | ${BAND_META[bandColor(t)].label} |`);
|
|
173
|
+
});
|
|
174
|
+
md.push('');
|
|
175
|
+
if (rest.length > 0) {
|
|
176
|
+
md.push('## Not measured');
|
|
177
|
+
md.push('');
|
|
178
|
+
md.push('No page: there is no number to show. The reason is recorded per candidate.');
|
|
179
|
+
md.push('');
|
|
180
|
+
md.push('| server | status |');
|
|
181
|
+
md.push('|---|---|');
|
|
182
|
+
for (const r of rest) {
|
|
183
|
+
const status = r.entry.remote ? 'remote-auth-wall' : (r.m?.status ?? 'not-yet-run');
|
|
184
|
+
md.push(`| ${mdCell(r.entry.name)} | ${mdCell(status)} |`);
|
|
185
|
+
}
|
|
186
|
+
md.push('');
|
|
187
|
+
}
|
|
188
|
+
md.push(`[Leaderboard](${BLOB}/results/leaderboard.md) · [Methodology](../METHODOLOGY.html) · [Dashboard](../dashboard.html)`);
|
|
189
|
+
md.push('');
|
|
190
|
+
return md.join('\n');
|
|
191
|
+
}
|
|
192
|
+
/** Write docs/servers/*.md for every measured server, plus the index. */
|
|
193
|
+
export function writeServerPages(entries, root = process.cwd()) {
|
|
194
|
+
const outDir = join(root, 'docs', 'servers');
|
|
195
|
+
mkdirSync(outDir, { recursive: true });
|
|
196
|
+
const historyPath = join(root, 'results', 'history.csv');
|
|
197
|
+
const history = existsSync(historyPath) ? parseHistory(readFileSync(historyPath, 'utf8')) : [];
|
|
198
|
+
const divergence = loadDivergence(root);
|
|
199
|
+
const rows = entries.map((entry) => {
|
|
200
|
+
const p = join(root, 'results', entry.name, 'measurement.json');
|
|
201
|
+
let m = null;
|
|
202
|
+
if (existsSync(p)) {
|
|
203
|
+
try {
|
|
204
|
+
m = JSON.parse(readFileSync(p, 'utf8'));
|
|
205
|
+
}
|
|
206
|
+
catch {
|
|
207
|
+
m = null; // a half-written measurement should not abort the whole run
|
|
208
|
+
}
|
|
209
|
+
}
|
|
210
|
+
return { entry, m };
|
|
211
|
+
});
|
|
212
|
+
let pages = 0;
|
|
213
|
+
for (const { entry, m } of rows) {
|
|
214
|
+
if (!isMeasured(m))
|
|
215
|
+
continue;
|
|
216
|
+
const series = history
|
|
217
|
+
.filter((h) => h.server === entry.name)
|
|
218
|
+
.sort((a, b) => a.date.localeCompare(b.date));
|
|
219
|
+
writeFileSync(join(outDir, `${entry.name}.md`), renderServerPage(entry, m, series, divergence));
|
|
220
|
+
pages++;
|
|
221
|
+
}
|
|
222
|
+
writeFileSync(join(outDir, 'index.md'), renderServerIndex(rows));
|
|
223
|
+
return { pages };
|
|
224
|
+
}
|
|
225
|
+
/** Public URL of a server's page — used by the leaderboard and the badge snippet. */
|
|
226
|
+
export function serverPageUrl(name) {
|
|
227
|
+
return `${PAGES_URL}/servers/${encodeURIComponent(name)}.html`;
|
|
228
|
+
}
|
package/dist/sweep/sweep-all.js
CHANGED
|
@@ -8,6 +8,7 @@ import { readFileSync } from 'node:fs';
|
|
|
8
8
|
import { parse } from 'yaml';
|
|
9
9
|
import { measureServer } from './run.js';
|
|
10
10
|
import { writeLeaderboard } from './report.js';
|
|
11
|
+
import { appendHistory } from './history.js';
|
|
11
12
|
function arg(name) {
|
|
12
13
|
const i = process.argv.indexOf(`--${name}`);
|
|
13
14
|
return i >= 0 ? process.argv[i + 1] : undefined;
|
|
@@ -45,6 +46,8 @@ async function worker() {
|
|
|
45
46
|
}
|
|
46
47
|
}
|
|
47
48
|
await Promise.all(Array.from({ length: Math.max(1, concurrency) }, () => worker()));
|
|
49
|
+
// Serial, after every worker has finished: history.csv is a read-modify-write.
|
|
48
50
|
writeLeaderboard(doc.servers);
|
|
51
|
+
const h = appendHistory();
|
|
49
52
|
const measured = Object.values(summary).filter((s) => s.includes('tokens')).length;
|
|
50
|
-
console.log(`done: ${measured}/${entries.length} measured; leaderboard regenerated`);
|
|
53
|
+
console.log(`done: ${measured}/${entries.length} measured; leaderboard + history (${h.rows} rows) regenerated`);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "mcp-context-cost",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.2.0",
|
|
4
4
|
"description": "Reproducible context-cost badges for MCP servers — measure what a server's tool schemas cost before the agent does any work",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -29,9 +29,11 @@
|
|
|
29
29
|
],
|
|
30
30
|
"scripts": {
|
|
31
31
|
"build": "tsc",
|
|
32
|
+
"prepublishOnly": "npm run build && npm test && npm run typecheck",
|
|
32
33
|
"test": "vitest run",
|
|
33
34
|
"test:badge": "./upstream/tests/badge-test.sh",
|
|
34
|
-
"typecheck": "tsc --noEmit",
|
|
35
|
+
"typecheck": "tsc --noEmit && tsc -p tsconfig.tools.json",
|
|
36
|
+
"divergence": "tsx tools/measure-divergence.ts",
|
|
35
37
|
"sweep": "tsx src/sweep/run.ts",
|
|
36
38
|
"sweep:all": "tsx src/sweep/sweep-all.ts",
|
|
37
39
|
"verify": "tsx src/cli.ts verify"
|
|
@@ -44,6 +46,7 @@
|
|
|
44
46
|
"yaml": "^2.9.0"
|
|
45
47
|
},
|
|
46
48
|
"devDependencies": {
|
|
49
|
+
"@anthropic-ai/sdk": "^0.117.1",
|
|
47
50
|
"@types/node": "^26.2.0",
|
|
48
51
|
"tsx": "^4.23.12",
|
|
49
52
|
"typescript": "^7.0.2",
|