toolmenu 0.17.1 → 0.20.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -23,7 +23,15 @@ connections and releases, and checks **when** it changes:
23
23
  It also checks the menu clients will actually show: a broken `x-mcp-header` or a
24
24
  `$ref` they can't resolve hides a tool or fails every call to it. And when a tool
25
25
  stops to ask the user something, `session` answers "no" and says whether the tool
26
- can take it.
26
+ can take it. Handles a server mints (`session_id`) should say how long they last and
27
+ fail readably when unknown; menus that differ by who asks (`--as`) shouldn't be
28
+ cached as public; prompts and resources (`--lists`) are checked like tools. Run
29
+ against the reference server, `@modelcontextprotocol/server-everything` 2.1.0, with
30
+ no false findings ([bench/](bench/README.md)).
31
+ - **For your client.** `--client claude-code | claude-api-search | openai` checks the
32
+ menu the way that client loads it, from its own docs: what it costs up front and per
33
+ tool a search loads, where it cuts descriptions, what its search can find, and names
34
+ its API refuses.
27
35
  - **Between releases.** `diff` finds breaking changes and token growth, and checks
28
36
  the version bump.
29
37
 
@@ -247,9 +255,9 @@ the way OpenAPI breaking-change checkers do it:
247
255
 
248
256
  | Class | Changes | Rules |
249
257
  |---|---|---|
250
- | **breaking** (error) | tool removed or renamed, parameter removed, new required parameter, narrower or different type, enum narrowed, unlisted properties now rejected (`additionalProperties: false` added), a tool becoming less safe (no longer read-only, or additive → destructive) | `diff/tool-removed`, `diff/tool-renamed`, `diff/param-removed`, `diff/param-required`, `diff/param-type`, `diff/enum-narrowed`, `diff/properties-closed`, `diff/safety-hint` |
251
- | **minor** (info) | tool added, new optional parameter, parameter now optional, type or enum widened, unlisted properties now accepted (`additionalProperties: false` removed) | `diff/tool-added`, `diff/param-added`, `diff/param-relaxed`, `diff/type-widened`, `diff/enum-widened`, `diff/properties-opened` |
252
- | **notice** (info) | optional parameter dropped (extra properties still allowed), description changed (with the text diff: it changes what the agent does), output schema, annotations, other fields, order, a schema restructured but accepting the same input, only the declared `$schema` dialect changed | `diff/param-dropped`, `diff/description`, `diff/schema-other`, `diff/annotations`, `diff/other`, `diff/order`, `diff/schema-equivalent`, `diff/schema-dialect` |
258
+ | **breaking** (error) | tool removed or renamed, parameter removed, new required parameter, narrower or different type, enum narrowed, unlisted properties now rejected (`additionalProperties: false` added), a tool becoming less safe (no longer read-only, or additive → destructive); with `--lists` on both sides, a prompt removed or newly requiring an argument, a resource template removed | `diff/tool-removed`, `diff/tool-renamed`, `diff/param-removed`, `diff/param-required`, `diff/param-type`, `diff/enum-narrowed`, `diff/properties-closed`, `diff/safety-hint`, `diff/prompt-removed`, `diff/prompt-argument-required`, `diff/resource-template-removed` |
259
+ | **minor** (info) | tool added, new optional parameter, parameter now optional, type or enum widened, unlisted properties now accepted (`additionalProperties: false` removed), a prompt or resource template added | `diff/tool-added`, `diff/param-added`, `diff/param-relaxed`, `diff/type-widened`, `diff/enum-widened`, `diff/properties-opened`, `diff/prompt-added`, `diff/resource-template-added` |
260
+ | **notice** (info) | optional parameter dropped (extra properties still allowed), description changed (with the text diff: it changes what the agent does), output schema, annotations, other fields, order, a schema restructured but accepting the same input, only the declared `$schema` dialect changed, a prompt's description or optional arguments, resources added or gone (data, so no bump) | `diff/param-dropped`, `diff/description`, `diff/schema-other`, `diff/annotations`, `diff/other`, `diff/order`, `diff/schema-equivalent`, `diff/schema-dialect`, `diff/prompt-changed`, `diff/resources` |
253
261
 
254
262
  The same rules apply inside parameters: fields of an object parameter, array items,
255
263
  and the options of an `anyOf`/`oneOf` union (zod's unions, discriminated unions and
@@ -394,6 +402,9 @@ server, or from the spec itself. If a rule can't point to one, it doesn't ship.
394
402
  | `write/no-dry-run` | info | Destructive tools with no dry-run parameter or preview tool, as one summary |
395
403
  | `spec/discover` | info | The server only speaks the 2025 protocol, so 2026-07-28 rules are skipped |
396
404
  | `spec/deprecated`, `spec/cache-hints` | info | 2026-07-28: deprecated features still advertised; `ttlMs: 0`, or `cacheScope: "public"` on a server that took credentials |
405
+ | `lists/variance` | warn | With `--lists`: `prompts/list`, `resources/list` or `resources/templates/list` differs between two calls, or from a second process or connection (2026-07-28: lists no longer vary per connection) |
406
+ | `lists/schema` | error | With `--lists`: a list result that doesn't match the official schema (2026-07-28 requires `ttlMs` and `cacheScope` on each), or a declared list that fails to answer (warn; info for Method not found, or a setup failure such as missing credentials) |
407
+ | `naming/client-name` | warn | With `--client`: tool names the client's API refuses: `^[a-zA-Z0-9_-]{1,128}$` for the Claude API, `^[a-zA-Z0-9_-]+$` up to 128 for OpenAI's functions. MCP allows names such as `files.read` that neither takes as they are |
397
408
  | `menu/auth-variance` | info | With `--as` (or `identities` in the config): the tool set differs by identity, what each adds, lacks or defines differently against the first, and its tokens. Allowed by the spec; a baseline of one identity's menu doesn't cover the others. An identity that can't be listed is said here too |
398
409
  | `spec/cache-scope-public` | warn | 2026-07-28, with `--as`: the tool set differs by identity, and `tools/list` says `cacheScope: "public"`, which lets shared caches serve one caller's list to another |
399
410
 
@@ -443,6 +454,23 @@ Clients that send descriptions in full don't need a setting beyond that. If your
443
454
  client cuts shorter, as mine did at 280, set its number. `fullDescriptions` lists the
444
455
  tools it sends uncut, so the description rules skip them.
445
456
 
457
+ ### Checking for one client (`--client`)
458
+
459
+ What an agent sees depends on its client. `snapshot --client <name>` (or `"client"` in the
460
+ config) checks the menu against what that client's own docs say, and every value
461
+ toolmenu uses names its source and the date it was read (`src/clients.ts`). What a
462
+ client doesn't document is left out, not guessed.
463
+
464
+ | `--client` | Up front | Search | Description cut | Names |
465
+ |---|---|---|---|---|
466
+ | `claude-code` | tool names; full definitions for tools marked `_meta["anthropic/alwaysLoad"]` | reads a description up to 4,096 characters | 16,384 through search, 4,096 up front | Claude API's |
467
+ | `claude-api-search` | nothing from tools you defer (keep the 3–5 used most loaded) | name, description, argument names and descriptions; 5 results by default | none documented | `^[a-zA-Z0-9_-]{1,128}$` |
468
+ | `openai` | each deferred function's name and description (its schema loads on demand); a deferred MCP server shows only its own name and description | how many results, and which fields, isn't documented, so `search/findable` doesn't run | none documented | `^[a-zA-Z0-9_-]+$`, up to 128 |
469
+
470
+ The snapshot adds a line such as `as Claude Code: up front ~524 tokens · on demand ~344
471
+ per tool a search loads (largest: update_contacts, ~2,093)`, and `--json` a `client`
472
+ object. With a client that defers tools, `search/findable` runs at any menu size.
473
+
446
474
  ## What toolmenu can't catch
447
475
 
448
476
  toolmenu sees what the server sends. Plenty of agent failures happen elsewhere:
@@ -475,6 +503,10 @@ toolmenu sees what the server sends. Plenty of agent failures happen elsewhere:
475
503
  (2026-07-28 only; an error if the server lacks it)
476
504
  --processes <n> server processes or connections to compare (default: 2);
477
505
  1 opens no second one, session's scope check included
506
+ --lists <kinds> snapshot: also read and check prompts, resources (comma-separated);
507
+ saved in the menu file beside the tools
508
+ --client <name> snapshot: check the menu for one client, from its own docs:
509
+ claude-code, claude-api-search or openai
478
510
  --as <name>:<cred> snapshot: list the menu as this identity too, repeatable:
479
511
  "reader:Authorization: Bearer …" (HTTP) or "admin:ROLE=admin"
480
512
  (stdio). The first is the menu saved; the others are compared
package/dist/cli.js CHANGED
@@ -2,6 +2,8 @@
2
2
  import { writeFile } from 'node:fs/promises';
3
3
  import { join } from 'node:path';
4
4
  import { parseArgs } from 'node:util';
5
+ import { CLIENT_NAMES, CLIENTS } from './clients.js';
6
+ import { LIST_WORDS } from './lists.js';
5
7
  import { loadConfig } from './config.js';
6
8
  import { ELICITATIONS, PROTOCOLS } from './connect.js';
7
9
  import { diffMenus } from './diff.js';
@@ -82,6 +84,16 @@ export async function main(argv) {
82
84
  const elicitation = (values.elicitation ?? 'decline');
83
85
  if (!ELICITATIONS.includes(elicitation))
84
86
  throw new UsageError('--elicitation must be decline or cancel');
87
+ if (values.lists !== undefined && sub !== 'snapshot')
88
+ throw new UsageError('--lists goes with snapshot.');
89
+ const lists = values.lists !== undefined ? values.lists.split(',').map((w) => w.trim()).filter(Boolean) : config.lists;
90
+ const badList = (lists ?? []).find((w) => !LIST_WORDS.includes(w));
91
+ if (badList)
92
+ throw new UsageError(`--lists takes ${LIST_WORDS.join(', ')}, comma-separated; not "${badList}"`);
93
+ if (values.client !== undefined && sub !== 'snapshot')
94
+ throw new UsageError('--client goes with snapshot.');
95
+ if (values.client !== undefined && !Object.hasOwn(CLIENTS, values.client))
96
+ throw new UsageError(`--client must be one of ${CLIENT_NAMES.join(', ')}`);
85
97
  if (values.as !== undefined && sub !== 'snapshot')
86
98
  throw new UsageError('--as goes with snapshot: it lists the menu once per identity.');
87
99
  if (values.elicitation !== undefined && sub !== 'session')
@@ -300,11 +312,12 @@ export async function main(argv) {
300
312
  const identities = values.as?.length ? parseIdentities(values.as, target.kind) : configIdentities(config.identities, target.kind);
301
313
  const routesPath = values.routes ?? config.routes;
302
314
  const routes = routesPath ? await loadRoutes(routesPath) : undefined;
303
- const { menu, findings } = await snapshot(target, { routes, timeoutMs, processes, identities, catalog: values.catalog ? (config.catalog ?? true) : undefined, rules: config.rules, ignore: config.ignore, descriptionLimit: config.descriptionLimit, fullDescriptions: config.fullDescriptions });
315
+ const client = values.client ?? config.client;
316
+ const { menu, findings, cost, notOffered } = await snapshot(target, { routes, timeoutMs, processes, identities, client, lists, catalog: values.catalog ? (config.catalog ?? true) : undefined, rules: config.rules, ignore: config.ignore, descriptionLimit: config.descriptionLimit, fullDescriptions: config.fullDescriptions });
304
317
  const outPath = values['no-write'] ? undefined : values.out ?? 'menu.json';
305
318
  if (outPath)
306
319
  await writeFile(outPath, JSON.stringify(menu, null, 2) + '\n');
307
- const output = formatSnapshot(menu, findings, format, outPath);
320
+ const output = formatSnapshot(menu, findings, format, outPath, cost, notOffered);
308
321
  if (output)
309
322
  process.stdout.write(output + '\n');
310
323
  return findings.some((f) => SEVERITY_RANK[f.severity] >= SEVERITY_RANK[failOn]) ? 1 : 0;
@@ -0,0 +1,69 @@
1
+ import type { Menu, MenuTool } from './types.js';
2
+ /** A value, where it was read, and when. */
3
+ export interface Sourced<T> {
4
+ value: T;
5
+ source: string;
6
+ read: string;
7
+ }
8
+ export interface ClientProfile {
9
+ /** How the client is named in messages. */
10
+ label: string;
11
+ /**
12
+ * Whether MCP tools are deferred behind a tool search: `all` (every tool unless
13
+ * marked to load up front), `chosen` (the integrator marks each tool), or `none`.
14
+ */
15
+ defers: Sourced<'all' | 'chosen' | 'none'>;
16
+ /** What the model has of a deferred tool before it searches: its name, or nothing. */
17
+ /**
18
+ * What the model has of a deferred tool before it searches: its name, nothing, or its
19
+ * name and description (OpenAI defers the parameter schema).
20
+ */
21
+ beforeSearch?: Sourced<'name' | 'nothing' | 'name-and-description'>;
22
+ /** Where the client cuts a description: for a tool loaded up front, and one loaded through search. */
23
+ descriptionCut?: Sourced<{
24
+ upfront: number;
25
+ searched: number;
26
+ }>;
27
+ /** How much of a description the search reads. */
28
+ searchReads?: Sourced<number>;
29
+ /** What the search matches on. */
30
+ searchFields?: Sourced<string[]>;
31
+ /** How many tools a search returns unless the model asks for more. */
32
+ searchLimit?: Sourced<number>;
33
+ /** Tool names the API accepts. */
34
+ namePattern?: Sourced<{
35
+ pattern: string;
36
+ max: number;
37
+ api: string;
38
+ }>;
39
+ /** Advice on which tools to keep loaded up front. */
40
+ keepLoaded?: Sourced<string>;
41
+ }
42
+ export declare const CLIENTS: Record<string, ClientProfile>;
43
+ export declare const CLIENT_NAMES: string[];
44
+ /** A client's profile by name: own keys only ("toString" names no client). */
45
+ export declare function profileOf(name: string | undefined): ClientProfile | undefined;
46
+ /** The profile for a client name, or undefined for none given. Throws for an unknown name. */
47
+ export declare function clientProfile(name: string | undefined): ClientProfile | undefined;
48
+ /** Whether a tool name fits the client's API, and why not. */
49
+ export declare function nameProblem(name: string, profile: ClientProfile): string | undefined;
50
+ /** Whether Claude Code loads this tool up front: marked so by its server (`_meta["anthropic/alwaysLoad"]: true`). */
51
+ export declare function loadsUpfront(tool: MenuTool): boolean;
52
+ export interface ClientCost {
53
+ client: string;
54
+ label: string;
55
+ /** Tokens before any search: the deferred tools' names (Claude Code) and the definitions of tools loaded up front. Undefined when the client's docs don't say what the model sees. */
56
+ upfront?: number;
57
+ /** Tools loaded up front, in full. */
58
+ loaded: string[];
59
+ /** Average tokens a deferred tool adds when a search loads it, and the largest. */
60
+ perTool: number;
61
+ largest?: {
62
+ tool: string;
63
+ tokens: number;
64
+ };
65
+ /** One line saying what the numbers stand on. */
66
+ basis: string;
67
+ }
68
+ /** What the menu costs under a client that defers tools: up front, and per tool a search loads. */
69
+ export declare function clientCost(name: string, menu: Menu): ClientCost | undefined;
@@ -0,0 +1,125 @@
1
+ /*
2
+ * What an agent sees depends on its client. Each profile holds only what that
3
+ * client's own docs or API spec say, and every value carries where it was read and
4
+ * when. A value nobody has sourced is not here: toolmenu leaves that part out
5
+ * rather than guess (docs/plans/0.15-2026-readiness.md, point 6).
6
+ */
7
+ import { countTokens } from './menu.js';
8
+ const CLAUDE_CODE_MCP = 'https://code.claude.com/docs/en/mcp';
9
+ const CLAUDE_TOOL_SEARCH = 'https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool';
10
+ const CLAUDE_DEFINE_TOOLS = 'https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools';
11
+ const OPENAI_SPEC = 'https://github.com/openai/openai-openapi (openapi.yaml: FunctionToolParam, NamespaceToolParam, MCPTool)';
12
+ const OPENAI_TOOL_SEARCH = 'https://developers.openai.com/api/docs/guides/tools-tool-search';
13
+ const READ = '2026-11-02';
14
+ const claudeName = { value: { pattern: '^[a-zA-Z0-9_-]{1,128}$', max: 128, api: 'the Claude API' }, source: CLAUDE_DEFINE_TOOLS, read: READ };
15
+ export const CLIENTS = {
16
+ 'claude-code': {
17
+ label: 'Claude Code',
18
+ // "Tool search is enabled by default: MCP tools are deferred and discovered on demand."
19
+ defers: { value: 'all', source: CLAUDE_CODE_MCP, read: READ },
20
+ // "Only tool names and server instructions load at session start."
21
+ beforeSearch: { value: 'name', source: CLAUDE_CODE_MCP, read: READ },
22
+ // "Server instructions, and the description of a tool that loads upfront: 4,096 characters";
23
+ // "The description of a tool that Claude loads through tool search: 16,384 characters".
24
+ descriptionCut: { value: { upfront: 4096, searched: 16384 }, source: CLAUDE_CODE_MCP, read: READ },
25
+ // "When tool search matches Claude's query against a tool's description, it reads only the part within the shorter limit."
26
+ searchReads: { value: 4096, source: CLAUDE_CODE_MCP, read: READ },
27
+ namePattern: claudeName,
28
+ },
29
+ 'claude-api-search': {
30
+ label: "Claude's tool search (Claude API)",
31
+ // "set defer_loading: true on the tools that shouldn't load up front"
32
+ defers: { value: 'chosen', source: CLAUDE_TOOL_SEARCH, read: READ },
33
+ // "Initially, Claude's context contains only the tool search tool and any non-deferred tools."
34
+ beforeSearch: { value: 'nothing', source: CLAUDE_TOOL_SEARCH, read: READ },
35
+ // "Both tool search variants (regex and bm25) search tool names, descriptions, argument names, and argument descriptions."
36
+ searchFields: { value: ['name', 'description', 'argument names', 'argument descriptions'], source: CLAUDE_TOOL_SEARCH, read: READ },
37
+ // "each search returns up to 5 matching tools by default"
38
+ searchLimit: { value: 5, source: CLAUDE_TOOL_SEARCH, read: READ },
39
+ keepLoaded: { value: 'Keep your 3–5 most frequently used tools non-deferred so Claude can call them without searching first.', source: CLAUDE_TOOL_SEARCH, read: READ },
40
+ namePattern: claudeName,
41
+ },
42
+ openai: {
43
+ label: "OpenAI's tool_search (Responses API)",
44
+ // FunctionToolParam.defer_loading, MCPTool.defer_loading: per function, or per MCP server.
45
+ defers: { value: 'chosen', source: OPENAI_SPEC, read: READ },
46
+ // "For an individual deferred function, the model still sees the function name and description,
47
+ // so in practice tool search is mostly deferring the parameter schema." For an MCP server or
48
+ // namespace, "the model sees only the namespace or server name and description at the beginning".
49
+ beforeSearch: { value: 'name-and-description', source: OPENAI_TOOL_SEARCH, read: '2026-11-03' },
50
+ // Not in the guide or the spec: how many tools a search returns, and which fields it matches on.
51
+ // So search/findable doesn't rank for OpenAI.
52
+ // FunctionToolParam.name: maxLength 128, pattern ^[a-zA-Z0-9_-]+$.
53
+ namePattern: { value: { pattern: '^[a-zA-Z0-9_-]+$', max: 128, api: "OpenAI's Responses API" }, source: OPENAI_SPEC, read: READ },
54
+ },
55
+ };
56
+ export const CLIENT_NAMES = Object.keys(CLIENTS);
57
+ /** A client's profile by name: own keys only ("toString" names no client). */
58
+ export function profileOf(name) {
59
+ return name !== undefined && Object.hasOwn(CLIENTS, name) ? CLIENTS[name] : undefined;
60
+ }
61
+ /** The profile for a client name, or undefined for none given. Throws for an unknown name. */
62
+ export function clientProfile(name) {
63
+ if (name === undefined)
64
+ return undefined;
65
+ const profile = profileOf(name);
66
+ if (!profile)
67
+ throw new Error(`unknown client "${name}": one of ${CLIENT_NAMES.join(', ')}`);
68
+ return profile;
69
+ }
70
+ /** Whether a tool name fits the client's API, and why not. */
71
+ export function nameProblem(name, profile) {
72
+ const rule = profile.namePattern?.value;
73
+ if (!rule)
74
+ return undefined;
75
+ if (name.length > rule.max)
76
+ return `${name.length} characters, over ${rule.max}`;
77
+ if (!new RegExp(rule.pattern).test(name)) {
78
+ const bad = [...new Set([...name].filter((c) => !/[a-zA-Z0-9_-]/.test(c)))];
79
+ return `has ${bad.map((c) => (c === ' ' ? 'a space' : `"${c}"`)).join(', ')}, outside letters, digits, _ and -`;
80
+ }
81
+ return undefined;
82
+ }
83
+ /** Whether Claude Code loads this tool up front: marked so by its server (`_meta["anthropic/alwaysLoad"]: true`). */
84
+ export function loadsUpfront(tool) {
85
+ return tool._meta?.['anthropic/alwaysLoad'] === true;
86
+ }
87
+ /** What the menu costs under a client that defers tools: up front, and per tool a search loads. */
88
+ export function clientCost(name, menu) {
89
+ const profile = profileOf(name);
90
+ if (!profile || profile.defers.value === 'none')
91
+ return undefined;
92
+ const loaded = profile.defers.value === 'all' ? menu.tools.filter(loadsUpfront) : [];
93
+ const deferred = menu.tools.filter((t) => !loaded.includes(t));
94
+ const loadedTokens = loaded.reduce((n, t) => n + t.tokens, 0);
95
+ const seen = profile.beforeSearch?.value;
96
+ // OpenAI keeps a deferred function's name and description up front: what a search adds is the rest.
97
+ const seenUpfront = (t) => countTokens(`${t.name}\n${t.description ?? ''}`);
98
+ const upfront = seen === 'name'
99
+ ? countTokens(deferred.map((t) => t.name).join('\n')) + loadedTokens
100
+ : seen === 'nothing'
101
+ ? loadedTokens
102
+ : seen === 'name-and-description'
103
+ ? deferred.reduce((n, t) => n + seenUpfront(t), 0) + loadedTokens
104
+ : undefined;
105
+ const onDemand = (t) => (seen === 'name-and-description' ? Math.max(0, t.tokens - seenUpfront(t)) : t.tokens);
106
+ const basis = seen === 'name'
107
+ ? `${profile.label} defers MCP tools and shows their names until a search loads one; the server's instructions load up front too and aren't counted here (${profile.beforeSearch.source}, read ${profile.beforeSearch.read})`
108
+ : seen === 'nothing'
109
+ ? `with every tool deferred, the model sees none of them until a search loads one, and the docs advise keeping the 3–5 tools used most loaded (${profile.beforeSearch.source}, read ${profile.beforeSearch.read})`
110
+ : seen === 'name-and-description'
111
+ ? `deferred as functions, each tool's name and description stay up front and its schema loads on demand; deferred as one MCP server, the model sees only the server's name and description (set where the server is added) until a search loads its tools (${profile.beforeSearch.source}, read ${profile.beforeSearch.read})`
112
+ : `${profile.label} defers per function or per MCP server; what the model sees before a search isn't documented, so no up-front figure`;
113
+ const perTool = deferred.length ? Math.round(deferred.reduce((n, t) => n + onDemand(t), 0) / deferred.length) : 0;
114
+ const biggest = [...deferred].sort((a, b) => onDemand(b) - onDemand(a))[0];
115
+ const largest = biggest ? { name: biggest.name, tokens: onDemand(biggest) } : undefined;
116
+ return {
117
+ client: name,
118
+ label: profile.label,
119
+ ...(upfront !== undefined ? { upfront } : {}),
120
+ loaded: loaded.map((t) => t.name),
121
+ perTool,
122
+ ...(largest ? { largest: { tool: largest.name, tokens: largest.tokens } } : {}),
123
+ basis,
124
+ };
125
+ }
package/dist/config.d.ts CHANGED
@@ -26,6 +26,10 @@ export interface Config {
26
26
  headers?: Record<string, string>;
27
27
  env?: Record<string, string>;
28
28
  }>;
29
+ /** snapshot: which lists to read besides tools: ["prompts", "resources"]. --lists wins. */
30
+ lists?: string[];
31
+ /** snapshot: the client to check the menu for (claude-code, claude-api-search, openai). --client wins. */
32
+ client?: string;
29
33
  /** Server processes (stdio) or connections (HTTP) to compare, the main one included (default 2). */
30
34
  processes?: number;
31
35
  /**
package/dist/config.js CHANGED
@@ -1,5 +1,7 @@
1
1
  import { existsSync } from 'node:fs';
2
2
  import { readFile } from 'node:fs/promises';
3
+ import { CLIENTS } from './clients.js';
4
+ import { LIST_WORDS } from './lists.js';
3
5
  import { CLIENT_LIMITS } from './rules/description.js';
4
6
  const DEFAULT_PATH = 'toolmenu.config.json';
5
7
  /** Optional: no config file is fine. */
@@ -23,12 +25,18 @@ export async function loadConfig(path) {
23
25
  throw new Error(`${file}: protocol must be auto, legacy or modern`);
24
26
  }
25
27
  const limit = config.descriptionLimit;
26
- if (limit !== undefined && !(typeof limit === 'number' && limit > 0) && !(typeof limit === 'string' && limit in CLIENT_LIMITS)) {
28
+ if (limit !== undefined && !(typeof limit === 'number' && limit > 0) && !(typeof limit === 'string' && Object.hasOwn(CLIENT_LIMITS, limit))) {
27
29
  throw new Error(`${file}: descriptionLimit must be a positive number or one of ${Object.keys(CLIENT_LIMITS).join(', ')}`);
28
30
  }
29
31
  if (config.processes !== undefined && !(Number.isInteger(config.processes) && config.processes >= 1)) {
30
32
  throw new Error(`${file}: processes must be a whole number, 1 or more`);
31
33
  }
34
+ if (config.lists !== undefined && !(Array.isArray(config.lists) && config.lists.every((l) => LIST_WORDS.includes(l)))) {
35
+ throw new Error(`${file}: lists must be a list of ${LIST_WORDS.join(', ')}`);
36
+ }
37
+ if (config.client !== undefined && !(typeof config.client === 'string' && Object.hasOwn(CLIENTS, config.client))) {
38
+ throw new Error(`${file}: client must be one of ${Object.keys(CLIENTS).join(', ')}`);
39
+ }
32
40
  if (config.identities !== undefined) {
33
41
  const strings = (v) => v === undefined || (typeof v === 'object' && v !== null && !Array.isArray(v) && Object.values(v).every((x) => typeof x === 'string'));
34
42
  const ids = config.identities;
package/dist/diff.js CHANGED
@@ -53,6 +53,17 @@ export const DIFF_RULES = {
53
53
  'diff/catalog-missing': { severity: 'info' },
54
54
  'diff/catalog-added': { severity: 'info' },
55
55
  'diff/catalog-queries': { severity: 'info' },
56
+ // snapshot --lists: prompts are called by name like tools, and resource templates are
57
+ // the URIs clients read through. Concrete resources are data (files come and go):
58
+ // counted, no bump. Only compared when both menus read the list.
59
+ 'diff/prompt-removed': { severity: 'error', class: 'breaking' },
60
+ 'diff/prompt-argument-required': { severity: 'error', class: 'breaking' },
61
+ 'diff/prompt-added': { severity: 'info', class: 'minor' },
62
+ 'diff/prompt-changed': { severity: 'info', class: 'notice' },
63
+ 'diff/resource-template-removed': { severity: 'error', class: 'breaking' },
64
+ 'diff/resource-template-added': { severity: 'info', class: 'minor' },
65
+ 'diff/resources': { severity: 'info', class: 'notice' },
66
+ 'diff/lists-unread': { severity: 'info' },
56
67
  };
57
68
  /**
58
69
  * In fixes: the release a breaking change should ship in. Resolved once the
@@ -135,6 +146,7 @@ export function diffMenus(before, after, options = {}) {
135
146
  });
136
147
  }
137
148
  raw.push(...compareCatalogs(before.catalog, after.catalog, ignored));
149
+ raw.push(...compareLists(before, after));
138
150
  // Ignored tools leave the token counts and the budget too.
139
151
  const tokens = tokenChange(oldTools, newTools);
140
152
  if (options.tokenBudget !== undefined && tokens.after > options.tokenBudget) {
@@ -1220,6 +1232,84 @@ function expandRefs(schema) {
1220
1232
  return { schema, expanded: false };
1221
1233
  }
1222
1234
  }
1235
+ /** The arguments a prompt requires. */
1236
+ function requiredArguments(p) {
1237
+ return (Array.isArray(p.arguments) ? p.arguments : []).filter((a) => a.required === true).map((a) => String(a.name));
1238
+ }
1239
+ /**
1240
+ * snapshot --lists: prompts, resource templates and resources between two menus.
1241
+ * A list only one menu read isn't compared (everything would read as added or removed).
1242
+ */
1243
+ function compareLists(before, after) {
1244
+ const out = [];
1245
+ const unread = [];
1246
+ const pair = (key, label) => {
1247
+ const a = before[key];
1248
+ const b = after[key];
1249
+ if (!a && !b)
1250
+ return undefined;
1251
+ if (!a || !b) {
1252
+ unread.push(`${label} (${a ? 'the old' : 'the new'} menu has them, the ${a ? 'new' : 'old'} one wasn't read with --lists)`);
1253
+ return undefined;
1254
+ }
1255
+ return [a, b];
1256
+ };
1257
+ const prompts = pair('prompts', 'prompts');
1258
+ if (prompts) {
1259
+ const [a, b] = prompts;
1260
+ const oldBy = new Map(a.map((p) => [String(p.name), p]));
1261
+ const newBy = new Map(b.map((p) => [String(p.name), p]));
1262
+ for (const [name] of oldBy) {
1263
+ if (!newBy.has(name))
1264
+ out.push({ rule: 'diff/prompt-removed', message: `Prompt ${code(name)} was removed. Users and clients that invoke it by name (a slash command, a saved workflow) now get an error.`, fix: `Keep ${code(name)} for one release, or ${BREAKING_RELEASE}.` });
1265
+ }
1266
+ for (const [name, p] of newBy) {
1267
+ const old = oldBy.get(name);
1268
+ if (!old) {
1269
+ out.push({ rule: 'diff/prompt-added', message: `Prompt ${code(name)} is new.` });
1270
+ continue;
1271
+ }
1272
+ const required = requiredArguments(p).filter((r) => !requiredArguments(old).includes(r));
1273
+ if (required.length) {
1274
+ out.push({ rule: 'diff/prompt-argument-required', message: `Prompt ${code(name)} now requires ${required.map(code).join(', ')}. Invocations that don't pass ${required.length === 1 ? 'it' : 'them'} now fail.`, fix: `Make ${required.length === 1 ? 'it' : 'them'} optional, or ${BREAKING_RELEASE}.` });
1275
+ }
1276
+ else if (JSON.stringify(old) !== JSON.stringify(p)) {
1277
+ out.push({ rule: 'diff/prompt-changed', message: `Prompt ${code(name)} changed (its description or arguments), and nothing it requires.` });
1278
+ }
1279
+ }
1280
+ }
1281
+ const templates = pair('resourceTemplates', 'resource templates');
1282
+ if (templates) {
1283
+ const [a, b] = templates;
1284
+ const oldSet = new Set(a.map((t) => String(t.uriTemplate)));
1285
+ const newSet = new Set(b.map((t) => String(t.uriTemplate)));
1286
+ for (const t of oldSet)
1287
+ if (!newSet.has(t))
1288
+ out.push({ rule: 'diff/resource-template-removed', message: `Resource template ${code(t)} was removed. Clients that read resources through it now get an error.`, fix: `Keep ${code(t)} for one release, or ${BREAKING_RELEASE}.` });
1289
+ for (const t of newSet)
1290
+ if (!oldSet.has(t))
1291
+ out.push({ rule: 'diff/resource-template-added', message: `Resource template ${code(t)} is new.` });
1292
+ }
1293
+ const resources = pair('resources', 'resources');
1294
+ if (resources) {
1295
+ const [a, b] = resources;
1296
+ const oldSet = new Set(a.map((r) => String(r.uri)));
1297
+ const newSet = new Set(b.map((r) => String(r.uri)));
1298
+ const gone = [...oldSet].filter((u) => !newSet.has(u));
1299
+ const fresh = [...newSet].filter((u) => !oldSet.has(u));
1300
+ if (gone.length || fresh.length) {
1301
+ out.push({
1302
+ rule: 'diff/resources',
1303
+ message: `The resources list changed: ${[fresh.length ? `${fresh.length} new` : '', gone.length ? `${gone.length} gone` : ''].filter(Boolean).join(', ')}. Resources are often data that changes between runs; a resource clients rely on by URI going away is worth a look.`,
1304
+ detail: [...gone.slice(0, 5).map((u) => `gone: ${u}`), ...fresh.slice(0, 5).map((u) => `new: ${u}`), ...(gone.length + fresh.length > 10 ? ['…'] : [])],
1305
+ });
1306
+ }
1307
+ }
1308
+ if (unread.length) {
1309
+ out.push({ rule: 'diff/lists-unread', message: `Not compared: ${unread.join('; ')}.`, fix: 'Snapshot both releases with the same --lists to compare them.' });
1310
+ }
1311
+ return out;
1312
+ }
1223
1313
  /** What the operations are behind: the search tool or router by name, or "the command routers" as words. */
1224
1314
  function catalogSource(c) {
1225
1315
  return (c.routers?.length ?? 0) > 1 ? c.tool : code(c.tool);
package/dist/help.js CHANGED
@@ -19,6 +19,8 @@ const OPTIONS = {
19
19
  env: ['--env <K=V>', ['environment variable for a stdio server, repeatable. The server only', 'gets a minimal environment (PATH, HOME, ...) plus these']],
20
20
  protocol: ['--protocol <auto|legacy|modern>', ['which handshake to make (default: auto)', 'legacy: the 2025 handshake only; modern: 2026-07-28 only, an error if the server lacks it']],
21
21
  as: ['--as <name>[:<credential>]', ['list the menu as this identity too, repeatable: a header ("reader:Authorization:', 'Bearer ...") or env var ("admin:ROLE=admin") on top of --header/--env. The first', 'is the main one (its menu is saved); the others are compared with it']],
22
+ lists: ['--lists <kinds>', ['which lists to read and check, comma-separated: tools (always),', 'prompts, resources (resources/list and resources/templates/list).', 'Saved in the menu file beside the tools']],
23
+ client: ['--client <name>', ['check the menu for one client, from its own docs: claude-code (tools deferred,', 'its description cut and search), claude-api-search (Claude API tool search)', 'or openai (Responses API tool_search): cost up front and per tool, names it refuses']],
22
24
  'no-auth': ['--no-auth', ["don't use a stored OAuth login (toolmenu auth login) for this server"]],
23
25
  timeout: ['--timeout <ms>', ['per-request timeout (default: 30000)']],
24
26
  processes: ['--processes <n>', ['server processes (stdio) or connections (HTTP) to compare, the main', 'one included (default: 2). 1 opens no second one: for servers with one', 'session per client']],
@@ -78,7 +80,7 @@ const HELP = {
78
80
  'reports what in the menu confuses agents or breaks prompt caches.',
79
81
  SERVER_NOTE,
80
82
  ],
81
- own: ['out', 'no-write', 'routes', 'catalog', 'as'],
83
+ own: ['out', 'no-write', 'routes', 'catalog', 'as', 'client', 'lists'],
82
84
  shared: ['config', 'format', 'json', 'fail-on', 'header', 'env', 'no-auth', 'protocol', 'timeout', 'processes'],
83
85
  examples: [
84
86
  'toolmenu snapshot -- npx -y @modelcontextprotocol/server-memory',
@@ -0,0 +1,40 @@
1
+ import { type Connection } from './connect.js';
2
+ export declare const LIST_KINDS: readonly ['prompts', 'resources', 'resourceTemplates'];
3
+ export type ListKind = (typeof LIST_KINDS)[number];
4
+ /** The --lists words: `resources` reads both resources/list and resources/templates/list. */
5
+ export declare const LIST_WORDS: readonly ['tools', 'prompts', 'resources'];
6
+ /** Each list's JSON-RPC method, result key, the key that names an item, and its words. */
7
+ export declare const LIST_INFO: Record<ListKind, {
8
+ method: string;
9
+ key: string;
10
+ id: string;
11
+ one: string;
12
+ capability: 'prompts' | 'resources';
13
+ }>;
14
+ /** Pages read at most: a resources list can be a whole file tree. */
15
+ export declare const MAX_PAGES = 50;
16
+ export interface ListRead {
17
+ kind: ListKind;
18
+ items: Record<string, unknown>[];
19
+ /** Raw result pages, as received (spec/schema checks them). */
20
+ pages: Record<string, unknown>[];
21
+ listMeta: {
22
+ ttlMs?: number;
23
+ cacheScope?: string;
24
+ };
25
+ /** Stopped at MAX_PAGES. */
26
+ truncated?: boolean;
27
+ /** Why it couldn't be read. */
28
+ error?: string;
29
+ /** The JSON-RPC error code it failed with, when there was one (-32601: method not found). */
30
+ code?: number;
31
+ }
32
+ /** Read one list, page by page, as the server sends it. */
33
+ export declare function readList(connection: Connection, kind: ListKind, timeoutMs?: number): Promise<ListRead>;
34
+ /** The list kinds the --lists words ask for that the server declares a capability for, and the ones it doesn't. */
35
+ export declare function kindsToRead(words: string[], capabilities: Record<string, unknown>): {
36
+ read: ListKind[];
37
+ undeclared: ListKind[];
38
+ };
39
+ /** Two reads of one list: what changed between them, by item id and in order. */
40
+ export declare function listChanges(kind: ListKind, before: Record<string, unknown>[], after: Record<string, unknown>[]): string[];
package/dist/lists.js ADDED
@@ -0,0 +1,91 @@
1
+ /*
2
+ * Prompts and resources: the other lists a client loads up front (2026-07-28:
3
+ * cacheable like tools/list, SEP-2549, and no longer varying per connection).
4
+ * snapshot --lists tools,prompts,resources reads them; the rules in
5
+ * rules/lists.ts check them the way the menu rules check tools.
6
+ */
7
+ import { quietExclusions } from './connect.js';
8
+ export const LIST_KINDS = ['prompts', 'resources', 'resourceTemplates'];
9
+ /** The --lists words: `resources` reads both resources/list and resources/templates/list. */
10
+ export const LIST_WORDS = ['tools', 'prompts', 'resources'];
11
+ /** Each list's JSON-RPC method, result key, the key that names an item, and its words. */
12
+ export const LIST_INFO = {
13
+ prompts: { method: 'prompts/list', key: 'prompts', id: 'name', one: 'prompt', capability: 'prompts' },
14
+ resources: { method: 'resources/list', key: 'resources', id: 'uri', one: 'resource', capability: 'resources' },
15
+ resourceTemplates: { method: 'resources/templates/list', key: 'resourceTemplates', id: 'uriTemplate', one: 'resource template', capability: 'resources' },
16
+ };
17
+ /** Pages read at most: a resources list can be a whole file tree. */
18
+ export const MAX_PAGES = 50;
19
+ /** Read one list, page by page, as the server sends it. */
20
+ export async function readList(connection, kind, timeoutMs) {
21
+ const info = LIST_INFO[kind];
22
+ const pages = [];
23
+ let cursor;
24
+ try {
25
+ for (let i = 0; i < MAX_PAGES; i++) {
26
+ const mark = connection.wire.responses.length;
27
+ const params = cursor ? { cursor } : undefined;
28
+ const call = () => {
29
+ const c = connection.client;
30
+ const fn = kind === 'prompts' ? 'listPrompts' : kind === 'resources' ? 'listResources' : 'listResourceTemplates';
31
+ return c[fn].call(connection.client, params, { timeout: timeoutMs, cacheMode: 'bypass' });
32
+ };
33
+ let failed;
34
+ await quietExclusions(call).catch((e) => (failed = e));
35
+ // The page as the server sent it: the SDK may reject a result toolmenu still checks.
36
+ const page = connection.wire.since(mark, info.method).at(-1)?.result;
37
+ if (!page)
38
+ throw failed ?? new Error(`no ${info.method} result`);
39
+ pages.push(page);
40
+ cursor = typeof page.nextCursor === 'string' && page.nextCursor ? page.nextCursor : undefined;
41
+ if (!cursor)
42
+ break;
43
+ }
44
+ }
45
+ catch (error) {
46
+ const code = error?.code;
47
+ return { kind, items: [], pages, listMeta: {}, error: (error instanceof Error ? error.message : String(error)).split('\n')[0].replace(/^MCP error -?\d+:\s*/, ''), ...(typeof code === 'number' ? { code } : {}) };
48
+ }
49
+ const first = pages[0] ?? {};
50
+ return {
51
+ kind,
52
+ items: pages.flatMap((p) => (Array.isArray(p[info.key]) ? p[info.key] : [])),
53
+ pages,
54
+ listMeta: {
55
+ ...(typeof first.ttlMs === 'number' ? { ttlMs: first.ttlMs } : {}),
56
+ ...(typeof first.cacheScope === 'string' ? { cacheScope: first.cacheScope } : {}),
57
+ },
58
+ ...(cursor ? { truncated: true } : {}),
59
+ };
60
+ }
61
+ /** The list kinds the --lists words ask for that the server declares a capability for, and the ones it doesn't. */
62
+ export function kindsToRead(words, capabilities) {
63
+ const wanted = [...(words.includes('prompts') ? ['prompts'] : []), ...(words.includes('resources') ? ['resources', 'resourceTemplates'] : [])];
64
+ return {
65
+ read: wanted.filter((k) => capabilities[LIST_INFO[k].capability] !== undefined),
66
+ undeclared: wanted.filter((k) => capabilities[LIST_INFO[k].capability] === undefined),
67
+ };
68
+ }
69
+ /** Two reads of one list: what changed between them, by item id and in order. */
70
+ export function listChanges(kind, before, after) {
71
+ const id = (x) => String(x[LIST_INFO[kind].id] ?? JSON.stringify(x));
72
+ const a = new Map(before.map((x) => [id(x), JSON.stringify(x)]));
73
+ const b = new Map(after.map((x) => [id(x), JSON.stringify(x)]));
74
+ const out = [];
75
+ for (const k of a.keys())
76
+ if (!b.has(k))
77
+ out.push(`${k}: gone`);
78
+ for (const k of b.keys())
79
+ if (!a.has(k))
80
+ out.push(`${k}: new`);
81
+ for (const [k, v] of b)
82
+ if (a.has(k) && a.get(k) !== v)
83
+ out.push(`${k}: ${sameKeys(a.get(k), v) ? 'key order changed' : 'changed'}`);
84
+ if (!out.length && [...a.keys()].join('\n') !== [...b.keys()].join('\n'))
85
+ out.push('same items, different order');
86
+ return out;
87
+ }
88
+ function sameKeys(x, y) {
89
+ const sort = (v) => (Array.isArray(v) ? v.map(sort) : v && typeof v === 'object' ? Object.fromEntries(Object.entries(v).sort(([p], [q]) => p.localeCompare(q)).map(([k, w]) => [k, sort(w)])) : v);
90
+ return JSON.stringify(sort(JSON.parse(x))) === JSON.stringify(sort(JSON.parse(y)));
91
+ }
package/dist/options.d.ts CHANGED
@@ -32,6 +32,12 @@ export declare const CLI_OPTIONS: {
32
32
  readonly type: 'string';
33
33
  readonly multiple: true;
34
34
  };
35
+ readonly client: {
36
+ readonly type: 'string';
37
+ };
38
+ readonly lists: {
39
+ readonly type: 'string';
40
+ };
35
41
  readonly timeout: {
36
42
  readonly type: 'string';
37
43
  };
package/dist/options.js CHANGED
@@ -13,6 +13,8 @@ export const CLI_OPTIONS = {
13
13
  header: { type: 'string', multiple: true },
14
14
  env: { type: 'string', multiple: true },
15
15
  as: { type: 'string', multiple: true },
16
+ client: { type: 'string' },
17
+ lists: { type: 'string' },
16
18
  timeout: { type: 'string' },
17
19
  protocol: { type: 'string' },
18
20
  elicitation: { type: 'string' },
package/dist/report.d.ts CHANGED
@@ -1,10 +1,11 @@
1
+ import type { ClientCost } from './clients.js';
1
2
  import type { DiffResult } from './diff.js';
2
3
  import type { HistoryResult } from './history.js';
3
4
  import { type Scenario, type SessionResult } from './session.js';
4
5
  import { type Finding, type Menu, type Severity } from './types.js';
5
6
  export type Format = 'text' | 'json' | 'github' | 'markdown';
6
7
  export declare function counts(findings: Finding[]): Record<Severity, number>;
7
- export declare function formatSnapshot(menu: Menu, findings: Finding[], format: Format, outPath?: string): string;
8
+ export declare function formatSnapshot(menu: Menu, findings: Finding[], format: Format, outPath?: string, cost?: ClientCost, notOffered?: string[]): string;
8
9
  export declare function formatDiff(d: DiffResult, beforeTools: number, afterTools: number, format: Format): string;
9
10
  export declare function formatHistory(h: HistoryResult, format: Format, outDir: string, csvPath?: string): string;
10
11
  export declare function formatSession(s: SessionResult, format: Format): string;
package/dist/report.js CHANGED
@@ -12,7 +12,23 @@ export function counts(findings) {
12
12
  function plural(n, word) {
13
13
  return `${n} ${word}${n === 1 ? '' : 's'}`;
14
14
  }
15
- export function formatSnapshot(menu, findings, format, outPath) {
15
+ /** snapshot --lists: "3 prompts · 5 resources · 2 resource templates", or nothing. */
16
+ function otherLists(menu) {
17
+ return [
18
+ menu.prompts ? plural(menu.prompts.length, 'prompt') : '',
19
+ menu.resources ? plural(menu.resources.length, 'resource') : '',
20
+ menu.resourceTemplates ? plural(menu.resourceTemplates.length, 'resource template') : '',
21
+ ].filter(Boolean).join(' · ');
22
+ }
23
+ /** --client: "as Claude Code: up front ~N tokens (…) · on demand ~M per tool loaded (largest: x, ~K)". */
24
+ function costLine(cost) {
25
+ const parts = [
26
+ cost.upfront !== undefined ? `up front ~${cost.upfront.toLocaleString('en-US')} tokens${cost.loaded.length ? ` (names, and ${plural(cost.loaded.length, 'tool')} loaded in full)` : ''}` : '',
27
+ `on demand ~${cost.perTool.toLocaleString('en-US')} per tool a search loads${cost.largest ? ` (largest: ${cost.largest.tool}, ~${cost.largest.tokens.toLocaleString('en-US')})` : ''}`,
28
+ ].filter(Boolean);
29
+ return `as ${cost.label}: ${parts.join(' · ')}`;
30
+ }
31
+ export function formatSnapshot(menu, findings, format, outPath, cost, notOffered = []) {
16
32
  if (format === 'json') {
17
33
  return JSON.stringify({
18
34
  command: 'snapshot',
@@ -21,6 +37,11 @@ export function formatSnapshot(menu, findings, format, outPath) {
21
37
  totalTokens: menu.totalTokens,
22
38
  listMeta: menu.listMeta,
23
39
  breakdown: breakdown(menu),
40
+ ...(cost ? { client: cost } : {}),
41
+ ...(menu.prompts ? { prompts: menu.prompts.length } : {}),
42
+ ...(menu.resources ? { resources: menu.resources.length } : {}),
43
+ ...(menu.resourceTemplates ? { resourceTemplates: menu.resourceTemplates.length } : {}),
44
+ ...(notOffered.length ? { notOffered } : {}),
24
45
  written: outPath ?? null,
25
46
  counts: counts(findings),
26
47
  findings,
@@ -34,6 +55,8 @@ export function formatSnapshot(menu, findings, format, outPath) {
34
55
  `### toolmenu snapshot: \`${s.name ?? 'server'}\` ${s.version ?? ''}`.trimEnd(),
35
56
  '',
36
57
  `Protocol ${s.protocolVersion ?? 'unknown'} · ${plural(menu.tools.length, 'tool')} · ~${menu.totalTokens.toLocaleString('en-US')} tokens (estimate)`,
58
+ ...(cost ? ['', `${costLine(cost)}. _${cost.basis}._`] : []),
59
+ ...(otherLists(menu) ? ['', `Also listed: ${otherLists(menu)}`] : []),
37
60
  '',
38
61
  ...mdFindings(findings),
39
62
  ...mdBreakdown(menu),
@@ -44,6 +67,13 @@ export function formatSnapshot(menu, findings, format, outPath) {
44
67
  `toolmenu snapshot ${s.name ?? 'unknown server'} ${s.version ?? ''}`.trimEnd(),
45
68
  ` protocol ${s.protocolVersion ?? 'unknown'} · ${plural(menu.tools.length, 'tool')} · ~${menu.totalTokens.toLocaleString('en-US')} tokens (estimate)`,
46
69
  ];
70
+ if (cost)
71
+ lines.push(` ${costLine(cost)}`, ` ${cost.basis}`);
72
+ const others = otherLists(menu);
73
+ // Lists asked for that the server doesn't offer: said once, not a finding.
74
+ const caps = [...new Set(notOffered.map((k) => (k === 'prompts' ? 'prompts' : 'resources')))];
75
+ if (others || caps.length)
76
+ lines.push(` ${[others ? `also listed: ${others}` : '', caps.length ? `no ${caps.join(' or ')} offered (not declared)` : ''].filter(Boolean).join(' · ')}`);
47
77
  if (outPath)
48
78
  lines.push(` wrote ${outPath}`);
49
79
  lines.push('');
@@ -0,0 +1,7 @@
1
+ import { type Rule } from './rule.js';
2
+ /**
3
+ * --client: tool names the client's API doesn't take. MCP allows names its APIs
4
+ * refuse ("files.read", "get user"): Claude's tool definitions must match
5
+ * ^[a-zA-Z0-9_-]{1,128}$, OpenAI's function names ^[a-zA-Z0-9_-]+$ up to 128.
6
+ */
7
+ export declare const clientName: Rule;
@@ -0,0 +1,31 @@
1
+ import { nameProblem, profileOf } from '../clients.js';
2
+ import { count, kept } from './rule.js';
3
+ /**
4
+ * --client: tool names the client's API doesn't take. MCP allows names its APIs
5
+ * refuse ("files.read", "get user"): Claude's tool definitions must match
6
+ * ^[a-zA-Z0-9_-]{1,128}$, OpenAI's function names ^[a-zA-Z0-9_-]+$ up to 128.
7
+ */
8
+ export const clientName = {
9
+ id: 'naming/client-name',
10
+ severity: 'warn',
11
+ summary: "Tool names the client's API doesn't accept as they are",
12
+ run(ctx) {
13
+ const profile = profileOf(ctx.client);
14
+ if (!profile?.namePattern)
15
+ return [];
16
+ const bad = kept(ctx.menu.tools, ctx)
17
+ .map((t) => ({ tool: t.name, why: nameProblem(t.name, profile) }))
18
+ .filter((b) => !!b.why);
19
+ if (!bad.length)
20
+ return [];
21
+ const rule = profile.namePattern;
22
+ return [
23
+ {
24
+ ...(bad.length === 1 ? { tool: bad[0].tool } : {}),
25
+ message: `${count(bad.length, 'tool name')} ${bad.length === 1 ? "doesn't" : "don't"} fit the tool name format of ${rule.value.api}${ctx.client === 'claude-code' ? `, which ${profile.label} sends tools to` : ''} (${rule.value.pattern}, at most ${rule.value.max} characters). MCP allows them, but a request that passes them to the API as they are is refused; a client that rewrites them shows the model a name that isn't the server's.`,
26
+ detail: [...bad.slice(0, 12).map((b) => `${b.tool}: ${b.why}`), ...(bad.length > 12 ? [`and ${bad.length - 12} more`] : []), `Source: ${rule.source} (read ${rule.read}).`],
27
+ fix: 'Use letters, digits, _ and - in tool names (files_read, not files.read), at most 128 characters.',
28
+ },
29
+ ];
30
+ },
31
+ };
@@ -25,7 +25,7 @@ const CLAUDE_CODE_UPFRONT = 4096;
25
25
  export function resolveLimit(setting) {
26
26
  if (typeof setting === 'number')
27
27
  return { limit: setting, who: 'your client', assumed: false, perTool: () => setting };
28
- const client = setting !== undefined && setting in CLIENT_LIMITS ? setting : DEFAULT_CLIENT;
28
+ const client = setting !== undefined && Object.hasOwn(CLIENT_LIMITS, setting) ? setting : DEFAULT_CLIENT;
29
29
  const limit = CLIENT_LIMITS[client];
30
30
  const upfront = (tool) => tool._meta?.['anthropic/alwaysLoad'] === true;
31
31
  return {
@@ -2,6 +2,8 @@ import { SEVERITY_RANK } from '../types.js';
2
2
  import { buried, cut, lateInstruction } from './description.js';
3
3
  import { connectionVariance, hostSpecific, nondeterministic, processVariance } from './determinism.js';
4
4
  import { authoredIds } from './ids.js';
5
+ import { clientName } from './clients.js';
6
+ import { listSchema, listVariance } from './lists.js';
5
7
  import { handleLifetime } from './handles.js';
6
8
  import { authVariance, cacheScopePublic } from './identities.js';
7
9
  import { route, sharedWord, vagueId } from './naming.js';
@@ -21,6 +23,7 @@ export const MENU_RULES = [
21
23
  vagueId,
22
24
  sharedWord,
23
25
  route,
26
+ clientName,
24
27
  findable,
25
28
  authoredIds,
26
29
  handleLifetime,
@@ -37,6 +40,8 @@ export const MENU_RULES = [
37
40
  deprecated,
38
41
  cacheHints,
39
42
  authVariance,
43
+ listVariance,
44
+ listSchema,
40
45
  cacheScopePublic,
41
46
  serverInfo,
42
47
  ];
@@ -0,0 +1,16 @@
1
+ import { type ListKind, type ListRead } from '../lists.js';
2
+ import { type Rule } from './rule.js';
3
+ /** What snapshot read of the other lists, for these rules. */
4
+ export interface ListsContext {
5
+ /** Each list read twice on one connection, and once from a second process (stdio) or connection (HTTP). */
6
+ read: {
7
+ kind: ListKind;
8
+ first: ListRead;
9
+ second?: ListRead;
10
+ probe?: ListRead;
11
+ }[];
12
+ /** Lists asked for that the server declares no capability for. */
13
+ undeclared: ListKind[];
14
+ }
15
+ export declare const listVariance: Rule;
16
+ export declare const listSchema: Rule;
@@ -0,0 +1,102 @@
1
+ import { LIST_INFO, listChanges } from '../lists.js';
2
+ import { classifyFailure, SETUP_FAILURES } from '../failures.js';
3
+ import { count } from './rule.js';
4
+ import { resultValidator } from './spec.js';
5
+ const RESULT = { prompts: 'ListPromptsResult', resources: 'ListResourcesResult', resourceTemplates: 'ListResourceTemplatesResult' };
6
+ export const listVariance = {
7
+ id: 'lists/variance',
8
+ severity: 'warn',
9
+ summary: 'Prompts and resources lists are the same every time they are listed, and from a fresh process or connection',
10
+ run(ctx) {
11
+ const findings = [];
12
+ for (const r of ctx.lists?.read ?? []) {
13
+ const info = LIST_INFO[r.kind];
14
+ if (r.first.error)
15
+ continue;
16
+ const again = r.second && !r.second.error ? listChanges(r.kind, r.first.items, r.second.items) : [];
17
+ if (again.length) {
18
+ findings.push({
19
+ message: `${info.method} differs between two calls on one connection: ${count(again.length, 'change')}. A client that caches it (ttlMs) or compares it to decide whether to refresh sees a new list every time.`,
20
+ detail: again.slice(0, 8).concat(again.length > 8 ? [`…and ${again.length - 8} more`] : []),
21
+ fix: `Build ${info.method} the same way every time: a stable order and no per-call values (timestamps, counters).`,
22
+ });
23
+ }
24
+ const fresh = r.probe && !r.probe.error ? listChanges(r.kind, r.first.items, r.probe.items) : [];
25
+ if (fresh.length && !again.length) {
26
+ const where = ctx.transport === 'http' ? 'connection' : 'process';
27
+ findings.push({
28
+ message: `${info.method} from a second ${where} differs from the first: ${count(fresh.length, 'change')}. 2026-07-28 lists no longer vary per connection, so clients and caches treat one ${where}'s list as everyone's.`,
29
+ detail: fresh.slice(0, 8).concat(fresh.length > 8 ? [`…and ${fresh.length - 8} more`] : []),
30
+ fix: `Build ${info.method} from what's the same for every ${where}: no per-start ordering (a set, a hash map), no per-${where} state.`,
31
+ });
32
+ }
33
+ }
34
+ return findings;
35
+ },
36
+ };
37
+ export const listSchema = {
38
+ id: 'lists/schema',
39
+ severity: 'error',
40
+ summary: 'Prompts and resources list results match the official schema for their protocol version',
41
+ run(ctx) {
42
+ if (!ctx.protocolVersion)
43
+ return [];
44
+ const findings = [];
45
+ for (const r of ctx.lists?.read ?? []) {
46
+ const info = LIST_INFO[r.kind];
47
+ if (r.first.error) {
48
+ const said = r.first.error.replace(/[.\s]+$/, '');
49
+ const failure = classifyFailure(r.first.error, { code: r.first.code });
50
+ if (r.first.code === -32601 || /method not found/i.test(r.first.error)) {
51
+ // A server that declares resources but answers one of their methods with Method not found.
52
+ findings.push({
53
+ severity: 'info',
54
+ message: `The server declares ${info.capability} but answers ${info.method} with Method not found (-32601), so a client that asks for ${info.key === 'resourceTemplates' ? 'resource templates' : info.key} gets an error, not an empty list.`,
55
+ fix: `Answer ${info.method} with an empty list if there are none.`,
56
+ });
57
+ }
58
+ else if (SETUP_FAILURES.has(failure)) {
59
+ // Credentials, this machine, a missing app: a failure of the setup, not of the server.
60
+ findings.push({
61
+ severity: 'info',
62
+ message: `${info.method} couldn't be read here (${said}), so it wasn't checked: a setup problem, not the server's.`,
63
+ fix: `Rerun where ${info.method} can answer (credentials, the app or data it needs) to check it.`,
64
+ });
65
+ }
66
+ else {
67
+ findings.push({
68
+ severity: 'warn',
69
+ message: `The server declares ${info.capability}, but ${info.method} failed: ${said}. Clients that load ${info.key === 'resourceTemplates' ? 'resource templates' : info.key} up front get an error instead.`,
70
+ fix: `Answer ${info.method} (an empty list is fine), or stop declaring the ${info.capability} capability.`,
71
+ });
72
+ }
73
+ continue;
74
+ }
75
+ if (r.first.truncated) {
76
+ // Not a defect: said so that no one reads the checks as covering the whole list.
77
+ findings.push({
78
+ severity: 'info',
79
+ message: `${info.method} has more than ${r.first.pages.length} pages; toolmenu read the first ${r.first.pages.length} (${r.first.items.length} items), so the checks and the menu file cover those only.`,
80
+ fix: 'Nothing to change for this; a client that loads the whole list up front pays for every page.',
81
+ });
82
+ }
83
+ const validate = resultValidator(ctx.protocolVersion, RESULT[r.kind]);
84
+ if (!validate)
85
+ continue;
86
+ r.first.pages.forEach((page, i) => {
87
+ if (validate(page))
88
+ return;
89
+ const errors = validate.errors ?? [];
90
+ const missing = [...new Set(errors.filter((e) => !e.instancePath && e.params && 'missingProperty' in e.params).map((e) => String(e.params.missingProperty)))];
91
+ findings.push({
92
+ message: `The ${info.method} result${r.first.pages.length > 1 ? ` (page ${i + 1})` : ''} doesn't match the official ${ctx.protocolVersion} schema, so a strict client may refuse it.`,
93
+ detail: errors.slice(0, 5).map((e) => `${e.instancePath || '(result)'} ${e.message}${e.params && 'missingProperty' in e.params ? `: ${e.params.missingProperty}` : ''}`).concat(errors.length > 5 ? [`…and ${errors.length - 5} more`] : []),
94
+ fix: missing.length
95
+ ? `Add ${missing.join(', ')} to the ${info.method} result${missing.some((m) => m === 'ttlMs' || m === 'cacheScope') ? ` (${ctx.protocolVersion} requires cache hints on it, SEP-2549)` : ''}.`
96
+ : `Fix ${errors[0]?.instancePath || 'the result'} first (${errors[0]?.message ?? 'see above'}), then rerun.`,
97
+ });
98
+ });
99
+ }
100
+ return findings;
101
+ },
102
+ };
@@ -28,6 +28,10 @@ export interface RuleContext {
28
28
  server?: string;
29
29
  /** snapshot --as: each identity's menu, the main one first (menu/auth-variance, spec/cache-scope-public). */
30
30
  identities?: import('./identities.js').IdentityMenu[];
31
+ /** snapshot --lists: the prompts and resources lists read (lists/* rules). */
32
+ lists?: import('./lists.js').ListsContext;
33
+ /** --client: the client profile the menu is checked for (src/clients.ts). */
34
+ client?: string;
31
35
  /** Where clients are assumed to cut tool descriptions (description/buried). */
32
36
  descriptionLimit?: number | string;
33
37
  /** Tools (names or globs) the client sends uncut, so description/buried skips them. */
@@ -1,3 +1,4 @@
1
+ import { profileOf } from '../clients.js';
1
2
  import { buildIndex, placeIn, placeOf, SEARCH_LIMIT } from '../search.js';
2
3
  import { commonWords, VERBS, words } from '../words.js';
3
4
  import { clip, count, kept, names, num } from './rule.js';
@@ -19,9 +20,18 @@ export const findable = {
19
20
  summary: 'Each tool comes up when searched for by its own name, for clients that defer tools',
20
21
  run(ctx) {
21
22
  const tools = ctx.menu.tools;
22
- if (tools.length < MIN_TOOLS || ((ctx.menu.totalTokens ?? 0) < MIN_TOKENS && tools.length < 30))
23
+ const profile = profileOf(ctx.client);
24
+ if (profile) {
25
+ // A client that defers tools does it at any size; one whose search results
26
+ // aren't documented (OpenAI's spec doesn't say how many) isn't ranked against a guess.
27
+ if (profile.defers.value === 'none' || tools.length < 2)
28
+ return [];
29
+ if (!profile.searchLimit && ctx.client !== 'claude-code')
30
+ return [];
31
+ }
32
+ else if (tools.length < MIN_TOOLS || ((ctx.menu.totalTokens ?? 0) < MIN_TOKENS && tools.length < 30))
23
33
  return [];
24
- const index = buildIndex(tools);
34
+ const index = buildIndex(tools, { descriptionChars: profile?.searchReads?.value });
25
35
  const findings = [];
26
36
  // The words of a name that say something: not a prefix most tools share (firecrawl_, API-).
27
37
  const shared = commonWords(tools.map((t) => t.name));
@@ -43,7 +53,7 @@ export const findable = {
43
53
  findings.push({
44
54
  ...(missed.length === 1 ? { tool: first.tool } : {}),
45
55
  confidence: 'unsure',
46
- message: `${count(missed.length, 'tool')} of ${num(tools.length)} ${missed.length === 1 ? "doesn't" : "don't"} come up in the top ${SEARCH_LIMIT} when searched for by ${missed.length === 1 ? 'its' : 'their'} own name. Clients that defer a menu this size let the model search it, and a search returns ${SEARCH_LIMIT} tools unless the model asks for more, so an agent looking for ${missed.length === 1 ? 'it' : 'them'} can be shown other tools instead. Ranked the way a BM25 tool search ranks them (name, description, argument names and descriptions); a client's own search can rank a little differently.`,
56
+ message: `${count(missed.length, 'tool')} of ${num(tools.length)} ${missed.length === 1 ? "doesn't" : "don't"} come up in the top ${SEARCH_LIMIT} when searched for by ${missed.length === 1 ? 'its' : 'their'} own name. Clients that defer a menu this size let the model search it, and a search returns ${SEARCH_LIMIT} tools unless the model asks for more, so an agent looking for ${missed.length === 1 ? 'it' : 'them'} can be shown other tools instead. Ranked the way a BM25 tool search ranks them (name, description, argument names and descriptions${profile?.searchReads ? `, the description up to character ${num(profile.searchReads.value)}, all ${profile.label}'s search reads` : ''}); a client's own search can rank a little differently.`,
47
57
  detail: missed.slice(0, 12).map((m) => `${m.tool}: “${m.query}” puts it ${m.place === undefined ? 'nowhere' : `at ${m.place}`}, behind ${names(m.ahead.slice(0, 4), 4)}${m.ahead.length > 4 ? ` and ${num(m.ahead.length - 4)} more` : ''}`).concat(missed.length > 12 ? [`and ${num(missed.length - 12)} more`] : []),
48
58
  fix: 'Say the words of each tool\'s name in its own description, early on (a search ranks the description too, and "Returns the llms.txt" never says "guide"), give it a word the tools ahead of it lack (the object it acts on, not the verb), and cut repeats of other tools\' words from long descriptions: a description that mentions "domain" ten times outranks the tool named for domains.',
49
59
  });
@@ -1,4 +1,6 @@
1
1
  import type { Rule } from './rule.js';
2
+ /** A validator for one result type ($defs name) of a protocol version, or null when this toolmenu has no schema for it. */
3
+ export declare function resultValidator(version: string, result?: string): import("ajv").ValidateFunction<unknown> | null;
2
4
  export declare const schema: Rule;
3
5
  export declare const toolsCapability: Rule;
4
6
  export declare const duplicateName: Rule;
@@ -7,24 +7,33 @@ import addFormatsImport from 'ajv-formats';
7
7
  const addFormats = (addFormatsImport.default ?? addFormatsImport);
8
8
  const SCHEMA_DIR = fileURLToPath(new URL('../../schemas/', import.meta.url));
9
9
  const validators = new Map();
10
- function listToolsValidator(version) {
11
- if (!validators.has(version)) {
10
+ /** A validator for one result type ($defs name) of a protocol version, or null when this toolmenu has no schema for it. */
11
+ export function resultValidator(version, result = 'ListToolsResult') {
12
+ const key = `${version}\0${result}`;
13
+ if (!validators.has(key)) {
12
14
  const file = `${SCHEMA_DIR}${version}.json`;
13
15
  if (!existsSync(file)) {
14
- validators.set(version, null);
16
+ validators.set(key, null);
15
17
  }
16
18
  else {
17
19
  const schema = JSON.parse(readFileSync(file, 'utf8'));
18
20
  // 2025-11-25 and later are draft 2020-12 with $defs; earlier ones are draft-07 with definitions.
19
21
  const modernDraft = String(schema.$schema).includes('2020-12');
20
- const ajv = modernDraft ? new Ajv2020({ strict: false, allErrors: true }) : new Ajv({ strict: false, allErrors: true });
21
- addFormats(ajv);
22
- ajv.addSchema(schema, 'mcp');
23
- validators.set(version, ajv.compile({ $ref: `mcp#/${modernDraft ? '$defs' : 'definitions'}/ListToolsResult` }));
22
+ const pool = modernDraft ? schema.$defs : schema.definitions;
23
+ if (!pool?.[result]) {
24
+ validators.set(key, null);
25
+ }
26
+ else {
27
+ const ajv = modernDraft ? new Ajv2020({ strict: false, allErrors: true }) : new Ajv({ strict: false, allErrors: true });
28
+ addFormats(ajv);
29
+ ajv.addSchema(schema, 'mcp');
30
+ validators.set(key, ajv.compile({ $ref: `mcp#/${modernDraft ? '$defs' : 'definitions'}/${result}` }));
31
+ }
24
32
  }
25
33
  }
26
- return validators.get(version) ?? null;
34
+ return validators.get(key) ?? null;
27
35
  }
36
+ const listToolsValidator = (version) => resultValidator(version, 'ListToolsResult');
28
37
  export const schema = {
29
38
  id: 'spec/schema',
30
39
  severity: 'error',
package/dist/search.d.ts CHANGED
@@ -2,7 +2,7 @@ import type { MenuTool } from './types.js';
2
2
  /** How many tools a search returns when the model doesn't set a limit. */
3
3
  export declare const SEARCH_LIMIT = 5;
4
4
  /** The words a search sees in a tool: its name, description, and every argument's name and description. */
5
- export declare function searchText(tool: MenuTool): string[];
5
+ export declare function searchText(tool: MenuTool, descriptionChars?: number): string[];
6
6
  /** Lowercase, singular, without stop words: what a query and a document are compared on. */
7
7
  export declare function terms(text: string): string[];
8
8
  export interface SearchIndex {
@@ -12,7 +12,9 @@ export interface SearchIndex {
12
12
  score: number;
13
13
  }[];
14
14
  }
15
- export declare function buildIndex(tools: MenuTool[]): SearchIndex;
15
+ export declare function buildIndex(tools: MenuTool[], options?: {
16
+ descriptionChars?: number;
17
+ }): SearchIndex;
16
18
  /**
17
19
  * Where a tool lands for a query: 1 is first. A tie counts in the tool's favour
18
20
  * (only tools that score strictly higher are ahead of it), so a miss is never a
package/dist/search.js CHANGED
@@ -9,8 +9,10 @@ const STOP = new Set([
9
9
  const K1 = 1.2;
10
10
  const B = 0.75;
11
11
  /** The words a search sees in a tool: its name, description, and every argument's name and description. */
12
- export function searchText(tool) {
13
- const parts = [tool.name, tool.description ?? ''];
12
+ export function searchText(tool, descriptionChars) {
13
+ // A client whose search reads only the start of a description (Claude Code: the first 4,096 characters).
14
+ const description = tool.description ?? '';
15
+ const parts = [tool.name, descriptionChars !== undefined ? description.slice(0, descriptionChars) : description];
14
16
  const walk = (schema, depth) => {
15
17
  if (!schema || typeof schema !== 'object' || depth > 6)
16
18
  return;
@@ -34,10 +36,10 @@ export function terms(text) {
34
36
  .map(singular)
35
37
  .filter((w) => w.length > 1 && !STOP.has(w));
36
38
  }
37
- export function buildIndex(tools) {
39
+ export function buildIndex(tools, options = {}) {
38
40
  const docs = tools.map((t) => {
39
41
  const tf = new Map();
40
- const text = searchText(t);
42
+ const text = searchText(t, options.descriptionChars);
41
43
  for (const w of text)
42
44
  tf.set(w, (tf.get(w) ?? 0) + 1);
43
45
  return { name: t.name, tf, length: text.length };
@@ -1,6 +1,8 @@
1
1
  import { type Target } from './connect.js';
2
2
  import { type Identity } from './probe.js';
3
+ import { type ListKind } from './lists.js';
3
4
  import type { IdentityMenu } from './rules/identities.js';
5
+ import { type ClientCost } from './clients.js';
4
6
  import { type RuleSettings } from './rules/index.js';
5
7
  import { type CatalogOptions } from './catalog.js';
6
8
  import type { Routes } from './routes.js';
@@ -22,12 +24,20 @@ export interface SnapshotOptions extends RuleSettings {
22
24
  * menu is the one saved and checked; the others are compared with it.
23
25
  */
24
26
  identities?: Identity[];
27
+ /** --lists: which lists besides tools to read and check (prompts, resources). */
28
+ lists?: string[];
29
+ /** --client: check the menu for this client (src/clients.ts): its description cut, its search, its name format. */
30
+ client?: string;
25
31
  }
26
32
  export interface SnapshotResult {
27
33
  menu: Menu;
28
34
  findings: Finding[];
29
35
  /** snapshot --as: each identity's tools and tokens, or why it couldn't be listed. */
30
36
  identities?: IdentityMenu[];
37
+ /** --client: what the menu costs up front and per tool a search loads. */
38
+ cost?: ClientCost;
39
+ /** --lists: lists asked for that the server declares no capability for, so weren't read. */
40
+ notOffered?: ListKind[];
31
41
  }
32
42
  /** Establish the menu: connect, list the tools twice, run the menu rules. */
33
43
  export declare function snapshot(given: Target, options?: SnapshotOptions): Promise<SnapshotResult>;
package/dist/snapshot.js CHANGED
@@ -2,7 +2,9 @@ import { listTools } from './connect.js';
2
2
  import { patiently, tooMany } from './failures.js';
3
3
  import { ConnectError } from './explain.js';
4
4
  import { buildMenu } from './menu.js';
5
- import { asIdentity, connectPatiently, isContainerWrapper, MAIN_SEED, probeMenu, probeVariance, seeded } from './probe.js';
5
+ import { asIdentity, connectPatiently, isContainerWrapper, MAIN_SEED, PROBE_SEED, probeMenu, probeVariance, seeded } from './probe.js';
6
+ import { kindsToRead, readList } from './lists.js';
7
+ import { clientCost } from './clients.js';
6
8
  import { MENU_RULES, runRules } from './rules/index.js';
7
9
  import { catalogFindings, readCatalog } from './catalog.js';
8
10
  import { SEVERITY_RANK } from './types.js';
@@ -17,6 +19,8 @@ export async function snapshot(given, options = {}) {
17
19
  let menu;
18
20
  let secondList;
19
21
  let first;
22
+ const reads = [];
23
+ let undeclared = [];
20
24
  try {
21
25
  first = await patiently(() => listTools(connection, options), { error: tooMany });
22
26
  const second = await patiently(() => listTools(connection, options), { error: tooMany });
@@ -28,6 +32,16 @@ export async function snapshot(given, options = {}) {
28
32
  };
29
33
  menu = buildMenu(first.tools, server, first.listMeta);
30
34
  secondList = buildMenu(second.tools, server, second.listMeta).tools;
35
+ // --lists: prompts and resources, each read twice on this connection.
36
+ const kinds = kindsToRead(options.lists ?? [], connection.capabilities);
37
+ undeclared = kinds.undeclared;
38
+ for (const kind of kinds.read) {
39
+ const first = await readList(connection, kind, options.timeoutMs);
40
+ const second = first.error ? undefined : await readList(connection, kind, options.timeoutMs);
41
+ reads.push({ kind, first, ...(second ? { second } : {}) });
42
+ if (!first.error)
43
+ menu[kind] = first.items;
44
+ }
31
45
  if (options.catalog) {
32
46
  menu.catalog = await readCatalog(connection, menu.tools, { ...(typeof options.catalog === 'object' ? options.catalog : {}), timeoutMs: options.timeoutMs });
33
47
  }
@@ -45,6 +59,23 @@ export async function snapshot(given, options = {}) {
45
59
  // After the main process has exited: a server that holds a file or a port
46
60
  // shouldn't have to share it with its own probe.
47
61
  const probes = await probeVariance(target, menu.tools, options.processes ?? 2, options.timeoutMs ?? 30_000);
62
+ // The other lists from a second process (stdio) or connection (HTTP), as tools get.
63
+ if ((options.processes ?? 2) > 1 && reads.some((r) => !r.first.error)) {
64
+ try {
65
+ const probe = await connectPatiently(seeded(target, PROBE_SEED), { timeoutMs: options.timeoutMs });
66
+ try {
67
+ for (const r of reads)
68
+ if (!r.first.error)
69
+ r.probe = await readList(probe, r.kind, options.timeoutMs);
70
+ }
71
+ finally {
72
+ await probe.close().catch(() => { });
73
+ }
74
+ }
75
+ catch {
76
+ // The tool probe above says why a second process or connection failed.
77
+ }
78
+ }
48
79
  // Each other identity's menu, from its own process (stdio) or connection (HTTP).
49
80
  const identities = mainIdentity
50
81
  ? [{ name: mainIdentity.name, tools: menu.tools, tokens: menu.totalTokens, cacheScope: menu.listMeta?.cacheScope }]
@@ -68,7 +99,10 @@ export async function snapshot(given, options = {}) {
68
99
  capabilities: connection.capabilities,
69
100
  usedAuth: connection.usedAuth,
70
101
  routes: options.routes,
71
- descriptionLimit: options.descriptionLimit,
102
+ // A client profile's cut, unless descriptionLimit says otherwise. A client whose docs
103
+ // give no cut (Claude's API, OpenAI) isn't checked against Claude Code's.
104
+ descriptionLimit: options.descriptionLimit ?? (options.client === undefined ? undefined : options.client === 'claude-code' ? 'claude-code' : Number.POSITIVE_INFINITY),
105
+ client: options.client,
72
106
  fullDescriptions: options.fullDescriptions,
73
107
  transport: target.kind,
74
108
  probes,
@@ -76,6 +110,7 @@ export async function snapshot(given, options = {}) {
76
110
  serverInfo: connection.server,
77
111
  server: serverArgs(target),
78
112
  identities,
113
+ ...(reads.length || undeclared.length ? { lists: { read: reads, undeclared } } : {}),
79
114
  }, options);
80
115
  // What --catalog called and what it got, under the same settings as the rules
81
116
  // (rules: { "catalog/read": "off" }, ignore).
@@ -89,7 +124,8 @@ export async function snapshot(given, options = {}) {
89
124
  }
90
125
  findings.sort((a, b) => SEVERITY_RANK[b.severity] - SEVERITY_RANK[a.severity]);
91
126
  }
92
- return { menu, findings, ...(identities ? { identities } : {}) };
127
+ const cost = options.client ? clientCost(options.client, menu) : undefined;
128
+ return { menu, findings, ...(identities ? { identities } : {}), ...(cost ? { cost } : {}), ...(undeclared.length ? { notOffered: undeclared } : {}) };
93
129
  }
94
130
  const firstLine = (error) => (error instanceof Error ? error.message : String(error)).split('\n')[0].trim();
95
131
  /**
package/dist/types.d.ts CHANGED
@@ -62,6 +62,10 @@ export interface Menu {
62
62
  ttlMs?: number;
63
63
  cacheScope?: string;
64
64
  };
65
+ /** snapshot --lists: the other lists as the server sent them (additive; readers that don't know them ignore them). */
66
+ prompts?: Record<string, unknown>[];
67
+ resources?: Record<string, unknown>[];
68
+ resourceTemplates?: Record<string, unknown>[];
65
69
  /**
66
70
  * 'session': every tool a session saw (session --union-out), in the order first
67
71
  * seen, which is the unlock order, not an order any client was served.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toolmenu",
3
- "version": "0.17.1",
3
+ "version": "0.20.0",
4
4
  "description": "Lint your MCP server's tool menu for changes that confuse agents or break caches.",
5
5
  "author": "Nikša Perović (https://niksa.me)",
6
6
  "main": "dist/index.js",