toolmenu 0.21.0 → 0.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/README.md +68 -37
  2. package/dist/apps.d.ts +56 -0
  3. package/dist/apps.js +99 -0
  4. package/dist/auto.js +2 -2
  5. package/dist/card.d.ts +18 -0
  6. package/dist/card.js +39 -0
  7. package/dist/cli.js +37 -10
  8. package/dist/colour.d.ts +8 -0
  9. package/dist/colour.js +70 -0
  10. package/dist/config.d.ts +2 -0
  11. package/dist/config.js +2 -0
  12. package/dist/connect.d.ts +6 -0
  13. package/dist/connect.js +22 -6
  14. package/dist/diff.js +111 -3
  15. package/dist/failures.d.ts +18 -5
  16. package/dist/failures.js +27 -12
  17. package/dist/help.js +3 -2
  18. package/dist/mrtr.d.ts +9 -1
  19. package/dist/mrtr.js +17 -0
  20. package/dist/options.d.ts +3 -0
  21. package/dist/options.js +1 -0
  22. package/dist/probe.d.ts +11 -1
  23. package/dist/probe.js +18 -3
  24. package/dist/report.js +12 -5
  25. package/dist/rule-docs.json +93 -3
  26. package/dist/rules/apps.d.ts +11 -0
  27. package/dist/rules/apps.js +156 -0
  28. package/dist/rules/capabilities.d.ts +21 -0
  29. package/dist/rules/capabilities.js +103 -0
  30. package/dist/rules/card.d.ts +7 -0
  31. package/dist/rules/card.js +65 -0
  32. package/dist/rules/clients.js +1 -0
  33. package/dist/rules/description.js +3 -0
  34. package/dist/rules/ids.js +35 -1
  35. package/dist/rules/index.js +18 -2
  36. package/dist/rules/naming.js +3 -0
  37. package/dist/rules/network.d.ts +21 -0
  38. package/dist/rules/network.js +63 -0
  39. package/dist/rules/rule.d.ts +18 -0
  40. package/dist/rules/search.js +1 -0
  41. package/dist/rules/spec.d.ts +9 -1
  42. package/dist/rules/spec.js +117 -3
  43. package/dist/rules/write.d.ts +0 -6
  44. package/dist/rules/write.js +11 -5
  45. package/dist/session.js +60 -11
  46. package/dist/snapshot.d.ts +5 -0
  47. package/dist/snapshot.js +89 -5
  48. package/dist/types.d.ts +5 -0
  49. package/dist/words.d.ts +8 -0
  50. package/dist/words.js +30 -2
  51. package/package.json +1 -1
  52. package/schemas/server-card.json +260 -0
package/README.md CHANGED
@@ -3,46 +3,69 @@
3
3
  [![npm](https://img.shields.io/npm/v/toolmenu)](https://www.npmjs.com/package/toolmenu)
4
4
  [![ci](https://github.com/niksa90/toolmenu/actions/workflows/ci.yml/badge.svg)](https://github.com/niksa90/toolmenu/actions/workflows/ci.yml)
5
5
  [![license: MIT](https://img.shields.io/badge/license-MIT-blue)](https://github.com/niksa90/toolmenu/blob/main/LICENSE)
6
+ [![MCP: 2026-07-28 and 2025](https://img.shields.io/badge/MCP-2026--07--28%20%C2%B7%202025-6E56CF)](https://modelcontextprotocol.io)
6
7
 
7
- **Lint your MCP server's tool menu for changes that confuse agents or break caches.**
8
-
9
- The MCP linters I found grade one snapshot of the menu an agent reads: tool names,
10
- descriptions, schemas. toolmenu observes how the menu behaves across calls,
11
- connections and releases, and checks **when** it changes:
12
-
13
- - **Mid-session.** A menu that changes after the conversation has started can
14
- invalidate the cached conversation: most clients put the tool list at the start of
15
- the prompt, so any change to it, even a tool added at the end, means everything
16
- after it is processed again. In my logs, one mid-conversation unlock rewrote
17
- 40–57K tokens of cached prefix (from my own logs; more on this, with a benchmark
18
- of the setups, in [this write-up](https://niksa.me/agent-tools-design-lessons)). `session` catches it, pins it to the step that
19
- caused it, and says where in the list it happened.
20
- - **Ready for 2026-07-28.** That protocol version says the tool set **must not** vary
21
- per connection or as a side effect of other requests (the 2025 versions allow it).
22
- Per-session "unlock" designs, like the one I built, are ruled out. `session` tells you whether yours is one of them.
23
- It also checks the menu clients will actually show: a broken `x-mcp-header` or a
24
- `$ref` they can't resolve hides a tool or fails every call to it. And when a tool
25
- stops to ask the user something, `session` answers "no" and says whether the tool
26
- can take it. Handles a server mints (`session_id`) should say how long they last and
27
- fail readably when unknown; menus that differ by who asks (`--as`) shouldn't be
28
- cached as public; prompts and resources (`--lists`) are checked like tools. Run
29
- against the reference server, `@modelcontextprotocol/server-everything` 2.1.0, with
30
- no false findings ([bench/](bench/README.md)).
31
- - **For your client.** `--client claude-code | claude-api-search | openai` checks the
32
- menu the way that client loads it, from its own docs: what it costs up front and per
33
- tool a search loads, where it cuts descriptions, what its search can find, and names
34
- its API refuses.
35
- - **Between releases.** `diff` finds breaking changes and token growth, and checks
36
- the version bump.
8
+ **Checks for MCP servers' tool lists: changes that confuse agents, break prompt caches or break clients, caught before your users hit them.**
37
9
 
38
- ![toolmenu session output: tools inserted mid-list, a description edited, each pinned to the step that caused it](https://raw.githubusercontent.com/niksa90/toolmenu/main/docs/demo.svg)
10
+ Your MCP server's tool list is the menu every agent reads before it does anything.
11
+ toolmenu checks that menu the way clients and agents will meet it: across calls,
12
+ connections, clients and releases. No LLM is involved, so the same inputs always give
13
+ the same answer and it fits in CI.
14
+
15
+ ```sh
16
+ npx toolmenu snapshot -- node dist/server.js # or: npx toolmenu snapshot https://example.com/mcp
17
+ npx toolmenu init -- node dist/server.js # CI: a baseline, a workflow, a comment on every PR
18
+ ```
39
19
 
40
- No LLM anywhere: the same inputs give the same answer, so it can sit in CI.
41
- (`history` installs old versions with today's dependencies, so it records what they
42
- serve now, which can differ from what they shipped with: FINDINGS F4.)
20
+ ![toolmenu session output: tools inserted mid-list, a description edited, each pinned to the step that caused it](https://raw.githubusercontent.com/niksa90/toolmenu/main/docs/demo.svg)
43
21
 
44
- > **Status: 0.16, early.** Spec in [docs/SPEC.md](https://github.com/niksa90/toolmenu/blob/main/docs/SPEC.md). What it has found on
45
- > real servers: [docs/FINDINGS.md](https://github.com/niksa90/toolmenu/blob/main/docs/FINDINGS.md).
22
+ ## What it catches
23
+
24
+ | | |
25
+ |---|---|
26
+ | **A menu that changes mid-session** | `session` calls your tools and watches the list. A tool added mid-conversation can rewrite the whole cached prompt: one unlock cost 40–57K tokens of cache in my logs. |
27
+ | **Breaking releases** | `diff` compares two menus: removed tools, new required parameters, narrowed types, token growth. It also says whether your version bump is big enough. |
28
+ | **The 2026-07-28 protocol** | The tool set must not vary per connection, and results need `resultType`. Elicitation, multi round-trip input and `server/discover` are checked too, as are the 2025 handshakes. |
29
+ | **MCP Apps, Tasks, Server Cards** | It reads `ui://` resources as a host does and checks who may see each tool. It catches tasks sent to clients that can't poll them, and Server Cards that contradict the live server. |
30
+ | **Your client's view** | `--client claude-code \| claude-api-search \| openai` shows what each client loads, cuts, can find and refuses. |
31
+ | **What agents misread** | It flags vague `id`s, IDs no tool returns, instructions buried late in descriptions, and writes without annotations or marked read-only. |
32
+ | **Menus that drift** | It catches defaults built from the clock or a hash order, menus that differ per process, per connection or offline, and listings that depend on who asks. |
33
+
34
+ Every finding says what's wrong, where it is and what toolmenu saw, and ends with a `→ Next:` step.
35
+ It runs on the 22-server [bench](bench/README.md) and on hosted servers (GitHub, Atlassian, Sentry,
36
+ Notion, Supabase, Vercel…). What it has found on real servers is in
37
+ [docs/FINDINGS.md](https://github.com/niksa90/toolmenu/blob/main/docs/FINDINGS.md).
38
+
39
+ > **Status: 0.23.** Spec in [docs/SPEC.md](https://github.com/niksa90/toolmenu/blob/main/docs/SPEC.md),
40
+ > every rule in [docs/RULES.md](docs/RULES.md), changes in [CHANGELOG.md](CHANGELOG.md).
41
+
42
+ <details>
43
+ <summary><b>Why "when it changes" matters</b></summary>
44
+
45
+ The MCP linters I found grade one snapshot of the menu: tool names, descriptions,
46
+ schemas. toolmenu also checks **when** the menu changes:
47
+
48
+ - **Mid-session.** Most clients put the tool list at the start of the prompt, so any
49
+ change to it, even a tool added at the end, means everything after it is processed
50
+ again. More on this, with a benchmark of the setups, in
51
+ [this write-up](https://niksa.me/agent-tools-design-lessons). `session` pins each
52
+ change to the step that caused it, and says where in the list it happened.
53
+ - **Per connection.** 2026-07-28 says the tool set **must not** vary per connection or
54
+ as a side effect of other requests (the 2025 versions allow it). Per-session
55
+ "unlock" designs, like the one I built, are ruled out. `session` tells you whether
56
+ yours is one of them. It also checks the menu clients will actually show: a broken
57
+ `x-mcp-header` or a `$ref` they can't resolve hides a tool or fails every call to
58
+ it. When a tool stops to ask the user something, `session` answers "no" and says
59
+ whether the tool can take it.
60
+ - **Between releases.** `diff` finds breaking changes and token growth, and checks the
61
+ version bump. `history` does it across a package's published releases. It installs
62
+ old versions with today's dependencies, so it records what they serve now
63
+ (FINDINGS F4).
64
+
65
+ Run against the reference server, `@modelcontextprotocol/server-everything`, it
66
+ reports no false findings ([bench/](bench/README.md)).
67
+
68
+ </details>
46
69
 
47
70
  ## Start here
48
71
 
@@ -235,6 +258,8 @@ serves, and 23 issue and pull-request tools that only the dynamic mode has.
235
258
  | `session/nothing-called` | warn | `--auto` called no tools (all open-world, missing values, or not read-only): the run only listed the menu |
236
259
  | `session/session-lost` | error | The server ended the session after toolmenu opened a second connection with the same credentials (servers with one session per client; 2025-era servers only). The run stops there; rerun with `--processes 1` |
237
260
  | `session/handle-error` | warn | 2026-07-28: `--auto` called a read-only tool with a handle that can't exist (`toolmenu-unknown-session`), and it answered with a protocol error instead of a tool error the model can read |
261
+ | `session/error-not-flagged` | warn | A tool answered with an HTTP error from the API behind it (`{"status":401,"object":"error",…}`) as a plain result, not marked `isError`, so a client that reads `isError` takes the call for a success (Notion's server). An auth one also counts toward `session/untested`. Tools that take a URL are left out: their answer is the page's (heuristic) |
262
+ | `session/server-unresponsive` | error | The server stopped answering after a step: even `tools/list` timed out. The run stops there with this one finding, instead of every later step waiting out its timeout (a server that polls a device-code sign-in inside a tool call) |
238
263
  | `session/handle-roundtrip` | warn | With `--create-handles`: the tool that creates a session, basket or cart was called, and the tools that take it back refused the handle it had just returned (info when its result had no handle to find) |
239
264
  | `session/rate-limited` | error | The server kept refusing requests for being too many (a 429) after toolmenu waited about a minute, or a tool not marked `readOnlyHint` said it was rate-limited (it isn't called again). The run stops there, with the steps it didn't check; give the run its own server instance or raise the limit for it |
240
265
  | `session/scope-unchecked` | info | A second server process or connection couldn't start mid-session (a server that holds a file or a port), so whether a change was global wasn't checked. Said once for the run |
@@ -450,6 +475,8 @@ server, or from the spec itself. If a rule can't point to one, it doesn't ship.
450
475
  | `naming/client-name` | warn | With `--client`: tool names the client's API refuses: `^[a-zA-Z0-9_-]{1,128}$` for the Claude API, `^[a-zA-Z0-9_-]+$` up to 128 for OpenAI's functions. MCP allows names such as `files.read` that neither takes as they are |
451
476
  | `menu/auth-variance` | info | With `--as` (or `identities` in the config): the tool set differs by identity, what each adds, lacks or defines differently against the first, and its tokens. Allowed by the spec; a baseline of one identity's menu doesn't cover the others. An identity that can't be listed is said here too |
452
477
  | `spec/cache-scope-public` | warn | 2026-07-28, with `--as`: the tool set differs by identity, and `tools/list` says `cacheScope: "public"`, which lets shared caches serve one caller's list to another |
478
+ | `menu/capability-dependent` | info | Tools a server lists only to a client that supports elicitation, or only to one that doesn't. `snapshot` lists as a client with elicitation, so the menu file holds those tools (github-mcp-server 1.14 lists `delete_repository` only to a client that can ask the user to confirm; before 0.22 a baseline missed it), and once more without |
479
+ | `menu/network-dependent` | warn | With `--offline-check` (stdio): the menu listed once more with the network cut off (the proxy variables pointed at a closed port, loopback left alone) differs: inite-visibility serves 3 tools offline and 10 online, so a baseline made with the network doesn't match a machine without it. A server that doesn't start offline is info. A server that ignores the proxy variables still reaches the network where the machine has it directly; then nothing is reported, which isn't a pass |
453
480
  | `auth/invalid-accepted` | warn | HTTP with credentials: the menu listed once more with an invalid credential in the same scheme. A server that serves a menu to it, not 401, gives a client whose token expired no reason to log in again; the agent just has fewer tools (Atlassian Rovo served 4 tools to any `Authorization` header). `--processes 1` turns it off |
454
481
 
455
482
  The full list, `diff` and `session` rules included, is [docs/RULES.md](docs/RULES.md).
@@ -483,10 +510,13 @@ meant to steer away from, and toolmenu says so.
483
510
  "routes": "routes.yml",
484
511
  "tokenBudget": 4000,
485
512
  "descriptionLimit": "claude-code",
486
- "fullDescriptions": ["search_capabilities", "lookup_*"]
513
+ "fullDescriptions": ["search_capabilities", "lookup_*"],
514
+ "offlineCheck": true
487
515
  }
488
516
  ```
489
517
 
518
+ `offlineCheck` turns on `--offline-check` for every `snapshot` of a stdio server.
519
+
490
520
  `descriptionLimit` is where your client cuts tool descriptions: a number, or a client
491
521
  whose cut has been reported: `"claude-code"` (the default: Claude Code's own docs,
492
522
  [code.claude.com/docs/en/mcp](https://code.claude.com/docs/en/mcp), read 2026-11-02:
@@ -556,6 +586,7 @@ toolmenu sees what the server sends. Plenty of agent failures happen elsewhere:
556
586
  --as <name>:<cred> snapshot: list the menu as this identity too, repeatable:
557
587
  "reader:Authorization: Bearer …" (HTTP) or "admin:ROLE=admin"
558
588
  (stdio). The first is the menu saved; the others are compared
589
+ --offline-check snapshot (stdio): also list the menu with the network cut off
559
590
  --concurrency <n> scan: servers checked at a time (default: 4)
560
591
  --out-dir <dir> scan: write each server's menu file there, as <name>.json
561
592
  ```
package/dist/apps.d.ts ADDED
@@ -0,0 +1,56 @@
1
+ import type { Connection } from './connect.js';
2
+ import type { Menu, MenuTool } from './types.js';
3
+ /** The deprecated flat key, before `_meta.ui` ("will be removed before GA"). */
4
+ export declare const LEGACY_URI_KEY = "ui/resourceUri";
5
+ export interface UiMeta {
6
+ /** `_meta.ui.resourceUri`. */
7
+ resourceUri?: unknown;
8
+ /** `_meta["ui/resourceUri"]`, the deprecated spelling. */
9
+ legacyUri?: unknown;
10
+ /** `_meta.ui.visibility`, as sent. */
11
+ visibility?: unknown;
12
+ }
13
+ /** The tool's MCP Apps metadata, or undefined when it has none. */
14
+ export declare function uiMeta(tool: object): UiMeta | undefined;
15
+ /** The UI resource the host renders for this tool: `_meta.ui.resourceUri`, else the deprecated key. */
16
+ export declare function resourceUriOf(tool: object): string | undefined;
17
+ /**
18
+ * Listed for the app only: `visibility` is a list without "model". An Apps host MUST
19
+ * NOT put such a tool in the agent's tool list, so the agent never reads its name,
20
+ * description or schema. Absent, visibility is ["model", "app"].
21
+ */
22
+ export declare function isAppOnly(tool: object): boolean;
23
+ /** The tools the agent is shown: all of them but the app-only ones. */
24
+ export declare function agentTools<T extends object>(tools: T[]): T[];
25
+ /**
26
+ * The menu the agent is shown, its token total recomputed, and what was left out: one
27
+ * view for the report, the agent-scope rules and the --client cost.
28
+ */
29
+ export declare function agentMenu(menu: Menu): {
30
+ shown: Menu;
31
+ appOnly?: {
32
+ tools: number;
33
+ tokens: number;
34
+ };
35
+ };
36
+ /** What resources/read gave for one UI resource. */
37
+ export interface AppResource {
38
+ uri: string;
39
+ /** The tools that name it. */
40
+ tools: string[];
41
+ /** resources/read failed: the error's first line. */
42
+ error?: string;
43
+ /** Each content item's mimeType, in order ("" when absent). */
44
+ mimeTypes?: string[];
45
+ /** Some content item for this URI carries text or a blob. */
46
+ hasContent?: boolean;
47
+ /** The content items' URIs that differ from the one asked for. */
48
+ otherUris?: string[];
49
+ }
50
+ /**
51
+ * Read each `ui://` resource the menu's tools name (resources/read: a read, not a tool
52
+ * call), at most `limit` of them. Each is read once, however many tools name it.
53
+ */
54
+ export declare function readAppResources(connection: Connection, tools: MenuTool[], timeoutMs?: number, limit?: number): Promise<AppResource[]>;
55
+ /** Whether a content type is the one MCP Apps allow, compared without case or spaces around `;`. */
56
+ export declare function isAppsMimeType(mime: string): boolean;
package/dist/apps.js ADDED
@@ -0,0 +1,99 @@
1
+ import { APPS_MIME_TYPE } from './connect.js';
2
+ /** The deprecated flat key, before `_meta.ui` ("will be removed before GA"). */
3
+ export const LEGACY_URI_KEY = 'ui/resourceUri';
4
+ const isObject = (v) => typeof v === 'object' && v !== null && !Array.isArray(v);
5
+ /** The tool's MCP Apps metadata, or undefined when it has none. */
6
+ export function uiMeta(tool) {
7
+ const raw = tool._meta;
8
+ const meta = isObject(raw) ? raw : undefined;
9
+ if (!meta)
10
+ return undefined;
11
+ const ui = isObject(meta.ui) ? meta.ui : undefined;
12
+ const out = {
13
+ ...(ui && 'resourceUri' in ui ? { resourceUri: ui.resourceUri } : {}),
14
+ ...(LEGACY_URI_KEY in meta ? { legacyUri: meta[LEGACY_URI_KEY] } : {}),
15
+ ...(ui && 'visibility' in ui ? { visibility: ui.visibility } : {}),
16
+ };
17
+ return Object.keys(out).length ? out : undefined;
18
+ }
19
+ /** The UI resource the host renders for this tool: `_meta.ui.resourceUri`, else the deprecated key. */
20
+ export function resourceUriOf(tool) {
21
+ const ui = uiMeta(tool);
22
+ // An empty _meta.ui.resourceUri names nothing: the deprecated key, when it has one, is what's left.
23
+ const uri = typeof ui?.resourceUri === 'string' && ui.resourceUri ? ui.resourceUri : typeof ui?.legacyUri === 'string' ? ui.legacyUri : undefined;
24
+ return uri || undefined;
25
+ }
26
+ /**
27
+ * Listed for the app only: `visibility` is a list without "model". An Apps host MUST
28
+ * NOT put such a tool in the agent's tool list, so the agent never reads its name,
29
+ * description or schema. Absent, visibility is ["model", "app"].
30
+ */
31
+ export function isAppOnly(tool) {
32
+ const v = uiMeta(tool)?.visibility;
33
+ return Array.isArray(v) && !v.includes('model');
34
+ }
35
+ /** The tools the agent is shown: all of them but the app-only ones. */
36
+ export function agentTools(tools) {
37
+ return tools.some(isAppOnly) ? tools.filter((t) => !isAppOnly(t)) : tools;
38
+ }
39
+ /**
40
+ * The menu the agent is shown, its token total recomputed, and what was left out: one
41
+ * view for the report, the agent-scope rules and the --client cost.
42
+ */
43
+ export function agentMenu(menu) {
44
+ const tools = agentTools(menu.tools);
45
+ if (tools === menu.tools)
46
+ return { shown: menu };
47
+ const tokens = tools.reduce((n, t) => n + t.tokens, 0);
48
+ return { shown: { ...menu, tools, totalTokens: tokens }, appOnly: { tools: menu.tools.length - tools.length, tokens: menu.totalTokens - tokens } };
49
+ }
50
+ /**
51
+ * Read each `ui://` resource the menu's tools name (resources/read: a read, not a tool
52
+ * call), at most `limit` of them. Each is read once, however many tools name it.
53
+ */
54
+ export async function readAppResources(connection, tools, timeoutMs = 30_000, limit = 50) {
55
+ const byUri = new Map();
56
+ for (const t of tools) {
57
+ const uri = resourceUriOf(t);
58
+ if (uri && uri.startsWith('ui://'))
59
+ byUri.set(uri, [...(byUri.get(uri) ?? []), t.name]);
60
+ }
61
+ const queue = [...byUri].slice(0, limit);
62
+ const out = new Array(queue.length);
63
+ // All of them within 2 × the request timeout, 4 at a time: a server whose reads hang
64
+ // costs one bounded wait, not one timeout per resource.
65
+ const deadline = Date.now() + 2 * timeoutMs;
66
+ const readOne = async (i) => {
67
+ const [uri, names] = queue[i];
68
+ const left = deadline - Date.now();
69
+ if (left <= 0) {
70
+ out[i] = { uri, tools: names, error: `not read: the reads took longer than ${Math.round((2 * timeoutMs) / 1000)} s in all` };
71
+ return;
72
+ }
73
+ try {
74
+ const result = await connection.client.readResource({ uri }, { timeout: Math.min(timeoutMs, left) });
75
+ const contents = Array.isArray(result?.contents) ? result.contents : [];
76
+ out[i] = {
77
+ uri,
78
+ tools: names,
79
+ mimeTypes: contents.map((c) => (typeof c.mimeType === 'string' ? c.mimeType : '')),
80
+ hasContent: contents.some((c) => (typeof c.text === 'string' && c.text.length > 0) || (typeof c.blob === 'string' && c.blob.length > 0)),
81
+ otherUris: contents.flatMap((c) => (typeof c.uri === 'string' && c.uri !== uri ? [c.uri] : [])),
82
+ };
83
+ }
84
+ catch (error) {
85
+ out[i] = { uri, tools: names, error: (error instanceof Error ? error.message : String(error)).split('\n')[0].trim() };
86
+ }
87
+ };
88
+ let next = 0;
89
+ const worker = async () => {
90
+ while (next < queue.length)
91
+ await readOne(next++);
92
+ };
93
+ await Promise.all(Array.from({ length: Math.min(4, queue.length) }, worker));
94
+ return out.filter((r) => r !== undefined);
95
+ }
96
+ /** Whether a content type is the one MCP Apps allow, compared without case or spaces around `;`. */
97
+ export function isAppsMimeType(mime) {
98
+ return mime.toLowerCase().replace(/\s*;\s*/g, ';') === APPS_MIME_TYPE;
99
+ }
package/dist/auto.js CHANGED
@@ -3,7 +3,7 @@ import { isMap, isNode, isScalar, parseDocument } from 'yaml';
3
3
  import { synthesizeArgs } from './args.js';
4
4
  import { CREATABLE_KINDS, createdKinds, creatorOf, handleParams } from './rules/handles.js';
5
5
  import { MAX_UNLOCKS, unlockers } from './session.js';
6
- import { COLLECTION_VERBS, commonWords, leadingVerb, LOOKUP_VERBS, verbOf, WRITE_VERBS } from './words.js';
6
+ import { COLLECTION_VERBS, prefixWords, leadingVerb, LOOKUP_VERBS, verbOf, WRITE_VERBS } from './words.js';
7
7
  /** Marked or named as a write: never called on the user's word. Says why, or undefined. */
8
8
  export function writeSign(tool) {
9
9
  const a = tool.annotations ?? {};
@@ -44,7 +44,7 @@ export function autoScenario(menu, options = {}) {
44
44
  }
45
45
  // The server's readOnlyHint, but not against a name that says it writes: a hosted
46
46
  // server marked create_my_ideal_account read-only (write/read-only-mismatch).
47
- const common = commonWords(menu.tools.map((t) => t.name));
47
+ const common = prefixWords(menu.tools.map((t) => t.name));
48
48
  const namedWrite = (t) => WRITE_VERBS.has(leadingVerb(t.name, common) ?? '');
49
49
  const readOnly = (t) => (t.annotations?.readOnlyHint === true && !namedWrite(t)) || assumed.has(t.name);
50
50
  const { perTool, ignored: unusedValues } = resolveValues(menu, options.values ?? {});
package/dist/card.d.ts ADDED
@@ -0,0 +1,18 @@
1
+ export declare const CARD_MEDIA_TYPE = "application/mcp-server-card+json";
2
+ export interface CardRead {
3
+ /** Where it was asked for: the server URL's origin and path, then /server-card. */
4
+ url: string;
5
+ status?: number;
6
+ contentType?: string;
7
+ /** The document, when the answer was JSON. */
8
+ card?: unknown;
9
+ /** Why it couldn't be read (network, timeout). */
10
+ error?: string;
11
+ }
12
+ /** The reserved card location for a Streamable HTTP URL: its query left out, /server-card appended to its path. */
13
+ export declare function cardUrl(serverUrl: string): string;
14
+ /**
15
+ * GET the card from its reserved location. No credentials: cards are public by design,
16
+ * and the request goes to a path the MCP endpoint didn't ask to receive them on.
17
+ */
18
+ export declare function readServerCard(serverUrl: string, timeoutMs?: number): Promise<CardRead>;
package/dist/card.js ADDED
@@ -0,0 +1,39 @@
1
+ /*
2
+ * MCP Server Cards (SEP-2127, ext-server-card): a static document describing a remote
3
+ * server, read before connecting. Its reserved location is `<streamable-http-url>/server-card`,
4
+ * served as application/mcp-server-card+json. Its version and protocol versions SHOULD NOT
5
+ * contradict what the live connection reports (ext-server-card docs/discovery.md,
6
+ * "Consistency with Runtime Behavior"). Schema: schemas/server-card.json, from
7
+ * modelcontextprotocol/ext-server-card at 526201bb.
8
+ */
9
+ export const CARD_MEDIA_TYPE = 'application/mcp-server-card+json';
10
+ /** The reserved card location for a Streamable HTTP URL: its query left out, /server-card appended to its path. */
11
+ export function cardUrl(serverUrl) {
12
+ const u = new URL(serverUrl);
13
+ return `${u.origin}${u.pathname.replace(/\/+$/, '')}/server-card`;
14
+ }
15
+ /**
16
+ * GET the card from its reserved location. No credentials: cards are public by design,
17
+ * and the request goes to a path the MCP endpoint didn't ask to receive them on.
18
+ */
19
+ export async function readServerCard(serverUrl, timeoutMs = 10_000) {
20
+ const url = cardUrl(serverUrl);
21
+ try {
22
+ const response = await fetch(url, { headers: { Accept: `${CARD_MEDIA_TYPE}, application/json;q=0.9` }, signal: AbortSignal.timeout(timeoutMs), redirect: 'follow' });
23
+ const contentType = response.headers.get('content-type') ?? '';
24
+ const out = { url, status: response.status, contentType };
25
+ if (!response.ok || !/json/i.test(contentType))
26
+ return out;
27
+ const text = await response.text();
28
+ try {
29
+ out.card = JSON.parse(text);
30
+ }
31
+ catch {
32
+ // Says JSON, isn't: no card.
33
+ }
34
+ return out;
35
+ }
36
+ catch (error) {
37
+ return { url, error: (error instanceof Error ? error.message : String(error)).split('\n')[0] };
38
+ }
39
+ }
package/dist/cli.js CHANGED
@@ -1,4 +1,5 @@
1
1
  #!/usr/bin/env node
2
+ import { colourFor, colourise, colouriseError } from './colour.js';
2
3
  import { mkdir, writeFile } from 'node:fs/promises';
3
4
  import { join } from 'node:path';
4
5
  import { parseArgs } from 'node:util';
@@ -64,9 +65,22 @@ export async function main(argv) {
64
65
  process.stdout.write(sub && isCommand(sub) ? commandHelp(sub) : overview());
65
66
  return sub || values.help ? 0 : 2;
66
67
  }
68
+ // An option another command takes is ignored here: say so, or scan --no-auth reads as
69
+ // done while the stored logins are used anyway. A warning, not an error: CI scripts pass
70
+ // the shared options (--timeout) to every command.
71
+ const takes = new Set([...commandOptions(sub), 'help', 'version']);
72
+ const at = (k) => before.findIndex((a) => a === `--${k}` || a.startsWith(`--${k}=`));
73
+ // In the order they were typed.
74
+ const stray = Object.keys(values).filter((k) => !takes.has(k)).sort((a, b) => at(a) - at(b)).map((k) => `--${k}`);
75
+ if (stray.length) {
76
+ const one = stray.length === 1;
77
+ process.stderr.write(`toolmenu: ${sub} ignores ${stray.join(', ')}: ${one ? 'it is' : 'they are'} for other commands.\n → Next: Leave ${one ? 'it' : 'them'} out to silence this; toolmenu ${sub} --help lists the options ${sub} takes.\n`);
78
+ }
67
79
  const format = (values.json ? 'json' : values.format ?? 'text');
68
80
  if (!['text', 'json', 'github', 'markdown'].includes(format))
69
81
  throw new UsageError(`--format must be text, json, github or markdown`);
82
+ // Colour for a report read in a terminal (colour.ts); piped, CI and other formats stay plain.
83
+ const show = (report) => (format === 'text' && colourFor(process.stdout) ? colourise(report) : report);
70
84
  const failOn = (values['fail-on'] ?? 'error');
71
85
  if (!(failOn in SEVERITY_RANK))
72
86
  throw new UsageError(`--fail-on must be error, warn or info`);
@@ -99,6 +113,8 @@ export async function main(argv) {
99
113
  throw new UsageError('--client goes with snapshot.');
100
114
  if (values.client !== undefined && !Object.hasOwn(CLIENTS, values.client))
101
115
  throw new UsageError(`--client must be one of ${CLIENT_NAMES.join(', ')}`);
116
+ if (values['offline-check'] && sub !== 'snapshot')
117
+ throw new UsageError('--offline-check goes with snapshot.');
102
118
  if (values.as !== undefined && sub !== 'snapshot')
103
119
  throw new UsageError('--as goes with snapshot: it lists the menu once per identity.');
104
120
  if (values.elicitation !== undefined && sub !== 'session')
@@ -125,7 +141,12 @@ export async function main(argv) {
125
141
  throw new UsageError(error.message);
126
142
  throw error;
127
143
  }
128
- const withProtocol = entries.map((e) => (e.target && protocol !== 'auto' ? { ...e, target: { ...e.target, protocol } } : e));
144
+ const noAuth = values['no-auth'] === true;
145
+ const withProtocol = entries.map((e) => {
146
+ if (!e.target || e.target.kind !== 'http')
147
+ return e.target && protocol !== 'auto' ? { ...e, target: { ...e.target, protocol } } : e;
148
+ return { ...e, target: { ...e.target, ...(protocol !== 'auto' ? { protocol } : {}), ...(noAuth ? { noAuth } : {}) } };
149
+ });
129
150
  const outDir = values['out-dir'];
130
151
  if (outDir)
131
152
  await mkdir(outDir, { recursive: true });
@@ -139,6 +160,8 @@ export async function main(argv) {
139
160
  descriptionLimit: config.descriptionLimit,
140
161
  fullDescriptions: config.fullDescriptions,
141
162
  ...(config.client ? { client: config.client } : {}),
163
+ // stdio servers only: snapshot skips it for HTTP.
164
+ ...(config.offlineCheck ? { offlineCheck: true } : {}),
142
165
  onDone: (row, done, total) => {
143
166
  if (tty)
144
167
  process.stderr.write(`\r\x1b[K ${done}/${total} ${row.name}`);
@@ -150,7 +173,7 @@ export async function main(argv) {
150
173
  for (const r of rows)
151
174
  if (r.menu)
152
175
  await writeFile(join(outDir, `${r.name.replace(/[^\w.@-]+/g, '_')}.json`), JSON.stringify(r.menu, null, 2) + '\n');
153
- process.stdout.write(formatScan(rows, format, rest[0]) + '\n');
176
+ process.stdout.write(show(formatScan(rows, format, rest[0])) + '\n');
154
177
  return scanExit(rows, failOn);
155
178
  }
156
179
  if (sub === 'diff') {
@@ -167,7 +190,7 @@ export async function main(argv) {
167
190
  release,
168
191
  serverVersionIsRelease: values['server-version-is-release'] ?? config.serverVersionIsRelease,
169
192
  });
170
- process.stdout.write(formatDiff(result, before.tools.length, after.tools.length, format) + '\n');
193
+ process.stdout.write(show(formatDiff(result, before.tools.length, after.tools.length, format)) + '\n');
171
194
  return result.findings.some((f) => SEVERITY_RANK[f.severity] >= SEVERITY_RANK[failOn]) ? 1 : 0;
172
195
  }
173
196
  if (sub === 'history') {
@@ -200,7 +223,7 @@ export async function main(argv) {
200
223
  });
201
224
  if (values.csv)
202
225
  await writeFile(values.csv, historyCsv(result));
203
- process.stdout.write(formatHistory(result, format, outDir, values.csv) + '\n');
226
+ process.stdout.write(show(formatHistory(result, format, outDir, values.csv)) + '\n');
204
227
  return result.rows.length > 0 && result.rows.every((r) => r.status === 'failed') ? 1 : 0;
205
228
  }
206
229
  if (sub === 'session' && values.init) {
@@ -242,7 +265,7 @@ export async function main(argv) {
242
265
  const plan = planInit({ cwd: process.cwd(), target, out: values.out, session: values['with-session'] });
243
266
  if (plan.existing.length)
244
267
  throw new UsageError(existingMessage(plan));
245
- const { menu, findings } = await snapshot(target, { timeoutMs, processes, rules: config.rules, ignore: config.ignore, descriptionLimit: config.descriptionLimit, fullDescriptions: config.fullDescriptions });
268
+ const { menu, findings } = await snapshot(target, { timeoutMs, processes, rules: config.rules, ignore: config.ignore, descriptionLimit: config.descriptionLimit, fullDescriptions: config.fullDescriptions, ...(config.offlineCheck && target.kind === 'stdio' ? { offlineCheck: true } : {}) });
246
269
  await writeInit(plan, JSON.stringify(menu, null, 2) + '\n');
247
270
  process.stdout.write(initReport(plan, { server: menu.server.name, version: menu.server.version, tools: menu.tools.length, tokens: menu.totalTokens, counts: counts(findings) }, target));
248
271
  return 0;
@@ -330,7 +353,7 @@ export async function main(argv) {
330
353
  await writeFile(values['union-out'], JSON.stringify(result.union, null, 2) + '\n');
331
354
  const output = formatSession(result, format);
332
355
  if (output)
333
- process.stdout.write(output + '\n');
356
+ process.stdout.write(show(output) + '\n');
334
357
  return result.findings.some((f) => SEVERITY_RANK[f.severity] >= SEVERITY_RANK[failOn]) ? 1 : 0;
335
358
  }
336
359
  if (sub === 'session') {
@@ -355,7 +378,7 @@ export async function main(argv) {
355
378
  await writeFile(values['union-out'], JSON.stringify(result.union, null, 2) + '\n');
356
379
  const output = formatSession(result, format);
357
380
  if (output)
358
- process.stdout.write(output + '\n');
381
+ process.stdout.write(show(output) + '\n');
359
382
  return result.findings.some((f) => SEVERITY_RANK[f.severity] >= SEVERITY_RANK[failOn]) ? 1 : 0;
360
383
  }
361
384
  const target = parseTarget(rest, command, values.header ?? [], values.env ?? [], values['no-auth'], prefix, protocol);
@@ -363,13 +386,16 @@ export async function main(argv) {
363
386
  const routesPath = values.routes ?? config.routes;
364
387
  const routes = routesPath ? await loadRoutes(routesPath) : undefined;
365
388
  const client = values.client ?? config.client;
366
- const { menu, findings, cost, notOffered } = await snapshot(target, { routes, timeoutMs, processes, identities, client, lists, catalog: values.catalog ? (config.catalog ?? true) : undefined, rules: config.rules, ignore: config.ignore, descriptionLimit: config.descriptionLimit, fullDescriptions: config.fullDescriptions });
389
+ const offlineCheck = values['offline-check'] ?? config.offlineCheck;
390
+ if (values['offline-check'] && target.kind !== 'stdio')
391
+ throw new UsageError("--offline-check goes with a stdio server: an HTTP server's network isn't this machine's to cut.");
392
+ const { menu, findings, cost, notOffered } = await snapshot(target, { offlineCheck: offlineCheck && target.kind === 'stdio', routes, timeoutMs, processes, identities, client, lists, catalog: values.catalog ? (config.catalog ?? true) : undefined, rules: config.rules, ignore: config.ignore, descriptionLimit: config.descriptionLimit, fullDescriptions: config.fullDescriptions });
367
393
  const outPath = values['no-write'] ? undefined : values.out ?? 'menu.json';
368
394
  if (outPath)
369
395
  await writeFile(outPath, JSON.stringify(menu, null, 2) + '\n');
370
396
  const output = formatSnapshot(menu, findings, format, outPath, cost, notOffered);
371
397
  if (output)
372
- process.stdout.write(output + '\n');
398
+ process.stdout.write(show(output) + '\n');
373
399
  return findings.some((f) => SEVERITY_RANK[f.severity] >= SEVERITY_RANK[failOn]) ? 1 : 0;
374
400
  }
375
401
  function parseTarget(positionals, command, headers, env, noAuth, prefix, protocol) {
@@ -484,7 +510,8 @@ for (const method of ['log', 'info', 'debug']) {
484
510
  console[method] = (...args) => console.error(...args);
485
511
  }
486
512
  main(process.argv.slice(2)).then(exitWhenFlushed, (error) => {
487
- process.stderr.write(`toolmenu: ${failureMessage(error)}\n`);
513
+ const message = `toolmenu: ${failureMessage(error)}`;
514
+ process.stderr.write(`${colourFor(process.stderr) ? colouriseError(message) : message}\n`);
488
515
  exitWhenFlushed(2);
489
516
  });
490
517
  function explainCommand(rest, command, format) {
@@ -0,0 +1,8 @@
1
+ /** Whether text written to this stream gets colour. */
2
+ export declare function colourFor(stream: {
3
+ isTTY?: boolean;
4
+ }, env?: NodeJS.ProcessEnv): boolean;
5
+ /** The report with ANSI colour: the same characters otherwise. */
6
+ export declare function colourise(text: string): string;
7
+ /** An error that stopped a command, with colour: what failed, the labelled evidence, the next step. */
8
+ export declare function colouriseError(text: string): string;
package/dist/colour.js ADDED
@@ -0,0 +1,70 @@
1
+ /*
2
+ * Colour for text output in a terminal: severities, the next step, the verdict. Applied
3
+ * to the finished text, line by line, so every command's report gets it the same way
4
+ * and the text itself never depends on it. Only when the stream is a terminal, unless
5
+ * NO_COLOR (https://no-color.org) or FORCE_COLOR says otherwise; never for json,
6
+ * markdown or github output.
7
+ */
8
+ const ESC = '\x1b[';
9
+ const paint = (codes, text) => (text ? `${ESC}${codes}m${text}${ESC}0m` : text);
10
+ const SEVERITY = { ERROR: '1;31', WARN: '1;33', INFO: '1;36' };
11
+ /** Whether text written to this stream gets colour. */
12
+ export function colourFor(stream, env = process.env) {
13
+ if (env.NO_COLOR !== undefined && env.NO_COLOR !== '')
14
+ return false;
15
+ if (env.FORCE_COLOR !== undefined)
16
+ return env.FORCE_COLOR !== '0' && env.FORCE_COLOR !== 'false';
17
+ return !!stream.isTTY && env.TERM !== 'dumb';
18
+ }
19
+ /** The report with ANSI colour: the same characters otherwise. */
20
+ export function colourise(text) {
21
+ // Lines deeper than the current finding's message are its evidence: dimmed.
22
+ let evidenceFrom;
23
+ return text
24
+ .split('\n')
25
+ .map((line) => {
26
+ const indent = line.length - line.trimStart().length;
27
+ const finding = /^(\s*)(ERROR|WARN|INFO)(\s+)(\S+)(.*)$/.exec(line);
28
+ if (finding) {
29
+ const [, lead, severity, gap, rule, rest] = finding;
30
+ evidenceFrom = lead.length + severity.length + gap.length + 1;
31
+ return `${lead}${paint(SEVERITY[severity], severity)}${gap}${paint('1', rule)}${rest.replace(/ · unsure\b/, paint('2;3', ' · unsure'))}`;
32
+ }
33
+ const next = /^(\s*)(→ Next:)(.*)$/.exec(line);
34
+ if (next)
35
+ return `${next[1]}${paint('1;32', next[2])}${paint('32', next[3])}`;
36
+ if (!line.trim() || indent === 0)
37
+ evidenceFrom = undefined;
38
+ if (evidenceFrom !== undefined && indent >= evidenceFrom)
39
+ return paint('2', line);
40
+ if (/^✗ /.test(line))
41
+ return paint('1;31', line);
42
+ if (/^✓ /.test(line))
43
+ return paint('1;32', line);
44
+ const title = /^(toolmenu \w+)(.*)$/.exec(line);
45
+ if (title)
46
+ return `${paint('1;35', title[1])}${paint('1', title[2])}`;
47
+ const step = /^((?:step \d+|final):)(.*)$/.exec(line);
48
+ if (step)
49
+ return `${paint('1', step[1])}${step[2].replace(/menu changed/g, paint('33', 'menu changed'))}`;
50
+ return line;
51
+ })
52
+ .join('\n');
53
+ }
54
+ /** An error that stopped a command, with colour: what failed, the labelled evidence, the next step. */
55
+ export function colouriseError(text) {
56
+ return text
57
+ .split('\n')
58
+ .map((line) => {
59
+ if (line.startsWith('toolmenu: '))
60
+ return `${paint('1;31', 'toolmenu:')}${paint('1', line.slice('toolmenu:'.length))}`;
61
+ const next = /^(\s*)(→ Next:)(.*)$/.exec(line);
62
+ if (next)
63
+ return `${next[1]}${paint('1;32', next[2])}${paint('32', next[3])}`;
64
+ const field = /^(\s{2})([a-z][a-z -]*?)(\s{2,})(.*)$/.exec(line);
65
+ if (field)
66
+ return `${field[1]}${paint('2', field[2])}${field[3]}${field[4]}`;
67
+ return line;
68
+ })
69
+ .join('\n');
70
+ }
package/dist/config.d.ts CHANGED
@@ -30,6 +30,8 @@ export interface Config {
30
30
  lists?: string[];
31
31
  /** snapshot: the client to check the menu for (claude-code, claude-api-search, openai). --client wins. */
32
32
  client?: string;
33
+ /** snapshot (stdio): also list the menu with the network cut off. --offline-check turns it on too. */
34
+ offlineCheck?: boolean;
33
35
  /** Server processes (stdio) or connections (HTTP) to compare, the main one included (default 2). */
34
36
  processes?: number;
35
37
  /**
package/dist/config.js CHANGED
@@ -34,6 +34,8 @@ export async function loadConfig(path) {
34
34
  if (config.lists !== undefined && !(Array.isArray(config.lists) && config.lists.every((l) => LIST_WORDS.includes(l)))) {
35
35
  throw new Error(`${file}: lists must be a list of ${LIST_WORDS.join(', ')}`);
36
36
  }
37
+ if (config.offlineCheck !== undefined && typeof config.offlineCheck !== 'boolean')
38
+ throw new Error(`${file}: offlineCheck must be true or false`);
37
39
  if (config.client !== undefined && !(typeof config.client === 'string' && Object.hasOwn(CLIENTS, config.client))) {
38
40
  throw new Error(`${file}: client must be one of ${Object.keys(CLIENTS).join(', ')}`);
39
41
  }