mcp-medic 1.2.3 → 1.2.5

package/README.md

@@ -50,6 +50,8 @@ This is the whole point (a real handshake, not a schema guess) — but it means
 
 `mcp-medic` computes a deterministic **MCP Quality Score** (0–100) from the same diagnostics shown in the report — there's no separate, opaque scoring model guessing independently. Every point deducted traces back to one or more real diagnostics, and the same input always produces the same score (no LLM, no randomness, no extra network calls beyond the MCP inspection already performed).
 
+**What the score means — and doesn't**: it's a measurement of what mcp-medic's passive checks actually found (or looked for), not a certification. "Security 93/100" means *mcp-medic's heuristic security checks found issues worth 7 points* — it is not a security audit, and a 100 is not a guarantee the server is safe. Every report includes this disclaimer and a coverage figure so you can tell what was actually inspected (see below).
+
 ```bash
 mcp-medic score path/to/config.json
 # or, alongside the normal report:
@@ -60,15 +62,27 @@ mcp-medic check path/to/config.json --score
 
 | Dimension | Weight | What it reflects |
 |---|---|---|
-| Protocol | 25% | Version negotiation compatibility, capability-inspection health (`resources/list`/`prompts/list` succeeding), `serverInfo` presence |
+| Protocol | 25% | Version negotiation compatibility, capability-inspection health (`resources/list`/`prompts/list` succeeding), `serverInfo` presence, version downgrades |
 | Schema | 20% | `schema.*` diagnostics — malformed/missing input schemas, type mismatches, missing required fields |
 | Agent usability | 20% | Tool/resource/prompt naming, description quality, output schemas, annotations, tool-surface bloat |
 | Security | 20% | `security.*` heuristic diagnostics (untrusted remotes, overbroad permissions, prompt-injection-risk patterns) |
 | Reliability | 15% | Whether the server connects at all (currently binary — see [Known Limitations](#known-limitations)) |
 
+These weights are a deliberate, documented choice — not arbitrary — and are not changed casually; see `.agent-room/DECISIONS.md` if you're curious why.
+
 ### How deductions work
 
-Each dimension starts at 100. Diagnostics are grouped by `checkId` and severity (`error`/`warning`/`info`), each contributing capped points (errors up to 25/checkId, warnings up to 15/checkId, info up to 5/checkId) — so **one noisy check can never dominate a dimension**: 20 tools sharing the same description problem cost at most 15 points, not 300. The report's "Deductions" list names exactly which checks cost how many points.
+Each dimension starts at 100. Diagnostics are grouped by `checkId` and severity (`error`/`warning`/`info`), each contributing capped points (errors up to 25/checkId, warnings up to 15/checkId, info up to 5/checkId) — so **one noisy check can never dominate a dimension**: 20 tools sharing the same description problem cost at most 15 points, not 300. The report's "Deductions" list names exactly which checks cost how many points. **Nothing is deducted that isn't also a visible diagnostic** — including protocol-level facts like a negotiated version downgrade or a capability that failed to list, which are their own `protocol.*` diagnostics, not a hidden number baked into the score.
+
+### Coverage: what was actually inspected
+
+A dimension's score only means something if the checks that produce it actually ran. If you run `mcp-medic` with a custom, restricted check set (`runChecks({ checks: [...] })` via the library API), the score's `coverage` tells you which dimensions were fully evaluated (`'covered'`), partially evaluated (`'partial'` — some but not all of that dimension's checks ran), or not evaluated at all (`'not-covered'`). A `security: 'not-covered'` next to `Security 100/100` means "nothing was looked for," not "nothing is wrong." `coveragePercent` summarizes all five as one number. The standard CLI (`check`/`score`) always runs the full built-in check set, so coverage is 100% there by default.
+
+A server that fails to connect is never silently averaged out of a fleet's score either — it's named in `quality.unscoredServers`, and the human report calls it out explicitly.
+
+### Protocol-version-aware quality rules
+
+Some things a check might flag could be a genuine protocol violation for a given negotiated MCP version, or merely an ecosystem style recommendation — mcp-medic never mislabels one as the other. `src/protocol/quality-rules.ts` centralizes what each supported version actually requires (today: no version defines a hard tool-name length or character-pattern constraint, so this distinction is currently latent — the abstraction exists so a future version that *does* add one only needs a new entry there, not a rewrite of every check).
 
 ### Diagnostic categories
 
@@ -187,6 +201,8 @@ Define organization-wide policies that compose with built-in checks:
 
 `quality.minimumScore` fails the run (adds an error diagnostic) if the computed MCP Quality Score falls below the threshold. `quality.maxTools` is an org-enforced hard limit — distinct from the built-in `quality.tool-surface` check's default 100-tool *warning*, which stays a recommendation. `quality.requireToolDescriptions` (or the equivalent top-level `requireToolDescriptions`) turns every missing tool description into a policy error rather than the default warning.
 
+A `minimumScore` gate is also coverage-aware: if the score was computed from incomplete coverage (or a server that couldn't be scored), a `policy.partial-coverage-with-minimum-score` **warning** is added alongside it — a passing score should never look like a clean bill of health when only part of the server was actually evaluated. This never changes the pass/fail outcome of the `minimumScore` check itself (that stays a plain score-vs-threshold comparison), it just makes a partial assessment visible.
+
 ---
 
 ## VS Code Extension (experimental)
@@ -292,6 +308,8 @@ The `security.*` checks are heuristic — they pattern-match on what a server *d
 - **The Reliability quality dimension is currently binary**: 100 if the server connected, 0 if it didn't (plus any future `reliability.*` diagnostics — none exist yet). Signals like latency trends, retry behavior, or flakiness across repeated runs aren't scored yet.
 - **`resources/list`/`prompts/list` pagination (`nextCursor`) is not followed** — mcp-medic inspects only the first page a server returns, matching the existing (also unpaginated) `tools/list` handling. A server with a very large resource/prompt catalog behind pagination will be under-inspected.
 - **The quality score never calls `resources/read`, `prompts/get`, or any tool** — it's entirely derived from the passive `initialize`/`tools/list`/`resources/list`/`prompts/list` responses already gathered during a normal `check`. See [SECURITY.md](./SECURITY.md) for the full passive-only guarantee.
+- **The CLI cannot currently restrict the check set** (no `--only-checks` flag), so `coverage` is always 100% via `check`/`score`. Partial coverage (and the coverage-aware `minimumScore` warning) only happens when the library API's `runChecks({ checks: [...] })` is called with a restricted list.
+- **No currently-supported MCP protocol version defines a hard tool-name length or character-pattern constraint**, so `src/protocol/quality-rules.ts`'s protocol-vs-quality distinction for tool names is real but currently dormant — every finding today is a quality recommendation, never a protocol violation, because no version actually requires one. The abstraction is there for when a future version does.
 - **npm README sync**: Latest docs live on GitHub main; npm README updates on the next publish.
 - **First run via `npx`** pays a one-time cost to resolve and download the package; once installed (or on a warm npx cache), `--help`/`--version` return in well under 100ms.
 

package/dist/checks/index.d.ts

@@ -6,6 +6,7 @@ export { sampleCallSimulationCheck } from './sample-call-simulation.js';
 export { securityUntrustedRemoteCheck } from './security-untrusted-remote.js';
 export { securityOverbroadPermissionsCheck } from './security-overbroad-permissions.js';
 export { securityPromptInjectionRiskCheck } from './security-prompt-injection-risk.js';
+export { securityHiddenUnicodeTagsCheck } from './security-hidden-unicode-tags.js';
 export { qualityToolNamesCheck } from './quality-tool-names.js';
 export { qualityToolDescriptionsCheck } from './quality-tool-descriptions.js';
 export { qualityToolOutputSchemaCheck } from './quality-tool-output-schema.js';
@@ -13,4 +14,5 @@ export { qualityToolAnnotationsCheck } from './quality-tool-annotations.js';
 export { qualityToolSurfaceCheck, createToolSurfaceCheck, DEFAULT_MAX_TOOLS_WARNING_THRESHOLD } from './quality-tool-surface.js';
 export { qualityResourcesCheck } from './quality-resources.js';
 export { qualityPromptsCheck } from './quality-prompts.js';
+export { protocolConnectionHealthCheck } from './protocol-connection-health.js';
 export declare const allChecks: import("../types.js").Check[];

package/dist/checks/index.js

@@ -6,6 +6,7 @@ export { sampleCallSimulationCheck } from './sample-call-simulation.js';
 export { securityUntrustedRemoteCheck } from './security-untrusted-remote.js';
 export { securityOverbroadPermissionsCheck } from './security-overbroad-permissions.js';
 export { securityPromptInjectionRiskCheck } from './security-prompt-injection-risk.js';
+export { securityHiddenUnicodeTagsCheck } from './security-hidden-unicode-tags.js';
 export { qualityToolNamesCheck } from './quality-tool-names.js';
 export { qualityToolDescriptionsCheck } from './quality-tool-descriptions.js';
 export { qualityToolOutputSchemaCheck } from './quality-tool-output-schema.js';
@@ -13,6 +14,7 @@ export { qualityToolAnnotationsCheck } from './quality-tool-annotations.js';
 export { qualityToolSurfaceCheck, createToolSurfaceCheck, DEFAULT_MAX_TOOLS_WARNING_THRESHOLD } from './quality-tool-surface.js';
 export { qualityResourcesCheck } from './quality-resources.js';
 export { qualityPromptsCheck } from './quality-prompts.js';
+export { protocolConnectionHealthCheck } from './protocol-connection-health.js';
 import { malformedSchemaCheck } from './malformed-schema.js';
 import { missingRequiredFieldsCheck } from './missing-required-fields.js';
 import { typeMismatchCheck } from './type-mismatch.js';
@@ -21,6 +23,7 @@ import { sampleCallSimulationCheck } from './sample-call-simulation.js';
 import { securityUntrustedRemoteCheck } from './security-untrusted-remote.js';
 import { securityOverbroadPermissionsCheck } from './security-overbroad-permissions.js';
 import { securityPromptInjectionRiskCheck } from './security-prompt-injection-risk.js';
+import { securityHiddenUnicodeTagsCheck } from './security-hidden-unicode-tags.js';
 import { qualityToolNamesCheck } from './quality-tool-names.js';
 import { qualityToolDescriptionsCheck } from './quality-tool-descriptions.js';
 import { qualityToolOutputSchemaCheck } from './quality-tool-output-schema.js';
@@ -28,7 +31,9 @@ import { qualityToolAnnotationsCheck } from './quality-tool-annotations.js';
 import { qualityToolSurfaceCheck } from './quality-tool-surface.js';
 import { qualityResourcesCheck } from './quality-resources.js';
 import { qualityPromptsCheck } from './quality-prompts.js';
+import { protocolConnectionHealthCheck } from './protocol-connection-health.js';
 export const allChecks = [
+    protocolConnectionHealthCheck,
     malformedSchemaCheck,
     missingRequiredFieldsCheck,
     typeMismatchCheck,
@@ -37,6 +42,7 @@ export const allChecks = [
     securityUntrustedRemoteCheck,
     securityOverbroadPermissionsCheck,
     securityPromptInjectionRiskCheck,
+    securityHiddenUnicodeTagsCheck,
     qualityToolNamesCheck,
     qualityToolDescriptionsCheck,
     qualityToolOutputSchemaCheck,

package/dist/checks/protocol-connection-health.d.ts

@@ -0,0 +1,17 @@
+import type { Check } from '../types.js';
+/**
+ * Surfaces connection-level protocol facts as real, visible diagnostics —
+ * previously these (capability-inspection failures, a version downgrade, a
+ * missing serverInfo) were deducted from the quality score directly from
+ * `MCPConnection` metadata inside src/quality-score.ts, with no
+ * corresponding diagnostic a user could actually see in the report or
+ * `--json` output. That meant a developer looking at "why did my protocol
+ * score drop" would find nothing in `diagnostics` explaining it.
+ *
+ * This check makes those same facts flow through the standard
+ * connection -> diagnostics -> dimension -> score pipeline like every
+ * other check, so every point deducted is explainable from a visible
+ * DiagnosticResult (see .agent-room/DECISIONS.md for the full rationale,
+ * including why this also changes some of the exact point values).
+ */
+export declare const protocolConnectionHealthCheck: Check;

package/dist/checks/protocol-connection-health.js

@@ -0,0 +1,78 @@
+/**
+ * Surfaces connection-level protocol facts as real, visible diagnostics —
+ * previously these (capability-inspection failures, a version downgrade, a
+ * missing serverInfo) were deducted from the quality score directly from
+ * `MCPConnection` metadata inside src/quality-score.ts, with no
+ * corresponding diagnostic a user could actually see in the report or
+ * `--json` output. That meant a developer looking at "why did my protocol
+ * score drop" would find nothing in `diagnostics` explaining it.
+ *
+ * This check makes those same facts flow through the standard
+ * connection -> diagnostics -> dimension -> score pipeline like every
+ * other check, so every point deducted is explainable from a visible
+ * DiagnosticResult (see .agent-room/DECISIONS.md for the full rationale,
+ * including why this also changes some of the exact point values).
+ */
+export const protocolConnectionHealthCheck = {
+    id: 'protocol.connection-health',
+    description: 'Flags protocol-level connection facts: a negotiated version downgrade, a missing serverInfo, or a declared capability whose list call failed.',
+    run(connection) {
+        const results = [];
+        try {
+            if (connection.protocolVersion?.negotiated &&
+                connection.protocolVersion.negotiated !== connection.protocolVersion.requested) {
+                results.push({
+                    checkId: 'protocol.version-downgrade',
+                    severity: 'info',
+                    message: `Server negotiated protocol version ${connection.protocolVersion.negotiated} instead of the requested ${connection.protocolVersion.requested}.`,
+                    serverName: connection.server.name,
+                    category: 'protocol',
+                    details: {
+                        requested: connection.protocolVersion.requested,
+                        negotiated: connection.protocolVersion.negotiated,
+                    },
+                });
+            }
+            if (!connection.serverInfo?.name) {
+                results.push({
+                    checkId: 'protocol.missing-server-info',
+                    severity: 'info',
+                    message: 'Server did not report its name/version in serverInfo during initialize.',
+                    serverName: connection.server.name,
+                    category: 'protocol',
+                    suggestedFix: { description: 'Have the server include a serverInfo.name in its initialize response.' },
+                });
+            }
+            if (connection.capabilityErrors?.resources) {
+                results.push({
+                    checkId: 'protocol.capability-error',
+                    severity: 'error',
+                    message: `Server declared the "resources" capability, but resources/list failed: ${connection.capabilityErrors.resources}`,
+                    serverName: connection.server.name,
+                    category: 'protocol',
+                    details: { capability: 'resources', error: connection.capabilityErrors.resources },
+                });
+            }
+            if (connection.capabilityErrors?.prompts) {
+                results.push({
+                    checkId: 'protocol.capability-error',
+                    severity: 'error',
+                    message: `Server declared the "prompts" capability, but prompts/list failed: ${connection.capabilityErrors.prompts}`,
+                    serverName: connection.server.name,
+                    category: 'protocol',
+                    details: { capability: 'prompts', error: connection.capabilityErrors.prompts },
+                });
+            }
+        }
+        catch (err) {
+            results.push({
+                checkId: 'protocol.connection-health',
+                severity: 'error',
+                message: `check failed internally: ${err instanceof Error ? err.message : String(err)}`,
+                serverName: connection.server.name,
+                category: 'protocol',
+            });
+        }
+        return results;
+    },
+};

package/dist/checks/quality-resources.d.ts

@@ -1,8 +1,8 @@
 import type { Check } from '../types.js';
 /**
- * Inspects only what `resources/list` already returned (see
- * MCPConnection.resources, populated passively in src/protocol/connect.ts).
- * Never calls `resources/read` — that would be real content retrieval, out
- * of scope for a passive `check`.
+ * Inspects only what `resources/list` and `resources/templates/list` already
+ * returned (see MCPConnection.resources/resourceTemplates, populated
+ * passively in src/protocol/connect.ts). Never calls `resources/read` —
+ * that would be real content retrieval, out of scope for a passive `check`.
  */
 export declare const qualityResourcesCheck: Check;

package/dist/checks/quality-resources.js

@@ -1,80 +1,135 @@
 /**
- * Inspects only what `resources/list` already returned (see
- * MCPConnection.resources, populated passively in src/protocol/connect.ts).
- * Never calls `resources/read` — that would be real content retrieval, out
- * of scope for a passive `check`.
+ * Inspects only what `resources/list` and `resources/templates/list` already
+ * returned (see MCPConnection.resources/resourceTemplates, populated
+ * passively in src/protocol/connect.ts). Never calls `resources/read` —
+ * that would be real content retrieval, out of scope for a passive `check`.
  */
 export const qualityResourcesCheck = {
     id: 'quality.resource',
-    description: 'Flags duplicate/empty resource URIs, missing required names, and other resources/list quality issues.',
+    description: 'Flags duplicate/empty resource(-template) URIs, missing required names, and other resources/list quality issues.',
     run(connection) {
         const results = [];
         try {
             const resources = connection.resources;
-            if (!resources || !Array.isArray(resources)) {
-                return results;
-            }
-            const byUri = new Map();
-            for (const resource of resources) {
-                const uri = resource.uri;
-                if (typeof uri !== 'string' || uri.trim() === '') {
-                    results.push({
-                        checkId: 'quality.resource',
-                        severity: 'error',
-                        message: 'Resource has an empty or invalid "uri" — the MCP spec requires resources to have a URI.',
-                        serverName: connection.server.name,
-                        category: 'schema',
-                        details: { resource },
-                    });
-                    continue;
-                }
-                byUri.set(uri, (byUri.get(uri) ?? 0) + 1);
-                // Per the MCP spec's Resource type (extends BaseMetadata), "name" is
-                // a required field, not optional.
-                if (!resource.name || resource.name.trim() === '') {
-                    results.push({
-                        checkId: 'quality.resource',
-                        severity: 'error',
-                        message: `Resource "${uri}" is missing a "name" — required by the MCP spec's Resource type.`,
-                        serverName: connection.server.name,
-                        category: 'schema',
-                        details: { uri },
-                        suggestedFix: { description: `Add a "name" field to the resource at "${uri}".` },
-                    });
+            if (Array.isArray(resources)) {
+                const byUri = new Map();
+                for (const resource of resources) {
+                    const uri = resource.uri;
+                    if (typeof uri !== 'string' || uri.trim() === '') {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'error',
+                            message: 'Resource has an empty or invalid "uri" — the MCP spec requires resources to have a URI.',
+                            serverName: connection.server.name,
+                            category: 'schema',
+                            details: { resource },
+                        });
+                        continue;
+                    }
+                    byUri.set(uri, (byUri.get(uri) ?? 0) + 1);
+                    // Per the MCP spec's Resource type (extends BaseMetadata), "name" is
+                    // a required field, not optional.
+                    if (!resource.name || resource.name.trim() === '') {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'error',
+                            message: `Resource "${uri}" is missing a "name" — required by the MCP spec's Resource type.`,
+                            serverName: connection.server.name,
+                            category: 'schema',
+                            details: { uri },
+                            suggestedFix: { description: `Add a "name" field to the resource at "${uri}".` },
+                        });
+                    }
+                    // description is optional per spec — absence is a quality
+                    // recommendation, not a violation.
+                    if (!resource.description || resource.description.trim() === '') {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'info',
+                            message: `Resource "${uri}" has no description, making it harder for an agent to know when to read it.`,
+                            serverName: connection.server.name,
+                            category: 'quality',
+                            details: { uri },
+                        });
+                    }
+                    if (resource.size !== undefined && (typeof resource.size !== 'number' || resource.size < 0)) {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'warning',
+                            message: `Resource "${uri}" declares an invalid "size" (${JSON.stringify(resource.size)}) — size must be a non-negative number.`,
+                            serverName: connection.server.name,
+                            category: 'schema',
+                            details: { uri, size: resource.size },
+                        });
+                    }
                 }
-                // description is optional per spec — absence is a quality
-                // recommendation, not a violation.
-                if (!resource.description || resource.description.trim() === '') {
-                    results.push({
-                        checkId: 'quality.resource',
-                        severity: 'info',
-                        message: `Resource "${uri}" has no description, making it harder for an agent to know when to read it.`,
-                        serverName: connection.server.name,
-                        category: 'quality',
-                        details: { uri },
-                    });
-                }
-                if (resource.size !== undefined && (typeof resource.size !== 'number' || resource.size < 0)) {
-                    results.push({
-                        checkId: 'quality.resource',
-                        severity: 'warning',
-                        message: `Resource "${uri}" declares an invalid "size" (${JSON.stringify(resource.size)}) — size must be a non-negative number.`,
-                        serverName: connection.server.name,
-                        category: 'schema',
-                        details: { uri, size: resource.size },
-                    });
+                for (const [uri, count] of byUri) {
+                    if (count > 1) {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'error',
+                            message: `Resource URI "${uri}" is declared ${count} times — resource URIs must be unique.`,
+                            serverName: connection.server.name,
+                            category: 'schema',
+                            details: { uri, duplicateCount: count },
+                        });
+                    }
                 }
             }
-            for (const [uri, count] of byUri) {
-                if (count > 1) {
-                    results.push({
-                        checkId: 'quality.resource',
-                        severity: 'error',
-                        message: `Resource URI "${uri}" is declared ${count} times — resource URIs must be unique.`,
-                        serverName: connection.server.name,
-                        category: 'schema',
-                        details: { uri, duplicateCount: count },
-                    });
+            const templates = connection.resourceTemplates;
+            if (Array.isArray(templates)) {
+                const byUriTemplate = new Map();
+                for (const template of templates) {
+                    const uriTemplate = template.uriTemplate;
+                    if (typeof uriTemplate !== 'string' || uriTemplate.trim() === '') {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'error',
+                            message: 'Resource template has an empty or invalid "uriTemplate" — the MCP spec requires resource templates to have one.',
+                            serverName: connection.server.name,
+                            category: 'schema',
+                            details: { template },
+                        });
+                        continue;
+                    }
+                    byUriTemplate.set(uriTemplate, (byUriTemplate.get(uriTemplate) ?? 0) + 1);
+                    // Per the MCP spec's ResourceTemplate type (extends BaseMetadata),
+                    // "name" is a required field, not optional.
+                    if (!template.name || template.name.trim() === '') {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'error',
+                            message: `Resource template "${uriTemplate}" is missing a "name" — required by the MCP spec's ResourceTemplate type.`,
+                            serverName: connection.server.name,
+                            category: 'schema',
+                            details: { uriTemplate },
+                            suggestedFix: { description: `Add a "name" field to the resource template "${uriTemplate}".` },
+                        });
+                    }
+                    // description is optional per spec — absence is a quality
+                    // recommendation, not a violation.
+                    if (!template.description || template.description.trim() === '') {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'info',
+                            message: `Resource template "${uriTemplate}" has no description, making it harder for an agent to know when to use it.`,
+                            serverName: connection.server.name,
+                            category: 'quality',
+                            details: { uriTemplate },
+                        });
+                    }
+                }
+                for (const [uriTemplate, count] of byUriTemplate) {
+                    if (count > 1) {
+                        results.push({
+                            checkId: 'quality.resource',
+                            severity: 'error',
+                            message: `Resource template "${uriTemplate}" is declared ${count} times — resource template URIs must be unique.`,
+                            serverName: connection.server.name,
+                            category: 'schema',
+                            details: { uriTemplate, duplicateCount: count },
+                        });
+                    }
                 }
             }
         }

package/dist/checks/quality-tool-names.d.ts

@@ -1,2 +1,35 @@
 import type { Check } from '../types.js';
+import { type ToolNameProtocolRules } from '../protocol/quality-rules.js';
+export interface ToolNameFinding {
+    severity: 'error' | 'warning';
+    /** 'protocol' only when `rules` itself defines a hard constraint the name
+     * violates; everything else is 'quality' (a recommendation, never a
+     * protocol violation) — see module doc. */
+    category: 'protocol' | 'quality';
+    message: string;
+    confidence?: 'low' | 'medium' | 'high';
+    suggestedFixDescription: string;
+}
+/**
+ * Pure evaluation of a single tool name, independent of the rest of the
+ * connection (duplicate detection is connection-wide and stays in `run()`).
+ * Exported so tests can inject a synthetic `ToolNameProtocolRules` (e.g. a
+ * hypothetical future protocol version with a real length/pattern
+ * constraint) without needing that version to actually exist yet — see
+ * src/protocol/quality-rules.ts.
+ *
+ * The A/B/C distinction this check makes:
+ *   A. PROTOCOL VIOLATION — only possible if `rules.maxLength`/`rules.pattern`
+ *      is defined by the negotiated version AND the name violates it.
+ *      Reported as category 'protocol', severity 'error'.
+ *   B. QUALITY WARNING — technically spec-valid, but likely to hurt agent
+ *      usability (too long by convention, odd characters, ambiguous name).
+ *      Reported as category 'quality', severity 'warning'.
+ *   C. Empty name is kept as a 'quality' error (not 'protocol') — no
+ *      supported version's spec actually forbids an empty string, but a
+ *      tool a client can't reference or distinguish is functionally
+ *      broken, which still deserves error severity without mislabeling it
+ *      a protocol violation.
+ */
+export declare function evaluateToolName(name: string, rules: ToolNameProtocolRules): ToolNameFinding[];
 export declare const qualityToolNamesCheck: Check;

package/dist/checks/quality-tool-names.js

@@ -1,9 +1,6 @@
-/** MCP's `Tool.name` (from `BaseMetadata`) has no documented pattern or
- * length constraint in the spec — so none of these are "protocol
- * violations" in the strict sense. Empty and duplicate names are still
- * treated as errors here because a tool a client cannot reference or
- * distinguish is functionally broken, not merely unstylish; everything
- * else is a warning. */
+import { getProtocolQualityRules } from '../protocol/quality-rules.js';
+/** Ecosystem recommendation, not a protocol constraint — see the module-level
+ * note on `evaluateToolName` for the A/B/C distinction this check makes. */
 const MAX_REASONABLE_NAME_LENGTH = 128;
 /** Conservative, curated list — exact (case-insensitive) matches only, to
  * avoid false-positiving on legitimately short/plain real tool names. */
@@ -31,64 +28,106 @@ function hasInvalidCharacters(name) {
     // eslint-disable-next-line no-control-regex
     return /[\t\n\r\x00-\x08\x0b\x0c\x0e-\x1f]/.test(name);
 }
+/**
+ * Pure evaluation of a single tool name, independent of the rest of the
+ * connection (duplicate detection is connection-wide and stays in `run()`).
+ * Exported so tests can inject a synthetic `ToolNameProtocolRules` (e.g. a
+ * hypothetical future protocol version with a real length/pattern
+ * constraint) without needing that version to actually exist yet — see
+ * src/protocol/quality-rules.ts.
+ *
+ * The A/B/C distinction this check makes:
+ *   A. PROTOCOL VIOLATION — only possible if `rules.maxLength`/`rules.pattern`
+ *      is defined by the negotiated version AND the name violates it.
+ *      Reported as category 'protocol', severity 'error'.
+ *   B. QUALITY WARNING — technically spec-valid, but likely to hurt agent
+ *      usability (too long by convention, odd characters, ambiguous name).
+ *      Reported as category 'quality', severity 'warning'.
+ *   C. Empty name is kept as a 'quality' error (not 'protocol') — no
+ *      supported version's spec actually forbids an empty string, but a
+ *      tool a client can't reference or distinguish is functionally
+ *      broken, which still deserves error severity without mislabeling it
+ *      a protocol violation.
+ */
+export function evaluateToolName(name, rules) {
+    const findings = [];
+    if (name.trim() === '') {
+        findings.push({
+            severity: 'error',
+            category: 'quality',
+            message: 'Tool has an empty name — a client cannot reference or distinguish it.',
+            suggestedFixDescription: 'Give the tool a non-empty, descriptive name.',
+        });
+        return findings; // nothing else meaningful to evaluate on an empty name
+    }
+    if (rules.maxLength !== undefined && name.length > rules.maxLength) {
+        findings.push({
+            severity: 'error',
+            category: 'protocol',
+            message: `Tool name "${name.slice(0, 40)}..." is ${name.length} characters, exceeding the negotiated protocol's maximum of ${rules.maxLength} — this is a protocol violation, not a style recommendation.`,
+            suggestedFixDescription: `Shorten the tool name to ${rules.maxLength} characters or fewer to comply with the negotiated protocol version.`,
+        });
+    }
+    else if (name.length > MAX_REASONABLE_NAME_LENGTH) {
+        findings.push({
+            severity: 'warning',
+            category: 'quality',
+            confidence: 'medium',
+            message: `Tool name "${name.slice(0, 40)}..." name is ${name.length} characters, exceeding the recommended ${MAX_REASONABLE_NAME_LENGTH}.`,
+            suggestedFixDescription: 'Shorten the tool name to something concise and memorable.',
+        });
+    }
+    if (rules.pattern && !rules.pattern.test(name)) {
+        findings.push({
+            severity: 'error',
+            category: 'protocol',
+            message: `Tool name "${name}" does not match the naming pattern required by the negotiated protocol version — this is a protocol violation, not a style recommendation.`,
+            suggestedFixDescription: 'Rename the tool to match the naming pattern required by the negotiated protocol version.',
+        });
+    }
+    else if (hasInvalidCharacters(name)) {
+        findings.push({
+            severity: 'warning',
+            category: 'quality',
+            message: `Tool name "${JSON.stringify(name)}" contains whitespace/control characters that may break client tooling.`,
+            suggestedFixDescription: 'Use only plain, printable characters in tool names (letters, digits, -, _).',
+        });
+    }
+    if (name.length === 1 || PLACEHOLDER_NAMES.has(name.trim().toLowerCase())) {
+        findings.push({
+            severity: 'warning',
+            category: 'quality',
+            confidence: 'medium',
+            message: `Tool name "${name}" is ambiguous or looks like a placeholder — it doesn't communicate what the tool does.`,
+            suggestedFixDescription: 'Rename the tool to describe its action, e.g. "search_flights" instead of "tool".',
+        });
+    }
+    return findings;
+}
 export const qualityToolNamesCheck = {
     id: 'quality.tool-name',
-    description: 'Flags empty, duplicate, overly long, or placeholder-looking tool names.',
+    description: 'Flags empty, duplicate, overly long, or placeholder-looking tool names; distinguishes protocol-version-defined violations from quality recommendations.',
     run(connection) {
         const results = [];
         try {
             if (!connection.tools || !Array.isArray(connection.tools)) {
                 return results;
             }
+            const rules = getProtocolQualityRules(connection.protocolVersion?.negotiated).toolName;
             const seen = new Map();
             for (const tool of connection.tools) {
                 const name = tool.name;
                 seen.set(name, (seen.get(name) ?? 0) + 1);
-                if (name.trim() === '') {
-                    results.push({
-                        checkId: 'quality.tool-name',
-                        severity: 'error',
-                        message: 'Tool has an empty name — a client cannot reference or distinguish it.',
-                        serverName: connection.server.name,
-                        toolName: name,
-                        category: 'quality',
-                        suggestedFix: { description: 'Give the tool a non-empty, descriptive name.' },
-                    });
-                    continue;
-                }
-                if (name.length > MAX_REASONABLE_NAME_LENGTH) {
-                    results.push({
-                        checkId: 'quality.tool-name',
-                        severity: 'warning',
-                        message: `Tool "${name.slice(0, 40)}..." name is ${name.length} characters, exceeding the recommended ${MAX_REASONABLE_NAME_LENGTH}.`,
-                        serverName: connection.server.name,
-                        toolName: name,
-                        category: 'quality',
-                        confidence: 'medium',
-                        suggestedFix: { description: 'Shorten the tool name to something concise and memorable.' },
-                    });
-                }
-                if (hasInvalidCharacters(name)) {
-                    results.push({
-                        checkId: 'quality.tool-name',
-                        severity: 'warning',
-                        message: `Tool name "${JSON.stringify(name)}" contains whitespace/control characters that may break client tooling.`,
-                        serverName: connection.server.name,
-                        toolName: name,
-                        category: 'quality',
-                        suggestedFix: { description: 'Use only plain, printable characters in tool names (letters, digits, -, _).' },
-                    });
-                }
-                if (name.length === 1 || PLACEHOLDER_NAMES.has(name.trim().toLowerCase())) {
+                for (const finding of evaluateToolName(name, rules)) {
                     results.push({
                         checkId: 'quality.tool-name',
-                        severity: 'warning',
-                        message: `Tool name "${name}" is ambiguous or looks like a placeholder — it doesn't communicate what the tool does.`,
+                        severity: finding.severity,
+                        message: finding.message,
                         serverName: connection.server.name,
                         toolName: name,
-                        category: 'quality',
-                        confidence: 'medium',
-                        suggestedFix: { description: 'Rename the tool to describe its action, e.g. "search_flights" instead of "tool".' },
+                        category: finding.category,
+                        ...(finding.confidence ? { confidence: finding.confidence } : {}),
+                        suggestedFix: { description: finding.suggestedFixDescription },
                     });
                 }
             }