mcp-medic 1.2.3 → 1.2.4

package/README.md

@@ -50,6 +50,8 @@ This is the whole point (a real handshake, not a schema guess) — but it means
 
 `mcp-medic` computes a deterministic **MCP Quality Score** (0–100) from the same diagnostics shown in the report — there's no separate, opaque scoring model guessing independently. Every point deducted traces back to one or more real diagnostics, and the same input always produces the same score (no LLM, no randomness, no extra network calls beyond the MCP inspection already performed).
 
+**What the score means — and doesn't**: it's a measurement of what mcp-medic's passive checks actually found (or looked for), not a certification. "Security 93/100" means *mcp-medic's heuristic security checks found issues worth 7 points* — it is not a security audit, and a 100 is not a guarantee the server is safe. Every report includes this disclaimer and a coverage figure so you can tell what was actually inspected (see below).
+
 ```bash
 mcp-medic score path/to/config.json
 # or, alongside the normal report:
@@ -60,15 +62,27 @@ mcp-medic check path/to/config.json --score
 
 | Dimension | Weight | What it reflects |
 |---|---|---|
-| Protocol | 25% | Version negotiation compatibility, capability-inspection health (`resources/list`/`prompts/list` succeeding), `serverInfo` presence |
+| Protocol | 25% | Version negotiation compatibility, capability-inspection health (`resources/list`/`prompts/list` succeeding), `serverInfo` presence, version downgrades |
 | Schema | 20% | `schema.*` diagnostics — malformed/missing input schemas, type mismatches, missing required fields |
 | Agent usability | 20% | Tool/resource/prompt naming, description quality, output schemas, annotations, tool-surface bloat |
 | Security | 20% | `security.*` heuristic diagnostics (untrusted remotes, overbroad permissions, prompt-injection-risk patterns) |
 | Reliability | 15% | Whether the server connects at all (currently binary — see [Known Limitations](#known-limitations)) |
 
+These weights are a deliberate, documented choice — not arbitrary — and are not changed casually; see `.agent-room/DECISIONS.md` if you're curious why.
+
 ### How deductions work
 
-Each dimension starts at 100. Diagnostics are grouped by `checkId` and severity (`error`/`warning`/`info`), each contributing capped points (errors up to 25/checkId, warnings up to 15/checkId, info up to 5/checkId) — so **one noisy check can never dominate a dimension**: 20 tools sharing the same description problem cost at most 15 points, not 300. The report's "Deductions" list names exactly which checks cost how many points.
+Each dimension starts at 100. Diagnostics are grouped by `checkId` and severity (`error`/`warning`/`info`), each contributing capped points (errors up to 25/checkId, warnings up to 15/checkId, info up to 5/checkId) — so **one noisy check can never dominate a dimension**: 20 tools sharing the same description problem cost at most 15 points, not 300. The report's "Deductions" list names exactly which checks cost how many points. **Nothing is deducted that isn't also a visible diagnostic** — including protocol-level facts like a negotiated version downgrade or a capability that failed to list, which are their own `protocol.*` diagnostics, not a hidden number baked into the score.
+
+### Coverage: what was actually inspected
+
+A dimension's score only means something if the checks that produce it actually ran. If you run `mcp-medic` with a custom, restricted check set (`runChecks({ checks: [...] })` via the library API), the score's `coverage` tells you which dimensions were fully evaluated (`'covered'`), partially evaluated (`'partial'` — some but not all of that dimension's checks ran), or not evaluated at all (`'not-covered'`). A `security: 'not-covered'` next to `Security 100/100` means "nothing was looked for," not "nothing is wrong." `coveragePercent` summarizes all five as one number. The standard CLI (`check`/`score`) always runs the full built-in check set, so coverage is 100% there by default.
+
+A server that fails to connect is never silently averaged out of a fleet's score either — it's named in `quality.unscoredServers`, and the human report calls it out explicitly.
+
+### Protocol-version-aware quality rules
+
+Some things a check might flag could be a genuine protocol violation for a given negotiated MCP version, or merely an ecosystem style recommendation — mcp-medic never mislabels one as the other. `src/protocol/quality-rules.ts` centralizes what each supported version actually requires (today: no version defines a hard tool-name length or character-pattern constraint, so this distinction is currently latent — the abstraction exists so a future version that *does* add one only needs a new entry there, not a rewrite of every check).
 
 ### Diagnostic categories
 
@@ -187,6 +201,8 @@ Define organization-wide policies that compose with built-in checks:
 
 `quality.minimumScore` fails the run (adds an error diagnostic) if the computed MCP Quality Score falls below the threshold. `quality.maxTools` is an org-enforced hard limit — distinct from the built-in `quality.tool-surface` check's default 100-tool *warning*, which stays a recommendation. `quality.requireToolDescriptions` (or the equivalent top-level `requireToolDescriptions`) turns every missing tool description into a policy error rather than the default warning.
 
+A `minimumScore` gate is also coverage-aware: if the score was computed from incomplete coverage (or a server that couldn't be scored), a `policy.partial-coverage-with-minimum-score` **warning** is added alongside it — a passing score should never look like a clean bill of health when only part of the server was actually evaluated. This never changes the pass/fail outcome of the `minimumScore` check itself (that stays a plain score-vs-threshold comparison), it just makes a partial assessment visible.
+
 ---
 
 ## VS Code Extension (experimental)
@@ -292,6 +308,8 @@ The `security.*` checks are heuristic — they pattern-match on what a server *d
 - **The Reliability quality dimension is currently binary**: 100 if the server connected, 0 if it didn't (plus any future `reliability.*` diagnostics — none exist yet). Signals like latency trends, retry behavior, or flakiness across repeated runs aren't scored yet.
 - **`resources/list`/`prompts/list` pagination (`nextCursor`) is not followed** — mcp-medic inspects only the first page a server returns, matching the existing (also unpaginated) `tools/list` handling. A server with a very large resource/prompt catalog behind pagination will be under-inspected.
 - **The quality score never calls `resources/read`, `prompts/get`, or any tool** — it's entirely derived from the passive `initialize`/`tools/list`/`resources/list`/`prompts/list` responses already gathered during a normal `check`. See [SECURITY.md](./SECURITY.md) for the full passive-only guarantee.
+- **The CLI cannot currently restrict the check set** (no `--only-checks` flag), so `coverage` is always 100% via `check`/`score`. Partial coverage (and the coverage-aware `minimumScore` warning) only happens when the library API's `runChecks({ checks: [...] })` is called with a restricted list.
+- **No currently-supported MCP protocol version defines a hard tool-name length or character-pattern constraint**, so `src/protocol/quality-rules.ts`'s protocol-vs-quality distinction for tool names is real but currently dormant — every finding today is a quality recommendation, never a protocol violation, because no version actually requires one. The abstraction is there for when a future version does.
 - **npm README sync**: Latest docs live on GitHub main; npm README updates on the next publish.
 - **First run via `npx`** pays a one-time cost to resolve and download the package; once installed (or on a warm npx cache), `--help`/`--version` return in well under 100ms.
 

package/dist/checks/index.d.ts

@@ -13,4 +13,5 @@ export { qualityToolAnnotationsCheck } from './quality-tool-annotations.js';
 export { qualityToolSurfaceCheck, createToolSurfaceCheck, DEFAULT_MAX_TOOLS_WARNING_THRESHOLD } from './quality-tool-surface.js';
 export { qualityResourcesCheck } from './quality-resources.js';
 export { qualityPromptsCheck } from './quality-prompts.js';
+export { protocolConnectionHealthCheck } from './protocol-connection-health.js';
 export declare const allChecks: import("../types.js").Check[];

package/dist/checks/index.js

@@ -13,6 +13,7 @@ export { qualityToolAnnotationsCheck } from './quality-tool-annotations.js';
 export { qualityToolSurfaceCheck, createToolSurfaceCheck, DEFAULT_MAX_TOOLS_WARNING_THRESHOLD } from './quality-tool-surface.js';
 export { qualityResourcesCheck } from './quality-resources.js';
 export { qualityPromptsCheck } from './quality-prompts.js';
+export { protocolConnectionHealthCheck } from './protocol-connection-health.js';
 import { malformedSchemaCheck } from './malformed-schema.js';
 import { missingRequiredFieldsCheck } from './missing-required-fields.js';
 import { typeMismatchCheck } from './type-mismatch.js';
@@ -28,7 +29,9 @@ import { qualityToolAnnotationsCheck } from './quality-tool-annotations.js';
 import { qualityToolSurfaceCheck } from './quality-tool-surface.js';
 import { qualityResourcesCheck } from './quality-resources.js';
 import { qualityPromptsCheck } from './quality-prompts.js';
+import { protocolConnectionHealthCheck } from './protocol-connection-health.js';
 export const allChecks = [
+    protocolConnectionHealthCheck,
     malformedSchemaCheck,
     missingRequiredFieldsCheck,
     typeMismatchCheck,

package/dist/checks/protocol-connection-health.d.ts

@@ -0,0 +1,17 @@
+import type { Check } from '../types.js';
+/**
+ * Surfaces connection-level protocol facts as real, visible diagnostics —
+ * previously these (capability-inspection failures, a version downgrade, a
+ * missing serverInfo) were deducted from the quality score directly from
+ * `MCPConnection` metadata inside src/quality-score.ts, with no
+ * corresponding diagnostic a user could actually see in the report or
+ * `--json` output. That meant a developer looking at "why did my protocol
+ * score drop" would find nothing in `diagnostics` explaining it.
+ *
+ * This check makes those same facts flow through the standard
+ * connection -> diagnostics -> dimension -> score pipeline like every
+ * other check, so every point deducted is explainable from a visible
+ * DiagnosticResult (see .agent-room/DECISIONS.md for the full rationale,
+ * including why this also changes some of the exact point values).
+ */
+export declare const protocolConnectionHealthCheck: Check;

package/dist/checks/protocol-connection-health.js

@@ -0,0 +1,78 @@
+/**
+ * Surfaces connection-level protocol facts as real, visible diagnostics —
+ * previously these (capability-inspection failures, a version downgrade, a
+ * missing serverInfo) were deducted from the quality score directly from
+ * `MCPConnection` metadata inside src/quality-score.ts, with no
+ * corresponding diagnostic a user could actually see in the report or
+ * `--json` output. That meant a developer looking at "why did my protocol
+ * score drop" would find nothing in `diagnostics` explaining it.
+ *
+ * This check makes those same facts flow through the standard
+ * connection -> diagnostics -> dimension -> score pipeline like every
+ * other check, so every point deducted is explainable from a visible
+ * DiagnosticResult (see .agent-room/DECISIONS.md for the full rationale,
+ * including why this also changes some of the exact point values).
+ */
+export const protocolConnectionHealthCheck = {
+    id: 'protocol.connection-health',
+    description: 'Flags protocol-level connection facts: a negotiated version downgrade, a missing serverInfo, or a declared capability whose list call failed.',
+    run(connection) {
+        const results = [];
+        try {
+            if (connection.protocolVersion?.negotiated &&
+                connection.protocolVersion.negotiated !== connection.protocolVersion.requested) {
+                results.push({
+                    checkId: 'protocol.version-downgrade',
+                    severity: 'info',
+                    message: `Server negotiated protocol version ${connection.protocolVersion.negotiated} instead of the requested ${connection.protocolVersion.requested}.`,
+                    serverName: connection.server.name,
+                    category: 'protocol',
+                    details: {
+                        requested: connection.protocolVersion.requested,
+                        negotiated: connection.protocolVersion.negotiated,
+                    },
+                });
+            }
+            if (!connection.serverInfo?.name) {
+                results.push({
+                    checkId: 'protocol.missing-server-info',
+                    severity: 'info',
+                    message: 'Server did not report its name/version in serverInfo during initialize.',
+                    serverName: connection.server.name,
+                    category: 'protocol',
+                    suggestedFix: { description: 'Have the server include a serverInfo.name in its initialize response.' },
+                });
+            }
+            if (connection.capabilityErrors?.resources) {
+                results.push({
+                    checkId: 'protocol.capability-error',
+                    severity: 'error',
+                    message: `Server declared the "resources" capability, but resources/list failed: ${connection.capabilityErrors.resources}`,
+                    serverName: connection.server.name,
+                    category: 'protocol',
+                    details: { capability: 'resources', error: connection.capabilityErrors.resources },
+                });
+            }
+            if (connection.capabilityErrors?.prompts) {
+                results.push({
+                    checkId: 'protocol.capability-error',
+                    severity: 'error',
+                    message: `Server declared the "prompts" capability, but prompts/list failed: ${connection.capabilityErrors.prompts}`,
+                    serverName: connection.server.name,
+                    category: 'protocol',
+                    details: { capability: 'prompts', error: connection.capabilityErrors.prompts },
+                });
+            }
+        }
+        catch (err) {
+            results.push({
+                checkId: 'protocol.connection-health',
+                severity: 'error',
+                message: `check failed internally: ${err instanceof Error ? err.message : String(err)}`,
+                serverName: connection.server.name,
+                category: 'protocol',
+            });
+        }
+        return results;
+    },
+};

package/dist/checks/quality-tool-names.d.ts

@@ -1,2 +1,35 @@
 import type { Check } from '../types.js';
+import { type ToolNameProtocolRules } from '../protocol/quality-rules.js';
+export interface ToolNameFinding {
+    severity: 'error' | 'warning';
+    /** 'protocol' only when `rules` itself defines a hard constraint the name
+     * violates; everything else is 'quality' (a recommendation, never a
+     * protocol violation) — see module doc. */
+    category: 'protocol' | 'quality';
+    message: string;
+    confidence?: 'low' | 'medium' | 'high';
+    suggestedFixDescription: string;
+}
+/**
+ * Pure evaluation of a single tool name, independent of the rest of the
+ * connection (duplicate detection is connection-wide and stays in `run()`).
+ * Exported so tests can inject a synthetic `ToolNameProtocolRules` (e.g. a
+ * hypothetical future protocol version with a real length/pattern
+ * constraint) without needing that version to actually exist yet — see
+ * src/protocol/quality-rules.ts.
+ *
+ * The A/B/C distinction this check makes:
+ *   A. PROTOCOL VIOLATION — only possible if `rules.maxLength`/`rules.pattern`
+ *      is defined by the negotiated version AND the name violates it.
+ *      Reported as category 'protocol', severity 'error'.
+ *   B. QUALITY WARNING — technically spec-valid, but likely to hurt agent
+ *      usability (too long by convention, odd characters, ambiguous name).
+ *      Reported as category 'quality', severity 'warning'.
+ *   C. Empty name is kept as a 'quality' error (not 'protocol') — no
+ *      supported version's spec actually forbids an empty string, but a
+ *      tool a client can't reference or distinguish is functionally
+ *      broken, which still deserves error severity without mislabeling it
+ *      a protocol violation.
+ */
+export declare function evaluateToolName(name: string, rules: ToolNameProtocolRules): ToolNameFinding[];
 export declare const qualityToolNamesCheck: Check;

package/dist/checks/quality-tool-names.js

@@ -1,9 +1,6 @@
-/** MCP's `Tool.name` (from `BaseMetadata`) has no documented pattern or
- * length constraint in the spec — so none of these are "protocol
- * violations" in the strict sense. Empty and duplicate names are still
- * treated as errors here because a tool a client cannot reference or
- * distinguish is functionally broken, not merely unstylish; everything
- * else is a warning. */
+import { getProtocolQualityRules } from '../protocol/quality-rules.js';
+/** Ecosystem recommendation, not a protocol constraint — see the module-level
+ * note on `evaluateToolName` for the A/B/C distinction this check makes. */
 const MAX_REASONABLE_NAME_LENGTH = 128;
 /** Conservative, curated list — exact (case-insensitive) matches only, to
  * avoid false-positiving on legitimately short/plain real tool names. */
@@ -31,64 +28,106 @@ function hasInvalidCharacters(name) {
     // eslint-disable-next-line no-control-regex
     return /[\t\n\r\x00-\x08\x0b\x0c\x0e-\x1f]/.test(name);
 }
+/**
+ * Pure evaluation of a single tool name, independent of the rest of the
+ * connection (duplicate detection is connection-wide and stays in `run()`).
+ * Exported so tests can inject a synthetic `ToolNameProtocolRules` (e.g. a
+ * hypothetical future protocol version with a real length/pattern
+ * constraint) without needing that version to actually exist yet — see
+ * src/protocol/quality-rules.ts.
+ *
+ * The A/B/C distinction this check makes:
+ *   A. PROTOCOL VIOLATION — only possible if `rules.maxLength`/`rules.pattern`
+ *      is defined by the negotiated version AND the name violates it.
+ *      Reported as category 'protocol', severity 'error'.
+ *   B. QUALITY WARNING — technically spec-valid, but likely to hurt agent
+ *      usability (too long by convention, odd characters, ambiguous name).
+ *      Reported as category 'quality', severity 'warning'.
+ *   C. Empty name is kept as a 'quality' error (not 'protocol') — no
+ *      supported version's spec actually forbids an empty string, but a
+ *      tool a client can't reference or distinguish is functionally
+ *      broken, which still deserves error severity without mislabeling it
+ *      a protocol violation.
+ */
+export function evaluateToolName(name, rules) {
+    const findings = [];
+    if (name.trim() === '') {
+        findings.push({
+            severity: 'error',
+            category: 'quality',
+            message: 'Tool has an empty name — a client cannot reference or distinguish it.',
+            suggestedFixDescription: 'Give the tool a non-empty, descriptive name.',
+        });
+        return findings; // nothing else meaningful to evaluate on an empty name
+    }
+    if (rules.maxLength !== undefined && name.length > rules.maxLength) {
+        findings.push({
+            severity: 'error',
+            category: 'protocol',
+            message: `Tool name "${name.slice(0, 40)}..." is ${name.length} characters, exceeding the negotiated protocol's maximum of ${rules.maxLength} — this is a protocol violation, not a style recommendation.`,
+            suggestedFixDescription: `Shorten the tool name to ${rules.maxLength} characters or fewer to comply with the negotiated protocol version.`,
+        });
+    }
+    else if (name.length > MAX_REASONABLE_NAME_LENGTH) {
+        findings.push({
+            severity: 'warning',
+            category: 'quality',
+            confidence: 'medium',
+            message: `Tool name "${name.slice(0, 40)}..." name is ${name.length} characters, exceeding the recommended ${MAX_REASONABLE_NAME_LENGTH}.`,
+            suggestedFixDescription: 'Shorten the tool name to something concise and memorable.',
+        });
+    }
+    if (rules.pattern && !rules.pattern.test(name)) {
+        findings.push({
+            severity: 'error',
+            category: 'protocol',
+            message: `Tool name "${name}" does not match the naming pattern required by the negotiated protocol version — this is a protocol violation, not a style recommendation.`,
+            suggestedFixDescription: 'Rename the tool to match the naming pattern required by the negotiated protocol version.',
+        });
+    }
+    else if (hasInvalidCharacters(name)) {
+        findings.push({
+            severity: 'warning',
+            category: 'quality',
+            message: `Tool name "${JSON.stringify(name)}" contains whitespace/control characters that may break client tooling.`,
+            suggestedFixDescription: 'Use only plain, printable characters in tool names (letters, digits, -, _).',
+        });
+    }
+    if (name.length === 1 || PLACEHOLDER_NAMES.has(name.trim().toLowerCase())) {
+        findings.push({
+            severity: 'warning',
+            category: 'quality',
+            confidence: 'medium',
+            message: `Tool name "${name}" is ambiguous or looks like a placeholder — it doesn't communicate what the tool does.`,
+            suggestedFixDescription: 'Rename the tool to describe its action, e.g. "search_flights" instead of "tool".',
+        });
+    }
+    return findings;
+}
 export const qualityToolNamesCheck = {
     id: 'quality.tool-name',
-    description: 'Flags empty, duplicate, overly long, or placeholder-looking tool names.',
+    description: 'Flags empty, duplicate, overly long, or placeholder-looking tool names; distinguishes protocol-version-defined violations from quality recommendations.',
     run(connection) {
         const results = [];
         try {
             if (!connection.tools || !Array.isArray(connection.tools)) {
                 return results;
             }
+            const rules = getProtocolQualityRules(connection.protocolVersion?.negotiated).toolName;
             const seen = new Map();
             for (const tool of connection.tools) {
                 const name = tool.name;
                 seen.set(name, (seen.get(name) ?? 0) + 1);
-                if (name.trim() === '') {
-                    results.push({
-                        checkId: 'quality.tool-name',
-                        severity: 'error',
-                        message: 'Tool has an empty name — a client cannot reference or distinguish it.',
-                        serverName: connection.server.name,
-                        toolName: name,
-                        category: 'quality',
-                        suggestedFix: { description: 'Give the tool a non-empty, descriptive name.' },
-                    });
-                    continue;
-                }
-                if (name.length > MAX_REASONABLE_NAME_LENGTH) {
-                    results.push({
-                        checkId: 'quality.tool-name',
-                        severity: 'warning',
-                        message: `Tool "${name.slice(0, 40)}..." name is ${name.length} characters, exceeding the recommended ${MAX_REASONABLE_NAME_LENGTH}.`,
-                        serverName: connection.server.name,
-                        toolName: name,
-                        category: 'quality',
-                        confidence: 'medium',
-                        suggestedFix: { description: 'Shorten the tool name to something concise and memorable.' },
-                    });
-                }
-                if (hasInvalidCharacters(name)) {
-                    results.push({
-                        checkId: 'quality.tool-name',
-                        severity: 'warning',
-                        message: `Tool name "${JSON.stringify(name)}" contains whitespace/control characters that may break client tooling.`,
-                        serverName: connection.server.name,
-                        toolName: name,
-                        category: 'quality',
-                        suggestedFix: { description: 'Use only plain, printable characters in tool names (letters, digits, -, _).' },
-                    });
-                }
-                if (name.length === 1 || PLACEHOLDER_NAMES.has(name.trim().toLowerCase())) {
+                for (const finding of evaluateToolName(name, rules)) {
                     results.push({
                         checkId: 'quality.tool-name',
-                        severity: 'warning',
-                        message: `Tool name "${name}" is ambiguous or looks like a placeholder — it doesn't communicate what the tool does.`,
+                        severity: finding.severity,
+                        message: finding.message,
                         serverName: connection.server.name,
                         toolName: name,
-                        category: 'quality',
-                        confidence: 'medium',
-                        suggestedFix: { description: 'Rename the tool to describe its action, e.g. "search_flights" instead of "tool".' },
+                        category: finding.category,
+                        ...(finding.confidence ? { confidence: finding.confidence } : {}),
+                        suggestedFix: { description: finding.suggestedFixDescription },
                     });
                 }
             }

package/dist/cli.js

@@ -14,7 +14,7 @@ import { loadPolicy, createPolicyChecks } from './policy.js';
 import { runFleetChecks, diffConfigs, filterDiagnosticsByBaseline } from './fleet.js';
 import { formatReportJUnit, formatFleetReportJUnit } from './junit.js';
 import { formatReportSarif } from './sarif.js';
-import { computeReportQualityScore } from './quality-score.js';
+import { computeReportQualityScore, checkMinimumScorePolicy } from './quality-score.js';
 import { SUPPORTED_PROTOCOL_VERSIONS } from './protocol/versions.js';
 import pc from 'picocolors';
 export function parseArgs(argv) {
@@ -286,7 +286,7 @@ async function executeCheck(config, args) {
                 // The quality score must reflect what's actually being reported —
                 // recompute it against the post-baseline-filter diagnostics rather
                 // than leaving the pre-filter score (computed by runChecks) stale.
-                report.quality = computeReportQualityScore(report);
+                report.quality = computeReportQualityScore(report, checks);
             }
             catch (err) {
                 console.error(pc.yellow(`Warning: Could not read snapshot baseline: ${String(err)}`));
@@ -298,16 +298,11 @@ async function executeCheck(config, args) {
     // check's output, not a single connection), so it's enforced here instead.
     const policy = loadPolicy(args.policyPath);
     if (typeof policy?.quality?.minimumScore === 'number' && report.quality) {
-        if (report.quality.overall < policy.quality.minimumScore) {
-            report.diagnostics.push({
-                checkId: 'policy.minimum-quality-score',
-                severity: 'error',
-                message: `MCP quality score ${report.quality.overall} is below the policy minimum of ${policy.quality.minimumScore}.`,
-                serverName: config.servers.map((s) => s.name).join(', ') || '(no servers)',
-                category: 'configuration',
-            });
-            report.summary.errors += 1;
-        }
+        const allServerNames = config.servers.map((s) => s.name).join(', ') || '(no servers)';
+        const policyDiagnostics = checkMinimumScorePolicy(report.quality, policy.quality.minimumScore, allServerNames);
+        report.diagnostics.push(...policyDiagnostics);
+        report.summary.errors += policyDiagnostics.filter((d) => d.severity === 'error').length;
+        report.summary.warnings += policyDiagnostics.filter((d) => d.severity === 'warning').length;
     }
     // Handle update snapshot
     if (args.updateSnapshotPath) {

package/dist/index.d.ts

@@ -22,5 +22,10 @@ export { SUPPORTED_PROTOCOL_VERSIONS, LATEST_SUPPORTED_PROTOCOL_VERSION, KNOWN_U
 export type { SupportedProtocolVersion, ProtocolVersionNegotiation } from './protocol/versions.js';
 export { isSecretKey, redactRecord, redactDeep, sanitizeServerConfig } from './redact.js';
 export { inferDiagnosticCategory, categoryOf } from './diagnostics.js';
-export { computeConnectionQualityScore, computeReportQualityScore, QUALITY_DIMENSIONS, QUALITY_DIMENSION_WEIGHTS, } from './quality-score.js';
+export { computeConnectionQualityScore, computeReportQualityScore, computeQualityCoverage, checkMinimumScorePolicy, QUALITY_DIMENSIONS, QUALITY_DIMENSION_WEIGHTS, QUALITY_SCORE_DISCLAIMER, } from './quality-score.js';
+export { getProtocolQualityRules } from './protocol/quality-rules.js';
+export type { ProtocolQualityRules, ToolNameProtocolRules } from './protocol/quality-rules.js';
+export { protocolConnectionHealthCheck } from './checks/protocol-connection-health.js';
+export { evaluateToolName } from './checks/quality-tool-names.js';
+export type { ToolNameFinding } from './checks/quality-tool-names.js';
 export * from './types.js';

package/dist/index.js

@@ -14,5 +14,8 @@ export { allChecks } from './checks/index.js';
 export { SUPPORTED_PROTOCOL_VERSIONS, LATEST_SUPPORTED_PROTOCOL_VERSION, KNOWN_UNSUPPORTED_PROTOCOL_VERSIONS, isSupportedProtocolVersion, resolveRequestedProtocolVersion, } from './protocol/versions.js';
 export { isSecretKey, redactRecord, redactDeep, sanitizeServerConfig } from './redact.js';
 export { inferDiagnosticCategory, categoryOf } from './diagnostics.js';
-export { computeConnectionQualityScore, computeReportQualityScore, QUALITY_DIMENSIONS, QUALITY_DIMENSION_WEIGHTS, } from './quality-score.js';
+export { computeConnectionQualityScore, computeReportQualityScore, computeQualityCoverage, checkMinimumScorePolicy, QUALITY_DIMENSIONS, QUALITY_DIMENSION_WEIGHTS, QUALITY_SCORE_DISCLAIMER, } from './quality-score.js';
+export { getProtocolQualityRules } from './protocol/quality-rules.js';
+export { protocolConnectionHealthCheck } from './checks/protocol-connection-health.js';
+export { evaluateToolName } from './checks/quality-tool-names.js';
 export * from './types.js';

package/dist/orchestrator.js

@@ -64,7 +64,7 @@ export async function runChecks(config, options = {}) {
         diagnostics,
         summary,
     };
-    const quality = computeReportQualityScore(report);
+    const quality = computeReportQualityScore(report, checks);
     if (quality) {
         report.quality = quality;
     }

package/dist/protocol/quality-rules.d.ts

@@ -0,0 +1,46 @@
+/**
+ * Protocol-version-aware quality rules.
+ *
+ * Some things a quality check might flag are genuine protocol violations
+ * for a given negotiated MCP version (hard requirement/prohibition); others
+ * are ecosystem recommendations that happen to be entirely valid per spec.
+ * Mixing these up means a check can wrongly call a style preference a
+ * "protocol violation," or (worse) miss a real one because the version
+ * that defines it was never consulted.
+ *
+ * This module is the single place that answers "does this negotiated
+ * version define a hard constraint here?" so quality checks don't each
+ * have to know MCP spec history — and so a future protocol version that
+ * *does* add a constraint only needs a new entry here, not a rewrite of
+ * every check that uses it.
+ *
+ * Current state of research (see src/protocol/versions.ts for the same
+ * source): every version in SUPPORTED_PROTOCOL_VERSIONS (2024-11-05
+ * through 2025-11-25) defines `Tool.name` as a plain `string` with no
+ * documented length or character-pattern constraint. So today, every
+ * version returns the same (empty) rule set — this module exists for the
+ * abstraction, not because any current version actually differs from
+ * another.
+ */
+export interface ToolNameProtocolRules {
+    /** A hard maximum length the negotiated version's spec actually defines
+     * for `Tool.name`, if any. Exceeding it is a protocol violation, not a
+     * style recommendation. `undefined` = the spec defines no such limit for
+     * this version — length-based feedback should be a quality warning, not
+     * an error. */
+    maxLength?: number;
+    /** A hard character-pattern the negotiated version's spec actually
+     * requires `Tool.name` to match, if any. `undefined` = no such
+     * constraint — character-based feedback should be a quality warning. */
+    pattern?: RegExp;
+}
+export interface ProtocolQualityRules {
+    toolName: ToolNameProtocolRules;
+}
+/**
+ * Returns the protocol-defined (hard) quality rules for a negotiated MCP
+ * version. `version` is normally `MCPConnection.protocolVersion.negotiated`.
+ * An unknown or absent version gets the same "no hard constraints" rules
+ * as every currently-supported version, rather than guessing.
+ */
+export declare function getProtocolQualityRules(version: string | undefined): ProtocolQualityRules;

package/dist/protocol/quality-rules.js

@@ -0,0 +1,37 @@
+/**
+ * Protocol-version-aware quality rules.
+ *
+ * Some things a quality check might flag are genuine protocol violations
+ * for a given negotiated MCP version (hard requirement/prohibition); others
+ * are ecosystem recommendations that happen to be entirely valid per spec.
+ * Mixing these up means a check can wrongly call a style preference a
+ * "protocol violation," or (worse) miss a real one because the version
+ * that defines it was never consulted.
+ *
+ * This module is the single place that answers "does this negotiated
+ * version define a hard constraint here?" so quality checks don't each
+ * have to know MCP spec history — and so a future protocol version that
+ * *does* add a constraint only needs a new entry here, not a rewrite of
+ * every check that uses it.
+ *
+ * Current state of research (see src/protocol/versions.ts for the same
+ * source): every version in SUPPORTED_PROTOCOL_VERSIONS (2024-11-05
+ * through 2025-11-25) defines `Tool.name` as a plain `string` with no
+ * documented length or character-pattern constraint. So today, every
+ * version returns the same (empty) rule set — this module exists for the
+ * abstraction, not because any current version actually differs from
+ * another.
+ */
+/** No currently-supported MCP version defines a hard length/pattern
+ * constraint on Tool.name — see the module doc comment. */
+const NO_HARD_CONSTRAINTS = { toolName: {} };
+/**
+ * Returns the protocol-defined (hard) quality rules for a negotiated MCP
+ * version. `version` is normally `MCPConnection.protocolVersion.negotiated`.
+ * An unknown or absent version gets the same "no hard constraints" rules
+ * as every currently-supported version, rather than guessing.
+ */
+export function getProtocolQualityRules(version) {
+    void version; // reserved for when a supported version actually defines a constraint
+    return NO_HARD_CONSTRAINTS;
+}

package/dist/quality-score.d.ts

@@ -1,33 +1,76 @@
-import type { MCPConnection, DiagnosticResult, RunReport, QualityDimension, QualityDeduction, QualityScoreBreakdown, ReportQualityScore } from './types.js';
+import type { MCPConnection, DiagnosticResult, RunReport, QualityDimension, QualityDeduction, QualityScoreBreakdown, ReportQualityScore, QualityCoverage, Check } from './types.js';
 /**
  * Deterministic MCP quality score. No LLM, no randomness, no network
  * calls beyond the MCP inspection `connect()` already performed — the same
  * report always produces the same score.
  *
- * Design: diagnostics -> normalized per-checkId deductions -> capped
- * per-dimension score -> weighted overall score. This keeps the score
- * fully derived from (and consistent with) the diagnostics that are also
- * shown in the report, rather than an independent "second opinion" — see
- * .agent-room/DECISIONS.md for the full rationale.
+ * Design: connection -> diagnostics -> normalized per-checkId deductions ->
+ * capped per-dimension score -> weighted overall score. Every deduction is
+ * derived from a visible `DiagnosticResult` — nothing is deducted directly
+ * from connection metadata that isn't also represented as a diagnostic
+ * (see .agent-room/DECISIONS.md, "v1.1 trust hardening" entry, for the
+ * `protocol.connection-health` check this replaced).
  *
  * (Result types — QualityDimension, QualityDeduction, QualityScoreBreakdown,
- * ReportQualityScore — live in src/types.ts alongside RunReport, since
- * RunReport.quality is one of them; this module just implements the math.)
+ * ReportQualityScore, QualityCoverage, CoverageStatus — live in src/types.ts
+ * alongside RunReport, since RunReport.quality is one of them; this module
+ * just implements the math.)
  */
 export type { QualityDimension, QualityDeduction, QualityScoreBreakdown, ReportQualityScore };
 export declare const QUALITY_DIMENSIONS: readonly QualityDimension[];
 /** Suggested weights from the product spec; documented here since every
- * point deduction must be explainable in terms of these. */
+ * point deduction must be explainable in terms of these. Do not change
+ * without a documented reason — see .agent-room/DECISIONS.md. */
 export declare const QUALITY_DIMENSION_WEIGHTS: Record<QualityDimension, number>;
 /**
- * Computes a quality score for one connection. Returns `undefined` for a
- * connection that never connected — there's nothing meaningful to score
- * (no tools/schema/capabilities were ever observed).
+ * The score is a measurement, not a certification: it reports what
+ * mcp-medic's passive checks actually found (or, per `coverage`, actually
+ * looked for) — never a formal audit of the server's real security or
+ * behavior. Surfaced in both JSON (`ReportQualityScore.disclaimer`) and the
+ * human report.
+ */
+export declare const QUALITY_SCORE_DISCLAIMER: string;
+/**
+ * Computes a quality score for one connection, purely from `diagnostics`
+ * (filtered to that server) — no connection metadata is consulted
+ * directly. Returns `undefined` for a connection that never connected —
+ * there's nothing meaningful to score (no tools/schema/capabilities were
+ * ever observed); see `computeReportQualityScore` for how the report-level
+ * score surfaces that instead of silently dropping it.
  */
 export declare function computeConnectionQualityScore(connection: MCPConnection, diagnostics: DiagnosticResult[]): QualityScoreBreakdown | undefined;
+/**
+ * Coverage answers "was this dimension actually evaluated," independent of
+ * what score it got — a 100 from zero checks run means "nothing was
+ * looked for," not "nothing is wrong." Derived from the *ids* of the
+ * checks that actually ran (`executedChecks`), compared against the
+ * built-in `allChecks` reference set grouped by dimension. Reliability is
+ * special-cased to always 'covered': its signal (did the connection
+ * succeed) isn't produced by any optional `Check` — it's inherent to
+ * attempting a connection at all, so it's never "not covered."
+ */
+export declare function computeQualityCoverage(executedChecks: Check[]): QualityCoverage;
 /**
  * Aggregate score across every connected server in a report (simple mean —
  * deliberately not weighted by tool count, so one huge server can't drown
- * out the rest of a fleet). Returns `undefined` if no server connected.
+ * out the rest of a fleet). `executedChecks` should be the exact list
+ * passed to `runChecks({ checks })` for this report, so `coverage` reflects
+ * what actually ran (defaults to the full built-in set if omitted, for
+ * callers that haven't been updated to pass it through).
+ * Returns `undefined` if no server connected — nothing to score at all.
+ */
+export declare function computeReportQualityScore(report: RunReport, executedChecks?: Check[]): ReportQualityScore | undefined;
+/**
+ * Enforces `.mcp-medic-policy.json`'s `quality.minimumScore` as a CI gate.
+ * Pure function so it's directly unit-testable without needing a real CLI
+ * invocation or a restricted check set — see PHASE 12 ("policy/CI") in
+ * .agent-room/DECISIONS.md.
+ *
+ * Returns diagnostics to append to the report (0, 1, or 2):
+ *  - an 'error' if the score is below `minimumScore` (fails the run).
+ *  - a 'warning' if the score was computed from a partial assessment
+ *    (incomplete coverage, or a server that couldn't be scored) — a
+ *    minimumScore gate must never silently "pass" as if a full check had
+ *    run when it didn't.
  */
-export declare function computeReportQualityScore(report: RunReport): ReportQualityScore | undefined;
+export declare function checkMinimumScorePolicy(quality: ReportQualityScore, minimumScore: number, serverNamesLabel: string): DiagnosticResult[];

package/dist/quality-score.js

@@ -1,4 +1,5 @@
-import { categoryOf } from './diagnostics.js';
+import { categoryOf, inferDiagnosticCategory } from './diagnostics.js';
+import { allChecks } from './checks/index.js';
 export const QUALITY_DIMENSIONS = [
     'protocol',
     'schema',
@@ -7,7 +8,8 @@ export const QUALITY_DIMENSIONS = [
     'reliability',
 ];
 /** Suggested weights from the product spec; documented here since every
- * point deduction must be explainable in terms of these. */
+ * point deduction must be explainable in terms of these. Do not change
+ * without a documented reason — see .agent-room/DECISIONS.md. */
 export const QUALITY_DIMENSION_WEIGHTS = {
     protocol: 0.25,
     schema: 0.2,
@@ -15,6 +17,15 @@ export const QUALITY_DIMENSION_WEIGHTS = {
     security: 0.2,
     reliability: 0.15,
 };
+/**
+ * The score is a measurement, not a certification: it reports what
+ * mcp-medic's passive checks actually found (or, per `coverage`, actually
+ * looked for) — never a formal audit of the server's real security or
+ * behavior. Surfaced in both JSON (`ReportQualityScore.disclaimer`) and the
+ * human report.
+ */
+export const QUALITY_SCORE_DISCLAIMER = "This score reflects issues detectable by mcp-medic's passive inspection (see \"coverage\") — " +
+    'it is not a certification of the server\'s actual security, correctness, or behavior.';
 // Deduction weights: chosen so a handful of real problems meaningfully move
 // the score, while a single noisy checkId can never wipe out a whole
 // dimension on its own (the per-checkId cap) — this directly implements
@@ -63,71 +74,23 @@ function deductionsFromDiagnostics(dimension, diagnostics) {
     }
     return deductions;
 }
-/** Protocol-dimension facts that live on `MCPConnection` rather than as
- * diagnostics (capability negotiation isn't produced by a `Check`). */
-function protocolConnectionDeductions(connection) {
-    const deductions = [];
-    if (connection.capabilityErrors?.resources) {
-        deductions.push({
-            dimension: 'protocol',
-            checkId: 'protocol.capability-error',
-            severity: 'warning',
-            count: 1,
-            points: 15,
-            description: 'resources/list failed despite being declared as a capability',
-        });
-    }
-    if (connection.capabilityErrors?.prompts) {
-        deductions.push({
-            dimension: 'protocol',
-            checkId: 'protocol.capability-error',
-            severity: 'warning',
-            count: 1,
-            points: 15,
-            description: 'prompts/list failed despite being declared as a capability',
-        });
-    }
-    if (connection.protocolVersion &&
-        connection.protocolVersion.negotiated &&
-        connection.protocolVersion.negotiated !== connection.protocolVersion.requested) {
-        deductions.push({
-            dimension: 'protocol',
-            checkId: 'protocol.version-downgrade',
-            severity: 'info',
-            count: 1,
-            points: 5,
-            description: `server negotiated ${connection.protocolVersion.negotiated} instead of the requested ${connection.protocolVersion.requested}`,
-        });
-    }
-    if (!connection.serverInfo?.name) {
-        deductions.push({
-            dimension: 'protocol',
-            checkId: 'protocol.missing-server-info',
-            severity: 'info',
-            count: 1,
-            points: 5,
-            description: 'server did not report its name/version in serverInfo',
-        });
-    }
-    return deductions;
-}
 function dimensionScore(deductions) {
     const totalPoints = deductions.reduce((sum, d) => sum + d.points, 0);
     return Math.max(0, Math.min(100, Math.round(100 - totalPoints)));
 }
 /**
- * Computes a quality score for one connection. Returns `undefined` for a
- * connection that never connected — there's nothing meaningful to score
- * (no tools/schema/capabilities were ever observed).
+ * Computes a quality score for one connection, purely from `diagnostics`
+ * (filtered to that server) — no connection metadata is consulted
+ * directly. Returns `undefined` for a connection that never connected —
+ * there's nothing meaningful to score (no tools/schema/capabilities were
+ * ever observed); see `computeReportQualityScore` for how the report-level
+ * score surfaces that instead of silently dropping it.
  */
 export function computeConnectionQualityScore(connection, diagnostics) {
     if (connection.status !== 'connected')
         return undefined;
     const relevant = diagnostics.filter((d) => d.serverName === connection.server.name);
-    const allDeductions = [
-        ...protocolConnectionDeductions(connection),
-        ...QUALITY_DIMENSIONS.flatMap((dim) => deductionsFromDiagnostics(dim, relevant)),
-    ];
+    const allDeductions = QUALITY_DIMENSIONS.flatMap((dim) => deductionsFromDiagnostics(dim, relevant));
     const dimensions = Object.fromEntries(QUALITY_DIMENSIONS.map((dim) => [dim, dimensionScore(allDeductions.filter((d) => d.dimension === dim))]));
     const overall = Math.round(QUALITY_DIMENSIONS.reduce((sum, dim) => sum + dimensions[dim] * QUALITY_DIMENSION_WEIGHTS[dim], 0));
     return {
@@ -136,28 +99,144 @@ export function computeConnectionQualityScore(connection, diagnostics) {
         deductions: allDeductions.filter((d) => d.points > 0).sort((a, b) => b.points - a.points),
     };
 }
+const COVERAGE_WEIGHT = { covered: 1, partial: 0.5, 'not-covered': 0 };
+/**
+ * Coverage answers "was this dimension actually evaluated," independent of
+ * what score it got — a 100 from zero checks run means "nothing was
+ * looked for," not "nothing is wrong." Derived from the *ids* of the
+ * checks that actually ran (`executedChecks`), compared against the
+ * built-in `allChecks` reference set grouped by dimension. Reliability is
+ * special-cased to always 'covered': its signal (did the connection
+ * succeed) isn't produced by any optional `Check` — it's inherent to
+ * attempting a connection at all, so it's never "not covered."
+ */
+export function computeQualityCoverage(executedChecks) {
+    const referenceByDimension = new Map();
+    for (const check of allChecks) {
+        const dim = categoryToDimension(inferDiagnosticCategory(check.id));
+        if (dim === 'reliability')
+            continue; // no built-in check drives reliability; handled unconditionally below
+        if (!referenceByDimension.has(dim))
+            referenceByDimension.set(dim, new Set());
+        referenceByDimension.get(dim).add(check.id);
+    }
+    const coverage = {};
+    for (const dim of QUALITY_DIMENSIONS) {
+        if (dim === 'reliability') {
+            coverage[dim] = 'covered';
+            continue;
+        }
+        const reference = referenceByDimension.get(dim) ?? new Set();
+        // Every executed check that maps to this dimension — including custom/
+        // community checks not in the built-in reference set, so an unmapped
+        // check still counts as (at least partial) coverage for the dimension
+        // it was inferred into, rather than being invisible to this count.
+        const executedForDim = executedChecks.filter((c) => categoryToDimension(inferDiagnosticCategory(c.id)) === dim);
+        if (executedForDim.length === 0) {
+            coverage[dim] = 'not-covered';
+            continue;
+        }
+        if (reference.size === 0) {
+            // No built-in check exists for this dimension at all, but something
+            // (necessarily a custom check) ran for it — can't call that "full"
+            // coverage without a reference to compare against.
+            coverage[dim] = 'partial';
+            continue;
+        }
+        const executedKnownIds = new Set(executedForDim.map((c) => c.id).filter((id) => reference.has(id)));
+        coverage[dim] = executedKnownIds.size === reference.size ? 'covered' : 'partial';
+    }
+    return coverage;
+}
+function coveragePercent(coverage) {
+    const total = QUALITY_DIMENSIONS.reduce((sum, dim) => sum + COVERAGE_WEIGHT[coverage[dim]], 0);
+    return Math.round((total / QUALITY_DIMENSIONS.length) * 100);
+}
 /**
  * Aggregate score across every connected server in a report (simple mean —
  * deliberately not weighted by tool count, so one huge server can't drown
- * out the rest of a fleet). Returns `undefined` if no server connected.
+ * out the rest of a fleet). `executedChecks` should be the exact list
+ * passed to `runChecks({ checks })` for this report, so `coverage` reflects
+ * what actually ran (defaults to the full built-in set if omitted, for
+ * callers that haven't been updated to pass it through).
+ * Returns `undefined` if no server connected — nothing to score at all.
  */
-export function computeReportQualityScore(report) {
+export function computeReportQualityScore(report, executedChecks = allChecks) {
     const perServer = {};
     for (const connection of report.connections) {
         const breakdown = computeConnectionQualityScore(connection, report.diagnostics);
         if (breakdown)
             perServer[connection.server.name] = breakdown;
     }
-    const names = Object.keys(perServer);
-    if (names.length === 0)
+    const scoredServers = Object.keys(perServer);
+    const unscoredServers = report.connections
+        .map((c) => c.server.name)
+        .filter((name) => !perServer[name]);
+    if (scoredServers.length === 0)
         return undefined;
     const dimensions = Object.fromEntries(QUALITY_DIMENSIONS.map((dim) => [
         dim,
-        Math.round(names.reduce((sum, n) => sum + perServer[n].dimensions[dim], 0) / names.length),
+        Math.round(scoredServers.reduce((sum, n) => sum + perServer[n].dimensions[dim], 0) / scoredServers.length),
     ]));
-    const overall = Math.round(names.reduce((sum, n) => sum + perServer[n].overall, 0) / names.length);
-    const deductions = names
+    const overall = Math.round(scoredServers.reduce((sum, n) => sum + perServer[n].overall, 0) / scoredServers.length);
+    const deductions = scoredServers
         .flatMap((n) => perServer[n].deductions)
         .sort((a, b) => b.points - a.points);
-    return { overall, dimensions, deductions, perServer };
+    const coverage = computeQualityCoverage(executedChecks);
+    return {
+        overall,
+        dimensions,
+        deductions,
+        perServer,
+        coverage,
+        coveragePercent: coveragePercent(coverage),
+        scoredServers,
+        unscoredServers,
+        disclaimer: QUALITY_SCORE_DISCLAIMER,
+    };
+}
+/**
+ * Enforces `.mcp-medic-policy.json`'s `quality.minimumScore` as a CI gate.
+ * Pure function so it's directly unit-testable without needing a real CLI
+ * invocation or a restricted check set — see PHASE 12 ("policy/CI") in
+ * .agent-room/DECISIONS.md.
+ *
+ * Returns diagnostics to append to the report (0, 1, or 2):
+ *  - an 'error' if the score is below `minimumScore` (fails the run).
+ *  - a 'warning' if the score was computed from a partial assessment
+ *    (incomplete coverage, or a server that couldn't be scored) — a
+ *    minimumScore gate must never silently "pass" as if a full check had
+ *    run when it didn't.
+ */
+export function checkMinimumScorePolicy(quality, minimumScore, serverNamesLabel) {
+    const results = [];
+    if (quality.overall < minimumScore) {
+        results.push({
+            checkId: 'policy.minimum-quality-score',
+            severity: 'error',
+            message: `MCP quality score ${quality.overall} is below the policy minimum of ${minimumScore}.`,
+            serverName: serverNamesLabel,
+            category: 'configuration',
+        });
+    }
+    if (quality.coveragePercent < 100 || quality.unscoredServers.length > 0) {
+        const gaps = Object.entries(quality.coverage)
+            .filter(([, status]) => status !== 'covered')
+            .map(([dim, status]) => `${dim}: ${status}`)
+            .join(', ');
+        const parts = [
+            gaps ? `coverage is ${quality.coveragePercent}% (${gaps})` : undefined,
+            quality.unscoredServers.length > 0
+                ? `${quality.unscoredServers.length} server(s) could not be scored: ${quality.unscoredServers.join(', ')}`
+                : undefined,
+        ].filter(Boolean);
+        results.push({
+            checkId: 'policy.partial-coverage-with-minimum-score',
+            severity: 'warning',
+            message: `quality.minimumScore was evaluated against a partial assessment (${parts.join('; ')}) — it may not reflect a full check of this server.`,
+            serverName: serverNamesLabel,
+            category: 'configuration',
+        });
+    }
+    return results;
 }

package/dist/report.js

@@ -21,6 +21,24 @@ function formatScoreBlock(label, score) {
     }
     return lines;
 }
+/** Printed once per report (not per-server) — coverage and the disclaimer
+ * are report-level facts, and repeating them per server would overwhelm
+ * the output for a multi-server fleet. */
+function formatCoverageAndDisclaimer(quality) {
+    const lines = [];
+    const gaps = Object.entries(quality.coverage).filter(([, status]) => status !== 'covered');
+    const coverageLine = gaps.length === 0
+        ? `Coverage: ${quality.coveragePercent}% (all dimensions fully evaluated)`
+        : `Coverage: ${quality.coveragePercent}% (${gaps
+            .map(([dim, status]) => `${DIMENSION_LABELS[dim] ?? dim}: ${status}`)
+            .join(', ')})`;
+    lines.push(coverageLine);
+    if (quality.unscoredServers.length > 0) {
+        lines.push(`Note: ${quality.unscoredServers.length} server(s) could not be scored (connection failed): ${quality.unscoredServers.join(', ')}`);
+    }
+    lines.push(quality.disclaimer);
+    return lines;
+}
 /** Human-readable plain text report formatter. */
 export function formatReportHuman(report, options = {}) {
     const lines = [];
@@ -59,9 +77,13 @@ export function formatReportHuman(report, options = {}) {
             lines.push(...formatScoreBlock('QUALITY', perServerScore));
         }
     }
-    if (options.showScore && report.quality && Object.keys(report.quality.perServer).length > 1) {
+    if (options.showScore && report.quality) {
+        if (Object.keys(report.quality.perServer).length > 1) {
+            lines.push('');
+            lines.push(...formatScoreBlock('OVERALL QUALITY (all servers)', report.quality));
+        }
         lines.push('');
-        lines.push(...formatScoreBlock('OVERALL QUALITY (all servers)', report.quality));
+        lines.push(...formatCoverageAndDisclaimer(report.quality));
     }
     if (report.diagnostics.length > 0) {
         lines.push('');

package/dist/types.d.ts

@@ -149,9 +149,27 @@ export interface QualityScoreBreakdown {
     /** Only deductions with points > 0, most-costly first — every deduction here explains itself. */
     deductions: QualityDeduction[];
 }
+/** Whether the checks that feed a dimension actually ran this time.
+ * 'covered': every built-in check for this dimension ran.
+ * 'partial': at least one check contributing to this dimension ran, but not all of them.
+ * 'not-covered': no check contributing to this dimension ran — a 100 in
+ * that dimension means "nothing flagged it," not "nothing wrong exists." */
+export type CoverageStatus = 'covered' | 'partial' | 'not-covered';
+export type QualityCoverage = Record<QualityDimension, CoverageStatus>;
 export interface ReportQualityScore extends QualityScoreBreakdown {
     /** Per-connected-server breakdown; a server that never connected has no entry (nothing to score). */
     perServer: Record<string, QualityScoreBreakdown>;
+    /** Per-dimension coverage, derived from which checks actually ran (`RunOptions.checks`) —
+     * never assume a dimension was fully evaluated just because it scored 100. */
+    coverage: QualityCoverage;
+    /** 0-100: 'covered' dimensions count as 1, 'partial' as 0.5, 'not-covered' as 0, averaged across all 5. */
+    coveragePercent: number;
+    /** Names of servers actually included in this score (connected + scored). */
+    scoredServers: string[];
+    /** Names of servers that could NOT be scored (failed to connect) — never silently dropped from view. */
+    unscoredServers: string[];
+    /** A short, load-bearing reminder of what this number does and doesn't mean — see src/quality-score.ts. */
+    disclaimer: string;
 }
 export interface RunReport {
     configSource?: string;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mcp-medic",
-  "version": "1.2.3",
+  "version": "1.2.4",
   "description": "Diagnose broken MCP (Model Context Protocol) server configs before they break your agent silently.",
   "type": "module",
   "main": "dist/index.js",