@k08200/mcp-probe 1.0.0 → 1.4.0

package/README.md

@@ -5,12 +5,23 @@
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 [![Node.js](https://img.shields.io/node/v/@k08200/mcp-probe)](package.json)
 
-**Quality checker for MCP servers.** Validates protocol handshake, discovery, optional tool-call dry-runs, and response latency in one command.
+**CI readiness gate for MCP servers.** Validates protocol handshake, discovery, optional tool-call dry-runs, stderr noise, and response latency in one command.
 
-The `npm audit` for the [MCP](https://modelcontextprotocol.io) ecosystem — because [awesome-mcp-servers](https://github.com/punkpeye/awesome-mcp-servers) lists 200+ servers and there was no way to know if they actually worked.
+The `npm audit` for the [MCP](https://modelcontextprotocol.io) ecosystem — because an MCP server can start, pass `tools/list`, and still fail every real tool call when auth handoff, browser OAuth, or downstream permissions are broken.
+
+Read the v1 launch post: [mcp-probe v1.0.0: A CI readiness gate for MCP servers](https://dev.to/k08200/mcp-probe-v100-a-ci-readiness-gate-for-mcp-servers-4ch0)
 
 ## Quick Start for CI
 
+Scaffold the config, sidecar, and GitHub Actions workflow:
+
+```bash
+npx @k08200/mcp-probe@latest init \
+  --target @your-org/your-mcp-server \
+  --discover \
+  --github-actions
+```
+
 Add this workflow to any project that depends on MCP servers:
 
 ```yaml
@@ -42,6 +53,27 @@ For teams running several MCP servers, use a config file:
 npx @k08200/mcp-probe --config mcp-probe.config.json --github-summary
 ```
 
+For production CI, add sidecar inputs so dry-runs call real read-only paths instead of schema-minimum placeholders:
+
+```json
+{
+  "tools": {
+    "logs_query": {
+      "input": {
+        "query": "service:web status:error",
+        "timeframe": "1h"
+      },
+      "expect": {
+        "status": "pass",
+        "not_error_code": [401, 403],
+        "requiredFields": ["source", "freshness"],
+        "maxRows": 100
+      }
+    }
+  }
+}
+```
+
 ```bash
 npx @k08200/mcp-probe @modelcontextprotocol/server-memory
 ```
@@ -77,6 +109,8 @@ mcp-probe  @modelcontextprotocol/server-memory
 
 ## Install
 
+Requires Node.js 20.19 or newer.
+
 ```bash
 # No install needed
 npx @k08200/mcp-probe <target>
@@ -91,6 +125,25 @@ npm install -g @k08200/mcp-probe
 # Check an npm package
 mcp-probe @modelcontextprotocol/server-memory
 
+# Scaffold config + .mcp-probe.json + optional GitHub Actions workflow
+mcp-probe init --target @modelcontextprotocol/server-memory --github-actions
+
+# Discover tool names first and scaffold sidecar entries automatically
+mcp-probe init --target @modelcontextprotocol/server-memory --discover --github-actions
+
+# Scaffold a remote server config with auth from an env var
+mcp-probe init \
+  --target https://mcp.example.com/mcp \
+  --transport http \
+  --header-env MCP_TOKEN \
+  --github-actions
+
+# Choose custom scaffold paths
+mcp-probe init \
+  --target @your-org/your-mcp-server \
+  --config-file ci/mcp-probe.config.json \
+  --sidecar-file ci/mcp-tools.json
+
 # Check a server that requires arguments (e.g. directories to serve)
 mcp-probe @modelcontextprotocol/server-filesystem /tmp /Users/me/projects
 
@@ -143,8 +196,65 @@ mcp-probe @scope/server --tools-file .mcp-probe.json
 | **Prompts discovery** | Runs `prompts/list` when the server advertises prompts. |
 | **Tool call dry-run** | Optional `tools/call` checks via `--probe-tools` or `--tools-file`. |
 
+## Issue codes and remediation hints
+
+When a check warns or fails, mcp-probe attaches stable issue metadata:
+
+```json
+{
+  "name": "Tool call dry-run",
+  "status": "warn",
+  "message": "1 auth/permission errors (1 sidecar, 0 auto)",
+  "issue": {
+    "code": "TOOL_CALL_AUTH",
+    "hint": "At least one tool call hit auth or permission handling. This often means CI needs tokens or the server needs non-browser auth."
+  }
+}
+```
+
+These hints appear in terminal output, JSON output, GitHub Actions summaries, and workflow annotations so PR failures point at the likely fix instead of only showing raw MCP errors.
+
+Common issue codes:
+
+| Code | Meaning |
+|------|---------|
+| `TARGET_NOT_FOUND` | The npm package, local file, or executable could not be started. |
+| `HANDSHAKE_TIMEOUT` | The server did not complete MCP `initialize` before the timeout. |
+| `HANDSHAKE_AUTH` | Initialization failed with an auth-like error. |
+| `NO_TOOLS` | The server responded but did not expose tools. |
+| `TOOL_SCHEMA_INVALID` | A discovered tool has an invalid schema. |
+| `TOOL_CALL_AUTH` | A real tool call reached auth or permission handling. |
+| `CONTRACT_ASSERTION_FAILED` | A tool call completed but failed one or more sidecar assertions. |
+| `AUTO_DRY_RUN_INPUT` | Auto-generated schema-minimum input failed; add sidecar inputs. |
+| `TOOL_CALL_FAILED` | A sidecar tool call returned a non-auth error. |
+
 ## Batch CI gate
 
+If you are starting from scratch, generate the files:
+
+```bash
+mcp-probe init --target @your-org/your-mcp-server --discover --github-actions
+```
+
+This creates:
+
+| File | Purpose |
+|------|---------|
+| `mcp-probe.config.json` | Batch config with one server and `probeTools: true`. |
+| `.mcp-probe.json` | Sidecar template for real tool-call sample inputs. |
+| `.github/workflows/mcp-probe.yml` | GitHub Actions readiness gate. |
+
+Existing files are skipped unless you pass `--force`.
+
+Generated config and sidecar files include JSON Schema references:
+
+| Schema | File |
+|--------|------|
+| [`mcp-probe.config.schema.json`](schemas/mcp-probe.config.schema.json) | `mcp-probe.config.json` |
+| [`mcp-probe.sidecar.schema.json`](schemas/mcp-probe.sidecar.schema.json) | `.mcp-probe.json` |
+
+When `--discover` is enabled, mcp-probe connects to the target server, runs discovery, and pre-populates `.mcp-probe.json` with the discovered tool names and schema-minimum sample inputs. Review those values before using them as a production CI gate.
+
 Use `--config` when a project depends on several MCP servers and you want one CI command to validate all of them:
 
 ```json
@@ -260,6 +370,55 @@ mcp-probe @your-org/datadog-mcp --tools-file ./ci/mcp-tools.json
 
 Sidecar inputs are used first; generated minimal inputs are fallback only. Auth and permission failures such as 401/403 are surfaced as warnings so CI can distinguish "OAuth handoff needed" from transport or runtime failure.
 
+## Tool call contract assertions
+
+For production MCP servers, especially database-backed servers, a successful `tools/call` is still not enough. Agents depend on a contract: read-only roles, scoped data, stable error codes, safe limits, and no leaked internals.
+
+Add assertions to `.mcp-probe.json` to validate that contract:
+
+```json
+{
+  "tools": {
+    "execute_sql": {
+      "input": {
+        "project_id": "YOUR_PROJECT_ID",
+        "query": "select 1 as health_check"
+      },
+      "expect": {
+        "status": "pass",
+        "requiredFields": ["rowCount", "limit", "source", "freshness"],
+        "maxRows": 100
+      }
+    },
+    "execute_sql_write_denied": {
+      "input": {
+        "project_id": "YOUR_PROJECT_ID",
+        "query": "delete from users where id = 1"
+      },
+      "expect": {
+        "status": "fail",
+        "errorCode": "WRITE_NOT_ALLOWED",
+        "notContains": ["DATABASE_URL", "password", "stack"]
+      }
+    }
+  }
+}
+```
+
+Supported assertions:
+
+| Assertion | Purpose |
+|-----------|---------|
+| `status` | Expected call status: `pass`, `fail`, or `warn`. Use `fail` for denied-write probes. |
+| `requiredFields` | Field names that must appear anywhere in the tool result payload. |
+| `maxRows` | Maximum allowed row count, using `rowCount`, `rowsReturned`, or common row arrays. |
+| `errorCode` | Stable error code expected in an error response. |
+| `contains` | Text snippets that must appear in the result or error payload. |
+| `notContains` | Text snippets that must not appear; useful for stack traces, secrets, and raw internals. |
+| `not_error_code` | HTTP/status codes that should be warnings instead of failures, usually auth handoff codes. |
+
+If an assertion fails, mcp-probe returns `CONTRACT_ASSERTION_FAILED` and includes per-assertion details in JSON and GitHub Actions summaries.
+
 ## Status badges
 
 Use `--badge-file` to write a [shields.io endpoint](https://shields.io/badges/endpoint-badge) JSON file:
@@ -377,25 +536,43 @@ Tool names vary by MCP server implementation. Run your server once with `--outpu
 ## JSON output
 
 ```bash
-mcp-probe @modelcontextprotocol/server-memory --probe-tools --output json
+mcp-probe @your-org/datadog-mcp --tools-file .mcp-probe.json --output json
 ```
 
 ```json
 {
-  "target": "@modelcontextprotocol/server-memory",
+  "target": "@your-org/datadog-mcp",
   "timestamp": "2026-05-17T12:00:00.000Z",
-  "overallStatus": "pass",
+  "overallStatus": "warn",
   "checks": [
-    { "name": "Target resolution", "status": "pass", "message": "npx --yes @modelcontextprotocol/server-memory" },
-    { "name": "MCP protocol handshake", "status": "pass", "message": "memory-server v0.6.3", "latencyMs": 1392 },
-    { "name": "Tools discovery", "status": "pass", "message": "Found 9 tools", "latencyMs": 33 },
+    { "name": "Target resolution", "status": "pass", "message": "npx --yes @your-org/datadog-mcp" },
+    { "name": "MCP protocol handshake", "status": "pass", "message": "datadog-mcp v1.0.0", "latencyMs": 1392 },
+    { "name": "Tools discovery", "status": "pass", "message": "Found 12 tools", "latencyMs": 33 },
     { "name": "Tool schema validation", "status": "pass", "message": "All tool schemas are valid" },
-    { "name": "Tool call dry-run", "status": "pass", "message": "9 passed (2 sidecar, 7 auto)" }
+    {
+      "name": "Tool call dry-run",
+      "status": "warn",
+      "message": "1 auth/permission errors (1 sidecar, 0 auto)",
+      "issue": {
+        "code": "TOOL_CALL_AUTH",
+        "hint": "At least one tool call hit auth or permission handling. This often means CI needs tokens or the server needs non-browser auth."
+      }
+    }
   ],
-  "serverInfo": { "name": "memory-server", "version": "0.6.3", "capabilities": ["tools"] },
-  "tools": [{ "name": "create_entities", "description": "Create multiple new entities in the knowledge graph" }],
+  "serverInfo": { "name": "datadog-mcp", "version": "1.0.0", "capabilities": ["tools"] },
+  "tools": [{ "name": "logs_query", "description": "Query Datadog logs" }],
   "toolCallResults": [
-    { "tool": "read_graph", "status": "pass", "latencyMs": 41, "source": "auto" }
+    {
+      "tool": "logs_query",
+      "status": "warn",
+      "latencyMs": 41,
+      "source": "sidecar",
+      "error": "401 Unauthorized",
+      "issue": {
+        "code": "TOOL_CALL_AUTH",
+        "hint": "The server registered this tool, but the call path hit auth or permission handling. Check OAuth/browser handoff, service tokens, and CI secrets."
+      }
+    }
   ],
   "totalLatencyMs": 1455
 }
@@ -423,6 +600,10 @@ mcp-probe @modelcontextprotocol/server-memory --probe-tools --output json
 
 Issues and PRs are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).
 
+## Changelog
+
+See [CHANGELOG.md](CHANGELOG.md).
+
 ## License
 
 [MIT](LICENSE)

package/dist/assertions.d.ts

@@ -0,0 +1,11 @@
+import type { AssertionResult, CheckStatus, ToolExpectations } from './types.js';
+type EvaluationInput = {
+    result?: unknown;
+    error?: string;
+    actualStatus: CheckStatus;
+    expect?: ToolExpectations;
+};
+export declare function evaluateToolAssertions(input: EvaluationInput): AssertionResult[];
+export declare function assertionFailureMessage(assertions: AssertionResult[]): string | undefined;
+export {};
+//# sourceMappingURL=assertions.d.ts.map

package/dist/assertions.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"assertions.d.ts","sourceRoot":"","sources":["../src/assertions.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,eAAe,EAAE,WAAW,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAEjF,KAAK,eAAe,GAAG;IACrB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,YAAY,EAAE,WAAW,CAAC;IAC1B,MAAM,CAAC,EAAE,gBAAgB,CAAC;CAC3B,CAAC;AA6GF,wBAAgB,sBAAsB,CAAC,KAAK,EAAE,eAAe,GAAG,eAAe,EAAE,CAsDhF;AAED,wBAAgB,uBAAuB,CAAC,UAAU,EAAE,eAAe,EAAE,GAAG,MAAM,GAAG,SAAS,CAIzF"}

package/dist/assertions.js

@@ -0,0 +1,156 @@
+function stableStringify(value) {
+    if (typeof value === 'string')
+        return value;
+    try {
+        return JSON.stringify(value);
+    }
+    catch {
+        return String(value);
+    }
+}
+function parseMaybeJson(text) {
+    try {
+        return JSON.parse(text);
+    }
+    catch {
+        return text;
+    }
+}
+function extractPayload(result, error) {
+    const values = [];
+    const textParts = [];
+    if (error) {
+        values.push(error);
+        textParts.push(error);
+    }
+    if (result !== undefined) {
+        values.push(result);
+        textParts.push(stableStringify(result));
+    }
+    if (result && typeof result === 'object') {
+        const content = result.content;
+        if (Array.isArray(content)) {
+            for (const part of content) {
+                if (!part || typeof part !== 'object')
+                    continue;
+                const record = part;
+                const text = typeof record.text === 'string' ? record.text : undefined;
+                if (text !== undefined) {
+                    values.push(parseMaybeJson(text));
+                    textParts.push(text);
+                }
+                for (const key of ['json', 'data', 'resource']) {
+                    if (record[key] !== undefined) {
+                        values.push(record[key]);
+                        textParts.push(stableStringify(record[key]));
+                    }
+                }
+            }
+        }
+    }
+    return { values, text: textParts.join('\n') };
+}
+function objectValues(value) {
+    if (Array.isArray(value))
+        return value.flatMap(objectValues);
+    if (!value || typeof value !== 'object')
+        return [];
+    const record = value;
+    return [record, ...Object.values(record).flatMap(objectValues)];
+}
+function hasField(value, field) {
+    if (!value || typeof value !== 'object')
+        return false;
+    if (Array.isArray(value))
+        return value.some((entry) => hasField(entry, field));
+    const record = value;
+    if (Object.prototype.hasOwnProperty.call(record, field))
+        return true;
+    return Object.values(record).some((entry) => hasField(entry, field));
+}
+function findNumberField(value, field) {
+    for (const candidate of objectValues(value)) {
+        const record = candidate;
+        const found = record[field];
+        if (typeof found === 'number')
+            return found;
+        if (typeof found === 'string' && found.trim() !== '' && Number.isFinite(Number(found))) {
+            return Number(found);
+        }
+    }
+    return undefined;
+}
+function findRowsLength(value) {
+    for (const candidate of objectValues(value)) {
+        const record = candidate;
+        const rows = record.rows ?? record.data ?? record.items ?? record.records;
+        if (Array.isArray(rows))
+            return rows.length;
+    }
+    return undefined;
+}
+function includesText(haystack, needle) {
+    return haystack.toLowerCase().includes(needle.toLowerCase());
+}
+function pass(name, message) {
+    return { name, status: 'pass', message };
+}
+function fail(name, message) {
+    return { name, status: 'fail', message };
+}
+export function evaluateToolAssertions(input) {
+    const { expect } = input;
+    if (!expect)
+        return [];
+    const payload = extractPayload(input.result, input.error);
+    const assertions = [];
+    if (expect.status) {
+        assertions.push(input.actualStatus === expect.status
+            ? pass('status', `Tool status matched expected ${expect.status}`)
+            : fail('status', `Expected tool status ${expect.status}, got ${input.actualStatus}`));
+    }
+    for (const field of expect.requiredFields ?? []) {
+        const found = payload.values.some((value) => hasField(value, field));
+        assertions.push(found
+            ? pass(`requiredFields.${field}`, `Found required field "${field}"`)
+            : fail(`requiredFields.${field}`, `Missing required field "${field}"`));
+    }
+    if (expect.maxRows !== undefined) {
+        const rowCount = payload.values
+            .map((value) => findNumberField(value, 'rowCount') ?? findNumberField(value, 'rowsReturned') ?? findRowsLength(value))
+            .find((value) => value !== undefined);
+        if (rowCount === undefined) {
+            assertions.push(fail('maxRows', 'Could not determine row count from result metadata'));
+        }
+        else {
+            assertions.push(rowCount <= expect.maxRows
+                ? pass('maxRows', `Row count ${rowCount} is within maxRows ${expect.maxRows}`)
+                : fail('maxRows', `Row count ${rowCount} exceeds maxRows ${expect.maxRows}`));
+        }
+    }
+    if (expect.errorCode) {
+        const found = payload.values.some((value) => hasField(value, 'code') && includesText(stableStringify(value), expect.errorCode))
+            || includesText(payload.text, expect.errorCode);
+        assertions.push(found
+            ? pass('errorCode', `Found expected error code ${expect.errorCode}`)
+            : fail('errorCode', `Missing expected error code ${expect.errorCode}`));
+    }
+    for (const expectedText of expect.contains ?? []) {
+        assertions.push(includesText(payload.text, expectedText)
+            ? pass(`contains.${expectedText}`, `Output contains "${expectedText}"`)
+            : fail(`contains.${expectedText}`, `Output does not contain "${expectedText}"`));
+    }
+    for (const forbiddenText of expect.notContains ?? []) {
+        assertions.push(!includesText(payload.text, forbiddenText)
+            ? pass(`notContains.${forbiddenText}`, `Output does not contain "${forbiddenText}"`)
+            : fail(`notContains.${forbiddenText}`, `Output leaked forbidden text "${forbiddenText}"`));
+    }
+    return assertions;
+}
+export function assertionFailureMessage(assertions) {
+    const failures = assertions.filter((assertion) => assertion.status === 'fail');
+    if (failures.length === 0)
+        return undefined;
+    return failures.map((assertion) => assertion.message).join('; ');
+}
+//# sourceMappingURL=assertions.js.map

package/dist/assertions.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"assertions.js","sourceRoot":"","sources":["../src/assertions.ts"],"names":[],"mappings":"AAcA,SAAS,eAAe,CAAC,KAAc;IACrC,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,KAAK,CAAC;IAC5C,IAAI,CAAC;QACH,OAAO,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC;IAC/B,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,MAAM,CAAC,KAAK,CAAC,CAAC;IACvB,CAAC;AACH,CAAC;AAED,SAAS,cAAc,CAAC,IAAY;IAClC,IAAI,CAAC;QACH,OAAO,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC1B,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,IAAI,CAAC;IACd,CAAC;AACH,CAAC;AAED,SAAS,cAAc,CAAC,MAAe,EAAE,KAAc;IACrD,MAAM,MAAM,GAAc,EAAE,CAAC;IAC7B,MAAM,SAAS,GAAa,EAAE,CAAC;IAC/B,IAAI,KAAK,EAAE,CAAC;QACV,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QACnB,SAAS,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACxB,CAAC;IAED,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;QACzB,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACpB,SAAS,CAAC,IAAI,CAAC,eAAe,CAAC,MAAM,CAAC,CAAC,CAAC;IAC1C,CAAC;IAED,IAAI,MAAM,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;QACzC,MAAM,OAAO,GAAI,MAAgC,CAAC,OAAO,CAAC;QAC1D,IAAI,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;YAC3B,KAAK,MAAM,IAAI,IAAI,OAAO,EAAE,CAAC;gBAC3B,IAAI,CAAC,IAAI,IAAI,OAAO,IAAI,KAAK,QAAQ;oBAAE,SAAS;gBAChD,MAAM,MAAM,GAAG,IAA+B,CAAC;gBAC/C,MAAM,IAAI,GAAG,OAAO,MAAM,CAAC,IAAI,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAC;gBACvE,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;oBACvB,MAAM,CAAC,IAAI,CAAC,cAAc,CAAC,IAAI,CAAC,CAAC,CAAC;oBAClC,SAAS,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBACvB,CAAC;gBACD,KAAK,MAAM,GAAG,IAAI,CAAC,MAAM,EAAE,MAAM,EAAE,UAAU,CAAC,EAAE,CAAC;oBAC/C,IAAI,MAAM,CAAC,GAAG,CAAC,KAAK,SAAS,EAAE,CAAC;wBAC9B,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC;wBACzB,SAAS,CAAC,IAAI,CAAC,eAAe,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;oBAC/C,CAAC;gBACH,CAAC;YACH,CAAC;QACH,CAAC;IACH,CAAC;IAED,OAAO,EAAE,MAAM,EAAE,IAAI,EAAE,SAAS,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;AAChD,CAAC;AAED,SAAS,YAAY,CAAC,KAAc;IAClC,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC;QAAE,OAAO,KAAK,CAAC,OAAO,CAAC,YAAY,CAAC,CAAC;IAC7D,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,EAAE,CAAC;IACnD,MAAM,MAAM,GAAG,KAAgC,CAAC;IAChD,OAAO,CAAC,MAAM,EAAE,GAAG,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,OAAO,CAAC,YAAY,CAAC,CAAC,CAAC;AAClE,CAAC;AAED,SAAS,QAAQ,CAAC,KAAc,EAAE,KAAa;IAC7C,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,KAAK,CAAC;IACtD,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC;QAAE,OAAO,KAAK,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,QAAQ,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC,CAAC;IAC/E,MAAM,MAAM,GAAG,KAAgC,CAAC;IAChD,IAAI,MAAM,CAAC,SAAS,CAAC,cAAc,CAAC,IAAI,CAAC,MAAM,EAAE,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IACrE,OAAO,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,QAAQ,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC,CAAC;AACvE,CAAC;AAED,SAAS,eAAe,CAAC,KAAc,EAAE,KAAa;IACpD,KAAK,MAAM,SAAS,IAAI,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC;QAC5C,MAAM,MAAM,GAAG,SAAoC,CAAC;QACpD,MAAM,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;QAC5B,IAAI,OAAO,KAAK,KAAK,QAAQ;YAAE,OAAO,KAAK,CAAC;QAC5C,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,IAAI,EAAE,KAAK,EAAE,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC;YACvF,OAAO,MAAM,CAAC,KAAK,CAAC,CAAC;QACvB,CAAC;IACH,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,cAAc,CAAC,KAAc;IACpC,KAAK,MAAM,SAAS,IAAI,YAAY,CAAC,KAAK,CAAC,EAAE,CAAC;QAC5C,MAAM,MAAM,GAAG,SAAoC,CAAC;QACpD,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,IAAI,MAAM,CAAC,IAAI,IAAI,MAAM,CAAC,KAAK,IAAI,MAAM,CAAC,OAAO,CAAC;QAC1E,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC;YAAE,OAAO,IAAI,CAAC,MAAM,CAAC;IAC9C,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,SAAS,YAAY,CAAC,QAAgB,EAAE,MAAc;IACpD,OAAO,QAAQ,CAAC,WAAW,EAAE,CAAC,QAAQ,CAAC,MAAM,CAAC,WAAW,EAAE,CAAC,CAAC;AAC/D,CAAC;AAED,SAAS,IAAI,CAAC,IAAY,EAAE,OAAe;IACzC,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,OAAO,EAAE,CAAC;AAC3C,CAAC;AAED,SAAS,IAAI,CAAC,IAAY,EAAE,OAAe;IACzC,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,OAAO,EAAE,CAAC;AAC3C,CAAC;AAED,MAAM,UAAU,sBAAsB,CAAC,KAAsB;IAC3D,MAAM,EAAE,MAAM,EAAE,GAAG,KAAK,CAAC;IACzB,IAAI,CAAC,MAAM;QAAE,OAAO,EAAE,CAAC;IAEvB,MAAM,OAAO,GAAG,cAAc,CAAC,KAAK,CAAC,MAAM,EAAE,KAAK,CAAC,KAAK,CAAC,CAAC;IAC1D,MAAM,UAAU,GAAsB,EAAE,CAAC;IAEzC,IAAI,MAAM,CAAC,MAAM,EAAE,CAAC;QAClB,UAAU,CAAC,IAAI,CAAC,KAAK,CAAC,YAAY,KAAK,MAAM,CAAC,MAAM;YAClD,CAAC,CAAC,IAAI,CAAC,QAAQ,EAAE,gCAAgC,MAAM,CAAC,MAAM,EAAE,CAAC;YACjE,CAAC,CAAC,IAAI,CAAC,QAAQ,EAAE,wBAAwB,MAAM,CAAC,MAAM,SAAS,KAAK,CAAC,YAAY,EAAE,CAAC,CAAC,CAAC;IAC1F,CAAC;IAED,KAAK,MAAM,KAAK,IAAI,MAAM,CAAC,cAAc,IAAI,EAAE,EAAE,CAAC;QAChD,MAAM,KAAK,GAAG,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,QAAQ,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC,CAAC;QACrE,UAAU,CAAC,IAAI,CAAC,KAAK;YACnB,CAAC,CAAC,IAAI,CAAC,kBAAkB,KAAK,EAAE,EAAE,yBAAyB,KAAK,GAAG,CAAC;YACpE,CAAC,CAAC,IAAI,CAAC,kBAAkB,KAAK,EAAE,EAAE,2BAA2B,KAAK,GAAG,CAAC,CAAC,CAAC;IAC5E,CAAC;IAED,IAAI,MAAM,CAAC,OAAO,KAAK,SAAS,EAAE,CAAC;QACjC,MAAM,QAAQ,GAAG,OAAO,CAAC,MAAM;aAC5B,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,eAAe,CAAC,KAAK,EAAE,UAAU,CAAC,IAAI,eAAe,CAAC,KAAK,EAAE,cAAc,CAAC,IAAI,cAAc,CAAC,KAAK,CAAC,CAAC;aACrH,IAAI,CAAC,CAAC,KAAK,EAAmB,EAAE,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC;QACzD,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;YAC3B,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,oDAAoD,CAAC,CAAC,CAAC;QACzF,CAAC;aAAM,CAAC;YACN,UAAU,CAAC,IAAI,CAAC,QAAQ,IAAI,MAAM,CAAC,OAAO;gBACxC,CAAC,CAAC,IAAI,CAAC,SAAS,EAAE,aAAa,QAAQ,sBAAsB,MAAM,CAAC,OAAO,EAAE,CAAC;gBAC9E,CAAC,CAAC,IAAI,CAAC,SAAS,EAAE,aAAa,QAAQ,oBAAoB,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC;QAClF,CAAC;IACH,CAAC;IAED,IAAI,MAAM,CAAC,SAAS,EAAE,CAAC;QACrB,MAAM,KAAK,GAAG,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC,IAAI,YAAY,CAAC,eAAe,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC,SAAU,CAAC,CAAC;eAC3H,YAAY,CAAC,OAAO,CAAC,IAAI,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;QAClD,UAAU,CAAC,IAAI,CAAC,KAAK;YACnB,CAAC,CAAC,IAAI,CAAC,WAAW,EAAE,6BAA6B,MAAM,CAAC,SAAS,EAAE,CAAC;YACpE,CAAC,CAAC,IAAI,CAAC,WAAW,EAAE,+BAA+B,MAAM,CAAC,SAAS,EAAE,CAAC,CAAC,CAAC;IAC5E,CAAC;IAED,KAAK,MAAM,YAAY,IAAI,MAAM,CAAC,QAAQ,IAAI,EAAE,EAAE,CAAC;QACjD,UAAU,CAAC,IAAI,CAAC,YAAY,CAAC,OAAO,CAAC,IAAI,EAAE,YAAY,CAAC;YACtD,CAAC,CAAC,IAAI,CAAC,YAAY,YAAY,EAAE,EAAE,oBAAoB,YAAY,GAAG,CAAC;YACvE,CAAC,CAAC,IAAI,CAAC,YAAY,YAAY,EAAE,EAAE,4BAA4B,YAAY,GAAG,CAAC,CAAC,CAAC;IACrF,CAAC;IAED,KAAK,MAAM,aAAa,IAAI,MAAM,CAAC,WAAW,IAAI,EAAE,EAAE,CAAC;QACrD,UAAU,CAAC,IAAI,CAAC,CAAC,YAAY,CAAC,OAAO,CAAC,IAAI,EAAE,aAAa,CAAC;YACxD,CAAC,CAAC,IAAI,CAAC,eAAe,aAAa,EAAE,EAAE,4BAA4B,aAAa,GAAG,CAAC;YACpF,CAAC,CAAC,IAAI,CAAC,eAAe,aAAa,EAAE,EAAE,iCAAiC,aAAa,GAAG,CAAC,CAAC,CAAC;IAC/F,CAAC;IAED,OAAO,UAAU,CAAC;AACpB,CAAC;AAED,MAAM,UAAU,uBAAuB,CAAC,UAA6B;IACnE,MAAM,QAAQ,GAAG,UAAU,CAAC,MAAM,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,SAAS,CAAC,MAAM,KAAK,MAAM,CAAC,CAAC;IAC/E,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,SAAS,CAAC;IAC5C,OAAO,QAAQ,CAAC,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,SAAS,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACnE,CAAC"}

package/dist/checker.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"checker.d.ts","sourceRoot":"","sources":["../src/checker.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAa,YAAY,EAAE,WAAW,EAAe,cAAc,EAAe,aAAa,EAAE,MAAM,YAAY,CAAC;AAQhI,wBAAgB,aAAa,CAAC,MAAM,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,aAAa,GAAG,cAAc,CAavF;AAiDD,wBAAsB,cAAc,CAAC,OAAO,EAAE,YAAY,GAAG,OAAO,CAAC,WAAW,CAAC,CA8HhF"}
+{"version":3,"file":"checker.d.ts","sourceRoot":"","sources":["../src/checker.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EAAa,YAAY,EAAE,WAAW,EAAe,cAAc,EAAe,aAAa,EAAE,MAAM,YAAY,CAAC;AAQhI,wBAAgB,aAAa,CAAC,MAAM,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,aAAa,GAAG,cAAc,CAavF;AA4ED,wBAAsB,cAAc,CAAC,OAAO,EAAE,YAAY,GAAG,OAAO,CAAC,WAAW,CAAC,CAiIhF"}

package/dist/checker.js

@@ -1,5 +1,7 @@
 import { existsSync, readFileSync } from 'fs';
+import { withIssue } from './issues.js';
 import { probeMcpServer } from './protocols/mcp-client.js';
+import { redactText, redactUnknown } from './redact.js';
 const SIDECAR_FILENAME = '.mcp-probe.json';
 function isUrlTarget(target) {
     return /^https?:\/\//i.test(target);
@@ -50,6 +52,22 @@ function loadSidecar(toolsFile) {
         if (codes !== undefined && (!Array.isArray(codes) || !codes.every((c) => typeof c === 'number'))) {
             throw new Error(`Invalid tools file: ${toolName}.expect.not_error_code must be a number array`);
         }
+        const status = toolEntry.expect?.status;
+        if (status !== undefined && status !== 'pass' && status !== 'fail' && status !== 'warn') {
+            throw new Error(`Invalid tools file: ${toolName}.expect.status must be pass, fail, or warn`);
+        }
+        for (const key of ['requiredFields', 'contains', 'notContains']) {
+            const value = toolEntry.expect?.[key];
+            if (value !== undefined && (!Array.isArray(value) || !value.every((item) => typeof item === 'string'))) {
+                throw new Error(`Invalid tools file: ${toolName}.expect.${key} must be a string array`);
+            }
+        }
+        if (toolEntry.expect?.maxRows !== undefined && (typeof toolEntry.expect.maxRows !== 'number' || toolEntry.expect.maxRows < 0)) {
+            throw new Error(`Invalid tools file: ${toolName}.expect.maxRows must be a non-negative number`);
+        }
+        if (toolEntry.expect?.errorCode !== undefined && typeof toolEntry.expect.errorCode !== 'string') {
+            throw new Error(`Invalid tools file: ${toolName}.expect.errorCode must be a string`);
+        }
     }
     return sidecar;
 }
@@ -63,17 +81,18 @@ function deriveOverallStatus(checks) {
 export async function checkMcpServer(options) {
     const startTime = Date.now();
     const checks = [];
+    const secretValues = Object.values(options.headers ?? {});
     const resolved = resolveTarget(options.target, options.transport);
     const args = [...(resolved.args ?? []), ...(options.serverArgs ?? [])];
     const probeTools = options.probeTools || Boolean(options.toolsFile);
     const resolutionMessage = resolved.transport === 'stdio'
         ? `${resolved.command} ${args.join(' ')}`
         : `${resolved.transport} ${resolved.url}`;
-    checks.push({
+    checks.push(withIssue({
         name: 'Target resolution',
         status: 'pass',
-        message: resolutionMessage,
-    });
+        message: redactText(resolutionMessage, secretValues),
+    }));
     try {
         const sidecar = probeTools ? loadSidecar(options.toolsFile) : undefined;
         const probe = await probeMcpServer({
@@ -87,68 +106,71 @@ export async function checkMcpServer(options) {
             probeTools,
             sidecar,
         });
-        checks.push({
+        checks.push(withIssue({
             name: 'MCP protocol handshake',
             status: 'pass',
             message: `${probe.serverInfo.name} v${probe.serverInfo.version}`,
             latencyMs: probe.connectLatencyMs,
-        });
+        }));
         const toolsStatus = probe.tools.length > 0 ? 'pass' : 'warn';
-        checks.push({
+        checks.push(withIssue({
             name: 'Tools discovery',
             status: toolsStatus,
             message: probe.tools.length > 0
                 ? `Found ${probe.tools.length} tool${probe.tools.length !== 1 ? 's' : ''}`
                 : 'No tools registered — server may be resources/prompts-only',
             latencyMs: probe.toolsLatencyMs,
-        });
+        }));
         const toolsMissingName = probe.tools.filter((t) => !t.name);
         if (probe.tools.length > 0) {
-            checks.push({
+            checks.push(withIssue({
                 name: 'Tool schema validation',
                 status: toolsMissingName.length > 0 ? 'warn' : 'pass',
                 message: toolsMissingName.length > 0
                     ? `${toolsMissingName.length} tool(s) missing required name field`
                     : 'All tool schemas are valid',
-            });
+            }));
         }
         if (probe.resources.length > 0 || probe.resourcesLatencyMs !== undefined) {
-            checks.push({
+            checks.push(withIssue({
                 name: 'Resources discovery',
                 status: 'pass',
                 message: `Found ${probe.resources.length} resource${probe.resources.length !== 1 ? 's' : ''}`,
                 latencyMs: probe.resourcesLatencyMs,
-            });
+            }));
         }
         if (probe.prompts.length > 0 || probe.promptsLatencyMs !== undefined) {
-            checks.push({
+            checks.push(withIssue({
                 name: 'Prompts discovery',
                 status: 'pass',
                 message: `Found ${probe.prompts.length} prompt${probe.prompts.length !== 1 ? 's' : ''}`,
                 latencyMs: probe.promptsLatencyMs,
-            });
+            }));
         }
         if (probe.toolCallResults && probe.toolCallResults.length > 0) {
             const failed = probe.toolCallResults.filter((r) => r.status === 'fail');
             const warned = probe.toolCallResults.filter((r) => r.status === 'warn');
             const passed = probe.toolCallResults.filter((r) => r.status === 'pass');
             const sidecarCount = probe.toolCallResults.filter((r) => r.source === 'sidecar').length;
+            const contractFailed = failed.filter((r) => r.issue?.code === 'CONTRACT_ASSERTION_FAILED').length;
             const status = failed.length > 0 ? 'fail' : warned.length > 0 ? 'warn' : 'pass';
             const parts = [];
             if (passed.length > 0)
                 parts.push(`${passed.length} passed`);
             if (warned.length > 0)
                 parts.push(`${warned.length} auth/permission errors`);
-            if (failed.length > 0)
-                parts.push(`${failed.length} failed`);
+            if (contractFailed > 0)
+                parts.push(`${contractFailed} contract assertion failed`);
+            if (failed.length - contractFailed > 0)
+                parts.push(`${failed.length - contractFailed} failed`);
             const sourceNote = sidecarCount > 0 ? ` (${sidecarCount} sidecar, ${probe.toolCallResults.length - sidecarCount} auto)` : '';
-            checks.push({
+            checks.push(withIssue({
                 name: 'Tool call dry-run',
                 status,
                 message: parts.join(', ') + sourceNote,
-            });
+            }));
         }
-        return {
+        return redactUnknown({
             target: options.target,
             timestamp: new Date().toISOString(),
             overallStatus: deriveOverallStatus(checks),
@@ -159,17 +181,17 @@ export async function checkMcpServer(options) {
             prompts: probe.prompts,
             toolCallResults: probe.toolCallResults,
             totalLatencyMs: Date.now() - startTime,
-        };
+        }, secretValues);
     }
     catch (error) {
-        const message = error instanceof Error ? error.message : String(error);
+        const message = redactText(error instanceof Error ? error.message : String(error), secretValues);
         const isSidecarError = message.includes('tools file');
-        checks.push({
+        checks.push(withIssue({
             name: isSidecarError ? 'Tool sidecar' : 'MCP protocol handshake',
             status: 'fail',
             message,
-        });
-        return {
+        }));
+        return redactUnknown({
             target: options.target,
             timestamp: new Date().toISOString(),
             overallStatus: 'fail',
@@ -178,7 +200,7 @@ export async function checkMcpServer(options) {
             resources: [],
             prompts: [],
             totalLatencyMs: Date.now() - startTime,
-        };
+        }, secretValues);
     }
 }
 //# sourceMappingURL=checker.js.map