@yawlabs/mcp-compliance 0.5.0 → 0.7.0

package/README.md

@@ -5,7 +5,7 @@
 [![GitHub stars](https://img.shields.io/github/stars/YawLabs/mcp-compliance)](https://github.com/YawLabs/mcp-compliance/stargazers)
 [![CI](https://github.com/YawLabs/mcp-compliance/actions/workflows/ci.yml/badge.svg)](https://github.com/YawLabs/mcp-compliance/actions/workflows/ci.yml)
 
-**Test any MCP server for spec compliance.** 43-test suite covering transport, lifecycle, tools, resources, prompts, error handling, and schema validation against the [MCP specification](https://modelcontextprotocol.io/specification/2025-11-25). CLI, MCP server, and programmatic API.
+**Test any MCP server for spec compliance.** 78-test suite covering transport, lifecycle, tools, resources, prompts, error handling, schema validation, and security against the [MCP specification](https://modelcontextprotocol.io/specification/2025-11-25). CLI, MCP server, and programmatic API.
 
 Built and maintained by [Yaw Labs](https://yaw.sh).
 
@@ -15,7 +15,7 @@ MCP servers are multiplying fast — but most ship without compliance testing. B
 
 This tool solves that:
 
-- **48 tests across 7 categories** — transport, lifecycle, tools, resources, prompts, error handling, and schema validation. No gaps.
+- **78 tests across 8 categories** — transport, lifecycle, tools, resources, prompts, error handling, schema validation, and security. No gaps.
 - **Capability-driven** — tests adapt to what the server declares. If it says it supports tools, tool tests become required. No false failures for features the server doesn't claim.
 - **Graded scoring** — A-F letter grade with a weighted score (required tests 70%, optional 30%). One number to communicate compliance.
 - **CI-ready** — `--strict` mode exits with code 1 on required test failures. Drop it into any pipeline.
@@ -100,23 +100,29 @@ mcp-compliance badge https://my-server.com/mcp
 
 Outputs the markdown embed for a compliance badge hosted at [mcp.hosting](https://mcp.hosting).
 
-## What the 48 tests check
+## What the 78 tests check
 
 <details>
-<summary><strong>Transport (7 tests)</strong></summary>
+<summary><strong>Transport (13 tests)</strong></summary>
 
 - **transport-post** — Server accepts HTTP POST requests (required)
 - **transport-content-type** — Responds with application/json or text/event-stream (required)
-- **transport-notification-202** — Notifications return 202 Accepted
+- **transport-notification-202** — Notifications return exactly 202 Accepted
+- **transport-content-type-reject** — Rejects non-JSON request Content-Type
 - **transport-session-id** — Enforces MCP-Session-Id after initialization
+- **transport-session-invalid** — Returns 404 for unknown session ID
 - **transport-get** — GET returns SSE stream or 405
 - **transport-delete** — DELETE accepted or returns 405
 - **transport-batch-reject** — Rejects JSON-RPC batch requests (required)
+- **transport-content-type-init** — Initialize response has valid content type
+- **transport-get-stream** — GET with session returns SSE or 405
+- **transport-concurrent** — Handles concurrent requests
+- **transport-sse-event-field** — SSE responses include required event: message field
 
 </details>
 
 <details>
-<summary><strong>Lifecycle (10 tests)</strong></summary>
+<summary><strong>Lifecycle (15 tests)</strong></summary>
 
 - **lifecycle-init** — Initialize handshake succeeds (required)
 - **lifecycle-proto-version** — Returns valid YYYY-MM-DD protocol version (required)
@@ -126,8 +132,13 @@ Outputs the markdown embed for a compliance badge hosted at [mcp.hosting](https:
 - **lifecycle-ping** — Responds to ping method (required)
 - **lifecycle-instructions** — Instructions field is valid string if present
 - **lifecycle-id-match** — Response ID matches request ID (required)
+- **lifecycle-string-id** — Supports string request IDs (JSON-RPC 2.0)
+- **lifecycle-version-negotiate** — Handles unknown protocol version gracefully
+- **lifecycle-reinit-reject** — Rejects second initialize request
 - **lifecycle-logging** — logging/setLevel accepted (required if logging capability declared)
 - **lifecycle-completions** — completion/complete accepted (required if completions capability declared)
+- **lifecycle-cancellation** — Handles cancellation notifications
+- **lifecycle-progress** — Handles progress notifications gracefully
 
 </details>
 
@@ -162,7 +173,7 @@ Outputs the markdown embed for a compliance badge hosted at [mcp.hosting](https:
 </details>
 
 <details>
-<summary><strong>Error Handling (8 tests)</strong></summary>
+<summary><strong>Error Handling (10 tests)</strong></summary>
 
 - **error-unknown-method** — Returns JSON-RPC error for unknown method (required)
 - **error-method-code** — Uses correct -32601 error code
@@ -172,6 +183,8 @@ Outputs the markdown embed for a compliance badge hosted at [mcp.hosting](https:
 - **error-parse-code** — Returns -32700 for invalid JSON
 - **error-invalid-request-code** — Returns -32600 for invalid request
 - **tools-call-unknown** — Returns error for nonexistent tool name
+- **error-capability-gated** — Rejects methods for undeclared capabilities
+- **error-invalid-cursor** — Handles invalid pagination cursor gracefully
 
 </details>
 
@@ -187,6 +200,34 @@ Outputs the markdown embed for a compliance badge hosted at [mcp.hosting](https:
 
 </details>
 
+<details>
+<summary><strong>Security (22 tests)</strong></summary>
+
+- **security-auth-required** — Rejects unauthenticated requests
+- **security-auth-malformed** — Rejects malformed auth credentials
+- **security-tls-required** — Enforces HTTPS/TLS
+- **security-session-entropy** — Session IDs are high-entropy
+- **security-session-not-auth** — Session ID does not bypass auth
+- **security-oauth-metadata** — Protected Resource Metadata endpoint exists (RFC 9728)
+- **security-token-in-uri** — Rejects auth tokens in query string
+- **security-cors-headers** — CORS headers are restrictive
+- **security-origin-validation** — Validates Origin header for DNS rebinding protection
+- **security-command-injection** — Resists command injection in tool params
+- **security-sql-injection** — Resists SQL injection in tool params
+- **security-path-traversal** — Resists path traversal in tool params
+- **security-ssrf-internal** — Resists SSRF to internal networks
+- **security-oversized-input** — Handles oversized inputs gracefully
+- **security-extra-params** — Rejects or ignores extra tool params
+- **security-tool-schema-defined** — All tools define inputSchema
+- **security-tool-rug-pull** — Tool definitions are stable across calls
+- **security-tool-description-poisoning** — Tool descriptions free of injection patterns
+- **security-tool-cross-reference** — Tools do not reference other tools by name
+- **security-error-no-stacktrace** — Error responses do not leak stack traces
+- **security-error-no-internal-ip** — Error responses do not leak internal IPs
+- **security-rate-limiting** — Rate limiting is enforced
+
+</details>
+
 ## Grading
 
 | Grade | Score  |
@@ -277,7 +318,7 @@ Restart your MCP client and approve the server when prompted.
 
 ### Tools
 
-- **mcp_compliance_test** — Run the full 43-test suite against a URL. Supports auth, custom headers, timeout, retries, and category/test filtering. Returns grade, score, and detailed results.
+- **mcp_compliance_test** — Run the full 78-test suite against a URL. Supports auth, custom headers, timeout, retries, and category/test filtering. Returns grade, score, and detailed results.
 - **mcp_compliance_badge** — Get the badge markdown/HTML for a server. Supports auth and custom headers.
 - **mcp_compliance_explain** — Explain what a specific test ID checks and why it matters.
 
@@ -304,7 +345,7 @@ const report2 = await runComplianceSuite('https://my-server.com/mcp', {
 
 The compliance testing methodology is published as an open specification:
 
-- **[MCP Compliance Testing Specification](./MCP_COMPLIANCE_SPEC.md)** — test execution model, scoring algorithm, all 43 test rules with pass/fail criteria (CC BY 4.0)
+- **[MCP Compliance Testing Specification](./MCP_COMPLIANCE_SPEC.md)** — test execution model, scoring algorithm, all 78 test rules with pass/fail criteria (CC BY 4.0)
 - **[Machine-readable rule catalog](./mcp-compliance-rules.json)** — JSON Schema-compliant catalog for programmatic consumption
 
 These are complementary to (not competing with) the [official MCP specification](https://modelcontextprotocol.io/specification/2025-11-25). The MCP spec defines what servers must do; this spec defines how to verify compliance.