@yawlabs/mcp-compliance 0.15.1 → 0.16.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -34
- package/dist/{chunk-B5HSDK4K.js → chunk-4RMQADAA.js} +1 -24
- package/dist/index.js +11 -412
- package/dist/mcp/server.js +1 -51
- package/dist/runner.d.ts +6 -22
- package/dist/runner.js +3 -7
- package/package.json +2 -2
- package/schemas/report.v1.json +4 -4
package/README.md
CHANGED
|
@@ -222,37 +222,13 @@ Check in a `mcp-compliance.config.json` so CI and your dev loop can run `mcp-com
|
|
|
222
222
|
|
|
223
223
|
Precedence: CLI flags > config file > defaults. Any field can be overridden on the command line.
|
|
224
224
|
|
|
225
|
-
###
|
|
225
|
+
### Local SVG badge
|
|
226
226
|
|
|
227
|
-
|
|
228
|
-
mcp-compliance badge https://my-server.com/mcp
|
|
229
|
-
```
|
|
230
|
-
|
|
231
|
-
Runs the compliance suite, publishes the report to [mcp.hosting](https://mcp.hosting), and prints the markdown embed for your README. The badge image reflects the real grade (A–F) and links to the full report.
|
|
232
|
-
|
|
233
|
-
| Option | Description |
|
|
234
|
-
|--------|-------------|
|
|
235
|
-
| `-H, --header <header>` | Add header to all requests, format `"Key: Value"` (repeatable) |
|
|
236
|
-
| `--auth <token>` | Shorthand for `-H "Authorization: <token>"` |
|
|
237
|
-
| `--timeout <ms>` | Request timeout in milliseconds (default: `15000`) |
|
|
238
|
-
| `--no-publish` | Skip publishing; print a local badge markdown only |
|
|
239
|
-
| `--output <file>` | Also write a local SVG badge to the given path |
|
|
240
|
-
|
|
241
|
-
Reports are kept for 90 days from last submission; resubmitting the same URL overwrites the previous report. Auth headers are stripped client-side before upload. Private/loopback URLs (`localhost`, `127.0.0.1`, `192.168.*`, etc.) trigger an interactive confirmation before publishing, and are rejected by the server in any case.
|
|
242
|
-
|
|
243
|
-
A delete token is returned at publish time and stored at `~/.mcp-compliance/tokens.json` (mode `0600`). Use it to take a report down:
|
|
244
|
-
|
|
245
|
-
```bash
|
|
246
|
-
mcp-compliance unpublish https://my-server.com/mcp
|
|
247
|
-
```
|
|
248
|
-
|
|
249
|
-
### Local SVG badge (any transport)
|
|
250
|
-
|
|
251
|
-
Stdio servers can't be published (no public URL to key on), but you can commit a local SVG reflecting the real grade:
|
|
227
|
+
Write a local SVG reflecting the real grade and commit it to your repo:
|
|
252
228
|
|
|
253
229
|
```bash
|
|
230
|
+
mcp-compliance test https://my-server.com/mcp --output badge.svg
|
|
254
231
|
mcp-compliance test node ./dist/server.js --output badge.svg
|
|
255
|
-
mcp-compliance badge npx -y @modelcontextprotocol/server-filesystem /tmp --output badge.svg
|
|
256
232
|
```
|
|
257
233
|
|
|
258
234
|
Then embed it in your README:
|
|
@@ -261,8 +237,6 @@ Then embed it in your README:
|
|
|
261
237
|

|
|
262
238
|
```
|
|
263
239
|
|
|
264
|
-
The `test` command never publishes — use it for CI, debugging, and local iteration. `badge` is the only command that publishes to mcp.hosting.
|
|
265
|
-
|
|
266
240
|
## What the 88 tests check
|
|
267
241
|
|
|
268
242
|
<details>
|
|
@@ -495,7 +469,6 @@ Restart your MCP client and approve the server when prompted.
|
|
|
495
469
|
### Tools
|
|
496
470
|
|
|
497
471
|
- **mcp_compliance_test** — Run the full 88-test suite against a URL or stdio command. Supports auth, custom headers, env vars, timeout, retries, and category/test filtering. Returns grade, score, and detailed results.
|
|
498
|
-
- **mcp_compliance_badge** — Get the badge markdown/HTML for a server. Supports auth and custom headers.
|
|
499
472
|
- **mcp_compliance_explain** — Explain what a specific test ID checks and why it matters.
|
|
500
473
|
|
|
501
474
|
All tools have [MCP tool annotations](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#annotations) (`readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`) so MCP clients can skip confirmation dialogs for safe operations.
|
|
@@ -560,11 +533,8 @@ The testing methodology is published openly so the grading is auditable:
|
|
|
560
533
|
- **[Why `mcp-compliance`](./docs/WHY.md)** — the problem, existing alternatives, what this tool does differently
|
|
561
534
|
- **[Fixing common failures](./docs/FIXES.md)** — recipes for the most frequent test failures with code snippets
|
|
562
535
|
- **[Spec version migration policy](./docs/SPEC_VERSION_MIGRATION.md)** — how this tool evolves with MCP spec releases
|
|
563
|
-
- **[mcp.hosting external API](./docs/EXT_API.md)** — public submit/retrieve/badge/delete endpoints used by `mcp-compliance badge` and any custom integrations
|
|
564
|
-
- **[Enterprise tier (draft)](./docs/ENTERPRISE.md)** — paid tier structure for organizations with scheduled/private/audit-track compliance needs
|
|
565
536
|
- **[Performance deep-dive](./docs/PERFORMANCE.md)** — why the suite is sequential and what parallel execution would cost
|
|
566
537
|
- **[Spec PR drafts](./docs/spec-prs/)** — our proposed MCP spec clarifications for ambiguous cases we've hit
|
|
567
|
-
- **[mcp.hosting integration spec](./docs/mcp-hosting-integration.md)** — the contract between this engine and the mcp.hosting platform: URL surfaces, data flow, storage model, badge API, leaderboard, router integration
|
|
568
538
|
|
|
569
539
|
The methodology is not an authoritative conformance standard — it's one tool's choices, published so they can be inspected, adopted, or forked. The [official MCP specification](https://modelcontextprotocol.io/specification/2025-11-25) defines what servers must do; this document describes how `@yawlabs/mcp-compliance` verifies it.
|
|
570
540
|
|
|
@@ -596,7 +566,6 @@ npm test
|
|
|
596
566
|
|
|
597
567
|
## Links
|
|
598
568
|
|
|
599
|
-
- [mcp.hosting](https://mcp.hosting) — Hosted MCP server infrastructure
|
|
600
569
|
- [MCP Specification](https://modelcontextprotocol.io/specification/2025-11-25)
|
|
601
570
|
- [Testing methodology](./COMPLIANCE_RUBRIC.md)
|
|
602
571
|
- [Yaw Labs](https://yaw.sh)
|
|
@@ -1,27 +1,6 @@
|
|
|
1
1
|
// src/runner.ts
|
|
2
2
|
import { request as request2 } from "undici";
|
|
3
3
|
|
|
4
|
-
// src/badge.ts
|
|
5
|
-
import { createHash } from "crypto";
|
|
6
|
-
function urlHash(url) {
|
|
7
|
-
return createHash("sha256").update(url).digest("hex").slice(0, 24);
|
|
8
|
-
}
|
|
9
|
-
function generateBadge(url) {
|
|
10
|
-
const hash = urlHash(url);
|
|
11
|
-
const imageUrl = `https://mcp.hosting/api/compliance/ext/${hash}/badge`;
|
|
12
|
-
const reportUrl = `https://mcp.hosting/compliance/ext/${hash}`;
|
|
13
|
-
return {
|
|
14
|
-
imageUrl,
|
|
15
|
-
reportUrl,
|
|
16
|
-
markdown: `[](${reportUrl})`,
|
|
17
|
-
// loading="lazy" so READMEs that embed many badges don't block first
|
|
18
|
-
// paint on this image. Markdown renderers (GitHub, npmjs.com) emit
|
|
19
|
-
// their own <img> from the markdown form so the attribute only
|
|
20
|
-
// matters for the HTML form people paste into custom pages.
|
|
21
|
-
html: `<a href="${reportUrl}"><img src="${imageUrl}" alt="MCP Compliant" loading="lazy"></a>`
|
|
22
|
-
};
|
|
23
|
-
}
|
|
24
|
-
|
|
25
4
|
// src/grader.ts
|
|
26
5
|
function computeGrade(score) {
|
|
27
6
|
if (score >= 90) return "A";
|
|
@@ -4132,7 +4111,7 @@ async function runComplianceSuite(target, options = {}) {
|
|
|
4132
4111
|
warnings.length = 0;
|
|
4133
4112
|
warnings.push(...capped);
|
|
4134
4113
|
const { score, grade, overall, summary, categories } = computeScore(tests);
|
|
4135
|
-
const badge =
|
|
4114
|
+
const badge = { imageUrl: "", reportUrl: "", markdown: "", html: "" };
|
|
4136
4115
|
return {
|
|
4137
4116
|
schemaVersion: REPORT_SCHEMA_VERSION,
|
|
4138
4117
|
specVersion: SPEC_VERSION,
|
|
@@ -4162,8 +4141,6 @@ async function runComplianceSuite(target, options = {}) {
|
|
|
4162
4141
|
}
|
|
4163
4142
|
|
|
4164
4143
|
export {
|
|
4165
|
-
urlHash,
|
|
4166
|
-
generateBadge,
|
|
4167
4144
|
computeGrade,
|
|
4168
4145
|
computeScore,
|
|
4169
4146
|
readPackageVersion,
|
package/dist/index.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
|
|
3
3
|
// src/index.ts
|
|
4
|
-
import { watch as fsWatch, readFileSync as
|
|
4
|
+
import { watch as fsWatch, readFileSync as readFileSync3, writeFileSync } from "fs";
|
|
5
5
|
import { createRequire } from "module";
|
|
6
6
|
import { createInterface } from "readline/promises";
|
|
7
7
|
import chalk2 from "chalk";
|
|
@@ -748,27 +748,6 @@ import { z } from "zod";
|
|
|
748
748
|
// src/runner.ts
|
|
749
749
|
import { request as request2 } from "undici";
|
|
750
750
|
|
|
751
|
-
// src/badge.ts
|
|
752
|
-
import { createHash } from "crypto";
|
|
753
|
-
function urlHash(url) {
|
|
754
|
-
return createHash("sha256").update(url).digest("hex").slice(0, 24);
|
|
755
|
-
}
|
|
756
|
-
function generateBadge(url) {
|
|
757
|
-
const hash = urlHash(url);
|
|
758
|
-
const imageUrl = `https://mcp.hosting/api/compliance/ext/${hash}/badge`;
|
|
759
|
-
const reportUrl = `https://mcp.hosting/compliance/ext/${hash}`;
|
|
760
|
-
return {
|
|
761
|
-
imageUrl,
|
|
762
|
-
reportUrl,
|
|
763
|
-
markdown: `[](${reportUrl})`,
|
|
764
|
-
// loading="lazy" so READMEs that embed many badges don't block first
|
|
765
|
-
// paint on this image. Markdown renderers (GitHub, npmjs.com) emit
|
|
766
|
-
// their own <img> from the markdown form so the attribute only
|
|
767
|
-
// matters for the HTML form people paste into custom pages.
|
|
768
|
-
html: `<a href="${reportUrl}"><img src="${imageUrl}" alt="MCP Compliant" loading="lazy"></a>`
|
|
769
|
-
};
|
|
770
|
-
}
|
|
771
|
-
|
|
772
751
|
// src/grader.ts
|
|
773
752
|
function computeGrade(score) {
|
|
774
753
|
if (score >= 90) return "A";
|
|
@@ -4491,7 +4470,7 @@ async function runComplianceSuite(target, options = {}) {
|
|
|
4491
4470
|
warnings.length = 0;
|
|
4492
4471
|
warnings.push(...capped);
|
|
4493
4472
|
const { score, grade, overall, summary, categories } = computeScore(tests);
|
|
4494
|
-
const badge =
|
|
4473
|
+
const badge = { imageUrl: "", reportUrl: "", markdown: "", html: "" };
|
|
4495
4474
|
return {
|
|
4496
4475
|
schemaVersion: REPORT_SCHEMA_VERSION,
|
|
4497
4476
|
specVersion: SPEC_VERSION,
|
|
@@ -4588,56 +4567,6 @@ ${JSON.stringify(report, null, 2)}` }
|
|
|
4588
4567
|
}
|
|
4589
4568
|
}
|
|
4590
4569
|
);
|
|
4591
|
-
server.tool(
|
|
4592
|
-
"mcp_compliance_badge",
|
|
4593
|
-
"Get the badge markdown embed code for an MCP server. Runs the compliance test suite first to determine the grade.",
|
|
4594
|
-
{
|
|
4595
|
-
url: z.string().url().describe("The MCP server URL to test"),
|
|
4596
|
-
auth: z.string().optional().describe('Authorization header value (e.g., "Bearer tok123")'),
|
|
4597
|
-
headers: z.record(z.string(), z.string()).optional().describe("Additional headers to include on all requests"),
|
|
4598
|
-
timeout: z.number().int().min(1).max(3e5).optional().describe("Request timeout in milliseconds (default: 15000, max: 300000)")
|
|
4599
|
-
},
|
|
4600
|
-
{
|
|
4601
|
-
title: "Get Compliance Badge",
|
|
4602
|
-
readOnlyHint: true,
|
|
4603
|
-
destructiveHint: false,
|
|
4604
|
-
idempotentHint: true,
|
|
4605
|
-
openWorldHint: true
|
|
4606
|
-
},
|
|
4607
|
-
async ({ url, auth, headers: extraHeaders, timeout }) => {
|
|
4608
|
-
try {
|
|
4609
|
-
const headers = { ...extraHeaders };
|
|
4610
|
-
if (auth) headers.Authorization = auth;
|
|
4611
|
-
const report = await runComplianceSuite(url, {
|
|
4612
|
-
headers: Object.keys(headers).length > 0 ? headers : void 0,
|
|
4613
|
-
timeout
|
|
4614
|
-
});
|
|
4615
|
-
const badge = report.badge;
|
|
4616
|
-
return {
|
|
4617
|
-
content: [
|
|
4618
|
-
{
|
|
4619
|
-
type: "text",
|
|
4620
|
-
text: [
|
|
4621
|
-
`Grade: ${report.grade} (${report.score}%)`,
|
|
4622
|
-
"",
|
|
4623
|
-
"Markdown:",
|
|
4624
|
-
badge.markdown,
|
|
4625
|
-
"",
|
|
4626
|
-
"HTML:",
|
|
4627
|
-
badge.html
|
|
4628
|
-
].join("\n")
|
|
4629
|
-
}
|
|
4630
|
-
]
|
|
4631
|
-
};
|
|
4632
|
-
} catch (err) {
|
|
4633
|
-
const message = err instanceof Error ? err.message : String(err);
|
|
4634
|
-
return {
|
|
4635
|
-
content: [{ type: "text", text: `Error: ${message}` }],
|
|
4636
|
-
isError: true
|
|
4637
|
-
};
|
|
4638
|
-
}
|
|
4639
|
-
}
|
|
4640
|
-
);
|
|
4641
4570
|
server.tool(
|
|
4642
4571
|
"mcp_compliance_explain",
|
|
4643
4572
|
"Explain what a specific compliance test ID checks and why it matters.",
|
|
@@ -4721,75 +4650,6 @@ if (isInvokedDirectly()) {
|
|
|
4721
4650
|
});
|
|
4722
4651
|
}
|
|
4723
4652
|
|
|
4724
|
-
// src/publish.ts
|
|
4725
|
-
import { request as request3 } from "undici";
|
|
4726
|
-
var PUBLISH_BASE = "https://mcp.hosting";
|
|
4727
|
-
var PUBLISH_PATH = "/api/compliance/ext";
|
|
4728
|
-
var PUBLISH_TIMEOUT_MS = 1e4;
|
|
4729
|
-
var PUBLISH_RETRIES = 2;
|
|
4730
|
-
function sanitizeReport(report) {
|
|
4731
|
-
const { headers: _h, auth: _a, ...rest } = report;
|
|
4732
|
-
void _h;
|
|
4733
|
-
void _a;
|
|
4734
|
-
return rest;
|
|
4735
|
-
}
|
|
4736
|
-
async function publishReport(report) {
|
|
4737
|
-
const body = JSON.stringify(sanitizeReport(report));
|
|
4738
|
-
let lastErr;
|
|
4739
|
-
for (let attempt = 0; attempt <= PUBLISH_RETRIES; attempt++) {
|
|
4740
|
-
try {
|
|
4741
|
-
const res = await request3(`${PUBLISH_BASE}${PUBLISH_PATH}`, {
|
|
4742
|
-
method: "POST",
|
|
4743
|
-
headers: { "content-type": "application/json" },
|
|
4744
|
-
body,
|
|
4745
|
-
signal: AbortSignal.timeout(PUBLISH_TIMEOUT_MS)
|
|
4746
|
-
});
|
|
4747
|
-
if (res.statusCode >= 500 && attempt < PUBLISH_RETRIES) {
|
|
4748
|
-
await res.body.dump();
|
|
4749
|
-
await new Promise((r) => setTimeout(r, 500 * (attempt + 1)));
|
|
4750
|
-
continue;
|
|
4751
|
-
}
|
|
4752
|
-
const text = await res.body.text();
|
|
4753
|
-
if (res.statusCode >= 400) {
|
|
4754
|
-
let message = `Publish failed: HTTP ${res.statusCode}`;
|
|
4755
|
-
try {
|
|
4756
|
-
const parsed2 = JSON.parse(text);
|
|
4757
|
-
if (parsed2.error) message = `Publish failed: ${parsed2.error}`;
|
|
4758
|
-
} catch {
|
|
4759
|
-
}
|
|
4760
|
-
throw new Error(message);
|
|
4761
|
-
}
|
|
4762
|
-
const parsed = JSON.parse(text);
|
|
4763
|
-
if (!parsed.hash || !parsed.deleteToken || !parsed.reportUrl || !parsed.badgeUrl) {
|
|
4764
|
-
throw new Error("Publish failed: malformed response from mcp.hosting");
|
|
4765
|
-
}
|
|
4766
|
-
return parsed;
|
|
4767
|
-
} catch (err) {
|
|
4768
|
-
lastErr = err;
|
|
4769
|
-
if (attempt < PUBLISH_RETRIES) {
|
|
4770
|
-
await new Promise((r) => setTimeout(r, 500 * (attempt + 1)));
|
|
4771
|
-
}
|
|
4772
|
-
}
|
|
4773
|
-
}
|
|
4774
|
-
throw lastErr instanceof Error ? lastErr : new Error(String(lastErr));
|
|
4775
|
-
}
|
|
4776
|
-
async function unpublishReport(hash, deleteToken2) {
|
|
4777
|
-
const res = await request3(`${PUBLISH_BASE}${PUBLISH_PATH}/${hash}`, {
|
|
4778
|
-
method: "DELETE",
|
|
4779
|
-
headers: { authorization: `Bearer ${deleteToken2}` },
|
|
4780
|
-
signal: AbortSignal.timeout(PUBLISH_TIMEOUT_MS)
|
|
4781
|
-
});
|
|
4782
|
-
const text = await res.body.text();
|
|
4783
|
-
if (res.statusCode === 204 || res.statusCode === 404) return;
|
|
4784
|
-
let message = `Unpublish failed: HTTP ${res.statusCode}`;
|
|
4785
|
-
try {
|
|
4786
|
-
const parsed = JSON.parse(text);
|
|
4787
|
-
if (parsed.error) message = `Unpublish failed: ${parsed.error}`;
|
|
4788
|
-
} catch {
|
|
4789
|
-
}
|
|
4790
|
-
throw new Error(message);
|
|
4791
|
-
}
|
|
4792
|
-
|
|
4793
4653
|
// src/reporter.ts
|
|
4794
4654
|
import chalk from "chalk";
|
|
4795
4655
|
var CATEGORY_LABELS = {
|
|
@@ -4957,13 +4817,7 @@ function formatTerminal(report) {
|
|
|
4957
4817
|
}
|
|
4958
4818
|
out.push("");
|
|
4959
4819
|
}
|
|
4960
|
-
|
|
4961
|
-
out.push(
|
|
4962
|
-
chalk.dim(" Badge: stdio targets aren't published. Run with --output badge.svg for a local badge image.")
|
|
4963
|
-
);
|
|
4964
|
-
} else {
|
|
4965
|
-
out.push(chalk.dim(` Badge: ${report.badge.markdown}`));
|
|
4966
|
-
}
|
|
4820
|
+
out.push(chalk.dim(" Badge: run with --output badge.svg for a local badge image."));
|
|
4967
4821
|
out.push("");
|
|
4968
4822
|
return out.join("\n");
|
|
4969
4823
|
}
|
|
@@ -5107,14 +4961,6 @@ function formatMarkdown(report) {
|
|
|
5107
4961
|
for (const w of report.warnings) lines.push(`- ${w}`);
|
|
5108
4962
|
lines.push("");
|
|
5109
4963
|
}
|
|
5110
|
-
if (!report.url.startsWith("stdio:")) {
|
|
5111
|
-
lines.push("## Badge");
|
|
5112
|
-
lines.push("");
|
|
5113
|
-
lines.push("```markdown");
|
|
5114
|
-
lines.push(report.badge.markdown);
|
|
5115
|
-
lines.push("```");
|
|
5116
|
-
lines.push("");
|
|
5117
|
-
}
|
|
5118
4964
|
return lines.join("\n");
|
|
5119
4965
|
}
|
|
5120
4966
|
function formatHtml(report) {
|
|
@@ -5133,7 +4979,6 @@ function formatHtml(report) {
|
|
|
5133
4979
|
const grouped = /* @__PURE__ */ new Map();
|
|
5134
4980
|
for (const cat of CATEGORY_ORDER) grouped.set(cat, []);
|
|
5135
4981
|
for (const t of report.tests) grouped.get(t.category)?.push(t);
|
|
5136
|
-
const isStdio = report.url.startsWith("stdio:");
|
|
5137
4982
|
return `<!doctype html>
|
|
5138
4983
|
<html lang="en">
|
|
5139
4984
|
<head>
|
|
@@ -5232,11 +5077,8 @@ function formatHtml(report) {
|
|
|
5232
5077
|
</tbody></table></div>`
|
|
5233
5078
|
).join("")}
|
|
5234
5079
|
|
|
5235
|
-
${
|
|
5236
|
-
<
|
|
5237
|
-
<p class="muted" style="margin-top:12px">Markdown:</p>
|
|
5238
|
-
<code style="display:block; padding:8px; background:#0b0f17">${esc(report.badge.markdown)}</code></div>` : `<div class="card"><h2>Local badge</h2>
|
|
5239
|
-
<p class="muted">Stdio servers can't be published to mcp.hosting (no public URL). Use <code>--output badge.svg</code> to write a local badge image.</p></div>`}
|
|
5080
|
+
${`<div class="card"><h2>Local badge</h2>
|
|
5081
|
+
<p class="muted">Use <code>--output badge.svg</code> to write a local badge image.</p></div>`}
|
|
5240
5082
|
|
|
5241
5083
|
<footer>
|
|
5242
5084
|
Generated by <a href="https://www.npmjs.com/package/@yawlabs/mcp-compliance">@yawlabs/mcp-compliance</a> v${esc(report.toolVersion)}
|
|
@@ -5260,62 +5102,6 @@ function splitStdioTarget(s) {
|
|
|
5260
5102
|
return { command: tokens[0], args: tokens.slice(1) };
|
|
5261
5103
|
}
|
|
5262
5104
|
|
|
5263
|
-
// src/token-store.ts
|
|
5264
|
-
import { existsSync as existsSync3, mkdirSync, readFileSync as readFileSync3, writeFileSync } from "fs";
|
|
5265
|
-
import { homedir } from "os";
|
|
5266
|
-
import { dirname as dirname3, join as join3 } from "path";
|
|
5267
|
-
var STORE_DIR = join3(homedir(), ".mcp-compliance");
|
|
5268
|
-
var STORE_PATH = join3(STORE_DIR, "tokens.json");
|
|
5269
|
-
function hashUrl(url) {
|
|
5270
|
-
return urlHash(url);
|
|
5271
|
-
}
|
|
5272
|
-
function readStore() {
|
|
5273
|
-
if (!existsSync3(STORE_PATH)) return {};
|
|
5274
|
-
try {
|
|
5275
|
-
const raw = readFileSync3(STORE_PATH, "utf8");
|
|
5276
|
-
const parsed = JSON.parse(raw);
|
|
5277
|
-
return parsed && typeof parsed === "object" ? parsed : {};
|
|
5278
|
-
} catch {
|
|
5279
|
-
return {};
|
|
5280
|
-
}
|
|
5281
|
-
}
|
|
5282
|
-
function writeStore(store) {
|
|
5283
|
-
const dir = dirname3(STORE_PATH);
|
|
5284
|
-
try {
|
|
5285
|
-
if (!existsSync3(dir)) mkdirSync(dir, { recursive: true, mode: 448 });
|
|
5286
|
-
writeFileSync(STORE_PATH, JSON.stringify(store, null, 2), { mode: 384 });
|
|
5287
|
-
} catch (err) {
|
|
5288
|
-
const code = err?.code;
|
|
5289
|
-
if (code === "EACCES" || code === "EPERM") {
|
|
5290
|
-
throw new Error(
|
|
5291
|
-
`Cannot write delete-token store at ${STORE_PATH}: permission denied. If you don't need to publish, re-run with --no-publish. Otherwise, ensure your home directory is writable.`
|
|
5292
|
-
);
|
|
5293
|
-
}
|
|
5294
|
-
if (code === "ENOSPC") {
|
|
5295
|
-
throw new Error(`Cannot write delete-token store at ${STORE_PATH}: no space left on device.`);
|
|
5296
|
-
}
|
|
5297
|
-
const message = err instanceof Error ? err.message : String(err);
|
|
5298
|
-
throw new Error(`Cannot write delete-token store at ${STORE_PATH}: ${message}`);
|
|
5299
|
-
}
|
|
5300
|
-
}
|
|
5301
|
-
function saveToken(hash, entry) {
|
|
5302
|
-
const store = readStore();
|
|
5303
|
-
store[hash] = entry;
|
|
5304
|
-
writeStore(store);
|
|
5305
|
-
}
|
|
5306
|
-
function getTokenForUrl(url) {
|
|
5307
|
-
const hash = hashUrl(url);
|
|
5308
|
-
const store = readStore();
|
|
5309
|
-
const entry = store[hash];
|
|
5310
|
-
return entry ? { hash, entry } : null;
|
|
5311
|
-
}
|
|
5312
|
-
function deleteToken(hash) {
|
|
5313
|
-
const store = readStore();
|
|
5314
|
-
if (!(hash in store)) return;
|
|
5315
|
-
delete store[hash];
|
|
5316
|
-
writeStore(store);
|
|
5317
|
-
}
|
|
5318
|
-
|
|
5319
5105
|
// src/index.ts
|
|
5320
5106
|
var require2 = createRequire(import.meta.url);
|
|
5321
5107
|
var { version: version2 } = require2("../package.json");
|
|
@@ -5361,7 +5147,7 @@ function parseEnvVar(value, prev) {
|
|
|
5361
5147
|
function readEnvFile(path) {
|
|
5362
5148
|
let contents;
|
|
5363
5149
|
try {
|
|
5364
|
-
contents =
|
|
5150
|
+
contents = readFileSync3(path, "utf8");
|
|
5365
5151
|
} catch (err) {
|
|
5366
5152
|
const code = err?.code;
|
|
5367
5153
|
if (code === "ENOENT") throw new Error(`--env-file: file not found: ${path}`);
|
|
@@ -5423,55 +5209,6 @@ function resolveTarget(cliTarget, cliExtraArgs, cliOpts, config) {
|
|
|
5423
5209
|
if (config?.target) return config.target;
|
|
5424
5210
|
throw new Error("No target specified. Pass a URL or command, or add 'target' to mcp-compliance.config.json.");
|
|
5425
5211
|
}
|
|
5426
|
-
var PRIVATE_TLD_SUFFIXES = [
|
|
5427
|
-
".local",
|
|
5428
|
-
".localhost",
|
|
5429
|
-
".internal",
|
|
5430
|
-
".corp",
|
|
5431
|
-
".home",
|
|
5432
|
-
".home.arpa",
|
|
5433
|
-
".lan",
|
|
5434
|
-
".intranet",
|
|
5435
|
-
".private",
|
|
5436
|
-
".test",
|
|
5437
|
-
".invalid",
|
|
5438
|
-
".example",
|
|
5439
|
-
".onion"
|
|
5440
|
-
];
|
|
5441
|
-
function isPrivateHost(urlStr) {
|
|
5442
|
-
let host;
|
|
5443
|
-
try {
|
|
5444
|
-
host = new URL(urlStr).hostname.toLowerCase();
|
|
5445
|
-
} catch {
|
|
5446
|
-
return false;
|
|
5447
|
-
}
|
|
5448
|
-
if (host === "localhost") return true;
|
|
5449
|
-
for (const suffix of PRIVATE_TLD_SUFFIXES) {
|
|
5450
|
-
if (host === suffix.slice(1) || host.endsWith(suffix)) return true;
|
|
5451
|
-
}
|
|
5452
|
-
const v4 = host.match(/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/);
|
|
5453
|
-
if (v4) {
|
|
5454
|
-
const [a, b] = v4.slice(1).map(Number);
|
|
5455
|
-
if (a === 10 || a === 127 || a === 169 && b === 254) return true;
|
|
5456
|
-
if (a === 172 && b >= 16 && b <= 31) return true;
|
|
5457
|
-
if (a === 192 && b === 168) return true;
|
|
5458
|
-
if (a === 0) return true;
|
|
5459
|
-
}
|
|
5460
|
-
const v6 = host.replace(/^\[|\]$/g, "");
|
|
5461
|
-
if (v6 === "::1") return true;
|
|
5462
|
-
if (v6.includes(":") && (v6.startsWith("fe80:") || v6.startsWith("fc") || v6.startsWith("fd"))) return true;
|
|
5463
|
-
return false;
|
|
5464
|
-
}
|
|
5465
|
-
async function promptYesNo(message) {
|
|
5466
|
-
if (!process.stdin.isTTY) return false;
|
|
5467
|
-
const rl = createInterface({ input: process.stdin, output: process.stderr });
|
|
5468
|
-
try {
|
|
5469
|
-
const answer = (await rl.question(`${message} [y/N]: `)).trim().toLowerCase();
|
|
5470
|
-
return answer === "y" || answer === "yes";
|
|
5471
|
-
} finally {
|
|
5472
|
-
rl.close();
|
|
5473
|
-
}
|
|
5474
|
-
}
|
|
5475
5212
|
var program = new Command();
|
|
5476
5213
|
program.name("mcp-compliance").description("Test MCP servers for spec compliance").version(version2);
|
|
5477
5214
|
program.command("test").description("Run the full compliance test suite against an MCP server (URL or stdio command)").argument(
|
|
@@ -5610,7 +5347,7 @@ Testing ${describeTarget(transportTarget)}...
|
|
|
5610
5347
|
}
|
|
5611
5348
|
if (opts.output) {
|
|
5612
5349
|
const svg = renderBadgeSvg({ grade: report2.grade, score: report2.score, timestamp: report2.timestamp });
|
|
5613
|
-
|
|
5350
|
+
writeFileSync(opts.output, svg, "utf8");
|
|
5614
5351
|
if (opts.format === "terminal") {
|
|
5615
5352
|
console.log(chalk2.dim(`
|
|
5616
5353
|
Badge SVG written to ${opts.output}`));
|
|
@@ -5693,144 +5430,6 @@ Error: ${message}
|
|
|
5693
5430
|
}
|
|
5694
5431
|
}
|
|
5695
5432
|
);
|
|
5696
|
-
program.command("badge").description("Run tests and publish a shareable compliance badge to mcp.hosting (HTTP targets only)").argument(
|
|
5697
|
-
"[target]",
|
|
5698
|
-
"Server URL, or command to spawn as a stdio server (optional when a config file defines 'target')"
|
|
5699
|
-
).argument("[extraArgs...]", "Additional args passed to the stdio command").option("--config <path>", "Load options from a config file").option(
|
|
5700
|
-
"-H, --header <header>",
|
|
5701
|
-
'Add header to all requests (format: "Key: Value", repeatable; HTTP only)',
|
|
5702
|
-
parseHeaderArg,
|
|
5703
|
-
{}
|
|
5704
|
-
).option("--auth <token>", 'Shorthand for -H "Authorization: <token>" (HTTP only)').option("-E, --env <var>", 'Set env var for stdio command ("KEY=VALUE", repeatable)', parseEnvVar, {}).option("--env-file <path>", "Load env vars from file (stdio only)").option("--cwd <dir>", "Working directory for stdio command").option("--timeout <ms>", "Request timeout in milliseconds (default: 15000, or `timeout` in config)").option("--no-publish", "Do not publish the report to mcp.hosting").option("--output <file>", "Write a local SVG badge to the given path (works for any transport)").option("--no-color", "Disable colored output (also honors NO_COLOR env var)").action(
|
|
5705
|
-
async (target, extraArgs, opts) => {
|
|
5706
|
-
if (opts.color === false) chalk2.level = 0;
|
|
5707
|
-
try {
|
|
5708
|
-
const config = loadConfig(opts.config);
|
|
5709
|
-
const transportTarget = resolveTarget(
|
|
5710
|
-
target,
|
|
5711
|
-
extraArgs,
|
|
5712
|
-
{
|
|
5713
|
-
header: opts.header,
|
|
5714
|
-
auth: opts.auth,
|
|
5715
|
-
env: opts.env,
|
|
5716
|
-
envFile: opts.envFile,
|
|
5717
|
-
cwd: opts.cwd
|
|
5718
|
-
},
|
|
5719
|
-
config
|
|
5720
|
-
);
|
|
5721
|
-
const shouldPublish = opts.publish && transportTarget.type === "http";
|
|
5722
|
-
if (shouldPublish && transportTarget.type === "http" && isPrivateHost(transportTarget.url)) {
|
|
5723
|
-
console.error(
|
|
5724
|
-
chalk2.yellow(
|
|
5725
|
-
`
|
|
5726
|
-
Warning: ${transportTarget.url} looks like a private/internal address. Publishing will send the report (with the URL) to mcp.hosting.`
|
|
5727
|
-
)
|
|
5728
|
-
);
|
|
5729
|
-
const ok = await promptYesNo(chalk2.yellow("Publish anyway?"));
|
|
5730
|
-
if (!ok) {
|
|
5731
|
-
console.error(chalk2.dim("\nAborted. Re-run with --no-publish to skip publishing.\n"));
|
|
5732
|
-
process.exit(1);
|
|
5733
|
-
}
|
|
5734
|
-
}
|
|
5735
|
-
console.log(chalk2.dim(`
|
|
5736
|
-
Testing ${describeTarget(transportTarget)}...
|
|
5737
|
-
`));
|
|
5738
|
-
let badgeTimeout = config?.timeout ?? 15e3;
|
|
5739
|
-
if (opts.timeout !== void 0) badgeTimeout = parsePositiveInt(opts.timeout, "--timeout", 1);
|
|
5740
|
-
const report = await runComplianceSuite(transportTarget, {
|
|
5741
|
-
timeout: badgeTimeout
|
|
5742
|
-
});
|
|
5743
|
-
let markdown = report.badge.markdown;
|
|
5744
|
-
if (shouldPublish && transportTarget.type === "http") {
|
|
5745
|
-
try {
|
|
5746
|
-
const res = await publishReport(report);
|
|
5747
|
-
saveToken(res.hash, {
|
|
5748
|
-
deleteToken: res.deleteToken,
|
|
5749
|
-
url: transportTarget.url,
|
|
5750
|
-
publishedAt: (/* @__PURE__ */ new Date()).toISOString()
|
|
5751
|
-
});
|
|
5752
|
-
markdown = `[](${res.reportUrl})`;
|
|
5753
|
-
console.log(`Grade: ${report.grade} (${report.score}%)
|
|
5754
|
-
`);
|
|
5755
|
-
console.log(markdown);
|
|
5756
|
-
console.log(chalk2.dim(`
|
|
5757
|
-
Report published: ${res.reportUrl}`));
|
|
5758
|
-
console.log(chalk2.dim(`Remove with: mcp-compliance unpublish ${transportTarget.url}
|
|
5759
|
-
`));
|
|
5760
|
-
} catch (err) {
|
|
5761
|
-
const message = err instanceof Error ? err.message : String(err);
|
|
5762
|
-
console.error(chalk2.yellow(`
|
|
5763
|
-
Warning: publish failed \u2014 ${message}`));
|
|
5764
|
-
console.error(chalk2.dim("Falling back to local badge markdown.\n"));
|
|
5765
|
-
console.log(`Grade: ${report.grade} (${report.score}%)
|
|
5766
|
-
`);
|
|
5767
|
-
console.log(markdown);
|
|
5768
|
-
console.log("");
|
|
5769
|
-
}
|
|
5770
|
-
} else {
|
|
5771
|
-
console.log(`Grade: ${report.grade} (${report.score}%)
|
|
5772
|
-
`);
|
|
5773
|
-
if (transportTarget.type === "stdio") {
|
|
5774
|
-
if (!opts.output) {
|
|
5775
|
-
console.log(
|
|
5776
|
-
chalk2.dim(
|
|
5777
|
-
"Stdio servers cannot be published. Pass --output <file.svg> to write a local badge image for your README."
|
|
5778
|
-
)
|
|
5779
|
-
);
|
|
5780
|
-
}
|
|
5781
|
-
} else {
|
|
5782
|
-
console.log(markdown);
|
|
5783
|
-
}
|
|
5784
|
-
console.log("");
|
|
5785
|
-
}
|
|
5786
|
-
if (opts.output) {
|
|
5787
|
-
const svg = renderBadgeSvg({ grade: report.grade, score: report.score, timestamp: report.timestamp });
|
|
5788
|
-
writeFileSync2(opts.output, svg, "utf8");
|
|
5789
|
-
console.log(chalk2.dim(`Badge SVG written to ${opts.output}
|
|
5790
|
-
`));
|
|
5791
|
-
}
|
|
5792
|
-
} catch (err) {
|
|
5793
|
-
const message = err instanceof Error ? err.message : String(err);
|
|
5794
|
-
console.error(chalk2.red(`
|
|
5795
|
-
Error: ${message}
|
|
5796
|
-
`));
|
|
5797
|
-
process.exit(1);
|
|
5798
|
-
}
|
|
5799
|
-
}
|
|
5800
|
-
);
|
|
5801
|
-
program.command("unpublish").description("Remove a previously-published compliance report from mcp.hosting").argument("<url>", "MCP server URL whose report should be removed").option("-y, --yes", "Skip the confirmation prompt (for automation)").action(async (url, opts) => {
|
|
5802
|
-
try {
|
|
5803
|
-
const stored = getTokenForUrl(url);
|
|
5804
|
-
if (!stored) {
|
|
5805
|
-
console.log(chalk2.dim(`
|
|
5806
|
-
No delete token found locally for ${url} \u2014 nothing to unpublish from this machine.`));
|
|
5807
|
-
console.log(
|
|
5808
|
-
chalk2.dim(
|
|
5809
|
-
"(Delete tokens are stored locally at publish time. If you published from a different machine, run unpublish from there.)\n"
|
|
5810
|
-
)
|
|
5811
|
-
);
|
|
5812
|
-
return;
|
|
5813
|
-
}
|
|
5814
|
-
if (!opts.yes) {
|
|
5815
|
-
const ok = await promptYesNo(chalk2.yellow(`Remove the published report for ${url}? This cannot be undone.`));
|
|
5816
|
-
if (!ok) {
|
|
5817
|
-
console.error(chalk2.dim("\nAborted. Re-run with --yes to skip this prompt.\n"));
|
|
5818
|
-
return;
|
|
5819
|
-
}
|
|
5820
|
-
}
|
|
5821
|
-
await unpublishReport(stored.hash, stored.entry.deleteToken);
|
|
5822
|
-
deleteToken(stored.hash);
|
|
5823
|
-
console.log(chalk2.green(`
|
|
5824
|
-
Removed report for ${url}.
|
|
5825
|
-
`));
|
|
5826
|
-
} catch (err) {
|
|
5827
|
-
const message = err instanceof Error ? err.message : String(err);
|
|
5828
|
-
console.error(chalk2.red(`
|
|
5829
|
-
Error: ${message}
|
|
5830
|
-
`));
|
|
5831
|
-
process.exit(1);
|
|
5832
|
-
}
|
|
5833
|
-
});
|
|
5834
5433
|
program.command("benchmark").description("Measure ping latency and throughput against an MCP server (URL or stdio command)").argument("[target]", "Server URL or stdio command").argument("[extraArgs...]", "Additional args for stdio command").option("-r, --requests <n>", "Number of ping requests to send", "100").option("-c, --concurrency <n>", "Concurrent in-flight requests", "1").option("--timeout <ms>", "Per-request timeout in milliseconds", "15000").option("--config <path>", "Load options from a config file").option("--format <format>", "terminal or json", "terminal").option("-H, --header <header>", "HTTP header (repeatable)", parseHeaderArg, {}).option("--auth <token>", 'Shorthand for -H "Authorization: <token>"').option("-E, --env <var>", "Env var for stdio (repeatable)", parseEnvVar, {}).option("--env-file <path>", "Load env vars from file").option("--cwd <dir>", "Working directory for stdio command").action(
|
|
5835
5434
|
async (target, extraArgs, opts) => {
|
|
5836
5435
|
try {
|
|
@@ -5863,8 +5462,8 @@ Error: ${message}
|
|
|
5863
5462
|
);
|
|
5864
5463
|
program.command("diff").description("Compare two compliance JSON reports; exit 1 if there are regressions").argument("<baseline>", "Baseline report JSON file").argument("<current>", "Current report JSON file").option("--format <format>", "terminal or json", "terminal").action((baselinePath, currentPath, opts) => {
|
|
5865
5464
|
try {
|
|
5866
|
-
const baseline = JSON.parse(
|
|
5867
|
-
const current = JSON.parse(
|
|
5465
|
+
const baseline = JSON.parse(readFileSync3(baselinePath, "utf8"));
|
|
5466
|
+
const current = JSON.parse(readFileSync3(currentPath, "utf8"));
|
|
5868
5467
|
const summary = diffReports(baseline, current);
|
|
5869
5468
|
if (opts.format === "json") {
|
|
5870
5469
|
console.log(JSON.stringify(summary, null, 2));
|
|
@@ -5881,10 +5480,10 @@ Error: ${message}
|
|
|
5881
5480
|
}
|
|
5882
5481
|
});
|
|
5883
5482
|
program.command("init").description("Scaffold a mcp-compliance.config.json in the current directory").option("--force", "Overwrite an existing config file without asking").action(async (opts) => {
|
|
5884
|
-
const { existsSync:
|
|
5483
|
+
const { existsSync: existsSync3, writeFileSync: write } = await import("fs");
|
|
5885
5484
|
const { join: joinPath } = await import("path");
|
|
5886
5485
|
const out = joinPath(process.cwd(), "mcp-compliance.config.json");
|
|
5887
|
-
if (
|
|
5486
|
+
if (existsSync3(out) && !opts.force) {
|
|
5888
5487
|
console.error(chalk2.red(`
|
|
5889
5488
|
A config already exists at ${out}.`));
|
|
5890
5489
|
console.error(chalk2.dim("Re-run with --force to overwrite.\n"));
|
package/dist/mcp/server.js
CHANGED
|
@@ -3,7 +3,7 @@ import {
|
|
|
3
3
|
TEST_DEFINITIONS,
|
|
4
4
|
readPackageVersion,
|
|
5
5
|
runComplianceSuite
|
|
6
|
-
} from "../chunk-
|
|
6
|
+
} from "../chunk-4RMQADAA.js";
|
|
7
7
|
|
|
8
8
|
// src/mcp/server.ts
|
|
9
9
|
import { realpathSync } from "fs";
|
|
@@ -81,56 +81,6 @@ ${JSON.stringify(report, null, 2)}` }
|
|
|
81
81
|
}
|
|
82
82
|
}
|
|
83
83
|
);
|
|
84
|
-
server.tool(
|
|
85
|
-
"mcp_compliance_badge",
|
|
86
|
-
"Get the badge markdown embed code for an MCP server. Runs the compliance test suite first to determine the grade.",
|
|
87
|
-
{
|
|
88
|
-
url: z.string().url().describe("The MCP server URL to test"),
|
|
89
|
-
auth: z.string().optional().describe('Authorization header value (e.g., "Bearer tok123")'),
|
|
90
|
-
headers: z.record(z.string(), z.string()).optional().describe("Additional headers to include on all requests"),
|
|
91
|
-
timeout: z.number().int().min(1).max(3e5).optional().describe("Request timeout in milliseconds (default: 15000, max: 300000)")
|
|
92
|
-
},
|
|
93
|
-
{
|
|
94
|
-
title: "Get Compliance Badge",
|
|
95
|
-
readOnlyHint: true,
|
|
96
|
-
destructiveHint: false,
|
|
97
|
-
idempotentHint: true,
|
|
98
|
-
openWorldHint: true
|
|
99
|
-
},
|
|
100
|
-
async ({ url, auth, headers: extraHeaders, timeout }) => {
|
|
101
|
-
try {
|
|
102
|
-
const headers = { ...extraHeaders };
|
|
103
|
-
if (auth) headers.Authorization = auth;
|
|
104
|
-
const report = await runComplianceSuite(url, {
|
|
105
|
-
headers: Object.keys(headers).length > 0 ? headers : void 0,
|
|
106
|
-
timeout
|
|
107
|
-
});
|
|
108
|
-
const badge = report.badge;
|
|
109
|
-
return {
|
|
110
|
-
content: [
|
|
111
|
-
{
|
|
112
|
-
type: "text",
|
|
113
|
-
text: [
|
|
114
|
-
`Grade: ${report.grade} (${report.score}%)`,
|
|
115
|
-
"",
|
|
116
|
-
"Markdown:",
|
|
117
|
-
badge.markdown,
|
|
118
|
-
"",
|
|
119
|
-
"HTML:",
|
|
120
|
-
badge.html
|
|
121
|
-
].join("\n")
|
|
122
|
-
}
|
|
123
|
-
]
|
|
124
|
-
};
|
|
125
|
-
} catch (err) {
|
|
126
|
-
const message = err instanceof Error ? err.message : String(err);
|
|
127
|
-
return {
|
|
128
|
-
content: [{ type: "text", text: `Error: ${message}` }],
|
|
129
|
-
isError: true
|
|
130
|
-
};
|
|
131
|
-
}
|
|
132
|
-
}
|
|
133
|
-
);
|
|
134
84
|
server.tool(
|
|
135
85
|
"mcp_compliance_explain",
|
|
136
86
|
"Explain what a specific compliance test ID checks and why it matters.",
|
package/dist/runner.d.ts
CHANGED
|
@@ -46,6 +46,11 @@ interface ComplianceReport {
|
|
|
46
46
|
resourceNames: string[];
|
|
47
47
|
promptCount: number;
|
|
48
48
|
promptNames: string[];
|
|
49
|
+
/**
|
|
50
|
+
* @deprecated The hosted badge renderer (mcp.hosting) is retired; these
|
|
51
|
+
* fields are always empty and will be removed in schema v2. For a local
|
|
52
|
+
* badge image, use the CLI's `--output <file>.svg`.
|
|
53
|
+
*/
|
|
49
54
|
badge: {
|
|
50
55
|
imageUrl: string;
|
|
51
56
|
reportUrl: string;
|
|
@@ -92,27 +97,6 @@ type TransportTarget = {
|
|
|
92
97
|
/** All 88 test IDs with descriptions for the explain command */
|
|
93
98
|
declare const TEST_DEFINITIONS: TestDefinition[];
|
|
94
99
|
|
|
95
|
-
/**
|
|
96
|
-
* Generate a short, deterministic hash of a URL for badge paths.
|
|
97
|
-
* SHA-256 truncated to 24 hex chars (96 bits of entropy) — matches the
|
|
98
|
-
* server-side hash width used by mcp.hosting for `/compliance/ext/<hash>`.
|
|
99
|
-
*
|
|
100
|
-
* Exported so mcp.hosting (and other consumers) can compute matching
|
|
101
|
-
* hashes when looking up reports/badges by URL. The hash is the canonical
|
|
102
|
-
* key for `/compliance/ext/<hash>` and `/api/compliance/ext/<hash>/badge`.
|
|
103
|
-
*/
|
|
104
|
-
declare function urlHash(url: string): string;
|
|
105
|
-
/**
|
|
106
|
-
* Generate badge URLs and markdown for a compliance report.
|
|
107
|
-
* Badge images are served by mcp.hosting.
|
|
108
|
-
*/
|
|
109
|
-
declare function generateBadge(url: string): {
|
|
110
|
-
imageUrl: string;
|
|
111
|
-
reportUrl: string;
|
|
112
|
-
markdown: string;
|
|
113
|
-
html: string;
|
|
114
|
-
};
|
|
115
|
-
|
|
116
100
|
declare function computeGrade(score: number): Grade;
|
|
117
101
|
declare function computeScore(tests: TestResult[]): {
|
|
118
102
|
score: number;
|
|
@@ -226,4 +210,4 @@ interface RunOptions {
|
|
|
226
210
|
*/
|
|
227
211
|
declare function runComplianceSuite(target: string | TransportTarget, options?: RunOptions): Promise<ComplianceReport>;
|
|
228
212
|
|
|
229
|
-
export { type ComplianceReport, type PreviewOptions, type RunOptions, SPEC_BASE, SPEC_VERSION, TEST_DEFINITIONS, type TestResult, computeGrade, computeScore, dedupAndCapWarnings,
|
|
213
|
+
export { type ComplianceReport, type PreviewOptions, type RunOptions, SPEC_BASE, SPEC_VERSION, TEST_DEFINITIONS, type TestResult, computeGrade, computeScore, dedupAndCapWarnings, parseSSEResponse, previewTests, runComplianceSuite };
|
package/dist/runner.js
CHANGED
|
@@ -5,12 +5,10 @@ import {
|
|
|
5
5
|
computeGrade,
|
|
6
6
|
computeScore,
|
|
7
7
|
dedupAndCapWarnings,
|
|
8
|
-
generateBadge,
|
|
9
8
|
parseSSEResponse,
|
|
10
9
|
previewTests,
|
|
11
|
-
runComplianceSuite
|
|
12
|
-
|
|
13
|
-
} from "./chunk-B5HSDK4K.js";
|
|
10
|
+
runComplianceSuite
|
|
11
|
+
} from "./chunk-4RMQADAA.js";
|
|
14
12
|
export {
|
|
15
13
|
SPEC_BASE,
|
|
16
14
|
SPEC_VERSION,
|
|
@@ -18,9 +16,7 @@ export {
|
|
|
18
16
|
computeGrade,
|
|
19
17
|
computeScore,
|
|
20
18
|
dedupAndCapWarnings,
|
|
21
|
-
generateBadge,
|
|
22
19
|
parseSSEResponse,
|
|
23
20
|
previewTests,
|
|
24
|
-
runComplianceSuite
|
|
25
|
-
urlHash
|
|
21
|
+
runComplianceSuite
|
|
26
22
|
};
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@yawlabs/mcp-compliance",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.16.1",
|
|
4
4
|
"mcpName": "io.github.YawLabs/mcp-compliance",
|
|
5
5
|
"description": "CLI tool and MCP server that tests MCP servers for spec compliance",
|
|
6
6
|
"license": "MIT",
|
|
@@ -79,5 +79,5 @@
|
|
|
79
79
|
"bugs": {
|
|
80
80
|
"url": "https://github.com/YawLabs/mcp-compliance/issues"
|
|
81
81
|
},
|
|
82
|
-
"homepage": "https://mcp
|
|
82
|
+
"homepage": "https://github.com/YawLabs/mcp-compliance"
|
|
83
83
|
}
|
package/schemas/report.v1.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
-
"$id": "https://
|
|
3
|
+
"$id": "https://yaw.sh/schemas/mcp-compliance/report.v1.json",
|
|
4
4
|
"title": "MCP Compliance Report",
|
|
5
|
-
"description": "Structured output of the mcp-compliance test suite. Stable, versioned contract — increment schemaVersion on breaking changes. Consumed by
|
|
5
|
+
"description": "Structured output of the mcp-compliance test suite. Stable, versioned contract — increment schemaVersion on breaking changes. Consumed by Yaw MCP and third-party dashboards.",
|
|
6
6
|
"type": "object",
|
|
7
7
|
"required": [
|
|
8
8
|
"schemaVersion",
|
|
@@ -120,8 +120,8 @@
|
|
|
120
120
|
"type": "object",
|
|
121
121
|
"required": ["imageUrl", "reportUrl", "markdown", "html"],
|
|
122
122
|
"properties": {
|
|
123
|
-
"imageUrl": { "type": "string"
|
|
124
|
-
"reportUrl": { "type": "string"
|
|
123
|
+
"imageUrl": { "type": "string" },
|
|
124
|
+
"reportUrl": { "type": "string" },
|
|
125
125
|
"markdown": { "type": "string" },
|
|
126
126
|
"html": { "type": "string" }
|
|
127
127
|
},
|