@yawlabs/mcp-compliance 0.15.0 → 0.16.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -222,37 +222,13 @@ Check in a `mcp-compliance.config.json` so CI and your dev loop can run `mcp-com
222
222
 
223
223
  Precedence: CLI flags > config file > defaults. Any field can be overridden on the command line.
224
224
 
225
- ### Publish a shareable badge (HTTP only)
225
+ ### Local SVG badge
226
226
 
227
- ```bash
228
- mcp-compliance badge https://my-server.com/mcp
229
- ```
230
-
231
- Runs the compliance suite, publishes the report to [mcp.hosting](https://mcp.hosting), and prints the markdown embed for your README. The badge image reflects the real grade (A–F) and links to the full report.
232
-
233
- | Option | Description |
234
- |--------|-------------|
235
- | `-H, --header <header>` | Add header to all requests, format `"Key: Value"` (repeatable) |
236
- | `--auth <token>` | Shorthand for `-H "Authorization: <token>"` |
237
- | `--timeout <ms>` | Request timeout in milliseconds (default: `15000`) |
238
- | `--no-publish` | Skip publishing; print a local badge markdown only |
239
- | `--output <file>` | Also write a local SVG badge to the given path |
240
-
241
- Reports are kept for 90 days from last submission; resubmitting the same URL overwrites the previous report. Auth headers are stripped client-side before upload. Private/loopback URLs (`localhost`, `127.0.0.1`, `192.168.*`, etc.) trigger an interactive confirmation before publishing, and are rejected by the server in any case.
242
-
243
- A delete token is returned at publish time and stored at `~/.mcp-compliance/tokens.json` (mode `0600`). Use it to take a report down:
244
-
245
- ```bash
246
- mcp-compliance unpublish https://my-server.com/mcp
247
- ```
248
-
249
- ### Local SVG badge (any transport)
250
-
251
- Stdio servers can't be published (no public URL to key on), but you can commit a local SVG reflecting the real grade:
227
+ Write a local SVG reflecting the real grade and commit it to your repo:
252
228
 
253
229
  ```bash
230
+ mcp-compliance test https://my-server.com/mcp --output badge.svg
254
231
  mcp-compliance test node ./dist/server.js --output badge.svg
255
- mcp-compliance badge npx -y @modelcontextprotocol/server-filesystem /tmp --output badge.svg
256
232
  ```
257
233
 
258
234
  Then embed it in your README:
@@ -261,8 +237,6 @@ Then embed it in your README:
261
237
  ![MCP Compliance](./badge.svg)
262
238
  ```
263
239
 
264
- The `test` command never publishes — use it for CI, debugging, and local iteration. `badge` is the only command that publishes to mcp.hosting.
265
-
266
240
  ## What the 88 tests check
267
241
 
268
242
  <details>
@@ -495,7 +469,6 @@ Restart your MCP client and approve the server when prompted.
495
469
  ### Tools
496
470
 
497
471
  - **mcp_compliance_test** — Run the full 88-test suite against a URL or stdio command. Supports auth, custom headers, env vars, timeout, retries, and category/test filtering. Returns grade, score, and detailed results.
498
- - **mcp_compliance_badge** — Get the badge markdown/HTML for a server. Supports auth and custom headers.
499
472
  - **mcp_compliance_explain** — Explain what a specific test ID checks and why it matters.
500
473
 
501
474
  All tools have [MCP tool annotations](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#annotations) (`readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`) so MCP clients can skip confirmation dialogs for safe operations.
@@ -560,11 +533,8 @@ The testing methodology is published openly so the grading is auditable:
560
533
  - **[Why `mcp-compliance`](./docs/WHY.md)** — the problem, existing alternatives, what this tool does differently
561
534
  - **[Fixing common failures](./docs/FIXES.md)** — recipes for the most frequent test failures with code snippets
562
535
  - **[Spec version migration policy](./docs/SPEC_VERSION_MIGRATION.md)** — how this tool evolves with MCP spec releases
563
- - **[mcp.hosting external API](./docs/EXT_API.md)** — public submit/retrieve/badge/delete endpoints used by `mcp-compliance badge` and any custom integrations
564
- - **[Enterprise tier (draft)](./docs/ENTERPRISE.md)** — paid tier structure for organizations with scheduled/private/audit-track compliance needs
565
536
  - **[Performance deep-dive](./docs/PERFORMANCE.md)** — why the suite is sequential and what parallel execution would cost
566
537
  - **[Spec PR drafts](./docs/spec-prs/)** — our proposed MCP spec clarifications for ambiguous cases we've hit
567
- - **[mcp.hosting integration spec](./docs/mcp-hosting-integration.md)** — the contract between this engine and the mcp.hosting platform: URL surfaces, data flow, storage model, badge API, leaderboard, router integration
568
538
 
569
539
  The methodology is not an authoritative conformance standard — it's one tool's choices, published so they can be inspected, adopted, or forked. The [official MCP specification](https://modelcontextprotocol.io/specification/2025-11-25) defines what servers must do; this document describes how `@yawlabs/mcp-compliance` verifies it.
570
540
 
@@ -596,7 +566,6 @@ npm test
596
566
 
597
567
  ## Links
598
568
 
599
- - [mcp.hosting](https://mcp.hosting) — Hosted MCP server infrastructure
600
569
  - [MCP Specification](https://modelcontextprotocol.io/specification/2025-11-25)
601
570
  - [Testing methodology](./COMPLIANCE_RUBRIC.md)
602
571
  - [Yaw Labs](https://yaw.sh)
@@ -1,27 +1,6 @@
1
1
  // src/runner.ts
2
2
  import { request as request2 } from "undici";
3
3
 
4
- // src/badge.ts
5
- import { createHash } from "crypto";
6
- function urlHash(url) {
7
- return createHash("sha256").update(url).digest("hex").slice(0, 24);
8
- }
9
- function generateBadge(url) {
10
- const hash = urlHash(url);
11
- const imageUrl = `https://mcp.hosting/api/compliance/ext/${hash}/badge`;
12
- const reportUrl = `https://mcp.hosting/compliance/ext/${hash}`;
13
- return {
14
- imageUrl,
15
- reportUrl,
16
- markdown: `[![MCP Compliant](${imageUrl})](${reportUrl})`,
17
- // loading="lazy" so READMEs that embed many badges don't block first
18
- // paint on this image. Markdown renderers (GitHub, npmjs.com) emit
19
- // their own <img> from the markdown form so the attribute only
20
- // matters for the HTML form people paste into custom pages.
21
- html: `<a href="${reportUrl}"><img src="${imageUrl}" alt="MCP Compliant" loading="lazy"></a>`
22
- };
23
- }
24
-
25
4
  // src/grader.ts
26
5
  function computeGrade(score) {
27
6
  if (score >= 90) return "A";
@@ -3209,7 +3188,7 @@ async function runComplianceSuite(target, options = {}) {
3209
3188
  return { passed: true, details: `HTTP ${res.statusCode} (unauthenticated request rejected)` };
3210
3189
  }
3211
3190
  return { passed: false, details: `HTTP ${res.statusCode} \u2014 server accepted unauthenticated request` };
3212
- } catch (err) {
3191
+ } catch (_err) {
3213
3192
  return { passed: true, details: "Connection rejected (acceptable)" };
3214
3193
  }
3215
3194
  }
@@ -3271,7 +3250,7 @@ async function runComplianceSuite(target, options = {}) {
3271
3250
  return { passed: true, details: `HTTP ${res.statusCode} (malformed auth rejected)` };
3272
3251
  }
3273
3252
  return { passed: false, details: `HTTP ${res.statusCode} \u2014 server accepted malformed auth token` };
3274
- } catch (err) {
3253
+ } catch (_err) {
3275
3254
  return { passed: true, details: "Connection rejected (acceptable)" };
3276
3255
  }
3277
3256
  }
@@ -3784,7 +3763,7 @@ async function runComplianceSuite(target, options = {}) {
3784
3763
  if (!toolsListOk) return { passed: true, details: "Skipped: tools/list not available" };
3785
3764
  const tools = cachedToolsList ?? [];
3786
3765
  if (tools.length === 0) return { passed: true, details: "No tools to validate" };
3787
- const missing = tools.filter((t) => !t.inputSchema || t.inputSchema.type !== "object");
3766
+ const missing = tools.filter((t) => t.inputSchema?.type !== "object");
3788
3767
  if (missing.length > 0) {
3789
3768
  return {
3790
3769
  passed: false,
@@ -4132,7 +4111,7 @@ async function runComplianceSuite(target, options = {}) {
4132
4111
  warnings.length = 0;
4133
4112
  warnings.push(...capped);
4134
4113
  const { score, grade, overall, summary, categories } = computeScore(tests);
4135
- const badge = displayUrl.startsWith("stdio:") ? { imageUrl: "", reportUrl: "", markdown: "", html: "" } : generateBadge(displayUrl);
4114
+ const badge = { imageUrl: "", reportUrl: "", markdown: "", html: "" };
4136
4115
  return {
4137
4116
  schemaVersion: REPORT_SCHEMA_VERSION,
4138
4117
  specVersion: SPEC_VERSION,
@@ -4162,8 +4141,6 @@ async function runComplianceSuite(target, options = {}) {
4162
4141
  }
4163
4142
 
4164
4143
  export {
4165
- urlHash,
4166
- generateBadge,
4167
4144
  computeGrade,
4168
4145
  computeScore,
4169
4146
  readPackageVersion,
package/dist/index.js CHANGED
@@ -1,7 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
 
3
3
  // src/index.ts
4
- import { watch as fsWatch, readFileSync as readFileSync4, writeFileSync as writeFileSync2 } from "fs";
4
+ import { watch as fsWatch, readFileSync as readFileSync3, writeFileSync } from "fs";
5
5
  import { createRequire } from "module";
6
6
  import { createInterface } from "readline/promises";
7
7
  import chalk2 from "chalk";
@@ -748,27 +748,6 @@ import { z } from "zod";
748
748
  // src/runner.ts
749
749
  import { request as request2 } from "undici";
750
750
 
751
- // src/badge.ts
752
- import { createHash } from "crypto";
753
- function urlHash(url) {
754
- return createHash("sha256").update(url).digest("hex").slice(0, 24);
755
- }
756
- function generateBadge(url) {
757
- const hash = urlHash(url);
758
- const imageUrl = `https://mcp.hosting/api/compliance/ext/${hash}/badge`;
759
- const reportUrl = `https://mcp.hosting/compliance/ext/${hash}`;
760
- return {
761
- imageUrl,
762
- reportUrl,
763
- markdown: `[![MCP Compliant](${imageUrl})](${reportUrl})`,
764
- // loading="lazy" so READMEs that embed many badges don't block first
765
- // paint on this image. Markdown renderers (GitHub, npmjs.com) emit
766
- // their own <img> from the markdown form so the attribute only
767
- // matters for the HTML form people paste into custom pages.
768
- html: `<a href="${reportUrl}"><img src="${imageUrl}" alt="MCP Compliant" loading="lazy"></a>`
769
- };
770
- }
771
-
772
751
  // src/grader.ts
773
752
  function computeGrade(score) {
774
753
  if (score >= 90) return "A";
@@ -3568,7 +3547,7 @@ async function runComplianceSuite(target, options = {}) {
3568
3547
  return { passed: true, details: `HTTP ${res.statusCode} (unauthenticated request rejected)` };
3569
3548
  }
3570
3549
  return { passed: false, details: `HTTP ${res.statusCode} \u2014 server accepted unauthenticated request` };
3571
- } catch (err) {
3550
+ } catch (_err) {
3572
3551
  return { passed: true, details: "Connection rejected (acceptable)" };
3573
3552
  }
3574
3553
  }
@@ -3630,7 +3609,7 @@ async function runComplianceSuite(target, options = {}) {
3630
3609
  return { passed: true, details: `HTTP ${res.statusCode} (malformed auth rejected)` };
3631
3610
  }
3632
3611
  return { passed: false, details: `HTTP ${res.statusCode} \u2014 server accepted malformed auth token` };
3633
- } catch (err) {
3612
+ } catch (_err) {
3634
3613
  return { passed: true, details: "Connection rejected (acceptable)" };
3635
3614
  }
3636
3615
  }
@@ -4143,7 +4122,7 @@ async function runComplianceSuite(target, options = {}) {
4143
4122
  if (!toolsListOk) return { passed: true, details: "Skipped: tools/list not available" };
4144
4123
  const tools = cachedToolsList ?? [];
4145
4124
  if (tools.length === 0) return { passed: true, details: "No tools to validate" };
4146
- const missing = tools.filter((t) => !t.inputSchema || t.inputSchema.type !== "object");
4125
+ const missing = tools.filter((t) => t.inputSchema?.type !== "object");
4147
4126
  if (missing.length > 0) {
4148
4127
  return {
4149
4128
  passed: false,
@@ -4491,7 +4470,7 @@ async function runComplianceSuite(target, options = {}) {
4491
4470
  warnings.length = 0;
4492
4471
  warnings.push(...capped);
4493
4472
  const { score, grade, overall, summary, categories } = computeScore(tests);
4494
- const badge = displayUrl.startsWith("stdio:") ? { imageUrl: "", reportUrl: "", markdown: "", html: "" } : generateBadge(displayUrl);
4473
+ const badge = { imageUrl: "", reportUrl: "", markdown: "", html: "" };
4495
4474
  return {
4496
4475
  schemaVersion: REPORT_SCHEMA_VERSION,
4497
4476
  specVersion: SPEC_VERSION,
@@ -4588,56 +4567,6 @@ ${JSON.stringify(report, null, 2)}` }
4588
4567
  }
4589
4568
  }
4590
4569
  );
4591
- server.tool(
4592
- "mcp_compliance_badge",
4593
- "Get the badge markdown embed code for an MCP server. Runs the compliance test suite first to determine the grade.",
4594
- {
4595
- url: z.string().url().describe("The MCP server URL to test"),
4596
- auth: z.string().optional().describe('Authorization header value (e.g., "Bearer tok123")'),
4597
- headers: z.record(z.string(), z.string()).optional().describe("Additional headers to include on all requests"),
4598
- timeout: z.number().int().min(1).max(3e5).optional().describe("Request timeout in milliseconds (default: 15000, max: 300000)")
4599
- },
4600
- {
4601
- title: "Get Compliance Badge",
4602
- readOnlyHint: true,
4603
- destructiveHint: false,
4604
- idempotentHint: true,
4605
- openWorldHint: true
4606
- },
4607
- async ({ url, auth, headers: extraHeaders, timeout }) => {
4608
- try {
4609
- const headers = { ...extraHeaders };
4610
- if (auth) headers.Authorization = auth;
4611
- const report = await runComplianceSuite(url, {
4612
- headers: Object.keys(headers).length > 0 ? headers : void 0,
4613
- timeout
4614
- });
4615
- const badge = report.badge;
4616
- return {
4617
- content: [
4618
- {
4619
- type: "text",
4620
- text: [
4621
- `Grade: ${report.grade} (${report.score}%)`,
4622
- "",
4623
- "Markdown:",
4624
- badge.markdown,
4625
- "",
4626
- "HTML:",
4627
- badge.html
4628
- ].join("\n")
4629
- }
4630
- ]
4631
- };
4632
- } catch (err) {
4633
- const message = err instanceof Error ? err.message : String(err);
4634
- return {
4635
- content: [{ type: "text", text: `Error: ${message}` }],
4636
- isError: true
4637
- };
4638
- }
4639
- }
4640
- );
4641
4570
  server.tool(
4642
4571
  "mcp_compliance_explain",
4643
4572
  "Explain what a specific compliance test ID checks and why it matters.",
@@ -4721,75 +4650,6 @@ if (isInvokedDirectly()) {
4721
4650
  });
4722
4651
  }
4723
4652
 
4724
- // src/publish.ts
4725
- import { request as request3 } from "undici";
4726
- var PUBLISH_BASE = "https://mcp.hosting";
4727
- var PUBLISH_PATH = "/api/compliance/ext";
4728
- var PUBLISH_TIMEOUT_MS = 1e4;
4729
- var PUBLISH_RETRIES = 2;
4730
- function sanitizeReport(report) {
4731
- const { headers: _h, auth: _a, ...rest } = report;
4732
- void _h;
4733
- void _a;
4734
- return rest;
4735
- }
4736
- async function publishReport(report) {
4737
- const body = JSON.stringify(sanitizeReport(report));
4738
- let lastErr;
4739
- for (let attempt = 0; attempt <= PUBLISH_RETRIES; attempt++) {
4740
- try {
4741
- const res = await request3(`${PUBLISH_BASE}${PUBLISH_PATH}`, {
4742
- method: "POST",
4743
- headers: { "content-type": "application/json" },
4744
- body,
4745
- signal: AbortSignal.timeout(PUBLISH_TIMEOUT_MS)
4746
- });
4747
- if (res.statusCode >= 500 && attempt < PUBLISH_RETRIES) {
4748
- await res.body.dump();
4749
- await new Promise((r) => setTimeout(r, 500 * (attempt + 1)));
4750
- continue;
4751
- }
4752
- const text = await res.body.text();
4753
- if (res.statusCode >= 400) {
4754
- let message = `Publish failed: HTTP ${res.statusCode}`;
4755
- try {
4756
- const parsed2 = JSON.parse(text);
4757
- if (parsed2.error) message = `Publish failed: ${parsed2.error}`;
4758
- } catch {
4759
- }
4760
- throw new Error(message);
4761
- }
4762
- const parsed = JSON.parse(text);
4763
- if (!parsed.hash || !parsed.deleteToken || !parsed.reportUrl || !parsed.badgeUrl) {
4764
- throw new Error("Publish failed: malformed response from mcp.hosting");
4765
- }
4766
- return parsed;
4767
- } catch (err) {
4768
- lastErr = err;
4769
- if (attempt < PUBLISH_RETRIES) {
4770
- await new Promise((r) => setTimeout(r, 500 * (attempt + 1)));
4771
- }
4772
- }
4773
- }
4774
- throw lastErr instanceof Error ? lastErr : new Error(String(lastErr));
4775
- }
4776
- async function unpublishReport(hash, deleteToken2) {
4777
- const res = await request3(`${PUBLISH_BASE}${PUBLISH_PATH}/${hash}`, {
4778
- method: "DELETE",
4779
- headers: { authorization: `Bearer ${deleteToken2}` },
4780
- signal: AbortSignal.timeout(PUBLISH_TIMEOUT_MS)
4781
- });
4782
- const text = await res.body.text();
4783
- if (res.statusCode === 204 || res.statusCode === 404) return;
4784
- let message = `Unpublish failed: HTTP ${res.statusCode}`;
4785
- try {
4786
- const parsed = JSON.parse(text);
4787
- if (parsed.error) message = `Unpublish failed: ${parsed.error}`;
4788
- } catch {
4789
- }
4790
- throw new Error(message);
4791
- }
4792
-
4793
4653
  // src/reporter.ts
4794
4654
  import chalk from "chalk";
4795
4655
  var CATEGORY_LABELS = {
@@ -4957,13 +4817,7 @@ function formatTerminal(report) {
4957
4817
  }
4958
4818
  out.push("");
4959
4819
  }
4960
- if (report.url.startsWith("stdio:")) {
4961
- out.push(
4962
- chalk.dim(" Badge: stdio targets aren't published. Run with --output badge.svg for a local badge image.")
4963
- );
4964
- } else {
4965
- out.push(chalk.dim(` Badge: ${report.badge.markdown}`));
4966
- }
4820
+ out.push(chalk.dim(" Badge: run with --output badge.svg for a local badge image."));
4967
4821
  out.push("");
4968
4822
  return out.join("\n");
4969
4823
  }
@@ -5107,14 +4961,6 @@ function formatMarkdown(report) {
5107
4961
  for (const w of report.warnings) lines.push(`- ${w}`);
5108
4962
  lines.push("");
5109
4963
  }
5110
- if (!report.url.startsWith("stdio:")) {
5111
- lines.push("## Badge");
5112
- lines.push("");
5113
- lines.push("```markdown");
5114
- lines.push(report.badge.markdown);
5115
- lines.push("```");
5116
- lines.push("");
5117
- }
5118
4964
  return lines.join("\n");
5119
4965
  }
5120
4966
  function formatHtml(report) {
@@ -5133,7 +4979,6 @@ function formatHtml(report) {
5133
4979
  const grouped = /* @__PURE__ */ new Map();
5134
4980
  for (const cat of CATEGORY_ORDER) grouped.set(cat, []);
5135
4981
  for (const t of report.tests) grouped.get(t.category)?.push(t);
5136
- const isStdio = report.url.startsWith("stdio:");
5137
4982
  return `<!doctype html>
5138
4983
  <html lang="en">
5139
4984
  <head>
@@ -5232,11 +5077,8 @@ function formatHtml(report) {
5232
5077
  </tbody></table></div>`
5233
5078
  ).join("")}
5234
5079
 
5235
- ${!isStdio ? `<div class="card"><h2>Embed badge</h2>
5236
- <div class="badge-img"><img src="${esc(report.badge.imageUrl)}" alt="MCP Compliance"></div>
5237
- <p class="muted" style="margin-top:12px">Markdown:</p>
5238
- <code style="display:block; padding:8px; background:#0b0f17">${esc(report.badge.markdown)}</code></div>` : `<div class="card"><h2>Local badge</h2>
5239
- <p class="muted">Stdio servers can't be published to mcp.hosting (no public URL). Use <code>--output badge.svg</code> to write a local badge image.</p></div>`}
5080
+ ${`<div class="card"><h2>Local badge</h2>
5081
+ <p class="muted">Use <code>--output badge.svg</code> to write a local badge image.</p></div>`}
5240
5082
 
5241
5083
  <footer>
5242
5084
  Generated by <a href="https://www.npmjs.com/package/@yawlabs/mcp-compliance">@yawlabs/mcp-compliance</a> v${esc(report.toolVersion)}
@@ -5260,62 +5102,6 @@ function splitStdioTarget(s) {
5260
5102
  return { command: tokens[0], args: tokens.slice(1) };
5261
5103
  }
5262
5104
 
5263
- // src/token-store.ts
5264
- import { existsSync as existsSync3, mkdirSync, readFileSync as readFileSync3, writeFileSync } from "fs";
5265
- import { homedir } from "os";
5266
- import { dirname as dirname3, join as join3 } from "path";
5267
- var STORE_DIR = join3(homedir(), ".mcp-compliance");
5268
- var STORE_PATH = join3(STORE_DIR, "tokens.json");
5269
- function hashUrl(url) {
5270
- return urlHash(url);
5271
- }
5272
- function readStore() {
5273
- if (!existsSync3(STORE_PATH)) return {};
5274
- try {
5275
- const raw = readFileSync3(STORE_PATH, "utf8");
5276
- const parsed = JSON.parse(raw);
5277
- return parsed && typeof parsed === "object" ? parsed : {};
5278
- } catch {
5279
- return {};
5280
- }
5281
- }
5282
- function writeStore(store) {
5283
- const dir = dirname3(STORE_PATH);
5284
- try {
5285
- if (!existsSync3(dir)) mkdirSync(dir, { recursive: true, mode: 448 });
5286
- writeFileSync(STORE_PATH, JSON.stringify(store, null, 2), { mode: 384 });
5287
- } catch (err) {
5288
- const code = err?.code;
5289
- if (code === "EACCES" || code === "EPERM") {
5290
- throw new Error(
5291
- `Cannot write delete-token store at ${STORE_PATH}: permission denied. If you don't need to publish, re-run with --no-publish. Otherwise, ensure your home directory is writable.`
5292
- );
5293
- }
5294
- if (code === "ENOSPC") {
5295
- throw new Error(`Cannot write delete-token store at ${STORE_PATH}: no space left on device.`);
5296
- }
5297
- const message = err instanceof Error ? err.message : String(err);
5298
- throw new Error(`Cannot write delete-token store at ${STORE_PATH}: ${message}`);
5299
- }
5300
- }
5301
- function saveToken(hash, entry) {
5302
- const store = readStore();
5303
- store[hash] = entry;
5304
- writeStore(store);
5305
- }
5306
- function getTokenForUrl(url) {
5307
- const hash = hashUrl(url);
5308
- const store = readStore();
5309
- const entry = store[hash];
5310
- return entry ? { hash, entry } : null;
5311
- }
5312
- function deleteToken(hash) {
5313
- const store = readStore();
5314
- if (!(hash in store)) return;
5315
- delete store[hash];
5316
- writeStore(store);
5317
- }
5318
-
5319
5105
  // src/index.ts
5320
5106
  var require2 = createRequire(import.meta.url);
5321
5107
  var { version: version2 } = require2("../package.json");
@@ -5361,7 +5147,7 @@ function parseEnvVar(value, prev) {
5361
5147
  function readEnvFile(path) {
5362
5148
  let contents;
5363
5149
  try {
5364
- contents = readFileSync4(path, "utf8");
5150
+ contents = readFileSync3(path, "utf8");
5365
5151
  } catch (err) {
5366
5152
  const code = err?.code;
5367
5153
  if (code === "ENOENT") throw new Error(`--env-file: file not found: ${path}`);
@@ -5423,55 +5209,6 @@ function resolveTarget(cliTarget, cliExtraArgs, cliOpts, config) {
5423
5209
  if (config?.target) return config.target;
5424
5210
  throw new Error("No target specified. Pass a URL or command, or add 'target' to mcp-compliance.config.json.");
5425
5211
  }
5426
- var PRIVATE_TLD_SUFFIXES = [
5427
- ".local",
5428
- ".localhost",
5429
- ".internal",
5430
- ".corp",
5431
- ".home",
5432
- ".home.arpa",
5433
- ".lan",
5434
- ".intranet",
5435
- ".private",
5436
- ".test",
5437
- ".invalid",
5438
- ".example",
5439
- ".onion"
5440
- ];
5441
- function isPrivateHost(urlStr) {
5442
- let host;
5443
- try {
5444
- host = new URL(urlStr).hostname.toLowerCase();
5445
- } catch {
5446
- return false;
5447
- }
5448
- if (host === "localhost") return true;
5449
- for (const suffix of PRIVATE_TLD_SUFFIXES) {
5450
- if (host === suffix.slice(1) || host.endsWith(suffix)) return true;
5451
- }
5452
- const v4 = host.match(/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/);
5453
- if (v4) {
5454
- const [a, b] = v4.slice(1).map(Number);
5455
- if (a === 10 || a === 127 || a === 169 && b === 254) return true;
5456
- if (a === 172 && b >= 16 && b <= 31) return true;
5457
- if (a === 192 && b === 168) return true;
5458
- if (a === 0) return true;
5459
- }
5460
- const v6 = host.replace(/^\[|\]$/g, "");
5461
- if (v6 === "::1") return true;
5462
- if (v6.includes(":") && (v6.startsWith("fe80:") || v6.startsWith("fc") || v6.startsWith("fd"))) return true;
5463
- return false;
5464
- }
5465
- async function promptYesNo(message) {
5466
- if (!process.stdin.isTTY) return false;
5467
- const rl = createInterface({ input: process.stdin, output: process.stderr });
5468
- try {
5469
- const answer = (await rl.question(`${message} [y/N]: `)).trim().toLowerCase();
5470
- return answer === "y" || answer === "yes";
5471
- } finally {
5472
- rl.close();
5473
- }
5474
- }
5475
5212
  var program = new Command();
5476
5213
  program.name("mcp-compliance").description("Test MCP servers for spec compliance").version(version2);
5477
5214
  program.command("test").description("Run the full compliance test suite against an MCP server (URL or stdio command)").argument(
@@ -5610,7 +5347,7 @@ Testing ${describeTarget(transportTarget)}...
5610
5347
  }
5611
5348
  if (opts.output) {
5612
5349
  const svg = renderBadgeSvg({ grade: report2.grade, score: report2.score, timestamp: report2.timestamp });
5613
- writeFileSync2(opts.output, svg, "utf8");
5350
+ writeFileSync(opts.output, svg, "utf8");
5614
5351
  if (opts.format === "terminal") {
5615
5352
  console.log(chalk2.dim(`
5616
5353
  Badge SVG written to ${opts.output}`));
@@ -5693,137 +5430,6 @@ Error: ${message}
5693
5430
  }
5694
5431
  }
5695
5432
  );
5696
- program.command("badge").description("Run tests and publish a shareable compliance badge to mcp.hosting (HTTP targets only)").argument(
5697
- "[target]",
5698
- "Server URL, or command to spawn as a stdio server (optional when a config file defines 'target')"
5699
- ).argument("[extraArgs...]", "Additional args passed to the stdio command").option("--config <path>", "Load options from a config file").option(
5700
- "-H, --header <header>",
5701
- 'Add header to all requests (format: "Key: Value", repeatable; HTTP only)',
5702
- parseHeaderArg,
5703
- {}
5704
- ).option("--auth <token>", 'Shorthand for -H "Authorization: <token>" (HTTP only)').option("-E, --env <var>", 'Set env var for stdio command ("KEY=VALUE", repeatable)', parseEnvVar, {}).option("--env-file <path>", "Load env vars from file (stdio only)").option("--cwd <dir>", "Working directory for stdio command").option("--timeout <ms>", "Request timeout in milliseconds (default: 15000, or `timeout` in config)").option("--no-publish", "Do not publish the report to mcp.hosting").option("--output <file>", "Write a local SVG badge to the given path (works for any transport)").option("--no-color", "Disable colored output (also honors NO_COLOR env var)").action(
5705
- async (target, extraArgs, opts) => {
5706
- if (opts.color === false) chalk2.level = 0;
5707
- try {
5708
- const config = loadConfig(opts.config);
5709
- const transportTarget = resolveTarget(
5710
- target,
5711
- extraArgs,
5712
- {
5713
- header: opts.header,
5714
- auth: opts.auth,
5715
- env: opts.env,
5716
- envFile: opts.envFile,
5717
- cwd: opts.cwd
5718
- },
5719
- config
5720
- );
5721
- const shouldPublish = opts.publish && transportTarget.type === "http";
5722
- if (shouldPublish && transportTarget.type === "http" && isPrivateHost(transportTarget.url)) {
5723
- console.error(
5724
- chalk2.yellow(
5725
- `
5726
- Warning: ${transportTarget.url} looks like a private/internal address. Publishing will send the report (with the URL) to mcp.hosting.`
5727
- )
5728
- );
5729
- const ok = await promptYesNo(chalk2.yellow("Publish anyway?"));
5730
- if (!ok) {
5731
- console.error(chalk2.dim("\nAborted. Re-run with --no-publish to skip publishing.\n"));
5732
- process.exit(1);
5733
- }
5734
- }
5735
- console.log(chalk2.dim(`
5736
- Testing ${describeTarget(transportTarget)}...
5737
- `));
5738
- let badgeTimeout = config?.timeout ?? 15e3;
5739
- if (opts.timeout !== void 0) badgeTimeout = parsePositiveInt(opts.timeout, "--timeout", 1);
5740
- const report = await runComplianceSuite(transportTarget, {
5741
- timeout: badgeTimeout
5742
- });
5743
- let markdown = report.badge.markdown;
5744
- if (shouldPublish && transportTarget.type === "http") {
5745
- try {
5746
- const res = await publishReport(report);
5747
- saveToken(res.hash, {
5748
- deleteToken: res.deleteToken,
5749
- url: transportTarget.url,
5750
- publishedAt: (/* @__PURE__ */ new Date()).toISOString()
5751
- });
5752
- markdown = `[![MCP Compliant](${res.badgeUrl})](${res.reportUrl})`;
5753
- console.log(`Grade: ${report.grade} (${report.score}%)
5754
- `);
5755
- console.log(markdown);
5756
- console.log(chalk2.dim(`
5757
- Report published: ${res.reportUrl}`));
5758
- console.log(chalk2.dim(`Remove with: mcp-compliance unpublish ${transportTarget.url}
5759
- `));
5760
- } catch (err) {
5761
- const message = err instanceof Error ? err.message : String(err);
5762
- console.error(chalk2.yellow(`
5763
- Warning: publish failed \u2014 ${message}`));
5764
- console.error(chalk2.dim("Falling back to local badge markdown.\n"));
5765
- console.log(`Grade: ${report.grade} (${report.score}%)
5766
- `);
5767
- console.log(markdown);
5768
- console.log("");
5769
- }
5770
- } else {
5771
- console.log(`Grade: ${report.grade} (${report.score}%)
5772
- `);
5773
- if (transportTarget.type === "stdio") {
5774
- if (!opts.output) {
5775
- console.log(
5776
- chalk2.dim(
5777
- "Stdio servers cannot be published. Pass --output <file.svg> to write a local badge image for your README."
5778
- )
5779
- );
5780
- }
5781
- } else {
5782
- console.log(markdown);
5783
- }
5784
- console.log("");
5785
- }
5786
- if (opts.output) {
5787
- const svg = renderBadgeSvg({ grade: report.grade, score: report.score, timestamp: report.timestamp });
5788
- writeFileSync2(opts.output, svg, "utf8");
5789
- console.log(chalk2.dim(`Badge SVG written to ${opts.output}
5790
- `));
5791
- }
5792
- } catch (err) {
5793
- const message = err instanceof Error ? err.message : String(err);
5794
- console.error(chalk2.red(`
5795
- Error: ${message}
5796
- `));
5797
- process.exit(1);
5798
- }
5799
- }
5800
- );
5801
- program.command("unpublish").description("Remove a previously-published compliance report from mcp.hosting").argument("<url>", "MCP server URL whose report should be removed").action(async (url) => {
5802
- try {
5803
- const stored = getTokenForUrl(url);
5804
- if (!stored) {
5805
- console.log(chalk2.dim(`
5806
- No delete token found locally for ${url} \u2014 nothing to unpublish from this machine.`));
5807
- console.log(
5808
- chalk2.dim(
5809
- "(Delete tokens are stored locally at publish time. If you published from a different machine, run unpublish from there.)\n"
5810
- )
5811
- );
5812
- return;
5813
- }
5814
- await unpublishReport(stored.hash, stored.entry.deleteToken);
5815
- deleteToken(stored.hash);
5816
- console.log(chalk2.green(`
5817
- Removed report for ${url}.
5818
- `));
5819
- } catch (err) {
5820
- const message = err instanceof Error ? err.message : String(err);
5821
- console.error(chalk2.red(`
5822
- Error: ${message}
5823
- `));
5824
- process.exit(1);
5825
- }
5826
- });
5827
5433
  program.command("benchmark").description("Measure ping latency and throughput against an MCP server (URL or stdio command)").argument("[target]", "Server URL or stdio command").argument("[extraArgs...]", "Additional args for stdio command").option("-r, --requests <n>", "Number of ping requests to send", "100").option("-c, --concurrency <n>", "Concurrent in-flight requests", "1").option("--timeout <ms>", "Per-request timeout in milliseconds", "15000").option("--config <path>", "Load options from a config file").option("--format <format>", "terminal or json", "terminal").option("-H, --header <header>", "HTTP header (repeatable)", parseHeaderArg, {}).option("--auth <token>", 'Shorthand for -H "Authorization: <token>"').option("-E, --env <var>", "Env var for stdio (repeatable)", parseEnvVar, {}).option("--env-file <path>", "Load env vars from file").option("--cwd <dir>", "Working directory for stdio command").action(
5828
5434
  async (target, extraArgs, opts) => {
5829
5435
  try {
@@ -5856,8 +5462,8 @@ Error: ${message}
5856
5462
  );
5857
5463
  program.command("diff").description("Compare two compliance JSON reports; exit 1 if there are regressions").argument("<baseline>", "Baseline report JSON file").argument("<current>", "Current report JSON file").option("--format <format>", "terminal or json", "terminal").action((baselinePath, currentPath, opts) => {
5858
5464
  try {
5859
- const baseline = JSON.parse(readFileSync4(baselinePath, "utf8"));
5860
- const current = JSON.parse(readFileSync4(currentPath, "utf8"));
5465
+ const baseline = JSON.parse(readFileSync3(baselinePath, "utf8"));
5466
+ const current = JSON.parse(readFileSync3(currentPath, "utf8"));
5861
5467
  const summary = diffReports(baseline, current);
5862
5468
  if (opts.format === "json") {
5863
5469
  console.log(JSON.stringify(summary, null, 2));
@@ -5874,10 +5480,10 @@ Error: ${message}
5874
5480
  }
5875
5481
  });
5876
5482
  program.command("init").description("Scaffold a mcp-compliance.config.json in the current directory").option("--force", "Overwrite an existing config file without asking").action(async (opts) => {
5877
- const { existsSync: existsSync4, writeFileSync: write } = await import("fs");
5483
+ const { existsSync: existsSync3, writeFileSync: write } = await import("fs");
5878
5484
  const { join: joinPath } = await import("path");
5879
5485
  const out = joinPath(process.cwd(), "mcp-compliance.config.json");
5880
- if (existsSync4(out) && !opts.force) {
5486
+ if (existsSync3(out) && !opts.force) {
5881
5487
  console.error(chalk2.red(`
5882
5488
  A config already exists at ${out}.`));
5883
5489
  console.error(chalk2.dim("Re-run with --force to overwrite.\n"));
@@ -3,7 +3,7 @@ import {
3
3
  TEST_DEFINITIONS,
4
4
  readPackageVersion,
5
5
  runComplianceSuite
6
- } from "../chunk-A3UG3J63.js";
6
+ } from "../chunk-4RMQADAA.js";
7
7
 
8
8
  // src/mcp/server.ts
9
9
  import { realpathSync } from "fs";
@@ -81,56 +81,6 @@ ${JSON.stringify(report, null, 2)}` }
81
81
  }
82
82
  }
83
83
  );
84
- server.tool(
85
- "mcp_compliance_badge",
86
- "Get the badge markdown embed code for an MCP server. Runs the compliance test suite first to determine the grade.",
87
- {
88
- url: z.string().url().describe("The MCP server URL to test"),
89
- auth: z.string().optional().describe('Authorization header value (e.g., "Bearer tok123")'),
90
- headers: z.record(z.string(), z.string()).optional().describe("Additional headers to include on all requests"),
91
- timeout: z.number().int().min(1).max(3e5).optional().describe("Request timeout in milliseconds (default: 15000, max: 300000)")
92
- },
93
- {
94
- title: "Get Compliance Badge",
95
- readOnlyHint: true,
96
- destructiveHint: false,
97
- idempotentHint: true,
98
- openWorldHint: true
99
- },
100
- async ({ url, auth, headers: extraHeaders, timeout }) => {
101
- try {
102
- const headers = { ...extraHeaders };
103
- if (auth) headers.Authorization = auth;
104
- const report = await runComplianceSuite(url, {
105
- headers: Object.keys(headers).length > 0 ? headers : void 0,
106
- timeout
107
- });
108
- const badge = report.badge;
109
- return {
110
- content: [
111
- {
112
- type: "text",
113
- text: [
114
- `Grade: ${report.grade} (${report.score}%)`,
115
- "",
116
- "Markdown:",
117
- badge.markdown,
118
- "",
119
- "HTML:",
120
- badge.html
121
- ].join("\n")
122
- }
123
- ]
124
- };
125
- } catch (err) {
126
- const message = err instanceof Error ? err.message : String(err);
127
- return {
128
- content: [{ type: "text", text: `Error: ${message}` }],
129
- isError: true
130
- };
131
- }
132
- }
133
- );
134
84
  server.tool(
135
85
  "mcp_compliance_explain",
136
86
  "Explain what a specific compliance test ID checks and why it matters.",
package/dist/runner.d.ts CHANGED
@@ -46,6 +46,11 @@ interface ComplianceReport {
46
46
  resourceNames: string[];
47
47
  promptCount: number;
48
48
  promptNames: string[];
49
+ /**
50
+ * @deprecated The hosted badge renderer (mcp.hosting) is retired; these
51
+ * fields are always empty and will be removed in schema v2. For a local
52
+ * badge image, use the CLI's `--output <file>.svg`.
53
+ */
49
54
  badge: {
50
55
  imageUrl: string;
51
56
  reportUrl: string;
@@ -92,27 +97,6 @@ type TransportTarget = {
92
97
  /** All 88 test IDs with descriptions for the explain command */
93
98
  declare const TEST_DEFINITIONS: TestDefinition[];
94
99
 
95
- /**
96
- * Generate a short, deterministic hash of a URL for badge paths.
97
- * SHA-256 truncated to 24 hex chars (96 bits of entropy) — matches the
98
- * server-side hash width used by mcp.hosting for `/compliance/ext/<hash>`.
99
- *
100
- * Exported so mcp.hosting (and other consumers) can compute matching
101
- * hashes when looking up reports/badges by URL. The hash is the canonical
102
- * key for `/compliance/ext/<hash>` and `/api/compliance/ext/<hash>/badge`.
103
- */
104
- declare function urlHash(url: string): string;
105
- /**
106
- * Generate badge URLs and markdown for a compliance report.
107
- * Badge images are served by mcp.hosting.
108
- */
109
- declare function generateBadge(url: string): {
110
- imageUrl: string;
111
- reportUrl: string;
112
- markdown: string;
113
- html: string;
114
- };
115
-
116
100
  declare function computeGrade(score: number): Grade;
117
101
  declare function computeScore(tests: TestResult[]): {
118
102
  score: number;
@@ -226,4 +210,4 @@ interface RunOptions {
226
210
  */
227
211
  declare function runComplianceSuite(target: string | TransportTarget, options?: RunOptions): Promise<ComplianceReport>;
228
212
 
229
- export { type ComplianceReport, type PreviewOptions, type RunOptions, SPEC_BASE, SPEC_VERSION, TEST_DEFINITIONS, type TestResult, computeGrade, computeScore, dedupAndCapWarnings, generateBadge, parseSSEResponse, previewTests, runComplianceSuite, urlHash };
213
+ export { type ComplianceReport, type PreviewOptions, type RunOptions, SPEC_BASE, SPEC_VERSION, TEST_DEFINITIONS, type TestResult, computeGrade, computeScore, dedupAndCapWarnings, parseSSEResponse, previewTests, runComplianceSuite };
package/dist/runner.js CHANGED
@@ -5,12 +5,10 @@ import {
5
5
  computeGrade,
6
6
  computeScore,
7
7
  dedupAndCapWarnings,
8
- generateBadge,
9
8
  parseSSEResponse,
10
9
  previewTests,
11
- runComplianceSuite,
12
- urlHash
13
- } from "./chunk-A3UG3J63.js";
10
+ runComplianceSuite
11
+ } from "./chunk-4RMQADAA.js";
14
12
  export {
15
13
  SPEC_BASE,
16
14
  SPEC_VERSION,
@@ -18,9 +16,7 @@ export {
18
16
  computeGrade,
19
17
  computeScore,
20
18
  dedupAndCapWarnings,
21
- generateBadge,
22
19
  parseSSEResponse,
23
20
  previewTests,
24
- runComplianceSuite,
25
- urlHash
21
+ runComplianceSuite
26
22
  };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@yawlabs/mcp-compliance",
3
- "version": "0.15.0",
3
+ "version": "0.16.1",
4
4
  "mcpName": "io.github.YawLabs/mcp-compliance",
5
5
  "description": "CLI tool and MCP server that tests MCP servers for spec compliance",
6
6
  "license": "MIT",
@@ -79,5 +79,5 @@
79
79
  "bugs": {
80
80
  "url": "https://github.com/YawLabs/mcp-compliance/issues"
81
81
  },
82
- "homepage": "https://mcp.hosting"
82
+ "homepage": "https://github.com/YawLabs/mcp-compliance"
83
83
  }
@@ -1,8 +1,8 @@
1
1
  {
2
2
  "$schema": "https://json-schema.org/draft/2020-12/schema",
3
- "$id": "https://mcp.hosting/schemas/compliance/report.v1.json",
3
+ "$id": "https://yaw.sh/schemas/mcp-compliance/report.v1.json",
4
4
  "title": "MCP Compliance Report",
5
- "description": "Structured output of the mcp-compliance test suite. Stable, versioned contract — increment schemaVersion on breaking changes. Consumed by mcp.hosting and third-party dashboards.",
5
+ "description": "Structured output of the mcp-compliance test suite. Stable, versioned contract — increment schemaVersion on breaking changes. Consumed by Yaw MCP and third-party dashboards.",
6
6
  "type": "object",
7
7
  "required": [
8
8
  "schemaVersion",
@@ -120,8 +120,8 @@
120
120
  "type": "object",
121
121
  "required": ["imageUrl", "reportUrl", "markdown", "html"],
122
122
  "properties": {
123
- "imageUrl": { "type": "string", "format": "uri" },
124
- "reportUrl": { "type": "string", "format": "uri" },
123
+ "imageUrl": { "type": "string" },
124
+ "reportUrl": { "type": "string" },
125
125
  "markdown": { "type": "string" },
126
126
  "html": { "type": "string" }
127
127
  },