@yawlabs/mcp-compliance 0.15.1 → 0.16.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -222,37 +222,13 @@ Check in a `mcp-compliance.config.json` so CI and your dev loop can run `mcp-com
222
222
 
223
223
  Precedence: CLI flags > config file > defaults. Any field can be overridden on the command line.
224
224
 
225
- ### Publish a shareable badge (HTTP only)
225
+ ### Local SVG badge
226
226
 
227
- ```bash
228
- mcp-compliance badge https://my-server.com/mcp
229
- ```
230
-
231
- Runs the compliance suite, publishes the report to [mcp.hosting](https://mcp.hosting), and prints the markdown embed for your README. The badge image reflects the real grade (A–F) and links to the full report.
232
-
233
- | Option | Description |
234
- |--------|-------------|
235
- | `-H, --header <header>` | Add header to all requests, format `"Key: Value"` (repeatable) |
236
- | `--auth <token>` | Shorthand for `-H "Authorization: <token>"` |
237
- | `--timeout <ms>` | Request timeout in milliseconds (default: `15000`) |
238
- | `--no-publish` | Skip publishing; print a local badge markdown only |
239
- | `--output <file>` | Also write a local SVG badge to the given path |
240
-
241
- Reports are kept for 90 days from last submission; resubmitting the same URL overwrites the previous report. Auth headers are stripped client-side before upload. Private/loopback URLs (`localhost`, `127.0.0.1`, `192.168.*`, etc.) trigger an interactive confirmation before publishing, and are rejected by the server in any case.
242
-
243
- A delete token is returned at publish time and stored at `~/.mcp-compliance/tokens.json` (mode `0600`). Use it to take a report down:
244
-
245
- ```bash
246
- mcp-compliance unpublish https://my-server.com/mcp
247
- ```
248
-
249
- ### Local SVG badge (any transport)
250
-
251
- Stdio servers can't be published (no public URL to key on), but you can commit a local SVG reflecting the real grade:
227
+ Write a local SVG reflecting the real grade and commit it to your repo:
252
228
 
253
229
  ```bash
230
+ mcp-compliance test https://my-server.com/mcp --output badge.svg
254
231
  mcp-compliance test node ./dist/server.js --output badge.svg
255
- mcp-compliance badge npx -y @modelcontextprotocol/server-filesystem /tmp --output badge.svg
256
232
  ```
257
233
 
258
234
  Then embed it in your README:
@@ -261,8 +237,6 @@ Then embed it in your README:
261
237
  ![MCP Compliance](./badge.svg)
262
238
  ```
263
239
 
264
- The `test` command never publishes — use it for CI, debugging, and local iteration. `badge` is the only command that publishes to mcp.hosting.
265
-
266
240
  ## What the 88 tests check
267
241
 
268
242
  <details>
@@ -495,7 +469,6 @@ Restart your MCP client and approve the server when prompted.
495
469
  ### Tools
496
470
 
497
471
  - **mcp_compliance_test** — Run the full 88-test suite against a URL or stdio command. Supports auth, custom headers, env vars, timeout, retries, and category/test filtering. Returns grade, score, and detailed results.
498
- - **mcp_compliance_badge** — Get the badge markdown/HTML for a server. Supports auth and custom headers.
499
472
  - **mcp_compliance_explain** — Explain what a specific test ID checks and why it matters.
500
473
 
501
474
  All tools have [MCP tool annotations](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#annotations) (`readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`) so MCP clients can skip confirmation dialogs for safe operations.
@@ -560,11 +533,8 @@ The testing methodology is published openly so the grading is auditable:
560
533
  - **[Why `mcp-compliance`](./docs/WHY.md)** — the problem, existing alternatives, what this tool does differently
561
534
  - **[Fixing common failures](./docs/FIXES.md)** — recipes for the most frequent test failures with code snippets
562
535
  - **[Spec version migration policy](./docs/SPEC_VERSION_MIGRATION.md)** — how this tool evolves with MCP spec releases
563
- - **[mcp.hosting external API](./docs/EXT_API.md)** — public submit/retrieve/badge/delete endpoints used by `mcp-compliance badge` and any custom integrations
564
- - **[Enterprise tier (draft)](./docs/ENTERPRISE.md)** — paid tier structure for organizations with scheduled/private/audit-track compliance needs
565
536
  - **[Performance deep-dive](./docs/PERFORMANCE.md)** — why the suite is sequential and what parallel execution would cost
566
537
  - **[Spec PR drafts](./docs/spec-prs/)** — our proposed MCP spec clarifications for ambiguous cases we've hit
567
- - **[mcp.hosting integration spec](./docs/mcp-hosting-integration.md)** — the contract between this engine and the mcp.hosting platform: URL surfaces, data flow, storage model, badge API, leaderboard, router integration
568
538
 
569
539
  The methodology is not an authoritative conformance standard — it's one tool's choices, published so they can be inspected, adopted, or forked. The [official MCP specification](https://modelcontextprotocol.io/specification/2025-11-25) defines what servers must do; this document describes how `@yawlabs/mcp-compliance` verifies it.
570
540
 
@@ -596,7 +566,6 @@ npm test
596
566
 
597
567
  ## Links
598
568
 
599
- - [mcp.hosting](https://mcp.hosting) — Hosted MCP server infrastructure
600
569
  - [MCP Specification](https://modelcontextprotocol.io/specification/2025-11-25)
601
570
  - [Testing methodology](./COMPLIANCE_RUBRIC.md)
602
571
  - [Yaw Labs](https://yaw.sh)
@@ -1,27 +1,6 @@
1
1
  // src/runner.ts
2
2
  import { request as request2 } from "undici";
3
3
 
4
- // src/badge.ts
5
- import { createHash } from "crypto";
6
- function urlHash(url) {
7
- return createHash("sha256").update(url).digest("hex").slice(0, 24);
8
- }
9
- function generateBadge(url) {
10
- const hash = urlHash(url);
11
- const imageUrl = `https://mcp.hosting/api/compliance/ext/${hash}/badge`;
12
- const reportUrl = `https://mcp.hosting/compliance/ext/${hash}`;
13
- return {
14
- imageUrl,
15
- reportUrl,
16
- markdown: `[![MCP Compliant](${imageUrl})](${reportUrl})`,
17
- // loading="lazy" so READMEs that embed many badges don't block first
18
- // paint on this image. Markdown renderers (GitHub, npmjs.com) emit
19
- // their own <img> from the markdown form so the attribute only
20
- // matters for the HTML form people paste into custom pages.
21
- html: `<a href="${reportUrl}"><img src="${imageUrl}" alt="MCP Compliant" loading="lazy"></a>`
22
- };
23
- }
24
-
25
4
  // src/grader.ts
26
5
  function computeGrade(score) {
27
6
  if (score >= 90) return "A";
@@ -78,6 +57,8 @@ import { existsSync, readFileSync } from "fs";
78
57
  import { dirname, join } from "path";
79
58
  import { fileURLToPath } from "url";
80
59
  function readPackageVersion(metaUrl) {
60
+ if (typeof __VERSION__ === "string" && __VERSION__) return __VERSION__;
61
+ if (!metaUrl) return "0.0.0";
81
62
  let dir = dirname(fileURLToPath(metaUrl));
82
63
  for (; ; ) {
83
64
  const pkgPath = join(dir, "package.json");
@@ -4132,7 +4113,7 @@ async function runComplianceSuite(target, options = {}) {
4132
4113
  warnings.length = 0;
4133
4114
  warnings.push(...capped);
4134
4115
  const { score, grade, overall, summary, categories } = computeScore(tests);
4135
- const badge = displayUrl.startsWith("stdio:") ? { imageUrl: "", reportUrl: "", markdown: "", html: "" } : generateBadge(displayUrl);
4116
+ const badge = { imageUrl: "", reportUrl: "", markdown: "", html: "" };
4136
4117
  return {
4137
4118
  schemaVersion: REPORT_SCHEMA_VERSION,
4138
4119
  specVersion: SPEC_VERSION,
@@ -4162,8 +4143,6 @@ async function runComplianceSuite(target, options = {}) {
4162
4143
  }
4163
4144
 
4164
4145
  export {
4165
- urlHash,
4166
- generateBadge,
4167
4146
  computeGrade,
4168
4147
  computeScore,
4169
4148
  readPackageVersion,
package/dist/index.js CHANGED
@@ -1,7 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
 
3
3
  // src/index.ts
4
- import { watch as fsWatch, readFileSync as readFileSync4, writeFileSync as writeFileSync2 } from "fs";
4
+ import { watch as fsWatch, readFileSync as readFileSync3, writeFileSync } from "fs";
5
5
  import { createRequire } from "module";
6
6
  import { createInterface } from "readline/promises";
7
7
  import chalk2 from "chalk";
@@ -726,6 +726,8 @@ import { existsSync as existsSync2, readFileSync as readFileSync2 } from "fs";
726
726
  import { dirname, join as join2 } from "path";
727
727
  import { fileURLToPath } from "url";
728
728
  function readPackageVersion(metaUrl) {
729
+ if (typeof __VERSION__ === "string" && __VERSION__) return __VERSION__;
730
+ if (!metaUrl) return "0.0.0";
729
731
  let dir = dirname(fileURLToPath(metaUrl));
730
732
  for (; ; ) {
731
733
  const pkgPath = join2(dir, "package.json");
@@ -748,27 +750,6 @@ import { z } from "zod";
748
750
  // src/runner.ts
749
751
  import { request as request2 } from "undici";
750
752
 
751
- // src/badge.ts
752
- import { createHash } from "crypto";
753
- function urlHash(url) {
754
- return createHash("sha256").update(url).digest("hex").slice(0, 24);
755
- }
756
- function generateBadge(url) {
757
- const hash = urlHash(url);
758
- const imageUrl = `https://mcp.hosting/api/compliance/ext/${hash}/badge`;
759
- const reportUrl = `https://mcp.hosting/compliance/ext/${hash}`;
760
- return {
761
- imageUrl,
762
- reportUrl,
763
- markdown: `[![MCP Compliant](${imageUrl})](${reportUrl})`,
764
- // loading="lazy" so READMEs that embed many badges don't block first
765
- // paint on this image. Markdown renderers (GitHub, npmjs.com) emit
766
- // their own <img> from the markdown form so the attribute only
767
- // matters for the HTML form people paste into custom pages.
768
- html: `<a href="${reportUrl}"><img src="${imageUrl}" alt="MCP Compliant" loading="lazy"></a>`
769
- };
770
- }
771
-
772
753
  // src/grader.ts
773
754
  function computeGrade(score) {
774
755
  if (score >= 90) return "A";
@@ -4491,7 +4472,7 @@ async function runComplianceSuite(target, options = {}) {
4491
4472
  warnings.length = 0;
4492
4473
  warnings.push(...capped);
4493
4474
  const { score, grade, overall, summary, categories } = computeScore(tests);
4494
- const badge = displayUrl.startsWith("stdio:") ? { imageUrl: "", reportUrl: "", markdown: "", html: "" } : generateBadge(displayUrl);
4475
+ const badge = { imageUrl: "", reportUrl: "", markdown: "", html: "" };
4495
4476
  return {
4496
4477
  schemaVersion: REPORT_SCHEMA_VERSION,
4497
4478
  specVersion: SPEC_VERSION,
@@ -4588,56 +4569,6 @@ ${JSON.stringify(report, null, 2)}` }
4588
4569
  }
4589
4570
  }
4590
4571
  );
4591
- server.tool(
4592
- "mcp_compliance_badge",
4593
- "Get the badge markdown embed code for an MCP server. Runs the compliance test suite first to determine the grade.",
4594
- {
4595
- url: z.string().url().describe("The MCP server URL to test"),
4596
- auth: z.string().optional().describe('Authorization header value (e.g., "Bearer tok123")'),
4597
- headers: z.record(z.string(), z.string()).optional().describe("Additional headers to include on all requests"),
4598
- timeout: z.number().int().min(1).max(3e5).optional().describe("Request timeout in milliseconds (default: 15000, max: 300000)")
4599
- },
4600
- {
4601
- title: "Get Compliance Badge",
4602
- readOnlyHint: true,
4603
- destructiveHint: false,
4604
- idempotentHint: true,
4605
- openWorldHint: true
4606
- },
4607
- async ({ url, auth, headers: extraHeaders, timeout }) => {
4608
- try {
4609
- const headers = { ...extraHeaders };
4610
- if (auth) headers.Authorization = auth;
4611
- const report = await runComplianceSuite(url, {
4612
- headers: Object.keys(headers).length > 0 ? headers : void 0,
4613
- timeout
4614
- });
4615
- const badge = report.badge;
4616
- return {
4617
- content: [
4618
- {
4619
- type: "text",
4620
- text: [
4621
- `Grade: ${report.grade} (${report.score}%)`,
4622
- "",
4623
- "Markdown:",
4624
- badge.markdown,
4625
- "",
4626
- "HTML:",
4627
- badge.html
4628
- ].join("\n")
4629
- }
4630
- ]
4631
- };
4632
- } catch (err) {
4633
- const message = err instanceof Error ? err.message : String(err);
4634
- return {
4635
- content: [{ type: "text", text: `Error: ${message}` }],
4636
- isError: true
4637
- };
4638
- }
4639
- }
4640
- );
4641
4572
  server.tool(
4642
4573
  "mcp_compliance_explain",
4643
4574
  "Explain what a specific compliance test ID checks and why it matters.",
@@ -4721,75 +4652,6 @@ if (isInvokedDirectly()) {
4721
4652
  });
4722
4653
  }
4723
4654
 
4724
- // src/publish.ts
4725
- import { request as request3 } from "undici";
4726
- var PUBLISH_BASE = "https://mcp.hosting";
4727
- var PUBLISH_PATH = "/api/compliance/ext";
4728
- var PUBLISH_TIMEOUT_MS = 1e4;
4729
- var PUBLISH_RETRIES = 2;
4730
- function sanitizeReport(report) {
4731
- const { headers: _h, auth: _a, ...rest } = report;
4732
- void _h;
4733
- void _a;
4734
- return rest;
4735
- }
4736
- async function publishReport(report) {
4737
- const body = JSON.stringify(sanitizeReport(report));
4738
- let lastErr;
4739
- for (let attempt = 0; attempt <= PUBLISH_RETRIES; attempt++) {
4740
- try {
4741
- const res = await request3(`${PUBLISH_BASE}${PUBLISH_PATH}`, {
4742
- method: "POST",
4743
- headers: { "content-type": "application/json" },
4744
- body,
4745
- signal: AbortSignal.timeout(PUBLISH_TIMEOUT_MS)
4746
- });
4747
- if (res.statusCode >= 500 && attempt < PUBLISH_RETRIES) {
4748
- await res.body.dump();
4749
- await new Promise((r) => setTimeout(r, 500 * (attempt + 1)));
4750
- continue;
4751
- }
4752
- const text = await res.body.text();
4753
- if (res.statusCode >= 400) {
4754
- let message = `Publish failed: HTTP ${res.statusCode}`;
4755
- try {
4756
- const parsed2 = JSON.parse(text);
4757
- if (parsed2.error) message = `Publish failed: ${parsed2.error}`;
4758
- } catch {
4759
- }
4760
- throw new Error(message);
4761
- }
4762
- const parsed = JSON.parse(text);
4763
- if (!parsed.hash || !parsed.deleteToken || !parsed.reportUrl || !parsed.badgeUrl) {
4764
- throw new Error("Publish failed: malformed response from mcp.hosting");
4765
- }
4766
- return parsed;
4767
- } catch (err) {
4768
- lastErr = err;
4769
- if (attempt < PUBLISH_RETRIES) {
4770
- await new Promise((r) => setTimeout(r, 500 * (attempt + 1)));
4771
- }
4772
- }
4773
- }
4774
- throw lastErr instanceof Error ? lastErr : new Error(String(lastErr));
4775
- }
4776
- async function unpublishReport(hash, deleteToken2) {
4777
- const res = await request3(`${PUBLISH_BASE}${PUBLISH_PATH}/${hash}`, {
4778
- method: "DELETE",
4779
- headers: { authorization: `Bearer ${deleteToken2}` },
4780
- signal: AbortSignal.timeout(PUBLISH_TIMEOUT_MS)
4781
- });
4782
- const text = await res.body.text();
4783
- if (res.statusCode === 204 || res.statusCode === 404) return;
4784
- let message = `Unpublish failed: HTTP ${res.statusCode}`;
4785
- try {
4786
- const parsed = JSON.parse(text);
4787
- if (parsed.error) message = `Unpublish failed: ${parsed.error}`;
4788
- } catch {
4789
- }
4790
- throw new Error(message);
4791
- }
4792
-
4793
4655
  // src/reporter.ts
4794
4656
  import chalk from "chalk";
4795
4657
  var CATEGORY_LABELS = {
@@ -4957,13 +4819,7 @@ function formatTerminal(report) {
4957
4819
  }
4958
4820
  out.push("");
4959
4821
  }
4960
- if (report.url.startsWith("stdio:")) {
4961
- out.push(
4962
- chalk.dim(" Badge: stdio targets aren't published. Run with --output badge.svg for a local badge image.")
4963
- );
4964
- } else {
4965
- out.push(chalk.dim(` Badge: ${report.badge.markdown}`));
4966
- }
4822
+ out.push(chalk.dim(" Badge: run with --output badge.svg for a local badge image."));
4967
4823
  out.push("");
4968
4824
  return out.join("\n");
4969
4825
  }
@@ -5107,14 +4963,6 @@ function formatMarkdown(report) {
5107
4963
  for (const w of report.warnings) lines.push(`- ${w}`);
5108
4964
  lines.push("");
5109
4965
  }
5110
- if (!report.url.startsWith("stdio:")) {
5111
- lines.push("## Badge");
5112
- lines.push("");
5113
- lines.push("```markdown");
5114
- lines.push(report.badge.markdown);
5115
- lines.push("```");
5116
- lines.push("");
5117
- }
5118
4966
  return lines.join("\n");
5119
4967
  }
5120
4968
  function formatHtml(report) {
@@ -5133,7 +4981,6 @@ function formatHtml(report) {
5133
4981
  const grouped = /* @__PURE__ */ new Map();
5134
4982
  for (const cat of CATEGORY_ORDER) grouped.set(cat, []);
5135
4983
  for (const t of report.tests) grouped.get(t.category)?.push(t);
5136
- const isStdio = report.url.startsWith("stdio:");
5137
4984
  return `<!doctype html>
5138
4985
  <html lang="en">
5139
4986
  <head>
@@ -5232,11 +5079,8 @@ function formatHtml(report) {
5232
5079
  </tbody></table></div>`
5233
5080
  ).join("")}
5234
5081
 
5235
- ${!isStdio ? `<div class="card"><h2>Embed badge</h2>
5236
- <div class="badge-img"><img src="${esc(report.badge.imageUrl)}" alt="MCP Compliance"></div>
5237
- <p class="muted" style="margin-top:12px">Markdown:</p>
5238
- <code style="display:block; padding:8px; background:#0b0f17">${esc(report.badge.markdown)}</code></div>` : `<div class="card"><h2>Local badge</h2>
5239
- <p class="muted">Stdio servers can't be published to mcp.hosting (no public URL). Use <code>--output badge.svg</code> to write a local badge image.</p></div>`}
5082
+ ${`<div class="card"><h2>Local badge</h2>
5083
+ <p class="muted">Use <code>--output badge.svg</code> to write a local badge image.</p></div>`}
5240
5084
 
5241
5085
  <footer>
5242
5086
  Generated by <a href="https://www.npmjs.com/package/@yawlabs/mcp-compliance">@yawlabs/mcp-compliance</a> v${esc(report.toolVersion)}
@@ -5260,65 +5104,8 @@ function splitStdioTarget(s) {
5260
5104
  return { command: tokens[0], args: tokens.slice(1) };
5261
5105
  }
5262
5106
 
5263
- // src/token-store.ts
5264
- import { existsSync as existsSync3, mkdirSync, readFileSync as readFileSync3, writeFileSync } from "fs";
5265
- import { homedir } from "os";
5266
- import { dirname as dirname3, join as join3 } from "path";
5267
- var STORE_DIR = join3(homedir(), ".mcp-compliance");
5268
- var STORE_PATH = join3(STORE_DIR, "tokens.json");
5269
- function hashUrl(url) {
5270
- return urlHash(url);
5271
- }
5272
- function readStore() {
5273
- if (!existsSync3(STORE_PATH)) return {};
5274
- try {
5275
- const raw = readFileSync3(STORE_PATH, "utf8");
5276
- const parsed = JSON.parse(raw);
5277
- return parsed && typeof parsed === "object" ? parsed : {};
5278
- } catch {
5279
- return {};
5280
- }
5281
- }
5282
- function writeStore(store) {
5283
- const dir = dirname3(STORE_PATH);
5284
- try {
5285
- if (!existsSync3(dir)) mkdirSync(dir, { recursive: true, mode: 448 });
5286
- writeFileSync(STORE_PATH, JSON.stringify(store, null, 2), { mode: 384 });
5287
- } catch (err) {
5288
- const code = err?.code;
5289
- if (code === "EACCES" || code === "EPERM") {
5290
- throw new Error(
5291
- `Cannot write delete-token store at ${STORE_PATH}: permission denied. If you don't need to publish, re-run with --no-publish. Otherwise, ensure your home directory is writable.`
5292
- );
5293
- }
5294
- if (code === "ENOSPC") {
5295
- throw new Error(`Cannot write delete-token store at ${STORE_PATH}: no space left on device.`);
5296
- }
5297
- const message = err instanceof Error ? err.message : String(err);
5298
- throw new Error(`Cannot write delete-token store at ${STORE_PATH}: ${message}`);
5299
- }
5300
- }
5301
- function saveToken(hash, entry) {
5302
- const store = readStore();
5303
- store[hash] = entry;
5304
- writeStore(store);
5305
- }
5306
- function getTokenForUrl(url) {
5307
- const hash = hashUrl(url);
5308
- const store = readStore();
5309
- const entry = store[hash];
5310
- return entry ? { hash, entry } : null;
5311
- }
5312
- function deleteToken(hash) {
5313
- const store = readStore();
5314
- if (!(hash in store)) return;
5315
- delete store[hash];
5316
- writeStore(store);
5317
- }
5318
-
5319
5107
  // src/index.ts
5320
- var require2 = createRequire(import.meta.url);
5321
- var { version: version2 } = require2("../package.json");
5108
+ var version2 = typeof __VERSION__ === "string" && __VERSION__ ? __VERSION__ : createRequire(import.meta.url)("../package.json").version;
5322
5109
  function parseHeaderArg(value, prev) {
5323
5110
  const idx = value.indexOf(":");
5324
5111
  if (idx === -1) {
@@ -5361,7 +5148,7 @@ function parseEnvVar(value, prev) {
5361
5148
  function readEnvFile(path) {
5362
5149
  let contents;
5363
5150
  try {
5364
- contents = readFileSync4(path, "utf8");
5151
+ contents = readFileSync3(path, "utf8");
5365
5152
  } catch (err) {
5366
5153
  const code = err?.code;
5367
5154
  if (code === "ENOENT") throw new Error(`--env-file: file not found: ${path}`);
@@ -5423,55 +5210,6 @@ function resolveTarget(cliTarget, cliExtraArgs, cliOpts, config) {
5423
5210
  if (config?.target) return config.target;
5424
5211
  throw new Error("No target specified. Pass a URL or command, or add 'target' to mcp-compliance.config.json.");
5425
5212
  }
5426
- var PRIVATE_TLD_SUFFIXES = [
5427
- ".local",
5428
- ".localhost",
5429
- ".internal",
5430
- ".corp",
5431
- ".home",
5432
- ".home.arpa",
5433
- ".lan",
5434
- ".intranet",
5435
- ".private",
5436
- ".test",
5437
- ".invalid",
5438
- ".example",
5439
- ".onion"
5440
- ];
5441
- function isPrivateHost(urlStr) {
5442
- let host;
5443
- try {
5444
- host = new URL(urlStr).hostname.toLowerCase();
5445
- } catch {
5446
- return false;
5447
- }
5448
- if (host === "localhost") return true;
5449
- for (const suffix of PRIVATE_TLD_SUFFIXES) {
5450
- if (host === suffix.slice(1) || host.endsWith(suffix)) return true;
5451
- }
5452
- const v4 = host.match(/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/);
5453
- if (v4) {
5454
- const [a, b] = v4.slice(1).map(Number);
5455
- if (a === 10 || a === 127 || a === 169 && b === 254) return true;
5456
- if (a === 172 && b >= 16 && b <= 31) return true;
5457
- if (a === 192 && b === 168) return true;
5458
- if (a === 0) return true;
5459
- }
5460
- const v6 = host.replace(/^\[|\]$/g, "");
5461
- if (v6 === "::1") return true;
5462
- if (v6.includes(":") && (v6.startsWith("fe80:") || v6.startsWith("fc") || v6.startsWith("fd"))) return true;
5463
- return false;
5464
- }
5465
- async function promptYesNo(message) {
5466
- if (!process.stdin.isTTY) return false;
5467
- const rl = createInterface({ input: process.stdin, output: process.stderr });
5468
- try {
5469
- const answer = (await rl.question(`${message} [y/N]: `)).trim().toLowerCase();
5470
- return answer === "y" || answer === "yes";
5471
- } finally {
5472
- rl.close();
5473
- }
5474
- }
5475
5213
  var program = new Command();
5476
5214
  program.name("mcp-compliance").description("Test MCP servers for spec compliance").version(version2);
5477
5215
  program.command("test").description("Run the full compliance test suite against an MCP server (URL or stdio command)").argument(
@@ -5610,7 +5348,7 @@ Testing ${describeTarget(transportTarget)}...
5610
5348
  }
5611
5349
  if (opts.output) {
5612
5350
  const svg = renderBadgeSvg({ grade: report2.grade, score: report2.score, timestamp: report2.timestamp });
5613
- writeFileSync2(opts.output, svg, "utf8");
5351
+ writeFileSync(opts.output, svg, "utf8");
5614
5352
  if (opts.format === "terminal") {
5615
5353
  console.log(chalk2.dim(`
5616
5354
  Badge SVG written to ${opts.output}`));
@@ -5693,144 +5431,6 @@ Error: ${message}
5693
5431
  }
5694
5432
  }
5695
5433
  );
5696
- program.command("badge").description("Run tests and publish a shareable compliance badge to mcp.hosting (HTTP targets only)").argument(
5697
- "[target]",
5698
- "Server URL, or command to spawn as a stdio server (optional when a config file defines 'target')"
5699
- ).argument("[extraArgs...]", "Additional args passed to the stdio command").option("--config <path>", "Load options from a config file").option(
5700
- "-H, --header <header>",
5701
- 'Add header to all requests (format: "Key: Value", repeatable; HTTP only)',
5702
- parseHeaderArg,
5703
- {}
5704
- ).option("--auth <token>", 'Shorthand for -H "Authorization: <token>" (HTTP only)').option("-E, --env <var>", 'Set env var for stdio command ("KEY=VALUE", repeatable)', parseEnvVar, {}).option("--env-file <path>", "Load env vars from file (stdio only)").option("--cwd <dir>", "Working directory for stdio command").option("--timeout <ms>", "Request timeout in milliseconds (default: 15000, or `timeout` in config)").option("--no-publish", "Do not publish the report to mcp.hosting").option("--output <file>", "Write a local SVG badge to the given path (works for any transport)").option("--no-color", "Disable colored output (also honors NO_COLOR env var)").action(
5705
- async (target, extraArgs, opts) => {
5706
- if (opts.color === false) chalk2.level = 0;
5707
- try {
5708
- const config = loadConfig(opts.config);
5709
- const transportTarget = resolveTarget(
5710
- target,
5711
- extraArgs,
5712
- {
5713
- header: opts.header,
5714
- auth: opts.auth,
5715
- env: opts.env,
5716
- envFile: opts.envFile,
5717
- cwd: opts.cwd
5718
- },
5719
- config
5720
- );
5721
- const shouldPublish = opts.publish && transportTarget.type === "http";
5722
- if (shouldPublish && transportTarget.type === "http" && isPrivateHost(transportTarget.url)) {
5723
- console.error(
5724
- chalk2.yellow(
5725
- `
5726
- Warning: ${transportTarget.url} looks like a private/internal address. Publishing will send the report (with the URL) to mcp.hosting.`
5727
- )
5728
- );
5729
- const ok = await promptYesNo(chalk2.yellow("Publish anyway?"));
5730
- if (!ok) {
5731
- console.error(chalk2.dim("\nAborted. Re-run with --no-publish to skip publishing.\n"));
5732
- process.exit(1);
5733
- }
5734
- }
5735
- console.log(chalk2.dim(`
5736
- Testing ${describeTarget(transportTarget)}...
5737
- `));
5738
- let badgeTimeout = config?.timeout ?? 15e3;
5739
- if (opts.timeout !== void 0) badgeTimeout = parsePositiveInt(opts.timeout, "--timeout", 1);
5740
- const report = await runComplianceSuite(transportTarget, {
5741
- timeout: badgeTimeout
5742
- });
5743
- let markdown = report.badge.markdown;
5744
- if (shouldPublish && transportTarget.type === "http") {
5745
- try {
5746
- const res = await publishReport(report);
5747
- saveToken(res.hash, {
5748
- deleteToken: res.deleteToken,
5749
- url: transportTarget.url,
5750
- publishedAt: (/* @__PURE__ */ new Date()).toISOString()
5751
- });
5752
- markdown = `[![MCP Compliant](${res.badgeUrl})](${res.reportUrl})`;
5753
- console.log(`Grade: ${report.grade} (${report.score}%)
5754
- `);
5755
- console.log(markdown);
5756
- console.log(chalk2.dim(`
5757
- Report published: ${res.reportUrl}`));
5758
- console.log(chalk2.dim(`Remove with: mcp-compliance unpublish ${transportTarget.url}
5759
- `));
5760
- } catch (err) {
5761
- const message = err instanceof Error ? err.message : String(err);
5762
- console.error(chalk2.yellow(`
5763
- Warning: publish failed \u2014 ${message}`));
5764
- console.error(chalk2.dim("Falling back to local badge markdown.\n"));
5765
- console.log(`Grade: ${report.grade} (${report.score}%)
5766
- `);
5767
- console.log(markdown);
5768
- console.log("");
5769
- }
5770
- } else {
5771
- console.log(`Grade: ${report.grade} (${report.score}%)
5772
- `);
5773
- if (transportTarget.type === "stdio") {
5774
- if (!opts.output) {
5775
- console.log(
5776
- chalk2.dim(
5777
- "Stdio servers cannot be published. Pass --output <file.svg> to write a local badge image for your README."
5778
- )
5779
- );
5780
- }
5781
- } else {
5782
- console.log(markdown);
5783
- }
5784
- console.log("");
5785
- }
5786
- if (opts.output) {
5787
- const svg = renderBadgeSvg({ grade: report.grade, score: report.score, timestamp: report.timestamp });
5788
- writeFileSync2(opts.output, svg, "utf8");
5789
- console.log(chalk2.dim(`Badge SVG written to ${opts.output}
5790
- `));
5791
- }
5792
- } catch (err) {
5793
- const message = err instanceof Error ? err.message : String(err);
5794
- console.error(chalk2.red(`
5795
- Error: ${message}
5796
- `));
5797
- process.exit(1);
5798
- }
5799
- }
5800
- );
5801
- program.command("unpublish").description("Remove a previously-published compliance report from mcp.hosting").argument("<url>", "MCP server URL whose report should be removed").option("-y, --yes", "Skip the confirmation prompt (for automation)").action(async (url, opts) => {
5802
- try {
5803
- const stored = getTokenForUrl(url);
5804
- if (!stored) {
5805
- console.log(chalk2.dim(`
5806
- No delete token found locally for ${url} \u2014 nothing to unpublish from this machine.`));
5807
- console.log(
5808
- chalk2.dim(
5809
- "(Delete tokens are stored locally at publish time. If you published from a different machine, run unpublish from there.)\n"
5810
- )
5811
- );
5812
- return;
5813
- }
5814
- if (!opts.yes) {
5815
- const ok = await promptYesNo(chalk2.yellow(`Remove the published report for ${url}? This cannot be undone.`));
5816
- if (!ok) {
5817
- console.error(chalk2.dim("\nAborted. Re-run with --yes to skip this prompt.\n"));
5818
- return;
5819
- }
5820
- }
5821
- await unpublishReport(stored.hash, stored.entry.deleteToken);
5822
- deleteToken(stored.hash);
5823
- console.log(chalk2.green(`
5824
- Removed report for ${url}.
5825
- `));
5826
- } catch (err) {
5827
- const message = err instanceof Error ? err.message : String(err);
5828
- console.error(chalk2.red(`
5829
- Error: ${message}
5830
- `));
5831
- process.exit(1);
5832
- }
5833
- });
5834
5434
  program.command("benchmark").description("Measure ping latency and throughput against an MCP server (URL or stdio command)").argument("[target]", "Server URL or stdio command").argument("[extraArgs...]", "Additional args for stdio command").option("-r, --requests <n>", "Number of ping requests to send", "100").option("-c, --concurrency <n>", "Concurrent in-flight requests", "1").option("--timeout <ms>", "Per-request timeout in milliseconds", "15000").option("--config <path>", "Load options from a config file").option("--format <format>", "terminal or json", "terminal").option("-H, --header <header>", "HTTP header (repeatable)", parseHeaderArg, {}).option("--auth <token>", 'Shorthand for -H "Authorization: <token>"').option("-E, --env <var>", "Env var for stdio (repeatable)", parseEnvVar, {}).option("--env-file <path>", "Load env vars from file").option("--cwd <dir>", "Working directory for stdio command").action(
5835
5435
  async (target, extraArgs, opts) => {
5836
5436
  try {
@@ -5863,8 +5463,8 @@ Error: ${message}
5863
5463
  );
5864
5464
  program.command("diff").description("Compare two compliance JSON reports; exit 1 if there are regressions").argument("<baseline>", "Baseline report JSON file").argument("<current>", "Current report JSON file").option("--format <format>", "terminal or json", "terminal").action((baselinePath, currentPath, opts) => {
5865
5465
  try {
5866
- const baseline = JSON.parse(readFileSync4(baselinePath, "utf8"));
5867
- const current = JSON.parse(readFileSync4(currentPath, "utf8"));
5466
+ const baseline = JSON.parse(readFileSync3(baselinePath, "utf8"));
5467
+ const current = JSON.parse(readFileSync3(currentPath, "utf8"));
5868
5468
  const summary = diffReports(baseline, current);
5869
5469
  if (opts.format === "json") {
5870
5470
  console.log(JSON.stringify(summary, null, 2));
@@ -5881,10 +5481,10 @@ Error: ${message}
5881
5481
  }
5882
5482
  });
5883
5483
  program.command("init").description("Scaffold a mcp-compliance.config.json in the current directory").option("--force", "Overwrite an existing config file without asking").action(async (opts) => {
5884
- const { existsSync: existsSync4, writeFileSync: write } = await import("fs");
5484
+ const { existsSync: existsSync3, writeFileSync: write } = await import("fs");
5885
5485
  const { join: joinPath } = await import("path");
5886
5486
  const out = joinPath(process.cwd(), "mcp-compliance.config.json");
5887
- if (existsSync4(out) && !opts.force) {
5487
+ if (existsSync3(out) && !opts.force) {
5888
5488
  console.error(chalk2.red(`
5889
5489
  A config already exists at ${out}.`));
5890
5490
  console.error(chalk2.dim("Re-run with --force to overwrite.\n"));
@@ -3,7 +3,7 @@ import {
3
3
  TEST_DEFINITIONS,
4
4
  readPackageVersion,
5
5
  runComplianceSuite
6
- } from "../chunk-B5HSDK4K.js";
6
+ } from "../chunk-XBHHNN6R.js";
7
7
 
8
8
  // src/mcp/server.ts
9
9
  import { realpathSync } from "fs";
@@ -81,56 +81,6 @@ ${JSON.stringify(report, null, 2)}` }
81
81
  }
82
82
  }
83
83
  );
84
- server.tool(
85
- "mcp_compliance_badge",
86
- "Get the badge markdown embed code for an MCP server. Runs the compliance test suite first to determine the grade.",
87
- {
88
- url: z.string().url().describe("The MCP server URL to test"),
89
- auth: z.string().optional().describe('Authorization header value (e.g., "Bearer tok123")'),
90
- headers: z.record(z.string(), z.string()).optional().describe("Additional headers to include on all requests"),
91
- timeout: z.number().int().min(1).max(3e5).optional().describe("Request timeout in milliseconds (default: 15000, max: 300000)")
92
- },
93
- {
94
- title: "Get Compliance Badge",
95
- readOnlyHint: true,
96
- destructiveHint: false,
97
- idempotentHint: true,
98
- openWorldHint: true
99
- },
100
- async ({ url, auth, headers: extraHeaders, timeout }) => {
101
- try {
102
- const headers = { ...extraHeaders };
103
- if (auth) headers.Authorization = auth;
104
- const report = await runComplianceSuite(url, {
105
- headers: Object.keys(headers).length > 0 ? headers : void 0,
106
- timeout
107
- });
108
- const badge = report.badge;
109
- return {
110
- content: [
111
- {
112
- type: "text",
113
- text: [
114
- `Grade: ${report.grade} (${report.score}%)`,
115
- "",
116
- "Markdown:",
117
- badge.markdown,
118
- "",
119
- "HTML:",
120
- badge.html
121
- ].join("\n")
122
- }
123
- ]
124
- };
125
- } catch (err) {
126
- const message = err instanceof Error ? err.message : String(err);
127
- return {
128
- content: [{ type: "text", text: `Error: ${message}` }],
129
- isError: true
130
- };
131
- }
132
- }
133
- );
134
84
  server.tool(
135
85
  "mcp_compliance_explain",
136
86
  "Explain what a specific compliance test ID checks and why it matters.",
package/dist/runner.d.ts CHANGED
@@ -46,6 +46,11 @@ interface ComplianceReport {
46
46
  resourceNames: string[];
47
47
  promptCount: number;
48
48
  promptNames: string[];
49
+ /**
50
+ * @deprecated The hosted badge renderer (mcp.hosting) is retired; these
51
+ * fields are always empty and will be removed in schema v2. For a local
52
+ * badge image, use the CLI's `--output <file>.svg`.
53
+ */
49
54
  badge: {
50
55
  imageUrl: string;
51
56
  reportUrl: string;
@@ -92,27 +97,6 @@ type TransportTarget = {
92
97
  /** All 88 test IDs with descriptions for the explain command */
93
98
  declare const TEST_DEFINITIONS: TestDefinition[];
94
99
 
95
- /**
96
- * Generate a short, deterministic hash of a URL for badge paths.
97
- * SHA-256 truncated to 24 hex chars (96 bits of entropy) — matches the
98
- * server-side hash width used by mcp.hosting for `/compliance/ext/<hash>`.
99
- *
100
- * Exported so mcp.hosting (and other consumers) can compute matching
101
- * hashes when looking up reports/badges by URL. The hash is the canonical
102
- * key for `/compliance/ext/<hash>` and `/api/compliance/ext/<hash>/badge`.
103
- */
104
- declare function urlHash(url: string): string;
105
- /**
106
- * Generate badge URLs and markdown for a compliance report.
107
- * Badge images are served by mcp.hosting.
108
- */
109
- declare function generateBadge(url: string): {
110
- imageUrl: string;
111
- reportUrl: string;
112
- markdown: string;
113
- html: string;
114
- };
115
-
116
100
  declare function computeGrade(score: number): Grade;
117
101
  declare function computeScore(tests: TestResult[]): {
118
102
  score: number;
@@ -226,4 +210,4 @@ interface RunOptions {
226
210
  */
227
211
  declare function runComplianceSuite(target: string | TransportTarget, options?: RunOptions): Promise<ComplianceReport>;
228
212
 
229
- export { type ComplianceReport, type PreviewOptions, type RunOptions, SPEC_BASE, SPEC_VERSION, TEST_DEFINITIONS, type TestResult, computeGrade, computeScore, dedupAndCapWarnings, generateBadge, parseSSEResponse, previewTests, runComplianceSuite, urlHash };
213
+ export { type ComplianceReport, type PreviewOptions, type RunOptions, SPEC_BASE, SPEC_VERSION, TEST_DEFINITIONS, type TestResult, computeGrade, computeScore, dedupAndCapWarnings, parseSSEResponse, previewTests, runComplianceSuite };
package/dist/runner.js CHANGED
@@ -5,12 +5,10 @@ import {
5
5
  computeGrade,
6
6
  computeScore,
7
7
  dedupAndCapWarnings,
8
- generateBadge,
9
8
  parseSSEResponse,
10
9
  previewTests,
11
- runComplianceSuite,
12
- urlHash
13
- } from "./chunk-B5HSDK4K.js";
10
+ runComplianceSuite
11
+ } from "./chunk-XBHHNN6R.js";
14
12
  export {
15
13
  SPEC_BASE,
16
14
  SPEC_VERSION,
@@ -18,9 +16,7 @@ export {
18
16
  computeGrade,
19
17
  computeScore,
20
18
  dedupAndCapWarnings,
21
- generateBadge,
22
19
  parseSSEResponse,
23
20
  previewTests,
24
- runComplianceSuite,
25
- urlHash
21
+ runComplianceSuite
26
22
  };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@yawlabs/mcp-compliance",
3
- "version": "0.15.1",
3
+ "version": "0.16.3",
4
4
  "mcpName": "io.github.YawLabs/mcp-compliance",
5
5
  "description": "CLI tool and MCP server that tests MCP servers for spec compliance",
6
6
  "license": "MIT",
@@ -40,11 +40,14 @@
40
40
  "prepublishOnly": "npm run build",
41
41
  "start": "node dist/index.js"
42
42
  },
43
+ "overrides": {
44
+ "esbuild": "^0.28.1"
45
+ },
43
46
  "dependencies": {
44
47
  "@modelcontextprotocol/sdk": "^1.29.0",
45
48
  "chalk": "^5.4.1",
46
49
  "commander": "^14.0.3",
47
- "undici": "^7.8.0",
50
+ "undici": "^7.28.0",
48
51
  "zod": "^4.4.3"
49
52
  },
50
53
  "devDependencies": {
@@ -52,8 +55,10 @@
52
55
  "@types/node": "^25.5.2",
53
56
  "ajv": "^8.18.0",
54
57
  "ajv-formats": "^3.0.1",
55
- "tsup": "^8.4.0",
56
- "tsx": "^4.21.0",
58
+ "esbuild": "^0.28.1",
59
+ "postject": "^1.0.0-alpha.6",
60
+ "tsup": "^8.5.1",
61
+ "tsx": "^4.22.4",
57
62
  "typescript": "^6.0.3",
58
63
  "vitest": "^4.1.8"
59
64
  },
@@ -79,5 +84,5 @@
79
84
  "bugs": {
80
85
  "url": "https://github.com/YawLabs/mcp-compliance/issues"
81
86
  },
82
- "homepage": "https://mcp.hosting"
87
+ "homepage": "https://github.com/YawLabs/mcp-compliance"
83
88
  }
@@ -1,8 +1,8 @@
1
1
  {
2
2
  "$schema": "https://json-schema.org/draft/2020-12/schema",
3
- "$id": "https://mcp.hosting/schemas/compliance/report.v1.json",
3
+ "$id": "https://yaw.sh/schemas/mcp-compliance/report.v1.json",
4
4
  "title": "MCP Compliance Report",
5
- "description": "Structured output of the mcp-compliance test suite. Stable, versioned contract — increment schemaVersion on breaking changes. Consumed by mcp.hosting and third-party dashboards.",
5
+ "description": "Structured output of the mcp-compliance test suite. Stable, versioned contract — increment schemaVersion on breaking changes. Consumed by Yaw MCP and third-party dashboards.",
6
6
  "type": "object",
7
7
  "required": [
8
8
  "schemaVersion",
@@ -120,8 +120,8 @@
120
120
  "type": "object",
121
121
  "required": ["imageUrl", "reportUrl", "markdown", "html"],
122
122
  "properties": {
123
- "imageUrl": { "type": "string", "format": "uri" },
124
- "reportUrl": { "type": "string", "format": "uri" },
123
+ "imageUrl": { "type": "string" },
124
+ "reportUrl": { "type": "string" },
125
125
  "markdown": { "type": "string" },
126
126
  "html": { "type": "string" }
127
127
  },