mcp-scorecard 0.1.1 → 0.1.3

package/AGENTS.md

@@ -0,0 +1,32 @@
+# Agent Development Notes
+
+## Scope
+
+This repo is an agent-readiness scorecard for MCP servers. It launches a target over stdio, inspects the MCP contract, and emits a scored report without persisting private tool payloads.
+
+## Commands
+
+- Install: `npm ci`
+- Typecheck: `npm run typecheck`
+- Build: `npm run build`
+- Self smoke: `npm run test:self`
+- Full gate: `npm test`
+- Package preview: `npm pack --dry-run`
+
+## Rules
+
+- Do not persist raw MCP responses from probed servers. Keep reports limited to scores, counts, labels, and redacted findings.
+- Preserve `MCP_PROBE=1` behavior so auth-heavy MCPs can expose contracts without live credentials.
+- New checks must be deterministic and explain the exact remediation in `fixes`.
+- Keep the score stable enough for CI gates; avoid changing weights casually.
+- Any new network behavior must be explicit in docs and disabled for local-path targets unless required.
+
+## Agent-readiness checklist
+
+Before publishing a new version:
+
+1. `npm test`
+2. `npm pack --dry-run`
+3. Run the built CLI against at least one known local-first MCP package
+4. Confirm markdown and JSON outputs both include actionable fixes
+5. Update `CHANGELOG.md` for score or output-shape changes

package/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## 0.1.3 — 2026-05-29
+
+### Docs
+
+- **README now shows a real captured run.** Added a `## Demo` section with the
+  verbatim output of `npx mcp-scorecard @modelcontextprotocol/server-everything`
+  — the official MCP reference server scores 44/100, demonstrating the
+  conventions checks against a real, well-known public server (no hand-edited
+  output).
+
+### Fixed
+
+- **`SERVER_VERSION` constant synced to the published version.** It had drifted
+  to `0.1.1` while package.json shipped `0.1.2`, so the report footer
+  (`_Generated by mcp-scorecard vX_`) printed a stale version. All four sources
+  (package.json, server.json ×2, `src/constants.ts`) are now `0.1.3`.
+
 ## 0.1.1 — 2026-05-23
 
 ### Fixed

package/README.md

@@ -34,6 +34,50 @@ npx -y mcp-scorecard my-mcp --min-score 80
 npx -y mcp-scorecard my-mcp --json
 ```
 
+## Demo
+
+Real captured run auditing the official MCP reference server
+[`@modelcontextprotocol/server-everything`](https://www.npmjs.com/package/@modelcontextprotocol/server-everything)
+— nothing here is hand-edited, this is exactly what the CLI printed:
+
+```console
+$ npx -y mcp-scorecard @modelcontextprotocol/server-everything
+
+# mcp-scorecard - @modelcontextprotocol/server-everything @2026.1.26
+
+**Agent-readiness score:** 44/100
+
+- [PASS] Schema validity              (13/13 tools have valid input schema)
+- [FAIL] Tool naming convention       (12/13 tools violate snake_case)
+- [FAIL] Privacy modes documented     (only 1 tool(s) mention privacy modes)
+- [PASS] Mutation gating              (no write tools — n/a)
+- [FAIL] Agent manifest               (no agent_manifest tool)
+- [FAIL] Smoke test                   (no smoke script and no test script)
+- [PASS] Resources advertised         (7 resources registered)
+- [PASS] Tool descriptions            (avg 88 chars across 13 tools)
+- [FAIL] Annotations                  (0/13 read tools annotated)
+- [FAIL] Manifest discoverability     (no discovery tools)
+
+## Details
+### Tool naming convention
+- Non-snake_case names: get-annotated-message, get-env, get-resource-links, get-resource-reference, get-structured-content, get-sum, get-tiny-image, gzip-file-as-resource, toggle-simulated-logging, toggle-subscriber-updates
+### Annotations
+- Missing readOnlyHint: echo, get-annotated-message, get-env, get-resource-links, get-resource-reference, get-structured-content, get-sum, get-tiny-image, gzip-file-as-resource, toggle-simulated-logging
+
+## Suggested fixes
+- Rename tools to lowercase snake_case (a-z, 0-9, _).
+- Add a `privacy_mode` parameter (summary | structured | raw) on read tools so agents can request only what they need.
+- Expose a `<prefix>_agent_manifest` tool that returns { recommended_first_calls, standard_tools, ... } so agents can self-onboard.
+- Add `scripts/smoke-tools.mjs` that boots the server via StdioClientTransport and asserts the tool list.
+- Add `annotations: { readOnlyHint: true, openWorldHint: false }` to every read tool definition.
+- Expose discovery tools so agents can self-onboard: `*_agent_manifest`, `*_data_inventory`, `*_capabilities`, `*_connection_status`.
+```
+
+The reference server is a feature showcase, not a production integration — a 44 is
+expected and is exactly why the conventions checks exist. Agent-oriented servers
+that adopt snake_case naming, a manifest tool, and read-only annotations land in
+the 80s and 90s.
+
 ## What it checks
 
 Ten quality dimensions, each scored 0-10. Final score is the sum, capped at 100.

package/dist/constants.d.ts

@@ -3,7 +3,7 @@
  * same string. Bumping this without updating package.json is intentional during
  * dev (keep src as source of truth); a release script can sync them.
  */
-export declare const SERVER_VERSION = "0.1.1";
+export declare const SERVER_VERSION = "0.1.3";
 /** Identifier used when the probe connects to a target MCP server. */
 export declare const PROBE_CLIENT_NAME = "mcp-scorecard";
 /** Env var set on the spawned target so MCP authors can detect we are probing

package/dist/constants.js

@@ -3,7 +3,7 @@
  * same string. Bumping this without updating package.json is intentional during
  * dev (keep src as source of truth); a release script can sync them.
  */
-export const SERVER_VERSION = '0.1.1';
+export const SERVER_VERSION = '0.1.3';
 /** Identifier used when the probe connects to a target MCP server. */
 export const PROBE_CLIENT_NAME = 'mcp-scorecard';
 /** Env var set on the spawned target so MCP authors can detect we are probing

package/llms.txt

@@ -0,0 +1,13 @@
+# mcp-scorecard
+Agent-readiness scorecard for MCP servers. Probes a target over stdio, runs 10 quality checks, and returns a 0-100 score with actionable fixes.
+
+Repository: https://github.com/davidmosiah/mcp-scorecard
+NPM: https://www.npmjs.com/package/mcp-scorecard
+Primary command: npm exec --yes --package=mcp-scorecard -- mcp-scorecard <package-or-repo>
+MCP command: none; this is a CLI auditor for MCP servers.
+
+- Use `mcp-scorecard <target> --json` when another agent or CI job needs structured output.
+- Use `mcp-scorecard <target> --min-score 80` as a release gate.
+- The probe launches targets with `MCP_PROBE=1`; MCP authors can use that env var to expose manifests without requiring live credentials.
+- The report is not a security audit. It checks contract shape, privacy affordances, mutation gating, resources, descriptions, annotations, and manifest discoverability.
+- Raw probed payloads should not be persisted. Reports should contain counts, labels, and redacted findings only.

package/package.json

@@ -1,6 +1,7 @@
 {
   "name": "mcp-scorecard",
-  "version": "0.1.1",
+  "version": "0.1.3",
+  "mcpName": "io.github.davidmosiah/mcp-scorecard",
   "description": "Agent-readiness scorecard for any MCP server. Probes a target, runs 10 checks, outputs a 0-100 score with actionable findings.",
   "type": "module",
   "bin": {
@@ -10,7 +11,9 @@
     "dist",
     "README.md",
     "LICENSE",
-    "CHANGELOG.md"
+    "CHANGELOG.md",
+    "AGENTS.md",
+    "llms.txt"
   ],
   "scripts": {
     "build": "tsc -p tsconfig.json",