mcpprobe-scanner 0.1.0__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (86) hide show
  1. mcpprobe_scanner-0.3.0/CHANGELOG.md +219 -0
  2. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/CHECKS.md +162 -26
  3. mcpprobe_scanner-0.3.0/LICENSE +201 -0
  4. mcpprobe_scanner-0.3.0/NOTICE +8 -0
  5. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/PKG-INFO +66 -12
  6. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/README.md +59 -10
  7. mcpprobe_scanner-0.3.0/docs/benchmark.md +43 -0
  8. mcpprobe_scanner-0.3.0/docs/github-action.md +126 -0
  9. mcpprobe_scanner-0.3.0/docs/judges.md +62 -0
  10. mcpprobe_scanner-0.3.0/docs/marketplace-listing.md +80 -0
  11. mcpprobe_scanner-0.3.0/docs/mcp-registry.md +179 -0
  12. mcpprobe_scanner-0.3.0/docs/report-schema.md +110 -0
  13. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/__init__.py +1 -1
  14. mcpprobe_scanner-0.3.0/mcpprobe/action.py +176 -0
  15. mcpprobe_scanner-0.3.0/mcpprobe/chains.py +103 -0
  16. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/checks/__init__.py +19 -0
  17. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/checks/auth.py +301 -48
  18. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/checks/base.py +66 -1
  19. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/checks/baseline.py +33 -2
  20. mcpprobe_scanner-0.3.0/mcpprobe/checks/capability.py +355 -0
  21. mcpprobe_scanner-0.3.0/mcpprobe/checks/metadata.py +527 -0
  22. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/checks/poisoning.py +181 -36
  23. mcpprobe_scanner-0.3.0/mcpprobe/checks/proof.py +163 -0
  24. mcpprobe_scanner-0.3.0/mcpprobe/checks/protocol_2026.py +749 -0
  25. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/checks/robustness.py +124 -8
  26. mcpprobe_scanner-0.3.0/mcpprobe/checks/tool_surface.py +339 -0
  27. mcpprobe_scanner-0.3.0/mcpprobe/checks/transport_conformance.py +647 -0
  28. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/cli.py +50 -0
  29. mcpprobe_scanner-0.3.0/mcpprobe/environment.py +51 -0
  30. mcpprobe_scanner-0.3.0/mcpprobe/judges/__init__.py +31 -0
  31. mcpprobe_scanner-0.3.0/mcpprobe/judges/hf.py +125 -0
  32. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/models.py +120 -3
  33. mcpprobe_scanner-0.3.0/mcpprobe/readiness.py +341 -0
  34. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/report/json_report.py +7 -0
  35. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/report/sarif.py +21 -3
  36. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/report/terminal.py +52 -3
  37. mcpprobe_scanner-0.3.0/mcpprobe/scanner.py +380 -0
  38. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/transport/base.py +25 -1
  39. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/transport/evidence.py +21 -45
  40. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/transport/fingerprint.py +7 -8
  41. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/transport/http.py +112 -2
  42. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/pyproject.toml +12 -3
  43. mcpprobe_scanner-0.3.0/scripts/benchmark_judge.py +222 -0
  44. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/conftest.py +34 -2
  45. mcpprobe_scanner-0.3.0/tests/data/meta01_corpus.json +51 -0
  46. mcpprobe_scanner-0.3.0/tests/data/meta01_heldout.json +67 -0
  47. mcpprobe_scanner-0.3.0/tests/fake_server.py +1087 -0
  48. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/fake_stdio_server.py +11 -3
  49. mcpprobe_scanner-0.3.0/tests/test_checks_03.py +369 -0
  50. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/test_checks_auth.py +40 -1
  51. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/test_checks_meta.py +44 -18
  52. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/test_checks_transport.py +25 -3
  53. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/test_docs_and_action.py +33 -3
  54. mcpprobe_scanner-0.3.0/tests/test_environment_and_evidence.py +103 -0
  55. mcpprobe_scanner-0.3.0/tests/test_fixture_outcomes.py +474 -0
  56. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/test_gate_and_policy.py +2 -2
  57. mcpprobe_scanner-0.3.0/tests/test_judges.py +151 -0
  58. mcpprobe_scanner-0.3.0/tests/test_meta01_corpus.py +46 -0
  59. mcpprobe_scanner-0.1.0/CHANGELOG.md +0 -44
  60. mcpprobe_scanner-0.1.0/LICENSE +0 -21
  61. mcpprobe_scanner-0.1.0/mcpprobe/action.py +0 -69
  62. mcpprobe_scanner-0.1.0/mcpprobe/checks/capability.py +0 -171
  63. mcpprobe_scanner-0.1.0/mcpprobe/checks/metadata.py +0 -341
  64. mcpprobe_scanner-0.1.0/mcpprobe/checks/transport_conformance.py +0 -429
  65. mcpprobe_scanner-0.1.0/mcpprobe/scanner.py +0 -211
  66. mcpprobe_scanner-0.1.0/tests/fake_server.py +0 -559
  67. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/.gitignore +0 -0
  68. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/SECURITY.md +0 -0
  69. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/authz.py +0 -0
  70. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/catalogue.py +0 -0
  71. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/checks/_util.py +0 -0
  72. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/checks/active_stubs.py +0 -0
  73. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/dnsutil.py +0 -0
  74. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/governance.py +0 -0
  75. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/policy.py +0 -0
  76. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/py.typed +0 -0
  77. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/redact.py +0 -0
  78. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/report/__init__.py +0 -0
  79. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/transport/__init__.py +0 -0
  80. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/transport/ratelimit.py +0 -0
  81. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/mcpprobe/transport/stdio.py +0 -0
  82. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/__init__.py +0 -0
  83. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/data/sarif-schema-2.1.0.json +0 -0
  84. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/test_checks_robustness.py +0 -0
  85. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/test_reports_and_cli.py +0 -0
  86. {mcpprobe_scanner-0.1.0 → mcpprobe_scanner-0.3.0}/tests/test_transport.py +0 -0
@@ -0,0 +1,219 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented here. The format follows
4
+ [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
5
+ [Semantic Versioning](https://semver.org/).
6
+
7
+ ## [Unreleased]
8
+
9
+ ## [0.3.0]
10
+
11
+ Directory readiness, the 2026-07-28 protocol rules, tool annotations and toxic flows.
12
+ Report schema 0.3 (additions only). 24 passive checks, up from 16.
13
+
14
+ ### Added
15
+
16
+ - **META-06 Tool annotation lint.** Missing `readOnlyHint`, `openWorldHint` and title on
17
+ every tool, `destructiveHint`/`idempotentHint` on every tool not marked read-only (the
18
+ spec says they only mean something then), non-boolean hints, and annotations that
19
+ contradict the tool's apparent capability (read-only or non-destructive on something
20
+ that executes, writes, deletes, sends or pays; closed-world on a network tool). Medium
21
+ when a dangerous-class tool is mislabelled, low when annotations are only missing;
22
+ one finding lists every tool. Fix text cites Anthropic's directory policy (5E) and
23
+ OpenAI's app review.
24
+ - **META-07 Toxic flow (lethal trifecta).** Warns when one server offers access to
25
+ private data, intake of untrusted content and a way to send data out, listing the
26
+ tools in each role. Medium; high when the endpoint needs no authentication. A tool
27
+ advertising arbitrary command execution counts for all three roles.
28
+ - **2026-07-28 checks**, each verified against the specification text on
29
+ modelcontextprotocol.io and not applicable (with the negotiated revision as the
30
+ reason) on servers that speak an earlier revision:
31
+ - TRANS-06 `server/discover` result: `supportedVersions` including 2026-07-28,
32
+ `capabilities`, `resultType: "complete"`; missing `serverInfo` is informational.
33
+ - TRANS-07 a protocol version the server does not list in `supportedVersions` is
34
+ served instead of rejected with HTTP 400 / `UnsupportedProtocolVersionError`
35
+ (-32022): downgrade acceptance. One request.
36
+ - TRANS-08 `Mcp-Method` header that differs from the body, or is missing, is not
37
+ rejected with HTTP 400 / `HeaderMismatch` (-32020). Two requests. `Mcp-Name` is
38
+ not tested passively (it exists only for `tools/call`, `resources/read`,
39
+ `prompts/get`).
40
+ - TRANS-09 `ttlMs` (number >= 0) and `cacheScope` (`public`/`private`) missing or
41
+ invalid on `server/discover` and list results. Sends nothing.
42
+ - AUTH-05 authorization server metadata does not advertise
43
+ `authorization_response_iss_parameter_supported: true` (RFC 9207). Sends nothing.
44
+ - AUTH-06 deprecated Dynamic Client Registration offered without Client ID Metadata
45
+ Documents (low), or alongside them (informational). Sends nothing.
46
+ - "Deprecated HTTP+SSE still reachable" stays TRANS-04 (no new ID).
47
+ - **Directory readiness** (`readiness` in the JSON report): Anthropic 5D/5E/2G/3F,
48
+ OpenAI annotations, HubSpot no-critical/high, exfiltration/poisoning risk and HTTPS,
49
+ each `met` / `not_met` / `not_assessable` with source URL, check IDs and a note.
50
+ Derived only from checks that ran; ownership (3F) is always not assessable from the CLI.
51
+ - **Rate-limit handling.** HTTP 429 is honoured with `Retry-After` (1 s when absent), at
52
+ most 3 retries per request, within a scan-wide waiting budget (`--rate-limit-wait`,
53
+ default 30 s). When the budget is used up, the checks that met the throttled
54
+ responses are reported `not_applicable` (blocked) and their findings withheld,
55
+ instead of drawing conclusions from throttled responses. DOS-01's burst still
56
+ observes 429 directly. A target that rate-limits negotiation is reported
57
+ `unreachable`, not "not an MCP server".
58
+ - **Optional Hugging Face judge for META-01** (`--judge hf:<model-id-or-path>`,
59
+ `--judge-download`, `--judge-threshold`; extra `mcpprobe-scanner[hf]`). Off by
60
+ default, safetensors only, no remote code, no download unless asked. Licence notes in
61
+ docs/judges.md.
62
+ - `scripts/benchmark_judge.py` and docs/benchmark.md: precision, recall and false-positive
63
+ rate of META-01 on the in-sample corpus and on a new held-out set (65% recall at medium
64
+ or above, 0 false positives on 40 benign descriptions). The Hugging Face Hub was not
65
+ reachable when the numbers were produced, so no public dataset or classifier was scored.
66
+ - Report schema 0.3: `checks[].blocked`, `fingerprint.supported_versions`, `readiness`.
67
+ - Fixture `trifecta` (a conformant 2026-07-28 server whose tools form the trifecta); all
68
+ its check outcomes are pinned in CI.
69
+
70
+ ### Changed
71
+
72
+ - Annotation contradictions moved from META-03 to META-06 (reported once). META-03 is
73
+ now titled "Dangerous tool capability exposed". E-mail and payment tools count as
74
+ writes only when a mutating verb (send, charge, refund, ...) is present, so a
75
+ read-only `list_emails` marked `readOnlyHint: true` is no longer a contradiction.
76
+ - 2026-07-28 requests now carry the required `io.modelcontextprotocol/clientCapabilities`
77
+ and the recommended `clientInfo` in `_meta`.
78
+ - The fingerprint reads `supportedVersions` and `_meta["io.modelcontextprotocol/serverInfo"]`
79
+ from `server/discover` as the 2026-07-28 schema defines them (the pre-release keys are
80
+ no longer merged into `server_info`).
81
+ - The `hardened` fixture is now dual-era (2026-07-28 and 2025-11-25) with
82
+ spec-conformant `server/discover`, header validation, caching hints, RFC 9207 and
83
+ CIMD metadata; it still produces zero findings.
84
+
85
+ ### GitHub Action
86
+
87
+ - The Action now names its outcome: one annotation per run (`mcpprobe: findings`,
88
+ `mcpprobe: not an MCP server`, `mcpprobe: scan error`, or a `mcpprobe: passed`
89
+ notice), a job summary with a severity table, and a new `result` output (`clean`,
90
+ `findings`, `error`, `not-mcp`). Exit code 3 (not an MCP server) is documented in
91
+ the `exit-code` output. `sarif-file` is empty when no SARIF was written.
92
+ - `action.yml` description shortened to 115 characters for GitHub Marketplace.
93
+ - README and new `docs/github-action.md` pin `@v0.3.0` and upload SARIF only for exit
94
+ codes 0 and 1: SARIF from an error or a non-MCP target has no results and would
95
+ close every open alert. New manual-only example workflow
96
+ `.github/workflows/example-scan.yml`.
97
+
98
+
99
+ ## [0.2.0]
100
+
101
+ Accuracy release, driven by an independent review of a 0.1.0 scan of the project's own
102
+ deliberately weak demo server. 0.1.0 missed the demo's textbook tool poisoning,
103
+ reported four checks as passed that had tested nothing, and rated an unauthenticated
104
+ remote shell low. All of that is fixed and pinned by fixtures in CI.
105
+
106
+ - Licence changes from MIT to Apache-2.0 for future releases. Version 0.1.0 remains available under MIT.
107
+
108
+ ### Fixed
109
+
110
+ - **META-01 missed obvious poisoning.** It now detects instructions aimed at the model
111
+ ("always call this first", "call this tool before any other", "you must", "do not
112
+ tell the user", "ignore previous instructions"), requests to send the model's context
113
+ out (conversation, chat history, system prompt, previous messages), requests to
114
+ harvest the user's secrets, and HTML comments, in tool, argument, prompt and resource
115
+ descriptions. A directive combined with a context-exfiltration request is high. The
116
+ demo's `run_command` description is now reported high. Several 0.1 patterns that
117
+ fired on benign text ("Read the file at the given path", "Forward an email",
118
+ "Silently ignores unknown fields", "Operating system: Linux", "rather than the user's
119
+ timezone") were tightened; a corpus of 33 benign real-world descriptions must stay
120
+ below medium in the test-suite.
121
+ - **Checks no longer report "passed" for things they did not test.** New check status
122
+ `not_applicable` (with a one-sentence reason and an optional `related_to`), raised
123
+ through `NotApplicable`. Every early exit was audited: AUTH-03/AUTH-04 are not
124
+ applicable when the server needs no authentication (related to AUTH-01), TRANS-03
125
+ when no `Mcp-Session-Id` is issued, META-* when no tool metadata could be read,
126
+ ERR-01/DOS-01 when every probe was stopped at authentication, META-02 without a
127
+ previous baseline, TRANS-01/TRANS-02 when a 401/429 hides the behaviour. Transport
128
+ failures are now `error`, not a pass. Reporters count only `ran` with no findings
129
+ as passed; SARIF lists not-applicable checks as notifications, never as passes.
130
+ - **AUTH-02 passed when the metadata was missing.** Missing RFC 9728 metadata is now a
131
+ finding on unauthenticated servers too (medium on public hosts, low on loopback).
132
+ - **AUTH-01 misreported servers scanned with credentials.** When `--header` supplies
133
+ credentials, one extra request is sent with them removed; a 401/403 marks the server
134
+ protected. AUTH-04's query-string probe no longer carries the operator's own token.
135
+ - **Not an MCP server.** A target that answers but does not speak MCP (for example an
136
+ ordinary web site) no longer aborts with a generic error or produces results: the
137
+ report has `target_kind: "not_mcp"`, every check is not applicable with the
138
+ observation ("Not an MCP server: initialize -> HTTP 200 text/html"), and the CLI
139
+ exits with the new code 3. Unreachable targets get `target_kind: "unreachable"`
140
+ (exit 2, as before).
141
+
142
+ ### Changed
143
+
144
+ - **META-03 rates by capability.** Tools that advertise arbitrary command execution or
145
+ arbitrary file read/write (a shell or file capability selected by a free-form string
146
+ argument) are high on their own and critical on an unauthenticated public endpoint.
147
+ The fix text says to remove or sandbox the tool with an allow-list; annotations come
148
+ after. The description states that a passive scan sees the advertised capability, not
149
+ whether the implementation really executes. Annotation contradictions are one finding
150
+ listing every affected tool. Check title updated accordingly.
151
+ - **One finding per check per root cause.** META-04 emits one finding listing every
152
+ affected tool, with each tool's complete `inputSchema` and its specific problems
153
+ (including arguments missing from `required` and `additionalProperties` not false).
154
+ - **Evidence and proof.** Every finding has `proof` (the complete object judged, built
155
+ from parsed data), `verify` (one runnable line, a `curl` against the real target for
156
+ HTTP), `affected`, `evidence_ids`, and `confidence_reason` whenever confidence is not
157
+ high (enforced). Evidence keeps every request and response header, with credential
158
+ headers redacted, and bodies up to 64 KiB (was 4 KiB).
159
+ - **Environment-aware fix text.** The edge is detected from response headers
160
+ (`apigw-requestid`/`x-amz-apigw-id` -> AWS API Gateway, CloudFront, Cloudflare
161
+ `cf-ray`, Vercel `x-vercel-id`) and public vs local hosts are told apart. TRANS-05 and
162
+ DOS-01 say where to configure TLS, headers and throttling; AUTH-01 and TRANS-01 drop
163
+ "bind to 127.0.0.1" on public hosts. AUTH-01 shows the exact `WWW-Authenticate` header
164
+ with the target's metadata URL and notes that authentication does not make a shell
165
+ tool safe. TRANS-01 also probes and advises on `Origin: null` and allows requests with
166
+ no Origin. TRANS-02 mentions version negotiation on `initialize`.
167
+ - **Attack chains.** New top-level `chains`: CHAIN-01 (unauthenticated + arbitrary
168
+ capability, + poisoned description when META-01 fired on that tool, critical) and
169
+ CHAIN-02 (the same on a local server that accepts foreign Origins). A chain is only
170
+ emitted when every constituent finding is present.
171
+ - Report schema 0.2 (`schema_version`, `target_kind`, `environment`, `chains`, the new
172
+ finding fields, `checks[].related_to`, `summary.checks_passed` /
173
+ `checks_not_applicable`), documented in `docs/report-schema.md`.
174
+
175
+ ### Tests
176
+
177
+ - Fixtures pinned in CI: an exact replica of the public demo server (`--preset demo`),
178
+ the observable passive behaviour of a hardened reference server (`--preset
179
+ hardened`), and an ordinary HTML site (`--preset html`). The exact outcome of every
180
+ check against each is asserted; any change fails the suite.
181
+
182
+ ## [0.1.0]
183
+
184
+ - Published on PyPI as `mcpprobe-scanner` (PyPI blocks `mcpprobe` and `mcpprobe-cli` as too similar to existing projects). The command and the Python package are both `mcpprobe`.
185
+ - 2026-10-27
186
+
187
+ First public release of the open-source scanner core.
188
+
189
+ ### Added
190
+
191
+ - MCP client for Streamable HTTP (JSON-RPC over POST, JSON or SSE responses)
192
+ and stdio, written to send deliberately non-conformant requests.
193
+ - Protocol fingerprinting: `server/discover` (2026-07-28) first, then
194
+ `initialize` with 2025-11-25 and 2025-06-18, then the deprecated
195
+ 2024-11-05 HTTP+SSE transport.
196
+ - Evidence log: every request/response with timestamp, method, status, a
197
+ header subset, bodies truncated to 4 KB, `Authorization` values and known
198
+ secret patterns redacted.
199
+ - Passive checks: AUTH-01..04, TRANS-01..05, META-01..05, ERR-01, DOS-01.
200
+ - Active checks INJ-01..05, SUPPLY-01/02 and SANDBOX-01 registered as gated
201
+ stubs (report `not_implemented`; no payloads are sent).
202
+ - Governance items GOV-01..05 listed as attestation-only in every report.
203
+ - Authorisation gate for active mode: `--authorised-by`, `--scope` and
204
+ ownership verification (`--verify dns-txt` via `_mcpprobe.<host>` TXT, or
205
+ `--verify local` for stdio); an authorisation record JSON is written next to
206
+ the report.
207
+ - Opt-out: DNS TXT `mcpprobe-opt-out=all` at `_mcpprobe.<host>` and
208
+ `/.well-known/mcpprobe-opt-out`, both checked before any MCP request.
209
+ - Rate limiting: 5 requests/second by default, hard ceiling 10.
210
+ - Reports: JSON (with evidence), SARIF 2.1.0 (validated against the official
211
+ schema in tests) and a rich terminal summary. Exit codes 0 / 1 / 2.
212
+ - META-02 baselines: canonical SHA-256 of tools, prompts, resources and
213
+ instructions with diffing against a stored baseline (rug-pull detection).
214
+ - Optional LLM-judge hook for META-01 (`ScanOptions.judge`); mcpprobe makes no
215
+ LLM or third-party network calls itself.
216
+ - Docker container GitHub Action, CI (ruff, strict mypy, pytest on Python
217
+ 3.11 to 3.13) and PyPI Trusted Publishing release workflow.
218
+
219
+ [0.1.0]: https://github.com/mcpprobe-dev/mcpprobe/releases/tag/v0.1.0
@@ -1,6 +1,6 @@
1
1
  # mcpprobe checks catalogue
2
2
 
3
- Generated from the check metadata of mcpprobe 0.1.0 (`mcpprobe checks --format markdown`). Do not edit by hand.
3
+ Generated from the check metadata of mcpprobe 0.3.0 (`mcpprobe checks --format markdown`). Do not edit by hand.
4
4
 
5
5
  Standards: OWASP MCP Top 10 (IDs `MCP0x:2025`, still a beta list, so shown with `(beta)`), OWASP Top 10 for Agentic Applications (`ASI0x`), OWASP Top 10 for LLM Applications 2026 (`LLM0x:2026`, primary) and 2025 (`LLM0x:2025`, kept for one release cycle). Severities are defaults; a check can raise or lower the severity of an individual finding (for example loopback-only servers are reported one level lower), and heuristic checks never rate a finding Critical on their own.
6
6
 
@@ -14,16 +14,24 @@ Passive checks send only what a normal MCP client would send plus a few benign m
14
14
  | [AUTH-02](#auth-02---protected-resource-metadata-rfc-9728-missing-or-invalid) | passive | high | Protected resource metadata (RFC 9728) missing or invalid | MCP07:2025 (beta) |
15
15
  | [AUTH-03](#auth-03---authorization-server-metadata-weak-or-missing-pkce-s256--https) | passive | high | Authorization server metadata weak or missing (PKCE S256 / HTTPS) | MCP07:2025 (beta), MCP01:2025 (beta) |
16
16
  | [AUTH-04](#auth-04---server-accepts-a-self-minted-token-with-the-wrong-audienceissuer) | passive | critical | Server accepts a self-minted token with the wrong audience/issuer | MCP07:2025 (beta), MCP01:2025 (beta), ASI03 |
17
+ | [AUTH-05](#auth-05---authorization-server-does-not-advertise-rfc-9207-iss-support-2026-07-28) | passive | low | Authorization server does not advertise RFC 9207 iss support (2026-07-28) | MCP07:2025 (beta) |
18
+ | [AUTH-06](#auth-06---deprecated-dynamic-client-registration-offered-2026-07-28) | passive | low | Deprecated Dynamic Client Registration offered (2026-07-28) | MCP07:2025 (beta) |
17
19
  | [TRANS-01](#trans-01---origin-header-not-validated-dns-rebinding-exposure) | passive | medium | Origin header not validated (DNS-rebinding exposure) | MCP07:2025 (beta) |
18
20
  | [TRANS-02](#trans-02---unsupported-mcp-protocol-version-header-not-rejected) | passive | low | Unsupported MCP-Protocol-Version header not rejected | MCP spec conformance / hygiene (no OWASP item) |
19
21
  | [TRANS-03](#trans-03---weak-predictable-or-unvalidated-mcp-session-id) | passive | high | Weak, predictable or unvalidated Mcp-Session-Id | MCP07:2025 (beta) |
20
22
  | [TRANS-04](#trans-04---deprecated-httpsse-2024-11-05-transport-in-use) | passive | info | Deprecated HTTP+SSE (2024-11-05) transport in use | MCP spec conformance / hygiene (no OWASP item) |
21
23
  | [TRANS-05](#trans-05---plain-http-weak-tls-or-missing-security-headers) | passive | high | Plain HTTP, weak TLS or missing security headers | MCP01:2025 (beta) |
24
+ | [TRANS-06](#trans-06---serverdiscover-result-incomplete-2026-07-28) | passive | low | server/discover result incomplete (2026-07-28) | MCP spec conformance / hygiene (no OWASP item) |
25
+ | [TRANS-07](#trans-07---protocol-version-the-server-does-not-advertise-is-accepted-downgrade) | passive | low | Protocol version the server does not advertise is accepted (downgrade) | MCP spec conformance / hygiene (no OWASP item) |
26
+ | [TRANS-08](#trans-08---mcp-method-header-not-validated-against-the-request-body-2026-07-28) | passive | medium | Mcp-Method header not validated against the request body (2026-07-28) | MCP spec conformance / hygiene (no OWASP item) |
27
+ | [TRANS-09](#trans-09---ttlms--cachescope-caching-hints-missing-or-invalid-2026-07-28) | passive | low | ttlMs / cacheScope caching hints missing or invalid (2026-07-28) | MCP spec conformance / hygiene (no OWASP item) |
22
28
  | [META-01](#meta-01---possible-tool-description-poisoning) | passive | high | Possible tool-description poisoning | MCP03:2025 (beta), ASI01, ASI04, LLM01:2026, LLM01:2025 |
23
29
  | [META-02](#meta-02---tool-surface-changed-since-the-stored-baseline-rug-pull-indicator) | passive | medium | Tool surface changed since the stored baseline (rug-pull indicator) | MCP03:2025 (beta), ASI04 |
24
- | [META-03](#meta-03---tool-annotations-contradict-the-tools-apparent-capability) | passive | medium | Tool annotations contradict the tool's apparent capability | MCP02:2025 (beta), MCP05:2025 (beta), ASI02, ASI05, LLM03:2026, LLM06:2025 |
30
+ | [META-03](#meta-03---dangerous-tool-capability-exposed) | passive | high | Dangerous tool capability exposed | MCP02:2025 (beta), MCP05:2025 (beta), ASI02, ASI05, LLM03:2026, LLM06:2025 |
25
31
  | [META-04](#meta-04---tool-schema-hygiene-problems) | passive | low | Tool schema hygiene problems | MCP05:2025 (beta) |
26
32
  | [META-05](#meta-05---secret-material-in-server-metadata-or-configuration) | passive | high | Secret material in server metadata or configuration | MCP01:2025 (beta), LLM02:2026, LLM02:2025 |
33
+ | [META-06](#meta-06---tool-annotations-missing-or-inconsistent-with-the-tool) | passive | medium | Tool annotations missing or inconsistent with the tool | MCP spec conformance / hygiene (no OWASP item) |
34
+ | [META-07](#meta-07---toxic-flow-private-data-untrusted-content-and-external-send-on-one-server) | passive | medium | Toxic flow: private data, untrusted content and external send on one server | ASI01, LLM01:2026, LLM01:2025 |
27
35
  | [ERR-01](#err-01---verbose-error-responses-leak-stack-traces-paths-or-versions) | passive | medium | Verbose error responses leak stack traces, paths or versions | MCP08:2025 (beta), LLM02:2026, LLM02:2025 |
28
36
  | [DOS-01](#dos-01---no-visible-limits-on-request-size-nesting-or-rate) | passive | low | No visible limits on request size, nesting or rate | ASI08, LLM06:2026, LLM10:2025 |
29
37
  | [INJ-01](#inj-01---commandcode-injection-safe-canaries) | active | critical | Command/code injection (safe canaries) | MCP05:2025 (beta), ASI05, LLM10:2026 |
@@ -50,13 +58,13 @@ Passive checks send only what a normal MCP client would send plus a few benign m
50
58
  | Default severity | high |
51
59
  | Transports | http |
52
60
  | Standards | MCP07:2025 (beta), ASI03, LLM04:2026, LLM03:2026, LLM03:2025, LLM06:2025 |
53
- | Evidence | The unauthenticated tools/list request and the 200 response. |
61
+ | Evidence | The unauthenticated request (Authorization shown as not sent) and the response. |
54
62
 
55
- POSTs initialize (or server/discover) and tools/list with no Authorization header. If the server answers with a tool list, anyone who can reach the endpoint can drive its tools. Missing authentication was the single most common issue in public MCP server studies.
63
+ POSTs initialize (or server/discover) and tools/list with no Authorization header. If the server answers with a tool list, anyone who can reach the endpoint can drive its tools. When the operator supplies credentials with --header, one extra request is sent with those credentials removed, so a protected server is not misreported. Passes only when the endpoint answers 401/403 without credentials.
56
64
 
57
65
  **False positives.** Servers protected by a network boundary (VPN, mTLS at a proxy, IP allow-list) will still be flagged because the scanner only sees what the endpoint returns. Local development servers bound to loopback are reported at reduced severity but are still exposed to DNS-rebinding and browser-based attacks (see CVE-2025-49596 class).
58
66
 
59
- **Fix.** Put the endpoint behind OAuth 2.1 as described in the MCP authorization spec: return 401 with WWW-Authenticate pointing at RFC 9728 protected-resource metadata, validate token audience, and reject requests without a valid bearer token. For local servers, bind to 127.0.0.1 and still require a token.
67
+ **Fix.** Put the endpoint behind OAuth 2.1 as described in the MCP authorization spec: return 401 with WWW-Authenticate: Bearer resource_metadata="<RFC 9728 metadata URL>", validate each token's signature, issuer, expiry and audience, and reject requests without a valid bearer token. Authentication does not make a dangerous tool safe.
60
68
 
61
69
  ### AUTH-02 - Protected resource metadata (RFC 9728) missing or invalid
62
70
 
@@ -66,11 +74,11 @@ POSTs initialize (or server/discover) and tools/list with no Authorization heade
66
74
  | Default severity | high |
67
75
  | Transports | http |
68
76
  | Standards | MCP07:2025 (beta) |
69
- | Evidence | The 401 response headers and each metadata GET with its status/body. |
77
+ | Evidence | The 401 response headers and each metadata GET with its status. |
70
78
 
71
- On a 401 the scanner parses WWW-Authenticate for resource_metadata, then tries /.well-known/oauth-protected-resource[/path] and the host root. The document must be JSON with a non-empty authorization_servers list. Without it, conformant clients cannot start the OAuth flow and users are pushed toward pasting long-lived static tokens.
79
+ Parses WWW-Authenticate for resource_metadata (when the server answers 401), then tries /.well-known/oauth-protected-resource[/path] and the host root. The document must be JSON with a non-empty authorization_servers list. Without it, conformant clients cannot start the OAuth flow and users are pushed toward pasting long-lived static tokens. A server that requires no authentication and serves no metadata is reported too (medium, the discovery half of AUTH-01).
72
80
 
73
- **False positives.** Servers that use a non-OAuth scheme (API keys at a gateway) will be flagged; the MCP spec still expects RFC 9728 metadata for HTTP transports.
81
+ **False positives.** Servers that use a non-OAuth scheme (API keys at a gateway) will be flagged; the MCP spec still expects RFC 9728 metadata for HTTP transports. A deliberately public, read-only server with no authentication will also be flagged; AUTH-01 is the root cause.
74
82
 
75
83
  **Fix.** Serve /.well-known/oauth-protected-resource (path-aware variant preferred) with 'resource' and 'authorization_servers', and include WWW-Authenticate: Bearer resource_metadata="<url>" on every 401.
76
84
 
@@ -106,6 +114,38 @@ Mints two syntactically valid JWTs the server cannot have issued (HS256 with a r
106
114
 
107
115
  **Fix.** Validate the token signature against the authorization server's keys, require 'aud' to equal this server's canonical URI (RFC 8707), reject alg=none, and never read tokens from query strings.
108
116
 
117
+ ### AUTH-05 - Authorization server does not advertise RFC 9207 iss support (2026-07-28)
118
+
119
+ | | |
120
+ |---|---|
121
+ | Mode | passive (implemented) |
122
+ | Default severity | low |
123
+ | Transports | http |
124
+ | Standards | MCP07:2025 (beta) |
125
+ | Evidence | The authorization server metadata document AUTH-03 fetched. |
126
+
127
+ 2026-07-28 says MCP authorization servers SHOULD include the RFC 9207 'iss' parameter in authorization responses and, when they do, MUST advertise authorization_response_iss_parameter_supported: true in their metadata; clients reject a response without iss only when that flag is true. Reads the authorization server metadata AUTH-03 already fetched and reports a flag that is absent or not true. Sends nothing. The revision says this SHOULD is expected to become a MUST.
128
+
129
+ **False positives.** The spec allows a server to emit iss before it updates its metadata; such a server is reported until the flag is published, because clients key their rejection on the flag.
130
+
131
+ **Fix.** Include iss in every authorization response, error responses too (RFC 9207 section 2), and publish "authorization_response_iss_parameter_supported": true in the authorization server metadata (https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization#authorization-response-validation). With a hosted identity provider, enable its issuer-identification option.
132
+
133
+ ### AUTH-06 - Deprecated Dynamic Client Registration offered (2026-07-28)
134
+
135
+ | | |
136
+ |---|---|
137
+ | Mode | passive (implemented) |
138
+ | Default severity | low |
139
+ | Transports | http |
140
+ | Standards | MCP07:2025 (beta) |
141
+ | Evidence | The authorization server metadata document AUTH-03 fetched. |
142
+
143
+ 2026-07-28 deprecates OAuth Dynamic Client Registration (RFC 7591) in favour of Client ID Metadata Documents, which authorization servers SHOULD support. Reads the authorization server metadata AUTH-03 already fetched: a registration_endpoint without client_id_metadata_document_supported: true is low (clients have only the deprecated path); a registration_endpoint next to CIMD support is informational (kept for backwards compatibility). No registration request is sent.
144
+
145
+ **False positives.** Keeping DCR for older clients is allowed; the informational finding is a reminder to plan its removal, not a defect. An authorization server that supports neither DCR nor CIMD (pre-registration only) is not reported here.
146
+
147
+ **Fix.** Support Client ID Metadata Documents and advertise "client_id_metadata_document_supported": true; keep registration_endpoint only while older clients still need it, and plan its removal (https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization/client-registration). Open registration endpoints also deserve rate limits and redirect-URI validation.
148
+
109
149
  ### TRANS-01 - Origin header not validated (DNS-rebinding exposure)
110
150
 
111
151
  | | |
@@ -114,13 +154,13 @@ Mints two syntactically valid JWTs the server cannot have issued (HS256 with a r
114
154
  | Default severity | medium |
115
155
  | Transports | http |
116
156
  | Standards | MCP07:2025 (beta) |
117
- | Evidence | The POST with the foreign Origin header and the non-403 response. |
157
+ | Evidence | Each POST with a foreign Origin header and the non-403 response. |
118
158
 
119
- Sends a normal negotiation request with 'Origin: https://evil.example'. The transport spec says servers MUST validate Origin and return 403 on an invalid one; otherwise a web page can drive a locally running MCP server through the victim's browser (DNS rebinding).
159
+ Sends a normal negotiation request with 'Origin: https://evil.example' and one with 'Origin: null'. The transport spec says servers MUST validate Origin and return 403 on an invalid one; otherwise a web page can drive the server through the victim's browser (DNS rebinding). Requests without an Origin header are not tested: non-browser clients do not send one.
120
160
 
121
- **False positives.** If the server answers 401 before looking at Origin the check is indeterminate and no finding is raised (see scan notes). Reverse proxies that strip Origin will cause a finding against the proxied endpoint even if the application validates it.
161
+ **False positives.** If the server answers 401 (or 429) before looking at Origin the check is not applicable and says so. Reverse proxies that strip Origin will cause a finding against the proxied endpoint even if the application validates it.
122
162
 
123
- **Fix.** Reject requests whose Origin is not on an allow-list with HTTP 403 before processing the body. For local servers also bind to 127.0.0.1 rather than 0.0.0.0.
163
+ **Fix.** Reject any request whose Origin header is present and not on an allow-list with HTTP 403, before processing the body, including 'Origin: null'. Allow requests with no Origin header.
124
164
 
125
165
  ### TRANS-02 - Unsupported MCP-Protocol-Version header not rejected
126
166
 
@@ -134,9 +174,9 @@ Sends a normal negotiation request with 'Origin: https://evil.example'. The tran
134
174
 
135
175
  Sends a request with 'MCP-Protocol-Version: 1999-01-01'. The spec requires HTTP 400 (or, on 2026-07-28, an UnsupportedProtocolVersionError). Accepting any version string suggests the server ignores version negotiation entirely.
136
176
 
137
- **False positives.** Pre-2025-06-18 servers never defined the header, so an older server that ignores it is noted at low severity only.
177
+ **False positives.** Pre-2025-06-18 servers never defined the header, so an older server that ignores it is noted at low severity only. A 401/429 before the header is read makes the check not applicable.
138
178
 
139
- **Fix.** Validate MCP-Protocol-Version against the supported list and reply 400 when it is unknown.
179
+ **Fix.** Validate the MCP-Protocol-Version header against the versions you support and reply 400 when it is unknown. Separately, when initialize asks for a protocolVersion you do not support, answer with the latest version you do support and let the client decide; never echo back whatever the client asked for.
140
180
 
141
181
  ### TRANS-03 - Weak, predictable or unvalidated Mcp-Session-Id
142
182
 
@@ -184,7 +224,71 @@ Flags non-loopback endpoints served over plain HTTP (bearer tokens in clear), TL
184
224
 
185
225
  **False positives.** Loopback plain-HTTP is reported as informational. Header checks look at the MCP endpoint only; a CDN may add headers on other paths.
186
226
 
187
- **Fix.** Serve the endpoint over HTTPS with TLS 1.2+ only, add Strict-Transport-Security and X-Content-Type-Options: nosniff.
227
+ **Fix.** Serve the endpoint over HTTPS with TLS 1.2+ only, add Strict-Transport-Security and X-Content-Type-Options: nosniff. Where an edge (API Gateway, CloudFront, Cloudflare, Vercel) is detected the fix text says where to configure it.
228
+
229
+ ### TRANS-06 - server/discover result incomplete (2026-07-28)
230
+
231
+ | | |
232
+ |---|---|
233
+ | Mode | passive (implemented) |
234
+ | Default severity | low |
235
+ | Transports | http, stdio |
236
+ | Standards | MCP spec conformance / hygiene (no OWASP item) |
237
+ | Evidence | The fingerprinting server/discover request and its complete result. |
238
+
239
+ A 2026-07-28 server MUST implement server/discover. The scanner's first request is server/discover; when it is answered, its result is checked for what the revision requires: supportedVersions (a list of version strings that includes the 2026-07-28 revision the request was served under), a capabilities object and resultType 'complete'. A missing _meta['io.modelcontextprotocol/serverInfo'] (SHOULD) is reported as informational. Caching hints on the same result are checked by TRANS-09. Not applicable to servers that speak an earlier revision through initialize.
240
+
241
+ **False positives.** A server that answers server/discover under an earlier result shape (for example a 'protocolVersions' key from a pre-release draft) is reported, because 2026-07-28 clients read 'supportedVersions'.
242
+
243
+ **Fix.** Return the DiscoverResult defined in the 2026-07-28 schema: {"resultType":"complete","supportedVersions":["2026-07-28", ...],"capabilities":{...},"_meta":{"io.modelcontextprotocol/serverInfo":{"name":...,"version":...}},"ttlMs":...,"cacheScope":"public"} (https://modelcontextprotocol.io/specification/2026-07-28/server/discover).
244
+
245
+ ### TRANS-07 - Protocol version the server does not advertise is accepted (downgrade)
246
+
247
+ | | |
248
+ |---|---|
249
+ | Mode | passive (implemented) |
250
+ | Default severity | low |
251
+ | Transports | http |
252
+ | Standards | MCP spec conformance / hygiene (no OWASP item) |
253
+ | Evidence | The server/discover result (supportedVersions) and the downgraded request with its response. |
254
+
255
+ Takes supportedVersions from the server/discover result and sends one more server/discover declared (in _meta and in the MCP-Protocol-Version header) as the newest earlier revision the server does not list (2025-11-25, 2025-06-18 or 2025-03-26). The spec says a server that does not implement the requested version MUST answer HTTP 400 with UnsupportedProtocolVersionError (-32022). Serving the request anyway means a client or intermediary can push the server onto a revision it claims not to support, without the header validation and other protections of 2026-07-28. One request, no tool is called.
256
+
257
+ **False positives.** A dual-era server that does serve the older revision but forgot to list it is reported too; listing it in supportedVersions is then the fix. Not applicable when every earlier revision is advertised.
258
+
259
+ **Fix.** Reject any request whose declared protocol version is not in your supportedVersions with HTTP 400 and {"code":-32022,"message":"Unsupported protocol version","data":{"supported":[...],"requested":"..."}}, before doing any work (https://modelcontextprotocol.io/specification/2026-07-28/basic/versioning). If you do serve the older revision, list it in supportedVersions.
260
+
261
+ ### TRANS-08 - Mcp-Method header not validated against the request body (2026-07-28)
262
+
263
+ | | |
264
+ |---|---|
265
+ | Mode | passive (implemented) |
266
+ | Default severity | medium |
267
+ | Transports | http |
268
+ | Standards | MCP spec conformance / hygiene (no OWASP item) |
269
+ | Evidence | Both probe requests (headers shown) and their responses. |
270
+
271
+ 2026-07-28 requires clients to mirror the JSON-RPC method into an Mcp-Method header, and servers that process the body MUST reject a request whose headers are missing or do not match the body with HTTP 400 and HeaderMismatch (-32020); otherwise a gateway that routes, rate-limits or authorises on the header can be bypassed by a body that says something else. Sends two server/discover requests: one with 'Mcp-Method: tools/list' and one with no Mcp-Method header. Mcp-Name is not tested, because it is only defined for tools/call, resources/read and prompts/get, which a passive scan does not send.
272
+
273
+ **False positives.** A server that rejects the probes with another 4xx is reported as informational only. When a gateway in front rejects mismatches itself, the finding concerns the server behind it only if the gateway can be bypassed.
274
+
275
+ **Fix.** Before dispatching, compare Mcp-Method (and, for tools/call, resources/read and prompts/get, the decoded Mcp-Name) with the body; when a required header is missing or differs, answer HTTP 400 with {"code":-32020,"message":"Header mismatch: ..."} (https://modelcontextprotocol.io/specification/2026-07-28/basic/transports/streamable-http#server-validation).
276
+
277
+ ### TRANS-09 - ttlMs / cacheScope caching hints missing or invalid (2026-07-28)
278
+
279
+ | | |
280
+ |---|---|
281
+ | Mode | passive (implemented) |
282
+ | Default severity | low |
283
+ | Transports | http, stdio |
284
+ | Standards | MCP spec conformance / hygiene (no OWASP item) |
285
+ | Evidence | The server/discover and list results the scan already received. |
286
+
287
+ 2026-07-28 servers MUST include caching hints on complete results of server/discover, tools/list, prompts/list and resources/list (among others): ttlMs, a number of milliseconds that MUST be >= 0, and cacheScope, either 'public' or 'private'. Checks the results the scan already received; sends nothing. A missing cacheScope leaves shared gateways to guess whether a list may be served to other users.
288
+
289
+ **False positives.** Only the first page of each list is checked. A public cacheScope on an authenticated server is allowed by the spec when the list is the same for every user, so it is not reported.
290
+
291
+ **Fix.** Add "ttlMs": <milliseconds, >= 0> and "cacheScope": "public" (identical for every user) or "private" (depends on the caller's authorization) to every cacheable result (https://modelcontextprotocol.io/specification/2026-07-28/server/utilities/caching).
188
292
 
189
293
  ### META-01 - Possible tool-description poisoning
190
294
 
@@ -196,11 +300,11 @@ Flags non-loopback endpoints served over plain HTTP (bearer tokens in clear), TL
196
300
  | Standards | MCP03:2025 (beta), ASI01, ASI04, LLM01:2026, LLM01:2025 |
197
301
  | Evidence | The tools/list (or prompts/resources/initialize) response and the matched excerpts. |
198
302
 
199
- Scans every tool, prompt and resource description, every inputSchema parameter description and the server 'instructions' for: prompt-style markup (<IMPORTANT>, <system>), instructions aimed at the model (conceal from user, override instructions, read-and-forward, prerequisite coercion, redirect/replace), credential file paths, references to other tools with behaviour-modifying language (shadowing), URLs and e-mail addresses with send/BCC wording, base64 blobs, zero-width/bidi/tag Unicode, mixed-script homoglyphs and excessive length. Findings carry tiered confidence and are never rated Critical on heuristics alone.
303
+ Scans every tool, prompt and resource description, every inputSchema parameter description and the server 'instructions' for: prompt-style markup (<IMPORTANT>, <system>, HTML comments), instructions aimed at the model (call this first / before any other tool, you must, conceal from the user, ignore previous instructions, redirect/replace), requests to send the model's context out (conversation, chat history, system prompt, previous messages), requests to harvest secrets (the user's API key, token, password, credentials, .env), credential file paths, references to other tools with behaviour-modifying language (shadowing), URLs and e-mail addresses with send/BCC wording, base64 blobs, zero-width/bidi/tag Unicode, mixed-script homoglyphs and excessive length. Findings carry tiered confidence and are never rated Critical on heuristics alone. A model directive combined with a request to send context out is rated high. Precision is guarded by a corpus of benign real-world descriptions in the test-suite that must stay below medium.
200
304
 
201
305
  **False positives.** Legitimate guidance such as 'IMPORTANT: call authenticate before this tool' matches the markup and prerequisite heuristics at medium confidence. URLs to documentation and mentions of sibling tools without behaviour-modifying words are reported at low severity. Review the matched excerpts; a second-opinion judge can be plugged in through the Python API (ScanOptions.judge); mcpprobe itself makes no LLM calls.
202
306
 
203
- **Fix.** Remove any text addressed to the model rather than describing the tool. Keep descriptions short and factual, strip invisible Unicode, and never reference other servers' tools. Pin and hash approved descriptions in clients (see META-02).
307
+ **Fix.** Rewrite the description so it only says what the tool does. Remove every instruction addressed to the model (when to call it, what to call first, what to hide from the user) and every request to pass along conversation, prompts, files or credentials. Strip invisible Unicode and HTML comments, and never reference other servers' tools. Pin and hash approved descriptions in clients (see META-02).
204
308
 
205
309
  ### META-02 - Tool surface changed since the stored baseline (rug-pull indicator)
206
310
 
@@ -218,21 +322,21 @@ Computes SHA-256 over the canonical JSON of each tool, prompt and resource defin
218
322
 
219
323
  **Fix.** Treat tool definitions as code: version them, review changes, and have clients pin and re-approve definitions when the hash changes.
220
324
 
221
- ### META-03 - Tool annotations contradict the tool's apparent capability
325
+ ### META-03 - Dangerous tool capability exposed
222
326
 
223
327
  | | |
224
328
  |---|---|
225
329
  | Mode | passive (implemented) |
226
- | Default severity | medium |
330
+ | Default severity | high |
227
331
  | Transports | http, stdio |
228
332
  | Standards | MCP02:2025 (beta), MCP05:2025 (beta), ASI02, ASI05, LLM03:2026, LLM06:2025 |
229
- | Evidence | The tools/list response; matched terms per tool are listed in details. |
333
+ | Evidence | The tools/list response; each affected tool's complete definition is shown as proof. |
230
334
 
231
- Classifies each tool by name, description and parameter names into capability classes (exec, file_write, file_read, database_*, network, email, credentials, payments, delete, search) and cross-checks the declared annotations: a tool that looks like it runs commands or sends e-mail but is marked readOnlyHint:true, or an exec/delete tool marked destructiveHint:false, is reported. A summary of the dangerous surface is reported at low severity. Annotations are treated as untrusted hints, as the spec requires of clients.
335
+ Classifies each tool by name, description and parameter names into capability classes. Tools that advertise arbitrary command execution, arbitrary file read or arbitrary file write (a shell/file capability selected by a free-form string argument with no enum, const or pattern) are rated high on their own and critical when the remote endpoint needs no authentication. The remaining high-impact surface (network, e-mail, credentials, payments) is summarised at low severity. Annotations that contradict the capability are reported by META-06 (until 0.2 they were reported here). Passive scanning sees the advertised capability, not whether the implementation really executes: the rating is based on the name, description and schema, and no tool is called.
232
336
 
233
- **False positives.** Classification is keyword-based: a tool named 'delete_draft' that only removes the user's own draft will be classed 'delete'. Confidence is medium for contradictions and the dangerous-surface summary is informational.
337
+ **False positives.** Classification is keyword-based: a tool named 'delete_draft' that only removes the user's own draft will be classed 'delete'. A tool that advertises a shell but whose implementation is a stub is still reported, because clients and models act on the advertised capability.
234
338
 
235
- **Fix.** Make annotations truthful (readOnlyHint only for tools with no side effects, destructiveHint:true for anything that deletes or executes), and consider splitting broad tools into narrower ones so clients can gate approval.
339
+ **Fix.** For arbitrary command execution or file access: remove the tool, or sandbox it and replace the free-form argument with an allow-list (enum) of commands or files. Annotations come after that (META-06); they help a careful client warn the user but do nothing about the risk.
236
340
 
237
341
  ### META-04 - Tool schema hygiene problems
238
342
 
@@ -242,13 +346,13 @@ Classifies each tool by name, description and parameter names into capability cl
242
346
  | Default severity | low |
243
347
  | Transports | http, stdio |
244
348
  | Standards | MCP05:2025 (beta) |
245
- | Evidence | The tools/list response. |
349
+ | Evidence | The tools/list response; each affected tool's complete inputSchema is shown as proof. |
246
350
 
247
- Checks tool names against [A-Za-z0-9_.-]{1,128} and for duplicates; checks that inputSchema is a JSON Schema object of type 'object'; flags free-form string arguments whose names suggest paths, commands, URLs or queries and that carry no enum, pattern, format or maxLength constraint. Missing outputSchema cannot be judged without calling the tool and is not reported.
351
+ Checks tool names against [A-Za-z0-9_.-]{1,128} and for duplicates; checks that inputSchema is a JSON Schema object of type 'object'; flags free-form string arguments whose names suggest paths, commands, URLs or queries and that carry no enum, pattern, format or maxLength constraint. One finding lists every affected tool, with each tool's complete inputSchema and its specific problems (including arguments missing from 'required' and additionalProperties not set to false). Missing outputSchema cannot be judged without calling the tool and is not reported.
248
352
 
249
353
  **False positives.** An unconstrained 'query' string is often unavoidable (search tools); the finding is Low severity and meant as a review prompt, not a defect.
250
354
 
251
- **Fix.** Use valid names, give every tool an object inputSchema, and constrain risky string arguments with enum/pattern/format/maxLength so clients and servers can validate.
355
+ **Fix.** Constrain every risky string argument (enum for a fixed set, pattern for a format, maxLength to bound size), list required arguments in 'required', set additionalProperties: false, and validate arguments on the server against the same schema instead of trusting the client to have done it.
252
356
 
253
357
  ### META-05 - Secret material in server metadata or configuration
254
358
 
@@ -266,6 +370,38 @@ Pattern and entropy scan for AWS/GitHub/Slack/Stripe/Google keys, private key bl
266
370
 
267
371
  **Fix.** Remove secrets from descriptions and resources, rotate anything that was exposed, and pass credentials through the environment or a secret store, never through metadata.
268
372
 
373
+ ### META-06 - Tool annotations missing or inconsistent with the tool
374
+
375
+ | | |
376
+ |---|---|
377
+ | Mode | passive (implemented) |
378
+ | Default severity | medium |
379
+ | Transports | http, stdio |
380
+ | Standards | MCP spec conformance / hygiene (no OWASP item) |
381
+ | Evidence | The tools/list response; each affected tool's name, title and annotations are shown as proof. |
382
+
383
+ Reads each tool's annotations from tools/list. Reported as missing: readOnlyHint, openWorldHint and a title (tool.title or annotations.title) on every tool, and destructiveHint and idempotentHint on every tool not marked readOnlyHint:true (the spec says those two are meaningful only when readOnlyHint is false). Reported as inconsistent: readOnlyHint:true on a tool that looks like it executes, writes, deletes, sends or pays; destructiveHint:false on a tool that looks like it executes, deletes or overwrites; openWorldHint:false on a tool that reaches the network; a hint that is not a boolean; and, as low-severity noise, destructiveHint:true on a tool that only appears to read. Medium when a dangerous-class tool is mislabelled, low when annotations are merely missing. One finding lists every affected tool. Anthropic's directory policy (5E) and OpenAI's app review both require accurate annotations.
384
+
385
+ **False positives.** The capability behind a contradiction is inferred from keywords in the tool's name, description and parameter names, so a narrowly scoped tool (for example 'delete_draft' that only removes the caller's own draft) may be misread. Missing annotations are a plain observation. Annotations are hints that clients must treat as untrusted; correcting them does not change what a tool can do.
386
+
387
+ **Fix.** Give every tool a title and set readOnlyHint and openWorldHint explicitly; for tools that are not read-only also set destructiveHint and idempotentHint. Make each value true to the implementation: readOnlyHint:true only when nothing is modified, destructiveHint:true for anything that executes, deletes or overwrites, openWorldHint:true for tools that reach the internet or other external systems. Anthropic's Software Directory Policy (5E, https://support.claude.com/en/articles/13145358-anthropic-software-directory-policy) requires all applicable annotations, and OpenAI's app review (https://developers.openai.com/plugins/deploy/app-review.md) checks readOnlyHint, destructiveHint and openWorldHint for accuracy.
388
+
389
+ ### META-07 - Toxic flow: private data, untrusted content and external send on one server
390
+
391
+ | | |
392
+ |---|---|
393
+ | Mode | passive (implemented) |
394
+ | Default severity | medium |
395
+ | Transports | http, stdio |
396
+ | Standards | ASI01, LLM01:2026, LLM01:2025 |
397
+ | Evidence | The tools/list response; each role lists its tools and the wording that put them there. |
398
+
399
+ Classifies every advertised tool into three roles from its capability class and wording: access to private data (files, databases, credentials, the user's mail), intake of untrusted content (fetching URLs, web search, reading mail or messages written by others) and a way to send data out (sending e-mail or messages, HTTP posts, webhooks, uploads). A tool advertising arbitrary command execution counts for all three; arbitrary file read counts as private data. When one server offers all three roles, text planted in the untrusted content can instruct the model to read the private data and send it away: the 'lethal trifecta' of prompt injection. Medium, high when the HTTP endpoint needs no authentication. One finding lists the tools in each role. No tool is called.
400
+
401
+ **False positives.** Roles are inferred from names, descriptions and capability keywords, not from behaviour. A 'fetch' tool restricted to one internal API, or a 'send' tool that only notifies the signed-in user, is still counted. Conversely a fetch tool with a free-form URL can leak data through the URL itself; that path is not counted as a send role.
402
+
403
+ **Fix.** Break the trifecta: split the tools across separate servers (or separate client configurations) so no single agent session combines private data, untrusted content and an outbound channel; or gate the send tool behind explicit per-call user confirmation and an allow-list of destinations; or constrain the intake tool to trusted sources. Keep the server authenticated so nobody else can drive the combination.
404
+
269
405
  ### ERR-01 - Verbose error responses leak stack traces, paths or versions
270
406
 
271
407
  | | |