@booyaka/mcp-vet 0.3.0 → 0.5.0

package/BENCHMARK.md

@@ -0,0 +1,112 @@
+# Benchmark: corpus, methodology, and honest limits
+
+The README claims high precision on real MCP code. This file is the evidence
+behind that claim — corpus, pinned commits, counts, labels, and what the
+scanner is *known to miss* — so the claim is checkable rather than vibes.
+
+> Prompted by community feedback on the launch post: *"'0 false positives' is
+> encouraging but incomplete without corpus size, commit SHAs, labeled
+> negatives, and recall."* Correct. Here they are.
+
+## Corpus (pinned)
+
+Scanned with `mcp-vet` v0.4.0 (`node dist/cli.js <roots> --json`), all rules
+enabled, default confidence (`low`), on 2026-07-23:
+
+| Repo | Commit | Scanned root | Files | LOC |
+| --- | --- | --- | --- | --- |
+| [modelcontextprotocol/servers](https://github.com/modelcontextprotocol/servers) | `d31124c982401739917fd817c2a59db344529c16` | `src/` | 78 | 14,742 |
+| [modelcontextprotocol/typescript-sdk](https://github.com/modelcontextprotocol/typescript-sdk) | `1e1392e3f91583884fe82a0b4b91335875c3fba6` | `examples/` | 144 | 17,224 |
+| [modelcontextprotocol/python-sdk](https://github.com/modelcontextprotocol/python-sdk) | `3a6f2996cdd8358957479791e8b26198c07d6a75` | `examples/` | 225 | 12,013 |
+| **Total** | | | **447** | **43,979** |
+
+File counts are candidate files (`.ts/.tsx/.js/.mjs/.cjs/.py`) under the
+scanned roots, excluding `node_modules`.
+
+## Results
+
+**105 findings across 41 files** (TypeScript/JavaScript: 93, Python: 12).
+By confidence: 66 high, 38 medium, 1 low.
+
+| Pattern | Findings |
+| --- | --- |
+| `MCP_SESSION_ID` | 49 |
+| `LOGGING_CAP` | 17 |
+| `SAMPLING_CAP` | 16 |
+| `ROOTS_CAP` | 15 |
+| `INITIALIZE_HANDLER` | 4 |
+| `TASKS_LEGACY` | 2 |
+| `TASKS_RESULT_REMOVED` | 2 |
+
+### Labeling
+
+Every finding was manually reviewed against its source line:
+
+- **104 / 105 true positives** — real references to a removed or deprecated
+  protocol surface (session headers/ids, handshake registration, legacy task
+  methods, deprecated capability declarations and method strings).
+- **1 / 105 false positive (0.95%)** —
+  `stories/json_response/client.py:62` in the typescript-sdk examples:
+  `assert "mcp-session-id" not in response.headers`. That line is
+  *already-migrated* test code asserting the header is **absent**; flagging it
+  as "will break" is wrong. It is exactly what inline suppression
+  (`# mcp-vet-disable-line MCP_SESSION_ID`) is for, but we count it as a false
+  positive rather than defining it away. So the honest headline is
+  **"1 false positive in 44k LOC"**, not zero.
+
+Notes on reading the numbers:
+
+- Two occurrences on one line (e.g. `transport.sessionId && sessions.delete(transport.sessionId)`)
+  are reported as two findings — column-level dedup, not line-level.
+- Findings in test files (`__tests__/…`) are counted as true positives: a test
+  that registers `sampling/createMessage` breaks the same way production code
+  does.
+
+### Labeled negatives
+
+Files asserted to stay **clean** are part of the repo's test suite and run in CI:
+
+- `test/fixtures/clean/` — a full server written in the 2026-07-28 style
+  (per-request `_meta`, `sessionIdGenerator: undefined`, `-32602`).
+- `test/fixtures/negatives/` — "false friend" patterns: `sessionId` on plain
+  app-level objects, `-32002` inside strings/comments, capability-like words
+  with no capabilities context.
+- `test/fixtures/adversarial/caught/` — obfuscations the scanner **must**
+  catch: aliased imports (TS + Python), namespace-qualified SDK constants,
+  client transports resuming a `sessionId`.
+
+Additionally, in the corpus above, comment-only mentions (e.g. `Mcp-Session-Id`
+in a comment, `initialize` in prose) produced zero findings — the AST layer
+distinguishes executable tokens from comments by construction.
+
+## Recall — what the scanner is known to miss
+
+Static token analysis proves known patterns are **absent**; it cannot prove
+your server **speaks the new wire contract**. Recall is bounded by
+construction, and the misses are locked into the test suite
+(`test/fixtures/adversarial/missed/`, asserted to produce zero findings so any
+silent claim-inflation fails CI):
+
+- split/computed method strings — `'tasks' + '/list'`, `` `tasks/${op}` ``, f-strings
+- computed capability keys — `{ ['roo'+'ts']: {} }`
+- generated/loop-driven registration from string fragments
+- framework-adapter indirection (route tables built at runtime)
+- cross-module renames — a wrapper re-exporting an SDK constant under a new
+  name is flagged in the wrapper file, but a consumer importing only the new
+  name scans clean on its own
+
+There is no corpus-wide recall *percentage*: that would require a labeled set
+of every legacy usage in the wild, which nobody has. What we can say is: for
+the pattern shapes listed in the README, detection is exact; for the shapes
+above, it is zero, and the tool says so — pair the scan with runtime checks
+(`mcp-vet fixtures`) to cover the difference.
+
+## Reproducing
+
+```bash
+git clone --depth 1 https://github.com/modelcontextprotocol/servers
+git clone --depth 1 https://github.com/modelcontextprotocol/typescript-sdk
+git clone --depth 1 https://github.com/modelcontextprotocol/python-sdk
+# check out the pinned SHAs above, then:
+npx @booyaka/mcp-vet servers/src typescript-sdk/examples python-sdk/examples --json --no-files
+```

package/CHANGELOG.md

@@ -4,6 +4,91 @@ All notable changes to `mcp-vet` are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres
 to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.0]
+
+The runtime-probe release — `mcp-vet probe` connects to a *running* MCP server
+(stdio command or Streamable HTTP URL) and detects the two 2026-07-28 violation
+categories that only exist on the wire, not in source.
+
+### Added
+
+- **`mcp-vet probe [options] <url | command...>`** — a runtime prober with two
+  new violation categories, reported in the same JSON + SARIF formats as the
+  static scan:
+  - **`json-schema-dialect` (WARN)** — calls `tools/list` and inspects every
+    tool's `inputSchema`/`outputSchema` (SEP-2106 lifts both to full JSON
+    Schema 2020-12). Flags an explicit draft-04/-06/-07 (or 2019-09) `$schema`
+    at high confidence, and — when `$schema` is absent — draft-only keyword
+    forms (`definitions`, `$ref: "#/definitions/…"`, boolean
+    `exclusiveMinimum`/`exclusiveMaximum`, array-form `items`, schema-form
+    `dependencies`) at medium confidence. The walker recurses only into schema
+    positions, so a *property* named `definitions` is never a false positive,
+    and an explicit 2020-12 `$schema` is trusted.
+  - **`requires-initialize-handshake` (ERROR)** — with
+    `--spec-version 2026-07-28`, makes a stateless first request (no
+    `initialize`; protocolVersion/clientInfo/capabilities travel in `_meta`
+    per the RC) and flags a server that rejects it or hangs. Cross-checked:
+    only emitted when the classic 2025-11-25 handshake path *does* work, so a
+    dead server is an operational error (exit 2), never a false violation.
+- **`--spec-version <2025-11-25|2026-07-28>`** (default `2025-11-25`) selects
+  the revision to vet against; `--timeout <ms>` bounds each request and doubles
+  as the hang-detection window; `--json` / `--sarif [file]` / `--fail-on` /
+  `--quiet` / `--color` work as in the scan.
+- **Runtime rules in SARIF** — probe rules join the driver metadata when they
+  fire (`ERROR` → `error`, `WARN` → `warning`); the static-scan SARIF keeps its
+  stable 9-rule shape.
+- **`test/probe-fixtures/`** — minimal real MCP servers used by 18 new tests:
+  `server-draft07.mjs` (explicit + inferable draft-07 tools, a modern 2020-12
+  tool, and a property literally named `definitions`), `server-requires-init.mjs`
+  (rejects pre-initialize requests with `-32002`), `server-stateless.mjs`
+  (2026-07-28-native, requires `_meta`, no initialize), and `server-http.mjs`
+  (Streamable HTTP, sessionful *and* stateless modes).
+- Verified against the official `@modelcontextprotocol/server-everything@2026.7.4`:
+  it answers stateless requests, but all of its tool schemas still declare
+  draft-07 — `probe` reports 14 true `json-schema-dialect` findings.
+
+## [0.4.0]
+
+The community-feedback release — everything in it traces to reader comments on
+the launch post (issues #1–#6). Static analysis got sharper, and the tool now
+ships the runtime half it was honest about not covering.
+
+### Added
+
+- **Client-side session-ownership detection** (#1) — a client transport
+  constructed with a real `sessionId`/`session_id` and reads of
+  `transport.sessionId` are flagged (`MCP_SESSION_ID`, medium). The migrated
+  `sessionId: undefined` / `session_id=None` forms are recognized as benign.
+  Servers going stateless is only half the migration; clients that still behave
+  as if they own a session break too.
+- **Aliased-import resolution** (#2) — `import { InitializeRequestSchema as Init }`
+  (TS) and `from mcp.types import RootsCapability as RC` (Python) now flag both
+  the import line and the aliased usage sites. Python import lines surface
+  imported names even when only the alias is used later.
+- **Adversarial regression suite** (#3) — `test/fixtures/adversarial/` locks in
+  what the scanner catches (`caught/`) *and* what it is known to miss
+  (`missed/`, asserted zero findings): computed strings, computed capability
+  keys, generated registration, framework adapters, cross-module renames.
+- **`mcp-vet fixtures [dir]`** (#4) — emits nine protocol-level conformance
+  fixtures + `CHECKLIST.md`: `server/discover`, per-request `_meta`,
+  `Mcp-Method`/`Mcp-Name` routing headers (incl. mismatch rejection), stateless
+  auth, task-handle lifecycle, duplicate deliveries, retry on another instance,
+  `tools/list` cache invalidation, and downgrade/refusal behavior. Also exported
+  programmatically (`CONFORMANCE_FIXTURES`, `emitConformanceFixtures`).
+- **BENCHMARK.md** (#5) — the precision claim is now evidence: pinned corpus
+  SHAs, 447 files / ~44k LOC, every finding labeled (105 findings, 104 TP,
+  1 FP), labeled negatives, and an explicit recall discussion.
+
+### Changed
+
+- **Docs: spec-date semantics** (#6) — July 28 is a specification release, not
+  a remote kill switch; breakage appears when a client/server pair negotiates
+  the new revision. README and the post-scan notice now say so, and recommend
+  the dual-version (2025-11-25 + 2026-07-28) rollout test matrix.
+- The README "0 false positives" claim is replaced by the measured, reproducible
+  numbers in BENCHMARK.md (1 FP in 44k LOC — an already-migrated negative
+  assertion in test code).
+
 ## [0.3.0]
 
 The completeness release — full detection coverage, a real migration path, and a

package/README.md

@@ -15,10 +15,21 @@ npx @booyaka/mcp-vet .
 ```
 
 <p align="center">
-  <img src="https://raw.githubusercontent.com/Booyaka101/mcp-vet/main/assets/demo.svg" alt="mcp-vet scanning a server — BREAKING and DEPRECATED findings with before/after fixes and confidence tags" width="720">
+  <img src="https://raw.githubusercontent.com/Booyaka101/mcp-vet/main/assets/demo.png" alt="mcp-vet scanning a server — BREAKING and DEPRECATED findings with before/after fixes and confidence tags" width="720">
 </p>
 
-No account, no API key, no network calls — it parses your code locally (ts-morph for TS/JS, a bundled Python `ast` script for `.py`) and exits non-zero if it finds anything **BREAKING**, so you can drop it straight into CI.
+No account, no API key — the scan parses your code locally (ts-morph for TS/JS, a bundled Python `ast` script for `.py`), makes no network calls, and exits non-zero if it finds anything **BREAKING**, so you can drop it straight into CI. (The opt-in [`mcp-vet probe`](#vet-a-running-server-mcp-vet-probe) is the one command that talks to a server — and only the one you point it at.)
+
+## What actually happens on July 28
+
+**July 28 is a specification release date, not a switch that remotely disables your deployment.** Nothing reaches into running servers and turns them off. Breakage appears when a **client and server pair negotiates or requires the new revision** — a client that sends `2026-07-28`-style requests (per-request `_meta`, no handshake, routing headers) against a server that still expects `2025-11-25` semantics, or vice versa.
+
+Two practical consequences:
+
+- **Your rollout is a window, not a day.** Until every client you care about has moved, keep **both** revisions in your production test matrix: a `2025-11-25` path and a `2026-07-28` path. `mcp-vet fixtures` emits wire-level test fixtures for exactly this (see [Runtime conformance fixtures](#runtime-conformance-fixtures)).
+- **Silent acceptance is the worst failure mode.** A server that quietly processes an old-revision request under new semantics (or the reverse) corrupts behavior instead of failing loudly. Verify *refusal* behavior, not just the happy path.
+
+The scan tells you *what to change in your source*; the date tells you *when clients start expecting it*.
 
 ---
 
@@ -43,7 +54,7 @@ sse-polling.ts:107:13    BREAKING   MCP_SESSION_ID [medium]
 6 finding(s): 5 BREAKING, 1 DEPRECATED
 ```
 
-Note it catches the `sessionIdGenerator` session usage — the real signal in SDK-based servers, which usually never write the literal `Mcp-Session-Id` string. And it stays quiet where it should: the `Mcp-Session-Id` mentioned in a *comment*, the `initialize` in a comment in `dual-era.ts`, and the `sampling/createMessage` in `sampling.ts` (which appears only in comments and behind the `requestSampling()` helper) are all left alone. That precision — structural AST checks, not text matching — is what keeps the noise down on a real codebase: **6 findings, 0 false positives.**
+Note it catches the `sessionIdGenerator` session usage — the real signal in SDK-based servers, which usually never write the literal `Mcp-Session-Id` string. And it stays quiet where it should: the `Mcp-Session-Id` mentioned in a *comment*, the `initialize` in a comment in `dual-era.ts`, and the `sampling/createMessage` in `sampling.ts` (which appears only in comments and behind the `requestSampling()` helper) are all left alone. That precision — structural AST checks, not text matching — is what keeps the noise down on a real codebase: **6 findings, 0 false positives on these files.** (Across the full labeled corpus it's 104/105 true positives — see [BENCHMARK.md](./BENCHMARK.md).)
 
 ---
 
@@ -53,7 +64,7 @@ Note it catches the `sessionIdGenerator` session usage — the real signal in SD
 
 | ID | Pattern |
 | --- | --- |
-| `MCP_SESSION_ID` | `Mcp-Session-Id` header / `mcpSessionId` variable |
+| `MCP_SESSION_ID` | `Mcp-Session-Id` header / `mcpSessionId` variable / client-side session ownership (`sessionId` passed to or read from a client transport) |
 | `INITIALIZE_HANDLER` | `initialize` / `notifications/initialized` handler registration |
 | `ERROR_CODE_32002` | the numeric error code `-32002` |
 | `TASKS_LEGACY` | `tasks/get` · `tasks/update` · `tasks/cancel` legacy method strings |
@@ -73,7 +84,7 @@ Note it catches the `sessionIdGenerator` session usage — the real signal in SD
 Every finding carries a **confidence** so you can tune signal-to-noise with `--min-confidence`:
 
 - **high** — exact/deterministic match (session id, `-32002`, tasks methods), a structurally-verified capability (the `roots`/`sampling`/`logging` key is really *inside* a `capabilities` object), or an `initialize` string used as a method name (handler registration, `switch` case, or `req.method === 'initialize'`).
-- **medium** — a `roots`/`sampling`/`logging` key/string within 5 lines of a `capabilities` mention but not structurally verified.
+- **medium** — a `roots`/`sampling`/`logging` key/string within 5 lines of a `capabilities` mention but not structurally verified; a real `sessionIdGenerator`; client-side session ownership (`sessionId`/`session_id` passed to or read from a transport/client).
 - **low** — a bare `'initialize'` string with no registration context.
 
 ---
@@ -96,6 +107,19 @@ function handle(req) {
 }
 ```
 
+This cuts both ways — **client-side session ownership breaks too**, even against a server that scans clean. A lot of tool-reliability bugs only show up when the server is stateless but the client still behaves as if it owns a session:
+
+```ts
+// ❌ before — the client resumes a stored session
+const transport = new StreamableHTTPClientTransport(url, { sessionId: stored });
+persist(transport.sessionId);
+
+// ✅ after — stateless: no stored session id, full _meta on every request
+const transport = new StreamableHTTPClientTransport(url, { sessionId: undefined });
+```
+
+`mcp-vet` flags a client transport constructed with a real `sessionId`/`session_id` and reads of `transport.sessionId` (medium confidence). The migrated `sessionId: undefined` / `session_id=None` forms are recognized and left alone.
+
 ### 2. `initialize` / `notifications/initialized` — the handshake is removed
 
 > *"The `initialize`/`initialized` handshake is removed. The protocol version, client info, and client capabilities that used to be exchanged once at connection time now travel in `_meta` on every request."*
@@ -163,10 +187,70 @@ case 'tasks/list': return listTasks();
 - **The long-lived server→client SSE push channel is removed** — a server may only send requests to the client *while it is actively processing a client request*. Standing push streams / out-of-band notifications need rework.
 - **Streamable HTTP now requires `Mcp-Method` and `Mcp-Name` headers** that mirror the JSON-RPC body; servers must reject requests where headers and body disagree.
 - **Auth hardening** — validate the RFC 9207 `iss` parameter, declare OIDC `application_type` on Dynamic Client Registration, and bind tokens to the issuing authorization server.
-- **Tool schemas may now be full JSON Schema 2020-12** (`oneOf`/`anyOf`/`$ref`/conditionals); do not auto-dereference external `$ref` URIs.
+- **Tool schemas may now be full JSON Schema 2020-12** (`oneOf`/`anyOf`/`$ref`/conditionals); do not auto-dereference external `$ref` URIs. The *dialect* half of this — schemas still declaring or using draft-07 forms — **is** detectable at runtime: [`mcp-vet probe`](#vet-a-running-server-mcp-vet-probe) checks it against your live server.
 
 The CLI prints a one-line reminder of these after every scan.
 
+## Runtime conformance fixtures
+
+Static analysis proves known legacy patterns are *absent* from your source. Only wire-level tests prove your running server actually *speaks* the 2026-07-28 contract. `mcp-vet` ships both halves:
+
+```bash
+npx @booyaka/mcp-vet fixtures ./mcp-fixtures
+```
+
+writes nine ready-to-fire JSON fixtures plus a `CHECKLIST.md`, covering the runtime behaviors a linter cannot see:
+
+1. `server/discover` replaces the initialize handshake
+2. per-request `_meta` (protocolVersion, clientInfo, capabilities) — including explicit refusal when `_meta` is missing
+3. `Mcp-Method` / `Mcp-Name` routing headers, including the header/body-mismatch rejection case
+4. stateless auth context (no session-bound token cache)
+5. task-handle lifecycle: creation, `tasks/get` polling, resume on another instance, `tasks/list` and `tasks/result` returning method-not-found
+6. duplicate request delivery (idempotency under retries)
+7. retry against a different server instance (no sticky in-memory state)
+8. `tools/list` cache invalidation
+9. downgrade/refusal: old-revision requests get an explicit error, never silent acceptance under the wrong semantics
+
+Each fixture is a plain JSON description (`send` headers + JSON-RPC body, `expect` notes) you can replay with curl, supertest, pytest + httpx, or any HTTP harness. The checklist also spells out the **dual-version rollout matrix** — run both `2025-11-25` and `2026-07-28` paths until your clients have all moved — and a **client-side assumptions** list (session resume, per-request `_meta`, retries landing on other instances, `tools/list` revalidation).
+
+## Vet a running server (`mcp-vet probe`)
+
+Where the scan reads your *source*, `probe` talks to your *running server* over the wire — stdio (a command it spawns) or Streamable HTTP (a URL) — and checks the two 2026-07-28 violations that only exist at runtime:
+
+| ID | Severity | What it checks |
+| --- | --- | --- |
+| `json-schema-dialect` | 🟡 WARN | calls `tools/list` and inspects every tool's `inputSchema`/`outputSchema` for a pre-2020-12 JSON Schema dialect ([SEP-2106](https://modelcontextprotocol.io/seps/2106-json-schema-2020-12)) — an explicit draft-04/-06/-07 `$schema` (**high** confidence), or no `$schema` but draft-only keyword forms: `definitions` instead of `$defs`, `$ref: "#/definitions/…"`, boolean `exclusiveMinimum`/`exclusiveMaximum`, array-form `items` (**medium** confidence) |
+| `requires-initialize-handshake` | 🔴 ERROR | with `--spec-version 2026-07-28`: makes a **stateless first request** — no `initialize`, capabilities/clientInfo/protocolVersion in `_meta` per the RC — and flags a server that rejects it or hangs waiting for the removed handshake |
+
+```bash
+# vet the schemas of a stdio server (spawns the command; a lone .js file runs with Node)
+npx @booyaka/mcp-vet probe node ./dist/server.js
+
+# full 2026-07-28 readiness: stateless first contact + schema dialects
+npx @booyaka/mcp-vet probe --spec-version 2026-07-28 http://localhost:3000/mcp
+```
+
+```text
+mcp-vet probe — node ./dist/server.js · spec 2026-07-28 · stdio · 12 tool(s) listed
+  stateless probe: stateless tools/list was rejected: -32002 Server not initialized
+  fallback probe: initialize handshake + tools/list succeeded
+
+ERROR  requires-initialize-handshake [high]
+    The server rejected (or hung on) a stateless 2026-07-28-style first request ...
+WARN   json-schema-dialect [high]
+    tool "echo" inputSchema: $schema = http://json-schema.org/draft-07/schema# (draft-07)
+```
+
+The stateless verdict is **cross-checked** before it becomes a violation: `requires-initialize-handshake` is only emitted when the classic `2025-11-25` handshake path *does* work — a dead or non-MCP server is an operational error (exit 2), never a false violation. The dialect walker recurses only into schema positions (applicators like `properties`/`allOf`), so a *property* literally named `definitions` is never mistaken for the draft-07 keyword, and an explicit 2020-12 `$schema` declaration is trusted.
+
+Probe findings use the same report formats as the scan: `--json` (machine-readable array on stdout) and `--sarif [file]` (SARIF 2.1.0 — `ERROR` maps to `error`, `WARN` to `warning`), plus `--fail-on breaking|any|none` (default `breaking`: exit 1 only on `ERROR`), `--timeout <ms>` (default 8000, also the hang-detection window), `--quiet`, and `--color`/`--no-color`.
+
+Try it against the official reference server — the July 2026 `@modelcontextprotocol/server-everything` answers stateless requests, but its tool schemas still declare draft-07, and `probe` catches all of them:
+
+```bash
+npx @booyaka/mcp-vet probe --spec-version 2026-07-28 npx -y mcp-server-everything stdio
+```
+
 ## Usage
 
 ```bash
@@ -174,6 +258,8 @@ npx @booyaka/mcp-vet [paths...]        # scan directories and/or files (default:
 npx @booyaka/mcp-vet . --fix           # scan, and auto-apply the mechanical -32002 → -32602 rewrite
 npx @booyaka/mcp-vet ./src ./packages  # multiple roots
 npx @booyaka/mcp-vet server.py         # a single file
+npx @booyaka/mcp-vet fixtures ./dir    # write runtime conformance fixtures + checklist (default: ./mcp-vet-fixtures)
+npx @booyaka/mcp-vet probe <url|cmd>   # vet a RUNNING server's wire behavior (see section above)
 ```
 
 Globs `**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}` and `**/*.py`, skipping `node_modules`, `.git`, `__pycache__`, `dist`, and `build`.
@@ -325,15 +411,25 @@ It matches the ways real servers are actually written, not just raw method strin
 - **SDK schema-constant registration** — `server.setRequestHandler(InitializeRequestSchema, …)` (how the official SDKs register handlers) maps `InitializeRequestSchema`, `ListRootsRequestSchema`, `CreateMessageRequestSchema`, `SetLevelRequestSchema`, `ListTasksRequestSchema`, `GetTaskResultRequestSchema`, … to the right rule.
 - **SDK capability constructors** — the Python SDK's `ClientCapabilities(roots=RootsCapability())` is recognized structurally (high confidence), and `RootsCapability` / `SamplingCapability` / `LoggingCapability` are matched directly.
 - **`sessionIdGenerator`** — flagged only when it's a real generator, not the migrated `sessionIdGenerator: undefined`.
+- **aliased imports** — `import { InitializeRequestSchema as Init }` (TS) and `from mcp.types import RootsCapability as RC` (Python) are resolved back to their canonical names, so both the import line and the aliased usage sites are flagged. Namespace access (`types.InitializeRequestSchema`) is matched too.
+- **client-side session ownership** — a client transport constructed with a real `sessionId`/`session_id`, or a read of `transport.sessionId`; the migrated `sessionId: undefined` / `session_id=None` forms are recognized as benign.
 
-Validated against a broad corpus of real MCP servers (the official reference servers, TS + Python): **0 false positives**.
+**Measured, not vibes:** scanned against the official MCP reference servers and both SDK example suites at pinned commits — 447 files / ~44k LOC — every finding manually labeled: **105 findings, 104 true positives, 1 false positive**. Corpus, commit SHAs, per-pattern counts, labeled negatives, and the recall discussion are in [BENCHMARK.md](./BENCHMARK.md).
 
 ### Known limitations
 
+These are locked into the test suite as `test/fixtures/adversarial/missed/` — fixtures asserted to produce **zero** findings, so the claims below can't silently rot in either direction:
+
+- **Split/computed method strings** — `"tasks" + "/list"`, `` `tasks/${op}` ``, or `f"tasks/{x}"` are not reconstructed.
+- **Computed capability keys** — `{ ['roo'+'ts']: {} }` never exists as a single token.
+- **Generated/loop-driven registration** — method tables assembled from string fragments at runtime.
+- **Framework-adapter indirection** — routes built dynamically (`app.post('/rpc/' + ns + '/' + action, ...)`).
+- **Cross-module renames** — a wrapper module re-exporting an SDK constant under a new name is flagged *in the wrapper file*, but a consumer importing only the new name scans clean on its own. Scan whole projects, not single files.
 - **Python SDK decorator/method registration** — a handler wired purely as `@server.list_roots()` or a bare `session.list_roots()` call (with no capability declaration or method string in the file) is not matched, to avoid false positives on generic method names. The capability declaration in the same server is normally caught.
-- **Split/computed method strings** — `"tasks" + "/list"` or `f"tasks/{x}"` are not reconstructed.
 - The **regex fallback** (no Python interpreter) covers only the deterministic rules at reduced precision; install Python for full `.py` fidelity.
 
+This is the recall boundary of static analysis: it proves known patterns are *absent*, not that the server *speaks the new wire contract*. Cover the difference with the [runtime conformance fixtures](#runtime-conformance-fixtures).
+
 ## Programmatic API
 
 The scanner is usable as a library (typed) as well as a CLI — for editor extensions, custom CI steps, or migration harnesses:
@@ -369,10 +465,10 @@ Also exported: `renderJson` / `renderMarkdown` / `renderSarif`, `RULES`, and the
 ```bash
 npm install      # installs deps and builds (via prepare)
 npm run build    # tsc -> dist/ + copies the Python script
-npm test         # builds, then runs the Node.js built-in test runner (30 tests)
+npm test         # builds, then runs the Node.js built-in test runner (55 tests)
 ```
 
-Test fixtures live in `test/fixtures/` (dirty TS + Python servers, a `clean/` server with zero violations, `negatives/` true-negatives, a `confidence/` gradient, and `suppress/` cases).
+Test fixtures live in `test/fixtures/` (dirty TS + Python servers, a `clean/` server with zero violations, `negatives/` true-negatives, a `confidence/` gradient, and `suppress/` cases). Runtime-probe fixtures live in `test/probe-fixtures/` — minimal stdio + Streamable-HTTP MCP servers: one returning draft-07 schemas, one requiring the initialize handshake, and one fully stateless 2026-07-28-native.
 
 ## License
 

package/dist/autofix.d.ts

@@ -1,4 +1,4 @@
-import { Finding, PatternId } from './types';
+import { Finding, ViolationId } from './types';
 export interface FixPreview {
     file: string;
     line: number;
@@ -16,7 +16,7 @@ export interface FixOptions {
     /** Compute and return the rewrites without touching any files. */
     dryRun?: boolean;
 }
-export declare function isFixable(id: PatternId): boolean;
+export declare function isFixable(id: ViolationId): boolean;
 /**
  * Apply the safe mechanical fixes in place. Returns which findings were fixed so
  * the caller can drop them from the report and the exit-code calculation.

package/dist/autofix.js

@@ -110,9 +110,9 @@ function applyFixes(findings, opts = {}) {
             preview.push(...localPreview);
             continue;
         }
-        // Only count/return findings as fixed once the write actually succeeds — a
+        // Only count/return findings as fixed once the write actually succeeds — a
         // failed write (read-only file, EACCES) must not report the code as fixed.
-        const out = (hasBom ? '' : '') + lines.join('\n');
+        const out = (hasBom ? '' : '') + lines.join('\n');
         try {
             fs.writeFileSync(absPath, out, 'utf8');
         }