@booyaka/mcp-vet 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/BENCHMARK.md ADDED
@@ -0,0 +1,112 @@
1
+ # Benchmark: corpus, methodology, and honest limits
2
+
3
+ The README claims high precision on real MCP code. This file is the evidence
4
+ behind that claim — corpus, pinned commits, counts, labels, and what the
5
+ scanner is *known to miss* — so the claim is checkable rather than vibes.
6
+
7
+ > Prompted by community feedback on the launch post: *"'0 false positives' is
8
+ > encouraging but incomplete without corpus size, commit SHAs, labeled
9
+ > negatives, and recall."* Correct. Here they are.
10
+
11
+ ## Corpus (pinned)
12
+
13
+ Scanned with `mcp-vet` v0.4.0 (`node dist/cli.js <roots> --json`), all rules
14
+ enabled, default confidence (`low`), on 2026-07-23:
15
+
16
+ | Repo | Commit | Scanned root | Files | LOC |
17
+ | --- | --- | --- | --- | --- |
18
+ | [modelcontextprotocol/servers](https://github.com/modelcontextprotocol/servers) | `d31124c982401739917fd817c2a59db344529c16` | `src/` | 78 | 14,742 |
19
+ | [modelcontextprotocol/typescript-sdk](https://github.com/modelcontextprotocol/typescript-sdk) | `1e1392e3f91583884fe82a0b4b91335875c3fba6` | `examples/` | 144 | 17,224 |
20
+ | [modelcontextprotocol/python-sdk](https://github.com/modelcontextprotocol/python-sdk) | `3a6f2996cdd8358957479791e8b26198c07d6a75` | `examples/` | 225 | 12,013 |
21
+ | **Total** | | | **447** | **43,979** |
22
+
23
+ File counts are candidate files (`.ts/.tsx/.js/.mjs/.cjs/.py`) under the
24
+ scanned roots, excluding `node_modules`.
25
+
26
+ ## Results
27
+
28
+ **105 findings across 41 files** (TypeScript/JavaScript: 93, Python: 12).
29
+ By confidence: 66 high, 38 medium, 1 low.
30
+
31
+ | Pattern | Findings |
32
+ | --- | --- |
33
+ | `MCP_SESSION_ID` | 49 |
34
+ | `LOGGING_CAP` | 17 |
35
+ | `SAMPLING_CAP` | 16 |
36
+ | `ROOTS_CAP` | 15 |
37
+ | `INITIALIZE_HANDLER` | 4 |
38
+ | `TASKS_LEGACY` | 2 |
39
+ | `TASKS_RESULT_REMOVED` | 2 |
40
+
41
+ ### Labeling
42
+
43
+ Every finding was manually reviewed against its source line:
44
+
45
+ - **104 / 105 true positives** — real references to a removed or deprecated
46
+ protocol surface (session headers/ids, handshake registration, legacy task
47
+ methods, deprecated capability declarations and method strings).
48
+ - **1 / 105 false positive (0.95%)** —
49
+ `stories/json_response/client.py:62` in the typescript-sdk examples:
50
+ `assert "mcp-session-id" not in response.headers`. That line is
51
+ *already-migrated* test code asserting the header is **absent**; flagging it
52
+ as "will break" is wrong. It is exactly what inline suppression
53
+ (`# mcp-vet-disable-line MCP_SESSION_ID`) is for, but we count it as a false
54
+ positive rather than defining it away. So the honest headline is
55
+ **"1 false positive in 44k LOC"**, not zero.
56
+
57
+ Notes on reading the numbers:
58
+
59
+ - Two occurrences on one line (e.g. `transport.sessionId && sessions.delete(transport.sessionId)`)
60
+ are reported as two findings — column-level dedup, not line-level.
61
+ - Findings in test files (`__tests__/…`) are counted as true positives: a test
62
+ that registers `sampling/createMessage` breaks the same way production code
63
+ does.
64
+
65
+ ### Labeled negatives
66
+
67
+ Files asserted to stay **clean** are part of the repo's test suite and run in CI:
68
+
69
+ - `test/fixtures/clean/` — a full server written in the 2026-07-28 style
70
+ (per-request `_meta`, `sessionIdGenerator: undefined`, `-32602`).
71
+ - `test/fixtures/negatives/` — "false friend" patterns: `sessionId` on plain
72
+ app-level objects, `-32002` inside strings/comments, capability-like words
73
+ with no capabilities context.
74
+ - `test/fixtures/adversarial/caught/` — obfuscations the scanner **must**
75
+ catch: aliased imports (TS + Python), namespace-qualified SDK constants,
76
+ client transports resuming a `sessionId`.
77
+
78
+ Additionally, in the corpus above, comment-only mentions (e.g. `Mcp-Session-Id`
79
+ in a comment, `initialize` in prose) produced zero findings — the AST layer
80
+ distinguishes executable tokens from comments by construction.
81
+
82
+ ## Recall — what the scanner is known to miss
83
+
84
+ Static token analysis proves known patterns are **absent**; it cannot prove
85
+ your server **speaks the new wire contract**. Recall is bounded by
86
+ construction, and the misses are locked into the test suite
87
+ (`test/fixtures/adversarial/missed/`, asserted to produce zero findings so any
88
+ silent claim-inflation fails CI):
89
+
90
+ - split/computed method strings — `'tasks' + '/list'`, `` `tasks/${op}` ``, f-strings
91
+ - computed capability keys — `{ ['roo'+'ts']: {} }`
92
+ - generated/loop-driven registration from string fragments
93
+ - framework-adapter indirection (route tables built at runtime)
94
+ - cross-module renames — a wrapper re-exporting an SDK constant under a new
95
+ name is flagged in the wrapper file, but a consumer importing only the new
96
+ name scans clean on its own
97
+
98
+ There is no corpus-wide recall *percentage*: that would require a labeled set
99
+ of every legacy usage in the wild, which nobody has. What we can say is: for
100
+ the pattern shapes listed in the README, detection is exact; for the shapes
101
+ above, it is zero, and the tool says so — pair the scan with runtime checks
102
+ (`mcp-vet fixtures`) to cover the difference.
103
+
104
+ ## Reproducing
105
+
106
+ ```bash
107
+ git clone --depth 1 https://github.com/modelcontextprotocol/servers
108
+ git clone --depth 1 https://github.com/modelcontextprotocol/typescript-sdk
109
+ git clone --depth 1 https://github.com/modelcontextprotocol/python-sdk
110
+ # check out the pinned SHAs above, then:
111
+ npx @booyaka/mcp-vet servers/src typescript-sdk/examples python-sdk/examples --json --no-files
112
+ ```
package/CHANGELOG.md CHANGED
@@ -4,6 +4,106 @@ All notable changes to `mcp-vet` are documented here. The format is based on
4
4
  [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres
5
5
  to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
+ ## [0.4.0]
8
+
9
+ The community-feedback release — everything in it traces to reader comments on
10
+ the launch post (issues #1–#6). Static analysis got sharper, and the tool now
11
+ ships the runtime half it was honest about not covering.
12
+
13
+ ### Added
14
+
15
+ - **Client-side session-ownership detection** (#1) — a client transport
16
+ constructed with a real `sessionId`/`session_id` and reads of
17
+ `transport.sessionId` are flagged (`MCP_SESSION_ID`, medium). The migrated
18
+ `sessionId: undefined` / `session_id=None` forms are recognized as benign.
19
+ Servers going stateless is only half the migration; clients that still behave
20
+ as if they own a session break too.
21
+ - **Aliased-import resolution** (#2) — `import { InitializeRequestSchema as Init }`
22
+ (TS) and `from mcp.types import RootsCapability as RC` (Python) now flag both
23
+ the import line and the aliased usage sites. Python import lines surface
24
+ imported names even when only the alias is used later.
25
+ - **Adversarial regression suite** (#3) — `test/fixtures/adversarial/` locks in
26
+ what the scanner catches (`caught/`) *and* what it is known to miss
27
+ (`missed/`, asserted zero findings): computed strings, computed capability
28
+ keys, generated registration, framework adapters, cross-module renames.
29
+ - **`mcp-vet fixtures [dir]`** (#4) — emits nine protocol-level conformance
30
+ fixtures + `CHECKLIST.md`: `server/discover`, per-request `_meta`,
31
+ `Mcp-Method`/`Mcp-Name` routing headers (incl. mismatch rejection), stateless
32
+ auth, task-handle lifecycle, duplicate deliveries, retry on another instance,
33
+ `tools/list` cache invalidation, and downgrade/refusal behavior. Also exported
34
+ programmatically (`CONFORMANCE_FIXTURES`, `emitConformanceFixtures`).
35
+ - **BENCHMARK.md** (#5) — the precision claim is now evidence: pinned corpus
36
+ SHAs, 447 files / ~44k LOC, every finding labeled (105 findings, 104 TP,
37
+ 1 FP), labeled negatives, and an explicit recall discussion.
38
+
39
+ ### Changed
40
+
41
+ - **Docs: spec-date semantics** (#6) — July 28 is a specification release, not
42
+ a remote kill switch; breakage appears when a client/server pair negotiates
43
+ the new revision. README and the post-scan notice now say so, and recommend
44
+ the dual-version (2025-11-25 + 2026-07-28) rollout test matrix.
45
+ - The README "0 false positives" claim is replaced by the measured, reproducible
46
+ numbers in BENCHMARK.md (1 FP in 44k LOC — an already-migrated negative
47
+ assertion in test code).
48
+
49
+ ## [0.3.0]
50
+
51
+ The completeness release — full detection coverage, a real migration path, and a
52
+ programmatic API. Informed by a deep audit of the spec and the codebase.
53
+
54
+ ### Added
55
+
56
+ - **Two removed-method rules**: `TASKS_LIST_REMOVED` and `TASKS_RESULT_REMOVED`
57
+ (both BREAKING). `tasks/list` and `tasks/result` are removed on 2026-07-28
58
+ (SEP-2663); previously `tasks/list` was (incorrectly) treated as a non-issue.
59
+ - **Deprecated-capability *method* strings** — `roots/list`,
60
+ `notifications/roots/list_changed`, `sampling/createMessage`, `logging/setLevel`,
61
+ `notifications/message` are now flagged (SEP-2577 deprecates the methods, not just
62
+ the capability keys), catching servers that reference them without a literal
63
+ `capabilities` object nearby.
64
+ - **SDK schema-constant detection** — `server.setRequestHandler(InitializeRequestSchema, …)`
65
+ and friends are how real SDK servers register handlers; these are now mapped to the
66
+ right rule (previously only bare string literals matched — a major false negative).
67
+ - **SDK capability-constructor detection** — the Python SDK's
68
+ `ClientCapabilities(roots=RootsCapability())` is now recognized structurally (high
69
+ confidence, not proximity-medium), and `RootsCapability`/`SamplingCapability`/
70
+ `LoggingCapability` are matched directly. (Found by validating against the official
71
+ reference servers — a broad real-corpus run measured **0 false positives**.)
72
+ - **`--fix --dry-run`** — preview the rewrites `--fix` would make without touching files.
73
+ - **`sessionIdGenerator`** detection (value-aware) — flagged only when it is a real
74
+ generator, not the migrated `sessionIdGenerator: undefined`.
75
+ - **`--fix`** — auto-applies the safe mechanical rewrites in place (`-32002` →
76
+ `-32602`). Fixed findings are dropped from the report and the exit code.
77
+ - **`--json`** — prints findings as a JSON array to stdout (pure JSON; notices go to
78
+ stderr), for piping in CI.
79
+ - **Programmatic API** — the package now exposes `main`/`types`/`exports` with
80
+ bundled `.d.ts`: `scan`, `applyFixes`, `renderJson`/`renderMarkdown`/`renderSarif`,
81
+ `RULES`, and the public types.
82
+ - **"Needs manual review" awareness** — changes that can't be found statically (SSE
83
+ push-channel removal, required `Mcp-Method`/`Mcp-Name` headers, auth hardening,
84
+ JSON Schema 2020-12 schemas) are documented and the CLI prints a reminder after
85
+ every scan.
86
+ - Config JSON Schema (`schema/mcpvetrc.schema.json`), a tag-triggered `release.yml`
87
+ with npm provenance, and README husky/pre-commit examples.
88
+
89
+ ### Fixed
90
+
91
+ - **`--fix` file corruption on Python (multibyte lines)** — the bundled `ast`
92
+ scanner emitted UTF-8 *byte* column offsets; on a line with non-ASCII characters
93
+ autofix could rewrite an unrelated `-32002` inside a string. Columns are now
94
+ character-accurate, and autofix requires an exact column match (no blind
95
+ `indexOf` fallback).
96
+ - **`--fix` false success** — findings were marked fixed even when the file write
97
+ failed; they are now only cleared after a successful write.
98
+ - **Config precedence** — a config-file `only` no longer silently cancels a CLI
99
+ `--disable`; `--disable` always applies on top.
100
+ - Negative numeric literals are anchored consistently at the `-` across the TS and
101
+ Python analyzers (correct SARIF regions and autofix positions).
102
+
103
+ ### Changed
104
+
105
+ - Rule count is now 9 (was 7); SARIF advertises all 9 rules.
106
+
7
107
  ## [0.2.0]
8
108
 
9
109
  The robustness and precision release. Everything below is covered by the test suite.
@@ -50,5 +150,6 @@ Initial release.
50
150
  shared rule engine, and four report formats (terminal, Markdown, JSON, GitHub
51
151
  Actions annotations).
52
152
 
153
+ [0.3.0]: https://github.com/Booyaka101/mcp-vet/releases/tag/v0.3.0
53
154
  [0.2.0]: https://github.com/Booyaka101/mcp-vet/releases/tag/v0.2.0
54
155
  [0.1.0]: https://github.com/Booyaka101/mcp-vet/releases/tag/v0.1.0
package/README.md CHANGED
@@ -14,8 +14,23 @@
14
14
  npx @booyaka/mcp-vet .
15
15
  ```
16
16
 
17
+ <p align="center">
18
+ <img src="https://raw.githubusercontent.com/Booyaka101/mcp-vet/main/assets/demo.png" alt="mcp-vet scanning a server — BREAKING and DEPRECATED findings with before/after fixes and confidence tags" width="720">
19
+ </p>
20
+
17
21
  No account, no API key, no network calls — it parses your code locally (ts-morph for TS/JS, a bundled Python `ast` script for `.py`) and exits non-zero if it finds anything **BREAKING**, so you can drop it straight into CI.
18
22
 
23
+ ## What actually happens on July 28
24
+
25
+ **July 28 is a specification release date, not a switch that remotely disables your deployment.** Nothing reaches into running servers and turns them off. Breakage appears when a **client and server pair negotiates or requires the new revision** — a client that sends `2026-07-28`-style requests (per-request `_meta`, no handshake, routing headers) against a server that still expects `2025-11-25` semantics, or vice versa.
26
+
27
+ Two practical consequences:
28
+
29
+ - **Your rollout is a window, not a day.** Until every client you care about has moved, keep **both** revisions in your production test matrix: a `2025-11-25` path and a `2026-07-28` path. `mcp-vet fixtures` emits wire-level test fixtures for exactly this (see [Runtime conformance fixtures](#runtime-conformance-fixtures)).
30
+ - **Silent acceptance is the worst failure mode.** A server that quietly processes an old-revision request under new semantics (or the reverse) corrupts behavior instead of failing loudly. Verify *refusal* behavior, not just the happy path.
31
+
32
+ The scan tells you *what to change in your source*; the date tells you *when clients start expecting it*.
33
+
19
34
  ---
20
35
 
21
36
  ## Real-world example
@@ -25,17 +40,21 @@ Pointed at the [official MCP TypeScript SDK's own example servers](https://githu
25
40
  ```text
26
41
  legacy-routing.ts:36:29 BREAKING MCP_SESSION_ID [high]
27
42
  const sid = req.headers['mcp-session-id'] as string | undefined;
43
+ legacy-routing.ts:41:13 BREAKING MCP_SESSION_ID [medium]
44
+ sessionIdGenerator: () => randomUUID(),
28
45
  legacy-routing.ts:70:26 BREAKING MCP_SESSION_ID [high]
29
46
  exposedHeaders: ['Mcp-Session-Id', 'WWW-Authenticate', ...]
30
47
  sse-polling.ts:34:29 DEPRECATED LOGGING_CAP [high]
31
48
  capabilities: { logging: {} }
32
49
  sse-polling.ts:102:29 BREAKING MCP_SESSION_ID [high]
33
50
  const sid = req.headers['mcp-session-id'] as string | undefined;
51
+ sse-polling.ts:107:13 BREAKING MCP_SESSION_ID [medium]
52
+ sessionIdGenerator: () => randomUUID(),
34
53
 
35
- 4 finding(s): 3 BREAKING, 1 DEPRECATED
54
+ 6 finding(s): 5 BREAKING, 1 DEPRECATED
36
55
  ```
37
56
 
38
- And it stays quiet where it should — the `initialize` mentioned in a *comment* in `dual-era.ts`, and the `sampling/createMessage` **method** in `sampling.ts`, are not flagged, because only the `sampling` **capability declaration** is deprecated, not the method. That precision (structural AST checks, not text matching) is what keeps the noise down on a real codebase.
57
+ Note it catches the `sessionIdGenerator` session usage — the real signal in SDK-based servers, which usually never write the literal `Mcp-Session-Id` string. And it stays quiet where it should: the `Mcp-Session-Id` mentioned in a *comment*, the `initialize` in a comment in `dual-era.ts`, and the `sampling/createMessage` in `sampling.ts` (which appears only in comments and behind the `requestSampling()` helper) are all left alone. That precision — structural AST checks, not text matching — is what keeps the noise down on a real codebase: **6 findings, 0 false positives on these files.** (Across the full labeled corpus it's 104/105 true positives — see [BENCHMARK.md](./BENCHMARK.md).)
39
58
 
40
59
  ---
41
60
 
@@ -45,10 +64,12 @@ And it stays quiet where it should — the `initialize` mentioned in a *comment*
45
64
 
46
65
  | ID | Pattern |
47
66
  | --- | --- |
48
- | `MCP_SESSION_ID` | `Mcp-Session-Id` header / `mcpSessionId` variable |
67
+ | `MCP_SESSION_ID` | `Mcp-Session-Id` header / `mcpSessionId` variable / client-side session ownership (`sessionId` passed to or read from a client transport) |
49
68
  | `INITIALIZE_HANDLER` | `initialize` / `notifications/initialized` handler registration |
50
69
  | `ERROR_CODE_32002` | the numeric error code `-32002` |
51
70
  | `TASKS_LEGACY` | `tasks/get` · `tasks/update` · `tasks/cancel` legacy method strings |
71
+ | `TASKS_LIST_REMOVED` | `tasks/list` — removed entirely (no replacement listing method) |
72
+ | `TASKS_RESULT_REMOVED` | `tasks/result` — removed; poll with `tasks/get` instead (SEP-2663) |
52
73
 
53
74
  ### 🟡 DEPRECATED (warns only — exit code 0, 12-month grace period)
54
75
 
@@ -63,7 +84,7 @@ And it stays quiet where it should — the `initialize` mentioned in a *comment*
63
84
  Every finding carries a **confidence** so you can tune signal-to-noise with `--min-confidence`:
64
85
 
65
86
  - **high** — exact/deterministic match (session id, `-32002`, tasks methods), a structurally-verified capability (the `roots`/`sampling`/`logging` key is really *inside* a `capabilities` object), or an `initialize` string used as a method name (handler registration, `switch` case, or `req.method === 'initialize'`).
66
- - **medium** — a `roots`/`sampling`/`logging` key/string within 5 lines of a `capabilities` mention but not structurally verified.
87
+ - **medium** — a `roots`/`sampling`/`logging` key/string within 5 lines of a `capabilities` mention but not structurally verified; a real `sessionIdGenerator`; client-side session ownership (`sessionId`/`session_id` passed to or read from a transport/client).
67
88
  - **low** — a bare `'initialize'` string with no registration context.
68
89
 
69
90
  ---
@@ -86,6 +107,19 @@ function handle(req) {
86
107
  }
87
108
  ```
88
109
 
110
+ This cuts both ways — **client-side session ownership breaks too**, even against a server that scans clean. A lot of tool-reliability bugs only show up when the server is stateless but the client still behaves as if it owns a session:
111
+
112
+ ```ts
113
+ // ❌ before — the client resumes a stored session
114
+ const transport = new StreamableHTTPClientTransport(url, { sessionId: stored });
115
+ persist(transport.sessionId);
116
+
117
+ // ✅ after — stateless: no stored session id, full _meta on every request
118
+ const transport = new StreamableHTTPClientTransport(url, { sessionId: undefined });
119
+ ```
120
+
121
+ `mcp-vet` flags a client transport constructed with a real `sessionId`/`session_id` and reads of `transport.sessionId` (medium confidence). The migrated `sessionId: undefined` / `session_id=None` forms are recognized and left alone.
122
+
89
123
  ### 2. `initialize` / `notifications/initialized` — the handshake is removed
90
124
 
91
125
  > *"The `initialize`/`initialized` handshake is removed. The protocol version, client info, and client capabilities that used to be exchanged once at connection time now travel in `_meta` on every request."*
@@ -113,6 +147,8 @@ return { error: { code: -32002, message: 'Resource not found' } };
113
147
  return { error: { code: -32602, message: 'Invalid params' } };
114
148
  ```
115
149
 
150
+ This one is purely mechanical, so `mcp-vet --fix` rewrites it for you in place.
151
+
116
152
  ### 4. Legacy Tasks methods — redesigned to a handle-based lifecycle
117
153
 
118
154
  > *"A server can answer `tools/call` with a task handle, and the client drives it with `tasks/get`, `tasks/update`, and `tasks/cancel`. Anyone who shipped against the `2025-11-25` experimental Tasks API will need to migrate to the new lifecycle."*
@@ -130,14 +166,61 @@ switch (method) {
130
166
  // the 2026-07-28 schema.
131
167
  ```
132
168
 
169
+ ### 5. `tasks/list` — removed entirely
170
+
171
+ > *"The `tasks/list` method is removed — it was unsafe once protocol-level sessions were gone. There is no replacement listing method."*
172
+
173
+ ```ts
174
+ // ❌ before
175
+ case 'tasks/list': return listTasks();
176
+
177
+ // ✅ after — there is nothing to enumerate server-side. A client tracks the
178
+ // task handles it got back from its own tools/call responses.
179
+ ```
180
+
133
181
  ---
134
182
 
183
+ ## Needs manual review (not statically detectable)
184
+
185
+ `mcp-vet` catches every 2026-07-28 change that has a concrete code-level signal (a header, a method string, an error code, a capability key). A few changes are real but **can't be found reliably by static analysis** — they're architectural or depend on runtime wiring. A clean scan is not a promise that these are handled, so check them by hand:
186
+
187
+ - **The long-lived server→client SSE push channel is removed** — a server may only send requests to the client *while it is actively processing a client request*. Standing push streams / out-of-band notifications need rework.
188
+ - **Streamable HTTP now requires `Mcp-Method` and `Mcp-Name` headers** that mirror the JSON-RPC body; servers must reject requests where headers and body disagree.
189
+ - **Auth hardening** — validate the RFC 9207 `iss` parameter, declare OIDC `application_type` on Dynamic Client Registration, and bind tokens to the issuing authorization server.
190
+ - **Tool schemas may now be full JSON Schema 2020-12** (`oneOf`/`anyOf`/`$ref`/conditionals); do not auto-dereference external `$ref` URIs.
191
+
192
+ The CLI prints a one-line reminder of these after every scan.
193
+
194
+ ## Runtime conformance fixtures
195
+
196
+ Static analysis proves known legacy patterns are *absent* from your source. Only wire-level tests prove your running server actually *speaks* the 2026-07-28 contract. `mcp-vet` ships both halves:
197
+
198
+ ```bash
199
+ npx @booyaka/mcp-vet fixtures ./mcp-fixtures
200
+ ```
201
+
202
+ writes nine ready-to-fire JSON fixtures plus a `CHECKLIST.md`, covering the runtime behaviors a linter cannot see:
203
+
204
+ 1. `server/discover` replaces the initialize handshake
205
+ 2. per-request `_meta` (protocolVersion, clientInfo, capabilities) — including explicit refusal when `_meta` is missing
206
+ 3. `Mcp-Method` / `Mcp-Name` routing headers, including the header/body-mismatch rejection case
207
+ 4. stateless auth context (no session-bound token cache)
208
+ 5. task-handle lifecycle: creation, `tasks/get` polling, resume on another instance, `tasks/list` and `tasks/result` returning method-not-found
209
+ 6. duplicate request delivery (idempotency under retries)
210
+ 7. retry against a different server instance (no sticky in-memory state)
211
+ 8. `tools/list` cache invalidation
212
+ 9. downgrade/refusal: old-revision requests get an explicit error, never silent acceptance under the wrong semantics
213
+
214
+ Each fixture is a plain JSON description (`send` headers + JSON-RPC body, `expect` notes) you can replay with curl, supertest, pytest + httpx, or any HTTP harness. The checklist also spells out the **dual-version rollout matrix** — run both `2025-11-25` and `2026-07-28` paths until your clients have all moved — and a **client-side assumptions** list (session resume, per-request `_meta`, retries landing on other instances, `tools/list` revalidation).
215
+
135
216
  ## Usage
136
217
 
137
218
  ```bash
138
219
  npx @booyaka/mcp-vet [paths...] # scan directories and/or files (default: current directory)
220
+ npx @booyaka/mcp-vet . --fix # scan, and auto-apply the mechanical -32002 → -32602 rewrite
139
221
  npx @booyaka/mcp-vet ./src ./packages # multiple roots
140
222
  npx @booyaka/mcp-vet server.py # a single file
223
+ npx @booyaka/mcp-vet fixtures ./dir # write runtime conformance fixtures + checklist (default: ./mcp-vet-fixtures)
141
224
  ```
142
225
 
143
226
  Globs `**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}` and `**/*.py`, skipping `node_modules`, `.git`, `__pycache__`, `dist`, and `build`.
@@ -153,6 +236,9 @@ Globs `**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}` and `**/*.py`, skipping `node_modul
153
236
  | `--only <ids>` | only run these pattern ids (comma/space separated) |
154
237
  | `--disable <ids>` | skip these pattern ids |
155
238
  | `--fail-on <level>` | non-zero exit on `breaking` (default), `any`, or `none` |
239
+ | `--fix` | auto-apply the safe mechanical fixes in place (currently `-32002` → `-32602`) |
240
+ | `--dry-run` | with `--fix`: print the rewrites that would be made, without changing files |
241
+ | `--json` | print findings as a JSON array to stdout (pure JSON — notices go to stderr) |
156
242
  | `--min-confidence <level>` | report only findings at/above `high`, `medium`, or `low` (default) |
157
243
  | `--ignore <glob>` | ignore paths matching a gitignore-style glob (repeatable) |
158
244
  | `--max-file-size <kb>` | skip files larger than N KB (default 1536; `0` = no limit) |
@@ -237,13 +323,98 @@ To upload results to GitHub code scanning instead:
237
323
  with: { sarif_file: mcp-vet.sarif }
238
324
  ```
239
325
 
326
+ ### Local git hooks
327
+
328
+ Catch it before it reaches CI. With [husky](https://typicode.github.io/husky/) + [lint-staged](https://github.com/lint-staged/lint-staged):
329
+
330
+ ```json
331
+ // package.json
332
+ {
333
+ "lint-staged": {
334
+ "*.{ts,tsx,js,jsx,mjs,cjs,py}": "mcp-vet"
335
+ }
336
+ }
337
+ ```
338
+
339
+ Or with [pre-commit](https://pre-commit.com) (Python projects):
340
+
341
+ ```yaml
342
+ # .pre-commit-config.yaml
343
+ repos:
344
+ - repo: local
345
+ hooks:
346
+ - id: mcp-vet
347
+ name: mcp-vet
348
+ entry: npx @booyaka/mcp-vet
349
+ language: system
350
+ files: \.(ts|tsx|js|jsx|mjs|cjs|py)$
351
+ ```
352
+
353
+ ### Why there's no `--baseline`
354
+
355
+ Some linters let you "grandfather" existing findings so CI stays green. `mcp-vet` deliberately doesn't: this is a **one-time migration to a spec that ships on a fixed date**, and a suppressed finding is code that will break on July 28. The point is for the build to fail until it's actually fixed. For the rare intentional exception, use targeted [inline suppression](#suppressing-findings-inline) — an explicit, reviewable, per-line decision.
356
+
357
+ ### Large repositories
358
+
359
+ `mcp-vet` skips `node_modules`, `.git`, `dist`, `build`, and `__pycache__` by default, chunks the Python subprocess, and takes `--max-file-size`. On a big monorepo, scope the scan to the packages that ship MCP servers (`mcp-vet ./packages/server ./services/mcp`) and add `--ignore` globs for generated code.
360
+
240
361
  ---
241
362
 
242
363
  ## How it works
243
364
 
244
365
  - **TypeScript / JavaScript** — parsed with [`ts-morph`](https://ts-morph.com); the analyzer walks the AST and emits normalized tokens (string literals, signed numeric literals, identifiers, object keys) annotated with structural capability context and registration context.
245
- - **Python** — a bundled script (`dist/python/mcp_ast_scan.py`) runs `ast.parse` + a context-tracking walk in a subprocess (chunked for large repos) and emits the same token shape. When no interpreter exists, a regex fallback covers the deterministic rules.
246
- - A single rule engine applies all 7 rules to those tokens, so TS and Python behave identically. Findings are de-duplicated per (line, column, rule) and can be suppressed inline.
366
+ - **Python** — a bundled script (`dist/python/mcp_ast_scan.py`) runs `ast.parse` + a context-tracking walk in a subprocess (chunked for large repos) and emits the same token shape (with character-accurate columns). When no interpreter exists, a regex fallback covers the deterministic rules.
367
+ - A single rule engine applies all 9 rules to those tokens, so TS and Python behave identically. Findings are de-duplicated per (line, column, rule) and can be suppressed inline.
368
+
369
+ It matches the ways real servers are actually written, not just raw method strings:
370
+
371
+ - **literal method strings** — `'tasks/list'`, `'sampling/createMessage'`, `'logging/setLevel'`, …
372
+ - **SDK schema-constant registration** — `server.setRequestHandler(InitializeRequestSchema, …)` (how the official SDKs register handlers) maps `InitializeRequestSchema`, `ListRootsRequestSchema`, `CreateMessageRequestSchema`, `SetLevelRequestSchema`, `ListTasksRequestSchema`, `GetTaskResultRequestSchema`, … to the right rule.
373
+ - **SDK capability constructors** — the Python SDK's `ClientCapabilities(roots=RootsCapability())` is recognized structurally (high confidence), and `RootsCapability` / `SamplingCapability` / `LoggingCapability` are matched directly.
374
+ - **`sessionIdGenerator`** — flagged only when it's a real generator, not the migrated `sessionIdGenerator: undefined`.
375
+ - **aliased imports** — `import { InitializeRequestSchema as Init }` (TS) and `from mcp.types import RootsCapability as RC` (Python) are resolved back to their canonical names, so both the import line and the aliased usage sites are flagged. Namespace access (`types.InitializeRequestSchema`) is matched too.
376
+ - **client-side session ownership** — a client transport constructed with a real `sessionId`/`session_id`, or a read of `transport.sessionId`; the migrated `sessionId: undefined` / `session_id=None` forms are recognized as benign.
377
+
378
+ **Measured, not vibes:** scanned against the official MCP reference servers and both SDK example suites at pinned commits — 447 files / ~44k LOC — every finding manually labeled: **105 findings, 104 true positives, 1 false positive**. Corpus, commit SHAs, per-pattern counts, labeled negatives, and the recall discussion are in [BENCHMARK.md](./BENCHMARK.md).
379
+
380
+ ### Known limitations
381
+
382
+ These are locked into the test suite as `test/fixtures/adversarial/missed/` — fixtures asserted to produce **zero** findings, so the claims below can't silently rot in either direction:
383
+
384
+ - **Split/computed method strings** — `"tasks" + "/list"`, `` `tasks/${op}` ``, or `f"tasks/{x}"` are not reconstructed.
385
+ - **Computed capability keys** — `{ ['roo'+'ts']: {} }` never exists as a single token.
386
+ - **Generated/loop-driven registration** — method tables assembled from string fragments at runtime.
387
+ - **Framework-adapter indirection** — routes built dynamically (`app.post('/rpc/' + ns + '/' + action, ...)`).
388
+ - **Cross-module renames** — a wrapper module re-exporting an SDK constant under a new name is flagged *in the wrapper file*, but a consumer importing only the new name scans clean on its own. Scan whole projects, not single files.
389
+ - **Python SDK decorator/method registration** — a handler wired purely as `@server.list_roots()` or a bare `session.list_roots()` call (with no capability declaration or method string in the file) is not matched, to avoid false positives on generic method names. The capability declaration in the same server is normally caught.
390
+ - The **regex fallback** (no Python interpreter) covers only the deterministic rules at reduced precision; install Python for full `.py` fidelity.
391
+
392
+ This is the recall boundary of static analysis: it proves known patterns are *absent*, not that the server *speaks the new wire contract*. Cover the difference with the [runtime conformance fixtures](#runtime-conformance-fixtures).
393
+
394
+ ## Programmatic API
395
+
396
+ The scanner is usable as a library (typed) as well as a CLI — for editor extensions, custom CI steps, or migration harnesses:
397
+
398
+ ```ts
399
+ import { scan, ALL_PATTERN_IDS, IgnoreMatcher, applyFixes } from '@booyaka/mcp-vet';
400
+
401
+ const result = scan(['./src'], {
402
+ enabled: new Set(ALL_PATTERN_IDS),
403
+ ignore: new IgnoreMatcher([]),
404
+ maxFileSizeKb: 0,
405
+ pythonFallback: true,
406
+ minConfidence: 'low',
407
+ });
408
+
409
+ for (const f of result.findings) {
410
+ console.log(`${f.file}:${f.line} ${f.severity} ${f.patternId}`);
411
+ }
412
+
413
+ // Apply the safe mechanical fixes:
414
+ applyFixes(result.findings);
415
+ ```
416
+
417
+ Also exported: `renderJson` / `renderMarkdown` / `renderSarif`, `RULES`, and the `Finding` / `PatternId` / `Severity` / `Confidence` types.
247
418
 
248
419
  ## Requirements
249
420
 
@@ -255,7 +426,7 @@ To upload results to GitHub code scanning instead:
255
426
  ```bash
256
427
  npm install # installs deps and builds (via prepare)
257
428
  npm run build # tsc -> dist/ + copies the Python script
258
- npm test # builds, then runs the Node.js built-in test runner (18 tests)
429
+ npm test # builds, then runs the Node.js built-in test runner (30 tests)
259
430
  ```
260
431
 
261
432
  Test fixtures live in `test/fixtures/` (dirty TS + Python servers, a `clean/` server with zero violations, `negatives/` true-negatives, a `confidence/` gradient, and `suppress/` cases).
@@ -0,0 +1,26 @@
1
+ import { Finding, PatternId } from './types';
2
+ export interface FixPreview {
3
+ file: string;
4
+ line: number;
5
+ before: string;
6
+ after: string;
7
+ }
8
+ export interface FixResult {
9
+ fixedCount: number;
10
+ filesChanged: string[];
11
+ fixedFindings: Finding[];
12
+ /** Every planned rewrite (populated in both real and dry-run modes). */
13
+ preview: FixPreview[];
14
+ }
15
+ export interface FixOptions {
16
+ /** Compute and return the rewrites without touching any files. */
17
+ dryRun?: boolean;
18
+ }
19
+ export declare function isFixable(id: PatternId): boolean;
20
+ /**
21
+ * Apply the safe mechanical fixes in place. Returns which findings were fixed so
22
+ * the caller can drop them from the report and the exit-code calculation.
23
+ * Replacements are same-length, so multiple fixes on one line never shift each
24
+ * other's positions.
25
+ */
26
+ export declare function applyFixes(findings: Finding[], opts?: FixOptions): FixResult;
@@ -0,0 +1,130 @@
1
+ "use strict";
2
+ var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
3
+ if (k2 === undefined) k2 = k;
4
+ var desc = Object.getOwnPropertyDescriptor(m, k);
5
+ if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
6
+ desc = { enumerable: true, get: function() { return m[k]; } };
7
+ }
8
+ Object.defineProperty(o, k2, desc);
9
+ }) : (function(o, m, k, k2) {
10
+ if (k2 === undefined) k2 = k;
11
+ o[k2] = m[k];
12
+ }));
13
+ var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
14
+ Object.defineProperty(o, "default", { enumerable: true, value: v });
15
+ }) : function(o, v) {
16
+ o["default"] = v;
17
+ });
18
+ var __importStar = (this && this.__importStar) || (function () {
19
+ var ownKeys = function(o) {
20
+ ownKeys = Object.getOwnPropertyNames || function (o) {
21
+ var ar = [];
22
+ for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
23
+ return ar;
24
+ };
25
+ return ownKeys(o);
26
+ };
27
+ return function (mod) {
28
+ if (mod && mod.__esModule) return mod;
29
+ var result = {};
30
+ if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
31
+ __setModuleDefault(result, mod);
32
+ return result;
33
+ };
34
+ })();
35
+ Object.defineProperty(exports, "__esModule", { value: true });
36
+ exports.isFixable = isFixable;
37
+ exports.applyFixes = applyFixes;
38
+ const fs = __importStar(require("node:fs"));
39
+ /**
40
+ * Rules whose fix is a safe, purely mechanical text substitution. Only the
41
+ * resource-not-found error code qualifies: -32002 -> -32602 is a same-length
42
+ * swap with no semantic ambiguity. Everything else (removed handshake, removed
43
+ * sessions, tasks/list removal, capability deprecations) requires human
44
+ * judgement and is deliberately NOT auto-fixed.
45
+ */
46
+ const FIXABLE = new Set(['ERROR_CODE_32002']);
47
+ function isFixable(id) {
48
+ return FIXABLE.has(id);
49
+ }
50
+ /**
51
+ * Apply the safe mechanical fixes in place. Returns which findings were fixed so
52
+ * the caller can drop them from the report and the exit-code calculation.
53
+ * Replacements are same-length, so multiple fixes on one line never shift each
54
+ * other's positions.
55
+ */
56
+ function applyFixes(findings, opts = {}) {
57
+ const byFile = new Map();
58
+ for (const f of findings) {
59
+ if (!isFixable(f.patternId) || !f.absPath)
60
+ continue;
61
+ const list = byFile.get(f.absPath) ?? [];
62
+ list.push(f);
63
+ byFile.set(f.absPath, list);
64
+ }
65
+ const filesChanged = [];
66
+ const fixedFindings = [];
67
+ const preview = [];
68
+ let fixedCount = 0;
69
+ for (const [absPath, fileFindings] of byFile) {
70
+ let raw;
71
+ try {
72
+ raw = fs.readFileSync(absPath, 'utf8');
73
+ }
74
+ catch {
75
+ continue;
76
+ }
77
+ const hasBom = raw.charCodeAt(0) === 0xfeff;
78
+ const text = hasBom ? raw.slice(1) : raw;
79
+ // Split on \n only; any \r stays attached to the line content, so CRLF
80
+ // endings are preserved verbatim on rejoin.
81
+ const lines = text.split('\n');
82
+ const applied = [];
83
+ const localPreview = [];
84
+ for (const f of fileFindings) {
85
+ const idx = f.line - 1;
86
+ if (idx < 0 || idx >= lines.length)
87
+ continue;
88
+ const L = lines[idx];
89
+ const col = (f.column ?? 0) - 1; // anchored at the '-' (both analyzers), else the first digit
90
+ // Require an exact column match. We deliberately do NOT fall back to a
91
+ // blind indexOf: a mislocated column (e.g. a skewed offset) could otherwise
92
+ // rewrite an unrelated `-32002` inside a string or comment and corrupt it.
93
+ let next = null;
94
+ if (col >= 0 && L.startsWith('-32002', col)) {
95
+ next = L.slice(0, col) + '-32602' + L.slice(col + 6);
96
+ }
97
+ else if (col >= 0 && L.startsWith('32002', col)) {
98
+ next = L.slice(0, col) + '32602' + L.slice(col + 5);
99
+ }
100
+ if (next !== null && next !== L) {
101
+ localPreview.push({ file: f.file, line: f.line, before: L.replace(/\r$/, ''), after: next.replace(/\r$/, '') });
102
+ lines[idx] = next;
103
+ applied.push(f);
104
+ }
105
+ }
106
+ if (applied.length === 0)
107
+ continue;
108
+ if (opts.dryRun) {
109
+ // Report what WOULD change; touch nothing.
110
+ preview.push(...localPreview);
111
+ continue;
112
+ }
113
+ // Only count/return findings as fixed once the write actually succeeds — a
114
+ // failed write (read-only file, EACCES) must not report the code as fixed.
115
+ const out = (hasBom ? '' : '') + lines.join('\n');
116
+ try {
117
+ fs.writeFileSync(absPath, out, 'utf8');
118
+ }
119
+ catch {
120
+ continue; // nothing fixed for this file
121
+ }
122
+ filesChanged.push(absPath);
123
+ preview.push(...localPreview);
124
+ for (const f of applied) {
125
+ fixedFindings.push(f);
126
+ fixedCount++;
127
+ }
128
+ }
129
+ return { fixedCount, filesChanged, fixedFindings, preview };
130
+ }