@booyaka/mcp-vet 0.10.2 → 0.10.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/BENCHMARK.md +40 -0
- package/CHANGELOG.md +544 -500
- package/README.md +1 -1
- package/dist/python/mcp_ast_scan.py +4 -0
- package/dist/rules.js +60 -9
- package/dist/ts-analyzer.js +4 -0
- package/package.json +71 -71
package/BENCHMARK.md
CHANGED
|
@@ -23,6 +23,46 @@ enabled, default confidence (`low`), on 2026-07-23:
|
|
|
23
23
|
File counts are candidate files (`.ts/.tsx/.js/.mjs/.cjs/.py`) under the
|
|
24
24
|
scanned roots, excluding `node_modules`.
|
|
25
25
|
|
|
26
|
+
## Validation against real-world hand-rolled OAuth clients (v0.10.3)
|
|
27
|
+
|
|
28
|
+
The pinned corpus above is almost entirely **SDK-routed** code, so it cannot
|
|
29
|
+
exercise the auth-hardening rules: the SDKs satisfy the requirements for you.
|
|
30
|
+
The rules exist for clients that do OAuth **by hand**, so that population was
|
|
31
|
+
sampled separately — three MCP OAuth clients found via GitHub code search
|
|
32
|
+
(`mcp-use/mcp-use`, `greeves89/AI-Employee`, `ogx-ai/ogx`), each read against
|
|
33
|
+
source.
|
|
34
|
+
|
|
35
|
+
| | v0.10.2 | v0.10.3 |
|
|
36
|
+
| --- | --- | --- |
|
|
37
|
+
| Findings | 4 | 2 |
|
|
38
|
+
| True positives | 2 (both on the **wrong line**) | **2, correctly anchored** |
|
|
39
|
+
| False positives | 2 | **0** |
|
|
40
|
+
|
|
41
|
+
The exercise found three defects the 447-file corpus never surfaced:
|
|
42
|
+
|
|
43
|
+
1. **Wrong anchor line.** `AUTH_ISS_UNVALIDATED` reported the DCR body's
|
|
44
|
+
`grant_types: ["authorization_code"]` at `ogx…:85`; the code is actually
|
|
45
|
+
redeemed at `:143`. `AUTH_DCR_NO_APPLICATION_TYPE` reported an unposted
|
|
46
|
+
local dict at `:58` instead of the posted body at `:83`. Both now anchor
|
|
47
|
+
correctly.
|
|
48
|
+
2. **Server-side DCR read as client-side.** `AI-Employee`'s registration
|
|
49
|
+
*endpoint* — which receives and stores an incoming client — was flagged
|
|
50
|
+
under rules that constrain clients.
|
|
51
|
+
3. **Request-body population read as a credential store.**
|
|
52
|
+
`data["client_secret"] = self.client_secret` at `ogx…:150`.
|
|
53
|
+
|
|
54
|
+
`ogx-ai/ogx` is the first real-world **`AUTH_ISS_UNVALIDATED` true positive**:
|
|
55
|
+
a hand-rolled `requests.post(token_endpoint, data={"grant_type":
|
|
56
|
+
"authorization_code", ...})` with no `iss` or `issuer` token anywhere in the
|
|
57
|
+
file — precisely the SEP-2468 MUST. The other two clients are `iss`-aware (12
|
|
58
|
+
and 9 mentions) and correctly stay clean.
|
|
59
|
+
|
|
60
|
+
Sample size is three files, so this is not a precision *rate* — it is evidence
|
|
61
|
+
that the rules work on the population they target, and that the SDK-heavy
|
|
62
|
+
corpus was systematically blind to a whole class of defect. Both wild false
|
|
63
|
+
positives are now regression fixtures (`negatives/server_dcr_handler.py`,
|
|
64
|
+
`auth/handrolled-client.py`).
|
|
65
|
+
|
|
26
66
|
## Results — v0.10.2 (21-rule engine, auth-hardening rules added)
|
|
27
67
|
|
|
28
68
|
Re-run 2026-08-01 with `mcp-vet` v0.10.2 on the SAME pinned corpus:
|