bug-bounty-intelligence-mcp 1.0.2 → 1.0.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/BENCHMARK.md +174 -160
- package/LICENSE +21 -0
- package/METHODOLOGY.md +94 -94
- package/README.md +70 -13
- package/index.mjs +30 -12
- package/package.json +30 -8
- package/vulnerability-acceptance-rates.json +97 -0
package/BENCHMARK.md
CHANGED
|
@@ -1,160 +1,174 @@
|
|
|
1
|
-
# Benchmark: Bug Bounty Intelligence vs Slither
|
|
2
|
-
|
|
3
|
-
**Target protocol:** 3FLabs/grunt — leveraged strategies for on-chain funds
|
|
4
|
-
**Protocol type:** ERC-4626/ERC-7540 vaults + Morpho lending + flash loan requests
|
|
5
|
-
**Scope:** `src/` (218 Solidity files)
|
|
6
|
-
**Slither version:** 0.10.x
|
|
7
|
-
**Date:** 2026-07-20
|
|
8
|
-
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
## TL;DR
|
|
12
|
-
|
|
13
|
-
| Metric | Slither | Bug Bounty Intelligence |
|
|
14
|
-
|--------|---------|------------------------|
|
|
15
|
-
| "High" findings reported | 27 | 0 |
|
|
16
|
-
| True positives (real bugs) | 0 | 0 |
|
|
17
|
-
| False positives | 27 (100%) | 0 |
|
|
18
|
-
| Out-of-scope library noise | 24 (89%) | 0 |
|
|
19
|
-
| Signals requiring human review | n/a | 17 |
|
|
20
|
-
| Reading time to triage all "Highs" | ~4 hours | ~15 min |
|
|
21
|
-
|
|
22
|
-
**Slither false positive rate on this codebase: 100%.**
|
|
23
|
-
|
|
24
|
-
---
|
|
25
|
-
|
|
26
|
-
## Test setup
|
|
27
|
-
|
|
28
|
-
We ran Slither on the full repo (including `lib/`), and Bug Bounty Intelligence on `src/` only. The scope for any real audit is `src/` — dependencies in `lib/` are battle-tested third-party code that has its own audit history.
|
|
29
|
-
|
|
30
|
-
---
|
|
31
|
-
|
|
32
|
-
## Slither's 27 "High" findings — breakdown
|
|
33
|
-
|
|
34
|
-
### Group A: 24 findings in `lib/solady/` (out of scope)
|
|
35
|
-
|
|
36
|
-
Slither flagged 24 High-severity issues, all in the `lib/solady/` dependency:
|
|
37
|
-
|
|
38
|
-
- 5x `incorrect-exp` — Solady uses XOR (`^`) for bitwise operations, not exponentiation. Slither confuses the two.
|
|
39
|
-
- 19x `incorrect-shift` — Solady uses `<<` / `>>` inside assembly blocks. Slither's detector fires on these but they are intentional and correct.
|
|
40
|
-
|
|
41
|
-
**Why these are not bugs:**
|
|
42
|
-
|
|
43
|
-
Solady is one of the most audited Solidity libraries in existence, used by hundreds of protocols. Its math implementations deliberately use low-level bit operations for gas efficiency. Slither's `incorrect-exp` and `incorrect-shift` detectors are known to produce false positives on Solady. Every human auditor knows to ignore these. Slither does not.
|
|
44
|
-
|
|
45
|
-
**Example:**
|
|
46
|
-
|
|
47
|
-
```
|
|
48
|
-
incorrect-shift | FixedPointMathLib.lnWad(int256) (lib/solady/src/utils/FixedPointMathLib.sol#269-341)
|
|
49
|
-
contains an incorrect shift operation: ...
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
This is in `lib/solady/`, a third-party library. It is out of scope for any 3FLabs audit.
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
2.
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
---
|
|
129
|
-
|
|
130
|
-
##
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
1
|
+
# Benchmark: Bug Bounty Intelligence vs Slither
|
|
2
|
+
|
|
3
|
+
**Target protocol:** 3FLabs/grunt — leveraged strategies for on-chain funds
|
|
4
|
+
**Protocol type:** ERC-4626/ERC-7540 vaults + Morpho lending + flash loan requests
|
|
5
|
+
**Scope:** `src/` (218 Solidity files)
|
|
6
|
+
**Slither version:** 0.10.x
|
|
7
|
+
**Date:** 2026-07-20
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## TL;DR
|
|
12
|
+
|
|
13
|
+
| Metric | Slither | Bug Bounty Intelligence |
|
|
14
|
+
|--------|---------|------------------------|
|
|
15
|
+
| "High" findings reported | 27 | 0 |
|
|
16
|
+
| True positives (real bugs) | 0 | 0 |
|
|
17
|
+
| False positives | 27 (100%) | 0 |
|
|
18
|
+
| Out-of-scope library noise | 24 (89%) | 0 |
|
|
19
|
+
| Signals requiring human review | n/a | 17 |
|
|
20
|
+
| Reading time to triage all "Highs" | ~4 hours | ~15 min |
|
|
21
|
+
|
|
22
|
+
**Slither false positive rate on this codebase: 100%.**
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## Test setup
|
|
27
|
+
|
|
28
|
+
We ran Slither on the full repo (including `lib/`), and Bug Bounty Intelligence on `src/` only. The scope for any real audit is `src/` — dependencies in `lib/` are battle-tested third-party code that has its own audit history.
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Slither's 27 "High" findings — breakdown
|
|
33
|
+
|
|
34
|
+
### Group A: 24 findings in `lib/solady/` (out of scope)
|
|
35
|
+
|
|
36
|
+
Slither flagged 24 High-severity issues, all in the `lib/solady/` dependency:
|
|
37
|
+
|
|
38
|
+
- 5x `incorrect-exp` — Solady uses XOR (`^`) for bitwise operations, not exponentiation. Slither confuses the two.
|
|
39
|
+
- 19x `incorrect-shift` — Solady uses `<<` / `>>` inside assembly blocks. Slither's detector fires on these but they are intentional and correct.
|
|
40
|
+
|
|
41
|
+
**Why these are not bugs:**
|
|
42
|
+
|
|
43
|
+
Solady is one of the most audited Solidity libraries in existence, used by hundreds of protocols. Its math implementations deliberately use low-level bit operations for gas efficiency. Slither's `incorrect-exp` and `incorrect-shift` detectors are known to produce false positives on Solady. Every human auditor knows to ignore these. Slither does not.
|
|
44
|
+
|
|
45
|
+
**Example:**
|
|
46
|
+
|
|
47
|
+
```
|
|
48
|
+
incorrect-shift | FixedPointMathLib.lnWad(int256) (lib/solady/src/utils/FixedPointMathLib.sol#269-341)
|
|
49
|
+
contains an incorrect shift operation: ...
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
This is in `lib/solady/`, a third-party library. It is out of scope for any 3FLabs audit.
|
|
53
|
+
|
|
54
|
+
**Update:** while triaging these, we noticed a related pattern on the same `lib/solady` code:
|
|
55
|
+
`divide-before-multiply` fired 38 times on `FixedPointMathLib`'s `fullMulDivUnchecked`, `cbrt`,
|
|
56
|
+
`invMod`, `rpow`, and `cbrtWad`. We initially described all 38 as the same false-positive class
|
|
57
|
+
(divided variable fully reassigned before the multiply) and reported it upstream —
|
|
58
|
+
[crytic/slither#3039](https://github.com/crytic/slither/issues/3039). That was an overclaim on our
|
|
59
|
+
part. A maintainer merged a fix for the specific reassignment pattern we described
|
|
60
|
+
([crytic/slither#3040](https://github.com/crytic/slither/pull/3040), not yet in a PyPI release as
|
|
61
|
+
of 2026-08-01), and re-testing against the merged code ourselves shows it correctly resolves 11 of
|
|
62
|
+
the 38 (down to 27 remaining on this file). The other 27 keep the divided variable in the same
|
|
63
|
+
expression rather than through a clean reassignment, which is a different pattern this fix
|
|
64
|
+
correctly leaves untouched — we haven't yet re-analyzed whether those are genuine false positives
|
|
65
|
+
too or something else. Filed as a lesson: verify a fix against the actual code before describing
|
|
66
|
+
it as resolving the full scope of what you reported.
|
|
67
|
+
|
|
68
|
+
### Group B: 3 findings in `src/` — all false positives
|
|
69
|
+
|
|
70
|
+
Slither flagged 3 uses of `safeTransferFrom(offer.maker, ...)` as "arbitrary from in transferFrom":
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
arbitrary-send-erc20 | Request.consume() uses arbitrary from in transferFrom:
|
|
74
|
+
_asset().safeTransferFrom(offer.maker, address(this), ptAmount)
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
**Why this is not a bug:**
|
|
78
|
+
|
|
79
|
+
`offer.maker` is not arbitrary. The offer is validated via EIP-712 signature before the transfer (`_validateOffer(offer, signature)` is called on line 421). The maker explicitly signed the offer, authorizing exactly this transfer of exactly this amount. Additionally, `consume()` is gated by `onlyOwnerOrRoles(_ROLE_CONSUMER)` — only trusted consumers can call it.
|
|
80
|
+
|
|
81
|
+
This is a correct and standard EIP-712 off-chain order pattern. Slither has no understanding of EIP-712 and cannot reason about signature-based authorization.
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## What Bug Bounty Intelligence flags instead
|
|
86
|
+
|
|
87
|
+
After filtering to `src/` and applying Al-Mizaan validation gates, we identified **17 signals requiring human review** — none rising to a confirmed finding:
|
|
88
|
+
|
|
89
|
+
| Category | Count | Description |
|
|
90
|
+
|----------|-------|-------------|
|
|
91
|
+
| `unsafe_keyword` | 15 | Explicit `forge-lint: disable-next-line(unsafe-typecast)` annotations — developer acknowledged unsafe casts. Require review for truncation risk. |
|
|
92
|
+
| `delegatecall` | 2 | `MorphoFlashLoanRequest.execute()` delegatecalls to whitelisted scripts. By design — whitelist check present. |
|
|
93
|
+
|
|
94
|
+
**What we do not flag:**
|
|
95
|
+
- Library code in `lib/` (out of scope)
|
|
96
|
+
- EIP-712 signed transfers (intentional pattern, not a vulnerability)
|
|
97
|
+
- Standard Solady bit operations (well-audited library behavior)
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
## Al-Mizaan filtering — why the 17 signals do not become findings
|
|
102
|
+
|
|
103
|
+
The 15 `unsafe_keyword` signals (explicit forge-lint disables) were put through Al-Mizaan's reachability chain:
|
|
104
|
+
|
|
105
|
+
1. **Code** — read the actual casts in `LibTokenController.sol`, `TokenController.sol`, and `PositionManagerLP.sol`
|
|
106
|
+
2. **Reachability** — are the truncated values reachable by an untrusted caller?
|
|
107
|
+
3. **Threat model** — who triggers these paths? In all cases, the entry points are role-gated (owner, ROLE_CONSUMER, ROLE_MANAGER)
|
|
108
|
+
4. **Invariant** — does the truncation break any accounting invariant?
|
|
109
|
+
|
|
110
|
+
Verdict: The typecasts are `uint256 → uint128` in share accounting. The values come from internal calculations that are capped by deposit limits. Overflow requires either extremely large TVL or a role-gated actor. **Trusted actor as sole trigger → ELIMINATE.**
|
|
111
|
+
|
|
112
|
+
The 2 `delegatecall` signals: `MorphoFlashLoanRequest` maintains an explicit `isScriptWhitelisted` mapping. Only the owner can whitelist. **Trusted actor (owner) as sole attacker → ELIMINATE.**
|
|
113
|
+
|
|
114
|
+
**Result: 0 confirmed findings.** This matches the expected difficulty level of a mature protocol with prior audits.
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
## Why Slither's false positive rate matters
|
|
119
|
+
|
|
120
|
+
A 100% false positive rate on High findings is not an edge case. It happens because:
|
|
121
|
+
|
|
122
|
+
1. **No scope awareness** — Slither scans everything, including dependencies. A real audit scopes to `src/` only.
|
|
123
|
+
2. **No protocol context** — Slither cannot distinguish between an arbitrary `transferFrom` and an EIP-712-authorized one.
|
|
124
|
+
3. **No library knowledge** — Solady's assembly patterns trigger false positives on multiple detectors.
|
|
125
|
+
|
|
126
|
+
For a developer running Slither for the first time, triaging 27 "High" findings (all false positives) takes hours. For a developer using Bug Bounty Intelligence, the 17 signals that remain after scope filtering take 15 minutes to review.
|
|
127
|
+
|
|
128
|
+
---
|
|
129
|
+
|
|
130
|
+
## When Slither IS useful
|
|
131
|
+
|
|
132
|
+
Slither is best used as a first-pass tool before human review, with explicit exclusions:
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
slither src/ --exclude incorrect-exp,incorrect-shift --filter-paths "lib/"
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
After filtering, Slither's Medium and Low findings become more useful starting points.
|
|
139
|
+
|
|
140
|
+
Bug Bounty Intelligence is designed to do this filtering for you, and adds protocol-context reasoning that static analysis cannot.
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
## Reproduce this benchmark
|
|
145
|
+
|
|
146
|
+
The 3FLabs/grunt codebase is public at [github.com/3FLabs/grunt](https://github.com/3FLabs/grunt).
|
|
147
|
+
|
|
148
|
+
To reproduce Slither's output:
|
|
149
|
+
```bash
|
|
150
|
+
git clone https://github.com/3FLabs/grunt
|
|
151
|
+
cd grunt
|
|
152
|
+
pip install slither-analyzer
|
|
153
|
+
slither . --json slither-output.json
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
To use Bug Bounty Intelligence:
|
|
157
|
+
```bash
|
|
158
|
+
npx -y bug-bounty-intelligence-mcp@latest
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Then in your MCP client:
|
|
162
|
+
```
|
|
163
|
+
scan_contract({ repo_url: "https://github.com/3FLabs/grunt", protocol_type: "LENDING" })
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
## About Bug Bounty Intelligence
|
|
169
|
+
|
|
170
|
+
Pattern-aware from a corpus of 27,681 submitted Sherlock/Code4rena findings across 105 contests; acceptance-rate numbers (used by `list_vulnerability_patterns`) are limited to the 1,032 findings across 10 contests that could be exact-reconciled against contest outcomes (461 accepted) — see METHODOLOGY.md. Uses the Al-Mizaan v3 framework — a 7-gate validation chain (Code, Reachability, Threat-Model, Invariant, Protocol-Intent, Impact, Proof) — to filter signals into confirmed findings.
|
|
171
|
+
|
|
172
|
+
**Cost:** $5 USDC on Base (eip155:8453) via x402.
|
|
173
|
+
**Install:** `npx -y bug-bounty-intelligence-mcp@latest`
|
|
174
|
+
**npm:** [npmjs.com/package/bug-bounty-intelligence-mcp](https://www.npmjs.com/package/bug-bounty-intelligence-mcp)
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Abdellah Ouadoudi
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/METHODOLOGY.md
CHANGED
|
@@ -1,94 +1,94 @@
|
|
|
1
|
-
# Methodology: acceptance-rate data behind `list_vulnerability_patterns`
|
|
2
|
-
|
|
3
|
-
**Last updated: 2026-07-21.**
|
|
4
|
-
|
|
5
|
-
## What this covers
|
|
6
|
-
|
|
7
|
-
`list_vulnerability_patterns` returns, per vulnerability class, what fraction of *submitted*
|
|
8
|
-
audit-competition findings were judged valid. This document explains exactly how those numbers
|
|
9
|
-
are computed, what they don't claim, and how to reproduce them from public data.
|
|
10
|
-
|
|
11
|
-
## Data source
|
|
12
|
-
|
|
13
|
-
Sherlock runs public GitHub repos for judging each audit contest
|
|
14
|
-
(`sherlock-audit/<contest>-judging`), where security researchers ("Watsons") open one GitHub
|
|
15
|
-
issue per submitted finding. 105 of these judging repos were crawled (27,681 raw submitted
|
|
16
|
-
findings total, 2024-02 through 2025-07).
|
|
17
|
-
|
|
18
|
-
## The problem: "was this accepted?" is not reliably answerable from GitHub alone
|
|
19
|
-
|
|
20
|
-
An early version of this pipeline inferred acceptance from GitHub issue state
|
|
21
|
-
(`OPEN` = accepted, `CLOSED` = rejected). That heuristic turned out to be wrong, in both
|
|
22
|
-
directions, depending on the contest — sometimes undercounting acceptance by an order of
|
|
23
|
-
magnitude, sometimes overcounting it. GitHub issue labels (`Reward`, `Non-Reward`, `Invalid`,
|
|
24
|
-
etc.) turned out to be applied inconsistently across contests too — some contests label every
|
|
25
|
-
issue, some label almost none.
|
|
26
|
-
|
|
27
|
-
## The fix: use Sherlock's own result structure, and only trust it when it reconciles
|
|
28
|
-
|
|
29
|
-
Sherlock's judging repos also contain, at the repo root, a folder per accepted finding
|
|
30
|
-
(`001-M/`, `002-H/`, etc. — number + severity), each containing one file per GitHub issue number
|
|
31
|
-
that was judged to be that finding (the primary write-up plus any duplicates), and a separate
|
|
32
|
-
`invalid/` folder for rejected issue numbers. This is Sherlock's own authoritative accounting,
|
|
33
|
-
not an inference.
|
|
34
|
-
|
|
35
|
-
We only trust this for a given contest if it **reconciles exactly**:
|
|
36
|
-
`accepted_issue_numbers + invalid_issue_numbers === total_submitted_issues`.
|
|
37
|
-
|
|
38
|
-
If a contest's folder structure doesn't exist, or doesn't reconcile exactly, that contest is
|
|
39
|
-
**excluded entirely** — not estimated, not approximated. Of 105 crawled contests:
|
|
40
|
-
|
|
41
|
-
- **10 reconcile exactly** and are used for `list_vulnerability_patterns`: 1,032 findings, 461
|
|
42
|
-
accepted.
|
|
43
|
-
- **95 do not** — either no accepted/invalid folder structure exists in that judging repo (a
|
|
44
|
-
likely explanation: some contests use a different Sherlock judging workflow that doesn't
|
|
45
|
-
generate this structure), or the counts don't sum to the known total (a likely explanation for
|
|
46
|
-
many of these: the original crawl capped issue retrieval at 500 per contest, so large contests'
|
|
47
|
-
true totals exceed what was captured — a fixable crawler limitation, not a Sherlock data
|
|
48
|
-
problem, but not yet re-verified).
|
|
49
|
-
|
|
50
|
-
## What the resulting numbers mean
|
|
51
|
-
|
|
52
|
-
For each pattern (keyword-matched against finding titles/bodies — e.g. "reentrancy",
|
|
53
|
-
"oracle manipulation"), the acceptance rate is: of all submissions whose issue number appears in
|
|
54
|
-
*any* accepted-finding folder (including duplicates of the same underlying bug), what fraction
|
|
55
|
-
of total tagged submissions that is. This measures **submission-level reliability** — "if
|
|
56
|
-
someone writes up and submits a claim of this bug class, how often does it hold up" — not the
|
|
57
|
-
number of distinct vulnerabilities found.
|
|
58
|
-
|
|
59
|
-
## What this does not claim
|
|
60
|
-
|
|
61
|
-
- Not a claim about real-world exploit frequency — this is pre-deployment audit-competition
|
|
62
|
-
data, not realized incidents.
|
|
63
|
-
- Not a claim about detection coverage — it describes submitted reports, not silent misses.
|
|
64
|
-
- The keyword-based pattern tagging is a simple substring matcher (see `detectPatterns()` in the
|
|
65
|
-
crawler scripts), not a semantic classifier — a manual audit found roughly 65% tagging
|
|
66
|
-
precision on a sample. Pattern labels are directional, not exact classifications.
|
|
67
|
-
- The 10-contest sample is small. Some pattern rows have thin n (e.g. `liquidation`, n=14) —
|
|
68
|
-
treat low-n rows as lower-confidence than high-n rows (e.g. `trusted-actor`, n=311).
|
|
69
|
-
|
|
70
|
-
## Reproducing this
|
|
71
|
-
|
|
72
|
-
Everything below uses only public data — no private API, no authentication required beyond
|
|
73
|
-
default `gh` CLI rate limits.
|
|
74
|
-
|
|
75
|
-
```bash
|
|
76
|
-
# 1. List Sherlock judging repos
|
|
77
|
-
gh repo list sherlock-audit --limit 1000 --json name | jq -r '.[] | select(.name | endswith("-judging")) | .name'
|
|
78
|
-
|
|
79
|
-
# 2. For a given contest, pull its full file tree and look for the NNN-severity / invalid folders
|
|
80
|
-
gh api repos/sherlock-audit/<contest>-judging/git/trees/main?recursive=1 \
|
|
81
|
-
| jq '[.tree[].path | select(test("^[0-9]{3}-[HM]/|^invalid/"))]'
|
|
82
|
-
|
|
83
|
-
# 3. Cross-check reconciliation against the contest's total issue count
|
|
84
|
-
gh api "repos/sherlock-audit/<contest>-judging/issues?state=all&per_page=100" | jq 'length'
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
A finding's issue number appearing under any `NNN-[HM]/` path = accepted. Under `invalid/` =
|
|
88
|
-
rejected. If `accepted + invalid !== total`, do not use that contest's data.
|
|
89
|
-
|
|
90
|
-
## Why we publish this instead of just the bigger, older number
|
|
91
|
-
|
|
92
|
-
An earlier draft of this tool advertised acceptance percentages computed from the full,
|
|
93
|
-
unverified 27,681-finding corpus. Independent review caught that those specific percentages
|
|
94
|
-
were unreliable. We would rather ship a smaller number we can prove than a bigger one we can't.
|
|
1
|
+
# Methodology: acceptance-rate data behind `list_vulnerability_patterns`
|
|
2
|
+
|
|
3
|
+
**Last updated: 2026-07-21.**
|
|
4
|
+
|
|
5
|
+
## What this covers
|
|
6
|
+
|
|
7
|
+
`list_vulnerability_patterns` returns, per vulnerability class, what fraction of *submitted*
|
|
8
|
+
audit-competition findings were judged valid. This document explains exactly how those numbers
|
|
9
|
+
are computed, what they don't claim, and how to reproduce them from public data.
|
|
10
|
+
|
|
11
|
+
## Data source
|
|
12
|
+
|
|
13
|
+
Sherlock runs public GitHub repos for judging each audit contest
|
|
14
|
+
(`sherlock-audit/<contest>-judging`), where security researchers ("Watsons") open one GitHub
|
|
15
|
+
issue per submitted finding. 105 of these judging repos were crawled (27,681 raw submitted
|
|
16
|
+
findings total, 2024-02 through 2025-07).
|
|
17
|
+
|
|
18
|
+
## The problem: "was this accepted?" is not reliably answerable from GitHub alone
|
|
19
|
+
|
|
20
|
+
An early version of this pipeline inferred acceptance from GitHub issue state
|
|
21
|
+
(`OPEN` = accepted, `CLOSED` = rejected). That heuristic turned out to be wrong, in both
|
|
22
|
+
directions, depending on the contest — sometimes undercounting acceptance by an order of
|
|
23
|
+
magnitude, sometimes overcounting it. GitHub issue labels (`Reward`, `Non-Reward`, `Invalid`,
|
|
24
|
+
etc.) turned out to be applied inconsistently across contests too — some contests label every
|
|
25
|
+
issue, some label almost none.
|
|
26
|
+
|
|
27
|
+
## The fix: use Sherlock's own result structure, and only trust it when it reconciles
|
|
28
|
+
|
|
29
|
+
Sherlock's judging repos also contain, at the repo root, a folder per accepted finding
|
|
30
|
+
(`001-M/`, `002-H/`, etc. — number + severity), each containing one file per GitHub issue number
|
|
31
|
+
that was judged to be that finding (the primary write-up plus any duplicates), and a separate
|
|
32
|
+
`invalid/` folder for rejected issue numbers. This is Sherlock's own authoritative accounting,
|
|
33
|
+
not an inference.
|
|
34
|
+
|
|
35
|
+
We only trust this for a given contest if it **reconciles exactly**:
|
|
36
|
+
`accepted_issue_numbers + invalid_issue_numbers === total_submitted_issues`.
|
|
37
|
+
|
|
38
|
+
If a contest's folder structure doesn't exist, or doesn't reconcile exactly, that contest is
|
|
39
|
+
**excluded entirely** — not estimated, not approximated. Of 105 crawled contests:
|
|
40
|
+
|
|
41
|
+
- **10 reconcile exactly** and are used for `list_vulnerability_patterns`: 1,032 findings, 461
|
|
42
|
+
accepted.
|
|
43
|
+
- **95 do not** — either no accepted/invalid folder structure exists in that judging repo (a
|
|
44
|
+
likely explanation: some contests use a different Sherlock judging workflow that doesn't
|
|
45
|
+
generate this structure), or the counts don't sum to the known total (a likely explanation for
|
|
46
|
+
many of these: the original crawl capped issue retrieval at 500 per contest, so large contests'
|
|
47
|
+
true totals exceed what was captured — a fixable crawler limitation, not a Sherlock data
|
|
48
|
+
problem, but not yet re-verified).
|
|
49
|
+
|
|
50
|
+
## What the resulting numbers mean
|
|
51
|
+
|
|
52
|
+
For each pattern (keyword-matched against finding titles/bodies — e.g. "reentrancy",
|
|
53
|
+
"oracle manipulation"), the acceptance rate is: of all submissions whose issue number appears in
|
|
54
|
+
*any* accepted-finding folder (including duplicates of the same underlying bug), what fraction
|
|
55
|
+
of total tagged submissions that is. This measures **submission-level reliability** — "if
|
|
56
|
+
someone writes up and submits a claim of this bug class, how often does it hold up" — not the
|
|
57
|
+
number of distinct vulnerabilities found.
|
|
58
|
+
|
|
59
|
+
## What this does not claim
|
|
60
|
+
|
|
61
|
+
- Not a claim about real-world exploit frequency — this is pre-deployment audit-competition
|
|
62
|
+
data, not realized incidents.
|
|
63
|
+
- Not a claim about detection coverage — it describes submitted reports, not silent misses.
|
|
64
|
+
- The keyword-based pattern tagging is a simple substring matcher (see `detectPatterns()` in the
|
|
65
|
+
crawler scripts), not a semantic classifier — a manual audit found roughly 65% tagging
|
|
66
|
+
precision on a sample. Pattern labels are directional, not exact classifications.
|
|
67
|
+
- The 10-contest sample is small. Some pattern rows have thin n (e.g. `liquidation`, n=14) —
|
|
68
|
+
treat low-n rows as lower-confidence than high-n rows (e.g. `trusted-actor`, n=311).
|
|
69
|
+
|
|
70
|
+
## Reproducing this
|
|
71
|
+
|
|
72
|
+
Everything below uses only public data — no private API, no authentication required beyond
|
|
73
|
+
default `gh` CLI rate limits.
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
# 1. List Sherlock judging repos
|
|
77
|
+
gh repo list sherlock-audit --limit 1000 --json name | jq -r '.[] | select(.name | endswith("-judging")) | .name'
|
|
78
|
+
|
|
79
|
+
# 2. For a given contest, pull its full file tree and look for the NNN-severity / invalid folders
|
|
80
|
+
gh api repos/sherlock-audit/<contest>-judging/git/trees/main?recursive=1 \
|
|
81
|
+
| jq '[.tree[].path | select(test("^[0-9]{3}-[HM]/|^invalid/"))]'
|
|
82
|
+
|
|
83
|
+
# 3. Cross-check reconciliation against the contest's total issue count
|
|
84
|
+
gh api "repos/sherlock-audit/<contest>-judging/issues?state=all&per_page=100" | jq 'length'
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
A finding's issue number appearing under any `NNN-[HM]/` path = accepted. Under `invalid/` =
|
|
88
|
+
rejected. If `accepted + invalid !== total`, do not use that contest's data.
|
|
89
|
+
|
|
90
|
+
## Why we publish this instead of just the bigger, older number
|
|
91
|
+
|
|
92
|
+
An earlier draft of this tool advertised acceptance percentages computed from the full,
|
|
93
|
+
unverified 27,681-finding corpus. Independent review caught that those specific percentages
|
|
94
|
+
were unreliable. We would rather ship a smaller number we can prove than a bigger one we can't.
|
package/README.md
CHANGED
|
@@ -1,27 +1,72 @@
|
|
|
1
|
-
# Bug Bounty Intelligence
|
|
1
|
+
# Bug Bounty Intelligence: MCP Server
|
|
2
2
|
|
|
3
3
|
AI-powered smart contract security analysis for AI agents and developers.
|
|
4
4
|
|
|
5
|
+
## Update, 24 October 2026, later the same day: version 2 of the scan
|
|
6
|
+
|
|
7
|
+
The defects described below were fixed and the new version is live. What changed: git submodules are fetched when they point to github.com, tests, mocks, scripts, interfaces and vendored libraries are left out, files are ranked by how likely they move funds and sent in at most six parts, every report states how much of the in-scope code was read, a scan without hypotheses is delivered as such and not as a normal report, hypotheses that name files or functions that are not in the code are dropped, and file links inside a repository are never followed. A free precheck at https://x402.tokenizen.nl/api/bug-intel/precheck?repo=owner/name says before you pay whether the scan counts any Solidity in your repository and roughly how much of it will be read.
|
|
8
|
+
|
|
9
|
+
The measurement of this version was pre-registered before it ran: [PREREGISTRATION-v2-2026-10-24.md](PREREGISTRATION-v2-2026-10-24.md). The frozen files are in [pipeline/v2](pipeline/v2).
|
|
10
|
+
|
|
11
|
+
Result, in [RESULTS-v2-2026-10-24.md](RESULTS-v2-2026-10-24.md): none of the 40 known High and Medium findings was found, 0 of 31 on the four contests of the first run and 0 of 9 on three new contests, while 29 percent of the in-scope code was read (8 to 100 percent per contest). 15 of the 40 known findings sit in a file that was read, and none of those was found. The scan produces hypotheses to review by hand. It has not been shown to find the issues that audit contests reward.
|
|
12
|
+
|
|
13
|
+
## Correction and limits (24 October 2026, before version 2)
|
|
14
|
+
|
|
15
|
+
Reading the production scan code on 24 October 2026 showed that earlier text on this page overstated what the paid scan does. What it does today:
|
|
16
|
+
|
|
17
|
+
- It sends the model at most about 22,000 characters of Solidity source per job, taken from up to 60 files in directory order, and it does not skip test or mock folders. On three recent audit-contest repositories (code-423n4/2025-11-merkl, 2025-11-sukukfi and 2026-04-monetrix) that is 3 to 7 percent of the Solidity source.
|
|
18
|
+
- The 7-gate filter is not part of the paid scan. It was taken out because the small model also eliminated real bugs. The report lists raw hypotheses with a self-reported confidence of 0.4 or higher, at most eight, and its disclaimer says so.
|
|
19
|
+
- The sample below ran on a repository that belongs to the corpus the framework was built from. It does not show how the scan does on projects it has not seen, and its label "CONFIRMED by Al-Mizaan validation" does not apply to the scan as it runs today.
|
|
20
|
+
- We have not measured how many real bugs the scan finds on contests it has not seen. A test on closed audit contests is pre-registered in [PREREGISTRATION-2026-10-24.md](PREREGISTRATION-2026-10-24.md). The baseline ran on 24 October 2026: 0 of 31 findings found, 2 hypotheses reported and neither matches, one job failed and two reports were empty. See [RESULTS-2026-10-24.md](RESULTS-2026-10-24.md). Until a changed version is pre-registered and measured, do not read this scan as something that finds bugs on projects it has not seen.
|
|
21
|
+
|
|
22
|
+
Official source: this repository and the payment address in the payment section below are the only official ones for Bug Bounty Intelligence. If a listing, a reply or a website gives a different payment address, it is not ours. Check the address against the one below before you pay.
|
|
23
|
+
|
|
5
24
|
**Drawn from a corpus of 27,681 submitted findings across 105 Sherlock and Code4rena contests.**
|
|
6
25
|
**Cost: $5 USDC on Base (eip155:8453) via x402.**
|
|
7
|
-
**Free tool: `list_vulnerability_patterns
|
|
26
|
+
**Free tool: `list_vulnerability_patterns`, no payment needed.**
|
|
8
27
|
|
|
9
28
|
`list_vulnerability_patterns`' acceptance-rate numbers are computed only from the subset where
|
|
10
|
-
results could be exactly reconciled against Sherlock's own published outcomes
|
|
29
|
+
results could be exactly reconciled against Sherlock's own published outcomes: 1,032 findings
|
|
11
30
|
across 10 contests. We'd rather show fewer, verified numbers than a bigger set we can't stand
|
|
12
31
|
behind. See [METHODOLOGY.md](METHODOLOGY.md) for exactly why, and how to reproduce it.
|
|
13
32
|
|
|
33
|
+
## Free dataset: no server, no MCP, no payment
|
|
34
|
+
|
|
35
|
+
[`vulnerability-acceptance-rates.json`](vulnerability-acceptance-rates.json) is the same data
|
|
36
|
+
`list_vulnerability_patterns` serves, as a static file. No API call, no account, no network
|
|
37
|
+
dependency at all. Download it, or copy it into your own tool's rule set. CC0 licensed, use it
|
|
38
|
+
however you want, credit optional.
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
curl -sO https://raw.githubusercontent.com/holistis/bug-bounty-intelligence-mcp/main/vulnerability-acceptance-rates.json
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
If you're building a local-first / offline security scanner and the paid `scan_contract` tool
|
|
45
|
+
below isn't a fit for your project's architecture, this file might still be useful on its own.
|
|
46
|
+
|
|
47
|
+
## Just want pattern search, nothing paid? Use 3ilm-mcp instead
|
|
48
|
+
|
|
49
|
+
[3ilm-mcp](https://github.com/holistis/3ilm-mcp) is a smaller, free-only MCP server built from
|
|
50
|
+
the same verified dataset. It has no paid tier at all, just pattern search
|
|
51
|
+
(`search_vulnerabilities`, `get_pattern_details`, `list_patterns`). Use this repo
|
|
52
|
+
(bug-bounty-intelligence-mcp) if you also want the paid `scan_contract` scan; use
|
|
53
|
+
3ilm-mcp if you only ever wanted the free lookup tools and prefer the lighter package.
|
|
54
|
+
|
|
55
|
+
Two more tools in the same Al-Mizaan toolkit, for the rest of the workflow: [al-mizaan-judge](https://github.com/holistis/al-mizaan-judge) is a local CLI that pressure-tests a candidate finding through the same 7-gate framework before you submit it, and [DeFi Signal API](https://github.com/holistis/defi-signal-api) exposes the same vuln-search/bug-intel routes as a pay-per-call HTTP API for anyone who wants this outside MCP.
|
|
56
|
+
|
|
14
57
|
## Benchmark: vs Slither
|
|
15
58
|
|
|
16
59
|
**[See the full benchmark →](BENCHMARK.md)**
|
|
17
60
|
|
|
18
61
|
On 3FLabs/grunt (ERC-4626 + Morpho vaults, 218 contracts): Slither reports 27 "High" findings. After analysis: 24 are in `lib/solady` (out-of-scope dependency with known Slither false positive patterns), and 3 are EIP-712 design patterns. False positive rate: **100%**.
|
|
19
62
|
|
|
20
|
-
Bug Bounty Intelligence
|
|
63
|
+
In that comparison Bug Bounty Intelligence was pointed at `src/` only. The paid scan now leaves out tests, mocks, scripts, interfaces and vendored libraries by rule. It does not run the 7-gate filter.
|
|
64
|
+
|
|
65
|
+
We also reported a related `divide-before-multiply` pattern on the same `lib/solady` code directly to the Slither project: [crytic/slither#3039](https://github.com/crytic/slither/issues/3039). A maintainer merged a fix for part of it ([crytic/slither#3040](https://github.com/crytic/slither/pull/3040)); see [BENCHMARK.md](BENCHMARK.md) for the honest scope of what that fix does and doesn't cover, we overclaimed in the original report.
|
|
21
66
|
|
|
22
67
|
## What it does
|
|
23
68
|
|
|
24
|
-
Submit a public GitHub repo containing Solidity smart contracts. Receive a
|
|
69
|
+
Submit a public GitHub repo containing Solidity smart contracts. Receive a report of AI-generated hypotheses, usually within the hour, that states how much of the code was read (see the update and the limits at the top of this page).
|
|
25
70
|
|
|
26
71
|
## Tools
|
|
27
72
|
|
|
@@ -33,6 +78,8 @@ Submit a public GitHub repo containing Solidity smart contracts. Receive a full
|
|
|
33
78
|
|
|
34
79
|
## Quick start (Claude Desktop / Claude Code)
|
|
35
80
|
|
|
81
|
+
Also listed in [PuPu](https://github.com/haoxiang-xu/PuPu)'s MCP store as of release v0.1.11 ([registration issue](https://github.com/haoxiang-xu/PuPu/issues/186)).
|
|
82
|
+
|
|
36
83
|
Add to your `claude_desktop_config.json`:
|
|
37
84
|
|
|
38
85
|
```json
|
|
@@ -54,13 +101,15 @@ npx -y bug-bounty-intelligence-mcp@latest
|
|
|
54
101
|
|
|
55
102
|
## Payment
|
|
56
103
|
|
|
57
|
-
If `scan_contract` returns `PAYMENT REQUIRED`, send exactly $5 USDC on Base to the address shown, then retry. Payment = acceptance of [service terms](https://
|
|
104
|
+
If `scan_contract` returns `PAYMENT REQUIRED`, send exactly $5 USDC on Base to the address shown, then retry. Payment = acceptance of [service terms](https://x402.tokenizen.nl/terms).
|
|
58
105
|
|
|
59
106
|
**x402 info:**
|
|
60
107
|
- Network: Base mainnet (eip155:8453)
|
|
61
108
|
- Amount: 5,000,000 (= 5 USDC, 6 decimals)
|
|
62
109
|
- Asset: 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913 (USDC)
|
|
63
|
-
- Wallet:
|
|
110
|
+
- Wallet: 0x015Eb036560216B3051339061FF68A207E0fe88f
|
|
111
|
+
|
|
112
|
+
The wallet address can change. Always pay the address `scan_contract` actually returns in its `PAYMENT REQUIRED` response, not a copy of this README. This document is updated when the address changes, but the live tool response is the source of truth.
|
|
64
113
|
|
|
65
114
|
## Example session
|
|
66
115
|
|
|
@@ -82,7 +131,7 @@ PAYMENT REQUIRED to start the scan.
|
|
|
82
131
|
|
|
83
132
|
Amount: 5.00 USDC
|
|
84
133
|
Network: Base mainnet (eip155:8453)
|
|
85
|
-
Pay to:
|
|
134
|
+
Pay to: 0x015Eb036560216B3051339061FF68A207E0fe88f
|
|
86
135
|
Asset: USDC — 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
|
|
87
136
|
|
|
88
137
|
After paying, retry scan_contract with the same repo_url.
|
|
@@ -124,7 +173,7 @@ Confidence: 0.90 (CONFIRMED by Al-Mizaan validation)
|
|
|
124
173
|
|
|
125
174
|
## About the analysis
|
|
126
175
|
|
|
127
|
-
The Al-Mizaan v3 framework
|
|
176
|
+
The Al-Mizaan v3 framework defines 7 gates:
|
|
128
177
|
|
|
129
178
|
1. Code reading (literal, not docs)
|
|
130
179
|
2. Reachability chain (entry to exploit)
|
|
@@ -134,11 +183,19 @@ The Al-Mizaan v3 framework checks 7 gates:
|
|
|
134
183
|
6. Impact (real financial damage)
|
|
135
184
|
7. Formal proof (reproducible PoC)
|
|
136
185
|
|
|
137
|
-
|
|
186
|
+
This is the framework used for manual review (the `al-mizaan-judge` CLI). The paid scan does not run it end to end, see the correction at the top of this page.
|
|
138
187
|
|
|
139
188
|
## Service info
|
|
140
189
|
|
|
141
|
-
- **API endpoint**: https://
|
|
142
|
-
- **
|
|
143
|
-
- **
|
|
190
|
+
- **API endpoint**: https://x402.tokenizen.nl/api/bug-intel
|
|
191
|
+
- **Free precheck**: https://x402.tokenizen.nl/api/bug-intel/precheck?repo=owner/name
|
|
192
|
+
- **Terms**: https://x402.tokenizen.nl/terms
|
|
193
|
+
- **Delivery**: usually within the hour, at most 48 hours according to the terms
|
|
194
|
+
- **If it fails or does not help**: write to info@holistischadviseur.nl with the job id for a new attempt or a refund, see the terms
|
|
144
195
|
- **Source code**: deleted after analysis
|
|
196
|
+
|
|
197
|
+
## Support this project
|
|
198
|
+
|
|
199
|
+
This is a free, open-source MCP server maintained in my spare time. If it caught something useful before you shipped, tips are welcome and appreciated (never expected):
|
|
200
|
+
|
|
201
|
+
`0xdffcC75a674257be6FE1b5549FE52e8f8a6A3A5A` (USDC, Base)
|
package/index.mjs
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
* Payment: $5 USDC on Base (eip155:8453) via x402.
|
|
11
11
|
* If you receive a payment_required response, pay to the address shown and retry.
|
|
12
12
|
*
|
|
13
|
-
* Run: node
|
|
13
|
+
* Run: node index.mjs (or: npx -y bug-bounty-intelligence-mcp@latest)
|
|
14
14
|
*/
|
|
15
15
|
|
|
16
16
|
import { Server } from '@modelcontextprotocol/sdk/server/index.js'
|
|
@@ -19,7 +19,7 @@ import {
|
|
|
19
19
|
CallToolRequestSchema,
|
|
20
20
|
ListToolsRequestSchema,
|
|
21
21
|
} from '@modelcontextprotocol/sdk/types.js'
|
|
22
|
-
const API_URL = 'https://
|
|
22
|
+
const API_URL = 'https://x402.tokenizen.nl'
|
|
23
23
|
|
|
24
24
|
// Embedded pattern library — Sherlock audit-competition findings, updated 2026-07-21.
|
|
25
25
|
// Methodology: only contests where Sherlock's own judging-repo result structure could be
|
|
@@ -93,11 +93,13 @@ server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
|
|
93
93
|
{
|
|
94
94
|
name: 'scan_contract',
|
|
95
95
|
description: [
|
|
96
|
-
'Submit a public GitHub repository for an automated
|
|
97
|
-
'
|
|
98
|
-
'
|
|
99
|
-
'
|
|
100
|
-
'If
|
|
96
|
+
'Submit a public GitHub repository for an automated Solidity analysis. The result is a list of AI-generated hypotheses to review by hand. It is not an audit and not a list of confirmed bugs.',
|
|
97
|
+
'The scan reads the files most likely to move funds first, in at most six parts. Tests, mocks, scripts, interfaces and vendored dependencies are left out. The report states what share of the in-scope code was actually read, and a report can say that no hypotheses came out of the part that was read.',
|
|
98
|
+
'Check the repository first, for free: GET https://x402.tokenizen.nl/api/bug-intel/precheck?repo=owner/name says whether the scan counts any Solidity in it and roughly how much of it will be read.',
|
|
99
|
+
'How often the scan finds known issues on closed audit contests, including the misses, is published in the README of the repository (github.com/holistis/bug-bounty-intelligence-mcp). The prompt uses acceptance-rate hints from the 1,032 findings that could be exact-reconciled against contest outcomes, see METHODOLOGY.md.',
|
|
100
|
+
'Cost: $5 USDC on Base (eip155:8453) via x402. Reports usually arrive within the hour. If the scan fails or the report does not help, write to info@holistischadviseur.nl for a new attempt or a refund, see the terms.',
|
|
101
|
+
'Returns a job_id. Use get_scan_report to poll for results.',
|
|
102
|
+
'If payment_required is true, pay $5 USDC to the payTo address on Base, then retry scan_contract with the same repo_url.',
|
|
101
103
|
].join(' '),
|
|
102
104
|
inputSchema: {
|
|
103
105
|
type: 'object',
|
|
@@ -117,7 +119,12 @@ server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
|
|
117
119
|
},
|
|
118
120
|
{
|
|
119
121
|
name: 'get_scan_report',
|
|
120
|
-
description:
|
|
122
|
+
description: [
|
|
123
|
+
'Poll the status of a job_id returned by scan_contract.',
|
|
124
|
+
'Call this every few minutes after submitting. Reports usually arrive within the hour; polling faster just re-checks a job that is still processing.',
|
|
125
|
+
'Returns status (queued/processing/complete/failed), the repo under scan, and once complete the reportUrl (the report of hypotheses, with the share of code that was read), the outcome and the coverage. A failed job says why and where to write for a new attempt or a refund.',
|
|
126
|
+
'A 404 means the job_id does not exist — double-check what scan_contract actually returned rather than retrying blindly.',
|
|
127
|
+
].join(' '),
|
|
121
128
|
inputSchema: {
|
|
122
129
|
type: 'object',
|
|
123
130
|
properties: {
|
|
@@ -163,6 +170,17 @@ server.setRequestHandler(CallToolRequestSchema, async (req) => {
|
|
|
163
170
|
if (status === 402) {
|
|
164
171
|
const accept = json.accepts?.[0] ?? {}
|
|
165
172
|
const amtUsdc = accept.amount ? (parseInt(accept.amount) / 1_000_000).toFixed(2) : '5.00'
|
|
173
|
+
if (!accept.payTo || !accept.asset) {
|
|
174
|
+
// Never fall back to a hardcoded address here: a stale one silently
|
|
175
|
+
// sends real USDC to a wallet nobody is watching. Fail loud instead.
|
|
176
|
+
return {
|
|
177
|
+
content: [{
|
|
178
|
+
type: 'text',
|
|
179
|
+
text: 'PAYMENT REQUIRED, but the server did not return a payment address. Do not guess one. Retry scan_contract; if this repeats, the payment endpoint is broken, not the address.',
|
|
180
|
+
}],
|
|
181
|
+
isError: true,
|
|
182
|
+
}
|
|
183
|
+
}
|
|
166
184
|
return {
|
|
167
185
|
content: [{
|
|
168
186
|
type: 'text',
|
|
@@ -171,11 +189,11 @@ server.setRequestHandler(CallToolRequestSchema, async (req) => {
|
|
|
171
189
|
'',
|
|
172
190
|
`Amount: ${amtUsdc} USDC`,
|
|
173
191
|
`Network: Base mainnet (eip155:8453)`,
|
|
174
|
-
`Pay to: ${accept.payTo
|
|
175
|
-
`Asset: USDC — ${accept.asset
|
|
192
|
+
`Pay to: ${accept.payTo}`,
|
|
193
|
+
`Asset: USDC — ${accept.asset}`,
|
|
176
194
|
'',
|
|
177
195
|
'After paying, retry scan_contract with the same repo_url.',
|
|
178
|
-
'Terms: https://
|
|
196
|
+
'Terms: https://x402.tokenizen.nl/terms',
|
|
179
197
|
].join('\n'),
|
|
180
198
|
}],
|
|
181
199
|
isError: false,
|
|
@@ -191,7 +209,7 @@ server.setRequestHandler(CallToolRequestSchema, async (req) => {
|
|
|
191
209
|
`Status: ${json.status ?? 'queued'}`,
|
|
192
210
|
'',
|
|
193
211
|
`Poll for results: get_scan_report({ job_id: "${json.jobId}" })`,
|
|
194
|
-
`
|
|
212
|
+
`Reports usually arrive within the hour. The report states how much of the code was read.`,
|
|
195
213
|
].join('\n'),
|
|
196
214
|
}],
|
|
197
215
|
}
|
package/package.json
CHANGED
|
@@ -1,18 +1,40 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "bug-bounty-intelligence-mcp",
|
|
3
|
-
"version": "1.0.
|
|
4
|
-
"description": "MCP server for AI-powered smart contract security analysis. $5 USDC/scan via x402. Drawn from 27,681 Sherlock/Code4rena submissions, verified acceptance rates from 1,032 exact-reconciled findings.",
|
|
3
|
+
"version": "1.0.5",
|
|
4
|
+
"description": "MCP server for AI-powered smart contract security analysis. $5 USDC/scan via x402. Drawn from 27,681 Sherlock/Code4rena submissions, verified acceptance rates from 1,032 exact-reconciled findings. Includes a free, standalone dataset (vulnerability-acceptance-rates.json) — no server required.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.mjs",
|
|
7
|
-
"bin": {
|
|
8
|
-
|
|
9
|
-
|
|
7
|
+
"bin": {
|
|
8
|
+
"bug-bounty-intelligence-mcp": "index.mjs"
|
|
9
|
+
},
|
|
10
|
+
"files": [
|
|
11
|
+
"index.mjs",
|
|
12
|
+
"README.md",
|
|
13
|
+
"BENCHMARK.md",
|
|
14
|
+
"METHODOLOGY.md",
|
|
15
|
+
"vulnerability-acceptance-rates.json"
|
|
16
|
+
],
|
|
17
|
+
"repository": {
|
|
18
|
+
"type": "git",
|
|
19
|
+
"url": "git+https://github.com/holistis/bug-bounty-intelligence-mcp.git"
|
|
20
|
+
},
|
|
10
21
|
"homepage": "https://github.com/holistis/bug-bounty-intelligence-mcp#readme",
|
|
11
|
-
"keywords": [
|
|
22
|
+
"keywords": [
|
|
23
|
+
"mcp",
|
|
24
|
+
"smart-contract",
|
|
25
|
+
"security",
|
|
26
|
+
"audit",
|
|
27
|
+
"solidity",
|
|
28
|
+
"defi",
|
|
29
|
+
"x402",
|
|
30
|
+
"bug-bounty"
|
|
31
|
+
],
|
|
12
32
|
"author": "holistis",
|
|
13
33
|
"license": "MIT",
|
|
14
34
|
"dependencies": {
|
|
15
35
|
"@modelcontextprotocol/sdk": "^1.29.0"
|
|
16
36
|
},
|
|
17
|
-
"engines": {
|
|
18
|
-
|
|
37
|
+
"engines": {
|
|
38
|
+
"node": ">=20.0.0"
|
|
39
|
+
}
|
|
40
|
+
}
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema_note": "Free, standalone dataset — no server, no API key, no payment. Embed directly in your own tool.",
|
|
3
|
+
"source": "Sherlock audit-competition judging repos (public GitHub)",
|
|
4
|
+
"license": "CC0 — public domain, use freely with or without attribution",
|
|
5
|
+
"methodology_url": "https://github.com/holistis/bug-bounty-intelligence-mcp/blob/main/METHODOLOGY.md",
|
|
6
|
+
"last_updated": "2026-07-21T19:51:20.564Z",
|
|
7
|
+
"contests_included": 10,
|
|
8
|
+
"total_findings": 1032,
|
|
9
|
+
"total_accepted": 461,
|
|
10
|
+
"note": "Only contests where accepted+invalid counts reconcile exactly against Sherlock own published results are included. See methodology_url for full scope, limitations, and reproduction steps.",
|
|
11
|
+
"patterns": {
|
|
12
|
+
"fee-miscalculation": {
|
|
13
|
+
"acceptance_rate": 0.5,
|
|
14
|
+
"accepted": 55,
|
|
15
|
+
"rejected": 51,
|
|
16
|
+
"duplicated": 3,
|
|
17
|
+
"total": 109
|
|
18
|
+
},
|
|
19
|
+
"dos-griefing": {
|
|
20
|
+
"acceptance_rate": 0.41,
|
|
21
|
+
"accepted": 64,
|
|
22
|
+
"rejected": 90,
|
|
23
|
+
"duplicated": 2,
|
|
24
|
+
"total": 156
|
|
25
|
+
},
|
|
26
|
+
"reentrancy": {
|
|
27
|
+
"acceptance_rate": 0.78,
|
|
28
|
+
"accepted": 40,
|
|
29
|
+
"rejected": 11,
|
|
30
|
+
"duplicated": 0,
|
|
31
|
+
"total": 51
|
|
32
|
+
},
|
|
33
|
+
"trusted-actor": {
|
|
34
|
+
"acceptance_rate": 0.51,
|
|
35
|
+
"accepted": 159,
|
|
36
|
+
"rejected": 148,
|
|
37
|
+
"duplicated": 4,
|
|
38
|
+
"total": 311
|
|
39
|
+
},
|
|
40
|
+
"overflow": {
|
|
41
|
+
"acceptance_rate": 0.58,
|
|
42
|
+
"accepted": 26,
|
|
43
|
+
"rejected": 19,
|
|
44
|
+
"duplicated": 0,
|
|
45
|
+
"total": 45
|
|
46
|
+
},
|
|
47
|
+
"staleness": {
|
|
48
|
+
"acceptance_rate": 0.49,
|
|
49
|
+
"accepted": 86,
|
|
50
|
+
"rejected": 84,
|
|
51
|
+
"duplicated": 4,
|
|
52
|
+
"total": 174
|
|
53
|
+
},
|
|
54
|
+
"rounding": {
|
|
55
|
+
"acceptance_rate": 0.4,
|
|
56
|
+
"accepted": 25,
|
|
57
|
+
"rejected": 38,
|
|
58
|
+
"duplicated": 0,
|
|
59
|
+
"total": 63
|
|
60
|
+
},
|
|
61
|
+
"access-control": {
|
|
62
|
+
"acceptance_rate": 0.47,
|
|
63
|
+
"accepted": 35,
|
|
64
|
+
"rejected": 39,
|
|
65
|
+
"duplicated": 0,
|
|
66
|
+
"total": 74
|
|
67
|
+
},
|
|
68
|
+
"oracle-manipulation": {
|
|
69
|
+
"acceptance_rate": 0.36,
|
|
70
|
+
"accepted": 47,
|
|
71
|
+
"rejected": 78,
|
|
72
|
+
"duplicated": 6,
|
|
73
|
+
"total": 131
|
|
74
|
+
},
|
|
75
|
+
"mev-slippage": {
|
|
76
|
+
"acceptance_rate": 0.49,
|
|
77
|
+
"accepted": 44,
|
|
78
|
+
"rejected": 44,
|
|
79
|
+
"duplicated": 1,
|
|
80
|
+
"total": 89
|
|
81
|
+
},
|
|
82
|
+
"liquidation": {
|
|
83
|
+
"acceptance_rate": 0.29,
|
|
84
|
+
"accepted": 4,
|
|
85
|
+
"rejected": 9,
|
|
86
|
+
"duplicated": 1,
|
|
87
|
+
"total": 14
|
|
88
|
+
},
|
|
89
|
+
"flash-loan": {
|
|
90
|
+
"acceptance_rate": 0.46,
|
|
91
|
+
"accepted": 6,
|
|
92
|
+
"rejected": 6,
|
|
93
|
+
"duplicated": 1,
|
|
94
|
+
"total": 13
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
}
|