@bryan-thompson/inspector-assessment 1.43.2 → 1.43.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1062 -224
- package/cli/build/assess-full.js +532 -106
- package/cli/build/assess-security.js +54 -90
- package/cli/build/lib/cli-parser.js +14 -1
- package/cli/build/lib/cli-parserSchemas.js +1 -0
- package/cli/build/lib/result-output.js +21 -0
- package/cli/build/profiles.js +20 -0
- package/cli/build/validate-testbed.js +0 -0
- package/cli/package.json +1 -1
- package/client/dist/assets/{OAuthCallback-BS8-A1sU.js → OAuthCallback-Chi58kRc.js} +1 -1
- package/client/dist/assets/{OAuthDebugCallback-025_TM2i.js → OAuthDebugCallback-BluD_Wxg.js} +1 -1
- package/client/dist/assets/{index-DEhlIjy-.js → index-KW2LwGdp.js} +4 -4
- package/client/dist/index.html +1 -1
- package/client/lib/lib/assessment/configSchemas.d.ts +64 -64
- package/client/lib/lib/assessment/jsonlEventSchemas.d.ts +286 -286
- package/client/lib/lib/assessment/resultTypes.d.ts +10 -0
- package/client/lib/lib/assessment/resultTypes.d.ts.map +1 -1
- package/client/lib/lib/assessmentTypes.d.ts +1 -20
- package/client/lib/lib/assessmentTypes.d.ts.map +1 -1
- package/client/lib/lib/assessmentTypes.js +1 -20
- package/client/lib/services/assessment/AssessmentOrchestrator.d.ts +57 -104
- package/client/lib/services/assessment/AssessmentOrchestrator.d.ts.map +1 -1
- package/client/lib/services/assessment/AssessmentOrchestrator.js +298 -133
- package/client/lib/services/assessment/modules/ErrorHandlingAssessor.d.ts +25 -0
- package/client/lib/services/assessment/modules/ErrorHandlingAssessor.d.ts.map +1 -0
- package/client/lib/services/assessment/modules/ErrorHandlingAssessor.deprecated.js +1 -1
- package/client/lib/services/assessment/modules/ErrorHandlingAssessor.js +564 -0
- package/client/lib/services/assessment/modules/SecurityAssessor.d.ts +5 -0
- package/client/lib/services/assessment/modules/SecurityAssessor.d.ts.map +1 -1
- package/client/lib/services/assessment/modules/SecurityAssessor.js +62 -0
- package/client/lib/services/assessment/modules/index.d.ts +1 -1
- package/client/lib/services/assessment/modules/index.js +1 -1
- package/client/lib/services/assessment/modules/securityTests/SecurityPayloadTester.d.ts +15 -0
- package/client/lib/services/assessment/modules/securityTests/SecurityPayloadTester.d.ts.map +1 -1
- package/client/lib/services/assessment/modules/securityTests/SecurityPayloadTester.js +72 -0
- package/client/lib/services/assessment/modules/securityTests/factory.d.ts +2 -0
- package/client/lib/services/assessment/modules/securityTests/factory.d.ts.map +1 -1
- package/client/lib/services/assessment/modules/securityTests/factory.js +1 -0
- package/client/lib/services/assessment/registry/AssessorDefinitions.js +1 -1
- package/client/lib/services/assessment/responseValidatorSchemas.d.ts +12 -12
- package/client/package.json +3 -3
- package/package.json +4 -2
- package/server/package.json +1 -1
- package/cli/build/lib/__tests__/zodErrorFormatter.test.js +0 -282
- package/client/lib/services/assessment/modules/ProtocolComplianceAssessor.d.ts +0 -109
- package/client/lib/services/assessment/modules/ProtocolComplianceAssessor.d.ts.map +0 -1
- package/client/lib/services/assessment/modules/ProtocolComplianceAssessor.deprecated.d.ts +0 -109
- package/client/lib/services/assessment/modules/ProtocolComplianceAssessor.deprecated.d.ts.map +0 -1
- package/client/lib/services/assessment/modules/ProtocolComplianceAssessor.deprecated.js +0 -852
- package/client/lib/services/assessment/modules/ProtocolComplianceAssessor.js +0 -852
package/README.md
CHANGED
|
@@ -3,364 +3,1202 @@
|
|
|
3
3
|
[](https://www.npmjs.com/package/@bryan-thompson/inspector-assessment)
|
|
4
4
|
[](https://www.npmjs.com/package/@bryan-thompson/inspector-assessment)
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
Test functionality, security, documentation, code quality, and policy compliance from the command line.
|
|
6
|
+
The MCP inspector is a developer tool for testing and debugging MCP servers with comprehensive assessment capabilities for validating server functionality, security, documentation, and compliance.
|
|
8
7
|
|
|
9
8
|

|
|
10
9
|
|
|
11
|
-
---
|
|
12
|
-
|
|
13
10
|
## Installation
|
|
14
11
|
|
|
12
|
+
**npm (global installation):**
|
|
13
|
+
|
|
15
14
|
```bash
|
|
16
|
-
# Install globally
|
|
17
15
|
npm install -g @bryan-thompson/inspector-assessment
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
**Or use directly with bunx (no installation):**
|
|
18
19
|
|
|
19
|
-
|
|
20
|
+
```bash
|
|
20
21
|
bunx @bryan-thompson/inspector-assessment
|
|
21
22
|
```
|
|
22
23
|
|
|
23
|
-
|
|
24
|
+
**Local installation for development:**
|
|
24
25
|
|
|
25
|
-
|
|
26
|
+
```bash
|
|
27
|
+
git clone https://github.com/triepod-ai/inspector-assessment.git
|
|
28
|
+
cd inspector-assessment
|
|
29
|
+
npm install
|
|
30
|
+
npm run build
|
|
31
|
+
npm run dev
|
|
32
|
+
```
|
|
26
33
|
|
|
27
|
-
|
|
34
|
+
## Quick Start
|
|
35
|
+
|
|
36
|
+
After installation, launch the inspector:
|
|
28
37
|
|
|
29
38
|
```bash
|
|
30
|
-
#
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
39
|
+
# Using global install
|
|
40
|
+
mcp-inspector-assess
|
|
41
|
+
|
|
42
|
+
# Using bunx
|
|
43
|
+
bunx @bryan-thompson/inspector-assessment
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
The web interface will open at http://localhost:6274
|
|
47
|
+
|
|
48
|
+
## For MCP Directory Reviewers
|
|
49
|
+
|
|
50
|
+
If you're reviewing MCP servers for the Anthropic MCP Directory, see our **[Reviewer Quick Start Guide](docs/REVIEWER_QUICK_START.md)** for:
|
|
51
|
+
|
|
52
|
+
- **60-second fast screening** workflow for approve/reject decisions
|
|
53
|
+
- **5-minute detailed review** process for borderline cases
|
|
54
|
+
- **Common pitfalls** explanation (false positives in security, informational vs scored tests)
|
|
55
|
+
- **Decision matrix** with clear approval criteria
|
|
56
|
+
- **Fast CLI analysis** commands for troubleshooting
|
|
57
|
+
|
|
58
|
+
The quick start guide is optimized for fast reviewer onboarding and provides clear guidance on interpreting assessment results.
|
|
59
|
+
|
|
60
|
+
## About This Fork
|
|
61
|
+
|
|
62
|
+
This is an enhanced fork of [Anthropic's MCP Inspector](https://github.com/modelcontextprotocol/inspector) with significantly expanded assessment capabilities for MCP server validation and testing.
|
|
63
|
+
|
|
64
|
+
**Original Repository**: https://github.com/modelcontextprotocol/inspector
|
|
65
|
+
**Our Enhanced Fork**: https://github.com/triepod-ai/inspector-assessment
|
|
66
|
+
|
|
67
|
+
**⚠️ Important**: This is a published fork with assessment enhancements. If you want the official Anthropic inspector without assessment features, use `npx @modelcontextprotocol/inspector`.
|
|
68
|
+
|
|
69
|
+
### What We Added
|
|
70
|
+
|
|
71
|
+
We've built a comprehensive assessment framework on top of the original inspector that transforms it from a debugging tool into a full validation suite for MCP servers. Our enhancements focus on accuracy, depth, and actionable insights for MCP server developers.
|
|
72
|
+
|
|
73
|
+
## Key Features
|
|
74
|
+
|
|
75
|
+
- **Interactive Testing**: Visual interface for testing MCP server tools, resources, and prompts
|
|
76
|
+
- **Comprehensive Assessment**: Automated validation of server functionality, error handling, documentation, security, and usability using multi-scenario testing with progressive complexity
|
|
77
|
+
- **Business Logic Validation**: Distinguishes between proper error handling and unintended failures
|
|
78
|
+
- **Detailed Test Reports**: Confidence scoring, test scenario details, and actionable recommendations
|
|
79
|
+
- **Multiple Transport Support**: STDIO, SSE, and Streamable HTTP transports
|
|
80
|
+
|
|
81
|
+
## Quality Metrics
|
|
82
|
+
|
|
83
|
+
Our enhanced fork maintains high code quality standards with comprehensive testing and validation:
|
|
84
|
+
|
|
85
|
+
- **Test Coverage**: ✅ 665/665 tests passing (100% pass rate)
|
|
86
|
+
- **Assessment Module Tests**: 291 tests specifically validating our assessment enhancements (including 83 new MCP Directory compliance tests)
|
|
87
|
+
- Business logic error detection with confidence scoring
|
|
88
|
+
- Progressive complexity testing (2 levels: minimal → simple)
|
|
89
|
+
- Context-aware security testing with zero false positives
|
|
90
|
+
- Realistic test data generation and boundary testing
|
|
91
|
+
- **Total Project Tests**: 582 tests including assessment modules, UI components, and core inspector functionality
|
|
92
|
+
- All tests updated to reflect focused backend testing (8 security patterns × 3 payloads per tool)
|
|
93
|
+
- Test files: `client/src/services/__tests__/` and `client/src/services/assessment/__tests__/`
|
|
94
|
+
- **Code Quality**: ✅ Production code uses proper TypeScript types
|
|
95
|
+
- 229 lint issues remaining (down 18% from 280 after recent cleanup)
|
|
96
|
+
- All source files migrated from `any` to `unknown` or proper types
|
|
97
|
+
- React Hooks follow best practices with proper dependency arrays
|
|
98
|
+
- **Build Status**: ✅ Production builds pass cleanly
|
|
99
|
+
- TypeScript compilation successful for all production code
|
|
100
|
+
- Vite build optimized and validated
|
|
101
|
+
- **Upstream Sync**: ✅ Up-to-date with v0.17.0
|
|
102
|
+
- Successfully integrated 121 commits from upstream
|
|
103
|
+
- New features: CustomHeaders, OAuth improvements, parameter validation
|
|
104
|
+
- All enhancements preserved during merge
|
|
105
|
+
|
|
106
|
+
**Testing Commands**:
|
|
107
|
+
|
|
108
|
+
```bash
|
|
109
|
+
npm test # Run all 665 tests
|
|
110
|
+
npm test -- assessment # Run all 291 assessment module tests
|
|
111
|
+
npm test -- assessmentService # Run assessment service integration tests (54 tests)
|
|
112
|
+
npm test -- SecurityAssessor # Run security assessment tests (16 tests)
|
|
113
|
+
npm test -- FunctionalityAssessor # Run functionality tests (11 tests)
|
|
114
|
+
npm test -- AUPCompliance # Run AUP compliance tests (26 tests)
|
|
115
|
+
npm test -- ToolAnnotation # Run tool annotation tests (13 tests)
|
|
116
|
+
npm run coverage # Generate coverage report
|
|
117
|
+
npm run lint # Check code quality
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
## Our Enhancements to the MCP Inspector
|
|
121
|
+
|
|
122
|
+
We've significantly expanded the original MCP Inspector's capabilities with advanced assessment features that go far beyond basic debugging. Here's what makes our fork unique:
|
|
123
|
+
|
|
124
|
+
### 1. Enhanced Business Logic Error Detection
|
|
125
|
+
|
|
126
|
+
**Problem**: The original inspector couldn't distinguish between broken tools and tools that correctly validate input. A tool returning "user not found" would be marked as broken.
|
|
127
|
+
|
|
128
|
+
**Our Solution**: Confidence-based validation system (ResponseValidator.ts:client/src/services/assessment/ResponseValidator.ts)
|
|
129
|
+
|
|
130
|
+
- **MCP Standard Error Code Recognition**: Properly identifies error codes like `-32602` (Invalid params) as successful validation
|
|
131
|
+
- **Confidence Scoring**: Multi-factor analysis determines if errors represent proper business logic
|
|
132
|
+
- **Tool Type Awareness**: Different validation thresholds for CRUD vs utility tools
|
|
133
|
+
- **Impact**: Estimated 80% reduction in false positives for resource-based tools (based on analysis in [FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md](docs/FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md#key-problems-addressed))
|
|
134
|
+
|
|
135
|
+
### 2. Optimized Progressive Complexity Testing
|
|
136
|
+
|
|
137
|
+
**Problem**: Testing tools with only complex inputs makes it hard to identify where functionality breaks down.
|
|
138
|
+
|
|
139
|
+
**Our Solution**: Two-level progressive diagnostic testing (TestScenarioEngine.ts:client/src/services/assessment/TestScenarioEngine.ts)
|
|
140
|
+
|
|
141
|
+
1. **Minimal**: Only required fields with simplest values - diagnoses basic setup issues
|
|
142
|
+
2. **Simple**: Required fields with realistic simple values - validates core functionality
|
|
143
|
+
|
|
144
|
+
Combined with multi-scenario comprehensive testing (Happy Path, Edge Cases, Boundary Testing) for full coverage.
|
|
145
|
+
|
|
146
|
+
**Benefits**:
|
|
147
|
+
|
|
148
|
+
- **50% faster** than previous 4-level approach (removed redundant typical/maximum tests)
|
|
149
|
+
- Identifies exact complexity level where tools fail (minimal vs simple)
|
|
150
|
+
- Provides specific, actionable recommendations
|
|
151
|
+
- Zero coverage loss - comprehensive scenarios provide full validation
|
|
152
|
+
- Helps developers understand tool limitations and requirements
|
|
153
|
+
|
|
154
|
+
**Performance**: For 10-tool servers, comprehensive testing now runs in ~4.2-8.3 minutes (down from ~7.5-11.7 minutes)
|
|
155
|
+
|
|
156
|
+
### 3. Realistic Test Data Generation
|
|
157
|
+
|
|
158
|
+
**Problem**: Generic test data like "test_value" and fake IDs trigger validation errors, causing false failures.
|
|
159
|
+
|
|
160
|
+
**Our Solution**: Context-aware test data generation (TestDataGenerator.ts:client/src/services/assessment/TestDataGenerator.ts)
|
|
161
|
+
|
|
162
|
+
- **Publicly Accessible URLs**: `https://www.google.com`, `https://api.github.com/users/octocat`
|
|
163
|
+
- **Real API Endpoints**: Uses actual test APIs like jsonplaceholder.typicode.com
|
|
164
|
+
- **Valid UUIDs**: Properly formatted identifiers that won't trigger format validation
|
|
165
|
+
- **Context Awareness**: Generates appropriate data based on field names (email, url, id, etc.)
|
|
166
|
+
|
|
167
|
+
### 4. Context-Aware Security Assessment with Zero False Positives
|
|
168
|
+
|
|
169
|
+
**Problem**: Traditional security scanners flag tools as vulnerable when they safely store or echo malicious input as data. For example, a database tool that stores `"<script>alert(1)"` would be incorrectly flagged as vulnerable to XSS.
|
|
170
|
+
|
|
171
|
+
**Our Solution**: Intelligent reflection detection (SecurityAssessor.ts:client/src/services/assessment/modules/SecurityAssessor.ts)
|
|
172
|
+
|
|
173
|
+
**Key Innovation**: Distinguishing **data reflection** (safe) from **command execution** (vulnerable)
|
|
174
|
+
|
|
175
|
+
**Examples**:
|
|
176
|
+
|
|
177
|
+
✅ **SAFE - Data Reflection**:
|
|
178
|
+
|
|
179
|
+
```
|
|
180
|
+
Payload: "<script>alert(1)</script>"
|
|
181
|
+
Response: "Stored in collection: <script>alert(1)</script>"
|
|
182
|
+
→ Tool is just storing data, not executing it
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
❌ **VULNERABLE - Command Execution** (Calculator Injection):
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
Payload: "2+2"
|
|
189
|
+
Response: "The answer is 4"
|
|
190
|
+
→ Tool executed the arithmetic expression via eval()!
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
**Detection Approach**:
|
|
194
|
+
|
|
195
|
+
1. **Reflection Pattern Recognition**: Identifies safe data operations through patterns like "stored", "created", "error getting info for", "collection doesn't exist"
|
|
196
|
+
2. **Execution Evidence Detection**: Only flags as vulnerable when actual execution is detected (calculator returning "4", API keys leaked, admin mode activated)
|
|
197
|
+
3. **Error Message Handling**: Recognizes that error messages echoing invalid input are safe reflection, not vulnerabilities
|
|
198
|
+
|
|
199
|
+
**Impact**:
|
|
200
|
+
|
|
201
|
+
- **Zero false positives** on data storage/retrieval tools (qdrant, databases, file systems)
|
|
202
|
+
- **18 injection patterns tested** (9 original + 9 advanced patterns)
|
|
203
|
+
- **Dual-mode testing**: Reviewer mode (3 critical patterns, fast) + Developer mode (all 13 patterns, comprehensive)
|
|
204
|
+
- **Real vulnerabilities still detected**: 100% test pass rate on detecting actual command injection, calculator injection, role override, data exfiltration
|
|
205
|
+
|
|
206
|
+
**Supported Injection Types**:
|
|
207
|
+
|
|
208
|
+
- **Command Injection**: System commands (whoami, ls -la, pwd)
|
|
209
|
+
- **Calculator Injection**: Arithmetic expressions and code injection via eval() (NEW - 7 payloads)
|
|
210
|
+
- **SQL Injection**: Database command injection
|
|
211
|
+
- **Path Traversal**: File system access outside intended directory
|
|
212
|
+
- Plus 9 additional patterns (Unicode Bypass, Nested Injection, Package Squatting, etc.)
|
|
213
|
+
|
|
214
|
+
**Validation**: See [VULNERABILITY_TESTING.md](VULNERABILITY_TESTING.md) for detailed testing guide and examples.
|
|
215
|
+
|
|
216
|
+
### 5. Streamlined Assessment Architecture
|
|
217
|
+
|
|
218
|
+
**Based on Real-World Testing**: Our methodology has been validated through systematic testing using the taskmanager MCP server as a case study (11 tools tested with 8 backend security patterns, detailed in [ASSESSMENT_METHODOLOGY.md](docs/ASSESSMENT_METHODOLOGY.md)).
|
|
219
|
+
|
|
220
|
+
**Eleven Core Assessors** aligned with Anthropic's MCP directory submission requirements:
|
|
221
|
+
|
|
222
|
+
**Original MCP Inspector Assessors:**
|
|
223
|
+
|
|
224
|
+
1. **FunctionalityAssessor** (225 lines)
|
|
225
|
+
- Multi-scenario validation with progressive complexity
|
|
226
|
+
- Coverage tracking and reliability scoring
|
|
227
|
+
- Business logic error detection
|
|
228
|
+
- Performance measurement
|
|
229
|
+
|
|
230
|
+
2. **SecurityAssessor** (443 lines)
|
|
231
|
+
- 13 distinct injection attack patterns (including Calculator Injection) with context-aware reflection detection
|
|
232
|
+
- Direct command injection, calculator injection (eval detection), role override, data exfiltration detection
|
|
233
|
+
- Vulnerability analysis with risk levels (HIGH/MEDIUM/LOW)
|
|
234
|
+
- Zero false positives through intelligent distinction between data reflection and command execution
|
|
235
|
+
|
|
236
|
+
3. **ErrorHandlingAssessor** (692 lines)
|
|
237
|
+
- MCP protocol compliance scoring
|
|
238
|
+
- Error response quality analysis
|
|
239
|
+
- Invalid input resilience testing
|
|
240
|
+
|
|
241
|
+
4. **DocumentationAssessor** (274 lines)
|
|
242
|
+
- README structure and completeness
|
|
243
|
+
- Code example extraction and validation
|
|
244
|
+
- API reference quality assessment
|
|
245
|
+
|
|
246
|
+
5. **UsabilityAssessor** (290 lines)
|
|
247
|
+
- Naming convention consistency
|
|
248
|
+
- Parameter clarity assessment
|
|
249
|
+
- Best practices compliance
|
|
250
|
+
|
|
251
|
+
6. **MCPSpecComplianceAssessor** (560 lines) - Extended
|
|
252
|
+
- JSON-RPC 2.0 compliance validation
|
|
253
|
+
- Protocol message format verification
|
|
254
|
+
- MCP specification adherence
|
|
255
|
+
|
|
256
|
+
**NEW: MCP Directory Compliance Assessors** (added 2025-12):
|
|
257
|
+
|
|
258
|
+
7. **AUPComplianceAssessor** - Policy compliance
|
|
259
|
+
- 14 AUP category violation detection (A-N)
|
|
260
|
+
- High-risk domain identification (Healthcare, Financial, Legal, Children)
|
|
261
|
+
- Tool name/description pattern analysis
|
|
262
|
+
- Source code scanning (enhanced mode)
|
|
263
|
+
|
|
264
|
+
8. **ToolAnnotationAssessor** - Policy #17 compliance
|
|
265
|
+
- readOnlyHint/destructiveHint verification
|
|
266
|
+
- Tool behavior inference from name patterns
|
|
267
|
+
- Annotation misalignment detection
|
|
268
|
+
- Policy #17 compliance reporting
|
|
269
|
+
|
|
270
|
+
9. **ProhibitedLibrariesAssessor** - Policy #28-30 compliance
|
|
271
|
+
- Financial library detection (Stripe, PayPal, Plaid, etc.)
|
|
272
|
+
- Media library detection (Sharp, FFmpeg, OpenCV, PIL)
|
|
273
|
+
- package.json and requirements.txt scanning
|
|
274
|
+
- Source code import analysis
|
|
275
|
+
|
|
276
|
+
10. **ManifestValidationAssessor** - MCPB manifest compliance
|
|
277
|
+
- manifest_version 0.3 validation
|
|
278
|
+
- Required field verification (name, version, mcp_config)
|
|
279
|
+
- Icon presence check
|
|
280
|
+
- ${BUNDLE_ROOT} anti-pattern detection
|
|
281
|
+
|
|
282
|
+
11. **PortabilityAssessor** - Cross-platform compatibility
|
|
283
|
+
- Hardcoded path detection (/Users/, /home/, C:\)
|
|
284
|
+
- Platform-specific code patterns
|
|
285
|
+
- ${\_\_dirname} usage validation
|
|
286
|
+
|
|
287
|
+
**Recent Updates**:
|
|
288
|
+
|
|
289
|
+
- (2025-12-07): Added 5 new MCP Directory compliance assessors with 83 tests
|
|
290
|
+
- (2025-10-05): Removed 2,707 lines of out-of-scope assessment modules to focus on core MCP validation requirements
|
|
291
|
+
|
|
292
|
+
### 6. Advanced Assessment Components
|
|
293
|
+
|
|
294
|
+
We've built a complete assessment architecture with specialized modules:
|
|
295
|
+
|
|
296
|
+
- **AssessmentOrchestrator.ts**: Coordinates multi-phase testing across all assessment dimensions
|
|
297
|
+
- **ResponseValidator.ts**: Advanced response validation with confidence scoring and business logic detection
|
|
298
|
+
- **TestScenarioEngine.ts**: Generates and executes optimized 2-level progressive complexity tests
|
|
299
|
+
- **TestDataGenerator.ts**: Context-aware realistic test data generation with conditional boundary testing
|
|
300
|
+
- **Assessment UI Components**: Rich visualization of test results and recommendations
|
|
301
|
+
|
|
302
|
+
### Documentation
|
|
303
|
+
|
|
304
|
+
Our enhancements include comprehensive documentation:
|
|
305
|
+
|
|
306
|
+
- **ASSESSMENT_METHODOLOGY.md**: Complete methodology with examples and best practices
|
|
307
|
+
- **FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md**: Implementation details and impact analysis
|
|
308
|
+
- **COMPREHENSIVE_TESTING_OPTIMIZATION_PLAN.md**: Detailed optimization strategy (Phases 1-2 complete)
|
|
309
|
+
- **PHASE1_OPTIMIZATION_COMPLETED.md**: Progressive complexity optimization (50% faster)
|
|
310
|
+
- **PHASE2_OPTIMIZATION_COMPLETED.md**: Business logic error detection enhancements
|
|
311
|
+
- **PROJECT_STATUS.md**: Current status, recent changes, and development roadmap
|
|
312
|
+
- **Test Coverage Reports**: Detailed validation of our assessment accuracy
|
|
313
|
+
|
|
314
|
+
## Architecture Overview
|
|
315
|
+
|
|
316
|
+
The MCP Inspector consists of two main components that work together:
|
|
317
|
+
|
|
318
|
+
- **MCP Inspector Client (MCPI)**: A React-based web UI that provides an interactive interface for testing and debugging MCP servers
|
|
319
|
+
- **MCP Proxy (MCPP)**: A Node.js server that acts as a protocol bridge, connecting the web UI to MCP servers via various transport methods (stdio, SSE, streamable-http)
|
|
320
|
+
|
|
321
|
+
Note that the proxy is not a network proxy for intercepting traffic. Instead, it functions as both an MCP client (connecting to your MCP server) and an HTTP server (serving the web UI), enabling browser-based interaction with MCP servers that use different transport protocols.
|
|
322
|
+
|
|
323
|
+
## Assessment Capabilities
|
|
324
|
+
|
|
325
|
+
Our enhanced MCP Inspector includes a comprehensive assessment system that validates MCP servers against Anthropic's directory submission requirements and MCP protocol standards:
|
|
326
|
+
|
|
327
|
+
### Assessment Categories
|
|
328
|
+
|
|
329
|
+
1. **Functionality Testing**
|
|
330
|
+
- Multi-scenario validation with happy path, edge cases, and boundary testing
|
|
331
|
+
- Optimized progressive complexity testing (2 levels: minimal → simple)
|
|
332
|
+
- Business logic validation to distinguish proper error handling from failures
|
|
333
|
+
- Confidence scoring based on test coverage and consistency
|
|
334
|
+
- Evidence-based recommendations with transparent methodology (e.g., "5/5 scenarios verified (happy path, edge cases, boundaries, error handling)")
|
|
335
|
+
|
|
336
|
+
2. **Error Handling**
|
|
337
|
+
- Invalid input resilience testing
|
|
338
|
+
- Comprehensive error message analysis
|
|
339
|
+
- Resource validation vs. unintended failures
|
|
340
|
+
- MCP protocol compliance scoring
|
|
341
|
+
- Quality scoring for descriptive error messages
|
|
342
|
+
|
|
343
|
+
3. **Documentation**
|
|
344
|
+
- Tool description completeness and clarity
|
|
345
|
+
- Parameter documentation validation
|
|
346
|
+
- README structure and examples evaluation
|
|
347
|
+
- API documentation quality assessment
|
|
348
|
+
|
|
349
|
+
4. **Security**
|
|
350
|
+
- 17 distinct injection attack patterns (8 original + 9 advanced patterns)
|
|
351
|
+
- Context-aware reflection detection distinguishes safe data operations from command execution
|
|
352
|
+
- Zero false positives - correctly handles tools that echo/store malicious input as data
|
|
353
|
+
- Input validation and sanitization checks
|
|
354
|
+
- Authentication/authorization testing
|
|
355
|
+
- Sensitive data exposure detection
|
|
356
|
+
- Dual-mode testing: Reviewer mode (3 critical patterns) + Developer mode (all 13 patterns)
|
|
357
|
+
|
|
358
|
+
5. **Usability**
|
|
359
|
+
- Tool naming consistency analysis
|
|
360
|
+
- Description quality assessment
|
|
361
|
+
- Schema completeness validation
|
|
362
|
+
- Parameter clarity evaluation
|
|
363
|
+
|
|
364
|
+
6. **MCP Spec Compliance** (Extended)
|
|
365
|
+
- JSON-RPC 2.0 protocol compliance
|
|
366
|
+
- MCP message format verification
|
|
367
|
+
- Error code standard compliance
|
|
368
|
+
- Protocol specification adherence
|
|
369
|
+
|
|
370
|
+
### Testing Features
|
|
371
|
+
|
|
372
|
+
**Note**: All testing uses comprehensive multi-scenario validation. See the "Our Enhancements" section above for detailed technical descriptions.
|
|
373
|
+
|
|
374
|
+
#### Multi-Scenario Validation
|
|
375
|
+
|
|
376
|
+
The inspector tests each tool with multiple scenarios:
|
|
377
|
+
|
|
378
|
+
- **Happy Path**: Valid inputs with expected success cases
|
|
379
|
+
- **Edge Cases**: Boundary values and unusual but valid inputs
|
|
380
|
+
- **Error Cases**: Invalid inputs to test error handling
|
|
381
|
+
- **Boundary Testing**: Maximum/minimum values and limits
|
|
382
|
+
|
|
383
|
+
#### Progressive Complexity Testing
|
|
384
|
+
|
|
385
|
+
Tools are tested with progressive diagnostic levels to identify where functionality breaks:
|
|
386
|
+
|
|
387
|
+
1. **Minimal**: Only required fields with simplest values - diagnoses basic setup issues
|
|
388
|
+
2. **Simple**: Required fields with realistic simple values - validates core functionality
|
|
389
|
+
|
|
390
|
+
This optimized 2-level approach is **50% faster** than previous 4-level testing while maintaining full coverage through comprehensive multi-scenario validation.
|
|
391
|
+
|
|
392
|
+
#### Business Logic Validation
|
|
393
|
+
|
|
394
|
+
The assessment distinguishes between:
|
|
395
|
+
|
|
396
|
+
- **Proper Validation**: Expected errors for invalid business logic (e.g., "User not found")
|
|
397
|
+
- **Tool Failures**: Unexpected errors indicating implementation issues
|
|
398
|
+
- **Resource Validation**: Proper handling of non-existent resources
|
|
399
|
+
- **Input Validation**: Appropriate rejection of malformed inputs
|
|
400
|
+
|
|
401
|
+
### Assessment Configuration
|
|
402
|
+
|
|
403
|
+
Configure assessment behavior through the UI:
|
|
404
|
+
|
|
405
|
+
| Setting | Description | Default |
|
|
406
|
+
| --------------------------------- | ------------------------------------------------------------------- | --------- |
|
|
407
|
+
| Tool Selection for Error Handling | Multi-select dropdown with checkboxes to choose which tools to test | All tools |
|
|
408
|
+
|
|
409
|
+
**Tool Selection** (as of 2025-10-10):
|
|
410
|
+
|
|
411
|
+
- Visual multi-select dropdown with checkboxes for each tool
|
|
412
|
+
- Search/filter functionality for large tool lists
|
|
413
|
+
- "Select All" / "Deselect All" bulk operations
|
|
414
|
+
- Shows "X of Y tools selected" count
|
|
415
|
+
- Select 0 tools to skip error handling tests entirely (fastest option)
|
|
416
|
+
|
|
417
|
+
**Note**: The old numeric "Error Handling Test Limit" has been replaced with the tool selector. The `maxToolsToTestForErrors` config field is deprecated but still works for backward compatibility.
|
|
418
|
+
|
|
419
|
+
### Viewing Assessment Results
|
|
420
|
+
|
|
421
|
+
The Assessment tab provides:
|
|
422
|
+
|
|
423
|
+
- **Overall Score**: Weighted aggregate score with letter grade (A-F)
|
|
424
|
+
- **Category Breakdown**: Individual scores for each assessment category
|
|
425
|
+
- **Tool Details**: Click any tool name to see detailed test results including:
|
|
426
|
+
- Test scenarios executed
|
|
427
|
+
- Input parameters used
|
|
428
|
+
- Actual responses received
|
|
429
|
+
- Pass/fail status with confidence scores
|
|
430
|
+
- Specific issues identified
|
|
431
|
+
- **Per-Tool JSON Display** (New in 2025-10-10): Each tool in security assessment has its own "Show JSON" button to view only that tool's test results without scrolling through all tools
|
|
432
|
+
- **Recommendations**: Actionable suggestions for improvement
|
|
433
|
+
- **Test Coverage**: Visual indicators of testing completeness
|
|
434
|
+
|
|
435
|
+
### Assessment Result Persistence
|
|
436
|
+
|
|
437
|
+
**New in 2025-10-06**: Assessment results are automatically saved to JSON files for fast CLI-based analysis and troubleshooting.
|
|
438
|
+
|
|
439
|
+
**Automatic Save**:
|
|
440
|
+
|
|
441
|
+
- Every assessment run automatically saves to `/tmp/inspector-assessment-{serverName}.json`
|
|
442
|
+
- Old results are automatically deleted before new runs
|
|
443
|
+
- No manual export needed - completely transparent
|
|
444
|
+
- Console shows: `✅ Assessment auto-saved: /tmp/inspector-assessment-{name}.json`
|
|
445
|
+
|
|
446
|
+
**Quick Analysis Examples**:
|
|
447
|
+
|
|
448
|
+
```bash
|
|
449
|
+
# View full assessment results
|
|
450
|
+
cat /tmp/inspector-assessment-memory-mcp.json | jq
|
|
451
|
+
|
|
452
|
+
# Check only functionality results
|
|
453
|
+
cat /tmp/inspector-assessment-memory-mcp.json | jq '.functionality'
|
|
454
|
+
|
|
455
|
+
# List broken tools
|
|
456
|
+
cat /tmp/inspector-assessment-memory-mcp.json | jq '.functionality.brokenTools'
|
|
457
|
+
|
|
458
|
+
# Get specific tool test results
|
|
459
|
+
cat /tmp/inspector-assessment-memory-mcp.json | jq '.functionality.enhancedResults[] | select(.toolName == "search_nodes")'
|
|
460
|
+
|
|
461
|
+
# Summary of all tools and their status
|
|
462
|
+
cat /tmp/inspector-assessment-memory-mcp.json | jq '.functionality.enhancedResults[] | {tool: .toolName, status: .overallStatus}'
|
|
463
|
+
|
|
464
|
+
# Count security vulnerabilities found
|
|
465
|
+
cat /tmp/inspector-assessment-memory-mcp.json | jq '.security.vulnerabilities | length'
|
|
466
|
+
|
|
467
|
+
# Check error handling coverage
|
|
468
|
+
cat /tmp/inspector-assessment-memory-mcp.json | jq '.errorHandling.metrics.validationCoverage'
|
|
469
|
+
```
|
|
470
|
+
|
|
471
|
+
**Benefits**:
|
|
472
|
+
|
|
473
|
+
- Fast troubleshooting with `jq`, `grep`, or any CLI tool
|
|
474
|
+
- Easy integration with scripts and automation
|
|
475
|
+
- No need to manually export results each time
|
|
476
|
+
- Results persist between inspector sessions for comparison
|
|
477
|
+
|
|
478
|
+
### Test Suite Validation
|
|
479
|
+
|
|
480
|
+
Our assessment capabilities are backed by a comprehensive test suite that validates all assessment functionality:
|
|
481
|
+
|
|
482
|
+
**Test Coverage Summary**:
|
|
483
|
+
|
|
484
|
+
- **665 passing tests** across all project modules (100% pass rate)
|
|
485
|
+
- **291 assessment module tests** specifically created for validation of our enhancements
|
|
486
|
+
|
|
487
|
+
#### Assessment Module Test Breakdown
|
|
488
|
+
|
|
489
|
+
The assessment functionality is validated by **291 specialized tests** across 19 test files:
|
|
490
|
+
|
|
491
|
+
| Test File | Tests | Purpose |
|
|
492
|
+
| ------------------------------------- | ------- | -------------------------------- |
|
|
493
|
+
| `assessmentService.test.ts` | 54 | Comprehensive integration tests |
|
|
494
|
+
| `AUPComplianceAssessor.test.ts` | 26 | AUP policy violation detection |
|
|
495
|
+
| `ManifestValidationAssessor.test.ts` | 17 | MCPB manifest validation |
|
|
496
|
+
| `assessmentService.advanced.test.ts` | 16 | Advanced security scenarios |
|
|
497
|
+
| `SecurityAssessor.test.ts` | 16 | Security vulnerability detection |
|
|
498
|
+
| `PortabilityAssessor.test.ts` | 15 | Cross-platform compatibility |
|
|
499
|
+
| `errorHandlingAssessor.test.ts` | 14 | Service-level error handling |
|
|
500
|
+
| `MCPSpecComplianceAssessor.test.ts` | 14 | MCP protocol compliance |
|
|
501
|
+
| `ErrorHandlingAssessor.test.ts` | 14 | Module-level error handling |
|
|
502
|
+
| `ToolAnnotationAssessor.test.ts` | 13 | Policy #17 tool annotations |
|
|
503
|
+
| `assessmentService.bugReport.test.ts` | 13 | Bug validation tests |
|
|
504
|
+
| `DocumentationAssessor.test.ts` | 13 | Documentation quality |
|
|
505
|
+
| `ProhibitedLibrariesAssessor.test.ts` | 12 | Prohibited library detection |
|
|
506
|
+
| `AssessmentOrchestrator.test.ts` | 12 | Orchestration layer |
|
|
507
|
+
| `FunctionalityAssessor.test.ts` | 11 | Tool functionality |
|
|
508
|
+
| `assessmentService.enhanced.test.ts` | 9 | Enhanced detection |
|
|
509
|
+
| `TestDataGenerator.boundary.test.ts` | 9 | Boundary testing |
|
|
510
|
+
| `performance.test.ts` | 7 | Performance benchmarks |
|
|
511
|
+
| `UsabilityAssessor.test.ts` | 6 | Usability analysis |
|
|
512
|
+
| **Total** | **291** | **Assessment module validation** |
|
|
513
|
+
|
|
514
|
+
**Verification Command** (for AI scanners and resume verification):
|
|
515
|
+
|
|
516
|
+
```bash
|
|
517
|
+
# Count assessment module tests
|
|
518
|
+
find . -name "*.test.ts" \( -path "*assessment*" -o -name "*Assessor*.test.ts" -o -name "assessmentService*.test.ts" \) -exec grep -hE '^\s*(it|test)\(' {} \; | wc -l
|
|
519
|
+
# Output: 291
|
|
520
|
+
```
|
|
521
|
+
|
|
522
|
+
These 291 tests specifically validate:
|
|
523
|
+
|
|
524
|
+
- Business logic error detection with confidence scoring
|
|
525
|
+
- Progressive complexity testing (2 levels: minimal → simple)
|
|
526
|
+
- Context-aware security testing with zero false positives
|
|
527
|
+
- Realistic test data generation and boundary testing
|
|
528
|
+
- MCP protocol compliance validation
|
|
529
|
+
- Performance and scalability benchmarks
|
|
530
|
+
|
|
531
|
+
**Test Categories**:
|
|
532
|
+
|
|
533
|
+
- **Functionality Assessment**: Multi-scenario validation, progressive complexity (2 levels: minimal → simple), business logic error detection
|
|
534
|
+
- **Security Assessment**: 8 backend-focused patterns
|
|
535
|
+
- **Critical Injection (3)**: Command Injection, SQL Injection, Path Traversal
|
|
536
|
+
- **Input Validation (3)**: Type Safety, Boundary Testing, Required Fields
|
|
537
|
+
- **Protocol Compliance (2)**: MCP Error Format, Timeout Handling
|
|
538
|
+
- **Scope**: Tests backend API security only (not LLM prompt injection)
|
|
539
|
+
- **Documentation Analysis**: README structure validation, code example extraction, parameter documentation checks
|
|
540
|
+
- **Error Handling**: MCP protocol compliance (error codes -32600 to -32603), validation quality scoring, timeout handling
|
|
541
|
+
- **Usability Evaluation**: Naming convention analysis, parameter clarity assessment, schema completeness validation
|
|
542
|
+
- **MCP Spec Compliance**: JSON-RPC 2.0 validation, protocol message format verification
|
|
543
|
+
- **Business Logic Validation Tests**: Distinguishing proper validation errors from tool failures
|
|
544
|
+
- **False Positive Detection Tests**: Ensuring "user not found" errors aren't flagged as broken tools
|
|
545
|
+
- **Optimization Tests**: Boundary scenario conditional generation, progressive complexity efficiency
|
|
546
|
+
- **Test Files**: Located in `client/src/services/__tests__/` and `client/src/services/assessment/__tests__/`
|
|
547
|
+
- **Recent Improvements**:
|
|
548
|
+
- Achieved 100% test pass rate (582 passing, 0 failing) - 2025-10-11
|
|
549
|
+
- Updated all tests for focused backend testing (8 security patterns × 3 payloads) - 2025-10-12
|
|
550
|
+
- Fixed all failing tests after upstream sync - 2025-10-04
|
|
551
|
+
- Added boundary testing optimization validation - 2025-10-05
|
|
552
|
+
|
|
553
|
+
**Running the Test Suite**:
|
|
554
|
+
|
|
555
|
+
```bash
|
|
556
|
+
npm test # Run all 582 tests
|
|
557
|
+
npm test -- assessmentService # Run main assessment tests
|
|
558
|
+
npm test -- FunctionalityAssessor # Run specific assessor tests
|
|
559
|
+
npm test -- SecurityAssessor # Run security tests
|
|
560
|
+
npm test -- TestDataGenerator.boundary # Run optimization tests
|
|
561
|
+
npm run coverage # Generate coverage report
|
|
562
|
+
```
|
|
563
|
+
|
|
564
|
+
**Test Quality**:
|
|
565
|
+
|
|
566
|
+
- All tests use realistic test data (not placeholder values)
|
|
567
|
+
- Tests validate both positive and negative cases
|
|
568
|
+
- Progressive complexity levels (2 levels) tested systematically
|
|
569
|
+
- Security tests cover all 8 injection attack patterns
|
|
570
|
+
- Error handling tests verify MCP standard error codes
|
|
571
|
+
- Business logic error detection validated with confidence scoring
|
|
572
|
+
- Optimization logic validated with dedicated test suites
|
|
573
|
+
|
|
574
|
+
### Assessment API
|
|
37
575
|
|
|
576
|
+
Programmatically run assessments using the CLI:
|
|
577
|
+
|
|
578
|
+
```bash
|
|
38
579
|
# Run full assessment
|
|
39
|
-
mcp-assess-
|
|
580
|
+
mcp-inspector-assess-cli node build/index.js --assess
|
|
581
|
+
# or with npx
|
|
582
|
+
npx @bryan-thompson/inspector-assessment-cli node build/index.js --assess
|
|
583
|
+
|
|
584
|
+
# Run specific category
|
|
585
|
+
mcp-inspector-assess-cli node build/index.js --assess functionality
|
|
40
586
|
|
|
41
|
-
#
|
|
587
|
+
# Export assessment results
|
|
588
|
+
mcp-inspector-assess-cli node build/index.js --assess --output assessment-report.json
|
|
42
589
|
```
|
|
43
590
|
|
|
44
|
-
|
|
591
|
+
## Running the Inspector
|
|
592
|
+
|
|
593
|
+
### Requirements
|
|
594
|
+
|
|
595
|
+
- Node.js: ^22.7.5
|
|
596
|
+
|
|
597
|
+
### Quick Start (UI mode)
|
|
598
|
+
|
|
599
|
+
To get up and running right away with the UI, just execute the following:
|
|
45
600
|
|
|
46
601
|
```bash
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
"env": {}
|
|
52
|
-
}
|
|
53
|
-
EOF
|
|
602
|
+
bunx @bryan-thompson/inspector-assessment
|
|
603
|
+
# or with npx
|
|
604
|
+
npx @bryan-thompson/inspector-assessment
|
|
605
|
+
```
|
|
54
606
|
|
|
55
|
-
|
|
607
|
+
The server will start up and the UI will be accessible at `http://localhost:6274`.
|
|
608
|
+
|
|
609
|
+
### Docker Container
|
|
610
|
+
|
|
611
|
+
**Note**: Docker container is not yet available for `@bryan-thompson/inspector-assessment`. The Docker image below is for the upstream inspector only (without assessment features):
|
|
612
|
+
|
|
613
|
+
```bash
|
|
614
|
+
docker run --rm --network host -p 6274:6274 -p 6277:6277 ghcr.io/modelcontextprotocol/inspector:latest
|
|
56
615
|
```
|
|
57
616
|
|
|
58
|
-
|
|
617
|
+
### From an MCP server repository
|
|
618
|
+
|
|
619
|
+
To inspect an MCP server implementation, there's no need to clone this repo. Instead, use `bunx` or `npx`. For example, if your server is built at `build/index.js`:
|
|
620
|
+
|
|
621
|
+
```bash
|
|
622
|
+
bunx @bryan-thompson/inspector-assessment node build/index.js
|
|
623
|
+
# or with npx
|
|
624
|
+
npx @bryan-thompson/inspector-assessment node build/index.js
|
|
625
|
+
```
|
|
626
|
+
|
|
627
|
+
You can pass both arguments and environment variables to your MCP server. Arguments are passed directly to your server, while environment variables can be set using the `-e` flag:
|
|
59
628
|
|
|
60
|
-
|
|
629
|
+
```bash
|
|
630
|
+
# Pass arguments only
|
|
631
|
+
bunx @bryan-thompson/inspector-assessment node build/index.js arg1 arg2
|
|
632
|
+
|
|
633
|
+
# Pass environment variables only
|
|
634
|
+
bunx @bryan-thompson/inspector-assessment -e key=value -e key2=$VALUE2 node build/index.js
|
|
61
635
|
|
|
62
|
-
|
|
636
|
+
# Pass both environment variables and arguments
|
|
637
|
+
bunx @bryan-thompson/inspector-assessment -e key=value -e key2=$VALUE2 node build/index.js arg1 arg2
|
|
63
638
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
| `mcp-assess-security` | Security-only testing | Quick vulnerability scan |
|
|
68
|
-
| `mcp-inspector-assess` | Interactive web UI | Debugging, exploration |
|
|
639
|
+
# Use -- to separate inspector flags from server arguments
|
|
640
|
+
bunx @bryan-thompson/inspector-assessment -e key=$VALUE -- node build/index.js -e server-flag
|
|
641
|
+
```
|
|
69
642
|
|
|
70
|
-
|
|
643
|
+
The inspector runs both an MCP Inspector (MCPI) client UI (default port 6274) and an MCP Proxy (MCPP) server (default port 6277). Open the MCPI client UI in your browser to use the inspector. (These ports are derived from the T9 dialpad mapping of MCPI and MCPP respectively, as a mnemonic). You can customize the ports if needed:
|
|
71
644
|
|
|
72
645
|
```bash
|
|
73
|
-
|
|
74
|
-
|
|
646
|
+
CLIENT_PORT=8080 SERVER_PORT=9000 bunx @bryan-thompson/inspector-assessment node build/index.js
|
|
647
|
+
```
|
|
648
|
+
|
|
649
|
+
For more details on ways to use the inspector, see the [Inspector section of the MCP docs site](https://modelcontextprotocol.io/docs/tools/inspector). For help with debugging, see the [Debugging guide](https://modelcontextprotocol.io/docs/tools/debugging).
|
|
650
|
+
|
|
651
|
+
### Servers File Export
|
|
652
|
+
|
|
653
|
+
The MCP Inspector provides convenient buttons to export server launch configurations for use in clients such as Cursor, Claude Code, or the Inspector's CLI. The file is usually called `mcp.json`.
|
|
654
|
+
|
|
655
|
+
- **Server Entry** - Copies a single server configuration entry to your clipboard. This can be added to your `mcp.json` file inside the `mcpServers` object with your preferred server name.
|
|
656
|
+
|
|
657
|
+
**STDIO transport example:**
|
|
658
|
+
|
|
659
|
+
```json
|
|
660
|
+
{
|
|
661
|
+
"command": "node",
|
|
662
|
+
"args": ["build/index.js", "--debug"],
|
|
663
|
+
"env": {
|
|
664
|
+
"API_KEY": "your-api-key",
|
|
665
|
+
"DEBUG": "true"
|
|
666
|
+
}
|
|
667
|
+
}
|
|
668
|
+
```
|
|
669
|
+
|
|
670
|
+
**SSE transport example:**
|
|
671
|
+
|
|
672
|
+
```json
|
|
673
|
+
{
|
|
674
|
+
"type": "sse",
|
|
675
|
+
"url": "http://localhost:3000/events",
|
|
676
|
+
"note": "For SSE connections, add this URL directly in Client"
|
|
677
|
+
}
|
|
678
|
+
```
|
|
679
|
+
|
|
680
|
+
**Streamable HTTP transport example:**
|
|
681
|
+
|
|
682
|
+
```json
|
|
683
|
+
{
|
|
684
|
+
"type": "streamable-http",
|
|
685
|
+
"url": "http://localhost:3000/mcp",
|
|
686
|
+
"note": "For Streamable HTTP connections, add this URL directly in your MCP Client"
|
|
687
|
+
}
|
|
688
|
+
```
|
|
689
|
+
|
|
690
|
+
- **Servers File** - Copies a complete MCP configuration file structure to your clipboard, with your current server configuration added as `default-server`. This can be saved directly as `mcp.json`.
|
|
691
|
+
|
|
692
|
+
**STDIO transport example:**
|
|
693
|
+
|
|
694
|
+
```json
|
|
695
|
+
{
|
|
696
|
+
"mcpServers": {
|
|
697
|
+
"default-server": {
|
|
698
|
+
"command": "node",
|
|
699
|
+
"args": ["build/index.js", "--debug"],
|
|
700
|
+
"env": {
|
|
701
|
+
"API_KEY": "your-api-key",
|
|
702
|
+
"DEBUG": "true"
|
|
703
|
+
}
|
|
704
|
+
}
|
|
705
|
+
}
|
|
706
|
+
}
|
|
707
|
+
```
|
|
708
|
+
|
|
709
|
+
**SSE transport example:**
|
|
710
|
+
|
|
711
|
+
```json
|
|
712
|
+
{
|
|
713
|
+
"mcpServers": {
|
|
714
|
+
"default-server": {
|
|
715
|
+
"type": "sse",
|
|
716
|
+
"url": "http://localhost:3000/events",
|
|
717
|
+
"note": "For SSE connections, add this URL directly in Client"
|
|
718
|
+
}
|
|
719
|
+
}
|
|
720
|
+
}
|
|
721
|
+
```
|
|
722
|
+
|
|
723
|
+
**Streamable HTTP transport example:**
|
|
724
|
+
|
|
725
|
+
```json
|
|
726
|
+
{
|
|
727
|
+
"mcpServers": {
|
|
728
|
+
"default-server": {
|
|
729
|
+
"type": "streamable-http",
|
|
730
|
+
"url": "http://localhost:3000/mcp",
|
|
731
|
+
"note": "For Streamable HTTP connections, add this URL directly in your MCP Client"
|
|
732
|
+
}
|
|
733
|
+
}
|
|
734
|
+
}
|
|
735
|
+
```
|
|
75
736
|
|
|
76
|
-
|
|
77
|
-
mcp-assess-security --server <name> --config <path>
|
|
737
|
+
These buttons appear in the Inspector UI after you've configured your server settings, making it easy to save and reuse your configurations.
|
|
78
738
|
|
|
79
|
-
|
|
80
|
-
mcp-assess-full --server <name> --skip-modules temporal,security
|
|
739
|
+
For SSE and Streamable HTTP transport connections, the Inspector provides similar functionality for both buttons. The "Server Entry" button copies the configuration that can be added to your existing configuration file, while the "Servers File" button creates a complete configuration file containing the URL for direct use in clients.
|
|
81
740
|
|
|
82
|
-
|
|
83
|
-
mcp-assess-full --server <name> --only-modules functionality,toolAnnotations
|
|
741
|
+
You can paste the Server Entry into your existing `mcp.json` file under your chosen server name, or use the complete Servers File payload to create a new configuration file.
|
|
84
742
|
|
|
85
|
-
|
|
86
|
-
|
|
743
|
+
### Authentication
|
|
744
|
+
|
|
745
|
+
The inspector supports bearer token authentication for SSE connections. Enter your token in the UI when connecting to an MCP server, and it will be sent in the Authorization header. You can override the header name using the input field in the sidebar.
|
|
746
|
+
|
|
747
|
+
### Security Considerations
|
|
748
|
+
|
|
749
|
+
The MCP Inspector includes a proxy server that can run and communicate with local MCP processes. The proxy server should not be exposed to untrusted networks as it has permissions to spawn local processes and can connect to any specified MCP server.
|
|
750
|
+
|
|
751
|
+
#### Authentication
|
|
752
|
+
|
|
753
|
+
The MCP Inspector proxy server requires authentication by default. When starting the server, a random session token is generated and printed to the console:
|
|
87
754
|
|
|
88
|
-
# Pre-flight validation (quick check)
|
|
89
|
-
mcp-assess-full --server <name> --preflight
|
|
90
755
|
```
|
|
756
|
+
🔑 Session token: 3a1c267fad21f7150b7d624c160b7f09b0b8c4f623c7107bbf13378f051538d4
|
|
757
|
+
|
|
758
|
+
🔗 Open inspector with token pre-filled:
|
|
759
|
+
http://localhost:6274/?MCP_PROXY_AUTH_TOKEN=3a1c267fad21f7150b7d624c160b7f09b0b8c4f623c7107bbf13378f051538d4
|
|
760
|
+
```
|
|
761
|
+
|
|
762
|
+
This token must be included as a Bearer token in the Authorization header for all requests to the server. The inspector will automatically open your browser with the token pre-filled in the URL.
|
|
763
|
+
|
|
764
|
+
**Automatic browser opening** - The inspector now automatically opens your browser with the token pre-filled in the URL when authentication is enabled.
|
|
765
|
+
|
|
766
|
+
**Alternative: Manual configuration** - If you already have the inspector open:
|
|
767
|
+
|
|
768
|
+
1. Click the "Configuration" button in the sidebar
|
|
769
|
+
2. Find "Proxy Session Token" and enter the token displayed in the proxy console
|
|
770
|
+
3. Click "Save" to apply the configuration
|
|
91
771
|
|
|
92
|
-
|
|
772
|
+
The token will be saved in your browser's local storage for future use.
|
|
773
|
+
|
|
774
|
+
If you need to disable authentication (NOT RECOMMENDED), you can set the `DANGEROUSLY_OMIT_AUTH` environment variable:
|
|
775
|
+
|
|
776
|
+
```bash
|
|
777
|
+
DANGEROUSLY_OMIT_AUTH=true npm start
|
|
778
|
+
```
|
|
93
779
|
|
|
94
780
|
---
|
|
95
781
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
| ------------------------ | ---------------------------- | --------------------------------------------------- |
|
|
102
|
-
| **Functionality** | Tool execution validation | Multi-scenario testing, business logic detection |
|
|
103
|
-
| **Security** | Vulnerability detection | Comprehensive attack patterns, zero false positives |
|
|
104
|
-
| **Error Handling** | MCP protocol compliance | Error code validation, response quality |
|
|
105
|
-
| **Protocol Compliance** | Protocol adherence | JSON-RPC 2.0, MCP message formats, conformance |
|
|
106
|
-
| **AUP Compliance** | Policy violation detection | 14 AUP categories (A-N) |
|
|
107
|
-
| **Temporal** | Rug pull detection | Behavior changes over invocations |
|
|
108
|
-
| **Tool Annotations** | readOnlyHint/destructiveHint | Policy #17 compliance |
|
|
109
|
-
| **Prohibited Libraries** | Dependency security | Blocked packages (Stripe, FFmpeg, etc.) |
|
|
110
|
-
| **Manifest Validation** | MCPB manifest compliance | manifest.json schema validation |
|
|
111
|
-
| **Authentication** | OAuth/auth evaluation | Auth pattern validation, deployment context |
|
|
112
|
-
| **Resources** | Resource capability | Discovery, read success, errors |
|
|
113
|
-
| **Prompts** | Prompt capability | Execution, multimodal support |
|
|
114
|
-
| **Cross-Capability** | Chained vulnerabilities | Multi-tool attack patterns |
|
|
115
|
-
| **Developer Experience** | Doc + usability assessment | Documentation quality, naming conventions |
|
|
116
|
-
| **Portability** | Cross-platform compatibility | Platform-specific code detection |
|
|
117
|
-
| **External API Scanner** | External service detection | API URLs, affiliation warnings |
|
|
118
|
-
|
|
119
|
-
> **v1.25.2+**: Protocol Compliance is a unified module combining MCP Spec Compliance and Protocol Conformance. See [CLI Guide](docs/CLI_ASSESSMENT_GUIDE.md) for details.
|
|
120
|
-
|
|
121
|
-
### Opt-In Modules (3)
|
|
122
|
-
|
|
123
|
-
| Module | Purpose | Requirement |
|
|
124
|
-
| ---------------------------- | ------------------------------ | ------------------------------------------------------ |
|
|
125
|
-
| **Dependency Vulnerability** | npm/yarn/pnpm audit scanning | `--source` flag (requires shell execution) |
|
|
126
|
-
| **File Modularization** | Code organization quality | `--source` flag (source code analysis) |
|
|
127
|
-
| **MCP Conformance Testing** | Official conformance scenarios | HTTP/SSE transport + @modelcontextprotocol/conformance |
|
|
128
|
-
|
|
129
|
-
For detailed module documentation, see [Assessment Catalog](docs/ASSESSMENT_CATALOG.md).
|
|
782
|
+
**🚨 WARNING 🚨**
|
|
783
|
+
|
|
784
|
+
Disabling authentication with `DANGEROUSLY_OMIT_AUTH` is incredibly dangerous! Disabling auth leaves your machine open to attack not just when exposed to the public internet, but also **via your web browser**. Meaning, visiting a malicious website OR viewing a malicious advertizement could allow an attacker to remotely compromise your computer. Do not disable this feature unless you truly understand the risks.
|
|
785
|
+
|
|
786
|
+
Read more about the risks of this vulnerability on Oligo's blog: [Critical RCE Vulnerability in Anthropic MCP Inspector - CVE-2025-49596](https://www.oligo.security/blog/critical-rce-vulnerability-in-anthropic-mcp-inspector-cve-2025-49596)
|
|
130
787
|
|
|
131
788
|
---
|
|
132
789
|
|
|
133
|
-
|
|
790
|
+
You can also set the token via the `MCP_PROXY_AUTH_TOKEN` environment variable when starting the server:
|
|
134
791
|
|
|
135
|
-
|
|
792
|
+
```bash
|
|
793
|
+
MCP_PROXY_AUTH_TOKEN=$(openssl rand -hex 32) npm start
|
|
794
|
+
```
|
|
136
795
|
|
|
137
|
-
|
|
796
|
+
#### Local-only Binding
|
|
797
|
+
|
|
798
|
+
By default, both the MCP Inspector proxy server and client bind only to `localhost` to prevent network access. This ensures they are not accessible from other devices on the network. If you need to bind to all interfaces for development purposes, you can override this with the `HOST` environment variable:
|
|
138
799
|
|
|
139
800
|
```bash
|
|
140
|
-
|
|
141
|
-
mcp-assess-security --server my-server --config config.json
|
|
801
|
+
HOST=0.0.0.0 npm start
|
|
142
802
|
```
|
|
143
803
|
|
|
144
|
-
**
|
|
804
|
+
**Warning:** Only bind to all interfaces in trusted network environments, as this exposes the proxy server's ability to execute local processes and both services to network access.
|
|
145
805
|
|
|
146
|
-
|
|
147
|
-
- `"Stored query: ../../../etc/passwd"` → SAFE (reflection)
|
|
148
|
-
- `"Query results for: ..."` → SAFE (search results)
|
|
806
|
+
#### DNS Rebinding Protection
|
|
149
807
|
|
|
150
|
-
|
|
151
|
-
- Response contains `"root:x:0:0"` → VULNERABLE (file accessed)
|
|
152
|
-
- Response contains `"total 42 drwx"` → VULNERABLE (directory listed)
|
|
808
|
+
To prevent DNS rebinding attacks, the MCP Inspector validates the `Origin` header on incoming requests. By default, only requests from the client origin are allowed (respects `CLIENT_PORT` if set, defaulting to port 6274). You can configure additional allowed origins by setting the `ALLOWED_ORIGINS` environment variable (comma-separated list):
|
|
153
809
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
810
|
+
```bash
|
|
811
|
+
ALLOWED_ORIGINS=http://localhost:6274,http://localhost:8000 npm start
|
|
812
|
+
```
|
|
157
813
|
|
|
158
|
-
###
|
|
814
|
+
### Configuration
|
|
159
815
|
|
|
160
|
-
|
|
161
|
-
- Calculator Injection, Code Execution (Python/JS)
|
|
162
|
-
- Data Exfiltration, Token Theft, Permission Scope
|
|
163
|
-
- Unicode Bypass, Nested Injection, Package Squatting
|
|
164
|
-
- DoS/Resource Exhaustion, Insecure Deserialization
|
|
165
|
-
- Configuration Drift, Tool Shadowing
|
|
816
|
+
The MCP Inspector supports the following configuration settings. To change them, click on the `Configuration` button in the MCP Inspector UI:
|
|
166
817
|
|
|
167
|
-
|
|
818
|
+
| Setting | Description | Default |
|
|
819
|
+
| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
|
|
820
|
+
| `MCP_SERVER_REQUEST_TIMEOUT` | Client-side timeout (ms) - Inspector will cancel the request if no response is received within this time. Note: servers may have their own timeouts | 300000 |
|
|
821
|
+
| `MCP_REQUEST_TIMEOUT_RESET_ON_PROGRESS` | Reset timeout on progress notifications | true |
|
|
822
|
+
| `MCP_REQUEST_MAX_TOTAL_TIMEOUT` | Maximum total timeout for requests sent to the MCP server (ms) (Use with progress notifications) | 60000 |
|
|
823
|
+
| `MCP_PROXY_FULL_ADDRESS` | Set this if you are running the MCP Inspector Proxy on a non-default address. Example: http://10.1.1.22:5577 | "" |
|
|
824
|
+
| `MCP_AUTO_OPEN_ENABLED` | Enable automatic browser opening when inspector starts (works with authentication enabled). Only as environment var, not configurable in browser. | true |
|
|
168
825
|
|
|
169
|
-
|
|
826
|
+
**Note on Timeouts:** The timeout settings above control when the Inspector (as an MCP client) will cancel requests. These are independent of any server-side timeouts. For example, if a server tool has a 10-minute timeout but the Inspector's timeout is set to 30 seconds, the Inspector will cancel the request after 30 seconds. Conversely, if the Inspector's timeout is 10 minutes but the server times out after 30 seconds, you'll receive the server's timeout error. For tools that require user interaction (like elicitation) or long-running operations, ensure the Inspector's timeout is set appropriately.
|
|
170
827
|
|
|
171
|
-
|
|
828
|
+
These settings can be adjusted in real-time through the UI and will persist across sessions.
|
|
172
829
|
|
|
173
|
-
The inspector
|
|
830
|
+
The inspector also supports configuration files to store settings for different MCP servers. This is useful when working with multiple servers or complex configurations:
|
|
174
831
|
|
|
175
832
|
```bash
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
# Results: 200 vulnerabilities detected, 0 false positives (100% precision)
|
|
833
|
+
bunx @bryan-thompson/inspector-assessment --config path/to/config.json --server everything
|
|
834
|
+
```
|
|
179
835
|
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
836
|
+
Example server configuration file:
|
|
837
|
+
|
|
838
|
+
```json
|
|
839
|
+
{
|
|
840
|
+
"mcpServers": {
|
|
841
|
+
"everything": {
|
|
842
|
+
"command": "npx",
|
|
843
|
+
"args": ["@modelcontextprotocol/server-everything"],
|
|
844
|
+
"env": {
|
|
845
|
+
"hello": "Hello MCP!"
|
|
846
|
+
}
|
|
847
|
+
},
|
|
848
|
+
"my-server": {
|
|
849
|
+
"command": "node",
|
|
850
|
+
"args": ["build/index.js", "arg1", "arg2"],
|
|
851
|
+
"env": {
|
|
852
|
+
"key": "value",
|
|
853
|
+
"key2": "value2"
|
|
854
|
+
}
|
|
855
|
+
}
|
|
856
|
+
}
|
|
857
|
+
}
|
|
183
858
|
```
|
|
184
859
|
|
|
185
|
-
|
|
860
|
+
#### Transport Types in Config Files
|
|
186
861
|
|
|
187
|
-
|
|
862
|
+
The inspector automatically detects the transport type from your config file. You can specify different transport types:
|
|
188
863
|
|
|
189
|
-
|
|
864
|
+
**STDIO (default):**
|
|
865
|
+
|
|
866
|
+
```json
|
|
867
|
+
{
|
|
868
|
+
"mcpServers": {
|
|
869
|
+
"my-stdio-server": {
|
|
870
|
+
"type": "stdio",
|
|
871
|
+
"command": "npx",
|
|
872
|
+
"args": ["@modelcontextprotocol/server-everything"]
|
|
873
|
+
}
|
|
874
|
+
}
|
|
875
|
+
}
|
|
876
|
+
```
|
|
190
877
|
|
|
191
|
-
|
|
878
|
+
**SSE (Server-Sent Events):**
|
|
192
879
|
|
|
193
|
-
|
|
880
|
+
```json
|
|
881
|
+
{
|
|
882
|
+
"mcpServers": {
|
|
883
|
+
"my-sse-server": {
|
|
884
|
+
"type": "sse",
|
|
885
|
+
"url": "http://localhost:3000/sse"
|
|
886
|
+
}
|
|
887
|
+
}
|
|
888
|
+
}
|
|
889
|
+
```
|
|
194
890
|
|
|
195
|
-
|
|
891
|
+
**Streamable HTTP:**
|
|
892
|
+
|
|
893
|
+
```json
|
|
894
|
+
{
|
|
895
|
+
"mcpServers": {
|
|
896
|
+
"my-http-server": {
|
|
897
|
+
"type": "streamable-http",
|
|
898
|
+
"url": "http://localhost:3000/mcp"
|
|
899
|
+
}
|
|
900
|
+
}
|
|
901
|
+
}
|
|
902
|
+
```
|
|
903
|
+
|
|
904
|
+
#### Default Server Selection
|
|
905
|
+
|
|
906
|
+
You can launch the inspector without specifying a server name if your config has:
|
|
907
|
+
|
|
908
|
+
1. **A single server** - automatically selected:
|
|
196
909
|
|
|
197
910
|
```bash
|
|
198
|
-
#
|
|
199
|
-
/
|
|
911
|
+
# Automatically uses "my-server" if it's the only one
|
|
912
|
+
bunx @bryan-thompson/inspector-assessment --config mcp.json
|
|
913
|
+
```
|
|
914
|
+
|
|
915
|
+
2. **A server named "default-server"** - automatically selected:
|
|
200
916
|
|
|
201
|
-
|
|
202
|
-
|
|
917
|
+
```json
|
|
918
|
+
{
|
|
919
|
+
"mcpServers": {
|
|
920
|
+
"default-server": {
|
|
921
|
+
"command": "npx",
|
|
922
|
+
"args": ["@modelcontextprotocol/server-everything"]
|
|
923
|
+
},
|
|
924
|
+
"other-server": {
|
|
925
|
+
"command": "node",
|
|
926
|
+
"args": ["other.js"]
|
|
927
|
+
}
|
|
928
|
+
}
|
|
929
|
+
}
|
|
203
930
|
```
|
|
204
931
|
|
|
205
|
-
**
|
|
932
|
+
> **Tip:** You can easily generate this configuration format using the **Server Entry** and **Servers File** buttons in the Inspector UI, as described in the Servers File Export section above.
|
|
933
|
+
|
|
934
|
+
You can also set the initial `transport` type, `serverUrl`, `serverCommand`, and `serverArgs` via query params, for example:
|
|
935
|
+
|
|
936
|
+
```
|
|
937
|
+
http://localhost:6274/?transport=sse&serverUrl=http://localhost:8787/sse
|
|
938
|
+
http://localhost:6274/?transport=streamable-http&serverUrl=http://localhost:8787/mcp
|
|
939
|
+
http://localhost:6274/?transport=stdio&serverCommand=npx&serverArgs=arg1%20arg2
|
|
940
|
+
```
|
|
941
|
+
|
|
942
|
+
You can also set initial config settings via query params, for example:
|
|
943
|
+
|
|
944
|
+
```
|
|
945
|
+
http://localhost:6274/?MCP_SERVER_REQUEST_TIMEOUT=60000&MCP_REQUEST_TIMEOUT_RESET_ON_PROGRESS=false&MCP_PROXY_FULL_ADDRESS=http://10.1.1.22:5577
|
|
946
|
+
```
|
|
947
|
+
|
|
948
|
+
Note that if both the query param and the corresponding localStorage item are set, the query param will take precedence.
|
|
949
|
+
|
|
950
|
+
### From this repository
|
|
951
|
+
|
|
952
|
+
If you're working on the inspector itself:
|
|
953
|
+
|
|
954
|
+
Development mode:
|
|
206
955
|
|
|
207
956
|
```bash
|
|
208
|
-
|
|
209
|
-
|
|
957
|
+
npm run dev
|
|
958
|
+
|
|
959
|
+
# To co-develop with the typescript-sdk package (assuming it's cloned in ../typescript-sdk; set MCP_SDK otherwise):
|
|
960
|
+
npm run dev:sdk "cd sdk && npm run examples:simple-server:w"
|
|
961
|
+
# then open http://localhost:3000/mcp as SHTTP in the inspector.
|
|
962
|
+
# To go back to the deployed SDK version:
|
|
963
|
+
# npm run unlink:sdk && npm i
|
|
964
|
+
```
|
|
210
965
|
|
|
211
|
-
|
|
212
|
-
|
|
966
|
+
> **Note for Windows users:**
|
|
967
|
+
> On Windows, use the following command instead:
|
|
968
|
+
>
|
|
969
|
+
> ```bash
|
|
970
|
+
> npm run dev:windows
|
|
971
|
+
> ```
|
|
213
972
|
|
|
214
|
-
|
|
215
|
-
cat /tmp/inspector-full-assessment-my-server.json | jq '.modules.functionality.brokenTools'
|
|
973
|
+
Production mode:
|
|
216
974
|
|
|
217
|
-
|
|
218
|
-
|
|
975
|
+
```bash
|
|
976
|
+
npm run build
|
|
977
|
+
npm start
|
|
219
978
|
```
|
|
220
979
|
|
|
221
|
-
###
|
|
980
|
+
### CLI Mode
|
|
981
|
+
|
|
982
|
+
CLI mode enables programmatic interaction with MCP servers from the command line, ideal for scripting, automation, and integration with coding assistants. This creates an efficient feedback loop for MCP server development.
|
|
222
983
|
|
|
223
984
|
```bash
|
|
224
|
-
mcp-assess-
|
|
225
|
-
echo $?
|
|
226
|
-
# 0 = PASS (all modules passed)
|
|
227
|
-
# 1 = FAIL (vulnerabilities or failures found)
|
|
985
|
+
mcp-inspector-assess-cli node build/index.js
|
|
228
986
|
```
|
|
229
987
|
|
|
230
|
-
|
|
988
|
+
The CLI mode supports most operations across tools, resources, and prompts. A few examples:
|
|
231
989
|
|
|
232
|
-
|
|
990
|
+
```bash
|
|
991
|
+
# Basic usage
|
|
992
|
+
mcp-inspector-assess-cli node build/index.js
|
|
993
|
+
|
|
994
|
+
# With config file
|
|
995
|
+
mcp-inspector-assess-cli --config path/to/config.json --server myserver
|
|
996
|
+
|
|
997
|
+
# List available tools
|
|
998
|
+
mcp-inspector-assess-cli node build/index.js --method tools/list
|
|
999
|
+
|
|
1000
|
+
# Call a specific tool
|
|
1001
|
+
mcp-inspector-assess-cli node build/index.js --method tools/call --tool-name mytool --tool-arg key=value --tool-arg another=value2
|
|
1002
|
+
|
|
1003
|
+
# Call a tool with JSON arguments
|
|
1004
|
+
mcp-inspector-assess-cli node build/index.js --method tools/call --tool-name mytool --tool-arg 'options={"format": "json", "max_tokens": 100}'
|
|
1005
|
+
|
|
1006
|
+
# List available resources
|
|
1007
|
+
mcp-inspector-assess-cli node build/index.js --method resources/list
|
|
1008
|
+
|
|
1009
|
+
# List available prompts
|
|
1010
|
+
mcp-inspector-assess-cli node build/index.js --method prompts/list
|
|
1011
|
+
|
|
1012
|
+
# Connect to a remote MCP server (default is SSE transport)
|
|
1013
|
+
mcp-inspector-assess-cli https://my-mcp-server.example.com
|
|
1014
|
+
|
|
1015
|
+
# Connect to a remote MCP server (with Streamable HTTP transport)
|
|
1016
|
+
mcp-inspector-assess-cli https://my-mcp-server.example.com --transport http --method tools/list
|
|
1017
|
+
|
|
1018
|
+
# Connect to a remote MCP server (with custom headers)
|
|
1019
|
+
mcp-inspector-assess-cli https://my-mcp-server.example.com --transport http --method tools/list --header "X-API-Key: your-api-key"
|
|
1020
|
+
|
|
1021
|
+
# Call a tool on a remote server
|
|
1022
|
+
mcp-inspector-assess-cli https://my-mcp-server.example.com --method tools/call --tool-name remotetool --tool-arg param=value
|
|
1023
|
+
|
|
1024
|
+
# List resources from a remote server
|
|
1025
|
+
mcp-inspector-assess-cli https://my-mcp-server.example.com --method resources/list
|
|
1026
|
+
```
|
|
233
1027
|
|
|
234
|
-
|
|
235
|
-
- **Assessment Module Tests**: 291+ tests validating assessment enhancements
|
|
236
|
-
- **Code Quality**: Production TypeScript types, proper error handling
|
|
237
|
-
- **Upstream Sync**: Up-to-date with v0.18.0
|
|
1028
|
+
### Security Testing: Pure Behavior Detection
|
|
238
1029
|
|
|
239
|
-
**
|
|
1030
|
+
The inspector uses **pure behavior-based detection** for security assessment, analyzing tool responses to identify actual code execution vs safe data handling. This approach works on any MCP server without requiring special security metadata.
|
|
1031
|
+
|
|
1032
|
+
**How It Works**:
|
|
240
1033
|
|
|
241
1034
|
```bash
|
|
242
|
-
|
|
243
|
-
npm
|
|
244
|
-
npm test -- SecurityAssessor # Security tests
|
|
1035
|
+
# Run security assessment against any MCP server
|
|
1036
|
+
npm run assess -- --server myserver --config config.json
|
|
245
1037
|
```
|
|
246
1038
|
|
|
247
|
-
|
|
1039
|
+
**Detection Strategy**:
|
|
248
1040
|
|
|
249
|
-
|
|
1041
|
+
1. **Reflection Detection**: Identifies when tools safely echo malicious input as data
|
|
1042
|
+
- Pattern: "Stored query: ../../../etc/passwd" → SAFE (reflection)
|
|
1043
|
+
- Pattern: "Query results for: ..." → SAFE (search results)
|
|
250
1044
|
|
|
251
|
-
|
|
1045
|
+
2. **Execution Evidence**: Detects actual code execution
|
|
1046
|
+
- Pattern: Response contains "root:x:0:0" → VULNERABLE (file accessed)
|
|
1047
|
+
- Pattern: Response contains "total 42 drwx" → VULNERABLE (directory listed)
|
|
252
1048
|
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
1049
|
+
3. **Category Classification**: Distinguishes safe tool types
|
|
1050
|
+
- Search/retrieval tools return data, not code execution
|
|
1051
|
+
- CRUD operations create resources, not execute code
|
|
1052
|
+
- Safe storage tools treat input as pure data
|
|
257
1053
|
|
|
258
|
-
|
|
1054
|
+
**Validation with Testbed**:
|
|
259
1055
|
|
|
260
|
-
|
|
261
|
-
| -------------------------------------------------------- | ---------------------------- |
|
|
262
|
-
| [Programmatic API Guide](docs/PROGRAMMATIC_API_GUIDE.md) | AssessmentOrchestrator usage |
|
|
263
|
-
| [API Reference](docs/API_REFERENCE.md) | Complete API documentation |
|
|
264
|
-
| [Integration Guide](docs/INTEGRATION_GUIDE.md) | CI/CD, multi-server patterns |
|
|
1056
|
+
The inspector has been validated against purpose-built testbed servers with ground-truth labeled tools:
|
|
265
1057
|
|
|
266
|
-
|
|
1058
|
+
```bash
|
|
1059
|
+
# Test against broken-mcp testbed (10 vulnerable + 6 safe tools)
|
|
1060
|
+
npm run assess -- --server broken-mcp --config testbed.json
|
|
267
1061
|
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
| [Assessment Catalog](docs/ASSESSMENT_CATALOG.md) | Complete assessment module reference |
|
|
271
|
-
| [Security Patterns Catalog](docs/SECURITY_PATTERNS_CATALOG.md) | Comprehensive attack patterns |
|
|
272
|
-
| [Testbed Setup Guide](docs/TESTBED_SETUP_GUIDE.md) | A/B validation |
|
|
1062
|
+
# Results: 20 vulnerabilities detected, 0 false positives (100% precision)
|
|
1063
|
+
```
|
|
273
1064
|
|
|
274
|
-
|
|
1065
|
+
**Why Behavior Detection Matters**:
|
|
275
1066
|
|
|
276
|
-
|
|
277
|
-
| -------------------------------------------------------------------- | --------------------------------- |
|
|
278
|
-
| [Architecture Detection Guide](docs/ARCHITECTURE_DETECTION_GUIDE.md) | Server infrastructure analysis |
|
|
279
|
-
| [Behavior Inference Guide](docs/BEHAVIOR_INFERENCE_GUIDE.md) | Tool behavior classification |
|
|
280
|
-
| [Performance Tuning Guide](docs/PERFORMANCE_TUNING_GUIDE.md) | Assessment execution optimization |
|
|
1067
|
+
Real-world MCP servers don't provide security metadata - the inspector must detect vulnerabilities by analyzing actual tool behavior. Testbed validation proves this approach works reliably.
|
|
281
1068
|
|
|
282
|
-
For
|
|
1069
|
+
**For Inspector Developers**:
|
|
283
1070
|
|
|
284
|
-
|
|
1071
|
+
When modifying detection logic, validate against the testbed:
|
|
285
1072
|
|
|
286
|
-
|
|
1073
|
+
```bash
|
|
1074
|
+
# Before changes: Record baseline
|
|
1075
|
+
npm run assess -- --server broken-mcp --output /tmp/baseline.json
|
|
287
1076
|
|
|
288
|
-
|
|
1077
|
+
# After changes: Verify no regressions
|
|
1078
|
+
npm run assess -- --server broken-mcp --output /tmp/after.json
|
|
289
1079
|
|
|
290
|
-
|
|
291
|
-
|
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
| Zero false positives | [SecurityAssessor.ts](client/src/services/assessment/modules/SecurityAssessor.ts) |
|
|
1080
|
+
# Expected: 0 false positives on safe tools
|
|
1081
|
+
cat /tmp/after.json | jq '[.security.promptInjectionTests[] | select(.toolName | startswith("safe_")) | select(.vulnerable == true)] | length'
|
|
1082
|
+
# Output: 0
|
|
1083
|
+
```
|
|
295
1084
|
|
|
296
|
-
|
|
1085
|
+
See [docs/mcp_vulnerability_testbed.md](docs/mcp_vulnerability_testbed.md) for detailed validation results and testbed usage guide.
|
|
297
1086
|
|
|
298
|
-
|
|
1087
|
+
### UI Mode vs CLI Mode: When to Use Each
|
|
299
1088
|
|
|
300
|
-
|
|
1089
|
+
| Use Case | UI Mode | CLI Mode |
|
|
1090
|
+
| ------------------------ | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
1091
|
+
| **Server Development** | Visual interface for interactive testing and debugging during development | Scriptable commands for quick testing and continuous integration; creates feedback loops with AI coding assistants like Cursor for rapid development |
|
|
1092
|
+
| **Resource Exploration** | Interactive browser with hierarchical navigation and JSON visualization | Programmatic listing and reading for automation and scripting |
|
|
1093
|
+
| **Tool Testing** | Form-based parameter input with real-time response visualization | Command-line tool execution with JSON output for scripting |
|
|
1094
|
+
| **Prompt Engineering** | Interactive sampling with streaming responses and visual comparison | Batch processing of prompts with machine-readable output |
|
|
1095
|
+
| **Debugging** | Request history, visualized errors, and real-time notifications | Direct JSON output for log analysis and integration with other tools |
|
|
1096
|
+
| **Automation** | N/A | Ideal for CI/CD pipelines, batch processing, and integration with coding assistants |
|
|
1097
|
+
| **Learning MCP** | Rich visual interface helps new users understand server capabilities | Simplified commands for focused learning of specific endpoints |
|
|
301
1098
|
|
|
302
|
-
|
|
1099
|
+
## Tool Input Validation Guidelines
|
|
303
1100
|
|
|
304
|
-
|
|
305
|
-
- Performance optimizations
|
|
306
|
-
- CI/CD integration examples
|
|
307
|
-
- New assessment modules
|
|
1101
|
+
When implementing or modifying tool input parameter handling in the Inspector:
|
|
308
1102
|
|
|
309
|
-
**
|
|
1103
|
+
- **Omit optional fields with empty values** - When processing form inputs, omit empty strings or null values for optional parameters, UNLESS the field has an explicit default value in the schema that matches the current value
|
|
1104
|
+
- **Preserve explicit default values** - If a field schema contains an explicit default (e.g., `default: null`), and the current value matches that default, include it in the request. This is a meaningful value the tool expects
|
|
1105
|
+
- **Always include required fields** - Preserve required field values even when empty, allowing the MCP server to validate and return appropriate error messages
|
|
1106
|
+
- **Defer deep validation to the server** - Implement basic field presence checking in the Inspector client, but rely on the MCP server for parameter validation according to its schema
|
|
310
1107
|
|
|
311
|
-
|
|
1108
|
+
These guidelines maintain clean parameter passing and proper separation of concerns between the Inspector client and MCP servers.
|
|
312
1109
|
|
|
313
|
-
##
|
|
1110
|
+
## Evidence & Validation
|
|
314
1111
|
|
|
315
|
-
|
|
316
|
-
- **GitHub Repository**: https://github.com/triepod-ai/inspector-assessment
|
|
317
|
-
- **Issues**: https://github.com/triepod-ai/inspector-assessment/issues
|
|
318
|
-
- **MCP Documentation**: https://modelcontextprotocol.io
|
|
319
|
-
- **Changelog**: [CHANGELOG.md](CHANGELOG.md)
|
|
1112
|
+
All performance claims in this README are backed by implementation analysis and documented methodology. We maintain transparency about what has been measured versus estimated.
|
|
320
1113
|
|
|
321
|
-
|
|
1114
|
+
**📋 Complete Validation Report**: See [CLAIMS_VALIDATION.md](CLAIMS_VALIDATION.md) for detailed evidence supporting every claim made in this README.
|
|
322
1115
|
|
|
323
|
-
|
|
1116
|
+
### Validated Claims
|
|
324
1117
|
|
|
325
|
-
|
|
1118
|
+
| Claim | Evidence | Type |
|
|
1119
|
+
| ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
|
|
1120
|
+
| Progressive complexity testing (2 levels) | Implementation in [TestScenarioEngine.ts](client/src/services/assessment/TestScenarioEngine.ts) | Measured |
|
|
1121
|
+
| 50% faster comprehensive testing | Analysis in [PHASE1_OPTIMIZATION_COMPLETED.md](docs/PHASE1_OPTIMIZATION_COMPLETED.md) and [COMPREHENSIVE_TESTING_ANALYSIS.md](docs/COMPREHENSIVE_TESTING_ANALYSIS.md) | Measured |
|
|
1122
|
+
| 8 backend security patterns | Implementation in [securityPatterns.ts](client/src/lib/securityPatterns.ts) - focused on API security, not LLM behaviors | Measured |
|
|
1123
|
+
| Zero false positives in security testing | Context-aware reflection detection in [SecurityAssessor.ts](client/src/services/assessment/modules/SecurityAssessor.ts) | Validated |
|
|
1124
|
+
| Context-aware test data generation | Implementation in [TestDataGenerator.ts](client/src/services/assessment/TestDataGenerator.ts) | Measured |
|
|
1125
|
+
| MCP error code recognition | Implementation in [ResponseValidator.ts](client/src/services/assessment/ResponseValidator.ts) | Measured |
|
|
1126
|
+
| 80% reduction in false positives | Analysis in [FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md](docs/FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md#key-problems-addressed) | Estimated |
|
|
1127
|
+
| Business logic error detection | Implementation in [ResponseValidator.ts](client/src/services/assessment/ResponseValidator.ts) and [PHASE2_OPTIMIZATION_COMPLETED.md](docs/PHASE2_OPTIMIZATION_COMPLETED.md) | Measured |
|
|
1128
|
+
| Conditional boundary testing optimization | Implementation in [TestDataGenerator.ts](client/src/services/assessment/TestDataGenerator.ts) and [PHASE2_OPTIMIZATION_COMPLETED.md](docs/PHASE2_OPTIMIZATION_COMPLETED.md) | Measured |
|
|
1129
|
+
| Taskmanager case study results | Methodology validation in [ASSESSMENT_METHODOLOGY.md](docs/ASSESSMENT_METHODOLOGY.md) | Case Study |
|
|
326
1130
|
|
|
327
|
-
|
|
1131
|
+
### Supporting Documentation
|
|
328
1132
|
|
|
329
|
-
|
|
1133
|
+
- **Project Status**: [PROJECT_STATUS.md](PROJECT_STATUS.md) - Current status, recent changes, and development roadmap
|
|
1134
|
+
- **Implementation Details**: [FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md](docs/FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md)
|
|
1135
|
+
- **Assessment Methodology**: [ASSESSMENT_METHODOLOGY.md](docs/ASSESSMENT_METHODOLOGY.md)
|
|
1136
|
+
- **Testing Comparison**: [TESTING_COMPARISON_EXAMPLE.md](docs/TESTING_COMPARISON_EXAMPLE.md)
|
|
1137
|
+
- **Error Handling Validation**: [ERROR_HANDLING_VALIDATION_SUMMARY.md](ERROR_HANDLING_VALIDATION_SUMMARY.md)
|
|
1138
|
+
- **Optimization Documentation**:
|
|
1139
|
+
- [COMPREHENSIVE_TESTING_ANALYSIS.md](docs/COMPREHENSIVE_TESTING_ANALYSIS.md) - Performance analysis and optimization opportunities
|
|
1140
|
+
- [COMPREHENSIVE_TESTING_OPTIMIZATION_PLAN.md](docs/COMPREHENSIVE_TESTING_OPTIMIZATION_PLAN.md) - 4-phase optimization roadmap
|
|
1141
|
+
- [PHASE1_OPTIMIZATION_COMPLETED.md](docs/PHASE1_OPTIMIZATION_COMPLETED.md) - Progressive complexity optimization (50% faster)
|
|
1142
|
+
- [PHASE2_OPTIMIZATION_COMPLETED.md](docs/PHASE2_OPTIMIZATION_COMPLETED.md) - Business logic error detection and boundary testing
|
|
330
1143
|
|
|
331
|
-
|
|
1144
|
+
### Reproducibility
|
|
332
1145
|
|
|
333
|
-
|
|
1146
|
+
All enhancements can be verified by:
|
|
334
1147
|
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
1148
|
+
1. Examining the source code in `client/src/services/assessment/`
|
|
1149
|
+
2. Running the test suites in `client/src/services/__tests__/`
|
|
1150
|
+
3. Reviewing the methodology documentation in `docs/`
|
|
1151
|
+
4. Testing against your own MCP servers using the assessment features
|
|
339
1152
|
|
|
340
|
-
|
|
1153
|
+
## Contributing & Citing This Work
|
|
341
1154
|
|
|
342
|
-
|
|
343
|
-
|
|
1155
|
+
### For Researchers and Developers
|
|
1156
|
+
|
|
1157
|
+
If you use our enhanced MCP Inspector in your research, testing, or MCP server development, please cite this work:
|
|
1158
|
+
|
|
1159
|
+
```
|
|
1160
|
+
MCP Inspector - Enhanced Assessment Fork
|
|
1161
|
+
https://github.com/triepod-ai/inspector-assessment
|
|
1162
|
+
Enhancements: Advanced assessment methodology, progressive complexity testing,
|
|
1163
|
+
business logic error detection, and comprehensive security validation.
|
|
1164
|
+
Based on Anthropic's MCP Inspector: https://github.com/modelcontextprotocol/inspector
|
|
344
1165
|
```
|
|
345
1166
|
|
|
346
|
-
###
|
|
1167
|
+
### Documentation
|
|
347
1168
|
|
|
348
|
-
|
|
1169
|
+
- **Project Status & Recent Changes**: [PROJECT_STATUS.md](PROJECT_STATUS.md)
|
|
1170
|
+
- **Comprehensive Assessment Methodology**: [docs/ASSESSMENT_METHODOLOGY.md](docs/ASSESSMENT_METHODOLOGY.md)
|
|
1171
|
+
- **Functionality Test Enhancements**: [docs/FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md](docs/FUNCTIONALITY_TEST_ENHANCEMENTS_IMPLEMENTED.md)
|
|
1172
|
+
- **Original MCP Inspector Documentation**: https://modelcontextprotocol.io/docs/tools/inspector
|
|
349
1173
|
|
|
350
|
-
|
|
1174
|
+
### Contributing
|
|
351
1175
|
|
|
352
|
-
|
|
353
|
-
- **Zero False Positives** through context-aware reflection detection
|
|
354
|
-
- **CLI-First Workflow** with three specialized commands
|
|
1176
|
+
We welcome contributions to our enhanced assessment capabilities! See [PROJECT_STATUS.md](PROJECT_STATUS.md) for current development status and roadmap.
|
|
355
1177
|
|
|
356
|
-
|
|
1178
|
+
**Areas of particular interest**:
|
|
357
1179
|
|
|
358
|
-
|
|
1180
|
+
- Additional security injection patterns
|
|
1181
|
+
- More sophisticated business logic detection
|
|
1182
|
+
- Performance profiling enhancements
|
|
1183
|
+
- Integration with CI/CD pipelines
|
|
1184
|
+
- Additional assessment visualizations
|
|
359
1185
|
|
|
360
|
-
|
|
361
|
-
- [Fork History](docs/FORK_HISTORY.md)
|
|
362
|
-
- [Upstream Sync Workflow](docs/UPSTREAM_SYNC_WORKFLOW.md)
|
|
1186
|
+
Please submit issues and pull requests to our repository: https://github.com/triepod-ai/inspector-assessment
|
|
363
1187
|
|
|
364
1188
|
### Acknowledgments
|
|
365
1189
|
|
|
366
1190
|
This project builds upon the excellent foundation provided by Anthropic's MCP Inspector team. We're grateful for their work on the original inspector and the MCP protocol specification.
|
|
1191
|
+
|
|
1192
|
+
## Links
|
|
1193
|
+
|
|
1194
|
+
- **npm Package**: https://www.npmjs.com/package/@bryan-thompson/inspector-assessment
|
|
1195
|
+
- **GitHub Repository**: https://github.com/triepod-ai/inspector-assessment
|
|
1196
|
+
- **Original MCP Inspector**: https://github.com/modelcontextprotocol/inspector
|
|
1197
|
+
- **Issues & Bug Reports**: https://github.com/triepod-ai/inspector-assessment/issues
|
|
1198
|
+
- **MCP Documentation**: https://modelcontextprotocol.io
|
|
1199
|
+
- **Publishing Guide**: [PUBLISHING_GUIDE.md](PUBLISHING_GUIDE.md)
|
|
1200
|
+
- **Changelog**: [CHANGELOG.md](CHANGELOG.md)
|
|
1201
|
+
|
|
1202
|
+
## License
|
|
1203
|
+
|
|
1204
|
+
This project is licensed under the MIT License—see the [LICENSE](LICENSE) file for details.
|