@bryan-thompson/inspector-assessment 1.23.8 → 1.23.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +38 -54
- package/cli/build/assess-full.js +10 -4
- package/cli/build/assess-security.js +7 -10
- package/client/dist/assets/{OAuthCallback-Dx-ZyJe7.js → OAuthCallback-8m79-pFb.js} +1 -1
- package/client/dist/assets/{OAuthDebugCallback-BPbrWgKx.js → OAuthDebugCallback-CL_C0bio.js} +1 -1
- package/client/dist/assets/{index-P1wj3llX.js → index-C9mj2Mhr.js} +4 -4
- package/client/dist/index.html +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
[](https://www.npmjs.com/package/@bryan-thompson/inspector-assessment)
|
|
4
4
|
[](https://www.npmjs.com/package/@bryan-thompson/inspector-assessment)
|
|
5
5
|
|
|
6
|
-
**Comprehensive MCP server validation with
|
|
6
|
+
**Comprehensive MCP server validation with 17 automated assessment modules.**
|
|
7
7
|
Test functionality, security, documentation, and policy compliance from the command line.
|
|
8
8
|
|
|
9
9
|

|
|
@@ -63,7 +63,7 @@ The inspector provides three CLI commands for different workflows:
|
|
|
63
63
|
|
|
64
64
|
| Command | Purpose | Use Case |
|
|
65
65
|
| ---------------------- | ----------------------------- | ---------------------------- |
|
|
66
|
-
| `mcp-assess-full` | Complete
|
|
66
|
+
| `mcp-assess-full` | Complete 17-module assessment | Full validation, CI/CD gates |
|
|
67
67
|
| `mcp-assess-security` | Security-only testing | Quick vulnerability scan |
|
|
68
68
|
| `mcp-inspector-assess` | Interactive web UI | Debugging, exploration |
|
|
69
69
|
|
|
@@ -93,54 +93,39 @@ For complete CLI documentation, see [CLI Assessment Guide](docs/CLI_ASSESSMENT_G
|
|
|
93
93
|
|
|
94
94
|
---
|
|
95
95
|
|
|
96
|
-
## Assessment Modules (
|
|
97
|
-
|
|
98
|
-
### Core Modules (
|
|
99
|
-
|
|
100
|
-
| Module
|
|
101
|
-
|
|
|
102
|
-
| **Functionality**
|
|
103
|
-
| **Security**
|
|
104
|
-
| **Documentation**
|
|
105
|
-
| **Error Handling**
|
|
106
|
-
| **Usability**
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
|
111
|
-
|
|
|
112
|
-
| **
|
|
113
|
-
| **
|
|
114
|
-
| **
|
|
115
|
-
| **
|
|
116
|
-
| **
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
|
122
|
-
|
|
|
123
|
-
| **
|
|
124
|
-
| **Temporal** | Rug pull detection | Behavior changes over invocations |
|
|
125
|
-
| **Resources** | Resource capability | Discovery, read success, errors |
|
|
126
|
-
| **Prompts** | Prompt capability | Execution, multimodal support |
|
|
127
|
-
| **Cross-Capability** | Chained vulnerabilities | Multi-tool attack patterns |
|
|
96
|
+
## Assessment Modules (17 Total)
|
|
97
|
+
|
|
98
|
+
### Core Modules (15)
|
|
99
|
+
|
|
100
|
+
| Module | Purpose | Key Features |
|
|
101
|
+
| ------------------------ | ---------------------------- | --------------------------------------------------- |
|
|
102
|
+
| **Functionality** | Tool execution validation | Multi-scenario testing, business logic detection |
|
|
103
|
+
| **Security** | Vulnerability detection | Comprehensive attack patterns, zero false positives |
|
|
104
|
+
| **Documentation** | README/description quality | Completeness scoring, example validation |
|
|
105
|
+
| **Error Handling** | MCP protocol compliance | Error code validation, response quality |
|
|
106
|
+
| **Usability** | Developer experience | Naming conventions, schema completeness |
|
|
107
|
+
| **MCP Spec Compliance** | Protocol adherence | JSON-RPC 2.0, MCP message formats |
|
|
108
|
+
| **AUP Compliance** | Policy violation detection | 14 AUP categories (A-N) |
|
|
109
|
+
| **Tool Annotations** | readOnlyHint/destructiveHint | Policy #17 compliance |
|
|
110
|
+
| **Prohibited Libraries** | Dependency security | Blocked packages (Stripe, FFmpeg, etc.) |
|
|
111
|
+
| **External API Scanner** | External service detection | API URLs, affiliation warnings |
|
|
112
|
+
| **Authentication** | OAuth/auth evaluation | Auth pattern validation, deployment context |
|
|
113
|
+
| **Temporal** | Rug pull detection | Behavior changes over invocations |
|
|
114
|
+
| **Resources** | Resource capability | Discovery, read success, errors |
|
|
115
|
+
| **Prompts** | Prompt capability | Execution, multimodal support |
|
|
116
|
+
| **Cross-Capability** | Chained vulnerabilities | Multi-tool attack patterns |
|
|
117
|
+
|
|
118
|
+
### Optional Modules (2) - MCPB Bundles
|
|
119
|
+
|
|
120
|
+
| Module | Purpose | Policy Alignment |
|
|
121
|
+
| ----------------------- | ---------------------------- | --------------------------------------- |
|
|
122
|
+
| **Manifest Validation** | MCPB manifest.json | manifest_version 0.3 spec |
|
|
123
|
+
| **Portability** | Cross-platform compatibility | Hardcoded paths, platform-specific code |
|
|
128
124
|
|
|
129
125
|
For detailed module documentation, see [Assessment Catalog](docs/ASSESSMENT_CATALOG.md).
|
|
130
126
|
|
|
131
127
|
---
|
|
132
128
|
|
|
133
|
-
## For MCP Directory Reviewers
|
|
134
|
-
|
|
135
|
-
If you're reviewing MCP servers for the Anthropic MCP Directory, see our **[Reviewer Quick Start Guide](docs/REVIEWER_QUICK_START.md)** for:
|
|
136
|
-
|
|
137
|
-
- **60-second fast screening** workflow for approve/reject decisions
|
|
138
|
-
- **5-minute detailed review** process for borderline cases
|
|
139
|
-
- **Common pitfalls** explanation (false positives in security, informational vs scored tests)
|
|
140
|
-
- **Decision matrix** with clear approval criteria
|
|
141
|
-
|
|
142
|
-
---
|
|
143
|
-
|
|
144
129
|
## Security Testing: Pure Behavior Detection
|
|
145
130
|
|
|
146
131
|
The inspector uses **pure behavior-based detection** for security assessment, analyzing tool responses to identify actual code execution vs safe data handling.
|
|
@@ -242,7 +227,7 @@ echo $?
|
|
|
242
227
|
|
|
243
228
|
## Quality Metrics
|
|
244
229
|
|
|
245
|
-
- **Test Coverage**: ~
|
|
230
|
+
- **Test Coverage**: ~1560 tests passing across 66 test suites
|
|
246
231
|
- **Assessment Module Tests**: 291+ tests validating assessment enhancements
|
|
247
232
|
- **Code Quality**: Production TypeScript types, proper error handling
|
|
248
233
|
- **Upstream Sync**: Up-to-date with v0.18.0
|
|
@@ -250,7 +235,7 @@ echo $?
|
|
|
250
235
|
**Run tests:**
|
|
251
236
|
|
|
252
237
|
```bash
|
|
253
|
-
npm test # All ~
|
|
238
|
+
npm test # All ~1560 tests
|
|
254
239
|
npm test -- assessment # Assessment module tests
|
|
255
240
|
npm test -- SecurityAssessor # Security tests
|
|
256
241
|
```
|
|
@@ -261,11 +246,10 @@ npm test -- SecurityAssessor # Security tests
|
|
|
261
246
|
|
|
262
247
|
### Quick Start
|
|
263
248
|
|
|
264
|
-
| Document | Purpose
|
|
265
|
-
| ------------------------------------------------------ |
|
|
266
|
-
| [
|
|
267
|
-
| [
|
|
268
|
-
| [Architecture & Value](docs/ARCHITECTURE_AND_VALUE.md) | What this provides and why |
|
|
249
|
+
| Document | Purpose |
|
|
250
|
+
| ------------------------------------------------------ | ------------------------------ |
|
|
251
|
+
| [CLI Assessment Guide](docs/CLI_ASSESSMENT_GUIDE.md) | Complete CLI modes and options |
|
|
252
|
+
| [Architecture & Value](docs/ARCHITECTURE_AND_VALUE.md) | What this provides and why |
|
|
269
253
|
|
|
270
254
|
### API & Integration
|
|
271
255
|
|
|
@@ -279,7 +263,7 @@ npm test -- SecurityAssessor # Security tests
|
|
|
279
263
|
|
|
280
264
|
| Document | Purpose |
|
|
281
265
|
| -------------------------------------------------------------- | ----------------------------- |
|
|
282
|
-
| [Assessment Catalog](docs/ASSESSMENT_CATALOG.md) | All
|
|
266
|
+
| [Assessment Catalog](docs/ASSESSMENT_CATALOG.md) | All 17 modules reference |
|
|
283
267
|
| [Security Patterns Catalog](docs/SECURITY_PATTERNS_CATALOG.md) | Comprehensive attack patterns |
|
|
284
268
|
| [Testbed Setup Guide](docs/TESTBED_SETUP_GUIDE.md) | A/B validation |
|
|
285
269
|
|
|
@@ -351,7 +335,7 @@ npx @modelcontextprotocol/inspector
|
|
|
351
335
|
|
|
352
336
|
We built a comprehensive assessment framework on top of the original inspector, transforming it from a debugging tool into a full validation suite. Key additions:
|
|
353
337
|
|
|
354
|
-
- **
|
|
338
|
+
- **17 Assessment Modules** covering functionality, security, compliance
|
|
355
339
|
- **Pure Behavior-Based Detection** analyzing responses, not tool names
|
|
356
340
|
- **Zero False Positives** through context-aware reflection detection
|
|
357
341
|
- **CLI-First Workflow** with three specialized commands
|
package/cli/build/assess-full.js
CHANGED
|
@@ -1010,9 +1010,10 @@ Module Selection:
|
|
|
1010
1010
|
Valid module names:
|
|
1011
1011
|
functionality, security, documentation, errorHandling, usability,
|
|
1012
1012
|
mcpSpecCompliance, aupCompliance, toolAnnotations, prohibitedLibraries,
|
|
1013
|
-
|
|
1013
|
+
externalAPIScanner, authentication, temporal, resources, prompts,
|
|
1014
|
+
crossCapability, manifestValidation, portability
|
|
1014
1015
|
|
|
1015
|
-
Assessment Modules (
|
|
1016
|
+
Assessment Modules (17 total):
|
|
1016
1017
|
• Functionality - Tests all tools work correctly
|
|
1017
1018
|
• Security - Prompt injection & vulnerability testing
|
|
1018
1019
|
• Documentation - README completeness checks
|
|
@@ -1022,9 +1023,14 @@ Assessment Modules (16 total):
|
|
|
1022
1023
|
• AUP Compliance - Acceptable Use Policy checks
|
|
1023
1024
|
• Tool Annotations - readOnlyHint/destructiveHint validation
|
|
1024
1025
|
• Prohibited Libs - Dependency security checks
|
|
1025
|
-
•
|
|
1026
|
-
•
|
|
1026
|
+
• External API - External service detection
|
|
1027
|
+
• Authentication - OAuth/auth evaluation
|
|
1027
1028
|
• Temporal - Rug pull/temporal behavior change detection
|
|
1029
|
+
• Resources - Resource capability assessment
|
|
1030
|
+
• Prompts - Prompt capability assessment
|
|
1031
|
+
• Cross-Capability - Chained vulnerability detection
|
|
1032
|
+
• Manifest - MCPB manifest.json validation (optional)
|
|
1033
|
+
• Portability - Cross-platform compatibility (optional)
|
|
1028
1034
|
|
|
1029
1035
|
Examples:
|
|
1030
1036
|
mcp-assess-full my-server
|
|
@@ -256,7 +256,7 @@ async function runSecurityAssessment(options) {
|
|
|
256
256
|
callTool: createCallToolWrapper(client),
|
|
257
257
|
config,
|
|
258
258
|
};
|
|
259
|
-
console.log(`🛡️ Running security assessment with
|
|
259
|
+
console.log(`🛡️ Running security assessment with 23 attack patterns...`);
|
|
260
260
|
const assessor = new SecurityAssessor(config);
|
|
261
261
|
const results = await assessor.assess(context);
|
|
262
262
|
await client.close();
|
|
@@ -372,7 +372,7 @@ function printHelp() {
|
|
|
372
372
|
console.log(`
|
|
373
373
|
Usage: mcp-assess-security [options] [server-name]
|
|
374
374
|
|
|
375
|
-
Run security assessment against an MCP server with
|
|
375
|
+
Run security assessment against an MCP server with 23 attack patterns.
|
|
376
376
|
|
|
377
377
|
Options:
|
|
378
378
|
--server, -s <name> Server name (required, or pass as first positional arg)
|
|
@@ -382,14 +382,11 @@ Options:
|
|
|
382
382
|
--verbose, -v Enable verbose logging
|
|
383
383
|
--help, -h Show this help message
|
|
384
384
|
|
|
385
|
-
Attack Patterns Tested (
|
|
386
|
-
•
|
|
387
|
-
•
|
|
388
|
-
•
|
|
389
|
-
•
|
|
390
|
-
• Encoding bypass
|
|
391
|
-
• Multi-turn manipulation
|
|
392
|
-
• Context poisoning
|
|
385
|
+
Attack Patterns Tested (23 total):
|
|
386
|
+
• Command Injection, SQL Injection, Path Traversal
|
|
387
|
+
• Calculator Injection, Code Execution, XXE
|
|
388
|
+
• Data Exfiltration, Token Theft, NoSQL Injection
|
|
389
|
+
• Unicode Bypass, Nested Injection, Package Squatting
|
|
393
390
|
• And more...
|
|
394
391
|
|
|
395
392
|
Examples:
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { u as useToast, r as reactExports, j as jsxRuntimeExports, p as parseOAuthCallbackParams, g as generateOAuthErrorDescription, S as SESSION_KEYS, I as InspectorOAuthClientProvider, a as auth } from "./index-
|
|
1
|
+
import { u as useToast, r as reactExports, j as jsxRuntimeExports, p as parseOAuthCallbackParams, g as generateOAuthErrorDescription, S as SESSION_KEYS, I as InspectorOAuthClientProvider, a as auth } from "./index-C9mj2Mhr.js";
|
|
2
2
|
const OAuthCallback = ({ onConnect }) => {
|
|
3
3
|
const { toast } = useToast();
|
|
4
4
|
const hasProcessedRef = reactExports.useRef(false);
|
package/client/dist/assets/{OAuthDebugCallback-BPbrWgKx.js → OAuthDebugCallback-CL_C0bio.js}
RENAMED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { r as reactExports, S as SESSION_KEYS, p as parseOAuthCallbackParams, j as jsxRuntimeExports, g as generateOAuthErrorDescription } from "./index-
|
|
1
|
+
import { r as reactExports, S as SESSION_KEYS, p as parseOAuthCallbackParams, j as jsxRuntimeExports, g as generateOAuthErrorDescription } from "./index-C9mj2Mhr.js";
|
|
2
2
|
const OAuthDebugCallback = ({ onConnect }) => {
|
|
3
3
|
reactExports.useEffect(() => {
|
|
4
4
|
let isProcessed = false;
|
|
@@ -16320,7 +16320,7 @@ object({
|
|
|
16320
16320
|
token_type_hint: string().optional()
|
|
16321
16321
|
}).strip();
|
|
16322
16322
|
const name = "@bryan-thompson/inspector-assessment-client";
|
|
16323
|
-
const version$1 = "1.23.
|
|
16323
|
+
const version$1 = "1.23.10";
|
|
16324
16324
|
const packageJson = {
|
|
16325
16325
|
name,
|
|
16326
16326
|
version: version$1
|
|
@@ -45217,7 +45217,7 @@ const useTheme = () => {
|
|
|
45217
45217
|
[theme, setThemeWithSideEffect]
|
|
45218
45218
|
);
|
|
45219
45219
|
};
|
|
45220
|
-
const version = "1.23.
|
|
45220
|
+
const version = "1.23.10";
|
|
45221
45221
|
var [createTooltipContext] = createContextScope("Tooltip", [
|
|
45222
45222
|
createPopperScope
|
|
45223
45223
|
]);
|
|
@@ -48774,13 +48774,13 @@ const App = () => {
|
|
|
48774
48774
|
) });
|
|
48775
48775
|
if (window.location.pathname === "/oauth/callback") {
|
|
48776
48776
|
const OAuthCallback = React.lazy(
|
|
48777
|
-
() => __vitePreload(() => import("./OAuthCallback-
|
|
48777
|
+
() => __vitePreload(() => import("./OAuthCallback-8m79-pFb.js"), true ? [] : void 0)
|
|
48778
48778
|
);
|
|
48779
48779
|
return /* @__PURE__ */ jsxRuntimeExports.jsx(reactExports.Suspense, { fallback: /* @__PURE__ */ jsxRuntimeExports.jsx("div", { children: "Loading..." }), children: /* @__PURE__ */ jsxRuntimeExports.jsx(OAuthCallback, { onConnect: onOAuthConnect }) });
|
|
48780
48780
|
}
|
|
48781
48781
|
if (window.location.pathname === "/oauth/callback/debug") {
|
|
48782
48782
|
const OAuthDebugCallback = React.lazy(
|
|
48783
|
-
() => __vitePreload(() => import("./OAuthDebugCallback-
|
|
48783
|
+
() => __vitePreload(() => import("./OAuthDebugCallback-CL_C0bio.js"), true ? [] : void 0)
|
|
48784
48784
|
);
|
|
48785
48785
|
return /* @__PURE__ */ jsxRuntimeExports.jsx(reactExports.Suspense, { fallback: /* @__PURE__ */ jsxRuntimeExports.jsx("div", { children: "Loading..." }), children: /* @__PURE__ */ jsxRuntimeExports.jsx(OAuthDebugCallback, { onConnect: onOAuthDebugConnect }) });
|
|
48786
48786
|
}
|
package/client/dist/index.html
CHANGED
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
<link rel="icon" type="image/svg+xml" href="/mcp.svg" />
|
|
6
6
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
|
7
7
|
<title>MCP Inspector</title>
|
|
8
|
-
<script type="module" crossorigin src="/assets/index-
|
|
8
|
+
<script type="module" crossorigin src="/assets/index-C9mj2Mhr.js"></script>
|
|
9
9
|
<link rel="stylesheet" crossorigin href="/assets/index-Df9Sx1jt.css">
|
|
10
10
|
</head>
|
|
11
11
|
<body>
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bryan-thompson/inspector-assessment",
|
|
3
|
-
"version": "1.23.
|
|
3
|
+
"version": "1.23.10",
|
|
4
4
|
"description": "Enhanced MCP Inspector with comprehensive assessment capabilities for server validation",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Bryan Thompson <bryan@triepod.ai>",
|