mcp-capdiff 0.1.2__tar.gz → 0.1.3__tar.gz

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/CHANGELOG.md

@@ -4,6 +4,23 @@ All notable changes follow Keep a Changelog. Releases use semantic versioning fo
 
 ## [Unreleased]
 
+## [0.1.3] - 2026-10-01
+
+### Fixed
+
+- Stop ordinary string, datetime, and other object `.replace()` calls from being classified as filesystem writes; retain detection for `os.replace(...)` and statically known `Path.replace(...)` calls.
+- Detect model-controlled interactive process execution through `pexpect.spawn(...)` and `pexpect.spawnu(...)`.
+- Require action verbs for semantic external-write inference so nouns such as `email` and `message` do not turn list/read tools into writes.
+- Prefer read-only tool names and explicit negation over incidental destructive words in descriptions while retaining destructive SQL detection for execution tools.
+
+### Verified
+
+- 71 automated tests, including positive controls for shell execution, filesystem replacement, destructive actions, and external writes.
+- `metabase-mcp` and `mcp_agent_mail`: string and datetime replacement no longer produce MCP002 findings.
+- `interactive-terminal-mcp`: `spawn_process` is classified as shell execution and produces MCP001 evidence at `pexpect.spawn(...)`.
+- `MCP-PostgreSQL-Ops` and `winremote-mcp`: the read-only statistics tool and GUI `Type` tool no longer produce MCP004/MCP010 findings.
+- The 14-repository benchmark covered 888 discovered tools; interprocedural network propagation remains deferred for a later release.
+
 ## [0.1.2] - 2026-10-01
 
 ### Fixed

{mcp_capdiff-0.1.2/src/mcp_capdiff.egg-info → mcp_capdiff-0.1.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mcp-capdiff
-Version: 0.1.2
+Version: 0.1.3
 Summary: Security regression scanner for Model Context Protocol servers.
 Author: MCP Audit contributors
 License-Expression: MIT

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/docs/precision-matrix.md

@@ -4,8 +4,8 @@ The release gate requires positive, negative, guarded, and edge coverage for the
 
 | Rule | True positive | True negative | Guarded safe | Tricky edge |
 | --- | --- | --- | --- | --- |
-| MCP001 | `shell_direct` | `safe_server` | `shell_safe_allowlist` | `shell_dynamic_no_shell` detects a dynamic executable without `shell=True` |
-| MCP002 | `filesystem_unrestricted` | `sensitive_read_only` | `filesystem_restricted` | `filesystem_guard_after_read` rejects a late guard |
+| MCP001 | `shell_direct` | `safe_server` | `shell_safe_allowlist` | `shell_dynamic_no_shell` and `benchmark_semantics` detect dynamic subprocess and pexpect execution |
+| MCP002 | `filesystem_unrestricted` | `sensitive_read_only` | `filesystem_restricted` | `filesystem_guard_after_read` rejects a late guard while generic `.replace()` calls remain non-filesystem operations |
 | MCP003 | `url_arbitrary` | fixed URL in demo baseline | `url_allowlisted` | hostname mention and guard-after-request remain findings |
 | MCP005 | `sensitive_to_network` | `sensitive_read_only` | constrained source-only server | root corpus proves independent MCP contexts are isolated |
 | MCP016 | network expansion diff | unchanged diff | allowlist-added demo resolves risk | a new high-impact tool is an escalation |
@@ -17,4 +17,4 @@ MCP004, MCP007, and MCP010 also have paired approved/bounded and unapproved/unbo
 
 Discovery is separately benchmarked by `fastmcp_async_annotations`, which must find both plain async `@mcp.tool()` and `@mcp.tool(annotations=ToolAnnotations(...))` decorators. Missing targets, unreadable trees, parse failures, and unexpected zero-tool scans fail closed.
 
-Real-world regression fixtures also cover imported `mcp.tool()(function)` registration, helper-based path containment, SQLite reads, local file modes, and local directory creation. MCP005 requires an actual outbound network destination; local writes and semantic-only action names cannot become exfiltration sinks.
+Real-world regression fixtures also cover imported `mcp.tool()(function)` registration, helper-based path containment, SQLite reads, local file modes, local directory creation, `pexpect.spawn`, typed `Path.replace`, and ordinary string/datetime `.replace()` calls. MCP005 requires an actual outbound network destination; local writes and semantic-only action names cannot become exfiltration sinks. Read/list/status tools and incidental destructive words in documentation are negative cases for side-effect inference.

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/docs/releasing.md

@@ -19,7 +19,7 @@ Install the wheel into a clean environment and verify `mcp-audit --version`, a p
 
 ## Publish
 
-1. Create and push the matching Git tag, such as `v0.1.2`.
+1. Create and push the matching Git tag, such as `v0.1.3`.
 2. Publish the matching GitHub Release. This automatically runs `release.yml`.
 3. Verify `pipx install mcp-capdiff` in a clean environment.
 4. Run the tagged Action from the demo repository and confirm SARIF appears in Code Scanning.

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/docs/versioning.md

@@ -4,7 +4,7 @@ MCP Audit maintains four independent versions:
 
 | Contract | Current | Change policy |
 | --- | --- | --- |
-| CLI/package | `0.1.2` | Semantic versioning; pre-1.0 minor releases may change CLI behavior |
+| CLI/package | `0.1.3` | Semantic versioning; pre-1.0 minor releases may change CLI behavior |
 | Ruleset | `0.1` | Minor adds or materially changes rules; patch corrects implementation without intended semantic change |
 | JSON report schema | `1.0` | Minor changes are additive; major changes may remove fields or change meaning |
 | Manifest schema | `1.0` | Minor changes are additive; major changes may remove fields or change meaning |

mcp_capdiff-0.1.3/fixtures/benchmark_semantics/server.py

@@ -0,0 +1,88 @@
+from datetime import datetime, timezone
+from pathlib import Path
+
+import os
+import pexpect
+from fastmcp import FastMCP
+
+
+mcp = FastMCP("benchmark-semantics")
+
+
+@mcp.tool()
+def list_tables(description: str) -> str:
+    """List database tables without modifying them."""
+    return description.replace("|", "\\|")
+
+
+@mcp.tool()
+def get_message_time() -> str:
+    """Read the current message timestamp."""
+    return datetime.now(timezone.utc).replace(tzinfo=None).isoformat()
+
+
+@mcp.tool()
+def spawn_process(command: str) -> str:
+    """Spawn an interactive command."""
+    return str(pexpect.spawn(command))
+
+
+@mcp.tool()
+def spawn_unicode_process(command: str) -> str:
+    """Spawn an interactive Unicode command."""
+    return str(pexpect.spawnu(command))
+
+
+@mcp.tool()
+def get_vacuum_analyze_stats() -> str:
+    """Retrieve read-only delete and revoke statistics; does not execute destructive operations."""
+    return "stats"
+
+
+@mcp.tool(name="Type")
+def type_text(text: str) -> str:
+    """Type text into the GUI. Clear existing content first with Ctrl+A, Delete."""
+    return text
+
+
+@mcp.tool()
+def list_email_tags() -> list[str]:
+    """List email tags without changing them."""
+    return []
+
+
+@mcp.tool()
+def delete_comment(comment_id: str) -> str:
+    """Delete a remote comment."""
+    return comment_id
+
+
+@mcp.tool()
+def send_email(message: str) -> str:
+    """Send an email message."""
+    return message
+
+
+@mcp.tool()
+def send_status(status: str) -> str:
+    """Send a status notification."""
+    return status
+
+
+@mcp.tool()
+def delete_history(history_id: str) -> str:
+    """Delete remote history."""
+    return history_id
+
+
+@mcp.tool()
+def replace_path(source: str, destination: str) -> str:
+    path = Path(source)
+    path.replace(destination)
+    return destination
+
+
+@mcp.tool()
+def replace_path_with_os(source: str, destination: str) -> str:
+    os.replace(source, destination)
+    return destination

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/src/mcp_audit/__init__.py

@@ -1,6 +1,6 @@
 """MCP Audit public package and independently evolving contract versions."""
 
-__version__ = "0.1.2"
+__version__ = "0.1.3"
 RULESET_VERSION = "0.1"
 REPORT_SCHEMA_VERSION = "1.0"
 MANIFEST_SCHEMA_VERSION = "1.0"

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/src/mcp_audit/scanners/source/python.py

@@ -318,7 +318,7 @@ def _tool_from_function(
                 location=_function_location(file_path, node),
             )
         )
-    if _looks_destructive(lower_text):
+    if _looks_destructive(name, description):
         capability.destructive = True
         capability.side_effect = "destructive_action"
         analyzer.evidence.append(
@@ -364,6 +364,11 @@ class FunctionAnalyzer(ast.NodeVisitor):
         self.file_path = file_path
         self.source_text = source_text
         self.parameter_names = {parameter.name for parameter in parameters}
+        self.path_objects = {
+            parameter.name
+            for parameter in parameters
+            if parameter.annotation and parameter.annotation.rsplit(".", 1)[-1] == "Path"
+        }
         self.path_helpers = path_helpers
         self.aliases: dict[str, set[str]] = {}
         self.host_guarded_parameters: set[str] = set()
@@ -387,14 +392,21 @@ class FunctionAnalyzer(ast.NodeVisitor):
     def visit_Assign(self, node: ast.Assign) -> None:
         roots = self._parameter_roots(node.value)
         for target in node.targets:
-            if isinstance(target, ast.Name) and roots:
-                self.aliases[target.id] = roots
+            if isinstance(target, ast.Name):
+                if roots:
+                    self.aliases[target.id] = roots
+                if _is_path_expression(node.value, self.path_objects):
+                    self.path_objects.add(target.id)
         self.generic_visit(node)
 
     def visit_AnnAssign(self, node: ast.AnnAssign) -> None:
         roots = self._parameter_roots(node.value)
-        if isinstance(node.target, ast.Name) and roots:
-            self.aliases[node.target.id] = roots
+        if isinstance(node.target, ast.Name):
+            if roots:
+                self.aliases[node.target.id] = roots
+            annotation = ast.unparse(node.annotation).rsplit(".", 1)[-1]
+            if annotation == "Path" or _is_path_expression(node.value, self.path_objects):
+                self.path_objects.add(node.target.id)
         self.generic_visit(node)
 
     def visit_For(self, node: ast.For) -> None:
@@ -426,6 +438,11 @@ class FunctionAnalyzer(ast.NodeVisitor):
             elif call_name.startswith("subprocess.") and _has_model_controlled_executable(node, self._parameter_roots):
                 self.shell_execution = True
                 self._record(node, f"{call_name}(...) uses a model-controlled executable or interpreter payload.", "execution")
+        if call_name in {"pexpect.spawn", "pexpect.spawnu"} and _has_model_controlled_executable(
+            node, self._parameter_roots
+        ):
+            self.shell_execution = True
+            self._record(node, f"{call_name}(...) starts a model-controlled interactive process.", "execution")
         open_access = _file_open_access(node, call_name)
         if open_access is not None:
             reads, writes = open_access
@@ -459,7 +476,9 @@ class FunctionAnalyzer(ast.NodeVisitor):
             "shutil.copyfile",
             "shutil.move",
             "shutil.rmtree",
-        } or call_name.endswith((".mkdir", ".unlink", ".rename", ".replace")):
+        } or call_name.endswith((".mkdir", ".unlink", ".rename")) or _is_path_replace(
+            node, call_name, self.path_objects
+        ):
             self.filesystem_write = True
             self.filesystem_operation_lines.append(node.lineno)
             self._mark_inline_path_guard(node)
@@ -610,6 +629,37 @@ def _file_open_access(node: ast.Call, call_name: str) -> tuple[bool, bool] | Non
     return reads, writes
 
 
+def _is_path_expression(node: ast.AST | None, path_objects: set[str]) -> bool:
+    if node is None:
+        return False
+    if isinstance(node, ast.Name):
+        return node.id in path_objects
+    if isinstance(node, ast.Call):
+        call_name = _call_name(node.func)
+        if call_name in {"Path", "pathlib.Path"}:
+            return True
+        if isinstance(node.func, ast.Attribute) and node.func.attr in {
+            "absolute",
+            "expanduser",
+            "resolve",
+            "with_name",
+            "with_stem",
+            "with_suffix",
+        }:
+            return _is_path_expression(node.func.value, path_objects)
+    return False
+
+
+def _is_path_replace(node: ast.Call, call_name: str, path_objects: set[str]) -> bool:
+    if call_name in {"os.replace", "Path.replace", "pathlib.Path.replace"}:
+        return True
+    return (
+        isinstance(node.func, ast.Attribute)
+        and node.func.attr == "replace"
+        and _is_path_expression(node.func.value, path_objects)
+    )
+
+
 def _has_model_controlled_executable(node: ast.Call, roots_for) -> bool:
     if not node.args:
         return False
@@ -694,7 +744,11 @@ def _function_location(file_path: Path, node: ast.FunctionDef | ast.AsyncFunctio
 
 
 def _semantic_tokens(text: str) -> set[str]:
-    return set(re.findall(r"[a-z0-9]+", text.lower()))
+    return set(_semantic_words(text))
+
+
+def _semantic_words(text: str) -> list[str]:
+    return re.findall(r"[a-z0-9]+", text.lower())
 
 
 def _highest_sensitivity(data_classes: set[str]) -> str:
@@ -705,11 +759,34 @@ def _highest_sensitivity(data_classes: set[str]) -> str:
 
 
 def _looks_external_write(text: str) -> bool:
-    return bool(
-        _semantic_tokens(text)
-        & {"send", "email", "slack", "webhook", "publish", "post", "message", "notify", "upload"}
-    )
+    words = _semantic_words(text)
+    if words and words[0] in {"get", "list", "read", "search", "show", "monitor"}:
+        return False
+    return bool(set(words) & {"send", "forward", "publish", "post", "notify", "upload"})
 
 
-def _looks_destructive(text: str) -> bool:
-    return bool(_semantic_tokens(text) & {"delete", "drop", "remove", "revoke", "terminate", "destroy"})
+def _looks_destructive(name: str, description: str) -> bool:
+    name_words = _semantic_words(name)
+    name_tokens = set(name_words)
+    if name_words and name_words[0] in {"get", "list", "read", "search", "show", "monitor"}:
+        return False
+    destructive = {"delete", "drop", "remove", "revoke", "terminate", "destroy"}
+    if name_tokens & destructive:
+        return True
+    normalized_description = " ".join(description.lower().split())
+    if any(
+        phrase in normalized_description
+        for phrase in {"does not execute", "doesn't execute", "read-only", "strictly prohibited", "monitor only"}
+    ):
+        return False
+    declares_destructive_action = bool(
+        re.match(
+            r"^(?:this tool\s+)?(?:delete|deletes|drop|drops|remove|removes|revoke|revokes|terminate|terminates|destroy|destroys)\b",
+            normalized_description,
+        )
+    )
+    execution_tool_describes_destructive_sql = bool(
+        name_tokens & {"execute", "run", "apply"}
+        and _semantic_tokens(normalized_description) & destructive
+    )
+    return declares_destructive_action or execution_tool_describes_destructive_sql

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3/src/mcp_capdiff.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mcp-capdiff
-Version: 0.1.2
+Version: 0.1.3
 Summary: Security regression scanner for Model Context Protocol servers.
 Author: MCP Audit contributors
 License-Expression: MIT

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/src/mcp_capdiff.egg-info/SOURCES.txt

@@ -34,6 +34,7 @@ examples/vulnerable-fastmcp/scenarios/network_allowlist_added/after.py
 examples/vulnerable-fastmcp/scenarios/network_allowlist_added/before.py
 examples/vulnerable-fastmcp/scenarios/unrestricted_fetch/after.py
 examples/vulnerable-fastmcp/scenarios/unrestricted_fetch/before.py
+fixtures/benchmark_semantics/server.py
 fixtures/destructive_with_approval/server.py
 fixtures/destructive_without_approval/server.py
 fixtures/fastmcp_async_annotations/server.py

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/tests/golden/scan.json

@@ -74,7 +74,7 @@
     }
   ],
   "versions": {
-    "cli": "0.1.2",
+    "cli": "0.1.3",
     "rules": "0.1"
   }
 }

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/tests/golden/scan.sarif.json

@@ -81,7 +81,7 @@
               }
             }
           ],
-          "semanticVersion": "0.1.2"
+          "semanticVersion": "0.1.3"
         }
       }
     }

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/tests/test_real_world_regressions.py

@@ -50,3 +50,43 @@ def test_sqlite_reads_are_not_writes_or_exfiltration_sinks() -> None:
     report = scan_path(ROOT / "fixtures/sqlite_read_only")
     assert all(tool.capability.side_effect == "none" for tool in report.tools)
     assert not {"MCP004", "MCP005"} & {finding.rule_id for finding in report.findings}
+
+
+def test_generic_replace_calls_are_not_filesystem_writes() -> None:
+    report = scan_path(ROOT / "fixtures/benchmark_semantics")
+    tools = {tool.name: tool for tool in report.tools}
+    assert tools["list_tables"].capability.filesystem == "none"
+    assert tools["get_message_time"].capability.filesystem == "none"
+    assert tools["replace_path"].capability.side_effect == "local_write"
+    assert tools["replace_path_with_os"].capability.side_effect == "local_write"
+
+
+def test_pexpect_spawn_is_model_controlled_execution() -> None:
+    report = scan_path(ROOT / "fixtures/benchmark_semantics")
+    tools = {tool.name: tool for tool in report.tools}
+    assert tools["spawn_process"].capability.execution == "shell"
+    assert tools["spawn_unicode_process"].capability.execution == "shell"
+    findings = {
+        item.tool: item for item in report.findings if item.rule_id == "MCP001"
+    }
+    assert findings["spawn_process"].evidence[0].snippet == "pexpect.spawn(command)"
+    assert findings["spawn_unicode_process"].evidence[0].snippet == "pexpect.spawnu(command)"
+
+
+def test_read_only_context_suppresses_destructive_lexical_matches() -> None:
+    report = scan_path(ROOT / "fixtures/benchmark_semantics")
+    destructive_tools = {
+        finding.tool for finding in report.findings if finding.rule_id in {"MCP004", "MCP010"}
+    }
+    assert "get_vacuum_analyze_stats" not in destructive_tools
+    assert "Type" not in destructive_tools
+    assert "delete_comment" in destructive_tools
+
+
+def test_external_write_inference_requires_an_action_verb() -> None:
+    report = scan_path(ROOT / "fixtures/benchmark_semantics")
+    tools = {tool.name: tool for tool in report.tools}
+    assert tools["list_email_tags"].capability.side_effect == "none"
+    assert tools["send_email"].capability.side_effect == "external_write"
+    assert tools["send_status"].capability.side_effect == "external_write"
+    assert tools["delete_history"].capability.side_effect == "destructive_action"

{mcp_capdiff-0.1.2 → mcp_capdiff-0.1.3}/tests/test_reporters.py

@@ -15,7 +15,7 @@ def test_json_schema_has_stable_version_and_structured_evidence() -> None:
     payload = scan_path(ROOT / "fixtures/url_arbitrary").as_dict()
     assert payload["schema_version"] == "1.0"
     assert payload["report_type"] == "scan"
-    assert payload["versions"] == {"cli": "0.1.2", "rules": "0.1"}
+    assert payload["versions"] == {"cli": "0.1.3", "rules": "0.1"}
     evidence = payload["findings"][0]["evidence"][0]
     assert set(evidence) == {"kind", "message", "location", "snippet"}
     assert payload["tools"][0]["context"] == "server.py:mcp"
@@ -31,7 +31,7 @@ def test_terminal_finding_answers_risk_and_remediation() -> None:
 def test_sarif_contains_source_region_snippet_and_help() -> None:
     payload = json.loads(render_sarif(scan_path(ROOT / "fixtures/url_arbitrary")))
     run = payload["runs"][0]
-    assert run["tool"]["driver"]["semanticVersion"] == "0.1.2"
+    assert run["tool"]["driver"]["semanticVersion"] == "0.1.3"
     assert run["tool"]["driver"]["properties"]["rulesVersion"] == "0.1"
     result = next(item for item in run["results"] if item["ruleId"] == "MCP003")
     region = result["locations"][0]["physicalLocation"]["region"]