mcp-capdiff 0.1.1__tar.gz

mcp_capdiff-0.1.1/CHANGELOG.md

@@ -0,0 +1,40 @@
+# Changelog
+
+All notable changes follow Keep a Changelog. Releases use semantic versioning for the CLI package; rules and output schemas are versioned independently.
+
+## [Unreleased]
+
+## [0.1.1] - 2026-10-01
+
+### Added
+
+- Dedicated PyPI Trusted Publishing workflow with tag-pinned builds and OIDC authentication.
+
+### Changed
+
+- Rename the PyPI distribution to `mcp-capdiff` while preserving the MCP Audit product name, `mcp_audit` import package, and `mcp-audit` command.
+- Prepare version `0.1.1` so the public `v0.1.0` tag remains immutable.
+
+## [0.1.0] - 2026-10-01
+
+### Fixed
+
+- Prevent `PostgreSQL` from being misclassified as an HTTP `POST` side effect.
+- Attach destructive semantic evidence to MCP004 and MCP010 instead of unrelated database-read evidence.
+- Fail closed on inaccessible source trees and zero-tool scans instead of reporting a false PASS.
+
+### Added
+
+- Discovery benchmark for async `@mcp.tool()` functions with FastMCP `ToolAnnotations`.
+- Explicit `--allow-empty` escape hatch for intentional zero-tool scans.
+- Python/FastMCP source discovery and capability extraction.
+- Nine security and regression rules in ruleset `0.1`.
+- Cross-tool sensitive-data path detection with MCP-context isolation.
+- Terminal, JSON, SARIF, manifest, and Git security-diff output.
+- Versioned report and manifest schema `1.0`.
+- Policy thresholds and reasoned, expiring suppressions.
+- Composite GitHub Action with Code Scanning upload.
+
+### Verified
+
+- External PostgreSQL MCP benchmark: 20 tools discovered, 6 legitimate findings, no known false positives from `PostgreSQL` token matching, and no false PASS.

mcp_capdiff-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 MCP Audit contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mcp_capdiff-0.1.1/MANIFEST.in

@@ -0,0 +1,8 @@
+include action.yml
+include CHANGELOG.md
+include SECURITY.md
+include mcp-audit.yaml
+recursive-include docs *.md
+recursive-include examples *
+recursive-include fixtures *.py
+recursive-include tests/golden *

mcp_capdiff-0.1.1/PKG-INFO

@@ -0,0 +1,173 @@
+Metadata-Version: 2.4
+Name: mcp-capdiff
+Version: 0.1.1
+Summary: Security regression scanner for Model Context Protocol servers.
+Author: MCP Audit contributors
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/robbyfa/MCP-Audit
+Project-URL: Repository, https://github.com/robbyfa/MCP-Audit
+Project-URL: Issues, https://github.com/robbyfa/MCP-Audit/issues
+Keywords: mcp,security,static-analysis,sarif,ci
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Security
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: PyYAML<7,>=6.0
+Dynamic: license-file
+
+# MCP Audit
+
+MCP Audit is an open-source security regression scanner for Model Context Protocol servers. It extracts tool capabilities and permissions, detects risky cross-tool data flows, compares security posture across Git revisions, and emits CI-friendly evidence.
+
+The first version focuses on Python/FastMCP projects and the core product thesis from the spec: show what an MCP server can do, what changed, and whether the change creates a dangerous capability path.
+
+MCP005 is the capability-graph rule: it identifies when one tool returns classified data and another tool in the same MCP registration context can send it to an external destination. Findings include the source, data classification, sink, destination, path, source evidence, impact, and remediation.
+
+## Install locally
+
+```bash
+uv sync
+```
+
+Run project commands through `uv run`, or activate the virtual environment with
+`source .venv/bin/activate` before using bare commands.
+
+The package also supports isolated CLI installation directly from a checkout:
+
+```bash
+pipx install .
+mcp-audit --version
+mcp-audit scan .
+```
+
+The product and CLI are named MCP Audit, while the PyPI distribution is named `mcp-capdiff`:
+
+```bash
+pipx install mcp-capdiff
+mcp-audit --version
+```
+
+## Scan a server
+
+```bash
+uv run mcp-audit scan .
+```
+
+Scans fail closed when source cannot be read or when no MCP tools are discovered. Use `--allow-empty` only when an empty result is intentional.
+
+Useful output formats:
+
+```bash
+uv run mcp-audit scan . --format json
+uv run mcp-audit scan . --format sarif --output mcp-audit.sarif
+```
+
+## Generate a capability manifest
+
+```bash
+uv run mcp-audit manifest .
+```
+
+## Compare against a Git baseline
+
+```bash
+uv run mcp-audit diff --base origin/main
+```
+
+The diff report focuses on security changes: before/after risk, added and changed tools, expanded capabilities, new findings, resolved findings, and approval removal.
+
+## Run tests
+
+```bash
+uv run pytest
+```
+
+The corpus contains positive, negative, and edge cases under `fixtures/`, including host allowlisting, path-root validation, approval gating, and cross-tool sensitive-data paths.
+
+## GitHub Action
+
+The repository includes a composite action that runs the security diff, uploads SARIF to GitHub Code Scanning, and fails the check when policy thresholds are crossed:
+
+```yaml
+permissions:
+  actions: read
+  contents: read
+  security-events: write
+
+steps:
+  - uses: actions/checkout@v4
+    with:
+      fetch-depth: 0
+  - uses: actions/setup-python@v5
+    with:
+      python-version: "3.12"
+  - uses: your-org/mcp-audit@v0.1
+    with:
+      target: .
+      base: origin/main
+      policy: mcp-audit.yaml
+```
+
+JSON reports and manifests use the versioned schema documented in `docs/report-schema.md`.
+
+The copyable [vulnerable FastMCP demo](examples/vulnerable-fastmcp/) includes four before/after pull-request scenarios and its own Action workflow.
+
+## V0.1 rules
+
+- `MCP001` arbitrary shell execution.
+- `MCP002` unrestricted filesystem access.
+- `MCP003` arbitrary URL or SSRF surface.
+- `MCP004` high-impact side effect without approval.
+- `MCP005` sensitive read to external write path.
+- `MCP007` unbounded security-sensitive input.
+- `MCP010` destructive tool exposed.
+- `MCP016` capability escalation in a diff.
+- `MCP017` approval removed in a diff.
+
+Each rule's detection behavior, examples, remediation, and limitations are documented in [docs/rules](docs/rules/README.md).
+
+## Demo fixtures
+
+```bash
+uv run mcp-audit scan fixtures/safe_server
+uv run mcp-audit scan fixtures/vulnerable_server
+```
+
+The vulnerable fixture includes a sensitive file read tool, an unrestricted URL fetcher, shell execution, and a destructive operation. MCP Audit should flag the individual findings and the cross-tool path:
+
+```text
+read_customer_file -> agent_context -> fetch_url
+```
+
+## Policy
+
+The default policy fails on high and critical findings. A minimal policy file can set a risk threshold and suppress reviewed findings:
+
+```yaml
+policy:
+  ci:
+    fail_on:
+      - critical
+      - high
+    max_risk_score: 60
+
+suppress:
+  - rule: MCP003
+    tool: internal_fetch
+    reason: "Network egress is enforced by service mesh"
+    expires: 2027-01-31
+```
+
+Suppression reasons are required. Expired suppressions no longer hide findings, and `mcp-audit policy check` reports them. See [policy documentation](docs/policy.md).
+
+## Version contracts
+
+CLI, ruleset, report schema, and manifest schema versions evolve independently. See [versioning](docs/versioning.md) and [report schema](docs/report-schema.md).
+
+Passing MCP Audit is technical security evidence, not a legal compliance determination.

mcp_capdiff-0.1.1/README.md

@@ -0,0 +1,150 @@
+# MCP Audit
+
+MCP Audit is an open-source security regression scanner for Model Context Protocol servers. It extracts tool capabilities and permissions, detects risky cross-tool data flows, compares security posture across Git revisions, and emits CI-friendly evidence.
+
+The first version focuses on Python/FastMCP projects and the core product thesis from the spec: show what an MCP server can do, what changed, and whether the change creates a dangerous capability path.
+
+MCP005 is the capability-graph rule: it identifies when one tool returns classified data and another tool in the same MCP registration context can send it to an external destination. Findings include the source, data classification, sink, destination, path, source evidence, impact, and remediation.
+
+## Install locally
+
+```bash
+uv sync
+```
+
+Run project commands through `uv run`, or activate the virtual environment with
+`source .venv/bin/activate` before using bare commands.
+
+The package also supports isolated CLI installation directly from a checkout:
+
+```bash
+pipx install .
+mcp-audit --version
+mcp-audit scan .
+```
+
+The product and CLI are named MCP Audit, while the PyPI distribution is named `mcp-capdiff`:
+
+```bash
+pipx install mcp-capdiff
+mcp-audit --version
+```
+
+## Scan a server
+
+```bash
+uv run mcp-audit scan .
+```
+
+Scans fail closed when source cannot be read or when no MCP tools are discovered. Use `--allow-empty` only when an empty result is intentional.
+
+Useful output formats:
+
+```bash
+uv run mcp-audit scan . --format json
+uv run mcp-audit scan . --format sarif --output mcp-audit.sarif
+```
+
+## Generate a capability manifest
+
+```bash
+uv run mcp-audit manifest .
+```
+
+## Compare against a Git baseline
+
+```bash
+uv run mcp-audit diff --base origin/main
+```
+
+The diff report focuses on security changes: before/after risk, added and changed tools, expanded capabilities, new findings, resolved findings, and approval removal.
+
+## Run tests
+
+```bash
+uv run pytest
+```
+
+The corpus contains positive, negative, and edge cases under `fixtures/`, including host allowlisting, path-root validation, approval gating, and cross-tool sensitive-data paths.
+
+## GitHub Action
+
+The repository includes a composite action that runs the security diff, uploads SARIF to GitHub Code Scanning, and fails the check when policy thresholds are crossed:
+
+```yaml
+permissions:
+  actions: read
+  contents: read
+  security-events: write
+
+steps:
+  - uses: actions/checkout@v4
+    with:
+      fetch-depth: 0
+  - uses: actions/setup-python@v5
+    with:
+      python-version: "3.12"
+  - uses: your-org/mcp-audit@v0.1
+    with:
+      target: .
+      base: origin/main
+      policy: mcp-audit.yaml
+```
+
+JSON reports and manifests use the versioned schema documented in `docs/report-schema.md`.
+
+The copyable [vulnerable FastMCP demo](examples/vulnerable-fastmcp/) includes four before/after pull-request scenarios and its own Action workflow.
+
+## V0.1 rules
+
+- `MCP001` arbitrary shell execution.
+- `MCP002` unrestricted filesystem access.
+- `MCP003` arbitrary URL or SSRF surface.
+- `MCP004` high-impact side effect without approval.
+- `MCP005` sensitive read to external write path.
+- `MCP007` unbounded security-sensitive input.
+- `MCP010` destructive tool exposed.
+- `MCP016` capability escalation in a diff.
+- `MCP017` approval removed in a diff.
+
+Each rule's detection behavior, examples, remediation, and limitations are documented in [docs/rules](docs/rules/README.md).
+
+## Demo fixtures
+
+```bash
+uv run mcp-audit scan fixtures/safe_server
+uv run mcp-audit scan fixtures/vulnerable_server
+```
+
+The vulnerable fixture includes a sensitive file read tool, an unrestricted URL fetcher, shell execution, and a destructive operation. MCP Audit should flag the individual findings and the cross-tool path:
+
+```text
+read_customer_file -> agent_context -> fetch_url
+```
+
+## Policy
+
+The default policy fails on high and critical findings. A minimal policy file can set a risk threshold and suppress reviewed findings:
+
+```yaml
+policy:
+  ci:
+    fail_on:
+      - critical
+      - high
+    max_risk_score: 60
+
+suppress:
+  - rule: MCP003
+    tool: internal_fetch
+    reason: "Network egress is enforced by service mesh"
+    expires: 2027-01-31
+```
+
+Suppression reasons are required. Expired suppressions no longer hide findings, and `mcp-audit policy check` reports them. See [policy documentation](docs/policy.md).
+
+## Version contracts
+
+CLI, ruleset, report schema, and manifest schema versions evolve independently. See [versioning](docs/versioning.md) and [report schema](docs/report-schema.md).
+
+Passing MCP Audit is technical security evidence, not a legal compliance determination.

mcp_capdiff-0.1.1/SECURITY.md

@@ -0,0 +1,11 @@
+# Security policy
+
+## Supported versions
+
+MCP Audit is pre-1.0. Security fixes are applied to the latest released minor version.
+
+## Reporting a vulnerability
+
+Please use GitHub private vulnerability reporting for the repository. Do not include secrets, production source code, or private scan output in a public issue.
+
+Include the affected MCP Audit version, a minimal reproducer, impact, and any known workaround. Maintainers should acknowledge a report within seven days and coordinate disclosure after a fix is available.

mcp_capdiff-0.1.1/action.yml

@@ -0,0 +1,58 @@
+name: MCP Audit
+description: Scan a Python/FastMCP server for security capability regressions and upload SARIF.
+inputs:
+  target:
+    description: Path to the MCP server source.
+    required: false
+    default: "."
+  base:
+    description: Git revision used as the approved security baseline.
+    required: false
+    default: "origin/main"
+  policy:
+    description: Optional MCP Audit policy file.
+    required: false
+    default: ""
+  sarif:
+    description: SARIF output path.
+    required: false
+    default: "mcp-audit.sarif"
+outputs:
+  exit-code:
+    description: MCP Audit exit code (0 pass, 1 policy failure).
+    value: ${{ steps.audit.outputs.exit-code }}
+runs:
+  using: composite
+  steps:
+    - name: Install MCP Audit
+      shell: bash
+      run: python -m pip install "${{ github.action_path }}"
+    - name: Generate security diff
+      id: audit
+      shell: bash
+      env:
+        MCP_AUDIT_TARGET: ${{ inputs.target }}
+        MCP_AUDIT_BASE: ${{ inputs.base }}
+        MCP_AUDIT_POLICY: ${{ inputs.policy }}
+        MCP_AUDIT_SARIF: ${{ inputs.sarif }}
+      run: |
+        set +e
+        args=(diff "$MCP_AUDIT_TARGET" --base "$MCP_AUDIT_BASE" --format sarif --output "$MCP_AUDIT_SARIF")
+        if [[ -n "$MCP_AUDIT_POLICY" ]]; then
+          args+=(--policy "$MCP_AUDIT_POLICY")
+        fi
+        mcp-audit "${args[@]}"
+        code=$?
+        echo "exit-code=$code" >> "$GITHUB_OUTPUT"
+        if [[ $code -eq 2 ]]; then
+          exit 2
+        fi
+        exit 0
+    - name: Upload SARIF
+      uses: github/codeql-action/upload-sarif@v4
+      with:
+        sarif_file: ${{ inputs.sarif }}
+    - name: Enforce policy result
+      if: steps.audit.outputs.exit-code == '1'
+      shell: bash
+      run: exit 1

mcp_capdiff-0.1.1/docs/policy.md

@@ -0,0 +1,20 @@
+# Policy and suppressions
+
+Policies define CI thresholds and reviewed exceptions:
+
+```yaml
+policy:
+  ci:
+    fail_on: [critical, high]
+    max_risk_score: 60
+
+suppress:
+  - rule: MCP003
+    tool: internal_fetch
+    reason: "Egress is restricted to api.example.com by the service mesh"
+    expires: 2027-01-31
+```
+
+`reason` is required. `tool` is optional and omission suppresses the rule for all tools. `expires` is optional and must use `YYYY-MM-DD`; after that date the finding is no longer suppressed. `mcp-audit policy check` exits with status `1` when a suppression is expired and status `2` for invalid policy syntax.
+
+Suppressions are policy decisions, not proof that a finding is safe. Prefer a narrow tool-specific suppression with a short expiry.

mcp_capdiff-0.1.1/docs/precision-matrix.md

@@ -0,0 +1,18 @@
+# Rule precision matrix
+
+The release gate requires positive, negative, guarded, and edge coverage for the differentiating rules.
+
+| Rule | True positive | True negative | Guarded safe | Tricky edge |
+| --- | --- | --- | --- | --- |
+| MCP001 | `shell_direct` | `safe_server` | `shell_safe_allowlist` | `shell_dynamic_no_shell` detects a dynamic executable without `shell=True` |
+| MCP002 | `filesystem_unrestricted` | `sensitive_read_only` | `filesystem_restricted` | `filesystem_guard_after_read` rejects a late guard |
+| MCP003 | `url_arbitrary` | fixed URL in demo baseline | `url_allowlisted` | hostname mention and guard-after-request remain findings |
+| MCP005 | `sensitive_to_network` | `sensitive_read_only` | constrained source-only server | root corpus proves independent MCP contexts are isolated |
+| MCP016 | network expansion diff | unchanged diff | allowlist-added demo resolves risk | a new high-impact tool is an escalation |
+| MCP017 | approval-removed diff | preserved approval | approval-added demo resolves risk | capability field change is retained in machine output |
+
+MCP004, MCP007, and MCP010 also have paired approved/bounded and unapproved/unbounded fixtures. The matrix is intentionally tied to fixture and test names so reviewers can audit the claim.
+
+`postgresql_read_only` protects the semantic classifier from matching `post` inside `PostgreSQL`; `sql_destructive` proves that actual `DELETE`/`DROP` semantics still produce MCP004 and MCP010 with relevant evidence.
+
+Discovery is separately benchmarked by `fastmcp_async_annotations`, which must find both plain async `@mcp.tool()` and `@mcp.tool(annotations=ToolAnnotations(...))` decorators. Missing targets, unreadable trees, parse failures, and unexpected zero-tool scans fail closed.

mcp_capdiff-0.1.1/docs/releasing.md

@@ -0,0 +1,39 @@
+# Release checklist
+
+## Local preflight
+
+```bash
+uv sync --locked
+uv run pytest
+uv build
+```
+
+Install the wheel into a clean environment and verify `mcp-audit --version`, a passing scan, a failing scan, and `policy check`.
+
+## Contract review
+
+- Confirm CLI/package, ruleset, report schema, and manifest schema versions.
+- Review golden-output changes explicitly.
+- Update `CHANGELOG.md` and every affected rule page.
+- Confirm the composite Action references supported major action versions.
+
+## Publish
+
+1. Create and push the matching Git tag, such as `v0.1.1`.
+2. Publish the matching GitHub Release. This automatically runs `release.yml`.
+3. Verify `pipx install mcp-capdiff` in a clean environment.
+4. Run the tagged Action from the demo repository and confirm SARIF appears in Code Scanning.
+
+For the first release, configure a GitHub Actions Pending Trusted Publisher on PyPI before running the workflow:
+
+```text
+PyPI project name: mcp-capdiff
+Owner: robbyfa
+Repository: MCP-Audit
+Workflow filename: release.yml
+Environment: pypi
+```
+
+The GitHub environment must be named `pypi`. Publishing uses OIDC and does not require a `PYPI_TOKEN` secret. Manual workflow dispatches require a release tag; the workflow checks out that immutable tag and verifies that the distribution version matches it.
+
+The PyPI distribution is `mcp-capdiff`; the product remains MCP Audit and the installed command remains `mcp-audit`. Name availability is only guaranteed when the first release is registered.

mcp_capdiff-0.1.1/docs/report-schema.md

@@ -0,0 +1,32 @@
+# Report schema 1.0
+
+MCP Audit uses the same dataclass-backed objects for terminal, JSON, SARIF, manifest, and diff output. JSON reports identify the contract with:
+
+```json
+{
+  "schema_version": "1.0",
+  "report_type": "scan"
+}
+```
+
+Reports also include the CLI and ruleset versions. Manifests carry `schema_version` and `rules_version`; SARIF uses `semanticVersion` and `properties.rulesVersion` on the tool driver.
+
+`report_type` is either `scan` or `diff`.
+
+## Finding contract
+
+Every finding includes:
+
+- `rule_id`, `title`, and `severity` for identity and prioritization.
+- `tool` and `location` for the affected MCP surface.
+- `message` describing what was detected.
+- `impact` describing the security consequence.
+- `recommendation` describing the expected remediation.
+- `evidence`, a list of typed source locations and snippets.
+- `path`, `data_classification`, and `destination` for capability-graph findings.
+
+## Diff contract
+
+Diff reports contain before/after risk, new/removed/changed tools, field-level capability changes, new findings, resolved findings, and synthetic regression findings (`MCP016` and `MCP017`). Policies are evaluated against new and regression findings, not the entire current scan.
+
+Additive fields may be introduced in schema `1.x`. Removing or changing the meaning of an existing field requires a new major schema version.

mcp_capdiff-0.1.1/docs/rules/MCP001.md

@@ -0,0 +1,37 @@
+# MCP001: Arbitrary shell execution
+
+**Severity:** Critical
+
+## What it detects
+
+FastMCP tools that invoke `os.system`, dynamic `eval`/`exec`/`compile`, subprocess APIs with `shell=True`, or a model-controlled executable/interpreter payload.
+
+## Why it matters
+
+Model-controlled input may execute commands with the MCP server process's privileges.
+
+## Vulnerable example
+
+```python
+@mcp.tool()
+def run(command: str):
+    return subprocess.run(command, shell=True)
+```
+
+## Safe example
+
+```python
+@mcp.tool()
+def status(service: str):
+    if service not in {"api", "worker"}:
+        raise ValueError("unsupported")
+    return subprocess.run(["systemctl", "status", service])
+```
+
+## Remediation
+
+Expose explicit operations, allowlist values, pass argument arrays, and avoid a shell interpreter.
+
+## Limitations
+
+Ruleset 0.1 recognizes common standard-library execution APIs. Wrapper functions and indirect aliases may be missed.

mcp_capdiff-0.1.1/docs/rules/MCP002.md

@@ -0,0 +1,36 @@
+# MCP002: Unrestricted filesystem access
+
+**Severity:** High
+
+## What it detects
+
+Filesystem reads or writes reached by an MCP tool without a detected approved-root check before the operation.
+
+## Why it matters
+
+An attacker may read secrets or modify files available to the server process.
+
+## Vulnerable example
+
+```python
+@mcp.tool()
+def read_file(path: str):
+    return Path(path).read_text()
+```
+
+## Safe example
+
+```python
+resolved = (REPORT_ROOT / path).resolve()
+if REPORT_ROOT not in resolved.parents:
+    raise ValueError("invalid path")
+return resolved.read_text()
+```
+
+## Remediation
+
+Resolve the requested path, reject traversal outside an explicit root, and perform the check before access.
+
+## Limitations
+
+Ruleset 0.1 recognizes direct `Path` and `open` patterns. Framework-specific storage wrappers may require future adapters.