mcplint-cli 0.1.0__tar.gz

mcplint_cli-0.1.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,29 @@
+---
+name: Bug report
+about: Something in MCPLint isn't working as documented
+title: ""
+labels: bug
+---
+
+**What happened**
+
+**What you expected**
+
+**Command you ran**
+
+```
+mcplint ...
+```
+
+**MCP server involved** (or a minimal repro server, if you can share one)
+
+**Environment**
+- MCPLint version: `mcplint --version` output, or commit SHA
+- Python version:
+- OS:
+
+**Relevant output**
+
+```
+paste terminal output / JSON / SARIF here
+```

mcplint_cli-0.1.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,23 @@
+---
+name: Feature request
+about: Propose a new rule, command, or reporter
+title: ""
+labels: enhancement
+---
+
+**Problem**
+
+What's the concrete scenario this doesn't handle today? (An example MCP
+server / tool description that should be flagged or reported differently is
+the most useful thing you can include.)
+
+**Proposed solution**
+
+- New rule? Name it, describe the deterministic check, and note the
+  severity/confidence you'd expect.
+- New command/flag? Sketch the CLI invocation.
+- New reporter/format? What should the output look like?
+
+**Alternatives considered**
+
+**Willing to submit a PR?**

mcplint_cli-0.1.0/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,11 @@
+## What
+
+## Why
+
+## Testing
+
+- [ ] `ruff check src tests examples` passes
+- [ ] `mypy src` passes
+- [ ] `pytest --cov=mcplint` passes, new code has tests
+- [ ] Updated `CHANGELOG.md` under `[Unreleased]`
+- [ ] Updated `IMPLEMENTATION_STATUS.md` if this changes what's done/incomplete

mcplint_cli-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,101 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+# Least privilege by default; no job in this workflow ever needs write
+# access to the repo, and nothing here ever touches PyPI or creates a
+# release — that only happens from release.yml, triggered by a tag push.
+permissions:
+  contents: read
+
+jobs:
+  lint-typecheck-test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install MCPLint (dev extras)
+        run: pip install -e ".[dev]"
+
+      - name: Ruff check
+        run: ruff check src tests examples
+
+      - name: Ruff format check
+        run: ruff format --check src tests examples
+
+      - name: MyPy
+        run: mypy src
+
+      - name: Unit + CLI tests with coverage
+        run: pytest --cov=mcplint --cov-report=term-missing --cov-fail-under=80 tests/unit tests/cli
+
+      - name: Integration tests (spawns real example MCP servers over stdio)
+        run: pytest tests/integration
+
+  build-and-verify-package:
+    runs-on: ubuntu-latest
+    needs: lint-typecheck-test
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install build backend and metadata checker
+        run: pip install build twine
+
+      - name: Build sdist and wheel
+        run: python -m build
+
+      - name: Inspect package metadata
+        run: twine check dist/*
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  install-and-smoke-test:
+    runs-on: ubuntu-latest
+    needs: build-and-verify-package
+    steps:
+      - uses: actions/checkout@v4 # only for examples/good_server, not for source install
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Install the built wheel into a clean environment
+        run: |
+          python -m venv /tmp/mcplint-smoke
+          /tmp/mcplint-smoke/bin/pip install --quiet dist/*.whl
+
+      - name: CLI smoke tests
+        # Activate the venv (rather than calling mcplint by full path) so the
+        # `python examples/good_server/server.py` subprocess mcplint spawns
+        # also resolves to this venv's python — which has `mcp` installed as
+        # mcplint's own runtime dependency. Calling the binary by full path
+        # would leave `python` on PATH pointing at the bare system
+        # interpreter, which doesn't have `mcp` installed, and the spawned
+        # server would fail to import.
+        run: |
+          source /tmp/mcplint-smoke/bin/activate
+          mcplint --version
+          mcplint rules
+          mcplint scan --server "python examples/good_server/server.py"

mcplint_cli-0.1.0/.github/workflows/example-scan-mcp-server.yml

@@ -0,0 +1,54 @@
+# Example workflow for MCPLint users: gate a PR on MCP tool-contract quality.
+#
+# This is a *documented example*, not part of MCPLint's own CI — copy it into
+# your MCP server's repo and adjust the "Start the MCP server" step. It:
+#   1. Installs MCPLint
+#   2. Invokes the local MCP server
+#   3. Runs the scanner
+#   4. Uploads SARIF to GitHub code scanning
+#   5. Fails the job on error-severity findings
+#
+# No custom JavaScript Action is required — this composite-style workflow
+# (plain steps) is sufficient, per MCPLint's own security stance of never
+# running a hosted service that executes arbitrary server commands.
+name: mcplint-scan-example
+
+on:
+  workflow_dispatch: # example only — trigger manually; adapt `on:` for real use
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      # 1. Install MCPLint (PyPI distribution name is mcplint-cli; the
+      # installed command is still `mcplint`)
+      - name: Install MCPLint
+        run: pip install mcplint-cli
+
+      # 2 & 3. Invoke the local MCP server and run the scanner.
+      # Replace the --server command with however your server is started —
+      # `python server.py`, `node server.js`, a built binary, etc.
+      - name: Scan the MCP server
+        run: |
+          mcplint scan \
+            --server "python examples/good_server/server.py" \
+            --format sarif \
+            --output mcplint-results.sarif \
+            --fail-on error
+
+      # 4. Upload SARIF to GitHub code scanning (visible in the Security tab).
+      - name: Upload SARIF
+        if: always()
+        uses: github/codeql-action/upload-sarif@v3
+        with:
+          sarif_file: mcplint-results.sarif
+
+      # 5. `mcplint scan` above already exits non-zero on error-severity
+      # findings (--fail-on error is the default), which fails this job.

mcplint_cli-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,125 @@
+name: Release
+
+# Production publishing happens ONLY from an explicit `vX.Y.Z` tag push.
+# Nothing in ci.yml (which runs on every PR and every push to main) has
+# any path to PyPI credentials or release-creation permissions — only this
+# workflow does, and only after a maintainer pushes a tag (tag creation
+# requires write access to the repo; it cannot be triggered by an
+# untrusted pull request).
+on:
+  push:
+    tags:
+      - "v*.*.*" # e.g. v0.1.0 — the build job's version-match step rejects anything non-semver
+
+permissions:
+  contents: read
+
+jobs:
+  # Build once. Every later job consumes these exact artifacts rather than
+  # rebuilding, so what's tested is byte-for-byte what gets published.
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install MCPLint (dev extras) for the pre-publish test run
+        run: pip install -e ".[dev]"
+
+      - name: Verify the tag matches the package version
+        run: |
+          TAG_VERSION="${GITHUB_REF_NAME#v}"
+          PKG_VERSION="$(python -c 'import mcplint; print(mcplint.__version__)')"
+          echo "Tag version:     $TAG_VERSION"
+          echo "Package version: $PKG_VERSION"
+          if [ "$TAG_VERSION" != "$PKG_VERSION" ]; then
+            echo "::error::Tag $GITHUB_REF_NAME does not match src/mcplint/__about__.py ($PKG_VERSION)"
+            exit 1
+          fi
+
+      - name: Ruff check
+        run: ruff check src tests examples
+
+      - name: Ruff format check
+        run: ruff format --check src tests examples
+
+      - name: MyPy
+        run: mypy src
+
+      - name: Full test suite
+        run: pytest --cov=mcplint --cov-fail-under=80 tests/unit tests/cli
+
+      - name: Integration tests
+        run: pytest tests/integration
+
+      - name: Install build backend and metadata checker
+        run: pip install build twine
+
+      - name: Build sdist and wheel
+        run: python -m build
+
+      - name: Inspect package metadata
+        run: twine check dist/*
+
+      - name: Install the built wheel into a clean environment and smoke-test it
+        run: |
+          python -m venv /tmp/mcplint-release-smoke
+          /tmp/mcplint-release-smoke/bin/pip install --quiet dist/*.whl
+          source /tmp/mcplint-release-smoke/bin/activate
+          mcplint --version
+          mcplint scan --server "python examples/good_server/server.py"
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  # Publish the exact artifacts from `build` to PyPI via Trusted Publishing.
+  # No long-lived token: the `id-token: write` permission lets PyPI verify
+  # this specific workflow run via OIDC. This is the only job in the whole
+  # repo with any PyPI-facing permission.
+  publish-to-pypi:
+    runs-on: ubuntu-latest
+    needs: build
+    environment:
+      name: pypi
+      url: https://pypi.org/p/mcplint-cli
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          attestations: true
+
+  # Create the GitHub Release and attach the same artifacts, after PyPI
+  # publish succeeds so a Release is never created for a build that failed
+  # to publish.
+  github-release:
+    runs-on: ubuntu-latest
+    needs: publish-to-pypi
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Create GitHub Release
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          gh release create "${{ github.ref_name }}" dist/* \
+            --repo "${{ github.repository }}" \
+            --title "${{ github.ref_name }}" \
+            --generate-notes

mcplint_cli-0.1.0/.gitignore

@@ -0,0 +1,11 @@
+.venv/
+__pycache__/
+*.pyc
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+*.egg-info/
+dist/
+build/
+.coverage
+htmlcov/

mcplint_cli-0.1.0/.pre-commit-config.yaml

@@ -0,0 +1,27 @@
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.6.9
+    hooks:
+      - id: ruff
+        args: [--fix]
+      - id: ruff-format
+
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.11.2
+    hooks:
+      - id: mypy
+        additional_dependencies:
+          - pydantic>=2.7
+          - types-pyyaml
+          - types-jsonschema
+        args: [src]
+        pass_filenames: false
+
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.6.0
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-toml
+      - id: check-merge-conflict

mcplint_cli-0.1.0/CHANGELOG.md

@@ -0,0 +1,96 @@
+# Changelog
+
+## [Unreleased]
+
+### Added
+- Project scaffolding (`pyproject.toml`, package layout, dev tooling).
+- Core Pydantic models: `ArtifactMetadata`, `SourceLocation`, `ParameterContract`,
+  `ToolAnnotation`, `ToolContract`, `MCPServerSnapshot`.
+- Canonicalization module: deterministic stable tool IDs, byte-stable snapshot JSON.
+- Stdio MCP client: `collect_stdio_snapshot`, mapping SDK `Tool` objects to `ToolContract`.
+- `examples/good_server`: a well-documented example MCP server fixture.
+- `mcplint inspect --server "<command>"` CLI command with Rich table output.
+- `Severity`, `Finding`, `RuleMetadata`, `LintReport` models.
+- `Rule` ABC, `RuleContext`, `RuleRegistry` (with entry-point plugin loading support).
+- Five built-in rules: `missing-tool-description`, `description-repeats-name`,
+  `vague-tool-description`, `missing-parameter-description`,
+  `schema-description-type-conflict`.
+- `lint_snapshot()` engine wiring the registry over every tool in a snapshot.
+- Snapshot persistence: `save_snapshot`/`load_snapshot`.
+- Terminal and JSON reporters (`render_terminal`, `render_json`).
+- `mcplint snapshot --server "<command>" --output PATH` CLI command.
+- `mcplint scan --server "<command>"` / `mcplint scan --snapshot PATH` CLI command
+  with `--format terminal|json` and `--fail-on error|warning|never`.
+- Remaining 10 built-in rules: `missing-return-semantics`,
+  `undocumented-error-behaviour`, `undocumented-required-constraint`,
+  `tool-name-action-conflict`, `destructive-tool-without-warning`,
+  `state-changing-tool-marked-read-only`, `ambiguous-tool-overlap`,
+  `missing-tool-distinction`, `excessive-description-length`,
+  `undefined-domain-term` — all 15 spec rules now implemented.
+- Semantic ambiguity engine (`compute_ambiguity`): name/description/parameter
+  Jaccard similarity with structured, inspectable evidence (shared verbs,
+  shared entities, overlapping parameters, absent exact-vs-search /
+  one-vs-many / read-vs-write distinctions).
+- `mcplint.yaml` configuration loading (`MCPLintConfig`): severity overrides,
+  ambiguity/description-length thresholds, per-tool rule ignores, benchmark
+  defaults. Wired into `scan --config`.
+- `mcplint rules` CLI command listing the full rule catalogue.
+- `examples/bad_server`: intentionally triggers all 15 built-in rules
+  (verified by an integration test).
+- `examples/ambiguous_customer_server`: get/search/update/delete customer
+  tools for the upcoming benchmark dataset, verified to trigger
+  `ambiguous-tool-overlap`.
+- `examples/good_server` reworked to use `Annotated[..., Field(description=...)]`
+  parameters and explicit tool annotations; verified to produce zero findings.
+- Benchmark models: `ExpectedToolCall`, `BenchmarkCase`, `BenchmarkDataset`,
+  `ActualToolCall`, `BenchmarkTrial`, `BenchmarkResult`.
+- `ToolCallingProvider` protocol + `ProviderResult`; `FakeProvider` for
+  network-free tests and dry runs.
+- Deterministic scorer (`score_trial`, `aggregate_metrics`): exact
+  tool-selection accuracy, valid-argument rate (JSON Schema validated),
+  required-argument accuracy, forbidden-tool invocation rate, no-tool rate,
+  mean/P95 latency, total estimated cost, per-case pass rate, and stability
+  across repeated trials. No LLM judge.
+- `mcplint benchmark DATASET --server "<command>" --provider fake --runs N`
+  CLI command with terminal/JSON output and `--output PATH`.
+  `--provider anthropic|openai` is recognized but raises a clear
+  "not implemented yet" error (Phase 5).
+- `examples/ambiguous_customer_server/customer-tools.evals.yaml`: the
+  get/search/update/delete-customer confusion dataset required by the spec.
+- `AnthropicProvider`: verified against the real `anthropic` SDK (0.120.0);
+  API errors surface as a scored `ProviderResult.error`, not a crash;
+  illustrative per-model cost table. `OpenAIProvider` is a typed stub.
+  `--provider anthropic` now works end-to-end via `mcplint benchmark`.
+- `ComparisonReport` model + `compare/differ.py` (`diff_tool_names`,
+  `diff_tool_contracts`, `diff_findings`, `diff_ambiguity`, `diff_benchmarks`).
+- `mcplint compare --baseline --candidate [--dataset --min-accuracy-delta]`
+  CLI command with a Rich terminal reporter; fails CI (exit 1) when the
+  accuracy delta falls below the threshold.
+- `RewriteSuggestion` model + `fix/suggest.py`: deterministic, schema-derived
+  rewrite suggestions (output shape, enum/numeric constraints, destructive
+  warnings, tool-distinction placeholders, description truncation).
+  Markdown patch report reporter.
+- `mcplint fix --snapshot [--output PATH]` CLI command — never overwrites
+  source files; `--llm-provider` is recognized but explicitly rejected
+  until LLM-assisted rewriting is implemented.
+- `ScoreBreakdown` model + `core/score.py::compute_score`: explainable 0-100
+  score, capped per category (critical/error, warning/info, ambiguity,
+  schema completeness, safety clarity, optional benchmark accuracy), with
+  an explicit not-a-scientific-metric disclaimer. Shown in the terminal
+  reporter.
+- SARIF 2.1.0 reporter (`reporters/sarif.py`) with a full rule catalogue
+  and per-finding results/locations.
+- Standalone, self-contained HTML report (`reporters/html.py` + Jinja2
+  template, embedded CSS, no external requests): score, findings by
+  severity, tool inventory, ambiguity pairs, and optional
+  benchmark/comparison/fix-suggestion sections.
+- `mcplint scan --format sarif|html` and a new global `--output PATH` to
+  also write the rendered report to a file.
+- `mcplint --version`.
+- Packaging: `LICENSE` (MIT), `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`,
+  `SECURITY.md`, GitHub issue/PR templates, `ci.yml`/`release.yml`
+  GitHub Actions, a documented example scan-and-upload-SARIF workflow,
+  `Dockerfile`, `.pre-commit-config.yaml`.
+- Full README rewrite: quickstart, real example output, architecture,
+  rule catalogue, ambiguity engine explanation, benchmark/compare/fix/CI/
+  plugin guides, limitations, roadmap, and a comparison section.

mcplint_cli-0.1.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,51 @@
+# Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and maintainers pledge to make participation in
+the MCPLint project a harassment-free experience for everyone, regardless of
+age, body size, visible or invisible disability, ethnicity, sex
+characteristics, gender identity and expression, level of experience,
+education, socio-economic status, nationality, personal appearance, race,
+religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment:
+
+- Demonstrating empathy and kindness toward other people
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Focusing on what is best for the community
+
+Examples of unacceptable behavior:
+
+- The use of sexualized language or imagery, and unwelcome sexual attention
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information without explicit permission
+
+## Enforcement Responsibilities
+
+Project maintainers are responsible for clarifying and enforcing our
+standards of acceptable behavior and will take appropriate and fair
+corrective action in response to any behavior they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies within all project spaces (issues, pull
+requests, discussions) and when an individual is officially representing the
+project in public spaces.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the project maintainers via the contact method listed in
+`SECURITY.md`. All complaints will be reviewed and investigated promptly and
+fairly.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org),
+version 2.1.

mcplint_cli-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,75 @@
+# Contributing to MCPLint
+
+Thanks for considering a contribution. MCPLint is a young project — the
+architecture in `IMPLEMENTATION_STATUS.md` and this guide are both likely to
+evolve.
+
+## Development setup
+
+```bash
+git clone https://github.com/mcplint/mcplint.git
+cd mcplint
+python3.11 -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+`dev` includes `anthropic` so the Anthropic provider's test suite runs by
+default. If you only want the deterministic linter, `pip install -e .` is
+enough — no extras are required for `inspect`, `snapshot`, `scan`, `rules`,
+or `fix`.
+
+## Before opening a PR
+
+Run the same gate CI runs:
+
+```bash
+ruff check src tests examples
+mypy src
+pytest --cov=mcplint --cov-report=term-missing
+```
+
+All three must pass. New code should come with tests — see
+`tests/unit/core/rules/` for the pattern most rules follow (a flag case and
+a pass case per behavior, using hand-built `ToolContract`/`RuleContext`
+fixtures, no live MCP server needed).
+
+## Adding a built-in rule
+
+1. Subclass `mcplint.core.rules.base.Rule` in the appropriate module under
+   `src/mcplint/core/rules/` (`description_rules.py`, `schema_rules.py`,
+   `safety_rules.py`, `completeness_rules.py`, or a new file if it's a new
+   category).
+2. Set `id`, `title`, `description`, `default_severity`, and optionally
+   `tags` as class attributes; implement `check(self, tool, context) ->
+   list[Finding]`.
+3. Register it in `BUILTIN_RULES` in `core/rules/builtin.py`.
+4. Add unit tests: at least one case that triggers the finding and one that
+   doesn't.
+5. If it's meant to fire on `examples/bad_server`, add a tool there and
+   confirm `tests/integration/test_bad_server.py` still passes (it asserts
+   every built-in rule ID fires at least once against that server).
+
+Third-party rules don't need to touch this repo at all — see the plugin
+guide in the README for the `mcplint.rules` entry-point mechanism.
+
+## Adding a benchmark provider
+
+Implement the `ToolCallingProvider` protocol (`benchmark/providers/base.py`):
+a `name`/`model` pair and an async `run(prompt, tools) -> ProviderResult`.
+See `benchmark/providers/anthropic_provider.py` for a real example and
+`benchmark/providers/fake.py` for a test-only one. Wire it into
+`benchmark/providers/factory.py::create_provider`.
+
+## Commit and PR conventions
+
+- Keep commits scoped to one logical change; the existing history (`git log
+  --oneline`) is the style to match.
+- Update `CHANGELOG.md` under `[Unreleased]` for user-visible changes.
+- Update `IMPLEMENTATION_STATUS.md` if you complete, start, or discover a
+  gap in a tracked feature.
+
+## Reporting bugs / requesting features
+
+Use the GitHub issue templates. For security issues, see `SECURITY.md`
+instead of opening a public issue.

mcplint_cli-0.1.0/Dockerfile

@@ -0,0 +1,17 @@
+# MCPLint CLI image. The MCP server you point --server at is spawned as a
+# subprocess *inside this container*, so it (and any language runtime it
+# needs, e.g. Node) must be available on the image or mounted in. This base
+# image ships Python only — extend it if your server needs something else.
+FROM python:3.11-slim AS base
+
+WORKDIR /app
+
+COPY pyproject.toml README.md ./
+COPY src ./src
+
+RUN pip install --no-cache-dir .
+
+WORKDIR /workspace
+
+ENTRYPOINT ["mcplint"]
+CMD ["--help"]