agentcheckup 0.4.0__tar.gz

agentcheckup-0.4.0/.gitattributes

@@ -0,0 +1,13 @@
+# Normalise line endings so Windows contributors don't produce whole-file diffs.
+* text=auto eol=lf
+
+# Windows-only scripts keep CRLF.
+*.ps1 text eol=crlf
+*.bat text eol=crlf
+*.cmd text eol=crlf
+
+# Binary-ish
+*.png binary
+*.jpg binary
+*.gif binary
+*.ico binary

agentcheckup-0.4.0/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,63 @@
+name: Bug report
+description: Something is wrong — a crash, a wrong finding, a missed finding.
+title: "[bug] "
+labels: ["bug"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for the report. The two most useful things you can include are the
+        **rule id** and a **minimal manifest** that reproduces it.
+
+  - type: textarea
+    id: what-happened
+    attributes:
+      label: What happened
+      description: What did agentcheckup do, and what did you expect instead?
+    validations:
+      required: true
+
+  - type: input
+    id: rule
+    attributes:
+      label: Rule id (if the report is about a specific rule)
+      placeholder: AGT040
+    validations:
+      required: false
+
+  - type: textarea
+    id: repro
+    attributes:
+      label: Minimal reproduction
+      description: >
+        The smallest manifest or config that triggers it. Redact anything
+        sensitive — a single tool entry is usually enough.
+      render: json
+      placeholder: |
+        {"tools": [{"name": "x", "description": "..."}]}
+    validations:
+      required: true
+
+  - type: input
+    id: command
+    attributes:
+      label: Command you ran
+      placeholder: agentcheckup scan tools.json --fail-on warning
+    validations:
+      required: true
+
+  - type: textarea
+    id: output
+    attributes:
+      label: Actual output
+      render: shell
+    validations:
+      required: false
+
+  - type: input
+    id: version
+    attributes:
+      label: agentcheckup version and Python version
+      placeholder: agentcheckup 0.1.0, Python 3.12.7, Windows 11
+    validations:
+      required: true

agentcheckup-0.4.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,8 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Question or idea
+    url: https://github.com/wanjiaqing/agentcheckup/discussions
+    about: For open-ended discussion, use Discussions instead of an issue.
+  - name: Security vulnerability
+    url: https://github.com/wanjiaqing/agentcheckup/security/advisories/new
+    about: Please report security issues privately, not in a public issue.

agentcheckup-0.4.0/.github/ISSUE_TEMPLATE/rule_request.yml

@@ -0,0 +1,72 @@
+name: Rule request
+description: Propose a new lint rule, or argue that an existing one is wrong.
+title: "[rule] "
+labels: ["rule"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Rules are the point of this project. A good proposal is mostly *evidence*:
+        what goes wrong in the real world, and why a linter is the right place to
+        catch it.
+
+  - type: dropdown
+    id: kind
+    attributes:
+      label: What kind of request is this?
+      options:
+        - New rule
+        - Existing rule has false positives
+        - Existing rule should be a different severity
+        - Existing rule misses a case
+    validations:
+      required: true
+
+  - type: input
+    id: rule-id
+    attributes:
+      label: Rule id (leave empty for a new rule)
+      placeholder: AGT015
+    validations:
+      required: false
+
+  - type: textarea
+    id: failure
+    attributes:
+      label: What goes wrong without this rule
+      description: >
+        Describe the concrete failure — a wrong tool call, a leaked credential, a
+        confusing retry loop. "It's bad practice" is not enough.
+    validations:
+      required: true
+
+  - type: textarea
+    id: example
+    attributes:
+      label: Example input that should (or should not) be flagged
+      render: json
+      placeholder: |
+        {"name": "read_file", "description": "TODO: describe this tool"}
+    validations:
+      required: true
+
+  - type: dropdown
+    id: severity
+    attributes:
+      label: Proposed severity
+      options:
+        - error — breaks a run or endangers the host
+        - warning — likely to cause wrong behaviour
+        - info — a human should decide
+    validations:
+      required: true
+
+  - type: textarea
+    id: false-positives
+    attributes:
+      label: What legitimate input might this wrongly flag?
+      description: >
+        This is the question that decides acceptance. If you cannot think of any,
+        you probably have not looked hard enough.
+    validations:
+      required: true

agentcheckup-0.4.0/.github/pull_request_template.md

@@ -0,0 +1,35 @@
+## What this changes
+
+<!-- One or two sentences. Link the issue if there is one. -->
+
+Closes #
+
+## Type
+
+- [ ] New rule
+- [ ] Rule fix (false positive / false negative / severity)
+- [ ] Loader or output format
+- [ ] Docs
+- [ ] Refactor (no behaviour change)
+
+## If this adds a rule
+
+- Rule id:
+- Severity:
+- What it catches:
+- **What legitimate input it might wrongly flag, and how you narrowed it:**
+
+## Checklist
+
+- [ ] `pytest` is green
+- [ ] `ruff check .` is clean
+- [ ] A test covers the new behaviour (both a positive and a negative case)
+- [ ] No new runtime dependency
+- [ ] `docs/rules.md` updated (for rule changes)
+- [ ] Rule id is unused — checked with `agentcheckup rules`
+- [ ] Severity is honest: `error` means "breaks a run or endangers the host"
+
+## Notes for the reviewer
+
+<!-- Anything you are unsure about. Saying "I'm not confident about the
+     false-positive rate here" is more useful than silence. -->

agentcheckup-0.4.0/.github/workflows/action-selftest.yml

@@ -0,0 +1,59 @@
+name: Action self-test
+
+# Exercises action.yml on a real runner, including spawning two MCP servers that
+# collide. Without this the action is only ever tested by reading it.
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  self-test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: ./
+        id: doctor
+        with:
+          config: tests/fixtures/ci_config.json
+          fail-on: never
+          sarif-file: agentcheckup.sarif
+
+      - name: Assert the action produced a usable result
+        run: |
+          set -eu
+          echo "score=${{ steps.doctor.outputs.score }}"
+          echo "grade=${{ steps.doctor.outputs.grade }}"
+          echo "findings=${{ steps.doctor.outputs.findings }}"
+
+          test -n "${{ steps.doctor.outputs.score }}" || { echo "::error::score output is empty"; exit 1; }
+          test -n "${{ steps.doctor.outputs.grade }}" || { echo "::error::grade output is empty"; exit 1; }
+          test -f agentcheckup.sarif || { echo "::error::sarif file was not written"; exit 1; }
+
+          # The fixture deliberately exposes read_file from two servers.
+          grep -q "AGT201" agentcheckup.sarif || {
+            echo "::error::expected a hard-collision finding in the SARIF output"
+            cat agentcheckup.sarif
+            exit 1
+          }
+
+      - name: Assert the gate fails when it should
+        run: |
+          set +e
+          # Re-run through the CLI directly so the action's own exit-code plumbing
+          # is not what we are measuring here.
+          PYTHONPATH=src python -m agentcheckup audit \
+            --no-discover \
+            --config tests/fixtures/ci_config.json \
+            --fail-on error
+          code=$?
+          if [ "$code" -ne 1 ]; then
+            echo "::error::expected exit 1 for an error-level finding, got $code"
+            exit 1
+          fi

agentcheckup-0.4.0/.github/workflows/ci.yml

@@ -0,0 +1,66 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+        python: ["3.10", "3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python }}
+      - name: Install dev tools
+        run: python -m pip install --upgrade pip pytest ruff
+      - name: Lint
+        run: ruff check .
+      - name: Test
+        run: pytest -q
+
+  self-lint:
+    # Dogfood: agentcheckup lints its own fixtures. If this job fails, the fixtures
+    # or a rule regressed.
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      # Required by upload-sarif. Without it the upload fails with
+      # "Resource not accessible by integration", which is how this job first
+      # broke in CI.
+      security-events: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Lint the clean fixture (must pass)
+        run: PYTHONPATH=src python -m agentcheckup scan tests/fixtures/good_tools.json --min-score 90
+      - name: Lint the dirty fixture (must fail)
+        run: |
+          set +e
+          PYTHONPATH=src python -m agentcheckup scan tests/fixtures/bad_tools.json --fail-on error
+          code=$?
+          if [ "$code" -ne 1 ]; then
+            echo "expected exit 1, got $code"
+            exit 1
+          fi
+      - name: Emit SARIF
+        if: always()
+        run: PYTHONPATH=src python -m agentcheckup scan tests/fixtures/bad_tools.json --format sarif --fail-on never > agentcheckup.sarif
+      - name: Upload SARIF
+        if: always()
+        # v3 is deprecated in December 2026.
+        uses: github/codeql-action/upload-sarif@v4
+        with:
+          sarif_file: agentcheckup.sarif

agentcheckup-0.4.0/.github/workflows/publish.yml

@@ -0,0 +1,59 @@
+name: Publish
+
+# Publishes to PyPI using Trusted Publishing (OIDC). No API token is stored
+# anywhere: PyPI trusts this repository's workflow identity directly.
+#
+# One-time setup on PyPI (the project does not exist yet, so use "pending
+# publisher"):
+#   https://pypi.org/manage/account/publishing/
+#     PyPI Project Name : agentcheckup
+#     Owner             : wanjiaqing
+#     Repository name   : agentcheckup
+#     Workflow name     : publish.yml
+#     Environment       : pypi
+
+on:
+  # Re-enable the release trigger once the PyPI pending publisher exists:
+  #   on: release: types: [published]
+  # Until then a release would run this and fail with "environment: MISSING",
+  # which is noise on the repository home page.
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Build sdist and wheel
+        run: |
+          python -m pip install --upgrade build
+          python -m build
+      - name: Check metadata
+        run: |
+          python -m pip install --upgrade twine
+          python -m twine check dist/*
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    # Must match the "Environment name" field of the PyPI pending publisher.
+    # PyPI recommends a dedicated environment so that only this job can publish.
+    environment: pypi
+    permissions:
+      id-token: write # required for Trusted Publishing
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1

agentcheckup-0.4.0/.gitignore

@@ -0,0 +1,31 @@
+# --- Python ---
+__pycache__/
+*.py[cod]
+*.egg-info/
+build/
+dist/
+.eggs/
+
+# --- Virtualenv (project-local, never committed) ---
+.venv/
+venv/
+env/
+.conda/
+
+# --- Test / lint caches ---
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.coverage
+htmlcov/
+
+# --- Editors / OS ---
+.vscode/
+.idea/
+*.swp
+.DS_Store
+Thumbs.db
+
+# --- Local scratch ---
+*.local.json
+scratch/

agentcheckup-0.4.0/CHANGELOG.md

@@ -0,0 +1,141 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and this project adheres
+to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+Rule ids are permanent: once shipped, an id never changes meaning. New rules get
+new ids, and removing a rule is a breaking change.
+
+## [Unreleased]
+
+## [0.4.0] - 2026-11-07
+
+Calibration release, driven by the first run against real servers rather than
+fixtures. Installing `mcp-server-fetch`, `mcp-server-git` and `mcp-server-time`
+and auditing all three produced **Grade F, 0/100, 89 findings across 18 tools**.
+A tool that scores the official reference implementations zero is a tool nobody
+trusts, so the scoring and the report both changed.
+
+### Changed
+
+- **Score is normalised per tool.** Budget is 20 points per tool; a flat
+  subtraction punished manifests for being large and scored the official servers
+  0/100. Calibrated against four real inputs: clean 100, official servers 87,
+  a collision-plus-token config 79, deliberately broken 36.
+- **Info findings are capped at 10 points total.** `AGT022` fired on 18 of 18
+  official tools and `AGT023` on 17 of 18 — a rule that fires on everything is a
+  preference, not a defect, and charging one point each buried the two findings
+  that mattered.
+- **Repeated findings collapse.** Twelve tools missing parameter descriptions now
+  print as one line naming all twelve instead of twelve near-identical blocks.
+  Info findings are summarised one line per rule. The official-server report went
+  from 40 lines to 18.
+- **`AGT011` (short description) is now `info`.** Short is a nudge; *missing* is
+  `AGT010`'s error.
+- **`AGT105` (server did not return a tool list) is now `info`.** It means "not
+  checked", not "your config is wrong" — it fired on a machine without `npx` and
+  dragged the score down for an environment issue.
+- **`AGT107` (plaintext credentials) is now `warning`.** It was `info`, which
+  under the new model made a stored token cost nothing.
+
+### Added
+
+- `tests/test_scoring.py`: score calibration, per-tool normalisation, info cap,
+  and aggregation behaviour, including the "12 findings print as one block" case.
+- README section documenting the score model with the four calibration inputs.
+
+### Notes
+
+- Conflict detection produced **zero false positives** across the 18 real tools
+  from the three official servers. That is the number this project lives or dies
+  on, and it held.
+
+## [0.3.0] - 2026-11-07
+
+Distribution release. The tool worked; nobody could reach it from CI.
+
+### Added
+
+- **GitHub Action** (`action.yml`). Installs from its own checkout, so pinning
+  the action version pins the tool version and a PyPI release is not required.
+  Inputs: `config`, `fail-on`, `min-score`, `sarif-file`, `args`. Outputs:
+  `score`, `grade`, `findings`, `exit-code` — published even when the gate fails,
+  so a workflow can gate on score without parsing the log.
+- `--json-out PATH` and `--sarif-out PATH`. One run now yields the human report,
+  machine-readable JSON and SARIF, instead of re-spawning every server to get a
+  second format.
+- Config discovery covers **27 paths**: Claude Desktop, Claude Code, Cursor,
+  VS Code, Cline, Roo Code, Kilo Code, Windsurf, Zed, Codex CLI, Continue, plus
+  project-local files.
+- VS Code's `mcp.servers` nesting and Zed's `context_servers` key are read.
+- Codex CLI's TOML config is parsed via `tomllib` on Python 3.11+, and skipped
+  rather than adding a dependency on 3.10.
+- `docs/github-action.md`.
+
+### Changed
+
+- Cline, Roo Code and Kilo Code paths are generated for every editor flavour on
+  every OS. Previously they were hardcoded Windows-only, so macOS and Linux users
+  got no Cline coverage at all.
+
+### Fixed
+
+- `discover()` walked a hardcoded Windows branch for extension paths; it now
+  derives from one platform table.
+
+## [0.2.0] - 2026-11-07
+
+Repositioned from "MCP manifest linter" to "agent setup doctor". The manifest
+rules were already there; what was missing was the one question no other tool
+answers: *what happens when five servers are loaded at once?*
+
+### Added
+
+- **Cross-server conflict detection** (`AGT201`–`AGT203`). Hard name collisions,
+  normalised-name collisions, and description overlap between tools from
+  different servers. No embeddings, no network — token Jaccard with a shared-head-
+  noun guard to keep the false-positive rate usable.
+- **`agentcheckup` with no arguments** now audits everything it can find: Claude
+  Desktop, Claude Code (including per-project `mcpServers`), Cursor, Windsurf,
+  Cline, and project-local `.mcp.json` / `.cursor/mcp.json` / `.vscode/mcp.json`.
+- **`audit` subcommand** with `--config`, `--no-discover`, `--no-spawn`, `--timeout`.
+- `AGT105` for servers that fail to start or cannot be introspected over stdio.
+  Reported and skipped, never fatal.
+- `StdioSession` accepts an argv list and an `env` dict, so servers are launched
+  without shell quoting (which matters on Windows) and with the environment their
+  config declares.
+- `discovery` module with config discovery and Claude Code's nested project layout.
+- Report splits findings into **CONFLICTS** and **RISKS** sections for `audit`.
+
+### Changed
+
+- Renamed from `mcplint` to `agentcheckup`; rule prefix `MCP` → `AGT`.
+- `Report` carries `server_count`; JSON output includes it.
+- The text header for `audit` reports servers, tools and tokens on one line.
+
+### Fixed
+
+- Plural stemming treated `files` as an `-es` plural and produced `fil`, so
+  `read_file` and `read_files` were not recognised as the same word.
+- Conflict scoring was name-weighted enough that two tools with 71% description
+  overlap scored 0.543 and slipped under a 0.55 threshold. Description overlap is
+  now a primary signal, because that is what the model actually reads.
+
+## [0.1.0] - 2026-11-07
+
+First release. 32 rules, three output formats, two input modes.
+
+### Added
+
+- `scan` for saved manifests (`{"tools": [...]}`, a bare list, a JSON-RPC
+  `result.tools` envelope, or a session log with `messages`) and for live stdio
+  servers via `--cmd`.
+- `config` to audit MCP client config files (`mcpServers`, `servers`,
+  `mcp_servers`).
+- `rules` to list every rule with its id and severity.
+- 32 rules across naming, documentation, schema, safety, injection, and config.
+- Output formats: `text`, `json`, `sarif` (SARIF 2.1.0 for GitHub code scanning).
+- `--fail-on error|warning|info|never` and `--min-score N` for CI gates.
+- Manifest token estimation, reported on every run.
+- Zero runtime dependencies. Python 3.10+.

agentcheckup-0.4.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,40 @@
+# Code of Conduct
+
+## The short version
+
+Be decent. Assume competence. Attack the idea, not the person.
+
+## Scope
+
+This applies to issues, pull requests, discussions, commit messages, and any
+other space where you are representing this project.
+
+## Expected
+
+- Critique the code, the rule, or the reasoning — with specifics.
+- Assume the other person knows something you do not. Most rule disagreements
+  come from different deployment contexts, not from carelessness.
+- Accept "no" on a PR gracefully. A declined rule is not a judgement of you.
+- Give credit for work you build on.
+
+## Not acceptable
+
+- Insults, dismissiveness, or "this is obviously wrong" without explanation.
+- Harassment of any kind, public or private.
+- Discriminatory language or imagery.
+- Publishing someone's private information.
+- Sustained disruption of discussions.
+
+## Enforcement
+
+Maintainers may edit, remove, or reject contributions that violate this, and may
+block repeat offenders. If you need to report something, open a private security
+advisory or contact the maintainer directly.
+
+Reports are read by the maintainer and kept confidential.
+
+## Attribution
+
+This is a short, project-specific code of conduct rather than a full adoption of
+the Contributor Covenant. It is deliberately brief because a long document nobody
+reads protects nobody.