mcpvitals 0.1.0__tar.gz

mcpvitals-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,20 @@
+name: release
+on:
+  push:
+    tags: ["v*"]
+  workflow_dispatch:
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write   # trusted publishing, no token needed
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: pip install build
+      - run: python -m build
+      - uses: pypa/gh-action-pypi-publish@release/v1

mcpvitals-0.1.0/.github/workflows/tests.yml

@@ -0,0 +1,12 @@
+name: tests
+on: [push, pull_request, workflow_dispatch]
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: pip install -e ".[dev]" mcp
+      - run: pytest -q

mcpvitals-0.1.0/.gitignore

@@ -0,0 +1,6 @@
+__pycache__/
+*.pyc
+.venv/
+.DS_Store
+dist/
+*.egg-info/

mcpvitals-0.1.0/.mcpvitals/badge.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="150" height="20" role="img" aria-label="mcp health: 100 A"><rect width="90" height="20" fill="#555"/><rect x="90" width="60" height="20" fill="#3fb950"/><g fill="#fff" font-family="Verdana" font-size="11"><text x="8" y="14">mcp health</text><text x="98" y="14">100 A</text></g></svg>

mcpvitals-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,31 @@
+# Contributing to mcpvitals
+
+Thanks for helping make MCP servers healthier.
+
+## Add a check in three steps
+
+Every check is a pure function over a `ServerSnapshot`, so adding one is small and testable:
+
+1. Write a function in the right module under `src/mcpvitals/checks/`:
+   ```python
+   from mcpvitals.checks import check
+   from mcpvitals.models import Finding, Level
+
+   @check
+   def my_rule(snapshot, config):
+       out = []
+       for t in snapshot.tools:
+           if is_bad(t):
+               out.append(Finding("MV0XX", Level.WARN, "what's wrong", hint="how to fix", tool=t.name))
+       return out
+   ```
+2. Give it a fresh, stable `MV###` code (never reuse or renumber an existing one).
+3. Add a golden-fixture test in `tests/` asserting the exact finding on a fixture snapshot. No network in unit tests.
+
+Run `pytest -q` before opening a PR. CI runs the full suite.
+
+## Principles
+
+- Checks do zero I/O. Only `introspect.py` touches the network.
+- Deterministic output. If a check needs token counting, take a counter via `config` so tests stay stable.
+- Human-voiced messages, no em-dashes.

mcpvitals-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ContextJet.ai
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mcpvitals-0.1.0/PKG-INFO

@@ -0,0 +1,109 @@
+Metadata-Version: 2.4
+Name: mcpvitals
+Version: 0.1.0
+Summary: Vital signs for your MCP servers: health score, token cost, tool-confusion, migration readiness.
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: mcp>=1.0
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == 'dev'
+Provides-Extra: tokens
+Requires-Dist: tiktoken>=0.7; extra == 'tokens'
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+# mcpvitals
+
+**Vital signs for your MCP servers. One command gives you a health score, plus the checks nobody else runs.**
+
+[![tests](https://github.com/ContextJet-ai/mcpvitals/actions/workflows/tests.yml/badge.svg)](https://github.com/ContextJet-ai/mcpvitals/actions/workflows/tests.yml)
+[![PyPI](https://img.shields.io/pypi/v/mcpvitals?color=1f6feb)](https://pypi.org/project/mcpvitals/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-lightgrey.svg)](LICENSE)
+[![by ContextJet.ai](https://img.shields.io/badge/by-ContextJet.ai-1f6feb)](https://www.contextjetai.com)
+
+</div>
+
+> [!NOTE]
+> An audit of ~1,850 public MCP servers found **52% abandoned and only 17% production-ready**, and a separate benchmark traced **56.7% of agent failures to tool misuse** (wrong tool, bad arguments). Registries have no health signal, so a broken server looks identical to a good one. `mcpvitals` is the checkup.
+
+## 60-second start
+
+No install. Point it at any MCP server (a stdio command or an http url):
+
+```bash
+uvx mcpvitals check "npx -y @your/mcp-server"
+```
+
+```text
+mcp health: 40 (F)
+
+  x MV020 [run_sql] tool 'run_sql' appears to contain a hard-coded secret
+      -> never ship keys in tool metadata; use env vars
+  x MV022 [run_sql] description contains model-directed instructions (poisoning surface)
+      -> tool descriptions should describe, not instruct the model
+  ! MV030 [run_sql] tool 'run_sql' costs ~410 tokens to expose
+      -> trim the description/schema; it inflates every request
+  ! MV040 tools 'get_user' and 'fetch_user' are 0.91 similar; agents may pick the wrong one
+      -> differentiate names/descriptions or merge them
+  i MV050 protocol 2025-03-26 predates the 2026-07-28 stateless spec
+```
+
+## What it checks
+
+Most linters stop at basic conformance. `mcpvitals` runs those, plus four things no other tool does:
+
+| Area | What it catches |
+|---|---|
+| **Conformance** | empty tool descriptions, duplicate tool names, malformed schemas |
+| **Reliability** | parameters with no `required` list, untyped params, error-path gaps |
+| **Security hygiene** | secrets in tool metadata, over-broad tools (`run_sql`, `exec`), model-directed instructions in descriptions (the tool-poisoning surface) |
+| **Token cost** | how many tokens each tool burns in the context window, and the total cost to connect the server |
+| **Tool confusion** | pairs of tools an agent will mix up, predicted from name/description overlap *before* it happens in production |
+| **Migration readiness** | what breaks under the 2026-07-28 stateless spec |
+
+Every check is a pure function scored deterministically. Run `mcpvitals check --json` for machine output.
+
+## Add the health badge to your README
+
+`mcpvitals` emits a shareable badge so a good score is visible at a glance:
+
+```bash
+mcpvitals check "npx your-server" --badge badge.svg
+```
+
+Or use the shields.io endpoint output to keep it live.
+
+## Use it in CI
+
+Gate your server on every push with the bundled GitHub Action:
+
+```yaml
+- uses: ContextJet-ai/mcpvitals@v0
+  with:
+    target: "python -m your_server"
+    strict: "true"   # fail the build on any error-level finding
+```
+
+## Track and monitor (not just a one-shot check)
+
+Pin a server's tools and get alerted when they change. This catches **rug-pulls**, where a server you already approved silently mutates a tool's schema or description after the fact:
+
+```bash
+mcpvitals watch "npx your-server"           # pins tools to mcp.lock
+mcpvitals watch "npx your-server" --check    # re-run in CI; fails if a pinned tool changed
+```
+
+Probe a running server for live health (uptime and latency):
+
+```bash
+mcpvitals monitor "https://your-server/mcp" --count 5
+# probe 1/5: up 210ms ... uptime 100.0%  avg 207ms
+```
+
+## Why this exists
+
+The MCP ecosystem grew faster than the tooling around it. There are thousands of servers and no standard way to tell whether one is any good, safe, or cheap to run. `mcpvitals` gives authors a one-command checkup before they publish, and gives everyone else a way to vet a server before they trust it.
+
+Built by [ContextJet.ai](https://www.contextjetai.com). MIT licensed. Contributions welcome, see [CONTRIBUTING.md](CONTRIBUTING.md).

mcpvitals-0.1.0/README.md

@@ -0,0 +1,95 @@
+<div align="center">
+
+# mcpvitals
+
+**Vital signs for your MCP servers. One command gives you a health score, plus the checks nobody else runs.**
+
+[![tests](https://github.com/ContextJet-ai/mcpvitals/actions/workflows/tests.yml/badge.svg)](https://github.com/ContextJet-ai/mcpvitals/actions/workflows/tests.yml)
+[![PyPI](https://img.shields.io/pypi/v/mcpvitals?color=1f6feb)](https://pypi.org/project/mcpvitals/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-lightgrey.svg)](LICENSE)
+[![by ContextJet.ai](https://img.shields.io/badge/by-ContextJet.ai-1f6feb)](https://www.contextjetai.com)
+
+</div>
+
+> [!NOTE]
+> An audit of ~1,850 public MCP servers found **52% abandoned and only 17% production-ready**, and a separate benchmark traced **56.7% of agent failures to tool misuse** (wrong tool, bad arguments). Registries have no health signal, so a broken server looks identical to a good one. `mcpvitals` is the checkup.
+
+## 60-second start
+
+No install. Point it at any MCP server (a stdio command or an http url):
+
+```bash
+uvx mcpvitals check "npx -y @your/mcp-server"
+```
+
+```text
+mcp health: 40 (F)
+
+  x MV020 [run_sql] tool 'run_sql' appears to contain a hard-coded secret
+      -> never ship keys in tool metadata; use env vars
+  x MV022 [run_sql] description contains model-directed instructions (poisoning surface)
+      -> tool descriptions should describe, not instruct the model
+  ! MV030 [run_sql] tool 'run_sql' costs ~410 tokens to expose
+      -> trim the description/schema; it inflates every request
+  ! MV040 tools 'get_user' and 'fetch_user' are 0.91 similar; agents may pick the wrong one
+      -> differentiate names/descriptions or merge them
+  i MV050 protocol 2025-03-26 predates the 2026-07-28 stateless spec
+```
+
+## What it checks
+
+Most linters stop at basic conformance. `mcpvitals` runs those, plus four things no other tool does:
+
+| Area | What it catches |
+|---|---|
+| **Conformance** | empty tool descriptions, duplicate tool names, malformed schemas |
+| **Reliability** | parameters with no `required` list, untyped params, error-path gaps |
+| **Security hygiene** | secrets in tool metadata, over-broad tools (`run_sql`, `exec`), model-directed instructions in descriptions (the tool-poisoning surface) |
+| **Token cost** | how many tokens each tool burns in the context window, and the total cost to connect the server |
+| **Tool confusion** | pairs of tools an agent will mix up, predicted from name/description overlap *before* it happens in production |
+| **Migration readiness** | what breaks under the 2026-07-28 stateless spec |
+
+Every check is a pure function scored deterministically. Run `mcpvitals check --json` for machine output.
+
+## Add the health badge to your README
+
+`mcpvitals` emits a shareable badge so a good score is visible at a glance:
+
+```bash
+mcpvitals check "npx your-server" --badge badge.svg
+```
+
+Or use the shields.io endpoint output to keep it live.
+
+## Use it in CI
+
+Gate your server on every push with the bundled GitHub Action:
+
+```yaml
+- uses: ContextJet-ai/mcpvitals@v0
+  with:
+    target: "python -m your_server"
+    strict: "true"   # fail the build on any error-level finding
+```
+
+## Track and monitor (not just a one-shot check)
+
+Pin a server's tools and get alerted when they change. This catches **rug-pulls**, where a server you already approved silently mutates a tool's schema or description after the fact:
+
+```bash
+mcpvitals watch "npx your-server"           # pins tools to mcp.lock
+mcpvitals watch "npx your-server" --check    # re-run in CI; fails if a pinned tool changed
+```
+
+Probe a running server for live health (uptime and latency):
+
+```bash
+mcpvitals monitor "https://your-server/mcp" --count 5
+# probe 1/5: up 210ms ... uptime 100.0%  avg 207ms
+```
+
+## Why this exists
+
+The MCP ecosystem grew faster than the tooling around it. There are thousands of servers and no standard way to tell whether one is any good, safe, or cheap to run. `mcpvitals` gives authors a one-command checkup before they publish, and gives everyone else a way to vet a server before they trust it.
+
+Built by [ContextJet.ai](https://www.contextjetai.com). MIT licensed. Contributions welcome, see [CONTRIBUTING.md](CONTRIBUTING.md).

mcpvitals-0.1.0/action.yml

@@ -0,0 +1,15 @@
+name: "mcpvitals"
+description: "Run an MCP server health check in CI"
+inputs:
+  target:
+    description: "stdio command or http url of the MCP server"
+    required: true
+  strict:
+    description: "fail the build on any error-level finding"
+    default: "false"
+runs:
+  using: "composite"
+  steps:
+    - shell: bash
+      run: |
+        pipx run mcpvitals check "${{ inputs.target }}" ${{ inputs.strict == 'true' && '--strict' || '' }}

mcpvitals-0.1.0/docs/DESIGN.md

@@ -0,0 +1,108 @@
+# mcpvitals - design spec
+
+_Status: approved 2026-07-05. Vital signs for your MCP servers._
+
+## Problem
+
+MCP servers exploded (7,000+ public, 2,000+ indexed), but the tooling to tell whether one is any good lagged far behind. Independent audits found ~52% abandoned and only ~17% production-ready; 56.7% of agent failures trace to tool misuse (wrong tool, bad args); connecting a few servers can burn 30%+ of the context window before the first message. The registries have "no health-check light," so a broken server looks identical to a solid one at discovery time. The 2026-07-28 spec RC adds six breaking changes (MCP goes stateless), so every production server must migrate this quarter.
+
+Several people independently started "lint / diagnose an MCP server" tools (max ~15 stars, all abandoned, all doing only basic checks). Demand is validated; nobody has won; the basic linter alone is undifferentiated.
+
+## What mcpvitals is
+
+A single, language-agnostic CLI that gives any MCP server a **health checkup with a score**, plus differentiated analysis nobody else does. It speaks MCP over JSON-RPC, so it inspects TypeScript and Python servers alike. Deterministic and fully unit-testable offline.
+
+Primary command:
+
+```
+uvx mcpvitals check <server>     # stdio command or http url
+```
+
+## Goals
+
+- Give a server author a one-command, actionable health report before they publish.
+- Produce a **standardized health score + shareable README badge** (the missing ecosystem quality signal, and the viral distribution loop).
+- Surface things no other tool does: token cost, tool-confusion risk, migration readiness.
+- Be deterministic and testable: the analysis engine is pure functions over introspected schema, tested against golden fixtures with zero network.
+- Ship a GitHub Action so authors gate it in CI and auto-refresh the badge.
+
+## Non-goals (v1)
+
+- Not a gateway/proxy, not a runtime security enforcer, not a registry.
+- Not a threat scanner for tool-poisoning/injection (that lane is closing: Cisco, Snyk, Invariant). We do *author-facing hygiene*, not *defender-facing threat detection*.
+- Not a hosted service or dashboard.
+
+## Architecture
+
+Three layers, clean boundaries:
+
+1. **Introspection layer** (`introspect/`) - connects to a server (stdio or streamable-http) via the MCP SDK, performs the handshake, and captures a plain-data snapshot: server info, protocol version, and the full list of tools/prompts/resources with their JSON schemas. Output is a pure dataclass/dict (`ServerSnapshot`). This is the only layer that touches the network.
+2. **Checks engine** (`checks/`) - a registry of independent **pure-function checks**. Each check takes a `ServerSnapshot` (plus config) and returns a list of `Finding(level, code, message, hint)`. No I/O. This is where all the logic lives and where all the tests point. Adding a check = adding one pure function + its golden fixtures.
+3. **Reporting layer** (`report/`) - aggregates findings into a **score**, renders human output (checklist + score), machine output (`--json`), and the **badge** (SVG + shields.io-compatible endpoint JSON). Deterministic given the findings.
+
+The live commands (`monitor`, `watch`) are a thin shell that repeatedly calls the introspection layer and diffs/records snapshots; the diff and scoring logic they use are pure functions in the checks/report layers, so they stay testable.
+
+### Data shapes
+
+```python
+Finding(level: "error"|"warn"|"info", code: str, message: str, hint: str | None,
+        tool: str | None, weight: int)
+ServerSnapshot(name, version, protocol_version, tools: list[ToolSpec], prompts, resources, raw)
+Report(score: int, grade: "A".."F", findings: list[Finding], token_cost, confusions, migration)
+```
+
+## The checks (v1)
+
+Core hygiene (deterministic rules over the snapshot):
+- **Conformance**: JSON-RPC error semantics, `isError` handling, valid `resources/list` URIs + MIME types, required tool fields, schema validity.
+- **Reliability smells**: numeric params typed as int but examples/coercion imply float; missing `required` on obviously-required params; empty/placeholder descriptions; no error-path signaling.
+- **Security hygiene** (author-facing, keeps the ContextJet flavor): secrets-shaped strings in tool descriptions/defaults; over-broad generic tools (`execute_sql`, `run_command`, `read_file` with unconstrained path); tool descriptions containing imperative "instructions to the model" (poisoning surface); unauthenticated http endpoint.
+
+The differentiated value-adds (the moat):
+- **Token-cost X-ray**: compute the token footprint of each tool's name+description+schema (tiktoken-style, with a stdlib fallback estimator), total "cost to connect," and flag the biggest offenders with a suggested trim target.
+- **Tool-confusion predictor**: pairwise similarity of tool names+descriptions (token-set / embedding-free cosine over n-grams); flag pairs above a threshold as likely mis-selection ("`get_user` vs `fetch_user`: 0.94"). Deterministic, no model call.
+- **Migration readiness (2026-07-28)**: detect reliance on soon-removed features (session state assumptions, deprecated Roots/Sampling/Logging usage, missing forthcoming headers) and report what breaks.
+
+## Scoring + badge
+
+- Each finding carries a weight by level. Score = 100 minus weighted deductions, clamped to 0-100, mapped to a letter grade. Deterministic and documented so it's not a black box.
+- `--badge` writes `badge.svg` and a shields.io endpoint JSON (`{schemaVersion, label:"mcp health", message:"92 A", color}`) so authors can embed either a static SVG or a live endpoint badge.
+- The scoring weights live in one documented table so authors can see exactly why they lost points.
+
+## CLI surface
+
+```
+mcpvitals check <server> [--json] [--badge out.svg] [--strict] [--config mcpvitals.toml]
+mcpvitals migrate-check <server>          # focused 2026-07-28 readiness report
+mcpvitals watch <server>                  # v1.1: pin schemas to mcp.lock, diff on change (rug-pull/drift)
+mcpvitals monitor <server> [--otel]       # v1.1: periodic health probe, latency/error/uptime, OTel metrics
+```
+
+Server arg accepts a stdio command (`"npx some-server"`) or an http(s) URL.
+
+## Testing strategy (the whole point)
+
+- **Golden fixtures**: a `tests/fixtures/` set of captured `ServerSnapshot` JSONs (a healthy server, a broken one, a token-bloated one, a confusable-tools one, a pre-2026-07-28 one). Every check has unit tests asserting exact findings against these. Zero network in the test suite.
+- **Scoring tests**: fixed findings -> exact score/grade/badge output.
+- **A tiny reference MCP server** (in `tests/servers/`) so a handful of end-to-end tests exercise the real introspection path in CI, but the bulk of coverage is the pure-function checks.
+- CI (GitHub Actions) runs the suite on every push -> green tests badge.
+
+## Distribution / trending
+
+- `uvx mcpvitals check <server>` - zero-install one-liner.
+- The **health badge** is the viral engine: every author who scores well embeds it, spreading the tool through READMEs (shields/codecov pattern).
+- A **`mcpvitals` GitHub Action** for CI gating + badge refresh.
+- Launch hook: "the 2026-07-28 spec breaks half the servers, run `mcpvitals` to see if yours survives" - rides the live migration wave. Plus the named-enemy hook: "52% of MCP servers are dead, here's a checkup."
+- README-as-landing-page, searchable name/topics, one-screenshot demo (the scored checklist).
+
+## Roadmap after v1
+
+- `watch` / `mcp.lock` (dependency drift + rug-pull tracking).
+- `monitor` (live latency/error/uptime, OpenTelemetry export).
+- Optional live tool-selection eval (the harness from candidate B) as a `mcpvitals eval` subcommand once the static core has traction.
+
+## Open decisions (resolve during planning)
+
+- Package/impl language: Python (fits `uvx`, our testing muscle, tiktoken). Confirmed for v1.
+- MCP client library: official `mcp` Python SDK for introspection.
+- Token counter: `tiktoken` if available, deterministic stdlib fallback otherwise (so tests never depend on a model).