mcp-halflist 0.2.0__tar.gz

mcp_halflist-0.2.0/.claude/settings.local.json

@@ -0,0 +1,19 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(git -C /Users/sabhishekh/personal-projects/mcp-halflist config user.name \"Abhishekh Singh\")",
+      "Bash(git -C /Users/sabhishekh/personal-projects/mcp-halflist config user.email \"abhishekcp.singh67@gmail.com\")",
+      "Bash(pip install *)",
+      "Bash(python3.11 -c \"import mcp; print\\(mcp.__version__\\)\")",
+      "Bash(pip show *)",
+      "Bash(python3.11 -c ' *)",
+      "Bash(python3.11 *)",
+      "Bash(halflist *)",
+      "Bash(ruff check *)",
+      "Bash(git *)",
+      "Bash(python3 -c \"import mcp\")",
+      "Bash(python *)",
+      "Bash(awk '{print $3}')"
+    ]
+  }
+}

mcp_halflist-0.2.0/.gitignore

@@ -0,0 +1,224 @@
+# Architecture / design docs (not for version control)
+mcp-halflist-architecture.md
+
+# macOS
+.DS_Store
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[codz]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#   Usually these files are written by a python script from a template
+#   before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py.cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+# Pipfile.lock
+
+# UV
+#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+# uv.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+# poetry.lock
+# poetry.toml
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#   pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
+#   https://pdm-project.org/en/latest/usage/project/#working-with-version-control
+# pdm.lock
+# pdm.toml
+.pdm-python
+.pdm-build/
+
+# pixi
+#   Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
+# pixi.lock
+#   Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
+#   in the .venv directory. It is recommended not to include this directory in version control.
+.pixi
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# Redis
+*.rdb
+*.aof
+*.pid
+
+# RabbitMQ
+mnesia/
+rabbitmq/
+rabbitmq-data/
+
+# ActiveMQ
+activemq-data/
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.envrc
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#   JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#   be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#   and can be added to the global gitignore or merged into this file.  For a more nuclear
+#   option (not recommended) you can uncomment the following to ignore the entire idea folder.
+# .idea/
+
+# Abstra
+#   Abstra is an AI-powered process automation framework.
+#   Ignore directories containing user credentials, local state, and settings.
+#   Learn more at https://abstra.io/docs
+.abstra/
+
+# Visual Studio Code
+#   Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore 
+#   that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
+#   and can be added to the global gitignore or merged into this file. However, if you prefer, 
+#   you could uncomment the following to ignore the entire vscode folder
+# .vscode/
+# Temporary file for partial code execution
+tempCodeRunnerFile.py
+
+# Ruff stuff:
+.ruff_cache/
+
+# PyPI configuration file
+.pypirc
+
+# Marimo
+marimo/_static/
+marimo/_lsp/
+__marimo__/
+
+# Streamlit
+.streamlit/secrets.toml

mcp_halflist-0.2.0/CHANGELOG.md

@@ -0,0 +1,38 @@
+# Changelog
+
+## 0.2.0 (2026-04-28)
+
+### Added
+- `halflist audit` — full conformance check + benchmark in one shot
+  - Runs all suites + benchmarks all tools with a single connection
+  - Combined `AuditReport` JSON with both suites and benchmarks
+  - `halflist report` auto-detects audit JSON for markdown generation
+- `halflist bench` — latency benchmarking per tool with p50/p95/p99 percentiles
+  - `--tool` filter (repeatable), `--all`, `--iterations/-n`, `--warmup/-w`
+  - Rich terminal output with live progress and results table
+  - JSON output (`--format json`)
+- `halflist watch` — continuous health monitoring with JSONL output
+  - `--interval/-i`, `--count/-c`, `--log/-l` for file logging
+  - Per-probe status: ok / degraded / down
+- `halflist report` — generate markdown or SVG badge from JSON output
+  - `--badge` for shields.io-style SVG
+  - `--format markdown` (default)
+  - `-o` to write to file
+- Schema-based dummy argument generation reused for bench tool calls
+- Differentiated empty vs absent tool description warnings in tools suite
+
+### Changed
+- `--quiet` / `-q` flag now suppresses server stderr (auto-enabled with `--format json`)
+
+## 0.1.0 (2026-04-28)
+
+Initial release.
+
+- `halflist check` command with stdio transport
+- Handshake suite (6 checks)
+- Tools suite (11 checks)
+- Terminal output with Rich formatting
+- JSON output (`--format json`)
+- Schema-based dummy argument generation for tool smoke tests
+- Suite filtering (`--suite`)
+- CI-friendly exit codes (0 = pass, 1 = fail, 2 = transport error, 3 = config error)

mcp_halflist-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Abhishekh Singh
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mcp_halflist-0.2.0/PKG-INFO

@@ -0,0 +1,287 @@
+Metadata-Version: 2.4
+Name: mcp-halflist
+Version: 0.2.0
+Summary: CI-first conformance testing and benchmarking CLI for MCP servers. Lint your MCP server before your users do.
+Project-URL: Homepage, https://github.com/abhishekhsingh/mcp-halflist
+Project-URL: Repository, https://github.com/abhishekhsingh/mcp-halflist
+Project-URL: Issues, https://github.com/abhishekhsingh/mcp-halflist/issues
+Author-email: Abhishekh Singh <abhishekcp.singh67@gmail.com>
+License: MIT
+License-File: LICENSE
+Keywords: cli,conformance,mcp,model-context-protocol,testing
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Requires-Dist: mcp>=1.0.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: typer>=0.9.0
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: ruff>=0.4; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# mcp-halflist
+
+**Lint your MCP server before your users do.**
+
+CI-first conformance testing CLI for [Model Context Protocol](https://modelcontextprotocol.io) servers. Point it at your server, get a scored health report.
+
+## Installation
+
+```bash
+pip install mcp-halflist
+```
+
+## Quick Start
+
+```bash
+# Full audit of the official MCP reference server
+halflist audit --stdio "npx -y @modelcontextprotocol/server-everything"
+
+# JSON output for CI pipelines
+halflist audit --stdio "npx -y @modelcontextprotocol/server-everything" --format json
+```
+
+### Test your own server
+
+```bash
+halflist audit --stdio "python3 my_server.py"
+halflist check --stdio "python3 my_server.py" --format json
+```
+
+### More examples
+
+```bash
+# Time server
+halflist check --stdio "npx -y @modelcontextprotocol/server-time"
+
+# Fetch server
+halflist check --stdio "npx -y @modelcontextprotocol/server-fetch"
+
+# Filesystem server
+halflist check --stdio "npx -y @modelcontextprotocol/server-filesystem /tmp"
+```
+
+## Commands
+
+### `halflist audit` — Full Audit (Recommended)
+
+Run conformance checks + benchmark every tool in a single shot.
+
+```bash
+# One command, full picture
+halflist audit --stdio "npx -y @modelcontextprotocol/server-everything"
+
+# JSON output for CI
+halflist audit --stdio "python3 my_server.py" --format json
+
+# Control benchmark iterations
+halflist audit --stdio "python3 my_server.py" --iterations 50
+```
+
+### `halflist check` — Conformance Testing
+
+Run protocol conformance checks against an MCP server.
+
+```bash
+halflist check --stdio "npx -y @modelcontextprotocol/server-everything"
+halflist check --stdio "python3 my_server.py" --format json
+halflist check --stdio "python3 my_server.py" --suite handshake --verbose
+```
+
+### `halflist bench` — Latency Benchmarking
+
+Benchmark latency per tool with p50/p95/p99 percentiles.
+
+```bash
+# Benchmark first 5 tools (default)
+halflist bench --stdio "npx -y @modelcontextprotocol/server-everything"
+
+# Benchmark specific tools
+halflist bench --stdio "python3 my_server.py" --tool search --tool get_topic
+
+# Benchmark all tools with 50 iterations
+halflist bench --stdio "python3 my_server.py" --all --iterations 50
+
+# JSON output
+halflist bench --stdio "python3 my_server.py" --format json
+```
+
+### `halflist watch` — Health Monitoring
+
+Continuously monitor an MCP server's health.
+
+```bash
+# Probe every 60 seconds (default)
+halflist watch --stdio "npx -y @modelcontextprotocol/server-time"
+
+# Custom interval, log to file
+halflist watch --stdio "python3 my_server.py" --interval 30 --log health.jsonl
+
+# Run 10 probes and exit
+halflist watch --stdio "python3 my_server.py" --count 10
+```
+
+### `halflist report` — Markdown & Badges
+
+Generate markdown or SVG badge from a halflist JSON report.
+
+```bash
+# Generate markdown from audit results
+halflist audit --stdio "python3 my_server.py" --format json > results.json
+halflist report results.json
+
+# Generate SVG badge
+halflist report results.json --badge -o badge.svg
+
+# Generate markdown from bench results
+halflist bench --stdio "python3 my_server.py" --format json > bench.json
+halflist report bench.json -o BENCH.md
+```
+
+## What It Checks
+
+### Handshake Suite
+- Server responds to `initialize` with valid response
+- Protocol version returned
+- Capabilities object present
+- Server info has name and version
+- `notifications/initialized` accepted
+- Ping returns pong
+
+### Tools Suite
+- `tools/list` returns valid array
+- At least 1 tool exists
+- Every tool has a non-empty name
+- Every tool has a description
+- `inputSchema` contains `"type": "object"`
+- No duplicate tool names
+- Tool names follow recommended pattern
+- Tool call with generated args returns valid response
+- Tool call with empty args handles gracefully
+- Response content is a list
+- Each content item has a type field
+
+## Output Formats
+
+### Terminal (default)
+
+Rich, colored output with a scored summary:
+
+```
+╭─ mcp-halflist v0.2.0 ──────────────────────────────────────╮
+│                                                              │
+│  Server:     my-server v1.0.0                               │
+│  Transport:  stdio                                           │
+│  Score:      94/100  ████████████████████░░  94%             │
+│                                                              │
+╰──────────────────────────────────────────────────────────────╯
+```
+
+### JSON (`--format json`)
+
+Full structured report suitable for CI integration. Returns a `HalflistReport`, `BenchReport`, or `AuditReport` object with score, suites, benchmarks, and timing data.
+
+### Markdown (`halflist report`)
+
+Generate markdown tables from JSON output, suitable for embedding in README or PR comments.
+
+### SVG Badge (`halflist report --badge`)
+
+Shields.io-style SVG badge showing score or benchmark summary.
+
+## Exit Codes
+
+| Code | Meaning |
+|------|---------|
+| 0 | All checks passed (warnings OK) |
+| 1 | One or more failures |
+| 2 | Connection/transport error |
+| 3 | Configuration error (bad flags) |
+
+## CLI Options
+
+```
+halflist audit --stdio <command>   Launch server via stdio (check + bench all tools)
+               -n / --iterations   Benchmark iterations per tool (default: 10)
+               -w / --warmup       Warmup iterations (default: 2)
+               --verbose           Show all check details
+               --format <fmt>      Output format: terminal or json
+               --quiet / -q        Suppress server stderr
+               --timeout <secs>    Timeout per operation (default: 30)
+
+halflist check --stdio <command>   Launch server via stdio
+               --format <fmt>      Output format: terminal (default) or json
+               --suite <name>      Filter to specific suite(s), repeatable
+               --verbose           Show all check details
+               --quiet / -q        Suppress server stderr
+               --timeout <secs>    Timeout per operation (default: 30)
+
+halflist bench --stdio <command>   Launch server via stdio
+               --tool <name>       Benchmark specific tool(s), repeatable
+               --all               Benchmark all tools (default: first 5)
+               -n / --iterations   Iterations per tool (default: 10)
+               -w / --warmup       Warmup iterations (default: 2)
+               --format <fmt>      Output format: terminal or json
+               --quiet / -q        Suppress server stderr
+               --timeout <secs>    Timeout per operation (default: 30)
+
+halflist watch --stdio <command>   Launch server via stdio
+               -i / --interval     Seconds between probes (default: 60)
+               -c / --count        Number of probes (default: infinite)
+               -l / --log <file>   Append JSONL probes to file
+               --quiet / -q        Suppress server stderr
+               --timeout <secs>    Timeout per operation (default: 30)
+
+halflist report <file.json>        Path to halflist JSON report
+               --format <fmt>      Output format: markdown (default)
+               --badge             Generate SVG badge instead
+               -o / --output       Write to file
+
+halflist --version                 Print version and exit
+```
+
+## How It Compares
+
+| | mcp-halflist | MCP Inspector | mcp-probe |
+|---|---|---|---|
+| **Approach** | Automated test suite | Interactive GUI | Request-level testing |
+| **CI-first** | Yes | No | Partial |
+| **Scored reports** | Yes | No | No |
+| **Benchmarking** | Yes | No | No |
+| **Zero config** | Yes | Yes | Yes |
+
+Think of it this way: MCP Inspector and mcp-probe are Postman. mcp-halflist is pytest.
+
+## Development
+
+```bash
+git clone https://github.com/abhishekhsingh/mcp-halflist.git
+cd mcp-halflist
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+
+# Run tests
+pytest
+
+# Run against fixture servers
+halflist check --stdio "python tests/servers/good_server.py"
+halflist check --stdio "python tests/servers/bad_server.py"
+
+# Lint
+ruff check src/ tests/
+```
+
+## License
+
+MIT
+
+## Author
+
+Abhishekh Singh