mcpbr-cli 0.4.0 → 0.4.1

package/README.md

@@ -19,6 +19,7 @@ Benchmark your MCP server against real GitHub issues. One command, hard numbers.
 **Model Context Protocol Benchmark Runner**
 
 [![PyPI version](https://badge.fury.io/py/mcpbr.svg)](https://pypi.org/project/mcpbr/)
+[![npm version](https://badge.fury.io/js/mcpbr-cli.svg)](https://www.npmjs.com/package/mcpbr-cli)
 [![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
 [![CI](https://github.com/greynewell/mcpbr/actions/workflows/ci.yml/badge.svg)](https://github.com/greynewell/mcpbr/actions/workflows/ci.yml)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
@@ -215,6 +216,8 @@ Run `mcpbr models` to see the full list.
 
 ### via npm
 
+[![npm package](https://img.shields.io/npm/v/mcpbr-cli.svg)](https://www.npmjs.com/package/mcpbr-cli)
+
 ```bash
 # Run with npx (no installation)
 npx mcpbr-cli run -c config.yaml
@@ -224,6 +227,8 @@ npm install -g mcpbr-cli
 mcpbr run -c config.yaml
 ```
 
+> **Package**: [`mcpbr-cli`](https://www.npmjs.com/package/mcpbr-cli) on npm
+>
 > **Note**: The npm package requires Python 3.11+ and the mcpbr Python package (`pip install mcpbr`)
 
 ### via pip
@@ -624,6 +629,7 @@ Run SWE-bench evaluation with the configured MCP server.
 | `--smtp-port PORT` | | SMTP server port (default: 587) |
 | `--smtp-user USER` | | SMTP username for authentication |
 | `--smtp-password PASS` | | SMTP password for authentication |
+| `--profile` | | Enable comprehensive performance profiling (tool latency, memory, overhead) |
 | `--help` | `-h` | Show help message |
 
 </details>
@@ -738,6 +744,58 @@ mcpbr cleanup -f
 
 </details>
 
+## Performance Profiling
+
+mcpbr includes comprehensive performance profiling to understand MCP server overhead and identify optimization opportunities.
+
+### Enable Profiling
+
+```bash
+# Via CLI flag
+mcpbr run -c config.yaml --profile
+
+# Or in config.yaml
+enable_profiling: true
+```
+
+### What Gets Measured
+
+- **Tool call latencies** with percentiles (p50, p95, p99)
+- **Memory usage** (peak and average RSS/VMS)
+- **Infrastructure overhead** (Docker and MCP server startup times)
+- **Tool discovery speed** (time to first tool use)
+- **Tool switching overhead** (time between tool calls)
+- **Automated insights** from profiling data
+
+### Example Profiling Output
+
+```json
+{
+  "profiling": {
+    "task_duration_seconds": 140.5,
+    "tool_call_latencies": {
+      "Read": {"count": 15, "avg_seconds": 0.8, "p95_seconds": 1.5},
+      "Bash": {"avg_seconds": 2.3, "p95_seconds": 5.1}
+    },
+    "memory_profile": {"peak_rss_mb": 512.3, "avg_rss_mb": 387.5},
+    "docker_startup_seconds": 2.1,
+    "mcp_server_startup_seconds": 1.8
+  }
+}
+```
+
+### Automated Insights
+
+The profiler automatically identifies performance issues:
+
+```text
+- Bash is the slowest tool (avg: 2.3s, p95: 5.1s)
+- Docker startup adds 2.1s overhead per task
+- Fast tool discovery: first tool use in 8.3s
+```
+
+See [docs/profiling.md](docs/profiling.md) for complete profiling documentation.
+
 ## Example Run
 
 Here's what a typical evaluation looks like:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mcpbr-cli",
-  "version": "0.4.0",
+  "version": "0.4.1",
   "description": "Model Context Protocol Benchmark Runner - CLI tool for evaluating MCP servers",
   "keywords": [
     "mcpbr",