@greynewell/mcpbr 0.3.21 → 0.3.23

package/README.md

@@ -402,6 +402,45 @@ mcp_server:
     SUPERMODEL_API_KEY: "${SUPERMODEL_API_KEY}"
 ```
 
+### MCP Timeout Configuration
+
+mcpbr supports configurable timeouts for MCP server operations to handle different server types and workloads.
+
+#### Configuration Fields
+
+| Field | Description | Default |
+|-------|-------------|---------|
+| `startup_timeout_ms` | Timeout in milliseconds for MCP server startup | 60000 (60s) |
+| `tool_timeout_ms` | Timeout in milliseconds for MCP tool execution | 900000 (15 min) |
+
+These fields map to the `MCP_TIMEOUT` and `MCP_TOOL_TIMEOUT` environment variables used by Claude Code. See the [Claude Code settings documentation](https://code.claude.com/docs/en/settings.md) for more details.
+
+#### Example Configuration
+
+```yaml
+mcp_server:
+  command: "npx"
+  args: ["-y", "@modelcontextprotocol/server-filesystem", "{workdir}"]
+  startup_timeout_ms: 60000      # 60 seconds for server to start
+  tool_timeout_ms: 900000        # 15 minutes for long-running tools
+```
+
+#### Common Timeout Values
+
+Different server types require different timeout settings based on their operational characteristics:
+
+| Server Type | startup_timeout_ms | tool_timeout_ms | Notes |
+|-------------|-------------------|-----------------|-------|
+| Fast (filesystem, git) | 10000 (10s) | 30000 (30s) | Local operations with minimal overhead |
+| Medium (web search, APIs) | 30000 (30s) | 120000 (2m) | Network I/O with moderate latency |
+| Slow (code analysis, databases) | 60000 (60s) | 900000 (15m) | Complex processing or large datasets |
+
+**When to adjust timeouts:**
+
+- **Increase `startup_timeout_ms`** if your server takes longer to initialize (e.g., loading large models, establishing database connections)
+- **Increase `tool_timeout_ms`** if your tools perform long-running operations (e.g., codebase analysis, file processing, AI inference)
+- **Decrease timeouts** for fast servers to fail quickly on connection issues
+
 ### Custom Agent Prompt
 
 You can customize the prompt sent to the agent using the `agent_prompt` field:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@greynewell/mcpbr",
-  "version": "0.3.21",
+  "version": "0.3.23",
   "description": "Model Context Protocol Benchmark Runner - CLI tool for evaluating MCP servers",
   "keywords": [
     "mcpbr",
@@ -25,13 +25,12 @@
   },
   "license": "MIT",
   "author": "mcpbr Contributors",
-  "readme": "NPM.md",
   "bin": {
     "mcpbr": "./bin/mcpbr.js"
   },
   "files": [
     "bin/",
-    "NPM.md"
+    "README.md"
   ],
   "scripts": {
     "test": "node bin/mcpbr.js --version",

package/NPM.md

@@ -1,176 +0,0 @@
-# mcpbr-cli
-
-> npm wrapper for the mcpbr CLI tool
-
-[![npm version](https://badge.fury.io/js/mcpbr-cli.svg)](https://www.npmjs.com/package/mcpbr-cli)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-
-This npm package provides convenient access to the [mcpbr](https://github.com/greynewell/mcpbr) command-line tool for Node.js and JavaScript developers.
-
-> **Note**: This package is also available as `@greynewell/mcpbr` for those who prefer scoped package names. Both packages provide the `mcpbr` command after installation.
-
-## What is mcpbr?
-
-**Model Context Protocol Benchmark Runner** - Benchmark your MCP server against real GitHub issues. Get hard numbers comparing tool-assisted vs. baseline agent performance.
-
-## Installation
-
-### Prerequisites
-
-- **Python 3.11+** (the mcpbr CLI is implemented in Python)
-- **mcpbr Python package** (`pip install mcpbr`)
-
-### Install via npm
-
-```bash
-# Run directly with npx (no installation)
-npx mcpbr-cli run -c config.yaml
-
-# Or install globally
-npm install -g mcpbr-cli
-
-# Then use anywhere
-mcpbr run -c config.yaml
-```
-
-### Install Python dependencies
-
-The npm package is a wrapper that requires the Python implementation:
-
-```bash
-# Install mcpbr Python package
-pip install mcpbr
-
-# Verify installation
-mcpbr --version
-```
-
-## Usage
-
-All `mcpbr` commands are available through the npm wrapper:
-
-```bash
-# Initialize configuration
-npx mcpbr-cli init
-
-# Run benchmark evaluation
-npx mcpbr-cli run -c config.yaml
-
-# List available models
-npx mcpbr-cli models
-
-# List available benchmarks
-npx mcpbr-cli benchmarks
-
-# Run with verbose output
-npx mcpbr-cli run -c config.yaml -v
-
-# Run specific tasks
-npx mcpbr-cli run -c config.yaml -t astropy__astropy-12907
-```
-
-## Quick Start
-
-```bash
-# 1. Install Python dependencies
-pip install mcpbr
-
-# 2. Set API key
-export ANTHROPIC_API_KEY="your-api-key"
-
-# 3. Run benchmark
-npx mcpbr-cli init
-npx mcpbr-cli run -c mcpbr.yaml -n 5 -v
-```
-
-## Why use the npm package?
-
-- **Familiar workflow** for Node.js/JavaScript developers
-- **No need to remember** pip vs npm for different tools
-- **Consistent versioning** with other npm packages
-- **Easy integration** into Node.js projects and scripts
-- **npx support** for one-off commands without installation
-
-## Documentation
-
-For full documentation, visit: <https://greynewell.github.io/mcpbr/>
-
-- [Installation Guide](https://greynewell.github.io/mcpbr/installation/)
-- [Configuration Reference](https://greynewell.github.io/mcpbr/configuration/)
-- [CLI Reference](https://greynewell.github.io/mcpbr/cli/)
-- [Benchmarks Guide](https://greynewell.github.io/mcpbr/benchmarks/)
-
-## Troubleshooting
-
-### Error: Python 3.11 or later required
-
-Install Python 3.11+:
-
-```bash
-# macOS
-brew install python@3.11
-
-# Ubuntu/Debian
-sudo apt install python3.11
-
-# Windows
-# Download from https://www.python.org/downloads/
-```
-
-### Error: mcpbr Python package not found
-
-Install the Python package:
-
-```bash
-pip install mcpbr
-# or
-pip3 install mcpbr
-```
-
-### Error: command not found: mcpbr
-
-If you installed globally, ensure npm global bin directory is in your PATH:
-
-```bash
-npm config get prefix
-# Add <prefix>/bin to your PATH
-```
-
-## Development
-
-This package is part of the mcpbr project:
-
-```bash
-# Clone the repository
-git clone https://github.com/greynewell/mcpbr.git
-cd mcpbr/cli
-
-# Install dependencies
-npm install
-
-# Test the wrapper
-npm test
-```
-
-## Related Packages
-
-- [`mcpbr`](https://pypi.org/project/mcpbr/) - Python package (core implementation)
-- [`mcpbr-claude-plugin`](https://www.npmjs.com/package/mcpbr-claude-plugin) - Claude Code plugin (also available as `@greynewell/mcpbr-claude-plugin`)
-
-## License
-
-MIT - see [LICENSE](../LICENSE) for details.
-
-## Contributing
-
-Contributions welcome! See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
-
-## Support
-
-- **Documentation**: <https://greynewell.github.io/mcpbr/>
-- **Issues**: <https://github.com/greynewell/mcpbr/issues>
-- **Discussions**: <https://github.com/greynewell/mcpbr/discussions>
-
----
-
-Built by [Grey Newell](https://greynewell.com)