@greynewell/mcpbr-claude-plugin 0.3.18

package/README.md

@@ -0,0 +1,127 @@
+# @greynewell/mcpbr-claude-plugin
+
+> Claude Code plugin for mcpbr - Makes Claude an expert at running MCP benchmarks
+
+[![npm version](https://badge.fury.io/js/%40greynewell%2Fmcpbr-claude-plugin.svg)](https://www.npmjs.com/package/@greynewell/mcpbr-claude-plugin)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+This Claude Code plugin provides specialized skills that make Claude an expert at using [mcpbr](https://github.com/greynewell/mcpbr) for MCP server benchmarking.
+
+## What is mcpbr?
+
+**Model Context Protocol Benchmark Runner** - Benchmark your MCP server against real GitHub issues. Get hard numbers comparing tool-assisted vs. baseline agent performance.
+
+## Installation
+
+### Option 1: Clone Repository (Automatic Detection)
+
+```bash
+git clone https://github.com/greynewell/mcpbr.git
+cd mcpbr
+# Claude Code automatically detects the plugin
+```
+
+### Option 2: Install via npm
+
+```bash
+npm install -g @greynewell/mcpbr-claude-plugin
+```
+
+### Option 3: Claude Code Plugin Manager
+
+```bash
+# In Claude Code, run:
+/plugin install github:greynewell/mcpbr
+```
+
+## What You Get
+
+When using Claude Code in a project with this plugin, Claude automatically gains expertise in:
+
+### 1. run-benchmark (`mcpbr-eval`)
+Expert at running evaluations with proper validation:
+- Verifies Docker is running
+- Checks for API keys
+- Validates configuration files
+- Uses correct CLI flags
+- Provides actionable error messages
+
+### 2. generate-config (`mcpbr-config`)
+Generates valid mcpbr configuration files:
+- Ensures `{workdir}` placeholder is included
+- Validates MCP server commands
+- Provides benchmark-specific templates
+- Applies best practices
+
+### 3. swe-bench-lite (`benchmark-swe-lite`)
+Quick-start command for demonstrations:
+- Pre-configured for 5-task evaluation
+- Sensible defaults for output files
+- Perfect for testing and demos
+
+## Example Interactions
+
+Simply ask Claude in natural language:
+
+- "Run the SWE-bench Lite benchmark"
+- "Generate a config for my MCP server"
+- "Run a quick test with 1 task"
+
+Claude will automatically:
+- Verify prerequisites before starting
+- Generate valid configurations
+- Use correct CLI commands
+- Handle errors gracefully
+
+## How It Works
+
+The plugin includes:
+- **plugin.json** - Claude Code plugin manifest
+- **skills/** - Three specialized SKILL.md files
+
+When Claude Code loads this plugin, it injects the skill instructions into Claude's context, making it an expert at mcpbr without you having to explain anything.
+
+## Development
+
+This package is part of the mcpbr project:
+
+```bash
+# Clone the repository
+git clone https://github.com/greynewell/mcpbr.git
+cd mcpbr
+
+# Test the plugin
+pytest tests/test_claude_plugin.py -v
+```
+
+## Related Packages
+
+- [`mcpbr`](https://pypi.org/project/mcpbr/) - Python package (core implementation)
+- [`@greynewell/mcpbr`](https://www.npmjs.com/package/@greynewell/mcpbr) - npm CLI wrapper
+
+## Documentation
+
+For full documentation, visit: <https://greynewell.github.io/mcpbr/>
+
+- [Claude Code Plugin Guide](https://greynewell.github.io/mcpbr/claude-code-plugin/)
+- [Installation Guide](https://greynewell.github.io/mcpbr/installation/)
+- [Configuration Reference](https://greynewell.github.io/mcpbr/configuration/)
+- [CLI Reference](https://greynewell.github.io/mcpbr/cli/)
+
+## License
+
+MIT - see [LICENSE](../LICENSE) for details.
+
+## Contributing
+
+Contributions welcome! See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
+
+## Support
+
+- **Documentation**: <https://greynewell.github.io/mcpbr/>
+- **Issues**: <https://github.com/greynewell/mcpbr/issues>
+- **Discussions**: <https://github.com/greynewell/mcpbr/discussions>
+
+---
+
+Built by [Grey Newell](https://greynewell.com)

package/package.json

@@ -0,0 +1,38 @@
+{
+  "name": "@greynewell/mcpbr-claude-plugin",
+  "version": "0.3.18",
+  "description": "Claude Code plugin for mcpbr - Expert benchmark runner for MCP servers with specialized skills",
+  "keywords": [
+    "claude-code",
+    "claude-plugin",
+    "mcpbr",
+    "mcp",
+    "benchmark",
+    "model-context-protocol",
+    "swe-bench",
+    "llm",
+    "agents",
+    "evaluation"
+  ],
+  "homepage": "https://github.com/greynewell/mcpbr",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/greynewell/mcpbr.git",
+    "directory": ".claude-plugin"
+  },
+  "bugs": {
+    "url": "https://github.com/greynewell/mcpbr/issues"
+  },
+  "license": "MIT",
+  "author": "mcpbr Contributors",
+  "files": [
+    "plugin.json",
+    "skills/"
+  ],
+  "scripts": {
+    "test": "pytest tests/test_claude_plugin.py -v"
+  },
+  "engines": {
+    "node": ">=18.0.0"
+  }
+}

package/plugin.json

@@ -0,0 +1,6 @@
+{
+  "name": "mcpbr",
+  "version": "0.3.18",
+  "description": "Expert benchmark runner for MCP servers using mcpbr. Handles Docker checks, config generation, and result parsing.",
+  "schema_version": "1.0"
+}

package/skills/README.md

@@ -0,0 +1,183 @@
+# mcpbr Claude Code Skills
+
+This directory contains specialized skills that make Claude Code an expert at using mcpbr.
+
+## What are Skills?
+
+Skills are instruction sets that teach Claude Code how to perform specific tasks correctly. When Claude works in this repository, it automatically detects these skills and gains domain expertise about mcpbr.
+
+## Available Skills
+
+### 1. `mcpbr-eval` - Run Benchmark
+
+**Purpose:** Expert at running evaluations with proper validation
+
+**Key Features:**
+- Validates prerequisites (Docker, API keys, config files)
+- Checks for common mistakes before running
+- Supports all benchmarks (SWE-bench, CyberGym, MCPToolBench++)
+- Provides actionable troubleshooting
+
+**When to use:** Anytime you want to run an evaluation with mcpbr
+
+**Example prompts:**
+- "Run the SWE-bench benchmark with 10 tasks"
+- "Evaluate my MCP server on CyberGym level 2"
+- "Run a quick test with 1 task"
+
+### 2. `mcpbr-config` - Generate Config
+
+**Purpose:** Generates valid mcpbr configuration files
+
+**Key Features:**
+- Ensures critical `{workdir}` placeholder is included
+- Validates MCP server commands
+- Provides templates for common MCP servers
+- Supports all benchmark types
+
+**When to use:** When creating or modifying mcpbr configuration files
+
+**Example prompts:**
+- "Generate a config for my Python MCP server"
+- "Create a config using the filesystem server"
+- "Help me configure my custom MCP server"
+
+### 3. `benchmark-swe-lite` - Quick Start
+
+**Purpose:** Streamlined command for SWE-bench Lite evaluation
+
+**Key Features:**
+- Pre-configured for 5-task evaluation
+- Sensible defaults for quick testing
+- Includes runtime/cost estimates
+- Perfect for demos and testing
+
+**When to use:** For quick validation or demonstrations
+
+**Example prompts:**
+- "Run a quick SWE-bench Lite test"
+- "Show me how mcpbr works"
+- "Do a fast evaluation"
+
+## How Skills Work
+
+When you clone this repository and work with Claude Code:
+
+1. Claude Code detects the `.claude-plugin/plugin.json` manifest
+2. It loads all skills from the `skills/` directory
+3. Each skill provides specialized knowledge about mcpbr commands
+4. Claude automatically follows best practices without being told
+
+## Skill Structure
+
+Each skill is a directory containing a `SKILL.md` file:
+
+```text
+skills/
+├── mcpbr-eval/
+│   └── SKILL.md          # Instructions for running evaluations
+├── mcpbr-config/
+│   └── SKILL.md          # Instructions for config generation
+└── benchmark-swe-lite/
+    └── SKILL.md          # Quick-start instructions
+```
+
+Each `SKILL.md` contains:
+
+1. **Frontmatter** - Metadata (name, description)
+2. **Instructions** - Main skill content
+3. **Examples** - Usage examples
+4. **Constraints** - Critical requirements
+5. **Troubleshooting** - Common issues and solutions
+
+## Benefits
+
+### Without Skills
+```text
+User: "Run the benchmark"
+Claude: *tries `mcpbr run` without config, fails*
+Claude: *forgets to check Docker, fails*
+Claude: *uses wrong flags, fails*
+```
+
+### With Skills
+```text
+User: "Run the benchmark"
+Claude: *checks Docker is running*
+Claude: *verifies config exists*
+Claude: *uses correct flags*
+Claude: *evaluation succeeds*
+```
+
+## Testing
+
+Skills are validated by comprehensive tests in `tests/test_claude_plugin.py`:
+
+- Validates plugin manifest structure
+- Checks skill file format and content
+- Ensures critical keywords are present (Docker, {workdir}, etc.)
+- Verifies documentation mentions all commands
+
+Run skill tests:
+```bash
+uv run pytest tests/test_claude_plugin.py -v
+```
+
+## Adding New Skills
+
+To add a new skill:
+
+1. Create a new directory: `skills/my-skill/`
+2. Create `SKILL.md` with frontmatter:
+   ```markdown
+   ---
+   name: my-skill
+   description: Brief description of what this skill does
+   ---
+
+   # Instructions
+   [Your skill content here]
+   ```
+3. Add tests in `tests/test_claude_plugin.py`
+4. Update this README
+
+## Version Management
+
+The plugin version in `.claude-plugin/plugin.json` is automatically synced with `pyproject.toml`:
+
+```bash
+# Sync versions manually
+make sync-version
+
+# Syncs automatically during
+make build
+pre-commit hooks
+CI/CD checks
+```
+
+## Learn More
+
+- **Plugin Manifest**: `.claude-plugin/plugin.json`
+- **Tests**: `tests/test_claude_plugin.py`
+- **Documentation**: Main README.md, CONTRIBUTING.md
+- **Issue**: [#262](https://github.com/greynewell/mcpbr/issues/262)
+
+## Contributing
+
+When modifying skills:
+
+1. Update the relevant `SKILL.md` file
+2. Run tests: `uv run pytest tests/test_claude_plugin.py`
+3. Run pre-commit: `pre-commit run --all-files`
+4. Submit a PR with your changes
+
+Skills should:
+- Be clear and concise
+- Include examples
+- Emphasize critical requirements
+- Provide troubleshooting guidance
+- Reference actual mcpbr commands
+
+## Questions?
+
+Open an issue or check the main project documentation.

package/skills/benchmark-swe-lite/SKILL.md

@@ -0,0 +1,102 @@
+---
+name: swe-bench-lite
+description: Quick-start command to run SWE-bench Lite evaluation with sensible defaults.
+---
+
+# Instructions
+This skill provides a streamlined way to run the SWE-bench Lite benchmark with pre-configured defaults.
+
+## What This Skill Does
+
+This skill runs a quick SWE-bench Lite evaluation with:
+- 5 sample tasks (configurable)
+- Verbose output for visibility
+- Results saved to `results.json`
+- Report saved to `report.md`
+
+## Prerequisites Check
+
+Before running, verify:
+
+1. **Docker is running:**
+   ```bash
+   docker ps
+   ```
+
+2. **API key is set:**
+   ```bash
+   echo $ANTHROPIC_API_KEY
+   ```
+
+3. **Config file exists:**
+   - Check for `mcpbr.yaml` in the current directory
+   - If missing, run `mcpbr init` to generate it
+
+## Default Command
+
+The default command for SWE-bench Lite:
+
+```bash
+mcpbr run -c mcpbr.yaml --dataset SWE-bench/SWE-bench_Lite -n 5 -v -o results.json -r report.md
+```
+
+## Customization Options
+
+Users can customize the run by modifying:
+
+- **Sample size:** Change `-n 5` to any number (or remove for full dataset)
+- **Config file:** Change `-c mcpbr.yaml` to point to a different config
+- **Verbosity:** Use `-vv` for very verbose output
+- **Output files:** Change `results.json` and `report.md` to different paths
+
+## Example Variations
+
+### Minimal quick test (1 task)
+```bash
+mcpbr run -c mcpbr.yaml -n 1 -v
+```
+
+### Full evaluation (all ~300 tasks)
+```bash
+mcpbr run -c mcpbr.yaml --dataset SWE-bench/SWE-bench_Lite -v -o results.json
+```
+
+### MCP-only (skip baseline)
+```bash
+mcpbr run -c mcpbr.yaml -n 5 -M -v -o results.json
+```
+
+### Specific tasks
+```bash
+mcpbr run -c mcpbr.yaml -t astropy__astropy-12907 -t django__django-11099 -v
+```
+
+## Expected Runtime & Cost
+
+For 5 tasks with default settings:
+- **Runtime:** 15-30 minutes (depends on task complexity)
+- **Cost:** $2-5 (depends on task complexity and model used)
+
+## What to Do If It Fails
+
+1. **Docker not running:** Start Docker Desktop
+2. **API key missing:** Set with `export ANTHROPIC_API_KEY="sk-ant-..."`
+3. **Config missing:** Run `mcpbr init` to generate default config
+4. **Config invalid:** Check that `{workdir}` placeholder is in the `args` array
+5. **MCP server fails:** Test the server command independently
+
+## After the Run
+
+Once complete, you'll have:
+- **results.json:** Full evaluation data with metrics, token usage, and per-task results
+- **report.md:** Human-readable summary with resolution rates and comparisons
+- **Console output:** Real-time progress and summary table
+
+Review the results to see how your MCP server performed compared to the baseline!
+
+## Pro Tips
+
+- Start with `-n 1` to verify everything works before running larger evaluations
+- Use `--log-dir logs/` to save detailed per-task logs for debugging
+- Compare multiple runs by changing the MCP server config between runs
+- Use `--baseline-results baseline.json` to detect regressions between versions

package/skills/mcpbr-config/SKILL.md

@@ -0,0 +1,204 @@
+---
+name: generate-config
+description: Generate and validate mcpbr configuration files for MCP server benchmarking.
+---
+
+# Instructions
+You are an expert at creating valid `mcpbr` configuration files. Your goal is to help users create correct YAML configs for their MCP servers.
+
+## Critical Requirements
+
+1. **Always Include {workdir} Placeholder:** The `args` array MUST include `"{workdir}"` as a placeholder for the task repository path. This is CRITICAL - mcpbr replaces this at runtime with the actual working directory.
+
+2. **Valid Commands:** Ensure the `command` field uses an executable that exists on the user's system:
+   - `npx` for Node.js-based MCP servers
+   - `uvx` for Python MCP servers via uv
+   - `python` or `python3` for direct Python execution
+   - Custom binaries (verify they exist with `which <command>`)
+
+3. **Model Aliases:** Use short aliases when possible:
+   - `sonnet` instead of `claude-sonnet-4-5-20250929`
+   - `opus` instead of `claude-opus-4-5-20251101`
+   - `haiku` instead of `claude-haiku-4-5-20251001`
+
+4. **Required Fields:** Every config MUST have:
+   - `mcp_server.command`
+   - `mcp_server.args` (with `"{workdir}"`)
+   - `provider` (usually `"anthropic"`)
+   - `agent_harness` (usually `"claude-code"`)
+   - `model`
+   - `dataset` (or rely on benchmark default)
+
+## Common MCP Server Configurations
+
+### Anthropic Filesystem Server
+```yaml
+mcp_server:
+  name: "filesystem"
+  command: "npx"
+  args:
+    - "-y"
+    - "@modelcontextprotocol/server-filesystem"
+    - "{workdir}"
+  env: {}
+```
+
+### Custom Python MCP Server
+```yaml
+mcp_server:
+  name: "my-server"
+  command: "uvx"
+  args:
+    - "my-mcp-server"
+    - "--workspace"
+    - "{workdir}"
+  env:
+    LOG_LEVEL: "debug"
+```
+
+### Supermodel Codebase Analysis
+```yaml
+mcp_server:
+  name: "supermodel"
+  command: "npx"
+  args:
+    - "-y"
+    - "@supermodeltools/mcp-server"
+  env:
+    SUPERMODEL_API_KEY: "${SUPERMODEL_API_KEY}"
+```
+
+## Configuration Template
+
+When generating a new config, use this template:
+
+```yaml
+mcp_server:
+  name: "<server-name>"
+  command: "<executable>"
+  args:
+    - "<arg1>"
+    - "<arg2>"
+    - "{workdir}"  # CRITICAL: Include this placeholder
+  env: {}
+
+provider: "anthropic"
+agent_harness: "claude-code"
+
+model: "sonnet"  # or "opus", "haiku"
+dataset: "SWE-bench/SWE-bench_Lite"  # or null to use benchmark default
+sample_size: 5
+timeout_seconds: 300
+max_concurrent: 4
+max_iterations: 30
+```
+
+## Validation Steps
+
+Before saving a config, validate:
+
+1. **Workdir Placeholder:** Ensure `"{workdir}"` appears in `args` array.
+2. **Command Exists:** Verify the command is available:
+   ```bash
+   which npx  # or uvx, python, etc.
+   ```
+3. **Syntax:** YAML syntax is correct (no tabs, proper indentation).
+4. **Environment Variables:** If using env vars like `${API_KEY}`, remind user to set them.
+
+## Benchmark-Specific Configurations
+
+### SWE-bench (Default)
+```yaml
+# ... mcp_server config ...
+provider: "anthropic"
+agent_harness: "claude-code"
+model: "sonnet"
+dataset: "SWE-bench/SWE-bench_Lite"  # or SWE-bench/SWE-bench_Verified
+sample_size: 10
+```
+
+### CyberGym
+```yaml
+# ... mcp_server config ...
+provider: "anthropic"
+agent_harness: "claude-code"
+model: "sonnet"
+benchmark: "cybergym"
+dataset: "sunblaze-ucb/cybergym"
+cybergym_level: 2  # 0-3
+sample_size: 10
+```
+
+### MCPToolBench++
+```yaml
+# ... mcp_server config ...
+provider: "anthropic"
+agent_harness: "claude-code"
+model: "sonnet"
+benchmark: "mcptoolbench"
+dataset: "MCPToolBench/MCPToolBenchPP"
+sample_size: 10
+```
+
+## Custom Agent Prompts
+
+Users can customize the agent prompt using the `agent_prompt` field:
+
+```yaml
+agent_prompt: |
+  Fix the following bug in this repository:
+
+  {problem_statement}
+
+  Make the minimal changes necessary to fix the issue.
+  Focus on the root cause, not symptoms.
+```
+
+**Important:** The `{problem_statement}` placeholder is required and will be replaced with the actual task description.
+
+## Common Mistakes to Avoid
+
+1. **Missing {workdir}:** Forgetting to include `"{workdir}"` in args.
+2. **Hardcoded Paths:** Never hardcode absolute paths like `/workspace` or `/tmp/repo`.
+3. **Invalid Commands:** Using commands that don't exist (e.g., `uv` instead of `uvx`).
+4. **Wrong Indentation:** YAML is whitespace-sensitive. Use 2 spaces, not tabs.
+5. **Missing Quotes:** Environment variable references like `"${VAR}"` need quotes.
+
+## Example Workflow
+
+When a user asks to create a config:
+
+1. Ask about their MCP server:
+   - What package/command runs the server?
+   - Does it need any special arguments or environment variables?
+   - Is it Node.js-based (npx) or Python-based (uvx)?
+
+2. Generate the config based on their answers.
+
+3. Validate the config:
+   - Check for `{workdir}` placeholder
+   - Verify command exists
+   - Confirm YAML syntax
+
+4. Save the config (usually to `mcpbr.yaml`).
+
+5. Optionally test the config with a small sample:
+   ```bash
+   mcpbr run -c mcpbr.yaml -n 1 -v
+   ```
+
+## Helpful Commands
+
+```bash
+# Generate a default config
+mcpbr init
+
+# List available models
+mcpbr models
+
+# List available benchmarks
+mcpbr benchmarks
+
+# Validate config by doing a dry run with 1 task
+mcpbr run -c config.yaml -n 1 -v
+```

package/skills/mcpbr-eval/SKILL.md

@@ -0,0 +1,123 @@
+---
+name: run-benchmark
+description: Run an MCP evaluation using mcpbr on SWE-bench or other datasets.
+---
+
+# Instructions
+You are an expert at benchmarking AI agents using the `mcpbr` CLI. Your goal is to run valid, reproducible evaluations.
+
+## Critical Constraints (DO NOT IGNORE)
+
+1. **Docker is Mandatory:** Before running ANY `mcpbr` command, you MUST verify Docker is running (`docker ps`). If not, tell the user to start it.
+
+2. **Config is Required:** `mcpbr run` FAILS without a config file. Never guess flags.
+   - IF no config exists: Run `mcpbr init` first to generate a template.
+   - IF config exists: Read it (`cat mcpbr.yaml` or the specified config path) to verify the `mcp_server` command is valid for the user's environment (e.g., check if `npx` or `uvx` is installed).
+
+3. **Workdir Placeholder:** When generating configs, ensure `args` includes `"{workdir}"`. Do not resolve this path yourself; `mcpbr` handles it.
+
+4. **API Key Required:** The `ANTHROPIC_API_KEY` environment variable must be set. Check for it before running evaluations.
+
+## Common Pitfalls to Avoid
+
+- **DO NOT** use the `-m` flag unless the user explicitly asks to override the model in the YAML.
+- **DO NOT** hallucinate dataset names. Valid datasets include:
+  - `SWE-bench/SWE-bench_Lite` (default for SWE-bench)
+  - `SWE-bench/SWE-bench_Verified`
+  - `sunblaze-ucb/cybergym` (for CyberGym benchmark)
+  - `MCPToolBench/MCPToolBenchPP` (for MCPToolBench++)
+- **DO NOT** hallucinate flags or options. Only use documented CLI flags.
+- **DO NOT** forget to specify the config file with `-c` or `--config`.
+
+## Supported Benchmarks
+
+mcpbr supports three benchmarks:
+1. **SWE-bench** (default): Real GitHub issues requiring bug fixes
+   - Dataset: `SWE-bench/SWE-bench_Lite` or `SWE-bench/SWE-bench_Verified`
+   - Use: `mcpbr run -c config.yaml` or `--benchmark swe-bench`
+
+2. **CyberGym**: Security vulnerabilities requiring PoC exploits
+   - Dataset: `sunblaze-ucb/cybergym`
+   - Use: `mcpbr run -c config.yaml --benchmark cybergym --level [0-3]`
+
+3. **MCPToolBench++**: Large-scale tool use evaluation
+   - Dataset: `MCPToolBench/MCPToolBenchPP`
+   - Use: `mcpbr run -c config.yaml --benchmark mcptoolbench`
+
+## Execution Steps
+
+Follow these steps in order:
+
+1. **Verify Prerequisites:**
+   ```bash
+   # Check Docker is running
+   docker ps
+
+   # Verify API key is set
+   echo $ANTHROPIC_API_KEY
+   ```
+
+2. **Check for Config File:**
+   - If `mcpbr.yaml` (or user-specified config) does NOT exist: Run `mcpbr init` to generate it.
+   - If config exists: Read it to understand the configuration.
+
+3. **Validate Config:**
+   - Ensure `mcp_server.command` is valid (e.g., `npx`, `uvx`, `python` are installed).
+   - Ensure `mcp_server.args` includes `"{workdir}"` placeholder.
+   - Verify `model`, `dataset`, and other parameters are correctly set.
+
+4. **Construct the Command:**
+   - Base command: `mcpbr run --config <path-to-config>`
+   - Add flags as needed based on user request:
+     - `-n <number>` or `--sample <number>`: Override sample size
+     - `-v` or `-vv`: Verbose output
+     - `-o <path>`: Save JSON results
+     - `-r <path>`: Save Markdown report
+     - `--log-dir <path>`: Save per-instance logs
+     - `-M`: MCP-only evaluation (skip baseline)
+     - `-B`: Baseline-only evaluation (skip MCP)
+     - `--benchmark <name>`: Override benchmark
+     - `--level <0-3>`: Set CyberGym difficulty level
+
+5. **Run the Command:**
+   Execute the constructed command and monitor the output.
+
+6. **Handle Results:**
+   - If the run completes successfully, inform the user about the results.
+   - If errors occur, diagnose and provide actionable feedback.
+
+## Example Commands
+
+```bash
+# Full evaluation with 5 tasks
+mcpbr run -c config.yaml -n 5 -v
+
+# MCP-only evaluation
+mcpbr run -c config.yaml -M -n 10
+
+# Save results and report
+mcpbr run -c config.yaml -o results.json -r report.md
+
+# Run CyberGym at level 2
+mcpbr run -c config.yaml --benchmark cybergym --level 2 -n 5
+
+# Run specific tasks
+mcpbr run -c config.yaml -t astropy__astropy-12907 -t django__django-11099
+```
+
+## Troubleshooting
+
+If you encounter errors:
+
+1. **Docker not running:** Remind user to start Docker Desktop or Docker daemon.
+2. **API key missing:** Ask user to set `export ANTHROPIC_API_KEY="sk-ant-..."`
+3. **Config file invalid:** Re-generate with `mcpbr init` or fix the YAML syntax.
+4. **MCP server fails to start:** Test the server command independently.
+5. **Timeout issues:** Suggest increasing `timeout_seconds` in config.
+
+## Important Reminders
+
+- Always read the config file before making assumptions about what's configured.
+- Never modify the config file without explicit user permission.
+- Use the `mcpbr models` command to check available models if needed.
+- Use the `mcpbr benchmarks` command to list available benchmarks.