mcpbr-cli 0.3.26 → 0.3.28

package/README.md

@@ -296,6 +296,9 @@ dataset: "SWE-bench/SWE-bench_Lite"
 sample_size: 10
 timeout_seconds: 300
 max_concurrent: 4
+
+# Optional: disable default logging (logs are saved to output_dir/logs/ by default)
+# disable_logs: true
 ```
 
 4. **Run the evaluation:**
@@ -519,7 +522,8 @@ Run SWE-bench evaluation with the configured MCP server.
 | `--output-junit PATH` | | Path to save JUnit XML report (for CI/CD integration) |
 | `--verbose` | `-v` | Verbose output (`-v` summary, `-vv` detailed) |
 | `--log-file PATH` | `-l` | Path to write raw JSON log output (single file) |
-| `--log-dir PATH` | | Directory to write per-instance JSON log files |
+| `--log-dir PATH` | | Directory to write per-instance JSON log files (default: `output_dir/logs/`) |
+| `--disable-logs` | | Disable detailed execution logs (overrides default and config) |
 | `--task TEXT` | `-t` | Run specific task(s) by instance_id (repeatable) |
 | `--prompt TEXT` | | Override agent prompt (use `{problem_statement}` placeholder) |
 | `--baseline-results PATH` | | Path to baseline results JSON for regression detection |
@@ -773,6 +777,38 @@ Results saved to results.json
 }
 ```
 
+### Output Directory Structure
+
+By default, mcpbr consolidates all outputs into a single timestamped directory:
+
+```text
+.mcpbr_run_20260126_133000/
+├── config.yaml                # Copy of configuration used
+├── evaluation_state.json      # Task results and state
+├── logs/                      # Detailed MCP server logs
+│   ├── task_1_mcp.log
+│   ├── task_2_mcp.log
+│   └── ...
+└── README.txt                 # Auto-generated explanation
+```
+
+This makes it easy to:
+- **Archive results**: `tar -czf results.tar.gz .mcpbr_run_*`
+- **Clean up**: `rm -rf .mcpbr_run_*`
+- **Share**: Just zip one directory
+
+You can customize the output directory:
+
+```bash
+# Custom output directory
+mcpbr run -c config.yaml --output-dir ./my-results
+
+# Or in config.yaml
+output_dir: "./my-results"
+```
+
+**Note:** The `--output-dir` CLI flag takes precedence over the `output_dir` config setting. This ensures that the README.txt file in the output directory reflects the final effective configuration values after all CLI overrides are applied.
+
 ### Markdown Report (`--report`)
 
 Generates a human-readable report with:
@@ -782,6 +818,17 @@ Generates a human-readable report with:
 
 ### Per-Instance Logs (`--log-dir`)
 
+**Logging is enabled by default** to prevent data loss. Detailed execution traces are automatically saved to `output_dir/logs/` unless disabled.
+
+To disable logging:
+```bash
+# Via CLI flag
+mcpbr run -c config.yaml --disable-logs
+
+# Or in config file
+disable_logs: true
+```
+
 Creates a directory with detailed JSON log files for each task run. Filenames include timestamps to prevent overwrites:
 
 ```text

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mcpbr-cli",
-  "version": "0.3.26",
+  "version": "0.3.28",
   "description": "Model Context Protocol Benchmark Runner - CLI tool for evaluating MCP servers",
   "keywords": [
     "mcpbr",