mini-coder 0.5.11 → 0.5.13

package/BENCHMARK.md

@@ -0,0 +1,107 @@
+# CORE GOAL
+
+**Terminal-Bench is a signal, not the product. The target is a better coding agent, not a higher benchmark score from benchmark-shaped patches.**
+
+- Do not add fixes to fix an issue with specific terminal bench evals, focus on improving the agent's behaviour.
+- Benchmarks run headless mode in one shot. Make sure you use `tmux` often to check that the TUI and multiturn UX is still
+  good.
+
+# Benchmark workflow
+
+Use this as the default tuning loop for `mini-coder` on Terminal-Bench.
+
+The goal is not to rerun the whole benchmark after every change. The goal is to get fast enough feedback that small changes can be judged quickly, then promote only the promising ones to bigger runs.
+
+## Principles
+
+- Keep changes small.
+- Change one thing at a time.
+- Compare against a fresh baseline, not an old leaderboard run.
+- Use fast suites for iteration, broad suites for promotion.
+- Keep structured `mc --json` logs in trial artifacts so behavior can be analyzed.
+- Optimize for general coding-agent behavior first.
+- Use benchmark failures to extract general behavior gaps, not to encode benchmark lore into the agent.
+- Prefer generic improvements over task-named patches, reminders, or stop-time nudges.
+
+## Suites
+
+There is a full 89 test baseline run with 2 attemps in the teminal-bench folder. Use the evals in it
+to determine your fast evals to start your optimization process and iterations.
+
+Settings:
+
+- `2` attempts
+- `2` concurrent
+- `0` retries
+
+## Experiment quality bar
+
+Before changing code, write the hypothesis in two layers:
+
+1. the benchmark symptom
+2. the general coding-agent behavior gap behind it
+
+Only run an experiment if you can answer all of these:
+
+- what general behavior is being improved?
+- why should that help outside Terminal-Bench?
+- what would make this change obviously overfit?
+
+Reject or redesign experiments that:
+
+- depend on benchmark-specific task names, file names, package names, or tool names in product logic
+- inject reminders or guards keyed to one benchmark noun unless that rule maps cleanly to a real product behavior
+- only make sense because a particular verifier is known
+- cannot be explained without citing a single task transcript
+
+Hard rule:
+
+- no task-specific nouns in agent logic unless they map to a real product feature
+
+## Iteration loop
+
+For each change:
+
+1. inspect the last fast / focused failures
+2. translate them into **one** general behavior gap
+3. reject benchmark-shaped ideas; if you cannot phrase the change without task-specific nouns, keep diagnosing
+4. if the change depends on a dynamic trigger, confirm that the trigger actually appears in the target failures
+5. form **one** narrow hypothesis
+6. make **one** small change
+7. run:
+   - fast suite
+8. compare to baseline or your reference run
+9. decide:
+   - keep
+   - revert
+   - refine
+
+## Behavior analysis requirements
+
+Behavior analysis depends on structured agent logs.
+
+Keep wrappers on:
+
+- `mc --json -p ...`
+
+Per trial, keep:
+
+- result JSON
+- verifier output
+- exception type
+- agent stderr
+- structured `agent/mini-coder.ndjson`
+- timestamps
+
+## Minimal experiment log format
+
+You are running in a loop, make sure to keep your progress tracked so you
+can continue between loop iterations, this is to avoid context pressure.:w
+Keep this in `PROGRESS.md`, a final summary for each completed change:
+
+- benchmark symptom
+- general behavior gap
+- why this should help outside Terminal-Bench
+- hypothesis
+- verification method
+- keep / revert / refine (Make the decision very visible in the file).

package/PROGRESS.md

@@ -0,0 +1,4 @@
+# CURRENT TASK PROGRESS
+
+- Progress reset for a fresh benchmark restart.
+- Benchmark for baseline is running (command in `benchmark-baseline.sh`)

package/README.md

@@ -39,15 +39,16 @@ $ mc
 
 ## Tools
 
-Four built-in tools, plus a read-only image tool:
+Six built-in tools, plus a conditional read-only image tool and any configured MCP tools:
 
 - **`shell`** — runs commands in the user's shell. Returns stdout, stderr, and exit code. Large output is truncated to protect model context.
+- **`read`** — reads UTF-8 text files from disk, optionally by line window.
+- **`grep`** — searches file contents with ripgrep-style options and returns structured matches.
 - **`edit`** — exact-text replacement in a single file. Fails deterministically if the target is missing or ambiguous. Creates new files when old text is empty.
 - **`todoWrite`** — creates or updates the session todo list incrementally and returns the full current snapshot.
 - **`todoRead`** — returns the full current session todo list snapshot.
 - **`readImage`** — reads PNG, JPEG, GIF, and WebP files as model input. Only registered when the active model supports images.
-
-Plugins can add more tools, but the core stays intentionally small.
+- **Configured MCP tools** — tools discovered from `settings.json` Streamable HTTP MCP servers. Imported tool names are prefixed with the server name, for example `docs__search`.
 
 ## Features
 
@@ -56,25 +57,27 @@ Plugins can add more tools, but the core stays intentionally small.
 - **Session persistence** — SQLite-backed sessions with undo, fork, resume, and cumulative usage stats. Sessions are scoped to the working directory.
 - **Reasoning and verbosity controls** — toggle thinking visibility and verbose tool rendering on demand. Preferences persist across launches.
 - **[AGENTS.md](https://agents.md) support** — project-specific instructions discovered root-to-leaf, with `~/.agents/` for global instructions.
-- **[Agent Skills](https://agentskills.io)** — skill catalogs exposed in the prompt. `/skill:name` injects a skill body into the next user message.
-- **Plugins** — optional tools, integrations, theme overrides, and prompt suffixes without bloating the core.
+- **[Agent Skills](https://agentskills.io)** — skill catalogs exposed in the prompt. `/skill:name` injects a skill body into the next user message, and `/skill` opens a picker that fills in the selected skill reference without submitting.
+- **Settings-driven MCP tools** — connect Streamable HTTP MCP servers from `~/.config/mini-coder/settings.json` and expose their tools directly in the core runtime.
 
 ## Commands
 
-| Command      | Description                                                                                            |
-| ------------ | ------------------------------------------------------------------------------------------------------ |
-| `/model`     | Switch models and save the choice as the global default.                                               |
-| `/session`   | Open the session picker for the current working directory.                                             |
-| `/new`       | Start a fresh session and reset the running token and cost totals.                                     |
-| `/fork`      | Fork the current chat into a new session, keep the original, and add a UI-only `Forked session.` note. |
-| `/undo`      | Remove the last conversational turn without touching filesystem changes.                               |
-| `/reasoning` | Show or hide model thinking. The setting is saved and restored on launch.                              |
-| `/verbose`   | Expand shell output plus edit previews and edit errors in the conversation log.                        |
-| `/todo`      | Show the current session todo list in the conversation log as a UI-only checklist block.               |
-| `/login`     | Sign in with a supported OAuth provider.                                                               |
-| `/logout`    | Remove saved OAuth credentials for a logged-in provider.                                               |
-| `/effort`    | Choose low, medium, high, or xhigh reasoning effort.                                                   |
-| `/help`      | Show commands, current toggles, loaded AGENTS.md files, skills, and plugins.                           |
+| Command       | Description                                                                                                                                                       |
+| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `/model`      | Switch models and save the choice as the global default.                                                                                                          |
+| `/session`    | Open the session picker for the current working directory.                                                                                                        |
+| `/new`        | Start a fresh session and reset the running token and cost totals.                                                                                                |
+| `/fork`       | Fork the current chat into a new session, keep the original, and add a UI-only `Forked session.` note.                                                            |
+| `/undo`       | Remove the last conversational turn without touching filesystem changes.                                                                                          |
+| `/reasoning`  | Show or hide model thinking. The setting is saved and restored on launch.                                                                                         |
+| `/verbose`    | Toggle compact vs full rendering for verbose-aware tool previews/results, including shell, read, grep, edit previews/errors, and MCP tool blocks.                 |
+| `/mcp`        | Open the MCP server picker and toggle configured servers on or off for future turns.                                                                              |
+| `/todo`       | Show the current session todo list in the conversation log as a UI-only checklist block.                                                                          |
+| `/login`      | Sign in with a supported OAuth provider.                                                                                                                          |
+| `/logout`     | Remove saved OAuth credentials for a logged-in provider.                                                                                                          |
+| `/effort`     | Choose low, medium, high, or xhigh reasoning effort.                                                                                                              |
+| `/help`       | Show commands, current toggles, loaded AGENTS.md files, skills, and MCP servers with on/off state.                                                                |
+| `/skill:name` | Insert a discovered skill into the next message. Submit `/skill` or pick `/skill:name` from slash-command autocomplete to choose a skill without auto-submitting. |
 
 ## Key bindings
 
@@ -83,7 +86,7 @@ Plugins can add more tools, but the core stays intentionally small.
 | `Enter`       | Submit message                                                                                             |
 | `Shift+Enter` | Insert newline                                                                                             |
 | `Escape`      | Dismiss the overlay without changing the draft; otherwise interrupt the running turn; otherwise do nothing |
-| `Tab`         | Autocomplete a path, or open the command picker when the draft starts with `/`                             |
+| `Tab`         | Autocomplete a path, or open the command picker when the draft starts with `/` without clearing the draft  |
 | `Ctrl+R`      | Search global raw input history                                                                            |
 | `Ctrl+C`      | Graceful exit                                                                                              |
 | `Ctrl+D`      | Graceful exit when the input is empty                                                                      |
@@ -103,9 +106,40 @@ $ printf '%s\n' 'fix the failing tests' | mc
 - Starts when `-p/--prompt` is provided or when stdin or stdout is not a TTY.
 - If stdout is redirected but stdin is still interactive, pass `-p`; headless mode will not fall back to an interactive prompt.
 - Uses the same parser as the TUI for plain text, `/skill:name`, and standalone image paths.
+- Without `--json`, keeps stdout script-friendly by writing only the final assistant text there, while lightweight assistant commentary snippets from tool-use turns go to stderr.
 - With `--json`, writes NDJSON events for completed assistant/tool-result messages plus `done` / `error` / `aborted` outcomes; queued `user_message` events may also appear. Streaming deltas are omitted.
 - Headless runs still persist like normal sessions and show up in `/session` history for that working directory.
-- Interactive slash commands such as `/model`, `/session`, and `/help` are not available in headless mode.
+- Interactive slash commands such as `/skill` without a name, `/model`, `/session`, `/mcp`, and `/help` are not available in headless mode.
+
+## Settings
+
+Global defaults live in `~/.config/mini-coder/settings.json`.
+
+```json
+{
+  "customProviders": [
+    {
+      "name": "lm-studio",
+      "baseUrl": "http://127.0.0.1:1234/v1"
+    }
+  ],
+  "mcp": {
+    "servers": [
+      {
+        "name": "docs",
+        "url": "http://127.0.0.1:8787/mcp",
+        "enabled": true
+      }
+    ]
+  }
+}
+```
+
+- `mcp.servers` currently supports Streamable HTTP MCP endpoints.
+- Each server `name` becomes the imported tool prefix, so a remote `search` tool appears as `docs__search`.
+- MCP servers with `enabled: true` connect at startup; disabled ones stay disconnected until you turn them back on.
+- Invalid MCP URLs are skipped immediately, and enabled servers that are unreachable are skipped with a warning.
+- `/mcp` can enable or disable configured MCP servers during the current app run, and that on/off state is persisted.
 
 ## Docs
 
@@ -123,6 +157,17 @@ bun run format
 bun run typecheck
 ```
 
+## Also makes LLMs smarter
+
+LLMs famously tell you to walk 50 meters to the car wash — forgetting the car needs to be there too. Not on our watch.
+
+<table align="center">
+  <tr>
+    <td><img src="assets/mc-claude-smart.png" alt="Claude correctly answering the car wash question" width="400" /></td>
+    <td><img src="assets/mc-gpt-smart.png" alt="GPT correctly answering the car wash question" width="400" /></td>
+  </tr>
+</table>
+
 ## License
 
 MIT

package/benchmark-baseline.sh

@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+cd -- "$(dirname -- "${BASH_SOURCE[0]}")"
+
+PYTHONPATH="$PWD/terminal-bench" harbor run -y \
+  --job-name benchmark-baseline-full-$(date +%F__%H-%M-%S) \
+  --jobs-dir "$PWD/terminal-bench/jobs" \
+  --agent-import-path mini_coder_agent:MiniCoderAgent \
+  --ak version=0.5.12 \
+  --model openai-codex/gpt-5.4 \
+  --dataset terminal-bench@2.0 \
+  --n-attempts 2 \
+  --n-concurrent 2 \
+  --max-retries 0

package/benchmark-loop.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+stop() {
+  echo "stopping loop"
+  exit 0
+}
+
+trap stop INT TERM
+
+while :; do
+  echo ""
+  echo "> Running step"
+  echo ""
+  mc -p "See BENCHMARK.md and PROGRESS.md first. Do the next step in the process. Once the step is complete update PROGRESS.md"
+
+  sleep 60
+done
+