autopilot-ai 2.0.0 → 2.1.0

package/README.md

@@ -390,7 +390,6 @@ autopilot
 | `/mode edit\|plan\|auto` | Switch permission mode |
 | `/shell auto\|bash\|cmd\|powershell` | Show or set the shell for the bash tool |
 | `/thinking low\|medium\|high` | Reasoning effort (persists) |
-| `/theme` | Interactive theme picker (`Ctrl+T`) |
 | `/resume` | Pick a past conversation to restore |
 | `/compact` | Summarize older turns to free context |
 | `/init` | Scan repo and write `KIMI.md` project context |
@@ -401,9 +400,10 @@ autopilot
 | `/provider` | List providers, or switch with `/provider <id> [key]` (see [Providers](#providers)) |
 | `/jev` | Ask Jev for a typed, scoped one-shot decision |
 | `/subagents` | List running subagents; `/subagents cancel <n>\|all` stops them; `/subagents off\|suggest\|auto` sets the delegation policy (default: `auto`) |
-| `/now` | Run your latest queued message now instead of after the current turn (also `Ctrl+G` in Ink) |
+| `/now` | Run your latest queued message now instead of after the current turn |
 | `/key` | Show the active provider's key — on OpenRouter with spend and credit (`/key set <key>`, `/key clear`) |
 | `/cost` | Show cost by session, day, month and all time (OpenRouter-confirmed, or estimated from the provider's prices) |
+| `/good` / `/bad` `[note]` | Rate the last turn; recorded in the session trace when [tracing](#session-traces-and-evals) is on |
 | `/update` | Check for updates |
 | `/help` | List all commands |
 
@@ -428,10 +428,8 @@ Type `!` followed by a command (for example `! gh auth login`) to run it in your
 | `Ctrl+C` / `Esc` | Interrupt current turn when busy; exit when idle |
 | `Ctrl+R` | Toggle reasoning display |
 | `Ctrl+O` | Toggle verbose tool output |
-| `Ctrl+T` | Open theme picker |
 | `Shift+Tab` | Cycle mode (edit → plan → auto) |
 | `↑` / `↓` | Walk prompt history |
-| `Ctrl+G` | While the agent works: run your latest queued message now instead of after the turn |
 
 ### Messages sent while the agent is working
 
@@ -439,7 +437,7 @@ A message you send mid-turn is triaged instead of always waiting in line:
 
 - **interrupt** ("stop, that's the wrong file"): the turn stops at the next safe point and your message runs next. Running subagents are never cancelled by an interrupt; while they run, the message is delivered as soon as they finish.
 - **steer** ("also handle the empty case"): folded into the current task at the agent's next step.
-- **queue** ("after this, update the changelog", or anything unclear): runs as its own turn afterwards. The queue shows `Ctrl+G run this now` so you can promote it.
+- **queue** ("after this, update the changelog", or anything unclear): runs as its own turn afterwards. Send `/now` to promote it.
 - **aside** ("how's it going?"): answered from the current state without disturbing the turn.
 
 Clear phrasing is decided instantly; ambiguous messages get a quick model check (OpenRouter only), and anything uncertain is queued. Use `/subagents cancel <n>` to stop a single subagent.
@@ -498,6 +496,33 @@ Each log entry maps to one OTel `LogRecord`. Correlation IDs
 encoding, and a `service.name=kimiflare` + `service.version` pair sits
 on the resource.
 
+## Session traces and evals
+
+Autopilot can record its own sessions so you can see how it performs on your work, and compare setups on your own tasks. Everything stays on your machine.
+
+### Session traces
+
+Tracing is off by default. Turn it on for one run with `KIMIFLARE_TRACE=1`, or always with `"trace": { "enabled": true }` in `~/.config/kimiflare/config.json`. Each session then writes a `trace.jsonl` under `~/.local/share/kimiflare/traces/` (`KIMIFLARE_TRACE_DIR` or `trace.root` to change it), one event per line:
+
+- every model request and response, with tokens, cost and latency
+- tool calls and their results, and each permission decision
+- compactions, completion checks, subagents and turn ends
+- **outcome signals:** the git changes a session made, test, lint, typecheck and build commands and whether they passed, your `/good` and `/bad` ratings (with an optional note), and whether your next message corrected the agent or told it to carry on
+
+Large bodies are stored once, by hash, next to the trace. Likely secrets are redacted before anything is written, nothing is uploaded, and old traces are pruned automatically: after 90 days, above 200 MB for one session, or when all traces pass 2 GB.
+
+### Evals
+
+`autopilot eval` runs Autopilot headlessly on a set of tasks and compares two configurations: a model, a setting, or a harness change.
+
+```bash
+autopilot eval run --tasks evals/tasks/autopilot-bench --config evals/configs/baseline.json --model <id> --out /tmp/eval-a
+autopilot eval run --tasks evals/tasks/autopilot-bench --config evals/configs/my-change.json --model <id> --out /tmp/eval-b
+autopilot eval compare /tmp/eval-a/results.jsonl /tmp/eval-b/results.jsonl
+```
+
+Each task is a JSON file naming a repo, a base commit, a prompt and a test command. Every run gets its own git worktree with tracing forced on, and the task passes if the test exits 0. `eval compare` prints success rates with a paired significance test, cost, turns and the tasks that flipped. This repository ships `autopilot-bench`, twenty tasks mined from its own merged pull requests; see [evals/README.md](evals/README.md) for the task format and how to build a set from your own repo.
+
 ## Hooks
 
 autopilot can fire shell commands at these points, configured