robot_lab-to 0.2.6

data/docs/concepts/verification.md

@@ -0,0 +1,75 @@
+# Verification Gate
+
+A robot reporting `success: true` is a *claim*, not a fact. The verification gate
+makes that claim count only when an independent command — one you choose — agrees.
+
+This is separation of duties applied to a single agent: the **robot does the
+work**, but the **orchestrator runs the deciding command**. The robot cannot
+self-certify a passing verdict.
+
+## Enabling it
+
+```bash
+robot-to "Fix the failing parser tests" \
+  --verify-command "bundle exec rake test" \
+  --verify-timeout 600
+```
+
+When `--verify-command` is set, every iteration the robot marks successful is
+gated:
+
+```
+robot says success ──► run verify-command ──► exit 0 ? ──► COMMIT
+                                                 │
+                                                 │ non-zero / timeout
+                                                 ▼
+                                              ROLLBACK  (recorded as [VERIFY FAILED])
+```
+
+If verification fails, the working tree is reset with `git reset --hard`, the
+iteration counts as a failure (incrementing the consecutive-failure counter), and
+the verification output is written to `notes.md` so the next robot can see *why*
+it failed.
+
+## How it runs
+
+The `Verifier`:
+
+- Runs the command in the project working directory via `Open3.popen2e`.
+- Launches it in its own **process group** so a timeout can kill the whole
+  process tree — test runners that spawn children won't survive a timeout.
+- Enforces `--verify-timeout` seconds (default **600**). On timeout the process
+  group is terminated and the gate fails.
+- Captures combined stdout+stderr, clamped to ~4 000 characters for the notes.
+
+A non-zero exit **or** a timeout fails the gate.
+
+## Choosing a verify command
+
+The command should be the real signal of correctness for your objective:
+
+| Objective | Good `--verify-command` |
+|-----------|-------------------------|
+| Fix failing tests | `bundle exec rake test` |
+| Keep the build green | `npm run build` |
+| Don't regress lint | `bundle exec rubocop` |
+| Full quality bar | `bundle exec rake` (tests + lint + coverage) |
+
+Tips:
+
+- Make it **fast enough to run every iteration** — it runs on each claimed
+  success.
+- Make it **deterministic** — flaky commands cause spurious rollbacks.
+- Combine checks with `&&` if you want several gates: `--verify-command "bundle
+  exec rake test && bundle exec rubocop"`.
+
+## Without a verify command
+
+If you don't set `--verify-command`, the robot's self-reported `success` is taken
+at face value and committed. This is fine for low-risk objectives (docs,
+comments, scaffolding) but for anything that must keep working, a verification
+gate is strongly recommended.
+
+---
+
+Next: [Evals: Scoring Iterations](evals.md).

data/docs/configuration/cli.md

@@ -0,0 +1,111 @@
+# CLI Reference
+
+The `robot-to` executable runs an autonomous loop toward an objective.
+
+## Synopsis
+
+```
+robot-to [objective] [options]
+```
+
+The **objective** is a single argument (quote it). If omitted, `robot-to` reads
+the objective from standard input — handy for long objectives:
+
+```bash
+robot-to "Add request logging middleware and tests"
+
+# or via stdin
+echo "Add request logging middleware and tests" | robot-to
+```
+
+## Options
+
+| Flag | Argument | Default | Description |
+|------|----------|---------|-------------|
+| `--provider` | `NAME` | `openai` | LLM provider (`anthropic`, `openai`, …). |
+| `--model` | `MODEL` | `gpt-5.5` | Model identifier for the provider. |
+| `--max-iterations` | `N` | *unlimited* | Stop after `N` iterations. |
+| `--max-tokens` | `N` | *unlimited* | Stop after `N` total tokens. |
+| `--stop-when` | `CONDITION` | *unset* | Natural-language stop condition the robot evaluates. |
+| `--max-consecutive-failures` | `N` | `3` | Abort after `N` consecutive failures. |
+| `--verify-command` | `CMD` | *unset* | Command that must pass before a success is committed. |
+| `--verify-timeout` | `SECONDS` | `600` | Timeout for `--verify-command`. |
+| `--max-verify-repairs` | `N` | `2` | Repair-in-place attempts when the eval's gate fails before rolling back. |
+| `--eval` | `code`\|`null`\|`prose` | *code if verify/measure/target set, else null* | Eval strategy — see [Evals](../concepts/evals.md). |
+| `--measure` | `CMD` | *unset* | Command printing a numeric score (higher = better); drives `Evals::Code`. |
+| `--target` | `FLOAT` | *unset* | Stop once the measured score reaches this. |
+| `--spec` | `PATH` | *unset* | Spec/outline artifact `Evals::Prose`'s judge measures against. |
+| `--floor` | `CMD` | *unset* | Mechanizable floor check for prose (links, outline coverage, AI-tells). |
+| `--judge-model` | `MODEL` | *`--model`* | Model for the pairwise prose judge. |
+| `--stop-on-plateau` | `N` | *unset — off* | Stop after `N` iterations with no committed improvement. |
+| `--[no-]require-improvement` | — | on | Roll back gate-passing iterations that don't beat the parent (default on). |
+| `--protect-path` | `GLOB` (repeatable) | *none* | Lock a grader file from robot edits (always-on `GraderLock`). |
+| `--commit-format` | `default`\|`conventional` | `default` | Commit message format. |
+| `--run-dir` | `PATH` | `.robot_lab_to` | Directory for run state. |
+| `--local-guards` | — | off | Add built-in file tools + small-model guardrails. |
+| `--no-stream` | — | streaming on | Disable streaming (required for local Ollama tool calls). |
+| `--debug` | — | off | Keep verbose provider logging enabled. |
+| `--version` | — | — | Print version and exit. |
+| `-h`, `--help` | — | — | Show help and exit. |
+
+See the [Settings Reference](settings.md) for the full meaning of each option and
+its config-file / environment-variable equivalents.
+
+## Examples
+
+A bounded run with a verification gate:
+
+```bash
+robot-to "Fix the failing parser tests" \
+  --provider anthropic \
+  --model claude-sonnet-4-6 \
+  --max-iterations 15 \
+  --verify-command "bundle exec rake test"
+```
+
+An overnight migration with a natural-language stop condition:
+
+```bash
+robot-to "Migrate from Minitest to RSpec" \
+  --max-iterations 60 \
+  --max-tokens 3000000 \
+  --verify-command "bundle exec rake" \
+  --stop-when "every test file is RSpec and the suite passes" \
+  --commit-format conventional
+```
+
+A scored run that descends toward a measured target (see [Evals](../concepts/evals.md)):
+
+```bash
+robot-to "Raise parser coverage to 90%" \
+  --verify  "bundle exec rake test" \
+  --measure "bundle exec rake coverage" \
+  --target  90 \
+  --protect-path test/coverage_helper.rb
+```
+
+A judged prose run that ends on plateau instead of a target:
+
+```bash
+robot-to "Write an opinionated guide to the Viable Systems Model" \
+  --eval prose --spec outline.md --floor "rake docs:lint" \
+  --stop-on-plateau 3
+```
+
+A fully local run on Ollama (see [Local Models](../local-models/index.md)):
+
+```bash
+robot-to "Add a greet(name) method in greeter.rb" \
+  --provider openai \
+  --model gpt-oss:20b \
+  --local-guards \
+  --no-stream \
+  --max-iterations 5
+```
+
+## Exit behavior
+
+`robot-to` runs to completion or until a stop condition trips, then prints an
+[exit summary](../getting-started/anatomy-of-a-run.md#the-exit-summary). Press
+**Ctrl-C** for a graceful stop — the in-flight iteration is rolled back, the loop
+exits, and the summary still prints.

data/docs/configuration/index.md

@@ -0,0 +1,93 @@
+# Configuration
+
+`robot_lab-to` is configured through a layered cascade built on
+[`myway_config`](https://github.com/MadBomber/myway_config). Lower layers provide
+defaults; higher layers override them.
+
+## The cascade
+
+From lowest to highest precedence:
+
+```
+1. Bundled defaults        lib/robot_lab/to/config/defaults.yml
+2. User config file        ~/.config/robot_lab/to.yml
+3. Environment variables   ROBOT_LAB_TO_*
+4. CLI flags / To.run args  (highest — always wins)
+```
+
+A value set at a higher layer overrides the same value from any lower layer.
+
+## 1. Bundled defaults
+
+Shipped with the gem. You never edit these, but they define the baseline:
+
+```yaml
+defaults:
+  provider: openai
+  model: gpt-5.5
+  max_tool_rounds: 100
+  max_consecutive_failures: 3
+  max_retries: 2
+  max_submit_nudges: 1
+  verify_timeout: 600
+  commit_format: default
+  run_dir: .robot_lab_to
+  local_guards: false
+  stream: true
+  debug: false
+```
+
+## 2. User config file
+
+Create `~/.config/robot_lab/to.yml` to set your own defaults — for example, to
+always use a local model:
+
+```yaml
+provider: openai          # routes to Ollama's OpenAI-compatible endpoint
+model: gpt-oss:20b
+local_guards: true
+stream: false
+max_consecutive_failures: 4
+```
+
+These apply to every `robot-to` run unless overridden by an env var or CLI flag.
+
+## 3. Environment variables
+
+Every setting can be set via an environment variable prefixed with
+`ROBOT_LAB_TO_`. Nested keys use a double underscore (rare here, since the config
+is flat):
+
+```bash
+export ROBOT_LAB_TO_MODEL="gpt-oss:20b"
+export ROBOT_LAB_TO_LOCAL_GUARDS=true
+export ROBOT_LAB_TO_MAX_CONSECUTIVE_FAILURES=4
+```
+
+Provider **API keys** are *not* `robot_lab-to` settings — they are read by the
+underlying provider (e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`).
+
+## 4. CLI flags / `To.run` arguments
+
+The highest-precedence layer. Anything passed on the command line (or as a keyword
+to `RobotLab::To.run`) wins over all other layers:
+
+```bash
+robot-to "..." --model claude-sonnet-4-6 --max-iterations 20
+```
+
+```ruby
+RobotLab::To.run("...", model: "claude-sonnet-4-6", max_iterations: 20)
+```
+
+Some options are **CLI-only** (no YAML default) because they are run-specific:
+`--max-iterations`, `--max-tokens`, `--stop-when`, `--verify-command`, and the
+whole [Evals](../concepts/evals.md) family (`--eval`, `--measure`, `--target`,
+`--spec`, `--floor`, `--judge-model`, `--stop-on-plateau`,
+`--[no-]require-improvement`, `--protect-path`). When unset they mean "no limit
+/ not configured".
+
+---
+
+- See the [Settings Reference](settings.md) for every option in detail.
+- See the [CLI Reference](cli.md) for every flag.

data/docs/configuration/settings.md

@@ -0,0 +1,270 @@
+# Settings Reference
+
+Every `robot_lab-to` setting, its default, and what it controls. Each setting can
+be supplied via the user config file, an `ROBOT_LAB_TO_*` environment variable, a
+CLI flag, or a `RobotLab::To.run` keyword argument (see
+[the cascade](index.md#the-cascade)).
+
+## Model & provider
+
+### `provider`
+
+- **Default:** `openai`
+- **CLI:** `--provider NAME`
+
+The LLM provider, passed through to RobotLab / RubyLLM. Use `anthropic`,
+`openai`, etc. For **local Ollama models**, set this to `openai` and point
+RubyLLM at Ollama's OpenAI-compatible endpoint — see [Ollama Setup](../local-models/ollama.md).
+
+### `model`
+
+- **Default:** `gpt-5.5`
+- **CLI:** `--model MODEL`
+
+The model identifier for the chosen provider (e.g. `claude-sonnet-4-6`,
+`gpt-5.5`, `gpt-oss:20b`).
+
+### `stream`
+
+- **Default:** `true`
+- **CLI:** `--no-stream` to disable
+
+Whether to stream the model response. Streaming enables per-chunk token
+accounting and mid-iteration token-budget enforcement. **Local Ollama models must
+run with `--no-stream`** — Ollama suppresses tool calls when streaming. With
+streaming off, tokens are accounted from each iteration's result instead. See
+[Streaming and tool calls](../local-models/ollama.md#streaming-and-tool-calls).
+
+## Loop control
+
+### `max_iterations`
+
+- **Default:** *(unset — no limit)*
+- **CLI:** `--max-iterations N`
+
+Stop after `N` iterations. CLI-only; strongly recommended for unattended runs.
+
+### `max_tokens`
+
+- **Default:** *(unset — no limit)*
+- **CLI:** `--max-tokens N`
+
+Stop once cumulative input+output tokens reach `N`. CLI-only.
+
+### `max_consecutive_failures`
+
+- **Default:** `3`
+- **CLI:** `--max-consecutive-failures N`
+
+Abort after `N` consecutive failed iterations. Resets to zero on each successful,
+committed iteration. The primary safety net against a run going nowhere.
+
+### `max_verify_repairs`
+
+- **Default:** `2`
+- **CLI:** `--max-verify-repairs N`
+
+When the robot claims success but `verify_command` fails, the orchestrator hands
+the failure output back to the **same** robot — its chat and working tree intact —
+and lets it fix the code, re-verifying after each attempt, up to `N` times before
+rolling the iteration back. Nothing commits until the gate is green, so the git
+history stays clean; a near-miss just isn't discarded. Set to `0` to roll back
+immediately on the first verify failure. See [Verification Gate](../concepts/verification.md).
+
+### `stop_when`
+
+- **Default:** *(unset)*
+- **CLI:** `--stop-when "<condition>"`
+
+A natural-language condition the robot evaluates after each successful iteration.
+When the robot judges it met, the run stops. CLI-only. See
+[Stop Conditions](../concepts/stop-conditions.md).
+
+### `max_submit_nudges`
+
+- **Default:** `1`
+
+How many times to re-prompt a robot that finished without calling
+`submit_iteration_result`. Recovers the "thinks but doesn't act" case before
+counting the iteration as a failure.
+
+### `max_retries`
+
+- **Default:** `2`
+
+How many consecutive *errors* (exceptions, e.g. transient API failures) to retry
+before aborting. Retries use an interruptible exponential backoff
+(`60 × 2ⁿ` seconds).
+
+### `max_tool_rounds`
+
+- **Default:** `100`
+
+Passed to the robot — the maximum number of tool-call rounds within a single
+iteration before RobotLab's circuit breaker stops it.
+
+## Verification
+
+### `verify_command`
+
+- **Default:** *(unset — no verification)*
+- **CLI:** `--verify-command "CMD"`
+
+A command that must exit `0` before a robot's claimed success is committed.
+CLI-only. See [Verification Gate](../concepts/verification.md).
+
+### `verify_timeout`
+
+- **Default:** `600` (seconds)
+- **CLI:** `--verify-timeout SECONDS`
+
+Maximum time `verify_command` may run before it is killed (whole process group)
+and the gate fails.
+
+## Evals (scoring)
+
+CLI-only (no YAML default) unless noted. See [Evals](../concepts/evals.md) for
+the full behavioral model.
+
+### `eval`
+
+- **Default:** *(unset — `Evals::Code` if `verify_command`/`eval_measure`/
+  `eval_target` is set, else the unscored `Evals::Null`)*
+- **CLI:** `--eval code|null|prose`
+
+Selects the scoring strategy. Via the Ruby API, `eval:` also accepts an instance
+responding to `#score`, a proc, or a `Symbol` registered with
+`RobotLab::To.register_eval` — the CLI flag itself only resolves the three named
+strings.
+
+### `eval_measure`
+
+- **Default:** *(unset)*
+- **CLI:** `--measure CMD`
+
+A command that prints a number to stdout (higher = better) — the descent signal
+for `Evals::Code`. Setting this alone (without `verify_command`) still selects
+`Evals::Code`, not `Evals::Null`.
+
+### `eval_target`
+
+- **Default:** *(unset)*
+- **CLI:** `--target FLOAT`
+
+Stop the run once `eval_measure`'s value reaches this (`Evals::Code`).
+
+### `eval_spec`
+
+- **Default:** *(unset)*
+- **CLI:** `--spec PATH`
+
+The spec/outline artifact `Evals::Prose`'s judge measures the draft against.
+Automatically added to `Evals::Prose#protected_paths` — locked from robot edits
+by `GraderLock`.
+
+### `eval_floor`
+
+- **Default:** *(unset)*
+- **CLI:** `--floor CMD`
+
+An optional mechanizable check (e.g. link validation, outline coverage) that
+must pass before `Evals::Prose` calls the judge at all. A failing floor is
+scored `:worse` without spending a judge call.
+
+### `eval_judge_model`
+
+- **Default:** *(unset — falls back to `model`)*
+- **CLI:** `--judge-model MODEL`
+
+The model used for `Evals::Prose`'s pairwise judge. Defaults to the doer's own
+`model`; set this to use a different (often cheaper, or deliberately more
+skeptical) model for the judge role.
+
+### `stop_on_plateau`
+
+- **Default:** *(unset — off)*
+- **CLI:** `--stop-on-plateau N`
+
+Stop after `N` consecutive iterations with no committed improvement. The
+primary terminator for `Evals::Prose`, which never sets a measurable target. See
+[Stop Conditions](../concepts/stop-conditions.md#plateau-no-improvement).
+
+### `require_improvement`
+
+- **Default:** `true` (CLI/Ruby-only hardcoded default; not in `defaults.yml`)
+- **CLI:** `--[no-]require-improvement`
+
+When true, a gate-passing iteration that doesn't beat the parent commit's score
+is rolled back rather than committed. `--no-require-improvement` restores
+"commit any gate-passing success."
+
+### `protect_paths`
+
+- **Default:** `[]`
+- **CLI:** `--protect-path GLOB` (repeatable)
+
+Extra grader files (globs) locked from robot writes/edits/bash, on top of
+whatever the configured eval's `#protected_paths` already locks. Always
+enforced via `Guards::GraderLock` when the resolved set is non-empty —
+independent of `local_guards`.
+
+## Local models
+
+### `local_guards`
+
+- **Default:** `false`
+- **CLI:** `--local-guards`
+
+Attach the built-in file tools (`read`, `write`, `edit`, `bash`) and the
+small-model guardrail hooks. Designed for local models that need a more
+constrained, forgiving tool surface. See [Local Models](../local-models/index.md).
+
+### `write_guard`
+
+- **Default:** `true` (Ruby-only — no CLI flag)
+- **Ruby:** `RobotLab::To.run(..., write_guard: false)`
+
+When `local_guards` is on, `write_guard: false` drops `Guards::WriteGuard` from
+the installed set (`Guards.install(..., except: [Guards::WriteGuard])`), letting
+the robot freely overwrite a file it is iteratively refining — WriteGuard
+otherwise refuses `write` on a file that already exists and redirects to `edit`,
+which is unhelpful for a robot rewriting a whole prose draft each pass. See
+[Guardrails](../local-models/guardrails.md).
+
+## Git & output
+
+### `commit_format`
+
+- **Default:** `default`
+- **CLI:** `--commit-format default|conventional`
+
+How commit messages are formatted:
+
+=== "default"
+
+    `robot-to <iteration>: <summary>`
+
+=== "conventional"
+
+    `<type>(<scope>): <summary>` — `type`/`scope` come from the robot's result
+    (defaulting to `chore`). Follows [Conventional Commits](https://www.conventionalcommits.org/).
+
+### `run_dir`
+
+- **Default:** `.robot_lab_to`
+- **CLI:** `--run-dir PATH`
+
+Where run state (`runs/<run_id>/`) is written. Added to `.git/info/exclude`
+automatically.
+
+### `debug`
+
+- **Default:** `false`
+- **CLI:** `--debug`
+
+Keep verbose RubyLLM/RobotLab provider logging enabled. The JSONL event log is
+written regardless.
+
+---
+
+See the [CLI Reference](cli.md) for the command-line form of every option.

data/docs/getting-started/anatomy-of-a-run.md

@@ -0,0 +1,111 @@
+# Anatomy of a Run
+
+A single `robot-to` invocation produces a git branch, a run directory, and a
+post-run summary. This page walks through each artifact.
+
+## The branch
+
+On startup the orchestrator:
+
+1. Verifies `HEAD` exists (at least one commit).
+2. Creates a branch named `robot-to/<slug>-<timestamp>` from the current `HEAD`.
+3. Records the `base_commit` SHA so the exit summary can diff the whole run.
+
+Every successful iteration is committed onto this branch. Your original branch is
+untouched.
+
+```
+* 6a7137c robot-to 2: add docstring and confirm behavior   ← iteration 2
+* 5810ae0 robot-to 1: create greeter.rb with greet method  ← iteration 1
+* 362b8d9 initial                                           ← base_commit
+```
+
+### Commit message format
+
+Controlled by `--commit-format`:
+
+=== "default"
+
+    ```
+    robot-to 1: <result summary>
+    ```
+
+=== "conventional"
+
+    ```
+    feat(scope): <result summary>
+    ```
+
+    The `type` and `scope` come from the robot's submitted result (defaulting to
+    `chore`). See [Commit conventions](../configuration/settings.md#commit_format).
+
+## The run directory
+
+All run state lives under `.robot_lab_to/runs/<run_id>/` (the directory is added
+to `.git/info/exclude` automatically):
+
+```
+.robot_lab_to/
+└── runs/
+    └── 20260623-144554-3e5c27/
+        ├── notes.md         # cross-iteration memory (human-readable)
+        ├── run.log          # JSONL event log (machine-readable)
+        └── checkpoints/     # pre-edit file snapshots (local_guards only)
+```
+
+- **`notes.md`** — written by the orchestrator after each iteration; read by the
+  robot at the start of the next. See [Cross-Iteration Memory](../concepts/notes.md).
+- **`run.log`** — one JSON object per line, one per lifecycle event. See
+  [Run State & Event Log](../concepts/run-state.md).
+- **`checkpoints/`** — first-write-wins file snapshots taken before a Write/Edit,
+  created only when `--local-guards` is active. See [Guardrails](../local-models/guardrails.md).
+
+## One iteration, step by step
+
+```
+iteration:start           → fresh robot built with prompt + notes
+agent:run:start           → robot.run() begins
+  (robot uses tools, calls submit_iteration_result)
+agent:run:end
+agent:nudge?              → if no result submitted, re-ask (max_submit_nudges)
+eval                      → the configured eval scores the working tree (if reported success)
+commit:success            → git add -A && commit                 (gate ok + improved)
+  — or —
+iteration:no_improvement  → git reset --hard                     (gate ok, not improved)
+iteration:gate_failure    → git reset --hard (after eval:repair)  (gate failed)
+iteration:failure         → git reset --hard                     (robot reported failure / no submit)
+```
+
+A committed iteration resets the consecutive-failure counter and the plateau
+counter; a failure or gate failure increments consecutive-failures, and any
+non-committing outcome increments the plateau counter (may trip
+`--max-consecutive-failures` or `--stop-on-plateau`). See
+[Evals](../concepts/evals.md) for the full scoring model — by default (no
+`--verify-command`/`--eval`), every reported success commits, matching this
+page's simpler mental model.
+
+## The exit summary
+
+When the loop ends — whether it completed, hit a stop condition, or aborted —
+`robot-to` prints a summary to stdout:
+
+```
+robot-to complete — claude-sonnet-4-6 — 4m 12s
+Branch: robot-to/migrate-to-rspec-20260623-144554
+
+  Iterations:   8   (6 committed, 2 rolled back)
+  Changes:      +312 / -47 across 14 files
+  Tokens:       1,204,883 in / 38,902 out
+
+Run state:  .robot_lab_to/runs/20260623-144554-3e5c27/
+Next steps:
+  git log robot-to/migrate-to-rspec-20260623-144554
+  git diff 362b8d9..HEAD
+```
+
+If the run was stopped early, the header shows the reason (e.g. `max iterations
+reached (8)`, `4 consecutive failures`, or `stop condition met: ...`).
+
+---
+
+Next: the full lifecycle in [The Iteration Loop](../concepts/iteration-loop.md).