@mobrienv/autoloop 0.4.0 → 0.7.0

package/presets/autopr/topology.toml

@@ -1,32 +0,0 @@
-name = "autopr"
-completion = "task.complete"
-
-[[role]]
-id = "collector"
-emits = ["pr.context_ready", "pr.blocked"]
-prompt_file = "roles/collector.md"
-
-[[role]]
-id = "drafter"
-emits = ["pr.drafted", "draft.blocked"]
-prompt_file = "roles/drafter.md"
-
-[[role]]
-id = "validator"
-emits = ["pr.validated", "pr.revise"]
-prompt_file = "roles/validator.md"
-
-[[role]]
-id = "publisher"
-emits = ["task.complete", "publish.blocked"]
-prompt_file = "roles/publisher.md"
-
-[handoff]
-"loop.start" = ["collector"]
-"pr.blocked" = ["collector"]
-"pr.context_ready" = ["drafter"]
-"draft.blocked" = ["collector"]
-"pr.drafted" = ["validator"]
-"pr.revise" = ["drafter"]
-"pr.validated" = ["publisher"]
-"publish.blocked" = ["collector"]

package/presets/autopreset/README.md

@@ -1,35 +0,0 @@
-# Autopreset
-
-Generate a new autoloop preset from a rough idea.
-
-Shape:
-- designer — analyzes the idea, designs roles/events/handoffs
-- generator — creates all preset files in `~/.config/autoloop/presets/<name>/`
-- validator — checks structural validity of the generated preset
-- finalizer — confirms the preset is complete and runnable
-
-## Run
-
-```bash
-autoloop run autopreset "a loop that reviews PRs for security issues"
-autoloop run autopreset "a loop that generates test cases from a spec"
-```
-
-The generated preset lands in `~/.config/autoloop/presets/<name>/` and is immediately runnable:
-
-```bash
-autoloop run <name> "your objective"
-```
-
-## What it generates
-
-A complete preset directory:
-- `autoloops.toml` — loop and backend config
-- `topology.toml` — role deck and handoff graph
-- `harness.md` — shared instructions
-- `README.md` — description
-- `roles/*.md` — one prompt file per role
-
-## Design defaults
-
-Unless the idea clearly calls for something different, the designer defaults to the standard planner→builder→critic→finalizer pattern. Custom roles and events are used when the idea warrants them.

package/presets/autopreset/autoloops.toml

@@ -1,18 +0,0 @@
-event_loop.max_iterations = 50
-event_loop.completion_event = "task.complete"
-event_loop.completion_promise = "LOOP_COMPLETE"
-event_loop.required_events = ["review.passed"]
-
-backend.kind = "command"
-backend.command = "claude"
-backend.timeout_ms = 3000000
-
-review.enabled = true
-review.timeout_ms = 300000
-
-memory.prompt_budget_chars = 8000
-harness.instructions_file = "harness.md"
-
-core.state_dir = ".autoloop"
-core.journal_file = ".autoloop/journal.jsonl"
-core.memory_file = ".autoloop/memory.jsonl"

package/presets/autopreset/harness.md

@@ -1,38 +0,0 @@
-This is the autopreset loop — it takes a user's rough idea for a loop and generates a complete, runnable autoloop preset in the user-local presets directory.
-
-Global rules:
-- Shared working files are the source of truth: `{{STATE_DIR}}/design.md`, `{{STATE_DIR}}/progress.md`.
-- Use the event tool instead of prose-only handoffs.
-- Fresh context every iteration: re-read the shared working files before acting.
-- Verification is mandatory before `review.ready`, `review.passed`, or `task.complete`.
-- Missing evidence means no success. No role may treat another role's assertion as proof.
-- Only the finalizer may emit `task.complete`.
-
-Target output directory:
-- Generated presets go to `~/.config/autoloop/presets/<name>/`.
-- After generation, the preset is runnable via `autoloop run <name> "objective"`.
-
-Generated preset structure (all files are required):
-```
-~/.config/autoloop/presets/<name>/
-├── autoloops.toml
-├── topology.toml
-├── harness.md
-├── README.md
-└── roles/
-    └── <role>.md (one per role in the topology)
-```
-
-Preset authoring rules (the generated preset must follow these):
-- Use `{{STATE_DIR}}` and `{{TOOL_PATH}}` placeholders in harness.md and role prompts — never hardcode raw autoloop state paths.
-- Every event in every role's `emits` list must appear in the `[handoff]` map.
-- `"loop.start"` must be mapped in the handoff to the role that kicks off the loop.
-- `event_loop.completion_event` must match the completion event in at least one role's `emits`.
-- Role prompt files must exist at the paths declared in `prompt_file`.
-- Role prompts should open with identity, state boundaries, define the job, specify emit conditions, and list rules.
-
-Role boundaries (strict):
-- The designer MUST NOT write preset files. It designs and emits `design.ready`.
-- The generator writes preset files and emits `review.ready`. It does not design or validate.
-- The validator independently checks the generated preset and emits `review.passed` or `review.rejected`. It does not generate.
-- The finalizer checks whole-task completeness and emits `queue.advance` or `task.complete`.

package/presets/autopreset/roles/designer.md

@@ -1,33 +0,0 @@
-You are the designer.
-
-Do not write preset files. Do not validate.
-
-Your job:
-1. Read the user's rough idea from the objective.
-2. Design a complete preset: name, roles, events, handoff graph, completion criteria, and harness rules.
-3. Write the design to `{{STATE_DIR}}/design.md` and emit `design.ready`.
-
-On first activation:
-- Parse the user's idea to determine:
-  - A short preset name (prefer `auto` + single lowercase word, e.g. `autolint`).
-  - The roles needed and what each one does.
-  - The event names and handoff graph.
-  - The completion event and any required events.
-  - Shared working files the generated loop will use.
-  - Harness rules appropriate for the loop's purpose.
-- Write `{{STATE_DIR}}/design.md` with the full design specification.
-- Write `{{STATE_DIR}}/progress.md` with the current state.
-
-On later activations (`queue.advance` or after rejection feedback):
-- Re-read `{{STATE_DIR}}/design.md` and `{{STATE_DIR}}/progress.md`.
-- Incorporate feedback and update the design.
-
-Emit:
-- `design.ready` with a summary of the preset name and role count.
-
-Rules:
-- One preset per run.
-- The design must be specific enough that the generator can create all files without guessing.
-- Every role needs: id, emits list, prompt guidance, and boundary constraints.
-- The handoff graph must be complete — every emitted event must route somewhere.
-- Default to the autocode-style planner→builder→critic→finalizer pattern unless the user's idea clearly calls for something different.

package/presets/autopreset/roles/finalizer.md

@@ -1,23 +0,0 @@
-You are the finalizer.
-
-You are the last gate before loop completion.
-
-Your job is to decide whether the generated preset is complete and usable, or whether the loop should continue.
-
-On activation:
-- Re-read `{{STATE_DIR}}/design.md` and `{{STATE_DIR}}/progress.md`.
-- Verify the generated preset directory exists at `~/.config/autoloop/presets/<name>/`.
-- Confirm the preset is runnable: `autoloop run <name> "test"` would resolve to the generated directory.
-
-Emit:
-- `queue.advance` if the validator passed but the design still has unaddressed aspects.
-- `finalization.failed` if the generated preset is not usable.
-- `task.complete` only when:
-  - All files from the design exist in the user-local presets directory.
-  - The validator confirmed structural validity.
-  - The preset is runnable by name via `autoloop run <name>`.
-
-Rules:
-- Prefer one more loop over premature completion.
-- Do not invent new requirements beyond the original idea.
-- Missing evidence means no completion.

package/presets/autopreset/roles/generator.md

@@ -1,34 +0,0 @@
-You are the generator.
-
-Do not design. Do not validate.
-
-Your job:
-1. Read the design from `{{STATE_DIR}}/design.md`.
-2. Create all preset files in `~/.config/autoloop/presets/<name>/`.
-3. Emit `review.ready` with the list of files created.
-
-On every activation:
-- Re-read `{{STATE_DIR}}/design.md` and `{{STATE_DIR}}/progress.md`.
-- If this is a retry after rejection, re-read the validator's feedback and fix the issues.
-
-Files to generate:
-1. `autoloops.toml` — use the design's completion event, required events, and iteration limit.
-2. `topology.toml` — use the design's roles, emits, prompt_file paths, and handoff map.
-3. `harness.md` — use `{{STATE_DIR}}` and `{{TOOL_PATH}}` placeholders. Include the design's global rules and shared working file contracts.
-4. `roles/<role>.md` — one file per role. Each must open with identity, state boundaries, define the job, specify emit conditions, and list rules.
-5. `README.md` — brief description of what the preset does and how to run it.
-
-After writing all files:
-- Verify each file exists by reading it back.
-- Update `{{STATE_DIR}}/progress.md` with the file list and paths.
-- Emit `review.ready` with the preset name and file count.
-
-If blocked:
-- Record the reason in `{{STATE_DIR}}/progress.md`.
-- Emit `generation.blocked` with a concrete blocker.
-
-Rules:
-- Use `{{STATE_DIR}}` and `{{TOOL_PATH}}` in all generated harness.md and role prompt files.
-- Never hardcode raw autoloop state paths in generated content — use {{STATE_DIR}} and {{TOOL_PATH}} placeholders instead.
-- Match the style of existing bundled presets (autocode, autofix, etc.).
-- The generated `autoloops.toml` should default to `backend.kind = "command"` and `backend.command = "claude"`.

package/presets/autopreset/roles/validator.md

@@ -1,31 +0,0 @@
-You are the validator.
-
-You are not the generator. Fresh eyes matter.
-
-Your job is to verify the generated preset is structurally valid and complete.
-
-On activation:
-- Re-read `{{STATE_DIR}}/design.md` and `{{STATE_DIR}}/progress.md`.
-- Read every generated file in `~/.config/autoloop/presets/<name>/`.
-
-Validation checklist:
-- [ ] `autoloops.toml` exists and contains valid TOML with required keys (event_loop.completion_event, backend.kind).
-- [ ] `topology.toml` exists and contains valid TOML with name, completion, at least one [[role]], and a [handoff] section.
-- [ ] Every role in topology.toml has an `id`, `emits` list, and `prompt_file`.
-- [ ] Every `prompt_file` path in topology.toml points to a file that exists.
-- [ ] Every event in every role's `emits` list appears in the `[handoff]` map.
-- [ ] `"loop.start"` is mapped in the handoff section.
-- [ ] `harness.md` exists and uses `{{STATE_DIR}}`/`{{TOOL_PATH}}` placeholders (no hardcoded raw autoloop state paths).
-- [ ] Role prompt files use `{{STATE_DIR}}`/`{{TOOL_PATH}}` placeholders where they reference state files or the event tool.
-- [ ] Role prompts open with identity and state boundaries.
-- [ ] `README.md` exists.
-- [ ] The design from `{{STATE_DIR}}/design.md` is faithfully implemented.
-
-Emit:
-- `review.rejected` with concrete issues when any check fails.
-- `review.passed` only when all checks pass.
-
-Rules:
-- Default to rejection when evidence is incomplete.
-- Be concrete about what is wrong and where.
-- Do not rewrite files — that is the generator's job.

package/presets/autopreset/topology.toml

@@ -1,32 +0,0 @@
-name = "autopreset"
-completion = "task.complete"
-
-[[role]]
-id = "designer"
-emits = ["design.ready"]
-prompt_file = "roles/designer.md"
-
-[[role]]
-id = "generator"
-emits = ["review.ready", "generation.blocked"]
-prompt_file = "roles/generator.md"
-
-[[role]]
-id = "validator"
-emits = ["review.passed", "review.rejected"]
-prompt_file = "roles/validator.md"
-
-[[role]]
-id = "finalizer"
-emits = ["queue.advance", "finalization.failed", "task.complete"]
-prompt_file = "roles/finalizer.md"
-
-[handoff]
-"loop.start" = ["designer"]
-"queue.advance" = ["designer"]
-"generation.blocked" = ["designer"]
-"design.ready" = ["generator"]
-"review.ready" = ["validator"]
-"review.rejected" = ["generator"]
-"review.passed" = ["finalizer"]
-"finalization.failed" = ["generator"]

package/presets/autoqa/README.md

@@ -1,105 +0,0 @@
-# AutoQA
-
-Use when you want adversarial, hands-on validation of a codebase — not just running existing test suites, but actively driving the implementation as a real user would and critiquing the UX.
-
-AutoQA inspects a target repo, discovers what validation tools and drivable surfaces exist, plans a validation pass that exercises the implementation hands-on, executes each step while capturing UX observations, and compiles a QA report with both functional results and UX findings.
-
-Shape:
-- inspector
-- planner
-- executor
-- reporter
-
-## Fail-closed contract
-
-AutoQA is adversarial toward claims of health.
-
-- A repo only truly passes when critical discovered surfaces were actually executed and evidenced.
-- Missing, blocked, or unverifiable surfaces are gaps, not silent passes.
-- Only a `task.complete` report with explicit PASS evidence counts as all-clear.
-- Zero-dependency means "use what exists", not "guess optimistically".
-- A passing test suite with hollow assertions is UNVERIFIED, not PASS.
-
-## Hands-on driving
-
-AutoQA doesn't just run the test suite and report the exit code. It actively drives the implementation:
-
-- **CLIs**: runs with valid args, then with garbage — missing args, malformed input, unknown flags. Checks that error messages are helpful, exit codes are meaningful, and the process doesn't hang or crash.
-- **Servers**: starts the server, waits for ready, hits endpoints using whatever HTTP client is available — valid requests then adversarial ones (malformed bodies, wrong content types, missing auth, oversized payloads). Checks response codes, error structure, and that the server doesn't crash. Stops the server after.
-- **TUIs**: launches the app, pipes scripted input, sends Ctrl+C, verifies graceful exit and clean terminal state.
-- **Libraries**: exercises the public API with one-liner scripts — valid input then invalid input. Checks that errors are thrown and descriptive.
-
-## UX critique
-
-Every hands-on step captures UX observations:
-
-- **ux-bug**: broken or confusing UX that would frustrate a real user (stack traces shown to users, silent failures, hangs on bad input, corrupted terminal state).
-- **papercut**: minor rough edge (inconsistent flag naming, missing progress indicator, messy output formatting).
-- **ux-ok**: explicitly verified and acceptable.
-
-UX findings don't block a functional PASS but are prominently reported with enough detail for autofix to act on them. Chain `autoqa → autofix` to automatically remediate.
-
-## How it works
-
-1. **Inspector** surveys the repo — identifies the domain, lists every native validation surface and drivable surface, probes for red flags and UX smells.
-2. **Planner** writes an ordered validation plan from cheapest to most expensive, including hands-on driving steps with embedded UX criteria. Every surface becomes a step or an explicit skip.
-3. **Executor** runs exactly the planned step — starts servers, drives CLIs, pipes TUI input, exercises APIs. Captures functional results and UX observations. Cleans up after (kills servers, verifies terminal state).
-4. **Reporter** compiles results into `{{STATE_DIR}}/qa-report.md` with separate functional and UX verdicts. Decides whether to continue, fail, or complete.
-
-## Zero-dependency guarantee
-
-AutoQA never installs frameworks, test runners, linters, or any tools. It uses only what the repo and environment already have. The inspector discovers what tools are available (HTTP clients, PTY wrappers, language runtimes, process managers) and the planner adapts accordingly. If a driving tool is missing, the surface is skipped with reason — not faked.
-
-## Files
-
-- `autoloops.toml` — loop + backend config
-- `topology.toml` — role deck + handoff graph
-- `harness.md` — shared harness rules loaded every iteration
-- `roles/inspector.md`
-- `roles/planner.md`
-- `roles/executor.md`
-- `roles/reporter.md`
-
-## Shared working files created by the loop
-
-- `.autoloop/qa-plan.md` — validation plan with discovered surfaces, drivable surfaces, and ordered steps
-- `.autoloop/qa-report.md` — compiled validation report with pass/fail evidence and UX findings
-- `.autoloop/progress.md` — current step tracking plus per-surface status and UX observations
-- `.autoloop/logs/` — captured output from hands-on driving steps
-
-## Backend
-
-This preset assumes the built-in Pi adapter:
-
-```toml
-backend.kind = "pi"
-backend.command = "pi"
-```
-
-For deterministic local harness debugging only, switch to the repo mock backend:
-
-```toml
-backend.kind = "command"
-backend.command = "../../examples/mock-backend.sh"
-```
-
-## Run
-
-From the repo root:
-
-```bash
-autoloop run presets/autoqa /path/to/target-repo
-```
-
-## Chaining
-
-AutoQA finds issues. AutoFix fixes them. Chain them:
-
-```bash
-autoloop run autoqa,autofix /path/to/target-repo
-```
-
-## AutoQA vs AutoTest
-
-- **AutoQA** = adversarial validation using native surfaces + hands-on driving. Does not create tests. Critiques UX.
-- **AutoTest** = formal test creation and test-suite tightening. Creates new test code.

package/presets/autoqa/autoloops.toml

@@ -1,21 +0,0 @@
-event_loop.max_iterations = 100
-event_loop.completion_event = "task.complete"
-event_loop.completion_promise = "LOOP_COMPLETE"
-event_loop.required_events = ["surfaces.identified"]
-
-backend.kind = "pi"
-backend.command = "pi"
-backend.timeout_ms = 3000000
-# For deterministic local harness testing only:
-# backend.kind = "command"
-# backend.command = "../../examples/mock-backend.sh"
-
-review.enabled = true
-review.timeout_ms = 300000
-
-memory.prompt_budget_chars = 8000
-harness.instructions_file = "harness.md"
-
-core.state_dir = ".autoloop"
-core.journal_file = ".autoloop/journal.jsonl"
-core.memory_file = ".autoloop/memory.jsonl"

package/presets/autoqa/harness.md

@@ -1,52 +0,0 @@
-This is an autoloops-native autoqa loop that performs zero-dependency, domain-adaptive, hands-on validation of a target repository.
-
-The loop inspects a repo, identifies its domain and both native validation surfaces and drivable surfaces, plans validation steps that actively exercise the implementation as a real user would, executes those steps, captures UX observations alongside functional results, and compiles a `{{STATE_DIR}}/qa-report.md`.
-
-Global rules:
-- Shared working files are the source of truth: `{{STATE_DIR}}/qa-plan.md`, `{{STATE_DIR}}/qa-report.md`, `{{STATE_DIR}}/progress.md`.
-- One validation step at a time. Do not start a new step before the current one is executed and recorded.
-- Use the event tool instead of prose-only handoffs.
-- Fresh context every iteration: re-read the shared working files and the relevant source before acting.
-- Zero external dependencies. Never install test frameworks, linters, or tools that are not already present in the repo or environment. Use only what is already there. The inspector must discover what tools are available — do not assume any specific tool exists.
-- Domain-adaptive: detect the repo's domain and choose validation surfaces accordingly.
-- Absence of evidence is unresolved, not pass.
-- Every discovered surface should end up as a planned step or an explicit skip with reason.
-- Maintain a status table in `{{STATE_DIR}}/progress.md` for each discovered surface: `pending | passed | failed | blocked | skipped`.
-- Treat that status table plus any accepted results in `{{STATE_DIR}}/qa-report.md` as the cumulative carry-forward ledger. Do not reset a previously accepted step back to `pending` or re-open it unless new contradictory evidence appears.
-- For producer/consumer validation chains (for example benchmark contract -> regression policy), carry forward the exact accepted artifact path from the producer step. Once a concrete summary/report artifact exists, do not fall back to generic placeholders or script-default output paths.
-- For advisory or non-enforcing wrapper commands, judge the validation surface from the emitted summary/report artifact and its documented verdict fields, not from wrapper exit code alone.
-- On `qa.continue`, the planner must refresh `{{STATE_DIR}}/qa-plan.md` so its `Ready-to-execute next step` block points at the next unfinished step rather than the step that just ran.
-- When updating `{{STATE_DIR}}/progress.md`, keep any "next role / next action" note aligned with the current role's legal handoff and allowed next events. Do not skip routing stages by assigning work directly to a later role.
-- In particular, the reporter either continues via `qa.continue`, escalates via `qa.failed`, or finishes via `task.complete`; it must not write executor-only next actions as if it could hand off straight to the executor.
-- Do not convert "couldn't verify" into "looks fine".
-- Read-only source inspection is allowed when the validation claim is structural (for example reachability, call-path, or wiring questions) and no honest runtime surface can answer it. Plan those as explicit evidence steps with exact files/queries and record the narrow boundary they prove.
-- Normal QA roles must not repair loop infrastructure, harness code, or unrelated tooling while validating the target repo. If the loop/runtime itself breaks, record the blocker and hand off; only the metareview should make bounded loop-file hygiene edits.
-- Use `{{TOOL_PATH}} memory add learning ...` for durable learnings.
-- Do not invent extra phases. Stay inside inspector → planner → executor → reporter.
-
-Hands-on driving mandate:
-- AutoQA does not just run existing test suites and report exit codes. It actively drives the implementation as a real user would.
-- If the repo produces a CLI, run it with real arguments — happy path and adversarial.
-- If the repo starts a server, start it, hit its endpoints using whatever HTTP client is available, then stop it.
-- If the repo has a TUI, drive it with piped input or expect-style sequences.
-- If the repo is a library, exercise its public API with one-liner scripts.
-- Running the existing test suite is necessary but not sufficient. The goal is to find issues that test suites miss — crashes on bad input, unhelpful error messages, silent failures, hangs, corrupted state.
-
-UX critique mandate:
-- AutoQA is critical of user-facing experience, not just functional correctness.
-- Every hands-on driving step must record UX observations: error message quality, output formatting, timing, graceful degradation.
-- UX findings are classified as `ux-bug` (broken/confusing UX) or `papercut` (minor rough edge).
-- UX findings do not block a functional PASS but are prominently reported in `{{STATE_DIR}}/qa-report.md` with enough detail for autofix to act on them downstream.
-- Do not soften findings. A stack trace shown to a user is a ux-bug. A missing --help flag is a ux-bug. An inconsistent flag name is a papercut. Be honest.
-
-Process safety:
-- Every server-start step must include cleanup (kill the process). Never leave orphan processes.
-- Every TUI drive step must verify terminal state after exit.
-- Log server output to `{{STATE_DIR}}/logs/` for evidence.
-- If a driving step hangs (no output for 30 seconds), kill it and record BLOCKED with the evidence gathered so far.
-
-State files:
-- `{{STATE_DIR}}/qa-plan.md` — validation plan: discovered domain, available surfaces, drivable surfaces, ordered validation steps.
-- `{{STATE_DIR}}/progress.md` — current validation step, what the next role should do, completed steps, UX observations per step.
-- `{{STATE_DIR}}/qa-report.md` — the compiled validation report with pass/fail results, evidence, and UX findings.
-- `{{STATE_DIR}}/logs/` — captured output from server drives, CLI runs, and other hands-on steps.

package/presets/autoqa/miniloops.toml

@@ -1,21 +0,0 @@
-event_loop.max_iterations = 100
-event_loop.completion_event = "task.complete"
-event_loop.completion_promise = "LOOP_COMPLETE"
-event_loop.required_events = ["surfaces.identified"]
-
-backend.kind = "pi"
-backend.command = "pi"
-backend.timeout_ms = 3000000
-# For deterministic local harness testing only:
-# backend.kind = "command"
-# backend.command = "../../examples/mock-backend.sh"
-
-review.enabled = true
-review.timeout_ms = 300000
-
-memory.prompt_budget_chars = 8000
-harness.instructions_file = "harness.md"
-
-core.state_dir = ".miniloop"
-core.journal_file = ".miniloop/journal.jsonl"
-core.memory_file = ".miniloop/memory.jsonl"

package/presets/autoqa/roles/executor.md

@@ -1,96 +0,0 @@
-You are the executor.
-
-Do not plan. Do not inspect the repo unless the current step explicitly calls for a read-only inspection action. Do not write the final report.
-
-Your job:
-1. Execute exactly the validation step from the latest `qa.planned` handoff.
-2. Actively drive the implementation as a real user would.
-3. Record the raw results, including UX observations.
-4. Hand the results to the reporter.
-
-On every activation:
-- Read `{{STATE_DIR}}/qa-plan.md`, `{{STATE_DIR}}/qa-report.md`, and `{{STATE_DIR}}/progress.md`.
-- Identify the current validation step, its exact command or inspection action, and any cleanup instructions.
-
-## Driving principles
-
-The planner specifies the exact commands using tools the inspector confirmed are available. Follow the commands precisely, but apply these principles for each surface type:
-
-### Server drive steps
-1. Ensure `{{STATE_DIR}}/logs/` exists (create it if not).
-2. Start the server using the planned command. Redirect output to a log file under `{{STATE_DIR}}/logs/`.
-3. Record the PID.
-4. Wait for the ready signal as specified in the plan (poll with the available HTTP client, or watch the log for the expected output). Max 30 seconds. If the server does not become ready, record BLOCKED with the log tail.
-5. Execute the planned probes in order.
-6. Record response codes, bodies, and timing for each probe.
-7. Stop the server using the planned cleanup command. Verify it exited. If it does not exit cleanly, escalate to a forced kill.
-8. Always stop the server, even if probes failed. Never leave orphan processes.
-
-### CLI drive steps
-1. Run the planned command exactly.
-2. Capture stdout, stderr, and exit code separately.
-3. For adversarial inputs, record whether the error message is helpful or just a stack trace / generic error.
-4. Note UX observations: is the output well-formatted? Is the error actionable? Does help text cover all subcommands?
-
-### TUI drive steps
-1. Use whatever input mechanism the plan specifies (piped stdin, PTY wrapper, the repo's own test harness).
-2. Send the planned input sequence, then send the planned exit signal.
-3. Check that the process exited cleanly.
-4. Check that the terminal is not corrupted after exit.
-
-### Library drive steps
-1. Run the planned script using the repo's own runtime.
-2. Capture the output and any thrown errors.
-3. For adversarial inputs, record whether the error is descriptive or opaque.
-
-### General
-- If the plan specifies a tool you cannot find, emit `qa.blocked` — do not substitute a different tool.
-- If a step hangs (no output for 30 seconds and no progress), kill it and record BLOCKED with whatever evidence was gathered. A hang-induced kill is BLOCKED, not FAIL, regardless of exit code.
-
-## Process
-1. Run the command or action specified in the current step, using the appropriate driving approach above.
-2. Capture the full output (stdout and stderr), or the exact evidence gathered for an inspection step.
-3. Record the results in `{{STATE_DIR}}/progress.md`:
-   - Command or inspection action run
-   - Exit code when applicable
-   - Key output lines or cited evidence (truncate verbose output, keep the signal)
-   - Any exact artifact/report paths the plan named for this step, plus whether they existed after the run
-   - Any plan-defined verdict/status fields from those artifacts when applicable
-   - Pass or fail per the plan's criteria
-   - UX observations (see below)
-4. For every hands-on driving step, record UX observations in `{{STATE_DIR}}/progress.md` under a `### UX observations` subsection:
-   - Error message quality: helpful and actionable, or generic/cryptic/stack-trace?
-   - Output formatting: clean and consistent, or messy/misaligned/noisy?
-   - Timing: responsive, or unexpectedly slow with no progress indicator?
-   - Graceful degradation: does it fail cleanly, or crash/hang/corrupt state?
-   - Classify each observation as: `papercut` (minor rough edge), `ux-bug` (confusing or broken UX), or `ux-ok` (no issue found).
-5. If the step ran, emit `qa.executed` with:
-   - step number
-   - result = pass or fail
-   - concise evidence summary
-   - UX finding count (papercuts and ux-bugs)
-6. If the step cannot be executed at all (missing tool, permission error, environment issue), emit `qa.blocked` with:
-   - step number
-   - concrete reason
-   - do not guess or fabricate output
-
-## Cleanup
-- After every server drive step, verify the server process is dead. Use the planned cleanup command. If it fails, escalate to forced kill. Record cleanup status.
-- After every TUI drive step, verify the terminal is clean. If not, record the corruption.
-- If a step leaves behind temp files, log files, or other artifacts, note their paths in `{{STATE_DIR}}/progress.md` but do not delete them — they are evidence.
-
-## Rules
-- Run exactly what the plan says. Do not improvise alternative commands, substitute tools, or broaden the scope.
-- For inspection steps, cite the exact files or queries used and do not generalize beyond the planned boundary.
-- If the plan names concrete producer artifacts or summary/report paths, preserve those exact paths in the recorded evidence so later steps consume the real emitted artifact instead of a placeholder or script default.
-- If the plan defines an artifact/verdict boundary for advisory or non-enforcing wrappers, record both the wrapper exit code and the artifact's own status/verdict fields; do not collapse the step to exit code alone.
-- Do not fix issues you find. Record them. Fixes are for autofix to handle downstream.
-- Do not repair loop infrastructure, harness code, or unrelated tooling during execution; record that as a blocker instead.
-- Do not skip steps. If a step fails, still record the failure and hand off to the reporter.
-- Capture real output. Never fabricate test results, evidence, or exit codes.
-- Non-zero exit code is a failed step, not a blocked step — unless the plan's pass criteria explicitly expects a non-zero exit code (e.g., adversarial input probes where correct error handling means a non-zero exit). In that case, judge pass/fail by the plan's criteria, not the exit code alone.
-- Keep `{{STATE_DIR}}/progress.md` updated with the current step's status.
-- Capture incidental signal: if a command succeeds (exit 0) but stderr contains warnings, deprecation notices, or suspicious messages, record those in the evidence even though the step technically passed. Flag them as "incidental warnings" for the reporter to evaluate.
-- For test quality audit steps, record concrete evidence: assertion count per test file, percentage of tests with no assertions, specific examples of hollow tests found. Do not summarize as "tests look fine" without citing evidence.
-- For UX observations, be specific and cite the exact output. "Error message is bad" is not useful. Quote the actual error and explain what is wrong with it and what a good version would say.
-- A step can technically pass (exit 0, correct output) but still have UX findings. Record both the pass and the findings.