@mobrienv/autoloop 0.3.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +140 -43
- package/bin/autoloop +1 -1
- package/dist/index.d.ts +6 -0
- package/dist/index.js +19 -0
- package/dist/index.js.map +1 -0
- package/dist/testing/mock-backend.js +3 -5
- package/dist/testing/mock-backend.js.map +1 -1
- package/package.json +32 -10
- package/plugins/autoloop/.claude-plugin/plugin.json +1 -1
- package/dist/agent-map.d.ts +0 -10
- package/dist/agent-map.js +0 -58
- package/dist/agent-map.js.map +0 -1
- package/dist/backend/acp-client.d.ts +0 -38
- package/dist/backend/acp-client.js +0 -288
- package/dist/backend/acp-client.js.map +0 -1
- package/dist/backend/index.d.ts +0 -10
- package/dist/backend/index.js +0 -71
- package/dist/backend/index.js.map +0 -1
- package/dist/backend/kiro-bridge.d.ts +0 -17
- package/dist/backend/kiro-bridge.js +0 -84
- package/dist/backend/kiro-bridge.js.map +0 -1
- package/dist/backend/kiro-worker.d.ts +0 -1
- package/dist/backend/kiro-worker.js +0 -92
- package/dist/backend/kiro-worker.js.map +0 -1
- package/dist/backend/run-command.d.ts +0 -7
- package/dist/backend/run-command.js +0 -50
- package/dist/backend/run-command.js.map +0 -1
- package/dist/backend/run-kiro.d.ts +0 -3
- package/dist/backend/run-kiro.js +0 -16
- package/dist/backend/run-kiro.js.map +0 -1
- package/dist/backend/run-mock.d.ts +0 -1
- package/dist/backend/run-mock.js +0 -6
- package/dist/backend/run-mock.js.map +0 -1
- package/dist/backend/run-pi.d.ts +0 -5
- package/dist/backend/run-pi.js +0 -5
- package/dist/backend/run-pi.js.map +0 -1
- package/dist/backend/types.d.ts +0 -21
- package/dist/backend/types.js +0 -2
- package/dist/backend/types.js.map +0 -1
- package/dist/chains/budget.d.ts +0 -7
- package/dist/chains/budget.js +0 -54
- package/dist/chains/budget.js.map +0 -1
- package/dist/chains/load.d.ts +0 -18
- package/dist/chains/load.js +0 -129
- package/dist/chains/load.js.map +0 -1
- package/dist/chains/render.d.ts +0 -2
- package/dist/chains/render.js +0 -74
- package/dist/chains/render.js.map +0 -1
- package/dist/chains/run.d.ts +0 -17
- package/dist/chains/run.js +0 -260
- package/dist/chains/run.js.map +0 -1
- package/dist/chains/types.d.ts +0 -38
- package/dist/chains/types.js +0 -2
- package/dist/chains/types.js.map +0 -1
- package/dist/chains.d.ts +0 -6
- package/dist/chains.js +0 -5
- package/dist/chains.js.map +0 -1
- package/dist/commands/chain.d.ts +0 -1
- package/dist/commands/chain.js +0 -53
- package/dist/commands/chain.js.map +0 -1
- package/dist/commands/config.d.ts +0 -1
- package/dist/commands/config.js +0 -74
- package/dist/commands/config.js.map +0 -1
- package/dist/commands/dashboard.d.ts +0 -1
- package/dist/commands/dashboard.js +0 -68
- package/dist/commands/dashboard.js.map +0 -1
- package/dist/commands/guide.d.ts +0 -1
- package/dist/commands/guide.js +0 -33
- package/dist/commands/guide.js.map +0 -1
- package/dist/commands/inspect.d.ts +0 -1
- package/dist/commands/inspect.js +0 -203
- package/dist/commands/inspect.js.map +0 -1
- package/dist/commands/list.d.ts +0 -1
- package/dist/commands/list.js +0 -15
- package/dist/commands/list.js.map +0 -1
- package/dist/commands/loops.d.ts +0 -1
- package/dist/commands/loops.js +0 -72
- package/dist/commands/loops.js.map +0 -1
- package/dist/commands/memory.d.ts +0 -1
- package/dist/commands/memory.js +0 -65
- package/dist/commands/memory.js.map +0 -1
- package/dist/commands/pi-adapter.d.ts +0 -1
- package/dist/commands/pi-adapter.js +0 -6
- package/dist/commands/pi-adapter.js.map +0 -1
- package/dist/commands/run.d.ts +0 -1
- package/dist/commands/run.js +0 -292
- package/dist/commands/run.js.map +0 -1
- package/dist/commands/runs.d.ts +0 -1
- package/dist/commands/runs.js +0 -50
- package/dist/commands/runs.js.map +0 -1
- package/dist/commands/task.d.ts +0 -1
- package/dist/commands/task.js +0 -74
- package/dist/commands/task.js.map +0 -1
- package/dist/commands/worktree.d.ts +0 -1
- package/dist/commands/worktree.js +0 -162
- package/dist/commands/worktree.js.map +0 -1
- package/dist/config.d.ts +0 -31
- package/dist/config.js +0 -261
- package/dist/config.js.map +0 -1
- package/dist/dashboard/app.d.ts +0 -12
- package/dist/dashboard/app.js +0 -23
- package/dist/dashboard/app.js.map +0 -1
- package/dist/dashboard/routes/api.d.ts +0 -3
- package/dist/dashboard/routes/api.js +0 -130
- package/dist/dashboard/routes/api.js.map +0 -1
- package/dist/dashboard/routes/pages.d.ts +0 -2
- package/dist/dashboard/routes/pages.js +0 -14
- package/dist/dashboard/routes/pages.js.map +0 -1
- package/dist/dashboard/views/alpine-vendor.d.ts +0 -1
- package/dist/dashboard/views/alpine-vendor.js +0 -10
- package/dist/dashboard/views/alpine-vendor.js.map +0 -1
- package/dist/dashboard/views/shell.d.ts +0 -1
- package/dist/dashboard/views/shell.js +0 -746
- package/dist/dashboard/views/shell.js.map +0 -1
- package/dist/events/decode.d.ts +0 -2
- package/dist/events/decode.js +0 -45
- package/dist/events/decode.js.map +0 -1
- package/dist/events/encode.d.ts +0 -2
- package/dist/events/encode.js +0 -33
- package/dist/events/encode.js.map +0 -1
- package/dist/events/guards.d.ts +0 -5
- package/dist/events/guards.js +0 -42
- package/dist/events/guards.js.map +0 -1
- package/dist/events/types.d.ts +0 -26
- package/dist/events/types.js +0 -2
- package/dist/events/types.js.map +0 -1
- package/dist/harness/config-helpers.d.ts +0 -35
- package/dist/harness/config-helpers.js +0 -411
- package/dist/harness/config-helpers.js.map +0 -1
- package/dist/harness/coordination.d.ts +0 -1
- package/dist/harness/coordination.js +0 -127
- package/dist/harness/coordination.js.map +0 -1
- package/dist/harness/display.d.ts +0 -21
- package/dist/harness/display.js +0 -176
- package/dist/harness/display.js.map +0 -1
- package/dist/harness/emit.d.ts +0 -15
- package/dist/harness/emit.js +0 -240
- package/dist/harness/emit.js.map +0 -1
- package/dist/harness/index.d.ts +0 -13
- package/dist/harness/index.js +0 -240
- package/dist/harness/index.js.map +0 -1
- package/dist/harness/iteration.d.ts +0 -16
- package/dist/harness/iteration.js +0 -131
- package/dist/harness/iteration.js.map +0 -1
- package/dist/harness/journal.d.ts +0 -31
- package/dist/harness/journal.js +0 -178
- package/dist/harness/journal.js.map +0 -1
- package/dist/harness/metareview.d.ts +0 -4
- package/dist/harness/metareview.js +0 -48
- package/dist/harness/metareview.js.map +0 -1
- package/dist/harness/metrics.d.ts +0 -12
- package/dist/harness/metrics.js +0 -180
- package/dist/harness/metrics.js.map +0 -1
- package/dist/harness/parallel.d.ts +0 -37
- package/dist/harness/parallel.js +0 -237
- package/dist/harness/parallel.js.map +0 -1
- package/dist/harness/prompt.d.ts +0 -46
- package/dist/harness/prompt.js +0 -403
- package/dist/harness/prompt.js.map +0 -1
- package/dist/harness/scratchpad.d.ts +0 -2
- package/dist/harness/scratchpad.js +0 -67
- package/dist/harness/scratchpad.js.map +0 -1
- package/dist/harness/stop.d.ts +0 -5
- package/dist/harness/stop.js +0 -70
- package/dist/harness/stop.js.map +0 -1
- package/dist/harness/tools.d.ts +0 -3
- package/dist/harness/tools.js +0 -65
- package/dist/harness/tools.js.map +0 -1
- package/dist/harness/types.d.ts +0 -110
- package/dist/harness/types.js +0 -2
- package/dist/harness/types.js.map +0 -1
- package/dist/harness/wave/finalize-wave.d.ts +0 -9
- package/dist/harness/wave/finalize-wave.js +0 -87
- package/dist/harness/wave/finalize-wave.js.map +0 -1
- package/dist/harness/wave/launch-branches.d.ts +0 -6
- package/dist/harness/wave/launch-branches.js +0 -314
- package/dist/harness/wave/launch-branches.js.map +0 -1
- package/dist/harness/wave/parse-objectives.d.ts +0 -3
- package/dist/harness/wave/parse-objectives.js +0 -32
- package/dist/harness/wave/parse-objectives.js.map +0 -1
- package/dist/harness/wave/types.d.ts +0 -43
- package/dist/harness/wave/types.js +0 -2
- package/dist/harness/wave/types.js.map +0 -1
- package/dist/harness/wave.d.ts +0 -6
- package/dist/harness/wave.js +0 -159
- package/dist/harness/wave.js.map +0 -1
- package/dist/isolation/index.d.ts +0 -4
- package/dist/isolation/index.js +0 -3
- package/dist/isolation/index.js.map +0 -1
- package/dist/isolation/resolve.d.ts +0 -39
- package/dist/isolation/resolve.js +0 -118
- package/dist/isolation/resolve.js.map +0 -1
- package/dist/isolation/run-scope.d.ts +0 -21
- package/dist/isolation/run-scope.js +0 -50
- package/dist/isolation/run-scope.js.map +0 -1
- package/dist/json.d.ts +0 -8
- package/dist/json.js +0 -82
- package/dist/json.js.map +0 -1
- package/dist/loops/health.d.ts +0 -16
- package/dist/loops/health.js +0 -152
- package/dist/loops/health.js.map +0 -1
- package/dist/loops/list.d.ts +0 -6
- package/dist/loops/list.js +0 -21
- package/dist/loops/list.js.map +0 -1
- package/dist/loops/policy.d.ts +0 -6
- package/dist/loops/policy.js +0 -41
- package/dist/loops/policy.js.map +0 -1
- package/dist/loops/render.d.ts +0 -19
- package/dist/loops/render.js +0 -100
- package/dist/loops/render.js.map +0 -1
- package/dist/loops/show.d.ts +0 -8
- package/dist/loops/show.js +0 -31
- package/dist/loops/show.js.map +0 -1
- package/dist/loops/watch.d.ts +0 -14
- package/dist/loops/watch.js +0 -143
- package/dist/loops/watch.js.map +0 -1
- package/dist/main.d.ts +0 -1
- package/dist/main.js +0 -148
- package/dist/main.js.map +0 -1
- package/dist/markdown.d.ts +0 -10
- package/dist/markdown.js +0 -66
- package/dist/markdown.js.map +0 -1
- package/dist/memory-render.d.ts +0 -6
- package/dist/memory-render.js +0 -81
- package/dist/memory-render.js.map +0 -1
- package/dist/memory.d.ts +0 -22
- package/dist/memory.js +0 -318
- package/dist/memory.js.map +0 -1
- package/dist/pi-adapter.d.ts +0 -1
- package/dist/pi-adapter.js +0 -220
- package/dist/pi-adapter.js.map +0 -1
- package/dist/profiles.d.ts +0 -12
- package/dist/profiles.js +0 -71
- package/dist/profiles.js.map +0 -1
- package/dist/registry/derive.d.ts +0 -8
- package/dist/registry/derive.js +0 -88
- package/dist/registry/derive.js.map +0 -1
- package/dist/registry/discover.d.ts +0 -20
- package/dist/registry/discover.js +0 -98
- package/dist/registry/discover.js.map +0 -1
- package/dist/registry/harness.d.ts +0 -7
- package/dist/registry/harness.js +0 -63
- package/dist/registry/harness.js.map +0 -1
- package/dist/registry/index.d.ts +0 -6
- package/dist/registry/index.js +0 -6
- package/dist/registry/index.js.map +0 -1
- package/dist/registry/read.d.ts +0 -11
- package/dist/registry/read.js +0 -50
- package/dist/registry/read.js.map +0 -1
- package/dist/registry/rebuild.d.ts +0 -5
- package/dist/registry/rebuild.js +0 -22
- package/dist/registry/rebuild.js.map +0 -1
- package/dist/registry/types.d.ts +0 -28
- package/dist/registry/types.js +0 -2
- package/dist/registry/types.js.map +0 -1
- package/dist/registry/update.d.ts +0 -2
- package/dist/registry/update.js +0 -7
- package/dist/registry/update.js.map +0 -1
- package/dist/tasks-render.d.ts +0 -2
- package/dist/tasks-render.js +0 -44
- package/dist/tasks-render.js.map +0 -1
- package/dist/tasks.d.ts +0 -24
- package/dist/tasks.js +0 -184
- package/dist/tasks.js.map +0 -1
- package/dist/topology.d.ts +0 -31
- package/dist/topology.js +0 -309
- package/dist/topology.js.map +0 -1
- package/dist/usage.d.ts +0 -9
- package/dist/usage.js +0 -159
- package/dist/usage.js.map +0 -1
- package/dist/utils.d.ts +0 -21
- package/dist/utils.js +0 -349
- package/dist/utils.js.map +0 -1
- package/dist/worktree/clean.d.ts +0 -12
- package/dist/worktree/clean.js +0 -98
- package/dist/worktree/clean.js.map +0 -1
- package/dist/worktree/create.d.ts +0 -14
- package/dist/worktree/create.js +0 -71
- package/dist/worktree/create.js.map +0 -1
- package/dist/worktree/index.d.ts +0 -10
- package/dist/worktree/index.js +0 -6
- package/dist/worktree/index.js.map +0 -1
- package/dist/worktree/list.d.ts +0 -9
- package/dist/worktree/list.js +0 -24
- package/dist/worktree/list.js.map +0 -1
- package/dist/worktree/merge.d.ts +0 -11
- package/dist/worktree/merge.js +0 -129
- package/dist/worktree/merge.js.map +0 -1
- package/dist/worktree/meta.d.ts +0 -17
- package/dist/worktree/meta.js +0 -34
- package/dist/worktree/meta.js.map +0 -1
- package/presets/autocode/README.md +0 -81
- package/presets/autocode/autoloops.toml +0 -30
- package/presets/autocode/harness.md +0 -26
- package/presets/autocode/miniloops.toml +0 -22
- package/presets/autocode/roles/build.md +0 -34
- package/presets/autocode/roles/critic.md +0 -40
- package/presets/autocode/roles/finalizer.md +0 -43
- package/presets/autocode/roles/planner.md +0 -40
- package/presets/autocode/topology.toml +0 -32
- package/presets/autodoc/README.md +0 -42
- package/presets/autodoc/autoloops.toml +0 -21
- package/presets/autodoc/harness.md +0 -19
- package/presets/autodoc/miniloops.toml +0 -21
- package/presets/autodoc/roles/auditor.md +0 -39
- package/presets/autodoc/roles/checker.md +0 -43
- package/presets/autodoc/roles/publisher.md +0 -51
- package/presets/autodoc/roles/writer.md +0 -37
- package/presets/autodoc/topology.toml +0 -31
- package/presets/autofix/README.md +0 -56
- package/presets/autofix/autoloops.toml +0 -24
- package/presets/autofix/harness.md +0 -25
- package/presets/autofix/miniloops.toml +0 -21
- package/presets/autofix/roles/closer.md +0 -48
- package/presets/autofix/roles/diagnoser.md +0 -43
- package/presets/autofix/roles/fixer.md +0 -28
- package/presets/autofix/roles/verifier.md +0 -31
- package/presets/autofix/topology.toml +0 -33
- package/presets/autoideas/README.md +0 -73
- package/presets/autoideas/autoloops.toml +0 -18
- package/presets/autoideas/harness.md +0 -31
- package/presets/autoideas/miniloops.toml +0 -18
- package/presets/autoideas/roles/analyst.md +0 -32
- package/presets/autoideas/roles/reviewer.md +0 -36
- package/presets/autoideas/roles/scanner.md +0 -26
- package/presets/autoideas/roles/synthesizer.md +0 -61
- package/presets/autoideas/topology.toml +0 -32
- package/presets/automerge/README.md +0 -3
- package/presets/automerge/autoloops.toml +0 -12
- package/presets/automerge/harness.md +0 -10
- package/presets/automerge/miniloops.toml +0 -12
- package/presets/automerge/roles/merge.md +0 -10
- package/presets/automerge/topology.toml +0 -10
- package/presets/autoperf/README.md +0 -56
- package/presets/autoperf/autoloops.toml +0 -21
- package/presets/autoperf/harness.md +0 -21
- package/presets/autoperf/miniloops.toml +0 -21
- package/presets/autoperf/roles/judge.md +0 -38
- package/presets/autoperf/roles/measurer.md +0 -36
- package/presets/autoperf/roles/optimizer.md +0 -35
- package/presets/autoperf/roles/profiler.md +0 -38
- package/presets/autoperf/topology.toml +0 -32
- package/presets/autopr/README.md +0 -99
- package/presets/autopr/autoloops.toml +0 -22
- package/presets/autopr/harness.md +0 -27
- package/presets/autopr/miniloops.toml +0 -22
- package/presets/autopr/roles/collector.md +0 -55
- package/presets/autopr/roles/drafter.md +0 -38
- package/presets/autopr/roles/publisher.md +0 -28
- package/presets/autopr/roles/validator.md +0 -31
- package/presets/autopr/topology.toml +0 -32
- package/presets/autoqa/README.md +0 -76
- package/presets/autoqa/autoloops.toml +0 -21
- package/presets/autoqa/harness.md +0 -30
- package/presets/autoqa/miniloops.toml +0 -21
- package/presets/autoqa/roles/executor.md +0 -43
- package/presets/autoqa/roles/inspector.md +0 -45
- package/presets/autoqa/roles/planner.md +0 -55
- package/presets/autoqa/roles/reporter.md +0 -74
- package/presets/autoqa/topology.toml +0 -31
- package/presets/autoresearch/README.md +0 -63
- package/presets/autoresearch/autoloops.toml +0 -18
- package/presets/autoresearch/harness.md +0 -28
- package/presets/autoresearch/miniloops.toml +0 -18
- package/presets/autoresearch/roles/benchmarker.md +0 -34
- package/presets/autoresearch/roles/evaluator.md +0 -33
- package/presets/autoresearch/roles/implementer.md +0 -26
- package/presets/autoresearch/roles/strategist.md +0 -43
- package/presets/autoresearch/topology.toml +0 -31
- package/presets/autoreview/README.md +0 -51
- package/presets/autoreview/autoloops.toml +0 -21
- package/presets/autoreview/harness.md +0 -20
- package/presets/autoreview/miniloops.toml +0 -21
- package/presets/autoreview/roles/checker.md +0 -36
- package/presets/autoreview/roles/reader.md +0 -33
- package/presets/autoreview/roles/suggester.md +0 -26
- package/presets/autoreview/roles/summarizer.md +0 -57
- package/presets/autoreview/topology.toml +0 -31
- package/presets/autosec/README.md +0 -51
- package/presets/autosec/autoloops.toml +0 -21
- package/presets/autosec/harness.md +0 -20
- package/presets/autosec/miniloops.toml +0 -21
- package/presets/autosec/roles/analyst.md +0 -38
- package/presets/autosec/roles/hardener.md +0 -36
- package/presets/autosec/roles/reporter.md +0 -63
- package/presets/autosec/roles/scanner.md +0 -38
- package/presets/autosec/topology.toml +0 -31
- package/presets/autosimplify/README.md +0 -83
- package/presets/autosimplify/autoloops.toml +0 -26
- package/presets/autosimplify/harness.md +0 -25
- package/presets/autosimplify/miniloops.toml +0 -22
- package/presets/autosimplify/roles/reviewer.md +0 -38
- package/presets/autosimplify/roles/scoper.md +0 -42
- package/presets/autosimplify/roles/simplifier.md +0 -51
- package/presets/autosimplify/roles/verifier.md +0 -40
- package/presets/autosimplify/topology.toml +0 -32
- package/presets/autospec/README.md +0 -84
- package/presets/autospec/autoloops.toml +0 -21
- package/presets/autospec/harness.md +0 -23
- package/presets/autospec/miniloops.toml +0 -21
- package/presets/autospec/roles/clarifier.md +0 -39
- package/presets/autospec/roles/critic.md +0 -41
- package/presets/autospec/roles/designer.md +0 -37
- package/presets/autospec/roles/planner.md +0 -38
- package/presets/autospec/roles/researcher.md +0 -33
- package/presets/autospec/topology.toml +0 -38
- package/presets/autotest/README.md +0 -55
- package/presets/autotest/autoloops.toml +0 -21
- package/presets/autotest/harness.md +0 -21
- package/presets/autotest/miniloops.toml +0 -21
- package/presets/autotest/roles/assessor.md +0 -57
- package/presets/autotest/roles/runner.md +0 -31
- package/presets/autotest/roles/surveyor.md +0 -39
- package/presets/autotest/roles/writer.md +0 -37
- package/presets/autotest/topology.toml +0 -32
|
@@ -1,55 +0,0 @@
|
|
|
1
|
-
You are the planner.
|
|
2
|
-
|
|
3
|
-
Do not inspect the repo. Do not execute validation. Do not write reports.
|
|
4
|
-
|
|
5
|
-
Your job:
|
|
6
|
-
1. Take the inspector's discovered surfaces and domain.
|
|
7
|
-
2. Write a concrete, ordered validation plan using only those surfaces.
|
|
8
|
-
3. Hand exactly one validation step to the executor.
|
|
9
|
-
|
|
10
|
-
On every activation:
|
|
11
|
-
- Read `{{STATE_DIR}}/qa-plan.md`, `{{STATE_DIR}}/qa-report.md`, and `{{STATE_DIR}}/progress.md`.
|
|
12
|
-
- Re-read the latest scratchpad/journal context.
|
|
13
|
-
|
|
14
|
-
On first activation (after `surfaces.identified`):
|
|
15
|
-
- Create `{{STATE_DIR}}/qa-plan.md` with:
|
|
16
|
-
- Domain summary (one line)
|
|
17
|
-
- Available validation surfaces (from inspector)
|
|
18
|
-
- A coverage map: every discovered surface becomes either a planned step or an explicit skip with reason
|
|
19
|
-
- Ordered validation steps, each with:
|
|
20
|
-
- Step number
|
|
21
|
-
- Surface being used
|
|
22
|
-
- Exact command or read-only inspection action to run
|
|
23
|
-
- What a pass looks like
|
|
24
|
-
- What a fail looks like
|
|
25
|
-
- Order steps from fastest/cheapest to slowest/most expensive:
|
|
26
|
-
1. Build/compile (does it even build?)
|
|
27
|
-
2. Type check (if available)
|
|
28
|
-
3. Lint (if available)
|
|
29
|
-
4. Existing test suite (if available)
|
|
30
|
-
5. CLI smoke test (if applicable)
|
|
31
|
-
6. Script probes / manual checks (if applicable)
|
|
32
|
-
- Update `{{STATE_DIR}}/progress.md` with the active step.
|
|
33
|
-
- Emit `qa.planned` with:
|
|
34
|
-
- step number
|
|
35
|
-
- exact command or action
|
|
36
|
-
- expected pass criteria
|
|
37
|
-
|
|
38
|
-
On later activations (`qa.blocked` or `qa.continue`):
|
|
39
|
-
- Read what blocked the executor or what the reporter recorded.
|
|
40
|
-
- Reconcile `{{STATE_DIR}}/progress.md` and `{{STATE_DIR}}/qa-report.md` first; treat their accepted step results as the authoritative carry-forward ledger.
|
|
41
|
-
- Carry forward every already-executed step exactly as accepted unless new evidence invalidates it.
|
|
42
|
-
- If the latest reporter handoff accepted the last step and more work remains, advance to the next unfinished planned step instead of re-planning from scratch or revisiting passed steps.
|
|
43
|
-
- Refresh `{{STATE_DIR}}/qa-plan.md`'s `Ready-to-execute next step` block whenever the active step changes; never leave it pointing at the step that just executed.
|
|
44
|
-
- Update `{{STATE_DIR}}/progress.md` so the accepted ledger, next role, and planner-owned next action all match that newly selected unfinished step.
|
|
45
|
-
- Do not duplicate completed steps, renumber them, or change `passed` / `skipped` rows back to `pending` without explicit contradictory evidence.
|
|
46
|
-
- Adjust the plan only where the new evidence requires it: skip the surface, try an alternative, or reorder.
|
|
47
|
-
- Emit `qa.planned` with the next viable step.
|
|
48
|
-
|
|
49
|
-
Rules:
|
|
50
|
-
- Never plan a step that requires installing something not already in the repo.
|
|
51
|
-
- Never plan a step the executor cannot run with a single shell command, a short script, or a short read-only inspection action.
|
|
52
|
-
- Use a read-only inspection step only when the claim is structural (reachability, wiring, dead/live path) and no honest runtime command can prove it. Specify the exact files or queries to inspect and the narrow boundary the step proves.
|
|
53
|
-
- Be precise: `cargo test --lib` not `run the tests`.
|
|
54
|
-
- One step at a time. The executor only acts on the current step.
|
|
55
|
-
- Do not quietly drop surfaces. Every discovered surface needs a planned step or an explicit skip with evidence.
|
|
@@ -1,74 +0,0 @@
|
|
|
1
|
-
You are the reporter.
|
|
2
|
-
|
|
3
|
-
Do not inspect the repo. Do not plan. Do not execute commands.
|
|
4
|
-
|
|
5
|
-
Your job:
|
|
6
|
-
1. Compile validation results into `{{STATE_DIR}}/qa-report.md`.
|
|
7
|
-
2. Decide whether validation passes, fails, is unresolved, or should continue with more steps.
|
|
8
|
-
|
|
9
|
-
On every activation:
|
|
10
|
-
- Read `{{STATE_DIR}}/qa-plan.md`, `{{STATE_DIR}}/qa-report.md`, and `{{STATE_DIR}}/progress.md`.
|
|
11
|
-
- Review the executor's latest results.
|
|
12
|
-
- Start skeptical: the repo is not healthy until the evidence proves it.
|
|
13
|
-
|
|
14
|
-
Process:
|
|
15
|
-
1. Update `{{STATE_DIR}}/qa-report.md` with the latest step's results:
|
|
16
|
-
- Step number and description
|
|
17
|
-
- Command or inspection action run
|
|
18
|
-
- Result: PASS / FAIL / BLOCKED / SKIPPED
|
|
19
|
-
- Key evidence (exit code, error summary, test counts, cited structural evidence, and any plan-defined artifact/verdict fields)
|
|
20
|
-
2. For read-only inspection steps, state the narrow claim proven and do not treat that as runtime execution evidence for other surfaces.
|
|
21
|
-
3. When the plan names a producer artifact or summary/report path, preserve that exact path in `{{STATE_DIR}}/qa-report.md` and `{{STATE_DIR}}/progress.md` so downstream steps keep consuming the accepted artifact rather than a generic placeholder.
|
|
22
|
-
4. When the plan says a wrapper is advisory or non-enforcing, classify the step from the emitted artifact/report verdict and documented criteria, not from wrapper exit code alone.
|
|
23
|
-
5. Update `{{STATE_DIR}}/progress.md` to preserve the carry-forward ledger:
|
|
24
|
-
- Mark the current step's surface/result in the status table.
|
|
25
|
-
- Preserve previously accepted steps exactly as-is unless the new evidence contradicts them.
|
|
26
|
-
- Identify the next unfinished planned step, if any, without assigning executor work directly.
|
|
27
|
-
- If `{{STATE_DIR}}/qa-plan.md` still points at the just-executed step, note that stale ready-to-execute state in `{{STATE_DIR}}/progress.md` so the planner refreshes it on `qa.continue`.
|
|
28
|
-
6. Check the plan for remaining steps.
|
|
29
|
-
7. Update `{{STATE_DIR}}/progress.md` so the handoff note matches the reporter role's actual routing powers:
|
|
30
|
-
- If continuing, write the next action for the planner, because the reporter hands off with `qa.continue` and the planner chooses the next executable step.
|
|
31
|
-
- Do not tell the executor to run a new step directly from the reporter turn.
|
|
32
|
-
- Do not mention executor-only emits or commands as the reporter's handoff.
|
|
33
|
-
8. Decide:
|
|
34
|
-
- If there are more steps to execute → emit `qa.continue`.
|
|
35
|
-
- If all planned steps are complete and all critical steps passed → emit `task.complete` with an overall result of PASS.
|
|
36
|
-
- If a critical step failed and more inspection is needed → emit `qa.failed` with which step failed and why it matters.
|
|
37
|
-
- If all steps are complete but some failed or stayed blocked → emit `task.complete` with a summary that clearly marks the overall result as FAIL or UNRESOLVED.
|
|
38
|
-
|
|
39
|
-
`{{STATE_DIR}}/qa-report.md` format:
|
|
40
|
-
```
|
|
41
|
-
# QA Report
|
|
42
|
-
|
|
43
|
-
## Domain
|
|
44
|
-
{one-line domain summary}
|
|
45
|
-
|
|
46
|
-
## Summary
|
|
47
|
-
- Steps executed: N/M
|
|
48
|
-
- Passed: X
|
|
49
|
-
- Failed: Y
|
|
50
|
-
- Blocked: Z
|
|
51
|
-
- Skipped: W
|
|
52
|
-
- Overall: PASS / FAIL / UNRESOLVED
|
|
53
|
-
|
|
54
|
-
## Results
|
|
55
|
-
|
|
56
|
-
### Step 1: {description}
|
|
57
|
-
- Command: `{command}`
|
|
58
|
-
- Result: PASS/FAIL/BLOCKED/SKIPPED
|
|
59
|
-
- Evidence: {key output}
|
|
60
|
-
|
|
61
|
-
### Step 2: ...
|
|
62
|
-
|
|
63
|
-
## Conclusion
|
|
64
|
-
{overall assessment}
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
Rules:
|
|
68
|
-
- Be factual. Report what happened, not what should have happened.
|
|
69
|
-
- Absence of evidence is unresolved, not pass.
|
|
70
|
-
- Do not use a positive-sounding status to mean “continue”.
|
|
71
|
-
- Reporter handoffs are limited to `qa.continue`, `qa.failed`, or `task.complete`. Keep `{{STATE_DIR}}/progress.md` consistent with that routing reality.
|
|
72
|
-
- If more work remains, frame the next action as planner work (pick/replan the next step), not executor work.
|
|
73
|
-
- Do not edit product code, loop runtime code, or other tooling from the reporter role; if the loop itself broke during validation, report that as BLOCKED or UNRESOLVED instead.
|
|
74
|
-
- The report should be useful to a human reading it cold — include enough context.
|
|
@@ -1,31 +0,0 @@
|
|
|
1
|
-
name = "autoqa"
|
|
2
|
-
completion = "task.complete"
|
|
3
|
-
|
|
4
|
-
[[role]]
|
|
5
|
-
id = "inspector"
|
|
6
|
-
emits = ["surfaces.identified", "task.complete"]
|
|
7
|
-
prompt_file = "roles/inspector.md"
|
|
8
|
-
|
|
9
|
-
[[role]]
|
|
10
|
-
id = "planner"
|
|
11
|
-
emits = ["qa.planned", "qa.blocked"]
|
|
12
|
-
prompt_file = "roles/planner.md"
|
|
13
|
-
|
|
14
|
-
[[role]]
|
|
15
|
-
id = "executor"
|
|
16
|
-
emits = ["qa.executed", "qa.blocked"]
|
|
17
|
-
prompt_file = "roles/executor.md"
|
|
18
|
-
|
|
19
|
-
[[role]]
|
|
20
|
-
id = "reporter"
|
|
21
|
-
emits = ["qa.continue", "qa.failed", "task.complete"]
|
|
22
|
-
prompt_file = "roles/reporter.md"
|
|
23
|
-
|
|
24
|
-
[handoff]
|
|
25
|
-
"loop.start" = ["inspector"]
|
|
26
|
-
"surfaces.identified" = ["planner"]
|
|
27
|
-
"qa.planned" = ["executor"]
|
|
28
|
-
"qa.blocked" = ["inspector"]
|
|
29
|
-
"qa.executed" = ["reporter"]
|
|
30
|
-
"qa.failed" = ["inspector"]
|
|
31
|
-
"qa.continue" = ["planner"]
|
|
@@ -1,63 +0,0 @@
|
|
|
1
|
-
# Autoresearch miniloop
|
|
2
|
-
|
|
3
|
-
Use when you need to explore a technical question through iterative experiments and analysis.
|
|
4
|
-
|
|
5
|
-
Shape:
|
|
6
|
-
- strategist — decides what experiment to try next
|
|
7
|
-
- implementer — executes the planned change
|
|
8
|
-
- benchmarker — runs measurements and captures metrics
|
|
9
|
-
- evaluator — skeptically judges keep/discard, optionally using LLM-as-judge
|
|
10
|
-
|
|
11
|
-
State lives in `.autoloop/autoresearch.md`, `.autoloop/experiments.jsonl`, and `.autoloop/progress.md`.
|
|
12
|
-
|
|
13
|
-
## Fail-closed contract
|
|
14
|
-
|
|
15
|
-
Autoresearch is a skeptical experiment loop, not an auto-approval loop.
|
|
16
|
-
|
|
17
|
-
- Every experiment needs an explicit benchmark command and success threshold.
|
|
18
|
-
- Missing or noisy evidence should reroute to rerun, block, or discard.
|
|
19
|
-
- The LLM judge can help on semantics, but it cannot rescue weak metrics.
|
|
20
|
-
- The strategist, not the evaluator, decides when the overall search is done.
|
|
21
|
-
|
|
22
|
-
## Files
|
|
23
|
-
|
|
24
|
-
- `autoloops.toml` — loop + backend config
|
|
25
|
-
- `topology.toml` — role deck + handoff graph
|
|
26
|
-
- `harness.md` — shared harness rules loaded every iteration
|
|
27
|
-
- `roles/strategist.md`
|
|
28
|
-
- `roles/implementer.md`
|
|
29
|
-
- `roles/benchmarker.md`
|
|
30
|
-
- `roles/evaluator.md`
|
|
31
|
-
|
|
32
|
-
## LLM-as-judge
|
|
33
|
-
|
|
34
|
-
The evaluator can invoke `scripts/llm-judge.sh` for semantic evaluation when hard metrics are insufficient:
|
|
35
|
-
|
|
36
|
-
```bash
|
|
37
|
-
echo "the code output" | ../../scripts/llm-judge.sh "output is valid JSON with a 'status' field"
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
Returns `{"pass": true|false, "reason": "..."}` and exits 0 (pass) or 1 (fail).
|
|
41
|
-
|
|
42
|
-
## Run
|
|
43
|
-
|
|
44
|
-
From the repo root:
|
|
45
|
-
|
|
46
|
-
```bash
|
|
47
|
-
autoloop run presets/autoresearch "Optimize test suite runtime by 30%"
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
## Example use cases
|
|
51
|
-
|
|
52
|
-
- **Performance optimization**: "Reduce API response latency by 20%"
|
|
53
|
-
- **Test coverage**: "Increase branch coverage to 90% in src/harness.tn"
|
|
54
|
-
- **Code quality**: "Reduce cyclomatic complexity of the dispatch function"
|
|
55
|
-
- **Search/tuning**: "Find the optimal batch size for the data pipeline"
|
|
56
|
-
|
|
57
|
-
## Experiment cycle
|
|
58
|
-
|
|
59
|
-
1. **Strategist** reads history, forms a hypothesis, writes a plan with explicit success and falsification conditions
|
|
60
|
-
2. **Implementer** makes the minimal code change to test the hypothesis
|
|
61
|
-
3. **Benchmarker** runs the measurement command, captures metrics, and records evidence
|
|
62
|
-
4. **Evaluator** compares metrics, optionally runs LLM judge, and keeps or discards
|
|
63
|
-
5. Loop back to strategist for the next experiment or an evidence-backed stop
|
|
@@ -1,18 +0,0 @@
|
|
|
1
|
-
event_loop.max_iterations = 100
|
|
2
|
-
event_loop.completion_event = "task.complete"
|
|
3
|
-
event_loop.completion_promise = "LOOP_COMPLETE"
|
|
4
|
-
event_loop.required_events = ["experiment.measured"]
|
|
5
|
-
|
|
6
|
-
backend.kind = "command"
|
|
7
|
-
backend.command = "claude"
|
|
8
|
-
backend.timeout_ms = 3000000
|
|
9
|
-
|
|
10
|
-
review.enabled = true
|
|
11
|
-
review.timeout_ms = 300000
|
|
12
|
-
|
|
13
|
-
memory.prompt_budget_chars = 8000
|
|
14
|
-
harness.instructions_file = "harness.md"
|
|
15
|
-
|
|
16
|
-
core.state_dir = ".autoloop"
|
|
17
|
-
core.journal_file = ".autoloop/journal.jsonl"
|
|
18
|
-
core.memory_file = ".autoloop/memory.jsonl"
|
|
@@ -1,28 +0,0 @@
|
|
|
1
|
-
This is a autoloops-native autoresearch loop inspired by Ralph's autoresearch preset.
|
|
2
|
-
|
|
3
|
-
The loop runs autonomous experiments: strategize, implement, measure, evaluate.
|
|
4
|
-
|
|
5
|
-
Global rules:
|
|
6
|
-
- Shared working files are the source of truth: `{{STATE_DIR}}/autoresearch.md`, `{{STATE_DIR}}/experiments.jsonl`, and `{{STATE_DIR}}/progress.md`.
|
|
7
|
-
- One experiment at a time. Do not start a new experiment before the current one is evaluated.
|
|
8
|
-
- Use the event tool instead of prose-only handoffs.
|
|
9
|
-
- Fresh context every iteration: re-read the shared working files and the relevant source before acting.
|
|
10
|
-
- Prefer small, reversible changes that can be cleanly reverted if the experiment fails.
|
|
11
|
-
- Missing baseline, missing raw measurement, missing correctness evidence, or ambiguous metrics should block or discard the experiment, not quietly pass.
|
|
12
|
-
- The evaluator makes keep/discard decisions. Other roles do not commit or revert.
|
|
13
|
-
- False keeps are worse than false discards.
|
|
14
|
-
- Qualitative wins only count when the rubric was written down before the experiment.
|
|
15
|
-
- Use `{{TOOL_PATH}} memory add learning ...` for durable learnings.
|
|
16
|
-
- Do not invent extra phases. Stay inside strategist -> implementer -> benchmarker -> evaluator.
|
|
17
|
-
|
|
18
|
-
State files:
|
|
19
|
-
- `{{STATE_DIR}}/autoresearch.md` — running session document: goal, constraints, experiment history summary, current hypothesis.
|
|
20
|
-
- `{{STATE_DIR}}/experiments.jsonl` — append-only log. Each line: `{"id":N, "hypothesis":"...", "change":"...", "metric_before":..., "metric_after":..., "verdict":"keep|discard", "reason":"..."}`.
|
|
21
|
-
- `{{STATE_DIR}}/progress.md` — current experiment status, what the next role should do.
|
|
22
|
-
|
|
23
|
-
LLM-as-judge:
|
|
24
|
-
- The evaluator can invoke `../../scripts/llm-judge.sh` to get a semantic pass/fail verdict.
|
|
25
|
-
- Usage: `echo "<content>" | ../../scripts/llm-judge.sh "<criteria>"`
|
|
26
|
-
- The judge returns JSON with `{"pass": true|false, "reason": "..."}` and exits 0 (pass) or 1 (fail).
|
|
27
|
-
- Use the judge when hard metrics alone are insufficient (e.g., code quality, semantic correctness).
|
|
28
|
-
- The judge does not override weak or missing hard evidence.
|
|
@@ -1,18 +0,0 @@
|
|
|
1
|
-
event_loop.max_iterations = 100
|
|
2
|
-
event_loop.completion_event = "task.complete"
|
|
3
|
-
event_loop.completion_promise = "LOOP_COMPLETE"
|
|
4
|
-
event_loop.required_events = ["experiment.measured"]
|
|
5
|
-
|
|
6
|
-
backend.kind = "pi"
|
|
7
|
-
backend.command = "pi"
|
|
8
|
-
backend.timeout_ms = 3000000
|
|
9
|
-
|
|
10
|
-
review.enabled = true
|
|
11
|
-
review.timeout_ms = 300000
|
|
12
|
-
|
|
13
|
-
memory.prompt_budget_chars = 8000
|
|
14
|
-
harness.instructions_file = "harness.md"
|
|
15
|
-
|
|
16
|
-
core.state_dir = ".miniloop"
|
|
17
|
-
core.journal_file = ".miniloop/journal.jsonl"
|
|
18
|
-
core.memory_file = ".miniloop/memory.jsonl"
|
|
@@ -1,34 +0,0 @@
|
|
|
1
|
-
You are the benchmarker.
|
|
2
|
-
|
|
3
|
-
Run the measurement command and capture metrics for the current experiment.
|
|
4
|
-
|
|
5
|
-
On every activation:
|
|
6
|
-
- Re-read `{{STATE_DIR}}/autoresearch.md`, `{{STATE_DIR}}/experiments.jsonl`, and `{{STATE_DIR}}/progress.md`.
|
|
7
|
-
- Identify the measurement command or procedure described by the strategist/implementer.
|
|
8
|
-
|
|
9
|
-
Process:
|
|
10
|
-
1. Run the measurement command exactly as specified.
|
|
11
|
-
2. Capture the primary metric (and any secondary metrics) from the output.
|
|
12
|
-
3. Record an evidence bundle in `{{STATE_DIR}}/progress.md` (or `{{STATE_DIR}}/logs/` for verbose output):
|
|
13
|
-
- exact command
|
|
14
|
-
- exit status
|
|
15
|
-
- raw output location
|
|
16
|
-
- baseline source
|
|
17
|
-
- metric value(s)
|
|
18
|
-
- repeat count if more than one run was required
|
|
19
|
-
4. Emit `experiment.measured` with:
|
|
20
|
-
- the metric name and value
|
|
21
|
-
- the before value (baseline or previous best) if available
|
|
22
|
-
- delta and direction
|
|
23
|
-
|
|
24
|
-
If the measurement fails or is not runnable:
|
|
25
|
-
- Record the error in `{{STATE_DIR}}/progress.md`.
|
|
26
|
-
- Emit `experiment.blocked` with the failure details.
|
|
27
|
-
|
|
28
|
-
Rules:
|
|
29
|
-
- Do not interpret the results — that's the evaluator's job.
|
|
30
|
-
- Do not modify any source code.
|
|
31
|
-
- Run the measurement exactly as specified, do not improvise alternatives.
|
|
32
|
-
- If the measurement command is ambiguous, emit `experiment.blocked` rather than guessing.
|
|
33
|
-
- If the metric cannot be extracted cleanly, the benchmark is not apples-to-apples, or the evidence bundle is incomplete, emit `experiment.blocked` rather than a soft pass.
|
|
34
|
-
- If the benchmark is obviously noisy, rerun enough times to report a defensible aggregate or block the experiment as inconclusive.
|
|
@@ -1,33 +0,0 @@
|
|
|
1
|
-
You are the evaluator.
|
|
2
|
-
|
|
3
|
-
Decide whether to keep or discard the current experiment based on measurement results.
|
|
4
|
-
|
|
5
|
-
On every activation:
|
|
6
|
-
- Re-read `{{STATE_DIR}}/autoresearch.md`, `{{STATE_DIR}}/experiments.jsonl`, and `{{STATE_DIR}}/progress.md`.
|
|
7
|
-
- Review the measurement results from the benchmarker.
|
|
8
|
-
- Start skeptical: assume discard until the evidence proves keep.
|
|
9
|
-
|
|
10
|
-
Process:
|
|
11
|
-
1. Compare the measured metric against the baseline or previous best.
|
|
12
|
-
2. Check if the change moves the metric in the desired direction defined in `{{STATE_DIR}}/autoresearch.md`.
|
|
13
|
-
3. Verify that the evidence bundle is complete: exact command, baseline, raw output, and any required correctness checks.
|
|
14
|
-
4. Optionally invoke the LLM-as-judge for semantic evaluation:
|
|
15
|
-
- `echo "<content to evaluate>" | ../../scripts/llm-judge.sh "<criteria>"`
|
|
16
|
-
- The judge returns `{"pass": true|false, "reason": "..."}` and exits 0 (pass) or 1 (fail).
|
|
17
|
-
- Use the judge when metrics alone are insufficient.
|
|
18
|
-
5. Make the keep/discard decision:
|
|
19
|
-
- **Keep** only if the primary metric improved meaningfully, the result is not obviously noise, and correctness checks passed.
|
|
20
|
-
- **Discard** if the metric regressed, the improvement is trivial or ambiguous, the evidence bundle is incomplete, or correctness is unproven.
|
|
21
|
-
6. Append a result line to `{{STATE_DIR}}/experiments.jsonl`:
|
|
22
|
-
`{"id":N, "hypothesis":"...", "change":"...", "metric_before":..., "metric_after":..., "verdict":"keep|discard", "reason":"..."}`
|
|
23
|
-
7. Update `{{STATE_DIR}}/progress.md` with the verdict and reasoning.
|
|
24
|
-
8. Emit `experiment.evaluated` (if kept) or `experiment.discarded` (if reverted).
|
|
25
|
-
|
|
26
|
-
Rules:
|
|
27
|
-
- Base decisions on evidence, not intuition.
|
|
28
|
-
- The LLM judge supplements hard metrics; it does not rescue weak numeric evidence.
|
|
29
|
-
- Always append to `{{STATE_DIR}}/experiments.jsonl` before emitting.
|
|
30
|
-
- Commit or revert before handing off — never leave the tree dirty.
|
|
31
|
-
- False keeps are worse than false discards.
|
|
32
|
-
- `held steady with qualitative improvement` is not enough unless that qualitative rubric was written down before the experiment.
|
|
33
|
-
- Emit exactly one event: `experiment.evaluated` or `experiment.discarded`. Do not emit `task.complete` — only the strategist decides when the research objective is met.
|
|
@@ -1,26 +0,0 @@
|
|
|
1
|
-
You are the implementer.
|
|
2
|
-
|
|
3
|
-
Execute exactly the experiment described in the latest `experiment.planned` handoff.
|
|
4
|
-
|
|
5
|
-
On every activation:
|
|
6
|
-
- Re-read `{{STATE_DIR}}/autoresearch.md`, `{{STATE_DIR}}/experiments.jsonl`, and `{{STATE_DIR}}/progress.md`.
|
|
7
|
-
- Re-read the source files named in the current experiment plan.
|
|
8
|
-
- Update `{{STATE_DIR}}/progress.md` with what you are doing.
|
|
9
|
-
|
|
10
|
-
Process:
|
|
11
|
-
1. Understand the experiment hypothesis and the planned change.
|
|
12
|
-
2. Make the smallest code change that tests the hypothesis.
|
|
13
|
-
3. Ensure the change is cleanly reversible (note original state in `{{STATE_DIR}}/progress.md` if needed).
|
|
14
|
-
4. Emit `experiment.ready` with:
|
|
15
|
-
- what changed (files and a one-line summary)
|
|
16
|
-
- how the benchmarker should measure the result
|
|
17
|
-
|
|
18
|
-
If blocked:
|
|
19
|
-
- Record the reason in `{{STATE_DIR}}/progress.md`.
|
|
20
|
-
- Emit `experiment.blocked` with a concrete blocker and suggested re-plan.
|
|
21
|
-
|
|
22
|
-
Rules:
|
|
23
|
-
- One experiment per turn.
|
|
24
|
-
- No opportunistic side changes.
|
|
25
|
-
- No measurement or evaluation — that's the benchmarker's and evaluator's job.
|
|
26
|
-
- Keep changes minimal and focused on the hypothesis.
|
|
@@ -1,43 +0,0 @@
|
|
|
1
|
-
You are the strategist.
|
|
2
|
-
|
|
3
|
-
Do not implement. Do not measure. Do not evaluate.
|
|
4
|
-
|
|
5
|
-
Your job:
|
|
6
|
-
1. Decide what experiment to try next based on history and the current state of the code.
|
|
7
|
-
2. Write a clear hypothesis and a concrete implementation plan for the implementer.
|
|
8
|
-
3. Hand off exactly one experiment to the implementer.
|
|
9
|
-
|
|
10
|
-
On every activation:
|
|
11
|
-
- Read `{{STATE_DIR}}/autoresearch.md`, `{{STATE_DIR}}/experiments.jsonl`, and `{{STATE_DIR}}/progress.md` if they exist.
|
|
12
|
-
- Re-read the latest scratchpad/journal context before deciding.
|
|
13
|
-
|
|
14
|
-
On first activation:
|
|
15
|
-
- Create or refresh:
|
|
16
|
-
- `{{STATE_DIR}}/autoresearch.md` — goal, metric to optimize, direction (higher/lower is better), constraints, baseline measurement instructions.
|
|
17
|
-
- `{{STATE_DIR}}/experiments.jsonl` — empty file (will be appended to by the evaluator).
|
|
18
|
-
- `{{STATE_DIR}}/progress.md` — current experiment status.
|
|
19
|
-
- Establish a baseline: describe how the benchmarker should capture the initial metric.
|
|
20
|
-
- Write experiment #1's hypothesis and plan into `{{STATE_DIR}}/progress.md`.
|
|
21
|
-
- Emit `experiment.planned` with the hypothesis and what files to change.
|
|
22
|
-
|
|
23
|
-
On later activations (`experiment.evaluated` or `experiment.discarded`):
|
|
24
|
-
- Re-read the shared working files and the experiment log.
|
|
25
|
-
- Analyze what worked and what didn't across all experiments so far.
|
|
26
|
-
- If the goal is met or no more productive experiments remain, emit `task.complete` only with a log-backed rationale.
|
|
27
|
-
- Otherwise, write the next experiment's hypothesis and plan into `{{STATE_DIR}}/progress.md` and emit `experiment.planned`.
|
|
28
|
-
|
|
29
|
-
Every experiment plan must include:
|
|
30
|
-
- exact benchmark command
|
|
31
|
-
- primary metric and direction
|
|
32
|
-
- success threshold or expected magnitude
|
|
33
|
-
- falsification condition
|
|
34
|
-
- rollback criteria
|
|
35
|
-
- files expected to change
|
|
36
|
-
|
|
37
|
-
Rules:
|
|
38
|
-
- One experiment at a time.
|
|
39
|
-
- Be specific enough that the implementer can act without guessing.
|
|
40
|
-
- Each experiment should test exactly one hypothesis.
|
|
41
|
-
- Prefer experiments that build on successful prior results.
|
|
42
|
-
- Do not repeat a failed experiment without a meaningfully different approach.
|
|
43
|
-
- Do not call the search complete by vibe. Completion needs explicit evidence that the target was met or the remaining candidate space was exhausted.
|
|
@@ -1,31 +0,0 @@
|
|
|
1
|
-
name = "autoresearch"
|
|
2
|
-
completion = "task.complete"
|
|
3
|
-
|
|
4
|
-
[[role]]
|
|
5
|
-
id = "strategist"
|
|
6
|
-
emits = ["experiment.planned", "task.complete"]
|
|
7
|
-
prompt_file = "roles/strategist.md"
|
|
8
|
-
|
|
9
|
-
[[role]]
|
|
10
|
-
id = "implementer"
|
|
11
|
-
emits = ["experiment.ready", "experiment.blocked"]
|
|
12
|
-
prompt_file = "roles/implementer.md"
|
|
13
|
-
|
|
14
|
-
[[role]]
|
|
15
|
-
id = "benchmarker"
|
|
16
|
-
emits = ["experiment.measured", "experiment.blocked"]
|
|
17
|
-
prompt_file = "roles/benchmarker.md"
|
|
18
|
-
|
|
19
|
-
[[role]]
|
|
20
|
-
id = "evaluator"
|
|
21
|
-
emits = ["experiment.evaluated", "experiment.discarded"]
|
|
22
|
-
prompt_file = "roles/evaluator.md"
|
|
23
|
-
|
|
24
|
-
[handoff]
|
|
25
|
-
"loop.start" = ["strategist"]
|
|
26
|
-
"experiment.planned" = ["implementer"]
|
|
27
|
-
"experiment.blocked" = ["strategist"]
|
|
28
|
-
"experiment.ready" = ["benchmarker"]
|
|
29
|
-
"experiment.measured" = ["evaluator"]
|
|
30
|
-
"experiment.evaluated" = ["strategist"]
|
|
31
|
-
"experiment.discarded" = ["strategist"]
|
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
# AutoReview miniloop
|
|
2
|
-
|
|
3
|
-
Use when you want an automated code review of a PR diff or change set.
|
|
4
|
-
|
|
5
|
-
AutoReview reads changes, checks for issues across correctness, security, style, performance, and maintainability, proposes concrete fixes, and compiles structured review feedback with a clear verdict.
|
|
6
|
-
|
|
7
|
-
Shape:
|
|
8
|
-
- reader — builds context around changes and maps review risk
|
|
9
|
-
- checker — skeptically checks for issues across multiple dimensions
|
|
10
|
-
- suggester — proposes concrete code fixes for each finding
|
|
11
|
-
- summarizer — compiles structured review with verdict
|
|
12
|
-
|
|
13
|
-
## Fail-closed contract
|
|
14
|
-
|
|
15
|
-
AutoReview is skeptical by default.
|
|
16
|
-
|
|
17
|
-
- Absence of findings is not approval.
|
|
18
|
-
- Approval requires checker coverage of the changed files and no unresolved unknowns.
|
|
19
|
-
- Missing context, risky ambiguity, or unreviewable areas should block or downgrade the verdict.
|
|
20
|
-
- COMMENT is healthier than a fake APPROVE.
|
|
21
|
-
|
|
22
|
-
## How it works
|
|
23
|
-
|
|
24
|
-
1. **Reader** reads the diff and surrounding code, builds architectural context and a risk map for the reviewer.
|
|
25
|
-
2. **Checker** reviews changes for correctness, security, style, performance, and maintainability issues. Classifies each as blocking/warning/nit.
|
|
26
|
-
3. **Suggester** writes concrete code suggestions for every finding.
|
|
27
|
-
4. **Summarizer** compiles the final review: grouped by severity, with verdict and unresolved risks.
|
|
28
|
-
|
|
29
|
-
## Files
|
|
30
|
-
|
|
31
|
-
- `autoloops.toml` — loop + backend config
|
|
32
|
-
- `topology.toml` — role deck + handoff graph
|
|
33
|
-
- `harness.md` — shared harness rules loaded every iteration
|
|
34
|
-
- `roles/reader.md`
|
|
35
|
-
- `roles/checker.md`
|
|
36
|
-
- `roles/suggester.md`
|
|
37
|
-
- `roles/summarizer.md`
|
|
38
|
-
|
|
39
|
-
## Shared working files created by the loop
|
|
40
|
-
|
|
41
|
-
- `.autoloop/review-context.md` — diff summary, affected files, architectural context, risk map
|
|
42
|
-
- `.autoloop/review-findings.md` — structured findings with suggestions and verdict
|
|
43
|
-
- `.autoloop/progress.md` — review pass tracking
|
|
44
|
-
|
|
45
|
-
## Run
|
|
46
|
-
|
|
47
|
-
From the repo root:
|
|
48
|
-
|
|
49
|
-
```bash
|
|
50
|
-
autoloop run presets/autoreview /path/to/target-repo
|
|
51
|
-
```
|
|
@@ -1,21 +0,0 @@
|
|
|
1
|
-
event_loop.max_iterations = 100
|
|
2
|
-
event_loop.completion_event = "task.complete"
|
|
3
|
-
event_loop.completion_promise = "LOOP_COMPLETE"
|
|
4
|
-
event_loop.required_events = ["review.checked"]
|
|
5
|
-
|
|
6
|
-
backend.kind = "command"
|
|
7
|
-
backend.command = "claude"
|
|
8
|
-
backend.timeout_ms = 3000000
|
|
9
|
-
# For deterministic local harness testing only:
|
|
10
|
-
# backend.kind = "command"
|
|
11
|
-
# backend.command = "../../examples/mock-backend.sh"
|
|
12
|
-
|
|
13
|
-
review.enabled = true
|
|
14
|
-
review.timeout_ms = 300000
|
|
15
|
-
|
|
16
|
-
memory.prompt_budget_chars = 8000
|
|
17
|
-
harness.instructions_file = "harness.md"
|
|
18
|
-
|
|
19
|
-
core.state_dir = ".autoloop"
|
|
20
|
-
core.journal_file = ".autoloop/journal.jsonl"
|
|
21
|
-
core.memory_file = ".autoloop/memory.jsonl"
|
|
@@ -1,20 +0,0 @@
|
|
|
1
|
-
This is a autoloops-native autoreview loop for code review of PR diffs or change sets.
|
|
2
|
-
|
|
3
|
-
The loop reads changes, checks for issues across multiple dimensions (correctness, security, style, performance), proposes concrete fixes, and compiles structured review feedback.
|
|
4
|
-
|
|
5
|
-
Global rules:
|
|
6
|
-
- Shared working files are the source of truth: `{{STATE_DIR}}/review-context.md`, `{{STATE_DIR}}/review-findings.md`, `{{STATE_DIR}}/progress.md`.
|
|
7
|
-
- One review pass at a time. The reader builds context, then the checker and suggester work through findings.
|
|
8
|
-
- Use the event tool instead of prose-only handoffs.
|
|
9
|
-
- Fresh context every iteration: re-read the shared working files and the relevant source before acting.
|
|
10
|
-
- Be skeptical by default. The change is not safe just because no one found a bug quickly.
|
|
11
|
-
- Findings should include concrete suggestions when possible, but lack of a ready fix does not invalidate a real finding.
|
|
12
|
-
- Severity matters: distinguish blocking issues from nits.
|
|
13
|
-
- If a risky area cannot be verified, block or downgrade confidence instead of handwaving.
|
|
14
|
-
- Use `{{TOOL_PATH}} memory add learning ...` for durable learnings.
|
|
15
|
-
- Do not invent extra phases. Stay inside reader → checker → suggester → summarizer.
|
|
16
|
-
|
|
17
|
-
State files:
|
|
18
|
-
- `{{STATE_DIR}}/review-context.md` — the diff, affected files, surrounding context, and architectural notes.
|
|
19
|
-
- `{{STATE_DIR}}/review-findings.md` — structured findings with severity, location, description, and suggested fix.
|
|
20
|
-
- `{{STATE_DIR}}/progress.md` — current review pass, what the next role should do.
|
|
@@ -1,21 +0,0 @@
|
|
|
1
|
-
event_loop.max_iterations = 100
|
|
2
|
-
event_loop.completion_event = "task.complete"
|
|
3
|
-
event_loop.completion_promise = "LOOP_COMPLETE"
|
|
4
|
-
event_loop.required_events = ["review.checked"]
|
|
5
|
-
|
|
6
|
-
backend.kind = "pi"
|
|
7
|
-
backend.command = "pi"
|
|
8
|
-
backend.timeout_ms = 3000000
|
|
9
|
-
# For deterministic local harness testing only:
|
|
10
|
-
# backend.kind = "command"
|
|
11
|
-
# backend.command = "../../examples/mock-backend.sh"
|
|
12
|
-
|
|
13
|
-
review.enabled = true
|
|
14
|
-
review.timeout_ms = 300000
|
|
15
|
-
|
|
16
|
-
memory.prompt_budget_chars = 8000
|
|
17
|
-
harness.instructions_file = "harness.md"
|
|
18
|
-
|
|
19
|
-
core.state_dir = ".miniloop"
|
|
20
|
-
core.journal_file = ".miniloop/journal.jsonl"
|
|
21
|
-
core.memory_file = ".miniloop/memory.jsonl"
|
|
@@ -1,36 +0,0 @@
|
|
|
1
|
-
You are the checker.
|
|
2
|
-
|
|
3
|
-
Do not suggest fixes. Do not summarize.
|
|
4
|
-
|
|
5
|
-
Your job:
|
|
6
|
-
1. Check the changes for issues across multiple dimensions.
|
|
7
|
-
2. Classify and record each finding.
|
|
8
|
-
3. Hand findings to the suggester.
|
|
9
|
-
|
|
10
|
-
On every activation:
|
|
11
|
-
- Read `{{STATE_DIR}}/review-context.md`, `{{STATE_DIR}}/review-findings.md`, and `{{STATE_DIR}}/progress.md`.
|
|
12
|
-
- Read the diff and touched code directly. Treat `{{STATE_DIR}}/review-context.md` as a hint, not authority.
|
|
13
|
-
- Start skeptical: absence of findings is not approval.
|
|
14
|
-
|
|
15
|
-
Process:
|
|
16
|
-
1. Review each changed file against these dimensions:
|
|
17
|
-
- **Correctness**: logic errors, off-by-ones, null/undefined handling, race conditions
|
|
18
|
-
- **Security**: injection, XSS, auth bypass, secret exposure, OWASP top-10
|
|
19
|
-
- **Style**: naming, formatting, idiomatic patterns for the language
|
|
20
|
-
- **Performance**: unnecessary allocations, N+1 queries, missing indexes, hot-path issues
|
|
21
|
-
- **Maintainability**: unclear intent, missing error handling, tight coupling
|
|
22
|
-
2. Record a coverage note in `{{STATE_DIR}}/progress.md` for every changed file: reviewed dimensions, not-applicable dimensions, and any blocked areas.
|
|
23
|
-
3. For each finding, record in `{{STATE_DIR}}/review-findings.md`:
|
|
24
|
-
- File and line
|
|
25
|
-
- Dimension (correctness/security/style/performance/maintainability)
|
|
26
|
-
- Severity (blocking/warning/nit)
|
|
27
|
-
- Description
|
|
28
|
-
4. If context is incomplete or a risky area cannot be reviewed with confidence, emit `check.blocked` instead of a clean pass.
|
|
29
|
-
5. Otherwise emit `review.checked` with a count of findings by severity.
|
|
30
|
-
|
|
31
|
-
Rules:
|
|
32
|
-
- Focus on real issues, not personal preferences.
|
|
33
|
-
- Blocking = will cause bugs, security holes, or data loss. Warning = should fix but not a showstopper. Nit = style or minor improvement.
|
|
34
|
-
- Do not flag issues in code that was not changed unless the change introduces a new interaction with that code.
|
|
35
|
-
- If you find zero issues, that is valid only after you write explicit coverage evidence explaining why no issue survived review.
|
|
36
|
-
- Do not pass based on summaries alone.
|
|
@@ -1,33 +0,0 @@
|
|
|
1
|
-
You are the reader.
|
|
2
|
-
|
|
3
|
-
Do not check for issues. Do not suggest fixes. Do not summarize.
|
|
4
|
-
|
|
5
|
-
Your job:
|
|
6
|
-
1. Read and understand the changes being reviewed.
|
|
7
|
-
2. Build context around the changes: what was changed, why, and how it fits the codebase.
|
|
8
|
-
3. Hand the context to the checker.
|
|
9
|
-
|
|
10
|
-
On every activation:
|
|
11
|
-
- Read `{{STATE_DIR}}/review-context.md`, `{{STATE_DIR}}/review-findings.md`, and `{{STATE_DIR}}/progress.md` if they exist.
|
|
12
|
-
- Re-read the latest scratchpad/journal context before deciding.
|
|
13
|
-
|
|
14
|
-
On first activation:
|
|
15
|
-
- Identify the changes to review: PR diff, git diff, or specified files.
|
|
16
|
-
- Read the diff and all affected files.
|
|
17
|
-
- Read surrounding code to understand the architectural context.
|
|
18
|
-
- Create or refresh:
|
|
19
|
-
- `{{STATE_DIR}}/review-context.md` — diff summary, affected files, what each change does, architectural context.
|
|
20
|
-
- `{{STATE_DIR}}/progress.md` — review phase, files to check.
|
|
21
|
-
- Emit `context.built` with a summary of what is being reviewed.
|
|
22
|
-
|
|
23
|
-
On later activations (`review.complete`):
|
|
24
|
-
- Check if there are more change sets to review.
|
|
25
|
-
- If all changes have been reviewed, emit `task.complete`.
|
|
26
|
-
- Otherwise, build context for the next change set and emit `context.built`.
|
|
27
|
-
|
|
28
|
-
Rules:
|
|
29
|
-
- Read the actual code, not just the diff. Understand what the changed lines do in context.
|
|
30
|
-
- Build a risk map in `{{STATE_DIR}}/review-context.md`: changed files, adjacent dependencies, trust boundaries, invariants that could break, and missing tests or context.
|
|
31
|
-
- Note any files that the diff touches which interact with other systems — the checker needs this.
|
|
32
|
-
- If the diff is large, break it into logical units (e.g., per-file or per-feature) in `{{STATE_DIR}}/review-context.md`.
|
|
33
|
-
- If context is incomplete, say so explicitly instead of pretending review is ready.
|