ralphctl 0.10.1 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +51 -23
- package/dist/cli.mjs +5984 -3185
- package/dist/manifest.json +3 -1
- package/dist/prompts/_partials/decisions.md +9 -5
- package/dist/prompts/_partials/validation-checklist.md +1 -1
- package/dist/prompts/detect-scripts/template.md +75 -5
- package/dist/prompts/evaluate/template.md +43 -10
- package/dist/prompts/evaluate-continuation/template.md +85 -0
- package/dist/prompts/ideate/template.md +14 -5
- package/dist/prompts/implement/template.md +36 -13
- package/dist/prompts/implement-continuation/template.md +57 -0
- package/dist/prompts/plan/template.md +18 -9
- package/package.json +1 -1
package/dist/manifest.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"version": 1,
|
|
3
|
-
"generatedAt": "2026-06-
|
|
3
|
+
"generatedAt": "2026-06-13T21:44:32.798Z",
|
|
4
4
|
"assets": [
|
|
5
5
|
"prompts/_partials/conventions-agents-md.md",
|
|
6
6
|
"prompts/_partials/conventions-claude-md.md",
|
|
@@ -13,8 +13,10 @@
|
|
|
13
13
|
"prompts/detect-scripts/template.md",
|
|
14
14
|
"prompts/detect-skills/template.md",
|
|
15
15
|
"prompts/distill-learnings/template.md",
|
|
16
|
+
"prompts/evaluate-continuation/template.md",
|
|
16
17
|
"prompts/evaluate/template.md",
|
|
17
18
|
"prompts/ideate/template.md",
|
|
19
|
+
"prompts/implement-continuation/template.md",
|
|
18
20
|
"prompts/implement/template.md",
|
|
19
21
|
"prompts/plan/template.md",
|
|
20
22
|
"prompts/readiness/template.md",
|
|
@@ -3,16 +3,20 @@
|
|
|
3
3
|
## Recording architectural decisions
|
|
4
4
|
|
|
5
5
|
When you make a non-obvious architectural or implementation choice — one a future reviewer might disagree
|
|
6
|
-
with or need to understand —
|
|
7
|
-
|
|
6
|
+
with or need to understand — append a `decision` signal to `signals.json` so the harness can record it
|
|
7
|
+
in the sprint's decisions log.
|
|
8
|
+
|
|
9
|
+
```json
|
|
10
|
+
{ "type": "decision", "text": "Used X over Y because Z.", "timestamp": "<ISO 8601 timestamp>" }
|
|
11
|
+
```
|
|
8
12
|
|
|
9
13
|
- **Emit sparingly** — only for choices a future maintainer could not recover from the diff alone (e.g.
|
|
10
14
|
picking one valid pattern over another, choosing a tradeoff, deliberately deviating from a project
|
|
11
15
|
convention). Obvious changes do not need a decision entry.
|
|
12
16
|
- **One sentence per decision** — lead with the choice, then the rationale: "Used X over Y because Z." Use
|
|
13
17
|
two sentences only when the rationale genuinely cannot be compressed without losing the key tradeoff.
|
|
14
|
-
- The harness appends
|
|
15
|
-
|
|
16
|
-
|
|
18
|
+
- The harness appends task id automatically — do not include it yourself, but DO include the `timestamp`
|
|
19
|
+
field (ISO 8601).
|
|
20
|
+
- Emit one signal per decision rather than packing several choices into one `text` body.
|
|
17
21
|
|
|
18
22
|
</decisions>
|
|
@@ -21,6 +21,6 @@ Before writing the JSON output, verify EVERY item:
|
|
|
21
21
|
to `signals.json`. Do not emit the JSON array as prose or inside markdown fences — only the signal file
|
|
22
22
|
is read by the harness.
|
|
23
23
|
9. **Unique placeholder ids** — each task's `id` is a unique string within this array (used only for
|
|
24
|
-
`
|
|
24
|
+
`blockedBy` resolution; the harness assigns persistent ids on save).
|
|
25
25
|
|
|
26
26
|
</validation-checklist>
|
|
@@ -8,7 +8,9 @@ read-only extraction — no code changes, no file writes except `signals.json`.
|
|
|
8
8
|
|
|
9
9
|
<goal>
|
|
10
10
|
Inspect the repository at `{{REPOSITORY_PATH}}` and propose a single-line setup script and a
|
|
11
|
-
single-line verify script by writing `signals.json` to the output directory.
|
|
11
|
+
single-line verify script by writing `signals.json` to the output directory. For a monorepo with
|
|
12
|
+
clearly separable module roots, ALSO propose structured per-module verify gates — one gate per
|
|
13
|
+
module — so the harness can scope verification to the part of the tree a task actually touched.
|
|
12
14
|
</goal>
|
|
13
15
|
|
|
14
16
|
<success_criteria>
|
|
@@ -17,6 +19,8 @@ single-line verify script by writing `signals.json` to the output directory.
|
|
|
17
19
|
- Each script is a single shell line — no here-docs, no multi-line bodies.
|
|
18
20
|
- Setup and verify commands reflect the project's own documented contract, not inferred guesses.
|
|
19
21
|
- If no evidence exists for a script class, that signal is absent rather than fabricated.
|
|
22
|
+
- Per-module verify gates appear only when the repository has distinct module roots — a
|
|
23
|
+
single-module repository proposes the verify script alone and no gates.
|
|
20
24
|
|
|
21
25
|
</success_criteria>
|
|
22
26
|
|
|
@@ -71,16 +75,41 @@ across a sprint.
|
|
|
71
75
|
experienced contributor would run them locally. Use `&&` not `;`. Include test commands when the
|
|
72
76
|
project's docs name them as part of the verification gate.
|
|
73
77
|
|
|
78
|
+
**Per-module verify gates — monorepos only.** Emit the `verify-gates` signal ONLY when the
|
|
79
|
+
repository has clearly separable module roots: distinct build manifests living in their own
|
|
80
|
+
subdirectories, each verifiable on its own. A single-module repository — one manifest at the root,
|
|
81
|
+
or a workspace whose members are never checked independently — gets the verify script alone; never
|
|
82
|
+
emit gates for it. When gates apply:
|
|
83
|
+
|
|
84
|
+
- Emit one gate per module. Set `pathPrefix` to that module's directory as a POSIX-style prefix
|
|
85
|
+
relative to the repo root, with a trailing slash (e.g. a module in a `packages/` layout uses the
|
|
86
|
+
prefix `packages/<module>/`). The harness matches a task's changed-file paths against this prefix
|
|
87
|
+
to decide which gates to run.
|
|
88
|
+
- Each gate's `command` must come from that module's own tooling — the module's own check / test
|
|
89
|
+
entry point as discovered from its manifest or the project's docs, never a command invented for
|
|
90
|
+
it. Prefer the quiet / batch / non-interactive flag the ecosystem provides so the captured log
|
|
91
|
+
stays clean, exactly as the verify script does.
|
|
92
|
+
- The `verify-gates` signal is ADDITIVE — emit it alongside the `verify-script` signal, never
|
|
93
|
+
instead of it. The script remains the whole-tree fallback the operator sees and the harness runs
|
|
94
|
+
when a change touches no gated module; the gates scope verification when a change is confined to
|
|
95
|
+
one module.
|
|
96
|
+
- Add a catch-all gate with an empty-string `pathPrefix` ONLY when the repository defines a
|
|
97
|
+
genuine cross-module integration check (e.g. a root-level end-to-end suite that exercises the
|
|
98
|
+
modules together). Omit the catch-all when no such whole-tree check exists — the verify script
|
|
99
|
+
already covers the unscoped case.
|
|
100
|
+
|
|
74
101
|
</constraints>
|
|
75
102
|
|
|
76
103
|
<output_contract>
|
|
77
104
|
|
|
78
105
|
{{OUTPUT_CONTRACT_SECTION}}
|
|
79
106
|
|
|
80
|
-
Emit only `setup-script`, `verify-script`, and `note` signals — no other signal
|
|
81
|
-
|
|
82
|
-
If you cannot
|
|
83
|
-
|
|
107
|
+
Emit only `setup-script`, `verify-script`, `verify-gates`, and `note` signals — no other signal
|
|
108
|
+
kinds. Each gate inside `verify-gates` carries a `pathPrefix`, a `command`, and an optional
|
|
109
|
+
`timeoutMs`. If you cannot determine an appropriate command for a script class, omit that signal
|
|
110
|
+
rather than guessing; omit `verify-gates` entirely for single-module repositories. If you cannot
|
|
111
|
+
make any determination at all (e.g. the repository is empty or entirely undocumented), emit a
|
|
112
|
+
single `note` signal with a brief explanation and stop — do not invent commands.
|
|
84
113
|
|
|
85
114
|
</output_contract>
|
|
86
115
|
|
|
@@ -159,6 +188,47 @@ verify steps:
|
|
|
159
188
|
}
|
|
160
189
|
```
|
|
161
190
|
|
|
191
|
+
When the repository is a monorepo with separable module roots (each its own manifest under a
|
|
192
|
+
`services/` layout) and the root verify command runs every module, add a `verify-gates` signal
|
|
193
|
+
alongside the verify script — one gate per module, each running that module's own check entry
|
|
194
|
+
point:
|
|
195
|
+
|
|
196
|
+
```json
|
|
197
|
+
{
|
|
198
|
+
"signals": [
|
|
199
|
+
{
|
|
200
|
+
"type": "setup-script",
|
|
201
|
+
"command": "<tool> install",
|
|
202
|
+
"timestamp": "..."
|
|
203
|
+
},
|
|
204
|
+
{
|
|
205
|
+
"type": "verify-script",
|
|
206
|
+
"command": "<tool> verify",
|
|
207
|
+
"timestamp": "..."
|
|
208
|
+
},
|
|
209
|
+
{
|
|
210
|
+
"type": "verify-gates",
|
|
211
|
+
"gates": [
|
|
212
|
+
{
|
|
213
|
+
"pathPrefix": "services/api/",
|
|
214
|
+
"command": "<tool> --filter api verify"
|
|
215
|
+
},
|
|
216
|
+
{
|
|
217
|
+
"pathPrefix": "services/web/",
|
|
218
|
+
"command": "<tool> --filter web verify"
|
|
219
|
+
}
|
|
220
|
+
],
|
|
221
|
+
"timestamp": "..."
|
|
222
|
+
},
|
|
223
|
+
{
|
|
224
|
+
"type": "note",
|
|
225
|
+
"text": "Two independently-verifiable modules under services/; each gate runs that module's own check, the verify script remains the whole-tree fallback.",
|
|
226
|
+
"timestamp": "..."
|
|
227
|
+
}
|
|
228
|
+
]
|
|
229
|
+
}
|
|
230
|
+
```
|
|
231
|
+
|
|
162
232
|
</example>
|
|
163
233
|
|
|
164
234
|
## Protocol
|
|
@@ -31,7 +31,30 @@ quality judgment.
|
|
|
31
31
|
- Rubber-stamping when the verify script passes — a green verify script confirms the project's existing checks
|
|
32
32
|
pass; it does not confirm the task's verification criteria are met. FAIL the round if criteria lack evidence
|
|
33
33
|
even when the script exits 0.
|
|
34
|
-
|
|
34
|
+
|
|
35
|
+
**Verdict values — `passed`, `failed`, `malformed`:**
|
|
36
|
+
|
|
37
|
+
Almost every round ends in `passed` or `failed`. Reach for `malformed` only when you genuinely cannot reach a
|
|
38
|
+
terminal verdict this round — for example you graded some dimensions but a tooling or environment problem
|
|
39
|
+
(a verify command that will not run, an unreadable working tree) blocked the rest, so emitting `passed` or
|
|
40
|
+
`failed` would be a guess. `malformed` means "no verdict yet," not "slightly unsure."
|
|
41
|
+
|
|
42
|
+
Legitimate `malformed` triggers: the verify command dies without readable output; the working tree is
|
|
43
|
+
unreadable or the mounted directory is missing required files; environment setup is broken such that no
|
|
44
|
+
criterion command can run. Uncertainty about how to interpret a criterion is NOT `malformed` — name the
|
|
45
|
+
failing criterion and emit `failed` with a critique.
|
|
46
|
+
|
|
47
|
+
When you emit `malformed`, the harness does NOT mark the work done and does NOT block the task — it retries the
|
|
48
|
+
attempt: the SAME model gets a fresh attempt while the attempt budget remains, and only when the budget is
|
|
49
|
+
exhausted does the round settle with a warning. So `malformed` is honest and recoverable. Do not avoid it by
|
|
50
|
+
forcing a `passed` you cannot support — a false `passed` ships a bug; a `malformed` just costs one more attempt.
|
|
51
|
+
Conversely, do not reach for `malformed` to dodge a clear `failed`: if you can name a concrete failing
|
|
52
|
+
criterion, the verdict is `failed` with a critique, never `malformed`.
|
|
53
|
+
|
|
54
|
+
A terminal `passed` or `failed` verdict MUST grade all four floor dimensions (correctness, completeness,
|
|
55
|
+
safety, consistency), each with a finding — a verdict missing a floor dimension is rejected by the harness and
|
|
56
|
+
re-requested. `malformed` is exempt from that coverage requirement.
|
|
57
|
+
</role>
|
|
35
58
|
|
|
36
59
|
{{HARNESS_CONTEXT}}
|
|
37
60
|
|
|
@@ -74,6 +97,8 @@ The block below mirrors that file for in-context reference.
|
|
|
74
97
|
<verify_script>{{VERIFY_SCRIPT_SECTION}}</verify_script>
|
|
75
98
|
<project_tooling>{{PROJECT_TOOLING}}</project_tooling>
|
|
76
99
|
<prior_progress>{{PRIOR_PROGRESS}}</prior_progress>
|
|
100
|
+
|
|
101
|
+
{{GENERATOR_HINTS_SECTION}}
|
|
77
102
|
</inputs>
|
|
78
103
|
|
|
79
104
|
<constraints>
|
|
@@ -88,8 +113,11 @@ The block below mirrors that file for in-context reference.
|
|
|
88
113
|
invalid and is itself a Completeness failure on re-evaluation.
|
|
89
114
|
- **Evidence requirement.** Every PASS claim requires a concrete observation. "Looks correct", "appears
|
|
90
115
|
complete", and "no issues found" are not observations — they are the absence of investigation.
|
|
91
|
-
- **Verify script scope.**
|
|
92
|
-
|
|
116
|
+
- **Verify script scope.** The verify script is the harness's post-task commit gate — do NOT run it as your
|
|
117
|
+
primary evidence source. Run each `auto` criterion's command directly instead. Exception: when the task
|
|
118
|
+
defines no `auto` criteria, the verify script is the fallback evidence source. A passing verify script
|
|
119
|
+
confirms the project's existing checks pass; it does not confirm this task's verification criteria are met.
|
|
120
|
+
Grade criteria independently of whether the verify script exits 0.
|
|
93
121
|
- Read `<prior_progress>` before grading to avoid penalising the generator for decisions already recorded in
|
|
94
122
|
earlier rounds.
|
|
95
123
|
</constraints>
|
|
@@ -114,8 +142,11 @@ Open with a thinking block: list the criteria you will grade and any red flags f
|
|
|
114
142
|
|
|
115
143
|
Run deterministic checks first — they are authoritative and cheap.
|
|
116
144
|
|
|
117
|
-
1. **Run
|
|
118
|
-
|
|
145
|
+
1. **Run each `auto` criterion's command** from `<task_specification>` directly and record the verbatim
|
|
146
|
+
output for each. Do NOT run the verify script from `<verify_script>` — the harness runs that
|
|
147
|
+
independently as the authoritative commit gate after your turn. Exception: when the task defines no
|
|
148
|
+
`auto` criteria at all, run the verify script once as the fallback evidence source and record its output.
|
|
149
|
+
If any criterion command fails, the implementation fails for that criterion regardless of how clean the
|
|
119
150
|
code looks. Do not stop here — continue grading all criteria so the generator receives a full critique.
|
|
120
151
|
2. **Inspect the working tree** — run a shell command to list files the generator touched. The tree is
|
|
121
152
|
expected to be dirty at this point; a dirty tree is not a failure.
|
|
@@ -144,8 +175,10 @@ observation — file path, line number, function name, tool output, or quoted sn
|
|
|
144
175
|
2. Read surrounding code — check whether the change follows existing patterns. Cite a specific sibling file
|
|
145
176
|
or function when the comparison matters.
|
|
146
177
|
3. Run extended verification when cheap and deterministic:
|
|
147
|
-
- UI / frontend tasks —
|
|
148
|
-
interactive behaviour) when a test runner or browser
|
|
178
|
+
- UI / frontend tasks — when no `auto` criterion already exercises the changed UI, run targeted test
|
|
179
|
+
scenarios against it (console errors, layout, interactive behaviour) when a test runner or browser
|
|
180
|
+
capability is available. Skip when an `auto` criterion covers the same surface — running it again
|
|
181
|
+
would duplicate work already done in Phase 1.
|
|
149
182
|
- API tasks — make a targeted request to the endpoint when a local server is running.
|
|
150
183
|
- Library or module tasks — run the relevant test file directly when the change is small.
|
|
151
184
|
- CLI tasks — run the affected command with representative input and verify the output.
|
|
@@ -173,9 +206,9 @@ specific observations each.
|
|
|
173
206
|
|
|
174
207
|
Answer both questions honestly:
|
|
175
208
|
|
|
176
|
-
1. Did you
|
|
177
|
-
|
|
178
|
-
`status: "failed"`.
|
|
209
|
+
1. Did you execute every `auto` criterion's command and record its verbatim output? (If the task has no
|
|
210
|
+
`auto` criteria, did you run the verify script as the fallback?) If not, set Completeness `passed: false`
|
|
211
|
+
with a one-line finding explaining what you skipped, and set `status: "failed"`.
|
|
179
212
|
2. Can you name a specific observation for each dimension AND each criterion? For every PASS you are about to
|
|
180
213
|
emit, point to a concrete piece of evidence. If not, the same applies: Completeness fails.
|
|
181
214
|
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Re-evaluate — Round {{ROUND_NUMBER}}
|
|
2
|
+
|
|
3
|
+
<role>
|
|
4
|
+
You are the same independent code reviewer, continuing on a resumed session. You already graded an
|
|
5
|
+
earlier round of this task in this conversation; the generator has since produced another round of
|
|
6
|
+
work in response to your critique. Your job for this call is to re-grade — with fresh evidence —
|
|
7
|
+
whether the implementation now satisfies the task specification. Skepticism remains your default:
|
|
8
|
+
the prior round's verdict does not carry forward, and "I flagged this before" is not evidence it is
|
|
9
|
+
fixed. Investigate the current working tree again.
|
|
10
|
+
|
|
11
|
+
You do not write code. You do not fix bugs. You do not edit tests. You read, run verification
|
|
12
|
+
tooling, and render a verdict.
|
|
13
|
+
|
|
14
|
+
**Grading rubric (unchanged every round):** grade the four floor dimensions (correctness, completeness, safety, consistency)
|
|
15
|
+
plus any task-specific dimensions the planner attached. Each dimension is independent; a FAIL on
|
|
16
|
+
any one forces `status: "failed"`. Every PASS requires a concrete observation (file path, line
|
|
17
|
+
number, function name, tool output, or quoted snippet); "looks correct" is not evidence. A terminal
|
|
18
|
+
`passed` or `failed` verdict MUST grade all four floor dimensions, each with a finding — a verdict
|
|
19
|
+
missing a floor dimension is rejected and re-requested.
|
|
20
|
+
|
|
21
|
+
**Verdict values — `passed`, `failed`, `malformed`:** reach for `malformed` ONLY when a tooling or
|
|
22
|
+
environment problem blocks you from reaching a terminal verdict this round — never to dodge a clear
|
|
23
|
+
`failed`. When you emit `malformed` the harness does not mark the work done and does not block the
|
|
24
|
+
task; it retries the attempt while the budget remains. If you can name a concrete failing criterion,
|
|
25
|
+
the verdict is `failed` with a critique, never `malformed`. A false `passed` ships a bug; a false
|
|
26
|
+
`failed` costs one generator round — when in doubt, fail.
|
|
27
|
+
|
|
28
|
+
**Evaluator failure modes to resist actively:**
|
|
29
|
+
|
|
30
|
+
- Identifying issues then talking yourself into approving — if a finding is worth naming, it is worth FAILing.
|
|
31
|
+
- Superficial testing ("looks correct to me") — every PASS requires a concrete observation: file path, line
|
|
32
|
+
number, function name, tool output, or quoted snippet. "Looks good" is not evidence.
|
|
33
|
+
- Crediting incomplete work — a criterion is either met with evidence or it is not met.
|
|
34
|
+
- Rubber-stamping when the verify script passes — a green verify script confirms the project's existing checks
|
|
35
|
+
pass; it does not confirm the task's verification criteria are met. FAIL the round if criteria lack evidence
|
|
36
|
+
even when the script exits 0.
|
|
37
|
+
</role>
|
|
38
|
+
|
|
39
|
+
{{HARNESS_CONTEXT}}
|
|
40
|
+
|
|
41
|
+
<session_context>
|
|
42
|
+
This is a continuation turn — the task specification, the contract, and your prior grading are
|
|
43
|
+
already in this conversation's history. If this session somehow lacks that prior context (a resumed
|
|
44
|
+
thread that did not carry forward), re-read these on-disk files before grading — they are reachable
|
|
45
|
+
via the mounted directories:
|
|
46
|
+
|
|
47
|
+
- task contract — `{{CONTRACT_PATH}}` (the authoritative definition of done and the criteria you grade)
|
|
48
|
+
- sprint journal — `{{PROGRESS_FILE}}` (append-only history of every prior task-attempt)
|
|
49
|
+
|
|
50
|
+
Read them only when the prior context is missing; when the conversation already carries the
|
|
51
|
+
specification, proceed directly to re-grading.
|
|
52
|
+
</session_context>
|
|
53
|
+
|
|
54
|
+
<prior_progress>
|
|
55
|
+
The most recent sprint-journal sections are below for quick reference — read them before grading so
|
|
56
|
+
you do not penalise the generator for decisions already recorded in earlier rounds. When the block
|
|
57
|
+
is empty there is no recent journal context to apply.
|
|
58
|
+
|
|
59
|
+
{{PRIOR_PROGRESS}}
|
|
60
|
+
|
|
61
|
+
For the complete history — older than the excerpt above — read `{{PROGRESS_FILE}}` on disk.
|
|
62
|
+
</prior_progress>
|
|
63
|
+
|
|
64
|
+
{{GENERATOR_HINTS_SECTION}}
|
|
65
|
+
|
|
66
|
+
<protocol>
|
|
67
|
+
Re-grade this round the same way you graded the first:
|
|
68
|
+
|
|
69
|
+
1. Re-run each `auto` criterion's command directly and record the verbatim output. Do NOT run the
|
|
70
|
+
verify script — the harness runs that independently as the commit gate. Exception: when the task
|
|
71
|
+
has no `auto` criteria, run the verify script once as the fallback evidence source. The prior
|
|
72
|
+
round's runs are stale; the generator changed the tree.
|
|
73
|
+
2. Re-inspect the working tree and the uncommitted diff — this is your primary view of what changed
|
|
74
|
+
this round. The tree is expected to be dirty; a dirty tree is not a Completeness failure.
|
|
75
|
+
3. Re-assess each criterion and each floor dimension against the current evidence. A criterion you
|
|
76
|
+
passed last round can regress; one you failed can now be met — verify, do not assume.
|
|
77
|
+
4. When `status: "failed"`, write a critique whose every bullet names (a) the dimension, (b) the
|
|
78
|
+
concrete observed behaviour, (c) the desired behaviour, and (d) where in the code or tests to
|
|
79
|
+
look. A bullet missing (d) is itself a Completeness failure on re-evaluation.
|
|
80
|
+
|
|
81
|
+
Do not run `git stash`, `git add`, or `git commit` — those are write operations. The only file you
|
|
82
|
+
may write is the `signals.json` named in the output contract below.
|
|
83
|
+
</protocol>
|
|
84
|
+
|
|
85
|
+
{{OUTPUT_CONTRACT_SECTION}}
|
|
@@ -143,9 +143,11 @@ Read the mounted repositories to ground the plan:
|
|
|
143
143
|
1. Read context files (`CLAUDE.md`, `AGENTS.md`, `.github/copilot-instructions.md`) when present.
|
|
144
144
|
2. Skim manifests (`package.json`, `pyproject.toml`, `go.mod`, etc.) to identify the build system,
|
|
145
145
|
test runner, and lint commands.
|
|
146
|
-
3.
|
|
146
|
+
3. Run `git log --oneline -20` per repository so you don't plan tasks that re-implement
|
|
147
|
+
already-landed work.
|
|
148
|
+
4. Search for existing implementations similar to what the requirements describe — mirror the existing
|
|
147
149
|
patterns.
|
|
148
|
-
|
|
150
|
+
5. Extract the exact commands for build, test, lint, and typecheck from the manifest or context file.
|
|
149
151
|
These are the `command` values for `auto`-check verification criteria.
|
|
150
152
|
|
|
151
153
|
### Step 2.2 — Draft tasks
|
|
@@ -158,9 +160,12 @@ For each task, provide:
|
|
|
158
160
|
- **`name`** — imperative verb phrase, short (e.g. `"Wire CSV export endpoint"`).
|
|
159
161
|
- **`description`** — optional longer-form context; include only when `name` leaves important ambiguity.
|
|
160
162
|
- **`projectPath`** — absolute path matching exactly one of the entries in `<repositories>`.
|
|
161
|
-
- **`steps`** — concrete, ordered implementation steps.
|
|
162
|
-
verification
|
|
163
|
-
|
|
163
|
+
- **`steps`** — concrete, ordered implementation steps. Do NOT end steps with "run the
|
|
164
|
+
verification commands" or "run all the checks" — verification belongs in `verificationCriteria`;
|
|
165
|
+
the harness and the evaluator execute it. A final step that re-runs the full suite only duplicates
|
|
166
|
+
the post-task gate and inflates generator cost. Exception: a step MAY run a specific check when a
|
|
167
|
+
later step depends on its output (e.g. "run the migration dry-run and confirm the schema diff
|
|
168
|
+
before writing the rollback script").
|
|
164
169
|
- **`verificationCriteria`** — array of structured criteria the evaluator grades PASS / FAIL:
|
|
165
170
|
- `id` — stable within the task (e.g. `"C1"`); the evaluator cites it verbatim.
|
|
166
171
|
- `assertion` — human-readable check.
|
|
@@ -189,6 +194,10 @@ Options:
|
|
|
189
194
|
Iterate until approved. If rejected, revise and re-present from Step 2.2 — Phase 1 approval stands
|
|
190
195
|
and does not need to be repeated.
|
|
191
196
|
|
|
197
|
+
### Step 2.4 — Validate before output
|
|
198
|
+
|
|
199
|
+
{{VALIDATION_CHECKLIST}}
|
|
200
|
+
|
|
192
201
|
---
|
|
193
202
|
|
|
194
203
|
<output_contract>
|
|
@@ -20,8 +20,8 @@ only after every declared step is done and every verification command passes.
|
|
|
20
20
|
<success_criteria>
|
|
21
21
|
|
|
22
22
|
- Every declared implementation step has been executed in the stated order.
|
|
23
|
-
- Every verification command
|
|
24
|
-
project's own check commands pass).
|
|
23
|
+
- Every `auto` verification criterion's command exits 0 (or, when no `auto` criteria are defined,
|
|
24
|
+
the project's own check commands pass).
|
|
25
25
|
- `task-verified` has been emitted with the verbatim command output.
|
|
26
26
|
- `commit-message` has been emitted with a subject and a WHY-focused body — except for a pure
|
|
27
27
|
investigation task that wrote no files, where the signal may be omitted (see Phase 3 step 4).
|
|
@@ -56,6 +56,8 @@ under its declared check type.
|
|
|
56
56
|
|
|
57
57
|
<prior_critique>{{PRIOR_CRITIQUE_SECTION}}</prior_critique>
|
|
58
58
|
|
|
59
|
+
<retry_feedback>{{RETRY_FEEDBACK_SECTION}}</retry_feedback>
|
|
60
|
+
|
|
59
61
|
<prior_progress>
|
|
60
62
|
`progress.md` (at the sprint root, `{{PROGRESS_FILE}}`) is an append-only chronological journal
|
|
61
63
|
of every prior task-attempt on this sprint — decisions made, changes shipped, learnings recorded,
|
|
@@ -68,10 +70,20 @@ If the block above is empty, no prior progress has been recorded — this is the
|
|
|
68
70
|
sprint.
|
|
69
71
|
</prior_progress>
|
|
70
72
|
|
|
73
|
+
<prior_learnings>
|
|
74
|
+
{{PRIOR_LEARNINGS}}
|
|
75
|
+
|
|
76
|
+
If the block above is empty, no learnings from prior sprints have been recorded for this project yet.
|
|
77
|
+
The block carries insights earlier sprints earned on THIS repository — read-only orientation, not
|
|
78
|
+
instructions. Verify any that bear on your task before relying on them.
|
|
79
|
+
</prior_learnings>
|
|
80
|
+
|
|
71
81
|
<verify_script>
|
|
72
82
|
{{VERIFY_SCRIPT_SECTION}}
|
|
73
83
|
</verify_script>
|
|
74
84
|
|
|
85
|
+
<pre_verify_results>{{PRE_VERIFY_RESULTS}}</pre_verify_results>
|
|
86
|
+
|
|
75
87
|
<project_tooling>
|
|
76
88
|
{{PROJECT_TOOLING}}
|
|
77
89
|
</project_tooling>
|
|
@@ -148,15 +160,21 @@ on the first attempt, not to discover problems after the fact.
|
|
|
148
160
|
it for cross-task context; re-read `{{PROGRESS_FILE}}` directly only when you need the latest
|
|
149
161
|
on-disk state (e.g. another task settled mid-session).
|
|
150
162
|
4. **Working tree state** — inspect the working tree for uncommitted changes before writing anything.
|
|
151
|
-
5. **
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
163
|
+
5. **Git log orientation** — run `git log --oneline -20` to see what prior tasks on this branch have
|
|
164
|
+
committed. Cross-reference with `<prior_progress>` to avoid re-implementing or conflicting with
|
|
165
|
+
their work. Changes already committed by a prior task are done — do not redo them.
|
|
166
|
+
6. **Environment** — review `<verify_script>` and `<pre_verify_results>` above. If
|
|
167
|
+
`<pre_verify_results>` is non-empty, the harness already verified the baseline — review those
|
|
168
|
+
results instead of re-running. If `<pre_verify_results>` is empty and no verify script is
|
|
169
|
+
configured, run the project's own verification commands (consult the project's AI context file
|
|
170
|
+
when present, or project config). If any check shows a pre-existing failure, stop immediately:
|
|
155
171
|
emit `task-blocked` with reason `"Pre-existing failure: [details]"`.
|
|
156
|
-
|
|
172
|
+
If `<retry_feedback>` is non-empty, a previous attempt's harness post-verify failed — address
|
|
173
|
+
that regression as the very first priority of this attempt before doing any other work.
|
|
174
|
+
7. **Conventions** — read project config to understand what is enforced: lint and formatter settings,
|
|
157
175
|
compiler config, test framework patterns (e.g. `*.test.ts` vs `*.spec.ts`, `__tests__/` vs
|
|
158
176
|
co-located).
|
|
159
|
-
|
|
177
|
+
8. **Existing patterns** — search for code similar to what you need to build. Matching existing
|
|
160
178
|
patterns is the single most important feedforward control — it prevents introducing new conventions
|
|
161
179
|
that conflict with neighbours.
|
|
162
180
|
|
|
@@ -176,17 +194,22 @@ Proceed to Phase 2 once Phase 1 passes.
|
|
|
176
194
|
verification criteria rather than signalling blocked. If steps appear incomplete relative to the
|
|
177
195
|
ticket, emit `task-blocked` rather than expanding scope — the planner may have scoped them
|
|
178
196
|
narrowly on purpose.
|
|
179
|
-
4. **
|
|
180
|
-
|
|
197
|
+
4. **After each meaningful change, run the cheapest check relevant to what you just touched** — for
|
|
198
|
+
example, the typecheck command or the test file for the module you edited — not the full suite.
|
|
199
|
+
The authoritative gate is Phase 3 step 2; interim runs are incremental sanity checks.
|
|
181
200
|
|
|
182
201
|
### Phase 3 — Completion
|
|
183
202
|
|
|
184
203
|
In order:
|
|
185
204
|
|
|
186
205
|
1. **Confirm all steps done** — every declared step has been completed.
|
|
187
|
-
2. **Run
|
|
188
|
-
|
|
189
|
-
|
|
206
|
+
2. **Run each `auto` criterion's command once** and fix any failures before proceeding. Do NOT run
|
|
207
|
+
the verify script from `<verify_script>` — the harness runs it after your turn as the
|
|
208
|
+
independent commit gate; running it yourself would duplicate that gate and inflate cost.
|
|
209
|
+
Exception: when the task defines no `auto` criteria, run the verify script once yourself to
|
|
210
|
+
confirm no regressions before signalling completion. When no verify script is configured either,
|
|
211
|
+
run the project's own check commands (as found during Phase 1 reconnaissance) before signalling
|
|
212
|
+
completion.
|
|
190
213
|
3. **Record verification results** — emit `task-verified` with the verbatim commands and their
|
|
191
214
|
combined stdout/stderr output in the `output` field.
|
|
192
215
|
4. **Propose the commit message** — emit `commit-message` with a real subject and a body explaining
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Continue — Round {{ROUND_NUMBER}}
|
|
2
|
+
|
|
3
|
+
<role>
|
|
4
|
+
You are the same AI coding agent, continuing the SAME task on a resumed session. You already have
|
|
5
|
+
the full task brief, the contract, and your own earlier work in this conversation — this prompt
|
|
6
|
+
does not repeat them. Your job for this call is one more round of the gen-eval loop: address the
|
|
7
|
+
evaluator's critique below and re-verify. The harness manages session lifecycle and context
|
|
8
|
+
compaction.
|
|
9
|
+
</role>
|
|
10
|
+
|
|
11
|
+
{{HARNESS_CONTEXT}}
|
|
12
|
+
|
|
13
|
+
<session_context>
|
|
14
|
+
This is a continuation turn — the brief, the contract, and your prior rounds are already in this
|
|
15
|
+
conversation's history. If this session somehow lacks that prior context (a resumed thread that
|
|
16
|
+
did not carry forward), re-read these on-disk files before acting — they are reachable via the
|
|
17
|
+
mounted directories:
|
|
18
|
+
|
|
19
|
+
- task contract — `{{CONTRACT_PATH}}`
|
|
20
|
+
- sprint journal — `{{PROGRESS_FILE}}` (append-only history of every prior task-attempt)
|
|
21
|
+
|
|
22
|
+
Read them only when the prior context is missing; when the conversation already carries the brief,
|
|
23
|
+
proceed directly to the critique below.
|
|
24
|
+
</session_context>
|
|
25
|
+
|
|
26
|
+
<plateau_directive>{{PLATEAU_DIRECTIVE_SECTION}}</plateau_directive>
|
|
27
|
+
|
|
28
|
+
<prior_critique>{{PRIOR_CRITIQUE_SECTION}}</prior_critique>
|
|
29
|
+
|
|
30
|
+
<retry_feedback>{{RETRY_FEEDBACK_SECTION}}</retry_feedback>
|
|
31
|
+
|
|
32
|
+
<pre_verify_results>{{PRE_VERIFY_RESULTS}}</pre_verify_results>
|
|
33
|
+
|
|
34
|
+
<prior_progress>
|
|
35
|
+
The most recent sprint-journal sections (decisions, changes, learnings, notes from prior
|
|
36
|
+
task-attempts) are below for quick reference. Honor prior decisions; do not re-litigate them
|
|
37
|
+
without a `decision` signal explaining why. When the block is empty there is no recent journal
|
|
38
|
+
context to apply.
|
|
39
|
+
|
|
40
|
+
{{PRIOR_PROGRESS}}
|
|
41
|
+
|
|
42
|
+
For the complete history — older than the excerpt above — read `{{PROGRESS_FILE}}` on disk.
|
|
43
|
+
</prior_progress>
|
|
44
|
+
|
|
45
|
+
<goal>
|
|
46
|
+
Address every dimension the evaluator flagged in `<prior_critique>`, then run each `auto`
|
|
47
|
+
criterion's command once. Do NOT run the verify script — the harness runs it after your turn as
|
|
48
|
+
the independent commit gate. Exception: when the task defines no `auto` criteria, run the verify
|
|
49
|
+
script once yourself. Emit `task-verified` with the verbatim command output, propose a
|
|
50
|
+
`commit-message` when you touched any file, and emit `task-complete` only after every flagged
|
|
51
|
+
dimension is resolved and every criterion command passes. Removing or disabling a test to make
|
|
52
|
+
verify pass counts as task failure — fix the implementation, not the test. When a flagged item is
|
|
53
|
+
genuinely blocked (missing dependency, contradictory input, unresolvable ambiguity), emit
|
|
54
|
+
`task-blocked` with the concrete reason instead of guessing.
|
|
55
|
+
</goal>
|
|
56
|
+
|
|
57
|
+
{{OUTPUT_CONTRACT_SECTION}}
|
|
@@ -100,11 +100,13 @@ Each task entry uses these fields:
|
|
|
100
100
|
`check === "manual"`. Use the project's own commands — read the project's AI context
|
|
101
101
|
file or manifest for the exact verification command this repository expects.
|
|
102
102
|
- **`blockedBy`** — `id`s of earlier tasks that must complete first.
|
|
103
|
-
- **`extraDimensions`** — optional kebab-case evaluator dimensions
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
103
|
+
- **`extraDimensions`** — optional kebab-case evaluator dimensions beyond the four floor
|
|
104
|
+
dimensions (correctness, completeness, safety, consistency). Attach an extra dimension
|
|
105
|
+
ONLY when an acceptance criterion explicitly demands a measurable property that no floor
|
|
106
|
+
dimension covers AND no manual criterion already encodes it. When in doubt, omit — the
|
|
107
|
+
floor dimensions are almost always sufficient. Example of a justified attachment:
|
|
108
|
+
`migration-safety` when the ticket requires a zero-downtime schema change that the four
|
|
109
|
+
floor dimensions cannot score on their own. Cap: 2–3 per task; hard max 6.
|
|
108
110
|
|
|
109
111
|
If you cannot produce a sound plan, emit the `task-plan` signal with `tasksJson` set to:
|
|
110
112
|
|
|
@@ -173,6 +175,12 @@ Right size:
|
|
|
173
175
|
structure.
|
|
174
176
|
- Micro-refactoring tasks (add directive, remove import) — fold into the task that needs
|
|
175
177
|
them.
|
|
178
|
+
- **Ending steps with "run the verification commands" or "run all the checks."** Verification
|
|
179
|
+
belongs in `verificationCriteria` — the harness and the evaluator execute it. A final step
|
|
180
|
+
that re-runs the full suite only duplicates the post-task gate and inflates generator
|
|
181
|
+
cost. Exception: a step MAY run a specific check when a later step depends on its output
|
|
182
|
+
(e.g. "run the migration dry-run and confirm the schema diff before writing the rollback
|
|
183
|
+
script").
|
|
176
184
|
|
|
177
185
|
### Dependency Graph
|
|
178
186
|
|
|
@@ -259,8 +267,7 @@ Good — precise steps with file paths and pattern references:
|
|
|
259
267
|
"Add AuthContext provider in src/contexts/AuthContext.tsx wrapping the app — follow the existing ThemeContext pattern",
|
|
260
268
|
"Create useAuth hook in src/hooks/useAuth.ts exposing auth state and actions",
|
|
261
269
|
"Add ProtectedRoute wrapper component in src/components/ProtectedRoute.tsx",
|
|
262
|
-
"Write unit tests in src/services/__tests__/auth.test.ts — follow patterns in src/services/__tests__/user.test.ts"
|
|
263
|
-
"Run the project's verification commands (read the project's AI context file or manifest for the exact commands — typecheck, lint, and tests must all pass)"
|
|
270
|
+
"Write unit tests in src/services/__tests__/auth.test.ts — follow patterns in src/services/__tests__/user.test.ts"
|
|
264
271
|
],
|
|
265
272
|
"verificationCriteria": [
|
|
266
273
|
{
|
|
@@ -328,8 +335,10 @@ Read the repositories mounted under `<repositories>` to:
|
|
|
328
335
|
1. Read repo instruction files (`CLAUDE.md`, `AGENTS.md`, `.github/copilot-instructions.md`)
|
|
329
336
|
when present.
|
|
330
337
|
2. Skim project structure and manifests (`package.json`, `pyproject.toml`, etc.).
|
|
331
|
-
3.
|
|
332
|
-
|
|
338
|
+
3. Run `git log --oneline -20` per repository so you don't plan tasks that re-implement
|
|
339
|
+
already-landed work.
|
|
340
|
+
4. Find similar implementations to mirror existing patterns.
|
|
341
|
+
5. Extract verification commands (build, test, lint, typecheck).
|
|
333
342
|
|
|
334
343
|
Remember: you are in the per-sprint plan unit root, not inside any repository. Use the
|
|
335
344
|
repository paths from `<repositories>` as the roots for all file reads and searches.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ralphctl",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.12.0",
|
|
4
4
|
"description": "Agent harness for long-running AI coding tasks — orchestrates Claude Code, GitHub Copilot, and OpenAI Codex across repositories",
|
|
5
5
|
"homepage": "https://github.com/lukas-grigis/ralphctl",
|
|
6
6
|
"type": "module",
|