ralphctl 0.23.0 → 0.25.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/chunk-BAAV7AGY.mjs +11 -0
- package/dist/{cli-J3MI4B6A.mjs → cli-GQBXYNFN.mjs} +9199 -7113
- package/dist/cli.mjs +3 -1
- package/dist/manifest.json +5 -3
- package/dist/prompts/_partials/approval-gate.md +19 -0
- package/dist/prompts/_partials/evaluator-grading-rules.md +25 -0
- package/dist/prompts/_partials/harness-context.md +2 -9
- package/dist/prompts/_partials/task-fields.md +4 -4
- package/dist/prompts/_partials/validation-checklist.md +3 -4
- package/dist/prompts/apply-feedback/template.md +8 -12
- package/dist/prompts/create-pr/template.md +0 -5
- package/dist/prompts/detect-scripts/template.md +38 -71
- package/dist/prompts/detect-skills/template.md +14 -19
- package/dist/prompts/distill-learnings/template.md +18 -16
- package/dist/prompts/evaluate/template.md +23 -50
- package/dist/prompts/evaluate-continuation/template.md +18 -38
- package/dist/prompts/ideate/template.md +13 -12
- package/dist/prompts/implement/template.md +29 -9
- package/dist/prompts/implement-continuation/template.md +5 -5
- package/dist/prompts/implement-crash-resume/template.md +33 -0
- package/dist/prompts/plan/template.md +21 -30
- package/dist/prompts/readiness/template.md +19 -16
- package/dist/prompts/refine/template.md +11 -14
- package/dist/prompts/reproduce/template.md +20 -9
- package/dist/prompts/select-candidate/template.md +7 -7
- package/dist/use-no-color-NEMCAVY3.mjs +9 -0
- package/package.json +2 -1
- package/dist/prompts/_partials/evaluation-checkpoint.md +0 -23
|
@@ -15,12 +15,16 @@ bottom of this prompt.
|
|
|
15
15
|
|
|
16
16
|
<success_criteria>
|
|
17
17
|
|
|
18
|
-
- `agents-md-proposal` signal emitted with `tag: "{{WIRE_TAG}}"`
|
|
18
|
+
- `agents-md-proposal` signal emitted with `tag: "{{WIRE_TAG}}"`. `content` is non-empty when
|
|
19
|
+
`<existing_context_file>` is empty (a fresh file); with an existing file it holds only additions and may be
|
|
20
|
+
an empty string when there is nothing to add.
|
|
19
21
|
- Every tech-stack claim in `content` is backed by a quoted file path or file content, not inferred.
|
|
20
|
-
-
|
|
21
|
-
below — the exact target and ceiling are provider-specific.
|
|
22
|
-
|
|
23
|
-
|
|
22
|
+
- The resulting file follows the length guidance for `{{CURRENT_TOOL}}` in `<target_file_conventions>`
|
|
23
|
+
below — the exact target and ceiling are provider-specific. The ceiling applies to the existing file plus
|
|
24
|
+
your additions, so count the existing lines and keep additions within the remaining room.
|
|
25
|
+
- When an existing context file is supplied in `<existing_context_file>`, `content` holds ONLY the new H2
|
|
26
|
+
sections to append — never the existing body; the harness appends them and keeps the existing file
|
|
27
|
+
byte-for-byte.
|
|
24
28
|
- Setup and verify skill proposals, when emitted, cite only commands that resolve in this specific repo
|
|
25
29
|
(shell commands verified against manifest files, not assumed from language defaults).
|
|
26
30
|
- `signals.json` is valid JSON and passes the harness schema check.
|
|
@@ -38,8 +42,6 @@ bottom of this prompt.
|
|
|
38
42
|
</target_file_conventions>
|
|
39
43
|
</inputs>
|
|
40
44
|
|
|
41
|
-
{{HARNESS_CONTEXT}}
|
|
42
|
-
|
|
43
45
|
<constraints>
|
|
44
46
|
|
|
45
47
|
**Read-only scope.** Read configuration and metadata files only — `package.json`, `pyproject.toml`,
|
|
@@ -71,12 +73,12 @@ for genuinely surprising rules — overuse erodes their meaning.
|
|
|
71
73
|
- Credentials, user-specific paths, or commands that touch remote services.
|
|
72
74
|
- Standard language conventions the agent already knows.
|
|
73
75
|
|
|
74
|
-
**Existing-context rule (fires when `<existing_context_file>`
|
|
75
|
-
The supplied prose is authoritative. The `agents-md-proposal`
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
76
|
+
**Existing-context rule (fires when `<existing_context_file>` is non-empty; an empty one means no file exists yet — emit a fresh body).**
|
|
77
|
+
The supplied prose is authoritative and the harness preserves it byte-for-byte. The `agents-md-proposal`
|
|
78
|
+
signal's `content` MUST contain only your additions, as new H2 sections — do not repeat, reword, or
|
|
79
|
+
summarise the existing body, and do not modify, prune, or merge into existing sections; the harness appends
|
|
80
|
+
`content` after the existing file. When you have nothing to add, still emit the `agents-md-proposal` signal
|
|
81
|
+
with an empty `content` string.
|
|
80
82
|
|
|
81
83
|
**Script safety (applies to setup and verify skill bodies).** Every command you document must resolve in
|
|
82
84
|
this repo. Cite a setup command only when its manifest file is present (a `package.json` install command
|
|
@@ -132,8 +134,9 @@ not what is typical for the apparent stack.
|
|
|
132
134
|
Draft each surviving section against the inclusion test. Drop any section an experienced engineer could
|
|
133
135
|
derive from the manifest or directory tree.
|
|
134
136
|
|
|
135
|
-
When `<existing_context_file>`
|
|
136
|
-
|
|
137
|
+
When `<existing_context_file>` is non-empty (empty means no file exists yet), the harness keeps the
|
|
138
|
+
existing prose first, byte-for-byte. Your `content` is only the additions — new H2 sections, never inline or
|
|
139
|
+
merged into existing ones.
|
|
137
140
|
|
|
138
141
|
### Phase 4 — Output
|
|
139
142
|
|
|
@@ -151,7 +154,7 @@ Do not invent stack claims without evidence.
|
|
|
151
154
|
### Signal semantics
|
|
152
155
|
|
|
153
156
|
- `agents-md-proposal` — REQUIRED. `tag` MUST equal `"{{WIRE_TAG}}"`. `content` is the project context
|
|
154
|
-
file body.
|
|
157
|
+
file body — or, when `<existing_context_file>` is non-empty, only the new sections to append to it.
|
|
155
158
|
- `setup-skill-proposal` — optional. Multi-paragraph markdown body describing the project's setup
|
|
156
159
|
convention. The harness lands it as `setup/SKILL.md`. Omit entirely when no setup skill is warranted.
|
|
157
160
|
- `verify-skill-proposal` — optional. Same shape as the setup skill but for verification (typecheck /
|
|
@@ -37,13 +37,11 @@ If `<prior_progress>` is empty, no prior work has been recorded for this sprint
|
|
|
37
37
|
If the `<context>` block above is empty or absent, no upstream issue body was available.
|
|
38
38
|
</inputs>
|
|
39
39
|
|
|
40
|
-
{{HARNESS_CONTEXT}}
|
|
41
|
-
|
|
42
40
|
<constraints>
|
|
43
|
-
-
|
|
41
|
+
- Stay implementation-agnostic. Frame requirements as observable behaviour ("user can filter by
|
|
44
42
|
date range"), not technical decisions ("add a SQL WHERE clause"). The planner that runs after you
|
|
45
43
|
needs maximum flexibility on HOW; your job is WHAT.
|
|
46
|
-
-
|
|
44
|
+
- Do not explore the repository — no source files are mounted in this session; only the output
|
|
47
45
|
directory is writable. If a question requires source context, note it in the requirements body
|
|
48
46
|
as an open question requiring repo investigation.
|
|
49
47
|
- One concern per question. Combining "what should it do AND how should it look" forces a fuzzy
|
|
@@ -63,7 +61,7 @@ ticket omits.
|
|
|
63
61
|
|
|
64
62
|
Then identify, in order:
|
|
65
63
|
|
|
66
|
-
1. What is already clear and
|
|
64
|
+
1. What is already clear and doesn't need clarification.
|
|
67
65
|
2. What is ambiguous, missing, or underspecified.
|
|
68
66
|
3. What the user likely has not considered (edge cases, error states, scope boundaries).
|
|
69
67
|
|
|
@@ -138,7 +136,7 @@ Header: "Formats"
|
|
|
138
136
|
multiSelect: true
|
|
139
137
|
Options:
|
|
140
138
|
- "CSV (Recommended)" — "Universal compatibility; simple structure."
|
|
141
|
-
- "JSON
|
|
139
|
+
- "JSON" — "API-friendly; structured data."
|
|
142
140
|
- "PDF" — "Human-readable reports; requires additional library."
|
|
143
141
|
```
|
|
144
142
|
|
|
@@ -148,10 +146,9 @@ Stop asking questions once every item in the Step 5 pre-output quality checklist
|
|
|
148
146
|
|
|
149
147
|
### Step 4 — Present requirements for approval
|
|
150
148
|
|
|
151
|
-
|
|
152
|
-
Make it easy to scan.
|
|
149
|
+
{{APPROVAL_GATE}}
|
|
153
150
|
|
|
154
|
-
|
|
151
|
+
The approval question:
|
|
155
152
|
|
|
156
153
|
```
|
|
157
154
|
Question: "Does this look correct? Any changes needed?"
|
|
@@ -162,12 +159,12 @@ Options:
|
|
|
162
159
|
- "Give feedback" — "Type specific corrections in my own words."
|
|
163
160
|
```
|
|
164
161
|
|
|
165
|
-
If the user selects "Needs changes" or "Give feedback", apply their input
|
|
166
|
-
until approved.
|
|
162
|
+
If the user selects "Needs changes" or "Give feedback", apply their input, then run the approval
|
|
163
|
+
gate again from the top. Iterate until approved.
|
|
167
164
|
|
|
168
165
|
### Step 5 — Pre-output quality check
|
|
169
166
|
|
|
170
|
-
Before emitting the signal, verify
|
|
167
|
+
Before emitting the signal, verify all of these are true:
|
|
171
168
|
|
|
172
169
|
- [ ] Problem statement is clear and agreed.
|
|
173
170
|
- [ ] Every requirement has acceptance criteria covering happy path, an alternate path, and an
|
|
@@ -245,7 +242,7 @@ them with `---`:
|
|
|
245
242
|
|
|
246
243
|
<output_contract>
|
|
247
244
|
Write `signals.json` to the output directory. Only `signals.json` is read by the harness; all
|
|
248
|
-
other session output is forensic and not persisted as data. The file
|
|
245
|
+
other session output is forensic and not persisted as data. The file must contain exactly one
|
|
249
246
|
`refined-ticket` signal. The harness validates this file after the session exits; a missing file,
|
|
250
247
|
unparseable JSON, or zero/multiple `refined-ticket` entries are all validation failures.
|
|
251
248
|
|
|
@@ -266,7 +263,7 @@ signal with whatever you have, appending a final `## Unresolved` section to the
|
|
|
266
263
|
gap. Also emit a `note` signal whose `text` explains what is missing. Do not silently invent
|
|
267
264
|
requirements.
|
|
268
265
|
|
|
269
|
-
|
|
266
|
+
The only file you write is `signals.json`. Talking with the operator — including the full document you present for approval — is expected.
|
|
270
267
|
|
|
271
268
|
{{OUTPUT_CONTRACT_SECTION}}
|
|
272
269
|
</output_contract>
|
|
@@ -5,12 +5,15 @@ as reported, run it to capture the failure, and report your findings by writing
|
|
|
5
5
|
NOT fix the defect — that is a separate session's job, run after yours.
|
|
6
6
|
</role>
|
|
7
7
|
|
|
8
|
-
{{
|
|
8
|
+
{{AUTONOMOUS_OPERATION}}
|
|
9
9
|
|
|
10
10
|
<goal>
|
|
11
11
|
Reproduce the defect described below against the project at `{{PROJECT_PATH}}`: write one new test that
|
|
12
12
|
fails for the reported reason, run it, and record the test path, the exact run command, and the observed
|
|
13
|
-
failure by writing `signals.json` to the output directory.
|
|
13
|
+
failure by writing `signals.json` to the output directory. If the description reads as vague or
|
|
14
|
+
under-specified, use the reading most consistent with the done criteria and record the interpretation in
|
|
15
|
+
`notes`. If a bounded attempt cannot make a test fail for the reported reason, that is a valid result —
|
|
16
|
+
report it as not reproduced (see Phase 3) rather than forcing a failure.
|
|
14
17
|
</goal>
|
|
15
18
|
|
|
16
19
|
<success_criteria>
|
|
@@ -24,6 +27,8 @@ failure by writing `signals.json` to the output directory.
|
|
|
24
27
|
up empty — an empty list is a meaningful answer, not an omission.
|
|
25
28
|
- No production code is modified. No existing test is modified, weakened, skipped, or deleted.
|
|
26
29
|
- A `reproduction` signal is written to `signals.json` with the observed failure output.
|
|
30
|
+
- A bounded attempt that ends in `"reproduced": false` with a concrete `reason` (and the added test
|
|
31
|
+
removed) is also a valid outcome — never a fabricated failure.
|
|
27
32
|
|
|
28
33
|
</success_criteria>
|
|
29
34
|
|
|
@@ -66,8 +71,8 @@ conventions (framework, file naming, directory layout) as found in the existing
|
|
|
66
71
|
|
|
67
72
|
**The test must fail for the reported reason.** Run it after writing it. If it errors on setup, a missing
|
|
68
73
|
import, or an unrelated fixture problem, that is not a reproduction yet — fix the harness issue in the
|
|
69
|
-
test itself (never in production code)
|
|
70
|
-
not an accident of the test's own plumbing.
|
|
74
|
+
test itself (never in production code), within the two-revision bound in Phase 3, until the failure you
|
|
75
|
+
observe is the actual reported behaviour, not an accident of the test's own plumbing.
|
|
71
76
|
|
|
72
77
|
**Do not commit.** Leave the new test uncommitted in the working tree — a later session commits it as
|
|
73
78
|
part of its own work, not this one.
|
|
@@ -100,7 +105,11 @@ test file following the project's own conventions.
|
|
|
100
105
|
|
|
101
106
|
Run the new test in isolation using the project's own test runner. Confirm it fails, and confirm the
|
|
102
107
|
failure is the reported defect — not a setup, import, or fixture error in the test itself. If it passes,
|
|
103
|
-
or fails for the wrong reason, revise the test
|
|
108
|
+
or fails for the wrong reason, revise the test and run it again — but stop after two revisions. If the
|
|
109
|
+
test still does not fail for the reported reason (the defect looks already fixed, depends on an
|
|
110
|
+
environment you cannot reach, or the report is not actionable), do not keep looping and do not fabricate
|
|
111
|
+
a failure: remove the test you added and report not reproduced in Phase 4. Otherwise record the exact
|
|
112
|
+
command.
|
|
104
113
|
|
|
105
114
|
{{EVIDENCE_BOUND}}
|
|
106
115
|
|
|
@@ -112,6 +121,10 @@ the observed failure excerpt, the relevant existing tests you found (possibly em
|
|
|
112
121
|
than a new one created). A `note` signal is available for anything that doesn't fit `reproduction`
|
|
113
122
|
itself — use it sparingly.
|
|
114
123
|
|
|
124
|
+
When the bounded attempt did not reproduce the defect, still emit the `reproduction` signal, with
|
|
125
|
+
`"reproduced": false` and a `"reason"` naming what you tried and why it did not fail (`testPath`,
|
|
126
|
+
`runCommand` and `observedFailure` may be empty strings). Omit `reproduced` on a successful reproduction.
|
|
127
|
+
|
|
115
128
|
<output_contract>
|
|
116
129
|
|
|
117
130
|
Only `signals.json` is read by the harness; all other session output is forensic and not persisted as
|
|
@@ -119,9 +132,7 @@ data.
|
|
|
119
132
|
|
|
120
133
|
{{OUTPUT_CONTRACT_SECTION}}
|
|
121
134
|
|
|
122
|
-
Emit exactly one `reproduction` signal — it is required, not optional.
|
|
123
|
-
|
|
124
|
-
stopping short; the `notes` field is where you record any interpretation you had to make. A `note`
|
|
125
|
-
signal may accompany it for anything else worth flagging, but never in place of it.
|
|
135
|
+
Emit exactly one `reproduction` signal — it is required, not optional. The `notes` field is where you
|
|
136
|
+
record any interpretation you had to make (see `<goal>`). A `note` signal may accompany it for anything else worth flagging, but never in place of it.
|
|
126
137
|
|
|
127
138
|
</output_contract>
|
|
@@ -6,8 +6,6 @@ better verdict than comparing summaries, at much higher cost. Skepticism is your
|
|
|
6
6
|
claim of success in a summary as unproven until the summary itself cites concrete evidence for it.
|
|
7
7
|
</role>
|
|
8
8
|
|
|
9
|
-
{{HARNESS_CONTEXT}}
|
|
10
|
-
|
|
11
9
|
<goal>
|
|
12
10
|
Compare Candidate 1 and Candidate 2 against the task specification below and pick the one more likely to
|
|
13
11
|
be a correct, complete, maintainable solution. Write exactly one `candidate-selection` signal to
|
|
@@ -16,7 +14,8 @@ be a correct, complete, maintainable solution. Write exactly one `candidate-sele
|
|
|
16
14
|
|
|
17
15
|
<success_criteria>
|
|
18
16
|
|
|
19
|
-
- The verdict names a winner — `1` or `2` —
|
|
17
|
+
- The verdict names a winner — `1` or `2` — or declares a tie (`0`) when the evidence genuinely does not
|
|
18
|
+
separate them. It never names both.
|
|
20
19
|
- The rationale cites concrete evidence from the summaries (a verification outcome, a specific file, a
|
|
21
20
|
notable signal) — not a general impression like "looks more thorough".
|
|
22
21
|
- A candidate's own confident claim of success, unsupported by a cited verification outcome, counts for
|
|
@@ -55,8 +54,9 @@ be a correct, complete, maintainable solution. Write exactly one `candidate-sele
|
|
|
55
54
|
one that claims full success without evidence.
|
|
56
55
|
- When both summaries are similarly strong on verification, prefer the one whose changed-files list stays
|
|
57
56
|
closest to the task's declared scope.
|
|
58
|
-
- When the two are genuinely indistinguishable on the evidence given,
|
|
59
|
-
|
|
57
|
+
- When the two are genuinely indistinguishable on the evidence given, declare a tie rather than inventing
|
|
58
|
+
a preference — state in the rationale what you compared and why nothing separates them. The order the
|
|
59
|
+
candidates are shown in carries no meaning; do not favor a candidate for being listed first or last.
|
|
60
60
|
|
|
61
61
|
</constraints>
|
|
62
62
|
|
|
@@ -83,7 +83,7 @@ criterion — do not penalise both candidates for a gap in what they chose to re
|
|
|
83
83
|
### Phase 3 — Decide and report
|
|
84
84
|
|
|
85
85
|
Weigh the per-criterion comparison from Phase 2 together with the evidence-over-confidence rule above.
|
|
86
|
-
Pick the stronger candidate. Write `signals.json` as described in `<output_contract>` below with the
|
|
86
|
+
Pick the stronger candidate, or declare a tie. Write `signals.json` as described in `<output_contract>` below with the
|
|
87
87
|
winning index and a rationale that cites the specific evidence that decided it.
|
|
88
88
|
|
|
89
89
|
<output_contract>
|
|
@@ -94,6 +94,6 @@ data.
|
|
|
94
94
|
{{OUTPUT_CONTRACT_SECTION}}
|
|
95
95
|
|
|
96
96
|
Emit exactly one `candidate-selection` signal — no other signal kinds. `winner` is `1` or `2`, matching
|
|
97
|
-
the `<candidate_1>` / `<candidate_2>` labelling above.
|
|
97
|
+
the `<candidate_1>` / `<candidate_2>` labelling above, or `0` for a tie.
|
|
98
98
|
|
|
99
99
|
</output_contract>
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ralphctl",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.25.0",
|
|
4
4
|
"description": "A ralph harness for long-running AI coding tasks — a generator-evaluator loop that drives Claude Code, GitHub Copilot CLI, OpenAI Codex CLI, OpenCode or Grok Build CLI across one or more repositories",
|
|
5
5
|
"homepage": "https://github.com/lukas-grigis/ralphctl",
|
|
6
6
|
"type": "module",
|
|
@@ -103,6 +103,7 @@
|
|
|
103
103
|
"coverage:unused": "tsx scripts/find-unused.ts",
|
|
104
104
|
"deadcode": "knip",
|
|
105
105
|
"skills:update": "tsx scripts/sync-skills.ts",
|
|
106
|
+
"eval": "tsx scripts/eval/main.ts",
|
|
106
107
|
"lint": "eslint . --max-warnings 0",
|
|
107
108
|
"lint:fix": "eslint . --fix",
|
|
108
109
|
"format": "prettier --write .",
|
|
@@ -1,23 +0,0 @@
|
|
|
1
|
-
```json
|
|
2
|
-
{
|
|
3
|
-
"schemaVersion": 1,
|
|
4
|
-
"signals": [
|
|
5
|
-
{
|
|
6
|
-
"type": "evaluation",
|
|
7
|
-
"status": "failed",
|
|
8
|
-
"dimensions": [
|
|
9
|
-
{ "dimension": "correctness", "passed": false, "finding": "assessment in progress" },
|
|
10
|
-
{ "dimension": "completeness", "passed": false, "finding": "assessment in progress" },
|
|
11
|
-
{ "dimension": "safety", "passed": false, "finding": "assessment in progress" },
|
|
12
|
-
{ "dimension": "consistency", "passed": false, "finding": "assessment in progress" },
|
|
13
|
-
{ "dimension": "robustness", "passed": false, "applicable": false, "finding": "assessment in progress" }
|
|
14
|
-
],
|
|
15
|
-
"timestamp": "2026-01-01T00:00:00.000Z"
|
|
16
|
-
}
|
|
17
|
-
]
|
|
18
|
-
}
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
Robustness carries the optional `applicable` field shown above — set it to `false` only if your
|
|
22
|
-
review determines the change touches no error/failure path (with the real reason in `finding`), or
|
|
23
|
-
omit it (default `true`) once you record an actual pass/fail.
|