@robmclarty/volley 0.11.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +23 -0
- package/README.md +248 -21
- package/dist/cli.js +3273 -1646
- package/dist/cli.js.map +1 -1
- package/dist/index.d.ts +120 -11
- package/dist/index.js +1210 -394
- package/dist/index.js.map +1 -1
- package/package.json +3 -1
- package/src/critic/presets/reviewer.md +7 -0
- package/src/tester/presets/tester.md +29 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,28 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v0.12.0 — 2026-11-09
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
- **`--effort-ladder low,medium,high` runs each build one effort rung further up, off by default.** Or `VOLLEY_EFFORT_LADDER`, or `effort_ladder` in a config file. Rungs come from `low`, `medium`, `high`, `xhigh`, and `max`, and the top rung repeats. A build climbs only after one that didn't end the run, and it records why: `check_failed`, else `critic_rejected`. Each build prints its rung (`[iter 2] rung medium (climbed: check_failed)`), and each iteration's `summary.json` records `rung`, `climbed` and `effort_ignored`. On `claude_cli` the effort reaches Claude Code as `CLAUDE_CODE_EFFORT_LEVEL`. On `ollama` and `lmstudio`, which ignore effort, the ladder is flat and config warns. A resumed run keeps its ladder and picks up at the rung it had reached.
|
|
7
|
+
- **`--critic-effort` sets the critic's effort on its own.** Or `VOLLEY_CRITIC_EFFORT`, or `critic_effort` in a config file. It takes the same five levels and defaults to `high` once a ladder is set. With neither set, the critic is sent no effort, as before.
|
|
8
|
+
- **`--tester on` writes a failing test from the criteria before the first build, off by default.** Or `VOLLEY_TESTER`, or `tester` in a config file. The tester runs on the critic's seat at the critic's effort; `--tester-model` (or `VOLLEY_TESTER_MODEL`) overrides the model. It writes only tests and fixtures under the gate paths, never `package.json`, a `*.config.*` file or `.github/`, and has no shell. The check must then fail. If it passes, the run ends as `tester_passed_early` (exit 2) with no builder spend. The tester needs a check, and a tester on any provider but `claude_cli` needs the same containment a builder does. Its spend (`tester_cost_usd`) and writes (`tester_changes`) are recorded apart, so its test isn't counted as a builder gate edit, and the critic reviews the tester's test, reporting any problems in an optional `test_findings`.
|
|
9
|
+
- **The tester runs under `volley acp` too.** Its writes go through Zed's approval prompts with their diff, as the builder's do, and a path it may not write is refused before you're asked. A `claude_cli` tester is refused at startup. Round 1 shows the tester's test and the build as two headings.
|
|
10
|
+
- **`volley run --protocol runciter.run/1` lets runciter's `process` kind run volley with no adapter.** volley reads one `RunInput` on stdin, writes exactly one `RunOutput` on stdout, puts `runciter.progress` and `runciter.usage` lines on stderr, and exits 0, 1 or 2 by the protocol's own table. It maps the input as runciter's `volley` kind did: the agent's prompt plus `context.md`, the task's criteria, the check from the stage's `verify`, state under the run directory, and the critic's last feedback as a `critic-feedback.md` artifact. It refuses `--worktree`, `--git`, `--json` and a positional prompt. The README's new "Run volley from runciter" section has an agent-file example and the exit table. `volley run` is now another name for `volley`.
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
- **Each iteration of a run's trajectory has two more spans.** The flow gained `test` and `work` branches, which pass straight through when the tester is off, so the builder's and the critic's calls, the output, and the exit codes are unchanged; the diagram draws the new branches.
|
|
14
|
+
|
|
15
|
+
### Internal
|
|
16
|
+
- `keyframes.md` lists `src/protocol/` under M1 (cli), and its C2 constraint now covers the tester. The build's record is in `.plumbbob/builds/2026-11-08-move-1-tester-seat-effort-ladder-and-runciter-run-1/`.
|
|
17
|
+
|
|
18
|
+
## v0.11.1 — 2026-11-04
|
|
19
|
+
|
|
20
|
+
### Internal
|
|
21
|
+
- **volley's intent is written down in `keyframes.md`.** It holds four principles, a seven-part map of `src/`, eleven constraints and eight scenarios, each linked to the test or rule that holds it, twelve decisions, one open question and five ideas not yet built. `AGENTS.md` tells agents how to treat each kind of item, and `CLAUDE.md` imports it, so Claude Code reads it.
|
|
22
|
+
- **`pnpm check` checks `keyframes.md` against the repo.** A new `intent` check runs `keyframes check --strict`, from `@robmclarty/keyframes` 0.3.0 as a dev dependency. It fails when a test, rule or path the plan points at is renamed or moved, or when the plan picks up a warning. It's a gate for developing volley and never runs in a volley run.
|
|
23
|
+
- **Agents ask before changing `keyframes.md`.** Claude Code asks before it edits the file or runs a shell command that names it. A pre-commit hook shows a macOS dialog before any commit that changes it can land, whichever agent made the change, and refuses where no dialog can be shown. A clone turns the hook on with `git config core.hooksPath .githooks`.
|
|
24
|
+
- **The published package is unchanged.** No command, flag or output changed since v0.11.0.
|
|
25
|
+
|
|
3
26
|
## v0.11.0 — 2026-11-02
|
|
4
27
|
|
|
5
28
|
### Added
|
package/README.md
CHANGED
|
@@ -101,6 +101,80 @@ workspace filesystem, the critic's feedback, and a failed check's output. Swap t
|
|
|
101
101
|
the loop optimizes for: `--critic reviewer` (default), `optimizer`,
|
|
102
102
|
`researcher`, or a path to your own system prompt.
|
|
103
103
|
|
|
104
|
+
### The effort ladder
|
|
105
|
+
|
|
106
|
+
`--effort-ladder low,medium,high` runs each build one rung further up than the
|
|
107
|
+
last: build 1 at `low`, build 2 at `medium`, and every build from the third on
|
|
108
|
+
at `high`, the top rung repeating to `--max-iterations`. A build climbs only
|
|
109
|
+
because the one before it didn't end the run: its check failed, or the critic
|
|
110
|
+
sent it back. So an easy task converges on a cheap first build, and a hard one
|
|
111
|
+
spends more only after a cheap attempt failed. A rung sets the call's effort,
|
|
112
|
+
never its model, so the seat stays the same seat on every rung; on `claude_cli`
|
|
113
|
+
it reaches Claude Code as `CLAUDE_CODE_EFFORT_LEVEL`. The rungs are `low`,
|
|
114
|
+
`medium`, `high`, `xhigh`, and `max`. `none` is refused, since on `claude_cli`
|
|
115
|
+
it sets nothing and leaves Claude Code at its own default, which can sit above
|
|
116
|
+
`low`.
|
|
117
|
+
|
|
118
|
+
The critic's effort is set apart, with `--critic-effort`, and defaults to `high`
|
|
119
|
+
once a ladder is set: reading a diff costs far less than writing it, and the
|
|
120
|
+
critic is what makes a cheap build safe. Without a ladder neither seat is sent
|
|
121
|
+
an effort unless `--critic-effort` names one, so a run without either flag makes
|
|
122
|
+
the calls it always has. Each build prints its rung as it opens
|
|
123
|
+
(`[iter 2] rung medium (climbed: check_failed)`), and the run's summaries record
|
|
124
|
+
them (see [Run artifacts](#run-artifacts)).
|
|
125
|
+
|
|
126
|
+
**The ladder is flat on Ollama and LM Studio.** Their transports drop effort, so
|
|
127
|
+
every rung runs the same there: volley warns when a ladder is set on such a
|
|
128
|
+
builder, marks each iteration `effort_ignored`, and the summary reads
|
|
129
|
+
`ladder_flat: true`. No rung means anything on a local model yet.
|
|
130
|
+
|
|
131
|
+
### The tester seat
|
|
132
|
+
|
|
133
|
+
`--tester on` adds a third seat that runs once, before the first build: a
|
|
134
|
+
tester that turns the acceptance criteria into a failing test. A critic can be
|
|
135
|
+
argued with about prose criteria, and a test can't. The tester reads the
|
|
136
|
+
workspace, writes tests and fixtures where the check will pick them up, and
|
|
137
|
+
then the check runs, and must fail on what it wrote. A red check goes on to the
|
|
138
|
+
build, and the builder's prompt names the tester's files and shows the check's
|
|
139
|
+
failure. A check that passes ends the run there, with no build paid for, as
|
|
140
|
+
`tester_passed_early` (exit 2): a test that passes before any work is either
|
|
141
|
+
vacuous or proof the work is already done. Later iterations start at the build,
|
|
142
|
+
with the test already in place. It's off by default.
|
|
143
|
+
|
|
144
|
+
- **Its seat.** The tester takes the critic's provider, URL, and effort, and the
|
|
145
|
+
critic's model unless `--tester-model` names one. Its spend is
|
|
146
|
+
`tester_cost_usd`, apart from the builder's and the critic's, and the cost cap
|
|
147
|
+
is checked after it: a tester that crosses the cap ends the run before the
|
|
148
|
+
check and the build.
|
|
149
|
+
- **What it may write.** Only paths a gate pattern matches (`--gate-paths`, or
|
|
150
|
+
the defaults under [What the builder changed](#what-the-builder-changed)),
|
|
151
|
+
and never `package.json`, a `*.config.*` file, or anything under `.github/`,
|
|
152
|
+
since a tester that could rewrite the check's configuration could make its
|
|
153
|
+
own result red or green. A write anywhere else is refused, naming the path.
|
|
154
|
+
It has no shell. On `claude_cli` it runs in Claude Code's default permission
|
|
155
|
+
mode, allowed `Read` anywhere in the build root and one `Edit` rule per test
|
|
156
|
+
and fixture pattern, with `Bash` and edits to those three kinds of file
|
|
157
|
+
denied.
|
|
158
|
+
- **It needs a check.** `--tester on` with `--check none`, or with a `--check
|
|
159
|
+
auto` that finds no checkride, is refused before anything runs, since a test
|
|
160
|
+
can't go red without a check.
|
|
161
|
+
- **Its test is its own.** Each file the tester wrote is hashed after it runs.
|
|
162
|
+
A build that leaves one at that hash isn't reported as a gate edit there, and
|
|
163
|
+
a build that changes it is, as any gate edit is. Under `--git`, a
|
|
164
|
+
`volley iter 1: test` commit lands before the build's.
|
|
165
|
+
- **The critic reads it too.** Its prompt lists the tester's files and asks
|
|
166
|
+
whether they test the criteria, and it names any gaps as `test_findings`. A
|
|
167
|
+
vacuous test doesn't decide the verdict by itself: the critic checks by hand
|
|
168
|
+
what the test misses.
|
|
169
|
+
|
|
170
|
+
**It's contained as a builder is.** The check runs what the tester writes, so a
|
|
171
|
+
tester on any provider but `claude_cli` runs only inside the
|
|
172
|
+
[sandbox](#local-builder), unless `--allow-unsandboxed-builder` opts out. A
|
|
173
|
+
`claude_cli` builder stays on the host, so with one, a local tester needs the
|
|
174
|
+
opt-out. A `claude_cli` tester runs under its own permission list, on the host.
|
|
175
|
+
Under [`volley acp`](#run-volley-from-zed) the tester's writes ask as the
|
|
176
|
+
builder's do, in a round of their own, and a `claude_cli` tester is refused.
|
|
177
|
+
|
|
104
178
|
### The flow
|
|
105
179
|
|
|
106
180
|
Under the hood the loop is one [fascicle](https://www.npmjs.com/package/fascicle)
|
|
@@ -112,22 +186,33 @@ the guard that decides when the loop stops.
|
|
|
112
186
|
|
|
113
187
|
<!-- flow diagram: pnpm diagram -->
|
|
114
188
|
```text
|
|
115
|
-
volley
|
|
116
|
-
├─ iteration
|
|
117
|
-
│ ├─
|
|
118
|
-
│ │
|
|
119
|
-
│ ├─
|
|
120
|
-
│ │
|
|
189
|
+
volley loop: one round per iteration, up to max_iterations
|
|
190
|
+
├─ iteration sequence
|
|
191
|
+
│ ├─ test branch: does the tester open this iteration? (--tester on, the run's first)
|
|
192
|
+
│ │ ├─ then sequence
|
|
193
|
+
│ │ │ ├─ write_test open the iteration, run the tester, record what it wrote
|
|
194
|
+
│ │ │ │ └─ tester model call writing a failing test under the test and fixture gate paths
|
|
195
|
+
│ │ │ └─ red branch: did the tester alone cross the cost cap?
|
|
196
|
+
│ │ │ ├─ then skip_red record the check as skipped and build nothing
|
|
197
|
+
│ │ │ └─ else red_check the check, which must fail on the tester's test
|
|
198
|
+
│ │ └─ else no_test no tester this iteration
|
|
199
|
+
│ ├─ work branch: did the test phase end the run: a green check, or the cost cap?
|
|
200
|
+
│ │ ├─ then hold build nothing, and leave the run to the guard
|
|
121
201
|
│ │ └─ else sequence
|
|
122
|
-
│ │ ├─
|
|
123
|
-
│ │ └─
|
|
124
|
-
│ │
|
|
125
|
-
│ │
|
|
126
|
-
│ │
|
|
127
|
-
│ │
|
|
128
|
-
│ │
|
|
129
|
-
│ └─
|
|
130
|
-
|
|
202
|
+
│ │ ├─ build open the iteration unless the tester did, run the builder, measure the change
|
|
203
|
+
│ │ │ └─ builder model call: one agentic session in the build root
|
|
204
|
+
│ │ └─ verify branch: did the build alone cross the cost cap?
|
|
205
|
+
│ │ ├─ then skip_verify record the check as skipped and call no critic
|
|
206
|
+
│ │ └─ else sequence
|
|
207
|
+
│ │ ├─ check the deterministic gate: checkride, a command, or none
|
|
208
|
+
│ │ └─ critique the read-only critic's verdict and feedback
|
|
209
|
+
│ │ └─ critic fallback: judge without tools if a critic on any provider but claude_cli keeps dying or runs out of steps
|
|
210
|
+
│ │ ├─ retry one more try after a stream death, on any provider but claude_cli
|
|
211
|
+
│ │ │ └─ critic_tools model call with read-only workspace tools
|
|
212
|
+
│ │ └─ pipe mark the verdict degraded
|
|
213
|
+
│ │ └─ critic_toolless model call judging from the check output and a file list
|
|
214
|
+
│ └─ record archive the iteration and rewrite the run summary
|
|
215
|
+
└─ guard gate stop on approval over a green check, the cost cap, a gate edit, or a green check before any build
|
|
131
216
|
```
|
|
132
217
|
|
|
133
218
|
`pnpm diagram` prints the same tree from a checkout. A test holds this copy and
|
|
@@ -189,8 +274,11 @@ volley matrix --builders <models> --critics <models> --config <path>
|
|
|
189
274
|
volley critique --criteria <criteria> [--baseline <rev>] [options]
|
|
190
275
|
volley acp --builder-provider <name> --builder-model <model> \
|
|
191
276
|
--critic-provider <name> --critic-model <model> [options]
|
|
277
|
+
volley run --protocol runciter.run/1 [options] < run-input.json
|
|
192
278
|
```
|
|
193
279
|
|
|
280
|
+
`volley run` is another name for `volley`, with the same options.
|
|
281
|
+
|
|
194
282
|
| Flag | Default | Description |
|
|
195
283
|
|---|---|---|
|
|
196
284
|
| `--prompt` | (required) | Task prompt. Literal string or `@path/to/file.md`. |
|
|
@@ -204,12 +292,16 @@ volley acp --builder-provider <name> --builder-model <model> \
|
|
|
204
292
|
| `--builder-max-tool-calls` | `1` | Tool calls a tool-loop builder may make per turn, not counting reads (`read_file`, `search_files`, `list_files`); past it, fascicle runs the calls before the first one over, drops the rest, and tells the model which it dropped. Raise it for a model and runtime that return parallel calls well. Or `VOLLEY_BUILDER_MAX_TOOL_CALLS`. Ignored for `claude_cli`. |
|
|
205
293
|
| `--builder-max-output-tokens` | `16384` (ollama, lmstudio), the provider's (openrouter) | Output tokens a tool-loop builder may generate per turn, reasoning included (`num_predict` on Ollama). A turn cut off at it before it calls a tool costs that step, not the session. Or `VOLLEY_BUILDER_MAX_OUTPUT_TOKENS`. Ignored for `claude_cli`. |
|
|
206
294
|
| `--builder-trim-history` | off | Experimental. Above this estimated request size, in tokens, a tool-loop builder's turns send older tool results as one-line stubs (see [Local builder](#local-builder)). Or `VOLLEY_BUILDER_TRIM_HISTORY`. Ignored for `claude_cli`. |
|
|
295
|
+
| `--effort-ladder` | off | Comma-separated builder efforts from `low`, `medium`, `high`, `xhigh`, and `max`: each build runs one rung further up than the last, and the top rung repeats (see [The effort ladder](#the-effort-ladder)). Or `VOLLEY_EFFORT_LADDER`, or `effort_ladder` in a config file. Flat on `ollama` and `lmstudio`, which ignore effort. |
|
|
207
296
|
| `--allow-unsandboxed-builder` | off | Permit a builder other than `claude_cli` to run without a sandbox (see [Local builder](#local-builder)). |
|
|
208
297
|
| `--critic-model` | `opus` | Model for the critic. |
|
|
209
298
|
| `--critic-provider` | `claude_cli` | `claude_cli`, `ollama`, `lmstudio`, or `openrouter` (see [Local critic](#local-critic) and [OpenRouter](#openrouter)). |
|
|
210
299
|
| `--critic-url` | the provider's env var | The critic's model server base URL, for `ollama`, `lmstudio`, or `openrouter` only. |
|
|
300
|
+
| `--critic-effort` | `high` with `--effort-ladder`, else none | The critic's effort on every call, from the same five levels as a rung. Or `VOLLEY_CRITIC_EFFORT`, or `critic_effort` in a config file. Unset with no ladder, the critic is sent no effort, as before. |
|
|
211
301
|
| `--builder-permission-mode` | `acceptEdits` | Or `bypassPermissions` for fully trusted workspaces. |
|
|
212
302
|
| `--critic` | `reviewer` | `reviewer`, `optimizer`, `researcher`, or a prompt file path. |
|
|
303
|
+
| `--tester` | `off` | `on` runs a tester before the first build, which writes a failing test from the criteria; needs a check (see [The tester seat](#the-tester-seat)). Or `VOLLEY_TESTER`, or `tester` in a config file. |
|
|
304
|
+
| `--tester-model` | the critic's | The tester's model, with `--tester on`; refused with it off. Or `VOLLEY_TESTER_MODEL`, or `tester_model` in a config file. |
|
|
213
305
|
| `--max-iterations` | `10` | Hard iteration cap. |
|
|
214
306
|
| `--max-cost-usd` | none | Hard USD ceiling. Set it. |
|
|
215
307
|
| `--git` | off | Commit after each phase (workspace must be a git repo). |
|
|
@@ -226,6 +318,7 @@ volley acp --builder-provider <name> --builder-model <model> \
|
|
|
226
318
|
| `--verbose` | off | Full tool inputs/outputs (truncated at 4000 chars). |
|
|
227
319
|
| `--quiet` | off | Phase transitions, cost lines, and the final summary only. |
|
|
228
320
|
| `--no-thinking` | off | Hide reasoning chunks (still recorded in the trajectory). |
|
|
321
|
+
| `--protocol` | off | `runciter.run/1`: read one `RunInput` on stdin, write one `RunOutput` on stdout, and exit by the protocol's codes (see [Run volley from runciter](#run-volley-from-runciter)). |
|
|
229
322
|
|
|
230
323
|
Human-readable progress goes to **stderr**; stdout carries machine output
|
|
231
324
|
only, so volley composes under bigger harnesses the same way it composes
|
|
@@ -236,8 +329,8 @@ checkride.
|
|
|
236
329
|
| Code | Meaning |
|
|
237
330
|
|---|---|
|
|
238
331
|
| 0 | Check passed and verdict approved. |
|
|
239
|
-
| 2 | Max iterations reached without success; for `volley critique`, the critic asked for changes or the check failed. |
|
|
240
|
-
| 3 | Builder
|
|
332
|
+
| 2 | Max iterations reached without success; for `volley critique`, the critic asked for changes or the check failed; with `--tester on`, the check passed on the tester's test before any build (`tester_passed_early`). |
|
|
333
|
+
| 3 | Builder, tester or provider error. |
|
|
241
334
|
| 4 | Check pipeline error (broken pipeline, not failing checks). |
|
|
242
335
|
| 5 | Configuration error. |
|
|
243
336
|
| 6 | Critic error (schema validation failed after repair attempts). |
|
|
@@ -245,6 +338,9 @@ checkride.
|
|
|
245
338
|
| 8 | The builder edited the gate and `--fail-on-gate-edit` was set. |
|
|
246
339
|
| 130 | Interrupted (SIGINT); resumable. |
|
|
247
340
|
|
|
341
|
+
Under `--protocol runciter.run/1` volley exits by the protocol's own table
|
|
342
|
+
instead (see [Run volley from runciter](#run-volley-from-runciter)).
|
|
343
|
+
|
|
248
344
|
### Config file
|
|
249
345
|
|
|
250
346
|
```ts
|
|
@@ -281,7 +377,10 @@ workspace. A run an older volley started keeps its state in the workspace's
|
|
|
281
377
|
inside it. A run directory with a symbolic link in it is refused, since volley
|
|
282
378
|
never writes one. The resumed run's first builder gets what the last completed
|
|
283
379
|
iteration left it, as it would have without the stop: the critic's feedback,
|
|
284
|
-
the check's output if it failed, and the step-limit note.
|
|
380
|
+
the check's output if it failed, and the step-limit note. The run's effort
|
|
381
|
+
ladder and its critic's effort are recorded in its `config.json`, and its rung
|
|
382
|
+
follows on from the builds the last iteration recorded, so build 3 still runs
|
|
383
|
+
at the third rung with no variable set.
|
|
285
384
|
|
|
286
385
|
**A resume on the host runs only a config this machine signed.** A resume runs
|
|
287
386
|
what the run's `config.json` says, its check command and its seats included.
|
|
@@ -798,8 +897,8 @@ for you ([below](#zed-setup)).
|
|
|
798
897
|
`--builder-provider`, `--builder-model`, `--critic-provider`, and
|
|
799
898
|
`--critic-model`. A missing one is refused with every missing flag named. The
|
|
800
899
|
rest are optional: `--builder-url`, `--critic-url`, `--builder-max-steps`,
|
|
801
|
-
`--builder-max-tool-calls`, `--builder-max-output-tokens`, `--allow-unsandboxed-builder`, `--critic`, `--check`, `--
|
|
802
|
-
`--max-cost-usd`, and `--state-dir`.
|
|
900
|
+
`--builder-max-tool-calls`, `--builder-max-output-tokens`, `--allow-unsandboxed-builder`, `--critic`, `--check`, `--tester`,
|
|
901
|
+
`--tester-model`, `--max-iterations`, `--max-cost-usd`, and `--state-dir`.
|
|
803
902
|
|
|
804
903
|
It refuses these before serving, each with exit 5:
|
|
805
904
|
|
|
@@ -809,9 +908,15 @@ It refuses these before serving, each with exit 5:
|
|
|
809
908
|
| `--git` | It commits to the open repo after every phase, which no permission prompt covers. |
|
|
810
909
|
| `--json` | Stdout carries the protocol alone. |
|
|
811
910
|
| a `claude_cli` builder | The Claude CLI runs its own tools under its own permission model, so its edits would never reach the editor's prompts. |
|
|
911
|
+
| a `claude_cli` tester | The tester takes the critic's provider, and a Claude CLI tester's writes would never reach the editor's prompts. `VOLLEY_TESTER=on` counts. |
|
|
812
912
|
|
|
813
913
|
A `claude_cli` critic is allowed: it only reads, and it runs on the host, where
|
|
814
|
-
its login works.
|
|
914
|
+
its login works. A tester on any other provider runs in a round of its own
|
|
915
|
+
before the build, headed **Round 1: the tester's test**, and each of its
|
|
916
|
+
writes asks with its diff, as the builder's do; a write it may not make is
|
|
917
|
+
refused before you're asked about it. Its `--check auto` resolves against the
|
|
918
|
+
folder each session opens, so a prompt in a folder with no check fails with
|
|
919
|
+
the reason rather than the startup. The whole config resolves once at startup, so a bad flag or an
|
|
815
920
|
unset `OPENROUTER_API_KEY` exits 5 there, where Zed's agent log shows it, rather
|
|
816
921
|
than failing every prompt.
|
|
817
922
|
|
|
@@ -927,6 +1032,107 @@ Two more round it out. Ask it to run `echo "key=$OPENROUTER_API_KEY"`, and the
|
|
|
927
1032
|
output should be `key=` with nothing after it. And the repository should hold no
|
|
928
1033
|
`.volley`.
|
|
929
1034
|
|
|
1035
|
+
## Run volley from runciter
|
|
1036
|
+
|
|
1037
|
+
`volley run --protocol runciter.run/1` makes volley a run child of runciter,
|
|
1038
|
+
the scheduler that runs tasks through pipelines of agents, over the protocol
|
|
1039
|
+
runciter's spec defines in its §5.4: one `RunInput` on stdin, exactly one
|
|
1040
|
+
`RunOutput` on stdout and nothing else, progress on stderr, and the exit code as
|
|
1041
|
+
the verdict. runciter's `process` kind runs it as it would any child that speaks
|
|
1042
|
+
the protocol, so runciter needs no adapter of its own for volley. Name the
|
|
1043
|
+
command in an agent file's frontmatter:
|
|
1044
|
+
|
|
1045
|
+
```markdown
|
|
1046
|
+
---
|
|
1047
|
+
label: Volley
|
|
1048
|
+
runner: process
|
|
1049
|
+
command: [volley, run, --protocol, runciter.run/1, --builder-model, sonnet, --max-iterations, "5"]
|
|
1050
|
+
---
|
|
1051
|
+
You are the build stage. Make the change the task asks for, and leave the tests green.
|
|
1052
|
+
```
|
|
1053
|
+
|
|
1054
|
+
Only the flag turns protocol mode on. runciter sets `RUNCITER_PROTOCOL` for
|
|
1055
|
+
every process under a run, a check that runs `volley` included, so volley
|
|
1056
|
+
ignores the variable.
|
|
1057
|
+
|
|
1058
|
+
### What it reads
|
|
1059
|
+
|
|
1060
|
+
The `RunInput` sets the task. Every other flag in the agent's argv, the seats,
|
|
1061
|
+
`--max-iterations`, `--critic`, `--tester` and `--config` among them, applies
|
|
1062
|
+
as it does on the command line.
|
|
1063
|
+
|
|
1064
|
+
| From the `RunInput` | Becomes |
|
|
1065
|
+
|---|---|
|
|
1066
|
+
| `agent.prompt`, then the text of `context.md` | The prompt. `context.md` carries the task and, on a retry, the feedback on the last attempt, so a retried task knows why it failed. |
|
|
1067
|
+
| `task.criteria` | The criteria. |
|
|
1068
|
+
| `workspace.path` | The workspace. |
|
|
1069
|
+
| `stage.verify` | The check, read as runciter's own verify reads it, so volley converges on the gate runciter checks afterwards: `none` and a command as they are, and `auto` as `pnpm check` when `package.json` has a `check` script, else no check. A `--check` in the argv wins. |
|
|
1070
|
+
| `budget.max_cost_usd` | The cost cap, over any `--max-cost-usd` in the argv, which stands when runciter gives no budget. |
|
|
1071
|
+
| `run_dir` and `run_id` | The run's state, at `<run_dir>/volley/<run_id>/`, outside the workspace runciter diffs and verifies. A `--state-dir` or `--run-id` in the argv wins. |
|
|
1072
|
+
| `budget.timeout_ms` | Nothing: runciter stops the run itself, with SIGTERM. |
|
|
1073
|
+
|
|
1074
|
+
### What it writes
|
|
1075
|
+
|
|
1076
|
+
On stdout, one `RunOutput`:
|
|
1077
|
+
|
|
1078
|
+
- **`status`** is `passed` when the run succeeded, else `failed`, with a
|
|
1079
|
+
`fail_reason`: `iterations_exhausted` when it ran out of iterations,
|
|
1080
|
+
`cost_cap`, `gate_edit_blocked`, `interrupted`, `builder_error` for a builder's
|
|
1081
|
+
or a tester's error, `check_error`, `critic_error`, and `not_passed` for any
|
|
1082
|
+
other outcome, `tester_passed_early` among them.
|
|
1083
|
+
- **`summary`** is a status line, then the critic's last feedback, cut to 4,000
|
|
1084
|
+
characters. **`detail`** is the error's message, when the run ended on one.
|
|
1085
|
+
- **`usage`** is the run's totals, its cost marked `estimated`, as Claude Code's
|
|
1086
|
+
cost is.
|
|
1087
|
+
- **`data`** is the run's `summary.json` whole, which runciter keeps as the
|
|
1088
|
+
run's `data.json`.
|
|
1089
|
+
- **`artifacts`** names `critic-feedback.md`, the critic's last `feedback.md`,
|
|
1090
|
+
which volley copies into `<run_dir>/artifacts/`, following no link in either
|
|
1091
|
+
file's place. **`lessons`** is empty.
|
|
1092
|
+
|
|
1093
|
+
On stderr, the run's progress without color, with runciter's typed lines among
|
|
1094
|
+
it: a `runciter.progress` line as each phase starts and ends
|
|
1095
|
+
(`iteration 2: check failing: test`, with `done` and `total` counting
|
|
1096
|
+
iterations), and a `runciter.usage` line as each iteration ends, the run's
|
|
1097
|
+
spend so far, and once more after the run, with its tokens.
|
|
1098
|
+
|
|
1099
|
+
### Exit codes in protocol mode
|
|
1100
|
+
|
|
1101
|
+
The protocol's own table, apart from the [one above](#exit-codes):
|
|
1102
|
+
|
|
1103
|
+
| Code | Meaning |
|
|
1104
|
+
|---|---|
|
|
1105
|
+
| 0 | One `RunOutput` on stdout, whatever the run's outcome, a builder, check or critic error included, so its summary and `data` reach runciter's drawer. |
|
|
1106
|
+
| 1 | volley failed and had no `RunOutput` to write. runciter records a `runner_error`. |
|
|
1107
|
+
| 2 | Bad input on stdin, a refused flag, or a config error. runciter records a `contract_violation`. |
|
|
1108
|
+
|
|
1109
|
+
On 1 and 2, stdout is empty and the last line on stderr is a `StdioFailure`,
|
|
1110
|
+
`{ "error": …, "stage"?: … }`, the shape fascicle's `run_stdio` writes, which
|
|
1111
|
+
runciter takes as the failure's detail.
|
|
1112
|
+
|
|
1113
|
+
### Refusals
|
|
1114
|
+
|
|
1115
|
+
Each exits 2 before stdin is read:
|
|
1116
|
+
|
|
1117
|
+
| Refused | Why |
|
|
1118
|
+
|---|---|
|
|
1119
|
+
| `--worktree` | runciter owns the workspace. |
|
|
1120
|
+
| `--git` | runciter owns the workspace and what's committed to it. |
|
|
1121
|
+
| `--json` | Stdout carries the `RunOutput`. |
|
|
1122
|
+
| a positional prompt | The `RunInput` gives the prompt. |
|
|
1123
|
+
|
|
1124
|
+
### Conformance
|
|
1125
|
+
|
|
1126
|
+
[`test/fixtures/runciter_run_input.json`](test/fixtures/runciter_run_input.json)
|
|
1127
|
+
is a complete `RunInput`, every field §5.4 names, and the conformance input
|
|
1128
|
+
runciter's planned `runciter-run conform <command>` will drive volley with. Its
|
|
1129
|
+
paths are placeholders: a driver points `workspace.path`, `run_dir` and
|
|
1130
|
+
`context` at real ones, as volley's own tests do, which parse it
|
|
1131
|
+
(`test/unit/protocol.test.ts`) and run it through the CLI
|
|
1132
|
+
(`test/integration/protocol_cli.test.ts`). volley's schemas in `src/protocol/`
|
|
1133
|
+
are its own copy of runciter's, with no dependency on runciter, so that test
|
|
1134
|
+
is what checks the two agree.
|
|
1135
|
+
|
|
930
1136
|
## What the builder changed
|
|
931
1137
|
|
|
932
1138
|
A check that exits 0 proves the check passed. It does not prove the builder did
|
|
@@ -1000,6 +1206,27 @@ the workspace: `<root>/<run id>/`, where the root is `--state-dir` (or
|
|
|
1000
1206
|
- **Pruning.** Nothing prunes old runs. A root volley creates is yours alone
|
|
1001
1207
|
(0700), since trajectories hold prompts and tool output. A reused `--run-id`
|
|
1002
1208
|
rotates the earlier run aside to `<run id>.bak.<timestamp>/`.
|
|
1209
|
+
- **The effort ladder.** Each `iterations/NNN/summary.json` records `builds`
|
|
1210
|
+
(the builder sessions so far), `rung` (the effort that build ran at, or null
|
|
1211
|
+
with no ladder), `climbed` (`check_failed`, `critic_rejected`, or null when
|
|
1212
|
+
it didn't climb), and `effort_ignored` (its provider dropped the effort), and
|
|
1213
|
+
a seat's record says `effort_ignored` too when its effort was dropped. The
|
|
1214
|
+
run's `summary.json` carries `comparison.rungs`, each iteration's rung in
|
|
1215
|
+
order, and `comparison.ladder_flat`, true when every build ran at one effort,
|
|
1216
|
+
an ignored effort counting as the provider's own
|
|
1217
|
+
([The effort ladder](#the-effort-ladder)).
|
|
1218
|
+
- **The tester.** With `--tester on`, iteration 1's `summary.json` records
|
|
1219
|
+
`tester` (its session, as `builder` and `critic` are), `tester_check` (the
|
|
1220
|
+
check run right after it, which must fail), and `tester_changes` (what it
|
|
1221
|
+
wrote, measured as `changes` is). Every iteration carries `tester_hashes`,
|
|
1222
|
+
each file the tester wrote by its content hash after it ran: a build that
|
|
1223
|
+
leaves one at that hash isn't counted as a gate edit there, and one that
|
|
1224
|
+
changes it is. The critic reads the tester's test too, and its record
|
|
1225
|
+
carries `test_findings` when it found the test wanting. A run that passes
|
|
1226
|
+
its check before any build has a null `builder` and `check`, ends
|
|
1227
|
+
`tester_passed_early`, and says so in `comparison.tester_passed_early`. The
|
|
1228
|
+
run's `summary.json` and its `comparison` carry `tester_cost_usd` beside the
|
|
1229
|
+
other two seats' costs.
|
|
1003
1230
|
- **Older runs.** An older volley kept the same files in the workspace's
|
|
1004
1231
|
`.volley/`. `volley resume` still finds such a run there, and resumes it in
|
|
1005
1232
|
place once you pass `--trust-config`, since an older volley signed nothing
|