@robmclarty/volley 0.8.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +44 -0
- package/README.md +194 -45
- package/dist/cli.js +1284 -457
- package/dist/cli.js.map +1 -1
- package/dist/index.d.ts +136 -12
- package/dist/index.js +1162 -543
- package/dist/index.js.map +1 -1
- package/package.json +2 -2
- package/src/builder/presets/harness_append_acp.md +4 -3
- package/src/builder/presets/harness_append_local.md +5 -3
- package/src/critic/presets/harness_append_local.md +2 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,49 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v0.10.0 — 2026-10-27
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
- **A builder sees what the last check said (#6).** When the previous iteration's check failed, the builder's prompt carries a `PREVIOUS CHECK RESULT` section: the exit code, the failing slots, and an excerpt capped at 6,000 characters, taken from checkride's digest (volley now runs `pnpm exec checkride --json --digest` and archives `check/digest.md`), else the start of each failing slot's raw output, else the end of a command's output. A resume rebuilds it from the archive, and a check the user rejected under `volley acp` reads as never having run, not as failed.
|
|
7
|
+
- **A tool-loop builder's last steps count down (#6).** The last five steps of a session, or a tenth of a longer budget, end with a note of how many are left and the command that runs the check, and the last says that only `finish` runs on it. When a session hits its step limit, the next prompt says so and asks for the check first.
|
|
8
|
+
- **`--builder-max-output-tokens <n>` caps a tool-loop builder's output per turn (#6).** Or `VOLLEY_BUILDER_MAX_OUTPUT_TOKENS`. Reasoning counts, and Ollama gets it as `num_predict`. An ollama or lmstudio builder is capped at 16,384 by default, an openrouter one only when you set a cap. A turn cut off before it calls a tool costs that step instead of ending the session, and each iteration's summary counts those turns as `builder.truncated_turns`.
|
|
9
|
+
- **`builder_max_steps_hits` counts the builder sessions that ran out of steps (#6).** It's on `RunResult`, so `--json` and `summary.json` carry it, since the warning each one prints goes to stderr, which `--json` silences.
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
- **Reads don't count against `--builder-max-tool-calls` (#6).** `read_file`, `search_files`, and `list_files` are read-only, so a builder that batches them gets them all, and past the cap the next request names the calls that were dropped. Above a cap of one, step 2 of the builder's prompt no longer names the cap.
|
|
13
|
+
- **The builder is asked to check as it goes (#6).** In a checkride workspace it's told to run `pnpm exec checkride --digest` after each batch of edits and to read `.check/digest.md` before any slot's raw output, and both harness prompts say to verify after each batch of edits, not only before finishing.
|
|
14
|
+
- **`RunResult` has `builder_max_steps_hits` and `ResolvedConfig` has `builder_max_output_tokens`.** Both are required, so SDK code that builds either by hand needs them.
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
- **A contained `--worktree` run keeps its work.** The sandbox image has no git identity, so volley's own commits failed inside it: the salvage commit that keeps a converged run's work on `volley/<run id>`, the `--git` checkpoints, and the squash merge. They now fall back to `volley <volley@volley.invalid>` for whatever git lacks, and an identity you configured still wins.
|
|
18
|
+
- **A failing slot's raw output is read only from inside the check directory.** Slot names and output files come from a summary the builder could have written, and a `../` in one could put a file from outside it in front of the critic or the next builder.
|
|
19
|
+
|
|
20
|
+
### Internal
|
|
21
|
+
- **Built on fascicle 0.17.1**, which names a turn's dropped tool calls to the model and can go on past a turn cut off at its output cap.
|
|
22
|
+
- **The publish workflow keeps its OIDC identity out of the job that runs dependencies.** A `build` job verifies the tag, installs, runs the gate, builds, and packs with read-only permissions, and a separate `publish` job holds `id-token: write` behind the `npm-publish` approval and publishes that tarball with `--ignore-scripts`.
|
|
23
|
+
- **The tests drive the step countdown, the per-turn cap's read-only reads, and the output cap through fascicle's real tool loop**, against a stand-in OpenAI server that now takes several calls a turn and can end one on `length`.
|
|
24
|
+
|
|
25
|
+
## v0.9.0 — 2026-10-20
|
|
26
|
+
|
|
27
|
+
### Added
|
|
28
|
+
- **`volley critique` checks and critiques a change already made, once, with no builder (#3).** The change is everything that differs from `--baseline` (default HEAD), committed or not, untracked files included, and the critic sees what a run's critic sees, so critics can be compared on fixed, known-good and known-bad changes without paying for builder turns. It exits 0 on approval over a green check and 2 when the critic asks for changes or the check is red (status `rejected`). Every builder or loop flag, and `--config`, is refused by name (exit 5), and a `--baseline` that isn't a commit fails before any state is written. `run_critique` is exported for harnesses.
|
|
29
|
+
- **`--builder-max-tool-calls <n>` lets a tool-loop builder make several tool calls a turn (#4).** Or `VOLLEY_BUILDER_MAX_TOOL_CALLS`; the default stays 1, since some model and runtime pairs return parallel calls badly. Above 1, the builder's prompt asks it to batch independent reads but make one change or command per turn. The cap is recorded with the run and kept on resume.
|
|
30
|
+
- **Each run names its state directory.** `RunResult`, and so `--json` and `summary.json`, carry `run_state_dir`, and a run prints `state: <dir>` when it starts and in its final summary. `--dry-run` names the state root.
|
|
31
|
+
- **`sandbox_run_args` mounts a contained run's own state directory.** Its `run_state` option binds the host's `<root>/<run id>/` at that run's directory under `SANDBOX_STATE_ROOT`, never the whole root, so the builder inside can't read or rewrite another run's state. `default_state_root` and `SANDBOX_STATE_ROOT` are exported.
|
|
32
|
+
|
|
33
|
+
### Changed
|
|
34
|
+
- **Run state leaves the workspace by default (#5).** Without `--state-dir`, a run's config, summary, trajectory, feedback, verdict, and iterations go to `<root>/<run id>/`, where the root is `$XDG_STATE_HOME/volley`, else `~/.local/state/volley`, as under `volley acp`. A check that reads every file, dot-directories included, no longer fails from the second iteration on over volley's own feedback. A harness that read `<workspace>/.volley/` reads `run_state_dir` instead. A run directory volley can't write is a configuration error (exit 5), and inside the sandbox the message names the mount. The critic's "prior iteration state" line now drops by default, since the iterations sit outside the tree it reads.
|
|
35
|
+
- **`volley resume <id>` finds a run from any directory**, under `--state-dir` or the default root, and falls back to the workspace's `.volley/` for a run an older volley started. It refuses a run directory with a symbolic link in it, an id that isn't a run id, and a critique, which has nothing to resume.
|
|
36
|
+
- **A resume on the host runs only a config this machine signed.** volley signs each run's `config.json` (as `config.sig`) under a key kept at `$XDG_CONFIG_HOME/volley/run-key`, else `~/.config/volley/run-key`, and signs nothing inside the sandbox, so a contained run's builder can't plant a check or seat for a host-side resume to run. A config without a valid signature (a contained run's, another machine's, an older volley's, or one edited by hand) resumes only with `--trust-config`, which first shows the check and seats it's about to run, escaped so a planted control sequence can't disguise them. Resume a contained run inside its container.
|
|
37
|
+
- **`ResolvedConfig` has `mode` and `builder_max_tool_calls`, and `RunResult` has `mode` and `run_state_dir`.** All are required, so SDK code that builds either by hand needs them. `resolve_config` always sets `state_dir`, where null still means the workspace's `.volley/`. `RunStatus` gains `rejected`, and a critique's `comparison.builder_transport` is null.
|
|
38
|
+
|
|
39
|
+
### Fixed
|
|
40
|
+
- **`search_files` and `list_files` on a file path search or list that file.** They walked it as a directory and answered "no matches", which led a local critic to reject a sound change. A path that doesn't exist now says so.
|
|
41
|
+
- **A critic verdict with null `feedback` or `unmet_criteria` reads as empty.** A critic that holds tools isn't constrain-decoded, and such an answer failed schema validation and exited 6.
|
|
42
|
+
- **A `--state-dir` inside the workspace stays out of the builder's work.** Its files showed up in the change list the critic reads and were committed by `--git` checkpoints. A `--state-dir` that is the workspace itself is refused (exit 5).
|
|
43
|
+
|
|
44
|
+
### Internal
|
|
45
|
+
- **The tests keep run state and the signing key out of your home directory.** A setup file gives each test file its own `XDG_STATE_HOME` and `XDG_CONFIG_HOME`, and fails a file that leaves a test run in the real state root or makes the real key.
|
|
46
|
+
|
|
3
47
|
## v0.8.0 — 2026-10-19
|
|
4
48
|
|
|
5
49
|
### Added
|
package/README.md
CHANGED
|
@@ -54,8 +54,8 @@ volley \
|
|
|
54
54
|
The cap is enforced in the loop guard after each phase. Sessions are never
|
|
55
55
|
killed mid-flight for cost (a builder killed mid-edit leaves the workspace
|
|
56
56
|
undefined), so set the cap below your true ceiling by roughly one iteration's
|
|
57
|
-
expected cost. Per-run totals land in
|
|
58
|
-
terminal summary. An [OpenRouter](#openrouter) seat's cost is the charge
|
|
57
|
+
expected cost. Per-run totals land in the run's `summary.json` (see
|
|
58
|
+
[Run artifacts](#run-artifacts)) and the final terminal summary. An [OpenRouter](#openrouter) seat's cost is the charge
|
|
59
59
|
OpenRouter reports for each call, so the cap trips on one as on any other.
|
|
60
60
|
|
|
61
61
|
## How it works
|
|
@@ -73,19 +73,28 @@ prompt ──> builder (full agentic session in your workspace)
|
|
|
73
73
|
```
|
|
74
74
|
|
|
75
75
|
- **Builder** gets the task, the acceptance criteria, and the previous
|
|
76
|
-
iteration's critic feedback.
|
|
77
|
-
|
|
76
|
+
iteration's critic feedback. When the previous iteration's check failed, it
|
|
77
|
+
also gets what the check said: checkride's digest of the failing slots, else
|
|
78
|
+
the start of each failing slot's raw output, else the end of a command's
|
|
79
|
+
output, capped at 6,000 characters. If the previous session hit its step
|
|
80
|
+
limit, the prompt says so and asks it to run the check first. In a
|
|
81
|
+
checkride workspace it's asked to run `pnpm exec checkride --digest` after
|
|
82
|
+
each batch of edits and to read `.check/digest.md` when that fails, one
|
|
83
|
+
capped file for every failing slot, before any slot's raw output. It has full tools
|
|
84
|
+
(Read/Write/Edit/Bash) and runs with `--permission-mode acceptEdits` by
|
|
85
|
+
default.
|
|
78
86
|
- **Check** is auto-detected: if the workspace uses checkride, volley runs
|
|
79
|
-
`pnpm exec checkride --json` and gates on the summary. Any shell
|
|
80
|
-
works too (`--check "npm test"`), or `--check none` for critic-only
|
|
87
|
+
`pnpm exec checkride --json --digest` and gates on the summary. Any shell
|
|
88
|
+
command works too (`--check "npm test"`), or `--check none` for critic-only
|
|
89
|
+
runs.
|
|
81
90
|
- **Critic** is read-only (`Read`, `Grep`, `Glob` — no write path at all) and
|
|
82
91
|
returns a schema-validated `{ verdict, feedback, unmet_criteria }`. It is told
|
|
83
92
|
which files the builder changed, so it reviews a change rather than a tree.
|
|
84
|
-
The harness writes
|
|
93
|
+
The harness writes the run's `feedback.md` and `verdict`; the critic
|
|
85
94
|
cannot get them wrong.
|
|
86
95
|
|
|
87
96
|
Each iteration is a fresh session — no context carries over except through the
|
|
88
|
-
workspace filesystem
|
|
97
|
+
workspace filesystem, the critic's feedback, and a failed check's output. Swap the critic to change what
|
|
89
98
|
the loop optimizes for: `--critic reviewer` (default), `optimizer`,
|
|
90
99
|
`researcher`, or a path to your own system prompt.
|
|
91
100
|
|
|
@@ -150,6 +159,7 @@ volley [options]
|
|
|
150
159
|
volley --config <path>
|
|
151
160
|
volley resume <run-id>
|
|
152
161
|
volley matrix --builders <models> --critics <models> --config <path>
|
|
162
|
+
volley critique --criteria <criteria> [--baseline <rev>] [options]
|
|
153
163
|
volley acp --builder-provider <name> --builder-model <model> \
|
|
154
164
|
--critic-provider <name> --critic-model <model> [options]
|
|
155
165
|
```
|
|
@@ -163,7 +173,9 @@ volley acp --builder-provider <name> --builder-model <model> \
|
|
|
163
173
|
| `--builder-model` | `opus` | `opus`/`sonnet`/`haiku` or a full model id. |
|
|
164
174
|
| `--builder-provider` | `claude_cli` | `claude_cli`, `ollama`, `lmstudio`, or `openrouter` (see [Local builder](#local-builder) and [OpenRouter](#openrouter)). |
|
|
165
175
|
| `--builder-url` | the provider's env var | The builder's model server base URL, for `ollama`, `lmstudio`, or `openrouter` only. Lets the builder and critic run on different machines with the same provider (see [Seats on different machines](#seats-on-different-machines)). |
|
|
166
|
-
| `--builder-max-steps` | `50` | Tool-loop builder step cap per iteration. Ignored for `claude_cli`. |
|
|
176
|
+
| `--builder-max-steps` | `50` | Tool-loop builder step cap per iteration. Or `VOLLEY_BUILDER_MAX_STEPS`. Ignored for `claude_cli`. `--json` and `summary.json` count the sessions that hit it as `builder_max_steps_hits`. |
|
|
177
|
+
| `--builder-max-tool-calls` | `1` | Tool calls a tool-loop builder may make per turn, not counting reads (`read_file`, `search_files`, `list_files`); past it, fascicle runs the calls before the first one over, drops the rest, and tells the model which it dropped. Raise it for a model and runtime that return parallel calls well. Or `VOLLEY_BUILDER_MAX_TOOL_CALLS`. Ignored for `claude_cli`. |
|
|
178
|
+
| `--builder-max-output-tokens` | `16384` (ollama, lmstudio), the provider's (openrouter) | Output tokens a tool-loop builder may generate per turn, reasoning included (`num_predict` on Ollama). A turn cut off at it before it calls a tool costs that step, not the session. Or `VOLLEY_BUILDER_MAX_OUTPUT_TOKENS`. Ignored for `claude_cli`. |
|
|
167
179
|
| `--allow-unsandboxed-builder` | off | Permit a builder other than `claude_cli` to run without a sandbox (see [Local builder](#local-builder)). |
|
|
168
180
|
| `--critic-model` | `opus` | Model for the critic. |
|
|
169
181
|
| `--critic-provider` | `claude_cli` | `claude_cli`, `ollama`, `lmstudio`, or `openrouter` (see [Local critic](#local-critic) and [OpenRouter](#openrouter)). |
|
|
@@ -178,7 +190,7 @@ volley acp --builder-provider <name> --builder-model <model> \
|
|
|
178
190
|
| `--gate-paths` | (built-in list) | Comma-separated globs naming the gate — tests, fixtures, check config. Replaces the defaults; see [What the builder changed](#what-the-builder-changed). |
|
|
179
191
|
| `--fail-on-gate-edit` | off | Halt the run (exit 8) if the builder edits a gate path, instead of reporting it. |
|
|
180
192
|
| `--sandbox-image` | `volley-sandbox:latest` | Container image for the local-builder sandbox. Or `VOLLEY_SANDBOX_IMAGE`. |
|
|
181
|
-
| `--state-dir` |
|
|
193
|
+
| `--state-dir` | `$XDG_STATE_HOME/volley`, else `~/.local/state/volley` | The root each run keeps its state under, at `<path>/<run id>/`, never in the workspace (see [Run artifacts](#run-artifacts)). Or `state_dir` in a config file. |
|
|
182
194
|
| `--run-id` | a fresh UUID | The run's id, naming its run state and its `volley/<id>` branch, so a harness that owns the id finds it in both. Letters, digits, `-`, and `_`, starting with a letter or digit. |
|
|
183
195
|
| `--dry-run` | off | Validate config, run `checkride doctor`, probe each tool-loop seat's model endpoint (local or OpenRouter), run the critic-seat canary (for a local critic), then exit. |
|
|
184
196
|
| `--config` | — | TypeScript config file (`VolleyConfig` default export). CLI flags override. |
|
|
@@ -196,7 +208,7 @@ checkride.
|
|
|
196
208
|
| Code | Meaning |
|
|
197
209
|
|---|---|
|
|
198
210
|
| 0 | Check passed and verdict approved. |
|
|
199
|
-
| 2 | Max iterations reached without success. |
|
|
211
|
+
| 2 | Max iterations reached without success; for `volley critique`, the critic asked for changes or the check failed. |
|
|
200
212
|
| 3 | Builder/provider error. |
|
|
201
213
|
| 4 | Check pipeline error (broken pipeline, not failing checks). |
|
|
202
214
|
| 5 | Configuration error. |
|
|
@@ -229,17 +241,37 @@ export default config;
|
|
|
229
241
|
### Resume
|
|
230
242
|
|
|
231
243
|
```sh
|
|
232
|
-
volley resume <run-id>
|
|
233
|
-
volley resume <run-id> --workspace ./my-project
|
|
244
|
+
volley resume <run-id> # from any directory
|
|
234
245
|
volley resume <run-id> --state-dir ~/volley-runs # a run started with --state-dir
|
|
246
|
+
volley resume <run-id> --workspace ./my-project # a run an older volley kept in .volley/
|
|
235
247
|
```
|
|
236
248
|
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
workspace.
|
|
249
|
+
`resume` finds the run at `<root>/<run-id>/` under `--state-dir`, else the
|
|
250
|
+
default root, from any directory, since the run's recorded config names its
|
|
251
|
+
workspace. A run an older volley started keeps its state in the workspace's
|
|
252
|
+
`.volley/`, and resumes there: point `--workspace` at it, or run `resume` from
|
|
253
|
+
inside it. A run directory with a symbolic link in it is refused, since volley
|
|
254
|
+
never writes one. The resumed run's first builder gets what the last completed
|
|
255
|
+
iteration left it, as it would have without the stop: the critic's feedback,
|
|
256
|
+
the check's output if it failed, and the step-limit note.
|
|
257
|
+
|
|
258
|
+
**A resume on the host runs only a config this machine signed.** A resume runs
|
|
259
|
+
what the run's `config.json` says, its check command and its seats included.
|
|
260
|
+
So volley on the host signs each config it writes, as `config.sig`, under a
|
|
261
|
+
key that stays on the machine (`$XDG_CONFIG_HOME/volley/run-key`, else
|
|
262
|
+
`~/.config/volley/run-key`), and a resume on the host refuses a config whose
|
|
263
|
+
signature doesn't hold (exit 5). That covers:
|
|
264
|
+
|
|
265
|
+
- **A contained run's config.** Nothing is signed inside the sandbox, and the
|
|
266
|
+
builder there can rewrite its own run's config, so resume it inside its
|
|
267
|
+
container, with the same mount. Inside the container, nothing is checked.
|
|
268
|
+
- **Another machine's config, an older volley's, or one you edited.** Read it,
|
|
269
|
+
then pass `--trust-config`, and the resume says which check and seats it's
|
|
270
|
+
about to run before it runs them.
|
|
240
271
|
|
|
241
272
|
Continues from the last completed iteration with the saved feedback and cost
|
|
242
|
-
totals. An iteration interrupted mid-phase is discarded and re-run.
|
|
273
|
+
totals. An iteration interrupted mid-phase is discarded and re-run. A
|
|
274
|
+
[critique](#critique-a-fixed-change) has one round and nothing to resume.
|
|
243
275
|
|
|
244
276
|
### Matrix sweep
|
|
245
277
|
|
|
@@ -272,9 +304,10 @@ local providers share one GPU, so parallel combos would thrash the model loader.
|
|
|
272
304
|
Each run forces `--worktree --discard-worktree` for a clean per-run reset — a
|
|
273
305
|
sweep wants verdicts, so each seat's effects are isolated and then thrown away,
|
|
274
306
|
leaving neither a branch nor a commit behind — so the **workspace must be a git
|
|
275
|
-
repository**.
|
|
276
|
-
|
|
277
|
-
`.volley-matrix/
|
|
307
|
+
repository**. Each attempt keeps its run state under the state root like any
|
|
308
|
+
run, and its summary, which names that state as `run_state_dir`, is copied to
|
|
309
|
+
`.volley-matrix/<builder>__<critic>/run-NN/summary.json`. The whole sweep goes
|
|
310
|
+
to `.volley-matrix/matrix.json`, and one aggregate table goes to stderr:
|
|
278
311
|
|
|
279
312
|
```
|
|
280
313
|
builder critic pass iters wall cost salvage flags why
|
|
@@ -309,6 +342,48 @@ A run that does not converge is a *result*, shown in the table; the sweep exits
|
|
|
309
342
|
0 as long as every attempt produced a summary, and nonzero only when one
|
|
310
343
|
produced none at all (`status: broke`).
|
|
311
344
|
|
|
345
|
+
### Critique a fixed change
|
|
346
|
+
|
|
347
|
+
`volley critique` runs the check and the critic once over a change already
|
|
348
|
+
made, with no builder. It's how you choose a critic: replay the same
|
|
349
|
+
known-good and known-bad changes through the critic seat, across models and
|
|
350
|
+
prompts, and count how many of the bad ones each critic rejects. A matrix sweep
|
|
351
|
+
can't tell a critic that reads from one that approves everything, since both
|
|
352
|
+
pass whenever the builder happens to be right.
|
|
353
|
+
|
|
354
|
+
```sh
|
|
355
|
+
volley critique --workspace ./repo --baseline <rev> \
|
|
356
|
+
--criteria "@criteria.md" \
|
|
357
|
+
--critic-provider ollama --critic-model qwen3:8b --critic reviewer
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
- **The change** is everything that differs from `--baseline`, committed or
|
|
361
|
+
not, untracked files included: the set a run reports for its builder. Without
|
|
362
|
+
`--baseline` it's HEAD, so the uncommitted and untracked edits are the
|
|
363
|
+
change. A revision that isn't a commit is a configuration error (exit 5).
|
|
364
|
+
- **The critic** sees what it sees in a run: the changed paths and gate edits,
|
|
365
|
+
the check's section, and the same preset or prompt file. `--prompt` is
|
|
366
|
+
optional and only recorded, since a critic judges against `--criteria`.
|
|
367
|
+
- **The state** is a one-round run's, under the state root
|
|
368
|
+
([Run artifacts](#run-artifacts)). The summary records `mode: "critique"`
|
|
369
|
+
and no builder: `builder` is null in the iteration's archive, and so is the
|
|
370
|
+
comparison block's `builder_transport`. The archived `critic.tool_calls` says
|
|
371
|
+
how much the critic read before it judged.
|
|
372
|
+
- **Exit codes**: 0 when the critic approves over a green check, 2 when it asks
|
|
373
|
+
for changes or the check is red (`status: "rejected"`), and the usual codes
|
|
374
|
+
for provider, check-pipeline, configuration, and critic errors.
|
|
375
|
+
- **Flags**: the critic's seat flags, `--check`, `--critic`, `--gate-paths`,
|
|
376
|
+
`--state-dir`, `--run-id`, `--dry-run` (with the critic-seat canary),
|
|
377
|
+
`--json`, and the output flags work as they do for a run. Every flag only a
|
|
378
|
+
builder or its loop uses, from `--builder-model` and `--max-iterations` to
|
|
379
|
+
`--worktree`, `--git`, and `--fail-on-gate-edit`, is refused by name (exit
|
|
380
|
+
5), and so is `--config`, since a run's config file sets a builder.
|
|
381
|
+
|
|
382
|
+
With no builder, a local critic needs no sandbox opt-in. The check still runs
|
|
383
|
+
whatever the repository's check runs, though, so review an untrusted change
|
|
384
|
+
with `volley critique` inside the container, its run directory mounted as a
|
|
385
|
+
run's would be ([Local builder](#local-builder)).
|
|
386
|
+
|
|
312
387
|
## Local critic
|
|
313
388
|
|
|
314
389
|
The critic is read-only and schema-constrained, which makes it a good fit for
|
|
@@ -338,7 +413,7 @@ Providers and their setup:
|
|
|
338
413
|
Install the peer for your provider in the project running volley (e.g.
|
|
339
414
|
`pnpm add ai-sdk-ollama@^4.3.0`); fascicle loads it lazily only when the local
|
|
340
415
|
critic actually runs. The version matters. The major must satisfy fascicle's
|
|
341
|
-
declared peer range (`^4` for fascicle 0.
|
|
416
|
+
declared peer range (`^4` for fascicle 0.17.1, which tracks the AI SDK v7
|
|
342
417
|
line), and volley declares 4.3.0 as its floor (an optional peer, so your
|
|
343
418
|
package manager flags an older copy at install time). Before 4.3.0,
|
|
344
419
|
ai-sdk-ollama streams a thinking model's reasoning only when the request sets
|
|
@@ -389,7 +464,40 @@ surface itself — `read_file`, `search_files`, `list_files`, `write_file`,
|
|
|
389
464
|
`edit_file`, `bash`, `fetch`, and an explicit `finish` — and runs one bounded
|
|
390
465
|
tool loop per iteration (capped by `--builder-max-steps`, default 50; hitting
|
|
391
466
|
the cap is treated as partial work, not an error). The produced workspace goes
|
|
392
|
-
to check + critic exactly like a `claude_cli` build.
|
|
467
|
+
to check + critic exactly like a `claude_cli` build. Each hit prints a warning,
|
|
468
|
+
and `--json`, which silences warnings, and `summary.json` count them as
|
|
469
|
+
`builder_max_steps_hits`.
|
|
470
|
+
|
|
471
|
+
So that a model that explores first still verifies, the last few steps of each
|
|
472
|
+
session count down: five, or a tenth of a longer budget. Each of those turns
|
|
473
|
+
ends with a note of how many steps are left and the command that runs the
|
|
474
|
+
check, and the last says that only `finish` runs on it, as fascicle refuses
|
|
475
|
+
any other call there. The note is added to what that one turn sends, never to
|
|
476
|
+
the transcript, so it never piles up and a local runtime's cached prefix holds.
|
|
477
|
+
|
|
478
|
+
A local builder makes one tool call per turn by default, since some model and
|
|
479
|
+
runtime pairs return parallel calls badly. Reads don't count: `read_file`,
|
|
480
|
+
`search_files`, and `list_files` are read-only, so a model that batches them,
|
|
481
|
+
as qwen3.8 on Ollama's MLX runner does, gets them all. A turn's calls run in
|
|
482
|
+
order, so a read can't race an edit. Past the cap, fascicle drops the
|
|
483
|
+
remaining calls and the next request names them, so the model can make them
|
|
484
|
+
again. For a pair that handles parallel calls, `--builder-max-tool-calls <n>`
|
|
485
|
+
lets each turn make up to `n` changes or commands, and the builder's prompt
|
|
486
|
+
then asks it to batch independent reads but make one change or command per
|
|
487
|
+
turn.
|
|
488
|
+
|
|
489
|
+
A thinking model can spend one turn reasoning for a long time: on Ollama's MLX
|
|
490
|
+
runner, with nothing capping a turn, one qwen3.8 turn ran to 30,847 tokens and
|
|
491
|
+
19 minutes before a single `read_file`, since the runner lets an unset
|
|
492
|
+
`num_predict` run to the model's trained context less the prompt. So a local
|
|
493
|
+
builder's turns are capped at 16,384 output tokens, reasoning included, and
|
|
494
|
+
`--builder-max-output-tokens <n>` sets another cap. A turn cut off there
|
|
495
|
+
before it makes a tool call costs that one step: fascicle tells the model it
|
|
496
|
+
hit the limit and the session goes on. Each iteration's `summary.json` counts
|
|
497
|
+
those turns as `builder.truncated_turns`. An OpenRouter builder is capped only
|
|
498
|
+
when you set one, since a hosted model has its own output limit and can refuse
|
|
499
|
+
a request for more; uncapped, a turn cut off at the provider's limit still ends
|
|
500
|
+
the session.
|
|
393
501
|
|
|
394
502
|
```sh
|
|
395
503
|
volley \
|
|
@@ -417,6 +525,26 @@ mangled crossing the boundary — so a *contained* Claude role must drive by API
|
|
|
417
525
|
key (`VOLLEY_AUTH_MODE=api_key` with `ANTHROPIC_API_KEY`); the all-Claude
|
|
418
526
|
subscription path stays on the host, Docker-free.
|
|
419
527
|
|
|
528
|
+
**A contained run's state needs a mount of its own.** The container's root
|
|
529
|
+
filesystem is read-only, so volley inside has nowhere to keep the run's state
|
|
530
|
+
until you mount the run's own host directory where it looks:
|
|
531
|
+
|
|
532
|
+
1. Create `<root>/<run id>/` on the host first, empty. On Linux, Docker creates
|
|
533
|
+
a missing one owned by root, and the container's user can't write it.
|
|
534
|
+
2. Mount it at `/home/node/.local/state/volley/<run id>`. That's
|
|
535
|
+
`SANDBOX_STATE_ROOT`, and `run_state` on `sandbox_run_args` renders the
|
|
536
|
+
flag.
|
|
537
|
+
3. Pass volley the same `--run-id`.
|
|
538
|
+
|
|
539
|
+
Without the mount, the run stops before any spend (exit 5) and says what to
|
|
540
|
+
mount. Mount that one directory, never the whole root: the builder runs
|
|
541
|
+
arbitrary commands, and with the root mounted it could read every other run's
|
|
542
|
+
trajectory or plant a check in another run's config. Its own run's config is
|
|
543
|
+
still the builder's to rewrite, so a resume on the host refuses it, since
|
|
544
|
+
volley signs only the configs it writes outside the sandbox
|
|
545
|
+
([Resume](#resume)). Resume a contained run inside the container, with the same
|
|
546
|
+
mount.
|
|
547
|
+
|
|
420
548
|
To run a local builder uncontained on the host anyway, opt out with
|
|
421
549
|
`--allow-unsandboxed-builder` (or `VOLLEY_ALLOW_UNSANDBOXED_BUILDER=1`), which
|
|
422
550
|
prints a one-time warning — only point it at a workspace you are willing to let a
|
|
@@ -435,6 +563,10 @@ surprise at teardown:
|
|
|
435
563
|
| `--worktree` | Committed onto `volley/<run id>` and left there. Recover it with `git switch volley/<run id>` or `git cherry-pick`; the branch is named in the final summary and in `--json` as `salvaged_branch`. |
|
|
436
564
|
| `--worktree --discard-worktree` | Thrown away with the branch — verdicts only, nothing kept. |
|
|
437
565
|
|
|
566
|
+
volley's own commits (`--git` checkpoints, the squash merge, and the salvage
|
|
567
|
+
commit) use your git identity. Where git has none, as in the sandbox image,
|
|
568
|
+
they fall back to `volley <volley@volley.invalid>` rather than fail.
|
|
569
|
+
|
|
438
570
|
Any run that does *not* converge (cost cap, budget exhausted, interrupt, error)
|
|
439
571
|
is discarded wholesale in all three modes: an abandoned phase leaves nothing
|
|
440
572
|
behind.
|
|
@@ -606,7 +738,7 @@ for you ([below](#zed-setup)).
|
|
|
606
738
|
`--builder-provider`, `--builder-model`, `--critic-provider`, and
|
|
607
739
|
`--critic-model`. A missing one is refused with every missing flag named. The
|
|
608
740
|
rest are optional: `--builder-url`, `--critic-url`, `--builder-max-steps`,
|
|
609
|
-
`--allow-unsandboxed-builder`, `--critic`, `--check`, `--max-iterations`,
|
|
741
|
+
`--builder-max-tool-calls`, `--builder-max-output-tokens`, `--allow-unsandboxed-builder`, `--critic`, `--check`, `--max-iterations`,
|
|
610
742
|
`--max-cost-usd`, and `--state-dir`.
|
|
611
743
|
|
|
612
744
|
It refuses these before serving, each with exit 5:
|
|
@@ -665,12 +797,13 @@ another way.
|
|
|
665
797
|
|
|
666
798
|
### Where its state goes
|
|
667
799
|
|
|
668
|
-
Nothing of volley's own lands in the repo. Each prompt's run state goes
|
|
800
|
+
Nothing of volley's own lands in the repo. Each prompt's run state goes under
|
|
801
|
+
the same state root as any run's ([Run artifacts](#run-artifacts)):
|
|
669
802
|
`<root>/<run id>/`, where the root is `--state-dir`, else
|
|
670
|
-
`$XDG_STATE_HOME/volley`, else `~/.local/state/volley`.
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
803
|
+
`$XDG_STATE_HOME/volley`, else `~/.local/state/volley`. An unwritable root
|
|
804
|
+
stops `volley acp` at startup, so it shows once in the agent log. The whole
|
|
805
|
+
root is safe to delete whenever no prompt is running. A run's
|
|
806
|
+
`trajectory.jsonl` there replays in `fascicle-viewer` like a CLI run's.
|
|
674
807
|
|
|
675
808
|
### Zed setup
|
|
676
809
|
|
|
@@ -748,7 +881,7 @@ iteration and reports what moved:
|
|
|
748
881
|
which ones moved.
|
|
749
882
|
- Changed paths matching the **gate** — the tests, fixtures, and check
|
|
750
883
|
configuration that decide whether the work passes — are called out separately
|
|
751
|
-
in the prompt, listed in
|
|
884
|
+
in the prompt, listed in the run's `summary.json` as `comparison.gate_edits`,
|
|
752
885
|
and warned about on stderr as they happen.
|
|
753
886
|
|
|
754
887
|
A gate edit is not misconduct: plenty of tasks are *about* the tests. It is the
|
|
@@ -777,34 +910,49 @@ that `--fail-on-gate-edit` cannot fire there.
|
|
|
777
910
|
|
|
778
911
|
## Run artifacts
|
|
779
912
|
|
|
780
|
-
|
|
913
|
+
Everything volley knows about a run lives in a directory of its own, outside
|
|
914
|
+
the workspace: `<root>/<run id>/`, where the root is `--state-dir` (or
|
|
915
|
+
`state_dir` in a config file), else `$XDG_STATE_HOME/volley`, else
|
|
916
|
+
`~/.local/state/volley`.
|
|
781
917
|
|
|
782
918
|
```
|
|
783
|
-
|
|
919
|
+
<root>/<run id>/
|
|
784
920
|
config.json # resolved run config (immutable per run)
|
|
921
|
+
config.sig # this machine's signature over config.json (none from inside the sandbox)
|
|
785
922
|
summary.json # run-level usage, cost, status
|
|
786
923
|
trajectory.jsonl # every fascicle trajectory event (replayable, below)
|
|
787
924
|
feedback.md # current critic feedback (harness-written)
|
|
788
925
|
verdict # approved | changes_requested (harness-written)
|
|
789
|
-
iterations/NNN/ # per-iteration archive: feedback, verdict, check artifacts, summary
|
|
926
|
+
iterations/NNN/ # per-iteration archive: feedback, verdict, check artifacts and digest, summary
|
|
790
927
|
```
|
|
791
928
|
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
-
|
|
796
|
-
|
|
797
|
-
|
|
798
|
-
|
|
799
|
-
|
|
800
|
-
|
|
929
|
+
- **Why outside.** Some checks read every file in the tree, dot-directories
|
|
930
|
+
included, as markdownlint-cli2 and checkride's links walk do. Kept in the
|
|
931
|
+
workspace, the run's own `feedback.md` would fail them from the second
|
|
932
|
+
iteration on, a failure the builder can't fix. Kept out, the workspace holds
|
|
933
|
+
only the builder's work, and `--git` checkpoints never commit volley's files.
|
|
934
|
+
- **Finding a run.** A run prints `state: <dir>` when it starts and again in its
|
|
935
|
+
final summary, and `--json` and `summary.json` carry it as `run_state_dir`.
|
|
936
|
+
`--dry-run` names the root, and `--run-id` names the directory.
|
|
937
|
+
- **Unwritable.** A run directory volley can't write is a configuration error
|
|
938
|
+
(exit 5) before any spend. In the sandbox container, mount the run's own
|
|
939
|
+
directory ([Local builder](#local-builder)).
|
|
940
|
+
- **Pruning.** Nothing prunes old runs. A root volley creates is yours alone
|
|
941
|
+
(0700), since trajectories hold prompts and tool output. A reused `--run-id`
|
|
942
|
+
rotates the earlier run aside to `<run id>.bak.<timestamp>/`.
|
|
943
|
+
- **Older runs.** An older volley kept the same files in the workspace's
|
|
944
|
+
`.volley/`. `volley resume` still finds such a run there, and resumes it in
|
|
945
|
+
place once you pass `--trust-config`, since an older volley signed nothing
|
|
946
|
+
([Resume](#resume)).
|
|
947
|
+
|
|
948
|
+
`volley matrix` writes its sweep to `.volley-matrix/` in the workspace, and a
|
|
949
|
+
base config's `state_dir` moves each seat's run state as it would any run's.
|
|
801
950
|
|
|
802
951
|
The trajectory replays in fascicle's bundled viewer — the `fascicle-viewer` bin
|
|
803
952
|
ships inside the `fascicle` package; there is no separate viewer package:
|
|
804
953
|
|
|
805
954
|
```sh
|
|
806
|
-
pnpm dlx --package=fascicle fascicle-viewer
|
|
807
|
-
pnpm dlx --package=fascicle fascicle-viewer <root>/<run id>/trajectory.jsonl # under --state-dir
|
|
955
|
+
pnpm dlx --package=fascicle fascicle-viewer ~/.local/state/volley/<run id>/trajectory.jsonl
|
|
808
956
|
```
|
|
809
957
|
|
|
810
958
|
## Environment variables
|
|
@@ -813,8 +961,9 @@ See [.env.example](./.env.example): `VOLLEY_CLAUDE_BIN`, `VOLLEY_AUTH_MODE`,
|
|
|
813
961
|
`VOLLEY_LOG_LEVEL`, `VOLLEY_NO_COLOR`, `VOLLEY_PRICING_PATH`, the
|
|
814
962
|
`ANTHROPIC_API_KEY` billing-meter warning, each seat's base URL
|
|
815
963
|
(`VOLLEY_OLLAMA_URL`, `VOLLEY_LMSTUDIO_URL`, `VOLLEY_OPENROUTER_URL`), and
|
|
816
|
-
`OPENROUTER_API_KEY` for [OpenRouter](#openrouter) seats.
|
|
817
|
-
|
|
964
|
+
`OPENROUTER_API_KEY` for [OpenRouter](#openrouter) seats. Every run reads
|
|
965
|
+
`XDG_STATE_HOME` for its default state root, and `XDG_CONFIG_HOME` for where
|
|
966
|
+
it keeps the key it signs run configs with.
|
|
818
967
|
|
|
819
968
|
## Development
|
|
820
969
|
|