@feigi/fleet-ctl 3.71.16 → 3.72.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -43,12 +43,13 @@ Three claims, none substituting for another: **run-binding** (this green belongs
43
43
 
44
44
  ## The review result file
45
45
 
46
- **Dispatched as a fix-applier, you read the review off one file: `<scratch>/pr<pr>/<run>/review.json`, the path your prompt names — `<run>` the review's own run root, the one the PR's latest `reviewed=` token names.** It holds the bare result object the review returned, in this key order — the digest `pr`, `head`, `resume`, `testEnvironment`, `dimensionsRun`, `dimensionsUnrun`, `cwdAudit`, `counts`, then `snapshot`, `survived`, `refuted`, `unverified` — the `review-pr-<pr#>` runner member (`agents/fleet-review-runner.agent.md`) writes it. The controller reads the digest and never the findings, so **every per-PR ruling in this section is yours** — nobody upstream has made one for you. A prompt that pastes the findings in instead of naming the file hands you the same buckets, and every ruling here applies to them unchanged. Read the file a bucket at a time with `jq`, never whole into your context — a review's result runs to tens of KB:
46
+ **Dispatched as a fix-applier, you read the review off one file: `<scratch>/pr<pr>/<run>/review.json`, the path your prompt names — `<run>` the review's own run root, the one the PR's latest `reviewed=` token names.** It holds the bare result object the review returned, in this key order — the digest `pr`, `head`, `resume`, `testEnvironment`, `suiteVerdict`, `dimensionsRun`, `dimensionsUnrun`, `cwdAudit`, `counts`, then `snapshot`, `survived`, `refuted`, `unverified` — the `review-pr-<pr#>` runner member (`agents/fleet-review-runner.agent.md`) writes it. The controller reads the digest and never the findings, so **every per-PR ruling in this section is yours** — nobody upstream has made one for you. A prompt that pastes the findings in instead of naming the file hands you the same buckets, and every ruling here applies to them unchanged. Read the file a bucket at a time with `jq`, never whole into your context — a review's result runs to tens of KB:
47
47
 
48
48
  ```bash
49
49
  R=<scratch>/pr<pr>/<run>/review.json
50
- jq '{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts}' "$R" # the digest
50
+ jq '{pr, head, resume, testEnvironment, suiteVerdict, dimensionsRun, dimensionsUnrun, cwdAudit, counts}' "$R" # the digest
51
51
  jq -r '.testEnvironment' "$R" # what the shared test run is evidence about
52
+ jq -r '.suiteVerdict' "$R" # whether the shared test run is usable, and why not
52
53
  jq '.survived' "$R" # apply candidates, every one
53
54
  jq '[.unverified[] | select(.refutersDispatched > (.refutersInconclusive // 0))]' "$R" # a refuter crashed: defer, naming the crash
54
55
  jq '[.unverified[] | select(.refutersDispatched > 0 and .refutersDispatched == .refutersInconclusive)]' "$R" # every refuter abstained: defer and file, citing the inconclusive runs in its votes' `reason`, never as crashed
@@ -58,7 +59,7 @@ jq '.refuted' "$R" # refuted: your
58
59
 
59
60
  **Every finding in the file is yours to rule on, not the ones that read sharpest.** The file is the only complete copy a member can reach — the specialists' transcripts exist, but nothing you hold addresses one — so a finding you never read is one nobody rules on. `counts` is the check that you read them all: your apply/defer split covers every `survived` and `unverified` finding it counts, and your disposition record (step 2) carries an entry for each `refuted` one you reversed, with the evidence in its `reason`, which the controller copies to the ledger's `ruled` line.
60
61
 
61
- **Read `testEnvironment` and `cwdAudit` before you act on any result of the shared test run.** `testEnvironment` says whether the snapshot the specialists ran in was verified as the reviewed tree: verified, a red there is a fact about the tree; UNVERIFIED, tests that ask git what ships decline or fail for the environment, so a red or a short count there is not a finding until you reproduce it in the worktree, and not a reason to rule a dimension unrun either. `cwdAudit` is each specialist's audit of the checkout it inherited, `{dimension, state, line}`: `dirty` and `missing` are equally loud, because either names a checkout a specialist may have written to. Check it with `git -C <that path> status --porcelain -uall` — the path is in a `dirty` entry's `line`, and a `missing` entry's is the main checkout, the parent of `git rev-parse --path-format=absolute --git-common-dir` — and report what it printed; never revert content you did not write. A `dimensionsUnrun` entry is a dimension that did not cover its ground, its `reason` naming which failure: name it unrun in your report, because an absence of findings is not coverage, and re-run nothing — the review already re-dispatched every crashed specialist once, in-run.
62
+ **Read `testEnvironment` and `cwdAudit` before you act on any result of the shared test run.** `testEnvironment` says whether the snapshot the specialists ran in was verified as the reviewed tree: verified, a red there is a fact about the tree; UNVERIFIED, tests that ask git what ships decline or fail for the environment, so a red or a short count there is not a finding until you reproduce it in the worktree. `cwdAudit` is each specialist's audit of the checkout it inherited, `{dimension, state, line}`: `dirty` and `missing` are equally loud, because either names a checkout a specialist may have written to. Check it with `git -C <that path> status --porcelain -uall` — the path is in a `dirty` entry's `line`, and a `missing` entry's is the main checkout, the parent of `git rev-parse --path-format=absolute --git-common-dir` — and report what it printed; never revert content you did not write. A `dimensionsUnrun` entry is a specialist that returned nothing, its `reason` the crash: name it unrun in your report, because an absence of findings is not coverage, and re-run nothing — the review already re-dispatched every crashed specialist once, in-run. `suiteVerdict` is the review's one judgement on its shared test run, `{usable, reason}`: usable does not mean green, and the verdict says nothing about whether a dimension reviewed. When `suiteVerdict.usable` is false, name `shared test run not usable: <reason>` in your final verdict and do nothing more about it — your own `<testCmd>` run and CI cover the suite.
62
63
 
63
64
  **Scan the findings against each other for MUTUAL EXCLUSION, and rule before the conflicting change reaches the tree.** Refuters are blind to their siblings, so two can approve changes that cannot both land — one preserving a comment block another's change deletes, one keeping a binding another collapses. Two runs produced it; in one, a refuter proved a variable collapse behavior-equivalent over 844 cases while its own blast-radius scan named the two comment blocks a *different* refuter had just established as guarding distinct pinned hazards. You hold every finding, so only you can see the pair — and the scan's two inputs arrive at different times, so run it twice. **Before you apply anything**, group the findings by `file:line` and by whether one's fix undoes another's; that is in the findings' own text. **Then as each refuter you dispatch reports**, read its blast radius against what you have already ruled on. **A conflicting pair goes to a SINGLE refuter, briefed with both claims** — that is what settled one such pair, one refuter ruling *"B wins: A's premise is false"* where two independent refuters would each have approved their own side. Defer only when no single refuter can settle it, with the equivalence evidence recorded in the filed issue so the work is not redone. Never let both reach the tree and hope the diff coheres.
64
65
 
@@ -83,7 +84,7 @@ It cannot ask the user anything and cannot wait on CI — rebasing, watching che
83
84
  - **Do not trust the obvious contamination check.** `diff -rq` and `md5` against the mutating tree both returned *clean* — because the probe had been reverted between the two reads. A clean diff against a live tree is not evidence in either direction; only `snap-ro` and the object store settle it.
84
85
  - **`git archive` carries tracked files only.** No installed dependencies, no gitignored runner or config. Provision in the same step — run the Install step in the copy (read it off the worktree or main checkout, `derive-testcmd.sh <worktree or main checkout> install`; pointed at the copy it refuses, since a copy holds no Recipe cache), from the copy's root, before anyone measures in it — or specialists silently have no runnable suite and reason from source instead of measuring. That failure is invisible: you get confident prose where you asked for a measurement. Never `cp -R` an installed tree, and never symlink or clone its ignored paths into a copy: an install can record absolute paths back into the tree it ran in, so the copy runs that tree's source and a mutation made in the copy never runs.
85
86
  - **Initialize the copy before anyone measures in it — a bare `git archive` tree cannot validly run every suite.** It is not a git repository, so anything that asks git about the checkout breaks: `derive_workspace_id` shells out to `git rev-parse`, falls back to the directory basename and the hook tests fail on the *name* — naming the dir after the repo makes those pass **by coincidence, not correctness** — while every sweep that asks git what ships declines instead of running, which is the silent half (measured on this repo at `ffa9026`: 2401 tests, 2382 pass, 19 skipped in an extraction against 2401/2401/0 in a worktree, the totals identical). `git init && git add -A -f && git commit` in the extracted copy is what makes a count mean what it looks like; the review's snapshot agent does exactly that, verifies it by comparing the new commit's tree hash against the reviewed commit's, and then runs the Install step in it, and a hand-cut tree needs the same before you read a count off it. **Every agent you send to work in its own copy of the snapshot makes that copy the same way:** copy the tracked files only (`git -C <snapshot> archive HEAD | tar -x -C <copy>`, or a `git clone` of the snapshot repository), `git init`, `git add -A -f` and commit, then run the Install step in the copy before measuring anything in it — never `cp -R` the snapshot. Suites that need the real worktree for some *other* reason — the stack below — still do.
86
- - **Give specialists a stack-free test command, not the worktree's runner, and their own scratch dir** — `<scratch>/pr<N>/fleet-review-<key>/`, resolved to an absolute path that you write into that specialist's prompt; a specialist never derives its own. Specialists inherit no environment, and one falling back to the default config runs a `globalSetup` that brings the shared compose stack up and tears it down, recreating the DB mid-run for every sibling. **The snapshot does not cover this** — the compose project name comes from the environment, not the working directory, so three agents on three copies still collide. But `./agent-test` does not fix it either: it exports **one** `TEST_COMPOSE_PROJECT`/port triple per *worktree* (`ab-<issue>`), so N concurrent specialists sharing it tear down each other's postgres mid-run. Observed: a probe returned `No test files found / No such container` and would have read as a test failure. So tell specialists the repo's test command directly — the Test entrypoint, read off the Recipe cache the way step 3 reads it (`derive-testcmd.sh <worktree or main checkout> test` prints it; the snapshot carries no `.fleet/` cache, so pointing it at the snapshot refuses) — when it brings up no shared stack, and the repo's own stack-free config when it does; never name a config the repo does not have, which sends every specialist into a missing-file error that reads as a broken tree rather than a bad instruction. Keep `./agent-test` for your own verification. **Hand that command with its reading rule: `tests 0` is a failed run, not a pass.** A command is not self-checking: a path glob or filter that matches nothing can exit 0 having run nothing. Measured with `node --test` over a glob matching no file: bash passes the pattern through literally, node globs it itself and finds no files, and you get `ℹ tests 0` / exit 0 — and `# tests 0` to a pipe on older node, so a rule that greps only for `ℹ` finds nothing in a CI log and cannot tell a zero-test run from a full one. Do not count on the shell to catch it: zsh errors (`no matches found`, exit 1) and bash does not, so the guard has to be the reading rule. **And zero is not the only no-work count.** **0 passes with no failures** is everything skipped, and a count **materially below the full suite** is a partial copy of the tree — which is the *documented* behavior, since the prompt tells specialists to run their probe work inside their own copy of the snapshot and nothing anchors an expected count for them. `review-core.mjs` classifies the first itself, in `unrunReason`; the second it structurally cannot, because that function is kept pure and so never learns how many tests the tree has. Handing the rule over with the command is the only guard the partial-copy case has. Carry the rule, not the command: wherever a repo has a stack, N concurrent specialists sharing one runner collide, and the escape is whatever that repo's stack-free config is.
87
+ - **Give specialists a stack-free test command, not the worktree's runner, and their own scratch dir** — `<scratch>/pr<N>/fleet-review-<key>/`, resolved to an absolute path that you write into that specialist's prompt; a specialist never derives its own. Specialists inherit no environment, and one falling back to the default config runs a `globalSetup` that brings the shared compose stack up and tears it down, recreating the DB mid-run for every sibling. **The snapshot does not cover this** — the compose project name comes from the environment, not the working directory, so three agents on three copies still collide. But `./agent-test` does not fix it either: it exports **one** `TEST_COMPOSE_PROJECT`/port triple per *worktree* (`ab-<issue>`), so N concurrent specialists sharing it tear down each other's postgres mid-run. Observed: a probe returned `No test files found / No such container` and would have read as a test failure. So tell specialists the repo's test command directly — the Test entrypoint, read off the Recipe cache the way step 3 reads it (`derive-testcmd.sh <worktree or main checkout> test` prints it; the snapshot carries no `.fleet/` cache, so pointing it at the snapshot refuses) — when it brings up no shared stack, and the repo's own stack-free config when it does; never name a config the repo does not have, which sends every specialist into a missing-file error that reads as a broken tree rather than a bad instruction. Keep `./agent-test` for your own verification. **Hand that command with its reading rule: `tests 0` is a failed run, not a pass.** A command is not self-checking: a path glob or filter that matches nothing can exit 0 having run nothing. Measured with `node --test` over a glob matching no file: bash passes the pattern through literally, node globs it itself and finds no files, and you get `ℹ tests 0` / exit 0 — and `# tests 0` to a pipe on older node, so a rule that greps only for `ℹ` finds nothing in a CI log and cannot tell a zero-test run from a full one. Do not count on the shell to catch it: zsh errors (`no matches found`, exit 1) and bash does not, so the guard has to be the reading rule. **And zero is not the only no-work count.** **0 passes with no failures** is everything skipped, and a count **materially below the full suite** is a partial copy of the tree — which is the *documented* behavior, since the prompt tells specialists to run their probe work inside their own copy of the snapshot and nothing anchors an expected count for them. `review-core.mjs` classifies the first itself, in `suiteVerdictOf`; the second it structurally cannot, because that function is kept pure and so never learns how many tests the tree has. Handing the rule over with the command is the only guard the partial-copy case has. Carry the rule, not the command: wherever a repo has a stack, N concurrent specialists sharing one runner collide, and the escape is whatever that repo's stack-free config is.
87
88
  - **Completion is not delivery — but you can reach a specialist, so never wait idle.** A specialist you dispatched reports when it settles; if a report is missing, `hub send` it by name and ask for the report. Judge delivery by whether a report came out. **Do not settle a dimension you hold no report for**: ask the specialist, then the controller by name; only when neither answers, rule and name that dimension **unrun** — a killed specialist never reports, so waiting on one is unbounded. Never ship "dispatched, never delivered" as settled; a ruling citing a report you do not hold gets verified from source, never applied on trust.
88
89
  - **Apply only once the fan-out is settled — every report either in hand or accounted for.** Collection is not guaranteed; a report that never arrived is an open dimension, not a smaller set. And editing while they read is the same defect as probing — one specialist reviewed uncommitted code that was never in the PR diff.
89
90
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@feigi/fleet-ctl",
3
- "version": "3.71.16",
3
+ "version": "3.72.1",
4
4
  "description": "Agent fleet: run-team controller, merge bot, PR reviewer, and the ticket pipeline they share",
5
5
  "license": "Apache-2.0",
6
6
  "repository": {
@@ -125,9 +125,17 @@ export function fleetFile(name, { cwd = process.cwd(), timeoutMs, canonicalise =
125
125
  export function scratchRoot({ cwd = process.cwd(), timeoutMs, home = homedir() } = {}) {
126
126
  const workspace = dirname(fleetFile(null, { cwd, timeoutMs, canonicalise: true }));
127
127
  const hex = createHash("sha256").update(workspace).digest("hex").slice(0, 8);
128
- return join(home, ".fleet", "scratch", `${basename(workspace)}-${hex}`);
128
+ return join(scratchParent(home), `${basename(workspace)}-${hex}`);
129
129
  }
130
130
 
131
+ /** The directory every run's scratch root sits directly in: `<home>/.fleet/scratch`. */
132
+ export function scratchParent(home = homedir()) {
133
+ return join(home, ".fleet", "scratch");
134
+ }
135
+
136
+ /** The shape of a scratch root's last segment: `<workspace basename>-<8 hex>`. */
137
+ export const SCRATCH_KEY_SHAPE = /-[0-9a-f]{8}$/;
138
+
131
139
  function nearestExisting(path) {
132
140
  let at = path;
133
141
  while (!existsSync(at)) {
@@ -40,8 +40,11 @@
40
40
  // cockpit `board.mjs serve --open` launched detached, `--spend-dir`
41
41
  // passed through. Recorded; never fails the run.
42
42
  // rotate `ledger.mjs rotate`. Exit 2 is a refusal and fails the step.
43
- // reclaim `reclaim-scratch.mjs --root <scratch> --after-rotate --apply`.
44
- // Recorded; never fails the run.
43
+ // reclaim `reclaim-scratch.mjs --root <scratch> --after-rotate --apply`,
44
+ // then `reclaim-scratch.mjs --siblings --root <scratch> --apply`
45
+ // for the other keys under the scratch parent, which runs
46
+ // whatever the first answered. What either receipt's failed[]
47
+ // names is listed. Recorded; never fails the run.
45
48
  // recipe `derive-testcmd.sh . install` and `. test --at origin/main`.
46
49
  // Either exiting 1 is a cache with nothing valid in it: pending,
47
50
  // `NEED derive-recipe`. Exit 3 (a tool the reader could not run)
@@ -277,8 +280,28 @@ function rotate(ctx) {
277
280
  }
278
281
 
279
282
  function reclaim({ cwd, scratch }) {
280
- const r = node("reclaim-scratch.mjs", ["--root", scratch, "--after-rotate", "--apply"], { cwd });
281
- return ok(r.status === 0 ? "scratch reclaimed" : withText(`reclaim-scratch.mjs ${r.how}, not enforced`, r));
283
+ const live = node("reclaim-scratch.mjs", ["--root", scratch, "--after-rotate", "--apply"], { cwd });
284
+ // The sibling pass runs whatever the live sweep answered: a failure there is recorded beside it, never a reason to skip it.
285
+ const siblings = node("reclaim-scratch.mjs", ["--siblings", "--root", scratch, "--apply"], { cwd });
286
+ return ok([
287
+ live.status === 0 ? "scratch reclaimed" : withText(`reclaim-scratch.mjs ${live.how}, not enforced`, live),
288
+ ...unreclaimed("scratch root", live),
289
+ ...(siblings.status === 0 ? unreclaimed("sibling keys", siblings) : [withText(`sibling keys: reclaim-scratch.mjs ${siblings.how}, not enforced`, siblings)]),
290
+ ].join("\n"));
291
+ }
292
+
293
+ // A pass that exited 0 still lists in its receipt's `failed[]` what it could
294
+ // not judge or remove: each is named here, and none fails the step.
295
+ function unreclaimed(what, r) {
296
+ if (r.status !== 0) return [];
297
+ let entries;
298
+ try {
299
+ ({ failed: entries } = JSON.parse(r.stdout));
300
+ } catch (e) {
301
+ return [`${what}: could not read reclaim-scratch.mjs's receipt: ${e.message}`];
302
+ }
303
+ if (!Array.isArray(entries)) return [`${what}: reclaim-scratch.mjs's receipt holds no failed[]`];
304
+ return entries.map((x) => `${what}: could not reclaim ${x?.path} (${x?.error})`);
282
305
  }
283
306
 
284
307
  function recipe({ cwd }) {
@@ -6,11 +6,19 @@
6
6
  //
7
7
  // reclaim-scratch.mjs --root <abs> [--after-rotate] [--apply]
8
8
  // reclaim-scratch.mjs --pr <N> --root <abs> [--apply]
9
+ // reclaim-scratch.mjs --siblings --root <abs> [--apply]
9
10
  //
10
11
  // Dry run by default; `--apply` removes. `--root` is required, an absolute
11
12
  // path to an existing directory: the controller passes the path
12
13
  // scratch-root.mjs prints.
13
14
  //
15
+ // `--siblings` is the sibling pass, its own invocation: `--root` names the live
16
+ // key, which must be this workspace's own scratch root (the path
17
+ // scratch-root.mjs prints), and the pass judges the other key-shaped
18
+ // directories beside it by age alone (judgeSiblings below), judging each again
19
+ // just before `--apply` removes it. It reads no ledger and is never combined
20
+ // with --pr or --after-rotate.
21
+ //
14
22
  // THE ROOT SWEEP judges every entry directly under `--root`:
15
23
  // pr<M>/, dispositions-fix-pr-<M>[-x].json
16
24
  // removed when PR #M is MERGED or CLOSED on GitHub and no live
@@ -86,10 +94,10 @@
86
94
 
87
95
  import { spawnSync } from "node:child_process";
88
96
  import { lstatSync, readdirSync, statSync } from "node:fs";
89
- import { dirname, isAbsolute, join, resolve } from "node:path";
97
+ import { basename, dirname, isAbsolute, join, resolve } from "node:path";
90
98
  import { fileURLToPath } from "node:url";
91
99
  import { defineFlags, isDigits, makeDie, writeAll } from "./arg.mjs";
92
- import { fleetFile, FleetDirUnresolvable } from "./fleet-dir.mjs";
100
+ import { fleetFile, FleetDirUnresolvable, SCRATCH_KEY_SHAPE, scratchRoot } from "./fleet-dir.mjs";
93
101
  import { gitEnv } from "./git-env.mjs";
94
102
  import { isCLI } from "./is-cli.mjs";
95
103
  import { liveOnLedger, parseMember, REVIEWER } from "./ledger-grammar.mjs";
@@ -143,8 +151,10 @@ function how(r) {
143
151
  * `cutoff`, or null when everything is older. Symlinks are not followed. A
144
152
  * directory that cannot be listed counts its own mtime only, and so does one
145
153
  * that can be listed but not searched: an entry inside it whose lstat is
146
- * refused is skipped, its parent's mtime already counted. */
147
- export function freshUnder(path, cutoff) {
154
+ * refused is skipped, its parent's mtime already counted. Given `unreadable`,
155
+ * an array, each path skipped for EACCES or EPERM is pushed onto it as
156
+ * `<path>: <code>`. */
157
+ export function freshUnder(path, cutoff, unreadable = null) {
148
158
  const stack = [path];
149
159
  while (stack.length) {
150
160
  const p = stack.pop();
@@ -152,6 +162,7 @@ export function freshUnder(path, cutoff) {
152
162
  try {
153
163
  st = lstatSync(p);
154
164
  } catch (e) {
165
+ if (e.code === "EACCES" || e.code === "EPERM") unreadable?.push(`${p}: ${e.code}`);
155
166
  if (e.code === "ENOENT" || e.code === "EACCES" || e.code === "EPERM") continue;
156
167
  throw e;
157
168
  }
@@ -161,6 +172,7 @@ export function freshUnder(path, cutoff) {
161
172
  try {
162
173
  names = readdirSync(p);
163
174
  } catch (e) {
175
+ if (e.code === "EACCES" || e.code === "EPERM") unreadable?.push(`${p}: ${e.code}`);
164
176
  if (e.code === "EACCES" || e.code === "EPERM" || e.code === "ENOENT") continue;
165
177
  throw e;
166
178
  }
@@ -421,9 +433,46 @@ function staleRunRoots(parent, cutoff, kept) {
421
433
  return remove;
422
434
  }
423
435
 
436
+ // THE SIBLING PASS judges every key-shaped directory directly under the scratch
437
+ // parent of `root`, other than the live key, by age alone: one with nothing
438
+ // modified within the retention goes whole; one with anything fresh stays whole,
439
+ // and so does one with any part that could not be read, since what that part
440
+ // holds is unknown. Other entries under the parent, links and the live key are
441
+ // not judged at all. The keys are judged in name order.
442
+ function judgeSiblings(root, cutoff) {
443
+ const parent = dirname(root);
444
+ const live = basename(root);
445
+ const remove = [];
446
+ const kept = [];
447
+ const failed = [];
448
+ const entries = readdirSync(parent, { withFileTypes: true }).sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
449
+ for (const entry of entries) {
450
+ if (!entry.isDirectory() || entry.name === live || !SCRATCH_KEY_SHAPE.test(entry.name)) continue;
451
+ const path = join(parent, entry.name);
452
+ let verdict;
453
+ try {
454
+ verdict = judgeSibling(path, cutoff);
455
+ } catch (e) {
456
+ failed.push({ path, error: e.message });
457
+ continue;
458
+ }
459
+ (verdict.idle ? remove : kept).push({ path, reason: verdict.reason });
460
+ }
461
+ return { remove, kept, failed };
462
+ }
463
+
464
+ // One sibling key's verdict: `idle` only when nothing under it is fresh and
465
+ // every part of it could be read.
466
+ function judgeSibling(path, cutoff) {
467
+ const unreadable = [];
468
+ if (freshUnder(path, cutoff, unreadable) !== null) return { idle: false, reason: "sibling key modified within 7 days" };
469
+ if (unreadable.length > 0) return { idle: false, reason: `sibling key could not be read whole (${unreadable[0]})` };
470
+ return { idle: true, reason: "sibling key idle past 7 days" };
471
+ }
472
+
424
473
  function main() {
425
474
  const { arg, has, sweep, stray } = defineFlags(die, {
426
- flags: { root: "value", pr: "value", "after-rotate": "bool", apply: "bool" },
475
+ flags: { root: "value", pr: "value", "after-rotate": "bool", apply: "bool", siblings: "bool" },
427
476
  });
428
477
  sweep();
429
478
  stray();
@@ -431,7 +480,8 @@ function main() {
431
480
  const pr = arg("pr");
432
481
  const afterRotate = has("after-rotate");
433
482
  const apply = has("apply");
434
- const usage = "usage: reclaim-scratch.mjs --root <abs> [--after-rotate] [--apply] | --pr <N> --root <abs> [--apply]";
483
+ const siblings = has("siblings");
484
+ const usage = "usage: reclaim-scratch.mjs --root <abs> [--after-rotate] [--apply] | --pr <N> --root <abs> [--apply] | --siblings --root <abs> [--apply]";
435
485
  if (rootArg === null) die(`--root is required — ${usage}`);
436
486
  if (!isAbsolute(rootArg)) die(`--root must be an absolute path, got ${JSON.stringify(rootArg)}`);
437
487
  if (pr !== null && !isDigits(pr)) die(`--pr must be a PR number, got ${JSON.stringify(pr)}`);
@@ -442,6 +492,19 @@ function main() {
442
492
  } catch (e) {
443
493
  die(`--root ${root} is not a directory: ${e.message}`);
444
494
  }
495
+ if (siblings) {
496
+ if (pr !== null) die("--siblings and --pr are separate passes");
497
+ if (afterRotate) die("--after-rotate is the root sweep's; --siblings reads no ledger");
498
+ // The live key is never judged, so it must be this workspace's own: any other
499
+ // key passed here would put the real live key among the judged siblings.
500
+ let own;
501
+ try {
502
+ own = scratchRoot();
503
+ } catch (e) {
504
+ die(`--siblings judges the keys beside this workspace's scratch root, which could not be resolved: ${e.message}`);
505
+ }
506
+ if (root !== own) die(`--siblings judges the keys beside the live one: --root must be this workspace's scratch root ${own}, got ${root}`);
507
+ }
445
508
 
446
509
  const now = Date.now();
447
510
  let ledger = null;
@@ -456,6 +519,16 @@ function main() {
456
519
  remove = [];
457
520
  failed.push({ path: parent, error: e.message });
458
521
  }
522
+ } else if (siblings) {
523
+ try {
524
+ const judged = judgeSiblings(root, now - RETENTION_MS);
525
+ remove = judged.remove;
526
+ kept = judged.kept;
527
+ failed.push(...judged.failed);
528
+ } catch (e) {
529
+ remove = [];
530
+ failed.push({ path: dirname(root), error: e.message });
531
+ }
459
532
  } else {
460
533
  ledger = ledgerFile();
461
534
  if (ledger === null && !afterRotate) die("no .fleet/ledger.md — without it no live member can hold its scratch, so nothing is reclaimed; pass --after-rotate only straight after `ledger.mjs rotate`");
@@ -477,6 +550,14 @@ function main() {
477
550
  }
478
551
  let error;
479
552
  try {
553
+ // A sibling key belongs to another workspace's runs, which can write to it
554
+ // while an earlier key is still being removed: it is judged again here, so
555
+ // a key that turned fresh since the pass judged it is kept.
556
+ const again = siblings ? judgeSibling(entry.path, now - RETENTION_MS) : null;
557
+ if (again !== null && !again.idle) {
558
+ kept.push({ path: entry.path, reason: again.reason });
559
+ continue;
560
+ }
480
561
  error = removeEntry(entry.path);
481
562
  } catch (e) {
482
563
  error = e.message;
@@ -3,7 +3,7 @@
3
3
  // This file holds every host-independent declaration PR review needs: the
4
4
  // JSON schemas, `usableDiff`, `readRules`, `resolveTestCmd`,
5
5
  // `decodeArgs`, `selectDimensions`, `resolveDimensions`, `snapshotMissing`,
6
- // `installReason`/`installFinding`, `unrunReason`/`unrunEntries`/`unrunCrashed`,
6
+ // `installReason`/`installFinding`, `suiteVerdictOf`/`unrunCrashed`,
7
7
  // `sharedRunNote`, `copyRule`,
8
8
  // `verdictFor`, `resumeFor`, and `runReview(host, args)` — the orchestration
9
9
  // function itself.
@@ -152,7 +152,7 @@ export const SNAPSHOT_SCHEMA = {
152
152
  // log path are NOT asked for: the caller handed both out, so it already knows
153
153
  // them, and an agent's echo of either could only disagree. Every count is
154
154
  // optional and stays absent when the log does not state it — an absent
155
- // `tests` is how `unrunReason` learns the run produced no counts at all
155
+ // `tests` is how `suiteVerdictOf` learns the run produced no counts at all
156
156
  // (crash, deadline, no summary), which a typed-in 0 would misreport as a run
157
157
  // that collected nothing.
158
158
  export const TEST_RUN_SCHEMA = {
@@ -652,26 +652,14 @@ function failingDesc(run) {
652
652
  return run.cancelled ? `${run.fail || 0} failing and ${run.cancelled} cancelled tests` : `${run.fail} failing tests`;
653
653
  }
654
654
 
655
- // Failing tests are owned by the review as a whole: `findings` is
656
- // every finding that reached the payload, and ONE from any dimension
657
- // satisfies the check for all — no dimension is asked to duplicate a
658
- // sibling's, and none is marked unrun because a sibling filed it. A refuted
659
- // finding does not count: the review rejected it, so an unrelated claim its
660
- // verifiers threw out would otherwise stand in for failures nobody reported.
661
- // A dimension whose verify stage died never reaches the payload, so its
662
- // findings are not here to count either.
663
- export function unrunReason(run, findings) {
664
- const why = testRunReason(run);
665
- if (why) return why;
666
- if (failingOf(run) > 0 && !findings.some((f) => f && f.verdict !== "refuted"))
667
- return `\`${run.command || "the test command"}\` reported ${failingDesc(run)} and no selected dimension filed a finding about them that the review kept`;
668
- return null;
669
- }
670
-
671
- // One verdict, every key: the shared run is the same run for each of them.
672
- export function unrunEntries(run, findings, keys) {
673
- const why = unrunReason(run, findings);
674
- return why ? keys.map((dimension) => ({ dimension, reason: why })) : [];
655
+ // The review's single judgement on its shared test run: `{usable, reason}`,
656
+ // `reason` null when the run is usable. It reads the run alone — never a
657
+ // specialist's findings, and never whether a dimension reviewed: a usable run
658
+ // is not a green one (failing tests are the owner dimension's to file), and an
659
+ // unusable one says nothing about what the specialists read.
660
+ export function suiteVerdictOf(run) {
661
+ const reason = testRunReason(run);
662
+ return { usable: reason === null, reason };
675
663
  }
676
664
 
677
665
  export const CRASHED_REASON = "the reviewer returned nothing — spend limit, timeout, or terminal error";
@@ -719,7 +707,7 @@ mutation in the copy never runs.`;
719
707
  export function sharedRunNote(run, owner, key) {
720
708
  if (run.unprovisioned)
721
709
  return `Tests: this review did NOT run its test command: ${run.unprovisioned}.
722
- Every dimension of this review is reported unrun for it — do not run the full
710
+ The review reports its suite verdict not usable for it — do not run the full
723
711
  command yourself to replace it, and file nothing about the install: the review
724
712
  reports it itself.
725
713
  command: ${run.command}`;
@@ -737,8 +725,8 @@ Read the log for anything the counts do not say.`,
737
725
  ];
738
726
  const why = testRunReason(run);
739
727
  if (why)
740
- lines.push(`This run is NOT usable: ${why}. Every dimension of this review is reported
741
- unrun for it — do not run the full command yourself to replace it.`);
728
+ lines.push(`This run is NOT usable: ${why}. The review reports its suite verdict not
729
+ usable for it — do not run the full command yourself to replace it.`);
742
730
  else if (failingOf(run) > 0)
743
731
  lines.push(
744
732
  key === owner
@@ -749,9 +737,8 @@ narrowed run in your own copy of the snapshot, where \`<testCmd>\` is the
749
737
  command above, and when that run does not show it failing, file no finding
750
738
  above \`suggestion\` on it and name both runs. A failure you cannot separate
751
739
  from the environment is still filed — say so in the finding, so it is
752
- reproduced in the worktree before anyone acts on it. If none is filed, or this
753
- review's refuters reject every finding filed, every dimension of this review
754
- is reported unrun. A red is read by these rules and nothing else:
740
+ reproduced in the worktree before anyone acts on it. A red is read by these
741
+ rules and nothing else:
755
742
 
756
743
  ${VALID_RED_RULE}
757
744
 
@@ -841,7 +828,7 @@ export function resumeFor(unverified) {
841
828
  // same reason a struct's fields stay ordered: a reader scanning top-down
842
829
  // meets the fields that matter first.
843
830
  // `snapshot`, `survived`, `refuted` and `unverified` follow it.
844
- export const DIGEST_KEYS = ["pr", "head", "resume", "testEnvironment", "dimensionsRun", "dimensionsUnrun", "cwdAudit", "counts"];
831
+ export const DIGEST_KEYS = ["pr", "head", "resume", "testEnvironment", "suiteVerdict", "dimensionsRun", "dimensionsUnrun", "cwdAudit", "counts"];
845
832
 
846
833
  export function digestOf(result) {
847
834
  return Object.fromEntries(DIGEST_KEYS.map((k) => [k, result[k]]));
@@ -1162,8 +1149,8 @@ and uncoveredError only replaces uncovered.`,
1162
1149
  const testCmd = resolveTestCmd(A.testCmd, snap);
1163
1150
  log(`testCmd ${A.testCmd ? "(caller override)" : "(derived)"} ${testCmd}`);
1164
1151
  // An Install step that failed or never ran leaves a snapshot that cannot
1165
- // measure the tree: the shared run is skipped, every dimension is reported
1166
- // unrun through `testRunReason`, and a non-zero exit is filed as a finding.
1152
+ // measure the tree: the shared run is skipped, the suite verdict is not
1153
+ // usable through `testRunReason`, and a non-zero exit is filed as a finding.
1167
1154
  const notInstalled = installReason(snap);
1168
1155
  log(notInstalled ? `install: ${notInstalled}` : `install: ${snap.installCmd ?? "the Install step"} exited 0 in the snapshot`);
1169
1156
 
@@ -1195,8 +1182,8 @@ and uncoveredError only replaces uncovered.`,
1195
1182
  // review was up to six full sweeps of one immutable snapshot. NOT wrapped in
1196
1183
  // `retryCrashed`: a crashed dispatch may already have launched the command,
1197
1184
  // so a retry would be a second launch — the load this step removes. A null
1198
- // or thrown answer is a run with no counts, which `unrunReason` reports for
1199
- // every dimension, never a pass.
1185
+ // or thrown answer is a run with no counts, which `suiteVerdictOf` reports as
1186
+ // not usable, never a pass.
1200
1187
  const logPath = `${snap.runRoot}/test-run.log`;
1201
1188
  phase("Test run");
1202
1189
  const ran = notInstalled ? null : await agent(
@@ -1247,7 +1234,7 @@ copied as printed; omit any the runner does not print. A test the runner reports
1247
1234
  as failing by design (vitest \`expected fail\`, pytest \`xfailed\`) goes in \`expectedFail\`, never in
1248
1235
  \`fail\`. Copy the counts, never judge them: 'tests 0' is a FAILED run, not a
1249
1236
  pass, and 0 passes with no failures is everything skipped — report both
1250
- exactly as printed, and the caller reports every dimension unrun for them.
1237
+ exactly as printed, and the caller reports its suite verdict not usable for them.
1251
1238
 
1252
1239
  If the command hit its deadline, crashed before printing a summary, or printed
1253
1240
  no counts at all, omit every count and say what happened in \`error\`. Never
@@ -1263,7 +1250,6 @@ learns the run produced none.`,
1263
1250
  // selected dimension is told to file them, and every other one not to.
1264
1251
  const failureOwner = dimensions[0].key;
1265
1252
 
1266
- const dimensionsUnrun = [];
1267
1253
  // Per-dimension record of the specialist's own CWD-AUDIT line (see
1268
1254
  // `cwdAuditFrom` above) — `{dimension, state, line}`, `state` one of
1269
1255
  // "clean"/"dirty"/"unrepo"/"missing". Populated for every dispatched
@@ -1282,13 +1268,13 @@ learns the run produced none.`,
1282
1268
  // tree it must not write to, `pwd` fixes which directory that is, and the
1283
1269
  // `CWD-AUDIT:` line is what makes a clean run say so — an audit reported
1284
1270
  // only when dirty is indistinguishable from one never run, the same reading
1285
- // `unrunReason` applies to a shared test run that reports nothing.
1271
+ // `suiteVerdictOf` applies to a shared test run that reports nothing.
1286
1272
  phase("Review");
1287
1273
  const reviewed = await pipeline(
1288
1274
  dimensions,
1289
1275
  // A crashed specialist (null or thrown) is re-dispatched once
1290
1276
  // before it counts as unrun. A specialist that RETURNED is never re-run
1291
- // here — the shared run, not the specialist, is what `unrunEntries` judges.
1277
+ // here — the shared run is judged once, by `suiteVerdictOf`.
1292
1278
  (d) =>
1293
1279
  retryCrashed(
1294
1280
  () =>
@@ -1489,14 +1475,10 @@ how three reviews from one cell left four files modified in that checkout.`,
1489
1475
  const refuted = all.filter((f) => f.verdict === "refuted");
1490
1476
  const unverified = all.filter((f) => f.verdict === "unverified");
1491
1477
 
1492
- // One verdict on the shared run, for every dimension whose chain did
1493
- // not die — a crashed one is named below with its own reason instead, so no
1494
- // key is listed twice. Read only once every dimension has been verified: the
1495
- // failing-tests check reads the findings that reached the payload, across
1496
- // ALL dimensions, refuted ones excluded.
1497
- const live = dimensions.filter((_, i) => reviewed[i]).map((d) => d.key);
1498
- dimensionsUnrun.push(...unrunEntries(sharedRun, all, live));
1499
- dimensionsUnrun.push(...unrunCrashed(reviewed, dimensions));
1478
+ // A crashed dimension is named with its own reason. The shared run is not
1479
+ // a dimension's crash: it is judged once, as `suiteVerdict`, below.
1480
+ const dimensionsUnrun = unrunCrashed(reviewed, dimensions);
1481
+ const suiteVerdict = suiteVerdictOf(sharedRun);
1500
1482
 
1501
1483
  const { crashed, resume } = resumeFor(unverified);
1502
1484
 
@@ -1516,19 +1498,24 @@ how three reviews from one cell left four files modified in that checkout.`,
1516
1498
  // checked" for "survived".
1517
1499
  // `dimensionsRun` names what was DISPATCHED after the size trim: a trimmed
1518
1500
  // fan-out must say so, never read as full coverage. It is not a coverage claim
1519
- // on its own and never was — a specialist can be dispatched and die, or the
1520
- // review's one shared test run can execute nothing — so
1521
- // `dimensionsUnrun` names which of those keys did not cover their ground,
1522
- // and why. A key in the first and NOT in the second ran a suite — not that
1523
- // it is covered. `unrunReason` reads the shared run's counts and
1501
+ // on its own and never was — a specialist can be dispatched and die — so
1502
+ // `dimensionsUnrun` names which of those keys returned nothing, and why. A
1503
+ // key in the first and NOT in the second returned a review — not that it is
1504
+ // covered.
1505
+ //
1506
+ // `suiteVerdict` is the review's one judgement on its shared test run, apart
1507
+ // from both: a run that executed nothing usable does not make a dimension
1508
+ // unrun, because the dimensions that returned reviewed and their findings
1509
+ // were verified as usual. `suiteVerdictOf` reads the shared run's counts and
1524
1510
  // quotes its command into its message; it never checks that the test-run
1525
1511
  // agent ran the command it was handed rather than a narrower one, so a
1526
- // substituted runner is not classified unrun.
1512
+ // substituted runner is not classified not usable.
1527
1513
  //
1528
- // The two are siblings rather than one filtered list because they answer
1529
- // different questions. Subtracting the unrun ones from `dimensionsRun` would
1530
- // make a crashed dimension indistinguishable from one the size tier never
1531
- // dispatched — this ticket set's own defect, moved one field over.
1514
+ // `dimensionsRun` and `dimensionsUnrun` are siblings rather than one
1515
+ // filtered list because they answer different questions. Subtracting the
1516
+ // unrun ones from `dimensionsRun` would make a crashed dimension
1517
+ // indistinguishable from one the size tier never dispatched — this ticket
1518
+ // set's own defect, moved one field over.
1532
1519
  //
1533
1520
  // Digest first, in DIGEST_KEYS order, bulk last — see DIGEST_KEYS
1534
1521
  // above for why the order is the contract.
@@ -1542,6 +1529,7 @@ how three reviews from one cell left four files modified in that checkout.`,
1542
1529
  // never be left unable to tell an environment artifact from a regression,
1543
1530
  // and that costs nothing when there is nothing wrong to report.
1544
1531
  testEnvironment: environmentNote(snap),
1532
+ suiteVerdict,
1545
1533
  dimensionsRun: dimensions.map((d) => d.key),
1546
1534
  dimensionsUnrun,
1547
1535
  // `cwdAuditFrom`'s per-dimension read of the specialist's own
@@ -1662,7 +1662,7 @@ since none runs against a finished PR, and a probe that
1662
1662
  cannot answer keeps the unit counted. A unit is not the agents a review fans
1663
1663
  out to — 1 snapshot + up to 6 specialists + 2 refuters per critical/important finding.
1664
1664
 
1665
- It returns `{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts, snapshot, survived, refuted, unverified}` —
1665
+ It returns `{pr, head, resume, testEnvironment, suiteVerdict, dimensionsRun, dimensionsUnrun, cwdAudit, counts, snapshot, survived, refuted, unverified}` —
1666
1666
  the **digest** first, then the snapshot and the findings — and it lands as
1667
1667
  one artefact, `<scratch>/pr<pr>/<run>/review.json`, holding that bare object.
1668
1668
  `<run>` is the review's own `run-XXXXXXXX` run root, the one its snapshot sits
@@ -1671,7 +1671,7 @@ of its own and never replaces the first. The `review-pr-<pr#>` runner writes it
1671
1671
  and reports the digest, the path and the `reviewed=` token naming that run.
1672
1672
 
1673
1673
  **You read the digest; the findings are the fix-applier's.**
1674
- `jq '{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts}' <scratch>/pr<pr>/<run>/review.json`
1674
+ `jq '{pr, head, resume, testEnvironment, suiteVerdict, dimensionsRun, dimensionsUnrun, cwdAudit, counts}' <scratch>/pr<pr>/<run>/review.json`
1675
1675
  is the whole of your read, and it is what you record: `reviewed=<head>:<survived>/<refuted>/<unverified>:<run>`,
1676
1676
  off `head`, `counts` and the run root's name — the runner's `ledger` field — on
1677
1677
  the PR's row, with `ledger.mjs review <pr>`. Every later path to the review is built from that token, before a
@@ -1714,16 +1714,26 @@ refutation the report defers.
1714
1714
 
1715
1715
  **`dimensionsRun` is the dispatch; `dimensionsUnrun` is what names a gap.** A
1716
1716
  specialist that dies contributes zero findings while its key stays in
1717
- `dimensionsRun`; a review whose one shared test run executed nothing — the
1718
- review runs the test command once, and every specialist reads that run instead
1719
- of running its own — leaves every key there with no suite behind it. So
1720
- read the two together. A key
1721
- in `dimensionsRun` and NOT in `dimensionsUnrun` ran a suite; every
1722
- `dimensionsUnrun` entry is `{dimension, reason}` naming which failure it was.
1717
+ `dimensionsRun`. So read the two together. A key
1718
+ in `dimensionsRun` and NOT in `dimensionsUnrun` returned a review; every
1719
+ `dimensionsUnrun` entry is `{dimension, reason}` naming a specialist that
1720
+ returned nothing, its `reason` the crash.
1723
1721
  The review already re-dispatched each crashed specialist once, so what
1724
1722
  `dimensionsUnrun` still names is residue: the fix-applier names it unrun in its
1725
1723
  report and re-runs nothing — an absence of findings is not coverage.
1726
1724
 
1725
+ **`suiteVerdict` is the review's one judgement on its shared test run:
1726
+ `{usable, reason}`, always present.** The review runs the test command once,
1727
+ and every specialist reads that run instead of running its own. `{usable: true,
1728
+ reason: null}` says the run executed real tests and its exit status is
1729
+ explained by its counts; usable is not green, so a usable run can have failing
1730
+ tests. `usable: false` carries `reason`, including a run that never started
1731
+ (install skipped, no suite slot). The verdict says nothing about whether a
1732
+ dimension reviewed: a specialist that returned reviewed, its findings verified
1733
+ as usual, whatever the suite verdict. With `usable: false` the
1734
+ fix-applier names `shared test run not usable: <reason>` in its final verdict
1735
+ and does nothing more — its own `<testCmd>` run and CI cover the suite.
1736
+
1727
1737
  The rule this replaces — treat any dimension with nothing in
1728
1738
  `survived`/`refuted`/`unverified` as unrun — was the workaround for having no
1729
1739
  such field, and it over-refuses in the direction that costs work: a dimension
@@ -1792,9 +1802,10 @@ seconds, 1800 by default, then exit **75** with `suite-slot: no slot after <n>s`
1792
1802
  on stderr, having launched nothing. On getting a slot the wrapper prints
1793
1803
  `SUITE_SLOT_WAIT_MS=<n>` — the wait in milliseconds — to stderr, which the
1794
1804
  review's test-run block captures into its log and the test-run agent reports as
1795
- `slotWaitMs`. **Exit 75 is not a test result.** In a review it leaves every
1796
- dimension unrun, for the reason "no suite slot within <n> s — the host is at its
1797
- suite bound; not a test failure", with no retry inside the run: re-review it
1805
+ `slotWaitMs`. **Exit 75 is not a test result.** In a review it makes
1806
+ `suiteVerdict` not usable, `suiteVerdict.reason` beginning "no suite slot within
1807
+ <n> s — the host is at its suite bound; not a test failure", with no retry
1808
+ inside the run: read that `reason` off the digest and re-review it
1798
1809
  under the next runner letter (`review-pr-<pr#>-b` after a first runner), as for
1799
1810
  a runner that reported `failed`. The implementer, fix-applier and finisher briefs
1800
1811
  carry the one-line rule: *Exit 75 from the suite is not a test result; run the
@@ -41,7 +41,7 @@ review already ran — and steps 4 and 6: the controller owns the CI wait and
41
41
  dispatches the finisher.
42
42
 
43
43
  **The review's result is `<scratch>/pr<pr>/<run>/review.json`** — the digest first
44
- (`jq '{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts}'`),
44
+ (`jq '{pr, head, resume, testEnvironment, suiteVerdict, dimensionsRun, dimensionsUnrun, cwdAudit, counts}'`),
45
45
  then every finding (`jq '.survived'`, `jq '.unverified'`, `jq '.refuted'`).
46
46
  Read it through `review-and-fix.md`'s **The review result file**, which holds
47
47
  the rest of the recipes and every ruling on the findings: both
@@ -265,6 +265,13 @@ as "no live token" — rotate's own verdict — where without it the script
265
265
  refuses. A ledger rotate did write is read as usual. Record nothing on the
266
266
  ledger and act on nothing in its receipt.
267
267
 
268
+ Straight after that, the same step reclaims the other keys under the scratch parent:
269
+ `~/.fleet/bin/fleet-run reclaim-scratch.mjs --siblings --root <scratch> --apply`. A key
270
+ goes whole when nothing under it has changed in 7 days, and stays whole otherwise. The
271
+ call runs whatever the root sweep answered, and records nothing on the ledger. A non-zero
272
+ exit, and each path its receipt's `failed[]` names, is listed in the step's output; neither
273
+ stops the run.
274
+
268
275
  ## Derive the Recipe
269
276
 
270
277
  **Derive the Recipe — the Recipe derivation step — before the fold-in below