@feigi/fleet-ctl 3.156.2 → 3.158.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agents/fleet-fix-applier.agent.md +33 -6
- package/agents/fleet-review-runner.agent.md +3 -5
- package/agents/fleet-review-tests.agent.md +4 -1
- package/commands/review-and-fix.md +6 -4
- package/package.json +1 -1
- package/scripts/evict.mjs +23 -7
- package/scripts/fleet-event.mjs +23 -45
- package/scripts/fleet-state.mjs +14 -11
- package/scripts/fleet-tick.mjs +71 -26
- package/scripts/ledger-grammar.mjs +25 -6
- package/scripts/ledger.mjs +114 -19
- package/scripts/measure-copy.mjs +242 -0
- package/scripts/reclaim-scratch.mjs +149 -29
- package/scripts/review-core.mjs +31 -20
- package/scripts/review-eval.mjs +11 -17
- package/skills/run-team/SKILL.md +5 -5
- package/skills/run-team/references/finisher.md +2 -2
- package/skills/run-team/references/isolation.md +16 -0
- package/skills/run-team/references/ledger.md +1 -1
- package/skills/run-team/references/member-lifecycle.md +2 -2
- package/skills/run-team/references/reviewers.md +12 -0
|
@@ -0,0 +1,242 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// A measuring copy as one command: the tree an agent mutates, runs a suite in
|
|
3
|
+
// or reproduces a failure in, built at a pinned commit with the Install step
|
|
4
|
+
// already run in it. Every copy a fix-applier, a fix-applier's refuter, a
|
|
5
|
+
// review specialist or a review refuter measures in is built by this script,
|
|
6
|
+
// so each is a repository of its own (the root sweep evicts only a directory
|
|
7
|
+
// that holds a `.git` of its own) and each has what the Install step leaves.
|
|
8
|
+
//
|
|
9
|
+
// measure-copy.mjs --worktree <abs> --pin <sha> --dest <abs> --pr <N> --scratch <abs>
|
|
10
|
+
//
|
|
11
|
+
// `--worktree` is the PR's worktree: git runs there, and every worktree of that
|
|
12
|
+
// repository shares its refs and Recipe cache, which a copy of tracked files
|
|
13
|
+
// holds none of — so the Install step is read from it, never from the copy or
|
|
14
|
+
// from a review snapshot. `--pin` is the full 40-hex commit to copy. `--dest`
|
|
15
|
+
// is the copy: a new directory inside a `fix-*` round root or a `run-*` run
|
|
16
|
+
// root that sits directly under `<scratch>/pr<N>/`. `--scratch` is the absolute
|
|
17
|
+
// scratch root the dispatch prompt names. Reached through the Resolver:
|
|
18
|
+
// `~/.omp/plugins/node_modules/@feigi/fleet-ctl/scripts/fleet-run measure-copy.mjs ...`.
|
|
19
|
+
//
|
|
20
|
+
// In order, each step only once every earlier one held:
|
|
21
|
+
//
|
|
22
|
+
// 1 The destination. It is written down from `--scratch` as
|
|
23
|
+
// `<scratch>/pr<N>/<root>/…/<dest>` (`<scratch>` as given or by realpath),
|
|
24
|
+
// `<root>` a directory named `fix-…` or `run-…` that is there, and it has no
|
|
25
|
+
// `.` or `..` segment; nothing from `<scratch>` down to the destination that
|
|
26
|
+
// exists may be a symlink; and nothing may exist at the destination itself,
|
|
27
|
+
// a dangling symlink or an empty directory included. Anything else builds
|
|
28
|
+
// nothing.
|
|
29
|
+
// 2 `git init` in it, then the pin's objects fetched from the worktree, its
|
|
30
|
+
// tree read into the index and checked out, `git add -A` and a commit.
|
|
31
|
+
// The commit's tree must be the pin's tree. Never `git archive`, which
|
|
32
|
+
// leaves out gitlinks and `export-ignore` paths and rewrites
|
|
33
|
+
// `export-subst` files.
|
|
34
|
+
// 3 The Install step, by evict.mjs's `installIn()`: read from the worktree
|
|
35
|
+
// with `derive-testcmd.sh <worktree> install --at origin/main`, run from the
|
|
36
|
+
// copy's root through `sh -c` with `CI` unset, no timeout, under no suite
|
|
37
|
+
// slot, a clean `git status --porcelain -uall` required before it. What the
|
|
38
|
+
// step leaves untracked afterwards is reported, never refused; a tracked
|
|
39
|
+
// file it changed is a refusal.
|
|
40
|
+
//
|
|
41
|
+
// Stdout is one JSON line, the receipt, whatever happened after the arguments
|
|
42
|
+
// parsed:
|
|
43
|
+
//
|
|
44
|
+
// {dest, pr, pin, head, tree, install, kind, exit, untracked, untrackedCount, message}
|
|
45
|
+
//
|
|
46
|
+
// dest the path the copy was, or would have been, built at
|
|
47
|
+
// head the copy's commit, null when never read
|
|
48
|
+
// tree the copy's commit tree, null when never read
|
|
49
|
+
// install the Install step's command, null when never read
|
|
50
|
+
// kind `built`, or what stopped it: `outside`, `occupied`,
|
|
51
|
+
// `unreadable`, `pin-unreadable`, `init-failed`,
|
|
52
|
+
// `fetch-failed`, `checkout-failed`, `tree-mismatch`, or one of
|
|
53
|
+
// evict.mjs's `installIn()` kinds: `derive`, `status-before`,
|
|
54
|
+
// `dirty-before`, `not-run`, `failed`, `dirtied`, `status-after`
|
|
55
|
+
// exit the Install step's own exit status, null when it produced none
|
|
56
|
+
// untracked the first 20 lines of `git status --porcelain -uall` in the
|
|
57
|
+
// copy after the Install step, null when it was never read
|
|
58
|
+
// untrackedCount how many lines there are in all
|
|
59
|
+
// message null when built, else the refusal in words
|
|
60
|
+
//
|
|
61
|
+
// Exit vocabulary:
|
|
62
|
+
//
|
|
63
|
+
// 0 built: the copy is the pin's tree and its Install step exited 0
|
|
64
|
+
// 1 no usable copy, nothing the Install step did: a destination outside a
|
|
65
|
+
// round root or run root, or one that exists; an init, a fetch, a
|
|
66
|
+
// checkout or a commit that failed; a tree that is not the pin's; a Recipe
|
|
67
|
+
// that could not be read; a status that could not be read before the
|
|
68
|
+
// Install step
|
|
69
|
+
// 2 a refusal on stderr with no receipt: a usage refusal, as in every other
|
|
70
|
+
// script, or a receipt that could not be written to stdout
|
|
71
|
+
// 3 the Install step did not run: exit 126 or 127, or it could not be started
|
|
72
|
+
// 4 the Install step failed: another non-zero exit, or a signal, or it
|
|
73
|
+
// changed a tracked file
|
|
74
|
+
// 5 the Install step ran and the copy's state could not be read after it
|
|
75
|
+
//
|
|
76
|
+
// Any non-zero exit leaves the copy where it stands: this script removes
|
|
77
|
+
// nothing.
|
|
78
|
+
|
|
79
|
+
import { spawnSync } from "node:child_process";
|
|
80
|
+
import { lstatSync, mkdirSync, realpathSync } from "node:fs";
|
|
81
|
+
import { isAbsolute, join, resolve, sep } from "node:path";
|
|
82
|
+
import { defineFlags, isDigits, makeDie, writeAll } from "./arg.mjs";
|
|
83
|
+
import { how } from "./evict-mark.mjs";
|
|
84
|
+
import { installIn } from "./evict.mjs";
|
|
85
|
+
import { gitEnv } from "./git-env.mjs";
|
|
86
|
+
import { isCLI } from "./is-cli.mjs";
|
|
87
|
+
|
|
88
|
+
const NAME = "measure-copy";
|
|
89
|
+
const die = makeDie(NAME);
|
|
90
|
+
const USAGE = "usage: measure-copy.mjs --worktree <abs> --pin <sha> --dest <abs> --pr <N> --scratch <abs>";
|
|
91
|
+
const SHA = /^[0-9a-f]{40}$/;
|
|
92
|
+
const ROOT = /^(?:fix|run)-./;
|
|
93
|
+
const GIT_TIMEOUT_MS = 5 * 60_000;
|
|
94
|
+
const MAX_BUFFER = 64 * 1024 * 1024;
|
|
95
|
+
|
|
96
|
+
// The installIn() refusal kinds an Install step that ran or tried to run
|
|
97
|
+
// answers for; every other refusal is exit 1.
|
|
98
|
+
const EXIT = {
|
|
99
|
+
"not-run": 3,
|
|
100
|
+
failed: 4,
|
|
101
|
+
dirtied: 4,
|
|
102
|
+
"status-after": 5,
|
|
103
|
+
};
|
|
104
|
+
|
|
105
|
+
const git = (dir, args) =>
|
|
106
|
+
spawnSync("git", ["-C", dir, ...args], { encoding: "utf8", maxBuffer: MAX_BUFFER, env: gitEnv(), timeout: GIT_TIMEOUT_MS });
|
|
107
|
+
|
|
108
|
+
/**
|
|
109
|
+
* Why `dest` is not a place this script builds, as `{kind, message}`, or null
|
|
110
|
+
* when it is. `scratch` and `pr` are the arguments; no path is created or
|
|
111
|
+
* changed.
|
|
112
|
+
*/
|
|
113
|
+
export function destRefusal(dest, scratch, pr) {
|
|
114
|
+
const outside = (message) => ({ kind: "outside", message });
|
|
115
|
+
if (dest.split(sep).some((s) => s === "." || s === "..")) return outside(`${dest} has a '.' or '..' segment`);
|
|
116
|
+
const bases = [resolve(scratch)];
|
|
117
|
+
try {
|
|
118
|
+
bases.push(realpathSync(scratch));
|
|
119
|
+
} catch (e) {
|
|
120
|
+
return { kind: "unreadable", message: `cannot resolve --scratch ${scratch} (${e.code ?? e.message})` };
|
|
121
|
+
}
|
|
122
|
+
const base = bases.find((b) => dest.startsWith(join(b, `pr${pr}`) + sep));
|
|
123
|
+
if (base === undefined) return outside(`${dest} is not inside <scratch>/pr${pr}/ (${scratch})`);
|
|
124
|
+
const parts = dest.slice(join(base, `pr${pr}`).length + 1).split(sep);
|
|
125
|
+
if (parts.some((s) => s === "")) return outside(`${dest} has an empty segment`);
|
|
126
|
+
if (parts.length < 2 || !ROOT.test(parts[0])) {
|
|
127
|
+
return outside(`${dest} is not inside a fix-* round root or a run-* run root directly under ${join(base, `pr${pr}`)}`);
|
|
128
|
+
}
|
|
129
|
+
let at = base;
|
|
130
|
+
for (const [i, part] of [`pr${pr}`, ...parts].entries()) {
|
|
131
|
+
at = join(at, part);
|
|
132
|
+
let st;
|
|
133
|
+
try {
|
|
134
|
+
st = lstatSync(at);
|
|
135
|
+
} catch (e) {
|
|
136
|
+
if (e.code !== "ENOENT") return { kind: "unreadable", message: `cannot look at ${at} (${e.code ?? e.message})` };
|
|
137
|
+
// The round root or run root is made before any copy: a missing one is
|
|
138
|
+
// a typo, not a place to create. Below it, nothing exists to refuse.
|
|
139
|
+
return i <= 1 ? outside(`${at} does not exist — make the round root or run root before the copy`) : null;
|
|
140
|
+
}
|
|
141
|
+
// Anything at the destination, a symlink included, is an occupied path.
|
|
142
|
+
if (at === dest) return { kind: "occupied", message: `${dest} already exists — a copy built over it would not be the pin's` };
|
|
143
|
+
if (st.isSymbolicLink()) return outside(`${at} is a symlink`);
|
|
144
|
+
if (!st.isDirectory()) return outside(`${at} is not a directory`);
|
|
145
|
+
}
|
|
146
|
+
return null;
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
function main() {
|
|
150
|
+
const { arg, sweep, stray } = defineFlags(die, {
|
|
151
|
+
flags: { worktree: "value", pin: "value", dest: "value", pr: "value", scratch: "value" },
|
|
152
|
+
});
|
|
153
|
+
// The sweep runs before the required-flag check so a misspelled flag is
|
|
154
|
+
// named as that, not buried under a missing-flag usage line.
|
|
155
|
+
sweep();
|
|
156
|
+
stray();
|
|
157
|
+
const worktree = arg("worktree");
|
|
158
|
+
const pinArg = arg("pin");
|
|
159
|
+
const destArg = arg("dest");
|
|
160
|
+
const pr = arg("pr");
|
|
161
|
+
const scratchArg = arg("scratch");
|
|
162
|
+
if ([worktree, pinArg, destArg, pr, scratchArg].includes(null)) die(USAGE);
|
|
163
|
+
if (!isAbsolute(worktree)) die(`--worktree needs an absolute path, got ${worktree}`);
|
|
164
|
+
if (!isAbsolute(destArg)) die(`--dest needs an absolute path, got ${destArg}`);
|
|
165
|
+
if (!isAbsolute(scratchArg)) die(`--scratch needs an absolute path, got ${scratchArg}`);
|
|
166
|
+
const pin = pinArg.toLowerCase();
|
|
167
|
+
if (!SHA.test(pin)) die(`--pin needs a full 40-character commit SHA, got ${pinArg}`);
|
|
168
|
+
if (!isDigits(pr) || Number(pr) < 1) die(`--pr needs a PR number, got ${pr}`);
|
|
169
|
+
|
|
170
|
+
// Only a trailing slash is forgiven; every other oddity is a refusal below.
|
|
171
|
+
const dest = destArg.length > 1 ? destArg.replace(/\/+$/, "") : destArg;
|
|
172
|
+
const receipt = { dest, pr: Number(pr), pin, head: null, tree: null, install: null, kind: null, exit: null, untracked: null, untrackedCount: null, message: null };
|
|
173
|
+
const finish = (code, kind, message = null) => {
|
|
174
|
+
receipt.kind = kind;
|
|
175
|
+
receipt.message = message;
|
|
176
|
+
// A receipt lost on its way to stdout must not leave its exit standing: an
|
|
177
|
+
// exit 0 with no receipt names no copy to work in, so a lost write is the
|
|
178
|
+
// no-receipt exit.
|
|
179
|
+
if (!writeAll(1, `${JSON.stringify(receipt)}\n`)) die("the receipt could not be written to stdout");
|
|
180
|
+
process.exitCode = code;
|
|
181
|
+
};
|
|
182
|
+
|
|
183
|
+
// 1: the destination.
|
|
184
|
+
const refusal = destRefusal(dest, scratchArg, pr);
|
|
185
|
+
if (refusal !== null) return finish(1, refusal.kind, refusal.message);
|
|
186
|
+
|
|
187
|
+
// 2: the copy, at the pin, as its own repository. The pin's tree is read
|
|
188
|
+
// first: a pin the worktree does not hold builds nothing, and the comparison
|
|
189
|
+
// at the end is against what git says the pin is, never against the copy.
|
|
190
|
+
const want = git(worktree, ["rev-parse", "--verify", "--end-of-options", `${pin}^{tree}`]);
|
|
191
|
+
const wanted = String(want.stdout ?? "").trim();
|
|
192
|
+
if (want.error || want.status !== 0 || wanted === "") return finish(1, "pin-unreadable", `could not read the tree of ${pin} in ${worktree} (${how(want)})`);
|
|
193
|
+
try {
|
|
194
|
+
mkdirSync(dest, { recursive: true });
|
|
195
|
+
} catch (e) {
|
|
196
|
+
return finish(1, "unreadable", `cannot make ${dest} (${e.code ?? e.message})`);
|
|
197
|
+
}
|
|
198
|
+
// The pin's objects are fetched into the copy and its tree checked out from
|
|
199
|
+
// them, never archived: `git archive` leaves out a submodule's gitlink and
|
|
200
|
+
// every `export-ignore` path and rewrites an `export-subst` file, so its copy
|
|
201
|
+
// of such a pin is never the pin's tree. A gitlink is checked out as an empty
|
|
202
|
+
// directory and stays in the index as the pin holds it. The add reads the
|
|
203
|
+
// checked-out files back, so one the checkout does not carry back unchanged
|
|
204
|
+
// is a tree mismatch below. Each step is its own child, so the one that
|
|
205
|
+
// failed names the refusal with git's own words.
|
|
206
|
+
const made = [
|
|
207
|
+
["init-failed", ["init", "-q"]],
|
|
208
|
+
["fetch-failed", ["-c", "protocol.version=2", "fetch", "-q", "--no-tags", "--depth=1", worktree, pin]],
|
|
209
|
+
["checkout-failed", ["read-tree", pin]],
|
|
210
|
+
["checkout-failed", ["checkout-index", "-a", "-f"]],
|
|
211
|
+
["init-failed", ["add", "-A"]],
|
|
212
|
+
["init-failed", ["-c", "user.name=fleet", "-c", "user.email=fleet@invalid", "-c", "commit.gpgsign=false", "commit", "-q", "--no-verify", "--allow-empty", "-m", `measuring copy of ${pin}`]],
|
|
213
|
+
];
|
|
214
|
+
for (const [kind, args] of made) {
|
|
215
|
+
const step = git(dest, args);
|
|
216
|
+
if (step.error || step.status !== 0) return finish(1, kind, `git ${args.filter((a) => a !== "-c" && !a.includes("=")).join(" ")} failed in ${dest} (${how(step)})`);
|
|
217
|
+
}
|
|
218
|
+
const head = git(dest, ["rev-parse", "HEAD", "HEAD^{tree}"]);
|
|
219
|
+
const [at, tree] = String(head.stdout ?? "").trim().split("\n");
|
|
220
|
+
if (head.error || head.status !== 0 || !at || !tree) return finish(1, "init-failed", `could not read HEAD in ${dest} (${how(head)})`);
|
|
221
|
+
receipt.head = at;
|
|
222
|
+
receipt.tree = tree;
|
|
223
|
+
if (tree !== wanted) return finish(1, "tree-mismatch", `the copy's tree ${tree} is not the tree ${wanted} of ${pin}`);
|
|
224
|
+
|
|
225
|
+
// 3: the Install step, read from the worktree, run in the copy. No retry in a
|
|
226
|
+
// dirty-before refusal: the caller runs none of this script's steps by hand
|
|
227
|
+
// to get past one.
|
|
228
|
+
const done = installIn(dest, true, { verb: "install", retry: false, source: worktree, untracked: true });
|
|
229
|
+
if (done.refusal) {
|
|
230
|
+
const { kind, message, install, exit } = done.refusal;
|
|
231
|
+
receipt.install = install;
|
|
232
|
+
receipt.exit = exit;
|
|
233
|
+
return finish(EXIT[kind] ?? 1, kind, message);
|
|
234
|
+
}
|
|
235
|
+
receipt.install = done.install;
|
|
236
|
+
receipt.exit = 0;
|
|
237
|
+
receipt.untracked = done.untracked.first;
|
|
238
|
+
receipt.untrackedCount = done.untracked.count;
|
|
239
|
+
finish(0, "built");
|
|
240
|
+
}
|
|
241
|
+
|
|
242
|
+
if (isCLI(import.meta.url)) main();
|
|
@@ -35,9 +35,10 @@
|
|
|
35
35
|
// finisher-pr-<M>[-x]. A live merge-bot-<n> token holds every one
|
|
36
36
|
// of them: a bot's token names no PR, and GitHub reads MERGED
|
|
37
37
|
// while the bot is still writing under pr<M>/merge-bot-<n>/. An
|
|
38
|
-
// OPEN PR's pr<M>/ is kept
|
|
39
|
-
//
|
|
40
|
-
// review evidence — a `<run>/review.json` under
|
|
38
|
+
// OPEN PR's pr<M>/ is kept, but for its run-* roots, which follow
|
|
39
|
+
// the --pr rule below, and the trees named below. A MERGED PR's
|
|
40
|
+
// entry that holds review evidence — a `<run>/review.json` under
|
|
41
|
+
// pr<M>/, or a
|
|
41
42
|
// Charges file naming its run — is also kept until origin/main
|
|
42
43
|
// holds every file of it where run-records.mjs's reviewTargets()
|
|
43
44
|
// puts it, under docs/metrics/reviews/<M>/: the Close-out PR
|
|
@@ -46,6 +47,21 @@
|
|
|
46
47
|
// or evidence that cannot be read, keeps the entry with that
|
|
47
48
|
// reason. A CLOSED PR gets no Ruling, and its entries keep the
|
|
48
49
|
// rule above.
|
|
50
|
+
// EVERY pr<M>/ the sweep keeps — an OPEN PR's, a live holder's or
|
|
51
|
+
// review evidence's, and one a live merge bot holds — also has
|
|
52
|
+
// the trees in it evicted (evict-tree.mjs's rule), by the kind of
|
|
53
|
+
// member that made them, once none of that kind is live on the
|
|
54
|
+
// ledger: the trees under its fix-* round roots, and under a
|
|
55
|
+
// mutate/ left from before, when no fix-pr-<M>[-x] is live; the
|
|
56
|
+
// trees under its run-* roots, when no review launch of PR #M is
|
|
57
|
+
// open; every finish-* tree, when no finisher-pr-<M>[-x] is live.
|
|
58
|
+
// The ledger is read for that after the directory's roots are
|
|
59
|
+
// listed, just before they are evicted, and a merge bot's hold
|
|
60
|
+
// keeps the directory but never its trees. A directory kept for
|
|
61
|
+
// a reason that names no state of the PR — GitHub did not answer,
|
|
62
|
+
// or the number names an issue — is not evicted. What is left in
|
|
63
|
+
// it — verdict files, prompts, receipts, logs, review.json, the
|
|
64
|
+
// mutants — stays.
|
|
49
65
|
// impl-<N>/
|
|
50
66
|
// removed when issue #N is CLOSED, or OPEN with no live
|
|
51
67
|
// impl-<N>[-x] token, no claim worktree standing, and idle past
|
|
@@ -120,13 +136,16 @@
|
|
|
120
136
|
// "kept":[{"path","reason"}],"failed":[{"path","error"}]}
|
|
121
137
|
// The sibling pass's kept entries also carry "idle": true when nothing under
|
|
122
138
|
// the key was found modified within the retention, false otherwise.
|
|
123
|
-
// `--pr` mode
|
|
124
|
-
// each tree that held ignored paths, `ignored` naming them
|
|
139
|
+
// `--pr` mode and the root sweep add `"evicted":[{"path","reason","ignored"}]`
|
|
140
|
+
// after `removed`: each tree that held ignored paths, `ignored` naming them and
|
|
141
|
+
// `reason` naming the rule that evicted it, and
|
|
125
142
|
// `"skipped":[{"path","reason"}]` after `failed`: each directory in a kept run
|
|
126
|
-
// root that evict-tree.mjs could not evaluate —
|
|
127
|
-
// resolve to it, one it cannot look in for a
|
|
128
|
-
// EACCES or EPERM. A skipped directory is
|
|
129
|
-
// step's marker, which reads `failed`, does
|
|
143
|
+
// root, round root or finisher tree that evict-tree.mjs could not evaluate —
|
|
144
|
+
// one with a `.git` git does not resolve to it, one it cannot look in for a
|
|
145
|
+
// `.git`, or one it cannot list for EACCES or EPERM. A skipped directory is
|
|
146
|
+
// never in `failed`, so the snapshot step's marker, which reads `failed`, does
|
|
147
|
+
// not fire for it. A kept pr<N>/ whose roots cannot be listed, or whose ledger
|
|
148
|
+
// cannot be read just before its eviction, is in `failed`.
|
|
130
149
|
// `ledger` is the file read, or null when none was (`--pr` mode, or
|
|
131
150
|
// `--after-rotate` with no file) — so "never read" stays distinct from "read
|
|
132
151
|
// and found empty". A dry run prints the same receipt with "applied":false,
|
|
@@ -149,7 +168,7 @@ import { fleetFile, FleetDirUnresolvable, SCRATCH_KEY_SHAPE, scratchRoot } from
|
|
|
149
168
|
import { gitEnv } from "./git-env.mjs";
|
|
150
169
|
import { ghAliasedRead, how } from "./gh-aliased-read.mjs";
|
|
151
170
|
import { isCLI } from "./is-cli.mjs";
|
|
152
|
-
import { liveOnLedger, parseMember, REVIEWER } from "./ledger-grammar.mjs";
|
|
171
|
+
import { liveOnLedger, parseMember, REVIEW, REVIEWER, rowNums } from "./ledger-grammar.mjs";
|
|
153
172
|
import { REVIEWS_DIR, reviewTargets } from "./run-records.mjs";
|
|
154
173
|
|
|
155
174
|
const NAME = "reclaim-scratch";
|
|
@@ -172,6 +191,8 @@ const REPORTS_DIR = "reports";
|
|
|
172
191
|
const REPORT_PR_ENTRY = /^(?:fix-pr|finisher-pr|review-pr)-([1-9][0-9]*)(?:-[a-z])?(?:\.check)?\.json$/;
|
|
173
192
|
const REPORT_IMPL_ENTRY = /^impl-([1-9][0-9]*)(?:-[a-z])?(?:\.check)?\.json$/;
|
|
174
193
|
const RUN_ROOT = /^run-/;
|
|
194
|
+
const FIX_ROOT = /^fix-/;
|
|
195
|
+
const FINISH_ROOT = /^finish-/;
|
|
175
196
|
|
|
176
197
|
/** The path of something under `path` (itself included) modified at or after
|
|
177
198
|
* `cutoff`, or null when everything is older. Symlinks are not followed. A
|
|
@@ -227,10 +248,18 @@ export function readLedger(file, die, undone) {
|
|
|
227
248
|
return { rows: d.rows, dispatched: d.dispatched };
|
|
228
249
|
}
|
|
229
250
|
|
|
230
|
-
// The live tokens the ledger holds,
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
251
|
+
// The live tokens the ledger holds, as `{name, launchOf}`: `launchOf` the PR
|
|
252
|
+
// whose rows carry the token when it is an open review launch — a runner's, or
|
|
253
|
+
// a workflow's, whose name is its runId — else null. `fail` and `undone` are
|
|
254
|
+
// readLedger()'s: a read made after removals have started throws, so the one
|
|
255
|
+
// directory's eviction is a failure on the receipt and the sweep goes on.
|
|
256
|
+
function liveTokens(file, fail = die, undone = "reclaimed") {
|
|
257
|
+
const { rows, dispatched } = readLedger(file, fail, undone);
|
|
258
|
+
const launchPr = new Map();
|
|
259
|
+
for (const text of rows) {
|
|
260
|
+
for (const word of text.split(/\s+/)) if (REVIEW.test(word) && !launchPr.has(word)) launchPr.set(word, rowNums(text).pr);
|
|
261
|
+
}
|
|
262
|
+
return liveOnLedger(rows, dispatched).tokens.map((t) => ({ name: t.name, launchOf: launchPr.get(t.token) ?? null }));
|
|
234
263
|
}
|
|
235
264
|
|
|
236
265
|
// The checkout the fleet dir sits in, and its ledger file — `file` null when it
|
|
@@ -345,11 +374,72 @@ function githubStates(numbers) {
|
|
|
345
374
|
return out;
|
|
346
375
|
}
|
|
347
376
|
|
|
348
|
-
// The
|
|
377
|
+
// The roots under a kept `pr<N>/` whose trees the sweep evicts when no member
|
|
378
|
+
// of the kind that made them is live, each as `{path, kind}`: a fix-applier's
|
|
379
|
+
// round root (`fix-*`) or a `mutate/` left from before, a review's run root
|
|
380
|
+
// (`run-*`) and a finisher's tree (`finish-*`). `skip` is the paths already
|
|
381
|
+
// being removed whole. An absent directory holds none; one that cannot be
|
|
382
|
+
// listed throws.
|
|
383
|
+
function treeRoots(pdir, skip) {
|
|
384
|
+
let entries;
|
|
385
|
+
try {
|
|
386
|
+
entries = readdirSync(pdir, { withFileTypes: true });
|
|
387
|
+
} catch (e) {
|
|
388
|
+
if (e.code === "ENOENT" || e.code === "ENOTDIR") return [];
|
|
389
|
+
throw e;
|
|
390
|
+
}
|
|
391
|
+
const roots = [];
|
|
392
|
+
for (const e of entries.sort((a, b) => (a.name < b.name ? -1 : 1))) {
|
|
393
|
+
if (!e.isDirectory()) continue;
|
|
394
|
+
const path = join(pdir, e.name);
|
|
395
|
+
if (skip.has(path)) continue;
|
|
396
|
+
const kind = e.name === "mutate" || FIX_ROOT.test(e.name) ? "fix" : RUN_ROOT.test(e.name) ? "run" : FINISH_ROOT.test(e.name) ? "finish" : null;
|
|
397
|
+
if (kind !== null) roots.push({ path, kind });
|
|
398
|
+
}
|
|
399
|
+
return roots;
|
|
400
|
+
}
|
|
401
|
+
|
|
402
|
+
// The live holder of the kind `kind` names on PR `n`, by name, or undefined:
|
|
403
|
+
// a `fix-pr-<n>[-x]`, a `finisher-pr-<n>[-x]`, or an open review launch of PR
|
|
404
|
+
// #n — `review-pr-<n>[-x]`, or a workflow launch on PR #n's rows, named by its
|
|
405
|
+
// runId. `live` is liveTokens()'s list.
|
|
406
|
+
function kindHolder(live, kind, n) {
|
|
407
|
+
return live.find(({ name, launchOf }) => {
|
|
408
|
+
if (kind === "run") {
|
|
409
|
+
const r = REVIEWER.exec(name);
|
|
410
|
+
return launchOf === n || (r !== null && Number(r[1]) === n);
|
|
411
|
+
}
|
|
412
|
+
const m = parseMember(name);
|
|
413
|
+
return m !== null && m.family === (kind === "fix" ? "fix-pr" : "finisher-pr") && m.number === n;
|
|
414
|
+
})?.name;
|
|
415
|
+
}
|
|
416
|
+
|
|
417
|
+
const EVICT_RULE = {
|
|
418
|
+
fix: (n, name) => (name === "mutate" ? `a tree in pr${n}/mutate/, left from before, with no live fix-pr-${n}[-x] on the ledger` : `a tree in a fix-applier round root, with no live fix-pr-${n}[-x] on the ledger`),
|
|
419
|
+
run: (n) => `a tree in a review run root, with no open review launch of PR #${n} on the ledger`,
|
|
420
|
+
finish: (n) => `a finisher tree, with no live finisher-pr-${n}[-x] on the ledger`,
|
|
421
|
+
};
|
|
422
|
+
|
|
423
|
+
// The root sweep's verdicts: `{remove: [{path, reason}], kept: [{path, reason}],
|
|
424
|
+
// evict: [{n, path, roots}]}`. `evict` is each kept `pr<N>/` directory whose
|
|
425
|
+
// trees the sweep evicts, `roots` the roots it listed in it, or `{n, path,
|
|
426
|
+
// error}` when it could not list them: whether a root's kind of member is live
|
|
427
|
+
// is read later, just before the eviction.
|
|
349
428
|
function judgeRoot(root, names, live, now, listing, checkout) {
|
|
350
429
|
const cutoff = now - RETENTION_MS;
|
|
351
430
|
const remove = [];
|
|
352
431
|
const kept = [];
|
|
432
|
+
const evict = [];
|
|
433
|
+
// A kept pr<N>/ keeps its trees only while a member of the kind that made
|
|
434
|
+
// them is live; a live merge bot's hold on the directory is no such member.
|
|
435
|
+
const evictIn = (path, n) => {
|
|
436
|
+
const skip = new Set(remove.map((r) => r.path));
|
|
437
|
+
try {
|
|
438
|
+
evict.push({ n, path, roots: treeRoots(path, skip) });
|
|
439
|
+
} catch (e) {
|
|
440
|
+
evict.push({ n, path, error: e.message });
|
|
441
|
+
}
|
|
442
|
+
};
|
|
353
443
|
const liveMembers = live.map(parseMember).filter((m) => m !== null);
|
|
354
444
|
const bot = liveMembers.find((m) => m.family === "merge-bot");
|
|
355
445
|
const prHolder = (n) => live.find((name) => {
|
|
@@ -452,18 +542,21 @@ function judgeRoot(root, names, live, now, listing, checkout) {
|
|
|
452
542
|
continue;
|
|
453
543
|
}
|
|
454
544
|
if (kind === "pr") {
|
|
545
|
+
const isDir = name === `pr${n}`;
|
|
455
546
|
if (bot !== undefined) {
|
|
456
547
|
kept.push({ path, reason: `live ${bot.name} holds every pr<N>` });
|
|
548
|
+
if (isDir) evictIn(path, n);
|
|
457
549
|
continue;
|
|
458
550
|
}
|
|
459
551
|
if (s.state === "OPEN") {
|
|
460
552
|
kept.push({ path, reason: `PR #${n} is OPEN` });
|
|
461
|
-
if (
|
|
553
|
+
if (isDir) {
|
|
462
554
|
try {
|
|
463
555
|
remove.push(...staleRunRoots(path, cutoff, kept));
|
|
464
556
|
} catch (e) {
|
|
465
557
|
kept.push({ path, reason: `its run-* roots cannot be listed: ${e.message}` });
|
|
466
558
|
}
|
|
559
|
+
evictIn(path, n);
|
|
467
560
|
}
|
|
468
561
|
continue;
|
|
469
562
|
}
|
|
@@ -474,11 +567,14 @@ function judgeRoot(root, names, live, now, listing, checkout) {
|
|
|
474
567
|
const holder = prHolder(n);
|
|
475
568
|
if (holder !== undefined) {
|
|
476
569
|
kept.push({ path, reason: `PR #${n} is ${s.state}, but live ${holder} names it` });
|
|
570
|
+
if (isDir) evictIn(path, n);
|
|
477
571
|
continue;
|
|
478
572
|
}
|
|
479
573
|
const held = s.state === "MERGED" ? evidenceHold(path, name, n) : null;
|
|
480
|
-
if (held !== null)
|
|
481
|
-
|
|
574
|
+
if (held !== null) {
|
|
575
|
+
kept.push({ path, reason: held });
|
|
576
|
+
if (isDir) evictIn(path, n);
|
|
577
|
+
} else remove.push({ path, reason: `PR #${n} is ${s.state}` });
|
|
482
578
|
continue;
|
|
483
579
|
}
|
|
484
580
|
if (s.state === "CLOSED") {
|
|
@@ -502,7 +598,7 @@ function judgeRoot(root, names, live, now, listing, checkout) {
|
|
|
502
598
|
if (freshUnder(path, cutoff) === null) remove.push({ path, reason: `issue #${n} is OPEN, no live impl-${n} token, idle past 7 days` });
|
|
503
599
|
else kept.push({ path, reason: `issue #${n} is OPEN; modified within 7 days` });
|
|
504
600
|
}
|
|
505
|
-
return { remove, kept };
|
|
601
|
+
return { remove, kept, evict };
|
|
506
602
|
}
|
|
507
603
|
|
|
508
604
|
// The run-* directories directly under `parent` idle past the retention, as
|
|
@@ -691,12 +787,20 @@ function main() {
|
|
|
691
787
|
|
|
692
788
|
const now = Date.now();
|
|
693
789
|
let ledger = null;
|
|
694
|
-
//
|
|
790
|
+
// The evictions and skipped directories of `--pr` mode and the root sweep.
|
|
695
791
|
const evicted = [];
|
|
696
792
|
const skipped = [];
|
|
697
793
|
let remove;
|
|
698
794
|
let kept = [];
|
|
795
|
+
let evict = [];
|
|
699
796
|
const failed = [];
|
|
797
|
+
// Evicts the trees under `path`, each named on the receipt with `reason`.
|
|
798
|
+
const evictRoot = (path, reason) => {
|
|
799
|
+
const done = evictTrees(path, { apply });
|
|
800
|
+
for (const t of done.evicted) evicted.push({ path: t.path, reason, ignored: t.ignored });
|
|
801
|
+
failed.push(...done.failed);
|
|
802
|
+
skipped.push(...done.skipped);
|
|
803
|
+
};
|
|
700
804
|
if (pr !== null) {
|
|
701
805
|
const parent = join(root, `pr${Number(pr)}`);
|
|
702
806
|
try {
|
|
@@ -706,12 +810,7 @@ function main() {
|
|
|
706
810
|
failed.push({ path: parent, error: e.message });
|
|
707
811
|
}
|
|
708
812
|
// `--pr` mode also evicts the trees in every run root it keeps.
|
|
709
|
-
for (const run of kept)
|
|
710
|
-
const done = evictTrees(run.path, { apply });
|
|
711
|
-
for (const t of done.evicted) evicted.push({ path: t.path, reason: "a tree in a kept review run root", ignored: t.ignored });
|
|
712
|
-
failed.push(...done.failed);
|
|
713
|
-
skipped.push(...done.skipped);
|
|
714
|
-
}
|
|
813
|
+
for (const run of kept) evictRoot(run.path, "a tree in a kept review run root");
|
|
715
814
|
} else if (siblings) {
|
|
716
815
|
if (apply) {
|
|
717
816
|
const wrote = writeCheckout(root);
|
|
@@ -730,7 +829,7 @@ function main() {
|
|
|
730
829
|
const found = ledgerFile(die);
|
|
731
830
|
ledger = found.file;
|
|
732
831
|
if (ledger === null && !afterRotate) die("no .fleet/ledger.md — without it no live member can hold its scratch, so nothing is reclaimed; pass --after-rotate only straight after `ledger.mjs rotate`");
|
|
733
|
-
const live = ledger === null ? [] :
|
|
832
|
+
const live = ledger === null ? [] : liveTokens(ledger).map((t) => t.name);
|
|
734
833
|
let names;
|
|
735
834
|
try {
|
|
736
835
|
names = readdirSync(root).sort();
|
|
@@ -738,7 +837,7 @@ function main() {
|
|
|
738
837
|
die(`could not list --root ${root}: ${e.message}`);
|
|
739
838
|
}
|
|
740
839
|
const listing = worktreeListing(found.checkout);
|
|
741
|
-
({ remove, kept } = judgeRoot(root, names, live, now, listing, found.checkout));
|
|
840
|
+
({ remove, kept, evict } = judgeRoot(root, names, live, now, listing, found.checkout));
|
|
742
841
|
}
|
|
743
842
|
|
|
744
843
|
const removed = [];
|
|
@@ -765,7 +864,28 @@ function main() {
|
|
|
765
864
|
if (error === null) removed.push(entry);
|
|
766
865
|
else failed.push({ path: entry.path, error });
|
|
767
866
|
}
|
|
768
|
-
|
|
867
|
+
// The trees of every kept pr<N>/, once the removals are done: a root is
|
|
868
|
+
// evicted only while no member of the kind that made it is live, and the
|
|
869
|
+
// ledger is read for that now, after the roots were listed, so a member
|
|
870
|
+
// dispatched in between holds its trees.
|
|
871
|
+
for (const dir of evict) {
|
|
872
|
+
if (dir.error !== undefined) {
|
|
873
|
+
failed.push({ path: dir.path, error: `its roots cannot be listed: ${dir.error}` });
|
|
874
|
+
continue;
|
|
875
|
+
}
|
|
876
|
+
if (dir.roots.length === 0) continue;
|
|
877
|
+
let liveNow;
|
|
878
|
+
try {
|
|
879
|
+
liveNow = ledger === null ? [] : liveTokens(ledger, (message) => { throw new Error(message); }, "evicted");
|
|
880
|
+
} catch (e) {
|
|
881
|
+
failed.push({ path: dir.path, error: e.message });
|
|
882
|
+
continue;
|
|
883
|
+
}
|
|
884
|
+
for (const r of dir.roots) {
|
|
885
|
+
if (kindHolder(liveNow, r.kind, dir.n) === undefined) evictRoot(r.path, EVICT_RULE[r.kind](dir.n, basename(r.path)));
|
|
886
|
+
}
|
|
887
|
+
}
|
|
888
|
+
const receipt = { applied: apply, ledger, removed, ...(siblings ? {} : { evicted }), kept, failed, ...(siblings ? {} : { skipped }) };
|
|
769
889
|
writeAll(1, `${JSON.stringify(receipt)}\n`);
|
|
770
890
|
}
|
|
771
891
|
|
package/scripts/review-core.mjs
CHANGED
|
@@ -743,24 +743,35 @@ function countsOf(run) {
|
|
|
743
743
|
}
|
|
744
744
|
|
|
745
745
|
// How a specialist or refuter makes "your own copy of the snapshot", in
|
|
746
|
-
// every prompt that sends one to work there
|
|
747
|
-
//
|
|
748
|
-
//
|
|
749
|
-
//
|
|
750
|
-
//
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
|
|
763
|
-
|
|
746
|
+
// every prompt that sends one to work there: by measure-copy.mjs, the one
|
|
747
|
+
// script that builds every measuring copy. It builds the copy from the PR
|
|
748
|
+
// worktree at the reviewed head, makes it a repository of its own and runs the
|
|
749
|
+
// Install step in it, so the copy holds what an install leaves — which one that
|
|
750
|
+
// shares the snapshot's installed output would not, since an install can record
|
|
751
|
+
// absolute paths back into the tree it ran in and a mutation made there would
|
|
752
|
+
// never run. `worktree` is the PR worktree, never the snapshot: the snapshot
|
|
753
|
+
// holds tracked files only, and the Install step is read from the repository's
|
|
754
|
+
// shared Recipe cache, which every worktree reads. `dir` is the agent's own
|
|
755
|
+
// directory of the run root; the copy is built under it. `scratch` and `pr`
|
|
756
|
+
// are runReview's own, the ones the run root was checked against, never read
|
|
757
|
+
// back off the run root the snapshot agent reported.
|
|
758
|
+
export function copyRule(snap, worktree, dir, scratch, pr) {
|
|
759
|
+
return `Make your own copy of the tree at the reviewed head with the one script that
|
|
760
|
+
builds every measuring copy, never by hand — it builds the copy from the PR
|
|
761
|
+
worktree at that commit, makes it a repository of its own and runs the Install
|
|
762
|
+
step from the copy's root:
|
|
763
|
+
~/.omp/plugins/node_modules/@feigi/fleet-ctl/scripts/fleet-run measure-copy.mjs --worktree ${worktree} --pin ${snap.head} --dest ${dir}/copy --pr ${pr} --scratch ${scratch}
|
|
764
|
+
For each further copy, give \`--dest\` another new directory under ${dir}/. The
|
|
765
|
+
Install step can take minutes: run the command as ONE blocking foreground
|
|
766
|
+
command with a command deadline of 3600 seconds, never backgrounded and never
|
|
767
|
+
polled. It prints one JSON line, the receipt, and exits 0 only when the copy is
|
|
768
|
+
the reviewed head's tree and its Install step exited 0; the receipt lists what
|
|
769
|
+
the step left untracked. Any other exit is no usable copy: measure nothing in
|
|
770
|
+
it, never make one by hand, and name the exit and the receipt's \`message\` in
|
|
771
|
+
what you report. \`--worktree\` is the one use of the PR worktree this prompt
|
|
772
|
+
allows, and the script's own: you never read or write it. Never \`cp -R\` or clone
|
|
773
|
+
the snapshot, and never symlink its ignored paths: it holds tracked files only,
|
|
774
|
+
and a copy sharing the snapshot's installed output runs the snapshot's source.`;
|
|
764
775
|
}
|
|
765
776
|
|
|
766
777
|
// The Tests paragraph of every specialist prompt: the shared run's
|
|
@@ -1464,7 +1475,7 @@ READ ONLY FROM THE SNAPSHOT: ${snap.path} (HEAD ${snap.head}) — plus the diff
|
|
|
1464
1475
|
file named below, if one is given.
|
|
1465
1476
|
Never read or write ${worktree} — other agents are using it.
|
|
1466
1477
|
Run any mutation or probe work inside your own copy of the snapshot.
|
|
1467
|
-
${copyRule(snap)}
|
|
1478
|
+
${copyRule(snap, worktree, `${snap.runRoot}/${d.key}`, scratch, pr)}
|
|
1468
1479
|
|
|
1469
1480
|
Your shell starts in NEITHER of those directories, and what it does start in is
|
|
1470
1481
|
a tree you must not write to: this dispatch carries no working directory of its
|
|
@@ -1563,7 +1574,7 @@ decide it and nothing else:
|
|
|
1563
1574
|
|
|
1564
1575
|
${KILL_RULES}
|
|
1565
1576
|
|
|
1566
|
-
${copyRule(snap)}
|
|
1577
|
+
${copyRule(snap, worktree, `${snap.runRoot}/verify-${d.key}/f${fi + 1}-l${i + 1}`, scratch, pr)}
|
|
1567
1578
|
|
|
1568
1579
|
${PATTERN_KILL_RULE}
|
|
1569
1580
|
|