@feigi/fleet-ctl 3.32.1 → 3.33.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agents/fleet-review-runner.agent.md +1 -1
- package/commands/review-and-fix.md +4 -4
- package/package.json +1 -1
- package/scripts/compute-board.mjs +4 -1
- package/scripts/dispositions-check.mjs +88 -44
- package/scripts/fleet-tick.mjs +41 -31
- package/scripts/ledger.mjs +6 -6
- package/scripts/review-eval.mjs +59 -11
- package/skills/run-team/SKILL.md +47 -33
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: fleet-review-runner
|
|
3
|
-
description: A `/skill:run-team` review runner on omp — dispatched by the controller as review-pr-<pr#> with pr, branch, worktree, testCmd and scratch (the scratch root); runs runReviewOnOmp to completion in its own eval cell, writes <scratch>/review
|
|
3
|
+
description: A `/skill:run-team` review runner on omp — dispatched by the controller as review-pr-<pr#> with pr, branch, worktree, testCmd and scratch (the scratch root); runs runReviewOnOmp to completion in its own eval cell, writes <scratch>/pr<pr>/<run>/review.json in the review's own run root, and reports the digest, the path and the reviewed= token naming that run. Never invoked directly.
|
|
4
4
|
model: "@smol:low"
|
|
5
5
|
spawns: fleet-review-snapshot, fleet-review-test-run, fleet-review-correctness, fleet-review-silent-failure, fleet-review-tests, fleet-review-comments, fleet-review-types, fleet-review-simplify, fleet-review-verifier
|
|
6
6
|
---
|
|
@@ -9,13 +9,13 @@ argument-hint: [pr-number]
|
|
|
9
9
|
|
|
10
10
|
See **Specialists** — those constraints are not optional.
|
|
11
11
|
2. Plan the actions. Split explicitly into **apply now** / **defer**. A finding is deferred, never dropped. **Every `suggestion` — the review's `severity`, the plugin's *Suggestions* bucket — arrives unchecked: the review budgets that band 0 refuters, so nothing has looked at it yet.** Checking it is now your job, not nobody's. So split it by SCOPE first. Out of the PR's ticket scope → defer and file. In scope → dispatch one refuter against it, biased to refuse, and hand it this instruction verbatim with one substitution: resolve `<scratch>/pr<N>/<finding>/` in it to an absolute path under your own run root — `mktemp -d` once per round for `<scratch>/pr<N>/fix-XXXXXXXX/`, never bare `pr<N>`, since finding ids restart at 1 each round too — and write that absolute path into the refuter's prompt in its place; a refuter never derives its own path: *Try to REFUTE this finding. Default to refuted=true if uncertain. Verify by RUNNING something — compile it, run the test, apply the mutation. Do not reason your way to agreement. Observe that run synchronously — run the command, wait for it, read its exit code. Never poll a log file for a completion marker: prefer ONE blocking run to a poll loop, and treat its return as permission to look, never as the answer. Reading a log the run has already finished writing is fine; waiting on one is not. If you match a test reporter's own output, accepting both `ℹ` and `#` is necessary but NOT sufficient — strip SGR escapes first as well. node's prefix moves with the node version and with whether stdout is a TTY, and color wraps the whole line so it begins with ESC and no prefix anchor matches at all, which returns empty at exit 0 — indistinguishable from a hung run and from a run of zero tests. For an uncolored baseline use `env -u FORCE_COLOR`; `FORCE_COLOR=` empty still enables color, so it is not a control. State your search scope AND what your pattern would have missed. A grep over one ref does not support a claim about history; a pattern built from the token a diff removed does not support a claim that the category is empty. A failure injection with no positive control has produced NO result, never a negative one. Before you read an injected fault — an env var, an argv word, a mutant — as having had no effect, prove the injection reached the child: one run whose output differs with it present versus absent, or the child echoing the injected value back. Uncontrolled, the cell is unrun — say so in your verdict instead of reporting a no-effect result. Build such an invocation as an array expanded braced and quoted — `cfg=(SETB=1 BADJ=1); env "${cfg[@]}" sh ./probe.sh` — or inline the assignments literally — `env SETB=1 BADJ=1 sh ./probe.sh`; NEVER from an unquoted scalar — `cfg="SETB=1 BADJ=1"; env $cfg sh ./probe.sh` — which under zsh passes ONE argument, sets a variable literally named `SETB` to `1 BADJ=1`, never sets `BADJ` at all, and still exits 0. `env $cfg[@]` is not the portable spelling either: measured, bash word-splits it into `SETB=1` and `BADJ=1[@]`, so the injection variable is set to a corrupted value, while zsh behaves exactly as with the bare `$cfg` — `BADJ` never set, exit 0. Everything you write — mutants, fixtures, scratch repos — goes under `<scratch>/pr<N>/<finding>/` and nowhere else; the checkout and any worktree are never write targets, though `git show`/`git archive` at a pinned ref read fine anywhere. Chain the directory change into the command, `cd "$D" && git …`, never `cd "$D"; git …`, so a failed `cd` cannot leave a `git` command running in the checkout — and bracket a fixture's own git with `git rev-parse --show-toplevel`: before `git init` it must NOT resolve to the repository, and a fresh scratch dir's `fatal: not a git repository` (exit 128) is the pass, not a failure; before any `git commit` it must resolve to your scratch path — compare resolved forms (`realpath`), since `--show-toplevel` can report `/private/tmp/…` for a `/tmp` scratch dir on macOS.* That last clause is the whole mechanism — a refuter left to reason its way to agreement rubber-stamps, and "apply only what survives" silently degrades into "apply everything in scope". Apply only what survives; a refuted one defers with the refutation in its body. "In scope" means the ticket's scope, not merely the same files — and applying one still never licenses editing a file the ticket has no business in. **A finding in `unverified` whose refuters ran and crashed always defers, whatever its severity** — and which of the two it is, you read off the finding's **`refutersDispatched`**, never off severity and never off an empty vote list: above zero with nothing surviving means every refuter dispatched against it died, and at `critical` that is every one of them. Only a `survived` finding has been checked; severity says how much a finding would matter if true, never whether anything looked. **Deferring a crash is your move, and re-running it is not** — the review already re-dispatched every crashed refuter pair once, in-run, so a crash that reaches you crashed twice, and the result's `resume` field names the one move left, which is not yours: nothing replays a review that ran in a runner's kernel — it is reported, not acted on. So say *in the deferral* that its refuters crashed rather than that it went unchecked, and re-run nothing for it. **A `suggestion` is also in `unverified`, for a different reason — `refutersDispatched` of zero, the 0-refuter budget the workflow gives that band by policy, so nothing looked *there* — and the scope split above, not this rule, covers it.**
|
|
12
|
-
**Write every ruling to `<scratch>/dispositions-<
|
|
12
|
+
**Write every ruling to `<scratch>/dispositions-<your name>.json`, `<your name>` the `fix-pr-<pr>[-x]` you were dispatched as (standalone, `fix-pr-<pr>`) — the disposition record `dispositions-check.mjs` judges against the review result file before any finisher is dispatched.** It is `{"head": "<the review file's head>", "run": "<the run root's name in the review path>", "entries": [ … ]}` — `run` the `<run>` of the review path your prompt names, which the check holds against the one the PR's latest `reviewed=` names, so a record written against an earlier review of the same PR, a same-head one included, is a mismatch (standalone, with no ledger naming a run, leave it out): one entry per `survived` finding and per `unverified` finding, plus one per `refuted` finding you reverse. Each entry carries `bucket` (`survived|unverified|refuted`), `index` (its position in that bucket of the review file — stable, since a round's file never changes), `scope` (`in|out`), `claimKind` (`behavior|shape`) and `disposition` (`apply|defer`), and where they apply `reason`, `issue` (the number you filed it to or commented on), `verdictPath` (an in-scope suggestion's refuter verdict file) and `remedyFiles` (the files its remedy names). A `refuted` entry carries `scope`, `claimKind` and `disposition` like every other entry, and a non-empty `reason` holding the evidence that reverses it; a missing or blank `reason` is a mismatch. The check reads scope off the diff before it reads your entry: a finding with no `line`, or on a line `git diff` touches from the PR's merge-base with `origin/main` to the review's `head`, is in scope whatever `scope` says, and only otherwise does the declared one stand. An in-scope `survived` finding deferred passes only with `reason` `false-rationale`, `mutual-exclusion`, `remedy-worse` or `remedy-outside-diff`. `remedy-outside-diff` — the remedy would edit a file the PR's diff does not touch — also needs `remedyFiles` to name at least one file absent from `git diff` from that merge-base to the review's `head`: none named, or every one in that diff, is a mismatch. A `critical` or `important` finding deferred for it goes to a human instead of passing: the script exits 1 with verdict `escalate`, `ledger.mjs dispatch` refuses the PR's finisher, and nothing retries it; a `suggestion` deferred for it passes. Any other reason, none, or a `survived` or `unverified` finding with no entry is a mismatch: the script exits 1 naming each violating entry's bucket, index and rule, and `ledger.mjs dispatch` refuses the PR's finisher. `disposition` has no `dismiss`: an in-scope `survived` finding verified correct, needing no change, is deferred with `reason` `false-rationale` and filed on the closed suggestion-band record — step 5's `PR #<pr> review: the suggestion band, checked`.
|
|
13
13
|
3. **Run `testCmd` before you commit**, from the worktree, copied verbatim — not a runner you picked. `testCmd` is the command your dispatcher handed you; **standalone, with no dispatcher to hand you one, it is the repo's own test command**, the Test entrypoint in the repo's Recipe cache (`derive-testcmd.sh <repo> test` prints it). Read it off the repo, never invent one: "not a runner you picked" is what this step forbids, and on the standalone path nothing else defines the referent. With no usable Recipe cache (`derive-testcmd.sh` refuses), run the Recipe derivation step first — dispatch a `task` member, agent `fleet-recipe-deriver`, its prompt naming the main checkout's absolute path as `<repo>` and a scratch directory of its own; it proves both commands in a throwaway worktree and writes the cache only on proof, and its `RECIPE NOT PROVEN` is a stop, reported with its reason. Never fall back to a runner of your own choosing. `tests 0` is a FAILED run, not a pass: a glob matching nothing still exits 0. **Zero is not the only no-work count.** A run reporting **0 passes with no failures** skipped everything it collected, and a count **materially below the whole suite** means you ran a partial tree — this suite reported `tests 937` when this line was written, so compare what you get against a full run rather than against zero. The number drifts; that is the safe direction, because a stale one produces a loud false red while its absence produces a silent false green. Red, zero-test, or either no-work count → fix it or move the finding to defer; never commit over it. Then commit and push — `git push origin HEAD:<branch>`, the explicit refspec: this step also runs from a `--detach` worktree (`run-team/SKILL.md`'s worktree-recreation form), where a bare `git push` exits 128 (`fatal: You are not currently on a branch`) and any push whose destination is `HEAD` rather than a full or branch refname (`-u` or not, forced or not) exits 1 (`error: The destination you provided is not a full refname`) — and report the test result alongside the SHA. **Then reconcile the PR body with what you actually did.** It merges into the permanent record and nothing downstream re-derives it: your commit messages are yours to fix and usually get fixed, the body is neither. Removing a change, or reversing one, leaves the body asserting work the diff no longer contains. Measured twice in one run — one body kept asserting a calibration its own fix had just deleted as false, another kept claiming a hunk the fix had removed as "this diff owns it" — both caught by a finisher's reading rather than by any rule, and both would otherwise have merged intact. `gh pr edit <pr> --body-file <path>`, and re-read the body against the final diff rather than against your memory of it. **Never `--no-verify`.** The fleet runs in repos it knows nothing about: it must not assume a pre-commit hook exists, and must never bypass one that does — a hook that fails is a finding to report, not an obstacle. Nothing re-reviews what you commit here, so this run is the only gate between your edit and a merge.
|
|
14
14
|
4. **Under the fleet, do not hold this wait.** The controller owns a persistent CI Monitor and dispatches a finisher once your report has arrived and CI is green — so push, then report to the controller a **final verdict** (every specialist you dispatched + its outcome + your apply/defer adjudication + the pushed SHA) and stop. A specialist that has neither reported nor been relayed to you is not a verdict line — ask the controller for it by name before ruling; see **Specialists**. The controller keys "review complete" on that verdict, **never on you merely going idle** — a slow specialist can still be running when your turn ends, and finishing off idle-alone ships a premature label (observed: a PR labelled while its contract-test specialist was still running and about to surface a vacuous pin). A turn-based reviewer re-reading `gh pr checks` each idle cycle rebuilds its whole context for nothing the Monitor lacks. **Standalone (no controller): watch** the run until the diff-validating jobs are green (bind via `ci-state.mjs` below); fix a **genuine** failure and repeat from 3. **Staleness is not a failure:** a `rebase-check` red or heavy jobs `skipped` **solely from a non-zero behind-count** never clears by waiting and must not be rebased away here — see **`skipped` is not `passed`** and **Do NOT rebase to label** below for what it looks like and why. Stop watching and go to 6. A turn-based reviewer that "waits for green" on a behind PR stalls forever and ends its turn without labelling. **`ci-state.mjs` returns `verdict: "no-ci"`** → there is no run to watch, so don't: go straight to 6 and let that step's `--declare-no-ci` branch decide.
|
|
15
15
|
5. File each deferred finding: `gh issue create --label <ready-for-agent|needs-triage>` with the finding, `file:line`, why deferred, `Deferred from PR #<pr> review`, and **the finding's `dimension` in the issue body** — the review returns it on every finding, and a backlog that does not carry it cannot be attributed to the specialist that generated it. **The label follows the finding's state, not this step** — but it is never omitted; an unlabelled issue is invisible to both `candidates.mjs --require-label` and the cockpit's pool query, reachable only by a human. **The bar here is whether the DEFECT is confirmed — never whether the remedy is settled.** Confirmed defect with the remedy still open → `ready-for-agent`, and say in the body that the remedy is open. Torn → `needs-triage`, narrowed: torn means **unsure anything is broken**, not unsure how to fix it. **A finding a refuter CHECKED and did not survive is neither** — it is not a confirmed defect, and you are not unsure whether anything is broken: you measured that nothing is. Refutations go as **one issue per review, titled `PR #<pr> review: the suggestion band, checked`, labelled `wontfix`, and CLOSED** — a run that files them open instead has to be corrected afterwards, as one was on 2026-08-27. Left open they read as queued agent work and rot into filing artifacts; a measured triage pass found a quarter of its `needs-triage` backlog was refuted records, bulk-closable by title shape. Closed is still a durable home — `check` reads the tracker at `--state all`, so a future run re-deriving the finding lands on the evidence rather than re-litigating it. Split the band only where an entry is **not** a refutation: a residual the refuter surfaced, or a finding refused as out of scope and never checked, is real deferred work — and real deferred work earns its own open issue only past a second bar. **Worth a claim: a finding whose own claim is that correct code could be shaped better — style, naming, layout, redundancy, a micro-simplification — never gets its own open issue.** Its only open question is "is this worth doing?", which answers itself under the repriced triage bar: measured 2026-08-15→31, 133 issues were triage-closed `wontfix` in sixteen days, 121 of them review deferrals, mostly this shape. Append it to the same closed record issue under a `Below the claim bar` heading — `file:line`, dimension, the claim, and a re-open trigger per entry. Closed is a durable home for these too: `check` reads `--state all`, so a run re-deriving the finding lands on the record instead of re-filing — and a later review returning the same finding as `survived`, or a maintainer hitting it, is the promotion signal: file it open then, citing the record. **An in-scope rediscovery step 2 applied is that same promotion signal, and it files nothing** — so the record carries no mark unless you write one: run `check` over each finding step 2 applied as well, and where its tracker rows put one on a `Below the claim bar` entry, comment on that record issue, opening `Promoted — applied, not filed`, and name the entry, that its refuter let it through, and the commit carrying the fix. That comment is the promotion mark this path leaves, and it is **not** a cross-reference — measured on this tracker, a comment mints `cross-referenced` on the issue its own text *names*, never on the issue it sits under (a comment citing a second issue landed on that issue's timeline and nowhere else), so the fixed opening is what carries the signal instead. **The bar reads the finding's claim, never its review band** — a finding that alleges wrong behavior, whatever band it sat in, `unverified`-with-crashed-refuters included, is above this bar and files open under the confirmed/torn split, which is why a band cannot carry this decision. Do **not** reach for phase 0's **decided?** test here. That is a *consumption* gate, priced for a seat where implementer divergence costs a claim, a worktree and a dispatch; at filing time divergence costs one PR review the maintainer already runs. Same test, two seats, opposite correct answers — and reading it in this seat took `needs-triage` from 11 to 135 before it was repriced (ruled 2026-08-18). **Dedupe through `~/.fleet/bin/fleet-run ledger.mjs check "<subject>"`, never a bare `gh issue list --search`** — the duplicate class that guard exists to catch was filed from *here*: the ledger row for the finding rediscovered five times in one measured run carries a reviewer's own source tag, `(review-pr-108)`. `check` reads this run's filed list *and* the tracker at `--state all`, so a closed duplicate still counts. Exit **1** is already filed in this run; exit **3** is a clean ledger with matching tracker rows that score against the subject — read the ones it prints and comment on the existing follow-up rather than filing a second issue. **Exit 0 is not automatically "safe to file"**: it covers `clean`, `soft-hit` and `unverified` alike, and an unreachable `gh` degrades to a ledger-only answer that says `TRACKER NOT CHECKED`, so read the stdout `verdict` to tell a tracker that was searched and clean from one nobody read — under `unverified`, search it yourself before filing — or, for the applied-promotion comment above, before treating a tracker nobody could read as a reason to skip it. **`soft-hit` is the answer that costs most to misread**: it means related rows were found and none of them was established as a duplicate — a near-miss above the floor, or tracker rows the scorer rates 0.00 — so read the rows it printed and decide, and never treat that exit 0 as permission. Exit **2** is a usage error, meaning nothing was checked at all: fix the invocation and re-run, never read it as a duplicate. Full exit-code and `verdict` semantics live in `$(~/.fleet/bin/fleet-run --root)/skills/run-team/SKILL.md`'s **Run ledger** section — read them there rather than restating them here. **The guard dedupes, it never drops:** a duplicate becomes a comment on the issue that already covers it, never a dropped finding. **Record every issue you create, at once:** immediately after each `gh issue create`, run `~/.fleet/bin/fleet-run ledger.mjs filed <N> "<subject>"` — `<N>` the number `gh issue create` printed, and **the same subject string you passed to `check`**, so the next `check` for that finding meets it in the filed list instead of scoring a rewording. Without it the guard's ledger half never holds a row, and its exit **1** can never fire. If `filed` fails, retry it once; if it fails again, leave the issue alone — it exists and is correct — and report it as `unrecorded: #N <subject>`, so the controller runs `filed` for it. Report `filed: #N <subject>` for every issue you created — under the fleet, in your report to the controller. **The one window the guard cannot see:** `check` → `gh issue create` → `filed` is three commands, and the ledger's lock covers each alone, so two filers can both read `clean` in the seconds before either records. That gap is accepted, never a reason to skip `filed`. Post the numbers as one PR comment.
|
|
16
16
|
**A duplicate that should hold an open deferral is relabelled up, never down.** Where the guard lands a finding whose home is an open issue on an existing one that is open and labelled `needs-triage` while the finding's state calls for `ready-for-agent` — a defect its refuters confirmed — relabel it with `gh issue edit <N> --remove-label needs-triage --add-label ready-for-agent`, and cite the refuters' verdict in the comment you leave there. Never relabel `ready-for-agent` down to `needs-triage`: a `ready-for-agent` issue already holds a `needs-triage` finding, and a later run's crashed refuters are no reason to send a confirmed defect back to triage. A closed one cannot hold it: file a new open issue citing it, as the promotion rule above does for the closed record.
|
|
17
17
|
**Name where each deferral went, in the disposition record.** Every entry you defer carries `issue`: the number of the issue you filed it to, commented it onto, or appended it to — the closed `PR #<pr> review: the suggestion band, checked` record included. An in-scope `suggestion`'s entry — an `unverified` finding with `refutersDispatched` of zero — also carries `verdictPath`, applied or deferred: the absolute path of a file holding its refuter's verdict as JSON, `{"refuted": <boolean>, "reason": "<string>"}`, under your own `<scratch>/pr<N>/fix-XXXXXXXX/<finding>/` run root. `dispositions-check.mjs` reads each named issue back through `gh issue view --json labels,state,title` and each verdict file off disk; a mismatch names the row of its filing table the entry broke, so move or relabel that filing to the home the row names and run the check again. With `gh` unreachable, or an issue it cannot read, that filing goes unchecked: with nothing else broken the verdict is `unchecked` — never `ok`, and it exits 1 — so run the check again once `gh` answers.
|
|
18
|
-
6. Diff-check green (run-bound per `ci-state.mjs`; under staleness see **`skipped` is not `passed`** for what the label then rests on) **and** deferrals filed **and** the PR carrying exactly one release label (`patch`/`minor`/`major`, counting only those three; **required on every PR** — `release-label.yml` auto-adds `patch` when none is present and hard-fails on more than one, so this never needs to check whether the repo defines the gate at all, and `gh label list` settles nothing here any more) **and** `dispositions-check.mjs --member fix-pr-<pr> --scratch <scratch>` run over your own record exiting 0 (standalone that is yours to run, from the worktree, after step 2 writes `<scratch>/dispositions-<pr>.json`;
|
|
18
|
+
6. Diff-check green (run-bound per `ci-state.mjs`; under staleness see **`skipped` is not `passed`** for what the label then rests on) **and** deferrals filed **and** the PR carrying exactly one release label (`patch`/`minor`/`major`, counting only those three; **required on every PR** — `release-label.yml` auto-adds `patch` when none is present and hard-fails on more than one, so this never needs to check whether the repo defines the gate at all, and `gh label list` settles nothing here any more) **and** `dispositions-check.mjs --member fix-pr-<pr> --scratch <scratch>` run over your own record exiting 0 (standalone that is yours to run, from the worktree, after step 2 writes `<scratch>/dispositions-fix-pr-<pr>.json`; `--review <path>` names the review file, which you write yourself from the findings step 1 collected, in the shape **The review result file** describes — `pr`, `head`, `snapshot`, `counts` and the `survived`, `refuted` and `unverified` buckets; standalone you pass `--no-ledger` with it, so it writes no token and the exit status is the verdict — a `.fleet/ledger.md` the repository already holds is a previous run's, and without the flag it exits 2 for a ledger with no row for you; under the fleet `ledger.mjs dispatch` has already refused a finisher without it) → `gh pr comment <pr> --body "ready-to-merge-head: <prHead>"`, `prHead` being the full SHA your green `ci-state.mjs` reading reported, which the merge bot reads as the head the label was applied to, then `gh pr edit <pr> --add-label ready-to-merge`. Do not merge. A non-zero dispositions exit halts you before the label, and you report the violations it printed. Zero or more than one release label halts you before the label, naming which you found — a timed-out `gh pr create` opens the PR with its `--label` unapplied and no exit status to react to, and nothing after this step re-derives the release label. **Under the fleet a finisher does this** — a fresh small agent the controller dispatches (after your final verdict, never on your bare idle). Gate it on the diff-validating **`check`** job, **not** on `ci-state --quiet` exit 0, which a behind PR — the normal case once any sibling merges — never reaches: a `rebase-check` red or heavy jobs `skipped` off the behind-count hold it short of full green, so an exit-0 gate strands every behind PR unlabelled. Label when `check` is present and green **and no heavy job (the diff-validating suites — *not* the `rebase-check` currency gate) is in `failure`** — a `skipped` heavy job is behind-count staleness and fine — **solely** off that count, which is a condition to establish rather than infer, see the five-condition note in `$(~/.fleet/bin/fleet-run --root)/skills/run-team/SKILL.md`'s fix-applier prompt block. A `failure` is the diff's own fault and blocks the label, else a genuinely broken PR gets labelled off a green `check` and burns a merge-bot rebase+CI cycle before the failure resurfaces. Read per-job state for this: run `ci-state.mjs` **without** `--quiet`, or read its `jobs` — the `--quiet` payload drops the per-job `jobs`/`missing` and the `dropped` list, keeping `reasons` (which still distinguishes `is failure` from `is skipped`). You reach step 6 yourself only when green is already in hand at push time.
|
|
19
19
|
|
|
20
20
|
**`verdict: "no-ci"` — this repo has no `check` job to gate on at all**, and that is a different state from red or unknown, not a third way to skip the wait. Label off step 3's own green `testCmd` run instead of a `check` job that will never exist — and only once `ci-state.mjs` reads `no-ci` **without** the flag, since a repo whose CI is merely misconfigured exits 2 there rather than reading no-ci. `--declare-no-ci` is your caller-side declaration that step 3's run **is** this repo's gate: it flips the exit code and echoes itself into `reasons`, so finding it in the payload confirms only that you passed it, never that anyone verified anything. **No green `testCmd` run on record → do not label.** Say in your verdict that the repo has no CI configured and no declaration, and stop — inferring a pass from silence here is exactly the misconfigured-CI case the declaration exists to rule out. See `ci-state.mjs --pr <n> --declare-no-ci` below.
|
|
21
21
|
|
|
@@ -41,10 +41,10 @@ Three claims, none substituting for another: **run-binding** (this green belongs
|
|
|
41
41
|
|
|
42
42
|
## The review result file
|
|
43
43
|
|
|
44
|
-
**Dispatched as a fix-applier, you read the review off one file: `<scratch>/review
|
|
44
|
+
**Dispatched as a fix-applier, you read the review off one file: `<scratch>/pr<pr>/<run>/review.json`, the path your prompt names — `<run>` the review's own run root, the one the PR's latest `reviewed=` token names.** It holds the bare result object the review returned, in this key order — the digest `pr`, `head`, `resume`, `testEnvironment`, `dimensionsRun`, `dimensionsUnrun`, `cwdAudit`, `counts`, then `snapshot`, `survived`, `refuted`, `unverified` — the `review-pr-<pr#>` runner member (`agents/fleet-review-runner.agent.md`) writes it. The controller reads the digest and never the findings, so **every per-PR ruling in this section is yours** — nobody upstream has made one for you. A prompt that pastes the findings in instead of naming the file hands you the same buckets, and every ruling here applies to them unchanged. Read the file a bucket at a time with `jq`, never whole into your context — a review's result runs to tens of KB:
|
|
45
45
|
|
|
46
46
|
```bash
|
|
47
|
-
R=<scratch>/review
|
|
47
|
+
R=<scratch>/pr<pr>/<run>/review.json
|
|
48
48
|
jq '{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts}' "$R" # the digest
|
|
49
49
|
jq -r '.testEnvironment' "$R" # what the shared test run is evidence about
|
|
50
50
|
jq '.survived' "$R" # apply candidates, every one
|
package/package.json
CHANGED
|
@@ -195,9 +195,12 @@ export function parseRow(row) {
|
|
|
195
195
|
// The head that review actually read — the cockpit's own copy
|
|
196
196
|
// of fleet-tick.mjs's `reviewedHead`, needed below to tell a past-pin
|
|
197
197
|
// halt already answered by this review apart from one still owed a
|
|
198
|
-
// fresh one.
|
|
198
|
+
// fresh one. A token fleet-tick.mjs refuses — one that names no review
|
|
199
|
+
// run — earns the row the same `ledger-error` flag any malformed token
|
|
200
|
+
// does, rather than a row reviewed at no head.
|
|
199
201
|
const m = REVIEWED.exec(tok);
|
|
200
202
|
if (m) reviewedHead = m[1].toLowerCase();
|
|
203
|
+
else malformed = true;
|
|
201
204
|
for (const r of runners) settled.add(r);
|
|
202
205
|
runners = [];
|
|
203
206
|
}
|
|
@@ -1,28 +1,32 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
// The disposition check: a review fix-applier's rulings on a review's
|
|
3
3
|
// findings, checked as code rather than read back from its report. The
|
|
4
|
-
// fix-applier writes its record to `<scratch>/dispositions-<
|
|
5
|
-
// the review result file
|
|
6
|
-
//
|
|
7
|
-
//
|
|
4
|
+
// fix-applier writes its record to `<scratch>/dispositions-fix-pr-<M>[-x].json`,
|
|
5
|
+
// one file per fix-applier; the review result file it answers is
|
|
6
|
+
// `<scratch>/pr<M>/<run>/review.json`, `<run>` the run root the PR's latest
|
|
7
|
+
// `reviewed=<head>:<survived>/<refuted>/<unverified>:<run>` names. This
|
|
8
|
+
// script reads the two together, judges the record, and writes its verdict
|
|
9
|
+
// onto the PR's ledger row as a token `ledger.mjs dispatch` gates the PR's
|
|
10
|
+
// finisher on:
|
|
8
11
|
//
|
|
9
|
-
// dispositions-ok=fix-pr-<M>[-x]:<
|
|
10
|
-
// dispositions-mismatch=fix-pr-<M>[-x]:<
|
|
11
|
-
// dispositions-escalate=fix-pr-<M>[-x]:<
|
|
12
|
-
//
|
|
13
|
-
//
|
|
14
|
-
//
|
|
15
|
-
//
|
|
16
|
-
//
|
|
17
|
-
// dispositions-unchecked=fix-pr-<M>[-x]:<
|
|
18
|
-
//
|
|
12
|
+
// dispositions-ok=fix-pr-<M>[-x]:<run> every rule below held
|
|
13
|
+
// dispositions-mismatch=fix-pr-<M>[-x]:<run> at least one entry broke one
|
|
14
|
+
// dispositions-escalate=fix-pr-<M>[-x]:<run> a human rules: either every rule held
|
|
15
|
+
// but a critical or important finding
|
|
16
|
+
// was deferred `remedy-outside-diff`,
|
|
17
|
+
// or a rule broke and an earlier
|
|
18
|
+
// fix-applier's record on the same
|
|
19
|
+
// review already had
|
|
20
|
+
// dispositions-unchecked=fix-pr-<M>[-x]:<run> no rule broke, but where a deferral was
|
|
21
|
+
// filed could not be read
|
|
19
22
|
//
|
|
20
|
-
// `<
|
|
21
|
-
//
|
|
23
|
+
// `<run>` is the review's own run, so a verdict on one review never answers
|
|
24
|
+
// a later one — a second review at the same head included.
|
|
22
25
|
//
|
|
23
26
|
// The record:
|
|
24
27
|
//
|
|
25
28
|
// { "head": "<the review file's head>",
|
|
29
|
+
// "run": "<the review's run root, as the review path names it>",
|
|
26
30
|
// "entries": [ { "bucket": "survived" | "unverified" | "refuted",
|
|
27
31
|
// "index": <position in that bucket of the review file>,
|
|
28
32
|
// "scope": "in" | "out",
|
|
@@ -42,6 +46,11 @@
|
|
|
42
46
|
// never changes.
|
|
43
47
|
//
|
|
44
48
|
// Rules — each broken one is reported as `<bucket>[<index>]: <rule>`:
|
|
49
|
+
// - Run: with a ledger, `run` is the run the PR's latest `reviewed=` names. A
|
|
50
|
+
// record naming another run, or none, answers a review that is not this
|
|
51
|
+
// one — a fix-applier that ruled on an earlier review at the same head
|
|
52
|
+
// included — and is a mismatch. Without a ledger nothing names the run,
|
|
53
|
+
// and `run` is not read.
|
|
45
54
|
// - Coverage: a `survived` or `unverified` finding with no entry was
|
|
46
55
|
// dropped. An entry naming no finding, or a second entry for one finding,
|
|
47
56
|
// answers nothing.
|
|
@@ -124,7 +133,9 @@
|
|
|
124
133
|
// the whole verdict: no token is written, no row is read, and `token` in the
|
|
125
134
|
// stdout payload is null. That is the shape of a standalone `/review-and-fix`
|
|
126
135
|
// with no controller, chosen by `--no-ledger` or by a ledger path that does
|
|
127
|
-
// not exist.
|
|
136
|
+
// not exist. With no `reviewed=` to name the review, `--review <path>` names
|
|
137
|
+
// its file, and is required; with a ledger it is refused, since the ledger
|
|
138
|
+
// already names one.
|
|
128
139
|
// A repository's `.fleet/ledger.md` outlives the fleet run that wrote it, so a
|
|
129
140
|
// standalone member in a worktree of such a repository would otherwise find
|
|
130
141
|
// the previous run's ledger, and a ledger that exists is never skipped: one
|
|
@@ -144,13 +155,17 @@ import { isCLI } from "./is-cli.mjs";
|
|
|
144
155
|
import { gitEnv } from "./git-env.mjs";
|
|
145
156
|
import { fleetFile, FleetDirUnresolvable } from "./fleet-dir.mjs";
|
|
146
157
|
import { parseMember, memberTokens } from "./ledger-grammar.mjs";
|
|
147
|
-
import { dispositionsToken,
|
|
158
|
+
import { deriveRun, dispositionsToken, LedgerError, rowNums } from "./fleet-tick.mjs";
|
|
148
159
|
import { VERDICT_SCHEMA } from "./review-core.mjs";
|
|
149
160
|
|
|
150
161
|
const NAME = "dispositions-check";
|
|
151
162
|
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));
|
|
152
163
|
const LEDGER_SCRIPT = join(SCRIPT_DIR, "ledger.mjs");
|
|
153
164
|
|
|
165
|
+
// Two spellings of one commit: either head a prefix of the other, the way a
|
|
166
|
+
// record's head (7-40 hex) is matched against its review's full head.
|
|
167
|
+
const sameHead = (a, b) => a.startsWith(b) || b.startsWith(a);
|
|
168
|
+
|
|
154
169
|
// ---------------------------------------------------------------------------
|
|
155
170
|
// pure core — no filesystem or process access until main()
|
|
156
171
|
// ---------------------------------------------------------------------------
|
|
@@ -372,7 +387,9 @@ function shapeErrors(e) {
|
|
|
372
387
|
* record, or null when there is none to read (`recordProblem` then says why).
|
|
373
388
|
* `touched` is touchedLines() for the review's head; `diffFiles` every file
|
|
374
389
|
* name `git diff <merge-base>...<head>` lists, required only when an entry
|
|
375
|
-
* defers `remedy-outside-diff`; `
|
|
390
|
+
* defers `remedy-outside-diff`; `run` the review's run root name, or null when
|
|
391
|
+
* no ledger names it — a record answers it only by carrying the same `run`;
|
|
392
|
+
* `roots` the directories an absolute finding
|
|
376
393
|
* or remedy path is read relative to. `filing` is `{pr, issue(n), verdict(path)}`,
|
|
377
394
|
* required only when an entry defers or is an in-scope suggestion: `issue`
|
|
378
395
|
* answers `{state, title, labels}` — `state` upper-case, `labels` names — or
|
|
@@ -381,7 +398,7 @@ function shapeErrors(e) {
|
|
|
381
398
|
* `review` must pass reviewProblem() — a bucket missing or holding a
|
|
382
399
|
* non-object is a TypeError here, not a violation.
|
|
383
400
|
*/
|
|
384
|
-
export function checkDispositions({ review, record, recordProblem = null, touched, diffFiles = null, roots = [], filing = null }) {
|
|
401
|
+
export function checkDispositions({ review, record, recordProblem = null, run = null, touched, diffFiles = null, roots = [], filing = null }) {
|
|
385
402
|
const violations = [];
|
|
386
403
|
const escalations = [];
|
|
387
404
|
const unchecked = [];
|
|
@@ -403,6 +420,9 @@ export function checkDispositions({ review, record, recordProblem = null, touche
|
|
|
403
420
|
if (!/^[0-9a-f]{7,40}$/.test(head) || !sameHead(head, reviewHead)) {
|
|
404
421
|
at(null, null, `the record answers head ${JSON.stringify(record.head ?? null)}, not the review's ${review.head}`);
|
|
405
422
|
}
|
|
423
|
+
if (run !== null && record.run !== run) {
|
|
424
|
+
at(null, null, `the record answers run ${JSON.stringify(record.run ?? null)}, not the review's ${run}`);
|
|
425
|
+
}
|
|
406
426
|
}
|
|
407
427
|
|
|
408
428
|
const seen = new Map();
|
|
@@ -527,7 +547,7 @@ export function formatEscalation({ bucket, index, severity, files }) {
|
|
|
527
547
|
}
|
|
528
548
|
|
|
529
549
|
// The row text with `token` in place of every verdict the same member wrote
|
|
530
|
-
// for the same review
|
|
550
|
+
// for the same review run — a re-run replaces its own earlier verdict rather
|
|
531
551
|
// than leaving two to disagree. Unchanged when the token is already the only
|
|
532
552
|
// one. Separators a removal leaves doubled are folded.
|
|
533
553
|
export function withVerdict(rowText, token) {
|
|
@@ -536,7 +556,7 @@ export function withVerdict(rowText, token) {
|
|
|
536
556
|
const words = String(rowText).trim().split(/\s+/).filter(Boolean);
|
|
537
557
|
const kept = words.filter((w) => {
|
|
538
558
|
const d = dispositionsToken(w);
|
|
539
|
-
return d === null || d.member.name !== mine.member.name ||
|
|
559
|
+
return d === null || d.member.name !== mine.member.name || d.run !== mine.run;
|
|
540
560
|
});
|
|
541
561
|
if (kept.length === words.length - 1 && words.includes(token)) return words.join(" ");
|
|
542
562
|
const folded = kept.join(" ").replace(/·(?:\s+·)+/g, "·").replace(/^·\s*|\s*·$/g, "").trim();
|
|
@@ -544,16 +564,16 @@ export function withVerdict(rowText, token) {
|
|
|
544
564
|
}
|
|
545
565
|
|
|
546
566
|
// Whether a fix-applier on PR `pr` with a lower retry suffix than `member`
|
|
547
|
-
// already has a mismatch or escalate on the ledger for `
|
|
548
|
-
// that resolves to the PR, since a row can be split. An unchecked verdict
|
|
549
|
-
// no failure of a record, and a verdict on another review
|
|
567
|
+
// already has a mismatch or escalate on the ledger for review `run` — on any
|
|
568
|
+
// row that resolves to the PR, since a row can be split. An unchecked verdict
|
|
569
|
+
// is no failure of a record, and a verdict on another review counts for
|
|
550
570
|
// nothing, so a new review starts the count again.
|
|
551
|
-
export function failedBefore(rows, pr, member,
|
|
571
|
+
export function failedBefore(rows, pr, member, run) {
|
|
552
572
|
const mine = member.retry ?? "";
|
|
553
573
|
return rows.some((r) => rowNums(r).pr === pr && r.split(/\s+/).some((w) => {
|
|
554
574
|
const d = dispositionsToken(w);
|
|
555
575
|
return d !== null && (d.verdict === "mismatch" || d.verdict === "escalate") && d.member.number === pr
|
|
556
|
-
&& (d.member.retry ?? "") < mine &&
|
|
576
|
+
&& (d.member.retry ?? "") < mine && d.run === run;
|
|
557
577
|
}));
|
|
558
578
|
}
|
|
559
579
|
|
|
@@ -563,7 +583,7 @@ export function failedBefore(rows, pr, member, head) {
|
|
|
563
583
|
|
|
564
584
|
const die = makeDie(NAME);
|
|
565
585
|
const { arg, has, sweep, stray } = defineFlags(die, {
|
|
566
|
-
flags: { member: "value", scratch: "value", repo: "value", ledger: "value", "no-ledger": "bool" },
|
|
586
|
+
flags: { member: "value", scratch: "value", repo: "value", ledger: "value", "no-ledger": "bool", review: "value" },
|
|
567
587
|
});
|
|
568
588
|
|
|
569
589
|
// The one git primitive. An ambient GIT_DIR or GIT_WORK_TREE — a hook, a
|
|
@@ -694,7 +714,7 @@ function readJson(path) {
|
|
|
694
714
|
}
|
|
695
715
|
|
|
696
716
|
function main() {
|
|
697
|
-
const usage = "usage: dispositions-check.mjs --member fix-pr-<M>[-x] --scratch <dir> [--repo <path>] [--ledger <path> | --no-ledger]";
|
|
717
|
+
const usage = "usage: dispositions-check.mjs --member fix-pr-<M>[-x] --scratch <dir> [--repo <path>] [--ledger <path> | --no-ledger --review <path>]";
|
|
698
718
|
const name = arg("member");
|
|
699
719
|
const scratch = arg("scratch");
|
|
700
720
|
if (!name || !scratch) die(usage);
|
|
@@ -706,8 +726,36 @@ function main() {
|
|
|
706
726
|
const repo = arg("repo") ?? process.cwd();
|
|
707
727
|
const standalone = has("no-ledger");
|
|
708
728
|
if (standalone && arg("ledger") !== null) die("--no-ledger and --ledger contradict: name one");
|
|
729
|
+
const ledgerFile = standalone ? null : ledgerInUse(arg("ledger"), repo);
|
|
709
730
|
|
|
710
|
-
|
|
731
|
+
// The review the record answers. With a ledger it is the one PR M's latest
|
|
732
|
+
// `reviewed=` names — `<scratch>/pr<M>/<run>/review.json`, read off the
|
|
733
|
+
// same per-PR fold `ledger.mjs dispatch` gates the finisher by, so the check
|
|
734
|
+
// and the gate never judge two different reviews. Without one nothing names
|
|
735
|
+
// it, so `--review` must.
|
|
736
|
+
let data = null, row = null, run = null, reviewPath;
|
|
737
|
+
if (ledgerFile === null) {
|
|
738
|
+
reviewPath = arg("review");
|
|
739
|
+
if (reviewPath === null) die(`no ledger names PR #${pr}'s review — pass --review <path> for the review file ${member.name}'s record answers`);
|
|
740
|
+
} else {
|
|
741
|
+
if (arg("review") !== null) die(`--review and a ledger contradict: with a ledger the review is the one PR #${pr}'s latest reviewed= names`);
|
|
742
|
+
data = JSON.parse(runLedger(ledgerFile, ["read"], "read the ledger"));
|
|
743
|
+
// The verdict lands on the row carrying the member on its own PR's row —
|
|
744
|
+
// where `ledger.mjs dispatch` wrote it, and where the gate reads PR M's
|
|
745
|
+
// tokens from.
|
|
746
|
+
row = data.rows.find((r) => rowNums(r).pr === pr && memberTokens(r).some((t) => t.name === member.name));
|
|
747
|
+
if (row === undefined) die(`${member.name} is on no row of PR #${pr} — \`ledger.mjs dispatch\` records a fix-applier before its record can be checked; with no controller, pass --no-ledger`);
|
|
748
|
+
let latest;
|
|
749
|
+
try {
|
|
750
|
+
latest = deriveRun({ rows: data.rows, dispatched: data.dispatched, drain: data.drain }, []).reviewed.find((r) => r.pr === pr);
|
|
751
|
+
} catch (e) {
|
|
752
|
+
if (!(e instanceof LedgerError)) throw e;
|
|
753
|
+
die(`${e.message} — fleet-tick.mjs refuses this ledger, so PR #${pr}'s latest review cannot be read off it`);
|
|
754
|
+
}
|
|
755
|
+
if (latest === undefined) die(`PR #${pr} has no reviewed= on the ledger — no review for ${member.name}'s record to answer`);
|
|
756
|
+
run = latest.run;
|
|
757
|
+
reviewPath = join(scratch, `pr${pr}`, run, "review.json");
|
|
758
|
+
}
|
|
711
759
|
let review;
|
|
712
760
|
try {
|
|
713
761
|
review = readJson(reviewPath);
|
|
@@ -718,7 +766,9 @@ function main() {
|
|
|
718
766
|
if (problem !== null) die(`${reviewPath} ${problem}`);
|
|
719
767
|
const head = String(review.head).toLowerCase();
|
|
720
768
|
|
|
721
|
-
|
|
769
|
+
// One record per fix-applier: a retry, or a fix-applier answering a later
|
|
770
|
+
// review, writes its own and never replaces an earlier one's.
|
|
771
|
+
const recordPath = join(scratch, `dispositions-${member.name}.json`);
|
|
722
772
|
let record = null, recordProblem = null;
|
|
723
773
|
if (!existsSync(recordPath)) {
|
|
724
774
|
recordProblem = `no disposition record at ${recordPath}`;
|
|
@@ -748,30 +798,24 @@ function main() {
|
|
|
748
798
|
// file the PR moved away or removed is a file the PR's diff names.
|
|
749
799
|
const diffFiles = git(repo, ["diff", "--no-color", "--no-ext-diff", "--no-renames", "--name-only", "-z", base, head],
|
|
750
800
|
`list the files of ${base}..${head}`).split("\0").filter(Boolean);
|
|
751
|
-
const ledgerFile = standalone ? null : ledgerInUse(arg("ledger"), repo);
|
|
752
801
|
|
|
753
802
|
const { violations, escalations, unchecked } = checkDispositions({
|
|
754
|
-
review, record, recordProblem, touched: touchedLines(diff), diffFiles, roots: [review.snapshot, top],
|
|
803
|
+
review, record, recordProblem, run, touched: touchedLines(diff), diffFiles, roots: [review.snapshot, top],
|
|
755
804
|
filing: { pr, issue: issueReader(repo), verdict: verdictReader(scratch, pr) },
|
|
756
805
|
});
|
|
757
|
-
//
|
|
758
|
-
//
|
|
759
|
-
// tokens from. No ledger, no row: the exit status below is the verdict, and
|
|
760
|
-
// with no earlier fix-applier's verdict to count, a broken rule is always a
|
|
806
|
+
// No ledger, no row: the exit status below is the verdict, and with no
|
|
807
|
+
// earlier fix-applier's verdict to count, a broken rule is always a
|
|
761
808
|
// mismatch.
|
|
762
|
-
const data = ledgerFile === null ? null : JSON.parse(runLedger(ledgerFile, ["read"], "read the ledger"));
|
|
763
809
|
// A mismatch an earlier fix-applier already drew on this review makes this
|
|
764
810
|
// one the second: it is written as an escalate, which no fix-applier answers.
|
|
765
811
|
// A broken rule outranks an escalation, and an escalation a filing the
|
|
766
812
|
// tracker could not answer.
|
|
767
|
-
const secondMismatch = violations.length > 0 && data !== null && failedBefore(data.rows, pr, member,
|
|
813
|
+
const secondMismatch = violations.length > 0 && data !== null && failedBefore(data.rows, pr, member, run);
|
|
768
814
|
const verdict = violations.length > 0 ? (secondMismatch ? "escalate" : "mismatch")
|
|
769
815
|
: escalations.length > 0 ? "escalate" : unchecked.length > 0 ? "unchecked" : "ok";
|
|
770
|
-
const token = `dispositions-${verdict}=${member.name}:${
|
|
816
|
+
const token = data === null ? null : `dispositions-${verdict}=${member.name}:${run}`;
|
|
771
817
|
|
|
772
818
|
if (data !== null) {
|
|
773
|
-
const row = data.rows.find((r) => rowNums(r).pr === pr && memberTokens(r).some((t) => t.name === member.name));
|
|
774
|
-
if (row === undefined) die(`${member.name} is on no row of PR #${pr} — \`ledger.mjs dispatch\` records a fix-applier before its record can be checked; with no controller, pass --no-ledger`);
|
|
775
819
|
const key = row.split(/\s/)[0];
|
|
776
820
|
if (!/^#[0-9]+$/.test(key)) die(`the row carrying ${member.name} has no #<n> key for \`ledger.mjs row\` to rewrite it by: ${row}`);
|
|
777
821
|
const text = row.slice(key.length).trim();
|
|
@@ -786,9 +830,9 @@ function main() {
|
|
|
786
830
|
console.error(`${member.name}: ${u.bucket}[${u.index}]: where it was filed is unchecked — #${u.issue} could not be read through gh: ${u.problem}`);
|
|
787
831
|
}
|
|
788
832
|
if (secondMismatch) {
|
|
789
|
-
console.error(`${member.name}: escalate — an earlier fix-applier's record on review ${
|
|
833
|
+
console.error(`${member.name}: escalate — an earlier fix-applier's record on review ${run} already failed this check; no further fix-applier answers PR #${pr}, a human does`);
|
|
790
834
|
}
|
|
791
|
-
console.log(JSON.stringify({ member: member.name, pr, head, verdict, token
|
|
835
|
+
console.log(JSON.stringify({ member: member.name, pr, head, verdict, token, violations, escalations, unchecked }));
|
|
792
836
|
// exitCode, not exit(): stdout to a pipe is written asynchronously, and an
|
|
793
837
|
// exit() here could cut the payload off.
|
|
794
838
|
process.exitCode = verdict === "ok" ? 0 : 1;
|
package/scripts/fleet-tick.mjs
CHANGED
|
@@ -371,9 +371,15 @@ export class LedgerError extends Error {}
|
|
|
371
371
|
|
|
372
372
|
// `review=wf:<runId>` | `review=member:review-pr-<n>` |
|
|
373
373
|
// `review=fallback:review-pr-<n>[-b]`, settled dead as `…=failed`; the result
|
|
374
|
-
// is `reviewed=<head>:<survived>/<refuted>/<unverified
|
|
374
|
+
// is `reviewed=<head>:<survived>/<refuted>/<unverified>:<run>`, `<run>` the
|
|
375
|
+
// name of the review's own run root under `<scratch>/pr<n>/` (`run-` and the
|
|
376
|
+
// eight characters `mktemp` chose), where its result file `review.json` sits.
|
|
375
377
|
const REVIEW = /^review=(?:wf|member|fallback):[^=\s]+(=failed)?$/;
|
|
376
|
-
export const REVIEWED = /^reviewed=([0-9a-
|
|
378
|
+
export const REVIEWED = /^reviewed=([0-9a-fA-F]{7,40}):(\d+)\/(\d+)\/(\d+):(run-[A-Za-z0-9]{8})$/;
|
|
379
|
+
// The `reviewed=` shape before it named a run: it names no result file, so
|
|
380
|
+
// nothing can tell which review a dispositions verdict answered. Only a ledger
|
|
381
|
+
// written before the plugin update and read after it can hold one.
|
|
382
|
+
const REVIEWED_RUNLESS = /^reviewed=[0-9a-fA-F]{7,40}:\d+\/\d+\/\d+$/;
|
|
377
383
|
const HELD = /^held-behind[:-]#?(\d+)$/;
|
|
378
384
|
// The durable record that a PR conflicts — written onto the held PR's own
|
|
379
385
|
// row by the merge bot, when its local-rebase fallback hits a conflict it would
|
|
@@ -386,43 +392,42 @@ const CONFLICT_HOLD = /^conflict-hold[:-]#?(\d+)$/;
|
|
|
386
392
|
// for a settled member whose transcript was never written.
|
|
387
393
|
const TIER_VERDICT = /^tier-(ok|mismatch|unverifiable)=([^:\s]+):\S+$/;
|
|
388
394
|
// dispositions-check.mjs's verdict on a review fix-applier's disposition
|
|
389
|
-
// record: `dispositions-ok=fix-pr-<M>[-x]:<
|
|
395
|
+
// record: `dispositions-ok=fix-pr-<M>[-x]:<run>`,
|
|
390
396
|
// `dispositions-mismatch=…`, `dispositions-escalate=…` (a critical or
|
|
391
397
|
// important deferral, or a second mismatch on one review, that a human rules
|
|
392
398
|
// on) or `dispositions-unchecked=…` (no rule broke, but where a deferral was
|
|
393
|
-
// filed could not be read), `<
|
|
394
|
-
// answers. `ledger.mjs dispatch` refuses a
|
|
399
|
+
// filed could not be read), `<run>` the run of the review the record
|
|
400
|
+
// answers, as its `reviewed=` names it. `ledger.mjs dispatch` refuses a
|
|
401
|
+
// finisher on anything but an ok. A
|
|
395
402
|
// mismatch returns the PR to fixDue for one retry; an escalate is a hold only
|
|
396
403
|
// a human answers, so the tick never re-offers the PR for it; an unchecked is
|
|
397
404
|
// answered by running the check again, so the tick never offers a
|
|
398
405
|
// fix-applier for it either. A token outside this shape is no verdict at all,
|
|
399
406
|
// which the gate reads as unchecked: fail closed.
|
|
400
407
|
export function dispositionsToken(tok) {
|
|
401
|
-
const m = /^dispositions-(ok|mismatch|escalate|unchecked)=([^:\s]+):([
|
|
408
|
+
const m = /^dispositions-(ok|mismatch|escalate|unchecked)=([^:\s]+):(run-[A-Za-z0-9]{8})$/i.exec(tok);
|
|
402
409
|
if (!m) return null;
|
|
403
410
|
const member = parseMember(m[2]);
|
|
404
411
|
if (member === null || member.family !== "fix-pr") return null;
|
|
405
|
-
return { verdict: m[1].toLowerCase(), member,
|
|
412
|
+
return { verdict: m[1].toLowerCase(), member, run: m[3] };
|
|
406
413
|
}
|
|
407
414
|
|
|
408
|
-
// Two spellings of one commit: either head a prefix of the other, the way a
|
|
409
|
-
// `reviewed=` head (7-40 hex) is matched against gh's full headRefOid.
|
|
410
|
-
export const sameHead = (a, b) => a.startsWith(b) || b.startsWith(a);
|
|
411
|
-
|
|
412
415
|
// Which verdict outranks which when one fix-applier carries more than one for
|
|
413
|
-
// a
|
|
416
|
+
// a review: the stricter reading wins.
|
|
414
417
|
const VERDICT_RANK = { ok: 0, unchecked: 1, mismatch: 2, escalate: 3 };
|
|
415
418
|
|
|
416
|
-
// A PR's current dispositions verdict against its latest review
|
|
417
|
-
// the tokens answering that
|
|
419
|
+
// A PR's current dispositions verdict against its latest review run: among
|
|
420
|
+
// the tokens answering that run, the one from the fix-applier with the
|
|
418
421
|
// highest retry suffix ("" < "b" < "c" …) — never row-text position, which a
|
|
419
422
|
// `row` rewrite can reorder. One fix-applier carrying two verdicts for one
|
|
420
|
-
//
|
|
423
|
+
// review reads as the stricter one. null when no token answers the run — so a
|
|
424
|
+
// second review at the same head is answered by none of the first one's
|
|
425
|
+
// verdicts.
|
|
421
426
|
/** @returns {{verdict: "ok"|"unchecked"|"mismatch"|"escalate", member: string}|null} */
|
|
422
|
-
export function currentDispositions(tokens,
|
|
427
|
+
export function currentDispositions(tokens, run) {
|
|
423
428
|
let best = null;
|
|
424
429
|
for (const t of tokens) {
|
|
425
|
-
if (
|
|
430
|
+
if (t.run !== run) continue;
|
|
426
431
|
const retry = t.member.retry ?? "";
|
|
427
432
|
const bestRetry = best?.member.retry ?? "";
|
|
428
433
|
if (best === null || retry > bestRetry
|
|
@@ -604,20 +609,21 @@ export function deriveRun({ rows, dispatched, drain }, prs, closed = new Set(),
|
|
|
604
609
|
// `fixMembers`: every fix-applier on this PR, wherever its token sits —
|
|
605
610
|
// one still unsettled anywhere holds the PR off fixDue, since `dispatch`
|
|
606
611
|
// refuses a second live one on the PR.
|
|
607
|
-
// `reviewedHead`: the latest `reviewed=<head>`.
|
|
612
|
+
// `reviewedHead`/`reviewedRun`: the latest `reviewed=<head>:…:<run>`.
|
|
613
|
+
// `pastPinHalt`: the latest
|
|
608
614
|
// finisher since that review settled `halted:past-pin` — any
|
|
609
615
|
// later finisher attempt, live or settled, replaces it, and a later
|
|
610
616
|
// returned review answers it. `unverified`: the latest `reviewed=`'s
|
|
611
617
|
// unverified count. `dispositions`: every dispositions verdict a fix-pr
|
|
612
|
-
// member of this PR carries, whatever
|
|
613
|
-
// by
|
|
618
|
+
// member of this PR carries, whatever review it answers — the gate picks
|
|
619
|
+
// by run and retry suffix, never by where a token sits.
|
|
614
620
|
// `workers`: every fix-applier and finisher bound to this PR, and every
|
|
615
621
|
// implementer on a row resolving to it — one still unsettled holds the
|
|
616
622
|
// CONFLICT line off, since it may be pushing to the branch right now.
|
|
617
623
|
const st = (pr !== null && byPr.get(pr)) || {
|
|
618
624
|
inFlight: false, reviewedAny: false, survived: 0, unverified: 0, reviewFixed: false,
|
|
619
625
|
fixMembers: new Set(), fixLanded: new Set(), conflictLanded: new Set(), held: [],
|
|
620
|
-
conflictOpen: false, reviewedHead: null, pastPinHalt: false, dispositions: [], workers: new Set(),
|
|
626
|
+
conflictOpen: false, reviewedHead: null, reviewedRun: null, pastPinHalt: false, dispositions: [], workers: new Set(),
|
|
621
627
|
};
|
|
622
628
|
// The finisher-pr token currently deciding `pastPinHalt`, picked by
|
|
623
629
|
// retry suffix ("" < "b" < "c" …) the same way compute-board.mjs's
|
|
@@ -689,11 +695,15 @@ export function deriveRun({ rows, dispatched, drain }, prs, closed = new Set(),
|
|
|
689
695
|
if (!m[1]) st.reviewedAny = true;
|
|
690
696
|
} else if (tok.startsWith("reviewed=")) {
|
|
691
697
|
const m = REVIEWED.exec(tok);
|
|
692
|
-
if (!m
|
|
698
|
+
if (!m && REVIEWED_RUNLESS.test(tok)) {
|
|
699
|
+
throw new LedgerError(`${where}: '${tok}' names no review run — reviewed= is now <head>:<survived>/<refuted>/<unverified>:<run>, `
|
|
700
|
+
+ "<run> the run-XXXXXXXX directory holding that review's review.json; rewrite the token with `ledger.mjs row`");
|
|
701
|
+
}
|
|
702
|
+
if (!m) throw new LedgerError(`${where}: '${tok}' is not reviewed=<head>:<survived>/<refuted>/<unverified>:<run>`);
|
|
693
703
|
latestFinisher = null;
|
|
694
704
|
Object.assign(st, {
|
|
695
705
|
inFlight: false, reviewedAny: true, survived: Number(m[2]), unverified: Number(m[4]), reviewFixed: false,
|
|
696
|
-
reviewedHead: m[1].toLowerCase(), pastPinHalt: false,
|
|
706
|
+
reviewedHead: m[1].toLowerCase(), reviewedRun: m[5], pastPinHalt: false,
|
|
697
707
|
});
|
|
698
708
|
} else if (labelOffMember(tok) === null) {
|
|
699
709
|
throw new LedgerError(`${where}: '${tok}' is not label-off=<finisher-pr member> — fix the row with \`ledger.mjs row\``);
|
|
@@ -788,8 +798,7 @@ export function deriveRun({ rows, dispatched, drain }, prs, closed = new Set(),
|
|
|
788
798
|
// nothing, and so did a conflict fix-applier, which is never checked. An
|
|
789
799
|
// escalate is a different verdict and is never due.
|
|
790
800
|
const mismatchDue = (st) => {
|
|
791
|
-
|
|
792
|
-
const cur = currentDispositions(st.dispositions, st.reviewedHead);
|
|
801
|
+
const cur = currentDispositions(st.dispositions, st.reviewedRun);
|
|
793
802
|
if (cur === null || cur.verdict !== "mismatch") return false;
|
|
794
803
|
const retry = parseMember(cur.member).retry ?? "";
|
|
795
804
|
return ![...st.fixLanded].some((n) => !st.conflictLanded.has(n) && (parseMember(n).retry ?? "") > retry);
|
|
@@ -863,16 +872,17 @@ export function deriveRun({ rows, dispatched, drain }, prs, closed = new Set(),
|
|
|
863
872
|
conflicts: conflicting.filter((p) => !treadmill(p)).map((p) => p.number).sort(asc),
|
|
864
873
|
conflictEscalate: conflicting.filter(treadmill).map((p) => p.number).sort(asc),
|
|
865
874
|
// Every PR, open or not, with a returned review: its latest `reviewed=`
|
|
866
|
-
// head and counts, the dispositions verdict currently answering that
|
|
867
|
-
//
|
|
868
|
-
// re-review
|
|
875
|
+
// head, run and counts, the dispositions verdict currently answering that
|
|
876
|
+
// run, and every fix-applier on the PR still unsettled (`fixLive`) — a
|
|
877
|
+
// re-review would otherwise read the earlier
|
|
869
878
|
// fix-applier's verdict while the one answering it is still working.
|
|
870
879
|
// `ledger.mjs dispatch` gates a finisher off this, read from the same
|
|
871
|
-
// per-PR fold as everything above
|
|
880
|
+
// per-PR fold as everything above, and dispositions-check.mjs reads the
|
|
881
|
+
// review file it judges against off `run`. Nothing in this tick acts on it.
|
|
872
882
|
reviewed: [...byPr.entries()].filter(([, st]) => st.reviewedHead !== null)
|
|
873
883
|
.map(([n, st]) => ({
|
|
874
|
-
pr: n, head: st.reviewedHead, survived: st.survived, unverified: st.unverified,
|
|
875
|
-
dispositions: currentDispositions(st.dispositions, st.
|
|
884
|
+
pr: n, head: st.reviewedHead, run: st.reviewedRun, survived: st.survived, unverified: st.unverified,
|
|
885
|
+
dispositions: currentDispositions(st.dispositions, st.reviewedRun),
|
|
876
886
|
fixLive: [...st.fixMembers].filter((name) => members.get(name).outcome === null).sort(),
|
|
877
887
|
}))
|
|
878
888
|
.sort((a, b) => a.pr - b.pr),
|
package/scripts/ledger.mjs
CHANGED
|
@@ -1641,30 +1641,30 @@ function refuseMalformedDispatched() {
|
|
|
1641
1641
|
function dispositionsRefusal(pr, review) {
|
|
1642
1642
|
if (review === undefined) return null;
|
|
1643
1643
|
if (review.fixLive.length > 0) {
|
|
1644
|
-
return `${review.fixLive.join(", ")} still live on PR #${pr} — a fix-applier answering review ${review.
|
|
1644
|
+
return `${review.fixLive.join(", ")} still live on PR #${pr} — a fix-applier answering review ${review.run} has not settled, `
|
|
1645
1645
|
+ "so no dispositions verdict answers its work yet; settle it, run dispositions-check.mjs for it, then dispatch the finisher again";
|
|
1646
1646
|
}
|
|
1647
1647
|
if (review.survived === 0 && review.unverified === 0) return null;
|
|
1648
1648
|
const v = review.dispositions;
|
|
1649
1649
|
if (v === null) {
|
|
1650
1650
|
return `dispositions unchecked — no dispositions-ok=/dispositions-mismatch=/dispositions-escalate=/dispositions-unchecked= token answers PR #${pr}'s latest review, `
|
|
1651
|
-
+
|
|
1651
|
+
+ `${review.run} at ${review.head}; run dispositions-check.mjs --member fix-pr-${pr}[-<x>] --scratch <scratch> for the fix-applier `
|
|
1652
1652
|
+ "that answered that review, then dispatch the finisher again";
|
|
1653
1653
|
}
|
|
1654
1654
|
if (v.verdict === "unchecked") {
|
|
1655
|
-
return `dispositions unchecked — ${v.member}'s check of review ${review.
|
|
1655
|
+
return `dispositions unchecked — ${v.member}'s check of review ${review.run} could not read where a deferral was filed (gh unreachable, `
|
|
1656
1656
|
+ "or an issue unreadable — an issue that does not exist is unreadable too, and no re-run answers that one: the check's stderr names "
|
|
1657
1657
|
+ `it, and the record's issue number needs correcting); run dispositions-check.mjs --member ${v.member} --scratch <scratch> again once gh answers, `
|
|
1658
1658
|
+ "then dispatch the finisher again";
|
|
1659
1659
|
}
|
|
1660
1660
|
if (v.verdict === "mismatch") {
|
|
1661
|
-
return `dispositions mismatch — ${v.member}'s disposition record fails the check against review ${review.
|
|
1661
|
+
return `dispositions mismatch — ${v.member}'s disposition record fails the check against review ${review.run}; `
|
|
1662
1662
|
+ `dispositions-check.mjs --member ${v.member} --scratch <scratch> names each violating entry by bucket, index and rule`;
|
|
1663
1663
|
}
|
|
1664
1664
|
if (v.verdict === "escalate") {
|
|
1665
1665
|
return `dispositions escalate — ${v.member} deferred a finding above suggestion severity because its remedy lies outside the PR's diff, `
|
|
1666
1666
|
+ "or its record drew a second mismatch on this review; either needs a human's ruling: "
|
|
1667
|
-
+ `review ${review.
|
|
1667
|
+
+ `review ${review.run} gets no finisher and no retry; dispositions-check.mjs --member ${v.member} `
|
|
1668
1668
|
+ "--scratch <scratch> names each escalated finding or violating entry by bucket and index, for the PR comment that hands it to a human";
|
|
1669
1669
|
}
|
|
1670
1670
|
return null;
|
|
@@ -1775,7 +1775,7 @@ function runDispatch() {
|
|
|
1775
1775
|
// known to conflict cannot merge, and a finisher would race the fix-applier
|
|
1776
1776
|
// rebasing it — and once its current dispositions verdict is ok — both read
|
|
1777
1777
|
// off the tick's own per-PR fold, the verdict at the same latest `reviewed=`
|
|
1778
|
-
//
|
|
1778
|
+
// run the tick reads — and no fix-applier on the PR is still live. A PR
|
|
1779
1779
|
// whose latest review counts no survived and no unverified finding had
|
|
1780
1780
|
// nothing for a fix-applier to rule on, and one never reviewed has no
|
|
1781
1781
|
// review file: neither is gated on a verdict. Refused here, before anything
|
package/scripts/review-eval.mjs
CHANGED
|
@@ -30,8 +30,8 @@
|
|
|
30
30
|
// runReview's own snapshot/verifier dispatch, unmodified — omp's `agent()`
|
|
31
31
|
// resolves a bare frontmatter `name:` exactly, which is already what those
|
|
32
32
|
// strings are.
|
|
33
|
-
import { mkdir, writeFile } from "node:fs/promises";
|
|
34
|
-
import { isAbsolute, join } from "node:path";
|
|
33
|
+
import { mkdir, unlink, writeFile } from "node:fs/promises";
|
|
34
|
+
import { basename, dirname, isAbsolute, join, normalize } from "node:path";
|
|
35
35
|
import { digestOf, runReview, runnerPrRefusal, runnerScratchRefusal } from "./review-core.mjs";
|
|
36
36
|
|
|
37
37
|
// eval's `agent()` returns a HANDLE, not data: `agent(prompt,
|
|
@@ -139,10 +139,17 @@ export async function runReviewOnOmp(args) {
|
|
|
139
139
|
|
|
140
140
|
// The whole of the omp review runner's job, so the agent's own cell is three
|
|
141
141
|
// lines and this contract can be run by a test: the full result
|
|
142
|
-
// object goes to `<
|
|
142
|
+
// object goes to `<run root>/review.json` — the one artefact handed to
|
|
143
143
|
// the fix-applier — and only the digest comes back, because the digest is
|
|
144
144
|
// all the controller reads and a 26–61 KB result is what the file exists to
|
|
145
|
-
// keep out of its context.
|
|
145
|
+
// keep out of its context. The run root is the review's own
|
|
146
|
+
// `<scratch>/pr<pr>/run-XXXXXXXX`, the directory review-core.mjs's snapshot
|
|
147
|
+
// step made with `mktemp` and the snapshot it returns sits in, so two reviews
|
|
148
|
+
// of one PR — at one head or two — never write one file, and the ledger token
|
|
149
|
+
// names the run whose file it is. No file is ever replaced. The file leaves
|
|
150
|
+
// with its run root: the snapshot step of a later review of the same PR
|
|
151
|
+
// removes that PR's run roots not touched for seven days, so a token older
|
|
152
|
+
// than that names a file that is gone.
|
|
146
153
|
//
|
|
147
154
|
// Failure is a throw or an empty return. The holder of the review call retries
|
|
148
155
|
// once; a second failure returns `failed` with both errors and the fallback
|
|
@@ -151,8 +158,9 @@ export async function runReviewOnOmp(args) {
|
|
|
151
158
|
// scratch that is not absolute (eval's cwd is the MAIN CHECKOUT, so a relative
|
|
152
159
|
// one would put the file there), a scratch that already ends in `pr<N>`
|
|
153
160
|
// — throws before any run: it is not a review failure, and retrying or falling
|
|
154
|
-
// back would only repeat it.
|
|
155
|
-
//
|
|
161
|
+
// back would only repeat it. So does a write fault other than a file already
|
|
162
|
+
// at the path: the review ran, the environment could not keep it. `run` is the
|
|
163
|
+
// seam the test injects; nothing else passes it.
|
|
156
164
|
export async function runReviewToFile(args, run = runReviewOnOmp) {
|
|
157
165
|
const pr = args?.pr;
|
|
158
166
|
const scratch = args?.scratch;
|
|
@@ -177,7 +185,7 @@ export async function runReviewToFile(args, run = runReviewOnOmp) {
|
|
|
177
185
|
// the file on disk — so a `run` that resolves to a truthy, object-shaped
|
|
178
186
|
// result missing `.counts` would otherwise throw on the destructure below
|
|
179
187
|
// OUTSIDE this function's own retry try/catch (which wraps only the
|
|
180
|
-
// `run(args)` call), leaving a half-written
|
|
188
|
+
// `run(args)` call), leaving a half-written review file on disk
|
|
181
189
|
// that a fix-applier reading it would mistake for a completed review that
|
|
182
190
|
// never actually finished. Treated the same as an empty return: retried,
|
|
183
191
|
// then reported `failed`, and no file is written for it.
|
|
@@ -185,15 +193,44 @@ export async function runReviewToFile(args, run = runReviewOnOmp) {
|
|
|
185
193
|
errors.push(`attempt ${attempt}: empty return (${String(result)})`);
|
|
186
194
|
continue;
|
|
187
195
|
}
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
196
|
+
// The run root is read off the result, never rebuilt: only the snapshot
|
|
197
|
+
// step's shell knew the `mktemp` name. A result whose snapshot is not in
|
|
198
|
+
// a run root of this PR under this scratch names no run the ledger token
|
|
199
|
+
// could point a reader back to, so it is a failed attempt like the one
|
|
200
|
+
// above.
|
|
201
|
+
const runRoot = runRootOf(result.snapshot, scratch, pr);
|
|
202
|
+
if (runRoot === null) {
|
|
203
|
+
errors.push(`attempt ${attempt}: the result's snapshot ${JSON.stringify(result.snapshot)} is not in a run root `
|
|
204
|
+
+ `${join(scratch, `pr${pr}`, "run-XXXXXXXX")} — no run the review file could be kept in`);
|
|
205
|
+
continue;
|
|
206
|
+
}
|
|
207
|
+
const path = join(runRoot, "review.json");
|
|
208
|
+
try {
|
|
209
|
+
await mkdir(runRoot, { recursive: true });
|
|
210
|
+
// `wx`: an existing file at this path is another review's result, and
|
|
211
|
+
// stays.
|
|
212
|
+
await writeFile(path, `${JSON.stringify(result, null, 2)}\n`, { flag: "wx" });
|
|
213
|
+
} catch (e) {
|
|
214
|
+
// Only that collision is a failed attempt: a second review run lands in
|
|
215
|
+
// a root of its own. Any other fault — permissions, a full disk — is
|
|
216
|
+
// the environment's, so it throws rather than re-run a full review that
|
|
217
|
+
// would hit it again and report it as a review failure. A write that
|
|
218
|
+
// fails partway has left its truncated bytes at the path: removed
|
|
219
|
+
// before the throw, never on a collision, where the file is another
|
|
220
|
+
// review's.
|
|
221
|
+
if (e?.code !== "EEXIST") {
|
|
222
|
+
await unlink(path).catch(() => {});
|
|
223
|
+
throw e;
|
|
224
|
+
}
|
|
225
|
+
errors.push(`attempt ${attempt}: could not write ${path}: ${e?.message ?? String(e)}`);
|
|
226
|
+
continue;
|
|
227
|
+
}
|
|
191
228
|
const { survived, refuted, unverified } = result.counts;
|
|
192
229
|
return {
|
|
193
230
|
status: "completed",
|
|
194
231
|
path,
|
|
195
232
|
// The review's result token, ready for `ledger.mjs row`.
|
|
196
|
-
ledger: `reviewed=${result.head}:${survived}/${refuted}/${unverified}`,
|
|
233
|
+
ledger: `reviewed=${result.head}:${survived}/${refuted}/${unverified}:${basename(runRoot)}`,
|
|
197
234
|
attempts: attempt,
|
|
198
235
|
errors,
|
|
199
236
|
digest: digestOf(result),
|
|
@@ -201,3 +238,14 @@ export async function runReviewToFile(args, run = runReviewOnOmp) {
|
|
|
201
238
|
}
|
|
202
239
|
return { status: "failed", errors, fallback: `review-pr-${pr}-b` };
|
|
203
240
|
}
|
|
241
|
+
|
|
242
|
+
// The run root a result's `snapshot` sits in — `<scratch>/pr<pr>/run-` and
|
|
243
|
+
// the eight characters `mktemp` put in place of `XXXXXXXX` — or null when it
|
|
244
|
+
// sits anywhere else. A `..` segment is refused before normalising, which
|
|
245
|
+
// would otherwise resolve it into a root the path never named.
|
|
246
|
+
function runRootOf(snapshot, scratch, pr) {
|
|
247
|
+
if (typeof snapshot !== "string" || !isAbsolute(snapshot) || snapshot.split("/").includes("..")) return null;
|
|
248
|
+
const runRoot = dirname(normalize(snapshot));
|
|
249
|
+
if (dirname(runRoot) !== join(scratch, `pr${pr}`) || !/^run-[A-Za-z0-9]{8}$/.test(basename(runRoot))) return null;
|
|
250
|
+
return runRoot;
|
|
251
|
+
}
|
package/skills/run-team/SKILL.md
CHANGED
|
@@ -1536,8 +1536,8 @@ for a token on a ticket's line — never a hand edit.
|
|
|
1536
1536
|
| Wake | Record, then tick |
|
|
1537
1537
|
|---|---|
|
|
1538
1538
|
| Implementer report | `verify-sha.sh`; `ledger.mjs settle impl-<N>=PR#<M>`, or `=bailed` and relabel by cause (**Implementer bails before implementing**, below) |
|
|
1539
|
-
| Review workflow notification / `review-pr-<n>` report |
|
|
1540
|
-
| Fix-applier report | `ledger.mjs settle fix-pr-<M>=…`; for a fix-applier that answered a review — not one that cleared a conflict hold, which has no review file — `~/.fleet/bin/fleet-run dispositions-check.mjs --member <that member> --scratch <scratch>`, from the checkout root: it judges `<scratch>/dispositions-<
|
|
1539
|
+
| Review workflow notification / `review-pr-<n>` report | the result file is `<scratch>/pr<pr>/<run>/review.json`, `<run>` the review's own `run-XXXXXXXX` run root; `reviewed=<head>:<survived>/<refuted>/<unverified>:<run>` on the PR's row (**Reviewers**) |
|
|
1540
|
+
| Fix-applier report | `ledger.mjs settle fix-pr-<M>=…`; for a fix-applier that answered a review — not one that cleared a conflict hold, which has no review file — `~/.fleet/bin/fleet-run dispositions-check.mjs --member <that member> --scratch <scratch>`, from the checkout root: it judges that member's own record `<scratch>/dispositions-<member>.json` against the review file the PR's latest `reviewed=` names, `<scratch>/pr<M>/<run>/review.json`, writes `dispositions-ok=`, `dispositions-mismatch=`, `dispositions-escalate=` or `dispositions-unchecked=<member>:<run>` onto the PR's row itself, and exits 1 on a mismatch, an escalation or an unchecked, naming each violating or escalated entry's bucket and index, a violation its rule too, and each entry whose filed issue `gh` could not read — **Then dispatch a fix-applier** says what a mismatch asks of you, the gate paragraph below what an escalation or an unchecked does; copy the refutations it reversed — the record's `refuted` entries — to `ruled`; `ledger.mjs filed <N> "<subject>"` for each `unrecorded:` line |
|
|
1541
1541
|
| Finisher report | `ledger.mjs settle finisher-pr-<M>=labelled` as reported, even when its read-back lacks `ready-to-merge` — the tick's `DISPATCH finisher PR#<M>` catches that next. A **repair** finisher's (one sent on that line) read-back lacking it also gets `gh pr comment <M>` with both finishers' read-backs: the one-shot escalation, where halts comment. `ledger.mjs filed <N> "<subject>"` for each `unrecorded:` line |
|
|
1542
1542
|
| Finisher report (halted) | `ledger.mjs settle finisher-pr-<M>=halted:<cause>`; `gh pr comment <M>` with the finisher's halt report, cause and evidence; `ledger.mjs filed <N> "<subject>"` for each `unrecorded:` line; then the per-cause rule (**Resolving a finisher halt**, below) |
|
|
1543
1543
|
| Finisher report (failed / killed) | `ledger.mjs settle finisher-pr-<M>=failed` for a finisher that crashed or gave up, `=killed` for one that was killed; no label, and the tick prints no dispatch for it. The cockpit flags the PR `finisher:failed` / `finisher:killed` at severity 4 and you resolve it by hand (**Resolving a finisher that died**, below) |
|
|
@@ -2109,7 +2109,7 @@ report recovery, never the exit code alone.
|
|
|
2109
2109
|
The tick names every PR owed one — `DISPATCH review PR#<M>`, oldest first —
|
|
2110
2110
|
and each gets one review, once per PR:
|
|
2111
2111
|
|
|
2112
|
-
a `task` member named `review-pr-<pr#>`, agent `fleet-review-runner`, its prompt the five args `pr`, `branch`, `worktree`, `testCmd` and `scratch` — it loads `review-eval.mjs` through the Resolver (`~/.fleet/bin/fleet-run --path review-eval.mjs`), awaits `runReviewOnOmp` in its own kernel, writes the result file and reports; record `review=member:review-pr-<pr#>` on the PR's row before the dispatch call. **`scratch` is the scratch root itself, never the review side's `<scratch>/pr<N>` partition:** the review creates `pr<N>/` under it, so a `pr<N>` passed in nests every run root at `pr<N>/pr<N>/`, and the review refuses a `scratch` whose last component is `pr` plus digits before dispatching anything.
|
|
2112
|
+
a `task` member named `review-pr-<pr#>`, agent `fleet-review-runner`, its prompt the five args `pr`, `branch`, `worktree`, `testCmd` and `scratch` — it loads `review-eval.mjs` through the Resolver (`~/.fleet/bin/fleet-run --path review-eval.mjs`), awaits `runReviewOnOmp` in its own kernel, writes the result file into the review's own run root and reports its path and `reviewed=` token; record `review=member:review-pr-<pr#>` on the PR's row before the dispatch call. **`scratch` is the scratch root itself, never the review side's `<scratch>/pr<N>` partition:** the review creates `pr<N>/` under it, so a `pr<N>` passed in nests every run root at `pr<N>/pr<N>/`, and the review refuses a `scratch` whose last component is `pr` plus digits before dispatching anything.
|
|
2113
2113
|
|
|
2114
2114
|
Write the `review=` token with `ledger.mjs row`, which **replaces the whole
|
|
2115
2115
|
line**, so carry every other field. The tick counts in-flight reviews off the
|
|
@@ -2169,13 +2169,19 @@ slots a review does not hold still serve fix-appliers.
|
|
|
2169
2169
|
|
|
2170
2170
|
It returns `{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts, snapshot, survived, refuted, unverified}` —
|
|
2171
2171
|
the **digest** first, then the snapshot and the findings — and it lands as
|
|
2172
|
-
one artefact, `<scratch>/review
|
|
2173
|
-
|
|
2172
|
+
one artefact, `<scratch>/pr<pr>/<run>/review.json`, holding that bare object.
|
|
2173
|
+
`<run>` is the review's own `run-XXXXXXXX` run root, the one its snapshot sits
|
|
2174
|
+
in, so a second review of the PR — at the same head or another — writes a file
|
|
2175
|
+
of its own and never replaces the first. The `review-pr-<pr#>` runner writes it
|
|
2176
|
+
and reports the digest, the path and the `reviewed=` token naming that run.
|
|
2174
2177
|
|
|
2175
2178
|
**You read the digest; the findings are the fix-applier's.**
|
|
2176
|
-
`jq '{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts}' <scratch>/review
|
|
2177
|
-
is the whole of your read, and it is what you record: `reviewed=<head>:<survived>/<refuted>/<unverified>`,
|
|
2178
|
-
off `head
|
|
2179
|
+
`jq '{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts}' <scratch>/pr<pr>/<run>/review.json`
|
|
2180
|
+
is the whole of your read, and it is what you record: `reviewed=<head>:<survived>/<refuted>/<unverified>:<run>`,
|
|
2181
|
+
off `head`, `counts` and the run root's name — the runner's `ledger` field — on
|
|
2182
|
+
the PR's row. Every later path to the review is built from that token, before a
|
|
2183
|
+
restart or after one: `<scratch>/pr<pr>/<run>/review.json`, `<run>` off the PR's
|
|
2184
|
+
latest `reviewed=`. Findings never enter your context —
|
|
2179
2185
|
a result runs 26-61 KB, 7-15k tokens that used to land in yours and then again,
|
|
2180
2186
|
up to ~10k characters, in every fix-applier prompt you wrote. The fix-applier
|
|
2181
2187
|
reads every one off the file and makes every ruling on them
|
|
@@ -2258,8 +2264,8 @@ named member per PR, `fix-pr-<pr#>` (the next suffix when that name is already
|
|
|
2258
2264
|
on record), never the PR's
|
|
2259
2265
|
implementer, recorded with `ledger.mjs dispatch <pr#> fix-pr-<pr#>` before the
|
|
2260
2266
|
call. Its prompt carries the PR number, the worktree abs path, the PR's branch,
|
|
2261
|
-
the same `testCmd` you passed the review, and the **path** `<scratch>/review
|
|
2262
|
-
— never the findings. The fix-applier owns every per-PR ruling you once made
|
|
2267
|
+
the same `testCmd` you passed the review, and the **path** `<scratch>/pr<pr>/<run>/review.json`,
|
|
2268
|
+
`<run>` off the PR's latest `reviewed=` — never the findings. The fix-applier owns every per-PR ruling you once made
|
|
2263
2269
|
before dispatch: both mutual-exclusion scans, the suggested-fix re-derivation,
|
|
2264
2270
|
`refuted=false` ≠ apply, per-site measurement for a sibling-site extension, and
|
|
2265
2271
|
`testEnvironment`/`cwdAudit` before any result of the shared test run. It
|
|
@@ -2293,17 +2299,17 @@ bot refuses it `head-moved-after-label-#<M>`: a fresh finisher, per **Failure
|
|
|
2293
2299
|
handling**.
|
|
2294
2300
|
|
|
2295
2301
|
**A `DISPATCH fix-pr PR#<M>` on a dispositions mismatch** — the PR's row carries
|
|
2296
|
-
`dispositions-mismatch=<member>:<
|
|
2302
|
+
`dispositions-mismatch=<member>:<run>` for its latest review's run, and no later
|
|
2297
2303
|
fix-applier has landed — is one automatic retry, a review fix-applier named the
|
|
2298
2304
|
next suffix (`fix-pr-<M>-b`) and dispatched like the first. Its prompt is the
|
|
2299
|
-
first one's, with
|
|
2300
|
-
`<scratch>/review
|
|
2301
|
-
it
|
|
2305
|
+
first one's, with three paths and a list copied in: the review file
|
|
2306
|
+
`<scratch>/pr<M>/<run>/review.json`, the mismatched member's record `<scratch>/dispositions-<member>.json`
|
|
2307
|
+
it starts from, its own record `<scratch>/dispositions-<its own name>.json` it writes, and the check's violation list **verbatim** — the lines
|
|
2302
2308
|
`dispositions-check.mjs` printed, each naming a bucket, an index and a rule;
|
|
2303
|
-
re-run the check for the mismatched member to reprint them. It
|
|
2309
|
+
re-run the check for the mismatched member to reprint them. It writes its own
|
|
2304
2310
|
record, and when it reports you run the check for it exactly as for the first
|
|
2305
2311
|
one. `ok` ends it. A second failure on the same review is written as
|
|
2306
|
-
`dispositions-escalate=fix-pr-<M>-b:<
|
|
2312
|
+
`dispositions-escalate=fix-pr-<M>-b:<run>` in place of a mismatch: the tick
|
|
2307
2313
|
prints no `DISPATCH fix-pr` for it, `dispatch` refuses the finisher naming
|
|
2308
2314
|
`dispositions escalate`, and you post the check's output with `gh pr comment <M>`
|
|
2309
2315
|
and flag the PR for a human. A new review of the PR counts again from none.
|
|
@@ -2378,7 +2384,7 @@ all.
|
|
|
2378
2384
|
> review already ran — and steps 4 and 6: the controller owns the CI wait and
|
|
2379
2385
|
> dispatches the finisher.
|
|
2380
2386
|
>
|
|
2381
|
-
> **The review's result is `<scratch>/review
|
|
2387
|
+
> **The review's result is `<scratch>/pr<pr>/<run>/review.json`** — the digest first
|
|
2382
2388
|
> (`jq '{pr, head, resume, testEnvironment, dimensionsRun, dimensionsUnrun, cwdAudit, counts}'`),
|
|
2383
2389
|
> then every finding (`jq '.survived'`, `jq '.unverified'`, `jq '.refuted'`).
|
|
2384
2390
|
> Read it through `review-and-fix.md`'s **The review result file**, which holds
|
|
@@ -2583,10 +2589,13 @@ all.
|
|
|
2583
2589
|
> that was two commits stale, and a finisher dispatched on either would have
|
|
2584
2590
|
> halted on a diverged head.
|
|
2585
2591
|
>
|
|
2586
|
-
> **Before you report, write every ruling to `<scratch>/dispositions-<
|
|
2587
|
-
>
|
|
2592
|
+
> **Before you report, write every ruling to `<scratch>/dispositions-<your name>.json`**,
|
|
2593
|
+
> `<your name>` the `fix-pr-<pr>[-x]` you were dispatched as — the record the controller checks with
|
|
2588
2594
|
> `dispositions-check.mjs` before it dispatches any finisher. It is
|
|
2589
|
-
> `{"head": "<the review file's head>", "entries": [ … ]}
|
|
2595
|
+
> `{"head": "<the review file's head>", "run": "<the run root's name in the review path>", "entries": [ … ]}`
|
|
2596
|
+
> — `run` the `<run>` of the review path above, which the check holds against the
|
|
2597
|
+
> review the PR's latest `reviewed=` names: a record written against an earlier
|
|
2598
|
+
> review, a same-head one included, is a mismatch. One entry per
|
|
2590
2599
|
> `survived` finding and per `unverified` finding, plus one per `refuted`
|
|
2591
2600
|
> finding you reverse. Each entry carries `bucket`
|
|
2592
2601
|
> (`survived|unverified|refuted`), `index` (its position in that bucket of the
|
|
@@ -2658,14 +2667,15 @@ and will act on it. Relay everything — reversals too — then dispatch.
|
|
|
2658
2667
|
|
|
2659
2668
|
**That `dispatch` refuses a finisher — exit 2, nothing written — on a PR whose
|
|
2660
2669
|
latest `reviewed=` counts a survived or unverified finding, unless the
|
|
2661
|
-
dispositions verdict answering that review's
|
|
2662
|
-
`dispositions-*=fix-pr-<M>[-x]:<
|
|
2663
|
-
from the highest-suffixed fix-applier
|
|
2670
|
+
dispositions verdict answering that review's run is `ok`**: among the
|
|
2671
|
+
`dispositions-*=fix-pr-<M>[-x]:<run>` tokens naming its run, the one
|
|
2672
|
+
from the highest-suffixed fix-applier — so a second review at the same head is
|
|
2673
|
+
answered by none of the first one's verdicts. A PR whose latest review counts
|
|
2664
2674
|
`0/<n>/0` is not gated. One counting `0/<n>/<u>` with `<u>` above zero is
|
|
2665
2675
|
gated like survivors, and the tick prints `DISPATCH fix-pr PR#<M>` for it: that
|
|
2666
2676
|
fix-applier's `no-op` and its `ok` verdict are what open the finisher. The
|
|
2667
2677
|
refusal names its cause. `dispositions
|
|
2668
|
-
unchecked` — no verdict answers that
|
|
2678
|
+
unchecked` — no verdict answers that review: run `dispositions-check.mjs` for
|
|
2669
2679
|
the fix-applier that answered the review, as the Fix-applier report edge
|
|
2670
2680
|
states, then dispatch again. It names a `dispositions-unchecked=` verdict too —
|
|
2671
2681
|
no rule broke, but the check could not read where a deferral was filed, `gh`
|
|
@@ -2681,15 +2691,14 @@ deferred `remedy-outside-diff`, or a second mismatch was drawn on one review, an
|
|
|
2681
2691
|
only a human can rule on either. Post the check's output, which names each
|
|
2682
2692
|
escalated finding or violating entry, with `gh pr comment <M>`, and flag the PR
|
|
2683
2693
|
for a human; re-running the check reprints it. Dispatch no fix-applier and no
|
|
2684
|
-
finisher for that review
|
|
2694
|
+
finisher for that review, and expect no `fix-due` row for it; a new
|
|
2685
2695
|
review of the PR starts the check again. **On any PR with a
|
|
2686
2696
|
returned review, `0/<n>/0` included, it refuses too while a fix-applier on the
|
|
2687
2697
|
PR is still live, verdict or not** —
|
|
2688
2698
|
`fix-pr-<M>[-x] still live`: settle it, run the check for it, then dispatch
|
|
2689
|
-
again.
|
|
2690
|
-
|
|
2691
|
-
|
|
2692
|
-
the current one.
|
|
2699
|
+
again. A re-review at the same head is a run of its own, so the earlier
|
|
2700
|
+
round's verdict never answers it; once the check has run for the fix-applier
|
|
2701
|
+
answering the new round, its verdict is the current one.
|
|
2693
2702
|
|
|
2694
2703
|
**It refuses a finisher, too, while the PR sits on a conflict hold no
|
|
2695
2704
|
fix-applier has cleared** — the same per-PR fold the tick prints `DISPATCH
|
|
@@ -3128,8 +3137,12 @@ guaranteed the way skill availability is. Tell it to check every added
|
|
|
3128
3137
|
assertion against the tree, including comments in files the diff does not
|
|
3129
3138
|
touch, and to read each corrected sentence literally, clause by clause. It then
|
|
3130
3139
|
does the fix-applier's job too:
|
|
3131
|
-
apply, defer, file, push, report, exit
|
|
3132
|
-
|
|
3140
|
+
apply, defer, file, push, report, exit — and before it reports, it writes its
|
|
3141
|
+
findings as a review result file in a run root of its own, made with
|
|
3142
|
+
`mktemp -d "<scratch>/pr<pr>/run-XXXXXXXX"`, in the shape `review-and-fix.md`'s
|
|
3143
|
+
**The review result file** describes, and reports that file's path. On its
|
|
3144
|
+
report, record `reviewed=<head>:0/<refuted>/<deferred>:<run>`, `<run>` that run
|
|
3145
|
+
root's name, against the head it pushed — it applied
|
|
3133
3146
|
its own survivors, so the zero is what tells the tick no fix-applier is owed for
|
|
3134
3147
|
them; a nonzero `<deferred>` is still fix-due, as any review's unverified band is —
|
|
3135
3148
|
and it reaches a finisher through the same gate as any other PR.
|
|
@@ -3874,8 +3887,9 @@ any other, never a section of its own.
|
|
|
3874
3887
|
A PR's review is not a member, so it carries its own pair of row tokens,
|
|
3875
3888
|
written with `row` (**Reviewers**): `review=wf:<runId>` |
|
|
3876
3889
|
`review=member:review-pr-<n>` | `review=fallback:review-pr-<n>[-b]` at launch,
|
|
3877
|
-
settled dead as `…=failed`, then `reviewed=<head>:<survived>/<refuted>/<unverified>`
|
|
3878
|
-
when the result lands
|
|
3890
|
+
settled dead as `…=failed`, then `reviewed=<head>:<survived>/<refuted>/<unverified>:<run>`
|
|
3891
|
+
when the result lands, `<run>` the review's `run-XXXXXXXX` run root under
|
|
3892
|
+
`<scratch>/pr<n>/`, which holds its `review.json`. A row with `review=` and no `reviewed=` is a review in
|
|
3879
3893
|
flight, and the tick counts it against the reviewer cap unless gh reports its PR
|
|
3880
3894
|
MERGED or CLOSED. `ci=<run-id>:<attempt>:<conclusion>`
|
|
3881
3895
|
and `held-behind:#<lower>` are row tokens the same way (Phase 3), and so is
|