rungate 0.8.3 → 0.8.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/gates/.gate-salt +1 -1
- package/hooks/BunTestPathGuard.hook.ts +106 -0
- package/hooks/TopLevelCdGuard.hook.ts +38 -6
- package/lib/ai-review-suppression.ts +295 -0
- package/lib/bun-test-paths.ts +253 -0
- package/lib/conformity.ts +9 -5
- package/lib/finding-jury.ts +20 -4
- package/lib/gate-check-audit.ts +535 -148
- package/lib/organize.ts +18 -8
- package/lib/release-preflight.ts +90 -1
- package/lib/report-provenance.ts +153 -0
- package/lib/scaffold/ai-review-findings.ts +485 -28
- package/lib/scaffold/ai-review-step.ts +384 -9
- package/lib/scaffold/steps.ts +26 -9
- package/lib/spec-updater.ts +11 -7
- package/lib/spec-write-ast.ts +728 -0
- package/lib/spec-write-sweep.ts +470 -0
- package/lib/test-suite-lock.ts +127 -13
- package/lib/ts-ast.ts +26 -0
- package/lib/validators/spec-validators.ts +15 -12
- package/lib/write-spec.ts +138 -0
- package/package.json +1 -1
- package/scripts/ai-review-findings.ts +45 -2
- package/scripts/audit-specs.ts +8 -9
- package/scripts/convert-spec.ts +7 -2
- package/scripts/create-sc.ts +7 -6
- package/scripts/create-spec.ts +5 -2
- package/scripts/split-spec.ts +9 -8
- package/scripts/sync-sc-status.ts +7 -6
|
@@ -0,0 +1,253 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* bun-test-paths.ts — the decision behind BunTestPathGuard (#1544)
|
|
3
|
+
*
|
|
4
|
+
* One question: does this Bash command hand `bun test` a path that matches
|
|
5
|
+
* no file?
|
|
6
|
+
*
|
|
7
|
+
* ## What bun does, measured in this tree on bun 1.3.14
|
|
8
|
+
*
|
|
9
|
+
* | command | exit | summary line |
|
|
10
|
+
* |---|---|---|
|
|
11
|
+
* | `bun test <absent>` | 1 | no test file matched at all |
|
|
12
|
+
* | `bun test <real> <absent>` | **0** | `Ran 4 tests across 1 file.` |
|
|
13
|
+
* | `bun test <real> ./<absent>` | **0** | nothing about the absent path |
|
|
14
|
+
*
|
|
15
|
+
* A path argument that matches no file is treated as a name FILTER. Bun
|
|
16
|
+
* exits non-zero only when the whole filter set matched zero files; if at
|
|
17
|
+
* least one path matched, every unmatched argument is silent and the run
|
|
18
|
+
* exits 0. So a typo'd path alongside real ones is indistinguishable from a
|
|
19
|
+
* suite that ran and passed — which is `.claude/rules/checks-must-be-able-to-fail.md`'s
|
|
20
|
+
* sixth shape, *green on the exit code, witnessing the wrong mechanism*, in
|
|
21
|
+
* the instrument every agent in this repo verifies its own work with.
|
|
22
|
+
*
|
|
23
|
+
* The live instance: on 2026-10-24, shipping #1536, a verification command
|
|
24
|
+
* named `test/sc-reservations.test.ts`. The real file is
|
|
25
|
+
* `test/sc-reservation-store.test.ts`. Five other paths ran, the command
|
|
26
|
+
* exited 0 reporting `181 pass, 0 fail`, and the suite that would have
|
|
27
|
+
* caught the defect was never opened. CI found it afterwards, on
|
|
28
|
+
* `test-shard (3)`.
|
|
29
|
+
*
|
|
30
|
+
* ## Why a pre-flight stat and not the summary line
|
|
31
|
+
*
|
|
32
|
+
* `Ran N tests across M files` is the only output that distinguishes the two
|
|
33
|
+
* cases — M must equal the number of path arguments. It is a sound reading
|
|
34
|
+
* and it is POST-HOC: it needs the run to have happened and the reader to
|
|
35
|
+
* have thought to look. This refuses before bun is invoked, costs one
|
|
36
|
+
* `existsSync` per argument, and does not depend on anybody remembering.
|
|
37
|
+
*
|
|
38
|
+
* ## Why a hook and not a wrapper script
|
|
39
|
+
*
|
|
40
|
+
* Measured on this tree: 55 sites execute `bun test` with a path argument,
|
|
41
|
+
* 49 of them inside `test/`. A wrapper covers 0 of them on the day it lands,
|
|
42
|
+
* and covers an agent typing `bun test …` by hand — the case that produced
|
|
43
|
+
* this issue — never. The Bash tool is the one chokepoint every agent-composed
|
|
44
|
+
* command passes through.
|
|
45
|
+
*
|
|
46
|
+
* ## The direction this is allowed to be wrong in
|
|
47
|
+
*
|
|
48
|
+
* Toward ALLOWING. The guard fires on every `bun test` an agent runs, so a
|
|
49
|
+
* false refusal is a wall in front of correct work, repeated. Three classes
|
|
50
|
+
* of argument are therefore skipped rather than resolved, because the hook
|
|
51
|
+
* sees the command BEFORE the shell expands it:
|
|
52
|
+
*
|
|
53
|
+
* - a glob (`test/*.test.ts`) — the shell expands it, bun never sees the
|
|
54
|
+
* metacharacters, and `existsSync` on the unexpanded string is always false
|
|
55
|
+
* - a variable or substitution (`$SUITE`, `$(…)`, backticks) — the hook has
|
|
56
|
+
* no access to shell state
|
|
57
|
+
* - anything that is not path-shaped — bun treats a bare word as a substring
|
|
58
|
+
* name filter and that is a legitimate thing to type
|
|
59
|
+
*
|
|
60
|
+
* ## WHAT THIS DOES NOT COVER
|
|
61
|
+
*
|
|
62
|
+
* Asserted as cases in `test/bun-test-path-guard.test.ts`, not merely listed,
|
|
63
|
+
* so a later widening goes red rather than silently changing behaviour.
|
|
64
|
+
*
|
|
65
|
+
* - A spawn from inside an already-running process. The 49 `test/` sites and
|
|
66
|
+
* `scripts/test-clean-env.ts` never reach the Bash tool, so no PreToolUse
|
|
67
|
+
* hook can see them. `suitePathArguments` is exported for a caller that
|
|
68
|
+
* wants to check its own arguments; nothing is forced to.
|
|
69
|
+
* - `bun test --cwd <dir> <path>`. The paths resolve against `--cwd`, not the
|
|
70
|
+
* tool call's cwd. `--cwd` is a flag and is skipped, so its paths are
|
|
71
|
+
* resolved against the wrong root and may read as absent — a false refusal,
|
|
72
|
+
* which is the direction above, and not one this has seen in the tree.
|
|
73
|
+
* - A path that exists but contains no test bun will collect. Present on
|
|
74
|
+
* disk is the whole reading; bun's own collection rules are not modelled.
|
|
75
|
+
*/
|
|
76
|
+
import { existsSync } from "fs";
|
|
77
|
+
import { isAbsolute, resolve } from "path";
|
|
78
|
+
|
|
79
|
+
import { suitePathArgumentsBySegment } from "./test-suite-lock";
|
|
80
|
+
|
|
81
|
+
/**
|
|
82
|
+
* The field the hook protocol reads for a refusal.
|
|
83
|
+
*
|
|
84
|
+
* Declared ONCE and exported, so the refusal has a single mutable site and
|
|
85
|
+
* `test/bun-test-path-guard.test.ts` can remove it and observe the guard
|
|
86
|
+
* stop refusing. A second inlined `'block'` would bypass the mutation and
|
|
87
|
+
* the case would go green having witnessed nothing.
|
|
88
|
+
*/
|
|
89
|
+
export const BUN_TEST_PATH_GUARD_DECISION = "block";
|
|
90
|
+
|
|
91
|
+
/**
|
|
92
|
+
* Shell syntax that makes a whole SEGMENT unresolvable.
|
|
93
|
+
*
|
|
94
|
+
* A command substitution spans several whitespace tokens — `$(bun scripts/x.ts)`
|
|
95
|
+
* tokenises to `$(bun` and `scripts/x.ts)` — so the second looks like an
|
|
96
|
+
* ordinary path and resolves to nothing. Judged per segment for that reason,
|
|
97
|
+
* not per argument. This was a live false refusal in the first implementation.
|
|
98
|
+
*/
|
|
99
|
+
const UNRESOLVABLE_SEGMENT = /\$\(|`|\$\{/;
|
|
100
|
+
|
|
101
|
+
/** Shell syntax that makes a single ARGUMENT unresolvable. */
|
|
102
|
+
const UNRESOLVABLE_ARGUMENT = /[*?\[\]$~]/;
|
|
103
|
+
|
|
104
|
+
/**
|
|
105
|
+
* A segment whose first command is `cd <target>`, behind any number of
|
|
106
|
+
* opening subshells. `cd /x && …`, `( cd /x && … )` and `( ( cd /x && … ) )`
|
|
107
|
+
* all match.
|
|
108
|
+
*
|
|
109
|
+
* The repetition is not speculative. A single optional `(` lost the root on
|
|
110
|
+
* `( ( cd X && bun test a.test.ts ) )` and refused a correct command — found
|
|
111
|
+
* by a 16-row battery over subshell spellings, which is the only row of the
|
|
112
|
+
* sixteen that was wrong, and not one that would have been hand-written.
|
|
113
|
+
*/
|
|
114
|
+
const LEADING_CD = /^\s*(?:\(\s*)*cd\s+(\S+)/;
|
|
115
|
+
|
|
116
|
+
/**
|
|
117
|
+
* Every directory `bun test` might run in: the tool call's cwd, plus every
|
|
118
|
+
* `cd` target the command names. Null when a `cd` target cannot be resolved,
|
|
119
|
+
* because then the set cannot be enumerated and nothing can be said.
|
|
120
|
+
*
|
|
121
|
+
* ## Why a SET of roots and not the one true cwd
|
|
122
|
+
*
|
|
123
|
+
* Raised by the AI review on PR #1562 and it was right in both directions.
|
|
124
|
+
* `splitTopLevel` does not track parenthesis depth, and this project mandates
|
|
125
|
+
* `( cd <dir> && … )` for any command working outside the session's own
|
|
126
|
+
* directory — so the first implementation resolved against the session cwd
|
|
127
|
+
* while bun would run in the cd'd tree. Measured: it REFUSED
|
|
128
|
+
* `( cd <target> && bun test test/real.test.ts )` naming a path that exists
|
|
129
|
+
* exactly where bun would have looked, and ALLOWED a path absent there but
|
|
130
|
+
* present in the session cwd.
|
|
131
|
+
*
|
|
132
|
+
* Modelling subshell scoping properly needs a depth counter, and a depth
|
|
133
|
+
* counter is a detector the next spelling defeats — the second shape in
|
|
134
|
+
* `.claude/rules/checks-must-be-able-to-fail.md`. Refusing only when a path
|
|
135
|
+
* is absent under EVERY candidate root makes the scoping question stop
|
|
136
|
+
* mattering instead of answering it: a typo is absent under all of them, and
|
|
137
|
+
* a correct path exists under at least one.
|
|
138
|
+
*
|
|
139
|
+
* The cost is declared rather than hidden: a path that exists under one root
|
|
140
|
+
* and not under the one bun actually uses is ALLOWED. That is the direction
|
|
141
|
+
* this guard is allowed to be wrong in, it is strictly better than the
|
|
142
|
+
* single-root reading it replaces, and it is not the #1536 shape — a typo'd
|
|
143
|
+
* filename exists nowhere.
|
|
144
|
+
*/
|
|
145
|
+
function candidateRoots(command: string, cwd: string): string[] | null {
|
|
146
|
+
const roots = [cwd];
|
|
147
|
+
for (const { segment } of suitePathArgumentsBySegment(command)) {
|
|
148
|
+
const cd = LEADING_CD.exec(segment);
|
|
149
|
+
if (!cd) continue;
|
|
150
|
+
const target = cd[1];
|
|
151
|
+
// `cd $WORKTREE`, `cd "$(git rev-parse …)"`, `cd -`: the set of roots
|
|
152
|
+
// cannot be enumerated, so the command is not judged at all.
|
|
153
|
+
if (UNRESOLVABLE_SEGMENT.test(target) || UNRESOLVABLE_ARGUMENT.test(target) || target === "-") {
|
|
154
|
+
return null;
|
|
155
|
+
}
|
|
156
|
+
roots.push(isAbsolute(target) ? target : resolve(cwd, target));
|
|
157
|
+
}
|
|
158
|
+
return roots;
|
|
159
|
+
}
|
|
160
|
+
|
|
161
|
+
export interface MissingTestPath {
|
|
162
|
+
/** The argument exactly as it was typed. */
|
|
163
|
+
argument: string;
|
|
164
|
+
/** Where it resolved to, so a reader can see which root was searched. */
|
|
165
|
+
resolved: string;
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
export interface BunTestPathRefusal {
|
|
169
|
+
missing: MissingTestPath[];
|
|
170
|
+
message: string;
|
|
171
|
+
}
|
|
172
|
+
|
|
173
|
+
function refusalMessage(missing: MissingTestPath[], roots: string[]): string {
|
|
174
|
+
const names = missing.map((m) => ` ${m.argument} → ${m.resolved}`).join("\n");
|
|
175
|
+
const plural = missing.length === 1 ? "a path that matches" : "paths that match";
|
|
176
|
+
// Every root is named. A refusal naming one directory while bun would have
|
|
177
|
+
// run in another is what made the first implementation's defect invisible.
|
|
178
|
+
const where = roots.length === 1 ? roots[0] : `any of ${roots.join(", ")}`;
|
|
179
|
+
return (
|
|
180
|
+
`This \`bun test\` names ${plural} no file under ${where}:\n${names}\n\n` +
|
|
181
|
+
`Bun treats an unmatched path argument as a NAME FILTER. It exits non-zero only ` +
|
|
182
|
+
`when NOTHING matched; with at least one real path alongside, the run exits 0 and ` +
|
|
183
|
+
`reports a pass total, so the argument above would contribute nothing and nobody ` +
|
|
184
|
+
`could see it from the outside. Measured on bun 1.3.14 (#1544).\n\n` +
|
|
185
|
+
`Check the spelling against the tree. If you want to confirm a run afterwards, the ` +
|
|
186
|
+
`tell is bun's own summary: \`Ran N tests across M files\` — M must equal the number ` +
|
|
187
|
+
`of paths you passed.\n\n` +
|
|
188
|
+
`RUNGATE_BUN_TEST_PATH_GUARD=warn or =off reverses this without a code change.`
|
|
189
|
+
);
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
/**
|
|
193
|
+
* The refusal, or null when every path argument resolves.
|
|
194
|
+
*
|
|
195
|
+
* An unreadable command — absent, or not a string — is NOT refused. Unlike
|
|
196
|
+
* the cd guard, whose false allow costs a peer half a day, this one's false
|
|
197
|
+
* refusal blocks the agent in front of it, and a payload with no command
|
|
198
|
+
* string is a hook-protocol problem rather than evidence of a bad path.
|
|
199
|
+
*/
|
|
200
|
+
export function bunTestPathRefusal(command: unknown, cwd: string): BunTestPathRefusal | null {
|
|
201
|
+
if (typeof command !== "string" || command.trim() === "") return null;
|
|
202
|
+
|
|
203
|
+
const roots = candidateRoots(command, cwd);
|
|
204
|
+
// A command naming a `cd` target this cannot resolve is not judged.
|
|
205
|
+
if (roots === null) return null;
|
|
206
|
+
|
|
207
|
+
const missing: MissingTestPath[] = [];
|
|
208
|
+
for (const { segment, paths } of suitePathArgumentsBySegment(command)) {
|
|
209
|
+
if (UNRESOLVABLE_SEGMENT.test(segment)) continue;
|
|
210
|
+
for (const argument of paths) {
|
|
211
|
+
if (UNRESOLVABLE_ARGUMENT.test(argument)) continue;
|
|
212
|
+
if (isAbsolute(argument)) {
|
|
213
|
+
if (!existsSync(argument)) missing.push({ argument, resolved: argument });
|
|
214
|
+
continue;
|
|
215
|
+
}
|
|
216
|
+
// Absent under EVERY root, or it is not missing. See `candidateRoots`.
|
|
217
|
+
const tried = roots.map((root) => resolve(root, argument));
|
|
218
|
+
if (tried.every((p) => !existsSync(p))) {
|
|
219
|
+
missing.push({ argument, resolved: tried.join(", ") });
|
|
220
|
+
}
|
|
221
|
+
}
|
|
222
|
+
}
|
|
223
|
+
|
|
224
|
+
if (missing.length === 0) return null;
|
|
225
|
+
return { missing, message: refusalMessage(missing, roots) };
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
export type BunTestPathGuardMode = "warn" | "block" | "off";
|
|
229
|
+
|
|
230
|
+
export const BUN_TEST_PATH_GUARD_MODE_VAR = "RUNGATE_BUN_TEST_PATH_GUARD";
|
|
231
|
+
|
|
232
|
+
/**
|
|
233
|
+
* `block`. The ladder exists so the decision is a configuration change rather
|
|
234
|
+
* than a code change made under pressure, and the rung is taken here.
|
|
235
|
+
*/
|
|
236
|
+
export const BUN_TEST_PATH_GUARD_DEFAULT_MODE: BunTestPathGuardMode = "block";
|
|
237
|
+
|
|
238
|
+
export interface BunTestPathGuardModeReading {
|
|
239
|
+
mode: BunTestPathGuardMode;
|
|
240
|
+
/** The value that was set and not understood. A guard must not turn itself off on a typo. */
|
|
241
|
+
unrecognised?: string;
|
|
242
|
+
}
|
|
243
|
+
|
|
244
|
+
export function bunTestPathGuardMode(
|
|
245
|
+
env: NodeJS.ProcessEnv = process.env,
|
|
246
|
+
): BunTestPathGuardModeReading {
|
|
247
|
+
const raw = env[BUN_TEST_PATH_GUARD_MODE_VAR];
|
|
248
|
+
if (raw === undefined || raw === "") return { mode: BUN_TEST_PATH_GUARD_DEFAULT_MODE };
|
|
249
|
+
if (raw === "warn" || raw === "block" || raw === "off") return { mode: raw };
|
|
250
|
+
// An unrecognised value falls back to the DEFAULT, not to `off`. A gate
|
|
251
|
+
// that switches itself off on a misspelling is a gate nobody can rely on.
|
|
252
|
+
return { mode: BUN_TEST_PATH_GUARD_DEFAULT_MODE, unrecognised: raw };
|
|
253
|
+
}
|
package/lib/conformity.ts
CHANGED
|
@@ -10,7 +10,8 @@ import { existsSync, mkdtempSync, readFileSync, readdirSync, realpathSync, rmSyn
|
|
|
10
10
|
import { tmpdir } from "os";
|
|
11
11
|
import type { Dirent } from "fs";
|
|
12
12
|
import { collectSpecFiles } from "./spec-files";
|
|
13
|
-
import {
|
|
13
|
+
import { skipLine } from "./derived-spec";
|
|
14
|
+
import { maySpecBeWritten, writeSpec } from "./write-spec";
|
|
14
15
|
import {
|
|
15
16
|
ambiguousSpecNames,
|
|
16
17
|
readSpecRequirements,
|
|
@@ -334,9 +335,12 @@ export function flipPassingCheckboxes(): number {
|
|
|
334
335
|
// on a spec a generator owns is discarded at the next regeneration,
|
|
335
336
|
// after breaking whatever pins the file — and a criterion counted as
|
|
336
337
|
// flipped in a file nobody wrote is a number that means nothing.
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
338
|
+
// #1552: the question is asked twice here and that is deliberate — once
|
|
339
|
+
// early so `flipped` does not count a file nobody wrote, and once inside
|
|
340
|
+
// `writeSpec`. Two consults of ONE decider, both landing on `mayWrite`.
|
|
341
|
+
const early = maySpecBeWritten(content);
|
|
342
|
+
if (!early.allowed) {
|
|
343
|
+
console.warn(skipLine(specPath, early.reading));
|
|
340
344
|
continue;
|
|
341
345
|
}
|
|
342
346
|
|
|
@@ -349,7 +353,7 @@ export function flipPassingCheckboxes(): number {
|
|
|
349
353
|
flipped++;
|
|
350
354
|
}
|
|
351
355
|
}
|
|
352
|
-
if (changed)
|
|
356
|
+
if (changed) writeSpec(specPath, content);
|
|
353
357
|
}
|
|
354
358
|
return flipped;
|
|
355
359
|
}
|
package/lib/finding-jury.ts
CHANGED
|
@@ -50,6 +50,7 @@
|
|
|
50
50
|
*/
|
|
51
51
|
|
|
52
52
|
import type { Finding } from "./finding-arbiter";
|
|
53
|
+
import { SEVERITY_RANK, type Severity } from "./scaffold/ai-review-step";
|
|
53
54
|
|
|
54
55
|
// ── Cardinalities ────────────────────────────────────────────────────────
|
|
55
56
|
|
|
@@ -214,11 +215,26 @@ export function classifyCategory(finding: Pick<Finding, "title" | "body">): Find
|
|
|
214
215
|
* `Pre-existing` is the one that matters most here and the one we had no way
|
|
215
216
|
* to say: a finding about code the change never touched is not the author's
|
|
216
217
|
* defect and should not be priced like one.
|
|
218
|
+
*
|
|
219
|
+
* **Declared in `./scaffold/ai-review-step` and re-exported here**, not
|
|
220
|
+
* declared twice. That module is the one the deployed CI path can import —
|
|
221
|
+
* it has no relative imports of its own, which is what lets the mutation
|
|
222
|
+
* batteries carry it — and until CRC-B7 this vocabulary lived here, where
|
|
223
|
+
* nothing reachable from CI could name it. Its docblock there carries the
|
|
224
|
+
* input on which this and `AI_REVIEW_SEVERITIES` must disagree.
|
|
225
|
+
*
|
|
226
|
+
* Re-exported rather than moved outright so `severityOf` and the three
|
|
227
|
+
* modules that name the vocabulary keep naming it here:
|
|
228
|
+
* `lib/finding-arbiter.ts`, `lib/ai-review-jury.ts` and
|
|
229
|
+
* `lib/issue-filing-gate.ts`. Measured 2026-10-24, not copied: #1587's own
|
|
230
|
+
* brief and the spec listed `lib/failure-intake.ts` and
|
|
231
|
+
* `hooks/VerifyPhaseLock.hook.ts` among them and NEITHER imports this module,
|
|
232
|
+
* directly or transitively. `scripts/arbitrate-finding.ts` does import from
|
|
233
|
+
* here — `parseVotes`, `renderComment`, `VerifierVote` — but never names
|
|
234
|
+
* `Severity` or `SEVERITY_RANK` itself.
|
|
217
235
|
*/
|
|
218
|
-
export
|
|
219
|
-
|
|
220
|
-
/** Sort order for the posted comments. A pre-existing bug is still a bug. */
|
|
221
|
-
export const SEVERITY_RANK: Record<Severity, number> = { Important: 0, "Pre-existing": 1, Nit: 2 };
|
|
236
|
+
export { SEVERITY_RANK };
|
|
237
|
+
export type { Severity };
|
|
222
238
|
|
|
223
239
|
// ── The vote ─────────────────────────────────────────────────────────────
|
|
224
240
|
|