@feigi/fleet-ctl 3.77.1 → 3.78.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -20,14 +20,25 @@ model: "@task:medium"
|
|
|
20
20
|
permanently. -->
|
|
21
21
|
|
|
22
22
|
You are a test-coverage analyst focused on whether tests actually
|
|
23
|
-
DISCRIMINATE, not on line coverage percentages.
|
|
23
|
+
DISCRIMINATE, not on line coverage percentages. The diff puts you in one of
|
|
24
|
+
two modes.
|
|
24
25
|
|
|
25
|
-
|
|
26
|
-
catch and confirm a kill, revert, then
|
|
27
|
-
and confirm the named test stays green.
|
|
28
|
-
mutation — a guard written to catch `//
|
|
29
|
-
trailing` through, and a test that only
|
|
30
|
-
|
|
26
|
+
**The diff touches or adds a test file.** For each test the diff touches or
|
|
27
|
+
adds, apply the mutation it is meant to catch and confirm a kill, revert, then
|
|
28
|
+
apply a mutation it should NOT catch and confirm the named test stays green.
|
|
29
|
+
Vary the syntactic form of the mutation — a guard written to catch `//
|
|
30
|
+
whole-line` comments may let `code; // trailing` through, and a test that only
|
|
31
|
+
exercises one form is not proven to discriminate the class.
|
|
32
|
+
|
|
33
|
+
**The diff changes source and touches no test file.** There is nothing in the
|
|
34
|
+
diff to discriminate, so hunt coverage gaps instead: for each piece of
|
|
35
|
+
behaviour the diff adds or changes in source, apply a mutation to that
|
|
36
|
+
changed source, run `<testCmd>`, and report changed behaviour that no
|
|
37
|
+
existing test kills. A mutant that survives, with a valid green on the
|
|
38
|
+
unmutated tree, is a finding; name the changed behaviour, the mutation and
|
|
39
|
+
the run that shows it surviving.
|
|
40
|
+
|
|
41
|
+
In both modes `<testCmd>` is the `command` your dispatch prompt's
|
|
31
42
|
Tests paragraph names, run only in your own copy of the snapshot. Make that
|
|
32
43
|
copy from the snapshot's tracked files only —
|
|
33
44
|
`git -C <snapshot> archive HEAD | tar -x -C <copy>`, or a `git clone` of the
|
|
@@ -92,7 +92,7 @@ Tell readers **`git show <sha>:<path>` is the source of truth** — the object s
|
|
|
92
92
|
|
|
93
93
|
**Brace a ref held in a variable — `git show "${SHA}:<path>"` — because quoting is not what fixes it.** zsh reads a `:x` suffix after an unbraced parameter as a history modifier, inside double quotes as well as outside, so git is handed a ref nobody typed; braces end the parameter name before the colon. Measured, fully quoted: the construct that fails loudly on one path exits 0 on another and prints the *commit* instead of the blob — and the silent shape is the hazard, because a survival `grep -c` scores it as a plausible hit and re-anchors a citation that was correct. A literal ref is unaffected either way; quote regardless — not for zsh's sake, which splits and globs an expansion's *result* only under `SH_WORD_SPLIT`/`GLOB_SUBST`, both off by default (measured: `V="a b"` unquoted still arrives as one argument), but so the same line survives a `bash` runner, where an unquoted path holding a space does split — measured, `fatal: path 'has' does not exist`.
|
|
94
94
|
|
|
95
|
-
**The fan-out scales to the diff — the review computes it, you don't.** Its snapshot agent runs `diff-stats.mjs`, and `selectDimensions` keeps the applicable subset of the six-specialist catalog (fleet-review-correctness, fleet-review-silent-failure, fleet-review-tests, fleet-review-comments, fleet-review-types, fleet-review-simplify): fleet-review-correctness always;
|
|
95
|
+
**The fan-out scales to the diff — the review computes it, you don't.** Its snapshot agent runs `diff-stats.mjs`, and `selectDimensions` keeps the applicable subset of the six-specialist catalog (fleet-review-correctness, fleet-review-silent-failure, fleet-review-tests, fleet-review-comments, fleet-review-types, fleet-review-simplify): fleet-review-correctness always; fleet-review-tests whenever the diff touches a test file, and also for a `production`-profile diff that changes source and touches no test file — there its job is the coverage gap, mutating the changed source and reporting behaviour no existing test kills, so untested source changes get asked that question during review — and dropped otherwise, so a diff with no source and no test file loses it; fleet-review-types / fleet-review-silent-failure / fleet-review-simplify with no source; a docs-only PR keeps only fleet-review-correctness + fleet-review-comments. **Then the size tier applies a floor on top:** a `single-file` or `small` profile keeps fleet-review-correctness + fleet-review-silent-failure + fleet-review-comments, plus fleet-review-tests when the diff touches a test file — so a small source PR runs three, not five, and the coverage-gap case above never reaches the size tier. **fleet-review-comments is unconditional on the floor**: it used to need a docs *file* in the diff, but `classify()` scores `.js`/`.mjs` `src` via the code-extension check and `.sh` `src` by falling through everything else, so a PR whose whole substance was prose in a `.js`/`.mjs`/`.sh` comment scored `docs: 0` and lost the one specialist whose job it was — measured five times in production, all correction-class PRs, with `dimensionsUnrun` empty each time because the trim was by design. **The floor outranks the no-source drop for fleet-review-silent-failure**: a one-file `.yml` or `.github/` shell change runs fleet-review-correctness + fleet-review-silent-failure, which is the pair a swallowed error in CI needs. Above the size tier the no-source drop still stands, and `tests-only` outranks the size tier for a diff with no config file — a no-src, no-config diff that also touches a test file gets no fleet-review-silent-failure; a tests-only diff that also carries a config file keeps it. `single-file` is one file at *any* size. Profile unknown → the full six, the safe direction. Pin a subset explicitly with `args.dimensions` (keys or dimension objects). Dispatching by hand instead of automatically: apply the same heuristic yourself — two or three for annotation-only or single-file, the full six for production code.
|
|
96
96
|
|
|
97
97
|
**If the ticket is a *correction* — stale docs, wrong comments, bad citations — hunt the defect inside the correction itself.** Four such tickets in one fleet run each shipped a new wrong claim in the fix: a misattributed package, a three-item list at inverted polarity (two items were the *negations* of the claims being marked stale, so the banner asserted currently-true facts were false), a commit body citing a line that held something else, and a banner whose own supporting sentence confirmed the bullet it marked as wrong. Nobody hunts this unprompted because the diff "obviously" improves accuracy. Read every corrected sentence literally and ask whether each clause is true under that reading.
|
|
98
98
|
|
package/package.json
CHANGED
package/scripts/review-core.mjs
CHANGED
|
@@ -209,7 +209,7 @@ export const DEFAULT_DIMENSIONS = [
|
|
|
209
209
|
{
|
|
210
210
|
key: "tests",
|
|
211
211
|
agentType: "fleet-review-tests",
|
|
212
|
-
prompt: `whether each test
|
|
212
|
+
prompt: `whether tests DISCRIMINATE, in whichever of two modes the diff puts you in. **The diff touches or adds a test file:** for each such test, apply the mutation it should catch and confirm a kill, revert, then apply one it should NOT catch and confirm the named test stays green. Vary the syntactic form — a guard catching \`// whole-line\` may let \`code; // trailing\` through. **The diff changes source and touches no test file:** hunt coverage gaps instead. For each piece of behaviour the diff adds or changes in source, apply a mutation to that changed source, run \`<testCmd>\`, and report changed behaviour that no existing test kills: a mutant that survives, with a valid green on the unmutated tree, is a finding. \`<testCmd>\` is the \`command\` your Tests paragraph names, run only in your own copy of the snapshot. A kill is scored by these rules and nothing else:
|
|
213
213
|
|
|
214
214
|
${KILL_RULES}
|
|
215
215
|
|
|
@@ -376,7 +376,13 @@ export function selectDimensions(all, stats) {
|
|
|
376
376
|
return docsOnlyDims;
|
|
377
377
|
}
|
|
378
378
|
let dims = all;
|
|
379
|
-
|
|
379
|
+
// `tests` has two jobs (see its prompt): discriminate the tests a diff touches,
|
|
380
|
+
// or — when the diff has source but no test — find changed behaviour no
|
|
381
|
+
// existing test kills. Only the second survives `hasTests === false`, and only
|
|
382
|
+
// above the size tier: a production-profile diff with source keeps it. The
|
|
383
|
+
// `!== false` on `hasSrc` keeps a blob that lost the field from narrowing.
|
|
384
|
+
const keepsCoverageGap = stats.profile === "production" && stats.hasSrc !== false;
|
|
385
|
+
if (stats.hasTests === false && !keepsCoverageGap) dims = dims.filter((d) => d.key !== "tests");
|
|
380
386
|
if (stats.hasSrc === false) {
|
|
381
387
|
const keepsSilentFailure =
|
|
382
388
|
SIZE_TIER_PROFILES.has(stats.profile) ||
|