mandrel 2.33.0 → 2.34.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/docs/configuration.md +2 -2
- package/.agents/schemas/agentrc.schema.json +2 -2
- package/.agents/scripts/coverage-capture.js +7 -1
- package/.agents/scripts/lib/baselines/kernel.js +20 -7
- package/.agents/scripts/lib/baselines/kinds/mutation.js +144 -14
- package/.agents/scripts/lib/config/gates/crap-incremental-coverage.schema.js +10 -7
- package/.agents/scripts/lib/config/quality.js +7 -0
- package/.agents/scripts/lib/coverage-capture-incremental.js +9 -2
- package/.agents/scripts/lib/coverage-capture-usage.js +55 -0
- package/.agents/scripts/lib/coverage-capture.js +10 -15
- package/.agents/scripts/lib/orchestration/plan-persist/story-ops.js +24 -7
- package/.agents/workflows/helpers/deliver-light.md +5 -2
- package/.agents/workflows/helpers/deliver-story-reference.md +23 -1
- package/.agents/workflows/helpers/deliver-story.md +4 -2
- package/docs/CHANGELOG.md +10 -0
- package/package.json +1 -1
|
@@ -210,8 +210,8 @@ Everything `/deliver` and `single-story-close` consume: execution timeouts, work
|
|
|
210
210
|
| `quality.gates.crap.refreshTag` | No | `string` | `"baseline-refresh:"` | Commit-subject substring that acknowledges a deliberate CRAP baseline refresh in the compared range. A range commit carrying it that also touches the baseline file demotes head-vs-base regressions; floors stay enforced. |
|
|
211
211
|
| `quality.gates.crap.refreshTimeoutMs` | No | `integer` | `60000` | Bounded timeout (ms) for `npm run crap:update` spawned by the baseline-attribution refresh path. Mirrors `coverage.timeoutMs`: a SIGKILL fired at the budget boundary maps to exit 124 so the close orchestrator can flip the Story to `agent::blocked`. Default 60000 (Story #2165). |
|
|
212
212
|
| `quality.gates.crap.ignoreGlobs` | No | `array<string>` | `[]` | Minimatch glob patterns matched against the canonicalised repo-relative path of each discovered file. Files matching any pattern are excluded from CRAP discovery before scoring. Orthogonal to `components` (grouping) — a file excluded here never appears in any component bucket. Absent or empty preserves the existing IGNORED_DIRS-only behaviour (Story #3217). |
|
|
213
|
-
| `quality.gates.crap.incrementalCoverage` | No | `object` | — | Story #4981 — opt-in incremental coverage-capture + CRAP-join scoping. Default (key absent) preserves today’s full-repo behaviour byte-for-byte. When `enabled: true`,
|
|
214
|
-
| `quality.gates.crap.incrementalCoverage.enabled` | No | `boolean` | — | Master switch for
|
|
213
|
+
| `quality.gates.crap.incrementalCoverage` | No | `object` | — | Story #4981 — opt-in incremental coverage-capture + CRAP-join scoping. Default (key absent) preserves today’s full-repo behaviour byte-for-byte. When `enabled: true`, the changed-file set against `baseRef` (default: the gate’s own `--ref` / `main`) decides WHETHER to capture — no changed file under `crap.targetDirs` means no capture at all — and the CRAP join treats a method in a file the diff did not touch as resolved by its committed baseline row instead of requiring fresh coverage for it. It does NOT narrow the capture run itself: a capture that does happen is the ordinary full `npm run test:coverage` (Story #5065). |
|
|
214
|
+
| `quality.gates.crap.incrementalCoverage.enabled` | No | `boolean` | — | Master switch for the capture skip and the baseline-resolved CRAP join. |
|
|
215
215
|
| `quality.gates.crap.incrementalCoverage.baseRef` | No | `string` | — | Git ref the changed-file set is computed against. Omitted falls back to the gate’s own `--ref` (`main`). |
|
|
216
216
|
| `quality.gates.maintainability` | No | `object` | — | Maintainability-index ratchet. Scores per file as the average over its methods, so deleting a small high-MI method can legitimately lower a file’s score. |
|
|
217
217
|
| `quality.gates.maintainability.enabled` | No | `boolean` | `true` | When false, the checker exits 0 with a skip line and the gate is reported as `skipped`, never omitted. |
|
|
@@ -978,11 +978,11 @@
|
|
|
978
978
|
},
|
|
979
979
|
"incrementalCoverage": {
|
|
980
980
|
"type": "object",
|
|
981
|
-
"description": "Story #4981 — opt-in incremental coverage-capture + CRAP-join scoping. Default (key absent) preserves today’s full-repo behaviour byte-for-byte. When `enabled: true`,
|
|
981
|
+
"description": "Story #4981 — opt-in incremental coverage-capture + CRAP-join scoping. Default (key absent) preserves today’s full-repo behaviour byte-for-byte. When `enabled: true`, the changed-file set against `baseRef` (default: the gate’s own `--ref` / `main`) decides WHETHER to capture — no changed file under `crap.targetDirs` means no capture at all — and the CRAP join treats a method in a file the diff did not touch as resolved by its committed baseline row instead of requiring fresh coverage for it. It does NOT narrow the capture run itself: a capture that does happen is the ordinary full `npm run test:coverage` (Story #5065).",
|
|
982
982
|
"properties": {
|
|
983
983
|
"enabled": {
|
|
984
984
|
"type": "boolean",
|
|
985
|
-
"description": "Master switch for
|
|
985
|
+
"description": "Master switch for the capture skip and the baseline-resolved CRAP join."
|
|
986
986
|
},
|
|
987
987
|
"baseRef": {
|
|
988
988
|
"type": "string",
|
|
@@ -33,6 +33,7 @@ import {
|
|
|
33
33
|
} from './lib/coverage-capture.js';
|
|
34
34
|
import { runFullScopeCapture } from './lib/coverage-capture-fullscope.js';
|
|
35
35
|
import { tryIncrementalCapture } from './lib/coverage-capture-incremental.js';
|
|
36
|
+
import { handleCoverageCaptureHelp } from './lib/coverage-capture-usage.js';
|
|
36
37
|
|
|
37
38
|
import { Logger } from './lib/Logger.js';
|
|
38
39
|
import { hasNpmScript, readPackageScripts } from './lib/npm-scripts.js';
|
|
@@ -160,7 +161,12 @@ export function runCoverageCapture(argv = process.argv, deps = {}) {
|
|
|
160
161
|
// invoked as a CLI the behaviour — exit code and log lines — is unchanged.
|
|
161
162
|
if (isDirectInvocation(import.meta.url)) {
|
|
162
163
|
try {
|
|
163
|
-
|
|
164
|
+
// `--help` is answered before the decision core runs: it used to fall
|
|
165
|
+
// through to the capture path, so asking this script to describe itself
|
|
166
|
+
// spawned the whole coverage suite.
|
|
167
|
+
process.exit(
|
|
168
|
+
handleCoverageCaptureHelp(process.argv) ? 0 : runCoverageCapture(),
|
|
169
|
+
);
|
|
164
170
|
} catch (err) {
|
|
165
171
|
Logger.error('[coverage-capture] unexpected error:', err);
|
|
166
172
|
process.exit(1);
|
|
@@ -107,6 +107,7 @@ import {
|
|
|
107
107
|
} from './kinds/maintainability.js';
|
|
108
108
|
import {
|
|
109
109
|
applyEpsilon as mutationApplyEpsilon,
|
|
110
|
+
assertBaselineCompatible as mutationAssertBaselineCompatible,
|
|
110
111
|
compare as mutationCompare,
|
|
111
112
|
kernelVersion as mutationKernelVersion,
|
|
112
113
|
keyField as mutationKeyField,
|
|
@@ -211,6 +212,7 @@ const KIND_MODULES = Object.freeze({
|
|
|
211
212
|
compare: mutationCompare,
|
|
212
213
|
applyEpsilon: mutationApplyEpsilon,
|
|
213
214
|
mergeRows: mutationMergeRows,
|
|
215
|
+
assertBaselineCompatible: mutationAssertBaselineCompatible,
|
|
214
216
|
}),
|
|
215
217
|
lighthouse: bindKindModule({
|
|
216
218
|
name: lighthouseName,
|
|
@@ -280,6 +282,22 @@ export function currentKernelVersion(kind) {
|
|
|
280
282
|
return getKindModule(kind).kernelVersion();
|
|
281
283
|
}
|
|
282
284
|
|
|
285
|
+
/**
|
|
286
|
+
* Resolve a kind module, or null when the kind is not registered. Lets the
|
|
287
|
+
* optional-hook callers below stay expression-shaped instead of threading a
|
|
288
|
+
* mutable binding through a try/catch.
|
|
289
|
+
*
|
|
290
|
+
* @param {string} kind
|
|
291
|
+
* @returns {object|null}
|
|
292
|
+
*/
|
|
293
|
+
function tryGetKindModule(kind) {
|
|
294
|
+
try {
|
|
295
|
+
return getKindModule(kind);
|
|
296
|
+
} catch {
|
|
297
|
+
return null;
|
|
298
|
+
}
|
|
299
|
+
}
|
|
300
|
+
|
|
283
301
|
/**
|
|
284
302
|
* Ask a kind whether a loaded baseline is compatible with the running
|
|
285
303
|
* scorer's SEMANTICS — a dimension `kernelVersion` cannot express, because a
|
|
@@ -291,13 +309,8 @@ export function currentKernelVersion(kind) {
|
|
|
291
309
|
* @returns {string|null} Operator-facing message, or null when compatible.
|
|
292
310
|
*/
|
|
293
311
|
export function checkBaselineSemantics(kind, baseline) {
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
mod = getKindModule(kind);
|
|
297
|
-
} catch {
|
|
298
|
-
return null;
|
|
299
|
-
}
|
|
300
|
-
if (typeof mod.assertBaselineCompatible !== 'function') return null;
|
|
312
|
+
const mod = tryGetKindModule(kind);
|
|
313
|
+
if (typeof mod?.assertBaselineCompatible !== 'function') return null;
|
|
301
314
|
return mod.assertBaselineCompatible(baseline);
|
|
302
315
|
}
|
|
303
316
|
|
|
@@ -2,13 +2,43 @@
|
|
|
2
2
|
* kinds/mutation.js — per-kind module for the mutation-testing baseline
|
|
3
3
|
* (Story #1891). Row shape: `{ path, score, killed, survived }`. Rollup
|
|
4
4
|
* carries score/killed/survived/noCoverage. Stryker is the upstream
|
|
5
|
-
* kernel; we pin a static
|
|
5
|
+
* kernel; we pin a static version until a Mandrel-side retrofit story
|
|
6
6
|
* wires the running Stryker version through (#1908).
|
|
7
7
|
*
|
|
8
8
|
* Higher score = better. New paths land in the `additions` bucket
|
|
9
9
|
* (Story #2012 — any real-world score under 100 must never flip to a
|
|
10
10
|
* regression); removed paths count as improvements when their score was
|
|
11
11
|
* non-perfect. Scaffold is generated by `makeBaselineKind` (Story #3983).
|
|
12
|
+
*
|
|
13
|
+
* ## Rollup weighting (Story #5058)
|
|
14
|
+
*
|
|
15
|
+
* The rollup score is a **mutant-weighted** mean —
|
|
16
|
+
* `sum(score_i * mutants_i) / sum(mutants_i)`, where
|
|
17
|
+
* `mutants_i = killed_i + survived_i`. It used to be `scoreSum / rows.length`,
|
|
18
|
+
* an unweighted mean over files, under which a 3-mutant file carried the same
|
|
19
|
+
* weight as a 300-mutant one: adding a handful of thinly-mutated new files
|
|
20
|
+
* dragged the whole-repo number down far enough to breach the floor arm while
|
|
21
|
+
* the compare arm reported no regression at all (a newly-scanned file has no
|
|
22
|
+
* baseline row, so it can only move the aggregate `applyFloors` scores).
|
|
23
|
+
*
|
|
24
|
+
* Both weights are already summed in the same loop and are already required
|
|
25
|
+
* by the row schema, which is `additionalProperties: false` — so the weighted
|
|
26
|
+
* score is computable from the existing row shape with no producer change and
|
|
27
|
+
* no schema change.
|
|
28
|
+
*
|
|
29
|
+
* **Limit of the approximation.** Rows carry no timeout or no-coverage counts
|
|
30
|
+
* (`noCoverage` in the rollup is hardcoded `0`), so `killed + survived` is not
|
|
31
|
+
* Stryker's full mutant population wherever timed-out or uncovered mutants
|
|
32
|
+
* exist. The weighted score therefore *approximates* Stryker's published
|
|
33
|
+
* overall score — it does not reproduce it. Closing that gap needs a producer
|
|
34
|
+
* change and its own Story.
|
|
35
|
+
*
|
|
36
|
+
* Because this changes what the stored number MEANS, it is a semantics
|
|
37
|
+
* migration: a baseline measured at 88.51 unweighted is 85.18 weighted over
|
|
38
|
+
* the same rows, so every floor calibrated on the old mean breaches on
|
|
39
|
+
* upgrade. `assertBaselineCompatible` below fails such a baseline closed —
|
|
40
|
+
* a bare `kernelVersion` bump would not, because `kernelMatch` feeds only the
|
|
41
|
+
* reporting-side drift count and reaches no exit code.
|
|
12
42
|
*/
|
|
13
43
|
|
|
14
44
|
import { canonicalise } from '../path-canon.js';
|
|
@@ -17,6 +47,24 @@ import { makeBaselineKind } from './kind-factory.js';
|
|
|
17
47
|
export const name = 'mutation';
|
|
18
48
|
export const keyField = 'path';
|
|
19
49
|
|
|
50
|
+
/**
|
|
51
|
+
* Kernel version of the mutation scorer. Bumped off the original `1.0.0` by
|
|
52
|
+
* Story #5058: the rollup score changed from an unweighted file mean to a
|
|
53
|
+
* mutant-weighted mean, so rows stamped below this version were aggregated
|
|
54
|
+
* under superseded semantics.
|
|
55
|
+
*/
|
|
56
|
+
const KERNEL_VERSION = '2.0.0';
|
|
57
|
+
|
|
58
|
+
/** Major of {@link KERNEL_VERSION} — the weighted-rollup boundary. */
|
|
59
|
+
const WEIGHTED_ROLLUP_MAJOR = 2;
|
|
60
|
+
|
|
61
|
+
const RESEED_REMEDY =
|
|
62
|
+
'Re-seed the baseline: re-run this project mutation run (Mandrel ships no ' +
|
|
63
|
+
"runner — Stryker is the upstream producer, e.g. 'npx stryker run') so " +
|
|
64
|
+
"'baselines/mutation.json' is rewritten under the weighted rollup, then " +
|
|
65
|
+
"commit it with a 'baseline-refresh:' subject and recalibrate the gate's " +
|
|
66
|
+
'floors against the new number.';
|
|
67
|
+
|
|
20
68
|
export function projectRow(row) {
|
|
21
69
|
return {
|
|
22
70
|
path: canonicalise(row.path),
|
|
@@ -26,20 +74,102 @@ export function projectRow(row) {
|
|
|
26
74
|
};
|
|
27
75
|
}
|
|
28
76
|
|
|
77
|
+
/**
|
|
78
|
+
* Parse the major component of a semver-ish stamp.
|
|
79
|
+
*
|
|
80
|
+
* @param {unknown} version
|
|
81
|
+
* @returns {number|null} The major, or null when unparseable/absent.
|
|
82
|
+
*/
|
|
83
|
+
function majorOf(version) {
|
|
84
|
+
const match = /^(\d+)\./.exec(String(version ?? ''));
|
|
85
|
+
return match ? Number(match[1]) : null;
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
/**
|
|
89
|
+
* Kind-module hook (Story #4775) — refuse a loaded baseline whose rollup was
|
|
90
|
+
* aggregated by the superseded unweighted mean. `checkBaselineSemantics`
|
|
91
|
+
* dispatches here and `check-baselines`' evaluate phase turns a non-null
|
|
92
|
+
* return into a fail-closed `semantics` schema error, so a pre-weighting
|
|
93
|
+
* baseline can never be silently scored against floors calibrated on a
|
|
94
|
+
* different definition of the number. Follows the `kinds/crap.js` precedent.
|
|
95
|
+
*
|
|
96
|
+
* An absent or unparseable stamp is rejected too: `kernelVersion` is required
|
|
97
|
+
* by the shared envelope schema, so its absence is not evidence of a newer
|
|
98
|
+
* writer.
|
|
99
|
+
*
|
|
100
|
+
* @param {object|null} baseline A loaded v2 baseline envelope.
|
|
101
|
+
* @returns {string|null} Operator-facing message, or null when compatible.
|
|
102
|
+
*/
|
|
103
|
+
export function assertBaselineCompatible(baseline) {
|
|
104
|
+
if (!baseline) return null;
|
|
105
|
+
const stamped = baseline.kernelVersion ?? null;
|
|
106
|
+
const major = majorOf(stamped);
|
|
107
|
+
if (major !== null && major >= WEIGHTED_ROLLUP_MAJOR) return null;
|
|
108
|
+
return (
|
|
109
|
+
`[mutation] rollup scoring semantics changed: baseline=${stamped ?? '<unstamped>'} ` +
|
|
110
|
+
`running=${KERNEL_VERSION}. The rollup score is now a mutant-weighted mean ` +
|
|
111
|
+
'(sum(score * mutants) / sum(mutants)) rather than an unweighted mean over ' +
|
|
112
|
+
'files, so the stored aggregate is a different number for the same rows and ' +
|
|
113
|
+
`the floors calibrated against it no longer mean what they did. ${RESEED_REMEDY}`
|
|
114
|
+
);
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
/**
|
|
118
|
+
* A row's mutant count — the weight it carries in the rollup score.
|
|
119
|
+
*
|
|
120
|
+
* @param {object} row
|
|
121
|
+
* @returns {number}
|
|
122
|
+
*/
|
|
123
|
+
function mutantsOf(row) {
|
|
124
|
+
return (row.killed ?? 0) + (row.survived ?? 0);
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
/**
|
|
128
|
+
* Sum one numeric row field across a row set.
|
|
129
|
+
*
|
|
130
|
+
* @param {object[]} rows
|
|
131
|
+
* @param {string} field
|
|
132
|
+
* @returns {number}
|
|
133
|
+
*/
|
|
134
|
+
function sumOf(rows, field) {
|
|
135
|
+
let total = 0;
|
|
136
|
+
for (const r of rows) total += r[field] ?? 0;
|
|
137
|
+
return total;
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
/**
|
|
141
|
+
* The mutant-weighted mean score: `sum(score * mutants) / sum(mutants)`.
|
|
142
|
+
*
|
|
143
|
+
* Zero-mutant guard: an empty row set, or one whose rows carry only zeroes,
|
|
144
|
+
* has no weight to divide by. Returning 0 rather than dividing keeps `NaN` and
|
|
145
|
+
* `Infinity` out of the envelope, where they would fail the schema's numeric
|
|
146
|
+
* bounds downstream.
|
|
147
|
+
*
|
|
148
|
+
* @param {object[]} rows
|
|
149
|
+
* @param {number} mutants Total mutant count across `rows`.
|
|
150
|
+
* @returns {number}
|
|
151
|
+
*/
|
|
152
|
+
function weightedScore(rows, mutants) {
|
|
153
|
+
if (mutants <= 0) return 0;
|
|
154
|
+
let weighted = 0;
|
|
155
|
+
for (const r of rows) weighted += (r.score ?? 0) * mutantsOf(r);
|
|
156
|
+
return Number((weighted / mutants).toFixed(2));
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
/**
|
|
160
|
+
* Aggregate rows into the rollup shape. `killed` and `survived` stay plain
|
|
161
|
+
* sums and `noCoverage` stays hardcoded 0 — the weighting changes `score`
|
|
162
|
+
* alone.
|
|
163
|
+
*
|
|
164
|
+
* @param {object[]} rows
|
|
165
|
+
* @returns {{score: number, killed: number, survived: number, noCoverage: number}}
|
|
166
|
+
*/
|
|
29
167
|
function aggregate(rows) {
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
let scoreSum = 0;
|
|
34
|
-
let killed = 0;
|
|
35
|
-
let survived = 0;
|
|
36
|
-
for (const r of rows) {
|
|
37
|
-
scoreSum += r.score ?? 0;
|
|
38
|
-
killed += r.killed ?? 0;
|
|
39
|
-
survived += r.survived ?? 0;
|
|
40
|
-
}
|
|
168
|
+
const scored = rows ?? [];
|
|
169
|
+
const killed = sumOf(scored, 'killed');
|
|
170
|
+
const survived = sumOf(scored, 'survived');
|
|
41
171
|
return {
|
|
42
|
-
score:
|
|
172
|
+
score: weightedScore(scored, killed + survived),
|
|
43
173
|
killed,
|
|
44
174
|
survived,
|
|
45
175
|
noCoverage: 0,
|
|
@@ -55,7 +185,7 @@ export const {
|
|
|
55
185
|
mergeRows,
|
|
56
186
|
} = makeBaselineKind({
|
|
57
187
|
keyField,
|
|
58
|
-
kernelVersion:
|
|
188
|
+
kernelVersion: KERNEL_VERSION,
|
|
59
189
|
axes: ['score'],
|
|
60
190
|
betterWhen: 'higher',
|
|
61
191
|
aggregate,
|
|
@@ -10,20 +10,23 @@
|
|
|
10
10
|
* spread into.
|
|
11
11
|
*
|
|
12
12
|
* Default (key absent) preserves today's full-repo behaviour byte-for-byte.
|
|
13
|
-
* When `enabled: true`,
|
|
14
|
-
*
|
|
15
|
-
* `
|
|
16
|
-
*
|
|
17
|
-
* fresh coverage for it.
|
|
13
|
+
* When `enabled: true`, the changed-file set against `baseRef` (default: the
|
|
14
|
+
* gate's own `--ref` / `main`) decides **whether** to capture — no changed
|
|
15
|
+
* file under `crap.targetDirs` means no capture at all — and the CRAP join
|
|
16
|
+
* treats a method in a file the diff did not touch as resolved by its
|
|
17
|
+
* committed baseline row instead of requiring fresh coverage for it. It does
|
|
18
|
+
* not narrow the capture run itself: a capture that does happen is the
|
|
19
|
+
* ordinary full `npm run test:coverage` (Story #5065).
|
|
18
20
|
*/
|
|
19
21
|
export const INCREMENTAL_COVERAGE_SCHEMA = {
|
|
20
22
|
type: 'object',
|
|
21
23
|
description:
|
|
22
|
-
'Story #4981 — opt-in incremental coverage-capture + CRAP-join scoping. Default (key absent) preserves today’s full-repo behaviour byte-for-byte. When `enabled: true`,
|
|
24
|
+
'Story #4981 — opt-in incremental coverage-capture + CRAP-join scoping. Default (key absent) preserves today’s full-repo behaviour byte-for-byte. When `enabled: true`, the changed-file set against `baseRef` (default: the gate’s own `--ref` / `main`) decides WHETHER to capture — no changed file under `crap.targetDirs` means no capture at all — and the CRAP join treats a method in a file the diff did not touch as resolved by its committed baseline row instead of requiring fresh coverage for it. It does NOT narrow the capture run itself: a capture that does happen is the ordinary full `npm run test:coverage` (Story #5065).',
|
|
23
25
|
properties: {
|
|
24
26
|
enabled: {
|
|
25
27
|
type: 'boolean',
|
|
26
|
-
description:
|
|
28
|
+
description:
|
|
29
|
+
'Master switch for the capture skip and the baseline-resolved CRAP join.',
|
|
27
30
|
},
|
|
28
31
|
baseRef: {
|
|
29
32
|
type: 'string',
|
|
@@ -78,6 +78,13 @@ const DEFAULT_MI_FLOORS = Object.freeze({
|
|
|
78
78
|
* `enabled: true`. `baseRef: null` means "use the caller's own ref
|
|
79
79
|
* resolution" (the gate's `--ref` flag / `main`) rather than a second,
|
|
80
80
|
* possibly-conflicting default.
|
|
81
|
+
*
|
|
82
|
+
* Story #5065 — what `enabled: true` actually buys, measured: the capture is
|
|
83
|
+
* **skipped entirely** when no changed file lives under `crap.targetDirs`,
|
|
84
|
+
* and the CRAP join resolves methods in untouched files from the committed
|
|
85
|
+
* baseline row instead of requiring fresh coverage. It does **not** shorten
|
|
86
|
+
* the capture run — when a capture does happen it is the ordinary full
|
|
87
|
+
* `npm run test:coverage`.
|
|
81
88
|
*/
|
|
82
89
|
const DEFAULT_INCREMENTAL_COVERAGE = Object.freeze({
|
|
83
90
|
enabled: false,
|
|
@@ -14,6 +14,14 @@ import path from 'node:path';
|
|
|
14
14
|
* Run the incremental capture path when
|
|
15
15
|
* `delivery.quality.gates.crap.incrementalCoverage.enabled` is true.
|
|
16
16
|
*
|
|
17
|
+
* **This does not shorten the capture run.** The changed-file set decides
|
|
18
|
+
* *whether* to capture, never *what* the capture executes: when nothing under
|
|
19
|
+
* `crap.targetDirs` changed there is no capture at all, and otherwise the
|
|
20
|
+
* ordinary full `npm run test:coverage` runs. The saving that makes the mode
|
|
21
|
+
* worth having is the skip; the other half is the CRAP join, which resolves
|
|
22
|
+
* methods in untouched files from the committed baseline row
|
|
23
|
+
* (`crap-baseline-join.js`) instead of demanding fresh coverage for them.
|
|
24
|
+
*
|
|
17
25
|
* Returns the process exit code when incremental mode handled the run
|
|
18
26
|
* (skip, capture, or a capture failure), or `null` when the caller should
|
|
19
27
|
* fall through to the full-scope path — either incremental mode is
|
|
@@ -81,13 +89,12 @@ export function tryIncrementalCapture({
|
|
|
81
89
|
}
|
|
82
90
|
|
|
83
91
|
logger.info(
|
|
84
|
-
`[coverage-capture] Incremental mode:
|
|
92
|
+
`[coverage-capture] Incremental mode: ${scopedFiles.length} changed file(s) under [${crap.targetDirs.join(', ')}] — capturing…`,
|
|
85
93
|
);
|
|
86
94
|
const code = runCaptureImpl({
|
|
87
95
|
cwd: args.cwd,
|
|
88
96
|
timeoutMs: coverage?.timeoutMs,
|
|
89
97
|
log: (m) => logger.info(m),
|
|
90
|
-
files: scopedFiles,
|
|
91
98
|
});
|
|
92
99
|
if (code !== 0) {
|
|
93
100
|
logger.error(
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* coverage-capture-usage.js — the `--help` spec for `coverage-capture.js`
|
|
3
|
+
* (Story #5063).
|
|
4
|
+
*
|
|
5
|
+
* The delivery workflow invokes `coverage-capture.js` by name
|
|
6
|
+
* (`helpers/deliver-story-reference.md` § Step 1), which brings it under the
|
|
7
|
+
* workflow-invoked self-description contract enforced by
|
|
8
|
+
* `tests/enforcement/workflow-script-help.test.js`. It failed that contract:
|
|
9
|
+
* `--help` fell through to the capture path and spawned the whole coverage
|
|
10
|
+
* suite instead of describing the script.
|
|
11
|
+
*
|
|
12
|
+
* The spec lives here rather than inline for the same reason
|
|
13
|
+
* `coverage-capture-incremental.js` does — a same-file expansion of the CLI
|
|
14
|
+
* shell costs maintainability index on a file already near its floor, and a
|
|
15
|
+
* usage table is data, not decision logic.
|
|
16
|
+
*/
|
|
17
|
+
|
|
18
|
+
import { respondToHelp } from './cli-usage.js';
|
|
19
|
+
|
|
20
|
+
/**
|
|
21
|
+
* Usage spec consumed by `cli-usage.js#respondToHelp`. `coverage-capture.js`
|
|
22
|
+
* does not route through `runAsCli` (its synchronous main returns an exit
|
|
23
|
+
* code that `process.exit` forwards), so the help short-circuit is wired by
|
|
24
|
+
* hand rather than declared on a `runAsCli` call.
|
|
25
|
+
*
|
|
26
|
+
* @type {{ invocation: string, summary: string, flags: Array<[string, string]> }}
|
|
27
|
+
*/
|
|
28
|
+
const COVERAGE_CAPTURE_USAGE = {
|
|
29
|
+
invocation:
|
|
30
|
+
'node .agents/scripts/coverage-capture.js [--skip-when-no-crap-files] [--ref <git-ref>] [--cwd <path>]',
|
|
31
|
+
summary:
|
|
32
|
+
'Ensure coverage/coverage-final.json is present and fresh before the CRAP gate fires, spawning `npm run test:coverage` only when it is stale. Writes a content-digest capture stamp that close-validation reads to skip a redundant re-run.',
|
|
33
|
+
flags: [
|
|
34
|
+
[
|
|
35
|
+
'--skip-when-no-crap-files',
|
|
36
|
+
'Exit 0 without capturing when no changed file under the CRAP target dirs differs from --ref.',
|
|
37
|
+
],
|
|
38
|
+
['--ref <git-ref>', 'Git ref the changed-file set is computed against.'],
|
|
39
|
+
['--cwd <path>', 'Repository root the capture runs in.'],
|
|
40
|
+
],
|
|
41
|
+
};
|
|
42
|
+
|
|
43
|
+
/**
|
|
44
|
+
* Answer `--help` / `-h` on stdout, returning whether the caller should stop.
|
|
45
|
+
* Takes the full `process.argv`-shaped array so the CLI shell hands over its
|
|
46
|
+
* own argv unchanged and the index arithmetic lives here rather than at the
|
|
47
|
+
* call site.
|
|
48
|
+
*
|
|
49
|
+
* @param {string[]} argv Full `process.argv`-shaped array.
|
|
50
|
+
* @param {{ write: (s: string) => void }} [out] Defaults to `process.stdout`.
|
|
51
|
+
* @returns {boolean} `true` when help was printed and the run must not proceed.
|
|
52
|
+
*/
|
|
53
|
+
export function handleCoverageCaptureHelp(argv = [], out = process.stdout) {
|
|
54
|
+
return respondToHelp(argv.slice(2), COVERAGE_CAPTURE_USAGE, out);
|
|
55
|
+
}
|
|
@@ -363,20 +363,21 @@ export const COVERAGE_TIMEOUT_EXIT_CODE = 124;
|
|
|
363
363
|
* `timeout(1)` convention exit code 124 so callers can pattern-match a
|
|
364
364
|
* runaway runner without inspecting signal names.
|
|
365
365
|
*
|
|
366
|
-
*
|
|
367
|
-
* test:coverage -- <files
|
|
368
|
-
*
|
|
369
|
-
*
|
|
370
|
-
*
|
|
371
|
-
*
|
|
372
|
-
*
|
|
366
|
+
* The spawn takes **no positional file arguments**. Story #4981 forwarded the
|
|
367
|
+
* changed-file list as `npm run test:coverage -- <files...>` on the premise
|
|
368
|
+
* that a test runner treats trailing positionals as filters over the suite.
|
|
369
|
+
* Node's runner does not: it treats each path as a test file to execute, so a
|
|
370
|
+
* forwarded *source* file runs as a trivially-passing test and the real suite
|
|
371
|
+
* never runs. `run-coverage.js` discarded the list, which is the only reason
|
|
372
|
+
* that never bit; Story #5063 measured it and Story #5065 removed the
|
|
373
|
+
* plumbing rather than leave a parameter whose obvious "fix" empties the
|
|
374
|
+
* coverage artifact.
|
|
373
375
|
*
|
|
374
376
|
* @param {{
|
|
375
377
|
* cwd: string,
|
|
376
378
|
* timeoutMs?: number,
|
|
377
379
|
* runner?: typeof spawnSync,
|
|
378
380
|
* log?: (m: string) => void,
|
|
379
|
-
* files?: string[] | null,
|
|
380
381
|
* }} opts
|
|
381
382
|
* @returns {number}
|
|
382
383
|
*/
|
|
@@ -385,14 +386,8 @@ export function runCapture({
|
|
|
385
386
|
timeoutMs,
|
|
386
387
|
runner = spawnSync,
|
|
387
388
|
log = () => {},
|
|
388
|
-
files = null,
|
|
389
389
|
} = {}) {
|
|
390
|
-
const
|
|
391
|
-
const args = [
|
|
392
|
-
'run',
|
|
393
|
-
'test:coverage',
|
|
394
|
-
...(scopedFiles ? ['--', ...scopedFiles] : []),
|
|
395
|
-
];
|
|
390
|
+
const args = ['run', 'test:coverage'];
|
|
396
391
|
log(`[coverage-capture] ▶ npm ${args.join(' ')}`);
|
|
397
392
|
const spawnOpts = {
|
|
398
393
|
cwd,
|
|
@@ -672,13 +672,30 @@ function renderStoryBodyForCreate(story, idBySlug) {
|
|
|
672
672
|
const dependencyRefs = story.depends_on.map(
|
|
673
673
|
(slug) => `#${idBySlug.get(slug)}`,
|
|
674
674
|
);
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
675
|
+
let base = story.body;
|
|
676
|
+
if (dependencyRefs.length > 0) {
|
|
677
|
+
// Re-serializing from `bodyObject` is what resolves the sibling slugs to
|
|
678
|
+
// real issue ids — but `bodyObject` never held the provenance footers
|
|
679
|
+
// (`assembleOnePlanStory` appends those to the body *string*), so this
|
|
680
|
+
// branch drops them unless the carry is re-applied. That is the exact
|
|
681
|
+
// loss site Story #4935 diagnosed, #4939 fixed, and #4956 reverted
|
|
682
|
+
// wholesale hours later; Story #5056 restored it with a persist-side
|
|
683
|
+
// regression test that reads the POSTed body.
|
|
684
|
+
//
|
|
685
|
+
// `from: story.body` — not the seed — is load-bearing: it re-carries the
|
|
686
|
+
// identities *this* Story was stamped with under Story #5045 attribution
|
|
687
|
+
// rather than reintroducing the whole seed's union. `carryProvenanceFooters`
|
|
688
|
+
// is additive, union-preserving and idempotent, so re-applying is safe by
|
|
689
|
+
// construction.
|
|
690
|
+
const reserialized = serializeStoryBody(
|
|
691
|
+
{ ...story.bodyObject, depends_on: dependencyRefs },
|
|
692
|
+
{ includeFooter: true },
|
|
693
|
+
);
|
|
694
|
+
base = carryProvenanceFooters({
|
|
695
|
+
from: story.body,
|
|
696
|
+
into: reserialized,
|
|
697
|
+
}).body;
|
|
698
|
+
}
|
|
682
699
|
return `${base}\n\n${planFingerprintMarker(story.fingerprint)}`;
|
|
683
700
|
}
|
|
684
701
|
|
|
@@ -131,8 +131,11 @@ answer).
|
|
|
131
131
|
invoked, not reimplemented.
|
|
132
132
|
|
|
133
133
|
3. **Implement + self-eval.** `cd` into `workCwd`, implement the change, run
|
|
134
|
-
|
|
135
|
-
|
|
134
|
+
the full suite once in the worktree **so close can credit it** — the
|
|
135
|
+
crediting invocation and the freshness contract are
|
|
136
|
+
[`deliver-story.md`](deliver-story.md) Step 1.3, unchanged here — then run
|
|
137
|
+
the bounded acceptance self-eval loop
|
|
138
|
+
([`deliver-story.md`](deliver-story.md) Step 1a). Commit
|
|
136
139
|
on `story-<id>` with `(refs #<storyId>)`.
|
|
137
140
|
|
|
138
141
|
4. **Diff backstop.** Before close, re-check the ACTUAL diff:
|
|
@@ -233,12 +233,34 @@ runs maker-blind at Story-scope review inside the close subprocess. The
|
|
|
233
233
|
dispatch step produces `checklistPath` from the Story's predicted footprint
|
|
234
234
|
before it spawns the worker — see [`/deliver`](../deliver.md).
|
|
235
235
|
|
|
236
|
-
**Pre-eval full-suite discipline (spine step
|
|
236
|
+
**Pre-eval full-suite discipline (spine step 1.3).** Repo-invariant guards —
|
|
237
237
|
drift-guard and schema tests living outside the Story's scoped greps — are
|
|
238
238
|
the failure class that actually bounces deliveries: close-validation
|
|
239
239
|
discovers them only after the whole close pipeline has run, at several times
|
|
240
240
|
the cost of one pre-eval full-suite run.
|
|
241
241
|
|
|
242
|
+
**Run it so close can credit it.** Close skips a gate that already passed at
|
|
243
|
+
the current HEAD, but a bare `npm test` deposits no such record — the suite
|
|
244
|
+
then runs twice per delivery, once here and once in the close gate chain.
|
|
245
|
+
Pick the invocation by the same predicate `close-validation/gates.js` uses to
|
|
246
|
+
choose its test gate:
|
|
247
|
+
|
|
248
|
+
```bash
|
|
249
|
+
# CRAP gate enabled (default) + a `test:coverage` script — writes the stamp
|
|
250
|
+
# the close `coverage-capture` gate reads:
|
|
251
|
+
node <main-repo>/.agents/scripts/coverage-capture.js --cwd <workCwd>
|
|
252
|
+
# otherwise — the evidence record the close `test` gate reads. <workCwd> must
|
|
253
|
+
# be ABSOLUTE and the runner exactly `npm test`: both sides hash
|
|
254
|
+
# {cmd, args, cwd}, so a relative path or a wrapper misses the credit.
|
|
255
|
+
node <main-repo>/.agents/scripts/evidence-gate.js --standalone \
|
|
256
|
+
--scope-id <storyId> --gate test --worktree <workCwd> -- npm test
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
The credit expires the moment it stops describing the tree: evidence is keyed
|
|
260
|
+
on HEAD, the capture stamp on a content digest of `crap.targetDirs`. A
|
|
261
|
+
self-eval fix — or any commit — invalidates it and close re-runs the suite for
|
|
262
|
+
real, so this never trades away the gate.
|
|
263
|
+
|
|
242
264
|
**Conflict with `main` mid-implementation** → resolve as you would any branch
|
|
243
265
|
rebase. There is no `epic/<id>` intermediate, so the rebase base is `main`
|
|
244
266
|
directly.
|
|
@@ -75,9 +75,11 @@ One branch, one PR to `main`, commits against the inline `acceptance[]` /
|
|
|
75
75
|
`## Slicing` rows as **intra-session checkpoints** (reference § Step 1).
|
|
76
76
|
2. Implement and commit on the Story branch, iterating with quick advisory
|
|
77
77
|
gates (`typecheck`, `lint`, scoped tests) — the full chain runs in Step 3.
|
|
78
|
-
3. Run
|
|
78
|
+
3. Run the full suite once in the worktree **before Step 1a**: repo-invariant
|
|
79
79
|
guards outside the Story's scoped greps are the failure class that bounces
|
|
80
|
-
deliveries. Fix and commit first, then run the self-eval loop.
|
|
80
|
+
deliveries. Fix and commit first, then run the self-eval loop. Run it **so
|
|
81
|
+
Step 3 credits it** — a bare `npm test` records nothing, so close re-runs
|
|
82
|
+
the identical suite (reference § Step 1, "Pre-eval full-suite discipline").
|
|
81
83
|
|
|
82
84
|
### Step 1a — Bounded acceptance self-eval loop (**required**)
|
|
83
85
|
|
package/docs/CHANGELOG.md
CHANGED
|
@@ -15,6 +15,16 @@ All notable changes to this project will be documented in this file.
|
|
|
15
15
|
-->
|
|
16
16
|
<!-- markdownlint-disable-file MD004 MD012 MD037 -->
|
|
17
17
|
|
|
18
|
+
## [2.34.0](https://github.com/dsj1984/mandrel/compare/mandrel-v2.33.0...mandrel-v2.34.0) (2026-08-27)
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
### Fixed
|
|
22
|
+
|
|
23
|
+
* **baselines:** weight the mutation rollup by mutant count and gate the migration (refs [#5058](https://github.com/dsj1984/mandrel/issues/5058)) ([#5062](https://github.com/dsj1984/mandrel/issues/5062)) ([a645d30](https://github.com/dsj1984/mandrel/commit/a645d3085450b4dae39ae5d0a9b9844014535f84))
|
|
24
|
+
* **coverage:** describe incrementalCoverage honestly, drop inert forwarding (refs [#5065](https://github.com/dsj1984/mandrel/issues/5065)) ([#5066](https://github.com/dsj1984/mandrel/issues/5066)) ([04ff450](https://github.com/dsj1984/mandrel/commit/04ff450cb1774a1faa0b4f901f1e09261b0ae00a))
|
|
25
|
+
* delivery lifecycle: credit the Step 1 full-suite run to close and enable incremental coverage by default ([#5063](https://github.com/dsj1984/mandrel/issues/5063)) ([#5064](https://github.com/dsj1984/mandrel/issues/5064)) ([39b1ea4](https://github.com/dsj1984/mandrel/commit/39b1ea421c9a8a43f35ad4c8cbc082e1f2ead8c4))
|
|
26
|
+
* **plan-persist:** re-carry audit provenance onto dependent Stories (refs [#5056](https://github.com/dsj1984/mandrel/issues/5056)) ([#5060](https://github.com/dsj1984/mandrel/issues/5060)) ([548cc8a](https://github.com/dsj1984/mandrel/commit/548cc8a46afc5b643452060cc3a03bf71a874b89))
|
|
27
|
+
|
|
18
28
|
## [2.33.0](https://github.com/dsj1984/mandrel/compare/mandrel-v2.32.0...mandrel-v2.33.0) (2026-08-07)
|
|
19
29
|
|
|
20
30
|
|
package/package.json
CHANGED