oxmutate 1.0.3 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +18 -9
- package/binding/oxmutate.darwin-arm64.node +0 -0
- package/binding/oxmutate.darwin-x64.node +0 -0
- package/binding/oxmutate.linux-arm64-gnu.node +0 -0
- package/binding/oxmutate.linux-x64-gnu.node +0 -0
- package/binding/oxmutate.linux-x64-musl.node +0 -0
- package/binding/oxmutate.win32-x64-msvc.node +0 -0
- package/dist/checker.js +1 -0
- package/dist/child.js +27 -1
- package/dist/coordinator.d.ts +6 -0
- package/dist/coordinator.js +193 -29
- package/dist/directives.js +55 -0
- package/dist/heartbeat.js +111 -0
- package/dist/lanes.js +1 -0
- package/dist/progress.js +1 -0
- package/dist/reports.js +1 -1
- package/dist/runner.d.ts +21 -0
- package/dist/runner.js +270 -11
- package/dist/runtime.js +1 -0
- package/dist/scheduler.js +51 -12
- package/dist/spec-oracle.js +1 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -45,7 +45,7 @@ oxc, not affiliated with the oxc project.
|
|
|
45
45
|
|
|
46
46
|
| | Stryker 10 | oxmutate 1.0 |
|
|
47
47
|
| :-- | :-- | :-- |
|
|
48
|
-
| Mutant set | 17 mutator families | the same 17, the same mutants (720 of 720 on es-toolkit) |
|
|
48
|
+
| Mutant set | 17 mutator families | the same 17, the same mutants (720 of 720 on es-toolkit), plus `NumericLiteral` (0 → 1, 1 → 0, n → n + 1): +6.6% mutants on burgee/paratext, 19 of them survivors Stryker cannot report. `--stryker-compat` gives Stryker's set exactly |
|
|
49
49
|
| Verdicts | — | the same as Stryker's on 99.9% of mutants ([measured](#compatibility)) |
|
|
50
50
|
| Instrumentation | Babel, file by file | Rust on oxc, every file in parallel: 720 mutants in 35 ms |
|
|
51
51
|
| Test runs | a Vitest run per mutant | each covering test re-run per mutant inside a warm Vitest worker |
|
|
@@ -187,10 +187,10 @@ setting a value inside the worker that already has the tests loaded.
|
|
|
187
187
|
| :-- | :-- |
|
|
188
188
|
| The dry run must pass | A suite that fails with no mutant armed scores nothing (exit 3) |
|
|
189
189
|
| Re-parse | Every instrumented file is parsed again before it runs; one that does not parse stops the run (exit 3) instead of scoring a broken file |
|
|
190
|
-
| Canary | The unmutated test must still pass after mutants run in a context; if a mutant left a cache, a registry or a fixture behind, the context restarts and the suspect verdicts run again |
|
|
190
|
+
| Canary | The unmutated test must still pass after mutants run in a context, with its `beforeEach`/`afterEach` hooks re-run around every run, file-level ones included; if a mutant left a cache, a registry or a fixture behind, the context restarts and the suspect verdicts run again. A kill that follows unchecked runs is checked at once and decided again from clean state, so no kill rests on another mutant's leftovers |
|
|
191
191
|
| Survivor confirmation | A survivor is re-run with the mutant armed across its covering tests in one file, as Stryker arms it, so a mutant that only fails across tests is killed |
|
|
192
|
-
| Watchdog | A synchronous runaway is stopped and its server replaced; the stop is re-run at 3x the budget before it is reported as a timeout |
|
|
193
|
-
| Static mutants | Code that runs at import is
|
|
192
|
+
| Watchdog | A synchronous runaway is stopped and its server replaced; the stop is re-run at 3x the budget before it is reported as a timeout. A stop whose worker was idle inside a test body (its event loop still turning, by a heartbeat, and its server under 25% of a core) is a hang, not a slow test: it is a Timeout as it stands, the way Stryker scores it, with no 3x re-run |
|
|
193
|
+
| Static mutants | Code that runs at import is decided in a live worker: the mutant is armed, the module registry reset and the test file collected again, against the files that load it. A file whose collection changes globals, `process.env` or a built-in prototype, or a mutant that leaves a trace, goes back to a fresh worker per mutant (`OXMUTATE_ISOLATE_STATIC=1` sends all of them there) |
|
|
194
194
|
| Flaky tests | Tests that behave differently across two dry runs have their mutants re-run; a verdict that changes is reported `flaky` |
|
|
195
195
|
| No orphans | Every Vitest server exits when its run does, killed or not |
|
|
196
196
|
|
|
@@ -249,11 +249,20 @@ Of the two disagreements left, one is a Stryker timeout on a loop that ends (its
|
|
|
249
249
|
it; the covering tests pass). The other is a mutant Stryker scored as a runtime error, so it has
|
|
250
250
|
no Stryker verdict to compare.
|
|
251
251
|
|
|
252
|
-
**Speed
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
252
|
+
**Speed, measured on a laptop, not yet a published number** (the published ones live at
|
|
253
|
+
[burgee.interlace.tools/docs/benchmarks](https://burgee.interlace.tools/docs/benchmarks)). Medians of 3
|
|
254
|
+
head-to-head runs (`bench/burgee-pair.mts`, concurrency 4, one tool after the other, load 6–19, 2026-10-30):
|
|
255
|
+
|
|
256
|
+
| Subject | Stryker 10 | oxmutate | Wall | CPU | Tests a mutant |
|
|
257
|
+
| :-- | --: | --: | --: | --: | --: |
|
|
258
|
+
| burgee/paratext | 108.3 s | 19.0 s | 5.70x | 5.92x | 31.7 → 16.8 |
|
|
259
|
+
| burgee/controlroom | 63.4 s | 45.6 s | 1.39x | 2.55x | 7.3 → 9.2 |
|
|
260
|
+
| oxmutate itself (Stryker's mutant set) | 508 s | 334 s | 1.52x | — | — |
|
|
261
|
+
|
|
262
|
+
The ceiling depends on the suite as much as the tool: controlroom's own `vitest run` takes ~8 s and
|
|
263
|
+
every fresh worker imports React and Ink, so even zero overhead would make it ~6x. The figures to
|
|
264
|
+
publish come from `bench/burgee-pair.mts`, run locally on a quiet machine: no mutation run happens
|
|
265
|
+
in GitHub Actions (`D-20261101-no-mutation-in-gha`).
|
|
257
266
|
|
|
258
267
|
## Benchmarks
|
|
259
268
|
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/dist/checker.js
CHANGED
|
@@ -29,6 +29,7 @@ export function applyMutant(source, m) {
|
|
|
29
29
|
const starts = [0];
|
|
30
30
|
for (let i = source.indexOf('\n'); i !== -1; i = source.indexOf('\n', i + 1))
|
|
31
31
|
starts.push(i + 1);
|
|
32
|
+
// Stryker disable next-line NumericLiteral: `starts` holds every line the checker reports, so the fallback never runs
|
|
32
33
|
const at = (p) => (starts[p.line - 1] ?? 0) + p.column - 1;
|
|
33
34
|
return source.slice(0, at(m.start)) + m.replacement + source.slice(at(m.end));
|
|
34
35
|
}
|
package/dist/child.js
CHANGED
|
@@ -14,11 +14,14 @@ import { readFileSync, writeFileSync } from 'node:fs';
|
|
|
14
14
|
import path from 'node:path';
|
|
15
15
|
import { fileURLToPath } from 'node:url';
|
|
16
16
|
import { ExitCode } from 'burgee';
|
|
17
|
+
import { BEAT_ENV, beatFile, CPU_SAMPLE_MS } from './heartbeat.js';
|
|
17
18
|
import { resolveEsm } from './resolve.js';
|
|
18
|
-
import { ACTIVE_MUTANT_ENV, argv, env, exit } from './runtime.js';
|
|
19
|
+
import { ACTIVE_MUTANT_ENV, argv, env, exit, pid } from './runtime.js';
|
|
19
20
|
import { instrumentedSources } from './vite-plugin.js';
|
|
20
21
|
const config = JSON.parse(readFileSync(argv[2] ?? '', 'utf8'));
|
|
21
22
|
env['NODE_ENV'] ??= 'test';
|
|
23
|
+
// Before Vitest starts any worker, so each one inherits it (heartbeat.ts).
|
|
24
|
+
env[BEAT_ENV] = beatFile(path.dirname(config.instrumentedMap), pid);
|
|
22
25
|
const vitestNode = (await import(resolveEsm('vitest/node', config.root)));
|
|
23
26
|
const runner = fileURLToPath(new URL('./runner.js', import.meta.url));
|
|
24
27
|
const silent = { onInit() { }, onFinished() { } };
|
|
@@ -42,6 +45,7 @@ const vitest = await vitestNode.createVitest('test', {
|
|
|
42
45
|
// spellings (top-level in Vitest 5, `experimental` in 4.x).
|
|
43
46
|
fsModuleCache: false,
|
|
44
47
|
experimental: { fsModuleCache: false },
|
|
48
|
+
...(config.inlineDeps && { server: { deps: { inline: true } } }),
|
|
45
49
|
},
|
|
46
50
|
// A private Vite cache per run, so no cache outside this run can answer for a mutated file.
|
|
47
51
|
{ plugins: [instrumentedSources(config.instrumentedMap)], cacheDir: path.join(path.dirname(config.instrumentedMap), 'vite-cache') });
|
|
@@ -72,6 +76,11 @@ async function runIsolated(unit) {
|
|
|
72
76
|
try {
|
|
73
77
|
for (const item of unit.batch ?? []) {
|
|
74
78
|
env[ACTIVE_MUTANT_ENV] = item.id;
|
|
79
|
+
// The runaway guard warm runs have: a loop the mutant causes stops at the limit, not the watchdog.
|
|
80
|
+
if (item.hitLimit)
|
|
81
|
+
env['OXMUTATE_HIT_LIMIT'] = String(item.hitLimit);
|
|
82
|
+
else
|
|
83
|
+
delete env['OXMUTATE_HIT_LIMIT'];
|
|
75
84
|
if (item.tests)
|
|
76
85
|
env['OXMUTATE_ONLY_TESTS'] = JSON.stringify(item.tests);
|
|
77
86
|
else
|
|
@@ -87,6 +96,7 @@ async function runIsolated(unit) {
|
|
|
87
96
|
}
|
|
88
97
|
delete env[ACTIVE_MUTANT_ENV];
|
|
89
98
|
delete env['OXMUTATE_ONLY_TESTS'];
|
|
99
|
+
delete env['OXMUTATE_HIT_LIMIT'];
|
|
90
100
|
}
|
|
91
101
|
finally {
|
|
92
102
|
vitest.config.bail = 0;
|
|
@@ -101,6 +111,14 @@ async function runUnit(unit) {
|
|
|
101
111
|
env['OXMUTATE_PROGRESS'] = unit.progress;
|
|
102
112
|
if (unit.plan)
|
|
103
113
|
env['OXMUTATE_PLAN'] = unit.plan;
|
|
114
|
+
if (unit.staticIds)
|
|
115
|
+
env['OXMUTATE_STATIC'] = JSON.stringify(unit.staticIds);
|
|
116
|
+
else
|
|
117
|
+
delete env['OXMUTATE_STATIC'];
|
|
118
|
+
if (unit.hitLimits)
|
|
119
|
+
env['OXMUTATE_HIT_LIMITS'] = JSON.stringify(unit.hitLimits);
|
|
120
|
+
else
|
|
121
|
+
delete env['OXMUTATE_HIT_LIMITS'];
|
|
104
122
|
delete env[ACTIVE_MUTANT_ENV];
|
|
105
123
|
if (unit.mode === 'isolated')
|
|
106
124
|
await runIsolated(unit);
|
|
@@ -127,5 +145,13 @@ process.on('message', (msg) => {
|
|
|
127
145
|
// and a worker thread may still be looping in a mutant. Two such servers were found spinning at
|
|
128
146
|
// 100% CPU 11 hours after their run ended, and slowed the next run 50x (2026-10-27).
|
|
129
147
|
process.on('disconnect', () => exit(ExitCode.OK));
|
|
148
|
+
// The server's CPU time, every thread (the test worker included), for the watchdog to tell an idle
|
|
149
|
+
// stall from a loop (heartbeat.ts). This thread stays free while a worker loops, so the samples come.
|
|
150
|
+
const sendCpu = () => {
|
|
151
|
+
const { user, system } = process.cpuUsage();
|
|
152
|
+
process.send?.({ type: 'cpu', at: Date.now(), us: user + system });
|
|
153
|
+
};
|
|
154
|
+
sendCpu();
|
|
155
|
+
setInterval(sendCpu, CPU_SAMPLE_MS).unref();
|
|
130
156
|
process.send?.({ type: 'ready' });
|
|
131
157
|
//# sourceMappingURL=child.js.map
|
package/dist/coordinator.d.ts
CHANGED
|
@@ -36,10 +36,16 @@ export interface RunStats {
|
|
|
36
36
|
instrumentMs: number;
|
|
37
37
|
dryRunMs: number;
|
|
38
38
|
mutationMs: number;
|
|
39
|
+
/** Static mutants decided in live workers (section 5b), between the warm and the isolated phases. */
|
|
40
|
+
staticMs: number;
|
|
41
|
+
/** Unarmed re-collections in that phase, the guard each live static verdict waits on (runner.ts). */
|
|
42
|
+
staticRecollections: number;
|
|
39
43
|
isolatedMs: number;
|
|
40
44
|
wallMs: number;
|
|
41
45
|
testExecutions: number;
|
|
42
46
|
watchdogKills: number;
|
|
47
|
+
/** Of `watchdogKills`, the stops whose worker was idle, waiting (heartbeat.ts). In an isolated run each is a Timeout as it stands. */
|
|
48
|
+
idleStalls: number;
|
|
43
49
|
crashes: number;
|
|
44
50
|
canaryFiles: number;
|
|
45
51
|
/** Contexts restarted because a mutant left state behind (the per-execution canary fired). */
|
package/dist/coordinator.js
CHANGED
|
@@ -17,6 +17,7 @@ import { ambientRuntime, run as spawn } from 'bellpull';
|
|
|
17
17
|
import { checkSurvivors } from './checker.js';
|
|
18
18
|
import { runCommandMutants } from './command-runner.js';
|
|
19
19
|
import { disables } from './config.js';
|
|
20
|
+
import { rangeDisables } from './directives.js';
|
|
20
21
|
import { globToRegExp, matchFiles } from './files.js';
|
|
21
22
|
import { changedLines, parseMutateSpec, shardOf, touches, withinRange } from './lanes.js';
|
|
22
23
|
import { native } from './native.js';
|
|
@@ -36,8 +37,16 @@ function readRecords(file) {
|
|
|
36
37
|
.map((l) => JSON.parse(l));
|
|
37
38
|
}
|
|
38
39
|
const sha = (s) => createHash('sha256').update(s).digest('hex').slice(0, 16);
|
|
39
|
-
/**
|
|
40
|
-
|
|
40
|
+
/**
|
|
41
|
+
* Work units of at most this many (mutant, test) executions: small enough to steal, large enough
|
|
42
|
+
* that each unit's fresh worker, which imports the test file's whole graph, is paid for few units.
|
|
43
|
+
* At 120, module loading was most of the CPU on burgee/controlroom (React/Ink in every worker); 400
|
|
44
|
+
* cut units 172 -> 93 and wall 44 -> 39 s (paratext 39 -> 35 s), and 1000 gained nothing more
|
|
45
|
+
* (2026-10-30). Smaller stays better for a pollution restart, which re-runs the rest of a unit.
|
|
46
|
+
*/
|
|
47
|
+
const UNIT_PAIRS = 400;
|
|
48
|
+
/** How many times its isolated budget a busy stop gets on its re-run. */
|
|
49
|
+
const RERUN_SLACK = 3;
|
|
41
50
|
const MS_PER_MINUTE = 60_000;
|
|
42
51
|
const PERCENT = 100;
|
|
43
52
|
/** Every test that reaches a mutant: the score's, then a spec oracle's replay tests. */
|
|
@@ -55,8 +64,8 @@ export async function run(options) {
|
|
|
55
64
|
const { root } = options;
|
|
56
65
|
const work = mkdtempSync(path.join(tmpdir(), 'oxmutate-'));
|
|
57
66
|
const stats = {
|
|
58
|
-
files: 0, mutants: 0, instrumentMs: 0, dryRunMs: 0, mutationMs: 0, isolatedMs: 0, wallMs: 0, testExecutions: 0,
|
|
59
|
-
watchdogKills: 0, crashes: 0, canaryFiles: 0, pollutionRestarts: 0, survivorsOverturned: 0, timeoutsOverturned: 0, compileErrors: 0, isolatedRuns: 0, reused: 0, flakyTests: 0, flakyMutants: 0, units: 0, serversSpawned: 0,
|
|
67
|
+
files: 0, mutants: 0, instrumentMs: 0, dryRunMs: 0, mutationMs: 0, staticMs: 0, staticRecollections: 0, isolatedMs: 0, wallMs: 0, testExecutions: 0,
|
|
68
|
+
watchdogKills: 0, idleStalls: 0, crashes: 0, canaryFiles: 0, pollutionRestarts: 0, survivorsOverturned: 0, timeoutsOverturned: 0, compileErrors: 0, isolatedRuns: 0, reused: 0, flakyTests: 0, flakyMutants: 0, units: 0, serversSpawned: 0,
|
|
60
69
|
};
|
|
61
70
|
let unitSeq = 0;
|
|
62
71
|
const fileFor = (kind, ext) => path.join(work, `${kind}-${String(unitSeq++)}.${ext}`);
|
|
@@ -87,8 +96,9 @@ export async function run(options) {
|
|
|
87
96
|
const directiveErrors = [];
|
|
88
97
|
for (const f of instrumented) {
|
|
89
98
|
const { disables: offs, errors } = disables(sources[f.path] ?? '', f.path);
|
|
99
|
+
const ranged = rangeDisables(sources[f.path] ?? '', f.path);
|
|
90
100
|
const ranges = rangesFor(f.path);
|
|
91
|
-
directiveErrors.push(...errors);
|
|
101
|
+
directiveErrors.push(...errors, ...ranged.errors);
|
|
92
102
|
let any = false;
|
|
93
103
|
for (const m of f.mutants) {
|
|
94
104
|
if (changed && !touches(changed, f.path, m.start.line, m.end.line))
|
|
@@ -98,13 +108,14 @@ export async function run(options) {
|
|
|
98
108
|
if (options.shard && shardOf(m.id, options.shard.count) !== options.shard.index)
|
|
99
109
|
continue;
|
|
100
110
|
const off = offs.find((d) => d.line === m.start.line && (d.mutators === 'all' || d.mutators.includes(m.mutatorName)));
|
|
111
|
+
const inRange = ranged.reasonFor(m.start.line, m.mutatorName);
|
|
101
112
|
const excluded = options.excludedMutations.includes(m.mutatorName);
|
|
102
113
|
mutants.push({
|
|
103
114
|
...m,
|
|
104
115
|
file: f.path,
|
|
105
116
|
coveredBy: [],
|
|
106
|
-
status: off || excluded || m.ignoredReason ? 'Ignored' : 'NoCoverage',
|
|
107
|
-
statusReason: off?.reason ?? m.ignoredReason ?? (excluded ? `excluded mutator ${m.mutatorName}` : undefined),
|
|
117
|
+
status: off || inRange !== undefined || excluded || m.ignoredReason ? 'Ignored' : 'NoCoverage',
|
|
118
|
+
statusReason: off?.reason ?? inRange ?? m.ignoredReason ?? (excluded ? `excluded mutator ${m.mutatorName}` : undefined),
|
|
108
119
|
});
|
|
109
120
|
any = true;
|
|
110
121
|
}
|
|
@@ -214,8 +225,10 @@ export async function run(options) {
|
|
|
214
225
|
for (const [testId, perMutant] of Object.entries(dry.perTest)) {
|
|
215
226
|
if (testId.startsWith(FILE_KEY)) {
|
|
216
227
|
const file = testId.slice(FILE_KEY.length);
|
|
217
|
-
for (const id of Object.
|
|
228
|
+
for (const [id, n] of Object.entries(perMutant)) {
|
|
218
229
|
(staticFiles.get(id) ?? staticFiles.set(id, new Set()).get(id))?.add(file);
|
|
230
|
+
hits.set(id, (hits.get(id) ?? 0) + n);
|
|
231
|
+
}
|
|
219
232
|
continue;
|
|
220
233
|
}
|
|
221
234
|
for (const [id, n] of Object.entries(perMutant)) {
|
|
@@ -294,7 +307,9 @@ export async function run(options) {
|
|
|
294
307
|
// false kills on burgee/paratext (an import-time registration the tests then read).
|
|
295
308
|
const toRun = mutants.filter((m) => !m.reused && m.status !== 'Ignored' && !m.static && allCovering(m).length > 0);
|
|
296
309
|
const byId = new Map(mutants.map((m) => [m.id, m]));
|
|
297
|
-
|
|
310
|
+
/** The switch-taken count past which a run is a runaway: 100x what the dry run saw, at least 1000. */
|
|
311
|
+
const hitLimitOf = (id) => Math.max(1000, (hits.get(id) ?? 1) * 100);
|
|
312
|
+
const hitLimits = Object.fromEntries(toRun.map((m) => [m.id, hitLimitOf(m.id)]));
|
|
298
313
|
const maxTestMs = (id) => { const m = byId.get(id); return Math.max(0, ...(m ? allCovering(m) : []).map((t) => testInfo.get(t)?.ms ?? 0)); };
|
|
299
314
|
// The watchdog is for runaways, which loop in fast code; a slow test under a loaded machine is
|
|
300
315
|
// not one. 3x the covering test's dry-run time, not Stryker's 1.5x: at 1.5x, subprocess-heavy
|
|
@@ -357,7 +372,8 @@ export async function run(options) {
|
|
|
357
372
|
const records = [];
|
|
358
373
|
const runaways = new Set();
|
|
359
374
|
const crashedIds = new Set();
|
|
360
|
-
|
|
375
|
+
// Dependencies inlined here too, so the static phase (5b) reuses these servers instead of starting its own.
|
|
376
|
+
const pool = createPool({ root, vitestConfig: options.vitestConfig, workers: 1, instrumentedMap: mapFile, nodeArgs: options.testRunnerNodeArgs, inlineDeps: true }, options.concurrency, work, 'mutant');
|
|
361
377
|
pools.push(pool);
|
|
362
378
|
/** Mutants decided on one verdict track: the tests', or a spec oracle's replay tests'. */
|
|
363
379
|
const killedSoFar = (spec) => new Set([...runaways, ...crashedIds, ...records.filter((r) => r.type === 'mutant' && r.status !== 'Survived' && Boolean(r.spec) === spec).map((r) => r.id ?? '')]);
|
|
@@ -410,7 +426,29 @@ export async function run(options) {
|
|
|
410
426
|
// One round, every covering file at once. A two-round plan (cheapest file first, survivors
|
|
411
427
|
// to the rest) cut executions 23% on burgee/paratext but took 2.9x longer: the second round
|
|
412
428
|
// waits for the first to drain, and the expensive files then run as a tail (2026-10-27).
|
|
413
|
-
|
|
429
|
+
// Kills are filtered out at dispatch, not when a unit is built: most units wait in the queue while
|
|
430
|
+
// others run, and a mutant another unit has killed (vouched for by its canary) needs no more of its
|
|
431
|
+
// covering tests. 1,022 of 6,550 warm executions on burgee/paratext ran after the mutant's kill
|
|
432
|
+
// (2026-10-30). Verdicts are unchanged: a Killed mutant needs no other pair (dispatch.qnt).
|
|
433
|
+
const beforeUnit = (unit) => {
|
|
434
|
+
const entry = planUnits.get(unit.results);
|
|
435
|
+
if (!entry)
|
|
436
|
+
return true;
|
|
437
|
+
const decided = { tests: killedSoFar(false), spec: killedSoFar(true) };
|
|
438
|
+
const left = {};
|
|
439
|
+
for (const [t, ids] of Object.entries(entry.plan)) {
|
|
440
|
+
const done = specOf.has(t) ? decided.spec : decided.tests;
|
|
441
|
+
const keep = ids.filter((id) => !done.has(id));
|
|
442
|
+
if (keep.length)
|
|
443
|
+
left[t] = keep;
|
|
444
|
+
}
|
|
445
|
+
if (Object.keys(left).length === 0)
|
|
446
|
+
return false;
|
|
447
|
+
entry.plan = left;
|
|
448
|
+
writeFileSync(unit.plan, JSON.stringify({ tests: left, hitLimits, skip: [], specTests: [...specOf.keys()] }));
|
|
449
|
+
return true;
|
|
450
|
+
};
|
|
451
|
+
await pool.drain(unitsForRound(toRun, (m) => fileOrder.get(m.id) ?? []), (_, id) => warmBudget(id), afterUnit, beforeUnit);
|
|
414
452
|
stats.mutationMs = performance.now() - m0;
|
|
415
453
|
// ── 5. Verdicts from the warm run ─────────────────────────────────────────────────────────
|
|
416
454
|
const byMutant = new Map();
|
|
@@ -449,9 +487,7 @@ export async function run(options) {
|
|
|
449
487
|
isolatedQueue.push(m); // unfinished: a crash left work undone
|
|
450
488
|
}
|
|
451
489
|
}
|
|
452
|
-
|
|
453
|
-
if (m.static && m.status !== 'Ignored' && !m.reused)
|
|
454
|
-
isolatedQueue.push(m);
|
|
490
|
+
const staticMutants = mutants.filter((m) => m.static && m.status !== 'Ignored' && !m.reused);
|
|
455
491
|
// ── 6. Isolated re-runs: the switch armed before import, Vitest run normally ──────────────
|
|
456
492
|
const allTestFiles = [...new Set(dry.tests.map((t) => t.file))];
|
|
457
493
|
// A static mutant needs the files that load it (plus any whose tests reach it); only a hit
|
|
@@ -460,6 +496,21 @@ export async function run(options) {
|
|
|
460
496
|
for (const t of dry.tests)
|
|
461
497
|
fileCost.set(t.file, (fileCost.get(t.file) ?? 0) + t.ms);
|
|
462
498
|
const byCost = (files) => [...files].sort((a, b) => (fileCost.get(a) ?? 0) - (fileCost.get(b) ?? 0));
|
|
499
|
+
// Test file -> source file -> how many of that source's mutants its tests reach. A static mutant
|
|
500
|
+
// rarely has covering tests of its own, so its loaders were tried cheapest first, and the cheapest
|
|
501
|
+
// seldom tests the module: on burgee/controlroom each tabs.ts static mutant ran 75 tests in two
|
|
502
|
+
// files where Stryker needed 37 (2026-10-30). The file that exercises the module goes first.
|
|
503
|
+
const affinity = new Map();
|
|
504
|
+
for (const mm of mutants) {
|
|
505
|
+
for (const t of mm.coveredBy) {
|
|
506
|
+
const f = testInfo.get(t)?.file;
|
|
507
|
+
if (!f)
|
|
508
|
+
continue;
|
|
509
|
+
const bySource = affinity.get(f) ?? affinity.set(f, new Map()).get(f);
|
|
510
|
+
bySource.set(mm.file, (bySource.get(mm.file) ?? 0) + 1);
|
|
511
|
+
}
|
|
512
|
+
}
|
|
513
|
+
const byAffinity = (source, files) => byCost(files).sort((a, b) => (affinity.get(b)?.get(source) ?? 0) - (affinity.get(a)?.get(source) ?? 0));
|
|
463
514
|
const filesFor = (m) => {
|
|
464
515
|
const tests = m.coveredBy.map((t) => testInfo.get(t)?.file ?? '');
|
|
465
516
|
if (!m.static)
|
|
@@ -471,27 +522,121 @@ export async function run(options) {
|
|
|
471
522
|
// ran a file that only imports it first, and 288 of 349 isolated kills on burgee/paratext
|
|
472
523
|
// came from a later file (2026-10-27).
|
|
473
524
|
const covering = byCost([...new Set(tests)]);
|
|
474
|
-
return [...covering, ...
|
|
525
|
+
return [...covering, ...byAffinity(m.file, [...loaders].filter((f) => !covering.includes(f)))];
|
|
475
526
|
};
|
|
476
527
|
const isolatedBudget = (id) => {
|
|
477
528
|
const m = byId.get(id);
|
|
478
529
|
const fs = m ? filesFor(m) : allTestFiles;
|
|
479
|
-
|
|
530
|
+
// A non-static mutant's isolated run executes only its covering tests (`tests` below), so they
|
|
531
|
+
// are its budget; every test of its files made a hang wait 30-40 s on oxmutate's own suite,
|
|
532
|
+
// where its covering tests take a fraction of that (2026-10-30). A static mutant runs them all.
|
|
533
|
+
const covering = new Set(m && !m.static ? m.coveredBy : []);
|
|
534
|
+
const ms = dry.tests.filter((t) => fs.includes(t.file) && (covering.size === 0 || covering.has(t.id))).reduce((sum, t) => sum + t.ms, 0);
|
|
480
535
|
return 1500 + options.timeoutMS + options.timeoutFactor * ms * 2;
|
|
481
536
|
};
|
|
482
537
|
// A non-static mutant runs only its covering tests, armed across all of them in file order:
|
|
483
538
|
// Stryker's semantics, so cross-test effects still show (a no-op reset() lets state pile up
|
|
484
539
|
// for the next covering test). A static mutant runs every test of its files, because an
|
|
485
540
|
// import-time change reaches tests that never execute the mutated line.
|
|
541
|
+
/**
|
|
542
|
+
* Static mutants decided in live workers, one unit per (test file, batch of its static mutants),
|
|
543
|
+
* on a pool that inlines dependencies. Returns the mutants this path could not decide.
|
|
544
|
+
*/
|
|
545
|
+
const decideStaticLive = async (items) => {
|
|
546
|
+
if (items.length === 0)
|
|
547
|
+
return [];
|
|
548
|
+
const STATIC_BATCH = 25;
|
|
549
|
+
/** Units of up to STATIC_BATCH mutants per test file, for the files each mutant gets this round. */
|
|
550
|
+
const unitsFor = (filesOf) => {
|
|
551
|
+
const byFile = new Map();
|
|
552
|
+
for (const m of items)
|
|
553
|
+
for (const f of filesOf(m))
|
|
554
|
+
(byFile.get(f) ?? byFile.set(f, []).get(f))?.push(m.id);
|
|
555
|
+
return [...byFile].flatMap(([file, ids]) => Array.from({ length: Math.ceil(ids.length / STATIC_BATCH) }, (_, i) => ({
|
|
556
|
+
mode: 'static', files: [file], results: fileFor('static', 'jsonl'), progress: fileFor('static', 'progress'), staticIds: ids.slice(i * STATIC_BATCH, (i + 1) * STATIC_BATCH),
|
|
557
|
+
hitLimits: Object.fromEntries(ids.slice(i * STATIC_BATCH, (i + 1) * STATIC_BATCH).map((id) => [id, hitLimitOf(id)])),
|
|
558
|
+
})));
|
|
559
|
+
};
|
|
560
|
+
const killed = new Map();
|
|
561
|
+
/** Decided by the hit-count limit: a runaway, Timeout like a warm one. */
|
|
562
|
+
const timedOut = new Set();
|
|
563
|
+
const passed = new Map(); // mutant -> files that ran it clean
|
|
564
|
+
const back = new Set();
|
|
565
|
+
const ran = new Map();
|
|
566
|
+
const staticPool = pool;
|
|
567
|
+
const afterStatic = (unit, outcome) => {
|
|
568
|
+
const file = unit.files[0] ?? '';
|
|
569
|
+
const decided = new Set();
|
|
570
|
+
for (const r of readRecords(unit.results)) {
|
|
571
|
+
if (r.type === 'recollection')
|
|
572
|
+
stats.staticRecollections++;
|
|
573
|
+
if (r.type === 'static-fallback')
|
|
574
|
+
for (const id of r.ids ?? [])
|
|
575
|
+
back.add(id);
|
|
576
|
+
if (r.type !== 'isolated' || !r.id)
|
|
577
|
+
continue;
|
|
578
|
+
decided.add(r.id);
|
|
579
|
+
ran.set(r.id, (ran.get(r.id) ?? 0) + (r.ran ?? 0));
|
|
580
|
+
if (r.timeout)
|
|
581
|
+
timedOut.add(r.id);
|
|
582
|
+
if (r.killed || r.timeout)
|
|
583
|
+
killed.set(r.id, { killedBy: [...(killed.get(r.id)?.killedBy ?? []), ...(r.failed ?? [])], ran: 0 });
|
|
584
|
+
else
|
|
585
|
+
(passed.get(r.id) ?? passed.set(r.id, new Set()).get(r.id))?.add(file);
|
|
586
|
+
}
|
|
587
|
+
// A stop or a crash leaves the rest of the batch undecided: the fresh-worker path decides it.
|
|
588
|
+
if (outcome.killedFor || outcome.crashed)
|
|
589
|
+
for (const id of unit.staticIds ?? [])
|
|
590
|
+
if (!decided.has(id))
|
|
591
|
+
back.add(id);
|
|
592
|
+
return [];
|
|
593
|
+
};
|
|
594
|
+
const keepUndecided = (unit) => {
|
|
595
|
+
unit.staticIds = (unit.staticIds ?? []).filter((id) => !killed.has(id) && !back.has(id));
|
|
596
|
+
return unit.staticIds.length > 0;
|
|
597
|
+
};
|
|
598
|
+
// Two rounds: each mutant first in the file most likely to kill it (filesFor puts the files whose
|
|
599
|
+
// tests reach it first, cheapest first), then its other files only if it survived. In one round
|
|
600
|
+
// every loading file ran a mutant at once, and a kill in one did not spare the others: 41 tests a
|
|
601
|
+
// static kill on burgee/controlroom against Stryker's 33.6 (2026-10-30).
|
|
602
|
+
const open = (m) => !killed.has(m.id) && !back.has(m.id);
|
|
603
|
+
const rounds = [(m) => filesFor(m).slice(0, 1), (m) => (open(m) ? filesFor(m).slice(1) : [])];
|
|
604
|
+
// the mutant's run and the unarmed canary after it
|
|
605
|
+
for (const round of rounds)
|
|
606
|
+
await staticPool.drain(unitsFor(round), (_, id) => 2 * isolatedBudget(id), afterStatic, keepUndecided);
|
|
607
|
+
const undecided = [];
|
|
608
|
+
for (const m of items) {
|
|
609
|
+
const k = killed.get(m.id);
|
|
610
|
+
if (k) {
|
|
611
|
+
m.status = timedOut.has(m.id) && !k.killedBy.length ? 'Timeout' : 'Killed';
|
|
612
|
+
m.killedBy = [...new Set(k.killedBy)];
|
|
613
|
+
m.testsCompleted = ran.get(m.id) ?? 0;
|
|
614
|
+
}
|
|
615
|
+
else if (!back.has(m.id) && filesFor(m).every((f) => passed.get(m.id)?.has(f))) {
|
|
616
|
+
m.status = 'Survived';
|
|
617
|
+
m.testsCompleted = ran.get(m.id) ?? 0;
|
|
618
|
+
}
|
|
619
|
+
else
|
|
620
|
+
undecided.push(m);
|
|
621
|
+
}
|
|
622
|
+
return undecided;
|
|
623
|
+
};
|
|
486
624
|
const runIsolated = async (items, slack = 1) => {
|
|
487
625
|
const verdicts = new Map();
|
|
488
626
|
if (items.length === 0)
|
|
489
627
|
return verdicts;
|
|
490
|
-
const
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
628
|
+
const mk = (batch) => ({ mode: 'isolated', files: [], results: fileFor('iso', 'jsonl'), progress: fileFor('iso', 'progress'), batch: batch.map((m) => ({ id: m.id, files: filesFor(m), tests: m.static ? undefined : m.coveredBy, hitLimit: hitLimitOf(m.id) })) });
|
|
629
|
+
// Packed by estimated cost, heaviest first into the lightest bin, and the heaviest bins run
|
|
630
|
+
// first. Equal mutant counts left one unit at 8.4 s while the others finished in under 3 s on
|
|
631
|
+
// burgee/controlroom (2026-10-30): a mutant's cost is its files' dry-run time, not one.
|
|
632
|
+
const cost = (m) => isolatedBudget(m.id);
|
|
633
|
+
const bins = Array.from({ length: Math.min(items.length, options.concurrency * 2) }, () => ({ load: 0, batch: [] }));
|
|
634
|
+
for (const m of [...items].sort((x, y) => cost(y) - cost(x))) {
|
|
635
|
+
const bin = bins.reduce((lightest, b) => (b.load < lightest.load ? b : lightest));
|
|
636
|
+
bin.load += cost(m);
|
|
637
|
+
bin.batch.push(m);
|
|
638
|
+
}
|
|
639
|
+
const units = bins.sort((x, y) => y.load - x.load).map((b) => mk(b.batch));
|
|
495
640
|
stats.isolatedRuns += items.length;
|
|
496
641
|
await pool.drain(units, (_, id) => slack * isolatedBudget(id), (unit, outcome) => {
|
|
497
642
|
// Vitest reports per test file: a mutant is killed if any of its files failed.
|
|
@@ -500,13 +645,15 @@ export async function run(options) {
|
|
|
500
645
|
continue;
|
|
501
646
|
const prev = verdicts.get(rec.id);
|
|
502
647
|
const testsCompleted = (prev?.testsCompleted ?? 0) + (rec.ran ?? 0);
|
|
503
|
-
if (rec.
|
|
648
|
+
if (rec.timeout)
|
|
649
|
+
verdicts.set(rec.id, { status: 'Timeout', testsCompleted });
|
|
650
|
+
else if (rec.killed)
|
|
504
651
|
verdicts.set(rec.id, { status: 'Killed', killedBy: [...(prev?.killedBy ?? []), ...(rec.failed ?? [])], testsCompleted });
|
|
505
652
|
else
|
|
506
653
|
verdicts.set(rec.id, { status: prev?.status ?? 'Survived', killedBy: prev?.killedBy, testsCompleted });
|
|
507
654
|
}
|
|
508
655
|
if (outcome.killedFor)
|
|
509
|
-
verdicts.set(outcome.killedFor, { status: 'Timeout' });
|
|
656
|
+
verdicts.set(outcome.killedFor, { status: 'Timeout', idle: outcome.stall === 'idle' });
|
|
510
657
|
if (outcome.crashed && outcome.crashedFor)
|
|
511
658
|
verdicts.set(outcome.crashedFor, { status: 'RuntimeError' });
|
|
512
659
|
if (!outcome.killedFor && !outcome.crashed)
|
|
@@ -520,14 +667,27 @@ export async function run(options) {
|
|
|
520
667
|
verdicts.set(m.id, { status: 'RuntimeError' });
|
|
521
668
|
return verdicts;
|
|
522
669
|
};
|
|
670
|
+
// ── 5b. Static mutants in live workers (.sdlc/intents/oxmutate/staticwarm.qnt) ─────────────
|
|
671
|
+
// A fresh worker per (static mutant, file) cost ~80 ms of start-up around tests that take a few
|
|
672
|
+
// ms: 29 of 35 s on burgee/paratext (2026-10-30). A live worker re-evaluates the project's modules
|
|
673
|
+
// per mutant instead; whatever it cannot vouch for comes back here for the fresh-worker path.
|
|
674
|
+
// OXMUTATE_ISOLATE_STATIC=1 sends every static mutant down the fresh-worker path, as before.
|
|
675
|
+
const s0 = performance.now();
|
|
676
|
+
const staticBack = env['OXMUTATE_ISOLATE_STATIC'] ? staticMutants : await decideStaticLive(staticMutants);
|
|
677
|
+
stats.staticMs = performance.now() - s0;
|
|
678
|
+
isolatedQueue.push(...staticBack);
|
|
523
679
|
progress.phase(`isolated runs (static mutants and stragglers: ${String(isolatedQueue.length)})`);
|
|
524
680
|
const i0 = performance.now();
|
|
525
681
|
const first = await runIsolated(isolatedQueue);
|
|
526
682
|
// A watchdog stop on a loaded machine is a slow test, not a runaway: 31 of them on one
|
|
527
683
|
// burgee/paratext run while the load average sat at 40, all Survived for Stryker (2026-10-27).
|
|
528
684
|
// Each stop gets one more run with 3x the budget; only one that is stopped again is a Timeout.
|
|
529
|
-
|
|
530
|
-
|
|
685
|
+
// An idle stop is not re-run: its worker was waiting with its event loop turning and no CPU spent,
|
|
686
|
+
// on something the mutant never lets happen, which no budget cures; it is a Timeout as Stryker
|
|
687
|
+
// scores it. Re-run at 3x, 375 of them on caique's 10-31 run "finished" only because Vitest's 30 s
|
|
688
|
+
// test timeout failed the test (D-20261101-idle-stall-is-timeout, stall.qnt).
|
|
689
|
+
const stopped = isolatedQueue.filter((m) => first.get(m.id)?.status === 'Timeout' && !first.get(m.id)?.idle);
|
|
690
|
+
const again = await runIsolated(stopped, RERUN_SLACK);
|
|
531
691
|
for (const m of stopped) {
|
|
532
692
|
const v = again.get(m.id);
|
|
533
693
|
if (v && v.status !== 'Timeout' && v.status !== 'RuntimeError') {
|
|
@@ -542,7 +702,7 @@ export async function run(options) {
|
|
|
542
702
|
m.killedBy = v.killedBy;
|
|
543
703
|
m.testsCompleted = v.testsCompleted;
|
|
544
704
|
if (v.status === 'Timeout')
|
|
545
|
-
m.statusReason = 'stopped by the watchdog twice, the second time with 3x the budget';
|
|
705
|
+
m.statusReason = v.idle ? 'its test waited past the budget with the worker idle: a hang, not a slow test' : 'stopped by the watchdog twice, the second time with 3x the budget';
|
|
546
706
|
}
|
|
547
707
|
}
|
|
548
708
|
// ── 6b. Confirm survivors under whole-file semantics ──────────────────────────────────────
|
|
@@ -551,13 +711,16 @@ export async function run(options) {
|
|
|
551
711
|
// survives warm and is killed when armed for the whole file, the way Stryker runs it (burgee/
|
|
552
712
|
// paratext plugin.ts, 2026-10-27). Survivors are what a person acts on, so each one is
|
|
553
713
|
// re-run once with the mutant armed across its covering files before it is reported.
|
|
554
|
-
|
|
714
|
+
// A survivor with one covering test is not re-run: confirmation runs only covering tests, so there
|
|
715
|
+
// is no later test for its leftover state to reach, and the warm run already armed that test
|
|
716
|
+
// through its hooks from clean state (.sdlc/intents/oxmutate/confirm.qnt).
|
|
717
|
+
const survivors = toRun.filter((m) => m.status === 'Survived' && new Set(m.coveredBy).size > 1);
|
|
555
718
|
const confirmed = await runIsolated(survivors);
|
|
556
719
|
// A timeout here is a watchdog stop like any other, and a loaded machine makes them: two
|
|
557
720
|
// closeout survivors were reported killed this way while a docs build shared the machine
|
|
558
721
|
// (2026-10-27). The same rule as section 6: one more run at 3x the budget before believing it.
|
|
559
|
-
const stoppedHere = survivors.filter((m) => confirmed.get(m.id)?.status === 'Timeout');
|
|
560
|
-
const recheck = await runIsolated(stoppedHere,
|
|
722
|
+
const stoppedHere = survivors.filter((m) => confirmed.get(m.id)?.status === 'Timeout' && !confirmed.get(m.id)?.idle);
|
|
723
|
+
const recheck = await runIsolated(stoppedHere, RERUN_SLACK);
|
|
561
724
|
for (const m of stoppedHere) {
|
|
562
725
|
const v = recheck.get(m.id);
|
|
563
726
|
if (v)
|
|
@@ -609,6 +772,7 @@ export async function run(options) {
|
|
|
609
772
|
}
|
|
610
773
|
stats.isolatedMs = performance.now() - i0;
|
|
611
774
|
stats.watchdogKills = pool.stats.killed;
|
|
775
|
+
stats.idleStalls = pool.stats.idle;
|
|
612
776
|
stats.crashes = pool.stats.crashed;
|
|
613
777
|
// ── 8. Score, cache, result ───────────────────────────────────────────────────────────────
|
|
614
778
|
if (options.incremental) {
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Stryker's range directives: a `disable` comment naming mutators (or `all`) and a reason turns them
|
|
3
|
+
* off from the next line on, until a `restore` comment naming them (either spelled with `Stryker` or
|
|
4
|
+
* `oxmutate`). The `next-line` form is config.ts's `disables`. Before this, oxmutate read only the
|
|
5
|
+
* `next-line` form and graded every mutant inside a range as if no directive were there: 116 such
|
|
6
|
+
* mutants in compat-oracle's `hosts.ts` alone (.sdlc/intents/oxmutate/sc8-profile.md).
|
|
7
|
+
*
|
|
8
|
+
* Read by line, as written. Stryker reads a directive only as the leading comment of the next AST
|
|
9
|
+
* node, so a `restore` that closes an object (`// Stryker restore all` above its `}`) is invisible to
|
|
10
|
+
* it, and its `disable` then runs to the end of the file. oxmutate does not reproduce that: the
|
|
11
|
+
* directive means what it says.
|
|
12
|
+
*/
|
|
13
|
+
const HEAD = /\/\/\s*(?:Stryker|oxmutate)\s+(disable|restore)\s+(\S.*)/i;
|
|
14
|
+
/** `all`, or a comma list of mutator names. */
|
|
15
|
+
const LIST = /^(?:all|\w+(?:\s*,\s*\w+)*)$/i;
|
|
16
|
+
const covers = (r, mutator) => r.mutators === 'all' || r.mutators.includes(mutator);
|
|
17
|
+
/** The range directives of a file. A `disable` without a reason is an error, as `next-line`'s is. */
|
|
18
|
+
export function rangeDisables(source, file) {
|
|
19
|
+
const rules = [];
|
|
20
|
+
const errors = [];
|
|
21
|
+
source.split('\n').forEach((text, i) => {
|
|
22
|
+
const m = HEAD.exec(text);
|
|
23
|
+
if (!m)
|
|
24
|
+
return;
|
|
25
|
+
// `next-line` is not a list (the hyphen), so that form is left to config.ts here too.
|
|
26
|
+
const body = m[2];
|
|
27
|
+
const colon = body.indexOf(':');
|
|
28
|
+
const list = (colon === -1 ? body : body.slice(0, colon)).trim();
|
|
29
|
+
if (!LIST.test(list))
|
|
30
|
+
return;
|
|
31
|
+
const kind = m[1].toLowerCase();
|
|
32
|
+
const reason = colon === -1 ? '' : body.slice(colon + 1).trim();
|
|
33
|
+
if (kind === 'disable' && !reason) {
|
|
34
|
+
errors.push(`${file}:${String(i + 1)}: disable comment without a reason. fix: add ": <why these mutants cannot be killed>"`);
|
|
35
|
+
return;
|
|
36
|
+
}
|
|
37
|
+
const mutators = list.toLowerCase() === 'all' ? 'all' : list.split(',').map((s) => s.trim().toLowerCase());
|
|
38
|
+
rules.push({ line: i + 1, kind, mutators, reason });
|
|
39
|
+
});
|
|
40
|
+
return {
|
|
41
|
+
errors,
|
|
42
|
+
reasonFor(line, mutator) {
|
|
43
|
+
const name = mutator.toLowerCase();
|
|
44
|
+
// The latest directive before the line that names this mutator decides it.
|
|
45
|
+
for (let k = rules.length - 1; k >= 0; k--) {
|
|
46
|
+
const r = rules[k];
|
|
47
|
+
if (r.line >= line || !covers(r, name))
|
|
48
|
+
continue;
|
|
49
|
+
return r.kind === 'disable' ? r.reason : undefined;
|
|
50
|
+
}
|
|
51
|
+
return undefined;
|
|
52
|
+
},
|
|
53
|
+
};
|
|
54
|
+
}
|
|
55
|
+
//# sourceMappingURL=directives.js.map
|