probatio 0.1.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,7 +5,7 @@ Probatio (Latin: a testing, a proof) is a free testing toolkit under the MIT lic
5
5
  It remembers what was fixed so agents with small context do not redo or undo work, and it shows
6
6
  which tests already touch a change.
7
7
 
8
- Status (2026-10-06): `mutate run` discovers the project suite and scores it. Node, pytest, C (when the binary was built with LLVM coverage), Go, Maven/Java, Rust, and C# collect a line map on the baseline. A line no test executes is `no coverage`, and the suite is not started. A line that runs only in a child process the map cannot see is `no coverage`. A suite with no line map, a baseline of 5 seconds or more, and more than 30 mutants stops before the first mutant. `verify-change` names the tests the line map ran. `ran` stays the direct-importer list. `mutate tally` does not delete a test. A kill is an assertion the suite already had. A sealed miss means the suite did not see the bug. The kill count is not a merge gate. On Commons CSV (Apache-2.0, commit `2c83a308`), two sealed runs agreed: with the new test file hidden, `CSVFormatTest.testFormatThrowsNullPointerException` failed on the reverted printer. `deletedTests` stayed 0. That older JUnit platform did not write a line map, so the first pass was the whole suite and confirm reran the older test. Auspex, markupsafe, and a hand-planted fixture are not that result. The package version in this repository is 0.1.0. The copy on npm is 0.0.5. The BugsInPy search row is still open, and the rows that apply only when a sealed run misses do not apply, because this catch held. The MCP server speaks one JSON object per line and returns the same JSON as the CLI.
8
+ Status (2026-10-06): `mutate run` discovers the project suite and scores it. Node, pytest, C (when the binary was built with LLVM coverage), Go, Maven/Java, Rust, C#, and Mocha collect a line map on the baseline. Mocha's map comes from its own root hooks. Loading the node:test collector under Mocha does not name Mocha tests. A line no test executes is `no coverage`, and the suite is not started. A plain Node script can write the lines it ran, and the parent test that reads that dump stores them under its own name. A child the parent does not read stays `no coverage`, and the suite is not started. A suite with no line map, a baseline of 5 seconds or more, and more than 30 mutants stops before the first mutant. `verify-change` names the tests the line map ran. It scores `src/`, a `lib/` file when the package tests import `lib/`, and a Go file beside `go.mod`. Tests, docs, `dist/`, and `node_modules` stay unseen. `ran` stays the direct-importer list. Covered mutants of one Node file run one after another inside the suite process that is already running. `--workers` stays 1. A Node test that sets its own `timeout` keeps that time when `--test-timeout-ms` is shorter. A mutant that crashes or leaves that process dirty ends it, and the next mutant starts clean. An uncovered line still does not start the suite. `mutate tally` does not delete a test. A kill is an assertion the suite already had. A sealed miss means the suite did not see the bug. The kill count is not a merge gate. On Commons CSV (Apache-2.0, commit `2c83a308`), two sealed runs agreed: with the new test file hidden, `CSVFormatTest.testFormatThrowsNullPointerException` failed on the reverted printer. `deletedTests` stayed 0. That older JUnit platform did not write a line map, so the first pass was the whole suite and confirm reran the older test. Auspex, markupsafe, and a hand-planted fixture are not that result. The package version in this repository is 0.1.2. npm latest is 0.1.0. The BugsInPy search row is still open, and the rows that apply only when a sealed run misses do not apply, because this catch held. The MCP server speaks one JSON object per line and returns the same JSON as the CLI.
9
9
 
10
10
  Read in this order:
11
11
 
@@ -31,7 +31,7 @@ npx probatio mutate tally --out .probatio/runs
31
31
 
32
32
  `run` discovers the suite: `run_tests.sh`, Cargo, Go, Swift, Maven, dotnet, pytest, unittest, node:test, Mocha, then `make test` when COBOL tests sit under that Makefile. An unknown layout stops and asks for `--suite-command`. It does not compile one file and call that the suite. `make test` that would curl or wget a missing file stops, and nothing is fetched. A worktree that has no `node_modules` uses the main checkout's. Omitting `--build` runs no build. Pass `--build` with a command when the suite needs one first.
33
33
 
34
- Node, pytest, Go, Maven/Java, Rust, and C# collect a line map on the baseline. C collects one when LLVM coverage was instrumented. The map stores the file, the line, and the test names that hit that line. A later mutant runs only those tests (`go test -run`, Maven `-Dtest`, `cargo test`, `dotnet test --filter`). A line no test executed is `no coverage`, and the suite is not started. A child process the map cannot see is `no coverage`. A baseline of 5 seconds or more, with no line map and more than 30 mutants still pending, stops before the first mutant. `next` names that baseline and tells you to narrow `--src` or pass a smaller patch directory. A one-file C, C++, Java, COBOL, or assembly launcher is not a suite discovery returns.
34
+ Node, pytest, Go, Maven/Java, Rust, C#, and Mocha collect a line map on the baseline. C collects one when LLVM coverage was instrumented. Mocha writes its own `{ files }` record from root hooks, keyed by the Mocha title. The node:test collector is a different file and does not name Mocha tests. The map stores the file, the line, and the test names that hit that line. A later mutant runs only those tests (`go test -run`, Maven `-Dtest`, `cargo test`, `dotnet test --filter`, Mocha `--grep`). A line no test executed is `no coverage`, and the suite is not started. A plain Node script can leave a nameless list of the lines it ran. The parent test that reads that list stores them under its own name. A child the parent does not read stays `no coverage`. Covered mutants of one Node file are applied one after another in the suite process already running, and the line map is reset between them. `--workers` stays 1. A crash or a process left dirty ends that process. The next mutant starts clean. A baseline of 5 seconds or more, with no line map and more than 30 mutants still pending, stops before the first mutant. `next` names that baseline and tells you to narrow `--src` or pass a smaller patch directory. A one-file C, C++, Java, COBOL, or assembly launcher is not a suite discovery returns.
35
35
 
36
36
  The mutant timeout is the baseline duration times 5, and at least 20 seconds, capped by `--suite-timeout-ms`. A timeout is not a kill. `--budget-ms` stops mutant work after the baseline. The first mutant still runs, except for that no-line-map stop. A kill is the test that failed, or a compiler token when the mutant did not build. Confirm is on by default and reruns the failing names. Pytest names that `-k` cannot express are passed as node ids. A usage error, including pytest exit 4, is not a kill. `verify-change` does not confirm a second time. Add `--affected` to limit the file list to tests that can see the change. The default, once a line map exists, runs the tests on the changed line.
37
37
 
@@ -0,0 +1,34 @@
1
+ // Nameless line dump for a plain script. This is not a node:test process and it
2
+ // does not write a coverage shard. The parent test's afterEach reads the dump.
3
+ import { randomBytes } from "node:crypto"
4
+ import { mkdirSync, writeFileSync } from "node:fs"
5
+ import path from "node:path"
6
+ import { openLineSampler } from "./precise-lines.mjs"
7
+
8
+ const mapPath = process.env.PROBATIO_COVERAGE_MAP
9
+ if (mapPath) {
10
+ let sampler = null
11
+ try {
12
+ sampler = await openLineSampler((rel) => rel.endsWith("child-lines.mjs") || rel.endsWith("precise-lines.mjs"))
13
+ } catch (err) {
14
+ writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
15
+ }
16
+ if (sampler) {
17
+ let wrote = false
18
+ const dump = () => {
19
+ if (wrote) return
20
+ wrote = true
21
+ sampler.positiveLines().then((hits) => {
22
+ const dir = `${mapPath}.children`
23
+ mkdirSync(dir, { recursive: true })
24
+ writeFileSync(
25
+ path.join(dir, `${process.pid}-${randomBytes(4).toString("hex")}.json`),
26
+ JSON.stringify({ hits }),
27
+ )
28
+ }).catch((err) => {
29
+ writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
30
+ })
31
+ }
32
+ process.on("beforeExit", dump)
33
+ }
34
+ }
@@ -0,0 +1,71 @@
1
+ // Mocha root hooks. This file is not node-coverage.mjs. Mocha does not run
2
+ // node:test afterEach/after, so that collector records no Mocha title.
3
+ // These hooks key inspector hits by the current Mocha title and write { files }.
4
+ const { mkdirSync, writeFileSync } = require("node:fs")
5
+ const path = require("node:path")
6
+
7
+ const mapPath = process.env.PROBATIO_COVERAGE_MAP
8
+ if (mapPath) {
9
+ /** @type {Map<string, Map<number, Set<string>>>} */
10
+ const byFile = new Map()
11
+ /** @type {Array<{ file: string, line: number }>} */
12
+ let prelude = []
13
+ /** @type {{ positiveLines: () => Promise<Array<{ file: string, line: number }>> } | null} */
14
+ let sampler = null
15
+
16
+ function add(hits, name) {
17
+ if (!name) return
18
+ for (const hit of hits) {
19
+ let lines = byFile.get(hit.file)
20
+ if (!lines) {
21
+ lines = new Map()
22
+ byFile.set(hit.file, lines)
23
+ }
24
+ let names = lines.get(hit.line)
25
+ if (!names) {
26
+ names = new Set()
27
+ lines.set(hit.line, names)
28
+ }
29
+ names.add(name)
30
+ }
31
+ }
32
+
33
+ function titleOf(test) {
34
+ if (!test) return ""
35
+ if (typeof test.fullTitle === "function") return test.fullTitle()
36
+ return typeof test.title === "string" ? test.title : ""
37
+ }
38
+
39
+ exports.mochaHooks = {
40
+ async beforeAll() {
41
+ try {
42
+ const opened = await import("./precise-lines.mjs")
43
+ sampler = await opened.openLineSampler((rel) => rel.endsWith("mocha-coverage.cjs") || rel.endsWith("precise-lines.mjs") || rel.endsWith("node-coverage.mjs"))
44
+ prelude = await sampler.positiveLines()
45
+ } catch (err) {
46
+ writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
47
+ }
48
+ },
49
+ async afterEach() {
50
+ if (!sampler) return
51
+ const title = titleOf(this.currentTest)
52
+ try {
53
+ add(prelude, title)
54
+ add(await sampler.positiveLines(), title)
55
+ } catch (err) {
56
+ writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
57
+ }
58
+ },
59
+ afterAll() {
60
+ if (byFile.size === 0) return
61
+ const files = {}
62
+ for (const [file, lines] of byFile) {
63
+ const bucket = {}
64
+ for (const [line, names] of lines) bucket[String(line)] = [...names].sort()
65
+ files[file] = bucket
66
+ }
67
+ mkdirSync(path.dirname(mapPath), { recursive: true })
68
+ writeFileSync(mapPath, `${JSON.stringify({ files }, null, 2)}\n`)
69
+ },
70
+ }
71
+ }
@@ -0,0 +1,85 @@
1
+ // Cache-busts package files so the next mutant in this process loads the patched source.
2
+ // The generation lives in a file because the loader thread does not see later env edits.
3
+ import { createRequire } from "node:module"
4
+ import { readFileSync, realpathSync } from "node:fs"
5
+ import path from "node:path"
6
+ import { fileURLToPath } from "node:url"
7
+
8
+ const require = createRequire(import.meta.url)
9
+ const ts = require("typescript")
10
+
11
+ function generation() {
12
+ const file = process.env.PB_GEN
13
+ if (!file) return ""
14
+ try {
15
+ return readFileSync(file, "utf8").trim()
16
+ } catch {
17
+ return ""
18
+ }
19
+ }
20
+
21
+ function packageRoot() {
22
+ const root = process.env.PB_ROOT
23
+ if (!root) return ""
24
+ try {
25
+ return realpathSync(root)
26
+ } catch {
27
+ return root
28
+ }
29
+ }
30
+
31
+ function underPackage(abs) {
32
+ const root = packageRoot()
33
+ if (!root) return false
34
+ let real = abs
35
+ try {
36
+ real = realpathSync(abs)
37
+ } catch {
38
+ return false
39
+ }
40
+ return real === root || real.startsWith(root.endsWith(path.sep) ? root : root + path.sep)
41
+ }
42
+
43
+ export async function resolve(specifier, context, nextResolve) {
44
+ const resolved = await nextResolve(specifier, context)
45
+ const gen = generation()
46
+ if (!gen) return resolved
47
+ let abs
48
+ try {
49
+ abs = fileURLToPath(resolved.url.split("?")[0])
50
+ } catch {
51
+ return resolved
52
+ }
53
+ if (!underPackage(abs)) return resolved
54
+ if (abs.split(path.sep).includes("node_modules")) return resolved
55
+ const url = new URL(resolved.url.split("?")[0])
56
+ url.searchParams.set("pb", gen)
57
+ return { url: url.href, shortCircuit: true }
58
+ }
59
+
60
+ export async function load(url, context, nextLoad) {
61
+ if (!url.includes("pb=")) return nextLoad(url, context)
62
+ const clean = url.split("?")[0]
63
+ let abs
64
+ try {
65
+ abs = fileURLToPath(clean)
66
+ } catch {
67
+ return nextLoad(url, context)
68
+ }
69
+ const source = readFileSync(abs, "utf8")
70
+ if (abs.endsWith(".ts") || abs.endsWith(".tsx") || abs.endsWith(".mts")) {
71
+ const transpiled = ts.transpileModule(source, {
72
+ fileName: abs,
73
+ compilerOptions: {
74
+ module: ts.ModuleKind.ESNext,
75
+ target: ts.ScriptTarget.ES2022,
76
+ sourceMap: false,
77
+ },
78
+ })
79
+ return { format: "module", source: transpiled.outputText, shortCircuit: true }
80
+ }
81
+ if (abs.endsWith(".mjs") || abs.endsWith(".js") || source.includes("import ") || source.includes("export ")) {
82
+ return { format: "module", source, shortCircuit: true }
83
+ }
84
+ return { format: "commonjs", source, shortCircuit: true }
85
+ }
@@ -0,0 +1,130 @@
1
+ // One suite process for covered mutants of one file. Jobs arrive on stdin.
2
+ // A mutant that exits this process ends it. The parent starts a clean process for the next mutant.
3
+ import { createInterface } from "node:readline"
4
+ import { realpathSync, writeFileSync } from "node:fs"
5
+ import { register } from "node:module"
6
+ import { run } from "node:test"
7
+ import path from "node:path"
8
+ import { openLineSampler } from "./precise-lines.mjs"
9
+
10
+ const loader = new URL("./node-batch-loader.mjs", import.meta.url).href
11
+ register(loader, { parentURL: import.meta.url })
12
+
13
+ let sampler = null
14
+ try {
15
+ sampler = await openLineSampler()
16
+ } catch {
17
+ sampler = null
18
+ }
19
+
20
+ writeFileSync(process.env.PB_READY, `${JSON.stringify({ pid: process.pid })}\n`)
21
+
22
+ function escapeRegExp(value) {
23
+ return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")
24
+ }
25
+
26
+ let generation = 0
27
+
28
+ async function runJob(job) {
29
+ generation += 1
30
+ writeFileSync(process.env.PB_GEN, String(generation))
31
+ if (sampler) await sampler.reset()
32
+ const files = (job.tests || []).map((rel) => path.join(job.root, rel))
33
+ const patterns = (job.names || []).filter((name) => name.length > 0).map((name) => new RegExp(`^${escapeRegExp(name)}$`))
34
+ const failed = []
35
+ const names = []
36
+ const testTimeout = job.testTimeoutMs > 0 ? job.testTimeoutMs : job.timeoutMs
37
+ const stream = run({
38
+ files,
39
+ isolation: "none",
40
+ timeout: testTimeout,
41
+ ...(patterns.length > 0 ? { testNamePatterns: patterns } : {}),
42
+ })
43
+ let lastEvent = Date.now()
44
+ const started = Date.now()
45
+ let ended = false
46
+ let timer = null
47
+ const collected = (async () => {
48
+ for await (const event of stream) {
49
+ lastEvent = Date.now()
50
+ if (event.type !== "test:pass" && event.type !== "test:fail") continue
51
+ const data = event.data || {}
52
+ const name = typeof data.name === "string" ? data.name : ""
53
+ if (!name || files.includes(name)) continue
54
+ if (!names.includes(name)) names.push(name)
55
+ if (event.type === "test:fail") {
56
+ const error = data.details && data.details.error
57
+ const text = error ? `${error.failureType || ""} ${error.message || ""} ${error.cause || ""}` : ""
58
+ const fileTimeout = /testTimeoutFailure|timed out/i.test(text) && fileNameIs(name, data.file || "")
59
+ failed.push({ name, file: data.file || "", line: data.line || 0, ...(fileTimeout ? { fileTimeout: true } : {}) })
60
+ }
61
+ }
62
+ ended = true
63
+ })()
64
+ let dirty = false
65
+ try {
66
+ dirty = await Promise.race([
67
+ collected.then(() => false),
68
+ new Promise((resolve) => {
69
+ timer = setInterval(() => {
70
+ if (ended) {
71
+ resolve(false)
72
+ return
73
+ }
74
+ const idle = Date.now() - lastEvent
75
+ const overdue = Date.now() - started >= testTimeout && idle >= 400 && names.length > 0
76
+ if (!overdue) return
77
+ resolve(true)
78
+ }, 50)
79
+ timer.unref()
80
+ }),
81
+ ])
82
+ } finally {
83
+ if (timer) clearInterval(timer)
84
+ }
85
+ if (dirty && failed.length === 0) {
86
+ for (const rel of job.tests || []) {
87
+ const posix = rel.split(path.sep).join("/")
88
+ let file = path.join(job.root, rel)
89
+ try {
90
+ file = realpathSync(file)
91
+ } catch {
92
+ // The reporter uses the path it opened. A missing file keeps the joined path.
93
+ }
94
+ failed.push({ name: posix, file, line: 1, fileTimeout: true })
95
+ if (!names.includes(posix)) names.push(posix)
96
+ }
97
+ }
98
+ if (sampler && !dirty) await sampler.reset()
99
+ const pattern = (job.names || []).join("|")
100
+ const command = ["node", "--test", ...(pattern ? [`--test-name-pattern=${pattern}`] : []), ...(job.tests || [])].join(" ")
101
+ return { id: job.id, pid: process.pid, failed, names, command, dirty }
102
+ }
103
+
104
+ function fileNameIs(name, file) {
105
+ const base = String(file).split(/[/\\]/).pop()
106
+ if (!base) return false
107
+ return name === file || name === base || name.endsWith(`/${base}`) || name.endsWith(`\\${base}`)
108
+ }
109
+
110
+ const lines = createInterface({ input: process.stdin })
111
+ for await (const line of lines) {
112
+ if (!line.trim()) continue
113
+ const job = JSON.parse(line)
114
+ // node:test with isolation "none" waits until the event loop is idle.
115
+ // An open readline on stdin is that handle, so the job would never finish.
116
+ lines.pause()
117
+ process.stdin.unref()
118
+ try {
119
+ const reply = await runJob(job)
120
+ writeFileSync(job.replyFile, `${JSON.stringify(reply)}\n`)
121
+ // An open handle after the tests finished is the same file timeout node --test reports.
122
+ // The process is dirty, so it ends and the next mutant starts clean.
123
+ if (reply.dirty) process.exit(0)
124
+ } catch (err) {
125
+ writeFileSync(job.replyFile, `${JSON.stringify({ id: job.id, pid: process.pid, failed: [], names: [], error: err instanceof Error ? err.message : String(err) })}\n`)
126
+ } finally {
127
+ if (!process.stdin.destroyed) process.stdin.ref()
128
+ lines.resume()
129
+ }
130
+ }
@@ -1,200 +1,29 @@
1
1
  // Baseline-only per-test line map for node --test. Loaded with --import in each
2
2
  // test-file process. Writes one shard; the runner merges shards after the suite.
3
3
  // takePreciseCoverage resets counters, so each sample is only what ran since the last take.
4
+ // A plain-script child does not run these hooks. It may leave a nameless hit list in
5
+ // ${map}.children. afterEach stores that list under the parent test before the shard is written.
4
6
  import { randomBytes } from "node:crypto"
5
- import { mkdirSync, writeFileSync } from "node:fs"
6
- import { Session } from "node:inspector/promises"
7
- import { findSourceMap } from "node:module"
7
+ import { existsSync, mkdirSync, readdirSync, readFileSync, unlinkSync, writeFileSync } from "node:fs"
8
8
  import { after, afterEach, before } from "node:test"
9
9
  import path from "node:path"
10
- import { fileURLToPath } from "node:url"
11
-
12
- const B64 = new Int32Array(128)
13
- for (let i = 0; i < 64; i++) B64["ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/".charCodeAt(i)] = i
14
-
15
- function decodeSegments(row) {
16
- const segments = []
17
- let i = 0
18
- while (i < row.length) {
19
- const fields = []
20
- while (i < row.length && row.charCodeAt(i) !== 44) {
21
- let value = 0
22
- let shift = 0
23
- let digit = 0
24
- do {
25
- digit = B64[row.charCodeAt(i++)]
26
- value |= (digit & 31) << shift
27
- shift += 5
28
- } while (digit & 32)
29
- fields.push(value & 1 ? -(value >> 1) : value >> 1)
30
- }
31
- if (row.charCodeAt(i) === 44) i++
32
- if (fields.length > 0) segments.push(fields)
33
- }
34
- return segments
35
- }
10
+ import { openLineSampler } from "./precise-lines.mjs"
36
11
 
37
12
  const mapPath = process.env.PROBATIO_COVERAGE_MAP
38
13
  if (mapPath) {
39
- const session = new Session()
40
- let started = false
14
+ let sampler = null
41
15
  try {
42
- session.connect()
43
- await session.post("Profiler.enable")
44
- await session.post("Debugger.enable")
45
- await session.post("Profiler.startPreciseCoverage", { callCount: true, detailed: true })
46
- started = true
16
+ sampler = await openLineSampler((rel) => rel.endsWith("node-coverage.mjs") || rel.endsWith("precise-lines.mjs") || rel.endsWith("child-lines.mjs"))
47
17
  } catch (err) {
48
18
  writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
49
19
  }
50
20
 
51
- if (started) {
52
- const textCache = new Map()
53
- const indexCache = new Map()
21
+ if (sampler) {
54
22
  /** @type {Map<string, Map<number, Set<string>>>} */
55
23
  const byFile = new Map()
56
24
  /** @type {Array<{ file: string, line: number }>} */
57
25
  let prelude = []
58
26
 
59
- async function scriptText(script) {
60
- if (textCache.has(script.scriptId)) return textCache.get(script.scriptId)
61
- let text = ""
62
- try {
63
- const got = await session.post("Debugger.getScriptSource", { scriptId: script.scriptId })
64
- text = got.scriptSource || ""
65
- } catch {
66
- text = ""
67
- }
68
- textCache.set(script.scriptId, text)
69
- return text
70
- }
71
-
72
- function relPath(url) {
73
- if (!url || url.startsWith("node:")) return null
74
- let abs
75
- try {
76
- abs = fileURLToPath(url.split("?")[0].split("#")[0])
77
- } catch {
78
- return null
79
- }
80
- const rel = path.relative(process.cwd(), abs).split(path.sep).join("/")
81
- if (!rel || rel.startsWith("..") || path.isAbsolute(rel)) return null
82
- if (rel.split("/").includes("node_modules")) return null
83
- if (rel.endsWith("node-coverage.mjs")) return null
84
- return rel
85
- }
86
-
87
- function indexFor(scriptId, text, sm) {
88
- if (indexCache.has(scriptId)) return indexCache.get(scriptId)
89
- const mapped = sm && sm.payload && typeof sm.payload.mappings === "string" ? linesFromMappings(text, sm.payload.mappings) : null
90
- const lines = mapped && hasLine(mapped) ? mapped : linesFromNewlines(text, sm)
91
- indexCache.set(scriptId, lines)
92
- return lines
93
- }
94
-
95
- function hasLine(lines) {
96
- for (let i = 0; i < lines.length; i++) if (lines[i] > 0) return true
97
- return false
98
- }
99
-
100
- function linesFromNewlines(text, sm) {
101
- const lines = new Int32Array(text.length + 1)
102
- if (!sm) {
103
- let line = 1
104
- for (let i = 0; i < text.length; i++) {
105
- lines[i] = line
106
- if (text.charCodeAt(i) === 10) line++
107
- }
108
- return lines
109
- }
110
- // Node 22 reports SourceMap.findEntry originalLine as 0-based.
111
- let genLine = 0
112
- let col = 0
113
- for (let i = 0; i < text.length; i++) {
114
- const entry = sm.findEntry(genLine, col)
115
- if (entry && Number.isInteger(entry.originalLine) && entry.originalLine >= 0) lines[i] = entry.originalLine + 1
116
- if (text.charCodeAt(i) === 10) {
117
- genLine++
118
- col = 0
119
- } else {
120
- col++
121
- }
122
- }
123
- return lines
124
- }
125
-
126
- function linesFromMappings(text, mappings) {
127
- const lines = new Int32Array(text.length + 1)
128
- const starts = [0]
129
- for (let i = 0; i < text.length; i++) if (text.charCodeAt(i) === 10) starts.push(i + 1)
130
- let carryLine = 0
131
- const rows = mappings.split(";")
132
- for (let genLine = 0; genLine < rows.length && genLine < starts.length; genLine++) {
133
- const start = starts[genLine]
134
- const next = genLine + 1 < starts.length ? starts[genLine + 1] : text.length
135
- let col = 0
136
- let cursor = start
137
- let active = -1
138
- for (const fields of decodeSegments(rows[genLine])) {
139
- col += fields[0]
140
- const at = Math.min(next, start + Math.max(0, col))
141
- if (active >= 0) {
142
- for (let i = cursor; i < at; i++) lines[i] = active + 1
143
- }
144
- cursor = at
145
- if (fields.length >= 4) {
146
- carryLine += fields[2]
147
- active = carryLine
148
- } else {
149
- active = -1
150
- }
151
- }
152
- if (active >= 0) {
153
- for (let i = cursor; i < next; i++) lines[i] = active + 1
154
- }
155
- }
156
- return lines
157
- }
158
-
159
- function applyRanges(text, script, lines) {
160
- const ranges = []
161
- for (const fn of script.functions || []) {
162
- for (const range of fn.ranges || []) ranges.push(range)
163
- }
164
- // Outer ranges first. A nested block then overwrites its own offsets, so a dead
165
- // branch stays at 0 while a condition earlier on that same line can stay covered.
166
- ranges.sort((a, b) => a.startOffset - b.startOffset || b.endOffset - a.endOffset)
167
- const offsets = new Int32Array(text.length)
168
- for (const range of ranges) {
169
- const end = Math.min(text.length, range.endOffset)
170
- for (let i = Math.max(0, range.startOffset); i < end; i++) offsets[i] = range.count
171
- }
172
- const counts = new Map()
173
- for (let i = 0; i < text.length; i++) {
174
- const line = lines[i]
175
- if (line <= 0 || offsets[i] <= 0) continue
176
- if (offsets[i] > (counts.get(line) || 0)) counts.set(line, offsets[i])
177
- }
178
- return counts
179
- }
180
-
181
- async function positiveLines() {
182
- const cov = await session.post("Profiler.takePreciseCoverage")
183
- /** @type {Array<{ file: string, line: number }>} */
184
- const found = []
185
- for (const script of cov.result || []) {
186
- const file = relPath(script.url)
187
- if (!file) continue
188
- const text = await scriptText(script)
189
- if (!text) continue
190
- const counts = applyRanges(text, script, indexFor(script.scriptId, text, findSourceMap(script.url)))
191
- for (const [line, count] of counts) {
192
- if (count > 0) found.push({ file, line })
193
- }
194
- }
195
- return found
196
- }
197
-
198
27
  function add(hits, name) {
199
28
  if (!name) return
200
29
  for (const hit of hits) {
@@ -212,9 +41,36 @@ if (mapPath) {
212
41
  }
213
42
  }
214
43
 
44
+ function takeChildHits() {
45
+ const dir = `${mapPath}.children`
46
+ if (!existsSync(dir)) return []
47
+ /** @type {Array<{ file: string, line: number }>} */
48
+ const hits = []
49
+ for (const name of readdirSync(dir)) {
50
+ if (!name.endsWith(".json")) continue
51
+ const full = path.join(dir, name)
52
+ try {
53
+ const parsed = JSON.parse(readFileSync(full, "utf8"))
54
+ const list = parsed && Array.isArray(parsed.hits) ? parsed.hits : []
55
+ for (const hit of list) {
56
+ if (!hit || typeof hit.file !== "string" || !Number.isInteger(hit.line)) continue
57
+ hits.push({ file: hit.file, line: hit.line })
58
+ }
59
+ } catch {
60
+ // A torn dump is ignored. The parent does not invent a line.
61
+ }
62
+ try {
63
+ unlinkSync(full)
64
+ } catch {
65
+ // The next test must not inherit this dump.
66
+ }
67
+ }
68
+ return hits
69
+ }
70
+
215
71
  before(async () => {
216
72
  try {
217
- prelude = await positiveLines()
73
+ prelude = await sampler.positiveLines()
218
74
  } catch (err) {
219
75
  writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
220
76
  }
@@ -223,7 +79,8 @@ if (mapPath) {
223
79
  afterEach(async (t) => {
224
80
  try {
225
81
  add(prelude, t.name)
226
- add(await positiveLines(), t.name)
82
+ add(await sampler.positiveLines(), t.name)
83
+ add(takeChildHits(), t.name)
227
84
  } catch (err) {
228
85
  writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
229
86
  }
@@ -0,0 +1,188 @@
1
+ // Inspector line samples shared by the node:test collector, the Mocha collector,
2
+ // and a plain-script child. This file has no test hooks and writes no shard.
3
+ import { Session } from "node:inspector/promises"
4
+ import { findSourceMap } from "node:module"
5
+ import path from "node:path"
6
+ import { fileURLToPath } from "node:url"
7
+
8
+ const B64 = new Int32Array(128)
9
+ for (let i = 0; i < 64; i++) B64["ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/".charCodeAt(i)] = i
10
+
11
+ function decodeSegments(row) {
12
+ const segments = []
13
+ let i = 0
14
+ while (i < row.length) {
15
+ const fields = []
16
+ while (i < row.length && row.charCodeAt(i) !== 44) {
17
+ let value = 0
18
+ let shift = 0
19
+ let digit = 0
20
+ do {
21
+ digit = B64[row.charCodeAt(i++)]
22
+ value |= (digit & 31) << shift
23
+ shift += 5
24
+ } while (digit & 32)
25
+ fields.push(value & 1 ? -(value >> 1) : value >> 1)
26
+ }
27
+ if (row.charCodeAt(i) === 44) i++
28
+ if (fields.length > 0) segments.push(fields)
29
+ }
30
+ return segments
31
+ }
32
+
33
+ /**
34
+ * @param {(rel: string) => boolean} [skipRel]
35
+ * @returns {Promise<{ positiveLines: () => Promise<Array<{ file: string, line: number }>>, reset: () => Promise<void> }>}
36
+ */
37
+ export async function openLineSampler(skipRel) {
38
+ const session = new Session()
39
+ session.connect()
40
+ await session.post("Profiler.enable")
41
+ await session.post("Debugger.enable")
42
+ await session.post("Profiler.startPreciseCoverage", { callCount: true, detailed: true })
43
+
44
+ const textCache = new Map()
45
+ const indexCache = new Map()
46
+
47
+ async function scriptText(script) {
48
+ if (textCache.has(script.scriptId)) return textCache.get(script.scriptId)
49
+ let text = ""
50
+ try {
51
+ const got = await session.post("Debugger.getScriptSource", { scriptId: script.scriptId })
52
+ text = got.scriptSource || ""
53
+ } catch {
54
+ text = ""
55
+ }
56
+ textCache.set(script.scriptId, text)
57
+ return text
58
+ }
59
+
60
+ function relPath(url) {
61
+ if (!url || url.startsWith("node:")) return null
62
+ let abs
63
+ try {
64
+ abs = fileURLToPath(url.split("?")[0].split("#")[0])
65
+ } catch {
66
+ return null
67
+ }
68
+ const rel = path.relative(process.cwd(), abs).split(path.sep).join("/")
69
+ if (!rel || rel.startsWith("..") || path.isAbsolute(rel)) return null
70
+ if (rel.split("/").includes("node_modules")) return null
71
+ if (skipRel && skipRel(rel)) return null
72
+ return rel
73
+ }
74
+
75
+ function indexFor(scriptId, text, sm) {
76
+ if (indexCache.has(scriptId)) return indexCache.get(scriptId)
77
+ const mapped = sm && sm.payload && typeof sm.payload.mappings === "string" ? linesFromMappings(text, sm.payload.mappings) : null
78
+ const lines = mapped && hasLine(mapped) ? mapped : linesFromNewlines(text, sm)
79
+ indexCache.set(scriptId, lines)
80
+ return lines
81
+ }
82
+
83
+ function hasLine(lines) {
84
+ for (let i = 0; i < lines.length; i++) if (lines[i] > 0) return true
85
+ return false
86
+ }
87
+
88
+ function linesFromNewlines(text, sm) {
89
+ const lines = new Int32Array(text.length + 1)
90
+ if (!sm) {
91
+ let line = 1
92
+ for (let i = 0; i < text.length; i++) {
93
+ lines[i] = line
94
+ if (text.charCodeAt(i) === 10) line++
95
+ }
96
+ return lines
97
+ }
98
+ let genLine = 0
99
+ let col = 0
100
+ for (let i = 0; i < text.length; i++) {
101
+ const entry = sm.findEntry(genLine, col)
102
+ if (entry && Number.isInteger(entry.originalLine) && entry.originalLine >= 0) lines[i] = entry.originalLine + 1
103
+ if (text.charCodeAt(i) === 10) {
104
+ genLine++
105
+ col = 0
106
+ } else {
107
+ col++
108
+ }
109
+ }
110
+ return lines
111
+ }
112
+
113
+ function linesFromMappings(text, mappings) {
114
+ const lines = new Int32Array(text.length + 1)
115
+ const starts = [0]
116
+ for (let i = 0; i < text.length; i++) if (text.charCodeAt(i) === 10) starts.push(i + 1)
117
+ let carryLine = 0
118
+ const rows = mappings.split(";")
119
+ for (let genLine = 0; genLine < rows.length && genLine < starts.length; genLine++) {
120
+ const start = starts[genLine]
121
+ const next = genLine + 1 < starts.length ? starts[genLine + 1] : text.length
122
+ let col = 0
123
+ let cursor = start
124
+ let active = -1
125
+ for (const fields of decodeSegments(rows[genLine])) {
126
+ col += fields[0]
127
+ const at = Math.min(next, start + Math.max(0, col))
128
+ if (active >= 0) {
129
+ for (let i = cursor; i < at; i++) lines[i] = active + 1
130
+ }
131
+ cursor = at
132
+ if (fields.length >= 4) {
133
+ carryLine += fields[2]
134
+ active = carryLine
135
+ } else {
136
+ active = -1
137
+ }
138
+ }
139
+ if (active >= 0) {
140
+ for (let i = cursor; i < next; i++) lines[i] = active + 1
141
+ }
142
+ }
143
+ return lines
144
+ }
145
+
146
+ function applyRanges(text, script, lines) {
147
+ const ranges = []
148
+ for (const fn of script.functions || []) {
149
+ for (const range of fn.ranges || []) ranges.push(range)
150
+ }
151
+ ranges.sort((a, b) => a.startOffset - b.startOffset || b.endOffset - a.endOffset)
152
+ const offsets = new Int32Array(text.length)
153
+ for (const range of ranges) {
154
+ const end = Math.min(text.length, range.endOffset)
155
+ for (let i = Math.max(0, range.startOffset); i < end; i++) offsets[i] = range.count
156
+ }
157
+ const counts = new Map()
158
+ for (let i = 0; i < text.length; i++) {
159
+ const line = lines[i]
160
+ if (line <= 0 || offsets[i] <= 0) continue
161
+ if (offsets[i] > (counts.get(line) || 0)) counts.set(line, offsets[i])
162
+ }
163
+ return counts
164
+ }
165
+
166
+ async function positiveLines() {
167
+ const cov = await session.post("Profiler.takePreciseCoverage")
168
+ const found = []
169
+ for (const script of cov.result || []) {
170
+ const file = relPath(script.url)
171
+ if (!file) continue
172
+ const text = await scriptText(script)
173
+ if (!text) continue
174
+ const counts = applyRanges(text, script, indexFor(script.scriptId, text, findSourceMap(script.url)))
175
+ for (const [line, count] of counts) {
176
+ if (count > 0) found.push({ file, line })
177
+ }
178
+ }
179
+ return found
180
+ }
181
+
182
+ return {
183
+ positiveLines,
184
+ reset: async () => {
185
+ await session.post("Profiler.takePreciseCoverage")
186
+ },
187
+ }
188
+ }
@@ -1,6 +1,6 @@
1
1
  import { spawn } from "node:child_process";
2
2
  import { createHash, randomBytes } from "node:crypto";
3
- import { copyFileSync, existsSync, mkdirSync, readdirSync, readFileSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
3
+ import { appendFileSync, copyFileSync, existsSync, mkdirSync, readdirSync, readFileSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
4
4
  import { tmpdir } from "node:os";
5
5
  import path from "node:path";
6
6
  import { fileURLToPath } from "node:url";
@@ -13,7 +13,7 @@ import { writeRustCoverage } from "./rust-coverage.js";
13
13
  import { applyPatch, changedLines, chooseDirection, filesInDiff, git, restoreTree } from "./patch.js";
14
14
  import { killLabel } from "./kill-label.js";
15
15
  import { TIMEOUT_FLOOR_MS, TIMEOUT_MULTIPLE, TIMEOUT_NEXT, WHOLE_PROGRAM_TEST, WHOLE_SUITE_NEXT, campaignSummary, decideSuite, hasLineMap, noCoverageNext, resumeNext, scaleStop, timeoutLimitMs, } from "./suite-decision.js";
16
- import { commandSuite, discoverSuite, runDiscoveredSuite, sourceCandidates } from "./suites.js";
16
+ import { commandSuite, discoverSuite, nodeTestTimeoutMs, runDiscoveredSuite, sourceCandidates } from "./suites.js";
17
17
  import { duplicateTitles } from "./titles.js";
18
18
  export const reporterPath = fileURLToPath(new URL("./fail-reporter.mjs", import.meta.url));
19
19
  export function parseReport(stdout) {
@@ -89,34 +89,61 @@ export async function runMutants(options) {
89
89
  budgetStarted = Date.now();
90
90
  const worker = async (worktree) => {
91
91
  const treePkg = packageIn(worktree, repo, packageDir);
92
- for (;;) {
93
- if (startedMutants > 0 && overBudget()) {
94
- budgetHit = true;
95
- return;
96
- }
97
- const patch = queue.shift();
98
- if (!patch)
99
- return;
100
- if (startedMutants > 0 && overBudget()) {
101
- queue.unshift(patch);
102
- budgetHit = true;
103
- return;
104
- }
105
- const id = path.basename(patch, ".patch");
106
- const raw = readFileSync(patch, "utf8");
107
- const prior = history.get(id);
108
- if (prior && reusable(prior, raw, treePkg)) {
92
+ let batch = null;
93
+ const dropBatch = () => {
94
+ batch?.stop();
95
+ batch = null;
96
+ };
97
+ try {
98
+ for (;;) {
99
+ if (startedMutants > 0 && overBudget()) {
100
+ budgetHit = true;
101
+ return;
102
+ }
103
+ const patch = queue.shift();
104
+ if (!patch)
105
+ return;
106
+ if (startedMutants > 0 && overBudget()) {
107
+ queue.unshift(patch);
108
+ budgetHit = true;
109
+ return;
110
+ }
111
+ const id = path.basename(patch, ".patch");
112
+ const raw = readFileSync(patch, "utf8");
113
+ const prior = history.get(id);
114
+ if (prior && reusable(prior, raw, treePkg)) {
115
+ startedMutants += 1;
116
+ const reused = reusedResult(prior, commit, run.id, options.agent);
117
+ writeResult(options.outDir, reused);
118
+ progress(`reused ${id}`);
119
+ continue;
120
+ }
109
121
  startedMutants += 1;
110
- const reused = reusedResult(prior, commit, run.id, options.agent);
111
- writeResult(options.outDir, reused);
112
- progress(`reused ${id}`);
113
- continue;
122
+ progress(`${id} start`);
123
+ const plan = batchPlan(raw, prep, options);
124
+ if (plan) {
125
+ if (!batch || batch.dead) {
126
+ dropBatch();
127
+ try {
128
+ batch = await NodeBatch.start(treePkg, options, options.outDir);
129
+ }
130
+ catch {
131
+ batch = null;
132
+ }
133
+ }
134
+ }
135
+ const launch = plan && batch
136
+ ? (pkg, files, names, timeoutMs) => batch.run(pkg, files, names, timeoutMs, id, options.outDir)
137
+ : undefined;
138
+ const result = await runOne(worktree, treePkg, patch, options, commit, run.id, prep, launch);
139
+ writeResult(options.outDir, result);
140
+ if (batch && (batch.dead || result.error === "suite process crashed"))
141
+ dropBatch();
142
+ progress(`${id} ${result.outcome}`);
114
143
  }
115
- startedMutants += 1;
116
- progress(`${id} start`);
117
- const result = await runOne(worktree, treePkg, patch, options, commit, run.id, prep);
118
- writeResult(options.outDir, result);
119
- progress(`${id} ${result.outcome}`);
144
+ }
145
+ finally {
146
+ dropBatch();
120
147
  }
121
148
  };
122
149
  await Promise.all(worktrees.dirs.map((dir) => worker(dir)));
@@ -217,7 +244,7 @@ async function prepare(pkg, options, progress) {
217
244
  const affectedMap = options.affected ? affectedTests(pkg, { testsDir: options.testsDir }) : null;
218
245
  return { ok: true, tests, affectedMap, baselineMs, spec, coverage: readCoverageMap(mapPath) };
219
246
  }
220
- async function runOne(worktree, pkg, patchPath, options, commit, runId, prep) {
247
+ async function runOne(worktree, pkg, patchPath, options, commit, runId, prep, launch) {
221
248
  const id = path.basename(patchPath, ".patch");
222
249
  const started = Date.now();
223
250
  const raw = readFileSync(patchPath, "utf8");
@@ -277,9 +304,15 @@ async function runOne(worktree, pkg, patchPath, options, commit, runId, prep) {
277
304
  // node --test loads every file it is given, so a name pattern still pays for the other files.
278
305
  const selected = narrowNodeFiles(pkg, selectTests(files, prep.tests, prep.affectedMap), names, prep.spec.kind);
279
306
  const commands = [];
280
- const main = await runSuite(pkg, selected, options, null, names && names.length > 0 ? names : null, prep.spec, limit, forceWhole, null, null);
307
+ const named = names && names.length > 0 ? names : null;
308
+ const main = launch
309
+ ? await launch(pkg, selected, named, limit)
310
+ : await runSuite(pkg, selected, options, null, named, prep.spec, limit, forceWhole, null, null);
281
311
  if (main.command)
282
312
  commands.push(main.command);
313
+ if (main.crashed) {
314
+ return finishMutant({ ...base(), direction, outcome: "error", error: "suite process crashed" }, pkg, commands.join(" | "), false, main.selectedTests);
315
+ }
283
316
  if (main.timedOut) {
284
317
  return finishMutant({ ...base(), direction, outcome: "timeout", error: "suite timed out", next: TIMEOUT_NEXT }, pkg, commands.join(" | "), main.wholeSuite || forceWhole, main.selectedTests);
285
318
  }
@@ -404,6 +437,164 @@ async function runSuite(pkg, files, options, pattern, names, spec, timeoutMs, fo
404
437
  selectedTests: result.selectedTests,
405
438
  };
406
439
  }
440
+ function batchPlan(raw, prep, options) {
441
+ if (options.workers !== 1 || prep.spec.kind !== "node")
442
+ return null;
443
+ if (!prep.coverage || Object.keys(prep.coverage.files).length === 0)
444
+ return null;
445
+ const edited = changedLines(raw);
446
+ const files = filesInDiff(raw);
447
+ if (new Set(files.map((item) => item.file)).size !== 1)
448
+ return null;
449
+ const picked = selectionFor(prep.coverage, edited.length > 0 ? edited : files);
450
+ if (picked.state !== "covered")
451
+ return null;
452
+ const names = picked.tests.filter((name) => name !== WHOLE_PROGRAM_TEST && name.length > 0);
453
+ if (names.length === 0)
454
+ return null;
455
+ return { names };
456
+ }
457
+ /** Covered node mutants of one file share this process. A crash sets dead and the next mutant starts another. */
458
+ class NodeBatch {
459
+ pid;
460
+ dead = false;
461
+ testTimeoutMs;
462
+ proc;
463
+ constructor(proc, pid, testTimeoutMs) {
464
+ this.proc = proc;
465
+ this.pid = pid;
466
+ this.testTimeoutMs = testTimeoutMs;
467
+ proc.on("close", () => {
468
+ this.dead = true;
469
+ });
470
+ }
471
+ static async start(pkg, options, outDir) {
472
+ const dir = path.join(outDir, "batch");
473
+ mkdirSync(dir, { recursive: true });
474
+ const ready = path.join(dir, `ready-${process.pid}-${Date.now()}.json`);
475
+ const gen = path.join(dir, `gen-${process.pid}-${Date.now()}.txt`);
476
+ writeFileSync(gen, "0\n");
477
+ const script = fileURLToPath(new URL("./node-batch.mjs", import.meta.url));
478
+ const env = childEnv(options.unset);
479
+ env.PB_READY = ready;
480
+ env.PB_GEN = gen;
481
+ env.PB_ROOT = pkg;
482
+ const proc = spawn(process.execPath, [script], { cwd: pkg, env, stdio: ["pipe", "ignore", "pipe"] });
483
+ const started = Date.now();
484
+ while (!existsSync(ready)) {
485
+ if (proc.exitCode !== null)
486
+ throw new Error("batch process exited before ready");
487
+ if (Date.now() - started > 10_000) {
488
+ proc.kill("SIGKILL");
489
+ throw new Error("batch process did not start");
490
+ }
491
+ await new Promise((resolve) => setTimeout(resolve, 20));
492
+ }
493
+ const body = JSON.parse(readFileSync(ready, "utf8"));
494
+ return new NodeBatch(proc, body.pid || proc.pid || 0, options.testTimeoutMs);
495
+ }
496
+ stop() {
497
+ this.dead = true;
498
+ try {
499
+ this.proc.stdin?.end();
500
+ }
501
+ catch {
502
+ // The pipe is already closed.
503
+ }
504
+ try {
505
+ if (this.proc.pid)
506
+ process.kill(this.proc.pid, "SIGTERM");
507
+ }
508
+ catch {
509
+ // The process has already exited.
510
+ }
511
+ }
512
+ async run(pkg, files, names, timeoutMs, id, outDir) {
513
+ const selected = names ?? [];
514
+ const replyFile = path.join(outDir, "batch", `${id}.reply.json`);
515
+ rmSync(replyFile, { force: true });
516
+ appendFileSync(path.join(outDir, "batch-processes.jsonl"), `${JSON.stringify({ id, pid: this.pid })}\n`);
517
+ const job = { id, root: pkg, tests: files, names: selected, timeoutMs, testTimeoutMs: nodeTestTimeoutMs(pkg, files, this.testTimeoutMs, timeoutMs), replyFile };
518
+ try {
519
+ this.proc.stdin?.write(`${JSON.stringify(job)}\n`);
520
+ }
521
+ catch {
522
+ this.dead = true;
523
+ return crashedCall(selected);
524
+ }
525
+ const waited = await waitForReply(replyFile, this.proc, timeoutMs);
526
+ if (waited === "timeout") {
527
+ this.dead = true;
528
+ this.stop();
529
+ return { report: null, timedOut: true, detail: "suite timed out", compileToken: null, command: "", wholeSuite: false, selectedTests: selected };
530
+ }
531
+ if (!waited)
532
+ return crashedCall(selected);
533
+ const failed = waited.failed ?? [];
534
+ const seen = waited.names ?? [];
535
+ if (seen.length === 0 && failed.length === 0) {
536
+ return {
537
+ report: null,
538
+ timedOut: false,
539
+ detail: waited.error || "test reporter returned nothing",
540
+ compileToken: null,
541
+ command: waited.command || "",
542
+ wholeSuite: false,
543
+ selectedTests: selected,
544
+ };
545
+ }
546
+ return {
547
+ report: { tests: Math.max(seen.length, failed.length), pass: Math.max(0, seen.length - failed.length), fail: failed.length, failed, names: seen.length > 0 ? seen : selected },
548
+ timedOut: false,
549
+ detail: "",
550
+ compileToken: null,
551
+ command: waited.command || "",
552
+ wholeSuite: false,
553
+ selectedTests: selected,
554
+ };
555
+ }
556
+ }
557
+ function crashedCall(selected) {
558
+ return { report: null, timedOut: false, detail: "suite process crashed", compileToken: null, command: "", wholeSuite: false, selectedTests: selected, crashed: true };
559
+ }
560
+ function waitForReply(file, proc, timeoutMs) {
561
+ return new Promise((resolve) => {
562
+ let settled = false;
563
+ const finish = (value) => {
564
+ if (settled)
565
+ return;
566
+ settled = true;
567
+ clearTimeout(timer);
568
+ clearInterval(poll);
569
+ proc.removeListener("close", onClose);
570
+ resolve(value);
571
+ };
572
+ const read = () => {
573
+ try {
574
+ return JSON.parse(readFileSync(file, "utf8"));
575
+ }
576
+ catch {
577
+ return null;
578
+ }
579
+ };
580
+ const onClose = () => {
581
+ const reply = existsSync(file) ? read() : null;
582
+ finish(reply);
583
+ };
584
+ const timer = setTimeout(() => finish("timeout"), timeoutMs);
585
+ const poll = setInterval(() => {
586
+ if (!existsSync(file))
587
+ return;
588
+ const reply = read();
589
+ if (reply)
590
+ finish(reply);
591
+ }, 20);
592
+ if (proc.exitCode !== null || proc.signalCode !== null)
593
+ onClose();
594
+ else
595
+ proc.once("close", onClose);
596
+ });
597
+ }
407
598
  function isNodeTestFile(file) {
408
599
  const base = path.posix.basename(file);
409
600
  return /\.(test|spec)\.(ts|tsx|mts|js|mjs|cjs)$/.test(base) || /_test\.(ts|tsx|js|mjs)$/.test(base);
@@ -30,7 +30,7 @@ export function timeoutLimitMs(baselineMs, multiple, floorMs) {
30
30
  return Math.max(floor, scaled);
31
31
  }
32
32
  /** Kinds that collect a per-line map on the baseline. Other kinds pay the whole suite. */
33
- const LINE_MAP_KINDS = new Set(["node", "pytest", "c", "go", "maven", "cargo", "dotnet"]);
33
+ const LINE_MAP_KINDS = new Set(["node", "pytest", "c", "go", "maven", "cargo", "dotnet", "mocha"]);
34
34
  /** A baseline this long, with no line map, is too expensive to repeat for a large batch. */
35
35
  export const SLOW_BASELINE_MS = 5_000;
36
36
  /** Pending mutants above this count, on a slow unmapped suite, are not started. */
@@ -400,8 +400,11 @@ async function launch(pkg, spec, files, pattern, names, timeoutMs, env, hooks) {
400
400
  }
401
401
  if (spec.kind === "dotnet-exe")
402
402
  return tag(runDotnetExe(pkg, spec, files, timeoutMs, env), spec.command, wantsNames, selection);
403
- if (spec.kind === "mocha")
404
- return tag(runMocha(pkg, files, timeoutMs, env), spec.command, wantsNames, selection);
403
+ if (spec.kind === "mocha") {
404
+ const grep = selection.length > 0 ? selection.map(escapeRegExp).join("|") : "";
405
+ const command = grep ? `${spec.command} --grep ${JSON.stringify(grep)}` : spec.command;
406
+ return tag(runMocha(pkg, files, timeoutMs, env, selection), command, false, selection);
407
+ }
405
408
  // A tty is only for a suite that runs `docker run -it` (tini). Everything else is bash.
406
409
  if (spec.kind === "script") {
407
410
  const launched = scriptInvocation(pkg);
@@ -481,18 +484,46 @@ function runShell(pkg, command, timeoutMs, env) {
481
484
  const child = /\b(java|javac|mvn)\b/.test(command) ? javaEnv(env) : env;
482
485
  return spawnCollected("sh", ["-c", command], pkg, { ...child, PYTHONDONTWRITEBYTECODE: "1" }, timeoutMs);
483
486
  }
487
+ /**
488
+ * Node's `--test-timeout` caps the file, not only a test that left the option unset.
489
+ * A test that sets a longer `timeout` is cancelled, and the failure name is the file path.
490
+ * The flag stays at least that long, and it stays inside the suite cap.
491
+ */
492
+ export function nodeTestTimeoutMs(pkg, files, testTimeoutMs, suiteTimeoutMs) {
493
+ let declared = 0;
494
+ for (const rel of files) {
495
+ let text = "";
496
+ try {
497
+ text = readFileSync(path.join(pkg, rel), "utf8");
498
+ }
499
+ catch {
500
+ continue;
501
+ }
502
+ for (const match of text.matchAll(/timeout\s*:\s*(\d[\d_]*)/g)) {
503
+ const value = Number(match[1].replaceAll("_", ""));
504
+ if (Number.isFinite(value) && value > declared)
505
+ declared = value;
506
+ }
507
+ }
508
+ const floor = Number.isFinite(testTimeoutMs) && testTimeoutMs > 0 ? testTimeoutMs : 0;
509
+ let chosen = Math.max(floor, declared);
510
+ if (Number.isFinite(suiteTimeoutMs) && suiteTimeoutMs > 0 && chosen > suiteTimeoutMs)
511
+ chosen = suiteTimeoutMs;
512
+ return chosen > 0 ? chosen : floor;
513
+ }
484
514
  async function runNode(pkg, files, pattern, timeoutMs, env, hooks) {
485
515
  const tsx = path.join(pkg, "node_modules", ".bin", "tsx");
486
516
  const bin = existsSync(tsx) ? tsx : process.execPath;
487
517
  const prefix = existsSync(tsx) ? [] : ["--experimental-strip-types"];
488
518
  const hook = env.PROBATIO_COVERAGE_MAP ? fileURLToPath(new URL("./node-coverage.mjs", import.meta.url)) : null;
519
+ const fileTimeout = nodeTestTimeoutMs(pkg, files, hooks.testTimeoutMs, timeoutMs);
489
520
  const result = await spawnCollected(bin, [
490
521
  ...prefix,
491
522
  ...(hook ? ["--import", hook] : []),
492
523
  "--test",
493
524
  `--test-reporter=${hooks.reporterPath}`,
494
525
  `--test-concurrency=${hooks.concurrency}`,
495
- `--test-timeout=${hooks.testTimeoutMs}`,
526
+ `--test-timeout=${fileTimeout}`,
496
527
  ...(pattern ? [`--test-name-pattern=${pattern}`] : []),
497
528
  ...files,
498
529
  ], pkg, env, timeoutMs);
@@ -545,7 +576,7 @@ function runPython(pkg, files, pattern, timeoutMs, env, script, nodeIds) {
545
576
  const targets = nodeIds && nodeIds.length > 0 ? nodeIds : files;
546
577
  return spawnCollected("python3", [reporter, ...targets, ...(pattern ? ["--pattern", pattern] : [])], pkg, next, timeoutMs);
547
578
  }
548
- async function runMocha(pkg, files, timeoutMs, env) {
579
+ async function runMocha(pkg, files, timeoutMs, env, names = null) {
549
580
  const mocha = ["mocha.js", "mocha"]
550
581
  .map((name) => path.join(pkg, "node_modules", "mocha", "bin", name))
551
582
  .find((file) => existsSync(file));
@@ -556,6 +587,10 @@ async function runMocha(pkg, files, timeoutMs, env) {
556
587
  return prepared;
557
588
  const config = readMochaConfig(pkg);
558
589
  const args = [mocha, ...buildMochaArgs(pkg, files)];
590
+ if (names && names.length > 0)
591
+ args.push("--grep", names.map(escapeRegExp).join("|"));
592
+ if (env.PROBATIO_COVERAGE_MAP)
593
+ args.push("--require", fileURLToPath(new URL("./mocha-coverage.cjs", import.meta.url)));
559
594
  if (config && config.require.includes("esm")) {
560
595
  // The `esm` loader throws on current Node before any test runs. Drop that one require and keep the rest of the project's mocha config.
561
596
  const requires = config.require.filter((name) => name !== "esm");
@@ -1,4 +1,4 @@
1
- import { mkdirSync, readFileSync, realpathSync, writeFileSync } from "node:fs";
1
+ import { existsSync, mkdirSync, readdirSync, readFileSync, realpathSync, statSync, writeFileSync } from "node:fs";
2
2
  import path from "node:path";
3
3
  import { SCHEMA_VERSION } from "../contract.js";
4
4
  import { asTable } from "../golden/check.js";
@@ -26,9 +26,10 @@ export async function verifyChange(options) {
26
26
  const goldenContract = goldenDiffs(repo, options.base, options.commit, changed);
27
27
  const packageFiles = [];
28
28
  const chosen = [];
29
+ const libImported = testsImportLib(packageDir);
29
30
  for (const [file, lines] of ranges) {
30
31
  const rel = toPackage(prefix, file);
31
- if (!rel || !rel.startsWith("src/") || rel.endsWith(".d.ts"))
32
+ if (!rel || !isVerifySource(rel, packageDir, libImported))
32
33
  continue;
33
34
  packageFiles.push(rel);
34
35
  const text = showFile(repo, options.commit, file);
@@ -269,3 +270,66 @@ function toPackage(prefix, file) {
269
270
  return null;
270
271
  return file.slice(prefix.length + 1);
271
272
  }
273
+ const SKIPPED_SOURCE = new Set(["node_modules", "dist", "docs", "tests", "test"]);
274
+ /** src/, a lib/ tree the tests import, or a Go file beside go.mod. Tests, docs, dist, and node_modules stay out. */
275
+ function isVerifySource(rel, packageDir, libImported) {
276
+ if (rel.endsWith(".d.ts"))
277
+ return false;
278
+ const parts = rel.split("/");
279
+ if (parts.some((part) => SKIPPED_SOURCE.has(part)))
280
+ return false;
281
+ if (rel.startsWith("src/"))
282
+ return true;
283
+ if (libImported && rel.startsWith("lib/"))
284
+ return true;
285
+ if (!rel.includes("/") && rel.endsWith(".go") && !rel.endsWith("_test.go") && existsSync(path.join(packageDir, "go.mod")))
286
+ return true;
287
+ return false;
288
+ }
289
+ function testsImportLib(packageDir) {
290
+ const pattern = /(?:require\s*\(\s*|from\s+)['"](?:@[^'"]+\/)?(?:\.\.\/|\.\/)*lib\//;
291
+ for (const dirName of ["tests", "test"]) {
292
+ const dir = path.join(packageDir, dirName);
293
+ if (!existsSync(dir))
294
+ continue;
295
+ if (walkText(dir).some((text) => pattern.test(text)))
296
+ return true;
297
+ }
298
+ return false;
299
+ }
300
+ function walkText(dir) {
301
+ const out = [];
302
+ const visit = (current) => {
303
+ let entries;
304
+ try {
305
+ entries = readdirSync(current);
306
+ }
307
+ catch {
308
+ return;
309
+ }
310
+ for (const name of entries) {
311
+ if (name === "node_modules" || name === "dist")
312
+ continue;
313
+ const full = path.join(current, name);
314
+ let info;
315
+ try {
316
+ info = statSync(full);
317
+ }
318
+ catch {
319
+ continue;
320
+ }
321
+ if (info.isDirectory())
322
+ visit(full);
323
+ else if (info.isFile() && info.size < 1_000_000) {
324
+ try {
325
+ out.push(readFileSync(full, "utf8"));
326
+ }
327
+ catch {
328
+ // An unreadable test file does not count as an import of lib/.
329
+ }
330
+ }
331
+ }
332
+ };
333
+ visit(dir);
334
+ return out;
335
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "probatio",
3
- "version": "0.1.0",
3
+ "version": "0.1.2",
4
4
  "description": "Mutation runner for AI agents. Scores a test suite by the bugs it catches.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -19,7 +19,7 @@
19
19
  "url": "https://github.com/IronAdamant/Probatio/issues"
20
20
  },
21
21
  "scripts": {
22
- "build": "tsc -p tsconfig.json && mkdir -p dist/mutate && cp src/mutate/fail-reporter.mjs src/mutate/node-coverage.mjs src/mutate/py-reporter.py src/mutate/py-pytest-reporter.py src/mutate/probatio-jacoco-run.java dist/mutate/",
22
+ "build": "tsc -p tsconfig.json && mkdir -p dist/mutate && cp src/mutate/fail-reporter.mjs src/mutate/node-coverage.mjs src/mutate/precise-lines.mjs src/mutate/child-lines.mjs src/mutate/mocha-coverage.cjs src/mutate/node-batch.mjs src/mutate/node-batch-loader.mjs src/mutate/py-reporter.py src/mutate/py-pytest-reporter.py src/mutate/probatio-jacoco-run.java dist/mutate/",
23
23
  "test": "tsx --test tests/*.test.ts",
24
24
  "probatio": "tsx src/cli.ts"
25
25
  },