probatio 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/dist/mutate/child-lines.mjs +34 -0
- package/dist/mutate/mocha-coverage.cjs +71 -0
- package/dist/mutate/node-batch-loader.mjs +85 -0
- package/dist/mutate/node-batch.mjs +130 -0
- package/dist/mutate/node-coverage.mjs +37 -180
- package/dist/mutate/precise-lines.mjs +188 -0
- package/dist/mutate/run.js +221 -30
- package/dist/mutate/suite-decision.js +1 -1
- package/dist/mutate/suites.js +39 -4
- package/dist/verify/change.js +66 -2
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -5,7 +5,7 @@ Probatio (Latin: a testing, a proof) is a free testing toolkit under the MIT lic
|
|
|
5
5
|
It remembers what was fixed so agents with small context do not redo or undo work, and it shows
|
|
6
6
|
which tests already touch a change.
|
|
7
7
|
|
|
8
|
-
Status (2026-10-06): `mutate run` discovers the project suite and scores it. Node, pytest, C (when the binary was built with LLVM coverage), Go, Maven/Java, Rust, and
|
|
8
|
+
Status (2026-10-06): `mutate run` discovers the project suite and scores it. Node, pytest, C (when the binary was built with LLVM coverage), Go, Maven/Java, Rust, C#, and Mocha collect a line map on the baseline. Mocha's map comes from its own root hooks. Loading the node:test collector under Mocha does not name Mocha tests. A line no test executes is `no coverage`, and the suite is not started. A plain Node script can write the lines it ran, and the parent test that reads that dump stores them under its own name. A child the parent does not read stays `no coverage`, and the suite is not started. A suite with no line map, a baseline of 5 seconds or more, and more than 30 mutants stops before the first mutant. `verify-change` names the tests the line map ran. It scores `src/`, a `lib/` file when the package tests import `lib/`, and a Go file beside `go.mod`. Tests, docs, `dist/`, and `node_modules` stay unseen. `ran` stays the direct-importer list. Covered mutants of one Node file run one after another inside the suite process that is already running. `--workers` stays 1. A Node test that sets its own `timeout` keeps that time when `--test-timeout-ms` is shorter. A mutant that crashes or leaves that process dirty ends it, and the next mutant starts clean. An uncovered line still does not start the suite. `mutate tally` does not delete a test. A kill is an assertion the suite already had. A sealed miss means the suite did not see the bug. The kill count is not a merge gate. On Commons CSV (Apache-2.0, commit `2c83a308`), two sealed runs agreed: with the new test file hidden, `CSVFormatTest.testFormatThrowsNullPointerException` failed on the reverted printer. `deletedTests` stayed 0. That older JUnit platform did not write a line map, so the first pass was the whole suite and confirm reran the older test. Auspex, markupsafe, and a hand-planted fixture are not that result. The package version in this repository is 0.1.2. npm latest is 0.1.0. The BugsInPy search row is still open, and the rows that apply only when a sealed run misses do not apply, because this catch held. The MCP server speaks one JSON object per line and returns the same JSON as the CLI.
|
|
9
9
|
|
|
10
10
|
Read in this order:
|
|
11
11
|
|
|
@@ -31,7 +31,7 @@ npx probatio mutate tally --out .probatio/runs
|
|
|
31
31
|
|
|
32
32
|
`run` discovers the suite: `run_tests.sh`, Cargo, Go, Swift, Maven, dotnet, pytest, unittest, node:test, Mocha, then `make test` when COBOL tests sit under that Makefile. An unknown layout stops and asks for `--suite-command`. It does not compile one file and call that the suite. `make test` that would curl or wget a missing file stops, and nothing is fetched. A worktree that has no `node_modules` uses the main checkout's. Omitting `--build` runs no build. Pass `--build` with a command when the suite needs one first.
|
|
33
33
|
|
|
34
|
-
Node, pytest, Go, Maven/Java, Rust, and
|
|
34
|
+
Node, pytest, Go, Maven/Java, Rust, C#, and Mocha collect a line map on the baseline. C collects one when LLVM coverage was instrumented. Mocha writes its own `{ files }` record from root hooks, keyed by the Mocha title. The node:test collector is a different file and does not name Mocha tests. The map stores the file, the line, and the test names that hit that line. A later mutant runs only those tests (`go test -run`, Maven `-Dtest`, `cargo test`, `dotnet test --filter`, Mocha `--grep`). A line no test executed is `no coverage`, and the suite is not started. A plain Node script can leave a nameless list of the lines it ran. The parent test that reads that list stores them under its own name. A child the parent does not read stays `no coverage`. Covered mutants of one Node file are applied one after another in the suite process already running, and the line map is reset between them. `--workers` stays 1. A crash or a process left dirty ends that process. The next mutant starts clean. A baseline of 5 seconds or more, with no line map and more than 30 mutants still pending, stops before the first mutant. `next` names that baseline and tells you to narrow `--src` or pass a smaller patch directory. A one-file C, C++, Java, COBOL, or assembly launcher is not a suite discovery returns.
|
|
35
35
|
|
|
36
36
|
The mutant timeout is the baseline duration times 5, and at least 20 seconds, capped by `--suite-timeout-ms`. A timeout is not a kill. `--budget-ms` stops mutant work after the baseline. The first mutant still runs, except for that no-line-map stop. A kill is the test that failed, or a compiler token when the mutant did not build. Confirm is on by default and reruns the failing names. Pytest names that `-k` cannot express are passed as node ids. A usage error, including pytest exit 4, is not a kill. `verify-change` does not confirm a second time. Add `--affected` to limit the file list to tests that can see the change. The default, once a line map exists, runs the tests on the changed line.
|
|
37
37
|
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
// Nameless line dump for a plain script. This is not a node:test process and it
|
|
2
|
+
// does not write a coverage shard. The parent test's afterEach reads the dump.
|
|
3
|
+
import { randomBytes } from "node:crypto"
|
|
4
|
+
import { mkdirSync, writeFileSync } from "node:fs"
|
|
5
|
+
import path from "node:path"
|
|
6
|
+
import { openLineSampler } from "./precise-lines.mjs"
|
|
7
|
+
|
|
8
|
+
const mapPath = process.env.PROBATIO_COVERAGE_MAP
|
|
9
|
+
if (mapPath) {
|
|
10
|
+
let sampler = null
|
|
11
|
+
try {
|
|
12
|
+
sampler = await openLineSampler((rel) => rel.endsWith("child-lines.mjs") || rel.endsWith("precise-lines.mjs"))
|
|
13
|
+
} catch (err) {
|
|
14
|
+
writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
|
|
15
|
+
}
|
|
16
|
+
if (sampler) {
|
|
17
|
+
let wrote = false
|
|
18
|
+
const dump = () => {
|
|
19
|
+
if (wrote) return
|
|
20
|
+
wrote = true
|
|
21
|
+
sampler.positiveLines().then((hits) => {
|
|
22
|
+
const dir = `${mapPath}.children`
|
|
23
|
+
mkdirSync(dir, { recursive: true })
|
|
24
|
+
writeFileSync(
|
|
25
|
+
path.join(dir, `${process.pid}-${randomBytes(4).toString("hex")}.json`),
|
|
26
|
+
JSON.stringify({ hits }),
|
|
27
|
+
)
|
|
28
|
+
}).catch((err) => {
|
|
29
|
+
writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
|
|
30
|
+
})
|
|
31
|
+
}
|
|
32
|
+
process.on("beforeExit", dump)
|
|
33
|
+
}
|
|
34
|
+
}
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
// Mocha root hooks. This file is not node-coverage.mjs. Mocha does not run
|
|
2
|
+
// node:test afterEach/after, so that collector records no Mocha title.
|
|
3
|
+
// These hooks key inspector hits by the current Mocha title and write { files }.
|
|
4
|
+
const { mkdirSync, writeFileSync } = require("node:fs")
|
|
5
|
+
const path = require("node:path")
|
|
6
|
+
|
|
7
|
+
const mapPath = process.env.PROBATIO_COVERAGE_MAP
|
|
8
|
+
if (mapPath) {
|
|
9
|
+
/** @type {Map<string, Map<number, Set<string>>>} */
|
|
10
|
+
const byFile = new Map()
|
|
11
|
+
/** @type {Array<{ file: string, line: number }>} */
|
|
12
|
+
let prelude = []
|
|
13
|
+
/** @type {{ positiveLines: () => Promise<Array<{ file: string, line: number }>> } | null} */
|
|
14
|
+
let sampler = null
|
|
15
|
+
|
|
16
|
+
function add(hits, name) {
|
|
17
|
+
if (!name) return
|
|
18
|
+
for (const hit of hits) {
|
|
19
|
+
let lines = byFile.get(hit.file)
|
|
20
|
+
if (!lines) {
|
|
21
|
+
lines = new Map()
|
|
22
|
+
byFile.set(hit.file, lines)
|
|
23
|
+
}
|
|
24
|
+
let names = lines.get(hit.line)
|
|
25
|
+
if (!names) {
|
|
26
|
+
names = new Set()
|
|
27
|
+
lines.set(hit.line, names)
|
|
28
|
+
}
|
|
29
|
+
names.add(name)
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
function titleOf(test) {
|
|
34
|
+
if (!test) return ""
|
|
35
|
+
if (typeof test.fullTitle === "function") return test.fullTitle()
|
|
36
|
+
return typeof test.title === "string" ? test.title : ""
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
exports.mochaHooks = {
|
|
40
|
+
async beforeAll() {
|
|
41
|
+
try {
|
|
42
|
+
const opened = await import("./precise-lines.mjs")
|
|
43
|
+
sampler = await opened.openLineSampler((rel) => rel.endsWith("mocha-coverage.cjs") || rel.endsWith("precise-lines.mjs") || rel.endsWith("node-coverage.mjs"))
|
|
44
|
+
prelude = await sampler.positiveLines()
|
|
45
|
+
} catch (err) {
|
|
46
|
+
writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
|
|
47
|
+
}
|
|
48
|
+
},
|
|
49
|
+
async afterEach() {
|
|
50
|
+
if (!sampler) return
|
|
51
|
+
const title = titleOf(this.currentTest)
|
|
52
|
+
try {
|
|
53
|
+
add(prelude, title)
|
|
54
|
+
add(await sampler.positiveLines(), title)
|
|
55
|
+
} catch (err) {
|
|
56
|
+
writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
|
|
57
|
+
}
|
|
58
|
+
},
|
|
59
|
+
afterAll() {
|
|
60
|
+
if (byFile.size === 0) return
|
|
61
|
+
const files = {}
|
|
62
|
+
for (const [file, lines] of byFile) {
|
|
63
|
+
const bucket = {}
|
|
64
|
+
for (const [line, names] of lines) bucket[String(line)] = [...names].sort()
|
|
65
|
+
files[file] = bucket
|
|
66
|
+
}
|
|
67
|
+
mkdirSync(path.dirname(mapPath), { recursive: true })
|
|
68
|
+
writeFileSync(mapPath, `${JSON.stringify({ files }, null, 2)}\n`)
|
|
69
|
+
},
|
|
70
|
+
}
|
|
71
|
+
}
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
// Cache-busts package files so the next mutant in this process loads the patched source.
|
|
2
|
+
// The generation lives in a file because the loader thread does not see later env edits.
|
|
3
|
+
import { createRequire } from "node:module"
|
|
4
|
+
import { readFileSync, realpathSync } from "node:fs"
|
|
5
|
+
import path from "node:path"
|
|
6
|
+
import { fileURLToPath } from "node:url"
|
|
7
|
+
|
|
8
|
+
const require = createRequire(import.meta.url)
|
|
9
|
+
const ts = require("typescript")
|
|
10
|
+
|
|
11
|
+
function generation() {
|
|
12
|
+
const file = process.env.PB_GEN
|
|
13
|
+
if (!file) return ""
|
|
14
|
+
try {
|
|
15
|
+
return readFileSync(file, "utf8").trim()
|
|
16
|
+
} catch {
|
|
17
|
+
return ""
|
|
18
|
+
}
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
function packageRoot() {
|
|
22
|
+
const root = process.env.PB_ROOT
|
|
23
|
+
if (!root) return ""
|
|
24
|
+
try {
|
|
25
|
+
return realpathSync(root)
|
|
26
|
+
} catch {
|
|
27
|
+
return root
|
|
28
|
+
}
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
function underPackage(abs) {
|
|
32
|
+
const root = packageRoot()
|
|
33
|
+
if (!root) return false
|
|
34
|
+
let real = abs
|
|
35
|
+
try {
|
|
36
|
+
real = realpathSync(abs)
|
|
37
|
+
} catch {
|
|
38
|
+
return false
|
|
39
|
+
}
|
|
40
|
+
return real === root || real.startsWith(root.endsWith(path.sep) ? root : root + path.sep)
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
export async function resolve(specifier, context, nextResolve) {
|
|
44
|
+
const resolved = await nextResolve(specifier, context)
|
|
45
|
+
const gen = generation()
|
|
46
|
+
if (!gen) return resolved
|
|
47
|
+
let abs
|
|
48
|
+
try {
|
|
49
|
+
abs = fileURLToPath(resolved.url.split("?")[0])
|
|
50
|
+
} catch {
|
|
51
|
+
return resolved
|
|
52
|
+
}
|
|
53
|
+
if (!underPackage(abs)) return resolved
|
|
54
|
+
if (abs.split(path.sep).includes("node_modules")) return resolved
|
|
55
|
+
const url = new URL(resolved.url.split("?")[0])
|
|
56
|
+
url.searchParams.set("pb", gen)
|
|
57
|
+
return { url: url.href, shortCircuit: true }
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
export async function load(url, context, nextLoad) {
|
|
61
|
+
if (!url.includes("pb=")) return nextLoad(url, context)
|
|
62
|
+
const clean = url.split("?")[0]
|
|
63
|
+
let abs
|
|
64
|
+
try {
|
|
65
|
+
abs = fileURLToPath(clean)
|
|
66
|
+
} catch {
|
|
67
|
+
return nextLoad(url, context)
|
|
68
|
+
}
|
|
69
|
+
const source = readFileSync(abs, "utf8")
|
|
70
|
+
if (abs.endsWith(".ts") || abs.endsWith(".tsx") || abs.endsWith(".mts")) {
|
|
71
|
+
const transpiled = ts.transpileModule(source, {
|
|
72
|
+
fileName: abs,
|
|
73
|
+
compilerOptions: {
|
|
74
|
+
module: ts.ModuleKind.ESNext,
|
|
75
|
+
target: ts.ScriptTarget.ES2022,
|
|
76
|
+
sourceMap: false,
|
|
77
|
+
},
|
|
78
|
+
})
|
|
79
|
+
return { format: "module", source: transpiled.outputText, shortCircuit: true }
|
|
80
|
+
}
|
|
81
|
+
if (abs.endsWith(".mjs") || abs.endsWith(".js") || source.includes("import ") || source.includes("export ")) {
|
|
82
|
+
return { format: "module", source, shortCircuit: true }
|
|
83
|
+
}
|
|
84
|
+
return { format: "commonjs", source, shortCircuit: true }
|
|
85
|
+
}
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
// One suite process for covered mutants of one file. Jobs arrive on stdin.
|
|
2
|
+
// A mutant that exits this process ends it. The parent starts a clean process for the next mutant.
|
|
3
|
+
import { createInterface } from "node:readline"
|
|
4
|
+
import { realpathSync, writeFileSync } from "node:fs"
|
|
5
|
+
import { register } from "node:module"
|
|
6
|
+
import { run } from "node:test"
|
|
7
|
+
import path from "node:path"
|
|
8
|
+
import { openLineSampler } from "./precise-lines.mjs"
|
|
9
|
+
|
|
10
|
+
const loader = new URL("./node-batch-loader.mjs", import.meta.url).href
|
|
11
|
+
register(loader, { parentURL: import.meta.url })
|
|
12
|
+
|
|
13
|
+
let sampler = null
|
|
14
|
+
try {
|
|
15
|
+
sampler = await openLineSampler()
|
|
16
|
+
} catch {
|
|
17
|
+
sampler = null
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
writeFileSync(process.env.PB_READY, `${JSON.stringify({ pid: process.pid })}\n`)
|
|
21
|
+
|
|
22
|
+
function escapeRegExp(value) {
|
|
23
|
+
return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
let generation = 0
|
|
27
|
+
|
|
28
|
+
async function runJob(job) {
|
|
29
|
+
generation += 1
|
|
30
|
+
writeFileSync(process.env.PB_GEN, String(generation))
|
|
31
|
+
if (sampler) await sampler.reset()
|
|
32
|
+
const files = (job.tests || []).map((rel) => path.join(job.root, rel))
|
|
33
|
+
const patterns = (job.names || []).filter((name) => name.length > 0).map((name) => new RegExp(`^${escapeRegExp(name)}$`))
|
|
34
|
+
const failed = []
|
|
35
|
+
const names = []
|
|
36
|
+
const testTimeout = job.testTimeoutMs > 0 ? job.testTimeoutMs : job.timeoutMs
|
|
37
|
+
const stream = run({
|
|
38
|
+
files,
|
|
39
|
+
isolation: "none",
|
|
40
|
+
timeout: testTimeout,
|
|
41
|
+
...(patterns.length > 0 ? { testNamePatterns: patterns } : {}),
|
|
42
|
+
})
|
|
43
|
+
let lastEvent = Date.now()
|
|
44
|
+
const started = Date.now()
|
|
45
|
+
let ended = false
|
|
46
|
+
let timer = null
|
|
47
|
+
const collected = (async () => {
|
|
48
|
+
for await (const event of stream) {
|
|
49
|
+
lastEvent = Date.now()
|
|
50
|
+
if (event.type !== "test:pass" && event.type !== "test:fail") continue
|
|
51
|
+
const data = event.data || {}
|
|
52
|
+
const name = typeof data.name === "string" ? data.name : ""
|
|
53
|
+
if (!name || files.includes(name)) continue
|
|
54
|
+
if (!names.includes(name)) names.push(name)
|
|
55
|
+
if (event.type === "test:fail") {
|
|
56
|
+
const error = data.details && data.details.error
|
|
57
|
+
const text = error ? `${error.failureType || ""} ${error.message || ""} ${error.cause || ""}` : ""
|
|
58
|
+
const fileTimeout = /testTimeoutFailure|timed out/i.test(text) && fileNameIs(name, data.file || "")
|
|
59
|
+
failed.push({ name, file: data.file || "", line: data.line || 0, ...(fileTimeout ? { fileTimeout: true } : {}) })
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
ended = true
|
|
63
|
+
})()
|
|
64
|
+
let dirty = false
|
|
65
|
+
try {
|
|
66
|
+
dirty = await Promise.race([
|
|
67
|
+
collected.then(() => false),
|
|
68
|
+
new Promise((resolve) => {
|
|
69
|
+
timer = setInterval(() => {
|
|
70
|
+
if (ended) {
|
|
71
|
+
resolve(false)
|
|
72
|
+
return
|
|
73
|
+
}
|
|
74
|
+
const idle = Date.now() - lastEvent
|
|
75
|
+
const overdue = Date.now() - started >= testTimeout && idle >= 400 && names.length > 0
|
|
76
|
+
if (!overdue) return
|
|
77
|
+
resolve(true)
|
|
78
|
+
}, 50)
|
|
79
|
+
timer.unref()
|
|
80
|
+
}),
|
|
81
|
+
])
|
|
82
|
+
} finally {
|
|
83
|
+
if (timer) clearInterval(timer)
|
|
84
|
+
}
|
|
85
|
+
if (dirty && failed.length === 0) {
|
|
86
|
+
for (const rel of job.tests || []) {
|
|
87
|
+
const posix = rel.split(path.sep).join("/")
|
|
88
|
+
let file = path.join(job.root, rel)
|
|
89
|
+
try {
|
|
90
|
+
file = realpathSync(file)
|
|
91
|
+
} catch {
|
|
92
|
+
// The reporter uses the path it opened. A missing file keeps the joined path.
|
|
93
|
+
}
|
|
94
|
+
failed.push({ name: posix, file, line: 1, fileTimeout: true })
|
|
95
|
+
if (!names.includes(posix)) names.push(posix)
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
if (sampler && !dirty) await sampler.reset()
|
|
99
|
+
const pattern = (job.names || []).join("|")
|
|
100
|
+
const command = ["node", "--test", ...(pattern ? [`--test-name-pattern=${pattern}`] : []), ...(job.tests || [])].join(" ")
|
|
101
|
+
return { id: job.id, pid: process.pid, failed, names, command, dirty }
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
function fileNameIs(name, file) {
|
|
105
|
+
const base = String(file).split(/[/\\]/).pop()
|
|
106
|
+
if (!base) return false
|
|
107
|
+
return name === file || name === base || name.endsWith(`/${base}`) || name.endsWith(`\\${base}`)
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
const lines = createInterface({ input: process.stdin })
|
|
111
|
+
for await (const line of lines) {
|
|
112
|
+
if (!line.trim()) continue
|
|
113
|
+
const job = JSON.parse(line)
|
|
114
|
+
// node:test with isolation "none" waits until the event loop is idle.
|
|
115
|
+
// An open readline on stdin is that handle, so the job would never finish.
|
|
116
|
+
lines.pause()
|
|
117
|
+
process.stdin.unref()
|
|
118
|
+
try {
|
|
119
|
+
const reply = await runJob(job)
|
|
120
|
+
writeFileSync(job.replyFile, `${JSON.stringify(reply)}\n`)
|
|
121
|
+
// An open handle after the tests finished is the same file timeout node --test reports.
|
|
122
|
+
// The process is dirty, so it ends and the next mutant starts clean.
|
|
123
|
+
if (reply.dirty) process.exit(0)
|
|
124
|
+
} catch (err) {
|
|
125
|
+
writeFileSync(job.replyFile, `${JSON.stringify({ id: job.id, pid: process.pid, failed: [], names: [], error: err instanceof Error ? err.message : String(err) })}\n`)
|
|
126
|
+
} finally {
|
|
127
|
+
if (!process.stdin.destroyed) process.stdin.ref()
|
|
128
|
+
lines.resume()
|
|
129
|
+
}
|
|
130
|
+
}
|
|
@@ -1,200 +1,29 @@
|
|
|
1
1
|
// Baseline-only per-test line map for node --test. Loaded with --import in each
|
|
2
2
|
// test-file process. Writes one shard; the runner merges shards after the suite.
|
|
3
3
|
// takePreciseCoverage resets counters, so each sample is only what ran since the last take.
|
|
4
|
+
// A plain-script child does not run these hooks. It may leave a nameless hit list in
|
|
5
|
+
// ${map}.children. afterEach stores that list under the parent test before the shard is written.
|
|
4
6
|
import { randomBytes } from "node:crypto"
|
|
5
|
-
import { mkdirSync, writeFileSync } from "node:fs"
|
|
6
|
-
import { Session } from "node:inspector/promises"
|
|
7
|
-
import { findSourceMap } from "node:module"
|
|
7
|
+
import { existsSync, mkdirSync, readdirSync, readFileSync, unlinkSync, writeFileSync } from "node:fs"
|
|
8
8
|
import { after, afterEach, before } from "node:test"
|
|
9
9
|
import path from "node:path"
|
|
10
|
-
import {
|
|
11
|
-
|
|
12
|
-
const B64 = new Int32Array(128)
|
|
13
|
-
for (let i = 0; i < 64; i++) B64["ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/".charCodeAt(i)] = i
|
|
14
|
-
|
|
15
|
-
function decodeSegments(row) {
|
|
16
|
-
const segments = []
|
|
17
|
-
let i = 0
|
|
18
|
-
while (i < row.length) {
|
|
19
|
-
const fields = []
|
|
20
|
-
while (i < row.length && row.charCodeAt(i) !== 44) {
|
|
21
|
-
let value = 0
|
|
22
|
-
let shift = 0
|
|
23
|
-
let digit = 0
|
|
24
|
-
do {
|
|
25
|
-
digit = B64[row.charCodeAt(i++)]
|
|
26
|
-
value |= (digit & 31) << shift
|
|
27
|
-
shift += 5
|
|
28
|
-
} while (digit & 32)
|
|
29
|
-
fields.push(value & 1 ? -(value >> 1) : value >> 1)
|
|
30
|
-
}
|
|
31
|
-
if (row.charCodeAt(i) === 44) i++
|
|
32
|
-
if (fields.length > 0) segments.push(fields)
|
|
33
|
-
}
|
|
34
|
-
return segments
|
|
35
|
-
}
|
|
10
|
+
import { openLineSampler } from "./precise-lines.mjs"
|
|
36
11
|
|
|
37
12
|
const mapPath = process.env.PROBATIO_COVERAGE_MAP
|
|
38
13
|
if (mapPath) {
|
|
39
|
-
|
|
40
|
-
let started = false
|
|
14
|
+
let sampler = null
|
|
41
15
|
try {
|
|
42
|
-
|
|
43
|
-
await session.post("Profiler.enable")
|
|
44
|
-
await session.post("Debugger.enable")
|
|
45
|
-
await session.post("Profiler.startPreciseCoverage", { callCount: true, detailed: true })
|
|
46
|
-
started = true
|
|
16
|
+
sampler = await openLineSampler((rel) => rel.endsWith("node-coverage.mjs") || rel.endsWith("precise-lines.mjs") || rel.endsWith("child-lines.mjs"))
|
|
47
17
|
} catch (err) {
|
|
48
18
|
writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
|
|
49
19
|
}
|
|
50
20
|
|
|
51
|
-
if (
|
|
52
|
-
const textCache = new Map()
|
|
53
|
-
const indexCache = new Map()
|
|
21
|
+
if (sampler) {
|
|
54
22
|
/** @type {Map<string, Map<number, Set<string>>>} */
|
|
55
23
|
const byFile = new Map()
|
|
56
24
|
/** @type {Array<{ file: string, line: number }>} */
|
|
57
25
|
let prelude = []
|
|
58
26
|
|
|
59
|
-
async function scriptText(script) {
|
|
60
|
-
if (textCache.has(script.scriptId)) return textCache.get(script.scriptId)
|
|
61
|
-
let text = ""
|
|
62
|
-
try {
|
|
63
|
-
const got = await session.post("Debugger.getScriptSource", { scriptId: script.scriptId })
|
|
64
|
-
text = got.scriptSource || ""
|
|
65
|
-
} catch {
|
|
66
|
-
text = ""
|
|
67
|
-
}
|
|
68
|
-
textCache.set(script.scriptId, text)
|
|
69
|
-
return text
|
|
70
|
-
}
|
|
71
|
-
|
|
72
|
-
function relPath(url) {
|
|
73
|
-
if (!url || url.startsWith("node:")) return null
|
|
74
|
-
let abs
|
|
75
|
-
try {
|
|
76
|
-
abs = fileURLToPath(url.split("?")[0].split("#")[0])
|
|
77
|
-
} catch {
|
|
78
|
-
return null
|
|
79
|
-
}
|
|
80
|
-
const rel = path.relative(process.cwd(), abs).split(path.sep).join("/")
|
|
81
|
-
if (!rel || rel.startsWith("..") || path.isAbsolute(rel)) return null
|
|
82
|
-
if (rel.split("/").includes("node_modules")) return null
|
|
83
|
-
if (rel.endsWith("node-coverage.mjs")) return null
|
|
84
|
-
return rel
|
|
85
|
-
}
|
|
86
|
-
|
|
87
|
-
function indexFor(scriptId, text, sm) {
|
|
88
|
-
if (indexCache.has(scriptId)) return indexCache.get(scriptId)
|
|
89
|
-
const mapped = sm && sm.payload && typeof sm.payload.mappings === "string" ? linesFromMappings(text, sm.payload.mappings) : null
|
|
90
|
-
const lines = mapped && hasLine(mapped) ? mapped : linesFromNewlines(text, sm)
|
|
91
|
-
indexCache.set(scriptId, lines)
|
|
92
|
-
return lines
|
|
93
|
-
}
|
|
94
|
-
|
|
95
|
-
function hasLine(lines) {
|
|
96
|
-
for (let i = 0; i < lines.length; i++) if (lines[i] > 0) return true
|
|
97
|
-
return false
|
|
98
|
-
}
|
|
99
|
-
|
|
100
|
-
function linesFromNewlines(text, sm) {
|
|
101
|
-
const lines = new Int32Array(text.length + 1)
|
|
102
|
-
if (!sm) {
|
|
103
|
-
let line = 1
|
|
104
|
-
for (let i = 0; i < text.length; i++) {
|
|
105
|
-
lines[i] = line
|
|
106
|
-
if (text.charCodeAt(i) === 10) line++
|
|
107
|
-
}
|
|
108
|
-
return lines
|
|
109
|
-
}
|
|
110
|
-
// Node 22 reports SourceMap.findEntry originalLine as 0-based.
|
|
111
|
-
let genLine = 0
|
|
112
|
-
let col = 0
|
|
113
|
-
for (let i = 0; i < text.length; i++) {
|
|
114
|
-
const entry = sm.findEntry(genLine, col)
|
|
115
|
-
if (entry && Number.isInteger(entry.originalLine) && entry.originalLine >= 0) lines[i] = entry.originalLine + 1
|
|
116
|
-
if (text.charCodeAt(i) === 10) {
|
|
117
|
-
genLine++
|
|
118
|
-
col = 0
|
|
119
|
-
} else {
|
|
120
|
-
col++
|
|
121
|
-
}
|
|
122
|
-
}
|
|
123
|
-
return lines
|
|
124
|
-
}
|
|
125
|
-
|
|
126
|
-
function linesFromMappings(text, mappings) {
|
|
127
|
-
const lines = new Int32Array(text.length + 1)
|
|
128
|
-
const starts = [0]
|
|
129
|
-
for (let i = 0; i < text.length; i++) if (text.charCodeAt(i) === 10) starts.push(i + 1)
|
|
130
|
-
let carryLine = 0
|
|
131
|
-
const rows = mappings.split(";")
|
|
132
|
-
for (let genLine = 0; genLine < rows.length && genLine < starts.length; genLine++) {
|
|
133
|
-
const start = starts[genLine]
|
|
134
|
-
const next = genLine + 1 < starts.length ? starts[genLine + 1] : text.length
|
|
135
|
-
let col = 0
|
|
136
|
-
let cursor = start
|
|
137
|
-
let active = -1
|
|
138
|
-
for (const fields of decodeSegments(rows[genLine])) {
|
|
139
|
-
col += fields[0]
|
|
140
|
-
const at = Math.min(next, start + Math.max(0, col))
|
|
141
|
-
if (active >= 0) {
|
|
142
|
-
for (let i = cursor; i < at; i++) lines[i] = active + 1
|
|
143
|
-
}
|
|
144
|
-
cursor = at
|
|
145
|
-
if (fields.length >= 4) {
|
|
146
|
-
carryLine += fields[2]
|
|
147
|
-
active = carryLine
|
|
148
|
-
} else {
|
|
149
|
-
active = -1
|
|
150
|
-
}
|
|
151
|
-
}
|
|
152
|
-
if (active >= 0) {
|
|
153
|
-
for (let i = cursor; i < next; i++) lines[i] = active + 1
|
|
154
|
-
}
|
|
155
|
-
}
|
|
156
|
-
return lines
|
|
157
|
-
}
|
|
158
|
-
|
|
159
|
-
function applyRanges(text, script, lines) {
|
|
160
|
-
const ranges = []
|
|
161
|
-
for (const fn of script.functions || []) {
|
|
162
|
-
for (const range of fn.ranges || []) ranges.push(range)
|
|
163
|
-
}
|
|
164
|
-
// Outer ranges first. A nested block then overwrites its own offsets, so a dead
|
|
165
|
-
// branch stays at 0 while a condition earlier on that same line can stay covered.
|
|
166
|
-
ranges.sort((a, b) => a.startOffset - b.startOffset || b.endOffset - a.endOffset)
|
|
167
|
-
const offsets = new Int32Array(text.length)
|
|
168
|
-
for (const range of ranges) {
|
|
169
|
-
const end = Math.min(text.length, range.endOffset)
|
|
170
|
-
for (let i = Math.max(0, range.startOffset); i < end; i++) offsets[i] = range.count
|
|
171
|
-
}
|
|
172
|
-
const counts = new Map()
|
|
173
|
-
for (let i = 0; i < text.length; i++) {
|
|
174
|
-
const line = lines[i]
|
|
175
|
-
if (line <= 0 || offsets[i] <= 0) continue
|
|
176
|
-
if (offsets[i] > (counts.get(line) || 0)) counts.set(line, offsets[i])
|
|
177
|
-
}
|
|
178
|
-
return counts
|
|
179
|
-
}
|
|
180
|
-
|
|
181
|
-
async function positiveLines() {
|
|
182
|
-
const cov = await session.post("Profiler.takePreciseCoverage")
|
|
183
|
-
/** @type {Array<{ file: string, line: number }>} */
|
|
184
|
-
const found = []
|
|
185
|
-
for (const script of cov.result || []) {
|
|
186
|
-
const file = relPath(script.url)
|
|
187
|
-
if (!file) continue
|
|
188
|
-
const text = await scriptText(script)
|
|
189
|
-
if (!text) continue
|
|
190
|
-
const counts = applyRanges(text, script, indexFor(script.scriptId, text, findSourceMap(script.url)))
|
|
191
|
-
for (const [line, count] of counts) {
|
|
192
|
-
if (count > 0) found.push({ file, line })
|
|
193
|
-
}
|
|
194
|
-
}
|
|
195
|
-
return found
|
|
196
|
-
}
|
|
197
|
-
|
|
198
27
|
function add(hits, name) {
|
|
199
28
|
if (!name) return
|
|
200
29
|
for (const hit of hits) {
|
|
@@ -212,9 +41,36 @@ if (mapPath) {
|
|
|
212
41
|
}
|
|
213
42
|
}
|
|
214
43
|
|
|
44
|
+
function takeChildHits() {
|
|
45
|
+
const dir = `${mapPath}.children`
|
|
46
|
+
if (!existsSync(dir)) return []
|
|
47
|
+
/** @type {Array<{ file: string, line: number }>} */
|
|
48
|
+
const hits = []
|
|
49
|
+
for (const name of readdirSync(dir)) {
|
|
50
|
+
if (!name.endsWith(".json")) continue
|
|
51
|
+
const full = path.join(dir, name)
|
|
52
|
+
try {
|
|
53
|
+
const parsed = JSON.parse(readFileSync(full, "utf8"))
|
|
54
|
+
const list = parsed && Array.isArray(parsed.hits) ? parsed.hits : []
|
|
55
|
+
for (const hit of list) {
|
|
56
|
+
if (!hit || typeof hit.file !== "string" || !Number.isInteger(hit.line)) continue
|
|
57
|
+
hits.push({ file: hit.file, line: hit.line })
|
|
58
|
+
}
|
|
59
|
+
} catch {
|
|
60
|
+
// A torn dump is ignored. The parent does not invent a line.
|
|
61
|
+
}
|
|
62
|
+
try {
|
|
63
|
+
unlinkSync(full)
|
|
64
|
+
} catch {
|
|
65
|
+
// The next test must not inherit this dump.
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
return hits
|
|
69
|
+
}
|
|
70
|
+
|
|
215
71
|
before(async () => {
|
|
216
72
|
try {
|
|
217
|
-
prelude = await positiveLines()
|
|
73
|
+
prelude = await sampler.positiveLines()
|
|
218
74
|
} catch (err) {
|
|
219
75
|
writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
|
|
220
76
|
}
|
|
@@ -223,7 +79,8 @@ if (mapPath) {
|
|
|
223
79
|
afterEach(async (t) => {
|
|
224
80
|
try {
|
|
225
81
|
add(prelude, t.name)
|
|
226
|
-
add(await positiveLines(), t.name)
|
|
82
|
+
add(await sampler.positiveLines(), t.name)
|
|
83
|
+
add(takeChildHits(), t.name)
|
|
227
84
|
} catch (err) {
|
|
228
85
|
writeFileSync(`${mapPath}.error`, `${err instanceof Error ? err.message : String(err)}\n`)
|
|
229
86
|
}
|
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
// Inspector line samples shared by the node:test collector, the Mocha collector,
|
|
2
|
+
// and a plain-script child. This file has no test hooks and writes no shard.
|
|
3
|
+
import { Session } from "node:inspector/promises"
|
|
4
|
+
import { findSourceMap } from "node:module"
|
|
5
|
+
import path from "node:path"
|
|
6
|
+
import { fileURLToPath } from "node:url"
|
|
7
|
+
|
|
8
|
+
const B64 = new Int32Array(128)
|
|
9
|
+
for (let i = 0; i < 64; i++) B64["ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/".charCodeAt(i)] = i
|
|
10
|
+
|
|
11
|
+
function decodeSegments(row) {
|
|
12
|
+
const segments = []
|
|
13
|
+
let i = 0
|
|
14
|
+
while (i < row.length) {
|
|
15
|
+
const fields = []
|
|
16
|
+
while (i < row.length && row.charCodeAt(i) !== 44) {
|
|
17
|
+
let value = 0
|
|
18
|
+
let shift = 0
|
|
19
|
+
let digit = 0
|
|
20
|
+
do {
|
|
21
|
+
digit = B64[row.charCodeAt(i++)]
|
|
22
|
+
value |= (digit & 31) << shift
|
|
23
|
+
shift += 5
|
|
24
|
+
} while (digit & 32)
|
|
25
|
+
fields.push(value & 1 ? -(value >> 1) : value >> 1)
|
|
26
|
+
}
|
|
27
|
+
if (row.charCodeAt(i) === 44) i++
|
|
28
|
+
if (fields.length > 0) segments.push(fields)
|
|
29
|
+
}
|
|
30
|
+
return segments
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
/**
|
|
34
|
+
* @param {(rel: string) => boolean} [skipRel]
|
|
35
|
+
* @returns {Promise<{ positiveLines: () => Promise<Array<{ file: string, line: number }>>, reset: () => Promise<void> }>}
|
|
36
|
+
*/
|
|
37
|
+
export async function openLineSampler(skipRel) {
|
|
38
|
+
const session = new Session()
|
|
39
|
+
session.connect()
|
|
40
|
+
await session.post("Profiler.enable")
|
|
41
|
+
await session.post("Debugger.enable")
|
|
42
|
+
await session.post("Profiler.startPreciseCoverage", { callCount: true, detailed: true })
|
|
43
|
+
|
|
44
|
+
const textCache = new Map()
|
|
45
|
+
const indexCache = new Map()
|
|
46
|
+
|
|
47
|
+
async function scriptText(script) {
|
|
48
|
+
if (textCache.has(script.scriptId)) return textCache.get(script.scriptId)
|
|
49
|
+
let text = ""
|
|
50
|
+
try {
|
|
51
|
+
const got = await session.post("Debugger.getScriptSource", { scriptId: script.scriptId })
|
|
52
|
+
text = got.scriptSource || ""
|
|
53
|
+
} catch {
|
|
54
|
+
text = ""
|
|
55
|
+
}
|
|
56
|
+
textCache.set(script.scriptId, text)
|
|
57
|
+
return text
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
function relPath(url) {
|
|
61
|
+
if (!url || url.startsWith("node:")) return null
|
|
62
|
+
let abs
|
|
63
|
+
try {
|
|
64
|
+
abs = fileURLToPath(url.split("?")[0].split("#")[0])
|
|
65
|
+
} catch {
|
|
66
|
+
return null
|
|
67
|
+
}
|
|
68
|
+
const rel = path.relative(process.cwd(), abs).split(path.sep).join("/")
|
|
69
|
+
if (!rel || rel.startsWith("..") || path.isAbsolute(rel)) return null
|
|
70
|
+
if (rel.split("/").includes("node_modules")) return null
|
|
71
|
+
if (skipRel && skipRel(rel)) return null
|
|
72
|
+
return rel
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
function indexFor(scriptId, text, sm) {
|
|
76
|
+
if (indexCache.has(scriptId)) return indexCache.get(scriptId)
|
|
77
|
+
const mapped = sm && sm.payload && typeof sm.payload.mappings === "string" ? linesFromMappings(text, sm.payload.mappings) : null
|
|
78
|
+
const lines = mapped && hasLine(mapped) ? mapped : linesFromNewlines(text, sm)
|
|
79
|
+
indexCache.set(scriptId, lines)
|
|
80
|
+
return lines
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
function hasLine(lines) {
|
|
84
|
+
for (let i = 0; i < lines.length; i++) if (lines[i] > 0) return true
|
|
85
|
+
return false
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
function linesFromNewlines(text, sm) {
|
|
89
|
+
const lines = new Int32Array(text.length + 1)
|
|
90
|
+
if (!sm) {
|
|
91
|
+
let line = 1
|
|
92
|
+
for (let i = 0; i < text.length; i++) {
|
|
93
|
+
lines[i] = line
|
|
94
|
+
if (text.charCodeAt(i) === 10) line++
|
|
95
|
+
}
|
|
96
|
+
return lines
|
|
97
|
+
}
|
|
98
|
+
let genLine = 0
|
|
99
|
+
let col = 0
|
|
100
|
+
for (let i = 0; i < text.length; i++) {
|
|
101
|
+
const entry = sm.findEntry(genLine, col)
|
|
102
|
+
if (entry && Number.isInteger(entry.originalLine) && entry.originalLine >= 0) lines[i] = entry.originalLine + 1
|
|
103
|
+
if (text.charCodeAt(i) === 10) {
|
|
104
|
+
genLine++
|
|
105
|
+
col = 0
|
|
106
|
+
} else {
|
|
107
|
+
col++
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
return lines
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
function linesFromMappings(text, mappings) {
|
|
114
|
+
const lines = new Int32Array(text.length + 1)
|
|
115
|
+
const starts = [0]
|
|
116
|
+
for (let i = 0; i < text.length; i++) if (text.charCodeAt(i) === 10) starts.push(i + 1)
|
|
117
|
+
let carryLine = 0
|
|
118
|
+
const rows = mappings.split(";")
|
|
119
|
+
for (let genLine = 0; genLine < rows.length && genLine < starts.length; genLine++) {
|
|
120
|
+
const start = starts[genLine]
|
|
121
|
+
const next = genLine + 1 < starts.length ? starts[genLine + 1] : text.length
|
|
122
|
+
let col = 0
|
|
123
|
+
let cursor = start
|
|
124
|
+
let active = -1
|
|
125
|
+
for (const fields of decodeSegments(rows[genLine])) {
|
|
126
|
+
col += fields[0]
|
|
127
|
+
const at = Math.min(next, start + Math.max(0, col))
|
|
128
|
+
if (active >= 0) {
|
|
129
|
+
for (let i = cursor; i < at; i++) lines[i] = active + 1
|
|
130
|
+
}
|
|
131
|
+
cursor = at
|
|
132
|
+
if (fields.length >= 4) {
|
|
133
|
+
carryLine += fields[2]
|
|
134
|
+
active = carryLine
|
|
135
|
+
} else {
|
|
136
|
+
active = -1
|
|
137
|
+
}
|
|
138
|
+
}
|
|
139
|
+
if (active >= 0) {
|
|
140
|
+
for (let i = cursor; i < next; i++) lines[i] = active + 1
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
return lines
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
function applyRanges(text, script, lines) {
|
|
147
|
+
const ranges = []
|
|
148
|
+
for (const fn of script.functions || []) {
|
|
149
|
+
for (const range of fn.ranges || []) ranges.push(range)
|
|
150
|
+
}
|
|
151
|
+
ranges.sort((a, b) => a.startOffset - b.startOffset || b.endOffset - a.endOffset)
|
|
152
|
+
const offsets = new Int32Array(text.length)
|
|
153
|
+
for (const range of ranges) {
|
|
154
|
+
const end = Math.min(text.length, range.endOffset)
|
|
155
|
+
for (let i = Math.max(0, range.startOffset); i < end; i++) offsets[i] = range.count
|
|
156
|
+
}
|
|
157
|
+
const counts = new Map()
|
|
158
|
+
for (let i = 0; i < text.length; i++) {
|
|
159
|
+
const line = lines[i]
|
|
160
|
+
if (line <= 0 || offsets[i] <= 0) continue
|
|
161
|
+
if (offsets[i] > (counts.get(line) || 0)) counts.set(line, offsets[i])
|
|
162
|
+
}
|
|
163
|
+
return counts
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
async function positiveLines() {
|
|
167
|
+
const cov = await session.post("Profiler.takePreciseCoverage")
|
|
168
|
+
const found = []
|
|
169
|
+
for (const script of cov.result || []) {
|
|
170
|
+
const file = relPath(script.url)
|
|
171
|
+
if (!file) continue
|
|
172
|
+
const text = await scriptText(script)
|
|
173
|
+
if (!text) continue
|
|
174
|
+
const counts = applyRanges(text, script, indexFor(script.scriptId, text, findSourceMap(script.url)))
|
|
175
|
+
for (const [line, count] of counts) {
|
|
176
|
+
if (count > 0) found.push({ file, line })
|
|
177
|
+
}
|
|
178
|
+
}
|
|
179
|
+
return found
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
return {
|
|
183
|
+
positiveLines,
|
|
184
|
+
reset: async () => {
|
|
185
|
+
await session.post("Profiler.takePreciseCoverage")
|
|
186
|
+
},
|
|
187
|
+
}
|
|
188
|
+
}
|
package/dist/mutate/run.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { spawn } from "node:child_process";
|
|
2
2
|
import { createHash, randomBytes } from "node:crypto";
|
|
3
|
-
import { copyFileSync, existsSync, mkdirSync, readdirSync, readFileSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
|
|
3
|
+
import { appendFileSync, copyFileSync, existsSync, mkdirSync, readdirSync, readFileSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
|
|
4
4
|
import { tmpdir } from "node:os";
|
|
5
5
|
import path from "node:path";
|
|
6
6
|
import { fileURLToPath } from "node:url";
|
|
@@ -13,7 +13,7 @@ import { writeRustCoverage } from "./rust-coverage.js";
|
|
|
13
13
|
import { applyPatch, changedLines, chooseDirection, filesInDiff, git, restoreTree } from "./patch.js";
|
|
14
14
|
import { killLabel } from "./kill-label.js";
|
|
15
15
|
import { TIMEOUT_FLOOR_MS, TIMEOUT_MULTIPLE, TIMEOUT_NEXT, WHOLE_PROGRAM_TEST, WHOLE_SUITE_NEXT, campaignSummary, decideSuite, hasLineMap, noCoverageNext, resumeNext, scaleStop, timeoutLimitMs, } from "./suite-decision.js";
|
|
16
|
-
import { commandSuite, discoverSuite, runDiscoveredSuite, sourceCandidates } from "./suites.js";
|
|
16
|
+
import { commandSuite, discoverSuite, nodeTestTimeoutMs, runDiscoveredSuite, sourceCandidates } from "./suites.js";
|
|
17
17
|
import { duplicateTitles } from "./titles.js";
|
|
18
18
|
export const reporterPath = fileURLToPath(new URL("./fail-reporter.mjs", import.meta.url));
|
|
19
19
|
export function parseReport(stdout) {
|
|
@@ -89,34 +89,61 @@ export async function runMutants(options) {
|
|
|
89
89
|
budgetStarted = Date.now();
|
|
90
90
|
const worker = async (worktree) => {
|
|
91
91
|
const treePkg = packageIn(worktree, repo, packageDir);
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
92
|
+
let batch = null;
|
|
93
|
+
const dropBatch = () => {
|
|
94
|
+
batch?.stop();
|
|
95
|
+
batch = null;
|
|
96
|
+
};
|
|
97
|
+
try {
|
|
98
|
+
for (;;) {
|
|
99
|
+
if (startedMutants > 0 && overBudget()) {
|
|
100
|
+
budgetHit = true;
|
|
101
|
+
return;
|
|
102
|
+
}
|
|
103
|
+
const patch = queue.shift();
|
|
104
|
+
if (!patch)
|
|
105
|
+
return;
|
|
106
|
+
if (startedMutants > 0 && overBudget()) {
|
|
107
|
+
queue.unshift(patch);
|
|
108
|
+
budgetHit = true;
|
|
109
|
+
return;
|
|
110
|
+
}
|
|
111
|
+
const id = path.basename(patch, ".patch");
|
|
112
|
+
const raw = readFileSync(patch, "utf8");
|
|
113
|
+
const prior = history.get(id);
|
|
114
|
+
if (prior && reusable(prior, raw, treePkg)) {
|
|
115
|
+
startedMutants += 1;
|
|
116
|
+
const reused = reusedResult(prior, commit, run.id, options.agent);
|
|
117
|
+
writeResult(options.outDir, reused);
|
|
118
|
+
progress(`reused ${id}`);
|
|
119
|
+
continue;
|
|
120
|
+
}
|
|
109
121
|
startedMutants += 1;
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
122
|
+
progress(`${id} start`);
|
|
123
|
+
const plan = batchPlan(raw, prep, options);
|
|
124
|
+
if (plan) {
|
|
125
|
+
if (!batch || batch.dead) {
|
|
126
|
+
dropBatch();
|
|
127
|
+
try {
|
|
128
|
+
batch = await NodeBatch.start(treePkg, options, options.outDir);
|
|
129
|
+
}
|
|
130
|
+
catch {
|
|
131
|
+
batch = null;
|
|
132
|
+
}
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
const launch = plan && batch
|
|
136
|
+
? (pkg, files, names, timeoutMs) => batch.run(pkg, files, names, timeoutMs, id, options.outDir)
|
|
137
|
+
: undefined;
|
|
138
|
+
const result = await runOne(worktree, treePkg, patch, options, commit, run.id, prep, launch);
|
|
139
|
+
writeResult(options.outDir, result);
|
|
140
|
+
if (batch && (batch.dead || result.error === "suite process crashed"))
|
|
141
|
+
dropBatch();
|
|
142
|
+
progress(`${id} ${result.outcome}`);
|
|
114
143
|
}
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
writeResult(options.outDir, result);
|
|
119
|
-
progress(`${id} ${result.outcome}`);
|
|
144
|
+
}
|
|
145
|
+
finally {
|
|
146
|
+
dropBatch();
|
|
120
147
|
}
|
|
121
148
|
};
|
|
122
149
|
await Promise.all(worktrees.dirs.map((dir) => worker(dir)));
|
|
@@ -217,7 +244,7 @@ async function prepare(pkg, options, progress) {
|
|
|
217
244
|
const affectedMap = options.affected ? affectedTests(pkg, { testsDir: options.testsDir }) : null;
|
|
218
245
|
return { ok: true, tests, affectedMap, baselineMs, spec, coverage: readCoverageMap(mapPath) };
|
|
219
246
|
}
|
|
220
|
-
async function runOne(worktree, pkg, patchPath, options, commit, runId, prep) {
|
|
247
|
+
async function runOne(worktree, pkg, patchPath, options, commit, runId, prep, launch) {
|
|
221
248
|
const id = path.basename(patchPath, ".patch");
|
|
222
249
|
const started = Date.now();
|
|
223
250
|
const raw = readFileSync(patchPath, "utf8");
|
|
@@ -277,9 +304,15 @@ async function runOne(worktree, pkg, patchPath, options, commit, runId, prep) {
|
|
|
277
304
|
// node --test loads every file it is given, so a name pattern still pays for the other files.
|
|
278
305
|
const selected = narrowNodeFiles(pkg, selectTests(files, prep.tests, prep.affectedMap), names, prep.spec.kind);
|
|
279
306
|
const commands = [];
|
|
280
|
-
const
|
|
307
|
+
const named = names && names.length > 0 ? names : null;
|
|
308
|
+
const main = launch
|
|
309
|
+
? await launch(pkg, selected, named, limit)
|
|
310
|
+
: await runSuite(pkg, selected, options, null, named, prep.spec, limit, forceWhole, null, null);
|
|
281
311
|
if (main.command)
|
|
282
312
|
commands.push(main.command);
|
|
313
|
+
if (main.crashed) {
|
|
314
|
+
return finishMutant({ ...base(), direction, outcome: "error", error: "suite process crashed" }, pkg, commands.join(" | "), false, main.selectedTests);
|
|
315
|
+
}
|
|
283
316
|
if (main.timedOut) {
|
|
284
317
|
return finishMutant({ ...base(), direction, outcome: "timeout", error: "suite timed out", next: TIMEOUT_NEXT }, pkg, commands.join(" | "), main.wholeSuite || forceWhole, main.selectedTests);
|
|
285
318
|
}
|
|
@@ -404,6 +437,164 @@ async function runSuite(pkg, files, options, pattern, names, spec, timeoutMs, fo
|
|
|
404
437
|
selectedTests: result.selectedTests,
|
|
405
438
|
};
|
|
406
439
|
}
|
|
440
|
+
function batchPlan(raw, prep, options) {
|
|
441
|
+
if (options.workers !== 1 || prep.spec.kind !== "node")
|
|
442
|
+
return null;
|
|
443
|
+
if (!prep.coverage || Object.keys(prep.coverage.files).length === 0)
|
|
444
|
+
return null;
|
|
445
|
+
const edited = changedLines(raw);
|
|
446
|
+
const files = filesInDiff(raw);
|
|
447
|
+
if (new Set(files.map((item) => item.file)).size !== 1)
|
|
448
|
+
return null;
|
|
449
|
+
const picked = selectionFor(prep.coverage, edited.length > 0 ? edited : files);
|
|
450
|
+
if (picked.state !== "covered")
|
|
451
|
+
return null;
|
|
452
|
+
const names = picked.tests.filter((name) => name !== WHOLE_PROGRAM_TEST && name.length > 0);
|
|
453
|
+
if (names.length === 0)
|
|
454
|
+
return null;
|
|
455
|
+
return { names };
|
|
456
|
+
}
|
|
457
|
+
/** Covered node mutants of one file share this process. A crash sets dead and the next mutant starts another. */
|
|
458
|
+
class NodeBatch {
|
|
459
|
+
pid;
|
|
460
|
+
dead = false;
|
|
461
|
+
testTimeoutMs;
|
|
462
|
+
proc;
|
|
463
|
+
constructor(proc, pid, testTimeoutMs) {
|
|
464
|
+
this.proc = proc;
|
|
465
|
+
this.pid = pid;
|
|
466
|
+
this.testTimeoutMs = testTimeoutMs;
|
|
467
|
+
proc.on("close", () => {
|
|
468
|
+
this.dead = true;
|
|
469
|
+
});
|
|
470
|
+
}
|
|
471
|
+
static async start(pkg, options, outDir) {
|
|
472
|
+
const dir = path.join(outDir, "batch");
|
|
473
|
+
mkdirSync(dir, { recursive: true });
|
|
474
|
+
const ready = path.join(dir, `ready-${process.pid}-${Date.now()}.json`);
|
|
475
|
+
const gen = path.join(dir, `gen-${process.pid}-${Date.now()}.txt`);
|
|
476
|
+
writeFileSync(gen, "0\n");
|
|
477
|
+
const script = fileURLToPath(new URL("./node-batch.mjs", import.meta.url));
|
|
478
|
+
const env = childEnv(options.unset);
|
|
479
|
+
env.PB_READY = ready;
|
|
480
|
+
env.PB_GEN = gen;
|
|
481
|
+
env.PB_ROOT = pkg;
|
|
482
|
+
const proc = spawn(process.execPath, [script], { cwd: pkg, env, stdio: ["pipe", "ignore", "pipe"] });
|
|
483
|
+
const started = Date.now();
|
|
484
|
+
while (!existsSync(ready)) {
|
|
485
|
+
if (proc.exitCode !== null)
|
|
486
|
+
throw new Error("batch process exited before ready");
|
|
487
|
+
if (Date.now() - started > 10_000) {
|
|
488
|
+
proc.kill("SIGKILL");
|
|
489
|
+
throw new Error("batch process did not start");
|
|
490
|
+
}
|
|
491
|
+
await new Promise((resolve) => setTimeout(resolve, 20));
|
|
492
|
+
}
|
|
493
|
+
const body = JSON.parse(readFileSync(ready, "utf8"));
|
|
494
|
+
return new NodeBatch(proc, body.pid || proc.pid || 0, options.testTimeoutMs);
|
|
495
|
+
}
|
|
496
|
+
stop() {
|
|
497
|
+
this.dead = true;
|
|
498
|
+
try {
|
|
499
|
+
this.proc.stdin?.end();
|
|
500
|
+
}
|
|
501
|
+
catch {
|
|
502
|
+
// The pipe is already closed.
|
|
503
|
+
}
|
|
504
|
+
try {
|
|
505
|
+
if (this.proc.pid)
|
|
506
|
+
process.kill(this.proc.pid, "SIGTERM");
|
|
507
|
+
}
|
|
508
|
+
catch {
|
|
509
|
+
// The process has already exited.
|
|
510
|
+
}
|
|
511
|
+
}
|
|
512
|
+
async run(pkg, files, names, timeoutMs, id, outDir) {
|
|
513
|
+
const selected = names ?? [];
|
|
514
|
+
const replyFile = path.join(outDir, "batch", `${id}.reply.json`);
|
|
515
|
+
rmSync(replyFile, { force: true });
|
|
516
|
+
appendFileSync(path.join(outDir, "batch-processes.jsonl"), `${JSON.stringify({ id, pid: this.pid })}\n`);
|
|
517
|
+
const job = { id, root: pkg, tests: files, names: selected, timeoutMs, testTimeoutMs: nodeTestTimeoutMs(pkg, files, this.testTimeoutMs, timeoutMs), replyFile };
|
|
518
|
+
try {
|
|
519
|
+
this.proc.stdin?.write(`${JSON.stringify(job)}\n`);
|
|
520
|
+
}
|
|
521
|
+
catch {
|
|
522
|
+
this.dead = true;
|
|
523
|
+
return crashedCall(selected);
|
|
524
|
+
}
|
|
525
|
+
const waited = await waitForReply(replyFile, this.proc, timeoutMs);
|
|
526
|
+
if (waited === "timeout") {
|
|
527
|
+
this.dead = true;
|
|
528
|
+
this.stop();
|
|
529
|
+
return { report: null, timedOut: true, detail: "suite timed out", compileToken: null, command: "", wholeSuite: false, selectedTests: selected };
|
|
530
|
+
}
|
|
531
|
+
if (!waited)
|
|
532
|
+
return crashedCall(selected);
|
|
533
|
+
const failed = waited.failed ?? [];
|
|
534
|
+
const seen = waited.names ?? [];
|
|
535
|
+
if (seen.length === 0 && failed.length === 0) {
|
|
536
|
+
return {
|
|
537
|
+
report: null,
|
|
538
|
+
timedOut: false,
|
|
539
|
+
detail: waited.error || "test reporter returned nothing",
|
|
540
|
+
compileToken: null,
|
|
541
|
+
command: waited.command || "",
|
|
542
|
+
wholeSuite: false,
|
|
543
|
+
selectedTests: selected,
|
|
544
|
+
};
|
|
545
|
+
}
|
|
546
|
+
return {
|
|
547
|
+
report: { tests: Math.max(seen.length, failed.length), pass: Math.max(0, seen.length - failed.length), fail: failed.length, failed, names: seen.length > 0 ? seen : selected },
|
|
548
|
+
timedOut: false,
|
|
549
|
+
detail: "",
|
|
550
|
+
compileToken: null,
|
|
551
|
+
command: waited.command || "",
|
|
552
|
+
wholeSuite: false,
|
|
553
|
+
selectedTests: selected,
|
|
554
|
+
};
|
|
555
|
+
}
|
|
556
|
+
}
|
|
557
|
+
function crashedCall(selected) {
|
|
558
|
+
return { report: null, timedOut: false, detail: "suite process crashed", compileToken: null, command: "", wholeSuite: false, selectedTests: selected, crashed: true };
|
|
559
|
+
}
|
|
560
|
+
function waitForReply(file, proc, timeoutMs) {
|
|
561
|
+
return new Promise((resolve) => {
|
|
562
|
+
let settled = false;
|
|
563
|
+
const finish = (value) => {
|
|
564
|
+
if (settled)
|
|
565
|
+
return;
|
|
566
|
+
settled = true;
|
|
567
|
+
clearTimeout(timer);
|
|
568
|
+
clearInterval(poll);
|
|
569
|
+
proc.removeListener("close", onClose);
|
|
570
|
+
resolve(value);
|
|
571
|
+
};
|
|
572
|
+
const read = () => {
|
|
573
|
+
try {
|
|
574
|
+
return JSON.parse(readFileSync(file, "utf8"));
|
|
575
|
+
}
|
|
576
|
+
catch {
|
|
577
|
+
return null;
|
|
578
|
+
}
|
|
579
|
+
};
|
|
580
|
+
const onClose = () => {
|
|
581
|
+
const reply = existsSync(file) ? read() : null;
|
|
582
|
+
finish(reply);
|
|
583
|
+
};
|
|
584
|
+
const timer = setTimeout(() => finish("timeout"), timeoutMs);
|
|
585
|
+
const poll = setInterval(() => {
|
|
586
|
+
if (!existsSync(file))
|
|
587
|
+
return;
|
|
588
|
+
const reply = read();
|
|
589
|
+
if (reply)
|
|
590
|
+
finish(reply);
|
|
591
|
+
}, 20);
|
|
592
|
+
if (proc.exitCode !== null || proc.signalCode !== null)
|
|
593
|
+
onClose();
|
|
594
|
+
else
|
|
595
|
+
proc.once("close", onClose);
|
|
596
|
+
});
|
|
597
|
+
}
|
|
407
598
|
function isNodeTestFile(file) {
|
|
408
599
|
const base = path.posix.basename(file);
|
|
409
600
|
return /\.(test|spec)\.(ts|tsx|mts|js|mjs|cjs)$/.test(base) || /_test\.(ts|tsx|js|mjs)$/.test(base);
|
|
@@ -30,7 +30,7 @@ export function timeoutLimitMs(baselineMs, multiple, floorMs) {
|
|
|
30
30
|
return Math.max(floor, scaled);
|
|
31
31
|
}
|
|
32
32
|
/** Kinds that collect a per-line map on the baseline. Other kinds pay the whole suite. */
|
|
33
|
-
const LINE_MAP_KINDS = new Set(["node", "pytest", "c", "go", "maven", "cargo", "dotnet"]);
|
|
33
|
+
const LINE_MAP_KINDS = new Set(["node", "pytest", "c", "go", "maven", "cargo", "dotnet", "mocha"]);
|
|
34
34
|
/** A baseline this long, with no line map, is too expensive to repeat for a large batch. */
|
|
35
35
|
export const SLOW_BASELINE_MS = 5_000;
|
|
36
36
|
/** Pending mutants above this count, on a slow unmapped suite, are not started. */
|
package/dist/mutate/suites.js
CHANGED
|
@@ -400,8 +400,11 @@ async function launch(pkg, spec, files, pattern, names, timeoutMs, env, hooks) {
|
|
|
400
400
|
}
|
|
401
401
|
if (spec.kind === "dotnet-exe")
|
|
402
402
|
return tag(runDotnetExe(pkg, spec, files, timeoutMs, env), spec.command, wantsNames, selection);
|
|
403
|
-
if (spec.kind === "mocha")
|
|
404
|
-
|
|
403
|
+
if (spec.kind === "mocha") {
|
|
404
|
+
const grep = selection.length > 0 ? selection.map(escapeRegExp).join("|") : "";
|
|
405
|
+
const command = grep ? `${spec.command} --grep ${JSON.stringify(grep)}` : spec.command;
|
|
406
|
+
return tag(runMocha(pkg, files, timeoutMs, env, selection), command, false, selection);
|
|
407
|
+
}
|
|
405
408
|
// A tty is only for a suite that runs `docker run -it` (tini). Everything else is bash.
|
|
406
409
|
if (spec.kind === "script") {
|
|
407
410
|
const launched = scriptInvocation(pkg);
|
|
@@ -481,18 +484,46 @@ function runShell(pkg, command, timeoutMs, env) {
|
|
|
481
484
|
const child = /\b(java|javac|mvn)\b/.test(command) ? javaEnv(env) : env;
|
|
482
485
|
return spawnCollected("sh", ["-c", command], pkg, { ...child, PYTHONDONTWRITEBYTECODE: "1" }, timeoutMs);
|
|
483
486
|
}
|
|
487
|
+
/**
|
|
488
|
+
* Node's `--test-timeout` caps the file, not only a test that left the option unset.
|
|
489
|
+
* A test that sets a longer `timeout` is cancelled, and the failure name is the file path.
|
|
490
|
+
* The flag stays at least that long, and it stays inside the suite cap.
|
|
491
|
+
*/
|
|
492
|
+
export function nodeTestTimeoutMs(pkg, files, testTimeoutMs, suiteTimeoutMs) {
|
|
493
|
+
let declared = 0;
|
|
494
|
+
for (const rel of files) {
|
|
495
|
+
let text = "";
|
|
496
|
+
try {
|
|
497
|
+
text = readFileSync(path.join(pkg, rel), "utf8");
|
|
498
|
+
}
|
|
499
|
+
catch {
|
|
500
|
+
continue;
|
|
501
|
+
}
|
|
502
|
+
for (const match of text.matchAll(/timeout\s*:\s*(\d[\d_]*)/g)) {
|
|
503
|
+
const value = Number(match[1].replaceAll("_", ""));
|
|
504
|
+
if (Number.isFinite(value) && value > declared)
|
|
505
|
+
declared = value;
|
|
506
|
+
}
|
|
507
|
+
}
|
|
508
|
+
const floor = Number.isFinite(testTimeoutMs) && testTimeoutMs > 0 ? testTimeoutMs : 0;
|
|
509
|
+
let chosen = Math.max(floor, declared);
|
|
510
|
+
if (Number.isFinite(suiteTimeoutMs) && suiteTimeoutMs > 0 && chosen > suiteTimeoutMs)
|
|
511
|
+
chosen = suiteTimeoutMs;
|
|
512
|
+
return chosen > 0 ? chosen : floor;
|
|
513
|
+
}
|
|
484
514
|
async function runNode(pkg, files, pattern, timeoutMs, env, hooks) {
|
|
485
515
|
const tsx = path.join(pkg, "node_modules", ".bin", "tsx");
|
|
486
516
|
const bin = existsSync(tsx) ? tsx : process.execPath;
|
|
487
517
|
const prefix = existsSync(tsx) ? [] : ["--experimental-strip-types"];
|
|
488
518
|
const hook = env.PROBATIO_COVERAGE_MAP ? fileURLToPath(new URL("./node-coverage.mjs", import.meta.url)) : null;
|
|
519
|
+
const fileTimeout = nodeTestTimeoutMs(pkg, files, hooks.testTimeoutMs, timeoutMs);
|
|
489
520
|
const result = await spawnCollected(bin, [
|
|
490
521
|
...prefix,
|
|
491
522
|
...(hook ? ["--import", hook] : []),
|
|
492
523
|
"--test",
|
|
493
524
|
`--test-reporter=${hooks.reporterPath}`,
|
|
494
525
|
`--test-concurrency=${hooks.concurrency}`,
|
|
495
|
-
`--test-timeout=${
|
|
526
|
+
`--test-timeout=${fileTimeout}`,
|
|
496
527
|
...(pattern ? [`--test-name-pattern=${pattern}`] : []),
|
|
497
528
|
...files,
|
|
498
529
|
], pkg, env, timeoutMs);
|
|
@@ -545,7 +576,7 @@ function runPython(pkg, files, pattern, timeoutMs, env, script, nodeIds) {
|
|
|
545
576
|
const targets = nodeIds && nodeIds.length > 0 ? nodeIds : files;
|
|
546
577
|
return spawnCollected("python3", [reporter, ...targets, ...(pattern ? ["--pattern", pattern] : [])], pkg, next, timeoutMs);
|
|
547
578
|
}
|
|
548
|
-
async function runMocha(pkg, files, timeoutMs, env) {
|
|
579
|
+
async function runMocha(pkg, files, timeoutMs, env, names = null) {
|
|
549
580
|
const mocha = ["mocha.js", "mocha"]
|
|
550
581
|
.map((name) => path.join(pkg, "node_modules", "mocha", "bin", name))
|
|
551
582
|
.find((file) => existsSync(file));
|
|
@@ -556,6 +587,10 @@ async function runMocha(pkg, files, timeoutMs, env) {
|
|
|
556
587
|
return prepared;
|
|
557
588
|
const config = readMochaConfig(pkg);
|
|
558
589
|
const args = [mocha, ...buildMochaArgs(pkg, files)];
|
|
590
|
+
if (names && names.length > 0)
|
|
591
|
+
args.push("--grep", names.map(escapeRegExp).join("|"));
|
|
592
|
+
if (env.PROBATIO_COVERAGE_MAP)
|
|
593
|
+
args.push("--require", fileURLToPath(new URL("./mocha-coverage.cjs", import.meta.url)));
|
|
559
594
|
if (config && config.require.includes("esm")) {
|
|
560
595
|
// The `esm` loader throws on current Node before any test runs. Drop that one require and keep the rest of the project's mocha config.
|
|
561
596
|
const requires = config.require.filter((name) => name !== "esm");
|
package/dist/verify/change.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { mkdirSync, readFileSync, realpathSync, writeFileSync } from "node:fs";
|
|
1
|
+
import { existsSync, mkdirSync, readdirSync, readFileSync, realpathSync, statSync, writeFileSync } from "node:fs";
|
|
2
2
|
import path from "node:path";
|
|
3
3
|
import { SCHEMA_VERSION } from "../contract.js";
|
|
4
4
|
import { asTable } from "../golden/check.js";
|
|
@@ -26,9 +26,10 @@ export async function verifyChange(options) {
|
|
|
26
26
|
const goldenContract = goldenDiffs(repo, options.base, options.commit, changed);
|
|
27
27
|
const packageFiles = [];
|
|
28
28
|
const chosen = [];
|
|
29
|
+
const libImported = testsImportLib(packageDir);
|
|
29
30
|
for (const [file, lines] of ranges) {
|
|
30
31
|
const rel = toPackage(prefix, file);
|
|
31
|
-
if (!rel || !rel
|
|
32
|
+
if (!rel || !isVerifySource(rel, packageDir, libImported))
|
|
32
33
|
continue;
|
|
33
34
|
packageFiles.push(rel);
|
|
34
35
|
const text = showFile(repo, options.commit, file);
|
|
@@ -269,3 +270,66 @@ function toPackage(prefix, file) {
|
|
|
269
270
|
return null;
|
|
270
271
|
return file.slice(prefix.length + 1);
|
|
271
272
|
}
|
|
273
|
+
const SKIPPED_SOURCE = new Set(["node_modules", "dist", "docs", "tests", "test"]);
|
|
274
|
+
/** src/, a lib/ tree the tests import, or a Go file beside go.mod. Tests, docs, dist, and node_modules stay out. */
|
|
275
|
+
function isVerifySource(rel, packageDir, libImported) {
|
|
276
|
+
if (rel.endsWith(".d.ts"))
|
|
277
|
+
return false;
|
|
278
|
+
const parts = rel.split("/");
|
|
279
|
+
if (parts.some((part) => SKIPPED_SOURCE.has(part)))
|
|
280
|
+
return false;
|
|
281
|
+
if (rel.startsWith("src/"))
|
|
282
|
+
return true;
|
|
283
|
+
if (libImported && rel.startsWith("lib/"))
|
|
284
|
+
return true;
|
|
285
|
+
if (!rel.includes("/") && rel.endsWith(".go") && !rel.endsWith("_test.go") && existsSync(path.join(packageDir, "go.mod")))
|
|
286
|
+
return true;
|
|
287
|
+
return false;
|
|
288
|
+
}
|
|
289
|
+
function testsImportLib(packageDir) {
|
|
290
|
+
const pattern = /(?:require\s*\(\s*|from\s+)['"](?:@[^'"]+\/)?(?:\.\.\/|\.\/)*lib\//;
|
|
291
|
+
for (const dirName of ["tests", "test"]) {
|
|
292
|
+
const dir = path.join(packageDir, dirName);
|
|
293
|
+
if (!existsSync(dir))
|
|
294
|
+
continue;
|
|
295
|
+
if (walkText(dir).some((text) => pattern.test(text)))
|
|
296
|
+
return true;
|
|
297
|
+
}
|
|
298
|
+
return false;
|
|
299
|
+
}
|
|
300
|
+
function walkText(dir) {
|
|
301
|
+
const out = [];
|
|
302
|
+
const visit = (current) => {
|
|
303
|
+
let entries;
|
|
304
|
+
try {
|
|
305
|
+
entries = readdirSync(current);
|
|
306
|
+
}
|
|
307
|
+
catch {
|
|
308
|
+
return;
|
|
309
|
+
}
|
|
310
|
+
for (const name of entries) {
|
|
311
|
+
if (name === "node_modules" || name === "dist")
|
|
312
|
+
continue;
|
|
313
|
+
const full = path.join(current, name);
|
|
314
|
+
let info;
|
|
315
|
+
try {
|
|
316
|
+
info = statSync(full);
|
|
317
|
+
}
|
|
318
|
+
catch {
|
|
319
|
+
continue;
|
|
320
|
+
}
|
|
321
|
+
if (info.isDirectory())
|
|
322
|
+
visit(full);
|
|
323
|
+
else if (info.isFile() && info.size < 1_000_000) {
|
|
324
|
+
try {
|
|
325
|
+
out.push(readFileSync(full, "utf8"));
|
|
326
|
+
}
|
|
327
|
+
catch {
|
|
328
|
+
// An unreadable test file does not count as an import of lib/.
|
|
329
|
+
}
|
|
330
|
+
}
|
|
331
|
+
}
|
|
332
|
+
};
|
|
333
|
+
visit(dir);
|
|
334
|
+
return out;
|
|
335
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "probatio",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "Mutation runner for AI agents. Scores a test suite by the bugs it catches.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
"url": "https://github.com/IronAdamant/Probatio/issues"
|
|
20
20
|
},
|
|
21
21
|
"scripts": {
|
|
22
|
-
"build": "tsc -p tsconfig.json && mkdir -p dist/mutate && cp src/mutate/fail-reporter.mjs src/mutate/node-coverage.mjs src/mutate/py-reporter.py src/mutate/py-pytest-reporter.py src/mutate/probatio-jacoco-run.java dist/mutate/",
|
|
22
|
+
"build": "tsc -p tsconfig.json && mkdir -p dist/mutate && cp src/mutate/fail-reporter.mjs src/mutate/node-coverage.mjs src/mutate/precise-lines.mjs src/mutate/child-lines.mjs src/mutate/mocha-coverage.cjs src/mutate/node-batch.mjs src/mutate/node-batch-loader.mjs src/mutate/py-reporter.py src/mutate/py-pytest-reporter.py src/mutate/probatio-jacoco-run.java dist/mutate/",
|
|
23
23
|
"test": "tsx --test tests/*.test.ts",
|
|
24
24
|
"probatio": "tsx src/cli.ts"
|
|
25
25
|
},
|