probatio 0.1.2 → 0.2.0

package/README.md

@@ -1,21 +1,76 @@
 # Probatio
 
-Probatio (Latin: a testing, a proof) is a free testing toolkit under the MIT license, built for AI agents
-(Claude, Grok, ChatGPT, others), working alone or as swarms. Every claim it makes is checkable.
-It remembers what was fixed so agents with small context do not redo or undo work, and it shows
-which tests already touch a change.
+[![npm version](https://img.shields.io/npm/v/probatio)](https://www.npmjs.com/package/probatio)
 
-Status (2026-10-06): `mutate run` discovers the project suite and scores it. Node, pytest, C (when the binary was built with LLVM coverage), Go, Maven/Java, Rust, C#, and Mocha collect a line map on the baseline. Mocha's map comes from its own root hooks. Loading the node:test collector under Mocha does not name Mocha tests. A line no test executes is `no coverage`, and the suite is not started. A plain Node script can write the lines it ran, and the parent test that reads that dump stores them under its own name. A child the parent does not read stays `no coverage`, and the suite is not started. A suite with no line map, a baseline of 5 seconds or more, and more than 30 mutants stops before the first mutant. `verify-change` names the tests the line map ran. It scores `src/`, a `lib/` file when the package tests import `lib/`, and a Go file beside `go.mod`. Tests, docs, `dist/`, and `node_modules` stay unseen. `ran` stays the direct-importer list. Covered mutants of one Node file run one after another inside the suite process that is already running. `--workers` stays 1. A Node test that sets its own `timeout` keeps that time when `--test-timeout-ms` is shorter. A mutant that crashes or leaves that process dirty ends it, and the next mutant starts clean. An uncovered line still does not start the suite. `mutate tally` does not delete a test. A kill is an assertion the suite already had. A sealed miss means the suite did not see the bug. The kill count is not a merge gate. On Commons CSV (Apache-2.0, commit `2c83a308`), two sealed runs agreed: with the new test file hidden, `CSVFormatTest.testFormatThrowsNullPointerException` failed on the reverted printer. `deletedTests` stayed 0. That older JUnit platform did not write a line map, so the first pass was the whole suite and confirm reran the older test. Auspex, markupsafe, and a hand-planted fixture are not that result. The package version in this repository is 0.1.2. npm latest is 0.1.0. The BugsInPy search row is still open, and the rows that apply only when a sealed run misses do not apply, because this catch held. The MCP server speaks one JSON object per line and returns the same JSON as the CLI.
+Probatio scores a test suite by the bugs it already catches. A kill is an assertion the suite had. The kill count is not a merge gate.
 
-Read in this order:
+Node 22.6 or newer:
 
-1. `HANDOFF.md` — why, the evidence from Auspex, the design, and the lessons that cost time.
-2. `FOR-GROK.md` — the builder brief: build order with acceptance tests, the agent contract,
-   memory for agents, anti-gaming, swarm protocol, traps.
-3. `seed/` — code copied from the Auspex experiment to start from (Auspex paths inside).
+```bash
+npm install probatio
+```
+
+```bash
+npx probatio mutate generate --package . --out .probatio/generate
+npx probatio mutate run --package . --patches .probatio/generate/mutants --out .probatio/runs
+npx probatio mutate tally --out .probatio/runs
+```
+
+Each command prints one JSON object and exits 0 only when `ok` is true.
+
+```json
+{
+  "schemaVersion": 1,
+  "ok": true,
+  "summary": "0 no coverage, 0 survived, 1 killed, 0 flaky, 0 timed out, 0 errored, of 1 finished.",
+  "next": "No survivor in this batch.",
+  "nextCall": null
+}
+```
+
+Agents changing this repo should read [AGENTS.md](AGENTS.md). The MCP server is `probatio mcp`. Snippets are in the MCP section below. People building Probatio should read [docs/builders.md](docs/builders.md). Small suites live in [examples/](examples/).
+
+## Status
+
+What a result means:
+
+- **killed**: a test the suite already had failed an assertion on the mutant. It is not proof the change is safe, and the count is not a merge gate.
+- **survived**: a test ran the line and did not notice. That is a gap. Add a test that fails on the mutant, then rerun.
+- **no coverage**: no test executed that line, so the suite was not started. It is not a pass and not a gap.
+- **did not build** (`unviable`): the compiler rejected the mutant, or on Node a test file no longer links (a removed export, a missing module, a file that does not parse). No test code ran, so it is not a kill and not a gap. Java, Rust, Go, and C# reject the same change at compile time, so every language gets the same verdict. For a ledger fix it means the reverted commit cannot be a regression check as it stands: write a hand-made mutant (`source=hand`) that keeps the API and puts the old behaviour back.
+- **timed out**: the baseline clock ran out. It is not a kill.
+- A red baseline, or a baseline with no test report, stops before the first mutant.
+
+What runs today:
+
+- Discovery finds the project's own suite: `run_tests.sh`, Cargo, Go, Swift, Maven, dotnet, pytest, unittest, node:test, Mocha, and `make test` for COBOL. An unknown layout asks for `--suite-command`.
+- Node, pytest, Go, Maven/Java, Rust, C#, Mocha, and C with LLVM coverage collect a line map on the baseline. A mutant runs only the tests that hit its line.
+- A Node child process started by a test is mapped under that test, with no import in the child and even when it ends with `process.exit`. A child started with a cleared environment cannot be seen and stays `no coverage`.
+- A slow suite with no line map (baseline 5 seconds or more, more than 30 mutants pending) stops before the first mutant.
+- A project that needs its own environment or test-line format says so in `.probatio.json` (below). Probatio does not patch a project's files or environment to make one repository pass.
+
+Sealed runs. A sealed run hides the test that came with a fix, puts the bug back, and asks whether the rest of the suite notices. Seven subjects are recorded (2026-10-05 and 10-06):
 
-Roles: Grok builds most of it; Claude refines and finishes. Grok keeps `NOTES-FOR-CLAUDE.md` here
-(decisions, guesses, shortcuts with file:line, acceptance output actually run).
+| Subject | Result |
+|---|---|
+| QuixBugs, Python program | no coverage: the planted line was not executed |
+| QuixBugs, Java program | survived |
+| Commons CSV, one bug | survived |
+| tqdm, one bug | survived |
+| First one-command `mutate sealed` run | survived, both runs |
+| Commons CSV `c15a06ee` (CSV-288) | the suite stayed green with the test hidden |
+| Commons CSV `2c83a308` (CSV-271) | killed on 10-06 with no line map; survived on 10-08 with the map; see below |
+
+The CSV-271 kill came from `CSVFormatTest.testFormatThrowsNullPointerException`. The fix commit itself edited that test: it changed the asserted stack-frame class from `CSVFormat` to `java.util.Objects`. With the test as it was before the fix, all 92 `CSVFormatTest` tests pass on the reverted code, and no other test fails because of the bug. So no older test caught it. The honest record is that these suites did not notice a hidden real bug, which is what a sealed run is for. A kill that pins an internal detail, such as which class threw, is not the same as a test that checks behaviour.
+
+Known limits:
+
+- The sealed list (`seal`) is a plain id list in the state dir. An agent that can read that directory can see it. Keep the state dir and every `mutate sealed` label file outside the workspace of the agents being scored.
+- `mutate sealed` checks whether the fix commit edited the killing test (`fixEdited`, `olderTestCatch`). It matches a kill to a file by path, class name, or a test name written in that file. A test renamed by the fix can slip past that match.
+- The Java line map comes from JaCoCo, which does not count a line as executed when a call on that line throws. A test that reaches a line only through an exception is not selected for it, so a mutant there can survive a test that would fail. On Commons CSV-271 the map leaves out `testFormatThrowsNullPointerException` at `CSVPrinter.java:284`, and the sealed run reports survived.
+- The operator set is small: condition swaps, relational swaps, booleans, a dropped `!`. No arithmetic, statement deletion, or constants.
+
+The package version in this repository is in `package.json`. The npm badge above is the published version.
 
 ## mutate
 
@@ -31,11 +86,29 @@ npx probatio mutate tally --out .probatio/runs
 
 `run` discovers the suite: `run_tests.sh`, Cargo, Go, Swift, Maven, dotnet, pytest, unittest, node:test, Mocha, then `make test` when COBOL tests sit under that Makefile. An unknown layout stops and asks for `--suite-command`. It does not compile one file and call that the suite. `make test` that would curl or wget a missing file stops, and nothing is fetched. A worktree that has no `node_modules` uses the main checkout's. Omitting `--build` runs no build. Pass `--build` with a command when the suite needs one first.
 
-Node, pytest, Go, Maven/Java, Rust, C#, and Mocha collect a line map on the baseline. C collects one when LLVM coverage was instrumented. Mocha writes its own `{ files }` record from root hooks, keyed by the Mocha title. The node:test collector is a different file and does not name Mocha tests. The map stores the file, the line, and the test names that hit that line. A later mutant runs only those tests (`go test -run`, Maven `-Dtest`, `cargo test`, `dotnet test --filter`, Mocha `--grep`). A line no test executed is `no coverage`, and the suite is not started. A plain Node script can leave a nameless list of the lines it ran. The parent test that reads that list stores them under its own name. A child the parent does not read stays `no coverage`. Covered mutants of one Node file are applied one after another in the suite process already running, and the line map is reset between them. `--workers` stays 1. A crash or a process left dirty ends that process. The next mutant starts clean. A baseline of 5 seconds or more, with no line map and more than 30 mutants still pending, stops before the first mutant. `next` names that baseline and tells you to narrow `--src` or pass a smaller patch directory. A one-file C, C++, Java, COBOL, or assembly launcher is not a suite discovery returns.
+Discovery reads the package root. An example project inside it, such as a directory with its own `go.mod` or `pytest.ini`, is a different project and is not collected. Go needs a `go.mod` at or above the package.
+
+Node, pytest, Go, Maven/Java, Rust, C#, and Mocha collect a line map on the baseline. C collects one when LLVM coverage was instrumented. Mocha writes its own `{ files }` record from root hooks, keyed by the Mocha title. The node:test collector is a different file and does not name Mocha tests. The map stores the file, the line, and the test names that hit that line. A later mutant runs only those tests (`go test -run`, Maven `-Dtest`, `cargo test`, `dotnet test --filter`, Mocha `--grep`). A line no test executed is `no coverage`, and the suite is not started. A Node child process started by a mapped test loads a small collector through `NODE_OPTIONS`. It writes the lines it ran when it exits, including after `process.exit`, and the parent test stores them under its own name. Each test process has its own dump directory, so parallel test files do not take each other's children. A child started with a cleared environment stays `no coverage`. On Node, a test filter that matches nothing is an error, not a survivor, and only the requested tests can be credited with a kill. A test file that fails before any test runs is one of two things. If it does not link or parse (a removed export, a missing module), no code ran, and the mutant is `unviable` unless some other test failed. If its own top-level code ran the mutated program and threw, that is a kill by that file, and confirm reruns the whole file. Covered mutants of one Node file are applied one after another in the suite process already running, and the line map is reset between them. `--workers` stays 1. A crash or a process left dirty ends that process. The next mutant starts clean. A baseline of 5 seconds or more, with no line map and more than 30 mutants still pending, stops before the first mutant. `next` names that baseline and tells you to narrow `--src` or pass a smaller patch directory. A one-file C, C++, Java, COBOL, or assembly launcher is not a suite discovery returns.
 
-The mutant timeout is the baseline duration times 5, and at least 20 seconds, capped by `--suite-timeout-ms`. A timeout is not a kill. `--budget-ms` stops mutant work after the baseline. The first mutant still runs, except for that no-line-map stop. A kill is the test that failed, or a compiler token when the mutant did not build. Confirm is on by default and reruns the failing names. Pytest names that `-k` cannot express are passed as node ids. A usage error, including pytest exit 4, is not a kill. `verify-change` does not confirm a second time. Add `--affected` to limit the file list to tests that can see the change. The default, once a line map exists, runs the tests on the changed line.
+The mutant timeout is the baseline duration times 5, and at least 20 seconds, capped by `--suite-timeout-ms`. A timeout is not a kill. `--budget-ms` stops mutant work after the baseline. The first mutant still runs, except for that no-line-map stop. A kill is the test that failed. A mutant the compiler rejects is `unviable`: the summary says `N did not build`, `unviable` and `unviableIds` list them, and they are neither kills nor gaps. Confirm is on by default and reruns the failing names. Pytest names that `-k` cannot express are passed as node ids. A usage error, including pytest exit 4, is not a kill. `verify-change` does not confirm a second time. Add `--affected` to limit the file list to tests that can see the change. The default, once a line map exists, runs the tests on the changed line.
 
-`tally` reads that run directory. The summary leads with the no-coverage count, then the survivors. It names tests that killed a mutant and tests that killed nothing. A gap is a survivor. An unseen line is not a gap and not a pass. A timeout is neither a kill nor a gap. `deletedTests` is 0. It does not delete a file.
+`tally` reads that run directory. The summary leads with the no-coverage count, then the survivors. `keep` names each test that killed a mutant, once, as an id `--only-test` can run. `noKillsYet` names tests that saw no mutant die in this batch. That is not a reason to delete them: one batch is not evidence, and the Auspex experiment caught 10 of 18 sealed bugs after pruning by kill evidence. `pruning.advice` stays empty and `deletedTests` is 0. A gap is a survivor. An unseen line is not a gap and not a pass. A timeout is neither a kill nor a gap.
+
+`nextCall` writes to a fresh out dir next to the old one, because the old out dir would skip every finished mutant and repeat the old result. With a gap, it reruns the batch on `HEAD` (`<out>.after-gaps`) once the new test is committed. With no gap, it rescores the keep ids at the same commit with `--only-test` (`<out>.rescore`).
+
+## .probatio.json
+
+A project can say how its suite runs. The file sits at the package root and is read even when it is not committed.
+
+```json
+{
+  "env": { "ARCH_NATIVE": "1" },
+  "testLine": "^(Testing .+|Running .+)$",
+  "passLine": "All done, tests as expected"
+}
+```
+
+`env` is added to the suite's environment, for a flag the project's own CI sets. `testLine` names the tests of a `run_tests.sh` suite: each matching line is a test, and the first capture group is the name when there is one. When the suite exits non-zero, the last test line is the one that failed, unless `passLine` was printed. Without `testLine`, a shell suite that prints TAP (`ok 1 - name`, `not ok 2 - name`) is read as TAP. Otherwise it is one command: exit 0 passes, and a failure is `::command`, not an invented test name. An invalid file stops the run and names the field.
 
 ## ledger
 
@@ -43,7 +116,7 @@ The mutant timeout is the baseline duration times 5, and at least 20 seconds, ca
 npx probatio ledger build --package . --commit HEAD --out .probatio/ledger
 ```
 
-A fix commit has a `Fixes-bug:` trailer, or it changes both `src` and a test. `ledger build` reverts that commit's src diff onto the tree you name. A diff that applies is written as a forward patch (`source=history`): apply it to put the bug back. A diff that does not apply is reported, and left for a hand-made patch in the same directory (`source=hand`, `fix=<commit>`). A rebuild keeps those hand-made files. Nothing is fuzzy-applied. A src diff over `--max-lines` (default 300) is skipped. `--max-commits N` reads only the newest N non-merge commits and says when older history was not scanned. The default reads the whole history. A tree is checked out only when a fix inside that cap has to be applied.
+A fix commit has a `Fixes-bug:` trailer, or it changes both `src` and a test. A commit that changes the version in `package.json`, `Cargo.toml`, `pyproject.toml`, or `setup.cfg` is a release, and it is not a fix unless it has the trailer: reverting it would put back several changes, not one bug. `ledger build` reverts that commit's src diff onto the tree you name. A diff that applies is written as a forward patch (`source=history`): apply it to put the bug back. A diff that does not apply is reported, and left for a hand-made patch in the same directory (`source=hand`, `fix=<commit>`). A rebuild keeps those hand-made files. Nothing is fuzzy-applied. A src diff over `--max-lines` (default 300) is skipped. `--max-commits N` reads only the newest N non-merge commits and says when older history was not scanned. The default reads the whole history. A tree is checked out only when a fix inside that cap has to be applied.
 
 ## matrix
 
@@ -81,4 +154,23 @@ Lists the tests that can see the diff, runs mutants on the changed lines, and re
 
 An item is `.probatio/queue/<id>.json`. `queue claim` renames it to `claimed/<agent>-<id>.json` with a lease. `queue reap` moves an expired lease back. `check-kill <id>` runs the tests that directly import the mutated file and confirms the killing test again. It is done only when that mutant dies and that suite is still green. `verify-change` is the edit-sized run: it uses the line map and does not confirm a second time.
 
-The MCP server speaks stdio JSON-RPC. Its one tool runs the CLI and returns that command's JSON.
+## MCP
+
+`probatio mcp` and the `probatio-mcp` bin speak stdio JSON-RPC, one JSON object per line. The tool name is `probatio`. `argv` is the CLI words. `ping` returns `{}`. An unknown method returns JSON-RPC `-32601` with the same id. A tool call runs in the background, so `ping` and other calls are answered while a long `mutate run` works. `notifications/cancelled` with that call's `requestId` stops the command and its suite, removes its worktrees, and sends no reply. The server answers with the client's `protocolVersion` when it is `2024-11-05`, `2025-03-26`, or `2025-06-18`.
+
+```json
+{
+  "mcpServers": {
+    "probatio": {
+      "command": "probatio",
+      "args": ["mcp"]
+    }
+  }
+}
+```
+
+That block is the Claude, Cursor, and Grok shape. The tool runs the CLI and returns that command's JSON. `nextCall.argv` keeps absolute paths so a later spawn does not have to expand `~`.
+
+`mutate generate` reads the commit (`HEAD` unless you pass `--commit`). `--working-tree` reads the checkout on disk and says so. `mutate run` scores the commit either way.
+
+`verify-change` does not score uncommitted edits when `--base` and `--commit` are the same. A clean empty diff says `No diff-scoped mutant.`

package/dist/cli.js

@@ -10,65 +10,82 @@ import { appendFinding, chooseSealed, fixGap, guardAllows, lineHash, loadState,
 import { git } from "./mutate/patch.js";
 import { generateMutants } from "./mutate/generate.js";
 import { runMutants } from "./mutate/run.js";
-import { LABEL_LEAK_SUMMARY, labelLeaked, readSealedLabel } from "./mutate/sealed.js";
+import { fixEditedKillers, LABEL_LEAK_SUMMARY, labelLeaked, readSealedLabel, sealedTreeLeaked } from "./mutate/sealed.js";
 import { generateNext } from "./mutate/suite-decision.js";
 import { discoverSuite } from "./mutate/suites.js";
 import { tallyRun } from "./mutate/tally.js";
 import { checkKill } from "./swarm/check-kill.js";
 import { claimItem, reapClaims, seedQueue } from "./swarm/queue.js";
 import { verifyChange } from "./verify/change.js";
-const parsed = parseArgs(process.argv.slice(2));
-const [group, action] = parsed.command;
-try {
-    if (group === "mutate" && action === "generate")
-        finish(await generateCommand(parsed.flags), parsed.human);
-    if (group === "mutate" && action === "run")
-        finish(await runCommand(parsed.flags), parsed.human);
-    if (group === "mutate" && action === "sealed")
-        finish(await sealedCommand(parsed.flags), parsed.human);
-    if (group === "mutate" && action === "tally")
-        finish(tallyCommand(parsed.flags), parsed.human);
-    if (group === "ledger" && action === "build")
-        finish(await ledgerCommand(parsed.flags), parsed.human);
-    if (group === "matrix" && action === "report")
-        finish(matrixCommand(parsed.flags), parsed.human);
-    if (group === "golden" && action === "check")
-        finish(goldenCommand(parsed.flags), parsed.human);
-    if (group === "gap" && action === "fix")
-        finish(gapFixCommand(parsed.flags), parsed.human);
-    if (group === "gap" && action === "revert")
-        finish(gapRevertCommand(parsed.flags), parsed.human);
-    if (group === "guard" && action === "check")
-        finish(guardCommand(parsed.flags), parsed.human);
-    if (group === "findings" && action === "add")
-        finish(findingsCommand(parsed.flags), parsed.human);
-    if (group === "status")
-        finish(statusCommand(parsed.flags), parsed.human);
-    if (group === "seal")
-        finish(sealCommand(parsed.flags), parsed.human);
-    if (group === "queue" && action === "seed")
-        finish(queueSeedCommand(parsed.flags), parsed.human);
-    if (group === "queue" && action === "claim")
-        finish(queueClaimCommand(parsed.flags), parsed.human);
-    if (group === "queue" && action === "reap")
-        finish(queueReapCommand(parsed.flags), parsed.human);
-    if (group === "check-kill")
-        finish(await checkKillCommand(parsed.flags, action), parsed.human);
-    if (group === "verify-change")
-        finish(await verifyCommand(parsed.flags), parsed.human);
-    finish(usage(false, "Use mutate, ledger, matrix, golden, gap, guard, findings, status, queue, check-kill, or verify-change."), parsed.human);
-}
-catch (error) {
-    const message = error instanceof Error ? error.message : String(error);
-    finish({
+import { nodeTooOld } from "./node-version.js";
+const tooOld = nodeTooOld(process.versions.node);
+if (tooOld) {
+    process.stdout.write(`${JSON.stringify({
         schemaVersion: SCHEMA_VERSION,
         ok: false,
-        command: `${group ?? "mutate"}.${action ?? "help"}`,
-        summary: message,
-        next: "Fix the command and run it again.",
+        command: "mutate.help",
+        summary: tooOld,
+        next: "Install Node 22.6 or newer and run the command again.",
         nextCall: null,
-    }, parsed.human);
+    })}\n`);
+    process.exit(1);
 }
+const parsed = parseArgs(process.argv.slice(2));
+const [group, action] = parsed.command;
+if (group === "mcp" && !action) {
+    await import("./mcp.js");
+}
+else
+    try {
+        if (group === "mutate" && action === "generate")
+            finish(await generateCommand(parsed.flags), parsed.human);
+        if (group === "mutate" && action === "run")
+            finish(await runCommand(parsed.flags), parsed.human);
+        if (group === "mutate" && action === "sealed")
+            finish(await sealedCommand(parsed.flags), parsed.human);
+        if (group === "mutate" && action === "tally")
+            finish(tallyCommand(parsed.flags), parsed.human);
+        if (group === "ledger" && action === "build")
+            finish(await ledgerCommand(parsed.flags), parsed.human);
+        if (group === "matrix" && action === "report")
+            finish(matrixCommand(parsed.flags), parsed.human);
+        if (group === "golden" && action === "check")
+            finish(goldenCommand(parsed.flags), parsed.human);
+        if (group === "gap" && action === "fix")
+            finish(gapFixCommand(parsed.flags), parsed.human);
+        if (group === "gap" && action === "revert")
+            finish(gapRevertCommand(parsed.flags), parsed.human);
+        if (group === "guard" && action === "check")
+            finish(guardCommand(parsed.flags), parsed.human);
+        if (group === "findings" && action === "add")
+            finish(findingsCommand(parsed.flags), parsed.human);
+        if (group === "status")
+            finish(statusCommand(parsed.flags), parsed.human);
+        if (group === "seal")
+            finish(sealCommand(parsed.flags), parsed.human);
+        if (group === "queue" && action === "seed")
+            finish(queueSeedCommand(parsed.flags), parsed.human);
+        if (group === "queue" && action === "claim")
+            finish(queueClaimCommand(parsed.flags), parsed.human);
+        if (group === "queue" && action === "reap")
+            finish(queueReapCommand(parsed.flags), parsed.human);
+        if (group === "check-kill")
+            finish(await checkKillCommand(parsed.flags, action), parsed.human);
+        if (group === "verify-change")
+            finish(await verifyCommand(parsed.flags), parsed.human);
+        finish(usage(false, "Use mutate, ledger, matrix, golden, gap, guard, findings, status, queue, check-kill, or verify-change."), parsed.human);
+    }
+    catch (error) {
+        const message = error instanceof Error ? error.message : String(error);
+        finish({
+            schemaVersion: SCHEMA_VERSION,
+            ok: false,
+            command: `${group ?? "mutate"}.${action ?? "help"}`,
+            summary: message,
+            next: "Fix the command and run it again.",
+            nextCall: null,
+        }, parsed.human);
+    }
 function usage(ok, summary) {
     return {
         schemaVersion: SCHEMA_VERSION,
@@ -93,7 +110,19 @@ async function generateCommand(flags) {
         maxMutants: int(flags, "max-mutants") ?? null,
         maxMinutes: int(flags, "max-minutes") ?? null,
         skipFiles: texts(flags, "skip-file"),
+        commit: text(flags, "commit") ?? "HEAD",
+        workingTree: bool(flags, "working-tree", false),
     });
+    if (result.error) {
+        return {
+            schemaVersion: SCHEMA_VERSION,
+            ok: false,
+            command: "mutate.generate",
+            summary: result.error,
+            next: "Commit the file, or pass --working-tree to mutate the checkout on disk. mutate run scores the commit.",
+            nextCall: null,
+        };
+    }
     if (result.violations.length > 0) {
         const first = result.violations[0];
         return {
@@ -117,9 +146,14 @@ async function generateCommand(flags) {
         summary: stopped
             ? `Stopped by the time budget after ${result.filesVisited} files. ${result.mutants.length} mutants written.`
             : `${result.mutants.length} mutants in ${result.filesVisited} files. No string-literal mutant.`,
-        next: stopped
-            ? "Narrow --src or raise --max-minutes, then run the same command."
-            : generateNext(result.mutants.length, suite ? { kind: suite.kind, command: suite.command } : null),
+        next: [
+            stopped
+                ? "Narrow --src or raise --max-minutes, then run the same command."
+                : generateNext(result.mutants.length, suite ? { kind: suite.kind, command: suite.command } : null),
+            result.source === "working-tree"
+                ? "These patches match the working tree. mutate run scores the commit, so commit the files before mutate run."
+                : "",
+        ].filter((line) => line.length > 0).join(" "),
         nextCall: result.mutants.length === 0
             ? null
             : { argv: ["mutate", "run", "--package", packageDir, "--patches", patches, "--out", path.join(outDir, "runs")] },
@@ -133,6 +167,19 @@ async function generateCommand(flags) {
     };
 }
 async function runCommand(flags, hide = null) {
+    const hidden = hide ?? texts(flags, "hide");
+    const escaped = hidden.find((rel) => rel.split(/[\\/]/).includes(".."));
+    if (escaped) {
+        return {
+            schemaVersion: SCHEMA_VERSION,
+            ok: false,
+            command: "mutate.run",
+            summary: `--hide stays inside the package. Refused ${escaped}.`,
+            next: "Pass a relative path with no .. segment.",
+            nextCall: null,
+            killed: 0,
+        };
+    }
     const onlyNames = texts(flags, "only-test");
     if (onlyNames.some((id) => id === "::command" || id.endsWith("::command"))) {
         return {
@@ -209,6 +256,8 @@ async function runCommand(flags, hide = null) {
         next: report.next,
         nextCall: report.ok ? resume : null,
         killed: report.killed,
+        unviable: report.unviable,
+        unviableIds: report.unviableIds,
         survived: report.survived,
         flaky: report.flaky,
         timeouts: report.timeouts,
@@ -234,10 +283,28 @@ function tallyCommand(flags) {
 async function sealedCommand(flags) {
     const label = readSealedLabel(path.resolve(requireText(flags, "label")));
     const hide = texts(flags, "hide");
+    const outDir = path.resolve(requireText(flags, "out"));
     const report = await runCommand(flags, hide);
-    const body = JSON.stringify(report);
-    if (!labelLeaked(body, label))
-        return report;
+    const packageDir = path.resolve(requireText(flags, "package"));
+    const repoDir = path.resolve(text(flags, "repo") ?? gitRoot(packageDir));
+    const kills = Array.isArray(report.kills) ? report.kills : [];
+    const commit = typeof report.commit === "string" ? report.commit : "";
+    const edited = report.ok && commit ? fixEditedKillers(repoDir, packageDir, commit, hide, kills) : { files: [], olderTestCatch: false };
+    const fromFix = kills.length > 0 && !edited.olderTestCatch;
+    const judged = {
+        ...report,
+        ...(fromFix
+            ? {
+                summary: `${report.summary} Every kill came from a test edited by the fix commit (${edited.files.join(", ")}). That is the fix's own test, so no older test caught the bug.`,
+                next: "Record this as a miss. Do not restore the hidden test.",
+            }
+            : {}),
+        fixEdited: edited.files,
+        olderTestCatch: edited.olderTestCatch,
+    };
+    const body = JSON.stringify(judged);
+    if (!labelLeaked(body, label) && !sealedTreeLeaked(outDir, label))
+        return judged;
     return {
         schemaVersion: SCHEMA_VERSION,
         ok: false,

package/dist/contract.js

@@ -1,18 +1,20 @@
 import { homedir } from "node:os";
 export const SCHEMA_VERSION = 1;
 /** Replace the home directory so a result never carries a user's name or home path. */
-export function scrub(value, home = homedir()) {
+export function scrub(value, home = homedir(), keepPaths = false) {
     if (typeof value === "string") {
-        if (!home || !value.includes(home))
+        if (keepPaths || !home || !value.includes(home))
             return value;
         return value.split(home).join("~");
     }
     if (Array.isArray(value))
-        return value.map((item) => scrub(item, home));
+        return value.map((item) => scrub(item, home, keepPaths));
     if (value && typeof value === "object") {
         const out = {};
-        for (const [key, item] of Object.entries(value))
-            out[key] = scrub(item, home);
+        for (const [key, item] of Object.entries(value)) {
+            // nextCall.argv is executed. A tilde is not expanded by spawn or by MCP.
+            out[key] = scrub(item, home, keepPaths || key === "nextCall");
+        }
         return out;
     }
     return value;

package/dist/ledger/build.js

@@ -106,6 +106,9 @@ function collect(repo, commit, srcPrefix, testsPrefix, maxCommits) {
             continue;
         if (!hasTrailer && lines === 0)
             continue;
+        // A version bump ships whatever was finished. Reverting it puts back several changes, not one bug.
+        if (!hasTrailer && bumpsVersion(repo, sha))
+            continue;
         found.push({
             commit: sha,
             date: date ?? "",
@@ -116,6 +119,17 @@ function collect(repo, commit, srcPrefix, testsPrefix, maxCommits) {
     }
     return { found, scanned: limited.length, budgetHit };
 }
+const MANIFESTS = ["package.json", "Cargo.toml", "pyproject.toml", "setup.cfg"].map((name) => `:(glob)**/${name}`);
+/** True when the commit changes a project version line in a manifest. */
+function bumpsVersion(repo, sha) {
+    const diff = git(repo, ["show", "--format=", "-U0", sha, "--", ...MANIFESTS]);
+    if (diff.status !== 0)
+        return false;
+    return diff.stdout
+        .split("\n")
+        .filter((line) => /^[-+]/.test(line) && !line.startsWith("+++") && !line.startsWith("---"))
+        .some((line) => /"version"\s*:/.test(line) || /^[-+]\s*version\s*=/.test(line));
+}
 function entry(candidate, id, status, files, reason) {
     return {
         id,

package/dist/mcp.js

@@ -1,8 +1,23 @@
 #!/usr/bin/env node
-import { spawnSync } from "node:child_process";
+import { spawn } from "node:child_process";
 import { existsSync, readFileSync } from "node:fs";
 import path from "node:path";
 import { fileURLToPath } from "node:url";
+import { nodeTooOld } from "./node-version.js";
+const tooOld = nodeTooOld(process.versions.node);
+if (tooOld) {
+    process.stdout.write(`${JSON.stringify({
+        schemaVersion: 1,
+        ok: false,
+        command: "mcp",
+        summary: tooOld,
+        next: "Install Node 22.6 or newer and run the command again.",
+        nextCall: null,
+    })}\n`);
+    process.exit(1);
+}
+/** Protocol versions this server speaks. The client's choice wins when it is one of these. */
+const PROTOCOLS = ["2024-11-05", "2025-03-26", "2025-06-18"];
 const here = path.dirname(fileURLToPath(import.meta.url));
 const version = readVersion();
 const tool = {
@@ -15,6 +30,9 @@ const tool = {
     },
 };
 let buffer = "";
+let closing = false;
+/** Tool calls still running, by request id. A cancel kills the child and drops the reply. */
+const running = new Map();
 process.stdin.setEncoding("utf8");
 process.stdin.on("data", (chunk) => {
     buffer += chunk;
@@ -22,7 +40,10 @@ process.stdin.on("data", (chunk) => {
 });
 process.stdin.on("end", () => {
     drain(true);
-    process.exit(0);
+    closing = true;
+    // A call already running still gets its reply before the server exits.
+    if (running.size === 0)
+        process.exit(0);
 });
 function drain(flush = false) {
     const lines = buffer.split("\n");
@@ -44,6 +65,22 @@ function drain(flush = false) {
 function handle(message) {
     if (!message.method)
         return;
+    if (message.method === "notifications/cancelled") {
+        const key = message.params?.requestId;
+        if (key === undefined)
+            return;
+        const child = running.get(String(key));
+        running.delete(String(key));
+        if (child?.pid) {
+            try {
+                process.kill(-child.pid, "SIGTERM");
+            }
+            catch {
+                child.kill("SIGTERM");
+            }
+        }
+        return;
+    }
     if (message.id === undefined || message.id === null)
         return;
     if (message.method === "initialize") {
@@ -51,7 +88,7 @@ function handle(message) {
             jsonrpc: "2.0",
             id: message.id,
             result: {
-                protocolVersion: "2024-11-05",
+                protocolVersion: PROTOCOLS.includes(message.params?.protocolVersion ?? "") ? message.params?.protocolVersion : PROTOCOLS[0],
                 capabilities: { tools: {} },
                 serverInfo: { name: "probatio", version },
                 tools: [tool],
@@ -63,36 +100,71 @@ function handle(message) {
         send({ jsonrpc: "2.0", id: message.id, result: { tools: [tool] } });
         return;
     }
+    if (message.method === "ping") {
+        send({ jsonrpc: "2.0", id: message.id, result: {} });
+        return;
+    }
     if (message.method === "tools/call") {
         const argv = message.params?.arguments?.argv;
         if (!Array.isArray(argv) || argv.some((item) => typeof item !== "string")) {
             send({ jsonrpc: "2.0", id: message.id, error: { code: -32602, message: "argv must be an array of strings" } });
             return;
         }
-        const cli = cliCommand();
-        const child = spawnSync(cli.bin, [...cli.args, ...argv], { encoding: "utf8" });
-        const text = child.stdout ?? "";
-        let envelope = null;
-        try {
-            envelope = JSON.parse(text);
-        }
-        catch {
-            envelope = null;
-        }
-        if (!envelope || typeof envelope !== "object") {
-            send({ jsonrpc: "2.0", id: message.id, error: { code: -32000, message: "the command did not return one JSON object" } });
-            return;
-        }
-        send({
-            jsonrpc: "2.0",
-            id: message.id,
-            result: {
-                content: [{ type: "text", text: text.trim() }],
-                structuredContent: envelope,
-                isError: envelope.ok !== true,
-            },
-        });
+        call(message.id, argv);
+        return;
+    }
+    send({
+        jsonrpc: "2.0",
+        id: message.id,
+        error: { code: -32601, message: `method not found: ${message.method}` },
+    });
+}
+/** One CLI command per call. It runs beside other calls, so ping and cancel are answered meanwhile. */
+function call(id, argv) {
+    const key = String(id);
+    const cli = cliCommand();
+    // Its own process group, so a cancel also stops the suite the command started.
+    const child = spawn(cli.bin, [...cli.args, ...argv], { stdio: ["ignore", "pipe", "ignore"], detached: true });
+    running.set(key, child);
+    let text = "";
+    child.stdout?.setEncoding("utf8");
+    child.stdout?.on("data", (chunk) => {
+        text += chunk;
+    });
+    const settle = (failure) => {
+        // A cancelled call is no longer in the map. Its reply is dropped.
+        const live = running.get(key) === child;
+        running.delete(key);
+        if (live)
+            reply(id, text, failure);
+        if (closing && running.size === 0)
+            process.exit(0);
+    };
+    child.on("error", (error) => settle(error.message));
+    child.on("close", () => settle(null));
+}
+function reply(id, text, failure) {
+    let envelope = null;
+    try {
+        envelope = JSON.parse(text);
+    }
+    catch {
+        envelope = null;
+    }
+    if (!envelope || typeof envelope !== "object") {
+        const message = failure ? `the command did not start: ${failure}` : "the command did not return one JSON object";
+        send({ jsonrpc: "2.0", id, error: { code: -32000, message } });
+        return;
     }
+    send({
+        jsonrpc: "2.0",
+        id,
+        result: {
+            content: [{ type: "text", text: text.trim() }],
+            structuredContent: envelope,
+            isError: envelope.ok !== true,
+        },
+    });
 }
 function cliCommand() {
     const compiled = path.join(here, "cli.js");