supercov 4.11.1 → 4.12.1

package/docs/cli.md

@@ -349,6 +349,19 @@ same run:
 
 ## Export for other tools
 
+Every obligation a run left uncovered, one JSON object per line:
+
+```sh supercov-example
+npx supercov runs latest gaps --format jsonl --metric mcdc --output gaps.jsonl
+npx supercov runs latest gaps --format jsonl > gaps.jsonl
+```
+
+Each line holds the file, the line, the source of that line and what `runs
+<id> file` says of the obligation: for an MC/DC condition its decision, the
+condition and the vectors observed. `--metric`, `--filter`, `--kind` and
+`--runner` narrow it as they narrow `gaps`, and nothing is paged, so a tool or
+an agent reads a run's every gap in one pass instead of a page at a time.
+
 ```sh supercov-example
 npx supercov runs report --format lcov --output coverage/lcov.info
 npx supercov runs report --format cobertura --output coverage/cobertura.xml
@@ -468,6 +481,20 @@ The same filters can focus a comparison:
 npx supercov diff <older-run> <newer-run> --kind e2e
 ```
 
+To see what a new test covered that the suite did not, measure it alone and
+compare it with the suite's run, keeping only what it gained:
+
+```sh supercov-example
+npx supercov -- <the new test alone>
+npx supercov diff <suite-run> latest --gained
+```
+
+A test run alone covers far less than the suite, so a plain `diff` of the two
+lists nearly everything as lost. `--gained` lists only the lines, branches and
+MC/DC conditions the newer run covers and the older does not, which is how to
+check that a test reached the conditions it was written for without measuring
+the whole suite. In `--json`, `gainedOnly` is `true` and `lost` is empty.
+
 ## Generate a portable HTML report
 
 ```sh
@@ -533,6 +560,16 @@ A source file
 added since the run is outside every test's record; the working-tree check
 says so, and the suite should run in full.
 
+A server the suite starts is recorded too: a mock server `webServer` starts,
+or the application's own. Each Node process of a Playwright run notes the
+project files it loads and reads, and the tests it serves, by the test a
+request names or, for one that names none (a webhook), the test running
+then. A change to a file such a process loaded or read, a mock's code, a
+helper it imports or the data it reads, affects the tests it served and no
+others. A mock keeps its state between tests, so a mock's files count whole,
+not route by route; a file the application's server loaded, a preload of its
+own, affects every test it served.
+
 ### Which affected tests check the change
 
 When the run has an [assertion assessment](assertions.md), `tests affected`
@@ -655,6 +692,19 @@ A run left out is named with the files and the functions that changed. The
 merged run lists what differed and why it does not reach the earlier runs:
 `merged across:` in `runs <id>`, and `mergedAcross` in JSON.
 
+A changed file that a server the run's tests reached loaded or read (see
+[Find the tests a change affects](#find-the-tests-a-change-affects)) takes
+only those tests from the run: measure them again, and the merge keeps the
+rest of the run and leaves their old records out:
+
+```sh supercov-example
+npx supercov runs <suite-run> tests affected --names   # after changing a mock
+npx supercov -- <those tests alone>
+npx supercov merge
+```
+
+Until they are measured again, the merge refuses and names them.
+
 When you know a change Supercov counts does not alter what the earlier tests
 ran, say so:
 
@@ -666,6 +716,22 @@ The product code, the dependencies, and what is measured must still be the
 same, and a run whose own test files changed is still left out. What was
 accepted is listed in `merged across:` as accepted.
 
+Runs measured on different machines merge the same way. Each machine
+measures its part of the suite on the same commit and toolchain, and one job
+merges them after copying each run's directory from `.supercov/runs/` into its
+own:
+
+```yaml
+# Each of two jobs:
+- run: npx supercov -- <test command for shard ${{ matrix.shard }}>
+- uses: actions/upload-artifact@v4
+  with: { name: "supercov-${{ matrix.shard }}", path: .supercov/runs, include-hidden-files: true }
+# Then one job:
+- uses: actions/download-artifact@v4
+  with: { pattern: "supercov-*", path: .supercov/runs, merge-multiple: true }
+- run: npx supercov merge
+```
+
 `assertions assess` assesses a merged run across every suite merged into it,
 reusing the answers saved when its parts were assessed. `tests affected` of a
 merged run is answered from the runs it was merged from, which hold the record

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "supercov",
-  "version": "4.11.1",
+  "version": "4.12.1",
   "description": "Coverage, security and code quality for coding agents",
   "keywords": [
     "coverage",
@@ -83,7 +83,7 @@
     "eval:skill": "claude plugin eval plugins/supercov --scaffold --trust-plugin --no-publish --model claude-sonnet-5 --judge-model claude-haiku-4-5 --threshold 0.66 --tag behavior trigger --allow-tools Bash Edit Write --max-cost-usd 15",
     "eval:skill-triggers": "claude plugin eval plugins/supercov --scaffold --trust-plugin --no-publish --model claude-sonnet-5 --judge-model claude-haiku-4-5 --threshold 0.66 --tag trigger --ablation none --max-cost-usd 5",
     "test:engine": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace && npm run test:runtime && npm run test:rust-assets && npm run test:engine-integrations",
-    "test:engine-integrations": "cargo build -p supercov && node scripts/html-report-integration.mjs && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/mocha-failure-output.mjs && node scripts/failure-output.mjs node ava tap && node scripts/side-by-side.mjs node && node scripts/merge-across.mjs && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/cross-origin-requests.mjs && node scripts/leaving-page.mjs && node scripts/background-by-time.mjs && node scripts/python-server-playwright.mjs && node scripts/ruby-server-playwright.mjs && node scripts/go-server-playwright.mjs && node scripts/jvm-server-playwright.mjs && node scripts/docker-server-playwright.mjs && node scripts/rust-classic-browser-integration.mjs && node scripts/rust-top-packages-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
+    "test:engine-integrations": "cargo build -p supercov && node scripts/html-report-integration.mjs && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/mocha-failure-output.mjs && node scripts/failure-output.mjs node ava tap && node scripts/side-by-side.mjs node && node scripts/merge-across.mjs && node scripts/mock-reach.mjs && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/cross-origin-requests.mjs && node scripts/leaving-page.mjs && node scripts/shared-page.mjs && node scripts/background-by-time.mjs && node scripts/python-server-playwright.mjs && node scripts/ruby-server-playwright.mjs && node scripts/go-server-playwright.mjs && node scripts/jvm-server-playwright.mjs && node scripts/docker-server-playwright.mjs && node scripts/rust-classic-browser-integration.mjs && node scripts/rust-top-packages-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
     "test:platform": "cargo test --workspace && cargo build -p supercov && node scripts/rust-process-supervision.mjs && node scripts/workspace-crash-integration.mjs",
     "test:native-package": "cargo build --release -p supercov && node scripts/native-package-integration.mjs && node scripts/native-release-set-integration.mjs",
     "test:pypi-wheel": "node scripts/pypi-wheel-integration.mjs",
@@ -120,14 +120,14 @@
     "test:windows-tls": "node scripts/windows-tls-test.mjs"
   },
   "optionalDependencies": {
-    "@supercov/cli-darwin-arm64": "4.11.1",
-    "@supercov/cli-darwin-x64": "4.11.1",
-    "@supercov/cli-linux-arm64-gnu": "4.11.1",
-    "@supercov/cli-linux-arm64-musl": "4.11.1",
-    "@supercov/cli-linux-x64-gnu": "4.11.1",
-    "@supercov/cli-linux-x64-musl": "4.11.1",
-    "@supercov/cli-win32-arm64": "4.11.1",
-    "@supercov/cli-win32-x64": "4.11.1"
+    "@supercov/cli-darwin-arm64": "4.12.1",
+    "@supercov/cli-darwin-x64": "4.12.1",
+    "@supercov/cli-linux-arm64-gnu": "4.12.1",
+    "@supercov/cli-linux-arm64-musl": "4.12.1",
+    "@supercov/cli-linux-x64-gnu": "4.12.1",
+    "@supercov/cli-linux-x64-musl": "4.12.1",
+    "@supercov/cli-win32-arm64": "4.12.1",
+    "@supercov/cli-win32-x64": "4.12.1"
   },
   "peerDependencies": {
     "@playwright/test": ">=1.55.0",

package/runtime/javascript/playwright.mjs

@@ -391,6 +391,42 @@ class CoveragePhaseController {
             }
         }
     }
+    /**
+     * Start the pages an earlier test left open (a page a serial describe
+     * opens in `beforeAll`, a worker-scoped one) with no evidence, and return
+     * what they held. Their documents kept every hit since they loaded, and
+     * each later test was credited with what the tests before it ran. What
+     * they held is the earlier tests' (already recorded) and work between
+     * tests, such as a `beforeAll` signing in: no test's. A document that
+     * already has one of this test's phases is left alone: a reset that
+     * arrives late must not drop what this test ran.
+     */
+    async takeCarriedEvidence() {
+        await Promise.all([...this.pendingRegistrations]);
+        const prefix = `${this.scope.attemptId}:`;
+        const take = (target) => bounded(target
+            .evaluate((prefix) => {
+            const coverageGlobal = globalThis;
+            if (String(coverageGlobal.__SUPERCOV_PHASE_ID__ ?? "").startsWith(prefix))
+                return undefined;
+            const getSnapshot = coverageGlobal.__SUPERCOV_COVERAGE_SNAPSHOT__;
+            const reset = coverageGlobal.__SUPERCOV_RESET__;
+            if (!getSnapshot || !reset)
+                return undefined;
+            const snapshot = JSON.stringify(getSnapshot());
+            reset();
+            return snapshot;
+        }, prefix)
+            .then((text) => (typeof text === "string" ? JSON.parse(text) : undefined))
+            .catch(() => undefined), WORKER_ACTIVATION_WAIT_MS);
+        const taken = await Promise.all(this.allPages().flatMap((page) => [
+            ...page.frames().map(take),
+            ...page.workers().map(take),
+        ]));
+        const snapshots = taken.filter((snapshot) => snapshot && snapshot !== WORKER_TIMED_OUT &&
+            (snapshot.hits?.length || snapshot.events?.length || snapshot.decisions?.length));
+        return snapshots.length > 0 ? mergeSnapshots(snapshots) : undefined;
+    }
     /**
      * A page is registered once, and whoever asks while that is under way
      * waits for it. Playwright announces a page to the context's listener
@@ -1458,6 +1494,25 @@ function mergeCollectedPhases(controllerPhases, fallbackPhases) {
     });
     return [...controllerPhases, ...remaining];
 }
+/**
+ * Pool runners may cycle-restore a VM immediately after Playwright exits,
+ * which can discard or overwrite the normal artifact copy. Each record is
+ * written to one uniquely named, one-shot evidence file in the runner's shared
+ * directory as well. No test streams into this path.
+ */
+function appendEvidenceLine(serialized) {
+    const evidenceDirectory = process.env["SUPERCOV_EVIDENCE_DIR"] ??
+        (GENERATED_EVIDENCE_DIRECTORY.startsWith("__")
+            ? undefined
+            : GENERATED_EVIDENCE_DIRECTORY);
+    if (!evidenceDirectory)
+        return;
+    const resolvedDirectory = resolve(process.cwd(), evidenceDirectory);
+    const append = process.env.SUPERCOV_DURABLE_EVIDENCE_EACH_TEST === "1"
+        ? appendJsonLineDurableSync
+        : appendJsonLineSync;
+    append(resolve(resolvedDirectory, `playwright-worker-${evidenceWriterIdentity()}-${process.pid}.mcdc.jsonl`), serialized);
+}
 const instrumentedFixtures = {
     page: async ({ page }, use, testInfo) => {
         const scope = executionScope(testInfo);
@@ -1507,6 +1562,17 @@ const instrumentedFixtures = {
             activeController = controller;
             directRuntime()?.activateCoverageScope(scope);
             await controller.adoptTrackedContexts().catch(() => undefined);
+            const carried = await controller.takeCarriedEvidence().catch(() => undefined);
+            if (carried) {
+                appendEvidenceLine(`${JSON.stringify({
+                    testId: `__supercov_background__:playwright:${scope.attemptId}`,
+                    test: "Playwright work between tests",
+                    status: "unknown",
+                    provenance: { runner: "background", kind: "background", source: "explicit" },
+                    role: "background",
+                    browser: [carried],
+                })}\n`);
+            }
             const serverOutput = serverEvidencePath(scope);
             mkdirSync(serverEvidenceDirectory(scope), { recursive: true });
             rmSync(serverOutput, { force: true });
@@ -1543,22 +1609,7 @@ const instrumentedFixtures = {
                         browser,
                         server,
                     };
-                    const serialized = `${JSON.stringify(payload)}\n`;
-                    // Pool runners may cycle-restore a VM immediately after Playwright
-                    // exits, which can discard or overwrite the normal artifact copy.
-                    // Write one uniquely named, one-shot evidence file to the runner's
-                    // shared directory as well. No test streams into this path.
-                    const evidenceDirectory = process.env["SUPERCOV_EVIDENCE_DIR"] ??
-                        (GENERATED_EVIDENCE_DIRECTORY.startsWith("__")
-                            ? undefined
-                            : GENERATED_EVIDENCE_DIRECTORY);
-                    if (evidenceDirectory) {
-                        const resolvedDirectory = resolve(process.cwd(), evidenceDirectory);
-                        const append = process.env.SUPERCOV_DURABLE_EVIDENCE_EACH_TEST === "1"
-                            ? appendJsonLineDurableSync
-                            : appendJsonLineSync;
-                        append(resolve(resolvedDirectory, `playwright-worker-${evidenceWriterIdentity()}-${process.pid}.mcdc.jsonl`), serialized);
-                    }
+                    appendEvidenceLine(`${JSON.stringify(payload)}\n`);
                 }
                 finally {
                     await timed("controller.dispose", () => controller.dispose());

package/runtime/javascript/register.mjs

@@ -847,8 +847,64 @@ function skipCoverageThresholds(tool) {
 // instead, where such a file is read as an ES module and the hook never
 // sees it; the loader is told so it can hand those files back.
 const typescriptHandler = Module._extensions[".ts"];
+// The tests a process serves reach the project files it loaded and read: a
+// mock server a suite starts, the app server. Where SUPERCOV_SUPPORT_DIR is
+// set this process notes them in a file of its own, which the loader adds
+// its modules to and the runtime the tests the process served; a change to
+// one of those files then affects those tests alone (`tests affected`,
+// `merge`).
+const supportDirectory = process.env.SUPERCOV_SUPPORT_DIR;
+let supportFile;
+if (supportDirectory) {
+    supportFile = path.join(supportDirectory, `${process.pid}-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}.jsonl`);
+    // This process's own, not a parent's it inherited.
+    process.env.__SUPERCOV_SUPPORT_FILE = supportFile;
+    try {
+        fs.mkdirSync(supportDirectory, { recursive: true });
+    }
+    catch {
+        // Then nothing is noted, and every test counts as before.
+    }
+    const projectRoot = process.env.SUPERCOV_PROJECT_ROOT;
+    const read = new Set();
+    const noteRead = (target) => {
+        try {
+            const file = typeof target === "string"
+                ? resolve(target)
+                : target instanceof URL && target.protocol === "file:"
+                    ? fileURLToPath(target)
+                    : undefined;
+            // The copy lives under `.supercov/`: what is the project's is
+            // decided by the project root.
+            const inside = file && projectRoot && file.startsWith(projectRoot + sep) ? file.slice(projectRoot.length + 1) : undefined;
+            if (!inside || inside.startsWith(`.supercov${sep}`) || inside.split(sep).includes("node_modules") || read.has(file))
+                return;
+            read.add(file);
+            fs.appendFileSync(supportFile, `${JSON.stringify({ read: file })}\n`);
+        }
+        catch {
+            // A read not noted counts for every test, as before.
+        }
+    };
+    for (const name of ["readFileSync", "readFile"]) {
+        const original = fs[name];
+        fs[name] = function supercovNotedRead(target, ...rest) {
+            noteRead(target);
+            return original.call(this, target, ...rest);
+        };
+    }
+    const readPromised = fs.promises.readFile;
+    fs.promises.readFile = function supercovNotedRead(target, ...rest) {
+        noteRead(target);
+        return readPromised.call(this, target, ...rest);
+    };
+    syncBuiltinESMExports();
+}
 register(new URL("./resolve-loader.mjs", import.meta.url), {
-    data: { typescriptRequireHook: typeof typescriptHandler === "function" && typescriptHandler.name !== "loadTS" },
+    data: {
+        typescriptRequireHook: typeof typescriptHandler === "function" && typescriptHandler.name !== "loadTS",
+        supportFile,
+    },
 });
 if (process.env.SUPERCOV_DEBUG === "1") {
     console.error("[supercov] preload", { entrypoint });

package/runtime/javascript/resolve-loader.mjs

@@ -1,4 +1,4 @@
-import { readFileSync } from "node:fs";
+import { appendFileSync, readFileSync } from "node:fs";
 import { dirname, resolve as resolvePath } from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";
 const GENERATED_TARGET = "__SUPERCOV_PLAYWRIGHT_MODULE__";
@@ -27,8 +27,40 @@ function belongsToProject(parentURL) {
     return (parentURL.startsWith(projectURL) && !parentURL.startsWith(generatedURL));
 }
 let typescriptRequireHook = false;
+// Where this process notes the project modules it loads (`register.mjs`
+// names it): a mock server a suite starts is reached by the tests it serves,
+// and a change to a file it loaded is a change for them alone.
+let supportFile;
+const supportModules = new Set();
+// A project file, not a dependency nor Supercov's own: the copy itself lives
+// under `.supercov/`, so the project root decides.
+function projectFile(file) {
+    if (!PROJECT_ROOT || !file.startsWith(PROJECT_ROOT))
+        return false;
+    const relative = file.slice(PROJECT_ROOT.length).replace(/^[\\/]/, "");
+    return relative !== "" && !/^\.supercov[\\/]/.test(relative) && !/(?:^|[\\/])node_modules[\\/]/.test(relative);
+}
+function noteLoaded(url) {
+    if (!supportFile || !url.startsWith("file:") || supportModules.has(url))
+        return;
+    supportModules.add(url);
+    try {
+        if (!projectFile(fileURLToPath(url)))
+            return;
+    }
+    catch {
+        return;
+    }
+    try {
+        appendFileSync(supportFile, `${JSON.stringify({ loaded: url })}\n`);
+    }
+    catch {
+        // Not recorded: the file counts for every test, as before.
+    }
+}
 export function initialize(data) {
     typescriptRequireHook = data?.typescriptRequireHook === true;
+    supportFile = typeof data?.supportFile === "string" ? data.supportFile : undefined;
 }
 const packageTypes = new Map();
 function packageType(directory) {
@@ -144,6 +176,7 @@ function mapToCopy(source, filename) {
 }
 export async function load(url, context, nextLoad) {
     const result = await nextLoad(url, context);
+    noteLoaded(url);
     if (!NODE_COVERAGE_FILTERS || result.source == null || !url.startsWith("file:"))
         return result;
     const text = typeof result.source === "string" ? result.source : Buffer.from(result.source).toString("utf8");

package/runtime/javascript/runtime.mjs

@@ -451,6 +451,12 @@ function persistBrowser() {
   persistBrowserListenersInstalled = true;
   try {
     addEventListener("beforeunload", handOverBrowser);
+    // An iframe the page removes (a client-side navigation away from the
+    // page that held it, an embed closed) gets no `beforeunload`: what it ran
+    // went with it, though the test clicked in it. Its `pagehide` reaches the
+    // collector, unlike a top-level document's.
+    if (typeof window !== "undefined" && window !== window.top)
+      addEventListener("pagehide", handOverBrowser);
     addEventListener("pagehide", persistBrowserNow);
     addEventListener("visibilitychange", () => {
       if (typeof document !== "undefined" && document.visibilityState === "hidden")
@@ -1425,6 +1431,55 @@ emitterContextMaps.set(runtimeInstance, emitterCoverageContexts);
 if (runtimeGlobal.__SUPERCOV_EMITTER_PATCHED_INSTANCES__ == null)
   runtimeGlobal.__SUPERCOV_EMITTER_PATCHED_INSTANCES__ = new Set();
 const patchedEmitterInstances = runtimeGlobal.__SUPERCOV_EMITTER_PATCHED_INSTANCES__;
+// Which tests this process served, and when it served a request that named
+// none, in the file `register.mjs` named for it: the project files it loaded
+// and read are reached by those tests, and a change to one is a change for
+// them alone (`tests affected`, `merge`). Its CommonJS modules are noted
+// here, once each, as requests come; the loader notes its ES modules.
+var supportNoted = { served: new Set(), modules: new Set(), lastUnscoped: 0 };
+function noteSupport(scope) {
+  const file = typeof process !== "undefined" && process.env ? process.env.__SUPERCOV_SUPPORT_FILE : void 0;
+  if (!file)
+    return;
+  const fs = getFs();
+  if (!fs)
+    return;
+  const lines = [];
+  if (scope) {
+    if (!supportNoted.served.has(scope.testId)) {
+      supportNoted.served.add(scope.testId);
+      lines.push({ served: scope.testId });
+    }
+  } else {
+    const now = Date.now();
+    if (now - supportNoted.lastUnscoped >= 5) {
+      supportNoted.lastUnscoped = now;
+      lines.push({ unscoped: now });
+    }
+  }
+  try {
+    const moduleBuiltin = typeof process.getBuiltinModule === "function" ? process.getBuiltinModule("node:module") : void 0;
+    const cache = moduleBuiltin && moduleBuiltin._cache;
+    const root = process.env.SUPERCOV_PROJECT_ROOT;
+    if (cache && root) {
+      for (const name of Object.keys(cache)) {
+        // The copy lives under `.supercov/`: the project root decides.
+        const inside = name.indexOf(root) === 0 ? name.slice(root.length).replace(/^[\\/]/, "") : "";
+        if (supportNoted.modules.has(name) || inside === "" || /^\.supercov[\\/]/.test(inside) || /(?:^|[\\/])node_modules[\\/]/.test(inside))
+          continue;
+        supportNoted.modules.add(name);
+        lines.push({ loaded: name });
+      }
+    }
+  } catch (e) {
+  }
+  if (lines.length === 0)
+    return;
+  try {
+    fs.appendFileSync(file, lines.map((line) => JSON.stringify(line)).join("\n") + "\n");
+  } catch (e) {
+  }
+}
 function installNodeRequestPropagation() {
   if (isBrowser || patchedEmitterInstances.has(runtimeInstance) || typeof process === "undefined") return;
   const builtin = (name) => typeof process.getBuiltinModule === "function" ? process.getBuiltinModule(name) : void 0;
@@ -1443,6 +1498,7 @@ function installNodeRequestPropagation() {
       // HTTP headers establish ownership even when Express/SDK listeners and
       // the server process were created by a shared before hook.
       const incoming = requestCoverageContext(args[0]);
+      noteSupport(incoming && incoming.scope);
       if (incoming) context = incoming.scope ? incoming : context != null ? context : currentRequestContext();
       if (incoming && incoming.scope)
         noteServedAttempt(incoming.scope);