supercov 3.0.4 → 4.0.0

package/docs/cli.md

@@ -142,14 +142,14 @@ npx supercov runs <run-id> [query] [options]
 | --- | --- |
 | no query | Read the overall result, test outcome, completeness, and timings |
 | `gaps` | See only files with uncovered behavior or measurement limits |
-| `files` | See every included file, including fully covered files |
+| `files` | See every included file, including fully covered files; `--group dir` shows coverage by directory, split by test kind |
 | `file <path>` | Inspect the open obligations in one file |
 | `decision <id \| path:line>` | Understand missing boolean outcomes and MC/DC witnesses |
 | `line <path:line>` | See one line's state, obligations, and covering tests |
 | `test <id \| name>` | See the coverage attributed to one test |
 | `kinds` | Group coverage by test level, such as unit or E2E |
 | `runners` | Group coverage by test runner |
-| `scope` | Review included, excluded, and ambiguous source files |
+| `scope` | See what is source and what is not, by directory and reason; `--files` lists every file |
 | `assertions` | Read the share of executed statements the tests assert, and the ones they do not; `assertions assess` works it out |
 | `source <path>` | Read matching current project source with line numbers |
 | `minimize` | Find a small test subset that preserves a coverage target |
@@ -164,6 +164,27 @@ npx supercov runs latest line app/routes/checkout.ts:57
 npx supercov runs latest test "checkout retry"
 ```
 
+To see where the coverage is, read it by directory. Each row is a directory,
+with what it holds and how much of it each kind of test covers; `--depth`
+keeps more levels, and `--metric` picks branches, functions, statements or
+MC/DC instead of lines:
+
+```sh supercov
+npx supercov runs latest files --group dir
+npx supercov runs latest files --group dir --depth 2 --metric branches
+```
+
+```
+Directory   Files  Lines     e2e    unit      All
+app             4     15  86.67%   0.00%  100.00%
+lib             1     10  90.00%  40.00%   90.00%
+components      1      2 100.00%   0.00%  100.00%
+```
+
+In `--json`, every file of `files` and `gaps` carries `totals` and `covered`
+for each metric beside what is missing, so a percentage can be worked out for
+any file or set of files.
+
 Run any query with `--help` to see only the options valid for that query:
 
 ```sh supercov
@@ -509,7 +530,8 @@ terminal or offline environment after the package has been downloaded.
 | --- | --- |
 | `SUPERCOV_SOURCE_ROOTS` | Comma-separated directories or files that hold your own code, in any language; everything else is left out |
 | `SUPERCOV_TEST_KIND` | Label the wrapped command as a test level such as `unit` or `e2e` |
-| `SUPERCOV_BUILD_COMMAND` | The build a JavaScript project's tests need, as words or a JSON array, such as `yarn workspace web build` in a monorepo whose root `build` builds every package. Without it Supercov runs the project's `build` script, through the package manager that started the tests, when the tests need built output |
+| `SUPERCOV_SOURCE_TOOLS` | Comma-separated names of tools the test command runs that read source as text, such as a linter or a spell checker, besides Biome, oxlint, dprint, cspell and knip. Each is run on your source as you wrote it instead of on the instrumented copy |
+| `SUPERCOV_KEEP_WORKSPACE` | Set to `1` to leave the instrumented copy of the project in `.supercov/workspaces/` after the run, to inspect what the command ran |
 
 Examples:
 

package/docs/coverage-model.md

@@ -94,6 +94,18 @@ Both are useful:
 - exact evidence helps you inspect or minimize individual tests;
 - aggregate evidence still shows whether the whole suite reached the source.
 
+Code that runs while no test is running counts in the run's total and in no
+test's: a server starting before the first test, modules loading, work between
+tests. When a run has any, the summary's `By test kind` table ends with a
+`no test` row for what only that covered, which is why a run with one kind of
+test can show a total above that kind's own:
+
+```
+By test kind
+  e2e             3 test(s)  lines  80.00%  branches  66.67%  MC/DC  25.00%
+  no test                    lines  17.14%  branches   0.00%  MC/DC   0.00%
+```
+
 See [Supported suites](supported-suites.md) for the attribution available from
 each runner.
 
@@ -128,6 +140,17 @@ If the summary reports ambiguous source scope, inspect it:
 npx supercov runs latest scope
 ```
 
+It prints the scope by directory and reason, the ambiguous places first, and
+the `SUPERCOV_SOURCE_ROOTS` value that would settle them. The summary names
+the same places under `Instrumentation`. Add `--files` to list every file.
+
+```
+Files  Status     Directory   Reason
+   79  AMBIGUOUS  components  unclassified first-party source
+  227  INCLUDED   app         discovered package source root
+  311  EXCLUDED   tests       test or fixture source
+```
+
 When first-party source lives in unusual directories, declare it explicitly:
 
 ```sh supercov
@@ -145,6 +168,20 @@ without a conventional source directory is measured as a whole. Functions passed
 to compile-time style macros (`stylex.create(...)`) are left as written because
 the bundler consumes them at build time; nothing about them runs.
 
+Inside the roots, tests, fixtures, tool scripts and configuration files are
+still recognised by their path and left out. A root you name settles what it
+names: with `SUPERCOV_SOURCE_ROOTS=src,scripts` the `scripts/` directory is
+source, and so is `src/test-utils` when it is named as a root. Only what lies
+below a named root is still judged by its path, and a root that is a single
+file is always measured.
+
+In a Next.js application, root `components/` and `pages/`, and the root files
+Next.js reads by name (`proxy`, `middleware`, `instrumentation`) are source
+without being declared. Under `app/` a folder is a URL segment, so the files
+the App Router serves (`route.ts`, `page.tsx`, `layout.tsx` and the rest) are
+source whatever their folders are called: `app/api/wordpress/test/route.ts` is
+the handler of `/api/wordpress/test`, not a test.
+
 Choose roots that describe code the repository owns. Do not include dependencies
 or generated output merely to make a warning disappear.
 

package/docs/performance.md

@@ -1,8 +1,9 @@
 # Speed and storage
 
-A Supercov run includes your test command, an instrumented build, and evidence
-publication. The first pass is usually the slowest; repeated passes can reuse
-the isolated build when the relevant inputs have not changed.
+A Supercov run includes your test command and evidence publication, and for a
+compiled language an instrumented build, which repeated passes can reuse when
+the relevant inputs have not changed. A JavaScript project has no such build:
+the command runs as given, and a build inside it is part of the command's time.
 
 ## See where the time went
 
@@ -17,7 +18,7 @@ The summary separates:
 | Initialization | Recovery, project discovery, and input checks |
 | Workspace preparation | Refreshing the isolated project copy |
 | Adapter setup | Preparing the runner integration |
-| Instrumented build | Building measured source, or almost nothing on a cache hit |
+| Instrumented build | Building measured source for a compiled language, or almost nothing on a cache hit; 0 for JavaScript |
 | Test command | The wrapped command, including browser, VM, or remote latency |
 | Evidence publication | Validating and storing the completed run |
 
@@ -41,7 +42,9 @@ before Supercov starts and is not coverage-engine overhead.
 
 Supercov reuses an instrumented build only when source, dependencies,
 configuration, toolchain, build mode, and instrumenter identity match. A
-possible mismatch triggers a fresh build rather than risking stale coverage.
+possible mismatch triggers a fresh build rather than risking stale coverage. A
+build inside a JavaScript project's command is the command's own and runs every
+time, as it does without Supercov.
 
 ## Use focused runs carefully
 

package/docs/troubleshooting.md

@@ -32,7 +32,8 @@ unusual directory, declare the source roots explicitly:
 SUPERCOV_SOURCE_ROOTS=src,app npx supercov -- npm test
 ```
 
-Then inspect what Supercov included and excluded:
+Then inspect what Supercov included and excluded, by directory, or file by
+file with `--files`:
 
 ```sh supercov
 npx supercov runs latest scope
@@ -41,6 +42,10 @@ npx supercov runs latest scope
 Do not broaden the roots to dependencies or generated output merely to remove a
 warning. The goal is an honest boundary around code the repository owns.
 
+A file `scope` lists as `test or fixture source` or `conventional tool script`
+was recognised by its path. If it is your code, name its directory, or the file
+itself, as a root: a named root is source whatever it is called.
+
 ## Too much is measured
 
 Vendored or generated code beside your own is measured like the rest of the
@@ -137,6 +142,21 @@ Its lines, methods and simple branches stay measured; everything that needs a
 probe is declared as a measurement limit for that file. Please report the file,
 since Supercov aims to instrument every Ruby source correctly.
 
+## The tests cannot find `dist/` or a production build
+
+Supercov runs your command in a copy of the project, and the copy starts
+without build output: `dist/`, `build/`, `.next/` and the like are not copied,
+because they hold code that was never instrumented. Supercov does not run your
+`build` script for you (earlier versions did). If the tests read build output, put
+the build in the command:
+
+```bash
+npx supercov -- sh -c "npm run build && npm test"
+```
+
+A `pretest` script, or a Playwright `webServer` that builds, does the same. The
+build then compiles the instrumented copy, and what the tests run is measured.
+
 ## A test that reads your source fails under Supercov
 
 A test that opens your source files and asserts on their text sees the probes,
@@ -215,10 +235,8 @@ runs remain intact.
 
 ## The first run is slow
 
-The first run may include the npm download, browser or toolchain startup,
-workspace creation, and an instrumented build. Repeated runs can reuse the
-isolated build when source, dependencies, configuration, toolchain, and build
-mode still match.
+The first run may include the npm download, browser or toolchain startup, and
+workspace creation.
 
 Inspect the recorded phases with:
 

package/docs/workspace-isolation.md

@@ -34,6 +34,15 @@ and the instrumented copy is not what you wrote:
 
 - Linters, formatters and `tsc --noEmit` (ESLint, Prettier, standard, xo and
   the like) read each file as you wrote it, and don't see `.supercov`.
+- The type check `next build` runs reads each file as you wrote it too, while
+  the build itself compiles the instrumented copy.
+- Biome, oxlint, dprint, cspell and knip run on your source as you wrote it:
+  the copy's `node_modules/.bin` starts each of them in a view of the copy
+  that holds every file in its original text. A file the tool writes there
+  (a report, a cache) is the command's output like any other. A change it
+  makes to a source file (`--write`, `--fix`) is not applied during a measured
+  run, and the run says so. To have another tool that reads source as text run
+  the same way, name it: `SUPERCOV_SOURCE_TOOLS=typos,stylelint`.
 - Coverage tools the command runs itself (tap, c8, nyc, Jest's and Vitest's
   `--coverage`) still collect and report coverage. They measure the
   instrumented copy, and report close to what they report without Supercov,
@@ -42,12 +51,33 @@ and the instrumented copy is not what you wrote:
 
 Files the wrapped command creates or changes inside the isolated workspace are
 synced back to the project after the run, so `supercov -- npm test -- -u`
-updates snapshots in the repository exactly as `npm test -- -u` would. Two
+updates snapshots in the repository exactly as `npm test -- -u` would. Three
 exceptions are reported instead of applied: changes the command makes to
 instrumented source files (the instrumented copies must never overwrite your
-sources) and deletions (never propagated automatically). Changes inside any
-`node_modules` directory are neither applied nor reported: dependency trees are
-not command outputs.
+sources), a build the command made from them, and deletions (never propagated
+automatically). Such a build stays behind whole: when a file in `.next/`,
+`dist/` or another directory that git ignores or the project does not have was
+built from instrumented source, nothing in that directory is copied, so the
+project never holds an instrumented build. Changes inside any `node_modules`
+directory are neither applied nor reported: dependency trees are not command
+outputs.
+
+The run prints where the synced files went, what stayed behind and what the
+command deleted, by top-level directory (`test-results/ 6`, `.next/ 2547`).
+
+The copy starts without build output: `dist/`, `build/`, `.next/` and the
+like are not copied, because they hold code that was never instrumented.
+Supercov runs the command you give it and no build of its own, so a suite that
+needs build output has the build in its command:
+
+```bash
+npx supercov -- sh -c "npm run build && npm test"
+```
+
+A `pretest` script, or a Playwright `webServer` that builds, does the same.
+When a run fails in a project that has a `build` script the command does not
+reach, it says this. `SUPERCOV_KEEP_WORKSPACE=1` leaves the instrumented copy
+in `.supercov/workspaces/` after a run, to inspect.
 
 ## Files Supercov creates
 
@@ -67,9 +97,10 @@ fallback instead of adopting or deleting it.
 
 ## Repeated runs and the build cache
 
-When source, dependencies, configuration, toolchain, and build mode still
-match, Supercov can reuse the isolated instrumented build. Test-only changes do
-not force an unrelated application rebuild.
+For a compiled language, when source, dependencies, configuration, toolchain,
+and build mode still match, Supercov can reuse the isolated instrumented build,
+so test-only changes do not force an unrelated rebuild. A JavaScript project's
+build is part of its command and runs every time.
 
 Workspace refreshes are prepared separately and become active only when
 complete. An interrupted refresh does not replace the last complete cache with

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "supercov",
-  "version": "3.0.4",
+  "version": "4.0.0",
   "description": "Coverage, security and code quality for coding agents",
   "keywords": [
     "coverage",
@@ -66,6 +66,7 @@
     "sync:rust-assets": "node scripts/sync-rust-package-assets.mjs",
     "test:rust-assets": "node scripts/sync-rust-package-assets.mjs --check",
     "test": "cargo test --workspace",
+    "test:next-playwright": "cargo build -p supercov && npm --prefix tests/fixtures/next-playwright ci && node scripts/next-playwright-integration.mjs",
     "test:react": "cargo build -p supercov && node scripts/rust-vitest-projects-integration.mjs && npm --prefix examples/react-verification ci && npm --prefix examples/react-verification run demo && node scripts/react-hydration-compatibility.mjs",
     "test:release-tooling": "node --test tests/release/*.test.mjs",
     "test:runtime": "node --test tests/runtime/*.test.mjs",
@@ -118,14 +119,14 @@
     "test:windows-tls": "node scripts/windows-tls-test.mjs"
   },
   "optionalDependencies": {
-    "@supercov/cli-darwin-arm64": "3.0.4",
-    "@supercov/cli-darwin-x64": "3.0.4",
-    "@supercov/cli-linux-arm64-gnu": "3.0.4",
-    "@supercov/cli-linux-arm64-musl": "3.0.4",
-    "@supercov/cli-linux-x64-gnu": "3.0.4",
-    "@supercov/cli-linux-x64-musl": "3.0.4",
-    "@supercov/cli-win32-arm64": "3.0.4",
-    "@supercov/cli-win32-x64": "3.0.4"
+    "@supercov/cli-darwin-arm64": "4.0.0",
+    "@supercov/cli-darwin-x64": "4.0.0",
+    "@supercov/cli-linux-arm64-gnu": "4.0.0",
+    "@supercov/cli-linux-arm64-musl": "4.0.0",
+    "@supercov/cli-linux-x64-gnu": "4.0.0",
+    "@supercov/cli-linux-x64-musl": "4.0.0",
+    "@supercov/cli-win32-arm64": "4.0.0",
+    "@supercov/cli-win32-x64": "4.0.0"
   },
   "peerDependencies": {
     "@playwright/test": ">=1.55.0",

package/runtime/javascript/register.mjs

@@ -178,6 +178,24 @@ const isAnalysisEntrypoint = new RegExp(`/node_modules/(?:\\.bin/(?:${analysisTo
         process.argv.includes("--noEmit"));
 if (isAnalysisEntrypoint)
     installAuthoredSourceView();
+// `next build` type-checks in a worker of its own instead of running tsc. A
+// probe in a condition stops TypeScript narrowing through it, so an inferred
+// return type widens, and a file outside the source roots that calls the
+// function fails the build on code nobody wrote ("'client' is possibly
+// 'null'"). Only that worker gets the authored view: the bundler, and `next
+// dev`, which sets TypeScript up in its main process, must read the copies.
+else if (/\/node_modules\/next\/dist\/compiled\/jest-worker\/processChild\.js$/.test(entrypoint) ||
+    (!workerThreads.isMainThread && /\/node_modules\/(?:\.bin\/next$|next\/dist\/)/.test(entrypoint))) {
+    const compile = Module.prototype._compile;
+    let installed = false;
+    Module.prototype._compile = function _compile(content, filename, ...rest) {
+        if (!installed && /[\\/]next[\\/]dist[\\/]lib[\\/]verify-typescript-setup\.js$/.test(filename)) {
+            installed = true;
+            installAuthoredSourceView();
+        }
+        return Reflect.apply(compile, this, [content, filename, ...rest]);
+    };
+}
 function installAuthoredSourceView() {
     let authored;
     try {
@@ -372,7 +390,16 @@ function skipCoverageThresholds(tool) {
             console.error(`[supercov] ${tool}'s coverage thresholds stay as configured`, error);
     }
 }
-register(new URL("./resolve-loader.mjs", import.meta.url));
+// A handler for `.ts` in the CommonJS loader that is not Node's own is a
+// transpiler's require hook: ts-node/register, @swc/register and the like.
+// It was there before this preload, and it is how the project's TypeScript
+// loads. Any `--import` sends the entry point through the module loader
+// instead, where such a file is read as an ES module and the hook never
+// sees it; the loader is told so it can hand those files back.
+const typescriptHandler = Module._extensions[".ts"];
+register(new URL("./resolve-loader.mjs", import.meta.url), {
+    data: { typescriptRequireHook: typeof typescriptHandler === "function" && typescriptHandler.name !== "loadTS" },
+});
 if (process.env.SUPERCOV_DEBUG === "1") {
     console.error("[supercov] preload", { entrypoint });
 }

package/runtime/javascript/resolve-loader.mjs

@@ -1,4 +1,5 @@
-import { resolve as resolvePath } from "node:path";
+import { readFileSync } from "node:fs";
+import { dirname, resolve as resolvePath } from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";
 const GENERATED_TARGET = "__SUPERCOV_PLAYWRIGHT_MODULE__";
 const TARGET = process.env.SUPERCOV_PLAYWRIGHT_MODULE ??
@@ -25,6 +26,41 @@ function belongsToProject(parentURL) {
     const generatedURL = `${projectURL}.supercov/`;
     return (parentURL.startsWith(projectURL) && !parentURL.startsWith(generatedURL));
 }
+let typescriptRequireHook = false;
+export function initialize(data) {
+    typescriptRequireHook = data?.typescriptRequireHook === true;
+}
+const packageTypes = new Map();
+function packageType(directory) {
+    if (packageTypes.has(directory))
+        return packageTypes.get(directory);
+    let type;
+    try {
+        type = JSON.parse(readFileSync(resolvePath(directory, "package.json"), "utf8")).type ?? "commonjs";
+    }
+    catch {
+        const parent = dirname(directory);
+        type = parent === directory ? "commonjs" : packageType(parent);
+    }
+    packageTypes.set(directory, type);
+    return type;
+}
+// `node -r ts-node/register --test tests/a.test.ts` in a package without
+// `"type"` passed alone and failed here with ERR_MODULE_NOT_FOUND on an
+// extensionless import: the preload made Node load the entry point as an ES
+// module, past the require hook that compiles it. A TypeScript file of a
+// CommonJS package is given back to the CommonJS loader, where that hook is.
+function throughRequireHook(resolved) {
+    if (!typescriptRequireHook || !resolved?.url?.startsWith("file:") || resolved.url.includes("/node_modules/"))
+        return resolved;
+    if (!/\.(?:ts|tsx|cts)$/.test(resolved.url) || /\.d\.c?ts$/.test(resolved.url))
+        return resolved;
+    if (resolved.format === "module" || resolved.format === "module-typescript")
+        return resolved;
+    if (!resolved.url.endsWith(".cts") && packageType(dirname(fileURLToPath(resolved.url))) === "module")
+        return resolved;
+    return { ...resolved, format: "commonjs" };
+}
 export async function resolve(specifier, context, nextResolve) {
     // Some transpilers preserve the source-relative runtime import while moving
     // only the transformed application file to an output directory. The
@@ -73,7 +109,7 @@ export async function resolve(specifier, context, nextResolve) {
         }
         return nextResolve(REPLACEMENT, context);
     }
-    return nextResolve(specifier, context);
+    return throughRequireHook(await nextResolve(specifier, context));
 }
 
 // Node's own test coverage filters files by `--test-coverage-include` and

package/runtime/javascript/runtime.mjs

@@ -890,7 +890,7 @@ function cleanInstrumentationStack(error) {
   if (!error || typeof error !== "object" || typeof error.stack !== "string")
     return error;
   const lines = error.stack.split("\n");
-  const visible = lines.filter((line, index) => index === 0 || !/[\\/]\.supercov[\\/](?:playwright|nodeTest|vitest|runtime|launchSupervisor|nodeAssert|nodeAssertStrict|nodeAssertAdapter|register|resolve-loader)\.(?:js|mjs)(?::|\))/u.test(line));
+  const visible = lines.filter((line, index) => index === 0 || !/[\\/]\.supercov[\\/](?:node_modules[\\/])?(?:playwright|nodeTest|vitest|runtime|launchSupervisor|nodeAssert|nodeAssertStrict|nodeAssertAdapter|register|resolve-loader)\.(?:js|mjs)(?::|\))/u.test(line));
   if (visible.length !== lines.length) {
     try {
       error.stack = visible.join("\n");
@@ -1402,6 +1402,10 @@ function registerProbeV2(definition) {
     // A selection's outcomes -- short falsy, short truthy, right falsy, right
     // truthy -- are four consecutive points per site from the first index.
     selectionPoints: definition.selectionPoints,
+    // What selectEndV2 and selectNextV2 read, for each tree of more than two
+    // leaves.
+    selectionSteps: (definition.selectionTrees || []).map((tree) => tree[0]),
+    selectionChoices: (definition.selectionTrees || []).map((tree) => tree[1]),
     none: optionalCallEmptySpread
   };
   state.probeV2Files.add(file);
@@ -1626,6 +1630,44 @@ function selectPathV2(file, value) {
   }
   return value;
 }
+// A selection tree whose leaves stay the program's own expressions: each is
+// the last operand of a comma that names it in the site's frame, so a type
+// checker reading the instrumented copy narrows through `a && a.b` as it does
+// in the source, and the tree's value is recorded here, once, for the leaf
+// that produced it. What a leaf before the last decided on the way is known
+// from where evaluation went next, and is recorded there as plain hits.
+// Two leaves: the left one is last only when it decided the result.
+function selectEnd2V2(file, first, value, right) {
+  coverageHitV2(file, first + (right ? 2 : 0) + (value ? 1 : 0));
+  return value;
+}
+// More: the leaf's steps, the ones selectPathV2 takes as arguments, come from
+// the file's table, whose entry for a tree is `[steps of each leaf, for each
+// leaf the points each leaf before it has decided by then]`.
+function selectEndV2(file, tree, value, leaf) {
+  const steps = file.selectionSteps[tree][leaf];
+  for (let index = 0; index < steps.length; index += 1) {
+    const step = steps[index];
+    const code = step & 3;
+    const first = (step - code) / 4;
+    if (code === 3)
+      coverageHitV2(file, value ? first + 3 : first + 2);
+    else if (code === 0 ? value : code === 1 ? !value : value !== null && value !== void 0)
+      coverageHitV2(file, value ? first + 1 : first);
+    else
+      break;
+  }
+  return value;
+}
+// Where one of several leaves can come before a leaf, the frame says which.
+// Written out in the program as a choice, it would be a branch of the user's
+// line to a coverage tool the tests run.
+function selectNextV2(file, tree, from, to) {
+  const points = file.selectionChoices[tree][to][from];
+  if (points)
+    for (let index = 0; index < points.length; index += 1)
+      coverageHitV2(file, points[index]);
+}
 // `x ||= y`, `x &&= y`, `x ??= y` keep their operator and their single
 // evaluation of the target: the right side goes through selectRightV2 (or the
 // named form, for an anonymous function the assignment would have named), and
@@ -1851,6 +1893,9 @@ const directRuntimeApi = {
   selectNamedRightV2,
   selectAssignEndV2,
   selectPathV2,
+  selectEnd2V2,
+  selectEndV2,
+  selectNextV2,
   takeNodeAssertionPhases,
   tryBegin,
   tryCatch,
@@ -1913,6 +1958,9 @@ export {
   selectNamedRightV2,
   selectAssignEndV2,
   selectPathV2,
+  selectEnd2V2,
+  selectEndV2,
+  selectNextV2,
   takeNodeAssertionPhases,
   tryBegin,
   tryCatch,