supercov 4.2.1 → 4.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -282,7 +282,7 @@ Java and Kotlin have no registry of their own here, so a JVM project takes the b
282
282
  | PHP | Coming soon | — | — |
283
283
  | C | Coming soon | — | — |
284
284
 
285
- The `npx` path requires Node.js 22 or newer; the other installs need no Node. Rust support currently uses Rust 1.95; cargo-nextest 0.9.138 and 0.9.140 are supported. Python support requires CPython 3.9 or newer and measures pytest and unittest runs. Ruby support requires Ruby 3.3 or newer (3.4 or newer for full measurement) and measures RSpec, Minitest, test-unit and Cucumber runs. Go support requires Go 1.22 or newer and measures `go test`. Java and Kotlin support requires a JDK 17 or newer with Maven or Gradle, and measures every JUnit Platform engine -- JUnit 5, JUnit 4 through Vintage, Kotest and Spock -- as well as TestNG.
285
+ The `npx` path requires Node.js 22 or newer; the other installs need no Node. Rust support currently uses Rust 1.95; cargo-nextest 0.9.138 and newer are supported. Python support requires CPython 3.9 or newer and measures pytest and unittest runs. Ruby support requires Ruby 3.3 or newer (3.4 or newer for full measurement) and measures RSpec, Minitest, test-unit and Cucumber runs. Go support requires Go 1.22 or newer and measures `go test`. Java and Kotlin support requires a JDK 17 or newer with Maven or Gradle, and measures every JUnit Platform engine -- JUnit 5, JUnit 4 through Vintage, Kotest and Spock -- as well as TestNG.
286
286
 
287
287
  ## Supported operating systems and architectures
288
288
 
@@ -83,9 +83,13 @@ npx supercov runs latest assertions
83
83
  ```
84
84
 
85
85
  `assertions assess` works it out and saves the result to the run. For each test,
86
- Supercov renders what the test ran -- its code, its helpers, and the source it
87
- executed -- and asks [Jev](https://typesafe.ai) whether the test would fail for
88
- each statement's change. It prints an estimate first; `--dry-run` sends nothing.
86
+ Supercov renders what the test ran -- its code and the source it executed --
87
+ and asks [Jev](https://typesafe.ai) whether the test would fail for each
88
+ statement's change. In JavaScript, TypeScript and Python the source shown is
89
+ the function each asked statement is in; in the other languages it is all the
90
+ test ran in the statement's file, with the helper files the test imports. It prints an estimate first; `--dry-run` sends nothing.
91
+ The estimate gives the first round and the most the pass can send, which is
92
+ what it sends when no test is judged to catch anything.
89
93
  It needs a TypeSafe AI API key in `TYPESAFE_API_KEY`.
90
94
 
91
95
  `assertions` reads the saved result with no network and no key, a page at a
@@ -125,7 +129,25 @@ ones most likely written for the statement: tests whose name shares words with
125
129
  the statement's line (`corsOrigin: a false value` for
126
130
  `return corsOrigin(origin)`), and among those the tests that ran the least
127
131
  other code. A statement many tests run can still be caught by one that was
128
- never asked; the statement view (`assertions <file>:<line>`) says how many were.
132
+ never asked; the statement view (`assertions <file>:<line>`) says how many were,
133
+ and names the test files none of whose tests were asked, which is where to look
134
+ before writing a test that may already exist.
135
+
136
+ An asserted statement is a judgment too: some test was judged to fail if it
137
+ changed, and the pass stops at the first that is. How strong that answer was
138
+ says how far to trust it. Checked by applying each change and running the
139
+ tests, an asserted verdict was wrong 22 times in 1,169 on an answer of 75% or
140
+ more, and 33 times in 711 on a weaker one. The overview says how many of the
141
+ asserted rest on a weaker answer, and `--weak` lists them:
142
+
143
+ ```bash
144
+ npx supercov runs latest assertions --weak # the files by statements asserted on a weak answer
145
+ npx supercov runs latest assertions src/adapters.ts --weak # those statements, the least certain first
146
+ ```
147
+
148
+ What reads as covered and is not is most often a deferred effect: a flag set
149
+ or an entry removed whose only consequence is what happens on a later event,
150
+ run by a test that asserts on the immediate reply.
129
151
 
130
152
  ## Which tests check a change
131
153
 
@@ -140,7 +162,7 @@ Answers are kept in `.supercov/assertions/`. Each is stored under the exact text
140
162
  Jev saw for that statement and test, and reused only when a new run would ask
141
163
  the same question with the same text. Nothing decides whether a change could
142
164
  matter: a change to the statement, to code around it that the test ran, to the
143
- test or its helpers, or to which tests run it, asks again. A dependency or
165
+ test, or to which tests run it, asks again. A dependency or
144
166
  configuration change asks everything again. After a typical commit only a few
145
167
  dozen questions are asked; the rest are reused.
146
168
 
package/docs/cli.md CHANGED
@@ -22,6 +22,7 @@ npx supercov --help
22
22
  | Find what is not asserted in one file | `npx supercov runs latest assertions <path>` |
23
23
  | See each test's answer for one statement | `npx supercov runs latest assertions <path>:<line>` |
24
24
  | See what one test is judged to catch | `npx supercov runs latest assertions --test <name>` |
25
+ | List what is asserted on a weak answer | `npx supercov runs latest assertions --weak` |
25
26
  | Read matching current source code | `npx supercov runs latest source <path>` |
26
27
  | Compare two runs | `npx supercov diff <older> <newer>` |
27
28
  | Open an interactive report | `npx supercov report` |
@@ -61,7 +62,8 @@ the score is arithmetic this command does over the answers. Text reports a band,
61
62
  `good`, `fair` or `weak`; `--json` carries the number and every check with what
62
63
  is known about it. `--all` includes test files, generated output and anything
63
64
  outside a source root, all of which are left out by default. `--dry-run` prints
64
- the exact requests and contacts nothing.
65
+ the exact requests and contacts nothing; how many there are and what they cost
66
+ is one line on stderr.
65
67
 
66
68
  `quality diff` reports what changed between two assessments: which files lost
67
69
  or gained health, each check that moved in them with its value before and after,
@@ -366,6 +368,19 @@ lost: 1 lines, 1 branches, 0 MC/DC conditions
366
368
  + branch src/fee.js:2 true
367
369
  ```
368
370
 
371
+ When the difference is in more than one file, the files come first, the one
372
+ that lost most at the top, so a component that did not render in one run reads
373
+ as one row and not as forty:
374
+
375
+ ```
376
+ in 2 files, most lost first:
377
+ src/panel.js: lost 6 lines, 1 branches
378
+ src/fee.js: lost 1 lines, 1 branches; gained 1 lines, 1 branches
379
+ ```
380
+
381
+ The counts are of the whole difference, whatever page of it is shown. In
382
+ `--json`, `files` and `byFile`.
383
+
369
384
  Two runs of the same code can differ a little: what an end-to-end suite
370
385
  reaches depends on timing, on the order of tests, and on state outside the
371
386
  code. When both runs measured the same source, `diff` says so under the
@@ -507,6 +522,11 @@ Merge creates a new run. The inputs must describe the same source,
507
522
  configuration, toolchain, schema, and coverage denominator. Supercov rejects an
508
523
  incompatible merge rather than publishing a misleading aggregate.
509
524
 
525
+ `assertions assess` assesses a merged run across every suite merged into it,
526
+ reusing the answers saved when its parts were assessed. `tests affected` is
527
+ asked of the runs that were merged, which hold the record of what each test
528
+ ran.
529
+
510
530
  ## Clean local data
511
531
 
512
532
  ```sh supercov
@@ -176,6 +176,21 @@ without a conventional source directory is measured as a whole. Functions passed
176
176
  to compile-time style macros (`stylex.create(...)`) are left as written because
177
177
  the bundler consumes them at build time; nothing about them runs.
178
178
 
179
+ In a Python or Ruby project every source file outside test directories,
180
+ virtual environments and build output is measured, except what lies beside
181
+ the package at the top of the project: `docs/`, `examples/`, benchmark
182
+ directories (`bench/`, `benchmarks/`) and `fuzz/`. A test runs none of it, and
183
+ counted in, a package covered to 99% reads 69%. Name one of them in
184
+ `SUPERCOV_SOURCE_ROOTS` and it is source.
185
+
186
+ In a Go module every `.go` file the toolchain compiles is measured: not
187
+ `vendor/` or `testdata/`, not a nested module, and not a file or directory
188
+ whose name begins with `_` or `.`, which `go build ./...` walks past. The
189
+ programs under a top-level `examples/` are left out too. A Rust run measures
190
+ what the build compiled, so a bench or an example `cargo test` does not build
191
+ is listed as not compiled. A Maven or Gradle project is measured by its source
192
+ sets: `src/main` is source, `src/test` is tests.
193
+
179
194
  Inside the roots, tests, fixtures, tool scripts and configuration files are
180
195
  still recognised by their path and left out. A root you name settles what it
181
196
  names: with `SUPERCOV_SOURCE_ROOTS=src,scripts` the `scripts/` directory is
package/docs/quality.md CHANGED
@@ -153,6 +153,12 @@ Source roots come from your package manifests and from what they declare, so a
153
153
  Test files, generated output, tool scripts, examples, benchmarks and
154
154
  documentation are left out, each with its reason. `--all` includes everything.
155
155
 
156
+ A JavaScript or TypeScript file a coverage run would measure is always read,
157
+ by `quality` and by `security`: the App Router handler at
158
+ `app/api/wordpress/test/route.ts` is a route, not a test, and a root named in
159
+ `SUPERCOV_SOURCE_ROOTS` is not second-guessed by its own name. Generated and
160
+ vendored files stay out.
161
+
156
162
  If some of your code sits somewhere none of that recognises, Supercov asks Jev
157
163
  about those paths with your whole tree as context, and says how many it decided
158
164
  that way. To decide yourself:
package/docs/security.md CHANGED
@@ -183,6 +183,12 @@ many of its measured lines no test reached. A handler that builds a query from
183
183
  request input is one thing; the same handler that no test ever executes is a
184
184
  stronger claim, and one that needs both halves of this product to make.
185
185
 
186
+ When the run's assertions were assessed (`runs <run> assertions assess`), the
187
+ report also says which flagged lines a test is judged to catch a change to, and
188
+ which flagged files tests run without catching any line of. A run that was not
189
+ assessed says so instead: nobody asked its tests, which is not the same as
190
+ their catching nothing.
191
+
186
192
  ## What a change introduced
187
193
 
188
194
  ```bash
@@ -150,7 +150,7 @@ whatever was still buffered.
150
150
  | --- | --- | --- |
151
151
  | Cargo's standard libtest runner | Exact test, attempt, and passing-assertion identity | Rust 1.95; run with `npx supercov -- cargo test` |
152
152
  | rustdoc doctests | Exact doctest identity; every doctest runs in a process of its own | Rust 1.95; part of `npx supercov -- cargo test` |
153
- | cargo-nextest | Exact test, attempt, retry, and binary identity | cargo-nextest 0.9.138 or 0.9.140 |
153
+ | cargo-nextest | Exact test, attempt, retry, and binary identity | cargo-nextest 0.9.138 or newer (tested with 0.9.140 and 0.9.148) |
154
154
 
155
155
  Supercov preserves Cargo's test selection, scheduling, fail-fast behavior,
156
156
  environment and exit status. Doctests run with their own identities; nextest
@@ -157,6 +157,13 @@ npx supercov -- sh -c "npm run build && npm test"
157
157
  A `pretest` script, or a Playwright `webServer` that builds, does the same. The
158
158
  build then compiles the instrumented copy, and what the tests run is measured.
159
159
 
160
+ A build made in an earlier step does not reach the copy either. A CI job that
161
+ ran `npm run build` and then `supercov -- node --test`, as a gate written for
162
+ Supercov 3 would, built the project and tested a copy without `dist/`: the
163
+ tests that need it fail, and the run still reports a coverage figure for the
164
+ rest. The message at the end of such a run says the copy starts without build
165
+ output and gives the command with the build in it.
166
+
160
167
  ## A test that reads your source fails under Supercov
161
168
 
162
169
  A test that opens your source files and asserts on their text sees the probes,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "supercov",
3
- "version": "4.2.1",
3
+ "version": "4.4.0",
4
4
  "description": "Coverage, security and code quality for coding agents",
5
5
  "keywords": [
6
6
  "coverage",
@@ -82,7 +82,7 @@
82
82
  "eval:skill": "claude plugin eval plugins/supercov --scaffold --trust-plugin --no-publish --model claude-sonnet-5 --judge-model claude-haiku-4-5 --threshold 0.66 --tag behavior trigger --allow-tools Bash Edit Write --max-cost-usd 15",
83
83
  "eval:skill-triggers": "claude plugin eval plugins/supercov --scaffold --trust-plugin --no-publish --model claude-sonnet-5 --judge-model claude-haiku-4-5 --threshold 0.66 --tag trigger --ablation none --max-cost-usd 5",
84
84
  "test:engine": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace && npm run test:runtime && npm run test:rust-assets && npm run test:engine-integrations",
85
- "test:engine-integrations": "cargo build -p supercov && node scripts/html-report-integration.mjs && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/rust-classic-browser-integration.mjs && node scripts/rust-top-packages-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
85
+ "test:engine-integrations": "cargo build -p supercov && node scripts/html-report-integration.mjs && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/mocha-failure-output.mjs && node scripts/failure-output.mjs node ava tap && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/rust-classic-browser-integration.mjs && node scripts/rust-top-packages-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
86
86
  "test:platform": "cargo test --workspace && cargo build -p supercov && node scripts/rust-process-supervision.mjs && node scripts/workspace-crash-integration.mjs",
87
87
  "test:native-package": "cargo build --release -p supercov && node scripts/native-package-integration.mjs && node scripts/native-release-set-integration.mjs",
88
88
  "test:pypi-wheel": "node scripts/pypi-wheel-integration.mjs",
@@ -104,11 +104,11 @@
104
104
  "prepublishOnly": "npm run release:check",
105
105
  "test:child-attribution": "cargo build -p supercov && node scripts/rust-child-attribution-integration.mjs",
106
106
  "test:host-loader": "cargo build -p supercov && node scripts/rust-host-loader-integration.mjs",
107
- "test:python": "cargo build -p supercov && node scripts/python-integration.mjs",
108
- "test:ruby-coverage": "cargo build -p supercov && node scripts/ruby-coverage-integration.mjs",
109
- "test:go": "cargo test --no-fail-fast -p supercov-engine --test go_frontend --test go_run",
107
+ "test:python": "cargo build -p supercov && node scripts/python-integration.mjs && node scripts/failure-output.mjs python",
108
+ "test:ruby-coverage": "cargo build -p supercov && node scripts/ruby-coverage-integration.mjs && node scripts/failure-output.mjs ruby",
109
+ "test:go": "cargo test --no-fail-fast -p supercov-engine --test go_frontend --test go_run && cargo build -p supercov && node scripts/failure-output.mjs go",
110
110
  "test:jvm": "cargo test --no-fail-fast -p supercov-engine --test jvm_frontend --test jvm_run -- --include-ignored --test-threads=1",
111
- "test:rust-public-cargo": "cargo build -p supercov && node scripts/rust-public-cargo-integration.mjs",
111
+ "test:rust-public-cargo": "cargo build -p supercov && node scripts/rust-public-cargo-integration.mjs && node scripts/failure-output.mjs rust",
112
112
  "test:assertions": "cargo build -p supercov && node scripts/assertions-integration.mjs",
113
113
  "docs:sync": "node scripts/sync-docs.mjs",
114
114
  "docs:check": "node scripts/sync-docs.mjs --check",
@@ -119,14 +119,14 @@
119
119
  "test:windows-tls": "node scripts/windows-tls-test.mjs"
120
120
  },
121
121
  "optionalDependencies": {
122
- "@supercov/cli-darwin-arm64": "4.2.1",
123
- "@supercov/cli-darwin-x64": "4.2.1",
124
- "@supercov/cli-linux-arm64-gnu": "4.2.1",
125
- "@supercov/cli-linux-arm64-musl": "4.2.1",
126
- "@supercov/cli-linux-x64-gnu": "4.2.1",
127
- "@supercov/cli-linux-x64-musl": "4.2.1",
128
- "@supercov/cli-win32-arm64": "4.2.1",
129
- "@supercov/cli-win32-x64": "4.2.1"
122
+ "@supercov/cli-darwin-arm64": "4.4.0",
123
+ "@supercov/cli-darwin-x64": "4.4.0",
124
+ "@supercov/cli-linux-arm64-gnu": "4.4.0",
125
+ "@supercov/cli-linux-arm64-musl": "4.4.0",
126
+ "@supercov/cli-linux-x64-gnu": "4.4.0",
127
+ "@supercov/cli-linux-x64-musl": "4.4.0",
128
+ "@supercov/cli-win32-arm64": "4.4.0",
129
+ "@supercov/cli-win32-x64": "4.4.0"
130
130
  },
131
131
  "peerDependencies": {
132
132
  "@playwright/test": ">=1.55.0",
@@ -11,7 +11,7 @@ import { realpathSync } from "node:fs";
11
11
  import { join } from "node:path";
12
12
  import { pathToFileURL } from "node:url";
13
13
  import { beginBufferedServerEvidence, flushBufferedServerEvidence, takeNodeAssertionPhases, withCoverageCarrier, } from "./runtime.mjs";
14
- import { runnerExecutionScope, writeRunnerEvidence } from "./runnerEvidence.mjs";
14
+ import { runnerExecutionScope, stackAsWritten, writeRunnerEvidence } from "./runnerEvidence.mjs";
15
15
 
16
16
  // The installed ava package the entrypoint belongs to.
17
17
  function avaRoot(entrypoint) {
@@ -26,6 +26,24 @@ function avaRoot(entrypoint) {
26
26
  return at < 0 ? undefined : real.slice(0, at + "/node_modules/ava".length);
27
27
  }
28
28
 
29
+ function asWritten(failure, depth = 0) {
30
+ if (!failure || typeof failure !== "object" || depth > 4)
31
+ return;
32
+ try {
33
+ // AVA prints the stack of an error a test threw as it is, and
34
+ // shortens a failed assertion's by its own working directory, which
35
+ // is the copy. A thrown error reaches here as an assertion's cause.
36
+ const shortened = failure.type === "ava" && !(failure.originalError?.cause instanceof Error);
37
+ if (typeof failure.stack === "string")
38
+ failure.stack = stackAsWritten(failure.stack, shortened);
39
+ for (const inner of Array.isArray(failure.errors) ? failure.errors : [])
40
+ asWritten(inner, depth + 1);
41
+ }
42
+ catch {
43
+ // A stack that cannot be written is printed as it is.
44
+ }
45
+ }
46
+
29
47
  export async function adaptAva(entrypoint) {
30
48
  const evidenceDirectory = process.env.SUPERCOV_EVIDENCE_DIR;
31
49
  const root = avaRoot(entrypoint);
@@ -66,6 +84,20 @@ export async function adaptAva(entrypoint) {
66
84
  });
67
85
  };
68
86
  Runner.prototype.runTest.supercovAva = true;
87
+ // AVA prints a failure's stack as it is. The test ran in the copy, so a
88
+ // frame of the test file named it there, and a frame of a rewritten
89
+ // module was named by its path where Node names a module by its URL:
90
+ // `at seal (/home/me/shop/src/crypto.js:3:11)` over
91
+ // `at file:///home/me/shop/.supercov/workspaces/…/test/crypto.test.js:5:18`.
92
+ // A failure is told to the reporter as a state change; its stack is put
93
+ // right on the way.
94
+ const emit = Runner.prototype.emit;
95
+ if (typeof emit === "function")
96
+ Runner.prototype.emit = function supercovAvaEmit(name, data, ...rest) {
97
+ if (name === "stateChange" && data?.err)
98
+ asWritten(data.err);
99
+ return Reflect.apply(emit, this, [name, data, ...rest]);
100
+ };
69
101
  // A skipped or todo test never reaches runTest; AVA announces it.
70
102
  const start = Runner.prototype.start;
71
103
  if (typeof start === "function")
@@ -64,6 +64,52 @@ export async function adaptMocha(entrypoint) {
64
64
  return Reflect.apply(run, this, args);
65
65
  };
66
66
  Runner.prototype.run.supercovMocha = true;
67
+ // Mocha takes its working directory off the front of each frame and
68
+ // prints the rest. Under Supercov that is the copy, while an application
69
+ // frame names the project's own file, so it printed whole:
70
+ // `at seal (/home/me/shop/src/crypto.js:3:11)` for `src/crypto.js:3:11`.
71
+ // And Supercov's hooks into asynchronous work put a frame of Node's under
72
+ // every stack that starts in a timer. A failure reads as it does without
73
+ // Supercov before Mocha or a reporter sees it.
74
+ const fail = Runner.prototype.fail;
75
+ if (typeof fail === "function") {
76
+ const place = asWritten();
77
+ Runner.prototype.fail = function supercovMochaFail(runnable, error, ...rest) {
78
+ place(error);
79
+ return Reflect.apply(fail, this, [runnable, error, ...rest]);
80
+ };
81
+ }
82
+ }
83
+
84
+ const HOOK_FRAME = /^\s*at process\.(?:callbackTrampoline|topLevelDomainCallback) \(node:(?:internal\/async_hooks|domain):\d+:\d+\)$/;
85
+
86
+ function asWritten() {
87
+ const real = (directory) => {
88
+ try {
89
+ return directory ? realpathSync(directory) : undefined;
90
+ }
91
+ catch {
92
+ return undefined;
93
+ }
94
+ };
95
+ const project = real(process.env.SUPERCOV_SOURCE_PROJECT_ROOT);
96
+ const copy = real(process.env.SUPERCOV_PROJECT_ROOT);
97
+ return (error) => {
98
+ if (!error || typeof error.stack !== "string")
99
+ return;
100
+ const lines = error.stack.split("\n").filter((line) => !HOOK_FRAME.test(line));
101
+ const placed = project && copy && project !== copy
102
+ ? lines.map((line) => /^\s*at /.test(line) && line.includes(project + sep) && !line.includes(copy + sep)
103
+ ? line.replace(`(${project}${sep}`, `(${copy}${sep}`).replace(` at ${project}${sep}`, ` at ${copy}${sep}`)
104
+ : line)
105
+ : lines;
106
+ try {
107
+ error.stack = placed.join("\n");
108
+ }
109
+ catch {
110
+ // A stack that cannot be written is printed as it is.
111
+ }
112
+ };
67
113
  }
68
114
 
69
115
  function observe(runner, runtime, write) {
@@ -2,8 +2,8 @@ import { AsyncLocalStorage } from "node:async_hooks";
2
2
  import * as native from "node:test";
3
3
  import { fileURLToPath } from "node:url";
4
4
  import vm from "node:vm";
5
- import { beginBufferedServerEvidence, flushBufferedServerEvidence, takeNodeAssertionPhases, withCoverageCarrier, } from "./runtime.mjs";
6
- import { callerLocation, runnerExecutionScope, runnerTestId, writeRunnerEvidence, } from "./runnerEvidence.mjs";
5
+ import { beginBufferedServerEvidence, flushBufferedServerEvidence, takeNodeAssertionPhases, withCoverageCarrier, withStackRoom, } from "./runtime.mjs";
6
+ import { callerLocation, runnerExecutionScope, runnerTestId, stackAsWritten, writeRunnerEvidence, } from "./runnerEvidence.mjs";
7
7
  function callbackIndex(args) {
8
8
  for (let index = args.length - 1; index >= 0; index -= 1)
9
9
  if (typeof args[index] === "function")
@@ -65,11 +65,7 @@ function restoreUserError(error, depth = 0) {
65
65
  : text;
66
66
  try {
67
67
  if (typeof error.stack === "string")
68
- error.stack = remap(error.stack
69
- .split("\n")
70
- .filter((line) => !(line.trimStart().startsWith("at ") &&
71
- line.includes("/.supercov/")))
72
- .join("\n"));
68
+ error.stack = stackAsWritten(error.stack);
73
69
  if (typeof error.message === "string")
74
70
  error.message = remap(error.message);
75
71
  }
@@ -227,9 +223,9 @@ function wrappedRegistration(original, parentTestId, forcedStatus) {
227
223
  finishBody();
228
224
  done?.(error);
229
225
  };
230
- return withCoverageCarrier({ version: 1, scope }, () => Reflect.apply(callback, callbackThis, [contextProxy, callbackDone]));
226
+ return withStackRoom(() => withCoverageCarrier({ version: 1, scope }, () => Reflect.apply(callback, callbackThis, [contextProxy, callbackDone])));
231
227
  }
232
- const result = withCoverageCarrier({ version: 1, scope }, () => Reflect.apply(callback, callbackThis, [contextProxy]));
228
+ const result = withStackRoom(() => withCoverageCarrier({ version: 1, scope }, () => Reflect.apply(callback, callbackThis, [contextProxy])));
233
229
  if (result && typeof result.then === "function")
234
230
  return Promise.resolve(result).then((value) => {
235
231
  finishBody();
@@ -250,6 +250,14 @@ process.__SUPERCOV_FAILURE_OUTPUT__ ??= function supercovFailureOutput(runner) {
250
250
  // name the copy's file, and coverage reports it by its path in the
251
251
  // project's layout.
252
252
  vitest: { prints: /[\\/]vitest[\\/]dist[\\/]/, names: true },
253
+ // A test's process makes each frame's path relative to where it
254
+ // runs (@tapjs/stack), and the process that reports reads the file
255
+ // under a frame by that path to print it (prismjs-terminal, for
256
+ // @tapjs/reporter): `../../../../src/crypto.js` over the right
257
+ // lines, or `src/crypto.js` over Supercov's.
258
+ // libtap, which tap 16 and older print with, reads the file itself,
259
+ // and their reporter reads it through cardinal.
260
+ tap: { prints: /[\\/](?:@tapjs[\\/](?:stack|reporter)|prismjs-terminal|libtap|cardinal)[\\/]/, names: false },
253
261
  }[runner];
254
262
  // Asked with no runner, the reads are in place and change nothing until
255
263
  // one is named: a process that may yet run a suite.
@@ -288,9 +296,11 @@ process.__SUPERCOV_FAILURE_OUTPUT__ ??= function supercovFailureOutput(runner) {
288
296
  rewritten = new Set();
289
297
  }
290
298
  }
299
+ // ts-node names a file with forward slashes on Windows too.
300
+ const named = sep === "\\" ? file.split("/").join(sep) : file;
291
301
  for (const root of roots) {
292
- if (file.startsWith(root)) {
293
- const local = file.slice(root.length).split(sep).join("/");
302
+ if (named.startsWith(root)) {
303
+ const local = named.slice(root.length).split(sep).join("/");
294
304
  return rewritten.has(local) ? local : undefined;
295
305
  }
296
306
  }
@@ -437,7 +447,8 @@ process.__SUPERCOV_FAILURE_OUTPUT__ ??= function supercovFailureOutput(runner) {
437
447
  function supercovReadFileSync(file, ...rest) {
438
448
  if (!failureOutput.prints)
439
449
  return Reflect.apply(readSync, this, [file, ...rest]);
440
- const local = rewrittenUnder(copies, file);
450
+ // A printer may name the file from where it runs.
451
+ const local = rewrittenUnder(copies, typeof file === "string" && !isAbsolute(file) ? resolve(file) : file);
441
452
  if (local === undefined) {
442
453
  const contents = Reflect.apply(readSync, this, [file, ...rest]);
443
454
  return failureOutput.composes && typeof contents === "string" && typeof file === "string" && file.endsWith(".map")
@@ -478,7 +489,61 @@ process.__SUPERCOV_FAILURE_OUTPUT__ ??= function supercovFailureOutput(runner) {
478
489
  }
479
490
  path.relative = supercovRelative;
480
491
  syncBuiltinESMExports();
492
+ // ts-node compiles the text it is given, which is the rewritten file,
493
+ // and its source map ends there: TypeScript reads no map in its input.
494
+ // A frame under Mocha read `src/crypto.ts:37:10` for line 8, and a test
495
+ // file's `:15:172` for column 12. ts-node keeps what it compiled in
496
+ // memory and has source-map-support read it from there, so that is
497
+ // where the map is carried on to the lines the author wrote: a handler
498
+ // of Supercov's in the list the library keeps on the global object,
499
+ // asked before its default.
500
+ const shared = Symbol.for("source-map-support/sharedData");
501
+ function supercovRetrieveSourceMap(source) {
502
+ try {
503
+ const file = typeof source === "string" && source.startsWith("file://") ? fileURLToPath(source) : source;
504
+ if (rewrittenUnder(copies, file) === undefined)
505
+ return null;
506
+ let code;
507
+ for (const read of globalThis[shared]?.retrieveFileHandlers ?? []) {
508
+ code = read(file);
509
+ if (code)
510
+ break;
511
+ }
512
+ const found = typeof code === "string" ? code.lastIndexOf("//# sourceMappingURL=data:application/json") : -1;
513
+ if (found < 0)
514
+ return null;
515
+ const text = Buffer.from(code.slice(code.indexOf(",", found) + 1).trim(), "base64").toString("utf8");
516
+ const through = composed(text);
517
+ return through === text ? null : { url: source, map: through };
518
+ }
519
+ catch {
520
+ return null;
521
+ }
522
+ }
523
+ failureOutput.adopt = () => {
524
+ const handlers = globalThis[shared]?.retrieveMapHandlers;
525
+ if (Array.isArray(handlers) && !handlers.includes(supercovRetrieveSourceMap))
526
+ handlers.unshift(supercovRetrieveSourceMap);
527
+ };
528
+ failureOutput.adopt();
481
529
  };
530
+ // source-map-support is ts-node's, loaded when ts-node is: by a runner's own
531
+ // `--require`, after this file, or by Node's, before it.
532
+ {
533
+ const adopt = () => {
534
+ process.__SUPERCOV_FAILURE_OUTPUT__();
535
+ failureOutput.adopt?.();
536
+ };
537
+ if (globalThis[Symbol.for("source-map-support/sharedData")])
538
+ adopt();
539
+ const load = Module._load;
540
+ Module._load = function supercovSourceMapSupportLoad(request, ...rest) {
541
+ const loaded = Reflect.apply(load, this, [request, ...rest]);
542
+ if (typeof request === "string" && request.includes("source-map-support"))
543
+ adopt();
544
+ return loaded;
545
+ };
546
+ }
482
547
  // Jest itself prints from here on. Any other process is Jest's from the
483
548
  // moment it loads Jest: a worker of its, or a tool that runs Jest in its own
484
549
  // process (`require("jest").run()`, as react-scripts and Vue's CLI do), where
@@ -487,6 +552,9 @@ process.__SUPERCOV_FAILURE_OUTPUT__ ??= function supercovFailureOutput(runner) {
487
552
  // of process) never loads it.
488
553
  if (isJestEntrypoint)
489
554
  process.__SUPERCOV_FAILURE_OUTPUT__("jest");
555
+ // tap prints from the process its command starts, and from each test's own.
556
+ else if (process.env.TAP_CHILD_ID || /\/node_modules\/(?:\.bin\/tap$|tap\/|@tapjs\/run\/)/.test(entrypoint))
557
+ process.__SUPERCOV_FAILURE_OUTPUT__("tap");
490
558
  else {
491
559
  const jest = /(?:^|[\\/])(?:jest|jest-cli|@jest[\\/]core|jest-runner)(?:[\\/]|$)/;
492
560
  const load = Module._load;
@@ -1,7 +1,7 @@
1
1
  import { createHash, randomBytes } from "node:crypto";
2
2
  import { readFileSync } from "node:fs";
3
- import { relative, resolve, sep } from "node:path";
4
- import { fileURLToPath } from "node:url";
3
+ import { dirname, isAbsolute, relative, resolve, sep } from "node:path";
4
+ import { fileURLToPath, pathToFileURL } from "node:url";
5
5
  import { appendEvidenceRecord } from "./atomic.mjs";
6
6
  import { inferTestProvenance } from "./provenance.mjs";
7
7
  import { serverEvidencePath } from "./transport.mjs";
@@ -133,3 +133,78 @@ export function writeRunnerEvidence(identity, status, scope, evidenceDirectoryOv
133
133
  };
134
134
  appendEvidenceRecord(resolve(process.cwd(), evidenceDirectory), identity.runner.replace(/[^A-Za-z0-9_-]/g, "_"), payload);
135
135
  }
136
+
137
+ // A failing test's stack as its author would have seen it, for the runners
138
+ // that print a stack as it is (node:test, AVA).
139
+ //
140
+ // - A frame of Supercov's is one in a file of its own, straight under a
141
+ // `.supercov` directory or its `node_modules`. The copy lies under a
142
+ // `.supercov` directory too, and every frame in it was once dropped for
143
+ // that: a failing node:test test printed no frame of its own file.
144
+ // `AsyncLocalStorage.run` is Node's frame for running a test under
145
+ // Supercov, and goes with the frames around it.
146
+ // - The test ran in the copy, and a frame of a test file named it there.
147
+ // - Node names a module's file by its URL and a CommonJS file by its path.
148
+ // A rewritten file is named through its source map, by path either way:
149
+ // `at seal (/home/me/shop/src/crypto.js:3:11)` for `file:///home/…`.
150
+ // - What is left is cut to the length the author's limit allows; the
151
+ // runtime made room for its own frames while they were on the stack.
152
+ const OWN_FRAME = /[\\/]\.supercov[\\/](?:node_modules[\\/])?[^\\/:]+\.[cm]?js:\d+/u;
153
+ const CARRIER_FRAME = /^\s*at AsyncLocalStorage\.run \(node:internal\//u;
154
+ const FRAME_PATH = /^(\s*at (?:.*? \()?)((?:\/|[A-Za-z]:[\\/])[^()]*?)(:\d+:\d+\)?)$/u;
155
+ const packageTypes = new Map();
156
+ function loadedAsModule(file) {
157
+ if (/\.m[jt]sx?$/u.test(file))
158
+ return true;
159
+ if (/\.c[jt]sx?$/u.test(file))
160
+ return false;
161
+ const visited = [];
162
+ let directory = dirname(file);
163
+ let type;
164
+ for (;;) {
165
+ if (packageTypes.has(directory)) {
166
+ type = packageTypes.get(directory);
167
+ break;
168
+ }
169
+ visited.push(directory);
170
+ try {
171
+ type = JSON.parse(readFileSync(resolve(directory, "package.json"), "utf8")).type === "module";
172
+ break;
173
+ }
174
+ catch {
175
+ // No manifest here, or not one that parses: the next directory up.
176
+ }
177
+ const parent = dirname(directory);
178
+ if (parent === directory) {
179
+ type = false;
180
+ break;
181
+ }
182
+ directory = parent;
183
+ }
184
+ for (const seen of visited)
185
+ packageTypes.set(seen, type);
186
+ return type;
187
+ }
188
+ //
189
+ // `whereItRan` leaves each file where the stack has it: for a printer that
190
+ // takes its working directory off a frame itself, and would not know the
191
+ // project's.
192
+ export function stackAsWritten(stack, whereItRan = false) {
193
+ const copy = process.env["SUPERCOV_PROJECT_ROOT"];
194
+ const project = whereItRan ? undefined : process.env["SUPERCOV_SOURCE_PROJECT_ROOT"];
195
+ const placed = (text) => copy && project && copy !== project ? text.split(copy).join(project) : text;
196
+ const lines = stack.split("\n");
197
+ const own = lines.map((line) => /^\s*at /u.test(line) && OWN_FRAME.test(line));
198
+ const limit = Error.stackTraceLimit;
199
+ let frames = 0;
200
+ return lines
201
+ .filter((line, index) => !own[index] && !(CARRIER_FRAME.test(line) && (own[index - 1] || own[index + 1])))
202
+ .map((line) => {
203
+ const frame = FRAME_PATH.exec(placed(line));
204
+ if (!frame || !project || !isAbsolute(frame[2]) || !frame[2].startsWith(project + sep) || !loadedAsModule(frame[2]))
205
+ return placed(line);
206
+ return `${frame[1]}${pathToFileURL(frame[2]).href}${frame[3]}`;
207
+ })
208
+ .filter((line) => !/^\s*at /u.test(line) || typeof limit !== "number" || !Number.isFinite(limit) || ++frames <= limit)
209
+ .join("\n");
210
+ }
@@ -892,31 +892,92 @@ const INSTRUMENTATION_FRAME = /[\\/]\.supercov[\\/](?:node_modules[\\/])?(?:play
892
892
  // added, at the assertion, and below the runtime's frames the function the
893
893
  // author wrote, at the wrapper. Vitest printed both, `:12:35` and `:12:1`.
894
894
  // They are one frame again: the author's function, where the assertion is.
895
+ //
896
+ // The runtime's frames are not always one run. `node:assert` called through
897
+ // its module is wrapped a second time, so an assertion that fails there has
898
+ // frames of the runtime above the added function too, and Node's own frame
899
+ // for `AsyncLocalStorage.run` lies among those below it: Mocha printed the
900
+ // added function, that frame and the author's function, three for one.
901
+ const NODE_FRAME = /^\s*at .*\(?node:internal\//u;
902
+ const CARRIER_FRAME = /^\s*at AsyncLocalStorage\.run \(node:internal\//u;
903
+ // An error keeps as many frames as `Error.stackTraceLimit` says, ten unless
904
+ // someone changed it, counted from where it was made. The runtime's frames
905
+ // were counted among them, so the author's own fell off the end: a failed
906
+ // `node:assert` under node:test kept the function Supercov added and lost
907
+ // the test's, and a test's stack lost the runner's frames below it. While
908
+ // the runtime's frames are on the stack there is room for them, and what is
909
+ // left when they are taken out is cut to the length the author asked for.
910
+ const OWN_ROOM = 24;
911
+ function withStackRoom(callback) {
912
+ const limit = Error.stackTraceLimit;
913
+ if (typeof limit !== "number" || !Number.isFinite(limit) || limit <= 0)
914
+ return callback();
915
+ Error.stackTraceLimit = limit + OWN_ROOM;
916
+ try {
917
+ return callback();
918
+ } finally {
919
+ // Unless the code it ran set a limit of its own.
920
+ if (Error.stackTraceLimit === limit + OWN_ROOM)
921
+ Error.stackTraceLimit = limit;
922
+ }
923
+ }
924
+ function withinStackLimit(lines) {
925
+ const limit = Error.stackTraceLimit;
926
+ if (typeof limit !== "number" || !Number.isFinite(limit) || limit < 0)
927
+ return lines;
928
+ let frames = 0;
929
+ return lines.filter((line) => !/^\s*at /.test(line) || ++frames <= limit);
930
+ }
895
931
  function cleanInstrumentationStack(error, lexical = false) {
896
932
  if (!error || typeof error !== "object" || typeof error.stack !== "string")
897
933
  return error;
898
934
  const lines = error.stack.split("\n");
899
935
  const hidden = lines.map((line, index) => index > 0 && INSTRUMENTATION_FRAME.test(line));
900
- const first = hidden.indexOf(true);
901
- if (first < 0)
936
+ if (!hidden.includes(true))
902
937
  return error;
903
- const last = hidden.lastIndexOf(true);
904
- let visible;
905
938
  const frame = /^(\s*at (?:async )?)(?:(.*?) \()?(.*?:\d+:\d+)\)?$/;
906
- const added = lexical && first > 1 ? frame.exec(lines[first - 1]) : null;
907
- const written = added && last + 1 < lines.length ? frame.exec(lines[last + 1]) : null;
908
939
  const file = (location) => location.replace(/:\d+:\d+$/, "");
909
- // Both are frames of the author's file, or the error is another's. A name
910
- // on the added one comes from a source map (Jest names a frame by what is
911
- // called at its position), and it is the name the author's line would have.
912
- if (added && written && file(added[3]) === file(written[3])) {
913
- const merged = added[2] !== void 0 || written[2] === void 0 ? lines[first - 1] : `${written[1]}${written[2]} (${added[3]})`;
914
- visible = [...lines.slice(0, first - 1), merged, ...lines.slice(last + 2)];
915
- } else {
916
- visible = lines.filter((line, index) => !hidden[index]);
917
- }
940
+ // The function Supercov added is the frame above a run of the runtime's
941
+ // that ends at a frame of the same file: the author's.
942
+ let merged;
943
+ let at = -1;
944
+ for (let start = 2; lexical && !merged && start < lines.length; start += 1) {
945
+ if (!hidden[start] || hidden[start - 1])
946
+ continue;
947
+ let last = start;
948
+ for (let index = start; index < lines.length && (hidden[index] || NODE_FRAME.test(lines[index])); index += 1) {
949
+ if (hidden[index])
950
+ last = index;
951
+ }
952
+ const added = frame.exec(lines[start - 1]);
953
+ const written = last + 1 < lines.length ? frame.exec(lines[last + 1]) : null;
954
+ // Both are frames of the author's file, or the error is another's. A name
955
+ // on the added one comes from a source map (Jest names a frame by what is
956
+ // called at its position), and it is the name the author's line would
957
+ // have; the function Supercov added has none of its own.
958
+ if (added && written && file(added[3]) === file(written[3])) {
959
+ const named = added[2] !== void 0 && added[2] !== "<anonymous>";
960
+ at = start - 1;
961
+ merged = {
962
+ last: last + 1,
963
+ text: named || written[2] === void 0 ? lines[at] : `${written[1]}${written[2]} (${added[3]})`
964
+ };
965
+ }
966
+ start = last;
967
+ }
968
+ const visible = lines.flatMap((line, index) => {
969
+ if (merged && index === at)
970
+ return [merged.text];
971
+ if (merged && index > at && index <= merged.last)
972
+ return [];
973
+ // Node's frame for running code under Supercov goes with the runtime's
974
+ // frames around it.
975
+ if (CARRIER_FRAME.test(line) && (hidden[index - 1] || hidden[index + 1]))
976
+ return [];
977
+ return hidden[index] ? [] : [line];
978
+ });
918
979
  try {
919
- error.stack = visible.join("\n");
980
+ error.stack = withinStackLimit(visible).join("\n");
920
981
  } catch (e) {
921
982
  }
922
983
  return error;
@@ -931,7 +992,7 @@ function withNodeAssertionPhase(operation, source, callback) {
931
992
  // with the frames its author would see.
932
993
  if (!scope) {
933
994
  try {
934
- return callback();
995
+ return withStackRoom(callback);
935
996
  } catch (error) {
936
997
  throw cleanInstrumentationStack(error, lexical);
937
998
  }
@@ -968,7 +1029,7 @@ function withNodeAssertionPhase(operation, source, callback) {
968
1029
  attempt.phases.push(phase);
969
1030
  attempt.phaseIds.add(phase.id);
970
1031
  try {
971
- const result = withCoverageCarrier({ version: 1, scope, phaseId: phase.id }, callback);
1032
+ const result = withStackRoom(() => withCoverageCarrier({ version: 1, scope, phaseId: phase.id }, callback));
972
1033
  if (result && typeof result.then === "function")
973
1034
  return Promise.resolve(result).then((value) => {
974
1035
  finishAssertionPhase(phase);
@@ -1886,6 +1947,7 @@ const directRuntimeApi = {
1886
1947
  coverageContextEnvironment,
1887
1948
  coverageContextHeaders,
1888
1949
  cleanInstrumentationStack,
1950
+ withStackRoom,
1889
1951
  coverageHit,
1890
1952
  coverageHitV2,
1891
1953
  coverageSnapshot,
@@ -1950,6 +2012,7 @@ export {
1950
2012
  coverageContextEnvironment,
1951
2013
  coverageContextHeaders,
1952
2014
  cleanInstrumentationStack,
2015
+ withStackRoom,
1953
2016
  coverageHit,
1954
2017
  coverageHitV2,
1955
2018
  coverageSnapshot,
@@ -19,9 +19,10 @@ import { runnerExecutionScope, writeRunnerEvidence } from "./runnerEvidence.mjs"
19
19
  async function subtestClasses() {
20
20
  const classes = [];
21
21
  let fromTap;
22
+ let tap;
22
23
  try {
23
24
  const fromProject = createRequire(join(process.cwd(), "package.json"));
24
- const tap = realpathSync(dirname(fromProject.resolve("tap/package.json")));
25
+ tap = realpathSync(dirname(fromProject.resolve("tap/package.json")));
25
26
  fromTap = createRequire(join(tap, "package.json"));
26
27
  }
27
28
  catch {
@@ -54,6 +55,22 @@ async function subtestClasses() {
54
55
  // libtap is CommonJS, and its exports hide lib/: reach it from its main.
55
56
  try {
56
57
  const lib = dirname(fromTap.resolve("libtap"));
58
+ // tap sets libtap up before it loads it: which frames are tap's own,
59
+ // and how a frame is taken through a source map. Loaded from here
60
+ // first, libtap kept its defaults for the whole file, and a failure
61
+ // listed tap's frames under the test's.
62
+ try {
63
+ fromTap(join(tap, "settings.js"));
64
+ }
65
+ catch {
66
+ // A tap that has no settings of its own leaves libtap's.
67
+ }
68
+ try {
69
+ nameFromTheCopy(createRequire(join(lib, "test.js"))("stack-utils"));
70
+ }
71
+ catch {
72
+ // Frames stay as libtap writes them.
73
+ }
57
74
  classes.push(createRequire(import.meta.url)(join(lib, "test.js")));
58
75
  }
59
76
  catch {
@@ -62,6 +79,49 @@ async function subtestClasses() {
62
79
  return classes;
63
80
  }
64
81
 
82
+ // libtap takes the directory the test runs in off a frame, which under
83
+ // Supercov is the copy, while a measured file's frame names the file its
84
+ // author wrote: the whole of `/home/me/shop/src/crypto.js` for
85
+ // `src/crypto.js`. A frame in the project is placed in the copy before
86
+ // libtap reads it, as Mocha's are.
87
+ function nameFromTheCopy(StackUtils) {
88
+ const real = (directory) => {
89
+ try {
90
+ return directory ? realpathSync(directory).replace(/\\/g, "/") : undefined;
91
+ }
92
+ catch {
93
+ return undefined;
94
+ }
95
+ };
96
+ const project = real(process.env.SUPERCOV_SOURCE_PROJECT_ROOT);
97
+ const copy = real(process.env.SUPERCOV_PROJECT_ROOT);
98
+ const { clean, at, parseLine } = StackUtils?.prototype ?? {};
99
+ if (!project || !copy || project === copy || typeof clean !== "function" || clean.supercovTap)
100
+ return;
101
+ const inTheProject = (text) => text.includes(`${project}/`) && !text.includes(`${copy}/`);
102
+ StackUtils.prototype.clean = function supercovTapClean(stack, ...rest) {
103
+ const lines = (Array.isArray(stack) ? stack : String(stack).split("\n")).map((line) => {
104
+ const text = String(line).replace(/\\/g, "/");
105
+ return inTheProject(text) ? text.split(`${project}/`).join(`${copy}/`) : line;
106
+ });
107
+ return Reflect.apply(clean, this, [lines, ...rest]);
108
+ };
109
+ StackUtils.prototype.clean.supercovTap = true;
110
+ const placed = (found) => {
111
+ if (typeof found?.file === "string" && found.file.startsWith(`${project}/`) && !found.file.startsWith(`${copy}/`))
112
+ found.file = found.file.slice(project.length + 1);
113
+ return found;
114
+ };
115
+ if (typeof at === "function")
116
+ StackUtils.prototype.at = function supercovTapAt(...given) {
117
+ return placed(Reflect.apply(at, this, given));
118
+ };
119
+ if (typeof parseLine === "function")
120
+ StackUtils.prototype.parseLine = function supercovTapParseLine(...given) {
121
+ return placed(Reflect.apply(parseLine, this, given));
122
+ };
123
+ }
124
+
65
125
  export async function adaptTap(testFile) {
66
126
  const evidenceDirectory = process.env.SUPERCOV_EVIDENCE_DIR;
67
127
  if (!evidenceDirectory)