supercov 4.19.0 → 4.20.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -199,12 +199,12 @@ Measured on four JavaScript and TypeScript projects (7,393 executed statements):
199
199
 
200
200
  | Project | Checked | Right | Flagged (truly not asserted) | Share reported / true |
201
201
  | --- | --- | --- | --- | --- |
202
- | google/uuid (Go) | 339 | 298 | 34 (28) | 90.0% / 81.4% |
203
- | hashicorp/go-version (Go) | 257 | 253 | 21 (19) | 91.8% / 91.8% |
202
+ | google/uuid (Go) | 291 of 326 | 254 | 16 (12) | 94.5% / 84.5% |
203
+ | hashicorp/go-version (Go) | 232 of 237 | 230 | 0 (0) | 100.0% / 99.1% |
204
204
  | dtolnay/semver (Rust) | 200 of a sample of 200 | 190 | 7 (2) | 96.5% / 96.5% |
205
205
  | apache/commons-cli (Java) | 196 of a sample of 200 | 167 | 49 (33) | 75.0% / 76.5% |
206
- | dry-rb/dry-inflector (Ruby 3.4) | 157 | 148 | 13 (8) | 91.7% / 92.4% |
207
- | sporkmonger/addressable (Ruby) | 198 of a sample of 200 | 182 | 25 (14) | 87.4% / 90.4% |
206
+ | dry-rb/dry-inflector (Ruby 3.3) | 157 of 157 | 151 | 12 (9) | 92.4% / 92.4% |
207
+ | sporkmonger/addressable (Ruby 3.3) | 199 of a sample of 200 | 183 | 17 (11) | 91.5% / 89.4% |
208
208
  | square/moshi (Kotlin) | 60 of a sample of 60 | 52 | 9 (6) | 85.0% / 81.7% |
209
209
 
210
210
  - **Held out:** five projects nothing was tuned on, each assessed once by the
@@ -217,14 +217,23 @@ Measured on four JavaScript and TypeScript projects (7,393 executed statements):
217
217
  | pallets/itsdangerous (Python) | 60 of 60 | 56 | 3 (2) | 95.0% / 91.7% |
218
218
  | — every statement | 238 of 238 | 224 | 13 (9) | 94.5% / 92.0% |
219
219
  | Masterminds/semver (Go) | 60 of 60 | 55 | 7 (7) | 88.3% / 80.0% |
220
- | — every statement | 497 of 498 | 472 | 46 (44) | 90.7% / 86.5% |
220
+ | — every statement | 443 of 461 | 422 | 9 (9) | 98.0% / 93.2% |
221
221
  | hashie/hashie (Ruby 3.4) | 60 of 60 | 57 | 3 (1) | 95.0% / 96.7% |
222
- | — every statement | 700 of 701 | 655 | 26 (10) | 96.3% / 94.4% |
222
+ | — every statement | 771 of 772 | 720 | 22 (7) | 97.1% / 94.4% |
223
223
  | FasterXML/java-classmate (Java) | 60 of 60 | 58 | 9 (7) | 85.0% / 88.3% |
224
224
  | bluss/arrayvec (Rust) | 54 of 60 | 48 | 1 (1) | 98.1% / 87.0% |
225
225
 
226
226
  Assessing all five (2,582 statements) cost $0.15, about 0.006 cents per
227
- statement. Go reads high: Jev takes a dropped error (`return false,
227
+ statement.
228
+
229
+ The Go and Ruby rows of both tables, the held-out ones' every-statement rows
230
+ among them, were measured again on 19 October 2026 by the current release,
231
+ at each project's latest commit and on Ruby 3.3; the projects have changed
232
+ since, so the counts differ from the first study's. The 60-statement samples
233
+ and the Rust, Java, Kotlin and Python rows are from that first study.
234
+ `node scripts/assertions-truth.mjs <project> -- <test command>` makes each
235
+ change, runs the tests and prints a row (Go and Ruby; a statement no
236
+ version of which builds is left out, so not every statement is checked). Go reads high: Jev takes a dropped error (`return false,
228
237
  fmt.Errorf(..)` becoming `return false, nil`) as caught where the tests check
229
238
  only the boolean. Rust reads high too on arrayvec, whose code is generic over
230
239
  its element type; six of its sampled changes cannot be made, because no value
package/docs/cli.md CHANGED
@@ -541,8 +541,8 @@ Neither input run is changed. Under the totals it lists what was lost, marked
541
541
  `-`, and then what was gained, marked `+`:
542
542
 
543
543
  ```
544
- gained: 1 lines, 1 branches, 0 MC/DC conditions
545
- lost: 1 lines, 1 branches, 0 MC/DC conditions
544
+ gained: 1 line, 1 branch, 0 MC/DC conditions
545
+ lost: 1 line, 1 branch, 0 MC/DC conditions
546
546
  - line src/fee.js:5
547
547
  - branch src/fee.js:2 false
548
548
  + line src/fee.js:3
@@ -555,8 +555,8 @@ as one row and not as forty:
555
555
 
556
556
  ```
557
557
  in 2 files, most lost first:
558
- src/panel.js: lost 6 lines, 1 branches
559
- src/fee.js: lost 1 lines, 1 branches; gained 1 lines, 1 branches
558
+ src/panel.js: lost 6 lines, 1 branch
559
+ src/fee.js: lost 1 line, 1 branch; gained 1 line, 1 branch
560
560
  ```
561
561
 
562
562
  The counts are of the whole difference, whatever page of it is shown. In
@@ -625,7 +625,9 @@ npx supercov runs latest minimize --metric branches --target 90
625
625
  ```
626
626
 
627
627
  `minimize` finds a small set of tests that preserves the selected coverage
628
- target. It does not edit, delete, or skip tests for you. Treat the result as an
628
+ target. Without `--target` it keeps everything the tests together cover, so
629
+ it works on a run that is not fully covered; `--target 90` asks for 90% of
630
+ each selected metric instead. It does not edit, delete, or skip tests for you. Treat the result as an
629
631
  analysis aid, not permission to remove tests that protect behavior outside the
630
632
  selected metric.
631
633
 
@@ -921,6 +923,7 @@ terminal or offline environment after the package has been downloaded.
921
923
  | `SUPERCOV_SOURCE_ROOTS` | Comma-separated directories or files that hold your own code, in any language; everything else is left out |
922
924
  | `SUPERCOV_TEST_KIND` | Label the wrapped command as a test level such as `unit` or `e2e` |
923
925
  | `SUPERCOV_SOURCE_TOOLS` | Comma-separated names of tools the test command runs that read source as text, such as a linter or a spell checker, besides Biome, oxlint, dprint, cspell and knip. Each is run on your source as you wrote it instead of on the instrumented copy |
926
+ | `SUPERCOV_RUN_ID` | Set by Supercov, not by you: in every process a measured command starts, the run's id, so a test or harness can tell it is measured |
924
927
  | `SUPERCOV_RUN_ID_FILE` | A file to write the recorded run's id to, for a script to query that run rather than `latest` |
925
928
  | `SUPERCOV_KEEP_WORKSPACE` | Set to `1` to leave the instrumented copy of the project in `.supercov/workspaces/` after the run, to inspect what the command ran |
926
929
 
@@ -937,6 +940,7 @@ SUPERCOV_TEST_KIND=e2e npx supercov -- npx playwright test
937
940
  | --- | --- |
938
941
  | `0` | The command or query succeeded |
939
942
  | Wrapped command's code | The test command failed and Supercov preserved its status |
943
+ | `127`, `126` | The test command could not start: no such program, or one that may not be run, as a shell reports it |
940
944
  | `1` | A valid measurement failed a policy you set, such as a coverage floor |
941
945
  | `2` | Supercov could not complete the request, or the evidence cannot answer it |
942
946
 
@@ -189,6 +189,14 @@ Files Status Directory Reason
189
189
  311 EXCLUDED tests test or fixture source
190
190
  ```
191
191
 
192
+ A file under no source root that first-party source imports is source:
193
+ Supercov follows the imports of the source it found, through what those load
194
+ in turn, before anything runs, and counts what they reach with the reason
195
+ `loaded by first-party source`. A file only tests import stays ambiguous,
196
+ since tests load their own helpers, mocks and simulators. What stays
197
+ ambiguous counts in no percentage, and `runs check` names it under its result
198
+ without failing, since the suite passes without Supercov.
199
+
192
200
  When first-party source lives in unusual directories, declare it explicitly:
193
201
 
194
202
  ```sh supercov
@@ -234,7 +234,11 @@ is your entry, and gives `[dev]` an inspector port unless you set one; your
234
234
  own configuration and command are not changed. While the suite runs,
235
235
  Supercov reads what the Worker ran through that inspector every 5 ms, and
236
236
  each read is credited to the browser test that was running then, if only one
237
- was, as for a [Go server](#a-go-server-under-browser-tests). The Worker's
237
+ was, as for a [Go server](#a-go-server-under-browser-tests). Several Workers
238
+ in one `wrangler dev` (`-c a/wrangler.toml -c b/wrangler.toml`) are each
239
+ measured: wrangler's inspector shows one of them, so Supercov reads each
240
+ through workerd's own inspector, whose address a small preload in wrangler's
241
+ Node process reports. The Worker's
238
242
  coverage is a Rust run of its own, published as the run or beside the browser
239
243
  run. Requests made before the first test, such as the readiness check
240
244
  `webServer` waits on, are the run's and no test's.
@@ -242,10 +246,11 @@ run. Requests made before the first test, such as the readiness check
242
246
  The build runs with `--cap-lints=warn`, so a crate that denies warnings still
243
247
  builds with the probes in it.
244
248
 
245
- ### A Rust Worker that `cargo test` starts
249
+ ### A Cloudflare Worker in Rust that `cargo test` starts
246
250
 
247
- Integration tests that start `wrangler dev` themselves -- to test a Worker in
248
- Cloudflare's own runtime -- measure the Worker too:
251
+ Integration tests that start `wrangler dev` themselves -- to test a Cloudflare
252
+ Worker written in Rust in workerd, Cloudflare's own runtime -- measure the
253
+ Worker too, Durable Objects included:
249
254
 
250
255
  ```sh
251
256
  npx supercov -- cargo test
@@ -257,13 +262,20 @@ dev` builds it in the copy as usual. The tests find a `wrangler` of
257
262
  Supercov's first on `PATH`: for `wrangler dev` it picks an inspector port,
258
263
  starts a reader of the Worker through it, and then runs your `wrangler` with
259
264
  the test's own arguments, process and signals. What the Worker ran is
260
- credited to the test that started it, exactly, as its own coverage. The
261
- Worker crate counts though `cargo test` never compiles it, once a test has
262
- run it. This needs a `wrangler` on `PATH` when the run starts; without one,
263
- nothing is put in front of it. A run whose tests start no Worker still compiles the crate under
264
- `cargo test --workspace`; its files then say they are a Rust Worker no test
265
- started, in `gaps` and the file view, rather than reading as code the tests
266
- missed.
265
+ credited to the test that started it, exactly, as its own coverage. This
266
+ needs a `wrangler` on `PATH` when the run starts; without one, nothing is
267
+ put in front of it.
268
+
269
+ A Worker crate's code is what worker-build compiles: for wasm, as a release
270
+ unless its `[build]` command passes `--dev`, never as a test, with the
271
+ features the command and the crate's defaults enable. Code a
272
+ `#[cfg(target_arch = "wasm32")]`, a feature or `debug_assertions` keeps out
273
+ of that build is not counted, and wasm-only code is, whatever target
274
+ `cargo test` builds the rest for. The crate counts whether or not
275
+ `cargo test` compiles it (`cargo test -p host` does not) and whether or not
276
+ a test started its Worker, so every run of the workspace has one
277
+ denominator. Where no test of a run started a Worker, its files say so, in
278
+ `gaps` and the file view, rather than reading as code the tests missed.
267
279
 
268
280
  `wrangler dev --remote` runs the Worker on Cloudflare's edge, and wrangler
269
281
  opens no inspector into it there, so nothing can read what it ran. A test
@@ -271,10 +283,64 @@ that starts it that way runs as usual; the Worker's files then say they ran
271
283
  remotely and could not be read. Start the Worker with `wrangler dev` to
272
284
  measure it.
273
285
 
286
+ Code in a Durable Object (`#[durable_object]` and its `impl DurableObject`)
287
+ is measured like the rest of the Worker and credited to the test whose
288
+ requests reached it. So is a Worker that runs beside another in one
289
+ `wrangler dev` (`-c plane/wrangler.toml -c runners/wrangler.toml`), reached
290
+ only through the first's service bindings: wrangler's inspector shows one
291
+ Worker of a session, and Supercov reads each through workerd's own, whose
292
+ address a small preload in wrangler's Node process reports. A test that
293
+ sets its own `NODE_OPTIONS` keeps them; the preload is added after. A workspace with several Worker crates counts each crate
294
+ once, and runs of different tests -- each starting different Workers, or
295
+ none -- merge with `supercov merge`. Tests that run a Worker only when its `build/` exists, as a test
296
+ kit that will not start an unbuilt Worker does, run it: a `build/` you made
297
+ beforehand and keep out of git is in the copy, and `wrangler dev` builds the
298
+ crate with probes over it there.
299
+
300
+ What a project needs:
301
+
302
+ - `wrangler` on `PATH` in the job that runs Supercov, before the run starts,
303
+ such as `node_modules/.bin` of a project that installs it.
304
+ - The `wasm32-unknown-unknown` target and `worker-build`, which the Worker's
305
+ `[build]` command runs.
306
+ - The Worker crates inside the source Supercov measures, as workspace members
307
+ or under the project root.
308
+
309
+ The project needs no `cfg` flag or build step of its own for Supercov.
310
+
311
+ #### What a test harness can rely on
312
+
313
+ - **Any configuration, anywhere.** A test may write its own wrangler
314
+ configuration, per test, outside the crate, with a `main` of its own: a
315
+ module that loads the built Worker and adds stand-ins or test controls.
316
+ Supercov leaves every configuration as it was written. When a test runs
317
+ `wrangler dev`, Supercov's `wrangler` hands it a copy of the configuration
318
+ it was given (`-c`, or the one wrangler finds from its directory), beside
319
+ the original as `.supercov-wrangler-<pid>-<n>.toml` (or `.json`), whose
320
+ `main` is a small entry that remembers the Worker's wasm instances and then
321
+ loads your `main`. The copy is removed once that `wrangler` exits. A Worker
322
+ is read when wasm built from the crate's measured source runs in it,
323
+ whatever the configuration's `main` imports.
324
+ - **The measured build.** The configuration's `[build]` builds the crate
325
+ with worker-build (any `cwd`), or its `main` imports a build made in the
326
+ run's copy. A `build/` made before the run, without probes, has nothing to
327
+ read.
328
+ - **Why a Worker was not measured.** Where tests started `wrangler dev` and
329
+ no measured build ran in it, the Worker's files name each `wrangler dev -c
330
+ <path>` the tests started, and why a configuration was left as it was (no
331
+ `main`, say), in `gaps` and the file view.
332
+ - **`--inspector-port`.** A test may pass its own: Supercov reads the Worker
333
+ through that port.
334
+ - **Knowing the run is measured.** `SUPERCOV_RUN_ID` is set in every process
335
+ a measured command starts, holding the run's id. The other `SUPERCOV_`
336
+ variables a test may see are Supercov's own wiring and change without
337
+ notice.
338
+ - **Output.** `cargo test -- --nocapture` shows each test's output as it is
339
+ written, as plain `cargo test` does.
340
+
274
341
  The Worker built with probes stays in the copy: its `build/` output is never
275
- returned to the project, and neither is the copy's configuration, which
276
- points `main` at Supercov's entry. wrangler's `.wrangler/` state is not a
277
- build input, so writing it does not make the run stale.
342
+ returned to the project, nor are wrangler's `.wrangler/` state and the
343
+ configuration copies above. Writing them does not make the run stale.
278
344
 
279
345
  ### Rust tests compiled to wasm
280
346
 
@@ -424,8 +490,13 @@ Ruby 3.3 and newer support statement, method, branch and MC/DC measurement,
424
490
  including loops, iterator blocks, short-circuit operators, pattern matching,
425
491
  optional calls and exception paths.
426
492
 
427
- Some constructs have narrower coverage. Code in a non-main Ractor keeps line
428
- coverage but may lack other observations. Certain nested-return expressions
493
+ Some constructs have narrower coverage. Code written inside a `Ractor.new`
494
+ block keeps line coverage but may lack other observations. Code a non-main
495
+ Ractor calls elsewhere runs as it would without Supercov, but what it runs
496
+ there is not recorded: the run says so, and a line only a Ractor reached reads
497
+ as not executed. A project whose code or tests mention `Ractor` gets probes a
498
+ Ractor can call, which cost more than the usual ones (about a fifth more in a
499
+ tight loop). Certain nested-return expressions
429
500
  limit normal-completion measurement. Constant predicates are folded as Ruby
430
501
  folds them, so unreachable alternatives do not become obligations.
431
502
 
@@ -508,8 +579,10 @@ beyond one deferred line per test that binds it to its evidence.
508
579
 
509
580
  `go test` builds one binary per package, so each test package records its own
510
581
  evidence and Supercov merges them. It also caches packages that passed, and a
511
- cached package does not run — so Supercov adds `-count=1` unless your command
512
- already says otherwise, and tells you it did.
582
+ cached package does not run — so Supercov adds `-count=1` to `GOFLAGS` unless
583
+ your command or `GOFLAGS` already says otherwise, and tells you it did. Through
584
+ `GOFLAGS` it reaches `go test` however your command starts it, `make test`
585
+ included.
513
586
 
514
587
  A test that calls `t.Parallel()` runs alongside others. Probes are a store into
515
588
  one array shared by the process, so nothing can say which of two concurrent
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "supercov",
3
- "version": "4.19.0",
3
+ "version": "4.20.1",
4
4
  "description": "Coverage, security and code quality for coding agents",
5
5
  "keywords": [
6
6
  "coverage",
@@ -95,7 +95,10 @@
95
95
  "test:test262": "cargo build --release -p supercov && node scripts/test262-equivalence.mjs",
96
96
  "benchmark:check": "cargo build --release -p supercov && node scripts/rust-transform-benchmark.mjs",
97
97
  "benchmark:python": "cargo build -p supercov && node scripts/python-benchmark.mjs",
98
- "check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:release-tooling && npm run test:runtime && npm run test:action && npm run test:rust-assets && npm run test:agent-commands && npm run test:assertions && npm run test:sweep-target && npm run test:skip-guard && npm run test:windows-tls && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
98
+ "check": "npm run check:lint && npm run check:tests",
99
+ "check:lint": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings",
100
+ "check:tests": "npm run test && npm run check:tooling",
101
+ "check:tooling": "npm run test:release-tooling && npm run test:runtime && npm run test:action && npm run test:rust-assets && npm run test:agent-commands && npm run test:assertions && npm run test:sweep-target && npm run test:skip-guard && npm run test:windows-tls && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
99
102
  "release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:go && npm run test:jvm && npm run test:launcher && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
100
103
  "release:bump": "node scripts/bump-version.mjs",
101
104
  "sweep": "node scripts/sweep-target.mjs",
@@ -119,14 +122,14 @@
119
122
  "test:windows-tls": "node scripts/windows-tls-test.mjs"
120
123
  },
121
124
  "optionalDependencies": {
122
- "@supercov/cli-darwin-arm64": "4.19.0",
123
- "@supercov/cli-darwin-x64": "4.19.0",
124
- "@supercov/cli-linux-arm64-gnu": "4.19.0",
125
- "@supercov/cli-linux-arm64-musl": "4.19.0",
126
- "@supercov/cli-linux-x64-gnu": "4.19.0",
127
- "@supercov/cli-linux-x64-musl": "4.19.0",
128
- "@supercov/cli-win32-arm64": "4.19.0",
129
- "@supercov/cli-win32-x64": "4.19.0"
125
+ "@supercov/cli-darwin-arm64": "4.20.1",
126
+ "@supercov/cli-darwin-x64": "4.20.1",
127
+ "@supercov/cli-linux-arm64-gnu": "4.20.1",
128
+ "@supercov/cli-linux-arm64-musl": "4.20.1",
129
+ "@supercov/cli-linux-x64-gnu": "4.20.1",
130
+ "@supercov/cli-linux-x64-musl": "4.20.1",
131
+ "@supercov/cli-win32-arm64": "4.20.1",
132
+ "@supercov/cli-win32-x64": "4.20.1"
130
133
  },
131
134
  "peerDependencies": {
132
135
  "@playwright/test": ">=1.55.0",
@@ -979,22 +979,21 @@ function appendServer(record) {
979
979
  var _a8, _b, _c, _d;
980
980
  if (state.runtimeSnapshots) {
981
981
  const timestampMs = (_a8 = record.timestampMs) != null ? _a8 : Date.now();
982
- const statement = record.statementId ? { statementId: record.statementId } : {};
983
982
  if (record.type === "decision") {
984
- recordBrowserEvent(__spreadProps(__spreadValues(__spreadValues({
983
+ recordBrowserEvent(__spreadProps(__spreadValues({
985
984
  type: "decision",
986
985
  id: record.meta.id,
987
986
  vector: record.vector,
988
987
  timestampMs
989
- }, record.phaseId ? { phaseId: record.phaseId } : {}), statement), {
988
+ }, record.phaseId ? { phaseId: record.phaseId } : {}), {
990
989
  environment: "server"
991
990
  }));
992
991
  } else {
993
- recordBrowserEvent(__spreadProps(__spreadValues(__spreadValues({
992
+ recordBrowserEvent(__spreadProps(__spreadValues({
994
993
  type: "hit",
995
994
  id: record.id,
996
995
  timestampMs
997
- }, record.phaseId ? { phaseId: record.phaseId } : {}), statement), {
996
+ }, record.phaseId ? { phaseId: record.phaseId } : {}), {
998
997
  environment: "server"
999
998
  }));
1000
999
  }