supercov 4.19.0 → 4.20.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -199,12 +199,12 @@ Measured on four JavaScript and TypeScript projects (7,393 executed statements):
199
199
 
200
200
  | Project | Checked | Right | Flagged (truly not asserted) | Share reported / true |
201
201
  | --- | --- | --- | --- | --- |
202
- | google/uuid (Go) | 339 | 298 | 34 (28) | 90.0% / 81.4% |
203
- | hashicorp/go-version (Go) | 257 | 253 | 21 (19) | 91.8% / 91.8% |
202
+ | google/uuid (Go) | 291 of 326 | 254 | 16 (12) | 94.5% / 84.5% |
203
+ | hashicorp/go-version (Go) | 232 of 237 | 230 | 0 (0) | 100.0% / 99.1% |
204
204
  | dtolnay/semver (Rust) | 200 of a sample of 200 | 190 | 7 (2) | 96.5% / 96.5% |
205
205
  | apache/commons-cli (Java) | 196 of a sample of 200 | 167 | 49 (33) | 75.0% / 76.5% |
206
- | dry-rb/dry-inflector (Ruby 3.4) | 157 | 148 | 13 (8) | 91.7% / 92.4% |
207
- | sporkmonger/addressable (Ruby) | 198 of a sample of 200 | 182 | 25 (14) | 87.4% / 90.4% |
206
+ | dry-rb/dry-inflector (Ruby 3.3) | 157 of 157 | 151 | 12 (9) | 92.4% / 92.4% |
207
+ | sporkmonger/addressable (Ruby 3.3) | 199 of a sample of 200 | 183 | 17 (11) | 91.5% / 89.4% |
208
208
  | square/moshi (Kotlin) | 60 of a sample of 60 | 52 | 9 (6) | 85.0% / 81.7% |
209
209
 
210
210
  - **Held out:** five projects nothing was tuned on, each assessed once by the
@@ -217,14 +217,23 @@ Measured on four JavaScript and TypeScript projects (7,393 executed statements):
217
217
  | pallets/itsdangerous (Python) | 60 of 60 | 56 | 3 (2) | 95.0% / 91.7% |
218
218
  | — every statement | 238 of 238 | 224 | 13 (9) | 94.5% / 92.0% |
219
219
  | Masterminds/semver (Go) | 60 of 60 | 55 | 7 (7) | 88.3% / 80.0% |
220
- | — every statement | 497 of 498 | 472 | 46 (44) | 90.7% / 86.5% |
220
+ | — every statement | 443 of 461 | 422 | 9 (9) | 98.0% / 93.2% |
221
221
  | hashie/hashie (Ruby 3.4) | 60 of 60 | 57 | 3 (1) | 95.0% / 96.7% |
222
- | — every statement | 700 of 701 | 655 | 26 (10) | 96.3% / 94.4% |
222
+ | — every statement | 771 of 772 | 720 | 22 (7) | 97.1% / 94.4% |
223
223
  | FasterXML/java-classmate (Java) | 60 of 60 | 58 | 9 (7) | 85.0% / 88.3% |
224
224
  | bluss/arrayvec (Rust) | 54 of 60 | 48 | 1 (1) | 98.1% / 87.0% |
225
225
 
226
226
  Assessing all five (2,582 statements) cost $0.15, about 0.006 cents per
227
- statement. Go reads high: Jev takes a dropped error (`return false,
227
+ statement.
228
+
229
+ The Go and Ruby rows of both tables, the held-out ones' every-statement rows
230
+ among them, were measured again on 19 October 2026 by the current release,
231
+ at each project's latest commit and on Ruby 3.3; the projects have changed
232
+ since, so the counts differ from the first study's. The 60-statement samples
233
+ and the Rust, Java, Kotlin and Python rows are from that first study.
234
+ `node scripts/assertions-truth.mjs <project> -- <test command>` makes each
235
+ change, runs the tests and prints a row (Go and Ruby; a statement no
236
+ version of which builds is left out, so not every statement is checked). Go reads high: Jev takes a dropped error (`return false,
228
237
  fmt.Errorf(..)` becoming `return false, nil`) as caught where the tests check
229
238
  only the boolean. Rust reads high too on arrayvec, whose code is generic over
230
239
  its element type; six of its sampled changes cannot be made, because no value
package/docs/cli.md CHANGED
@@ -541,8 +541,8 @@ Neither input run is changed. Under the totals it lists what was lost, marked
541
541
  `-`, and then what was gained, marked `+`:
542
542
 
543
543
  ```
544
- gained: 1 lines, 1 branches, 0 MC/DC conditions
545
- lost: 1 lines, 1 branches, 0 MC/DC conditions
544
+ gained: 1 line, 1 branch, 0 MC/DC conditions
545
+ lost: 1 line, 1 branch, 0 MC/DC conditions
546
546
  - line src/fee.js:5
547
547
  - branch src/fee.js:2 false
548
548
  + line src/fee.js:3
@@ -555,8 +555,8 @@ as one row and not as forty:
555
555
 
556
556
  ```
557
557
  in 2 files, most lost first:
558
- src/panel.js: lost 6 lines, 1 branches
559
- src/fee.js: lost 1 lines, 1 branches; gained 1 lines, 1 branches
558
+ src/panel.js: lost 6 lines, 1 branch
559
+ src/fee.js: lost 1 line, 1 branch; gained 1 line, 1 branch
560
560
  ```
561
561
 
562
562
  The counts are of the whole difference, whatever page of it is shown. In
@@ -625,7 +625,9 @@ npx supercov runs latest minimize --metric branches --target 90
625
625
  ```
626
626
 
627
627
  `minimize` finds a small set of tests that preserves the selected coverage
628
- target. It does not edit, delete, or skip tests for you. Treat the result as an
628
+ target. Without `--target` it keeps everything the tests together cover, so
629
+ it works on a run that is not fully covered; `--target 90` asks for 90% of
630
+ each selected metric instead. It does not edit, delete, or skip tests for you. Treat the result as an
629
631
  analysis aid, not permission to remove tests that protect behavior outside the
630
632
  selected metric.
631
633
 
@@ -937,6 +939,7 @@ SUPERCOV_TEST_KIND=e2e npx supercov -- npx playwright test
937
939
  | --- | --- |
938
940
  | `0` | The command or query succeeded |
939
941
  | Wrapped command's code | The test command failed and Supercov preserved its status |
942
+ | `127`, `126` | The test command could not start: no such program, or one that may not be run, as a shell reports it |
940
943
  | `1` | A valid measurement failed a policy you set, such as a coverage floor |
941
944
  | `2` | Supercov could not complete the request, or the evidence cannot answer it |
942
945
 
@@ -189,6 +189,14 @@ Files Status Directory Reason
189
189
  311 EXCLUDED tests test or fixture source
190
190
  ```
191
191
 
192
+ A file under no source root that first-party source imports is source:
193
+ Supercov follows the imports of the source it found, through what those load
194
+ in turn, before anything runs, and counts what they reach with the reason
195
+ `loaded by first-party source`. A file only tests import stays ambiguous,
196
+ since tests load their own helpers, mocks and simulators. What stays
197
+ ambiguous counts in no percentage, and `runs check` names it under its result
198
+ without failing, since the suite passes without Supercov.
199
+
192
200
  When first-party source lives in unusual directories, declare it explicitly:
193
201
 
194
202
  ```sh supercov
@@ -234,7 +234,11 @@ is your entry, and gives `[dev]` an inspector port unless you set one; your
234
234
  own configuration and command are not changed. While the suite runs,
235
235
  Supercov reads what the Worker ran through that inspector every 5 ms, and
236
236
  each read is credited to the browser test that was running then, if only one
237
- was, as for a [Go server](#a-go-server-under-browser-tests). The Worker's
237
+ was, as for a [Go server](#a-go-server-under-browser-tests). Several Workers
238
+ in one `wrangler dev` (`-c a/wrangler.toml -c b/wrangler.toml`) are each
239
+ measured: wrangler's inspector shows one of them, so Supercov reads each
240
+ through workerd's own inspector, whose address a small preload in wrangler's
241
+ Node process reports. The Worker's
238
242
  coverage is a Rust run of its own, published as the run or beside the browser
239
243
  run. Requests made before the first test, such as the readiness check
240
244
  `webServer` waits on, are the run's and no test's.
@@ -242,10 +246,11 @@ run. Requests made before the first test, such as the readiness check
242
246
  The build runs with `--cap-lints=warn`, so a crate that denies warnings still
243
247
  builds with the probes in it.
244
248
 
245
- ### A Rust Worker that `cargo test` starts
249
+ ### A Cloudflare Worker in Rust that `cargo test` starts
246
250
 
247
- Integration tests that start `wrangler dev` themselves -- to test a Worker in
248
- Cloudflare's own runtime -- measure the Worker too:
251
+ Integration tests that start `wrangler dev` themselves -- to test a Cloudflare
252
+ Worker written in Rust in workerd, Cloudflare's own runtime -- measure the
253
+ Worker too, Durable Objects included:
249
254
 
250
255
  ```sh
251
256
  npx supercov -- cargo test
@@ -257,13 +262,20 @@ dev` builds it in the copy as usual. The tests find a `wrangler` of
257
262
  Supercov's first on `PATH`: for `wrangler dev` it picks an inspector port,
258
263
  starts a reader of the Worker through it, and then runs your `wrangler` with
259
264
  the test's own arguments, process and signals. What the Worker ran is
260
- credited to the test that started it, exactly, as its own coverage. The
261
- Worker crate counts though `cargo test` never compiles it, once a test has
262
- run it. This needs a `wrangler` on `PATH` when the run starts; without one,
263
- nothing is put in front of it. A run whose tests start no Worker still compiles the crate under
264
- `cargo test --workspace`; its files then say they are a Rust Worker no test
265
- started, in `gaps` and the file view, rather than reading as code the tests
266
- missed.
265
+ credited to the test that started it, exactly, as its own coverage. This
266
+ needs a `wrangler` on `PATH` when the run starts; without one, nothing is
267
+ put in front of it.
268
+
269
+ A Worker crate's code is what worker-build compiles: for wasm, as a release
270
+ unless its `[build]` command passes `--dev`, never as a test, with the
271
+ features the command and the crate's defaults enable. Code a
272
+ `#[cfg(target_arch = "wasm32")]`, a feature or `debug_assertions` keeps out
273
+ of that build is not counted, and wasm-only code is, whatever target
274
+ `cargo test` builds the rest for. The crate counts whether or not
275
+ `cargo test` compiles it (`cargo test -p host` does not) and whether or not
276
+ a test started its Worker, so every run of the workspace has one
277
+ denominator. Where no test of a run started a Worker, its files say so, in
278
+ `gaps` and the file view, rather than reading as code the tests missed.
267
279
 
268
280
  `wrangler dev --remote` runs the Worker on Cloudflare's edge, and wrangler
269
281
  opens no inspector into it there, so nothing can read what it ran. A test
@@ -271,6 +283,31 @@ that starts it that way runs as usual; the Worker's files then say they ran
271
283
  remotely and could not be read. Start the Worker with `wrangler dev` to
272
284
  measure it.
273
285
 
286
+ Code in a Durable Object (`#[durable_object]` and its `impl DurableObject`)
287
+ is measured like the rest of the Worker and credited to the test whose
288
+ requests reached it. So is a Worker that runs beside another in one
289
+ `wrangler dev` (`-c plane/wrangler.toml -c runners/wrangler.toml`), reached
290
+ only through the first's service bindings: wrangler's inspector shows one
291
+ Worker of a session, and Supercov reads each through workerd's own, whose
292
+ address a small preload in wrangler's Node process reports. A test that
293
+ sets its own `NODE_OPTIONS` keeps them; the preload is added after. A workspace with several Worker crates counts each crate
294
+ once, and runs of different tests -- each starting different Workers, or
295
+ none -- merge with `supercov merge`. Tests that run a Worker only when its `build/` exists, as a test
296
+ kit that will not start an unbuilt Worker does, run it: a `build/` you made
297
+ beforehand and keep out of git is in the copy, and `wrangler dev` builds the
298
+ crate with probes over it there.
299
+
300
+ What a project needs:
301
+
302
+ - `wrangler` on `PATH` in the job that runs Supercov, before the run starts,
303
+ such as `node_modules/.bin` of a project that installs it.
304
+ - The `wasm32-unknown-unknown` target and `worker-build`, which the Worker's
305
+ `[build]` command runs.
306
+ - The Worker crates inside the source Supercov measures, as workspace members
307
+ or under the project root.
308
+
309
+ The project needs no `cfg` flag or build step of its own for Supercov.
310
+
274
311
  The Worker built with probes stays in the copy: its `build/` output is never
275
312
  returned to the project, and neither is the copy's configuration, which
276
313
  points `main` at Supercov's entry. wrangler's `.wrangler/` state is not a
@@ -424,8 +461,13 @@ Ruby 3.3 and newer support statement, method, branch and MC/DC measurement,
424
461
  including loops, iterator blocks, short-circuit operators, pattern matching,
425
462
  optional calls and exception paths.
426
463
 
427
- Some constructs have narrower coverage. Code in a non-main Ractor keeps line
428
- coverage but may lack other observations. Certain nested-return expressions
464
+ Some constructs have narrower coverage. Code written inside a `Ractor.new`
465
+ block keeps line coverage but may lack other observations. Code a non-main
466
+ Ractor calls elsewhere runs as it would without Supercov, but what it runs
467
+ there is not recorded: the run says so, and a line only a Ractor reached reads
468
+ as not executed. A project whose code or tests mention `Ractor` gets probes a
469
+ Ractor can call, which cost more than the usual ones (about a fifth more in a
470
+ tight loop). Certain nested-return expressions
429
471
  limit normal-completion measurement. Constant predicates are folded as Ruby
430
472
  folds them, so unreachable alternatives do not become obligations.
431
473
 
@@ -508,8 +550,10 @@ beyond one deferred line per test that binds it to its evidence.
508
550
 
509
551
  `go test` builds one binary per package, so each test package records its own
510
552
  evidence and Supercov merges them. It also caches packages that passed, and a
511
- cached package does not run — so Supercov adds `-count=1` unless your command
512
- already says otherwise, and tells you it did.
553
+ cached package does not run — so Supercov adds `-count=1` to `GOFLAGS` unless
554
+ your command or `GOFLAGS` already says otherwise, and tells you it did. Through
555
+ `GOFLAGS` it reaches `go test` however your command starts it, `make test`
556
+ included.
513
557
 
514
558
  A test that calls `t.Parallel()` runs alongside others. Probes are a store into
515
559
  one array shared by the process, so nothing can say which of two concurrent
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "supercov",
3
- "version": "4.19.0",
3
+ "version": "4.20.0",
4
4
  "description": "Coverage, security and code quality for coding agents",
5
5
  "keywords": [
6
6
  "coverage",
@@ -95,7 +95,9 @@
95
95
  "test:test262": "cargo build --release -p supercov && node scripts/test262-equivalence.mjs",
96
96
  "benchmark:check": "cargo build --release -p supercov && node scripts/rust-transform-benchmark.mjs",
97
97
  "benchmark:python": "cargo build -p supercov && node scripts/python-benchmark.mjs",
98
- "check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:release-tooling && npm run test:runtime && npm run test:action && npm run test:rust-assets && npm run test:agent-commands && npm run test:assertions && npm run test:sweep-target && npm run test:skip-guard && npm run test:windows-tls && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
98
+ "check": "npm run check:lint && npm run check:tests",
99
+ "check:lint": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings",
100
+ "check:tests": "npm run test && npm run test:release-tooling && npm run test:runtime && npm run test:action && npm run test:rust-assets && npm run test:agent-commands && npm run test:assertions && npm run test:sweep-target && npm run test:skip-guard && npm run test:windows-tls && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
99
101
  "release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:go && npm run test:jvm && npm run test:launcher && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
100
102
  "release:bump": "node scripts/bump-version.mjs",
101
103
  "sweep": "node scripts/sweep-target.mjs",
@@ -119,14 +121,14 @@
119
121
  "test:windows-tls": "node scripts/windows-tls-test.mjs"
120
122
  },
121
123
  "optionalDependencies": {
122
- "@supercov/cli-darwin-arm64": "4.19.0",
123
- "@supercov/cli-darwin-x64": "4.19.0",
124
- "@supercov/cli-linux-arm64-gnu": "4.19.0",
125
- "@supercov/cli-linux-arm64-musl": "4.19.0",
126
- "@supercov/cli-linux-x64-gnu": "4.19.0",
127
- "@supercov/cli-linux-x64-musl": "4.19.0",
128
- "@supercov/cli-win32-arm64": "4.19.0",
129
- "@supercov/cli-win32-x64": "4.19.0"
124
+ "@supercov/cli-darwin-arm64": "4.20.0",
125
+ "@supercov/cli-darwin-x64": "4.20.0",
126
+ "@supercov/cli-linux-arm64-gnu": "4.20.0",
127
+ "@supercov/cli-linux-arm64-musl": "4.20.0",
128
+ "@supercov/cli-linux-x64-gnu": "4.20.0",
129
+ "@supercov/cli-linux-x64-musl": "4.20.0",
130
+ "@supercov/cli-win32-arm64": "4.20.0",
131
+ "@supercov/cli-win32-x64": "4.20.0"
130
132
  },
131
133
  "peerDependencies": {
132
134
  "@playwright/test": ">=1.55.0",
@@ -979,22 +979,21 @@ function appendServer(record) {
979
979
  var _a8, _b, _c, _d;
980
980
  if (state.runtimeSnapshots) {
981
981
  const timestampMs = (_a8 = record.timestampMs) != null ? _a8 : Date.now();
982
- const statement = record.statementId ? { statementId: record.statementId } : {};
983
982
  if (record.type === "decision") {
984
- recordBrowserEvent(__spreadProps(__spreadValues(__spreadValues({
983
+ recordBrowserEvent(__spreadProps(__spreadValues({
985
984
  type: "decision",
986
985
  id: record.meta.id,
987
986
  vector: record.vector,
988
987
  timestampMs
989
- }, record.phaseId ? { phaseId: record.phaseId } : {}), statement), {
988
+ }, record.phaseId ? { phaseId: record.phaseId } : {}), {
990
989
  environment: "server"
991
990
  }));
992
991
  } else {
993
- recordBrowserEvent(__spreadProps(__spreadValues(__spreadValues({
992
+ recordBrowserEvent(__spreadProps(__spreadValues({
994
993
  type: "hit",
995
994
  id: record.id,
996
995
  timestampMs
997
- }, record.phaseId ? { phaseId: record.phaseId } : {}), statement), {
996
+ }, record.phaseId ? { phaseId: record.phaseId } : {}), {
998
997
  environment: "server"
999
998
  }));
1000
999
  }