supercov 4.4.0 → 4.6.0

package/README.md

@@ -344,6 +344,29 @@ npx supercov report
 
 Collections accept `--limit` and `--offset` and print a copyable next-page command. Machine-readable output is available with `--json` when an integration needs it.
 
+## Comment on pull requests
+
+In GitHub Actions, one step runs your tests and tells the pull request which
+changed lines no test ran:
+
+```yaml
+permissions:
+  contents: read
+  pull-requests: write
+
+steps:
+  - uses: actions/checkout@v6
+  - run: npm ci
+  - uses: supercorp-ai/supercov@v4
+    with:
+      command: npm test
+```
+
+It works with every supported language, keeps one comment up to date, marks the
+lines in the diff, and can fail the job below a floor (`min-changed-lines: 90`).
+No account, no upload token and no server: the report never leaves the runner.
+See [Comment on a pull request](docs/cli.md#comment-on-a-pull-request).
+
 ## Local, private, and zero-edit
 
 Everything Supercov writes lives under one hidden `.supercov/` directory: run evidence in `.supercov/runs/<run-id>/` and the isolated build cache in `.supercov/workspaces/`. It ignores itself in Git, so there is nothing to add to your `.gitignore`.

package/docs/cli.md

@@ -279,6 +279,62 @@ code nothing ran.
 adjacent misses into one range and capping the total (`--max-annotations`). It
 needs no token and posts no comment.
 
+## Comment on a pull request
+
+In GitHub Actions, one step runs the tests, says in a comment which changed
+lines no test ran, and marks them in the diff:
+
+```yaml
+on: pull_request
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+      - run: npm ci
+      - uses: supercorp-ai/supercov@v4
+        with:
+          command: npm test
+```
+
+`command` is the test command in any supported language: `pytest`,
+`cargo test`, `go test ./...`, `bundle exec rspec`, `mvn test`. The step takes
+the place of the one that ran the tests, and fails when they fail. It needs no
+account and no server: the comment is written with the workflow's own token,
+and everything else stays on the runner.
+
+The comment gives the coverage of the changed lines, the run's line, branch and
+MC/DC coverage, and each changed file with the lines no test ran, linked to the
+commit. A later push rewrites the same comment. The job summary holds the same
+report, which is what a pull request from a fork gets: its token cannot write a
+comment.
+
+| Input | Default | |
+| --- | --- | --- |
+| `command` | required | The test command. One command; put several in a script. |
+| `working-directory` | `.` | Where to run it. |
+| `min-changed-lines` | none | Fail below this percentage of changed lines covered. |
+| `min-lines`, `min-branches`, `min-mcdc` | none | Fail below this percentage for the whole run. |
+| `title` | `Coverage` | The comment's heading. Give each job its own when several comment on one pull request. |
+| `base` | the pull request's base | What the change is compared with. |
+| `comment`, `annotate` | `true` | Turn the comment or the marks in the diff off. |
+| `version` | the action's release | The Supercov release to run. |
+| `binary` | none | A `supercov` already on the runner, used instead of downloading one. |
+
+The default checkout fetches one commit; the step fetches the history between
+it and the base itself. It puts `supercov` on the `PATH` and sets the outputs
+`run-id`, `lines`, `branches` and `changed-lines`, so later steps can read the
+same run:
+
+```yaml
+      - run: supercov runs report --format html --output coverage/report
+```
+
 ## Export for other tools
 
 ```sh supercov-example
@@ -300,7 +356,13 @@ one to fill a field.
 
 MC/DC conditions are not exported as ordinary branches. A consumer would then
 show condition obligations as branch coverage, which is a different
-measurement; that evidence stays in the JSON view and the HTML report. A report
+measurement. The LCOV export writes them as LCOV 2's own records, `MCDC:` for
+each condition with `MCF:` and `MCH:` for a file, so `lcov --mcdc-coverage` and
+`genhtml --mcdc-coverage` show a per-file MC/DC figure from the export alone.
+LCOV counts a condition twice, once for each way it can turn the outcome, and
+Supercov's covered condition is both: hit over found is the run's MC/DC. A
+reader of the older format passes the records over. Cobertura has no such
+record; there the evidence stays in the JSON view and the HTML report. A report
 from a failed or stale run is still written, with a warning on stderr — only
 `check` refuses to pass on one.
 
@@ -449,8 +511,12 @@ pass in the run is listed regardless.
 per line, for a runner's filter. A Vitest, Playwright or node:test test inside
 `describe` blocks is printed as the blocks and the test joined by spaces
 (`small charges`), which is what `vitest -t`, `--grep` and
-`--test-name-pattern` match. A dependency, lockfile, configuration or
-toolchain change affects every test and is reported as such. A source file
+`--test-name-pattern` match. A helper beside the tests that changed affects
+the tests whose files import it (a helper that only gained functions affects
+none), and every test where Supercov cannot read the imports. A dependency,
+lockfile, configuration or toolchain change affects every test and is reported
+as such; a test runner's settings affect only the runs that used that runner.
+A source file
 added since the run is outside every test's record; the working-tree check
 says so, and the suite should run in full.
 
@@ -516,12 +582,71 @@ assessment, each commit cost under a cent.
 
 ```sh supercov
 npx supercov merge <shard-a> <shard-b> <shard-c>
+npx supercov merge
 ```
 
 Merge creates a new run. The inputs must describe the same source,
 configuration, toolchain, schema, and coverage denominator. Supercov rejects an
 incompatible merge rather than publishing a misleading aggregate.
 
+With no run named, `merge` takes the runs recorded on the source as it is now:
+the newest of each command, leaving out runs that are themselves a merge. It
+prints the runs it took. This is what a script uses after measuring its
+suites, one after another or side by side, without knowing what each run was
+called. Asked again with nothing new, it names the merge already made.
+
+A test that is in two of the runs is one test in the merge. If it failed in
+either, it is failed, which is why a command run again replaces its earlier
+run and is not merged with it.
+
+The runs may differ in their tests, the helpers beside them, and the test
+runners' settings, as long as nothing that changed reaches what an earlier
+run's tests did. Measuring a new test alone and adding it to the runs before it
+is then the whole suite's figure, without the whole suite's time:
+
+```sh supercov-example
+npx supercov -- npm test               # the suite, once
+npx supercov -- <the new test alone>   # a test added since
+npx supercov merge
+```
+
+Each file that differs from what an earlier run was recorded with is judged by
+what that run's tests did with it:
+
+- Another run's test file: it was not in this run. A file of the run's own
+  tests: what it recorded is of other tests, and the run is left out.
+- A helper, fixture or support file: left out of a run only when that run's
+  test files import it, directly or through other files. For JavaScript and
+  TypeScript tests Supercov reads the imports; a computed import path, or tests
+  in another language, count every helper as loaded. A helper that only gained
+  functions, which code written before cannot have called, leaves every run
+  in.
+- A test runner's settings: `playwright.config.*`, `vitest.config.*`,
+  `jest.config.*` and `ava.config.*` belong to their runner, and a run that
+  did not use that runner keeps. Of Playwright's settings, only those that
+  decide what a test does count: `use`, `testDir`, `webServer`,
+  `globalSetup`, `globalTeardown`, and a project's `dependencies` and
+  `teardown`. Timeouts, retries, workers, reporters, which files and names to
+  run, and projects added since schedule tests and change nothing an earlier
+  test reached. This needs the newer run to have measured Playwright.
+- Any other configuration, such as `tsconfig.json`, `vite.config.*` or a
+  Babel file, may set how any test runs, and leaves the run out.
+
+A run left out is named with the files and the functions that changed. The
+merged run lists what differed and why it does not reach the earlier runs:
+`merged across:` in `runs <id>`, and `mergedAcross` in JSON.
+
+When you know a change Supercov counts does not alter what the earlier tests
+ran, say so:
+
+```sh supercov-example
+npx supercov merge --allow-changed
+```
+
+The product code, the dependencies, and what is measured must still be the
+same, and a run whose own test files changed is still left out. What was
+accepted is listed in `merged across:` as accepted.
+
 `assertions assess` assesses a merged run across every suite merged into it,
 reusing the answers saved when its parts were assessed. `tests affected` is
 asked of the runs that were merged, which hold the record of what each test

package/docs/workspace-isolation.md

@@ -114,12 +114,56 @@ partial output.
 
 ## Interrupted and overlapping commands
 
-One project can run one coverage or cleanup transaction at a time. A second
-operation fails clearly instead of racing the first.
+Runs of one project can be started at the same moment. Each takes a copy of
+the project to itself: a run alone always has the first,
+`.supercov/workspaces/workspace/<name>`, and one started while that is in use
+has `workspace-2`, the next `workspace-3`. None waits and none is refused:
+
+```sh
+npx supercov -- npm run test:unit &
+npx supercov -- npm run test:integration &
+npx supercov -- npm run test:e2e &
+wait
+npx supercov merge
+```
+
+`merge` with no run named adds up the runs recorded on the source as it is
+now, so a script does not have to know what each was called.
+
+A copy past the first is kept for a day after its last run, so that runs that
+overlap again start from what they left, and is removed after that. Rust runs
+build into one directory, as two `cargo test` do, and Cargo makes the second
+wait while the first compiles.
+
+Suites can also be started from one command. They are then one run with
+nothing to merge, and each test keeps the kind its own process was given:
+
+```sh
+npx supercov -- sh -c '
+  SUPERCOV_TEST_KIND=integration npm run test:integration & integration=$!
+  SUPERCOV_TEST_KIND=e2e npm run test:e2e & e2e=$!
+  wait $integration; first=$?
+  wait $e2e && exit $first
+'
+```
+
+`wait` with no argument ends with 0 whatever the suites did, which would turn a
+failing suite green; waiting for each by its id keeps the status.
+
+`runs clean` wants the store to itself. While a run is active it ends with
+status 2 and names the run; a run started while it works is told so, and no
+test of it runs.
 
 After interruption or host restart, the next command recovers unpublished
 staging state. Completed runs remain immutable.
 
+Ctrl-C ends the test command and the run: nothing is recorded, and the exit
+status is 130. A test command that reads the keyboard (a debugger at a
+breakpoint, a watch mode, a question before an install) is given the terminal
+when it first reads, on macOS and Linux. From then on Ctrl-C and Ctrl-Z go to
+it, as they do without Supercov, and a command that Ctrl-C ended is still a
+run that was interrupted.
+
 ## Clean up local data
 
 Preview cleanup before removing anything:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "supercov",
-  "version": "4.4.0",
+  "version": "4.6.0",
   "description": "Coverage, security and code quality for coding agents",
   "keywords": [
     "coverage",
@@ -70,6 +70,7 @@
     "test:react": "cargo build -p supercov && node scripts/rust-vitest-projects-integration.mjs && npm --prefix examples/react-verification ci && npm --prefix examples/react-verification run demo && node scripts/react-hydration-compatibility.mjs",
     "test:release-tooling": "node --test tests/release/*.test.mjs",
     "test:runtime": "node --test tests/runtime/*.test.mjs",
+    "test:action": "cargo build -p supercov && node --test tests/action/*.test.mjs",
     "test:fixture": "cargo build -p supercov && node scripts/rust-fixture-matrix.mjs",
     "test:packed-npx": "cargo build --release -p supercov && node scripts/packed-npx-integration.mjs",
     "test:isolation": "cargo build -p supercov && node scripts/isolation-integration.mjs",
@@ -82,7 +83,7 @@
     "eval:skill": "claude plugin eval plugins/supercov --scaffold --trust-plugin --no-publish --model claude-sonnet-5 --judge-model claude-haiku-4-5 --threshold 0.66 --tag behavior trigger --allow-tools Bash Edit Write --max-cost-usd 15",
     "eval:skill-triggers": "claude plugin eval plugins/supercov --scaffold --trust-plugin --no-publish --model claude-sonnet-5 --judge-model claude-haiku-4-5 --threshold 0.66 --tag trigger --ablation none --max-cost-usd 5",
     "test:engine": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace && npm run test:runtime && npm run test:rust-assets && npm run test:engine-integrations",
-    "test:engine-integrations": "cargo build -p supercov && node scripts/html-report-integration.mjs && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/mocha-failure-output.mjs && node scripts/failure-output.mjs node ava tap && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/rust-classic-browser-integration.mjs && node scripts/rust-top-packages-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
+    "test:engine-integrations": "cargo build -p supercov && node scripts/html-report-integration.mjs && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/mocha-failure-output.mjs && node scripts/failure-output.mjs node ava tap && node scripts/side-by-side.mjs node && node scripts/merge-across.mjs && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/rust-classic-browser-integration.mjs && node scripts/rust-top-packages-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
     "test:platform": "cargo test --workspace && cargo build -p supercov && node scripts/rust-process-supervision.mjs && node scripts/workspace-crash-integration.mjs",
     "test:native-package": "cargo build --release -p supercov && node scripts/native-package-integration.mjs && node scripts/native-release-set-integration.mjs",
     "test:pypi-wheel": "node scripts/pypi-wheel-integration.mjs",
@@ -95,7 +96,7 @@
     "test:test262": "cargo build --release -p supercov && node scripts/test262-equivalence.mjs",
     "benchmark:check": "cargo build --release -p supercov && node scripts/rust-transform-benchmark.mjs",
     "benchmark:python": "cargo build -p supercov && node scripts/python-benchmark.mjs",
-    "check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:release-tooling && npm run test:runtime && npm run test:rust-assets && npm run test:agent-commands && npm run test:assertions && npm run test:sweep-target && npm run test:skip-guard && npm run test:windows-tls && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
+    "check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:release-tooling && npm run test:runtime && npm run test:action && npm run test:rust-assets && npm run test:agent-commands && npm run test:assertions && npm run test:sweep-target && npm run test:skip-guard && npm run test:windows-tls && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
     "release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:go && npm run test:jvm && npm run test:launcher && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
     "release:bump": "node scripts/bump-version.mjs",
     "sweep": "node scripts/sweep-target.mjs",
@@ -104,11 +105,11 @@
     "prepublishOnly": "npm run release:check",
     "test:child-attribution": "cargo build -p supercov && node scripts/rust-child-attribution-integration.mjs",
     "test:host-loader": "cargo build -p supercov && node scripts/rust-host-loader-integration.mjs",
-    "test:python": "cargo build -p supercov && node scripts/python-integration.mjs && node scripts/failure-output.mjs python",
-    "test:ruby-coverage": "cargo build -p supercov && node scripts/ruby-coverage-integration.mjs && node scripts/failure-output.mjs ruby",
-    "test:go": "cargo test --no-fail-fast -p supercov-engine --test go_frontend --test go_run && cargo build -p supercov && node scripts/failure-output.mjs go",
-    "test:jvm": "cargo test --no-fail-fast -p supercov-engine --test jvm_frontend --test jvm_run -- --include-ignored --test-threads=1",
-    "test:rust-public-cargo": "cargo build -p supercov && node scripts/rust-public-cargo-integration.mjs && node scripts/failure-output.mjs rust",
+    "test:python": "cargo build -p supercov && node scripts/python-integration.mjs && node scripts/failure-output.mjs python && node scripts/side-by-side.mjs python",
+    "test:ruby-coverage": "cargo build -p supercov && node scripts/ruby-coverage-integration.mjs && node scripts/failure-output.mjs ruby && node scripts/side-by-side.mjs ruby",
+    "test:go": "cargo test --no-fail-fast -p supercov-engine --test go_frontend --test go_run && cargo build -p supercov && node scripts/failure-output.mjs go && node scripts/side-by-side.mjs go",
+    "test:jvm": "cargo test --no-fail-fast -p supercov-engine --test jvm_frontend --test jvm_run -- --include-ignored --test-threads=1 && cargo build -p supercov && node scripts/side-by-side.mjs jvm",
+    "test:rust-public-cargo": "cargo build -p supercov && node scripts/rust-public-cargo-integration.mjs && node scripts/failure-output.mjs rust && node scripts/side-by-side.mjs rust",
     "test:assertions": "cargo build -p supercov && node scripts/assertions-integration.mjs",
     "docs:sync": "node scripts/sync-docs.mjs",
     "docs:check": "node scripts/sync-docs.mjs --check",
@@ -119,14 +120,14 @@
     "test:windows-tls": "node scripts/windows-tls-test.mjs"
   },
   "optionalDependencies": {
-    "@supercov/cli-darwin-arm64": "4.4.0",
-    "@supercov/cli-darwin-x64": "4.4.0",
-    "@supercov/cli-linux-arm64-gnu": "4.4.0",
-    "@supercov/cli-linux-arm64-musl": "4.4.0",
-    "@supercov/cli-linux-x64-gnu": "4.4.0",
-    "@supercov/cli-linux-x64-musl": "4.4.0",
-    "@supercov/cli-win32-arm64": "4.4.0",
-    "@supercov/cli-win32-x64": "4.4.0"
+    "@supercov/cli-darwin-arm64": "4.6.0",
+    "@supercov/cli-darwin-x64": "4.6.0",
+    "@supercov/cli-linux-arm64-gnu": "4.6.0",
+    "@supercov/cli-linux-arm64-musl": "4.6.0",
+    "@supercov/cli-linux-x64-gnu": "4.6.0",
+    "@supercov/cli-linux-x64-musl": "4.6.0",
+    "@supercov/cli-win32-arm64": "4.6.0",
+    "@supercov/cli-win32-x64": "4.6.0"
   },
   "peerDependencies": {
     "@playwright/test": ">=1.55.0",

package/runtime/javascript/register.mjs

@@ -280,9 +280,10 @@ process.__SUPERCOV_FAILURE_OUTPUT__ ??= function supercovFailureOutput(runner) {
             return [copy];
         }
     })();
-    // The copy lies in the project at .supercov/workspaces/workspace/<name>.
+    // The copy lies in the project at .supercov/workspaces/workspace/<name>,
+    // or at workspace-2 where it is one of several runs at once.
     const projects = copies
-        .filter((copy) => /[\\/]\.supercov[\\/]workspaces[\\/]workspace[\\/][^\\/]+[\\/]$/.test(copy))
+        .filter((copy) => /[\\/]\.supercov[\\/]workspaces[\\/]workspace(?:-\d+)?[\\/][^\\/]+[\\/]$/.test(copy))
         .map((copy) => resolve(copy, "../../../..") + sep);
     // The path inside the copy of a rewritten file that lies under one of `roots`.
     const rewrittenUnder = (roots, file) => {