supercov 0.0.47 → 0.0.49
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -2
- package/docs/assertion-maps.md +27 -8
- package/docs/cli.md +111 -1
- package/docs/getting-started.md +3 -1
- package/docs/supported-suites.md +131 -1
- package/docs/troubleshooting.md +27 -0
- package/package.json +12 -10
package/README.md
CHANGED
|
@@ -152,11 +152,14 @@ cargo binstall supercov && supercov -- cargo test # crates.io
|
|
|
152
152
|
| Rust | Available | `npx supercov -- cargo test` |
|
|
153
153
|
| Python | Available | `npx supercov -- pytest` |
|
|
154
154
|
| Ruby | Available | `npx supercov -- rspec` |
|
|
155
|
+
| Go | Available | `npx supercov -- go test ./...` |
|
|
156
|
+
| Java | Available | `npx supercov -- mvn test` |
|
|
157
|
+
| Kotlin | Available | `npx supercov -- ./gradlew test` |
|
|
155
158
|
| Zig | Coming soon | — |
|
|
156
159
|
| PHP | Coming soon | — |
|
|
157
160
|
| C | Coming soon | — |
|
|
158
161
|
|
|
159
|
-
Supercov requires Node.js 22 or newer. Rust support currently uses Rust 1.95; cargo-nextest 0.9.138 and 0.9.140 are supported. Python support requires CPython 3.12 or newer and measures pytest and unittest runs. Ruby support requires Ruby 3.3 or newer (3.4 or newer for full measurement) and measures RSpec, Minitest, test-unit and Cucumber runs.
|
|
162
|
+
Supercov requires Node.js 22 or newer. Rust support currently uses Rust 1.95; cargo-nextest 0.9.138 and 0.9.140 are supported. Python support requires CPython 3.12 or newer and measures pytest and unittest runs. Ruby support requires Ruby 3.3 or newer (3.4 or newer for full measurement) and measures RSpec, Minitest, test-unit and Cucumber runs. Go support requires Go 1.22 or newer and measures `go test`. Java and Kotlin support requires a JDK 17 or newer with Maven or Gradle, and measures every JUnit Platform engine -- JUnit 5, JUnit 4 through Vintage, Kotest and Spock -- as well as TestNG.
|
|
160
163
|
|
|
161
164
|
## Supported operating systems and architectures
|
|
162
165
|
|
|
@@ -168,7 +171,7 @@ The CLI is a single native binary. `npx supercov` selects the build for the mach
|
|
|
168
171
|
| Linux | arm64, x64 | glibc 2.28 or newer (Debian 10, Ubuntu 18.10, RHEL 8 and later), or musl (Alpine); chosen automatically |
|
|
169
172
|
| Windows | arm64, x64 | Windows 10 or newer |
|
|
170
173
|
|
|
171
|
-
JavaScript, TypeScript, Python, Ruby and
|
|
174
|
+
JavaScript, TypeScript, Python, Ruby, Rust, Go, Java and Kotlin suites are verified on Linux, macOS and Windows -- CPython 3.12, 3.13 and 3.14, Ruby 3.3, 3.4 and 4.0, and Go 1.22 and 1.25 on every one, and JDK 17 and 21 on every one that has them (Temurin publishes no 17 for arm64 Windows, so that pairing is 21 alone). Containers, VMs, and remote executors run the same Linux builds. Every release publishes the same binaries to npm, PyPI (`supercov-cli`, a wheel per platform) and RubyGems (`supercov`, a gem per platform except arm64 Windows, which Ruby has no platform for), and the source to crates.io (`supercov`).
|
|
172
175
|
|
|
173
176
|
## Supported test suites
|
|
174
177
|
|
package/docs/assertion-maps.md
CHANGED
|
@@ -70,7 +70,7 @@ examined and acknowledged.
|
|
|
70
70
|
| `appliesTo` | Select tests by project-relative file and exact displayed test name. |
|
|
71
71
|
| `nodes`, `edges` | Record source locations and relationships ending at `$assertion`. |
|
|
72
72
|
| `countsAsAsserted` | List the node IDs you judge to be checked by the assertion. |
|
|
73
|
-
| `watch` | List additional files the explanation depends on, such as helpers or configuration. |
|
|
73
|
+
| `watch` | List additional files the explanation depends on, such as helpers or configuration. Manifests, lockfiles and runner configuration are already tracked for the whole run; naming one here catches nothing, and the report says so. |
|
|
74
74
|
| `questions` | Record unresolved investigation questions. Questions inside a flow block its credit. |
|
|
75
75
|
|
|
76
76
|
Source anchors use project-relative paths with `/`, one-based lines and one-based
|
|
@@ -108,13 +108,32 @@ token stale.
|
|
|
108
108
|
### What makes a review token stale
|
|
109
109
|
|
|
110
110
|
A flow needs a fresh review when its claim changes, when a file it watches or a
|
|
111
|
-
test it selects changes, or when the
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
111
|
+
test it selects changes, or when the configuration that decides what executes
|
|
112
|
+
changes: a transpiler, a test runner, an interpreter pin.
|
|
113
|
+
|
|
114
|
+
Several things that sound like they should count do not, because an
|
|
115
|
+
acknowledgement demanded for all of them at once stops being read.
|
|
116
|
+
|
|
117
|
+
Cutting a release does not. A manifest is fingerprinted by what it declares, so
|
|
118
|
+
a version number moving in `package.json`, `Cargo.toml`, `pyproject.toml` or a
|
|
119
|
+
lockfile changes nothing. Neither does reformatting one.
|
|
120
|
+
|
|
121
|
+
Upgrading Supercov does not. Your claims are about your code, and a new release
|
|
122
|
+
re-derives the evidence they rest on rather than making them wrong. Only a
|
|
123
|
+
deliberate change to the instrumenter contract counts.
|
|
124
|
+
|
|
125
|
+
Upgrading a dependency does not make every flow stale either. It is recorded
|
|
126
|
+
once, as a change to assess, and flows keep their credit until that assessment
|
|
127
|
+
says otherwise. One explanation answers for the upgrade.
|
|
128
|
+
|
|
129
|
+
Linters, formatters, type checkers and coverage settings never count, because
|
|
130
|
+
none of them change what the code does when it runs.
|
|
131
|
+
|
|
132
|
+
The ambient environment does not count: running from another directory, a new
|
|
133
|
+
terminal session, a different package manager or another Node installation
|
|
134
|
+
leaves current flows current. A behavioural difference that matters still shows
|
|
135
|
+
up on its own, because credit requires a passing assertion occurrence and
|
|
136
|
+
execution of the claimed statement in the same selected test.
|
|
118
137
|
|
|
119
138
|
If your suite genuinely depends on particular variables, name them; only the
|
|
120
139
|
ones you name participate, and an unset variable is recorded as absent.
|
package/docs/cli.md
CHANGED
|
@@ -132,6 +132,115 @@ report. Follow the printed next-page command or JSON `pagination.nextOffset`.
|
|
|
132
132
|
Validation supports `--view flows`, `--view changes` and `--view errors` for large
|
|
133
133
|
maps. The [map reference](assertion-maps.md) describes all fields and gates.
|
|
134
134
|
|
|
135
|
+
## Fail CI below a coverage floor
|
|
136
|
+
|
|
137
|
+
```sh supercov-example
|
|
138
|
+
supercov runs check --min-lines 90 --min-branches 80 --min-mcdc 80
|
|
139
|
+
supercov runs check --min-lines 100 --per-file --json
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
`check` reads a recorded run; it never runs tests again. Give a floor per metric
|
|
143
|
+
with `--min-lines`, `--min-statements`, `--min-functions`, `--min-branches` or
|
|
144
|
+
`--min-mcdc`. `--per-file` applies the same floors to every file that has
|
|
145
|
+
eligible obligations, in addition to the whole run. Both report the counts
|
|
146
|
+
behind the percentage and, for lines, where the gaps are.
|
|
147
|
+
|
|
148
|
+
Floors are compared against the counts, never a rounded percentage: 9,999
|
|
149
|
+
covered lines out of 10,000 displays as 99.99% and fails a 100% floor, and a
|
|
150
|
+
run that displayed `100.00%` could never pass one while something is uncovered.
|
|
151
|
+
|
|
152
|
+
A check answers only when the run can answer. These end the command with `2`
|
|
153
|
+
rather than a pass or a failure:
|
|
154
|
+
|
|
155
|
+
- the wrapped test command did not pass, so a gate over it would turn a red CI
|
|
156
|
+
run green
|
|
157
|
+
- the run no longer matches the current checkout
|
|
158
|
+
- a requested metric has nothing eligible, which is not the same as complete
|
|
159
|
+
- a requested metric left obligations unmeasured, so no exact judgement exists
|
|
160
|
+
- a requested metric is not recorded by the language adapter
|
|
161
|
+
|
|
162
|
+
Assertion coverage keeps its own check. Whether a test *examines* what it
|
|
163
|
+
executes is a different question from whether a line ran, and
|
|
164
|
+
`runs <id> assertions check` carries the freshness and acknowledgement rules
|
|
165
|
+
that answer needs.
|
|
166
|
+
|
|
167
|
+
## Check the lines a change touches
|
|
168
|
+
|
|
169
|
+
```sh supercov-example
|
|
170
|
+
supercov runs patch --base origin/main --min-lines 100
|
|
171
|
+
supercov runs patch --base origin/main --annotate github
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
`patch` answers whether the lines this change added or modified are tested. It
|
|
175
|
+
compares against the **merge base** with `--base`, not that branch's tip, so
|
|
176
|
+
commits other people landed after you branched are not counted as your
|
|
177
|
+
obligation. A shallow checkout has no merge base; fetch with full history
|
|
178
|
+
(`actions/checkout` takes `fetch-depth: 0`).
|
|
179
|
+
|
|
180
|
+
The denominator is the changed lines the run measured. Comments, blank lines and
|
|
181
|
+
declarations fall out because the language adapter already decided they are not
|
|
182
|
+
executable, not because `patch` guesses at syntax. Deleted lines are excluded:
|
|
183
|
+
there is nothing left to cover. Untracked new source counts as entirely added.
|
|
184
|
+
|
|
185
|
+
A change with nothing executable in it reports **No executable changes** and
|
|
186
|
+
passes, rather than claiming 100% for a patch that changed only comments. A
|
|
187
|
+
changed file that looks like product source but is absent from the run is named
|
|
188
|
+
separately, because treating it as zero uncovered lines would report success for
|
|
189
|
+
code nothing ran.
|
|
190
|
+
|
|
191
|
+
`--annotate github` prints workflow-command annotations on stdout, combining
|
|
192
|
+
adjacent misses into one range and capping the total (`--max-annotations`). It
|
|
193
|
+
needs no token and posts no comment.
|
|
194
|
+
|
|
195
|
+
## Export for other tools
|
|
196
|
+
|
|
197
|
+
```sh supercov-example
|
|
198
|
+
supercov runs report --format lcov --output coverage/lcov.info
|
|
199
|
+
supercov runs report --format cobertura --output coverage/cobertura.xml
|
|
200
|
+
supercov runs report --format html --output coverage/report
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
Both are written from the same view `check` and `patch` read, so a viewer,
|
|
204
|
+
hosted service or CI integration sees the totals Supercov enforced. Paths are
|
|
205
|
+
repository-relative with forward slashes, ordering is stable, and the file is
|
|
206
|
+
written atomically; an existing file is kept unless you pass `--force`. Without
|
|
207
|
+
`--output` the report goes to stdout and diagnostics to stderr, so a redirect
|
|
208
|
+
captures only the report.
|
|
209
|
+
|
|
210
|
+
Supercov records that a line ran, not how many times, so `DA:` and `hits` state
|
|
211
|
+
`1` or `0`. They are not execution frequencies, and Supercov will not invent
|
|
212
|
+
one to fill a field.
|
|
213
|
+
|
|
214
|
+
MC/DC conditions are not exported as ordinary branches. A consumer would then
|
|
215
|
+
show condition obligations as branch coverage, which is a different
|
|
216
|
+
measurement; that evidence stays in the JSON view and the HTML report. A report
|
|
217
|
+
from a failed or stale run is still written, with a warning on stderr — only
|
|
218
|
+
`check` refuses to pass on one.
|
|
219
|
+
|
|
220
|
+
### The HTML report
|
|
221
|
+
|
|
222
|
+
`--format html` writes one self-contained document. It opens from a copied CI
|
|
223
|
+
artifact with no server, no network and no login, and nothing is fetched from a
|
|
224
|
+
CDN. Because a source path is never used as an output path, a filename cannot
|
|
225
|
+
write outside the directory you named, and no directory is ever cleared to
|
|
226
|
+
regenerate a report.
|
|
227
|
+
|
|
228
|
+
It has three levels: the run's metric counts, a filterable and sortable file
|
|
229
|
+
table, and a source view marking each line covered or not covered in words and
|
|
230
|
+
a glyph as well as colour. Every line links as `#<file>:<line>`, so a CI summary
|
|
231
|
+
can point someone at the obligation rather than at the report.
|
|
232
|
+
|
|
233
|
+
Four states stay distinct, because collapsing them into one score is how a
|
|
234
|
+
report starts to mislead: **uncovered** (measured, nothing reached it), **not
|
|
235
|
+
applicable** (nothing eligible), **partly measured** (Supercov declined some
|
|
236
|
+
obligations, which are excluded from every count) and **stale** (the run no
|
|
237
|
+
longer matches the checkout). A failed suite says so beside its numbers.
|
|
238
|
+
|
|
239
|
+
Source text is embedded only when the run still matches the checkout; otherwise
|
|
240
|
+
the report shows line numbers and explains why. Embedding makes a report
|
|
241
|
+
portable and also means it contains your code — worth knowing before uploading
|
|
242
|
+
one as a public artifact.
|
|
243
|
+
|
|
135
244
|
## Narrow a view
|
|
136
245
|
|
|
137
246
|
| Option | Meaning |
|
|
@@ -233,4 +342,5 @@ SUPERCOV_TEST_KIND=e2e npx supercov -- npx playwright test
|
|
|
233
342
|
| --- | --- |
|
|
234
343
|
| `0` | The command or query succeeded |
|
|
235
344
|
| Wrapped command's code | The test command failed and Supercov preserved its status |
|
|
236
|
-
| `
|
|
345
|
+
| `1` | A valid measurement failed a policy you set, such as a coverage floor |
|
|
346
|
+
| `2` | Supercov could not complete the request, or the evidence cannot answer it |
|
package/docs/getting-started.md
CHANGED
|
@@ -16,7 +16,7 @@ When it finishes, review the test change and coverage comparison in your
|
|
|
16
16
|
conversation. Ask separately if you want a commit or pull request.
|
|
17
17
|
|
|
18
18
|
No account, config file, import, custom reporter, or hosted service is required.
|
|
19
|
-
Supercov supports JavaScript, TypeScript, Rust, Python, and
|
|
19
|
+
Supercov supports JavaScript, TypeScript, Rust, Python, Ruby, Go, Java, and Kotlin today.
|
|
20
20
|
|
|
21
21
|
## What the agent does
|
|
22
22
|
|
|
@@ -108,6 +108,8 @@ if anything is missing.
|
|
|
108
108
|
- a working test command, including its dependencies, environment variables,
|
|
109
109
|
and any local services;
|
|
110
110
|
- for Rust, the Rust 1.95 toolchain;
|
|
111
|
+
- for Go, Go 1.22 or newer;
|
|
112
|
+
- for Java and Kotlin, JDK 17 or newer, with Maven or Gradle;
|
|
111
113
|
- for Python, CPython 3.12 or newer with pytest or unittest;
|
|
112
114
|
- for Ruby, Ruby 3.4 or newer with RSpec, Minitest, test-unit or Cucumber (3.3 measures lines, methods and simple branches only).
|
|
113
115
|
|
package/docs/supported-suites.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Supported languages and test suites
|
|
2
2
|
|
|
3
|
-
Supercov supports JavaScript, TypeScript, Rust, Python, and
|
|
3
|
+
Supercov supports JavaScript, TypeScript, Rust, Python, Ruby, Go, Java, and Kotlin today. Start with
|
|
4
4
|
the same test command the repository already uses; Supercov detects supported
|
|
5
5
|
runners inside that command.
|
|
6
6
|
|
|
@@ -10,6 +10,8 @@ npx supercov -- npx playwright test
|
|
|
10
10
|
npx supercov -- cargo test
|
|
11
11
|
npx supercov -- pytest
|
|
12
12
|
npx supercov -- rspec
|
|
13
|
+
npx supercov -- go test ./...
|
|
14
|
+
npx supercov -- mvn test
|
|
13
15
|
```
|
|
14
16
|
|
|
15
17
|
## Language support
|
|
@@ -21,6 +23,9 @@ npx supercov -- rspec
|
|
|
21
23
|
| Rust | Available | `npx supercov -- cargo test` |
|
|
22
24
|
| Python | Available | `npx supercov -- pytest` |
|
|
23
25
|
| Ruby | Available | `npx supercov -- rspec` |
|
|
26
|
+
| Go | Available | `npx supercov -- go test ./...` |
|
|
27
|
+
| Java | Available | `npx supercov -- mvn test` |
|
|
28
|
+
| Kotlin | Available | `npx supercov -- ./gradlew test` |
|
|
24
29
|
| Zig | Coming soon | — |
|
|
25
30
|
| PHP | Coming soon | — |
|
|
26
31
|
| C | Coming soon | — |
|
|
@@ -211,6 +216,131 @@ npx supercov -- ruby -Itest test/shapes_test.rb
|
|
|
211
216
|
npx supercov -- bin/rails test
|
|
212
217
|
```
|
|
213
218
|
|
|
219
|
+
## Go
|
|
220
|
+
|
|
221
|
+
| Runner | Attribution | Current requirement |
|
|
222
|
+
| --- | --- | --- |
|
|
223
|
+
| `go test` | Exact per test | Go 1.22 or newer |
|
|
224
|
+
| A test that calls `t.Parallel()` | Aggregate: its coverage counts run-wide | — |
|
|
225
|
+
| An `Example` with an `Output` comment, and a `Fuzz` target's seed corpus | Aggregate: measured, but named by no test | — |
|
|
226
|
+
|
|
227
|
+
Supercov instruments an isolated copy of the module and runs your own command
|
|
228
|
+
against it. Your tree is not touched, and your test sources are not rewritten
|
|
229
|
+
beyond one deferred line per test that binds it to its evidence.
|
|
230
|
+
|
|
231
|
+
`go test` builds one binary per package, so each test package records its own
|
|
232
|
+
evidence and Supercov merges them. It also caches packages that passed, and a
|
|
233
|
+
cached package does not run — so Supercov adds `-count=1` unless your command
|
|
234
|
+
already says otherwise, and tells you it did.
|
|
235
|
+
|
|
236
|
+
A test that calls `t.Parallel()` runs alongside others. Probes are a store into
|
|
237
|
+
one array shared by the process, so nothing can say which of two concurrent
|
|
238
|
+
tests reached a line. That coverage is reported against the run rather than
|
|
239
|
+
assigned to a test by guesswork: the lines count as covered, and no test claims
|
|
240
|
+
them. It counts only when the run passed, for the same reason a failing test's
|
|
241
|
+
coverage never counts — a failed run cannot say which of it came from the test
|
|
242
|
+
that failed.
|
|
243
|
+
|
|
244
|
+
An `Example` with an `Output` comment and a `Fuzz` target's seed corpus are
|
|
245
|
+
measured the same way. `go test` runs both, so what they reach is real
|
|
246
|
+
coverage, but neither takes a `*testing.T` for a result to be named by.
|
|
247
|
+
|
|
248
|
+
Supercov also writes evidence as the suite runs, not only at the end. Go offers
|
|
249
|
+
no way to run code on `os.Exit`, and a `TestMain` need not reach the `m.Run()`
|
|
250
|
+
call Supercov wraps — `goleak.VerifyTestMain(m)` runs the suite and exits
|
|
251
|
+
itself. Without periodic writes such a run recorded nothing at all.
|
|
252
|
+
|
|
253
|
+
For assertion maps, `t.Error`, `t.Errorf`, `t.Fatal`, `t.Fatalf` and testify's
|
|
254
|
+
`assert` and `require` are inventoried. A Go test states its claim with an `if`
|
|
255
|
+
and reports the violation, so the report is the site.
|
|
256
|
+
|
|
257
|
+
A repository with several modules works either way it is laid out. A directory
|
|
258
|
+
with a `go.mod` of its own that no `go.work` names is a different module, and
|
|
259
|
+
`go test ./...` walks past it, so Supercov leaves it alone. A `go.work`
|
|
260
|
+
workspace has no module at its root, so each module it names gets a runtime of
|
|
261
|
+
its own.
|
|
262
|
+
|
|
263
|
+
```sh
|
|
264
|
+
npx supercov -- go test ./...
|
|
265
|
+
npx supercov -- go test -run TestParser ./internal/...
|
|
266
|
+
npx supercov -- go test ./core/... ./app/...
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
## Java and Kotlin
|
|
270
|
+
|
|
271
|
+
| Runner | Attribution | Current requirement |
|
|
272
|
+
| --- | --- | --- |
|
|
273
|
+
| JUnit 5 (Jupiter) | Exact per test | JDK 17 or newer, Maven or Gradle |
|
|
274
|
+
| JUnit 4 (through Vintage) | Exact per test | — |
|
|
275
|
+
| Kotest | Exact per test, under the names Kotest itself reports | — |
|
|
276
|
+
| Spock | Exact per feature, under the names Spock itself reports | — |
|
|
277
|
+
| TestNG | Exact per test, each data-provider invocation its own | — |
|
|
278
|
+
| JUnit 4 alone | Exact per test, through Vintage — see below | Maven |
|
|
279
|
+
|
|
280
|
+
Multi-module builds are measured module by module: each compiles its own source
|
|
281
|
+
set and forks its own JVM, so each gets a runtime and records evidence of its
|
|
282
|
+
own, and the run merges them. A build that forks several JVMs to run tests in
|
|
283
|
+
parallel — Gradle's `maxParallelForks`, surefire's `forkCount` — is measured
|
|
284
|
+
the same way: each JVM writes evidence of its own and the run merges every
|
|
285
|
+
one.
|
|
286
|
+
|
|
287
|
+
Attribution comes from the framework's own lifecycle rather than from rewritten
|
|
288
|
+
test sources: a JUnit Platform listener sees every engine built on the platform,
|
|
289
|
+
which is what covers Kotest and Spock, whose tests are not annotated methods any
|
|
290
|
+
rewriter could find. TestNG is not a platform engine and has a listener of its
|
|
291
|
+
own. Tests keep the names their framework chose, so a coverage report and a test
|
|
292
|
+
report name the same thing.
|
|
293
|
+
|
|
294
|
+
JUnit 4 on its own is not a platform engine and does not run on one. Maven and
|
|
295
|
+
Gradle choose a test provider from what is on the classpath, so putting the
|
|
296
|
+
platform there makes the build pick a provider that finds no engine and fail.
|
|
297
|
+
For a Maven module, Supercov adds `junit-vintage-engine` to the copy — the
|
|
298
|
+
platform's own way of running exactly those JUnit 4 tests through the lifecycle
|
|
299
|
+
it listens to — and measures them; your own build still runs JUnit 4 as it did.
|
|
300
|
+
A Gradle module is left alone and told about rather than broken.
|
|
301
|
+
|
|
302
|
+
Supercov instruments an isolated copy and leaves your build file alone. In the
|
|
303
|
+
copy it adds a test-scoped `junit-platform-launcher`, because the listener is
|
|
304
|
+
compiled from the project's test sources and neither Maven nor Gradle puts that
|
|
305
|
+
API on the compile classpath; it disables JUnit's parallel execution, keeping
|
|
306
|
+
whatever else your `junit-platform.properties` set; and it stops the copy
|
|
307
|
+
failing its build on warnings, because the copy holds instrumented code your
|
|
308
|
+
project never wrote a style policy for. Warnings are still reported.
|
|
309
|
+
|
|
310
|
+
If tests do run concurrently anyway, Supercov says so and stops attributing
|
|
311
|
+
rather than reporting numbers nobody can trust: statements and branches still
|
|
312
|
+
count run-wide, and condition coverage is dropped, because concurrent
|
|
313
|
+
evaluations corrupt the state it is computed from.
|
|
314
|
+
|
|
315
|
+
Some conditions are read by the compiler as well as evaluated at runtime, and
|
|
316
|
+
those Supercov leaves exactly as written. `x instanceof String s`, a record
|
|
317
|
+
deconstruction pattern, and Kotlin's `x is String` or `x != null` all narrow a
|
|
318
|
+
type for the code beneath them; wrapping such a condition to observe its
|
|
319
|
+
operands would take the narrowing away and the code would stop compiling. An
|
|
320
|
+
`if` is still measured — which way it went is recorded from inside its arms
|
|
321
|
+
instead — but it carries no condition vectors, so it contributes no MC/DC
|
|
322
|
+
obligation. A loop is a harder case: it has one arm, and no place to record an
|
|
323
|
+
exit a `break` would not also reach, so a loop whose condition narrows a type
|
|
324
|
+
carries no branch obligation at all rather than one no test could close. The
|
|
325
|
+
same goes for a loop over a constant, `while (true)`, which can only go one
|
|
326
|
+
way. A Kotlin `contract { }` has to stay the first statement of its function,
|
|
327
|
+
so the probe that records the function being entered is written after it.
|
|
328
|
+
|
|
329
|
+
Every one of these is named in the run: ask for `supercov runs latest
|
|
330
|
+
limitations` and each appears with its file, its line, and why it was left
|
|
331
|
+
alone. A source file the parser cannot read is declared there too, so a hole in
|
|
332
|
+
the denominator stays visible after the build log is gone.
|
|
333
|
+
|
|
334
|
+
For assertion maps, forms spelled `assertSomething`, `assertThat` or `fail` are
|
|
335
|
+
inventoried, which covers JUnit, TestNG, AssertJ, Hamcrest and kotlin.test.
|
|
336
|
+
Kotest's infix matchers are not.
|
|
337
|
+
|
|
338
|
+
```sh
|
|
339
|
+
npx supercov -- mvn test
|
|
340
|
+
npx supercov -- ./gradlew test
|
|
341
|
+
npx supercov -- ./mvnw verify
|
|
342
|
+
```
|
|
343
|
+
|
|
214
344
|
## Containers, VMs, and remote execution
|
|
215
345
|
|
|
216
346
|
Supercov can collect from supported processes launched through a container, VM,
|
package/docs/troubleshooting.md
CHANGED
|
@@ -75,6 +75,33 @@ Its lines, methods and simple branches stay measured; everything that needs a
|
|
|
75
75
|
probe is declared as a measurement limit for that file. Please report the file,
|
|
76
76
|
since Supercov aims to instrument every Ruby source correctly.
|
|
77
77
|
|
|
78
|
+
## A test that reads your source fails under Supercov
|
|
79
|
+
|
|
80
|
+
A test that opens your source files and asserts on their text sees the probes,
|
|
81
|
+
because that is what instrumentation is. RxJava has one: `CheckCatchThrowIfFatal`
|
|
82
|
+
reads every `catch` block in the project and requires the first statement to be
|
|
83
|
+
`Exceptions.throwIfFatal`, and under Supercov the first statement is a probe. An
|
|
84
|
+
architecture or convention test that scans bytecode or source can fail the same
|
|
85
|
+
way.
|
|
86
|
+
|
|
87
|
+
Nothing is wrong with the measurement and nothing your code does has changed --
|
|
88
|
+
the run still records every test that ran -- but the command exited non-zero, so
|
|
89
|
+
its coverage is reported as diagnostic and cannot gate. Leave that one test out
|
|
90
|
+
of the command you give Supercov. Surefire excludes by name:
|
|
91
|
+
|
|
92
|
+
```sh supercov
|
|
93
|
+
npx supercov -- mvn test -Dtest='!CheckCatchThrowIfFatalTest'
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Gradle's `--tests` only includes, so exclude it in your build file instead --
|
|
97
|
+
Supercov measures the copy, and your own build is unaffected by what it runs:
|
|
98
|
+
|
|
99
|
+
```groovy
|
|
100
|
+
test { filter { excludeTestsMatching "*CheckCatchThrowIfFatal*" } }
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Either way your own build still runs the test, against your own sources.
|
|
104
|
+
|
|
78
105
|
## A run is marked stale
|
|
79
106
|
|
|
80
107
|
A stored run remains valid history, but it stops describing the current
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "supercov",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.49",
|
|
4
4
|
"description": "Coverage for coding agents and software factories \ud83c\udf19",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -76,7 +76,7 @@
|
|
|
76
76
|
"benchmark:check": "cargo build --release -p supercov && node scripts/rust-transform-benchmark.mjs",
|
|
77
77
|
"benchmark:python-monitoring": "cargo build -p supercov && node scripts/python-monitoring-benchmark.mjs",
|
|
78
78
|
"check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:runtime && npm run test:rust-assets && npm run test:assertion-maps && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
|
|
79
|
-
"release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python-monitoring && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
|
|
79
|
+
"release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:go && npm run test:jvm && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python-monitoring && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
|
|
80
80
|
"release:bump": "node scripts/bump-version.mjs",
|
|
81
81
|
"sweep": "node scripts/sweep-target.mjs",
|
|
82
82
|
"oracle:rust": "cargo build --release -p supercov && node scripts/rust-coverage-oracle.mjs",
|
|
@@ -86,6 +86,8 @@
|
|
|
86
86
|
"test:host-loader": "cargo build -p supercov && node scripts/rust-host-loader-integration.mjs",
|
|
87
87
|
"test:python-monitoring": "cargo build -p supercov && node scripts/python-monitoring-integration.mjs",
|
|
88
88
|
"test:ruby-coverage": "cargo build -p supercov && node scripts/ruby-coverage-integration.mjs",
|
|
89
|
+
"test:go": "cargo test --no-fail-fast -p supercov-engine --test go_frontend --test go_run",
|
|
90
|
+
"test:jvm": "cargo test --no-fail-fast -p supercov-engine --test jvm_frontend --test jvm_run -- --include-ignored --test-threads=1",
|
|
89
91
|
"test:rust-public-cargo": "cargo build -p supercov && node scripts/rust-public-cargo-integration.mjs",
|
|
90
92
|
"test:assertion-maps": "cargo build -p supercov && node scripts/assertion-map-schema.mjs --check && node scripts/assertion-map-integration.mjs && node scripts/assertion-map-js-integration.mjs && node scripts/assertion-dogfood-integration.mjs",
|
|
91
93
|
"test:assertion-maps:js": "cargo build -p supercov && node scripts/assertion-map-schema.mjs --check && node scripts/assertion-map-js-integration.mjs && node scripts/assertion-dogfood-integration.mjs",
|
|
@@ -94,14 +96,14 @@
|
|
|
94
96
|
"docs:check": "node scripts/sync-docs.mjs --check"
|
|
95
97
|
},
|
|
96
98
|
"optionalDependencies": {
|
|
97
|
-
"@supercov/cli-darwin-arm64": "0.0.
|
|
98
|
-
"@supercov/cli-darwin-x64": "0.0.
|
|
99
|
-
"@supercov/cli-linux-arm64-gnu": "0.0.
|
|
100
|
-
"@supercov/cli-linux-arm64-musl": "0.0.
|
|
101
|
-
"@supercov/cli-linux-x64-gnu": "0.0.
|
|
102
|
-
"@supercov/cli-linux-x64-musl": "0.0.
|
|
103
|
-
"@supercov/cli-win32-arm64": "0.0.
|
|
104
|
-
"@supercov/cli-win32-x64": "0.0.
|
|
99
|
+
"@supercov/cli-darwin-arm64": "0.0.49",
|
|
100
|
+
"@supercov/cli-darwin-x64": "0.0.49",
|
|
101
|
+
"@supercov/cli-linux-arm64-gnu": "0.0.49",
|
|
102
|
+
"@supercov/cli-linux-arm64-musl": "0.0.49",
|
|
103
|
+
"@supercov/cli-linux-x64-gnu": "0.0.49",
|
|
104
|
+
"@supercov/cli-linux-x64-musl": "0.0.49",
|
|
105
|
+
"@supercov/cli-win32-arm64": "0.0.49",
|
|
106
|
+
"@supercov/cli-win32-x64": "0.0.49"
|
|
105
107
|
},
|
|
106
108
|
"peerDependencies": {
|
|
107
109
|
"@playwright/test": ">=1.55.0",
|