supercov 1.0.1 → 1.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -17
- package/docs/cli.md +1 -1
- package/docs/getting-started.md +1 -1
- package/docs/performance.md +2 -2
- package/docs/supported-suites.md +51 -3
- package/docs/troubleshooting.md +14 -0
- package/package.json +18 -12
package/README.md
CHANGED
|
@@ -7,13 +7,15 @@
|
|
|
7
7
|
Paste this to your coding agent to start:
|
|
8
8
|
|
|
9
9
|
```text supercov-prompt
|
|
10
|
-
Measure code quality with npx supercov
|
|
10
|
+
Measure code quality with npx supercov.
|
|
11
11
|
```
|
|
12
12
|
|
|
13
13
|
Scoring needs a [TypeSafe AI](https://typesafe.ai) API key, and your coding agent will usually ask you for it. It costs about a cent per megabyte of source. Coverage needs no account, config file, import, custom reporter, or hosted service. Supercov is local, free, open source, and MIT licensed.
|
|
14
14
|
|
|
15
15
|
[Website](https://supercov.com) · [Documentation](https://supercov.com/docs) · [npm](https://www.npmjs.com/package/supercov) · [GitHub](https://github.com/supercorp-ai/supercov)
|
|
16
16
|
|
|
17
|
+
Supported languages: [JavaScript](https://supercov.com/docs/javascript) · [TypeScript](https://supercov.com/docs/javascript) · [Rust](https://supercov.com/docs/rust) · [Python](https://supercov.com/docs/python) · [Ruby](https://supercov.com/docs/ruby) · [Go](https://supercov.com/docs/go) · [Java](https://supercov.com/docs/java) · [Kotlin](https://supercov.com/docs/kotlin).
|
|
18
|
+
|
|
17
19
|
Supported by [Supercorp](https://supercorp.ai).
|
|
18
20
|
|
|
19
21
|
[Agent workflow](https://supercov.com/docs/agent-loop): ask your coding agent
|
|
@@ -217,21 +219,21 @@ Java and Kotlin have no registry of their own here, so a JVM project takes the b
|
|
|
217
219
|
|
|
218
220
|
## Supported languages
|
|
219
221
|
|
|
220
|
-
| Language | Status | Start with |
|
|
221
|
-
| --- | --- | --- |
|
|
222
|
-
| JavaScript | Available | `npx supercov -- npm test` |
|
|
223
|
-
| TypeScript | Available | `npx supercov -- npm test` |
|
|
224
|
-
| Rust | Available | `
|
|
225
|
-
| Python | Available | `
|
|
226
|
-
| Ruby | Available | `
|
|
227
|
-
| Go | Available | `go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./...` |
|
|
228
|
-
| Java | Available | `
|
|
229
|
-
| Kotlin | Available | `
|
|
230
|
-
| Zig | Coming soon | — |
|
|
231
|
-
| PHP | Coming soon | — |
|
|
232
|
-
| C | Coming soon | — |
|
|
233
|
-
|
|
234
|
-
|
|
222
|
+
| Language | Status | Install | Start with |
|
|
223
|
+
| --- | --- | --- | --- |
|
|
224
|
+
| [JavaScript](https://supercov.com/docs/javascript) | Available | nothing, `npx` fetches it | `npx supercov -- npm test` |
|
|
225
|
+
| [TypeScript](https://supercov.com/docs/javascript) | Available | nothing, `npx` fetches it | `npx supercov -- npm test` |
|
|
226
|
+
| [Rust](https://supercov.com/docs/rust) | Available | `cargo install supercov` | `supercov -- cargo test` |
|
|
227
|
+
| [Python](https://supercov.com/docs/python) | Available | `python -m pip install supercov-cli` | `supercov -- python -m pytest` |
|
|
228
|
+
| [Ruby](https://supercov.com/docs/ruby) | Available | `gem install supercov` | `supercov -- bundle exec rspec` |
|
|
229
|
+
| [Go](https://supercov.com/docs/go) | Available | nothing, `go run` fetches it | `go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./...` |
|
|
230
|
+
| [Java](https://supercov.com/docs/java) | Available | `brew install supercorp-ai/tap/supercov` | `supercov -- mvn test` |
|
|
231
|
+
| [Kotlin](https://supercov.com/docs/kotlin) | Available | `brew install supercorp-ai/tap/supercov` | `supercov -- ./gradlew test` |
|
|
232
|
+
| Zig | Coming soon | — | — |
|
|
233
|
+
| PHP | Coming soon | — | — |
|
|
234
|
+
| C | Coming soon | — | — |
|
|
235
|
+
|
|
236
|
+
The `npx` path requires Node.js 22 or newer; the other installs need no Node. Rust support currently uses Rust 1.95; cargo-nextest 0.9.138 and 0.9.140 are supported. Python support requires CPython 3.12 or newer and measures pytest and unittest runs. Ruby support requires Ruby 3.3 or newer (3.4 or newer for full measurement) and measures RSpec, Minitest, test-unit and Cucumber runs. Go support requires Go 1.22 or newer and measures `go test`. Java and Kotlin support requires a JDK 17 or newer with Maven or Gradle, and measures every JUnit Platform engine -- JUnit 5, JUnit 4 through Vintage, Kotest and Spock -- as well as TestNG.
|
|
235
237
|
|
|
236
238
|
## Supported operating systems and architectures
|
|
237
239
|
|
|
@@ -243,7 +245,7 @@ The CLI is a single native binary. `npx supercov` selects the build for the mach
|
|
|
243
245
|
| Linux | arm64, x64 | glibc 2.28 or newer (Debian 10, Ubuntu 18.10, RHEL 8 and later), or musl (Alpine); chosen automatically |
|
|
244
246
|
| Windows | arm64, x64 | Windows 10 or newer |
|
|
245
247
|
|
|
246
|
-
JavaScript, TypeScript, Python, Ruby, Rust, Go, Java and Kotlin suites are verified on Linux, macOS and Windows -- CPython 3.12, 3.13 and 3.14, Ruby 3.3, 3.4 and 4.0, and Go 1.22 and 1.
|
|
248
|
+
JavaScript, TypeScript, Python, Ruby, Rust, Go, Java and Kotlin suites are verified on Linux, macOS and Windows -- CPython 3.12, 3.13 and 3.14, Ruby 3.3, 3.4 and 4.0, and Go 1.22, 1.25, 1.26 and 1.27 on every one, and JDK 17 and 21 on every one that has them (Temurin publishes no 17 for arm64 Windows, so that pairing is 21 alone). Containers, VMs, and remote executors run the same Linux builds. Every release publishes the same binaries to npm, PyPI (`supercov-cli`, a wheel per platform) and RubyGems (`supercov`, a gem per platform except arm64 Windows, which Ruby has no platform for), and the source to crates.io (`supercov`).
|
|
247
249
|
|
|
248
250
|
## Supported test suites
|
|
249
251
|
|
|
@@ -320,3 +322,7 @@ npx supercov clean # remove all runs and the build cache
|
|
|
320
322
|
## Free and open source
|
|
321
323
|
|
|
322
324
|
[MIT licensed](LICENSE). Inspect, extend, and run it anywhere.
|
|
325
|
+
|
|
326
|
+
## Contributors
|
|
327
|
+
|
|
328
|
+
- [@DaVinciTachyon](https://github.com/DaVinciTachyon)
|
package/docs/cli.md
CHANGED
|
@@ -410,7 +410,7 @@ terminal or offline environment after the package has been downloaded.
|
|
|
410
410
|
|
|
411
411
|
| Variable | Use |
|
|
412
412
|
| --- | --- |
|
|
413
|
-
| `SUPERCOV_SOURCE_ROOTS` |
|
|
413
|
+
| `SUPERCOV_SOURCE_ROOTS` | Comma-separated directories or files that hold your own code, in any language; everything else is left out |
|
|
414
414
|
| `SUPERCOV_TEST_KIND` | Label the wrapped command as a test level such as `unit` or `e2e` |
|
|
415
415
|
|
|
416
416
|
Examples:
|
package/docs/getting-started.md
CHANGED
|
@@ -16,7 +16,7 @@ To score the code itself rather than its tests, set a [Jev](https://typesafe.ai)
|
|
|
16
16
|
key and ask:
|
|
17
17
|
|
|
18
18
|
```text supercov-prompt
|
|
19
|
-
Measure code quality with npx supercov
|
|
19
|
+
Measure code quality with npx supercov.
|
|
20
20
|
```
|
|
21
21
|
|
|
22
22
|
See [Understanding quality](https://supercov.com/docs/quality).
|
package/docs/performance.md
CHANGED
|
@@ -95,6 +95,6 @@ flow, `--view nodes` or `--view edges` pages the details; `--compact` omits repe
|
|
|
95
95
|
source text while keeping locations and credit reasons. See
|
|
96
96
|
[Investigating assertion evidence](assertion-evidence.md#read-a-large-flow).
|
|
97
97
|
|
|
98
|
-
The disposable `assertions.report.cache.json`
|
|
99
|
-
or damaged cache is rebuilt. Removing it affects the next query's speed, not
|
|
98
|
+
The disposable `assertions.report.cache.json` and `assertions.summary.cache.json`
|
|
99
|
+
files live beside the map. A missing or damaged cache is rebuilt. Removing it affects the next query's speed, not
|
|
100
100
|
your saved explanations.
|
package/docs/supported-suites.md
CHANGED
|
@@ -218,12 +218,24 @@ npx supercov -- python -m unittest
|
|
|
218
218
|
| Runner | Attribution | Current requirement |
|
|
219
219
|
| --- | --- | --- |
|
|
220
220
|
| RSpec | Exact example and before/example/after phase identity | Ruby 3.4 or newer for full measurement; run with `npx supercov -- rspec` or `bundle exec rspec` |
|
|
221
|
-
| Minitest (including Minitest::Spec and ActiveSupport::TestCase) | Exact test and setup/test/teardown identity; skips recorded | `ruby -Itest ...`, `rake test`, `rails test` |
|
|
221
|
+
| Minitest (including Minitest::Spec and ActiveSupport::TestCase) | Exact test and setup/test/teardown identity; a test's coverage is a lower bound (see below); skips recorded | `ruby -Itest ...`, `rake test`, `rails test` |
|
|
222
222
|
| test-unit | Exact test and setup/test/teardown identity; omissions and pendings recorded | `ruby -Itest ...`, `rake test` |
|
|
223
223
|
| parallel_tests, Rails process workers | Exact per worker process | Workers inherit the run through `RUBYOPT`; verified on a Rails app with bootsnap, Zeitwerk and two forked workers |
|
|
224
224
|
| Thread-parallel Minitest (`parallelize_me!`, `parallelize(with: :threads)`) | Probe observations exact per test; line, method and simple-branch observations made while phases overlapped go to the run, declared | |
|
|
225
225
|
| Cucumber | Exact scenario identity (`features/x.feature:LINE`), hook steps as setup/teardown | `cucumber`, `bundle exec cucumber` |
|
|
226
226
|
|
|
227
|
+
Ruby reports a line the first time it executes and never again, which is what
|
|
228
|
+
makes collecting coverage cheap enough to leave on. So a test is credited with
|
|
229
|
+
the lines it was first to reach, and a later test running the same lines is
|
|
230
|
+
credited with none of them: what a test is credited with is really its own, and
|
|
231
|
+
what it is not credited with is not evidence it did not run the code.
|
|
232
|
+
|
|
233
|
+
Totals are unaffected — every line is credited to exactly one test. What this
|
|
234
|
+
changes is per-test reporting: `supercov runs <id> test <name>` gives its
|
|
235
|
+
numbers as "at least", and `tests affected` reports a test whose own record
|
|
236
|
+
cannot settle the question as **undetermined** rather than unaffected, so
|
|
237
|
+
`--names` includes it in the set to run.
|
|
238
|
+
|
|
227
239
|
Your project runs in place with its own interpreter and bundle. Supercov loads
|
|
228
240
|
through `RUBYOPT`; application files on disk and their backtrace line numbers
|
|
229
241
|
stay unchanged. RSpec, Minitest and test-unit assertions can identify execution
|
|
@@ -261,6 +273,31 @@ npx supercov -- ruby -Itest test/shapes_test.rb
|
|
|
261
273
|
npx supercov -- bin/rails test
|
|
262
274
|
```
|
|
263
275
|
|
|
276
|
+
## Crediting a test with what it reached
|
|
277
|
+
|
|
278
|
+
Most runners attribute exactly at no cost: Supercov runs each Rust test in its
|
|
279
|
+
own process, Python's workers are processes, and JavaScript keeps a coverage
|
|
280
|
+
map per test. Two cases cannot, because probes are a store into one array the
|
|
281
|
+
whole process shares and two tests running at once cannot both be credited
|
|
282
|
+
with what they reached:
|
|
283
|
+
|
|
284
|
+
| | Left as written | `--exact-attribution` |
|
|
285
|
+
| --- | --- | --- |
|
|
286
|
+
| A Go test calling `t.Parallel()` | counted run-wide | announced after it resumes, with `-parallel=1` |
|
|
287
|
+
| A JUnit suite with parallel execution enabled | counted run-wide | run in order |
|
|
288
|
+
|
|
289
|
+
```sh supercov
|
|
290
|
+
npx supercov --exact-attribution -- go test ./...
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
By default your command runs as you wrote it, and the run says what it could
|
|
294
|
+
not credit. `--exact-attribution` buys that back by running the suite in order,
|
|
295
|
+
which costs whatever its parallelism was worth — on sixteen I/O-bound JUnit
|
|
296
|
+
tests across fifteen cores, 3.6x. Either way the coverage totals are the same;
|
|
297
|
+
what changes is whether a line can be traced to the test that ran it.
|
|
298
|
+
|
|
299
|
+
`supercov runs latest` reports which you got, under `Attribution`.
|
|
300
|
+
|
|
264
301
|
## Go
|
|
265
302
|
|
|
266
303
|
| Runner | Attribution | Current requirement |
|
|
@@ -287,8 +324,19 @@ coverage never counts — a failed run cannot say which of it came from the test
|
|
|
287
324
|
that failed.
|
|
288
325
|
|
|
289
326
|
An `Example` with an `Output` comment and a `Fuzz` target's seed corpus are
|
|
290
|
-
measured the same way
|
|
291
|
-
coverage
|
|
327
|
+
measured the same way: `go test` runs both, and what they reach is real
|
|
328
|
+
coverage no test can claim.
|
|
329
|
+
|
|
330
|
+
Such a test is still recorded as having run, with its outcome, and credited
|
|
331
|
+
with no coverage. `supercov runs <id> test <name>` says that rather than
|
|
332
|
+
reporting zeroes, coverage percentages leave it out, and `tests affected` lists
|
|
333
|
+
it as **undetermined**: nothing can say a change missed it, so `--names`
|
|
334
|
+
includes it in the set to run. A package where every test calls `t.Parallel()`
|
|
335
|
+
is published like any other.
|
|
336
|
+
|
|
337
|
+
A file Supercov cannot parse is a hole, not the end of the run. It is named on
|
|
338
|
+
the line that reports the result and recorded in the run, and a run is refused
|
|
339
|
+
only when nothing is left to measure.
|
|
292
340
|
|
|
293
341
|
Supercov also writes evidence as the suite runs, not only at the end. Go offers
|
|
294
342
|
no way to run code on `os.Exit`, and a `TestMain` need not reach the `m.Run()`
|
package/docs/troubleshooting.md
CHANGED
|
@@ -41,6 +41,20 @@ npx supercov runs latest scope
|
|
|
41
41
|
Do not broaden the roots to dependencies or generated output merely to remove a
|
|
42
42
|
warning. The goal is an honest boundary around code the repository owns.
|
|
43
43
|
|
|
44
|
+
## Too much is measured
|
|
45
|
+
|
|
46
|
+
Vendored or generated code beside your own is measured like the rest of the
|
|
47
|
+
repository. Name the directories or files that hold your own code; this works
|
|
48
|
+
for every language:
|
|
49
|
+
|
|
50
|
+
```sh supercov
|
|
51
|
+
SUPERCOV_SOURCE_ROOTS=src npx supercov -- pytest
|
|
52
|
+
SUPERCOV_SOURCE_ROOTS=app.py,core.py npx supercov -- python -m unittest
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Everything outside the roots is left out, and `npx supercov runs latest scope`
|
|
56
|
+
lists what was.
|
|
57
|
+
|
|
44
58
|
## The tests pass but coverage is missing
|
|
45
59
|
|
|
46
60
|
First check runner and source scope:
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "supercov",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.1.1",
|
|
4
4
|
"description": "Code quality and coverage for coding agents",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -56,6 +56,7 @@
|
|
|
56
56
|
"test:rust-compiler-spikes": "cargo build -p supercov && node scripts/rust-libtest-companion-spike.mjs && node scripts/rust-async-attribution-spike.mjs && node scripts/rust-subprocess-attribution-spike.mjs && node scripts/rust-custom-harness-spike.mjs && node scripts/rust-libtest-builder-lifecycle-spike.mjs",
|
|
57
57
|
"test": "cargo test --workspace",
|
|
58
58
|
"test:react": "cargo build -p supercov && node scripts/rust-vitest-projects-integration.mjs && npm --prefix examples/react-verification ci && npm --prefix examples/react-verification run demo && node scripts/react-hydration-compatibility.mjs",
|
|
59
|
+
"test:release-tooling": "node --test tests/release/*.test.mjs",
|
|
59
60
|
"test:runtime": "node --test tests/runtime/*.test.mjs",
|
|
60
61
|
"test:fixture": "cargo build -p supercov && node scripts/rust-fixture-matrix.mjs",
|
|
61
62
|
"test:packed-npx": "cargo build --release -p supercov && node scripts/packed-npx-integration.mjs",
|
|
@@ -64,7 +65,8 @@
|
|
|
64
65
|
"test:watchdog": "cargo build -p supercov && node scripts/watchdog-integration.mjs",
|
|
65
66
|
"test:engine-contract": "cargo build -p supercov && node scripts/engine-contract.mjs",
|
|
66
67
|
"test:agent": "cargo build -p supercov && node scripts/agent-query-eval.mjs",
|
|
67
|
-
"test:engine": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace &&
|
|
68
|
+
"test:engine": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace && npm run test:runtime && npm run test:rust-assets && npm run test:engine-integrations",
|
|
69
|
+
"test:engine-integrations": "cargo build -p supercov && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
|
|
68
70
|
"test:platform": "cargo test --workspace && cargo build -p supercov && node scripts/rust-process-supervision.mjs && node scripts/workspace-crash-integration.mjs",
|
|
69
71
|
"test:native-package": "cargo build --release -p supercov && node scripts/native-package-integration.mjs && node scripts/native-release-set-integration.mjs",
|
|
70
72
|
"test:pypi-wheel": "node scripts/pypi-wheel-integration.mjs",
|
|
@@ -77,7 +79,7 @@
|
|
|
77
79
|
"test:test262": "cargo build --release -p supercov && node scripts/test262-equivalence.mjs",
|
|
78
80
|
"benchmark:check": "cargo build --release -p supercov && node scripts/rust-transform-benchmark.mjs",
|
|
79
81
|
"benchmark:python-monitoring": "cargo build -p supercov && node scripts/python-monitoring-benchmark.mjs",
|
|
80
|
-
"check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:runtime && npm run test:rust-assets && npm run test:assertion-maps && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
|
|
82
|
+
"check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:release-tooling && npm run test:runtime && npm run test:rust-assets && npm run test:assertion-maps && npm run test:sweep-target && npm run test:skip-guard && npm run test:windows-tls && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
|
|
81
83
|
"release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:go && npm run test:jvm && npm run test:launcher && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python-monitoring && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
|
|
82
84
|
"release:bump": "node scripts/bump-version.mjs",
|
|
83
85
|
"sweep": "node scripts/sweep-target.mjs",
|
|
@@ -96,17 +98,21 @@
|
|
|
96
98
|
"sync:assertion-schema": "cargo build -p supercov && node scripts/assertion-map-schema.mjs",
|
|
97
99
|
"docs:sync": "node scripts/sync-docs.mjs",
|
|
98
100
|
"docs:check": "node scripts/sync-docs.mjs --check",
|
|
99
|
-
"test:launcher": "go vet ./cmd/... && go test ./cmd/..."
|
|
101
|
+
"test:launcher": "go vet ./cmd/... && go test ./cmd/...",
|
|
102
|
+
"test:sweep-target": "node scripts/sweep-target-test.mjs",
|
|
103
|
+
"test:go-versions": "cargo build -p supercov-engine --tests && node scripts/go-version-matrix.mjs",
|
|
104
|
+
"test:skip-guard": "node scripts/skip-guard-test.mjs",
|
|
105
|
+
"test:windows-tls": "node scripts/windows-tls-test.mjs"
|
|
100
106
|
},
|
|
101
107
|
"optionalDependencies": {
|
|
102
|
-
"@supercov/cli-darwin-arm64": "1.
|
|
103
|
-
"@supercov/cli-darwin-x64": "1.
|
|
104
|
-
"@supercov/cli-linux-arm64-gnu": "1.
|
|
105
|
-
"@supercov/cli-linux-arm64-musl": "1.
|
|
106
|
-
"@supercov/cli-linux-x64-gnu": "1.
|
|
107
|
-
"@supercov/cli-linux-x64-musl": "1.
|
|
108
|
-
"@supercov/cli-win32-arm64": "1.
|
|
109
|
-
"@supercov/cli-win32-x64": "1.
|
|
108
|
+
"@supercov/cli-darwin-arm64": "1.1.1",
|
|
109
|
+
"@supercov/cli-darwin-x64": "1.1.1",
|
|
110
|
+
"@supercov/cli-linux-arm64-gnu": "1.1.1",
|
|
111
|
+
"@supercov/cli-linux-arm64-musl": "1.1.1",
|
|
112
|
+
"@supercov/cli-linux-x64-gnu": "1.1.1",
|
|
113
|
+
"@supercov/cli-linux-x64-musl": "1.1.1",
|
|
114
|
+
"@supercov/cli-win32-arm64": "1.1.1",
|
|
115
|
+
"@supercov/cli-win32-x64": "1.1.1"
|
|
110
116
|
},
|
|
111
117
|
"peerDependencies": {
|
|
112
118
|
"@playwright/test": ">=1.55.0",
|