supercov 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -17
- package/docs/cli.md +1 -1
- package/docs/getting-started.md +2 -3
- package/docs/quality.md +4 -2
- package/docs/supported-suites.md +51 -3
- package/package.json +15 -11
package/README.md
CHANGED
|
@@ -4,10 +4,18 @@
|
|
|
4
4
|
|
|
5
5
|
**Supercov tells your coding agent what to fix and what to test.** It scores your code quality with [Jev](https://typesafe.ai), runs the test command you already use, and turns uncovered paths into small, actionable queries. Your agent picks a target, writes a focused test or a focused refactor, proves what improved, and keeps going.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Paste this to your coding agent to start:
|
|
8
|
+
|
|
9
|
+
```text supercov-prompt
|
|
10
|
+
Measure code quality with npx supercov.
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Scoring needs a [TypeSafe AI](https://typesafe.ai) API key, and your coding agent will usually ask you for it. It costs about a cent per megabyte of source. Coverage needs no account, config file, import, custom reporter, or hosted service. Supercov is local, free, open source, and MIT licensed.
|
|
8
14
|
|
|
9
15
|
[Website](https://supercov.com) · [Documentation](https://supercov.com/docs) · [npm](https://www.npmjs.com/package/supercov) · [GitHub](https://github.com/supercorp-ai/supercov)
|
|
10
16
|
|
|
17
|
+
Supported languages: [JavaScript](https://supercov.com/docs/javascript) · [TypeScript](https://supercov.com/docs/javascript) · [Rust](https://supercov.com/docs/rust) · [Python](https://supercov.com/docs/python) · [Ruby](https://supercov.com/docs/ruby) · [Go](https://supercov.com/docs/go) · [Java](https://supercov.com/docs/java) · [Kotlin](https://supercov.com/docs/kotlin).
|
|
18
|
+
|
|
11
19
|
Supported by [Supercorp](https://supercorp.ai).
|
|
12
20
|
|
|
13
21
|
[Agent workflow](https://supercov.com/docs/agent-loop): ask your coding agent
|
|
@@ -211,21 +219,21 @@ Java and Kotlin have no registry of their own here, so a JVM project takes the b
|
|
|
211
219
|
|
|
212
220
|
## Supported languages
|
|
213
221
|
|
|
214
|
-
| Language | Status | Start with |
|
|
215
|
-
| --- | --- | --- |
|
|
216
|
-
| JavaScript | Available | `npx supercov -- npm test` |
|
|
217
|
-
| TypeScript | Available | `npx supercov -- npm test` |
|
|
218
|
-
| Rust | Available | `
|
|
219
|
-
| Python | Available | `
|
|
220
|
-
| Ruby | Available | `
|
|
221
|
-
| Go | Available | `go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./...` |
|
|
222
|
-
| Java | Available | `
|
|
223
|
-
| Kotlin | Available | `
|
|
224
|
-
| Zig | Coming soon | — |
|
|
225
|
-
| PHP | Coming soon | — |
|
|
226
|
-
| C | Coming soon | — |
|
|
227
|
-
|
|
228
|
-
|
|
222
|
+
| Language | Status | Install | Start with |
|
|
223
|
+
| --- | --- | --- | --- |
|
|
224
|
+
| [JavaScript](https://supercov.com/docs/javascript) | Available | nothing, `npx` fetches it | `npx supercov -- npm test` |
|
|
225
|
+
| [TypeScript](https://supercov.com/docs/javascript) | Available | nothing, `npx` fetches it | `npx supercov -- npm test` |
|
|
226
|
+
| [Rust](https://supercov.com/docs/rust) | Available | `cargo install supercov` | `supercov -- cargo test` |
|
|
227
|
+
| [Python](https://supercov.com/docs/python) | Available | `python -m pip install supercov-cli` | `supercov -- python -m pytest` |
|
|
228
|
+
| [Ruby](https://supercov.com/docs/ruby) | Available | `gem install supercov` | `supercov -- bundle exec rspec` |
|
|
229
|
+
| [Go](https://supercov.com/docs/go) | Available | nothing, `go run` fetches it | `go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./...` |
|
|
230
|
+
| [Java](https://supercov.com/docs/java) | Available | `brew install supercorp-ai/tap/supercov` | `supercov -- mvn test` |
|
|
231
|
+
| [Kotlin](https://supercov.com/docs/kotlin) | Available | `brew install supercorp-ai/tap/supercov` | `supercov -- ./gradlew test` |
|
|
232
|
+
| Zig | Coming soon | — | — |
|
|
233
|
+
| PHP | Coming soon | — | — |
|
|
234
|
+
| C | Coming soon | — | — |
|
|
235
|
+
|
|
236
|
+
The `npx` path requires Node.js 22 or newer; the other installs need no Node. Rust support currently uses Rust 1.95; cargo-nextest 0.9.138 and 0.9.140 are supported. Python support requires CPython 3.12 or newer and measures pytest and unittest runs. Ruby support requires Ruby 3.3 or newer (3.4 or newer for full measurement) and measures RSpec, Minitest, test-unit and Cucumber runs. Go support requires Go 1.22 or newer and measures `go test`. Java and Kotlin support requires a JDK 17 or newer with Maven or Gradle, and measures every JUnit Platform engine -- JUnit 5, JUnit 4 through Vintage, Kotest and Spock -- as well as TestNG.
|
|
229
237
|
|
|
230
238
|
## Supported operating systems and architectures
|
|
231
239
|
|
|
@@ -237,7 +245,7 @@ The CLI is a single native binary. `npx supercov` selects the build for the mach
|
|
|
237
245
|
| Linux | arm64, x64 | glibc 2.28 or newer (Debian 10, Ubuntu 18.10, RHEL 8 and later), or musl (Alpine); chosen automatically |
|
|
238
246
|
| Windows | arm64, x64 | Windows 10 or newer |
|
|
239
247
|
|
|
240
|
-
JavaScript, TypeScript, Python, Ruby, Rust, Go, Java and Kotlin suites are verified on Linux, macOS and Windows -- CPython 3.12, 3.13 and 3.14, Ruby 3.3, 3.4 and 4.0, and Go 1.22 and 1.
|
|
248
|
+
JavaScript, TypeScript, Python, Ruby, Rust, Go, Java and Kotlin suites are verified on Linux, macOS and Windows -- CPython 3.12, 3.13 and 3.14, Ruby 3.3, 3.4 and 4.0, and Go 1.22, 1.25, 1.26 and 1.27 on every one, and JDK 17 and 21 on every one that has them (Temurin publishes no 17 for arm64 Windows, so that pairing is 21 alone). Containers, VMs, and remote executors run the same Linux builds. Every release publishes the same binaries to npm, PyPI (`supercov-cli`, a wheel per platform) and RubyGems (`supercov`, a gem per platform except arm64 Windows, which Ruby has no platform for), and the source to crates.io (`supercov`).
|
|
241
249
|
|
|
242
250
|
## Supported test suites
|
|
243
251
|
|
package/docs/cli.md
CHANGED
|
@@ -55,7 +55,7 @@ the exact requests and contacts nothing.
|
|
|
55
55
|
`quality diff` reports what declined between two assessments: which files lost
|
|
56
56
|
health, which properties appeared, and which files entered or left the scope.
|
|
57
57
|
|
|
58
|
-
Assessing needs a TypeSafe API key in `TYPESAFE_API_KEY`; reading a saved
|
|
58
|
+
Assessing needs a TypeSafe AI API key in `TYPESAFE_API_KEY`; reading a saved
|
|
59
59
|
assessment does not. The command prints a cost estimate before sending anything
|
|
60
60
|
and caches answers by content, so a second run pays only for what changed.
|
|
61
61
|
|
package/docs/getting-started.md
CHANGED
|
@@ -15,9 +15,8 @@ tests; you don't need to do those steps yourself.
|
|
|
15
15
|
To score the code itself rather than its tests, set a [Jev](https://typesafe.ai)
|
|
16
16
|
key and ask:
|
|
17
17
|
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
npx supercov quality
|
|
18
|
+
```text supercov-prompt
|
|
19
|
+
Measure code quality with npx supercov.
|
|
21
20
|
```
|
|
22
21
|
|
|
23
22
|
See [Understanding quality](https://supercov.com/docs/quality).
|
package/docs/quality.md
CHANGED
|
@@ -9,8 +9,10 @@ arithmetic, so every part of a score is a claim you can check against the file.
|
|
|
9
9
|
|
|
10
10
|
## Set your key
|
|
11
11
|
|
|
12
|
-
Assessing needs a TypeSafe API key,
|
|
13
|
-
[typesafe.ai](https://typesafe.ai).
|
|
12
|
+
Assessing needs a TypeSafe AI API key, and your coding agent will usually ask
|
|
13
|
+
you for it. Get one at [typesafe.ai](https://typesafe.ai).
|
|
14
|
+
|
|
15
|
+
Supercov looks for it in `TYPESAFE_API_KEY`:
|
|
14
16
|
|
|
15
17
|
```bash
|
|
16
18
|
export TYPESAFE_API_KEY=... # for this shell
|
package/docs/supported-suites.md
CHANGED
|
@@ -218,12 +218,24 @@ npx supercov -- python -m unittest
|
|
|
218
218
|
| Runner | Attribution | Current requirement |
|
|
219
219
|
| --- | --- | --- |
|
|
220
220
|
| RSpec | Exact example and before/example/after phase identity | Ruby 3.4 or newer for full measurement; run with `npx supercov -- rspec` or `bundle exec rspec` |
|
|
221
|
-
| Minitest (including Minitest::Spec and ActiveSupport::TestCase) | Exact test and setup/test/teardown identity; skips recorded | `ruby -Itest ...`, `rake test`, `rails test` |
|
|
221
|
+
| Minitest (including Minitest::Spec and ActiveSupport::TestCase) | Exact test and setup/test/teardown identity; a test's coverage is a lower bound (see below); skips recorded | `ruby -Itest ...`, `rake test`, `rails test` |
|
|
222
222
|
| test-unit | Exact test and setup/test/teardown identity; omissions and pendings recorded | `ruby -Itest ...`, `rake test` |
|
|
223
223
|
| parallel_tests, Rails process workers | Exact per worker process | Workers inherit the run through `RUBYOPT`; verified on a Rails app with bootsnap, Zeitwerk and two forked workers |
|
|
224
224
|
| Thread-parallel Minitest (`parallelize_me!`, `parallelize(with: :threads)`) | Probe observations exact per test; line, method and simple-branch observations made while phases overlapped go to the run, declared | |
|
|
225
225
|
| Cucumber | Exact scenario identity (`features/x.feature:LINE`), hook steps as setup/teardown | `cucumber`, `bundle exec cucumber` |
|
|
226
226
|
|
|
227
|
+
Ruby reports a line the first time it executes and never again, which is what
|
|
228
|
+
makes collecting coverage cheap enough to leave on. So a test is credited with
|
|
229
|
+
the lines it was first to reach, and a later test running the same lines is
|
|
230
|
+
credited with none of them: what a test is credited with is really its own, and
|
|
231
|
+
what it is not credited with is not evidence it did not run the code.
|
|
232
|
+
|
|
233
|
+
Totals are unaffected — every line is credited to exactly one test. What this
|
|
234
|
+
changes is per-test reporting: `supercov runs <id> test <name>` gives its
|
|
235
|
+
numbers as "at least", and `tests affected` reports a test whose own record
|
|
236
|
+
cannot settle the question as **undetermined** rather than unaffected, so
|
|
237
|
+
`--names` includes it in the set to run.
|
|
238
|
+
|
|
227
239
|
Your project runs in place with its own interpreter and bundle. Supercov loads
|
|
228
240
|
through `RUBYOPT`; application files on disk and their backtrace line numbers
|
|
229
241
|
stay unchanged. RSpec, Minitest and test-unit assertions can identify execution
|
|
@@ -261,6 +273,31 @@ npx supercov -- ruby -Itest test/shapes_test.rb
|
|
|
261
273
|
npx supercov -- bin/rails test
|
|
262
274
|
```
|
|
263
275
|
|
|
276
|
+
## Crediting a test with what it reached
|
|
277
|
+
|
|
278
|
+
Most runners attribute exactly at no cost: Supercov runs each Rust test in its
|
|
279
|
+
own process, Python's workers are processes, and JavaScript keeps a coverage
|
|
280
|
+
map per test. Two cases cannot, because probes are a store into one array the
|
|
281
|
+
whole process shares and two tests running at once cannot both be credited
|
|
282
|
+
with what they reached:
|
|
283
|
+
|
|
284
|
+
| | Left as written | `--exact-attribution` |
|
|
285
|
+
| --- | --- | --- |
|
|
286
|
+
| A Go test calling `t.Parallel()` | counted run-wide | announced after it resumes, with `-parallel=1` |
|
|
287
|
+
| A JUnit suite with parallel execution enabled | counted run-wide | run in order |
|
|
288
|
+
|
|
289
|
+
```sh supercov
|
|
290
|
+
npx supercov --exact-attribution -- go test ./...
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
By default your command runs as you wrote it, and the run says what it could
|
|
294
|
+
not credit. `--exact-attribution` buys that back by running the suite in order,
|
|
295
|
+
which costs whatever its parallelism was worth — on sixteen I/O-bound JUnit
|
|
296
|
+
tests across fifteen cores, 3.6x. Either way the coverage totals are the same;
|
|
297
|
+
what changes is whether a line can be traced to the test that ran it.
|
|
298
|
+
|
|
299
|
+
`supercov runs latest` reports which you got, under `Attribution`.
|
|
300
|
+
|
|
264
301
|
## Go
|
|
265
302
|
|
|
266
303
|
| Runner | Attribution | Current requirement |
|
|
@@ -287,8 +324,19 @@ coverage never counts — a failed run cannot say which of it came from the test
|
|
|
287
324
|
that failed.
|
|
288
325
|
|
|
289
326
|
An `Example` with an `Output` comment and a `Fuzz` target's seed corpus are
|
|
290
|
-
measured the same way
|
|
291
|
-
coverage
|
|
327
|
+
measured the same way: `go test` runs both, and what they reach is real
|
|
328
|
+
coverage no test can claim.
|
|
329
|
+
|
|
330
|
+
Such a test is still recorded as having run, with its outcome, and credited
|
|
331
|
+
with no coverage. `supercov runs <id> test <name>` says that rather than
|
|
332
|
+
reporting zeroes, coverage percentages leave it out, and `tests affected` lists
|
|
333
|
+
it as **undetermined**: nothing can say a change missed it, so `--names`
|
|
334
|
+
includes it in the set to run. A package where every test calls `t.Parallel()`
|
|
335
|
+
is published like any other.
|
|
336
|
+
|
|
337
|
+
A file Supercov cannot parse is a hole, not the end of the run. It is named on
|
|
338
|
+
the line that reports the result and recorded in the run, and a run is refused
|
|
339
|
+
only when nothing is left to measure.
|
|
292
340
|
|
|
293
341
|
Supercov also writes evidence as the suite runs, not only at the end. Go offers
|
|
294
342
|
no way to run code on `os.Exit`, and a `TestMain` need not reach the `m.Run()`
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "supercov",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "Code quality and coverage for coding agents",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -77,7 +77,7 @@
|
|
|
77
77
|
"test:test262": "cargo build --release -p supercov && node scripts/test262-equivalence.mjs",
|
|
78
78
|
"benchmark:check": "cargo build --release -p supercov && node scripts/rust-transform-benchmark.mjs",
|
|
79
79
|
"benchmark:python-monitoring": "cargo build -p supercov && node scripts/python-monitoring-benchmark.mjs",
|
|
80
|
-
"check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:runtime && npm run test:rust-assets && npm run test:assertion-maps && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
|
|
80
|
+
"check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:runtime && npm run test:rust-assets && npm run test:assertion-maps && npm run test:sweep-target && npm run test:skip-guard && npm run test:windows-tls && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
|
|
81
81
|
"release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:go && npm run test:jvm && npm run test:launcher && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python-monitoring && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
|
|
82
82
|
"release:bump": "node scripts/bump-version.mjs",
|
|
83
83
|
"sweep": "node scripts/sweep-target.mjs",
|
|
@@ -96,17 +96,21 @@
|
|
|
96
96
|
"sync:assertion-schema": "cargo build -p supercov && node scripts/assertion-map-schema.mjs",
|
|
97
97
|
"docs:sync": "node scripts/sync-docs.mjs",
|
|
98
98
|
"docs:check": "node scripts/sync-docs.mjs --check",
|
|
99
|
-
"test:launcher": "go vet ./cmd/... && go test ./cmd/..."
|
|
99
|
+
"test:launcher": "go vet ./cmd/... && go test ./cmd/...",
|
|
100
|
+
"test:sweep-target": "node scripts/sweep-target-test.mjs",
|
|
101
|
+
"test:go-versions": "cargo build -p supercov-engine --tests && node scripts/go-version-matrix.mjs",
|
|
102
|
+
"test:skip-guard": "node scripts/skip-guard-test.mjs",
|
|
103
|
+
"test:windows-tls": "node scripts/windows-tls-test.mjs"
|
|
100
104
|
},
|
|
101
105
|
"optionalDependencies": {
|
|
102
|
-
"@supercov/cli-darwin-arm64": "1.
|
|
103
|
-
"@supercov/cli-darwin-x64": "1.
|
|
104
|
-
"@supercov/cli-linux-arm64-gnu": "1.
|
|
105
|
-
"@supercov/cli-linux-arm64-musl": "1.
|
|
106
|
-
"@supercov/cli-linux-x64-gnu": "1.
|
|
107
|
-
"@supercov/cli-linux-x64-musl": "1.
|
|
108
|
-
"@supercov/cli-win32-arm64": "1.
|
|
109
|
-
"@supercov/cli-win32-x64": "1.
|
|
106
|
+
"@supercov/cli-darwin-arm64": "1.1.0",
|
|
107
|
+
"@supercov/cli-darwin-x64": "1.1.0",
|
|
108
|
+
"@supercov/cli-linux-arm64-gnu": "1.1.0",
|
|
109
|
+
"@supercov/cli-linux-arm64-musl": "1.1.0",
|
|
110
|
+
"@supercov/cli-linux-x64-gnu": "1.1.0",
|
|
111
|
+
"@supercov/cli-linux-x64-musl": "1.1.0",
|
|
112
|
+
"@supercov/cli-win32-arm64": "1.1.0",
|
|
113
|
+
"@supercov/cli-win32-x64": "1.1.0"
|
|
110
114
|
},
|
|
111
115
|
"peerDependencies": {
|
|
112
116
|
"@playwright/test": ">=1.55.0",
|