supercov 0.0.32 → 0.0.33
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -1
- package/docs/getting-started.md +6 -2
- package/docs/supported-suites.md +84 -1
- package/docs/troubleshooting.md +15 -0
- package/package.json +10 -9
package/README.md
CHANGED
|
@@ -109,11 +109,12 @@ The denominator comes from source structure before the run, so adding or removin
|
|
|
109
109
|
| TypeScript | Available | `npx supercov -- npm test` |
|
|
110
110
|
| Rust | Available | `npx supercov -- cargo test` |
|
|
111
111
|
| Python | Available | `npx supercov -- pytest` |
|
|
112
|
+
| Ruby | Available | `npx supercov -- rspec` |
|
|
112
113
|
| Zig | Coming soon | — |
|
|
113
114
|
| PHP | Coming soon | — |
|
|
114
115
|
| C | Coming soon | — |
|
|
115
116
|
|
|
116
|
-
Supercov requires Node.js 22 or newer. Rust support currently uses Rust 1.95; cargo-nextest 0.9.138 and 0.9.140 are supported. Python support requires CPython 3.12 or newer and measures pytest and unittest runs.
|
|
117
|
+
Supercov requires Node.js 22 or newer. Rust support currently uses Rust 1.95; cargo-nextest 0.9.138 and 0.9.140 are supported. Python support requires CPython 3.12 or newer and measures pytest and unittest runs. Ruby support requires Ruby 3.3 or newer (3.4 or newer for full measurement) and measures RSpec, Minitest, test-unit and Cucumber runs.
|
|
117
118
|
|
|
118
119
|
## Supported test suites
|
|
119
120
|
|
package/docs/getting-started.md
CHANGED
|
@@ -9,7 +9,7 @@ npx supercov -- npm test
|
|
|
9
9
|
```
|
|
10
10
|
|
|
11
11
|
No account, config file, import, custom reporter, or hosted service is required.
|
|
12
|
-
Supercov supports JavaScript, TypeScript, Rust, and
|
|
12
|
+
Supercov supports JavaScript, TypeScript, Rust, Python, and Ruby today.
|
|
13
13
|
|
|
14
14
|
## Before you start
|
|
15
15
|
|
|
@@ -18,7 +18,8 @@ You need:
|
|
|
18
18
|
- Node.js 22 or newer;
|
|
19
19
|
- a test command that already works in the repository; and
|
|
20
20
|
- for Rust, the Rust 1.95 toolchain;
|
|
21
|
-
- for Python, CPython 3.12 or newer with pytest or unittest
|
|
21
|
+
- for Python, CPython 3.12 or newer with pytest or unittest;
|
|
22
|
+
- for Ruby, Ruby 3.4 or newer with RSpec, Minitest, test-unit or Cucumber (3.3 measures lines, methods and simple branches only).
|
|
22
23
|
|
|
23
24
|
The CLI is distributed through npm, even for Rust projects. The first `npx`
|
|
24
25
|
invocation may download Supercov from the npm registry. Supercov itself does not
|
|
@@ -41,6 +42,9 @@ npx supercov -- cargo nextest run
|
|
|
41
42
|
|
|
42
43
|
# Python
|
|
43
44
|
npx supercov -- pytest
|
|
45
|
+
|
|
46
|
+
# Ruby
|
|
47
|
+
npx supercov -- rspec
|
|
44
48
|
```
|
|
45
49
|
|
|
46
50
|
Supercov runs that command in an isolated, instrumented copy of the project.
|
package/docs/supported-suites.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Supported languages and test suites
|
|
2
2
|
|
|
3
|
-
Supercov supports JavaScript, TypeScript, Rust, and
|
|
3
|
+
Supercov supports JavaScript, TypeScript, Rust, Python, and Ruby today. Start with
|
|
4
4
|
the same test command the repository already uses; Supercov detects supported
|
|
5
5
|
runners inside that command.
|
|
6
6
|
|
|
@@ -9,6 +9,7 @@ npx supercov -- npm test
|
|
|
9
9
|
npx supercov -- npx playwright test
|
|
10
10
|
npx supercov -- cargo test
|
|
11
11
|
npx supercov -- pytest
|
|
12
|
+
npx supercov -- rspec
|
|
12
13
|
```
|
|
13
14
|
|
|
14
15
|
## Language support
|
|
@@ -19,6 +20,7 @@ npx supercov -- pytest
|
|
|
19
20
|
| TypeScript | Available | `npx supercov -- npm test` |
|
|
20
21
|
| Rust | Available | `npx supercov -- cargo test` |
|
|
21
22
|
| Python | Available | `npx supercov -- pytest` |
|
|
23
|
+
| Ruby | Available | `npx supercov -- rspec` |
|
|
22
24
|
| Zig | Coming soon | — |
|
|
23
25
|
| PHP | Coming soon | — |
|
|
24
26
|
| C | Coming soon | — |
|
|
@@ -137,6 +139,87 @@ npx supercov -- uv run pytest
|
|
|
137
139
|
npx supercov -- python -m unittest
|
|
138
140
|
```
|
|
139
141
|
|
|
142
|
+
## Ruby
|
|
143
|
+
|
|
144
|
+
| Runner | Attribution | Current requirement |
|
|
145
|
+
| --- | --- | --- |
|
|
146
|
+
| RSpec | Exact example and before/example/after phase identity | Ruby 3.4 or newer for full measurement; run with `npx supercov -- rspec` or `bundle exec rspec` |
|
|
147
|
+
| Minitest (including Minitest::Spec and ActiveSupport::TestCase) | Exact test and setup/test/teardown identity; skips recorded | `ruby -Itest ...`, `rake test`, `rails test` |
|
|
148
|
+
| test-unit | Exact test and setup/test/teardown identity; omissions and pendings recorded | `ruby -Itest ...`, `rake test` |
|
|
149
|
+
| parallel_tests, Rails process workers | Exact per worker process | Workers inherit the run through `RUBYOPT`; verified on a Rails app with bootsnap, Zeitwerk and two forked workers |
|
|
150
|
+
| Thread-parallel Minitest (`parallelize_me!`, `parallelize(with: :threads)`) | Probe observations exact per test; line, method and simple-branch observations made while phases overlapped go to the run, declared | |
|
|
151
|
+
| Cucumber | Exact scenario identity (`features/x.feature:LINE`), hook steps as setup/teardown | `cucumber`, `bundle exec cucumber` |
|
|
152
|
+
|
|
153
|
+
Supercov measures Ruby with Ruby's own `Coverage` module plus probe calls it
|
|
154
|
+
splices into application files in memory as they load. Nothing on disk is
|
|
155
|
+
rewritten or copied; the project runs with its own interpreter and bundle, and
|
|
156
|
+
Supercov only adds a `-r` entry to `RUBYOPT` and a few `SUPERCOV_*` variables.
|
|
157
|
+
No insertion adds a line, so backtraces keep their line numbers.
|
|
158
|
+
|
|
159
|
+
Measured obligations are statements, method definitions, `if`/`unless`/
|
|
160
|
+
ternary/`while`/`until` decisions with MC/DC vectors over `&&`/`||` operands,
|
|
161
|
+
`while`/`until`/`for` iteration and the idiomatic iterator blocks (`each`,
|
|
162
|
+
`map`, `times`, `select`, ...), `&&`/`||`/`||=`/`&&=` short-circuiting,
|
|
163
|
+
`case`/`when`, `case`/`in` and `&.` selection, and `begin`/`rescue` completion,
|
|
164
|
+
handler selection and propagation. `||=` and `&&=` on method-call, index and
|
|
165
|
+
constant targets are exact too: an arrival probe and a right-side probe count
|
|
166
|
+
the skipped side without re-reading the target. Blocks and lambdas are
|
|
167
|
+
statements inside their methods, not function entry points, and that includes
|
|
168
|
+
a `define_method` block: Ruby's own method coverage reports one entry per
|
|
169
|
+
method it defines, but the block's body is measured statement by statement
|
|
170
|
+
instead of as a definition.
|
|
171
|
+
|
|
172
|
+
A statement on a line Ruby's own line table never counts (`x = case`, a
|
|
173
|
+
multi-line literal assignment, a bare `begin`, `if false`) gets a probe at load
|
|
174
|
+
time instead.
|
|
175
|
+
`if true`/`if false`/`if nil` and other literal predicates are folded the way
|
|
176
|
+
Ruby folds them: no branch, and the dead arm is not an obligation. A Spring
|
|
177
|
+
preloader started before the run has no hook and fails closed; JRuby and
|
|
178
|
+
TruffleRuby are not supported.
|
|
179
|
+
|
|
180
|
+
The runtime loads through `RUBYOPT` before Bundler and requires only
|
|
181
|
+
`coverage`, so it never activates a gem an application's Gemfile pins
|
|
182
|
+
differently. Insertions are checked against Ruby itself by a sweep
|
|
183
|
+
(`scripts/ruby-position-sweep.rb`) over Ruby's whole standard library and the
|
|
184
|
+
Rails, Rack, RSpec, Minitest and Cucumber gems, about 3,000 files: every file
|
|
185
|
+
is transformed and compiled, and every position Supercov expects is compared
|
|
186
|
+
with what Ruby reports. With `--load` each file also runs twice, untouched and
|
|
187
|
+
transformed, so the probes are proven to preserve behaviour and every method
|
|
188
|
+
position is checked.
|
|
189
|
+
|
|
190
|
+
A `begin` whose body ends in an expression that can `return` from inside
|
|
191
|
+
itself has its handlers and propagation measured as usual, but its normal
|
|
192
|
+
completion is declared instead of probed unless every branch of that
|
|
193
|
+
expression can carry the probe: Ruby cannot pass such an expression as an
|
|
194
|
+
argument, which is what a probe wrapper does.
|
|
195
|
+
|
|
196
|
+
Measuring never breaks the program being measured. If a file cannot be
|
|
197
|
+
compiled with its probes, it loads unmodified: Ruby's `Coverage` still
|
|
198
|
+
measures its lines, methods and own branches, and only the obligations a
|
|
199
|
+
probe would have proven are declared for that file. Setting
|
|
200
|
+
`SUPERCOV_RUBY_SKIP_PROBES` to a comma-separated list of path fragments puts
|
|
201
|
+
chosen files on that same path deliberately, which is the escape hatch if
|
|
202
|
+
instrumentation ever disagrees with one of yours.
|
|
203
|
+
|
|
204
|
+
An obligation Supercov declares unmeasured leaves the obligation totals and is
|
|
205
|
+
reported as a measurement limit rather than a gap. Its line still counts in
|
|
206
|
+
the line total, as it does for every other language, so a file with declared
|
|
207
|
+
obligations reads slightly pessimistically on lines.
|
|
208
|
+
|
|
209
|
+
Ruby 3.3 does not apply its `Coverage` module to code compiled by a load hook,
|
|
210
|
+
so on 3.3 Supercov measures through `Coverage` alone: lines, methods and the
|
|
211
|
+
branches Ruby reports itself. Everything that needs a probe (multi-condition
|
|
212
|
+
decisions, `||=`, loops, `rescue` flow, a second statement on a line) is
|
|
213
|
+
declared unmeasured on that interpreter rather than shown as a gap. Ruby 3.4
|
|
214
|
+
and newer measure everything.
|
|
215
|
+
|
|
216
|
+
```sh
|
|
217
|
+
npx supercov -- rspec
|
|
218
|
+
npx supercov -- bundle exec rspec
|
|
219
|
+
npx supercov -- ruby -Itest test/shapes_test.rb
|
|
220
|
+
npx supercov -- bin/rails test
|
|
221
|
+
```
|
|
222
|
+
|
|
140
223
|
## Containers, VMs, and remote execution
|
|
141
224
|
|
|
142
225
|
Supercov can collect from supported processes launched through a container, VM,
|
package/docs/troubleshooting.md
CHANGED
|
@@ -60,6 +60,21 @@ Compare the command with [Supported suites](supported-suites.md). If the runner
|
|
|
60
60
|
should be supported, preserve the summary and exact command when reporting the
|
|
61
61
|
problem.
|
|
62
62
|
|
|
63
|
+
## A Ruby file behaves differently while Supercov measures it
|
|
64
|
+
|
|
65
|
+
Supercov splices probe calls into Ruby sources in memory as they load, and a
|
|
66
|
+
file that cannot be compiled with them is measured through Ruby's `Coverage`
|
|
67
|
+
module alone rather than failing the run. To put a file on that path
|
|
68
|
+
deliberately, name a fragment of its path:
|
|
69
|
+
|
|
70
|
+
```sh
|
|
71
|
+
SUPERCOV_RUBY_SKIP_PROBES=app/models/order.rb npx supercov -- bundle exec rspec
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Its lines, methods and simple branches stay measured; everything that needs a
|
|
75
|
+
probe is declared as a measurement limit for that file. Please report the file,
|
|
76
|
+
since Supercov aims to instrument every Ruby source correctly.
|
|
77
|
+
|
|
63
78
|
## A run is marked stale
|
|
64
79
|
|
|
65
80
|
A stored run remains valid history, but it stops describing the current
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "supercov",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.33",
|
|
4
4
|
"description": "Zero-edit, runner-aware coverage completeness for JavaScript, TypeScript, Rust, and Python test suites",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -64,18 +64,19 @@
|
|
|
64
64
|
"benchmark:check": "cargo build --release -p supercov && node scripts/rust-transform-benchmark.mjs",
|
|
65
65
|
"benchmark:python-monitoring": "cargo build -p supercov && node scripts/python-monitoring-benchmark.mjs",
|
|
66
66
|
"check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:runtime && npm run test:rust-assets && node scripts/package-preflight.mjs",
|
|
67
|
-
"release:check": "npm run check && npm run test:engine && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:python-monitoring && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
|
|
67
|
+
"release:check": "npm run check && npm run test:engine && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:python-monitoring && npm run test:ruby-coverage && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
|
|
68
68
|
"prepack": "node scripts/package-preflight.mjs",
|
|
69
69
|
"prepublishOnly": "npm run release:check",
|
|
70
|
-
"test:python-monitoring": "cargo build -p supercov && node scripts/python-monitoring-integration.mjs"
|
|
70
|
+
"test:python-monitoring": "cargo build -p supercov && node scripts/python-monitoring-integration.mjs",
|
|
71
|
+
"test:ruby-coverage": "cargo build -p supercov && node scripts/ruby-coverage-integration.mjs"
|
|
71
72
|
},
|
|
72
73
|
"optionalDependencies": {
|
|
73
|
-
"@supercov/cli-darwin-arm64": "0.0.
|
|
74
|
-
"@supercov/cli-darwin-x64": "0.0.
|
|
75
|
-
"@supercov/cli-linux-arm64-gnu": "0.0.
|
|
76
|
-
"@supercov/cli-linux-arm64-musl": "0.0.
|
|
77
|
-
"@supercov/cli-linux-x64-gnu": "0.0.
|
|
78
|
-
"@supercov/cli-linux-x64-musl": "0.0.
|
|
74
|
+
"@supercov/cli-darwin-arm64": "0.0.33",
|
|
75
|
+
"@supercov/cli-darwin-x64": "0.0.33",
|
|
76
|
+
"@supercov/cli-linux-arm64-gnu": "0.0.33",
|
|
77
|
+
"@supercov/cli-linux-arm64-musl": "0.0.33",
|
|
78
|
+
"@supercov/cli-linux-x64-gnu": "0.0.33",
|
|
79
|
+
"@supercov/cli-linux-x64-musl": "0.0.33"
|
|
79
80
|
},
|
|
80
81
|
"peerDependencies": {
|
|
81
82
|
"@playwright/test": ">=1.55.0",
|