supercov 4.8.1 → 4.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/cli.md +8 -5
- package/docs/coverage-model.md +13 -10
- package/docs/supported-suites.md +78 -0
- package/docs/troubleshooting.md +5 -0
- package/package.json +10 -10
package/docs/cli.md
CHANGED
|
@@ -632,11 +632,14 @@ what that run's tests did with it:
|
|
|
632
632
|
- A helper, fixture or support file, or source outside what is measured (a
|
|
633
633
|
simulator kept out of `SUPERCOV_SOURCE_ROOTS`): left out of a run only when
|
|
634
634
|
that run's test files import it, directly or through other files. Supercov
|
|
635
|
-
reads the imports of JavaScript, TypeScript and
|
|
636
|
-
`conftest.py` files and `pytest_plugins
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
635
|
+
reads the imports of JavaScript, TypeScript, Python and Ruby tests, with
|
|
636
|
+
pytest's `conftest.py` files and `pytest_plugins`, and what `.rspec`
|
|
637
|
+
requires; a computed import, a glob of support files, a change to
|
|
638
|
+
`sys.path` or `$LOAD_PATH`, or tests in another language count every helper
|
|
639
|
+
as loaded. A JavaScript, TypeScript or Python helper that only gained
|
|
640
|
+
functions, which code written before cannot have called, leaves every run
|
|
641
|
+
in. In Python that is a plain `def`: a decorator runs when the module
|
|
642
|
+
loads.
|
|
640
643
|
- A test runner's settings: `playwright.config.*`, `vitest.config.*`,
|
|
641
644
|
`jest.config.*` and `ava.config.*` belong to their runner, and a run that
|
|
642
645
|
did not use that runner keeps. Of Playwright's settings, only those that
|
package/docs/coverage-model.md
CHANGED
|
@@ -110,16 +110,19 @@ By test kind
|
|
|
110
110
|
directory has any, so a directory's `All` never stands above its kinds
|
|
111
111
|
unexplained.
|
|
112
112
|
|
|
113
|
-
A request to your app server that names no test -- a
|
|
114
|
-
Stripe server Supercov does not measure -- is credited to
|
|
115
|
-
the server handled it, if that test was the only one
|
|
116
|
-
server serves one worker's tests, as a server started per
|
|
117
|
-
that worker's tests count, so the request is credited while
|
|
118
|
-
theirs. A mock that runs under Supercov passes the test's
|
|
119
|
-
own requests, and its webhook is credited exactly. The run's
|
|
120
|
-
line counts tests credited by time as `with server work credited
|
|
121
|
-
each one's `runs <id> test` page says so.
|
|
122
|
-
|
|
113
|
+
A request to your app server, Node, Python or Ruby, that names no test -- a
|
|
114
|
+
webhook from a mock Stripe server Supercov does not measure -- is credited to
|
|
115
|
+
the test running when the server handled it, if that test was the only one
|
|
116
|
+
running then. Where a server serves one worker's tests, as a server started per
|
|
117
|
+
worker does, only that worker's tests count, so the request is credited while
|
|
118
|
+
other workers run theirs. A mock that runs under Supercov passes the test's
|
|
119
|
+
identity on in its own requests, and its webhook is credited exactly. The run's
|
|
120
|
+
`Attribution` line counts tests credited by time as `with server work credited
|
|
121
|
+
by time`, and each one's `runs <id> test` page says so. A server the suite runs
|
|
122
|
+
in its own process -- a pytest live server, Capybara's Puma under a Rails
|
|
123
|
+
system test -- is judged by which of the suite's tests that process was
|
|
124
|
+
running. Work done while tests ran at once on the same server, or while none
|
|
125
|
+
ran, stays no test's.
|
|
123
126
|
|
|
124
127
|
A test that made assertions and has no covered line to its name is counted in
|
|
125
128
|
a warning on the summary. `npx supercov runs latest tests without-evidence`
|
package/docs/supported-suites.md
CHANGED
|
@@ -224,6 +224,45 @@ npx supercov -- uv run pytest
|
|
|
224
224
|
npx supercov -- python -m unittest
|
|
225
225
|
```
|
|
226
226
|
|
|
227
|
+
### A Python server under browser tests
|
|
228
|
+
|
|
229
|
+
A Playwright suite that starts a Python server, through `webServer`, a fixture
|
|
230
|
+
or a test, measures the server when you run it under Supercov:
|
|
231
|
+
|
|
232
|
+
```sh
|
|
233
|
+
npx supercov -- npx playwright test
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
Each request runs as the test that sent it. Supercov's Playwright runtime names
|
|
237
|
+
the test in a header on the page's own requests and the request fixture's, and
|
|
238
|
+
in a cookie on the app's host; the server reads it in the standard library's
|
|
239
|
+
HTTP server, in any WSGI server running Flask or Django (gunicorn, waitress,
|
|
240
|
+
their own development servers) and in any ASGI server running Starlette,
|
|
241
|
+
FastAPI or Django (uvicorn, hypercorn). A request that names no test, such as
|
|
242
|
+
a webhook from a mock, is credited by time as it is for a Node server; see the
|
|
243
|
+
[coverage model](coverage-model.md). A Python process a test starts belongs to
|
|
244
|
+
that test.
|
|
245
|
+
|
|
246
|
+
The server's coverage is a Python run of its own. In a project with no
|
|
247
|
+
JavaScript source that run is the run. In one with a Node server or front end
|
|
248
|
+
too, the browser run is published as usual and the server's run beside it.
|
|
249
|
+
`runs latest` is the browser run, and its summary names the server's run
|
|
250
|
+
(`companions` in JSON); the server's run names the browser run it took its
|
|
251
|
+
tests from. The pull-request comment Action reports each server's run in a
|
|
252
|
+
section of its own. Each run's figures stand alone, and `supercov merge`
|
|
253
|
+
merges each language's runs separately.
|
|
254
|
+
|
|
255
|
+
A server the suite runs in its own process works without Playwright: a
|
|
256
|
+
pytest-django `live_server`, Django's `LiveServerTestCase`, or a Flask app in
|
|
257
|
+
a thread that a Selenium or pytest-playwright suite drives. Its requests name
|
|
258
|
+
no test, and each is credited to the test that was running in that process
|
|
259
|
+
when it came, and said to be.
|
|
260
|
+
|
|
261
|
+
The server runs in Supercov's copy of the project. A virtual environment
|
|
262
|
+
inside the project is used as it is, and a package installed editable from the
|
|
263
|
+
checkout is measured too. `SUPERCOV_SOURCE_ROOTS` applies to both languages:
|
|
264
|
+
when you set it, list the Python directories as well.
|
|
265
|
+
|
|
227
266
|
## Ruby
|
|
228
267
|
|
|
229
268
|
| Runner | Attribution | Current requirement |
|
|
@@ -280,6 +319,26 @@ npx supercov -- ruby -Itest test/shapes_test.rb
|
|
|
280
319
|
npx supercov -- bin/rails test
|
|
281
320
|
```
|
|
282
321
|
|
|
322
|
+
### A Ruby server under browser tests
|
|
323
|
+
|
|
324
|
+
A Playwright suite that starts a Ruby server, a Rails, Sinatra or other Rack
|
|
325
|
+
app under Puma, WEBrick or another server, measures it the way it measures a
|
|
326
|
+
[Python server](#a-python-server-under-browser-tests): each request runs as the
|
|
327
|
+
test that sent it, a request that names no test is credited by time, a Ruby
|
|
328
|
+
process a test starts belongs to that test, and the server's coverage is a Ruby
|
|
329
|
+
run of its own, published as the run or beside the browser run. Gems installed
|
|
330
|
+
into the project (`bundle config set path vendor/bundle`) are used from the
|
|
331
|
+
copy the suite runs in.
|
|
332
|
+
|
|
333
|
+
A Rails system test runs its app in Capybara's Puma, in the test process: each
|
|
334
|
+
request is credited to the test that was running when it came, and said to
|
|
335
|
+
be.
|
|
336
|
+
|
|
337
|
+
```sh
|
|
338
|
+
npx supercov -- npx playwright test
|
|
339
|
+
npx supercov -- bin/rails test:system
|
|
340
|
+
```
|
|
341
|
+
|
|
283
342
|
## Crediting a test with what it reached
|
|
284
343
|
|
|
285
344
|
Most runners attribute exactly at no cost: Supercov runs each Rust test in its
|
|
@@ -362,6 +421,25 @@ npx supercov -- go test -run TestParser ./internal/...
|
|
|
362
421
|
npx supercov -- go test ./core/... ./app/...
|
|
363
422
|
```
|
|
364
423
|
|
|
424
|
+
### A Go server under browser tests
|
|
425
|
+
|
|
426
|
+
A Playwright suite that starts a Go server with `go run` measures it: the copy
|
|
427
|
+
the suite runs in has its Go rewritten with probes, as for `go test`, and the
|
|
428
|
+
server is built from that copy. Go has no per-request context Supercov can
|
|
429
|
+
carry a test's name in to every probe, so the server sweeps what ran every
|
|
430
|
+
5 ms, and each sweep is credited to the browser test that was running then,
|
|
431
|
+
if only one was, and said to be. Work done while two tests ran at once, as
|
|
432
|
+
with parallel workers, is the run's and no test's. The server's coverage is a
|
|
433
|
+
Go run of its own, published as the run or beside the browser run, as for a
|
|
434
|
+
[Python server](#a-python-server-under-browser-tests).
|
|
435
|
+
|
|
436
|
+
A server binary built before the run is not the one Supercov instruments:
|
|
437
|
+
start it with `go run`, or build it in the test command.
|
|
438
|
+
|
|
439
|
+
```sh
|
|
440
|
+
npx supercov -- npx playwright test
|
|
441
|
+
```
|
|
442
|
+
|
|
365
443
|
## Java and Kotlin
|
|
366
444
|
|
|
367
445
|
| Runner | Attribution | Current requirement |
|
package/docs/troubleshooting.md
CHANGED
|
@@ -42,6 +42,11 @@ npx supercov runs latest scope
|
|
|
42
42
|
Do not broaden the roots to dependencies or generated output merely to remove a
|
|
43
43
|
warning. The goal is an honest boundary around code the repository owns.
|
|
44
44
|
|
|
45
|
+
Browser tests of a Python or Ruby server need a Playwright configuration in
|
|
46
|
+
the project, and the server's source under the roots: Supercov then measures
|
|
47
|
+
the server the suite starts. See
|
|
48
|
+
[Supported suites](supported-suites.md#a-python-server-under-browser-tests).
|
|
49
|
+
|
|
45
50
|
A file `scope` lists as `test or fixture source` or `conventional tool script`
|
|
46
51
|
was recognised by its path. If it is your code, name its directory, or the file
|
|
47
52
|
itself, as a root: a named root is source whatever it is called.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "supercov",
|
|
3
|
-
"version": "4.
|
|
3
|
+
"version": "4.9.0",
|
|
4
4
|
"description": "Coverage, security and code quality for coding agents",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"coverage",
|
|
@@ -83,7 +83,7 @@
|
|
|
83
83
|
"eval:skill": "claude plugin eval plugins/supercov --scaffold --trust-plugin --no-publish --model claude-sonnet-5 --judge-model claude-haiku-4-5 --threshold 0.66 --tag behavior trigger --allow-tools Bash Edit Write --max-cost-usd 15",
|
|
84
84
|
"eval:skill-triggers": "claude plugin eval plugins/supercov --scaffold --trust-plugin --no-publish --model claude-sonnet-5 --judge-model claude-haiku-4-5 --threshold 0.66 --tag trigger --ablation none --max-cost-usd 5",
|
|
85
85
|
"test:engine": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace && npm run test:runtime && npm run test:rust-assets && npm run test:engine-integrations",
|
|
86
|
-
"test:engine-integrations": "cargo build -p supercov && node scripts/html-report-integration.mjs && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/mocha-failure-output.mjs && node scripts/failure-output.mjs node ava tap && node scripts/side-by-side.mjs node && node scripts/merge-across.mjs && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/cross-origin-requests.mjs && node scripts/background-by-time.mjs && node scripts/rust-classic-browser-integration.mjs && node scripts/rust-top-packages-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
|
|
86
|
+
"test:engine-integrations": "cargo build -p supercov && node scripts/html-report-integration.mjs && node scripts/rust-process-supervision.mjs && node scripts/rust-direct-node-integration.mjs && node scripts/rust-public-run-integration.mjs && node scripts/rust-embedded-runtime-integration.mjs && node scripts/rust-direct-vitest-integration.mjs && node scripts/rust-direct-jest-integration.mjs && node scripts/mocha-failure-output.mjs && node scripts/failure-output.mjs node ava tap && node scripts/side-by-side.mjs node && node scripts/merge-across.mjs && node scripts/rust-direct-playwright-integration.mjs && node scripts/rust-custom-browser-playwright-integration.mjs && node scripts/cross-origin-requests.mjs && node scripts/background-by-time.mjs && node scripts/python-server-playwright.mjs && node scripts/ruby-server-playwright.mjs && node scripts/go-server-playwright.mjs && node scripts/rust-classic-browser-integration.mjs && node scripts/rust-top-packages-integration.mjs && node scripts/rust-generic-esbuild-integration.mjs && node scripts/rust-generic-tsc-integration.mjs && node scripts/rust-generic-build-matrix.mjs && node scripts/rust-vite-playwright-integration.mjs",
|
|
87
87
|
"test:platform": "cargo test --workspace && cargo build -p supercov && node scripts/rust-process-supervision.mjs && node scripts/workspace-crash-integration.mjs",
|
|
88
88
|
"test:native-package": "cargo build --release -p supercov && node scripts/native-package-integration.mjs && node scripts/native-release-set-integration.mjs",
|
|
89
89
|
"test:pypi-wheel": "node scripts/pypi-wheel-integration.mjs",
|
|
@@ -120,14 +120,14 @@
|
|
|
120
120
|
"test:windows-tls": "node scripts/windows-tls-test.mjs"
|
|
121
121
|
},
|
|
122
122
|
"optionalDependencies": {
|
|
123
|
-
"@supercov/cli-darwin-arm64": "4.
|
|
124
|
-
"@supercov/cli-darwin-x64": "4.
|
|
125
|
-
"@supercov/cli-linux-arm64-gnu": "4.
|
|
126
|
-
"@supercov/cli-linux-arm64-musl": "4.
|
|
127
|
-
"@supercov/cli-linux-x64-gnu": "4.
|
|
128
|
-
"@supercov/cli-linux-x64-musl": "4.
|
|
129
|
-
"@supercov/cli-win32-arm64": "4.
|
|
130
|
-
"@supercov/cli-win32-x64": "4.
|
|
123
|
+
"@supercov/cli-darwin-arm64": "4.9.0",
|
|
124
|
+
"@supercov/cli-darwin-x64": "4.9.0",
|
|
125
|
+
"@supercov/cli-linux-arm64-gnu": "4.9.0",
|
|
126
|
+
"@supercov/cli-linux-arm64-musl": "4.9.0",
|
|
127
|
+
"@supercov/cli-linux-x64-gnu": "4.9.0",
|
|
128
|
+
"@supercov/cli-linux-x64-musl": "4.9.0",
|
|
129
|
+
"@supercov/cli-win32-arm64": "4.9.0",
|
|
130
|
+
"@supercov/cli-win32-x64": "4.9.0"
|
|
131
131
|
},
|
|
132
132
|
"peerDependencies": {
|
|
133
133
|
"@playwright/test": ">=1.55.0",
|