supercov 0.0.52 → 0.0.54
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +12 -6
- package/docs/supported-suites.md +25 -1
- package/package.json +9 -9
- package/runtime/javascript/runtime.mjs +43 -6
- package/runtime/javascript/vitestBrowser.mjs +107 -0
package/README.md
CHANGED
|
@@ -136,17 +136,23 @@ The denominator comes from source structure before the run, so adding or removin
|
|
|
136
136
|
|
|
137
137
|
## Install for your language
|
|
138
138
|
|
|
139
|
-
|
|
139
|
+
Measuring a project should not start by installing another language. The same binary, at the same version, from one release:
|
|
140
140
|
|
|
141
141
|
```bash
|
|
142
|
-
npx supercov -- npm test
|
|
143
|
-
uvx --from supercov-cli supercov -- pytest
|
|
144
|
-
gem install supercov && supercov -- bundle exec rspec
|
|
145
|
-
cargo binstall supercov && supercov -- cargo test
|
|
142
|
+
npx supercov -- npm test # npm
|
|
143
|
+
uvx --from supercov-cli supercov -- pytest # PyPI
|
|
144
|
+
gem install supercov && supercov -- bundle exec rspec # RubyGems
|
|
145
|
+
cargo binstall supercov && supercov -- cargo test # crates.io
|
|
146
|
+
go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./... # Go
|
|
147
|
+
brew install supercorp-ai/tap/supercov && supercov -- ./gradlew test # Homebrew
|
|
146
148
|
```
|
|
147
149
|
|
|
148
150
|
`pip install supercov-cli` and `gem install supercov` install a wheel or gem that carries the binary for your platform; nothing is compiled. `cargo binstall` downloads that same binary from the GitHub release, while plain `cargo install supercov` builds it from source and needs Rust 1.95.
|
|
149
151
|
|
|
152
|
+
A Go project needs Go and nothing else. Like any `go run` with a version suffix it resolves by module path and ignores the `go.mod` in your current directory, so it neither needs nor touches your module.
|
|
153
|
+
|
|
154
|
+
Java and Kotlin have no registry of their own here, so a JVM project takes the binary directly -- Homebrew above, `npx` if Node is already present, or the platform archive from the [latest release](https://github.com/supercorp-ai/supercov/releases/latest). Maven and Gradle are driven as your test command, not as a plugin. Supercov adds the JUnit Platform launcher its measurement needs to the build file inside its own isolated workspace copy -- your `pom.xml` or `build.gradle` is never edited.
|
|
155
|
+
|
|
150
156
|
## Supported languages
|
|
151
157
|
|
|
152
158
|
| Language | Status | Start with |
|
|
@@ -156,7 +162,7 @@ cargo binstall supercov && supercov -- cargo test # crates.io
|
|
|
156
162
|
| Rust | Available | `npx supercov -- cargo test` |
|
|
157
163
|
| Python | Available | `npx supercov -- pytest` |
|
|
158
164
|
| Ruby | Available | `npx supercov -- rspec` |
|
|
159
|
-
| Go | Available | `
|
|
165
|
+
| Go | Available | `go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./...` |
|
|
160
166
|
| Java | Available | `npx supercov -- mvn test` |
|
|
161
167
|
| Kotlin | Available | `npx supercov -- ./gradlew test` |
|
|
162
168
|
| Zig | Coming soon | — |
|
package/docs/supported-suites.md
CHANGED
|
@@ -49,7 +49,7 @@ Supercov reports the level it actually observed. It does not guess.
|
|
|
49
49
|
| Runner | Attribution |
|
|
50
50
|
| --- | --- |
|
|
51
51
|
| Playwright | Exact per test, worker, retry, outcome, action, and assertion phase |
|
|
52
|
-
| Vitest | Exact per test, with setup execution kept separate |
|
|
52
|
+
| Vitest | Exact per test, with setup execution kept separate; Browser Mode included |
|
|
53
53
|
| Jest | Exact per test, including parameterized tests, with the user's own configuration, setup files and reporters kept; passing `expect` occurrences are identified for assertion maps |
|
|
54
54
|
| `node:test` | Exact per test |
|
|
55
55
|
| AVA and Mocha | Aggregate structural coverage |
|
|
@@ -59,6 +59,30 @@ Supercov reports the level it actually observed. It does not guess.
|
|
|
59
59
|
One command may launch several runners. Supercov combines their evidence into
|
|
60
60
|
one run and preserves runner identity wherever the runner exposes it.
|
|
61
61
|
|
|
62
|
+
### Vitest Browser Mode
|
|
63
|
+
|
|
64
|
+
Vitest Browser Mode runs the test file in a real browser. Supercov measures it
|
|
65
|
+
per test like any other Vitest run: lines, branches, MC/DC and assertion
|
|
66
|
+
coverage, with no extra configuration. Every provider is supported, because
|
|
67
|
+
evidence travels over Vitest's own browser command channel rather than anything
|
|
68
|
+
provider-specific.
|
|
69
|
+
|
|
70
|
+
Component code is instrumented the same way as any other source, with one
|
|
71
|
+
addition that matters most here. A JSX tree is a single statement, so an
|
|
72
|
+
expression rendered inside it -- `aria-label={label(state)}`, a child
|
|
73
|
+
`{formatted(value)}` -- is measured on its own rather than counted as covered
|
|
74
|
+
because the component rendered once. This is also what lets a UI assertion be
|
|
75
|
+
explained precisely: `toHaveAccessibleName` names the attribute and
|
|
76
|
+
`toHaveTextContent` names the child, and each is credited separately.
|
|
77
|
+
|
|
78
|
+
Expressions that cannot independently fail to evaluate do not become
|
|
79
|
+
obligations. `{value}` is reached exactly when the tree is, and
|
|
80
|
+
`onClick={() => save()}` is measured where the handler is called rather than
|
|
81
|
+
where it is created.
|
|
82
|
+
|
|
83
|
+
`expect.element(...)`, `expect.soft(...)` and `expect.poll(...)` are recognised
|
|
84
|
+
as assertions, so their passing occurrences are available to assertion maps.
|
|
85
|
+
|
|
62
86
|
### Builds and source formats
|
|
63
87
|
|
|
64
88
|
JavaScript and TypeScript projects may use Vite, Next, Turbopack, Webpack,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "supercov",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.54",
|
|
4
4
|
"description": "Coverage for coding agents and software factories \ud83c\udf19",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -97,14 +97,14 @@
|
|
|
97
97
|
"test:launcher": "go vet ./cmd/... && go test ./cmd/..."
|
|
98
98
|
},
|
|
99
99
|
"optionalDependencies": {
|
|
100
|
-
"@supercov/cli-darwin-arm64": "0.0.
|
|
101
|
-
"@supercov/cli-darwin-x64": "0.0.
|
|
102
|
-
"@supercov/cli-linux-arm64-gnu": "0.0.
|
|
103
|
-
"@supercov/cli-linux-arm64-musl": "0.0.
|
|
104
|
-
"@supercov/cli-linux-x64-gnu": "0.0.
|
|
105
|
-
"@supercov/cli-linux-x64-musl": "0.0.
|
|
106
|
-
"@supercov/cli-win32-arm64": "0.0.
|
|
107
|
-
"@supercov/cli-win32-x64": "0.0.
|
|
100
|
+
"@supercov/cli-darwin-arm64": "0.0.54",
|
|
101
|
+
"@supercov/cli-darwin-x64": "0.0.54",
|
|
102
|
+
"@supercov/cli-linux-arm64-gnu": "0.0.54",
|
|
103
|
+
"@supercov/cli-linux-arm64-musl": "0.0.54",
|
|
104
|
+
"@supercov/cli-linux-x64-gnu": "0.0.54",
|
|
105
|
+
"@supercov/cli-linux-x64-musl": "0.0.54",
|
|
106
|
+
"@supercov/cli-win32-arm64": "0.0.54",
|
|
107
|
+
"@supercov/cli-win32-x64": "0.0.54"
|
|
108
108
|
},
|
|
109
109
|
"peerDependencies": {
|
|
110
110
|
"@playwright/test": ">=1.55.0",
|
|
@@ -751,8 +751,19 @@ function withNodeAssertionPhase(operation, source, callback) {
|
|
|
751
751
|
const scope = context.scope;
|
|
752
752
|
if (!scope)
|
|
753
753
|
return callback();
|
|
754
|
-
|
|
755
|
-
|
|
754
|
+
// A lexically instrumented occurrence is an authored assertion site in its
|
|
755
|
+
// own right, even when it runs inside another assertion's callback: the
|
|
756
|
+
// validator passed to assert.throws is the ordinary case, and its inner
|
|
757
|
+
// assertions are what claim *what* the error says. Skipping every nested
|
|
758
|
+
// phase left those with no passing occurrence, so they could never earn
|
|
759
|
+
// credit however carefully they were mapped.
|
|
760
|
+
//
|
|
761
|
+
// What must still be skipped is the same call seen twice -- the module proxy
|
|
762
|
+
// firing inside the lexical wrapper. That path has no lexical source and
|
|
763
|
+
// arrives as a lazy stack fallback, so the shape of `source` tells them
|
|
764
|
+
// apart without comparing coordinates the transform has moved.
|
|
765
|
+
const nested = context.phaseId && assertionPhaseState(scope).phaseIds.has(context.phaseId);
|
|
766
|
+
if (nested && typeof source !== "string")
|
|
756
767
|
return callback();
|
|
757
768
|
// A lexical occurrence already identifies its source. Stack fallback is lazy
|
|
758
769
|
// and qualified so transformed coordinates cannot impersonate original ones.
|
|
@@ -1196,6 +1207,15 @@ function selectionEnd(frame, value) {
|
|
|
1196
1207
|
}
|
|
1197
1208
|
return value;
|
|
1198
1209
|
}
|
|
1210
|
+
// A JSX expression container has nowhere to put a statement and cannot hold a
|
|
1211
|
+
// bare comma operator, so a rendered expression carries its probe as a call
|
|
1212
|
+
// that returns the value through. Like optionalSelect, it records after the
|
|
1213
|
+
// value evaluates: an expression that throws is reported unevaluated rather
|
|
1214
|
+
// than covered.
|
|
1215
|
+
function renderedValueV2(file, index, value) {
|
|
1216
|
+
coverageHitV2(file, index);
|
|
1217
|
+
return value;
|
|
1218
|
+
}
|
|
1199
1219
|
function optionalSelect(shortId, continuedId, value) {
|
|
1200
1220
|
coverageHit(value === null || value === void 0 ? shortId : continuedId);
|
|
1201
1221
|
return value;
|
|
@@ -1244,20 +1264,35 @@ function tryBegin(successId, catchId) {
|
|
|
1244
1264
|
return { successId, catchId, caught: false };
|
|
1245
1265
|
}
|
|
1246
1266
|
function tryCatch(frame, value) {
|
|
1247
|
-
|
|
1267
|
+
// Record the outcome the moment it is known, not when the construct is left.
|
|
1268
|
+
// The commit used to sit in the generated `finally`, and `process.exit()` in
|
|
1269
|
+
// a catch body skips `finally` -- so a catch that ran and whose statements
|
|
1270
|
+
// were credited was still reported as never entered. Statement probes fire
|
|
1271
|
+
// in place and survived, which is what made the report contradict itself.
|
|
1272
|
+
if (!frame.caught) {
|
|
1273
|
+
frame.caught = true;
|
|
1274
|
+
coverageHit(frame.catchId);
|
|
1275
|
+
}
|
|
1248
1276
|
return value;
|
|
1249
1277
|
}
|
|
1250
1278
|
function tryEnd(frame) {
|
|
1251
|
-
|
|
1279
|
+
// Completing without catching is only knowable here.
|
|
1280
|
+
if (!frame.caught) coverageHit(frame.successId);
|
|
1252
1281
|
}
|
|
1253
1282
|
function loopBegin(zeroId, enteredId) {
|
|
1254
1283
|
return { zeroId, enteredId, entered: false };
|
|
1255
1284
|
}
|
|
1256
1285
|
function loopEntered(frame) {
|
|
1257
|
-
|
|
1286
|
+
// Same reason as tryCatch: a loop body that exits the process would
|
|
1287
|
+
// otherwise report zero iterations. Guarded so a loop still counts once,
|
|
1288
|
+
// however many times it iterates.
|
|
1289
|
+
if (!frame.entered) {
|
|
1290
|
+
frame.entered = true;
|
|
1291
|
+
coverageHit(frame.enteredId);
|
|
1292
|
+
}
|
|
1258
1293
|
}
|
|
1259
1294
|
function loopEnd(frame) {
|
|
1260
|
-
|
|
1295
|
+
if (!frame.entered) coverageHit(frame.zeroId);
|
|
1261
1296
|
}
|
|
1262
1297
|
function mcdcBegin(id, meta) {
|
|
1263
1298
|
if (!state.decisions.has(id)) {
|
|
@@ -1330,6 +1365,7 @@ const directRuntimeApi = {
|
|
|
1330
1365
|
optionalCallEnd,
|
|
1331
1366
|
optionalCallReached,
|
|
1332
1367
|
optionalSelect,
|
|
1368
|
+
renderedValueV2,
|
|
1333
1369
|
parenthesizedAssignmentValue,
|
|
1334
1370
|
phaseBelongsToAttempt,
|
|
1335
1371
|
registerProbeV2,
|
|
@@ -1380,6 +1416,7 @@ export {
|
|
|
1380
1416
|
optionalCallEnd,
|
|
1381
1417
|
optionalCallReached,
|
|
1382
1418
|
optionalSelect,
|
|
1419
|
+
renderedValueV2,
|
|
1383
1420
|
parenthesizedAssignmentValue,
|
|
1384
1421
|
phaseBelongsToAttempt,
|
|
1385
1422
|
registerProbeV2,
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
// Vitest Browser Mode runs the test file in the browser, so the node setup
|
|
2
|
+
// cannot be used: it imports node:fs through atomic.mjs, and Vite externalises
|
|
3
|
+
// that for the client. Loading it fails the whole suite before a single test
|
|
4
|
+
// collects, which is what a browser-mode project saw.
|
|
5
|
+
//
|
|
6
|
+
// This setup mirrors the node one's payload contract exactly. What it cannot do
|
|
7
|
+
// in the browser -- resolve a path against the project root, infer provenance,
|
|
8
|
+
// write a file -- it hands to node over Vitest's browser command channel, so
|
|
9
|
+
// that contract has one implementation.
|
|
10
|
+
import { afterEach, beforeEach } from "vitest";
|
|
11
|
+
import { coverageSnapshot, activateCoverageScope, enableRuntimeSnapshotEvidence, resetCoverage, takeNodeAssertionPhases, } from "./runtime.mjs";
|
|
12
|
+
// Vitest exposes the command channel at "vitest/browser" from v4 and at
|
|
13
|
+
// "@vitest/browser/context" before that. Both are tried so a project is not
|
|
14
|
+
// forced onto one Vitest line to be measured.
|
|
15
|
+
const browserContext = await import("vitest/browser").catch(() =>
|
|
16
|
+
import("@vitest/browser/context"));
|
|
17
|
+
const { commands } = browserContext;
|
|
18
|
+
const attempts = new Map();
|
|
19
|
+
const activeScopes = new Map();
|
|
20
|
+
enableRuntimeSnapshotEvidence();
|
|
21
|
+
function attemptStatus(state) {
|
|
22
|
+
if (state === "pass")
|
|
23
|
+
return "passed";
|
|
24
|
+
if (state === "fail")
|
|
25
|
+
return "failed";
|
|
26
|
+
if (state === "skip" || state === "todo")
|
|
27
|
+
return "skipped";
|
|
28
|
+
return "unknown";
|
|
29
|
+
}
|
|
30
|
+
function titlePath(task) {
|
|
31
|
+
const names = [task.name];
|
|
32
|
+
let suite = task.suite;
|
|
33
|
+
while (suite?.name) {
|
|
34
|
+
names.unshift(suite.name);
|
|
35
|
+
suite = suite.suite;
|
|
36
|
+
}
|
|
37
|
+
return names;
|
|
38
|
+
}
|
|
39
|
+
// The node setup keys an attempt by sha256(testId). Browsers have the same
|
|
40
|
+
// digest behind an async API, so the hook awaits it rather than substituting a
|
|
41
|
+
// different hash and giving the same test two identities across environments.
|
|
42
|
+
async function testKeyOf(testId) {
|
|
43
|
+
const bytes = new TextEncoder().encode(testId);
|
|
44
|
+
const digest = await crypto.subtle.digest("SHA-256", bytes);
|
|
45
|
+
return [...new Uint8Array(digest)]
|
|
46
|
+
.map((byte) => byte.toString(16).padStart(2, "0"))
|
|
47
|
+
.join("")
|
|
48
|
+
.slice(0, 24);
|
|
49
|
+
}
|
|
50
|
+
// Evidence leaves the browser over Vitest's own command channel: node runs the
|
|
51
|
+
// handler, the test awaits it, and the failure is reported rather than
|
|
52
|
+
// swallowed -- silently losing a test's evidence is the worst shape a
|
|
53
|
+
// measurement bug takes.
|
|
54
|
+
async function sendEvidence(payload, suffix) {
|
|
55
|
+
try {
|
|
56
|
+
await commands.__supercovEvidence(payload, suffix);
|
|
57
|
+
}
|
|
58
|
+
catch (error) {
|
|
59
|
+
console.error("[supercov] failed to record browser evidence:", error);
|
|
60
|
+
throw error;
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
beforeEach(async (context) => {
|
|
64
|
+
const task = context.task;
|
|
65
|
+
const testId = `vitest:${task.id}`;
|
|
66
|
+
const retry = attempts.get(testId) ?? 0;
|
|
67
|
+
attempts.set(testId, retry + 1);
|
|
68
|
+
const testKey = await testKeyOf(testId);
|
|
69
|
+
const scope = {
|
|
70
|
+
version: 1,
|
|
71
|
+
runId: globalThis.__SUPERCOV_RUN_ID__ ?? "unscoped",
|
|
72
|
+
workerId: "vitest-browser",
|
|
73
|
+
testId,
|
|
74
|
+
testKey,
|
|
75
|
+
retry,
|
|
76
|
+
attemptId: `${testKey}-${retry}`,
|
|
77
|
+
};
|
|
78
|
+
activeScopes.set(task.id, scope);
|
|
79
|
+
activateCoverageScope(scope);
|
|
80
|
+
resetCoverage(testId);
|
|
81
|
+
});
|
|
82
|
+
afterEach(async (context) => {
|
|
83
|
+
const task = context.task;
|
|
84
|
+
const scope = activeScopes.get(task.id);
|
|
85
|
+
const retry = scope?.retry ?? task.result?.retryCount ?? 0;
|
|
86
|
+
// Node relativises the test file Vitest reports and infers provenance, so
|
|
87
|
+
// both environments describe a test the same way and neither trusts the
|
|
88
|
+
// browser realm to say which file it was.
|
|
89
|
+
const payload = {
|
|
90
|
+
testId: scope?.testId ?? `vitest:${task.id}`,
|
|
91
|
+
...(scope ? { scope } : {}),
|
|
92
|
+
test: [...titlePath(task)].join(" > "),
|
|
93
|
+
projectName: task.file?.projectName,
|
|
94
|
+
title: task.name,
|
|
95
|
+
retry,
|
|
96
|
+
status: attemptStatus(task.result?.state),
|
|
97
|
+
...(scope ? { phases: takeNodeAssertionPhases(scope) } : {}),
|
|
98
|
+
// Browser-mode evidence is a runtime snapshot like node's: the code
|
|
99
|
+
// under test runs in the same realm as the probes.
|
|
100
|
+
runtime: [coverageSnapshot()],
|
|
101
|
+
browser: [],
|
|
102
|
+
server: [],
|
|
103
|
+
};
|
|
104
|
+
await sendEvidence(payload, `vitest-${task.id}-${retry}`);
|
|
105
|
+
activeScopes.delete(task.id);
|
|
106
|
+
activateCoverageScope();
|
|
107
|
+
});
|