supercov 0.0.51 → 0.0.53
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +12 -6
- package/docs/assertion-maps.md +20 -7
- package/package.json +9 -9
- package/runtime/javascript/provenance.mjs +71 -12
- package/runtime/javascript/runtime.mjs +32 -6
package/README.md
CHANGED
|
@@ -136,17 +136,23 @@ The denominator comes from source structure before the run, so adding or removin
|
|
|
136
136
|
|
|
137
137
|
## Install for your language
|
|
138
138
|
|
|
139
|
-
|
|
139
|
+
Measuring a project should not start by installing another language. The same binary, at the same version, from one release:
|
|
140
140
|
|
|
141
141
|
```bash
|
|
142
|
-
npx supercov -- npm test
|
|
143
|
-
uvx --from supercov-cli supercov -- pytest
|
|
144
|
-
gem install supercov && supercov -- bundle exec rspec
|
|
145
|
-
cargo binstall supercov && supercov -- cargo test
|
|
142
|
+
npx supercov -- npm test # npm
|
|
143
|
+
uvx --from supercov-cli supercov -- pytest # PyPI
|
|
144
|
+
gem install supercov && supercov -- bundle exec rspec # RubyGems
|
|
145
|
+
cargo binstall supercov && supercov -- cargo test # crates.io
|
|
146
|
+
go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./... # Go
|
|
147
|
+
brew install supercorp-ai/tap/supercov && supercov -- ./gradlew test # Homebrew
|
|
146
148
|
```
|
|
147
149
|
|
|
148
150
|
`pip install supercov-cli` and `gem install supercov` install a wheel or gem that carries the binary for your platform; nothing is compiled. `cargo binstall` downloads that same binary from the GitHub release, while plain `cargo install supercov` builds it from source and needs Rust 1.95.
|
|
149
151
|
|
|
152
|
+
A Go project needs Go and nothing else. Like any `go run` with a version suffix it resolves by module path and ignores the `go.mod` in your current directory, so it neither needs nor touches your module.
|
|
153
|
+
|
|
154
|
+
Java and Kotlin have no registry of their own here, so a JVM project takes the binary directly -- Homebrew above, `npx` if Node is already present, or the platform archive from the [latest release](https://github.com/supercorp-ai/supercov/releases/latest). Maven and Gradle are driven as your test command, not as a plugin. Supercov adds the JUnit Platform launcher its measurement needs to the build file inside its own isolated workspace copy -- your `pom.xml` or `build.gradle` is never edited.
|
|
155
|
+
|
|
150
156
|
## Supported languages
|
|
151
157
|
|
|
152
158
|
| Language | Status | Start with |
|
|
@@ -156,7 +162,7 @@ cargo binstall supercov && supercov -- cargo test # crates.io
|
|
|
156
162
|
| Rust | Available | `npx supercov -- cargo test` |
|
|
157
163
|
| Python | Available | `npx supercov -- pytest` |
|
|
158
164
|
| Ruby | Available | `npx supercov -- rspec` |
|
|
159
|
-
| Go | Available | `
|
|
165
|
+
| Go | Available | `go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./...` |
|
|
160
166
|
| Java | Available | `npx supercov -- mvn test` |
|
|
161
167
|
| Kotlin | Available | `npx supercov -- ./gradlew test` |
|
|
162
168
|
| Zig | Coming soon | — |
|
package/docs/assertion-maps.md
CHANGED
|
@@ -70,7 +70,7 @@ examined and acknowledged.
|
|
|
70
70
|
| `appliesTo` | Select tests by project-relative file and exact displayed test name. |
|
|
71
71
|
| `nodes`, `edges` | Record source locations and relationships ending at `$assertion`. |
|
|
72
72
|
| `countsAsAsserted` | List the node IDs you judge to be checked by the assertion. |
|
|
73
|
-
| `watch` | List additional files the explanation depends on, such as helpers or configuration. Manifests, lockfiles and runner configuration are already tracked for the whole run; naming one
|
|
73
|
+
| `watch` | List additional files the explanation depends on, such as helpers or configuration. A watched file is depended on **as a whole**: any change to it that is not a comment is a review. Use it for a file your claim reasons about but holds no node of yours -- including claims about what a file does *not* contain, which no node can anchor. Naming a file that already holds this flow's nodes widens the flow from those declarations to the whole file, so a neighbouring function's body becomes a review again; that is sometimes what you mean, and the report says when you have done it. Manifests, lockfiles and runner configuration are already tracked for the whole run; naming one catches nothing, and the report says so. A redundant entry is free to take back out: it is not part of what the acknowledgement rests on, so removing one keeps the flow's credit. |
|
|
74
74
|
| `questions` | Record unresolved investigation questions. Questions inside a flow block its credit. |
|
|
75
75
|
|
|
76
76
|
Source anchors use project-relative paths with `/`, one-based lines and one-based
|
|
@@ -279,12 +279,25 @@ response in the map's top-level `changeAssessments` array:
|
|
|
279
279
|
}
|
|
280
280
|
```
|
|
281
281
|
|
|
282
|
-
This is an excerpt to add to your existing map.
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
282
|
+
This is an excerpt to add to your existing map. `affectedFlows` is your
|
|
283
|
+
judgement: the flows this change invalidates, and only those. It is not a
|
|
284
|
+
restatement of `knownFlows` -- Supercov already holds that list and does not
|
|
285
|
+
ask you to retype it. **Every flow you name loses its acknowledgement and has
|
|
286
|
+
to be reread**, so naming a flow is a decision, and naming all of them is the
|
|
287
|
+
expensive one.
|
|
288
|
+
|
|
289
|
+
To make that judgement, read what the change view gives you. `knownFlows`
|
|
290
|
+
counts the flows the change has already made stale and samples them. `exposed`
|
|
291
|
+
counts the flows whose selected tests ran the changed code and names those
|
|
292
|
+
tests -- not stale for it, but the ones most worth thinking about. Neither is
|
|
293
|
+
a verdict: under an integration suite every test runs most of the code, so
|
|
294
|
+
exposure is a place to look, not a list to copy.
|
|
295
|
+
|
|
296
|
+
An empty list is the normal outcome for a change that touches no claim, and it
|
|
297
|
+
is the contract this channel exists to provide: one explanation answers for the
|
|
298
|
+
change and every dependent keeps its credit. It still needs an explanation of
|
|
299
|
+
why existing claims are unaffected, and it does not mean the changed code is
|
|
300
|
+
tested.
|
|
288
301
|
|
|
289
302
|
Save the assessments and graph edits, validate, and copy the examined change
|
|
290
303
|
tokens. Save again, then validate once more before copying final flow tokens.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "supercov",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.53",
|
|
4
4
|
"description": "Coverage for coding agents and software factories \ud83c\udf19",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -97,14 +97,14 @@
|
|
|
97
97
|
"test:launcher": "go vet ./cmd/... && go test ./cmd/..."
|
|
98
98
|
},
|
|
99
99
|
"optionalDependencies": {
|
|
100
|
-
"@supercov/cli-darwin-arm64": "0.0.
|
|
101
|
-
"@supercov/cli-darwin-x64": "0.0.
|
|
102
|
-
"@supercov/cli-linux-arm64-gnu": "0.0.
|
|
103
|
-
"@supercov/cli-linux-arm64-musl": "0.0.
|
|
104
|
-
"@supercov/cli-linux-x64-gnu": "0.0.
|
|
105
|
-
"@supercov/cli-linux-x64-musl": "0.0.
|
|
106
|
-
"@supercov/cli-win32-arm64": "0.0.
|
|
107
|
-
"@supercov/cli-win32-x64": "0.0.
|
|
100
|
+
"@supercov/cli-darwin-arm64": "0.0.53",
|
|
101
|
+
"@supercov/cli-darwin-x64": "0.0.53",
|
|
102
|
+
"@supercov/cli-linux-arm64-gnu": "0.0.53",
|
|
103
|
+
"@supercov/cli-linux-arm64-musl": "0.0.53",
|
|
104
|
+
"@supercov/cli-linux-x64-gnu": "0.0.53",
|
|
105
|
+
"@supercov/cli-linux-x64-musl": "0.0.53",
|
|
106
|
+
"@supercov/cli-win32-arm64": "0.0.53",
|
|
107
|
+
"@supercov/cli-win32-x64": "0.0.53"
|
|
108
108
|
},
|
|
109
109
|
"peerDependencies": {
|
|
110
110
|
"@playwright/test": ">=1.55.0",
|
|
@@ -4,13 +4,50 @@ const NORMALIZED_KINDS = [
|
|
|
4
4
|
["integration", /(^|[/_.-])(integration|int)([/_.-]|$)/i],
|
|
5
5
|
["e2e", /(^|[/_.-])(e2e|end-to-end|offline|online)([/_.-]|$)/i],
|
|
6
6
|
];
|
|
7
|
-
|
|
8
|
-
|
|
7
|
+
// A runner that can only drive the whole system through its external interface
|
|
8
|
+
// is evidence, not a guess. Nothing else here is: a token in a filename is a
|
|
9
|
+
// convention someone may not have followed.
|
|
10
|
+
const STRONG_RUNNER_KINDS = { playwright: "e2e" };
|
|
11
|
+
// The most specific token in a segment wins, not the first one this list
|
|
12
|
+
// happens to hold. `checkout-integration-e2e` is an e2e spec whose name notes
|
|
13
|
+
// what it integrates; ordering by array position called it an integration.
|
|
14
|
+
function segmentKind(segment) {
|
|
15
|
+
if (!segment)
|
|
9
16
|
return undefined;
|
|
10
17
|
// gatewayE2e.test.ts and responseIntegration.test.ts are conventional
|
|
11
18
|
// camel-case paths too. Do not infer kinds from test titles or API usage.
|
|
12
|
-
const words =
|
|
13
|
-
|
|
19
|
+
const words = segment.replace(/([a-z0-9])([A-Z])/g, "$1-$2");
|
|
20
|
+
let best;
|
|
21
|
+
for (const [kind, pattern] of NORMALIZED_KINDS) {
|
|
22
|
+
const found = words.search(pattern);
|
|
23
|
+
if (found >= 0 && (best === undefined || found > best.at))
|
|
24
|
+
best = { kind, at: found };
|
|
25
|
+
}
|
|
26
|
+
return best?.kind;
|
|
27
|
+
}
|
|
28
|
+
function classifiedKind(value) {
|
|
29
|
+
if (!value)
|
|
30
|
+
return undefined;
|
|
31
|
+
return segmentKind(value);
|
|
32
|
+
}
|
|
33
|
+
// A directory is a deliberate choice about where a suite lives; a filename is
|
|
34
|
+
// often just a description of the thing under test. Read directories from the
|
|
35
|
+
// deepest inwards, so the nearest enclosing suite wins.
|
|
36
|
+
function directoryKind(file) {
|
|
37
|
+
if (!file)
|
|
38
|
+
return undefined;
|
|
39
|
+
const segments = file.split(/[/\\]/).slice(0, -1);
|
|
40
|
+
for (let index = segments.length - 1; index >= 0; index -= 1) {
|
|
41
|
+
const kind = segmentKind(segments[index]);
|
|
42
|
+
if (kind)
|
|
43
|
+
return kind;
|
|
44
|
+
}
|
|
45
|
+
return undefined;
|
|
46
|
+
}
|
|
47
|
+
function basenameKind(file) {
|
|
48
|
+
if (!file)
|
|
49
|
+
return undefined;
|
|
50
|
+
return segmentKind(file.split(/[/\\]/).pop());
|
|
14
51
|
}
|
|
15
52
|
export function inferTestProvenance({ runner, file, project, explicitKind, }) {
|
|
16
53
|
if (explicitKind?.trim()) {
|
|
@@ -30,20 +67,42 @@ export function inferTestProvenance({ runner, file, project, explicitKind, }) {
|
|
|
30
67
|
source: "project",
|
|
31
68
|
};
|
|
32
69
|
}
|
|
33
|
-
|
|
34
|
-
|
|
70
|
+
// A directory outranks the runner: putting a Playwright spec under
|
|
71
|
+
// `tests/integration/` is a statement about that suite.
|
|
72
|
+
const directory = directoryKind(file);
|
|
73
|
+
if (directory) {
|
|
74
|
+
return {
|
|
75
|
+
runner,
|
|
76
|
+
kind: directory,
|
|
77
|
+
...(project ? { project } : {}),
|
|
78
|
+
source: "path",
|
|
79
|
+
};
|
|
80
|
+
}
|
|
81
|
+
// A filename token does not. `storefront-empire-integration.spec.ts` drove
|
|
82
|
+
// a real browser; reading `integration` out of its name reported 102 lines
|
|
83
|
+
// as untouched by E2E while a browser had rendered them. The runner knows
|
|
84
|
+
// better than the filename here, so it is asked first.
|
|
85
|
+
const strong = STRONG_RUNNER_KINDS[runner];
|
|
86
|
+
if (strong) {
|
|
87
|
+
return {
|
|
88
|
+
runner,
|
|
89
|
+
kind: strong,
|
|
90
|
+
...(project ? { project } : {}),
|
|
91
|
+
source: "runner-default",
|
|
92
|
+
};
|
|
93
|
+
}
|
|
94
|
+
const basename = basenameKind(file);
|
|
95
|
+
if (basename) {
|
|
35
96
|
return {
|
|
36
97
|
runner,
|
|
37
|
-
kind:
|
|
98
|
+
kind: basename,
|
|
38
99
|
...(project ? { project } : {}),
|
|
39
100
|
source: "path",
|
|
40
101
|
};
|
|
41
102
|
}
|
|
42
|
-
const defaultKind = runner === "
|
|
43
|
-
? "
|
|
44
|
-
:
|
|
45
|
-
? "unit"
|
|
46
|
-
: "unknown";
|
|
103
|
+
const defaultKind = runner === "vitest" || runner === "jest" || runner === "node:test"
|
|
104
|
+
? "unit"
|
|
105
|
+
: "unknown";
|
|
47
106
|
return {
|
|
48
107
|
runner,
|
|
49
108
|
kind: defaultKind,
|
|
@@ -751,8 +751,19 @@ function withNodeAssertionPhase(operation, source, callback) {
|
|
|
751
751
|
const scope = context.scope;
|
|
752
752
|
if (!scope)
|
|
753
753
|
return callback();
|
|
754
|
-
|
|
755
|
-
|
|
754
|
+
// A lexically instrumented occurrence is an authored assertion site in its
|
|
755
|
+
// own right, even when it runs inside another assertion's callback: the
|
|
756
|
+
// validator passed to assert.throws is the ordinary case, and its inner
|
|
757
|
+
// assertions are what claim *what* the error says. Skipping every nested
|
|
758
|
+
// phase left those with no passing occurrence, so they could never earn
|
|
759
|
+
// credit however carefully they were mapped.
|
|
760
|
+
//
|
|
761
|
+
// What must still be skipped is the same call seen twice -- the module proxy
|
|
762
|
+
// firing inside the lexical wrapper. That path has no lexical source and
|
|
763
|
+
// arrives as a lazy stack fallback, so the shape of `source` tells them
|
|
764
|
+
// apart without comparing coordinates the transform has moved.
|
|
765
|
+
const nested = context.phaseId && assertionPhaseState(scope).phaseIds.has(context.phaseId);
|
|
766
|
+
if (nested && typeof source !== "string")
|
|
756
767
|
return callback();
|
|
757
768
|
// A lexical occurrence already identifies its source. Stack fallback is lazy
|
|
758
769
|
// and qualified so transformed coordinates cannot impersonate original ones.
|
|
@@ -1244,20 +1255,35 @@ function tryBegin(successId, catchId) {
|
|
|
1244
1255
|
return { successId, catchId, caught: false };
|
|
1245
1256
|
}
|
|
1246
1257
|
function tryCatch(frame, value) {
|
|
1247
|
-
|
|
1258
|
+
// Record the outcome the moment it is known, not when the construct is left.
|
|
1259
|
+
// The commit used to sit in the generated `finally`, and `process.exit()` in
|
|
1260
|
+
// a catch body skips `finally` -- so a catch that ran and whose statements
|
|
1261
|
+
// were credited was still reported as never entered. Statement probes fire
|
|
1262
|
+
// in place and survived, which is what made the report contradict itself.
|
|
1263
|
+
if (!frame.caught) {
|
|
1264
|
+
frame.caught = true;
|
|
1265
|
+
coverageHit(frame.catchId);
|
|
1266
|
+
}
|
|
1248
1267
|
return value;
|
|
1249
1268
|
}
|
|
1250
1269
|
function tryEnd(frame) {
|
|
1251
|
-
|
|
1270
|
+
// Completing without catching is only knowable here.
|
|
1271
|
+
if (!frame.caught) coverageHit(frame.successId);
|
|
1252
1272
|
}
|
|
1253
1273
|
function loopBegin(zeroId, enteredId) {
|
|
1254
1274
|
return { zeroId, enteredId, entered: false };
|
|
1255
1275
|
}
|
|
1256
1276
|
function loopEntered(frame) {
|
|
1257
|
-
|
|
1277
|
+
// Same reason as tryCatch: a loop body that exits the process would
|
|
1278
|
+
// otherwise report zero iterations. Guarded so a loop still counts once,
|
|
1279
|
+
// however many times it iterates.
|
|
1280
|
+
if (!frame.entered) {
|
|
1281
|
+
frame.entered = true;
|
|
1282
|
+
coverageHit(frame.enteredId);
|
|
1283
|
+
}
|
|
1258
1284
|
}
|
|
1259
1285
|
function loopEnd(frame) {
|
|
1260
|
-
|
|
1286
|
+
if (!frame.entered) coverageHit(frame.zeroId);
|
|
1261
1287
|
}
|
|
1262
1288
|
function mcdcBegin(id, meta) {
|
|
1263
1289
|
if (!state.decisions.has(id)) {
|