supercov 0.0.43 → 0.0.44
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/analyzers/typescript/README.md +4 -0
- package/analyzers/typescript/bin/identity.mjs +4 -0
- package/analyzers/typescript/dist/analyze.js +1352 -51
- package/analyzers/typescript/dist/archive.js +31 -3
- package/analyzers/typescript/dist/awaited-observations.js +376 -0
- package/analyzers/typescript/dist/build-identity.json +1 -1
- package/analyzers/typescript/dist/mock-counts.js +2517 -0
- package/analyzers/typescript/dist/pragmas.js +59 -16
- package/analyzers/typescript/src/analyze.ts +1738 -96
- package/analyzers/typescript/src/archive.ts +36 -3
- package/analyzers/typescript/src/awaited-observations.ts +561 -0
- package/analyzers/typescript/src/mock-counts.ts +3219 -0
- package/analyzers/typescript/src/pragmas.ts +90 -24
- package/docs/agent-loop.md +116 -31
- package/docs/assertion-evidence.md +560 -1
- package/docs/cli.md +15 -15
- package/docs/code-verification.md +3 -181
- package/docs/coverage-model.md +6 -6
- package/docs/evidence.md +6 -6
- package/docs/getting-started.md +60 -72
- package/docs/performance.md +4 -4
- package/docs/troubleshooting.md +8 -8
- package/docs/verification.md +2 -2
- package/docs/workspace-isolation.md +1 -1
- package/package.json +11 -9
- package/runtime/javascript/nodeAssertAdapter.mjs +32 -8
- package/runtime/javascript/nodeTest.mjs +13 -5
- package/runtime/javascript/runnerEvidence.mjs +33 -11
- package/runtime/javascript/runtime.mjs +22 -3
package/docs/cli.md
CHANGED
|
@@ -4,7 +4,7 @@ Supercov has one command for measuring a suite and a small set of commands for
|
|
|
4
4
|
reading the result. Text output is designed for people and coding agents. Add
|
|
5
5
|
`--json` only when an integration needs a stable machine-readable response.
|
|
6
6
|
|
|
7
|
-
```sh
|
|
7
|
+
```sh supercov
|
|
8
8
|
npx supercov --help
|
|
9
9
|
```
|
|
10
10
|
|
|
@@ -25,13 +25,13 @@ npx supercov --help
|
|
|
25
25
|
|
|
26
26
|
## Measure a test command
|
|
27
27
|
|
|
28
|
-
```sh
|
|
28
|
+
```sh supercov
|
|
29
29
|
npx supercov -- <test command>
|
|
30
30
|
```
|
|
31
31
|
|
|
32
32
|
Everything after `--` is passed to the test command:
|
|
33
33
|
|
|
34
|
-
```sh
|
|
34
|
+
```sh supercov
|
|
35
35
|
npx supercov -- npm test
|
|
36
36
|
npx supercov -- npx playwright test --project=chromium
|
|
37
37
|
npx supercov -- cargo test
|
|
@@ -43,7 +43,7 @@ preserves the wrapped command's exit status, so it can remain a CI gate.
|
|
|
43
43
|
|
|
44
44
|
## List and select runs
|
|
45
45
|
|
|
46
|
-
```sh
|
|
46
|
+
```sh supercov
|
|
47
47
|
npx supercov runs
|
|
48
48
|
npx supercov runs --limit 5
|
|
49
49
|
npx supercov runs latest
|
|
@@ -56,7 +56,7 @@ sessions.
|
|
|
56
56
|
|
|
57
57
|
## Query a run
|
|
58
58
|
|
|
59
|
-
```sh
|
|
59
|
+
```sh supercov
|
|
60
60
|
npx supercov runs <run-id> [query] [options]
|
|
61
61
|
```
|
|
62
62
|
|
|
@@ -77,7 +77,7 @@ npx supercov runs <run-id> [query] [options]
|
|
|
77
77
|
|
|
78
78
|
Common examples:
|
|
79
79
|
|
|
80
|
-
```sh
|
|
80
|
+
```sh supercov
|
|
81
81
|
npx supercov runs latest gaps --limit 10
|
|
82
82
|
npx supercov runs latest file app/routes/checkout.ts
|
|
83
83
|
npx supercov runs latest decision app/routes/checkout.ts:42
|
|
@@ -87,7 +87,7 @@ npx supercov runs latest test "checkout retry"
|
|
|
87
87
|
|
|
88
88
|
Run any query with `--help` to see only the options valid for that query:
|
|
89
89
|
|
|
90
|
-
```sh
|
|
90
|
+
```sh supercov
|
|
91
91
|
npx supercov runs latest --help
|
|
92
92
|
npx supercov runs latest file --help
|
|
93
93
|
npx supercov runs latest assertions --help
|
|
@@ -115,14 +115,14 @@ Collection output includes a copyable command for the next page.
|
|
|
115
115
|
|
|
116
116
|
For a large file, group and rank its decisions:
|
|
117
117
|
|
|
118
|
-
```sh
|
|
118
|
+
```sh supercov
|
|
119
119
|
npx supercov runs latest file app/routes/checkout.ts \
|
|
120
120
|
--group decision --sort missing
|
|
121
121
|
```
|
|
122
122
|
|
|
123
123
|
## Compare runs
|
|
124
124
|
|
|
125
|
-
```sh
|
|
125
|
+
```sh supercov
|
|
126
126
|
npx supercov diff <older-run> <newer-run>
|
|
127
127
|
```
|
|
128
128
|
|
|
@@ -132,13 +132,13 @@ Neither input run is changed.
|
|
|
132
132
|
|
|
133
133
|
The same filters can focus a comparison:
|
|
134
134
|
|
|
135
|
-
```sh
|
|
135
|
+
```sh supercov
|
|
136
136
|
npx supercov diff <older-run> <newer-run> --kind e2e
|
|
137
137
|
```
|
|
138
138
|
|
|
139
139
|
## Find a smaller test set
|
|
140
140
|
|
|
141
|
-
```sh
|
|
141
|
+
```sh supercov
|
|
142
142
|
npx supercov runs latest minimize
|
|
143
143
|
npx supercov runs latest minimize --metric branches --target 90
|
|
144
144
|
```
|
|
@@ -150,7 +150,7 @@ selected metric.
|
|
|
150
150
|
|
|
151
151
|
## Combine shards
|
|
152
152
|
|
|
153
|
-
```sh
|
|
153
|
+
```sh supercov
|
|
154
154
|
npx supercov merge <shard-a> <shard-b> <shard-c>
|
|
155
155
|
```
|
|
156
156
|
|
|
@@ -160,7 +160,7 @@ incompatible merge rather than publishing a misleading aggregate.
|
|
|
160
160
|
|
|
161
161
|
## Clean local data
|
|
162
162
|
|
|
163
|
-
```sh
|
|
163
|
+
```sh supercov
|
|
164
164
|
npx supercov clean --dry-run
|
|
165
165
|
npx supercov clean --keep 20
|
|
166
166
|
npx supercov clean
|
|
@@ -172,7 +172,7 @@ storage.
|
|
|
172
172
|
|
|
173
173
|
## Read bundled documentation
|
|
174
174
|
|
|
175
|
-
```sh
|
|
175
|
+
```sh supercov
|
|
176
176
|
npx supercov docs
|
|
177
177
|
npx supercov docs getting-started
|
|
178
178
|
npx supercov docs troubleshooting
|
|
@@ -190,7 +190,7 @@ terminal or offline environment after the package has been downloaded.
|
|
|
190
190
|
|
|
191
191
|
Examples:
|
|
192
192
|
|
|
193
|
-
```sh
|
|
193
|
+
```sh supercov
|
|
194
194
|
SUPERCOV_SOURCE_ROOTS=src,app npx supercov -- npm test
|
|
195
195
|
SUPERCOV_TEST_KIND=e2e npx supercov -- npx playwright test
|
|
196
196
|
```
|
|
@@ -1,182 +1,4 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Agent workflow
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
comparing the results.
|
|
6
|
-
|
|
7
|
-
## Before you start
|
|
8
|
-
|
|
9
|
-
You need Node.js 22 or newer and npm. Clone the repository and install the
|
|
10
|
-
example's dependencies:
|
|
11
|
-
|
|
12
|
-
```sh
|
|
13
|
-
git clone --depth 1 https://github.com/supercorp-ai/supercov.git
|
|
14
|
-
cd supercov/examples/checkout-verification
|
|
15
|
-
npm ci
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
Run the commands below from this directory. The example uses Supercov 0.0.42
|
|
19
|
-
and Node's built-in test runner. Both the original tests and the additional
|
|
20
|
-
test are included. The first three steps do not require any file edits.
|
|
21
|
-
|
|
22
|
-
## 1. Run the original tests
|
|
23
|
-
|
|
24
|
-
In `src/session.js`, checkout is allowed only if the customer is signed in and
|
|
25
|
-
their session has not expired:
|
|
26
|
-
|
|
27
|
-
```js
|
|
28
|
-
export function canCheckout(signedIn, expired) {
|
|
29
|
-
if (signedIn && !expired) return true;
|
|
30
|
-
return false;
|
|
31
|
-
}
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
The two tests in `tests/session.test.js` check a valid session and a signed-out
|
|
35
|
-
visitor:
|
|
36
|
-
|
|
37
|
-
```js
|
|
38
|
-
assert.equal(canCheckout(true, false), true);
|
|
39
|
-
assert.equal(canCheckout(false, false), false);
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
Run those tests through Supercov, then open the summary:
|
|
43
|
-
|
|
44
|
-
```sh
|
|
45
|
-
npx supercov -- node --test tests/session.test.js
|
|
46
|
-
npx supercov runs latest
|
|
47
|
-
```
|
|
48
|
-
|
|
49
|
-
Everything after `--` is the test command Supercov runs. In your own project,
|
|
50
|
-
use your existing test command there.
|
|
51
|
-
|
|
52
|
-
Both tests pass. The coverage section shows:
|
|
53
|
-
|
|
54
|
-
```text
|
|
55
|
-
Coverage
|
|
56
|
-
Lines 100.00% (3/3)
|
|
57
|
-
Branches 100.00% (2/2)
|
|
58
|
-
MC/DC 50.00% (1/2)
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
Line and branch coverage are 100% because the tests reach both `return true`
|
|
62
|
-
and `return false`. The MC/DC result shows there is still a condition to test.
|
|
63
|
-
|
|
64
|
-
Keep the run ID printed at the top of the summary. You'll use it to compare
|
|
65
|
-
this run with the next one.
|
|
66
|
-
|
|
67
|
-
## 2. Inspect the missing condition
|
|
68
|
-
|
|
69
|
-
Ask about the decision on line 2:
|
|
70
|
-
|
|
71
|
-
```sh
|
|
72
|
-
npx supercov runs latest decision src/session.js:2
|
|
73
|
-
```
|
|
74
|
-
|
|
75
|
-
```text
|
|
76
|
-
signedIn && !expired
|
|
77
|
-
C1 covered + asserted: signedIn
|
|
78
|
-
C2 MISSING: !expired
|
|
79
|
-
confidence asserted; asserted MC/DC 1/2
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
MC/DC stands for Modified Condition/Decision Coverage. It checks whether each
|
|
83
|
-
condition has independently affected the decision. The original tests show
|
|
84
|
-
that changing `signedIn` changes the result, but neither test changes `expired`.
|
|
85
|
-
That leaves one of two conditions covered: 50%.
|
|
86
|
-
|
|
87
|
-
`C2 MISSING: !expired` points to the case to test: a customer who is still
|
|
88
|
-
signed in, but whose session has expired. Checkout should be denied.
|
|
89
|
-
|
|
90
|
-
## 3. Include the expired-session test
|
|
91
|
-
|
|
92
|
-
`tests/expired-session.test.js` contains that test:
|
|
93
|
-
|
|
94
|
-
```js
|
|
95
|
-
test('an expired session cannot check out', () => {
|
|
96
|
-
assert.equal(canCheckout(true, true), false);
|
|
97
|
-
});
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
Run both test files and open the new summary:
|
|
101
|
-
|
|
102
|
-
```sh
|
|
103
|
-
npx supercov -- node --test tests/session.test.js tests/expired-session.test.js
|
|
104
|
-
npx supercov runs latest
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
All three tests pass:
|
|
108
|
-
|
|
109
|
-
```text
|
|
110
|
-
Coverage
|
|
111
|
-
Lines 100.00% (3/3)
|
|
112
|
-
Branches 100.00% (2/2)
|
|
113
|
-
MC/DC 100.00% (2/2)
|
|
114
|
-
```
|
|
115
|
-
|
|
116
|
-
Query the same decision again:
|
|
117
|
-
|
|
118
|
-
```sh
|
|
119
|
-
npx supercov runs latest decision src/session.js:2
|
|
120
|
-
```
|
|
121
|
-
|
|
122
|
-
The expiry condition is now covered. `asserted MC/DC 2/2` means both conditions
|
|
123
|
-
have coverage evidence linked to passing assertions:
|
|
124
|
-
|
|
125
|
-
```text
|
|
126
|
-
C2 covered + asserted: !expired
|
|
127
|
-
confidence asserted; asserted MC/DC 2/2
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
Compare the runs, replacing `<before-run-id>` with the ID you saved in step 1:
|
|
131
|
-
|
|
132
|
-
```sh
|
|
133
|
-
npx supercov diff <before-run-id> latest
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
```text
|
|
137
|
-
lines +0pp, branches +0pp, MC/DC +50pp
|
|
138
|
-
gained: 0 lines, 0 branches, 1 MC/DC conditions
|
|
139
|
-
lost: 0 lines, 0 branches, 0 MC/DC conditions
|
|
140
|
-
+ MC/DC src/session.js:2 C2 !expired
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
Line and branch coverage have not changed. The new test covers the missing
|
|
144
|
-
expiry condition without changing application code.
|
|
145
|
-
|
|
146
|
-
## 4. Check that the test catches a regression
|
|
147
|
-
|
|
148
|
-
In this example only, temporarily remove the expiry check from `src/session.js`:
|
|
149
|
-
|
|
150
|
-
```diff
|
|
151
|
-
- if (signedIn && !expired) return true;
|
|
152
|
-
+ if (signedIn) return true;
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
Run the original two tests against the changed function:
|
|
156
|
-
|
|
157
|
-
```sh
|
|
158
|
-
npx supercov -- node --test tests/session.test.js
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
Then include the expired-session test:
|
|
162
|
-
|
|
163
|
-
```sh
|
|
164
|
-
npx supercov -- node --test tests/session.test.js tests/expired-session.test.js
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
| Tests run against the changed function | Result |
|
|
168
|
-
| --- | --- |
|
|
169
|
-
| Original two tests | Both pass. |
|
|
170
|
-
| All three tests | The expired-session test fails; the other two pass. |
|
|
171
|
-
|
|
172
|
-
The new test expects `false`, but the changed function returns `true`. The
|
|
173
|
-
second command should fail: that is the test catching the removed expiry check.
|
|
174
|
-
|
|
175
|
-
Restore `&& !expired` in `src/session.js` when you finish, then rerun all three
|
|
176
|
-
tests with the same command. They should pass again.
|
|
177
|
-
|
|
178
|
-
## Next
|
|
179
|
-
|
|
180
|
-
- [Full example and recorded output](https://github.com/supercorp-ai/supercov/tree/main/examples/checkout-verification) — source, tests, and the complete output excerpted above.
|
|
181
|
-
- [Understanding coverage](coverage-model.md) — what each metric measures and what 100% means.
|
|
182
|
-
- [Agent workflow](agent-loop.md) — use the same run, inspect, test, and compare steps with a coding agent.
|
|
3
|
+
The tutorial is now part of [Agent workflow](agent-loop.md), including the
|
|
4
|
+
prompt, recorded checkout example, and guidance for longer runs.
|
package/docs/coverage-model.md
CHANGED
|
@@ -17,7 +17,7 @@ runner boundary.
|
|
|
17
17
|
Supercov keeps those states separate. It does not turn “unknown” into
|
|
18
18
|
“uncovered,” and it does not round either one away to produce a reassuring 100%.
|
|
19
19
|
|
|
20
|
-
```sh
|
|
20
|
+
```sh supercov
|
|
21
21
|
npx supercov runs latest
|
|
22
22
|
npx supercov runs latest gaps
|
|
23
23
|
npx supercov runs latest scope
|
|
@@ -39,7 +39,7 @@ The exact obligations depend on the language and source construct. You do not
|
|
|
39
39
|
need to reason about all of them at once. Start with a file, then open a decision
|
|
40
40
|
or line only when the missing behavior needs explanation:
|
|
41
41
|
|
|
42
|
-
```sh
|
|
42
|
+
```sh supercov
|
|
43
43
|
npx supercov runs latest file app/checkout/session.ts
|
|
44
44
|
npx supercov runs latest decision app/checkout/session.ts:64
|
|
45
45
|
npx supercov runs latest line app/checkout/session.ts:64
|
|
@@ -92,7 +92,7 @@ each runner.
|
|
|
92
92
|
|
|
93
93
|
The same stored run can answer different questions:
|
|
94
94
|
|
|
95
|
-
```sh
|
|
95
|
+
```sh supercov
|
|
96
96
|
npx supercov runs latest --filter all
|
|
97
97
|
npx supercov runs latest --filter passed
|
|
98
98
|
npx supercov runs latest --filter failed
|
|
@@ -103,7 +103,7 @@ attempts. `failed` isolates failed attempts, including failed retries.
|
|
|
103
103
|
|
|
104
104
|
You can also focus on a test level or runner:
|
|
105
105
|
|
|
106
|
-
```sh
|
|
106
|
+
```sh supercov
|
|
107
107
|
npx supercov runs latest gaps --kind e2e
|
|
108
108
|
npx supercov runs latest gaps --runner playwright
|
|
109
109
|
```
|
|
@@ -115,13 +115,13 @@ percentage that was computed from a different set of tests.
|
|
|
115
115
|
|
|
116
116
|
If the summary reports ambiguous source scope, inspect it:
|
|
117
117
|
|
|
118
|
-
```sh
|
|
118
|
+
```sh supercov
|
|
119
119
|
npx supercov runs latest scope
|
|
120
120
|
```
|
|
121
121
|
|
|
122
122
|
When first-party source lives in unusual directories, declare it explicitly:
|
|
123
123
|
|
|
124
|
-
```sh
|
|
124
|
+
```sh supercov
|
|
125
125
|
SUPERCOV_SOURCE_ROOTS=src,app npx supercov -- npm test
|
|
126
126
|
```
|
|
127
127
|
|
package/docs/evidence.md
CHANGED
|
@@ -6,7 +6,7 @@ without rerunning the tests.
|
|
|
6
6
|
|
|
7
7
|
## Find the run you want
|
|
8
8
|
|
|
9
|
-
```sh
|
|
9
|
+
```sh supercov
|
|
10
10
|
npx supercov runs
|
|
11
11
|
npx supercov runs --limit 10
|
|
12
12
|
npx supercov runs latest
|
|
@@ -23,7 +23,7 @@ A run id is immutable. `latest` is only a convenient selector.
|
|
|
23
23
|
|
|
24
24
|
## Ask the same run different questions
|
|
25
25
|
|
|
26
|
-
```sh
|
|
26
|
+
```sh supercov
|
|
27
27
|
npx supercov runs latest gaps --limit 10
|
|
28
28
|
npx supercov runs latest file app/checkout/session.ts
|
|
29
29
|
npx supercov runs latest line app/checkout/session.ts:64
|
|
@@ -45,7 +45,7 @@ new work from it.
|
|
|
45
45
|
|
|
46
46
|
## Focus on passed or failed attempts
|
|
47
47
|
|
|
48
|
-
```sh
|
|
48
|
+
```sh supercov
|
|
49
49
|
npx supercov runs latest --filter all
|
|
50
50
|
npx supercov runs latest --filter passed
|
|
51
51
|
npx supercov runs latest --filter failed
|
|
@@ -60,7 +60,7 @@ reports that can drift apart.
|
|
|
60
60
|
|
|
61
61
|
## Compare before and after
|
|
62
62
|
|
|
63
|
-
```sh
|
|
63
|
+
```sh supercov
|
|
64
64
|
npx supercov diff <older-run> <newer-run>
|
|
65
65
|
```
|
|
66
66
|
|
|
@@ -76,7 +76,7 @@ need to reproduce the result.
|
|
|
76
76
|
|
|
77
77
|
## Combine distributed shards
|
|
78
78
|
|
|
79
|
-
```sh
|
|
79
|
+
```sh supercov
|
|
80
80
|
npx supercov merge <shard-a> <shard-b> <shard-c>
|
|
81
81
|
```
|
|
82
82
|
|
|
@@ -100,7 +100,7 @@ Completed runs live under `.supercov/runs/<run-id>/`. The isolated workspace and
|
|
|
100
100
|
instrumented build cache may use more space than the compressed run itself.
|
|
101
101
|
Nothing is pruned in the background.
|
|
102
102
|
|
|
103
|
-
```sh
|
|
103
|
+
```sh supercov
|
|
104
104
|
npx supercov clean --dry-run
|
|
105
105
|
npx supercov clean --keep 20
|
|
106
106
|
npx supercov clean
|
package/docs/getting-started.md
CHANGED
|
@@ -1,68 +1,50 @@
|
|
|
1
1
|
# Getting started
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
compare the result.
|
|
3
|
+
Open your project in a coding agent that can run terminal commands, then paste
|
|
4
|
+
this prompt:
|
|
6
5
|
|
|
7
|
-
```
|
|
8
|
-
npx supercov
|
|
6
|
+
```text supercov-prompt
|
|
7
|
+
Measure code coverage with npx supercov and write one missing test.
|
|
8
|
+
Only change tests. Rerun the full test suite and show me the test you
|
|
9
|
+
added and the before-and-after coverage.
|
|
9
10
|
```
|
|
10
11
|
|
|
11
|
-
|
|
12
|
-
|
|
12
|
+
Your agent can install Supercov if needed. It runs the commands and edits the
|
|
13
|
+
tests; you don't need to do those steps yourself.
|
|
13
14
|
|
|
14
|
-
|
|
15
|
+
When it finishes, review the test change and coverage comparison in your
|
|
16
|
+
conversation. Ask separately if you want a commit or pull request.
|
|
15
17
|
|
|
16
|
-
|
|
18
|
+
No account, config file, import, custom reporter, or hosted service is required.
|
|
19
|
+
Supercov supports JavaScript, TypeScript, Rust, Python, and Ruby today.
|
|
17
20
|
|
|
18
|
-
|
|
19
|
-
Windows (arm64 or x64);
|
|
20
|
-
- Node.js 22 or newer;
|
|
21
|
-
- a test command that already works in the repository; and
|
|
22
|
-
- for Rust, the Rust 1.95 toolchain;
|
|
23
|
-
- for Python, CPython 3.12 or newer with pytest or unittest;
|
|
24
|
-
- for Ruby, Ruby 3.4 or newer with RSpec, Minitest, test-unit or Cucumber (3.3 measures lines, methods and simple branches only).
|
|
21
|
+
## What the agent does
|
|
25
22
|
|
|
26
|
-
The
|
|
27
|
-
|
|
28
|
-
on PyPI as `supercov-cli` (`uvx --from supercov-cli supercov`) and on RubyGems
|
|
29
|
-
as `supercov` (`gem install supercov`), at the same version, and the source is
|
|
30
|
-
on crates.io (`cargo install supercov`). The first invocation may download
|
|
31
|
-
Supercov from the registry. Supercov itself does not upload your source or
|
|
32
|
-
coverage evidence to a Supercov service.
|
|
23
|
+
The commands below show how the agent measures coverage and checks its work.
|
|
24
|
+
You can also run them yourself if you prefer using the terminal.
|
|
33
25
|
|
|
34
|
-
|
|
26
|
+
### 1. Run your real test command
|
|
35
27
|
|
|
36
|
-
|
|
37
|
-
complete command you trust before merging or deploying:
|
|
28
|
+
The agent runs your repository's complete test suite through Supercov:
|
|
38
29
|
|
|
39
|
-
```sh
|
|
40
|
-
# JavaScript or TypeScript
|
|
30
|
+
```sh supercov-example
|
|
41
31
|
npx supercov -- npm test
|
|
42
|
-
npx supercov -- npx playwright test
|
|
43
|
-
npx supercov -- pnpm test:e2e
|
|
44
|
-
|
|
45
|
-
# Rust
|
|
46
|
-
npx supercov -- cargo test
|
|
47
|
-
npx supercov -- cargo nextest run
|
|
48
|
-
|
|
49
|
-
# Python
|
|
50
|
-
npx supercov -- pytest
|
|
51
|
-
|
|
52
|
-
# Ruby
|
|
53
|
-
npx supercov -- rspec
|
|
54
32
|
```
|
|
55
33
|
|
|
34
|
+
Everything after `--` is the command Supercov measures. The agent should use
|
|
35
|
+
the same complete command your project uses before merging or deploying.
|
|
36
|
+
If the project has several suites, tell it which one to use.
|
|
37
|
+
|
|
56
38
|
Supercov runs that command in an isolated, instrumented copy of the project.
|
|
57
39
|
The command keeps its normal arguments, environment, output, and exit status.
|
|
58
40
|
If one command launches several supported runners, their evidence lands in one
|
|
59
41
|
run.
|
|
60
42
|
|
|
61
|
-
|
|
43
|
+
### 2. Read the first result
|
|
62
44
|
|
|
63
|
-
|
|
45
|
+
The agent reads the newest run:
|
|
64
46
|
|
|
65
|
-
```sh
|
|
47
|
+
```sh supercov-example
|
|
66
48
|
npx supercov runs latest
|
|
67
49
|
```
|
|
68
50
|
|
|
@@ -76,18 +58,19 @@ An uncovered gap is a candidate for a test. A measurement limit is different:
|
|
|
76
58
|
it means Supercov cannot honestly account for that code yet. Do not try to test
|
|
77
59
|
away a measurement limit.
|
|
78
60
|
|
|
79
|
-
|
|
61
|
+
### 3. Choose one useful gap
|
|
80
62
|
|
|
81
|
-
|
|
63
|
+
The agent asks for a short list of gaps, then inspects one file. It uses your
|
|
64
|
+
project's file paths in place of the examples:
|
|
82
65
|
|
|
83
|
-
```sh
|
|
66
|
+
```sh supercov-example
|
|
84
67
|
npx supercov runs latest gaps --limit 10
|
|
85
68
|
npx supercov runs latest file app/checkout/session.ts
|
|
86
69
|
```
|
|
87
70
|
|
|
88
|
-
|
|
71
|
+
It can use more specific queries to understand the gap:
|
|
89
72
|
|
|
90
|
-
```sh
|
|
73
|
+
```sh supercov-example
|
|
91
74
|
npx supercov runs latest decision app/checkout/session.ts:64
|
|
92
75
|
npx supercov runs latest line app/checkout/session.ts:64
|
|
93
76
|
```
|
|
@@ -96,40 +79,45 @@ npx supercov runs latest line app/checkout/session.ts:64
|
|
|
96
79
|
outcomes and MC/DC witnesses. `line` shows the obligations and tests associated
|
|
97
80
|
with one source line.
|
|
98
81
|
|
|
99
|
-
|
|
82
|
+
### 4. Add a test and prove the gain
|
|
100
83
|
|
|
101
|
-
|
|
102
|
-
command and
|
|
84
|
+
The agent writes one focused test with a meaningful assertion. It then reruns
|
|
85
|
+
the same complete command and compares the two runs:
|
|
103
86
|
|
|
104
|
-
```sh
|
|
87
|
+
```sh supercov-example
|
|
105
88
|
npx supercov -- npm test
|
|
106
89
|
npx supercov diff <previous-run-id> latest
|
|
107
90
|
```
|
|
108
91
|
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
92
|
+
A useful change leaves the suite passing and shows the expected gain without
|
|
93
|
+
an unexplained loss elsewhere. Review what the new test actually checks, not
|
|
94
|
+
just the percentage.
|
|
112
95
|
|
|
113
|
-
|
|
96
|
+
For a recorded example and prompts for longer runs, see
|
|
97
|
+
[Agent workflow](agent-loop.md).
|
|
114
98
|
|
|
115
|
-
|
|
99
|
+
## Environment requirements
|
|
116
100
|
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
101
|
+
These requirements apply wherever the agent runs commands: your machine,
|
|
102
|
+
a container, or a remote workspace. The agent can check them and tell you
|
|
103
|
+
if anything is missing.
|
|
120
104
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
of an ordinary gap.
|
|
130
|
-
```
|
|
105
|
+
- macOS (arm64 or x64), Linux (arm64 or x64, glibc 2.28 or newer or musl), or
|
|
106
|
+
Windows (arm64 or x64);
|
|
107
|
+
- Node.js 22 or newer when using the npm package;
|
|
108
|
+
- a working test command, including its dependencies, environment variables,
|
|
109
|
+
and any local services;
|
|
110
|
+
- for Rust, the Rust 1.95 toolchain;
|
|
111
|
+
- for Python, CPython 3.12 or newer with pytest or unittest;
|
|
112
|
+
- for Ruby, Ruby 3.4 or newer with RSpec, Minitest, test-unit or Cucumber (3.3 measures lines, methods and simple branches only).
|
|
131
113
|
|
|
132
|
-
|
|
114
|
+
The CLI is a native binary. `npx supercov` picks the build for your operating
|
|
115
|
+
system and architecture, and nothing is compiled on install; the same binary is
|
|
116
|
+
on PyPI as `supercov-cli` (`uvx --from supercov-cli supercov`) and on RubyGems
|
|
117
|
+
as `supercov` (`gem install supercov`), at the same version, and the source is
|
|
118
|
+
on crates.io (`cargo install supercov`). The first invocation may download
|
|
119
|
+
Supercov from the registry. Supercov itself does not upload your source or
|
|
120
|
+
coverage evidence to a Supercov service.
|
|
133
121
|
|
|
134
122
|
## Files and cleanup
|
|
135
123
|
|
|
@@ -138,7 +126,7 @@ workspace for instrumented builds. These files are local and ignored by Git.
|
|
|
138
126
|
Supercov does not rewrite your source, tests, imports, runner configuration,
|
|
139
127
|
dependencies, or ordinary build output.
|
|
140
128
|
|
|
141
|
-
```sh
|
|
129
|
+
```sh supercov
|
|
142
130
|
npx supercov clean --dry-run # preview what would be removed
|
|
143
131
|
npx supercov clean --keep 20 # keep the 20 newest runs
|
|
144
132
|
npx supercov clean # remove all runs and the build cache
|
package/docs/performance.md
CHANGED
|
@@ -6,7 +6,7 @@ the isolated build when the relevant inputs have not changed.
|
|
|
6
6
|
|
|
7
7
|
## See where the time went
|
|
8
8
|
|
|
9
|
-
```sh
|
|
9
|
+
```sh supercov
|
|
10
10
|
npx supercov runs latest
|
|
11
11
|
```
|
|
12
12
|
|
|
@@ -42,7 +42,7 @@ possible mismatch triggers a fresh build rather than risking stale coverage.
|
|
|
42
42
|
|
|
43
43
|
A narrow test command can shorten the inner loop:
|
|
44
44
|
|
|
45
|
-
```sh
|
|
45
|
+
```sh supercov
|
|
46
46
|
npx supercov -- npx vitest run app/checkout/session.test.ts
|
|
47
47
|
```
|
|
48
48
|
|
|
@@ -54,7 +54,7 @@ baseline.
|
|
|
54
54
|
|
|
55
55
|
Compare the original and wrapped command under similar cache conditions:
|
|
56
56
|
|
|
57
|
-
```sh
|
|
57
|
+
```sh supercov
|
|
58
58
|
/usr/bin/time -p npm test
|
|
59
59
|
/usr/bin/time -p npx supercov -- npm test
|
|
60
60
|
```
|
|
@@ -73,7 +73,7 @@ than stored as a full report for every filter.
|
|
|
73
73
|
The isolated workspace may be larger because it can contain an instrumented
|
|
74
74
|
build cache. Supercov does not delete history in the background.
|
|
75
75
|
|
|
76
|
-
```sh
|
|
76
|
+
```sh supercov
|
|
77
77
|
npx supercov clean --dry-run
|
|
78
78
|
npx supercov clean --keep 20
|
|
79
79
|
npx supercov clean
|