supercov 0.0.26 → 0.0.27

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/cli.md CHANGED
@@ -1,18 +1,34 @@
1
1
  # CLI reference
2
2
 
3
- Supercov runs locally. No command uploads source or coverage evidence.
3
+ Supercov has one command for measuring a suite and a small set of commands for
4
+ reading the result. Text output is designed for people and coding agents. Add
5
+ `--json` only when an integration needs a stable machine-readable response.
4
6
 
5
7
  ```sh
6
8
  npx supercov --help
7
9
  ```
8
10
 
11
+ ## Quick reference
12
+
13
+ | Goal | Command |
14
+ | --- | --- |
15
+ | Measure a suite | `npx supercov -- <test command>` |
16
+ | List recent runs | `npx supercov runs` |
17
+ | Read the newest run | `npx supercov runs latest` |
18
+ | Find useful gaps | `npx supercov runs latest gaps` |
19
+ | Inspect one file | `npx supercov runs latest file <path>` |
20
+ | Compare two runs | `npx supercov diff <older> <newer>` |
21
+ | Combine shards | `npx supercov merge <id> <id> [...]` |
22
+ | Remove local data | `npx supercov clean` |
23
+ | Read bundled guides | `npx supercov docs` |
24
+
9
25
  ## Measure a test command
10
26
 
11
27
  ```sh
12
28
  npx supercov -- <test command>
13
29
  ```
14
30
 
15
- Everything after `--` is the command Supercov measures.
31
+ Everything after `--` is passed to the test command:
16
32
 
17
33
  ```sh
18
34
  npx supercov -- npm test
@@ -21,42 +37,45 @@ npx supercov -- cargo test
21
37
  npx supercov -- cargo nextest run
22
38
  ```
23
39
 
24
- A coverage run exits with the test command's own status, so the wrapped command
25
- can remain a CI gate.
40
+ Use the complete command you rely on before merging or deploying. A coverage run
41
+ preserves the wrapped command's exit status, so it can remain a CI gate.
26
42
 
27
- ## List runs
43
+ ## List and select runs
28
44
 
29
45
  ```sh
30
- npx supercov runs [--limit N] [--json]
46
+ npx supercov runs
47
+ npx supercov runs --limit 5
48
+ npx supercov runs latest
49
+ npx supercov runs <run-id>
31
50
  ```
32
51
 
33
- Runs are listed newest first. Use an immutable run id when work spans a session;
34
- use `latest` for interactive work.
52
+ Runs are listed newest first. `latest` is convenient during an interactive
53
+ loop. Use the immutable run id in automation, review notes, and work that spans
54
+ sessions.
35
55
 
36
- ## Query one run
56
+ ## Query a run
37
57
 
38
58
  ```sh
39
59
  npx supercov runs <run-id> [query] [options]
40
60
  ```
41
61
 
42
- | Query | What it answers |
62
+ | Query | Use it to |
43
63
  | --- | --- |
44
- | no query | Overall completeness and measurement limits |
45
- | `kinds` | Coverage by semantic level, such as unit or E2E |
46
- | `runners` | Coverage by test runner |
47
- | `scope` | Included, excluded, and ambiguous source files |
48
- | `files` | All included files, ranked |
49
- | `gaps` | Files with useful open obligations or measurement limits |
50
- | `file <path>` | Open obligations in one file |
51
- | `decision <id \| path:line>` | Observed decision vectors and missing witnesses |
52
- | `line <path:line>` | Line state, nested obligations, and covering tests |
53
- | `test <id \| name>` | What one test contributes |
54
- | `minimize` | The smallest test subset that preserves selected coverage |
64
+ | no query | Read the overall result, test outcome, completeness, and timings |
65
+ | `gaps` | See only files with uncovered behavior or measurement limits |
66
+ | `files` | See every included file, including fully covered files |
67
+ | `file <path>` | Inspect the open obligations in one file |
68
+ | `decision <id \| path:line>` | Understand missing boolean outcomes and MC/DC witnesses |
69
+ | `line <path:line>` | See one line's state, obligations, and covering tests |
70
+ | `test <id \| name>` | See the coverage attributed to one test |
71
+ | `kinds` | Group coverage by test level, such as unit or E2E |
72
+ | `runners` | Group coverage by test runner |
73
+ | `scope` | Review included, excluded, and ambiguous source files |
74
+ | `minimize` | Find a small test subset that preserves a coverage target |
55
75
 
56
76
  Common examples:
57
77
 
58
78
  ```sh
59
- npx supercov runs latest
60
79
  npx supercov runs latest gaps --limit 10
61
80
  npx supercov runs latest file app/routes/checkout.ts
62
81
  npx supercov runs latest decision app/routes/checkout.ts:42
@@ -64,38 +83,70 @@ npx supercov runs latest line app/routes/checkout.ts:57
64
83
  npx supercov runs latest test "checkout retry"
65
84
  ```
66
85
 
67
- ## Query options
86
+ Run any query with `--help` to see only the options valid for that query:
87
+
88
+ ```sh
89
+ npx supercov runs latest --help
90
+ npx supercov runs latest file --help
91
+ ```
92
+
93
+ ## Narrow a view
68
94
 
69
95
  | Option | Meaning |
70
96
  | --- | --- |
71
- | `--filter all \| passed \| failed` | Choose which test attempts contribute. `all` is the default. |
72
- | `--kind <kind>` | Restrict to a test level such as `unit`, `integration`, or `e2e`. |
73
- | `--runner <runner>` | Restrict a summary to one runner. |
74
- | `--limit N`, `--offset N` | Page through collection results. |
75
- | `--metric all \| lines \| statements \| functions \| branches \| mcdc` | Choose the obligations preserved by `minimize`. |
76
- | `--target 0..100` | Stop `minimize` when the selected metric reaches the target. |
77
- | `--json` | Return the stable machine-readable form when an integration needs it. |
97
+ | `--filter all \| passed \| failed` | Recalculate the view from all, successful, or failed attempts |
98
+ | `--kind <kind>` | Restrict to a test level such as `unit`, `integration`, or `e2e` |
99
+ | `--runner <runner>` | Restrict to one runner |
100
+ | `--metric all \| lines \| statements \| functions \| branches \| mcdc` | Choose a metric for `files`, `gaps`, `diff`, or `minimize` |
101
+ | `--limit N`, `--offset N` | Page through a collection |
102
+ | `--json` | Return the machine-readable form |
78
103
 
79
- Collections print a copyable next-page command. Ordinary text output is intended
80
- to work well for both people and coding agents.
104
+ Collection output includes a copyable command for the next page.
105
+
106
+ For a large file, group and rank its decisions:
107
+
108
+ ```sh
109
+ npx supercov runs latest file app/routes/checkout.ts \
110
+ --group decision --sort missing
111
+ ```
81
112
 
82
113
  ## Compare runs
83
114
 
84
115
  ```sh
85
- npx supercov diff <older-run> <newer-run> [--limit N] [--json]
116
+ npx supercov diff <older-run> <newer-run>
86
117
  ```
87
118
 
88
- `diff` shows both gains and losses. Neither input run is changed.
119
+ `diff` reports gains and losses. Use it after adding a test to prove that the
120
+ expected behavior became covered without an unexplained regression elsewhere.
121
+ Neither input run is changed.
89
122
 
90
- ## Merge shards
123
+ The same filters can focus a comparison:
91
124
 
92
125
  ```sh
93
- npx supercov merge <run-id> <run-id> [...]
126
+ npx supercov diff <older-run> <newer-run> --kind e2e
94
127
  ```
95
128
 
96
- `merge` creates a new run from compatible shards. If source, configuration,
97
- toolchain, schema, or denominator fingerprints differ, it fails clearly rather
98
- than producing an invalid aggregate.
129
+ ## Find a smaller test set
130
+
131
+ ```sh
132
+ npx supercov runs latest minimize
133
+ npx supercov runs latest minimize --metric branches --target 90
134
+ ```
135
+
136
+ `minimize` finds a small set of tests that preserves the selected coverage
137
+ target. It does not edit, delete, or skip tests for you. Treat the result as an
138
+ analysis aid, not permission to remove tests that protect behavior outside the
139
+ selected metric.
140
+
141
+ ## Combine shards
142
+
143
+ ```sh
144
+ npx supercov merge <shard-a> <shard-b> <shard-c>
145
+ ```
146
+
147
+ Merge creates a new run. The inputs must describe the same source,
148
+ configuration, toolchain, schema, and coverage denominator. Supercov rejects an
149
+ incompatible merge rather than publishing a misleading aggregate.
99
150
 
100
151
  ## Clean local data
101
152
 
@@ -106,27 +157,38 @@ npx supercov clean
106
157
  ```
107
158
 
108
159
  By default, `clean` removes all stored runs and the isolated build cache.
109
- `--keep N` preserves the newest N runs. Cleanup only removes marker-owned
110
- Supercov storage.
160
+ `--keep N` keeps the newest N runs. Cleanup removes only marker-owned Supercov
161
+ storage.
111
162
 
112
163
  ## Read bundled documentation
113
164
 
114
165
  ```sh
115
166
  npx supercov docs
116
167
  npx supercov docs getting-started
168
+ npx supercov docs troubleshooting
117
169
  ```
118
170
 
171
+ The guides are installed with the package, so they remain available in a
172
+ terminal or offline environment after the package has been downloaded.
173
+
119
174
  ## Environment variables
120
175
 
121
176
  | Variable | Use |
122
177
  | --- | --- |
123
- | `SUPERCOV_SOURCE_ROOTS` | Declare the authoritative first-party source roots when automatic scope is ambiguous. |
124
- | `SUPERCOV_TEST_KIND` | Declare the semantic level of the tests in the wrapped command. |
178
+ | `SUPERCOV_SOURCE_ROOTS` | Set comma-separated first-party source roots when automatic discovery is ambiguous |
179
+ | `SUPERCOV_TEST_KIND` | Label the wrapped command as a test level such as `unit` or `e2e` |
180
+
181
+ Examples:
182
+
183
+ ```sh
184
+ SUPERCOV_SOURCE_ROOTS=src,app npx supercov -- npm test
185
+ SUPERCOV_TEST_KIND=e2e npx supercov -- npx playwright test
186
+ ```
125
187
 
126
188
  ## Exit codes
127
189
 
128
190
  | Code | Meaning |
129
191
  | --- | --- |
130
- | `0` | The run or query succeeded. |
131
- | Test command's status | A coverage run preserves the wrapped command's exit status. |
132
- | `2` | Supercov itself could not complete the request. |
192
+ | `0` | The command or query succeeded |
193
+ | Wrapped command's code | The test command failed and Supercov preserved its status |
194
+ | `2` | Supercov could not complete the request |
@@ -1,51 +1,95 @@
1
- # Coverage model
1
+ # Understanding coverage
2
2
 
3
- Supercov turns source structure into a fixed set of obligations before the test
4
- run. Tests can cover those obligations, but they cannot silently change what
5
- 100% means.
3
+ Supercov answers a more useful question than “which lines ran?” It shows which
4
+ behaviors were exercised, which paths still need a test, and where measurement
5
+ was incomplete.
6
+
7
+ ## Gaps and measurement limits are different
8
+
9
+ An **uncovered gap** is behavior Supercov measured but did not observe. A test
10
+ may be able to close it.
11
+
12
+ A **measurement limit** means Supercov could not establish a complete boundary
13
+ for some code or execution. Common causes include ambiguous source scope,
14
+ dynamically created source, self-inspecting code, and an unsupported process or
15
+ runner boundary.
16
+
17
+ Supercov keeps those states separate. It does not turn “unknown” into
18
+ “uncovered,” and it does not round either one away to produce a reassuring 100%.
19
+
20
+ ```sh
21
+ npx supercov runs latest
22
+ npx supercov runs latest gaps
23
+ npx supercov runs latest scope
24
+ ```
6
25
 
7
26
  ## What Supercov measures
8
27
 
9
- | Obligation | Question |
28
+ | Metric | The question it answers |
10
29
  | --- | --- |
11
30
  | Line | Did execution reach this source line? |
12
31
  | Statement | Did this executable statement run? |
13
32
  | Function | Was this function entered? |
14
33
  | Branch | Did each alternative execute? |
15
- | Decision vector | Which combinations of conditions were observed? |
16
- | MC/DC witness | Was each condition shown to affect its decision independently? |
17
- | Value path | Did language constructs such as defaults, optional chains, and logical assignments take each meaningful path? |
34
+ | Decision vector | Which combinations of boolean conditions occurred? |
35
+ | MC/DC witness | Was each condition shown to affect the decision independently? |
36
+ | Value path | Did defaults, optional chains, logical assignments, and similar constructs take each meaningful path? |
18
37
 
19
- The exact obligations depend on the language and source construct. Query one
20
- file or decision to see the concrete missing behavior:
38
+ The exact obligations depend on the language and source construct. You do not
39
+ need to reason about all of them at once. Start with a file, then open a decision
40
+ or line only when the missing behavior needs explanation:
21
41
 
22
42
  ```sh
23
43
  npx supercov runs latest file app/checkout/session.ts
24
44
  npx supercov runs latest decision app/checkout/session.ts:64
45
+ npx supercov runs latest line app/checkout/session.ts:64
25
46
  ```
26
47
 
27
- ## Why a line percentage is not enough
48
+ ## Why line coverage is not enough
28
49
 
29
- A line can execute while an important outcome remains untested. For example,
30
- this decision has two conditions:
50
+ Consider:
31
51
 
32
52
  ```js
33
53
  if (user.isAdmin || user.ownsDocument) allowEdit();
34
54
  ```
35
55
 
36
- Executing the line proves very little by itself. Useful tests should show the
37
- admin condition matters, the ownership condition matters, and the denied path
38
- still works. Decision-vector and MC/DC queries expose the missing cases directly.
56
+ The line can run even if the suite never proves that administrators are allowed,
57
+ owners are allowed, and everyone else is denied. Branches, decision vectors, and
58
+ MC/DC expose those missing cases instead of treating one executed line as proof
59
+ that the decision is safe.
39
60
 
40
61
  The same principle applies to Rust boolean expressions and control flow.
41
62
 
42
- ## Evidence confidence
63
+ ## What 100% means
64
+
65
+ Supercov derives the coverage denominator from source structure before the test
66
+ run. Adding or removing tests cannot silently change the definition of 100%.
67
+
68
+ A complete result means every declared obligation was measured and covered. It
69
+ does not mean the product has no bugs, the assertions are meaningful, or every
70
+ possible input was tested. Review test quality and user-visible behavior, not
71
+ only the percentage.
72
+
73
+ If source cannot be measured safely, Supercov reports a measurement limit
74
+ instead of claiming completeness.
43
75
 
44
- Where the runner exposes exact test boundaries, Supercov can show which test and
45
- attempt covered an obligation. Where it does not, execution is recorded as
46
- aggregate background evidence rather than assigned to a guessed test.
76
+ ## Exact and aggregate evidence
47
77
 
48
- Use attempt filters to choose the evidence included in a view:
78
+ When a runner exposes test and attempt boundaries, Supercov can show which test
79
+ covered an obligation. When it cannot, Supercov records aggregate background
80
+ coverage without guessing which test caused it.
81
+
82
+ Both are useful:
83
+
84
+ - exact evidence helps you inspect or minimize individual tests;
85
+ - aggregate evidence still shows whether the whole suite reached the source.
86
+
87
+ See [Supported suites](supported-suites.md) for the attribution available from
88
+ each runner.
89
+
90
+ ## Recalculate a view from selected tests
91
+
92
+ The same stored run can answer different questions:
49
93
 
50
94
  ```sh
51
95
  npx supercov runs latest --filter all
@@ -53,32 +97,32 @@ npx supercov runs latest --filter passed
53
97
  npx supercov runs latest --filter failed
54
98
  ```
55
99
 
56
- Use `--kind` when the project distinguishes test levels:
100
+ `all` matches the usual whole-run view. `passed` shows evidence from successful
101
+ attempts. `failed` isolates failed attempts, including failed retries.
102
+
103
+ You can also focus on a test level or runner:
57
104
 
58
105
  ```sh
59
106
  npx supercov runs latest gaps --kind e2e
107
+ npx supercov runs latest gaps --runner playwright
60
108
  ```
61
109
 
62
- A filtered view recomputes obligations from the selected evidence. It does not
63
- filter a percentage that was already calculated from something else.
110
+ These views are recalculated from stored evidence. Supercov is not filtering a
111
+ percentage that was computed from a different set of tests.
64
112
 
65
- ## Complete, uncovered, and blocked
113
+ ## Fix source scope before chasing gaps
66
114
 
67
- An ordinary uncovered obligation can be closed by a test. A completeness
68
- blocker means Supercov cannot honestly claim the source was fully measured.
69
- Common blockers are:
115
+ If the summary reports ambiguous source scope, inspect it:
70
116
 
71
- - ambiguous first-party source scope;
72
- - source that must remain uninstrumented because code observes its own text;
73
- - dynamically created source without a stable pre-run denominator; and
74
- - execution that crossed an unsupported or unattributed runner boundary.
117
+ ```sh
118
+ npx supercov runs latest scope
119
+ ```
75
120
 
76
- Inspect source scope with:
121
+ When first-party source lives in unusual directories, declare it explicitly:
77
122
 
78
123
  ```sh
79
- npx supercov runs latest scope
124
+ SUPERCOV_SOURCE_ROOTS=src,app npx supercov -- npm test
80
125
  ```
81
126
 
82
- If automatic scope is ambiguous, declare the authoritative roots with
83
- `SUPERCOV_SOURCE_ROOTS`. Other blockers remain visible with their location and
84
- reason. Supercov does not round them away to produce a comfortable 100%.
127
+ Choose roots that describe code the repository owns. Do not include dependencies
128
+ or generated output merely to make a warning disappear.
package/docs/evidence.md CHANGED
@@ -1,43 +1,49 @@
1
- # Evidence and runs
1
+ # Runs and evidence
2
2
 
3
- Every completed Supercov run is an immutable local record of what the suite
4
- executed. Queries, comparisons, and filtered views are derived from that record.
3
+ Every completed Supercov run is a local snapshot of what the suite executed.
4
+ You can return to it, ask different questions, or compare it with a later run
5
+ without rerunning the tests.
5
6
 
6
- ## What a run contains
7
+ ## Find the run you want
7
8
 
8
- A run records:
9
+ ```sh
10
+ npx supercov runs
11
+ npx supercov runs --limit 10
12
+ npx supercov runs latest
13
+ ```
14
+
15
+ Use `latest` during an interactive loop. Use the printed run id when:
9
16
 
10
- - the fixed coverage denominator for the measured source;
11
- - observed lines, statements, functions, branches, and decision vectors;
12
- - test, attempt, runner, outcome, and phase identity where the runner exposes it;
13
- - source, test, dependency, configuration, toolchain, and schema fingerprints;
14
- - completeness blockers and unattributed background execution; and
15
- - phase timings and integrity information.
17
+ - a task spans more than one session;
18
+ - an automated worker must not race a newer run;
19
+ - you are recording evidence in a pull request; or
20
+ - you need a reproducible comparison later.
16
21
 
17
- Completed runs live under `.supercov/runs/<run-id>/`. The original evidence is
18
- not rewritten when you query it. Supercov may build a disposable local index to
19
- answer later queries faster; that index is derived data and can always be
20
- recreated from the immutable run.
22
+ A run id is immutable. `latest` is only a convenient selector.
21
23
 
22
- ## Read a run
24
+ ## Ask the same run different questions
23
25
 
24
26
  ```sh
25
- npx supercov runs --limit 10
26
- npx supercov runs latest
27
27
  npx supercov runs latest gaps --limit 10
28
28
  npx supercov runs latest file app/checkout/session.ts
29
+ npx supercov runs latest line app/checkout/session.ts:64
29
30
  ```
30
31
 
31
- Use `latest` while working interactively. Use the run id printed by `runs` for
32
- automation, review notes, and work that spans sessions.
32
+ These queries read stored evidence. They do not run the test suite again or
33
+ rewrite the original run. Supercov may create a disposable local index to make
34
+ later queries faster; it can rebuild that index from the run.
35
+
36
+ ## Understand stale runs
33
37
 
34
- Queries compare the stored fingerprint with the current workspace. A stale run
35
- remains valid history, but it is no longer presented as a description of the
36
- current source.
38
+ Supercov compares a stored run with the current workspace. If relevant source,
39
+ tests, dependencies, configuration, or toolchain inputs changed, the run is
40
+ marked stale.
37
41
 
38
- ## Filter attempts
42
+ A stale run is still valid history. It simply should not be presented as the
43
+ current state of the repository. Run the complete suite again before choosing
44
+ new work from it.
39
45
 
40
- The same run can answer different questions without rerunning the suite:
46
+ ## Focus on passed or failed attempts
41
47
 
42
48
  ```sh
43
49
  npx supercov runs latest --filter all
@@ -45,46 +51,54 @@ npx supercov runs latest --filter passed
45
51
  npx supercov runs latest --filter failed
46
52
  ```
47
53
 
48
- - `all` includes every executed attempt and matches conventional coverage tools.
49
- - `passed` includes successful attempts of tests that ultimately passed.
54
+ - `all` includes every executed attempt and matches the normal whole-run view.
55
+ - `passed` uses successful attempts from tests that ultimately passed.
50
56
  - `failed` isolates failed attempts, including failed retries of flaky tests.
51
57
 
52
- Filtered views are recomputed from the stored evidence. They are not separate
53
- report files that can drift apart.
58
+ The views are recalculated from the same stored evidence. They are not separate
59
+ reports that can drift apart.
54
60
 
55
- ## Compare two runs
61
+ ## Compare before and after
56
62
 
57
63
  ```sh
58
64
  npx supercov diff <older-run> <newer-run>
59
65
  ```
60
66
 
61
- The diff shows newly covered and newly uncovered obligations. It is the easiest
62
- way to prove that a focused test changed coverage without losing behavior
63
- elsewhere.
67
+ The diff shows both newly covered and newly uncovered obligations. Use it after
68
+ a focused test to answer:
69
+
70
+ 1. Did the expected behavior become covered?
71
+ 2. Did anything unexpectedly become uncovered?
72
+ 3. Did the source boundary or measurement completeness change?
73
+
74
+ Keep the compared run ids in the agent summary or pull request when someone may
75
+ need to reproduce the result.
64
76
 
65
- ## Combine shards
77
+ ## Combine distributed shards
66
78
 
67
79
  ```sh
68
- npx supercov merge <shard-a> <shard-b> [...]
80
+ npx supercov merge <shard-a> <shard-b> <shard-c>
69
81
  ```
70
82
 
71
- Merge creates a new immutable run. Shards must have matching source,
72
- configuration, toolchain, schema, and denominator fingerprints. Input runs are
73
- never changed.
83
+ Merge creates a new run and leaves the inputs unchanged. Shards must describe
84
+ the same source, configuration, toolchain, schema, and coverage denominator.
85
+ Supercov rejects incompatible inputs rather than producing a misleading total.
74
86
 
75
- ## Integrity and incomplete evidence
87
+ ## What a run remembers
76
88
 
77
- Supercov validates evidence before publishing a run. Corrupt, truncated,
78
- duplicated, or contradictory input is rejected or surfaced as an explicit
79
- measurement limit. It is never silently converted into a clean percentage.
89
+ A run contains the coverage denominator, observed obligations, test and runner
90
+ identity where available, test outcomes and attempts, source and configuration
91
+ identity, measurement limits, and phase timings.
80
92
 
81
- Likewise, ambiguous source scope, uninstrumented code, and execution without a
82
- reliable test identity remain visible. See [Coverage model](/docs/coverage-model)
83
- for how these states affect completeness.
93
+ That is enough to answer later queries while keeping the original evidence
94
+ immutable. Corrupt, truncated, stale, or incompatible data is surfaced as such;
95
+ it is not opened as a plausible clean report.
84
96
 
85
- ## Retention
97
+ ## Storage and retention
86
98
 
87
- Runs remain until you remove them:
99
+ Completed runs live under `.supercov/runs/<run-id>/`. The isolated workspace and
100
+ instrumented build cache may use more space than the compressed run itself.
101
+ Nothing is pruned in the background.
88
102
 
89
103
  ```sh
90
104
  npx supercov clean --dry-run
@@ -92,5 +106,6 @@ npx supercov clean --keep 20
92
106
  npx supercov clean
93
107
  ```
94
108
 
95
- Cleanup takes the same project lock as a coverage run and removes only
109
+ Preview cleanup first. The final command removes all runs and the isolated build
110
+ cache; `--keep 20` preserves the 20 newest runs. Cleanup removes only
96
111
  marker-owned Supercov data.