supercov 0.0.26 → 0.0.28

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,39 +1,38 @@
1
- # Verification
1
+ # Trusting a result
2
2
 
3
- Coverage is useful only if instrumentation preserves program behavior and the
4
- reported obligations match what actually executed. Supercov fails closed when
5
- it cannot establish either condition.
3
+ Coverage is useful only when the test command still behaves normally and the
4
+ report distinguishes what was covered from what could not be measured.
5
+ Supercov is designed to fail clearly rather than turn uncertainty into a clean
6
+ percentage.
6
7
 
7
- ## What release checks cover
8
+ ## Review a test written by an agent
8
9
 
9
- Every release is checked for:
10
+ Before merging, check three things:
10
11
 
11
- - identical return values, thrown errors, and side-effect order before and
12
- after instrumentation;
13
- - short-circuiting, getters, proxies, optional calls, `this` binding, defaults,
14
- exceptions, loops, async functions, and generators;
15
- - exact line, branch, decision-vector, and MC/DC results;
16
- - JavaScript behavior across a pinned TC39 Test262 corpus;
17
- - supported JavaScript, TypeScript, and Rust runner contracts;
18
- - Chromium, Firefox, and WebKit browser execution;
19
- - source isolation, interrupted-run recovery, and atomic publication; and
20
- - package installation and execution from a clean project.
12
+ 1. **The complete wrapped test command passes.** A focused test command is useful
13
+ during iteration, but it is not the final gate.
14
+ 2. **The diff shows the expected gain.** Look for unexplained coverage losses or
15
+ a changed source boundary.
16
+ 3. **The test protects behavior.** It should make a meaningful assertion without
17
+ weakening existing assertions or changing application code for the metric.
21
18
 
22
- MC/DC cases are also compared with an independent LLVM implementation so a
23
- self-consistent error in Supercov's own calculation does not pass unnoticed.
19
+ ```sh
20
+ npx supercov diff <baseline-run> <new-run>
21
+ npx supercov runs <new-run> test "new test name"
22
+ ```
24
23
 
25
- ## What happens when code cannot be measured safely
24
+ Keep the compared run ids in the pull request or agent summary when the result
25
+ needs to be reviewed later.
26
26
 
27
- Supercov does not force a transform through code that observes its own source
28
- text or creates source dynamically without a stable denominator. It leaves the
29
- affected behavior uninstrumented and records a completeness blocker with the
30
- reason and location.
27
+ ## Read the status honestly
31
28
 
32
- Similarly, evidence from an unsupported runner or hidden remote boundary is
33
- reported as aggregate, unattributed, or missing. It is not assigned to a test
34
- that may not have caused it.
29
+ Supercov separates four states:
35
30
 
36
- Inspect these states with:
31
+ - **covered** the obligation was measured and observed;
32
+ - **uncovered** — it was measured but not observed, so a test may close it;
33
+ - **measurement limit** — Supercov could not establish a complete boundary;
34
+ - **stale** — the run is valid history but no longer describes the current
35
+ workspace.
37
36
 
38
37
  ```sh
39
38
  npx supercov runs latest
@@ -41,28 +40,43 @@ npx supercov runs latest scope
41
40
  npx supercov runs latest gaps
42
41
  ```
43
42
 
44
- ## How to review a coverage change
43
+ Do not treat a measurement limit as a test-writing task. Read its reason, fix
44
+ source scope or configuration when possible, and otherwise report the boundary.
45
45
 
46
- For a test added by a coding agent, check three things:
46
+ ## How Supercov avoids false confidence
47
47
 
48
- 1. The wrapped test command still passes.
49
- 2. `diff` shows the expected obligations gained and no unexplained loss.
50
- 3. The test contains meaningful assertions and does not weaken application
51
- behavior merely to improve a percentage.
48
+ - The coverage denominator is derived before the suite runs, so adding tests
49
+ cannot silently redefine 100%.
50
+ - Instrumentation runs in an isolated copy instead of rewriting your source.
51
+ - Evidence is attributed to a test only when the runner exposes a reliable
52
+ identity; everything else remains aggregate or background evidence.
53
+ - Completed runs are immutable and checked against the current workspace.
54
+ - Missing, corrupt, contradictory, or unsupported evidence is surfaced as a
55
+ limitation instead of being guessed or discarded.
52
56
 
53
- ```sh
54
- npx supercov diff <baseline-run> <new-run>
55
- npx supercov runs <new-run> test "new test name"
56
- ```
57
+ ## What happens when code cannot be measured safely
58
+
59
+ Some code observes its own source text, creates source dynamically, or executes
60
+ behind a launcher Supercov cannot see. Forcing ordinary instrumentation through
61
+ those boundaries could change behavior or invent a denominator.
62
+
63
+ Supercov leaves the affected behavior uninstrumented and reports the location
64
+ and reason. Likewise, coverage from a runner without exact test identity stays
65
+ aggregate instead of being assigned to whichever test happened to be nearby.
66
+
67
+ ## How releases are checked
57
68
 
58
- Store the compared run ids in the review or agent summary when the evidence
59
- needs to be reproducible later.
69
+ Release checks cover program behavior before and after instrumentation, line and
70
+ branch results, decision vectors and MC/DC, supported JavaScript, TypeScript,
71
+ and Rust runner contracts, Chromium, Firefox, and WebKit execution, clean
72
+ installation, interrupted-run recovery, and isolated publication.
60
73
 
61
- ## Integrity of stored runs
74
+ JavaScript behavior is exercised against a pinned TC39 Test262 corpus. MC/DC
75
+ cases are also compared with an independent LLVM implementation so a
76
+ self-consistent calculation error does not pass unnoticed.
62
77
 
63
- Completed runs are immutable and integrity-bound to their evidence,
64
- fingerprints, and schema. A run whose bytes are missing, corrupt, stale, or
65
- incompatible is surfaced as such instead of being opened as a plausible report.
78
+ These checks reduce risk; they do not replace reviewing the assertions and
79
+ behavior protected by a new test.
66
80
 
67
- See [Evidence and runs](/docs/evidence) for retention, comparison, and shard
68
- merging.
81
+ See [Runs and evidence](evidence.md) for immutable run ids, comparisons, and
82
+ retention.
@@ -1,25 +1,18 @@
1
- # Workspace isolation
1
+ # Files, privacy, and cleanup
2
2
 
3
3
  Supercov measures an instrumented copy of the project. It does not rewrite the
4
- source tree you edit or the ordinary build output your project already owns.
4
+ source tree you edit or send your source and coverage evidence to a hosted
5
+ Supercov service.
5
6
 
6
- ## What Supercov may write
7
+ ## What uses the network
7
8
 
8
- | Location | Purpose |
9
- | --- | --- |
10
- | `.supercov/runs/<run-id>/` | Immutable completed runs |
11
- | `.supercov/work/` | In-progress state and evidence staging |
12
- | `.supercov/locks/` | Prevents overlapping run and cleanup operations |
13
- | `supercov/workspace/<project>/` | Marker-protected isolated source and build cache |
14
-
15
- Supercov owns these locations only when its exact marker is present. If the
16
- project already contains a user-created `supercov/` directory, Supercov does
17
- not adopt or delete it; it chooses a deterministic fallback location instead.
9
+ The first `npx supercov` invocation may contact the npm registry to download the
10
+ package. Your wrapped test command may also use the network if it normally does.
18
11
 
19
- The managed directories contain their own gitignore rules so run evidence and
20
- instrumented builds do not become normal repository changes.
12
+ The Supercov CLI does not need a Supercov account or upload a coverage run to
13
+ Supercov. Run evidence and query indexes stay on the machine.
21
14
 
22
- ## What remains untouched
15
+ ## What stays untouched
23
16
 
24
17
  Supercov does not intentionally edit:
25
18
 
@@ -27,33 +20,57 @@ Supercov does not intentionally edit:
27
20
  - imports or dependency declarations;
28
21
  - test-runner configuration or reporter lists;
29
22
  - the project's ordinary build output; or
30
- - files outside its marker-owned storage.
23
+ - files outside marker-owned Supercov storage.
24
+
25
+ The wrapped command still has its normal side effects. If `npm test` writes
26
+ snapshots, calls a service, or changes a database, wrapping it does not remove
27
+ that behavior. The isolation guarantee applies to Supercov's instrumentation
28
+ and evidence work.
31
29
 
32
- The wrapped test command can still create anything it normally creates. The
33
- isolation guarantee applies to Supercov's additional instrumentation, evidence,
34
- and build work—not to side effects authored into the command itself.
30
+ Files the wrapped command creates or changes inside the isolated workspace are
31
+ synced back to the project after the run, so `supercov -- npm test -- -u`
32
+ updates snapshots in the repository exactly as `npm test -- -u` would. Two
33
+ exceptions are reported instead of applied: changes the command makes to
34
+ instrumented source files (the instrumented copies must never overwrite your
35
+ sources) and deletions (never propagated automatically).
35
36
 
36
- ## Repeated runs
37
+ ## Files Supercov creates
38
+
39
+ | Location | What it is for |
40
+ | --- | --- |
41
+ | `.supercov/runs/<run-id>/` | Completed immutable runs |
42
+ | `.supercov/work/` | Temporary state while a run is being prepared |
43
+ | `.supercov/locks/` | Prevents two operations from racing |
44
+ | `.supercov-workspace/workspace/<project>/` | Isolated source and reusable instrumented build cache (safe to delete) |
37
45
 
38
- When source, configuration, dependencies, toolchain, and build mode match,
39
- Supercov can reuse the isolated instrumented build. Test-only changes do not
40
- force an unrelated application rebuild.
46
+ Managed directories include Git ignore rules so run evidence and instrumented
47
+ builds do not become ordinary repository changes.
41
48
 
42
- Workspace updates are prepared separately and published only when complete, so
43
- a failed refresh does not replace the last complete cache with a partial one.
49
+ Supercov owns a directory only when its exact marker is present. If the project
50
+ already has a user-created `supercov/` directory, the CLI chooses a deterministic
51
+ fallback instead of adopting or deleting it.
44
52
 
45
- ## Crashes and concurrent commands
53
+ ## Repeated runs and the build cache
46
54
 
47
- One project can have one coverage or cleanup transaction at a time. A second
55
+ When source, dependencies, configuration, toolchain, and build mode still
56
+ match, Supercov can reuse the isolated instrumented build. Test-only changes do
57
+ not force an unrelated application rebuild.
58
+
59
+ Workspace refreshes are prepared separately and become active only when
60
+ complete. An interrupted refresh does not replace the last complete cache with
61
+ partial output.
62
+
63
+ ## Interrupted and overlapping commands
64
+
65
+ One project can run one coverage or cleanup transaction at a time. A second
48
66
  operation fails clearly instead of racing the first.
49
67
 
50
- In-progress state records allow the next command to recover after interruption,
51
- forced termination, or host restart. Unpublished staging data is discarded;
52
- completed runs are published atomically and remain immutable.
68
+ After interruption or host restart, the next command recovers unpublished
69
+ staging state. Completed runs remain immutable.
53
70
 
54
- ## Cleanup
71
+ ## Clean up local data
55
72
 
56
- Preview cleanup before deleting local Supercov data:
73
+ Preview cleanup before removing anything:
57
74
 
58
75
  ```sh
59
76
  npx supercov clean --dry-run
@@ -61,12 +78,16 @@ npx supercov clean --keep 20
61
78
  npx supercov clean
62
79
  ```
63
80
 
64
- Cleanup follows marker ownership and refuses to race an active run. It does not
65
- scan for similarly named directories or delete paths supplied by run metadata.
81
+ The final command removes all runs and the isolated build cache. `--keep 20`
82
+ retains the 20 newest runs. Cleanup follows marker ownership, waits for the
83
+ project lock, and does not scan for similarly named directories.
66
84
 
67
85
  ## Containers and remote workspaces
68
86
 
69
87
  When a suite launches a container or VM from a mounted workspace, Supercov uses
70
- the isolated workspace as the source presented to that environment. If an
71
- executor hides its launch or mount boundary, Supercov reports the limitation
72
- instead of claiming that unseen code was measured.
88
+ the isolated workspace as the source presented to that environment. The runtime
89
+ must be able to cross the launch boundary and return evidence.
90
+
91
+ If a remote executor hides the launch or mount boundary, Supercov reports the
92
+ limitation instead of claiming unseen code was measured. See
93
+ [Supported suites](supported-suites.md) for the current boundary.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "supercov",
3
- "version": "0.0.26",
3
+ "version": "0.0.28",
4
4
  "description": "Zero-edit, runner-aware coverage completeness for JavaScript test suites",
5
5
  "license": "MIT",
6
6
  "repository": {
@@ -68,12 +68,12 @@
68
68
  "prepublishOnly": "npm run release:check"
69
69
  },
70
70
  "optionalDependencies": {
71
- "@supercov/cli-darwin-arm64": "0.0.26",
72
- "@supercov/cli-darwin-x64": "0.0.26",
73
- "@supercov/cli-linux-arm64-gnu": "0.0.26",
74
- "@supercov/cli-linux-arm64-musl": "0.0.26",
75
- "@supercov/cli-linux-x64-gnu": "0.0.26",
76
- "@supercov/cli-linux-x64-musl": "0.0.26"
71
+ "@supercov/cli-darwin-arm64": "0.0.28",
72
+ "@supercov/cli-darwin-x64": "0.0.28",
73
+ "@supercov/cli-linux-arm64-gnu": "0.0.28",
74
+ "@supercov/cli-linux-arm64-musl": "0.0.28",
75
+ "@supercov/cli-linux-x64-gnu": "0.0.28",
76
+ "@supercov/cli-linux-x64-musl": "0.0.28"
77
77
  },
78
78
  "peerDependencies": {
79
79
  "@playwright/test": ">=1.55.0",