supercov 0.0.44 → 0.0.45

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/README.md +21 -12
  2. package/docs/agent-loop.md +12 -8
  3. package/docs/assertion-agent.md +156 -0
  4. package/docs/assertion-evidence.md +9 -694
  5. package/docs/assertion-maps.md +252 -0
  6. package/docs/assertions.md +82 -0
  7. package/docs/cli.md +23 -8
  8. package/docs/coverage-model.md +12 -0
  9. package/package.json +34 -35
  10. package/runtime/javascript/runtime.mjs +18 -33
  11. package/schemas/assertions.schema.json +276 -0
  12. package/analyzers/typescript/README.md +0 -59
  13. package/analyzers/typescript/bin/compiler-identity.mjs +0 -78
  14. package/analyzers/typescript/bin/identity.mjs +0 -71
  15. package/analyzers/typescript/bin/query.mjs +0 -29
  16. package/analyzers/typescript/dist/analyze.js +0 -5273
  17. package/analyzers/typescript/dist/archive.js +0 -337
  18. package/analyzers/typescript/dist/awaited-observations.js +0 -376
  19. package/analyzers/typescript/dist/build-identity.json +0 -1
  20. package/analyzers/typescript/dist/compiler.js +0 -32
  21. package/analyzers/typescript/dist/frontend.js +0 -75
  22. package/analyzers/typescript/dist/mock-counts.js +0 -2517
  23. package/analyzers/typescript/dist/native-frontend.js +0 -271
  24. package/analyzers/typescript/dist/pragmas.js +0 -186
  25. package/analyzers/typescript/dist/types.js +0 -1
  26. package/analyzers/typescript/package.json +0 -27
  27. package/analyzers/typescript/src/analyze.ts +0 -6180
  28. package/analyzers/typescript/src/archive.ts +0 -471
  29. package/analyzers/typescript/src/awaited-observations.ts +0 -561
  30. package/analyzers/typescript/src/compiler.ts +0 -49
  31. package/analyzers/typescript/src/frontend.ts +0 -136
  32. package/analyzers/typescript/src/mock-counts.ts +0 -3219
  33. package/analyzers/typescript/src/native-frontend.ts +0 -315
  34. package/analyzers/typescript/src/pragmas.ts +0 -284
  35. package/analyzers/typescript/src/types.ts +0 -45
  36. package/analyzers/typescript/tsconfig.json +0 -12
  37. package/docs/code-verification.md +0 -4
package/README.md CHANGED
@@ -46,22 +46,30 @@ npx supercov -- python -m unittest
46
46
  npx supercov -- bundle exec rspec
47
47
  ```
48
48
 
49
- ## Inspect assertion evidence (JS/TS)
49
+ ## Map what assertions check
50
50
 
51
- After a run, inspect what existing assertions appear to check:
51
+ Each normal test run creates an assertion map automatically. An agent can
52
+ optionally fill in what the assertions check:
52
53
 
53
54
  ```sh
54
- npx supercov runs latest assertions --limit 5
55
- npx supercov runs latest assertions --pragmas --json
56
- npx supercov runs latest assertions --evidence /tests --limit 5 --json
55
+ npx supercov runs latest assertions --json
56
+ # Pin the returned run ID. Inspect assertions and the source that ran:
57
+ npx supercov runs <run> assertion <assertion-id>
58
+ npx supercov runs <run> source src/example.ts
59
+ # Edit assertions.json, then validate and acknowledge:
60
+ npx supercov runs <run> assertions validate --json
61
+ # Copy examined expectedBasis tokens into assertions.json; save again.
62
+ npx supercov runs <run> assertions check --require-mappings --require-observed --json
63
+ npx supercov runs <run> # includes the assertion percentage in the regular report
57
64
  ```
58
65
 
59
- This npm-only query uses the run archive and matching source; it adds no new
60
- test-time instrumentation. It requires a compatible project TypeScript compiler
61
- API (5.8.3 and native 7.0.2 are tested). Results are candidates, not proof
62
- that changes are safe or a verified assertion percentage. Follow
63
- [`assertion-evidence.md`](docs/assertion-evidence.md) for provenance, optional
64
- assertion hints, pagination, and limitations.
66
+ After code or tests change, run the same test command again. Supercov reuses the
67
+ newest available map for that command and language and identifies dirty flows. Rust owns
68
+ validation, change tracking and reporting; the agent supplies semantic reasoning.
69
+ The score is agent-assessed and separate from MC/DC. See
70
+ [assertion maps](docs/assertion-maps.md) for the format and JS/TS limits, or run
71
+ `supercov docs assertion-agent` for the agent workflow. `supercov assertions schema`
72
+ exports the editor schema; `assertions validate --file <path>` checks JSON syntax.
65
73
 
66
74
  ## Give Supercov a job
67
75
 
@@ -170,7 +178,7 @@ Supercov uses exact per-test attribution where an adapter is available. For othe
170
178
  | --- | --- |
171
179
  | Playwright | Exact per test, worker, retry, outcome, action, and assertion phase |
172
180
  | Vitest | Exact per test, with setup execution kept separate |
173
- | Jest | Exact per test, including parameterized tests; `expect` assertions link the evidence they check |
181
+ | Jest | Exact per test, including parameterized tests; exact `expect` occurrence identity for assertion maps |
174
182
  | `node:test` | Exact per test |
175
183
  | AVA and Mocha | Aggregate structural coverage |
176
184
  | Cargo's standard libtest runner | Exact test, attempt, and passing-assertion identity |
@@ -223,6 +231,7 @@ npx supercov clean # remove all runs and the build cache
223
231
 
224
232
  - [Getting started](https://supercov.com/docs/getting-started)
225
233
  - [Agent workflow](https://supercov.com/docs/agent-loop)
234
+ - [Understanding assertions](docs/assertions.md)
226
235
  - [Troubleshooting](https://supercov.com/docs/troubleshooting)
227
236
  - [CLI reference](https://supercov.com/docs/cli)
228
237
  - [Supported languages and test suites](https://supercov.com/docs/supported-suites)
@@ -53,18 +53,22 @@ The `line` query is useful before writing a test because it shows which tests
53
53
  already reach that line. Extending a nearby test is often better than adding a
54
54
  duplicate.
55
55
 
56
- For supported JS/TS projects, inspect assertion evidence before adding tests:
56
+ Every new test run creates `assertions.json`. To investigate what tests assert:
57
57
 
58
58
  ```sh
59
- npx supercov runs latest assertions --file app/checkout/session.ts --limit 5 --json
60
- npx supercov runs latest assertions --pragmas --json
59
+ npx supercov runs latest assertions --json
60
+ # Pin data.run and edit the file at data.map.
61
61
  ```
62
62
 
63
- This post-run query needs the matching source and a compatible
64
- project TypeScript API. An `evident` candidate is not a proof of safety; keep
65
- test gaps separate from analysis limits. Follow the returned evidence pointers
66
- and `pagination.nextOffset`, pinning `--analysis` and the run id while paging.
67
- See [assertion evidence](assertion-evidence.md) for requirements and examples.
63
+ Inspect an entry with `runs <run> assertion <id>` and read archived code with
64
+ `runs <run> source <path>`. Edit its `assertions.json`, declare credited nodes
65
+ and broader watch inputs, then obtain expectedBasis tokens with `assertions validate --json` and save the examined tokens in the map or repeated `--flow ASSERTION/FLOW`. After the next run of
66
+ the same command, its map automatically inherits prior work; repair affected entries. Supercov
67
+ validates references and freshness, while the agent owns semantic meaning.
68
+ Run `supercov docs assertion-agent` for the full mapping instructions. Validate
69
+ with `runs <run> assertions validate`, save examined expectedBasis tokens in the map, then use
70
+ `runs <run> assertions check --require-mappings --require-observed` as the
71
+ completion gate. See [assertion maps](assertion-maps.md) for the contract and JS/TS limits.
68
72
 
69
73
  ## Example
70
74
 
@@ -0,0 +1,156 @@
1
+ # Agent workflow for assertion maps
2
+
3
+ You are the semantic author. Supercov supplies source identities, execution
4
+ records, file hashes and a JSON validator. It does not infer or prove your
5
+ explanations. Use your ordinary source-reading tools and edit one run-owned
6
+ `assertions.json`; no model is embedded in Supercov.
7
+
8
+ ## Start with a real run
9
+
10
+ 1. Run the intended suite through `supercov -- <test command>` if no matching
11
+ current run exists. Use the full suite for a suite-wide result.
12
+ 2. Read `supercov runs --json`, pin its concrete run ID, then read
13
+ `supercov runs <run> assertions --json`. Edit `data.map`; the run creates it
14
+ automatically and inherits compatible prior work. Check `data.inheritance`
15
+ for fallback errors. Keep old runs and their maps.
16
+ 3. Read `supercov docs assertion-maps`. Page the assertion, statement, test and
17
+ change views. Do not mistake one page for the whole suite:
18
+
19
+ ```sh
20
+ supercov runs <run> assertions --limit 100 --json
21
+ supercov runs <run> assertions report --view statements --limit 100 --json
22
+ supercov runs <run> assertions report --view tests --limit 100 --json
23
+ supercov runs <run> assertions report --view changes --limit 100 --json
24
+ ```
25
+
26
+ Follow `pagination.nextOffset` with `--offset`; restart if `revision` changes.
27
+ Use a single map writer. Save edits atomically where your editor supports it.
28
+ The outer object contains `schemaVersion: 2` and `assertions`; do not insert run IDs.
29
+
30
+ ## Investigate precise claims
31
+
32
+ Read current test setup, inputs, mocks, callbacks, branches, callees, helper
33
+ modules and relevant configuration. Current files must match the pinned run.
34
+ Supercov retains hashes and identities, not a source checkout. Optional
35
+ `runs <run> source <path> --offset 0 --limit 100` prints matching code with line
36
+ numbers; `assertions files` lists input hashes even when files no longer match.
37
+
38
+ Use `runs <run> assertion <id>` for the exact site and graph. Preserve its ID and
39
+ full `at` anchor, usually the complete assertion call without a trailing semicolon.
40
+ Use one-based UTF-8 byte columns, not character counts. An `inMap: false` site
41
+ was recognized but is missing from the map; restore it when investigating.
42
+
43
+ For each assertion:
44
+
45
+ - Describe exactly what its predicate distinguishes in `observes`. Truthiness,
46
+ existence, length or substring checks do not imply equality of every field.
47
+ - Write independently maintainable flows with stable IDs, `basis: null`, a
48
+ concrete explanation, exact nodes and authored edges to `$assertion`.
49
+ - Use explicit `appliesTo: [{file, name}]` from the test view. File and name
50
+ must resolve unambiguously. Empty means no credit. Runtime IDs never belong
51
+ in selectors. Shared/parameterized sites may need multiple cases or flows.
52
+ - Take production statement anchors from the `statements` view when possible.
53
+ Put only nodes you judge asserted in `countsAsAsserted`. Each counted node
54
+ needs an authored path to `$assertion`; a block does not credit nested code.
55
+ - Put additional dependency paths in `watch`, as whole-file strings. Assertion,
56
+ selected-test and node files are already dependencies. Include setup, guards,
57
+ alternate paths and helpers your explanation relies on, even without nodes.
58
+ - Preserve uncertainty in `questions`. Flow questions block that flow's credit.
59
+ Assertion questions record unfinished exploration without implying a known
60
+ total. There is no `analysis`, `mapped`, or `complete` flag.
61
+ - A fixture-only or absence explanation can have `countsAsAsserted: []`. Explain
62
+ the absence, including ordering/barriers and the observation window. Never
63
+ credit an unexecuted body just because its execution would violate a check.
64
+
65
+ Inspect `selectors`, `blockers`, `reasons`, empty `observedPassingTests`, and
66
+ statement `at: null` entries. They may represent skipped tests, unsupported
67
+ syntax, ambiguous test names or missing evidence. Do not borrow another site's
68
+ identity, fabricate events, delete inconvenient entries or edit managed state.
69
+ Zero-credit explanations are useful; invented credit is not.
70
+
71
+ ## Account for changes before finalizing flow tokens
72
+
73
+ When inheriting a map, inspect **every** item in `--view changes`, including files
74
+ already watched by some flows. Dependencies may be missing. In root
75
+ `changeAssessments`, write one response per managed change ID:
76
+
77
+ ```json
78
+ {
79
+ "id": "c_copy_from_changes_view",
80
+ "basis": null,
81
+ "affectedFlows": ["assertion-id/flow-id"],
82
+ "explanation": "What changed, which claims it affects, and why other existing claims remain valid."
83
+ }
84
+ ```
85
+
86
+ Include all known dependent flows still present, plus any others affected.
87
+ Repair watches if the change reveals a missing dependency. An empty list needs
88
+ an actual explanation. Removing a response does not clear the outstanding change.
89
+ A changed test/assertion can affect all its flows; a production file used by one
90
+ sibling may affect only that flow. No machine can tell you an expected flow total.
91
+
92
+ Save the responses and graph edits. Run:
93
+
94
+ ```sh
95
+ supercov assertions validate --file <map-path> --json
96
+ supercov runs <run> assertions validate --json
97
+ ```
98
+
99
+ For large maps, page `validate --view changes`, `validate --view flows`, or
100
+ `validate --view errors` with `--offset` and `--limit`; tokens appear under
101
+ `items` and overall validity still covers the whole map. Pin `revision` across pages.
102
+
103
+ Validation checks shape, IDs, graph links, anchors and state binding. It returns
104
+ `changes[].expectedBasis` and `flows[].expectedBasis`; null/stale tokens alone
105
+ are unfinished work, not syntax errors. Copy change tokens **only after inspecting
106
+ those impact assessments**, then save. Validate again and copy flow tokens only
107
+ for claims you have examined. Save the map again. Change acknowledgements can
108
+ invalidate additional flows, so obtain final flow tokens after change tokens.
109
+ Changing a graph, selector, observation or watch after obtaining its token makes
110
+ that token stale. Do not implement the hashing algorithm or manufacture tokens.
111
+
112
+ There is no `review` command. All commands are read-only; acknowledgement is your
113
+ explicit file edit. Passing reference validation does not prove your reasoning.
114
+
115
+ ## Check and report
116
+
117
+ ```sh
118
+ supercov runs <run> assertions check --require-mappings --require-observed --json
119
+ supercov runs <run> assertions report --view statements --limit 100 --json
120
+ supercov runs <run>
121
+ ```
122
+
123
+ Basic `check` permits untouched sites without flows, but fails on invalid or
124
+ unfinished authored claims, open questions and outstanding change impact.
125
+ `--require-mappings` requires at least one current observed explanation for every
126
+ recognized passing site, including legitimate zero-credit explanations.
127
+ `--require-observed` checks every site and selector. Neither means every semantic
128
+ flow was found. Use `--min <percentage>` only for the project's chosen target.
129
+ Do not lower a target or hide unknowns to make a check pass.
130
+
131
+ The normal report includes the percentage immediately from the edited file.
132
+ `notAssessed`, `pending`, `unavailable` and `notApplicable` have null percentages,
133
+ not fabricated zeroes. A numeric partial-map result still reports counts of
134
+ assertions without flows and claims needing work. Describe the score as
135
+ agent-assessed statements; successful same-test coexecution is evidence, not
136
+ proof of causality or mutation resistance.
137
+
138
+ ## Continue after edits
139
+
140
+ Run the same suite command again. The new run carries IDs, explanations and
141
+ unchanged acknowledgements using prior file hashes, without needing old source.
142
+ Use `assertions --needs-attention` and the change view. Repair only affected
143
+ claims; keep current siblings. Any dependency byte edit, including comments,
144
+ requires rechecking. Unique relocation preserves identity as a suggestion;
145
+ ambiguous/removed sites remain in `retiredAssertions`. Old v1 flows are imported
146
+ as drafts with questions about selectors and graph paths; inspect them explicitly.
147
+
148
+ Do not edit `assertions.state.json`. Dirty generations persist across reruns and
149
+ reverts until current tokens are recorded. Resolved change responses are folded
150
+ and retired on the next publication. Runtime evidence never carries forward.
151
+ If source differs from the pinned run, rerun tests before continuing.
152
+
153
+ Finish with the pinned run ID, status, numeric percentage/counts when available,
154
+ assertions without flows, remaining questions/stale or unobserved claims, and
155
+ verification commands used. Explain remaining limits. The property in `observes`
156
+ matters: changing a credited line while preserving that property can still pass.