supercov 0.0.43 → 0.0.45

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/README.md +23 -14
  2. package/docs/agent-loop.md +128 -39
  3. package/docs/assertion-agent.md +156 -0
  4. package/docs/assertion-evidence.md +9 -135
  5. package/docs/assertion-maps.md +252 -0
  6. package/docs/assertions.md +82 -0
  7. package/docs/cli.md +38 -23
  8. package/docs/coverage-model.md +18 -6
  9. package/docs/evidence.md +6 -6
  10. package/docs/getting-started.md +60 -72
  11. package/docs/performance.md +4 -4
  12. package/docs/troubleshooting.md +8 -8
  13. package/docs/verification.md +2 -2
  14. package/docs/workspace-isolation.md +1 -1
  15. package/package.json +34 -33
  16. package/runtime/javascript/nodeAssertAdapter.mjs +32 -8
  17. package/runtime/javascript/nodeTest.mjs +13 -5
  18. package/runtime/javascript/runnerEvidence.mjs +33 -11
  19. package/runtime/javascript/runtime.mjs +27 -23
  20. package/schemas/assertions.schema.json +276 -0
  21. package/analyzers/typescript/README.md +0 -55
  22. package/analyzers/typescript/bin/compiler-identity.mjs +0 -78
  23. package/analyzers/typescript/bin/identity.mjs +0 -67
  24. package/analyzers/typescript/bin/query.mjs +0 -29
  25. package/analyzers/typescript/dist/analyze.js +0 -3972
  26. package/analyzers/typescript/dist/archive.js +0 -309
  27. package/analyzers/typescript/dist/build-identity.json +0 -1
  28. package/analyzers/typescript/dist/compiler.js +0 -32
  29. package/analyzers/typescript/dist/frontend.js +0 -75
  30. package/analyzers/typescript/dist/native-frontend.js +0 -271
  31. package/analyzers/typescript/dist/pragmas.js +0 -143
  32. package/analyzers/typescript/dist/types.js +0 -1
  33. package/analyzers/typescript/package.json +0 -27
  34. package/analyzers/typescript/src/analyze.ts +0 -4538
  35. package/analyzers/typescript/src/archive.ts +0 -438
  36. package/analyzers/typescript/src/compiler.ts +0 -49
  37. package/analyzers/typescript/src/frontend.ts +0 -136
  38. package/analyzers/typescript/src/native-frontend.ts +0 -315
  39. package/analyzers/typescript/src/pragmas.ts +0 -218
  40. package/analyzers/typescript/src/types.ts +0 -45
  41. package/analyzers/typescript/tsconfig.json +0 -12
  42. package/docs/code-verification.md +0 -182
package/README.md CHANGED
@@ -10,8 +10,8 @@ No account, config file, import, custom reporter, or hosted service is required.
10
10
 
11
11
  Supported by [Supercorp](https://supercorp.ai).
12
12
 
13
- [Try the worked example](https://supercov.com/docs/code-verification): find an
14
- untested session-expiry condition, run an additional test, and compare the results.
13
+ [Agent workflow](https://supercov.com/docs/agent-loop): ask your coding agent
14
+ to add a test in your own project, with a recorded example to follow along.
15
15
 
16
16
  ## Start with the suite you already have
17
17
 
@@ -46,22 +46,30 @@ npx supercov -- python -m unittest
46
46
  npx supercov -- bundle exec rspec
47
47
  ```
48
48
 
49
- ## Inspect assertion evidence (JS/TS)
49
+ ## Map what assertions check
50
50
 
51
- After a run, inspect what existing assertions appear to check:
51
+ Each normal test run creates an assertion map automatically. An agent can
52
+ optionally fill in what the assertions check:
52
53
 
53
54
  ```sh
54
- npx supercov runs latest assertions --limit 5
55
- npx supercov runs latest assertions --pragmas --json
56
- npx supercov runs latest assertions --evidence /tests --limit 5 --json
55
+ npx supercov runs latest assertions --json
56
+ # Pin the returned run ID. Inspect assertions and the source that ran:
57
+ npx supercov runs <run> assertion <assertion-id>
58
+ npx supercov runs <run> source src/example.ts
59
+ # Edit assertions.json, then validate and acknowledge:
60
+ npx supercov runs <run> assertions validate --json
61
+ # Copy examined expectedBasis tokens into assertions.json; save again.
62
+ npx supercov runs <run> assertions check --require-mappings --require-observed --json
63
+ npx supercov runs <run> # includes the assertion percentage in the regular report
57
64
  ```
58
65
 
59
- This npm-only query uses the run archive and matching source; it adds no new
60
- test-time instrumentation. It requires a compatible project TypeScript compiler
61
- API (5.8.3 and native 7.0.2 are tested). Results are candidates, not proof
62
- that changes are safe or a verified assertion percentage. Follow
63
- [`assertion-evidence.md`](docs/assertion-evidence.md) for provenance, optional
64
- assertion hints, pagination, and limitations.
66
+ After code or tests change, run the same test command again. Supercov reuses the
67
+ newest available map for that command and language and identifies dirty flows. Rust owns
68
+ validation, change tracking and reporting; the agent supplies semantic reasoning.
69
+ The score is agent-assessed and separate from MC/DC. See
70
+ [assertion maps](docs/assertion-maps.md) for the format and JS/TS limits, or run
71
+ `supercov docs assertion-agent` for the agent workflow. `supercov assertions schema`
72
+ exports the editor schema; `assertions validate --file <path>` checks JSON syntax.
65
73
 
66
74
  ## Give Supercov a job
67
75
 
@@ -170,7 +178,7 @@ Supercov uses exact per-test attribution where an adapter is available. For othe
170
178
  | --- | --- |
171
179
  | Playwright | Exact per test, worker, retry, outcome, action, and assertion phase |
172
180
  | Vitest | Exact per test, with setup execution kept separate |
173
- | Jest | Exact per test, including parameterized tests; `expect` assertions link the evidence they check |
181
+ | Jest | Exact per test, including parameterized tests; exact `expect` occurrence identity for assertion maps |
174
182
  | `node:test` | Exact per test |
175
183
  | AVA and Mocha | Aggregate structural coverage |
176
184
  | Cargo's standard libtest runner | Exact test, attempt, and passing-assertion identity |
@@ -223,6 +231,7 @@ npx supercov clean # remove all runs and the build cache
223
231
 
224
232
  - [Getting started](https://supercov.com/docs/getting-started)
225
233
  - [Agent workflow](https://supercov.com/docs/agent-loop)
234
+ - [Understanding assertions](docs/assertions.md)
226
235
  - [Troubleshooting](https://supercov.com/docs/troubleshooting)
227
236
  - [CLI reference](https://supercov.com/docs/cli)
228
237
  - [Supported languages and test suites](https://supercov.com/docs/supported-suites)
@@ -1,41 +1,33 @@
1
1
  # Agent workflow
2
2
 
3
- Supercov works best as a small, repeatable loop: run the suite, choose one useful
4
- gap, write one test, rerun, and prove what improved.
3
+ Use Supercov with your coding agent and the test suite you already have.
4
+ Supercov reports coverage and gaps. Your agent writes a test, reruns the suite,
5
+ and checks what improved.
5
6
 
6
- ```text
7
- run the suite → choose a gap → write one test → rerun → compare
8
- ↑ |
9
- └────────────────────────────────────────────────────────────┘
10
- ```
11
-
12
- Supercov supplies the coverage signal and evidence. Your coding agent writes
13
- the tests.
7
+ ## Start with one test
14
8
 
15
- ## Choose the job
9
+ Open your own repository in your coding agent and paste this prompt. You don't
10
+ need to install Supercov first; the agent can handle that.
16
11
 
17
- For one careful first pass, ask:
18
-
19
- ```text
20
- Measure code coverage with `npx supercov` and write the first useful test based
21
- on coverage. Only edit tests. Rerun the complete suite and report what improved.
12
+ ```text supercov-prompt
13
+ Measure code coverage with npx supercov and write one missing test.
14
+ Only change tests. Rerun the full test suite and show me the test you
15
+ added and the before-and-after coverage.
22
16
  ```
23
17
 
24
- For an overnight run or leftover token budget, ask:
18
+ If the project has several test commands, tell the agent which full suite to
19
+ use. Let it run the commands and edit the tests, approving those actions if
20
+ your agent asks.
25
21
 
26
- ```text
27
- Use `npx supercov` to improve coverage. Only write tests. Keep going while
28
- useful gaps remain. Never weaken assertions or change application code to make
29
- coverage easier. Stop at a measurement limit, unreachable behavior, or the end
30
- of the available time budget. Report the run ids compared and what improved.
31
- ```
32
-
33
- The second prompt is intentionally open-ended, but 100% is a direction rather
34
- than permission to write meaningless tests or reshape application code.
22
+ The result is a normal test-file change and a coverage comparison in the
23
+ conversation. Ask separately if you want a commit or pull request.
35
24
 
36
25
  ## One safe pass
37
26
 
38
- ```sh
27
+ The agent should run the suite, inspect a gap, write a test, then rerun the
28
+ same suite and compare. These are the commands it can use:
29
+
30
+ ```sh supercov-example
39
31
  # 1. Establish a baseline.
40
32
  npx supercov -- npm test
41
33
 
@@ -52,29 +44,126 @@ npx supercov -- npm test
52
44
  npx supercov diff <previous-run-id> latest
53
45
  ```
54
46
 
55
- For Rust, use `cargo test` or `cargo nextest run` in both runs. Keep the baseline
56
- and verification commands identical.
47
+ Everything after `--` is your project's test command. Use your actual command
48
+ and file paths in place of the examples. For example,
49
+ Rust projects can use `cargo test`, Python projects `pytest`, and Ruby projects
50
+ `bundle exec rspec`. Keep the baseline and verification commands identical.
57
51
 
58
52
  The `line` query is useful before writing a test because it shows which tests
59
53
  already reach that line. Extending a nearby test is often better than adding a
60
54
  duplicate.
61
55
 
62
- For supported JS/TS projects, inspect assertion evidence before adding tests:
56
+ Every new test run creates `assertions.json`. To investigate what tests assert:
63
57
 
64
58
  ```sh
65
- npx supercov runs latest assertions --file app/checkout/session.ts --limit 5 --json
66
- npx supercov runs latest assertions --pragmas --json
59
+ npx supercov runs latest assertions --json
60
+ # Pin data.run and edit the file at data.map.
67
61
  ```
68
62
 
69
- This post-run query needs the matching source and a compatible
70
- project TypeScript API. An `evident` candidate is not a proof of safety; keep
71
- test gaps separate from analysis limits. Follow the returned evidence pointers
72
- and `pagination.nextOffset`, pinning `--analysis` and the run id while paging.
73
- See [assertion evidence](assertion-evidence.md) for requirements and examples.
63
+ Inspect an entry with `runs <run> assertion <id>` and read archived code with
64
+ `runs <run> source <path>`. Edit its `assertions.json`, declare credited nodes
65
+ and broader watch inputs, then obtain expectedBasis tokens with `assertions validate --json` and save the examined tokens in the map or repeated `--flow ASSERTION/FLOW`. After the next run of
66
+ the same command, its map automatically inherits prior work; repair affected entries. Supercov
67
+ validates references and freshness, while the agent owns semantic meaning.
68
+ Run `supercov docs assertion-agent` for the full mapping instructions. Validate
69
+ with `runs <run> assertions validate`, save examined expectedBasis tokens in the map, then use
70
+ `runs <run> assertions check --require-mappings --require-observed` as the
71
+ completion gate. See [assertion maps](assertion-maps.md) for the contract and JS/TS limits.
72
+
73
+ ## Example
74
+
75
+ Here's a recorded Codex run in a JavaScript project, using the first prompt.
76
+ The files are from our checkout example; you don't need to add them to your
77
+ project.
78
+
79
+ [`src/session.js`](https://github.com/supercorp-ai/supercov/blob/main/examples/checkout-verification/starter/src/session.js)
80
+ allows checkout only when the customer is signed in and their session has
81
+ not expired:
82
+
83
+ ```js
84
+ export function canCheckout(signedIn, expired) {
85
+ if (signedIn && !expired) return true;
86
+ return false;
87
+ }
88
+ ```
74
89
 
75
- ## A complete prompt for longer runs
90
+ The two tests in
91
+ [`tests/session.test.js`](https://github.com/supercorp-ai/supercov/blob/main/examples/checkout-verification/starter/tests/session.test.js)
92
+ check a valid session and a signed-out visitor:
93
+
94
+ ```js
95
+ assert.equal(canCheckout(true, false), true);
96
+ assert.equal(canCheckout(false, false), false);
97
+ ```
98
+
99
+ The agent ran `npx supercov -- npm test`. Both tests passed, and the summary
100
+ from `npx supercov runs latest` showed:
76
101
 
77
102
  ```text
103
+ Coverage
104
+ Lines 100.00% (3/3)
105
+ Branches 100.00% (2/2)
106
+ MC/DC 50.00% (1/2)
107
+ ```
108
+
109
+ It listed the gaps and inspected the file. You can open those views with:
110
+
111
+ ```sh
112
+ npx supercov runs latest gaps
113
+ npx supercov runs latest file src/session.js
114
+ ```
115
+
116
+ The file query explained the gap:
117
+
118
+ ```text
119
+ LINE STATUS SOURCE
120
+ 2 PARTIAL signedIn && !expired
121
+ Unobserved: no witness pair shows `!expired` independently changing the decision result
122
+ ```
123
+
124
+ Both return paths had run, but neither test checked an expired session. MC/DC
125
+ checks whether each condition has been shown to affect the decision
126
+ independently. Here, `signedIn` had; `!expired` had not.
127
+
128
+ The agent added one test to `tests/session.test.js`, leaving the application
129
+ code and existing tests unchanged:
130
+
131
+ ```js
132
+ test('a signed-in visitor with an expired session cannot check out', () => {
133
+ assert.equal(canCheckout(true, true), false);
134
+ });
135
+ ```
136
+
137
+ It reran the same full suite. All three tests passed, and MC/DC reached 100%.
138
+ The comparison from `npx supercov diff <before-run-id> latest` showed:
139
+
140
+ ```text
141
+ lines +0pp, branches +0pp, MC/DC +50pp
142
+ gained: 0 lines, 0 branches, 1 MC/DC conditions
143
+ lost: 0 lines, 0 branches, 0 MC/DC conditions
144
+ + MC/DC src/session.js:2 C2 !expired
145
+ ```
146
+
147
+ The new assertion checks that checkout is denied when a signed-in customer's
148
+ session has expired. Removing the expiry check makes this test fail; the
149
+ original two tests still pass.
150
+
151
+ In your project, look for the same evidence: the test checks the behavior the
152
+ agent identified, and the full suite passes. One useful test won't necessarily
153
+ take coverage to 100%.
154
+
155
+ To try these exact files, [download the starter](https://supercov.com/downloads/supercov-tutorial.zip),
156
+ extract it, open the `supercov-tutorial` folder in your agent, and run `npm ci`.
157
+ Then use the JavaScript prompt above. The completed test is not included in the
158
+ download. The [recorded run](https://github.com/supercorp-ai/supercov/tree/main/examples/checkout-verification/agent-run)
159
+ includes the commands, full output, and completed test.
160
+
161
+ ## A complete prompt for longer runs
162
+
163
+ Once you've reviewed the first test, use this prompt to continue through
164
+ useful gaps—for example, during an overnight run:
165
+
166
+ ```text supercov-prompt
78
167
  Use `npx supercov` to improve coverage. Only write tests. Keep going while
79
168
  useful gaps remain.
80
169
 
@@ -110,7 +199,7 @@ reason to manufacture a test.
110
199
 
111
200
  If the repository separates test levels, narrow the view:
112
201
 
113
- ```sh
202
+ ```sh supercov
114
203
  npx supercov runs latest gaps --kind e2e --limit 10
115
204
  ```
116
205
 
@@ -0,0 +1,156 @@
1
+ # Agent workflow for assertion maps
2
+
3
+ You are the semantic author. Supercov supplies source identities, execution
4
+ records, file hashes and a JSON validator. It does not infer or prove your
5
+ explanations. Use your ordinary source-reading tools and edit one run-owned
6
+ `assertions.json`; no model is embedded in Supercov.
7
+
8
+ ## Start with a real run
9
+
10
+ 1. Run the intended suite through `supercov -- <test command>` if no matching
11
+ current run exists. Use the full suite for a suite-wide result.
12
+ 2. Read `supercov runs --json`, pin its concrete run ID, then read
13
+ `supercov runs <run> assertions --json`. Edit `data.map`; the run creates it
14
+ automatically and inherits compatible prior work. Check `data.inheritance`
15
+ for fallback errors. Keep old runs and their maps.
16
+ 3. Read `supercov docs assertion-maps`. Page the assertion, statement, test and
17
+ change views. Do not mistake one page for the whole suite:
18
+
19
+ ```sh
20
+ supercov runs <run> assertions --limit 100 --json
21
+ supercov runs <run> assertions report --view statements --limit 100 --json
22
+ supercov runs <run> assertions report --view tests --limit 100 --json
23
+ supercov runs <run> assertions report --view changes --limit 100 --json
24
+ ```
25
+
26
+ Follow `pagination.nextOffset` with `--offset`; restart if `revision` changes.
27
+ Use a single map writer. Save edits atomically where your editor supports it.
28
+ The outer object contains `schemaVersion: 2` and `assertions`; do not insert run IDs.
29
+
30
+ ## Investigate precise claims
31
+
32
+ Read current test setup, inputs, mocks, callbacks, branches, callees, helper
33
+ modules and relevant configuration. Current files must match the pinned run.
34
+ Supercov retains hashes and identities, not a source checkout. Optional
35
+ `runs <run> source <path> --offset 0 --limit 100` prints matching code with line
36
+ numbers; `assertions files` lists input hashes even when files no longer match.
37
+
38
+ Use `runs <run> assertion <id>` for the exact site and graph. Preserve its ID and
39
+ full `at` anchor, usually the complete assertion call without a trailing semicolon.
40
+ Use one-based UTF-8 byte columns, not character counts. An `inMap: false` site
41
+ was recognized but is missing from the map; restore it when investigating.
42
+
43
+ For each assertion:
44
+
45
+ - Describe exactly what its predicate distinguishes in `observes`. Truthiness,
46
+ existence, length or substring checks do not imply equality of every field.
47
+ - Write independently maintainable flows with stable IDs, `basis: null`, a
48
+ concrete explanation, exact nodes and authored edges to `$assertion`.
49
+ - Use explicit `appliesTo: [{file, name}]` from the test view. File and name
50
+ must resolve unambiguously. Empty means no credit. Runtime IDs never belong
51
+ in selectors. Shared/parameterized sites may need multiple cases or flows.
52
+ - Take production statement anchors from the `statements` view when possible.
53
+ Put only nodes you judge asserted in `countsAsAsserted`. Each counted node
54
+ needs an authored path to `$assertion`; a block does not credit nested code.
55
+ - Put additional dependency paths in `watch`, as whole-file strings. Assertion,
56
+ selected-test and node files are already dependencies. Include setup, guards,
57
+ alternate paths and helpers your explanation relies on, even without nodes.
58
+ - Preserve uncertainty in `questions`. Flow questions block that flow's credit.
59
+ Assertion questions record unfinished exploration without implying a known
60
+ total. There is no `analysis`, `mapped`, or `complete` flag.
61
+ - A fixture-only or absence explanation can have `countsAsAsserted: []`. Explain
62
+ the absence, including ordering/barriers and the observation window. Never
63
+ credit an unexecuted body just because its execution would violate a check.
64
+
65
+ Inspect `selectors`, `blockers`, `reasons`, empty `observedPassingTests`, and
66
+ statement `at: null` entries. They may represent skipped tests, unsupported
67
+ syntax, ambiguous test names or missing evidence. Do not borrow another site's
68
+ identity, fabricate events, delete inconvenient entries or edit managed state.
69
+ Zero-credit explanations are useful; invented credit is not.
70
+
71
+ ## Account for changes before finalizing flow tokens
72
+
73
+ When inheriting a map, inspect **every** item in `--view changes`, including files
74
+ already watched by some flows. Dependencies may be missing. In root
75
+ `changeAssessments`, write one response per managed change ID:
76
+
77
+ ```json
78
+ {
79
+ "id": "c_copy_from_changes_view",
80
+ "basis": null,
81
+ "affectedFlows": ["assertion-id/flow-id"],
82
+ "explanation": "What changed, which claims it affects, and why other existing claims remain valid."
83
+ }
84
+ ```
85
+
86
+ Include all known dependent flows still present, plus any others affected.
87
+ Repair watches if the change reveals a missing dependency. An empty list needs
88
+ an actual explanation. Removing a response does not clear the outstanding change.
89
+ A changed test/assertion can affect all its flows; a production file used by one
90
+ sibling may affect only that flow. No machine can tell you an expected flow total.
91
+
92
+ Save the responses and graph edits. Run:
93
+
94
+ ```sh
95
+ supercov assertions validate --file <map-path> --json
96
+ supercov runs <run> assertions validate --json
97
+ ```
98
+
99
+ For large maps, page `validate --view changes`, `validate --view flows`, or
100
+ `validate --view errors` with `--offset` and `--limit`; tokens appear under
101
+ `items` and overall validity still covers the whole map. Pin `revision` across pages.
102
+
103
+ Validation checks shape, IDs, graph links, anchors and state binding. It returns
104
+ `changes[].expectedBasis` and `flows[].expectedBasis`; null/stale tokens alone
105
+ are unfinished work, not syntax errors. Copy change tokens **only after inspecting
106
+ those impact assessments**, then save. Validate again and copy flow tokens only
107
+ for claims you have examined. Save the map again. Change acknowledgements can
108
+ invalidate additional flows, so obtain final flow tokens after change tokens.
109
+ Changing a graph, selector, observation or watch after obtaining its token makes
110
+ that token stale. Do not implement the hashing algorithm or manufacture tokens.
111
+
112
+ There is no `review` command. All commands are read-only; acknowledgement is your
113
+ explicit file edit. Passing reference validation does not prove your reasoning.
114
+
115
+ ## Check and report
116
+
117
+ ```sh
118
+ supercov runs <run> assertions check --require-mappings --require-observed --json
119
+ supercov runs <run> assertions report --view statements --limit 100 --json
120
+ supercov runs <run>
121
+ ```
122
+
123
+ Basic `check` permits untouched sites without flows, but fails on invalid or
124
+ unfinished authored claims, open questions and outstanding change impact.
125
+ `--require-mappings` requires at least one current observed explanation for every
126
+ recognized passing site, including legitimate zero-credit explanations.
127
+ `--require-observed` checks every site and selector. Neither means every semantic
128
+ flow was found. Use `--min <percentage>` only for the project's chosen target.
129
+ Do not lower a target or hide unknowns to make a check pass.
130
+
131
+ The normal report includes the percentage immediately from the edited file.
132
+ `notAssessed`, `pending`, `unavailable` and `notApplicable` have null percentages,
133
+ not fabricated zeroes. A numeric partial-map result still reports counts of
134
+ assertions without flows and claims needing work. Describe the score as
135
+ agent-assessed statements; successful same-test coexecution is evidence, not
136
+ proof of causality or mutation resistance.
137
+
138
+ ## Continue after edits
139
+
140
+ Run the same suite command again. The new run carries IDs, explanations and
141
+ unchanged acknowledgements using prior file hashes, without needing old source.
142
+ Use `assertions --needs-attention` and the change view. Repair only affected
143
+ claims; keep current siblings. Any dependency byte edit, including comments,
144
+ requires rechecking. Unique relocation preserves identity as a suggestion;
145
+ ambiguous/removed sites remain in `retiredAssertions`. Old v1 flows are imported
146
+ as drafts with questions about selectors and graph paths; inspect them explicitly.
147
+
148
+ Do not edit `assertions.state.json`. Dirty generations persist across reruns and
149
+ reverts until current tokens are recorded. Resolved change responses are folded
150
+ and retired on the next publication. Runtime evidence never carries forward.
151
+ If source differs from the pinned run, rerun tests before continuing.
152
+
153
+ Finish with the pinned run ID, status, numeric percentage/counts when available,
154
+ assertions without flows, remaining questions/stale or unobserved claims, and
155
+ verification commands used. Explain remaining limits. The property in `observes`
156
+ matters: changing a credited line while preserving that property can still pass.
@@ -1,135 +1,9 @@
1
- # Assertion evidence (JS/TS)
2
-
3
- Supercov can analyze which source behaviors existing assertions appear to check,
4
- using an ordinary run archive, its matching source, and the existing statement
5
- and assertion-phase evidence. This work happens after tests. It adds no new
6
- test-time probes, does not run mutants, and does not rewrite your tests.
7
-
8
- This is candidate evidence, not a proof that arbitrary changes are safe.
9
- `assertionScore` remains null. Execution-only links, test gaps and analysis limits
10
- are different things; do not interpret a passing assertion nearby as protection.
11
-
12
- ## Requirements
13
-
14
- Use the **npm Supercov launcher**, which supplies the installed analyzer assets.
15
- The current JS/TS adapter is exercised with Node's test runner and Vitest.
16
- Browser, background, merged, retried and ambiguously attributed records have
17
- explicit limitations; other language assertion analyzers are not enabled here.
18
-
19
- The analyzed project must provide a TypeScript compiler API, **even for a
20
- JavaScript project**. For example, add TypeScript as a development dependency
21
- before recording the run. The analyzer uses that project's compiler, not a
22
- silently substituted global or bundled version. Installing it afterward changes
23
- the dependency fingerprint, so rerun the suite. Missing/incompatible compiler APIs
24
- produce an error. **TypeScript 5.8.3 and native 7.0.2 are tested.** Version 7.0.2
25
- uses its own native parser/checker, not a fallback to TypeScript 5. Install its
26
- platform-specific optional dependency too; the report hashes both the JS client
27
- and the native compiler package, including its standard libraries. This backend
28
- requires Node 22.12 or newer (Node 24 tested), and uses original-source evidence
29
- from ordinary Supercov archives, not legacy ts-node/V8 generated-line coverage.
30
- Native module resolution currently follows actual import/export references;
31
- unresolved helper-only specifiers remain visible as compiler limitations.
32
- Other native compiler versions are not enabled until separately calibrated.
33
- Do not downgrade an application's compiler just to improve an assertion report.
34
-
35
- Compiler compatibility and regression checks do not prove general correctness.
36
- All candidates remain unverified and `assertionScore` remains null. Query-side
37
- native compiler work adds no test-time instrumentation.
38
-
39
- Standalone native/Python/Ruby distributions do not currently bundle this JS/TS
40
- analyzer. Use the npm launcher, or explicitly set `SUPERCOV_PACKAGE_ROOT` to an
41
- installed npm package directory. Normal coverage commands are unaffected.
42
-
43
- ## Run, inspect, follow evidence
44
-
45
- `assertions` analyzes source behaviors and their assertion evidence; it does not
46
- just count or list assertion calls. Use `npx supercov docs assertion-evidence`
47
- to read this guide from the installed package.
48
-
49
- ```sh
50
- npx supercov -- npm test
51
- npx supercov runs latest assertions --limit 5
52
- npx supercov runs latest assertions --file src/core.ts --json
53
- npx supercov runs latest assertions --site '<site-id>' --json
54
- ```
55
-
56
- The summary reports candidate counts and the site denominator, without claiming
57
- a verified assertion percentage. Each ordinary site row contains its source,
58
- candidate classification, available facts, and an `evidence.pointer` for full
59
- details. Global tests, attempts, execution links, diagnostics and source-scope
60
- limits are referenced under `evidence`, not repeated inside every site page.
61
-
62
- ```sh
63
- npx supercov runs latest assertions --evidence /tests --limit 5 --json
64
- npx supercov runs latest assertions --evidence /diagnostics --json
65
- npx supercov runs latest assertions --evidence /sites/0/facts --json
66
- ```
67
-
68
- Pointers use JSON Pointer syntax: escape `/` in a property name as `~1` and `~`
69
- as `~0`. Use an empty pointer (`--evidence ''`) for the document root. Follow
70
- returned pointers rather than constructing indices from filtered page offsets:
71
- site pointers index the complete, stable inventory.
72
-
73
- ## Pagination and oversized records
74
-
75
- All JSON responses stay within the normal response budget. Pages may contain
76
- fewer than `--limit` results: **follow `pagination.nextOffset`**, not offset plus
77
- the requested limit. `hasMore: false` marks the end.
78
-
79
- An oversized site/hint is returned as `detailOnly: true` with an
80
- `evidence.pointer`. Nothing has been discarded. An evidence page returns immediate
81
- object members or array entries as `items`. Small entries contain `value`; large
82
- ones contain `detailOnly: true` and another `pointer` to inspect. String leaves
83
- return `text` chunks, with offsets/counts measured in Unicode scalar values,
84
- not bytes. This allows reading one large observation or diagnostic completely.
85
-
86
- ```sh
87
- npx supercov runs '<run-id>' assertions --evidence '<returned-pointer>' \
88
- --offset 0 --limit 4000 --analysis '<analysisId>' --json
89
- ```
90
-
91
- Every page carries an `analysisId` hashing the complete derived document and its
92
- provenance. Pass the first page's id as `--analysis` on follow-ups to reject mixed
93
- analyses if the compiler, analyzer, or derived results change. Use a fixed run id
94
- instead of `latest` while paging. Source/run freshness checks still run on every
95
- query. Derived results are not cached or written into the archive.
96
-
97
- JSON reports use `reportSchema: 2`. Shared evidence is accessed through the
98
- evidence pointers above.
99
- Success and error JSON envelopes identify this query as `coverage.assertions`.
100
-
101
- ## Optional assertion hints
102
-
103
- ```ts
104
- // observes: src/core.ts#compute return value
105
- assert.equal(compute(), 4);
106
- ```
107
-
108
- The comment must precede a statement containing exactly one recognized assertion.
109
- The project-relative file, exact function/owner name, and optional literal source
110
- substring must select one inventory site. Optional `via ...` is explanation only.
111
-
112
- ```sh
113
- npx supercov runs latest assertions --pragmas --json
114
- ```
115
-
116
- Origin is `user-suggested`; validation is separately `analyzer-supported`,
117
- `unresolved`, or `invalid`. Support requires the named assertion's exact passing
118
- witness and a connection supported by the ordinary effect rules. Strength remains
119
- presence/value/total; presence does not mean value protection. Unsupported
120
- decision, absence and internal-state paths remain unresolved. A missing inventory
121
- match may reflect unsupported source rather than a bad declaration.
122
-
123
- Hints cannot inject observations, borrow another assertion's evidence, inflate
124
- coverage or remove sites from the denominator. Editing comments changes test
125
- source and requires a new run. Supported hints retain the analyzer's limitations;
126
- they are not formal proofs.
127
-
128
- ## Agent workflow
129
-
130
- Read coverage first, then inspect assertion candidates and their evidence before
131
- writing another test. An execution gap may need a reachable scenario; an assertion
132
- gap may need a stronger check. An analysis limit needs investigation—not a test
133
- written just to satisfy the analyzer. Keep the original denominator and unresolved
134
- work visible. Never automatically treat an `evident` candidate as permission to
135
- change application behavior.
1
+ # Assertion evidence
2
+
3
+ The earlier automatic JS/TS analyzer has been replaced by agent-authored
4
+ assertion maps. Use [Understanding assertion coverage](assertions.md) to get
5
+ started, [Agent-authored assertion maps](assertion-maps.md) for the file format
6
+ and CLI reference, and [the agent instructions](assertion-agent.md) to build or
7
+ update a map. Existing structural runs remain readable; mapping requires a new
8
+ run with an assertion input manifest. Old `--pragmas`, `--analysis` and `--evidence`
9
+ options do not apply to the map workflow.