supercov 0.0.45 → 0.0.46

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -59,7 +59,7 @@ npx supercov runs <run> source src/example.ts
59
59
  # Edit assertions.json, then validate and acknowledge:
60
60
  npx supercov runs <run> assertions validate --json
61
61
  # Copy examined expectedBasis tokens into assertions.json; save again.
62
- npx supercov runs <run> assertions check --require-mappings --require-observed --json
62
+ npx supercov runs <run> assertions check --require-mappings --json
63
63
  npx supercov runs <run> # includes the assertion percentage in the regular report
64
64
  ```
65
65
 
@@ -53,22 +53,26 @@ The `line` query is useful before writing a test because it shows which tests
53
53
  already reach that line. Extending a nearby test is often better than adding a
54
54
  duplicate.
55
55
 
56
- Every new test run creates `assertions.json`. To investigate what tests assert:
56
+ ## Add assertion coverage
57
57
 
58
- ```sh
59
- npx supercov runs latest assertions --json
60
- # Pin data.run and edit the file at data.map.
58
+ For JavaScript and TypeScript projects, you can also ask the agent to explain
59
+ what each test assertion checks. Every normal run creates an `assertions.json`
60
+ map and carries forward compatible work from earlier runs of the same command.
61
+
62
+ ```sh supercov
63
+ npx supercov runs latest assertions
64
+ npx supercov docs assertion-agent
61
65
  ```
62
66
 
63
- Inspect an entry with `runs <run> assertion <id>` and read archived code with
64
- `runs <run> source <path>`. Edit its `assertions.json`, declare credited nodes
65
- and broader watch inputs, then obtain expectedBasis tokens with `assertions validate --json` and save the examined tokens in the map or repeated `--flow ASSERTION/FLOW`. After the next run of
66
- the same command, its map automatically inherits prior work; repair affected entries. Supercov
67
- validates references and freshness, while the agent owns semantic meaning.
68
- Run `supercov docs assertion-agent` for the full mapping instructions. Validate
69
- with `runs <run> assertions validate`, save examined expectedBasis tokens in the map, then use
70
- `runs <run> assertions check --require-mappings --require-observed` as the
71
- completion gate. See [assertion maps](assertion-maps.md) for the contract and JS/TS limits.
67
+ Keep the printed run ID fixed while editing. The agent reads matching current
68
+ source, updates the map, validates its references and saves review tokens for
69
+ the explanations it examined. The regular run report then shows the assertion
70
+ percentage. It does not need another test run just to read the edited map.
71
+
72
+ Start with [Understanding assertion coverage](assertions.md) for a copyable
73
+ prompt. [Mapping assertions with an agent](assertion-agent.md) gives the complete
74
+ editing and checking loop. Use the evidence and recorded observations alongside
75
+ MC/DC when deciding which tests need stronger checks.
72
76
 
73
77
  ## Example
74
78
 
@@ -1,156 +1,146 @@
1
- # Agent workflow for assertion maps
2
-
3
- You are the semantic author. Supercov supplies source identities, execution
4
- records, file hashes and a JSON validator. It does not infer or prove your
5
- explanations. Use your ordinary source-reading tools and edit one run-owned
6
- `assertions.json`; no model is embedded in Supercov.
7
-
8
- ## Start with a real run
9
-
10
- 1. Run the intended suite through `supercov -- <test command>` if no matching
11
- current run exists. Use the full suite for a suite-wide result.
12
- 2. Read `supercov runs --json`, pin its concrete run ID, then read
13
- `supercov runs <run> assertions --json`. Edit `data.map`; the run creates it
14
- automatically and inherits compatible prior work. Check `data.inheritance`
15
- for fallback errors. Keep old runs and their maps.
16
- 3. Read `supercov docs assertion-maps`. Page the assertion, statement, test and
17
- change views. Do not mistake one page for the whole suite:
18
-
19
- ```sh
20
- supercov runs <run> assertions --limit 100 --json
21
- supercov runs <run> assertions report --view statements --limit 100 --json
22
- supercov runs <run> assertions report --view tests --limit 100 --json
23
- supercov runs <run> assertions report --view changes --limit 100 --json
24
- ```
25
-
26
- Follow `pagination.nextOffset` with `--offset`; restart if `revision` changes.
27
- Use a single map writer. Save edits atomically where your editor supports it.
28
- The outer object contains `schemaVersion: 2` and `assertions`; do not insert run IDs.
29
-
30
- ## Investigate precise claims
31
-
32
- Read current test setup, inputs, mocks, callbacks, branches, callees, helper
33
- modules and relevant configuration. Current files must match the pinned run.
34
- Supercov retains hashes and identities, not a source checkout. Optional
35
- `runs <run> source <path> --offset 0 --limit 100` prints matching code with line
36
- numbers; `assertions files` lists input hashes even when files no longer match.
37
-
38
- Use `runs <run> assertion <id>` for the exact site and graph. Preserve its ID and
39
- full `at` anchor, usually the complete assertion call without a trailing semicolon.
40
- Use one-based UTF-8 byte columns, not character counts. An `inMap: false` site
41
- was recognized but is missing from the map; restore it when investigating.
1
+ # Mapping assertions with an agent
2
+
3
+ Use these instructions when asking a coding agent to create or update an
4
+ assertion map. The agent reads your tests and source, explains what each
5
+ assertion checks, and edits the run's `assertions.json`. Supercov supplies the
6
+ run evidence and checks the file's references and freshness.
7
+
8
+ Start with [Understanding assertion coverage](assertions.md) for a prompt you
9
+ can copy. The steps below are the agent's working instructions.
10
+
11
+ ## Choose a run and keep it fixed
12
+
13
+ 1. Use a matching current run, or run the requested suite through
14
+ `supercov -- <test command>`. Use the full suite for a suite-wide result.
15
+ 2. Read `supercov runs latest assertions --json` and keep `data.run` fixed for
16
+ this investigation. Edit the file at `data.map`.
17
+ 3. Check `data.inheritance` for reused work or fallback errors. Keep the old
18
+ runs and maps. Inspect changes before renewing inherited flows.
19
+ 4. Read the [map format](assertion-maps.md), also available through
20
+ `supercov docs assertion-maps`.
21
+
22
+ Use a single writer for the map. Save atomically if your editor supports it.
23
+ Do not change application code or tests unless the user also requested those
24
+ changes. Never edit `assertions.state.json` or the run's recorded evidence.
25
+
26
+ ## Inspect the assertions and source
27
+
28
+ ```sh supercov-example
29
+ npx supercov runs <run-id> assertions --limit 100 --json
30
+ npx supercov runs <run-id> assertions report --view statements --limit 100 --json
31
+ npx supercov runs <run-id> assertions report --view tests --limit 100 --json
32
+ npx supercov runs <run-id> assertions report --view changes --limit 100 --json
33
+ npx supercov runs <run-id> assertion <assertion-id>
34
+ npx supercov runs <run-id> source src/shipping.js --offset 0 --limit 100
35
+ ```
42
36
 
43
- For each assertion:
37
+ Follow every `pagination.nextOffset`; a page is not the whole result. Restart a
38
+ paged read if its `revision` changes. Current source must match the pinned run.
39
+ Use ordinary source-reading tools to inspect setup, inputs, mocks, callbacks,
40
+ branches, called functions, helpers and relevant configuration.
44
41
 
45
- - Describe exactly what its predicate distinguishes in `observes`. Truthiness,
46
- existence, length or substring checks do not imply equality of every field.
47
- - Write independently maintainable flows with stable IDs, `basis: null`, a
48
- concrete explanation, exact nodes and authored edges to `$assertion`.
49
- - Use explicit `appliesTo: [{file, name}]` from the test view. File and name
50
- must resolve unambiguously. Empty means no credit. Runtime IDs never belong
51
- in selectors. Shared/parameterized sites may need multiple cases or flows.
52
- - Take production statement anchors from the `statements` view when possible.
53
- Put only nodes you judge asserted in `countsAsAsserted`. Each counted node
54
- needs an authored path to `$assertion`; a block does not credit nested code.
55
- - Put additional dependency paths in `watch`, as whole-file strings. Assertion,
56
- selected-test and node files are already dependencies. Include setup, guards,
57
- alternate paths and helpers your explanation relies on, even without nodes.
58
- - Preserve uncertainty in `questions`. Flow questions block that flow's credit.
59
- Assertion questions record unfinished exploration without implying a known
60
- total. There is no `analysis`, `mapped`, or `complete` flag.
61
- - A fixture-only or absence explanation can have `countsAsAsserted: []`. Explain
62
- the absence, including ordering/barriers and the observation window. Never
63
- credit an unexecuted body just because its execution would violate a check.
64
-
65
- Inspect `selectors`, `blockers`, `reasons`, empty `observedPassingTests`, and
66
- statement `at: null` entries. They may represent skipped tests, unsupported
67
- syntax, ambiguous test names or missing evidence. Do not borrow another site's
68
- identity, fabricate events, delete inconvenient entries or edit managed state.
69
- Zero-credit explanations are useful; invented credit is not.
70
-
71
- ## Account for changes before finalizing flow tokens
72
-
73
- When inheriting a map, inspect **every** item in `--view changes`, including files
74
- already watched by some flows. Dependencies may be missing. In root
75
- `changeAssessments`, write one response per managed change ID:
76
-
77
- ```json
78
- {
79
- "id": "c_copy_from_changes_view",
80
- "basis": null,
81
- "affectedFlows": ["assertion-id/flow-id"],
82
- "explanation": "What changed, which claims it affects, and why other existing claims remain valid."
83
- }
84
- ```
42
+ Preserve each assertion's ID and complete `at` anchor. Locations use one-based
43
+ lines and UTF-8 byte columns. Restore recognized sites marked `inMap: false`
44
+ when investigating them; removing an entry does not remove the assertion from
45
+ Supercov's list.
85
46
 
86
- Include all known dependent flows still present, plus any others affected.
87
- Repair watches if the change reveals a missing dependency. An empty list needs
88
- an actual explanation. Removing a response does not clear the outstanding change.
89
- A changed test/assertion can affect all its flows; a production file used by one
90
- sibling may affect only that flow. No machine can tell you an expected flow total.
47
+ ## Write precise explanations
91
48
 
92
- Save the responses and graph edits. Run:
49
+ For each assertion:
93
50
 
94
- ```sh
95
- supercov assertions validate --file <map-path> --json
96
- supercov runs <run> assertions validate --json
51
+ - Describe exactly what its predicate distinguishes in `observes`. Existence,
52
+ truthiness, length and substring checks do not imply equality of every field.
53
+ - Split independently maintainable explanations into flows with stable IDs.
54
+ Start a new or changed flow with `basis: null`.
55
+ - Choose `appliesTo` tests from the tests view using their exact file and name.
56
+ Select applicable cases explicitly for shared or parameterized assertions.
57
+ - Use complete source statements from the statements view as node anchors when
58
+ possible. Write the relationships that connect those nodes to `$assertion`.
59
+ - Put a node in `countsAsAsserted` only when its behavior is checked by this
60
+ assertion. Every counted node needs a recorded path to `$assertion`.
61
+ Counting a guard or block does not count the nested body automatically.
62
+ - Add helper, configuration and other dependency files to `watch`. Assertion,
63
+ selected-test and node files are already dependencies. Include relevant guards
64
+ and alternatives even when they do not appear as graph nodes.
65
+ - Keep uncertainty in `questions`. A flow's unresolved questions block its
66
+ credit. Assertion-level questions record broader unfinished investigation.
67
+
68
+ A fixture-only or absence explanation can have `countsAsAsserted: []`. For
69
+ absence, describe the ordering or barrier and the observation window. Never
70
+ credit an unexecuted body merely because executing it would violate the check.
71
+
72
+ There is no known number of flows an assertion should have. Do not label an
73
+ assertion complete just because you recorded one explanation.
74
+
75
+ ## Check why a node receives no credit
76
+
77
+ Read the node's credit reason in assertion detail, or its computed `nodeCredit`
78
+ entry in JSON. Compare the statement's execution evidence and selected test's
79
+ outcome. Keep these report fields out of the editable map.
80
+
81
+ A source node can be context only, lack a measured statement, have stale inputs,
82
+ or lack matching passing assertion and execution evidence. Shared setup and
83
+ background execution do not count as execution by every consuming test. Skipped
84
+ and TODO tests supply no passing witness. An unobserved site in a passing test
85
+ can indicate an untaken branch or missing measurement.
86
+
87
+ Do not borrow another assertion's identity or another test's execution. Preserve
88
+ zero-credit explanations when they accurately describe what the test checks.
89
+ Use `assertions report --view excludedStatements` to inspect erased TypeScript
90
+ imports. See [Investigating assertion evidence](assertion-evidence.md) for
91
+ asynchronous and missing-evidence cases.
92
+
93
+ For a large flow, page `assertion <id> --flow <flow-id> --view nodes --json` and
94
+ `--view edges` separately. `--compact` omits repeated source text from the
95
+ report; read the matching source separately and never save the compact objects
96
+ back into the map.
97
+
98
+ ## Assess changes before renewing flows
99
+
100
+ When inheriting a map, read every entry in `--view changes`. Recorded file
101
+ watches are a starting point; investigate whether other flows are affected too.
102
+
103
+ Add a `changeAssessments` entry for each managed change ID. Include all listed
104
+ `knownFlows` that still exist, plus any other affected flows. Explain why the
105
+ remaining claims are unaffected. An empty affected-flow list needs a reason.
106
+ Repair missing watches and update the affected graphs or selectors.
107
+
108
+ Save these edits, then validate:
109
+
110
+ ```sh supercov-example
111
+ npx supercov assertions validate --file <map-path> --json
112
+ npx supercov runs <run-id> assertions validate --view changes --json
113
+ npx supercov runs <run-id> assertions validate --view flows --json
97
114
  ```
98
115
 
99
- For large maps, page `validate --view changes`, `validate --view flows`, or
100
- `validate --view errors` with `--offset` and `--limit`; tokens appear under
101
- `items` and overall validity still covers the whole map. Pin `revision` across pages.
116
+ Page the validation views too. Overall validity covers the whole map even when
117
+ only one page is returned. Copy change `expectedBasis` tokens only after
118
+ examining the corresponding impact assessments, then save. Validate again and
119
+ copy flow tokens only for the claims you have examined.
102
120
 
103
- Validation checks shape, IDs, graph links, anchors and state binding. It returns
104
- `changes[].expectedBasis` and `flows[].expectedBasis`; null/stale tokens alone
105
- are unfinished work, not syntax errors. Copy change tokens **only after inspecting
106
- those impact assessments**, then save. Validate again and copy flow tokens only
107
- for claims you have examined. Save the map again. Change acknowledgements can
108
- invalidate additional flows, so obtain final flow tokens after change tokens.
109
- Changing a graph, selector, observation or watch after obtaining its token makes
110
- that token stale. Do not implement the hashing algorithm or manufacture tokens.
121
+ Obtain final flow tokens after saving change tokens: a change assessment can
122
+ invalidate additional flows. Treat tokens as opaque; do not implement their
123
+ hashing or manufacture them. Changing a claim after getting its token makes
124
+ that token stale.
111
125
 
112
- There is no `review` command. All commands are read-only; acknowledgement is your
113
- explicit file edit. Passing reference validation does not prove your reasoning.
126
+ ## Check and summarize
114
127
 
115
- ## Check and report
116
-
117
- ```sh
118
- supercov runs <run> assertions check --require-mappings --require-observed --json
119
- supercov runs <run> assertions report --view statements --limit 100 --json
120
- supercov runs <run>
128
+ ```sh supercov-example
129
+ npx supercov runs <run-id> assertions check --require-mappings --json
130
+ npx supercov runs <run-id> assertions report --view statements --limit 100 --json
131
+ npx supercov runs <run-id>
121
132
  ```
122
133
 
123
- Basic `check` permits untouched sites without flows, but fails on invalid or
124
- unfinished authored claims, open questions and outstanding change impact.
125
- `--require-mappings` requires at least one current observed explanation for every
126
- recognized passing site, including legitimate zero-credit explanations.
127
- `--require-observed` checks every site and selector. Neither means every semantic
128
- flow was found. Use `--min <percentage>` only for the project's chosen target.
129
- Do not lower a target or hide unknowns to make a check pass.
130
-
131
- The normal report includes the percentage immediately from the edited file.
132
- `notAssessed`, `pending`, `unavailable` and `notApplicable` have null percentages,
133
- not fabricated zeroes. A numeric partial-map result still reports counts of
134
- assertions without flows and claims needing work. Describe the score as
135
- agent-assessed statements; successful same-test coexecution is evidence, not
136
- proof of causality or mutation resistance.
137
-
138
- ## Continue after edits
139
-
140
- Run the same suite command again. The new run carries IDs, explanations and
141
- unchanged acknowledgements using prior file hashes, without needing old source.
142
- Use `assertions --needs-attention` and the change view. Repair only affected
143
- claims; keep current siblings. Any dependency byte edit, including comments,
144
- requires rechecking. Unique relocation preserves identity as a suggestion;
145
- ambiguous/removed sites remain in `retiredAssertions`. Old v1 flows are imported
146
- as drafts with questions about selectors and graph paths; inspect them explicitly.
147
-
148
- Do not edit `assertions.state.json`. Dirty generations persist across reruns and
149
- reverts until current tokens are recorded. Resolved change responses are folded
150
- and retired on the next publication. Runtime evidence never carries forward.
151
- If source differs from the pinned run, rerun tests before continuing.
152
-
153
- Finish with the pinned run ID, status, numeric percentage/counts when available,
154
- assertions without flows, remaining questions/stale or unobserved claims, and
155
- verification commands used. Explain remaining limits. The property in `observes`
156
- matters: changing a credited line while preserving that property can still pass.
134
+ Add `--require-observed` when every mapped site and test selector is expected to
135
+ have passing evidence. Report skipped, TODO or untaken cases when that gate is
136
+ unmet. Use `--min <percentage>` only for the user's chosen target; do not lower
137
+ it or hide gaps to make a check pass.
138
+
139
+ Report the run ID, map path, assertion percentage and counts, unresolved changes,
140
+ missing execution and useful next tests. Distinguish statements credited by the
141
+ map from raw line coverage and MC/DC. Mention which claims were newly examined
142
+ and which were reused.
143
+
144
+ A valid file does not prove the explanation or its completeness. A credited
145
+ statement can change without failing a test when the edit preserves the observed
146
+ property. Keep that distinction clear when recommending code changes.
@@ -1,9 +1,103 @@
1
- # Assertion evidence
2
-
3
- The earlier automatic JS/TS analyzer has been replaced by agent-authored
4
- assertion maps. Use [Understanding assertion coverage](assertions.md) to get
5
- started, [Agent-authored assertion maps](assertion-maps.md) for the file format
6
- and CLI reference, and [the agent instructions](assertion-agent.md) to build or
7
- update a map. Existing structural runs remain readable; mapping requires a new
8
- run with an assertion input manifest. Old `--pragmas`, `--analysis` and `--evidence`
9
- options do not apply to the map workflow.
1
+ # Investigating assertion evidence
2
+
3
+ Use assertion details when a statement runs but receives no assertion credit,
4
+ or when a mapped assertion has no passing occurrence. The report shows the
5
+ recorded explanation and the evidence supporting it separately.
6
+
7
+ ```sh supercov-example
8
+ npx supercov runs <run-id> assertions --needs-attention --limit 5
9
+ npx supercov runs <run-id> assertion <assertion-id>
10
+ ```
11
+
12
+ Replace the placeholders with the run and assertion IDs from your report.
13
+
14
+ ## Follow one assertion
15
+
16
+ The detail identifies the exact assertion in the test and the source nodes in
17
+ each flow. Each node has an explanation and a credit decision:
18
+
19
+ | Decision | Meaning |
20
+ | --- | --- |
21
+ | Credited | The map counts this statement, its flow is current, and the selected passing test supplies both assertion and execution evidence. |
22
+ | Not credited | The map claims this statement, but a reference, freshness or execution requirement is missing. Read the reason beside it. |
23
+ | Context only | The node helps explain the assertion; the map does not claim it as an asserted statement. |
24
+
25
+ The agent's explanation describes why a value or behavior reaches the assertion.
26
+ Supercov's credit reason describes what the recorded evidence supports. A credit
27
+ decision does not independently prove that explanation.
28
+
29
+ ## When a test has no source coverage
30
+
31
+ You may see a warning that a test made assertions but has no source-coverage
32
+ evidence. For example, a test might check a third-party parser, a constant, or
33
+ configuration without calling measured application code.
34
+
35
+ It can also mean application code ran in shared setup or across a process or
36
+ network boundary that was not attributed to the test. Missing evidence alone
37
+ cannot distinguish these cases. Inspect the test and the relevant statement:
38
+
39
+ ```sh supercov-example
40
+ npx supercov runs <run-id> assertions report --view tests --limit 20
41
+ npx supercov runs <run-id> assertions report --view statements --file src/shipping.js
42
+ ```
43
+
44
+ In JSON output, a statement's `executionEvidence` distinguishes any recorded
45
+ execution, passing tests and execution outside those tests. The tests view
46
+ includes outcomes and identifies setup scopes. Use that information to decide
47
+ whether to add a test, adjust the test setup or investigate measurement.
48
+
49
+ ## When a mapped assertion was not observed
50
+
51
+ An assertion can appear in source without being reached in this run. Common
52
+ causes include skipped or TODO tests and untaken branches inside passing tests.
53
+ An unsupported custom assertion form or missing attribution can also prevent
54
+ Supercov from recording the occurrence.
55
+
56
+ Read the selected test's outcome before changing the map. A recorded skipped
57
+ test and a test with no record are different cases. A passing sibling test does
58
+ not supply evidence for an assertion it never reached.
59
+
60
+ Use the stricter gate when you expect every mapped assertion and selected test
61
+ to have passing evidence:
62
+
63
+ ```sh supercov-example
64
+ npx supercov runs <run-id> assertions check --require-observed
65
+ ```
66
+
67
+ This gate can intentionally fail for a suite with TODO or skipped cases. Keep
68
+ those entries visible instead of deleting them to make the check pass.
69
+
70
+ ## Shared setup and asynchronous work
71
+
72
+ Code executed once in shared setup does not count as execution by every test
73
+ that later uses its result. Setup remains visible separately. If a statement
74
+ needs its own test evidence, arrange a test that executes it and checks its
75
+ behavior.
76
+
77
+ Per-test `t.after` assertions are included for Node's test runner. Supported HTTP
78
+ requests and test-owned WebSocket connections can carry a test's attribution
79
+ into callbacks. A WebSocket shared across tests may still lack enough information
80
+ to distinguish individual messages. Do not assign that work to a test merely
81
+ because it ran around the same time.
82
+
83
+ The regular report's **Runtime action phases** section is also separate from
84
+ assertion coverage. Zero lines recorded inside action phases does not mean the
85
+ agent-authored map credits zero statements.
86
+
87
+ ## Read a large flow
88
+
89
+ Page the nodes and edges of one flow when the complete graph is too large:
90
+
91
+ ```sh supercov-example
92
+ npx supercov runs <run-id> assertion <assertion-id> --flow <flow-id> --view nodes --limit 10 --compact
93
+ npx supercov runs <run-id> assertion <assertion-id> --flow <flow-id> --view edges --limit 10
94
+ npx supercov runs <run-id> source src/shipping.js --offset 0 --limit 20
95
+ ```
96
+
97
+ `--compact` leaves source locations and credit reasons visible while omitting
98
+ repeated source text. It only changes the displayed report. Follow the printed
99
+ next-page command, or `pagination.nextOffset` in JSON, until the view is complete.
100
+ Never copy compact report objects back into `assertions.json`.
101
+
102
+ See [Assertion map format](assertion-maps.md) for fields and validation rules,
103
+ and [Mapping assertions with an agent](assertion-agent.md) for the editing loop.