supercov 0.0.44 → 0.0.45

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/README.md +21 -12
  2. package/docs/agent-loop.md +12 -8
  3. package/docs/assertion-agent.md +156 -0
  4. package/docs/assertion-evidence.md +9 -694
  5. package/docs/assertion-maps.md +252 -0
  6. package/docs/assertions.md +82 -0
  7. package/docs/cli.md +23 -8
  8. package/docs/coverage-model.md +12 -0
  9. package/package.json +34 -35
  10. package/runtime/javascript/runtime.mjs +18 -33
  11. package/schemas/assertions.schema.json +276 -0
  12. package/analyzers/typescript/README.md +0 -59
  13. package/analyzers/typescript/bin/compiler-identity.mjs +0 -78
  14. package/analyzers/typescript/bin/identity.mjs +0 -71
  15. package/analyzers/typescript/bin/query.mjs +0 -29
  16. package/analyzers/typescript/dist/analyze.js +0 -5273
  17. package/analyzers/typescript/dist/archive.js +0 -337
  18. package/analyzers/typescript/dist/awaited-observations.js +0 -376
  19. package/analyzers/typescript/dist/build-identity.json +0 -1
  20. package/analyzers/typescript/dist/compiler.js +0 -32
  21. package/analyzers/typescript/dist/frontend.js +0 -75
  22. package/analyzers/typescript/dist/mock-counts.js +0 -2517
  23. package/analyzers/typescript/dist/native-frontend.js +0 -271
  24. package/analyzers/typescript/dist/pragmas.js +0 -186
  25. package/analyzers/typescript/dist/types.js +0 -1
  26. package/analyzers/typescript/package.json +0 -27
  27. package/analyzers/typescript/src/analyze.ts +0 -6180
  28. package/analyzers/typescript/src/archive.ts +0 -471
  29. package/analyzers/typescript/src/awaited-observations.ts +0 -561
  30. package/analyzers/typescript/src/compiler.ts +0 -49
  31. package/analyzers/typescript/src/frontend.ts +0 -136
  32. package/analyzers/typescript/src/mock-counts.ts +0 -3219
  33. package/analyzers/typescript/src/native-frontend.ts +0 -315
  34. package/analyzers/typescript/src/pragmas.ts +0 -284
  35. package/analyzers/typescript/src/types.ts +0 -45
  36. package/analyzers/typescript/tsconfig.json +0 -12
  37. package/docs/code-verification.md +0 -4
@@ -1,694 +1,9 @@
1
- # Assertion evidence (JS/TS)
2
-
3
- Supercov can analyze which source behaviors existing assertions appear to check,
4
- using an ordinary run archive, its matching source, and the existing statement
5
- and assertion-phase evidence. This work happens after tests. It adds no new
6
- test-time probes, does not run mutants, and does not rewrite your tests.
7
-
8
- This is candidate evidence, not a proof that arbitrary changes are safe.
9
- `assertionScore` remains null. Execution-only links, test gaps and analysis limits
10
- are different things; do not interpret a passing assertion nearby as protection.
11
-
12
- Repeated Node test registrations retain separate test and attempt identities,
13
- including same-name loop entries, nested `t.test` registrations and separate
14
- worker executions. This prevents one attempt from overwriting another; it does
15
- not prove which source-table row produced an ambiguous title. Such row guidance
16
- remains unresolved. Rerun suites to collect corrected evidence: already lost
17
- attempts cannot be recovered from an older archive.
18
-
19
- An unsupported operand in a passing assertion can leave its relationship to
20
- code covered by that test unknown. Such sites report an operand-shape limit,
21
- not proof that an assertion is absent. This does not give those sites assertion
22
- credit or hide known execution gaps.
23
-
24
- Native Node `assert(value)` and `assert.ok(value)` use the same recorded `ok`
25
- operation. Discovery resolves native import bindings, including renamed imports;
26
- an unrelated helper named `assert` does not acquire that identity. The optional
27
- failure message is not a truthiness-predicate operand and gives its returned value
28
- no assertion credit. Evaluating that message can still throw or have side effects:
29
- excluding its return value does not establish that removing the call is safe.
30
- Passing witnesses still require the exact source location and consistent passed
31
- outcomes; caught failed assertions and mixed outcomes do not supply them.
32
-
33
- For passing native `doesNotThrow` and `doesNotReject` calls, the optional second
34
- argument is not inspected after normal completion and receives no returned-value
35
- credit. Its evaluation can still have effects. `throws` and `rejects` are
36
- different: their second operand can supply an actual error matcher, and string
37
- overloads also have an ambiguity check. That operand's producer is not attributed
38
- as the operation whose exception the assertion catches.
39
-
40
- The ordinary first-operand analysis of these four native exception methods
41
- remains an explicit completion-analysis limit. A recognizable function name is not enough: the
42
- analyzer must separate callback production from invocation, synchronous throws
43
- from promise rejection, and normal completion from the returned/fulfilled value.
44
- The ordinary returned-value model supplies no credit for that operand, including
45
- when a pragma names its producer. This also leaves genuine exception checks
46
- unresolved until their invocation/completion path is established; it does not
47
- report them as absent assertions. The checked synchronous completion recipe
48
- below can resolve one precise omission question separately. Error-matcher
49
- operands retain their separate analysis. No additional execution probes or test
50
- failures are introduced.
51
-
52
- Every passed archived test remains in the analysis inventory, including tests
53
- whose custom or aliased registration cannot be linked to a source body.
54
- Those entries have no invented observations and carry
55
- `witnessIssues: [{ kind: "test-source-unlinked" }]`. The corresponding
56
- `diagnostics.unlinkedTests` identifies the runtime attempts; `linkedTests` counts
57
- only successfully linked bodies. Covered sites retain the source-linking limit
58
- instead of treating missing analysis as an absent assertion. Known execution
59
- gaps and independently supported assertions remain separate. An inline hint
60
- cannot bypass a missing owning source link.
61
-
62
- For some custom Node registrations, exact recorded passing assertion locations
63
- and native assertion import bindings identify a callback body. These partial
64
- links appear in `diagnostics.witnessedBodyLinks`, with the source assertions,
65
- and carry `test-registration-scope-unverified` in the test's witness issues.
66
- The body observations are candidates, not proof of the wrapper's behavior,
67
- captured loop values, failure propagation or all observers in the test. Pragma
68
- checking remains unresolved until that scope is supported. No title matching,
69
- extra test instrumentation or user declaration supplies the missing premise.
70
-
71
- Supported Node console-mock observations preserve the assertion's source projection:
72
- a call count, an argument, the whole history, or a slice or map. Their `mock` details
73
- retain the receiver and access path. Predicate strength applies to that projected
74
- value: equality of a count is not equality of the logged arguments. These records
75
- do not supply production-site credit or validate an observation hint until the
76
- mock lifetime, selected calls and source-site dependence are established. This
77
- remains separate from establishing that the count itself was checked.
78
-
79
- For a bounded synchronous subset, `mock.countEvidence` now models the exact mock
80
- creation, reset boundary, read location and participating calls. It distinguishes
81
- the installed method from saved mock references, restores a previous instance,
82
- and preserves copied counts and call-history snapshots across later calls or
83
- resets. Instance locations are scoped to their owning test record. `calls` are
84
- reconstructed from checked source, not a new runtime call trace. The modeled
85
- count must match an independent integer comparison with a passing witness.
86
-
87
- `source-checked` means accepted by the versioned `node-sync-console-count-v2`
88
- model, not formally verified application behavior. The model assumes unmodified
89
- native Node assertion/mock APIs, standard JavaScript built-ins (including array
90
- iteration) and the standard global console. It accepts empty console-mock
91
- replacements and a bounded synchronous source subset: constant bindings,
92
- fresh plain objects/arrays, returned closures, primitive-input production branches,
93
- own-property parameter binding, defaults, rest parameters and fresh-array spreads.
94
- An installed empty mock, or a saved reference to it, can also count calls carrying
95
- source-modeled plain objects, arrays or closures without inspecting their payload.
96
- Argument-evaluation calls still participate in the history in evaluation order.
97
- Fresh dense arrays support source-defined synchronous `map` callbacks, including
98
- their element, index and array parameters. Receiver and callback expressions are
99
- evaluated before callback invocations. Own source methods named `map` or `slice`
100
- are evaluated as those methods, not treated as native array operations. Sparse
101
- arrays, callback mutation, opaque callbacks and a `map` this-argument remain limits.
102
- Fresh arrays and native mock-history snapshots also support `slice` with bounded
103
- integer indices. History slices retain `historySelections`: each selection's
104
- source location, input count and effective half-open `[from, to)` range. Selected
105
- calls remain tied to the original instance and snapshot across later resets;
106
- index-argument effects occur after the snapshot is taken. An empty selection is
107
- not evidence that the original mock had no calls. Coercing indices remain limits.
108
- This does not pin payload fields or grant production-value credit. Unmocked
109
- console calls with nonprimitive arguments remain unsupported because formatting
110
- can invoke user code; getter-bearing or otherwise opaque payloads remain limits.
111
- Module factories must be source-checked as pure: setup side effects are never
112
- replayed under the test's mock. Shared module objects remain unresolved because
113
- `const` does not rule out mutation by earlier callers. Opaque calls, escaped
114
- mocks, getters, missing own properties, mutable bindings, test branching,
115
- unsupported initialization and async suspension leave explicit reasons. Evidence
116
- inside a rejected assertion witness is not a passing observation.
117
-
118
- For a closed top-level `for (const { ... } of cases)` table using native
119
- `node:test`, `countEvidence.rowBinding` can retain the selected source row and
120
- its scalar bindings. The table must be a private literal with no aliases or
121
- other reads/writes. Supercov reproduces each registration title from the source
122
- values and requires a unique exact match to the archived title; it does not
123
- guess from title prefixes or record order. The existing original-source passing
124
- assertion witness is still required. Duplicate titles, transformed titles,
125
- mutable/escaping tables, custom registration helpers and unsupported loops stay
126
- unresolved. A source-checked row binding alone grants no count or value credit;
127
- for example, the producer's shared-object history may still be unknown.
128
-
129
- Count evidence does not grant general site, payload or pragma credit, even for
130
- listed calls: an aggregate number does not pin each call's arguments or rule out
131
- compensating changes. Such value relationships remain operand-shape limits.
132
- This analysis is post-run and requires no new probes, pragmas or contract files.
133
-
134
- Native Node equality observations also retain both operand source locations and
135
- their comparison relation. Comparing a `const` value with itself, or with an
136
- immutable alias, checks no property of that returned value. Such a comparison
137
- cannot supply value credit or validate a hint; an independent assertion still
138
- can. Node's strict equality uses `Object.is`, so even `NaN` compared with itself
139
- passes. Repeated calls, property reads, mutable aliases and `await` are not
140
- assumed to be identical evaluations. This bounded source check needs no pragma,
141
- contract file or additional runtime probe; it is not general relational analysis.
142
-
143
- For native equality comparisons, bounded source analysis also follows `const`
144
- aliases and `await`. If both operands lead to the same stable input through an
145
- await, `comparison.relation` is `shared-input-through-await`; each operand's
146
- `input` records the binding and await source locations. This does **not** mean
147
- the resulting values are equal or the predicate is a tautology: a Promise differs
148
- from its resolved value, and a stateful thenable can resolve differently on two
149
- awaits. The passing witness remains visible, but this comparison alone cannot
150
- supply producer-value, absence, control-flow or pragma credit. A relevant site
151
- reports `limit:predicate-dependence` unless a more specific existing limitation
152
- or known execution gap applies. Independent assertions still contribute normally.
153
- The traversal is bounded, does not equate calls/getters or follow mutable aliases,
154
- and does not infer independence when no shared input was found. It runs only
155
- during querying; the test-time instrumentation is unchanged.
156
-
157
- For a narrow direct-call shape, `decision.primitive` retains literal branch
158
- results and the witnessed native strict equality/inequality predicates. The
159
- `js-primitive-decision-v1` model checks whether forcing either outcome would
160
- still satisfy each modeled assertion. Both branches returning the same value,
161
- or different values both accepted by `notEqual`, do not establish decision
162
- sensitivity. Numeric values retain signed zero and remain distinct from strings.
163
- The model requires synchronous, literal-only branches and direct boolean-input
164
- calls in simple Node tests; it rejects transformations, effects and mutable
165
- function bindings. It assumes normal module loading and unmodified native
166
- assertion semantics. Unsupported shapes retain the existing candidate analysis,
167
- not a sensitivity proof. These query-time checks add no probes, and their
168
- bounded results remain unverified candidates, not a global assertion score.
169
- Within an accepted primitive model, `stuckTrueCaught` and `stuckFalseCaught`
170
- describe rejection of those two forced outcomes only. A rejected or incomplete
171
- model omits these fields: unknown is not a known non-rejection. Outside this
172
- model, the same fields are legacy branch-observation heuristics, not checked
173
- counterfactuals. Neither form predicts arbitrary edits such as changing a regex
174
- anchor or a comparison boundary merely because the containing site is `evident`.
175
-
176
- Decision candidates expose `sensitivityBasis` in JSON and as `sensitivity:` in
177
- text output:
178
-
179
- - `bounded-source-model`: the supported source model evaluated both forced
180
- outcomes against the witnessed predicates, within its stated assumptions.
181
- - `branch-observation-heuristic`: the flags come from observed branch evidence,
182
- not evaluation of the alternative behavior. Do not use them as verified
183
- change-detection results.
184
- - `unavailable`: forced-outcome results are not available. Missing flags are not
185
- `false` and must not be counted as known non-rejections.
186
-
187
- Effect candidates omit this decision-only field. These labels describe the
188
- analysis method, not accuracy percentages or global safety; `analysisCertainty`
189
- stays `unverified`. Evidence pagination preserves the same labels.
190
-
191
- Child-exit observations retain `processExit` source evidence instead of assuming
192
- that every Promise containing an exit listener checks the covered process. The
193
- bounded `node-child-exit-source-v1` model follows const projections and simple
194
- local/imported function returns by declaration identity. It records helper call
195
- sites, the native spawn call, Promise, exit/close listener and selected field.
196
- An accepted `resolution` describes which event argument the callback passes to
197
- the resolver, directly or in a fresh plain object's field (`code` or `signal`).
198
- This is a source-checked resolver mapping under unmodified native Node APIs and
199
- global Promise semantics, not a checked end-to-end assertion link. Calls from
200
- project-defined Promise constructors, transformed fields, competing settlements,
201
- non-child emitters and unsupported shapes retain explicit reasons.
202
-
203
- The optional `consumer` check follows the selected read's local const bindings
204
- and checks their other uses, including nested closures. It can establish that
205
- the fresh resolver field reaches this read unchanged under the model's
206
- unmodified built-in/prototype assumptions. Aliases that escape, extra Promise
207
- consumers, mutations, reflective access and nonlocal result carriers remain
208
- unresolved. Simple discarded awaits (including native `Promise.race`) and
209
- independent arrow readers on a fresh helper return are supported. The traversal
210
- is bounded and happens after the test run. A source-checked consumer is not a
211
- claim that the assertion distinguishes every change, or that the event came
212
- from the covered producer.
213
-
214
- In particular, a different child can run the same source. `processExit.status` remains
215
- `unresolved`; even a source-checked `resolution` supplies no production-value,
216
- absence, control-flow or pragma credit. Relevant candidates report
217
- `limit:process-exit-link` unless a more specific existing limitation or known
218
- execution gap applies. Assertion witnesses remain visible. A bare `exit` channel
219
- without these source details is not an escape hatch for credit. This source
220
- analysis adds no test-time probes and makes no claim about complete pipe capture.
221
-
222
- Fallback native assertion locations prefixed `runtime-stack:` identify runtime
223
- stack coordinates, which a compiler or loader may have shifted. They are not
224
- original-source witnesses and cannot select a static test or validate a hint.
225
- Lexical assertion markers retain their exact original-source locations.
226
-
227
- For statically identified native Node assertions with awaited arguments, Supercov
228
- binds that source location to the call before evaluating the arguments. It
229
- records the phase only when the assertion is invoked: a rejected operand creates
230
- no assertion witness, and a caught assertion failure remains a failed witness.
231
- Earlier async work is not attributed to that invocation phase. This preserves
232
- the original `await` expressions without relying on a shared current-statement
233
- marker. Optional calls, generators, matcher calls with awaited operands, constructed
234
- `Assert` instances and awaited arguments to `rejects`/`doesNotReject` still use
235
- qualified runtime-stack fallback where the runner supports it. Exact source
236
- identity alone does not establish an operand's relationship to production code.
237
-
238
- ## Requirements
239
-
240
- Use the **npm Supercov launcher**, which supplies the installed analyzer assets.
241
- The current JS/TS adapter is exercised with Node's test runner and Vitest.
242
- Browser, background, merged, retried and ambiguously attributed records have
243
- explicit limitations; other language assertion analyzers are not enabled here.
244
-
245
- Assertion analysis requires a source file for every accepted passing test attempt.
246
- If the runner or a custom stack formatter prevents source attribution, the query
247
- fails rather than treating omitted tests as execution gaps. Restore attribution
248
- and recapture the suite; ordinary coverage queries remain available.
249
-
250
- The analyzed project must provide a TypeScript compiler API, **even for a
251
- JavaScript project**. For example, add TypeScript as a development dependency
252
- before recording the run. The analyzer uses that project's compiler, not a
253
- silently substituted global or bundled version. Installing it afterward changes
254
- the dependency fingerprint, so rerun the suite. Missing/incompatible compiler APIs
255
- produce an error. **TypeScript 5.8.3 and native 7.0.2 are tested.** Version 7.0.2
256
- uses its own native parser/checker, not a fallback to TypeScript 5. Install its
257
- platform-specific optional dependency too; the report hashes both the JS client
258
- and the native compiler package, including its standard libraries. This backend
259
- requires Node 22.12 or newer (Node 24 tested), and uses original-source evidence
260
- from ordinary Supercov archives, not legacy ts-node/V8 generated-line coverage.
261
- Native module resolution currently follows actual import/export references;
262
- unresolved helper-only specifiers remain visible as compiler limitations.
263
- Other native compiler versions are not enabled until separately calibrated.
264
- Do not downgrade an application's compiler just to improve an assertion report.
265
-
266
- Compiler compatibility and regression checks do not prove general correctness.
267
- All candidates remain unverified and `assertionScore` remains null. Query-side
268
- native compiler work adds no test-time instrumentation.
269
-
270
- Standalone native/Python/Ruby distributions do not currently bundle this JS/TS
271
- analyzer. Use the npm launcher, or explicitly set `SUPERCOV_PACKAGE_ROOT` to an
272
- installed npm package directory. Normal coverage commands are unaffected.
273
-
274
- ## Run, inspect, follow evidence
275
-
276
- `assertions` analyzes source behaviors and their assertion evidence; it does not
277
- just count or list assertion calls. Use `npx supercov docs assertion-evidence`
278
- to read this guide from the installed package.
279
-
280
- ```sh
281
- npx supercov -- npm test
282
- npx supercov runs latest assertions --limit 5
283
- npx supercov runs latest assertions --file src/core.ts --json
284
- npx supercov runs latest assertions --site '<site-id>' --json
285
- ```
286
-
287
- The summary reports candidate counts and the site denominator, without claiming
288
- a verified assertion percentage. Each ordinary site row contains its source,
289
- candidate classification, available facts, and an `evidence.pointer` for full
290
- details. Global tests, attempts, execution links, diagnostics and source-scope
291
- limits are referenced under `evidence`, not repeated inside every site page.
292
-
293
- ```sh
294
- npx supercov runs latest assertions --evidence /tests --limit 5 --json
295
- npx supercov runs latest assertions --evidence /diagnostics --json
296
- npx supercov runs latest assertions --evidence /sites/0/facts --json
297
- ```
298
-
299
- Pointers use JSON Pointer syntax: escape `/` in a property name as `~1` and `~`
300
- as `~0`. Use an empty pointer (`--evidence ''`) for the document root. Follow
301
- returned pointers rather than constructing indices from filtered page offsets:
302
- site pointers index the complete, stable inventory.
303
-
304
- ## Pagination and oversized records
305
-
306
- All JSON responses stay within the normal response budget. Pages may contain
307
- fewer than `--limit` results: **follow `pagination.nextOffset`**, not offset plus
308
- the requested limit. `hasMore: false` marks the end.
309
-
310
- An oversized site/hint is returned as `detailOnly: true` with an
311
- `evidence.pointer`. Nothing has been discarded. An evidence page returns immediate
312
- object members or array entries as `items`. Small entries contain `value`; large
313
- ones contain `detailOnly: true` and another `pointer` to inspect. String leaves
314
- return `text` chunks, with offsets/counts measured in Unicode scalar values,
315
- not bytes. This allows reading one large observation or diagnostic completely.
316
-
317
- ```sh
318
- npx supercov runs '<run-id>' assertions --evidence '<returned-pointer>' \
319
- --offset 0 --limit 4000 --analysis '<analysisId>' --json
320
- ```
321
-
322
- Every page carries an `analysisId` hashing the complete derived document and its
323
- provenance. Pass the first page's id as `--analysis` on follow-ups to reject mixed
324
- analyses if the compiler, analyzer, or derived results change. Use a fixed run id
325
- instead of `latest` while paging. Source/run freshness checks still run on every
326
- query. Derived results are not cached or written into the archive.
327
-
328
- JSON reports use `reportSchema: 2`. Shared evidence is accessed through the
329
- evidence pointers above.
330
- Success and error JSON envelopes identify this query as `coverage.assertions`.
331
-
332
- ## Optional assertion hints
333
-
334
- ```ts
335
- // observes: src/core.ts#compute return value
336
- assert.equal(compute(), 4);
337
- ```
338
-
339
- The comment must precede a statement containing exactly one recognized assertion
340
- or a supported directly awaited observation helper.
341
- The project-relative file, exact function/owner name, and optional literal source
342
- substring must select one inventory site. Optional `via ...` is explanation only.
343
-
344
- ```sh
345
- npx supercov runs latest assertions --pragmas --json
346
- ```
347
-
348
- Origin is `user-suggested`; validation is separately `analyzer-supported`,
349
- `unresolved`, or `invalid`. Support requires the named assertion's exact passing
350
- witness and a connection supported by the ordinary effect rules. Strength remains
351
- presence/value/total; presence does not mean value protection. Unsupported
352
- decision, absence and internal-state paths remain unresolved. A missing inventory
353
- match may reflect unsupported source rather than a bad declaration.
354
-
355
- Hints cannot inject observations, borrow another assertion's evidence, inflate
356
- coverage or remove sites from the denominator. Editing comments changes test
357
- source and requires a new run. Supported hints retain the analyzer's limitations;
358
- they are not formal proofs.
359
-
360
- ### A checked missing-call question
361
-
362
- ```ts
363
- // observes: src/logger.ts#stdout console.log; check missing call
364
- assert.equal(log.mock.callCount(), 1);
365
- ```
366
-
367
- The optional `; check missing call` suffix selects a bounded source-analysis
368
- recipe. It targets a console emission, excluding a containing callback-return
369
- inventory entry. Existing `via` text remains explanatory, not executable guidance.
370
- The question is precise: replace the selected expression-bodied arrow callback
371
- with `() => undefined`. This removes its body on **every invocation**, not just
372
- one recorded call. It says nothing about changing argument values or arbitrary
373
- edits in the enclosing function.
374
-
375
- The `node-first-test-call-omission-v1` model checks a closed test-file prefix and
376
- the first synchronous native Node test (or first row of a checked registration
377
- table). Only native test/assert imports and the selected production module may
378
- initialize that test scope. The production module's initialization must be pure
379
- under the existing bounded source interpreter. The supported source prefix has
380
- no earlier registrations, hooks, async suspension, opaque operations or mutations.
381
- This is what allows the model to read the module's initial shared objects; a
382
- comment never disables the ordinary shared-history guard for other analysis.
383
-
384
- The model interprets the supported prefix twice after the run, with and without
385
- that callback body, through the selected independent integer count assertion.
386
- The original modeled count must match the expectation and the assertion must
387
- have its own exact passing witness. The named call must participate in the
388
- selected history. Wrong histories, resets that erase the evidence, later rows,
389
- unsupported source and absent witnesses stay unresolved. Source-driven helper,
390
- receiver and variable names are not fixed by this recipe.
391
-
392
- `--pragmas` returns `hint.callOmission` with the source locations, mock instance,
393
- original/omitted/expected counts and an outcome of `rejected` or `not-rejected`.
394
- A sliced history can hide an omitted call when another call fills its slot;
395
- `not-rejected` concerns **this assertion only**, never whole-suite survival.
396
- Support uses `validation: analyzer-supported`, with no site `strength` and no
397
- change to ordinary candidate verdicts or the global assertion score.
398
-
399
- These conclusions depend on normal isolated Node test-file loading and serial
400
- registration execution, the matching compiler, unmodified native assertion/mock
401
- APIs and built-ins, and finite ordinary execution. The source checker does not
402
- prove those environmental assumptions or implement arbitrary JavaScript semantics.
403
- Do not apply its result to custom preloads, non-isolated module histories or
404
- different scheduling contracts. The existing source-freshness checks still apply.
405
- No mutation is executed during querying and no new test-time probe is installed.
406
- Invalid or unsupported hints only affect analysis; they never fail a test.
407
-
408
- ### A checked count/routing question
409
-
410
- ```ts
411
- // observes: src/logger.ts#selectLogger mode === 'quiet'; check count
412
- assert.equal(log.mock.callCount(), 1);
413
- ```
414
-
415
- `; check count` selects a decision or a return containing a conditional expression.
416
- The bounded `node-closed-count-sensitivity-v1` model checks three explicit changes
417
- to the selected primitive equality: replace it with true, replace it with false,
418
- or invert it. Each change applies on every evaluation of that expression in the
419
- modeled prefix. This is not arbitrary-change or whole-site protection.
420
-
421
- Unlike the first-test omission recipe, this rule can handle later source-checked
422
- registration rows. It checks a closed synchronous test module, the sole imported
423
- production factory, source closure origins, nonescaping receivers and enumerated
424
- shared object allocations. Only those allocations receive permission; captured
425
- arrays/objects do not inherit a blanket shared-state exemption. Setup effects,
426
- hooks, native aliases, dynamic escape and unsupported calls remain unresolved.
427
-
428
- For call counts, `util.inspect` on bounded fresh plain data can be represented as
429
- an opaque string. Native TTY state remains opaque too. The model does not guess
430
- formatted bytes, and an opaque comparison or unsupported earlier assertion stops
431
- the selected variant. Earlier rejecting assertions are not credited to a later
432
- count assertion.
433
-
434
- The owning count assertion must have an exact passing witness and an independent
435
- integer expectation matching its modeled original count. `hint.countSensitivity`
436
- reports the checked source locations, allocations, mock instance, original and
437
- expected counts, and separate `condition-true`, `condition-false` and
438
- `condition-inverted` variants. Every variant has its own status and, when checked,
439
- count and `rejected`/`not-rejected` outcome. Partial results retain unknown variants.
440
-
441
- `validation: analyzer-supported` applies to these bounded results only. It never
442
- promotes ordinary observations, site strength or a global assertion score.
443
- `not-rejected` concerns the selected assertion, not the whole suite. The native
444
- API/compiler/pristine-environment and isolated-loading assumptions above still
445
- apply; this source checker is not a formally verified JavaScript semantics.
446
- Hints are inert comments. All additional work occurs during querying, with no
447
- new test-time probes or external user-maintained contract files.
448
-
449
- ### A checked selected-argument question
450
-
451
- ```ts
452
- // observes: src/logger.ts#formatData args.map; check value
453
- assert.equal(infoCalls[0].arguments[1], "hello");
454
-
455
- // observes: src/logger.ts#selectLogger mode === 'verbose'; check value
456
- assert.deepEqual(infoCalls[0].arguments[2], { a: 1 });
457
-
458
- // observes: src/logger.ts#formatData typeof arg === 'object'; check value
459
- assert.match(String(infoCalls[0].arguments[2]), /a: 1/);
460
- ```
461
-
462
- `; check value` uses the same closed synchronous scope, but follows a selected
463
- mock call and argument to its existing native strict equality, deep-strict
464
- equality or restricted literal-regexp assertion. `hint.payloadSensitivity` retains the mock instance, ordered
465
- call index, emission source, history slices, argument index, original value,
466
- independent literal expectation and per-change outcome. A call index is relative
467
- to the selected history; recorded slices describe how that history was obtained.
468
-
469
- For a primitive equality target (including `typeof` a bound identifier), the changes are force true, force false and
470
- invert. For a native `map` call with an inline block-bodied arrow callback, the
471
- single `map-callback-empty` change empties that callback body on every invocation;
472
- its parameters and the surrounding call still evaluate. Native array origin is
473
- checked, not inferred from a method name. This does not model arbitrary edits.
474
-
475
- The current value domain contains primitives, fresh plain objects/dense arrays,
476
- and abstract strings returned by the restricted native `util.inspect` summary.
477
- An opaque string can be distinguished from an expected object without guessing
478
- formatted bytes. For inputs of at most 64 ASCII letters, digits, spaces, underscores
479
- or hyphens, `quoted-string` records only that native inspection includes a single
480
- quote. It can reject an apostrophe-free expected string without predicting colors
481
- or complete output. Two abstract strings cannot be assumed equal.
482
-
483
- The only supported coercion is unshadowed native `String` on a string or fresh
484
- plain object without custom conversion. Evidence records the input and coercion
485
- rule; default plain-object conversion produces `[object Object]`. Native
486
- `assert.match` accepts direct regex literals containing 1–80 ASCII letters,
487
- digits, spaces, underscores, colons or hyphens, with no flags. Their predicate is
488
- exact substring inclusion. Other regexes, custom conversions, arbitrary transforms,
489
- object identity comparisons and nonliteral expected operands remain limits.
490
-
491
- In `node-closed-payload-sensitivity-v2`, a fully derived original path with an
492
- opaque selected string can use its exact existing passing assertion witness:
493
- the original outcome is explicitly `witnessed-pass`, not source-calculated
494
- `not-rejected`. This never overrides a known contradiction, supplies an earlier
495
- unsupported assertion, or lends the original pass to any changed path. Every
496
- resolved variant still needs a source-calculated predicate result; unknown
497
- variants remain unresolved. Earlier rejecting or unsupported assertions stop a
498
- variant and cannot provide credit to a later assertion.
499
-
500
- The bounded interpreter can derive a history-alias relationship that the older
501
- boundary analyzer did not recognize. Such a result does not require a legacy
502
- boundary observation, but still requires native assertion identity, a derived
503
- original path and the exact owning passing assertion witness.
504
- The engine checks value/outcome consistency and variant scope; the versioned
505
- source analyzer and documented native-environment assumptions remain trusted.
506
- This is not a formally verified JavaScript implementation or a general assertion
507
- score. `not-rejected` concerns only the selected predicate, not the suite.
508
- The native inspection summary is calibrated on supported Node versions, not
509
- proven for every future version: Node documents inspection output as changeable,
510
- and these archives do not currently pin the native runtime version. That remains
511
- an explicit compatibility assumption, not an independently checked guarantee.
512
-
513
- No comments create observations or promote ordinary site strength. No application
514
- execution, mutations, runtime value sampling or additional probes occur during
515
- the query. Invalid hints affect analysis only; they cannot fail native tests.
516
-
517
- ### Checked direct return values
518
-
519
- The same `; check value` recipe also supports direct source-call results:
520
-
521
- ```ts
522
- // observes: src/options.ts#select return false; check value
523
- assert.equal(select({ items: undefined }), false);
524
-
525
- // observes: src/options.ts#select items.length === 0; check value
526
- assert.equal(select({ items: [] }), "*");
527
- ```
528
-
529
- `hint.directReturnSensitivity` uses `node-first-test-direct-return-v1`, separate
530
- from mock argument/history evidence. It checks the first synchronous native Node
531
- test prefix, a production module containing only function declarations or
532
- constant arrow functions, and a direct call or immutable local aliases leading
533
- to the selected strict/deep-strict equality. The expected operand must be an
534
- independent literal. The original predicate must agree with the source-calculated
535
- value and have its own exact passing assertion witness.
536
-
537
- The target must be evaluated inside that selected call. An earlier call reaching
538
- the same source cannot lend it a relationship. Unsupported earlier operations,
539
- getters, missing own properties, transformed/discarded results, setup and imported
540
- initializers remain limits. The model may finish an earlier prefix without
541
- evaluating operations in unexecuted bodies; it does not assume those operations
542
- are pure. Native APIs, pristine builtins, the matching compiler and isolated,
543
- serial Node test loading remain explicit model assumptions.
544
-
545
- For a strict equality expression, the questions are force true, force false and
546
- invert. For a Boolean literal, the question is `boolean-literal-inverted`, not
547
- an MC/DC decision. Each result carries its call source, target evaluation count,
548
- actual and expected values, and `rejected` or `not-rejected` predicate outcome.
549
- Earlier rejecting assertions stop a variant and are not attributed to the selected
550
- later assertion. Unknown variants remain visible even when another is supported.
551
-
552
- Rust validates the evidence scope, source locations, owning witness and value/
553
- outcome consistency. The source analyzer remains trusted; this is not a formally
554
- verified JavaScript implementation. Support never changes ordinary candidate
555
- strength, marks an MC/DC outcome executed, or supplies a global assertion score.
556
- `not-rejected` is local to the selected predicate, not whole-suite survival.
557
- All new work occurs after the run; hints remain inert comments.
558
-
559
- ### Checked synchronous exception completion
560
-
561
- ```ts
562
- // observes: src/validate.ts#validate throw Error('invalid'); check completion
563
- assert.throws(() => validate(badInput));
564
-
565
- // observes: src/operation.ts#operation return 101; check completion
566
- assert.doesNotThrow(operation);
567
- ```
568
-
569
- `; check completion` asks whether replacing the selected complete `throw` or
570
- `return` statement with an empty statement (`;`) changes this assertion's
571
- outcome. The empty statement preserves a single-statement `if` body; it is not
572
- arbitrary token deletion, function removal, a changed error message or an MC/DC
573
- condition force. The hint must select exactly one source statement and attach
574
- to an existing assertion with its own passing witness.
575
-
576
- `hint.completionSensitivity` uses `node-first-test-completion-v1`. It shares the
577
- bounded source evaluator and declaration-only production-module/first
578
- synchronous native Node test-prefix scope with direct-return checks. It accepts
579
- one-argument native `throws` and `doesNotThrow`, plus the missing-exception
580
- shortcut for `throws` with a matcher and optional message described below.
581
- Asynchronous settlement is not modeled. Source calls, immutable local aliases,
582
- source-created callback factories and inline test callbacks follow actual closure
583
- environments. Operand evaluation happens before the assertion's catch, while
584
- callback invocation happens inside it. The evaluator follows source `try`,
585
- `catch`, rethrow and `finally`; an evaluator limitation is not a JavaScript error
586
- that a catch or finally return can conceal.
587
-
588
- Unshadowed `Error` construction accepts at most one primitive message and no
589
- options. The model retains its primitive-derived message for native failure
590
- diagnostics, not general properties, stack or matcher semantics. Unsupported
591
- operations on an original or changed path remain unresolved. A changed path that
592
- no longer supplies a source callback also remains unresolved; no callable is
593
- invented to finish the proof.
594
- Native assertion semantics, pristine builtins and isolated serial Node loading
595
- remain explicit model assumptions. This is not a formally verified JavaScript
596
- implementation, a general proof for every runtime, or an independently checked
597
- certificate of the entire source interpreter.
598
-
599
- An escaping exception alone does not prove that `doesNotThrow` reports failure:
600
- Node reads and formats the thrown value's `message` first. That access or
601
- conversion can run application code, including code that exits successfully.
602
- The model therefore requires a safe diagnostic path: a native Error's derived
603
- string message, a fresh own primitive data property, or an absent message on an
604
- accepted value under the pristine-prototype assumption. Nonprimitive messages
605
- and unsupported accessors remain analysis limitations even if a native control
606
- happens to fail. `throws` without a matcher does not inspect that message and
607
- does not require this additional evidence.
608
-
609
- The original and omitted paths each record callback location, `normal` or
610
- `throw` completion, the escaping throw's source where applicable, target visit
611
- count, and a scoped outcome (`rejected` or `not-rejected` for single-argument
612
- forms). The original must agree with its passing witness. A rejecting
613
- `doesNotThrow` path also records a `diagnostic` descriptor
614
- with its basis and primitive message. Rust rejects missing or inconsistent
615
- descriptors, including older unconditional throw-to-rejection claims. Rust
616
- validates scope, source identities and completion/predicate consistency. A caught
617
- inner throw can therefore receive a checked `not-rejected` answer even when
618
- another throw in the same function supplies the assertion's
619
- witness. A no-error assertion can ignore a normal return value, yet reject an
620
- omission that exposes a later throw.
621
-
622
- These are exact, scoped answers, not ordinary site-strength promotions.
623
- `not-rejected` does not mean the entire suite would pass. The comment cannot
624
- supply a missing assertion, an unexecuted MC/DC pair or a missing witness, and
625
- cannot make native tests fail. All added work is post-run analysis; there are no
626
- new test-time probes or executions of application source during the query.
627
-
628
- #### Error matchers: a checked missing-exception shortcut
629
-
630
- ```ts
631
- // observes: src/validate.ts#validate throw Error('invalid'); check completion
632
- assert.throws(() => validate(badInput), /invalid/);
633
- ```
634
-
635
- If omitting the selected statement makes the callback complete normally, native
636
- `throws` does not execute its error-matching logic. The analyzer can establish
637
- rejection without interpreting an arbitrary regex or predicate body. It first
638
- evaluates all source arguments before invoking the callback, just as JavaScript
639
- does; a throwing matcher factory is not a callback exception or a passing witness.
640
-
641
- The original `CompletionCheck` records `outcome: "witnessed-pass"`, its matcher
642
- kind and argument source. This uses only the selected assertion's own archived
643
- pass, not a claim that matching was simulated. The changed check must have
644
- `completion: "normal"`, `outcome: "rejected"` and a
645
- `missingExceptionDiagnostic`. A changed callback that still throws remains
646
- unresolved; neither its matching result nor its survival is borrowed from the
647
- original witness. Earlier matcher assertions cannot borrow that witness either.
648
-
649
- Skipping the matcher is not enough by itself. Node still reads the matcher's
650
- `name` and formats a supplied message while reporting the missing exception.
651
- The descriptor records a safe name basis and primitive message. Supported bases
652
- include fresh regex/array values without a name, fresh source functions whose
653
- names are strings (without inventing their inferred spelling), pristine native
654
- Error constructors/instances, and fresh objects with absent or own primitive
655
- names. Explicit null/undefined matchers and the string message overload are also
656
- handled. Arbitrary name access, name/message conversion hooks and unsupported
657
- argument evaluation remain limits. Regex literals are represented without running
658
- them; the query does not call user matchers or conversion hooks.
659
-
660
- Rust requires matching argument source locations, the original passing witness,
661
- the changed normal completion, and consistent diagnostic metadata. This is
662
- still a bounded trusted source model, not a general interpreter certificate,
663
- whole-suite survival result or automatic site-credit promotion.
664
-
665
- ### Awaited child-process capture helpers
666
-
667
- For a bounded Node `test` pattern, a hint can attach to an awaited helper that
668
- polls a regex over append-only UTF-8 child-process output and throws on exit or
669
- timeout. Supercov checks the actual imports, bindings, capture callbacks,
670
- predicate and polling loop—not a method name such as `ready`. The receiver must
671
- be a local `const`, with one factory launch immediately followed by the awaited
672
- call. Escaped receivers, transformed buffers, stateful regexes, caught polling
673
- failures and unsupported control flow do not satisfy this source model.
674
-
675
- `--pragmas` reports recognized source structure in `hint.awaitedObservation`,
676
- including the factory, capture and predicate locations and the regex literal.
677
- This is **source support, not a recorded read**. The result remains `unresolved`
678
- with reason `observation-capture-unavailable` and `witness: unavailable`: current
679
- archives do not supply a supported read receipt for this model. A passing test
680
- or similarly named assertion phase cannot substitute for that receipt.
681
-
682
- This query-only analysis adds no probes, changes no test result and supplies no
683
- presence, value or whole-site assertion credit. A bad or unsupported pragma never
684
- adds a test failure. Native API behavior and no external monkey-patching remain
685
- model assumptions; source recognition does not prove end-to-end pipe provenance.
686
-
687
- ## Agent workflow
688
-
689
- Read coverage first, then inspect assertion candidates and their evidence before
690
- writing another test. An execution gap may need a reachable scenario; an assertion
691
- gap may need a stronger check. An analysis limit needs investigation—not a test
692
- written just to satisfy the analyzer. Keep the original denominator and unresolved
693
- work visible. Never automatically treat an `evident` candidate as permission to
694
- change application behavior.
1
+ # Assertion evidence
2
+
3
+ The earlier automatic JS/TS analyzer has been replaced by agent-authored
4
+ assertion maps. Use [Understanding assertion coverage](assertions.md) to get
5
+ started, [Agent-authored assertion maps](assertion-maps.md) for the file format
6
+ and CLI reference, and [the agent instructions](assertion-agent.md) to build or
7
+ update a map. Existing structural runs remain readable; mapping requires a new
8
+ run with an assertion input manifest. Old `--pragmas`, `--analysis` and `--evidence`
9
+ options do not apply to the map workflow.