supercov 0.0.49 → 0.0.51
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -0
- package/docs/assertion-agent.md +9 -5
- package/docs/assertion-maps.md +38 -11
- package/docs/cli.md +22 -0
- package/docs/getting-started.md +14 -0
- package/package.json +12 -11
- package/schemas/assertions.schema.json +2 -2
package/README.md
CHANGED
|
@@ -19,6 +19,10 @@ to add a test in your own project, with a recorded example to follow along.
|
|
|
19
19
|
npx supercov -- npm test
|
|
20
20
|
```
|
|
21
21
|
|
|
22
|
+
Other ways in, all the same binary at the same version: `brew install
|
|
23
|
+
supercorp-ai/tap/supercov`, or for a Go project `go run
|
|
24
|
+
github.com/supercorp-ai/supercov/cmd/supercov@latest`.
|
|
25
|
+
|
|
22
26
|
Everything after `--` is your test command. Supercov runs it without changing your source, tests, runner configuration, or normal build output.
|
|
23
27
|
|
|
24
28
|
Then ask what is still uncovered:
|
package/docs/assertion-agent.md
CHANGED
|
@@ -59,9 +59,10 @@ For each assertion:
|
|
|
59
59
|
- Put a node in `countsAsAsserted` only when its behavior is checked by this
|
|
60
60
|
assertion. Every counted node needs a recorded path to `$assertion`.
|
|
61
61
|
Counting a guard or block does not count the nested body automatically.
|
|
62
|
-
- Add helper, configuration and other dependency files to `watch`.
|
|
63
|
-
|
|
64
|
-
and
|
|
62
|
+
- Add helper, configuration and other dependency files to `watch`. The
|
|
63
|
+
declarations holding your nodes, the top level of their files, the assertion
|
|
64
|
+
file and the selected test files are already dependencies. Include relevant
|
|
65
|
+
guards and alternatives even when they do not appear as graph nodes.
|
|
65
66
|
- Keep uncertainty in `questions`. A flow's unresolved questions block its
|
|
66
67
|
credit. Assertion-level questions record broader unfinished investigation.
|
|
67
68
|
|
|
@@ -97,8 +98,11 @@ back into the map.
|
|
|
97
98
|
|
|
98
99
|
## Assess changes before renewing flows
|
|
99
100
|
|
|
100
|
-
When inheriting a map, read every entry in `--view changes`.
|
|
101
|
-
|
|
101
|
+
When inheriting a map, read every entry in `--view changes`. Each names the
|
|
102
|
+
flows it already made stale (`knownFlows`) and, under `exposed`, the tests that
|
|
103
|
+
ran the changed code and how many flows they carry; start with those, and
|
|
104
|
+
investigate whether other flows are affected too. A flow's own `notices` list changes near its
|
|
105
|
+
nodes that could not have reached it.
|
|
102
106
|
|
|
103
107
|
Add a `changeAssessments` entry for each managed change ID. Include all listed
|
|
104
108
|
`knownFlows` that still exist, plus any other affected flows. Explain why the
|
package/docs/assertion-maps.md
CHANGED
|
@@ -107,13 +107,36 @@ token stale.
|
|
|
107
107
|
|
|
108
108
|
### What makes a review token stale
|
|
109
109
|
|
|
110
|
-
A flow needs a fresh review when its claim changes
|
|
111
|
-
|
|
112
|
-
|
|
110
|
+
A flow needs a fresh review when its claim changes; when a declaration holding
|
|
111
|
+
one of its nodes changes -- the function, method or class the node sits in, or
|
|
112
|
+
the top level of that file, its imports and constants; when that file's set of
|
|
113
|
+
declarations changes, one added, removed or renamed; when a file it watches, a
|
|
114
|
+
test it selects or its assertion's file changes; or when the configuration
|
|
115
|
+
that decides what executes changes: a transpiler, a test runner, an
|
|
116
|
+
interpreter pin.
|
|
117
|
+
|
|
118
|
+
Each reason names what moved -- `src/server.js: Server.start (line 12)
|
|
119
|
+
changed (holds this flow's return:31)` -- so you can look rather than reread.
|
|
113
120
|
|
|
114
121
|
Several things that sound like they should count do not, because an
|
|
115
122
|
acknowledgement demanded for all of them at once stops being read.
|
|
116
123
|
|
|
124
|
+
Comments do not, nor blank lines or trailing whitespace: no program can tell.
|
|
125
|
+
A comment the language itself reads is the exception and does count -- a Go
|
|
126
|
+
`//go:embed` directive, a Ruby magic comment, a Rust doctest.
|
|
127
|
+
|
|
128
|
+
A change to another declaration in a node's file does not. The claim rests on
|
|
129
|
+
the code it names; the rest of the file is the author's to name in `watch` if
|
|
130
|
+
it matters. Such a change is a notice on the flow (`notices` in the report),
|
|
131
|
+
not a review, and it is asked about once, as a change to assess.
|
|
132
|
+
|
|
133
|
+
Code the flow's test ran elsewhere does not make the flow stale either. A claim
|
|
134
|
+
does not pass through every function its test happened to execute. What each
|
|
135
|
+
test ran is recorded, and a changed file's change record says which tests ran
|
|
136
|
+
the changed code and how many flows that exposes, so the one assessment the
|
|
137
|
+
change asks for is asked of the right people -- and a change no selected test
|
|
138
|
+
ran is not asked about at all.
|
|
139
|
+
|
|
117
140
|
Cutting a release does not. A manifest is fingerprinted by what it declares, so
|
|
118
141
|
a version number moving in `package.json`, `Cargo.toml`, `pyproject.toml` or a
|
|
119
142
|
lockfile changes nothing. Neither does reformatting one.
|
|
@@ -233,11 +256,13 @@ Run the same test command again, then edit the new run's map. Supercov carries
|
|
|
233
256
|
forward compatible mappings and leaves the previous run unchanged. If a newer
|
|
234
257
|
map cannot be reused, `inheritance.skipped` explains the fallback.
|
|
235
258
|
|
|
236
|
-
Each flow depends on
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
259
|
+
Each flow depends on the declarations holding its nodes and the top level of
|
|
260
|
+
their files, on its assertion file, its selected test files and its extra
|
|
261
|
+
`watch` files. A change to any of those requires another look at that flow;
|
|
262
|
+
a comment, a blank line or another declaration's body does not. Independent
|
|
263
|
+
sibling flows stay current. Changing the assertion or its observation affects
|
|
264
|
+
all its flows. Dependency, configuration and instrumentation changes can
|
|
265
|
+
affect many flows.
|
|
241
266
|
|
|
242
267
|
Start with `assertions report --view changes`. Investigate every listed change,
|
|
243
268
|
including effects on flows that did not yet watch the changed file. Add a
|
|
@@ -255,9 +280,11 @@ response in the map's top-level `changeAssessments` array:
|
|
|
255
280
|
```
|
|
256
281
|
|
|
257
282
|
This is an excerpt to add to your existing map. Include every entry in `knownFlows`
|
|
258
|
-
that still exists and any additional affected flows.
|
|
259
|
-
|
|
260
|
-
code
|
|
283
|
+
that still exists and any additional affected flows. `knownFlows` are the flows
|
|
284
|
+
the change has already made stale. `exposed` counts the flows whose selected
|
|
285
|
+
tests ran the changed code and names those tests -- not stale for it, but the
|
|
286
|
+
ones to think about; add the flows the change actually reaches. An empty list needs an explanation of why
|
|
287
|
+
existing claims are unaffected; it does not mean the changed code is tested.
|
|
261
288
|
|
|
262
289
|
Save the assessments and graph edits, validate, and copy the examined change
|
|
263
290
|
tokens. Save again, then validate once more before copying final flow tokens.
|
package/docs/cli.md
CHANGED
|
@@ -21,6 +21,7 @@ npx supercov --help
|
|
|
21
21
|
| Inspect one assertion and its flows | `npx supercov runs latest assertion <id>` |
|
|
22
22
|
| Read matching current source code | `npx supercov runs latest source <path>` |
|
|
23
23
|
| Compare two runs | `npx supercov diff <older> <newer>` |
|
|
24
|
+
| Find the tests a change affects | `npx supercov runs latest tests affected` |
|
|
24
25
|
| Combine shards | `npx supercov merge <id> <id> [...]` |
|
|
25
26
|
| Remove local data | `npx supercov clean` |
|
|
26
27
|
| Read bundled guides | `npx supercov docs` |
|
|
@@ -289,6 +290,27 @@ target. It does not edit, delete, or skip tests for you. Treat the result as an
|
|
|
289
290
|
analysis aid, not permission to remove tests that protect behavior outside the
|
|
290
291
|
selected metric.
|
|
291
292
|
|
|
293
|
+
## Find the tests a change affects
|
|
294
|
+
|
|
295
|
+
```sh supercov
|
|
296
|
+
npx supercov runs latest tests affected
|
|
297
|
+
npx supercov runs latest tests affected --files
|
|
298
|
+
npx supercov runs latest tests affected --json
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
`tests affected` names the tests of a run whose recorded execution the changes
|
|
302
|
+
since that run could have reached: a change in code the test ran, in its test
|
|
303
|
+
file, or in the shape of a file it ran code in -- a declaration added, removed
|
|
304
|
+
or renamed. A change confined to code the test never ran does not count, and
|
|
305
|
+
neither do comments, blank lines or trailing whitespace. A test that did not
|
|
306
|
+
pass in the run is listed regardless.
|
|
307
|
+
|
|
308
|
+
`--names` prints one affected test name per line and `--files` one test file
|
|
309
|
+
per line, for a runner's filter. A dependency, lockfile, configuration or
|
|
310
|
+
toolchain change affects every test and is reported as such. A source file
|
|
311
|
+
added since the run is outside every test's record; the working-tree check
|
|
312
|
+
says so, and the suite should run in full.
|
|
313
|
+
|
|
292
314
|
## Combine shards
|
|
293
315
|
|
|
294
316
|
```sh supercov
|
package/docs/getting-started.md
CHANGED
|
@@ -121,6 +121,20 @@ on crates.io (`cargo install supercov`). The first invocation may download
|
|
|
121
121
|
Supercov from the registry. Supercov itself does not upload your source or
|
|
122
122
|
coverage evidence to a Supercov service.
|
|
123
123
|
|
|
124
|
+
Whichever language your project is in, you should not have to install another
|
|
125
|
+
one first. Go runs it the way Go runs any tool, and without touching your
|
|
126
|
+
`go.mod`:
|
|
127
|
+
|
|
128
|
+
```sh supercov
|
|
129
|
+
go run github.com/supercorp-ai/supercov/cmd/supercov@latest -- go test ./...
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
Homebrew installs the same binary for everything else, a JVM project included:
|
|
133
|
+
|
|
134
|
+
```sh
|
|
135
|
+
brew install supercorp-ai/tap/supercov
|
|
136
|
+
```
|
|
137
|
+
|
|
124
138
|
## Files and cleanup
|
|
125
139
|
|
|
126
140
|
Completed runs live under `.supercov/runs/`. Supercov also keeps an isolated
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "supercov",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.51",
|
|
4
4
|
"description": "Coverage for coding agents and software factories \ud83c\udf19",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -76,7 +76,7 @@
|
|
|
76
76
|
"benchmark:check": "cargo build --release -p supercov && node scripts/rust-transform-benchmark.mjs",
|
|
77
77
|
"benchmark:python-monitoring": "cargo build -p supercov && node scripts/python-monitoring-benchmark.mjs",
|
|
78
78
|
"check": "cargo fmt --all -- --check && cargo clippy --workspace --all-targets -- -D warnings && npm run test && npm run test:runtime && npm run test:rust-assets && npm run test:assertion-maps && node scripts/package-preflight.mjs && node scripts/verify-binstall-metadata.mjs --offline",
|
|
79
|
-
"release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:go && npm run test:jvm && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python-monitoring && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
|
|
79
|
+
"release:check": "cargo clean && node scripts/sweep-target.mjs && npm run check && npm run test:engine && npm run test:go && npm run test:jvm && npm run test:launcher && npm run test:fixture && npm run test:watchdog && npm run test:engine-contract && npm run test:agent && npm run test:child-attribution && npm run test:host-loader && npm run test:python-monitoring && npm run test:ruby-coverage && npm run test:rust-public-cargo && npm run test:packed-npx && npm run test:clang-mcdc && npm run benchmark:check",
|
|
80
80
|
"release:bump": "node scripts/bump-version.mjs",
|
|
81
81
|
"sweep": "node scripts/sweep-target.mjs",
|
|
82
82
|
"oracle:rust": "cargo build --release -p supercov && node scripts/rust-coverage-oracle.mjs",
|
|
@@ -93,17 +93,18 @@
|
|
|
93
93
|
"test:assertion-maps:js": "cargo build -p supercov && node scripts/assertion-map-schema.mjs --check && node scripts/assertion-map-js-integration.mjs && node scripts/assertion-dogfood-integration.mjs",
|
|
94
94
|
"sync:assertion-schema": "cargo build -p supercov && node scripts/assertion-map-schema.mjs",
|
|
95
95
|
"docs:sync": "node scripts/sync-docs.mjs",
|
|
96
|
-
"docs:check": "node scripts/sync-docs.mjs --check"
|
|
96
|
+
"docs:check": "node scripts/sync-docs.mjs --check",
|
|
97
|
+
"test:launcher": "go vet ./cmd/... && go test ./cmd/..."
|
|
97
98
|
},
|
|
98
99
|
"optionalDependencies": {
|
|
99
|
-
"@supercov/cli-darwin-arm64": "0.0.
|
|
100
|
-
"@supercov/cli-darwin-x64": "0.0.
|
|
101
|
-
"@supercov/cli-linux-arm64-gnu": "0.0.
|
|
102
|
-
"@supercov/cli-linux-arm64-musl": "0.0.
|
|
103
|
-
"@supercov/cli-linux-x64-gnu": "0.0.
|
|
104
|
-
"@supercov/cli-linux-x64-musl": "0.0.
|
|
105
|
-
"@supercov/cli-win32-arm64": "0.0.
|
|
106
|
-
"@supercov/cli-win32-x64": "0.0.
|
|
100
|
+
"@supercov/cli-darwin-arm64": "0.0.51",
|
|
101
|
+
"@supercov/cli-darwin-x64": "0.0.51",
|
|
102
|
+
"@supercov/cli-linux-arm64-gnu": "0.0.51",
|
|
103
|
+
"@supercov/cli-linux-arm64-musl": "0.0.51",
|
|
104
|
+
"@supercov/cli-linux-x64-gnu": "0.0.51",
|
|
105
|
+
"@supercov/cli-linux-x64-musl": "0.0.51",
|
|
106
|
+
"@supercov/cli-win32-arm64": "0.0.51",
|
|
107
|
+
"@supercov/cli-win32-x64": "0.0.51"
|
|
107
108
|
},
|
|
108
109
|
"peerDependencies": {
|
|
109
110
|
"@playwright/test": ">=1.55.0",
|
|
@@ -110,7 +110,7 @@
|
|
|
110
110
|
"string",
|
|
111
111
|
"null"
|
|
112
112
|
],
|
|
113
|
-
"pattern": "^
|
|
113
|
+
"pattern": "^scov[23]:[0-9a-f]{64}$"
|
|
114
114
|
},
|
|
115
115
|
"explanation": {
|
|
116
116
|
"type": "string"
|
|
@@ -236,7 +236,7 @@
|
|
|
236
236
|
"string",
|
|
237
237
|
"null"
|
|
238
238
|
],
|
|
239
|
-
"pattern": "^
|
|
239
|
+
"pattern": "^scov[23]:[0-9a-f]{64}$"
|
|
240
240
|
},
|
|
241
241
|
"affectedFlows": {
|
|
242
242
|
"type": "array",
|