supercov 0.0.43 → 0.0.45
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -14
- package/docs/agent-loop.md +128 -39
- package/docs/assertion-agent.md +156 -0
- package/docs/assertion-evidence.md +9 -135
- package/docs/assertion-maps.md +252 -0
- package/docs/assertions.md +82 -0
- package/docs/cli.md +38 -23
- package/docs/coverage-model.md +18 -6
- package/docs/evidence.md +6 -6
- package/docs/getting-started.md +60 -72
- package/docs/performance.md +4 -4
- package/docs/troubleshooting.md +8 -8
- package/docs/verification.md +2 -2
- package/docs/workspace-isolation.md +1 -1
- package/package.json +34 -33
- package/runtime/javascript/nodeAssertAdapter.mjs +32 -8
- package/runtime/javascript/nodeTest.mjs +13 -5
- package/runtime/javascript/runnerEvidence.mjs +33 -11
- package/runtime/javascript/runtime.mjs +27 -23
- package/schemas/assertions.schema.json +276 -0
- package/analyzers/typescript/README.md +0 -55
- package/analyzers/typescript/bin/compiler-identity.mjs +0 -78
- package/analyzers/typescript/bin/identity.mjs +0 -67
- package/analyzers/typescript/bin/query.mjs +0 -29
- package/analyzers/typescript/dist/analyze.js +0 -3972
- package/analyzers/typescript/dist/archive.js +0 -309
- package/analyzers/typescript/dist/build-identity.json +0 -1
- package/analyzers/typescript/dist/compiler.js +0 -32
- package/analyzers/typescript/dist/frontend.js +0 -75
- package/analyzers/typescript/dist/native-frontend.js +0 -271
- package/analyzers/typescript/dist/pragmas.js +0 -143
- package/analyzers/typescript/dist/types.js +0 -1
- package/analyzers/typescript/package.json +0 -27
- package/analyzers/typescript/src/analyze.ts +0 -4538
- package/analyzers/typescript/src/archive.ts +0 -438
- package/analyzers/typescript/src/compiler.ts +0 -49
- package/analyzers/typescript/src/frontend.ts +0 -136
- package/analyzers/typescript/src/native-frontend.ts +0 -315
- package/analyzers/typescript/src/pragmas.ts +0 -218
- package/analyzers/typescript/src/types.ts +0 -45
- package/analyzers/typescript/tsconfig.json +0 -12
- package/docs/code-verification.md +0 -182
|
@@ -1,182 +0,0 @@
|
|
|
1
|
-
# Example
|
|
2
|
-
|
|
3
|
-
This example measures coverage for a small checkout function. It includes the
|
|
4
|
-
original tests, an additional test for an expired session, and commands for
|
|
5
|
-
comparing the results.
|
|
6
|
-
|
|
7
|
-
## Before you start
|
|
8
|
-
|
|
9
|
-
You need Node.js 22 or newer and npm. Clone the repository and install the
|
|
10
|
-
example's dependencies:
|
|
11
|
-
|
|
12
|
-
```sh
|
|
13
|
-
git clone --depth 1 https://github.com/supercorp-ai/supercov.git
|
|
14
|
-
cd supercov/examples/checkout-verification
|
|
15
|
-
npm ci
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
Run the commands below from this directory. The example uses Supercov 0.0.42
|
|
19
|
-
and Node's built-in test runner. Both the original tests and the additional
|
|
20
|
-
test are included. The first three steps do not require any file edits.
|
|
21
|
-
|
|
22
|
-
## 1. Run the original tests
|
|
23
|
-
|
|
24
|
-
In `src/session.js`, checkout is allowed only if the customer is signed in and
|
|
25
|
-
their session has not expired:
|
|
26
|
-
|
|
27
|
-
```js
|
|
28
|
-
export function canCheckout(signedIn, expired) {
|
|
29
|
-
if (signedIn && !expired) return true;
|
|
30
|
-
return false;
|
|
31
|
-
}
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
The two tests in `tests/session.test.js` check a valid session and a signed-out
|
|
35
|
-
visitor:
|
|
36
|
-
|
|
37
|
-
```js
|
|
38
|
-
assert.equal(canCheckout(true, false), true);
|
|
39
|
-
assert.equal(canCheckout(false, false), false);
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
Run those tests through Supercov, then open the summary:
|
|
43
|
-
|
|
44
|
-
```sh
|
|
45
|
-
npx supercov -- node --test tests/session.test.js
|
|
46
|
-
npx supercov runs latest
|
|
47
|
-
```
|
|
48
|
-
|
|
49
|
-
Everything after `--` is the test command Supercov runs. In your own project,
|
|
50
|
-
use your existing test command there.
|
|
51
|
-
|
|
52
|
-
Both tests pass. The coverage section shows:
|
|
53
|
-
|
|
54
|
-
```text
|
|
55
|
-
Coverage
|
|
56
|
-
Lines 100.00% (3/3)
|
|
57
|
-
Branches 100.00% (2/2)
|
|
58
|
-
MC/DC 50.00% (1/2)
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
Line and branch coverage are 100% because the tests reach both `return true`
|
|
62
|
-
and `return false`. The MC/DC result shows there is still a condition to test.
|
|
63
|
-
|
|
64
|
-
Keep the run ID printed at the top of the summary. You'll use it to compare
|
|
65
|
-
this run with the next one.
|
|
66
|
-
|
|
67
|
-
## 2. Inspect the missing condition
|
|
68
|
-
|
|
69
|
-
Ask about the decision on line 2:
|
|
70
|
-
|
|
71
|
-
```sh
|
|
72
|
-
npx supercov runs latest decision src/session.js:2
|
|
73
|
-
```
|
|
74
|
-
|
|
75
|
-
```text
|
|
76
|
-
signedIn && !expired
|
|
77
|
-
C1 covered + asserted: signedIn
|
|
78
|
-
C2 MISSING: !expired
|
|
79
|
-
confidence asserted; asserted MC/DC 1/2
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
MC/DC stands for Modified Condition/Decision Coverage. It checks whether each
|
|
83
|
-
condition has independently affected the decision. The original tests show
|
|
84
|
-
that changing `signedIn` changes the result, but neither test changes `expired`.
|
|
85
|
-
That leaves one of two conditions covered: 50%.
|
|
86
|
-
|
|
87
|
-
`C2 MISSING: !expired` points to the case to test: a customer who is still
|
|
88
|
-
signed in, but whose session has expired. Checkout should be denied.
|
|
89
|
-
|
|
90
|
-
## 3. Include the expired-session test
|
|
91
|
-
|
|
92
|
-
`tests/expired-session.test.js` contains that test:
|
|
93
|
-
|
|
94
|
-
```js
|
|
95
|
-
test('an expired session cannot check out', () => {
|
|
96
|
-
assert.equal(canCheckout(true, true), false);
|
|
97
|
-
});
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
Run both test files and open the new summary:
|
|
101
|
-
|
|
102
|
-
```sh
|
|
103
|
-
npx supercov -- node --test tests/session.test.js tests/expired-session.test.js
|
|
104
|
-
npx supercov runs latest
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
All three tests pass:
|
|
108
|
-
|
|
109
|
-
```text
|
|
110
|
-
Coverage
|
|
111
|
-
Lines 100.00% (3/3)
|
|
112
|
-
Branches 100.00% (2/2)
|
|
113
|
-
MC/DC 100.00% (2/2)
|
|
114
|
-
```
|
|
115
|
-
|
|
116
|
-
Query the same decision again:
|
|
117
|
-
|
|
118
|
-
```sh
|
|
119
|
-
npx supercov runs latest decision src/session.js:2
|
|
120
|
-
```
|
|
121
|
-
|
|
122
|
-
The expiry condition is now covered. `asserted MC/DC 2/2` means both conditions
|
|
123
|
-
have coverage evidence linked to passing assertions:
|
|
124
|
-
|
|
125
|
-
```text
|
|
126
|
-
C2 covered + asserted: !expired
|
|
127
|
-
confidence asserted; asserted MC/DC 2/2
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
Compare the runs, replacing `<before-run-id>` with the ID you saved in step 1:
|
|
131
|
-
|
|
132
|
-
```sh
|
|
133
|
-
npx supercov diff <before-run-id> latest
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
```text
|
|
137
|
-
lines +0pp, branches +0pp, MC/DC +50pp
|
|
138
|
-
gained: 0 lines, 0 branches, 1 MC/DC conditions
|
|
139
|
-
lost: 0 lines, 0 branches, 0 MC/DC conditions
|
|
140
|
-
+ MC/DC src/session.js:2 C2 !expired
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
Line and branch coverage have not changed. The new test covers the missing
|
|
144
|
-
expiry condition without changing application code.
|
|
145
|
-
|
|
146
|
-
## 4. Check that the test catches a regression
|
|
147
|
-
|
|
148
|
-
In this example only, temporarily remove the expiry check from `src/session.js`:
|
|
149
|
-
|
|
150
|
-
```diff
|
|
151
|
-
- if (signedIn && !expired) return true;
|
|
152
|
-
+ if (signedIn) return true;
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
Run the original two tests against the changed function:
|
|
156
|
-
|
|
157
|
-
```sh
|
|
158
|
-
npx supercov -- node --test tests/session.test.js
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
Then include the expired-session test:
|
|
162
|
-
|
|
163
|
-
```sh
|
|
164
|
-
npx supercov -- node --test tests/session.test.js tests/expired-session.test.js
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
| Tests run against the changed function | Result |
|
|
168
|
-
| --- | --- |
|
|
169
|
-
| Original two tests | Both pass. |
|
|
170
|
-
| All three tests | The expired-session test fails; the other two pass. |
|
|
171
|
-
|
|
172
|
-
The new test expects `false`, but the changed function returns `true`. The
|
|
173
|
-
second command should fail: that is the test catching the removed expiry check.
|
|
174
|
-
|
|
175
|
-
Restore `&& !expired` in `src/session.js` when you finish, then rerun all three
|
|
176
|
-
tests with the same command. They should pass again.
|
|
177
|
-
|
|
178
|
-
## Next
|
|
179
|
-
|
|
180
|
-
- [Full example and recorded output](https://github.com/supercorp-ai/supercov/tree/main/examples/checkout-verification) — source, tests, and the complete output excerpted above.
|
|
181
|
-
- [Understanding coverage](coverage-model.md) — what each metric measures and what 100% means.
|
|
182
|
-
- [Agent workflow](agent-loop.md) — use the same run, inspect, test, and compare steps with a coding agent.
|