@codacy/verity-cli 0.33.3-experimental.52f1656 → 0.33.3-experimental.7124bbc

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@codacy/verity-cli",
3
- "version": "0.33.3-experimental.52f1656",
3
+ "version": "0.33.3-experimental.7124bbc",
4
4
  "description": "CLI for Verity quality gate service",
5
5
  "license": "SEE LICENSE IN LICENSE",
6
6
  "homepage": "https://verity.md",
@@ -1,174 +0,0 @@
1
- ---
2
- description: >-
3
- Formally verify a small, risky piece of code with a Lean 4 model: pick candidates from git
4
- history, CONFIRM with the user which code to model, write the model, run it locally with
5
- `verity prove check`, and replay any counterexample on the real code before calling it a bug.
6
- Use when the user asks to verify, prove, or model-check code, or asks "can this race / double
7
- run / fail open?". Anything typed after /verity-prove is the user's hint: the file, function,
8
- or worry to check.
9
- ---
10
- # /verity-prove — Formal verification of small pieces of code
11
-
12
- **The words after `/verity-prove` are the hint.** `/verity-prove` alone means "find me
13
- something worth checking"; `/verity-prove can the Stripe webhook double-charge on a
14
- redelivered event?` means that is the target and, if it is a worry, the property.
15
- Carry it through every step below as `$ARGUMENTS`.
16
-
17
- Verity keeps **Lean 4 models of the riskiest code**, not of the codebase. A model is
18
- another tool for finding bugs: a few dozen lines that abstract ONE function and check
19
- a few properties by exhaustive bounded search. When a property fails you get the
20
- **shortest sequence of actions** that breaks it — a hypothesis to replay on the real
21
- code. When it holds you know exactly how far it was checked.
22
-
23
- This skill is the **explicit, user-driven** entry; nothing in it blocks a commit or a
24
- push. (The push-time phase — `verity config formal push` — is a separate, opt-in path
25
- that uses the same tools.) Lean runs on this machine because the user asked for it.
26
-
27
- ## The flow
28
-
29
- ### 1. Find what deserves a model
30
-
31
- ```bash
32
- verity prove candidates --limit 8 # git history + risk signals
33
- verity prove candidates --rank luna --json # + a cheap model re-ranks and names a symbol
34
- verity prove candidates src/payments # scoped to a path
35
- ```
36
-
37
- Each candidate says why: fix commits over commits, check-then-act, retries, caches,
38
- locks/leases, status transitions, money.
39
-
40
- **With a hint** (`$ARGUMENTS` non-empty) — a file, a function, or a worry in the user's
41
- own words ("can the webhook double-charge?", "does the retry ever lose the feedback?"):
42
-
43
- ```bash
44
- verity prove candidates --hint "$ARGUMENTS"
45
- ```
46
-
47
- A named file or function skips the ranking entirely: go to step 2 with it. A *worry*
48
- becomes an `intent` property in step 4, with `source := "user, <date>: <their words>"`;
49
- it needs no further confirmation, they just gave it. The candidates list is still shown,
50
- so the user sees what else the history says — but the hint decides.
51
-
52
- ### 2. Confirm with the user — always
53
-
54
- **Do not write a model until the user has said which code.** Present at most three
55
- candidates in one short message, each as: file, the function you would model, the
56
- property you would check first, and the reason from the signals. Then ask. The user may
57
- pick one, name something else, or narrow the property. Their answer is the scope; do not
58
- widen it.
59
-
60
- A model is a claim about what the code *means*. Only the person who owns the code can
61
- say whether that claim is worth checking.
62
-
63
- ### 3. Set up the project (once per repo)
64
-
65
- ```bash
66
- verity prove init
67
- ```
68
-
69
- Creates `.verity/proofs/` — a Lake project with Verity's library (`Verity.System`,
70
- `Verity.check`, a bounded explorer). Re-running refreshes the library and never touches
71
- `Models/`. If Lean is missing, install it only after telling the user what will be
72
- installed and where: `verity lean install` (official release, ≈800 MB download, 2.7 GB
73
- on disk, into `~/.verity/lean`). Ranking (`--hint`, `--rank luna`) and `verity prove
74
- replay` run on the Verity service with the user's login (`verity login`); without a
75
- login, ranking is deterministic and replay is by hand (step 6.2).
76
-
77
- ### 4. Write the model — `Models/<Name>.lean`
78
-
79
- Rules that keep models small and honest:
80
-
81
- - **One symbol.** The header names it: `Verity.model { name, abstracts := [{ file, symbol, sha256 }] }`.
82
- Get the hash with `verity prove hash <file>`. `verity prove check` reports drift when the file changes.
83
- When the code is a *region* rather than a named function (a branch inside a large
84
- handler), add `lines := "a-b"` (1-based, inclusive) — `verity prove replay` needs it
85
- to know which real lines must run.
86
- - **State = only what the property reads.** Actions = the operations that can interleave
87
- (a retry, a second call, a concurrent writer). Steps = what the code does, not what it should.
88
- - **Properties are tagged.** `kind := .structural` for the catalogue (idempotence, no lost
89
- update, status only moves forward, retries terminate, never fails open, bounded size,
90
- check-then-act is atomic, no double effect). `kind := .intent` **only** when a human
91
- wrote the expectation down — a doc comment, an ADR, a ticket, a Standard rule — and
92
- `source :=` cites it. Never invent intent.
93
- - **End with the checks.** `#eval Verity.model …` then one `#eval Verity.check …` per
94
- property (bounded exploration), `Verity.checkData` for input properties, `Verity.proved`
95
- only beside a real theorem. `sorry` fails the run.
96
- - Keep it under ~120 lines. If it wants more, the scope is wrong — go back to step 2.
97
-
98
- `Models/README.md` has the skeleton; `cli/tests/fixtures/proofs/SeedRunner.lean` in
99
- Verity's own repo is a worked example (the VRT-243 seed-duplication bug).
100
-
101
- ### 5. Check it
102
-
103
- ```bash
104
- verity prove check <Name> # one model
105
- verity prove check # every model
106
- verity prove check <Name> --json # for scripts
107
- ```
108
-
109
- Per property: `✓ holds to bound k`, `✓ proved`, `✗ COUNTEREXAMPLE` with the trace and
110
- the state after each step, or `? inconclusive` (state cap).
111
-
112
- **Hard caps — never loop:**
113
- - elaboration errors: at most **3** fix rounds, then stop and say the model could not be
114
- built;
115
- - a counterexample that **diverges** on replay (the model was wrong about the code): at
116
- most **2** model revisions, each naming what the replay showed was different; after
117
- that, stop and report "no faithful model found for `<symbol>`" — that is a valid result;
118
- - never weaken or delete a property to make it pass; never add a property to make
119
- something fail.
120
-
121
- A wrong model is expected some of the time. It costs a replay and is discarded; it never
122
- costs a false bug, because only a *reproduced* counterexample is reported (step 6).
123
-
124
- ### 6. Replay before you call it a bug
125
-
126
- A counterexample is a fact about the **model**. Before reporting a bug:
127
-
128
- 1. Run the replay in OpenAI's disposable container (network off, deleted afterwards):
129
-
130
- ```bash
131
- verity prove replay <Name> # every counterexample
132
- verity prove replay <Name> --property P2 # one
133
- ```
134
-
135
- It hands a model the trace, the Lean file and the real source — the abstracted
136
- file's content leaves the machine for the Verity service and OpenAI's container;
137
- only git-tracked files are ever read — and requires three things before accepting
138
- its answer: code actually ran (a shell call with output), the real function's lines
139
- appear verbatim in what it wrote, and the harness itself printed
140
- `REPLAY_RESULT=reproduced`. `REPRODUCED` → the
141
- real code does this. `DIVERGED` → the model is wrong about the code. `UNVERIFIED` →
142
- not evidence either way. `UNREPLAYABLE` → it could not run the function faithfully.
143
- Read its reason: the stubs it chose are where a false reproduction hides.
144
- 2. When the replay is unverified or unreplayable — or the code needs a runtime the
145
- container lacks (Deno edge functions, a real database) — replay by hand: read the
146
- real function again, map every trace action to a real call, and reproduce it with a
147
- throwaway harness in a scratch directory that loads the real module and stubs only
148
- I/O boundaries. **Never edit the user's source files to do this.**
149
- 3. Reproduced → report it as a bug, with the trace and the reproduction. Not reproduced
150
- → the model is wrong (`model-divergence`): say so, fix the model, do not report a bug.
151
-
152
- This is the rule Verity applies everywhere (memory: *replay formal counterexamples
153
- against changed code before blocking*). A model that finds a bug in itself is noise.
154
-
155
- ### 7. Report, in a few lines
156
-
157
- State what was modelled, what was checked to what bound, what was refuted and whether
158
- it replayed, and what remains `intent` awaiting the user's confirmation. **Never offer
159
- to commit the model** — `.verity/proofs/` is machine-local and ignores itself; models
160
- would only confuse reviewers. What reaches the team is the bug report and, when asked,
161
- the fix. Then record the outcome:
162
-
163
- ```bash
164
- verity reflect --user-input "Modelled runSeed (cli/src/lib/seed-runner.ts): P1 at-most-once refuted in 2 steps, replayed — guard checks a path the server never writes." --kind gotcha
165
- ```
166
-
167
- ## Do not
168
-
169
- - Model code the user did not confirm.
170
- - Present a model-only counterexample as a bug.
171
- - Put `sorry` in a model, or weaken a property to get green.
172
- - Touch source files, run Lean from a hook, or install anything silently.
173
- - Commit, stage, or un-ignore anything under `.verity/proofs/`.
174
- - Model a whole module. If the state has more than ~6 fields, narrow it.