@codacy/verity-cli 0.33.3-experimental.52f1656 → 0.33.3-experimental.5a3f573

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,46 +1,62 @@
1
1
  ---
2
2
  description: >-
3
- Set Verity up in this project: ask the two setup questions, run the deterministic
4
- install, and repair a project the Claude Code plugin has just been added to. Use
5
- right after installing the Verity plugin, or when Verity says this project is not
6
- set up.
3
+ Set Verity up in this project: ask the setup questions in the session, run the
4
+ deterministic install with those answers, sign in, and repair a project the Claude
5
+ Code plugin has just been added to. Use right after installing the Verity plugin,
6
+ or when Verity says this project is not set up.
7
7
  disable-model-invocation: true
8
8
  ---
9
9
  # /verity-init — set Verity up in this project
10
10
 
11
- You are running the setup a plugin install cannot do for itself.
11
+ You are running, inside this session, the same setup `verity init` runs at a
12
+ terminal: the same questions, the same install, the same sign-in. The only
13
+ difference is who asks. At a terminal the CLI prompts; here **you** ask, and pass
14
+ the answers down as flags.
12
15
 
13
16
  **Why this skill exists.** Claude Code has no post-install hook and a plugin
14
17
  cannot invoke a skill, so installing Verity repairs nothing: it cannot remove a
15
18
  project's superseded skills, stand down duplicate hooks, or derive a Standard.
16
19
  All of that needs a command someone runs — this one.
17
20
 
18
- **`verity init` prompts on a terminal, and you are not one.** That is the whole
19
- problem this skill solves. Left to itself the CLI would take `--yes` and silently
20
- choose **stop-only** review — no pre-commit gate, no pre-push gate — for every
21
- plugin user, which is exactly the outcome we treat as a serious bug when it
22
- happens by accident. So **you** ask, in the session, where the user actually is.
21
+ **`verity init` prompts on a terminal, and you are not one.** Run with `--yes`
22
+ alone it takes the unattended answer to every question: stop-only review (no
23
+ commit or push gate), no telemetry, no sign-in, nothing untracked. Every one of
24
+ those is a decision that belongs to the user, so every one is asked here.
25
+
26
+ Run every `verity` command below the way this session's context says (the
27
+ plugin's own copy), not with plain `verity`, whenever the context names one.
23
28
 
24
29
  ---
25
30
 
26
- ## Step 0: Check the CLI before asking anything
31
+ ## Step 0: Read where the project stands — before asking anything
32
+
33
+ ```bash
34
+ verity doctor --json
35
+ ```
27
36
 
28
- If this session's context says the `verity` on PATH is a different version from
29
- the plugin's, **tell the user that first**, in one line, before the questions —
30
- with the `npm install -g` command the context gives. Finding out after they have
31
- answered is the failure this step exists to prevent: an older CLI cannot take the
32
- answers.
37
+ From that one report:
33
38
 
34
- Then run every command below the way the context says (the plugin's own copy),
35
- not with plain `verity`.
39
+ - **`blocked` is true** → show the `prerequisites[].remedy` lines and stop.
40
+ Nothing below can work.
41
+ - **`hooks.versions.skew` is set** → tell the user first, in one line, with
42
+ `hooks.versions.skew.remedy`. An older CLI cannot take the answers, so they
43
+ should hear it now, not after answering.
44
+ - **`answers`** holds what a previous run recorded (`intensity`, `moments`,
45
+ `telemetry`). On a re-run those are the defaults — pre-select them. The shipped
46
+ defaults are only for a project with nothing recorded.
47
+ - **`telemetry.enabled`** decides whether the telemetry question is asked at all.
48
+ - **`memory.trackedFiles > 0` and `memory.optedOut` is false** → the graph
49
+ question applies.
50
+ - **`trackedState.files > 0`** → the committed-state question applies.
36
51
 
37
- ## Step 1: Ask both questions in ONE prompt
52
+ ## Step 1: Ask — once, in one prompt
38
53
 
39
- Use `AskUserQuestion` with **both** questions in a single call. Do not ask them
40
- one at a time, and do not ask them in prose — this is the only interruption the
41
- setup gets, so spend it once.
54
+ Use `AskUserQuestion` with every question that applies **in a single call**, in
55
+ this order. It takes at most four; if more apply, put the rest in ONE follow-up
56
+ call. Do not ask one at a time, and do not ask in prose unless `AskUserQuestion`
57
+ is unavailable (then ask in prose and **wait** — never choose for the user).
42
58
 
43
- **Question 1 — "When should Verity review your code?"** (multi-select)
59
+ **1 — "When should Verity review your code?"** (multi-select, always asked)
44
60
 
45
61
  | Option | Description |
46
62
  |---|---|
@@ -48,12 +64,12 @@ setup gets, so spend it once.
48
64
  | `Before commit` | Reviews the staged diff and blocks the commit on a failing review. |
49
65
  | `Before push / PR` | Reviews the commits about to be pushed and blocks on a failing review. |
50
66
 
51
- Pre-select **After every turn**, and do not call it "recommended": under the
52
- plugin it runs whatever the answer, and a user who unticks it should not find
53
- that out afterwards. Say plainly in the question text that choosing only the
54
- first means **git commits and pushes are not gated**.
67
+ Pre-select the recorded moments, or **After every turn** on a first run, and do
68
+ not call it "recommended": under the plugin it runs whatever the answer. Say
69
+ plainly in the question text that choosing only the first means **git commits
70
+ and pushes are not gated**.
55
71
 
56
- **Question 2 — "How deeply should Verity review?"** (single-select)
72
+ **2 — "How deeply should Verity review?"** (single-select, always asked)
57
73
 
58
74
  | Option | Description |
59
75
  |---|---|
@@ -61,75 +77,157 @@ first means **git commits and pushes are not gated**.
61
77
  | `Lightweight` | Critical security only, about 3s. |
62
78
  | `Thorough` | All tools, all rules, about 15s. |
63
79
 
80
+ **3 — "Send cost & usage telemetry to Verity?"** (single-select; **only when
81
+ `telemetry.enabled` is false**)
82
+
83
+ | Option | Description |
84
+ |---|---|
85
+ | `Yes, enable it` | Powers the /usage dashboard. Sends Claude Code's own OpenTelemetry metrics and traces: model names, token counts, cost, session ids. Not your prompts, code, or tool input/output. |
86
+ | `No` | /usage stays empty. Enable later with `verity telemetry install`. |
87
+
88
+ **4 — "Sign in to Verity now?"** (single-select, always asked)
89
+
90
+ | Option | Description |
91
+ |---|---|
92
+ | `Sign in now (recommended)` | One GitHub login covers every repository you can write to, for 90 days. Needed for run history, cloud memory and the dashboard. If you are already signed in this just confirms it. |
93
+ | `Skip for now` | The gate still reviews your code locally; nothing is saved. Sign in later with `/verity-init` or `verity login`. |
94
+
95
+ If asked, explain: the GitHub token is used once to check which repositories they
96
+ can write to, then discarded; signing in does not give Verity access to their
97
+ code, which is analyzed in memory and never stored.
98
+
99
+ **5 — "Stop tracking Verity's knowledge graph in git?"** (only when the graph
100
+ question applies; say how many files)
101
+
102
+ | Option | Description |
103
+ |---|---|
104
+ | `Yes, untrack it (recommended)` | Files stay on disk; their removal from the index is staged for you to commit. Generated notes stop appearing in every diff and PR. |
105
+ | `No, not now` | Nothing changes. `verity memory track` records it as deliberate. |
106
+
107
+ **6 — "Untrack committed Verity state?"** (only when `trackedState.files > 0`;
108
+ say how many files)
109
+
110
+ | Option | Description |
111
+ |---|---|
112
+ | `Yes, untrack it (recommended)` | Machine-local files that should never have been committed — they can include copies of flagged secrets. Files stay on disk; the removal is staged. |
113
+ | `No, not now` | Nothing changes. |
114
+
64
115
  ## Step 2: Run the install with those answers
65
116
 
66
- Map the answers to flags and run it — one command, no prompts, nothing silently
67
- defaulted:
117
+ One command, no prompts, nothing silently defaulted:
68
118
 
69
119
  ```bash
70
- verity init --yes --no-setup --moments <stop[,pre-commit][,pre-push]> --intensity <balanced|lightweight|thorough>
120
+ verity init --yes --no-setup \
121
+ --moments <stop[,pre-commit][,pre-push] | none> \
122
+ --intensity <balanced|lightweight|thorough> \
123
+ [--telemetry on|off] [--untrack-memory] [--untrack-state]
71
124
  ```
72
125
 
73
- `--moments none` is the deliberate way to gate nothing; an unrecognised value is
74
- refused rather than dropped, so a typo cannot quietly remove a gate.
126
+ - Pass `--telemetry` only when question 3 was asked. Omitting it leaves the
127
+ recorded choice alone.
128
+ - Pass `--untrack-memory` / `--untrack-state` only on a yes.
129
+ - Pass `stop` only if they ticked it. Under the plugin the Stop review runs either
130
+ way and `verity init` says so.
131
+ - An unrecognised value is refused rather than dropped, so a typo cannot quietly
132
+ remove a gate. If init exits non-zero, show its error and stop.
133
+
134
+ This is idempotent and does the whole install: removes the project's own
135
+ `.claude/skills/verity-*` copies that the plugin supersedes, stands down duplicate
136
+ `.claude/settings.json` hooks, adopts the team's Standard from the service or
137
+ derives one from the codebase, heals a stale analysis config, and records the
138
+ plugin version — which is what stops Verity asking at every session start.
75
139
 
76
- Pass `stop` only if they ticked it. Under the plugin the Stop review runs either
77
- way and `verity init` says so; recording their real answer is what lets it be
78
- honoured once Stop can be turned off.
140
+ Init runs before sign-in, so on a first run it says it skipped authentication and
141
+ that telemetry needs a token. That is expected — Step 3 handles both. Do not
142
+ repeat those lines to the user as problems.
79
143
 
80
- This is idempotent and does the whole job: removes the project's own
81
- `.claude/skills/verity-*` copies that the plugin supersedes (they are why the
82
- slash-command list shows every skill twice), stands down duplicate
83
- `.claude/settings.json` hooks so a turn is reviewed once, derives the Standard
84
- and the analysis config if there is none, heals a stale one, and records the
85
- plugin version — which is what stops Verity asking at every session start.
144
+ ## Step 3: Sign in — here, in two halves
145
+
146
+ Skip this step if they chose **Skip for now**; say the gate runs locally and how
147
+ to sign in later.
148
+
149
+ `verity login` on its own prints a code and then blocks until it is approved, and
150
+ you would only see the code after it ended. So it is split:
151
+
152
+ ```bash
153
+ verity login --start
154
+ ```
155
+
156
+ It prints one JSON object on stdout:
157
+
158
+ - **`"status": "logged_in"`** → they already are. Say so and go to Step 4.
159
+ - **`"status": "pending"`** → show the user `verification_uri` and `user_code`
160
+ exactly as given, and ask them to approve it in their browser and tell you when
161
+ they have. Wait for their reply. Do not run anything in the meantime.
162
+ - **`"status": "failed"`** → show the error, and tell them to run `verity login`
163
+ in their own terminal later. Go to Step 4.
164
+
165
+ When they say it is approved (or ask you to check):
86
166
 
87
- Report what it removed. Those skill directories are usually committed, so the
88
- deletions will show in `git status` and are the user's to commit.
167
+ ```bash
168
+ verity login --finish
169
+ ```
170
+
171
+ Give the tool call at least a 150-second timeout; it waits up to 100 seconds.
172
+
173
+ - **exit 0, `"status": "logged_in"`** → report the lines it printed about the
174
+ account and whether this repository is covered, word for word where it names a
175
+ link to grant the Verity GitHub App.
176
+ - **exit 2, `"status": "pending"`** → not approved yet. Show the code again, wait
177
+ for them, and run `--finish` again.
178
+ - **exit 1** → `denied`, `expired`, `service_changed`, `no_pending` or `failed`.
179
+ Show the error. For `expired` or `no_pending`, offer to start again with
180
+ `--start`.
181
+
182
+ If they change their mind while a code is pending, run `verity login --cancel`.
89
183
 
90
- ## Step 3: Tell the user to sign in — in their own terminal
184
+ ⚠ Never run plain `verity login` from here, and never tell a user whose Step 0
185
+ reported a version skew to run it in a terminal until they have fixed the skew —
186
+ the old global CLI can sign the machine in to a different server, and the plugin
187
+ follows that sign-in.
91
188
 
92
- **Do not run `verity login` yourself.** It is a GitHub device flow: it prints a
93
- code, waits for the user to approve it in a browser, and blocks until they do.
94
- Started from a tool call it just hangs, and the user never sees the code.
189
+ ## Step 4: Finish telemetry, if it waited on sign-in
95
190
 
96
- Say this, and stop:
191
+ If they said yes to telemetry and init reported it needs a token, and Step 3
192
+ signed them in:
97
193
 
98
- > Verity is set up. One more step, in your own terminal — not here:
99
- >
100
- > ```
101
- > verity login
102
- > ```
103
- >
104
- > It prints a code and a link; approve it in your browser. One login covers every
105
- > repository you can write to, for 90 days. Without it the gate still reviews
106
- > your code — it just records nothing, so there is no history, no cloud memory
107
- > and no dashboard.
194
+ ```bash
195
+ verity telemetry install
196
+ ```
197
+
198
+ It takes effect on their next Claude Code session.
108
199
 
109
- If `verity status` already reports an account, skip this step and say so.
200
+ ## Step 5: Report what happened
110
201
 
111
- If Step 0 found a version mismatch, the `verity` in their terminal is the old
112
- global one, not the plugin's: tell them to update it first (the `npm install -g`
113
- command from Step 0), then log in. An older or experimental build can sign the
114
- machine in to a different server, and the plugin follows that sign-in.
202
+ ```bash
203
+ verity doctor --json
204
+ ```
115
205
 
116
- ## Step 4: Offer the part a lookup cannot produce
206
+ Summarise from this report, not from memory of what you ran:
117
207
 
118
- The Standard `init` derives covers the research-backed patterns for the
119
- languages and tools it found. What it deliberately leaves empty is
120
- `custom_patterns` — rules like *"every route under `api/` uses the auth
121
- middleware"*, which need something to read the code for intent.
208
+ - what init removed (superseded skills, duplicate hooks) — those skill
209
+ directories are usually committed, so the deletions show in `git status` and
210
+ are the user's to commit;
211
+ - whether the Standard was **adopted** from the service or **derived** from the
212
+ codebase (init's output says which);
213
+ - the moments and intensity now in effect, and who wires the hooks
214
+ (`hooks.source`);
215
+ - anything untracked, and that the staged removal is theirs to commit;
216
+ - sign-in and telemetry state;
217
+ - each line of `next`, if any.
122
218
 
123
- Offer `/verity-setup` for that, and say it is optional: the gate is already
124
- running.
219
+ Then offer `/verity-setup` for the part a lookup cannot produce — project-specific
220
+ `custom_patterns` such as *"every route under `api/` uses the auth middleware"* —
221
+ and say it is optional: the gate is already running.
125
222
 
126
223
  ---
127
224
 
128
225
  ## What NOT to do here
129
226
 
130
227
  - **Do not wire hooks or edit `.gitignore` yourself.** `verity init` owns that.
131
- Doing it here is how the two flows drifted apart before — the skill reconciled
132
- hooks init had just installed and silently removed them.
133
- - **Do not synthesize a Standard by hand.** The CLI derives it in about two
134
- seconds from the same catalogue you would be reading.
135
- - **Do not run `verity login`.** Step 3.
228
+ Doing it here is how the two flows drifted apart before.
229
+ - **Do not synthesize a Standard by hand.** The CLI derives it in seconds from
230
+ the same catalogue you would be reading.
231
+ - **Do not answer a question for the user**, including by leaving a flag off
232
+ because it seemed obvious. An omitted question is an unattended answer.
233
+ - **Do not run plain `verity login`.** Step 3.
@@ -1,174 +1,135 @@
1
1
  ---
2
2
  description: >-
3
- Formally verify a small, risky piece of code with a Lean 4 model: pick candidates from git
4
- history, CONFIRM with the user which code to model, write the model, run it locally with
5
- `verity prove check`, and replay any counterexample on the real code before calling it a bug.
6
- Use when the user asks to verify, prove, or model-check code, or asks "can this race / double
7
- run / fail open?". Anything typed after /verity-prove is the user's hint: the file, function,
8
- or worry to check.
3
+ Formally verify a small, risky piece of code with a Lean 4 model written by the
4
+ Verity service: rank candidates from git history and the user's hint, CONFIRM with
5
+ the user which code to model, have the service generate the model, check it locally
6
+ with `verity prove check`, and replay any counterexample on the real code before
7
+ calling it a bug. Use when the user asks to verify, prove, or model-check code, or
8
+ asks "can this race / double run / fail open?". Anything typed after /verity-prove
9
+ is the user's hint: the file, function, or worry to check.
9
10
  ---
10
11
  # /verity-prove — Formal verification of small pieces of code
11
12
 
12
13
  **The words after `/verity-prove` are the hint.** `/verity-prove` alone means "find me
13
14
  something worth checking"; `/verity-prove can the Stripe webhook double-charge on a
14
- redelivered event?` means that is the target and, if it is a worry, the property.
15
- Carry it through every step below as `$ARGUMENTS`.
15
+ redelivered event?` names the target and the worry. Carry it through every step as
16
+ `$ARGUMENTS`.
16
17
 
17
- Verity keeps **Lean 4 models of the riskiest code**, not of the codebase. A model is
18
- another tool for finding bugs: a few dozen lines that abstract ONE function and check
19
- a few properties by exhaustive bounded search. When a property fails you get the
20
- **shortest sequence of actions** that breaks it — a hypothesis to replay on the real
21
- code. When it holds you know exactly how far it was checked.
18
+ Verity keeps **Lean 4 models of the riskiest code**, not of the codebase: a few dozen
19
+ lines that abstract ONE function and check a few properties by exhaustive bounded
20
+ search. When a property fails you get the **shortest sequence of actions** that breaks
21
+ it — a hypothesis to replay on the real code.
22
22
 
23
- This skill is the **explicit, user-driven** entry; nothing in it blocks a commit or a
24
- push. (The push-time phase — `verity config formal push` — is a separate, opt-in path
25
- that uses the same tools.) Lean runs on this machine because the user asked for it.
23
+ **The service does the work; you orchestrate and present.** Ranking, writing the model
24
+ and replaying its counterexample all run on the Verity service through the CLI. You
25
+ do not search the codebase by hand, read the Lean library, or write Lean yourself
26
+ unless a step below says so. Each step is one command; read its output, decide, move on.
27
+ Lean runs on this machine only because the user asked for it; nothing here blocks a
28
+ commit or a push.
26
29
 
27
30
  ## The flow
28
31
 
29
- ### 1. Find what deserves a model
32
+ ### 1. Candidates — one command
30
33
 
31
34
  ```bash
32
- verity prove candidates --limit 8 # git history + risk signals
33
- verity prove candidates --rank luna --json # + a cheap model re-ranks and names a symbol
34
- verity prove candidates src/payments # scoped to a path
35
+ verity prove candidates --hint "$ARGUMENTS" --json # hint given
36
+ verity prove candidates --limit 8 --json # no hint
35
37
  ```
36
38
 
37
- Each candidate says why: fix commits over commits, check-then-act, retries, caches,
38
- locks/leases, status transitions, money.
39
+ The hint **widens the pool** (files it names by path or content come first) and the
40
+ service ranks it, naming a symbol and the properties worth checking per file. Do not
41
+ search for the code yourself: if the ranker's list does not contain what the user
42
+ meant, say so and ask them for the file or function. A named file or function in the
43
+ hint skips ranking: go straight to step 2 with it.
39
44
 
40
- **With a hint** (`$ARGUMENTS` non-empty) — a file, a function, or a worry in the user's
41
- own words ("can the webhook double-charge?", "does the retry ever lose the feedback?"):
45
+ ### 2. Confirm with the user — always
42
46
 
43
- ```bash
44
- verity prove candidates --hint "$ARGUMENTS"
45
- ```
47
+ **Do not generate a model until the user has said which code.** Present at most three
48
+ candidates in one short message: file, the symbol the ranker named, the first property
49
+ it suggested, the reason. Ask. Their answer is the scope; do not widen it. If the hint
50
+ is a worry in the user's words, it becomes an `intent` property — that needs no further
51
+ confirmation, they just gave it.
46
52
 
47
- A named file or function skips the ranking entirely: go to step 2 with it. A *worry*
48
- becomes an `intent` property in step 4, with `source := "user, <date>: <their words>"`;
49
- it needs no further confirmation, they just gave it. The candidates list is still shown,
50
- so the user sees what else the history says — but the hint decides.
53
+ A hint that names the code, or a candidate the user saw in an earlier session, is
54
+ **not** a confirmation of the exact symbol and property. Still ask, in one line:
55
+ "I'll model `<file>#<symbol>` for `<property>` — go ahead?" It costs the user one word.
51
56
 
52
- ### 2. Confirm with the user — always
57
+ ### 3. Set up once per repo
53
58
 
54
- **Do not write a model until the user has said which code.** Present at most three
55
- candidates in one short message, each as: file, the function you would model, the
56
- property you would check first, and the reason from the signals. Then ask. The user may
57
- pick one, name something else, or narrow the property. Their answer is the scope; do not
58
- widen it.
59
+ ```bash
60
+ verity prove init
61
+ ```
59
62
 
60
- A model is a claim about what the code *means*. Only the person who owns the code can
61
- say whether that claim is worth checking.
63
+ If Lean is missing, say what will be installed and where before running
64
+ `verity lean install` (official release, ≈800 MB download, 2.7 GB on disk, into
65
+ `~/.verity/lean`). Everything else needs only `verity login`.
62
66
 
63
- ### 3. Set up the project (once per repo)
67
+ ### 4. Generate the model — one command
64
68
 
65
69
  ```bash
66
- verity prove init
70
+ verity prove generate src/pay.ts#charge --hint "$ARGUMENTS"
71
+ verity prove generate src/handler.ts --lines 120-180 --hint "$ARGUMENTS" # a region
67
72
  ```
68
73
 
69
- Creates `.verity/proofs/` — a Lake project with Verity's library (`Verity.System`,
70
- `Verity.check`, a bounded explorer). Re-running refreshes the library and never touches
71
- `Models/`. If Lean is missing, install it only after telling the user what will be
72
- installed and where: `verity lean install` (official release, ≈800 MB download, 2.7 GB
73
- on disk, into `~/.verity/lean`). Ranking (`--hint`, `--rank luna`) and `verity prove
74
- replay` run on the Verity service with the user's login (`verity login`); without a
75
- login, ranking is deterministic and replay is by hand (step 6.2).
76
-
77
- ### 4. Write the model — `Models/<Name>.lean`
78
-
79
- Rules that keep models small and honest:
80
-
81
- - **One symbol.** The header names it: `Verity.model { name, abstracts := [{ file, symbol, sha256 }] }`.
82
- Get the hash with `verity prove hash <file>`. `verity prove check` reports drift when the file changes.
83
- When the code is a *region* rather than a named function (a branch inside a large
84
- handler), add `lines := "a-b"` (1-based, inclusive) — `verity prove replay` needs it
85
- to know which real lines must run.
86
- - **State = only what the property reads.** Actions = the operations that can interleave
87
- (a retry, a second call, a concurrent writer). Steps = what the code does, not what it should.
88
- - **Properties are tagged.** `kind := .structural` for the catalogue (idempotence, no lost
89
- update, status only moves forward, retries terminate, never fails open, bounded size,
90
- check-then-act is atomic, no double effect). `kind := .intent` **only** when a human
91
- wrote the expectation down — a doc comment, an ADR, a ticket, a Standard rule — and
92
- `source :=` cites it. Never invent intent.
93
- - **End with the checks.** `#eval Verity.model …` then one `#eval Verity.check …` per
94
- property (bounded exploration), `Verity.checkData` for input properties, `Verity.proved`
95
- only beside a real theorem. `sorry` fails the run.
96
- - Keep it under ~120 lines. If it wants more, the scope is wrong — go back to step 2.
97
-
98
- `Models/README.md` has the skeleton; `cli/tests/fixtures/proofs/SeedRunner.lean` in
99
- Verity's own repo is a worked example (the VRT-243 seed-duplication bug).
100
-
101
- ### 5. Check it
74
+ The service writes the model, the linter fences it, this machine elaborates it, with
75
+ up to **3** rounds on compiler errors — all inside the command. It prints the check:
76
+ per property `✓ holds to bound k`, `✗ COUNTEREXAMPLE` with the trace, or `? inconclusive`.
77
+ Only if it ends with "no model elaborated" do you write one by hand from
78
+ `.verity/proofs/Models/README.md` (one symbol, ≤ ~120 lines, `#eval Verity.model …`
79
+ header, one `#eval Verity.check …` per property, tagged `.structural` or `.intent` with
80
+ `source :=`); check it with `verity prove check <Name>`.
81
+
82
+ ### 5. Replay before you call it a bug — one command
102
83
 
103
84
  ```bash
104
- verity prove check <Name> # one model
105
- verity prove check # every model
106
- verity prove check <Name> --json # for scripts
85
+ verity prove replay <Name> # every counterexample
86
+ verity prove replay <Name> --property P2 # one
107
87
  ```
108
88
 
109
- Per property: `✓ holds to bound k`, `✓ proved`, `✗ COUNTEREXAMPLE` with the trace and
110
- the state after each step, or `? inconclusive` (state cap).
111
-
112
- **Hard caps — never loop:**
113
- - elaboration errors: at most **3** fix rounds, then stop and say the model could not be
114
- built;
115
- - a counterexample that **diverges** on replay (the model was wrong about the code): at
116
- most **2** model revisions, each naming what the replay showed was different; after
117
- that, stop and report "no faithful model found for `<symbol>`" — that is a valid result;
118
- - never weaken or delete a property to make it pass; never add a property to make
119
- something fail.
120
-
121
- A wrong model is expected some of the time. It costs a replay and is discarded; it never
122
- costs a false bug, because only a *reproduced* counterexample is reported (step 6).
123
-
124
- ### 6. Replay before you call it a bug
125
-
126
- A counterexample is a fact about the **model**. Before reporting a bug:
127
-
128
- 1. Run the replay in OpenAI's disposable container (network off, deleted afterwards):
129
-
130
- ```bash
131
- verity prove replay <Name> # every counterexample
132
- verity prove replay <Name> --property P2 # one
133
- ```
134
-
135
- It hands a model the trace, the Lean file and the real source — the abstracted
136
- file's content leaves the machine for the Verity service and OpenAI's container;
137
- only git-tracked files are ever read — and requires three things before accepting
138
- its answer: code actually ran (a shell call with output), the real function's lines
139
- appear verbatim in what it wrote, and the harness itself printed
140
- `REPLAY_RESULT=reproduced`. `REPRODUCED` → the
141
- real code does this. `DIVERGED` → the model is wrong about the code. `UNVERIFIED` →
142
- not evidence either way. `UNREPLAYABLE` → it could not run the function faithfully.
143
- Read its reason: the stubs it chose are where a false reproduction hides.
144
- 2. When the replay is unverified or unreplayable — or the code needs a runtime the
145
- container lacks (Deno edge functions, a real database) — replay by hand: read the
146
- real function again, map every trace action to a real call, and reproduce it with a
147
- throwaway harness in a scratch directory that loads the real module and stubs only
148
- I/O boundaries. **Never edit the user's source files to do this.**
149
- 3. Reproduced → report it as a bug, with the trace and the reproduction. Not reproduced
150
- → the model is wrong (`model-divergence`): say so, fix the model, do not report a bug.
151
-
152
- This is the rule Verity applies everywhere (memory: *replay formal counterexamples
153
- against changed code before blocking*). A model that finds a bug in itself is noise.
154
-
155
- ### 7. Report, in a few lines
156
-
157
- State what was modelled, what was checked to what bound, what was refuted and whether
158
- it replayed, and what remains `intent` awaiting the user's confirmation. **Never offer
159
- to commit the model** — `.verity/proofs/` is machine-local and ignores itself; models
160
- would only confuse reviewers. What reaches the team is the bug report and, when asked,
161
- the fix. Then record the outcome:
89
+ The service replays the trace on the real code in a network-less container and accepts
90
+ an answer only when code ran, the real function's lines appear verbatim in the harness,
91
+ and the harness itself printed `REPLAY_RESULT=reproduced`. A model's counterexamples
92
+ replay in parallel, so the command takes about as long as one replay (~30 s). **Run it
93
+ in the foreground and wait for it**: do not background it, do other work, or report
94
+ before every verdict is in. Read the verdict:
95
+
96
+ - **REPRODUCED** — the real code does this. **This is final: do not replay by hand
97
+ as well.** Report it as a bug (step 6).
98
+ - **DIVERGED** — the model is wrong about the code. Revise it **at most twice**, each
99
+ time saying what the replay showed:
100
+ `verity prove generate <target> --revise <Name> --note "<what the replay showed>"`,
101
+ then replay again. After two divergences, report "no faithful model found for
102
+ `<symbol>`" — that is a valid result, not a bug.
103
+ - **UNVERIFIED / UNREPLAYABLE** — not evidence either way. Only now replay by hand:
104
+ a throwaway harness in a scratch directory that loads the real module and stubs only
105
+ I/O boundaries **in memory**. Never edit the user's source files, and never read from
106
+ or write to the user's databases, services or local stacks to do this — ask first if
107
+ a stub cannot be faithful.
108
+
109
+ Never weaken or delete a property to get green; never add one to make something fail.
110
+
111
+ ### 6. Report, in a few lines
112
+
113
+ What was modelled, what was checked to what bound, what was refuted and whether it
114
+ replayed, and what remains `intent` awaiting the user's confirmation. **Never offer to
115
+ commit the model** — `.verity/proofs/` is machine-local and ignores itself. What reaches
116
+ the team is the bug report and, when asked, the fix. Then record the outcome:
162
117
 
163
118
  ```bash
164
119
  verity reflect --user-input "Modelled runSeed (cli/src/lib/seed-runner.ts): P1 at-most-once refuted in 2 steps, replayed — guard checks a path the server never writes." --kind gotcha
165
120
  ```
166
121
 
122
+ The abstracted file's content leaves the machine for the Verity service and its
123
+ replay container; only git-tracked files are ever read.
124
+
167
125
  ## Do not
168
126
 
127
+ - Search the codebase yourself before the ranker has answered, or instead of asking the user.
169
128
  - Model code the user did not confirm.
170
- - Present a model-only counterexample as a bug.
171
- - Put `sorry` in a model, or weaken a property to get green.
172
- - Touch source files, run Lean from a hook, or install anything silently.
129
+ - Write Lean by hand while `verity prove generate` can still try.
130
+ - Replay by hand after a REPRODUCED verdict, or present a model-only counterexample as a bug.
131
+ - Background `verity prove replay`, or report before its verdicts are in.
132
+ - Write to the user's databases or local services, even throwaway rows.
133
+ - Put `sorry` in a model, weaken a property to get green, or touch source files.
134
+ - Run Lean from a hook, or install anything silently.
173
135
  - Commit, stage, or un-ignore anything under `.verity/proofs/`.
174
- - Model a whole module. If the state has more than ~6 fields, narrow it.