fv-skills-baif 2.3.0 → 2.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +17 -0
- package/README.md +24 -10
- package/commands/fvs/crypto-followup.md +31 -6
- package/commands/fvs/crypto-plan.md +38 -6
- package/commands/fvs/crypto-review.md +70 -127
- package/commands/fvs/formalise.md +4 -3
- package/commands/fvs/help.md +21 -12
- package/commands/fvs/lean-spec-review.md +12 -0
- package/commands/fvs/lean-specify.md +21 -6
- package/fv-skills/VERSION +1 -1
- package/fv-skills/references/crypto-plan-review.md +18 -3
- package/fv-skills/references/fc-spec-review.md +14 -5
- package/fv-skills/templates/config.json +3 -0
- package/fv-skills/workflows/crypto-followup.md +29 -4
- package/fv-skills/workflows/crypto-plan.md +26 -2
- package/fv-skills/workflows/crypto-review.md +42 -63
- package/fv-skills/workflows/lean-spec-review.md +21 -11
- package/fv-skills/workflows/lean-specify.md +21 -7
- package/package.json +1 -1
- package/scripts/fvs-codex-think.mjs +289 -197
- package/scripts/fvs-spec-review.mjs +126 -66
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,23 @@ All notable changes to FVS (Formal Verification Skills) will be documented in th
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
|
6
6
|
|
|
7
|
+
## [2.3.1] - 2026-09-09
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- Crypto plan and follow-up review now support selected Codex, Claude, or external reviewers with
|
|
11
|
+
explicit model/effort choices, read-only execution, honest provenance, immutable hash-bound
|
|
12
|
+
packets, and a three-round author/reviewer loop. Automatic handoff stays interactive, while
|
|
13
|
+
one-run Skip and `crypto_review.automatic: false` retain an explicit unreviewed trusted path.
|
|
14
|
+
This resolves
|
|
15
|
+
[GitHub issue #51](https://github.com/Beneficial-AI-Foundation/formal-verification-skills/issues/51).
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
- Crypto and functional-correctness review now treat `APPROVE-WITH-EDITS` as terminal once the
|
|
19
|
+
authoring seat applies the exhaustively named bounded edits and reruns its gates. Reviewer output
|
|
20
|
+
remains immutable; only true `REJECT`, `REVISE`, or `BLOCKED` verdicts start a fresh review.
|
|
21
|
+
This resolves
|
|
22
|
+
[GitHub issue #52](https://github.com/Beneficial-AI-Foundation/formal-verification-skills/issues/52).
|
|
23
|
+
|
|
7
24
|
## [2.3.0] - 2026-09-07
|
|
8
25
|
|
|
9
26
|
### Added
|
package/README.md
CHANGED
|
@@ -172,10 +172,17 @@ Commands are grouped into five bundles. Each bundle has a **router** command tha
|
|
|
172
172
|
rejects ordinary Lean identifiers with three or more namespace dots, steering generated code
|
|
173
173
|
toward scoped namespaces, `open`, and local names.
|
|
174
174
|
|
|
175
|
-
After `lean-specify`,
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
and
|
|
175
|
+
After `lean-specify`, an interactive review menu asks reviewer, then model, then effort; it never
|
|
176
|
+
auto-selects a choice. It offers the other runtime first, a fresh reviewer in the current runtime,
|
|
177
|
+
or another provider. Choose GPT Sol or Astra for Codex, Fable for Claude, or a cheaper/custom model;
|
|
178
|
+
effort defaults to `max` and can be lowered. Other providers use an exported source packet and
|
|
179
|
+
imported response. One-run `Skip review` records `Unreviewed (user skipped)` and does not begin
|
|
180
|
+
proof work. Reviews and source hashes live under `.formalising/spec-reviews/`.
|
|
181
|
+
|
|
182
|
+
The reviewer stays read-only and `review.md` stays immutable. The `lean-specify` authoring seat
|
|
183
|
+
writes separate `triage.md`. PASS proceeds; APPROVE-WITH-EDITS also proceeds after accepted bounded
|
|
184
|
+
edits pass the structure/style/build gates, with no redundant second review. Only REVISE/BLOCKED
|
|
185
|
+
starts another fresh review, carrying prior records as untrusted history, with a three-round cap.
|
|
179
186
|
|
|
180
187
|
Automatic review works even without a config file. To disable the automatic menu, merge this
|
|
181
188
|
setting into `.formalising/fvs-config.json` (or create that file with just this object):
|
|
@@ -216,7 +223,7 @@ secrets, raw transcripts, ephemeral error dumps, unsupported guesses, or inferre
|
|
|
216
223
|
| `/fvs:lean-formalise` | Formalise paper/math content into Lean 4 specs and definitions (one-shot) |
|
|
217
224
|
| `/fvs:lean-refactor` | Refactor, simplify, and decompose verified proofs — *also in Formal-Core* |
|
|
218
225
|
| `/fvs:crypto-plan` | Author the next bounded, runtime-neutral plan for a topic-based crypto formalisation iteration (KB-grounded, cached under `sources/`) |
|
|
219
|
-
| `/fvs:crypto-review` |
|
|
226
|
+
| `/fvs:crypto-review` | Review a crypto plan with selected Codex, Claude, or Other runtime/model/effort |
|
|
220
227
|
| `/fvs:crypto-execute` | Run the current iteration's bounded plan under the green-build guard |
|
|
221
228
|
| `/fvs:crypto-eval` | Adversarially evaluate the iteration; ends in one of ACCEPT / FOLLOWUP / HUMAN_RULING / BLOCKED |
|
|
222
229
|
| `/fvs:crypto-followup` | Convert eval findings into the next follow-up plan; HALTs on HUMAN_RULING |
|
|
@@ -274,12 +281,12 @@ The paper track formalises papers directly into Lean 4 — no Rust, no Aeneas. T
|
|
|
274
281
|
|
|
275
282
|
- **One-shot:** `/fvs:lean-formalise` reads your PDFs / images / LaTeX (optionally grounded in a NotebookLM knowledge base via `/fvs:kb-setup`) and produces Lean definition and spec files in a single pass.
|
|
276
283
|
- **Iterative crypto loop:** for larger crypto formalisations, a topic-based, restartable loop with
|
|
277
|
-
|
|
284
|
+
a fresh pre-execution review gate:
|
|
278
285
|
|
|
279
286
|
`/fvs:crypto-plan` → `/fvs:crypto-review` → `/fvs:crypto-execute` → `/fvs:crypto-eval` → `/fvs:crypto-followup` → `/fvs:crypto-review` → repeat
|
|
280
287
|
|
|
281
|
-
A high-effort thinker authors each bounded plan. Before execution,
|
|
282
|
-
|
|
288
|
+
A high-effort thinker authors each bounded plan. Before execution, a selected Codex, Claude, or
|
|
289
|
+
Other fresh reviewer attacks the plan or follow-up under read-only controls and returns an
|
|
283
290
|
evidence-backed APPROVE / APPROVE-WITH-EDITS / REJECT verdict. The executor runs an approved plan
|
|
284
291
|
under a green-build guard; the post-execution adversarial eval tries to refute the spec, proof,
|
|
285
292
|
and assumptions and ends in exactly one of ACCEPT / FOLLOWUP / HUMAN_RULING / BLOCKED. Follow-up
|
|
@@ -289,8 +296,15 @@ The paper track formalises papers directly into Lean 4 — no Rust, no Aeneas. T
|
|
|
289
296
|
The authoring, execution, eval, and follow-up stages use the lightweight proof-engineering overlay:
|
|
290
297
|
they load at most eight relevant `crypto`/`shared` lessons and propose at most three reviewed
|
|
291
298
|
updates. Modeling lessons require paper or standard citations and remain provisional until an
|
|
292
|
-
accepted adversarial eval or explicit human ruling. The
|
|
293
|
-
|
|
299
|
+
accepted adversarial eval or explicit human ruling. The `crypto-review` gate is deliberately
|
|
300
|
+
memory-blind. It labels cross-runtime, same-runtime fresh reviewer, and unverified provenance;
|
|
301
|
+
Other uses packet export/import. APPROVE-WITH-EDITS becomes terminal after the authoring seat
|
|
302
|
+
applies accepted edits and reruns gates, with no second review. REJECT starts a fresh reviewed
|
|
303
|
+
revision with prior history, capped at three reviewer rounds per invocation. Automatic plan and
|
|
304
|
+
follow-up handoff asks reviewer -> model -> effort and never auto-selects. One-run `Skip review`
|
|
305
|
+
records `Unreviewed (user skipped)` and never starts execution. To opt out persistently, merge
|
|
306
|
+
`"crypto_review": {"automatic": false}` into `.formalising/fvs-config.json`; standalone review
|
|
307
|
+
and explicit trusted-path execution remain available without fabricating approval.
|
|
294
308
|
|
|
295
309
|
**Single- vs dual-runtime (`--codex`).** By default the loop is single-runtime — the thinking stages (`crypto-plan`, `crypto-eval`, `crypto-followup`) run the in-runtime `fvs-crypto-thinker`. Pass `--codex` to hand a stage's thinking to an independent **Codex CLI** thinker instead, so the adversarial planner/evaluator runs on a *different engine* than the executor and blind spots don't correlate. `crypto-execute` is the runtime-neutral executor and takes no `--codex`. Without the Codex CLI installed, a `--codex` stage stops with an install hint rather than silently falling back.
|
|
296
310
|
|
|
@@ -185,8 +185,9 @@ The artifact MUST also record:
|
|
|
185
185
|
Authoring runtime: {Claude Code | OpenCode | Gemini | Codex | Codex CLI}
|
|
186
186
|
```
|
|
187
187
|
|
|
188
|
-
Use `Codex CLI` for `--codex`; otherwise name the actual host runtime.
|
|
189
|
-
|
|
188
|
+
Use `Codex CLI` for `--codex`; otherwise name the actual host runtime. `/fvs:crypto-review` uses
|
|
189
|
+
this provenance to label cross-runtime review as independent and same-runtime review as fresh but
|
|
190
|
+
not independent; missing provenance fails closed.
|
|
190
191
|
|
|
191
192
|
## Step 4a: Reconcile Follow-Up Lessons
|
|
192
193
|
|
|
@@ -196,6 +197,31 @@ required. Strengthen an equivalent record or create one file per new lesson unde
|
|
|
196
197
|
`lessons/crypto/`, updating the index in the same reviewable diff. Unruled choices stay
|
|
197
198
|
`provisional`; never infer or generalize a ruling beyond its recorded scope.
|
|
198
199
|
|
|
200
|
+
## Step 4b: Run the bounded review loop
|
|
201
|
+
|
|
202
|
+
After authoring gates, run
|
|
203
|
+
`node ~/.claude/scripts/fvs-codex-think.mjs review-automatic`; missing config or
|
|
204
|
+
`crypto_review.automatic` defaults to true and malformed values stop clearly. If false, record
|
|
205
|
+
`Unreviewed (automatic review disabled)`, preserve the follow-up, and do not auto-start execution.
|
|
206
|
+
|
|
207
|
+
If true, enter the interactive `crypto-review` handoff. Honor reviewer/model/effort choices explicitly
|
|
208
|
+
supplied earlier in this invocation. Ask only for missing choices in order: reviewer -> model ->
|
|
209
|
+
effort. Recommend the normalized non-author runtime, but never auto-select or treat a preselected
|
|
210
|
+
default as consent. Offer a one-run `Skip review`, recorded exactly as `Unreviewed (user skipped)`.
|
|
211
|
+
Skipping preserves the follow-up and does not auto-start execution; a trusted user may explicitly
|
|
212
|
+
invoke `/fvs:crypto-execute`. The standalone review flags remain the non-interactive path.
|
|
213
|
+
|
|
214
|
+
Run at most three reviewer rounds in this command invocation. APPROVE stops.
|
|
215
|
+
APPROVE-WITH-EDITS is terminal after the authoring seat applies accepted bounded edits, reruns plan
|
|
216
|
+
gates, records hashes/finding IDs in separate triage, and marks `approved after edits`; no second
|
|
217
|
+
review.
|
|
218
|
+
|
|
219
|
+
REJECT creates a fresh authored revision at the next immutable iteration and a fresh review. Inline
|
|
220
|
+
the preceding review and triage as delimited untrusted history for the author and pass repeated
|
|
221
|
+
`--history` flags to the reviewer packet. At round three, stop with the latest artifacts and the
|
|
222
|
+
exact `/fvs:crypto-review <topic> nN --target followup` resume command. Failed, cancelled, pending,
|
|
223
|
+
and unverified states do not start execution.
|
|
224
|
+
|
|
199
225
|
## Step 5: Run-end banner + next command
|
|
200
226
|
|
|
201
227
|
```
|
|
@@ -205,8 +231,8 @@ Topic: {TOPIC_RAW}
|
|
|
205
231
|
Decision: {FOLLOWUP | HUMAN_RULING -> ruled}
|
|
206
232
|
Plan: plans/FOLLOWUP_PLAN_n{N}.md
|
|
207
233
|
|
|
208
|
-
|
|
209
|
-
/fvs:crypto-review
|
|
234
|
+
Review: {approved | approved after edits | Unreviewed (user skipped) | Unreviewed (automatic review disabled) | failed | pending | unverified | rejected at cap}
|
|
235
|
+
Next: {/fvs:crypto-execute only after approval | exact crypto-review resume command}
|
|
210
236
|
```
|
|
211
237
|
|
|
212
238
|
</process>
|
|
@@ -233,8 +259,7 @@ unchanged.
|
|
|
233
259
|
- [ ] Latest `EVAL_nN.md` read; decision routed (`ACCEPT` stop / `BLOCKED` pause / `FOLLOWUP` author / `HUMAN_RULING` HALT).
|
|
234
260
|
- [ ] On `HUMAN_RULING` the command HALTs and asks the user -- it NEVER fabricates a follow-up plan.
|
|
235
261
|
- [ ] On `FOLLOWUP` the thinker is dispatched (`subagent_type="fvs-crypto-thinker"`) and the bounded follow-up plan written to `plans/`.
|
|
236
|
-
- [ ] The follow-up records truthful
|
|
237
|
-
`/fvs:crypto-review --target followup`.
|
|
262
|
+
- [ ] The follow-up records truthful provenance and runs at most three review rounds before stop.
|
|
238
263
|
- [ ] At most three source/ruling-evidenced candidates reconciled as one file each plus index updates.
|
|
239
264
|
- [ ] No bare `lake build`, no `gh` open/create, no generated-Lean write.
|
|
240
265
|
</success_criteria>
|
|
@@ -205,8 +205,8 @@ Authoring runtime: {Claude Code | OpenCode | Gemini | Codex | Codex CLI}
|
|
|
205
205
|
```
|
|
206
206
|
|
|
207
207
|
Use `Codex CLI` when `--codex` authored the plan; otherwise name the actual host runtime. Never
|
|
208
|
-
write a generic or guessed marker. `/fvs:crypto-review` uses it to
|
|
209
|
-
|
|
208
|
+
write a generic or guessed marker. `/fvs:crypto-review` uses it to label cross-runtime review as
|
|
209
|
+
independent and same-runtime review as fresh but not independent; missing provenance fails closed.
|
|
210
210
|
|
|
211
211
|
Carry the BOUNDED-PLAN CONTRACT verbatim into `EXEC_PLAN_nN.md`:
|
|
212
212
|
1. **Branch and current state** -- the branch name and what already compiles / is proven.
|
|
@@ -229,6 +229,38 @@ accepted adversarial eval or explicit human ruling validates them. Strengthen an
|
|
|
229
229
|
or create one `lessons/crypto/<date>-<slug>.md` file per new lesson and update the index in the same
|
|
230
230
|
reviewable diff. Never persist uncited claims, raw transcripts, full error dumps, or secrets.
|
|
231
231
|
|
|
232
|
+
## Step 5b: Run the bounded review loop
|
|
233
|
+
|
|
234
|
+
Read the automatic-review setting after the plan gates pass; missing config or
|
|
235
|
+
`crypto_review.automatic` defaults to true, while malformed values stop clearly:
|
|
236
|
+
|
|
237
|
+
```bash
|
|
238
|
+
AUTOMATIC_REVIEW=$(node ~/.claude/scripts/fvs-codex-think.mjs review-automatic) || exit 1
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
If true, automatically enter the interactive `crypto-review` handoff. Honor reviewer/model/effort
|
|
242
|
+
choices explicitly supplied earlier in this invocation, then ask only for missing choices in order:
|
|
243
|
+
reviewer -> model -> effort. Recommend the normalized non-author runtime. Never auto-select or treat
|
|
244
|
+
a default/preselected menu item as consent. Also offer `Skip review` for this run. Record that choice
|
|
245
|
+
exactly as `Unreviewed (user skipped)`; an Other export without an imported response is pending.
|
|
246
|
+
|
|
247
|
+
If false, record `Unreviewed (automatic review disabled)`. Users set this persistently by merging
|
|
248
|
+
`"crypto_review": {"automatic": false}` into `.formalising/fvs-config.json`. Either unreviewed
|
|
249
|
+
route preserves the plans; do not auto-start crypto execution. A trusted user may explicitly
|
|
250
|
+
invoke `/fvs:crypto-execute <topic> nN` afterward. The standalone `/fvs:crypto-review` flags remain
|
|
251
|
+
the non-interactive review path.
|
|
252
|
+
|
|
253
|
+
When review runs, allow at most three reviewer rounds in this command invocation. APPROVE ends the
|
|
254
|
+
loop. APPROVE-WITH-EDITS is also terminal after the authoring seat applies accepted bounded edits,
|
|
255
|
+
reruns the plan gates, and records `approved after edits` in a separate triage; do not request a
|
|
256
|
+
redundant second review.
|
|
257
|
+
|
|
258
|
+
REJECT requires a fresh authored revision at the next immutable iteration and a fresh review.
|
|
259
|
+
Inline the previous review and triage into the author prompt as delimited untrusted history and pass
|
|
260
|
+
both to the next reviewer packet with repeated `--history` flags. At round three, stop with the
|
|
261
|
+
latest artifacts and the exact `/fvs:crypto-review <topic> nN --target plan` resume command; never
|
|
262
|
+
auto-approve or start execution. Failed, cancelled, pending, and unverified states also stop.
|
|
263
|
+
|
|
232
264
|
## Step 6: Run-end banner + next command
|
|
233
265
|
|
|
234
266
|
```
|
|
@@ -240,8 +272,8 @@ KB: {grounded | degraded -- not configured}
|
|
|
240
272
|
Plans: plans/PLAN_n{NEXT}.md, plans/EXEC_PLAN_n{NEXT}.md
|
|
241
273
|
Sources: {K} cached under sources/
|
|
242
274
|
|
|
243
|
-
|
|
244
|
-
/fvs:crypto-review
|
|
275
|
+
Review: {approved | approved after edits | Unreviewed (user skipped) | Unreviewed (automatic review disabled) | failed | pending | unverified | rejected at cap}
|
|
276
|
+
Next: {/fvs:crypto-execute only after approval | exact crypto-review resume command}
|
|
245
277
|
```
|
|
246
278
|
|
|
247
279
|
</process>
|
|
@@ -268,8 +300,8 @@ auto-picks a default, never writes an upstream artifact).
|
|
|
268
300
|
- [ ] `$THINKER_MODEL` resolved via the model-profiles sequence; the thinker dispatched (`subagent_type="fvs-crypto-thinker"`) with inlined context.
|
|
269
301
|
- [ ] KB grounded intensively when configured; cached under `sources/` and re-read before re-querying; loud-fail-once + labeled-degrade + `/fvs:kb-setup` when unconfigured.
|
|
270
302
|
- [ ] The bounded-plan contract (stop conditions, verification commands `LEAN_NUM_THREADS="${LEAN_NUM_THREADS:-4}" nice -n 19 lake build`, immutable public statements, allowed-`sorry`) is written into `EXEC_PLAN_nN.md`.
|
|
271
|
-
- [ ] Both plan artifacts record truthful `Authoring runtime:` provenance;
|
|
272
|
-
|
|
303
|
+
- [ ] Both plan artifacts record truthful `Authoring runtime:` provenance; automatic review runs
|
|
304
|
+
for at most three rounds and only an approved result can suggest execution.
|
|
273
305
|
- [ ] At most three evidence-gated lesson candidates reconciled as one file each plus index updates.
|
|
274
306
|
- [ ] No bare `lake build`, no `gh` open/create, no generated-Lean write.
|
|
275
307
|
</success_criteria>
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: fvs:crypto-review
|
|
3
|
-
description:
|
|
4
|
-
argument-hint: "<topic> [nN] [--target plan|followup]"
|
|
3
|
+
description: Adversarially review a crypto plan with a chosen runtime, model, and effort
|
|
4
|
+
argument-hint: "<topic> [nN] [--target plan|followup] [--reviewer codex|claude|other] [--model ID] [--effort LEVEL]"
|
|
5
5
|
allowed-tools:
|
|
6
6
|
- Read
|
|
7
7
|
- Bash
|
|
@@ -9,20 +9,14 @@ allowed-tools:
|
|
|
9
9
|
- Grep
|
|
10
10
|
- Write
|
|
11
11
|
- Edit
|
|
12
|
+
- AskUserQuestion
|
|
13
|
+
- Task
|
|
12
14
|
---
|
|
13
15
|
|
|
14
16
|
<objective>
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
primary planning seat verify and triage every finding.
|
|
19
|
-
|
|
20
|
-
This is not the post-execution `/fvs:crypto-eval` stage. It attacks the PLAN before an executor
|
|
21
|
-
spends effort. Codex is the independent reviewer; it never authors or edits the plan.
|
|
22
|
-
|
|
23
|
-
This gate is deliberately proof-engineering-memory-blind. Do not load
|
|
24
|
-
`.formalising/proof-engineering/` or the topic's `sources/proof-engineering-context.md` snapshot into
|
|
25
|
-
the reviewer: independence includes re-challenging assumptions without inherited lesson framing.
|
|
17
|
+
Run a fresh, read-only adversarial review before crypto execution. The reviewer returns evidence;
|
|
18
|
+
the distinct planning/authoring seat owns triage and any plan edits. Preserve every review and
|
|
19
|
+
triage record, and never load proof-engineering memory into the reviewer.
|
|
26
20
|
</objective>
|
|
27
21
|
|
|
28
22
|
<execution_context>
|
|
@@ -31,45 +25,14 @@ the reviewer: independence includes re-challenging assumptions without inherited
|
|
|
31
25
|
@~/.claude/fv-skills/references/ui-brand.md
|
|
32
26
|
</execution_context>
|
|
33
27
|
|
|
34
|
-
<context>
|
|
35
|
-
Topic and optional iteration/target: $ARGUMENTS.
|
|
36
|
-
|
|
37
|
-
Default target selection is `followup` when `FOLLOWUP_PLAN_nN.md` exists, otherwise `plan`.
|
|
38
|
-
The optional `--target` makes that choice explicit.
|
|
39
|
-
</context>
|
|
28
|
+
<context>Topic, iteration, target, and reviewer options: $ARGUMENTS.</context>
|
|
40
29
|
|
|
41
30
|
<process>
|
|
42
31
|
|
|
43
|
-
##
|
|
44
|
-
|
|
45
|
-
Before reading plan contents or doing any later work, verify that the Codex CLI is installed and
|
|
46
|
-
signed in:
|
|
47
|
-
|
|
48
|
-
```bash
|
|
49
|
-
command -v codex >/dev/null 2>&1 \
|
|
50
|
-
&& codex login status >/dev/null 2>&1 \
|
|
51
|
-
&& echo "CODEX_OK" \
|
|
52
|
-
|| echo "CODEX_NOT_READY"
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
If the result is `CODEX_NOT_READY`, STOP:
|
|
56
|
-
|
|
57
|
-
```
|
|
58
|
-
FVS >> CODEX ISN'T READY
|
|
59
|
-
|
|
60
|
-
This review needs the OpenAI Codex CLI installed and signed in.
|
|
61
|
-
1. Install: npm install -g @openai/codex
|
|
62
|
-
2. Sign in: codex login
|
|
63
|
-
3. Verify: codex login status
|
|
32
|
+
## 1. Resolve target and reviewer choices
|
|
64
33
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
## Step 1: Resolve topic, iteration, and target safely
|
|
70
|
-
|
|
71
|
-
Treat all arguments as untrusted. Collapse topic whitespace to `-`, preserve meaningful
|
|
72
|
-
capitalization, reject shell metacharacters, `..`, and `/`, quote every path, and never `eval`.
|
|
34
|
+
Treat arguments as untrusted. Collapse topic whitespace to `-`, preserve capitalization, reject
|
|
35
|
+
shell metacharacters, `..`, and `/`, quote every path, and never `eval`:
|
|
73
36
|
|
|
74
37
|
```bash
|
|
75
38
|
TOPIC_RAW="$1"
|
|
@@ -81,107 +44,87 @@ SLUG=$(printf '%s' "$TOPIC_RAW" | tr -s '[:space:]' '-')
|
|
|
81
44
|
ROOT=".formalising/fv-plans/$SLUG"
|
|
82
45
|
```
|
|
83
46
|
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
Resolve the target:
|
|
89
|
-
|
|
90
|
-
- `plan`: require both `plans/PLAN_nN.md` and `plans/EXEC_PLAN_nN.md`; output
|
|
91
|
-
`reviews/PLAN_REVIEW_nN.md`.
|
|
92
|
-
- `followup`: require `plans/FOLLOWUP_PLAN_nN.md`; also expose the matching eval and original plan
|
|
93
|
-
when present; output `reviews/FOLLOWUP_REVIEW_nN.md`.
|
|
94
|
-
- omitted/auto: choose `followup` when its file exists, otherwise `plan`.
|
|
95
|
-
|
|
96
|
-
Refuse to overwrite an existing output. Preserve prior review history and ask the user to choose a
|
|
97
|
-
new iteration or archive the old review deliberately.
|
|
47
|
+
Resolve numeric `nN` and `--target plan|followup`. Initial review consumes `PLAN_nN.md` plus
|
|
48
|
+
`EXEC_PLAN_nN.md` and owns `PLAN_REVIEW_nN.md`; follow-up consumes `FOLLOWUP_PLAN_nN.md` plus
|
|
49
|
+
available original-plan/eval context and owns `FOLLOWUP_REVIEW_nN.md`. Refuse an occupied output.
|
|
98
50
|
|
|
99
|
-
|
|
51
|
+
Normalize the target's `Authoring runtime:` marker to `codex`, `claude`, `other`, or `unknown`.
|
|
52
|
+
Missing, foreign, or conflicting markers are `unverified`; they do not block review, but never
|
|
53
|
+
claim independence. Label a different known runtime `cross-runtime`, an explicitly selected
|
|
54
|
+
matching runtime `same-runtime, fresh reviewer`, and Other `unverified`.
|
|
100
55
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
56
|
+
Honor explicit `--reviewer`, `--model`, and `--effort`. Ask only for missing choices in exactly
|
|
57
|
+
this order: reviewer -> model -> effort. Supplying all three flags is the standalone
|
|
58
|
+
non-interactive path; never replace an explicit choice.
|
|
104
59
|
|
|
105
|
-
|
|
106
|
-
|
|
60
|
+
1. Reviewer: recommend the normalized non-author runtime first. Offer `Codex`, `Claude`, and
|
|
61
|
+
`Other`; a same-runtime choice is opt-in.
|
|
62
|
+
2. Model: Codex offers `gpt-5.6-sol` then `gpt-6-astra` and custom; Claude offers `fable` then
|
|
63
|
+
`sonnet` and custom. Other requires the exact external model ID.
|
|
64
|
+
3. Effort: offer `max` first, then supported lower levels and `runtime-default`; offer Codex
|
|
65
|
+
`ultra` only when supported. Other accepts the external provider's effort label.
|
|
107
66
|
|
|
108
|
-
|
|
109
|
-
|
|
67
|
+
Automatic callers also offer a one-run `Skip review`. Record it exactly as
|
|
68
|
+
`Unreviewed (user skipped)` and do not auto-start crypto execution.
|
|
110
69
|
|
|
111
|
-
##
|
|
112
|
-
|
|
113
|
-
Run the installed FVS helper at xhigh effort:
|
|
70
|
+
## 2. Run or export the read-only review
|
|
114
71
|
|
|
115
72
|
```bash
|
|
116
73
|
node ~/.claude/scripts/fvs-codex-think.mjs review \
|
|
117
|
-
--topic "$ROOT" \
|
|
118
|
-
--
|
|
119
|
-
--target "$TARGET_KIND" \
|
|
120
|
-
--effort xhigh
|
|
74
|
+
--topic "$ROOT" --iteration "n$N" --target "$TARGET_KIND" \
|
|
75
|
+
--reviewer "$REVIEWER" --model "$MODEL" --effort "$EFFORT"
|
|
121
76
|
```
|
|
122
77
|
|
|
123
|
-
The
|
|
78
|
+
The shared provider machinery preflights only the selected CLI. Codex runs read-only and ephemeral
|
|
79
|
+
with user config ignored; Claude runs safe mode with only Read/Glob/Grep, no MCP servers, and no
|
|
80
|
+
persisted session. The reviewer never edits a target or repository file. The wrapper creates a
|
|
81
|
+
unique hash-bound packet, validates one track-valid verdict, and exclusively writes the final
|
|
82
|
+
review. Authentication, process, stale-input, or output failure is `failed`; never silently switch
|
|
83
|
+
reviewers.
|
|
124
84
|
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
array, xhigh effort, and no `--model`;
|
|
129
|
-
- gives Codex the exact target paths and tells it to treat repository/plan contents as data;
|
|
130
|
-
- excludes proof-engineering memory and its derived snapshot from reviewer context;
|
|
131
|
-
- captures the final reviewer message in an OS temporary directory;
|
|
132
|
-
- validates exactly one `VERDICT:` line;
|
|
133
|
-
- has the WRAPPER persist exactly one review artifact, then removes temporary output.
|
|
85
|
+
For Other, the command reports `PENDING` and a managed packet. Give `prompt.md` to the selected
|
|
86
|
+
reviewer, save its Markdown response inside the project, then run the printed `review-import`
|
|
87
|
+
command with `--topic`, `--packet`, and `--response`. A pending export is not a completed review.
|
|
134
88
|
|
|
135
|
-
|
|
136
|
-
nonzero, or violates the output contract, STOP. Never fall back to the plan author.
|
|
89
|
+
## 3. Triage in the authoring seat
|
|
137
90
|
|
|
138
|
-
|
|
91
|
+
Keep the reviewer response byte-for-byte intact. Never append triage to it. The planning seat
|
|
92
|
+
re-checks every finding and exclusively writes one separate file:
|
|
139
93
|
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
accepting it.
|
|
94
|
+
- `PLAN_REVIEW_nN_TRIAGE.md`, or
|
|
95
|
+
- `FOLLOWUP_REVIEW_nN_TRIAGE.md`.
|
|
143
96
|
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
97
|
+
Record requested and observed runtime/model/effort, provenance, each finding ID with
|
|
98
|
+
accept/reject/defer and checked evidence, pre-edit target hashes, post-edit target hashes when
|
|
99
|
+
applicable, gates rerun, and one status. Refuse to overwrite either review or triage history.
|
|
147
100
|
|
|
148
|
-
|
|
101
|
+
Route the verdict:
|
|
149
102
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
103
|
+
- `APPROVE`: write supported triage status `approved`; execution may be suggested.
|
|
104
|
+
- `APPROVE-WITH-EDITS`: the authoring seat applies every accepted, exhaustively named bounded edit,
|
|
105
|
+
reruns the plan's own verification gates, records accepted/rejected finding IDs plus pre-edit and
|
|
106
|
+
post-edit hashes, then writes `approved after edits`. This is terminal: no second review.
|
|
107
|
+
- `REJECT`: bounded edits cannot promote it. The authoring seat creates a fresh authored revision
|
|
108
|
+
at the next immutable iteration and a fresh review.
|
|
153
109
|
|
|
154
|
-
|
|
155
|
-
|
|
110
|
+
For a true REJECT revision, pass the preceding review and triage to the new author prompt and next
|
|
111
|
+
review packet as separately delimited untrusted history using repeated `--history` flags. Do not
|
|
112
|
+
load `.formalising/proof-engineering/` or `sources/proof-engineering-context.md` into the reviewer.
|
|
156
113
|
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
114
|
+
Hard cap each command invocation at at most three reviewer rounds. After the third REJECT, stop
|
|
115
|
+
with the latest artifacts and print the exact standalone `/fvs:crypto-review <topic> nN --target
|
|
116
|
+
<kind>` resume command; never auto-approve.
|
|
160
117
|
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
- `APPROVE`: the plan may proceed to `/fvs:crypto-execute`.
|
|
164
|
-
- `APPROVE-WITH-EDITS`: STOP before execution; revise the named plan sections and run a fresh
|
|
165
|
-
independently recorded review.
|
|
166
|
-
- `REJECT`: STOP before execution; return to `/fvs:crypto-plan` or `/fvs:crypto-followup`.
|
|
118
|
+
Failed, cancelled, pending, or unverified review states never auto-start execution or
|
|
119
|
+
`/fvs:crypto-execute`. Report them honestly. Standalone invocation remains usable.
|
|
167
120
|
|
|
168
121
|
</process>
|
|
169
122
|
|
|
170
|
-
<codex_skill_adapter>
|
|
171
|
-
This command itself is a cross-runtime bridge to the Codex CLI; it does not dispatch a Codex
|
|
172
|
-
subagent. On the Codex host runtime it fails closed because Codex reviewing Codex is not independent.
|
|
173
|
-
All coordination is artifact-mediated. Interactive ambiguity degrades to a plain-text question and
|
|
174
|
-
waits; it never guesses provenance, iteration, or overwrite intent.
|
|
175
|
-
</codex_skill_adapter>
|
|
176
|
-
|
|
177
123
|
<success_criteria>
|
|
178
|
-
- [ ]
|
|
179
|
-
- [ ]
|
|
180
|
-
- [ ]
|
|
181
|
-
- [ ]
|
|
182
|
-
- [ ]
|
|
183
|
-
- [ ]
|
|
184
|
-
- [ ] Wrapper persisted exactly one well-formed review with one allowed verdict.
|
|
185
|
-
- [ ] Planning seat re-verified and triaged findings without softening Codex's review.
|
|
186
|
-
- [ ] Non-APPROVE verdicts stop before execution.
|
|
124
|
+
- [ ] Reviewer -> model -> effort selection honored, including explicit same-runtime and Other.
|
|
125
|
+
- [ ] Provenance says only cross-runtime, same-runtime fresh reviewer, or unverified as observed.
|
|
126
|
+
- [ ] Reviewer remained read-only and memory-blind; final response and separate triage are immutable.
|
|
127
|
+
- [ ] APPROVE-WITH-EDITS becomes approved after edits once author edits and local gates pass.
|
|
128
|
+
- [ ] REJECT alone starts a fresh review round; at most three reviews run per invocation.
|
|
129
|
+
- [ ] Failed/cancelled/pending/unverified states do not start execution.
|
|
187
130
|
</success_criteria>
|
|
@@ -17,17 +17,18 @@ When invoked WITH a request, match it against the table below and invoke the mat
|
|
|
17
17
|
| Formalise a paper/topic into Lean (one-shot) | fvs:lean-formalise |
|
|
18
18
|
| Refactor / simplify / decompose a proof | fvs:lean-refactor |
|
|
19
19
|
| Start/plan a topic-based crypto formalisation iteration | fvs:crypto-plan |
|
|
20
|
-
|
|
|
20
|
+
| Fresh-review an initial or follow-up crypto plan | fvs:crypto-review |
|
|
21
21
|
| Run the current iteration's plan | fvs:crypto-execute |
|
|
22
22
|
| Adversarially evaluate the iteration | fvs:crypto-eval |
|
|
23
23
|
| Write a follow-up plan from eval findings | fvs:crypto-followup |
|
|
24
24
|
|
|
25
25
|
The crypto iteration loop is
|
|
26
|
-
plan ->
|
|
26
|
+
plan -> fresh review -> execute -> eval -> follow-up -> fresh review -> repeat,
|
|
27
27
|
restartable from records under `fv-plans/<topic>/`. `lean-formalise` stays the one-shot paper-track
|
|
28
28
|
command; the loop sits beside it for topic-based, multi-iteration crypto work.
|
|
29
29
|
|
|
30
30
|
The one-shot and iterative authoring stages share the bounded, indexed learning loop under
|
|
31
|
-
`.formalising/proof-engineering/`.
|
|
31
|
+
`.formalising/proof-engineering/`. `crypto-review` remains memory-blind; only verified
|
|
32
|
+
cross-runtime provenance is labeled independent.
|
|
32
33
|
|
|
33
34
|
Invoke the matched skill directly using the Skill tool.
|
package/commands/fvs/help.md
CHANGED
|
@@ -140,8 +140,11 @@ Adversarially review an FC specification against Rust, extracted Lean, and inter
|
|
|
140
140
|
- Effort defaults to `max`; lower settings and `runtime-default` remain selectable
|
|
141
141
|
- Uses fresh reviewers and labels cross-runtime versus same-runtime review
|
|
142
142
|
- Other providers use an exported packet and imported response; missing CLIs offer setup/fallback
|
|
143
|
-
- Preserves
|
|
144
|
-
-
|
|
143
|
+
- Preserves immutable `review.md` plus separate author-owned `triage.md` and input hashes
|
|
144
|
+
- Never auto-selects a menu item; supplying all flags is the non-interactive path
|
|
145
|
+
- One-run `Skip review` records `Unreviewed (user skipped)` and never starts proof work
|
|
146
|
+
- PASS proceeds; APPROVE-WITH-EDITS proceeds after author edits and gates, without another review
|
|
147
|
+
- REVISE/BLOCKED create a fresh review with prior history; each invocation stops after three rounds
|
|
145
148
|
|
|
146
149
|
Runs automatically after `lean-specify` by default, even on projects without config. To disable
|
|
147
150
|
only automation, merge `"spec_review": {"automatic": false}` into `.formalising/fvs-config.json`,
|
|
@@ -242,22 +245,28 @@ Author the next bounded, runtime-neutral executor plan for a topic, grounded in
|
|
|
242
245
|
Usage: `/fvs:crypto-plan "CKA from KEM"`
|
|
243
246
|
Usage: `/fvs:crypto-plan "CKA from KEM" --codex` # hand the planning think-step to Codex
|
|
244
247
|
|
|
245
|
-
**`/fvs:crypto-review <topic> [nN] [--target plan|followup]`**
|
|
246
|
-
Send
|
|
247
|
-
adversarial review.
|
|
248
|
+
**`/fvs:crypto-review <topic> [nN] [--target plan|followup] [--reviewer codex|claude|other] [--model ID] [--effort LEVEL]`**
|
|
249
|
+
Send a plan to a selected fresh adversarial reviewer.
|
|
248
250
|
|
|
249
|
-
-
|
|
250
|
-
-
|
|
251
|
-
- Runs
|
|
251
|
+
- Recommends the non-author runtime, then model (Sol/Astra or Fable/Sonnet), then max-first effort
|
|
252
|
+
- Labels cross-runtime, same-runtime fresh reviewer, and unverified provenance honestly
|
|
253
|
+
- Runs Codex or Claude with read-only ephemeral controls and no repository write tools
|
|
252
254
|
- Attacks source fidelity, statement soundness, semantic closure, interfaces, gates, boundedness,
|
|
253
255
|
security/data-loss risks, and roadmap coherence
|
|
254
|
-
- Wrapper
|
|
255
|
-
verifies and triages every finding
|
|
256
|
+
- Wrapper preserves immutable review evidence; the authoring seat writes separate hash-bound triage
|
|
256
257
|
- Deliberately excludes canonical and snapshotted proof-engineering memory from reviewer context
|
|
257
|
-
-
|
|
258
|
+
- APPROVE-WITH-EDITS proceeds after accepted author edits and gates, with no second review
|
|
259
|
+
- REJECT creates a fresh reviewed revision; each invocation stops after three reviewer rounds
|
|
260
|
+
- Automatic handoff asks reviewer -> model -> effort and never auto-selects a choice
|
|
261
|
+
- One-run `Skip review` records `Unreviewed (user skipped)` and never starts execution
|
|
258
262
|
|
|
259
263
|
Usage: `/fvs:crypto-review "CKA from KEM" n1 --target plan`
|
|
260
|
-
Usage: `/fvs:crypto-review "CKA from KEM" n1 --target followup`
|
|
264
|
+
Usage: `/fvs:crypto-review "CKA from KEM" n1 --target followup --reviewer claude --model sonnet --effort max`
|
|
265
|
+
|
|
266
|
+
Crypto plan and follow-up enter the interactive review handoff by default. To disable only that
|
|
267
|
+
automatic handoff, merge `"crypto_review": {"automatic": false}` into
|
|
268
|
+
`.formalising/fvs-config.json`. Standalone review remains available, and a trusted user may
|
|
269
|
+
explicitly invoke crypto execution from an unreviewed plan.
|
|
261
270
|
|
|
262
271
|
**`/fvs:crypto-execute <topic> nN`**
|
|
263
272
|
Run the current iteration's bounded plan under the green-build guard; a failed proof triggers a short interactive redirect early. (Executor stage — takes no `--codex`.)
|
|
@@ -26,4 +26,16 @@ runtime, model, and effort menu; preserve the spec and record the review and fin
|
|
|
26
26
|
Follow the review workflow with `$ARGUMENTS`. Explicit invocation always runs the selection flow,
|
|
27
27
|
even when `spec_review.automatic` is false. Automatic invocation from `lean-specify` enters the
|
|
28
28
|
same workflow after generation checks, with the resolved spec/source paths and author runtime.
|
|
29
|
+
|
|
30
|
+
Honor explicit reviewer/model/effort choices. Supplying all three standalone flags is the
|
|
31
|
+
non-interactive path; otherwise ask only for missing choices in order: reviewer -> model -> effort.
|
|
32
|
+
Automatic callers record `Skip review` exactly as `Unreviewed (user skipped)` and do not auto-start
|
|
33
|
+
proof work.
|
|
34
|
+
|
|
35
|
+
The reviewer is read-only; the `lean-specify` authoring seat keeps `review.md` unchanged and writes
|
|
36
|
+
separate `triage.md` with finding IDs, old/new hashes (pre-edit/post-edit), and rerun structure,
|
|
37
|
+
style, and optional build gates. PASS is terminal. APPROVE-WITH-EDITS becomes `approved after edits`
|
|
38
|
+
after accepted bounded edits pass those gates, with no second review. REVISE and BLOCKED
|
|
39
|
+
require a fresh revision or evidence packet and another review with prior review/triage history.
|
|
40
|
+
Run at most three reviewer rounds per invocation; stop at the cap with an exact resume command.
|
|
29
41
|
</process>
|