@zephyrdeng/pi-review 0.19.0 → 0.20.0

package/resources/review-presets.json

@@ -3,18 +3,18 @@
     "description": "Code review only: inspect provided files, diffs, or paths and return review conclusions without editing.",
     "tools": ["read", "grep", "find", "ls"],
     "thinking": "high",
-    "instructions": "Review code or code changes only. Do not edit, write, patch, format, commit, or otherwise mutate files. Use the enabled read-only tools for inspection. Focus on correctness, regressions, security, concurrency, API contracts, edge cases, maintainability, and missing tests."
+    "instructions": "Review code or code changes only. Do not edit, write, patch, format, commit, or otherwise mutate files. Use the enabled read-only tools for inspection. Focus on correctness, regressions, security, concurrency, API contracts, edge cases, maintainability, and missing tests. Ground every finding in facts: before concluding, use grep/find/ls to look for the repository's own evidence — design docs, ADRs, README/docs, CHANGELOG, any spec or issue text included in the request, and the existing tests that cover the touched code — and cite what you found. If no spec or test evidence exists for a claim, say so explicitly in Risks and Blind Spots instead of assuming the code is correct, and do not mark a finding Actionable when it rests on inference alone."
   },
   "plan": {
     "description": "High-level plan review with multiple expert lenses; output a decisive conclusion.",
     "tools": ["read", "grep", "find", "ls"],
     "thinking": "high",
-    "instructions": "Review the proposal, design, architecture, or product plan through multiple expert lenses: engineering lead, product owner, security, QA, operations, and developer experience. Do not implement. Identify fatal flaws, trade-offs, missing constraints, sequencing risks, and the smallest changes that would make the plan safer."
+    "instructions": "Review the proposal, design, architecture, or product plan through multiple expert lenses: engineering lead, product owner, security, QA, operations, and developer experience. Do not implement. Identify fatal flaws, trade-offs, missing constraints, sequencing risks, and the smallest changes that would make the plan safer. Verify the plan's factual claims against the repository (grep/find/read the code, docs, ADRs, and tests it refers to) and cite what you checked; mark claims you could not verify as unverified rather than accepting them."
   },
   "challenge": {
     "description": "Adversarial plan review: pressure-test assumptions, boundaries, dependencies, and terminology.",
     "tools": ["read", "grep", "find", "ls"],
     "thinking": "high",
-    "instructions": "Challenge the plan as an adversarial design reviewer. Do not ask a sequence of live interview questions; produce a one-shot review conclusion. Pressure-test terminology, hidden dependencies, reversibility, edge cases, failure modes, migration paths, ownership boundaries, and what evidence would invalidate the plan."
+    "instructions": "Challenge the plan as an adversarial design reviewer. Do not ask a sequence of live interview questions; produce a one-shot review conclusion. Pressure-test terminology, hidden dependencies, reversibility, edge cases, failure modes, migration paths, ownership boundaries, and what evidence would invalidate the plan. For every assumption you attack, check it against the repository's actual code, docs, and tests first and cite the evidence; state which assumptions you could not verify."
   }
 }

package/skills/pi-review/SKILL.md

@@ -156,6 +156,29 @@ Use this protocol when the host is closing out implementation work and may edit
 
 The shell exit policy is: `0` clean, `1` status is `has_findings`, `2` usage error, `3` needs human, `4` blocked/runtime failure. A non-zero loop result is a gate signal, not permission for the child reviewer to edit.
 
+## Verification before review (`--verify`)
+
+Opt-in. pi-review runs verification commands once, in a disposable isolated copy of the working tree, before reviewers start, and gives every reviewer the results as `test-output` evidence. Reviewers stay read-only; pi-review's own code runs the commands. Guide: [docs/guide/verify-mode.md](https://github.com/ZephyrDeng/pi-review/blob/main/docs/guide/verify-mode.md).
+
+**When to use it:** the user asks for a review that should rest on executed tests/type checks, or the change makes claims (a bug fix, a behavior change) that a test can confirm. Skip it for plan/design reviews and for untrusted checkouts.
+
+**Trust boundary:** verification executes repository code on this machine with the user's privileges. The isolated copy protects the working tree, not the machine. Never add `--verify` on a repository the user would not run its tests on.
+
+**What runs without asking:** only `verify.commands` from `.pi-review.json` committed at `HEAD` (standing pre-approval). `pi-review --verify -- <target>` runs exactly those. Everything else needs a human decision.
+
+**Agent-proposed verification — host steps (mandatory order):**
+
+1. **Propose.** Write `plan.json` yourself (default: you know the session, the issue, the design docs, and what you changed). Pick the smallest set of commands that confirms or refutes the change's claims; add a minimal repro test via `files` when a key claim has no test. Every step needs `id`, `argv` (no shell), and `purpose`; add `expectedDurationSec`, `sideEffects`, `idempotent` as your honest self-assessment. Fallback when you lack context: `pi-review verify propose -- <target> > plan.json` (read-only Pi child, same format).
+2. **Show.** Run `pi-review verify show plan.json`. Exit `0` = everything is pre-approved, go to step 5 without `--verify-approval`. Exit `5` = a human must decide. Exit `2` = invalid plan; fix it.
+3. **Ask the user.** Present every step that says `NEEDS APPROVAL` — command, purpose, new files, your self-assessment — with your host's question tool (Claude Code `AskUserQuestion`; Codex/Cursor approval prompt). Offer approve all / choose steps / reject. **Never approve on the user's behalf, never skip the question, and never run a step yourself in the shell instead.**
+4. **Record.** `pi-review verify approve plan.json --all | --steps a,b [--reject c] --by "<user> via <tool>" > approval.json`. The record binds to each step's content hash; editing a step afterwards voids its approval, so re-ask after any edit.
+5. **Review.** `pi-review [loop] --verify-plan plan.json --verify-approval approval.json [panel flags] -- <target>`. Unapproved steps show up as `skipped_unapproved` in the prompt and meta.
+6. **Iterate (optional, capped).** To see results without a full review: `pi-review verify run --plan plan.json --approval approval.json`. After a failure you may append `"round": 2` steps (e.g. a minimal repro), then repeat steps 2-4 for the new steps only and pass both approval files (`--verify-approval` is repeatable). The default cap is one follow-up round (`verify.maxFollowUpRounds`).
+
+CI / non-interactive callers: pass `--verify` (whitelist) or a plan without approval; only pre-approved steps run.
+
+**Reading results:** the footer shows `Verify  <status> · N run / M not run`; `PI_REVIEW_META_JSON.verification` carries `status` (`passed | partial | failed | unverified`), `reason`, per-step `approval`, `status`, `exitCode`, `stepSha256`, and the redacted output tail. A failing command is evidence, not a pi-review failure. Report `unverified` honestly; never describe it as passing.
+
 ## Gate-grade fast screening (`pi-review screen`)
 
 `pi-review screen <@files|paths...>` is a sub-second gate check that completely skips LLM prose generation. It slices files at declaration boundaries into hunks, evaluates each hunk against defect patterns via single-call typed Jev judgments, and assembles findings from catalog templates in **~1.0–1.2s**:
@@ -197,6 +220,8 @@ A finding whose summary ends `matched no known pattern` is an unmatched signal:
 | `--until clean` | In `loop` mode, automates iterative convergence (stops when confirmed actionable issues drop to 0 or hard `--max-rounds` cap is hit). |
 | `--progress-log <path>` | Streams compact JSON events to a file; essential for background runs in buffered-output hosts like Claude Code, Codex, or Cursor. |
 | `--ui web` | Starts a loopback browser dashboard for panel runs; recommended for visual inspection on agent hosts. |
+| `--verify` / `--verify-plan <file>` / `--verify-approval <file>` | Run whitelisted or human-approved verification once in an isolated copy before reviewers; see [Verification before review](#verification-before-review---verify). |
+| `verify propose\|show\|approve\|run` | The host-side handshake for agent-proposed verification plans. `show` exits `5` when the user must approve. |
 
 
 ## Panel review