@codyswann/lisa 3.6.2 → 3.8.0

package/expo/create-only/.github/required-checks.json

@@ -7,7 +7,12 @@
     "A transcription expires after 90 days, because a ruleset can be edited with no signal in this repository. The shipped `.github/workflows/required-checks-drift.yml` runs `--remote` weekly to catch that; `--remote` reads the ruleset live and so answers even when the cache is untrusted.",
     "Lisa's quality.yml runs the offline arm on every pull request. This seed ships `\"enforcement\": \"warn\"`, which downgrades findings AND the NOT-CHECKED refusal to reports so a fresh install does not go red on arrival. Delete the key once you have transcribed the list — then it blocks.",
     "`skip_jobs` is matched as an exact comma-delimited token and GitHub Actions expression syntax has no string-replace, so write the list with no spaces: `skip_jobs: 'a,b'`, never `skip_jobs: 'a, b'`. A spaced token matches nothing and the job runs — reported as `whitespace_in_skip_token`.",
-    "`_example_required_contexts` below is a STARTING POINT FOR TYPING, never read by the guard. Verify every line against your own ruleset before promoting any of it."
+    "`_example_required_contexts` below is a STARTING POINT FOR TYPING, never read by the guard. Verify every line against your own ruleset before promoting any of it.",
+    "THE FAMILY: required-and-red is loud; required-and-vacuous is not; advisory-and-stale is invisible. All three are one gate reporting satisfied without proving anything. `required_contexts` + `skip_job_declarations` above cover the SKIPPED variant; `evidence_bearing_checks` below covers the VACUOUS one.",
+    "VACUOUS, measured (CodySwannGT/lisa#2497): a required `CodeRabbit` context posted `success` with the description `Review rate limited`, having reviewed nothing, on two security-relevant PRs that then merged and shipped. `gh pr checks` prints `pass` for that exactly as it does for a real review — only the DESCRIPTION tells them apart.",
+    "Run it per PR: `npm run check:vacuous-required-checks -- --pr=1234` (or `node scripts/check-skipped-required-checks.mjs --pr=1234`). It reads `gh pr checks --json name,state,bucket,description`, which is the only route that carries the description for a legacy commit status like CodeRabbit's.",
+    "`evidence_bearing_checks` names the checks whose GREEN is supposed to mean something reviewed the code. Use `{}` to accept the shipped description vocabulary, or add `proof` / `no_work` arrays to extend it — extensions ADD to the defaults, they do not replace them. Undeclared checks are never examined, because most CI jobs ship an empty description and flagging them all would bury the one finding that matters.",
+    "This arm REPORTS AND NEVER BLOCKS, in every enforcement mode. A review bot can go hollow because an org-wide SPENDING CAP was hit, and a gate that reddens every PR on a billing state is worse than the one it criticises. What it changes is what you may CLAIM: a PR carrying a `vacuous_required_check` finding has not been shown to be reviewed, so do not record it as reviewed."
   ],
   "enforcement": "warn",
   "ruleset": {
@@ -54,5 +59,8 @@
       "ruleset_required": false,
       "reason": "DAST needs a deployed target and is run against staging, not on every PR. Not a required context."
     }
+  },
+  "evidence_bearing_checks": {
+    "CodeRabbit": {}
   }
 }

package/nestjs/create-only/.github/required-checks.json

@@ -7,7 +7,12 @@
     "A transcription expires after 90 days, because a ruleset can be edited with no signal in this repository. The shipped `.github/workflows/required-checks-drift.yml` runs `--remote` weekly to catch that; `--remote` reads the ruleset live and so answers even when the cache is untrusted.",
     "Lisa's quality.yml runs the offline arm on every pull request. This seed ships `\"enforcement\": \"warn\"`, which downgrades findings AND the NOT-CHECKED refusal to reports so a fresh install does not go red on arrival. Delete the key once you have transcribed the list — then it blocks.",
     "`skip_jobs` is matched as an exact comma-delimited token and GitHub Actions expression syntax has no string-replace, so write the list with no spaces: `skip_jobs: 'a,b'`, never `skip_jobs: 'a, b'`. A spaced token matches nothing and the job runs — reported as `whitespace_in_skip_token`.",
-    "`_example_required_contexts` below is a STARTING POINT FOR TYPING, never read by the guard. Verify every line against your own ruleset before promoting any of it."
+    "`_example_required_contexts` below is a STARTING POINT FOR TYPING, never read by the guard. Verify every line against your own ruleset before promoting any of it.",
+    "THE FAMILY: required-and-red is loud; required-and-vacuous is not; advisory-and-stale is invisible. All three are one gate reporting satisfied without proving anything. `required_contexts` + `skip_job_declarations` above cover the SKIPPED variant; `evidence_bearing_checks` below covers the VACUOUS one.",
+    "VACUOUS, measured (CodySwannGT/lisa#2497): a required `CodeRabbit` context posted `success` with the description `Review rate limited`, having reviewed nothing, on two security-relevant PRs that then merged and shipped. `gh pr checks` prints `pass` for that exactly as it does for a real review — only the DESCRIPTION tells them apart.",
+    "Run it per PR: `npm run check:vacuous-required-checks -- --pr=1234` (or `node scripts/check-skipped-required-checks.mjs --pr=1234`). It reads `gh pr checks --json name,state,bucket,description`, which is the only route that carries the description for a legacy commit status like CodeRabbit's.",
+    "`evidence_bearing_checks` names the checks whose GREEN is supposed to mean something reviewed the code. Use `{}` to accept the shipped description vocabulary, or add `proof` / `no_work` arrays to extend it — extensions ADD to the defaults, they do not replace them. Undeclared checks are never examined, because most CI jobs ship an empty description and flagging them all would bury the one finding that matters.",
+    "This arm REPORTS AND NEVER BLOCKS, in every enforcement mode. A review bot can go hollow because an org-wide SPENDING CAP was hit, and a gate that reddens every PR on a billing state is worse than the one it criticises. What it changes is what you may CLAIM: a PR carrying a `vacuous_required_check` finding has not been shown to be reviewed, so do not record it as reviewed."
   ],
   "enforcement": "warn",
   "ruleset": {
@@ -53,5 +58,8 @@
       "ruleset_required": false,
       "reason": "The shipped ci.yml runs ZAP as its own `zap` job against a deployed target after quality passes, not inside the quality callee. Not a required context."
     }
+  },
+  "evidence_bearing_checks": {
+    "CodeRabbit": {}
   }
 }

package/package.json

@@ -120,7 +120,7 @@
     }
   },
   "name": "@codyswann/lisa",
-  "version": "3.6.2",
+  "version": "3.8.0",
   "description": "Claude Code governance framework that applies guardrails, guidance, and automated enforcement to projects",
   "main": "dist/index.js",
   "exports": {

package/plugins/lisa/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "lisa",
-  "version": "3.6.2",
+  "version": "3.8.0",
   "description": "Universal governance — agents, skills, commands, hooks, and rules for all projects",
   "author": {
     "name": "Cody Swann"

package/plugins/lisa/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "lisa",
-  "version": "3.6.2",
+  "version": "3.8.0",
   "description": "Universal governance: agents, skills, commands, hooks, and rules for all projects.",
   "author": {
     "name": "Cody Swann"

package/plugins/lisa/.codex-plugin/skills/lisa-drive-pr-to-merge/SKILL.md

@@ -291,7 +291,22 @@ Delegate to the `pull-request-review` skill with the PR number. It owns the whol
 comment cycle: fetch every unresolved human + bot thread (with resolution state via
 GraphQL), implement valid feedback (commit + push), reply to invalid feedback, and
 resolve every thread via `resolveReviewThread` so the branch-protection
-thread-resolution gate clears. If that skill needs to push a commit, leave
+thread-resolution gate clears.
+
+**A green review check is not proof a review happened.** That skill's Step 1b
+returns a `reviewed` / `NOT REVIEWED` verdict — record it, and repeat it in this
+skill's final report. Measured (CodySwannGT/lisa#2497): `CodeRabbit` was a
+*required* context and posted `success — "Review rate limited"` on #2483 and
+#2484, so branch protection recorded a satisfied review gate for two
+security-relevant PRs nothing had read; both merged and shipped in `v3.5.1`.
+
+`NOT REVIEWED` is **not a blocker** — do not hold the merge on it, do not treat
+it as a failing check, and do not try to force the bot to re-run. A hollow
+review check is usually an org-wide vendor spending cap, which is a billing
+matter no amount of driving will clear, and whether such a check belongs in the
+required set at all is an open owner decision. It is a **reporting** obligation:
+the PR merged unreviewed, and the report has to say so instead of implying a
+review it did not get. If that skill needs to push a commit, leave
 auto-merge armed (section 1); when it returns, re-read `headRefOid` and reset
 `verify_commit` to the returned/pushed head, then continue. Do not re-implement review handling here
 — it is the single source of truth for review-thread handling.
@@ -452,3 +467,16 @@ At every terminal state, release the babysitter lease
 (`gh pr edit <pr> --remove-label "lisa:babysitter-on-duty"`) so the CI
 auto-fix workflow can take over as fixer of last resort if the branch goes
 red later with nobody driving it.
+
+**Every terminal report carries the step-(d) review verdict**, including a
+successful `MERGED`. "Merged, all checks green" is exactly the sentence that hid
+#2483 and #2484: both were green, both were merged, and neither had been read by
+anything. Green means *no gate objected*; it does not mean *something looked*.
+So state the verdict alongside the outcome:
+
+- `MERGED — reviewed (CodeRabbit "Review approved")`
+- `MERGED — NOT REVIEWED: CodeRabbit posted success but "Review rate limited"`
+
+This is reporting, never a terminal state of its own. `NOT REVIEWED` does not
+turn a merged PR into a blocked one, and it must never be used to withhold a
+merge — it changes what the record says, not what the loop does.

package/plugins/lisa/.codex-plugin/skills/lisa-pull-request-review/SKILL.md

@@ -37,8 +37,49 @@ gh api graphql -f query='
   -F owner=<owner> -F repo=<repo> -F pr=<pr>
 ```
 
-Keep only threads where `isResolved == false`. If there are none, report success
-and exit (nothing to do).
+Keep only threads where `isResolved == false`.
+
+**If there are none, you have not yet learned anything — run Step 1b before
+concluding.** Zero unresolved threads has two completely different causes, and
+this query cannot tell them apart: *a reviewer looked and found nothing*, or
+*nothing ever looked*. Reporting the first when it was the second is the defect
+in CodySwannGT/lisa#2497.
+
+## Step 1b: Did any review actually happen? (required before reporting)
+
+A required review check can post `success` having reviewed nothing. Measured on
+PRs #2483 and #2484: `CodeRabbit` reported `success — "Review rate limited"`,
+zero reviews, and both merged on that green carrying security-relevant changes.
+
+```bash
+gh pr checks <pr> --json name,state,bucket,description \
+  --jq '.[] | select(.name | test("(?i)coderabbit|review")) | "\(.name)\t\(.state)\t\(.description)"'
+```
+
+**The state column reads `SUCCESS` whether the review was real or hollow — only
+the description distinguishes them.** `Review approved` / `Review completed` is
+a real review; `Review rate limited`, `Review queued`, or a missing context is
+not. Never read `gh pr view --json statusCheckRollup` for this: CodeRabbit posts
+a legacy commit status, which that route returns *without* the description.
+
+Where the project ships Lisa's guard, prefer its machine-readable form, which
+also says whether the check is ruleset-required (so whether branch protection
+recorded a satisfied review gate for a review that did not happen):
+
+```bash
+node scripts/check-skipped-required-checks.mjs --pr=<pr> --json
+```
+
+It reports and never fails — a hollow check is often a vendor spending cap, not
+a repository defect, and it is not this skill's call to block on one.
+
+Carry the finding into your Step 4 report. **Do not write "no unresolved review
+threads" on its own** — it is true of an unreviewed PR too. Say which you
+observed:
+
+- `reviewed — CodeRabbit "Review approved", 0 unresolved threads`
+- `NOT REVIEWED — CodeRabbit success but "Review rate limited" (vacuous); 0 threads means nobody looked`
+- `NOT REVIEWED — no review check reported on this PR at all`
 
 ## Step 2: Triage and act on each unresolved thread
 
@@ -75,6 +116,13 @@ judgment. This skill resolves **threads**; it does not dismiss review-decision
 gates (`CHANGES_REQUESTED`) or merge the PR — the caller (`drive-pr-to-merge`)
 owns those.
 
+**Open the report with the Step 1b verdict, before the thread counts.** A thread
+summary describes what was done about review findings; it says nothing about
+whether a review produced any. State `reviewed` or `NOT REVIEWED (<why>)` first,
+then the counts. A caller that records "reviews addressed" in evidence must
+carry that verdict through verbatim — a PR whose only review check was vacuous
+has not been reviewed, no matter how clean its thread list is.
+
 ## Composition
 
 - **Standalone**: `/lisa:pull-request:review <pr>`.

package/plugins/lisa/rules/eager/falsifiable-checks.md

@@ -18,7 +18,9 @@ Each has been observed in real runs; each reported success while asserting nothi
 ## Mandatory
 
 - **Falsify before reporting.** No clean result is reportable until the check has been shown to fail on a deliberate break. **"Mentally reverting" does not count** — reasoning that the assertion *would* fail is precisely the step that lets a non-functional guard ship, because the author already believes it is load-bearing. Run the break.
-- **Count the failures: introduce the exact regression the guard exists to prevent, and confirm **exactly one** test fails, then revert.** Zero means the guard is inert; many means it is over-broad and its next failure will not name the cause. A guard that pinned one field of a structure shipped with all 50 tests green while the regression walked straight through.
+- **Mutation-prove the guard, then read the cardinality.** Neuter the protection **in production code**, run the **whole suite**, count the failures and **read their names**, then revert. **Zero** ⇒ the guard is inert (one pinning a single field shipped with all 50 tests green while the regression walked through). **Many, unrelated** ⇒ over-broad; its next failure will not name the cause. **Exactly one, or several all named for the same regression** ⇒ load-bearing and correctly scoped. A **zero is robust to contention** — load only ever adds failures — so only the non-zero counts need a quiet machine.
+- **Never scope the probe by filename, and never infer absent coverage from a grep.** Suites split at the line cap, so a protection's tests often sit in a sibling file: a filename-scoped probe reported cardinality 0 on a live security fix that a whole-suite run showed failing 4 tests, every one named for it.
+- **Assert that no copy is wrong, not that some copy is right.** An existential assertion over duplicated content is satisfied by the correct copy and cannot see the broken one — rewriting one suite's assertions to universal negatives took the same mutation from **0 to 18** failures. Enumerate the property (*every* marker uses an em-dash), never the one known-bad spelling.
 - **Reconstruct a red state; never narrate one you did not run.** Evidence describing a failing or dirty state must come from a run you actually observed — an evidence file once claimed "working tree clean" while listing two untracked scripts in the same file. To observe a state that no longer exists, check that commit out in a **throwaway detached-HEAD worktree** and run it there. Writing down what the output would have said is fabrication, however confident the reasoning.
 - **Say how it was falsified.** "0 findings" alone is not a result; state what you broke and that the check caught it. A gate whose falsification is untested must be reported as *unvalidated*, not as passing.
 - **Prefer structural over textual checks.** Parse the AST/structure instead of matching source text: text matching cannot distinguish a field from a comment, an alias, or a nested occurrence, and it produces false positives that mask the real ones.

package/plugins/lisa/rules/reference/derived-branch-plan.md

@@ -85,9 +85,25 @@ Stated plainly, because the session that produced this contract exists to make e
 
 This is **not** an executable control. `ready-role-filing`'s sibling guard is a hook that exits 2 and physically refuses a tool call; nothing here refuses anything. What enforces this contract is a validator agent reading `SKILL.md` and choosing to apply it — the same rung of the ladder that failed to bind 13 filings out of 13.
 
-What the regression suite adds is narrower than enforcement and worth naming precisely: it pins the **wording** across all six generated skill roots, so the contract cannot be silently deleted, softened, or lost in a regeneration. It cannot catch a validator that documents the rule and then ignores it at runtime.
+The regression suite has two halves. **Why they bind differently is `falsifiable-checks`' subject, not this rule's** — read the quantifier guidance there. What belongs here is the measurement, because this contract happens to be where it was taken.
 
-That is the ceiling for now rather than a shortcut, because `SKILL.md` *is* the validators' execution substrate — there is no compiled artifact underneath to attach a hook to. The honest summary is: absence-as-`false` was undecidable and is now *decidable*, which is a real improvement and a strictly weaker claim than *enforced*. Treat a gate verdict resting on this contract as evidence, not proof, until a deterministic checker exists to read the declaration off a live item.
+- `runtime-behavior-change-decidability.test.ts` asserts the wording is *present* across all six generated skill roots. It pins the contract against silent deletion or a lossy regeneration, and proves nothing about any particular copy.
+- `runtime-behavior-change-consistency.test.ts` asserts no copy says the *wrong* thing, over every occurrence in the corpus: every declaration marker uses the em-dash discriminator, every absence-derivation site resolves to *underivable* within its own sentence, and no phrasing maps an absent section to `false` or hands authority back to the caller.
+
+This document states its rule **twice** — once in gate S8, once in the execution step — which is what made the gap observable. `lisa-linear-validate-issue` once spelled the discriminator with an ASCII hyphen in its execution step while its own S8 used the em-dash, and 192 assertions stayed green through that mutation twice running. The fix pinned the execution step and left S8's copy bare. Mutating each site separately, whole suite, before and after the second half landed:
+
+| mutated site | before | after |
+|---|---|---|
+| execution step reads an absent section as `false` | 18 | 36 |
+| **gate S8's prose reads an absent section as `false`** | **0** | **18** |
+| writers stop rendering the section | 36 | 54 |
+| a caller's assertion beats the stored declaration | 18 | 36 |
+
+Zero out of 11,270 — same file, same rule, same commit, one site pinned and its twin bare.
+
+(The wrong spelling is deliberately not written out above. This section sits inside the corpus the consistency suite scans, and the first draft of it quoted the defect as an illustration and failed its own check.)
+
+**The executable half of this contract is nothing.** Both suites read markdown. `SKILL.md` *is* the validators' execution substrate — there is no compiled artifact underneath to attach a hook to, so there is no function to call and no return value to assert. What they prove is that the instruction the agent reads is internally consistent; what neither can prove is that the agent then obeys it. Absence-as-`false` was undecidable, is now *decidable* and *non-contradictory*, and both of those are strictly weaker than *enforced*. Treat a gate verdict resting on this contract as evidence, not proof, until a deterministic checker exists to read the declaration off a live item.
 
 This costs nothing in derivation difficulty. A single-environment project — Lisa itself is one, `deploy.branches: { production: main }` — recomputes every applicable item to the same branch. The gate was never hard to satisfy; it was impossible to *audit*, and those are different problems with different fixes. Weakening S19 would have fixed neither.
 

package/plugins/lisa/rules/reference/falsifiable-checks.md

@@ -63,17 +63,69 @@ A clean result is a statement about what the check can perceive, not about the c
 
 1. Author the check.
 2. Deliberately break the guarded property — specifically, introduce **the exact regression the guard exists to prevent**, not a nearby or convenient break.
-3. Confirm the check **fails, names the right file/line, and that exactly one test fails**. A failure that does not localize is weak evidence the check is measuring the right thing.
+3. Confirm the check **fails and names the right file/line**, then read the failure count and the failing test *names* per the yardstick below.
 4. Restore, and confirm green again.
 5. Report the falsification alongside the result.
 
 ### The failure count is part of the evidence
 
-Step 3's cardinality is not pedantry; each deviation names a distinct defect.
+Step 3's cardinality is not pedantry; each deviation names a distinct defect. Run it as four steps, in this order:
 
-- **Zero failures** — the guard is inert. It was authored, it is green, and it asserts nothing about the property it names. The originating incident: a guard pinned one field of a structure and the regression it was written to stop walked straight through it with all fifty tests green, because nothing in the suite touched the field that actually moved.
-- **More than one failure** — the break is too coarse, the guard is over-broad, or unrelated tests share the fixture. Any of the three means the guard's next real failure will not tell the reader what broke, which is most of a guard's value.
-- **Exactly one failure** — the guard localizes. That is the evidence, and it is what gets reported.
+1. **Neuter the protection in production code** — delete the branch, drop the token from the enum, remove the check. Editing a *test* proves nothing about the guard; it only proves the test file is loaded.
+2. **Run the whole suite.** Never a single test file, and never a path filter.
+3. **Count the failures and read their names.** The count alone is ambiguous; the names are what separate "several failures of one regression" from "several unrelated failures."
+4. **Revert, and confirm green again.**
+
+Then read the result:
+
+- **Zero failures** — the guard is inert. It was authored, it is green, and it asserts nothing about the property it names. Two real instances: a guard pinned one field of a structure and the regression it was written to stop walked through it with all fifty tests green; and removing either `conflicting` or `unreadable` from `design-source-gate.mjs`'s `VIOLATION_STATUSES` failed **0 of 38 tests** while flipping the gate's verdict from FAIL to PASS.
+- **Many failures, unrelated to each other** — the break is too coarse, the guard is over-broad, or unrelated tests share a fixture. Any of the three means the guard's next real failure will not tell the reader what broke, which is most of a guard's value.
+- **Exactly one failure, or several all named for the same regression** — the guard localizes. Both readings are load-bearing and correctly scoped. Removing the two-token `--config-env` check failed **3** tests and pinning an index failed **5**; every failing name in both runs described the removed behaviour, so both guards were correct. This is a reading exercise, not arithmetic — "more than one is bad" is the wrong rule.
+
+### A zero is robust to contention; a positive count is not
+
+The obvious objection to any cardinality measured on a shared machine — *"your probe ran on a loaded box, so how do you trust the number?"* — has an asymmetric answer, and the asymmetry is worth stating because it decides which numbers you must re-measure.
+
+**Load can only add failures, never remove them.** A test that passes under contention would also have passed on a quiet machine; contention causes timeouts and lock races, which turn green into red, never red into green. So:
+
+- **A zero stands regardless of what else was running.** If the whole suite reports zero failures attributable to your mutation on a loaded box, a quiet box cannot produce fewer. `0 of 11,270` is `0 of 11,270`.
+- **A non-zero count needs a quiet machine**, because contention inflates it. Failures that are really flakes get miscounted as the guard's, which reads as "over-broad" and gets a correctly-scoped guard deleted.
+
+Practical consequence: an inert-guard finding is safe to report from a busy machine, while the "exactly one, or several all named alike" reading is only trustworthy once you have separated your mutation's failures from the load-flake population — which is what reading the *names* is for.
+
+### Two reasons a real protection reports zero
+
+A cardinality of zero has two distinct innocent causes, and they need different fixes. Rule out both before concluding a guard is inert:
+
+1. **Wrong probe scope** — the probe was scoped to a filename or path and is blind to a split suite. Fix: run the whole suite.
+2. **Wrong quantifier** — the assertions are existential over a corpus that contains duplicates, so a correct copy satisfies them on the broken copy's behalf. Fix: assert the universal negative.
+
+### Never scope the probe by filename
+
+**A cardinality probe scoped to a filename reports zero when a suite splits. Follow the protection, not the path.**
+
+Observed live: a reviewer neutered `before_end_of_options` (an end-of-options security fix), grepped `block-direct-issue-create.test.ts` for coverage, found none, and reported **cardinality 0 — the fix ships untested**. It was relayed as fact and used to question the author's discipline. Re-measuring against the whole suite showed neutering the protection fails **4** tests, each named for it: the suite had split at the 300-effective-line lint ceiling, and the declaration arm — including all five end-of-options tests — had moved to `block-direct-issue-create-declarations.test.ts`.
+
+The false negative fires exactly where this repo is most likely to split a file: at the line cap, on the suites that have grown *because* someone added protections to them. So:
+
+- Never scope the probe by filename or path filter.
+- Never infer absence of coverage from a grep. A grep locates text; it cannot enumerate what a mutation breaks.
+- A prior cardinality-0 finding measured with a path-scoped probe must be **re-measured** with a whole-suite probe before anyone acts on it.
+
+### Assert that no copy is wrong, not that some copy is right
+
+A test asserting that the **correct** marker is *present* passes trivially when the corpus happens to state it twice — the second statement satisfies the assertion no matter what the first one says, so the test survives the exact edit it exists to catch. The falsifiable form asserts the **wrong** spelling is *absent*: it has no second copy to fall back on, and it goes red the moment the wrong form reappears.
+
+Stated as quantifiers, which is the general form:
+
+- **Existential** — *"some copy says the right thing"* — is trivially satisfied by any other copy. A wrong copy is undetectable.
+- **Universal negative** — *"no copy says the wrong thing"* — cannot be satisfied by a correct copy elsewhere.
+
+Observed: a suite over a corpus containing two copies of the same rule returned **cardinality 0** on a real mutation because every assertion was existential. Rewriting the assertions to universal negatives — same code, same corpus, only the quantifier changed — took the same mutation from **0 to 18** failures. The defect it had been hiding: one PR pinned the *execution-step* copy of a rule and left the *gate* copy entirely unpinned, so reverting the gate's "absent section means underivable — never `false`" back to "means `false`" failed **0 of 11,270 tests** while the validator's documented gate contradicted its own execution step.
+
+### Enumerate the property, not the known-bad instance
+
+A universal negative against one hard-coded wrong spelling still only sees that spelling. Assert the **property** instead: *every* marker uses an em-dash catches the en-dash and the missing dash too, which `not.toContain("<one wrong spelling>")` never would. When that generalisation was made, it went red immediately on a wrong marker its own author had just written into rule prose — the discipline catching the person applying it, in the same run.
 
 ### Never narrate a red state you did not run
 

package/plugins/lisa/scripts/design-source-gate.mjs

@@ -294,8 +294,17 @@ export function classifyDesignSource(file, config) {
   return { ...base, status: "undeclared" };
 }
 
-/** Per-file statuses that fail the gate. */
-const VIOLATION_STATUSES = new Set([
+/**
+ * Per-file statuses that fail the gate.
+ *
+ * Exported so the suite can pin the set itself, not just its members' behavior.
+ * Classifying a file `unreadable` is not a guard; *failing the change* because
+ * of it is. Issue #2492 measured the difference: with only classification
+ * asserted, deleting `conflicting` or `unreadable` from this list flipped the
+ * verdict to PASS with the whole suite still green. The pinning test now fails
+ * on any edit to this set — a removal and an addition alike.
+ */
+export const VIOLATION_STATUSES = new Set([
   "undeclared",
   "malformed",
   "conflicting",

package/plugins/lisa/skills/lisa-drive-pr-to-merge/SKILL.md

@@ -291,7 +291,22 @@ Delegate to the `pull-request-review` skill with the PR number. It owns the whol
 comment cycle: fetch every unresolved human + bot thread (with resolution state via
 GraphQL), implement valid feedback (commit + push), reply to invalid feedback, and
 resolve every thread via `resolveReviewThread` so the branch-protection
-thread-resolution gate clears. If that skill needs to push a commit, leave
+thread-resolution gate clears.
+
+**A green review check is not proof a review happened.** That skill's Step 1b
+returns a `reviewed` / `NOT REVIEWED` verdict — record it, and repeat it in this
+skill's final report. Measured (CodySwannGT/lisa#2497): `CodeRabbit` was a
+*required* context and posted `success — "Review rate limited"` on #2483 and
+#2484, so branch protection recorded a satisfied review gate for two
+security-relevant PRs nothing had read; both merged and shipped in `v3.5.1`.
+
+`NOT REVIEWED` is **not a blocker** — do not hold the merge on it, do not treat
+it as a failing check, and do not try to force the bot to re-run. A hollow
+review check is usually an org-wide vendor spending cap, which is a billing
+matter no amount of driving will clear, and whether such a check belongs in the
+required set at all is an open owner decision. It is a **reporting** obligation:
+the PR merged unreviewed, and the report has to say so instead of implying a
+review it did not get. If that skill needs to push a commit, leave
 auto-merge armed (section 1); when it returns, re-read `headRefOid` and reset
 `verify_commit` to the returned/pushed head, then continue. Do not re-implement review handling here
 — it is the single source of truth for review-thread handling.
@@ -452,3 +467,16 @@ At every terminal state, release the babysitter lease
 (`gh pr edit <pr> --remove-label "lisa:babysitter-on-duty"`) so the CI
 auto-fix workflow can take over as fixer of last resort if the branch goes
 red later with nobody driving it.
+
+**Every terminal report carries the step-(d) review verdict**, including a
+successful `MERGED`. "Merged, all checks green" is exactly the sentence that hid
+#2483 and #2484: both were green, both were merged, and neither had been read by
+anything. Green means *no gate objected*; it does not mean *something looked*.
+So state the verdict alongside the outcome:
+
+- `MERGED — reviewed (CodeRabbit "Review approved")`
+- `MERGED — NOT REVIEWED: CodeRabbit posted success but "Review rate limited"`
+
+This is reporting, never a terminal state of its own. `NOT REVIEWED` does not
+turn a merged PR into a blocked one, and it must never be used to withhold a
+merge — it changes what the record says, not what the loop does.

package/plugins/lisa/skills/lisa-pull-request-review/SKILL.md

@@ -37,8 +37,49 @@ gh api graphql -f query='
   -F owner=<owner> -F repo=<repo> -F pr=<pr>
 ```
 
-Keep only threads where `isResolved == false`. If there are none, report success
-and exit (nothing to do).
+Keep only threads where `isResolved == false`.
+
+**If there are none, you have not yet learned anything — run Step 1b before
+concluding.** Zero unresolved threads has two completely different causes, and
+this query cannot tell them apart: *a reviewer looked and found nothing*, or
+*nothing ever looked*. Reporting the first when it was the second is the defect
+in CodySwannGT/lisa#2497.
+
+## Step 1b: Did any review actually happen? (required before reporting)
+
+A required review check can post `success` having reviewed nothing. Measured on
+PRs #2483 and #2484: `CodeRabbit` reported `success — "Review rate limited"`,
+zero reviews, and both merged on that green carrying security-relevant changes.
+
+```bash
+gh pr checks <pr> --json name,state,bucket,description \
+  --jq '.[] | select(.name | test("(?i)coderabbit|review")) | "\(.name)\t\(.state)\t\(.description)"'
+```
+
+**The state column reads `SUCCESS` whether the review was real or hollow — only
+the description distinguishes them.** `Review approved` / `Review completed` is
+a real review; `Review rate limited`, `Review queued`, or a missing context is
+not. Never read `gh pr view --json statusCheckRollup` for this: CodeRabbit posts
+a legacy commit status, which that route returns *without* the description.
+
+Where the project ships Lisa's guard, prefer its machine-readable form, which
+also says whether the check is ruleset-required (so whether branch protection
+recorded a satisfied review gate for a review that did not happen):
+
+```bash
+node scripts/check-skipped-required-checks.mjs --pr=<pr> --json
+```
+
+It reports and never fails — a hollow check is often a vendor spending cap, not
+a repository defect, and it is not this skill's call to block on one.
+
+Carry the finding into your Step 4 report. **Do not write "no unresolved review
+threads" on its own** — it is true of an unreviewed PR too. Say which you
+observed:
+
+- `reviewed — CodeRabbit "Review approved", 0 unresolved threads`
+- `NOT REVIEWED — CodeRabbit success but "Review rate limited" (vacuous); 0 threads means nobody looked`
+- `NOT REVIEWED — no review check reported on this PR at all`
 
 ## Step 2: Triage and act on each unresolved thread
 
@@ -75,6 +116,13 @@ judgment. This skill resolves **threads**; it does not dismiss review-decision
 gates (`CHANGES_REQUESTED`) or merge the PR — the caller (`drive-pr-to-merge`)
 owns those.
 
+**Open the report with the Step 1b verdict, before the thread counts.** A thread
+summary describes what was done about review findings; it says nothing about
+whether a review produced any. State `reviewed` or `NOT REVIEWED (<why>)` first,
+then the counts. A caller that records "reviews addressed" in evidence must
+carry that verdict through verbatim — a PR whose only review check was vacuous
+has not been reviewed, no matter how clean its thread list is.
+
 ## Composition
 
 - **Standalone**: `/lisa:pull-request:review <pr>`.

package/plugins/lisa-agy/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "lisa",
-  "version": "3.6.2",
+  "version": "3.8.0",
   "description": "Universal governance — agents, skills, commands, hooks, and rules for all projects",
   "author": {
     "name": "Cody Swann"

package/plugins/lisa-agy/scripts/design-source-gate.mjs

@@ -294,8 +294,17 @@ export function classifyDesignSource(file, config) {
   return { ...base, status: "undeclared" };
 }
 
-/** Per-file statuses that fail the gate. */
-const VIOLATION_STATUSES = new Set([
+/**
+ * Per-file statuses that fail the gate.
+ *
+ * Exported so the suite can pin the set itself, not just its members' behavior.
+ * Classifying a file `unreadable` is not a guard; *failing the change* because
+ * of it is. Issue #2492 measured the difference: with only classification
+ * asserted, deleting `conflicting` or `unreadable` from this list flipped the
+ * verdict to PASS with the whole suite still green. The pinning test now fails
+ * on any edit to this set — a removal and an addition alike.
+ */
+export const VIOLATION_STATUSES = new Set([
   "undeclared",
   "malformed",
   "conflicting",

package/plugins/lisa-agy/skills/lisa-drive-pr-to-merge/SKILL.md

@@ -291,7 +291,22 @@ Delegate to the `pull-request-review` skill with the PR number. It owns the whol
 comment cycle: fetch every unresolved human + bot thread (with resolution state via
 GraphQL), implement valid feedback (commit + push), reply to invalid feedback, and
 resolve every thread via `resolveReviewThread` so the branch-protection
-thread-resolution gate clears. If that skill needs to push a commit, leave
+thread-resolution gate clears.
+
+**A green review check is not proof a review happened.** That skill's Step 1b
+returns a `reviewed` / `NOT REVIEWED` verdict — record it, and repeat it in this
+skill's final report. Measured (CodySwannGT/lisa#2497): `CodeRabbit` was a
+*required* context and posted `success — "Review rate limited"` on #2483 and
+#2484, so branch protection recorded a satisfied review gate for two
+security-relevant PRs nothing had read; both merged and shipped in `v3.5.1`.
+
+`NOT REVIEWED` is **not a blocker** — do not hold the merge on it, do not treat
+it as a failing check, and do not try to force the bot to re-run. A hollow
+review check is usually an org-wide vendor spending cap, which is a billing
+matter no amount of driving will clear, and whether such a check belongs in the
+required set at all is an open owner decision. It is a **reporting** obligation:
+the PR merged unreviewed, and the report has to say so instead of implying a
+review it did not get. If that skill needs to push a commit, leave
 auto-merge armed (section 1); when it returns, re-read `headRefOid` and reset
 `verify_commit` to the returned/pushed head, then continue. Do not re-implement review handling here
 — it is the single source of truth for review-thread handling.
@@ -452,3 +467,16 @@ At every terminal state, release the babysitter lease
 (`gh pr edit <pr> --remove-label "lisa:babysitter-on-duty"`) so the CI
 auto-fix workflow can take over as fixer of last resort if the branch goes
 red later with nobody driving it.
+
+**Every terminal report carries the step-(d) review verdict**, including a
+successful `MERGED`. "Merged, all checks green" is exactly the sentence that hid
+#2483 and #2484: both were green, both were merged, and neither had been read by
+anything. Green means *no gate objected*; it does not mean *something looked*.
+So state the verdict alongside the outcome:
+
+- `MERGED — reviewed (CodeRabbit "Review approved")`
+- `MERGED — NOT REVIEWED: CodeRabbit posted success but "Review rate limited"`
+
+This is reporting, never a terminal state of its own. `NOT REVIEWED` does not
+turn a merged PR into a blocked one, and it must never be used to withhold a
+merge — it changes what the record says, not what the loop does.

package/plugins/lisa-agy/skills/lisa-pull-request-review/SKILL.md

@@ -37,8 +37,49 @@ gh api graphql -f query='
   -F owner=<owner> -F repo=<repo> -F pr=<pr>
 ```
 
-Keep only threads where `isResolved == false`. If there are none, report success
-and exit (nothing to do).
+Keep only threads where `isResolved == false`.
+
+**If there are none, you have not yet learned anything — run Step 1b before
+concluding.** Zero unresolved threads has two completely different causes, and
+this query cannot tell them apart: *a reviewer looked and found nothing*, or
+*nothing ever looked*. Reporting the first when it was the second is the defect
+in CodySwannGT/lisa#2497.
+
+## Step 1b: Did any review actually happen? (required before reporting)
+
+A required review check can post `success` having reviewed nothing. Measured on
+PRs #2483 and #2484: `CodeRabbit` reported `success — "Review rate limited"`,
+zero reviews, and both merged on that green carrying security-relevant changes.
+
+```bash
+gh pr checks <pr> --json name,state,bucket,description \
+  --jq '.[] | select(.name | test("(?i)coderabbit|review")) | "\(.name)\t\(.state)\t\(.description)"'
+```
+
+**The state column reads `SUCCESS` whether the review was real or hollow — only
+the description distinguishes them.** `Review approved` / `Review completed` is
+a real review; `Review rate limited`, `Review queued`, or a missing context is
+not. Never read `gh pr view --json statusCheckRollup` for this: CodeRabbit posts
+a legacy commit status, which that route returns *without* the description.
+
+Where the project ships Lisa's guard, prefer its machine-readable form, which
+also says whether the check is ruleset-required (so whether branch protection
+recorded a satisfied review gate for a review that did not happen):
+
+```bash
+node scripts/check-skipped-required-checks.mjs --pr=<pr> --json
+```
+
+It reports and never fails — a hollow check is often a vendor spending cap, not
+a repository defect, and it is not this skill's call to block on one.
+
+Carry the finding into your Step 4 report. **Do not write "no unresolved review
+threads" on its own** — it is true of an unreviewed PR too. Say which you
+observed:
+
+- `reviewed — CodeRabbit "Review approved", 0 unresolved threads`
+- `NOT REVIEWED — CodeRabbit success but "Review rate limited" (vacuous); 0 threads means nobody looked`
+- `NOT REVIEWED — no review check reported on this PR at all`
 
 ## Step 2: Triage and act on each unresolved thread
 
@@ -75,6 +116,13 @@ judgment. This skill resolves **threads**; it does not dismiss review-decision
 gates (`CHANGES_REQUESTED`) or merge the PR — the caller (`drive-pr-to-merge`)
 owns those.
 
+**Open the report with the Step 1b verdict, before the thread counts.** A thread
+summary describes what was done about review findings; it says nothing about
+whether a review produced any. State `reviewed` or `NOT REVIEWED (<why>)` first,
+then the counts. A caller that records "reviews addressed" in evidence must
+carry that verdict through verbatim — a PR whose only review check was vacuous
+has not been reviewed, no matter how clean its thread list is.
+
 ## Composition
 
 - **Standalone**: `/lisa:pull-request:review <pr>`.

package/plugins/lisa-cdk/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "lisa-cdk",
-  "version": "3.6.2",
+  "version": "3.8.0",
   "description": "AWS CDK-specific plugin",
   "author": {
     "name": "Cody Swann"

package/plugins/lisa-cdk/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "lisa-cdk",
-  "version": "3.6.2",
+  "version": "3.8.0",
   "description": "AWS CDK-specific Lisa plugin.",
   "author": {
     "name": "Cody Swann"

package/plugins/lisa-cdk-agy/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "lisa-cdk",
-  "version": "3.6.2",
+  "version": "3.8.0",
   "description": "AWS CDK-specific plugin",
   "author": {
     "name": "Cody Swann"

package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "lisa-cdk",
-  "version": "3.6.2",
+  "version": "3.8.0",
   "description": "AWS CDK-specific plugin",
   "author": {
     "name": "Cody Swann"