@codyswann/lisa 2.263.0 → 2.265.0

package/plugins/src/base/skills/lisa-security-review/SKILL.md

@@ -16,6 +16,57 @@ Identify vulnerabilities, evaluate threats, and recommend mitigations for code c
 5. **Check auth/authz** -- are access controls properly enforced for new endpoints or features?
 6. **Review dependencies** -- do new dependencies introduce known vulnerabilities?
 
+## The impact-or-exploitability bar
+
+Severity is **earned, not pattern-matched**. Every security-shaped finding is classified
+mechanically, before it is written up:
+
+| Field | What it holds |
+|-------|---------------|
+| `reproducer` | an evidence ref of a kind that reaches the claim's boundary, or `none` |
+| `impact` | a bounded impact/exploitability statement (who can do what, to what data, under what preconditions), or `unproven` |
+| `reason` | one line saying why the finding landed in its bucket |
+
+**The bar:** a finding is **proven** only when it carries **both** a reproducer **and** a bounded
+impact statement. **Missing either ⇒ unproven.** No other input changes the bucket.
+
+What counts as a reaching reproducer is defined by the `claim-evidence-mapping` contract (**BCE-1**,
+#1835), not here: an injection claim at the `http-api` boundary needs an `http-transcript`; a UI
+claim needs a `screenshot` or `recording`. A passing unit `test-run-log` reaches `code-unit` only and
+never discharges either.
+
+**Each field stands on its own.** The two halves are recorded independently: a finding with a bounded
+impact but no reproducer keeps its impact statement verbatim and only `reproducer` reads `none`; a
+finding with a reproducer but no bounded impact keeps the evidence ref and only `impact` reads
+`unproven`. **Never overwrite** a field you actually have with a missing-value placeholder — the
+`reason` line names which half is missing, and the surviving half is the head start the next reviewer
+needs.
+
+## The two buckets — conservative by default
+
+Findings render in two clearly-labeled buckets: **Security (proven)** and **Security (unproven)**.
+
+A reproducer-less finding **stays in the security section**, labeled `unproven` with its reason. It
+is **never auto-demoted** to a `maintenance` bucket and it is **not removed** from the report —
+under-reporting a real vulnerability is the worse failure, so the conservative default keeps it
+visible where a security reader looks.
+
+**Single policy point.** The unproven bucket's label is the only thing an owner may change:
+`security.review.unprovenBucket` in `.lisa.config.json`, default `security-unproven`. An owner who
+prefers true demotion sets it to a maintenance label; the finding then renders under that bucket and
+**no other classification logic changes** — the bar, the fields, and the reasons are identical.
+
+Write both buckets in operator voice (`factory-model` rule 5): a person who does not code reads this
+at the gate. "Anyone who can reach the search box can read other customers' orders — reproduced with
+the request transcript below" is usable; "possible SQLi in handler" is not.
+
+This bar governs *code-review* security findings. Dependency CVE remediation keeps its own decision
+ladder in the `security-audit-handling` rule — cite it, do not restate or fork it.
+
+Bucketing is **advisory** — it shapes the report, it does not block a merge — on the same terms as
+the boundary checks, which stay reporting-only until `verification.gate.enforceBoundaries` is `true`
+in `.lisa.config.json`.
+
 ## Output Format
 
 Structure findings as:
@@ -41,17 +92,33 @@ Structure findings as:
 - [ ] No XSS vectors in user-facing output
 - [ ] Dependencies free of known CVEs
 
-### Vulnerabilities Found
-- [vulnerability] -- where in the code, how to prevent
+### Security (proven)
+- [finding] -- where in the code, how to prevent
+  - reproducer: [evidence ref, e.g. evidence/<ticket>/http-transcript-01.txt]
+  - impact: [who can do what, to what data, under what preconditions]
+  - reason: reproducer + bounded impact
+
+### Security (unproven)
+- [finding] -- where in the code, how to prevent
+  - reproducer: [evidence ref if one exists, else `none`]
+  - impact: [bounded statement if one exists, else `unproven`]
+  - reason: [which half is missing -- e.g. "impact bounded, but never reproduced"]
+    -- kept in the security section, not demoted
 
 ### Recommendations
 - [recommendation] -- priority (critical/warning/suggestion)
 ```
 
+Rename the unproven heading only when `security.review.unprovenBucket` is set to something other
+than `security-unproven`; everything else stays as written.
+
 ## Rules
 
 - Focus on the specific changes proposed, not a full security audit of the entire codebase
 - Flag only real risks -- do not invent hypothetical threats for internal tooling with no user input
+- Classify every finding against the bar before writing it up; never leave a finding unbucketed
+- Never silently drop or downgrade a finding out of the security section -- `unproven` is the
+  conservative landing spot, and the reason line says why
 - Prioritize OWASP Top 10 vulnerabilities
 - If the changes are purely internal (config, refactoring, docs), report "No security concerns" and explain why
 - Always check `.gitleaksignore` patterns to understand what secrets scanning is already in place

package/plugins/src/base/skills/lisa-security-zap-scan/SKILL.md

@@ -22,10 +22,30 @@ Run a ZAP baseline security scan against the local application.
    - After the scan completes, read `zap-report.html` (or `zap-report.md` for text)
    - Summarize findings:
      - Total number of alerts by risk level (High, Medium, Low, Informational)
-     - List each Medium+ finding with its rule ID, name, and recommended fix
+     - **Every alert reaches classification** -- High, Medium, Low, and Informational alike. Risk
+       level orders the summary; it never filters it. **Nothing is dropped before classification**,
+       so no alert can leave the report unclassified. Medium+ alerts are listed first, in full (rule
+       ID, name, recommended fix); Low/Informational alerts are still listed, bucketed, and given a
+       `reason`, even when compressed to one line each.
      - Categorize findings as "infrastructure-level" (fix at CDN/proxy) vs "application-level" (fix in code)
 
-4. **Handle failures**:
+4. **Apply the impact-or-exploitability bar** -- the same bar the `lisa-security-review` skill
+   defines; follow that skill, do not restate it. A ZAP alert is not a reproducer by itself: the
+   alert names a pattern, not an exercised impact path.
+   - **Security (proven)** -- the alert carries a reproducer **and** a bounded impact statement. The
+     reproducer counts only if its evidence kind **reaches the claim's boundary** under the
+     `claim-evidence-mapping` contract (BCE-1, #1835): a ZAP request/response transcript is an
+     `http-transcript` and reaches the `http-api` boundary only. An alert whose claim is about
+     rendered UI (`browser`) or persisted state (`data`) needs evidence at *that* boundary -- a
+     transcript never proves it.
+   - **Security (unproven)** -- everything else, each with a one-line `reason` (typically
+     "alert only, no reproducer / no bounded impact", or "transcript does not reach the claim's
+     boundary"). Unproven alerts are **not dropped** and not demoted out of the security summary --
+     they render in the unproven bucket so a reader still sees them.
+   - Rename the unproven heading only if `security.review.unprovenBucket` is set to something other
+     than `security-unproven`; no other classification changes.
+
+5. **Handle failures**:
    - If the scan failed, explain what failed and suggest concrete remediation steps
 
 ## Execution

package/plugins/src/base/skills/lisa-tracker-evidence/SKILL.md

@@ -28,6 +28,10 @@ See the `config-resolution` rule for configuration and dispatch table.
 - Never post evidence to a different ticket than the one named — `$ARGUMENTS` is the source of truth.
 - Never invent a verify-specific usage footer. Evidence artifact usage must flow through `lisa-usage-accounting`, preserve the canonical `## Lisa Usage` section, and surface `source: unavailable` explicitly when the runtime cannot provide trustworthy numbers.
 - **Evidence-manifest gate (leaf work units).** Before dispatching to a vendor skill that transitions the ticket, confirm `EVIDENCE_DIR` contains a non-empty artifact **of the declared type** for every typed `[EVIDENCE: <artifact-type>: <name>]` marker declared in the ticket's Validation Journey — a `screenshot` marker needs an actual image, an `http-transcript` marker needs the request + response text, a `perf-trace` marker needs measured numbers; a prose claim satisfies nothing. If any declared marker has no captured artifact, an empty one, or one whose content/extension does not match its declared type, stop and report the offending markers by name instead of posting — a leaf work unit (Bug / Task / Sub-task / Improvement) may not advance to its review/Done state with an unsatisfied manifest (see the "Per-Work-Unit Evidence Contract" in the `verification` rule). Epics / Stories / Spikes, and leaf units without a Validation Journey, are exempt.
+- **Claim↔boundary binding (S14 upgrade).** Satisfying the manifest by *type* is not enough: each `[EVIDENCE: <artifact-type>: <name>]` marker is also cited for a **claim**, and the marker's artifact type must **reach the boundary that claim declares** per the `claim-evidence-mapping` rule's taxonomy. A `browser` claim (user-visible UI behavior) is reached by `screenshot` / `recording` and never by a unit `test-run-log`; an `http-api` claim needs an `http-transcript`; a `deploy-health` claim needs a `deploy-log` and no pre-deploy artifact. On a mismatch, report the offending marker **by name** together with the claim, its boundary, and the **required evidence kinds** for that boundary — e.g. `[EVIDENCE: test-run-log: unit-suite] cited for claim AC-2 [boundary browser] — required evidence kinds: screenshot, recording`. Swapping in a marker of a reaching kind with a real captured artifact satisfies the gate. **Advisory-first:** until `verification.gate.enforceBoundaries` is `true` in `.lisa.config.json` (the same ratchet flag the Stop-hook gate reads, default `false`), a boundary mismatch is reported to the operator but does not block the post; once ratcheted on it refuses the post exactly like a missing or wrong-type artifact. Missing/empty/wrong-type artifacts keep refusing the post regardless of the flag.
+- **The "Not established" section is required.** Before posting, confirm `evidence/comment.md` (and `comment.txt` where the vendor uses it) contains a `## Not established` heading and that the verdict carries `not_established_reviewed: true`. The heading is **never omitted and never blank**: with nothing outstanding it still renders `None outstanding — reviewed`; otherwise it lists, in plain operator language, each thing the verification did not prove — boundaries not exercised, environments not tested, behavior consciously out of scope. A comment with no such heading, a heading with nothing under it, or a verdict whose `not_established_reviewed` flag is absent is **refused**: stop and report the missing Not-established review instead of posting. (The list may be empty; the flag may never be omitted.) Definition and exemplars live in the `claim-evidence-mapping` rule; this generalizes `lisa-improve-harness`'s required, never-empty `Known limits` field.
+- **Artifact identity is required, with values (S14 extension).** Before posting, confirm the comment body carries a `## Artifact identity` heading populated with **values, not placeholders**: the `repository`, the `head_sha` the verification observed, the `environment`, and for each committed artifact its `sha256` digest and `captured_at`. Then check both identity failures. **`artifact_mismatch`** — an evidence entry whose recorded `artifact_head_sha` differs from the verdict's `artifact.head_sha`: refuse the post and report the offending evidence id together with **both SHAs** (the one the artifact was captured at and the one the verdict claims), e.g. `EV-2 captured at 4f1c9ab but artifact.head_sha is 9de0c31`. **`evidence_digest_mismatch`** — recompute the `sha256` of each committed evidence file on read; if the bytes no longer match the recorded digest (or the file is absent), stop and report the evidence id by name. Never summarize either as "verification failed" — name the field, the ids, and both SHAs so a non-engineer can read it at the gate. **Advisory-first:** until `verification.gate.enforceBoundaries` is `true` in `.lisa.config.json`, an identity failure is reported to the operator but does not block the post; once ratcheted on it refuses the post exactly like a missing artifact. Definition: the `claim-evidence-mapping` rule.
+- **Merge-race reconciliation (at completion, not at post).** The evidence's pinned `head_sha` is reconciled against the merged head using the ancestry + deploy-run definition of "what shipped" that `lisa-drive-pr-to-merge` already owns — cite that skill, never re-implement or restate its checks here. Pre-merge evidence counts for the merge commit only when its head is a parent of the merge, ancestry alone never justifies reporting shipped, and on a mismatch verification re-runs against the merged head before completion is declared.
 - **Evidence references are not manifest entries.** Extract obligations using the exact `[EVIDENCE:` prefix (and the legacy local `[SCREENSHOT:` form). Exclude both the canonical `[EVIDENCE-REF: <work-item-ref> | <artifact-type>: <kebab-case-name>]` and the Lisa 2.223.0 legacy alias `[EVIDENCE-REF: <tracker-ref>: <artifact-type>: <kebab-case-name>]` from artifact lookup, missing-artifact reporting, and duplicate-name checks: either belongs to another work item and cannot satisfy this item's S14 gate. A runtime-changing leaf with references but no local claiming marker must be rejected before dispatch.
 
 ## UI Evidence Checklist (when work is UI-visible)
@@ -47,8 +51,11 @@ The checklist is tracker-agnostic — the same shape works on JIRA, GitHub Issue
 5. **"What this shows" section.** Tailor to ticket type:
    - **Bug repro:** state plainly whether the bug reproduces or not, and the most likely 1–2 reasons their retest still failed (different env, native app vs. web, stuck backend row, etc.).
    - **Feature/UX completion:** state plainly which acceptance criteria each screenshot covers, and call out any deferred or out-of-scope surface explicitly so QA/PM doesn't have to infer.
-6. **"What would help me confirm" (bug) / "How to QA" (feature) section.** Concrete actionable retest steps with the exact selection criteria (e.g., "pick a record whose Status column shows `—`, not `Processing` or `Pending Review`").
-7. **Explicit invitation to be corrected.** End with a line like *"If any of the steps I listed are different from what you expected / actually did, please tell me explicitly which step I got wrong."* Non-optional — small differences (which record, which device, exact tap order, expected behavior) change everything, and naming the door open short-circuits ticket bounce-loops.
-8. **Workflow transition** is the vendor skill's job, not yours — it'll move the ticket per the configured tracker (JIRA: Reassign to reporter for bug repro / move to the configured review status when one exists, otherwise leave it in `claimed`; GitHub: direct `claimed` → configured `done` after a successful build; Linear: equivalent state). You don't transition manually.
+6. **"Artifact identity" and "Not established" sections (both required).** Right after "What this shows":
+   - `## Artifact identity` — what the evidence was collected against, as **values, not placeholders**: the repository, the `head_sha` the verification observed, the `environment`, and per artifact its `sha256` digest and `captured_at`. A heading rendered with `<unknown>` in place of a SHA is not identity.
+   - `## Not established` — **never omitted, never blank.** List in plain language what this verification did *not* prove: boundaries not exercised (e.g. "the persisted order row was never queried"), environments not tested (e.g. "checked on desktop Chrome only — not mobile Safari"), and behavior consciously out of scope (e.g. "refunds were not touched or tested"). Name the specific thing, not a category. With nothing outstanding, the heading still renders a single line: `None outstanding — reviewed`. Never state a quality check as proof of behavior — "unit tests pass" belongs here as a limit, not above as evidence.
+7. **"What would help me confirm" (bug) / "How to QA" (feature) section.** Concrete actionable retest steps with the exact selection criteria (e.g., "pick a record whose Status column shows `—`, not `Processing` or `Pending Review`").
+8. **Explicit invitation to be corrected.** End with a line like *"If any of the steps I listed are different from what you expected / actually did, please tell me explicitly which step I got wrong."* Non-optional — small differences (which record, which device, exact tap order, expected behavior) change everything, and naming the door open short-circuits ticket bounce-loops.
+9. **Workflow transition** is the vendor skill's job, not yours — it'll move the ticket per the configured tracker (JIRA: Reassign to reporter for bug repro / move to the configured review status when one exists, otherwise leave it in `claimed`; GitHub: direct `claimed` → configured `done` after a successful build; Linear: equivalent state). You don't transition manually.
 
 **Why this format:** It (a) gives the reporter a frame-by-frame they can compare against, (b) avoids the JIRA image-collapse failure mode while still working everywhere else, (c) names the most plausible discrepancies up front so the loop short-circuits, (d) explicitly opens the door to being corrected so tickets don't bounce on assumed alignment. The same mechanics that resolve a stuck bug ticket also give QA an unambiguous handoff for a freshly-built feature.