hex-audit 0.5.5 → 0.6.0

package/dist/skills/{generate-overleaf.md → generate-report.md}

@@ -1,23 +1,44 @@
 ---
-description: "Generate the four LaTeX section files for the final audit report (Overleaf-ready)"
+description: "Generate the LaTeX report sections for the final audit report (Overleaf-ready)"
 ---
 
-# Skill: Generate Overleaf
+# Skill: Generate Report
 
 **Recommended model:** Sonnet
 
-This skill produces the four LaTeX section files an auditor uploads into the Nethermind Overleaf template at the end of an engagement. It runs from the GitHub-synced issues (the report's source of truth), after findings have been pushed via /sync-issues.
+This skill produces the LaTeX section files an auditor assembles into the Nethermind Overleaf report template at the end of an engagement. It runs from the GitHub-synced issues (the report's source of truth), after findings have been pushed via /sync-issues, and reuses the analysis Hex already produced during /init-audit (`overview.md`, `stats.json`, `access-control.json`, `deps.json`).
 
 ## What it writes
 
-Four `.txt` files in `<output_dir>/overleaf/`:
+Seven report sections in `<output_dir>/report/`, each emitted in **two forms** (see "Output convention"):
 
-1. `executive_summary.txt` — `\section{Executive Summary}` body with project synopsis, severity histogram, status histogram, summary table of audit metadata.
-2. `audited_files.txt` — `\section{Audited Files}` body with a LaTeX table of in-scope files (LoC, comments, ratio, blank, total).
-3. `summary_of_findings.txt` — `\section{Summary of Issues}` body with hyperlinked finding rows.
-4. `findings.txt` — `\section{Issues}` body with one `\subsection{[Sev] title}\label{issue:N}` per synced finding, including description, code blocks, recommendation, status, client update.
+1. **Executive Summary** (§1) — project synopsis, severity histogram, status histogram, summary table of audit metadata.
+2. **Audited Files** (§2) — a LaTeX table of in-scope files (LoC, comments, ratio, blank, total).
+3. **Summary of Issues** (§3) — hyperlinked finding rows.
+4. **System Overview** (§4) — descriptive protocol architecture: components, actors & roles, per-module or per-flow subsections. Built from `overview.md` + access-control/deps analysis.
+5. **Issues** (§6) — one `\subsection{[Sev] title}\label{issue:N}` per synced finding, with description, code blocks, recommendation, status, client update.
+6. **Documentation Evaluation** (§7) — a `tcolorbox` remarks paragraph about the client's documentation.
+7. **Test Suite Evaluation** (§8) — coverage summary and automated-tooling note, generated from `stats.json.test_coverage`.
 
-Output is plain `.txt` so the auditor pastes content into the matching Overleaf template slots. The skill does NOT compile LaTeX itself.
+The skill does NOT compile LaTeX itself. Sections §5 (risk methodology) and §9 (conclusion) are static in the Overleaf template and are not generated here.
+
+## Output convention
+
+Every section is generated **once** as a LaTeX string, then written to two paths in `<output_dir>/report/`:
+
+| Section | `.txt` (dashboard viewer / copy-paste) | `.tex` (`\input{}` into `main.tex`) |
+|---|---|---|
+| §1 Executive Summary | `executive_summary.txt` | `01-executive-summary.tex` |
+| §2 Audited Files | `audited_files.txt` | `02-audited-files.tex` |
+| §3 Summary of Issues | `summary_of_findings.txt` | `03-summary-of-findings.tex` |
+| §4 System Overview | `system_overview.txt` | `04-system-overview.tex` |
+| §6 Issues | `findings.txt` | `06-issues.tex` |
+| §7 Documentation Evaluation | `documentation_eval.txt` | `07-documentation-eval.tex` |
+| §8 Test Suite Evaluation | `test_suite.txt` | `08-test-suite.tex` |
+
+The `.txt` and `.tex` bodies are byte-identical. The `.txt` copies power the dashboard `/report` viewer (with copy buttons); the `.tex` copies drop straight into an Overleaf `main.tex` that `\input`s the numbered files.
+
+All prose written by this skill must follow the project writing style (no em dashes, "Consider..." recommendations, banned-word list in the project CLAUDE.md).
 
 ## Phase 0 — Gather report metadata
 
@@ -31,12 +52,14 @@ Read `<output_dir>/config.json`. Look under `settings.report.*` for:
 - `audit_type` — defaults to `Security Review`.
 - `doc_assessment` — `High` / `Medium` / `Low`.
 - `test_assessment` — `High` / `Medium` / `Low`.
+- `kickoff_call` — optional focus note for the §7 kick-off sentence (e.g. "with a focus on the upgrade changes"). Empty means omit the kick-off sentence.
+- `client_availability` — optional free-text for the §7 closing sentence. Empty falls back to the standard availability sentence.
 
 For each missing field, use **AskUserQuestion** to prompt the auditor. After collection, persist all values back to `config.json` under `settings.report.*` so re-runs don't re-prompt.
 
 If `repository_url` is not set, also offer to derive it from the project's git remote (`git -C <project_dir> remote get-url origin`).
 
-## Phase 0.5 — Require synced issues (GitHub is the source of truth)
+## Phase 1 — Require synced issues (GitHub is the source of truth)
 
 The report is generated **only** from issues that are synced to GitHub (`tracking.status === "synced"`). GitHub is the single source of truth for finding content; the local Potential/Verified columns are a staging area.
 
@@ -44,7 +67,7 @@ Before doing anything else:
 - If `settings.github.repo` is not set, **abort**: "The report is generated from GitHub-synced issues. Configure `settings.github.repo` (or pass `--github-repo` to `hex init`) and run `/sync-issues` first."
 - If there are zero `synced` findings, **abort**: "No synced issues found. Verify findings, then run `/sync-issues` to push them to GitHub before generating the report."
 
-## Phase 1 — Compute the histograms and counts
+## Phase 2 — Compute the histograms and counts
 
 Read `<output_dir>/findings.json` and `<output_dir>/tracking.json`.
 
@@ -62,9 +85,9 @@ Resolution histogram (count of synced findings per `resolution` field):
 ```
 Findings without a `resolution` field count as `Unresolved`.
 
-## Phase 2 — Build the protocol synopsis paragraph
+## Phase 3 — Build the protocol synopsis paragraph
 
-Read `<output_dir>/overview.md`. Produce a roughly half-page (~120-180 words) plain-language summary of what the protocol does and the security-relevant mechanics. This becomes the second paragraph of `executive_summary.txt`. Match the tone of the Nethermind example:
+Read `<output_dir>/overview.md`. Produce a roughly half-page (~120-180 words) plain-language summary of what the protocol does and the security-relevant mechanics. This becomes the second paragraph of the Executive Summary. Match the tone of the Nethermind example:
 
 - Open with one sentence on the project's purpose and the in-scope contract(s).
 - Use a short `\begin{itemize}[leftmargin=*,itemsep=4pt] ... \end{itemize}` to list 2-3 of the security-relevant axes (e.g., access-control gates, asset-flow surfaces, upgradeability boundaries).
@@ -75,7 +98,9 @@ Style hints:
 - Bold key terms with `\textbf{...}`.
 - No code blocks in this paragraph — those belong in the per-finding sections.
 
-## Phase 3 — Write `executive_summary.txt`
+## Phase 4 — Write the Executive Summary (§1)
+
+Write to `executive_summary.txt` **and** `01-executive-summary.tex`.
 
 ```latex
 \pagebreak \vspace*{0cm}
@@ -84,7 +109,7 @@ Style hints:
 \black
 This document presents the results of the security review conducted by \href{https://www.nethermind.io/smart-contract-audits}{Nethermind Security} for <PROJECT_NAME>'s <SCOPE_DESCRIPTION>.
 
-<AI-GENERATED PROTOCOL SYNOPSIS PARAGRAPH FROM PHASE 2>
+<AI-GENERATED PROTOCOL SYNOPSIS PARAGRAPH FROM PHASE 3>
 
 \textbf{The audit comprises <TOTAL_NSLOC>} lines of Solidity code. \textbf{The audit was performed using} (a)~manual analysis of the codebase, and (b)~automated analysis tools.
 
@@ -143,7 +168,7 @@ The issues are summarized in Fig.~1.
 \textbf{Final Commit} & \href{<REPOSITORY_URL>/commit/<FINAL_COMMIT>}{<FINAL_COMMIT_SHORT>} \\
 \textbf{Documentation Assessment} & <DOC_ASSESSMENT> \\
 \textbf{Test Suite Assessment} & <TEST_ASSESSMENT> \\ \hline
-\end{longtable>
+\end{longtable}
 \color{black}
 ```
 
@@ -157,7 +182,9 @@ Substitutions:
 - `<INITIAL_COMMIT_SHORT>` ← first 7 chars of the commit hash.
 - `<FIRST_SCOPE_FILE>` ← relative path of the first file in `config.json.project.scope` (for the initial commit hyperref, which the template anchors to a specific file).
 
-## Phase 4 — Write `audited_files.txt`
+## Phase 5 — Write the Audited Files table (§2)
+
+Write to `audited_files.txt` **and** `02-audited-files.tex`.
 
 Compute per-file line counts from each scope file. For each in-scope file in `config.json.project.scope`:
 
@@ -200,9 +227,11 @@ Render the table:
 - Number rows sequentially `1, 2, 3 ...`.
 - Aggregate ratio = `round(sum-comments * 100 / max(sum-code, 1), 0)`.
 
-## Phase 5 — Write `summary_of_findings.txt`
+## Phase 6 — Write the Summary of Issues (§3)
 
-For each synced finding in severity-sorted order (Critical → Info), assign a sequential `issue:N` label starting at `issue:0`. This same label is used in `findings.txt` for the `\hyperref`.
+Write to `summary_of_findings.txt` **and** `03-summary-of-findings.tex`.
+
+For each synced finding in severity-sorted order (Critical → Info), assign a sequential `issue:N` label starting at `issue:0`. This same label is used in the Issues section for the `\hyperref`.
 
 ```latex
 %\pagebreak \vspace*{0cm}
@@ -228,13 +257,59 @@ For each synced finding in severity-sorted order (Critical → Info), assign a s
 ```
 
 - `<N>` is the 1-based row counter.
-- `<I>` matches the `\label{issue:N}` index used in `findings.txt`.
+- `<I>` matches the `\label{issue:N}` index used in the Issues section.
 - `<Severity-display>` maps `Info` → `Info`, others use their name as-is.
 - `<Resolution-display>` maps the `resolution` field directly (`Fixed`, `Acknowledged`, `Mitigated`, `Not Fixed`); blank/`Unresolved` shows `Unresolved`.
 
-## Phase 6 — Write `findings.txt`
+## Phase 7 — Write the System Overview (§4)
+
+Write to `system_overview.txt` **and** `04-system-overview.tex`.
 
-For each synced finding in the same order (severity-sorted), render one subsection. Use the index `i` (0-based) matching `summary_of_findings.txt`.
+Primary source: `<output_dir>/overview.md` (produced by /init-audit — it already carries an intro, a `## Key Contracts` table, `## External Dependencies`, and `## Architecture Notes`). Enrich the actors/roles material from `<output_dir>/access-control.json` (roles, modifiers, gated functions) and cross-contract relationships from `<output_dir>/deps.json`.
+
+**If `overview.md` is absent, abort**: "System Overview needs `<output_dir>/overview.md`. Run `/init-audit` first, then re-run `/generate-report`."
+
+This section is descriptive only: no vulnerability observations (those live in Issues), no findings, no severities.
+
+Structure the LaTeX:
+
+```latex
+\pagebreak \vspace*{0cm}
+\sectioncolor
+\section{System Overview}
+\black
+
+<intro paragraph(s): what the protocol does; if scope is a subset, focus on scope. State what each in-scope contract inherits from using \texttt{}.>
+
+<a short paragraph or itemize listing the main components: one line each — \texttt{ContractName}: responsibility.>
+
+\subsectioncolor
+\subsection{Actors and Roles}
+\black
+<bullet list of actors, their permissions, the functions they can trigger, trust level. If there is a formal access-control system, name the library and explain the role model, using \texttt{ROLE\_NAME} for role constants.>
+
+\subsectioncolor
+\subsection{<Module or Flow Name>}
+\black
+<one subsection per module (Option A) or per flow (Option B). Choose per gauss's rule: if explaining each contract independently makes the system clear, use per-module; if the contracts only make sense together in motion, use per-flow. For each: entry points as a bullet list (functionName: one-line description), main internal flows, key state owned/mutated, how it interfaces with other modules. Use \begin{minted}[linenos]{solidity} ... \end{minted} for signature-only snippets when a flow depends on the parameters. Do not include full function bodies.>
+
+... (repeat subsections) ...
+```
+
+Formatting rules (from gauss `to-latex.md`):
+- Each `\subsection{}` is preceded by `\subsectioncolor` and followed by `\black`. Subsection titles use title case.
+- Contract / function / variable / interface names in `\texttt{}`. Hex values and constants in `\texttt{}`. Standard identifiers (ERC-1271, UUPS, etc.) as-is.
+- Do not start a subsection with a bullet list — add an introductory sentence first.
+- For a genuinely central formula, use an `equation` block. Do not do this for every calculation.
+- Keep it under ~1 page for simple systems, up to ~2 pages for complex ones.
+
+Apply the markdown → LaTeX conversion rules (below) to any prose carried over from `overview.md`.
+
+## Phase 8 — Write the Issues section (§6)
+
+Write to `findings.txt` **and** `06-issues.tex`.
+
+For each synced finding in the same order (severity-sorted), render one subsection. Use the index `i` (0-based) matching the Summary of Issues.
 
 ```latex
 \pagebreak \vspace*{0cm}
@@ -297,7 +372,7 @@ Apply to `description`, `recommendation`, and `update_from_client` (in that orde
 
 End list items with `;` (replace trailing `.` with `;`). Last item still ends with `;`.
 
-The markdown-to-LaTeX logic matches the spirit of the Python script `create-section-findings.py` referenced by the auditor — port that semantically into the skill's instructions rather than calling it.
+These conversion rules match the spirit of the Python script `create-section-findings.py` referenced by the auditor — port that semantically into these instructions rather than calling it. The same rules apply to the System Overview prose in Phase 7.
 
 ### Severity-display mapping
 - `Critical` → `Critical`
@@ -306,16 +381,93 @@ The markdown-to-LaTeX logic matches the spirit of the Python script `create-sect
 - `Low` → `Low`
 - `Info` → `Info` (the Nethermind template uses `Info` in the subsection headers and `Informational` in the histogram caption)
 
-## Phase 7 — Final report
+## Phase 9 — Write the Documentation Evaluation (§7)
+
+Write to `documentation_eval.txt` **and** `07-documentation-eval.tex`.
+
+Determine what documentation was available:
+- NatSpec presence: infer from `stats.json.totals.comment_lines` (a high comment ratio suggests NatSpec) and by skimming 2-3 in-scope files from `config.json.project.scope`.
+- README / external docs: `config.json.project.docs_url` if set, and any README in the repo describing the in-scope system.
+- Kick-off call: only if `settings.report.kickoff_call` is set (append its text as the focus).
+- Client availability: `settings.report.client_availability` if set, otherwise the standard closing sentence.
+
+Emit the exact `tcolorbox` wrapper (from gauss `documentation-eval.md`):
+
+```latex
+\pagebreak \vspace*{0cm}
+\sectioncolor
+\section{Documentation Evaluation}
+\black
+
+\begin{tcolorbox}[colback=blue!5,colframe=blue!50!black!70,title=Remarks about <CLIENT> <PRODUCT> documentation]
+<one paragraph describing the documentation actually provided — NatSpec, README/docs, and what the README covers (high-level overview, objectives, architecture). Only mention what exists and is relevant to the code in scope.>
+\vspace{0.3em}
+
+<kick-off call sentence, only if settings.report.kickoff_call is set. Example: "Additionally, the <CLIENT> team provided a general overview of the system during the kick-off call <kickoff_call>.">
+\vspace{0.3em}
+
+<closing sentence on team availability. Use settings.report.client_availability if set, else: "Moreover, the team addressed all questions and concerns raised by the Nethermind Security team, providing valuable insights and a comprehensive understanding of the project's technical aspects.">
+\end{tcolorbox}
+```
+
+- `<CLIENT>` ← client/org name (derive from `repository_url` org or `project.name`).
+- `<PRODUCT>` ← `config.json.project.name`.
+- Do not include full file paths when referring to a file — use the basename.
+- Keep it factual: do not claim documentation exists that the analysis does not support.
+
+## Phase 10 — Write the Test Suite Evaluation (§8)
+
+Write to `test_suite.txt` **and** `08-test-suite.tex`.
+
+Read `stats.json.test_coverage`.
+
+If `test_coverage.status === "available"`:
+- Report `overall_line_pct` and `overall_branch_pct`.
+- Render a per-contract coverage table from `test_coverage.per_contract[]` (`contract`, `line_pct`, `branch_pct`).
+
+If `test_coverage.status !== "available"` (`failed` / `skipped` / null):
+- State plainly that automated coverage could not be collected (include `failure_reason` if present), and describe the test suite qualitatively, falling back to the `settings.report.test_assessment` grade.
+
+Always close with an automated-tools paragraph: manual analysis plus automated tooling. If `config.json.settings.ai.auditagent_scan_id` is set, name Nethermind AuditAgent explicitly.
+
+```latex
+\pagebreak \vspace*{0cm}
+\sectioncolor
+\section{Test Suite Evaluation}
+\black
+
+<paragraph on the test suite: overall line/branch coverage, or the reason coverage is unavailable, plus a qualitative read tied to the Test Suite Assessment grade.>
+
+\begin{table}[H]
+    \centering
+    \begin{footnotesize}
+    \begin{tabular}{|p{9cm}|p{2cm}|p{2cm}|}
+        \hline
+        \textbf{Contract} & \textbf{Line \%} & \textbf{Branch \%} \\ \hline
+        \texttt{<contract>} & <line_pct>\% & <branch_pct>\% \\ \hline
+        ...
+    \end{tabular}
+    \end{footnotesize}
+\end{table}
+
+<automated-tools paragraph: manual review + automated analysis; name Nethermind AuditAgent if a scan id is configured.>
+```
+
+(Omit the coverage table entirely when coverage is unavailable.)
+
+## Phase 11 — Final report
 
-After writing all four files, print:
+After writing all sections, print:
 
 ```
-Wrote 4 LaTeX section files to <output_dir>/overleaf/:
-  - executive_summary.txt     (<N> bytes)
-  - audited_files.txt          (<N> bytes)
-  - summary_of_findings.txt    (<N> bytes)
-  - findings.txt               (<N> bytes)
+Wrote 7 report sections (14 files) to <output_dir>/report/:
+  §1 Executive Summary       executive_summary.txt     01-executive-summary.tex
+  §2 Audited Files           audited_files.txt         02-audited-files.tex
+  §3 Summary of Issues       summary_of_findings.txt   03-summary-of-findings.tex
+  §4 System Overview         system_overview.txt       04-system-overview.tex
+  §6 Issues                  findings.txt              06-issues.tex
+  §7 Documentation Eval      documentation_eval.txt    07-documentation-eval.tex
+  §8 Test Suite Evaluation   test_suite.txt            08-test-suite.tex
 
 Synced findings included: <count>
   Critical=<C> High=<H> Medium=<M> Low=<L> Info=<I>
@@ -323,12 +475,14 @@ Synced findings included: <count>
 Status distribution:
   Fixed=<F> Acknowledged=<A> Mitigated=<MI> Unresolved=<U>
 
-Upload these into your Overleaf project's Section 1 / 2 / 3 / 6 slots respectively.
+Read the sections on the dashboard's /report page, or \input the NN-*.tex files into your
+Nethermind Overleaf main.tex. Sections 5 (risk methodology) and 9 (conclusion) are template-static.
 ```
 
 ## What this skill does NOT do
 
 - Does not compile LaTeX (no `pdflatex` invocation).
-- Does not modify any file outside `<output_dir>/overleaf/` and (with permission) `config.json` under `settings.report.*`.
+- Does not generate §5 (risk rating methodology) or §9 (conclusion) — those are static in the Overleaf template.
+- Does not modify any file outside `<output_dir>/report/` and (with permission) `config.json` under `settings.report.*`.
 - Does not push to Overleaf, GitHub, or any remote — output is local plain-text.
 - Does not include `rejected`, `duplicate`, `pending_validation`, `unverified`, or `verified`-but-not-yet-synced findings. Only `synced` (on GitHub) makes the cut.

package/dist/skills/ingest-aa-report.md

@@ -142,7 +142,7 @@ Then record the match in `<output_dir>/comparison.json` under `duplicates`. The
 
 If `comparison.json` doesn't exist, create it with `{ "duplicates": [], "novel": [], "rejected": [] }`.
 
-For entries that don't match anything, leave them as `status: "unverified"` (the auditor will run `/validate-issue AA-<NNN>` on them).
+For entries that don't match anything, leave them as `status: "unverified"` (the auditor will run `/validate-finding AA-<NNN>` on them).
 
 ## Step 7 — Report
 
@@ -154,7 +154,7 @@ Ingested N AuditAgent findings (scan <scan_id>).
 - K flagged as Duplicate of existing entries
 - 0 errors
 
-Run /validate-issue <id> on any new finding to confirm and promote it to Verified.
+Run /validate-finding <id> on any new finding to confirm and promote it to Verified.
 ```
 
 ## Outputs (recap)

package/dist/skills/init-audit.md

@@ -25,7 +25,7 @@ Ask the auditor for:
 - **Commit hash** — the specific commit being audited (default: HEAD).
 - **Chain** — target chain (default: ethereum).
 - **Documentation URL** — if available.
-- **GitHub repo** (optional) — the `owner/repo` the audit's findings sync to (e.g. `nethermind/audit-vaultx`). Pass it to `hex init --github-repo` so `/sync-issues` and `/generate-overleaf` work later. Leave blank if the engagement isn't using GitHub.
+- **GitHub repo** (optional) — the `owner/repo` the audit's findings sync to (e.g. `nethermind/audit-vaultx`). Pass it to `hex init --github-repo` so `/sync-issues` and `/generate-report` work later. Leave blank if the engagement isn't using GitHub.
 
 ### 1.2 Audit dependencies (BEFORE compiling)
 
@@ -247,7 +247,7 @@ npx hex issue new --source conformance --source-ref "<spec-item-id, e.g. SC-001>
   --severity "<spec-item.severity_hint, default Info>"
 ```
 
-`--source-ref` is the spec item's own id (e.g. `SC-001`) — Hex keeps that as the back-reference; the board card itself gets the uniform `H-NNN`. These appear on the Issues board as Potential cards immediately; the auditor validates them later with `/validate-issue <id>` (or `/validate-all-findings`).
+`--source-ref` is the spec item's own id (e.g. `SC-001`) — Hex keeps that as the back-reference; the board card itself gets the uniform `H-NNN`. These appear on the Issues board as Potential cards immediately; the auditor validates them later with `/validate-finding <id>` (or `/validate-all-findings`).
 
 **Idempotency:** before creating, check whether a tracking entry already has this `source_ref` (re-running `/init-audit` shouldn't double-create). Skip items whose `source_ref` is already present.
 

package/dist/skills/sync-issues.md

@@ -10,7 +10,7 @@ Hex's GitHub Issues integration. Hex never authenticates with GitHub — you aut
 
 ## The model: GitHub is the source of truth, identity is the issue number
 
-Once an issue is on GitHub it is **read-only in Hex** (status `synced`, locked on the board). Edits — severity, description, recommendation, status — happen on GitHub; this skill pulls them back into the local finding. `/generate-overleaf` reports **only** from synced issues.
+Once an issue is on GitHub it is **read-only in Hex** (status `synced`, locked on the board). Edits — severity, description, recommendation, status — happen on GitHub; this skill pulls them back into the local finding. `/generate-report` reports **only** from synced issues.
 
 **Identity is the GitHub issue number `#N`.** There is no hidden footer in the issue body (auditors create issues by hand, so a footer can't be relied on). Reconciliation matches a local finding to the issue number it has stored in `github.issue_number`. The body is exactly the five-field template, nothing hidden.
 

package/dist/skills/validate-all-findings.md

@@ -6,7 +6,9 @@ description: "Walk every Potential issue on the board and validate them one by o
 
 **Recommended model:** Opus
 
-Runs `/validate-issue` over every issue currently in the **Potential** column, one at a time, so the auditor can clear the backlog in a single pass without naming each id.
+Runs `/validate-finding` over every issue currently in the **Potential** column, one at a time, so the auditor can clear the backlog in a single pass without naming each id.
+
+Each issue gets `/validate-finding`'s full two-axis verdict — correctness **and** the Step-2.5 security-relevance gate. **`auditagent` / `github` findings skew recall-over-precision, so the realism gate is mandatory for them**; `manual` / `conformance` items already passed an auditor's eye and can lean lighter.
 
 ## Step 1 — Build the work list
 
@@ -14,6 +16,16 @@ Read `<output_dir>/tracking.json`. The Potential column is every entry whose `st
 
 If `N` is 0, report "No Potential issues to validate." and stop.
 
+## Step 1.5 — Cheap realism pre-pass
+
+Before deep-diving, do a fast first pass over all `N` and emit a one-line realism read per issue — the Step-2.5 reality-gate take only, no full code trace yet:
+
+```
+<id> [<sev>] <title> — likely: security | by-design | self-inflicted | incorrect | unsure
+```
+
+Print it as a table. This front-loads "which of these even deserve a deep trace" so effort concentrates on the genuine security candidates instead of spreading evenly. It's a triage hint, not a verdict — every issue still gets its real verdict in Step 2.
+
 ## Step 2 — Loop
 
 For each issue `i` of `N` (1-based), print a header:
@@ -22,15 +34,16 @@ For each issue `i` of `N` (1-based), print a header:
 ── Validating i/N: <id> [<severity>] <title> ──
 ```
 
-Then run the full **`/validate-issue`** flow for that `<id>` exactly as that skill defines it:
+Then run the full **`/validate-finding`** flow for that `<id>` exactly as that skill defines it:
 1. Assemble context (the tracking entry + its source record + the affected source).
 2. Trace the claim in the code.
-3. Reach a verdict:
-   - **Valid** → ask PoC-vs-memo (AskUserQuestion), write the validation memo, optionally generate the PoC, then `hex issue patch <id> ...` + `hex issue move <id> --to verified`; offer the severity-adjustment prompt.
-   - **Invalid** → `hex issue move <id> --to invalid`, memo records why.
+3. Reach a verdict (correctness + the Step-2.5 security-relevance gate):
+   - **Verified** (true + security-relevant) → re-rate severity, ask PoC-vs-memo, write the memo, `hex issue patch <id> ...` + `hex issue move <id> --to verified`.
+   - **Rejected — by-design** (true, not security-relevant) → memo with `Disposition: by-design`, `hex issue patch <id> --resolution Acknowledged` + `hex issue move <id> --to rejected`.
+   - **Rejected — incorrect** (false) → `hex issue move <id> --to rejected` (no resolution), memo records why.
    - **Uncertain** → leave in place, `hex issue patch <id> --notes "<open questions>"`.
 
-Do not batch or skip the interactive prompts — this skill is the same per-issue UX as `/validate-issue`, just iterated. The auditor stays in control of each verdict.
+Batch by tier — don't rubber-stamp. **Security-relevant candidates get the full per-issue prompt flow** (PoC-vs-memo and severity), one at a time — the auditor stays in control of every real finding. **Info / by-design / self-inflicted items may be batched**: collect them and confirm their dispositions (and any PoC/severity calls) in a single consolidated **AskUserQuestion** instead of N separate modals. That keeps a large sweep controllable without diluting scrutiny on the findings that matter.
 
 ## Step 3 — Resume safety
 
@@ -42,9 +55,10 @@ After the loop, print a tally:
 
 ```
 Validated N Potential issues:
-  → Verified: V
-  → Invalid:  X
-  → Left for review (uncertain): U
+  → Verified:                       V
+  → Rejected (incorrect):           X
+  → Rejected (valid, not security): B
+  → Left for review (uncertain):    U
 ```
 
-Remind the auditor of the next steps: `/sync-issues` to push the Verified findings to GitHub, then `/generate-overleaf` for the report.
+Remind the auditor of the next steps: `/sync-issues` to push the Verified findings to GitHub, then `/generate-report` for the report.

package/dist/skills/{validate-issue.md → validate-finding.md}

@@ -1,8 +1,8 @@
 ---
-description: "Validate any potential issue on the board (manual / auditagent / conformance / github) and promote it to Verified or Invalid"
+description: "Validate any potential finding on the board (manual / auditagent / conformance / github): check both correctness and security relevance, then route to Verified, Rejected, or leave Uncertain"
 ---
 
-# Skill: Validate Issue
+# Skill: Validate Finding
 
 **Recommended model:** Opus
 
@@ -15,17 +15,17 @@ Takes a single potential issue from the dashboard's Issues board and runs an ind
 - **Conformance** (materialized by `/init-audit`) — `source: "conformance"`, `status: "pending_validation"`.
 - **GitHub** (teammate findings via `/sync-issues`) — `source: "github"`, `status: "pending_validation"`.
 
-The skill writes a validation memo (always), optionally generates a PoC, and updates the tracking entry to `verified` or `rejected`. Card moves on the board.
+The skill judges **two** things, not one: first **correctness** (is the claim true?), then for true claims **security relevance** (is it actually a vuln worth reporting, or true-but-by-design / a missing feature / self-inflicted?). It writes a validation memo (always), optionally generates a PoC, and moves the card. Outcomes: **Verified** (true + security-relevant), **Rejected** (either factually incorrect, *or* true-but-not-a-security-issue), or **Uncertain**. Both Rejected kinds land in the board's **Rejected** column, distinguished by disposition.
 
 ## Invocation
 
 The auditor invokes by id or by free-text reference:
 
 ```
-/validate-issue F003
-/validate-issue SC-007
-/validate-issue AA-N002
-/validate-issue for the rounding issue in Vault.sol
+/validate-finding F003
+/validate-finding SC-007
+/validate-finding AA-N002
+/validate-finding for the rounding issue in Vault.sol
 ```
 
 Resolve to a single tracking entry by:
@@ -65,12 +65,35 @@ For conformance items, the "claim" is `spec_text` + the finding paragraph — i.
 - Check existing protections (modifiers, reentrancy guards, input validation, economic constraints).
 - For conformance items: re-read the cited spec section, compare to the implementation.
 
+### 2.5 Reality gate — is it security-relevant?
+
+Correctness is not enough. AI scanners (and sometimes teammates) surface claims that are **factually true but not security issues** — by-design behavior, missing features, defense-in-depth nice-to-haves, or risks only the deployer can inflict on themselves. This is the precision pass. **Run it for every finding; treat it as mandatory for `source: auditagent | github`** (those skew recall-over-precision). `manual` / `conformance` items already passed an auditor's eye, so they can lean lighter.
+
+For a claim you found **true**, run these five checks — each one pushes toward *not-security-relevant*:
+
+- **Trigger model.** Can only the deployer/owner harm their own deployment? Then it's self-inflicted, not an attack.
+- **Out-of-scope dependency.** Does the impact require an out-of-scope component (e.g. the integrating contract) to be buggy or malicious — and if so, does the finding grant a capability that component didn't already have? No capability gain → defense-in-depth noise.
+- **Trivial workaround.** Is there an integrator-controlled or already-documented mitigation (a constructor param, a config, standard usage)?
+- **Class.** Defect, deliberate design decision, or missing feature? Only defects are findings.
+- **Net test.** Would a senior auditor put this in a client report as a security issue, as an informational note, or not at all?
+
+Carry the answers into the verdict and the memo (Triggerability / Depends-on-out-of-scope / Disposition).
+
 ### 3. Verdict
 
-**Valid:**
+Combine the two axes — correctness (Step 2) and security relevance (Step 2.5):
+
+| Correctness | Security-relevant? | → Outcome |
+|---|---|---|
+| True | Yes | **Verified** |
+| True | No | **Rejected** (by-design / informational) |
+| False | — | **Rejected** (incorrect) |
+| Uncertain | — | **Uncertain** |
+
+**Verified — true and security-relevant:**
 
 1. Use **AskUserQuestion** to ask:
-   > The issue appears **VALID**. How would you like to proceed?
+   > The finding appears **VALID and security-relevant**. How would you like to proceed?
    >
    > 1. Generate a full PoC test, then promote to Verified.
    > 2. Accept with rational verification only — validation memo, no PoC.
@@ -81,7 +104,9 @@ For conformance items, the "claim" is `spec_text` + the finding paragraph — i.
 
 3. **If memo-only chosen:** no PoC file.
 
-4. **Persist via the CLI — do not hand-edit `findings.json` or `tracking.json`.** The `hex issue` command is the single source of truth for board mutations; it materializes a `findings.json` entry from the source record (conformance / auditagent / github) when one does not exist yet, and routes fields correctly. Write the validation memo and the recommendation to temp files, then:
+4. **Calibrate severity — don't inherit it.** The source's severity (especially AuditAgent's) is a *hypothesis*, not authoritative. Re-derive it from realistic impact × likelihood; many technically-true findings collapse to Info. A finding whose worst case is self-inflicted or chain-conditional must say so and re-rate accordingly.
+
+5. **Persist via the CLI — do not hand-edit `findings.json` or `tracking.json`.** The `hex issue` command is the single source of truth for board mutations; it materializes a `findings.json` entry from the source record (conformance / auditagent / github) when one does not exist yet, and routes fields correctly. Write the validation memo and the recommendation to temp files, then:
 
    ```bash
    # Land the editable fields (materializes a finding for conformance/auditagent/github ids):
@@ -99,21 +124,34 @@ For conformance items, the "claim" is `spec_text` + the finding paragraph — i.
 
    Hex does not write `@audit-issue` comments into the client's source files — the board and finding record are the only place the issue lives.
 
-5. Use **AskUserQuestion** to ask:
+6. Use **AskUserQuestion** to ask:
    > Issue verified as **\<id\>** with severity **\<current severity\>**.
    >
    > Would you like to change the severity? Reply with the new severity (Critical / High / Medium / Low / Info), or **no** to keep it.
 
-6. If the user provides a new severity, run `npx hex issue patch <id> --severity <NewSeverity>`.
+7. If the user provides a new severity, run `npx hex issue patch <id> --severity <NewSeverity>`.
 
-**Invalid:**
+**Rejected — true but not a security issue (by-design / informational):**
 
-1. Explain why:
-   - Is the attack path blocked? By what specifically (modifier, require, state check)?
-   - Are the preconditions impossible? Why?
-   - Is the impact assessment wrong? What actually happens?
-2. Move it to the Invalid column: `npx hex issue move <id> --to invalid`.
-3. Do NOT write a finding body. The memo alone records the reasoning.
+The claim is correct, but Step 2.5 says it isn't a vuln worth reporting (self-inflicted, design decision, missing feature, defense-in-depth noise). This is **not** the same as "incorrect" — give it its own disposition so it's distinguishable from a false positive and AuditAgent gets honest feedback.
+
+1. Write the memo with `Disposition: by-design` (or `informational`) and the Step-2.5 reasoning.
+2. Mark the disposition and move it:
+   ```bash
+   npx hex issue patch <id> --resolution Acknowledged
+   npx hex issue move <id> --to rejected
+   ```
+   `Acknowledged` on a Rejected card means "AA was right, but it's by-design / not a security issue" — the board shows a **By-design** chip. (A Rejected card with *no* resolution = factually incorrect; see below.)
+3. Do NOT write a finding body or carry it into the report.
+
+**Rejected — incorrect (false positive):**
+
+The claim is factually wrong. Explain why:
+- Is the attack path blocked? By what specifically (modifier, require, state check)?
+- Are the preconditions impossible? Why?
+- Is the impact assessment wrong? What actually happens?
+
+Move it to the Rejected column with **no** resolution (the board shows an **Invalid** chip): `npx hex issue move <id> --to rejected`. Do NOT write a finding body — the memo records the reasoning.
 
 **Uncertain:**
 
@@ -126,11 +164,15 @@ For conformance items, the "claim" is `spec_text` + the finding paragraph — i.
 Write `<output_dir>/validations/<id>_memo.md` (always — for valid, invalid, and uncertain verdicts):
 
 ```markdown
-# Issue Validation: <ID> — <Title>
+# Finding Validation: <ID> — <Title>
 
 **Source:** <manual | auditagent | conformance | github>
 **Original record:** <pointer to the specific record being validated — e.g., findings.json#F003, ai-results/auditagent/findings.json#AA-014>
-**Verdict:** Valid / Invalid / Uncertain
+**Correctness:** True / False / Uncertain
+**Security relevance:** security-relevant / not-security-relevant / n/a
+**Triggerability:** external | self-inflicted
+**Depends on out-of-scope component:** no | yes (capability gain: yes/no)
+**Disposition:** security-finding | by-design | informational | false
 **Date:** <YYYY-MM-DD>
 
 ## Claim being validated
@@ -142,13 +184,17 @@ Write `<output_dir>/validations/<id>_memo.md` (always — for valid, invalid, an
 ## Protections evaluated
 [Access control, reentrancy guards, input validation, economic constraints, etc.]
 
+## Reality gate
+[The five Step-2.5 checks for true claims: trigger model, out-of-scope dependency, trivial workaround, class, net test. Skip with "n/a (claim is false)" for incorrect findings.]
+
 ## Verdict reasoning
-[Why valid / invalid / uncertain]
+[Why Verified / Rejected (by-design or incorrect) / Uncertain — tie it to the two axes above]
 
 ## Action taken
-[If valid + PoC: PoC at <path>, test passing, finding F<NNN> updated/created]
-[If valid + memo-only: finding F<NNN> updated/created, no PoC]
-[If invalid: tracking entry rejected, no finding written]
+[If Verified + PoC: PoC at <path>, test passing, finding updated/created; severity re-rated to <X>]
+[If Verified + memo-only: finding updated/created, no PoC; severity <X>]
+[If Rejected by-design: moved to Rejected with resolution Acknowledged, no finding written]
+[If Rejected incorrect: moved to Rejected, no resolution, no finding written]
 [If uncertain: open questions listed; tracking status unchanged]
 ```
 
@@ -158,4 +204,4 @@ The board reflects the new state automatically (the dashboard watches `tracking.
 
 - Move on to the next Potential card.
 - Run `/sync-issues` to push the newly Verified finding to GitHub if `settings.github.repo` is configured.
-- Run `/generate-overleaf` once all verifications are done, to produce the LaTeX report sections.
+- Run `/generate-report` once all verifications are done, to produce the LaTeX report sections.