mandrel 2.3.0 → 2.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. package/.agents/audit-checklists/accessibility.md +29 -0
  2. package/.agents/audit-checklists/architecture.md +4 -5
  3. package/.agents/audit-checklists/clean-code.md +10 -0
  4. package/.agents/audit-checklists/data-model.md +22 -0
  5. package/.agents/audit-checklists/dependencies.md +11 -2
  6. package/.agents/audit-checklists/devops.md +4 -0
  7. package/.agents/audit-checklists/navigability.md +3 -0
  8. package/.agents/audit-checklists/performance.md +8 -11
  9. package/.agents/audit-checklists/privacy.md +3 -4
  10. package/.agents/audit-checklists/quality.md +2 -0
  11. package/.agents/audit-checklists/security.md +4 -5
  12. package/.agents/audit-checklists/seo.md +7 -1
  13. package/.agents/audit-checklists/sre.md +14 -12
  14. package/.agents/audit-checklists/ux-ui.md +4 -0
  15. package/.agents/docs/configuration.md +3 -0
  16. package/.agents/docs/workflows.md +4 -3
  17. package/.agents/schemas/agentrc.schema.json +17 -0
  18. package/.agents/schemas/audit-rules.json +134 -19
  19. package/.agents/schemas/audit-rules.schema.json +6 -2
  20. package/.agents/scripts/audit-labels-bootstrap.js +4 -4
  21. package/.agents/scripts/audit-to-stories.js +244 -19
  22. package/.agents/scripts/lib/audit-suite/checklist-threading.js +26 -3
  23. package/.agents/scripts/lib/audit-suite/dispatch-checklist.js +132 -0
  24. package/.agents/scripts/lib/audit-suite/index.js +1 -0
  25. package/.agents/scripts/lib/audit-suite/selector.js +290 -14
  26. package/.agents/scripts/lib/audit-to-stories/audit-lenses.js +2 -1
  27. package/.agents/scripts/lib/audit-to-stories/build-story-body.js +5 -1
  28. package/.agents/scripts/lib/audit-to-stories/dedupe-against-github.js +23 -3
  29. package/.agents/scripts/lib/audit-to-stories/finding-adapter.js +38 -0
  30. package/.agents/scripts/lib/audit-to-stories/ledger.js +256 -0
  31. package/.agents/scripts/lib/audit-to-stories/parse-audit-md.js +41 -7
  32. package/.agents/scripts/lib/audit-to-stories/seed-from-findings.js +20 -2
  33. package/.agents/scripts/lib/command-header.js +1 -1
  34. package/.agents/scripts/lib/config-settings-schema-delivery.js +21 -0
  35. package/.agents/scripts/lib/dynamic-workflow/performance-report-contract.js +5 -3
  36. package/.agents/scripts/lib/feedback-loop/audit-results-graduator.js +56 -0
  37. package/.agents/scripts/lib/findings/route-finding.js +108 -10
  38. package/.agents/scripts/lib/observability/runtime-friction.js +62 -0
  39. package/.agents/scripts/lib/orchestration/resolve-stories.js +9 -1
  40. package/.agents/scripts/lib/orchestration/retro-proposals.js +49 -1
  41. package/.agents/scripts/lib/orchestration/single-story-close/phases/post-land.js +73 -15
  42. package/.agents/scripts/lib/orchestration/story-close/phases/local-lens-review.js +81 -1
  43. package/.agents/scripts/lib/orchestration/story-close/phases/review-core.js +1 -0
  44. package/.agents/scripts/lib/orchestration/ticket-lease.js +78 -10
  45. package/.agents/scripts/lib/orchestration/ticketing/transition.js +68 -16
  46. package/.agents/scripts/lib/single-story-sweep/sweep-lock.js +73 -0
  47. package/.agents/scripts/lib/wave-runner/live-probe.js +93 -12
  48. package/.agents/scripts/nav-registry-diff.js +449 -0
  49. package/.agents/scripts/single-story-init.js +189 -51
  50. package/.agents/scripts/stories-wave-tick.js +39 -2
  51. package/.agents/workflows/audit-accessibility.md +243 -0
  52. package/.agents/workflows/audit-architecture.md +89 -71
  53. package/.agents/workflows/audit-clean-code.md +87 -53
  54. package/.agents/workflows/audit-data-model.md +198 -0
  55. package/.agents/workflows/audit-dependencies.md +143 -28
  56. package/.agents/workflows/audit-devops.md +109 -18
  57. package/.agents/workflows/audit-documentation.md +25 -53
  58. package/.agents/workflows/audit-navigability.md +78 -22
  59. package/.agents/workflows/audit-performance.md +207 -103
  60. package/.agents/workflows/audit-privacy.md +51 -13
  61. package/.agents/workflows/audit-quality.md +71 -61
  62. package/.agents/workflows/audit-security.md +94 -71
  63. package/.agents/workflows/audit-seo.md +80 -25
  64. package/.agents/workflows/audit-sre.md +99 -66
  65. package/.agents/workflows/audit-to-stories.md +44 -5
  66. package/.agents/workflows/audit-ux-ui.md +71 -17
  67. package/.agents/workflows/deliver.md +32 -8
  68. package/.agents/workflows/helpers/audit-dual-path.md +59 -0
  69. package/.agents/workflows/helpers/audit-self-check.md +70 -0
  70. package/.agents/workflows/helpers/audit-severity-scale.md +19 -0
  71. package/.agents/workflows/helpers/deliver-story.md +25 -0
  72. package/docs/CHANGELOG.md +29 -0
  73. package/package.json +1 -1
  74. package/.agents/audit-checklists/lighthouse.md +0 -15
  75. package/.agents/schemas/audit-results.schema.json +0 -69
  76. package/.agents/workflows/audit-lighthouse.md +0 -269
@@ -38,89 +38,98 @@ before this section existed.
38
38
 
39
39
  ## Execution strategy (dual-path)
40
40
 
41
- This lens runs along one of two execution paths. Both emit the **identical**
42
- report contract (Step 3); downstream consumers (`audit-to-stories`) are agnostic to which path produced it.
43
-
44
- - **Orchestrated (dynamic-workflow) path.** When Claude Code's
45
- [dynamic workflows](https://code.claude.com/docs/en/workflows) are
46
- available, the saved project workflow
47
- `.claude/workflows/audit-security.workflow.js` fans the dimensions below
48
- out as parallel read-only subagents, runs an **adversarial cross-check**
49
- stage (an independent agent reviews each dimension's findings and drops
50
- false positives before they enter the report), then synthesises the Step 3
51
- report. The orchestrator derives its per-dimension prompts from *this*
52
- markdown at run time the lens stays the single source of truth; the
53
- script does not fork a second copy of the spec.
54
- - **Sequential (single-pass) path.** When dynamic workflows are unavailable,
55
- follow Steps 1–3 below turn-by-turn exactly as before. This is the default
56
- fallback and changes nothing about the existing behaviour.
57
-
58
- **Strategy selection** is computed by
59
- [`lib/dynamic-workflow/capability.js`](../scripts/lib/dynamic-workflow/capability.js)
60
- (`selectAuditStrategy`). The orchestrated path is chosen only when the runtime
61
- is Claude Code, `disableWorkflows` is not set (settings.json **or**
62
- `CLAUDE_CODE_DISABLE_WORKFLOWS`), and the Claude Code version meets the
63
- research-preview floor (`>= 2.1.154`). Any other runtime, a disabled setting,
64
- or an older version degrades gracefully to the sequential path.
65
-
66
- > **Capability degradation, not a contract shim.** This dual path is **not**
67
- > covered by the No-Shim / hard-cutover rule in
68
- > [`git-conventions.md`](../rules/git-conventions.md). That rule forbids
69
- > running two shapes of the *same contract* side by side. Here there is **one**
70
- > report contract; only the *execution strategy* is selected from a runtime
71
- > capability — the same pattern the protocol already endorses for live-docs
72
- > fallback in [`instructions.md` §1.C/§1.D](../instructions.md). The full
73
- > capability-degradation rationale lives in the
74
- > [`capability.js`](../scripts/lib/dynamic-workflow/capability.js) module
75
- > docstring; the orchestrated-run evidence and per-lens cost/precision gate
76
- > verdicts live in [`docs/roadmap.md`](../../docs/roadmap.md) (Part 3 —
77
- > Dynamic-Workflow Orchestration).
78
-
79
- **Forcing a path (for testing).** Set `MANDREL_AUDIT_STRATEGY=sequential` to
80
- verify the fallback path with the feature notionally disabled, or
81
- `MANDREL_AUDIT_STRATEGY=orchestrated` to pin the dynamic path. To exercise the
82
- real disable signals instead, set `CLAUDE_CODE_DISABLE_WORKFLOWS=1` (env) or
83
- `disableWorkflows: true` in `.claude/settings.json` and re-run the lens — both
84
- degrade to the sequential path.
85
-
86
- > **Read-only on both paths.** The lens is read-only (see Constraint). The
87
- > orchestrated subagents run in `acceptEdits` and inherit the session tool
88
- > allowlist, but the workflow script grants the analysis agents only
89
- > read/search tools (`Read`, `Grep`, `Glob`) — no write/edit/shell-mutation
90
- > tools. The single write in an orchestrated run is the final report artifact.
91
-
92
- ## Step 1: Vulnerability Surface Analysis
41
+ This lens runs along one of two execution paths (orchestrated dynamic-workflow
42
+ or sequential single-pass). Both emit the **identical** Step 3 report contract;
43
+ downstream consumers (`audit-to-stories`) are agnostic to which path produced
44
+ it. See [`helpers/audit-dual-path.md`](helpers/audit-dual-path.md) for strategy
45
+ selection, the forcing flags, and the read-only guarantee — read `audit-<lens>`
46
+ there as this lens's name.
47
+
48
+ ## Rubric `rules/security-baseline.md` is the contract
49
+
50
+ This lens does **not** grade against recalled OWASP lore. The authoritative
51
+ rubric is [`../rules/security-baseline.md`](../rules/security-baseline.md) the
52
+ project's inviolable security MUSTs (Input Validation, Authentication,
53
+ Authorization, Output & Rendering, Data Leakage & Logging, Transport & Headers,
54
+ Secrets Management, Dependency Hygiene). Read it first. **Every finding MUST
55
+ name the specific `security-baseline.md` MUST it violates** (e.g. "violates
56
+ _Secrets Management_: 'Fallback or placeholder secrets MUST NOT be committed'").
57
+ A finding that cannot be tied to a baseline MUST — or to a CWE where the
58
+ baseline is silent — is out of scope for this lens.
59
+
60
+ ## Step 1: Detection Battery (Tool-First, Read-Only)
93
61
 
94
62
  > Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
95
63
 
96
- Scan the codebase for:
97
-
98
- - **Input Validation:** Check where user input enters the system (API endpoints,
99
- forms). Is it sanitized/validated?
100
- - **Injection Risks:** Search for raw SQL queries, `dangerouslySetInnerHTML`,
101
- `eval()`, or command execution logic.
102
- - **Authentication/Authorization:** Review how sessions/tokens are handled. Are
103
- there missing checks on sensitive routes?
104
- - **Dependency Security:** Check `package.json` for known-vulnerable versions of
105
- libraries.
106
- - **Secret Management:** Scan for `.env` files in git, hardcoded keys, or
107
- exposed credentials.
64
+ Ground every finding in tool output, not vibes. Run the ladder below; each rung
65
+ is **presence-gated** — when a scanner is absent, fall through to the next rung
66
+ and note the missing tool as a `Security Misconfiguration` finding (recommend
67
+ adopting it).
68
+
69
+ 1. **Dependency CVEs (`npm audit`).** Never recall CVEs from memory:
70
+
71
+ ```bash
72
+ npm audit --omit=dev --json 2>/dev/null || echo "npm audit unavailable"
73
+ ```
74
+
75
+ Each advisory reachable in production (`--omit=dev`) at `high` or `critical`
76
+ is a _Vulnerable Components_ finding citing the advisory id and the violated
77
+ _Dependency Hygiene_ MUST.
78
+
79
+ 2. **Secret scanning (`gitleaks` / `trufflehog`), with a grep fallback.** When
80
+ `gitleaks` is installed, prefer it:
81
+
82
+ ```bash
83
+ command -v gitleaks >/dev/null 2>&1 && gitleaks detect --no-banner --redact -v || \
84
+ command -v trufflehog >/dev/null 2>&1 && trufflehog filesystem . --no-update || \
85
+ echo "no secret scanner installed — running the grep battery below"
86
+ ```
87
+
88
+ 3. **Grep battery (deterministic fallback / augmentation).** Run these
89
+ regardless — they are cheap and catch what a scanner's ruleset may miss:
90
+
91
+ ```bash
92
+ # Hardcoded key material (violates Secrets Management)
93
+ rg -n -i "(api[_-]?key|secret|password|token)\s*[:=]\s*['\"][^'\"]{8,}" --glob '!**/*.test.*'
94
+ # eval / exec sinks (violates Output & Rendering)
95
+ rg -n "\b(eval|new Function|child_process\.exec|execSync)\s*\(" --glob '!**/*.test.*'
96
+ # Template-literal SQL (violates Output & Rendering — parameterize)
97
+ rg -n "(SELECT|INSERT|UPDATE|DELETE)\b[^;]*\$\{" -i
98
+ # Committed .env with real values (violates Secrets Management)
99
+ git ls-files | rg "(^|/)\.env($|\.)" | rg -v "\.env\.example$"
100
+ ```
101
+
102
+ 4. **Manual surface review.** Then read the surfaces the battery flags plus:
103
+ input-validation edges (API endpoints, form handlers — is input validated at
104
+ the boundary with a strict schema?), auth/session handling (token storage,
105
+ missing ownership checks on sensitive routes), and injection sinks
106
+ (`dangerouslySetInnerHTML`, raw SQL, command execution).
108
107
 
109
108
  ## Step 2: Evaluation Dimensions
110
109
 
111
- 1. **Injection:** SQL, NoSQL, OS Command, and Cross-Site Scripting (XSS).
112
- 2. **Broken Access Control:** Can a user access data they don't own?
110
+ Grade each finding against the `security-baseline.md` MUST it breaks (and the
111
+ CWE where one applies):
112
+
113
+ 1. **Injection:** SQL, NoSQL, OS Command, and Cross-Site Scripting (XSS) —
114
+ _Output & Rendering_.
115
+ 2. **Broken Access Control:** Can a user access data they don't own? —
116
+ _Authorization_.
113
117
  3. **Cryptographic Failures:** Is sensitive data (passwords, PII) hashed or
114
- encrypted using modern standards?
118
+ encrypted using modern standards? — _Authentication_ / _Data Leakage &
119
+ Logging_.
115
120
  4. **Security Misconfiguration:** Are there default passwords, verbose error
116
- messages in production, or insecure headers?
117
- 5. **Vulnerable Components:** Are outdated libraries introducing risks?
121
+ messages in production, or insecure headers? — _Transport & Headers_.
122
+ 5. **Vulnerable Components:** Are outdated libraries introducing risks?
123
+ _Dependency Hygiene_.
118
124
 
119
125
  ## Step 3: Output Requirements
120
126
 
121
127
  Generate and save a highly structured Markdown audit report to
122
128
  `{{auditOutputDir}}/audit-security-results.md`, using the exact template below.
123
129
 
130
+ > Grade every finding's severity on the shared
131
+ > [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md).
132
+
124
133
  ```markdown
125
134
  # Security Audit Report
126
135
 
@@ -131,16 +140,20 @@ security posture.]
131
140
 
132
141
  ## Detailed Findings
133
142
 
134
- [For every vulnerability identified, use the following strict structure:]
143
+ [For every vulnerability identified, use the following strict structure. Lead
144
+ each title with the primary file the vulnerability lives in:]
135
145
 
136
- ### [Short Title of the Vulnerability]
146
+ ### `path/to/primary-file.ext` — [Short title of the vulnerability]
137
147
 
138
148
  - **Dimension:** [e.g., Injection | Broken Access Control]
139
149
  - **Severity:** [Critical | High | Medium | Low]
140
150
  - **CWE ID:** [e.g., CWE-89 for SQL Injection]
151
+ - **Baseline MUST:** [the violated `security-baseline.md` MUST — e.g. "Secrets Management: fallback secrets MUST NOT be committed"]
152
+ - **Location:** `path/to/primary-file.ext:line`
141
153
  - **Current State:** [Technical explanation of the flaw and its location]
142
154
  - **Recommendation & Rationale:** [Step-by-step fix and defensive hardening
143
155
  strategy]
156
+ - **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. the exploit no longer reproducing, an added regression test, or a re-run of this lens]
144
157
  - **Agent Prompt:**
145
158
  `[A copy-pasteable, highly specific prompt to execute this remediation independently]`
146
159
 
@@ -154,3 +167,13 @@ security posture.]
154
167
 
155
168
  This is a **read-only** audit. Your priority is accuracy and clear impact
156
169
  assessment. Do not attempt to exploit the system or modify code.
170
+
171
+ ## Self-cross-check (mandatory — filter false positives before you finalize)
172
+
173
+ Before you write the report artifact from the previous step, run the shared
174
+ adversarial self-cross-check over your Detailed Findings — see
175
+ [`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
176
+ per-finding evidence bar, the exclusion list, and the final re-open-and-drop
177
+ pass whose `kept <k> / dropped <d>` counts you record in the Executive
178
+ Summary, so the sequential single-pass path filters unverified findings just as
179
+ the orchestrated path's adversarial reviewer does.
@@ -49,41 +49,79 @@ before this section existed.
49
49
  proceed with the full codebase-wide scan defined in the remaining
50
50
  steps.
51
51
 
52
- ## Step 1: Context Gathering (Read-Only Scan)
52
+ ## Step 0: Indexability gate (run first)
53
53
 
54
- > Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
54
+ **Open every SEO audit by deciding whether the surface is meant to be indexed at
55
+ all.** SEO findings on an auth-walled, private, or internal surface are noise —
56
+ a login-gated dashboard is *supposed* to be invisible to crawlers, so a missing
57
+ `<meta name="description">` there is not a defect.
58
+
59
+ - **Auth-walled / private surface** (every route under the change set sits
60
+ behind an authentication guard, a `noindex` directive, or a `Disallow`-all
61
+ `robots.txt`) ⇒ record a single **"SEO not applicable — surface is not
62
+ indexable"** note and stop. Do not emit per-file findings.
63
+ - **Publicly indexable surface** (marketing pages, docs, blog, product pages, a
64
+ public app shell) ⇒ proceed to Step 1.
65
+ - **Mixed** ⇒ scope the remaining steps to the indexable routes only, and say so
66
+ in the Executive Summary.
55
67
 
56
- Before generating the report, silently scan the codebase. Pay special attention
57
- to:
68
+ ## Step 1: Framework-aware metadata detection matrix
69
+
70
+ > Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
58
71
 
59
- - Page `<head>` elements: `<title>`, `<meta name="description">`, canonical
60
- tags, Open Graph, and Twitter Card tags.
61
- - Semantic HTML structure: heading hierarchy (`h1`–`h6`), landmark elements
62
- (`<main>`, `<nav>`, `<article>`, `<section>`), and `<img alt>` attributes.
63
- - Structured data: JSON-LD blocks and Schema.org types in use.
64
- - Internal linking patterns and URL structure.
65
- - Content layout: answer-friendly formatting (FAQs, numbered steps, definition
66
- lists) vs. dense prose.
72
+ Modern web consumers almost never ship literal `<head><meta></head>` HTML — the
73
+ metadata is produced by a framework mechanism. **Identify the mechanism first,
74
+ then probe the surfaces that mechanism uses.** Reporting "no `<meta>` tags found"
75
+ on a Next.js app that sets them through `generateMetadata` is a false finding.
76
+
77
+ - **Step 1a Identify the meta mechanism.** Determine which one (or more) of the
78
+ following the consumer uses, from its dependencies and source layout:
79
+
80
+ | Framework / library | Metadata mechanism | Where to probe |
81
+ | --- | --- | --- |
82
+ | Next.js (App Router) | `metadata` export / `generateMetadata()` | `app/**/{layout,page}.{js,jsx,ts,tsx}` |
83
+ | Next.js (Pages Router) | `next/head` `<Head>` | `pages/**/*.{js,jsx,ts,tsx}` |
84
+ | React (generic) | `react-helmet` / `react-helmet-async` | components importing `Helmet` |
85
+ | Vue / Nuxt | `@unhead/vue` / `useHead()` / `nuxt.config` `head` | `*.vue`, `nuxt.config.*` |
86
+ | Svelte / SvelteKit | `<svelte:head>` | `*.svelte` |
87
+ | Astro | frontmatter `<head>` in layouts | `*.astro` |
88
+ | Plain static | literal `<head>` HTML | `*.html` |
89
+
90
+ - **Step 1b — Enumerate the routes.** Take the route list from the navigability
91
+ `routeGlobs` SSOT (the same route tree the navigability lens enumerates), not
92
+ from a guess. Each public route is a page whose metadata you assess.
93
+ - **Step 1c — Per-route metadata probe.** For each indexable route, assert the
94
+ detected mechanism supplies: a `<title>`, a meta description, canonical URL,
95
+ Open Graph / Twitter Card tags, and (where relevant) JSON-LD structured data.
96
+ A route whose mechanism sets none of these is a real finding.
67
97
 
68
98
  ## Step 2: Analysis Dimensions
69
99
 
70
100
  Evaluate the gathered context against the following dimensions:
71
101
 
72
- 1. **Traditional SEO:** Meta tags, semantic structure, accessibility, internal
73
- linking logic, and keyword placement.
102
+ 1. **Traditional SEO:** Meta mechanism coverage (per Step 1), semantic structure,
103
+ heading hierarchy, `<img alt>`, internal linking logic, and canonical URLs.
74
104
  2. **AIO & GEO (Answer Engine Optimization):** Entity clarity, concise answer
75
105
  formatting, structured data (Schema.org), and token efficiency for LLM
76
106
  retrieval.
77
- 3. **Core Web Vitals:** CLS, LCP, and INP risk factors visible from the codebase
78
- (e.g., unsized images, render-blocking resources, large layout shifts).
79
- 4. **Crawlability:** `robots.txt`, `sitemap.xml`, and any `noindex` directives
80
- that may unintentionally block pages.
107
+ 3. **Statically-provable Core Web Vitals defects only:** flag *code-visible*
108
+ regressions unsized images or media embeds (CLS risk), render-blocking synchronous
109
+ scripts, and fonts loaded without `display=swap`. **Do not estimate or score
110
+ measured CWV** (LCP/INP/CLS numbers): measured Core Web Vitals are owned by
111
+ the `audit-performance` lens — defer them there explicitly rather than
112
+ guessing a score from source.
113
+ 4. **Crawlability:** `robots.txt`, `sitemap.xml` (including generated
114
+ `sitemap.*`/`robots.*` route handlers), and any `noindex` directives that may
115
+ unintentionally block indexable pages.
81
116
 
82
117
  ## Step 3: Output Requirements
83
118
 
84
119
  Generate and save a highly structured Markdown audit report to
85
120
  `{{auditOutputDir}}/audit-seo-results.md`, using the exact template below.
86
121
 
122
+ > Grade every finding's severity on the shared
123
+ > [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md).
124
+
87
125
  ```markdown
88
126
  # SEO & GEO Audit Report
89
127
 
@@ -94,9 +132,13 @@ primary gaps and the most impactful opportunities.]
94
132
 
95
133
  ## Detailed Audit Table
96
134
 
97
- | Issue | Impact | Category | Suggested Fix |
98
- | ------------------- | ---------------- | ---------- | ------------- |
99
- | [Issue description] | High / Med / Low | SEO or GEO | [Brief fix] |
135
+ [A supplementary at-a-glance index only. Every row MUST also have a full
136
+ `## Detailed Findings` entry below the Detailed Findings blocks are the
137
+ machine-parsed source of record; this table is not parsed.]
138
+
139
+ | Issue | Impact | Category | Suggested Fix |
140
+ | ------------------- | --------------------------- | ---------- | ------------- |
141
+ | [Issue description] | Critical / High / Med / Low | SEO or GEO | [Brief fix] |
100
142
 
101
143
  ## GEO-Specific Recommendations
102
144
 
@@ -106,16 +148,19 @@ reformatting key content as FAQ blocks.]
106
148
 
107
149
  ## Detailed Findings
108
150
 
109
- [For any issue requiring deeper explanation, use the following strict
110
- structure:]
151
+ [Mandatory: emit one entry per issue in the Detailed Audit Table above, using
152
+ the following strict structure. Lead each title with the primary file the
153
+ finding lives in:]
111
154
 
112
- ### [Short Title of the Issue]
155
+ ### `path/to/primary-file.ext` — [Short title of the issue]
113
156
 
114
157
  - **Category:** [SEO | GEO | Core Web Vitals | Crawlability]
115
- - **Impact:** [High | Medium | Low]
158
+ - **Impact:** [Critical | High | Medium | Low]
159
+ - **Location:** `path/to/primary-file.ext:line`
116
160
  - **Current State:** [What exists in the codebase and why it's suboptimal]
117
161
  - **Recommendation & Rationale:** [The specific fix and how it improves
118
162
  discoverability or LLM retrieval]
163
+ - **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. the meta tag now present in the rendered head, or a re-run of this lens]
119
164
  - **Agent Prompt:**
120
165
  `[A copy-pasteable, highly specific prompt to execute this fix independently]`
121
166
  ```
@@ -126,3 +171,13 @@ structure:]
126
171
 
127
172
  Do NOT rewrite or modify any files. Do NOT implement the changes. Focus strictly
128
173
  on analyzing the code. Output the report and stop.
174
+
175
+ ## Self-cross-check (mandatory — filter false positives before you finalize)
176
+
177
+ Before you write the report artifact from the previous step, run the shared
178
+ adversarial self-cross-check over your Detailed Findings — see
179
+ [`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
180
+ per-finding evidence bar, the exclusion list, and the final re-open-and-drop
181
+ pass whose `kept <k> / dropped <d>` counts you record in the Executive
182
+ Summary, so the sequential single-pass path filters unverified findings just as
183
+ the orchestrated path's adversarial reviewer does.
@@ -10,9 +10,10 @@ Senior Site Reliability Engineer (SRE) & Lead Developer
10
10
 
11
11
  ## Context & Objective
12
12
 
13
- You are conducting a rigorous, read-only final code audit for a production
14
- release candidate. Your goal is to surface critical risks across configuration
15
- integrity, security, observability, and code quality — providing a prioritized,
13
+ You are conducting a rigorous, read-only operational-readiness audit for a
14
+ production release candidate. Your goal is to surface critical risks across
15
+ rollback & recovery paths, observability & instrumentation, resilience &
16
+ failure handling, and runbooks & operational docs — providing a prioritized,
16
17
  actionable report that can be handed off for remediation before deployment.
17
18
 
18
19
  ## Scope (Story / plan-run mode)
@@ -36,69 +37,89 @@ before this section existed.
36
37
  proceed with the full codebase-wide scan defined in the remaining
37
38
  steps.
38
39
 
39
- ## Step 1: Context Gathering (Read-Only Scan)
40
+ ## Step 1: Resilience Detection Battery (Read-Only, Tool-First)
40
41
 
41
42
  > Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
42
43
 
43
- Before generating the report, silently scan the workspace. Pay special attention
44
- to:
45
-
46
- - Application configuration files (e.g., `site.config.ts`, `.env.example`,
47
- `wrangler.toml`, `app.config.ts`).
48
- - Source files for hardcoded values (strings resembling secrets, IDs, or
49
- environment-specific data).
50
- - Error handling patterns across services, API routes, and background jobs.
51
- - `package.json` for unused, deprecated, or overly heavy dependencies.
52
- - Any debugging artifacts likely introduced during development.
44
+ This lens audits **operational readiness** can this release be observed,
45
+ survive failure, and be rolled back? It deliberately does **not** re-audit
46
+ secrets, injection, dead code, or complexity; those are owned by
47
+ [`audit-security`](audit-security.md) and [`audit-clean-code`](audit-clean-code.md).
48
+ Ground findings in the greps below, then read the operational surfaces they
49
+ flag.
50
+
51
+ 1. **Run the resilience battery.** Each grep maps to a Step 2 dimension:
52
+
53
+ ```bash
54
+ # Network calls without a timeout (a hung upstream stalls the whole request)
55
+ rg -n "\b(fetch|axios(\.\w+)?|http\.request|got|ky)\s*\(" --glob '!**/*.test.*' | \
56
+ rg -v -i "timeout|signal|AbortController"
57
+ # Graceful-shutdown handlers (their ABSENCE is the finding for long-lived processes)
58
+ rg -n "process\.on\(\s*['\"]SIG(TERM|INT)['\"]" || echo "no SIGTERM/SIGINT handler found"
59
+ # Error-swallowing empty catch blocks (silent failure — no signal to observe)
60
+ rg -n "catch\s*(\([^)]*\))?\s*\{\s*\}"
61
+ # Retry / backoff on network-dependent work (resilience to transient failure)
62
+ rg -n -i "retr(y|ies)|backoff|circuit.?breaker|p-retry"
63
+ # Health / readiness endpoints (needed for orchestrated rollout & rollback)
64
+ rg -n -i "/(health|healthz|readyz|livez|ping)\b|healthCheck"
65
+ ```
53
66
 
54
67
  ## Step 2: Analysis Dimensions
55
68
 
56
- Evaluate the gathered context against the following production-readiness
57
- criteria:
58
-
59
- ### 1. Configuration Architecture
60
-
61
- - **Config Integrity:** Audit the application config to ensure it defines a
62
- clear schema for all variable/environment-specific data.
63
- - **Hardcoding Scan:** Scan components, utils, and services for any hardcoded
64
- values that should come from config or environment variables (e.g., API URLs,
65
- feature flags, region/locale data, identifiers).
66
- - **Fallback Logic:** Verify how the app behaves if a required config value is
67
- missing does it fail gracefully or crash silently?
68
-
69
- ### 2. Security & Secrets Management
70
-
71
- - **Secret Leaks:** Check for hardcoded API keys, tokens, or credentials
72
- committed to source. Ensure all secrets use environment variables.
73
- - **Input Sanitization:** Identify potential XSS or injection vectors,
74
- particularly where user input or URL parameters are reflected in the DOM or
75
- database.
76
- - **Dependency Risks:** Flag obviously deprecated, unmaintained, or unused heavy
77
- dependencies in `package.json`.
78
-
79
- ### 3. Error Handling & Observability
80
-
81
- - **Console Hygiene:** Identify debugging artifacts (`console.log`, `debugger`,
82
- commented-out test code) that must be removed before release.
83
- - **Error Swallowing:** Flag empty `catch` blocks or places where errors are
84
- silently ignored rather than logged or re-thrown.
85
- - **Boundary Handling:** Ensure the app handles unexpected or invalid inputs
86
- (e.g., bad URL params, missing DB records) with appropriate error responses.
87
-
88
- ### 4. Code Quality & Performance
89
-
90
- - **Dead Code:** Identify unused variables, imports, functions, or unreachable
91
- code blocks.
92
- - **Complexity:** Highlight logic with high cyclomatic complexity (deeply nested
93
- `if/else`, massive switch statements) that violates DRY principles.
94
- - **Asset Loading:** Flag synchronous heavy operations or unoptimized asset
95
- loading patterns that could hurt Core Web Vitals or API response times.
69
+ Evaluate the release candidate against these **production-readiness** criteria.
70
+
71
+ ### 1. Rollback & Recovery Paths
72
+
73
+ - **Rollback Path:** Is there a defined, tested way to revert this release —
74
+ a versioned deploy, blue-green/canary, or a documented `git revert` + redeploy
75
+ path? A release with no rollback path is a Critical finding.
76
+ - **Migration Reversibility:** Any schema migration in scope must ship a
77
+ down-migration (or a documented forward-fix); an irreversible destructive
78
+ migration blocks the release.
79
+ - **Feature-Flag Kill Switch:** Risky new behaviour should sit behind a flag
80
+ that can be disabled without a redeploy.
81
+
82
+ ### 2. Observability & Instrumentation
83
+
84
+ - **Structured Logging:** Are operational events emitted through a structured
85
+ logger (levels, correlation ids) rather than bare `console.*`, so they are
86
+ queryable in production?
87
+ - **Metrics & Tracing:** Are latency/error/throughput metrics and trace spans
88
+ emitted for the new code path? Missing instrumentation on a critical path is
89
+ a High finding.
90
+ - **Alerting & SLOs:** Is there an SLO (or error budget) and an alert wired to
91
+ the signals above, so a regression pages someone?
92
+
93
+ ### 3. Resilience & Failure Handling
94
+
95
+ - **Timeouts & Cancellation:** Every outbound call needs a timeout /
96
+ `AbortSignal` (grep 1 in Step 1). A call without one is a Reliability finding.
97
+ - **Retry & Backoff:** Transient-failure-prone calls should retry with backoff;
98
+ flag network work that fails hard on the first error.
99
+ - **Graceful Shutdown:** Long-lived processes must handle `SIGTERM`/`SIGINT`
100
+ and drain in-flight work (grep 2). Its absence risks dropped requests on
101
+ every deploy.
102
+ - **Error Boundaries:** Empty `catch` blocks (grep 3) swallow failures with no
103
+ signal flag each as an Observability + Resilience finding.
104
+
105
+ ### 4. Runbooks & Operational Docs
106
+
107
+ - **Runbook Coverage:** Does an operational runbook exist for this
108
+ service/feature (how to deploy, roll back, and respond to the top failure
109
+ modes)? A new production surface with no runbook is a finding.
110
+ - **Health & Readiness:** Are health/readiness endpoints (grep 5) present and
111
+ wired into the orchestrator so a bad rollout is caught before it takes
112
+ traffic?
113
+ - **On-Call Escalation:** Is ownership / escalation for this surface documented?
96
114
 
97
115
  ## Step 3: Output Requirements
98
116
 
99
117
  Generate and save a highly structured Markdown audit report to
100
118
  `{{auditOutputDir}}/audit-sre-results.md`, using the exact template below.
101
119
 
120
+ > Grade every finding's severity on the shared
121
+ > [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md).
122
+
102
123
  ```markdown
103
124
  # Production Release Candidate Audit
104
125
 
@@ -107,28 +128,30 @@ Generate and save a highly structured Markdown audit report to
107
128
  [A brief overview of the release candidate's health. Highlight the most critical
108
129
  risks that must be resolved before deployment.]
109
130
 
110
- ## Findings
131
+ ## Detailed Findings
111
132
 
112
- [Group findings by the categories below. Use this structure for each item:]
133
+ [Group findings by the categories below. Use this structure for each item.
134
+ Lead each title with the primary file the finding lives in:]
113
135
 
114
- ### [Short Title of the Issue]
136
+ ### `path/to/primary-file.ext` — [Short title of the issue]
115
137
 
116
- - **Category:** [Configuration | Security | Observability | Code Quality]
117
- - **Severity:** [High | Medium | Low]
118
- - **Location:** [`path/to/file.ts` or relevant area]
138
+ - **Category:** [Rollback & Recovery | Observability | Resilience | Runbooks]
139
+ - **Severity:** [Critical | High | Medium | Low]
140
+ - **Location:** `path/to/primary-file.ext:line`
119
141
  - **Current State:** [What exists and why it's a risk]
120
142
  - **Recommendation:** [The specific fix and rationale]
143
+ - **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. `npm test`, a grep that now returns empty, or a re-run of this lens]
121
144
  - **Agent Prompt:**
122
145
  `[A copy-pasteable, highly specific prompt to execute this fix independently]`
123
146
 
124
147
  ## Release Readiness Checklist
125
148
 
126
- | Category | Status |
127
- | ----------------------- | -------------------------- |
128
- | Configuration Integrity | ✅ Clear / ⚠️ Issues Found |
129
- | Security & Secrets | ✅ Clear / ⚠️ Issues Found |
130
- | Error Handling | ✅ Clear / ⚠️ Issues Found |
131
- | Code Quality | ✅ Clear / ⚠️ Issues Found |
149
+ | Category | Status |
150
+ | -------------------- | -------------------------- |
151
+ | Rollback & Recovery | ✅ Clear / ⚠️ Issues Found |
152
+ | Observability | ✅ Clear / ⚠️ Issues Found |
153
+ | Resilience | ✅ Clear / ⚠️ Issues Found |
154
+ | Runbooks | ✅ Clear / ⚠️ Issues Found |
132
155
  ```
133
156
 
134
157
  ---
@@ -137,3 +160,13 @@ risks that must be resolved before deployment.]
137
160
 
138
161
  Do NOT generate code fixes, edit files, or create branches. This is strictly a
139
162
  read-only analysis. Output the report and stop.
163
+
164
+ ## Self-cross-check (mandatory — filter false positives before you finalize)
165
+
166
+ Before you write the report artifact from the previous step, run the shared
167
+ adversarial self-cross-check over your Detailed Findings — see
168
+ [`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
169
+ per-finding evidence bar, the exclusion list, and the final re-open-and-drop
170
+ pass whose `kept <k> / dropped <d>` counts you record in the Executive
171
+ Summary, so the sequential single-pass path filters unverified findings just as
172
+ the orchestrated path's adversarial reviewer does.