mandrel 2.4.0 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/audit-checklists/accessibility.md +29 -0
- package/.agents/audit-checklists/architecture.md +4 -5
- package/.agents/audit-checklists/clean-code.md +10 -0
- package/.agents/audit-checklists/data-model.md +22 -0
- package/.agents/audit-checklists/dependencies.md +11 -2
- package/.agents/audit-checklists/devops.md +4 -0
- package/.agents/audit-checklists/navigability.md +3 -0
- package/.agents/audit-checklists/performance.md +8 -11
- package/.agents/audit-checklists/privacy.md +3 -4
- package/.agents/audit-checklists/quality.md +2 -0
- package/.agents/audit-checklists/security.md +4 -5
- package/.agents/audit-checklists/seo.md +7 -1
- package/.agents/audit-checklists/sre.md +14 -12
- package/.agents/audit-checklists/ux-ui.md +4 -0
- package/.agents/docs/configuration.md +3 -0
- package/.agents/docs/workflows.md +4 -3
- package/.agents/schemas/agentrc.schema.json +17 -0
- package/.agents/schemas/audit-rules.json +134 -19
- package/.agents/schemas/audit-rules.schema.json +6 -2
- package/.agents/scripts/audit-labels-bootstrap.js +4 -4
- package/.agents/scripts/audit-to-stories.js +244 -19
- package/.agents/scripts/lib/audit-suite/checklist-threading.js +26 -3
- package/.agents/scripts/lib/audit-suite/dispatch-checklist.js +132 -0
- package/.agents/scripts/lib/audit-suite/index.js +1 -0
- package/.agents/scripts/lib/audit-suite/selector.js +290 -14
- package/.agents/scripts/lib/audit-to-stories/audit-lenses.js +2 -1
- package/.agents/scripts/lib/audit-to-stories/build-story-body.js +5 -1
- package/.agents/scripts/lib/audit-to-stories/dedupe-against-github.js +23 -3
- package/.agents/scripts/lib/audit-to-stories/finding-adapter.js +38 -0
- package/.agents/scripts/lib/audit-to-stories/ledger.js +256 -0
- package/.agents/scripts/lib/audit-to-stories/parse-audit-md.js +41 -7
- package/.agents/scripts/lib/audit-to-stories/seed-from-findings.js +20 -2
- package/.agents/scripts/lib/command-header.js +1 -1
- package/.agents/scripts/lib/config-settings-schema-delivery.js +21 -0
- package/.agents/scripts/lib/dynamic-workflow/performance-report-contract.js +5 -3
- package/.agents/scripts/lib/feedback-loop/audit-results-graduator.js +56 -0
- package/.agents/scripts/lib/findings/route-finding.js +108 -10
- package/.agents/scripts/lib/observability/runtime-friction.js +137 -6
- package/.agents/scripts/lib/orchestration/retro-proposals.js +0 -0
- package/.agents/scripts/lib/orchestration/run-epilogue.js +5 -19
- package/.agents/scripts/lib/orchestration/single-story-close/phases/post-land.js +11 -0
- package/.agents/scripts/lib/orchestration/story-close/phases/local-lens-review.js +81 -1
- package/.agents/scripts/lib/orchestration/story-close/phases/review-core.js +1 -0
- package/.agents/scripts/lib/orchestration/story-follow-ups.js +53 -13
- package/.agents/scripts/nav-registry-diff.js +449 -0
- package/.agents/workflows/audit-accessibility.md +243 -0
- package/.agents/workflows/audit-architecture.md +89 -71
- package/.agents/workflows/audit-clean-code.md +87 -53
- package/.agents/workflows/audit-data-model.md +198 -0
- package/.agents/workflows/audit-dependencies.md +143 -28
- package/.agents/workflows/audit-devops.md +109 -18
- package/.agents/workflows/audit-documentation.md +25 -53
- package/.agents/workflows/audit-navigability.md +78 -22
- package/.agents/workflows/audit-performance.md +207 -103
- package/.agents/workflows/audit-privacy.md +51 -13
- package/.agents/workflows/audit-quality.md +71 -61
- package/.agents/workflows/audit-security.md +94 -71
- package/.agents/workflows/audit-seo.md +80 -25
- package/.agents/workflows/audit-sre.md +99 -66
- package/.agents/workflows/audit-to-stories.md +44 -5
- package/.agents/workflows/audit-ux-ui.md +71 -17
- package/.agents/workflows/helpers/audit-dual-path.md +59 -0
- package/.agents/workflows/helpers/audit-self-check.md +70 -0
- package/.agents/workflows/helpers/audit-severity-scale.md +19 -0
- package/.agents/workflows/helpers/deliver-story.md +25 -0
- package/docs/CHANGELOG.md +23 -0
- package/package.json +1 -1
- package/.agents/audit-checklists/lighthouse.md +0 -15
- package/.agents/schemas/audit-results.schema.json +0 -69
- package/.agents/workflows/audit-lighthouse.md +0 -269
|
@@ -38,89 +38,98 @@ before this section existed.
|
|
|
38
38
|
|
|
39
39
|
## Execution strategy (dual-path)
|
|
40
40
|
|
|
41
|
-
This lens runs along one of two execution paths
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
is Claude Code, `disableWorkflows` is not set (settings.json **or**
|
|
62
|
-
`CLAUDE_CODE_DISABLE_WORKFLOWS`), and the Claude Code version meets the
|
|
63
|
-
research-preview floor (`>= 2.1.154`). Any other runtime, a disabled setting,
|
|
64
|
-
or an older version degrades gracefully to the sequential path.
|
|
65
|
-
|
|
66
|
-
> **Capability degradation, not a contract shim.** This dual path is **not**
|
|
67
|
-
> covered by the No-Shim / hard-cutover rule in
|
|
68
|
-
> [`git-conventions.md`](../rules/git-conventions.md). That rule forbids
|
|
69
|
-
> running two shapes of the *same contract* side by side. Here there is **one**
|
|
70
|
-
> report contract; only the *execution strategy* is selected from a runtime
|
|
71
|
-
> capability — the same pattern the protocol already endorses for live-docs
|
|
72
|
-
> fallback in [`instructions.md` §1.C/§1.D](../instructions.md). The full
|
|
73
|
-
> capability-degradation rationale lives in the
|
|
74
|
-
> [`capability.js`](../scripts/lib/dynamic-workflow/capability.js) module
|
|
75
|
-
> docstring; the orchestrated-run evidence and per-lens cost/precision gate
|
|
76
|
-
> verdicts live in [`docs/roadmap.md`](../../docs/roadmap.md) (Part 3 —
|
|
77
|
-
> Dynamic-Workflow Orchestration).
|
|
78
|
-
|
|
79
|
-
**Forcing a path (for testing).** Set `MANDREL_AUDIT_STRATEGY=sequential` to
|
|
80
|
-
verify the fallback path with the feature notionally disabled, or
|
|
81
|
-
`MANDREL_AUDIT_STRATEGY=orchestrated` to pin the dynamic path. To exercise the
|
|
82
|
-
real disable signals instead, set `CLAUDE_CODE_DISABLE_WORKFLOWS=1` (env) or
|
|
83
|
-
`disableWorkflows: true` in `.claude/settings.json` and re-run the lens — both
|
|
84
|
-
degrade to the sequential path.
|
|
85
|
-
|
|
86
|
-
> **Read-only on both paths.** The lens is read-only (see Constraint). The
|
|
87
|
-
> orchestrated subagents run in `acceptEdits` and inherit the session tool
|
|
88
|
-
> allowlist, but the workflow script grants the analysis agents only
|
|
89
|
-
> read/search tools (`Read`, `Grep`, `Glob`) — no write/edit/shell-mutation
|
|
90
|
-
> tools. The single write in an orchestrated run is the final report artifact.
|
|
91
|
-
|
|
92
|
-
## Step 1: Vulnerability Surface Analysis
|
|
41
|
+
This lens runs along one of two execution paths (orchestrated dynamic-workflow
|
|
42
|
+
or sequential single-pass). Both emit the **identical** Step 3 report contract;
|
|
43
|
+
downstream consumers (`audit-to-stories`) are agnostic to which path produced
|
|
44
|
+
it. See [`helpers/audit-dual-path.md`](helpers/audit-dual-path.md) for strategy
|
|
45
|
+
selection, the forcing flags, and the read-only guarantee — read `audit-<lens>`
|
|
46
|
+
there as this lens's name.
|
|
47
|
+
|
|
48
|
+
## Rubric — `rules/security-baseline.md` is the contract
|
|
49
|
+
|
|
50
|
+
This lens does **not** grade against recalled OWASP lore. The authoritative
|
|
51
|
+
rubric is [`../rules/security-baseline.md`](../rules/security-baseline.md) — the
|
|
52
|
+
project's inviolable security MUSTs (Input Validation, Authentication,
|
|
53
|
+
Authorization, Output & Rendering, Data Leakage & Logging, Transport & Headers,
|
|
54
|
+
Secrets Management, Dependency Hygiene). Read it first. **Every finding MUST
|
|
55
|
+
name the specific `security-baseline.md` MUST it violates** (e.g. "violates
|
|
56
|
+
_Secrets Management_: 'Fallback or placeholder secrets MUST NOT be committed'").
|
|
57
|
+
A finding that cannot be tied to a baseline MUST — or to a CWE where the
|
|
58
|
+
baseline is silent — is out of scope for this lens.
|
|
59
|
+
|
|
60
|
+
## Step 1: Detection Battery (Tool-First, Read-Only)
|
|
93
61
|
|
|
94
62
|
> Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
|
|
95
63
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
64
|
+
Ground every finding in tool output, not vibes. Run the ladder below; each rung
|
|
65
|
+
is **presence-gated** — when a scanner is absent, fall through to the next rung
|
|
66
|
+
and note the missing tool as a `Security Misconfiguration` finding (recommend
|
|
67
|
+
adopting it).
|
|
68
|
+
|
|
69
|
+
1. **Dependency CVEs (`npm audit`).** Never recall CVEs from memory:
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
npm audit --omit=dev --json 2>/dev/null || echo "npm audit unavailable"
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Each advisory reachable in production (`--omit=dev`) at `high` or `critical`
|
|
76
|
+
is a _Vulnerable Components_ finding citing the advisory id and the violated
|
|
77
|
+
_Dependency Hygiene_ MUST.
|
|
78
|
+
|
|
79
|
+
2. **Secret scanning (`gitleaks` / `trufflehog`), with a grep fallback.** When
|
|
80
|
+
`gitleaks` is installed, prefer it:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
command -v gitleaks >/dev/null 2>&1 && gitleaks detect --no-banner --redact -v || \
|
|
84
|
+
command -v trufflehog >/dev/null 2>&1 && trufflehog filesystem . --no-update || \
|
|
85
|
+
echo "no secret scanner installed — running the grep battery below"
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
3. **Grep battery (deterministic fallback / augmentation).** Run these
|
|
89
|
+
regardless — they are cheap and catch what a scanner's ruleset may miss:
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
# Hardcoded key material (violates Secrets Management)
|
|
93
|
+
rg -n -i "(api[_-]?key|secret|password|token)\s*[:=]\s*['\"][^'\"]{8,}" --glob '!**/*.test.*'
|
|
94
|
+
# eval / exec sinks (violates Output & Rendering)
|
|
95
|
+
rg -n "\b(eval|new Function|child_process\.exec|execSync)\s*\(" --glob '!**/*.test.*'
|
|
96
|
+
# Template-literal SQL (violates Output & Rendering — parameterize)
|
|
97
|
+
rg -n "(SELECT|INSERT|UPDATE|DELETE)\b[^;]*\$\{" -i
|
|
98
|
+
# Committed .env with real values (violates Secrets Management)
|
|
99
|
+
git ls-files | rg "(^|/)\.env($|\.)" | rg -v "\.env\.example$"
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
4. **Manual surface review.** Then read the surfaces the battery flags plus:
|
|
103
|
+
input-validation edges (API endpoints, form handlers — is input validated at
|
|
104
|
+
the boundary with a strict schema?), auth/session handling (token storage,
|
|
105
|
+
missing ownership checks on sensitive routes), and injection sinks
|
|
106
|
+
(`dangerouslySetInnerHTML`, raw SQL, command execution).
|
|
108
107
|
|
|
109
108
|
## Step 2: Evaluation Dimensions
|
|
110
109
|
|
|
111
|
-
|
|
112
|
-
|
|
110
|
+
Grade each finding against the `security-baseline.md` MUST it breaks (and the
|
|
111
|
+
CWE where one applies):
|
|
112
|
+
|
|
113
|
+
1. **Injection:** SQL, NoSQL, OS Command, and Cross-Site Scripting (XSS) —
|
|
114
|
+
_Output & Rendering_.
|
|
115
|
+
2. **Broken Access Control:** Can a user access data they don't own? —
|
|
116
|
+
_Authorization_.
|
|
113
117
|
3. **Cryptographic Failures:** Is sensitive data (passwords, PII) hashed or
|
|
114
|
-
encrypted using modern standards?
|
|
118
|
+
encrypted using modern standards? — _Authentication_ / _Data Leakage &
|
|
119
|
+
Logging_.
|
|
115
120
|
4. **Security Misconfiguration:** Are there default passwords, verbose error
|
|
116
|
-
messages in production, or insecure headers?
|
|
117
|
-
5. **Vulnerable Components:** Are outdated libraries introducing risks?
|
|
121
|
+
messages in production, or insecure headers? — _Transport & Headers_.
|
|
122
|
+
5. **Vulnerable Components:** Are outdated libraries introducing risks? —
|
|
123
|
+
_Dependency Hygiene_.
|
|
118
124
|
|
|
119
125
|
## Step 3: Output Requirements
|
|
120
126
|
|
|
121
127
|
Generate and save a highly structured Markdown audit report to
|
|
122
128
|
`{{auditOutputDir}}/audit-security-results.md`, using the exact template below.
|
|
123
129
|
|
|
130
|
+
> Grade every finding's severity on the shared
|
|
131
|
+
> [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md).
|
|
132
|
+
|
|
124
133
|
```markdown
|
|
125
134
|
# Security Audit Report
|
|
126
135
|
|
|
@@ -131,16 +140,20 @@ security posture.]
|
|
|
131
140
|
|
|
132
141
|
## Detailed Findings
|
|
133
142
|
|
|
134
|
-
[For every vulnerability identified, use the following strict structure
|
|
143
|
+
[For every vulnerability identified, use the following strict structure. Lead
|
|
144
|
+
each title with the primary file the vulnerability lives in:]
|
|
135
145
|
|
|
136
|
-
### [Short
|
|
146
|
+
### `path/to/primary-file.ext` — [Short title of the vulnerability]
|
|
137
147
|
|
|
138
148
|
- **Dimension:** [e.g., Injection | Broken Access Control]
|
|
139
149
|
- **Severity:** [Critical | High | Medium | Low]
|
|
140
150
|
- **CWE ID:** [e.g., CWE-89 for SQL Injection]
|
|
151
|
+
- **Baseline MUST:** [the violated `security-baseline.md` MUST — e.g. "Secrets Management: fallback secrets MUST NOT be committed"]
|
|
152
|
+
- **Location:** `path/to/primary-file.ext:line`
|
|
141
153
|
- **Current State:** [Technical explanation of the flaw and its location]
|
|
142
154
|
- **Recommendation & Rationale:** [Step-by-step fix and defensive hardening
|
|
143
155
|
strategy]
|
|
156
|
+
- **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. the exploit no longer reproducing, an added regression test, or a re-run of this lens]
|
|
144
157
|
- **Agent Prompt:**
|
|
145
158
|
`[A copy-pasteable, highly specific prompt to execute this remediation independently]`
|
|
146
159
|
|
|
@@ -154,3 +167,13 @@ security posture.]
|
|
|
154
167
|
|
|
155
168
|
This is a **read-only** audit. Your priority is accuracy and clear impact
|
|
156
169
|
assessment. Do not attempt to exploit the system or modify code.
|
|
170
|
+
|
|
171
|
+
## Self-cross-check (mandatory — filter false positives before you finalize)
|
|
172
|
+
|
|
173
|
+
Before you write the report artifact from the previous step, run the shared
|
|
174
|
+
adversarial self-cross-check over your Detailed Findings — see
|
|
175
|
+
[`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
|
|
176
|
+
per-finding evidence bar, the exclusion list, and the final re-open-and-drop
|
|
177
|
+
pass whose `kept <k> / dropped <d>` counts you record in the Executive
|
|
178
|
+
Summary, so the sequential single-pass path filters unverified findings just as
|
|
179
|
+
the orchestrated path's adversarial reviewer does.
|
|
@@ -49,41 +49,79 @@ before this section existed.
|
|
|
49
49
|
proceed with the full codebase-wide scan defined in the remaining
|
|
50
50
|
steps.
|
|
51
51
|
|
|
52
|
-
## Step
|
|
52
|
+
## Step 0: Indexability gate (run first)
|
|
53
53
|
|
|
54
|
-
|
|
54
|
+
**Open every SEO audit by deciding whether the surface is meant to be indexed at
|
|
55
|
+
all.** SEO findings on an auth-walled, private, or internal surface are noise —
|
|
56
|
+
a login-gated dashboard is *supposed* to be invisible to crawlers, so a missing
|
|
57
|
+
`<meta name="description">` there is not a defect.
|
|
58
|
+
|
|
59
|
+
- **Auth-walled / private surface** (every route under the change set sits
|
|
60
|
+
behind an authentication guard, a `noindex` directive, or a `Disallow`-all
|
|
61
|
+
`robots.txt`) ⇒ record a single **"SEO not applicable — surface is not
|
|
62
|
+
indexable"** note and stop. Do not emit per-file findings.
|
|
63
|
+
- **Publicly indexable surface** (marketing pages, docs, blog, product pages, a
|
|
64
|
+
public app shell) ⇒ proceed to Step 1.
|
|
65
|
+
- **Mixed** ⇒ scope the remaining steps to the indexable routes only, and say so
|
|
66
|
+
in the Executive Summary.
|
|
55
67
|
|
|
56
|
-
|
|
57
|
-
|
|
68
|
+
## Step 1: Framework-aware metadata detection matrix
|
|
69
|
+
|
|
70
|
+
> Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
|
|
58
71
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
-
|
|
65
|
-
|
|
66
|
-
|
|
72
|
+
Modern web consumers almost never ship literal `<head><meta></head>` HTML — the
|
|
73
|
+
metadata is produced by a framework mechanism. **Identify the mechanism first,
|
|
74
|
+
then probe the surfaces that mechanism uses.** Reporting "no `<meta>` tags found"
|
|
75
|
+
on a Next.js app that sets them through `generateMetadata` is a false finding.
|
|
76
|
+
|
|
77
|
+
- **Step 1a — Identify the meta mechanism.** Determine which one (or more) of the
|
|
78
|
+
following the consumer uses, from its dependencies and source layout:
|
|
79
|
+
|
|
80
|
+
| Framework / library | Metadata mechanism | Where to probe |
|
|
81
|
+
| --- | --- | --- |
|
|
82
|
+
| Next.js (App Router) | `metadata` export / `generateMetadata()` | `app/**/{layout,page}.{js,jsx,ts,tsx}` |
|
|
83
|
+
| Next.js (Pages Router) | `next/head` `<Head>` | `pages/**/*.{js,jsx,ts,tsx}` |
|
|
84
|
+
| React (generic) | `react-helmet` / `react-helmet-async` | components importing `Helmet` |
|
|
85
|
+
| Vue / Nuxt | `@unhead/vue` / `useHead()` / `nuxt.config` `head` | `*.vue`, `nuxt.config.*` |
|
|
86
|
+
| Svelte / SvelteKit | `<svelte:head>` | `*.svelte` |
|
|
87
|
+
| Astro | frontmatter `<head>` in layouts | `*.astro` |
|
|
88
|
+
| Plain static | literal `<head>` HTML | `*.html` |
|
|
89
|
+
|
|
90
|
+
- **Step 1b — Enumerate the routes.** Take the route list from the navigability
|
|
91
|
+
`routeGlobs` SSOT (the same route tree the navigability lens enumerates), not
|
|
92
|
+
from a guess. Each public route is a page whose metadata you assess.
|
|
93
|
+
- **Step 1c — Per-route metadata probe.** For each indexable route, assert the
|
|
94
|
+
detected mechanism supplies: a `<title>`, a meta description, canonical URL,
|
|
95
|
+
Open Graph / Twitter Card tags, and (where relevant) JSON-LD structured data.
|
|
96
|
+
A route whose mechanism sets none of these is a real finding.
|
|
67
97
|
|
|
68
98
|
## Step 2: Analysis Dimensions
|
|
69
99
|
|
|
70
100
|
Evaluate the gathered context against the following dimensions:
|
|
71
101
|
|
|
72
|
-
1. **Traditional SEO:** Meta
|
|
73
|
-
linking logic, and
|
|
102
|
+
1. **Traditional SEO:** Meta mechanism coverage (per Step 1), semantic structure,
|
|
103
|
+
heading hierarchy, `<img alt>`, internal linking logic, and canonical URLs.
|
|
74
104
|
2. **AIO & GEO (Answer Engine Optimization):** Entity clarity, concise answer
|
|
75
105
|
formatting, structured data (Schema.org), and token efficiency for LLM
|
|
76
106
|
retrieval.
|
|
77
|
-
3. **Core Web Vitals
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
107
|
+
3. **Statically-provable Core Web Vitals defects only:** flag *code-visible*
|
|
108
|
+
regressions — unsized images or media embeds (CLS risk), render-blocking synchronous
|
|
109
|
+
scripts, and fonts loaded without `display=swap`. **Do not estimate or score
|
|
110
|
+
measured CWV** (LCP/INP/CLS numbers): measured Core Web Vitals are owned by
|
|
111
|
+
the `audit-performance` lens — defer them there explicitly rather than
|
|
112
|
+
guessing a score from source.
|
|
113
|
+
4. **Crawlability:** `robots.txt`, `sitemap.xml` (including generated
|
|
114
|
+
`sitemap.*`/`robots.*` route handlers), and any `noindex` directives that may
|
|
115
|
+
unintentionally block indexable pages.
|
|
81
116
|
|
|
82
117
|
## Step 3: Output Requirements
|
|
83
118
|
|
|
84
119
|
Generate and save a highly structured Markdown audit report to
|
|
85
120
|
`{{auditOutputDir}}/audit-seo-results.md`, using the exact template below.
|
|
86
121
|
|
|
122
|
+
> Grade every finding's severity on the shared
|
|
123
|
+
> [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md).
|
|
124
|
+
|
|
87
125
|
```markdown
|
|
88
126
|
# SEO & GEO Audit Report
|
|
89
127
|
|
|
@@ -94,9 +132,13 @@ primary gaps and the most impactful opportunities.]
|
|
|
94
132
|
|
|
95
133
|
## Detailed Audit Table
|
|
96
134
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
135
|
+
[A supplementary at-a-glance index only. Every row MUST also have a full
|
|
136
|
+
`## Detailed Findings` entry below — the Detailed Findings blocks are the
|
|
137
|
+
machine-parsed source of record; this table is not parsed.]
|
|
138
|
+
|
|
139
|
+
| Issue | Impact | Category | Suggested Fix |
|
|
140
|
+
| ------------------- | --------------------------- | ---------- | ------------- |
|
|
141
|
+
| [Issue description] | Critical / High / Med / Low | SEO or GEO | [Brief fix] |
|
|
100
142
|
|
|
101
143
|
## GEO-Specific Recommendations
|
|
102
144
|
|
|
@@ -106,16 +148,19 @@ reformatting key content as FAQ blocks.]
|
|
|
106
148
|
|
|
107
149
|
## Detailed Findings
|
|
108
150
|
|
|
109
|
-
[
|
|
110
|
-
structure
|
|
151
|
+
[Mandatory: emit one entry per issue in the Detailed Audit Table above, using
|
|
152
|
+
the following strict structure. Lead each title with the primary file the
|
|
153
|
+
finding lives in:]
|
|
111
154
|
|
|
112
|
-
### [Short
|
|
155
|
+
### `path/to/primary-file.ext` — [Short title of the issue]
|
|
113
156
|
|
|
114
157
|
- **Category:** [SEO | GEO | Core Web Vitals | Crawlability]
|
|
115
|
-
- **Impact:** [High | Medium | Low]
|
|
158
|
+
- **Impact:** [Critical | High | Medium | Low]
|
|
159
|
+
- **Location:** `path/to/primary-file.ext:line`
|
|
116
160
|
- **Current State:** [What exists in the codebase and why it's suboptimal]
|
|
117
161
|
- **Recommendation & Rationale:** [The specific fix and how it improves
|
|
118
162
|
discoverability or LLM retrieval]
|
|
163
|
+
- **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. the meta tag now present in the rendered head, or a re-run of this lens]
|
|
119
164
|
- **Agent Prompt:**
|
|
120
165
|
`[A copy-pasteable, highly specific prompt to execute this fix independently]`
|
|
121
166
|
```
|
|
@@ -126,3 +171,13 @@ structure:]
|
|
|
126
171
|
|
|
127
172
|
Do NOT rewrite or modify any files. Do NOT implement the changes. Focus strictly
|
|
128
173
|
on analyzing the code. Output the report and stop.
|
|
174
|
+
|
|
175
|
+
## Self-cross-check (mandatory — filter false positives before you finalize)
|
|
176
|
+
|
|
177
|
+
Before you write the report artifact from the previous step, run the shared
|
|
178
|
+
adversarial self-cross-check over your Detailed Findings — see
|
|
179
|
+
[`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
|
|
180
|
+
per-finding evidence bar, the exclusion list, and the final re-open-and-drop
|
|
181
|
+
pass whose `kept <k> / dropped <d>` counts you record in the Executive
|
|
182
|
+
Summary, so the sequential single-pass path filters unverified findings just as
|
|
183
|
+
the orchestrated path's adversarial reviewer does.
|
|
@@ -10,9 +10,10 @@ Senior Site Reliability Engineer (SRE) & Lead Developer
|
|
|
10
10
|
|
|
11
11
|
## Context & Objective
|
|
12
12
|
|
|
13
|
-
You are conducting a rigorous, read-only
|
|
14
|
-
release candidate. Your goal is to surface critical risks across
|
|
15
|
-
|
|
13
|
+
You are conducting a rigorous, read-only operational-readiness audit for a
|
|
14
|
+
production release candidate. Your goal is to surface critical risks across
|
|
15
|
+
rollback & recovery paths, observability & instrumentation, resilience &
|
|
16
|
+
failure handling, and runbooks & operational docs — providing a prioritized,
|
|
16
17
|
actionable report that can be handed off for remediation before deployment.
|
|
17
18
|
|
|
18
19
|
## Scope (Story / plan-run mode)
|
|
@@ -36,69 +37,89 @@ before this section existed.
|
|
|
36
37
|
proceed with the full codebase-wide scan defined in the remaining
|
|
37
38
|
steps.
|
|
38
39
|
|
|
39
|
-
## Step 1:
|
|
40
|
+
## Step 1: Resilience Detection Battery (Read-Only, Tool-First)
|
|
40
41
|
|
|
41
42
|
> Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
|
|
42
43
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
44
|
+
This lens audits **operational readiness** — can this release be observed,
|
|
45
|
+
survive failure, and be rolled back? It deliberately does **not** re-audit
|
|
46
|
+
secrets, injection, dead code, or complexity; those are owned by
|
|
47
|
+
[`audit-security`](audit-security.md) and [`audit-clean-code`](audit-clean-code.md).
|
|
48
|
+
Ground findings in the greps below, then read the operational surfaces they
|
|
49
|
+
flag.
|
|
50
|
+
|
|
51
|
+
1. **Run the resilience battery.** Each grep maps to a Step 2 dimension:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
# Network calls without a timeout (a hung upstream stalls the whole request)
|
|
55
|
+
rg -n "\b(fetch|axios(\.\w+)?|http\.request|got|ky)\s*\(" --glob '!**/*.test.*' | \
|
|
56
|
+
rg -v -i "timeout|signal|AbortController"
|
|
57
|
+
# Graceful-shutdown handlers (their ABSENCE is the finding for long-lived processes)
|
|
58
|
+
rg -n "process\.on\(\s*['\"]SIG(TERM|INT)['\"]" || echo "no SIGTERM/SIGINT handler found"
|
|
59
|
+
# Error-swallowing empty catch blocks (silent failure — no signal to observe)
|
|
60
|
+
rg -n "catch\s*(\([^)]*\))?\s*\{\s*\}"
|
|
61
|
+
# Retry / backoff on network-dependent work (resilience to transient failure)
|
|
62
|
+
rg -n -i "retr(y|ies)|backoff|circuit.?breaker|p-retry"
|
|
63
|
+
# Health / readiness endpoints (needed for orchestrated rollout & rollback)
|
|
64
|
+
rg -n -i "/(health|healthz|readyz|livez|ping)\b|healthCheck"
|
|
65
|
+
```
|
|
53
66
|
|
|
54
67
|
## Step 2: Analysis Dimensions
|
|
55
68
|
|
|
56
|
-
Evaluate the
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
- **
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
- **
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
### 2.
|
|
70
|
-
|
|
71
|
-
- **
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
- **
|
|
95
|
-
|
|
69
|
+
Evaluate the release candidate against these **production-readiness** criteria.
|
|
70
|
+
|
|
71
|
+
### 1. Rollback & Recovery Paths
|
|
72
|
+
|
|
73
|
+
- **Rollback Path:** Is there a defined, tested way to revert this release —
|
|
74
|
+
a versioned deploy, blue-green/canary, or a documented `git revert` + redeploy
|
|
75
|
+
path? A release with no rollback path is a Critical finding.
|
|
76
|
+
- **Migration Reversibility:** Any schema migration in scope must ship a
|
|
77
|
+
down-migration (or a documented forward-fix); an irreversible destructive
|
|
78
|
+
migration blocks the release.
|
|
79
|
+
- **Feature-Flag Kill Switch:** Risky new behaviour should sit behind a flag
|
|
80
|
+
that can be disabled without a redeploy.
|
|
81
|
+
|
|
82
|
+
### 2. Observability & Instrumentation
|
|
83
|
+
|
|
84
|
+
- **Structured Logging:** Are operational events emitted through a structured
|
|
85
|
+
logger (levels, correlation ids) rather than bare `console.*`, so they are
|
|
86
|
+
queryable in production?
|
|
87
|
+
- **Metrics & Tracing:** Are latency/error/throughput metrics and trace spans
|
|
88
|
+
emitted for the new code path? Missing instrumentation on a critical path is
|
|
89
|
+
a High finding.
|
|
90
|
+
- **Alerting & SLOs:** Is there an SLO (or error budget) and an alert wired to
|
|
91
|
+
the signals above, so a regression pages someone?
|
|
92
|
+
|
|
93
|
+
### 3. Resilience & Failure Handling
|
|
94
|
+
|
|
95
|
+
- **Timeouts & Cancellation:** Every outbound call needs a timeout /
|
|
96
|
+
`AbortSignal` (grep 1 in Step 1). A call without one is a Reliability finding.
|
|
97
|
+
- **Retry & Backoff:** Transient-failure-prone calls should retry with backoff;
|
|
98
|
+
flag network work that fails hard on the first error.
|
|
99
|
+
- **Graceful Shutdown:** Long-lived processes must handle `SIGTERM`/`SIGINT`
|
|
100
|
+
and drain in-flight work (grep 2). Its absence risks dropped requests on
|
|
101
|
+
every deploy.
|
|
102
|
+
- **Error Boundaries:** Empty `catch` blocks (grep 3) swallow failures with no
|
|
103
|
+
signal — flag each as an Observability + Resilience finding.
|
|
104
|
+
|
|
105
|
+
### 4. Runbooks & Operational Docs
|
|
106
|
+
|
|
107
|
+
- **Runbook Coverage:** Does an operational runbook exist for this
|
|
108
|
+
service/feature (how to deploy, roll back, and respond to the top failure
|
|
109
|
+
modes)? A new production surface with no runbook is a finding.
|
|
110
|
+
- **Health & Readiness:** Are health/readiness endpoints (grep 5) present and
|
|
111
|
+
wired into the orchestrator so a bad rollout is caught before it takes
|
|
112
|
+
traffic?
|
|
113
|
+
- **On-Call Escalation:** Is ownership / escalation for this surface documented?
|
|
96
114
|
|
|
97
115
|
## Step 3: Output Requirements
|
|
98
116
|
|
|
99
117
|
Generate and save a highly structured Markdown audit report to
|
|
100
118
|
`{{auditOutputDir}}/audit-sre-results.md`, using the exact template below.
|
|
101
119
|
|
|
120
|
+
> Grade every finding's severity on the shared
|
|
121
|
+
> [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md).
|
|
122
|
+
|
|
102
123
|
```markdown
|
|
103
124
|
# Production Release Candidate Audit
|
|
104
125
|
|
|
@@ -107,28 +128,30 @@ Generate and save a highly structured Markdown audit report to
|
|
|
107
128
|
[A brief overview of the release candidate's health. Highlight the most critical
|
|
108
129
|
risks that must be resolved before deployment.]
|
|
109
130
|
|
|
110
|
-
## Findings
|
|
131
|
+
## Detailed Findings
|
|
111
132
|
|
|
112
|
-
[Group findings by the categories below. Use this structure for each item
|
|
133
|
+
[Group findings by the categories below. Use this structure for each item.
|
|
134
|
+
Lead each title with the primary file the finding lives in:]
|
|
113
135
|
|
|
114
|
-
### [Short
|
|
136
|
+
### `path/to/primary-file.ext` — [Short title of the issue]
|
|
115
137
|
|
|
116
|
-
- **Category:** [
|
|
117
|
-
- **Severity:** [High | Medium | Low]
|
|
118
|
-
- **Location:**
|
|
138
|
+
- **Category:** [Rollback & Recovery | Observability | Resilience | Runbooks]
|
|
139
|
+
- **Severity:** [Critical | High | Medium | Low]
|
|
140
|
+
- **Location:** `path/to/primary-file.ext:line`
|
|
119
141
|
- **Current State:** [What exists and why it's a risk]
|
|
120
142
|
- **Recommendation:** [The specific fix and rationale]
|
|
143
|
+
- **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. `npm test`, a grep that now returns empty, or a re-run of this lens]
|
|
121
144
|
- **Agent Prompt:**
|
|
122
145
|
`[A copy-pasteable, highly specific prompt to execute this fix independently]`
|
|
123
146
|
|
|
124
147
|
## Release Readiness Checklist
|
|
125
148
|
|
|
126
|
-
| Category
|
|
127
|
-
|
|
|
128
|
-
|
|
|
129
|
-
|
|
|
130
|
-
|
|
|
131
|
-
|
|
|
149
|
+
| Category | Status |
|
|
150
|
+
| -------------------- | -------------------------- |
|
|
151
|
+
| Rollback & Recovery | ✅ Clear / ⚠️ Issues Found |
|
|
152
|
+
| Observability | ✅ Clear / ⚠️ Issues Found |
|
|
153
|
+
| Resilience | ✅ Clear / ⚠️ Issues Found |
|
|
154
|
+
| Runbooks | ✅ Clear / ⚠️ Issues Found |
|
|
132
155
|
```
|
|
133
156
|
|
|
134
157
|
---
|
|
@@ -137,3 +160,13 @@ risks that must be resolved before deployment.]
|
|
|
137
160
|
|
|
138
161
|
Do NOT generate code fixes, edit files, or create branches. This is strictly a
|
|
139
162
|
read-only analysis. Output the report and stop.
|
|
163
|
+
|
|
164
|
+
## Self-cross-check (mandatory — filter false positives before you finalize)
|
|
165
|
+
|
|
166
|
+
Before you write the report artifact from the previous step, run the shared
|
|
167
|
+
adversarial self-cross-check over your Detailed Findings — see
|
|
168
|
+
[`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
|
|
169
|
+
per-finding evidence bar, the exclusion list, and the final re-open-and-drop
|
|
170
|
+
pass whose `kept <k> / dropped <d>` counts you record in the Executive
|
|
171
|
+
Summary, so the sequential single-pass path filters unverified findings just as
|
|
172
|
+
the orchestrated path's adversarial reviewer does.
|