mandrel 2.3.0 → 2.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. package/.agents/audit-checklists/accessibility.md +29 -0
  2. package/.agents/audit-checklists/architecture.md +4 -5
  3. package/.agents/audit-checklists/clean-code.md +10 -0
  4. package/.agents/audit-checklists/data-model.md +22 -0
  5. package/.agents/audit-checklists/dependencies.md +11 -2
  6. package/.agents/audit-checklists/devops.md +4 -0
  7. package/.agents/audit-checklists/navigability.md +3 -0
  8. package/.agents/audit-checklists/performance.md +8 -11
  9. package/.agents/audit-checklists/privacy.md +3 -4
  10. package/.agents/audit-checklists/quality.md +2 -0
  11. package/.agents/audit-checklists/security.md +4 -5
  12. package/.agents/audit-checklists/seo.md +7 -1
  13. package/.agents/audit-checklists/sre.md +14 -12
  14. package/.agents/audit-checklists/ux-ui.md +4 -0
  15. package/.agents/docs/configuration.md +3 -0
  16. package/.agents/docs/workflows.md +4 -3
  17. package/.agents/schemas/agentrc.schema.json +17 -0
  18. package/.agents/schemas/audit-rules.json +134 -19
  19. package/.agents/schemas/audit-rules.schema.json +6 -2
  20. package/.agents/scripts/audit-labels-bootstrap.js +4 -4
  21. package/.agents/scripts/audit-to-stories.js +244 -19
  22. package/.agents/scripts/lib/audit-suite/checklist-threading.js +26 -3
  23. package/.agents/scripts/lib/audit-suite/dispatch-checklist.js +132 -0
  24. package/.agents/scripts/lib/audit-suite/index.js +1 -0
  25. package/.agents/scripts/lib/audit-suite/selector.js +290 -14
  26. package/.agents/scripts/lib/audit-to-stories/audit-lenses.js +2 -1
  27. package/.agents/scripts/lib/audit-to-stories/build-story-body.js +5 -1
  28. package/.agents/scripts/lib/audit-to-stories/dedupe-against-github.js +23 -3
  29. package/.agents/scripts/lib/audit-to-stories/finding-adapter.js +38 -0
  30. package/.agents/scripts/lib/audit-to-stories/ledger.js +256 -0
  31. package/.agents/scripts/lib/audit-to-stories/parse-audit-md.js +41 -7
  32. package/.agents/scripts/lib/audit-to-stories/seed-from-findings.js +20 -2
  33. package/.agents/scripts/lib/command-header.js +1 -1
  34. package/.agents/scripts/lib/config-settings-schema-delivery.js +21 -0
  35. package/.agents/scripts/lib/dynamic-workflow/performance-report-contract.js +5 -3
  36. package/.agents/scripts/lib/feedback-loop/audit-results-graduator.js +56 -0
  37. package/.agents/scripts/lib/findings/route-finding.js +108 -10
  38. package/.agents/scripts/lib/observability/runtime-friction.js +62 -0
  39. package/.agents/scripts/lib/orchestration/resolve-stories.js +9 -1
  40. package/.agents/scripts/lib/orchestration/retro-proposals.js +49 -1
  41. package/.agents/scripts/lib/orchestration/single-story-close/phases/post-land.js +73 -15
  42. package/.agents/scripts/lib/orchestration/story-close/phases/local-lens-review.js +81 -1
  43. package/.agents/scripts/lib/orchestration/story-close/phases/review-core.js +1 -0
  44. package/.agents/scripts/lib/orchestration/ticket-lease.js +78 -10
  45. package/.agents/scripts/lib/orchestration/ticketing/transition.js +68 -16
  46. package/.agents/scripts/lib/single-story-sweep/sweep-lock.js +73 -0
  47. package/.agents/scripts/lib/wave-runner/live-probe.js +93 -12
  48. package/.agents/scripts/nav-registry-diff.js +449 -0
  49. package/.agents/scripts/single-story-init.js +189 -51
  50. package/.agents/scripts/stories-wave-tick.js +39 -2
  51. package/.agents/workflows/audit-accessibility.md +243 -0
  52. package/.agents/workflows/audit-architecture.md +89 -71
  53. package/.agents/workflows/audit-clean-code.md +87 -53
  54. package/.agents/workflows/audit-data-model.md +198 -0
  55. package/.agents/workflows/audit-dependencies.md +143 -28
  56. package/.agents/workflows/audit-devops.md +109 -18
  57. package/.agents/workflows/audit-documentation.md +25 -53
  58. package/.agents/workflows/audit-navigability.md +78 -22
  59. package/.agents/workflows/audit-performance.md +207 -103
  60. package/.agents/workflows/audit-privacy.md +51 -13
  61. package/.agents/workflows/audit-quality.md +71 -61
  62. package/.agents/workflows/audit-security.md +94 -71
  63. package/.agents/workflows/audit-seo.md +80 -25
  64. package/.agents/workflows/audit-sre.md +99 -66
  65. package/.agents/workflows/audit-to-stories.md +44 -5
  66. package/.agents/workflows/audit-ux-ui.md +71 -17
  67. package/.agents/workflows/deliver.md +32 -8
  68. package/.agents/workflows/helpers/audit-dual-path.md +59 -0
  69. package/.agents/workflows/helpers/audit-self-check.md +70 -0
  70. package/.agents/workflows/helpers/audit-severity-scale.md +19 -0
  71. package/.agents/workflows/helpers/deliver-story.md +25 -0
  72. package/docs/CHANGELOG.md +29 -0
  73. package/package.json +1 -1
  74. package/.agents/audit-checklists/lighthouse.md +0 -15
  75. package/.agents/schemas/audit-results.schema.json +0 -69
  76. package/.agents/workflows/audit-lighthouse.md +0 -269
@@ -1,5 +1,5 @@
1
1
  ---
2
- description: Audit hot paths, algorithmic complexity, and I/O bottlenecks in the tooling surface (`single-story-close`, dispatcher, gates); propose remediations.
2
+ description: Audit performance by measuring first — profile hot paths, I/O, memory, and payload against the repo's own numbers — and audit interleaving/partial-failure correctness (TOCTOU, unawaited promises, non-atomic writes) as a first-class dimension.
3
3
  ---
4
4
 
5
5
  # Performance & Bottleneck Audit
@@ -10,9 +10,23 @@ Performance Engineer & Systems Architect
10
10
 
11
11
  ## Context & Objective
12
12
 
13
- Analyze the application for performance regressions, bottlenecks, and efficiency
14
- gaps. Your goal is to identify why a system is slow or where it might fail under
15
- load.
13
+ Find where a system is slow, wasteful, or unsafe under concurrency — and prove
14
+ it with numbers, not opinions. This lens has three standing commitments that
15
+ separate it from a prose read-through:
16
+
17
+ 1. **Measure before you judge.** A performance claim that is not backed by a
18
+ profile, a timing, or a byte count is a hypothesis, not a finding. Step 0
19
+ times the repo's own suite and entry points and produces the evidence every
20
+ finding must cite.
21
+ 2. **Adapt to the repo profile.** A CLI/tooling repo has no bundle and no Core
22
+ Web Vitals; a frontend app does. Step 1 detects the target profile and
23
+ activates only the dimensions that apply, declaring the rest inapplicable
24
+ rather than fabricating findings for a surface that does not exist.
25
+ 3. **Interleaving correctness is a performance concern.** The most expensive
26
+ defects this repo has shipped were not slow loops — they were races
27
+ (check-then-act on a lease, non-atomic checkout mutation under concurrent
28
+ close, shared-cache poisoning). Step 2 treats interleaving & partial-failure
29
+ correctness as a first-class dimension, statically and repo-observably.
16
30
 
17
31
  ## Scope (Story / plan-run mode)
18
32
 
@@ -37,88 +51,154 @@ before this section existed.
37
51
 
38
52
  ## Execution strategy (dual-path)
39
53
 
40
- This lens runs along one of two execution paths. Both emit the **identical**
41
- report contract (Step 3); downstream consumers (`audit-to-stories`) are agnostic to which path produced it.
42
-
43
- - **Orchestrated (dynamic-workflow) path.** When Claude Code's
44
- [dynamic workflows](https://code.claude.com/docs/en/workflows) are
45
- available, the saved project workflow
46
- `.claude/workflows/audit-performance.workflow.js` fans the dimensions below
47
- out as parallel read-only subagents, runs an **adversarial cross-check**
48
- stage (an independent agent reviews each dimension's findings and drops
49
- false positives before they enter the report), then synthesises the Step 3
50
- report. The orchestrator derives its per-dimension prompts from *this*
51
- markdown at run time the lens stays the single source of truth; the
52
- script does not fork a second copy of the spec. The three-phase fan-out
53
- itself is the shared
54
- [`runAuditOrchestration`](../scripts/lib/dynamic-workflow/audit-orchestrator.js)
55
- engine, not a per-lens copy.
56
- - **Sequential (single-pass) path.** When dynamic workflows are unavailable,
57
- follow Steps 1–3 below turn-by-turn exactly as before. This is the default
58
- fallback and changes nothing about the existing behaviour.
59
-
60
- **Strategy selection** is computed by
61
- [`lib/dynamic-workflow/capability.js`](../scripts/lib/dynamic-workflow/capability.js)
62
- (`selectAuditStrategy`). The orchestrated path is chosen only when the runtime
63
- is Claude Code, `disableWorkflows` is not set (settings.json **or**
64
- `CLAUDE_CODE_DISABLE_WORKFLOWS`), and the Claude Code version meets the
65
- research-preview floor (`>= 2.1.154`). Any other runtime, a disabled setting,
66
- or an older version degrades gracefully to the sequential path.
67
-
68
- > **Capability degradation, not a contract shim.** This dual path is **not**
69
- > covered by the No-Shim / hard-cutover rule in
70
- > [`git-conventions.md`](../rules/git-conventions.md). That rule forbids
71
- > running two shapes of the *same contract* side by side. Here there is **one**
72
- > report contract; only the *execution strategy* is selected from a runtime
73
- > capability the same pattern the protocol already endorses for live-docs
74
- > fallback in [`instructions.md` §1.C/§1.D](../instructions.md). The full
75
- > capability-degradation rationale lives in the
76
- > [`capability.js`](../scripts/lib/dynamic-workflow/capability.js) module
77
- > docstring; the orchestrated-run evidence and per-lens cost/precision gate
78
- > verdicts live in [`docs/roadmap.md`](../../docs/roadmap.md) (Part 3 —
79
- > Dynamic-Workflow Orchestration).
80
-
81
- **Forcing a path (for testing).** Set `MANDREL_AUDIT_STRATEGY=sequential` to
82
- verify the fallback path with the feature notionally disabled, or
83
- `MANDREL_AUDIT_STRATEGY=orchestrated` to pin the dynamic path. To exercise the
84
- real disable signals instead, set `CLAUDE_CODE_DISABLE_WORKFLOWS=1` (env) or
85
- `disableWorkflows: true` in `.claude/settings.json` and re-run the lens — both
86
- degrade to the sequential path.
87
-
88
- > **Read-only on both paths.** The lens is read-only (see Constraint). The
89
- > orchestrated subagents run in `acceptEdits` and inherit the session tool
90
- > allowlist, but the workflow script grants the analysis agents only
91
- > read/search tools (`Read`, `Grep`, `Glob`) no write/edit/shell-mutation
92
- > tools. The single write in an orchestrated run is the final report artifact.
93
-
94
- ## Step 1: Bottleneck Discovery
95
-
96
- > Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
97
-
98
- Investigate the following areas:
99
-
100
- - **Database/API Efficiency:** Look for N+1 query patterns, missing indexes, or
101
- oversized JSON payloads.
102
- - **Frontend Rendering:** Identify unnecessary re-renders (in React/Vue), large
103
- DOM trees, or layout thrashing.
104
- - **Bundle Size:** Check for heavy dependencies, missing code-splitting, or
105
- unoptimized assets.
106
- - **Resource Usage:** Identify potential memory leaks or high CPU usage logic
107
- (e.g., synchronous loops over large datasets).
108
- - **Network Path:** Check for excessive round-trips or lack of caching headers.
109
-
110
- ## Step 2: Evaluation Dimensions
111
-
112
- 1. **Latency:** How long does it take for a user action to complete?
113
- 2. **Throughput:** How many concurrent operations can the system handle before
114
- degrading?
115
- 3. **Efficiency:** Is the code using the minimum amount of CPU/Memory/Network
116
- required?
117
- 4. **Scalability:** Does the performance hold as the data size or user count
118
- increases?
119
- 5. **Core Web Vitals:** (For frontend) LCP, FID, and CLS metrics.
120
-
121
- ## Step 3: Output Requirements
54
+ This lens runs along one of two execution paths (orchestrated dynamic-workflow
55
+ or sequential single-pass). Both emit the **identical** Step 4 report contract;
56
+ downstream consumers (`audit-to-stories`) are agnostic to which path produced
57
+ it. See [`helpers/audit-dual-path.md`](helpers/audit-dual-path.md) for strategy
58
+ selection, the forcing flags, and the read-only guarantee — read `audit-<lens>`
59
+ there as this lens's name.
60
+
61
+ > **Measurement is non-mutating, not forbidden.** This lens is read-only with
62
+ > respect to source, but it MUST be allowed to *run* measurements. The
63
+ > orchestrated path grants its measurement agents a `Bash` tool restricted to a
64
+ > **non-mutating command allowlist** (profilers, timers, bundle-stat and
65
+ > file-size probesnever a command that writes source, installs, or mutates
66
+ > git/labels). See the allowlist in
67
+ > [`.claude/workflows/audit-performance.workflow.js`](../../.claude/workflows/audit-performance.workflow.js).
68
+
69
+ ## Step 0: Measure before you judge (mandatory)
70
+
71
+ Produce evidence first; every finding in Step 4 carries an **Evidence** field
72
+ that cites a repro command and tags itself `measured` or `estimated`. Run the
73
+ measurements that apply to the repo (Step 1 tells you which), preferring the
74
+ repo's own scripts over invented ones.
75
+
76
+ **Timing (all repos).** Time the project's own test suite and any CLI entry
77
+ points. Prefer `hyperfine` for stable multi-run statistics; fall back to
78
+ `/usr/bin/time -v` (or `time`) when it is absent:
79
+
80
+ ```bash
81
+ hyperfine --warmup 1 'npm test' # suite wall-clock + variance
82
+ hyperfine --warmup 1 'node bin/<entry>.js --help' # CLI cold-start cost
83
+ /usr/bin/time -v node bin/<entry>.js <args> 2>&1 | tail -n 20 # fallback + RSS
84
+ ```
85
+
86
+ **CPU profile (all repos).** Profile a detected entry script with the V8
87
+ sampling profiler and read the hottest self-time frames:
88
+
89
+ ```bash
90
+ node --cpu-prof --cpu-prof-dir=temp/audits/cpuprof bin/<entry>.js <args>
91
+ ```
92
+
93
+ Then inspect the emitted `.cpuprofile` (the top self-time nodes) for the real
94
+ hot path.
95
+
96
+ **Payload / bundle (web repos only).** Emit build stats and size the shipped
97
+ payload; do not estimate what the bundler will tell you exactly:
98
+
99
+ ```bash
100
+ npx vite build --profile # or the repo's own build script
101
+ du -sh dist/ && find dist -name '*.js' -exec wc -c {} + | sort -n | tail
102
+ ```
103
+
104
+ **Evidence discipline.** A finding tagged `measured` names the command above
105
+ whose output produced its number. A finding tagged `estimated` (e.g. a
106
+ Big-O argument read off the code without a runnable repro) says so plainly and
107
+ is graded no higher than **Medium** unless a measurement upgrades it. Attach
108
+ the repro command to the finding so the operator can reproduce it in one paste.
109
+
110
+ ### Step 0b: Perf baseline artifact + diff-aware trend
111
+
112
+ Write a per-run baseline capturing the Step 0 numbers to
113
+ `{{auditOutputDir}}/perf-baseline.json` (suite time, per-entry cold-start, RSS,
114
+ bundle bytes where applicable, and the hot-frame list). On every run:
115
+
116
+ - **Diff against the previous baseline** when one exists. A metric that
117
+ regressed past the previous run's value is an **automatic High** finding with
118
+ the delta (old new, absolute and %) as its Evidence.
119
+ - **Suppress unchanged known findings.** A finding whose measured value is
120
+ within noise of the prior baseline is reported as *unchanged* in a trend
121
+ summary line, not re-litigated as a fresh finding. New or regressed metrics
122
+ are what the run surfaces.
123
+ - Record the baseline path and the trend verdict (`first-run` /
124
+ `improved` / `unchanged` / `regressed`) in the Executive Summary.
125
+
126
+ ## Step 1: Repo profile & active dimensions
127
+
128
+ Detect the target profile from repo-observable markers, then activate only the
129
+ dimensions that apply. Declare the inapplicable ones explicitly in the report
130
+ (so a reader knows they were considered and ruled out, not forgotten).
131
+
132
+ Detection signals:
133
+
134
+ - A **CLI / tooling / library** repo — a `bin` field or executable entry
135
+ scripts, no framework marker, no frontend directory: bundle and Core Web
136
+ Vitals are **inapplicable**.
137
+ - A **web / frontend** repo — a framework marker (`react`, `vue`, `svelte`,
138
+ `next`, `vite`, `webpack`) and/or a frontend surface (`src/components/**`,
139
+ `*.html`, `*.css`, a `dist/` build): payload/bundle and CWV **apply**.
140
+ - A **service / backend** repo — server entry, route/controller/API surface:
141
+ I/O and interleaving dominate; bundle is **inapplicable**.
142
+
143
+ The interleaving & partial-failure dimension and the CPU / I/O / memory
144
+ dimensions apply to **every** profile. Only payload/bundle (and its re-homed
145
+ CWV material) is web-gated.
146
+
147
+ ## Step 2: Analysis dimensions (orthogonal set)
148
+
149
+ The historically overlapping ten dimensions collapse to four orthogonal
150
+ resource dimensions plus one correctness dimension. Audit each *active*
151
+ dimension (per Step 1) against measured evidence from Step 0.
152
+
153
+ - **CPU & algorithmic hot paths:** super-linear complexity on a path that runs
154
+ under load, redundant recomputation, synchronous work blocking the event
155
+ loop. Ground every claim in a `--cpu-prof` hot frame or a timed repro.
156
+ - **I/O & syscall efficiency:** N+1 queries/reads, unbatched network or
157
+ filesystem round-trips, missing caching, oversized payloads crossing a
158
+ boundary, chatty `fs` calls in a loop. Cite the call site and a timing.
159
+ - **Memory & leaks:** unbounded caches/arrays, retained closures, listeners
160
+ never removed, growth across a repeated operation. Cite RSS from Step 0 or a
161
+ heap delta.
162
+ - **Payload & bundle (web only):** heavy or duplicated dependencies, missing
163
+ code-splitting, unoptimized assets, render-blocking resources. Re-homed from
164
+ the retired lighthouse lens: capture a **per-route Core-Web-Vitals score
165
+ baseline** and measure with a **median-of-3** protocol (three runs, report
166
+ the median LCP/CLS/INP/TBT per route) so single-run variance never drives a
167
+ finding. Inapplicable — and omitted — on non-web repos.
168
+ - **Interleaving & partial-failure correctness:** the concurrency dimension.
169
+ Statically, repo-observably, look for: unawaited / floating promises;
170
+ `Promise.all` over independently-failing branches where `allSettled` is
171
+ required; **check-then-act (TOCTOU)** on files, locks, labels, or uniqueness
172
+ constraints; non-atomic read-modify-write; in-place writes where
173
+ temp-file-plus-rename is required for crash-atomicity; missing
174
+ transaction/compensation around a multi-step write; non-idempotent retried
175
+ side effects; and shared-cache poisoning (one run writing state a concurrent
176
+ run reads). **Speculative races are the known failure mode of this
177
+ dimension** — a claimed race with no concrete interleaving and no
178
+ repo-observable shared-state path is a false positive; lean hard on the
179
+ self-cross-check to drop it.
180
+
181
+ ## Step 3: Severity rubric (performance-anchored)
182
+
183
+ Grade every finding on the shared
184
+ [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md),
185
+ anchored to performance/correctness cost rather than gut feel:
186
+
187
+ - Grade **Critical** for a guaranteed data-loss or corruption path under normal
188
+ concurrency (e.g. a lost-update TOCTOU on persisted state), or a hang/outage
189
+ under expected load.
190
+ - Grade **High** for a measured regression past the previous baseline (delta is
191
+ the Evidence), or a hot-path cost (frequency × per-call cost) on a path proven
192
+ to run under load, or a race with a concrete losing interleaving.
193
+ - Grade **Medium** for a real but bounded cost: an `estimated`-only algorithmic
194
+ concern, a cold-path inefficiency, or a concurrency smell with a plausible but
195
+ unproven interleaving.
196
+ - Grade **Low** for minor or opportunistic issues; fix when nearby.
197
+
198
+ Latency thresholds, when a user-facing route is in scope, follow the CWV bands
199
+ in the payload/bundle dimension (LCP ≤2.5s good / ≤4.0s needs-improvement).
200
+
201
+ ## Step 4: Output Requirements
122
202
 
123
203
  Generate and save a highly structured Markdown audit report to
124
204
  `{{auditOutputDir}}/audit-performance-results.md`, using the exact template
@@ -129,22 +209,32 @@ below.
129
209
 
130
210
  ## Executive Summary
131
211
 
132
- [Overview of performance summary vs target benchmarks.]
212
+ [Overview of performance posture vs the Step 0 measurements. State the repo
213
+ profile detected (Step 1) and which dimensions were inapplicable. State the
214
+ baseline trend verdict (first-run / improved / unchanged / regressed) and the
215
+ `perf-baseline.json` path. Close with the self-cross-check `kept k / dropped d`
216
+ line.]
133
217
 
134
218
  ## Detailed Findings
135
219
 
136
- [For every bottleneck identified, use the following strict structure:]
137
-
138
- ### [Short Title of the Bottleneck]
139
-
140
- - **Dimension:** [e.g., Latency | Throughput | Efficiency]
141
- - **Impact:** [High | Medium | Low]
142
- - **Current State:** [Technical explanation of where and why the bottleneck
143
- occurs]
144
- - **Recommendation & Rationale:** [Specific optimization tactic and expected
145
- performance gain]
220
+ [For every bottleneck or correctness defect identified, use the following
221
+ strict structure. Lead each title with the primary file it lives in:]
222
+
223
+ ### `path/to/primary-file.ext` — [Short title of the finding]
224
+
225
+ - **Dimension:** [CPU & algorithmic | I/O | Memory & leaks | Payload & bundle | Interleaving & partial-failure]
226
+ - **Impact:** [Critical | High | Medium | Low]
227
+ - **Location:** `path/to/primary-file.ext:line`
228
+ - **Evidence:** [A repro command from Step 0 (or a quoted code path) and a
229
+ `measured` or `estimated` tag — e.g. "`measured`: `hyperfine 'npm test'` →
230
+ regressed 8.1s → 11.4s (+40%) vs perf-baseline.json"]
231
+ - **Current State:** [Technical explanation of where and why the bottleneck or
232
+ race occurs]
233
+ - **Recommendation & Rationale:** [Specific optimization/fix tactic and the
234
+ expected gain or the interleaving it closes]
235
+ - **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. a benchmark below the target threshold, a re-run of this lens showing the baseline no longer regressed, or a test exercising the losing interleaving]
146
236
  - **Agent Prompt:**
147
- `[A copy-pasteable, highly specific prompt to execute this optimization independently]`
237
+ `[A copy-pasteable, highly specific prompt to execute this fix independently]`
148
238
 
149
239
  ## Low-Hanging Fruit
150
240
 
@@ -153,8 +243,22 @@ below.
153
243
 
154
244
  ## Constraint
155
245
 
156
- This is a **read-only** audit. Note: This workflow differs from
157
- `audit-lighthouse.md` (which runs Lighthouse and reports per-category scores
158
- and findings) by focusing on deep architectural and logic bottlenecks across
159
- the whole stack backend, data access, and runtime hot paths — rather than
160
- the page-load surface Lighthouse measures.
246
+ This is a **read-only** audit **with respect to source**: it does not edit,
247
+ create, or delete application code, dependencies, or configuration. It **does**
248
+ run non-mutating measurements (Step 0) and writes exactly two artifacts — the
249
+ report and `perf-baseline.json`. Note: this lens supersedes the retired
250
+ `audit-lighthouse` lens by folding its measured Core-Web-Vitals material (the
251
+ per-route score baseline and median-of-3 protocol) into the web branch of the
252
+ payload/bundle dimension.
253
+
254
+ ## Self-cross-check (mandatory — filter false positives before you finalize)
255
+
256
+ Before you write the report artifact from the previous step, run the shared
257
+ adversarial self-cross-check over your Detailed Findings — see
258
+ [`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
259
+ per-finding evidence bar, the exclusion list, and the final re-open-and-drop
260
+ pass whose `kept <k> / dropped <d>` counts you record in the Executive
261
+ Summary, so the sequential single-pass path filters unverified findings just as
262
+ the orchestrated path's adversarial reviewer does. The **interleaving**
263
+ dimension leans on this pass hardest: drop every claimed race that lacks a
264
+ concrete losing interleaving over a repo-observable shared-state path.
@@ -36,19 +36,41 @@ before this section existed.
36
36
  proceed with the full codebase-wide scan defined in the remaining
37
37
  steps.
38
38
 
39
- ## Step 1: Scanning for PII Patterns
39
+ ## Step 1: Sink-First Detection
40
40
 
41
41
  > Apply [`helpers/parallel-tooling.md`](helpers/parallel-tooling.md) when batching the scan below — independent reads belong in one turn, long shells run via `run_in_background` + `Monitor`.
42
42
 
43
- Scan the codebase for patterns related to sensitive data. Pay attention to:
44
-
45
- - **Log Statements:** Search for `console.log`, `logger.info`, etc., that might
46
- be outputting `user`, `email`, `password`, `token`, `address`, or `phone`.
47
- - **Storage:** Check `localStorage`, `sessionStorage`, and database schemas for
48
- unencrypted sensitive fields.
49
- - **API Requests:** Review outgoing requests to ensure PII is not leaked in URLs
50
- (query params) or unencrypted headers.
51
- - **Analytics:** Ensure third-party analytics calls are anonymized.
43
+ A PII leak is a **source sink** flow: sensitive data reaching an egress point.
44
+ Enumerate the **sinks** first, then trace which ones receive PII. Report only
45
+ **proven flows** a sink that never touches a PII source is not a finding.
46
+
47
+ 1. **Enumerate the sinks.** Run these verbatim `rg` commands (they anchor every
48
+ finding to a real line):
49
+
50
+ ```bash
51
+ # Logging sinks
52
+ rg -n "\b(console\.(log|info|warn|error|debug)|logger\.(info|warn|error|debug|log))\s*\(" --glob '!**/*.test.*'
53
+ # Telemetry / analytics sinks
54
+ rg -n "\b(track|capture|analytics|telemetry|reportEvent|Sentry\.(captureException|captureMessage))\s*\(" --glob '!**/*.test.*'
55
+ # Persistence sinks
56
+ rg -n "\b(localStorage|sessionStorage|\.set\(|db\.(insert|update|save)|prisma\.\w+\.(create|update|upsert))\b" --glob '!**/*.test.*'
57
+ # Outbound-HTTP sinks (PII in URLs / bodies / headers)
58
+ rg -n "\b(fetch|axios|http\.request|got|ky)\s*\(" --glob '!**/*.test.*'
59
+ ```
60
+
61
+ 2. **Secret scan.** Prefer `gitleaks`; fall back to `rg`:
62
+
63
+ ```bash
64
+ command -v gitleaks >/dev/null 2>&1 && gitleaks detect --no-banner --redact -v || \
65
+ rg -n -i "(api[_-]?key|secret|password|token|salt)\s*[:=]\s*['\"][^'\"]{8,}" --glob '!**/*.test.*'
66
+ ```
67
+
68
+ 3. **Trace PII sources to the enumerated sinks.** PII source tokens to follow:
69
+ `email`, `password`, `token`, `phone`, `address`, `ssn`, `dob`, `ip`,
70
+ `fullName`, `firstName`/`lastName`, `creditCard`, `user` object spreads. For
71
+ each sink from step 1, decide whether a PII source reaches it (directly, or
72
+ via a variable/object logged whole). Only a **proven** source→sink flow
73
+ becomes a finding; cite both the source line and the sink line.
52
74
 
53
75
  ## Step 2: Analysis Dimensions
54
76
 
@@ -70,6 +92,9 @@ Evaluate the codebase against these privacy pillars:
70
92
  Generate and save a highly structured Markdown audit report to
71
93
  `{{auditOutputDir}}/audit-privacy-results.md`, using the exact template below.
72
94
 
95
+ > Grade every finding's severity on the shared
96
+ > [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md).
97
+
73
98
  ```markdown
74
99
  # Privacy & PII Audit Report
75
100
 
@@ -85,15 +110,18 @@ Generate and save a highly structured Markdown audit report to
85
110
 
86
111
  ## Detailed Findings
87
112
 
88
- [For every gap identified, use the following strict structure:]
113
+ [For every gap identified, use the following strict structure. Lead each title
114
+ with the primary file the finding lives in:]
89
115
 
90
- ### [Short Title of the Issue]
116
+ ### `path/to/primary-file.ext` — [Short title of the issue]
91
117
 
92
- - **Type:** [Leaky Log | Insecure Storage | Data Over-collection]
118
+ - **Dimension:** [Leaky Log | Insecure Storage | Data Over-collection]
93
119
  - **Impact:** [Critical | High | Medium | Low]
120
+ - **Location:** `path/to/primary-file.ext:line`
94
121
  - **Current State:** [The specific file/line/module and why it is problematic]
95
122
  - **Recommendation & Rationale:** [How to remediate and why it's necessary for
96
123
  compliance]
124
+ - **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. a grep for the leaky log that now returns empty, or a re-run of this lens]
97
125
  - **Agent Prompt:**
98
126
  `[A copy-pasteable, highly specific prompt to execute this remediation independently]`
99
127
  ```
@@ -102,3 +130,13 @@ Generate and save a highly structured Markdown audit report to
102
130
 
103
131
  This is a **read-only** audit. Do not modify any code. Focus on identifying
104
132
  risks and providing clear remediation steps.
133
+
134
+ ## Self-cross-check (mandatory — filter false positives before you finalize)
135
+
136
+ Before you write the report artifact from the previous step, run the shared
137
+ adversarial self-cross-check over your Detailed Findings — see
138
+ [`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
139
+ per-finding evidence bar, the exclusion list, and the final re-open-and-drop
140
+ pass whose `kept <k> / dropped <d>` counts you record in the Executive
141
+ Summary, so the sequential single-pass path filters unverified findings just as
142
+ the orchestrated path's adversarial reviewer does.
@@ -47,62 +47,52 @@ before this section existed.
47
47
 
48
48
  ## Execution strategy (dual-path)
49
49
 
50
- This lens runs along one of two execution paths. Both emit the **identical**
51
- report contract (Step 3); downstream consumers (`audit-to-stories`) are agnostic to which path produced it.
52
-
53
- - **Orchestrated (dynamic-workflow) path.** When Claude Code's
54
- [dynamic workflows](https://code.claude.com/docs/en/workflows) are
55
- available, the saved project workflow
56
- `.claude/workflows/audit-quality.workflow.js` fans the dimensions below
57
- out as parallel read-only subagents, runs an **adversarial cross-check**
58
- stage (an independent agent reviews each dimension's findings and drops
59
- false positives before they enter the report), then synthesises the Step 3
60
- report. The orchestrator derives its per-dimension prompts from *this*
61
- markdown at run time — the lens stays the single source of truth; the
62
- script does not fork a second copy of the spec.
63
- - **Sequential (single-pass) path.** When dynamic workflows are unavailable,
64
- follow Steps 1–3 below turn-by-turn exactly as before. This is the default
65
- fallback and changes nothing about the existing behaviour.
66
-
67
- **Strategy selection** is computed by
68
- [`lib/dynamic-workflow/capability.js`](../scripts/lib/dynamic-workflow/capability.js)
69
- (`selectAuditStrategy`). The orchestrated path is chosen only when the runtime
70
- is Claude Code, `disableWorkflows` is not set (settings.json **or**
71
- `CLAUDE_CODE_DISABLE_WORKFLOWS`), and the Claude Code version meets the
72
- research-preview floor (`>= 2.1.154`). Any other runtime, a disabled setting,
73
- or an older version degrades gracefully to the sequential path.
74
-
75
- > **Capability degradation, not a contract shim.** This dual path is **not**
76
- > covered by the No-Shim / hard-cutover rule in
77
- > [`git-conventions.md`](../rules/git-conventions.md). That rule forbids
78
- > running two shapes of the *same contract* side by side. Here there is **one**
79
- > report contract; only the *execution strategy* is selected from a runtime
80
- > capabilitythe same pattern the protocol already endorses for live-docs
81
- > fallback in [`instructions.md` §1.C/§1.D](../instructions.md). The full
82
- > capability-degradation rationale lives in the
83
- > [`capability.js`](../scripts/lib/dynamic-workflow/capability.js) module
84
- > docstring; the orchestrated-run evidence and per-lens cost/precision gate
85
- > verdicts live in [`docs/roadmap.md`](../../docs/roadmap.md) (Part 3
86
- > Dynamic-Workflow Orchestration).
87
-
88
- **Forcing a path (for testing).** Set `MANDREL_AUDIT_STRATEGY=sequential` to
89
- verify the fallback path with the feature notionally disabled, or
90
- `MANDREL_AUDIT_STRATEGY=orchestrated` to pin the dynamic path. To exercise the
91
- real disable signals instead, set `CLAUDE_CODE_DISABLE_WORKFLOWS=1` (env) or
92
- `disableWorkflows: true` in `.claude/settings.json` and re-run the lens — both
93
- degrade to the sequential path.
94
-
95
- > **Read-only on both paths.** The lens is read-only (see Constraint). The
96
- > orchestrated subagents run in `acceptEdits` and inherit the session tool
97
- > allowlist, but the workflow script grants the analysis agents only
98
- > read/search tools (`Read`, `Grep`, `Glob`) — no write/edit/shell-mutation
99
- > tools. The single write in an orchestrated run is the final report artifact.
100
-
101
- ## Step 0 - Project Context
102
-
103
- 1. Read the Story under audit — its `## Goal`, inline `acceptance[]` /
104
- `verify[]`, and folded `## Spec` — to identify the target features.
105
- 2. Identify the target codebase paths for the audit.
50
+ This lens runs along one of two execution paths (orchestrated dynamic-workflow
51
+ or sequential single-pass). Both emit the **identical** Step 3 report contract;
52
+ downstream consumers (`audit-to-stories`) are agnostic to which path produced
53
+ it. See [`helpers/audit-dual-path.md`](helpers/audit-dual-path.md) for strategy
54
+ selection, the forcing flags, and the read-only guarantee — read `audit-<lens>`
55
+ there as this lens's name.
56
+
57
+ ## Step 0 - Mode split + tool-first artifact read (mandatory)
58
+
59
+ **Resolve the mode first**, then read the numbers before judging. The two modes
60
+ do not share a Step 0 — a codebase-wide run must not try to read a Story it was
61
+ never given.
62
+
63
+ - **Story-scoped mode** (the `## Scope` block above is populated with a change
64
+ set): read the Story under audit its `## Goal`, inline `acceptance[]` /
65
+ `verify[]`, and folded `## Spec` — to identify the target features, and scope
66
+ the audit to the change set and its direct dependencies.
67
+ - **Codebase-wide mode** (the `## Scope` block renders the literal
68
+ `{{changedFiles}}` token): there is **no Story** — do not look for one. Audit
69
+ the whole test surface, ranked (below).
70
+
71
+ **Read the committed test-quality artifacts as evidence** (both modes). This
72
+ lens grounds every coverage/quality claim in the metrics the delivery gates
73
+ already compute and commit, rather than prose-scanning the tests:
74
+
75
+ - `baselines/coverage.json` per-file line/branch coverage. Cite the covered
76
+ ratio for any file you flag as under-tested.
77
+ - `baselines/crap.json` — the CRAP score (complexity × uncoveredness). A high
78
+ CRAP row is a measured "complex **and** under-tested" hotspot the single
79
+ strongest coverage-gap signal.
80
+ - `baselines/mutation.json`mutation-testing survivors where present: tests
81
+ that execute code without asserting on it (coverage without confidence).
82
+
83
+ **Rank churn-by-coverage.** Order candidate findings by **churn × coverage
84
+ gap** frequently-changed files (`git log --format= --name-only -n 200 | sort
85
+ | uniq -c | sort -rn`) that also score low coverage / high CRAP are the
86
+ highest-value gaps. Lead the report with them; cap the Detailed Findings at the
87
+ top hotspots so the output is an actionable batch, not an exhaustive dump.
88
+
89
+ **Anchor the rubric** to [`rules/testing-standards.md`](../rules/testing-standards.md):
90
+ the three-tier pyramid, assertion-placement, and mocking/isolation MUSTs are the
91
+ standard a finding is measured against cite the rule the test violates rather
92
+ than asserting a bare opinion.
93
+
94
+ Reading these committed artifacts is **read-only** and explicitly permitted (see
95
+ the Constraint) it is not "running the suite".
106
96
 
107
97
  ## Step 1: Context Gathering (Read-Only Scan)
108
98
 
@@ -149,6 +139,9 @@ Evaluate the gathered context against the following test quality dimensions:
149
139
  Generate and save a highly structured Markdown audit report to
150
140
  `{{auditOutputDir}}/audit-quality-results.md`, using the exact template below.
151
141
 
142
+ > Grade every finding's severity on the shared
143
+ > [`Critical | High | Medium | Low` scale](helpers/audit-severity-scale.md).
144
+
152
145
  ```markdown
153
146
  # Testing & Quality Assurance Audit
154
147
 
@@ -168,16 +161,19 @@ primary vulnerabilities, coverage gaps, and areas causing developer friction.]
168
161
 
169
162
  ## Detailed Findings
170
163
 
171
- [For every gap identified, use the following strict structure:]
164
+ [For every gap identified, use the following strict structure. Lead each title
165
+ with the primary file the finding lives in:]
172
166
 
173
- ### [Short Title of the Issue]
167
+ ### `path/to/primary-file.ext` — [Short title of the issue]
174
168
 
175
169
  - **Category:** [Flakiness | Coverage | Performance | Mocking | Test Plans]
176
- - **Impact:** [High | Medium | Low]
170
+ - **Impact:** [Critical | High | Medium | Low]
171
+ - **Location:** `path/to/primary-file.ext:line`
177
172
  - **Current State:** [How the tests are currently written and why it's
178
173
  problematic]
179
174
  - **Recommendation & Rationale:** [The specific testing pattern or refactor
180
175
  strategy to fix the issue]
176
+ - **Acceptance signal:** [the command or observable that proves this finding is remediated — e.g. the new test failing before / passing after the fix, a coverage re-check, or a re-run of this lens]
181
177
  - **Agent Prompt:**
182
178
  `[A copy-pasteable, highly specific prompt to execute this fix independently]`
183
179
  ```
@@ -186,5 +182,19 @@ primary vulnerabilities, coverage gaps, and areas causing developer friction.]
186
182
 
187
183
  ## Constraint
188
184
 
189
- Do NOT execute any code modifications, edit files, create branches, or run the
190
- test suite. This is strictly a read-only analysis. Output the report and stop.
185
+ Do NOT execute any code modifications, edit files, create branches, or **run**
186
+ the test suite (do not invoke `npm test`, a coverage run, or a mutation run
187
+ those mutate state and cost minutes). Reading the **committed** coverage / CRAP
188
+ / mutation artifacts under `baselines/` is explicitly permitted and required
189
+ (Step 0): citing an already-computed metric is read-only analysis, not a suite
190
+ run. Output the report and stop.
191
+
192
+ ## Self-cross-check (mandatory — filter false positives before you finalize)
193
+
194
+ Before you write the report artifact from the previous step, run the shared
195
+ adversarial self-cross-check over your Detailed Findings — see
196
+ [`helpers/audit-self-check.md`](helpers/audit-self-check.md). It defines the
197
+ per-finding evidence bar, the exclusion list, and the final re-open-and-drop
198
+ pass whose `kept <k> / dropped <d>` counts you record in the Executive
199
+ Summary, so the sequential single-pass path filters unverified findings just as
200
+ the orchestrated path's adversarial reviewer does.