nightytidy 0.3.7 → 0.3.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/nightytidy.js +1 -1
- package/package.json +1 -1
- package/src/agent/git-integration.js +4 -1
- package/src/claude.js +1 -1
- package/src/prompts/manifest.json +138 -138
- package/src/prompts/steps/02-test-coverage.md +181 -181
- package/src/prompts/steps/03-test-hardening.md +181 -181
- package/src/prompts/steps/04-test-architecture.md +130 -130
- package/src/prompts/steps/05-test-consolidation.md +165 -165
- package/src/prompts/steps/06-test-quality.md +211 -211
- package/src/prompts/steps/07-api-design.md +165 -165
- package/src/prompts/steps/08-security-sweep.md +207 -207
- package/src/prompts/steps/09-dependency-health.md +217 -217
- package/src/prompts/steps/10-codebase-cleanup.md +189 -189
- package/src/prompts/steps/11-crosscutting-concerns.md +196 -196
- package/src/prompts/steps/12-file-decomposition.md +263 -263
- package/src/prompts/steps/13-code-elegance.md +329 -329
- package/src/prompts/steps/14-architectural-complexity.md +297 -297
- package/src/prompts/steps/15-type-safety.md +192 -192
- package/src/prompts/steps/16-logging-error-message.md +173 -173
- package/src/prompts/steps/17-data-integrity.md +139 -139
- package/src/prompts/steps/18-performance.md +183 -183
- package/src/prompts/steps/19-cost-resource-optimization.md +136 -136
- package/src/prompts/steps/20-error-recovery.md +145 -145
- package/src/prompts/steps/21-race-condition-audit.md +178 -178
- package/src/prompts/steps/22-bug-hunt.md +229 -229
- package/src/prompts/steps/23-frontend-quality.md +210 -210
- package/src/prompts/steps/24-uiux-audit.md +284 -284
- package/src/prompts/steps/25-state-management.md +170 -170
- package/src/prompts/steps/26-perceived-performance.md +190 -190
- package/src/prompts/steps/27-devops.md +165 -165
- package/src/prompts/steps/28-scheduled-job-chron-jobs.md +141 -141
- package/src/prompts/steps/29-observability.md +152 -152
- package/src/prompts/steps/30-backup-check.md +155 -155
- package/src/prompts/steps/31-product-polish-ux-friction.md +122 -122
- package/src/prompts/steps/32-feature-discovery-opportunity.md +128 -128
- package/src/prompts/steps/33-strategic-opportunities.md +217 -217
|
@@ -1,229 +1,229 @@
|
|
|
1
|
-
# Bug Hunt
|
|
2
|
-
|
|
3
|
-
Overnight bug detection pass. Find bugs — logic errors, off-by-ones, unhandled edge cases, silent failures, incorrect assumptions. Be thorough and skeptical. Read every file.
|
|
4
|
-
|
|
5
|
-
**Default posture: SURFACE bugs, not fix them.** Only fix if ALL criteria are met: (1) ≥90% confident it's a bug, (2) mechanical/obvious fix, (3) tests exist to verify, (4) no business logic or user-facing behavior change. Everything else: document only.
|
|
6
|
-
|
|
7
|
-
Branch: `bug-hunt-[date]` · Commit format: `fix: [brief description] in [file/module]`
|
|
8
|
-
|
|
9
|
-
## Global Rules
|
|
10
|
-
|
|
11
|
-
- Run tests after every fix. Fail → revert immediately → reclassify as document-only.
|
|
12
|
-
- DO NOT change business logic, refactor, or install new tools.
|
|
13
|
-
- A bug fix changes the minimum lines to correct the defect — nothing else.
|
|
14
|
-
- False positives are cheap. Missed bugs are expensive. When in doubt, report it.
|
|
15
|
-
- When you find a bug, search for the same pattern elsewhere. Bugs cluster.
|
|
16
|
-
- For every finding, include: what's wrong, why, likely correct behavior, trigger/reproduction, confidence (High/Medium/Low), and a suggested test.
|
|
17
|
-
|
|
18
|
-
---
|
|
19
|
-
|
|
20
|
-
## Phase 1: Static Analysis — Pattern-Based Detection
|
|
21
|
-
|
|
22
|
-
Scan the entire codebase for each category explicitly.
|
|
23
|
-
|
|
24
|
-
### 1. Comparison & Equality
|
|
25
|
-
- `=` vs `==`/`===` in conditionals
|
|
26
|
-
- `==` vs `===` (type coercion: `0 == ""`, `null == undefined`, `"0" == false`)
|
|
27
|
-
- Inverted comparisons (`>` vs `<`, `>=` vs `>`)
|
|
28
|
-
- Unhandled `null`/`undefined`/`NaN` in comparisons (`NaN !== NaN`; `null >= 0` is true but `null > 0` is false)
|
|
29
|
-
- Float equality without epsilon
|
|
30
|
-
- String vs numeric comparison (`"10" < "9"` is true)
|
|
31
|
-
- Reference equality where deep equality was intended
|
|
32
|
-
|
|
33
|
-
### 2. Off-by-One & Boundaries
|
|
34
|
-
- `array[array.length]`, `array[-1]` (both undefined in JS)
|
|
35
|
-
- Loop bounds: `<` vs `<=`, start index 0 vs 1, `.length` vs `.length - 1`
|
|
36
|
-
- Substring/slice inclusive vs exclusive end
|
|
37
|
-
- Pagination: page 0 vs 1, last page calc (`ceil` vs `floor`), empty last page
|
|
38
|
-
- Date/time: midnight, month boundaries (31→28), timezone crossing, DST
|
|
39
|
-
- Chained range checks (`min <= value <= max` doesn't work in most languages)
|
|
40
|
-
- Fence-post errors in counting/partitioning
|
|
41
|
-
|
|
42
|
-
### 3. Null/Undefined/Empty Handling
|
|
43
|
-
- Property access on potentially null/undefined without checks
|
|
44
|
-
- Missing `?.` or wrong fallback (`?? "Unknown"` when null means something different)
|
|
45
|
-
- Empty string as falsy when valid (`if (!input)` rejects `""` and `null`)
|
|
46
|
-
- Empty array/object as truthy (`if (results)` is always true for `[]`)
|
|
47
|
-
- Default params masking caller bugs (`f(x = 0)` — is 0 valid or hiding a missing arg?)
|
|
48
|
-
- Destructuring without nested defaults
|
|
49
|
-
- Missing `.length === 0` before accessing first/last element
|
|
50
|
-
|
|
51
|
-
### 4. Async & Promises
|
|
52
|
-
- Missing `await` (returns Promise instead of value — often silent)
|
|
53
|
-
- `await` inside `forEach` (doesn't await — use `for...of` or `Promise.all(arr.map(...))`)
|
|
54
|
-
- Missing `.catch()` / try-catch on promises
|
|
55
|
-
- Race conditions assuming sequential async execution
|
|
56
|
-
- `async` functions that never await (unnecessary wrapper or forgotten await)
|
|
57
|
-
- `new Promise(async (resolve) => ...)` anti-pattern
|
|
58
|
-
- Swallowed errors in middleware catch blocks
|
|
59
|
-
- `Promise.all` where `Promise.allSettled` was needed
|
|
60
|
-
- Async in constructors or synchronous-looking paths
|
|
61
|
-
|
|
62
|
-
### 5. Logic Errors
|
|
63
|
-
- De Morgan violations, double negatives inverting intent
|
|
64
|
-
- Short-circuit side effects: `a && doSomething()` where `doSomething` should always run
|
|
65
|
-
- Switch missing `break` (fall-through), missing `default`, incomplete enum coverage
|
|
66
|
-
- Early returns skipping cleanup (resource release, state reset)
|
|
67
|
-
- Identical then/else branches (copy-paste)
|
|
68
|
-
- Always-true/false conditions (dead branches)
|
|
69
|
-
- Variable shadowing (inner scope hiding outer value)
|
|
70
|
-
- Operator precedence: `a & b == c` → `a & (b == c)`
|
|
71
|
-
- Chained ternary associativity
|
|
72
|
-
- `x || default` failing on falsy valid values (`0`, `""`, `false`)
|
|
73
|
-
|
|
74
|
-
### 6. Data & Type Bugs
|
|
75
|
-
- Mutating shared objects/arrays passed by reference
|
|
76
|
-
- Sort without comparator (JS default is lexicographic: `[10,9,80].sort()` → `[10,80,9]`)
|
|
77
|
-
- Integer overflow/underflow
|
|
78
|
-
- `parseInt` pitfalls: `parseInt("08")` octal, `parseInt("123abc")` → 123, `Number("")` → 0
|
|
79
|
-
- Regex: unescaped specials, missing anchors, greedy vs lazy, catastrophic backtracking
|
|
80
|
-
- `JSON.parse`/`stringify`: `undefined` dropped, `Date` → string, `BigInt` throws, circular refs
|
|
81
|
-
- Spread shallow copy (nested objects share references)
|
|
82
|
-
- Map/Set with object keys (reference equality)
|
|
83
|
-
|
|
84
|
-
### 7. API & Integration
|
|
85
|
-
- HTTP status codes unchecked (assuming success)
|
|
86
|
-
- Response body structure assumed without validation
|
|
87
|
-
- No timeout on HTTP requests
|
|
88
|
-
- Retry on non-idempotent operations (POST retry = duplicate)
|
|
89
|
-
- Non-idempotent webhook handlers (redelivery = duplicate processing)
|
|
90
|
-
- URL construction: missing `encodeURIComponent`, double slashes, query param bugs
|
|
91
|
-
- Content-Type mismatches
|
|
92
|
-
- Pagination: not fetching all pages, off-by-one, ignoring `hasMore`/`nextCursor`
|
|
93
|
-
|
|
94
|
-
### 8. Security-Adjacent Logic
|
|
95
|
-
- Auth check without permission-level check
|
|
96
|
-
- IDOR: user-supplied ID without ownership verification
|
|
97
|
-
- Rate limit bypass via alternate routes
|
|
98
|
-
- Timing leaks (different response times for user-exists vs not)
|
|
99
|
-
- Mass assignment: request body spread into DB update without field filtering
|
|
100
|
-
- Permission checked at entry but not on downstream operations
|
|
101
|
-
|
|
102
|
-
---
|
|
103
|
-
|
|
104
|
-
## Phase 2: Semantic Analysis — Intent vs Implementation
|
|
105
|
-
|
|
106
|
-
### Function Contract Analysis
|
|
107
|
-
For every function on critical paths (auth, payments, data mutation, core logic):
|
|
108
|
-
- Infer intended contract from name, params, docs, callers
|
|
109
|
-
- Compare to actual behavior for ALL inputs, including edge cases
|
|
110
|
-
- Check if callers can violate the function's assumptions
|
|
111
|
-
|
|
112
|
-
### State Machine Verification
|
|
113
|
-
For every entity with status/state (orders, payments, subscriptions, etc.):
|
|
114
|
-
- Map all states and transitions (code locations)
|
|
115
|
-
- Check for: unguarded impossible transitions, states with no exit, skipped intermediate states, concurrent transition conflicts
|
|
116
|
-
|
|
117
|
-
### Business Rule Consistency
|
|
118
|
-
- Same rules applied consistently everywhere? (Discount in checkout AND summary AND invoice)
|
|
119
|
-
- Frontend-only validation without backend enforcement?
|
|
120
|
-
- Hardcoded values duplicated with different values across files?
|
|
121
|
-
|
|
122
|
-
### Error Path Analysis
|
|
123
|
-
For every error handler/catch/fallback:
|
|
124
|
-
- Does it accomplish the intended recovery?
|
|
125
|
-
- Does the error propagate correctly or get swallowed?
|
|
126
|
-
- Is the user informed? Or does it fail silently?
|
|
127
|
-
- Is the system left in a consistent state? (Partial writes, uncommitted transactions, half-updated UI)
|
|
128
|
-
|
|
129
|
-
---
|
|
130
|
-
|
|
131
|
-
## Phase 3: Data Flow Analysis
|
|
132
|
-
|
|
133
|
-
### Trace Critical Flows End-to-End (top 5-10 operations)
|
|
134
|
-
- Data correctness through every transformation
|
|
135
|
-
- Types preserved/converted correctly at each boundary
|
|
136
|
-
- Precision preserved (float money, timestamp truncation, timezone loss)
|
|
137
|
-
- Required fields guaranteed present at every stage
|
|
138
|
-
|
|
139
|
-
### Cross-Feature Interference
|
|
140
|
-
- Two features writing to the same field with different expectations
|
|
141
|
-
- Stale cached values modified by another feature
|
|
142
|
-
- Feature A's error handler resetting state Feature B depends on
|
|
143
|
-
- Event handler ordering across features
|
|
144
|
-
|
|
145
|
-
### Migration & Backwards Compatibility
|
|
146
|
-
- DB fields that changed meaning over time (old records have different semantics)
|
|
147
|
-
- API consumers sending old-format requests
|
|
148
|
-
- Serialized objects (DB, cache, queue) in old format
|
|
149
|
-
- Code reading data without accounting for schema changes without data migration
|
|
150
|
-
|
|
151
|
-
---
|
|
152
|
-
|
|
153
|
-
## Phase 4: Test-Informed Detection
|
|
154
|
-
|
|
155
|
-
### Existing Failures & Skips
|
|
156
|
-
- Skipped tests with `// BUG`, `// FIXME`, `// broken`, `// flaky` = known unfixed bugs
|
|
157
|
-
- Tests asserting surprising behavior (`// this is weird but correct`) — verify it IS correct
|
|
158
|
-
|
|
159
|
-
### Coverage Gaps
|
|
160
|
-
- Code with NO test coverage = most likely bug locations
|
|
161
|
-
- Functions tested only happy-path (edge cases are where bugs live)
|
|
162
|
-
- Untested error paths
|
|
163
|
-
|
|
164
|
-
### Test Correctness
|
|
165
|
-
- Tests asserting the wrong thing (passes but doesn't verify correct behavior)
|
|
166
|
-
- Tautological tests (`expect(mock).toHaveBeenCalled()` on unconditionally-called mock)
|
|
167
|
-
- Tests that test the mock more than the code
|
|
168
|
-
|
|
169
|
-
---
|
|
170
|
-
|
|
171
|
-
## Phase 5: Fix High-Confidence Bugs
|
|
172
|
-
|
|
173
|
-
For each finding meeting ALL fix criteria:
|
|
174
|
-
1. Write minimal fix → 2. Run full test suite → 3. Pass: commit → 4. Fail: revert, reclassify as document-only
|
|
175
|
-
|
|
176
|
-
**Fixable:** Missing null checks, `==`→`===`, missing `await`, pagination off-by-one, swallowed errors, missing `break`, numeric sort without comparator.
|
|
177
|
-
|
|
178
|
-
**Document-only:** Business logic that might be intentional, race conditions needing architecture changes, state machine gaps needing product decisions, performance issues, ambiguous "correct" behavior.
|
|
179
|
-
|
|
180
|
-
---
|
|
181
|
-
|
|
182
|
-
## Output
|
|
183
|
-
|
|
184
|
-
Save to `audit-reports/22_BUG_HUNT_REPORT_[run-number]_[date]_[time in user's local time].md` (create dir if needed, increment run number).
|
|
185
|
-
|
|
186
|
-
### Report Sections
|
|
187
|
-
1. **Executive Summary** — Total bugs, confidence breakdown, fixed vs documented, critical count, highest-density areas.
|
|
188
|
-
2. **Critical Bugs** — Data loss, security, core UX. Per bug: Title, Location (file/line/function), Confidence + reasoning, Description, Impact, Trigger condition, Suggested fix, Status (Fixed w/ commit or Document Only w/ reason).
|
|
189
|
-
3. **High-Priority** — Incorrect behavior, data inconsistency, degraded UX. Same format.
|
|
190
|
-
4. **Medium-Priority** — Edge cases, minor issues. Same format.
|
|
191
|
-
5. **Low-Priority / Potential** — Suspicious but uncertain. Emphasize confidence and what would confirm/disprove.
|
|
192
|
-
6. **Bugs Fixed Table** — | File | Bug | Fix | Confidence | Tests Pass? | Commit |
|
|
193
|
-
7. **State Machine Analysis** — Per entity: state diagram, valid transitions w/ code locations, missing guards, stuck/impossible states.
|
|
194
|
-
8. **Data Flow Findings** — Transformation issues, cross-feature interference, schema/migration concerns.
|
|
195
|
-
9. **Test Suite Observations** — Known-bug tests, suspicious assertions, coverage gaps, test correctness issues.
|
|
196
|
-
10. **Bug Density Map** — Files/modules with most findings.
|
|
197
|
-
11. **Recommendations** — Recurring patterns, missing validation, tests to write.
|
|
198
|
-
|
|
199
|
-
---
|
|
200
|
-
|
|
201
|
-
## Chat Output (Primary Deliverable)
|
|
202
|
-
|
|
203
|
-
The user should NOT need to open the report. Print everything that matters in the conversation.
|
|
204
|
-
|
|
205
|
-
### 1. Status Line
|
|
206
|
-
One sentence: what you did, duration, whether tests pass.
|
|
207
|
-
|
|
208
|
-
### 2. Bugs Fixed
|
|
209
|
-
Per fix: file/line, what was wrong (1 sentence), what you changed (1 sentence), confidence + reasoning (e.g. "99% — mechanical, tests cover this"), commit. Flag any ambiguity with ⚠️.
|
|
210
|
-
|
|
211
|
-
### 3. Bugs Found — Needs Human Review
|
|
212
|
-
ALL bugs not fixed, by severity (Critical → Low). Per bug:
|
|
213
|
-
- **Severity + confidence** (e.g. "HIGH (Medium confidence)")
|
|
214
|
-
- **File/line**
|
|
215
|
-
- **What's wrong** (specific: code does X, should do Y)
|
|
216
|
-
- **Trigger condition** (specific: "when user submits empty array for line items")
|
|
217
|
-
- **Impact** (data corruption, wrong price, silent failure, crash, etc.)
|
|
218
|
-
- **Suggested fix** (plain language or short snippet)
|
|
219
|
-
- **Why not fixed** ("not confident it's unintentional" / "no test coverage" / "requires business decision" / "too many call sites")
|
|
220
|
-
|
|
221
|
-
List ALL findings. Completeness beats brevity.
|
|
222
|
-
|
|
223
|
-
### 4. Patterns & Hot Spots
|
|
224
|
-
- Bug clusters (e.g. "`payments/` had 6 findings")
|
|
225
|
-
- Recurring patterns (e.g. "missing null checks ×8 — consider lint rule")
|
|
226
|
-
- Risky untested areas
|
|
227
|
-
|
|
228
|
-
### 5. Report Location
|
|
229
|
-
Full path to detailed report.
|
|
1
|
+
# Bug Hunt
|
|
2
|
+
|
|
3
|
+
Overnight bug detection pass. Find bugs — logic errors, off-by-ones, unhandled edge cases, silent failures, incorrect assumptions. Be thorough and skeptical. Read every file.
|
|
4
|
+
|
|
5
|
+
**Default posture: SURFACE bugs, not fix them.** Only fix if ALL criteria are met: (1) ≥90% confident it's a bug, (2) mechanical/obvious fix, (3) tests exist to verify, (4) no business logic or user-facing behavior change. Everything else: document only.
|
|
6
|
+
|
|
7
|
+
Branch: `bug-hunt-[date]` · Commit format: `fix: [brief description] in [file/module]`
|
|
8
|
+
|
|
9
|
+
## Global Rules
|
|
10
|
+
|
|
11
|
+
- Run tests after every fix. Fail → revert immediately → reclassify as document-only.
|
|
12
|
+
- DO NOT change business logic, refactor, or install new tools.
|
|
13
|
+
- A bug fix changes the minimum lines to correct the defect — nothing else.
|
|
14
|
+
- False positives are cheap. Missed bugs are expensive. When in doubt, report it.
|
|
15
|
+
- When you find a bug, search for the same pattern elsewhere. Bugs cluster.
|
|
16
|
+
- For every finding, include: what's wrong, why, likely correct behavior, trigger/reproduction, confidence (High/Medium/Low), and a suggested test.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Phase 1: Static Analysis — Pattern-Based Detection
|
|
21
|
+
|
|
22
|
+
Scan the entire codebase for each category explicitly.
|
|
23
|
+
|
|
24
|
+
### 1. Comparison & Equality
|
|
25
|
+
- `=` vs `==`/`===` in conditionals
|
|
26
|
+
- `==` vs `===` (type coercion: `0 == ""`, `null == undefined`, `"0" == false`)
|
|
27
|
+
- Inverted comparisons (`>` vs `<`, `>=` vs `>`)
|
|
28
|
+
- Unhandled `null`/`undefined`/`NaN` in comparisons (`NaN !== NaN`; `null >= 0` is true but `null > 0` is false)
|
|
29
|
+
- Float equality without epsilon
|
|
30
|
+
- String vs numeric comparison (`"10" < "9"` is true)
|
|
31
|
+
- Reference equality where deep equality was intended
|
|
32
|
+
|
|
33
|
+
### 2. Off-by-One & Boundaries
|
|
34
|
+
- `array[array.length]`, `array[-1]` (both undefined in JS)
|
|
35
|
+
- Loop bounds: `<` vs `<=`, start index 0 vs 1, `.length` vs `.length - 1`
|
|
36
|
+
- Substring/slice inclusive vs exclusive end
|
|
37
|
+
- Pagination: page 0 vs 1, last page calc (`ceil` vs `floor`), empty last page
|
|
38
|
+
- Date/time: midnight, month boundaries (31→28), timezone crossing, DST
|
|
39
|
+
- Chained range checks (`min <= value <= max` doesn't work in most languages)
|
|
40
|
+
- Fence-post errors in counting/partitioning
|
|
41
|
+
|
|
42
|
+
### 3. Null/Undefined/Empty Handling
|
|
43
|
+
- Property access on potentially null/undefined without checks
|
|
44
|
+
- Missing `?.` or wrong fallback (`?? "Unknown"` when null means something different)
|
|
45
|
+
- Empty string as falsy when valid (`if (!input)` rejects `""` and `null`)
|
|
46
|
+
- Empty array/object as truthy (`if (results)` is always true for `[]`)
|
|
47
|
+
- Default params masking caller bugs (`f(x = 0)` — is 0 valid or hiding a missing arg?)
|
|
48
|
+
- Destructuring without nested defaults
|
|
49
|
+
- Missing `.length === 0` before accessing first/last element
|
|
50
|
+
|
|
51
|
+
### 4. Async & Promises
|
|
52
|
+
- Missing `await` (returns Promise instead of value — often silent)
|
|
53
|
+
- `await` inside `forEach` (doesn't await — use `for...of` or `Promise.all(arr.map(...))`)
|
|
54
|
+
- Missing `.catch()` / try-catch on promises
|
|
55
|
+
- Race conditions assuming sequential async execution
|
|
56
|
+
- `async` functions that never await (unnecessary wrapper or forgotten await)
|
|
57
|
+
- `new Promise(async (resolve) => ...)` anti-pattern
|
|
58
|
+
- Swallowed errors in middleware catch blocks
|
|
59
|
+
- `Promise.all` where `Promise.allSettled` was needed
|
|
60
|
+
- Async in constructors or synchronous-looking paths
|
|
61
|
+
|
|
62
|
+
### 5. Logic Errors
|
|
63
|
+
- De Morgan violations, double negatives inverting intent
|
|
64
|
+
- Short-circuit side effects: `a && doSomething()` where `doSomething` should always run
|
|
65
|
+
- Switch missing `break` (fall-through), missing `default`, incomplete enum coverage
|
|
66
|
+
- Early returns skipping cleanup (resource release, state reset)
|
|
67
|
+
- Identical then/else branches (copy-paste)
|
|
68
|
+
- Always-true/false conditions (dead branches)
|
|
69
|
+
- Variable shadowing (inner scope hiding outer value)
|
|
70
|
+
- Operator precedence: `a & b == c` → `a & (b == c)`
|
|
71
|
+
- Chained ternary associativity
|
|
72
|
+
- `x || default` failing on falsy valid values (`0`, `""`, `false`)
|
|
73
|
+
|
|
74
|
+
### 6. Data & Type Bugs
|
|
75
|
+
- Mutating shared objects/arrays passed by reference
|
|
76
|
+
- Sort without comparator (JS default is lexicographic: `[10,9,80].sort()` → `[10,80,9]`)
|
|
77
|
+
- Integer overflow/underflow
|
|
78
|
+
- `parseInt` pitfalls: `parseInt("08")` octal, `parseInt("123abc")` → 123, `Number("")` → 0
|
|
79
|
+
- Regex: unescaped specials, missing anchors, greedy vs lazy, catastrophic backtracking
|
|
80
|
+
- `JSON.parse`/`stringify`: `undefined` dropped, `Date` → string, `BigInt` throws, circular refs
|
|
81
|
+
- Spread shallow copy (nested objects share references)
|
|
82
|
+
- Map/Set with object keys (reference equality)
|
|
83
|
+
|
|
84
|
+
### 7. API & Integration
|
|
85
|
+
- HTTP status codes unchecked (assuming success)
|
|
86
|
+
- Response body structure assumed without validation
|
|
87
|
+
- No timeout on HTTP requests
|
|
88
|
+
- Retry on non-idempotent operations (POST retry = duplicate)
|
|
89
|
+
- Non-idempotent webhook handlers (redelivery = duplicate processing)
|
|
90
|
+
- URL construction: missing `encodeURIComponent`, double slashes, query param bugs
|
|
91
|
+
- Content-Type mismatches
|
|
92
|
+
- Pagination: not fetching all pages, off-by-one, ignoring `hasMore`/`nextCursor`
|
|
93
|
+
|
|
94
|
+
### 8. Security-Adjacent Logic
|
|
95
|
+
- Auth check without permission-level check
|
|
96
|
+
- IDOR: user-supplied ID without ownership verification
|
|
97
|
+
- Rate limit bypass via alternate routes
|
|
98
|
+
- Timing leaks (different response times for user-exists vs not)
|
|
99
|
+
- Mass assignment: request body spread into DB update without field filtering
|
|
100
|
+
- Permission checked at entry but not on downstream operations
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## Phase 2: Semantic Analysis — Intent vs Implementation
|
|
105
|
+
|
|
106
|
+
### Function Contract Analysis
|
|
107
|
+
For every function on critical paths (auth, payments, data mutation, core logic):
|
|
108
|
+
- Infer intended contract from name, params, docs, callers
|
|
109
|
+
- Compare to actual behavior for ALL inputs, including edge cases
|
|
110
|
+
- Check if callers can violate the function's assumptions
|
|
111
|
+
|
|
112
|
+
### State Machine Verification
|
|
113
|
+
For every entity with status/state (orders, payments, subscriptions, etc.):
|
|
114
|
+
- Map all states and transitions (code locations)
|
|
115
|
+
- Check for: unguarded impossible transitions, states with no exit, skipped intermediate states, concurrent transition conflicts
|
|
116
|
+
|
|
117
|
+
### Business Rule Consistency
|
|
118
|
+
- Same rules applied consistently everywhere? (Discount in checkout AND summary AND invoice)
|
|
119
|
+
- Frontend-only validation without backend enforcement?
|
|
120
|
+
- Hardcoded values duplicated with different values across files?
|
|
121
|
+
|
|
122
|
+
### Error Path Analysis
|
|
123
|
+
For every error handler/catch/fallback:
|
|
124
|
+
- Does it accomplish the intended recovery?
|
|
125
|
+
- Does the error propagate correctly or get swallowed?
|
|
126
|
+
- Is the user informed? Or does it fail silently?
|
|
127
|
+
- Is the system left in a consistent state? (Partial writes, uncommitted transactions, half-updated UI)
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## Phase 3: Data Flow Analysis
|
|
132
|
+
|
|
133
|
+
### Trace Critical Flows End-to-End (top 5-10 operations)
|
|
134
|
+
- Data correctness through every transformation
|
|
135
|
+
- Types preserved/converted correctly at each boundary
|
|
136
|
+
- Precision preserved (float money, timestamp truncation, timezone loss)
|
|
137
|
+
- Required fields guaranteed present at every stage
|
|
138
|
+
|
|
139
|
+
### Cross-Feature Interference
|
|
140
|
+
- Two features writing to the same field with different expectations
|
|
141
|
+
- Stale cached values modified by another feature
|
|
142
|
+
- Feature A's error handler resetting state Feature B depends on
|
|
143
|
+
- Event handler ordering across features
|
|
144
|
+
|
|
145
|
+
### Migration & Backwards Compatibility
|
|
146
|
+
- DB fields that changed meaning over time (old records have different semantics)
|
|
147
|
+
- API consumers sending old-format requests
|
|
148
|
+
- Serialized objects (DB, cache, queue) in old format
|
|
149
|
+
- Code reading data without accounting for schema changes without data migration
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## Phase 4: Test-Informed Detection
|
|
154
|
+
|
|
155
|
+
### Existing Failures & Skips
|
|
156
|
+
- Skipped tests with `// BUG`, `// FIXME`, `// broken`, `// flaky` = known unfixed bugs
|
|
157
|
+
- Tests asserting surprising behavior (`// this is weird but correct`) — verify it IS correct
|
|
158
|
+
|
|
159
|
+
### Coverage Gaps
|
|
160
|
+
- Code with NO test coverage = most likely bug locations
|
|
161
|
+
- Functions tested only happy-path (edge cases are where bugs live)
|
|
162
|
+
- Untested error paths
|
|
163
|
+
|
|
164
|
+
### Test Correctness
|
|
165
|
+
- Tests asserting the wrong thing (passes but doesn't verify correct behavior)
|
|
166
|
+
- Tautological tests (`expect(mock).toHaveBeenCalled()` on unconditionally-called mock)
|
|
167
|
+
- Tests that test the mock more than the code
|
|
168
|
+
|
|
169
|
+
---
|
|
170
|
+
|
|
171
|
+
## Phase 5: Fix High-Confidence Bugs
|
|
172
|
+
|
|
173
|
+
For each finding meeting ALL fix criteria:
|
|
174
|
+
1. Write minimal fix → 2. Run full test suite → 3. Pass: commit → 4. Fail: revert, reclassify as document-only
|
|
175
|
+
|
|
176
|
+
**Fixable:** Missing null checks, `==`→`===`, missing `await`, pagination off-by-one, swallowed errors, missing `break`, numeric sort without comparator.
|
|
177
|
+
|
|
178
|
+
**Document-only:** Business logic that might be intentional, race conditions needing architecture changes, state machine gaps needing product decisions, performance issues, ambiguous "correct" behavior.
|
|
179
|
+
|
|
180
|
+
---
|
|
181
|
+
|
|
182
|
+
## Output
|
|
183
|
+
|
|
184
|
+
Save to `audit-reports/22_BUG_HUNT_REPORT_[run-number]_[date]_[time in user's local time].md` (create dir if needed, increment run number).
|
|
185
|
+
|
|
186
|
+
### Report Sections
|
|
187
|
+
1. **Executive Summary** — Total bugs, confidence breakdown, fixed vs documented, critical count, highest-density areas.
|
|
188
|
+
2. **Critical Bugs** — Data loss, security, core UX. Per bug: Title, Location (file/line/function), Confidence + reasoning, Description, Impact, Trigger condition, Suggested fix, Status (Fixed w/ commit or Document Only w/ reason).
|
|
189
|
+
3. **High-Priority** — Incorrect behavior, data inconsistency, degraded UX. Same format.
|
|
190
|
+
4. **Medium-Priority** — Edge cases, minor issues. Same format.
|
|
191
|
+
5. **Low-Priority / Potential** — Suspicious but uncertain. Emphasize confidence and what would confirm/disprove.
|
|
192
|
+
6. **Bugs Fixed Table** — | File | Bug | Fix | Confidence | Tests Pass? | Commit |
|
|
193
|
+
7. **State Machine Analysis** — Per entity: state diagram, valid transitions w/ code locations, missing guards, stuck/impossible states.
|
|
194
|
+
8. **Data Flow Findings** — Transformation issues, cross-feature interference, schema/migration concerns.
|
|
195
|
+
9. **Test Suite Observations** — Known-bug tests, suspicious assertions, coverage gaps, test correctness issues.
|
|
196
|
+
10. **Bug Density Map** — Files/modules with most findings.
|
|
197
|
+
11. **Recommendations** — Recurring patterns, missing validation, tests to write.
|
|
198
|
+
|
|
199
|
+
---
|
|
200
|
+
|
|
201
|
+
## Chat Output (Primary Deliverable)
|
|
202
|
+
|
|
203
|
+
The user should NOT need to open the report. Print everything that matters in the conversation.
|
|
204
|
+
|
|
205
|
+
### 1. Status Line
|
|
206
|
+
One sentence: what you did, duration, whether tests pass.
|
|
207
|
+
|
|
208
|
+
### 2. Bugs Fixed
|
|
209
|
+
Per fix: file/line, what was wrong (1 sentence), what you changed (1 sentence), confidence + reasoning (e.g. "99% — mechanical, tests cover this"), commit. Flag any ambiguity with ⚠️.
|
|
210
|
+
|
|
211
|
+
### 3. Bugs Found — Needs Human Review
|
|
212
|
+
ALL bugs not fixed, by severity (Critical → Low). Per bug:
|
|
213
|
+
- **Severity + confidence** (e.g. "HIGH (Medium confidence)")
|
|
214
|
+
- **File/line**
|
|
215
|
+
- **What's wrong** (specific: code does X, should do Y)
|
|
216
|
+
- **Trigger condition** (specific: "when user submits empty array for line items")
|
|
217
|
+
- **Impact** (data corruption, wrong price, silent failure, crash, etc.)
|
|
218
|
+
- **Suggested fix** (plain language or short snippet)
|
|
219
|
+
- **Why not fixed** ("not confident it's unintentional" / "no test coverage" / "requires business decision" / "too many call sites")
|
|
220
|
+
|
|
221
|
+
List ALL findings. Completeness beats brevity.
|
|
222
|
+
|
|
223
|
+
### 4. Patterns & Hot Spots
|
|
224
|
+
- Bug clusters (e.g. "`payments/` had 6 findings")
|
|
225
|
+
- Recurring patterns (e.g. "missing null checks ×8 — consider lint rule")
|
|
226
|
+
- Risky untested areas
|
|
227
|
+
|
|
228
|
+
### 5. Report Location
|
|
229
|
+
Full path to detailed report.
|