@codyswann/lisa 2.309.2 → 2.309.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (79) hide show
  1. package/dist/core/upstream-evidence-manifest.js +4 -4
  2. package/package.json +1 -1
  3. package/plugins/lisa/.claude-plugin/plugin.json +1 -1
  4. package/plugins/lisa/.codex-plugin/plugin.json +1 -1
  5. package/plugins/lisa/.codex-plugin/skills/lisa-bug-triage/SKILL.md +2 -2
  6. package/plugins/lisa/.codex-plugin/skills/lisa-reproduce-bug/SKILL.md +40 -69
  7. package/plugins/lisa/.codex-plugin/skills/lisa-reproduce-bug/agents/openai.yaml +2 -2
  8. package/plugins/lisa/.codex-plugin/skills/lisa-root-cause-analysis/SKILL.md +88 -108
  9. package/plugins/lisa/.codex-plugin/skills/lisa-root-cause-analysis/agents/openai.yaml +2 -2
  10. package/plugins/lisa/agents/debug-specialist.md +13 -97
  11. package/plugins/lisa/skills/lisa-bug-triage/SKILL.md +2 -2
  12. package/plugins/lisa/skills/lisa-reproduce-bug/SKILL.md +40 -69
  13. package/plugins/lisa/skills/lisa-reproduce-bug/agents/openai.yaml +2 -2
  14. package/plugins/lisa/skills/lisa-root-cause-analysis/SKILL.md +88 -108
  15. package/plugins/lisa/skills/lisa-root-cause-analysis/agents/openai.yaml +2 -2
  16. package/plugins/lisa-agy/agents/debug-specialist.md +13 -97
  17. package/plugins/lisa-agy/plugin.json +1 -1
  18. package/plugins/lisa-agy/skills/lisa-bug-triage/SKILL.md +2 -2
  19. package/plugins/lisa-agy/skills/lisa-reproduce-bug/SKILL.md +40 -69
  20. package/plugins/lisa-agy/skills/lisa-root-cause-analysis/SKILL.md +88 -108
  21. package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
  22. package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
  23. package/plugins/lisa-cdk-agy/plugin.json +1 -1
  24. package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
  25. package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
  26. package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
  27. package/plugins/lisa-copilot/agents/debug-specialist.agent.md +13 -97
  28. package/plugins/lisa-copilot/skills/lisa-bug-triage/SKILL.md +2 -2
  29. package/plugins/lisa-copilot/skills/lisa-reproduce-bug/SKILL.md +40 -69
  30. package/plugins/lisa-copilot/skills/lisa-root-cause-analysis/SKILL.md +88 -108
  31. package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
  32. package/plugins/lisa-cursor/agents/debug-specialist.md +13 -97
  33. package/plugins/lisa-cursor/skills/lisa-bug-triage/SKILL.md +2 -2
  34. package/plugins/lisa-cursor/skills/lisa-reproduce-bug/SKILL.md +40 -69
  35. package/plugins/lisa-cursor/skills/lisa-root-cause-analysis/SKILL.md +88 -108
  36. package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
  37. package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
  38. package/plugins/lisa-expo-agy/plugin.json +1 -1
  39. package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
  40. package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
  41. package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
  42. package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
  43. package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
  44. package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
  45. package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
  46. package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
  47. package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
  48. package/plugins/lisa-nestjs-agy/plugin.json +1 -1
  49. package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
  50. package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
  51. package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
  52. package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
  53. package/plugins/lisa-openclaw-agy/plugin.json +1 -1
  54. package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
  55. package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
  56. package/plugins/lisa-phaser/.claude-plugin/plugin.json +1 -1
  57. package/plugins/lisa-phaser/.codex-plugin/plugin.json +1 -1
  58. package/plugins/lisa-phaser-agy/plugin.json +1 -1
  59. package/plugins/lisa-phaser-copilot/.claude-plugin/plugin.json +1 -1
  60. package/plugins/lisa-phaser-cursor/.claude-plugin/plugin.json +1 -1
  61. package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
  62. package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
  63. package/plugins/lisa-rails-agy/plugin.json +1 -1
  64. package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
  65. package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
  66. package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
  67. package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
  68. package/plugins/lisa-typescript-agy/plugin.json +1 -1
  69. package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
  70. package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
  71. package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
  72. package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
  73. package/plugins/lisa-wiki-agy/plugin.json +1 -1
  74. package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
  75. package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
  76. package/plugins/src/base/agents/debug-specialist.md +13 -97
  77. package/plugins/src/base/skills/lisa-bug-triage/SKILL.md +2 -2
  78. package/plugins/src/base/skills/lisa-reproduce-bug/SKILL.md +40 -69
  79. package/plugins/src/base/skills/lisa-root-cause-analysis/SKILL.md +88 -108
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-rails",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "Ruby on Rails-specific hooks — RuboCop linting/formatting and ast-grep scanning on edit",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-rails",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "Ruby on Rails-specific skills and hooks for RuboCop and ast-grep scanning on edit.",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-rails",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "Ruby on Rails-specific hooks — RuboCop linting/formatting and ast-grep scanning on edit",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-rails",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "Ruby on Rails-specific hooks — RuboCop linting/formatting and ast-grep scanning on edit",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-rails",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "Ruby on Rails-specific hooks — RuboCop linting/formatting and ast-grep scanning on edit",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-typescript",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "TypeScript-specific hooks — Prettier formatting, ESLint linting, ast-grep scanning, and error-suppression blocking on edit",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-typescript",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "TypeScript-specific hooks for formatting, linting, and ast-grep scanning on edit.",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-typescript",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "TypeScript-specific hooks — Prettier formatting, ESLint linting, ast-grep scanning, and error-suppression blocking on edit",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-typescript",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "TypeScript-specific hooks — Prettier formatting, ESLint linting, ast-grep scanning, and error-suppression blocking on edit",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-typescript",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "TypeScript-specific hooks — Prettier formatting, ESLint linting, ast-grep scanning, and error-suppression blocking on edit",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-wiki",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "LLM Wiki — a distributable, git-native markdown knowledge base for Claude Code and Codex",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-wiki",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "Distributable LLM Wiki kernel — ingest, query, lint, and maintain a git-native markdown knowledge base across Claude and Codex.",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-wiki",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "LLM Wiki — a distributable, git-native markdown knowledge base for Claude Code and Codex",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-wiki",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "LLM Wiki — a distributable, git-native markdown knowledge base for Claude Code and Codex",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "lisa-wiki",
3
- "version": "2.309.2",
3
+ "version": "2.309.3",
4
4
  "description": "LLM Wiki — a distributable, git-native markdown knowledge base for Claude Code and Codex",
5
5
  "author": {
6
6
  "name": "Cody Swann"
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: debug-specialist
3
- description: Debug specialist agent. Expert at root cause analysis, log investigation (local and remote via AWS CloudWatch, scripts, and project tooling), strategic log statement placement, and definitive proof of bug causation. Finds what is causing the problem without a doubt.
3
+ description: Debug specialist agent. Proves what causes a defect — reproduction on the real path, hypotheses confirmed by execution, evidence chains, and log investigation both local and remote (CloudWatch, Sentry, project tooling). Escalates an unresolved verdict with a decision-ready packet rather than guessing when a cause will not yield.
4
4
  skills:
5
5
  - reproduce-bug
6
6
  - root-cause-analysis
@@ -8,107 +8,23 @@ skills:
8
8
 
9
9
  # Debug Specialist Agent
10
10
 
11
- You are a debug specialist whose mission is to **definitively prove** what is causing a problem. You do not guess. You do not theorize without evidence. You trace the actual execution path, read real logs, and produce irrefutable proof of root cause.
11
+ You prove causes. A conclusion you have not executed against is a hypothesis, however well it reads.
12
12
 
13
- ## Core Philosophy
13
+ Both procedures live in your skills — `reproduce-bug` for establishing the failure, `root-cause-analysis` for proving its cause, including the verdict vocabulary, the stopping rule, and both output contracts. Follow them; nothing here restates them, so there is one place to change them.
14
14
 
15
- **"Show me the proof."** Every conclusion must be backed by concrete evidence -- a log line, a stack trace, a reproducible sequence, or a failing test. If you cannot prove it, you have not found the root cause.
15
+ ## What you route
16
16
 
17
- ## Clean Up Log Statements
17
+ - **Which skill the work is in.** No investigation begins before `reproduce-bug` yields a reproduction or a blocked verdict. When it yields neither, that is your finding to report, not a step to work around.
18
+ - **Which technique the symptom calls for.** `root-cause-analysis` carries the menu; choosing badly costs more than any other decision in the session, and a regression with a nameable good commit goes to `git bisect` before anyone reads code.
19
+ - **When the session ends.** You own the budget and the escalation, and an unresolved verdict handed over clearly is a valid end — not a failure to be dressed up as a finding.
18
20
 
19
- After root cause is confirmed, **remove all debug log statements** that were added during investigation. Leave only:
21
+ ## What you hand to bug-fixer
20
22
 
21
- - Log statements that belong in the application permanently (error logging, audit trails)
22
- - Statements explicitly requested by the user
23
+ You do not implement the fix. Pass on, in the forms the two skills define:
23
24
 
24
- Verify cleanup with:
25
- ```bash
26
- # Search for any remaining debug markers
27
- grep -rn "\[DEBUG:" src/ --include="*.ts" --include="*.tsx" --include="*.js"
28
- ```
25
+ - The reproduction — its entry point, its form (failing test, script, or manual steps), and its observed failure rate. **Do not require it to be a failing test**: `reproduce-bug` permits a script or manual steps where the real path allows nothing better, and `bug-fixer` codifies a regression test from whichever form arrived.
26
+ - The verdict, and for a confirmed one, proximate and root cause with `file:line` plus the confirming execution. For an inconclusive or unresolved verdict, the unblocker instead — never a cause invented to fill the field.
29
27
 
30
- ## Output Format
28
+ ## How you are judged
31
29
 
32
- Structure your findings as:
33
-
34
- ```
35
- ## Debug Investigation
36
-
37
- ### Symptom
38
- What was observed -- exact error message, stack trace, or behavior description.
39
-
40
- ### Reproduction
41
- The exact command or sequence that triggers the issue.
42
-
43
- ### Evidence Trail
44
- | Step | Location | Evidence | Conclusion |
45
- |------|----------|----------|------------|
46
- | 1 | file:line | Log output or observed value | What this proves |
47
- | 2 | file:line | Log output or observed value | What this proves |
48
- | ... | ... | ... | ... |
49
-
50
- ### Root Cause
51
- **Proximate cause:** The line that directly produces the error.
52
- **Root cause:** The underlying reason this line behaves incorrectly.
53
- **Proof:** The specific evidence that confirms this beyond doubt.
54
-
55
- ### Fix
56
- What needs to change and why. Include file:line references.
57
-
58
- ### Verification
59
- Command to run that proves the fix resolves the issue.
60
- Expected output after the fix.
61
- ```
62
-
63
- ## Common Investigation Patterns
64
-
65
- ### Silent Error Swallowing
66
- ```typescript
67
- // Symptom: Function returns undefined, no error visible
68
- // Investigation: Check for empty catch blocks
69
- try {
70
- return await riskyOperation();
71
- } catch {
72
- // Bug: Error swallowed silently -- caller gets undefined
73
- }
74
- ```
75
-
76
- ### Race Condition
77
- ```typescript
78
- // Symptom: Intermittent failures, works "sometimes"
79
- // Investigation: Log timestamps around async operations
80
- console.log("[DEBUG] before await:", Date.now());
81
- const result = await asyncOp();
82
- console.log("[DEBUG] after await:", Date.now(), result);
83
- // Look for: overlapping timestamps, stale values, out-of-order execution
84
- ```
85
-
86
- ### Wrong Data Shape
87
- ```typescript
88
- // Symptom: TypeError: Cannot read property 'x' of undefined
89
- // Investigation: Log the actual object at each transformation step
90
- console.log("[DEBUG] raw response:", JSON.stringify(response, null, 2));
91
- console.log("[DEBUG] after transform:", JSON.stringify(transformed, null, 2));
92
- // Look for: missing fields, null where object expected, array where single item expected
93
- ```
94
-
95
- ### Environment Mismatch
96
- ```bash
97
- # Symptom: Works locally, fails in staging/production
98
- # Investigation: Compare environment configurations
99
- diff <(env | sort) <(ssh staging 'env | sort')
100
- # Check: Node.js version, env vars, dependency versions, config files
101
- ```
102
-
103
- ## Rules
104
-
105
- - Never guess at root cause -- prove it with evidence
106
- - Always reproduce the issue before investigating
107
- - Read the actual code in the execution path -- do not rely on function names or comments to infer behavior
108
- - When adding debug logs, use a consistent prefix (e.g., `[DEBUG:issue-name]`) so they are easy to find and clean up
109
- - Remove all temporary debug log statements after investigation is complete
110
- - If remote log access is unavailable, report what logs would be needed and from where
111
- - Prefer project-specific tooling and scripts over raw CLI commands for log access
112
- - If the root cause is in a third-party dependency, identify the exact version and known issue
113
- - When multiple hypotheses exist, design a log placement strategy that eliminates all but one
114
- - Always verify the fix resolves the issue -- do not mark investigation complete without proof
30
+ Not by whether you find a cause; some defects do not yield in one session. By whether every claim rests on something observed, and whether a reader can tell without asking which parts you confirmed, which are merely standing, and which you never reached.
@@ -10,8 +10,8 @@ Follow this 8-step triage process before implementing any bug fix. Do not skip t
10
10
  ## Triage Steps
11
11
 
12
12
  1. Verify you have all information needed to reproduce the bug (authentication requirements, environment information, etc.). Do not make assumptions. If anything is missing, stop and ask before proceeding.
13
- 2. Reproduce the bug. If you cannot reproduce it, stop and report what you tried and what you observed.
14
- 3. Once reproduced, verify you are 100% positive on how to fix it. If not, determine what you need to do to be 100% positive (e.g. add logging, trace the code path, inspect state) and do that first.
13
+ 2. Reproduce the bug on the path the user actually hits. Prerequisites the real path needs — seeded data, auth state, flags — are part of the reproduction; scaffolding that substitutes real behaviour is not, and a failure that survives only with it in place is a lead rather than a reproduction. Say which you have. Without a real-path reproduction you may not claim a root cause at all. If you cannot reproduce it, stop and report what you tried and what you observed.
14
+ 3. Once reproduced, name a candidate cause and the observation that would disprove it, and go get that observation. Surviving the disproof is not proof — it leaves the candidate standing, not confirmed so before implementing, execute something whose output the candidate predicts and a different cause would not produce. If you cannot get that confirmation, record the verdict as inconclusive and say so rather than proceeding as if certain; add logging, trace the path, or bisect until you can.
15
15
  4. Verify you have access to the tools, environments, and permissions needed to deploy and verify this fix (e.g. CI/CD pipelines, deployment targets, logging/monitoring systems, API access, database access). If any are missing or inaccessible, stop and raise them before starting implementation.
16
16
  5. Define the tests you will write to confirm the fix and prevent a regression.
17
17
  6. Define the documentation you will create or update to explain this bug so another developer understands the "how" and "what" behind it.
@@ -1,96 +1,67 @@
1
1
  ---
2
2
  name: lisa-reproduce-bug
3
- description: "How to create reliable bug reproduction scenarios. Covers failing tests, minimal scripts, environment verification, and reproduction evidence capture."
3
+ description: "How to reproduce a bug reliably and on the real path — choosing a reproduction method from the symptom, distinguishing prerequisites from behaviour-substituting scaffolding, and reporting the observed failure rate instead of rounding it."
4
4
  ---
5
5
 
6
6
  # Reproduce Bug
7
7
 
8
- Before investigating root cause, reproduce the issue empirically. A bug that cannot be reproduced cannot be verified as fixed.
8
+ A bug that cannot be reproduced cannot be verified as fixed. Root cause analysis does not begin until a reliable reproduction exists.
9
9
 
10
- ## Reproduction Process
10
+ ## The reproduction must exercise the real path
11
11
 
12
- ### 1. Run the Failing Scenario
12
+ Left alone, an agent asked to demonstrate a defect will build the environment in which the defect appears — a stub, a fake, a fresh harness — and present that as the reproduction. It looks like proof and it is not: a fix that satisfies it may never touch the path the user is on.
13
13
 
14
- - Execute the exact command, test, or request that triggers the bug
15
- - Capture the complete error output, stack trace, or unexpected behavior
16
- - Record the exact command used so it can be repeated
14
+ Two kinds of setup get confused here, and only one is a problem:
17
15
 
18
- ### 2. Capture Evidence
16
+ - **Prerequisites** — state the real path genuinely needs: a seeded record, a logged-in user, a feature flag, a fixture recreating production-equivalent data. These are part of the reproduction. Removing them changes the precondition rather than testing anything, so do not remove them.
17
+ - **Replacement scaffolding** — anything standing in for behaviour the real path would perform: a mocked service, a stubbed function, a fake clock, a bespoke harness that bypasses the normal entry point. This is what makes a reproduction suspect.
19
18
 
20
- - Save the full error output (not just a summary)
21
- - Note the timestamp and environment details (OS, runtime version, dependency versions)
22
- - Screenshot or log any visual/UI issues
23
- - Record the actual behavior vs. the expected behavior
19
+ So the reproduction states, explicitly:
24
20
 
25
- ### 3. Investigate Environment Differences (If Cannot Reproduce)
21
+ - **The entry point the user actually hits** — the route, endpoint, command, or interaction.
22
+ - **Prerequisites**, listed as setup.
23
+ - **Replacement scaffolding**, each item with the real behaviour it substitutes.
24
+ - **Whether the failure survives with prerequisites in place and replacement scaffolding removed.** If it does not, this is a lead rather than a reproduction. Say so and keep going.
26
25
 
27
- If the issue does not reproduce locally:
26
+ If the real path is unreachable — no credentials, no environment, no data — that is a blocked reproduction. Report the missing access instead of substituting scaffolding and calling the bug reproduced.
28
27
 
29
- - Compare environment configurations (env vars, config files, feature flags)
30
- - Check runtime versions (Node.js, Python, Java, etc.)
31
- - Compare dependency versions (`package-lock.json`, `poetry.lock`, etc.)
32
- - Check data differences (database state, seed data, user roles)
33
- - Verify network conditions (DNS, proxies, firewalls, VPN)
34
- - Check for platform-specific behavior (OS, architecture, container vs. host)
28
+ ## Choose the method from the symptom
35
29
 
36
- ### 4. Create a Minimal Reproduction
30
+ | Symptom | Reach for |
31
+ | --- | --- |
32
+ | Wrong value, bad output, thrown error | Failing test at the narrowest layer that still crosses the real path |
33
+ | Broken interface or journey | Browser or device driver against a running build (Playwright, Maestro) |
34
+ | Service or API behaviour | Direct request to the running service — client script or `curl` |
35
+ | Depends on particular data | Seeded fixture recreating that state, recorded as a prerequisite |
36
+ | Intermittent or timing-shaped | Loop the trigger; capture timestamps around async boundaries |
37
+ | Works locally, fails deployed | Do not chase it locally — reproduce against the environment that fails |
37
38
 
38
- Create the smallest possible reproduction that triggers the bug:
39
+ **A failing test is the preferred form** wherever it can cross the real path: it runs in CI, it becomes the regression guard, and `codify-verification` expects it. A script is the fallback. Manual steps are the last resort and must carry their prerequisites.
39
40
 
40
- **Preferred: Failing test**
41
- - Write a test that exercises the exact code path and asserts the expected behavior
42
- - The test should fail with the same symptom as the reported bug
43
- - A failing test is the most reliable reproduction because it runs in CI and prevents regression
41
+ ## Report the failure rate, do not round it
44
42
 
45
- **Fallback: Reproduction script**
46
- - Write a standalone script that triggers the issue
47
- - Minimize dependencies -- remove anything not needed to reproduce
48
- - Include setup steps (data seeding, config) in the script itself
49
- - The script should be runnable by anyone with access to the repo
43
+ Run the reproduction enough times to state a rate. A reproduction that fails half the time cannot prove a fix — one passing run afterwards means nothing at that rate. If the rate is too low to distinguish a fix from luck, say what would raise it: more iterations, a forced schedule, a narrowed trigger, a seeded clock.
50
44
 
51
- **Last resort: Manual steps**
52
- - Document exact click-by-click or command-by-command steps
53
- - Include prerequisite state (logged-in user, specific data, feature flags)
54
- - Note any timing-sensitive aspects (race conditions, timeouts)
45
+ ## When it will not reproduce
55
46
 
56
- ### 5. Verify Reproduction Is Reliable
47
+ The difference is nearly always one of: runtime version, configuration or feature flags, data state, credentials and permissions, network posture, or platform. Diff the two environments along those axes rather than guessing between them, and report which axes you compared and what you found. That comparison is the finding when the bug stays hidden.
57
48
 
58
- - Run the reproduction multiple times to confirm it consistently fails
59
- - For intermittent bugs, run enough iterations to establish the failure rate
60
- - If intermittent, note any patterns (timing, load, specific data)
61
-
62
- ## Output Format
49
+ ## Output
63
50
 
64
51
  ```text
65
52
  ## Reproduction
66
53
 
67
- ### Command/Steps
68
- The exact command or steps to trigger the bug.
69
-
70
- ### Actual Behavior
71
- What happens (error message, wrong output, crash).
72
-
73
- ### Expected Behavior
74
- What should happen instead.
75
-
76
- ### Environment
77
- - Runtime: [version]
78
- - OS: [platform]
79
- - Dependencies: [relevant versions]
80
-
81
- ### Reproduction Type
82
- [ ] Failing test: [path to test file]
83
- [ ] Script: [path to script]
84
- [ ] Manual steps: [documented above]
85
-
86
- ### Reliability
87
- [Always / Intermittent (N/M runs) / Conditional (only when X)]
54
+ **Entry point:** the route, command, or action the user hits
55
+ **Command or steps:** exactly what to run
56
+ **Actual:** what happens · **Expected:** what should happen
57
+ **Prerequisites:** seeded data, auth state, flags the real path needs — or "none"
58
+ **Replacement scaffolding:** each mock, stub, fake, or bespoke harness and the
59
+ real behaviour it substitutes — or "none"
60
+ **Survives without replacement scaffolding:** yes / no — if no, this is a lead,
61
+ not a reproduction
62
+ **Observed failure rate:** n failures in m runs
63
+ **Form:** failing test `path` | script `path` | manual steps above
64
+ **Environment:** runtime, platform, relevant dependency versions
88
65
  ```
89
66
 
90
- ## Rules
91
-
92
- - Never skip reproduction. If you cannot reproduce, report what you tried and what you observed.
93
- - A failing test is always the preferred reproduction method.
94
- - Capture complete error output -- do not truncate or summarize.
95
- - If the bug is environment-specific, document exactly which environment triggers it.
96
- - Do not begin root cause analysis until you have a reliable reproduction.
67
+ Capture output whole — a truncated stack trace loses the line that mattered. But evidence carries whatever the system was holding, so **redact secrets, tokens, credentials, and personal data before a reproduction is handed on or attached to a work item**, and keep any unredacted capture only where the data class it contains is already permitted to live.