@amrhas82/agentic-kit 1.5.0 → 1.7.5

package/packages/opencode/command/systematic-debugging.md

@@ -1,19 +1,32 @@
 ---
-description: Systematic four-phase debugging framework - investigate root cause before any fixes
-argument-hint: <bug-or-error-description>
+id: systematic-debugging
+name: systematic-debugging
+description: Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes - four-phase framework (root cause investigation, pattern analysis, hypothesis testing, implementation) that ensures understanding before attempting solutions
+usage: /systematic-debugging <bug-or-error-description>
+auto_trigger: false
 ---
 
+# Systematic Debugging
+
+## Overview
+
+Random fixes waste time and create new bugs. Quick patches mask underlying issues.
+
 **Core principle:** ALWAYS find root cause before attempting fixes. Symptom fixes are failure.
 
+**Violating the letter of this process is violating the spirit of debugging.**
+
 ## The Iron Law
 
 ```
 NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST
 ```
 
+If you haven't completed Phase 1, you cannot propose fixes.
+
 ## When to Use
 
-**Use for ANY technical issue:**
+Use for ANY technical issue:
 - Test failures
 - Bugs in production
 - Unexpected behavior
@@ -21,16 +34,21 @@ NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST
 - Build failures
 - Integration issues
 
-**Use ESPECIALLY when:**
+**Use this ESPECIALLY when:**
 - Under time pressure (emergencies make guessing tempting)
 - "Just one quick fix" seems obvious
 - You've already tried multiple fixes
 - Previous fix didn't work
 - You don't fully understand the issue
 
+**Don't skip when:**
+- Issue seems simple (simple bugs have root causes too)
+- You're in a hurry (rushing guarantees rework)
+- Manager wants it fixed NOW (systematic is faster than thrashing)
+
 ## The Four Phases
 
-Complete each phase before proceeding to the next.
+You MUST complete each phase before proceeding to the next.
 
 ### Phase 1: Root Cause Investigation
 
@@ -56,160 +74,225 @@ Complete each phase before proceeding to the next.
 
 4. **Gather Evidence in Multi-Component Systems**
 
-For systems with multiple components (CI → build → signing, API → service → database):
+   **WHEN system has multiple components (CI → build → signing, API → service → database):**
 
-**Add diagnostic instrumentation at EACH component boundary:**
-```bash
-# Layer 1: Input validation
-echo "=== Input data received: ==="
-echo "IDENTITY: ${IDENTITY:+SET}${IDENTITY:-UNSET}"
+   **BEFORE proposing fixes, add diagnostic instrumentation:**
+   ```
+   For EACH component boundary:
+     - Log what data enters component
+     - Log what data exits component
+     - Verify environment/config propagation
+     - Check state at each layer
 
-# Layer 2: Processing
-echo "=== Env vars in process: ==="
-env | grep IDENTITY || echo "IDENTITY not found"
+   Run once to gather evidence showing WHERE it breaks
+   THEN analyze evidence to identify failing component
+   THEN investigate that specific component
+   ```
 
-# Layer 3: Output
-echo "=== Processing result: ==="
-echo "Keychain state: $(security list-keychains)"
-```
+   **Example (multi-layer system):**
+   ```bash
+   # Layer 1: Workflow
+   echo "=== Secrets available in workflow: ==="
+   echo "IDENTITY: ${IDENTITY:+SET}${IDENTITY:-UNSET}"
+
+   # Layer 2: Build script
+   echo "=== Env vars in build script: ==="
+   env | grep IDENTITY || echo "IDENTITY not in environment"
+
+   # Layer 3: Signing script
+   echo "=== Keychain state: ==="
+   security list-keychains
+   security find-identity -v
+
+   # Layer 4: Actual signing
+   codesign --sign "$IDENTITY" --verbose=4 "$APP"
+   ```
 
-**Run once to gather evidence showing WHERE it breaks, THEN analyze evidence to identify failing component**
+   **This reveals:** Which layer fails (secrets → workflow ✓, workflow → build ✗)
+
+5. **Trace Data Flow**
+
+   **WHEN error is deep in call stack:**
+
+   **REQUIRED SUB-SKILL:** Use root-cause-tracing for backward tracing technique
+
+   **Quick version:**
+   - Where does bad value originate?
+   - What called this with bad value?
+   - Keep tracing up until you find the source
+   - Fix at source, not at symptom
 
 ### Phase 2: Pattern Analysis
 
-**After gathering evidence:**
+**Find the pattern before fixing:**
 
-1. **Look for Patterns**
-   - Does it fail the same way every time?
-   - Are there common conditions?
-   - Does it work in some environments but not others?
-   - Is there timing involved?
+1. **Find Working Examples**
+   - Locate similar working code in same codebase
+   - What works that's similar to what's broken?
 
-2. **Map the Failure Chain**
-   - What happens just before failure?
-   - What data/state exists at each step?
-   - Where does the expected flow break?
+2. **Compare Against References**
+   - If implementing pattern, read reference implementation COMPLETELY
+   - Don't skim - read every line
+   - Understand the pattern fully before applying
 
-3. **Identify Contradictions**
-   - What should happen vs. what actually happens?
-   - Where do assumptions break down?
-   - What evidence contradicts initial hypotheses?
+3. **Identify Differences**
+   - What's different between working and broken?
+   - List every difference, however small
+   - Don't assume "that can't matter"
 
-### Phase 3: Hypothesis Testing
+4. **Understand Dependencies**
+   - What other components does this need?
+   - What settings, config, environment?
+   - What assumptions does it make?
 
-**Form and test hypotheses:**
+### Phase 3: Hypothesis and Testing
 
-1. **Formulate Specific Hypotheses**
-   - Not "it's probably a timing issue"
-   - But "the callback fires before the database connection is established"
+**Scientific method:**
 
-2. **Design Tests to Prove/Disprove**
-   - Add logging to verify assumptions
-   - Create minimal test cases
-   - Isolate variables
+1. **Form Single Hypothesis**
+   - State clearly: "I think X is the root cause because Y"
+   - Write it down
+   - Be specific, not vague
 
-3. **Test Each Hypothesis**
+2. **Test Minimally**
+   - Make the SMALLEST possible change to test hypothesis
    - One variable at a time
-   - Document results clearly
-   - Eliminate possibilities systematically
-
-**Example:**
-```typescript
-// Hypothesis: Race condition between async operations
-// Test: Add console.log timestamps to verify order
-console.log('1. Starting operation A');
-operationA().then(() => {
-  console.log('2. Operation A completed');
-  operationB();
-});
-
-console.log('3. Operation B started');
-operationB().then(() => {
-  console.log('4. Operation B completed');
-});
-```
+   - Don't fix multiple things at once
 
-### Phase 4: Implementation
-
-**Only after completing phases 1-3:**
-
-1. **Design Fix Based on Root Cause**
-   - Address the actual problem, not symptoms
-   - Consider side effects and interactions
-   - Plan for failure cases
-
-2. **Implement Minimal Fix**
-   - Change only what's necessary
-   - Don't over-engineer
-   - Test thoroughly
-
-3. **Verify Fix Works**
-   - Reproduce original failure
-   - Confirm fix resolves it
-   - Test edge cases
-
-## Common Debugging Mistakes
-
-**❌ Skipping Phase 1:**
-- "Let me try this quick fix"
-- Result: Hidden bugs, regression issues
-
-**❌ Guessing at root cause:**
-- "It's probably a timing issue"
-- Result: Wrong fixes, wasted time
-
-**❌ Fixing symptoms:**
-- "If I just add a delay here..."
-- Result: Fragile solutions, new bugs
-
-**✅ Following systematic approach:**
-- Evidence gathering first
-- Hypothesis testing
-- Minimal fix based on understanding
-
-## Debugging Tools
-
-**Essential Commands:**
-```bash
-# Git investigation
-git log --oneline -10
-git diff HEAD~1..HEAD
-git blame <file>
-
-# Environment debugging
-env | sort
-echo $PATH
-which <command>
-
-# Process debugging
-ps aux | grep <process>
-lsof -p <pid>
-strace -p <pid>  # Linux
-dtruss -p <pid>  # macOS
-
-# Network debugging
-curl -v <url>
-netstat -an | grep <port>
-ss -tulpn | grep <port>
-
-# Log analysis
-tail -f <logfile>
-grep -n "ERROR" <logfile>
-awk '/ERROR/ {print NR, $0}' <logfile>
-```
+3. **Verify Before Continuing**
+   - Did it work? Yes → Phase 4
+   - Didn't work? Form NEW hypothesis
+   - DON'T add more fixes on top
 
-## Usage Examples
+4. **When You Don't Know**
+   - Say "I don't understand X"
+   - Don't pretend to know
+   - Ask for help
+   - Research more
 
-**Complex production bug:**
-`/systematic-debugging "API returns 500 errors intermittently"`
+### Phase 4: Implementation
 
-**Failing test suite:**
-`/systematic-debugging "integration tests timing out in CI"`
+**Fix the root cause, not the symptom:**
+
+1. **Create Failing Test Case**
+   - Simplest possible reproduction
+   - Automated test if possible
+   - One-off test script if no framework
+   - MUST have before fixing
+   - **REQUIRED SUB-SKILL:** Use test-driven-development for writing proper failing tests
 
-**Build failures:**
-`/systematic-debugging "npm install fails with dependency conflicts"`
+2. **Implement Single Fix**
+   - Address the root cause identified
+   - ONE change at a time
+   - No "while I'm here" improvements
+   - No bundled refactoring
+
+3. **Verify Fix**
+   - Test passes now?
+   - No other tests broken?
+   - Issue actually resolved?
 
-**Performance issue:**
-`/systematic-debugging "database queries taking 30+ seconds"`
+4. **If Fix Doesn't Work**
+   - STOP
+   - Count: How many fixes have you tried?
+   - If < 3: Return to Phase 1, re-analyze with new information
+   - **If ≥ 3: STOP and question the architecture (step 5 below)**
+   - DON'T attempt Fix #4 without architectural discussion
+
+5. **If 3+ Fixes Failed: Question Architecture**
+
+   **Pattern indicating architectural problem:**
+   - Each fix reveals new shared state/coupling/problem in different place
+   - Fixes require "massive refactoring" to implement
+   - Each fix creates new symptoms elsewhere
+
+   **STOP and question fundamentals:**
+   - Is this pattern fundamentally sound?
+   - Are we "sticking with it through sheer inertia"?
+   - Should we refactor architecture vs. continue fixing symptoms?
+
+   **Discuss with your human partner before attempting more fixes**
+
+   This is NOT a failed hypothesis - this is a wrong architecture.
+
+## Red Flags - STOP and Follow Process
+
+If you catch yourself thinking:
+- "Quick fix for now, investigate later"
+- "Just try changing X and see if it works"
+- "Add multiple changes, run tests"
+- "Skip the test, I'll manually verify"
+- "It's probably X, let me fix that"
+- "I don't fully understand but this might work"
+- "Pattern says X but I'll adapt it differently"
+- "Here are the main problems: [lists fixes without investigation]"
+- Proposing solutions before tracing data flow
+- **"One more fix attempt" (when already tried 2+)**
+- **Each fix reveals new problem in different place**
+
+**ALL of these mean: STOP. Return to Phase 1.**
+
+**If 3+ fixes failed:** Question the architecture (see Phase 4.5)
+
+## your human partner's Signals You're Doing It Wrong
+
+**Watch for these redirections:**
+- "Is that not happening?" - You assumed without verifying
+- "Will it show us...?" - You should have added evidence gathering
+- "Stop guessing" - You're proposing fixes without understanding
+- "Ultrathink this" - Question fundamentals, not just symptoms
+- "We're stuck?" (frustrated) - Your approach isn't working
+
+**When you see these:** STOP. Return to Phase 1.
+
+## Common Rationalizations
+
+| Excuse | Reality |
+|--------|---------|
+| "Issue is simple, don't need process" | Simple issues have root causes too. Process is fast for simple bugs. |
+| "Emergency, no time for process" | Systematic debugging is FASTER than guess-and-check thrashing. |
+| "Just try this first, then investigate" | First fix sets the pattern. Do it right from the start. |
+| "I'll write test after confirming fix works" | Untested fixes don't stick. Test first proves it. |
+| "Multiple fixes at once saves time" | Can't isolate what worked. Causes new bugs. |
+| "Reference too long, I'll adapt the pattern" | Partial understanding guarantees bugs. Read it completely. |
+| "I see the problem, let me fix it" | Seeing symptoms ≠ understanding root cause. |
+| "One more fix attempt" (after 2+ failures) | 3+ failures = architectural problem. Question pattern, don't fix again. |
+
+## Quick Reference
+
+| Phase | Key Activities | Success Criteria |
+|-------|---------------|------------------|
+| **1. Root Cause** | Read errors, reproduce, check changes, gather evidence | Understand WHAT and WHY |
+| **2. Pattern** | Find working examples, compare | Identify differences |
+| **3. Hypothesis** | Form theory, test minimally | Confirmed or new hypothesis |
+| **4. Implementation** | Create test, fix, verify | Bug resolved, tests pass |
+
+## When Process Reveals "No Root Cause"
+
+If systematic investigation reveals issue is truly environmental, timing-dependent, or external:
+
+1. You've completed the process
+2. Document what you investigated
+3. Implement appropriate handling (retry, timeout, error message)
+4. Add monitoring/logging for future investigation
+
+**But:** 95% of "no root cause" cases are incomplete investigation.
+
+## Integration with Other Skills
+
+**This skill requires using:**
+- **root-cause-tracing** - REQUIRED when error is deep in call stack (see Phase 1, Step 5)
+- **test-driven-development** - REQUIRED for creating failing test case (see Phase 4, Step 1)
+
+**Complementary skills:**
+- **defense-in-depth** - Add validation at multiple layers after finding root cause
+- **condition-based-waiting** - Replace arbitrary timeouts identified in Phase 2
+- **verification-before-completion** - Verify fix worked before claiming success
+
+## Real-World Impact
 
-Remember: The time spent on systematic debugging saves 10x the time in rework and maintenance.
+From debugging sessions:
+- Systematic approach: 15-30 minutes to fix
+- Random fixes approach: 2-3 hours of thrashing
+- First-time fix rate: 95% vs 40%
+- New bugs introduced: Near zero vs common