@kata-sh/cli 0.1.0 → 0.1.1

package/src/resources/skills/debug-like-expert/references/hypothesis-testing.md

@@ -0,0 +1,373 @@
+
+<overview>
+Debugging is applied scientific method. You observe a phenomenon (the bug), form hypotheses about its cause, design experiments to test those hypotheses, and revise based on evidence. This isn't metaphorical - it's literal experimental science.
+</overview>
+
+
+<principle name="falsifiability">
+A good hypothesis can be proven wrong. If you can't design an experiment that could disprove it, it's not a useful hypothesis.
+
+**Bad hypotheses** (unfalsifiable):
+- "Something is wrong with the state"
+- "The timing is off"
+- "There's a race condition somewhere"
+- "The library is buggy"
+
+**Good hypotheses** (falsifiable):
+- "The user state is being reset because the component remounts when the route changes"
+- "The API call completes after the component unmounts, causing the state update on unmounted component warning"
+- "Two async operations are modifying the same array without locking, causing data loss"
+- "The library's caching mechanism is returning stale data because our cache key doesn't include the timestamp"
+
+**The difference**: Specificity. Good hypotheses make specific, testable claims.
+</principle>
+
+<how_to_form>
+**Process for forming hypotheses**:
+
+1. **Observe the behavior precisely**
+   - Not "it's broken"
+   - But "the counter shows 3 when clicking once, should show 1"
+
+2. **Ask "What could cause this?"**
+   - List every possible cause you can think of
+   - Don't judge them yet, just brainstorm
+
+3. **Make each hypothesis specific**
+   - Not "state is wrong"
+   - But "state is being updated twice because handleClick is called twice"
+
+4. **Identify what evidence would support/refute each**
+   - If hypothesis X is true, I should see Y
+   - If hypothesis X is false, I should see Z
+
+<example>
+**Observation**: Button click sometimes saves data, sometimes doesn't.
+
+**Vague hypothesis**: "The save isn't working reliably"
+❌ Unfalsifiable, not specific
+
+**Specific hypotheses**:
+1. "The save API call is timing out when network is slow"
+   - Testable: Check network tab for timeout errors
+   - Falsifiable: If all requests complete successfully, this is wrong
+
+2. "The save button is being double-clicked, and the second request overwrites with stale data"
+   - Testable: Add logging to count clicks
+   - Falsifiable: If only one click is registered, this is wrong
+
+3. "The save is successful but the UI doesn't update because the response is being ignored"
+   - Testable: Check if API returns success
+   - Falsifiable: If UI updates on successful response, this is wrong
+</example>
+</how_to_form>
+
+
+<experimental_design>
+An experiment is a test that produces evidence supporting or refuting a hypothesis.
+
+**Good experiments**:
+- Test one hypothesis at a time
+- Have clear success/failure criteria
+- Produce unambiguous results
+- Are repeatable
+
+**Bad experiments**:
+- Test multiple things at once
+- Have unclear outcomes ("maybe it works better?")
+- Rely on subjective judgment
+- Can't be reproduced
+
+<framework>
+For each hypothesis, design an experiment:
+
+**1. Prediction**: If hypothesis H is true, then I will observe X
+**2. Test setup**: What do I need to do to test this?
+**3. Measurement**: What exactly am I measuring?
+**4. Success criteria**: What result confirms H? What result refutes H?
+**5. Run the experiment**: Execute the test
+**6. Observe the result**: Record what actually happened
+**7. Conclude**: Does this support or refute H?
+
+</framework>
+
+<example>
+**Hypothesis**: "The component is re-rendering excessively because the parent is passing a new object reference on every render"
+
+**1. Prediction**: If true, the component will re-render even when the object's values haven't changed
+
+**2. Test setup**:
+   - Add console.log in component body to count renders
+   - Add console.log in parent to track when object is created
+   - Add useEffect with the object as dependency to log when it changes
+
+**3. Measurement**: Count of renders and object creations
+
+**4. Success criteria**:
+   - Confirms H: Component re-renders match parent renders, object reference changes each time
+   - Refutes H: Component only re-renders when object values actually change
+
+**5. Run**: Execute the code with logging
+
+**6. Observe**:
+   ```
+   [Parent] Created user object
+   [Child] Rendering (1)
+   [Parent] Created user object
+   [Child] Rendering (2)
+   [Parent] Created user object
+   [Child] Rendering (3)
+   ```
+
+**7. Conclude**: CONFIRMED. New object every parent render → child re-renders
+</example>
+</experimental_design>
+
+
+<evidence_quality>
+Not all evidence is equal. Learn to distinguish strong from weak evidence.
+
+**Strong evidence**:
+- Directly observable ("I can see in the logs that X happens")
+- Repeatable ("This fails every time I do Y")
+- Unambiguous ("The value is definitely null, not undefined")
+- Independent ("This happens even in a fresh browser with no cache")
+
+**Weak evidence**:
+- Hearsay ("I think I saw this fail once")
+- Non-repeatable ("It failed that one time but I can't reproduce it")
+- Ambiguous ("Something seems off")
+- Confounded ("It works after I restarted the server and cleared the cache and updated the package")
+
+<examples>
+**Strong**:
+```javascript
+console.log('User ID:', userId); // Output: User ID: undefined
+console.log('Type:', typeof userId); // Output: Type: undefined
+```
+✅ Direct observation, unambiguous
+
+**Weak**:
+"I think the user ID might not be set correctly sometimes"
+❌ Vague, not verified, uncertain
+
+**Strong**:
+```javascript
+for (let i = 0; i < 100; i++) {
+  const result = processData(testData);
+  if (result !== expected) {
+    console.log('Failed on iteration', i);
+  }
+}
+// Output: Failed on iterations: 3, 7, 12, 23, 31...
+```
+✅ Repeatable, shows pattern
+
+**Weak**:
+"It usually works, but sometimes fails"
+❌ Not quantified, no pattern identified
+</examples>
+</evidence_quality>
+
+
+<decision_point>
+Don't act too early (premature fix) or too late (analysis paralysis).
+
+**Act when you can answer YES to all**:
+
+1. **Do you understand the mechanism?**
+   - Not just "what fails" but "why it fails"
+   - Can you explain the chain of events that produces the bug?
+
+2. **Can you reproduce it reliably?**
+   - Either always reproduces, or you understand the conditions that trigger it
+   - If you can't reproduce, you don't understand it yet
+
+3. **Do you have evidence, not just theory?**
+   - You've observed the behavior directly
+   - You've logged the values, traced the execution
+   - You're not guessing
+
+4. **Have you ruled out alternatives?**
+   - You've considered other hypotheses
+   - Evidence contradicts the alternatives
+   - This is the most likely cause, not just the first idea
+
+**Don't act if**:
+- "I think it might be X" - Too uncertain
+- "This could be the issue" - Not confident enough
+- "Let me try changing Y and see" - Random changes, not hypothesis-driven
+- "I'll fix it and if it works, great" - Outcome-based, not understanding-based
+
+<example>
+**Too early** (don't act):
+- Hypothesis: "Maybe the API is slow"
+- Evidence: None, just a guess
+- Action: Add caching
+- Result: Bug persists, now you have caching to debug too
+
+**Right time** (act):
+- Hypothesis: "API response is missing the 'status' field when user is inactive, causing the app to crash"
+- Evidence:
+  - Logged API response for active user: has 'status' field
+  - Logged API response for inactive user: missing 'status' field
+  - Logged app behavior: crashes on accessing undefined status
+- Action: Add defensive check for missing status field
+- Result: Bug fixed because you understood the cause
+</example>
+</decision_point>
+
+
+<recovery>
+You will be wrong sometimes. This is normal. The skill is recovering gracefully.
+
+**When your hypothesis is disproven**:
+
+1. **Acknowledge it explicitly**
+   - "This hypothesis was wrong because [evidence]"
+   - Don't gloss over it or rationalize
+   - Intellectual honesty with yourself
+
+2. **Extract the learning**
+   - What did this experiment teach you?
+   - What did you rule out?
+   - What new information do you have?
+
+3. **Revise your understanding**
+   - Update your mental model
+   - What does the evidence actually suggest?
+
+4. **Form new hypotheses**
+   - Based on what you now know
+   - Avoid just moving to "second-guess" - use the evidence
+
+5. **Don't get attached to hypotheses**
+   - You're not your ideas
+   - Being wrong quickly is better than being wrong slowly
+
+<example>
+**Initial hypothesis**: "The memory leak is caused by event listeners not being cleaned up"
+
+**Experiment**: Check Chrome DevTools for listener counts
+**Result**: Listener count stays stable, doesn't grow over time
+
+**Recovery**:
+1. ✅ "Event listeners are NOT the cause. The count doesn't increase."
+2. ✅ "I've ruled out event listeners as the culprit"
+3. ✅ "But the memory profile shows objects accumulating. What objects? Let me check the heap snapshot..."
+4. ✅ "New hypothesis: Large arrays are being cached and never released. Let me test by checking the heap for array sizes..."
+
+This is good debugging. Wrong hypothesis, quick recovery, better understanding.
+</example>
+</recovery>
+
+
+<multiple_hypotheses>
+Don't fall in love with your first hypothesis. Generate multiple alternatives.
+
+**Strategy**: "Strong inference" - Design experiments that differentiate between competing hypotheses.
+
+<example>
+**Problem**: Form submission fails intermittently
+
+**Competing hypotheses**:
+1. Network timeout
+2. Validation failure
+3. Race condition with auto-save
+4. Server-side rate limiting
+
+**Design experiment that differentiates**:
+
+Add logging at each stage:
+```javascript
+try {
+  console.log('[1] Starting validation');
+  const validation = await validate(formData);
+  console.log('[1] Validation passed:', validation);
+
+  console.log('[2] Starting submission');
+  const response = await api.submit(formData);
+  console.log('[2] Response received:', response.status);
+
+  console.log('[3] Updating UI');
+  updateUI(response);
+  console.log('[3] Complete');
+} catch (error) {
+  console.log('[ERROR] Failed at stage:', error);
+}
+```
+
+**Observe results**:
+- Fails at [2] with timeout error → Hypothesis 1
+- Fails at [1] with validation error → Hypothesis 2
+- Succeeds but [3] has wrong data → Hypothesis 3
+- Fails at [2] with 429 status → Hypothesis 4
+
+**One experiment, differentiates between four hypotheses.**
+</example>
+</multiple_hypotheses>
+
+
+<workflow>
+```
+1. Observe unexpected behavior
+     ↓
+2. Form specific hypotheses (plural)
+     ↓
+3. For each hypothesis: What would prove/disprove?
+     ↓
+4. Design experiment to test
+     ↓
+5. Run experiment
+     ↓
+6. Observe results
+     ↓
+7. Evaluate: Confirmed, refuted, or inconclusive?
+     ↓
+8a. If CONFIRMED → Design fix based on understanding
+8b. If REFUTED → Return to step 2 with new hypotheses
+8c. If INCONCLUSIVE → Redesign experiment or gather more data
+```
+
+**Key insight**: This is a loop, not a line. You'll cycle through multiple times. That's expected.
+</workflow>
+
+
+<pitfalls>
+
+**Pitfall: Testing multiple hypotheses at once**
+- You change three things and it works
+- Which one fixed it? You don't know
+- Solution: Test one hypothesis at a time
+
+**Pitfall: Confirmation bias in experiments**
+- You only look for evidence that confirms your hypothesis
+- You ignore evidence that contradicts it
+- Solution: Actively seek disconfirming evidence
+
+**Pitfall: Acting on weak evidence**
+- "It seems like maybe this could be..."
+- Solution: Wait for strong, unambiguous evidence
+
+**Pitfall: Not documenting results**
+- You forget what you tested
+- You repeat the same experiments
+- Solution: Write down each hypothesis and its result
+
+**Pitfall: Giving up on the scientific method**
+- Under pressure, you start making random changes
+- "Let me just try this..."
+- Solution: Double down on rigor when pressure increases
+</pitfalls>
+
+<excellence>
+**Great debuggers**:
+- Form multiple competing hypotheses
+- Design clever experiments that differentiate between them
+- Follow the evidence wherever it leads
+- Revise their beliefs when proven wrong
+- Act only when they have strong evidence
+- Understand the mechanism, not just the symptom
+
+This is the difference between guessing and debugging.
+</excellence>

package/src/resources/skills/debug-like-expert/references/investigation-techniques.md

@@ -0,0 +1,337 @@
+
+<overview>
+These are systematic approaches to narrowing down bugs. Each technique is a tool in your debugging toolkit. The skill is knowing which tool to use when.
+</overview>
+
+
+<technique name="binary_search">
+**When to use**: Large codebase, long execution path, or many possible failure points.
+
+**How it works**: Cut the problem space in half repeatedly until you isolate the issue.
+
+**In practice**:
+
+1. **Identify the boundaries**: Where does it work? Where does it fail?
+2. **Find the midpoint**: Add logging/testing at the middle of the execution path
+3. **Determine which half**: Does the bug occur before or after the midpoint?
+4. **Repeat**: Cut that half in half, test again
+5. **Converge**: Keep halving until you find the exact line
+
+<example>
+Problem: API request returns wrong data
+
+1. Test: Does the data leave the database correctly? YES
+2. Test: Does the data reach the frontend correctly? NO
+3. Test: Does the data leave the API route correctly? YES
+4. Test: Does the data survive serialization? NO
+5. **Found it**: Bug is in the serialization layer
+
+You just eliminated 90% of the code in 4 tests.
+</example>
+</technique>
+
+<technique name="comment_out_bisection">
+**Variant**: Commenting out code to find the breaking change.
+
+1. Comment out the second half of a function
+2. Does it work now? The bug is in the commented section
+3. Uncomment half of that, repeat
+4. Converge on the problematic lines
+
+**Warning**: Only works for code you can safely comment out. Don't use for initialization code.
+</technique>
+
+
+<technique name="rubber_duck">
+**When to use**: You're stuck, confused, or your mental model doesn't match reality.
+
+**How it works**: Explain the problem out loud (to a rubber duck, a colleague, or in writing) in complete detail.
+
+**Why it works**: Articulating forces you to:
+- Make assumptions explicit
+- Notice gaps in your understanding
+- Hear how convoluted your explanation sounds
+- Realize what you haven't actually verified
+
+**In practice**:
+
+Write or say out loud:
+1. "The system should do X"
+2. "Instead it does Y"
+3. "I think this is because Z"
+4. "The code path is: A → B → C → D"
+5. "I've verified that..." (List what you've actually tested)
+6. "I'm assuming that..." (List assumptions)
+
+Often you'll spot the bug mid-explanation: "Wait, I never actually verified that B returns what I think it does."
+
+<example>
+"So when the user clicks the button, it calls handleClick, which dispatches an action, which... wait, does the reducer actually handle this action type? Let me check... Oh. The reducer is looking for 'UPDATE_USER' but I'm dispatching 'USER_UPDATE'."
+</example>
+</technique>
+
+
+<technique name="minimal_reproduction">
+**When to use**: Complex system, many moving parts, unclear which part is failing.
+
+**How it works**: Strip away everything until you have the smallest possible code that reproduces the bug.
+
+**Why it works**:
+- Removes distractions
+- Isolates the actual issue
+- Often reveals the bug during the stripping process
+- Makes it easier to reason about
+
+**Process**:
+
+1. **Copy the failing code to a new file**
+2. **Remove one piece** (a dependency, a function, a feature)
+3. **Test**: Does it still reproduce?
+   - YES: Keep it removed, continue
+   - NO: Put it back, it's needed
+4. **Repeat** until you have the bare minimum
+5. **The bug is now obvious** in the stripped-down code
+
+<example>
+Start with: 500-line React component with 15 props, 8 hooks, 3 contexts
+
+End with:
+```jsx
+function MinimalRepro() {
+  const [count, setCount] = useState(0);
+
+  useEffect(() => {
+    setCount(count + 1); // Bug: infinite loop, missing dependency array
+  });
+
+  return <div>{count}</div>;
+}
+```
+
+The bug was hidden in complexity. Minimal reproduction made it obvious.
+</example>
+</technique>
+
+
+<technique name="working_backwards">
+**When to use**: You know what the correct output should be, but don't know why you're not getting it.
+
+**How it works**: Start from the desired end state and trace backwards through the execution path.
+
+**Process**:
+
+1. **Define the desired output precisely**
+2. **Ask**: What function produces this output?
+3. **Test that function**: Give it the input it should receive. Does it produce correct output?
+   - YES: The bug is earlier (wrong input to this function)
+   - NO: The bug is here
+4. **Repeat backwards** through the call stack
+5. **Find the divergence point**: Where does expected vs actual first differ?
+
+<example>
+Problem: UI shows "User not found" when user exists
+
+Trace backwards:
+1. UI displays: `user.error` → Is this the right value to display? YES
+2. Component receives: `user.error = "User not found"` → Is this correct? NO, should be null
+3. API returns: `{ error: "User not found" }` → Why?
+4. Database query: `SELECT * FROM users WHERE id = 'undefined'` → AH!
+5. **Found it**: The user ID is 'undefined' (string) instead of a number
+
+Working backwards revealed the bug was in how the ID was passed to the query.
+</example>
+</technique>
+
+
+<technique name="differential_debugging">
+**When to use**: Something used to work and now doesn't. A feature works in one environment but not another.
+
+**How it works**: Compare the working vs broken states to find what's different.
+
+**Questions to ask**:
+
+**Time-based** (it worked, now it doesn't):
+- What changed in the code since it worked?
+- What changed in the environment? (Node version, OS, dependencies)
+- What changed in the data? (Database schema, API responses)
+- What changed in the configuration?
+
+**Environment-based** (works in dev, fails in prod):
+- What's different between environments?
+- Configuration values
+- Environment variables
+- Network conditions
+- Data volume
+- Third-party service behavior
+
+**Process**:
+
+1. **Make a list of differences** between working and broken
+2. **Test each difference** in isolation
+3. **Find the difference that causes the failure**
+4. **That difference reveals the root cause**
+
+<example>
+Works locally, fails in CI:
+
+Differences:
+- Node version: Same ✓
+- Environment variables: Same ✓
+- Timezone: Different! ✗
+
+Test: Set local timezone to UTC (like CI)
+Result: Now fails locally too
+
+**Found it**: Date comparison logic assumes local timezone
+</example>
+</technique>
+
+
+<technique name="observability_first">
+**When to use**: Always. Before making any fix.
+
+**Why it matters**: You can't fix what you can't see. Add visibility before changing behavior.
+
+**Approaches**:
+
+**1. Strategic logging**
+```javascript
+// Not this (useless):
+console.log('in function');
+
+// This (useful):
+console.log('[handleSubmit] Input:', { email, password: '***' });
+console.log('[handleSubmit] Validation result:', validationResult);
+console.log('[handleSubmit] API response:', response);
+```
+
+**2. Assertion checks**
+```javascript
+function processUser(user) {
+  console.assert(user !== null, 'User is null!');
+  console.assert(user.id !== undefined, 'User ID is undefined!');
+  // ... rest of function
+}
+```
+
+**3. Timing measurements**
+```javascript
+console.time('Database query');
+const result = await db.query(sql);
+console.timeEnd('Database query');
+```
+
+**4. Stack traces at key points**
+```javascript
+console.log('[updateUser] Called from:', new Error().stack);
+```
+
+**The workflow**:
+1. **Add logging/instrumentation** at suspected points
+2. **Run the code**
+3. **Observe the output**
+4. **Form hypothesis** based on what you see
+5. **Only then** make changes
+
+Don't code in the dark. Light up the execution path first.
+</technique>
+
+
+<technique name="comment_out_everything">
+**When to use**: Many possible interactions, unclear which code is causing the issue.
+
+**How it works**:
+
+1. **Comment out everything** in a function/file
+2. **Verify the bug is gone**
+3. **Uncomment one piece at a time**
+4. **After each uncomment, test**
+5. **When the bug returns**, you found the culprit
+
+**Variant**: For config files, reset to defaults and add back one setting at a time.
+
+<example>
+Problem: Some middleware breaks requests, but you have 8 middleware functions.
+
+```javascript
+app.use(helmet()); // Uncomment, test → works
+app.use(cors()); // Uncomment, test → works
+app.use(compression()); // Uncomment, test → works
+app.use(bodyParser.json({ limit: '50mb' })); // Uncomment, test → BREAKS
+
+// Found it: Body size limit too high causes memory issues
+```
+</example>
+</technique>
+
+
+<technique name="git_bisect">
+**When to use**: Feature worked in the past, broke at some unknown commit.
+
+**How it works**: Binary search through git history to find the breaking commit.
+
+**Process**:
+
+```bash
+git bisect start
+
+git bisect bad
+
+git bisect good abc123
+
+git bisect bad
+
+git bisect good
+
+```
+
+**Why it's powerful**: Turns "it broke sometime in the last 100 commits" into "it broke in commit abc123" in ~7 tests (log₂ 100 ≈ 7).
+
+<example>
+100 commits between working and broken
+Manual testing: 100 commits to check
+Git bisect: 7 commits to check
+
+Time saved: Massive
+</example>
+</technique>
+
+
+<decision_tree>
+**Large codebase, many files**:
+→ Binary search / Divide and conquer
+
+**Confused about what's happening**:
+→ Rubber duck debugging
+→ Observability first (add logging)
+
+**Complex system with many interactions**:
+→ Minimal reproduction
+
+**Know the desired output**:
+→ Working backwards
+
+**Used to work, now doesn't**:
+→ Differential debugging
+→ Git bisect
+
+**Many possible causes**:
+→ Comment out everything
+→ Binary search
+
+**Always**:
+→ Observability first before making changes
+</decision_tree>
+
+<combining_techniques>
+Often you'll use multiple techniques together:
+
+1. **Differential debugging** to identify what changed
+2. **Binary search** to narrow down where in the code
+3. **Observability first** to add logging at that point
+4. **Rubber duck** to articulate what you're seeing
+5. **Minimal reproduction** to isolate just that behavior
+6. **Working backwards** to find the root cause
+
+Techniques compose. Use as many as needed.
+</combining_techniques>