@hienlh/ppm 0.9.80 → 0.9.81

package/.opencode/skills/ck-debug/references/investigation-methodology.md

@@ -0,0 +1,101 @@
+# Investigation Methodology
+
+Five-step structured investigation for system-level issues, incidents, and multi-component failures.
+
+## When to Use
+
+- Server returning 500 errors or unexpected responses
+- System behavior changed without obvious code changes
+- Multi-component failures spanning services/databases/infrastructure
+- Need to understand "what happened" before fixing
+
+## Step 1: Initial Assessment
+
+**Gather scope and impact before diving in.**
+
+1. **Collect symptoms** - Error messages, affected endpoints, user reports
+2. **Identify affected components** - Which services, databases, queues involved?
+3. **Determine timeframe** - When did issue start? Correlate with deployments/changes
+4. **Assess severity** - Users affected? Data at risk? Revenue impact?
+5. **Check recent changes** - Git log, deployment history, config changes, dependency updates
+
+```bash
+# Recent deployments
+gh run list --limit 10
+# Recent commits
+git log --oneline -20 --since="2 days ago"
+# Config changes
+git diff HEAD~5 -- '*.env*' '*.config*' '*.yml' '*.yaml' '*.json'
+```
+
+## Step 2: Data Collection
+
+**Gather evidence systematically before analysis.**
+
+1. **Server/application logs** - Filter by timeframe and affected components
+2. **CI/CD pipeline logs** - Use `gh run view <run-id> --log-failed` for GitHub Actions
+3. **Database state** - Query relevant tables, check recent migrations
+4. **System metrics** - CPU, memory, disk, network utilization
+5. **External dependencies** - Third-party API status, DNS, CDN
+
+```bash
+# GitHub Actions: list recent workflow runs
+gh run list --workflow=<workflow> --limit 5
+# View failed run logs
+gh run view <run-id> --log-failed
+# Download full logs
+gh run view <run-id> --log > /tmp/ci-logs.txt
+```
+
+**For codebase understanding:**
+- Read `docs/codebase-summary.md` if exists and up-to-date (<2 days old)
+- Otherwise use `ck:repomix` to generate fresh codebase summary
+- Use `/ck:scout` or `/ck:scout ext` to find relevant files
+- Use `ck:docs-seeker` skill for package/plugin documentation
+
+## Step 3: Analysis Process
+
+**Correlate evidence across sources.**
+
+1. **Timeline reconstruction** - Order events chronologically across all log sources
+2. **Pattern identification** - Recurring errors, timing patterns, affected user segments
+3. **Execution path tracing** - Follow request flow through system components
+4. **Database analysis** - Query performance, table relationships, data integrity
+5. **Dependency mapping** - Which components depend on the failing one?
+
+**Key questions:**
+- Does issue correlate with specific deployments or time windows?
+- Is it intermittent or consistent?
+- Does it affect all users or a subset?
+- Are there related errors in upstream/downstream services?
+
+## Step 4: Root Cause Identification
+
+**Systematic elimination with evidence.**
+
+1. **List hypotheses** ranked by evidence strength
+2. **Test each** - Design smallest experiment to confirm/eliminate
+3. **Validate with evidence** - Logs, metrics, reproduction steps
+4. **Consider environmental factors** - Race conditions, resource limits, config drift
+5. **Document the chain** - Full event sequence from trigger to symptom
+
+**Avoid:** Fixing first hypothesis without testing alternatives. Multiple plausible causes require elimination.
+
+## Step 5: Solution Development
+
+**Design targeted, evidence-backed fixes.**
+
+1. **Immediate fix** - Minimum change to restore service (hotfix, rollback, config change)
+2. **Root cause fix** - Address underlying issue permanently
+3. **Preventive measures** - Monitoring, alerting, validation to catch recurrence early
+4. **Verification plan** - How to confirm fix works in production
+
+**Prioritize:** Impact × urgency. Restore service first, then fix root cause, then prevent recurrence.
+
+## Integration with Code-Level Debugging
+
+When investigation narrows to specific code:
+- Switch to `systematic-debugging.md` for the code-level fix
+- Use `root-cause-tracing.md` if error is deep in call stack
+- Apply `defense-in-depth.md` after fixing
+- Always finish with `verification.md`

package/.opencode/skills/ck-debug/references/log-and-ci-analysis.md

@@ -0,0 +1,97 @@
+# Log & CI/CD Analysis
+
+Collect and analyze logs from servers, CI/CD pipelines, and application layers to diagnose failures.
+
+## GitHub Actions Analysis
+
+### List and Inspect Runs
+
+```bash
+# List recent runs (all workflows)
+gh run list --limit 10
+
+# List runs for specific workflow
+gh run list --workflow=ci.yml --limit 5
+
+# View specific run details
+gh run view <run-id>
+
+# View failed job logs only
+gh run view <run-id> --log-failed
+
+# Download complete logs
+gh run view <run-id> --log > /tmp/ci-full.txt
+
+# Re-run failed jobs
+gh run rerun <run-id> --failed
+```
+
+### Common CI/CD Failure Patterns
+
+| Pattern | Likely Cause | Investigation |
+|---------|-------------|---------------|
+| Passes locally, fails CI | Environment diff | Check Node/Python version, OS, env vars |
+| Intermittent failures | Race conditions, flaky tests | Run 3x, check timing, shared state |
+| Timeout failures | Resource limits, infinite loops | Check resource usage, add timeouts |
+| Permission errors | Token/secret misconfiguration | Verify `GITHUB_TOKEN`, secret names |
+| Dependency install fails | Registry issues, version conflicts | Check lockfile, registry status |
+| Build succeeds, tests fail | Test environment setup | Check test config, database setup, fixtures |
+
+### Analyzing Failed Steps
+
+1. **Identify which step failed** - `gh run view <id>` shows step-by-step status
+2. **Get the logs** - `gh run view <id> --log-failed` for focused output
+3. **Search for error patterns** - Look for `Error:`, `FAIL`, `exit code`, stack traces
+4. **Check annotations** - `gh api repos/{owner}/{repo}/check-runs/{id}/annotations`
+
+## Server Log Analysis
+
+### Log Collection Strategy
+
+1. **Identify log locations** - Application logs, system logs, web server logs
+2. **Filter by timeframe** - Narrow to incident window
+3. **Correlate request IDs** - Trace single request across services
+4. **Look for patterns** - Repeated errors, error rate changes, unusual payloads
+
+### Structured Log Queries
+
+```bash
+# Search application logs for errors (use Grep tool when possible)
+# Pattern: timestamp, level, message
+# Filter by time range and severity
+
+# PostgreSQL slow query log
+psql -c "SELECT query, calls, mean_exec_time FROM pg_stat_statements ORDER BY mean_exec_time DESC LIMIT 10;"
+
+# Check database connections
+psql -c "SELECT count(*), state FROM pg_stat_activity GROUP BY state;"
+```
+
+### Cross-Source Correlation
+
+1. **Align timestamps** across all log sources (timezone awareness)
+2. **Build timeline** - First error → propagation → user impact
+3. **Identify trigger** - What changed immediately before first error?
+4. **Map blast radius** - Which services/endpoints affected?
+
+## Application Log Analysis
+
+### Error Pattern Recognition
+
+- **Sudden spike** → Deployment, config change, external dependency failure
+- **Gradual increase** → Resource leak, data growth, degradation
+- **Periodic failures** → Cron jobs, scheduled tasks, resource contention
+- **Single endpoint** → Code bug, data issue, specific dependency
+- **All endpoints** → Infrastructure, database, network issue
+
+### Key Log Fields
+
+Prioritize: timestamp, level, error message, stack trace, request ID, user ID, endpoint, response code, duration
+
+### Evidence Preservation
+
+Always capture relevant log excerpts for the diagnostic report. Include:
+- Exact error messages and stack traces
+- Timestamps and request IDs
+- Before/after comparison (normal vs error state)
+- Counts and frequencies

package/.opencode/skills/ck-debug/references/performance-diagnostics.md

@@ -0,0 +1,113 @@
+# Performance Diagnostics
+
+Identify bottlenecks, analyze query performance, and develop optimization strategies.
+
+## When to Use
+
+- Response times increased significantly
+- Application feels slow or unresponsive
+- Database queries taking too long
+- High CPU/memory/disk usage
+- Resource exhaustion or OOM errors
+
+## Diagnostic Process
+
+### 1. Quantify the Problem
+
+**Measure before optimizing.** Establish baseline and current state.
+
+- What is the expected response time vs actual?
+- When did degradation start? (correlate with changes)
+- Which endpoints/operations are affected?
+- Is it consistent or intermittent?
+
+### 2. Identify the Bottleneck Layer
+
+```
+Request → Network → Web Server → Application → Database → Filesystem
+                                      ↓
+                              External APIs / Services
+```
+
+**Elimination approach:** Measure time at each layer to find where delay occurs.
+
+| Layer | Check | Tool |
+|-------|-------|------|
+| Network | Latency, DNS, TLS | `curl -w` timing, network logs |
+| Web server | Request queue, connections | Server metrics, access logs |
+| Application | CPU profiling, memory | Profiler, APM, `process.memoryUsage()` |
+| Database | Query time, connections | `EXPLAIN ANALYZE`, `pg_stat_statements` |
+| Filesystem | I/O wait, disk usage | `iostat`, `df -h` |
+| External APIs | Response time, timeouts | Request logging with durations |
+
+### 3. Database Performance
+
+#### PostgreSQL Diagnostics
+
+```sql
+-- Slow queries (requires pg_stat_statements extension)
+SELECT query, calls, mean_exec_time, total_exec_time
+FROM pg_stat_statements
+ORDER BY mean_exec_time DESC LIMIT 20;
+
+-- Active queries right now
+SELECT pid, now() - pg_stat_activity.query_start AS duration, query, state
+FROM pg_stat_activity
+WHERE state != 'idle'
+ORDER BY duration DESC;
+
+-- Table sizes and bloat
+SELECT relname, pg_size_pretty(pg_total_relation_size(relid))
+FROM pg_catalog.pg_statio_user_tables
+ORDER BY pg_total_relation_size(relid) DESC LIMIT 20;
+
+-- Missing indexes (sequential scans on large tables)
+SELECT relname, seq_scan, seq_tup_read, idx_scan
+FROM pg_stat_user_tables
+WHERE seq_scan > 100 AND seq_tup_read > 10000
+ORDER BY seq_tup_read DESC;
+
+-- Connection pool status
+SELECT count(*), state FROM pg_stat_activity GROUP BY state;
+```
+
+#### Query Optimization
+
+```sql
+-- Analyze specific query execution plan
+EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT) <your-query>;
+```
+
+**Look for:** Sequential scans on large tables, nested loops with high row counts, sorts without indexes, excessive buffer hits.
+
+### 4. Application Performance
+
+**Common bottlenecks:**
+
+| Issue | Symptom | Fix |
+|-------|---------|-----|
+| N+1 queries | Many small DB calls per request | Eager loading, batch queries |
+| Memory leaks | Growing memory over time | Profile heap, check event listeners |
+| Blocking I/O | High response time, low CPU | Async operations, connection pooling |
+| CPU-bound | High CPU, proportional to load | Optimize algorithms, caching |
+| Connection exhaustion | Intermittent timeouts | Pool sizing, connection reuse |
+| Large payloads | Slow transfers, high memory | Pagination, compression, streaming |
+
+### 5. Optimization Strategy
+
+**Priority order:**
+1. **Quick wins** - Add missing index, fix N+1 query, enable caching
+2. **Configuration** - Pool sizes, timeouts, buffer sizes, worker counts
+3. **Code changes** - Algorithm optimization, data structure changes
+4. **Architecture** - Caching layer, read replicas, async processing, CDN
+
+**Always:** Measure after each change to verify improvement. One change at a time.
+
+## Reporting Performance Issues
+
+Include in diagnostic report:
+- **Baseline vs current** metrics (with numbers)
+- **Bottleneck identification** with evidence
+- **Root cause** explanation
+- **Recommended fixes** with expected impact
+- **Verification plan** to confirm improvement

package/.opencode/skills/ck-debug/references/reporting-standards.md

@@ -0,0 +1,122 @@
+# Reporting Standards
+
+Structured format for diagnostic and investigation reports. Sacrifice grammar for concision.
+
+## When to Use
+
+- After completing system investigation
+- Summarizing debugging session findings
+- Producing incident post-mortems
+- Reporting performance analysis results
+
+## Report Structure
+
+### 1. Executive Summary (3-5 lines)
+
+- **Issue:** One-line description
+- **Impact:** Users/systems affected, severity
+- **Root cause:** One-line explanation
+- **Status:** Resolved / Mitigated / Under investigation
+- **Fix:** What was done or recommended
+
+### 2. Technical Analysis
+
+**Timeline:**
+```
+HH:MM - Event description
+HH:MM - Next event
+...
+```
+
+**Evidence:**
+- Relevant log excerpts (trimmed to essential lines)
+- Query results with key metrics
+- Error messages and stack traces
+- Before/after comparisons
+
+**Findings:**
+- List each finding with supporting evidence
+- Distinguish confirmed facts from hypotheses
+- Note correlation vs causation
+
+### 3. Actionable Recommendations
+
+**Immediate (P0):**
+- [ ] Critical fix with implementation steps
+
+**Short-term (P1):**
+- [ ] Follow-up improvements
+
+**Long-term (P2):**
+- [ ] Monitoring/alerting enhancements
+- [ ] Architecture improvements
+- [ ] Preventive measures
+
+Each recommendation: what to do, why, expected impact, effort estimate (low/medium/high).
+
+### 4. Supporting Evidence
+
+- Relevant log excerpts
+- Query results and execution plans
+- Performance metrics
+- Test results and error traces
+- Screenshots or diagrams if applicable
+
+### 5. Unresolved Questions
+
+List anything that remains unclear:
+- Items needing further investigation
+- Questions for the team
+- Assumptions that need validation
+
+## Report File Naming
+
+Use naming pattern from `## Naming` section injected by hooks. Pattern includes full path and computed date.
+
+**Example:** `plans/reports/debugger-260205-2215-api-500-investigation.md`
+
+## Writing Guidelines
+
+- **Concise:** Facts and evidence, not narrative. Sacrifice grammar for brevity
+- **Evidence-backed:** Every claim supported by logs, metrics, or reproduction steps
+- **Actionable:** Recommendations are specific with clear next steps
+- **Honest:** State unknowns explicitly. "Likely cause" vs "confirmed cause"
+- **Structured:** Use headers, tables, and bullet points for scanability
+
+## Template
+
+```markdown
+# [Issue Title] - Investigation Report
+
+## Executive Summary
+- **Issue:**
+- **Impact:**
+- **Root cause:**
+- **Status:**
+- **Fix:**
+
+## Timeline
+- HH:MM -
+- HH:MM -
+
+## Technical Analysis
+### Findings
+1.
+2.
+
+### Evidence
+[logs, queries, metrics]
+
+## Recommendations
+### Immediate (P0)
+- [ ]
+
+### Short-term (P1)
+- [ ]
+
+### Long-term (P2)
+- [ ]
+
+## Unresolved Questions
+-
+```

package/.opencode/skills/ck-debug/references/root-cause-tracing.md

@@ -0,0 +1,122 @@
+# Root Cause Tracing
+
+Systematically trace bugs backward through call stack to find original trigger.
+
+## Core Principle
+
+**Trace backward through call chain until finding original trigger, then fix at source.**
+
+Bugs often manifest deep in call stack (git init in wrong directory, file created in wrong location). Instinct is to fix where error appears, but that's treating symptom.
+
+## When to Use
+
+**Use when:**
+- Error happens deep in execution (not at entry point)
+- Stack trace shows long call chain
+- Unclear where invalid data originated
+- Need to find which test/code triggers problem
+
+## The Tracing Process
+
+### 1. Observe the Symptom
+```
+Error: git init failed in /Users/jesse/project/packages/core
+```
+
+### 2. Find Immediate Cause
+What code directly causes this?
+```typescript
+await execFileAsync('git', ['init'], { cwd: projectDir });
+```
+
+### 3. Ask: What Called This?
+```typescript
+WorktreeManager.createSessionWorktree(projectDir, sessionId)
+  → called by Session.initializeWorkspace()
+  → called by Session.create()
+  → called by test at Project.create()
+```
+
+### 4. Keep Tracing Up
+What value was passed?
+- `projectDir = ''` (empty string!)
+- Empty string as `cwd` resolves to `process.cwd()`
+- That's the source code directory!
+
+### 5. Find Original Trigger
+Where did empty string come from?
+```typescript
+const context = setupCoreTest(); // Returns { tempDir: '' }
+Project.create('name', context.tempDir); // Accessed before beforeEach!
+```
+
+## Adding Stack Traces
+
+When can't trace manually, add instrumentation:
+
+```typescript
+async function gitInit(directory: string) {
+  const stack = new Error().stack;
+  console.error('DEBUG git init:', {
+    directory,
+    cwd: process.cwd(),
+    stack,
+  });
+
+  await execFileAsync('git', ['init'], { cwd: directory });
+}
+```
+
+**Critical:** Use `console.error()` in tests (not logger - may not show)
+
+**Run and capture:**
+```bash
+npm test 2>&1 | grep 'DEBUG git init'
+```
+
+**Analyze stack traces:**
+- Look for test file names
+- Find line number triggering call
+- Identify pattern (same test? same parameter?)
+
+## Finding Which Test Causes Pollution
+
+If something appears during tests but don't know which test:
+
+Use bisection script: `scripts/find-polluter.sh`
+
+```bash
+./scripts/find-polluter.sh '.git' 'src/**/*.test.ts'
+```
+
+Runs tests one-by-one, stops at first polluter.
+
+## Key Principle
+
+**NEVER fix just where error appears.** Trace back to find original trigger.
+
+When found immediate cause:
+- Can trace one level up? → Trace backwards
+- Is this the source? → Fix at source
+- Then add validation at each layer (see defense-in-depth.md)
+
+## Real Example
+
+**Symptom:** `.git` created in `packages/core/` (source code)
+
+**Trace chain:**
+1. `git init` runs in `process.cwd()` ← empty cwd parameter
+2. WorktreeManager called with empty projectDir
+3. Session.create() passed empty string
+4. Test accessed `context.tempDir` before beforeEach
+5. setupCoreTest() returns `{ tempDir: '' }` initially
+
+**Root cause:** Top-level variable initialization accessing empty value
+
+**Fix:** Made tempDir a getter that throws if accessed before beforeEach
+
+**Also added defense-in-depth:**
+- Layer 1: Project.create() validates directory
+- Layer 2: WorkspaceManager validates not empty
+- Layer 3: NODE_ENV guard refuses git init outside tmpdir
+- Layer 4: Stack trace logging before git init

package/.opencode/skills/ck-debug/references/systematic-debugging.md

@@ -0,0 +1,102 @@
+# Systematic Debugging
+
+Four-phase debugging framework that ensures root cause investigation before attempting fixes.
+
+## The Iron Law
+
+```
+NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST
+```
+
+If haven't completed Phase 1, cannot propose fixes.
+
+## The Four Phases
+
+Must complete each phase before proceeding to next.
+
+### Phase 1: Root Cause Investigation
+
+**BEFORE attempting ANY fix:**
+
+1. **Read Error Messages Carefully** - Don't skip past errors/warnings, read stack traces completely
+2. **Reproduce Consistently** - Can trigger reliably? Exact steps? If not reproducible → gather more data
+3. **Check Recent Changes** - What changed? Git diff, recent commits, new dependencies, config changes
+4. **Gather Evidence in Multi-Component Systems**
+   - For EACH component boundary: log data entering/exiting, verify environment propagation
+   - Run once to gather evidence showing WHERE it breaks
+   - THEN analyze to identify failing component
+5. **Trace Data Flow** - Where does bad value originate? Trace up call stack until finding source (see root-cause-tracing.md)
+
+### Phase 2: Pattern Analysis
+
+**Find pattern before fixing:**
+
+1. **Find Working Examples** - Locate similar working code in same codebase
+2. **Compare Against References** - Read reference implementation COMPLETELY, understand fully before applying
+3. **Identify Differences** - List every difference however small, don't assume "that can't matter"
+4. **Understand Dependencies** - What other components, settings, config, environment needed?
+
+### Phase 3: Hypothesis and Testing
+
+**Scientific method:**
+
+1. **Form Single Hypothesis** - "I think X is root cause because Y", be specific not vague
+2. **Test Minimally** - SMALLEST possible change to test hypothesis, one variable at a time
+3. **Verify Before Continuing** - Worked? → Phase 4. Didn't work? → NEW hypothesis. DON'T add more fixes
+4. **When Don't Know** - Say "I don't understand X", don't pretend, ask for help
+
+### Phase 4: Implementation
+
+**Fix root cause, not symptom:**
+
+1. **Create Failing Test Case** - Simplest reproduction, automated if possible, MUST have before fixing
+2. **Implement Single Fix** - Address root cause identified, ONE change, no "while I'm here" improvements
+3. **Verify Fix** - Test passes? No other tests broken? Issue actually resolved?
+4. **If Fix Doesn't Work**
+   - STOP. Count: How many fixes tried?
+   - If < 3: Return to Phase 1, re-analyze with new information
+   - **If ≥ 3: STOP and question architecture**
+5. **If 3+ Fixes Failed: Question Architecture**
+   - Pattern: Each fix reveals new shared state/coupling problem elsewhere
+   - STOP and question fundamentals: Is pattern sound? Wrong architecture?
+   - Discuss with human partner before more fixes
+
+## Red Flags - STOP and Follow Process
+
+If catch yourself thinking:
+- "Quick fix for now, investigate later"
+- "Just try changing X and see if it works"
+- "Add multiple changes, run tests"
+- "Skip the test, I'll manually verify"
+- "It's probably X, let me fix that"
+- "I don't fully understand but this might work"
+- "One more fix attempt" (when already tried 2+)
+
+**ALL mean:** STOP. Return to Phase 1.
+
+## Human Partner Signals You're Doing It Wrong
+
+- "Is that not happening?" - Assumed without verifying
+- "Will it show us...?" - Should have added evidence gathering
+- "Stop guessing" - Proposing fixes without understanding
+- "Ultrathink this" - Question fundamentals, not just symptoms
+- "We're stuck?" (frustrated) - Approach isn't working
+
+**When see these:** STOP. Return to Phase 1.
+
+## Common Rationalizations
+
+| Excuse | Reality |
+|--------|---------|
+| "Issue is simple, don't need process" | Simple issues have root causes too |
+| "Emergency, no time for process" | Systematic is FASTER than guess-and-check |
+| "Just try this first, then investigate" | First fix sets pattern. Do right from start |
+| "One more fix attempt" (after 2+ failures) | 3+ failures = architectural problem |
+
+## Real-World Impact
+
+From debugging sessions:
+- Systematic approach: 15-30 minutes to fix
+- Random fixes approach: 2-3 hours of thrashing
+- First-time fix rate: 95% vs 40%
+- New bugs introduced: Near zero vs common