claude-mpm 4.20.3__py3-none-any.whl → 4.25.10__py3-none-any.whl

claude_mpm/skills/bundled/debugging/verification-before-completion/references/verification-patterns.md

@@ -24,6 +24,18 @@ Complete patterns for verifying different types of claims before making them.
 - Exact pass/fail counts
 - Exit code confirmed (0 = success)
 
+**Edge Cases:**
+- Flaky tests: Run multiple times, document flakiness
+- Timeout tests: Verify timeout values appropriate
+- Skipped tests: Count and document why skipped
+- Warnings in output: Document and address
+
+**Troubleshooting:**
+- **Tests hang**: Check for infinite loops, deadlocks
+- **Random failures**: Run 10x, identify flaky tests
+- **Environment issues**: Verify dependencies, config
+- **Coverage gaps**: Check coverage report for holes
+
 ## Regression Test Verification (TDD Red-Green Cycle)
 
 **Correct Pattern:**
@@ -49,6 +61,11 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ Test passes on first run (never confirmed it catches bug)
 - ❌ Single pass without reverting (could be false positive)
 
+**Edge Cases:**
+- Test passes immediately: Test doesn't catch bug, rewrite
+- Test fails differently: Fix changed behavior, investigate
+- Can't revert cleanly: Use version control, stash changes
+
 ## Build Verification
 
 **Correct Pattern:**
@@ -71,6 +88,18 @@ Complete patterns for verifying different types of claims before making them.
 - Local build ≠ CI build
 - Dev build ≠ production build
 
+**Build Types to Verify:**
+- Development build: Fast iteration
+- Production build: Minification, optimization
+- Test build: Coverage instrumentation
+- Distribution build: Platform-specific artifacts
+
+**Troubleshooting:**
+- **Build succeeds but artifacts missing**: Check output directory
+- **Incremental build issues**: Clean build, verify again
+- **Platform-specific failures**: Test on target platform
+- **Resource exhaustion**: Check memory, disk space
+
 ## Linter Verification
 
 **Correct Pattern:**
@@ -87,6 +116,20 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ "Linter passed on one file" (not comprehensive)
 - ❌ "Should be clean now" (no verification)
 
+**Multiple Linters Pattern:**
+```
+1. Run each linter separately
+2. Verify each independently
+3. Document results for each
+4. Only claim clean if ALL pass
+```
+
+**Edge Cases:**
+- Warnings vs errors: Document acceptable warnings
+- Auto-fix available: Run auto-fix, verify results
+- Custom rules: Verify custom rules active
+- Ignored files: Document why files ignored
+
 ## Bug Fix Verification
 
 **Correct Pattern:**
@@ -105,6 +148,19 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ "Logic looks correct" (theory, not evidence)
 - ❌ "Can't reproduce anymore" (didn't verify with test)
 
+**Bug Types:**
+- **Logic bugs**: Unit test verifies correct behavior
+- **Race conditions**: Stress test, run 100x
+- **Edge cases**: Test boundary conditions
+- **Integration bugs**: End-to-end test verifies
+- **Performance bugs**: Benchmark before/after
+
+**Troubleshooting:**
+- **Can't reproduce**: Document steps, environment
+- **Intermittent bug**: Increase test iterations
+- **Different in production**: Test in production-like environment
+- **Fix causes regression**: Test full suite, not just bug test
+
 ## Requirements Verification
 
 **Correct Pattern:**
@@ -121,6 +177,22 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ "I implemented what was asked" (subjective)
 - ❌ "Phase complete" (vague, no checklist)
 
+**Requirement Types:**
+- **Functional**: Feature works as specified
+- **Non-functional**: Performance, scalability, security
+- **User experience**: UI/UX matches design
+- **Integration**: Works with other systems
+- **Documentation**: Docs complete and accurate
+
+**Checklist Format:**
+```
+[ ] Requirement 1: [Evidence: test_feature_x passes]
+[ ] Requirement 2: [Evidence: benchmark shows 50ms response]
+[ ] Requirement 3: [Evidence: manual testing confirms behavior]
+[X] Requirement 4: MISSING - not yet implemented
+[ ] Requirement 5: [Evidence: integration test passes]
+```
+
 ## Agent Delegation Verification
 
 **Correct Pattern:**
@@ -138,6 +210,31 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ "Agent said success, moving on"
 - ❌ Not checking actual changes made
 
+**Agent Verification Checklist:**
+```
+1. Review git diff: What changed?
+2. Run tests: Do tests still pass?
+3. Check task requirements: All met?
+4. Look for unexpected changes: Any surprises?
+5. Verify no secrets added: .env, keys, tokens?
+6. Check for commented code: Any TODOs added?
+7. Verify imports/dependencies: Any new ones?
+```
+
+**Common Agent Failures:**
+- Agent adds TODO instead of implementing
+- Agent introduces syntax errors
+- Agent misunderstands requirements
+- Agent makes unrelated changes
+- Agent adds secrets to version control
+- Agent breaks existing functionality
+
+**Troubleshooting:**
+- **Agent claims success but tests fail**: Review diff, find issue
+- **Agent made unexpected changes**: Revert, clarify task
+- **Agent added placeholder code**: Complete implementation
+- **Agent broke unrelated code**: Partial revert, fix issue
+
 ## Deployment Verification
 
 **Correct Pattern:**
@@ -155,6 +252,39 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ "Should be live now" (no verification)
 - ❌ "Deployed to staging" (didn't verify it works)
 
+**Environment-Specific Verification:**
+
+**Staging:**
+```
+1. Deploy to staging
+2. Run full test suite against staging
+3. Check all endpoints
+4. Verify database migrations
+5. Test with production-like data
+```
+
+**Production:**
+```
+1. Deploy to production
+2. Verify zero-downtime deployment
+3. Check health endpoints
+4. Monitor error rates
+5. Verify key user flows
+6. Keep rollback ready
+```
+
+**Deployment Types:**
+- **Blue-green**: Verify both environments
+- **Canary**: Monitor canary metrics
+- **Rolling**: Verify each batch
+- **Feature flags**: Verify flag state
+
+**Troubleshooting:**
+- **Deployment succeeds but service down**: Check logs, restart
+- **Configuration mismatch**: Verify environment variables
+- **Database migration failed**: Rollback, investigate
+- **Partial deployment**: Complete or rollback fully
+
 ## Performance Verification
 
 **Correct Pattern:**
@@ -174,6 +304,26 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ "Looks quicker" (subjective)
 - ❌ Single measurement (could be outlier)
 
+**Benchmark Requirements:**
+- **Sample size**: Minimum 10 runs
+- **Statistical significance**: Calculate standard deviation
+- **Cold vs warm**: Test both scenarios
+- **Load levels**: Test under different loads
+- **Environment**: Same environment for before/after
+
+**Metrics to Measure:**
+- **Response time**: p50, p95, p99 latency
+- **Throughput**: Requests per second
+- **Resource usage**: CPU, memory, disk I/O
+- **Database queries**: Query count, duration
+- **Network**: Bandwidth, connection count
+
+**Edge Cases:**
+- Performance regression: Identify cause, revert if needed
+- Inconsistent results: Check for external factors
+- Different under load: Load test before claiming
+- Memory leaks: Profile memory over time
+
 ## Security Verification
 
 **Correct Pattern:**
@@ -192,6 +342,42 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ "No obvious vulnerabilities" (didn't scan)
 - ❌ "Should be safe" (assumption)
 
+**Security Layers to Verify:**
+
+**Static Analysis:**
+```
+1. Code scanning: bandit, semgrep
+2. Dependency scanning: safety check
+3. Secret scanning: git secrets
+4. License compliance: license checker
+```
+
+**Dynamic Analysis:**
+```
+1. Penetration testing: OWASP ZAP
+2. Fuzzing: Input validation
+3. Load testing: DoS resistance
+4. Authentication testing: Auth flows
+```
+
+**Security Checklist:**
+- [ ] Input validation implemented
+- [ ] SQL injection prevention
+- [ ] XSS prevention
+- [ ] CSRF protection
+- [ ] Authentication secure
+- [ ] Authorization checks
+- [ ] Secrets not in code
+- [ ] HTTPS enforced
+- [ ] Headers configured
+- [ ] Dependencies updated
+
+**Troubleshooting:**
+- **False positives**: Document and suppress with justification
+- **Critical findings**: Fix immediately, don't suppress
+- **Dependency vulnerabilities**: Update or replace
+- **Configuration issues**: Fix in deployment configs
+
 ## Documentation Verification
 
 **Correct Pattern:**
@@ -210,6 +396,92 @@ Complete patterns for verifying different types of claims before making them.
 - ❌ "Clear to me" (author bias)
 - ❌ "Should be understandable" (no verification)
 
+**Documentation Types:**
+
+**API Documentation:**
+```
+1. Document all endpoints
+2. Include request/response examples
+3. Document error codes
+4. Test with curl/Postman
+5. Verify examples work
+```
+
+**User Documentation:**
+```
+1. Write step-by-step guide
+2. Include screenshots
+3. Test with fresh user
+4. Record where they get stuck
+5. Improve those sections
+```
+
+**Developer Documentation:**
+```
+1. Document setup steps
+2. Fresh clone, follow steps
+3. Verify can build and run
+4. Document all prerequisites
+5. Test on clean machine
+```
+
+**Troubleshooting:**
+- **User gets stuck**: Add more detail, screenshots
+- **Prerequisites missing**: Document all dependencies
+- **Examples don't work**: Verify and fix examples
+- **Outdated documentation**: Regular review and updates
+
+## CI/CD Pipeline Verification
+
+**Correct Pattern:**
+```
+1. Push to branch
+2. Watch CI pipeline run
+3. Verify all stages pass: build, test, lint, security
+4. Check pipeline logs for warnings
+5. Verify artifacts created
+6. Claim: "CI pipeline passes, all stages green"
+```
+
+**Pipeline Stages to Verify:**
+- **Build**: Code compiles, artifacts created
+- **Test**: All tests pass, coverage threshold met
+- **Lint**: Code style compliance
+- **Security**: No vulnerabilities found
+- **Deploy**: Successful deployment to target
+
+**Edge Cases:**
+- Flaky pipeline: Investigate root cause, fix
+- Timeout issues: Optimize slow stages
+- Cache issues: Clear cache, rebuild
+- Secrets missing: Verify environment variables
+
+## Database Migration Verification
+
+**Correct Pattern:**
+```
+1. Backup database
+2. Run migration on backup
+3. Verify data integrity
+4. Test rollback
+5. Run on staging
+6. Verify application works
+7. Document any data changes
+8. Claim: "Migration tested, rollback verified"
+```
+
+**Critical Checks:**
+- **Data loss prevention**: Verify no data dropped
+- **Rollback tested**: Must be able to undo
+- **Application compatibility**: App works during migration
+- **Performance impact**: Migration doesn't lock tables long
+
+**Edge Cases:**
+- Large tables: Use online migration tools
+- Zero-downtime: Test deployment order
+- Data transformation: Verify transformation logic
+- Foreign key constraints: Check constraint violations
+
 ## The Universal Pattern
 
 All verification follows this structure: