@dzhechkov/keysarium 1.3.0 → 1.5.0

package/templates/.claude/skills/feature-adr/references/agentic-qe/regression-testing.md

@@ -0,0 +1,233 @@
+---
+name: regression-testing
+description: "Strategic regression testing with test selection, impact analysis, and continuous regression management. Use when verifying fixes don't break existing functionality, planning regression suites, or optimizing test execution for faster feedback."
+category: specialized-testing
+priority: high
+tokenEstimate: 1000
+agents: [qe-regression-risk-analyzer, qe-test-executor, qe-coverage-analyzer]
+implementation_status: optimized
+optimization_version: 1.0
+last_optimized: 2025-12-02
+dependencies: []
+quick_reference_card: true
+tags: [regression, test-selection, impact-analysis, ci-cd, change-based, risk-based]
+trust_tier: 3
+validation:
+  schema_path: schemas/output.json
+  validator_path: scripts/validate-config.json
+  eval_path: evals/regression-testing.yaml
+
+---
+
+# Regression Testing
+
+<default_to_action>
+When verifying changes don't break existing functionality:
+1. ANALYZE what changed (git diff, impact analysis)
+2. SELECT tests based on change + risk (not everything)
+3. RUN in priority order (smoke → selective → full)
+4. OPTIMIZE execution (parallel, sharding)
+5. MONITOR suite health (flakiness, execution time)
+
+**Quick Regression Strategy:**
+- Per-commit: Smoke + changed code tests (5-10 min)
+- Nightly: Extended regression (30-60 min)
+- Pre-release: Full regression (2-4 hours)
+
+**Critical Success Factors:**
+- Smart selection catches 90% of regressions in 10% of time
+- Flaky tests waste more time than they save
+- Every production bug becomes a regression test
+</default_to_action>
+
+## Quick Reference Card
+
+### When to Use
+- After any code change
+- Before release
+- After dependency updates
+- After environment changes
+
+### Regression Types
+| Type | When | Scope |
+|------|------|-------|
+| **Corrective** | No code change | Full suite |
+| **Progressive** | New features | Existing + new |
+| **Selective** | Specific changes | Changed + dependent |
+| **Complete** | Major refactor | Everything |
+
+### Test Selection Strategies
+| Strategy | How | Reduction |
+|----------|-----|-----------|
+| **Change-based** | Git diff analysis | 70-90% |
+| **Risk-based** | Priority by impact | 50-70% |
+| **Historical** | Frequently failing | 40-60% |
+| **Time-budget** | Fixed time window | Variable |
+
+---
+
+## Change-Based Test Selection
+
+```typescript
+// Analyze changed files and select impacted tests
+function selectTests(changedFiles: string[]): string[] {
+  const testsToRun = new Set<string>();
+
+  for (const file of changedFiles) {
+    // Direct tests
+    testsToRun.add(`${file.replace('.ts', '.test.ts')}`);
+
+    // Dependent tests (via coverage mapping)
+    const dependentTests = testCoverage[file] || [];
+    dependentTests.forEach(t => testsToRun.add(t));
+  }
+
+  return Array.from(testsToRun);
+}
+
+// Example: payment.ts changed
+// Runs: payment.test.ts, checkout.integration.test.ts, e2e/purchase.test.ts
+```
+
+---
+
+## Regression Suite Pyramid
+
+```
+         /\
+        /  \  Full Regression (weekly)
+       /    \  - All tests (2-4 hours)
+      /------\
+     /        \  Extended Regression (nightly)
+    /          \  - Unit + integration + critical E2E (30-60 min)
+   /------------\
+  /              \  Quick Regression (per commit)
+ /________________\  - Changed code + smoke tests (5-10 min)
+```
+
+---
+
+## CI/CD Integration
+
+```yaml
+# .github/workflows/regression.yml
+jobs:
+  quick-regression:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - name: Analyze changes
+        id: changes
+        uses: dorny/paths-filter@v2
+        with:
+          filters: |
+            payment:
+              - 'src/payment/**'
+            auth:
+              - 'src/auth/**'
+
+      - name: Run affected tests
+        run: npm run test:affected
+
+      - name: Smoke tests (always)
+        run: npm run test:smoke
+
+  nightly-regression:
+    if: github.event_name == 'schedule'
+    timeout-minutes: 120
+    steps:
+      - run: npm test -- --coverage
+```
+
+---
+
+## Optimization Techniques
+
+```javascript
+// 1. Parallel execution
+module.exports = {
+  maxWorkers: '50%', // Use half CPU cores
+  testTimeout: 30000
+};
+
+// 2. Sharding across CI workers
+// npm test -- --shard=1/4
+
+// 3. Incremental testing (only changed since last run)
+// Track last run state, skip passing unchanged tests
+
+// 4. Fast-fail on smoke
+// Run critical tests first, abort if they fail
+```
+
+---
+
+## Agent-Driven Regression
+
+```typescript
+// Smart test selection
+await Task("Regression Analysis", {
+  pr: 1234,
+  strategy: 'change-based-with-risk',
+  timeBudget: '15min'
+}, "qe-regression-risk-analyzer");
+
+// Returns:
+// {
+//   mustRun: ['payment.test.ts', 'checkout.integration.test.ts'],
+//   shouldRun: ['order.test.ts'],
+//   canSkip: ['profile.test.ts', 'search.test.ts'],
+//   estimatedTime: '12 min',
+//   riskCoverage: 0.94
+// }
+
+// Generate regression test from production bug
+await Task("Bug Regression Test", {
+  bug: { id: 'BUG-567', description: 'Checkout fails > 100 items' },
+  preventRecurrence: true
+}, "qe-test-generator");
+```
+
+---
+
+## Agent Coordination Hints
+
+### Memory Namespace
+```
+aqe/regression-testing/
+├── test-selection/*     - Impact analysis results
+├── suite-health/*       - Flakiness, timing trends
+├── coverage-maps/*      - Test-to-code mapping
+└── bug-regressions/*    - Tests from production bugs
+```
+
+### Fleet Coordination
+```typescript
+const regressionFleet = await FleetManager.coordinate({
+  strategy: 'comprehensive-regression',
+  agents: [
+    'qe-regression-risk-analyzer',  // Analyze changes, select tests
+    'qe-test-executor',             // Execute selected tests
+    'qe-coverage-analyzer',         // Analyze coverage gaps
+    'qe-quality-gate'               // Go/no-go decision
+  ],
+  topology: 'sequential'
+});
+```
+
+---
+
+## Related Skills
+- [risk-based-testing](../risk-based-testing/) - Risk-based prioritization
+- [test-automation-strategy](../test-automation-strategy/) - Automation pyramid
+- [continuous-testing-shift-left](../continuous-testing-shift-left/) - CI/CD integration
+
+---
+
+## Remember
+
+**Regression testing is insurance against change.** Every code change is a risk. Smart regression testing mitigates that risk by testing what matters based on what changed.
+
+**Good regression testing is strategic, not exhaustive.** You cannot test everything, every time. Select based on changes, risk, and time budget.
+
+**With Agents:** `qe-regression-risk-analyzer` provides intelligent test selection achieving 90% defect detection in 10% of execution time. Agents generate regression tests from production bugs automatically.

package/templates/.claude/skills/feature-adr/references/agentic-qe/risk-based-testing.md

@@ -0,0 +1,212 @@
+---
+name: risk-based-testing
+description: "Focus testing effort on highest-risk areas using risk assessment and prioritization. Use when planning test strategy, allocating testing resources, or making coverage decisions."
+category: testing-methodologies
+priority: high
+tokenEstimate: 1000
+agents: [qe-regression-risk-analyzer, qe-test-generator, qe-production-intelligence, qe-quality-gate]
+implementation_status: optimized
+optimization_version: 1.0
+last_optimized: 2025-12-02
+dependencies: []
+quick_reference_card: true
+tags: [risk, prioritization, test-planning, coverage, impact-analysis]
+trust_tier: 3
+validation:
+  schema_path: schemas/output.json
+  validator_path: scripts/validate-config.json
+  eval_path: evals/risk-based-testing.yaml
+
+---
+
+# Risk-Based Testing
+
+<default_to_action>
+When planning tests or allocating testing resources:
+1. IDENTIFY risks: What can go wrong? What's the impact? What's the likelihood?
+2. CALCULATE risk: Risk = Probability × Impact (use 1-5 scale for each)
+3. PRIORITIZE: Critical (20+) → High (12-19) → Medium (6-11) → Low (1-5)
+4. ALLOCATE effort: 60% critical, 25% high, 10% medium, 5% low
+5. REASSESS continuously: New info, changes, production incidents
+
+**Quick Risk Assessment:**
+- Probability factors: Complexity, change frequency, developer experience, technical debt
+- Impact factors: User count, revenue, safety, reputation, regulatory
+- Dynamic adjustment: Production bugs increase risk; stable code decreases
+
+**Critical Success Factors:**
+- Test where bugs hurt most, not everywhere equally
+- Risk is dynamic - reassess with new information
+- Production data informs risk (shift-right feeds shift-left)
+</default_to_action>
+
+## Quick Reference Card
+
+### When to Use
+- Planning sprint/release test strategy
+- Deciding what to automate first
+- Allocating limited testing time
+- Justifying test coverage decisions
+
+### Risk Calculation
+```
+Risk Score = Probability (1-5) × Impact (1-5)
+```
+
+| Score | Priority | Effort | Action |
+|-------|----------|--------|--------|
+| 20-25 | Critical | 60% | Comprehensive testing, multiple techniques |
+| 12-19 | High | 25% | Thorough testing, automation priority |
+| 6-11 | Medium | 10% | Standard testing, basic automation |
+| 1-5 | Low | 5% | Smoke test, exploratory only |
+
+### Probability Factors
+| Factor | Low (1) | Medium (3) | High (5) |
+|--------|---------|------------|----------|
+| Complexity | Simple CRUD | Business logic | Algorithms, integrations |
+| Change Rate | Stable 6+ months | Monthly changes | Weekly/daily changes |
+| Developer Experience | Senior, domain expert | Mid-level | Junior, new to codebase |
+| Technical Debt | Clean code | Some debt | Legacy, no tests |
+
+### Impact Factors
+| Factor | Low (1) | Medium (3) | High (5) |
+|--------|---------|------------|----------|
+| Users Affected | Admin only | Department | All users |
+| Revenue | None | Indirect | Direct (checkout) |
+| Safety | Convenience | Data loss | Physical harm |
+| Reputation | Internal | Industry | Public scandal |
+
+---
+
+## Risk Assessment Workflow
+
+### Step 1: List Features/Components
+```
+Feature | Probability | Impact | Risk | Priority
+--------|-------------|--------|------|----------
+Checkout | 4 | 5 | 20 | Critical
+User Auth | 3 | 5 | 15 | High
+Admin Panel | 2 | 2 | 4 | Low
+Search | 3 | 3 | 9 | Medium
+```
+
+### Step 2: Apply Test Depth
+```typescript
+await Task("Risk-Based Test Generation", {
+  critical: {
+    features: ['checkout', 'payment'],
+    depth: 'comprehensive',
+    techniques: ['unit', 'integration', 'e2e', 'performance', 'security']
+  },
+  high: {
+    features: ['auth', 'user-profile'],
+    depth: 'thorough',
+    techniques: ['unit', 'integration', 'e2e']
+  },
+  medium: {
+    features: ['search', 'notifications'],
+    depth: 'standard',
+    techniques: ['unit', 'integration']
+  },
+  low: {
+    features: ['admin-panel', 'settings'],
+    depth: 'smoke',
+    techniques: ['smoke-tests']
+  }
+}, "qe-test-generator");
+```
+
+### Step 3: Reassess Dynamically
+```typescript
+// Production incident increases risk
+await Task("Update Risk Score", {
+  feature: 'search',
+  event: 'production-incident',
+  previousRisk: 9,
+  newProbability: 5,  // Increased due to incident
+  newRisk: 15         // Now HIGH priority
+}, "qe-regression-risk-analyzer");
+```
+
+---
+
+## ML-Enhanced Risk Analysis
+
+```typescript
+// Agent predicts risk using historical data
+const riskAnalysis = await Task("ML Risk Analysis", {
+  codeChanges: changedFiles,
+  historicalBugs: bugDatabase,
+  prediction: {
+    model: 'gradient-boosting',
+    factors: ['complexity', 'change-frequency', 'author-experience', 'file-age']
+  }
+}, "qe-regression-risk-analyzer");
+
+// Output: 95% accuracy risk prediction per file
+```
+
+---
+
+## Agent Coordination Hints
+
+### Memory Namespace
+```
+aqe/risk-based/
+├── risk-scores/*        - Current risk assessments
+├── historical-bugs/*    - Bug patterns by area
+├── production-data/*    - Incident data for risk
+└── coverage-map/*       - Test depth by risk level
+```
+
+### Fleet Coordination
+```typescript
+const riskFleet = await FleetManager.coordinate({
+  strategy: 'risk-based-testing',
+  agents: [
+    'qe-regression-risk-analyzer',  // Risk scoring
+    'qe-test-generator',            // Risk-appropriate tests
+    'qe-production-intelligence',   // Production feedback
+    'qe-quality-gate'               // Risk-based gates
+  ],
+  topology: 'sequential'
+});
+```
+
+---
+
+## Integration with CI/CD
+
+```yaml
+# Risk-based test selection in pipeline
+- name: Risk Analysis
+  run: aqe risk-analyze --changes ${{ github.event.pull_request.files }}
+
+- name: Run Critical Tests
+  if: risk.critical > 0
+  run: npm run test:critical
+
+- name: Run High Tests
+  if: risk.high > 0
+  run: npm run test:high
+
+- name: Skip Low Risk
+  if: risk.low_only
+  run: npm run test:smoke
+```
+
+---
+
+## Related Skills
+- [agentic-quality-engineering](../agentic-quality-engineering/) - Risk-aware agents
+- [context-driven-testing](../context-driven-testing/) - Context affects risk
+- [regression-testing](../regression-testing/) - Risk-based regression selection
+- [shift-right-testing](../shift-right-testing/) - Production informs risk
+
+---
+
+## Remember
+
+**Risk = Probability × Impact.** Test where bugs hurt most. Critical gets 60%, low gets 5%. Risk is dynamic - reassess with new info. Production incidents raise risk scores.
+
+**With Agents:** Agents calculate risk using ML on historical data, select risk-appropriate tests, and adjust scores from production feedback. Use agents to maintain dynamic risk profiles at scale.