agentvibes 2.13.6 → 2.13.8

package/.bmad/bmm/testarch/knowledge/test-priorities-matrix.md

@@ -1,373 +0,0 @@
-<!-- Powered by BMAD-CORE™ -->
-
-# Test Priorities Matrix
-
-Guide for prioritizing test scenarios based on risk, criticality, and business impact.
-
-## Priority Levels
-
-### P0 - Critical (Must Test)
-
-**Criteria:**
-
-- Revenue-impacting functionality
-- Security-critical paths
-- Data integrity operations
-- Regulatory compliance requirements
-- Previously broken functionality (regression prevention)
-
-**Examples:**
-
-- Payment processing
-- Authentication/authorization
-- User data creation/deletion
-- Financial calculations
-- GDPR/privacy compliance
-
-**Testing Requirements:**
-
-- Comprehensive coverage at all levels
-- Both happy and unhappy paths
-- Edge cases and error scenarios
-- Performance under load
-
-### P1 - High (Should Test)
-
-**Criteria:**
-
-- Core user journeys
-- Frequently used features
-- Features with complex logic
-- Integration points between systems
-- Features affecting user experience
-
-**Examples:**
-
-- User registration flow
-- Search functionality
-- Data import/export
-- Notification systems
-- Dashboard displays
-
-**Testing Requirements:**
-
-- Primary happy paths required
-- Key error scenarios
-- Critical edge cases
-- Basic performance validation
-
-### P2 - Medium (Nice to Test)
-
-**Criteria:**
-
-- Secondary features
-- Admin functionality
-- Reporting features
-- Configuration options
-- UI polish and aesthetics
-
-**Examples:**
-
-- Admin settings panels
-- Report generation
-- Theme customization
-- Help documentation
-- Analytics tracking
-
-**Testing Requirements:**
-
-- Happy path coverage
-- Basic error handling
-- Can defer edge cases
-
-### P3 - Low (Test if Time Permits)
-
-**Criteria:**
-
-- Rarely used features
-- Nice-to-have functionality
-- Cosmetic issues
-- Non-critical optimizations
-
-**Examples:**
-
-- Advanced preferences
-- Legacy feature support
-- Experimental features
-- Debug utilities
-
-**Testing Requirements:**
-
-- Smoke tests only
-- Can rely on manual testing
-- Document known limitations
-
-## Risk-Based Priority Adjustments
-
-### Increase Priority When:
-
-- High user impact (affects >50% of users)
-- High financial impact (>$10K potential loss)
-- Security vulnerability potential
-- Compliance/legal requirements
-- Customer-reported issues
-- Complex implementation (>500 LOC)
-- Multiple system dependencies
-
-### Decrease Priority When:
-
-- Feature flag protected
-- Gradual rollout planned
-- Strong monitoring in place
-- Easy rollback capability
-- Low usage metrics
-- Simple implementation
-- Well-isolated component
-
-## Test Coverage by Priority
-
-| Priority | Unit Coverage | Integration Coverage | E2E Coverage       |
-| -------- | ------------- | -------------------- | ------------------ |
-| P0       | >90%          | >80%                 | All critical paths |
-| P1       | >80%          | >60%                 | Main happy paths   |
-| P2       | >60%          | >40%                 | Smoke tests        |
-| P3       | Best effort   | Best effort          | Manual only        |
-
-## Priority Assignment Rules
-
-1. **Start with business impact** - What happens if this fails?
-2. **Consider probability** - How likely is failure?
-3. **Factor in detectability** - Would we know if it failed?
-4. **Account for recoverability** - Can we fix it quickly?
-
-## Priority Decision Tree
-
-```
-Is it revenue-critical?
-├─ YES → P0
-└─ NO → Does it affect core user journey?
-    ├─ YES → Is it high-risk?
-    │   ├─ YES → P0
-    │   └─ NO → P1
-    └─ NO → Is it frequently used?
-        ├─ YES → P1
-        └─ NO → Is it customer-facing?
-            ├─ YES → P2
-            └─ NO → P3
-```
-
-## Test Execution Order
-
-1. Execute P0 tests first (fail fast on critical issues)
-2. Execute P1 tests second (core functionality)
-3. Execute P2 tests if time permits
-4. P3 tests only in full regression cycles
-
-## Continuous Adjustment
-
-Review and adjust priorities based on:
-
-- Production incident patterns
-- User feedback and complaints
-- Usage analytics
-- Test failure history
-- Business priority changes
-
----
-
-## Automated Priority Classification
-
-### Example: Priority Calculator (Risk-Based Automation)
-
-```typescript
-// src/testing/priority-calculator.ts
-
-export type Priority = 'P0' | 'P1' | 'P2' | 'P3';
-
-export type PriorityFactors = {
-  revenueImpact: 'critical' | 'high' | 'medium' | 'low' | 'none';
-  userImpact: 'all' | 'majority' | 'some' | 'few' | 'minimal';
-  securityRisk: boolean;
-  complianceRequired: boolean;
-  previousFailure: boolean;
-  complexity: 'high' | 'medium' | 'low';
-  usage: 'frequent' | 'regular' | 'occasional' | 'rare';
-};
-
-/**
- * Calculate test priority based on multiple factors
- * Mirrors the priority decision tree with objective criteria
- */
-export function calculatePriority(factors: PriorityFactors): Priority {
-  const { revenueImpact, userImpact, securityRisk, complianceRequired, previousFailure, complexity, usage } = factors;
-
-  // P0: Revenue-critical, security, or compliance
-  if (revenueImpact === 'critical' || securityRisk || complianceRequired || (previousFailure && revenueImpact === 'high')) {
-    return 'P0';
-  }
-
-  // P0: High revenue + high complexity + frequent usage
-  if (revenueImpact === 'high' && complexity === 'high' && usage === 'frequent') {
-    return 'P0';
-  }
-
-  // P1: Core user journey (majority impacted + frequent usage)
-  if (userImpact === 'all' || userImpact === 'majority') {
-    if (usage === 'frequent' || complexity === 'high') {
-      return 'P1';
-    }
-  }
-
-  // P1: High revenue OR high complexity with regular usage
-  if ((revenueImpact === 'high' && usage === 'regular') || (complexity === 'high' && usage === 'frequent')) {
-    return 'P1';
-  }
-
-  // P2: Secondary features (some impact, occasional usage)
-  if (userImpact === 'some' || usage === 'occasional') {
-    return 'P2';
-  }
-
-  // P3: Rarely used, low impact
-  return 'P3';
-}
-
-/**
- * Generate priority justification (for audit trail)
- */
-export function justifyPriority(factors: PriorityFactors): string {
-  const priority = calculatePriority(factors);
-  const reasons: string[] = [];
-
-  if (factors.revenueImpact === 'critical') reasons.push('critical revenue impact');
-  if (factors.securityRisk) reasons.push('security-critical');
-  if (factors.complianceRequired) reasons.push('compliance requirement');
-  if (factors.previousFailure) reasons.push('regression prevention');
-  if (factors.userImpact === 'all' || factors.userImpact === 'majority') {
-    reasons.push(`impacts ${factors.userImpact} users`);
-  }
-  if (factors.complexity === 'high') reasons.push('high complexity');
-  if (factors.usage === 'frequent') reasons.push('frequently used');
-
-  return `${priority}: ${reasons.join(', ')}`;
-}
-
-/**
- * Example: Payment scenario priority calculation
- */
-const paymentScenario: PriorityFactors = {
-  revenueImpact: 'critical',
-  userImpact: 'all',
-  securityRisk: true,
-  complianceRequired: true,
-  previousFailure: false,
-  complexity: 'high',
-  usage: 'frequent',
-};
-
-console.log(calculatePriority(paymentScenario)); // 'P0'
-console.log(justifyPriority(paymentScenario));
-// 'P0: critical revenue impact, security-critical, compliance requirement, impacts all users, high complexity, frequently used'
-```
-
-### Example: Test Suite Tagging Strategy
-
-```typescript
-// tests/e2e/checkout.spec.ts
-import { test, expect } from '@playwright/test';
-
-// Tag tests with priority for selective execution
-test.describe('Checkout Flow', () => {
-  test('valid payment completes successfully @p0 @smoke @revenue', async ({ page }) => {
-    // P0: Revenue-critical happy path
-    await page.goto('/checkout');
-    await page.getByTestId('payment-method').selectOption('credit-card');
-    await page.getByTestId('card-number').fill('4242424242424242');
-    await page.getByRole('button', { name: 'Place Order' }).click();
-
-    await expect(page.getByText('Order confirmed')).toBeVisible();
-  });
-
-  test('expired card shows user-friendly error @p1 @error-handling', async ({ page }) => {
-    // P1: Core error scenario (frequent user impact)
-    await page.goto('/checkout');
-    await page.getByTestId('payment-method').selectOption('credit-card');
-    await page.getByTestId('card-number').fill('4000000000000069'); // Test card: expired
-    await page.getByRole('button', { name: 'Place Order' }).click();
-
-    await expect(page.getByText('Card expired. Please use a different card.')).toBeVisible();
-  });
-
-  test('coupon code applies discount correctly @p2', async ({ page }) => {
-    // P2: Secondary feature (nice-to-have)
-    await page.goto('/checkout');
-    await page.getByTestId('coupon-code').fill('SAVE10');
-    await page.getByRole('button', { name: 'Apply' }).click();
-
-    await expect(page.getByText('10% discount applied')).toBeVisible();
-  });
-
-  test('gift message formatting preserved @p3', async ({ page }) => {
-    // P3: Cosmetic feature (rarely used)
-    await page.goto('/checkout');
-    await page.getByTestId('gift-message').fill('Happy Birthday!\n\nWith love.');
-    await page.getByRole('button', { name: 'Place Order' }).click();
-
-    // Message formatting preserved (linebreaks intact)
-    await expect(page.getByTestId('order-summary')).toContainText('Happy Birthday!');
-  });
-});
-```
-
-**Run tests by priority:**
-
-```bash
-# P0 only (smoke tests, 2-5 min)
-npx playwright test --grep @p0
-
-# P0 + P1 (core functionality, 10-15 min)
-npx playwright test --grep "@p0|@p1"
-
-# Full regression (all priorities, 30+ min)
-npx playwright test
-```
-
----
-
-## Integration with Risk Scoring
-
-Priority should align with risk score from `probability-impact.md`:
-
-| Risk Score | Typical Priority | Rationale                                  |
-| ---------- | ---------------- | ------------------------------------------ |
-| 9          | P0               | Critical blocker (probability=3, impact=3) |
-| 6-8        | P0 or P1         | High risk (requires mitigation)            |
-| 4-5        | P1 or P2         | Medium risk (monitor closely)              |
-| 1-3        | P2 or P3         | Low risk (document and defer)              |
-
-**Example**: Risk score 9 (checkout API failure) → P0 priority → comprehensive coverage required.
-
----
-
-## Priority Checklist
-
-Before finalizing test priorities:
-
-- [ ] **Revenue impact assessed**: Payment, subscription, billing features → P0
-- [ ] **Security risks identified**: Auth, data exposure, injection attacks → P0
-- [ ] **Compliance requirements documented**: GDPR, PCI-DSS, SOC2 → P0
-- [ ] **User impact quantified**: >50% users → P0/P1, <10% → P2/P3
-- [ ] **Previous failures reviewed**: Regression prevention → increase priority
-- [ ] **Complexity evaluated**: >500 LOC or multiple dependencies → increase priority
-- [ ] **Usage metrics consulted**: Frequent use → P0/P1, rare use → P2/P3
-- [ ] **Monitoring coverage confirmed**: Strong monitoring → can decrease priority
-- [ ] **Rollback capability verified**: Easy rollback → can decrease priority
-- [ ] **Priorities tagged in tests**: @p0, @p1, @p2, @p3 for selective execution
-
-## Integration Points
-
-- **Used in workflows**: `*automate` (priority-based test generation), `*test-design` (scenario prioritization), `*trace` (coverage validation by priority)
-- **Related fragments**: `risk-governance.md` (risk scoring), `probability-impact.md` (impact assessment), `selective-testing.md` (tag-based execution)
-- **Tools**: Playwright/Cypress grep for tag filtering, CI scripts for priority-based execution
-
-_Source: Risk-based testing practices, test prioritization strategies, production incident analysis_