bmad-method 6.0.0-alpha.3 → 6.0.0-alpha.5

package/bmad/bmm/workflows/testarch/automate/instructions.md

@@ -0,0 +1,1303 @@
+<!-- Powered by BMAD-CORE™ -->
+
+# Test Automation Expansion
+
+**Workflow ID**: `bmad/bmm/testarch/automate`
+**Version**: 4.0 (BMad v6)
+
+---
+
+## Overview
+
+Expands test automation coverage by generating comprehensive test suites at appropriate levels (E2E, API, Component, Unit) with supporting infrastructure. This workflow operates in **dual mode**:
+
+1. **BMad-Integrated Mode**: Works WITH BMad artifacts (story, tech-spec, PRD, test-design) to expand coverage after story implementation
+2. **Standalone Mode**: Works WITHOUT BMad artifacts - analyzes existing codebase and generates tests independently
+
+**Core Principle**: Generate prioritized, deterministic tests that avoid duplicate coverage and follow testing best practices.
+
+---
+
+## Preflight Requirements
+
+**Flexible:** This workflow can run with minimal prerequisites. Only HALT if framework is completely missing.
+
+### Required (Always)
+
+- ✅ Framework scaffolding configured (run `framework` workflow if missing)
+- ✅ Test framework configuration available (playwright.config.ts or cypress.config.ts)
+
+### Optional (BMad-Integrated Mode)
+
+- Story markdown with acceptance criteria (enhances coverage targeting)
+- Tech spec or PRD (provides architectural context)
+- Test design document (provides risk/priority context)
+
+### Optional (Standalone Mode)
+
+- Source code to analyze (feature implementation)
+- Existing tests (for gap analysis)
+
+**If framework is missing:** HALT with message: "Framework scaffolding required. Run `bmad tea *framework` first."
+
+---
+
+## Step 1: Determine Execution Mode and Load Context
+
+### Actions
+
+1. **Detect Execution Mode**
+
+   Check if BMad artifacts are available:
+   - If `{story_file}` variable is set → BMad-Integrated Mode
+   - If `{target_feature}` or `{target_files}` set → Standalone Mode
+   - If neither set → Auto-discover mode (scan codebase for features needing tests)
+
+2. **Load BMad Artifacts (If Available)**
+
+   **BMad-Integrated Mode:**
+   - Read story markdown from `{story_file}`
+   - Extract acceptance criteria and technical requirements
+   - Load tech-spec.md if `{use_tech_spec}` is true
+   - Load test-design.md if `{use_test_design}` is true
+   - Load PRD.md if `{use_prd}` is true
+   - Note: These are **optional enhancements**, not hard requirements
+
+   **Standalone Mode:**
+   - Skip BMad artifact loading
+   - Proceed directly to source code analysis
+
+3. **Load Framework Configuration**
+   - Read test framework config (playwright.config.ts or cypress.config.ts)
+   - Identify test directory structure from `{test_dir}`
+   - Check existing test patterns in `{test_dir}`
+   - Note test runner capabilities (parallel execution, fixtures, etc.)
+
+4. **Analyze Existing Test Coverage**
+
+   If `{analyze_coverage}` is true:
+   - Search `{test_dir}` for existing test files
+   - Identify tested features vs untested features
+   - Map tests to source files (coverage gaps)
+   - Check existing fixture and factory patterns
+
+5. **Load Knowledge Base Fragments**
+
+   **Critical:** Consult `{project-root}/bmad/bmm/testarch/tea-index.csv` to load:
+   - `test-levels-framework.md` - Test level selection (E2E vs API vs Component vs Unit with decision matrix, 467 lines, 4 examples)
+   - `test-priorities-matrix.md` - Priority classification (P0-P3 with automated scoring, risk mapping, 389 lines, 2 examples)
+   - `fixture-architecture.md` - Test fixture patterns (pure function → fixture → mergeTests, auto-cleanup, 406 lines, 5 examples)
+   - `data-factories.md` - Factory patterns with faker (overrides, nested factories, API seeding, 498 lines, 5 examples)
+   - `selective-testing.md` - Targeted test execution strategies (tag-based, spec filters, diff-based, promotion rules, 727 lines, 4 examples)
+   - `ci-burn-in.md` - Flaky test detection patterns (10-iteration burn-in, sharding, selective execution, 678 lines, 4 examples)
+   - `test-quality.md` - Test design principles (deterministic, isolated, explicit assertions, length/time limits, 658 lines, 5 examples)
+   - `network-first.md` - Route interception patterns (intercept before navigate, HAR capture, deterministic waiting, 489 lines, 5 examples)
+
+   **Healing Knowledge (If `{auto_heal_failures}` is true):**
+   - `test-healing-patterns.md` - Common failure patterns and automated fixes (stale selectors, race conditions, dynamic data, network errors, hard waits, 648 lines, 5 examples)
+   - `selector-resilience.md` - Selector debugging and refactoring guide (data-testid > ARIA > text > CSS hierarchy, anti-patterns, 541 lines, 4 examples)
+   - `timing-debugging.md` - Race condition identification and fixes (network-first, deterministic waiting, async debugging, 370 lines, 3 examples)
+
+---
+
+## Step 2: Identify Automation Targets
+
+### Actions
+
+1. **Determine What Needs Testing**
+
+   **BMad-Integrated Mode (story available):**
+   - Map acceptance criteria from story to test scenarios
+   - Identify features implemented in this story
+   - Check if story has existing ATDD tests (from `*atdd` workflow)
+   - Expand beyond ATDD with edge cases and negative paths
+
+   **Standalone Mode (no story):**
+   - If `{target_feature}` specified: Analyze that specific feature
+   - If `{target_files}` specified: Analyze those specific files
+   - If `{auto_discover_features}` is true: Scan `{source_dir}` for features
+   - Prioritize features with:
+     - No test coverage (highest priority)
+     - Complex business logic
+     - External integrations (API calls, database, auth)
+     - Critical user paths (login, checkout, etc.)
+
+2. **Apply Test Level Selection Framework**
+
+   **Knowledge Base Reference**: `test-levels-framework.md`
+
+   For each feature or acceptance criterion, determine appropriate test level:
+
+   **E2E (End-to-End)**:
+   - Critical user journeys (login, checkout, core workflows)
+   - Multi-system integration
+   - Full user-facing scenarios
+   - Characteristics: High confidence, slow, brittle
+
+   **API (Integration)**:
+   - Business logic validation
+   - Service contracts and data transformations
+   - Backend integration without UI
+   - Characteristics: Fast feedback, stable, good balance
+
+   **Component**:
+   - UI component behavior (buttons, forms, modals)
+   - Interaction testing (click, hover, keyboard)
+   - State management within component
+   - Characteristics: Fast, isolated, granular
+
+   **Unit**:
+   - Pure business logic and algorithms
+   - Edge cases and error handling
+   - Minimal dependencies
+   - Characteristics: Fastest, most granular
+
+3. **Avoid Duplicate Coverage**
+
+   **Critical principle:** Don't test same behavior at multiple levels unless necessary
+   - Use E2E for critical happy path only
+   - Use API tests for business logic variations
+   - Use component tests for UI interaction edge cases
+   - Use unit tests for pure logic edge cases
+
+   **Example:**
+   - E2E: User can log in with valid credentials → Dashboard loads
+   - API: POST /auth/login returns 401 for invalid credentials
+   - API: POST /auth/login returns 200 and JWT token for valid credentials
+   - Component: LoginForm disables submit button when fields are empty
+   - Unit: validateEmail() returns false for malformed email addresses
+
+4. **Assign Test Priorities**
+
+   **Knowledge Base Reference**: `test-priorities-matrix.md`
+
+   **P0 (Critical - Every commit)**:
+   - Critical user paths that must always work
+   - Security-critical functionality (auth, permissions)
+   - Data integrity scenarios
+   - Run in pre-commit hooks or PR checks
+
+   **P1 (High - PR to main)**:
+   - Important features with high user impact
+   - Integration points between systems
+   - Error handling for common failures
+   - Run before merging to main branch
+
+   **P2 (Medium - Nightly)**:
+   - Edge cases with moderate impact
+   - Less-critical feature variations
+   - Performance/load testing
+   - Run in nightly CI builds
+
+   **P3 (Low - On-demand)**:
+   - Nice-to-have validations
+   - Rarely-used features
+   - Exploratory testing scenarios
+   - Run manually or weekly
+
+   **Priority Variables:**
+   - `{include_p0}` - Always include (default: true)
+   - `{include_p1}` - High priority (default: true)
+   - `{include_p2}` - Medium priority (default: true)
+   - `{include_p3}` - Low priority (default: false)
+
+5. **Create Test Coverage Plan**
+
+   Document what will be tested at each level with priorities:
+
+   ```markdown
+   ## Test Coverage Plan
+
+   ### E2E Tests (P0)
+
+   - User login with valid credentials → Dashboard loads
+   - User logout → Redirects to login page
+
+   ### API Tests (P1)
+
+   - POST /auth/login - valid credentials → 200 + JWT token
+   - POST /auth/login - invalid credentials → 401 + error message
+   - POST /auth/login - missing fields → 400 + validation errors
+
+   ### Component Tests (P1)
+
+   - LoginForm - empty fields → submit button disabled
+   - LoginForm - valid input → submit button enabled
+
+   ### Unit Tests (P2)
+
+   - validateEmail() - valid email → returns true
+   - validateEmail() - malformed email → returns false
+   ```
+
+---
+
+## Step 3: Generate Test Infrastructure
+
+### Actions
+
+1. **Enhance Fixture Architecture**
+
+   **Knowledge Base Reference**: `fixture-architecture.md`
+
+   Check existing fixtures in `tests/support/fixtures/`:
+   - If missing or incomplete, create fixture architecture
+   - Use Playwright's `test.extend()` pattern
+   - Ensure all fixtures have auto-cleanup in teardown
+
+   **Common fixtures to create/enhance:**
+   - **authenticatedUser**: User with valid session (auto-deletes user after test)
+   - **apiRequest**: Authenticated API client with base URL and headers
+   - **mockNetwork**: Network mocking for external services
+   - **testDatabase**: Database with test data (auto-cleanup after test)
+
+   **Example fixture:**
+
+   ```typescript
+   // tests/support/fixtures/auth.fixture.ts
+   import { test as base } from '@playwright/test';
+   import { createUser, deleteUser } from '../factories/user.factory';
+
+   export const test = base.extend({
+     authenticatedUser: async ({ page }, use) => {
+       // Setup: Create and authenticate user
+       const user = await createUser();
+       await page.goto('/login');
+       await page.fill('[data-testid="email"]', user.email);
+       await page.fill('[data-testid="password"]', user.password);
+       await page.click('[data-testid="login-button"]');
+       await page.waitForURL('/dashboard');
+
+       // Provide to test
+       await use(user);
+
+       // Cleanup: Delete user automatically
+       await deleteUser(user.id);
+     },
+   });
+   ```
+
+2. **Enhance Data Factories**
+
+   **Knowledge Base Reference**: `data-factories.md`
+
+   Check existing factories in `tests/support/factories/`:
+   - If missing or incomplete, create factory architecture
+   - Use `@faker-js/faker` for all random data (no hardcoded values)
+   - Support overrides for specific test scenarios
+
+   **Common factories to create/enhance:**
+   - User factory (email, password, name, role)
+   - Product factory (name, price, description, SKU)
+   - Order factory (items, total, status, customer)
+
+   **Example factory:**
+
+   ```typescript
+   // tests/support/factories/user.factory.ts
+   import { faker } from '@faker-js/faker';
+
+   export const createUser = (overrides = {}) => ({
+     id: faker.number.int(),
+     email: faker.internet.email(),
+     password: faker.internet.password(),
+     name: faker.person.fullName(),
+     role: 'user',
+     createdAt: faker.date.recent().toISOString(),
+     ...overrides,
+   });
+
+   export const createUsers = (count: number) => Array.from({ length: count }, () => createUser());
+
+   // API helper for cleanup
+   export const deleteUser = async (userId: number) => {
+     await fetch(`/api/users/${userId}`, { method: 'DELETE' });
+   };
+   ```
+
+3. **Create/Enhance Helper Utilities**
+
+   If `{update_helpers}` is true:
+
+   Check `tests/support/helpers/` for common utilities:
+   - **waitFor**: Polling helper for complex conditions
+   - **retry**: Retry helper for flaky operations
+   - **testData**: Test data generation helpers
+   - **assertions**: Custom assertion helpers
+
+   **Example helper:**
+
+   ```typescript
+   // tests/support/helpers/wait-for.ts
+   export const waitFor = async (condition: () => Promise<boolean>, timeout = 5000, interval = 100): Promise<void> => {
+     const startTime = Date.now();
+     while (Date.now() - startTime < timeout) {
+       if (await condition()) return;
+       await new Promise((resolve) => setTimeout(resolve, interval));
+     }
+     throw new Error(`Condition not met within ${timeout}ms`);
+   };
+   ```
+
+---
+
+## Step 4: Generate Test Files
+
+### Actions
+
+1. **Create Test File Structure**
+
+   ```
+   tests/
+   ├── e2e/
+   │   └── {feature-name}.spec.ts        # E2E tests (P0-P1)
+   ├── api/
+   │   └── {feature-name}.api.spec.ts    # API tests (P1-P2)
+   ├── component/
+   │   └── {ComponentName}.test.tsx      # Component tests (P1-P2)
+   ├── unit/
+   │   └── {module-name}.test.ts         # Unit tests (P2-P3)
+   └── support/
+       ├── fixtures/                      # Test fixtures
+       ├── factories/                     # Data factories
+       └── helpers/                       # Utility functions
+   ```
+
+2. **Write E2E Tests (If Applicable)**
+
+   **Follow Given-When-Then format:**
+
+   ```typescript
+   import { test, expect } from '@playwright/test';
+
+   test.describe('User Authentication', () => {
+     test('[P0] should login with valid credentials and load dashboard', async ({ page }) => {
+       // GIVEN: User is on login page
+       await page.goto('/login');
+
+       // WHEN: User submits valid credentials
+       await page.fill('[data-testid="email-input"]', 'user@example.com');
+       await page.fill('[data-testid="password-input"]', 'Password123!');
+       await page.click('[data-testid="login-button"]');
+
+       // THEN: User is redirected to dashboard
+       await expect(page).toHaveURL('/dashboard');
+       await expect(page.locator('[data-testid="user-name"]')).toBeVisible();
+     });
+
+     test('[P1] should display error for invalid credentials', async ({ page }) => {
+       // GIVEN: User is on login page
+       await page.goto('/login');
+
+       // WHEN: User submits invalid credentials
+       await page.fill('[data-testid="email-input"]', 'invalid@example.com');
+       await page.fill('[data-testid="password-input"]', 'wrongpassword');
+       await page.click('[data-testid="login-button"]');
+
+       // THEN: Error message is displayed
+       await expect(page.locator('[data-testid="error-message"]')).toHaveText('Invalid email or password');
+     });
+   });
+   ```
+
+   **Critical patterns:**
+   - Tag tests with priority: `[P0]`, `[P1]`, `[P2]`, `[P3]` in test name
+   - One assertion per test (atomic tests)
+   - Explicit waits (no hard waits/sleeps)
+   - Network-first approach (route interception before navigation)
+   - data-testid selectors for stability
+   - Clear Given-When-Then structure
+
+3. **Write API Tests (If Applicable)**
+
+   ```typescript
+   import { test, expect } from '@playwright/test';
+
+   test.describe('User Authentication API', () => {
+     test('[P1] POST /api/auth/login - should return token for valid credentials', async ({ request }) => {
+       // GIVEN: Valid user credentials
+       const credentials = {
+         email: 'user@example.com',
+         password: 'Password123!',
+       };
+
+       // WHEN: Logging in via API
+       const response = await request.post('/api/auth/login', {
+         data: credentials,
+       });
+
+       // THEN: Returns 200 and JWT token
+       expect(response.status()).toBe(200);
+       const body = await response.json();
+       expect(body).toHaveProperty('token');
+       expect(body.token).toMatch(/^[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+$/); // JWT format
+     });
+
+     test('[P1] POST /api/auth/login - should return 401 for invalid credentials', async ({ request }) => {
+       // GIVEN: Invalid credentials
+       const credentials = {
+         email: 'invalid@example.com',
+         password: 'wrongpassword',
+       };
+
+       // WHEN: Attempting login
+       const response = await request.post('/api/auth/login', {
+         data: credentials,
+       });
+
+       // THEN: Returns 401 with error
+       expect(response.status()).toBe(401);
+       const body = await response.json();
+       expect(body).toMatchObject({
+         error: 'Invalid credentials',
+       });
+     });
+   });
+   ```
+
+4. **Write Component Tests (If Applicable)**
+
+   **Knowledge Base Reference**: `component-tdd.md`
+
+   ```typescript
+   import { test, expect } from '@playwright/experimental-ct-react';
+   import { LoginForm } from './LoginForm';
+
+   test.describe('LoginForm Component', () => {
+     test('[P1] should disable submit button when fields are empty', async ({ mount }) => {
+       // GIVEN: LoginForm is mounted
+       const component = await mount(<LoginForm />);
+
+       // WHEN: Form is initially rendered
+       const submitButton = component.locator('button[type="submit"]');
+
+       // THEN: Submit button is disabled
+       await expect(submitButton).toBeDisabled();
+     });
+
+     test('[P1] should enable submit button when fields are filled', async ({ mount }) => {
+       // GIVEN: LoginForm is mounted
+       const component = await mount(<LoginForm />);
+
+       // WHEN: User fills in email and password
+       await component.locator('[data-testid="email-input"]').fill('user@example.com');
+       await component.locator('[data-testid="password-input"]').fill('Password123!');
+
+       // THEN: Submit button is enabled
+       const submitButton = component.locator('button[type="submit"]');
+       await expect(submitButton).toBeEnabled();
+     });
+   });
+   ```
+
+5. **Write Unit Tests (If Applicable)**
+
+   ```typescript
+   import { validateEmail } from './validation';
+
+   describe('Email Validation', () => {
+     test('[P2] should return true for valid email', () => {
+       // GIVEN: Valid email address
+       const email = 'user@example.com';
+
+       // WHEN: Validating email
+       const result = validateEmail(email);
+
+       // THEN: Returns true
+       expect(result).toBe(true);
+     });
+
+     test('[P2] should return false for malformed email', () => {
+       // GIVEN: Malformed email addresses
+       const invalidEmails = ['notanemail', '@example.com', 'user@', 'user @example.com'];
+
+       // WHEN/THEN: Each should fail validation
+       invalidEmails.forEach((email) => {
+         expect(validateEmail(email)).toBe(false);
+       });
+     });
+   });
+   ```
+
+6. **Apply Network-First Pattern (E2E tests)**
+
+   **Knowledge Base Reference**: `network-first.md`
+
+   **Critical pattern to prevent race conditions:**
+
+   ```typescript
+   test('should load user dashboard after login', async ({ page }) => {
+     // CRITICAL: Intercept routes BEFORE navigation
+     await page.route('**/api/user', (route) =>
+       route.fulfill({
+         status: 200,
+         body: JSON.stringify({ id: 1, name: 'Test User' }),
+       }),
+     );
+
+     // NOW navigate
+     await page.goto('/dashboard');
+
+     await expect(page.locator('[data-testid="user-name"]')).toHaveText('Test User');
+   });
+   ```
+
+7. **Enforce Quality Standards**
+
+   **For every test:**
+   - ✅ Uses Given-When-Then format
+   - ✅ Has clear, descriptive name with priority tag
+   - ✅ One assertion per test (atomic)
+   - ✅ No hard waits or sleeps (use explicit waits)
+   - ✅ Self-cleaning (uses fixtures with auto-cleanup)
+   - ✅ Deterministic (no flaky patterns)
+   - ✅ Fast (under {max_test_duration} seconds)
+   - ✅ Lean (test file under {max_file_lines} lines)
+
+   **Forbidden patterns:**
+   - ❌ Hard waits: `await page.waitForTimeout(2000)`
+   - ❌ Conditional flow: `if (await element.isVisible()) { ... }`
+   - ❌ Try-catch for test logic (use for cleanup only)
+   - ❌ Hardcoded test data (use factories)
+   - ❌ Page objects (keep tests simple and direct)
+   - ❌ Shared state between tests
+
+---
+
+## Step 5: Execute, Validate & Heal Generated Tests (NEW - Phase 2.5)
+
+**Purpose**: Automatically validate generated tests and heal common failures before delivery
+
+### Actions
+
+1. **Validate Generated Tests**
+
+   Always validate (auto_validate is always true):
+   - Run generated tests to verify they work
+   - Continue with healing if config.tea_use_mcp_enhancements is true
+
+2. **Run Generated Tests**
+
+   Execute the full test suite that was just generated:
+
+   ```bash
+   npx playwright test {generated_test_files}
+   ```
+
+   Capture results:
+   - Total tests run
+   - Passing tests count
+   - Failing tests count
+   - Error messages and stack traces for failures
+
+3. **Evaluate Results**
+
+   **If ALL tests pass:**
+   - ✅ Generate report with success summary
+   - Proceed to Step 6 (Documentation and Scripts)
+
+   **If tests FAIL:**
+   - Check config.tea_use_mcp_enhancements setting
+   - If true: Enter healing loop (Step 5.4)
+   - If false: Document failures for manual review, proceed to Step 6
+
+4. **Healing Loop (If config.tea_use_mcp_enhancements is true)**
+
+   **Iteration limit**: 3 attempts per test (constant)
+
+   **For each failing test:**
+
+   **A. Load Healing Knowledge Fragments**
+
+   Consult `tea-index.csv` to load healing patterns:
+   - `test-healing-patterns.md` - Common failure patterns and fixes
+   - `selector-resilience.md` - Selector debugging and refactoring
+   - `timing-debugging.md` - Race condition identification and fixes
+
+   **B. Identify Failure Pattern**
+
+   Analyze error message and stack trace to classify failure type:
+
+   **Stale Selector Failure:**
+   - Error contains: "locator resolved to 0 elements", "element not found", "unable to find element"
+   - Extract selector from error message
+   - Apply selector healing (knowledge from `selector-resilience.md`):
+     - If CSS class → Replace with `page.getByTestId()`
+     - If nth() → Replace with `filter({ hasText })`
+     - If ID → Replace with data-testid
+     - If complex XPath → Replace with ARIA role
+
+   **Race Condition Failure:**
+   - Error contains: "timeout waiting for", "element not visible", "timed out retrying"
+   - Detect missing network waits or hard waits in test code
+   - Apply timing healing (knowledge from `timing-debugging.md`):
+     - Add network-first interception before navigate
+     - Replace `waitForTimeout()` with `waitForResponse()`
+     - Add explicit element state waits (`waitFor({ state: 'visible' })`)
+
+   **Dynamic Data Failure:**
+   - Error contains: "Expected 'User 123' but received 'User 456'", timestamp mismatches
+   - Identify hardcoded assertions
+   - Apply data healing (knowledge from `test-healing-patterns.md`):
+     - Replace hardcoded IDs with regex (`/User \d+/`)
+     - Replace hardcoded dates with dynamic generation
+     - Capture dynamic values and use in assertions
+
+   **Network Error Failure:**
+   - Error contains: "API call failed", "500 error", "network error"
+   - Detect missing route interception
+   - Apply network healing (knowledge from `test-healing-patterns.md`):
+     - Add `page.route()` or `cy.intercept()` for API mocking
+     - Mock error scenarios (500, 429, timeout)
+
+   **Hard Wait Detection:**
+   - Scan test code for `page.waitForTimeout()`, `cy.wait(number)`, `sleep()`
+   - Apply hard wait healing (knowledge from `timing-debugging.md`):
+     - Replace with event-based waits
+     - Add network response waits
+     - Use element state changes
+
+   **C. MCP Healing Mode (If MCP Tools Available)**
+
+   If Playwright MCP tools are available in your IDE:
+
+   Use MCP tools for interactive healing:
+   - `playwright_test_debug_test`: Pause on failure for visual inspection
+   - `browser_snapshot`: Capture visual context at failure point
+   - `browser_console_messages`: Retrieve console logs for JS errors
+   - `browser_network_requests`: Analyze network activity
+   - `browser_generate_locator`: Generate better selectors interactively
+
+   Apply MCP-generated fixes to test code.
+
+   **D. Pattern-Based Healing Mode (Fallback)**
+
+   If MCP unavailable, use pattern-based analysis:
+   - Parse error message and stack trace
+   - Match against failure patterns from knowledge base
+   - Apply fixes programmatically:
+     - Selector fixes: Use suggestions from `selector-resilience.md`
+     - Timing fixes: Apply patterns from `timing-debugging.md`
+     - Data fixes: Use patterns from `test-healing-patterns.md`
+
+   **E. Apply Healing Fix**
+   - Modify test file with healed code
+   - Re-run test to validate fix
+   - If test passes: Mark as healed, move to next failure
+   - If test fails: Increment iteration count, try different pattern
+
+   **F. Iteration Limit Handling**
+
+   After 3 failed healing attempts:
+
+   Always mark unfixable tests:
+   - Mark test with `test.fixme()` instead of `test()`
+   - Add detailed comment explaining:
+     - What failure occurred
+     - What healing was attempted (3 iterations)
+     - Why healing failed
+     - Manual investigation needed
+
+   ```typescript
+   test.fixme('[P1] should handle complex interaction', async ({ page }) => {
+     // FIXME: Test healing failed after 3 attempts
+     // Failure: "Locator 'button[data-action="submit"]' resolved to 0 elements"
+     // Attempted fixes:
+     //   1. Replaced with page.getByTestId('submit-button') - still failing
+     //   2. Replaced with page.getByRole('button', { name: 'Submit' }) - still failing
+     //   3. Added waitForLoadState('networkidle') - still failing
+     // Manual investigation needed: Selector may require application code changes
+     // TODO: Review with team, may need data-testid added to button component
+     // Original test code...
+   });
+   ```
+
+   **Note**: Workflow continues even with unfixable tests (marked as test.fixme() for manual review)
+
+5. **Generate Healing Report**
+
+   Document healing outcomes:
+
+   ```markdown
+   ## Test Healing Report
+
+   **Auto-Heal Enabled**: {auto_heal_failures}
+   **Healing Mode**: {use_mcp_healing ? "MCP-assisted" : "Pattern-based"}
+   **Iterations Allowed**: {max_healing_iterations}
+
+   ### Validation Results
+
+   - **Total tests**: {total_tests}
+   - **Passing**: {passing_tests}
+   - **Failing**: {failing_tests}
+
+   ### Healing Outcomes
+
+   **Successfully Healed ({healed_count} tests):**
+
+   - `tests/e2e/login.spec.ts:15` - Stale selector (CSS class → data-testid)
+   - `tests/e2e/checkout.spec.ts:42` - Race condition (added network-first interception)
+   - `tests/api/users.spec.ts:28` - Dynamic data (hardcoded ID → regex pattern)
+
+   **Unable to Heal ({unfixable_count} tests):**
+
+   - `tests/e2e/complex-flow.spec.ts:67` - Marked as test.fixme() with manual investigation needed
+     - Failure: Locator not found after 3 healing attempts
+     - Requires application code changes (add data-testid to component)
+
+   ### Healing Patterns Applied
+
+   - **Selector fixes**: 2 (CSS class → data-testid, nth() → filter())
+   - **Timing fixes**: 1 (added network-first interception)
+   - **Data fixes**: 1 (hardcoded ID → regex)
+
+   ### Knowledge Base References
+
+   - `test-healing-patterns.md` - Common failure patterns
+   - `selector-resilience.md` - Selector refactoring guide
+   - `timing-debugging.md` - Race condition prevention
+   ```
+
+6. **Update Test Files with Healing Results**
+   - Save healed test code to files
+   - Mark unfixable tests with `test.fixme()` and detailed comments
+   - Preserve original test logic in comments (for debugging)
+
+---
+
+## Step 6: Update Documentation and Scripts
+
+### Actions
+
+1. **Update Test README**
+
+   If `{update_readme}` is true:
+
+   Create or update `tests/README.md` with:
+   - Overview of test suite structure
+   - How to run tests (all, specific files, by priority)
+   - Fixture and factory usage examples
+   - Priority tagging convention ([P0], [P1], [P2], [P3])
+   - How to write new tests
+   - Common patterns and anti-patterns
+
+   **Example section:**
+
+   ````markdown
+   ## Running Tests
+
+   ```bash
+   # Run all tests
+   npm run test:e2e
+
+   # Run by priority
+   npm run test:e2e -- --grep "@P0"
+   npm run test:e2e -- --grep "@P1"
+
+   # Run specific file
+   npm run test:e2e -- user-authentication.spec.ts
+
+   # Run in headed mode
+   npm run test:e2e -- --headed
+
+   # Debug specific test
+   npm run test:e2e -- user-authentication.spec.ts --debug
+   ```
+   ````
+
+   ## Priority Tags
+   - **[P0]**: Critical paths, run every commit
+   - **[P1]**: High priority, run on PR to main
+   - **[P2]**: Medium priority, run nightly
+   - **[P3]**: Low priority, run on-demand
+
+   ```
+
+   ```
+
+2. **Update package.json Scripts**
+
+   If `{update_package_scripts}` is true:
+
+   Add or update test execution scripts:
+
+   ```json
+   {
+     "scripts": {
+       "test:e2e": "playwright test",
+       "test:e2e:p0": "playwright test --grep '@P0'",
+       "test:e2e:p1": "playwright test --grep '@P1|@P0'",
+       "test:api": "playwright test tests/api",
+       "test:component": "playwright test tests/component",
+       "test:unit": "vitest"
+     }
+   }
+   ```
+
+3. **Run Test Suite**
+
+   If `{run_tests_after_generation}` is true:
+   - Run full test suite locally
+   - Capture results (passing/failing counts)
+   - Verify no flaky patterns (tests should be deterministic)
+   - Document any setup requirements or known issues
+
+---
+
+## Step 6: Generate Automation Summary
+
+### Actions
+
+1. **Create Automation Summary Document**
+
+   Save to `{output_summary}` with:
+
+   **BMad-Integrated Mode:**
+
+   ````markdown
+   # Automation Summary - {feature_name}
+
+   **Date:** {date}
+   **Story:** {story_id}
+   **Coverage Target:** {coverage_target}
+
+   ## Tests Created
+
+   ### E2E Tests (P0-P1)
+
+   - `tests/e2e/user-authentication.spec.ts` (2 tests, 87 lines)
+     - [P0] Login with valid credentials → Dashboard loads
+     - [P1] Display error for invalid credentials
+
+   ### API Tests (P1-P2)
+
+   - `tests/api/auth.api.spec.ts` (3 tests, 102 lines)
+     - [P1] POST /auth/login - valid credentials → 200 + token
+     - [P1] POST /auth/login - invalid credentials → 401 + error
+     - [P2] POST /auth/login - missing fields → 400 + validation
+
+   ### Component Tests (P1)
+
+   - `tests/component/LoginForm.test.tsx` (2 tests, 45 lines)
+     - [P1] Empty fields → submit button disabled
+     - [P1] Valid input → submit button enabled
+
+   ## Infrastructure Created
+
+   ### Fixtures
+
+   - `tests/support/fixtures/auth.fixture.ts` - authenticatedUser with auto-cleanup
+
+   ### Factories
+
+   - `tests/support/factories/user.factory.ts` - createUser(), deleteUser()
+
+   ### Helpers
+
+   - `tests/support/helpers/wait-for.ts` - Polling helper for complex conditions
+
+   ## Test Execution
+
+   ```bash
+   # Run all new tests
+   npm run test:e2e
+
+   # Run by priority
+   npm run test:e2e:p0  # Critical paths only
+   npm run test:e2e:p1  # P0 + P1 tests
+   ```
+   ````
+
+   ## Coverage Analysis
+
+   **Total Tests:** 7
+   - P0: 1 test (critical path)
+   - P1: 5 tests (high priority)
+   - P2: 1 test (medium priority)
+
+   **Test Levels:**
+   - E2E: 2 tests (user journeys)
+   - API: 3 tests (business logic)
+   - Component: 2 tests (UI behavior)
+
+   **Coverage Status:**
+   - ✅ All acceptance criteria covered
+   - ✅ Happy path covered (E2E + API)
+   - ✅ Error cases covered (API)
+   - ✅ UI validation covered (Component)
+   - ⚠️ Edge case: Password reset flow not yet covered (future story)
+
+   ## Definition of Done
+   - [x] All tests follow Given-When-Then format
+   - [x] All tests use data-testid selectors
+   - [x] All tests have priority tags
+   - [x] All tests are self-cleaning (fixtures with auto-cleanup)
+   - [x] No hard waits or flaky patterns
+   - [x] Test files under 300 lines
+   - [x] All tests run under 1.5 minutes each
+   - [x] README updated with test execution instructions
+   - [x] package.json scripts updated
+
+   ## Next Steps
+   1. Review generated tests with team
+   2. Run tests in CI pipeline: `npm run test:e2e`
+   3. Integrate with quality gate: `bmad tea *gate`
+   4. Monitor for flaky tests in burn-in loop
+
+   ````
+
+   **Standalone Mode:**
+   ```markdown
+   # Automation Summary - {target_feature}
+
+   **Date:** {date}
+   **Target:** {target_feature} (standalone analysis)
+   **Coverage Target:** {coverage_target}
+
+   ## Feature Analysis
+
+   **Source Files Analyzed:**
+   - `src/auth/login.ts` - Login logic and validation
+   - `src/auth/session.ts` - Session management
+   - `src/auth/validation.ts` - Email/password validation
+
+   **Existing Coverage:**
+   - E2E tests: 0 found
+   - API tests: 0 found
+   - Component tests: 0 found
+   - Unit tests: 0 found
+
+   **Coverage Gaps Identified:**
+   - ❌ No E2E tests for login flow
+   - ❌ No API tests for /auth/login endpoint
+   - ❌ No component tests for LoginForm
+   - ❌ No unit tests for validateEmail()
+
+   ## Tests Created
+
+   {Same structure as BMad-Integrated Mode}
+
+   ## Recommendations
+
+   1. **High Priority (P0-P1):**
+      - Add E2E test for password reset flow
+      - Add API tests for token refresh endpoint
+      - Add component tests for logout button
+
+   2. **Medium Priority (P2):**
+      - Add unit tests for session timeout logic
+      - Add E2E test for "remember me" functionality
+
+   3. **Future Enhancements:**
+      - Consider contract testing for auth API
+      - Add visual regression tests for login page
+      - Set up burn-in loop for flaky test detection
+
+   ## Definition of Done
+
+   {Same checklist as BMad-Integrated Mode}
+   ````
+
+2. **Provide Summary to User**
+
+   Output concise summary:
+
+   ```markdown
+   ## Automation Complete
+
+   **Coverage:** {total_tests} tests created across {test_levels} levels
+   **Priority Breakdown:** P0: {p0_count}, P1: {p1_count}, P2: {p2_count}, P3: {p3_count}
+   **Infrastructure:** {fixture_count} fixtures, {factory_count} factories
+   **Output:** {output_summary}
+
+   **Run tests:** `npm run test:e2e`
+   **Next steps:** Review tests, run in CI, integrate with quality gate
+   ```
+
+---
+
+## Important Notes
+
+### Dual-Mode Operation
+
+**BMad-Integrated Mode** (story available):
+
+- Uses story acceptance criteria for coverage targeting
+- Aligns with test-design risk/priority assessment
+- Expands ATDD tests with edge cases and negative paths
+- Updates BMad status tracking
+
+**Standalone Mode** (no story):
+
+- Analyzes source code independently
+- Identifies coverage gaps automatically
+- Generates tests based on code analysis
+- Works with any project (BMad or non-BMad)
+
+**Auto-discover Mode** (no targets specified):
+
+- Scans codebase for features needing tests
+- Prioritizes features with no coverage
+- Generates comprehensive test plan
+
+### Avoid Duplicate Coverage
+
+**Critical principle:** Don't test same behavior at multiple levels
+
+**Good coverage:**
+
+- E2E: User can login → Dashboard loads (critical happy path)
+- API: POST /auth/login returns correct status codes (variations)
+- Component: LoginForm validates input (UI edge cases)
+
+**Bad coverage (duplicate):**
+
+- E2E: User can login → Dashboard loads
+- E2E: User can login with different emails → Dashboard loads (unnecessary duplication)
+- API: POST /auth/login returns 200 (already covered in E2E)
+
+Use E2E sparingly for critical paths. Use API/Component for variations and edge cases.
+
+### Priority Tagging
+
+**Tag every test with priority in test name:**
+
+```typescript
+test('[P0] should login with valid credentials', async ({ page }) => { ... });
+test('[P1] should display error for invalid credentials', async ({ page }) => { ... });
+test('[P2] should remember login preference', async ({ page }) => { ... });
+```
+
+**Enables selective test execution:**
+
+```bash
+# Run only P0 tests (critical paths)
+npm run test:e2e -- --grep "@P0"
+
+# Run P0 + P1 tests (pre-merge)
+npm run test:e2e -- --grep "@P0|@P1"
+```
+
+### No Page Objects
+
+**Do NOT create page object classes.** Keep tests simple and direct:
+
+```typescript
+// ✅ CORRECT: Direct test
+test('should login', async ({ page }) => {
+  await page.goto('/login');
+  await page.fill('[data-testid="email"]', 'user@example.com');
+  await page.click('[data-testid="login-button"]');
+  await expect(page).toHaveURL('/dashboard');
+});
+
+// ❌ WRONG: Page object abstraction
+class LoginPage {
+  async login(email, password) { ... }
+}
+```
+
+Use fixtures for setup/teardown, not page objects for actions.
+
+### Deterministic Tests Only
+
+**No flaky patterns allowed:**
+
+```typescript
+// ❌ WRONG: Hard wait
+await page.waitForTimeout(2000);
+
+// ✅ CORRECT: Explicit wait
+await page.waitForSelector('[data-testid="user-name"]');
+await expect(page.locator('[data-testid="user-name"]')).toBeVisible();
+
+// ❌ WRONG: Conditional flow
+if (await element.isVisible()) {
+  await element.click();
+}
+
+// ✅ CORRECT: Deterministic assertion
+await expect(element).toBeVisible();
+await element.click();
+
+// ❌ WRONG: Try-catch for test logic
+try {
+  await element.click();
+} catch (e) {
+  // Test shouldn't catch errors
+}
+
+// ✅ CORRECT: Let test fail if element not found
+await element.click();
+```
+
+### Self-Cleaning Tests
+
+**Every test must clean up its data:**
+
+```typescript
+// ✅ CORRECT: Fixture with auto-cleanup
+export const test = base.extend({
+  testUser: async ({ page }, use) => {
+    const user = await createUser();
+    await use(user);
+    await deleteUser(user.id); // Auto-cleanup
+  },
+});
+
+// ❌ WRONG: Manual cleanup (can be forgotten)
+test('should login', async ({ page }) => {
+  const user = await createUser();
+  // ... test logic ...
+  // Forgot to delete user!
+});
+```
+
+### File Size Limits
+
+**Keep test files lean (under {max_file_lines} lines):**
+
+- If file exceeds limit, split into multiple files by feature area
+- Group related tests in describe blocks
+- Extract common setup to fixtures
+
+### Knowledge Base Integration
+
+**Core Fragments (Auto-loaded in Step 1):**
+
+- `test-levels-framework.md` - E2E vs API vs Component vs Unit decision framework with characteristics matrix (467 lines, 4 examples)
+- `test-priorities-matrix.md` - P0-P3 classification with automated scoring and risk mapping (389 lines, 2 examples)
+- `fixture-architecture.md` - Pure function → fixture → mergeTests composition with auto-cleanup (406 lines, 5 examples)
+- `data-factories.md` - Factory patterns with faker: overrides, nested factories, API seeding (498 lines, 5 examples)
+- `selective-testing.md` - Tag-based, spec filters, diff-based selection, promotion rules (727 lines, 4 examples)
+- `ci-burn-in.md` - 10-iteration burn-in loop, parallel sharding, selective execution (678 lines, 4 examples)
+- `test-quality.md` - Deterministic tests, isolated with cleanup, explicit assertions, length/time optimization (658 lines, 5 examples)
+- `network-first.md` - Intercept before navigate, HAR capture, deterministic waiting strategies (489 lines, 5 examples)
+
+**Healing Fragments (Auto-loaded if `{auto_heal_failures}` enabled):**
+
+- `test-healing-patterns.md` - Common failure patterns: stale selectors, race conditions, dynamic data, network errors, hard waits (648 lines, 5 examples)
+- `selector-resilience.md` - Selector hierarchy (data-testid > ARIA > text > CSS), dynamic patterns, anti-patterns refactoring (541 lines, 4 examples)
+- `timing-debugging.md` - Race condition prevention, deterministic waiting, async debugging techniques (370 lines, 3 examples)
+
+**Manual Reference (Optional):**
+
+- Use `tea-index.csv` to find additional specialized fragments as needed
+
+---
+
+## Output Summary
+
+After completing this workflow, provide a summary:
+
+````markdown
+## Automation Complete
+
+**Mode:** {standalone_mode ? "Standalone" : "BMad-Integrated"}
+**Target:** {story_id || target_feature || "Auto-discovered features"}
+
+**Tests Created:**
+
+- E2E: {e2e_count} tests ({p0_count} P0, {p1_count} P1, {p2_count} P2)
+- API: {api_count} tests ({p0_count} P0, {p1_count} P1, {p2_count} P2)
+- Component: {component_count} tests ({p1_count} P1, {p2_count} P2)
+- Unit: {unit_count} tests ({p2_count} P2, {p3_count} P3)
+
+**Infrastructure:**
+
+- Fixtures: {fixture_count} created/enhanced
+- Factories: {factory_count} created/enhanced
+- Helpers: {helper_count} created/enhanced
+
+**Documentation Updated:**
+
+- ✅ Test README with execution instructions
+- ✅ package.json scripts for test execution
+
+**Test Execution:**
+
+```bash
+# Run all tests
+npm run test:e2e
+
+# Run by priority
+npm run test:e2e:p0  # Critical paths only
+npm run test:e2e:p1  # P0 + P1 tests
+
+# Run specific file
+npm run test:e2e -- {first_test_file}
+```
+````
+
+**Coverage Status:**
+
+- ✅ {coverage_percentage}% of features covered
+- ✅ All P0 scenarios covered
+- ✅ All P1 scenarios covered
+- ⚠️ {gap_count} coverage gaps identified (documented in summary)
+
+**Quality Checks:**
+
+- ✅ All tests follow Given-When-Then format
+- ✅ All tests have priority tags
+- ✅ All tests use data-testid selectors
+- ✅ All tests are self-cleaning
+- ✅ No hard waits or flaky patterns
+- ✅ All test files under {max_file_lines} lines
+
+**Output File:** {output_summary}
+
+**Next Steps:**
+
+1. Review generated tests with team
+2. Run tests in CI pipeline
+3. Monitor for flaky tests in burn-in loop
+4. Integrate with quality gate: `bmad tea *gate`
+
+**Knowledge Base References Applied:**
+
+- Test level selection framework (E2E vs API vs Component vs Unit)
+- Priority classification (P0-P3)
+- Fixture architecture patterns with auto-cleanup
+- Data factory patterns using faker
+- Selective testing strategies
+- Test quality principles
+
+```
+
+---
+
+## Validation
+
+After completing all steps, verify:
+
+- [ ] Execution mode determined (BMad-Integrated, Standalone, or Auto-discover)
+- [ ] BMad artifacts loaded if available (story, tech-spec, test-design, PRD)
+- [ ] Framework configuration loaded
+- [ ] Existing test coverage analyzed (gaps identified)
+- [ ] Knowledge base fragments loaded (test-levels, test-priorities, fixture-architecture, data-factories, selective-testing)
+- [ ] Automation targets identified (what needs testing)
+- [ ] Test levels selected appropriately (E2E, API, Component, Unit)
+- [ ] Duplicate coverage avoided (same behavior not tested at multiple levels)
+- [ ] Test priorities assigned (P0, P1, P2, P3)
+- [ ] Fixture architecture created/enhanced (with auto-cleanup)
+- [ ] Data factories created/enhanced (using faker)
+- [ ] Helper utilities created/enhanced (if needed)
+- [ ] E2E tests written (Given-When-Then, priority tags, data-testid selectors)
+- [ ] API tests written (Given-When-Then, priority tags, comprehensive coverage)
+- [ ] Component tests written (Given-When-Then, priority tags, UI behavior)
+- [ ] Unit tests written (Given-When-Then, priority tags, pure logic)
+- [ ] Network-first pattern applied (route interception before navigation)
+- [ ] Quality standards enforced (no hard waits, no flaky patterns, self-cleaning, deterministic)
+- [ ] Test README updated (execution instructions, priority tagging, patterns)
+- [ ] package.json scripts updated (test execution commands)
+- [ ] Test suite run locally (results captured)
+- [ ] Tests validated (if auto_validate enabled)
+- [ ] Failures healed (if auto_heal_failures enabled)
+- [ ] Healing report generated (if healing attempted)
+- [ ] Unfixable tests marked with test.fixme() (if any)
+- [ ] Automation summary created (tests, infrastructure, coverage, healing, DoD)
+- [ ] Output file formatted correctly
+
+Refer to `checklist.md` for comprehensive validation criteria.
+```