bmad-method 6.0.0-alpha.0 → 6.0.0-alpha.2

package/src/modules/bmm/workflows/testarch/test-design/README.md

@@ -0,0 +1,493 @@
+# Test Design and Risk Assessment Workflow
+
+Plans comprehensive test coverage strategy with risk assessment (probability × impact scoring), priority classification (P0-P3), and resource estimation. This workflow generates a test design document that identifies high-risk areas, maps requirements to appropriate test levels, and provides execution ordering for optimal feedback.
+
+## Usage
+
+```bash
+bmad tea *test-design
+```
+
+The TEA agent runs this workflow when:
+
+- Planning test coverage before development starts
+- Assessing risks for an epic or story
+- Prioritizing test scenarios by business impact
+- Estimating testing effort and resources
+
+## Inputs
+
+**Required Context Files:**
+
+- **Story markdown**: Acceptance criteria and requirements
+- **PRD or epics.md**: High-level product context
+- **Architecture docs** (optional): Technical constraints and integration points
+
+**Workflow Variables:**
+
+- `epic_num`: Epic number for scoped design
+- `story_path`: Specific story for design (optional)
+- `design_level`: full/targeted/minimal (default: full)
+- `risk_threshold`: Score for high-priority flag (default: 6)
+- `risk_categories`: TECH,SEC,PERF,DATA,BUS,OPS (all enabled)
+- `priority_levels`: P0,P1,P2,P3 (all enabled)
+
+## Outputs
+
+**Primary Deliverable:**
+
+**Test Design Document** (`test-design-epic-{N}.md`):
+
+1. **Risk Assessment Matrix**
+   - Risk ID, category, description
+   - Probability (1-3) × Impact (1-3) = Score
+   - Scores ≥6 flagged as high-priority
+   - Mitigation plans with owners and timelines
+
+2. **Coverage Matrix**
+   - Requirement → Test Level (E2E/API/Component/Unit)
+   - Priority assignment (P0-P3)
+   - Risk linkage
+   - Test count estimates
+
+3. **Execution Order**
+   - Smoke tests (P0 subset, <5 min)
+   - P0 tests (critical paths, <10 min)
+   - P1 tests (important features, <30 min)
+   - P2/P3 tests (full regression, <60 min)
+
+4. **Resource Estimates**
+   - Hours per priority level
+   - Total effort in days
+   - Tooling and data prerequisites
+
+5. **Quality Gate Criteria**
+   - P0 pass rate: 100%
+   - P1 pass rate: ≥95%
+   - High-risk mitigations: 100%
+   - Coverage target: ≥80%
+
+## Key Features
+
+### Risk Scoring Framework
+
+**Probability × Impact = Risk Score**
+
+**Probability** (1-3):
+
+- 1 (Unlikely): <10% chance
+- 2 (Possible): 10-50% chance
+- 3 (Likely): >50% chance
+
+**Impact** (1-3):
+
+- 1 (Minor): Cosmetic, workaround exists
+- 2 (Degraded): Feature impaired, difficult workaround
+- 3 (Critical): System failure, no workaround
+
+**Scores**:
+
+- 1-2: Low risk (monitor)
+- 3-4: Medium risk (plan mitigation)
+- **6-9: High risk** (immediate mitigation required)
+
+### Risk Categories (6 types)
+
+**TECH** (Technical/Architecture):
+
+- Architecture flaws, integration failures
+- Scalability issues, technical debt
+
+**SEC** (Security):
+
+- Missing access controls, auth bypass
+- Data exposure, injection vulnerabilities
+
+**PERF** (Performance):
+
+- SLA violations, response time degradation
+- Resource exhaustion, scalability limits
+
+**DATA** (Data Integrity):
+
+- Data loss/corruption, inconsistent state
+- Migration failures
+
+**BUS** (Business Impact):
+
+- UX degradation, business logic errors
+- Revenue impact, compliance violations
+
+**OPS** (Operations):
+
+- Deployment failures, configuration errors
+- Monitoring gaps, rollback issues
+
+### Priority Classification (P0-P3)
+
+**P0 (Critical)** - Run on every commit:
+
+- Blocks core user journey
+- High-risk (score ≥6)
+- Revenue-impacting or security-critical
+
+**P1 (High)** - Run on PR to main:
+
+- Important user features
+- Medium-risk (score 3-4)
+- Common workflows
+
+**P2 (Medium)** - Run nightly/weekly:
+
+- Secondary features
+- Low-risk (score 1-2)
+- Edge cases
+
+**P3 (Low)** - Run on-demand:
+
+- Nice-to-have, exploratory
+- Performance benchmarks
+
+### Test Level Selection
+
+**E2E (End-to-End)**:
+
+- Critical user journeys
+- Multi-system integration
+- Highest confidence, slowest
+
+**API (Integration)**:
+
+- Service contracts
+- Business logic validation
+- Fast feedback, stable
+
+**Component**:
+
+- UI component behavior
+- Visual regression
+- Fast, isolated
+
+**Unit**:
+
+- Business logic, edge cases
+- Error handling
+- Fastest, most granular
+
+**Key principle**: Avoid duplicate coverage - don't test same behavior at multiple levels.
+
+### Exploratory Mode (NEW - Phase 2.5)
+
+**test-design** supports UI exploration for brownfield applications with missing documentation.
+
+**Activation**: Automatic when requirements missing/incomplete for brownfield apps
+
+- If config.tea_use_mcp_enhancements is true + MCP available → MCP-assisted exploration
+- Otherwise → Manual exploration with user documentation
+
+**When to Use Exploratory Mode:**
+
+- ✅ Brownfield projects with missing documentation
+- ✅ Legacy systems lacking requirements
+- ✅ Undocumented features needing test coverage
+- ✅ Unknown user journeys requiring discovery
+- ❌ NOT for greenfield projects with clear requirements
+
+**Exploration Modes:**
+
+1. **MCP-Assisted Exploration** (if Playwright MCP available):
+   - Interactive browser exploration using MCP tools
+   - `planner_setup_page` - Initialize browser
+   - `browser_navigate` - Explore pages
+   - `browser_click` - Interact with UI elements
+   - `browser_hover` - Reveal hidden menus
+   - `browser_snapshot` - Capture state at each step
+   - `browser_screenshot` - Document visually
+   - `browser_console_messages` - Find JavaScript errors
+   - `browser_network_requests` - Identify API endpoints
+
+2. **Manual Exploration** (fallback without MCP):
+   - User explores application manually
+   - Documents findings in markdown:
+     - Pages/features discovered
+     - User journeys identified
+     - API endpoints observed (DevTools Network)
+     - JavaScript errors noted (DevTools Console)
+     - Critical workflows mapped
+   - Provides exploration findings to workflow
+
+**Exploration Workflow:**
+
+```
+1. Enable exploratory_mode and set exploration_url
+2. IF MCP available:
+   - Use planner_setup_page to init browser
+   - Explore UI with browser_* tools
+   - Capture snapshots and screenshots
+   - Monitor console and network
+   - Document discoveries
+3. IF MCP unavailable:
+   - Notify user to explore manually
+   - Wait for exploration findings
+4. Convert discoveries to testable requirements
+5. Continue with standard risk assessment (Step 2)
+```
+
+**Example Output from Exploratory Mode:**
+
+```markdown
+## Exploration Findings - Legacy Admin Panel
+
+**Exploration URL**: https://admin.example.com
+**Mode**: MCP-Assisted
+
+### Discovered Features:
+
+1. User Management (/admin/users)
+   - List users (table with 10 columns)
+   - Edit user (modal form)
+   - Delete user (confirmation dialog)
+   - Export to CSV (download button)
+
+2. Reporting Dashboard (/admin/reports)
+   - Date range picker
+   - Filter by department
+   - Generate PDF report
+   - Email report to stakeholders
+
+3. API Endpoints Discovered:
+   - GET /api/admin/users
+   - PUT /api/admin/users/:id
+   - DELETE /api/admin/users/:id
+   - POST /api/reports/generate
+
+### User Journeys Mapped:
+
+1. Admin deletes inactive user
+   - Navigate to /admin/users
+   - Click delete icon
+   - Confirm in modal
+   - User removed from table
+
+2. Admin generates monthly report
+   - Navigate to /admin/reports
+   - Select date range (last month)
+   - Click generate
+   - Download PDF
+
+### Risks Identified (from exploration):
+
+- R-001 (SEC): No RBAC check observed (any admin can delete any user)
+- R-002 (DATA): No confirmation on bulk delete
+- R-003 (PERF): User table loads slowly (5s for 1000 rows)
+
+**Next**: Proceed to risk assessment with discovered requirements
+```
+
+**Graceful Degradation:**
+
+- Exploratory mode is OPTIONAL (default: disabled)
+- Works without Playwright MCP (manual fallback)
+- If exploration fails, can disable mode and provide requirements documentation
+- Seamlessly transitions to standard risk assessment workflow
+
+### Knowledge Base Integration
+
+Automatically consults TEA knowledge base:
+
+- `risk-governance.md` - Risk classification framework
+- `probability-impact.md` - Risk scoring methodology
+- `test-levels-framework.md` - Test level selection
+- `test-priorities-matrix.md` - P0-P3 prioritization
+
+## Integration with Other Workflows
+
+**Before test-design:**
+
+- **prd** (Phase 2): Creates PRD and epics
+- **architecture** (Phase 3): Defines technical approach
+- **tech-spec** (Phase 3): Implementation details
+
+**After test-design:**
+
+- **atdd**: Generate failing tests for P0 scenarios
+- **automate**: Expand coverage for P1/P2 scenarios
+- **trace (Phase 2)**: Use quality gate criteria for release decisions
+
+**Coordinates with:**
+
+- **framework**: Test infrastructure must exist
+- **ci**: Execution order maps to CI stages
+
+**Updates:**
+
+- `bmm-workflow-status.md`: Adds test design to Quality & Testing Progress
+
+## Important Notes
+
+### Evidence-Based Assessment
+
+**Critical principle**: Base risk assessment on **evidence**, not speculation.
+
+**Evidence sources:**
+
+- PRD and user research
+- Architecture documentation
+- Historical bug data
+- User feedback
+- Security audit results
+
+**When uncertain**: Document assumptions, request user clarification.
+
+**Avoid**:
+
+- Guessing business impact
+- Assuming user behavior
+- Inventing requirements
+
+### Resource Estimation Formula
+
+```
+P0: 2 hours per test (setup + complex scenarios)
+P1: 1 hour per test (standard coverage)
+P2: 0.5 hours per test (simple scenarios)
+P3: 0.25 hours per test (exploratory)
+
+Total Days = Total Hours / 8
+```
+
+Example:
+
+- 15 P0 × 2h = 30h
+- 25 P1 × 1h = 25h
+- 40 P2 × 0.5h = 20h
+- **Total: 75 hours (~10 days)**
+
+### Execution Order Strategy
+
+**Smoke tests** (subset of P0, <5 min):
+
+- Login successful
+- Dashboard loads
+- Core API responds
+
+**Purpose**: Fast feedback, catch build-breaking issues immediately.
+
+**P0 tests** (critical paths, <10 min):
+
+- All scenarios blocking user journeys
+- Security-critical flows
+
+**P1 tests** (important features, <30 min):
+
+- Common workflows
+- Medium-risk areas
+
+**P2/P3 tests** (full regression, <60 min):
+
+- Edge cases
+- Performance benchmarks
+
+### Quality Gate Criteria
+
+**Pass/Fail thresholds:**
+
+- P0: 100% pass (no exceptions)
+- P1: ≥95% pass (2-3 failures acceptable with waivers)
+- P2/P3: ≥90% pass (informational)
+- High-risk items: All mitigated or have approved waivers
+
+**Coverage targets:**
+
+- Critical paths: ≥80%
+- Security scenarios: 100%
+- Business logic: ≥70%
+
+## Validation Checklist
+
+After workflow completion:
+
+- [ ] Risk assessment complete (all categories)
+- [ ] Risks scored (probability × impact)
+- [ ] High-priority risks (≥6) flagged
+- [ ] Coverage matrix maps requirements to test levels
+- [ ] Priorities assigned (P0-P3)
+- [ ] Execution order defined
+- [ ] Resource estimates provided
+- [ ] Quality gate criteria defined
+- [ ] Output file created
+
+Refer to `checklist.md` for comprehensive validation.
+
+## Example Execution
+
+**Scenario: E-commerce checkout epic**
+
+```bash
+bmad tea *test-design
+# Epic 3: Checkout flow redesign
+
+# Risk Assessment identifies:
+- R-001 (SEC): Payment bypass, P=2 × I=3 = 6 (HIGH)
+- R-002 (PERF): Cart load time, P=3 × I=2 = 6 (HIGH)
+- R-003 (BUS): Order confirmation email, P=2 × I=2 = 4 (MEDIUM)
+
+# Coverage Plan:
+P0 scenarios: 12 tests (payment security, order creation)
+P1 scenarios: 18 tests (cart management, promo codes)
+P2 scenarios: 25 tests (edge cases, error handling)
+
+Total effort: 65 hours (~8 days)
+
+# Test Levels:
+- E2E: 8 tests (critical checkout path)
+- API: 30 tests (business logic, payment processing)
+- Unit: 17 tests (calculations, validations)
+
+# Execution Order:
+1. Smoke: Payment successful, order created (2 min)
+2. P0: All payment & security flows (8 min)
+3. P1: Cart & promo codes (20 min)
+4. P2: Edge cases (40 min)
+
+# Quality Gates:
+- P0 pass rate: 100%
+- P1 pass rate: ≥95%
+- R-001 mitigated: Add payment validation layer
+- R-002 mitigated: Implement cart caching
+```
+
+## Troubleshooting
+
+**Issue: "Unable to score risks - missing context"**
+
+- **Cause**: Insufficient documentation
+- **Solution**: Request PRD, architecture docs, or user clarification
+
+**Issue: "All tests marked as P0"**
+
+- **Cause**: Over-prioritization
+- **Solution**: Apply strict P0 criteria (blocks core journey + high risk + no workaround)
+
+**Issue: "Duplicate coverage at multiple test levels"**
+
+- **Cause**: Not following test pyramid
+- **Solution**: Use E2E for critical paths only, API for logic, unit for edge cases
+
+**Issue: "Resource estimates too high"**
+
+- **Cause**: Complex test setup or insufficient automation
+- **Solution**: Invest in fixtures/factories upfront, reduce per-test setup time
+
+## Related Workflows
+
+- **atdd**: Generate failing tests → [atdd/README.md](../atdd/README.md)
+- **automate**: Expand regression coverage → [automate/README.md](../automate/README.md)
+- **trace**: Traceability and quality gate decisions → [trace/README.md](../trace/README.md)
+- **framework**: Test infrastructure → [framework/README.md](../framework/README.md)
+
+## Version History
+
+- **v4.0 (BMad v6)**: Pure markdown instructions, risk scoring framework, template-based output
+- **v3.x**: XML format instructions
+- **v2.x**: Legacy task-based approach

package/src/modules/bmm/workflows/testarch/test-design/checklist.md

@@ -0,0 +1,234 @@
+# Test Design and Risk Assessment - Validation Checklist
+
+## Prerequisites
+
+- [ ] Story markdown with clear acceptance criteria exists
+- [ ] PRD or epic documentation available
+- [ ] Architecture documents available (optional)
+- [ ] Requirements are testable and unambiguous
+
+## Process Steps
+
+### Step 1: Context Loading
+
+- [ ] PRD.md read and requirements extracted
+- [ ] Epics.md or specific epic documentation loaded
+- [ ] Story markdown with acceptance criteria analyzed
+- [ ] Architecture documents reviewed (if available)
+- [ ] Existing test coverage analyzed
+- [ ] Knowledge base fragments loaded (risk-governance, probability-impact, test-levels, test-priorities)
+
+### Step 2: Risk Assessment
+
+- [ ] Genuine risks identified (not just features)
+- [ ] Risks classified by category (TECH/SEC/PERF/DATA/BUS/OPS)
+- [ ] Probability scored (1-3 for each risk)
+- [ ] Impact scored (1-3 for each risk)
+- [ ] Risk scores calculated (probability × impact)
+- [ ] High-priority risks (score ≥6) flagged
+- [ ] Mitigation plans defined for high-priority risks
+- [ ] Owners assigned for each mitigation
+- [ ] Timelines set for mitigations
+- [ ] Residual risk documented
+
+### Step 3: Coverage Design
+
+- [ ] Acceptance criteria broken into atomic scenarios
+- [ ] Test levels selected (E2E/API/Component/Unit)
+- [ ] No duplicate coverage across levels
+- [ ] Priority levels assigned (P0/P1/P2/P3)
+- [ ] P0 scenarios meet strict criteria (blocks core + high risk + no workaround)
+- [ ] Data prerequisites identified
+- [ ] Tooling requirements documented
+- [ ] Execution order defined (smoke → P0 → P1 → P2/P3)
+
+### Step 4: Deliverables Generation
+
+- [ ] Risk assessment matrix created
+- [ ] Coverage matrix created
+- [ ] Execution order documented
+- [ ] Resource estimates calculated
+- [ ] Quality gate criteria defined
+- [ ] Output file written to correct location
+- [ ] Output file uses template structure
+
+## Output Validation
+
+### Risk Assessment Matrix
+
+- [ ] All risks have unique IDs (R-001, R-002, etc.)
+- [ ] Each risk has category assigned
+- [ ] Probability values are 1, 2, or 3
+- [ ] Impact values are 1, 2, or 3
+- [ ] Scores calculated correctly (P × I)
+- [ ] High-priority risks (≥6) clearly marked
+- [ ] Mitigation strategies specific and actionable
+
+### Coverage Matrix
+
+- [ ] All requirements mapped to test levels
+- [ ] Priorities assigned to all scenarios
+- [ ] Risk linkage documented
+- [ ] Test counts realistic
+- [ ] Owners assigned where applicable
+- [ ] No duplicate coverage (same behavior at multiple levels)
+
+### Execution Order
+
+- [ ] Smoke tests defined (<5 min target)
+- [ ] P0 tests listed (<10 min target)
+- [ ] P1 tests listed (<30 min target)
+- [ ] P2/P3 tests listed (<60 min target)
+- [ ] Order optimizes for fast feedback
+
+### Resource Estimates
+
+- [ ] P0 hours calculated (count × 2 hours)
+- [ ] P1 hours calculated (count × 1 hour)
+- [ ] P2 hours calculated (count × 0.5 hours)
+- [ ] P3 hours calculated (count × 0.25 hours)
+- [ ] Total hours summed
+- [ ] Days estimate provided (hours / 8)
+- [ ] Estimates include setup time
+
+### Quality Gate Criteria
+
+- [ ] P0 pass rate threshold defined (should be 100%)
+- [ ] P1 pass rate threshold defined (typically ≥95%)
+- [ ] High-risk mitigation completion required
+- [ ] Coverage targets specified (≥80% recommended)
+
+## Quality Checks
+
+### Evidence-Based Assessment
+
+- [ ] Risk assessment based on documented evidence
+- [ ] No speculation on business impact
+- [ ] Assumptions clearly documented
+- [ ] Clarifications requested where needed
+- [ ] Historical data referenced where available
+
+### Risk Classification Accuracy
+
+- [ ] TECH risks are architecture/integration issues
+- [ ] SEC risks are security vulnerabilities
+- [ ] PERF risks are performance/scalability concerns
+- [ ] DATA risks are data integrity issues
+- [ ] BUS risks are business/revenue impacts
+- [ ] OPS risks are deployment/operational issues
+
+### Priority Assignment Accuracy
+
+- [ ] P0: Truly blocks core functionality
+- [ ] P0: High-risk (score ≥6)
+- [ ] P0: No workaround exists
+- [ ] P1: Important but not blocking
+- [ ] P2/P3: Nice-to-have or edge cases
+
+### Test Level Selection
+
+- [ ] E2E used only for critical paths
+- [ ] API tests cover complex business logic
+- [ ] Component tests for UI interactions
+- [ ] Unit tests for edge cases and algorithms
+- [ ] No redundant coverage
+
+## Integration Points
+
+### Knowledge Base Integration
+
+- [ ] risk-governance.md consulted
+- [ ] probability-impact.md applied
+- [ ] test-levels-framework.md referenced
+- [ ] test-priorities-matrix.md used
+- [ ] Additional fragments loaded as needed
+
+### Status File Integration
+
+- [ ] bmm-workflow-status.md exists
+- [ ] Test design logged in Quality & Testing Progress
+- [ ] Epic number and scope documented
+- [ ] Completion timestamp recorded
+
+### Workflow Dependencies
+
+- [ ] Can proceed to `atdd` workflow with P0 scenarios
+- [ ] Can proceed to `automate` workflow with full coverage plan
+- [ ] Risk assessment informs `gate` workflow criteria
+- [ ] Integrates with `ci` workflow execution order
+
+## Completion Criteria
+
+**All must be true:**
+
+- [ ] All prerequisites met
+- [ ] All process steps completed
+- [ ] All output validations passed
+- [ ] All quality checks passed
+- [ ] All integration points verified
+- [ ] Output file complete and well-formatted
+- [ ] Team review scheduled (if required)
+
+## Post-Workflow Actions
+
+**User must complete:**
+
+1. [ ] Review risk assessment with team
+2. [ ] Prioritize mitigation for high-priority risks (score ≥6)
+3. [ ] Allocate resources per estimates
+4. [ ] Run `atdd` workflow to generate P0 tests
+5. [ ] Set up test data factories and fixtures
+6. [ ] Schedule team review of test design document
+
+**Recommended next workflows:**
+
+1. [ ] Run `atdd` workflow for P0 test generation
+2. [ ] Run `framework` workflow if not already done
+3. [ ] Run `ci` workflow to configure pipeline stages
+
+## Rollback Procedure
+
+If workflow fails:
+
+1. [ ] Delete output file
+2. [ ] Review error logs
+3. [ ] Fix missing context (PRD, architecture docs)
+4. [ ] Clarify ambiguous requirements
+5. [ ] Retry workflow
+
+## Notes
+
+### Common Issues
+
+**Issue**: Too many P0 tests
+
+- **Solution**: Apply strict P0 criteria - must block core AND high risk AND no workaround
+
+**Issue**: Risk scores all high
+
+- **Solution**: Differentiate between high-impact (3) and degraded (2) impacts
+
+**Issue**: Duplicate coverage across levels
+
+- **Solution**: Use test pyramid - E2E for critical paths only
+
+**Issue**: Resource estimates too high
+
+- **Solution**: Invest in fixtures/factories to reduce per-test setup time
+
+### Best Practices
+
+- Base risk assessment on evidence, not assumptions
+- High-priority risks (≥6) require immediate mitigation
+- P0 tests should cover <10% of total scenarios
+- Avoid testing same behavior at multiple levels
+- Include smoke tests (P0 subset) for fast feedback
+
+---
+
+**Checklist Complete**: Sign off when all items validated.
+
+**Completed by:** **\*\***\_\_\_**\*\***
+**Date:** **\*\***\_\_\_**\*\***
+**Epic:** **\*\***\_\_\_**\*\***
+**Notes:** \***\*\*\*\*\***\*\*\***\*\*\*\*\***\_\_\_\***\*\*\*\*\***\*\*\***\*\*\*\*\***