bmad-method 5.1.3 → 6.0.0-alpha.1

package/src/modules/bmm/workflows/testarch/framework/README.md

@@ -0,0 +1,340 @@
+# Test Framework Setup Workflow
+
+Initializes a production-ready test framework architecture (Playwright or Cypress) with fixtures, helpers, configuration, and industry best practices. This workflow scaffolds the complete testing infrastructure for modern web applications, providing a robust foundation for test automation.
+
+## Usage
+
+```bash
+bmad tea *framework
+```
+
+The TEA agent runs this workflow when:
+
+- Starting a new project that needs test infrastructure
+- Migrating from an older testing approach
+- Setting up testing from scratch
+- Standardizing test architecture across teams
+
+## Inputs
+
+**Required Context Files:**
+
+- **package.json**: Project dependencies and scripts to detect project type and bundler
+
+**Optional Context Files:**
+
+- **Architecture docs** (architecture.md, tech-spec.md): Informs framework configuration decisions
+- **Existing tests**: Detects current framework to avoid conflicts
+
+**Workflow Variables:**
+
+- `test_framework`: Auto-detected (playwright/cypress) or manually specified
+- `project_type`: Auto-detected from package.json (react/vue/angular/next/node)
+- `bundler`: Auto-detected from package.json (vite/webpack/rollup/esbuild)
+- `test_dir`: Root test directory (default: `{project-root}/tests`)
+- `use_typescript`: Prefer TypeScript configuration (default: true)
+- `framework_preference`: Auto-detection or force specific framework (default: "auto")
+
+## Outputs
+
+**Primary Deliverables:**
+
+1. **Configuration File**
+   - `playwright.config.ts` or `cypress.config.ts` with production-ready settings
+   - Timeouts: action 15s, navigation 30s, test 60s
+   - Reporters: HTML + JUnit XML
+   - Failure-only artifacts (traces, screenshots, videos)
+
+2. **Directory Structure**
+
+   ```
+   tests/
+   ├── e2e/                          # Test files (organize as needed)
+   ├── support/                      # Framework infrastructure (key pattern)
+   │   ├── fixtures/                 # Test fixtures with auto-cleanup
+   │   │   ├── index.ts             # Fixture merging
+   │   │   └── factories/           # Data factories (faker-based)
+   │   ├── helpers/                 # Utility functions
+   │   └── page-objects/            # Page object models (optional)
+   └── README.md                    # Setup and usage guide
+   ```
+
+   **Note**: Test organization (e2e/, api/, integration/, etc.) is flexible. The **support/** folder contains reusable fixtures, helpers, and factories - the core framework pattern.
+
+3. **Environment Configuration**
+   - `.env.example` with `TEST_ENV`, `BASE_URL`, `API_URL`, auth credentials
+   - `.nvmrc` with Node version (LTS)
+
+4. **Test Infrastructure**
+   - Fixture architecture using `mergeTests` pattern
+   - Data factories with auto-cleanup (faker-based)
+   - Sample tests demonstrating best practices
+   - Helper utilities for common operations
+
+5. **Documentation**
+   - `tests/README.md` with comprehensive setup instructions
+   - Inline comments explaining configuration choices
+   - References to TEA knowledge base
+
+**Secondary Deliverables:**
+
+- Updated `package.json` with minimal test script (`test:e2e`)
+- Sample test demonstrating fixture usage
+- Network-first testing patterns
+- Selector strategy guidance (data-testid)
+
+**Validation Safeguards:**
+
+- ✅ No existing framework detected (prevents conflicts)
+- ✅ package.json exists and is valid
+- ✅ Framework auto-detection successful or explicit choice provided
+- ✅ Sample test runs successfully
+- ✅ All generated files are syntactically correct
+
+## Key Features
+
+### Smart Framework Selection
+
+- **Auto-detection logic** based on project characteristics:
+  - **Playwright** recommended for: Large repos (100+ files), performance-critical apps, multi-browser support, complex debugging needs
+  - **Cypress** recommended for: Small teams prioritizing DX, component testing focus, real-time test development
+- Falls back to Playwright as default if uncertain
+
+### Production-Ready Patterns
+
+- **Fixture Architecture**: Pure function → fixture → `mergeTests` composition pattern
+- **Auto-Cleanup**: Fixtures automatically clean up test data in teardown
+- **Network-First**: Route interception before navigation to prevent race conditions
+- **Failure-Only Artifacts**: Screenshots/videos/traces only captured on failure to reduce storage
+- **Parallel Execution**: Configured for optimal CI performance
+
+### Industry Best Practices
+
+- **Selector Strategy**: Prescriptive guidance on `data-testid` attributes
+- **Data Factories**: Faker-based factories for realistic test data
+- **Contract Testing**: Recommends Pact for microservices architectures
+- **Error Handling**: Comprehensive timeout and retry configuration
+- **Reporting**: Multiple reporter formats (HTML, JUnit, console)
+
+### Knowledge Base Integration
+
+Automatically consults TEA knowledge base:
+
+- `fixture-architecture.md` - Pure function → fixture → mergeTests pattern
+- `data-factories.md` - Faker-based factories with auto-cleanup
+- `network-first.md` - Network interception before navigation
+- `playwright-config.md` - Playwright-specific best practices
+- `test-config.md` - General configuration guidelines
+
+## Integration with Other Workflows
+
+**Before framework:**
+
+- **prd** (Phase 2): Determines project scope and testing needs
+- **workflow-status**: Verifies project readiness
+
+**After framework:**
+
+- **ci**: Scaffold CI/CD pipeline using framework configuration
+- **test-design**: Plan test coverage strategy for the project
+- **atdd**: Generate failing acceptance tests using the framework
+
+**Coordinates with:**
+
+- **architecture** (Phase 3): Aligns test structure with system architecture
+- **tech-spec**: Uses technical specifications to inform test configuration
+
+**Updates:**
+
+- `bmm-workflow-status.md`: Adds framework initialization to Quality & Testing Progress section
+
+## Important Notes
+
+### Preflight Checks
+
+**Critical requirements** verified before scaffolding:
+
+- package.json exists in project root
+- No modern E2E framework already configured
+- Architecture/stack context available
+
+If any check fails, workflow **HALTS** and notifies user.
+
+### Framework-Specific Guidance
+
+**Playwright Advantages:**
+
+- Worker parallelism (significantly faster for large suites)
+- Trace viewer (powerful debugging with screenshots, network, console logs)
+- Multi-language support (TypeScript, JavaScript, Python, C#, Java)
+- Built-in API testing capabilities
+- Better handling of multiple browser contexts
+
+**Cypress Advantages:**
+
+- Superior developer experience (real-time reloading)
+- Excellent for component testing
+- Simpler setup for small teams
+- Better suited for watch mode during development
+
+**Avoid Cypress when:**
+
+- API chains are heavy and complex
+- Multi-tab/window scenarios are common
+- Worker parallelism is critical for CI performance
+
+### Selector Strategy
+
+**Always recommend:**
+
+- `data-testid` attributes for UI elements (framework-agnostic)
+- `data-cy` attributes if Cypress is chosen (Cypress-specific)
+- Avoid brittle CSS selectors or XPath
+
+### Standalone Operation
+
+This workflow operates independently:
+
+- **No story required**: Can be run at project initialization
+- **No epic context needed**: Works for greenfield and brownfield projects
+- **Autonomous**: Auto-detects configuration and proceeds without user input
+
+### Output Summary Format
+
+After completion, provides structured summary:
+
+```markdown
+## Framework Scaffold Complete
+
+**Framework Selected**: Playwright (or Cypress)
+
+**Artifacts Created**:
+
+- ✅ Configuration file: playwright.config.ts
+- ✅ Directory structure: tests/e2e/, tests/support/
+- ✅ Environment config: .env.example
+- ✅ Node version: .nvmrc
+- ✅ Fixture architecture: tests/support/fixtures/
+- ✅ Data factories: tests/support/fixtures/factories/
+- ✅ Sample tests: tests/e2e/example.spec.ts
+- ✅ Documentation: tests/README.md
+
+**Next Steps**:
+
+1. Copy .env.example to .env and fill in environment variables
+2. Run npm install to install test dependencies
+3. Run npm run test:e2e to execute sample tests
+4. Review tests/README.md for detailed setup instructions
+
+**Knowledge Base References Applied**:
+
+- Fixture architecture pattern (pure functions + mergeTests)
+- Data factories with auto-cleanup (faker-based)
+- Network-first testing safeguards
+- Failure-only artifact capture
+```
+
+## Validation Checklist
+
+After workflow completion, verify:
+
+- [ ] Configuration file created and syntactically valid
+- [ ] Directory structure exists with all folders
+- [ ] Environment configuration generated (.env.example, .nvmrc)
+- [ ] Sample tests run successfully (npm run test:e2e)
+- [ ] Documentation complete and accurate (tests/README.md)
+- [ ] No errors or warnings during scaffold
+- [ ] package.json scripts updated correctly
+- [ ] Fixtures and factories follow patterns from knowledge base
+
+Refer to `checklist.md` for comprehensive validation criteria.
+
+## Example Execution
+
+**Scenario 1: New React + Vite project**
+
+```bash
+# User runs framework workflow
+bmad tea *framework
+
+# TEA detects:
+# - React project (from package.json)
+# - Vite bundler
+# - No existing test framework
+# - 150+ files (recommends Playwright)
+
+# TEA scaffolds:
+# - playwright.config.ts with Vite detection
+# - Component testing configuration
+# - React Testing Library helpers
+# - Sample component + E2E tests
+```
+
+**Scenario 2: Existing Node.js API project**
+
+```bash
+# User runs framework workflow
+bmad tea *framework
+
+# TEA detects:
+# - Node.js backend (no frontend framework)
+# - Express framework
+# - Small project (50 files)
+# - API endpoints in routes/
+
+# TEA scaffolds:
+# - playwright.config.ts focused on API testing
+# - tests/api/ directory structure
+# - API helper utilities
+# - Sample API tests with auth
+```
+
+**Scenario 3: Cypress preferred (explicit)**
+
+```bash
+# User sets framework preference
+# (in workflow config: framework_preference: "cypress")
+
+bmad tea *framework
+
+# TEA scaffolds:
+# - cypress.config.ts
+# - tests/e2e/ with Cypress patterns
+# - Cypress-specific commands
+# - data-cy selector strategy
+```
+
+## Troubleshooting
+
+**Issue: "Existing test framework detected"**
+
+- **Cause**: playwright.config._ or cypress.config._ already exists
+- **Solution**: Use `upgrade-framework` workflow (TBD) or manually remove existing config
+
+**Issue: "Cannot detect project type"**
+
+- **Cause**: package.json missing or malformed
+- **Solution**: Ensure package.json exists and has valid dependencies
+
+**Issue: "Sample test fails to run"**
+
+- **Cause**: Missing dependencies or incorrect BASE_URL
+- **Solution**: Run `npm install` and configure `.env` with correct URLs
+
+**Issue: "TypeScript compilation errors"**
+
+- **Cause**: Missing @types packages or tsconfig misconfiguration
+- **Solution**: Ensure TypeScript and type definitions are installed
+
+## Related Workflows
+
+- **ci**: Scaffold CI/CD pipeline → [ci/README.md](../ci/README.md)
+- **test-design**: Plan test coverage → [test-design/README.md](../test-design/README.md)
+- **atdd**: Generate acceptance tests → [atdd/README.md](../atdd/README.md)
+- **automate**: Expand regression suite → [automate/README.md](../automate/README.md)
+
+## Version History
+
+- **v4.0 (BMad v6)**: Pure markdown instructions, enhanced workflow.yaml, comprehensive README
+- **v3.x**: XML format instructions
+- **v2.x**: Legacy task-based approach

package/src/modules/bmm/workflows/testarch/framework/checklist.md

@@ -0,0 +1,321 @@
+# Test Framework Setup - Validation Checklist
+
+This checklist ensures the framework workflow completes successfully and all deliverables meet quality standards.
+
+---
+
+## Prerequisites
+
+Before starting the workflow:
+
+- [ ] Project root contains valid `package.json`
+- [ ] No existing modern E2E framework detected (`playwright.config.*`, `cypress.config.*`)
+- [ ] Project type identifiable (React, Vue, Angular, Next.js, Node, etc.)
+- [ ] Bundler identifiable (Vite, Webpack, Rollup, esbuild) or not applicable
+- [ ] User has write permissions to create directories and files
+
+---
+
+## Process Steps
+
+### Step 1: Preflight Checks
+
+- [ ] package.json successfully read and parsed
+- [ ] Project type extracted correctly
+- [ ] Bundler identified (or marked as N/A for backend projects)
+- [ ] No framework conflicts detected
+- [ ] Architecture documents located (if available)
+
+### Step 2: Framework Selection
+
+- [ ] Framework auto-detection logic executed
+- [ ] Framework choice justified (Playwright vs Cypress)
+- [ ] Framework preference respected (if explicitly set)
+- [ ] User notified of framework selection and rationale
+
+### Step 3: Directory Structure
+
+- [ ] `tests/` root directory created
+- [ ] `tests/e2e/` directory created (or user's preferred structure)
+- [ ] `tests/support/` directory created (critical pattern)
+- [ ] `tests/support/fixtures/` directory created
+- [ ] `tests/support/fixtures/factories/` directory created
+- [ ] `tests/support/helpers/` directory created
+- [ ] `tests/support/page-objects/` directory created (if applicable)
+- [ ] All directories have correct permissions
+
+**Note**: Test organization is flexible (e2e/, api/, integration/). The **support/** folder is the key pattern.
+
+### Step 4: Configuration Files
+
+- [ ] Framework config file created (`playwright.config.ts` or `cypress.config.ts`)
+- [ ] Config file uses TypeScript (if `use_typescript: true`)
+- [ ] Timeouts configured correctly (action: 15s, navigation: 30s, test: 60s)
+- [ ] Base URL configured with environment variable fallback
+- [ ] Trace/screenshot/video set to retain-on-failure
+- [ ] Multiple reporters configured (HTML + JUnit + console)
+- [ ] Parallel execution enabled
+- [ ] CI-specific settings configured (retries, workers)
+- [ ] Config file is syntactically valid (no compilation errors)
+
+### Step 5: Environment Configuration
+
+- [ ] `.env.example` created in project root
+- [ ] `TEST_ENV` variable defined
+- [ ] `BASE_URL` variable defined with default
+- [ ] `API_URL` variable defined (if applicable)
+- [ ] Authentication variables defined (if applicable)
+- [ ] Feature flag variables defined (if applicable)
+- [ ] `.nvmrc` created with appropriate Node version
+
+### Step 6: Fixture Architecture
+
+- [ ] `tests/support/fixtures/index.ts` created
+- [ ] Base fixture extended from Playwright/Cypress
+- [ ] Type definitions for fixtures created
+- [ ] mergeTests pattern implemented (if multiple fixtures)
+- [ ] Auto-cleanup logic included in fixtures
+- [ ] Fixture architecture follows knowledge base patterns
+
+### Step 7: Data Factories
+
+- [ ] At least one factory created (e.g., UserFactory)
+- [ ] Factories use @faker-js/faker for realistic data
+- [ ] Factories track created entities (for cleanup)
+- [ ] Factories implement `cleanup()` method
+- [ ] Factories integrate with fixtures
+- [ ] Factories follow knowledge base patterns
+
+### Step 8: Sample Tests
+
+- [ ] Example test file created (`tests/e2e/example.spec.ts`)
+- [ ] Test uses fixture architecture
+- [ ] Test demonstrates data factory usage
+- [ ] Test uses proper selector strategy (data-testid)
+- [ ] Test follows Given-When-Then structure
+- [ ] Test includes proper assertions
+- [ ] Network interception demonstrated (if applicable)
+
+### Step 9: Helper Utilities
+
+- [ ] API helper created (if API testing needed)
+- [ ] Network helper created (if network mocking needed)
+- [ ] Auth helper created (if authentication needed)
+- [ ] Helpers follow functional patterns
+- [ ] Helpers have proper error handling
+
+### Step 10: Documentation
+
+- [ ] `tests/README.md` created
+- [ ] Setup instructions included
+- [ ] Running tests section included
+- [ ] Architecture overview section included
+- [ ] Best practices section included
+- [ ] CI integration section included
+- [ ] Knowledge base references included
+- [ ] Troubleshooting section included
+
+### Step 11: Package.json Updates
+
+- [ ] Minimal test script added to package.json: `test:e2e`
+- [ ] Test framework dependency added (if not already present)
+- [ ] Type definitions added (if TypeScript)
+- [ ] Users can extend with additional scripts as needed
+
+---
+
+## Output Validation
+
+### Configuration Validation
+
+- [ ] Config file loads without errors
+- [ ] Config file passes linting (if linter configured)
+- [ ] Config file uses correct syntax for chosen framework
+- [ ] All paths in config resolve correctly
+- [ ] Reporter output directories exist or are created on test run
+
+### Test Execution Validation
+
+- [ ] Sample test runs successfully
+- [ ] Test execution produces expected output (pass/fail)
+- [ ] Test artifacts generated correctly (traces, screenshots, videos)
+- [ ] Test report generated successfully
+- [ ] No console errors or warnings during test run
+
+### Directory Structure Validation
+
+- [ ] All required directories exist
+- [ ] Directory structure matches framework conventions
+- [ ] No duplicate or conflicting directories
+- [ ] Directories accessible with correct permissions
+
+### File Integrity Validation
+
+- [ ] All generated files are syntactically correct
+- [ ] No placeholder text left in files (e.g., "TODO", "FIXME")
+- [ ] All imports resolve correctly
+- [ ] No hardcoded credentials or secrets in files
+- [ ] All file paths use correct separators for OS
+
+---
+
+## Quality Checks
+
+### Code Quality
+
+- [ ] Generated code follows project coding standards
+- [ ] TypeScript types are complete and accurate (no `any` unless necessary)
+- [ ] No unused imports or variables
+- [ ] Consistent code formatting (matches project style)
+- [ ] No linting errors in generated files
+
+### Best Practices Compliance
+
+- [ ] Fixture architecture follows pure function → fixture → mergeTests pattern
+- [ ] Data factories implement auto-cleanup
+- [ ] Network interception occurs before navigation
+- [ ] Selectors use data-testid strategy
+- [ ] Artifacts only captured on failure
+- [ ] Tests follow Given-When-Then structure
+- [ ] No hard-coded waits or sleeps
+
+### Knowledge Base Alignment
+
+- [ ] Fixture pattern matches `fixture-architecture.md`
+- [ ] Data factories match `data-factories.md`
+- [ ] Network handling matches `network-first.md`
+- [ ] Config follows `playwright-config.md` or `test-config.md`
+- [ ] Test quality matches `test-quality.md`
+
+### Security Checks
+
+- [ ] No credentials in configuration files
+- [ ] .env.example contains placeholders, not real values
+- [ ] Sensitive test data handled securely
+- [ ] API keys and tokens use environment variables
+- [ ] No secrets committed to version control
+
+---
+
+## Integration Points
+
+### Status File Integration
+
+- [ ] `bmm-workflow-status.md` exists
+- [ ] Framework initialization logged in Quality & Testing Progress section
+- [ ] Status file updated with completion timestamp
+- [ ] Status file shows framework: Playwright or Cypress
+
+### Knowledge Base Integration
+
+- [ ] Relevant knowledge fragments identified from tea-index.csv
+- [ ] Knowledge fragments successfully loaded
+- [ ] Patterns from knowledge base applied correctly
+- [ ] Knowledge base references included in documentation
+
+### Workflow Dependencies
+
+- [ ] Can proceed to `ci` workflow after completion
+- [ ] Can proceed to `test-design` workflow after completion
+- [ ] Can proceed to `atdd` workflow after completion
+- [ ] Framework setup compatible with downstream workflows
+
+---
+
+## Completion Criteria
+
+**All of the following must be true:**
+
+- [ ] All prerequisite checks passed
+- [ ] All process steps completed without errors
+- [ ] All output validations passed
+- [ ] All quality checks passed
+- [ ] All integration points verified
+- [ ] Sample test executes successfully
+- [ ] User can run `npm run test:e2e` without errors
+- [ ] Documentation is complete and accurate
+- [ ] No critical issues or blockers identified
+
+---
+
+## Post-Workflow Actions
+
+**User must complete:**
+
+1. [ ] Copy `.env.example` to `.env`
+2. [ ] Fill in environment-specific values in `.env`
+3. [ ] Run `npm install` to install test dependencies
+4. [ ] Run `npm run test:e2e` to verify setup
+5. [ ] Review `tests/README.md` for project-specific guidance
+
+**Recommended next workflows:**
+
+1. [ ] Run `ci` workflow to set up CI/CD pipeline
+2. [ ] Run `test-design` workflow to plan test coverage
+3. [ ] Run `atdd` workflow when ready to develop stories
+
+---
+
+## Rollback Procedure
+
+If workflow fails and needs to be rolled back:
+
+1. [ ] Delete `tests/` directory
+2. [ ] Remove test scripts from package.json
+3. [ ] Delete `.env.example` (if created)
+4. [ ] Delete `.nvmrc` (if created)
+5. [ ] Delete framework config file
+6. [ ] Remove test dependencies from package.json (if added)
+7. [ ] Run `npm install` to clean up node_modules
+
+---
+
+## Notes
+
+### Common Issues
+
+**Issue**: Config file has TypeScript errors
+
+- **Solution**: Ensure `@playwright/test` or `cypress` types are installed
+
+**Issue**: Sample test fails to run
+
+- **Solution**: Check BASE_URL in .env, ensure app is running
+
+**Issue**: Fixture cleanup not working
+
+- **Solution**: Verify cleanup() is called in fixture teardown
+
+**Issue**: Network interception not working
+
+- **Solution**: Ensure route setup occurs before page.goto()
+
+### Framework-Specific Considerations
+
+**Playwright:**
+
+- Requires Node.js 18+
+- Browser binaries auto-installed on first run
+- Trace viewer requires running `npx playwright show-trace`
+
+**Cypress:**
+
+- Requires Node.js 18+
+- Cypress app opens on first run
+- Component testing requires additional setup
+
+### Version Compatibility
+
+- [ ] Node.js version matches .nvmrc
+- [ ] Framework version compatible with Node.js version
+- [ ] TypeScript version compatible with framework
+- [ ] All peer dependencies satisfied
+
+---
+
+**Checklist Complete**: Sign off when all items checked and validated.
+
+**Completed by:** **\*\***\_\_\_**\*\***
+**Date:** **\*\***\_\_\_**\*\***
+**Framework:** **\*\***\_\_\_**\*\*** (Playwright / Cypress)
+**Notes:** \***\*\*\*\*\***\*\*\***\*\*\*\*\***\_\_\_\***\*\*\*\*\***\*\*\***\*\*\*\*\***