bmad-method 6.0.0-alpha.9 → 6.0.2

package/src/modules/bmm/workflows/testarch/ci/instructions.md

@@ -1,517 +0,0 @@
-<!-- Powered by BMAD-CORE™ -->
-
-# CI/CD Pipeline Setup
-
-**Workflow ID**: `{bmad_folder}/bmm/testarch/ci`
-**Version**: 4.0 (BMad v6)
-
----
-
-## Overview
-
-Scaffolds a production-ready CI/CD quality pipeline with test execution, burn-in loops for flaky test detection, parallel sharding, artifact collection, and notification configuration. This workflow creates platform-specific CI configuration optimized for fast feedback and reliable test execution.
-
----
-
-## Preflight Requirements
-
-**Critical:** Verify these requirements before proceeding. If any fail, HALT and notify the user.
-
-- ✅ Git repository is initialized (`.git/` directory exists)
-- ✅ Local test suite passes (`npm run test:e2e` succeeds)
-- ✅ Test framework is configured (from `framework` workflow)
-- ✅ Team agrees on target CI platform (GitHub Actions, GitLab CI, Circle CI, etc.)
-- ✅ Access to CI platform settings/secrets available (if updating existing pipeline)
-
----
-
-## Step 1: Run Preflight Checks
-
-### Actions
-
-1. **Verify Git Repository**
-   - Check for `.git/` directory
-   - Confirm remote repository configured (`git remote -v`)
-   - If not initialized, HALT with message: "Git repository required for CI/CD setup"
-
-2. **Validate Test Framework**
-   - Look for `playwright.config.*` or `cypress.config.*`
-   - Read framework configuration to extract:
-     - Test directory location
-     - Test command
-     - Reporter configuration
-     - Timeout settings
-   - If not found, HALT with message: "Run `framework` workflow first to set up test infrastructure"
-
-3. **Run Local Tests**
-   - Execute `npm run test:e2e` (or equivalent from package.json)
-   - Ensure tests pass before CI setup
-   - If tests fail, HALT with message: "Fix failing tests before setting up CI/CD"
-
-4. **Detect CI Platform**
-   - Check for existing CI configuration:
-     - `.github/workflows/*.yml` (GitHub Actions)
-     - `.gitlab-ci.yml` (GitLab CI)
-     - `.circleci/config.yml` (Circle CI)
-     - `Jenkinsfile` (Jenkins)
-   - If found, ask user: "Update existing CI configuration or create new?"
-   - If not found, detect platform from git remote:
-     - `github.com` → GitHub Actions (default)
-     - `gitlab.com` → GitLab CI
-     - Ask user if unable to auto-detect
-
-5. **Read Environment Configuration**
-   - Check for `.nvmrc` to determine Node version
-   - Default to Node 20 LTS if not found
-   - Read `package.json` to identify dependencies (affects caching strategy)
-
-**Halt Condition:** If preflight checks fail, stop immediately and report which requirement failed.
-
----
-
-## Step 2: Scaffold CI Pipeline
-
-### Actions
-
-1. **Select CI Platform Template**
-
-   Based on detection or user preference, use the appropriate template:
-
-   **GitHub Actions** (`.github/workflows/test.yml`):
-   - Most common platform
-   - Excellent caching and matrix support
-   - Free for public repos, generous free tier for private
-
-   **GitLab CI** (`.gitlab-ci.yml`):
-   - Integrated with GitLab
-   - Built-in registry and runners
-   - Powerful pipeline features
-
-   **Circle CI** (`.circleci/config.yml`):
-   - Fast execution with parallelism
-   - Docker-first approach
-   - Enterprise features
-
-   **Jenkins** (`Jenkinsfile`):
-   - Self-hosted option
-   - Maximum customization
-   - Requires infrastructure management
-
-2. **Generate Pipeline Configuration**
-
-   Use templates from `{installed_path}/` directory:
-   - `github-actions-template.yml`
-   - `gitlab-ci-template.yml`
-
-   **Key pipeline stages:**
-
-   ```yaml
-   stages:
-     - lint # Code quality checks
-     - test # Test execution (parallel shards)
-     - burn-in # Flaky test detection
-     - report # Aggregate results and publish
-   ```
-
-3. **Configure Test Execution**
-
-   **Parallel Sharding:**
-
-   ```yaml
-   strategy:
-     fail-fast: false
-     matrix:
-       shard: [1, 2, 3, 4]
-
-   steps:
-     - name: Run tests
-       run: npm run test:e2e -- --shard=${{ matrix.shard }}/${{ strategy.job-total }}
-   ```
-
-   **Purpose:** Splits tests into N parallel jobs for faster execution (target: <10 min per shard)
-
-4. **Add Burn-In Loop**
-
-   **Critical pattern from production systems:**
-
-   ```yaml
-   burn-in:
-     name: Flaky Test Detection
-     runs-on: ubuntu-latest
-     steps:
-       - uses: actions/checkout@v4
-
-       - name: Setup Node
-         uses: actions/setup-node@v4
-         with:
-           node-version-file: '.nvmrc'
-
-       - name: Install dependencies
-         run: npm ci
-
-       - name: Run burn-in loop (10 iterations)
-         run: |
-           for i in {1..10}; do
-             echo "🔥 Burn-in iteration $i/10"
-             npm run test:e2e || exit 1
-           done
-
-       - name: Upload failure artifacts
-         if: failure()
-         uses: actions/upload-artifact@v4
-         with:
-           name: burn-in-failures
-           path: test-results/
-           retention-days: 30
-   ```
-
-   **Purpose:** Runs tests multiple times to catch non-deterministic failures before they reach main branch.
-
-   **When to run:**
-   - On pull requests to main/develop
-   - Weekly on cron schedule
-   - After significant test infrastructure changes
-
-5. **Configure Caching**
-
-   **Node modules cache:**
-
-   ```yaml
-   - name: Cache dependencies
-     uses: actions/cache@v4
-     with:
-       path: ~/.npm
-       key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
-       restore-keys: |
-         ${{ runner.os }}-node-
-   ```
-
-   **Browser binaries cache (Playwright):**
-
-   ```yaml
-   - name: Cache Playwright browsers
-     uses: actions/cache@v4
-     with:
-       path: ~/.cache/ms-playwright
-       key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
-   ```
-
-   **Purpose:** Reduces CI execution time by 2-5 minutes per run.
-
-6. **Configure Artifact Collection**
-
-   **Failure artifacts only:**
-
-   ```yaml
-   - name: Upload test results
-     if: failure()
-     uses: actions/upload-artifact@v4
-     with:
-       name: test-results-${{ matrix.shard }}
-       path: |
-         test-results/
-         playwright-report/
-       retention-days: 30
-   ```
-
-   **Artifacts to collect:**
-   - Traces (Playwright) - full debugging context
-   - Screenshots - visual evidence of failures
-   - Videos - interaction playback
-   - HTML reports - detailed test results
-   - Console logs - error messages and warnings
-
-7. **Add Retry Logic**
-
-   ```yaml
-   - name: Run tests with retries
-     uses: nick-invision/retry@v2
-     with:
-       timeout_minutes: 30
-       max_attempts: 3
-       retry_on: error
-       command: npm run test:e2e
-   ```
-
-   **Purpose:** Handles transient failures (network issues, race conditions)
-
-8. **Configure Notifications** (Optional)
-
-   If `notify_on_failure` is enabled:
-
-   ```yaml
-   - name: Notify on failure
-     if: failure()
-     uses: 8398a7/action-slack@v3
-     with:
-       status: ${{ job.status }}
-       text: 'Test failures detected in PR #${{ github.event.pull_request.number }}'
-       webhook_url: ${{ secrets.SLACK_WEBHOOK }}
-   ```
-
-9. **Generate Helper Scripts**
-
-   **Selective testing script** (`scripts/test-changed.sh`):
-
-   ```bash
-   #!/bin/bash
-   # Run only tests for changed files
-
-   CHANGED_FILES=$(git diff --name-only HEAD~1)
-
-   if echo "$CHANGED_FILES" | grep -q "src/.*\.ts$"; then
-     echo "Running affected tests..."
-     npm run test:e2e -- --grep="$(echo $CHANGED_FILES | sed 's/src\///g' | sed 's/\.ts//g')"
-   else
-     echo "No test-affecting changes detected"
-   fi
-   ```
-
-   **Local mirror script** (`scripts/ci-local.sh`):
-
-   ```bash
-   #!/bin/bash
-   # Mirror CI execution locally for debugging
-
-   echo "🔍 Running CI pipeline locally..."
-
-   # Lint
-   npm run lint || exit 1
-
-   # Tests
-   npm run test:e2e || exit 1
-
-   # Burn-in (reduced iterations)
-   for i in {1..3}; do
-     echo "🔥 Burn-in $i/3"
-     npm run test:e2e || exit 1
-   done
-
-   echo "✅ Local CI pipeline passed"
-   ```
-
-10. **Generate Documentation**
-
-    **CI README** (`docs/ci.md`):
-    - Pipeline stages and purpose
-    - How to run locally
-    - Debugging failed CI runs
-    - Secrets and environment variables needed
-    - Notification setup
-    - Badge URLs for README
-
-    **Secrets checklist** (`docs/ci-secrets-checklist.md`):
-    - Required secrets list (SLACK_WEBHOOK, etc.)
-    - Where to configure in CI platform
-    - Security best practices
-
----
-
-## Step 3: Deliverables
-
-### Primary Artifacts Created
-
-1. **CI Configuration File**
-   - `.github/workflows/test.yml` (GitHub Actions)
-   - `.gitlab-ci.yml` (GitLab CI)
-   - `.circleci/config.yml` (Circle CI)
-
-2. **Pipeline Stages**
-   - **Lint**: Code quality checks (ESLint, Prettier)
-   - **Test**: Parallel test execution (4 shards)
-   - **Burn-in**: Flaky test detection (10 iterations)
-   - **Report**: Result aggregation and publishing
-
-3. **Helper Scripts**
-   - `scripts/test-changed.sh` - Selective testing
-   - `scripts/ci-local.sh` - Local CI mirror
-   - `scripts/burn-in.sh` - Standalone burn-in execution
-
-4. **Documentation**
-   - `docs/ci.md` - CI pipeline guide
-   - `docs/ci-secrets-checklist.md` - Required secrets
-   - Inline comments in CI configuration
-
-5. **Optimization Features**
-   - Dependency caching (npm, browser binaries)
-   - Parallel sharding (4 jobs default)
-   - Retry logic (2 retries on failure)
-   - Failure-only artifact upload
-
-### Performance Targets
-
-- **Lint stage**: <2 minutes
-- **Test stage** (per shard): <10 minutes
-- **Burn-in stage**: <30 minutes (10 iterations)
-- **Total pipeline**: <45 minutes
-
-**Speedup:** 20× faster than sequential execution through parallelism and caching.
-
----
-
-## Important Notes
-
-### Knowledge Base Integration
-
-**Critical:** Consult `{project-root}/{bmad_folder}/bmm/testarch/tea-index.csv` to identify and load relevant knowledge fragments:
-
-- `ci-burn-in.md` - Burn-in loop patterns: 10-iteration detection, GitHub Actions workflow, shard orchestration, selective execution (678 lines, 4 examples)
-- `selective-testing.md` - Changed test detection strategies: tag-based, spec filters, diff-based selection, promotion rules (727 lines, 4 examples)
-- `visual-debugging.md` - Artifact collection best practices: trace viewer, HAR recording, custom artifacts, accessibility integration (522 lines, 5 examples)
-- `test-quality.md` - CI-specific test quality criteria: deterministic tests, isolated with cleanup, explicit assertions, length/time optimization (658 lines, 5 examples)
-- `playwright-config.md` - CI-optimized configuration: parallelization, artifact output, project dependencies, sharding (722 lines, 5 examples)
-
-### CI Platform-Specific Guidance
-
-**GitHub Actions:**
-
-- Use `actions/cache` for caching
-- Matrix strategy for parallelism
-- Secrets in repository settings
-- Free 2000 minutes/month for private repos
-
-**GitLab CI:**
-
-- Use `.gitlab-ci.yml` in root
-- `cache:` directive for caching
-- Parallel execution with `parallel: 4`
-- Variables in project CI/CD settings
-
-**Circle CI:**
-
-- Use `.circleci/config.yml`
-- Docker executors recommended
-- Parallelism with `parallelism: 4`
-- Context for shared secrets
-
-### Burn-In Loop Strategy
-
-**When to run:**
-
-- ✅ On PRs to main/develop branches
-- ✅ Weekly on schedule (cron)
-- ✅ After test infrastructure changes
-- ❌ Not on every commit (too slow)
-
-**Iterations:**
-
-- **10 iterations** for thorough detection
-- **3 iterations** for quick feedback
-- **100 iterations** for high-confidence stability
-
-**Failure threshold:**
-
-- Even ONE failure in burn-in → tests are flaky
-- Must fix before merging
-
-### Artifact Retention
-
-**Failure artifacts only:**
-
-- Saves storage costs
-- Maintains debugging capability
-- 30-day retention default
-
-**Artifact types:**
-
-- Traces (Playwright) - 5-10 MB per test
-- Screenshots - 100-500 KB per screenshot
-- Videos - 2-5 MB per test
-- HTML reports - 1-2 MB per run
-
-### Selective Testing
-
-**Detect changed files:**
-
-```bash
-git diff --name-only HEAD~1
-```
-
-**Run affected tests only:**
-
-- Faster feedback for small changes
-- Full suite still runs on main branch
-- Reduces CI time by 50-80% for focused PRs
-
-**Trade-off:**
-
-- May miss integration issues
-- Run full suite at least on merge
-
-### Local CI Mirror
-
-**Purpose:** Debug CI failures locally
-
-**Usage:**
-
-```bash
-./scripts/ci-local.sh
-```
-
-**Mirrors CI environment:**
-
-- Same Node version
-- Same test command
-- Same stages (lint → test → burn-in)
-- Reduced burn-in iterations (3 vs 10)
-
----
-
-## Output Summary
-
-After completing this workflow, provide a summary:
-
-```markdown
-## CI/CD Pipeline Complete
-
-**Platform**: GitHub Actions (or GitLab CI, etc.)
-
-**Artifacts Created**:
-
-- ✅ Pipeline configuration: .github/workflows/test.yml
-- ✅ Burn-in loop: 10 iterations for flaky detection
-- ✅ Parallel sharding: 4 jobs for fast execution
-- ✅ Caching: Dependencies + browser binaries
-- ✅ Artifact collection: Failure-only traces/screenshots/videos
-- ✅ Helper scripts: test-changed.sh, ci-local.sh, burn-in.sh
-- ✅ Documentation: docs/ci.md, docs/ci-secrets-checklist.md
-
-**Performance:**
-
-- Lint: <2 min
-- Test (per shard): <10 min
-- Burn-in: <30 min
-- Total: <45 min (20× speedup vs sequential)
-
-**Next Steps**:
-
-1. Commit CI configuration: `git add .github/workflows/test.yml && git commit -m "ci: add test pipeline"`
-2. Push to remote: `git push`
-3. Configure required secrets in CI platform settings (see docs/ci-secrets-checklist.md)
-4. Open a PR to trigger first CI run
-5. Monitor pipeline execution and adjust parallelism if needed
-
-**Knowledge Base References Applied**:
-
-- Burn-in loop pattern (ci-burn-in.md)
-- Selective testing strategy (selective-testing.md)
-- Artifact collection (visual-debugging.md)
-- Test quality criteria (test-quality.md)
-```
-
----
-
-## Validation
-
-After completing all steps, verify:
-
-- [ ] CI configuration file created and syntactically valid
-- [ ] Burn-in loop configured (10 iterations)
-- [ ] Parallel sharding enabled (4 jobs)
-- [ ] Caching configured (dependencies + browsers)
-- [ ] Artifact collection on failure only
-- [ ] Helper scripts created and executable (`chmod +x`)
-- [ ] Documentation complete (ci.md, secrets checklist)
-- [ ] No errors or warnings during scaffold
-
-Refer to `checklist.md` for comprehensive validation criteria.

package/src/modules/bmm/workflows/testarch/ci/workflow.yaml

@@ -1,55 +0,0 @@
-# Test Architect workflow: ci
-name: testarch-ci
-description: "Scaffold CI/CD quality pipeline with test execution, burn-in loops, and artifact collection"
-author: "BMad"
-
-# Critical variables from config
-config_source: "{project-root}/{bmad_folder}/bmm/config.yaml"
-output_folder: "{config_source}:output_folder"
-user_name: "{config_source}:user_name"
-communication_language: "{config_source}:communication_language"
-document_output_language: "{config_source}:document_output_language"
-date: system-generated
-
-# Workflow components
-installed_path: "{project-root}/{bmad_folder}/bmm/workflows/testarch/ci"
-instructions: "{installed_path}/instructions.md"
-validation: "{installed_path}/checklist.md"
-
-# Variables and inputs
-variables:
-  ci_platform: "auto" # auto, github-actions, gitlab-ci, circle-ci, jenkins - user can override
-  test_dir: "{project-root}/tests" # Root test directory
-
-# Output configuration
-default_output_file: "{project-root}/.github/workflows/test.yml" # GitHub Actions default
-
-# Required tools
-required_tools:
-  - read_file # Read .nvmrc, package.json, framework config
-  - write_file # Create CI config, scripts, documentation
-  - create_directory # Create .github/workflows/ or .gitlab-ci/ directories
-  - list_files # Detect existing CI configuration
-  - search_repo # Find test files for selective testing
-
-# Recommended inputs
-recommended_inputs:
-  - framework_config: "Framework configuration (playwright.config.ts, cypress.config.ts)"
-  - package_json: "Project dependencies and scripts"
-  - nvmrc: ".nvmrc for Node version (optional, defaults to LTS)"
-  - existing_ci: "Existing CI configuration to update (optional)"
-  - git_info: "Git repository information for platform detection"
-
-tags:
-  - qa
-  - ci-cd
-  - test-architect
-  - pipeline
-  - automation
-
-execution_hints:
-  interactive: false # Minimize prompts, auto-detect when possible
-  autonomous: true # Proceed without user input unless blocked
-  iterative: true
-
-web_bundle: false