ma-agents 2.19.2 → 2.20.0

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-ci/checklist.md

@@ -0,0 +1,289 @@
+# CI/CD Pipeline Setup - Validation Checklist
+
+## Prerequisites
+
+- [ ] Git repository initialized (`.git/` exists)
+- [ ] Git remote configured (`git remote -v` shows origin)
+- [ ] Test framework configured (appropriate config for detected stack type)
+- [ ] Local tests pass (test command succeeds)
+- [ ] Team agrees on CI platform
+- [ ] Access to CI platform settings (if updating)
+
+### Multi-Stack Detection
+
+- [ ] Test stack type detected or configured (`frontend`, `backend`, `fullstack`)
+- [ ] Test framework detected or configured (Playwright, Cypress, Jest, Vitest, etc.)
+- [ ] Stack-appropriate test commands identified
+
+### Multi-Platform Detection
+
+- [ ] CI platform detected or configured
+- [ ] Supported platform: GitHub Actions, GitLab CI, Jenkins, Azure DevOps, Harness, or Circle CI
+- [ ] Platform-specific template selected
+
+Note: CI setup is typically a one-time task per repo and can be run any time after the test framework is configured.
+
+## Process Steps
+
+### Step 1: Preflight Checks
+
+- [ ] Git repository validated
+- [ ] Framework configuration detected
+- [ ] Local test execution successful
+- [ ] CI platform detected or selected
+- [ ] Node version identified (.nvmrc or default)
+- [ ] No blocking issues found
+
+### Step 2: CI Pipeline Configuration
+
+- [ ] CI configuration file created at platform-correct path
+  - GitHub Actions: `.github/workflows/test.yml`
+  - GitLab CI: `.gitlab-ci.yml`
+  - Jenkins: `Jenkinsfile`
+  - Azure DevOps: `azure-pipelines.yml`
+  - Harness: `.harness/pipeline.yaml`
+  - Circle CI: `.circleci/config.yml`
+- [ ] File is syntactically valid (no YAML/Groovy errors)
+- [ ] Correct framework commands configured for detected stack type
+- [ ] Node version matches project
+- [ ] Test directory paths correct
+- [ ] Stack-conditional steps applied:
+  - [ ] Browser install included for frontend/fullstack stacks
+  - [ ] Browser install omitted for backend-only stacks
+  - [ ] Test commands match detected framework
+
+### Step 3: Parallel Sharding
+
+- [ ] Matrix strategy configured (4 shards default)
+- [ ] Shard syntax correct for framework
+- [ ] fail-fast set to false
+- [ ] Shard count appropriate for test suite size
+
+### Step 4: Burn-In Loop
+
+- [ ] Burn-in job created (frontend/fullstack stacks) or intentionally skipped (backend-only)
+- [ ] 10 iterations configured (when enabled)
+- [ ] Proper exit on failure (`|| exit 1`)
+- [ ] Runs on appropriate triggers (PR, cron)
+- [ ] Failure artifacts uploaded
+- [ ] Backend-only stacks: burn-in skipped by default (documented reason: targets UI flakiness)
+
+### Step 5: Caching Configuration
+
+- [ ] Dependency cache configured (npm/yarn)
+- [ ] Cache key uses lockfile hash
+- [ ] Browser cache configured (Playwright/Cypress)
+- [ ] Restore-keys defined for fallback
+- [ ] Cache paths correct for platform
+
+### Step 6: Artifact Collection
+
+- [ ] Artifacts upload on failure only
+- [ ] Correct artifact paths (test-results/, traces/, etc.)
+- [ ] Retention days set (30 default)
+- [ ] Artifact names unique per shard
+- [ ] No sensitive data in artifacts
+
+### Step 7: Retry Logic
+
+- [ ] Retry action/strategy configured
+- [ ] Max attempts: 2-3
+- [ ] Timeout appropriate (30 min)
+- [ ] Retry only on transient errors
+
+### Step 8: Helper Scripts
+
+- [ ] `scripts/test-changed.sh` created
+- [ ] `scripts/ci-local.sh` created
+- [ ] `scripts/burn-in.sh` created (optional)
+- [ ] Scripts are executable (`chmod +x`)
+- [ ] Scripts use correct test commands
+- [ ] Shebang present (`#!/bin/bash`)
+
+### Step 9: Documentation
+
+- [ ] `docs/ci.md` created with pipeline guide
+- [ ] `docs/ci-secrets-checklist.md` created
+- [ ] Required secrets documented
+- [ ] Setup instructions clear
+- [ ] Troubleshooting section included
+- [ ] Badge URLs provided (optional)
+
+## Output Validation
+
+### Configuration Validation
+
+- [ ] CI file loads without errors
+- [ ] All paths resolve correctly
+- [ ] No hardcoded values (use env vars)
+- [ ] Triggers configured (push, pull_request, schedule)
+- [ ] Platform-specific syntax correct
+
+### Execution Validation
+
+- [ ] First CI run triggered (push to remote)
+- [ ] Pipeline starts without errors
+- [ ] All jobs appear in CI dashboard
+- [ ] Caching works (check logs for cache hit)
+- [ ] Tests execute in parallel
+- [ ] Artifacts collected on failure
+
+### Performance Validation
+
+- [ ] Lint stage: <2 minutes
+- [ ] Test stage (per shard): <10 minutes
+- [ ] Burn-in stage: <30 minutes
+- [ ] Total pipeline: <45 minutes
+- [ ] Cache reduces install time by 2-5 minutes
+
+## Quality Checks
+
+### Best Practices Compliance
+
+- [ ] Burn-in loop follows production patterns
+- [ ] Parallel sharding configured optimally
+- [ ] Failure-only artifact collection
+- [ ] Selective testing enabled (optional)
+- [ ] Retry logic handles transient failures only
+- [ ] No secrets in configuration files
+
+### Knowledge Base Alignment
+
+- [ ] Burn-in pattern matches `ci-burn-in.md`
+- [ ] Selective testing matches `selective-testing.md`
+- [ ] Artifact collection matches `visual-debugging.md`
+- [ ] Test quality matches `test-quality.md`
+
+### Security Checks
+
+- [ ] No credentials in CI configuration
+- [ ] Secrets use platform secret management
+- [ ] Environment variables for sensitive data
+- [ ] Artifact retention appropriate (not too long)
+- [ ] No debug output exposing secrets
+- [ ] **MUST**: No `${{ inputs.* }}` or user-controlled GitHub context (`github.event.pull_request.title`, `github.event.issue.body`, `github.event.comment.body`, `github.head_ref`) directly in `run:` blocks — all passed through `env:` intermediaries and referenced as `"$ENV_VAR"`
+
+## Integration Points
+
+### Status File Integration
+
+- [ ] CI setup logged in Quality & Testing Progress section
+- [ ] Status updated with completion timestamp
+- [ ] Platform and configuration noted
+
+### Knowledge Base Integration
+
+- [ ] Relevant knowledge fragments loaded
+- [ ] Patterns applied from knowledge base
+- [ ] Documentation references knowledge base
+- [ ] Knowledge base references in README
+
+### Workflow Dependencies
+
+- [ ] `framework` workflow completed first
+- [ ] Can proceed to `atdd` workflow after CI setup
+- [ ] Can proceed to `automate` workflow
+- [ ] CI integrates with `gate` workflow
+
+## Completion Criteria
+
+**All must be true:**
+
+- [ ] All prerequisites met
+- [ ] All process steps completed
+- [ ] All output validations passed
+- [ ] All quality checks passed
+- [ ] All integration points verified
+- [ ] First CI run successful
+- [ ] Performance targets met
+- [ ] Documentation complete
+
+## Post-Workflow Actions
+
+**User must complete:**
+
+1. [ ] Commit CI configuration
+2. [ ] Push to remote repository
+3. [ ] Configure required secrets in CI platform
+4. [ ] Open PR to trigger first CI run
+5. [ ] Monitor and verify pipeline execution
+6. [ ] Adjust parallelism if needed (based on actual run times)
+7. [ ] Set up notifications (optional)
+
+**Recommended next workflows:**
+
+1. [ ] Run `atdd` workflow for test generation
+2. [ ] Run `automate` workflow for coverage expansion
+3. [ ] Run `gate` workflow for quality gates
+
+## Rollback Procedure
+
+If workflow fails:
+
+1. [ ] Delete CI configuration file
+2. [ ] Remove helper scripts directory
+3. [ ] Remove documentation (docs/ci.md, etc.)
+4. [ ] Clear CI platform secrets (if added)
+5. [ ] Review error logs
+6. [ ] Fix issues and retry workflow
+
+## Notes
+
+### Common Issues
+
+**Issue**: CI file syntax errors
+
+- **Solution**: Validate YAML syntax online or with linter
+
+**Issue**: Tests fail in CI but pass locally
+
+- **Solution**: Use `scripts/ci-local.sh` to mirror CI environment
+
+**Issue**: Caching not working
+
+- **Solution**: Check cache key formula, verify paths
+
+**Issue**: Burn-in too slow
+
+- **Solution**: Reduce iterations or run on cron only
+
+### Platform-Specific
+
+**GitHub Actions:**
+
+- Secrets: Repository Settings → Secrets and variables → Actions
+- Runners: Ubuntu latest recommended
+- Concurrency limits: 20 jobs for free tier
+
+**GitLab CI:**
+
+- Variables: Project Settings → CI/CD → Variables
+- Runners: Shared or project-specific
+- Pipeline quota: 400 minutes/month free tier
+
+**Jenkins:**
+
+- Credentials: Manage Jenkins → Manage Credentials
+- Agents: Configure build agents with Node.js
+- Plugins: Pipeline, JUnit, HTML Publisher recommended
+
+**Azure DevOps:**
+
+- Variables: Pipelines → Library → Variable groups
+- Agent pools: Azure-hosted or self-hosted
+- Parallel jobs: 1 free (Microsoft-hosted)
+
+**Harness:**
+
+- Connectors: Configure container registry and code repo connectors
+- Delegates: Install Harness delegate in target infrastructure
+- Steps: Use Run steps with appropriate container images
+
+---
+
+**Checklist Complete**: Sign off when all items validated.
+
+**Completed by:** {name}
+**Date:** {date}
+**Platform:** {GitHub Actions, GitLab CI, Other}
+**Notes:** {notes}

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-ci/github-actions-template.yaml

@@ -0,0 +1,328 @@
+# GitHub Actions CI/CD Pipeline for Test Execution
+# Generated by BMad TEA Agent - Test Architect Module
+# Optimized for: Parallel Sharding, Burn-In Loop
+# Stack: {test_stack_type} | Framework: {test_framework}
+#
+# Variables to customize per project:
+#   INSTALL_CMD       - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile, yarn --frozen-lockfile)
+#   TEST_CMD          - main test command (e.g., npm run test:e2e, npm test, npx vitest)
+#   LINT_CMD          - lint command (e.g., npm run lint)
+#   BROWSER_INSTALL   - browser install command (frontend/fullstack only; omit for backend)
+#   BROWSER_CACHE_PATH - browser cache path (frontend/fullstack only; omit for backend)
+
+name: Test Pipeline
+
+on:
+  push:
+    branches: [main, develop]
+  pull_request:
+    branches: [main, develop]
+  schedule:
+    # Weekly burn-in on Sundays at 2 AM UTC
+    - cron: "0 2 * * 0"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  # Lint stage - Code quality checks
+  lint:
+    name: Lint
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Determine Node version
+        id: node-version
+        run: |
+          if [ -f .nvmrc ]; then
+            echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
+            echo "Using Node from .nvmrc"
+          else
+            echo "value=24" >> "$GITHUB_OUTPUT"
+            echo "Using default Node 24 (current LTS)"
+          fi
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: ${{ steps.node-version.outputs.value }}
+          cache: "npm"
+
+      - name: Install dependencies
+        run: npm ci # Replace with INSTALL_CMD
+
+      - name: Run linter
+        run: npm run lint # Replace with LINT_CMD
+
+  # Test stage - Parallel execution with sharding
+  test:
+    name: Test (Shard ${{ matrix.shard }})
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    needs: lint
+
+    strategy:
+      fail-fast: false
+      matrix:
+        shard: [1, 2, 3, 4]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Determine Node version
+        id: node-version
+        run: |
+          if [ -f .nvmrc ]; then
+            echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
+            echo "Using Node from .nvmrc"
+          else
+            echo "value=22" >> "$GITHUB_OUTPUT"
+            echo "Using default Node 22 (current LTS)"
+          fi
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: ${{ steps.node-version.outputs.value }}
+          cache: "npm"
+
+      - name: Cache Playwright browsers
+        uses: actions/cache@v4
+        with:
+          path: ~/.cache/ms-playwright
+          key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-playwright-
+
+      - name: Install dependencies
+        run: npm ci # Replace with INSTALL_CMD
+
+      # Frontend/Fullstack only — remove this step for backend-only stacks
+      - name: Install Playwright browsers
+        run: npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL
+
+      - name: Run tests (shard ${{ matrix.shard }}/4)
+        run: npm run test:e2e -- --shard=${{ matrix.shard }}/4 # Replace with TEST_CMD + shard args
+
+      - name: Upload test results
+        if: failure()
+        uses: actions/upload-artifact@v4
+        with:
+          name: test-results-${{ matrix.shard }}
+          path: |
+            test-results/
+            playwright-report/
+          retention-days: 30
+
+  # Burn-in stage - Flaky test detection
+  burn-in:
+    name: Burn-In (Flaky Detection)
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+    needs: test
+    # Only run burn-in on PRs to main/develop or on schedule
+    if: github.event_name == 'pull_request' || github.event_name == 'schedule'
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Determine Node version
+        id: node-version
+        run: |
+          if [ -f .nvmrc ]; then
+            echo "value=$(cat .nvmrc)" >> "$GITHUB_OUTPUT"
+            echo "Using Node from .nvmrc"
+          else
+            echo "value=22" >> "$GITHUB_OUTPUT"
+            echo "Using default Node 22 (current LTS)"
+          fi
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: ${{ steps.node-version.outputs.value }}
+          cache: "npm"
+
+      # Frontend/Fullstack only — remove this step for backend-only stacks
+      - name: Cache Playwright browsers
+        uses: actions/cache@v4
+        with:
+          path: ~/.cache/ms-playwright # Replace with BROWSER_CACHE_PATH
+          key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
+
+      - name: Install dependencies
+        run: npm ci # Replace with INSTALL_CMD
+
+      # Frontend/Fullstack only — remove this step for backend-only stacks
+      - name: Install Playwright browsers
+        run: npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL
+
+      # Note: Burn-in targets UI flakiness. For backend-only stacks, remove this job entirely.
+      - name: Run burn-in loop (10 iterations)
+        run: |
+          echo "🔥 Starting burn-in loop - detecting flaky tests"
+          for i in {1..10}; do
+            echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+            echo "🔥 Burn-in iteration $i/10"
+            echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+            npm run test:e2e || exit 1  # Replace with TEST_CMD
+          done
+          echo "✅ Burn-in complete - no flaky tests detected"
+
+      - name: Upload burn-in failure artifacts
+        if: failure()
+        uses: actions/upload-artifact@v4
+        with:
+          name: burn-in-failures
+          path: |
+            test-results/
+            playwright-report/
+          retention-days: 30
+
+  # Report stage - Aggregate and publish results
+  report:
+    name: Test Report
+    runs-on: ubuntu-latest
+    needs: [test, burn-in]
+    if: always()
+
+    steps:
+      - name: Download all artifacts
+        uses: actions/download-artifact@v4
+        with:
+          path: artifacts
+
+      - name: Generate summary
+        run: |
+          echo "## Test Execution Summary" >> $GITHUB_STEP_SUMMARY
+          echo "" >> $GITHUB_STEP_SUMMARY
+          echo "- **Status**: ${{ needs.test.result }}" >> $GITHUB_STEP_SUMMARY
+          echo "- **Burn-in**: ${{ needs.burn-in.result }}" >> $GITHUB_STEP_SUMMARY
+          echo "- **Shards**: 4" >> $GITHUB_STEP_SUMMARY
+          echo "" >> $GITHUB_STEP_SUMMARY
+
+          if [ "${{ needs.burn-in.result }}" == "failure" ]; then
+            echo "⚠️ **Flaky tests detected** - Review burn-in artifacts" >> $GITHUB_STEP_SUMMARY
+          fi
+
+# ============================================================================
+# EXTENSION PATTERNS — Script Injection Prevention
+# ============================================================================
+# When extending this template into reusable workflows, manual dispatch
+# workflows, or composite actions, NEVER use ${{ inputs.* }} directly in
+# run: blocks. Always pass through env: intermediaries.
+#
+# KEY PRINCIPLE: Inputs must be DATA, not COMMANDS.
+# Pass inputs through env: and interpolate as quoted arguments into fixed
+# commands. NEVER accept command-shaped inputs (e.g., install-command,
+# test-command) that get executed as shell code — even through env:.
+#
+# --- Reusable Workflow (workflow_call) ---
+#
+# on:
+#   workflow_call:
+#     inputs:
+#       test-grep:
+#         description: 'Test grep filter (data only — not a command)'
+#         type: string
+#         required: false
+#         default: ''
+#       base-ref:
+#         description: 'Base branch for diff'
+#         type: string
+#         required: false
+#         default: 'main'
+#       burn-in-count:
+#         description: 'Number of burn-in iterations'
+#         type: string
+#         required: false
+#         default: '10'
+#
+# jobs:
+#   test:
+#     runs-on: ubuntu-latest
+#     steps:
+#       - uses: actions/checkout@v4
+#       # Fixed command — not derived from inputs
+#       - name: Install dependencies
+#         run: npm ci
+#       # ✅ SAFE — input is DATA passed as an argument to a fixed command
+#       - name: Run tests
+#         env:
+#           TEST_GREP: ${{ inputs.test-grep }}
+#         run: |
+#           # Security: inputs passed through env: to prevent script injection
+#           if [ -n "$TEST_GREP" ]; then
+#             npx playwright test --grep "$TEST_GREP"
+#           else
+#             npx playwright test
+#           fi
+#
+# --- Manual Dispatch (workflow_dispatch) ---
+#
+# on:
+#   workflow_dispatch:
+#     inputs:
+#       test-grep:
+#         description: 'Test grep filter (data only — not a command)'
+#         type: string
+#         required: false
+#       environment:
+#         description: 'Target environment'
+#         type: choice
+#         options: [staging, production]
+#
+# jobs:
+#   run-tests:
+#     runs-on: ubuntu-latest
+#     steps:
+#       - uses: actions/checkout@v4
+#       # ✅ SAFE — input is DATA interpolated into a fixed command
+#       - name: Run selected tests
+#         env:
+#           TEST_GREP: ${{ inputs.test-grep }}
+#         run: |
+#           # Security: inputs passed through env: to prevent script injection
+#           npx playwright test --grep "$TEST_GREP"
+#
+# --- Composite Action (action.yml) ---
+#
+# inputs:
+#   test-grep:
+#     description: 'Test grep filter (data only — not a command)'
+#     required: false
+#     default: ''
+#   burn-in-count:
+#     description: 'Number of burn-in iterations'
+#     required: false
+#     default: '10'
+#
+# runs:
+#   using: composite
+#   steps:
+#     # ✅ SAFE — inputs are DATA arguments to fixed commands
+#     - name: Run burn-in
+#       shell: bash
+#       env:
+#         TEST_GREP: ${{ inputs.test-grep }}
+#         BURN_IN_COUNT: ${{ inputs.burn-in-count }}
+#       run: |
+#         # Security: inputs passed through env: to prevent script injection
+#         for i in $(seq 1 "$BURN_IN_COUNT"); do
+#           echo "Burn-in iteration $i/$BURN_IN_COUNT"
+#           npx playwright test --grep "$TEST_GREP" || exit 1
+#         done
+#
+# ❌ NEVER DO THIS:
+#   # Direct ${{ inputs.* }} in run: — GitHub expression injection
+#   - run: npx playwright test --grep "${{ inputs.test-grep }}"
+#
+#   # Executing input-derived env var as a command — still command injection
+#   - env:
+#       CMD: ${{ inputs.test-command }}
+#     run: $CMD
+# ============================================================================