bmad-method 6.0.0-alpha.11 → 6.0.0-alpha.13

package/src/modules/bmm/testarch/knowledge/burn-in.md

@@ -0,0 +1,273 @@
+# Burn-in Test Runner
+
+## Principle
+
+Use smart test selection with git diff analysis to run only affected tests. Filter out irrelevant changes (configs, types, docs) and control test volume with percentage-based execution. Reduce unnecessary CI runs while maintaining reliability.
+
+## Rationale
+
+Playwright's `--only-changed` triggers all affected tests:
+
+- Config file changes trigger hundreds of tests
+- Type definition changes cause full suite runs
+- No volume control (all or nothing)
+- Slow CI pipelines
+
+The `burn-in` utility provides:
+
+- **Smart filtering**: Skip patterns for irrelevant files (configs, types, docs)
+- **Volume control**: Run percentage of affected tests after filtering
+- **Custom dependency analysis**: More accurate than Playwright's built-in
+- **CI optimization**: Faster pipelines without sacrificing confidence
+- **Process of elimination**: Start with all → filter irrelevant → control volume
+
+## Pattern Examples
+
+### Example 1: Basic Burn-in Setup
+
+**Context**: Run burn-in on changed files compared to main branch.
+
+**Implementation**:
+
+```typescript
+// Step 1: Create burn-in script
+// playwright/scripts/burn-in-changed.ts
+import { runBurnIn } from '@seontechnologies/playwright-utils/burn-in'
+
+async function main() {
+  await runBurnIn({
+    configPath: 'playwright/config/.burn-in.config.ts',
+    baseBranch: 'main'
+  })
+}
+
+main().catch(console.error)
+
+// Step 2: Create config
+// playwright/config/.burn-in.config.ts
+import type { BurnInConfig } from '@seontechnologies/playwright-utils/burn-in'
+
+const config: BurnInConfig = {
+  // Files that never trigger tests (first filter)
+  skipBurnInPatterns: [
+    '**/config/**',
+    '**/*constants*',
+    '**/*types*',
+    '**/*.md',
+    '**/README*'
+  ],
+
+  // Run 30% of remaining tests after skip filter
+  burnInTestPercentage: 0.3,
+
+  // Burn-in repetition
+  burnIn: {
+    repeatEach: 3,  // Run each test 3 times
+    retries: 1      // Allow 1 retry
+  }
+}
+
+export default config
+
+// Step 3: Add package.json script
+{
+  "scripts": {
+    "test:pw:burn-in-changed": "tsx playwright/scripts/burn-in-changed.ts"
+  }
+}
+```
+
+**Key Points**:
+
+- Two-stage filtering: skip patterns, then volume control
+- `skipBurnInPatterns` eliminates irrelevant files
+- `burnInTestPercentage` controls test volume (0.3 = 30%)
+- Custom dependency analysis finds actually affected tests
+
+### Example 2: CI Integration
+
+**Context**: Use burn-in in GitHub Actions for efficient CI runs.
+
+**Implementation**:
+
+```yaml
+# .github/workflows/burn-in.yml
+name: Burn-in Changed Tests
+
+on:
+  pull_request:
+    branches: [main]
+
+jobs:
+  burn-in:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0 # Need git history
+
+      - name: Setup Node
+        uses: actions/setup-node@v4
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Run burn-in on changed tests
+        run: npm run test:pw:burn-in-changed -- --base-branch=origin/main
+
+      - name: Upload artifacts
+        if: failure()
+        uses: actions/upload-artifact@v4
+        with:
+          name: burn-in-failures
+          path: test-results/
+```
+
+**Key Points**:
+
+- `fetch-depth: 0` for full git history
+- Pass `--base-branch=origin/main` for PR comparison
+- Upload artifacts only on failure
+- Significantly faster than full suite
+
+### Example 3: How It Works (Process of Elimination)
+
+**Context**: Understanding the filtering pipeline.
+
+**Scenario:**
+
+```
+Git diff finds: 21 changed files
+├─ Step 1: Skip patterns filter
+│  Removed: 6 files (*.md, config/*, *types*)
+│  Remaining: 15 files
+│
+├─ Step 2: Dependency analysis
+│  Tests that import these 15 files: 45 tests
+│
+└─ Step 3: Volume control (30%)
+   Final tests to run: 14 tests (30% of 45)
+
+Result: Run 14 targeted tests instead of 147 with --only-changed!
+```
+
+**Key Points**:
+
+- Three-stage pipeline: skip → analyze → control
+- Custom dependency analysis (not just imports)
+- Percentage applies AFTER filtering
+- Dramatically reduces CI time
+
+### Example 4: Environment-Specific Configuration
+
+**Context**: Different settings for local vs CI environments.
+
+**Implementation**:
+
+```typescript
+import type { BurnInConfig } from '@seontechnologies/playwright-utils/burn-in';
+
+const config: BurnInConfig = {
+  skipBurnInPatterns: ['**/config/**', '**/*types*', '**/*.md'],
+
+  // CI runs fewer iterations, local runs more
+  burnInTestPercentage: process.env.CI ? 0.2 : 0.3,
+
+  burnIn: {
+    repeatEach: process.env.CI ? 2 : 3,
+    retries: process.env.CI ? 0 : 1, // No retries in CI
+  },
+};
+
+export default config;
+```
+
+**Key Points**:
+
+- `process.env.CI` for environment detection
+- Lower percentage in CI (20% vs 30%)
+- Fewer iterations in CI (2 vs 3)
+- No retries in CI (fail fast)
+
+### Example 5: Sharding Support
+
+**Context**: Distribute burn-in tests across multiple CI workers.
+
+**Implementation**:
+
+```typescript
+// burn-in-changed.ts with sharding
+import { runBurnIn } from '@seontechnologies/playwright-utils/burn-in';
+
+async function main() {
+  const shardArg = process.argv.find((arg) => arg.startsWith('--shard='));
+
+  if (shardArg) {
+    process.env.PW_SHARD = shardArg.split('=')[1];
+  }
+
+  await runBurnIn({
+    configPath: 'playwright/config/.burn-in.config.ts',
+  });
+}
+```
+
+```yaml
+# GitHub Actions with sharding
+jobs:
+  burn-in:
+    strategy:
+      matrix:
+        shard: [1/3, 2/3, 3/3]
+    steps:
+      - run: npm run test:pw:burn-in-changed -- --shard=${{ matrix.shard }}
+```
+
+**Key Points**:
+
+- Pass `--shard=1/3` for parallel execution
+- Burn-in respects Playwright sharding
+- Distribute across multiple workers
+- Reduces total CI time further
+
+## Integration with CI Workflow
+
+When setting up CI with `*ci` workflow, recommend burn-in for:
+
+- Pull request validation
+- Pre-merge checks
+- Nightly builds (subset runs)
+
+## Related Fragments
+
+- `ci-burn-in.md` - Traditional burn-in patterns (10-iteration loops)
+- `selective-testing.md` - Test selection strategies
+- `overview.md` - Installation
+
+## Anti-Patterns
+
+**❌ Over-aggressive skip patterns:**
+
+```typescript
+skipBurnInPatterns: [
+  '**/*', // Skips everything!
+];
+```
+
+**✅ Targeted skip patterns:**
+
+```typescript
+skipBurnInPatterns: ['**/config/**', '**/*types*', '**/*.md', '**/*constants*'];
+```
+
+**❌ Too low percentage (false confidence):**
+
+```typescript
+burnInTestPercentage: 0.05; // Only 5% - might miss issues
+```
+
+**✅ Balanced percentage:**
+
+```typescript
+burnInTestPercentage: 0.2; // 20% in CI, provides good coverage
+```

package/src/modules/bmm/testarch/knowledge/file-utils.md

@@ -0,0 +1,260 @@
+# File Utilities
+
+## Principle
+
+Read and validate files (CSV, XLSX, PDF, ZIP) with automatic parsing, type-safe results, and download handling. Simplify file operations in Playwright tests with built-in format support and validation helpers.
+
+## Rationale
+
+Testing file operations in Playwright requires boilerplate:
+
+- Manual download handling
+- External parsing libraries for each format
+- No validation helpers
+- Type-unsafe results
+- Repetitive path handling
+
+The `file-utils` module provides:
+
+- **Auto-parsing**: CSV, XLSX, PDF, ZIP automatically parsed
+- **Download handling**: Single function for UI or API-triggered downloads
+- **Type-safe**: TypeScript interfaces for parsed results
+- **Validation helpers**: Row count, header checks, content validation
+- **Format support**: Multiple sheet support (XLSX), text extraction (PDF), archive extraction (ZIP)
+
+## Pattern Examples
+
+### Example 1: UI-Triggered CSV Download
+
+**Context**: User clicks button, CSV downloads, validate contents.
+
+**Implementation**:
+
+```typescript
+import { handleDownload, readCSV } from '@seontechnologies/playwright-utils/file-utils';
+import path from 'node:path';
+
+const DOWNLOAD_DIR = path.join(__dirname, '../downloads');
+
+test('should download and validate CSV', async ({ page }) => {
+  const downloadPath = await handleDownload({
+    page,
+    downloadDir: DOWNLOAD_DIR,
+    trigger: () => page.click('[data-testid="export-csv"]'),
+  });
+
+  const { content } = await readCSV({ filePath: downloadPath });
+
+  // Validate headers
+  expect(content.headers).toEqual(['ID', 'Name', 'Email', 'Role']);
+
+  // Validate data
+  expect(content.data).toHaveLength(10);
+  expect(content.data[0]).toMatchObject({
+    ID: expect.any(String),
+    Name: expect.any(String),
+    Email: expect.stringMatching(/@/),
+  });
+});
+```
+
+**Key Points**:
+
+- `handleDownload` waits for download, returns file path
+- `readCSV` auto-parses to `{ headers, data }`
+- Type-safe access to parsed content
+- Clean up downloads in `afterEach`
+
+### Example 2: XLSX with Multiple Sheets
+
+**Context**: Excel file with multiple sheets (e.g., Summary, Details, Errors).
+
+**Implementation**:
+
+```typescript
+import { readXLSX } from '@seontechnologies/playwright-utils/file-utils';
+
+test('should read multi-sheet XLSX', async () => {
+  const downloadPath = await handleDownload({
+    page,
+    downloadDir: DOWNLOAD_DIR,
+    trigger: () => page.click('[data-testid="export-xlsx"]'),
+  });
+
+  const { content } = await readXLSX({ filePath: downloadPath });
+
+  // Access specific sheets
+  const summarySheet = content.sheets.find((s) => s.name === 'Summary');
+  const detailsSheet = content.sheets.find((s) => s.name === 'Details');
+
+  // Validate summary
+  expect(summarySheet.data).toHaveLength(1);
+  expect(summarySheet.data[0].TotalRecords).toBe('150');
+
+  // Validate details
+  expect(detailsSheet.data).toHaveLength(150);
+  expect(detailsSheet.headers).toContain('TransactionID');
+});
+```
+
+**Key Points**:
+
+- `sheets` array with `name` and `data` properties
+- Access sheets by name
+- Each sheet has its own headers and data
+- Type-safe sheet iteration
+
+### Example 3: PDF Text Extraction
+
+**Context**: Validate PDF report contains expected content.
+
+**Implementation**:
+
+```typescript
+import { readPDF } from '@seontechnologies/playwright-utils/file-utils';
+
+test('should validate PDF report', async () => {
+  const downloadPath = await handleDownload({
+    page,
+    downloadDir: DOWNLOAD_DIR,
+    trigger: () => page.click('[data-testid="download-report"]'),
+  });
+
+  const { content } = await readPDF({ filePath: downloadPath });
+
+  // content.text is extracted text from all pages
+  expect(content.text).toContain('Financial Report Q4 2024');
+  expect(content.text).toContain('Total Revenue:');
+
+  // Validate page count
+  expect(content.numpages).toBeGreaterThan(10);
+});
+```
+
+**Key Points**:
+
+- `content.text` contains all extracted text
+- `content.numpages` for page count
+- PDF parsing handles multi-page documents
+- Search for specific phrases
+
+### Example 4: ZIP Archive Validation
+
+**Context**: Validate ZIP contains expected files and extract specific file.
+
+**Implementation**:
+
+```typescript
+import { readZIP } from '@seontechnologies/playwright-utils/file-utils';
+
+test('should validate ZIP archive', async () => {
+  const downloadPath = await handleDownload({
+    page,
+    downloadDir: DOWNLOAD_DIR,
+    trigger: () => page.click('[data-testid="download-backup"]'),
+  });
+
+  const { content } = await readZIP({ filePath: downloadPath });
+
+  // Check file list
+  expect(content.files).toContain('data.csv');
+  expect(content.files).toContain('config.json');
+  expect(content.files).toContain('readme.txt');
+
+  // Read specific file from archive
+  const configContent = content.zip.readAsText('config.json');
+  const config = JSON.parse(configContent);
+
+  expect(config.version).toBe('2.0');
+});
+```
+
+**Key Points**:
+
+- `content.files` lists all files in archive
+- `content.zip.readAsText()` extracts specific files
+- Validate archive structure
+- Read and parse individual files from ZIP
+
+### Example 5: API-Triggered Download
+
+**Context**: API endpoint returns file download (not UI click).
+
+**Implementation**:
+
+```typescript
+test('should download via API', async ({ page, request }) => {
+  const downloadPath = await handleDownload({
+    page,
+    downloadDir: DOWNLOAD_DIR,
+    trigger: async () => {
+      const response = await request.get('/api/export/csv', {
+        headers: { Authorization: 'Bearer token' },
+      });
+
+      if (!response.ok()) {
+        throw new Error(`Export failed: ${response.status()}`);
+      }
+    },
+  });
+
+  const { content } = await readCSV({ filePath: downloadPath });
+
+  expect(content.data).toHaveLength(100);
+});
+```
+
+**Key Points**:
+
+- `trigger` can be async API call
+- API must return `Content-Disposition` header
+- Still need `page` for download events
+- Works with authenticated endpoints
+
+## Validation Helpers
+
+```typescript
+// CSV validation
+const { isValid, errors } = await validateCSV({
+  filePath: downloadPath,
+  expectedRowCount: 10,
+  requiredHeaders: ['ID', 'Name', 'Email'],
+});
+
+expect(isValid).toBe(true);
+expect(errors).toHaveLength(0);
+```
+
+## Download Cleanup Pattern
+
+```typescript
+test.afterEach(async () => {
+  // Clean up downloaded files
+  await fs.remove(DOWNLOAD_DIR);
+});
+```
+
+## Related Fragments
+
+- `overview.md` - Installation and imports
+- `api-request.md` - API-triggered downloads
+- `recurse.md` - Poll for file generation completion
+
+## Anti-Patterns
+
+**❌ Not cleaning up downloads:**
+
+```typescript
+test('creates file', async () => {
+  await handleDownload({ ... })
+  // File left in downloads folder
+})
+```
+
+**✅ Clean up after tests:**
+
+```typescript
+test.afterEach(async () => {
+  await fs.remove(DOWNLOAD_DIR);
+});
+```