bmad-method 6.0.0-alpha.12 → 6.0.0-alpha.14

package/src/modules/bmm/testarch/knowledge/network-error-monitor.md

@@ -0,0 +1,272 @@
+# Network Error Monitor
+
+## Principle
+
+Automatically detect and fail tests when HTTP 4xx/5xx errors occur during execution. Act like Sentry for tests - catch silent backend failures even when UI passes assertions.
+
+## Rationale
+
+Traditional Playwright tests focus on UI:
+
+- Backend 500 errors ignored if UI looks correct
+- Silent failures slip through
+- No visibility into background API health
+- Tests pass while features are broken
+
+The `network-error-monitor` provides:
+
+- **Automatic detection**: All HTTP 4xx/5xx responses tracked
+- **Test failures**: Fail tests with backend errors (even if UI passes)
+- **Structured artifacts**: JSON reports with error details
+- **Smart opt-out**: Disable for validation tests expecting errors
+- **Deduplication**: Group repeated errors by pattern
+- **Domino effect prevention**: Limit test failures per error pattern
+
+## Pattern Examples
+
+### Example 1: Basic Auto-Monitoring
+
+**Context**: Automatically fail tests when backend errors occur.
+
+**Implementation**:
+
+```typescript
+import { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+
+// Monitoring automatically enabled
+test('should load dashboard', async ({ page }) => {
+  await page.goto('/dashboard');
+  await expect(page.locator('h1')).toContainText('Dashboard');
+
+  // ✅ Passes if no HTTP errors
+  // ❌ Fails if any 4xx/5xx errors detected with clear message:
+  //    "Network errors detected: 2 request(s) failed"
+  //    Failed requests:
+  //      GET 500 https://api.example.com/users
+  //      POST 503 https://api.example.com/metrics
+});
+```
+
+**Key Points**:
+
+- Zero setup - auto-enabled for all tests
+- Fails on any 4xx/5xx response
+- Structured error message with URLs and status codes
+- JSON artifact attached to test report
+
+### Example 2: Opt-Out for Validation Tests
+
+**Context**: Some tests expect errors (validation, error handling, edge cases).
+
+**Implementation**:
+
+```typescript
+import { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+
+// Opt-out with annotation
+test('should show error on invalid input', { annotation: [{ type: 'skipNetworkMonitoring' }] }, async ({ page }) => {
+  await page.goto('/form');
+  await page.click('#submit'); // Triggers 400 error
+
+  // Monitoring disabled - test won't fail on 400
+  await expect(page.getByText('Invalid input')).toBeVisible();
+});
+
+// Or opt-out entire describe block
+test.describe('error handling', { annotation: [{ type: 'skipNetworkMonitoring' }] }, () => {
+  test('handles 404', async ({ page }) => {
+    // All tests in this block skip monitoring
+  });
+
+  test('handles 500', async ({ page }) => {
+    // Monitoring disabled
+  });
+});
+```
+
+**Key Points**:
+
+- Use annotation `{ type: 'skipNetworkMonitoring' }`
+- Can opt-out single test or entire describe block
+- Monitoring still active for other tests
+- Perfect for intentional error scenarios
+
+### Example 3: Integration with Merged Fixtures
+
+**Context**: Combine network-error-monitor with other utilities.
+
+**Implementation**:
+
+```typescript
+// playwright/support/merged-fixtures.ts
+import { mergeTests } from '@playwright/test';
+import { test as authFixture } from '@seontechnologies/playwright-utils/auth-session/fixtures';
+import { test as networkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+
+export const test = mergeTests(
+  authFixture,
+  networkErrorMonitorFixture,
+  // Add other fixtures
+);
+
+// In tests
+import { test, expect } from '../support/merged-fixtures';
+
+test('authenticated with monitoring', async ({ page, authToken }) => {
+  // Both auth and network monitoring active
+  await page.goto('/protected');
+
+  // Fails if backend returns errors during auth flow
+});
+```
+
+**Key Points**:
+
+- Combine with `mergeTests`
+- Works alongside all other utilities
+- Monitoring active automatically
+- No extra setup needed
+
+### Example 4: Domino Effect Prevention
+
+**Context**: One failing endpoint shouldn't fail all tests.
+
+**Implementation**:
+
+```typescript
+// Configuration (internal to utility)
+const config = {
+  maxTestsPerError: 3, // Max 3 tests fail per unique error pattern
+};
+
+// Scenario:
+// Test 1: GET /api/broken → 500 error → Test fails ❌
+// Test 2: GET /api/broken → 500 error → Test fails ❌
+// Test 3: GET /api/broken → 500 error → Test fails ❌
+// Test 4: GET /api/broken → 500 error → Test passes ⚠️ (limit reached, warning logged)
+// Test 5: Different error pattern → Test fails ❌ (new pattern, counter resets)
+```
+
+**Key Points**:
+
+- Limits cascading failures
+- Groups errors by URL + status code pattern
+- Warns when limit reached
+- Prevents flaky backend from failing entire suite
+
+### Example 5: Artifact Structure
+
+**Context**: Debugging failed tests with network error artifacts.
+
+**Implementation**:
+
+When test fails due to network errors, artifact attached:
+
+```json
+// test-results/my-test/network-errors.json
+{
+  "errors": [
+    {
+      "url": "https://api.example.com/users",
+      "method": "GET",
+      "status": 500,
+      "statusText": "Internal Server Error",
+      "timestamp": "2024-08-13T10:30:45.123Z"
+    },
+    {
+      "url": "https://api.example.com/metrics",
+      "method": "POST",
+      "status": 503,
+      "statusText": "Service Unavailable",
+      "timestamp": "2024-08-13T10:30:46.456Z"
+    }
+  ],
+  "summary": {
+    "totalErrors": 2,
+    "uniquePatterns": 2
+  }
+}
+```
+
+**Key Points**:
+
+- JSON artifact per failed test
+- Full error details (URL, method, status, timestamp)
+- Summary statistics
+- Easy debugging with structured data
+
+## Comparison with Manual Error Checks
+
+| Manual Approach                                        | network-error-monitor      |
+| ------------------------------------------------------ | -------------------------- |
+| `page.on('response', resp => { if (!resp.ok()) ... })` | Auto-enabled, zero setup   |
+| Check each response manually                           | Automatic for all requests |
+| Custom error tracking logic                            | Built-in deduplication     |
+| No structured artifacts                                | JSON artifacts attached    |
+| Easy to forget                                         | Never miss a backend error |
+
+## When to Use
+
+**Auto-enabled for:**
+
+- ✅ All E2E tests
+- ✅ Integration tests
+- ✅ Any test hitting real APIs
+
+**Opt-out for:**
+
+- ❌ Validation tests (expecting 4xx)
+- ❌ Error handling tests (expecting 5xx)
+- ❌ Offline tests (network-recorder playback)
+
+## Integration with Framework Setup
+
+In `*framework` workflow, mention network-error-monitor:
+
+```typescript
+// Add to merged-fixtures.ts
+import { test as networkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+
+export const test = mergeTests(
+  // ... other fixtures
+  networkErrorMonitorFixture,
+);
+```
+
+## Related Fragments
+
+- `overview.md` - Installation and fixtures
+- `fixtures-composition.md` - Merging with other utilities
+- `error-handling.md` - Traditional error handling patterns
+
+## Anti-Patterns
+
+**❌ Opting out of monitoring globally:**
+
+```typescript
+// Every test skips monitoring
+test.use({ annotation: [{ type: 'skipNetworkMonitoring' }] });
+```
+
+**✅ Opt-out only for specific error tests:**
+
+```typescript
+test.describe('error scenarios', { annotation: [{ type: 'skipNetworkMonitoring' }] }, () => {
+  // Only these tests skip monitoring
+});
+```
+
+**❌ Ignoring network error artifacts:**
+
+```typescript
+// Test fails, artifact shows 500 errors
+// Developer: "Works on my machine" ¯\_(ツ)_/¯
+```
+
+**✅ Check artifacts for root cause:**
+
+```typescript
+// Read network-errors.json artifact
+// Identify failing endpoint: GET /api/users → 500
+// Fix backend issue before merging
+```

package/src/modules/bmm/testarch/knowledge/network-recorder.md

@@ -0,0 +1,265 @@
+# Network Recorder Utility
+
+## Principle
+
+Record network traffic to HAR files during test execution, then play back from disk for offline testing. Enables frontend tests to run in complete isolation from backend services with intelligent stateful CRUD detection for realistic API behavior.
+
+## Rationale
+
+Traditional E2E tests require live backend services:
+
+- Slow (real network latency)
+- Flaky (backend instability affects tests)
+- Expensive (full stack running for UI tests)
+- Coupled (UI tests break when API changes)
+
+HAR-based recording/playback provides:
+
+- **True offline testing**: UI tests run without backend
+- **Deterministic behavior**: Same responses every time
+- **Fast execution**: No network latency
+- **Stateful mocking**: CRUD operations work naturally (not just read-only)
+- **Environment flexibility**: Map URLs for any environment
+
+## Pattern Examples
+
+### Example 1: Basic Record and Playback
+
+**Context**: The fundamental pattern - record traffic once, play back for all subsequent runs.
+
+**Implementation**:
+
+```typescript
+import { test } from '@seontechnologies/playwright-utils/network-recorder/fixtures';
+
+// Set mode in test file (recommended)
+process.env.PW_NET_MODE = 'playback'; // or 'record'
+
+test('CRUD operations work offline', async ({ page, context, networkRecorder }) => {
+  // Setup recorder (records or plays back based on PW_NET_MODE)
+  await networkRecorder.setup(context);
+
+  await page.goto('/');
+
+  // First time (record mode): Records all network traffic to HAR
+  // Subsequent runs (playback mode): Plays back from HAR (no backend!)
+  await page.fill('#movie-name', 'Inception');
+  await page.click('#add-movie');
+
+  // Intelligent CRUD detection makes this work offline!
+  await expect(page.getByText('Inception')).toBeVisible();
+});
+```
+
+**Key Points**:
+
+- `PW_NET_MODE=record` captures traffic to HAR files
+- `PW_NET_MODE=playback` replays from HAR files
+- Set mode in test file or via environment variable
+- HAR files auto-organized by test name
+- Stateful mocking detects CRUD operations
+
+### Example 2: Complete CRUD Flow with HAR
+
+**Context**: Full create-read-update-delete flow that works completely offline.
+
+**Implementation**:
+
+```typescript
+process.env.PW_NET_MODE = 'playback';
+
+test.describe('Movie CRUD - offline with network recorder', () => {
+  test.beforeEach(async ({ page, networkRecorder, context }) => {
+    await networkRecorder.setup(context);
+    await page.goto('/');
+  });
+
+  test('should add, edit, delete movie browser-only', async ({ page, interceptNetworkCall }) => {
+    // Create
+    await page.fill('#movie-name', 'Inception');
+    await page.fill('#year', '2010');
+    await page.click('#add-movie');
+
+    // Verify create (reads from stateful HAR)
+    await expect(page.getByText('Inception')).toBeVisible();
+
+    // Update
+    await page.getByText('Inception').click();
+    await page.fill('#movie-name', "Inception Director's Cut");
+
+    const updateCall = interceptNetworkCall({
+      method: 'PUT',
+      url: '/movies/*',
+    });
+
+    await page.click('#save');
+    await updateCall; // Wait for update
+
+    // Verify update (HAR reflects state change!)
+    await page.click('#back');
+    await expect(page.getByText("Inception Director's Cut")).toBeVisible();
+
+    // Delete
+    await page.click(`[data-testid="delete-Inception Director's Cut"]`);
+
+    // Verify delete (HAR reflects removal!)
+    await expect(page.getByText("Inception Director's Cut")).not.toBeVisible();
+  });
+});
+```
+
+**Key Points**:
+
+- Full CRUD operations work offline
+- Stateful HAR mocking tracks creates/updates/deletes
+- Combine with `interceptNetworkCall` for deterministic waits
+- First run records, subsequent runs replay
+
+### Example 3: Environment Switching
+
+**Context**: Record in dev environment, play back in CI with different base URLs.
+
+**Implementation**:
+
+```typescript
+// playwright.config.ts - Map URLs for different environments
+export default defineConfig({
+  use: {
+    baseURL: process.env.CI ? 'https://app.ci.example.com' : 'http://localhost:3000',
+  },
+});
+
+// Test works in both environments
+test('cross-environment playback', async ({ page, context, networkRecorder }) => {
+  await networkRecorder.setup(context);
+
+  // In dev: hits http://localhost:3000/api/movies
+  // In CI: HAR replays with https://app.ci.example.com/api/movies
+  await page.goto('/movies');
+
+  // Network recorder auto-maps URLs
+  await expect(page.getByTestId('movie-list')).toBeVisible();
+});
+```
+
+**Key Points**:
+
+- HAR files record absolute URLs
+- Playback maps to current baseURL
+- Same HAR works across environments
+- No manual URL rewriting needed
+
+### Example 4: Automatic vs Manual Mode Control
+
+**Context**: Choose between environment-based switching or in-test mode control.
+
+**Implementation**:
+
+```typescript
+// Option 1: Environment variable (recommended for CI)
+PW_NET_MODE=record npm run test:pw   # Record traffic
+PW_NET_MODE=playback npm run test:pw # Playback traffic
+
+// Option 2: In-test control (recommended for development)
+process.env.PW_NET_MODE = 'record'  // Set at top of test file
+
+test('my test', async ({ page, context, networkRecorder }) => {
+  await networkRecorder.setup(context)
+  // ...
+})
+
+// Option 3: Auto-fallback (record if HAR missing, else playback)
+// This is the default behavior when PW_NET_MODE not set
+test('auto mode', async ({ page, context, networkRecorder }) => {
+  await networkRecorder.setup(context)
+  // First run: auto-records
+  // Subsequent runs: auto-plays back
+})
+```
+
+**Key Points**:
+
+- Three mode options: record, playback, auto
+- `PW_NET_MODE` environment variable
+- In-test `process.env.PW_NET_MODE` assignment
+- Auto-fallback when no mode specified
+
+## Why Use This Instead of Native Playwright?
+
+| Native Playwright (`routeFromHAR`) | network-recorder Utility       |
+| ---------------------------------- | ------------------------------ |
+| ~80 lines setup boilerplate        | ~5 lines total                 |
+| Manual HAR file management         | Automatic file organization    |
+| Complex setup/teardown             | Automatic cleanup via fixtures |
+| **Read-only tests**                | **Full CRUD support**          |
+| **Stateless**                      | **Stateful mocking**           |
+| Manual URL mapping                 | Automatic environment mapping  |
+
+**The game-changer: Stateful CRUD detection**
+
+Native Playwright HAR playback is stateless - a POST create followed by GET list won't show the created item. This utility intelligently tracks CRUD operations in memory to reflect state changes, making offline tests behave like real APIs.
+
+## Integration with Other Utilities
+
+**With interceptNetworkCall** (deterministic waits):
+
+```typescript
+test('use both utilities', async ({ page, context, networkRecorder, interceptNetworkCall }) => {
+  await networkRecorder.setup(context);
+
+  const createCall = interceptNetworkCall({
+    method: 'POST',
+    url: '/api/movies',
+  });
+
+  await page.click('#add-movie');
+  await createCall; // Wait for create (works with HAR!)
+
+  // Network recorder provides playback, intercept provides determinism
+});
+```
+
+## Related Fragments
+
+- `overview.md` - Installation and fixture patterns
+- `intercept-network-call.md` - Combine for deterministic offline tests
+- `auth-session.md` - Record authenticated traffic
+- `network-first.md` - Core pattern for intercept-before-navigate
+
+## Anti-Patterns
+
+**❌ Mixing record and playback in same test:**
+
+```typescript
+process.env.PW_NET_MODE = 'record';
+// ... some test code ...
+process.env.PW_NET_MODE = 'playback'; // Don't switch mid-test
+```
+
+**✅ One mode per test:**
+
+```typescript
+process.env.PW_NET_MODE = 'playback'; // Set once at top
+
+test('my test', async ({ page, context, networkRecorder }) => {
+  await networkRecorder.setup(context);
+  // Entire test uses playback mode
+});
+```
+
+**❌ Forgetting to call setup:**
+
+```typescript
+test('broken', async ({ page, networkRecorder }) => {
+  await page.goto('/'); // HAR not active!
+});
+```
+
+**✅ Always call setup before navigation:**
+
+```typescript
+test('correct', async ({ page, context, networkRecorder }) => {
+  await networkRecorder.setup(context); // Must setup first
+  await page.goto('/'); // Now HAR is active
+});
+```