@zeyue0329/xiaoma-cli 1.6.4 → 1.7.0

package/src/xmc/testarch/knowledge/playwright-config.md

@@ -1,730 +0,0 @@
-# Playwright Configuration Guardrails
-
-## Principle
-
-Load environment configs via a central map (`envConfigMap`), standardize timeouts (action 15s, navigation 30s, expect 10s, test 60s), emit HTML + JUnit reporters, and store artifacts under `test-results/` for CI upload. Keep `.env.example`, `.nvmrc`, and browser dependencies versioned so local and CI runs stay aligned.
-
-## Rationale
-
-Environment-specific configuration prevents hardcoded URLs, timeouts, and credentials from leaking into tests. A central config map with fail-fast validation catches missing environments early. Standardized timeouts reduce flakiness while remaining long enough for real-world network conditions. Consistent artifact storage (`test-results/`, `playwright-report/`) enables CI pipelines to upload failure evidence automatically. Versioned dependencies (`.nvmrc`, `package.json` browser versions) eliminate "works on my machine" issues between local and CI environments.
-
-## Pattern Examples
-
-### Example 1: Environment-Based Configuration
-
-**Context**: When testing against multiple environments (local, staging, production), use a central config map that loads environment-specific settings and fails fast if `TEST_ENV` is invalid.
-
-**Implementation**:
-
-```typescript
-// playwright.config.ts - Central config loader
-import { config as dotenvConfig } from 'dotenv';
-import path from 'path';
-
-// Load .env from project root
-dotenvConfig({
-  path: path.resolve(__dirname, '../../.env'),
-});
-
-// Central environment config map
-const envConfigMap = {
-  local: require('./playwright/config/local.config').default,
-  staging: require('./playwright/config/staging.config').default,
-  production: require('./playwright/config/production.config').default,
-};
-
-const environment = process.env.TEST_ENV || 'local';
-
-// Fail fast if environment not supported
-if (!Object.keys(envConfigMap).includes(environment)) {
-  console.error(`❌ No configuration found for environment: ${environment}`);
-  console.error(`   Available environments: ${Object.keys(envConfigMap).join(', ')}`);
-  process.exit(1);
-}
-
-console.log(`✅ Running tests against: ${environment.toUpperCase()}`);
-
-export default envConfigMap[environment as keyof typeof envConfigMap];
-```
-
-```typescript
-// playwright/config/base.config.ts - Shared base configuration
-import { defineConfig } from '@playwright/test';
-import path from 'path';
-
-export const baseConfig = defineConfig({
-  testDir: path.resolve(__dirname, '../tests'),
-  outputDir: path.resolve(__dirname, '../../test-results'),
-  fullyParallel: true,
-  forbidOnly: !!process.env.CI,
-  retries: process.env.CI ? 2 : 0,
-  workers: process.env.CI ? 1 : undefined,
-  reporter: [
-    ['html', { outputFolder: 'playwright-report', open: 'never' }],
-    ['junit', { outputFile: 'test-results/results.xml' }],
-    ['list'],
-  ],
-  use: {
-    actionTimeout: 15000,
-    navigationTimeout: 30000,
-    trace: 'on-first-retry',
-    screenshot: 'only-on-failure',
-    video: 'retain-on-failure',
-  },
-  globalSetup: path.resolve(__dirname, '../support/global-setup.ts'),
-  timeout: 60000,
-  expect: { timeout: 10000 },
-});
-```
-
-```typescript
-// playwright/config/local.config.ts - Local environment
-import { defineConfig } from '@playwright/test';
-import { baseConfig } from './base.config';
-
-export default defineConfig({
-  ...baseConfig,
-  use: {
-    ...baseConfig.use,
-    baseURL: 'http://localhost:3000',
-    video: 'off', // No video locally for speed
-  },
-  webServer: {
-    command: 'npm run dev',
-    url: 'http://localhost:3000',
-    reuseExistingServer: !process.env.CI,
-    timeout: 120000,
-  },
-});
-```
-
-```typescript
-// playwright/config/staging.config.ts - Staging environment
-import { defineConfig } from '@playwright/test';
-import { baseConfig } from './base.config';
-
-export default defineConfig({
-  ...baseConfig,
-  use: {
-    ...baseConfig.use,
-    baseURL: 'https://staging.example.com',
-    ignoreHTTPSErrors: true, // Allow self-signed certs in staging
-  },
-});
-```
-
-```typescript
-// playwright/config/production.config.ts - Production environment
-import { defineConfig } from '@playwright/test';
-import { baseConfig } from './base.config';
-
-export default defineConfig({
-  ...baseConfig,
-  retries: 3, // More retries in production
-  use: {
-    ...baseConfig.use,
-    baseURL: 'https://example.com',
-    video: 'on', // Always record production failures
-  },
-});
-```
-
-```bash
-# .env.example - Template for developers
-TEST_ENV=local
-API_KEY=your_api_key_here
-DATABASE_URL=postgresql://localhost:5432/test_db
-```
-
-**Key Points**:
-
-- Central `envConfigMap` prevents environment misconfiguration
-- Fail-fast validation with clear error message (available envs listed)
-- Base config defines shared settings, environment configs override
-- `.env.example` provides template for required secrets
-- `TEST_ENV=local` as default for local development
-- Production config increases retries and enables video recording
-
-### Example 2: Timeout Standards
-
-**Context**: When tests fail due to inconsistent timeout settings, standardize timeouts across all tests: action 15s, navigation 30s, expect 10s, test 60s. Expose overrides through fixtures rather than inline literals.
-
-**Implementation**:
-
-```typescript
-// playwright/config/base.config.ts - Standardized timeouts
-import { defineConfig } from '@playwright/test';
-
-export default defineConfig({
-  // Global test timeout: 60 seconds
-  timeout: 60000,
-
-  use: {
-    // Action timeout: 15 seconds (click, fill, etc.)
-    actionTimeout: 15000,
-
-    // Navigation timeout: 30 seconds (page.goto, page.reload)
-    navigationTimeout: 30000,
-  },
-
-  // Expect timeout: 10 seconds (all assertions)
-  expect: {
-    timeout: 10000,
-  },
-});
-```
-
-```typescript
-// playwright/support/fixtures/timeout-fixture.ts - Timeout override fixture
-import { test as base } from '@playwright/test';
-
-type TimeoutOptions = {
-  extendedTimeout: (timeoutMs: number) => Promise<void>;
-};
-
-export const test = base.extend<TimeoutOptions>({
-  extendedTimeout: async ({}, use, testInfo) => {
-    const originalTimeout = testInfo.timeout;
-
-    await use(async (timeoutMs: number) => {
-      testInfo.setTimeout(timeoutMs);
-    });
-
-    // Restore original timeout after test
-    testInfo.setTimeout(originalTimeout);
-  },
-});
-
-export { expect } from '@playwright/test';
-```
-
-```typescript
-// Usage in tests - Standard timeouts (implicit)
-import { test, expect } from '@playwright/test';
-
-test('user can log in', async ({ page }) => {
-  await page.goto('/login'); // Uses 30s navigation timeout
-  await page.fill('[data-testid="email"]', 'test@example.com'); // Uses 15s action timeout
-  await page.click('[data-testid="login-button"]'); // Uses 15s action timeout
-
-  await expect(page.getByText('Welcome')).toBeVisible(); // Uses 10s expect timeout
-});
-```
-
-```typescript
-// Usage in tests - Per-test timeout override
-import { test, expect } from '../support/fixtures/timeout-fixture';
-
-test('slow data processing operation', async ({ page, extendedTimeout }) => {
-  // Override default 60s timeout for this slow test
-  await extendedTimeout(180000); // 3 minutes
-
-  await page.goto('/data-processing');
-  await page.click('[data-testid="process-large-file"]');
-
-  // Wait for long-running operation
-  await expect(page.getByText('Processing complete')).toBeVisible({
-    timeout: 120000, // 2 minutes for assertion
-  });
-});
-```
-
-```typescript
-// Per-assertion timeout override (inline)
-test('API returns quickly', async ({ page }) => {
-  await page.goto('/dashboard');
-
-  // Override expect timeout for fast API (reduce flakiness detection)
-  await expect(page.getByTestId('user-name')).toBeVisible({ timeout: 5000 }); // 5s instead of 10s
-
-  // Override expect timeout for slow external API
-  await expect(page.getByTestId('weather-widget')).toBeVisible({ timeout: 20000 }); // 20s instead of 10s
-});
-```
-
-**Key Points**:
-
-- **Standardized timeouts**: action 15s, navigation 30s, expect 10s, test 60s (global defaults)
-- Fixture-based override (`extendedTimeout`) for slow tests (preferred over inline)
-- Per-assertion timeout override via `{ timeout: X }` option (use sparingly)
-- Avoid hard waits (`page.waitForTimeout(3000)`) - use event-based waits instead
-- CI environments may need longer timeouts (handle in environment-specific config)
-
-### Example 3: Artifact Output Configuration
-
-**Context**: When debugging failures in CI, configure artifacts (screenshots, videos, traces, HTML reports) to be captured on failure and stored in consistent locations for upload.
-
-**Implementation**:
-
-```typescript
-// playwright.config.ts - Artifact configuration
-import { defineConfig } from '@playwright/test';
-import path from 'path';
-
-export default defineConfig({
-  // Output directory for test artifacts
-  outputDir: path.resolve(__dirname, './test-results'),
-
-  use: {
-    // Screenshot on failure only (saves space)
-    screenshot: 'only-on-failure',
-
-    // Video recording on failure + retry
-    video: 'retain-on-failure',
-
-    // Trace recording on first retry (best debugging data)
-    trace: 'on-first-retry',
-  },
-
-  reporter: [
-    // HTML report (visual, interactive)
-    [
-      'html',
-      {
-        outputFolder: 'playwright-report',
-        open: 'never', // Don't auto-open in CI
-      },
-    ],
-
-    // JUnit XML (CI integration)
-    [
-      'junit',
-      {
-        outputFile: 'test-results/results.xml',
-      },
-    ],
-
-    // List reporter (console output)
-    ['list'],
-  ],
-});
-```
-
-```typescript
-// playwright/support/fixtures/artifact-fixture.ts - Custom artifact capture
-import { test as base } from '@playwright/test';
-import fs from 'fs';
-import path from 'path';
-
-export const test = base.extend({
-  // Auto-capture console logs on failure
-  page: async ({ page }, use, testInfo) => {
-    const logs: string[] = [];
-
-    page.on('console', (msg) => {
-      logs.push(`[${msg.type()}] ${msg.text()}`);
-    });
-
-    await use(page);
-
-    // Save logs on failure
-    if (testInfo.status !== testInfo.expectedStatus) {
-      const logsPath = path.join(testInfo.outputDir, 'console-logs.txt');
-      fs.writeFileSync(logsPath, logs.join('\n'));
-      testInfo.attachments.push({
-        name: 'console-logs',
-        contentType: 'text/plain',
-        path: logsPath,
-      });
-    }
-  },
-});
-```
-
-```yaml
-# .github/workflows/e2e.yml - CI artifact upload
-name: E2E Tests
-on: [push, pull_request]
-
-jobs:
-  test:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-node@v4
-        with:
-          node-version-file: '.nvmrc'
-
-      - name: Install dependencies
-        run: npm ci
-
-      - name: Install Playwright browsers
-        run: npx playwright install --with-deps
-
-      - name: Run tests
-        run: npm run test
-        env:
-          TEST_ENV: staging
-
-      # Upload test artifacts on failure
-      - name: Upload test results
-        if: failure()
-        uses: actions/upload-artifact@v4
-        with:
-          name: test-results
-          path: test-results/
-          retention-days: 30
-
-      - name: Upload Playwright report
-        if: failure()
-        uses: actions/upload-artifact@v4
-        with:
-          name: playwright-report
-          path: playwright-report/
-          retention-days: 30
-```
-
-```typescript
-// Example: Custom screenshot on specific condition
-test('capture screenshot on specific error', async ({ page }) => {
-  await page.goto('/checkout');
-
-  try {
-    await page.click('[data-testid="submit-payment"]');
-    await expect(page.getByText('Order Confirmed')).toBeVisible();
-  } catch (error) {
-    // Capture custom screenshot with timestamp
-    await page.screenshot({
-      path: `test-results/payment-error-${Date.now()}.png`,
-      fullPage: true,
-    });
-    throw error;
-  }
-});
-```
-
-**Key Points**:
-
-- `screenshot: 'only-on-failure'` saves space (not every test)
-- `video: 'retain-on-failure'` captures full flow on failures
-- `trace: 'on-first-retry'` provides deep debugging data (network, DOM, console)
-- HTML report at `playwright-report/` (visual debugging)
-- JUnit XML at `test-results/results.xml` (CI integration)
-- CI uploads artifacts on failure with 30-day retention
-- Custom fixture can capture console logs, network logs, etc.
-
-### Example 4: Parallelization Configuration
-
-**Context**: When tests run slowly in CI, configure parallelization with worker count, sharding, and fully parallel execution to maximize speed while maintaining stability.
-
-**Implementation**:
-
-```typescript
-// playwright.config.ts - Parallelization settings
-import { defineConfig } from '@playwright/test';
-import os from 'os';
-
-export default defineConfig({
-  // Run tests in parallel within single file
-  fullyParallel: true,
-
-  // Worker configuration
-  workers: process.env.CI
-    ? 1 // Serial in CI for stability (or 2 for faster CI)
-    : os.cpus().length - 1, // Parallel locally (leave 1 CPU for OS)
-
-  // Prevent accidentally committed .only() from blocking CI
-  forbidOnly: !!process.env.CI,
-
-  // Retry failed tests in CI
-  retries: process.env.CI ? 2 : 0,
-
-  // Shard configuration (split tests across multiple machines)
-  shard:
-    process.env.SHARD_INDEX && process.env.SHARD_TOTAL
-      ? {
-          current: parseInt(process.env.SHARD_INDEX, 10),
-          total: parseInt(process.env.SHARD_TOTAL, 10),
-        }
-      : undefined,
-});
-```
-
-```yaml
-# .github/workflows/e2e-parallel.yml - Sharded CI execution
-name: E2E Tests (Parallel)
-on: [push, pull_request]
-
-jobs:
-  test:
-    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        shard: [1, 2, 3, 4] # Split tests across 4 machines
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-node@v4
-        with:
-          node-version-file: '.nvmrc'
-
-      - name: Install dependencies
-        run: npm ci
-
-      - name: Install Playwright browsers
-        run: npx playwright install --with-deps
-
-      - name: Run tests (shard ${{ matrix.shard }})
-        run: npm run test
-        env:
-          SHARD_INDEX: ${{ matrix.shard }}
-          SHARD_TOTAL: 4
-          TEST_ENV: staging
-
-      - name: Upload test results
-        if: failure()
-        uses: actions/upload-artifact@v4
-        with:
-          name: test-results-shard-${{ matrix.shard }}
-          path: test-results/
-```
-
-```typescript
-// playwright/config/serial.config.ts - Serial execution for flaky tests
-import { defineConfig } from '@playwright/test';
-import { baseConfig } from './base.config';
-
-export default defineConfig({
-  ...baseConfig,
-
-  // Disable parallel execution
-  fullyParallel: false,
-  workers: 1,
-
-  // Used for: authentication flows, database-dependent tests, feature flag tests
-});
-```
-
-```typescript
-// Usage: Force serial execution for specific tests
-import { test } from '@playwright/test';
-
-// Serial execution for auth tests (shared session state)
-test.describe.configure({ mode: 'serial' });
-
-test.describe('Authentication Flow', () => {
-  test('user can log in', async ({ page }) => {
-    // First test in serial block
-  });
-
-  test('user can access dashboard', async ({ page }) => {
-    // Depends on previous test (serial)
-  });
-});
-```
-
-```typescript
-// Usage: Parallel execution for independent tests (default)
-import { test } from '@playwright/test';
-
-test.describe('Product Catalog', () => {
-  test('can view product 1', async ({ page }) => {
-    // Runs in parallel with other tests
-  });
-
-  test('can view product 2', async ({ page }) => {
-    // Runs in parallel with other tests
-  });
-});
-```
-
-**Key Points**:
-
-- `fullyParallel: true` enables parallel execution within single test file
-- Workers: 1 in CI (stability), N-1 CPUs locally (speed)
-- Sharding splits tests across multiple CI machines (4x faster with 4 shards)
-- `test.describe.configure({ mode: 'serial' })` for dependent tests
-- `forbidOnly: true` in CI prevents `.only()` from blocking pipeline
-- Matrix strategy in CI runs shards concurrently
-
-### Example 5: Project Configuration
-
-**Context**: When testing across multiple browsers, devices, or configurations, use Playwright projects to run the same tests against different environments (chromium, firefox, webkit, mobile).
-
-**Implementation**:
-
-```typescript
-// playwright.config.ts - Multiple browser projects
-import { defineConfig, devices } from '@playwright/test';
-
-export default defineConfig({
-  projects: [
-    // Desktop browsers
-    {
-      name: 'chromium',
-      use: { ...devices['Desktop Chrome'] },
-    },
-    {
-      name: 'firefox',
-      use: { ...devices['Desktop Firefox'] },
-    },
-    {
-      name: 'webkit',
-      use: { ...devices['Desktop Safari'] },
-    },
-
-    // Mobile browsers
-    {
-      name: 'mobile-chrome',
-      use: { ...devices['Pixel 5'] },
-    },
-    {
-      name: 'mobile-safari',
-      use: { ...devices['iPhone 13'] },
-    },
-
-    // Tablet
-    {
-      name: 'tablet',
-      use: { ...devices['iPad Pro'] },
-    },
-  ],
-});
-```
-
-```typescript
-// playwright.config.ts - Authenticated vs. unauthenticated projects
-import { defineConfig } from '@playwright/test';
-import path from 'path';
-
-export default defineConfig({
-  projects: [
-    // Setup project (runs first, creates auth state)
-    {
-      name: 'setup',
-      testMatch: /global-setup\.ts/,
-    },
-
-    // Authenticated tests (reuse auth state)
-    {
-      name: 'authenticated',
-      dependencies: ['setup'],
-      use: {
-        storageState: path.resolve(__dirname, './playwright/.auth/user.json'),
-      },
-      testMatch: /.*authenticated\.spec\.ts/,
-    },
-
-    // Unauthenticated tests (public pages)
-    {
-      name: 'unauthenticated',
-      testMatch: /.*unauthenticated\.spec\.ts/,
-    },
-  ],
-});
-```
-
-```typescript
-// playwright/support/global-setup.ts - Setup project for auth
-import { chromium, FullConfig } from '@playwright/test';
-import path from 'path';
-
-async function globalSetup(config: FullConfig) {
-  const browser = await chromium.launch();
-  const page = await browser.newPage();
-
-  // Perform authentication
-  await page.goto('http://localhost:3000/login');
-  await page.fill('[data-testid="email"]', 'test@example.com');
-  await page.fill('[data-testid="password"]', 'password123');
-  await page.click('[data-testid="login-button"]');
-
-  // Wait for authentication to complete
-  await page.waitForURL('**/dashboard');
-
-  // Save authentication state
-  await page.context().storageState({
-    path: path.resolve(__dirname, '../.auth/user.json'),
-  });
-
-  await browser.close();
-}
-
-export default globalSetup;
-```
-
-```bash
-# Run specific project
-npx playwright test --project=chromium
-npx playwright test --project=mobile-chrome
-npx playwright test --project=authenticated
-
-# Run multiple projects
-npx playwright test --project=chromium --project=firefox
-
-# Run all projects (default)
-npx playwright test
-```
-
-```typescript
-// Usage: Project-specific test
-import { test, expect } from '@playwright/test';
-
-test('mobile navigation works', async ({ page, isMobile }) => {
-  await page.goto('/');
-
-  if (isMobile) {
-    // Open mobile menu
-    await page.click('[data-testid="hamburger-menu"]');
-  }
-
-  await page.click('[data-testid="products-link"]');
-  await expect(page).toHaveURL(/.*products/);
-});
-```
-
-```yaml
-# .github/workflows/e2e-cross-browser.yml - CI cross-browser testing
-name: E2E Tests (Cross-Browser)
-on: [push, pull_request]
-
-jobs:
-  test:
-    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        project: [chromium, firefox, webkit, mobile-chrome]
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-node@v4
-      - run: npm ci
-      - run: npx playwright install --with-deps
-
-      - name: Run tests (${{ matrix.project }})
-        run: npx playwright test --project=${{ matrix.project }}
-```
-
-**Key Points**:
-
-- Projects enable testing across browsers, devices, and configurations
-- `devices` from `@playwright/test` provide preset configurations (Pixel 5, iPhone 13, etc.)
-- `dependencies` ensures setup project runs first (auth, data seeding)
-- `storageState` shares authentication across tests (0 seconds auth per test)
-- `testMatch` filters which tests run in which project
-- CI matrix strategy runs projects in parallel (4x faster with 4 projects)
-- `isMobile` context property for conditional logic in tests
-
-## Integration Points
-
-- **Used in workflows**: `*framework` (config setup), `*ci` (parallelization, artifact upload)
-- **Related fragments**:
-  - `fixture-architecture.md` - Fixture-based timeout overrides
-  - `ci-burn-in.md` - CI pipeline artifact upload
-  - `test-quality.md` - Timeout standards (no hard waits)
-  - `data-factories.md` - Per-test isolation (no shared global state)
-
-## Configuration Checklist
-
-**Before deploying tests, verify**:
-
-- [ ] Environment config map with fail-fast validation
-- [ ] Standardized timeouts (action 15s, navigation 30s, expect 10s, test 60s)
-- [ ] Artifact storage at `test-results/` and `playwright-report/`
-- [ ] HTML + JUnit reporters configured
-- [ ] `.env.example`, `.nvmrc`, browser versions committed
-- [ ] Parallelization configured (workers, sharding)
-- [ ] Projects defined for cross-browser/device testing (if needed)
-- [ ] CI uploads artifacts on failure with 30-day retention
-
-_Source: Playwright book repo, SEON configuration example, Murat testing philosophy (lines 216-271)._