qa-workflow-cc 1.0.0

package/skills/qa/templates/test-standards.md

@@ -0,0 +1,652 @@
+# Testing Standards
+
+> Generic testing standards template. Load this skill when writing tests. Not needed every session.
+
+## Template Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `{{testRunner}}` | Test runner name (vitest, jest) | vitest |
+| `{{testCommand}}` | Command to run tests | pnpm test |
+| `{{testCoverageCommand}}` | Command for coverage report | pnpm test:coverage |
+| `{{e2eCommand}}` | Command to run E2E tests | pnpm e2e |
+| `{{coverageThresholds}}` | Coverage threshold block | See defaults below |
+| `{{testCommands}}` | Project-specific test commands block | See defaults below |
+
+---
+
+## Coverage Thresholds
+
+### Global Defaults
+
+{{coverageThresholds}}
+
+| Metric | Threshold |
+|--------|-----------|
+| Statements | 80% |
+| Branches | 70% |
+| Functions | 80% |
+| Lines | 80% |
+
+### Coverage by Code Area
+
+Define coverage targets by area based on criticality:
+
+| Area | Target | Priority | Rationale |
+|------|--------|----------|-----------|
+| API / Server Routes | 90% | CRITICAL | Data integrity, security boundaries |
+| Authentication / Authorization | 95% | CRITICAL | Security-critical code paths |
+| Data Access Layer | 85% | HIGH | Database queries, data transformations |
+| Utility Functions | 85% | HIGH | Widely reused, high leverage |
+| Hooks / State Management | 80% | HIGH | Complex state logic |
+| UI Components | 70% | MEDIUM | Visual correctness via snapshot + behavior tests |
+| E2E Flows | Critical paths only | CRITICAL | User journeys that must never break |
+| Configuration / Constants | 50% | LOW | Mostly static, low risk |
+
+> **Project-specific override:** Replace the table above with your project's actual coverage targets during bootstrapping.
+
+---
+
+## Test Type Requirements
+
+### Unit Tests (Required for all new logic)
+
+Every new utility function, hook, or pure logic module MUST have unit tests.
+
+**File naming:** `[module].test.ts` or `__tests__/[module].test.ts`
+
+**What to test:**
+- Input/output mapping for all cases (happy path, edge cases, errors)
+- Boundary conditions (empty, null, undefined, max values)
+- Error throwing and error messages
+- Default values and optional parameters
+
+**Example areas:**
+- Utility functions (formatters, validators, calculators)
+- Custom hooks (state changes, effects, cleanup)
+- Data transformation functions
+- Schema validation logic (Zod, Yup, etc.)
+- Business logic modules
+
+### API / Router Tests (Required for all endpoints)
+
+Every API endpoint or server-side route MUST have integration tests covering:
+
+- CRUD operations (create, read, update, delete)
+- Input validation (valid, invalid, edge cases)
+- Authorization checks (authenticated vs. unauthenticated)
+- Multi-tenancy isolation (if applicable -- data scoped to organization/user)
+- Error handling (not found, forbidden, conflict, bad request)
+- Pagination (cursor-based or offset-based)
+
+**File naming:** `routers/__tests__/[router].test.ts` or `routes/__tests__/[route].test.ts`
+
+### Component Tests (Required for complex components)
+
+Cover:
+- Rendering with different props (variants, states)
+- User interactions (clicks, form submission, keyboard)
+- Loading, error, and empty states
+- Accessibility (ARIA attributes, keyboard navigation)
+- Conditional rendering logic
+
+**Use Testing Library patterns, not implementation details.**
+
+**File naming:** `components/__tests__/[Component].test.tsx`
+
+### E2E Tests (Required for critical flows)
+
+Cover the critical user journeys that MUST NOT break:
+
+- Authentication flow (sign up, sign in, sign out)
+- Primary CRUD operations (create, edit, delete core entities)
+- Navigation between major sections
+- Form wizards / multi-step flows
+- Payment / checkout flows (if applicable)
+- File upload flows (if applicable)
+
+**File naming:** `e2e/[flow-name].spec.ts`
+
+---
+
+## Test Patterns
+
+### DO (Recommended Patterns)
+
+| Pattern | Description |
+|---------|-------------|
+| Semantic queries | `getByRole`, `getByLabelText`, `getByText` first |
+| `userEvent` | Simulates real user interactions (not `fireEvent` for web) |
+| `waitFor` / `findBy` | Handles async rendering and data fetching |
+| Behavior testing | Test what the user sees and does, not internal state |
+| `describe` blocks | Group related tests for readability |
+| `it.each` / parameterized | Reduce duplication for similar test cases |
+| Factory functions | Create test data with sensible defaults |
+| Custom render | Wrap providers in test-utils for consistent setup |
+| Mock cleanup | `vi.clearAllMocks()` or `jest.clearAllMocks()` in `beforeEach` |
+| Fake timers | `vi.useFakeTimers()` for time-dependent code |
+
+### DON'T (Anti-Patterns)
+
+| Anti-Pattern | Why |
+|--------------|-----|
+| `getByClassName` | Implementation detail, breaks on refactor |
+| `getByTestId` as default | Use semantic queries first, testId as last resort |
+| Hardcoded waits (`sleep(2000)`) | Flaky, slow -- use `waitFor` or `findBy` |
+| Testing internal state | Test behavior, not `useState` values |
+| Testing implementation details | Internal methods, private functions, CSS classes |
+| Snapshot-only tests | Snapshots detect changes but don't verify correctness |
+| Shared mutable state | Each test must be independent |
+| `any` in test types | Weakens test reliability |
+| Console.log debugging left in | Clean up before committing |
+
+---
+
+## Multi-Tenancy / Isolation Testing (CRITICAL)
+
+If your application is multi-tenant (organizations, workspaces, teams), every database query test MUST verify isolation:
+
+```typescript
+describe('data isolation', () => {
+  it('does not return other tenant data', async () => {
+    // Create data for Org A
+    const itemA = await createItem({ orgId: 'org-a', name: 'Item A' })
+
+    // Query as Org B -- should NOT see Org A's data
+    const result = await listItems({ orgId: 'org-b' })
+
+    expect(result.items).not.toContainEqual(
+      expect.objectContaining({ id: itemA.id })
+    )
+  })
+
+  it('rejects access to other tenant resources', async () => {
+    const item = await createItem({ orgId: 'org-a', name: 'Private Item' })
+
+    // Try to access as different org
+    await expect(
+      getItem({ orgId: 'org-b', itemId: item.id })
+    ).rejects.toThrow() // Should throw FORBIDDEN or NOT_FOUND
+  })
+
+  it('scopes mutations to current tenant', async () => {
+    const item = await createItem({ orgId: 'org-a', name: 'Original' })
+
+    // Try to update as different org
+    await expect(
+      updateItem({ orgId: 'org-b', itemId: item.id, name: 'Hacked' })
+    ).rejects.toThrow()
+
+    // Verify original is unchanged
+    const unchanged = await getItem({ orgId: 'org-a', itemId: item.id })
+    expect(unchanged.name).toBe('Original')
+  })
+})
+```
+
+**Checklist for isolation:**
+- [ ] All list queries filter by tenant identifier
+- [ ] All single-resource queries verify tenant ownership
+- [ ] All mutations verify tenant ownership before modifying
+- [ ] Aggregation queries (counts, sums) scope to tenant
+- [ ] Search/filter queries scope to tenant
+
+---
+
+## Test Data Patterns
+
+### Factory Functions
+
+```typescript
+// test/factories.ts
+let idCounter = 0
+
+export function createTestUser(overrides: Partial<User> = {}): User {
+  idCounter++
+  return {
+    id: `user-${idCounter}`,
+    name: `Test User ${idCounter}`,
+    email: `user${idCounter}@test.com`,
+    role: 'member',
+    createdAt: new Date(),
+    ...overrides,
+  }
+}
+
+export function createTestItem(overrides: Partial<Item> = {}): Item {
+  idCounter++
+  return {
+    id: `item-${idCounter}`,
+    name: `Test Item ${idCounter}`,
+    status: 'DRAFT',
+    createdAt: new Date(),
+    ...overrides,
+  }
+}
+
+// Usage
+const user = createTestUser({ role: 'admin' })
+const item = createTestItem({ status: 'ACTIVE', name: 'Custom Name' })
+```
+
+### Builder Pattern (For Complex Objects)
+
+```typescript
+export class TestItemBuilder {
+  private item: Partial<Item> = {}
+
+  withName(name: string) { this.item.name = name; return this }
+  withStatus(status: string) { this.item.status = status; return this }
+  withOwner(userId: string) { this.item.ownerId = userId; return this }
+  withTags(tags: string[]) { this.item.tags = tags; return this }
+
+  build(): Item {
+    return createTestItem(this.item)
+  }
+}
+
+// Usage
+const item = new TestItemBuilder()
+  .withName('Important')
+  .withStatus('ACTIVE')
+  .withTags(['urgent', 'client'])
+  .build()
+```
+
+---
+
+## Test Organization
+
+### Directory Structure
+
+```
+project/
+  src/
+    __tests__/           # Unit tests (mirrors src/ structure)
+      utils/
+        currency.test.ts
+        date.test.ts
+      hooks/
+        useDebounce.test.ts
+      services/
+        email.test.ts
+    components/
+      __tests__/         # Component tests (co-located)
+        ItemCard.test.tsx
+        Dashboard.test.tsx
+  e2e/                   # E2E tests
+    fixtures/
+      auth.ts
+      testData.ts
+    pages/               # Page objects
+      ItemsPage.ts
+      DashboardPage.ts
+    item-management.spec.ts
+    auth-flow.spec.ts
+  test/                  # Test utilities and setup
+    setup.ts
+    factories.ts
+    test-utils.tsx
+    mocks/
+      handlers.ts
+```
+
+### Test Naming Guidelines
+
+```typescript
+// Describe blocks: noun (the thing being tested)
+describe('formatCurrency', () => { ... })
+describe('ItemCard', () => { ... })
+describe('useDebounce', () => { ... })
+
+// Test names: "it [does something expected]"
+it('formats positive numbers with commas', () => { ... })
+it('renders item name and status', () => { ... })
+it('debounces rapid value changes', () => { ... })
+
+// Group by scenario
+describe('ItemCard', () => {
+  describe('when item is active', () => { ... })
+  describe('when item is archived', () => { ... })
+  describe('user interactions', () => { ... })
+  describe('accessibility', () => { ... })
+})
+```
+
+---
+
+## Security Testing Patterns
+
+### Authentication Boundary Tests
+
+```typescript
+describe('authentication boundaries', () => {
+  it('rejects unauthenticated requests', async () => {
+    await expect(
+      callEndpoint({ authToken: undefined })
+    ).rejects.toThrow('UNAUTHORIZED')
+  })
+
+  it('rejects expired tokens', async () => {
+    await expect(
+      callEndpoint({ authToken: expiredToken })
+    ).rejects.toThrow('UNAUTHORIZED')
+  })
+
+  it('rejects invalid tokens', async () => {
+    await expect(
+      callEndpoint({ authToken: 'invalid-garbage' })
+    ).rejects.toThrow('UNAUTHORIZED')
+  })
+})
+```
+
+### Authorization Tests
+
+```typescript
+describe('authorization', () => {
+  it('allows admin to delete items', async () => {
+    const result = await deleteItem({ role: 'admin', itemId: '1' })
+    expect(result.success).toBe(true)
+  })
+
+  it('forbids regular user from deleting items', async () => {
+    await expect(
+      deleteItem({ role: 'member', itemId: '1' })
+    ).rejects.toThrow('FORBIDDEN')
+  })
+
+  it('allows item owner to edit their own item', async () => {
+    const result = await editItem({ userId: 'owner-1', itemId: '1' })
+    expect(result.success).toBe(true)
+  })
+
+  it('forbids editing another user item', async () => {
+    await expect(
+      editItem({ userId: 'other-user', itemId: '1' })
+    ).rejects.toThrow('FORBIDDEN')
+  })
+})
+```
+
+### Input Sanitization Tests
+
+```typescript
+describe('input sanitization', () => {
+  it('strips HTML from user input', () => {
+    const result = sanitize('<script>alert("xss")</script>Hello')
+    expect(result).toBe('Hello')
+    expect(result).not.toContain('<script>')
+  })
+
+  it('rejects SQL injection attempts', async () => {
+    await expect(
+      searchItems({ query: "'; DROP TABLE items; --" })
+    ).resolves.not.toThrow()
+    // Items table should still exist
+  })
+})
+```
+
+---
+
+## Performance Testing Patterns
+
+### Response Time Assertions
+
+```typescript
+describe('performance', () => {
+  it('list query responds within 300ms', async () => {
+    const start = performance.now()
+    await listItems({ limit: 20 })
+    const duration = performance.now() - start
+
+    expect(duration).toBeLessThan(300)
+  })
+
+  it('search responds within 500ms', async () => {
+    const start = performance.now()
+    await searchItems({ query: 'test' })
+    const duration = performance.now() - start
+
+    expect(duration).toBeLessThan(500)
+  })
+})
+```
+
+### Lighthouse CI (E2E)
+
+```typescript
+// e2e/performance.spec.ts
+test('meets Lighthouse performance targets', async ({ page }) => {
+  await page.goto('{{baseUrl}}/dashboard')
+
+  // Check critical metrics
+  const metrics = await page.evaluate(() => {
+    return new Promise(resolve => {
+      new PerformanceObserver(list => {
+        const entries = list.getEntries()
+        resolve({
+          lcp: entries.find(e => e.entryType === 'largest-contentful-paint')?.startTime,
+          fid: entries.find(e => e.entryType === 'first-input')?.processingStart,
+        })
+      }).observe({ entryTypes: ['largest-contentful-paint', 'first-input'] })
+    })
+  })
+})
+```
+
+---
+
+## Performance Benchmarks
+
+| Metric | Target | Priority |
+|--------|--------|----------|
+| API simple queries | p95 < 300ms | HIGH |
+| API mutations | p95 < 500ms | HIGH |
+| API complex queries | p95 < 1000ms | MEDIUM |
+| Lighthouse Performance | >= 80 | HIGH |
+| Lighthouse Accessibility | >= 85 | HIGH |
+| Lighthouse Best Practices | >= 90 | MEDIUM |
+| Time to Interactive (TTI) | < 3.5s | HIGH |
+| Largest Contentful Paint (LCP) | < 2.5s | HIGH |
+| Cumulative Layout Shift (CLS) | < 0.1 | MEDIUM |
+
+---
+
+## CI Integration
+
+### Running Tests in CI
+
+{{testCommands}}
+
+```bash
+# Unit + Integration tests
+{{testCommand}}
+
+# With coverage report
+{{testCoverageCommand}}
+
+# E2E tests (headless)
+CI=true {{e2eCommand}}
+
+# Run only changed files (for PR checks)
+{{testCommand}} --changed
+
+# Generate coverage report for upload
+{{testCoverageCommand}} --reporter=json --outputFile=coverage.json
+```
+
+### CI Pipeline Stages
+
+```
+1. Lint          → eslint, prettier check
+2. Type Check    → tsc --noEmit
+3. Unit Tests    → {{testCommand}} --coverage
+4. Build         → build the application
+5. E2E Tests     → {{e2eCommand}} (against build)
+6. Coverage Gate → check thresholds
+```
+
+### Coverage Report Upload
+
+```yaml
+# Example CI step (GitHub Actions)
+- name: Upload coverage
+  uses: codecov/codecov-action@v4
+  with:
+    file: ./coverage/coverage-final.json
+    fail_ci_if_error: true
+```
+
+---
+
+## When to Write Tests
+
+| Scenario | Test Type | Priority |
+|----------|-----------|----------|
+| New API endpoint / route | Unit + Integration | REQUIRED |
+| New utility function | Unit | REQUIRED |
+| New custom hook | Unit | REQUIRED |
+| New page / screen | Component + E2E | REQUIRED |
+| New complex component | Component | REQUIRED |
+| New form | Component (validation) + E2E | REQUIRED |
+| Bug fix | Regression test first | REQUIRED |
+| Refactoring | Verify existing tests pass | REQUIRED |
+| Performance optimization | Benchmark before/after | RECOMMENDED |
+| Security fix | Security test case | REQUIRED |
+| Config change | Verify build still works | REQUIRED |
+| Dependency update | Run full suite | REQUIRED |
+
+---
+
+## QA Program Integration
+
+Test standards are part of a broader QA program. When bootstrapping a new project, create these QA resources:
+
+### Resource Files to Generate
+
+| Resource | Purpose |
+|----------|---------|
+| `test-matrix.md` | Map features to test cases |
+| `security-checklist.md` | OWASP + auth + isolation checks |
+| `nielsen-heuristics.md` | UX quality rubric |
+| `performance-benchmarks.md` | Response time budgets |
+| `qa-report-template.md` | Standardized QA report format |
+
+### QA Agents (When Using Agent Framework)
+
+| Agent | Role |
+|-------|------|
+| `qa-test-executor` | Runs functional + API tests |
+| `qa-security-auditor` | Auth boundaries + data isolation |
+| `qa-ux-optimizer` | Heuristic evaluation + WCAG |
+| `qa-report-writer` | Consolidates findings |
+| `qa-fix-planner` | Prioritizes defects |
+| `qa-verifier` | Re-runs failed tests after fixes |
+
+### QA Cycle
+
+```
+1. Plan tests (from PRD / feature spec)
+2. Write tests (using skill templates)
+3. Execute tests (via test runner + E2E)
+4. Report findings (qa-report-template)
+5. Fix defects (prioritized by severity)
+6. Re-verify fixes (regression suite)
+7. Update test matrix (mark as covered)
+```
+
+---
+
+## Test Configuration Templates
+
+### Vitest Config
+
+```typescript
+// vitest.config.ts
+import { defineConfig } from 'vitest/config'
+import path from 'path'
+
+export default defineConfig({
+  test: {
+    globals: true,
+    environment: 'jsdom', // or 'node' for API tests
+    setupFiles: ['./test/setup.ts'],
+    include: ['**/*.test.{ts,tsx}'],
+    exclude: ['node_modules', 'dist', 'e2e'],
+    coverage: {
+      provider: 'v8',
+      reporter: ['text', 'json', 'html'],
+      thresholds: {
+        statements: 80,
+        branches: 70,
+        functions: 80,
+        lines: 80,
+      },
+    },
+  },
+  resolve: {
+    alias: {
+      '@': path.resolve(__dirname, './src'),
+    },
+  },
+})
+```
+
+### Jest Config
+
+```typescript
+// jest.config.ts
+import type { Config } from 'jest'
+
+const config: Config = {
+  preset: 'ts-jest',
+  testEnvironment: 'jsdom', // or 'node'
+  setupFilesAfterSetup: ['./test/setup.ts'],
+  moduleNameMapper: {
+    '^@/(.*)$': '<rootDir>/src/$1',
+  },
+  coverageThreshold: {
+    global: {
+      statements: 80,
+      branches: 70,
+      functions: 80,
+      lines: 80,
+    },
+  },
+  collectCoverageFrom: [
+    'src/**/*.{ts,tsx}',
+    '!src/**/*.d.ts',
+    '!src/**/index.ts',
+  ],
+}
+
+export default config
+```
+
+### Test Setup File
+
+```typescript
+// test/setup.ts
+import '@testing-library/jest-dom' // or '@testing-library/jest-dom/vitest'
+
+// Global mocks
+beforeAll(() => {
+  // Mock environment variables for tests
+  process.env.NODE_ENV = 'test'
+})
+
+afterEach(() => {
+  // Clean up between tests
+  vi.clearAllMocks() // or jest.clearAllMocks()
+})
+```
+
+---
+
+## See Also
+
+- `unit-test.md` - Detailed unit test patterns
+- `component-test.md` - Component test patterns
+- `e2e-test.md` - End-to-end test patterns