@jamie-tam/forge 6.0.0

package/skills/quality-test-plan/SKILL.md

@@ -0,0 +1,297 @@
+---
+name: quality-test-plan
+description: "Use when implementation is complete and a comprehensive test strategy covering unit, integration, E2E, smoke, load, and contract tests is needed."
+---
+
+# Test Plan
+
+## Overview
+
+Generate a comprehensive test plan covering every test type needed for the feature. Each test maps to a requirement for full traceability.
+
+**Core principle:** Every acceptance criterion needs a test that traces back to it.
+
+**Announce at start:** "I'm using the quality-test-plan skill to generate the test strategy."
+
+## When to Use
+
+- After implementation is complete (all TDD cycles done)
+- Before running the full test execution phase
+- During `/feature` and `/greenfield` commands at the test planning phase
+
+**Not for:**
+- Writing tests during implementation (that is build-tdd)
+- Executing tests (that is quality-test-execution)
+
+## When to load references
+
+- **`references/test-type-guide.md`** — purpose, structure, examples, and key rules for each of the six test types (unit, integration, E2E, smoke, load, contract). Load when generating any test-type section.
+
+## Prerequisites
+
+Before generating a test plan, the following must exist:
+
+1. **Requirements** -- `.forge/work/{type}/{name}/requirements.md` (for acceptance criteria mapping)
+2. **Architecture artifacts** -- `.forge/work/{type}/{name}/architecture/` (for API contracts, DB schema)
+3. **Implementation code** -- The actual code to test (from build-tdd phase)
+4. **Existing TDD tests** -- Unit tests written during build-tdd already exist
+
+If any are missing, stop and request them. Do not generate a test plan against imagined requirements.
+
+## The Test Plan Process
+
+### Step 1: Read Input Artifacts
+
+Read and understand:
+
+```
+1. requirements.md
+   - All requirements and their acceptance criteria
+   - Priority levels (must/should/could/won't)
+   - User stories and their scenarios
+
+2. architecture/api-contract.md
+   - Every endpoint: method, path, request/response shapes
+   - Error codes and error response format
+   - Authentication requirements per endpoint
+
+3. architecture/db-schema.md
+   - Tables, columns, types, constraints
+   - Indexes, unique constraints, foreign keys
+   - Cascade behavior
+
+4. Implementation code
+   - All new/modified source files
+   - All existing TDD tests
+   - Entry points, routes, handlers, services
+
+5. Decision log
+   - Architecture decisions that affect test strategy
+   - Known limitations or accepted risks
+```
+
+### Step 1.5: Codex Mode Check
+
+Now that input artifacts are read, run the Codex consent flow from `protocols/codex.md`. The selected mode applies for the rest of this skill's invocation.
+
+- **Takeover:** Dispatch Codex with the artifacts to generate the full test plan. Claude reviews coverage and gap analysis.
+- **Verify** or **Skip / Codex unavailable:** Proceed with the steps below. Step 4.5 will dispatch Codex to review Claude's test plan (Verify only).
+
+### Step 2: Map Requirements to Tests
+
+Create a traceability matrix:
+
+```markdown
+| Requirement | Acceptance Criteria | Test Type | Test ID |
+|-------------|-------------------|-----------|---------|
+| REQ-001: User registration | AC-001.1: Email must be valid | Unit | UT-001 |
+| REQ-001: User registration | AC-001.2: Password min 8 chars | Unit | UT-002 |
+| REQ-001: User registration | AC-001.3: Duplicate email rejected | Integration | IT-001 |
+| REQ-001: User registration | AC-001.4: Success creates user | Integration | IT-002 |
+| REQ-001: User registration | Full flow | E2E | E2E-001 |
+| REQ-002: User login | AC-002.1: Valid credentials succeed | Integration | IT-003 |
+| REQ-002: User login | AC-002.2: Invalid credentials fail | Integration | IT-004 |
+| REQ-002: User login | Full flow | E2E | E2E-002 |
+```
+
+**Rule:** Every acceptance criterion MUST have at least one test. If a criterion has no test, the plan is incomplete.
+
+### Step 2.5: Map User Journeys to E2E Scenarios
+
+If the requirements document includes a User Journeys section, create a business flow traceability table. Every journey MUST map to an E2E test scenario. Gaps are flagged before test execution.
+
+```markdown
+## Business Flow Traceability
+
+| Journey | Priority | E2E Scenario | Steps | Requirements Covered | Status |
+|---|---|---|---|---|---|
+| {journey name} | Must | E2E-001 | {N} steps | {FR-xxx, FR-yyy} | COVERED |
+| {journey name} | Must | E2E-002 | {N} steps | {FR-xxx, FR-yyy} | COVERED |
+| {journey name} | Should | — | {N} steps | {FR-xxx} | GAP |
+```
+
+**Rule:** Every "Must" priority journey MUST have a corresponding E2E scenario. "Should" journeys without E2E scenarios are flagged as gaps but do not block.
+
+### Step 3: Generate Test Sections
+
+Before generating output, check this skill's `templates/` directory and use the provided templates as the output format for each test type.
+
+Generate a plan section for each test type below. For each, load **`references/test-type-guide.md`** to see the purpose, structure, illustrative examples, and key rules:
+
+| Test type | Purpose | Output template |
+|---|---|---|
+| **Unit** | Individual functions/methods in isolation | `templates/unit-test-plan.md` |
+| **Integration** | Component interactions, API endpoints, DB ops | `templates/integration-test-plan.md` |
+| **E2E** | Full user flows through the UI (Playwright) | `templates/e2e-test-plan.md` |
+| **Smoke** | Critical-path sanity check post-deploy | `templates/smoke-test-plan.md` |
+| **Load/Stress** | Performance under realistic and peak load | `templates/load-test-plan.md` |
+| **API Contract** | Frontend/backend agree on API shapes | (no separate output template — follow the example in `references/test-type-guide.md`) |
+
+Coverage targets, key rules, traceability requirements, and per-type examples all live in `references/test-type-guide.md`.
+
+### Step 4: Coverage Gap Analysis
+
+After generating all test sections, verify completeness:
+
+```markdown
+### Coverage Gap Analysis
+
+| Requirement | Unit | Integration | E2E | Smoke | Contract | Status |
+|-------------|------|-------------|-----|-------|----------|--------|
+| REQ-001 | 3 tests | 2 tests | 1 flow | - | 1 test | COVERED |
+| REQ-002 | 2 tests | 2 tests | 1 flow | 1 test | 1 test | COVERED |
+| REQ-003 | 1 test | 0 tests | 0 flows | - | - | GAP |
+
+GAPS FOUND: 1
+- REQ-003 has no integration or E2E tests. Must add before plan is complete.
+```
+
+**Rule:** No gaps allowed. Every requirement must have coverage at the appropriate test levels. If a gap exists, add the missing tests to the plan.
+
+### Step 4.5: Codex Verify
+
+After gap analysis, check the mode recorded at Step 1.5. If **Verify** was selected, dispatch Codex to review the test plan for missing scenarios, edge cases, mock-vs-real blind spots, and coverage gaps. Add missing scenarios if user approves. If **Takeover** was selected, skip this step (Codex already generated the plan). If **Skip**, do nothing. Do NOT re-run the consent flow. See **Codex Integration** section below for full details.
+
+### Step 5: Write the Test Plan Document
+
+Output to `.forge/work/{type}/{name}/test-plan.md` with this structure:
+
+```markdown
+# Test Plan: {Feature Name}
+
+## Metadata
+- **Feature:** {name}
+- **Date:** {date}
+- **Requirements:** {link to requirements.md}
+- **Architecture:** {link to architecture/}
+
+## Traceability Matrix
+[Full requirement-to-test mapping table]
+
+## Unit Tests
+[All unit test specifications]
+
+## Integration Tests
+[All integration test specifications]
+
+## E2E Tests
+[All E2E test specifications]
+
+## Smoke Tests
+[All smoke test specifications]
+
+## Load/Stress Tests
+[All load test specifications, if applicable]
+
+## API Contract Tests
+[All contract test specifications]
+
+## Coverage Gap Analysis
+[Completeness verification]
+
+## Test Environment Requirements
+- Database: {test database config}
+- Services: {external services needed}
+- Tools: {Playwright, k6, etc.}
+- Data: {seed data requirements}
+```
+
+### Step 5.5: Critic Review of Test Plan
+
+After writing the test plan, dispatch the **critic** subagent (Opus) to review the plan for completeness against requirements and API contracts. Provide the critic with the test plan, requirements document, and architecture artifacts.
+
+If the critic identifies coverage gaps or missing scenarios, address them before presenting the plan to the user.
+
+## Common Mistakes
+
+| Mistake | Fix |
+|---------|-----|
+| Skipping E2E for "simple" features | No feature is too simple. E2E catches integration issues. |
+| Only testing happy paths | Every error code, every validation failure, every edge case. |
+| No traceability | Every test must link to a requirement. No orphan tests. |
+| Vague test descriptions | Specify exact inputs, exact expected outputs. |
+| Missing error scenarios | For every success scenario, there is at least one error scenario. |
+| No load test thresholds | "Should be fast" is not a threshold. Specify numbers. |
+| Ignoring existing TDD tests | TDD tests are part of the plan. Include them in the traceability matrix. |
+| Tests that depend on each other | Each test must be independent. No ordering requirements. |
+
+## Red Flags
+
+**Never:**
+- Generate a test plan without reading requirements first
+- Skip any test type without explicit project profile justification
+- Leave gaps in the traceability matrix
+- Use vague descriptions ("test that it works")
+- Assume existing TDD tests cover everything
+- Generate load tests for projects that do not need them (check project profile)
+
+**Always:**
+- Read requirements, architecture, and implementation before planning
+- Map every acceptance criterion to at least one test
+- Include both happy and error paths for every scenario
+- Specify exact inputs and expected outputs
+- Verify completeness with gap analysis
+- Output to `.forge/work/{type}/{name}/test-plan.md`
+
+## I/O Contract
+
+| Field | Value |
+|---|---|
+| **Requires** | Implementation code (from `build-tdd`), architecture artifacts (`.forge/work/{type}/{name}/architecture/`), requirements (`.forge/work/{type}/{name}/requirements.md`) |
+| **Produces** | `.forge/work/{type}/{name}/test-plan.md` |
+| **Feeds into** | `quality-test-execution` (test plan drives execution) |
+| **Updates manifest** | `artifacts.test-plan: test-plan.md` |
+
+## Graphify Context
+
+**Protocol:** `protocols/graphify.md` | **Guard:** Run the status check from the protocol before Step 1.
+
+Graph data directly informs test prioritization — god nodes need the most integration tests, and community boundaries reveal integration gaps.
+
+**How graph data maps to this skill:**
+- **God nodes (highest-degree)** → components with most connections need the most integration tests
+- **Community boundaries** → cross-community edges are integration test priorities
+- **INFERRED edges** → relationships worth validating with contract tests
+
+**CLI queries** (if graph exists and CLI available):
+- `graphify query "what are the most connected components" --budget 1000 --graph graphify-out/graph.json` — identify integration test priorities
+
+**Codex bridge:** Pass graph-informed test priorities to the Codex verify step below.
+
+---
+
+## Codex Integration
+**Modes:** Verify or Takeover | **Protocol:** `protocols/codex.md`
+
+- **Verify:** Claude generates the test plan, Codex reviews for gaps.
+- **Takeover:** Codex generates the test plan, Claude reviews coverage.
+
+**When:** After Claude generates the test plan, before user approval.
+
+**Context to pass:**
+- Path to `test-plan.md`
+- Path to `architecture/api-contract.md`
+- Path to `requirements.md`
+
+**What Codex reviews:**
+- Missing test scenarios (edge cases, error paths, boundary conditions)
+- Mock-vs-real blind spots (tests that mock away the thing they should test)
+- Coverage gaps against requirements (must-haves without corresponding tests)
+
+**Prompt focus:** "Review this test plan against the requirements and architecture. For each test scenario, assess: does this test the real behavior or just a mock? What edge cases are missing? What requirements have no corresponding test?"
+
+**Presentation:** Codex findings merged into the test plan review. Missing scenarios added to the plan if user approves.
+
+---
+
+## Integration
+
+**Called by:**
+- `/feature` command (after implementation loop)
+- `/greenfield` command (after implementation loop)
+
+**Pairs with:**
+- `build-tdd` (existing unit tests become part of the plan)
+- `quality-test-execution` (executes every test in this plan)
+- `plan-architecture` (API contracts and DB schema inform integration/contract tests)
+- `discover-requirements` (acceptance criteria drive the traceability matrix)

package/skills/quality-test-plan/references/test-type-guide.md

@@ -0,0 +1,263 @@
+# Test-type guide
+
+Per-test-type purpose, structure, illustrative examples, and key rules. Load this when generating any of the six test-type sections in the test plan.
+
+For each test type below, the **Structure** block shows the markdown shape of the plan entry. The corresponding output template in `templates/{type}-test-plan.md` (if present) is the authoritative output format — the structure here is illustrative.
+
+---
+
+## Unit Tests
+
+Use the template from `templates/unit-test-plan.md` as the output format.
+
+**Purpose:** Test individual functions and methods in isolation.
+
+**Coverage targets:**
+- 80% minimum overall coverage
+- 100% for critical paths (auth, payments, data validation)
+- All error paths tested
+- All edge cases tested (null, empty, boundary values, overflow)
+
+**What to include:**
+
+```markdown
+### Unit Tests
+
+#### UT-001: Email validation accepts valid emails
+- **File:** tests/unit/auth/validation.test.ts
+- **Function under test:** validateEmail()
+- **Scenarios:**
+  - Valid standard email: user@example.com -> true
+  - Valid with subdomain: user@sub.example.com -> true
+  - Invalid missing @: userexample.com -> false
+  - Invalid missing domain: user@ -> false
+  - Empty string: "" -> false
+  - Null input: null -> false
+- **Traces to:** REQ-001, AC-001.1
+
+#### UT-002: Password validation enforces minimum length
+- **File:** tests/unit/auth/validation.test.ts
+- **Function under test:** validatePassword()
+- **Scenarios:**
+  - Valid 8 chars: "abcd1234" -> true
+  - Invalid 7 chars: "abcd123" -> false
+  - Empty string: "" -> false
+  - Exactly 8 chars boundary: "12345678" -> true
+- **Traces to:** REQ-001, AC-001.2
+```
+
+**Key rules for unit tests:**
+- One behavior per test (if name has "and", split it)
+- Test behavior, not implementation
+- Real code preferred over mocks
+- Edge cases: null, undefined, empty, boundary, overflow, special characters
+
+---
+
+## Integration Tests
+
+Use the template from `templates/integration-test-plan.md` as the output format.
+
+**Purpose:** Test component interactions, API endpoints, database operations.
+
+**What to include:**
+
+```markdown
+### Integration Tests
+
+#### IT-001: POST /api/users rejects duplicate email
+- **File:** tests/integration/auth/registration.test.ts
+- **Components:** UserController -> UserService -> UserRepository -> Database
+- **Setup:** Create existing user with email "test@example.com"
+- **Action:** POST /api/users { email: "test@example.com", password: "valid123" }
+- **Expected:** 409 Conflict, { error: { code: "EMAIL_EXISTS" } }
+- **Cleanup:** Remove test users
+- **Traces to:** REQ-001, AC-001.3
+- **API contract ref:** api-contract.md, POST /api/users, error codes
+
+#### IT-002: POST /api/users creates user successfully
+- **File:** tests/integration/auth/registration.test.ts
+- **Components:** UserController -> UserService -> UserRepository -> Database
+- **Setup:** Clean database
+- **Action:** POST /api/users { email: "new@example.com", password: "valid123", name: "Test" }
+- **Expected:** 201 Created, { id: string, email: "new@example.com", name: "Test", createdAt: string }
+- **Verify:** User exists in database with hashed password
+- **Cleanup:** Remove test users
+- **Traces to:** REQ-001, AC-001.4
+- **API contract ref:** api-contract.md, POST /api/users, success response
+```
+
+**Key rules for integration tests:**
+- Test against real database (use test database, not mocks)
+- Validate against API contract shapes exactly
+- Test both success and error paths per endpoint
+- Clean up test data (use transactions or explicit cleanup)
+- Test database constraints (unique, NOT NULL, foreign keys)
+
+---
+
+## E2E Tests
+
+Use the template from `templates/e2e-test-plan.md` as the output format.
+
+**Purpose:** Test complete user flows through the UI using Playwright.
+
+**What to include:**
+
+```markdown
+### E2E Tests (Playwright)
+
+#### E2E-001: User registration flow
+- **File:** tests/e2e/auth/registration.spec.ts
+- **Preconditions:** Application running, database clean
+- **Steps:**
+  1. Navigate to /register
+  2. Fill in email: "e2e-test@example.com"
+  3. Fill in password: "ValidPass123"
+  4. Fill in name: "E2E Test User"
+  5. Click "Create Account" button
+  6. Wait for redirect to /dashboard
+- **Expected results:**
+  - Registration form accepts input
+  - Loading state shown during submission
+  - Redirected to /dashboard after success
+  - Welcome message displays user name
+  - User can log out and log back in with same credentials
+- **Error scenarios:**
+  - Submit with empty email -> validation error shown inline
+  - Submit with short password -> validation error shown inline
+  - Submit with duplicate email -> error message shown
+- **Traces to:** REQ-001
+- **Screenshots:** Capture at each step for visual regression
+```
+
+**Key rules for E2E tests:**
+- Test complete user journeys, not individual components
+- Include both happy path and error scenarios
+- Test loading states, error states, empty states
+- No skipping "obvious" flows (login, logout, navigation)
+- Use realistic test data, not "test" and "12345"
+- Capture screenshots for visual verification
+
+---
+
+## Smoke Tests
+
+Use the template from `templates/smoke-test-plan.md` as the output format.
+
+**Purpose:** Quick sanity check that critical paths work after deployment.
+
+**What to include:**
+
+```markdown
+### Smoke Tests
+
+#### SMOKE-001: Application starts and responds
+- **Check:** GET /health returns 200
+- **Timeout:** 5 seconds
+- **Traces to:** Infrastructure
+
+#### SMOKE-002: Authentication works
+- **Check:** POST /api/auth/login with valid credentials returns 200 + token
+- **Timeout:** 10 seconds
+- **Traces to:** REQ-002
+
+#### SMOKE-003: Core API responds
+- **Check:** GET /api/users (authenticated) returns 200
+- **Timeout:** 10 seconds
+- **Traces to:** REQ-003
+
+#### SMOKE-004: Database connected
+- **Check:** Any database query succeeds
+- **Timeout:** 5 seconds
+- **Traces to:** Infrastructure
+```
+
+**Key rules for smoke tests:**
+- Only critical paths (not comprehensive)
+- Fast execution (under 60 seconds total)
+- Run after every deployment
+- Binary pass/fail (no partial success)
+
+---
+
+## Load/Stress Tests
+
+Use the template from `templates/load-test-plan.md` as the output format.
+
+**Purpose:** Verify performance under realistic and peak load.
+
+**What to include:**
+
+```markdown
+### Load/Stress Tests
+
+#### LOAD-001: Registration endpoint under load
+- **Tool:** k6 / artillery
+- **Endpoint:** POST /api/users
+- **Scenarios:**
+  - Normal load: 50 concurrent users, 5 min duration
+  - Peak load: 200 concurrent users, 2 min duration
+  - Stress: Ramp to 500 concurrent users, find breaking point
+- **Thresholds:**
+  - p95 response time < 500ms under normal load
+  - p99 response time < 2000ms under peak load
+  - Error rate < 1% under normal load
+  - Error rate < 5% under peak load
+- **Traces to:** NFR-001 (performance requirements)
+
+#### LOAD-002: Authentication endpoint under load
+- **Tool:** k6 / artillery
+- **Endpoint:** POST /api/auth/login
+- **Scenarios:**
+  - Normal load: 100 concurrent users, 5 min duration
+  - Peak load: 500 concurrent users, 2 min duration
+- **Thresholds:**
+  - p95 response time < 200ms under normal load
+  - Error rate < 0.1% under normal load
+- **Traces to:** NFR-001
+```
+
+**Key rules for load tests:**
+- Define specific thresholds (not "should be fast")
+- Test against realistic data volumes
+- Include ramp-up period
+- Test the most-called endpoints
+- Only include if project profile indicates load testing is relevant
+
+---
+
+## API Contract Tests
+
+**Purpose:** Verify frontend and backend agree on API shapes.
+
+**What to include:**
+
+```markdown
+### API Contract Tests
+
+#### CONTRACT-001: POST /api/users request/response shape
+- **File:** tests/contract/auth/users.contract.test.ts
+- **Contract source:** api-contract.md
+- **Validates:**
+  - Request body schema matches contract (required fields, types)
+  - Success response (201) body schema matches contract
+  - Error response (400) body schema matches contract
+  - Error response (409) body schema matches contract
+- **Traces to:** api-contract.md, POST /api/users
+
+#### CONTRACT-002: GET /api/users/:id response shape
+- **File:** tests/contract/auth/users.contract.test.ts
+- **Contract source:** api-contract.md
+- **Validates:**
+  - Response (200) body schema matches contract
+  - Response (404) body schema matches contract
+  - All fields present with correct types
+- **Traces to:** api-contract.md, GET /api/users/:id
+```
+
+**Key rules for contract tests:**
+- Validate exact shapes (field names, types, required vs optional)
+- Test all status codes defined in the contract
+- Run on both frontend and backend independently
+- Fail if contract changes without corresponding code change

package/skills/quality-test-plan/templates/e2e-test-plan.md

@@ -0,0 +1,72 @@
+# E2E Test Plan
+
+## User Flow: {flow_name}
+
+### Description
+{What user journey this flow tests. One sentence.}
+
+### Preconditions
+- {User is logged in as: role}
+- {Database contains: specific test data}
+- {Feature flag: enabled/disabled}
+- {Browser: chromium / firefox / webkit}
+
+### Steps
+
+| # | Action | Expected Result | Assertion |
+|---|--------|----------------|-----------|
+| 1 | Navigate to {URL} | Page loads with {element} visible | `expect(page.locator('{selector}')).toBeVisible()` |
+| 2 | Fill {field} with "{value}" | Field accepts input | `expect(input).toHaveValue('{value}')` |
+| 3 | Click "{button text}" | {Loading state appears, then result} | `expect(page.locator('{selector}')).toContainText('{text}')` |
+| 4 | Wait for {condition} | {Expected state after async operation} | `expect(response.status()).toBe(200)` |
+| 5 | Verify {final state} | {Page shows expected content} | `expect(page.locator('{selector}')).toBeVisible()` |
+
+### Screenshots/Visual Assertions
+- Capture screenshot after step {N}: {what it should look like}
+- Capture screenshot on failure: {automatic}
+
+### Error Flow Variant: {variant_name}
+| # | Action | Expected Result | Assertion |
+|---|--------|----------------|-----------|
+| 1 | {Same setup as happy path} | | |
+| 2 | Fill {field} with "{invalid value}" | | |
+| 3 | Click "{button text}" | Error message: "{expected error}" | `expect(page.locator('.error')).toContainText('{message}')` |
+
+### Cleanup
+- {Delete test user/data created during the flow}
+- {Reset application state}
+
+---
+
+## User Flow: {next_flow_name}
+
+### Description
+{What this flow tests.}
+
+### Preconditions
+- {list}
+
+### Steps
+
+| # | Action | Expected Result | Assertion |
+|---|--------|----------------|-----------|
+| 1 | {action} | {result} | {assertion} |
+
+### Cleanup
+- {cleanup steps}
+
+---
+
+## Flow Summary
+
+| Flow | Steps | Variants | Priority | Requirement |
+|------|-------|----------|----------|-------------|
+| {flow name} | {count} | {count} | Critical / High / Medium | {FR-XXX} |
+| {flow name} | {count} | {count} | Critical / High / Medium | {FR-XXX} |
+
+## Browser Matrix
+
+| Flow | Chromium | Firefox | WebKit |
+|------|---------|---------|--------|
+| {flow} | Required | Required | Optional |
+| {flow} | Required | Optional | Optional |