valent-pipeline 0.2.19 → 0.2.21

package/pipeline/steps/qa-a/mcp-server.md

@@ -0,0 +1,31 @@
+# QA-A Step: MCP Server Testing
+
+## Protocol Smoke Test Specification
+
+For every MCP tool in this story, write a **Protocol Smoke Test** table:
+
+```
+## Protocol Smoke Tests
+
+| ID | JSON-RPC Method | Params | Expected Result Shape | Expected Error Code |
+```
+
+Rules:
+- Initialize handshake first: `initialize` request must be the first row, verifying server info and capabilities
+- `tools/list` must follow, verifying all tools are registered with correct inputSchema
+- One `tools/call` row per tool with valid params (happy path)
+- One `tools/call` row per tool with invalid params (schema violation, expecting `-32602`)
+- Two-tier error model coverage: at least one row triggering `isError: true` (tool failure) and at least one row triggering a JSON-RPC error code (protocol failure)
+- One row for unknown tool name (expecting `-32601` Method not found or `-32602` Invalid params)
+- One row for malformed JSON-RPC (expecting `-32700` or `-32600`)
+- Expected result shape: key fields and content types QA-B must verify
+- Params: exact JSON payload for the JSON-RPC params field
+
+## Quality Gate Additions
+
+- [ ] Protocol smoke test table covers initialize handshake
+- [ ] Protocol smoke test table covers tools/list
+- [ ] Protocol smoke test table covers tools/call for every tool (happy path + error paths)
+- [ ] Two-tier error model tested: at least one JSON-RPC error code and one isError:true
+- [ ] Invalid args test included for every tool
+- [ ] Unknown tool / unknown method test included

package/pipeline/steps/qa-a/mobile-app.md

@@ -0,0 +1,59 @@
+# QA-A Step: Mobile App Testing
+
+## Maestro Flow Specification
+
+For every mobile screen/flow in this story, write a **Maestro Flow Specification** table:
+
+### Maestro Flow Specifications
+
+| ID | Flow Name | App State Setup | Steps | Expected Result | Platform |
+|----|-----------|-----------------|-------|-----------------|----------|
+
+Column rules:
+- **ID:** Sequential flow identifier (MF-001, MF-002, ...)
+- **Flow Name:** Descriptive name matching the AC being tested
+- **App State Setup:** How to reach the required starting state. Every flow starts with `clearState` + `launchApp`. If seed data is needed, specify the API call or fixture.
+- **Steps:** Sequence of Maestro actions (`launchApp`, `tapOn`, `assertVisible`, `inputText`, `scroll`, `back`, `swipe`, `openLink`, `takeScreenshot`)
+- **Expected Result:** What the user should see after the flow completes (assert conditions)
+- **Platform:** `both` (default) | `android-only` | `ios-only`
+
+### Flow Writing Rules
+
+1. Every flow MUST start with `clearState` or `launchApp` with `clearState: true` — no state carryover from previous flows
+2. Every flow MUST be independent — no ordering dependency between flows
+3. Use `assertVisible` and `assertNotVisible` for assertions, not fixed-time waits
+4. Use `waitForAnimationToEnd` for animation settling, not `extendedWaitUntil` with hardcoded durations
+5. For deep link tests, use `openLink` command with the URI pattern from reqs-brief
+6. Include `takeScreenshot` at key assertion points for evidence capture
+
+## Platform-Conditional Test Requirements
+
+For each AC, specify:
+- **Both platforms** (default): tests that verify identical behavior on Android and iOS
+- **Android-specific:** tests for Android-only behavior (hardware back button, specific permissions, Android notification channels)
+- **iOS-specific:** tests for iOS-only behavior (swipe-to-go-back, Face ID/Touch ID, iOS notification categories)
+
+Mark iOS-specific tests with `[DEFER-IOS]` if the pipeline host may be Windows/Linux. The MOBILE agent will move these to the deferred queue.
+
+## State Isolation Requirements
+
+- Every Maestro flow must be independent (no ordering dependency)
+- App state cleared between flows via `adb shell pm clear {package}` / simulator equivalent
+- If a flow requires seed data, specify the exact API call or fixture to set it up before the flow
+- If a flow requires specific permissions, specify which permissions must be pre-granted
+
+## API Integration for Mobile
+
+When a story's mobile app calls backend APIs, the spec MUST include:
+- At least one Maestro flow per API-calling AC that exercises the real API round-trip (app → API → database → response → UI update)
+- No mocked API responses in Maestro flows (Maestro does not support API interception — this is enforced by design)
+- Infrastructure prerequisite: "API server must be running before Maestro flow execution"
+
+## Quality Gate Additions
+
+- [ ] Every AC has at least one Maestro flow specification
+- [ ] State isolation documented for every flow (clearState + seed data if needed)
+- [ ] Platform-specific tests explicitly tagged (both / android-only / ios-only)
+- [ ] No flow depends on another flow's output state
+- [ ] Deep link tests included for screens with URI patterns
+- [ ] Infrastructure prerequisites documented (API server, emulator, permissions)

package/pipeline/steps/qa-b/data-pipeline.md

@@ -0,0 +1,48 @@
+# QA-B Step: Data Pipeline Testing
+
+## Pipeline Execution Tests
+
+Mandatory for all stories with data pipeline stages. Run the real pipeline against real data and real data stores.
+
+**Procedure:**
+
+1. **Seed sample data.** Per smoke test table in qa-test-spec.md. Use fixture files, seed scripts, or direct data store insertion. Include happy-path data, null/malformed records, and edge cases.
+2. **Run pipeline.** Execute the full pipeline from ingest to output. Capture all logs.
+3. **Validate row counts.** At each stage boundary, verify row counts match expected values from the smoke test table. Record actual vs expected.
+4. **Spot-check values.** For each transform stage, verify a sample of output records against expected values. Check data types, formats, null handling, and edge cases.
+5. **Re-run pipeline (idempotency).** Run the same pipeline again with the same input. Assert that the output is identical -- no duplicate rows, no changed values, no side effects from the second run.
+6. **Kill and restart (checkpoint/resume).** If the pipeline supports checkpointing: run the pipeline, kill it mid-execution (after at least one checkpoint), restart, and verify that the final output matches a clean full run.
+7. **Record results** in `## Pipeline Execution Results` of execution-report.md.
+
+**Pipeline Execution Results table:**
+
+```
+| ID | Stage | Input Rows | Expected Output Rows | Actual Output Rows | Spot-Check Values | Idempotency | Result |
+```
+
+**Row Count Reconciliation:**
+
+```
+| Stage | Input | Output | Dropped | Drop Reason | Expected Delta | Actual Delta | Match |
+```
+
+Include full pipeline logs and commands for reproducibility.
+
+**Failure handling:**
+- Pipeline fails to start: file P1 bug, record error, continue.
+- Stage produces wrong row count: file bug at appropriate priority, continue remaining stages.
+- Idempotency fails (duplicates on re-run): file P1 bug, record both run outputs.
+- Checkpoint/resume produces different output than clean run: file P1 bug, record both outputs.
+- Pipeline crashes mid-run: file P1 bug, record crash output, attempt restart.
+
+**This step cannot be skipped.** If qa-test-spec.md lacks a Pipeline Smoke Tests section, construct the table from pipeline stages in reqs-brief.md and execute.
+
+## Execution Report Additions
+
+The execution report MUST include:
+- `## Pipeline Execution Results` table with actual vs expected for every stage
+- `## Row Count Reconciliation` table showing data flow through the pipeline
+- Full pipeline execution logs
+- Pipeline start command and configuration
+- Idempotency verification results (both runs compared)
+- Checkpoint/resume verification results (if applicable)

package/pipeline/steps/qa-b/document-generation.md

@@ -0,0 +1,47 @@
+# QA-B Step: Document Generation Testing
+
+## Render Validation Tests
+
+Mandatory for all stories with document generation. Invoke the real render pipeline and validate actual output.
+
+**Procedure:**
+
+1. **Seed template and input data.** Per render smoke test table in qa-test-spec.md. Load templates and prepare input data fixtures (JSON, database records, or programmatic setup).
+2. **Invoke document generation.** Call the render pipeline with the seeded template and input data. Capture the generated output (file, buffer, or stream).
+3. **Verify output format and MIME type.** Confirm the output matches the expected format (PDF, HTML, Markdown) and has the correct MIME type.
+4. **Parse output structure.** For HTML: parse the DOM. For PDF: extract text and metadata. For Markdown: parse structure. Verify the output is well-formed and parseable.
+5. **Check variable substitution.** Verify all expected variable values appear in the output at correct positions. Verify no unsubstituted template markers (`{{varName}}`, `${varName}`, `{% raw %}`, etc.) remain.
+6. **Verify encoding.** Confirm UTF-8 encoding. Test with unicode data (CJK, emoji, RTL) and verify output preserves characters correctly -- no mojibake, no replacement characters.
+7. **Execute edge-case data tests.** Per render smoke test table: null variables, missing optional fields, empty collections, extremely long strings, special characters. Verify graceful handling.
+8. **Record results** in `## Render Validation Results` of execution-report.md.
+
+```
+## Render Validation Results
+
+| ID | Template | Input Data | Expected Format | Actual Format | Substitution Check | Encoding Check | Result |
+```
+
+Include raw generation commands and output excerpts for reproducibility.
+
+### Variable Substitution Audit
+
+After all render tests execute, build a substitution audit:
+
+| Template | Total Variables | Substituted Correctly | Null Handled | Missing Handled | Unsubstituted Markers Found |
+|----------|----------------|----------------------|--------------|-----------------|---------------------------|
+
+**Failure handling:**
+- Render pipeline fails to start: file P1 bug, record error, continue.
+- Render test fails: file bug at appropriate priority, continue remaining tests.
+- Output contains unsubstituted markers: file P1 bug -- this is a data leak / presentation defect.
+- Encoding errors (mojibake, replacement characters): file P2 bug.
+
+**This step cannot be skipped.** If qa-test-spec.md lacks a Render Smoke Tests section, construct the table from template definitions in reqs-brief.md and execute.
+
+## Execution Report Additions
+
+The execution report MUST include:
+- `## Render Validation Results` table with actual vs expected for every row
+- Variable substitution audit table
+- Raw generation commands and output excerpts
+- Encoding verification details (input data with unicode, output confirmation)

package/pipeline/steps/qa-b/iac.md

@@ -0,0 +1,44 @@
+# QA-B Step: Infrastructure Testing
+
+## Infrastructure Validation Tests
+
+Mandatory for all stories with infrastructure resources. Validate infrastructure definitions against real plan output and live state.
+
+**Procedure:**
+
+1. **Initialize.** Run `terraform init` (or equivalent) to initialize providers and modules. Verify initialization succeeds without errors.
+2. **Plan validation.** Run `terraform plan` (or equivalent). Verify no errors. Capture plan output for review.
+3. **Apply (if test environment available).** Run `terraform apply` against the test environment. Capture apply output.
+4. **Verify resources exist.** For each resource in the smoke test table, verify it exists in the target environment using CLI or API queries.
+5. **Verify tags.** For each resource, verify all standard tags are present (environment, project, owner, managed-by).
+6. **Verify IAM policies.** For each IAM role/policy, verify least-privilege: no wildcard actions, no overly broad resource scopes.
+7. **Idempotency check.** Run `terraform plan` again after apply. Expect no changes (zero diff). Any unexpected diff is a bug.
+8. **Record results** in `## Infrastructure Validation Results` of execution-report.md.
+
+**Infrastructure Validation Results table:**
+
+```
+| ID | Resource | Operation | Expected State | Actual State | Tags Valid | IAM Valid | Result |
+```
+
+Include full command output for reproducibility.
+
+**Failure handling:**
+- Init fails: file P1 bug, record error, continue.
+- Plan fails: file P1 bug, record plan output, continue.
+- Apply fails: file P1 bug, record apply output, continue.
+- Missing resource after apply: file P1 bug, continue remaining checks.
+- Missing tags: file P2 bug, continue.
+- Overly permissive IAM: file P1 bug, continue.
+- Idempotency fails (diff after apply): file P1 bug, record both plan outputs.
+
+**This step cannot be skipped.** If qa-test-spec.md lacks an Infrastructure Smoke Tests section, construct the table from infrastructure resources in reqs-brief.md and execute.
+
+## Execution Report Additions
+
+The execution report MUST include:
+- `## Infrastructure Validation Results` table with actual vs expected for every resource
+- Full terraform init/plan/apply output
+- Tag verification results per resource
+- IAM policy verification results
+- Idempotency verification (second plan output showing no changes)

package/pipeline/steps/qa-b/library.md

@@ -0,0 +1,61 @@
+# QA-B Step: Library Testing
+
+## Library Consumer Tests
+
+Mandatory for all stories with library/package exports. Install the library in an isolated environment and test it as a real consumer would.
+
+**Procedure:**
+
+1. **Create isolated temp directory.** Initialize a fresh project with the appropriate package manager. This must be outside the library source tree.
+2. **Install the library.** Use a local path install (`npm install ../path-to-lib`, `pip install -e ../path-to-lib`, etc.) to simulate a real consumer installation.
+3. **Execute CJS import tests.** For each export in the Export Smoke Test table, `require()` the module path and verify the expected export is present and has the correct type/signature. Record PASS/FAIL.
+4. **Execute ESM import tests.** For each export in the Export Smoke Test table, `import` the module path and verify the expected export is present and has the correct type/signature. Record PASS/FAIL.
+5. **Execute API contract tests.** Call each exported function/class with the documented usage example. Verify return values match declared types and documented behavior.
+6. **Measure bundle size.** If applicable, bundle the library with a standard bundler (esbuild, webpack, rollup) and record the output size. Flag if significantly larger than expected.
+7. **Verify tree-shaking.** If `sideEffects: false` is declared, bundle a selective import and verify unused exports are excluded from the output.
+8. **Clean up temp directory.**
+
+**Record results** in `## Library Consumer Test Results` of execution-report.md:
+
+```
+## Export Audit Results
+
+| Export Name | CJS require() | ESM import | Type Declaration | Contract Test | Result |
+|-------------|---------------|------------|-----------------|---------------|--------|
+| {name} | {PASS/FAIL} | {PASS/FAIL} | {PASS/FAIL} | {PASS/FAIL} | {overall} |
+
+## CJS/ESM Interop
+
+| Test | Result | Notes |
+|------|--------|-------|
+| CJS require() resolves | {PASS/FAIL} | {details} |
+| ESM import resolves | {PASS/FAIL} | {details} |
+| Named exports match | {PASS/FAIL} | {details} |
+| Default export consistent | {PASS/FAIL} | {details} |
+| No dual-instance corruption | {PASS/FAIL} | {details} |
+
+## Bundle Size
+
+| Entry Point | Raw Size | Minified | Gzipped | Notes |
+|-------------|----------|----------|---------|-------|
+| {entry} | {bytes} | {bytes} | {bytes} | {flags if oversized} |
+```
+
+Include raw commands and full output for reproducibility.
+
+**Failure handling:**
+- Install fails: file P1 bug, record error, continue.
+- CJS/ESM import fails: file bug at appropriate priority, continue remaining tests.
+- Type declarations missing: file P2 bug, continue.
+- Bundle size exceeds threshold: file P3 bug as advisory.
+
+**This step cannot be skipped.** If qa-test-spec.md lacks an Export Smoke Tests section, construct the table from exports in reqs-brief.md and execute.
+
+## Execution Report Additions
+
+The execution report MUST include:
+- `## Export Audit Results` table with PASS/FAIL for each export and import method
+- `## CJS/ESM Interop` table with detailed interop verification
+- `## Bundle Size` table with measured sizes
+- Raw install, import, and bundle commands with full output
+- Isolated temp directory path and cleanup confirmation

package/pipeline/steps/qa-b/mcp-server.md

@@ -0,0 +1,40 @@
+# QA-B Step: MCP Server Testing
+
+## Protocol Compliance Tests
+
+Mandatory for all stories with MCP server tools. Spawn a real server and communicate over the actual transport.
+
+**Procedure:**
+
+1. **Spawn server.** Start the MCP server process with the configured transport (stdio/SSE/HTTP). For stdio, connect to stdin/stdout pipes. For SSE/HTTP, poll the endpoint (max 10 retries, 1s apart) until the server accepts connections.
+2. **JSON-RPC initialize.** Send `initialize` request. Verify server info (name, version) and capability declarations match expected values from qa-test-spec.md.
+3. **tools/list.** Send `tools/list` request. Verify all expected tools are present with correct names, descriptions, and inputSchema.
+4. **tools/call per tool.** For each tool in the protocol smoke test table, send `tools/call` with the specified params. Capture the result. Compare content types, result shape, and values against expected.
+5. **Error paths.** Execute error path rows from the smoke test table:
+   - Unknown tool: send `tools/call` with a non-existent tool name. Verify JSON-RPC error response.
+   - Bad params: send `tools/call` with params violating the declared inputSchema. Verify `-32602` error code.
+   - Tool failure: trigger a tool-level error. Verify `isError: true` in the result.
+   - Malformed JSON: send raw invalid JSON over the transport. Verify `-32700` error code.
+6. **Stop server.**
+7. **Record results** in `## Protocol Compliance Results` of execution-report.md:
+
+```
+| ID | JSON-RPC Method | Params | Expected Result | Actual Result | Expected Error Code | Actual Error Code | Result |
+```
+
+Include raw JSON-RPC request/response pairs for reproducibility.
+
+**Failure handling:**
+- Server fails to start: file P1 bug, record error, continue.
+- Protocol test fails: file bug at appropriate priority, continue remaining tests.
+- Server crashes mid-test: file P1 bug, record crash output, attempt restart.
+
+**This step cannot be skipped.** If qa-test-spec.md lacks a Protocol Smoke Tests section, construct the table from tools in reqs-brief.md and execute.
+
+## Execution Report Additions
+
+The execution report MUST include:
+- `## Protocol Compliance Results` table with actual vs expected for every row
+- Raw JSON-RPC request/response pairs
+- Server startup command and output
+- Transport type and connection confirmation

package/pipeline/steps/qa-b/mobile-app.md

@@ -0,0 +1,71 @@
+# QA-B Step: Mobile App Testing
+
+## Pre-Execution: Verify Mobile Infrastructure
+
+Before running Maestro flows:
+
+1. **Emulator check:** Verify `adb devices` shows at least one connected emulator/device in `device` state (not `offline` or `unauthorized`)
+2. **If no emulator:** Follow boot procedure from MOBILE's emulator-lifecycle step. If boot fails after retries, file P1 bug: "Android emulator failed to boot. Cannot execute Maestro flows."
+3. **App installed check:** `adb shell pm list packages | grep {app_package}`. If app not listed, build and install per MOBILE procedure. If install fails, file P1 bug.
+4. **iOS check (Mac only):** Verify `xcrun simctl list devices | grep Booted` shows at least one booted simulator. If iOS flows exist but no simulator is booted, boot one. If not on Mac, iOS flows are deferred (not a bug).
+5. **API server check (if story calls APIs):** Send health check to API server. If unreachable, start via `docker compose up -d db api` or project-equivalent. If still unreachable, file P1 bug.
+
+## Maestro Flow Execution
+
+For each Maestro flow from qa-test-spec.md:
+1. **State isolation:** `adb shell pm clear {app_package}` (Android) / `xcrun simctl terminate + privacy reset` (iOS)
+2. **Pre-grant permissions:** per flow's permission requirements
+3. **Execute:** `maestro test {flow_file}` (Android) or `maestro test {flow_file} --device {simulator}` (iOS)
+4. **Capture evidence:** Maestro test output + any `takeScreenshot` captures
+5. **Record result:** pass/fail with output details
+
+## Post-Execution: Platform Coverage Audit
+
+After all Maestro flows execute, audit platform coverage:
+
+### Step 1: Scan Flow Results by Platform
+
+| Flow File | Android Result | iOS Result | Both Required |
+|-----------|---------------|------------|---------------|
+| {flow} | {PASS \| FAIL} | {PASS \| FAIL \| DEFERRED} | {yes \| no} |
+
+### Step 2: Check for Deferred iOS Tests
+
+If host is not Mac:
+- All iOS-only and `both` flows that could not run on iOS are deferred
+- This is expected — NOT a bug
+- Record in execution report as "iOS deferred — host OS limitation"
+
+### Step 3: File Bugs
+
+- **Android failures are always bugs** — Android emulator is always available
+- **iOS failures on Mac are bugs** — if the simulator was available and a flow failed, that is a real failure
+- **iOS inability on Windows/Linux is deferred, not a bug** — record as deferred with platform reason
+
+### Step 4: Mock Coverage (N/A for Maestro)
+
+Maestro flows inherently hit the real app and real API server — there is no API interception mechanism. Unlike Playwright `route.fulfill()`, Maestro cannot mock API responses. This means mock coverage audit is not applicable for Maestro E2E flows. However, verify:
+- The app is configured to hit the real API server (check base URL in build config / environment)
+- Actual network traffic is flowing (check `adb shell dumpsys netstats uid {app_uid}` for non-zero bytes)
+- If all API calls show zero bytes, file P1 bug: "App appears to use hardcoded/cached data — no real API traffic detected"
+
+## Execution Report Additions
+
+The execution report MUST include:
+
+```
+## Mobile Platform Coverage Audit
+
+- Host platform: {Mac | Windows | Linux}
+- Android emulator: {PASS — booted | FAIL — details}
+- iOS simulator: {PASS — booted | N/A — not Mac | FAIL — details}
+- Total Maestro flows: {count}
+- Android flows executed: {count} / passed: {count} / failed: {count}
+- iOS flows executed: {count} / passed: {count} / deferred: {count}
+- Platform coverage: {FULL | PARTIAL-IOS-DEFERRED}
+- API traffic verified: {yes | no | N/A — no API calls}
+```
+
+## Quality Gate: This Step Cannot Be Skipped
+
+If `testing_profiles` includes `mobile-app`, this audit is mandatory. The mobile platform coverage audit must appear in the execution report.

package/pipeline/steps/readiness/standalone-review.md

@@ -10,6 +10,10 @@ Read the story's ACs and scope. Independently assess which `testing_profiles` sh
 - `api` — story has API endpoints, backend logic, or database changes
 - `ui` — story has UI components, pages, or visual elements
 - `data-pipeline` — story has ETL, data transformation, or batch processing
+- `mcp-server` — story has MCP server tools, handlers, or protocol work
+- `library` — story is shared library/package (exports, packaging, versioning)
+- `document-generation` — story has document/report template or generation pipeline work
+- `iac` — story has infrastructure work (Terraform, CloudFormation, Kubernetes, CI/CD)
 
 Compare your assessment against the `testing_profiles` tagged on the backlog entry:
 - **Missing profile that should be present** → reject to Lead: `[READINESS-REJECTION] Story {story_id}: testing_profiles missing '{profile}'. Re-tag and re-groom.` **STOP.**
@@ -42,8 +46,9 @@ Read `{story_output_dir}/reqs-brief.md`. Record in `inputsRead`.
 | Pre-mortem addressed | Findings folded-in or accepted-risk with justification |
 | Required sections populated | All template sections present and non-empty (or N/A with rationale) |
 | Business rules complete | Cover all ACs with edge case notes |
-| Database changes specified | Present or explicitly "none" with rationale |
-| API endpoints specified | Present or explicitly "none" with rationale |
+| Database changes specified | Present or explicitly "none" with rationale (only check if api or data-pipeline in testing_profiles) |
+| API endpoints specified | Present or explicitly "none" with rationale (only check if api in testing_profiles) |
+| Domain-specific sections | Conditional sections populated per active testing_profiles (e.g., Tool Definitions for mcp-server, Public API Surface for library) |
 | Cross-cutting concerns | Error handling, validation, auth defined |
 
 **If ANY fails:** Reject to `readiness-review.md#reqs-rejection-reasons`. Send `[READINESS-REJECTION]` to **REQS** AND to Lead: `[READINESS-REJECTION] Story {story_id}: ACs need rework. See readiness-review.md#reqs-rejection-reasons.` **STOP**.

package/pipeline/steps/reqs/data-pipeline.md

@@ -0,0 +1,56 @@
+# REQS Step: Data Pipeline Requirements
+
+## Step 4b: Data-Pipeline-Specific Requirement Extraction
+
+For stories with `data-pipeline` in `testing_profiles`, extract additional requirements:
+
+### Source and Destination Definitions
+- Data format per source/destination (CSV, JSON, Parquet, database table, API response)
+- Schema definition (column names, types, nullable, constraints)
+- Location and access method (file path, connection string, API endpoint, bucket)
+- Volume expectations (row counts, file sizes, batch frequency)
+
+### Transform Specifications
+- Business rules per transform stage (filtering criteria, aggregation logic, joins)
+- Computation definitions (derived fields, formulas, lookups)
+- Stage ordering and dependencies (which transforms must precede others)
+- Data type coercions and casting rules
+
+### Data Quality Rules
+- Null handling per field (reject, default, propagate, coalesce)
+- Deduplication strategy (natural keys, window, first/last wins)
+- Referential integrity checks (foreign key validation, orphan handling)
+- Row count expectations per stage (exact, range, ratio)
+- Value range and format validation (regex, min/max, enum membership)
+
+### Idempotency Strategy
+- Natural keys or surrogate keys for upsert behavior
+- Upsert vs insert-only vs replace semantics per destination
+- Rerun safety guarantees (running the same input twice produces identical output)
+- Timestamp or version column handling on reruns
+
+### Checkpoint and Resume Design
+- State persistence mechanism (file marker, database row, offset tracking)
+- Checkpoint granularity (per-batch, per-file, per-row, per-stage)
+- Recovery behavior on restart (replay from last checkpoint, skip completed)
+- Partial output cleanup on failure (rollback, leave partial, quarantine)
+
+### Error Handling per Stage
+- Per-stage error strategy: skip row, fail batch, quarantine row, log and continue
+- Quarantine destination and format (dead-letter table, error file, log)
+- Error threshold before batch abort (count or percentage)
+- Alerting and notification requirements on failure
+
+### Schema Evolution Strategy
+- How added columns are handled (ignore, default, fail)
+- How removed columns are handled (ignore, fail, null-fill)
+- How type changes are handled (coerce, fail, flag)
+- Schema version tracking and compatibility policy
+
+## Quality Gate Additions
+
+- [ ] Every AC involving data movement has malformed-input behavior defined
+- [ ] Every source and destination has format, schema, and location specified
+- [ ] Idempotency strategy defined for every write stage
+- [ ] Error handling strategy specified per pipeline stage
+- [ ] Null handling defined for every nullable field

package/pipeline/steps/reqs/document-generation.md

@@ -0,0 +1,55 @@
+# REQS Step: Document Generation Requirements
+
+## Step 4b: Document-Generation-Specific Requirement Extraction
+
+For stories with `document-generation` in `testing_profiles`, extract additional requirements:
+
+### Template Definitions
+- Which templates exist and what each produces
+- Template format and engine (Handlebars, Jinja, EJS, custom)
+- Template inheritance or composition (partials, layouts, includes)
+- Template versioning and selection logic
+
+### Variable Schema per Template
+- Required variables with types and validation rules
+- Optional variables with default values
+- Null/missing variable behavior per field (blank, placeholder text, omit section, error)
+- Variable nesting (objects, arrays, conditionals within variable data)
+
+### Output Format Requirements
+- Output format(s): PDF, HTML, Markdown, DOCX, other
+- MIME type for each output format
+- Page layout specifications (margins, orientation, page size) for paginated formats
+- Styling requirements (CSS, embedded styles, theme)
+
+### Conditional Section Rules
+- Include/exclude logic per section (which variables or conditions control visibility)
+- Empty collection handling (show "no items" message vs hide section entirely)
+- Mutual exclusion rules (sections that cannot appear together)
+- Ordering rules for dynamic sections
+
+### Encoding Requirements
+- Character encoding: UTF-8 handling and validation
+- Unicode support (emoji, CJK, diacritics)
+- RTL language support (if applicable)
+- Special character escaping per output format (HTML entities, PDF text encoding)
+
+### Asset Dependencies
+- Fonts: which fonts, embedded or referenced, fallback chain
+- Images and logos: source location, embedded or referenced, format (PNG, SVG, base64)
+- Stylesheets: external or inline, caching behavior
+- Asset resolution strategy (relative paths, CDN, bundled)
+
+### Large Document Handling
+- Streaming or chunked generation for large outputs
+- Maximum document size or page count limits
+- Memory constraints and timeout behavior
+- Progress reporting for long-running generation
+
+## Quality Gate Additions
+
+- [ ] Every template variable has defined null/missing behavior
+- [ ] Every template has output format and MIME type specified
+- [ ] Conditional section logic documented with include/exclude rules
+- [ ] Asset dependencies listed with embedding vs reference strategy
+- [ ] Encoding requirements specified for non-ASCII content

package/pipeline/steps/reqs/draft-brief.md

@@ -25,3 +25,13 @@ Assign priority based on business impact, complexity, and novelty:
 | P1 | Feature broken, degraded UX | Medium | Moderate | CRUD operations with business rules, form validation |
 | P2 | Minor inconvenience | Low | Familiar | Display formatting, sort order, pagination |
 | P3 | Cosmetic, nice-to-have | Trivial | Standard | Label text, tooltip content, logging |
+
+### Profile-Aware Section Selection
+
+Populate template sections based on active `{testing_profiles}`:
+- **Database Changes**: include if `api` or `data-pipeline` in profiles. Skip entirely for other profiles.
+- **API Endpoints**: include if `api` in profiles. Skip entirely for other profiles.
+- **Authentication and Authorization**: include if `api` or `ui` in profiles. Skip entirely for other profiles.
+- **Domain-specific sections**: include the conditional sections from `reqs-brief.template.md` that match active profiles (Data Pipeline Specification, Tool Definitions, Public API Surface, Template Specifications, Infrastructure Resources).
+
+For multi-profile stories (e.g., `[api, iac]`), include ALL applicable sections.

package/pipeline/steps/reqs/iac.md

@@ -0,0 +1,63 @@
+# REQS Step: Infrastructure Requirements
+
+## Step 4b: IaC-Specific Requirement Extraction
+
+For stories with `iac` in `testing_profiles`, extract additional requirements:
+
+### Resource Inventory
+- What resources are created, modified, or destroyed (with provider and resource type)
+- Purpose of each resource (why it exists, what depends on it)
+- Resource dependencies and creation order
+- Naming convention per resource (prefix, environment, region, random suffix)
+
+### Tagging Policy
+- Required tags per resource (environment, project, owner, cost-center, managed-by)
+- Tag naming conventions and allowed values
+- Tags that must propagate to child resources
+- Tag enforcement mechanism (policy, validation, linting)
+
+### IAM Requirements
+- Roles and policies per resource (what identity, what permissions, on what scope)
+- Least-privilege justification (why each permission is needed)
+- Service account or managed identity requirements
+- Cross-account or cross-subscription access (if applicable)
+- Temporary credential strategy (assume role, workload identity federation)
+
+### State Management
+- Remote backend configuration (storage account, bucket, table)
+- State locking mechanism and timeout
+- State access control (who can read/write state)
+- State file segmentation (per-environment, per-module, monolith)
+
+### Environment Topology
+- Environments: dev, staging, prod (and any others)
+- Differences between environments (instance sizes, replica counts, feature flags)
+- Environment promotion strategy (apply same config with different vars)
+- Shared vs isolated resources across environments
+
+### Compliance Constraints
+- Encryption requirements: at rest (which resources, key management) and in transit (TLS version)
+- Network isolation: VPC/VNet, subnets, security groups, private endpoints
+- Audit logging: which resources, log destination, retention period
+- Data residency: region constraints, data sovereignty requirements
+- Regulatory framework alignment (SOC2, HIPAA, GDPR -- if applicable)
+
+### Destroy Protection
+- Which resources must never be accidentally destroyed (databases, storage, DNS)
+- Safeguards per protected resource (lifecycle prevent_destroy, deletion locks, backups)
+- Manual approval gates before destructive changes
+- Backup and recovery requirements before destroy is permitted
+
+### Drift Detection
+- Detection method: scheduled plan, continuous monitoring, cloud-native service
+- Alerting on drift detection (who is notified, channel, severity)
+- Auto-remediation policy (auto-apply, alert-only, manual review)
+- Expected drift sources (manual console changes, other automation)
+
+## Quality Gate Additions
+
+- [ ] Every resource has tagging and destroy-protection policy defined
+- [ ] IAM permissions follow least-privilege with justification
+- [ ] State management backend and locking strategy specified
+- [ ] Environment topology differences documented
+- [ ] Encryption and network isolation requirements specified per resource