@zigrivers/scaffold 2.1.2 → 2.38.0

package/pipeline/pre/create-prd.md

@@ -1,33 +1,38 @@
 ---
 name: create-prd
 description: Create a product requirements document from a project idea
+summary: "Translates your vision (or idea, if no vision exists) into a product requirements document with problem statement, user personas, prioritized feature list, constraints, non-functional requirements, and measurable success criteria."
 phase: "pre"
-order: 1
+order: 110
 dependencies: []
-outputs: [docs/prd.md]
+outputs: [docs/plan.md]
 conditional: null
 knowledge-base: [prd-craft]
+reads: [create-vision]
 ---
 
 ## Purpose
 Transform a project idea into a structured product requirements document that
 defines the problem, target users, features, constraints, and success criteria.
 This is the foundation document that all subsequent phases reference.
+The PRD drives user stories, architecture decisions, and implementation planning
+throughout the entire pipeline.
 
 ## Inputs
 - Project idea (provided by user verbally or in a brief)
 - Existing project files (if brownfield — any README, docs, or code)
 
 ## Expected Outputs
-- docs/prd.md — Product requirements document
+- docs/plan.md — Product requirements document
 
 ## Quality Criteria
-- Problem statement is specific and testable (not vague aspirations)
-- Target users are identified with their needs
-- Features are scoped with clear boundaries (what's in, what's out)
-- Success criteria are measurable
-- Constraints (technical, timeline, budget, team) are documented
-- Non-functional requirements are explicit (performance, security, accessibility)
+- (mvp) Problem statement names a specific user group, a specific pain point, and a falsifiable hypothesis about the solution
+- (mvp) Target users are identified with their needs
+- (mvp) Features are scoped with clear boundaries (what's in, what's out)
+- (mvp) Success criteria are measurable
+- (mvp) Each non-functional requirement has a measurable target or threshold (e.g., 'page load < 2s', 'WCAG AA')
+- (mvp) No two sections contain contradictory statements about the same concept
+- (deep) Constraints (technical, timeline, budget, team) are documented
 
 ## Methodology Scaling
 - **deep**: Comprehensive PRD. Competitive analysis, detailed user personas,
@@ -40,6 +45,25 @@ This is the foundation document that all subsequent phases reference.
   phased delivery.
 
 ## Mode Detection
-If docs/prd.md exists, operate in update mode: read existing content, identify
+If docs/plan.md exists, operate in update mode: read existing content, identify
 what has changed or been learned since it was written, propose targeted updates.
 Preserve existing decisions unless explicitly revisiting them.
+
+## Update Mode Specifics
+- **Detect prior artifact**: docs/plan.md exists
+- **Preserve**: problem statement, existing feature definitions, success criteria,
+  user personas, scope boundaries, and enhancement markers (`<!-- enhancement: ... -->`)
+  unless user explicitly requests changes
+- **Triggers for update**: user provides new requirements, scope adjustment
+  requested, constraints changed (timeline, budget, team), new user research
+- **Conflict resolution**: new features are appended to the feature list with
+  clear versioning; changed constraints are documented with rationale for change
+
+### Understand the Vision
+
+**If `docs/vision.md` exists**: Read it completely. This is your strategic foundation — the vision document has already established the problem space, target audience, value proposition, competitive landscape, and guiding principles. Skip the vision discovery questions below and use the vision document as the North Star for this PRD. Reference it throughout, ensuring every requirement aligns with the stated vision and guiding principles. Focus your discovery questions on translating the vision into concrete product requirements rather than re-exploring strategic direction.
+
+**If `docs/vision.md` does NOT exist**:
+- What problem does this solve and for whom? Push me to be specific about the target user.
+- What does success look like? How will we know this is working?
+- What's the single most important thing this app must do well?

package/pipeline/pre/innovate-prd.md

@@ -1,12 +1,13 @@
 ---
 name: innovate-prd
 description: Discover feature-level innovation opportunities in the PRD
+summary: "Analyzes the PRD for feature-level gaps — competitive blind spots, UX enhancements, AI-native possibilities — and proposes additions for your approval."
 phase: "pre"
-order: 3
+order: 130
 dependencies: [review-prd]
-outputs: [docs/prd-innovation.md]
+outputs: [docs/prd-innovation.md, docs/plan.md, docs/reviews/prd-innovation/review-summary.md, docs/reviews/prd-innovation/codex-review.json, docs/reviews/prd-innovation/gemini-review.json]
 conditional: "if-needed"
-knowledge-base: [prd-innovation, prd-craft]
+knowledge-base: [prd-innovation, prd-craft, multi-model-review-dispatch]
 ---
 
 ## Purpose
@@ -15,33 +16,63 @@ new capabilities, competitive positioning, and defensive product gaps. It is
 NOT UX-level enhancement (that belongs in user story innovation) — it focuses
 on whether the right features are in the PRD at all.
 
+At depth 4+, dispatches to external AI models (Codex, Gemini) for
+independent innovation brainstorming — different models surface different
+creative opportunities and competitive insights.
+
 ## Inputs
-- docs/prd.md (required) — PRD to analyze for innovation opportunities
+- docs/plan.md (required) — PRD to analyze for innovation opportunities
 - docs/reviews/pre-review-prd.md (optional) — review findings for context
 
 ## Expected Outputs
 - docs/prd-innovation.md — innovation findings, suggestions with cost/impact
   assessment, and disposition (accepted/rejected/deferred)
-- docs/prd.md — updated with approved innovations
+- docs/plan.md — updated with approved innovations
+- docs/reviews/prd-innovation/review-summary.md (depth 4+) — multi-model innovation synthesis
+- docs/reviews/prd-innovation/codex-review.json (depth 4+, if available) — raw Codex suggestions
+- docs/reviews/prd-innovation/gemini-review.json (depth 4+, if available) — raw Gemini suggestions
 
 ## Quality Criteria
-- Enhancements are feature-level, not UX-level polish
-- Each suggestion has a cost estimate (trivial/moderate/significant)
-- Each suggestion has a clear user benefit and impact assessment
-- Approved innovations are documented to the same standard as existing features
-- PRD scope boundaries are respected — no uncontrolled scope creep
+- (mvp) Enhancements are feature-level, not UX-level polish
+- (mvp) Each suggestion has a cost estimate (trivial/moderate/significant)
+- (mvp) Each suggestion has a clear user benefit and impact assessment
+- (mvp) Each approved innovation includes: problem it solves, target users, scope boundaries, and success criteria
+- (mvp) PRD scope boundaries are respected — no uncontrolled scope creep
 - User approval is obtained before modifying the PRD
+- User approval for each accepted innovation documented as a question-response pair with timestamp (e.g., "Q: Accept feature X? A: Yes — 2025-01-15T14:30Z")
+- (depth 4+) Multi-model suggestions deduplicated and synthesized with unique ideas from each model highlighted
 
 ## Methodology Scaling
 - **deep**: Full innovation pass across all categories (competitive research,
   UX gaps, AI-native opportunities, defensive product thinking). Cost/impact
-  matrix. Detailed integration of approved innovations into PRD.
+  matrix. Detailed integration of approved innovations into PRD. Multi-model
+  innovation dispatched to Codex and Gemini if available, with graceful
+  fallback to Claude-only enhanced brainstorming.
 - **mvp**: Not applicable — this step is conditional and skipped in MVP.
-- **custom:depth(1-5)**: Depth 1-2: not typically enabled. Depth 3: quick scan
-  for obvious gaps and missing expected features. Depth 4-5: full innovation
-  pass with evaluation framework.
+- **custom:depth(1-5)**: Depth 1-2: skip (not enough context for meaningful innovation at this depth). Depth 3: quick scan
+  for obvious gaps and missing expected features. Depth 4: full innovation
+  pass + one external model (if CLI available). Depth 5: full innovation pass
+  + multi-model with deduplication and synthesis.
+
+## Conditional Evaluation
+Enable when: project has a competitive landscape section in plan.md, user explicitly
+requests an innovation pass, or the PRD review (review-prd) identifies feature gaps
+or missing capabilities. Skip when: PRD is minimal/exploratory, depth < 3, or user
+explicitly declines innovation.
 
 ## Mode Detection
 If docs/prd-innovation.md exists, this is a re-innovation pass. Read previous
 suggestions and their disposition (accepted/rejected/deferred), focus on new
-opportunities from PRD changes since last run.
+opportunities from PRD changes since last run. If multi-model artifacts exist
+under docs/reviews/prd-innovation/, preserve prior suggestion dispositions.
+
+## Update Mode Specifics
+- **Detect prior artifact**: docs/prd-innovation.md exists with suggestion
+  dispositions
+- **Preserve**: accepted/rejected/deferred dispositions from prior runs,
+  cost/impact assessments already reviewed by user, multi-model review artifacts
+- **Triggers for update**: PRD scope changed (new features added or removed),
+  user requests re-evaluation of deferred suggestions, new external model
+  available for additional perspectives
+- **Conflict resolution**: if a previously rejected suggestion is now relevant
+  due to PRD changes, re-propose with updated rationale referencing the change

package/pipeline/pre/innovate-user-stories.md

@@ -1,12 +1,13 @@
 ---
 name: innovate-user-stories
 description: Discover UX-level enhancements and innovation opportunities in user stories
+summary: "Identifies UX enhancement opportunities — progressive disclosure, smart defaults, accessibility improvements — and integrates approved changes into existing stories."
 phase: "pre"
-order: 6
+order: 160
 dependencies: [review-user-stories]
-outputs: [docs/user-stories-innovation.md]
+outputs: [docs/user-stories-innovation.md, docs/reviews/user-stories-innovation/review-summary.md, docs/reviews/user-stories-innovation/codex-review.json, docs/reviews/user-stories-innovation/gemini-review.json]
 conditional: "if-needed"
-knowledge-base: [user-stories, user-story-innovation]
+knowledge-base: [user-stories, user-story-innovation, multi-model-review-dispatch]
 ---
 
 ## Purpose
@@ -16,32 +17,64 @@ innovation — `innovate-prd`) — it focuses on making existing features better
 through smart defaults,
 progressive disclosure, accessibility improvements, and AI-native capabilities.
 
+At depth 4+, dispatches to external AI models (Codex, Gemini) for
+independent UX innovation brainstorming — different models surface different
+enhancement opportunities.
+
 ## Inputs
 - docs/user-stories.md (required) — stories to enhance
-- docs/prd.md (required) — PRD boundaries (innovation must not exceed scope)
+- docs/plan.md (required) — PRD boundaries (innovation must not exceed scope)
 
 ## Expected Outputs
 - docs/user-stories-innovation.md — innovation findings, suggestions with
   cost/impact assessment, and disposition (accepted/rejected/deferred)
 - docs/user-stories.md — updated with approved enhancements
+- docs/reviews/user-stories-innovation/review-summary.md (depth 4+) — multi-model innovation synthesis
+- docs/reviews/user-stories-innovation/codex-review.json (depth 4+, if available) — raw Codex suggestions
+- docs/reviews/user-stories-innovation/gemini-review.json (depth 4+, if available) — raw Gemini suggestions
 
 ## Quality Criteria
-- Enhancements are UX-level, not new features
-- Each suggestion has a cost estimate (trivial/moderate/significant)
-- Each suggestion has a clear user benefit
-- Approved enhancements are integrated into existing stories (not new stories)
-- PRD scope boundaries are respected — no scope creep
+- (mvp) Enhancements are UX-level, not new features
+- (mvp) Each suggestion has a cost estimate (trivial/moderate/significant)
+- (mvp) Each suggestion has a clear user benefit
+- (mvp) Approved enhancements are integrated into existing stories (not new stories)
+- (mvp) PRD scope boundaries are respected — no scope creep
+- User approval for each accepted innovation documented as a question-response pair with timestamp (e.g., "Q: Accept enhancement X? A: Yes — 2025-01-15T14:30Z")
+- (depth 4+) Multi-model suggestions deduplicated and synthesized with unique ideas from each model highlighted
 
 ## Methodology Scaling
 - **deep**: Full innovation pass across all three categories (high-value
   low-effort, differentiators, defensive gaps). Cost/impact matrix.
-  Detailed integration of approved enhancements into stories.
+  Detailed integration of approved enhancements into stories. Multi-model
+  innovation dispatched to Codex and Gemini if available, with graceful
+  fallback to Claude-only enhanced brainstorming.
 - **mvp**: Not applicable — this step is conditional and skipped in MVP.
-- **custom:depth(1-5)**: Depth 1-2: not typically enabled. Depth 3: quick
-  scan for obvious improvements. Depth 4-5: full innovation pass with
-  evaluation framework.
+- **custom:depth(1-5)**: Depth 1-2: skip (not enough context for meaningful innovation at this depth). Depth 3: quick
+  scan for obvious improvements. Depth 4: full innovation pass + one external
+  model (if CLI available). Depth 5: full innovation pass + multi-model with
+  deduplication and synthesis.
+
+## Conditional Evaluation
+Enable when: user stories review identifies UX gaps, project targets a consumer-facing
+audience, or progressive disclosure patterns would benefit users. Skip when: stories
+are backend-only with no user-facing UI, depth < 3, or user explicitly declines
+innovation.
 
 ## Mode Detection
 If docs/user-stories-innovation.md exists, this is a re-innovation pass. Read
 previous suggestions and their disposition (accepted/rejected), focus on new
-opportunities from story changes since last run.
+opportunities from story changes since last run. If multi-model artifacts
+exist under docs/reviews/user-stories-innovation/, preserve prior suggestion
+dispositions.
+
+## Update Mode Specifics
+- **Detect prior artifact**: docs/user-stories-innovation.md exists with
+  suggestion dispositions
+- **Preserve**: accepted/rejected dispositions from prior runs, cost/impact
+  assessments already reviewed, multi-model review artifacts
+- **Triggers for update**: user stories changed (new stories added, existing
+  stories rewritten), PRD innovation accepted new features that need UX
+  enhancement analysis
+- **Conflict resolution**: if a previously rejected UX enhancement is now
+  relevant due to story changes, re-propose with updated rationale; never
+  re-suggest rejected enhancements without a material change in context

package/pipeline/pre/review-prd.md

@@ -1,12 +1,13 @@
 ---
 name: review-prd
 description: Multi-pass review of the PRD for completeness, clarity, and downstream readiness
+summary: "Reviews the PRD across eight passes — problem rigor, persona coverage, feature scoping, success criteria, internal consistency, constraints, non-functional requirements — and fixes blocking issues."
 phase: "pre"
-order: 2
+order: 120
 dependencies: [create-prd]
-outputs: [docs/reviews/pre-review-prd.md]
+outputs: [docs/reviews/pre-review-prd.md, docs/reviews/prd/review-summary.md, docs/reviews/prd/codex-review.json, docs/reviews/prd/gemini-review.json]
 conditional: null
-knowledge-base: [review-methodology, review-prd, prd-craft, gap-analysis]
+knowledge-base: [review-methodology, review-prd, prd-craft, gap-analysis, multi-model-review-dispatch, review-step-template]
 ---
 
 ## Purpose
@@ -15,30 +16,50 @@ product requirements artifacts. Identify issues, create a fix plan, execute
 fixes, and re-validate. Ensures the PRD is complete, clear, consistent, and
 ready for User Stories to consume.
 
+At depth 4+, dispatches to external AI models (Codex, Gemini) for
+independent review validation.
+
 ## Inputs
-- docs/prd.md (required) — PRD to review
+- docs/plan.md (required) — PRD to review
 - Project idea or brief (context from user, if available)
 
 ## Expected Outputs
 - docs/reviews/pre-review-prd.md — review findings, fix plan, and resolution log
-- docs/prd.md — updated with fixes
+- docs/plan.md — updated with fixes
+- docs/reviews/prd/review-summary.md (depth 4+) — multi-model review synthesis
+- docs/reviews/prd/codex-review.json (depth 4+, if available) — raw Codex findings
+- docs/reviews/prd/gemini-review.json (depth 4+, if available) — raw Gemini findings
 
 ## Quality Criteria
+- (mvp) Passes 1-2 executed with findings documented
 - All review passes executed with findings documented
 - Every finding categorized by severity (P0-P3)
 - Fix plan created for P0 and P1 findings
 - Fixes applied and re-validated
-- Downstream readiness confirmed (User Stories can proceed)
+- (mvp) Downstream readiness confirmed (User Stories can proceed)
+- (depth 4+) Multi-model findings synthesized with consensus/disagreement analysis
 
 ## Methodology Scaling
 - **deep**: All 8 review passes from the knowledge base. Full findings report
-  with severity categorization. Fixes applied and re-validated.
+  with severity categorization. Fixes applied and re-validated. Multi-model
+  review dispatched to Codex and Gemini if available, with graceful fallback
+  to Claude-only enhanced review.
 - **mvp**: Passes 1-2 only (Problem Statement Rigor, Persona Coverage). Focus
   on blocking gaps — requirements too vague to write stories from.
 - **custom:depth(1-5)**: Depth 1-2: passes 1-2 only (Problem Statement Rigor,
   Persona Coverage). Depth 3: passes 1-4 (add Feature Scoping, Success
-  Criteria). Depth 4-5: all 8 passes.
+  Criteria). Depth 4: all 8 passes + one external model review (if CLI
+  available). Depth 5: all 8 passes + multi-model review with reconciliation.
 
 ## Mode Detection
 If docs/reviews/pre-review-prd.md exists, this is a re-review. Read previous
 findings, check which were addressed, run review passes again on updated PRD.
+If multi-model review artifacts exist under docs/reviews/prd/, preserve prior
+findings still valid.
+
+## Update Mode Specifics
+
+- **Detect**: `docs/reviews/review-prd.md` exists with tracking comment
+- **Preserve**: Prior findings still valid, resolution decisions, multi-model review artifacts
+- **Triggers**: Upstream artifact changed since last review (compare tracking comment dates)
+- **Conflict resolution**: Previously resolved findings reappearing = regression; flag and re-evaluate

package/pipeline/pre/review-user-stories.md

@@ -1,12 +1,13 @@
 ---
 name: review-user-stories
 description: Multi-pass review of user stories for PRD coverage, quality, and downstream readiness
+summary: "Verifies every PRD feature maps to at least one story, checks that acceptance criteria are specific enough to test, validates story independence, and builds a requirements traceability index at higher depths."
 phase: "pre"
-order: 5
+order: 150
 dependencies: [user-stories]
-outputs: [docs/reviews/pre-review-user-stories.md]
+outputs: [docs/reviews/pre-review-user-stories.md, docs/reviews/user-stories/requirements-index.md, docs/reviews/user-stories/coverage.json, docs/reviews/user-stories/review-summary.md]
 conditional: null
-knowledge-base: [review-methodology, review-user-stories]
+knowledge-base: [review-methodology, review-user-stories, multi-model-review-dispatch, review-step-template]
 ---
 
 ## Purpose
@@ -14,30 +15,55 @@ Deep multi-pass review of user stories, targeting failure modes specific to
 story artifacts. Identify coverage gaps, quality issues, and downstream
 readiness problems. Create a fix plan, execute fixes, and re-validate.
 
+At higher depths, builds a formal requirements index with traceability matrix
+and optionally dispatches to external AI models (Codex, Gemini) for
+independent coverage validation.
+
 ## Inputs
 - docs/user-stories.md (required) — stories to review
-- docs/prd.md (required) — source requirements for coverage checking
+- docs/plan.md (required) — source requirements for coverage checking
+- docs/reviews/user-stories/ artifacts (optional) — prior review findings in update mode
 
 ## Expected Outputs
 - docs/reviews/pre-review-user-stories.md — review findings, fix plan, and resolution log
 - docs/user-stories.md — updated with fixes
+- docs/reviews/user-stories/requirements-index.md (depth 4+) — atomic requirements
+  extracted from PRD with REQ-xxx IDs
+- docs/reviews/user-stories/coverage.json (depth 4+) — requirement-to-story mapping
+- docs/reviews/user-stories/review-summary.md (depth 5) — multi-model review
+  synthesis with coverage verification
 
 ## Quality Criteria
+- (mvp) Pass 1 (PRD coverage) executed with findings documented
 - All review passes executed with findings documented
 - Every finding categorized by severity (P0-P3)
 - Fix plan created for P0 and P1 findings
 - Fixes applied and re-validated
-- Downstream readiness confirmed (modeling phase can proceed)
+- (mvp) Every story has at least one testable acceptance criterion, and every PRD feature maps to at least one story
+- (depth 4+) Every atomic PRD requirement has a REQ-xxx ID in the requirements index
+- (depth 4+) Coverage matrix maps every REQ to at least one US (100% coverage target)
+- (depth 4+) Multi-model findings synthesized with consensus/disagreement analysis
 
 ## Methodology Scaling
 - **deep**: All 6 review passes from the knowledge base. Full findings report
-  with severity categorization. Fixes applied and re-validated.
+  with severity categorization. Fixes applied and re-validated. Requirements
+  index and coverage matrix built. Multi-model review dispatched to Codex and
+  Gemini if available, with graceful fallback to Claude-only enhanced review.
 - **mvp**: Pass 1 only (PRD coverage). Focus on blocking gaps — PRD features
   with no corresponding story.
 - **custom:depth(1-5)**: Depth 1: pass 1 only. Depth 2: passes 1-2.
-  Depth 3: passes 1-4. Depth 4-5: all 6 passes.
+  Depth 3: passes 1-4. Depth 4: all 6 passes + requirements index + coverage
+  matrix. Depth 5: all of depth 4 + multi-model review (if CLIs available).
 
 ## Mode Detection
 If docs/reviews/pre-review-user-stories.md exists, this is a re-review. Read
 previous findings, check which were addressed, run review passes again on
-updated stories.
+updated stories. If docs/reviews/user-stories/requirements-index.md exists,
+preserve requirement IDs — never renumber REQ-xxx IDs.
+
+## Update Mode Specifics
+
+- **Detect**: `docs/reviews/pre-review-user-stories.md` exists with tracking comment
+- **Preserve**: Prior findings still valid, REQ-xxx IDs, resolution decisions, multi-model review artifacts
+- **Triggers**: Upstream artifact changed since last review (compare tracking comment dates)
+- **Conflict resolution**: Previously resolved findings reappearing = regression; flag and re-evaluate

package/pipeline/pre/user-stories.md

@@ -1,10 +1,12 @@
 ---
 name: user-stories
 description: Translate PRD features into user stories with acceptance criteria
+summary: "Breaks every PRD feature into user stories organized by epic, each with testable acceptance criteria in Given/When/Then format."
 phase: "pre"
-order: 4
+order: 140
 dependencies: [review-prd]
 outputs: [docs/user-stories.md]
+reads: [innovate-prd]
 conditional: null
 knowledge-base: [user-stories]
 ---
@@ -16,7 +18,7 @@ that are testable and specific enough to drive domain modeling, UX design, and
 task decomposition downstream.
 
 ## Inputs
-- docs/prd.md (required) — features, personas, and requirements to translate
+- docs/plan.md (required) — features, personas, and requirements to translate
 - docs/reviews/pre-review-prd.md (optional) — review findings for context
 - docs/prd-innovation.md (optional) — innovation findings and approved enhancements
 
@@ -25,12 +27,13 @@ task decomposition downstream.
   criteria scaled to the configured depth level
 
 ## Quality Criteria
-- Every PRD feature maps to at least one user story
-- Stories follow INVEST criteria (Independent, Negotiable, Valuable, Estimable, Small, Testable)
-- Acceptance criteria are testable — unambiguous pass/fail
-- No story too large to implement in 1-3 focused agent sessions
-- Every PRD persona is represented in at least one story
-- Stories describe user behavior, not implementation details
+- (mvp) Every PRD feature maps to at least one user story
+- (deep) Stories follow INVEST criteria (Independent, Negotiable, Valuable, Estimable, Small, Testable)
+- (mvp) Acceptance criteria are testable — unambiguous pass/fail
+- (deep) No story has more than 7 acceptance criteria
+- (mvp) Every PRD persona is represented in at least one story
+- (mvp) Stories describe user behavior, not implementation details
+- (mvp) Each story is independent — reordering stories does not break acceptance criteria
 
 ## Methodology Scaling
 - **deep**: Full story template with IDs, persona journey maps, cross-story
@@ -46,3 +49,15 @@ task decomposition downstream.
 If docs/user-stories.md exists, operate in update mode: read existing stories,
 identify changes needed based on updated PRD, categorize as ADD/RESTRUCTURE/
 PRESERVE, get approval before modifying. Preserve existing story IDs.
+
+## Update Mode Specifics
+- **Detect prior artifact**: docs/user-stories.md exists
+- **Preserve**: existing story IDs, epic groupings, acceptance criteria that
+  haven't been invalidated, story-to-PRD-feature traceability, enhancement
+  markers (`<!-- enhancement: ... -->`), priority decisions, story ID format
+  (US-xxx)
+- **Triggers for update**: PRD features added or changed, innovation suggestions
+  accepted, user personas expanded, review findings require story adjustments
+- **Conflict resolution**: never reuse a retired story ID; if a story's scope
+  changed, update its acceptance criteria in-place rather than creating a
+  duplicate story

package/pipeline/quality/create-evals.md

@@ -0,0 +1,106 @@
+---
+name: create-evals
+description: Generate project-specific eval checks from standards documentation
+summary: "Generates automated checks that verify your code matches your documented standards — file placement, naming conventions, feature-to-test coverage, API contract alignment — using your project's own test framework."
+phase: "quality"
+order: 920
+dependencies: [tdd, story-tests]
+outputs: [tests/evals/, docs/eval-standards.md]
+reads: [security, dev-env-setup, api-contracts, database-schema, ux-spec]
+conditional: null
+knowledge-base: [eval-craft, testing-strategy]
+---
+
+## Purpose
+Generate automated eval checks that verify AI-generated code meets the project's
+own documented standards. Evals are test files in the project's own test framework
+— not a separate tool. They check up to 13 categories: 5 core (always generated)
+and 8 conditional (generated when their source document exists). Core: consistency,
+structure, adherence, coverage, cross-doc. Conditional: architecture conformance,
+API contract, security patterns, database schema, accessibility, performance budget,
+configuration validation, error handling completeness.
+
+## Inputs
+- docs/tech-stack.md (required) — determines test framework and stack-specific patterns
+- docs/coding-standards.md (required) — adherence and error handling patterns
+- docs/tdd-standards.md (required) — test co-location rules, mocking strategy
+- docs/project-structure.md (required) — file placement rules for structure evals
+- CLAUDE.md (required) — Key Commands table for consistency evals
+- Makefile or package.json (required) — build targets to match against
+- tests/acceptance/ (optional) — story test skeletons for coverage validation
+- docs/user-stories.md (optional) — acceptance criteria for coverage evals
+- docs/plan.md (optional) — feature list for coverage evals, performance NFRs
+- docs/system-architecture.md (optional) — architecture conformance evals
+- docs/api-contracts.md (optional) — API contract validation evals
+- docs/security-review.md (optional) — security pattern verification evals
+- docs/database-schema.md (optional) — database schema conformance evals
+- docs/ux-spec.md (optional) — accessibility compliance evals
+- docs/dev-setup.md (optional) — configuration validation evals
+
+## Expected Outputs
+
+Core (always generated):
+- tests/evals/consistency.test.* — command matching, format checking, cross-doc refs
+- tests/evals/structure.test.* — file placement, shared code rules, test co-location
+- tests/evals/adherence.test.* — coding convention patterns, mock rules, TODO format
+- tests/evals/coverage.test.* — feature-to-code mapping, AC-to-test mapping
+- tests/evals/cross-doc.test.* — tech stack consistency, path consistency, terminology
+
+Conditional (generated when source doc exists):
+- tests/evals/architecture.test.* — layer direction, module boundaries, circular deps
+- tests/evals/api-contract.test.* — endpoint existence, methods, error codes
+- tests/evals/security.test.* — auth middleware, secrets, input validation, SQL injection
+- tests/evals/database.test.* — migration coverage, columns, indexes, relationships
+- tests/evals/accessibility.test.* — ARIA, alt text, focus styles, contrast
+- tests/evals/performance.test.* — budget files, bundle tracking, perf test existence
+- tests/evals/config.test.* — env var docs, dead config, startup validation
+- tests/evals/error-handling.test.* — bare catches, error responses tested, custom errors
+
+Supporting:
+- tests/evals/helpers.* — shared utilities
+- docs/eval-standards.md — documents what is and isn't checked
+- make eval target added to Makefile/package.json
+
+## Quality Criteria
+- (mvp) Consistency + Structure evals generated
+- (mvp) Evals use the project's own test framework from docs/tech-stack.md
+- (mvp) All generated evals pass on the current codebase (no false positives)
+- (mvp) Eval results are binary PASS/FAIL, not scores
+- (mvp) make eval is separate from make test and make check (opt-in for CI)
+- (deep) All applicable eval categories generated including security, API, DB, accessibility (conditional on source doc existence)
+- (deep) Adherence, security, and error-handling evals include exclusion mechanisms
+- (deep) docs/eval-standards.md explicitly documents what evals do NOT check
+- (deep) Full eval suite runs in under 30 seconds
+- (mvp) `make eval` (or equivalent) runs and all generated evals pass
+- (deep) Eval false-positive assessment: each eval category documents at least one scenario where valid code might incorrectly fail, with exclusion mechanism
+
+## Methodology Scaling
+- **deep**: All 13 eval categories (conditional on doc existence). Stack-specific
+  patterns. Coverage with keyword extraction. Cross-doc consistency. Architecture
+  conformance. API contract validation. Security patterns. Full suite.
+- **mvp**: Consistency + Structure only. Skip everything else.
+- **custom:depth(1-5)**:
+  - Depth 1-2: Consistency + Structure
+  - Depth 3: Add Adherence + Cross-doc
+  - Depth 4: Add Coverage + Architecture + Config + Error handling
+  - Depth 5: All 13 categories (Security, API, Database, Accessibility, Performance)
+
+## Mode Detection
+Update mode if tests/evals/ directory or docs/eval-standards.md exists. In
+update mode: regenerate consistency, structure, cross-doc, and conditional
+category evals. Preserve adherence, security, and error-handling eval
+exclusions. Regenerate coverage evals only if plan.md or user-stories.md
+changed. Add/remove conditional categories based on whether their source doc
+exists.
+
+## Update Mode Specifics
+- **Detect prior artifact**: tests/evals/ directory exists with eval test files
+- **Preserve**: adherence eval exclusions, security eval exclusions,
+  error-handling eval exclusions, custom helper utilities in tests/evals/helpers,
+  make eval target configuration
+- **Triggers for update**: source docs changed (coding-standards, project-structure,
+  tech-stack), new conditional source docs appeared (e.g., security-review.md
+  now exists), Makefile targets changed, user-stories.md changed
+- **Conflict resolution**: if a source doc was removed, archive its conditional
+  eval category rather than deleting; if exclusion patterns conflict with new
+  standards, flag for user review