fraim-framework 2.0.67 → 2.0.68
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -2
- package/dist/registry/ai-manager-rules/customer-development-phases/phase1-customer-profiling.md +0 -101
- package/dist/registry/ai-manager-rules/customer-development-phases/phase2-platform-discovery.md +0 -235
- package/dist/registry/ai-manager-rules/customer-development-phases/phase3-prospect-qualification.md +0 -243
- package/dist/registry/ai-manager-rules/customer-development-phases/phase4-inventory-compilation.md +0 -206
- package/dist/registry/ai-manager-rules/design-phases/design-completeness-review.md +0 -73
- package/dist/registry/ai-manager-rules/design-phases/design-design.md +0 -145
- package/dist/registry/ai-manager-rules/design-phases/design.md +0 -108
- package/dist/registry/ai-manager-rules/design-phases/finalize.md +0 -60
- package/dist/registry/ai-manager-rules/design-phases/validate.md +0 -125
- package/dist/registry/ai-manager-rules/design.json +0 -97
- package/dist/registry/ai-manager-rules/implement-phases/code.md +0 -323
- package/dist/registry/ai-manager-rules/implement-phases/completeness-review.md +0 -94
- package/dist/registry/ai-manager-rules/implement-phases/finalize.md +0 -177
- package/dist/registry/ai-manager-rules/implement-phases/implement-code.md +0 -283
- package/dist/registry/ai-manager-rules/implement-phases/implement-completeness-review.md +0 -120
- package/dist/registry/ai-manager-rules/implement-phases/implement-regression.md +0 -173
- package/dist/registry/ai-manager-rules/implement-phases/implement-repro.md +0 -104
- package/dist/registry/ai-manager-rules/implement-phases/implement-scoping.md +0 -100
- package/dist/registry/ai-manager-rules/implement-phases/implement-smoke.md +0 -237
- package/dist/registry/ai-manager-rules/implement-phases/implement-spike.md +0 -121
- package/dist/registry/ai-manager-rules/implement-phases/implement-validate.md +0 -375
- package/dist/registry/ai-manager-rules/implement-phases/quality-review.md +0 -304
- package/dist/registry/ai-manager-rules/implement-phases/regression.md +0 -159
- package/dist/registry/ai-manager-rules/implement-phases/repro.md +0 -101
- package/dist/registry/ai-manager-rules/implement-phases/scoping.md +0 -93
- package/dist/registry/ai-manager-rules/implement-phases/smoke.md +0 -225
- package/dist/registry/ai-manager-rules/implement-phases/spike.md +0 -118
- package/dist/registry/ai-manager-rules/implement-phases/validate.md +0 -347
- package/dist/registry/ai-manager-rules/implement.json +0 -153
- package/dist/registry/ai-manager-rules/retrospective.md +0 -116
- package/dist/registry/ai-manager-rules/shared-phases/address-pr-feedback.md +0 -188
- package/dist/registry/ai-manager-rules/shared-phases/finalize.md +0 -169
- package/dist/registry/ai-manager-rules/shared-phases/submit-pr.md +0 -202
- package/dist/registry/ai-manager-rules/shared-phases/wait-for-pr-review.md +0 -170
- package/dist/registry/ai-manager-rules/spec-phases/finalize.md +0 -60
- package/dist/registry/ai-manager-rules/spec-phases/spec-competitor-analysis.md +0 -105
- package/dist/registry/ai-manager-rules/spec-phases/spec-completeness-review.md +0 -66
- package/dist/registry/ai-manager-rules/spec-phases/spec-spec.md +0 -139
- package/dist/registry/ai-manager-rules/spec-phases/spec.md +0 -102
- package/dist/registry/ai-manager-rules/spec-phases/validate.md +0 -118
- package/dist/registry/ai-manager-rules/spec.json +0 -112
- package/dist/registry/ai-manager-rules/test.json +0 -98
- package/dist/registry/ai-manager-rules/user-survey-phases/phase1-survey-scoping.md +0 -60
- package/dist/registry/ai-manager-rules/user-survey-phases/phase2-survey-build-linkedin.md +0 -23
- package/dist/registry/ai-manager-rules/user-survey-phases/phase3-survey-build-reddit.md +0 -22
- package/dist/registry/ai-manager-rules/user-survey-phases/phase4-survey-build-x.md +0 -21
- package/dist/registry/ai-manager-rules/user-survey-phases/phase5-survey-build-facebook.md +0 -19
- package/dist/registry/ai-manager-rules/user-survey-phases/phase6-survey-build-custom.md +0 -15
- package/dist/registry/ai-manager-rules/user-survey-phases/phase7-survey-dispatch.md +0 -45
- package/dist/registry/delivery/address-feedback.md +0 -83
- package/dist/registry/delivery/retrospective.md +0 -78
- package/dist/registry/delivery/submit.md +0 -79
- package/dist/registry/providers/ado.json +0 -19
- package/dist/registry/providers/delivery-conversation.json +0 -9
- package/dist/registry/providers/delivery-pr.json +0 -9
- package/dist/registry/providers/github.json +0 -19
- package/dist/registry/rules/agent-success-criteria.md +0 -101
- package/dist/registry/rules/agent-testing-guidelines.md +0 -502
- package/dist/registry/rules/architecture.md +0 -126
- package/dist/registry/rules/communication.md +0 -122
- package/dist/registry/rules/continuous-learning.md +0 -55
- package/dist/registry/rules/debugging-multitenancy-issues.md +0 -85
- package/dist/registry/rules/ephemeral-execution.md +0 -54
- package/dist/registry/rules/git-safe-commands.md +0 -34
- package/dist/registry/rules/hitl-ppe-record-analysis.md +0 -302
- package/dist/registry/rules/integrity-and-test-ethics.md +0 -275
- package/dist/registry/rules/local-development.md +0 -275
- package/dist/registry/rules/mandatory-pre-completion-reflection.md +0 -193
- package/dist/registry/rules/merge-requirements.md +0 -238
- package/dist/registry/rules/simplicity.md +0 -118
- package/dist/registry/rules/software-development-lifecycle.md +0 -105
- package/dist/registry/rules/spike-first-development.md +0 -205
- package/dist/registry/rules/successful-debugging-patterns.md +0 -488
- package/dist/registry/scripts/build-scripts-generator.js +0 -205
- package/dist/registry/scripts/cleanup-branch.js +0 -287
- package/dist/registry/scripts/evaluate-code-quality.js +0 -66
- package/dist/registry/scripts/exec-with-timeout.js +0 -142
- package/dist/registry/scripts/fraim-config.js +0 -61
- package/dist/registry/scripts/generate-engagement-emails.js +0 -705
- package/dist/registry/scripts/generic-issues-api.js +0 -100
- package/dist/registry/scripts/newsletter-helpers.js +0 -671
- package/dist/registry/scripts/openapi-generator.js +0 -664
- package/dist/registry/scripts/performance/profile-server.js +0 -390
- package/dist/registry/scripts/profile-server.js +0 -389
- package/dist/registry/scripts/run-thank-you-workflow.js +0 -92
- package/dist/registry/scripts/send-newsletter-simple.js +0 -85
- package/dist/registry/scripts/send-thank-you-emails.js +0 -54
- package/dist/registry/scripts/validate-openapi-limits.js +0 -311
- package/dist/registry/scripts/validate-test-coverage.js +0 -262
- package/dist/registry/scripts/verify-test-coverage.js +0 -66
- package/dist/registry/stubs/workflows/azure/cost-optimization.md +0 -11
- package/dist/registry/stubs/workflows/bootstrap/create-architecture.md +0 -11
- package/dist/registry/stubs/workflows/bootstrap/detect-broken-windows.md +0 -11
- package/dist/registry/stubs/workflows/bootstrap/evaluate-code-quality.md +0 -11
- package/dist/registry/stubs/workflows/bootstrap/verify-test-coverage.md +0 -11
- package/dist/registry/stubs/workflows/brainstorming/blue-sky-brainstorming.md +0 -11
- package/dist/registry/stubs/workflows/brainstorming/codebase-brainstorming.md +0 -11
- package/dist/registry/stubs/workflows/business-development/create-business-plan.md +0 -11
- package/dist/registry/stubs/workflows/business-development/ideate-business-opportunity.md +0 -11
- package/dist/registry/stubs/workflows/business-development/price-product.md +0 -18
- package/dist/registry/stubs/workflows/compliance/detect-compliance-requirements.md +0 -11
- package/dist/registry/stubs/workflows/compliance/generate-audit-evidence.md +0 -11
- package/dist/registry/stubs/workflows/compliance/soc2-evidence-generator.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/insight-analysis.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/insight-triage.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/interview-preparation.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/linkedin-outreach.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/strategic-brainstorming.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/thank-customers.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/user-survey-dispatch.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/users-to-target.md +0 -11
- package/dist/registry/stubs/workflows/customer-development/weekly-newsletter.md +0 -11
- package/dist/registry/stubs/workflows/deploy/cloud-deployment.md +0 -11
- package/dist/registry/stubs/workflows/improve-fraim/contribute.md +0 -11
- package/dist/registry/stubs/workflows/improve-fraim/file-issue.md +0 -11
- package/dist/registry/stubs/workflows/learning/build-skillset.md +0 -11
- package/dist/registry/stubs/workflows/learning/synthesize-learnings.md +0 -11
- package/dist/registry/stubs/workflows/legal/contract-review-analysis.md +0 -11
- package/dist/registry/stubs/workflows/legal/nda.md +0 -11
- package/dist/registry/stubs/workflows/legal/patent-filing.md +0 -11
- package/dist/registry/stubs/workflows/legal/saas-contract-development.md +0 -11
- package/dist/registry/stubs/workflows/legal/trademark-filing.md +0 -11
- package/dist/registry/stubs/workflows/marketing/content-creation.md +0 -11
- package/dist/registry/stubs/workflows/marketing/convert-to-pdf.md +0 -11
- package/dist/registry/stubs/workflows/marketing/create-modern-website.md +0 -11
- package/dist/registry/stubs/workflows/marketing/domain-registration.md +0 -11
- package/dist/registry/stubs/workflows/marketing/evangelize.md +0 -11
- package/dist/registry/stubs/workflows/marketing/hbr-article.md +0 -11
- package/dist/registry/stubs/workflows/marketing/launch-checklist.md +0 -11
- package/dist/registry/stubs/workflows/marketing/marketing-strategy.md +0 -11
- package/dist/registry/stubs/workflows/marketing/storytelling.md +0 -11
- package/dist/registry/stubs/workflows/performance/analyze-performance.md +0 -11
- package/dist/registry/stubs/workflows/product-building/design.md +0 -11
- package/dist/registry/stubs/workflows/product-building/implement.md +0 -11
- package/dist/registry/stubs/workflows/product-building/iterate-on-pr-comments.md +0 -11
- package/dist/registry/stubs/workflows/product-building/prep-issue.md +0 -11
- package/dist/registry/stubs/workflows/product-building/prototype.md +0 -11
- package/dist/registry/stubs/workflows/product-building/resolve.md +0 -11
- package/dist/registry/stubs/workflows/product-building/retrospect.md +0 -11
- package/dist/registry/stubs/workflows/product-building/spec.md +0 -11
- package/dist/registry/stubs/workflows/product-building/test.md +0 -11
- package/dist/registry/stubs/workflows/productivity-report/productivity-report.md +0 -11
- package/dist/registry/stubs/workflows/quality-assurance/browser-validation.md +0 -11
- package/dist/registry/stubs/workflows/quality-assurance/iterative-improvement-cycle.md +0 -11
- package/dist/registry/stubs/workflows/replicate/replicate-discovery.md +0 -11
- package/dist/registry/stubs/workflows/replicate/replicate-to-issues.md +0 -11
- package/dist/registry/stubs/workflows/reviewer/review-implementation-vs-design-spec.md +0 -11
- package/dist/registry/stubs/workflows/reviewer/review-implementation-vs-feature-spec.md +0 -11
- package/dist/registry/stubs/workflows/startup-credits/aws-activate-application.md +0 -11
- package/dist/registry/stubs/workflows/startup-credits/google-cloud-application.md +0 -11
- package/dist/registry/stubs/workflows/startup-credits/microsoft-azure-application.md +0 -11
- package/dist/registry/templates/bootstrap/ARCHITECTURE-TEMPLATE.md +0 -53
- package/dist/registry/templates/bootstrap/CODE-QUALITY-REPORT-TEMPLATE.md +0 -37
- package/dist/registry/templates/bootstrap/TEST-COVERAGE-REPORT-TEMPLATE.md +0 -35
- package/dist/registry/templates/business-development/IDEATION-REPORT-TEMPLATE.md +0 -29
- package/dist/registry/templates/business-development/PRICING-STRATEGY-TEMPLATE.md +0 -126
- package/dist/registry/templates/customer-development/customer-interview-template.md +0 -99
- package/dist/registry/templates/customer-development/customer-persona-template.md +0 -69
- package/dist/registry/templates/customer-development/follow-up-email-templates.md +0 -132
- package/dist/registry/templates/customer-development/insight-analysis-template.md +0 -74
- package/dist/registry/templates/customer-development/prospect-inventory-template.csv +0 -3
- package/dist/registry/templates/customer-development/search-strategy-template.md +0 -123
- package/dist/registry/templates/customer-development/strategic-recommendations-template.md +0 -53
- package/dist/registry/templates/customer-development/thank-you-email-template.html +0 -124
- package/dist/registry/templates/customer-development/thank-you-note-template.md +0 -16
- package/dist/registry/templates/customer-development/triage-log-template.md +0 -278
- package/dist/registry/templates/customer-development/weekly-newsletter-template.html +0 -204
- package/dist/registry/templates/evidence/Design-Evidence.md +0 -30
- package/dist/registry/templates/evidence/Implementation-BugEvidence.md +0 -122
- package/dist/registry/templates/evidence/Implementation-FeatureEvidence.md +0 -160
- package/dist/registry/templates/evidence/Spec-Evidence.md +0 -19
- package/dist/registry/templates/help/HelpNeeded.md +0 -14
- package/dist/registry/templates/legal/NDA-TEMPLATE.md +0 -170
- package/dist/registry/templates/legal/PATENT-TEMPLATE.md +0 -372
- package/dist/registry/templates/legal/TRADEMARK-TEMPLATE.md +0 -339
- package/dist/registry/templates/legal/contract-review-checklist.md +0 -193
- package/dist/registry/templates/legal/review-report-template.md +0 -198
- package/dist/registry/templates/legal/saas-terms-template.md +0 -174
- package/dist/registry/templates/legal/sow-template.md +0 -117
- package/dist/registry/templates/legal/template-variables.md +0 -131
- package/dist/registry/templates/marketing/DOMAIN-REGISTRATION-TEMPLATE.md +0 -194
- package/dist/registry/templates/marketing/EVANGELIZE-TEMPLATE.md +0 -102
- package/dist/registry/templates/marketing/HBR-ARTICLE-TEMPLATE.md +0 -66
- package/dist/registry/templates/marketing/STORYTELLING-TEMPLATE.md +0 -130
- package/dist/registry/templates/marketing/WEBSITE-TEMPLATE.md +0 -262
- package/dist/registry/templates/marketing/github-pages-workflow.yml +0 -64
- package/dist/registry/templates/replicate/implementation-checklist.md +0 -39
- package/dist/registry/templates/replicate/use-cases-template.md +0 -88
- package/dist/registry/templates/retrospective/RETROSPECTIVE-TEMPLATE.md +0 -55
- package/dist/registry/templates/specs/BUGSPEC-TEMPLATE.md +0 -37
- package/dist/registry/templates/specs/FEATURESPEC-TEMPLATE.md +0 -66
- package/dist/registry/templates/specs/TECHSPEC-TEMPLATE.md +0 -39
- package/dist/registry/workflows/azure/cost-optimization.md +0 -772
- package/dist/registry/workflows/bootstrap/create-architecture.md +0 -37
- package/dist/registry/workflows/bootstrap/detect-broken-windows.md +0 -300
- package/dist/registry/workflows/bootstrap/evaluate-code-quality.md +0 -35
- package/dist/registry/workflows/bootstrap/verify-test-coverage.md +0 -36
- package/dist/registry/workflows/brainstorming/blue-sky-brainstorming.md +0 -211
- package/dist/registry/workflows/brainstorming/codebase-brainstorming.md +0 -159
- package/dist/registry/workflows/business-development/create-business-plan.md +0 -737
- package/dist/registry/workflows/business-development/ideate-business-opportunity.md +0 -55
- package/dist/registry/workflows/business-development/price-product.md +0 -325
- package/dist/registry/workflows/compliance/detect-compliance-requirements.md +0 -80
- package/dist/registry/workflows/compliance/generate-audit-evidence.md +0 -80
- package/dist/registry/workflows/compliance/soc2-evidence-generator.md +0 -332
- package/dist/registry/workflows/customer-development/insight-analysis.md +0 -156
- package/dist/registry/workflows/customer-development/insight-triage.md +0 -933
- package/dist/registry/workflows/customer-development/interview-preparation.md +0 -452
- package/dist/registry/workflows/customer-development/linkedin-outreach.md +0 -599
- package/dist/registry/workflows/customer-development/strategic-brainstorming.md +0 -146
- package/dist/registry/workflows/customer-development/thank-customers.md +0 -203
- package/dist/registry/workflows/customer-development/user-survey-dispatch.md +0 -312
- package/dist/registry/workflows/customer-development/users-to-target.md +0 -427
- package/dist/registry/workflows/customer-development/weekly-newsletter.md +0 -366
- package/dist/registry/workflows/deploy/cloud-deployment.md +0 -310
- package/dist/registry/workflows/improve-fraim/contribute.md +0 -32
- package/dist/registry/workflows/improve-fraim/file-issue.md +0 -32
- package/dist/registry/workflows/learning/build-skillset.md +0 -212
- package/dist/registry/workflows/learning/synthesize-learnings.md +0 -369
- package/dist/registry/workflows/legal/contract-review-analysis.md +0 -382
- package/dist/registry/workflows/legal/nda.md +0 -69
- package/dist/registry/workflows/legal/patent-filing.md +0 -76
- package/dist/registry/workflows/legal/saas-contract-development.md +0 -204
- package/dist/registry/workflows/legal/trademark-filing.md +0 -77
- package/dist/registry/workflows/marketing/content-creation.md +0 -40
- package/dist/registry/workflows/marketing/convert-to-pdf.md +0 -265
- package/dist/registry/workflows/marketing/create-modern-website.md +0 -456
- package/dist/registry/workflows/marketing/domain-registration.md +0 -323
- package/dist/registry/workflows/marketing/evangelize.md +0 -426
- package/dist/registry/workflows/marketing/hbr-article.md +0 -73
- package/dist/registry/workflows/marketing/launch-checklist.md +0 -37
- package/dist/registry/workflows/marketing/marketing-strategy.md +0 -45
- package/dist/registry/workflows/marketing/storytelling.md +0 -65
- package/dist/registry/workflows/performance/analyze-performance.md +0 -65
- package/dist/registry/workflows/product-building/design.md +0 -263
- package/dist/registry/workflows/product-building/implement.md +0 -771
- package/dist/registry/workflows/product-building/iterate-on-pr-comments.md +0 -70
- package/dist/registry/workflows/product-building/prep-issue.md +0 -41
- package/dist/registry/workflows/product-building/prototype.md +0 -60
- package/dist/registry/workflows/product-building/resolve.md +0 -175
- package/dist/registry/workflows/product-building/retrospect.md +0 -86
- package/dist/registry/workflows/product-building/spec.md +0 -303
- package/dist/registry/workflows/product-building/test.md +0 -125
- package/dist/registry/workflows/productivity-report/productivity-report.md +0 -263
- package/dist/registry/workflows/quality-assurance/browser-validation.md +0 -221
- package/dist/registry/workflows/quality-assurance/iterative-improvement-cycle.md +0 -562
- package/dist/registry/workflows/replicate/replicate-discovery.md +0 -336
- package/dist/registry/workflows/replicate/replicate-to-issues.md +0 -319
- package/dist/registry/workflows/reviewer/review-implementation-vs-design-spec.md +0 -694
- package/dist/registry/workflows/reviewer/review-implementation-vs-feature-spec.md +0 -734
- package/dist/registry/workflows/startup-credits/aws-activate-application.md +0 -535
- package/dist/registry/workflows/startup-credits/google-cloud-application.md +0 -647
- package/dist/registry/workflows/startup-credits/microsoft-azure-application.md +0 -538
|
@@ -1,694 +0,0 @@
|
|
|
1
|
-
# Review Implementation vs Design Spec
|
|
2
|
-
|
|
3
|
-
## INTENT
|
|
4
|
-
To systematically verify that the implementation matches the approved technical design (RFC), ensuring **completeness**, **correctness**, and **quality** of all technical requirements, test cases, and validation scenarios.
|
|
5
|
-
|
|
6
|
-
## PRINCIPLES
|
|
7
|
-
- **Completeness First**: Verify ALL RFC components are implemented before checking correctness
|
|
8
|
-
- **Design-Driven Validation**: Every requirement in the RFC must be verified
|
|
9
|
-
- **Test Matrix Compliance**: All test cases from design spec must exist and pass
|
|
10
|
-
- **Architecture Alignment**: Implementation must follow design decisions
|
|
11
|
-
- **Evidence-Based**: All claims must be backed by actual evidence
|
|
12
|
-
- **Gap Identification**: Systematically identify what's missing, not just what exists
|
|
13
|
-
- **Quality Standards**: Code must meet quality, maintainability, and documentation standards
|
|
14
|
-
|
|
15
|
-
{{include:orchestration/independent-reviewer-guidance.md}}
|
|
16
|
-
|
|
17
|
-
## REVIEW WORKFLOW
|
|
18
|
-
|
|
19
|
-
### Step 1: Issue Identification and Evidence Loading
|
|
20
|
-
- Get {issue_number} from context
|
|
21
|
-
|
|
22
|
-
- Verify issue has `phase:impl` and `status:needs-review` labels
|
|
23
|
-
- Locate RFC: `docs/rfcs/{issue}-*.md`
|
|
24
|
-
- If RFC doesn't exist: ❌ BLOCKER - Cannot review without design spec. Request design phase first.
|
|
25
|
-
- **Determine Issue Type**: Check if this is a bug fix or feature
|
|
26
|
-
- Check RFC title/description or issue title for "bug", "fix", "error", etc.
|
|
27
|
-
- Bug fixes require regression tests (see Step 4.4 Regression Test Verification)
|
|
28
|
-
- **Load Implementation Evidence**: Read `docs/evidence/{issue}-implementation-evidence.md`
|
|
29
|
-
- If evidence doesn't exist: ❌ BLOCKER - Implementation agent must create evidence document first
|
|
30
|
-
- Review evidence for completeness and quality
|
|
31
|
-
- **Check for Existing Feedback**: Check if `docs/evidence/{issue}-design-reviewer-feedback.md` exists
|
|
32
|
-
- If exists, this is an iteration (check iteration count)
|
|
33
|
-
- Track iteration number (max 3 iterations)
|
|
34
|
-
|
|
35
|
-
### Step 2: RFC Completeness Check
|
|
36
|
-
|
|
37
|
-
**CRITICAL**: Before checking technical correctness, verify ALL RFC components are addressed.
|
|
38
|
-
|
|
39
|
-
#### 2.1: Extract Full RFC Scope
|
|
40
|
-
**Read the ENTIRE RFC and extract**:
|
|
41
|
-
- All "Changes" sections (Component Changes, API Changes, Schema Changes, etc.)
|
|
42
|
-
- All new files to be created
|
|
43
|
-
- All existing files to be modified
|
|
44
|
-
- All new endpoints/APIs
|
|
45
|
-
- All configuration changes
|
|
46
|
-
- All test requirements (from Test Matrix)
|
|
47
|
-
- All validation scenarios (from Validation Plan)
|
|
48
|
-
- All risk mitigations
|
|
49
|
-
- All observability requirements
|
|
50
|
-
|
|
51
|
-
#### 2.2: Create Completeness Checklist
|
|
52
|
-
**Format**:
|
|
53
|
-
```markdown
|
|
54
|
-
## RFC Completeness Checklist
|
|
55
|
-
|
|
56
|
-
### Part 1: [Component Name from RFC]
|
|
57
|
-
- [ ] File: path/to/file.ts - [description from RFC]
|
|
58
|
-
- [ ] Test: path/to/test.ts - [description from RFC]
|
|
59
|
-
- [ ] Config: .fraim/config.json - [fields from RFC]
|
|
60
|
-
|
|
61
|
-
### Part 2: [Component Name from RFC]
|
|
62
|
-
- [ ] Endpoint: POST /api/endpoint - [description from RFC]
|
|
63
|
-
- [ ] File: path/to/file.ts - [description from RFC]
|
|
64
|
-
|
|
65
|
-
### Part 3: [Component Name from RFC]
|
|
66
|
-
...
|
|
67
|
-
|
|
68
|
-
**Total Items**: X
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
#### 2.3: Verify Each Item
|
|
72
|
-
**For each checklist item**:
|
|
73
|
-
- ✅ **Implemented**: Point to code/test files/config that implements it
|
|
74
|
-
- ⏸️ **Deferred**: Check if follow-up issue exists and is documented
|
|
75
|
-
- ❌ **Missing**: BLOCKER - must be implemented or explicitly deferred
|
|
76
|
-
|
|
77
|
-
#### 2.4: Calculate Completeness
|
|
78
|
-
```
|
|
79
|
-
Implemented: X/Y items (Z%)
|
|
80
|
-
Deferred: A items (with follow-up issues #123, #456)
|
|
81
|
-
Missing: B items (BLOCKER if >0 without follow-up issues)
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
**If completeness <100%**:
|
|
85
|
-
- ❌ REQUEST CHANGES - List all missing items
|
|
86
|
-
- Require either implementation or explicit deferral with follow-up issues
|
|
87
|
-
- Do NOT proceed to correctness checks until completeness is 100%
|
|
88
|
-
|
|
89
|
-
### Step 3: RFC Analysis (Correctness Check)
|
|
90
|
-
|
|
91
|
-
**ONLY proceed here if Step 2 shows 100% completeness**
|
|
92
|
-
|
|
93
|
-
**MUST read and extract:**
|
|
94
|
-
- Technical requirements (API changes, schema changes, etc.)
|
|
95
|
-
- Test Matrix section (unit, integration, E2E tests required)
|
|
96
|
-
- Validation Plan section (all validation scenarios)
|
|
97
|
-
- Architecture decisions (service boundaries, patterns)
|
|
98
|
-
- Risk mitigations (how they were addressed)
|
|
99
|
-
- Observability requirements (logs, metrics, alerts)
|
|
100
|
-
|
|
101
|
-
**Create a correctness checklist mapping:**
|
|
102
|
-
- List each requirement from RFC
|
|
103
|
-
- List each test case from Test Matrix
|
|
104
|
-
- List each validation scenario from Validation Plan
|
|
105
|
-
- This becomes your correctness review checklist
|
|
106
|
-
|
|
107
|
-
### Step 4: Implementation Code Review (Correctness Check)
|
|
108
|
-
**For each technical requirement in RFC:**
|
|
109
|
-
- [ ] Verify requirement is implemented in code
|
|
110
|
-
- Search codebase for implementation
|
|
111
|
-
- Check file paths match design
|
|
112
|
-
- Verify code exists and is not just stubbed
|
|
113
|
-
- [ ] Check code follows architecture decisions
|
|
114
|
-
- Service boundaries match design
|
|
115
|
-
- Patterns match design (e.g., spike-first, deterministic separation)
|
|
116
|
-
- No over-engineering beyond design scope
|
|
117
|
-
- [ ] Verify API/schema changes match RFC exactly
|
|
118
|
-
- Check OpenAPI spec matches (if applicable)
|
|
119
|
-
- Check database schema matches (if applicable)
|
|
120
|
-
- Verify field names, types, constraints match
|
|
121
|
-
- [ ] Check error handling matches design
|
|
122
|
-
- Error responses match design
|
|
123
|
-
- Error codes match design
|
|
124
|
-
- Error messages match design
|
|
125
|
-
- [ ] Verify observability (logs, metrics) matches design
|
|
126
|
-
- Log statements exist where specified
|
|
127
|
-
- Metrics tracked where specified
|
|
128
|
-
- Alerts configured where specified
|
|
129
|
-
|
|
130
|
-
**Document findings:**
|
|
131
|
-
- Create table: Requirement | Status | Evidence Location | Notes
|
|
132
|
-
|
|
133
|
-
### Step 4: Test Matrix Validation
|
|
134
|
-
**For each test type in RFC Test Matrix:**
|
|
135
|
-
|
|
136
|
-
#### Unit Tests
|
|
137
|
-
- [ ] **Count Required**: How many unit tests does RFC specify?
|
|
138
|
-
- [ ] **Count Found**: How many unit test files exist? (`test-{issue}-*.ts` or similar)
|
|
139
|
-
- [ ] **Verify Existence**: Check each listed test case exists
|
|
140
|
-
- [ ] **Run Tests**: Execute `npx tsx ~/.fraim/scripts/exec-with-timeout.ts --timeout 120 -- npm run test test-{issue}-*.ts` (or appropriate command)
|
|
141
|
-
- [ ] **Verify Exit Code**: Must be 0 (success), 124 indicates timeout (test hung)
|
|
142
|
-
- [ ] **Check Output**: All tests must pass (not just "some tests pass")
|
|
143
|
-
- [ ] **Document Results**: Include actual test output in review evidence
|
|
144
|
-
|
|
145
|
-
#### Integration Tests
|
|
146
|
-
- [ ] **Count Required**: How many integration tests does RFC specify?
|
|
147
|
-
- [ ] **Count Found**: How many integration test files exist?
|
|
148
|
-
- [ ] **Verify Existence**: Check each listed test case exists
|
|
149
|
-
- [ ] **Run Tests**: Execute integration tests with proper mocks
|
|
150
|
-
- [ ] **Verify Service Interactions**: Check mocks validate service calls match design
|
|
151
|
-
- [ ] **Verify Exit Code**: Must be 0 (success)
|
|
152
|
-
- [ ] **Document Results**: Include actual test output in review evidence
|
|
153
|
-
|
|
154
|
-
#### E2E Tests
|
|
155
|
-
- [ ] **Count Required**: How many E2E tests does RFC specify?
|
|
156
|
-
- [ ] **Count Found**: How many E2E test files exist?
|
|
157
|
-
- [ ] **Verify Existence**: Check each listed test case exists
|
|
158
|
-
- [ ] **Run Tests**: Execute E2E tests (may require server running)
|
|
159
|
-
- [ ] **Verify End-to-End Flow**: Complete workflow must work
|
|
160
|
-
- [ ] **Verify Exit Code**: Must be 0 (success)
|
|
161
|
-
- [ ] **Document Results**: Include actual test output in review evidence
|
|
162
|
-
|
|
163
|
-
**If test matrix says "X test cases" but only Y exist:**
|
|
164
|
-
- ❌ BLOCKER: Missing test cases
|
|
165
|
-
- Must identify which tests are missing (list specific test names)
|
|
166
|
-
- Cannot approve until all tests from matrix exist and pass
|
|
167
|
-
|
|
168
|
-
**Document findings:**
|
|
169
|
-
- Create table: Test Type | Required | Found | Passing | Evidence | Missing Tests
|
|
170
|
-
|
|
171
|
-
#### Test Quality Validation (Anti-Pattern Detection)
|
|
172
|
-
**🚨 CRITICAL: Verify tests validate runtime behavior, not code structure**
|
|
173
|
-
|
|
174
|
-
**For each test file found (test-{issue}-*.ts):**
|
|
175
|
-
- [ ] **Check for Static Analysis Anti-Pattern**: Verify tests don't use `fs.readFileSync()` for code structure checks
|
|
176
|
-
- Check: `grep -E "fs\.readFileSync|readFileSync" test-{issue}-*.ts`
|
|
177
|
-
- ❌ BLOCKER if found - tests are doing static analysis, not runtime testing
|
|
178
|
-
- Expected: Tests should execute code, not read source files
|
|
179
|
-
- Reference: `retrospectives/issue-723-weak-test-validation-anti-pattern-postmortem.md`
|
|
180
|
-
- [ ] **Verify Runtime Behavior Testing**: Check tests actually execute code paths
|
|
181
|
-
- Review test code to ensure it calls actual functions/services
|
|
182
|
-
- Verify tests check observable outcomes (API responses, database state, logs)
|
|
183
|
-
- ❌ BLOCKER if tests only check code structure without execution
|
|
184
|
-
- [ ] **Check for Mock Anti-Pattern**: Verify tests don't mock the core functionality being tested
|
|
185
|
-
- Check: Are mocks used for dependencies (✅) or for the thing being tested (❌)?
|
|
186
|
-
- ❌ BLOCKER if core functionality is mocked
|
|
187
|
-
- Expected: Mock dependencies, test actual implementation
|
|
188
|
-
- [ ] **Verify State Validation**: Check tests validate state changes
|
|
189
|
-
- Tests should check database state, API responses, or service interactions
|
|
190
|
-
- ❌ BLOCKER if tests don't validate any state changes
|
|
191
|
-
- Expected: Tests verify before/after state or observable outcomes
|
|
192
|
-
|
|
193
|
-
**Document findings:**
|
|
194
|
-
- Create table: Test File | Static Analysis Check | Runtime Behavior | Mock Usage | State Validation | Status
|
|
195
|
-
|
|
196
|
-
#### Regression Test Verification (For Bug Fixes)
|
|
197
|
-
**If this is a bug fix (check RFC or issue description):**
|
|
198
|
-
- [ ] **Identify Bug**: Understand what the bug was from RFC/issue
|
|
199
|
-
- [ ] **Check for Regression Test**: Verify test file includes regression test for the bug
|
|
200
|
-
- Check test file for test case that reproduces the bug
|
|
201
|
-
- Test should be named something like "regression test for bug X" or similar
|
|
202
|
-
- [ ] **Verify Test Pattern**: Regression test must follow correct pattern
|
|
203
|
-
- ❌ BLOCKER if test passes/fails regardless of bug (wrong pattern)
|
|
204
|
-
- ✅ CORRECT: Test fails with bug → Test passes with fix
|
|
205
|
-
- ❌ WRONG: Test passes with bug AND with fix (doesn't reproduce bug)
|
|
206
|
-
- ❌ WRONG: Test fails with bug AND with fix (environmental issue)
|
|
207
|
-
- [ ] **Verify Test Behavior** (if possible):
|
|
208
|
-
- Ideally: Revert fix temporarily, run test → should fail
|
|
209
|
-
- Reapply fix, run test → should pass
|
|
210
|
-
- If not possible, verify test logic would fail with buggy code
|
|
211
|
-
|
|
212
|
-
**Document findings:**
|
|
213
|
-
- Bug Fix: ✅ Yes / ❌ No
|
|
214
|
-
- Regression Test: ✅ Found / ❌ Missing
|
|
215
|
-
- Test Pattern: ✅ Correct (fails with bug, passes with fix) / ❌ Wrong pattern
|
|
216
|
-
- Notes: [Any issues with regression test]
|
|
217
|
-
|
|
218
|
-
### Step 4.5: Common Error Pattern Checks
|
|
219
|
-
**🚨 CRITICAL: Check for common test and code quality issues**
|
|
220
|
-
|
|
221
|
-
#### Test Structure Validation
|
|
222
|
-
**For each test file found (test-{issue}-*.ts):**
|
|
223
|
-
- [ ] **BaseTestCase Extension**: Verify test cases extend `BaseTestCase` interface
|
|
224
|
-
- Check: `grep -E "extends BaseTestCase|interface.*extends BaseTestCase" test-{issue}-*.ts`
|
|
225
|
-
- ❌ BLOCKER if test cases don't extend BaseTestCase
|
|
226
|
-
- Expected pattern: `interface MyTestCase extends BaseTestCase { ... }`
|
|
227
|
-
- All test case objects must use interface that extends BaseTestCase
|
|
228
|
-
- [ ] **Main Function**: Verify test file has `main()` function
|
|
229
|
-
- Check: `grep -E "async function main\(\)|function main\(\)" test-{issue}-*.ts`
|
|
230
|
-
- ❌ BLOCKER if main() function missing
|
|
231
|
-
- Expected pattern: `async function main() { ... runTests(...) }`
|
|
232
|
-
- Must have `main().catch(console.error)` at end of file
|
|
233
|
-
- [ ] **runTests Usage**: Verify test file uses `runTests()` function
|
|
234
|
-
- Check: `grep "runTests" test-{issue}-*.ts`
|
|
235
|
-
- ❌ BLOCKER if runTests() not used
|
|
236
|
-
- Expected: `runTests(TEST_CASES, runMyTest, 'Test Suite Name')`
|
|
237
|
-
- Must import runTests from './test-utils' or './test-utils.ts'
|
|
238
|
-
|
|
239
|
-
**Document findings:**
|
|
240
|
-
- Create table: Test File | BaseTestCase | main() | runTests() | Status
|
|
241
|
-
|
|
242
|
-
#### Code Quality Check
|
|
243
|
-
- [ ] **Run Code Quality Check**: Execute the code quality script directly from synced location.
|
|
244
|
-
- Command: `~/.fraim/scripts/code-quality-check.sh pre-pr`
|
|
245
|
-
- Verify exit code is 0 (success)
|
|
246
|
-
- Check for critical failures (❌) vs warnings (⚠️)
|
|
247
|
-
- ❌ BLOCKER if critical checks fail (e.g., `as any` usage, TypeScript errors)
|
|
248
|
-
- ⚠️ WARNING if warnings exist (document in feedback)
|
|
249
|
-
- [ ] **Review Quality Check Output**:
|
|
250
|
-
- Check for `as any` type bypassing in src/
|
|
251
|
-
- Check TypeScript compilation passes
|
|
252
|
-
- Check linter passes (warnings acceptable, errors are blockers)
|
|
253
|
-
- Check all test files referenced in evidence have been executed
|
|
254
|
-
|
|
255
|
-
**Document findings:**
|
|
256
|
-
- Code Quality Check: ✅ Passed / ❌ Failed / ⚠️ Warnings
|
|
257
|
-
- Critical Issues: [list any critical failures]
|
|
258
|
-
- Warnings: [list any warnings]
|
|
259
|
-
|
|
260
|
-
#### Evidence File Location Validation
|
|
261
|
-
- [ ] **Check Evidence Files**: Verify evidence files are ONLY in `docs/evidence/` folder
|
|
262
|
-
- Check: `find . -name "*evidence*.md" -not -path "./docs/evidence/*" -not -path "./.git/*" -not -path "./node_modules/*"`
|
|
263
|
-
- ❌ BLOCKER if evidence files found outside `docs/evidence/`
|
|
264
|
-
- Expected: All evidence files should be in `docs/evidence/{issue}-*.md`
|
|
265
|
-
- [ ] **Check for Evidence in Git**: Verify evidence folder is gitignored
|
|
266
|
-
- Check: `git check-ignore docs/evidence/` should return `docs/evidence/`
|
|
267
|
-
- ⚠️ WARNING if evidence folder not in .gitignore (should be gitignored)
|
|
268
|
-
- [ ] **Check PR for Evidence Files**: Verify no evidence files committed to repo
|
|
269
|
-
- Check git status for any files in docs/evidence/
|
|
270
|
-
- ❌ BLOCKER if evidence files are staged/committed
|
|
271
|
-
|
|
272
|
-
**Document findings:**
|
|
273
|
-
- Evidence Location Check: ✅ All in docs/evidence/ / ❌ Found outside
|
|
274
|
-
- Git Ignore Check: ✅ Ignored / ⚠️ Not ignored
|
|
275
|
-
- Committed Evidence: ✅ None / ❌ Found in commits
|
|
276
|
-
|
|
277
|
-
**If any blockers found:**
|
|
278
|
-
- ❌ BLOCKER: Must fix before approval
|
|
279
|
-
- List specific files and issues
|
|
280
|
-
- Cannot approve until all blockers resolved
|
|
281
|
-
|
|
282
|
-
### Step 5: Validation Plan Execution
|
|
283
|
-
**For each validation scenario in RFC Validation Plan:**
|
|
284
|
-
- [ ] **Scenario Identified**: Map each scenario from validation plan
|
|
285
|
-
- [ ] **Evidence Found**: Check PR evidence for scenario validation
|
|
286
|
-
- Look in PR comments for evidence
|
|
287
|
-
- Check for screenshots, curl outputs, test results
|
|
288
|
-
- Verify evidence shows actual execution (not just "tested")
|
|
289
|
-
- [ ] **Method Verified**: Ensure validation method matches (UI, API, DB, etc.)
|
|
290
|
-
- If RFC says "API validation", check for curl/Postman results
|
|
291
|
-
- If RFC says "Browser validation", check for screenshots/recordings
|
|
292
|
-
- If RFC says "Database validation", check for DB query results
|
|
293
|
-
- [ ] **Result Verified**: Ensure scenario actually passes (not just "tested")
|
|
294
|
-
- Check evidence shows success, not just attempt
|
|
295
|
-
- Verify no "pending" or "TODO" markers
|
|
296
|
-
|
|
297
|
-
**If validation plan has 10 scenarios but only 7 have evidence:**
|
|
298
|
-
- ❌ BLOCKER: Missing validation evidence
|
|
299
|
-
- Must identify which scenarios are missing (list specific scenario names)
|
|
300
|
-
- Cannot approve until all scenarios validated with evidence
|
|
301
|
-
|
|
302
|
-
**Document findings:**
|
|
303
|
-
- Create table: Scenario | Method | Evidence | Status | Missing Evidence
|
|
304
|
-
|
|
305
|
-
### Step 6: Architecture Compliance Check
|
|
306
|
-
**Verify implementation matches design:**
|
|
307
|
-
- [ ] Service boundaries match design
|
|
308
|
-
- Check service files match design boundaries
|
|
309
|
-
- Verify no cross-boundary violations
|
|
310
|
-
- [ ] Patterns match design (e.g., spike-first, deterministic separation)
|
|
311
|
-
- Check code follows patterns specified in RFC
|
|
312
|
-
- Verify no anti-patterns introduced
|
|
313
|
-
- [ ] Dependencies match design
|
|
314
|
-
- Check package.json matches design dependencies
|
|
315
|
-
- Verify no unexpected dependencies added
|
|
316
|
-
- [ ] No over-engineering beyond design scope
|
|
317
|
-
- Check code doesn't add features not in RFC
|
|
318
|
-
- Verify minimal implementation (not over-engineered)
|
|
319
|
-
|
|
320
|
-
**Document findings:**
|
|
321
|
-
- List any architecture deviations
|
|
322
|
-
- Note if deviations are acceptable or blockers
|
|
323
|
-
|
|
324
|
-
### Step 7: Risk Mitigation Verification
|
|
325
|
-
**For each risk in RFC Risks & Mitigations:**
|
|
326
|
-
- [ ] Verify mitigation is implemented
|
|
327
|
-
- Check code implements mitigation strategy
|
|
328
|
-
- Verify mitigation code exists and is not just commented
|
|
329
|
-
- [ ] Check mitigation actually addresses the risk
|
|
330
|
-
- Verify mitigation logic matches risk description
|
|
331
|
-
- Check mitigation is complete (not partial)
|
|
332
|
-
- [ ] Verify observability for risk detection
|
|
333
|
-
- Check logs/alerts exist for risk scenarios
|
|
334
|
-
- Verify monitoring is in place
|
|
335
|
-
|
|
336
|
-
**Document findings:**
|
|
337
|
-
- Create table: Risk | Mitigation Status | Implementation Location | Notes
|
|
338
|
-
|
|
339
|
-
### Step 8: Evidence Quality Review
|
|
340
|
-
**Verify PR evidence quality:**
|
|
341
|
-
- [ ] All test outputs included (not just "tests pass")
|
|
342
|
-
- Check for actual test command output
|
|
343
|
-
- Verify exit codes shown
|
|
344
|
-
- Check for test case names and results
|
|
345
|
-
- [ ] All validation scenarios documented with results
|
|
346
|
-
- Check each scenario has evidence
|
|
347
|
-
- Verify evidence shows actual execution
|
|
348
|
-
- [ ] No "pending" items that could be validated automatically
|
|
349
|
-
- Flag any "pending" items that could be tested
|
|
350
|
-
- Check for "TODO" or "needs manual validation" markers
|
|
351
|
-
- [ ] Build verification included
|
|
352
|
-
- Check for `npm run build` output
|
|
353
|
-
- Verify build succeeds
|
|
354
|
-
- [ ] Quality gate results included
|
|
355
|
-
- Check for quality gate script output
|
|
356
|
-
- Verify all gates pass (or warnings documented)
|
|
357
|
-
|
|
358
|
-
**Document findings:**
|
|
359
|
-
- List any evidence quality issues
|
|
360
|
-
- Note if evidence is sufficient or needs improvement
|
|
361
|
-
|
|
362
|
-
### Step 9: Review Decision and Feedback Creation
|
|
363
|
-
|
|
364
|
-
**Check Iteration Count:**
|
|
365
|
-
- If `docs/evidence/{issue}-design-reviewer-feedback.md` exists, check iteration number
|
|
366
|
-
- Maximum 3 iterations allowed
|
|
367
|
-
- If iteration 3 and still issues: ❌ REJECT (max iterations reached)
|
|
368
|
-
|
|
369
|
-
**Options:**
|
|
370
|
-
- ✅ **APPROVE**: All requirements met, tests pass, evidence complete
|
|
371
|
-
- ❌ **REJECT**: Missing requirements, tests fail, or evidence incomplete (after max iterations)
|
|
372
|
-
- ⚠️ **REQUEST CHANGES**: Minor gaps that need addressing (iteration < 3)
|
|
373
|
-
|
|
374
|
-
**If REQUEST CHANGES (iteration < 3):**
|
|
375
|
-
- **Create Feedback Document**: Create `docs/evidence/{issue}-design-reviewer-feedback.md`
|
|
376
|
-
- Use template below for feedback structure
|
|
377
|
-
- List specific blockers with evidence locations
|
|
378
|
-
- Provide actionable feedback for each issue
|
|
379
|
-
- Include iteration number
|
|
380
|
-
- Label issue `status:wip` (remove `status:needs-review`)
|
|
381
|
-
- Add PR comment referencing feedback document
|
|
382
|
-
- Implementation agent must address feedback and update evidence
|
|
383
|
-
- Implementation agent re-submits by marking `status:needs-review` again
|
|
384
|
-
|
|
385
|
-
**If REJECT (max iterations reached):**
|
|
386
|
-
- Create final feedback document with all remaining issues
|
|
387
|
-
- Label issue `status:wip` (remove `status:needs-review`)
|
|
388
|
-
- Add PR comment with final rejection notice
|
|
389
|
-
- Implementation agent must address all issues before re-submission
|
|
390
|
-
|
|
391
|
-
**If APPROVE:**
|
|
392
|
-
- **Create Summary for Feature Spec Review**: Create `docs/evidence/{issue}-design-review-summary.md`
|
|
393
|
-
- Summarize what was reviewed and approved
|
|
394
|
-
- Include key findings and evidence locations
|
|
395
|
-
- Provide context for feature spec review agent
|
|
396
|
-
- Label issue `status:design-review-passed` (remove `status:needs-review`)
|
|
397
|
-
- Add PR comment confirming design spec compliance
|
|
398
|
-
- Issue ready for Feature Spec Review (next workflow)
|
|
399
|
-
|
|
400
|
-
## FEEDBACK DOCUMENT TEMPLATE
|
|
401
|
-
|
|
402
|
-
Create `docs/evidence/{issue}-design-reviewer-feedback.md`:
|
|
403
|
-
|
|
404
|
-
```markdown
|
|
405
|
-
# Design Review Feedback - Issue #{issue}
|
|
406
|
-
|
|
407
|
-
## Iteration Information
|
|
408
|
-
- Iteration Number: {1, 2, or 3}
|
|
409
|
-
- Review Date: {date}
|
|
410
|
-
- Reviewer: Design Spec Review Agent
|
|
411
|
-
|
|
412
|
-
## Review Summary
|
|
413
|
-
- RFC Location: `docs/rfcs/{issue}-*.md`
|
|
414
|
-
- Evidence Reviewed: `docs/evidence/{issue}-implementation-evidence.md`
|
|
415
|
-
- Overall Status: ⚠️ REQUEST CHANGES
|
|
416
|
-
|
|
417
|
-
## Issues Found
|
|
418
|
-
|
|
419
|
-
### Critical Blockers (Must Fix)
|
|
420
|
-
1. **Issue Title**
|
|
421
|
-
- **Location**: `src/file.ts:line`
|
|
422
|
-
- **Problem**: [Specific problem description]
|
|
423
|
-
- **Expected**: [What RFC requires]
|
|
424
|
-
- **Found**: [What implementation has]
|
|
425
|
-
- **Action Required**: [Specific fix needed]
|
|
426
|
-
|
|
427
|
-
2. **Issue Title**
|
|
428
|
-
- [Same structure]
|
|
429
|
-
|
|
430
|
-
### Missing Requirements
|
|
431
|
-
1. **Requirement Name**
|
|
432
|
-
- **RFC Section**: [Section reference]
|
|
433
|
-
- **What's Missing**: [Specific requirement]
|
|
434
|
-
- **Action Required**: [What needs to be implemented]
|
|
435
|
-
|
|
436
|
-
### Test Matrix Issues
|
|
437
|
-
1. **Missing Test**: Test case name
|
|
438
|
-
- **RFC Test Matrix**: [Reference]
|
|
439
|
-
- **Action Required**: Create test file `test-{issue}-*.ts` with test case
|
|
440
|
-
|
|
441
|
-
### Test Structure Issues (Common Error Patterns)
|
|
442
|
-
1. **Test Not Extending BaseTestCase**: `test-{issue}-*.ts`
|
|
443
|
-
- **Problem**: Test case interface doesn't extend BaseTestCase
|
|
444
|
-
- **Expected**: `interface MyTestCase extends BaseTestCase { ... }`
|
|
445
|
-
- **Action Required**: Update test case interface to extend BaseTestCase
|
|
446
|
-
|
|
447
|
-
2. **Missing main() Function**: `test-{issue}-*.ts`
|
|
448
|
-
- **Problem**: Test file doesn't have main() function
|
|
449
|
-
- **Expected**: `async function main() { ... runTests(...) }` with `main().catch(console.error)`
|
|
450
|
-
- **Action Required**: Add main() function that calls runTests()
|
|
451
|
-
|
|
452
|
-
3. **Not Using runTests()**: `test-{issue}-*.ts`
|
|
453
|
-
- **Problem**: Test file doesn't use runTests() function
|
|
454
|
-
- **Expected**: Import runTests from './test-utils' and call it in main()
|
|
455
|
-
- **Action Required**: Refactor to use runTests() function
|
|
456
|
-
|
|
457
|
-
### Test Quality Issues (Anti-Patterns)
|
|
458
|
-
1. **Static Analysis Anti-Pattern**: `test-{issue}-*.ts`
|
|
459
|
-
- **Problem**: Tests use `fs.readFileSync()` to check code structure instead of runtime behavior
|
|
460
|
-
- **Expected**: Tests should execute code and validate outcomes, not read source files
|
|
461
|
-
- **Action Required**: Rewrite tests to validate runtime behavior
|
|
462
|
-
- **Reference**: `retrospectives/issue-723-weak-test-validation-anti-pattern-postmortem.md`
|
|
463
|
-
|
|
464
|
-
2. **No Runtime Behavior Validation**: `test-{issue}-*.ts`
|
|
465
|
-
- **Problem**: Tests don't execute actual code paths or validate observable outcomes
|
|
466
|
-
- **Expected**: Tests should call functions/services and check results
|
|
467
|
-
- **Action Required**: Add runtime behavior validation to tests
|
|
468
|
-
|
|
469
|
-
3. **Mocking Core Functionality**: `test-{issue}-*.ts`
|
|
470
|
-
- **Problem**: Tests mock the thing being tested instead of dependencies
|
|
471
|
-
- **Expected**: Mock dependencies, test actual implementation
|
|
472
|
-
- **Action Required**: Refactor mocks to only mock dependencies
|
|
473
|
-
|
|
474
|
-
4. **No State Validation**: `test-{issue}-*.ts`
|
|
475
|
-
- **Problem**: Tests don't validate state changes (database, API responses, etc.)
|
|
476
|
-
- **Expected**: Tests should verify before/after state or observable outcomes
|
|
477
|
-
- **Action Required**: Add state validation to tests
|
|
478
|
-
|
|
479
|
-
### Regression Test Issues (For Bug Fixes)
|
|
480
|
-
1. **Missing Regression Test**: `test-{issue}-*.ts`
|
|
481
|
-
- **Problem**: Bug fix doesn't have regression test
|
|
482
|
-
- **Expected**: Regression test that fails with bug, passes with fix
|
|
483
|
-
- **Action Required**: Add regression test following correct pattern
|
|
484
|
-
|
|
485
|
-
2. **Wrong Regression Test Pattern**: `test-{issue}-*.ts`
|
|
486
|
-
- **Problem**: Test passes/fails regardless of bug (doesn't reproduce bug)
|
|
487
|
-
- **Expected**: Test fails with bug → Test passes with fix
|
|
488
|
-
- **Action Required**: Fix regression test to follow correct pattern
|
|
489
|
-
- **Reference**: `retrospectives/task-api-calendar-event-id-bug-postmortem.md`
|
|
490
|
-
|
|
491
|
-
### Code Quality Issues
|
|
492
|
-
1. **Code Quality Check Failed**: [Specific issue]
|
|
493
|
-
- **Problem**: [What failed - e.g., "as any" usage, TypeScript errors]
|
|
494
|
-
- **Action Required**: Fix code quality issues and re-run check
|
|
495
|
-
- **Command**: Run quality check (fetch `scripts/code-quality-check.sh` via `get_fraim_file`)
|
|
496
|
-
|
|
497
|
-
### Evidence File Location Issues
|
|
498
|
-
1. **Evidence File Outside docs/evidence/**: [File path]
|
|
499
|
-
- **Problem**: Evidence file found outside `docs/evidence/` folder
|
|
500
|
-
- **Expected**: All evidence files should be in `docs/evidence/{issue}-*.md`
|
|
501
|
-
- **Action Required**: Move evidence file to `docs/evidence/` folder
|
|
502
|
-
|
|
503
|
-
### Validation Plan Issues
|
|
504
|
-
1. **Missing Validation**: Scenario name
|
|
505
|
-
- **RFC Validation Plan**: [Reference]
|
|
506
|
-
- **Action Required**: Execute validation and add evidence
|
|
507
|
-
|
|
508
|
-
### Evidence Quality Issues
|
|
509
|
-
1. **Missing Evidence**: [What's missing]
|
|
510
|
-
- **Action Required**: [What evidence needs to be added]
|
|
511
|
-
|
|
512
|
-
## Action Items for Implementation Agent
|
|
513
|
-
- [ ] Fix critical blocker 1
|
|
514
|
-
- [ ] Fix critical blocker 2
|
|
515
|
-
- [ ] Implement missing requirement X
|
|
516
|
-
- [ ] Add missing test case Y
|
|
517
|
-
- [ ] Execute missing validation Z
|
|
518
|
-
- [ ] Update evidence document with fixes
|
|
519
|
-
|
|
520
|
-
## Next Steps
|
|
521
|
-
1. Implementation agent addresses all feedback items
|
|
522
|
-
2. Implementation agent updates `docs/evidence/{issue}-implementation-evidence.md`
|
|
523
|
-
3. Implementation agent marks issue `status:needs-review` again
|
|
524
|
-
4. Design review agent will re-review (iteration {next_number})
|
|
525
|
-
```
|
|
526
|
-
|
|
527
|
-
## SUMMARY DOCUMENT TEMPLATE (For Feature Spec Review)
|
|
528
|
-
|
|
529
|
-
Create `docs/evidence/{issue}-design-review-summary.md` when approving:
|
|
530
|
-
|
|
531
|
-
```markdown
|
|
532
|
-
# Design Review Summary - Issue #{issue}
|
|
533
|
-
|
|
534
|
-
## Review Outcome
|
|
535
|
-
- Status: ✅ APPROVED
|
|
536
|
-
- Iterations: {number} (1-3)
|
|
537
|
-
- RFC Location: `docs/rfcs/{issue}-*.md`
|
|
538
|
-
|
|
539
|
-
## What Was Reviewed
|
|
540
|
-
- Technical Requirements: X/Y implemented ✅
|
|
541
|
-
- Test Matrix: All test cases exist and pass ✅
|
|
542
|
-
- Validation Plan: All scenarios validated ✅
|
|
543
|
-
- Architecture: Matches design ✅
|
|
544
|
-
- Risk Mitigations: All implemented ✅
|
|
545
|
-
|
|
546
|
-
## Key Findings
|
|
547
|
-
- All RFC requirements implemented correctly
|
|
548
|
-
- All tests from test matrix exist and pass
|
|
549
|
-
- All validation scenarios executed with evidence
|
|
550
|
-
- Architecture follows design decisions
|
|
551
|
-
- No over-engineering detected
|
|
552
|
-
|
|
553
|
-
## Evidence Locations
|
|
554
|
-
- Implementation Evidence: `docs/evidence/{issue}-implementation-evidence.md`
|
|
555
|
-
- Test Results: [locations]
|
|
556
|
-
- Validation Evidence: [locations]
|
|
557
|
-
|
|
558
|
-
## Notes for Feature Spec Review
|
|
559
|
-
- Technical implementation is complete and correct
|
|
560
|
-
- All technical requirements met
|
|
561
|
-
- Ready for functional/user experience validation
|
|
562
|
-
- No technical blockers for feature spec review
|
|
563
|
-
```
|
|
564
|
-
|
|
565
|
-
## REVIEW EVIDENCE TEMPLATE (PR Comment)
|
|
566
|
-
|
|
567
|
-
Add this as a PR comment:
|
|
568
|
-
|
|
569
|
-
```markdown
|
|
570
|
-
# Design Spec Review - Issue #{issue}
|
|
571
|
-
|
|
572
|
-
## RFC Compliance Summary
|
|
573
|
-
- RFC Location: `docs/rfcs/{issue}-*.md`
|
|
574
|
-
- Technical Requirements: X/Y implemented ✅/❌
|
|
575
|
-
- Test Matrix Compliance: X/Y test types complete ✅/❌
|
|
576
|
-
- Validation Plan Coverage: X/Y scenarios validated ✅/❌
|
|
577
|
-
- Architecture Compliance: ✅/❌
|
|
578
|
-
- Risk Mitigations: X/Y implemented ✅/❌
|
|
579
|
-
|
|
580
|
-
## Detailed Findings
|
|
581
|
-
|
|
582
|
-
### Technical Requirements
|
|
583
|
-
| Requirement | Status | Evidence Location | Notes |
|
|
584
|
-
|------------|--------|-------------------|-------|
|
|
585
|
-
| API endpoint X | ✅ | `src/api/...` | Matches RFC |
|
|
586
|
-
| Schema change Y | ❌ | Missing | RFC requires Z field |
|
|
587
|
-
|
|
588
|
-
### Test Matrix Validation
|
|
589
|
-
| Test Type | Required | Found | Passing | Evidence | Missing Tests |
|
|
590
|
-
|-----------|----------|-------|---------|----------|---------------|
|
|
591
|
-
| Unit Tests | 5 | 5 | ✅ | `test-{issue}.ts` lines 10-50 | None |
|
|
592
|
-
| Integration | 3 | 2 | ❌ | Partial | Missing test for scenario Z |
|
|
593
|
-
| E2E | 1 | 0 | ❌ | Not found | E2E test not implemented |
|
|
594
|
-
|
|
595
|
-
### Test Quality Validation
|
|
596
|
-
| Test File | Static Analysis | Runtime Behavior | Mock Usage | State Validation | Status |
|
|
597
|
-
|-----------|----------------|-------------------|------------|------------------|--------|
|
|
598
|
-
| test-{issue}.ts | ✅ No fs.readFileSync | ✅ Executes code | ✅ Mocks dependencies | ✅ Validates state | ✅ Pass |
|
|
599
|
-
| test-{issue}-2.ts | ❌ Uses fs.readFileSync | ❌ No execution | ❌ Mocks core | ❌ No validation | ❌ Fail |
|
|
600
|
-
|
|
601
|
-
### Regression Test Verification (If Bug Fix)
|
|
602
|
-
- Bug Fix: ✅ Yes / ❌ No
|
|
603
|
-
- Regression Test: ✅ Found / ❌ Missing
|
|
604
|
-
- Test Pattern: ✅ Correct (fails with bug, passes with fix) / ❌ Wrong pattern
|
|
605
|
-
- Notes: [Any issues with regression test]
|
|
606
|
-
|
|
607
|
-
### Validation Plan Coverage
|
|
608
|
-
| Scenario | Method | Evidence | Status | Missing Evidence |
|
|
609
|
-
|----------|--------|----------|--------|------------------|
|
|
610
|
-
| Create operation | API | PR comment | ✅ | None |
|
|
611
|
-
| Update operation | API | Missing | ❌ | No curl output found |
|
|
612
|
-
|
|
613
|
-
### Architecture Compliance
|
|
614
|
-
- Service Boundaries: ✅ Match design
|
|
615
|
-
- Patterns: ✅ Follow design patterns
|
|
616
|
-
- Dependencies: ⚠️ Extra dependency X added (not in RFC)
|
|
617
|
-
- Over-engineering: ✅ Minimal implementation
|
|
618
|
-
|
|
619
|
-
### Risk Mitigations
|
|
620
|
-
| Risk | Mitigation Status | Implementation Location | Notes |
|
|
621
|
-
|------|-------------------|-------------------------|-------|
|
|
622
|
-
| Risk X | ✅ Implemented | `src/...` | Matches design |
|
|
623
|
-
| Risk Y | ❌ Missing | N/A | Not implemented |
|
|
624
|
-
|
|
625
|
-
### Common Error Pattern Checks
|
|
626
|
-
- Test Structure: ✅ All tests extend BaseTestCase / ❌ Issues found
|
|
627
|
-
- Main Functions: ✅ All tests have main() / ❌ Missing in some tests
|
|
628
|
-
- runTests Usage: ✅ All tests use runTests() / ❌ Not used in some tests
|
|
629
|
-
- Code Quality Check: ✅ Passed / ❌ Failed / ⚠️ Warnings
|
|
630
|
-
- Evidence Location: ✅ All in docs/evidence/ / ❌ Found outside
|
|
631
|
-
|
|
632
|
-
### Evidence Quality
|
|
633
|
-
- Test Outputs: ✅ Complete (all test results included)
|
|
634
|
-
- Validation Evidence: ⚠️ Partial (missing scenario Y evidence)
|
|
635
|
-
- Build Verification: ✅ Included
|
|
636
|
-
- Quality Gate: ✅ Passed
|
|
637
|
-
|
|
638
|
-
## Decision
|
|
639
|
-
✅ APPROVE / ❌ REJECT / ⚠️ REQUEST CHANGES
|
|
640
|
-
|
|
641
|
-
## Feedback Document
|
|
642
|
-
- Location: `docs/evidence/{issue}-design-reviewer-feedback.md`
|
|
643
|
-
- Iteration: {1, 2, or 3}
|
|
644
|
-
|
|
645
|
-
## Blockers (if any)
|
|
646
|
-
1. Missing test case for scenario X (RFC Test Matrix requires it)
|
|
647
|
-
2. Validation scenario Y not executed (RFC Validation Plan requires it)
|
|
648
|
-
3. Technical requirement Z not implemented (RFC Technical Details requires it)
|
|
649
|
-
|
|
650
|
-
## Next Steps (if REQUEST CHANGES)
|
|
651
|
-
- [ ] Implementation agent addresses feedback in `docs/evidence/{issue}-design-reviewer-feedback.md`
|
|
652
|
-
- [ ] Implementation agent updates `docs/evidence/{issue}-implementation-evidence.md`
|
|
653
|
-
- [ ] Implementation agent marks `status:needs-review` for re-review
|
|
654
|
-
- [ ] Design review agent will re-review (max 3 iterations)
|
|
655
|
-
```
|
|
656
|
-
|
|
657
|
-
## EXAMPLES
|
|
658
|
-
|
|
659
|
-
### Good: Complete Review
|
|
660
|
-
```
|
|
661
|
-
Issue #533: Design Spec Review
|
|
662
|
-
✅ RFC Compliance: 15/15 requirements implemented
|
|
663
|
-
✅ Test Matrix: All 8 test cases exist and pass
|
|
664
|
-
✅ Validation Plan: All 5 scenarios validated
|
|
665
|
-
✅ Architecture: Matches design
|
|
666
|
-
✅ Evidence: Complete with test outputs
|
|
667
|
-
Decision: ✅ APPROVE
|
|
668
|
-
```
|
|
669
|
-
|
|
670
|
-
### Bad: Incomplete Review
|
|
671
|
-
```
|
|
672
|
-
Issue #533: Design Spec Review
|
|
673
|
-
⚠️ RFC Compliance: 12/15 requirements (3 missing)
|
|
674
|
-
❌ Test Matrix: 5/8 test cases exist (3 missing)
|
|
675
|
-
⚠️ Validation Plan: 3/5 scenarios validated (2 missing)
|
|
676
|
-
Decision: ❌ REJECT
|
|
677
|
-
Blockers: Missing tests, missing validations
|
|
678
|
-
```
|
|
679
|
-
|
|
680
|
-
## INTEGRATION
|
|
681
|
-
|
|
682
|
-
### Status Label Flow
|
|
683
|
-
```
|
|
684
|
-
phase:impl + status:needs-review
|
|
685
|
-
↓ (Design Spec Review)
|
|
686
|
-
status:design-review-passed (or status:wip if rejected)
|
|
687
|
-
↓ (Feature Spec Review - next workflow)
|
|
688
|
-
status:feature-review-passed
|
|
689
|
-
```
|
|
690
|
-
|
|
691
|
-
### After This Review
|
|
692
|
-
- If APPROVE: Issue moves to Feature Spec Review
|
|
693
|
-
- If REJECT: Issue returns to implementation (status:wip)
|
|
694
|
-
- Implementation agent addresses blockers and re-submits
|