fraim-framework 2.0.27 → 2.0.30

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (98) hide show
  1. package/.github/workflows/deploy-fraim.yml +1 -1
  2. package/dist/registry/scripts/build-scripts-generator.js +205 -0
  3. package/dist/registry/scripts/cleanup-branch.js +258 -0
  4. package/dist/registry/scripts/evaluate-code-quality.js +66 -0
  5. package/dist/registry/scripts/exec-with-timeout.js +142 -0
  6. package/dist/registry/scripts/fraim-config.js +61 -0
  7. package/dist/registry/scripts/generate-engagement-emails.js +630 -0
  8. package/dist/registry/scripts/generic-issues-api.js +100 -0
  9. package/dist/registry/scripts/newsletter-helpers.js +731 -0
  10. package/dist/registry/scripts/openapi-generator.js +664 -0
  11. package/dist/registry/scripts/performance/profile-server.js +390 -0
  12. package/dist/registry/scripts/run-thank-you-workflow.js +92 -0
  13. package/dist/registry/scripts/send-newsletter-simple.js +85 -0
  14. package/dist/registry/scripts/send-thank-you-emails.js +54 -0
  15. package/dist/registry/scripts/validate-openapi-limits.js +311 -0
  16. package/dist/registry/scripts/validate-test-coverage.js +262 -0
  17. package/dist/registry/scripts/verify-test-coverage.js +66 -0
  18. package/dist/src/cli/commands/init.js +14 -14
  19. package/dist/src/cli/commands/sync.js +4 -4
  20. package/dist/src/cli/fraim.js +24 -22
  21. package/dist/src/cli/setup/first-run.js +13 -6
  22. package/dist/src/fraim/db-service.js +26 -15
  23. package/dist/src/fraim/issues.js +67 -0
  24. package/dist/src/fraim-mcp-server.js +272 -18
  25. package/dist/src/utils/git-utils.js +1 -1
  26. package/dist/tests/debug-tools.js +79 -0
  27. package/dist/tests/esm-compat.js +11 -0
  28. package/dist/tests/test-chalk-esm-issue.js +159 -0
  29. package/dist/tests/test-chalk-real-world.js +265 -0
  30. package/dist/tests/test-chalk-regression.js +327 -0
  31. package/dist/tests/test-chalk-resolution-issue.js +304 -0
  32. package/dist/tests/test-fraim-install-chalk-issue.js +254 -0
  33. package/dist/tests/test-fraim-issues.js +59 -0
  34. package/dist/tests/test-genericization.js +1 -1
  35. package/dist/tests/test-mcp-connection.js +166 -0
  36. package/dist/tests/test-mcp-issue-integration.js +144 -0
  37. package/dist/tests/test-mcp-lifecycle-methods.js +312 -0
  38. package/dist/tests/test-node-compatibility.js +71 -0
  39. package/dist/tests/test-npm-install.js +66 -0
  40. package/dist/tests/test-npm-resolution-diagnostic.js +140 -0
  41. package/dist/tests/test-session-rehydration.js +145 -0
  42. package/dist/tests/test-standalone.js +2 -8
  43. package/dist/tests/test-telemetry.js +190 -0
  44. package/package.json +10 -8
  45. package/registry/agent-guardrails.md +62 -54
  46. package/registry/rules/agent-success-criteria.md +52 -0
  47. package/registry/rules/agent-testing-guidelines.md +502 -502
  48. package/registry/rules/communication.md +121 -121
  49. package/registry/rules/continuous-learning.md +54 -54
  50. package/registry/rules/ephemeral-execution.md +10 -5
  51. package/registry/rules/hitl-ppe-record-analysis.md +302 -302
  52. package/registry/rules/local-development.md +251 -251
  53. package/registry/rules/software-development-lifecycle.md +104 -104
  54. package/registry/rules/successful-debugging-patterns.md +482 -478
  55. package/registry/rules/telemetry.md +67 -0
  56. package/registry/scripts/build-scripts-generator.ts +216 -215
  57. package/registry/scripts/cleanup-branch.ts +303 -284
  58. package/registry/scripts/code-quality-check.sh +559 -559
  59. package/registry/scripts/detect-tautological-tests.sh +38 -38
  60. package/registry/scripts/evaluate-code-quality.ts +1 -1
  61. package/registry/scripts/generate-engagement-emails.ts +744 -744
  62. package/registry/scripts/generic-issues-api.ts +110 -150
  63. package/registry/scripts/newsletter-helpers.ts +874 -874
  64. package/registry/scripts/openapi-generator.ts +695 -693
  65. package/registry/scripts/performance/profile-server.ts +5 -3
  66. package/registry/scripts/prep-issue.sh +468 -455
  67. package/registry/scripts/validate-openapi-limits.ts +366 -365
  68. package/registry/scripts/validate-test-coverage.ts +280 -280
  69. package/registry/scripts/verify-pr-comments.sh +70 -70
  70. package/registry/scripts/verify-test-coverage.ts +1 -1
  71. package/registry/templates/bootstrap/ARCHITECTURE-TEMPLATE.md +53 -53
  72. package/registry/templates/evidence/Implementation-BugEvidence.md +85 -85
  73. package/registry/templates/evidence/Implementation-FeatureEvidence.md +120 -120
  74. package/registry/templates/marketing/HBR-ARTICLE-TEMPLATE.md +66 -0
  75. package/registry/workflows/bootstrap/create-architecture.md +2 -2
  76. package/registry/workflows/bootstrap/evaluate-code-quality.md +3 -3
  77. package/registry/workflows/bootstrap/verify-test-coverage.md +2 -2
  78. package/registry/workflows/customer-development/insight-analysis.md +156 -156
  79. package/registry/workflows/customer-development/interview-preparation.md +421 -421
  80. package/registry/workflows/customer-development/strategic-brainstorming.md +146 -146
  81. package/registry/workflows/customer-development/thank-customers.md +193 -191
  82. package/registry/workflows/customer-development/weekly-newsletter.md +362 -352
  83. package/registry/workflows/improve-fraim/contribute.md +32 -0
  84. package/registry/workflows/improve-fraim/file-issue.md +32 -0
  85. package/registry/workflows/marketing/hbr-article.md +73 -0
  86. package/registry/workflows/performance/analyze-performance.md +63 -59
  87. package/registry/workflows/product-building/design.md +3 -2
  88. package/registry/workflows/product-building/implement.md +4 -3
  89. package/registry/workflows/product-building/prep-issue.md +28 -17
  90. package/registry/workflows/product-building/resolve.md +3 -2
  91. package/registry/workflows/product-building/retrospect.md +3 -2
  92. package/registry/workflows/product-building/spec.md +5 -4
  93. package/registry/workflows/product-building/test.md +3 -2
  94. package/registry/workflows/quality-assurance/iterative-improvement-cycle.md +562 -562
  95. package/registry/workflows/replicate/website-discovery-analysis.md +3 -3
  96. package/registry/workflows/reviewer/review-implementation-vs-design-spec.md +632 -632
  97. package/registry/workflows/reviewer/review-implementation-vs-feature-spec.md +669 -669
  98. package/tsconfig.json +2 -1
@@ -1,669 +1,669 @@
1
- # Review Implementation vs Feature Spec
2
-
3
- ## INTENT
4
- To systematically verify that the implementation solves the customer problem and delivers the user experience described in the feature spec, ensuring functional completeness and user value.
5
-
6
- ## PRINCIPLES
7
- - **User-Centric Validation**: Every user scenario must work as specified
8
- - **Problem-Solution Fit**: Implementation must solve the customer problem
9
- - **Experience Fidelity**: UI/UX must match feature spec and mocks
10
- - **Validation Plan Execution**: All validation methods must be executed
11
- - **End-to-End Testing**: Complete user workflows must be validated with authentication
12
-
13
- ## REVIEW WORKFLOW
14
-
15
- ### Step 1: Issue Identification and Evidence Loading
16
- - Get {issue_number} from context
17
- - Verify issue has `status:design-review-passed` label
18
- - Locate feature spec: `docs/feature specs/{issue}-*.md`
19
- - If feature spec doesn't exist: ⚠️ Check if this is a bug fix (may not have feature spec). If feature, BLOCKER - request spec phase first.
20
- - **Load Design Review Summary**: Read `docs/evidence/{issue}-design-review-summary.md`
21
- - If summary doesn't exist: ❌ BLOCKER - Design review must complete first
22
- - Review summary to understand what was approved in design review
23
- - **Load Implementation Evidence**: Read `docs/evidence/{issue}-implementation-evidence.md`
24
- - Review evidence for functional/user experience aspects
25
- - **Check for Existing Feedback**: Check if `docs/evidence/{issue}-feature-reviewer-feedback.md` exists
26
- - If exists, this is an iteration (check iteration count)
27
- - Track iteration number (max 3 iterations)
28
-
29
- ### Step 2: Feature Spec Analysis
30
- **MUST read and extract:**
31
- - Customer problem statement
32
- - Customer's desired outcome
33
- - User experience workflow (step-by-step)
34
- - Validation Plan (all validation methods)
35
- - UI mocks (if applicable): `docs/feature specs/mocks/{issue}-*.png` (or similar)
36
- - Alternatives considered (verify chosen approach)
37
-
38
- **Create a checklist mapping:**
39
- - List each step in user experience workflow
40
- - List each validation method from Validation Plan
41
- - List each UI mock element (if applicable)
42
- - This becomes your review checklist
43
-
44
- ### Step 3: Customer Problem Verification
45
- **Verify implementation solves the problem:**
46
- - [ ] Read customer problem statement from spec
47
- - [ ] Understand desired outcome from spec
48
- - [ ] Verify implementation addresses the problem
49
- - Check code solves the stated problem
50
- - Verify solution matches desired outcome
51
- - [ ] Check if solution matches chosen alternative (if alternatives documented)
52
- - Review alternatives table in spec
53
- - Verify implementation matches chosen approach
54
- - Note if different approach was used (and why)
55
-
56
- **Document findings:**
57
- - Problem Statement: [from spec]
58
- - Desired Outcome: [from spec]
59
- - Solution Status: ✅ Addresses problem / ❌ Doesn't address problem
60
- - Notes: [any concerns or deviations]
61
-
62
- ### Step 3.5: Authentication Setup for End-to-End Testing
63
- **🚨 CRITICAL: Set up authentication before testing user workflows**
64
-
65
- **Determine if authentication is needed:**
66
- - [ ] Check if user experience workflow requires authentication
67
- - [ ] Check if validation plan includes browser/UI validation
68
- - [ ] Check if feature requires logged-in user state
69
-
70
- **If authentication is needed, choose one approach:**
71
-
72
- #### Option A: Request Credentials (if applicable)
73
- - [ ] **When to use**: If OAuth/SSO automation is not possible or if testing requires specific test accounts
74
- - [ ] Ask user: "I need credentials to test the end-to-end user workflow. Please provide:"
75
- - Username/email
76
- - Password (or test account credentials)
77
- - Any additional auth requirements (2FA, etc.)
78
- - [ ] Store credentials securely (do not commit to repo)
79
- - [ ] Use credentials to authenticate during testing
80
- - [ ] Document: "Using provided credentials for end-to-end testing"
81
-
82
- #### Option B: Cookie-Driven Browser Session (preferred for UI features)
83
- - [ ] **When to use**: For UI features that require authentication - this is the preferred method
84
- - [ ] Ask user: "I need to test the end-to-end user workflow. Please log in to the application in a browser session I can access, so I can validate the complete user experience. Once you're logged in, I'll use that session for all my validations."
85
- - [ ] **Wait for user confirmation**: User logs in manually (complies with OAuth ToS, avoids bot detection)
86
- - [ ] **Verify login state**:
87
- - Navigate to application URL
88
- - Check for authenticated state (user menu, profile icon, etc.)
89
- - Take snapshot to verify login succeeded
90
- - [ ] **Session persistence**:
91
- - Browser MCP tools automatically maintain cookies between invocations
92
- - Or use Playwright with `storageState` to save/load auth state
93
- - Session persists for all subsequent validations
94
- - [ ] Document: "Using cookie-driven browser session for end-to-end testing (user logged in manually)"
95
-
96
- **Browser Session Setup (if using Option B):**
97
-
98
- **Using Browser MCP Tools (if available):**
99
- - Browser MCP tools (like `browser_navigate`, `browser_snapshot`) automatically maintain session state
100
- - Cookies persist between browser invocations
101
- - User logs in once, agent can reuse session for all validations
102
- - Reference: `.ai-agents/workflows/customer-development/linkedin-outreach.md` shows this pattern
103
- - **Process**:
104
- 1. Use `browser_navigate` to open application
105
- 2. Use `browser_snapshot` to check if already logged in
106
- 3. If not logged in, ask user to log in manually
107
- 4. Use `browser_wait_for` to detect login success (e.g., wait for "Dashboard" text)
108
- 5. Verify with `browser_snapshot` - should see authenticated UI elements
109
- 6. Session automatically persists for future browser operations
110
-
111
- **Using Playwright (if MCP tools not available):**
112
- - Use Playwright with `storageState` to save/load authentication
113
- - Reference: `tests/auth-setup.ts` and `tests/playwright.config.ts`
114
- - **Process**:
115
- 1. Launch browser: `chromium.launch()`
116
- 2. Create context: `browser.newContext()`
117
- 3. Navigate to application
118
- 4. Ask user to log in manually in the browser window
119
- 5. Wait for login success (check URL or UI elements)
120
- 6. Save auth state: `context.storageState({ path: 'tests/auth-state.json' })`
121
- 7. Reuse auth state in future: `browser.newContext({ storageState: 'tests/auth-state.json' })`
122
-
123
- **Setup Steps (General):**
124
- 1. Ask user: "I need to test the end-to-end user workflow. Please log in to the application now. I'll wait for you to complete the login."
125
- 2. Wait for user confirmation that login is complete
126
- 3. Verify login: Navigate to authenticated page, check for user menu/profile/authenticated UI
127
- 4. Document: "Using authenticated browser session for end-to-end validation (user logged in manually)"
128
- 5. Reuse session for all subsequent workflow validations
129
- 6. Take screenshots at key validation points to show authenticated state
130
-
131
- **Verification:**
132
- - [ ] Test authentication works: Navigate to protected page
133
- - [ ] Verify user is logged in (check for user menu, profile, etc.)
134
- - [ ] Confirm session persists across multiple page navigations
135
- - [ ] Document authentication method used in review evidence
136
-
137
- **If authentication setup fails:**
138
- - [ ] Document what prevented authentication
139
- - [ ] Ask user for alternative approach
140
- - [ ] Note in review: "End-to-end testing limited due to authentication setup"
141
- - [ ] Proceed with available validation methods
142
-
143
- ### Step 4: User Experience Validation
144
- **For each step in the user experience workflow:**
145
-
146
- - [ ] **Step Identified**: Map each workflow step from spec
147
- - [ ] **Implementation Verified**: Check code implements the step
148
- - Search codebase for relevant code
149
- - Verify UI components exist (if UI feature)
150
- - Verify API endpoints exist (if API feature)
151
- - [ ] **Functionality Tested**: Actually test the step (browser, API, etc.)
152
- - Use authenticated browser session (from Step 3.5)
153
- - Navigate through workflow step-by-step
154
- - Verify each step works as described
155
- - [ ] **Result Verified**: Ensure step works as described
156
- - Check actual behavior matches spec description
157
- - Verify data appears correctly (not just UI renders)
158
- - Take screenshots at key points
159
-
160
- **Example for UI feature (with authentication):**
161
- - Step: "User clicks on tab X"
162
- - [ ] Verify tab X exists in UI code
163
- - [ ] **Use authenticated browser session** (from Step 3.5)
164
- - [ ] Navigate to page, click tab X
165
- - [ ] Verify tab X actually works (not just exists)
166
- - [ ] Take screenshot for evidence
167
- - Step: "User sees list Y"
168
- - [ ] Verify list Y is rendered
169
- - [ ] **Use authenticated browser session** to check list Y
170
- - [ ] Check list Y shows correct data (verify data matches expected)
171
- - [ ] Verify list Y matches spec description
172
- - [ ] Take screenshot showing list Y with data
173
-
174
- **Example for API feature:**
175
- - Step: "Call API A with params X, Y, Z"
176
- - [ ] Verify API endpoint exists
177
- - [ ] Test with curl: `curl -X POST /api/a -d '{"x":..., "y":..., "z":...}'`
178
- - [ ] Verify response matches expected outcome
179
- - [ ] Include curl output in evidence
180
-
181
- **Document findings:**
182
- - Create table: Step | Description | Status | Evidence | Notes
183
-
184
- ### Step 5: UI Mock Compliance (if applicable)
185
- **If feature spec includes UI mocks:**
186
- - [ ] Locate mocks: `docs/feature specs/mocks/{issue}-*.png` (or similar)
187
- - [ ] Compare implementation to mocks
188
- - Open implementation in browser (with authentication)
189
- - Compare layout to mock
190
- - Compare components to mock
191
- - Compare interactions to mock
192
- - [ ] Verify layout matches
193
- - Check positioning, spacing, colors
194
- - Verify responsive behavior (if specified)
195
- - [ ] Verify components match
196
- - Check all components from mock exist
197
- - Verify component styling matches
198
- - [ ] Verify interactions match
199
- - Test interactions described in spec
200
- - Verify behavior matches mock expectations
201
- - [ ] Document any intentional deviations with rationale
202
- - Note if deviations are acceptable
203
- - Explain why deviations exist
204
-
205
- **Document findings:**
206
- - Create table: Mock Element | Spec | Implementation | Match | Notes
207
-
208
- ### Step 6: Validation Plan Execution
209
- **For each validation method in feature spec Validation Plan:**
210
- - [ ] **Method Identified**: Map each validation method from spec
211
- - [ ] **Execution Verified**: Check evidence shows method was executed
212
- - Look in PR evidence for validation results
213
- - Check for screenshots, curl outputs, test results
214
- - [ ] **Result Verified**: Ensure validation actually passed
215
- - Check evidence shows success, not just attempt
216
- - Verify no "pending" or "TODO" markers
217
- - [ ] **Evidence Quality**: Verify evidence is complete (screenshots, curl output, etc.)
218
- - Screenshots for UI validation
219
- - Curl outputs for API validation
220
- - Database queries for data validation
221
-
222
- **Validation methods typically include:**
223
- - Browser validation (UI testing) - **REQUIRES authentication if feature needs login**
224
- - API validation (curl/Postman) - May require auth tokens
225
- - Database validation (check data state)
226
- - User workflow validation (end-to-end) - **REQUIRES full authentication setup**
227
-
228
- **For browser/UI validation with authentication:**
229
- - [ ] Use authenticated browser session from Step 3.5
230
- - [ ] Navigate through complete user workflow
231
- - [ ] Verify each step works as described in spec
232
- - [ ] Take screenshots at key points
233
- - [ ] Verify data appears correctly (not just UI renders)
234
- - [ ] Test error states if applicable
235
- - [ ] Document any issues found during end-to-end testing
236
-
237
- **Document findings:**
238
- - Create table: Validation Method | Required | Executed | Result | Evidence | Auth Status
239
-
240
- ### Step 7: Edge Cases, Negative Cases, and Error Handling
241
- **🚨 CRITICAL: Validate both positive and negative scenarios**
242
-
243
- #### Positive Case Validation (Already in Step 4)
244
- - [ ] All user workflow steps work (from Step 4)
245
- - [ ] Success states work correctly
246
-
247
- #### Negative Case Validation (What Shouldn't Work)
248
- **For each user workflow step, test negative scenarios:**
249
- - [ ] **Invalid Input Handling**: Test with invalid/malformed inputs
250
- - Example: Empty fields, wrong data types, out-of-range values
251
- - Verify feature rejects invalid inputs gracefully
252
- - Verify error messages are clear and actionable
253
- - [ ] **Unauthorized Access**: Test with wrong permissions/authentication
254
- - Example: Try to access feature without login (if required)
255
- - Example: Try to access another user's data
256
- - Verify proper error handling (401, 403, etc.)
257
- - [ ] **Missing Data Scenarios**: Test with missing required data
258
- - Example: Required fields not provided
259
- - Example: Dependencies not set up
260
- - Verify feature handles missing data gracefully
261
- - [ ] **Boundary Conditions**: Test at limits/edges
262
- - Example: Maximum/minimum values, empty lists, null states
263
- - Verify feature handles boundaries correctly
264
- - [ ] **Concurrent Operations**: Test simultaneous actions (if applicable)
265
- - Example: Multiple users accessing same resource
266
- - Verify no race conditions or data corruption
267
-
268
- **Document findings:**
269
- - Create table: Negative Scenario | Tested | Result | Error Handling | Notes
270
-
271
- #### Edge Cases from Spec
272
- - [ ] Review alternatives considered in spec
273
- - Read alternatives table from spec
274
- - Understand why chosen approach was selected
275
- - [ ] Verify chosen approach is implemented
276
- - Check code matches chosen alternative
277
- - Verify not using discarded alternatives
278
- - [ ] Check if edge cases from spec are handled
279
- - Review spec for edge case mentions
280
- - Verify edge cases are handled in code
281
- - Test each edge case explicitly
282
-
283
- **Document findings:**
284
- - Create table: Edge Case | From Spec | Tested | Result | Notes
285
-
286
- #### Error Handling Validation
287
- - [ ] **Error Messages**: Verify error messages are user-friendly
288
- - Test error scenarios and check messages
289
- - Verify messages explain what went wrong
290
- - Verify messages suggest how to fix (if applicable)
291
- - [ ] **Error States**: Verify error states match spec descriptions
292
- - Check UI shows appropriate error states
293
- - Verify error states don't break the UI
294
- - Verify users can recover from errors
295
- - [ ] **Error Logging**: Verify errors are logged appropriately
296
- - Check server logs for error details
297
- - Verify errors don't expose sensitive information
298
- - Verify errors are actionable for debugging
299
-
300
- **Document findings:**
301
- - Error Handling: ✅ User-friendly / ❌ Confusing
302
- - Error States: ✅ Match spec / ❌ Don't match
303
- - Error Logging: ✅ Appropriate / ❌ Missing or excessive
304
-
305
- ### Step 8: Functional Completeness Check
306
- **Verify feature is functionally complete:**
307
- - [ ] All user workflow steps work
308
- - Test complete workflow end-to-end
309
- - Verify no broken steps
310
- - [ ] No broken links or missing functionality
311
- - Check all links work
312
- - Verify no 404s or missing pages
313
- - [ ] **Positive Cases Work**: Success scenarios function correctly
314
- - Test all success paths
315
- - Verify success feedback is clear
316
- - Verify data is saved/updated correctly
317
- - [ ] **Negative Cases Handled**: Invalid inputs rejected gracefully
318
- - Test invalid inputs (from Step 7)
319
- - Verify proper error handling
320
- - Verify no crashes or data corruption
321
- - [ ] **Edge Cases Handled**: Boundary conditions work correctly
322
- - Test edge cases (from Step 7)
323
- - Verify feature handles limits correctly
324
- - Verify no unexpected behavior at boundaries
325
- - [ ] **Error Recovery**: Users can recover from errors
326
- - Test error scenarios
327
- - Verify users can retry or fix issues
328
- - Verify no dead-end error states
329
-
330
- **Document findings:**
331
- - Positive Cases: ✅ All working / ❌ Some broken
332
- - Negative Cases: ✅ Handled gracefully / ❌ Not handled
333
- - Edge Cases: ✅ Handled / ❌ Issues found
334
- - Error Recovery: ✅ Users can recover / ❌ Dead-ends found
335
-
336
- ### Step 9: Evidence Quality Review
337
- **Verify validation evidence quality:**
338
- - [ ] All validation methods have evidence
339
- - Check each method from validation plan has evidence
340
- - Verify evidence is complete
341
- - [ ] Evidence shows actual execution (not just "tested")
342
- - Check for screenshots, curl outputs, test results
343
- - Verify evidence shows real execution
344
- - [ ] Screenshots/recordings for UI features
345
- - Check screenshots show actual UI
346
- - Verify screenshots show authenticated state (if applicable)
347
- - [ ] API outputs for API features
348
- - Check curl outputs are included
349
- - Verify responses match expected
350
- - [ ] Database state for data features
351
- - Check database queries/results included
352
- - Verify data state matches expected
353
-
354
- **Document findings:**
355
- - List any missing evidence
356
- - Note if evidence quality is sufficient
357
-
358
- ### Step 10: Review Decision and Feedback Creation
359
-
360
- **Check Iteration Count:**
361
- - If `docs/evidence/{issue}-feature-reviewer-feedback.md` exists, check iteration number
362
- - Maximum 3 iterations allowed
363
- - If iteration 3 and still issues: ❌ REJECT (max iterations reached)
364
-
365
- **Options:**
366
- - ✅ **APPROVE**: All user scenarios work, problem solved, evidence complete
367
- - ❌ **REJECT**: User scenarios broken, problem not solved, or evidence incomplete (after max iterations)
368
- - ⚠️ **REQUEST CHANGES**: Minor UX issues or missing validations (iteration < 3)
369
-
370
- **If REQUEST CHANGES (iteration < 3):**
371
- - **Create Feedback Document**: Create `docs/evidence/{issue}-feature-reviewer-feedback.md`
372
- - Use template below for feedback structure
373
- - List specific blockers with evidence locations
374
- - Provide actionable feedback for each issue
375
- - Include iteration number
376
- - Label issue `status:wip` (remove `status:design-review-passed`)
377
- - Add PR comment referencing feedback document
378
- - Implementation agent must address feedback and update evidence
379
- - Implementation agent re-submits by marking `status:design-review-passed` again
380
-
381
- **If REJECT (max iterations reached):**
382
- - Create final feedback document with all remaining issues
383
- - Label issue `status:wip` (remove `status:design-review-passed`)
384
- - Add PR comment with final rejection notice
385
- - Implementation agent must address all issues before re-submission
386
-
387
- **If APPROVE:**
388
- - **Create Final Summary**: Create `docs/evidence/{issue}-feature-review-summary.md`
389
- - Summarize what was reviewed and approved
390
- - Include key findings and evidence locations
391
- - Confirm feature is ready for merge
392
- - Label issue `status:feature-review-passed` (remove `status:design-review-passed`)
393
- - Add PR comment confirming feature spec compliance
394
- - Issue ready for final approval/merge
395
-
396
- ## FEEDBACK DOCUMENT TEMPLATE
397
-
398
- Create `docs/evidence/{issue}-feature-reviewer-feedback.md`:
399
-
400
- ```markdown
401
- # Feature Review Feedback - Issue #{issue}
402
-
403
- ## Iteration Information
404
- - Iteration Number: {1, 2, or 3}
405
- - Review Date: {date}
406
- - Reviewer: Feature Spec Review Agent
407
-
408
- ## Review Summary
409
- - Feature Spec Location: `docs/feature specs/{issue}-*.md`
410
- - Design Review Summary: `docs/evidence/{issue}-design-review-summary.md`
411
- - Evidence Reviewed: `docs/evidence/{issue}-implementation-evidence.md`
412
- - Overall Status: ⚠️ REQUEST CHANGES
413
-
414
- ## Issues Found
415
-
416
- ### Critical Blockers (Must Fix)
417
- 1. **Issue Title**
418
- - **Location**: [UI component, API endpoint, or workflow step]
419
- - **Problem**: [Specific problem description]
420
- - **Expected**: [What feature spec requires]
421
- - **Found**: [What implementation has]
422
- - **Action Required**: [Specific fix needed]
423
-
424
- 2. **Issue Title**
425
- - [Same structure]
426
-
427
- ### User Experience Issues
428
- 1. **Workflow Step**: Step name
429
- - **Feature Spec Section**: [Section reference]
430
- - **Problem**: [What doesn't work]
431
- - **Action Required**: [What needs to be fixed]
432
-
433
- ### UI Mock Compliance Issues
434
- 1. **Mock Element**: Element name
435
- - **Mock Location**: `docs/feature specs/mocks/{issue}-*.png`
436
- - **Problem**: [What doesn't match]
437
- - **Action Required**: [What needs to be updated]
438
-
439
- ### Validation Plan Issues
440
- 1. **Missing Validation**: Method name
441
- - **Feature Spec Validation Plan**: [Reference]
442
- - **Action Required**: [What validation needs to be executed]
443
-
444
- ### Customer Problem Issues
445
- 1. **Problem Not Solved**: [Description]
446
- - **Feature Spec Problem**: [Reference]
447
- - **Action Required**: [What needs to be fixed]
448
-
449
- ### Negative Case Issues
450
- 1. **Invalid Input Not Handled**: [Scenario]
451
- - **Problem**: Feature doesn't reject invalid inputs gracefully
452
- - **Expected**: Clear error message, no crashes
453
- - **Action Required**: Add input validation and error handling
454
-
455
- 2. **Unauthorized Access Not Blocked**: [Scenario]
456
- - **Problem**: Feature allows unauthorized access
457
- - **Expected**: Proper 401/403 errors
458
- - **Action Required**: Add authorization checks
459
-
460
- 3. **Missing Data Not Handled**: [Scenario]
461
- - **Problem**: Feature crashes or behaves incorrectly with missing data
462
- - **Expected**: Graceful handling with clear error messages
463
- - **Action Required**: Add null/undefined checks
464
-
465
- ### Edge Case Issues
466
- 1. **Edge Case Not Handled**: [Edge case from spec]
467
- - **Feature Spec Section**: [Reference]
468
- - **Problem**: [What doesn't work]
469
- - **Action Required**: [What needs to be fixed]
470
-
471
- ### Error Handling Issues
472
- 1. **Error Messages Not User-Friendly**: [Error scenario]
473
- - **Problem**: Error messages are technical or confusing
474
- - **Expected**: Clear, actionable error messages
475
- - **Action Required**: Improve error messages
476
-
477
- 2. **Error Recovery Not Possible**: [Error scenario]
478
- - **Problem**: Users can't recover from errors (dead-end)
479
- - **Expected**: Users can retry or fix issues
480
- - **Action Required**: Add error recovery paths
481
-
482
- ## Action Items for Implementation Agent
483
- - [ ] Fix critical blocker 1
484
- - [ ] Fix critical blocker 2
485
- - [ ] Fix user experience issue X
486
- - [ ] Update UI to match mock Y
487
- - [ ] Execute missing validation Z
488
- - [ ] Update evidence document with fixes
489
-
490
- ## Next Steps
491
- 1. Implementation agent addresses all feedback items
492
- 2. Implementation agent updates `docs/evidence/{issue}-implementation-evidence.md`
493
- 3. Implementation agent marks issue `status:design-review-passed` again
494
- 4. Feature review agent will re-review (iteration {next_number})
495
- ```
496
-
497
- ## FINAL SUMMARY TEMPLATE (When Approving)
498
-
499
- Create `docs/evidence/{issue}-feature-review-summary.md` when approving:
500
-
501
- ```markdown
502
- # Feature Review Summary - Issue #{issue}
503
-
504
- ## Review Outcome
505
- - Status: ✅ APPROVED
506
- - Iterations: {number} (1-3)
507
- - Feature Spec Location: `docs/feature specs/{issue}-*.md`
508
-
509
- ## What Was Reviewed
510
- - Customer Problem: ✅ Solved
511
- - User Experience: All X steps working ✅
512
- - Validation Plan: All Y methods executed ✅
513
- - UI Mocks: Match ✅ (if applicable)
514
- - Authentication: Set up and working ✅
515
-
516
- ## Key Findings
517
- - Customer problem is solved
518
- - All user workflow steps work correctly
519
- - All validation methods executed with evidence
520
- - UI matches mocks (if applicable)
521
- - End-to-end testing completed with authentication
522
-
523
- ## Evidence Locations
524
- - Implementation Evidence: `docs/evidence/{issue}-implementation-evidence.md`
525
- - Design Review Summary: `docs/evidence/{issue}-design-review-summary.md`
526
- - Validation Evidence: [locations]
527
- - Screenshots: [locations]
528
-
529
- ## Final Status
530
- - Technical Review: ✅ Passed (from design review)
531
- - Functional Review: ✅ Passed (this review)
532
- - Ready for merge: ✅ Yes
533
- ```
534
-
535
- ## REVIEW EVIDENCE TEMPLATE (PR Comment)
536
-
537
- Add this as a PR comment:
538
-
539
- ```markdown
540
- # Feature Spec Review - Issue #{issue}
541
-
542
- ## Feature Spec Compliance Summary
543
- - Feature Spec Location: `docs/feature specs/{issue}-*.md`
544
- - Customer Problem: Solved ✅/❌
545
- - User Experience: Complete X/Y steps ✅/❌
546
- - Validation Plan: X/Y methods executed ✅/❌
547
- - UI Mocks: Match ✅/❌ (if applicable)
548
-
549
- ## Detailed Findings
550
-
551
- ### Customer Problem Verification
552
- - Problem Statement: [from spec]
553
- - Desired Outcome: [from spec]
554
- - Solution Status: ✅ Addresses problem / ❌ Doesn't address problem
555
- - Notes: [any concerns]
556
-
557
- ### Authentication Setup
558
- - Method Used: [Credentials provided / Cookie-driven browser session]
559
- - Status: ✅ Authenticated / ❌ Failed / ⚠️ Partial
560
- - Notes: [Any authentication issues encountered]
561
-
562
- ### User Experience Workflow Validation
563
- | Step | Description | Status | Evidence | Notes |
564
- |------|-------------|--------|----------|-------|
565
- | 1 | User clicks tab X | ✅ | Screenshot | Works correctly (tested with auth) |
566
- | 2 | User sees list Y | ❌ | Missing | List not rendering (tested with auth) |
567
- | 3 | User approves item Z | ⚠️ | Partial | Works but slow (tested with auth) |
568
-
569
- ### UI Mock Compliance (if applicable)
570
- | Mock Element | Spec | Implementation | Match | Notes |
571
- |--------------|------|----------------|-------|-------|
572
- | Layout | [description] | [description] | ✅ | Matches |
573
- | Component X | [description] | [description] | ❌ | Missing feature Y |
574
-
575
- ### Validation Plan Execution
576
- | Validation Method | Required | Executed | Result | Evidence | Auth Status |
577
- |------------------|----------|----------|--------|----------|-------------|
578
- | Browser validation | Yes | ✅ | Pass | Screenshot in PR | ✅ Authenticated |
579
- | API validation | Yes | ❌ | Missing | No curl output | ⚠️ Auth token missing |
580
- | Database validation | Yes | ✅ | Pass | DB query results | N/A |
581
- | End-to-end workflow | Yes | ✅ | Pass | Full workflow screenshots | ✅ Cookie session |
582
-
583
- ### Edge Cases, Negative Cases, and Error Handling
584
- - Alternatives Considered: ✅ Matches chosen approach
585
- - Edge Cases: ✅ All tested / ⚠️ Some not tested / ❌ Issues found
586
- - Negative Cases: ✅ All tested / ⚠️ Some not tested / ❌ Issues found
587
- - Error Handling: ✅ User-friendly / ❌ Confusing
588
- - Error Recovery: ✅ Users can recover / ❌ Dead-ends found
589
-
590
- ### Functional Completeness
591
- - All Workflow Steps: ✅ Working
592
- - Broken Links: ✅ None found
593
- - Positive Cases: ✅ All working
594
- - Negative Cases: ✅ Handled gracefully / ❌ Not handled
595
- - Edge Cases: ✅ Handled / ❌ Issues found
596
- - Error States: ✅ Handled gracefully
597
- - Error Recovery: ✅ Users can recover / ❌ Dead-ends found
598
- - Success States: ✅ Work correctly
599
-
600
- ### Evidence Quality
601
- - Validation Evidence: ✅ Complete
602
- - Screenshots: ✅ Included for UI features
603
- - API Outputs: ⚠️ Missing for some endpoints
604
- - Database State: ✅ Included
605
-
606
- ## Decision
607
- ✅ APPROVE / ❌ REJECT / ⚠️ REQUEST CHANGES
608
-
609
- ## Feedback Document
610
- - Location: `docs/evidence/{issue}-feature-reviewer-feedback.md`
611
- - Iteration: {1, 2, or 3}
612
-
613
- ## Blockers (if any)
614
- 1. User workflow step X doesn't work (tested with authentication)
615
- 2. Validation method Y not executed
616
- 3. UI doesn't match mock Z
617
- 4. Customer problem not actually solved
618
-
619
- ## Next Steps (if REQUEST CHANGES)
620
- - [ ] Implementation agent addresses feedback in `docs/evidence/{issue}-feature-reviewer-feedback.md`
621
- - [ ] Implementation agent updates `docs/evidence/{issue}-implementation-evidence.md`
622
- - [ ] Implementation agent marks `status:design-review-passed` for re-review
623
- - [ ] Feature review agent will re-review (max 3 iterations)
624
- ```
625
-
626
- ## EXAMPLES
627
-
628
- ### Good: Complete Review
629
- ```
630
- Issue #533: Feature Spec Review
631
- ✅ Customer Problem: Solved
632
- ✅ User Experience: 5/5 steps working
633
- ✅ Validation Plan: 4/4 methods executed
634
- ✅ UI Mocks: Match
635
- ✅ Authentication: Cookie session working
636
- Decision: ✅ APPROVE
637
- ```
638
-
639
- ### Bad: Incomplete Review
640
- ```
641
- Issue #533: Feature Spec Review
642
- ❌ Customer Problem: Not fully addressed
643
- ⚠️ User Experience: 3/5 steps working (2 broken)
644
- ❌ Validation Plan: 2/4 methods executed (2 missing)
645
- ❌ UI Mocks: Don't match
646
- Decision: ❌ REJECT
647
- Blockers: Broken workflow steps, missing validations
648
- ```
649
-
650
- ## INTEGRATION
651
-
652
- ### Status Label Flow
653
- ```
654
- status:design-review-passed
655
- ↓ (Feature Spec Review)
656
- status:feature-review-passed (or status:wip if rejected)
657
- ↓ (Final approval)
658
- Ready for merge
659
- ```
660
-
661
- ### After This Review
662
- - If APPROVE: Issue ready for final approval/merge
663
- - If REJECT: Issue returns to implementation (status:wip)
664
- - Implementation agent addresses blockers and re-submits
665
-
666
- ### Relationship to Design Spec Review
667
- - Design Spec Review validates technical correctness
668
- - Feature Spec Review validates user value and experience
669
- - Both must pass for issue to be complete
1
+ # Review Implementation vs Feature Spec
2
+
3
+ ## INTENT
4
+ To systematically verify that the implementation solves the customer problem and delivers the user experience described in the feature spec, ensuring functional completeness and user value.
5
+
6
+ ## PRINCIPLES
7
+ - **User-Centric Validation**: Every user scenario must work as specified
8
+ - **Problem-Solution Fit**: Implementation must solve the customer problem
9
+ - **Experience Fidelity**: UI/UX must match feature spec and mocks
10
+ - **Validation Plan Execution**: All validation methods must be executed
11
+ - **End-to-End Testing**: Complete user workflows must be validated with authentication
12
+
13
+ ## REVIEW WORKFLOW
14
+
15
+ ### Step 1: Issue Identification and Evidence Loading
16
+ - Get {issue_number} from context
17
+ - Verify issue has `status:design-review-passed` label
18
+ - Locate feature spec: `docs/feature specs/{issue}-*.md`
19
+ - If feature spec doesn't exist: ⚠️ Check if this is a bug fix (may not have feature spec). If feature, BLOCKER - request spec phase first.
20
+ - **Load Design Review Summary**: Read `docs/evidence/{issue}-design-review-summary.md`
21
+ - If summary doesn't exist: ❌ BLOCKER - Design review must complete first
22
+ - Review summary to understand what was approved in design review
23
+ - **Load Implementation Evidence**: Read `docs/evidence/{issue}-implementation-evidence.md`
24
+ - Review evidence for functional/user experience aspects
25
+ - **Check for Existing Feedback**: Check if `docs/evidence/{issue}-feature-reviewer-feedback.md` exists
26
+ - If exists, this is an iteration (check iteration count)
27
+ - Track iteration number (max 3 iterations)
28
+
29
+ ### Step 2: Feature Spec Analysis
30
+ **MUST read and extract:**
31
+ - Customer problem statement
32
+ - Customer's desired outcome
33
+ - User experience workflow (step-by-step)
34
+ - Validation Plan (all validation methods)
35
+ - UI mocks (if applicable): `docs/feature specs/mocks/{issue}-*.png` (or similar)
36
+ - Alternatives considered (verify chosen approach)
37
+
38
+ **Create a checklist mapping:**
39
+ - List each step in user experience workflow
40
+ - List each validation method from Validation Plan
41
+ - List each UI mock element (if applicable)
42
+ - This becomes your review checklist
43
+
44
+ ### Step 3: Customer Problem Verification
45
+ **Verify implementation solves the problem:**
46
+ - [ ] Read customer problem statement from spec
47
+ - [ ] Understand desired outcome from spec
48
+ - [ ] Verify implementation addresses the problem
49
+ - Check code solves the stated problem
50
+ - Verify solution matches desired outcome
51
+ - [ ] Check if solution matches chosen alternative (if alternatives documented)
52
+ - Review alternatives table in spec
53
+ - Verify implementation matches chosen approach
54
+ - Note if different approach was used (and why)
55
+
56
+ **Document findings:**
57
+ - Problem Statement: [from spec]
58
+ - Desired Outcome: [from spec]
59
+ - Solution Status: ✅ Addresses problem / ❌ Doesn't address problem
60
+ - Notes: [any concerns or deviations]
61
+
62
+ ### Step 3.5: Authentication Setup for End-to-End Testing
63
+ **🚨 CRITICAL: Set up authentication before testing user workflows**
64
+
65
+ **Determine if authentication is needed:**
66
+ - [ ] Check if user experience workflow requires authentication
67
+ - [ ] Check if validation plan includes browser/UI validation
68
+ - [ ] Check if feature requires logged-in user state
69
+
70
+ **If authentication is needed, choose one approach:**
71
+
72
+ #### Option A: Request Credentials (if applicable)
73
+ - [ ] **When to use**: If OAuth/SSO automation is not possible or if testing requires specific test accounts
74
+ - [ ] Ask user: "I need credentials to test the end-to-end user workflow. Please provide:"
75
+ - Username/email
76
+ - Password (or test account credentials)
77
+ - Any additional auth requirements (2FA, etc.)
78
+ - [ ] Store credentials securely (do not commit to repo)
79
+ - [ ] Use credentials to authenticate during testing
80
+ - [ ] Document: "Using provided credentials for end-to-end testing"
81
+
82
+ #### Option B: Cookie-Driven Browser Session (preferred for UI features)
83
+ - [ ] **When to use**: For UI features that require authentication - this is the preferred method
84
+ - [ ] Ask user: "I need to test the end-to-end user workflow. Please log in to the application in a browser session I can access, so I can validate the complete user experience. Once you're logged in, I'll use that session for all my validations."
85
+ - [ ] **Wait for user confirmation**: User logs in manually (complies with OAuth ToS, avoids bot detection)
86
+ - [ ] **Verify login state**:
87
+ - Navigate to application URL
88
+ - Check for authenticated state (user menu, profile icon, etc.)
89
+ - Take snapshot to verify login succeeded
90
+ - [ ] **Session persistence**:
91
+ - Browser MCP tools automatically maintain cookies between invocations
92
+ - Or use Playwright with `storageState` to save/load auth state
93
+ - Session persists for all subsequent validations
94
+ - [ ] Document: "Using cookie-driven browser session for end-to-end testing (user logged in manually)"
95
+
96
+ **Browser Session Setup (if using Option B):**
97
+
98
+ **Using Browser MCP Tools (if available):**
99
+ - Browser MCP tools (like `browser_navigate`, `browser_snapshot`) automatically maintain session state
100
+ - Cookies persist between browser invocations
101
+ - User logs in once, agent can reuse session for all validations
102
+ - Reference: Read `workflows/customer-development/linkedin-outreach.md` (via `get_fraim_file`) to see this pattern
103
+ - **Process**:
104
+ 1. Use `browser_navigate` to open application
105
+ 2. Use `browser_snapshot` to check if already logged in
106
+ 3. If not logged in, ask user to log in manually
107
+ 4. Use `browser_wait_for` to detect login success (e.g., wait for "Dashboard" text)
108
+ 5. Verify with `browser_snapshot` - should see authenticated UI elements
109
+ 6. Session automatically persists for future browser operations
110
+
111
+ **Using Playwright (if MCP tools not available):**
112
+ - Use Playwright with `storageState` to save/load authentication
113
+ - Reference: Look for authentication setup and Playwright configuration in the project's test files.
114
+ - **Process**:
115
+ 1. Launch browser: `chromium.launch()`
116
+ 2. Create context: `browser.newContext()`
117
+ 3. Navigate to application
118
+ 4. Ask user to log in manually in the browser window
119
+ 5. Wait for login success (check URL or UI elements)
120
+ 6. Save auth state: `context.storageState({ path: 'auth-state.json' })`
121
+ 7. Reuse auth state in future: `browser.newContext({ storageState: 'auth-state.json' })`
122
+
123
+ **Setup Steps (General):**
124
+ 1. Ask user: "I need to test the end-to-end user workflow. Please log in to the application now. I'll wait for you to complete the login."
125
+ 2. Wait for user confirmation that login is complete
126
+ 3. Verify login: Navigate to authenticated page, check for user menu/profile/authenticated UI
127
+ 4. Document: "Using authenticated browser session for end-to-end validation (user logged in manually)"
128
+ 5. Reuse session for all subsequent workflow validations
129
+ 6. Take screenshots at key validation points to show authenticated state
130
+
131
+ **Verification:**
132
+ - [ ] Test authentication works: Navigate to protected page
133
+ - [ ] Verify user is logged in (check for user menu, profile, etc.)
134
+ - [ ] Confirm session persists across multiple page navigations
135
+ - [ ] Document authentication method used in review evidence
136
+
137
+ **If authentication setup fails:**
138
+ - [ ] Document what prevented authentication
139
+ - [ ] Ask user for alternative approach
140
+ - [ ] Note in review: "End-to-end testing limited due to authentication setup"
141
+ - [ ] Proceed with available validation methods
142
+
143
+ ### Step 4: User Experience Validation
144
+ **For each step in the user experience workflow:**
145
+
146
+ - [ ] **Step Identified**: Map each workflow step from spec
147
+ - [ ] **Implementation Verified**: Check code implements the step
148
+ - Search codebase for relevant code
149
+ - Verify UI components exist (if UI feature)
150
+ - Verify API endpoints exist (if API feature)
151
+ - [ ] **Functionality Tested**: Actually test the step (browser, API, etc.)
152
+ - Use authenticated browser session (from Step 3.5)
153
+ - Navigate through workflow step-by-step
154
+ - Verify each step works as described
155
+ - [ ] **Result Verified**: Ensure step works as described
156
+ - Check actual behavior matches spec description
157
+ - Verify data appears correctly (not just UI renders)
158
+ - Take screenshots at key points
159
+
160
+ **Example for UI feature (with authentication):**
161
+ - Step: "User clicks on tab X"
162
+ - [ ] Verify tab X exists in UI code
163
+ - [ ] **Use authenticated browser session** (from Step 3.5)
164
+ - [ ] Navigate to page, click tab X
165
+ - [ ] Verify tab X actually works (not just exists)
166
+ - [ ] Take screenshot for evidence
167
+ - Step: "User sees list Y"
168
+ - [ ] Verify list Y is rendered
169
+ - [ ] **Use authenticated browser session** to check list Y
170
+ - [ ] Check list Y shows correct data (verify data matches expected)
171
+ - [ ] Verify list Y matches spec description
172
+ - [ ] Take screenshot showing list Y with data
173
+
174
+ **Example for API feature:**
175
+ - Step: "Call API A with params X, Y, Z"
176
+ - [ ] Verify API endpoint exists
177
+ - [ ] Test with curl: `curl -X POST /api/a -d '{"x":..., "y":..., "z":...}'`
178
+ - [ ] Verify response matches expected outcome
179
+ - [ ] Include curl output in evidence
180
+
181
+ **Document findings:**
182
+ - Create table: Step | Description | Status | Evidence | Notes
183
+
184
+ ### Step 5: UI Mock Compliance (if applicable)
185
+ **If feature spec includes UI mocks:**
186
+ - [ ] Locate mocks: `docs/feature specs/mocks/{issue}-*.png` (or similar)
187
+ - [ ] Compare implementation to mocks
188
+ - Open implementation in browser (with authentication)
189
+ - Compare layout to mock
190
+ - Compare components to mock
191
+ - Compare interactions to mock
192
+ - [ ] Verify layout matches
193
+ - Check positioning, spacing, colors
194
+ - Verify responsive behavior (if specified)
195
+ - [ ] Verify components match
196
+ - Check all components from mock exist
197
+ - Verify component styling matches
198
+ - [ ] Verify interactions match
199
+ - Test interactions described in spec
200
+ - Verify behavior matches mock expectations
201
+ - [ ] Document any intentional deviations with rationale
202
+ - Note if deviations are acceptable
203
+ - Explain why deviations exist
204
+
205
+ **Document findings:**
206
+ - Create table: Mock Element | Spec | Implementation | Match | Notes
207
+
208
+ ### Step 6: Validation Plan Execution
209
+ **For each validation method in feature spec Validation Plan:**
210
+ - [ ] **Method Identified**: Map each validation method from spec
211
+ - [ ] **Execution Verified**: Check evidence shows method was executed
212
+ - Look in PR evidence for validation results
213
+ - Check for screenshots, curl outputs, test results
214
+ - [ ] **Result Verified**: Ensure validation actually passed
215
+ - Check evidence shows success, not just attempt
216
+ - Verify no "pending" or "TODO" markers
217
+ - [ ] **Evidence Quality**: Verify evidence is complete (screenshots, curl output, etc.)
218
+ - Screenshots for UI validation
219
+ - Curl outputs for API validation
220
+ - Database queries for data validation
221
+
222
+ **Validation methods typically include:**
223
+ - Browser validation (UI testing) - **REQUIRES authentication if feature needs login**
224
+ - API validation (curl/Postman) - May require auth tokens
225
+ - Database validation (check data state)
226
+ - User workflow validation (end-to-end) - **REQUIRES full authentication setup**
227
+
228
+ **For browser/UI validation with authentication:**
229
+ - [ ] Use authenticated browser session from Step 3.5
230
+ - [ ] Navigate through complete user workflow
231
+ - [ ] Verify each step works as described in spec
232
+ - [ ] Take screenshots at key points
233
+ - [ ] Verify data appears correctly (not just UI renders)
234
+ - [ ] Test error states if applicable
235
+ - [ ] Document any issues found during end-to-end testing
236
+
237
+ **Document findings:**
238
+ - Create table: Validation Method | Required | Executed | Result | Evidence | Auth Status
239
+
240
+ ### Step 7: Edge Cases, Negative Cases, and Error Handling
241
+ **🚨 CRITICAL: Validate both positive and negative scenarios**
242
+
243
+ #### Positive Case Validation (Already in Step 4)
244
+ - [ ] All user workflow steps work (from Step 4)
245
+ - [ ] Success states work correctly
246
+
247
+ #### Negative Case Validation (What Shouldn't Work)
248
+ **For each user workflow step, test negative scenarios:**
249
+ - [ ] **Invalid Input Handling**: Test with invalid/malformed inputs
250
+ - Example: Empty fields, wrong data types, out-of-range values
251
+ - Verify feature rejects invalid inputs gracefully
252
+ - Verify error messages are clear and actionable
253
+ - [ ] **Unauthorized Access**: Test with wrong permissions/authentication
254
+ - Example: Try to access feature without login (if required)
255
+ - Example: Try to access another user's data
256
+ - Verify proper error handling (401, 403, etc.)
257
+ - [ ] **Missing Data Scenarios**: Test with missing required data
258
+ - Example: Required fields not provided
259
+ - Example: Dependencies not set up
260
+ - Verify feature handles missing data gracefully
261
+ - [ ] **Boundary Conditions**: Test at limits/edges
262
+ - Example: Maximum/minimum values, empty lists, null states
263
+ - Verify feature handles boundaries correctly
264
+ - [ ] **Concurrent Operations**: Test simultaneous actions (if applicable)
265
+ - Example: Multiple users accessing same resource
266
+ - Verify no race conditions or data corruption
267
+
268
+ **Document findings:**
269
+ - Create table: Negative Scenario | Tested | Result | Error Handling | Notes
270
+
271
+ #### Edge Cases from Spec
272
+ - [ ] Review alternatives considered in spec
273
+ - Read alternatives table from spec
274
+ - Understand why chosen approach was selected
275
+ - [ ] Verify chosen approach is implemented
276
+ - Check code matches chosen alternative
277
+ - Verify not using discarded alternatives
278
+ - [ ] Check if edge cases from spec are handled
279
+ - Review spec for edge case mentions
280
+ - Verify edge cases are handled in code
281
+ - Test each edge case explicitly
282
+
283
+ **Document findings:**
284
+ - Create table: Edge Case | From Spec | Tested | Result | Notes
285
+
286
+ #### Error Handling Validation
287
+ - [ ] **Error Messages**: Verify error messages are user-friendly
288
+ - Test error scenarios and check messages
289
+ - Verify messages explain what went wrong
290
+ - Verify messages suggest how to fix (if applicable)
291
+ - [ ] **Error States**: Verify error states match spec descriptions
292
+ - Check UI shows appropriate error states
293
+ - Verify error states don't break the UI
294
+ - Verify users can recover from errors
295
+ - [ ] **Error Logging**: Verify errors are logged appropriately
296
+ - Check server logs for error details
297
+ - Verify errors don't expose sensitive information
298
+ - Verify errors are actionable for debugging
299
+
300
+ **Document findings:**
301
+ - Error Handling: ✅ User-friendly / ❌ Confusing
302
+ - Error States: ✅ Match spec / ❌ Don't match
303
+ - Error Logging: ✅ Appropriate / ❌ Missing or excessive
304
+
305
+ ### Step 8: Functional Completeness Check
306
+ **Verify feature is functionally complete:**
307
+ - [ ] All user workflow steps work
308
+ - Test complete workflow end-to-end
309
+ - Verify no broken steps
310
+ - [ ] No broken links or missing functionality
311
+ - Check all links work
312
+ - Verify no 404s or missing pages
313
+ - [ ] **Positive Cases Work**: Success scenarios function correctly
314
+ - Test all success paths
315
+ - Verify success feedback is clear
316
+ - Verify data is saved/updated correctly
317
+ - [ ] **Negative Cases Handled**: Invalid inputs rejected gracefully
318
+ - Test invalid inputs (from Step 7)
319
+ - Verify proper error handling
320
+ - Verify no crashes or data corruption
321
+ - [ ] **Edge Cases Handled**: Boundary conditions work correctly
322
+ - Test edge cases (from Step 7)
323
+ - Verify feature handles limits correctly
324
+ - Verify no unexpected behavior at boundaries
325
+ - [ ] **Error Recovery**: Users can recover from errors
326
+ - Test error scenarios
327
+ - Verify users can retry or fix issues
328
+ - Verify no dead-end error states
329
+
330
+ **Document findings:**
331
+ - Positive Cases: ✅ All working / ❌ Some broken
332
+ - Negative Cases: ✅ Handled gracefully / ❌ Not handled
333
+ - Edge Cases: ✅ Handled / ❌ Issues found
334
+ - Error Recovery: ✅ Users can recover / ❌ Dead-ends found
335
+
336
+ ### Step 9: Evidence Quality Review
337
+ **Verify validation evidence quality:**
338
+ - [ ] All validation methods have evidence
339
+ - Check each method from validation plan has evidence
340
+ - Verify evidence is complete
341
+ - [ ] Evidence shows actual execution (not just "tested")
342
+ - Check for screenshots, curl outputs, test results
343
+ - Verify evidence shows real execution
344
+ - [ ] Screenshots/recordings for UI features
345
+ - Check screenshots show actual UI
346
+ - Verify screenshots show authenticated state (if applicable)
347
+ - [ ] API outputs for API features
348
+ - Check curl outputs are included
349
+ - Verify responses match expected
350
+ - [ ] Database state for data features
351
+ - Check database queries/results included
352
+ - Verify data state matches expected
353
+
354
+ **Document findings:**
355
+ - List any missing evidence
356
+ - Note if evidence quality is sufficient
357
+
358
+ ### Step 10: Review Decision and Feedback Creation
359
+
360
+ **Check Iteration Count:**
361
+ - If `docs/evidence/{issue}-feature-reviewer-feedback.md` exists, check iteration number
362
+ - Maximum 3 iterations allowed
363
+ - If iteration 3 and still issues: ❌ REJECT (max iterations reached)
364
+
365
+ **Options:**
366
+ - ✅ **APPROVE**: All user scenarios work, problem solved, evidence complete
367
+ - ❌ **REJECT**: User scenarios broken, problem not solved, or evidence incomplete (after max iterations)
368
+ - ⚠️ **REQUEST CHANGES**: Minor UX issues or missing validations (iteration < 3)
369
+
370
+ **If REQUEST CHANGES (iteration < 3):**
371
+ - **Create Feedback Document**: Create `docs/evidence/{issue}-feature-reviewer-feedback.md`
372
+ - Use template below for feedback structure
373
+ - List specific blockers with evidence locations
374
+ - Provide actionable feedback for each issue
375
+ - Include iteration number
376
+ - Label issue `status:wip` (remove `status:design-review-passed`)
377
+ - Add PR comment referencing feedback document
378
+ - Implementation agent must address feedback and update evidence
379
+ - Implementation agent re-submits by marking `status:design-review-passed` again
380
+
381
+ **If REJECT (max iterations reached):**
382
+ - Create final feedback document with all remaining issues
383
+ - Label issue `status:wip` (remove `status:design-review-passed`)
384
+ - Add PR comment with final rejection notice
385
+ - Implementation agent must address all issues before re-submission
386
+
387
+ **If APPROVE:**
388
+ - **Create Final Summary**: Create `docs/evidence/{issue}-feature-review-summary.md`
389
+ - Summarize what was reviewed and approved
390
+ - Include key findings and evidence locations
391
+ - Confirm feature is ready for merge
392
+ - Label issue `status:feature-review-passed` (remove `status:design-review-passed`)
393
+ - Add PR comment confirming feature spec compliance
394
+ - Issue ready for final approval/merge
395
+
396
+ ## FEEDBACK DOCUMENT TEMPLATE
397
+
398
+ Create `docs/evidence/{issue}-feature-reviewer-feedback.md`:
399
+
400
+ ```markdown
401
+ # Feature Review Feedback - Issue #{issue}
402
+
403
+ ## Iteration Information
404
+ - Iteration Number: {1, 2, or 3}
405
+ - Review Date: {date}
406
+ - Reviewer: Feature Spec Review Agent
407
+
408
+ ## Review Summary
409
+ - Feature Spec Location: `docs/feature specs/{issue}-*.md`
410
+ - Design Review Summary: `docs/evidence/{issue}-design-review-summary.md`
411
+ - Evidence Reviewed: `docs/evidence/{issue}-implementation-evidence.md`
412
+ - Overall Status: ⚠️ REQUEST CHANGES
413
+
414
+ ## Issues Found
415
+
416
+ ### Critical Blockers (Must Fix)
417
+ 1. **Issue Title**
418
+ - **Location**: [UI component, API endpoint, or workflow step]
419
+ - **Problem**: [Specific problem description]
420
+ - **Expected**: [What feature spec requires]
421
+ - **Found**: [What implementation has]
422
+ - **Action Required**: [Specific fix needed]
423
+
424
+ 2. **Issue Title**
425
+ - [Same structure]
426
+
427
+ ### User Experience Issues
428
+ 1. **Workflow Step**: Step name
429
+ - **Feature Spec Section**: [Section reference]
430
+ - **Problem**: [What doesn't work]
431
+ - **Action Required**: [What needs to be fixed]
432
+
433
+ ### UI Mock Compliance Issues
434
+ 1. **Mock Element**: Element name
435
+ - **Mock Location**: `docs/feature specs/mocks/{issue}-*.png`
436
+ - **Problem**: [What doesn't match]
437
+ - **Action Required**: [What needs to be updated]
438
+
439
+ ### Validation Plan Issues
440
+ 1. **Missing Validation**: Method name
441
+ - **Feature Spec Validation Plan**: [Reference]
442
+ - **Action Required**: [What validation needs to be executed]
443
+
444
+ ### Customer Problem Issues
445
+ 1. **Problem Not Solved**: [Description]
446
+ - **Feature Spec Problem**: [Reference]
447
+ - **Action Required**: [What needs to be fixed]
448
+
449
+ ### Negative Case Issues
450
+ 1. **Invalid Input Not Handled**: [Scenario]
451
+ - **Problem**: Feature doesn't reject invalid inputs gracefully
452
+ - **Expected**: Clear error message, no crashes
453
+ - **Action Required**: Add input validation and error handling
454
+
455
+ 2. **Unauthorized Access Not Blocked**: [Scenario]
456
+ - **Problem**: Feature allows unauthorized access
457
+ - **Expected**: Proper 401/403 errors
458
+ - **Action Required**: Add authorization checks
459
+
460
+ 3. **Missing Data Not Handled**: [Scenario]
461
+ - **Problem**: Feature crashes or behaves incorrectly with missing data
462
+ - **Expected**: Graceful handling with clear error messages
463
+ - **Action Required**: Add null/undefined checks
464
+
465
+ ### Edge Case Issues
466
+ 1. **Edge Case Not Handled**: [Edge case from spec]
467
+ - **Feature Spec Section**: [Reference]
468
+ - **Problem**: [What doesn't work]
469
+ - **Action Required**: [What needs to be fixed]
470
+
471
+ ### Error Handling Issues
472
+ 1. **Error Messages Not User-Friendly**: [Error scenario]
473
+ - **Problem**: Error messages are technical or confusing
474
+ - **Expected**: Clear, actionable error messages
475
+ - **Action Required**: Improve error messages
476
+
477
+ 2. **Error Recovery Not Possible**: [Error scenario]
478
+ - **Problem**: Users can't recover from errors (dead-end)
479
+ - **Expected**: Users can retry or fix issues
480
+ - **Action Required**: Add error recovery paths
481
+
482
+ ## Action Items for Implementation Agent
483
+ - [ ] Fix critical blocker 1
484
+ - [ ] Fix critical blocker 2
485
+ - [ ] Fix user experience issue X
486
+ - [ ] Update UI to match mock Y
487
+ - [ ] Execute missing validation Z
488
+ - [ ] Update evidence document with fixes
489
+
490
+ ## Next Steps
491
+ 1. Implementation agent addresses all feedback items
492
+ 2. Implementation agent updates `docs/evidence/{issue}-implementation-evidence.md`
493
+ 3. Implementation agent marks issue `status:design-review-passed` again
494
+ 4. Feature review agent will re-review (iteration {next_number})
495
+ ```
496
+
497
+ ## FINAL SUMMARY TEMPLATE (When Approving)
498
+
499
+ Create `docs/evidence/{issue}-feature-review-summary.md` when approving:
500
+
501
+ ```markdown
502
+ # Feature Review Summary - Issue #{issue}
503
+
504
+ ## Review Outcome
505
+ - Status: ✅ APPROVED
506
+ - Iterations: {number} (1-3)
507
+ - Feature Spec Location: `docs/feature specs/{issue}-*.md`
508
+
509
+ ## What Was Reviewed
510
+ - Customer Problem: ✅ Solved
511
+ - User Experience: All X steps working ✅
512
+ - Validation Plan: All Y methods executed ✅
513
+ - UI Mocks: Match ✅ (if applicable)
514
+ - Authentication: Set up and working ✅
515
+
516
+ ## Key Findings
517
+ - Customer problem is solved
518
+ - All user workflow steps work correctly
519
+ - All validation methods executed with evidence
520
+ - UI matches mocks (if applicable)
521
+ - End-to-end testing completed with authentication
522
+
523
+ ## Evidence Locations
524
+ - Implementation Evidence: `docs/evidence/{issue}-implementation-evidence.md`
525
+ - Design Review Summary: `docs/evidence/{issue}-design-review-summary.md`
526
+ - Validation Evidence: [locations]
527
+ - Screenshots: [locations]
528
+
529
+ ## Final Status
530
+ - Technical Review: ✅ Passed (from design review)
531
+ - Functional Review: ✅ Passed (this review)
532
+ - Ready for merge: ✅ Yes
533
+ ```
534
+
535
+ ## REVIEW EVIDENCE TEMPLATE (PR Comment)
536
+
537
+ Add this as a PR comment:
538
+
539
+ ```markdown
540
+ # Feature Spec Review - Issue #{issue}
541
+
542
+ ## Feature Spec Compliance Summary
543
+ - Feature Spec Location: `docs/feature specs/{issue}-*.md`
544
+ - Customer Problem: Solved ✅/❌
545
+ - User Experience: Complete X/Y steps ✅/❌
546
+ - Validation Plan: X/Y methods executed ✅/❌
547
+ - UI Mocks: Match ✅/❌ (if applicable)
548
+
549
+ ## Detailed Findings
550
+
551
+ ### Customer Problem Verification
552
+ - Problem Statement: [from spec]
553
+ - Desired Outcome: [from spec]
554
+ - Solution Status: ✅ Addresses problem / ❌ Doesn't address problem
555
+ - Notes: [any concerns]
556
+
557
+ ### Authentication Setup
558
+ - Method Used: [Credentials provided / Cookie-driven browser session]
559
+ - Status: ✅ Authenticated / ❌ Failed / ⚠️ Partial
560
+ - Notes: [Any authentication issues encountered]
561
+
562
+ ### User Experience Workflow Validation
563
+ | Step | Description | Status | Evidence | Notes |
564
+ |------|-------------|--------|----------|-------|
565
+ | 1 | User clicks tab X | ✅ | Screenshot | Works correctly (tested with auth) |
566
+ | 2 | User sees list Y | ❌ | Missing | List not rendering (tested with auth) |
567
+ | 3 | User approves item Z | ⚠️ | Partial | Works but slow (tested with auth) |
568
+
569
+ ### UI Mock Compliance (if applicable)
570
+ | Mock Element | Spec | Implementation | Match | Notes |
571
+ |--------------|------|----------------|-------|-------|
572
+ | Layout | [description] | [description] | ✅ | Matches |
573
+ | Component X | [description] | [description] | ❌ | Missing feature Y |
574
+
575
+ ### Validation Plan Execution
576
+ | Validation Method | Required | Executed | Result | Evidence | Auth Status |
577
+ |------------------|----------|----------|--------|----------|-------------|
578
+ | Browser validation | Yes | ✅ | Pass | Screenshot in PR | ✅ Authenticated |
579
+ | API validation | Yes | ❌ | Missing | No curl output | ⚠️ Auth token missing |
580
+ | Database validation | Yes | ✅ | Pass | DB query results | N/A |
581
+ | End-to-end workflow | Yes | ✅ | Pass | Full workflow screenshots | ✅ Cookie session |
582
+
583
+ ### Edge Cases, Negative Cases, and Error Handling
584
+ - Alternatives Considered: ✅ Matches chosen approach
585
+ - Edge Cases: ✅ All tested / ⚠️ Some not tested / ❌ Issues found
586
+ - Negative Cases: ✅ All tested / ⚠️ Some not tested / ❌ Issues found
587
+ - Error Handling: ✅ User-friendly / ❌ Confusing
588
+ - Error Recovery: ✅ Users can recover / ❌ Dead-ends found
589
+
590
+ ### Functional Completeness
591
+ - All Workflow Steps: ✅ Working
592
+ - Broken Links: ✅ None found
593
+ - Positive Cases: ✅ All working
594
+ - Negative Cases: ✅ Handled gracefully / ❌ Not handled
595
+ - Edge Cases: ✅ Handled / ❌ Issues found
596
+ - Error States: ✅ Handled gracefully
597
+ - Error Recovery: ✅ Users can recover / ❌ Dead-ends found
598
+ - Success States: ✅ Work correctly
599
+
600
+ ### Evidence Quality
601
+ - Validation Evidence: ✅ Complete
602
+ - Screenshots: ✅ Included for UI features
603
+ - API Outputs: ⚠️ Missing for some endpoints
604
+ - Database State: ✅ Included
605
+
606
+ ## Decision
607
+ ✅ APPROVE / ❌ REJECT / ⚠️ REQUEST CHANGES
608
+
609
+ ## Feedback Document
610
+ - Location: `docs/evidence/{issue}-feature-reviewer-feedback.md`
611
+ - Iteration: {1, 2, or 3}
612
+
613
+ ## Blockers (if any)
614
+ 1. User workflow step X doesn't work (tested with authentication)
615
+ 2. Validation method Y not executed
616
+ 3. UI doesn't match mock Z
617
+ 4. Customer problem not actually solved
618
+
619
+ ## Next Steps (if REQUEST CHANGES)
620
+ - [ ] Implementation agent addresses feedback in `docs/evidence/{issue}-feature-reviewer-feedback.md`
621
+ - [ ] Implementation agent updates `docs/evidence/{issue}-implementation-evidence.md`
622
+ - [ ] Implementation agent marks `status:design-review-passed` for re-review
623
+ - [ ] Feature review agent will re-review (max 3 iterations)
624
+ ```
625
+
626
+ ## EXAMPLES
627
+
628
+ ### Good: Complete Review
629
+ ```
630
+ Issue #533: Feature Spec Review
631
+ ✅ Customer Problem: Solved
632
+ ✅ User Experience: 5/5 steps working
633
+ ✅ Validation Plan: 4/4 methods executed
634
+ ✅ UI Mocks: Match
635
+ ✅ Authentication: Cookie session working
636
+ Decision: ✅ APPROVE
637
+ ```
638
+
639
+ ### Bad: Incomplete Review
640
+ ```
641
+ Issue #533: Feature Spec Review
642
+ ❌ Customer Problem: Not fully addressed
643
+ ⚠️ User Experience: 3/5 steps working (2 broken)
644
+ ❌ Validation Plan: 2/4 methods executed (2 missing)
645
+ ❌ UI Mocks: Don't match
646
+ Decision: ❌ REJECT
647
+ Blockers: Broken workflow steps, missing validations
648
+ ```
649
+
650
+ ## INTEGRATION
651
+
652
+ ### Status Label Flow
653
+ ```
654
+ status:design-review-passed
655
+ ↓ (Feature Spec Review)
656
+ status:feature-review-passed (or status:wip if rejected)
657
+ ↓ (Final approval)
658
+ Ready for merge
659
+ ```
660
+
661
+ ### After This Review
662
+ - If APPROVE: Issue ready for final approval/merge
663
+ - If REJECT: Issue returns to implementation (status:wip)
664
+ - Implementation agent addresses blockers and re-submits
665
+
666
+ ### Relationship to Design Spec Review
667
+ - Design Spec Review validates technical correctness
668
+ - Feature Spec Review validates user value and experience
669
+ - Both must pass for issue to be complete