agentic-loop 3.14.2 → 3.16.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/prd/SKILL.md +43 -86
- package/.claude/skills/prd-check/SKILL.md +48 -0
- package/README.md +7 -1
- package/bin/ralph.sh +3 -0
- package/dist/loopgram/prd-generator.d.ts +2 -0
- package/dist/loopgram/prd-generator.d.ts.map +1 -1
- package/dist/loopgram/prd-generator.js +2 -0
- package/dist/loopgram/prd-generator.js.map +1 -1
- package/package.json +1 -1
- package/ralph/hooks/protect-prd.sh +1 -1
- package/ralph/loop.sh +37 -17
- package/ralph/prd-check.sh +171 -3
- package/ralph/prd.sh +9 -32
- package/ralph/setup.sh +19 -0
- package/templates/checks/prd/check-example.sh +25 -0
- package/templates/prd-example.json +45 -26
- package/.claude/commands/prd.md +0 -770
package/.claude/commands/prd.md
DELETED
|
@@ -1,770 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
description: Generate an executable PRD for Ralph from an idea file or description.
|
|
3
|
-
---
|
|
4
|
-
|
|
5
|
-
# /prd - Generate PRD for Ralph
|
|
6
|
-
|
|
7
|
-
Generate executable stories for Ralph's autonomous development loop.
|
|
8
|
-
|
|
9
|
-
**CRITICAL: This command does NOT write code. It produces `.ralph/prd.json` only.**
|
|
10
|
-
|
|
11
|
-
## User Input
|
|
12
|
-
|
|
13
|
-
```text
|
|
14
|
-
$ARGUMENTS
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
## Workflow
|
|
18
|
-
|
|
19
|
-
### Step 1: Determine Input Type
|
|
20
|
-
|
|
21
|
-
**If `$ARGUMENTS` is empty:**
|
|
22
|
-
1. Check for idea files:
|
|
23
|
-
```bash
|
|
24
|
-
ls docs/ideas/*.md 2>/dev/null || echo "No ideas found"
|
|
25
|
-
```
|
|
26
|
-
2. Ask: "Would you like to:
|
|
27
|
-
- Convert an idea file (e.g., `/prd auth` for `docs/ideas/auth.md`)
|
|
28
|
-
- Describe a feature directly (e.g., `/prd 'Add user logout button'`)"
|
|
29
|
-
|
|
30
|
-
**If `$ARGUMENTS` looks like a file reference** (no spaces, matches `docs/ideas/*.md`):
|
|
31
|
-
- If it's a full path, use it directly
|
|
32
|
-
- If it's just a name like `content-engine`, look for `docs/ideas/content-engine.md`
|
|
33
|
-
- Proceed to "Read and Understand the Idea"
|
|
34
|
-
|
|
35
|
-
**If `$ARGUMENTS` is a description** (has spaces, is a sentence):
|
|
36
|
-
- This is the **quick PRD flow** - no `docs/ideas/` file created
|
|
37
|
-
- Good for small features that don't need documentation
|
|
38
|
-
- Skip to "Confirm Understanding" below
|
|
39
|
-
|
|
40
|
-
### Step 2a: Read and Understand the Idea (from file)
|
|
41
|
-
|
|
42
|
-
Read the idea file and summarize:
|
|
43
|
-
|
|
44
|
-
Say: "I've read `{path}`. Here's my understanding:
|
|
45
|
-
|
|
46
|
-
**Feature:** {name}
|
|
47
|
-
**Problem:** {one line}
|
|
48
|
-
**Solution:** {one line}
|
|
49
|
-
**Scope:** {key items}
|
|
50
|
-
|
|
51
|
-
I'll now split this into {N} stories for Ralph. Continue?"
|
|
52
|
-
|
|
53
|
-
**STOP and wait for user confirmation.**
|
|
54
|
-
|
|
55
|
-
### Step 2b: Confirm Understanding (from description)
|
|
56
|
-
|
|
57
|
-
If working from a direct description, first explore the codebase briefly:
|
|
58
|
-
```bash
|
|
59
|
-
ls -la src/ app/ 2>/dev/null | head -20
|
|
60
|
-
cat package.json 2>/dev/null | jq '{name, dependencies}' || true
|
|
61
|
-
cat pyproject.toml 2>/dev/null | head -20 || true
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
Then say: "I'll create a PRD for: **{description}**
|
|
65
|
-
|
|
66
|
-
Before I generate stories, quick questions:
|
|
67
|
-
1. **Type:** Frontend or backend?
|
|
68
|
-
2. **Scale:** Any specific limits (users, items, rate limits)?
|
|
69
|
-
3. **Anything else** I should know?
|
|
70
|
-
|
|
71
|
-
(Or say 'go' to proceed with defaults)"
|
|
72
|
-
|
|
73
|
-
**STOP and wait for user input** (can be brief or 'go').
|
|
74
|
-
|
|
75
|
-
### Step 3: Check for Existing PRD
|
|
76
|
-
|
|
77
|
-
```bash
|
|
78
|
-
cat .ralph/prd.json 2>/dev/null
|
|
79
|
-
```
|
|
80
|
-
|
|
81
|
-
If it exists, read it and say:
|
|
82
|
-
"`.ralph/prd.json` exists with {N} stories ({M} completed, {P} pending).
|
|
83
|
-
|
|
84
|
-
Options:
|
|
85
|
-
- **'append'** - Add new stories to the existing PRD (recommended)
|
|
86
|
-
- **'overwrite'** - Replace it entirely
|
|
87
|
-
- **'cancel'** - Stop here"
|
|
88
|
-
|
|
89
|
-
**STOP and wait for user choice.**
|
|
90
|
-
|
|
91
|
-
If user chooses **'append'**:
|
|
92
|
-
- Find highest existing story number (ignore prefix - could be US-005 or TASK-005)
|
|
93
|
-
- **Always use TASK- prefix** for new stories (e.g., if highest is US-005 or TASK-005, new stories start at TASK-006)
|
|
94
|
-
- New stories will be added after existing ones
|
|
95
|
-
|
|
96
|
-
### Step 4: Split into Stories
|
|
97
|
-
|
|
98
|
-
Break the idea into small, executable stories:
|
|
99
|
-
|
|
100
|
-
- Each story completable in one Claude session (~10-15 min)
|
|
101
|
-
- Max 3-4 acceptance criteria per story
|
|
102
|
-
- Max 10 stories (suggest phases if more needed)
|
|
103
|
-
- If appending, start IDs from the next available number
|
|
104
|
-
|
|
105
|
-
### Step 5: Write Draft PRD
|
|
106
|
-
|
|
107
|
-
Write the initial PRD to `.ralph/prd.json`:
|
|
108
|
-
|
|
109
|
-
1. Ensure .ralph directory exists:
|
|
110
|
-
```bash
|
|
111
|
-
mkdir -p .ralph && touch .ralph/.prd-edit-allowed
|
|
112
|
-
```
|
|
113
|
-
|
|
114
|
-
2. Write all stories to `.ralph/prd.json`
|
|
115
|
-
- If **appending**: Read existing JSON, add new stories, update count
|
|
116
|
-
|
|
117
|
-
**Do not present to user yet - validation comes next.**
|
|
118
|
-
|
|
119
|
-
### Step 6: Validate and Fix (MANDATORY)
|
|
120
|
-
|
|
121
|
-
**Read back the PRD you just wrote and validate EVERY story.**
|
|
122
|
-
|
|
123
|
-
```bash
|
|
124
|
-
cat .ralph/prd.json
|
|
125
|
-
```
|
|
126
|
-
|
|
127
|
-
For EACH story, check:
|
|
128
|
-
|
|
129
|
-
#### 6a. Testability
|
|
130
|
-
- ❌ `grep -q 'function' file.py` → Only checks code exists, not behavior
|
|
131
|
-
- ❌ `test -f src/component.tsx` → Only checks file exists
|
|
132
|
-
- ❌ `npm test` alone for backend → Mocks can pass without real behavior
|
|
133
|
-
- ✅ `curl ... | jq -e` → Tests actual API response
|
|
134
|
-
- ✅ `npx playwright test` → Real browser tests
|
|
135
|
-
- ✅ `npx tsc --noEmit` → Real type checking
|
|
136
|
-
|
|
137
|
-
#### 6b. Dependencies
|
|
138
|
-
- Can this story's tests pass given prior stories completed?
|
|
139
|
-
- If TASK-003 needs a user, does TASK-001/002 create one?
|
|
140
|
-
|
|
141
|
-
#### 6c. Security (for auth/input stories)
|
|
142
|
-
Does acceptanceCriteria include:
|
|
143
|
-
- Password handling → "Passwords hashed with bcrypt (cost 10+)"
|
|
144
|
-
- Auth responses → "Password/tokens NEVER in response body"
|
|
145
|
-
- User input → "Input sanitized to prevent SQL injection/XSS"
|
|
146
|
-
- Login endpoints → "Rate limited to N attempts per minute"
|
|
147
|
-
- Token expiry → "JWT expires after N hours"
|
|
148
|
-
|
|
149
|
-
#### 6d. Scale (for list/data stories)
|
|
150
|
-
Does acceptanceCriteria include:
|
|
151
|
-
- List endpoints → "Returns paginated results (max 100 per page)"
|
|
152
|
-
- Query params → "Accepts ?page=N&limit=N"
|
|
153
|
-
- Large datasets → "Database query uses index on [column]"
|
|
154
|
-
|
|
155
|
-
#### 6e. Context (for frontend stories)
|
|
156
|
-
- Does `contextFiles` include the idea file (has ASCII mockups)?
|
|
157
|
-
- Does `contextFiles` include styleguide (if exists)?
|
|
158
|
-
- Is `testUrl` set?
|
|
159
|
-
|
|
160
|
-
**Fix any issues you find:**
|
|
161
|
-
|
|
162
|
-
| Problem | Fix |
|
|
163
|
-
|---------|-----|
|
|
164
|
-
| testSteps use grep/test only | Replace with curl, playwright |
|
|
165
|
-
| Backend story has only `npm test` | Add curl commands that hit real endpoints |
|
|
166
|
-
| Story depends on something not created | Reorder or add missing dependency |
|
|
167
|
-
| Auth story missing security criteria | Add password hashing, rate limiting to acceptanceCriteria |
|
|
168
|
-
| List endpoint missing pagination | Add pagination criteria to acceptanceCriteria |
|
|
169
|
-
| Frontend missing contextFiles | Add idea file + styleguide paths |
|
|
170
|
-
| Frontend missing testUrl | Add URL from config |
|
|
171
|
-
|
|
172
|
-
### Step 7: Reorder if Needed
|
|
173
|
-
|
|
174
|
-
If validation found dependency issues, reorder stories:
|
|
175
|
-
|
|
176
|
-
1. Stories that create foundations (DB schemas, base components) come first
|
|
177
|
-
2. Stories that depend on others come after their dependencies
|
|
178
|
-
3. Update `dependsOn` arrays to reflect the order
|
|
179
|
-
4. Re-number story IDs if needed (TASK-001, TASK-002, etc.)
|
|
180
|
-
|
|
181
|
-
**After reordering, re-run Step 6 validation to confirm the new order works.**
|
|
182
|
-
|
|
183
|
-
### Step 8: Present Final PRD
|
|
184
|
-
|
|
185
|
-
Open the PRD for review:
|
|
186
|
-
```bash
|
|
187
|
-
open -a TextEdit .ralph/prd.json
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
Say: "I've {created|updated} the PRD with {N} stories and opened it in TextEdit.
|
|
191
|
-
|
|
192
|
-
Review the PRD and let me know:
|
|
193
|
-
- **'approved'** - Ready for `ralph run`
|
|
194
|
-
- **'edit [changes]'** - Tell me what to change
|
|
195
|
-
- Or edit the JSON directly and say **'done'**"
|
|
196
|
-
|
|
197
|
-
**STOP and wait for user response.**
|
|
198
|
-
|
|
199
|
-
### Step 9: Final Instructions
|
|
200
|
-
|
|
201
|
-
Once approved, say:
|
|
202
|
-
|
|
203
|
-
"PRD is ready!
|
|
204
|
-
|
|
205
|
-
**Source:** `{idea-file-path}`
|
|
206
|
-
**PRD:** `.ralph/prd.json` ({N} stories)
|
|
207
|
-
|
|
208
|
-
To start autonomous development:
|
|
209
|
-
```bash
|
|
210
|
-
ralph run
|
|
211
|
-
```
|
|
212
|
-
|
|
213
|
-
Ralph will work through each story, running tests and committing as it goes."
|
|
214
|
-
|
|
215
|
-
**DO NOT start implementing code.**
|
|
216
|
-
|
|
217
|
-
---
|
|
218
|
-
|
|
219
|
-
## Complete PRD JSON Schema
|
|
220
|
-
|
|
221
|
-
**Full working example:** See `templates/prd-example.json` for a complete, valid PRD.
|
|
222
|
-
|
|
223
|
-
```json
|
|
224
|
-
{
|
|
225
|
-
"feature": {
|
|
226
|
-
"name": "Feature Name",
|
|
227
|
-
"ideaFile": "docs/ideas/{feature-name}.md",
|
|
228
|
-
"branch": "feature/{feature-name}",
|
|
229
|
-
"status": "pending"
|
|
230
|
-
},
|
|
231
|
-
|
|
232
|
-
"originalContext": "docs/ideas/{feature-name}.md",
|
|
233
|
-
|
|
234
|
-
"techStack": {
|
|
235
|
-
"frontend": "{detected from package.json}",
|
|
236
|
-
"backend": "{detected from pyproject.toml/go.mod}",
|
|
237
|
-
"database": "{detected or asked}"
|
|
238
|
-
},
|
|
239
|
-
|
|
240
|
-
"testing": {
|
|
241
|
-
"approach": "TDD",
|
|
242
|
-
"unit": {
|
|
243
|
-
"frontend": "{vitest|jest - detected from package.json}",
|
|
244
|
-
"backend": "{pytest|go test - detected from project}"
|
|
245
|
-
},
|
|
246
|
-
"integration": "{playwright|cypress}",
|
|
247
|
-
"e2e": "{playwright|cypress}",
|
|
248
|
-
"coverage": {
|
|
249
|
-
"minimum": 80,
|
|
250
|
-
"enforced": false
|
|
251
|
-
}
|
|
252
|
-
},
|
|
253
|
-
|
|
254
|
-
"architecture": {
|
|
255
|
-
"frontend": "src/components",
|
|
256
|
-
"backend": "src/api",
|
|
257
|
-
"doNotCreate": ["new database tables without migration"]
|
|
258
|
-
},
|
|
259
|
-
|
|
260
|
-
"globalConstraints": [
|
|
261
|
-
"All API calls must have error handling",
|
|
262
|
-
"No console.log in production code",
|
|
263
|
-
"Use existing UI components from src/components/ui"
|
|
264
|
-
],
|
|
265
|
-
|
|
266
|
-
"testUsers": {
|
|
267
|
-
"admin": {"email": "admin@test.com", "password": "test123"},
|
|
268
|
-
"user": {"email": "user@test.com", "password": "test123"}
|
|
269
|
-
},
|
|
270
|
-
|
|
271
|
-
"metadata": {
|
|
272
|
-
"createdAt": "ISO timestamp",
|
|
273
|
-
"estimatedStories": 5,
|
|
274
|
-
"complexity": "low|medium|high"
|
|
275
|
-
},
|
|
276
|
-
|
|
277
|
-
"stories": [
|
|
278
|
-
{
|
|
279
|
-
"id": "TASK-001",
|
|
280
|
-
"type": "frontend|backend",
|
|
281
|
-
"title": "Short description",
|
|
282
|
-
"priority": 1,
|
|
283
|
-
"passes": false,
|
|
284
|
-
|
|
285
|
-
"files": {
|
|
286
|
-
"create": ["paths to new files"],
|
|
287
|
-
"modify": ["paths to existing files"],
|
|
288
|
-
"reuse": ["existing files to import from"]
|
|
289
|
-
},
|
|
290
|
-
|
|
291
|
-
"acceptanceCriteria": [
|
|
292
|
-
"What it should do"
|
|
293
|
-
],
|
|
294
|
-
|
|
295
|
-
"errorHandling": [
|
|
296
|
-
"What happens when things fail"
|
|
297
|
-
],
|
|
298
|
-
|
|
299
|
-
"testing": {
|
|
300
|
-
"types": ["unit", "integration"],
|
|
301
|
-
"approach": "TDD",
|
|
302
|
-
"files": {
|
|
303
|
-
"unit": ["src/components/Dashboard.test.tsx"],
|
|
304
|
-
"integration": ["tests/integration/dashboard.test.ts"],
|
|
305
|
-
"e2e": ["tests/e2e/dashboard.spec.ts"]
|
|
306
|
-
}
|
|
307
|
-
},
|
|
308
|
-
|
|
309
|
-
"testSteps": [
|
|
310
|
-
"curl -s {config.urls.backend}/endpoint | jq -e '.expected == true'",
|
|
311
|
-
"npx playwright test tests/e2e/feature.spec.ts"
|
|
312
|
-
],
|
|
313
|
-
|
|
314
|
-
"testUrl": "{config.urls.frontend}/feature-page",
|
|
315
|
-
|
|
316
|
-
"mcp": ["playwright", "devtools"],
|
|
317
|
-
|
|
318
|
-
"contextFiles": [
|
|
319
|
-
"docs/ideas/feature.md",
|
|
320
|
-
"src/styles/styleguide.html"
|
|
321
|
-
],
|
|
322
|
-
|
|
323
|
-
"skills": [
|
|
324
|
-
{"name": "styleguide", "usage": "Reference for UI components"},
|
|
325
|
-
{"name": "vibe-check", "usage": "Run after implementation"}
|
|
326
|
-
],
|
|
327
|
-
|
|
328
|
-
"apiContract": {
|
|
329
|
-
"endpoint": "GET /api/resource",
|
|
330
|
-
"response": {"field": "type"}
|
|
331
|
-
},
|
|
332
|
-
|
|
333
|
-
"prerequisites": [
|
|
334
|
-
"Backend server running",
|
|
335
|
-
"Database seeded"
|
|
336
|
-
],
|
|
337
|
-
|
|
338
|
-
"notes": "Human guidance - preferences, warnings, tips",
|
|
339
|
-
|
|
340
|
-
"scale": "small|medium|large",
|
|
341
|
-
|
|
342
|
-
"architecture": {
|
|
343
|
-
"pattern": "React Query for data fetching",
|
|
344
|
-
"constraints": ["No Redux"]
|
|
345
|
-
},
|
|
346
|
-
|
|
347
|
-
"dependsOn": []
|
|
348
|
-
}
|
|
349
|
-
]
|
|
350
|
-
}
|
|
351
|
-
```
|
|
352
|
-
|
|
353
|
-
---
|
|
354
|
-
|
|
355
|
-
## Field Reference
|
|
356
|
-
|
|
357
|
-
### PRD-Level Fields
|
|
358
|
-
|
|
359
|
-
| Field | Required | Description |
|
|
360
|
-
|-------|----------|-------------|
|
|
361
|
-
| `feature` | Yes | Feature name, branch, status |
|
|
362
|
-
| `originalContext` | Yes | Path to idea file (Claude reads this for full context) |
|
|
363
|
-
| `techStack` | No | Technologies in use (auto-detect from project) |
|
|
364
|
-
| `testing` | Yes | Testing strategy, tools, coverage requirements |
|
|
365
|
-
| `architecture` | No | Directory structure, patterns, constraints |
|
|
366
|
-
| `globalConstraints` | No | Rules that apply to ALL stories |
|
|
367
|
-
| `testUsers` | No | Test accounts for auth flows |
|
|
368
|
-
| `metadata` | Yes | Created date, complexity estimate |
|
|
369
|
-
|
|
370
|
-
**Note:** URLs come from `.ralph/config.json`, not the PRD. Use `{config.urls.backend}` in testSteps.
|
|
371
|
-
|
|
372
|
-
### Story-Level Fields
|
|
373
|
-
|
|
374
|
-
| Field | Required | Description |
|
|
375
|
-
|-------|----------|-------------|
|
|
376
|
-
| `id` | Yes | Unique ID (TASK-001, TASK-002, etc.) |
|
|
377
|
-
| `type` | Yes | frontend or backend (keep stories atomic) |
|
|
378
|
-
| `title` | Yes | Short description |
|
|
379
|
-
| `priority` | No | Order of importance (1 = highest) |
|
|
380
|
-
| `passes` | Yes | Always starts as `false` |
|
|
381
|
-
| `files` | Yes | create, modify, reuse arrays |
|
|
382
|
-
| `acceptanceCriteria` | Yes | What must be true when done |
|
|
383
|
-
| `errorHandling` | Yes | How to handle failures |
|
|
384
|
-
| `testing` | Yes | Test types, approach, and files for this story |
|
|
385
|
-
| `testSteps` | Yes | Executable shell commands |
|
|
386
|
-
| `testUrl` | Frontend | URL to verify the feature |
|
|
387
|
-
| `mcp` | Frontend | MCP tools for verification |
|
|
388
|
-
| `contextFiles` | No | Files Claude should read (idea files, styleguides) |
|
|
389
|
-
| `skills` | No | Relevant skills with usage hints |
|
|
390
|
-
| `apiContract` | Backend | Expected request/response format |
|
|
391
|
-
| `prerequisites` | No | What must be running/ready |
|
|
392
|
-
| `notes` | No | Human guidance for Claude |
|
|
393
|
-
| `scale` | No | small, medium, large |
|
|
394
|
-
| `architecture` | No | Story-specific patterns/constraints |
|
|
395
|
-
| `dependsOn` | No | Story IDs that must complete first |
|
|
396
|
-
|
|
397
|
-
---
|
|
398
|
-
|
|
399
|
-
## Testing Strategy
|
|
400
|
-
|
|
401
|
-
### PRD-Level Testing Config
|
|
402
|
-
|
|
403
|
-
Define the overall testing strategy for the feature. **Auto-detect tools from project config files:**
|
|
404
|
-
|
|
405
|
-
```json
|
|
406
|
-
"testing": {
|
|
407
|
-
"approach": "TDD",
|
|
408
|
-
"unit": {
|
|
409
|
-
"frontend": "vitest",
|
|
410
|
-
"backend": "pytest"
|
|
411
|
-
},
|
|
412
|
-
"integration": "playwright",
|
|
413
|
-
"e2e": "playwright",
|
|
414
|
-
"coverage": {
|
|
415
|
-
"minimum": 80,
|
|
416
|
-
"enforced": false
|
|
417
|
-
}
|
|
418
|
-
}
|
|
419
|
-
```
|
|
420
|
-
|
|
421
|
-
**Detection hints:**
|
|
422
|
-
- Check `package.json` for `vitest`, `jest`, `playwright`, `cypress`
|
|
423
|
-
- Check `pyproject.toml` for `pytest`
|
|
424
|
-
- Check `go.mod` for Go projects (use `go test`)
|
|
425
|
-
|
|
426
|
-
| Field | Values | Description |
|
|
427
|
-
|-------|--------|-------------|
|
|
428
|
-
| `approach` | `TDD`, `test-after` | Write tests first (TDD) or after implementation |
|
|
429
|
-
| `unit.frontend` | `vitest`, `jest` | Frontend unit test runner (detect from package.json) |
|
|
430
|
-
| `unit.backend` | `pytest`, `go test` | Backend unit test runner (detect from project) |
|
|
431
|
-
| `integration` | `playwright`, `cypress` | Integration test tool |
|
|
432
|
-
| `e2e` | `playwright`, `cypress` | End-to-end test tool |
|
|
433
|
-
| `coverage.minimum` | `0-100` | Minimum coverage percentage |
|
|
434
|
-
| `coverage.enforced` | `true/false` | Fail if coverage not met |
|
|
435
|
-
|
|
436
|
-
### Story-Level Testing Config
|
|
437
|
-
|
|
438
|
-
Specify what tests each story needs:
|
|
439
|
-
|
|
440
|
-
```json
|
|
441
|
-
"testing": {
|
|
442
|
-
"types": ["unit", "integration"],
|
|
443
|
-
"approach": "TDD",
|
|
444
|
-
"files": {
|
|
445
|
-
"unit": ["src/components/Dashboard.test.tsx"],
|
|
446
|
-
"integration": ["tests/integration/dashboard.test.ts"],
|
|
447
|
-
"e2e": ["tests/e2e/dashboard.spec.ts"]
|
|
448
|
-
}
|
|
449
|
-
}
|
|
450
|
-
```
|
|
451
|
-
|
|
452
|
-
| Field | Description |
|
|
453
|
-
|-------|-------------|
|
|
454
|
-
| `types` | Required test types: `unit`, `integration`, `e2e` |
|
|
455
|
-
| `approach` | Override PRD-level approach for this story |
|
|
456
|
-
| `files.unit` | Unit test files to create |
|
|
457
|
-
| `files.integration` | Integration test files to create |
|
|
458
|
-
| `files.e2e` | E2E test files to create |
|
|
459
|
-
|
|
460
|
-
### Test Types
|
|
461
|
-
|
|
462
|
-
| Type | What it Tests | When to Use |
|
|
463
|
-
|------|---------------|-------------|
|
|
464
|
-
| **Unit** | Individual functions, components in isolation | Always - every new file needs unit tests |
|
|
465
|
-
| **Integration** | How pieces work together (API + DB, Component + Hook) | When story involves multiple modules |
|
|
466
|
-
| **E2E** | Full user flows in browser | User-facing features with interactions |
|
|
467
|
-
|
|
468
|
-
### TDD Workflow
|
|
469
|
-
|
|
470
|
-
When `approach: "TDD"`:
|
|
471
|
-
|
|
472
|
-
1. **Write failing test first** - Define expected behavior
|
|
473
|
-
2. **Implement minimum code** - Make the test pass
|
|
474
|
-
3. **Refactor** - Clean up while tests stay green
|
|
475
|
-
4. **Repeat** - Next acceptance criterion
|
|
476
|
-
|
|
477
|
-
Example for a Dashboard component:
|
|
478
|
-
```
|
|
479
|
-
1. Write test: "renders user name in header"
|
|
480
|
-
2. Run test → FAIL (component doesn't exist)
|
|
481
|
-
3. Create Dashboard.tsx with user name
|
|
482
|
-
4. Run test → PASS
|
|
483
|
-
5. Write test: "shows loading state"
|
|
484
|
-
6. Run test → FAIL
|
|
485
|
-
7. Add loading state
|
|
486
|
-
8. Run test → PASS
|
|
487
|
-
```
|
|
488
|
-
|
|
489
|
-
### Testing Anti-Patterns (AVOID THESE)
|
|
490
|
-
|
|
491
|
-
**Missing integration points:**
|
|
492
|
-
```json
|
|
493
|
-
// ❌ BAD - creates function but doesn't verify callers use it
|
|
494
|
-
{
|
|
495
|
-
"files": {"modify": ["graph.py"]},
|
|
496
|
-
"acceptanceCriteria": ["Create stream_agent function"]
|
|
497
|
-
}
|
|
498
|
-
|
|
499
|
-
// ✅ GOOD - verifies the full chain
|
|
500
|
-
{
|
|
501
|
-
"files": {"modify": ["graph.py", "service.py"]},
|
|
502
|
-
"acceptanceCriteria": [
|
|
503
|
-
"service.py calls stream_agent() (not run_agent)",
|
|
504
|
-
"POST /chat returns progress SSE events"
|
|
505
|
-
]
|
|
506
|
-
}
|
|
507
|
-
```
|
|
508
|
-
|
|
509
|
-
**(See "The Grep for Code Trap" section above for the #1 anti-pattern)**
|
|
510
|
-
|
|
511
|
-
### Removing/Modifying UI - Update Tests!
|
|
512
|
-
|
|
513
|
-
**CRITICAL: When a story removes or modifies UI elements, it MUST update related tests.**
|
|
514
|
-
|
|
515
|
-
Stories that remove UI must include:
|
|
516
|
-
```json
|
|
517
|
-
{
|
|
518
|
-
"files": {
|
|
519
|
-
"modify": ["src/components/Dashboard.tsx"],
|
|
520
|
-
"delete": ["src/components/SelectionPanel.tsx"]
|
|
521
|
-
},
|
|
522
|
-
"acceptanceCriteria": [
|
|
523
|
-
"Selection panel removed from dashboard",
|
|
524
|
-
"All tests referencing 'Auto-select' button updated or removed"
|
|
525
|
-
],
|
|
526
|
-
"testSteps": [
|
|
527
|
-
"grep -r 'Auto-select' tests/ && exit 1 || echo 'No stale test references'",
|
|
528
|
-
"npx playwright test tests/e2e/dashboard.spec.ts"
|
|
529
|
-
]
|
|
530
|
-
}
|
|
531
|
-
```
|
|
532
|
-
|
|
533
|
-
The `grep ... && exit 1` pattern ensures the story fails if stale test references exist.
|
|
534
|
-
|
|
535
|
-
### Acceptance Criteria Rules
|
|
536
|
-
|
|
537
|
-
1. **Behavior over implementation** - Describe what the user/API sees, not what code exists
|
|
538
|
-
2. **Verifiable** - Each criterion must be testable with a curl, pytest, or playwright
|
|
539
|
-
3. **Include callers** - If adding a new function, verify callers use it
|
|
540
|
-
4. **Update tests** - If removing UI, verify no tests reference removed elements
|
|
541
|
-
|
|
542
|
-
```
|
|
543
|
-
❌ "Use astream_events() for progress"
|
|
544
|
-
✅ "POST /chat streams progress events before final response"
|
|
545
|
-
|
|
546
|
-
❌ "Create stream_agent function"
|
|
547
|
-
✅ "service.py send_message_stream() calls stream_agent()"
|
|
548
|
-
```
|
|
549
|
-
|
|
550
|
-
### Integration Test Requirements
|
|
551
|
-
|
|
552
|
-
Backend stories that modify internal functions MUST have integration tests that verify the API behavior:
|
|
553
|
-
|
|
554
|
-
```python
|
|
555
|
-
# ✅ GOOD - tests actual API behavior
|
|
556
|
-
async def test_send_message_streams_progress_events():
|
|
557
|
-
"""Verify the API actually streams progress events."""
|
|
558
|
-
async with client.stream("POST", f"/chat/{conv_id}/messages",
|
|
559
|
-
json={"content": "test"}) as response:
|
|
560
|
-
events = [e async for e in parse_sse(response)]
|
|
561
|
-
progress_events = [e for e in events if e["event_type"] == "progress"]
|
|
562
|
-
assert len(progress_events) > 0, "No progress events streamed"
|
|
563
|
-
```
|
|
564
|
-
|
|
565
|
-
### Example Stories by Type
|
|
566
|
-
|
|
567
|
-
**Frontend story:**
|
|
568
|
-
```json
|
|
569
|
-
"testing": {
|
|
570
|
-
"types": ["unit", "e2e"],
|
|
571
|
-
"approach": "TDD",
|
|
572
|
-
"files": {
|
|
573
|
-
"unit": ["src/components/Dashboard.test.tsx"],
|
|
574
|
-
"e2e": ["tests/e2e/dashboard.spec.ts"]
|
|
575
|
-
}
|
|
576
|
-
}
|
|
577
|
-
```
|
|
578
|
-
|
|
579
|
-
**Backend API story:**
|
|
580
|
-
```json
|
|
581
|
-
"testing": {
|
|
582
|
-
"types": ["unit", "integration"],
|
|
583
|
-
"approach": "TDD",
|
|
584
|
-
"files": {
|
|
585
|
-
"unit": ["tests/unit/test_stream_agent.py"],
|
|
586
|
-
"integration": ["tests/integration/test_chat_streaming.py"]
|
|
587
|
-
}
|
|
588
|
-
},
|
|
589
|
-
"acceptanceCriteria": [
|
|
590
|
-
"service.py calls stream_agent() instead of run_agent()",
|
|
591
|
-
"POST /chat/messages returns SSE stream with progress events",
|
|
592
|
-
"Progress events include tool name and status"
|
|
593
|
-
],
|
|
594
|
-
"testSteps": [
|
|
595
|
-
"pytest tests/integration/test_chat_streaming.py -v",
|
|
596
|
-
"curl -N {config.urls.backend}/chat/1/messages -d '{\"content\":\"test\"}' | grep -q 'progress'"
|
|
597
|
-
]
|
|
598
|
-
```
|
|
599
|
-
|
|
600
|
-
---
|
|
601
|
-
|
|
602
|
-
## MCP Tools
|
|
603
|
-
|
|
604
|
-
Specify which MCP tools Claude should use for verification:
|
|
605
|
-
|
|
606
|
-
| Tool | When to Use |
|
|
607
|
-
|------|-------------|
|
|
608
|
-
| `playwright` | UI testing, screenshots, form interactions, a11y |
|
|
609
|
-
| `devtools` | Console errors, network inspection, DOM debugging |
|
|
610
|
-
| `postgres` | Database verification (future) |
|
|
611
|
-
|
|
612
|
-
**Frontend stories** default to `["playwright", "devtools"]`.
|
|
613
|
-
**Backend-only stories** can use `[]` or omit.
|
|
614
|
-
|
|
615
|
-
---
|
|
616
|
-
|
|
617
|
-
## Skills Reference
|
|
618
|
-
|
|
619
|
-
Point Claude to relevant skills for guidance:
|
|
620
|
-
|
|
621
|
-
| Skill | When to Use |
|
|
622
|
-
|-------|-------------|
|
|
623
|
-
| `styleguide` | Frontend stories - reference UI components |
|
|
624
|
-
| `vibe-check` | Any story - check for AI anti-patterns after |
|
|
625
|
-
| `review` | Security-sensitive stories - OWASP checks |
|
|
626
|
-
| `explain` | Complex logic - document decisions |
|
|
627
|
-
|
|
628
|
-
Example:
|
|
629
|
-
```json
|
|
630
|
-
"skills": [
|
|
631
|
-
{"name": "styleguide", "usage": "Use existing Card, Button components"},
|
|
632
|
-
{"name": "vibe-check", "usage": "Run after implementation to catch issues"}
|
|
633
|
-
]
|
|
634
|
-
```
|
|
635
|
-
|
|
636
|
-
---
|
|
637
|
-
|
|
638
|
-
## Test Steps - CRITICAL
|
|
639
|
-
|
|
640
|
-
⚠️ **THE #1 CAUSE OF FALSE PASSES: grep-only test steps that verify code exists but not behavior.**
|
|
641
|
-
|
|
642
|
-
**Test steps MUST be executable shell commands.** Ralph runs them with bash.
|
|
643
|
-
|
|
644
|
-
### The "Grep for Code" Trap - NEVER DO THIS
|
|
645
|
-
|
|
646
|
-
```json
|
|
647
|
-
// ❌ BAD - This will PASS even when the feature is completely broken!
|
|
648
|
-
"testSteps": [
|
|
649
|
-
"grep -q 'astream_events' app/domains/chat/agent/graph.py",
|
|
650
|
-
"grep -q 'export function' src/api/users.ts"
|
|
651
|
-
]
|
|
652
|
-
|
|
653
|
-
// ✅ GOOD - This actually tests if the feature works
|
|
654
|
-
"testSteps": [
|
|
655
|
-
"curl -N {config.urls.backend}/chat -d '{\"message\":\"test\"}' | grep -q 'progress'",
|
|
656
|
-
"curl -s {config.urls.backend}/users | jq -e '.data | length >= 0'"
|
|
657
|
-
]
|
|
658
|
-
```
|
|
659
|
-
|
|
660
|
-
**Why is grep bad?** Ralph runs `grep -q 'function' file.py` → returns 0 → marks story as PASSED. But the function could be completely broken, have wrong parameters, or never get called. The test passed but the feature doesn't work.
|
|
661
|
-
|
|
662
|
-
### Backend Stories MUST Have Curl Tests
|
|
663
|
-
|
|
664
|
-
**CRITICAL: Every backend story MUST include curl commands that verify actual API behavior.**
|
|
665
|
-
|
|
666
|
-
Use `{config.urls.backend}` - Ralph expands this from `.ralph/config.json`:
|
|
667
|
-
|
|
668
|
-
```json
|
|
669
|
-
// ✅ REQUIRED for backend stories
|
|
670
|
-
"testSteps": [
|
|
671
|
-
"curl -s {config.urls.backend}/users | jq -e '.data | length > 0'",
|
|
672
|
-
"curl -s -X POST {config.urls.backend}/users -d '{\"email\":\"test@test.com\"}' | jq -e '.id'",
|
|
673
|
-
"curl -N {config.urls.backend}/chat/1/messages -d '{\"content\":\"test\"}' | grep -q 'progress'"
|
|
674
|
-
]
|
|
675
|
-
```
|
|
676
|
-
|
|
677
|
-
Ralph reads `.ralph/config.json` and expands `{config.urls.backend}` before running.
|
|
678
|
-
|
|
679
|
-
**Why?** Grep tests verify code exists. Curl tests verify the feature works. (See "The Grep for Code Trap" above.)
|
|
680
|
-
|
|
681
|
-
### Test Steps by Story Type
|
|
682
|
-
|
|
683
|
-
| Story Type | Required testSteps |
|
|
684
|
-
|------------|-------------------|
|
|
685
|
-
| `backend` | curl commands using `{config.urls.backend}` to verify API behavior |
|
|
686
|
-
| `frontend` | `tsc --noEmit` (type errors) + `npm test` (unit) + playwright (e2e) |
|
|
687
|
-
| `e2e` | playwright test commands |
|
|
688
|
-
|
|
689
|
-
**Frontend stories MUST include TypeScript check** - curl won't catch type errors:
|
|
690
|
-
```json
|
|
691
|
-
// ✅ Frontend story testSteps
|
|
692
|
-
"testSteps": [
|
|
693
|
-
"npx tsc --noEmit",
|
|
694
|
-
"npm test -- --testPathPattern=Dashboard",
|
|
695
|
-
"npx playwright test tests/e2e/dashboard.spec.ts"
|
|
696
|
-
]
|
|
697
|
-
```
|
|
698
|
-
|
|
699
|
-
### Good Test Steps (executable)
|
|
700
|
-
```json
|
|
701
|
-
// Backend story - use {config.urls.backend}
|
|
702
|
-
"testSteps": [
|
|
703
|
-
"curl -s {config.urls.backend}/health | jq -e '.status == \"ok\"'",
|
|
704
|
-
"curl -s -X POST {config.urls.backend}/users -H 'Content-Type: application/json' -d '{\"email\":\"test@example.com\"}' | jq -e '.id'",
|
|
705
|
-
"pytest tests/integration/test_users.py -v"
|
|
706
|
-
]
|
|
707
|
-
|
|
708
|
-
// Frontend story
|
|
709
|
-
"testSteps": [
|
|
710
|
-
"npm test -- --testPathPattern=Button.test.tsx",
|
|
711
|
-
"npx tsc --noEmit"
|
|
712
|
-
]
|
|
713
|
-
|
|
714
|
-
// E2E story
|
|
715
|
-
"testSteps": [
|
|
716
|
-
"npx playwright test tests/e2e/user-signup.spec.ts"
|
|
717
|
-
]
|
|
718
|
-
```
|
|
719
|
-
|
|
720
|
-
### Bad Test Steps (will PASS but miss bugs)
|
|
721
|
-
```json
|
|
722
|
-
"testSteps": [
|
|
723
|
-
"grep -q 'function createUser' app/services/user.py", // ❌ PASSES if code exists, even if broken
|
|
724
|
-
"grep -q 'export default' src/components/Dashboard.tsx", // ❌ PASSES even if component crashes
|
|
725
|
-
"test -f src/api/users.ts", // ❌ PASSES if file exists, even if empty
|
|
726
|
-
"Visit http://localhost:3000/dashboard", // ❌ Not executable
|
|
727
|
-
"User can see the dashboard" // ❌ Not executable
|
|
728
|
-
]
|
|
729
|
-
```
|
|
730
|
-
|
|
731
|
-
**NEVER use grep/test to verify behavior.** These will mark stories as PASSED when the feature is broken.
|
|
732
|
-
|
|
733
|
-
**If a step can't be automated**, put it in `acceptanceCriteria` instead. Claude will verify it visually using MCP tools.
|
|
734
|
-
|
|
735
|
-
---
|
|
736
|
-
|
|
737
|
-
## Context Files
|
|
738
|
-
|
|
739
|
-
Use `contextFiles` to point Claude to important reference material:
|
|
740
|
-
|
|
741
|
-
```json
|
|
742
|
-
"contextFiles": [
|
|
743
|
-
"docs/ideas/dashboard.md",
|
|
744
|
-
"src/styles/styleguide.html",
|
|
745
|
-
"docs/api-spec.md"
|
|
746
|
-
]
|
|
747
|
-
```
|
|
748
|
-
|
|
749
|
-
This is where ASCII mockups, design specs, and detailed requirements live. Claude reads these during the Orient step.
|
|
750
|
-
|
|
751
|
-
---
|
|
752
|
-
|
|
753
|
-
## Guidelines
|
|
754
|
-
|
|
755
|
-
- **Keep stories small** - Max 3-4 acceptance criteria (~1000 tokens)
|
|
756
|
-
- **Order by dependency** - Foundation stories first
|
|
757
|
-
- **Specify files explicitly** - Max 3-4 files per story
|
|
758
|
-
- **Define error handling** - Every story specifies failure behavior
|
|
759
|
-
- **Include contextFiles** - Point to idea files with full context (ASCII art, mockups)
|
|
760
|
-
- **Add relevant skills** - Help Claude find the right patterns
|
|
761
|
-
|
|
762
|
-
### UI Stories Must Include
|
|
763
|
-
- `testUrl` - Where to verify
|
|
764
|
-
- `mcp: ["playwright", "devtools"]` - Browser tools
|
|
765
|
-
- Acceptance criteria for: page loads, elements render, mobile works
|
|
766
|
-
|
|
767
|
-
### API Stories Must Include
|
|
768
|
-
- `apiContract` - Expected request/response
|
|
769
|
-
- `errorHandling` - What happens on 400, 401, 500, etc.
|
|
770
|
-
- `testSteps` with curl commands to verify endpoints
|