@haystackeditor/cli 0.7.0 → 0.7.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/utils/skill.js +72 -4
- package/package.json +1 -1
package/dist/utils/skill.js
CHANGED
|
@@ -50,7 +50,75 @@ Your flows should describe THIS journey, not just "pages load".
|
|
|
50
50
|
|
|
51
51
|
---
|
|
52
52
|
|
|
53
|
-
## Step 3:
|
|
53
|
+
## Step 3: Verify ALL Services Have Flows (CRITICAL)
|
|
54
|
+
|
|
55
|
+
**Every service in \`.haystack.json\` must have at least one verification flow.** This is the most common mistake - adding services but forgetting to add flows that exercise them.
|
|
56
|
+
|
|
57
|
+
### Check for Uncovered Services
|
|
58
|
+
|
|
59
|
+
\`\`\`bash
|
|
60
|
+
# List all services
|
|
61
|
+
grep -A1 '"services"' .haystack.json | grep -E '^\\s+"[^"]+":' | sed 's/.*"\\([^"]*\\)".*/\\1/'
|
|
62
|
+
|
|
63
|
+
# List all flows
|
|
64
|
+
grep '"name":' .haystack.json
|
|
65
|
+
|
|
66
|
+
# MANUAL CHECK: Does every service have a flow?
|
|
67
|
+
\`\`\`
|
|
68
|
+
|
|
69
|
+
### Types of Services
|
|
70
|
+
|
|
71
|
+
| Service Type | How to Verify |
|
|
72
|
+
|--------------|---------------|
|
|
73
|
+
| **Web frontend** | UI flows (navigate, wait_for, screenshot) |
|
|
74
|
+
| **API/Worker** | UI flows that hit API endpoints |
|
|
75
|
+
| **CLI/Batch** | Backend flows with golden inputs |
|
|
76
|
+
| **Analysis pipeline** | Backend flows with known PR input |
|
|
77
|
+
|
|
78
|
+
### Backend Verification (CLI tools, analysis pipelines, batch jobs)
|
|
79
|
+
|
|
80
|
+
**Not everything is a web page.** For CLI tools and batch processes, you need:
|
|
81
|
+
|
|
82
|
+
1. **Golden input** - A known-good input that produces predictable output
|
|
83
|
+
2. **Run command** - How to execute the service
|
|
84
|
+
3. **Output assertion** - What to check in the output
|
|
85
|
+
|
|
86
|
+
\`\`\`json
|
|
87
|
+
{
|
|
88
|
+
"name": "Backend pipeline - golden input",
|
|
89
|
+
"description": "Run pipeline on known input to verify it works",
|
|
90
|
+
"trigger": "on_change",
|
|
91
|
+
"watch_patterns": ["packages/my-pipeline/src/**"],
|
|
92
|
+
"service": "my-pipeline",
|
|
93
|
+
"type": "backend",
|
|
94
|
+
"steps": [
|
|
95
|
+
{
|
|
96
|
+
"action": "run",
|
|
97
|
+
"command": "pnpm start",
|
|
98
|
+
"env": { "INPUT_ID": "known-good-input-123" },
|
|
99
|
+
"timeout": 120
|
|
100
|
+
},
|
|
101
|
+
{ "action": "assert_exit_code", "code": 0 },
|
|
102
|
+
{ "action": "assert_output_contains", "pattern": "Processing complete" }
|
|
103
|
+
]
|
|
104
|
+
}
|
|
105
|
+
\`\`\`
|
|
106
|
+
|
|
107
|
+
### Finding Golden Inputs
|
|
108
|
+
|
|
109
|
+
For analysis/processing pipelines:
|
|
110
|
+
- Pick a **small, stable PR** that won't change
|
|
111
|
+
- Document what the expected output should be
|
|
112
|
+
- Add it as a comment in the flow description
|
|
113
|
+
|
|
114
|
+
\`\`\`bash
|
|
115
|
+
# Find a good golden PR (small, merged, stable)
|
|
116
|
+
gh pr list --state merged --limit 10 --json number,title,changedFiles | jq '.[] | select(.changedFiles < 5)'
|
|
117
|
+
\`\`\`
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## Step 4: Assess Data Needs
|
|
54
122
|
|
|
55
123
|
\`\`\`bash
|
|
56
124
|
# Find API calls
|
|
@@ -85,7 +153,7 @@ grep -r "https://" src/ --include="*.ts" --include="*.tsx" | grep -v node_module
|
|
|
85
153
|
|
|
86
154
|
---
|
|
87
155
|
|
|
88
|
-
## Step
|
|
156
|
+
## Step 5: Write Flows
|
|
89
157
|
|
|
90
158
|
Flows tell the Planner about your app's routes, UI elements, and user journeys.
|
|
91
159
|
|
|
@@ -157,7 +225,7 @@ For flows that only matter when certain files change:
|
|
|
157
225
|
|
|
158
226
|
---
|
|
159
227
|
|
|
160
|
-
## Step
|
|
228
|
+
## Step 6: Configure Fixtures (if needed)
|
|
161
229
|
|
|
162
230
|
Based on user's answer in Step 3:
|
|
163
231
|
|
|
@@ -189,7 +257,7 @@ fixtures:
|
|
|
189
257
|
|
|
190
258
|
---
|
|
191
259
|
|
|
192
|
-
## Step
|
|
260
|
+
## Step 7: Commit
|
|
193
261
|
|
|
194
262
|
\`\`\`bash
|
|
195
263
|
git add .haystack.json fixtures/
|