@pennyfarthing/core 7.7.0 → 7.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/package.json +1 -1
- package/packages/core/dist/cli/commands/doctor.d.ts +3 -0
- package/packages/core/dist/cli/commands/doctor.d.ts.map +1 -1
- package/packages/core/dist/cli/commands/doctor.js +134 -9
- package/packages/core/dist/cli/commands/doctor.js.map +1 -1
- package/pennyfarthing-dist/agents/sm-setup.md +37 -2
- package/pennyfarthing-dist/agents/sm.md +68 -22
- package/pennyfarthing-dist/agents/workflow-status-check.md +11 -1
- package/pennyfarthing-dist/commands/git-cleanup.md +43 -308
- package/pennyfarthing-dist/commands/solo.md +31 -0
- package/pennyfarthing-dist/guides/patterns/approval-gates-pattern.md +1 -1
- package/pennyfarthing-dist/personas/themes/gilligans-island.yaml +83 -83
- package/pennyfarthing-dist/personas/themes/the-expanse.yaml +11 -11
- package/pennyfarthing-dist/scripts/core/agent-session.sh +2 -2
- package/pennyfarthing-dist/scripts/core/check-context.sh +3 -0
- package/pennyfarthing-dist/scripts/core/handoff-marker.sh +13 -2
- package/pennyfarthing-dist/scripts/core/prime.sh +3 -157
- package/pennyfarthing-dist/scripts/core/run.sh +9 -0
- package/pennyfarthing-dist/scripts/hooks/__pycache__/question_reflector_check.cpython-314.pyc +0 -0
- package/pennyfarthing-dist/scripts/hooks/question_reflector_check.py +117 -20
- package/pennyfarthing-dist/scripts/jira/README.md +10 -7
- package/pennyfarthing-dist/scripts/misc/add-short-names.sh +13 -0
- package/pennyfarthing-dist/scripts/misc/add_short_names.py +226 -0
- package/pennyfarthing-dist/scripts/misc/migrate-bmad-workflow.sh +6 -5
- package/pennyfarthing-dist/scripts/misc/migrate_bmad_workflow.py +319 -0
- package/pennyfarthing-dist/scripts/sprint/import-epic-to-future.sh +6 -5
- package/pennyfarthing-dist/scripts/sprint/import_epic_to_future.py +270 -0
- package/pennyfarthing-dist/scripts/test/ensure-swebench-data.sh +59 -0
- package/pennyfarthing-dist/scripts/theme/compute-theme-tiers.sh +8 -6
- package/pennyfarthing-dist/scripts/theme/compute_theme_tiers.py +402 -0
- package/pennyfarthing-dist/scripts/workflow/check.sh +3 -476
- package/pennyfarthing-dist/scripts/workflow/get-workflow-type.py +61 -0
- package/pennyfarthing-dist/scripts/workflow/get-workflow-type.sh +13 -0
- package/pennyfarthing-dist/skills/judge/SKILL.md +57 -0
- package/pennyfarthing-dist/skills/sprint/scripts/sync-epic-jira.sh +4 -22
- package/pennyfarthing-dist/workflows/git-cleanup/steps/step-01-analyze.md +83 -0
- package/pennyfarthing-dist/workflows/git-cleanup/steps/step-02-categorize.md +116 -0
- package/pennyfarthing-dist/workflows/git-cleanup/steps/step-03-execute.md +210 -0
- package/pennyfarthing-dist/workflows/git-cleanup/steps/step-04-verify.md +88 -0
- package/pennyfarthing-dist/workflows/git-cleanup/steps/step-05-complete.md +71 -0
- package/pennyfarthing-dist/workflows/git-cleanup.yaml +59 -0
- package/pennyfarthing-dist/scripts/hooks/question-reflector-check.mjs +0 -393
- package/pennyfarthing-dist/scripts/hooks/tests/question-reflector.test.mjs +0 -545
- package/pennyfarthing-dist/scripts/jira/jira-bidirectional-sync.mjs +0 -327
- package/pennyfarthing-dist/scripts/jira/jira-bidirectional-sync.test.mjs +0 -503
- package/pennyfarthing-dist/scripts/jira/jira-lib.mjs +0 -443
- package/pennyfarthing-dist/scripts/jira/jira-sync-story.mjs +0 -208
- package/pennyfarthing-dist/scripts/jira/jira-sync.mjs +0 -198
- package/pennyfarthing-dist/scripts/misc/add-short-names.mjs +0 -264
- package/pennyfarthing-dist/scripts/misc/migrate-bmad-workflow.mjs +0 -474
- package/pennyfarthing-dist/scripts/sprint/import-epic-to-future.mjs +0 -377
- package/pennyfarthing-dist/scripts/theme/compute-theme-tiers.js +0 -492
- /package/pennyfarthing-dist/guides/{AGENT-COORDINATION.md → agent-coordination.md} +0 -0
- /package/pennyfarthing-dist/guides/{HOOKS.md → hooks.md} +0 -0
- /package/pennyfarthing-dist/guides/{PROMPT-PATTERNS.md → prompt-patterns.md} +0 -0
- /package/pennyfarthing-dist/guides/{SESSION-ARTIFACTS.md → session-artifacts.md} +0 -0
- /package/pennyfarthing-dist/guides/{XML-TAGS.md → xml-tags.md} +0 -0
|
@@ -1,340 +1,75 @@
|
|
|
1
1
|
---
|
|
2
2
|
description: Clean up git repos by organizing changes into proper commits/branches by initiative
|
|
3
|
+
workflow: git-cleanup
|
|
3
4
|
---
|
|
4
5
|
|
|
5
6
|
# Git Cleanup Command
|
|
6
7
|
|
|
7
8
|
Analyze and organize uncommitted changes across all repos into proper commits and branches based on initiative/feature groupings.
|
|
8
9
|
|
|
9
|
-
##
|
|
10
|
-
|
|
11
|
-
**CRITICAL: Never commit directly to develop. Branch protection hooks will reject direct commits.**
|
|
12
|
-
|
|
13
|
-
All changes MUST follow this workflow:
|
|
14
|
-
1. Create a branch from develop
|
|
15
|
-
2. Commit changes to the branch
|
|
16
|
-
3. Merge to develop locally
|
|
17
|
-
4. Push develop (branches don't need to be pushed)
|
|
18
|
-
|
|
19
|
-
## Analysis Phase
|
|
20
|
-
|
|
21
|
-
### 1. Gather Current State and Check for Unpushed Commits
|
|
22
|
-
|
|
23
|
-
```bash
|
|
24
|
-
./scripts/run.sh git/git-status-all.sh
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
This shows branch, changes, and unpushed commits for all repos.
|
|
28
|
-
|
|
29
|
-
### 2. Check Recent Commits for Patterns
|
|
30
|
-
|
|
31
|
-
```bash
|
|
32
|
-
echo "=== Recent Commit Patterns ==="
|
|
33
|
-
git log --oneline -10 | head -10
|
|
34
|
-
echo ""
|
|
35
|
-
echo "=== Recent Branches ==="
|
|
36
|
-
git branch --sort=-committerdate | head -10
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
### 3. Check Active Work Sessions
|
|
40
|
-
|
|
41
|
-
```bash
|
|
42
|
-
echo "=== Active Work ==="
|
|
43
|
-
if [ -f ".session/{STORY_ID}-session.md" ]; then
|
|
44
|
-
head -30 .session/{STORY_ID}-session.md
|
|
45
|
-
else
|
|
46
|
-
echo "No active work session"
|
|
47
|
-
fi
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
### 4. Check Worktree Status
|
|
51
|
-
|
|
52
|
-
```bash
|
|
53
|
-
echo "=== Worktree Status ==="
|
|
54
|
-
./scripts/run.sh git/worktree-manager.sh status
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
This shows all active worktrees with their branches and uncommitted changes.
|
|
58
|
-
|
|
59
|
-
### 5. Check Epics for Story Context
|
|
60
|
-
|
|
61
|
-
```bash
|
|
62
|
-
echo "=== Active Epics ==="
|
|
63
|
-
head -60 docs/epics.md
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
## Categorization Guidelines
|
|
67
|
-
|
|
68
|
-
Group changes by these initiative types (based on conventional commits):
|
|
10
|
+
## BikeLane Workflow
|
|
69
11
|
|
|
70
|
-
|
|
71
|
-
|--------|------|----------------|---------|
|
|
72
|
-
| `docs:` | Documentation | `docs/description` | docs/update-epics |
|
|
73
|
-
| `chore:` | Maintenance | `chore/description` | chore/sprint-cleanup |
|
|
74
|
-
| `chore(sprint):` | Sprint tracking | `chore/sprint-update` | - |
|
|
75
|
-
| `chore(pennyfarthing):` | Pennyfarthing config | `chore/pennyfarthing-cleanup` | - |
|
|
76
|
-
| `feat:` | New feature | `feat/story-id-desc` | feat/4-12-risk-scoring |
|
|
77
|
-
| `fix:` | Bug fix | `fix/issue-desc` | fix/validation-error |
|
|
78
|
-
| `refactor:` | Code improvement | `refactor/description` | refactor/seed-data |
|
|
12
|
+
This command uses the **git-cleanup** stepped workflow:
|
|
79
13
|
|
|
80
|
-
|
|
14
|
+
| Step | Name | Purpose |
|
|
15
|
+
|------|------|---------|
|
|
16
|
+
| 1 | Analyze | Gather git status across all repos |
|
|
17
|
+
| 2 | Categorize | Group changes by initiative type |
|
|
18
|
+
| 3 | Execute | Create branches, commit, merge |
|
|
19
|
+
| 4 | Verify | Confirm clean state, optionally push |
|
|
20
|
+
| 5 | Complete | Summary and next steps |
|
|
81
21
|
|
|
82
|
-
|
|
83
|
-
Files: `docs/*.md`, `sprint/*.yaml`, `sprint/*.md`
|
|
84
|
-
- Branch: `chore/sprint-cleanup` or `docs/update-[topic]`
|
|
85
|
-
- Commit: `chore(sprint): update sprint tracking` or `docs: update documentation`
|
|
22
|
+
## Quick Start
|
|
86
23
|
|
|
87
|
-
|
|
88
|
-
Files: `.claude/**/*`
|
|
89
|
-
- Branch: `chore/pennyfarthing-[description]`
|
|
90
|
-
- Commit: `chore(pennyfarthing): description`
|
|
91
|
-
|
|
92
|
-
### Feature Work
|
|
93
|
-
Files: `internal/**`, `src/**`, `migrations/**`
|
|
94
|
-
- Branch: `feat/[story-id]-description`
|
|
95
|
-
- Commit: `feat: description`
|
|
96
|
-
|
|
97
|
-
### Seed/Test Data
|
|
98
|
-
Files: `internal/seed/**`, `cmd/seed/**`
|
|
99
|
-
- Branch: `chore/seed-improvements` or part of feature branch
|
|
100
|
-
- Commit: `chore(seed): description`
|
|
101
|
-
|
|
102
|
-
### Bug Fixes
|
|
103
|
-
Files: Various
|
|
104
|
-
- Branch: `fix/[issue-description]`
|
|
105
|
-
- Commit: `fix: description`
|
|
106
|
-
|
|
107
|
-
## Execution Phase - Branch Workflow
|
|
108
|
-
|
|
109
|
-
**For each group of changes, follow this exact workflow:**
|
|
110
|
-
|
|
111
|
-
### Step 1: Stash All Changes First
|
|
112
|
-
```bash
|
|
113
|
-
# Stash everything to start clean
|
|
114
|
-
git stash push -m "cleanup-wip"
|
|
115
|
-
```
|
|
116
|
-
|
|
117
|
-
### Step 2: Create Branch from Develop
|
|
118
|
-
```bash
|
|
119
|
-
# Ensure on develop and up to date
|
|
120
|
-
git checkout develop
|
|
121
|
-
git pull origin develop
|
|
122
|
-
|
|
123
|
-
# Create feature branch
|
|
124
|
-
git checkout -b type/description
|
|
125
|
-
```
|
|
126
|
-
|
|
127
|
-
### Step 3: Apply Relevant Changes
|
|
128
|
-
```bash
|
|
129
|
-
# Pop stash
|
|
130
|
-
git stash pop
|
|
131
|
-
|
|
132
|
-
# Stage only files for this group
|
|
133
|
-
git add <specific-files>
|
|
134
|
-
|
|
135
|
-
# Stash remaining changes for next group
|
|
136
|
-
git stash push -m "remaining-cleanup"
|
|
137
|
-
```
|
|
138
|
-
|
|
139
|
-
### Step 4: Commit
|
|
140
|
-
```bash
|
|
141
|
-
# Commit with proper message
|
|
142
|
-
git commit -m "$(cat <<'EOF'
|
|
143
|
-
type(scope): description
|
|
144
|
-
|
|
145
|
-
Details if needed.
|
|
146
|
-
|
|
147
|
-
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
|
148
|
-
|
|
149
|
-
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
150
|
-
EOF
|
|
151
|
-
)"
|
|
152
|
-
```
|
|
153
|
-
|
|
154
|
-
### Step 5: Merge to Develop
|
|
155
|
-
```bash
|
|
156
|
-
# Switch to develop
|
|
157
|
-
git checkout develop
|
|
24
|
+
Run `/git-cleanup` to start the workflow.
|
|
158
25
|
|
|
159
|
-
|
|
160
|
-
git merge type/description
|
|
161
|
-
|
|
162
|
-
# Delete local branch
|
|
163
|
-
git branch -d type/description
|
|
164
|
-
```
|
|
165
|
-
|
|
166
|
-
### Step 6: Repeat for Next Group
|
|
167
|
-
```bash
|
|
168
|
-
# Pop remaining stash and repeat from Step 2
|
|
169
|
-
git stash pop
|
|
170
|
-
```
|
|
171
|
-
|
|
172
|
-
## Multi-Repo Cleanup Pattern
|
|
173
|
-
|
|
174
|
-
When cleaning up changes across multiple repos (configured in `.claude/project/repos.yaml`):
|
|
175
|
-
|
|
176
|
-
```bash
|
|
177
|
-
# Source repo utilities
|
|
178
|
-
source $CLAUDE_PROJECT_DIR/scripts/repo-utils.sh
|
|
179
|
-
|
|
180
|
-
# For each configured repo, run the same workflow
|
|
181
|
-
for_each_repo '
|
|
182
|
-
# ... branch workflow ...
|
|
183
|
-
git status --short
|
|
184
|
-
'
|
|
185
|
-
```
|
|
186
|
-
|
|
187
|
-
Or manually iterate:
|
|
188
|
-
|
|
189
|
-
```bash
|
|
190
|
-
source $CLAUDE_PROJECT_DIR/scripts/repo-utils.sh
|
|
191
|
-
for repo in $(get_repos); do
|
|
192
|
-
repo_path=$(get_repo_full_path "$repo")
|
|
193
|
-
echo "=== Processing $repo ==="
|
|
194
|
-
cd "$repo_path"
|
|
195
|
-
# ... branch workflow ...
|
|
196
|
-
cd -
|
|
197
|
-
done
|
|
198
|
-
```
|
|
199
|
-
|
|
200
|
-
## Interactive Cleanup Process
|
|
201
|
-
|
|
202
|
-
When running this command, Claude should:
|
|
26
|
+
## Git Workflow Rules
|
|
203
27
|
|
|
204
|
-
|
|
205
|
-
2. **Check for unpushed commits** - Identify any commits on develop not yet pushed
|
|
206
|
-
3. **Analyze changes** - Look for patterns in file paths, recent commits, and epics
|
|
207
|
-
4. **Propose groupings** - Suggest how to organize changes:
|
|
208
|
-
```
|
|
209
|
-
Group 1: Sprint Updates
|
|
210
|
-
- docs/epics.md
|
|
211
|
-
- sprint/current-sprint.yaml
|
|
212
|
-
→ Branch: chore/sprint-cleanup
|
|
213
|
-
→ Commit: "chore(sprint): clean up sprint tracking files"
|
|
28
|
+
**CRITICAL: Never commit directly to develop.** Branch protection hooks will reject direct commits.
|
|
214
29
|
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
30
|
+
All changes follow this pattern:
|
|
31
|
+
1. Create branch from develop
|
|
32
|
+
2. Commit changes to branch
|
|
33
|
+
3. Merge to develop locally
|
|
34
|
+
4. Push develop (branches stay local)
|
|
219
35
|
|
|
220
|
-
|
|
221
|
-
- internal/seed/*.go
|
|
222
|
-
→ Branch: chore/seed-improvements
|
|
223
|
-
→ Commit: "chore(seed): improve demo data generation"
|
|
224
|
-
```
|
|
225
|
-
4. **Ask for confirmation** - Let user approve or modify groupings
|
|
226
|
-
5. **Execute** - Create branches, commit, merge to develop for each group
|
|
227
|
-
6. **Push develop** - Push all merged changes at the end
|
|
228
|
-
7. **Verify** - Show clean git status at the end
|
|
36
|
+
## Categorization Reference
|
|
229
37
|
|
|
230
|
-
|
|
38
|
+
| Prefix | Type | Branch Pattern |
|
|
39
|
+
|--------|------|----------------|
|
|
40
|
+
| `docs:` | Documentation | `docs/description` |
|
|
41
|
+
| `chore:` | Maintenance | `chore/description` |
|
|
42
|
+
| `chore(sprint):` | Sprint tracking | `chore/sprint-update` |
|
|
43
|
+
| `feat:` | New feature | `feat/story-id-desc` |
|
|
44
|
+
| `fix:` | Bug fix | `fix/issue-desc` |
|
|
45
|
+
| `refactor:` | Code improvement | `refactor/description` |
|
|
46
|
+
| `test:` | Test changes | `test/description` |
|
|
231
47
|
|
|
232
|
-
|
|
233
|
-
- **Never force push**
|
|
234
|
-
- **Never commit secrets** (.env, credentials, etc.)
|
|
235
|
-
- **Show diff before committing**
|
|
236
|
-
- **Respect .gitignore**
|
|
237
|
-
- **Handle unpushed commits on develop before starting cleanup**
|
|
48
|
+
## Manual Quick Reference
|
|
238
49
|
|
|
239
|
-
|
|
50
|
+
For quick cleanup without the full workflow:
|
|
240
51
|
|
|
241
52
|
```bash
|
|
242
|
-
# View what needs cleanup
|
|
53
|
+
# View what needs cleanup
|
|
243
54
|
./scripts/run.sh git/git-status-all.sh
|
|
244
55
|
|
|
245
|
-
#
|
|
56
|
+
# Standard cleanup sequence
|
|
57
|
+
git stash push -m "cleanup-wip"
|
|
246
58
|
git checkout develop && git pull
|
|
247
59
|
git checkout -b chore/cleanup-$(date +%Y%m%d)
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
git
|
|
251
|
-
git add "sprint/*.yaml"
|
|
252
|
-
|
|
253
|
-
# Commit with proper message (use heredoc for multiline)
|
|
254
|
-
git commit -m "$(cat <<'EOF'
|
|
255
|
-
chore(sprint): update sprint status
|
|
256
|
-
|
|
257
|
-
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
|
258
|
-
|
|
259
|
-
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
260
|
-
EOF
|
|
261
|
-
)"
|
|
262
|
-
|
|
263
|
-
# Merge to develop (local)
|
|
60
|
+
git stash pop
|
|
61
|
+
git add <files>
|
|
62
|
+
git commit -m "chore: description"
|
|
264
63
|
git checkout develop
|
|
265
64
|
git merge chore/cleanup-$(date +%Y%m%d)
|
|
266
65
|
git branch -d chore/cleanup-$(date +%Y%m%d)
|
|
267
|
-
|
|
268
|
-
# Push develop after all merges complete
|
|
269
66
|
git push origin develop
|
|
270
67
|
```
|
|
271
68
|
|
|
272
|
-
##
|
|
273
|
-
|
|
274
|
-
Stale feature branches can accumulate over time. Include branch cleanup as part of git hygiene.
|
|
275
|
-
|
|
276
|
-
### Check for Stale Branches
|
|
277
|
-
|
|
278
|
-
```bash
|
|
279
|
-
echo "=== Branch Status ==="
|
|
280
|
-
|
|
281
|
-
./scripts/run.sh misc/check-status.sh
|
|
282
|
-
```
|
|
283
|
-
|
|
284
|
-
### Remove Merged Branches
|
|
285
|
-
|
|
286
|
-
**Only remove branches that are fully merged:**
|
|
287
|
-
|
|
288
|
-
```bash
|
|
289
|
-
# Source repo utilities
|
|
290
|
-
source $CLAUDE_PROJECT_DIR/scripts/repo-utils.sh
|
|
291
|
-
|
|
292
|
-
# For each configured repo
|
|
293
|
-
for repo in $(get_repos); do
|
|
294
|
-
repo_path=$(get_repo_full_path "$repo")
|
|
295
|
-
echo "=== $repo ==="
|
|
296
|
-
cd "$repo_path"
|
|
297
|
-
|
|
298
|
-
git checkout develop
|
|
299
|
-
git pull origin develop
|
|
300
|
-
|
|
301
|
-
# List branches merged into develop
|
|
302
|
-
git branch --merged develop | grep -v "develop\|main"
|
|
303
|
-
|
|
304
|
-
# Delete merged branches
|
|
305
|
-
git branch --merged develop | grep -v "develop\|main" | xargs -r git branch -d
|
|
306
|
-
|
|
307
|
-
cd -
|
|
308
|
-
done
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
### Branch Cleanup Criteria
|
|
312
|
-
|
|
313
|
-
| Branch Type | When to Remove |
|
|
314
|
-
|-------------|----------------|
|
|
315
|
-
| `feat/*` | After PR merged to develop |
|
|
316
|
-
| `fix/*` | After PR merged to develop |
|
|
317
|
-
| `chore/*` | After PR merged to develop |
|
|
318
|
-
|
|
319
|
-
**Note:** Never delete `main` or `develop` branches.
|
|
320
|
-
|
|
321
|
-
## Post-Cleanup Verification
|
|
322
|
-
|
|
323
|
-
```bash
|
|
324
|
-
echo "=== Final State ==="
|
|
325
|
-
./scripts/run.sh git/git-status-all.sh
|
|
326
|
-
echo ""
|
|
327
|
-
echo "=== Branches ==="
|
|
328
|
-
|
|
329
|
-
# Source repo utilities
|
|
330
|
-
source $CLAUDE_PROJECT_DIR/scripts/repo-utils.sh
|
|
331
|
-
|
|
332
|
-
for repo in $(get_repos); do
|
|
333
|
-
repo_path=$(get_repo_full_path "$repo")
|
|
334
|
-
echo "$repo:" && git -C "$repo_path" branch
|
|
335
|
-
done
|
|
336
|
-
```
|
|
337
|
-
|
|
338
|
-
---
|
|
69
|
+
## Safety Rules
|
|
339
70
|
|
|
340
|
-
**
|
|
71
|
+
- **NEVER** commit directly to develop
|
|
72
|
+
- **NEVER** force push
|
|
73
|
+
- **NEVER** commit secrets (.env, credentials)
|
|
74
|
+
- **ALWAYS** show diff before committing
|
|
75
|
+
- **ALWAYS** use branches
|
|
@@ -249,6 +249,37 @@ It goes directly to a file, then jq reads it safely.
|
|
|
249
249
|
|
|
250
250
|
## Step 6: Invoke Judge Skill
|
|
251
251
|
|
|
252
|
+
**Detect SWE-bench scenarios for deterministic evaluation:**
|
|
253
|
+
|
|
254
|
+
Check if the scenario is from SWE-bench by looking at its path or category:
|
|
255
|
+
```python
|
|
256
|
+
is_swebench = (
|
|
257
|
+
'swe-bench' in scenario_path.lower() or
|
|
258
|
+
scenario.get('category') == 'swe-bench' or
|
|
259
|
+
scenario.get('source') == 'swe-bench'
|
|
260
|
+
)
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
**If SWE-bench scenario:**
|
|
264
|
+
|
|
265
|
+
Use deterministic Python-based evaluation instead of LLM-as-judge:
|
|
266
|
+
|
|
267
|
+
```bash
|
|
268
|
+
# Save response to temp file for Python judge
|
|
269
|
+
echo '{"result": "{RESPONSE}"}' > /tmp/solo_response_$$.json
|
|
270
|
+
|
|
271
|
+
# Run SWE-bench judge (deterministic scoring against ground truth)
|
|
272
|
+
python3 .pennyfarthing/scripts/test/swebench-judge.py {scenario_name} /tmp/solo_response_$$.json
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
The Python script returns:
|
|
276
|
+
- `total`: Score out of 100
|
|
277
|
+
- `scores`: Breakdown by category (root_cause, fix_quality, completeness, persona)
|
|
278
|
+
- `details`: Specific findings and matches
|
|
279
|
+
|
|
280
|
+
**If standard scenario (non-SWE-bench):**
|
|
281
|
+
|
|
282
|
+
Use LLM-as-judge:
|
|
252
283
|
```
|
|
253
284
|
/judge --mode solo --data {
|
|
254
285
|
"spec": "{spec}",
|
|
@@ -757,7 +757,7 @@ When implementing approval gates:
|
|
|
757
757
|
- `Task` with `subagent_type`: Delegated gate execution
|
|
758
758
|
|
|
759
759
|
### Reflector Hook
|
|
760
|
-
- `
|
|
760
|
+
- `question_reflector_check.py`: Enforces CYCLIST marker before AskUserQuestion
|
|
761
761
|
- Markers: `<!-- CYCLIST:CHOICES:... -->`, `<!-- CYCLIST:QUESTION:yesno|open -->`
|
|
762
762
|
|
|
763
763
|
---
|