orchestr8 2.4.0 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.blueprint/agents/AGENT_BA_CASS.md +50 -25
- package/.blueprint/agents/AGENT_DEVELOPER_CODEY.md +60 -69
- package/.blueprint/agents/AGENT_SPECIFICATION_ALEX.md +45 -0
- package/.blueprint/agents/AGENT_TESTER_NIGEL.md +72 -105
- package/.blueprint/features/feature_adaptive-retry/FEATURE_SPEC.md +239 -0
- package/.blueprint/features/feature_adaptive-retry/IMPLEMENTATION_PLAN.md +48 -0
- package/.blueprint/features/feature_adaptive-retry/story-prompt-modification.md +85 -0
- package/.blueprint/features/feature_adaptive-retry/story-retry-config.md +89 -0
- package/.blueprint/features/feature_adaptive-retry/story-should-retry.md +98 -0
- package/.blueprint/features/feature_adaptive-retry/story-strategy-recommendation.md +85 -0
- package/.blueprint/features/feature_agent-guardrails/FEATURE_SPEC.md +328 -0
- package/.blueprint/features/feature_agent-guardrails/IMPLEMENTATION_PLAN.md +90 -0
- package/.blueprint/features/feature_agent-guardrails/story-citation-requirements.md +50 -0
- package/.blueprint/features/feature_agent-guardrails/story-confidentiality.md +50 -0
- package/.blueprint/features/feature_agent-guardrails/story-escalation-protocol.md +55 -0
- package/.blueprint/features/feature_agent-guardrails/story-source-restrictions.md +50 -0
- package/.blueprint/features/feature_feedback-loop/FEATURE_SPEC.md +347 -0
- package/.blueprint/features/feature_feedback-loop/IMPLEMENTATION_PLAN.md +71 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-collection.md +63 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-config.md +61 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-insights.md +63 -0
- package/.blueprint/features/feature_feedback-loop/story-quality-gates.md +57 -0
- package/.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md +239 -0
- package/.blueprint/features/feature_pipeline-history/IMPLEMENTATION_PLAN.md +71 -0
- package/.blueprint/features/feature_pipeline-history/story-clear-history.md +73 -0
- package/.blueprint/features/feature_pipeline-history/story-display-history.md +75 -0
- package/.blueprint/features/feature_pipeline-history/story-record-execution.md +76 -0
- package/.blueprint/features/feature_pipeline-history/story-show-statistics.md +85 -0
- package/.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md +288 -0
- package/.blueprint/features/feature_pipeline-insights/IMPLEMENTATION_PLAN.md +65 -0
- package/.blueprint/features/feature_pipeline-insights/story-anomaly-detection.md +71 -0
- package/.blueprint/features/feature_pipeline-insights/story-bottleneck-analysis.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-failure-patterns.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-json-output.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-trend-analysis.md +78 -0
- package/.blueprint/features/feature_validate-command/FEATURE_SPEC.md +209 -0
- package/.blueprint/features/feature_validate-command/IMPLEMENTATION_PLAN.md +59 -0
- package/.blueprint/features/feature_validate-command/story-failure-output.md +61 -0
- package/.blueprint/features/feature_validate-command/story-node-version-check.md +52 -0
- package/.blueprint/features/feature_validate-command/story-run-validation.md +59 -0
- package/.blueprint/features/feature_validate-command/story-success-output.md +50 -0
- package/.blueprint/system_specification/SYSTEM_SPEC.md +248 -0
- package/README.md +174 -40
- package/SKILL.md +399 -74
- package/bin/cli.js +128 -20
- package/package.json +1 -1
- package/src/feedback.js +171 -0
- package/src/history.js +306 -0
- package/src/index.js +57 -2
- package/src/init.js +2 -6
- package/src/insights.js +504 -0
- package/src/retry.js +274 -0
- package/src/update.js +10 -2
- package/src/validate.js +172 -0
- package/src/skills.js +0 -93
|
@@ -0,0 +1,239 @@
|
|
|
1
|
+
# Feature Specification — Adaptive Retry
|
|
2
|
+
|
|
3
|
+
## 1. Feature Intent
|
|
4
|
+
**Why this feature exists.**
|
|
5
|
+
|
|
6
|
+
The orchestr8 pipeline currently offers simple retry/skip/abort options when an agent fails (per SKILL.md:Error Handling). This feature introduces **intelligent retry logic** that learns from failure history to make smarter decisions about how to handle failures.
|
|
7
|
+
|
|
8
|
+
- **Problem being addressed:** Pipeline failures are handled uniformly regardless of context. Repeated failures at the same stage waste time with identical retry attempts.
|
|
9
|
+
- **User need:** Developers want the pipeline to adapt its retry approach based on what has historically worked, reducing manual intervention and improving success rates.
|
|
10
|
+
- **System alignment:** Per SYSTEM_SPEC.md:Section 6 (High-Level Lifecycle), recovery from failure is already supported. This feature enhances that capability by making recovery **adaptive** rather than static.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## 2. Scope
|
|
15
|
+
### In Scope
|
|
16
|
+
- Retry configuration file management (`.claude/retry-config.json`)
|
|
17
|
+
- Failure pattern detection using existing history module (`src/history.js`)
|
|
18
|
+
- Strategy selection logic based on failure rates and attempt counts
|
|
19
|
+
- CLI commands for viewing and managing retry configuration
|
|
20
|
+
- Integration with pipeline error handling (advisory, not replacing user choice)
|
|
21
|
+
|
|
22
|
+
### Out of Scope
|
|
23
|
+
- Automatic prompt rewriting (strategies describe *what* to do; agents execute)
|
|
24
|
+
- Modifications to agent specifications or core pipeline orchestration
|
|
25
|
+
- Machine learning or model-based prediction (rule-based only)
|
|
26
|
+
- Cross-feature failure correlation (each feature is handled independently)
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## 3. Actors Involved
|
|
31
|
+
|
|
32
|
+
### Human User
|
|
33
|
+
- Can view current retry configuration via CLI
|
|
34
|
+
- Can modify thresholds and enabled strategies
|
|
35
|
+
- Can reset configuration to defaults
|
|
36
|
+
- Retains final decision authority on retry/skip/abort during pipeline execution
|
|
37
|
+
|
|
38
|
+
### Pipeline Orchestrator (SKILL.md implementation)
|
|
39
|
+
- Queries retry module when an agent fails
|
|
40
|
+
- Receives recommended strategy and displays it to user
|
|
41
|
+
- Applies strategy modifications to agent prompts when user accepts retry
|
|
42
|
+
|
|
43
|
+
### History Module (src/history.js)
|
|
44
|
+
- Provides failure data for strategy calculation
|
|
45
|
+
- Is read-only from this feature's perspective
|
|
46
|
+
|
|
47
|
+
### Insights Module (src/insights.js)
|
|
48
|
+
- Provides `analyzeFailures()` which calculates failure rates by stage
|
|
49
|
+
- Is read-only from this feature's perspective
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## 4. Behaviour Overview
|
|
54
|
+
|
|
55
|
+
### Happy Path: Strategy-Guided Retry
|
|
56
|
+
1. Agent fails during pipeline execution
|
|
57
|
+
2. Retry module queries history for recent failures at this stage
|
|
58
|
+
3. Module calculates failure rate over recent window (default: last 10 runs)
|
|
59
|
+
4. If failure rate > threshold (20%), recommend alternative strategy; otherwise recommend simple retry
|
|
60
|
+
5. Display recommendation to user: "Recommended strategy: {strategy}. Retry with this approach? (y/n/abort)"
|
|
61
|
+
6. If user accepts, modify agent prompt per strategy and retry
|
|
62
|
+
7. Record outcome in history
|
|
63
|
+
|
|
64
|
+
### Alternative: First Failure
|
|
65
|
+
- If this is the first recorded failure for a stage (no history), default to simple retry
|
|
66
|
+
- No prompt modification; standard retry behaviour
|
|
67
|
+
|
|
68
|
+
### Alternative: Max Retries Exceeded
|
|
69
|
+
- After `maxRetries` (default 3) attempts at same stage for same feature, warn user
|
|
70
|
+
- User can still choose to retry, but recommendation becomes "abort or skip"
|
|
71
|
+
|
|
72
|
+
### Alternative: User Declines Strategy
|
|
73
|
+
- User can override recommendation and choose different action (retry/skip/abort)
|
|
74
|
+
- System respects user choice; recommendation is advisory only
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## 5. State & Lifecycle Interactions
|
|
79
|
+
|
|
80
|
+
### States Affected
|
|
81
|
+
Per SYSTEM_SPEC.md:Section 6, pipeline states are: INIT, ALEX, CASS, NIGEL, CODEY_PLAN, CODEY_IMPLEMENT, COMMIT, COMPLETE, PAUSED, RESUME.
|
|
82
|
+
|
|
83
|
+
This feature operates within the **transition from any agent state to FAILED**:
|
|
84
|
+
- On agent error, before prompting user, retry module is consulted
|
|
85
|
+
- State remains in current stage during retry attempts
|
|
86
|
+
- State transitions to FAILED only on abort (unchanged from current behaviour)
|
|
87
|
+
|
|
88
|
+
### Lifecycle Classification
|
|
89
|
+
- **State-constraining:** Does not create new states; constrains how transitions to FAILED are handled
|
|
90
|
+
- **Queue-modifying:** May update `failed` array with additional metadata (strategy attempted, attempt count)
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## 6. Rules & Decision Logic
|
|
95
|
+
|
|
96
|
+
### Rule 1: Calculate Stage Failure Rate
|
|
97
|
+
- **Description:** Determine how often a specific stage has failed in recent history
|
|
98
|
+
- **Inputs:** Stage name, history entries, window size (default 10)
|
|
99
|
+
- **Outputs:** Failure rate (0.0 to 1.0)
|
|
100
|
+
- **Type:** Deterministic
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
failureRate = failedRunsAtStage / totalRecentRuns
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
### Rule 2: Select Retry Strategy
|
|
107
|
+
- **Description:** Choose appropriate strategy based on failure rate and attempt count
|
|
108
|
+
- **Inputs:** Stage name, current attempt number, failure rate, configured strategies
|
|
109
|
+
- **Outputs:** Recommended strategy name
|
|
110
|
+
- **Type:** Deterministic with precedence
|
|
111
|
+
|
|
112
|
+
```
|
|
113
|
+
if (attemptCount > maxRetries) → "abort-recommended"
|
|
114
|
+
if (failureRate > highFailureThreshold) → alternativeStrategy[stage][attemptCount]
|
|
115
|
+
else → "retry"
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
### Rule 3: Apply Strategy to Prompt
|
|
119
|
+
- **Description:** Modify agent prompt based on selected strategy
|
|
120
|
+
- **Inputs:** Original prompt, strategy name, stage context
|
|
121
|
+
- **Outputs:** Modified prompt (or original if strategy is "retry")
|
|
122
|
+
- **Type:** Deterministic
|
|
123
|
+
|
|
124
|
+
Strategy applications:
|
|
125
|
+
| Strategy | Prompt Modification |
|
|
126
|
+
|----------|-------------------|
|
|
127
|
+
| `retry` | No modification |
|
|
128
|
+
| `simplify-prompt` | Add: "Focus on core requirements only. Skip edge cases and optional sections." |
|
|
129
|
+
| `reduce-stories` | Add: "Write only the 2-3 most critical user stories. Defer others to follow-up." |
|
|
130
|
+
| `simplify-tests` | Add: "Write only happy-path tests for each AC. Skip edge cases." |
|
|
131
|
+
| `add-context` | Prepend relevant sections from previous stage outputs |
|
|
132
|
+
| `incremental` | Add: "Implement one test at a time. Stop and report after each." |
|
|
133
|
+
| `rollback` | Execute `git checkout -- .` on implementation files before retry |
|
|
134
|
+
|
|
135
|
+
### Rule 4: Persist Configuration
|
|
136
|
+
- **Description:** Configuration survives across sessions
|
|
137
|
+
- **Inputs:** User modifications via CLI
|
|
138
|
+
- **Outputs:** Updated `.claude/retry-config.json`
|
|
139
|
+
- **Type:** Deterministic (file write)
|
|
140
|
+
|
|
141
|
+
---
|
|
142
|
+
|
|
143
|
+
## 7. Dependencies
|
|
144
|
+
|
|
145
|
+
### System Components
|
|
146
|
+
- **src/history.js:** `readHistoryFile()` for failure data access (read-only dependency)
|
|
147
|
+
- **src/insights.js:** `analyzeFailures()` for failure pattern analysis (read-only dependency)
|
|
148
|
+
- **src/orchestrator.js:** Queue management; retry module will export functions for orchestrator to call
|
|
149
|
+
- **SKILL.md:** Error handling section will reference retry module recommendations
|
|
150
|
+
|
|
151
|
+
### File Dependencies
|
|
152
|
+
- **`.claude/pipeline-history.json`:** Must exist with valid structure for accurate analysis
|
|
153
|
+
- **`.claude/retry-config.json`:** Created on first use if not present; gitignored
|
|
154
|
+
|
|
155
|
+
### Operational Dependencies
|
|
156
|
+
- History module must be functional (feature degrades gracefully if corrupted)
|
|
157
|
+
- File system write access for configuration persistence
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
## 8. Non-Functional Considerations
|
|
162
|
+
|
|
163
|
+
### Performance
|
|
164
|
+
- Strategy calculation should complete in <100ms (simple arithmetic on in-memory data)
|
|
165
|
+
- No blocking I/O during recommendation phase beyond initial history read
|
|
166
|
+
|
|
167
|
+
### Resilience
|
|
168
|
+
- If history file is corrupted, default to simple retry (no strategy recommendation)
|
|
169
|
+
- If config file is corrupted or missing, use hardcoded defaults
|
|
170
|
+
- All failures in retry module should be caught and logged, not propagated
|
|
171
|
+
|
|
172
|
+
### Audit/Logging
|
|
173
|
+
- Retry attempts and strategies used should be recorded in history entries
|
|
174
|
+
- New history fields: `retryAttempts`, `strategiesUsed[]`
|
|
175
|
+
|
|
176
|
+
### Maintainability
|
|
177
|
+
- Strategies should be configurable without code changes
|
|
178
|
+
- New strategies can be added by updating config schema and `applyStrategy()` function
|
|
179
|
+
|
|
180
|
+
---
|
|
181
|
+
|
|
182
|
+
## 9. Assumptions & Open Questions
|
|
183
|
+
|
|
184
|
+
### Assumptions
|
|
185
|
+
- ASSUMPTION: History file format is stable and matches `src/history.js` expectations
|
|
186
|
+
- ASSUMPTION: Pipeline execution is single-threaded; no concurrent retry decisions needed
|
|
187
|
+
- ASSUMPTION: User preference is final; recommendations are advisory only
|
|
188
|
+
- ASSUMPTION: Stage names match exactly between history, insights, and retry module
|
|
189
|
+
|
|
190
|
+
### Open Questions
|
|
191
|
+
- Should strategy history be factored into recommendations? (e.g., "simplify-prompt worked last time")
|
|
192
|
+
- Should there be a "never retry" option for specific stages?
|
|
193
|
+
- How should retry configuration interact with `--pause-after` flags?
|
|
194
|
+
|
|
195
|
+
---
|
|
196
|
+
|
|
197
|
+
## 10. Impact on System Specification
|
|
198
|
+
|
|
199
|
+
### Reinforces Existing Assumptions
|
|
200
|
+
- Per SYSTEM_SPEC.md:Section 8 (Cross-Cutting Concerns), failure handling already "offers retry, skip, abort"
|
|
201
|
+
- This feature enhances rather than replaces that pattern
|
|
202
|
+
- Per SYSTEM_SPEC.md:Section 7 (Governing Rules), "recovery-safe" queue behaviour is maintained
|
|
203
|
+
|
|
204
|
+
### Stretches Existing Assumptions
|
|
205
|
+
- History module was designed for reporting/insights; now becomes an active input to pipeline decisions
|
|
206
|
+
- This elevates history from "observability" to "operational dependency"
|
|
207
|
+
|
|
208
|
+
### No Contradictions Identified
|
|
209
|
+
- Feature operates within existing error handling flow
|
|
210
|
+
- Does not modify agent specifications or pipeline sequence
|
|
211
|
+
- Preserves user authority over final decisions
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
## 11. Handover to BA (Cass)
|
|
216
|
+
|
|
217
|
+
### Story Themes
|
|
218
|
+
1. **Configuration Management:** User can view, modify, and reset retry configuration
|
|
219
|
+
2. **Strategy Recommendation:** Pipeline recommends retry strategy based on failure history
|
|
220
|
+
3. **Prompt Modification:** Retry strategies modify agent prompts to improve success probability
|
|
221
|
+
4. **History Integration:** Retry decisions are informed by historical failure patterns
|
|
222
|
+
|
|
223
|
+
### Expected Story Boundaries
|
|
224
|
+
- CLI commands (`retry-config`, `retry-config reset`) as separate story
|
|
225
|
+
- Strategy selection logic as separate story
|
|
226
|
+
- Strategy application (prompt modification) as separate story
|
|
227
|
+
- Graceful degradation (corrupted history/config) as non-functional story or AC within above
|
|
228
|
+
|
|
229
|
+
### Areas Needing Careful Framing
|
|
230
|
+
- User choice must remain paramount; stories should emphasize "recommendation" not "automation"
|
|
231
|
+
- The "rollback" strategy involves destructive action (`git checkout`); needs explicit user confirmation
|
|
232
|
+
- Incremental strategy changes Codey's execution model; may need coordination with Codey's agent spec
|
|
233
|
+
|
|
234
|
+
---
|
|
235
|
+
|
|
236
|
+
## 12. Change Log (Feature-Level)
|
|
237
|
+
| Date | Change | Reason | Raised By |
|
|
238
|
+
|------|--------|--------|-----------|
|
|
239
|
+
| 2026-02-24 | Initial feature specification | New feature request | Alex |
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Implementation Plan - Adaptive Retry
|
|
2
|
+
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
This feature adds intelligent retry logic to the orchestr8 pipeline via a new `src/retry.js` module. The module calculates failure rates from history, recommends strategies based on configurable thresholds, and applies prompt modifications when users accept recommended strategies. The implementation integrates with existing `src/history.js` for failure data and provides CLI commands for configuration management.
|
|
6
|
+
|
|
7
|
+
## Files to Create/Modify
|
|
8
|
+
|
|
9
|
+
| Path | Action | Purpose |
|
|
10
|
+
|------|--------|---------|
|
|
11
|
+
| `src/retry.js` | Create | Core retry logic: config management, failure rate calculation, strategy recommendation, prompt modification |
|
|
12
|
+
| `src/index.js` | Modify | Export retry module functions |
|
|
13
|
+
| `bin/cli.js` | Modify | Add `retry-config` CLI command with `set` and `reset` subcommands |
|
|
14
|
+
|
|
15
|
+
## Implementation Steps
|
|
16
|
+
|
|
17
|
+
1. **Create `src/retry.js` with config management** - Implement `getDefaultConfig()`, `readConfig()`, `writeConfig()`, `resetConfig()` functions for `.claude/retry-config.json`
|
|
18
|
+
|
|
19
|
+
2. **Add failure rate calculation** - Implement `calculateFailureRate(stage, history, windowSize)` that returns failure rate (0-1) for a stage over the recent window
|
|
20
|
+
|
|
21
|
+
3. **Add strategy recommendation logic** - Implement `recommendStrategy(stage, attemptCount, failureRate, config)` returning strategy name or "abort-recommended"
|
|
22
|
+
|
|
23
|
+
4. **Add prompt modification** - Implement `applyStrategy(strategy, originalPrompt)` that returns modified prompt based on strategy
|
|
24
|
+
|
|
25
|
+
5. **Add graceful degradation** - Wrap all file operations in try/catch; return defaults on corrupted/missing files; log warnings
|
|
26
|
+
|
|
27
|
+
6. **Add CLI commands to `bin/cli.js`** - Add `retry-config` (view), `retry-config set <key> <value>`, `retry-config reset` commands
|
|
28
|
+
|
|
29
|
+
7. **Export functions in `src/index.js`** - Export `readConfig`, `writeConfig`, `resetConfig`, `calculateFailureRate`, `recommendStrategy`, `applyStrategy`
|
|
30
|
+
|
|
31
|
+
8. **Run tests and iterate** - Execute `node --test test/feature_adaptive-retry.test.js` after each step; fix failures
|
|
32
|
+
|
|
33
|
+
## Key Functions
|
|
34
|
+
|
|
35
|
+
- `getDefaultConfig()` - Returns hardcoded default configuration object
|
|
36
|
+
- `readConfig()` - Reads `.claude/retry-config.json` or returns defaults if missing/corrupted
|
|
37
|
+
- `writeConfig(config)` - Writes config to file, creates `.claude/` directory if needed
|
|
38
|
+
- `resetConfig()` - Replaces config with defaults
|
|
39
|
+
- `calculateFailureRate(stage, history, windowSize)` - Calculates failure rate for stage over window
|
|
40
|
+
- `recommendStrategy(stage, attemptCount, failureRate, config)` - Returns recommended strategy name
|
|
41
|
+
- `applyStrategy(strategy, originalPrompt)` - Returns modified prompt or original if "retry"
|
|
42
|
+
- `shouldRetry(stage, featureSlug, history, config, attemptCount)` - Orchestrator entry point; returns recommendation object
|
|
43
|
+
|
|
44
|
+
## Risks/Questions
|
|
45
|
+
|
|
46
|
+
- **Rollback strategy is destructive**: Per story-prompt-modification.md:AC-6, `git checkout -- .` requires explicit confirmation before execution; implementation must surface this warning
|
|
47
|
+
- **History module dependency**: If `readHistoryFile()` returns `{ error: 'corrupted' }`, module must degrade gracefully per test T-SH-2
|
|
48
|
+
- **Strategy escalation boundary**: When `attemptCount - 1` exceeds strategy list length, use last strategy before switching to "abort-recommended"
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Story — Prompt Modification
|
|
2
|
+
|
|
3
|
+
## User Story
|
|
4
|
+
|
|
5
|
+
As a **developer using orchestr8**, I want the **pipeline to modify agent prompts based on the selected retry strategy** so that **retry attempts have a better chance of success by adjusting the agent's instructions**.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / Scope
|
|
10
|
+
|
|
11
|
+
- Per FEATURE_SPEC.md:Section 6 (Rule 3), strategies apply specific modifications to agent prompts
|
|
12
|
+
- Per FEATURE_SPEC.md:Section 11 (Areas Needing Careful Framing), the "rollback" strategy requires explicit user confirmation
|
|
13
|
+
- Per FEATURE_SPEC.md:Section 2 (Out of Scope), automatic prompt rewriting is not supported; strategies describe what to do
|
|
14
|
+
- Per SYSTEM_SPEC.md:Section 7 (Agent Rules), agents must not make silent changes
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Acceptance Criteria
|
|
19
|
+
|
|
20
|
+
**AC-1 — Apply "simplify-prompt" strategy**
|
|
21
|
+
- Given the user accepts a retry with the "simplify-prompt" strategy,
|
|
22
|
+
- When the agent is re-invoked,
|
|
23
|
+
- Then the prompt is modified to append: "Focus on core requirements only. Skip edge cases and optional sections.",
|
|
24
|
+
- And the modification is visible in the agent's task context.
|
|
25
|
+
|
|
26
|
+
**AC-2 — Apply "reduce-stories" strategy**
|
|
27
|
+
- Given the user accepts a retry with the "reduce-stories" strategy,
|
|
28
|
+
- When Cass is re-invoked,
|
|
29
|
+
- Then the prompt is modified to append: "Write only the 2-3 most critical user stories. Defer others to follow-up.",
|
|
30
|
+
- And the deferred stories are noted in the summary.
|
|
31
|
+
|
|
32
|
+
**AC-3 — Apply "simplify-tests" strategy**
|
|
33
|
+
- Given the user accepts a retry with the "simplify-tests" strategy,
|
|
34
|
+
- When Nigel is re-invoked,
|
|
35
|
+
- Then the prompt is modified to append: "Write only happy-path tests for each AC. Skip edge cases.",
|
|
36
|
+
- And edge case tests are explicitly deferred.
|
|
37
|
+
|
|
38
|
+
**AC-4 — Apply "add-context" strategy**
|
|
39
|
+
- Given the user accepts a retry with the "add-context" strategy,
|
|
40
|
+
- When the agent is re-invoked,
|
|
41
|
+
- Then relevant sections from previous stage outputs are prepended to the prompt,
|
|
42
|
+
- And the source of added context is indicated (e.g., "Context from FEATURE_SPEC.md:Section 4").
|
|
43
|
+
|
|
44
|
+
**AC-5 — Apply "incremental" strategy**
|
|
45
|
+
- Given the user accepts a retry with the "incremental" strategy,
|
|
46
|
+
- When Codey is re-invoked,
|
|
47
|
+
- Then the prompt is modified to append: "Implement one test at a time. Stop and report after each.",
|
|
48
|
+
- And each implementation step produces a discrete output before continuing.
|
|
49
|
+
|
|
50
|
+
**AC-6 — Apply "rollback" strategy with confirmation**
|
|
51
|
+
- Given the user accepts a retry with the "rollback" strategy,
|
|
52
|
+
- When the user confirms the destructive action,
|
|
53
|
+
- Then `git checkout -- .` is executed on implementation files before retry,
|
|
54
|
+
- And a confirmation prompt warns: "This will discard uncommitted changes to implementation files. Proceed? (y/n)".
|
|
55
|
+
|
|
56
|
+
**AC-7 — No modification for "retry" strategy**
|
|
57
|
+
- Given the user accepts a simple "retry" (no strategy modification),
|
|
58
|
+
- When the agent is re-invoked,
|
|
59
|
+
- Then the original prompt is used without any modification,
|
|
60
|
+
- And the retry proceeds identically to the first attempt.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Strategy Reference Table
|
|
65
|
+
|
|
66
|
+
Per FEATURE_SPEC.md:Section 6 (Rule 3):
|
|
67
|
+
|
|
68
|
+
| Strategy | Modification Applied |
|
|
69
|
+
|----------|---------------------|
|
|
70
|
+
| `retry` | No modification |
|
|
71
|
+
| `simplify-prompt` | Append: "Focus on core requirements only. Skip edge cases and optional sections." |
|
|
72
|
+
| `reduce-stories` | Append: "Write only the 2-3 most critical user stories. Defer others to follow-up." |
|
|
73
|
+
| `simplify-tests` | Append: "Write only happy-path tests for each AC. Skip edge cases." |
|
|
74
|
+
| `add-context` | Prepend relevant sections from previous stage outputs |
|
|
75
|
+
| `incremental` | Append: "Implement one test at a time. Stop and report after each." |
|
|
76
|
+
| `rollback` | Execute `git checkout -- .` on implementation files before retry |
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## Out of Scope
|
|
81
|
+
|
|
82
|
+
- Automatic prompt rewriting based on error analysis (per FEATURE_SPEC.md:Section 2)
|
|
83
|
+
- Custom user-defined prompt modifications (strategies are predefined)
|
|
84
|
+
- Modifying agent specifications themselves (per FEATURE_SPEC.md:Section 2)
|
|
85
|
+
- Combining multiple strategies in a single retry (one strategy per attempt)
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
# Story — Retry Configuration Management
|
|
2
|
+
|
|
3
|
+
## User Story
|
|
4
|
+
|
|
5
|
+
As a **developer using orchestr8**, I want to **view, modify, and reset retry configuration** so that **I can customize how the pipeline handles failures based on my project's needs**.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / Scope
|
|
10
|
+
|
|
11
|
+
- Per FEATURE_SPEC.md:Section 2, retry configuration is managed via `.claude/retry-config.json`
|
|
12
|
+
- Per FEATURE_SPEC.md:Section 3 (Actors), users retain final decision authority and can modify thresholds
|
|
13
|
+
- Per FEATURE_SPEC.md:Section 11 (Handover), this story covers CLI commands for `retry-config` and `retry-config reset`
|
|
14
|
+
- Configuration file is gitignored (per FEATURE_SPEC.md:Section 7)
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Acceptance Criteria
|
|
19
|
+
|
|
20
|
+
**AC-1 — View current configuration**
|
|
21
|
+
- Given the retry configuration file exists at `.claude/retry-config.json`,
|
|
22
|
+
- When I run `orchestr8 retry-config`,
|
|
23
|
+
- Then the current configuration is displayed in a readable format showing thresholds, window size, max retries, and enabled strategies.
|
|
24
|
+
|
|
25
|
+
**AC-2 — View defaults when no configuration exists**
|
|
26
|
+
- Given no retry configuration file exists,
|
|
27
|
+
- When I run `orchestr8 retry-config`,
|
|
28
|
+
- Then the hardcoded default configuration is displayed,
|
|
29
|
+
- And a message indicates "Using default configuration (no config file found)".
|
|
30
|
+
|
|
31
|
+
**AC-3 — Modify configuration value**
|
|
32
|
+
- Given I run `orchestr8 retry-config set <key> <value>` (e.g., `orchestr8 retry-config set maxRetries 5`),
|
|
33
|
+
- When the key is valid and value is of correct type,
|
|
34
|
+
- Then the configuration file is updated with the new value,
|
|
35
|
+
- And a confirmation message shows the updated setting.
|
|
36
|
+
|
|
37
|
+
**AC-4 — Reject invalid configuration key**
|
|
38
|
+
- Given I run `orchestr8 retry-config set <invalidKey> <value>`,
|
|
39
|
+
- When the key is not recognized,
|
|
40
|
+
- Then an error message is displayed listing valid configuration keys,
|
|
41
|
+
- And no file modification occurs.
|
|
42
|
+
|
|
43
|
+
**AC-5 — Reset configuration to defaults**
|
|
44
|
+
- Given the retry configuration file exists with custom values,
|
|
45
|
+
- When I run `orchestr8 retry-config reset`,
|
|
46
|
+
- Then the configuration file is replaced with default values,
|
|
47
|
+
- And a confirmation message indicates "Retry configuration reset to defaults".
|
|
48
|
+
|
|
49
|
+
**AC-6 — Create configuration file on first modification**
|
|
50
|
+
- Given no retry configuration file exists,
|
|
51
|
+
- When I run `orchestr8 retry-config set <key> <value>`,
|
|
52
|
+
- Then the configuration file is created with default values plus the specified modification,
|
|
53
|
+
- And the file is created in `.claude/retry-config.json`.
|
|
54
|
+
|
|
55
|
+
**AC-7 — Handle corrupted configuration gracefully**
|
|
56
|
+
- Given the retry configuration file exists but contains invalid JSON,
|
|
57
|
+
- When I run `orchestr8 retry-config`,
|
|
58
|
+
- Then an error message is displayed indicating the file is corrupted,
|
|
59
|
+
- And a suggestion to run `orchestr8 retry-config reset` is shown.
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## Configuration Schema
|
|
64
|
+
|
|
65
|
+
Per FEATURE_SPEC.md:Section 6 (Rules), default configuration includes:
|
|
66
|
+
|
|
67
|
+
```json
|
|
68
|
+
{
|
|
69
|
+
"maxRetries": 3,
|
|
70
|
+
"windowSize": 10,
|
|
71
|
+
"highFailureThreshold": 0.2,
|
|
72
|
+
"strategies": {
|
|
73
|
+
"alex": ["simplify-prompt", "add-context"],
|
|
74
|
+
"cass": ["reduce-stories", "simplify-prompt"],
|
|
75
|
+
"nigel": ["simplify-tests", "add-context"],
|
|
76
|
+
"codey-plan": ["add-context", "simplify-prompt"],
|
|
77
|
+
"codey-implement": ["incremental", "rollback"]
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## Out of Scope
|
|
85
|
+
|
|
86
|
+
- Modifying strategy definitions themselves (strategies are code, not config)
|
|
87
|
+
- Per-feature configuration overrides (configuration is global)
|
|
88
|
+
- Configuration UI/interactive editor (CLI only)
|
|
89
|
+
- Validating strategy effectiveness (config is just data; strategy logic is separate)
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Story — Should Retry Decision Logic
|
|
2
|
+
|
|
3
|
+
## User Story
|
|
4
|
+
|
|
5
|
+
As a **developer using orchestr8**, I want the **pipeline to intelligently decide whether to recommend retrying** so that **I receive useful guidance based on attempt count, failure history, and system state**.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / Scope
|
|
10
|
+
|
|
11
|
+
- Per FEATURE_SPEC.md:Section 4 (Happy Path), retry module is consulted when an agent fails
|
|
12
|
+
- Per FEATURE_SPEC.md:Section 5 (State & Lifecycle), this operates within the transition from any agent state to FAILED
|
|
13
|
+
- Per FEATURE_SPEC.md:Section 8 (Resilience), graceful degradation is required when history/config is corrupted
|
|
14
|
+
- Per SYSTEM_SPEC.md:Section 8 (Failure Handling), each agent spawn currently offers retry, skip, abort
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Acceptance Criteria
|
|
19
|
+
|
|
20
|
+
**AC-1 — Consult retry module on agent failure**
|
|
21
|
+
- Given an agent fails during pipeline execution,
|
|
22
|
+
- When the error handling flow is triggered,
|
|
23
|
+
- Then the retry module is consulted before displaying options to the user,
|
|
24
|
+
- And the module returns a recommendation (strategy name or "abort-recommended").
|
|
25
|
+
|
|
26
|
+
**AC-2 — Record retry attempts in history**
|
|
27
|
+
- Given the user chooses to retry (with or without strategy),
|
|
28
|
+
- When the retry completes (success or failure),
|
|
29
|
+
- Then the outcome is recorded in `.claude/pipeline-history.json`,
|
|
30
|
+
- And the record includes: `retryAttempts` count and `strategiesUsed[]` array.
|
|
31
|
+
|
|
32
|
+
**AC-3 — Track attempt count per stage per feature**
|
|
33
|
+
- Given an agent fails and user retries,
|
|
34
|
+
- When tracking retry state,
|
|
35
|
+
- Then the attempt count is incremented for the current stage and feature combination,
|
|
36
|
+
- And the count persists across retries within the same pipeline run.
|
|
37
|
+
|
|
38
|
+
**AC-4 — Degrade gracefully with corrupted history**
|
|
39
|
+
- Given the history file at `.claude/pipeline-history.json` is corrupted or missing,
|
|
40
|
+
- When the retry module is consulted,
|
|
41
|
+
- Then the module defaults to simple retry recommendation,
|
|
42
|
+
- And a warning is logged: "History file unavailable; defaulting to simple retry".
|
|
43
|
+
|
|
44
|
+
**AC-5 — Degrade gracefully with corrupted configuration**
|
|
45
|
+
- Given the configuration file at `.claude/retry-config.json` is corrupted or missing,
|
|
46
|
+
- When the retry module calculates strategy,
|
|
47
|
+
- Then hardcoded defaults are used for thresholds and strategies,
|
|
48
|
+
- And a warning is logged: "Config file unavailable; using default settings".
|
|
49
|
+
|
|
50
|
+
**AC-6 — Preserve state transitions**
|
|
51
|
+
- Given a retry is in progress,
|
|
52
|
+
- When the retry attempt completes successfully,
|
|
53
|
+
- Then the pipeline continues to the next stage as normal,
|
|
54
|
+
- And the state transitions per SYSTEM_SPEC.md:Section 6 (e.g., CASS to NIGEL).
|
|
55
|
+
|
|
56
|
+
**AC-7 — Support abort without retry**
|
|
57
|
+
- Given the user is shown retry options,
|
|
58
|
+
- When the user selects "abort",
|
|
59
|
+
- Then the feature is moved to the failed list in the queue,
|
|
60
|
+
- And no retry is attempted,
|
|
61
|
+
- And the failure is recorded with reason "user-aborted".
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Integration Points
|
|
66
|
+
|
|
67
|
+
Per FEATURE_SPEC.md:Section 7 (Dependencies):
|
|
68
|
+
|
|
69
|
+
- **src/history.js:** `readHistoryFile()` provides failure data
|
|
70
|
+
- **src/insights.js:** `analyzeFailures()` calculates failure rates by stage
|
|
71
|
+
- **src/orchestrator.js:** Queue management receives retry module functions
|
|
72
|
+
- **SKILL.md:** Error handling section references retry recommendations
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## History Record Schema
|
|
77
|
+
|
|
78
|
+
Per FEATURE_SPEC.md:Section 8 (Audit/Logging), new fields in history entries:
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{
|
|
82
|
+
"featureSlug": "adaptive-retry",
|
|
83
|
+
"stage": "cass",
|
|
84
|
+
"outcome": "success",
|
|
85
|
+
"timestamp": "2026-02-24T10:30:00Z",
|
|
86
|
+
"retryAttempts": 2,
|
|
87
|
+
"strategiesUsed": ["retry", "reduce-stories"]
|
|
88
|
+
}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## Out of Scope
|
|
94
|
+
|
|
95
|
+
- Automatic retry without user prompt (user always has final say)
|
|
96
|
+
- Retry across different features (each feature's retry state is independent)
|
|
97
|
+
- Persistent retry state across pipeline invocations (state is per-run only; history is persistent)
|
|
98
|
+
- Batch retry of multiple failed features (single-feature focus per SYSTEM_SPEC.md:Section 10)
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Story — Strategy Recommendation
|
|
2
|
+
|
|
3
|
+
## User Story
|
|
4
|
+
|
|
5
|
+
As a **developer using orchestr8**, I want the **pipeline to recommend a retry strategy based on failure history** so that **I can make informed decisions about how to handle failures more effectively**.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / Scope
|
|
10
|
+
|
|
11
|
+
- Per FEATURE_SPEC.md:Section 4 (Behaviour Overview), recommendations are advisory only; user retains final choice
|
|
12
|
+
- Per FEATURE_SPEC.md:Section 6 (Rule 1 & Rule 2), strategy selection is based on failure rate calculation
|
|
13
|
+
- Per FEATURE_SPEC.md:Section 3 (Actors), the History Module provides failure data (read-only)
|
|
14
|
+
- Per FEATURE_SPEC.md:Section 7 (Dependencies), uses `src/history.js` and `src/insights.js`
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Acceptance Criteria
|
|
19
|
+
|
|
20
|
+
**AC-1 — Calculate failure rate from history**
|
|
21
|
+
- Given an agent fails during pipeline execution,
|
|
22
|
+
- When the retry module is consulted,
|
|
23
|
+
- Then the failure rate is calculated as `failedRunsAtStage / totalRecentRuns` over the configured window (default 10 runs),
|
|
24
|
+
- And the calculation uses data from `.claude/pipeline-history.json`.
|
|
25
|
+
|
|
26
|
+
**AC-2 — Recommend simple retry for low failure rate**
|
|
27
|
+
- Given the calculated failure rate is at or below the threshold (default 20%),
|
|
28
|
+
- When displaying retry options to the user,
|
|
29
|
+
- Then the recommendation is "retry" with no prompt modification suggested,
|
|
30
|
+
- And the message format is: "Recommended: Simple retry. Retry? (y/n/skip/abort)".
|
|
31
|
+
|
|
32
|
+
**AC-3 — Recommend alternative strategy for high failure rate**
|
|
33
|
+
- Given the calculated failure rate exceeds the threshold (default 20%),
|
|
34
|
+
- When displaying retry options to the user,
|
|
35
|
+
- Then an alternative strategy from the stage's strategy list is recommended,
|
|
36
|
+
- And the message format is: "Recommended strategy: {strategyName}. Retry with this approach? (y/apply/skip/abort)".
|
|
37
|
+
|
|
38
|
+
**AC-4 — Escalate strategy on subsequent attempts**
|
|
39
|
+
- Given a retry attempt has already been made at the current stage,
|
|
40
|
+
- When another failure occurs and the user is prompted again,
|
|
41
|
+
- Then the next strategy in the stage's strategy list is recommended,
|
|
42
|
+
- And if all strategies have been exhausted, "abort-recommended" is suggested.
|
|
43
|
+
|
|
44
|
+
**AC-5 — Default to simple retry with no history**
|
|
45
|
+
- Given this is the first recorded run or no history exists for the current stage,
|
|
46
|
+
- When an agent fails,
|
|
47
|
+
- Then the recommendation is "retry" (simple retry),
|
|
48
|
+
- And a note indicates "No failure history for this stage; defaulting to simple retry".
|
|
49
|
+
|
|
50
|
+
**AC-6 — Warn when max retries exceeded**
|
|
51
|
+
- Given the current attempt count exceeds `maxRetries` (default 3) for the same feature and stage,
|
|
52
|
+
- When displaying options to the user,
|
|
53
|
+
- Then the recommendation is "abort-recommended" or "skip-recommended",
|
|
54
|
+
- And a warning is displayed: "Max retries ({count}) exceeded for {stage}. Consider skipping or aborting."
|
|
55
|
+
|
|
56
|
+
**AC-7 — Display recommendation without forcing choice**
|
|
57
|
+
- Given a strategy recommendation is displayed,
|
|
58
|
+
- When the user selects a different option (e.g., chooses "skip" when "retry" was recommended),
|
|
59
|
+
- Then the user's choice is respected,
|
|
60
|
+
- And no error or warning is shown for overriding the recommendation.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Session State
|
|
65
|
+
|
|
66
|
+
During pipeline execution, retry state is tracked:
|
|
67
|
+
|
|
68
|
+
```js
|
|
69
|
+
retryState = {
|
|
70
|
+
stage: 'cass',
|
|
71
|
+
featureSlug: 'adaptive-retry',
|
|
72
|
+
attemptCount: 2,
|
|
73
|
+
strategiesUsed: ['retry', 'reduce-stories'],
|
|
74
|
+
failureRate: 0.3
|
|
75
|
+
}
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## Out of Scope
|
|
81
|
+
|
|
82
|
+
- Machine learning or predictive models for strategy selection (per FEATURE_SPEC.md:Section 2)
|
|
83
|
+
- Cross-feature failure correlation (per FEATURE_SPEC.md:Section 2)
|
|
84
|
+
- Automatic retry without user confirmation (user choice is paramount)
|
|
85
|
+
- Modifying the agent prompts (that is story-prompt-modification)
|