orchestr8 2.4.0 → 2.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/.blueprint/agents/AGENT_BA_CASS.md +50 -25
  2. package/.blueprint/agents/AGENT_DEVELOPER_CODEY.md +60 -69
  3. package/.blueprint/agents/AGENT_SPECIFICATION_ALEX.md +45 -0
  4. package/.blueprint/agents/AGENT_TESTER_NIGEL.md +72 -105
  5. package/.blueprint/features/feature_adaptive-retry/FEATURE_SPEC.md +239 -0
  6. package/.blueprint/features/feature_adaptive-retry/IMPLEMENTATION_PLAN.md +48 -0
  7. package/.blueprint/features/feature_adaptive-retry/story-prompt-modification.md +85 -0
  8. package/.blueprint/features/feature_adaptive-retry/story-retry-config.md +89 -0
  9. package/.blueprint/features/feature_adaptive-retry/story-should-retry.md +98 -0
  10. package/.blueprint/features/feature_adaptive-retry/story-strategy-recommendation.md +85 -0
  11. package/.blueprint/features/feature_agent-guardrails/FEATURE_SPEC.md +328 -0
  12. package/.blueprint/features/feature_agent-guardrails/IMPLEMENTATION_PLAN.md +90 -0
  13. package/.blueprint/features/feature_agent-guardrails/story-citation-requirements.md +50 -0
  14. package/.blueprint/features/feature_agent-guardrails/story-confidentiality.md +50 -0
  15. package/.blueprint/features/feature_agent-guardrails/story-escalation-protocol.md +55 -0
  16. package/.blueprint/features/feature_agent-guardrails/story-source-restrictions.md +50 -0
  17. package/.blueprint/features/feature_feedback-loop/FEATURE_SPEC.md +347 -0
  18. package/.blueprint/features/feature_feedback-loop/IMPLEMENTATION_PLAN.md +71 -0
  19. package/.blueprint/features/feature_feedback-loop/story-feedback-collection.md +63 -0
  20. package/.blueprint/features/feature_feedback-loop/story-feedback-config.md +61 -0
  21. package/.blueprint/features/feature_feedback-loop/story-feedback-insights.md +63 -0
  22. package/.blueprint/features/feature_feedback-loop/story-quality-gates.md +57 -0
  23. package/.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md +239 -0
  24. package/.blueprint/features/feature_pipeline-history/IMPLEMENTATION_PLAN.md +71 -0
  25. package/.blueprint/features/feature_pipeline-history/story-clear-history.md +73 -0
  26. package/.blueprint/features/feature_pipeline-history/story-display-history.md +75 -0
  27. package/.blueprint/features/feature_pipeline-history/story-record-execution.md +76 -0
  28. package/.blueprint/features/feature_pipeline-history/story-show-statistics.md +85 -0
  29. package/.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md +288 -0
  30. package/.blueprint/features/feature_pipeline-insights/IMPLEMENTATION_PLAN.md +65 -0
  31. package/.blueprint/features/feature_pipeline-insights/story-anomaly-detection.md +71 -0
  32. package/.blueprint/features/feature_pipeline-insights/story-bottleneck-analysis.md +75 -0
  33. package/.blueprint/features/feature_pipeline-insights/story-failure-patterns.md +75 -0
  34. package/.blueprint/features/feature_pipeline-insights/story-json-output.md +75 -0
  35. package/.blueprint/features/feature_pipeline-insights/story-trend-analysis.md +78 -0
  36. package/.blueprint/features/feature_validate-command/FEATURE_SPEC.md +209 -0
  37. package/.blueprint/features/feature_validate-command/IMPLEMENTATION_PLAN.md +59 -0
  38. package/.blueprint/features/feature_validate-command/story-failure-output.md +61 -0
  39. package/.blueprint/features/feature_validate-command/story-node-version-check.md +52 -0
  40. package/.blueprint/features/feature_validate-command/story-run-validation.md +59 -0
  41. package/.blueprint/features/feature_validate-command/story-success-output.md +50 -0
  42. package/.blueprint/system_specification/SYSTEM_SPEC.md +248 -0
  43. package/README.md +174 -40
  44. package/SKILL.md +399 -74
  45. package/bin/cli.js +128 -20
  46. package/package.json +1 -1
  47. package/src/feedback.js +171 -0
  48. package/src/history.js +306 -0
  49. package/src/index.js +57 -2
  50. package/src/init.js +2 -6
  51. package/src/insights.js +504 -0
  52. package/src/retry.js +274 -0
  53. package/src/update.js +10 -2
  54. package/src/validate.js +172 -0
  55. package/src/skills.js +0 -93
@@ -0,0 +1,239 @@
1
+ # Feature Specification — Adaptive Retry
2
+
3
+ ## 1. Feature Intent
4
+ **Why this feature exists.**
5
+
6
+ The orchestr8 pipeline currently offers simple retry/skip/abort options when an agent fails (per SKILL.md:Error Handling). This feature introduces **intelligent retry logic** that learns from failure history to make smarter decisions about how to handle failures.
7
+
8
+ - **Problem being addressed:** Pipeline failures are handled uniformly regardless of context. Repeated failures at the same stage waste time with identical retry attempts.
9
+ - **User need:** Developers want the pipeline to adapt its retry approach based on what has historically worked, reducing manual intervention and improving success rates.
10
+ - **System alignment:** Per SYSTEM_SPEC.md:Section 6 (High-Level Lifecycle), recovery from failure is already supported. This feature enhances that capability by making recovery **adaptive** rather than static.
11
+
12
+ ---
13
+
14
+ ## 2. Scope
15
+ ### In Scope
16
+ - Retry configuration file management (`.claude/retry-config.json`)
17
+ - Failure pattern detection using existing history module (`src/history.js`)
18
+ - Strategy selection logic based on failure rates and attempt counts
19
+ - CLI commands for viewing and managing retry configuration
20
+ - Integration with pipeline error handling (advisory, not replacing user choice)
21
+
22
+ ### Out of Scope
23
+ - Automatic prompt rewriting (strategies describe *what* to do; agents execute)
24
+ - Modifications to agent specifications or core pipeline orchestration
25
+ - Machine learning or model-based prediction (rule-based only)
26
+ - Cross-feature failure correlation (each feature is handled independently)
27
+
28
+ ---
29
+
30
+ ## 3. Actors Involved
31
+
32
+ ### Human User
33
+ - Can view current retry configuration via CLI
34
+ - Can modify thresholds and enabled strategies
35
+ - Can reset configuration to defaults
36
+ - Retains final decision authority on retry/skip/abort during pipeline execution
37
+
38
+ ### Pipeline Orchestrator (SKILL.md implementation)
39
+ - Queries retry module when an agent fails
40
+ - Receives recommended strategy and displays it to user
41
+ - Applies strategy modifications to agent prompts when user accepts retry
42
+
43
+ ### History Module (src/history.js)
44
+ - Provides failure data for strategy calculation
45
+ - Is read-only from this feature's perspective
46
+
47
+ ### Insights Module (src/insights.js)
48
+ - Provides `analyzeFailures()` which calculates failure rates by stage
49
+ - Is read-only from this feature's perspective
50
+
51
+ ---
52
+
53
+ ## 4. Behaviour Overview
54
+
55
+ ### Happy Path: Strategy-Guided Retry
56
+ 1. Agent fails during pipeline execution
57
+ 2. Retry module queries history for recent failures at this stage
58
+ 3. Module calculates failure rate over recent window (default: last 10 runs)
59
+ 4. If failure rate > threshold (20%), recommend alternative strategy; otherwise recommend simple retry
60
+ 5. Display recommendation to user: "Recommended strategy: {strategy}. Retry with this approach? (y/n/abort)"
61
+ 6. If user accepts, modify agent prompt per strategy and retry
62
+ 7. Record outcome in history
63
+
64
+ ### Alternative: First Failure
65
+ - If this is the first recorded failure for a stage (no history), default to simple retry
66
+ - No prompt modification; standard retry behaviour
67
+
68
+ ### Alternative: Max Retries Exceeded
69
+ - After `maxRetries` (default 3) attempts at same stage for same feature, warn user
70
+ - User can still choose to retry, but recommendation becomes "abort or skip"
71
+
72
+ ### Alternative: User Declines Strategy
73
+ - User can override recommendation and choose different action (retry/skip/abort)
74
+ - System respects user choice; recommendation is advisory only
75
+
76
+ ---
77
+
78
+ ## 5. State & Lifecycle Interactions
79
+
80
+ ### States Affected
81
+ Per SYSTEM_SPEC.md:Section 6, pipeline states are: INIT, ALEX, CASS, NIGEL, CODEY_PLAN, CODEY_IMPLEMENT, COMMIT, COMPLETE, PAUSED, RESUME.
82
+
83
+ This feature operates within the **transition from any agent state to FAILED**:
84
+ - On agent error, before prompting user, retry module is consulted
85
+ - State remains in current stage during retry attempts
86
+ - State transitions to FAILED only on abort (unchanged from current behaviour)
87
+
88
+ ### Lifecycle Classification
89
+ - **State-constraining:** Does not create new states; constrains how transitions to FAILED are handled
90
+ - **Queue-modifying:** May update `failed` array with additional metadata (strategy attempted, attempt count)
91
+
92
+ ---
93
+
94
+ ## 6. Rules & Decision Logic
95
+
96
+ ### Rule 1: Calculate Stage Failure Rate
97
+ - **Description:** Determine how often a specific stage has failed in recent history
98
+ - **Inputs:** Stage name, history entries, window size (default 10)
99
+ - **Outputs:** Failure rate (0.0 to 1.0)
100
+ - **Type:** Deterministic
101
+
102
+ ```
103
+ failureRate = failedRunsAtStage / totalRecentRuns
104
+ ```
105
+
106
+ ### Rule 2: Select Retry Strategy
107
+ - **Description:** Choose appropriate strategy based on failure rate and attempt count
108
+ - **Inputs:** Stage name, current attempt number, failure rate, configured strategies
109
+ - **Outputs:** Recommended strategy name
110
+ - **Type:** Deterministic with precedence
111
+
112
+ ```
113
+ if (attemptCount > maxRetries) → "abort-recommended"
114
+ if (failureRate > highFailureThreshold) → alternativeStrategy[stage][attemptCount]
115
+ else → "retry"
116
+ ```
117
+
118
+ ### Rule 3: Apply Strategy to Prompt
119
+ - **Description:** Modify agent prompt based on selected strategy
120
+ - **Inputs:** Original prompt, strategy name, stage context
121
+ - **Outputs:** Modified prompt (or original if strategy is "retry")
122
+ - **Type:** Deterministic
123
+
124
+ Strategy applications:
125
+ | Strategy | Prompt Modification |
126
+ |----------|-------------------|
127
+ | `retry` | No modification |
128
+ | `simplify-prompt` | Add: "Focus on core requirements only. Skip edge cases and optional sections." |
129
+ | `reduce-stories` | Add: "Write only the 2-3 most critical user stories. Defer others to follow-up." |
130
+ | `simplify-tests` | Add: "Write only happy-path tests for each AC. Skip edge cases." |
131
+ | `add-context` | Prepend relevant sections from previous stage outputs |
132
+ | `incremental` | Add: "Implement one test at a time. Stop and report after each." |
133
+ | `rollback` | Execute `git checkout -- .` on implementation files before retry |
134
+
135
+ ### Rule 4: Persist Configuration
136
+ - **Description:** Configuration survives across sessions
137
+ - **Inputs:** User modifications via CLI
138
+ - **Outputs:** Updated `.claude/retry-config.json`
139
+ - **Type:** Deterministic (file write)
140
+
141
+ ---
142
+
143
+ ## 7. Dependencies
144
+
145
+ ### System Components
146
+ - **src/history.js:** `readHistoryFile()` for failure data access (read-only dependency)
147
+ - **src/insights.js:** `analyzeFailures()` for failure pattern analysis (read-only dependency)
148
+ - **src/orchestrator.js:** Queue management; retry module will export functions for orchestrator to call
149
+ - **SKILL.md:** Error handling section will reference retry module recommendations
150
+
151
+ ### File Dependencies
152
+ - **`.claude/pipeline-history.json`:** Must exist with valid structure for accurate analysis
153
+ - **`.claude/retry-config.json`:** Created on first use if not present; gitignored
154
+
155
+ ### Operational Dependencies
156
+ - History module must be functional (feature degrades gracefully if corrupted)
157
+ - File system write access for configuration persistence
158
+
159
+ ---
160
+
161
+ ## 8. Non-Functional Considerations
162
+
163
+ ### Performance
164
+ - Strategy calculation should complete in <100ms (simple arithmetic on in-memory data)
165
+ - No blocking I/O during recommendation phase beyond initial history read
166
+
167
+ ### Resilience
168
+ - If history file is corrupted, default to simple retry (no strategy recommendation)
169
+ - If config file is corrupted or missing, use hardcoded defaults
170
+ - All failures in retry module should be caught and logged, not propagated
171
+
172
+ ### Audit/Logging
173
+ - Retry attempts and strategies used should be recorded in history entries
174
+ - New history fields: `retryAttempts`, `strategiesUsed[]`
175
+
176
+ ### Maintainability
177
+ - Strategies should be configurable without code changes
178
+ - New strategies can be added by updating config schema and `applyStrategy()` function
179
+
180
+ ---
181
+
182
+ ## 9. Assumptions & Open Questions
183
+
184
+ ### Assumptions
185
+ - ASSUMPTION: History file format is stable and matches `src/history.js` expectations
186
+ - ASSUMPTION: Pipeline execution is single-threaded; no concurrent retry decisions needed
187
+ - ASSUMPTION: User preference is final; recommendations are advisory only
188
+ - ASSUMPTION: Stage names match exactly between history, insights, and retry module
189
+
190
+ ### Open Questions
191
+ - Should strategy history be factored into recommendations? (e.g., "simplify-prompt worked last time")
192
+ - Should there be a "never retry" option for specific stages?
193
+ - How should retry configuration interact with `--pause-after` flags?
194
+
195
+ ---
196
+
197
+ ## 10. Impact on System Specification
198
+
199
+ ### Reinforces Existing Assumptions
200
+ - Per SYSTEM_SPEC.md:Section 8 (Cross-Cutting Concerns), failure handling already "offers retry, skip, abort"
201
+ - This feature enhances rather than replaces that pattern
202
+ - Per SYSTEM_SPEC.md:Section 7 (Governing Rules), "recovery-safe" queue behaviour is maintained
203
+
204
+ ### Stretches Existing Assumptions
205
+ - History module was designed for reporting/insights; now becomes an active input to pipeline decisions
206
+ - This elevates history from "observability" to "operational dependency"
207
+
208
+ ### No Contradictions Identified
209
+ - Feature operates within existing error handling flow
210
+ - Does not modify agent specifications or pipeline sequence
211
+ - Preserves user authority over final decisions
212
+
213
+ ---
214
+
215
+ ## 11. Handover to BA (Cass)
216
+
217
+ ### Story Themes
218
+ 1. **Configuration Management:** User can view, modify, and reset retry configuration
219
+ 2. **Strategy Recommendation:** Pipeline recommends retry strategy based on failure history
220
+ 3. **Prompt Modification:** Retry strategies modify agent prompts to improve success probability
221
+ 4. **History Integration:** Retry decisions are informed by historical failure patterns
222
+
223
+ ### Expected Story Boundaries
224
+ - CLI commands (`retry-config`, `retry-config reset`) as separate story
225
+ - Strategy selection logic as separate story
226
+ - Strategy application (prompt modification) as separate story
227
+ - Graceful degradation (corrupted history/config) as non-functional story or AC within above
228
+
229
+ ### Areas Needing Careful Framing
230
+ - User choice must remain paramount; stories should emphasize "recommendation" not "automation"
231
+ - The "rollback" strategy involves destructive action (`git checkout`); needs explicit user confirmation
232
+ - Incremental strategy changes Codey's execution model; may need coordination with Codey's agent spec
233
+
234
+ ---
235
+
236
+ ## 12. Change Log (Feature-Level)
237
+ | Date | Change | Reason | Raised By |
238
+ |------|--------|--------|-----------|
239
+ | 2026-02-24 | Initial feature specification | New feature request | Alex |
@@ -0,0 +1,48 @@
1
+ # Implementation Plan - Adaptive Retry
2
+
3
+ ## Summary
4
+
5
+ This feature adds intelligent retry logic to the orchestr8 pipeline via a new `src/retry.js` module. The module calculates failure rates from history, recommends strategies based on configurable thresholds, and applies prompt modifications when users accept recommended strategies. The implementation integrates with existing `src/history.js` for failure data and provides CLI commands for configuration management.
6
+
7
+ ## Files to Create/Modify
8
+
9
+ | Path | Action | Purpose |
10
+ |------|--------|---------|
11
+ | `src/retry.js` | Create | Core retry logic: config management, failure rate calculation, strategy recommendation, prompt modification |
12
+ | `src/index.js` | Modify | Export retry module functions |
13
+ | `bin/cli.js` | Modify | Add `retry-config` CLI command with `set` and `reset` subcommands |
14
+
15
+ ## Implementation Steps
16
+
17
+ 1. **Create `src/retry.js` with config management** - Implement `getDefaultConfig()`, `readConfig()`, `writeConfig()`, `resetConfig()` functions for `.claude/retry-config.json`
18
+
19
+ 2. **Add failure rate calculation** - Implement `calculateFailureRate(stage, history, windowSize)` that returns failure rate (0-1) for a stage over the recent window
20
+
21
+ 3. **Add strategy recommendation logic** - Implement `recommendStrategy(stage, attemptCount, failureRate, config)` returning strategy name or "abort-recommended"
22
+
23
+ 4. **Add prompt modification** - Implement `applyStrategy(strategy, originalPrompt)` that returns modified prompt based on strategy
24
+
25
+ 5. **Add graceful degradation** - Wrap all file operations in try/catch; return defaults on corrupted/missing files; log warnings
26
+
27
+ 6. **Add CLI commands to `bin/cli.js`** - Add `retry-config` (view), `retry-config set <key> <value>`, `retry-config reset` commands
28
+
29
+ 7. **Export functions in `src/index.js`** - Export `readConfig`, `writeConfig`, `resetConfig`, `calculateFailureRate`, `recommendStrategy`, `applyStrategy`
30
+
31
+ 8. **Run tests and iterate** - Execute `node --test test/feature_adaptive-retry.test.js` after each step; fix failures
32
+
33
+ ## Key Functions
34
+
35
+ - `getDefaultConfig()` - Returns hardcoded default configuration object
36
+ - `readConfig()` - Reads `.claude/retry-config.json` or returns defaults if missing/corrupted
37
+ - `writeConfig(config)` - Writes config to file, creates `.claude/` directory if needed
38
+ - `resetConfig()` - Replaces config with defaults
39
+ - `calculateFailureRate(stage, history, windowSize)` - Calculates failure rate for stage over window
40
+ - `recommendStrategy(stage, attemptCount, failureRate, config)` - Returns recommended strategy name
41
+ - `applyStrategy(strategy, originalPrompt)` - Returns modified prompt or original if "retry"
42
+ - `shouldRetry(stage, featureSlug, history, config, attemptCount)` - Orchestrator entry point; returns recommendation object
43
+
44
+ ## Risks/Questions
45
+
46
+ - **Rollback strategy is destructive**: Per story-prompt-modification.md:AC-6, `git checkout -- .` requires explicit confirmation before execution; implementation must surface this warning
47
+ - **History module dependency**: If `readHistoryFile()` returns `{ error: 'corrupted' }`, module must degrade gracefully per test T-SH-2
48
+ - **Strategy escalation boundary**: When `attemptCount - 1` exceeds strategy list length, use last strategy before switching to "abort-recommended"
@@ -0,0 +1,85 @@
1
+ # Story — Prompt Modification
2
+
3
+ ## User Story
4
+
5
+ As a **developer using orchestr8**, I want the **pipeline to modify agent prompts based on the selected retry strategy** so that **retry attempts have a better chance of success by adjusting the agent's instructions**.
6
+
7
+ ---
8
+
9
+ ## Context / Scope
10
+
11
+ - Per FEATURE_SPEC.md:Section 6 (Rule 3), strategies apply specific modifications to agent prompts
12
+ - Per FEATURE_SPEC.md:Section 11 (Areas Needing Careful Framing), the "rollback" strategy requires explicit user confirmation
13
+ - Per FEATURE_SPEC.md:Section 2 (Out of Scope), automatic prompt rewriting is not supported; strategies describe what to do
14
+ - Per SYSTEM_SPEC.md:Section 7 (Agent Rules), agents must not make silent changes
15
+
16
+ ---
17
+
18
+ ## Acceptance Criteria
19
+
20
+ **AC-1 — Apply "simplify-prompt" strategy**
21
+ - Given the user accepts a retry with the "simplify-prompt" strategy,
22
+ - When the agent is re-invoked,
23
+ - Then the prompt is modified to append: "Focus on core requirements only. Skip edge cases and optional sections.",
24
+ - And the modification is visible in the agent's task context.
25
+
26
+ **AC-2 — Apply "reduce-stories" strategy**
27
+ - Given the user accepts a retry with the "reduce-stories" strategy,
28
+ - When Cass is re-invoked,
29
+ - Then the prompt is modified to append: "Write only the 2-3 most critical user stories. Defer others to follow-up.",
30
+ - And the deferred stories are noted in the summary.
31
+
32
+ **AC-3 — Apply "simplify-tests" strategy**
33
+ - Given the user accepts a retry with the "simplify-tests" strategy,
34
+ - When Nigel is re-invoked,
35
+ - Then the prompt is modified to append: "Write only happy-path tests for each AC. Skip edge cases.",
36
+ - And edge case tests are explicitly deferred.
37
+
38
+ **AC-4 — Apply "add-context" strategy**
39
+ - Given the user accepts a retry with the "add-context" strategy,
40
+ - When the agent is re-invoked,
41
+ - Then relevant sections from previous stage outputs are prepended to the prompt,
42
+ - And the source of added context is indicated (e.g., "Context from FEATURE_SPEC.md:Section 4").
43
+
44
+ **AC-5 — Apply "incremental" strategy**
45
+ - Given the user accepts a retry with the "incremental" strategy,
46
+ - When Codey is re-invoked,
47
+ - Then the prompt is modified to append: "Implement one test at a time. Stop and report after each.",
48
+ - And each implementation step produces a discrete output before continuing.
49
+
50
+ **AC-6 — Apply "rollback" strategy with confirmation**
51
+ - Given the user accepts a retry with the "rollback" strategy,
52
+ - When the user confirms the destructive action,
53
+ - Then `git checkout -- .` is executed on implementation files before retry,
54
+ - And a confirmation prompt warns: "This will discard uncommitted changes to implementation files. Proceed? (y/n)".
55
+
56
+ **AC-7 — No modification for "retry" strategy**
57
+ - Given the user accepts a simple "retry" (no strategy modification),
58
+ - When the agent is re-invoked,
59
+ - Then the original prompt is used without any modification,
60
+ - And the retry proceeds identically to the first attempt.
61
+
62
+ ---
63
+
64
+ ## Strategy Reference Table
65
+
66
+ Per FEATURE_SPEC.md:Section 6 (Rule 3):
67
+
68
+ | Strategy | Modification Applied |
69
+ |----------|---------------------|
70
+ | `retry` | No modification |
71
+ | `simplify-prompt` | Append: "Focus on core requirements only. Skip edge cases and optional sections." |
72
+ | `reduce-stories` | Append: "Write only the 2-3 most critical user stories. Defer others to follow-up." |
73
+ | `simplify-tests` | Append: "Write only happy-path tests for each AC. Skip edge cases." |
74
+ | `add-context` | Prepend relevant sections from previous stage outputs |
75
+ | `incremental` | Append: "Implement one test at a time. Stop and report after each." |
76
+ | `rollback` | Execute `git checkout -- .` on implementation files before retry |
77
+
78
+ ---
79
+
80
+ ## Out of Scope
81
+
82
+ - Automatic prompt rewriting based on error analysis (per FEATURE_SPEC.md:Section 2)
83
+ - Custom user-defined prompt modifications (strategies are predefined)
84
+ - Modifying agent specifications themselves (per FEATURE_SPEC.md:Section 2)
85
+ - Combining multiple strategies in a single retry (one strategy per attempt)
@@ -0,0 +1,89 @@
1
+ # Story — Retry Configuration Management
2
+
3
+ ## User Story
4
+
5
+ As a **developer using orchestr8**, I want to **view, modify, and reset retry configuration** so that **I can customize how the pipeline handles failures based on my project's needs**.
6
+
7
+ ---
8
+
9
+ ## Context / Scope
10
+
11
+ - Per FEATURE_SPEC.md:Section 2, retry configuration is managed via `.claude/retry-config.json`
12
+ - Per FEATURE_SPEC.md:Section 3 (Actors), users retain final decision authority and can modify thresholds
13
+ - Per FEATURE_SPEC.md:Section 11 (Handover), this story covers CLI commands for `retry-config` and `retry-config reset`
14
+ - Configuration file is gitignored (per FEATURE_SPEC.md:Section 7)
15
+
16
+ ---
17
+
18
+ ## Acceptance Criteria
19
+
20
+ **AC-1 — View current configuration**
21
+ - Given the retry configuration file exists at `.claude/retry-config.json`,
22
+ - When I run `orchestr8 retry-config`,
23
+ - Then the current configuration is displayed in a readable format showing thresholds, window size, max retries, and enabled strategies.
24
+
25
+ **AC-2 — View defaults when no configuration exists**
26
+ - Given no retry configuration file exists,
27
+ - When I run `orchestr8 retry-config`,
28
+ - Then the hardcoded default configuration is displayed,
29
+ - And a message indicates "Using default configuration (no config file found)".
30
+
31
+ **AC-3 — Modify configuration value**
32
+ - Given I run `orchestr8 retry-config set <key> <value>` (e.g., `orchestr8 retry-config set maxRetries 5`),
33
+ - When the key is valid and value is of correct type,
34
+ - Then the configuration file is updated with the new value,
35
+ - And a confirmation message shows the updated setting.
36
+
37
+ **AC-4 — Reject invalid configuration key**
38
+ - Given I run `orchestr8 retry-config set <invalidKey> <value>`,
39
+ - When the key is not recognized,
40
+ - Then an error message is displayed listing valid configuration keys,
41
+ - And no file modification occurs.
42
+
43
+ **AC-5 — Reset configuration to defaults**
44
+ - Given the retry configuration file exists with custom values,
45
+ - When I run `orchestr8 retry-config reset`,
46
+ - Then the configuration file is replaced with default values,
47
+ - And a confirmation message indicates "Retry configuration reset to defaults".
48
+
49
+ **AC-6 — Create configuration file on first modification**
50
+ - Given no retry configuration file exists,
51
+ - When I run `orchestr8 retry-config set <key> <value>`,
52
+ - Then the configuration file is created with default values plus the specified modification,
53
+ - And the file is created in `.claude/retry-config.json`.
54
+
55
+ **AC-7 — Handle corrupted configuration gracefully**
56
+ - Given the retry configuration file exists but contains invalid JSON,
57
+ - When I run `orchestr8 retry-config`,
58
+ - Then an error message is displayed indicating the file is corrupted,
59
+ - And a suggestion to run `orchestr8 retry-config reset` is shown.
60
+
61
+ ---
62
+
63
+ ## Configuration Schema
64
+
65
+ Per FEATURE_SPEC.md:Section 6 (Rules), default configuration includes:
66
+
67
+ ```json
68
+ {
69
+ "maxRetries": 3,
70
+ "windowSize": 10,
71
+ "highFailureThreshold": 0.2,
72
+ "strategies": {
73
+ "alex": ["simplify-prompt", "add-context"],
74
+ "cass": ["reduce-stories", "simplify-prompt"],
75
+ "nigel": ["simplify-tests", "add-context"],
76
+ "codey-plan": ["add-context", "simplify-prompt"],
77
+ "codey-implement": ["incremental", "rollback"]
78
+ }
79
+ }
80
+ ```
81
+
82
+ ---
83
+
84
+ ## Out of Scope
85
+
86
+ - Modifying strategy definitions themselves (strategies are code, not config)
87
+ - Per-feature configuration overrides (configuration is global)
88
+ - Configuration UI/interactive editor (CLI only)
89
+ - Validating strategy effectiveness (config is just data; strategy logic is separate)
@@ -0,0 +1,98 @@
1
+ # Story — Should Retry Decision Logic
2
+
3
+ ## User Story
4
+
5
+ As a **developer using orchestr8**, I want the **pipeline to intelligently decide whether to recommend retrying** so that **I receive useful guidance based on attempt count, failure history, and system state**.
6
+
7
+ ---
8
+
9
+ ## Context / Scope
10
+
11
+ - Per FEATURE_SPEC.md:Section 4 (Happy Path), retry module is consulted when an agent fails
12
+ - Per FEATURE_SPEC.md:Section 5 (State & Lifecycle), this operates within the transition from any agent state to FAILED
13
+ - Per FEATURE_SPEC.md:Section 8 (Resilience), graceful degradation is required when history/config is corrupted
14
+ - Per SYSTEM_SPEC.md:Section 8 (Failure Handling), each agent spawn currently offers retry, skip, abort
15
+
16
+ ---
17
+
18
+ ## Acceptance Criteria
19
+
20
+ **AC-1 — Consult retry module on agent failure**
21
+ - Given an agent fails during pipeline execution,
22
+ - When the error handling flow is triggered,
23
+ - Then the retry module is consulted before displaying options to the user,
24
+ - And the module returns a recommendation (strategy name or "abort-recommended").
25
+
26
+ **AC-2 — Record retry attempts in history**
27
+ - Given the user chooses to retry (with or without strategy),
28
+ - When the retry completes (success or failure),
29
+ - Then the outcome is recorded in `.claude/pipeline-history.json`,
30
+ - And the record includes: `retryAttempts` count and `strategiesUsed[]` array.
31
+
32
+ **AC-3 — Track attempt count per stage per feature**
33
+ - Given an agent fails and user retries,
34
+ - When tracking retry state,
35
+ - Then the attempt count is incremented for the current stage and feature combination,
36
+ - And the count persists across retries within the same pipeline run.
37
+
38
+ **AC-4 — Degrade gracefully with corrupted history**
39
+ - Given the history file at `.claude/pipeline-history.json` is corrupted or missing,
40
+ - When the retry module is consulted,
41
+ - Then the module defaults to simple retry recommendation,
42
+ - And a warning is logged: "History file unavailable; defaulting to simple retry".
43
+
44
+ **AC-5 — Degrade gracefully with corrupted configuration**
45
+ - Given the configuration file at `.claude/retry-config.json` is corrupted or missing,
46
+ - When the retry module calculates strategy,
47
+ - Then hardcoded defaults are used for thresholds and strategies,
48
+ - And a warning is logged: "Config file unavailable; using default settings".
49
+
50
+ **AC-6 — Preserve state transitions**
51
+ - Given a retry is in progress,
52
+ - When the retry attempt completes successfully,
53
+ - Then the pipeline continues to the next stage as normal,
54
+ - And the state transitions per SYSTEM_SPEC.md:Section 6 (e.g., CASS to NIGEL).
55
+
56
+ **AC-7 — Support abort without retry**
57
+ - Given the user is shown retry options,
58
+ - When the user selects "abort",
59
+ - Then the feature is moved to the failed list in the queue,
60
+ - And no retry is attempted,
61
+ - And the failure is recorded with reason "user-aborted".
62
+
63
+ ---
64
+
65
+ ## Integration Points
66
+
67
+ Per FEATURE_SPEC.md:Section 7 (Dependencies):
68
+
69
+ - **src/history.js:** `readHistoryFile()` provides failure data
70
+ - **src/insights.js:** `analyzeFailures()` calculates failure rates by stage
71
+ - **src/orchestrator.js:** Queue management receives retry module functions
72
+ - **SKILL.md:** Error handling section references retry recommendations
73
+
74
+ ---
75
+
76
+ ## History Record Schema
77
+
78
+ Per FEATURE_SPEC.md:Section 8 (Audit/Logging), new fields in history entries:
79
+
80
+ ```json
81
+ {
82
+ "featureSlug": "adaptive-retry",
83
+ "stage": "cass",
84
+ "outcome": "success",
85
+ "timestamp": "2026-02-24T10:30:00Z",
86
+ "retryAttempts": 2,
87
+ "strategiesUsed": ["retry", "reduce-stories"]
88
+ }
89
+ ```
90
+
91
+ ---
92
+
93
+ ## Out of Scope
94
+
95
+ - Automatic retry without user prompt (user always has final say)
96
+ - Retry across different features (each feature's retry state is independent)
97
+ - Persistent retry state across pipeline invocations (state is per-run only; history is persistent)
98
+ - Batch retry of multiple failed features (single-feature focus per SYSTEM_SPEC.md:Section 10)
@@ -0,0 +1,85 @@
1
+ # Story — Strategy Recommendation
2
+
3
+ ## User Story
4
+
5
+ As a **developer using orchestr8**, I want the **pipeline to recommend a retry strategy based on failure history** so that **I can make informed decisions about how to handle failures more effectively**.
6
+
7
+ ---
8
+
9
+ ## Context / Scope
10
+
11
+ - Per FEATURE_SPEC.md:Section 4 (Behaviour Overview), recommendations are advisory only; user retains final choice
12
+ - Per FEATURE_SPEC.md:Section 6 (Rule 1 & Rule 2), strategy selection is based on failure rate calculation
13
+ - Per FEATURE_SPEC.md:Section 3 (Actors), the History Module provides failure data (read-only)
14
+ - Per FEATURE_SPEC.md:Section 7 (Dependencies), uses `src/history.js` and `src/insights.js`
15
+
16
+ ---
17
+
18
+ ## Acceptance Criteria
19
+
20
+ **AC-1 — Calculate failure rate from history**
21
+ - Given an agent fails during pipeline execution,
22
+ - When the retry module is consulted,
23
+ - Then the failure rate is calculated as `failedRunsAtStage / totalRecentRuns` over the configured window (default 10 runs),
24
+ - And the calculation uses data from `.claude/pipeline-history.json`.
25
+
26
+ **AC-2 — Recommend simple retry for low failure rate**
27
+ - Given the calculated failure rate is at or below the threshold (default 20%),
28
+ - When displaying retry options to the user,
29
+ - Then the recommendation is "retry" with no prompt modification suggested,
30
+ - And the message format is: "Recommended: Simple retry. Retry? (y/n/skip/abort)".
31
+
32
+ **AC-3 — Recommend alternative strategy for high failure rate**
33
+ - Given the calculated failure rate exceeds the threshold (default 20%),
34
+ - When displaying retry options to the user,
35
+ - Then an alternative strategy from the stage's strategy list is recommended,
36
+ - And the message format is: "Recommended strategy: {strategyName}. Retry with this approach? (y/apply/skip/abort)".
37
+
38
+ **AC-4 — Escalate strategy on subsequent attempts**
39
+ - Given a retry attempt has already been made at the current stage,
40
+ - When another failure occurs and the user is prompted again,
41
+ - Then the next strategy in the stage's strategy list is recommended,
42
+ - And if all strategies have been exhausted, "abort-recommended" is suggested.
43
+
44
+ **AC-5 — Default to simple retry with no history**
45
+ - Given this is the first recorded run or no history exists for the current stage,
46
+ - When an agent fails,
47
+ - Then the recommendation is "retry" (simple retry),
48
+ - And a note indicates "No failure history for this stage; defaulting to simple retry".
49
+
50
+ **AC-6 — Warn when max retries exceeded**
51
+ - Given the current attempt count exceeds `maxRetries` (default 3) for the same feature and stage,
52
+ - When displaying options to the user,
53
+ - Then the recommendation is "abort-recommended" or "skip-recommended",
54
+ - And a warning is displayed: "Max retries ({count}) exceeded for {stage}. Consider skipping or aborting."
55
+
56
+ **AC-7 — Display recommendation without forcing choice**
57
+ - Given a strategy recommendation is displayed,
58
+ - When the user selects a different option (e.g., chooses "skip" when "retry" was recommended),
59
+ - Then the user's choice is respected,
60
+ - And no error or warning is shown for overriding the recommendation.
61
+
62
+ ---
63
+
64
+ ## Session State
65
+
66
+ During pipeline execution, retry state is tracked:
67
+
68
+ ```js
69
+ retryState = {
70
+ stage: 'cass',
71
+ featureSlug: 'adaptive-retry',
72
+ attemptCount: 2,
73
+ strategiesUsed: ['retry', 'reduce-stories'],
74
+ failureRate: 0.3
75
+ }
76
+ ```
77
+
78
+ ---
79
+
80
+ ## Out of Scope
81
+
82
+ - Machine learning or predictive models for strategy selection (per FEATURE_SPEC.md:Section 2)
83
+ - Cross-feature failure correlation (per FEATURE_SPEC.md:Section 2)
84
+ - Automatic retry without user confirmation (user choice is paramount)
85
+ - Modifying the agent prompts (that is story-prompt-modification)