orchestr8 2.5.0 → 2.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.blueprint/agents/AGENT_BA_CASS.md +42 -19
- package/.blueprint/agents/AGENT_DEVELOPER_CODEY.md +42 -38
- package/.blueprint/agents/AGENT_SPECIFICATION_ALEX.md +45 -0
- package/.blueprint/agents/AGENT_TESTER_NIGEL.md +42 -21
- package/.blueprint/features/feature_adaptive-retry/FEATURE_SPEC.md +239 -0
- package/.blueprint/features/feature_adaptive-retry/IMPLEMENTATION_PLAN.md +48 -0
- package/.blueprint/features/feature_adaptive-retry/story-prompt-modification.md +85 -0
- package/.blueprint/features/feature_adaptive-retry/story-retry-config.md +89 -0
- package/.blueprint/features/feature_adaptive-retry/story-should-retry.md +98 -0
- package/.blueprint/features/feature_adaptive-retry/story-strategy-recommendation.md +85 -0
- package/.blueprint/features/feature_agent-guardrails/FEATURE_SPEC.md +328 -0
- package/.blueprint/features/feature_agent-guardrails/IMPLEMENTATION_PLAN.md +90 -0
- package/.blueprint/features/feature_agent-guardrails/story-citation-requirements.md +50 -0
- package/.blueprint/features/feature_agent-guardrails/story-confidentiality.md +50 -0
- package/.blueprint/features/feature_agent-guardrails/story-escalation-protocol.md +55 -0
- package/.blueprint/features/feature_agent-guardrails/story-source-restrictions.md +50 -0
- package/.blueprint/features/feature_feedback-loop/FEATURE_SPEC.md +347 -0
- package/.blueprint/features/feature_feedback-loop/IMPLEMENTATION_PLAN.md +71 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-collection.md +63 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-config.md +61 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-insights.md +63 -0
- package/.blueprint/features/feature_feedback-loop/story-quality-gates.md +57 -0
- package/.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md +239 -0
- package/.blueprint/features/feature_pipeline-history/IMPLEMENTATION_PLAN.md +71 -0
- package/.blueprint/features/feature_pipeline-history/story-clear-history.md +73 -0
- package/.blueprint/features/feature_pipeline-history/story-display-history.md +75 -0
- package/.blueprint/features/feature_pipeline-history/story-record-execution.md +76 -0
- package/.blueprint/features/feature_pipeline-history/story-show-statistics.md +85 -0
- package/.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md +288 -0
- package/.blueprint/features/feature_pipeline-insights/IMPLEMENTATION_PLAN.md +65 -0
- package/.blueprint/features/feature_pipeline-insights/story-anomaly-detection.md +71 -0
- package/.blueprint/features/feature_pipeline-insights/story-bottleneck-analysis.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-failure-patterns.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-json-output.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-trend-analysis.md +78 -0
- package/.blueprint/features/feature_validate-command/FEATURE_SPEC.md +209 -0
- package/.blueprint/features/feature_validate-command/IMPLEMENTATION_PLAN.md +59 -0
- package/.blueprint/features/feature_validate-command/story-failure-output.md +61 -0
- package/.blueprint/features/feature_validate-command/story-node-version-check.md +52 -0
- package/.blueprint/features/feature_validate-command/story-run-validation.md +59 -0
- package/.blueprint/features/feature_validate-command/story-success-output.md +50 -0
- package/.blueprint/system_specification/SYSTEM_SPEC.md +248 -0
- package/README.md +182 -38
- package/SKILL.md +333 -23
- package/bin/cli.js +128 -20
- package/package.json +2 -2
- package/src/feedback.js +171 -0
- package/src/history.js +306 -0
- package/src/index.js +57 -2
- package/src/init.js +2 -6
- package/src/insights.js +504 -0
- package/src/retry.js +274 -0
- package/src/validate.js +172 -0
- package/src/skills.js +0 -93
|
@@ -380,23 +380,46 @@ You have done your job well when:
|
|
|
380
380
|
|
|
381
381
|
---
|
|
382
382
|
|
|
383
|
-
##
|
|
383
|
+
## Guardrails
|
|
384
|
+
|
|
385
|
+
### Allowed Sources
|
|
386
|
+
You may use ONLY information from these sources:
|
|
387
|
+
- System specification (`.blueprint/system_specification/SYSTEM_SPEC.md`)
|
|
388
|
+
- Feature specifications (`.blueprint/features/*/FEATURE_SPEC.md`)
|
|
389
|
+
- User stories (`story-*.md`) and test artifacts (`test-spec.md`, `*.test.js`)
|
|
390
|
+
- Implementation code in the project
|
|
391
|
+
- Business context (`.business_context/*`)
|
|
392
|
+
- Templates (`.blueprint/templates/*`) and agent specifications
|
|
393
|
+
|
|
394
|
+
### Prohibited Sources
|
|
395
|
+
Do not use:
|
|
396
|
+
- Social media, forums, blog posts, or external APIs
|
|
397
|
+
- Training data for domain facts—do not invent business rules
|
|
398
|
+
- External project or company references by name
|
|
399
|
+
|
|
400
|
+
### Citation Requirements
|
|
401
|
+
- Cite sources using: "Per [filename]: [claim]" or "[filename:section] states..."
|
|
402
|
+
- Use section-level citations where feasible (e.g., "story-login.md:AC-3")
|
|
403
|
+
- Reference `.business_context/` files for domain definitions
|
|
404
|
+
- Maintain a traceable chain: downstream artifacts cite upstream sources
|
|
405
|
+
|
|
406
|
+
### Assumptions vs Facts
|
|
407
|
+
- Label assumptions explicitly: "ASSUMPTION: [statement]" or "NOTE: Assuming..."
|
|
408
|
+
- Distinguish clearly between cited facts and assumptions
|
|
409
|
+
- Do not guess—state "This information is not available in the provided inputs"
|
|
410
|
+
|
|
411
|
+
### Confidentiality
|
|
412
|
+
- Do not reproduce `.business_context/` content verbatim; summarise or use generic descriptions
|
|
413
|
+
- Do not reference external entities, companies, or projects by name
|
|
414
|
+
- Do not use external services that would expose project data
|
|
415
|
+
- Outputs must be self-contained and understandable without access to confidential sources
|
|
416
|
+
|
|
417
|
+
### Escalation Protocol
|
|
418
|
+
Escalate to the user when:
|
|
419
|
+
- Critical information is missing and cannot be safely assumed
|
|
420
|
+
- Inputs are ambiguous with multiple possible interpretations—list options and ask for clarification
|
|
421
|
+
- Source documents conflict—cite both sources and request resolution
|
|
422
|
+
- Output would require violating confidentiality constraints
|
|
423
|
+
|
|
424
|
+
When escalation is not warranted, you may proceed with an explicit assumption labelled as such.
|
|
384
425
|
|
|
385
|
-
You have access to the following skills that can help with your work:
|
|
386
|
-
|
|
387
|
-
### `/user-story-writing`
|
|
388
|
-
|
|
389
|
-
**When to use:** When creating user stories and acceptance criteria.
|
|
390
|
-
|
|
391
|
-
**What it provides:**
|
|
392
|
-
- User story template with INVEST criteria
|
|
393
|
-
- Acceptance criteria examples in Given/When/Then format
|
|
394
|
-
- Story refinement process and quality gates
|
|
395
|
-
- Story splitting strategies for large features
|
|
396
|
-
- Estimation guidance
|
|
397
|
-
|
|
398
|
-
**How to invoke:** Use `/user-story-writing` when you need guidance on structuring a user story or acceptance criteria.
|
|
399
|
-
|
|
400
|
-
**Location:** `.agents/skills/user-story-writing/SKILL.md`
|
|
401
|
-
|
|
402
|
-
---
|
|
@@ -423,41 +423,45 @@ By following this guide, Codey and Nigel can work together in a tight loop: Nige
|
|
|
423
423
|
|
|
424
424
|
---
|
|
425
425
|
|
|
426
|
-
##
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
-
|
|
440
|
-
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
-
|
|
456
|
-
-
|
|
457
|
-
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
426
|
+
## Guardrails
|
|
427
|
+
|
|
428
|
+
### Allowed Sources
|
|
429
|
+
You may use ONLY information from these sources:
|
|
430
|
+
- System specification (`.blueprint/system_specification/SYSTEM_SPEC.md`)
|
|
431
|
+
- Feature specifications (`.blueprint/features/*/FEATURE_SPEC.md`)
|
|
432
|
+
- User stories (`story-*.md`) and test artifacts (`test-spec.md`, `*.test.js`)
|
|
433
|
+
- Implementation code in the project
|
|
434
|
+
- Business context (`.business_context/*`)
|
|
435
|
+
- Templates (`.blueprint/templates/*`) and agent specifications
|
|
436
|
+
|
|
437
|
+
### Prohibited Sources
|
|
438
|
+
Do not use:
|
|
439
|
+
- Social media, forums, blog posts, or external APIs
|
|
440
|
+
- Training data for domain facts—do not invent business rules
|
|
441
|
+
- External project or company references by name
|
|
442
|
+
|
|
443
|
+
### Citation Requirements
|
|
444
|
+
- Cite sources using: "Per [filename]: [claim]" or "[filename:section] states..."
|
|
445
|
+
- Use section-level citations where feasible (e.g., "story-login.md:AC-3")
|
|
446
|
+
- Reference `.business_context/` files for domain definitions
|
|
447
|
+
- Maintain a traceable chain: downstream artifacts cite upstream sources
|
|
448
|
+
|
|
449
|
+
### Assumptions vs Facts
|
|
450
|
+
- Label assumptions explicitly: "ASSUMPTION: [statement]" or "NOTE: Assuming..."
|
|
451
|
+
- Distinguish clearly between cited facts and assumptions
|
|
452
|
+
- Do not guess—state "This information is not available in the provided inputs"
|
|
453
|
+
|
|
454
|
+
### Confidentiality
|
|
455
|
+
- Do not reproduce `.business_context/` content verbatim; summarise or use generic descriptions
|
|
456
|
+
- Do not reference external entities, companies, or projects by name
|
|
457
|
+
- Do not use external services that would expose project data
|
|
458
|
+
- Outputs must be self-contained and understandable without access to confidential sources
|
|
459
|
+
|
|
460
|
+
### Escalation Protocol
|
|
461
|
+
Escalate to the user when:
|
|
462
|
+
- Critical information is missing and cannot be safely assumed
|
|
463
|
+
- Inputs are ambiguous with multiple possible interpretations—list options and ask for clarification
|
|
464
|
+
- Source documents conflict—cite both sources and request resolution
|
|
465
|
+
- Output would require violating confidentiality constraints
|
|
466
|
+
|
|
467
|
+
When escalation is not warranted, you may proceed with an explicit assumption labelled as such.
|
|
@@ -163,3 +163,48 @@ He ensures that what gets built is:
|
|
|
163
163
|
- intentional,
|
|
164
164
|
- coherent over time,
|
|
165
165
|
- and traceable back to a clearly articulated system design — even as that design evolves.
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## Guardrails
|
|
170
|
+
|
|
171
|
+
### Allowed Sources
|
|
172
|
+
You may use ONLY information from these sources:
|
|
173
|
+
- System specification (`.blueprint/system_specification/SYSTEM_SPEC.md`)
|
|
174
|
+
- Feature specifications (`.blueprint/features/*/FEATURE_SPEC.md`)
|
|
175
|
+
- User stories (`story-*.md`) and test artifacts (`test-spec.md`, `*.test.js`)
|
|
176
|
+
- Implementation code in the project
|
|
177
|
+
- Business context (`.business_context/*`)
|
|
178
|
+
- Templates (`.blueprint/templates/*`) and agent specifications
|
|
179
|
+
|
|
180
|
+
### Prohibited Sources
|
|
181
|
+
Do not use:
|
|
182
|
+
- Social media, forums, blog posts, or external APIs
|
|
183
|
+
- Training data for domain facts—do not invent business rules
|
|
184
|
+
- External project or company references by name
|
|
185
|
+
|
|
186
|
+
### Citation Requirements
|
|
187
|
+
- Cite sources using: "Per [filename]: [claim]" or "[filename:section] states..."
|
|
188
|
+
- Use section-level citations where feasible (e.g., "story-login.md:AC-3")
|
|
189
|
+
- Reference `.business_context/` files for domain definitions
|
|
190
|
+
- Maintain a traceable chain: downstream artifacts cite upstream sources
|
|
191
|
+
|
|
192
|
+
### Assumptions vs Facts
|
|
193
|
+
- Label assumptions explicitly: "ASSUMPTION: [statement]" or "NOTE: Assuming..."
|
|
194
|
+
- Distinguish clearly between cited facts and assumptions
|
|
195
|
+
- Do not guess—state "This information is not available in the provided inputs"
|
|
196
|
+
|
|
197
|
+
### Confidentiality
|
|
198
|
+
- Do not reproduce `.business_context/` content verbatim; summarise or use generic descriptions
|
|
199
|
+
- Do not reference external entities, companies, or projects by name
|
|
200
|
+
- Do not use external services that would expose project data
|
|
201
|
+
- Outputs must be self-contained and understandable without access to confidential sources
|
|
202
|
+
|
|
203
|
+
### Escalation Protocol
|
|
204
|
+
Escalate to the user when:
|
|
205
|
+
- Critical information is missing and cannot be safely assumed
|
|
206
|
+
- Inputs are ambiguous with multiple possible interpretations—list options and ask for clarification
|
|
207
|
+
- Source documents conflict—cite both sources and request resolution
|
|
208
|
+
- Output would require violating confidentiality constraints
|
|
209
|
+
|
|
210
|
+
When escalation is not warranted, you may proceed with an explicit assumption labelled as such.
|
|
@@ -171,25 +171,46 @@ When you receive a new story or feature, you can structure your response like th
|
|
|
171
171
|
|
|
172
172
|
---
|
|
173
173
|
|
|
174
|
-
##
|
|
174
|
+
## Guardrails
|
|
175
|
+
|
|
176
|
+
### Allowed Sources
|
|
177
|
+
You may use ONLY information from these sources:
|
|
178
|
+
- System specification (`.blueprint/system_specification/SYSTEM_SPEC.md`)
|
|
179
|
+
- Feature specifications (`.blueprint/features/*/FEATURE_SPEC.md`)
|
|
180
|
+
- User stories (`story-*.md`) and test artifacts (`test-spec.md`, `*.test.js`)
|
|
181
|
+
- Implementation code in the project
|
|
182
|
+
- Business context (`.business_context/*`)
|
|
183
|
+
- Templates (`.blueprint/templates/*`) and agent specifications
|
|
184
|
+
|
|
185
|
+
### Prohibited Sources
|
|
186
|
+
Do not use:
|
|
187
|
+
- Social media, forums, blog posts, or external APIs
|
|
188
|
+
- Training data for domain facts—do not invent business rules
|
|
189
|
+
- External project or company references by name
|
|
190
|
+
|
|
191
|
+
### Citation Requirements
|
|
192
|
+
- Cite sources using: "Per [filename]: [claim]" or "[filename:section] states..."
|
|
193
|
+
- Use section-level citations where feasible (e.g., "story-login.md:AC-3")
|
|
194
|
+
- Reference `.business_context/` files for domain definitions
|
|
195
|
+
- Maintain a traceable chain: downstream artifacts cite upstream sources
|
|
196
|
+
|
|
197
|
+
### Assumptions vs Facts
|
|
198
|
+
- Label assumptions explicitly: "ASSUMPTION: [statement]" or "NOTE: Assuming..."
|
|
199
|
+
- Distinguish clearly between cited facts and assumptions
|
|
200
|
+
- Do not guess—state "This information is not available in the provided inputs"
|
|
201
|
+
|
|
202
|
+
### Confidentiality
|
|
203
|
+
- Do not reproduce `.business_context/` content verbatim; summarise or use generic descriptions
|
|
204
|
+
- Do not reference external entities, companies, or projects by name
|
|
205
|
+
- Do not use external services that would expose project data
|
|
206
|
+
- Outputs must be self-contained and understandable without access to confidential sources
|
|
207
|
+
|
|
208
|
+
### Escalation Protocol
|
|
209
|
+
Escalate to the user when:
|
|
210
|
+
- Critical information is missing and cannot be safely assumed
|
|
211
|
+
- Inputs are ambiguous with multiple possible interpretations—list options and ask for clarification
|
|
212
|
+
- Source documents conflict—cite both sources and request resolution
|
|
213
|
+
- Output would require violating confidentiality constraints
|
|
214
|
+
|
|
215
|
+
When escalation is not warranted, you may proceed with an explicit assumption labelled as such.
|
|
175
216
|
|
|
176
|
-
You have access to the following skills that can help with your work:
|
|
177
|
-
|
|
178
|
-
### `/javascript-testing-patterns`
|
|
179
|
-
|
|
180
|
-
**When to use:** When writing executable tests, setting up test infrastructure, or implementing test patterns.
|
|
181
|
-
|
|
182
|
-
**What it provides:**
|
|
183
|
-
- Jest and Vitest configuration and setup
|
|
184
|
-
- Unit testing patterns for functions and classes
|
|
185
|
-
- Async testing patterns (promises, async/await)
|
|
186
|
-
- Mocking patterns (modules, dependencies, spies)
|
|
187
|
-
- Integration testing with supertest
|
|
188
|
-
- Test fixtures and factories
|
|
189
|
-
- Best practices (AAA pattern, test organization)
|
|
190
|
-
|
|
191
|
-
**How to invoke:** Use `/javascript-testing-patterns` when you need guidance on test structure, mocking strategies, or testing async code.
|
|
192
|
-
|
|
193
|
-
**Location:** `.agents/skills/javascript-testing-patterns/SKILL.md`
|
|
194
|
-
|
|
195
|
-
---
|
|
@@ -0,0 +1,239 @@
|
|
|
1
|
+
# Feature Specification — Adaptive Retry
|
|
2
|
+
|
|
3
|
+
## 1. Feature Intent
|
|
4
|
+
**Why this feature exists.**
|
|
5
|
+
|
|
6
|
+
The orchestr8 pipeline currently offers simple retry/skip/abort options when an agent fails (per SKILL.md:Error Handling). This feature introduces **intelligent retry logic** that learns from failure history to make smarter decisions about how to handle failures.
|
|
7
|
+
|
|
8
|
+
- **Problem being addressed:** Pipeline failures are handled uniformly regardless of context. Repeated failures at the same stage waste time with identical retry attempts.
|
|
9
|
+
- **User need:** Developers want the pipeline to adapt its retry approach based on what has historically worked, reducing manual intervention and improving success rates.
|
|
10
|
+
- **System alignment:** Per SYSTEM_SPEC.md:Section 6 (High-Level Lifecycle), recovery from failure is already supported. This feature enhances that capability by making recovery **adaptive** rather than static.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## 2. Scope
|
|
15
|
+
### In Scope
|
|
16
|
+
- Retry configuration file management (`.claude/retry-config.json`)
|
|
17
|
+
- Failure pattern detection using existing history module (`src/history.js`)
|
|
18
|
+
- Strategy selection logic based on failure rates and attempt counts
|
|
19
|
+
- CLI commands for viewing and managing retry configuration
|
|
20
|
+
- Integration with pipeline error handling (advisory, not replacing user choice)
|
|
21
|
+
|
|
22
|
+
### Out of Scope
|
|
23
|
+
- Automatic prompt rewriting (strategies describe *what* to do; agents execute)
|
|
24
|
+
- Modifications to agent specifications or core pipeline orchestration
|
|
25
|
+
- Machine learning or model-based prediction (rule-based only)
|
|
26
|
+
- Cross-feature failure correlation (each feature is handled independently)
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## 3. Actors Involved
|
|
31
|
+
|
|
32
|
+
### Human User
|
|
33
|
+
- Can view current retry configuration via CLI
|
|
34
|
+
- Can modify thresholds and enabled strategies
|
|
35
|
+
- Can reset configuration to defaults
|
|
36
|
+
- Retains final decision authority on retry/skip/abort during pipeline execution
|
|
37
|
+
|
|
38
|
+
### Pipeline Orchestrator (SKILL.md implementation)
|
|
39
|
+
- Queries retry module when an agent fails
|
|
40
|
+
- Receives recommended strategy and displays it to user
|
|
41
|
+
- Applies strategy modifications to agent prompts when user accepts retry
|
|
42
|
+
|
|
43
|
+
### History Module (src/history.js)
|
|
44
|
+
- Provides failure data for strategy calculation
|
|
45
|
+
- Is read-only from this feature's perspective
|
|
46
|
+
|
|
47
|
+
### Insights Module (src/insights.js)
|
|
48
|
+
- Provides `analyzeFailures()` which calculates failure rates by stage
|
|
49
|
+
- Is read-only from this feature's perspective
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## 4. Behaviour Overview
|
|
54
|
+
|
|
55
|
+
### Happy Path: Strategy-Guided Retry
|
|
56
|
+
1. Agent fails during pipeline execution
|
|
57
|
+
2. Retry module queries history for recent failures at this stage
|
|
58
|
+
3. Module calculates failure rate over recent window (default: last 10 runs)
|
|
59
|
+
4. If failure rate > threshold (20%), recommend alternative strategy; otherwise recommend simple retry
|
|
60
|
+
5. Display recommendation to user: "Recommended strategy: {strategy}. Retry with this approach? (y/n/abort)"
|
|
61
|
+
6. If user accepts, modify agent prompt per strategy and retry
|
|
62
|
+
7. Record outcome in history
|
|
63
|
+
|
|
64
|
+
### Alternative: First Failure
|
|
65
|
+
- If this is the first recorded failure for a stage (no history), default to simple retry
|
|
66
|
+
- No prompt modification; standard retry behaviour
|
|
67
|
+
|
|
68
|
+
### Alternative: Max Retries Exceeded
|
|
69
|
+
- After `maxRetries` (default 3) attempts at same stage for same feature, warn user
|
|
70
|
+
- User can still choose to retry, but recommendation becomes "abort or skip"
|
|
71
|
+
|
|
72
|
+
### Alternative: User Declines Strategy
|
|
73
|
+
- User can override recommendation and choose different action (retry/skip/abort)
|
|
74
|
+
- System respects user choice; recommendation is advisory only
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## 5. State & Lifecycle Interactions
|
|
79
|
+
|
|
80
|
+
### States Affected
|
|
81
|
+
Per SYSTEM_SPEC.md:Section 6, pipeline states are: INIT, ALEX, CASS, NIGEL, CODEY_PLAN, CODEY_IMPLEMENT, COMMIT, COMPLETE, PAUSED, RESUME.
|
|
82
|
+
|
|
83
|
+
This feature operates within the **transition from any agent state to FAILED**:
|
|
84
|
+
- On agent error, before prompting user, retry module is consulted
|
|
85
|
+
- State remains in current stage during retry attempts
|
|
86
|
+
- State transitions to FAILED only on abort (unchanged from current behaviour)
|
|
87
|
+
|
|
88
|
+
### Lifecycle Classification
|
|
89
|
+
- **State-constraining:** Does not create new states; constrains how transitions to FAILED are handled
|
|
90
|
+
- **Queue-modifying:** May update `failed` array with additional metadata (strategy attempted, attempt count)
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## 6. Rules & Decision Logic
|
|
95
|
+
|
|
96
|
+
### Rule 1: Calculate Stage Failure Rate
|
|
97
|
+
- **Description:** Determine how often a specific stage has failed in recent history
|
|
98
|
+
- **Inputs:** Stage name, history entries, window size (default 10)
|
|
99
|
+
- **Outputs:** Failure rate (0.0 to 1.0)
|
|
100
|
+
- **Type:** Deterministic
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
failureRate = failedRunsAtStage / totalRecentRuns
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
### Rule 2: Select Retry Strategy
|
|
107
|
+
- **Description:** Choose appropriate strategy based on failure rate and attempt count
|
|
108
|
+
- **Inputs:** Stage name, current attempt number, failure rate, configured strategies
|
|
109
|
+
- **Outputs:** Recommended strategy name
|
|
110
|
+
- **Type:** Deterministic with precedence
|
|
111
|
+
|
|
112
|
+
```
|
|
113
|
+
if (attemptCount > maxRetries) → "abort-recommended"
|
|
114
|
+
if (failureRate > highFailureThreshold) → alternativeStrategy[stage][attemptCount]
|
|
115
|
+
else → "retry"
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
### Rule 3: Apply Strategy to Prompt
|
|
119
|
+
- **Description:** Modify agent prompt based on selected strategy
|
|
120
|
+
- **Inputs:** Original prompt, strategy name, stage context
|
|
121
|
+
- **Outputs:** Modified prompt (or original if strategy is "retry")
|
|
122
|
+
- **Type:** Deterministic
|
|
123
|
+
|
|
124
|
+
Strategy applications:
|
|
125
|
+
| Strategy | Prompt Modification |
|
|
126
|
+
|----------|-------------------|
|
|
127
|
+
| `retry` | No modification |
|
|
128
|
+
| `simplify-prompt` | Add: "Focus on core requirements only. Skip edge cases and optional sections." |
|
|
129
|
+
| `reduce-stories` | Add: "Write only the 2-3 most critical user stories. Defer others to follow-up." |
|
|
130
|
+
| `simplify-tests` | Add: "Write only happy-path tests for each AC. Skip edge cases." |
|
|
131
|
+
| `add-context` | Prepend relevant sections from previous stage outputs |
|
|
132
|
+
| `incremental` | Add: "Implement one test at a time. Stop and report after each." |
|
|
133
|
+
| `rollback` | Execute `git checkout -- .` on implementation files before retry |
|
|
134
|
+
|
|
135
|
+
### Rule 4: Persist Configuration
|
|
136
|
+
- **Description:** Configuration survives across sessions
|
|
137
|
+
- **Inputs:** User modifications via CLI
|
|
138
|
+
- **Outputs:** Updated `.claude/retry-config.json`
|
|
139
|
+
- **Type:** Deterministic (file write)
|
|
140
|
+
|
|
141
|
+
---
|
|
142
|
+
|
|
143
|
+
## 7. Dependencies
|
|
144
|
+
|
|
145
|
+
### System Components
|
|
146
|
+
- **src/history.js:** `readHistoryFile()` for failure data access (read-only dependency)
|
|
147
|
+
- **src/insights.js:** `analyzeFailures()` for failure pattern analysis (read-only dependency)
|
|
148
|
+
- **src/orchestrator.js:** Queue management; retry module will export functions for orchestrator to call
|
|
149
|
+
- **SKILL.md:** Error handling section will reference retry module recommendations
|
|
150
|
+
|
|
151
|
+
### File Dependencies
|
|
152
|
+
- **`.claude/pipeline-history.json`:** Must exist with valid structure for accurate analysis
|
|
153
|
+
- **`.claude/retry-config.json`:** Created on first use if not present; gitignored
|
|
154
|
+
|
|
155
|
+
### Operational Dependencies
|
|
156
|
+
- History module must be functional (feature degrades gracefully if corrupted)
|
|
157
|
+
- File system write access for configuration persistence
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
## 8. Non-Functional Considerations
|
|
162
|
+
|
|
163
|
+
### Performance
|
|
164
|
+
- Strategy calculation should complete in <100ms (simple arithmetic on in-memory data)
|
|
165
|
+
- No blocking I/O during recommendation phase beyond initial history read
|
|
166
|
+
|
|
167
|
+
### Resilience
|
|
168
|
+
- If history file is corrupted, default to simple retry (no strategy recommendation)
|
|
169
|
+
- If config file is corrupted or missing, use hardcoded defaults
|
|
170
|
+
- All failures in retry module should be caught and logged, not propagated
|
|
171
|
+
|
|
172
|
+
### Audit/Logging
|
|
173
|
+
- Retry attempts and strategies used should be recorded in history entries
|
|
174
|
+
- New history fields: `retryAttempts`, `strategiesUsed[]`
|
|
175
|
+
|
|
176
|
+
### Maintainability
|
|
177
|
+
- Strategies should be configurable without code changes
|
|
178
|
+
- New strategies can be added by updating config schema and `applyStrategy()` function
|
|
179
|
+
|
|
180
|
+
---
|
|
181
|
+
|
|
182
|
+
## 9. Assumptions & Open Questions
|
|
183
|
+
|
|
184
|
+
### Assumptions
|
|
185
|
+
- ASSUMPTION: History file format is stable and matches `src/history.js` expectations
|
|
186
|
+
- ASSUMPTION: Pipeline execution is single-threaded; no concurrent retry decisions needed
|
|
187
|
+
- ASSUMPTION: User preference is final; recommendations are advisory only
|
|
188
|
+
- ASSUMPTION: Stage names match exactly between history, insights, and retry module
|
|
189
|
+
|
|
190
|
+
### Open Questions
|
|
191
|
+
- Should strategy history be factored into recommendations? (e.g., "simplify-prompt worked last time")
|
|
192
|
+
- Should there be a "never retry" option for specific stages?
|
|
193
|
+
- How should retry configuration interact with `--pause-after` flags?
|
|
194
|
+
|
|
195
|
+
---
|
|
196
|
+
|
|
197
|
+
## 10. Impact on System Specification
|
|
198
|
+
|
|
199
|
+
### Reinforces Existing Assumptions
|
|
200
|
+
- Per SYSTEM_SPEC.md:Section 8 (Cross-Cutting Concerns), failure handling already "offers retry, skip, abort"
|
|
201
|
+
- This feature enhances rather than replaces that pattern
|
|
202
|
+
- Per SYSTEM_SPEC.md:Section 7 (Governing Rules), "recovery-safe" queue behaviour is maintained
|
|
203
|
+
|
|
204
|
+
### Stretches Existing Assumptions
|
|
205
|
+
- History module was designed for reporting/insights; now becomes an active input to pipeline decisions
|
|
206
|
+
- This elevates history from "observability" to "operational dependency"
|
|
207
|
+
|
|
208
|
+
### No Contradictions Identified
|
|
209
|
+
- Feature operates within existing error handling flow
|
|
210
|
+
- Does not modify agent specifications or pipeline sequence
|
|
211
|
+
- Preserves user authority over final decisions
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
## 11. Handover to BA (Cass)
|
|
216
|
+
|
|
217
|
+
### Story Themes
|
|
218
|
+
1. **Configuration Management:** User can view, modify, and reset retry configuration
|
|
219
|
+
2. **Strategy Recommendation:** Pipeline recommends retry strategy based on failure history
|
|
220
|
+
3. **Prompt Modification:** Retry strategies modify agent prompts to improve success probability
|
|
221
|
+
4. **History Integration:** Retry decisions are informed by historical failure patterns
|
|
222
|
+
|
|
223
|
+
### Expected Story Boundaries
|
|
224
|
+
- CLI commands (`retry-config`, `retry-config reset`) as separate story
|
|
225
|
+
- Strategy selection logic as separate story
|
|
226
|
+
- Strategy application (prompt modification) as separate story
|
|
227
|
+
- Graceful degradation (corrupted history/config) as non-functional story or AC within above
|
|
228
|
+
|
|
229
|
+
### Areas Needing Careful Framing
|
|
230
|
+
- User choice must remain paramount; stories should emphasize "recommendation" not "automation"
|
|
231
|
+
- The "rollback" strategy involves destructive action (`git checkout`); needs explicit user confirmation
|
|
232
|
+
- Incremental strategy changes Codey's execution model; may need coordination with Codey's agent spec
|
|
233
|
+
|
|
234
|
+
---
|
|
235
|
+
|
|
236
|
+
## 12. Change Log (Feature-Level)
|
|
237
|
+
| Date | Change | Reason | Raised By |
|
|
238
|
+
|------|--------|--------|-----------|
|
|
239
|
+
| 2026-02-24 | Initial feature specification | New feature request | Alex |
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Implementation Plan - Adaptive Retry
|
|
2
|
+
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
This feature adds intelligent retry logic to the orchestr8 pipeline via a new `src/retry.js` module. The module calculates failure rates from history, recommends strategies based on configurable thresholds, and applies prompt modifications when users accept recommended strategies. The implementation integrates with existing `src/history.js` for failure data and provides CLI commands for configuration management.
|
|
6
|
+
|
|
7
|
+
## Files to Create/Modify
|
|
8
|
+
|
|
9
|
+
| Path | Action | Purpose |
|
|
10
|
+
|------|--------|---------|
|
|
11
|
+
| `src/retry.js` | Create | Core retry logic: config management, failure rate calculation, strategy recommendation, prompt modification |
|
|
12
|
+
| `src/index.js` | Modify | Export retry module functions |
|
|
13
|
+
| `bin/cli.js` | Modify | Add `retry-config` CLI command with `set` and `reset` subcommands |
|
|
14
|
+
|
|
15
|
+
## Implementation Steps
|
|
16
|
+
|
|
17
|
+
1. **Create `src/retry.js` with config management** - Implement `getDefaultConfig()`, `readConfig()`, `writeConfig()`, `resetConfig()` functions for `.claude/retry-config.json`
|
|
18
|
+
|
|
19
|
+
2. **Add failure rate calculation** - Implement `calculateFailureRate(stage, history, windowSize)` that returns failure rate (0-1) for a stage over the recent window
|
|
20
|
+
|
|
21
|
+
3. **Add strategy recommendation logic** - Implement `recommendStrategy(stage, attemptCount, failureRate, config)` returning strategy name or "abort-recommended"
|
|
22
|
+
|
|
23
|
+
4. **Add prompt modification** - Implement `applyStrategy(strategy, originalPrompt)` that returns modified prompt based on strategy
|
|
24
|
+
|
|
25
|
+
5. **Add graceful degradation** - Wrap all file operations in try/catch; return defaults on corrupted/missing files; log warnings
|
|
26
|
+
|
|
27
|
+
6. **Add CLI commands to `bin/cli.js`** - Add `retry-config` (view), `retry-config set <key> <value>`, `retry-config reset` commands
|
|
28
|
+
|
|
29
|
+
7. **Export functions in `src/index.js`** - Export `readConfig`, `writeConfig`, `resetConfig`, `calculateFailureRate`, `recommendStrategy`, `applyStrategy`
|
|
30
|
+
|
|
31
|
+
8. **Run tests and iterate** - Execute `node --test test/feature_adaptive-retry.test.js` after each step; fix failures
|
|
32
|
+
|
|
33
|
+
## Key Functions
|
|
34
|
+
|
|
35
|
+
- `getDefaultConfig()` - Returns hardcoded default configuration object
|
|
36
|
+
- `readConfig()` - Reads `.claude/retry-config.json` or returns defaults if missing/corrupted
|
|
37
|
+
- `writeConfig(config)` - Writes config to file, creates `.claude/` directory if needed
|
|
38
|
+
- `resetConfig()` - Replaces config with defaults
|
|
39
|
+
- `calculateFailureRate(stage, history, windowSize)` - Calculates failure rate for stage over window
|
|
40
|
+
- `recommendStrategy(stage, attemptCount, failureRate, config)` - Returns recommended strategy name
|
|
41
|
+
- `applyStrategy(strategy, originalPrompt)` - Returns modified prompt or original if "retry"
|
|
42
|
+
- `shouldRetry(stage, featureSlug, history, config, attemptCount)` - Orchestrator entry point; returns recommendation object
|
|
43
|
+
|
|
44
|
+
## Risks/Questions
|
|
45
|
+
|
|
46
|
+
- **Rollback strategy is destructive**: Per story-prompt-modification.md:AC-6, `git checkout -- .` requires explicit confirmation before execution; implementation must surface this warning
|
|
47
|
+
- **History module dependency**: If `readHistoryFile()` returns `{ error: 'corrupted' }`, module must degrade gracefully per test T-SH-2
|
|
48
|
+
- **Strategy escalation boundary**: When `attemptCount - 1` exceeds strategy list length, use last strategy before switching to "abort-recommended"
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Story — Prompt Modification
|
|
2
|
+
|
|
3
|
+
## User Story
|
|
4
|
+
|
|
5
|
+
As a **developer using orchestr8**, I want the **pipeline to modify agent prompts based on the selected retry strategy** so that **retry attempts have a better chance of success by adjusting the agent's instructions**.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / Scope
|
|
10
|
+
|
|
11
|
+
- Per FEATURE_SPEC.md:Section 6 (Rule 3), strategies apply specific modifications to agent prompts
|
|
12
|
+
- Per FEATURE_SPEC.md:Section 11 (Areas Needing Careful Framing), the "rollback" strategy requires explicit user confirmation
|
|
13
|
+
- Per FEATURE_SPEC.md:Section 2 (Out of Scope), automatic prompt rewriting is not supported; strategies describe what to do
|
|
14
|
+
- Per SYSTEM_SPEC.md:Section 7 (Agent Rules), agents must not make silent changes
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Acceptance Criteria
|
|
19
|
+
|
|
20
|
+
**AC-1 — Apply "simplify-prompt" strategy**
|
|
21
|
+
- Given the user accepts a retry with the "simplify-prompt" strategy,
|
|
22
|
+
- When the agent is re-invoked,
|
|
23
|
+
- Then the prompt is modified to append: "Focus on core requirements only. Skip edge cases and optional sections.",
|
|
24
|
+
- And the modification is visible in the agent's task context.
|
|
25
|
+
|
|
26
|
+
**AC-2 — Apply "reduce-stories" strategy**
|
|
27
|
+
- Given the user accepts a retry with the "reduce-stories" strategy,
|
|
28
|
+
- When Cass is re-invoked,
|
|
29
|
+
- Then the prompt is modified to append: "Write only the 2-3 most critical user stories. Defer others to follow-up.",
|
|
30
|
+
- And the deferred stories are noted in the summary.
|
|
31
|
+
|
|
32
|
+
**AC-3 — Apply "simplify-tests" strategy**
|
|
33
|
+
- Given the user accepts a retry with the "simplify-tests" strategy,
|
|
34
|
+
- When Nigel is re-invoked,
|
|
35
|
+
- Then the prompt is modified to append: "Write only happy-path tests for each AC. Skip edge cases.",
|
|
36
|
+
- And edge case tests are explicitly deferred.
|
|
37
|
+
|
|
38
|
+
**AC-4 — Apply "add-context" strategy**
|
|
39
|
+
- Given the user accepts a retry with the "add-context" strategy,
|
|
40
|
+
- When the agent is re-invoked,
|
|
41
|
+
- Then relevant sections from previous stage outputs are prepended to the prompt,
|
|
42
|
+
- And the source of added context is indicated (e.g., "Context from FEATURE_SPEC.md:Section 4").
|
|
43
|
+
|
|
44
|
+
**AC-5 — Apply "incremental" strategy**
|
|
45
|
+
- Given the user accepts a retry with the "incremental" strategy,
|
|
46
|
+
- When Codey is re-invoked,
|
|
47
|
+
- Then the prompt is modified to append: "Implement one test at a time. Stop and report after each.",
|
|
48
|
+
- And each implementation step produces a discrete output before continuing.
|
|
49
|
+
|
|
50
|
+
**AC-6 — Apply "rollback" strategy with confirmation**
|
|
51
|
+
- Given the user accepts a retry with the "rollback" strategy,
|
|
52
|
+
- When the user confirms the destructive action,
|
|
53
|
+
- Then `git checkout -- .` is executed on implementation files before retry,
|
|
54
|
+
- And a confirmation prompt warns: "This will discard uncommitted changes to implementation files. Proceed? (y/n)".
|
|
55
|
+
|
|
56
|
+
**AC-7 — No modification for "retry" strategy**
|
|
57
|
+
- Given the user accepts a simple "retry" (no strategy modification),
|
|
58
|
+
- When the agent is re-invoked,
|
|
59
|
+
- Then the original prompt is used without any modification,
|
|
60
|
+
- And the retry proceeds identically to the first attempt.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Strategy Reference Table
|
|
65
|
+
|
|
66
|
+
Per FEATURE_SPEC.md:Section 6 (Rule 3):
|
|
67
|
+
|
|
68
|
+
| Strategy | Modification Applied |
|
|
69
|
+
|----------|---------------------|
|
|
70
|
+
| `retry` | No modification |
|
|
71
|
+
| `simplify-prompt` | Append: "Focus on core requirements only. Skip edge cases and optional sections." |
|
|
72
|
+
| `reduce-stories` | Append: "Write only the 2-3 most critical user stories. Defer others to follow-up." |
|
|
73
|
+
| `simplify-tests` | Append: "Write only happy-path tests for each AC. Skip edge cases." |
|
|
74
|
+
| `add-context` | Prepend relevant sections from previous stage outputs |
|
|
75
|
+
| `incremental` | Append: "Implement one test at a time. Stop and report after each." |
|
|
76
|
+
| `rollback` | Execute `git checkout -- .` on implementation files before retry |
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## Out of Scope
|
|
81
|
+
|
|
82
|
+
- Automatic prompt rewriting based on error analysis (per FEATURE_SPEC.md:Section 2)
|
|
83
|
+
- Custom user-defined prompt modifications (strategies are predefined)
|
|
84
|
+
- Modifying agent specifications themselves (per FEATURE_SPEC.md:Section 2)
|
|
85
|
+
- Combining multiple strategies in a single retry (one strategy per attempt)
|