orchestr8 2.5.0 → 2.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.blueprint/agents/AGENT_BA_CASS.md +42 -19
- package/.blueprint/agents/AGENT_DEVELOPER_CODEY.md +42 -38
- package/.blueprint/agents/AGENT_SPECIFICATION_ALEX.md +45 -0
- package/.blueprint/agents/AGENT_TESTER_NIGEL.md +42 -21
- package/.blueprint/features/feature_adaptive-retry/FEATURE_SPEC.md +239 -0
- package/.blueprint/features/feature_adaptive-retry/IMPLEMENTATION_PLAN.md +48 -0
- package/.blueprint/features/feature_adaptive-retry/story-prompt-modification.md +85 -0
- package/.blueprint/features/feature_adaptive-retry/story-retry-config.md +89 -0
- package/.blueprint/features/feature_adaptive-retry/story-should-retry.md +98 -0
- package/.blueprint/features/feature_adaptive-retry/story-strategy-recommendation.md +85 -0
- package/.blueprint/features/feature_agent-guardrails/FEATURE_SPEC.md +328 -0
- package/.blueprint/features/feature_agent-guardrails/IMPLEMENTATION_PLAN.md +90 -0
- package/.blueprint/features/feature_agent-guardrails/story-citation-requirements.md +50 -0
- package/.blueprint/features/feature_agent-guardrails/story-confidentiality.md +50 -0
- package/.blueprint/features/feature_agent-guardrails/story-escalation-protocol.md +55 -0
- package/.blueprint/features/feature_agent-guardrails/story-source-restrictions.md +50 -0
- package/.blueprint/features/feature_feedback-loop/FEATURE_SPEC.md +347 -0
- package/.blueprint/features/feature_feedback-loop/IMPLEMENTATION_PLAN.md +71 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-collection.md +63 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-config.md +61 -0
- package/.blueprint/features/feature_feedback-loop/story-feedback-insights.md +63 -0
- package/.blueprint/features/feature_feedback-loop/story-quality-gates.md +57 -0
- package/.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md +239 -0
- package/.blueprint/features/feature_pipeline-history/IMPLEMENTATION_PLAN.md +71 -0
- package/.blueprint/features/feature_pipeline-history/story-clear-history.md +73 -0
- package/.blueprint/features/feature_pipeline-history/story-display-history.md +75 -0
- package/.blueprint/features/feature_pipeline-history/story-record-execution.md +76 -0
- package/.blueprint/features/feature_pipeline-history/story-show-statistics.md +85 -0
- package/.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md +288 -0
- package/.blueprint/features/feature_pipeline-insights/IMPLEMENTATION_PLAN.md +65 -0
- package/.blueprint/features/feature_pipeline-insights/story-anomaly-detection.md +71 -0
- package/.blueprint/features/feature_pipeline-insights/story-bottleneck-analysis.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-failure-patterns.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-json-output.md +75 -0
- package/.blueprint/features/feature_pipeline-insights/story-trend-analysis.md +78 -0
- package/.blueprint/features/feature_validate-command/FEATURE_SPEC.md +209 -0
- package/.blueprint/features/feature_validate-command/IMPLEMENTATION_PLAN.md +59 -0
- package/.blueprint/features/feature_validate-command/story-failure-output.md +61 -0
- package/.blueprint/features/feature_validate-command/story-node-version-check.md +52 -0
- package/.blueprint/features/feature_validate-command/story-run-validation.md +59 -0
- package/.blueprint/features/feature_validate-command/story-success-output.md +50 -0
- package/.blueprint/system_specification/SYSTEM_SPEC.md +248 -0
- package/README.md +182 -38
- package/SKILL.md +333 -23
- package/bin/cli.js +128 -20
- package/package.json +2 -2
- package/src/feedback.js +171 -0
- package/src/history.js +306 -0
- package/src/index.js +57 -2
- package/src/init.js +2 -6
- package/src/insights.js +504 -0
- package/src/retry.js +274 -0
- package/src/validate.js +172 -0
- package/src/skills.js +0 -93
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# Story — Bottleneck Analysis
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
|
|
5
|
+
As a developer, I want to identify which pipeline stage consistently takes the longest so that I can focus optimization efforts where they will have the greatest impact.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / scope
|
|
10
|
+
|
|
11
|
+
- User has executed multiple pipeline runs via `/implement-feature`
|
|
12
|
+
- History data exists in `.claude/pipeline-history.json`
|
|
13
|
+
- This is a read-only analysis; no pipeline state is modified
|
|
14
|
+
- Route: `orchestr8 insights` or `orchestr8 insights --bottlenecks`
|
|
15
|
+
|
|
16
|
+
Per FEATURE_SPEC.md:Section 6 (Rule: Bottleneck Detection):
|
|
17
|
+
- Bottleneck threshold: >35% of total pipeline time
|
|
18
|
+
- Recommendation threshold: >40% of total pipeline time
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Acceptance criteria
|
|
23
|
+
|
|
24
|
+
**AC-1 — Display bottleneck stage**
|
|
25
|
+
- Given the history file contains at least 3 successful pipeline runs,
|
|
26
|
+
- When the user runs `orchestr8 insights`,
|
|
27
|
+
- Then the output includes a "Bottlenecks" section identifying the stage with the highest average duration.
|
|
28
|
+
|
|
29
|
+
**AC-2 — Show percentage of total time**
|
|
30
|
+
- Given a bottleneck stage is identified,
|
|
31
|
+
- When the analysis completes,
|
|
32
|
+
- Then the output displays the stage name, average duration in milliseconds, and percentage of total pipeline time.
|
|
33
|
+
|
|
34
|
+
**AC-3 — Bottleneck threshold reporting**
|
|
35
|
+
- Given a stage accounts for more than 35% of total pipeline time,
|
|
36
|
+
- When the analysis completes,
|
|
37
|
+
- Then that stage is flagged as a bottleneck in the output.
|
|
38
|
+
|
|
39
|
+
**AC-4 — Generate recommendation for severe bottleneck**
|
|
40
|
+
- Given a stage accounts for more than 40% of total pipeline time,
|
|
41
|
+
- When the analysis completes,
|
|
42
|
+
- Then the output includes the recommendation: "Consider simplifying {stage} requirements or splitting features".
|
|
43
|
+
|
|
44
|
+
**AC-5 — Filter to bottlenecks only**
|
|
45
|
+
- Given the user runs `orchestr8 insights --bottlenecks`,
|
|
46
|
+
- When the analysis completes,
|
|
47
|
+
- Then only the bottleneck analysis section is displayed (other analysis types are omitted).
|
|
48
|
+
|
|
49
|
+
**AC-6 — Insufficient data handling**
|
|
50
|
+
- Given the history file contains fewer than 3 runs,
|
|
51
|
+
- When the user runs `orchestr8 insights`,
|
|
52
|
+
- Then the output displays: "Insufficient data for insights. Complete at least 3 pipeline runs."
|
|
53
|
+
|
|
54
|
+
**AC-7 — Missing history file handling**
|
|
55
|
+
- Given no history file exists at `.claude/pipeline-history.json`,
|
|
56
|
+
- When the user runs `orchestr8 insights`,
|
|
57
|
+
- Then the output displays: "No history found. Run some pipelines first."
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## Out of scope
|
|
62
|
+
|
|
63
|
+
- Modifying the history file or pipeline configuration
|
|
64
|
+
- Customizing the 35%/40% threshold values
|
|
65
|
+
- Providing automated remediation
|
|
66
|
+
- Stage-specific threshold configuration
|
|
67
|
+
- Analysis of partial/in-progress runs
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## References
|
|
72
|
+
|
|
73
|
+
- Feature spec: `.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md`
|
|
74
|
+
- Upstream dependency: `.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md`
|
|
75
|
+
- System spec: `.blueprint/system_specification/SYSTEM_SPEC.md`
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# Story — Failure Pattern Analysis
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
|
|
5
|
+
As a developer, I want to analyze which pipeline stages fail most frequently so that I can identify systemic issues and improve pipeline reliability.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / scope
|
|
10
|
+
|
|
11
|
+
- User has executed multiple pipeline runs, some of which have failed
|
|
12
|
+
- History data includes entries with `status: "failed"` and associated stage information
|
|
13
|
+
- This is a read-only analysis; no pipeline state is modified
|
|
14
|
+
- Route: `orchestr8 insights` or `orchestr8 insights --failures`
|
|
15
|
+
|
|
16
|
+
Per FEATURE_SPEC.md:Section 6 (Rule: Failure Pattern Analysis):
|
|
17
|
+
- Failure rate threshold: >15% is reported as concerning
|
|
18
|
+
- Recommendation threshold: >20% triggers specific recommendation
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Acceptance criteria
|
|
23
|
+
|
|
24
|
+
**AC-1 — Display failure rates per stage**
|
|
25
|
+
- Given the history file contains at least 3 pipeline runs with at least one failure,
|
|
26
|
+
- When the user runs `orchestr8 insights`,
|
|
27
|
+
- Then the output includes a "Failure Patterns" section showing failure rate for each stage that has experienced failures.
|
|
28
|
+
|
|
29
|
+
**AC-2 — Identify most common failure stage**
|
|
30
|
+
- Given failures exist in the history,
|
|
31
|
+
- When the analysis completes,
|
|
32
|
+
- Then the output identifies the stage with the highest failure count as the "most common failure stage".
|
|
33
|
+
|
|
34
|
+
**AC-3 — Flag concerning failure rates**
|
|
35
|
+
- Given a stage has a failure rate greater than 15%,
|
|
36
|
+
- When the analysis completes,
|
|
37
|
+
- Then that stage is flagged as having a concerning failure rate.
|
|
38
|
+
|
|
39
|
+
**AC-4 — Generate recommendation for high failure rate**
|
|
40
|
+
- Given a stage has a failure rate greater than 20%,
|
|
41
|
+
- When the analysis completes,
|
|
42
|
+
- Then the output includes the recommendation: "Review {stage} agent configuration or specification clarity".
|
|
43
|
+
|
|
44
|
+
**AC-5 — Identify features with repeated failures**
|
|
45
|
+
- Given the same feature slug has failed multiple times,
|
|
46
|
+
- When the analysis completes,
|
|
47
|
+
- Then those features are listed as correlation hints (e.g., "Feature 'complex-auth' has failed 3 times").
|
|
48
|
+
|
|
49
|
+
**AC-6 — Filter to failures only**
|
|
50
|
+
- Given the user runs `orchestr8 insights --failures`,
|
|
51
|
+
- When the analysis completes,
|
|
52
|
+
- Then only the failure pattern analysis section is displayed (other analysis types are omitted).
|
|
53
|
+
|
|
54
|
+
**AC-7 — No failures recorded**
|
|
55
|
+
- Given all pipeline runs in history have status "completed" (no failures),
|
|
56
|
+
- When the user runs `orchestr8 insights`,
|
|
57
|
+
- Then the failure analysis section displays: "No failures recorded" and is omitted from recommendations.
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## Out of scope
|
|
62
|
+
|
|
63
|
+
- Automatic correlation with feature complexity metrics
|
|
64
|
+
- Root cause analysis beyond stage identification
|
|
65
|
+
- Failure notification or alerting
|
|
66
|
+
- Retry or remediation automation
|
|
67
|
+
- Classification of failure types (timeout vs error vs abort)
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## References
|
|
72
|
+
|
|
73
|
+
- Feature spec: `.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md`
|
|
74
|
+
- Upstream dependency: `.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md`
|
|
75
|
+
- System spec: `.blueprint/system_specification/SYSTEM_SPEC.md`
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# Story — JSON Output
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
|
|
5
|
+
As a developer, I want to export pipeline insights as structured JSON so that I can integrate the analysis with other tools or process the data programmatically.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / scope
|
|
10
|
+
|
|
11
|
+
- User wants machine-readable output instead of human-readable text
|
|
12
|
+
- JSON output contains all the same analysis data as text output
|
|
13
|
+
- Enables integration with CI/CD pipelines, dashboards, or custom tooling
|
|
14
|
+
- This is a read-only analysis; no pipeline state is modified
|
|
15
|
+
- Route: `orchestr8 insights --json`
|
|
16
|
+
|
|
17
|
+
Per FEATURE_SPEC.md:Section 4 (Key alternatives or branches):
|
|
18
|
+
- `--json` flag produces structured JSON instead of formatted text
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Acceptance criteria
|
|
23
|
+
|
|
24
|
+
**AC-1 — Output JSON when flag provided**
|
|
25
|
+
- Given the history file contains valid pipeline data,
|
|
26
|
+
- When the user runs `orchestr8 insights --json`,
|
|
27
|
+
- Then the output is valid JSON (parseable by `JSON.parse()`).
|
|
28
|
+
|
|
29
|
+
**AC-2 — Include bottleneck data in JSON**
|
|
30
|
+
- Given bottleneck analysis completes successfully,
|
|
31
|
+
- When `--json` flag is provided,
|
|
32
|
+
- Then the JSON output includes a `bottlenecks` object with: `stage`, `averageDurationMs`, `percentageOfTotal`, `isBottleneck`, `recommendation` (if applicable).
|
|
33
|
+
|
|
34
|
+
**AC-3 — Include failure data in JSON**
|
|
35
|
+
- Given failure analysis completes successfully,
|
|
36
|
+
- When `--json` flag is provided,
|
|
37
|
+
- Then the JSON output includes a `failures` object with: `failuresByStage` (array), `mostCommonFailureStage`, `featuresWithRepeatedFailures` (array), `recommendation` (if applicable).
|
|
38
|
+
|
|
39
|
+
**AC-4 — Include anomaly data in JSON**
|
|
40
|
+
- Given anomaly detection completes successfully,
|
|
41
|
+
- When `--json` flag is provided,
|
|
42
|
+
- Then the JSON output includes an `anomalies` object with: `detected` (array of anomalous runs), `recommendation` (if applicable).
|
|
43
|
+
|
|
44
|
+
**AC-5 — Include trend data in JSON**
|
|
45
|
+
- Given trend analysis completes successfully,
|
|
46
|
+
- When `--json` flag is provided,
|
|
47
|
+
- Then the JSON output includes a `trends` object with: `successRate` (trend + percentage), `duration` (trend + percentage), `recommendation` (if applicable).
|
|
48
|
+
|
|
49
|
+
**AC-6 — Combine JSON with filter flags**
|
|
50
|
+
- Given the user runs `orchestr8 insights --json --bottlenecks`,
|
|
51
|
+
- When the analysis completes,
|
|
52
|
+
- Then the JSON output includes only the `bottlenecks` section (other analysis types are omitted).
|
|
53
|
+
|
|
54
|
+
**AC-7 — Handle insufficient data in JSON**
|
|
55
|
+
- Given there is insufficient data for analysis,
|
|
56
|
+
- When `--json` flag is provided,
|
|
57
|
+
- Then the JSON output includes an `error` field with the appropriate message (e.g., `{"error": "Insufficient data for insights. Complete at least 3 pipeline runs."}`).
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## Out of scope
|
|
62
|
+
|
|
63
|
+
- Exporting JSON to a file (output to stdout only)
|
|
64
|
+
- JSON schema validation or versioning
|
|
65
|
+
- Compressed or minified JSON output options
|
|
66
|
+
- Integration with specific external platforms
|
|
67
|
+
- Historical JSON output comparison
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## References
|
|
72
|
+
|
|
73
|
+
- Feature spec: `.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md`
|
|
74
|
+
- Upstream dependency: `.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md`
|
|
75
|
+
- System spec: `.blueprint/system_specification/SYSTEM_SPEC.md`
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# Story — Trend Analysis
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
|
|
5
|
+
As a developer, I want to track whether pipeline performance is improving or degrading over time so that I can understand the impact of changes and maintain development efficiency.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Context / scope
|
|
10
|
+
|
|
11
|
+
- User has accumulated sufficient history to compare earlier vs later runs
|
|
12
|
+
- Analysis compares first half vs second half of history data
|
|
13
|
+
- Requires minimum 6 runs to compute meaningful trends
|
|
14
|
+
- This is a read-only analysis; no pipeline state is modified
|
|
15
|
+
- Route: `orchestr8 insights` (trends section included by default)
|
|
16
|
+
|
|
17
|
+
Per FEATURE_SPEC.md:Section 6 (Rule: Trend Analysis):
|
|
18
|
+
- Improving: >10% better in second half
|
|
19
|
+
- Degrading: >10% worse in second half
|
|
20
|
+
- Stable: within 10% variance
|
|
21
|
+
- Minimum data: 6 runs required
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Acceptance criteria
|
|
26
|
+
|
|
27
|
+
**AC-1 — Display success rate trend**
|
|
28
|
+
- Given the history file contains at least 6 pipeline runs,
|
|
29
|
+
- When the user runs `orchestr8 insights`,
|
|
30
|
+
- Then the output includes a "Trends" section showing success rate trend as "improving", "stable", or "degrading".
|
|
31
|
+
|
|
32
|
+
**AC-2 — Display duration trend**
|
|
33
|
+
- Given the history file contains at least 6 pipeline runs,
|
|
34
|
+
- When the user runs `orchestr8 insights`,
|
|
35
|
+
- Then the output includes average duration trend as "improving" (faster), "stable", or "degrading" (slower).
|
|
36
|
+
|
|
37
|
+
**AC-3 — Calculate trend by comparing halves**
|
|
38
|
+
- Given the history contains N runs (where N >= 6),
|
|
39
|
+
- When trend analysis is performed,
|
|
40
|
+
- Then the first N/2 runs are compared against the last N/2 runs to determine trend direction.
|
|
41
|
+
|
|
42
|
+
**AC-4 — Apply threshold for trend classification**
|
|
43
|
+
- Given the percentage change between halves is calculated,
|
|
44
|
+
- When classifying the trend,
|
|
45
|
+
- Then >10% improvement shows "improving", >10% degradation shows "degrading", and within 10% shows "stable".
|
|
46
|
+
|
|
47
|
+
**AC-5 — Generate recommendation for degrading trend**
|
|
48
|
+
- Given either success rate or duration trend is "degrading",
|
|
49
|
+
- When the analysis completes,
|
|
50
|
+
- Then the output includes the recommendation: "Review recent changes to agent specifications or system spec".
|
|
51
|
+
|
|
52
|
+
**AC-6 — Insufficient data for trends**
|
|
53
|
+
- Given the history file contains fewer than 6 runs,
|
|
54
|
+
- When trend analysis is attempted,
|
|
55
|
+
- Then it is skipped with explanation: "Insufficient data for trend analysis. Need at least 6 runs."
|
|
56
|
+
|
|
57
|
+
**AC-7 — Show percentage change**
|
|
58
|
+
- Given trend analysis completes successfully,
|
|
59
|
+
- When the output is displayed,
|
|
60
|
+
- Then the percentage change for both success rate and duration is shown alongside the trend indicator.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Out of scope
|
|
65
|
+
|
|
66
|
+
- Sliding window trend analysis (uses fixed first-half/second-half)
|
|
67
|
+
- Time-based trend analysis (e.g., last 30 days)
|
|
68
|
+
- Per-stage trend analysis
|
|
69
|
+
- Trend visualization or charting
|
|
70
|
+
- Predictive modelling of future performance
|
|
71
|
+
|
|
72
|
+
---
|
|
73
|
+
|
|
74
|
+
## References
|
|
75
|
+
|
|
76
|
+
- Feature spec: `.blueprint/features/feature_pipeline-insights/FEATURE_SPEC.md`
|
|
77
|
+
- Upstream dependency: `.blueprint/features/feature_pipeline-history/FEATURE_SPEC.md`
|
|
78
|
+
- System spec: `.blueprint/system_specification/SYSTEM_SPEC.md`
|
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
# Feature Specification — Validate Command
|
|
2
|
+
|
|
3
|
+
## 1. Feature Intent
|
|
4
|
+
**Why this feature exists.**
|
|
5
|
+
|
|
6
|
+
The `orchestr8 validate` command provides pre-flight checks to ensure the environment is correctly configured before running the `/implement-feature` pipeline. This addresses a common failure mode where users invoke the pipeline without required artifacts in place, leading to mid-pipeline failures that are harder to diagnose and recover from.
|
|
7
|
+
|
|
8
|
+
**Problem being addressed:**
|
|
9
|
+
- Pipeline failures mid-execution due to missing system spec, agent files, or skills
|
|
10
|
+
- Poor developer experience when prerequisites are unclear
|
|
11
|
+
- No visibility into "readiness" state before committing to a full pipeline run
|
|
12
|
+
|
|
13
|
+
**User need:**
|
|
14
|
+
- Developers need confidence that their project is correctly set up before invoking the pipeline
|
|
15
|
+
- Teams need actionable guidance when setup is incomplete
|
|
16
|
+
|
|
17
|
+
**How this supports the system purpose:**
|
|
18
|
+
- Aligns with the system's goal of reducing ambiguity and drift by enforcing explicit prerequisites
|
|
19
|
+
- Supports the "System spec gate" rule defined in the system specification (Section 7)
|
|
20
|
+
- Reduces failed pipeline runs by catching issues early
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Scope
|
|
25
|
+
|
|
26
|
+
### In Scope
|
|
27
|
+
- New CLI command `orchestr8 validate` accessible via `bin/cli.js`
|
|
28
|
+
- Check: System spec exists at `.blueprint/system_specification/SYSTEM_SPEC.md`
|
|
29
|
+
- Check: Required directories exist (`.blueprint/`, `.business_context/`, `.claude/commands/`)
|
|
30
|
+
- Check: Agent spec files exist in `.blueprint/agents/` (AGENT_SPECIFICATION_ALEX.md, AGENT_BA_CASS.md, AGENT_TESTER_NIGEL.md, AGENT_DEVELOPER_CODEY.md)
|
|
31
|
+
- Check: Business context directory has at least one file (not empty)
|
|
32
|
+
- Check: Skills installed (`.claude/commands/implement-feature.md` exists)
|
|
33
|
+
- Check: Node.js version meets requirements (>=18)
|
|
34
|
+
- Success output: Print success message with checkmarks for each passed check
|
|
35
|
+
- Failure output: Print what is missing with actionable fix suggestions for each failed check
|
|
36
|
+
- Exit code: 0 on all checks pass, non-zero on any failure
|
|
37
|
+
|
|
38
|
+
### Out of Scope
|
|
39
|
+
- Validation of file contents (e.g., SYSTEM_SPEC.md is well-formed)
|
|
40
|
+
- Validation of business context quality or completeness
|
|
41
|
+
- Automatic remediation (the command reports issues, not fixes them)
|
|
42
|
+
- Network checks (e.g., can reach skills.sh)
|
|
43
|
+
- Validation of queue state or in-progress features
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## 3. Actors Involved
|
|
48
|
+
|
|
49
|
+
### Human User (Developer)
|
|
50
|
+
- **Can do:** Invoke `orchestr8 validate` to check project readiness
|
|
51
|
+
- **Can do:** Review validation output to understand what is missing
|
|
52
|
+
- **Cannot do:** Auto-fix issues via this command (must run `init` or manually create files)
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## 4. Behaviour Overview
|
|
57
|
+
|
|
58
|
+
**Happy-path behaviour:**
|
|
59
|
+
1. User runs `orchestr8 validate` in a project directory
|
|
60
|
+
2. Command performs all checks in sequence
|
|
61
|
+
3. Each check prints a status line (checkmark for pass, X for fail)
|
|
62
|
+
4. If all checks pass, prints overall success message and exits with code 0
|
|
63
|
+
|
|
64
|
+
**Failure behaviour:**
|
|
65
|
+
1. User runs `orchestr8 validate` in an incomplete project
|
|
66
|
+
2. Command performs all checks in sequence
|
|
67
|
+
3. Failed checks print X with a description of what is missing
|
|
68
|
+
4. After all checks, failed items include actionable fix suggestions (e.g., "Run `orchestr8 init` to create .blueprint directory")
|
|
69
|
+
5. Command exits with non-zero exit code
|
|
70
|
+
|
|
71
|
+
**User-visible outcomes:**
|
|
72
|
+
- Clear, scannable output showing pass/fail status for each prerequisite
|
|
73
|
+
- Actionable guidance for resolving failures
|
|
74
|
+
- Machine-parseable exit code for scripting/CI integration
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## 5. State & Lifecycle Interactions
|
|
79
|
+
|
|
80
|
+
This feature is **state-inspecting only**. It does not create, transition, or modify any system state.
|
|
81
|
+
|
|
82
|
+
- **States inspected:** File system state (existence of files/directories), Node.js runtime version
|
|
83
|
+
- **States entered:** None
|
|
84
|
+
- **States exited:** None
|
|
85
|
+
- **States modified:** None
|
|
86
|
+
|
|
87
|
+
The command is idempotent and safe to run repeatedly without side effects.
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
## 6. Rules & Decision Logic
|
|
92
|
+
|
|
93
|
+
### R1: Directory Existence Check
|
|
94
|
+
- **Description:** Verify required directories exist
|
|
95
|
+
- **Inputs:** Directory paths (`.blueprint/`, `.business_context/`, `.claude/commands/`)
|
|
96
|
+
- **Outputs:** Pass/fail for each directory
|
|
97
|
+
- **Deterministic:** Yes
|
|
98
|
+
|
|
99
|
+
### R2: System Spec Existence Check
|
|
100
|
+
- **Description:** Verify SYSTEM_SPEC.md exists at expected path
|
|
101
|
+
- **Inputs:** Path `.blueprint/system_specification/SYSTEM_SPEC.md`
|
|
102
|
+
- **Outputs:** Pass/fail
|
|
103
|
+
- **Deterministic:** Yes
|
|
104
|
+
|
|
105
|
+
### R3: Agent Specs Existence Check
|
|
106
|
+
- **Description:** Verify all four agent specification files exist
|
|
107
|
+
- **Inputs:** Paths in `.blueprint/agents/` (AGENT_SPECIFICATION_ALEX.md, AGENT_BA_CASS.md, AGENT_TESTER_NIGEL.md, AGENT_DEVELOPER_CODEY.md)
|
|
108
|
+
- **Outputs:** Pass/fail, listing any missing files
|
|
109
|
+
- **Deterministic:** Yes
|
|
110
|
+
|
|
111
|
+
### R4: Business Context Non-Empty Check
|
|
112
|
+
- **Description:** Verify `.business_context/` contains at least one file
|
|
113
|
+
- **Inputs:** Directory listing of `.business_context/`
|
|
114
|
+
- **Outputs:** Pass/fail
|
|
115
|
+
- **Deterministic:** Yes
|
|
116
|
+
|
|
117
|
+
### R5: Skills Installed Check
|
|
118
|
+
- **Description:** Verify implement-feature skill is installed
|
|
119
|
+
- **Inputs:** Path `.claude/commands/implement-feature.md`
|
|
120
|
+
- **Outputs:** Pass/fail
|
|
121
|
+
- **Deterministic:** Yes
|
|
122
|
+
|
|
123
|
+
### R6: Node.js Version Check
|
|
124
|
+
- **Description:** Verify Node.js version is 18 or higher
|
|
125
|
+
- **Inputs:** `process.version`
|
|
126
|
+
- **Outputs:** Pass/fail with current version
|
|
127
|
+
- **Deterministic:** Yes
|
|
128
|
+
|
|
129
|
+
### R7: Exit Code Logic
|
|
130
|
+
- **Description:** Return exit code based on overall result
|
|
131
|
+
- **Inputs:** All check results
|
|
132
|
+
- **Outputs:** Exit code 0 if all pass, exit code 1 if any fail
|
|
133
|
+
- **Deterministic:** Yes
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## 7. Dependencies
|
|
138
|
+
|
|
139
|
+
### System Components
|
|
140
|
+
- `fs` module for file system checks
|
|
141
|
+
- `process.version` for Node.js version check
|
|
142
|
+
- CLI routing in `bin/cli.js`
|
|
143
|
+
|
|
144
|
+
### Technical Dependencies
|
|
145
|
+
- Node.js >= 18 (the check itself should work on lower versions to report the failure)
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## 8. Non-Functional Considerations
|
|
150
|
+
|
|
151
|
+
### Performance
|
|
152
|
+
- All checks are local file system operations; should complete in < 100ms
|
|
153
|
+
|
|
154
|
+
### Error Tolerance
|
|
155
|
+
- Command should not throw exceptions; all checks should be wrapped to handle missing paths gracefully
|
|
156
|
+
|
|
157
|
+
### User Experience
|
|
158
|
+
- Output should be colorized if terminal supports it (green checkmarks, red X marks)
|
|
159
|
+
- Output should be readable in non-color terminals (using ASCII fallback)
|
|
160
|
+
|
|
161
|
+
---
|
|
162
|
+
|
|
163
|
+
## 9. Assumptions & Open Questions
|
|
164
|
+
|
|
165
|
+
### Assumptions
|
|
166
|
+
- The four agent spec files have fixed names matching current convention
|
|
167
|
+
- `.business_context/` having any file (including README.md) counts as non-empty
|
|
168
|
+
- The Node.js version check should report failure but not crash on old versions
|
|
169
|
+
|
|
170
|
+
### Open Questions
|
|
171
|
+
- Should there be a `--json` flag for machine-readable output?
|
|
172
|
+
- Should there be a `--fix` flag that runs `init` if missing? (Deferred - out of scope)
|
|
173
|
+
|
|
174
|
+
---
|
|
175
|
+
|
|
176
|
+
## 10. Impact on System Specification
|
|
177
|
+
|
|
178
|
+
This feature **reinforces** existing system assumptions:
|
|
179
|
+
- Validates the prerequisites implied by "System spec gate" rule in Section 7
|
|
180
|
+
- Aligns with system boundary definition (CLI tooling in scope)
|
|
181
|
+
- Consistent with existing command patterns (`init`, `update`, `queue`)
|
|
182
|
+
|
|
183
|
+
No contradictions or tensions with the current system specification.
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## 11. Handover to BA (Cass)
|
|
188
|
+
|
|
189
|
+
### Story Themes
|
|
190
|
+
1. **Core validation flow** - User runs validate and sees results
|
|
191
|
+
2. **Success path** - All checks pass, user sees green checkmarks
|
|
192
|
+
3. **Failure path with guidance** - Checks fail, user sees actionable fixes
|
|
193
|
+
4. **Node.js version edge case** - Running on unsupported Node version
|
|
194
|
+
|
|
195
|
+
### Expected Story Boundaries
|
|
196
|
+
- Each check type could be a separate acceptance criterion within a single story
|
|
197
|
+
- Success/failure output formatting is a story concern
|
|
198
|
+
- Exit codes are a story concern (affects CI integration)
|
|
199
|
+
|
|
200
|
+
### Areas Needing Careful Story Framing
|
|
201
|
+
- The distinction between "check failed" and "command error" (the command itself should not fail/throw, even if checks fail)
|
|
202
|
+
- How to phrase fix suggestions to be helpful without being prescriptive
|
|
203
|
+
|
|
204
|
+
---
|
|
205
|
+
|
|
206
|
+
## 12. Change Log (Feature-Level)
|
|
207
|
+
| Date | Change | Reason | Raised By |
|
|
208
|
+
|------|--------|--------|-----------|
|
|
209
|
+
| 2026-02-24 | Initial feature specification | New feature request for pre-flight validation | Alex |
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Implementation Plan — Validate Command
|
|
2
|
+
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
Implement a new `orchestr8 validate` CLI command that performs pre-flight checks for directory existence, file presence, and Node.js version. The command outputs pass/fail status for each check with colorized indicators, provides actionable fix suggestions for failures, and returns appropriate exit codes (0 for success, 1 for failure).
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Files to Create/Modify
|
|
10
|
+
|
|
11
|
+
| Path | Action | Purpose |
|
|
12
|
+
|------|--------|---------|
|
|
13
|
+
| `src/validate.js` | Create | Core validation logic with `validate()`, `formatOutput()`, and `checkNodeVersion()` exports |
|
|
14
|
+
| `src/index.js` | Modify | Export validate module for library consumers |
|
|
15
|
+
| `bin/cli.js` | Modify | Add `validate` command routing |
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Implementation Steps
|
|
20
|
+
|
|
21
|
+
1. **Create `src/validate.js` with check functions**
|
|
22
|
+
- Implement `checkDirectories()` for `.blueprint/`, `.business_context/`, `.claude/commands/`
|
|
23
|
+
- Implement `checkSystemSpec()` for `.blueprint/system_specification/SYSTEM_SPEC.md`
|
|
24
|
+
- Implement `checkAgentSpecs()` for the four agent files in `.blueprint/agents/`
|
|
25
|
+
- Implement `checkBusinessContext()` to verify directory is non-empty
|
|
26
|
+
- Implement `checkSkillsInstalled()` for `.claude/commands/implement-feature.md`
|
|
27
|
+
- Implement `checkNodeVersion()` exported separately for tests, parsing `process.version`
|
|
28
|
+
|
|
29
|
+
2. **Implement main `validate()` function**
|
|
30
|
+
- Run all checks sequentially, collecting results in an array
|
|
31
|
+
- Each check returns `{ name, passed, message, fix?, detectedVersion? }`
|
|
32
|
+
- Compute `success` (all passed) and `exitCode` (0 or 1)
|
|
33
|
+
- Return `{ success, exitCode, checks }`
|
|
34
|
+
|
|
35
|
+
3. **Implement `formatOutput()` function**
|
|
36
|
+
- Accept result object and color flag (detect via `process.stdout.isTTY`)
|
|
37
|
+
- Use `✓`/`✗` for color terminals, `[PASS]`/`[FAIL]` for ASCII fallback
|
|
38
|
+
- Apply green/red ANSI codes when color is enabled
|
|
39
|
+
- Include fix suggestions after failed checks
|
|
40
|
+
- Print overall success/failure summary at end
|
|
41
|
+
|
|
42
|
+
4. **Add CLI routing in `bin/cli.js`**
|
|
43
|
+
- Import `validate` and `formatOutput` from `src/validate.js`
|
|
44
|
+
- Add `validate` command that runs validation, prints output, and calls `process.exit(result.exitCode)`
|
|
45
|
+
|
|
46
|
+
5. **Export from `src/index.js`**
|
|
47
|
+
- Add `validate` to module exports for programmatic usage
|
|
48
|
+
|
|
49
|
+
6. **Run tests and iterate**
|
|
50
|
+
- Execute `node --test test/feature_validate-command.test.js`
|
|
51
|
+
- Fix any failing assertions
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## Risks/Questions
|
|
56
|
+
|
|
57
|
+
- **Color detection**: Tests mock TTY state; implementation should use `process.stdout.isTTY` or accept a parameter for testability
|
|
58
|
+
- **Agent file names**: Tests expect exact names (AGENT_SPECIFICATION_ALEX.md, AGENT_BA_CASS.md, AGENT_TESTER_NIGEL.md, AGENT_DEVELOPER_CODEY.md) - verify these match production
|
|
59
|
+
- **Node version parsing**: Use `parseInt(process.version.slice(1).split('.')[0], 10)` to extract major version safely
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# Story — Validation Failure Output
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
As a developer, I want to see what is missing and how to fix it when validation checks fail so that I can resolve issues and get my project ready for the pipeline.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Context / scope
|
|
9
|
+
- Developer using orchestr8 CLI
|
|
10
|
+
- Project is missing one or more required artifacts
|
|
11
|
+
- This story covers failure output and actionable guidance
|
|
12
|
+
|
|
13
|
+
See feature spec: `.blueprint/features/feature_validate-command/FEATURE_SPEC.md`
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Acceptance criteria
|
|
18
|
+
|
|
19
|
+
**AC-1 — X mark displayed for failed checks**
|
|
20
|
+
- Given one or more required files/directories are missing,
|
|
21
|
+
- When I run `orchestr8 validate`,
|
|
22
|
+
- Then each failed check displays an X indicator.
|
|
23
|
+
|
|
24
|
+
**AC-2 — Colorized failure output when supported**
|
|
25
|
+
- Given my terminal supports color output,
|
|
26
|
+
- When I run `orchestr8 validate` and checks fail,
|
|
27
|
+
- Then X marks are displayed in red.
|
|
28
|
+
|
|
29
|
+
**AC-3 — Description of what is missing**
|
|
30
|
+
- Given a check fails,
|
|
31
|
+
- When the status line is printed,
|
|
32
|
+
- Then it includes a description of what is missing (e.g., "Missing: .blueprint/system_specification/SYSTEM_SPEC.md").
|
|
33
|
+
|
|
34
|
+
**AC-4 — Actionable fix suggestions**
|
|
35
|
+
- Given one or more checks fail,
|
|
36
|
+
- When all checks have completed,
|
|
37
|
+
- Then actionable fix suggestions are displayed:
|
|
38
|
+
- Missing `.blueprint/` directory: "Run `orchestr8 init` to initialize project"
|
|
39
|
+
- Missing agent specs: "Run `orchestr8 init` to create agent specification files"
|
|
40
|
+
- Missing skills: "Run `orchestr8 init` to install required skills"
|
|
41
|
+
- Empty business context: "Add at least one file to `.business_context/` directory"
|
|
42
|
+
- Node.js version: "Upgrade Node.js to version 18 or higher"
|
|
43
|
+
|
|
44
|
+
**AC-5 — Exit code non-zero on failure**
|
|
45
|
+
- Given one or more validation checks fail,
|
|
46
|
+
- When the command completes,
|
|
47
|
+
- Then the process exits with a non-zero exit code (1)
|
|
48
|
+
- And this exit code can be used in scripts/CI pipelines to detect failures.
|
|
49
|
+
|
|
50
|
+
**AC-6 — All checks still run on failure**
|
|
51
|
+
- Given the first check fails,
|
|
52
|
+
- When I run `orchestr8 validate`,
|
|
53
|
+
- Then all remaining checks are still executed
|
|
54
|
+
- And a complete picture of missing items is shown.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Out of scope
|
|
59
|
+
- `--fix` flag to automatically remediate issues
|
|
60
|
+
- Prioritization of which issues to fix first
|
|
61
|
+
- JSON output format (deferred)
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Story — Node.js Version Check
|
|
2
|
+
|
|
3
|
+
## User story
|
|
4
|
+
As a developer, I want the validation to check my Node.js version so that I am warned before attempting to run the pipeline on an unsupported runtime.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Context / scope
|
|
9
|
+
- Developer using orchestr8 CLI
|
|
10
|
+
- Node.js version may or may not meet minimum requirement (>=18)
|
|
11
|
+
- This check should work even on older Node.js versions to report the issue
|
|
12
|
+
|
|
13
|
+
See feature spec: `.blueprint/features/feature_validate-command/FEATURE_SPEC.md`
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Acceptance criteria
|
|
18
|
+
|
|
19
|
+
**AC-1 — Version check passes on Node 18+**
|
|
20
|
+
- Given I am running Node.js version 18 or higher,
|
|
21
|
+
- When I run `orchestr8 validate`,
|
|
22
|
+
- Then the Node.js version check displays a pass indicator.
|
|
23
|
+
|
|
24
|
+
**AC-2 — Version check fails on Node < 18**
|
|
25
|
+
- Given I am running Node.js version lower than 18,
|
|
26
|
+
- When I run `orchestr8 validate`,
|
|
27
|
+
- Then the Node.js version check displays a fail indicator
|
|
28
|
+
- And the current version is shown in the output.
|
|
29
|
+
|
|
30
|
+
**AC-3 — Command does not crash on old Node**
|
|
31
|
+
- Given I am running an older Node.js version,
|
|
32
|
+
- When I run `orchestr8 validate`,
|
|
33
|
+
- Then the command executes and reports the version failure
|
|
34
|
+
- And the command does not throw a runtime exception.
|
|
35
|
+
|
|
36
|
+
**AC-4 — Clear upgrade guidance**
|
|
37
|
+
- Given the Node.js version check fails,
|
|
38
|
+
- When fix suggestions are displayed,
|
|
39
|
+
- Then guidance includes "Upgrade Node.js to version 18 or higher"
|
|
40
|
+
- And the current detected version is shown.
|
|
41
|
+
|
|
42
|
+
**AC-5 — Version detected from runtime**
|
|
43
|
+
- Given I run `orchestr8 validate`,
|
|
44
|
+
- When the Node.js version check executes,
|
|
45
|
+
- Then the version is detected from `process.version` (not from package.json or external sources).
|
|
46
|
+
|
|
47
|
+
---
|
|
48
|
+
|
|
49
|
+
## Out of scope
|
|
50
|
+
- Checking for specific minor/patch versions
|
|
51
|
+
- Checking for Node.js feature availability beyond version number
|
|
52
|
+
- nvm or version manager integration
|