orchestr8 2.7.1 → 3.0.0

package/.blueprint/features/feature_parallel-features/FEATURE_SPEC.md

@@ -0,0 +1,291 @@
+# Feature Specification — Parallel Features
+
+## 1. Feature Intent
+
+**Why this feature exists.**
+
+orchestr8 currently processes features sequentially through the Alex, Cass, Nigel, Codey pipeline. For projects with large backlogs, this creates a bottleneck: only one feature can be in progress at any time. The system specification explicitly notes this as a known limitation: "Pipeline is sequential; no parallel agent execution" and "Single-feature focus; no batch processing" (see `.blueprint/system_specification/SYSTEM_SPEC.md` Section 10).
+
+This feature enables **parallel execution of multiple feature pipelines** using git worktrees for isolation. Each feature runs in its own worktree/branch, allowing simultaneous development. Results are merged back to the main branch when complete.
+
+**Problem being addressed:**
+- Sequential processing is slow for large backlogs
+- Users with multiple features to implement must wait for each to complete
+- Development throughput is constrained by single-feature pipeline design
+
+**User need:**
+- Developers want to queue multiple features and have them processed concurrently
+- Teams want faster turnaround on feature backlogs
+
+**Alignment with system purpose:**
+- Extends the core pipeline without modifying agent behaviour
+- Maintains specification-first, test-driven approach per feature
+- Preserves traceability and artifact integrity within each worktree
+
+---
+
+## 2. Scope
+
+### In Scope
+
+- New CLI command: `orchestr8 parallel <slug1> <slug2> ... <slugN>`
+- Git worktree creation per feature (one worktree = one feature)
+- Independent pipeline execution in each worktree
+- Merge strategy for completed features back to main branch
+- Queue management for parallel pipelines
+- Status reporting across parallel executions
+- Conflict detection and handling on merge
+
+### Out of Scope
+
+- Parallelisation within a single feature pipeline (agents remain sequential per feature)
+- Cross-feature dependency resolution during parallel execution
+- CI/CD integration for parallel pipelines
+- IDE integration for worktree management
+- Custom merge strategies beyond standard git merge
+
+---
+
+## 3. Actors Involved
+
+### Human User
+- **Can do:** Invoke parallel command with multiple feature slugs, monitor progress, resolve merge conflicts, abort individual pipelines
+- **Cannot do:** Run parallel pipelines without git repository, modify running pipeline's worktree directly without risking state corruption
+
+### orchestr8 CLI
+- **Can do:** Create/manage worktrees, spawn parallel pipelines, merge completed features, report status
+- **Cannot do:** Automatically resolve merge conflicts (escalates to user)
+
+### Agent Pipeline (per worktree)
+- **Can do:** Execute standard Alex, Cass, Nigel, Codey flow in isolation
+- **Cannot do:** Access or modify other worktrees, communicate with parallel pipeline instances
+
+---
+
+## 4. Behaviour Overview
+
+### Happy Path
+
+1. User invokes `orchestr8 parallel feat-a feat-b feat-c`
+2. System validates git repository state (clean working tree, on main/target branch)
+3. For each feature slug:
+   - Create git worktree at `.claude/worktrees/feat-{slug}`
+   - Create new branch `feature/{slug}` from current HEAD
+   - Initialize pipeline queue in worktree
+4. Spawn parallel pipeline processes (one per worktree)
+5. Each pipeline executes independently: Alex, Cass, Nigel, Codey
+6. As each completes:
+   - Merge feature branch back to main (fast-forward when possible)
+   - Remove worktree
+   - Update aggregate status
+7. Report final status: completed, failed, merge conflicts
+
+### Key Alternatives
+
+- **Merge conflict:** Pipeline completes but merge fails. Feature branch preserved, user notified, manual resolution required.
+- **Pipeline failure:** Individual feature fails. Other pipelines continue. Failed worktree preserved for debugging.
+- **User abort:** Single feature or all features can be aborted. Aborted worktrees are cleaned up.
+
+### User-Visible Outcomes
+
+- Multiple features processed simultaneously (wall-clock time reduction)
+- Independent commits per feature on main branch
+- Clear status reporting per feature
+- Preserved worktrees on failure for inspection
+
+---
+
+## 5. State & Lifecycle Interactions
+
+### States Introduced
+
+| State | Description |
+|-------|-------------|
+| **parallel_queued** | Feature slug accepted for parallel processing |
+| **worktree_created** | Git worktree and branch created for feature |
+| **parallel_running** | Pipeline executing in worktree |
+| **merge_pending** | Pipeline complete, awaiting merge to main |
+| **merge_conflict** | Merge attempted but conflicts detected |
+| **parallel_complete** | Feature merged successfully |
+| **parallel_failed** | Pipeline or merge failed |
+
+### State Transitions
+
+```
+parallel_queued → worktree_created → parallel_running → merge_pending
+                                           ↓                  ↓
+                                    parallel_failed   ┌──────────────┐
+                                           ↓          │merge_conflict│
+                                    (preserve worktree)└──────────────┘
+                                                             ↓
+                                                   (user resolves)
+                                                             ↓
+                                                   parallel_complete
+```
+
+### This Feature Is
+
+- **State-creating:** Introduces parallel pipeline state model
+- **State-transitioning:** Manages transitions through parallel lifecycle
+- **State-constraining:** Enforces clean git state before starting
+
+---
+
+## 6. Rules & Decision Logic
+
+### Rule: Worktree Isolation
+
+- **Description:** Each feature runs in an isolated git worktree
+- **Inputs:** Feature slug, current git HEAD
+- **Outputs:** Worktree path, branch name
+- **Deterministic:** Yes - same slug always produces same paths
+
+### Rule: Concurrency Limit
+
+- **Description:** Maximum number of concurrent pipelines (default: 3, configurable)
+- **Inputs:** Requested feature count, `maxConcurrency` setting
+- **Outputs:** Number of pipelines to spawn, remaining queued
+- **Deterministic:** Yes - capped at limit, overflow queued
+
+### Rule: Merge Order
+
+- **Description:** Features merge in completion order (first finished, first merged)
+- **Inputs:** Completion timestamps per feature
+- **Outputs:** Merge sequence
+- **Deterministic:** No - depends on pipeline execution time
+
+### Rule: Conflict Escalation
+
+- **Description:** Merge conflicts escalate to user rather than auto-resolve
+- **Inputs:** Git merge result
+- **Outputs:** Success or conflict report with preserved branch
+- **Deterministic:** Yes - conflicts always escalate
+
+---
+
+## 7. Dependencies
+
+### System Components
+
+- Git (worktree support requires git 2.5+)
+- orchestr8 queue management (`src/orchestrator.js`)
+- Pipeline execution (`SKILL.md` / implement-feature)
+- Node.js child process spawning
+
+### Technical Dependencies
+
+- Clean git working tree (uncommitted changes block parallel start)
+- Sufficient disk space for worktrees (each is a full checkout)
+- Available system processes for concurrent execution
+
+### Operational Dependencies
+
+- Network access if agents require external resources
+- Claude API availability for all parallel pipelines
+
+---
+
+## 8. Non-Functional Considerations
+
+### Performance
+
+- Pipeline execution time reduced by parallelism factor (N features in ~1 pipeline time vs N x pipeline time)
+- Disk space increases linearly with concurrent worktrees
+- Memory/CPU scales with concurrent agent processes
+
+### Observability
+
+- Aggregate status command: `orchestr8 parallel status`
+- Per-feature status visible in each worktree's queue file
+- Completion/failure notifications per feature
+
+### Error Handling
+
+- Pipeline failures isolated to single worktree
+- Failed worktrees preserved for debugging (not auto-cleaned)
+- Partial success possible (some features complete, others fail)
+
+### Resource Management
+
+- Worktrees cleaned up on success
+- Failed/conflict worktrees require manual cleanup or `--cleanup` flag
+- Queue tracks worktree locations for recovery
+
+---
+
+## 9. Assumptions & Open Questions
+
+### Assumptions
+
+- Users have git 2.5+ installed (worktree support)
+- Working tree is clean before invoking parallel command
+- Sufficient disk space for N concurrent worktrees
+- Features are independent (no cross-feature code dependencies)
+- Main branch is the merge target (configurable?)
+
+### Open Questions
+
+1. **Merge strategy:** Should we support squash merge, rebase, or merge commit options?
+2. **Failure handling:** Should failed features block others from merging, or proceed independently?
+3. **Resource limits:** What is a sensible default for maxConcurrency? (Proposed: 3)
+4. **Progress reporting:** Real-time progress per worktree, or periodic aggregate status?
+5. **Branch naming:** Should branch names be configurable, or always `feature/{slug}`?
+6. **Cleanup policy:** Auto-cleanup failed worktrees after N days, or require explicit cleanup?
+
+---
+
+## 10. Impact on System Specification
+
+### Reinforces Existing Assumptions
+
+- Maintains agent role boundaries (each agent behaves identically in worktree)
+- Preserves artifact structure (same paths, just in worktree)
+- Keeps sequential agent flow per feature
+
+### Stretches Existing Assumptions
+
+- Queue model expands: system-level parallel queue + per-worktree queues
+- "Single current feature" invariant becomes "single current feature per worktree"
+- Recovery now spans multiple worktrees
+
+### Tensions Requiring Decision
+
+**Tension 1:** System spec states "Single current: Only one feature can be current at a time" (Section 7). This feature introduces multiple concurrent "current" features across worktrees.
+
+**Proposed resolution:** Reframe the invariant: "Only one feature can be current per pipeline instance (worktree)." The parallel orchestrator manages multiple pipeline instances, each respecting the single-current invariant internally.
+
+**Tension 2:** System spec notes "Pipeline is sequential" as a known limitation. This feature partially addresses this at the feature level but not the agent level.
+
+**Proposed resolution:** Update Known Limitations section to distinguish "sequential agents within feature" (unchanged) from "sequential features across pipeline" (now optional via parallel mode).
+
+---
+
+## 11. Handover to BA (Cass)
+
+### Story Themes
+
+1. **Parallel invocation:** User starts multiple features in parallel mode
+2. **Worktree lifecycle:** Creation, isolation, and cleanup of worktrees
+3. **Status monitoring:** User tracks progress across parallel pipelines
+4. **Merge handling:** Successful merges and conflict resolution
+5. **Failure recovery:** Handling failed pipelines without blocking others
+
+### Expected Story Boundaries
+
+- Each story should cover one actor's interaction with one lifecycle phase
+- Merge conflict story should be separate from happy-path merge
+- Status reporting could be its own story
+
+### Areas Needing Careful Story Framing
+
+- **Concurrency limits:** Story should clarify what happens when user requests more features than maxConcurrency
+- **Partial failure:** What user actions are available when some features succeed and others fail?
+- **Cleanup:** When and how are worktrees removed? Manual vs automatic?
+
+---
+
+## 12. Change Log (Feature-Level)
+
+| Date | Change | Reason | Raised By |
+|------|--------|--------|-----------|
+| 2026-02-25 | Initial feature specification | Feature proposed in FEATURE_IDEAS.md | Alex |

package/.blueprint/features/feature_parallel-features/IMPLEMENTATION_PLAN.md

@@ -0,0 +1,73 @@
+# Implementation Plan - Parallel Features
+
+## Summary
+
+Implement a new `src/parallel.js` module that enables concurrent execution of multiple feature pipelines using git worktrees for isolation. The module provides functions for worktree management, pre-flight validation, concurrency control, merge handling, status reporting, and state management. CLI integration adds `parallel` and `parallel status` commands to `bin/cli.js`.
+
+---
+
+## Files to Create/Modify
+
+| Path | Action | Purpose |
+|------|--------|---------|
+| `src/parallel.js` | Create | Core parallel execution module with all exported functions |
+| `bin/cli.js` | Modify | Add `parallel` and `parallel status` command routing |
+| `.claude/parallel-queue.json` | Runtime | State persistence for parallel pipelines (auto-created) |
+
+---
+
+## Implementation Steps
+
+1. **Create `src/parallel.js` with utility functions**
+   - `buildWorktreePath(slug)` - returns `.claude/worktrees/feat-{slug}`
+   - `buildBranchName(slug)` - returns `feature/{slug}`
+   - `getDefaultConfig()` - returns `{ maxConcurrency: 3 }`
+   - `getQueuePath(worktreePath)` - returns worktree-specific queue file path
+
+2. **Add pre-flight validation functions**
+   - `validatePreFlight({ isGitRepo, isDirty, gitVersion })` - returns `{ valid, errors }`
+   - `isGitVersionSupported(versionString)` - parses version, returns true if >= 2.5.0
+
+3. **Implement concurrency control**
+   - `splitByLimit(slugs, maxConcurrency)` - returns `{ active, queued }`
+   - `promoteFromQueue(state)` - moves first queued item to active when below limit
+
+4. **Add worktree lifecycle helpers**
+   - `shouldCleanupWorktree(state)` - returns true for `parallel_complete` or `aborted`, false for `parallel_failed` or `merge_conflict`
+
+5. **Implement merge handling functions**
+   - `canFastForward({ mainHead, branchBase })` - returns boolean
+   - `hasMergeConflict(gitOutput)` - detects "CONFLICT" in output
+   - `handleMergeConflict(state, conflictOutput?)` - transitions to `merge_conflict` status
+   - `orderByCompletion(features)` - sorts by `completedAt` timestamp
+
+6. **Add state management**
+   - `transition(state, newStatus)` - validates and applies state transitions
+   - Valid transitions: queued->worktree_created->running->merge_pending->complete
+   - Error transitions: running->failed, merge_pending->conflict
+
+7. **Implement status reporting**
+   - `formatStatus(states)` - multi-line status output
+   - `formatFeatureStatus(state)` - single feature line
+   - `summarizeFinal(results)` - returns `{ completed, failed, conflicts }`
+   - `aggregateResults(results)` - returns `{ completed, failed, total }`
+
+8. **Add user control functions**
+   - `abortFeature(states, slug)` - sets single feature to `aborted`
+   - `abortAll(states)` - sets all features to `aborted`
+
+9. **Implement pipeline execution helpers**
+   - `buildPipelineCommand(slug, worktreePath)` - constructs claude command string
+
+10. **Integrate with CLI in `bin/cli.js`**
+    - Add `parallel` command: parse slugs and flags, invoke parallel execution
+    - Add `parallel status` subcommand: invoke `formatStatus` with current state
+
+---
+
+## Risks/Questions
+
+- **Git availability**: Functions assume synchronous git operations via `child_process.execSync`; actual git calls deferred to integration phase
+- **Queue path collision**: Per-worktree queues use existing `QUEUE_PATH` pattern; ensure no conflicts with main queue
+- **Process spawning**: Actual concurrent pipeline spawning not covered by unit tests; requires integration testing
+- **Merge ordering**: First-completed-first-merged may cause conflicts; document as expected behavior

package/.blueprint/features/feature_parallel-lock/FEATURE_SPEC.md

@@ -0,0 +1,119 @@
+# Feature Specification — Parallel Lock
+
+## 1. Feature Intent
+
+Prevent running multiple parallel executions simultaneously by using a lock file. If a user accidentally runs `orchestr8 parallel` twice, the second invocation should fail with a clear message rather than creating chaos.
+
+**Problem:** Without locking, two parallel runs could create conflicting worktrees, compete for resources, and produce unpredictable results.
+
+**Solution:** Create a lock file on start, check for existing lock, clean up on completion.
+
+---
+
+## 2. Scope
+
+### In Scope
+- Create lock file with PID on start
+- Check if lock exists and process is running
+- Clean up lock on completion (success or failure)
+- `--force` flag to override lock (with warning)
+- Clear error message when locked
+
+### Out of Scope
+- Distributed locking (multiple machines)
+- Queue management for sequential runs
+
+---
+
+## 3. Behaviour Overview
+
+### First Run (No Lock)
+
+```
+$ orchestr8 parallel feat-a feat-b
+
+[Creates .claude/parallel.lock]
+Starting parallel pipelines...
+...
+[Removes .claude/parallel.lock on completion]
+```
+
+### Second Run (Locked)
+
+```
+$ orchestr8 parallel feat-c feat-d
+
+Error: Another parallel execution is in progress (PID: 12345)
+Started at: 2026-02-25T10:30:00Z
+
+Options:
+  • Wait for it to complete
+  • Run: orchestr8 parallel status
+  • Force override: orchestr8 parallel feat-c feat-d --force
+```
+
+### Force Override
+
+```
+$ orchestr8 parallel feat-c feat-d --force
+
+Warning: Overriding existing lock (PID: 12345)
+This may cause conflicts if another execution is still running.
+
+Continue? [y/N] y
+```
+
+### Stale Lock (Process Dead)
+
+```
+$ orchestr8 parallel feat-a feat-b
+
+Warning: Found stale lock file (PID 12345 not running)
+Removing stale lock and continuing...
+
+Starting parallel pipelines...
+```
+
+---
+
+## 4. Lock File Format
+
+Location: `.claude/parallel.lock`
+
+```json
+{
+  "pid": 12345,
+  "startedAt": "2026-02-25T10:30:00Z",
+  "features": ["feat-a", "feat-b"],
+  "maxConcurrency": 3
+}
+```
+
+---
+
+## 5. Implementation Notes
+
+- Check if PID is running: `process.kill(pid, 0)` returns without error if alive
+- Lock created after confirmation, before worktree creation
+- Lock removed in `finally` block to ensure cleanup
+- `--force` bypasses lock check but still creates new lock
+
+---
+
+## 6. Test Themes
+
+- Lock file created on start
+- Lock file removed on success
+- Lock file removed on failure
+- Error when lock exists with running PID
+- Stale lock detected and removed
+- `--force` flag overrides lock
+- Lock contains correct metadata
+
+---
+
+## 7. Change Log
+
+| Date | Change | Reason |
+|------|--------|--------|
+| 2026-02-25 | Initial spec | Parallel safeguards |

package/.blueprint/features/feature_parallel-logging/FEATURE_SPEC.md

@@ -0,0 +1,105 @@
+# Feature Specification — Parallel Logging
+
+## 1. Feature Intent
+
+Write each pipeline's output to a dedicated log file instead of interleaving on console. When multiple pipelines run simultaneously, their output becomes unreadable. Logging to files keeps the console clean while preserving full output for debugging.
+
+**Problem:** With 3 pipelines running in parallel, console output is an unreadable mess of interleaved messages from different agents.
+
+**Solution:** Write each pipeline's stdout/stderr to a log file; show only summary status on console.
+
+---
+
+## 2. Scope
+
+### In Scope
+- Log file per pipeline: `.claude/worktrees/feat-{slug}/pipeline.log`
+- Summary status on console (started, completed, failed)
+- `--verbose` flag to also stream to console
+- Timestamps on log entries
+
+### Out of Scope
+- Log rotation
+- Log compression
+- Remote log shipping
+
+---
+
+## 3. Behaviour Overview
+
+### Default (Quiet Console)
+
+```
+$ orchestr8 parallel feat-a feat-b feat-c
+
+Starting parallel pipelines...
+
+[10:30:01] feat-a: Started (log: .claude/worktrees/feat-feat-a/pipeline.log)
+[10:30:02] feat-b: Started (log: .claude/worktrees/feat-feat-b/pipeline.log)
+[10:30:03] feat-c: Started (log: .claude/worktrees/feat-feat-c/pipeline.log)
+[10:35:15] feat-b: Completed ✓
+[10:36:22] feat-a: Completed ✓
+[10:38:45] feat-c: Failed ✗ (see log for details)
+
+Summary: 2 completed, 1 failed
+```
+
+### Verbose Mode
+
+```
+$ orchestr8 parallel feat-a --verbose
+
+Starting parallel pipelines...
+
+[10:30:01] feat-a: Started
+--- feat-a output ---
+You are Alex, the System Specification Agent...
+[Alex output streams here]
+...
+--- end feat-a output ---
+
+[10:35:15] feat-a: Completed ✓
+```
+
+---
+
+## 4. Log File Format
+
+Location: `.claude/worktrees/feat-{slug}/pipeline.log`
+
+```
+[2026-02-25T10:30:01.123Z] Pipeline started for feat-a
+[2026-02-25T10:30:01.456Z] [stdout] You are Alex, the System Specification Agent...
+[2026-02-25T10:30:02.789Z] [stdout] Reading feature spec...
+[2026-02-25T10:35:15.000Z] [stderr] Warning: some warning
+[2026-02-25T10:35:15.100Z] Pipeline completed with exit code 0
+```
+
+---
+
+## 5. Implementation Notes
+
+- Change `stdio: 'inherit'` to `stdio: 'pipe'`
+- Create write stream to log file
+- Prefix each line with timestamp and stream type
+- On `--verbose`, also pipe to `process.stdout`
+- Console shows only status updates (not full output)
+
+---
+
+## 6. Test Themes
+
+- Log file created in worktree directory
+- Log contains timestamped output
+- Console shows only status by default
+- `--verbose` streams output to console
+- Log preserved on failure for debugging
+- Multiple pipelines don't interleave in logs
+
+---
+
+## 7. Change Log
+
+| Date | Change | Reason |
+|------|--------|--------|
+| 2026-02-25 | Initial spec | Parallel safeguards |