@litmers/cursorflow-orchestrator 0.1.13 → 0.1.15

package/CHANGELOG.md

@@ -5,9 +5,46 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.14] - 2025-12-21
+
+### Added
+- **Enhanced Logging System**: Comprehensive terminal output capture and management
+  - ANSI escape sequence stripping for clean, readable logs
+  - Automatic timestamps on each log line (ISO, relative, or short format)
+  - Log rotation with configurable max size and file retention
+  - Separate raw logs (with ANSI codes) and clean logs
+  - Structured JSON logs (`terminal.jsonl`) for programmatic access
+  - **Streaming JSON output** from cursor-agent (`--output-format stream-json`)
+  - Streaming message parser for real-time log processing
+  - Session headers with context (lane name, task, model, timestamps)
+- **Log Viewer CLI**: New `cursorflow logs` command
+  - View logs for specific lanes with `--lane <name>`
+  - **View all lanes merged**: `--all` or `-a` for unified timeline view
+  - Follow logs in real-time with `--follow` or `-f` (works with `--all`)
+  - Export to multiple formats: text, json, markdown, html
+  - Filter by regex pattern with `--filter`
+  - Filter by log level with `--level`
+  - Tail last N lines with `--tail`
+  - Color-coded lanes in merged view for easy identification
+- **Logging Configuration**: New `enhancedLogging` section in `cursorflow.config.js`
+  - `enabled`: Toggle enhanced logging (default: true)
+  - `stripAnsi`: Remove ANSI codes from clean logs (default: true)
+  - `addTimestamps`: Prepend timestamps to lines (default: true)
+  - `maxFileSize`: Rotation threshold in bytes (default: 50MB)
+  - `maxFiles`: Number of rotated files to keep (default: 5)
+  - `keepRawLogs`: Store raw logs separately (default: true)
+  - `writeJsonLog`: Generate structured JSON logs (default: true)
+  - `timestampFormat`: 'iso' | 'relative' | 'short' (default: 'iso')
+- **Full Lifecycle Integration Test**: Added `scripts/test-real-cursor-lifecycle.sh` (and `npm run test:lifecycle`) to verify end-to-end orchestration, result verification, and resource cleanup.
+- **Enhanced Log Customization**: Added `agentOutputFormat` to `cursorflow.config.js` to control `cursor-agent` output format (defaults to `stream-json`).
+- **Readable Logs by Default**: `cursorflow logs` now defaults to showing parsed, human-readable AI outputs for better visibility.
+- **Event System**: New `events.ts` module for structured event handling
+- **Webhook Support**: New `webhook.ts` for external integrations
+
 ## [0.1.13] - 2025-12-21
 
 ### Fixed
+- **Orchestration**: Fixed a bug where blocked lanes (exit code 2) were treated as completed, causing dependent lanes to start prematurely.
 - **Security**: Fixed command injection vulnerability in `checkDiskSpace` by using `spawnSync` instead of `execSync`.
 - **Workflow**: Updated security scanner to latest Semgrep Action.
 

package/README.md

@@ -101,7 +101,7 @@ Within the `cursorflow monitor` dashboard:
 {
   "baseBranch": "main",
   "branchPrefix": "feature/lane-1-",
-  "timeout": 300000,
+  "timeout": 600000,
   "enableIntervention": false,
   "dependsOn": ["01-lane-1"],
   "enableReview": true,
@@ -121,7 +121,7 @@ Within the `cursorflow monitor` dashboard:
 
 | Option | Type | Default | Description |
 |--------|------|---------|-------------|
-| `timeout` | number | 300000 | Task timeout in milliseconds (5 min) |
+| `timeout` | number | 600000 | Task timeout in milliseconds (10 min) |
 | `enableIntervention` | boolean | false | Enable stdin piping for intervention |
 | `model` | string | "sonnet-4.5" | AI model to use |
 | `dependsOn` | string[] | [] | Lane dependencies |
@@ -168,6 +168,87 @@ cursorflow doctor --tasks-dir _cursorflow/tasks/my-feature
 | `cursorflow clean` | Clean branches/worktrees |
 | `cursorflow signal` | Send message to running agent |
 | `cursorflow models` | List available AI models |
+| `cursorflow logs` | View, export, and follow logs |
+
+## 📝 Enhanced Logging
+
+CursorFlow provides comprehensive logging with automatic cleanup and export options:
+
+### Features
+- **ANSI Stripping**: Clean logs without terminal escape codes
+- **Timestamps**: Automatic timestamps on each line (ISO, relative, or short format)
+- **Log Rotation**: Automatic rotation when files exceed size limits
+- **Multiple Formats**: 
+  - `terminal.log` - Clean, readable logs
+  - `terminal-raw.log` - Raw logs with ANSI codes
+  - `terminal.jsonl` - Structured JSON for programmatic access
+
+### Usage
+
+```bash
+# View logs summary for latest run
+cursorflow logs
+
+# View specific lane logs
+cursorflow logs --lane api-setup
+
+# View ALL lanes merged (unified timeline)
+cursorflow logs --all
+
+# Follow all lanes in real-time
+cursorflow logs --all --follow
+
+# Follow logs in real-time
+cursorflow logs --lane api-setup --follow
+
+# Export to different formats
+cursorflow logs --lane api-setup --format json --output logs.json
+cursorflow logs --all --format html --output all-logs.html
+
+# Filter logs
+cursorflow logs --all --filter "error|failed"
+cursorflow logs --all --level stderr
+cursorflow logs --lane api-setup --level error --tail 50
+```
+
+### Merged Logs View (`--all`)
+
+When running multiple lanes, use `--all` to see a unified timeline:
+
+```
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+  🔀 Merged Logs - run-123 (45 entries from 3 lanes)
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+  Lanes: ■ api-setup  ■ frontend  ■ database
+
+[10:15:30] [api-setup   ] [STDOUT] Starting API setup...
+[10:15:31] [frontend    ] [STDOUT] Setting up React...
+[10:15:32] [database    ] [STDOUT] Creating schema...
+[10:15:33] [api-setup   ] [STDOUT] Endpoints created
+[10:15:34] [frontend    ] [STDERR] Warning: Deprecated API
+...
+```
+
+### Configuration
+
+Add to `cursorflow.config.js`:
+
+```javascript
+module.exports = {
+  // ... other config ...
+  enhancedLogging: {
+    enabled: true,           // Enable enhanced logging
+    stripAnsi: true,         // Strip ANSI codes for clean logs
+    addTimestamps: true,     // Add timestamps to each line
+    maxFileSize: 52428800,   // 50MB max before rotation
+    maxFiles: 5,             // Keep 5 rotated files
+    keepRawLogs: true,       // Keep raw logs separately
+    writeJsonLog: true,      // Generate JSON logs
+    timestampFormat: 'iso',  // 'iso' | 'relative' | 'short'
+  },
+};
+```
 
 ## 📖 Documentation
 

package/commands/cursorflow-clean.md

@@ -1,7 +1,7 @@
 # CursorFlow Clean
 
 ## Overview
-Clean up temporary resources created by CursorFlow, including Git worktrees, feature branches, and log files.
+Clean up temporary resources created by CursorFlow, including Git worktrees, feature branches, log files, and task definitions.
 
 ## Usage
 
@@ -16,7 +16,8 @@ cursorflow clean <type> [options]
 | `branches` | Remove local feature branches created by CursorFlow |
 | `worktrees` | Remove temporary Git worktrees |
 | `logs` | Clear all run and terminal logs |
-| `all` | Clean everything (branches, worktrees, and logs) |
+| `tasks` | Remove task definition directories (keeps `example/`) |
+| `all` | Clean everything (branches, worktrees, logs, and tasks) |
 
 ## Options
 
@@ -24,23 +25,29 @@ cursorflow clean <type> [options]
 |------|------|
 | `--dry-run` | Show what would be removed without actually deleting anything |
 | `--force` | Force removal (ignore uncommitted changes in worktrees) |
+| `--include-latest` | Also remove the most recent item (by default, latest is kept) |
 | `--help`, `-h` | Show help |
 
 ## Examples
 
-### Review before deleting
+### Review before deleting (latest is kept by default)
 ```bash
 cursorflow clean all --dry-run
 ```
 
-### Clean only worktrees
+### Clean only worktrees (keeps the latest worktree)
 ```bash
 cursorflow clean worktrees
 ```
 
-### Force clean everything
+### Force clean everything including the latest
 ```bash
-cursorflow clean all --force
+cursorflow clean all --force --include-latest
+```
+
+### Remove all worktrees including the latest
+```bash
+cursorflow clean worktrees --include-latest
 ```
 
 ## Notes
@@ -49,3 +56,10 @@ cursorflow clean all --force
 2. **Worktrees**: The command identifies CursorFlow worktrees by their location (usually in `_cursorflow/worktrees/`) or their prefix.
 3. **Branches**: Only branches starting with the configured `branchPrefix` (default: `cursorflow/`) are targeted.
 4. **Irreversible**: Once logs are deleted, they cannot be recovered.
+5. **Default Behavior**: By default, the most recent item is preserved. The "most recent" is determined by:
+   - **Worktrees**: Directory modification time
+   - **Branches**: Latest commit timestamp
+   - **Logs**: File/directory modification time
+   - **Tasks**: Directory modification time
+   
+   Use `--include-latest` to remove everything including the most recent item.

package/commands/cursorflow-prepare.md

@@ -380,7 +380,7 @@ _cursorflow/tasks/2412211530_FeatureName/
 {
   "baseBranch": "main",
   "branchPrefix": "featurename/lane-1-",
-  "timeout": 300000,
+  "timeout": 600000,
   "enableIntervention": false,
   "dependencyPolicy": {
     "allowDependencyChange": false,

package/commands/cursorflow-resume.md

@@ -1,12 +1,14 @@
 # CursorFlow Resume
 
 ## Overview
-Resume a lane that was interrupted or failed. CursorFlow allows you to continue from where the agent left off or restart the lane from the first task.
+Resume lanes that were interrupted or failed. CursorFlow allows you to continue from where the agent left off, restart from the first task, or resume all incomplete lanes at once.
 
 ## Usage
 
 ```bash
-cursorflow resume <lane-name> [options]
+cursorflow resume [lane-name] [options]
+cursorflow resume --status              # Check status of all lanes
+cursorflow resume --all                 # Resume all incomplete lanes
 ```
 
 ## Options
@@ -14,17 +16,82 @@ cursorflow resume <lane-name> [options]
 | Option | Description |
 |------|------|
 | `<lane-name>` | The name of the lane to resume (e.g., `lane-1`) |
+| `--status` | Show status of all lanes in the run (no resume) |
+| `--all` | Resume ALL incomplete/failed lanes automatically |
 | `--run-dir <path>` | Use a specific run directory (default: latest) |
-| `--restart` | Restart the lane from the very first task |
+| `--max-concurrent <n>` | Max lanes to run in parallel when using `--all` (default: 3) |
+| `--restart` | Restart from the first task (index 0) |
 | `--clean` | Clean up the existing worktree before resuming |
 | `--skip-doctor` | Skip pre-resume checks (not recommended) |
 
+## Checking Lane Status
+
+Before resuming, you can check the status of all lanes:
+
+```bash
+cursorflow resume --status
+```
+
+This displays a table showing:
+- Lane name
+- Current status (completed, failed, running, paused, etc.)
+- Progress (current task / total tasks)
+- Whether the lane needs to be resumed
+
+Example output:
+```
+📊 Lane Status (run-1703145600000)
+
+  Lane                     Status      Progress    Needs Resume
+  ------------------------------------------------------------
+  01-lane-auth             completed   3/3         
+  02-lane-api              failed      1/3         ✓
+   └─ Error: cursor-agent timed out...
+  03-lane-ui               running     2/3         ✓
+
+  Total: 3 | Completed: 1 | Needs Resume: 2
+
+  Tip: Run cursorflow resume --all to resume all incomplete lanes
+```
+
+## Resuming All Incomplete Lanes
+
+The most common use case after interruption:
+
+```bash
+# Check what needs to be resumed
+cursorflow resume --status
+
+# Resume all incomplete lanes
+cursorflow resume --all
+
+# Resume with custom concurrency
+cursorflow resume --all --max-concurrent 2
+
+# Restart all incomplete lanes from the beginning
+cursorflow resume --all --restart
+```
+
 ## How it works
+
+### Single Lane Resume
 1. **Pre-flight Checks**: Runs doctor validation to check for branch conflicts and Git issues.
 2. **State Loading**: Reads the `state.json` for the specified lane to find the last successful task.
 3. **Environment Restore**: Verifies the Git worktree and branch for the lane.
 4. **Execution**: Spawns a new runner that starts either from the current task index or from index 0 if `--restart` is used.
 
+### Resume All (`--all`)
+1. **Status Check**: Scans all lanes in the run directory.
+2. **Filter**: Identifies lanes that need resuming (failed, paused, interrupted).
+3. **Dependency Analysis**: 
+   - Checks each lane's `dependsOn` field
+   - Skips lanes with unresolvable dependencies (deps not completed and not in resume list)
+   - Orders execution so lanes wait for their dependencies to complete first
+4. **Pre-flight Checks**: Runs doctor validation once for the entire run.
+5. **Parallel Execution**: Spawns runners for multiple lanes with concurrency control.
+6. **Dependency-Aware Scheduling**: Only starts a lane when all its dependencies have completed.
+7. **Progress Tracking**: Reports success/failure/skipped for each lane.
+
 ## Pre-resume Validation
 
 Before resuming, CursorFlow automatically runs validation checks:
@@ -35,11 +102,22 @@ Before resuming, CursorFlow automatically runs validation checks:
 To skip these checks (not recommended):
 ```bash
 cursorflow resume lane-1 --skip-doctor
+cursorflow resume --all --skip-doctor
 ```
 
 ## Examples
 
-### Resume a failed lane
+### Check status of all lanes
+```bash
+cursorflow resume --status
+```
+
+### Resume all incomplete lanes
+```bash
+cursorflow resume --all
+```
+
+### Resume a single failed lane
 ```bash
 cursorflow resume 01-lane-1
 ```
@@ -51,13 +129,49 @@ cursorflow resume 02-lane-2 --restart
 
 ### Resume from an older run
 ```bash
-cursorflow resume 01-lane-1 --run-dir _cursorflow/logs/runs/run-123456789/
+cursorflow resume --status --run-dir _cursorflow/logs/runs/run-123456789/
+cursorflow resume --all --run-dir _cursorflow/logs/runs/run-123456789/
 ```
 
+### Resume with limited parallelism
+```bash
+cursorflow resume --all --max-concurrent 1  # One at a time
+```
+
+## Dependency Handling
+
+When using `--all`, CursorFlow respects the `dependsOn` field in each lane's configuration:
+
+- **Automatic ordering**: Lanes will wait for their dependencies to complete before starting
+- **Skipped lanes**: If a lane depends on another that isn't completed and isn't in the resume queue, it will be skipped
+- **Parallel with deps**: Independent lanes run in parallel; dependent lanes wait
+
+Example status output with dependencies:
+```
+📊 Lane Status (run-1703145600000)
+
+  Lane                     Status      Progress    DependsOn      Resumable
+  ---------------------------------------------------------------------------
+  01-lane-core             completed   3/3         -              
+  02-lane-api              failed      1/3         01-lane-core   ✓
+  03-lane-ui               failed      0/3         02-lane-api    ⏳ waiting
+   └─ waiting for: 02-lane-api
+
+  Total: 3 | Completed: 1 | Needs Resume: 2
+
+  Tip: Run cursorflow resume --all to resume all incomplete lanes
+       Lanes with dependencies will wait until their dependencies complete.
+```
+
+In this example:
+- `01-lane-core` is already completed
+- `02-lane-api` can start immediately (its dependency `01-lane-core` is completed)
+- `03-lane-ui` will wait until `02-lane-api` completes
+
 ## Troubleshooting
 
 ### State not found
-If the command fails because the state is missing, ensure you are providing the correct lane name. Use `cursorflow monitor` to see the names of the lanes in the latest run.
+If the command fails because the state is missing, ensure you are providing the correct lane name. Use `cursorflow resume --status` to see the names of the lanes in the latest run.
 
 ### Worktree issues
 If the worktree directory was manually deleted, use the `--clean` or `--restart` flag to allow CursorFlow to recreate the environment.
@@ -82,3 +196,10 @@ If the task JSON file's `branchPrefix` was changed after the initial run:
 1. Either restore the original prefix in the JSON
 2. Or use `--restart` to start fresh with the new prefix
 3. Or manually clean up old branches with `cursorflow clean branches`
+
+### Some lanes still failing after `--all`
+If some lanes continue to fail after using `--all`:
+1. Check the specific error with `cursorflow resume --status`
+2. Try resuming the problematic lane individually with more visibility
+3. Use `cursorflow monitor` to watch the lane in real-time
+4. Check the lane's terminal log in `_cursorflow/logs/runs/<run>/lanes/<lane>/`

package/commands/cursorflow-run.md

@@ -136,7 +136,7 @@ Each JSON file in the tasks directory follows this schema:
 {
   "baseBranch": "main",
   "branchPrefix": "feature/lane-1-",
-  "timeout": 300000,
+  "timeout": 600000,
   "enableIntervention": false,
   "dependsOn": ["01-lane-1"],
   "dependencyPolicy": {
@@ -165,7 +165,7 @@ Each JSON file in the tasks directory follows this schema:
 | `branchPrefix` | string | Yes | Prefix for feature branch naming |
 | `tasks` | Task[] | Yes | Array of task objects to execute |
 | `dependsOn` | string[] | No | Lane names to wait for and merge |
-| `timeout` | number | No | Task timeout in ms (default: 300000) |
+| `timeout` | number | No | Task timeout in ms (default: 600000) |
 | `enableIntervention` | boolean | No | Allow stdin injection during execution |
 | `dependencyPolicy.allowDependencyChange` | boolean | No | Allow package.json modifications |
 | `dependencyPolicy.lockfileReadOnly` | boolean | No | Keep lockfile read-only |

package/commands/cursorflow-signal.md

@@ -7,6 +7,7 @@ Directly intervene in a running lane by sending a message to the agent. This is
 
 ```bash
 cursorflow signal <lane-name> "<message>" [options]
+cursorflow signal <lane-name> --timeout <ms>
 ```
 
 ## Options
@@ -15,24 +16,30 @@ cursorflow signal <lane-name> "<message>" [options]
 |------|------|
 | `<lane-name>` | The name of the lane to signal |
 | `"<message>"` | The text message to send to the agent |
+| `--timeout <ms>` | Update the execution timeout (in milliseconds) |
 | `--run-dir <path>` | Use a specific run directory (default: latest) |
 
 ## How it works
-1. **Logging**: The message is recorded in the lane's conversation history as a system/commander message.
-2. **Injection**: If the lane's task configuration has `enableIntervention: true`, the message is injected into the agent's input stream.
+1. **Logging**: Intervention messages are recorded in the lane's conversation history.
+2. **Injection**: If `enableIntervention: true`, messages are injected into the agent's input stream.
+3. **Dynamic Timeout**: If `--timeout` is used, the active runner receives a signal to reset its internal timer to the new value.
 
-## Example
+## Examples
 
 ```bash
 # Provide a hint to a running agent
 cursorflow signal 01-lane-1 "Make sure to export the new function from index.ts"
+
+# Increase timeout to 10 minutes mid-execution
+cursorflow signal 01-lane-1 --timeout 600000
 ```
 
 ## Dashboard Alternative
 You can also use the interactive monitor to send signals:
 1. Run `cursorflow monitor latest`.
 2. Select a lane and enter details (`→`).
-3. Press `I` to type and send an intervention message.
+3. Press `I` to send an intervention message.
+4. Press `O` to update the execution timeout.
 
 ## Note on Intervention
 For the agent to receive the signal immediately, the task must be configured with: