@caseyharalson/orrery 0.12.0 → 0.13.1

package/HELP.md

@@ -78,10 +78,13 @@ Options:
   --review        Enable code review loop after each step
   --parallel      Enable parallel execution with git worktrees for isolation
   --background    Run orchestration as a detached background process
+  --on-complete <command>  Run a shell command when the orchestrator finishes
 ```
 
-Only one `exec`/`resume` can run at a time per project. A lock file
-(`exec.lock`) prevents concurrent runs and is automatically cleaned up.
+A lock file prevents concurrent runs and is automatically cleaned up. Without
+`--plan`, a global lock (`exec.lock`) allows only one execution at a time.
+With `--plan`, each plan gets its own lock (`exec-<planId>.lock`) and runs in
+an isolated worktree, so multiple plans can execute concurrently.
 
 Example:
 
@@ -103,17 +106,21 @@ Options:
   --step <id>    Unblock a specific step before resuming
   --all          Unblock all blocked steps (default behavior)
   --dry-run      Preview what would be unblocked without making changes
+  --background   Run resume as a detached background process
+  --on-complete <command>  Run a shell command when the orchestrator finishes
 ```
 
-When `--plan` is provided, the plan's `work_branch` must match the current
-branch. If the plan hasn't been dispatched yet (no `work_branch`), use
-`orrery exec --plan` first.
+When `--plan` is provided and a worktree exists for the plan, resume runs
+directly inside the worktree. Otherwise, the plan's `work_branch` must match
+the current branch. If the plan hasn't been dispatched yet (no `work_branch`),
+use `orrery exec --plan` first.
 
 Example:
 
 ```bash
 orrery resume
 orrery resume --plan my-feature.yaml
+orrery resume --plan my-feature.yaml --background
 orrery resume --step step-2
 orrery resume --dry-run
 ```
@@ -264,16 +271,69 @@ max iteration limit is reached (default: 3).
 
 ### Background Execution
 
-Run orchestration as a detached background process:
+Run orchestration or resume as a detached background process:
 
 ```bash
 orrery exec --background
 orrery exec --plan my-feature.yaml --background
+orrery resume --plan my-feature.yaml --background
 ```
 
-The process runs detached and logs output to `<work-dir>/exec.log`. Use
-`orrery status` to check progress. A lock file prevents starting a second
-execution while one is already running.
+The process runs detached and logs output to `<work-dir>/exec.log` (or
+`exec-<planId>.log` for per-plan execution). Use `orrery status` to check
+progress. Each execution acquires its own lock file — a global `exec.lock` for non-plan
+runs, or `exec-<planId>.lock` when `--plan` is used — so background per-plan
+executions can run concurrently.
+
+### Concurrent Plan Execution
+
+Run multiple plans at the same time by using `--plan` in separate terminals.
+Each plan executes in its own git worktree (`.worktrees/plan-<planId>`) with
+an independent lock file, so plans do not interfere with each other.
+
+```bash
+# Terminal 1
+orrery exec --plan auth-feature.yaml
+
+# Terminal 2
+orrery exec --plan search-feature.yaml
+
+# Or run both in the background from a single terminal
+orrery exec --plan auth-feature.yaml --background
+orrery exec --plan search-feature.yaml --background
+```
+
+Use `orrery status` to see all running plans at once. Resume a specific plan
+with `orrery resume --plan <file>`, which automatically re-enters the plan's
+existing worktree.
+
+Note: Without `--plan`, execution uses a global lock and only one run is
+allowed at a time.
+
+### Completion Hook
+
+Run a shell command when the orchestrator finishes using `--on-complete`:
+
+```bash
+orrery exec --on-complete "notify-send 'Plan finished'"
+orrery resume --plan my-feature.yaml --on-complete "./scripts/on-done.sh"
+```
+
+The command receives plan context through environment variables:
+
+| Variable                 | Description                                    | Example                                                |
+| :----------------------- | :--------------------------------------------- | :----------------------------------------------------- |
+| `ORRERY_PLAN_NAME`       | Plan file name                                 | `my-feature.yaml`                                      |
+| `ORRERY_PLAN_FILE`       | Absolute path to the plan file                 | `/home/user/project/.agent-work/plans/my-feature.yaml` |
+| `ORRERY_PLAN_OUTCOME`    | Outcome: `success`, `partial`, or `incomplete` | `success`                                              |
+| `ORRERY_WORK_BRANCH`     | Work branch name                               | `plan/my-feature`                                      |
+| `ORRERY_SOURCE_BRANCH`   | Source branch name                             | `main`                                                 |
+| `ORRERY_PR_URL`          | PR URL (empty if not created)                  | `https://github.com/user/repo/pull/42`                 |
+| `ORRERY_STEPS_TOTAL`     | Total number of steps                          | `5`                                                    |
+| `ORRERY_STEPS_COMPLETED` | Number of completed steps                      | `4`                                                    |
+| `ORRERY_STEPS_BLOCKED`   | Number of blocked steps                        | `1`                                                    |
+
+Hook failures are logged but do not fail the orchestrator.
 
 ### Parallel Execution
 
@@ -393,11 +453,13 @@ Orrery maintains state in `.agent-work/` (configurable via `ORRERY_WORK_DIR`):
 
 ```
 .agent-work/
-  plans/        Active plan files (new and in-progress)
-  reports/      Step-level execution logs and outcomes
-  completed/    Successfully executed plans (archived)
-  exec.lock     Lock file when orchestration is running
-  exec.log      Output log for background execution
+  plans/                Active plan files (new and in-progress)
+  reports/              Step-level execution logs and outcomes
+  completed/            Successfully executed plans (archived)
+  exec.lock             Lock file when orchestration is running
+  exec-<planId>.lock    Per-plan lock file for concurrent execution
+  exec.log              Output log for background execution
+  exec-<planId>.log     Per-plan output log for concurrent background execution
 ```
 
 When `ORRERY_WORK_DIR` is set, Orrery automatically scopes to a project

package/README.md

@@ -89,6 +89,7 @@ For power users, Orrery offers additional capabilities:
 - **Review Loop** - Iterative code review after each step with automatic fixes
 - **Parallel Execution** - Run independent steps concurrently with git worktree isolation
 - **Background Execution** - Run orchestration as a detached process and poll status
+- **Completion Hook** - Run a command when orchestration finishes
 - **Handling Blocked Plans** - Recovery workflows when steps cannot complete
 
 See [Advanced Workflows](docs/advanced-workflows.md) for details.
@@ -121,14 +122,14 @@ The Orchestrator (`orrery exec`) is the engine that drives the process. It loads
 
 ## Command Reference
 
-| Command              | Description                                                                                                                                     |
-| :------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------- |
-| `orrery`             | Command reference.                                                                                                                              |
-| `orrery init`        | Initialize Orrery: install skills to detected agents.                                                                                           |
-| `orrery manual`      | Show the full CLI reference manual.                                                                                                             |
-| `orrery orchestrate` | Executes the active plan. Use `--review` for review loop, `--parallel` for parallel execution, `--background` for detached mode. Alias: `exec`. |
-| `orrery resume`      | Unblock steps and resume orchestration. Use `--plan` to target a specific plan.                                                                 |
-| `orrery status`      | Shows the progress of current plans and active execution status.                                                                                |
+| Command              | Description                                                                                                                                                                            |
+| :------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `orrery`             | Command reference.                                                                                                                                                                     |
+| `orrery init`        | Initialize Orrery: install skills to detected agents.                                                                                                                                  |
+| `orrery manual`      | Show the full CLI reference manual.                                                                                                                                                    |
+| `orrery orchestrate` | Executes the active plan. Use `--review` for review loop, `--parallel` for parallel execution, `--background` for detached mode, `--on-complete` for a completion hook. Alias: `exec`. |
+| `orrery resume`      | Unblock steps and resume orchestration. Use `--plan` to target a specific plan, `--background` for detached mode.                                                                      |
+| `orrery status`      | Shows the progress of current plans and active execution status.                                                                                                                       |
 
 ## Directory Structure
 

package/lib/cli/commands/orchestrate.js

@@ -4,6 +4,7 @@ const path = require("path");
 
 const { orchestrate } = require("../../orchestration");
 const { getWorkDir } = require("../../utils/paths");
+const { derivePlanId } = require("../../utils/git");
 
 module.exports = function registerOrchestrateCommand(program) {
   program
@@ -23,6 +24,10 @@ module.exports = function registerOrchestrateCommand(program) {
       "--background",
       "Run orchestration as a detached background process"
     )
+    .option(
+      "--on-complete <command>",
+      "Run a shell command when the orchestrator finishes"
+    )
     .action(async (options) => {
       // Background mode: re-spawn as detached process
       if (options.background) {
@@ -38,8 +43,15 @@ module.exports = function registerOrchestrateCommand(program) {
           if (options.resume) args.push("--resume");
           if (options.review) args.push("--review");
           if (options.parallel) args.push("--parallel");
+          if (options.onComplete)
+            args.push("--on-complete", options.onComplete);
 
-          const logFile = path.join(getWorkDir(), "exec.log");
+          let logFileName = "exec.log";
+          if (options.plan) {
+            const planId = derivePlanId(path.basename(options.plan));
+            logFileName = `exec-${planId}.log`;
+          }
+          const logFile = path.join(getWorkDir(), logFileName);
           const logFd = fs.openSync(logFile, "a");
 
           const binPath = path.join(
@@ -74,7 +86,8 @@ module.exports = function registerOrchestrateCommand(program) {
           verbose: options.verbose,
           resume: options.resume,
           review: options.review,
-          parallel: options.parallel
+          parallel: options.parallel,
+          onComplete: options.onComplete
         });
       } catch (error) {
         console.error(error && error.message ? error.message : error);

package/lib/cli/commands/resume.js

@@ -1,3 +1,4 @@
+const { spawn } = require("child_process");
 const fs = require("fs");
 const path = require("path");
 
@@ -6,8 +7,8 @@ const {
   loadPlan,
   updateStepsStatus
 } = require("../../orchestration/plan-loader");
-const { commit, getCurrentBranch } = require("../../utils/git");
-const { getPlansDir } = require("../../utils/paths");
+const { commit, getCurrentBranch, derivePlanId } = require("../../utils/git");
+const { getPlansDir, getWorkDir } = require("../../utils/paths");
 const { orchestrate } = require("../../orchestration");
 
 function supportsColor() {
@@ -36,7 +37,60 @@ module.exports = function registerResumeCommand(program) {
       "--dry-run",
       "Preview what would be unblocked without making changes"
     )
+    .option("--background", "Run resume as a detached background process")
+    .option(
+      "--on-complete <command>",
+      "Run a shell command when the orchestrator finishes"
+    )
     .action(async (options) => {
+      // Background mode: re-spawn as detached process
+      if (options.background) {
+        if (options.dryRun) {
+          console.log(
+            "Note: --background with --dry-run runs in foreground.\n"
+          );
+          // Fall through to normal execution
+        } else {
+          const args = [];
+          if (options.plan) args.push("--plan", options.plan);
+          if (options.step) args.push("--step", options.step);
+          if (options.all) args.push("--all");
+          if (options.onComplete)
+            args.push("--on-complete", options.onComplete);
+
+          let logFileName = "exec.log";
+          if (options.plan) {
+            const planId = derivePlanId(path.basename(options.plan));
+            logFileName = `exec-${planId}.log`;
+          }
+          const logFile = path.join(getWorkDir(), logFileName);
+          const logFd = fs.openSync(logFile, "a");
+
+          const binPath = path.join(
+            __dirname,
+            "..",
+            "..",
+            "..",
+            "bin",
+            "orrery.js"
+          );
+          const child = spawn(process.execPath, [binPath, "resume", ...args], {
+            detached: true,
+            stdio: ["ignore", logFd, logFd],
+            cwd: process.cwd(),
+            env: process.env
+          });
+
+          child.unref();
+          fs.closeSync(logFd);
+
+          console.log(`Background resume started (PID ${child.pid})`);
+          console.log(`Log file: ${logFile}`);
+          console.log("\nUse 'orrery status' to check progress.");
+          return;
+        }
+      }
+
       let planFile;
       let plan;
 
@@ -73,23 +127,34 @@ module.exports = function registerResumeCommand(program) {
           return;
         }
 
-        // Verify current branch matches plan's work_branch
-        let currentBranch;
-        try {
-          currentBranch = getCurrentBranch(process.cwd());
-        } catch {
-          console.error("Error detecting current branch.");
-          process.exitCode = 1;
-          return;
-        }
+        // Check for existing worktree — if found, skip branch validation
+        const planId = derivePlanId(path.basename(resolvedPath));
+        const worktreePath = path.join(
+          process.cwd(),
+          ".worktrees",
+          `plan-${planId}`
+        );
+        const hasWorktree = require("fs").existsSync(worktreePath);
 
-        if (currentBranch !== plan.metadata.work_branch) {
-          console.error(
-            `Plan expects branch '${plan.metadata.work_branch}' but you are on '${currentBranch}'.`
-          );
-          console.log(`\nRun: git checkout ${plan.metadata.work_branch}`);
-          process.exitCode = 1;
-          return;
+        if (!hasWorktree) {
+          // Verify current branch matches plan's work_branch
+          let currentBranch;
+          try {
+            currentBranch = getCurrentBranch(process.cwd());
+          } catch {
+            console.error("Error detecting current branch.");
+            process.exitCode = 1;
+            return;
+          }
+
+          if (currentBranch !== plan.metadata.work_branch) {
+            console.error(
+              `Plan expects branch '${plan.metadata.work_branch}' but you are on '${currentBranch}'.`
+            );
+            console.log(`\nRun: git checkout ${plan.metadata.work_branch}`);
+            process.exitCode = 1;
+            return;
+          }
         }
       } else {
         // 1. Find plan for current branch (existing behavior)
@@ -137,7 +202,11 @@ module.exports = function registerResumeCommand(program) {
         }
 
         console.log("Resuming orchestration...\n");
-        await orchestrate({ resume: true, plan: options.plan });
+        await orchestrate({
+          resume: true,
+          plan: options.plan,
+          onComplete: options.onComplete
+        });
         return;
       }
 
@@ -204,6 +273,10 @@ module.exports = function registerResumeCommand(program) {
 
       // 8. Resume orchestration
       console.log("\nResuming orchestration...\n");
-      await orchestrate({ resume: true, plan: options.plan });
+      await orchestrate({
+        resume: true,
+        plan: options.plan,
+        onComplete: options.onComplete
+      });
     });
 };

package/lib/cli/commands/status.js

@@ -1,10 +1,15 @@
+const fs = require("fs");
 const path = require("path");
 
-const { getPlansDir } = require("../../utils/paths");
+const {
+  getPlansDir,
+  getCompletedDir,
+  getWorkDir
+} = require("../../utils/paths");
 const { getPlanFiles, loadPlan } = require("../../orchestration/plan-loader");
 const { findPlanForCurrentBranch } = require("../../utils/plan-detect");
-const { getCurrentBranch } = require("../../utils/git");
-const { getLockStatus } = require("../../utils/lock");
+const { getCurrentBranch, derivePlanId } = require("../../utils/git");
+const { getLockStatus, listPlanLocks } = require("../../utils/lock");
 
 function supportsColor() {
   return Boolean(process.stdout.isTTY) && !process.env.NO_COLOR;
@@ -48,7 +53,21 @@ function resolvePlanPath(planArg) {
   if (!planArg) return null;
   if (path.isAbsolute(planArg)) return planArg;
   if (planArg.includes(path.sep)) return path.resolve(process.cwd(), planArg);
-  return path.join(getPlansDir(), planArg);
+
+  // Build candidate filenames: exact name, then with .yaml/.yml appended
+  const names = [planArg];
+  if (!planArg.endsWith(".yaml") && !planArg.endsWith(".yml")) {
+    names.push(`${planArg}.yaml`, `${planArg}.yml`);
+  }
+
+  // Check active plans first, then completed
+  for (const dir of [getPlansDir(), getCompletedDir()]) {
+    for (const name of names) {
+      const candidate = path.join(dir, name);
+      if (fs.existsSync(candidate)) return candidate;
+    }
+  }
+  return path.join(getPlansDir(), planArg); // default so caller's error message still works
 }
 
 function summarizePlans(plans) {
@@ -76,6 +95,14 @@ function renderPlanList(plans) {
 function renderPlanDetail(plan) {
   const status = getPlanStatus(plan);
   console.log(`${formatStatusLabel(status)} ${plan.fileName}`);
+  if (plan.metadata.completed_at) {
+    console.log(`  Completed: ${plan.metadata.completed_at}`);
+  }
+  const planId = derivePlanId(plan.fileName);
+  const logFile = path.join(getWorkDir(), `exec-${planId}.log`);
+  if (fs.existsSync(logFile)) {
+    console.log(`  Log: ${logFile}`);
+  }
   for (const step of plan.steps) {
     const stepLabel = formatStatusLabel(step.status || "pending");
     const description = step.description ? ` - ${step.description}` : "";
@@ -93,7 +120,7 @@ module.exports = function registerStatusCommand(program) {
     .description("Show orchestration status for plans in the current project")
     .option("--plan <file>", "Show detailed status for a specific plan")
     .action((options) => {
-      // Check for active execution
+      // Check for active global execution
       const lock = getLockStatus();
       if (lock.locked) {
         console.log(
@@ -105,12 +132,32 @@ module.exports = function registerStatusCommand(program) {
         );
       }
 
+      // Check for per-plan executions
+      const planLocks = listPlanLocks();
+      const activePlanLocks = planLocks.filter((l) => l.active);
+      const stalePlanLocks = planLocks.filter((l) => l.stale);
+
+      if (activePlanLocks.length > 0) {
+        console.log(`Active plan executions (${activePlanLocks.length}):`);
+        for (const pl of activePlanLocks) {
+          let line = `  - ${pl.planId} (PID ${pl.pid}, started ${pl.startedAt})`;
+          if (pl.worktreePath) line += `\n    worktree: ${pl.worktreePath}`;
+          console.log(line);
+        }
+        console.log();
+      }
+      if (stalePlanLocks.length > 0) {
+        console.log(
+          `Note: ${stalePlanLocks.length} stale per-plan lock(s) detected\n`
+        );
+      }
+
       const plansDir = getPlansDir();
       const planArg = options.plan;
 
       if (planArg) {
         const planPath = resolvePlanPath(planArg);
-        if (!planPath || !require("fs").existsSync(planPath)) {
+        if (!planPath || !fs.existsSync(planPath)) {
           console.error(`Plan not found: ${planArg}`);
           process.exitCode = 1;
           return;
@@ -135,16 +182,28 @@ module.exports = function registerStatusCommand(program) {
       }
 
       const planFiles = getPlanFiles(plansDir);
-      if (planFiles.length === 0) {
+      const completedDir = getCompletedDir();
+      const completedFiles = getPlanFiles(completedDir);
+
+      if (planFiles.length === 0 && completedFiles.length === 0) {
         console.log("No plans found");
         return;
       }
 
-      const plans = planFiles.map((planFile) => loadPlan(planFile));
-      const { pendingSteps, completedSteps } = summarizePlans(plans);
-      console.log(
-        `${plans.length} plans, ${pendingSteps} pending steps, ${completedSteps} completed`
-      );
-      renderPlanList(plans);
+      if (planFiles.length > 0) {
+        const plans = planFiles.map((planFile) => loadPlan(planFile));
+        const { pendingSteps, completedSteps } = summarizePlans(plans);
+        console.log(
+          `${plans.length} plans, ${pendingSteps} pending steps, ${completedSteps} completed`
+        );
+        renderPlanList(plans);
+      }
+
+      if (completedFiles.length > 0) {
+        if (planFiles.length > 0) console.log();
+        console.log(`Completed (${completedFiles.length}):`);
+        const completedPlans = completedFiles.map((f) => loadPlan(f));
+        renderPlanList(completedPlans);
+      }
     });
 };

package/lib/orchestration/agent-invoker.js

@@ -251,14 +251,28 @@ function createDefaultResult(stepId, exitCode, stderr) {
 }
 
 /**
- * Check if an error condition should trigger failover to another agent
+ * Check if agent stdout contains valid structured output (indicating a
+ * legitimate task-level result rather than an infrastructure failure).
+ * @param {string} stdout - Raw stdout from agent
+ * @returns {boolean}
+ */
+function hasValidAgentOutput(stdout) {
+  if (!stdout) return false;
+  const results = parseAgentResults(stdout);
+  return results.length > 0;
+}
+
+/**
+ * Check if an error condition should trigger failover to another agent.
+ * Failover triggers on any non-zero exit unless the agent produced valid
+ * structured output (indicating a legitimate task-level failure, not an
+ * infrastructure issue).
  * @param {Object} result - Process result with exitCode, stdout, stderr
  * @param {Error} spawnError - Error from spawn (if any)
  * @param {boolean} timedOut - Whether the process timed out
- * @param {Object} errorPatterns - Regex patterns for error detection
  * @returns {{shouldFailover: boolean, reason: string}}
  */
-function shouldTriggerFailover(result, spawnError, timedOut, errorPatterns) {
+function shouldTriggerFailover(result, spawnError, timedOut) {
   // 1. Spawn failures (command not found, ENOENT)
   if (spawnError) {
     if (spawnError.code === "ENOENT") {
@@ -272,23 +286,12 @@ function shouldTriggerFailover(result, spawnError, timedOut, errorPatterns) {
     return { shouldFailover: true, reason: "timeout" };
   }
 
-  // 3. Non-zero exit with error patterns in stderr (but NOT legitimate blocked)
+  // 3. Non-zero exit — failover unless agent produced valid structured output
   if (result && result.exitCode !== 0) {
-    const stderr = result.stderr || "";
-
-    // Check API error patterns
-    for (const pattern of errorPatterns.apiError || []) {
-      if (pattern.test(stderr)) {
-        return { shouldFailover: true, reason: "api_error" };
-      }
-    }
-
-    // Check token limit patterns
-    for (const pattern of errorPatterns.tokenLimit || []) {
-      if (pattern.test(stderr)) {
-        return { shouldFailover: true, reason: "token_limit" };
-      }
+    if (hasValidAgentOutput(result.stdout)) {
+      return { shouldFailover: false, reason: null };
     }
+    return { shouldFailover: true, reason: "agent_error" };
   }
 
   return { shouldFailover: false, reason: null };
@@ -513,8 +516,7 @@ function invokeAgentWithFailover(
         const { shouldFailover, reason } = shouldTriggerFailover(
           result,
           null,
-          result.timedOut,
-          failoverConfig.errorPatterns || {}
+          result.timedOut
         );
 
         if (shouldFailover && i < availableAgents.length - 1) {
@@ -535,8 +537,7 @@ function invokeAgentWithFailover(
         const { shouldFailover, reason } = shouldTriggerFailover(
           null,
           spawnError,
-          false,
-          failoverConfig.errorPatterns || {}
+          false
         );
 
         if (shouldFailover && i < availableAgents.length - 1) {
@@ -595,6 +596,7 @@ module.exports = {
   invokeAgentWithFailover,
   parseAgentResults,
   createDefaultResult,
+  shouldTriggerFailover,
   waitForAll,
   waitForAny
 };

package/lib/orchestration/config.js

@@ -146,30 +146,7 @@ module.exports = {
 
     // Timeout in milliseconds before trying next agent (15 minutes)
     // Can be overridden via ORRERY_AGENT_TIMEOUT environment variable
-    timeoutMs: 900000,
-
-    // Patterns to detect failover-triggering errors from stderr
-    errorPatterns: {
-      // API/connection errors
-      apiError: [
-        /API error/i,
-        /connection refused/i,
-        /ECONNRESET/i,
-        /ETIMEDOUT/i,
-        /network error/i,
-        /rate limit/i,
-        /429/,
-        /502/,
-        /503/
-      ],
-      // Token/context limit errors
-      tokenLimit: [
-        /token limit/i,
-        /context.*(limit|length|exceeded)/i,
-        /maximum.*tokens/i,
-        /too long/i
-      ]
-    }
+    timeoutMs: 900000
   },
 
   // Concurrency control