@signalridge/pi-workflows 1.10.0 → 1.10.1

package/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## 1.10.1
+### Patch Changes
+
+- c00f9f9: Wait for the exact terminal journal append receipt before publishing workflow completion, and recheck soft run/phase budgets after queue admission and before retry attempts.
+
 ## 1.10.0
 ### Minor Changes
 

package/README.md

@@ -62,7 +62,13 @@ This is not a wall-clock deadline for legitimate agent awaits, and never polls a
 model. Cancellation/failure closes the actual owned helper through TERM, KILL
 escalation after 250ms, and a bounded 1.5-second close wait. Managed children must
 still settle exact-owner cleanup before terminal publication or resumability; a
-cleanup failure is fenced, not claimed as success. A returned value closes new
+cleanup failure is fenced, not claimed as success. Terminal status, final results,
+waiters and background notifications also wait for the exact accepted terminal
+transition's journal append to succeed (an append receipt, not an fsync guarantee).
+A transient append rejection retains the immutable transition for bounded retries.
+Exhaustion never reports success: a receipted persistence failure may settle the run;
+otherwise terminal publication remains pending. Stop or branch replacement
+invalidates a queued success. A returned value closes new
 root-call admission but drains every accepted native operation and nested frame
 before success; callbacks needed by those operations remain available. Infinite
 awaits in forgotten helpers still require cancellation. Ignored native RPC promises
@@ -163,6 +169,14 @@ user's own LLM and run on the user's own machine.
 - `phase(title, { budget })`, `log(message)`, `args`, `cwd`, `process.cwd()`,
   `budget` (`{ total, spent(), remaining() }`).
 
+Run and phase token budgets are soft pre-dispatch gates: they recheck completed
+spend after acquiring runtime capacity and before every provider-agent attempt,
+including empty-output retries and schema repairs. Queued calls do not retain an
+earlier budget approval. Attempts already running can overshoot; this is not a
+hard token or cost cap. Nested live calls share run spend, while phase allowances
+remain frame-local and queued calls keep their captured phase allowance. Journal
+replay neither re-executes nor charges completed calls.
+
 ### Resume
 
 A script edit replays the longest unchanged prefix from the journal and runs

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@signalridge/pi-workflows",
-  "version": "1.10.0",
+  "version": "1.10.1",
   "description": "Deterministic JavaScript orchestration workflows backed by pi-subagents managed spawning.",
   "type": "module",
   "license": "MIT",

package/skills/workflow-authoring/SKILL.md

@@ -2,7 +2,7 @@
 name: workflow-authoring
 description: Design, write, review, and debug Pi JavaScript workflows that coordinate multiple subagents. Use when a request needs named dependencies, bounded fan-out, per-item pipelines, quality gates, saved or nested workflows, pause/resume, or workflow progress; do not use it merely to run an existing named workflow.
 metadata:
-  version: "1.10.0"
+  version: "1.10.1"
 ---
 
 # Workflow authoring

package/skills/workflow-authoring/references/capabilities.md

@@ -2,7 +2,7 @@
 # Workflow capability index
 
 Contract format: `2.0.0`<br>
-Contract content / skill / extension: `1.10.0`
+Contract content / skill / extension: `1.10.1`
 
 This compact generated index covers supported runtime globals and workflow-tool inputs. For constraints, compatibility behavior, internal boundaries, and dynamic-reference ownership, follow the [exhaustive generated facts](capability-details.md).
 

package/skills/workflow-authoring/references/capability-details.md

@@ -2,7 +2,7 @@
 # Exhaustive workflow capability facts
 
 Contract format: `2.0.0`<br>
-Contract content / skill / extension: `1.10.0`
+Contract content / skill / extension: `1.10.1`
 
 Every exact fact below is projected from the installed extension's capability contract. Explanatory judgment belongs in the hand-written references next to this file.
 

package/skills/workflow-authoring/references/lifecycle.md

@@ -8,7 +8,7 @@ Bound the work where the work is decided: graph tasks, fan-out items, loop round
 
 That pool is typically small — 4 slots unless the host was configured otherwise — so assume a wide fan-out runs in waves, not all at once. This does not change how wide to make it: the agents all run either way, and the reason to split work is coverage rather than wall-clock. It does mean a script should not promise the user a fast answer because it fanned out, and should not add a checkpoint or a progress phase that only makes sense if everything starts together.
 
-Enter a phase budget with `phase("Name", { budget: N })`; phase metadata does not carry budgets. `N` is a token allowance, not a call or round count: size it for the intended agent work instead of copying a small iteration limit. Token and phase budgets are soft pre-call gates. Spend lands after agents finish, so concurrent work can overshoot. A phase budget gates later calls in that phase; it neither reserves tokens nor cancels active calls. `budget.spent()` and `budget.remaining()` include nested work.
+Enter a phase budget with `phase("Name", { budget: N })`; phase metadata does not carry budgets. `N` is a token allowance, not a call or round count: size it for the intended agent work instead of copying a small iteration limit. Token and phase budgets are soft pre-dispatch gates, rechecked after acquiring runtime capacity and before every provider-agent attempt (including empty-output retries and schema repairs). Queued calls keep their captured phase allowance, not an earlier budget approval or a later phase rebase. Spend lands after agents finish, so concurrent work can overshoot; these gates are not hard token or cost caps. Phase allowances remain frame-local; a phase budget neither reserves tokens nor cancels active calls. `budget.spent()` and `budget.remaining()` include shared nested run spend.
 
 ## Checkpoints
 
@@ -30,6 +30,8 @@ Resume replays only the longest unchanged prefix of journaled calls. Once one ca
 
 Only a call that finishes with a real result is journaled. A call whose every attempt was recoverable (including one that only ever produced `AGENT_EMPTY_OUTPUT`) contributes no journal entry, so resuming that run reruns exactly that call and everything lexically after it live; the earlier, already-succeeded prefix still replays from cache. Schema repair retries include the concrete validation errors in the next prompt.
 
+Terminal status, final results, waiters and background notifications wait for the exact accepted terminal transition's journal append receipt, as well as existing owned-cleanup fences. This is an append guarantee, not an fsync guarantee. Transient append rejection retains the immutable transition for bounded retries. Exhaustion never reports success: a receipted persistence failure may settle the run; otherwise terminal publication remains pending. Stop or branch replacement invalidates a queued success.
+
 The runtime blocks common accidental nondeterminism, but this is not a security boundary. Pass timestamps, randomness, and external decisions through `args`.
 
 ## Nesting and shared state

package/skills/workflow-patterns/SKILL.md

@@ -2,7 +2,7 @@
 name: workflow-patterns
 description: Choose and run one of Pi's built-in multi-agent workflows for research, adversarial review, code review, multi-perspective analysis, or codebase auditing. Use when the request matches a named pattern; use workflow-authoring instead for a new script or custom topology.
 metadata:
-  version: "1.10.0"
+  version: "1.10.1"
 ---
 
 # Built-in workflow patterns

package/skills/workflow-review/SKILL.md

@@ -2,7 +2,7 @@
 name: workflow-review
 description: Audit an existing Pi JavaScript workflow for topology, prompt/data flow, limits, failure handling, cancellation, resume safety, and publication compatibility. Use when reviewing or debugging a workflow script or saved run; do not use to run a standard named workflow.
 metadata:
-  version: "1.10.0"
+  version: "1.10.1"
 ---
 
 # Workflow review

package/src/engine.ts

@@ -245,6 +245,8 @@ export interface ScriptRunState {
   >;
   /** Journal retry bookkeeping. */
   pendingJournal: Set<string>;
+  /** Exact terminal append still awaiting its receipt; not a published run status. */
+  pendingTerminal?: object;
   /** Whether a lifecycle suspension is in effect (branch change). */
   lifecycleSuspended: boolean;
 }
@@ -490,6 +492,8 @@ export class WorkflowEngine {
     const epoch = this.branchEpoch;
     if (
       !this.isCurrentState(state, generation, epoch) ||
+      !isTerminalWorkflow(state.run.status) ||
+      state.pendingTerminal !== undefined ||
       state.pendingCleanup.size > 0 ||
       this.terminalNotified.has(state)
     )
@@ -519,6 +523,7 @@ export class WorkflowEngine {
           state.pendingSpawns.size > 0 ||
           state.agentWaiters.size > 0 ||
           state.pendingJournal.size > 0 ||
+          state.pendingTerminal !== undefined ||
           state.pendingCleanup.size > 0 ||
           state.activeExecutionGenerations.size > 0 ||
           state.stopOperation !== undefined ||
@@ -1111,8 +1116,11 @@ export class WorkflowEngine {
       if (!(error instanceof WorkflowWaitAbortedError)) throw error;
       // Foreground cancellation owns execution, not just this waiter. Do not
       // advertise resumability until runtime children and checkpoint UI unwind.
+      // A failed receipt can outlive execution; retire it before aborting too.
+      state.pendingTerminal = undefined;
       state.controller.abort();
       await this.drainExecution(state, execution);
+      this.recoverInterruptedExecution(state, generation, epoch);
       const current = this.runs.get(runId)?.run ?? run;
       return { runId: current.runId, status: current.status, background: false, waitAborted: true };
     }
@@ -1274,8 +1282,10 @@ export class WorkflowEngine {
       return { runId, status: settled.status, background: false };
     } catch (error: unknown) {
       if (!(error instanceof WorkflowWaitAbortedError)) throw error;
+      state.pendingTerminal = undefined;
       state.controller.abort();
       await this.drainExecution(state, execution);
+      this.recoverInterruptedExecution(state, generation, epoch);
       return { runId, status: run.status, background: false, waitAborted: true };
     }
   }
@@ -1297,6 +1307,8 @@ export class WorkflowEngine {
     if (action === "get") return { action, run: this.summary(run, true) };
     if (action === "pause") {
       if (run.status === "running") {
+        const generation = state.executionGeneration;
+        const epoch = this.branchEpoch;
         this.setWorkflowStatus(run, "pausing");
         this.reconcilePendingSpawns(state);
         // Abort the script execution; the run settles as interrupted and can be
@@ -1304,6 +1316,7 @@ export class WorkflowEngine {
         // the runtime's abort signal via the dispatch waiters.
         state.controller.abort();
         await this.drainExecution(state, state.execution);
+        this.recoverInterruptedExecution(state, generation, epoch);
       } else if (run.status !== "paused" && run.status !== "interrupted") {
         throw new Error(`cannot pause workflow in state ${run.status}`);
       }
@@ -1381,7 +1394,15 @@ export class WorkflowEngine {
   }
 
   private settleWaiters(runId: string, run: ScriptRun): void {
-    if (this.runs.get(runId)?.run !== run || this.hasPendingCleanup(runId)) return;
+    const state = this.runs.get(runId);
+    if (
+      state?.run !== run ||
+      !this.isCurrentState(state, state.executionGeneration, this.branchEpoch) ||
+      state.pendingTerminal !== undefined ||
+      this.hasPendingCleanup(runId) ||
+      (!isTerminalWorkflow(run.status) && run.status !== "paused" && run.status !== "interrupted")
+    )
+      return;
     const all = this.waiters.get(runId);
     if (!all) return;
     for (const waiter of [...all]) {
@@ -1396,6 +1417,43 @@ export class WorkflowEngine {
 
   // ── execution ──────────────────────────────────────────────────────────────
 
+  private recoverInterruptedExecution(state: ScriptRunState, generation: number, epoch: number): void {
+    const run = state.run;
+    const current = () =>
+      this.runs.get(run.runId) === state &&
+      state.executionGeneration === generation &&
+      this.branchEpoch === epoch &&
+      !this.disposed &&
+      !state.stopOperation &&
+      (run.status === "running" || run.status === "pausing");
+    if (!current()) return;
+    // Cancellation takes ownership before interruption is visible. An abandoned
+    // completed/failed receipt must neither publish later nor block old waiters.
+    state.pendingTerminal = undefined;
+    const timestamp = Date.now();
+    const recovery: RunRecoveryEvent = {
+      kind: "run_recovery",
+      schemaVersion: JOURNAL_SCHEMA_VERSION,
+      runId: run.runId,
+      status: "interrupted",
+      branchGeneration: this.recoveryBranchGeneration,
+      rotations: [],
+      recoveryId: deriveRecoveryId({ runId: run.runId, rotations: [], timestamp }),
+      timestamp,
+    };
+    this.persist(run.runId, "abort-recovery", () => this.write(recovery));
+    if (!current()) return;
+    run.status = "interrupted";
+    run.updatedAt = timestamp;
+    // A pending failed receipt can outlive execute(). In that case there is no
+    // finally left to publish interruption or settle pre-existing waiters.
+    if (!state.activeExecutionGenerations.has(generation)) {
+      state.presentation.settle(run.status);
+      this.notifyInspection(state);
+      this.settleWaiters(run.runId, run);
+    }
+  }
+
   private async execute(
     state: ScriptRunState,
     script: string,
@@ -1579,7 +1637,13 @@ export class WorkflowEngine {
       const run = live?.run;
       if (run && isSameBranch() && !isTerminalWorkflow(run.status)) {
         this.setWorkflowStatus(run, "completed", undefined, undefined, result.result);
-        live.result = result.result;
+        // Keep execution ownership until this newly accepted terminal fact has
+        // its receipt too. Pause/stop may invalidate it while retries drain.
+        await this.drainJournal(runId);
+        if (!isCurrent() || executionSignal.aborted || run.status !== "completed")
+          throw new WorkflowError("workflow execution aborted", WorkflowErrorCode.WORKFLOW_ABORTED, {
+            recoverable: true,
+          });
       }
       return result;
     } catch (error: unknown) {
@@ -1593,20 +1657,7 @@ export class WorkflowEngine {
           this.cleanupAgents(runId);
           this.scheduleProviderResume(runId, workflowError.resetHint);
         } else if (workflowError.code === WorkflowErrorCode.WORKFLOW_ABORTED && executionSignal.aborted) {
-          this.persist(runId, "abort-recovery", () =>
-            this.write({
-              kind: "run_recovery",
-              schemaVersion: JOURNAL_SCHEMA_VERSION,
-              runId,
-              status: "interrupted",
-              branchGeneration: this.recoveryBranchGeneration,
-              rotations: [],
-              recoveryId: deriveRecoveryId({ runId, rotations: [], timestamp: Date.now() }),
-              timestamp: Date.now(),
-            }),
-          );
-          run.status = "interrupted";
-          run.updatedAt = Date.now();
+          this.recoverInterruptedExecution(state, generation, epoch);
         } else {
           this.cleanupAgents(runId);
           this.setWorkflowStatus(run, "failed", message);
@@ -2382,6 +2433,7 @@ export class WorkflowEngine {
     const epoch = this.branchEpoch;
     if (!this.isCurrentState(state, generation, epoch)) return;
     if (run.nonResumable && run.status === "stopped") return;
+    state.pendingTerminal = undefined;
     run.nonResumable = true;
     const blockedNodeIds = Object.keys(run.callStatus).filter((nodeId) => run.callStatus[nodeId] !== "completed");
     for (const nodeId of blockedNodeIds) run.callStatus[nodeId] = "stopped";
@@ -2573,9 +2625,9 @@ export class WorkflowEngine {
   /**
    * A Pi lifecycle event is delivered once. If its durable append is transiently
    * unavailable, retain the mutation in memory and retry the append with
-   * backoff without requiring the peer to re-emit. The in-memory run state is
-   * authoritative; the journal is best-effort with bounded retries, and a run
-   * whose appends keep failing is marked blocked (reported once, not spammed).
+   * backoff without requiring the peer to re-emit. Most in-memory facts can
+   * lead the journal; terminal transitions instead publish inside their exact
+   * append's receipt callback. Exhausted retries mark the journal blocked.
    */
   private scheduleJournalRetry(runId: string, key: string, mutation: () => void): void {
     // Every accepted fact has its own slot, including later retry generations
@@ -2660,10 +2712,8 @@ export class WorkflowEngine {
   }
 
   private write(event: JournalEvent): void {
-    // The in-memory mutation is applied by the caller BEFORE calling write;
-    // here we only persist. A transient failure schedules a retry of this exact
-    // append. Terminal bookkeeping already happened, so a retry that never
-    // succeeds degrades to a warning rather than corrupting run state.
+    // Only append here. persist turns rejection into a retry of this exact
+    // fact; receipt-gated callers publish only after write returns successfully.
     try {
       this.journal.append(event);
     } catch (error: unknown) {
@@ -2675,9 +2725,9 @@ export class WorkflowEngine {
   }
 
   /**
-   * Persist one journal event with bounded retry. The caller applies the
-   * in-memory mutation itself; `persist` only makes the append durable and, on
-   * a transient failure, schedules a retry of the exact same append.
+   * Persist one journal event with bounded retry. The caller owns in-memory
+   * publication; terminal transitions do it inside mutation after write returns.
+   * A transient failure schedules a retry of the exact same accepted append.
    */
   private persist(runId: string, key: string, mutation: () => void): void {
     const acceptedState = this.runs.get(runId);
@@ -2768,36 +2818,53 @@ export class WorkflowEngine {
     const generation = state?.executionGeneration ?? 0;
     const epoch = this.branchEpoch;
     if (!state || state.run !== run || !this.isCurrentState(state, generation, epoch)) return;
-    // Validate and isolate terminal values before best-effort persistence or any
-    // status mutation. A bad result must still be able to transition the run to failed.
+    // Capture once: retries must append the exact accepted transition, not a
+    // newly timestamped status or a result still owned by the runtime.
     const durableFinalResult = finalResult === undefined ? undefined : cloneWorkflowResult(finalResult);
     const runFinalResult = finalResult === undefined ? undefined : cloneWorkflowResult(finalResult);
     const timestamp = Date.now();
-    this.persist(run.runId, `transition:${status}`, () =>
-      this.write({
-        kind: "workflow_transition",
-        schemaVersion: JOURNAL_SCHEMA_VERSION,
-        runId: run.runId,
-        status,
-        ...(error !== undefined ? { error } : {}),
-        ...(durableFinalResult !== undefined ? { finalResult: durableFinalResult } : {}),
-        ...(terminalIntent !== undefined ? { terminalIntent } : {}),
-        timestamp,
-      }),
-    );
-    if (!this.isCurrentState(state, generation, epoch)) return;
-    run.status = status;
-    if (status === "running") run.error = undefined;
-    if (error !== undefined) run.error = error;
-    if (runFinalResult !== undefined) run.finalResult = runFinalResult;
-    if (terminalIntent !== undefined) run.terminalIntent = terminalIntent;
-    run.updatedAt = timestamp;
-    this.notifyInspection(state);
-    if (!this.isCurrentState(state, generation, epoch)) return;
-    if (isTerminalWorkflow(status)) {
-      state.bufferedTerminals.clear();
-      this.notifyTerminal(state);
-    }
+    const event: JournalEvent = {
+      kind: "workflow_transition",
+      schemaVersion: JOURNAL_SCHEMA_VERSION,
+      runId: run.runId,
+      status,
+      ...(error !== undefined ? { error } : {}),
+      ...(durableFinalResult !== undefined ? { finalResult: durableFinalResult } : {}),
+      ...(terminalIntent !== undefined ? { terminalIntent } : {}),
+      timestamp,
+    };
+    const terminal = isTerminalWorkflow(status);
+    const intent = terminal ? {} : undefined;
+    // Stop/pause takes ownership synchronously, invalidating any queued success.
+    state.pendingTerminal = intent;
+    const current = () =>
+      this.isCurrentState(state, generation, epoch) &&
+      (!terminal || (state.pendingTerminal === intent && (status !== "completed" || !state.controller.signal.aborted)));
+    const publish = () => {
+      if (!current()) return;
+      state.pendingTerminal = undefined;
+      run.status = status;
+      if (status === "running") run.error = undefined;
+      if (error !== undefined) run.error = error;
+      if (runFinalResult !== undefined) run.finalResult = runFinalResult;
+      if (status === "completed") state.result = runFinalResult;
+      if (terminalIntent !== undefined) run.terminalIntent = terminalIntent;
+      run.updatedAt = timestamp;
+      this.notifyInspection(state);
+      if (!this.isCurrentState(state, generation, epoch)) return;
+      if (terminal) {
+        state.bufferedTerminals.clear();
+        this.notifyTerminal(state);
+      }
+    };
+    this.persist(run.runId, `transition:${status}`, () => {
+      if (!current()) return;
+      this.write(event);
+      // write throws on append rejection inside persist, including its retries.
+      // Only a receipt from this exact append may publish terminal state/result.
+      if (terminal) publish();
+    });
+    if (!terminal) publish();
   }
 
   // ── restore / summary ──────────────────────────────────────────────────────

package/src/runtime.ts

@@ -1700,29 +1700,34 @@ export async function runWorkflow<T = unknown>(
     // Reserve a real dispatch synchronously — no await occurs between this
     // check and the increments, so a parallel() burst cannot overshoot the cap.
     if (shared.physicalAgentCount >= maxAgents) throw agentLimitError();
-    if (budget.total !== null && budget.remaining() <= 0) {
-      throw new WorkflowError("workflow token budget exhausted", WorkflowErrorCode.TOKEN_BUDGET_EXHAUSTED, {
-        recoverable: false,
-      });
-    }
-    // Per-phase soft sub-budget gate: warns once at ~80%, throws at 100%.
-    if (assignedPhase) {
-      const pb = state.phaseBudgets.get(assignedPhase);
-      if (pb) {
-        const phaseSpent = shared.spent - pb.startSpent;
-        if (phaseSpent >= pb.budget) {
+    // Capture this frame's phase allowance at lexical admission. A later phase
+    // change/rebase must not donate a fresh allowance to queued work or retries.
+    const phaseBudget = assignedPhase ? state.phaseBudgets.get(assignedPhase) : undefined;
+    const checkBudget = () => {
+      if (budget.total !== null && budget.remaining() <= 0) {
+        throw new WorkflowError("workflow token budget exhausted", WorkflowErrorCode.TOKEN_BUDGET_EXHAUSTED, {
+          recoverable: false,
+        });
+      }
+      // Soft gate: completed spend is shared, but active calls reserve no tokens.
+      if (phaseBudget) {
+        const phaseSpent = shared.spent - phaseBudget.startSpent;
+        if (phaseSpent >= phaseBudget.budget) {
           throw new WorkflowError(
-            `phase "${assignedPhase}" token sub-budget exhausted (${pb.budget})`,
+            `phase "${assignedPhase}" token sub-budget exhausted (${phaseBudget.budget})`,
             WorkflowErrorCode.TOKEN_BUDGET_EXHAUSTED,
             { recoverable: false },
           );
         }
-        if (!pb.warned && phaseSpent >= pb.budget * 0.8) {
-          pb.warned = true;
-          log(`phase "${assignedPhase}" at ${Math.round((phaseSpent / pb.budget) * 100)}% of its token sub-budget`);
+        if (!phaseBudget.warned && phaseSpent >= phaseBudget.budget * 0.8) {
+          phaseBudget.warned = true;
+          log(
+            `phase "${assignedPhase}" at ${Math.round((phaseSpent / phaseBudget.budget) * 100)}% of its token sub-budget`,
+          );
         }
       }
-    }
+    };
+    checkBudget();
     shared.agentCount++;
     frameAgentCount++;
     shared.physicalAgentCount++;
@@ -1737,6 +1742,7 @@ export async function runWorkflow<T = unknown>(
       await Promise.resolve();
       throwIfAborted();
       if (batch?.cancelled) throw agentLimitError();
+      checkBudget();
 
       const timeout =
         agentOptions.timeoutMs !== undefined ? normalizeAgentTimeout(agentOptions.timeoutMs) : agentTimeoutMs;
@@ -1787,6 +1793,9 @@ export async function runWorkflow<T = unknown>(
 
       try {
         for (let attempt = 1; attempt <= maxAttempts; attempt++) {
+          // Outside the attempt catch: a rejected admission is not a provider
+          // attempt and must not be charged an estimated failure or retried.
+          checkBudget();
           usage = undefined;
           const externalSignal = options.signal;
           let onExternalAbort: (() => void) | undefined;