@osolmaz/pi-workflows 0.15.2 → 0.15.3

package/docs/2026-09-01-restore-session-delivery-controls-plan.md

@@ -0,0 +1,139 @@
+---
+title: Restore workflow session delivery and controls
+author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
+date: 2026-09-01
+---
+
+# Restore workflow session delivery and controls
+
+The out-of-process workflow host removed the Pi workflow widget and Escape-to-pause behavior. A later delivery safety fix also caused normal polling to report `Interactive request presentation claim conflict`. This plan restores those features and fixes delivery without bringing back the embedded workflow runtime.
+
+[Workflow host](WORKFLOW_HOST.md) remains the process and state specification. [Workflow step messages](WORKFLOW_STEP_MESSAGES.md) remains the session message specification. This plan records the cause, scope, implementation order, and acceptance checks for the repair.
+
+## Observed problems
+
+- The active workflow widget no longer appears in Pi.
+- Escape aborts the current model turn but leaves its workflow unpaused.
+- Polling can report a presentation claim conflict before the workflow step appears.
+- A visible message can become eligible for another send if saving its durable receipt fails.
+
+The affected live run remained durably parked at its interactive request. No workflow state or Pi session message was lost.
+
+## Root cause
+
+The out-of-process redesign removed the embedded extension executor. The old widget and `agent_end` pause handler were coupled to that executor, so both were removed with it. The redesign did not add host-backed replacements.
+
+The delivery coordinator checks Pi again after an asynchronous host claim. If Pi becomes busy during that claim, the coordinator currently drops the claim. The next poll tries to claim the same interaction before its ten-second presentation lease expires. The host correctly rejects that second claim. The extension incorrectly exposes the expected rejection as a workflow tool failure.
+
+The coordinator also removes its local queued guard before durable settlement finishes. A settlement error can therefore make later polling treat a message that is already visible in Pi as sendable work.
+
+## Requirements
+
+- Keep one global host as the normal workflow state writer.
+- Keep workflow and controller code in supervised child processes.
+- Use documented Pi extension APIs only.
+- Preserve one ordered session delivery path for steps, decisions, notifications, and final results.
+- Never send through an expired claim.
+- Never resend a message that is visible in Pi only because its durable receipt failed.
+- Pause a presented workflow interaction when its Pi model turn ends with stop reason `aborted`.
+- Reject workflow updates and submissions while that interaction is paused.
+- Restore the widget as a read-only view of durable host state.
+- Keep schema identifiers at version 1 and add no compatibility path.
+
+## Delivery coordinator
+
+The coordinator will have three in-memory states for one delivery:
+
+1. `claimed`: the host granted a lease, but Pi became busy before send;
+2. `queued`: `pi.sendMessage()` was called and the matching Pi entry is not yet durably settled;
+3. settled or ambiguous: the host accepted the public Pi entry ID, or settlement could not be proved.
+
+The coordinator records `claimExpiresAt` with every claim. A later poll can use the same claim while it is live. Immediately before send, the extension revalidates that exact claim and durable resource through the host, then checks Pi and the lease again. It discards cancelled, paused, replaced, or expired work. It does not request another claim while a live claim is remembered.
+
+The extension reads the presentation claim owner and expiry from the existing version-1 interaction row. A live claim held by any extension is normal unavailable work. It is not a tool error. Notification and terminal-turn claim receipts also include their exact expiry.
+
+The coordinator keeps a queued delivery until durable settlement succeeds. If the Pi entry is visible and settlement fails, the coordinator reports an ambiguous receipt and keeps the send blocked. Recovery may acquire a fresh claim and adopt the existing Pi entry, but it cannot send that delivery again.
+
+## Widget
+
+The extension will read the active origin-session run and render the existing workflow widget from its durable run state and definition snapshot. This path is read-only. It does not load workflow source, execute workflow code, or write workflow state.
+
+The widget will:
+
+- show running, waiting, and paused state;
+- use the existing bounded ten-line renderer;
+- support `Shift+Up` and `Shift+Down` scrolling;
+- use serializable lines outside TUI mode;
+- clear when the session has no active run or shuts down.
+
+## Escape and pause
+
+The extension will use Pi's documented `agent_end` event and public extension context abort signal. It will act only when:
+
+- the active context signal is aborted, or an assistant message has stop reason `aborted`;
+- the origin session has a pending agent or assistant interaction;
+- the same `agent_end` event contains that interaction's workflow prompt; and
+- the run is not already paused.
+
+The extension will send `run.pause` to the host. A live worker uses the existing exact-claim pause transaction. A waiting interaction has no worker and no live run claim, so the host will atomically set `paused = 1` on the parked run. The host will reject updates, submissions, and decision answers while paused.
+
+Resume will clear the pause on the same pending interaction without creating a worker or a second prompt. Other paused work will keep the existing behavior: take a new claim generation and resume in a supervised child.
+
+## Scope
+
+The change may update:
+
+- the extension delivery coordinator and host client integration;
+- the read-only workflow widget projection;
+- host pause and resume handling for parked interactions;
+- existing version-1 interaction response fields;
+- focused unit, integration, and live Pi tests;
+- the workflow host and authoring documentation.
+
+## Non-goals
+
+- Do not change Pi core, private Pi APIs, or Pi session schemas.
+- Do not restore the embedded workflow executor.
+- Do not add a second production runtime, database, service, feature flag, migration, or compatibility reader.
+- Do not claim exactly-once execution for an external effect that cannot prove it.
+- Do not release until the repair passes a live test in a new Pi session.
+
+## Implementation order
+
+1. Keep a live claim in the shared delivery coordinator and add claim expiry to all delivery receipts.
+2. Treat an existing live presentation claim as unavailable work.
+3. Keep visible messages blocked until durable settlement succeeds or recovery adopts them.
+4. Add atomic pause and resume operations for a parked interaction.
+5. Reject updates and submissions while the run is paused.
+6. Add the public `agent_end` Escape handler.
+7. Restore the widget as a read-only durable-state view.
+8. Add regression tests and update the canonical documentation.
+9. Run repository checks, Pi Reviewer, a new-session live test, and CI before release.
+
+## Acceptance criteria
+
+- Pi can remain busy longer than both the poll interval and presentation lease without a duplicate message or claim error.
+- Every delivery ID creates at most one Pi session message and one model turn.
+- A visible message with a failed receipt remains blocked from resend.
+- A competing live presentation claim does not appear as a workflow tool failure.
+- The widget appears for an active origin-session run and shows paused state after Escape.
+- Escape pauses the matching waiting run through the host.
+- A paused interaction rejects `update` and `submit`.
+- Resume keeps the same request and allows submission without another prompt.
+- Non-aborted turns and unrelated sessions do not pause the workflow.
+- The extension and host execute no workflow or controller code in their own event loops.
+
+## Verification
+
+Run:
+
+```bash
+npm run check
+npm run test:e2e
+git diff --check
+npx slophammer-ts@latest dry .
+npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
+npx -y @simpledoc/simpledoc check
+```
+
+Then run Pi Reviewer against `main` with a ten-minute tool timeout until no P0 or P1 finding remains. Test the built package in a new Pi session and a new Herdr tab. The live test must show the widget, start one workflow step, pause it with Escape, resume it, and complete it without a duplicate prompt or claim error.

package/docs/WORKFLOW_HOST.md

@@ -1,6 +1,6 @@
 # Workflow host
 
-Status: implemented. [Run workflows outside Pi](2026-08-30-out-of-process-workflow-host-plan.md) records the approved implementation plan.
+Status: implemented. [Run workflows outside Pi](2026-08-30-out-of-process-workflow-host-plan.md) records the approved redesign. [Restore workflow session delivery and controls](2026-09-01-restore-session-delivery-controls-plan.md) records the delivery, widget, and Escape-control repair.
 
 ## Purpose
 
@@ -254,6 +254,8 @@ The first command set is:
 - `host.status`
 - `host.stop`
 
+`notification.claim` and `turn.claim` can create a claim or revalidate the exact retained claim before delivery. Revalidation checks the in-memory client claim and its durable lease without creating another claim.
+
 Read operations may use the existing read-only store directly in viewers. Mutating Pi and CLI paths use the host.
 
 ## Worker protocol
@@ -316,9 +318,11 @@ Agent and assistant-message steps for an interactive run execute in the origin P
 
 The worker commits the node's resolved wall-clock deadline before it proposes `interaction.requested`. The host commits the request, changes the node attempt to waiting, parks the queue row, releases the claim, and acknowledges the worker. The worker then exits. The host continues to enforce the durable deadline while no worker exists. If the deadline passes, one control claim atomically closes the stale request and schedules a supervised timeout-resume child. The child preserves the same attempt and deadline, records `timed_out`, and follows any `$result.outcome` edge. A run with no timeout recovery edge becomes terminal and releases its session reservation. Restart recovery starts this timeout path before it schedules other work.
 
-The extension finds pending requests during `session_start`, after `agent_settled`, and once per second while the session is open. One shared session-delivery coordinator handles step prompts, protected decisions, notifications, and terminal presentation turns. It waits until Pi is idle and has no pending messages before it claims new work. Because the host claim is asynchronous, it checks those conditions again immediately before the synchronous call to the documented `pi.sendMessage()` API. An unused claim expires and is never sent later. After a claim, the coordinator records the delivery ID in memory before it sends. Polling only checks for delivery evidence while that ID remains queued; it cannot send the message again.
+The extension finds pending requests during `session_start`, after `agent_settled`, and once per second while the session is open. One shared session-delivery coordinator handles step prompts, protected decisions, notifications, and terminal presentation turns. It waits until Pi is idle and has no pending messages before it claims new work. Because the host claim is asynchronous, it checks those conditions again immediately before the synchronous call to the documented `pi.sendMessage()` API. The coordinator remembers the claimed delivery before that final check. If Pi became busy, a later poll can send with that exact claim while its lease remains live. An expired unused claim is discarded. Polling cannot acquire a second claim or send a delivery that is already queued.
+
+The host grants one live presentation claim. The current presenter cannot claim the same request again before that claim expires. A poll that sees any live presentation claim treats it as unavailable, not as a tool failure. When the matching custom message appears in the active Pi branch, the coordinator records its public session entry ID through the host and clears the local queued state. If Pi becomes idle without exposing a matching entry after the confirmation interval, the coordinator reports the delivery as ambiguous and keeps it blocked. A failed durable receipt also keeps the visible message blocked. Neither case can send the message again. The normal `workflow` tool contract then submits updates and results.
 
-The host grants one live presentation claim. The current presenter cannot claim the same request again before that claim expires. When the matching custom message appears in the active Pi branch, the coordinator records its public session entry ID through the host and clears the local queued state. If Pi becomes idle without exposing a matching entry after the confirmation interval, the coordinator reports the delivery as ambiguous and keeps it blocked. It does not infer success or retry. The normal `workflow` tool contract then submits updates and results.
+The extension projects the active origin-session run into Pi's widget and status APIs by reading host-owned durable state. This projection never runs workflow code and never writes run state. `Shift+Up` and `Shift+Down` scroll the widget.
 
 A tool update or submission goes to the host. It includes the exact request, node, attempt, expected revision, and tool-call idempotency key. The host first checks this transport contract and records a provisional `validating` submission. It then schedules a supervised workflow child. Only that child loads workflow code and runs the node's `validate` function. The child reports `interaction.accepted` or `interaction.rejected` to the host. The host settles the request only after acceptance. A rejected payload leaves the same request pending and returns the stored actionable error to the model. If the child stops before it reports a result, the host rejects the provisional submission and leaves the request ready for a corrected retry.
 
@@ -340,7 +344,7 @@ The run binding records `interactive` or `headless` execution mode. Viewers show
 
 ## Pause and cancellation
 
-Pause atomically commits `paused = 1`, parks the queue, releases the exact claim, and stores the command receipt. The fenced worker process group then stops. An uncommitted pure node can run again after resume. Resume takes a new generation and starts another worker from the last durable boundary.
+Pause atomically commits `paused = 1`, parks the queue, releases the exact claim, and stores the command receipt. The fenced worker process group then stops. If a Pi model turn ends while the public extension context signal is aborted, or with public stop reason `aborted`, the extension sends this same host pause command only when that `agent_end` event contains the pending interaction's workflow prompt. A parked interaction has no worker or live run claim, so the host marks it paused in place. While paused, updates, submissions, and decision answers are rejected. Resume clears the pause on that same pending interaction; other paused work takes a new generation and starts another worker from the last durable boundary. An uncommitted pure node can run again after resume.
 
 Cancellation against a live worker atomically commits terminal cancellation, cancels pending attempt and interaction state, settles effect recovery state, releases the exact claim, and stores the command receipt. A pending effect becomes cancelled. An applying effect becomes ambiguous because the host cannot prove its external outcome. The host then stops the fenced worker process group. A host crash after the receipt cannot resume the cancelled run or retry the ambiguous effect. If the child does not stop by the deadline, the host kills its process group.
 

package/docs/WORKFLOW_STEP_MESSAGES.md

@@ -1,6 +1,6 @@
 # Workflow step messages
 
-This specification defines how pi-workflows shows agent-step instructions in an interactive Pi session. The model receives the complete step prompt, while the user sees a small workflow card that can be expanded.
+This specification defines how pi-workflows shows agent-step instructions in an interactive Pi session. The model receives the complete step prompt, while the user sees a small workflow card that can be expanded. [Restore workflow session delivery and controls](2026-09-01-restore-session-delivery-controls-plan.md) records the delivery and session-control repair.
 
 ## Goal
 
@@ -48,11 +48,11 @@ pi.sendMessage(
 
 ## Session delivery
 
-One shared coordinator delivers step prompts, protected decisions, notifications, and final workflow results. It does not claim or send a new message while Pi is busy or another message is pending. It checks these conditions again after the asynchronous host claim and immediately before the synchronous send. If Pi became busy, the unused claim expires and is never sent later. Before it calls `pi.sendMessage()`, the coordinator records the stable delivery ID in a process-local queued map. The one-second poll can then look for the matching session entry, but it cannot send that ID again.
+One shared coordinator delivers step prompts, protected decisions, notifications, and final workflow results. It does not claim or send a new message while Pi is busy or another message is pending. It checks these conditions again after the asynchronous host claim and immediately before the synchronous send. The coordinator remembers the stable delivery ID and exact claim expiry before that final check. If Pi became busy, a later poll can use that same claim while it remains live. Before it sends retained work, it revalidates the exact claim and durable resource through the host, then checks Pi and the lease again. Cancelled, paused, or replaced work is discarded. An expired unused claim is also discarded.
 
-The coordinator clears the queued ID only after it observes the custom message in the active branch and saves the public Pi session entry ID through the workflow host. If Pi becomes idle but does not expose that entry after the confirmation interval, the coordinator reports an ambiguous delivery and keeps it blocked. Reload and restart recovery first search the branch for that stable ID. An existing entry is adopted. A new message is sent only when no entry exists and the host grants a new claim.
+Before the coordinator calls `pi.sendMessage()`, it records the delivery in a process-local queued map. The one-second poll can look for the matching session entry, but it cannot send that ID again. The coordinator clears the queued ID only after it observes the custom message in the active branch and saves the public Pi session entry ID through the workflow host. If Pi does not expose that entry after the confirmation interval, or if saving its receipt fails, the coordinator reports an ambiguous delivery and keeps it blocked.
 
-The host does not grant another live presentation claim to the same extension client. Claim expiry permits recovery by a new claimant; it is not permission for the current process to resend a queued message.
+Reload and restart recovery first search the branch for that stable ID. An existing entry is adopted. A new message is sent only when no entry exists and the host grants a new claim. The host does not grant a second live presentation claim. Polling treats another live claim as unavailable work rather than a workflow error.
 
 ## Engine boundary
 

package/docs/workflows.md

@@ -475,8 +475,18 @@ The model sees one `workflow` tool. Its `action` field supports:
 A direct user request to continue or resume the active workflow maps to
 `resume` immediately. The model does not call `status` instead of `resume` or
 use it as a prerequisite. An already active run adopts the resume request. A
-paused or parked run gets a new claim generation and worker. With no resumable
-run, the host rejects the request.
+paused interaction remains the same durable request and resumes without a
+worker. Other paused or parked work gets a new claim generation and worker.
+With no resumable run, the host rejects the request.
+
+The origin Pi session shows its active run in the workflow widget. `Shift+Up`
+and `Shift+Down` scroll it. If Escape aborts the model turn started by a
+presented workflow interaction, the extension detects the public context abort
+signal and pauses that run through the host. It also accepts Pi's public
+`aborted` stop reason. The matching `agent_end` event must contain the workflow
+prompt, so an unrelated interrupted turn cannot pause old pending work. The
+paused run does not accept updates, submissions, or decision answers until
+`resume`.
 
 `status` reports the durable queue projection. A host command succeeds only
 after its transaction commits. The protocol stores request fingerprints and

package/herdr-plugin.toml

@@ -1,6 +1,6 @@
 id = "osolmaz.pi-workflows"
 name = "pi-workflows"
-version = "0.15.2"
+version = "0.15.3"
 min_herdr_version = "0.7.0"
 description = "Open the active pi-workflows run in piw from a managed Herdr pane."
 platforms = ["linux", "macos"]

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@osolmaz/pi-workflows",
-  "version": "0.15.2",
+  "version": "0.15.3",
   "description": "Workflow and controller runtime with a live terminal viewer for the pi coding agent",
   "keywords": [
     "pi-package"

package/src/controllers/sqlite.ts

@@ -191,6 +191,7 @@ type RunRow = {
   workflowName: string;
   workflowRef: string;
   runStatus: string;
+  paused: number;
   definitionDigest: Buffer;
   definitionHash: Buffer;
   inputHash: Buffer;
@@ -1561,6 +1562,104 @@ export class SqliteControllerStore implements ControllerStore {
     return this.releaseRunClaim(options.runId, options.claimToken, "parked", options.now);
   }
 
+  pauseParkedWorkflowRun(options: { runId: string; now?: string }): boolean {
+    const now = epoch(validTimestamp(options.now));
+    return this.state.transaction(() => {
+      const row = this.workflowRunRow(options.runId);
+      if (row === undefined || row.status !== "parked") return false;
+      const lease = this.requireLease(row.resourceId);
+      if (lease.ownerId !== null && lease.expiresAt !== null && lease.expiresAt > now) return false;
+      const pending = this.state.connection
+        .prepare(
+          `SELECT 1 FROM interactive_requests
+           WHERE run_id = ? AND status IN ('pending', 'presenting') LIMIT 1`,
+        )
+        .get(options.runId);
+      if (pending === undefined) return false;
+      if (row.paused === 1) return true;
+      if (!["running", "waiting"].includes(row.runStatus)) return false;
+      const changed = this.state.connection
+        .prepare(
+          `UPDATE runs SET paused = 1, status_detail = 'paused', updated_at = ?
+           WHERE run_id = ? AND paused = 0 AND status IN ('running', 'waiting')`,
+        )
+        .run(now, options.runId);
+      if (changed.changes !== 1) return false;
+      const revision = this.resourceRevision(row.resourceId);
+      this.bumpResource(row.resourceId, revision, now);
+      this.insertEvent(
+        row.resourceId,
+        revision + 1,
+        "run.paused",
+        "control",
+        null,
+        { status: "parked" },
+        now,
+      );
+      recordViewerDeltas(
+        this.state,
+        options.runId,
+        [{ targetType: "summary" }, { targetType: "replay" }],
+        now,
+      );
+      return true;
+    });
+  }
+
+  resumePausedInteraction(options: { runId: string; now?: string }): boolean {
+    const now = epoch(validTimestamp(options.now));
+    return this.state.transaction(() => {
+      const row = this.workflowRunRow(options.runId);
+      if (
+        row === undefined ||
+        row.status !== "parked" ||
+        row.runStatus !== "waiting" ||
+        row.paused !== 1
+      ) {
+        return false;
+      }
+      const lease = this.requireLease(row.resourceId);
+      if (lease.ownerId !== null && lease.expiresAt !== null && lease.expiresAt > now) return false;
+      const pending = this.state.connection
+        .prepare(
+          `SELECT 1 FROM interactive_requests
+           WHERE run_id = ? AND status IN ('pending', 'presenting') LIMIT 1`,
+        )
+        .get(options.runId);
+      if (pending === undefined) return false;
+      const changed = this.state.connection
+        .prepare(
+          `UPDATE runs
+           SET paused = 0, status_detail = 'waiting for origin-session input', updated_at = ?
+           WHERE run_id = ? AND status = 'waiting' AND paused = 1`,
+        )
+        .run(now, options.runId);
+      if (changed.changes !== 1) return false;
+      const revision = this.resourceRevision(row.resourceId);
+      this.bumpResource(row.resourceId, revision, now);
+      this.insertEvent(
+        row.resourceId,
+        revision + 1,
+        "run.resumed",
+        "control",
+        null,
+        { status: "waiting" },
+        now,
+      );
+      recordViewerDeltas(
+        this.state,
+        options.runId,
+        [{ targetType: "summary" }, { targetType: "replay" }],
+        now,
+      );
+      return true;
+    });
+  }
+
+  isWorkflowRunPaused(runId: string): boolean {
+    return this.workflowRunRow(runId)?.paused === 1;
+  }
+
   parkWorkflowRunForSourceChange(options: {
     runId: string;
     claimToken: string;
@@ -2038,6 +2137,21 @@ export class SqliteControllerStore implements ControllerStore {
     return isTurnIntentRow(row) ? this.mapTurnIntent(row) : undefined;
   }
 
+  isWorkflowTurnIntentClaimLive(options: {
+    intentId: string;
+    targetSessionId: string;
+    claimToken: string;
+    now?: string;
+  }): boolean {
+    const row = this.turnIntentRow(options.intentId);
+    return (
+      row !== undefined &&
+      row.targetSessionId === options.targetSessionId &&
+      row.resolvedAt === null &&
+      this.verifyEffectToken(row.effectId, options.claimToken, options.now)
+    );
+  }
+
   claimWorkflowTurnIntent(options: {
     intentId: string;
     targetSessionId: string;
@@ -2279,6 +2393,20 @@ export class SqliteControllerStore implements ControllerStore {
     return result;
   }
 
+  isWorkflowNotificationClaimLive(options: {
+    notificationId: string;
+    targetSessionId: string;
+    claimToken: string;
+    now?: string;
+  }): boolean {
+    const row = this.notificationRowById(options.notificationId);
+    return (
+      row !== undefined &&
+      row.targetSessionId === options.targetSessionId &&
+      this.verifyEffectToken(row.effectId, options.claimToken, options.now)
+    );
+  }
+
   markWorkflowNotificationDelivered(options: {
     notificationId: string;
     targetSessionId: string;
@@ -3452,7 +3580,7 @@ function workflowSelect(clause: string): string {
 function workflowRunSelect(clause: string): string {
   return `SELECT r.run_id AS runId, r.resource_id AS resourceId,
     d.workflow_name AS workflowName, r.workflow_ref AS workflowRef, r.status AS runStatus,
-    r.definition_digest AS definitionDigest, d.definition_hash AS definitionHash,
+    r.paused, r.definition_digest AS definitionDigest, d.definition_hash AS definitionHash,
     r.input_hash AS inputHash,
     r.launch_options_hash AS launchOptionsHash,
     q.status, q.available_at AS availableAt, q.affinity_runner_id AS affinityRunnerId,