@osolmaz/pi-workflows 0.2.0 → 0.4.0

package/docs/run-bundles.md

@@ -77,9 +77,9 @@ Consequences for readers:
 
 Large payloads are stored once, content-addressed, under `artifacts/` and
 referenced from the documents that use them. This applies uniformly to every
-**persisted value position**: `input`, `outputs.*`, `results.*.output`,
-`steps[*].prompt`, `steps[*].output`, `finalOutput`, and trace event payload
-values.
+**persisted value position**. These positions include `input`, `outputs.*`,
+`results.*.output`, `steps[*].prompt`, `steps[*].output`, and `finalOutput`.
+Trace event payload values follow the same rule.
 
 Encoding rule, applied recursively to a persisted value:
 
@@ -109,9 +109,9 @@ Artifact rules:
 - Artifacts are immutable once written and deduplicate by content hash.
 - Readers must tolerate unknown `mediaType` values.
 
-Because the same output can legitimately appear in `outputs`, `results`,
-`steps`, and the trace, externalization makes that duplication cheap: each
-copy is the same small reference.
+The same output can legitimately appear in several places. Externalization
+keeps each copy in `outputs`, `results`, `steps`, or the trace as the same small
+reference.
 
 ## manifest.json
 
@@ -123,7 +123,11 @@ Identity and pointers, kept in sync with the state on every snapshot:
   "runId": "20260729T023912Z-autoimplement-3f2a9c1b",
   "workflowName": "autoimplement",
   "runTitle": "autoimplement: fix the flaky test",
-  "workflowPath": "/repo/.pi/workflows/autoimplement.workflow.ts",
+  "workflowSource": {
+    "kind": "file",
+    "path": "/repo/.pi/workflows/autoimplement.workflow.ts",
+    "hash": "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"
+  },
   "startedAt": "2026-07-29T02:39:12.412Z",
   "finishedAt": "2026-07-29T02:41:03.977Z",
   "status": "completed",
@@ -138,6 +142,11 @@ Identity and pointers, kept in sync with the state on every snapshot:
 }
 ```
 
+`workflowSource` identifies the definition used by the run. User workflow
+files use an absolute path and SHA-256 hash. Package-provided workflows use a
+stable identity such as `{ "kind": "builtin", "id": "monitor", "revision": "1" }`.
+A built-in identity does not contain an installation path.
+
 `paths.artifacts` is declared from bundle creation so a live session-event
 patch can safely reference a newly written artifact before the next workflow
 state projection. The directory itself is created only when needed.
@@ -154,6 +163,30 @@ validators are not serialized. Each node keeps only its metadata (`nodeType`,
 and edges are copied verbatim. The snapshot is what lets viewers draw all
 nodes, including ones that have not run yet. It is immutable after run start.
 
+## Resume and repair
+
+An interrupted run (status `running` with no terminal trace event) can resume
+instead of failing. Resume is a named operation with strict rules:
+
+1. The caller must hold the run's queue claim. Only the current claim holder
+   may resume or interrupt a bundle, and every bundle write verifies the
+   claim token first (write fencing).
+2. A torn trace tail (a crash mid-append) is truncated to the last complete
+   line. Trace events the state projection never recorded are dropped, so
+   `state.traceSeq` and the trace agree again before any new event.
+3. Completed nodes replay from the projection. The in-flight node reruns with
+   a fresh attempt; a `run_resumed` trace event marks the boundary.
+4. `state.workflowSource` pins the workflow source from run start. File
+   sources require the same hash. Built-in sources require the same catalog
+   id and revision. Resume refuses a mismatch unless forced, and a forced
+   resume records the mismatch in the `run_resumed` payload.
+
+Continuation runs (answering a checkpoint) are new bundles, not resumed ones.
+They link back through `state.parentRunId`, carry the parent's outputs,
+results, and step records forward, and note `continuedFrom` in their
+`run_started` payload. Bundles stay append-only; a continuation is the only
+way work follows a terminal `waiting` state.
+
 ## state.json
 
 The full run projection (`WorkflowRunState` in
@@ -166,6 +199,11 @@ The full run projection (`WorkflowRunState` in
   "traceSeq": 17,
   "runId": "20260729T023912Z-autoimplement-3f2a9c1b",
   "workflowName": "autoimplement",
+  "workflowSource": {
+    "kind": "file",
+    "path": "/repo/.pi/workflows/autoimplement.workflow.ts",
+    "hash": "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"
+  },
   "startedAt": "…",
   "updatedAt": "…",
   "status": "running",
@@ -176,11 +214,20 @@ The full run projection (`WorkflowRunState` in
 }
 ```
 
+- `workflowSource` is the canonical source identity. Resuming a file requires
+  the same hash. Resuming a built-in requires the same catalog revision. A
+  mismatch refuses the resume instead of loading another definition.
 - `status` is one of `running`, `waiting`, `completed`, `failed`, `timed_out`,
-  or `cancelled`.
-- While a node is executing, `currentNode`, `currentAttemptId`,
-  `currentNodeStartedAt`, and `statusDetail` describe it; they disappear when
-  the node finishes. The executing node's type comes from the definition
+  or `cancelled`. A controller host records an abandoned bundle as `failed`
+  with a final `run_interrupted` trace event. Before doing that, recovery checks
+  the trace tail and repairs a stale projection when the terminal event was
+  already appended. The controller store can then retry a genuinely abandoned
+  child attempt without changing this schema. If startup stopped before a
+  manifest existed, the scheduler preserves that incomplete directory as a
+  hidden sibling before creating the reserved run.
+- While a node is executing, `currentNode` and `currentAttemptId` identify it.
+  `currentNodeStartedAt` and `statusDetail` add timing and display context.
+  These fields disappear when the node finishes. The executing node's type comes from the definition
   snapshot, not from the state.
 - While a pause request holds the run at a step boundary, `paused` is `true`
   (with matching `run_paused`/`run_resumed` trace events); it disappears when
@@ -218,10 +265,9 @@ The full run projection (`WorkflowRunState` in
   produced by this attempt, from prompt delivery through accepted submission.
   Viewers must use this explicit linkage and never infer it heuristically.
 - Action steps carry an `action` receipt with `actionType`
-  (`shell`/`function`) and, for shell actions, `command`, `args`, `cwd`,
-  `exitCode`, `signal`, and `durationMs`. Shell stdout/stderr live in the
-  step output (the parsed or raw shell result) and are externalized when
-  large.
+  (`shell`/`function`). Shell actions also record `command`, `args`, `cwd`,
+  `exitCode`, `signal`, and `durationMs`. Shell stdout/stderr live in the step
+  output (the parsed or raw shell result) and are externalized when large.
 - When a run pauses at a checkpoint, `waitingOn` names the checkpoint node.
   Terminal runs carry `finalOutput` on success and `error` on failure.
 
@@ -248,8 +294,8 @@ One event per line, monotonically sequenced per run, schema
 ignore unknown event types and unknown payload fields so new ones can be added
 within the same schema version.
 
-The trace alone is sufficient to reconstruct the run: outputs and receipts are
-part of the terminal node events, not only of `state.json`.
+The trace alone is sufficient to reconstruct the run because terminal node
+events carry outputs and receipts.
 
 Event catalog and payload contracts:
 
@@ -268,6 +314,7 @@ Event catalog and payload contracts:
 | `run_failed`        | run     | `status`, `error`                                                   |
 | `run_timed_out`     | run     | `status`, `error`                                                   |
 | `run_cancelled`     | run     | `status`, `error?`                                                  |
+| `run_interrupted`   | run     | `error`                                                             |
 
 Invariants:
 
@@ -283,6 +330,15 @@ Present when the run executed inside a Pi conversation. The extension records
 the conversation into the bundle so replay never depends on Pi's global
 session store.
 
+A run that outlives its first session (parked, then resumed by another
+session or the host) gains a second capture under
+`session/segments/<attemptId>/` with the same file layout (`binding.json`,
+`entries.ndjson`, `events.ndjson`, `capture.json`). The first capture stays
+flat at `session/`; only captures from the second bind onward become
+segments, so readers that predate segments keep working on single-session
+bundles. An interrupted run finalizes any segment still `recording` as
+`failed` with the interruption reason.
+
 ### binding.json
 
 Written once when the run binds to the conversation
@@ -325,22 +381,23 @@ while the run was active, schema `pi-workflows.session-entry.v1`:
 - `entry` is the verbatim Pi session entry (Pi's own versioned format),
   including user messages, assistant messages, tool results, model changes,
   and compaction entries. Nothing is normalized or rewritten.
-- Entries include everything that happened in the conversation during the run:
-  workflow prompts, nudges, and user interruptions are all part of the record.
+- Entries include everything that happened in the conversation during the run.
+  This includes workflow prompts and nudges together with user interruptions.
 - `conversation` ranges in step records and `node_finished` events address
   entries by Pi entry id (`entry.id`).
 
 ### events.ndjson
 
-The temporal journal records documented Pi `turn_*`, `message_*`, and
+The temporal journal records documented Pi `turn_*` and `message_*` hooks plus
 `tool_execution_*` hooks with schema `pi-workflows.session-event.v1`. Each
-record has a per-file `seq`, timestamp, `nodeId`, `attemptId`, optional
-`turnId`, `messageId`, and `toolCallId`, a normalized `type`, and `payload`.
+record has a per-file `seq`, timestamp, `nodeId`, and `attemptId`. Optional
+turn, message, and tool call IDs link related records. A normalized `type` and
+`payload` carry the event data.
 The full contract and event catalog are in
 [session-event-journal.md](session-event-journal.md).
 
-Events preserve semantic deltas. Assistant `partial`, terminal `message`, and
-terminal `error` snapshots are never stored. Tool update records omit Pi's
+Events preserve semantic deltas. Assistant `partial` snapshots are never
+stored, and neither are terminal `message` or `error` snapshots. Tool update records omit Pi's
 cumulative `partialResult`. Final `message_finished` records link to settled
 Pi entries with `entryId`; after that linkage, `entries.ndjson` is the
 verbatim content authority.
@@ -364,10 +421,10 @@ complete lines, sequence gaps, and terminal torn tails are integrity failures.
 }
 ```
 
-`status` is `recording`, `complete`, or `failed`. Failed capture adds
-`failure` with `failedAt`, `code`, and `message`. Capture failure is visible to
-readers but does not fail the workflow. Terminal readers verify the counts,
-last sequence, schemas, and contiguous event sequence. Missing temporal files
+`status` starts as `recording` and ends as `complete` or `failed`. Failed
+capture adds `failure` with `failedAt` plus a code and message. Capture failure
+is visible to readers but does not fail the workflow. Terminal readers verify
+the counts and last sequence, then check schemas and contiguous event order. Missing temporal files
 in a session-bound bundle are invalid, not an older supported layout.
 
 ## Versioning

package/docs/workflows.md

@@ -9,10 +9,21 @@ model sees, and how runs behave at runtime. For the on-disk run format, see
 
 A workflow is a TypeScript module whose default export is `defineWorkflow(...)`.
 Files are discovered by suffix (`.workflow.ts`, `.workflow.js`, `.workflow.mts`,
-`.workflow.mjs`) from two directories, in precedence order:
+`.workflow.mjs`) from these sources, in precedence order:
 
 1. `.pi/workflows/` in the project (highest precedence on name collisions)
 2. `~/.pi/agent/workflows/` globally
+3. Workflows built into Pi Workflows
+
+Pi Workflows includes a built-in `monitor` workflow. A project or global file
+named `monitor.workflow.ts` replaces it. The package registers each built-in in
+a process-local catalog with a stable reference such as `builtin:monitor` and
+an explicit revision. Built-ins are imported with the engine when a Pi process
+starts. They are not read from the package directory when a run starts or
+resumes. Updating the package on disk cannot mix a new built-in with that
+process's old engine; reload or restart Pi to use the new built-in. A revision
+mismatch refuses resume. Project and global workflow files still reload on
+each run and use their path and SHA-256 hash as their source identity.
 
 The workflow's command name is the file stem, so `.pi/workflows/triage.workflow.ts`
 runs as `/workflow triage`. A direct path also works: `/workflow ./somewhere/x.workflow.ts`.
@@ -44,7 +55,7 @@ Top-level fields:
 
 | Field                | Type                   | Notes                                                                                                                                                                                                              |
 | -------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| `name`               | `string`               | Required. Used in run ids and the step contract. `cancel`, `list`, `pause`, and `resume` are reserved for `/workflow` subcommands.                                                                                 |
+| `name`               | `string`               | Required. Used in run ids and the step contract. `answer`, `cancel`, `list`, `pause`, `resume`, and `status` are reserved for `/workflow` subcommands.                                                             |
 | `title`              | `string` or function   | Optional run title, resolved once at start from `{ input, workflowName }`. Async resolution is bounded (30s) and cancellable.                                                                                      |
 | `presentationPrompt` | `string` or function   | Optional instructions for a normal assistant response after the run. A function receives `{ state, finalOutput, signal }` and may return a prompt or `undefined`. See [Result presentation](#result-presentation). |
 | `startAt`            | `string`               | Required. Id of the first node.                                                                                                                                                                                    |
@@ -79,6 +90,38 @@ Long-running compute, action, and checkpoint callbacks should observe
 steps). When the node times out or the run is cancelled, the engine stops
 waiting immediately, but only cooperative callbacks stop doing work.
 
+## Durable runs, parking, and resume
+
+Every interactive `/workflow` run is tracked in the project run queue (see
+[CONTROLLERS.md](CONTROLLERS.md) for the store). The session that starts a run
+claims it and owns it while it executes; every bundle write proves the claim
+first (write fencing).
+
+Closing the Pi session mid-run no longer cancels the run. The engine **parks**:
+it stops without a terminal event, releases the claim, and leaves the bundle
+resumable. When a runner is available again (a reopened Pi session or the
+standalone host), the run **resumes** at the node it stopped on. Completed
+nodes replay from the recorded state; only the interrupted node and everything
+downstream rerun. Resume repairs a torn trace tail, drops trace events the
+state projection never recorded, and refuses to continue if the workflow
+source changed since the run started (a forced resume records the mismatch).
+
+The standalone host runs without any Pi session:
+
+```bash
+pi-workflows host --project /path/to/project
+```
+
+The host claims parked runs, resumes them, and reconciles durable controllers.
+Conversation nodes execute in headless `pi --mode rpc` children that load a
+small bridge extension; the model sees the same `workflow` tool contract as an
+in-session run. The host is a foreground process: start it in a terminal and
+stop it with Ctrl-C. A second host for the same project refuses to start, and
+a host that dies has its orphaned children reaped by the next one. While the
+host works, any open Pi session stays current: a per-session watermark over
+the shared run event feed produces catch-up summaries and quiet context
+updates.
+
 ## Node types
 
 ### agent
@@ -91,7 +134,10 @@ agent({
   prompt: ({ outputs }) => `Review this: ${JSON.stringify(outputs.implement)}`,
   expectedOutput: `{ "verdict": "clean" | "issues_found" }`,
   validate: (output) => output, // optional; throw to reject the submission
-  timeoutMs: 30 * 60_000, // optional; default 15 minutes
+  timeoutMs: ({ input }) =>
+    (input as { timeoutMinutes?: number }).timeoutMinutes
+      ? (input as { timeoutMinutes: number }).timeoutMinutes * 60_000
+      : 30 * 60_000, // optional number or context callback; default 15 minutes
   statusDetail: "reviewing", // optional; shown in widget and viewer
 });
 ```
@@ -101,7 +147,15 @@ calls the tool, the output passes through normalization (a JSON string is
 parsed tolerantly) and then `validate`. If `validate` throws, the tool call
 returns an error and the model can retry within the same step. If the agent
 ends its turn without submitting, the extension nudges it, twice by default,
-then fails the step.
+then fails the step. If an agent node times out or the workflow is cancelled,
+the extension also aborts its active Pi turn. The model cannot continue to use
+tools after the engine has closed that attempt.
+
+`timeoutMs` can be a finite positive number or a function of the normal node
+context. A timeout function can use prepared outputs to select a deadline for
+this run. It has 30 seconds to return a value. Computed timeout functions are
+runtime code, so definition snapshots omit them; fixed numeric timeouts remain
+in the snapshot.
 
 ### compute
 
@@ -147,10 +201,15 @@ command fails.
 
 ### checkpoint
 
-Ends the run in a `waiting` state for human review. Runs after a checkpoint do
-not resume automatically; the checkpoint output is the run's final output.
-Because nothing resumes past a checkpoint, graph validation rejects outgoing
-edges from checkpoint nodes.
+Ends the run in a `waiting` state for human review. The checkpoint bundle is
+terminal, so no process keeps running while the run waits. The human answers
+with `/workflow answer <json>` (or plain text), which starts a **continuation
+run**: a new run with its own bundle and trace, linked to the checkpointed run
+through `parentRunId`. The continuation receives the answer as its input,
+carries forward every output the parent produced (including the checkpoint's),
+and continues routing along the checkpoint's outgoing edge. Outgoing edges
+from checkpoint nodes are allowed exactly so continuations have somewhere to
+go; step accounting carries over, so `maxSteps` bounds the whole chain.
 
 ```typescript
 checkpoint({
@@ -207,6 +266,60 @@ A missing case for the resolved value fails the run with a routing error. A
 node with no outgoing edge (or no matching failure route) ends the run:
 `completed` on success, `failed`/`timed_out`/`cancelled` otherwise.
 
+## Model workflow control
+
+The model sees one `workflow` tool. Its `action` field supports:
+
+- `list` for discovered workflow names and sources.
+- `start` with a workflow name or path and structured input.
+- `status` for the active run or a supplied run ID.
+- `pause`, `resume`, and `cancel` for the active run.
+- `answer` with checkpoint input and an optional run ID.
+- `submit` for the current workflow step contract.
+
+A model-started run is queued until the model's current turn settles. The first
+workflow prompt then starts a new turn. This keeps the requesting turn outside
+the workflow's first attempt and prevents an early missing-submission reminder.
+The normal extension offers all actions. The headless RPC bridge offers only
+`submit`, so a workflow child cannot recursively control other runs.
+
+### Built-in monitor
+
+The built-in `monitor` workflow turns a plain request for repeated checks into
+one looping workflow run. Its input is:
+
+```json
+{
+  "task": "Check pull request 123",
+  "everyMinutes": 30,
+  "reportWhen": "Checks fail or the state changes materially",
+  "stopWhen": "The pull request is merged or closed",
+  "maxChecks": 1000,
+  "checkTimeoutMinutes": 60
+}
+```
+
+The first check runs immediately. Each accepted check records a bounded current
+observation and chooses whether to continue, report, or stop. A report uses a
+separate agent node so its structured check result is validated before the user
+sees the message. The next check can read the previous accepted observation.
+
+Intervals must be whole minutes from 1 through 1,440. `maxChecks` defaults to
+1,000 and cannot exceed 1,000. `checkTimeoutMinutes` is optional and applies to
+check and report agent nodes. It must be from 5 through 1,440 minutes. Its
+default is the larger of 60 minutes and `everyMinutes`. The workflow also has
+a finite step limit and bounded observation and report sizes.
+
+The interval uses the existing shell action to launch the current Node
+executable with a timer. This works on every platform supported by Pi. The node
+and command timeouts are higher than the maximum interval. Cancelling the
+workflow aborts the timer process immediately. If the owning Pi process or
+standalone host stops during the wait, normal parking rules abort the shell node
+and resume later by running that wait again from the beginning.
+
+A monitor uses the session's single active workflow slot. It does not provide
+cron syntax, calendar scheduling, OS notifications, or a background service.
+
 ## The step contract
 
 Every `agent` prompt ends with a step contract block naming the workflow, the
@@ -217,16 +330,17 @@ step id, the attempt id, and the expected output shape:
 Workflow step contract (workflow: autoimplement, step: review, attempt: 6f9d…)
 
 Complete this step by calling the `workflow` tool exactly once with:
-{"step": "review", "attempt": "6f9d…", "output": <your result>}
+{"action": "submit", "step": "review", "attempt": "6f9d…", "output": <your result>}
 Expected output: { "route": "clean" | "issues_found", "reason": "short justification" }
 The step is complete only after the workflow tool accepts the output.
 If the tool reports a validation error, correct the output and call it again.
 ```
 
-The `workflow` tool takes `{ step, attempt, output }`. Submissions are
-rejected (with a reason the model sees) when no step is pending, the step id
-is wrong, the attempt id belongs to an earlier attempt of the same node (loops
-revisit node ids, so each attempt gets a fresh id), or `validate` throws.
+The `workflow` tool uses `{ action: "submit", step, attempt, output }` for step
+results. Submissions are rejected (with a reason the model sees) when no step
+is pending, the step id is wrong, the attempt id belongs to an earlier attempt
+of the same node (loops revisit node ids, so each attempt gets a fresh id), or
+`validate` throws.
 Acceptance resolves the step and the engine advances; the next agent prompt
 arrives as a new user message in the same conversation.
 
@@ -251,8 +365,9 @@ After the final run state has been persisted, the Pi extension sends the
 presentation instructions and bounded final result to the model as a hidden
 follow-up message. The next visible message is a normal assistant response.
 Returning `undefined`, returning an empty string, or omitting
-`presentationPrompt` produces no follow-up. Cancelled runs are never
-presented. Async prompt builders have 30 seconds to finish and receive an
+`presentationPrompt` produces no follow-up. Failed, timed-out, and cancelled
+runs are never presented; the extension reports their persisted status and
+error directly. Async prompt builders have 30 seconds to finish and receive an
 `AbortSignal` that fires on timeout, session shutdown, or when a new workflow
 or normal user turn starts; stale presentations are discarded. Once a presentation message has
 been queued, another workflow cannot start until that assistant response
@@ -271,8 +386,10 @@ Runs execute one node at a time. Every transition is persisted to the run
 bundle before the engine moves on, which is what makes the live viewer
 possible. Defaults worth knowing:
 
-- Node timeout is 15 minutes unless the node sets `timeoutMs`. A timed-out
-  node has outcome `timed_out` and can be routed with `$result.outcome`.
+- Node timeout is 15 minutes unless the node sets `timeoutMs` to a positive
+  number or context callback. A timed-out node has outcome `timed_out` and can
+  be routed with `$result.outcome`. A timed-out agent node also aborts its Pi
+  turn, and late output for that attempt is rejected.
 - `maxSteps` (workflow-level, default 100) bounds loops built from cycles in
   the graph.
 - `/workflow pause` requests a pause: the current step finishes normally,
@@ -290,6 +407,33 @@ possible. Defaults worth knowing:
 - Agent nudges: if the model ends its turn without submitting the pending
   step, it gets a reminder, twice by default, then the step fails.
 
+## Workflows started by controllers
+
+A controller can start a workflow as a finite child job with `ctx.workflows.ensure()`. The request key is stable across reconciliation passes, and the input fingerprint prevents one key from being reused for different work.
+
+```typescript
+const run = await ctx.workflows.ensure({
+  requestKey: `repair:${resource.metadata.generation}`,
+  workflow: "repair-pull-request",
+  input: { repository: resource.spec.repository, number: resource.spec.number },
+});
+
+if (run.state !== "succeeded") {
+  return ctx.requeueAfter(5_000, {
+    workflowRun: {
+      requestId: run.requestId,
+      ...(run.runId ? { runId: run.runId } : {}),
+      state: run.state,
+      attempt: run.attempt,
+    },
+  });
+}
+```
+
+Child workflow completion queues the parent resource again. A running child left by a stopped host is recorded as a failed run bundle with a `run_interrupted` event. The controller treats that child attempt as interrupted, and the next parent reconciliation starts another immutable attempt. Consequential external mutations should use the controller effect API so uncertain results are observed before retry.
+
+See [CONTROLLERS.md](CONTROLLERS.md) for controller definitions and the full recovery contract.
+
 ## Using the engine outside pi
 
 The engine is pi-agnostic. `WorkflowEngine` takes any `AgentStepExecutor`, so