@osolmaz/pi-workflows 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +72 -5
- package/dist/builtins/catalog.d.ts +2 -0
- package/dist/builtins/catalog.js +22 -0
- package/dist/builtins/catalog.js.map +1 -0
- package/dist/builtins/monitor.workflow.d.ts +71 -0
- package/dist/builtins/monitor.workflow.js +250 -0
- package/dist/builtins/monitor.workflow.js.map +1 -0
- package/dist/controllers/conditions.d.ts +6 -0
- package/dist/controllers/conditions.js +68 -0
- package/dist/controllers/conditions.js.map +1 -0
- package/dist/controllers/definition.d.ts +6 -0
- package/dist/controllers/definition.js +45 -0
- package/dist/controllers/definition.js.map +1 -0
- package/dist/controllers/effects.d.ts +14 -0
- package/dist/controllers/effects.js +104 -0
- package/dist/controllers/effects.js.map +1 -0
- package/dist/controllers/errors.d.ts +12 -0
- package/dist/controllers/errors.js +25 -0
- package/dist/controllers/errors.js.map +1 -0
- package/dist/controllers/index.d.ts +13 -0
- package/dist/controllers/index.js +13 -0
- package/dist/controllers/index.js.map +1 -0
- package/dist/controllers/json.d.ts +5 -0
- package/dist/controllers/json.js +57 -0
- package/dist/controllers/json.js.map +1 -0
- package/dist/controllers/loader.d.ts +23 -0
- package/dist/controllers/loader.js +74 -0
- package/dist/controllers/loader.js.map +1 -0
- package/dist/controllers/manager.d.ts +58 -0
- package/dist/controllers/manager.js +399 -0
- package/dist/controllers/manager.js.map +1 -0
- package/dist/controllers/results.d.ts +5 -0
- package/dist/controllers/results.js +32 -0
- package/dist/controllers/results.js.map +1 -0
- package/dist/controllers/sqlite.d.ts +235 -0
- package/dist/controllers/sqlite.js +1087 -0
- package/dist/controllers/sqlite.js.map +1 -0
- package/dist/controllers/store.d.ts +112 -0
- package/dist/controllers/store.js +32 -0
- package/dist/controllers/store.js.map +1 -0
- package/dist/controllers/types.d.ts +159 -0
- package/dist/controllers/types.js +2 -0
- package/dist/controllers/types.js.map +1 -0
- package/dist/controllers/workflow-engine-scheduler.d.ts +25 -0
- package/dist/controllers/workflow-engine-scheduler.js +95 -0
- package/dist/controllers/workflow-engine-scheduler.js.map +1 -0
- package/dist/controllers/workflows.d.ts +27 -0
- package/dist/controllers/workflows.js +109 -0
- package/dist/controllers/workflows.js.map +1 -0
- package/dist/extension/controller-host.d.ts +47 -0
- package/dist/extension/controller-host.js +110 -0
- package/dist/extension/controller-host.js.map +1 -0
- package/dist/extension/executor.d.ts +3 -0
- package/dist/extension/executor.js +12 -2
- package/dist/extension/executor.js.map +1 -1
- package/dist/extension/index.d.ts +7 -0
- package/dist/extension/index.js +1118 -106
- package/dist/extension/index.js.map +1 -1
- package/dist/extension/recorder.d.ts +6 -0
- package/dist/extension/recorder.js +20 -8
- package/dist/extension/recorder.js.map +1 -1
- package/dist/extension/workflow-tool.d.ts +28 -0
- package/dist/extension/workflow-tool.js +33 -0
- package/dist/extension/workflow-tool.js.map +1 -0
- package/dist/host/processes.d.ts +24 -0
- package/dist/host/processes.js +114 -0
- package/dist/host/processes.js.map +1 -0
- package/dist/host/rpc-bridge.d.ts +9 -0
- package/dist/host/rpc-bridge.js +39 -0
- package/dist/host/rpc-bridge.js.map +1 -0
- package/dist/host/rpc-executor.d.ts +38 -0
- package/dist/host/rpc-executor.js +254 -0
- package/dist/host/rpc-executor.js.map +1 -0
- package/dist/host/runner.d.ts +50 -0
- package/dist/host/runner.js +379 -0
- package/dist/host/runner.js.map +1 -0
- package/dist/viewer/cli.d.ts +7 -3
- package/dist/viewer/cli.js +150 -19
- package/dist/viewer/cli.js.map +1 -1
- package/dist/workflows/catalog.d.ts +43 -0
- package/dist/workflows/catalog.js +79 -0
- package/dist/workflows/catalog.js.map +1 -0
- package/dist/workflows/engine.d.ts +37 -2
- package/dist/workflows/engine.js +297 -29
- package/dist/workflows/engine.js.map +1 -1
- package/dist/workflows/errors.d.ts +23 -0
- package/dist/workflows/errors.js +38 -0
- package/dist/workflows/errors.js.map +1 -1
- package/dist/workflows/graph.js +0 -5
- package/dist/workflows/graph.js.map +1 -1
- package/dist/workflows/index.d.ts +2 -2
- package/dist/workflows/index.js +1 -1
- package/dist/workflows/index.js.map +1 -1
- package/dist/workflows/loader.d.ts +19 -15
- package/dist/workflows/loader.js +63 -19
- package/dist/workflows/loader.js.map +1 -1
- package/dist/workflows/migrate-sources.d.ts +41 -0
- package/dist/workflows/migrate-sources.js +129 -0
- package/dist/workflows/migrate-sources.js.map +1 -0
- package/dist/workflows/schema.js +3 -2
- package/dist/workflows/schema.js.map +1 -1
- package/dist/workflows/store.d.ts +50 -6
- package/dist/workflows/store.js +448 -53
- package/dist/workflows/store.js.map +1 -1
- package/dist/workflows/types.d.ts +27 -3
- package/docs/CONTROLLERS.md +215 -0
- package/docs/development.md +14 -9
- package/docs/plans/2026-08-04-controller-runtime-plan.md +169 -0
- package/docs/plans/2026-08-05-always-on-workflows-plan.md +125 -0
- package/docs/plans/2026-08-10-agent-managed-monitor-workflows-plan.md +184 -0
- package/docs/plans/2026-08-12-coordinated-workflow-timeouts-plan.md +74 -0
- package/docs/plans/2026-08-13-built-in-workflow-catalog-plan.md +97 -0
- package/docs/run-bundles.md +85 -28
- package/docs/workflows.md +161 -17
- package/examples/controllers/pull-request.controller.ts +215 -0
- package/package.json +10 -2
- package/src/builtins/catalog.ts +22 -0
- package/src/builtins/monitor.workflow.ts +302 -0
- package/src/controllers/conditions.ts +110 -0
- package/src/controllers/definition.ts +65 -0
- package/src/controllers/effects.ts +123 -0
- package/src/controllers/errors.ts +27 -0
- package/src/controllers/index.ts +90 -0
- package/src/controllers/json.ts +62 -0
- package/src/controllers/loader.ts +104 -0
- package/src/controllers/manager.ts +533 -0
- package/src/controllers/results.ts +46 -0
- package/src/controllers/sqlite.ts +1545 -0
- package/src/controllers/store.ts +160 -0
- package/src/controllers/types.ts +183 -0
- package/src/controllers/workflow-engine-scheduler.ts +148 -0
- package/src/controllers/workflows.ts +152 -0
- package/src/extension/controller-host.ts +163 -0
- package/src/extension/executor.ts +13 -2
- package/src/extension/index.ts +1325 -130
- package/src/extension/recorder.ts +65 -36
- package/src/extension/workflow-tool.ts +59 -0
- package/src/host/processes.ts +119 -0
- package/src/host/rpc-bridge.ts +44 -0
- package/src/host/rpc-executor.ts +299 -0
- package/src/host/runner.ts +445 -0
- package/src/viewer/cli.ts +167 -21
- package/src/workflows/catalog.ts +135 -0
- package/src/workflows/engine.ts +404 -41
- package/src/workflows/errors.ts +45 -0
- package/src/workflows/graph.ts +0 -5
- package/src/workflows/index.ts +2 -0
- package/src/workflows/loader.ts +76 -22
- package/src/workflows/migrate-sources.ts +167 -0
- package/src/workflows/schema.ts +3 -2
- package/src/workflows/store.ts +557 -48
- package/src/workflows/types.ts +22 -3
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Add always-on workflow execution
|
|
3
|
+
author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
|
|
4
|
+
date: 2026-08-05
|
|
5
|
+
updated: 2026-08-05
|
|
6
|
+
status: implemented
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Always-on workflows plan
|
|
10
|
+
|
|
11
|
+
Pi Workflows should feel the same whether the user watches a run or walks away from it. In the user's words: "I might start a workflow locally in Pi then I wait for it to complete. All the while I am looking at the screen and I'm not closing the Pi window. When the workflow ends I just want to be able to continue the same Pi session like normal with a session up to date with what happened in the workflow." And: "I just want to interact by starting a workflow, closing it, and then coming back and then still being able to continue it when I open it up. It's syncing continuously or something."
|
|
12
|
+
|
|
13
|
+
These are not two modes. The user asked for "both in a single unified system." This plan makes the Pi window irrelevant to execution: closing or opening the window is a change in observation, not in the run. The work stays on one machine, uses the merged controller runtime as its foundation, and does not modify Pi core.
|
|
14
|
+
|
|
15
|
+
This revision incorporates an external design review of the first draft. The review confirmed the three design rules and found five structural gaps in the details. The Decisions section resolves each one before implementation starts.
|
|
16
|
+
|
|
17
|
+
## Design
|
|
18
|
+
|
|
19
|
+
Four rules define the system:
|
|
20
|
+
|
|
21
|
+
1. **Durable state is the only source of truth.** Run bundles, the queue, and the event log live on disk. Nothing important lives in a session's memory.
|
|
22
|
+
2. **Runners are interchangeable, but ownership is exclusive.** Any live process running the engine claims work from the durable queue. A run has exactly one owning runner at a time, proven by a claim token, and only that runner may write to the run.
|
|
23
|
+
3. **The session is always a view.** It attaches to a run's event stream and renders it. An open window sees a live tail; a reopened window catches up from the same stream. Interaction such as approvals uses durable waiting states, never prompts tied to the window's lifetime.
|
|
24
|
+
4. **Claims arbitrate every lifecycle decision.** Only the current claim holder may resume, interrupt, or write a terminal event for a run. Recovery code that finds an abandoned run goes through the queue instead of writing to the bundle directly.
|
|
25
|
+
|
|
26
|
+
The controller runtime already provides most of the machinery: a deduplicated queue with expiring claims, a structured event table, crash recovery through the trace tail, and guarded effect records. This plan extends that treatment to runs the user starts interactively and adds the view layer.
|
|
27
|
+
|
|
28
|
+
## Decisions
|
|
29
|
+
|
|
30
|
+
These points were open in the first draft. External review showed each one is load-bearing, so they are decided here.
|
|
31
|
+
|
|
32
|
+
1. **Write fencing on bundles.** Every claim records a runner ID, a token, and a lease expiry in the store. Every bundle write verifies the token inside the run lock before appending and fails the runner fast when the token no longer matches. Expiring claims alone do not stop a stalled runner from writing; fencing does.
|
|
33
|
+
2. **Explicit resume protocol.** Resume is a named operation, not a restart. It truncates a torn trace tail to the last complete line, rebuilds run state from the trace, seeds the trace sequence from the tail, accounts for already-executed steps against the step limit, and records a resume boundary event before continuing.
|
|
34
|
+
3. **Resume scope.** User-started runs get node-level resume. Controller child runs keep their current new-attempt semantics, because attempt immutability and parent-side retry already work and are tested. Both behaviors are explicit; nothing mixes silently.
|
|
35
|
+
4. **A dedicated run queue.** Interactive runs do not fit the controller queue, which is keyed to controller resources. A new `workflow_run_queue` table shares the claim, lease, and fencing pattern and adds runner affinity fields. Runs stay out of `/controller list`.
|
|
36
|
+
5. **Origin affinity.** An interactively started run is inserted and claimed in one transaction, so the session that started it owns it from birth. The standalone host takes over only when that claim is released or expires. This guarantees a watched run's conversation happens in the watching session.
|
|
37
|
+
6. **Close-to-park shutdown.** `session_shutdown` stops writing a terminal cancel event for queued runs. It aborts in-flight work without a terminal event and releases the claim, leaving a resumable bundle. A clean close releases claims explicitly; a crash relies on lease expiry.
|
|
38
|
+
7. **Per-attempt capture.** Session capture becomes segmented per attempt, keyed by attempt ID, so a run handed off to the host or a new session starts a fresh capture segment instead of failing integrity checks.
|
|
39
|
+
8. **Continuation runs for waiting.** A run that needs human input ends at a checkpoint with status `waiting`, preserving bundle immutability. An answered checkpoint starts a continuation run with a new run ID chained to its parent, carrying forward prior outputs. The parent link makes the chain inspectable.
|
|
40
|
+
9. **Source pinning.** The manifest stores a content hash of the workflow source at run start. Resume refuses to continue against changed source unless forced, and a forced resume records the mismatch in the trace.
|
|
41
|
+
10. **Snapshot catch-up.** Notifications are idempotent snapshots of run state as of a store sequence number, recomputed from the store, not a stream of one-off messages. The watermark persists per session before sending. A skipped incremental notification is subsumed by the next snapshot.
|
|
42
|
+
11. **A run-level event feed.** Run lifecycle transitions write rows into a store table, so one watermark covers both controller and run events. Tailing `trace.ndjson` is reserved for the single actively watched run.
|
|
43
|
+
12. **Store-error backoff.** Worker loops treat store errors such as `SQLITE_BUSY` as transient and back off instead of letting a worker die silently. The host's advisory lock guards host-versus-host only; the embedded runner does not take it, and a second host refuses to start.
|
|
44
|
+
13. **Orphan reaping.** The host spawns `pi --mode rpc` children in their own process group, records child PIDs in the bundle, and reaps known orphans on startup. Consequential actions stay behind guarded effects regardless.
|
|
45
|
+
|
|
46
|
+
## Requirements
|
|
47
|
+
|
|
48
|
+
- Starting `/workflow run` in a Pi session creates a durable queued run claimed by that session, and shows its progress live.
|
|
49
|
+
- Closing Pi mid-run never loses the run and never writes a spurious terminal event. With the standalone host alive, the host reclaims and resumes the run. Without a host, the run waits and resumes when a runner returns.
|
|
50
|
+
- Reopening a session brings it up to date with an idempotent snapshot: what finished, what failed, what waits for input. No state is duplicated and no information is permanently lost.
|
|
51
|
+
- A run that needs human input ends at a checkpoint. The user answers with a command, and a continuation run carries the work forward. The wait survives any process lifetime.
|
|
52
|
+
- Killing any process at any point recovers without duplicate trace sequences, duplicate notifications of record, or duplicate external effects. A stalled runner that loses its claim can never write again.
|
|
53
|
+
- Everything uses documented Pi public APIs. No Pi core changes.
|
|
54
|
+
|
|
55
|
+
## Work items
|
|
56
|
+
|
|
57
|
+
1. **Run queue and fencing.** Add the `workflow_run_queue` table with claim tokens, lease expiry, and runner affinity. Gate every bundle write on the token inside the run lock. Insert-and-claim interactive starts in one transaction.
|
|
58
|
+
2. **Node-level resume.** Implement the explicit resume protocol from Decision 2, with source pinning from Decision 9. User-started runs resume at the interrupted node; controller child runs are untouched.
|
|
59
|
+
3. **Close-to-park and capture segments.** Change `session_shutdown` to abort-without-terminal plus claim release for queued runs. Split session capture into per-attempt segments the integrity checker understands.
|
|
60
|
+
4. **Continuation runs.** Chain an answered checkpoint to a new run ID with a parent link and carried-forward outputs. Render the chain as one logical run in views.
|
|
61
|
+
5. **Session sync.** Add the run-level event feed, per-session watermarks, snapshot catch-up on `session_start`, and noteworthy-event messages. Live watching tails `trace.ndjson` from a remembered byte offset with `fs.watch`, reusing the TUI viewer's file-tail path.
|
|
62
|
+
6. **Standalone host.** A `pi-workflows` CLI subcommand loads controller definitions, opens the project store, and runs claiming in a loop with store-error backoff. Conversation child nodes run in spawned headless `pi --mode rpc` sessions with orphan reaping. The host takes an advisory lock against other hosts, drains on SIGTERM, and recovers on restart. It is a foreground process the user runs in a terminal; it is not a service.
|
|
63
|
+
|
|
64
|
+
## Non-goals
|
|
65
|
+
|
|
66
|
+
- Pi core changes of any kind. Every integration uses public APIs: commands, session events, widgets, `sendUserMessage`, and `pi --mode rpc`.
|
|
67
|
+
- Multi-machine execution, a remote store, or leader election. SQLite and one machine are in scope; the store contracts leave room for a remote implementation later.
|
|
68
|
+
- Installing or configuring a system or user service. The host is a process the user starts and stops.
|
|
69
|
+
- A push channel from an external process into a live session. Polling the shared store is the mechanism, and it is fast enough.
|
|
70
|
+
- Exactly-once chat notifications. The guarantee is no duplicated state and no permanently lost information, delivered through idempotent snapshots.
|
|
71
|
+
- Exactly-once external side effects beyond the existing guarded effect records.
|
|
72
|
+
|
|
73
|
+
## Assumptions
|
|
74
|
+
|
|
75
|
+
- One machine and one user, with the store and run bundles on the local filesystem.
|
|
76
|
+
- Spawning `pi --mode rpc` per conversation child run is acceptable at the expected cadence. If startup cost proves too high, the host keeps a small pool of persistent RPC sessions instead. Both options stay outside Pi core.
|
|
77
|
+
- A polling interval of a few seconds is responsive enough for the session view. File watching covers the live tail of a watched run.
|
|
78
|
+
- The user does not need machine-sleep or power-loss coverage beyond crash recovery. A stopped machine stops work until a runner returns.
|
|
79
|
+
|
|
80
|
+
## Open questions
|
|
81
|
+
|
|
82
|
+
- Which events deserve a chat message and which belong only in the widget. The default should be quiet.
|
|
83
|
+
- The exact host command shape, for example `pi-workflows run --project <dir>` versus a subcommand under `controllers`.
|
|
84
|
+
|
|
85
|
+
## Departures from the decisions
|
|
86
|
+
|
|
87
|
+
The implementation matches the decisions above with these refinements:
|
|
88
|
+
|
|
89
|
+
- Graph validation now allows outgoing edges from checkpoint nodes. The old
|
|
90
|
+
rejection encoded terminal-forever checkpoints; continuations make those
|
|
91
|
+
edges live. This is a deliberate contract change for workflow authors.
|
|
92
|
+
- The host command is `pi-workflows host`, chosen over `run` because the
|
|
93
|
+
viewer CLI's vocabulary already uses runs for bundles.
|
|
94
|
+
- The first capture stays flat at `session/`; only binds from the second
|
|
95
|
+
recorder onward write segments under `session/segments/`. This keeps the
|
|
96
|
+
layout of every existing single-session bundle and the viewer unchanged.
|
|
97
|
+
- Resume always starts unpaused; the operator can pause again.
|
|
98
|
+
- Headless conversation children keep the exact tool contract: a bridge
|
|
99
|
+
extension loaded with `-e` registers the `workflow` tool and reports
|
|
100
|
+
submissions to the host over stderr, so no engine prompt changes were
|
|
101
|
+
needed.
|
|
102
|
+
- The host accepts explicit `storeFile` and `runsDir` options. Its defaults
|
|
103
|
+
resolve from the project and environment like every other entry point.
|
|
104
|
+
|
|
105
|
+
## Acceptance criteria
|
|
106
|
+
|
|
107
|
+
- Start a run in Pi, then close Pi while a conversation node is mid-response and the host is running. The host reclaims and resumes the run, and reopening Pi shows the catch-up snapshot and allows normal conversation about the result.
|
|
108
|
+
- The same flow without the host: the run resumes at the interrupted node when Pi reopens and completes.
|
|
109
|
+
- A stalled runner that loses its claim cannot write to the bundle afterward; fencing rejects its writes.
|
|
110
|
+
- An open session reflects host-driven progress within a few seconds. Two sessions on one project each get complete catch-up; neither starves the other.
|
|
111
|
+
- A waiting-for-input run survives closing and reopening. Answering it starts a continuation run that carries forward prior outputs.
|
|
112
|
+
- `kill -9` on the host mid-append, followed by a restart, repairs the torn trace tail and recovers without duplicate trace sequence numbers and without repeating an applied external effect. Orphaned RPC children are reaped.
|
|
113
|
+
- No changes to Pi core; the diff touches only this package.
|
|
114
|
+
|
|
115
|
+
## Verification
|
|
116
|
+
|
|
117
|
+
- `npm run check` and `npm run test:e2e`, including new real-Pi E2E tests that start a run, kill the host, restart it, and assert continuation.
|
|
118
|
+
- A fencing test with both runners alive and a forced lease loss.
|
|
119
|
+
- A two-session watermark test on one project.
|
|
120
|
+
- A `kill -9` mid-append torn-tail resume test.
|
|
121
|
+
- A checkpoint answer-to-continuation round-trip test.
|
|
122
|
+
- Extension tests with fake timers for the polling loop, watermark, and snapshot catch-up.
|
|
123
|
+
- `npx slophammer-ts@latest dry .` and the dependency-boundary check.
|
|
124
|
+
- `npx -y @simpledoc/simpledoc check` for documentation changes.
|
|
125
|
+
- Manual pass through both usage patterns from the user's request with the host in a terminal.
|
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Let agents run and manage monitor workflows
|
|
3
|
+
author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
|
|
4
|
+
date: 2026-08-10
|
|
5
|
+
updated: 2026-08-10
|
|
6
|
+
status: implemented
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Agent-managed monitor workflows plan
|
|
10
|
+
|
|
11
|
+
The user should be able to tell an agent, "Monitor this every 30 minutes," and have the agent start the right workflow. The user must not write controller records or JSON. The existing `workflow` model tool should manage workflows instead of serving only as a step-submission tool.
|
|
12
|
+
|
|
13
|
+
A monitor is one Pi Workflows graph. It checks the target, reports a meaningful change, sleeps for the requested interval with the existing shell node, and loops. This plan does not use controllers, Unified Exec, a new wait node, or a second scheduler.
|
|
14
|
+
|
|
15
|
+
## Shipped design
|
|
16
|
+
|
|
17
|
+
The implementation follows this plan. The built-in monitor defaults to 1,000 checks, caps the interval at 24 hours, and uses a 5,010-step engine limit as a second guard. The normal Pi extension exposes all workflow tool actions. The headless RPC bridge exposes only `submit`. A real-Pi end-to-end test proves that a normal model turn can start the built-in monitor and complete its first check. The automated test inspects the exact 30-minute Node timer command and both timeout margins instead of making the test suite wait for 30 minutes. The Node timer keeps the shell action portable across Pi's supported platforms.
|
|
18
|
+
|
|
19
|
+
## User experience
|
|
20
|
+
|
|
21
|
+
A normal request looks like this:
|
|
22
|
+
|
|
23
|
+
> Monitor PR 123 every 30 minutes. Tell me when checks fail. Stop when it is merged or closed.
|
|
24
|
+
|
|
25
|
+
The agent recognizes repeated monitoring from the `workflow` tool description and starts the built-in `monitor` workflow. The tool call contains the structured input, but the user does not write it.
|
|
26
|
+
|
|
27
|
+
The monitor checks immediately, then waits between later checks. It writes a normal assistant message only when the requested report condition is met. It stops when the requested stop condition is met, when it reaches its safety limit, or when the user or agent cancels it.
|
|
28
|
+
|
|
29
|
+
Existing controls remain available:
|
|
30
|
+
|
|
31
|
+
```text
|
|
32
|
+
/workflow pause
|
|
33
|
+
/workflow resume
|
|
34
|
+
/workflow cancel
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
The same operations become available to the model through the `workflow` tool. One active workflow per Pi session remains the rule. A long monitor therefore occupies that session's workflow slot.
|
|
38
|
+
|
|
39
|
+
## Workflow tool
|
|
40
|
+
|
|
41
|
+
Replace the current submit-only schema with one discriminated tool schema. Use Pi's documented `registerTool` API and `StringEnum` helper.
|
|
42
|
+
|
|
43
|
+
The tool supports these actions:
|
|
44
|
+
|
|
45
|
+
- `list`: list discovered workflows and their source.
|
|
46
|
+
- `start`: start a workflow by name or path with structured input.
|
|
47
|
+
- `status`: return a bounded summary of the active or named run.
|
|
48
|
+
- `pause`: pause the active run at its next node boundary.
|
|
49
|
+
- `resume`: resume the active paused run.
|
|
50
|
+
- `cancel`: cancel the active run.
|
|
51
|
+
- `answer`: answer a waiting checkpoint, with an optional run ID.
|
|
52
|
+
- `submit`: submit the result required by the current workflow step contract.
|
|
53
|
+
|
|
54
|
+
The `submit` form becomes:
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
{
|
|
58
|
+
"action": "submit",
|
|
59
|
+
"step": "check",
|
|
60
|
+
"attempt": "attempt-id",
|
|
61
|
+
"output": {}
|
|
62
|
+
}
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Update workflow step prompts and reminders to include `action: "submit"`. Do not retain the old submit shape as a compatibility alias.
|
|
66
|
+
|
|
67
|
+
Slash commands and tool actions must call the same internal lifecycle functions. This avoids separate behavior for humans and models. Tool results must be structured and bounded. Errors must state whether the operation failed because no run exists, a run is already active, a checkpoint is not waiting, or the requested workflow cannot be found.
|
|
68
|
+
|
|
69
|
+
A tool call happens during an agent turn. The `start` action must validate and queue the launch, then deliver the first workflow prompt after the initiating turn settles. This prevents the new run from treating the initiating turn as a failed workflow step. It also prevents a workflow prompt from being injected in the middle of the tool call.
|
|
70
|
+
|
|
71
|
+
The headless RPC bridge keeps a submit-only version of the tool. Headless workflow children must not start or control other workflows. Its submit schema and parser still change to require `action: "submit"`.
|
|
72
|
+
|
|
73
|
+
## Agent guidance
|
|
74
|
+
|
|
75
|
+
The tool description must tell the model when to use the built-in workflow:
|
|
76
|
+
|
|
77
|
+
- Use `start` with workflow `monitor` when the user asks to watch, monitor, poll, or check something repeatedly.
|
|
78
|
+
- Start directly when the task and interval are clear.
|
|
79
|
+
- Ask one short question when the target or interval is missing.
|
|
80
|
+
- Use observation-only behavior unless the user explicitly authorizes a mutation.
|
|
81
|
+
- Do not create repeated work without a user request or an existing workflow instruction that authorizes it.
|
|
82
|
+
|
|
83
|
+
The `list` result identifies `monitor` as a built-in workflow and gives a short description. This keeps discovery inside the tool instead of adding a skill or a system-prompt injection.
|
|
84
|
+
|
|
85
|
+
## Built-in monitor workflow
|
|
86
|
+
|
|
87
|
+
Ship `monitor` as a built-in workflow in the Pi Workflows package. Built-ins have the lowest discovery precedence:
|
|
88
|
+
|
|
89
|
+
1. Project workflows under `.pi/workflows/`
|
|
90
|
+
2. Global workflows under `~/.pi/agent/workflows/`
|
|
91
|
+
3. Workflows bundled with Pi Workflows
|
|
92
|
+
|
|
93
|
+
A project or global `monitor.workflow.ts` can therefore replace the default. The built-in remains a real workflow file so run bundles can record its path and source hash with the existing rules.
|
|
94
|
+
|
|
95
|
+
The workflow input contains:
|
|
96
|
+
|
|
97
|
+
- `task`: what to inspect.
|
|
98
|
+
- `everyMinutes`: the interval between checks.
|
|
99
|
+
- `reportWhen`: changes or states that deserve a user message.
|
|
100
|
+
- `stopWhen`: the condition that ends monitoring.
|
|
101
|
+
- `maxChecks`: a hard safety limit.
|
|
102
|
+
|
|
103
|
+
The model supplies these fields from the user's request. The workflow validates the input and applies documented bounds. The first release supports intervals from 1 minute through 24 hours. It defaults `maxChecks` to a documented finite value when the request gives no end limit.
|
|
104
|
+
|
|
105
|
+
The graph is:
|
|
106
|
+
|
|
107
|
+
```text
|
|
108
|
+
prepare -> guard -> check
|
|
109
|
+
| continue quietly -> sleep -> guard
|
|
110
|
+
| continue and report -> report -> sleep -> guard
|
|
111
|
+
| stop quietly -> finish
|
|
112
|
+
| stop and report -> report-final -> finish
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
`prepare` validates and normalizes the input. `guard` enforces `maxChecks`. `check` is an agent node that performs one observation and returns a validated route, a bounded observation, and an optional report. The next `check` can read the previous accepted `check` output, which Pi Workflows already keeps for looped nodes.
|
|
116
|
+
|
|
117
|
+
The report nodes write a normal assistant message and then submit an acknowledgement. Keeping reporting after accepted check output prevents the agent from showing a report before the structured result passes validation. The final presentation reports why the monitor stopped without repeating a report that the user already saw.
|
|
118
|
+
|
|
119
|
+
The sleep node uses the existing Pi Workflows shell action to launch the current Node executable with a timer. Set the node timeout above the largest supported interval because the engine default is 15 minutes. Set the shell execution timeout above the requested wait by a small fixed margin. Cancellation aborts the timer process immediately.
|
|
120
|
+
|
|
121
|
+
If the Pi TUI or standalone workflow host stops during sleep, Pi Workflows parks the run and kills the shell child. Resuming the run starts that sleep node again from the beginning. This is existing workflow behavior and is acceptable for this feature. No special timer persistence is added.
|
|
122
|
+
|
|
123
|
+
The workflow uses a high but finite `maxSteps` value as a second safety guard. Check and report values have explicit size limits so a long run cannot grow its bundle without bound.
|
|
124
|
+
|
|
125
|
+
## Repository changes
|
|
126
|
+
|
|
127
|
+
Make the feature in `osolmaz/pi-workflows`:
|
|
128
|
+
|
|
129
|
+
- Refactor workflow lifecycle operations out of the slash-command handler.
|
|
130
|
+
- Expand the normal Pi `workflow` tool and update its tests.
|
|
131
|
+
- Update the RPC bridge submit contract.
|
|
132
|
+
- Add built-in workflow discovery with project and global override precedence.
|
|
133
|
+
- Add the built-in monitor workflow and focused tests.
|
|
134
|
+
- Update `README.md` and `docs/workflows.md`.
|
|
135
|
+
|
|
136
|
+
After the upstream change is complete, update the pinned Pi Workflows commit in OnurPi's thin `packages/workflows` wrapper. Do not add a new OnurPi extension or copy a monitor file into live global state.
|
|
137
|
+
|
|
138
|
+
## State and API impact
|
|
139
|
+
|
|
140
|
+
- **Session state:** Normal workflow prompts, model replies, tool calls, and monitor reports are appended through Pi's normal session behavior.
|
|
141
|
+
- **Other persistent data:** No new data model. The feature uses existing run bundles and the existing workflow run queue.
|
|
142
|
+
- **Pi internals:** None.
|
|
143
|
+
- **Pi public API:** `registerTool`, `registerCommand`, `sendUserMessage`, and documented agent and session lifecycle events.
|
|
144
|
+
- **Pi Workflows API:** The workflow definition and run-state models do not change. The model-facing `workflow` tool contract changes, and discovery gains a lowest-priority built-in source.
|
|
145
|
+
|
|
146
|
+
## Non-goals
|
|
147
|
+
|
|
148
|
+
This work does not add cron expressions, calendar schedules, a background service, controller resources, OS notifications, concurrent workflows in one session, resumable shell processes, or guaranteed wall-clock wake times across runner shutdowns. It does not let headless child agents recursively start workflows.
|
|
149
|
+
|
|
150
|
+
## Acceptance criteria
|
|
151
|
+
|
|
152
|
+
- A user can ask for repeated monitoring in plain language, and the agent starts `monitor` through the existing `workflow` tool.
|
|
153
|
+
- The user does not type JSON or a slash command to start the monitor.
|
|
154
|
+
- The monitor checks immediately, sleeps for the requested interval, and checks again.
|
|
155
|
+
- An unchanged observation produces no normal assistant report.
|
|
156
|
+
- A matching report condition produces one concise assistant report.
|
|
157
|
+
- A matching stop condition reports as requested and ends the run.
|
|
158
|
+
- The safety limit ends a monitor that never reaches its stop condition.
|
|
159
|
+
- The model can list, start, inspect, pause, resume, cancel, answer, and submit through one tool.
|
|
160
|
+
- Slash commands and tool actions use the same lifecycle code.
|
|
161
|
+
- Starting from a model tool call does not cause an early nudge or record the initiating turn as the first workflow attempt.
|
|
162
|
+
- A 30-minute sleep is not stopped by the default 15-minute node timeout.
|
|
163
|
+
- Cancelling during sleep stops the shell child and ends the workflow.
|
|
164
|
+
- Project and global workflows override the built-in `monitor` name.
|
|
165
|
+
- Existing workflow, controller, host, viewer, and run-bundle tests continue to pass.
|
|
166
|
+
- OnurPi loads the updated wrapper and Pi starts successfully.
|
|
167
|
+
|
|
168
|
+
## Verification
|
|
169
|
+
|
|
170
|
+
Add unit and integration coverage for tool action validation, lifecycle dispatch, deferred model-start launches, built-in discovery precedence, monitor routing, quiet checks, reports, stop conditions, safety limits, cancellation during sleep, and the headless submit bridge.
|
|
171
|
+
|
|
172
|
+
Run:
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
npm run check
|
|
176
|
+
npm run test:e2e
|
|
177
|
+
npx slophammer-ts@latest dry .
|
|
178
|
+
npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
|
|
179
|
+
npx -y @simpledoc/simpledoc check
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
Test the extension from the Pi Workflows checkout with `pi -e src/extension/index.ts`. Use a short test interval in a controlled fixture, then perform one manual 30-minute monitor run to confirm that the configured node timeout does not stop it. Test plain-language startup with the normal model, then test list, status, pause, resume, cancel, and checkpoint answer actions.
|
|
183
|
+
|
|
184
|
+
After updating OnurPi, run its full checks and start Pi with the installed OnurPi package. Confirm that the model sees one `workflow` tool, discovers `monitor`, and can start it from a plain-language request.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Coordinate Workflow Timeouts With Pi Turns
|
|
3
|
+
author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
|
|
4
|
+
date: 2026-08-12
|
|
5
|
+
status: implemented
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Coordinate Workflow Timeouts With Pi Turns
|
|
9
|
+
|
|
10
|
+
A workflow agent node can time out while its Pi turn keeps running. The model can then continue to use tools and submit output for an attempt that the workflow engine has already closed. A failed or timed-out monitor can also call its normal result presenter without a final output. This plan makes the workflow deadline own the corresponding Pi turn and gives monitor checks an explicit timeout.
|
|
11
|
+
|
|
12
|
+
## Requirements
|
|
13
|
+
|
|
14
|
+
- Stop the active Pi turn when its workflow agent node times out, is cancelled, parks, or loses its claim.
|
|
15
|
+
- Keep user Escape behavior separate. A user interrupt must still pause the workflow until `/workflow resume`.
|
|
16
|
+
- Reject late submissions after the pending attempt closes.
|
|
17
|
+
- Let workflow nodes derive a timeout from their input through a documented callback.
|
|
18
|
+
- Give the built-in monitor an optional `checkTimeoutMinutes` input with safe bounds and a useful default.
|
|
19
|
+
- Run normal result presentation only for completed runs and waiting checkpoints.
|
|
20
|
+
- Keep the workflow engine independent of Pi.
|
|
21
|
+
- Preserve existing run-bundle and workflow file formats.
|
|
22
|
+
|
|
23
|
+
## Scope
|
|
24
|
+
|
|
25
|
+
The workflow engine will expose agent-step aborts through its existing `AbortSignal`. The Pi extension will bind that signal to the Pi turn by calling the documented `ExtensionContext.abort()` method. It will record the attempt that requested the abort so the later `agent_end` event does not treat that abort as a user interrupt.
|
|
26
|
+
|
|
27
|
+
`WorkflowNodeCommon.timeoutMs` will accept a positive number or a callback that returns a positive number from the node context. The engine will resolve and validate that value before it dispatches the node. Existing numeric timeouts and the 15-minute engine default will continue to work.
|
|
28
|
+
|
|
29
|
+
The monitor will accept `checkTimeoutMinutes`. Its default will be the larger of 60 minutes and `everyMinutes`, with a maximum of 24 hours. The timeout will apply to check and report agent nodes. The shell wait keeps its current interval-specific timeout.
|
|
30
|
+
|
|
31
|
+
## Non-goals
|
|
32
|
+
|
|
33
|
+
- Do not change Pi internals.
|
|
34
|
+
- Do not add a persistent timeout registry or change the run-state schema.
|
|
35
|
+
- Do not make timed-out runs resumable.
|
|
36
|
+
- Do not remove safety deadlines from workflow nodes.
|
|
37
|
+
- Do not add compatibility code for submissions that omit `action: "submit"`.
|
|
38
|
+
|
|
39
|
+
## Implementation
|
|
40
|
+
|
|
41
|
+
1. Extend the conversation executor with an `onAbort` callback. Call it once when the engine abort signal closes a pending agent step. Pass the attempt contract and abort reason.
|
|
42
|
+
2. In the Pi extension, keep a short-lived in-memory marker for the system-aborted attempt. Call `ctx.abort()` only when Pi is still busy. Clear the marker after the matching aborted agent end or when no turn was active.
|
|
43
|
+
3. In `agent_end`, pause only for unmarked aborted turns. System aborts leave the engine's terminal or parked state in control.
|
|
44
|
+
4. Resolve node timeout callbacks before creating the timer. Validate that the result is a finite positive number. Abort timeout resolution after a short fixed deadline so a callback cannot block node dispatch forever.
|
|
45
|
+
5. Add and validate `checkTimeoutMinutes` in the monitor. Apply the derived timeout to `check`, `report_continue`, and `report_stop`.
|
|
46
|
+
6. Restrict normal result presentation to `completed` and `waiting` states. Terminal failures continue to use the existing deterministic notification with the persisted error.
|
|
47
|
+
7. Add tests for timeout-driven Pi abort, user Escape, late submissions, dynamic timeout validation, monitor timeout selection, and failed-run presentation.
|
|
48
|
+
8. Add a real Pi RPC regression test where a model turn exceeds a short workflow timeout. Confirm that Pi aborts the turn, the run times out, and later tool work does not occur.
|
|
49
|
+
|
|
50
|
+
## Acceptance criteria
|
|
51
|
+
|
|
52
|
+
- A timed-out workflow agent node aborts its active Pi turn exactly once.
|
|
53
|
+
- The timed-out attempt no longer accepts output.
|
|
54
|
+
- A system-triggered abort does not pause the workflow as a user interrupt.
|
|
55
|
+
- Escape continues to pause and `/workflow resume` continues the pending step.
|
|
56
|
+
- Monitor checks use `checkTimeoutMinutes`, or the documented default when it is absent.
|
|
57
|
+
- Failed, timed-out, and cancelled runs never call `presentationPrompt`.
|
|
58
|
+
- Completed and waiting runs keep their current presentation behavior.
|
|
59
|
+
- Existing workflow definitions with numeric or omitted timeouts continue to work.
|
|
60
|
+
- No Pi internal API or persistent schema changes.
|
|
61
|
+
|
|
62
|
+
## Verification
|
|
63
|
+
|
|
64
|
+
Run:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
npm run check
|
|
68
|
+
npm run test:e2e
|
|
69
|
+
npx slophammer-ts@latest dry .
|
|
70
|
+
npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
|
|
71
|
+
npx -y @simpledoc/simpledoc check
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Also start Pi through the package in RPC mode and complete a `get_state` request. The real Pi end-to-end suite must exercise timeout-driven turn cancellation.
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Give Built-in Workflows Stable Identities
|
|
3
|
+
author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
|
|
4
|
+
date: 2026-08-13
|
|
5
|
+
status: implemented
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Give Built-in Workflows Stable Identities
|
|
9
|
+
|
|
10
|
+
A long-lived Pi process must never combine an old workflow engine with a new built-in workflow file. Built-in workflows also need identities that survive package paths, source and distribution layouts, and later package updates. This plan replaces path-based built-in loading with a general catalog for all current and future built-ins.
|
|
11
|
+
|
|
12
|
+
## Requirements
|
|
13
|
+
|
|
14
|
+
- Load every built-in workflow and its validator from one process module graph.
|
|
15
|
+
- Identify built-ins with stable references such as `builtin:monitor`.
|
|
16
|
+
- Record a built-in revision and refuse resume when the loaded revision differs.
|
|
17
|
+
- Keep project and global workflow files hot-reloadable and path-based.
|
|
18
|
+
- Keep project and global workflows able to override a built-in by name.
|
|
19
|
+
- Let the Pi extension and standalone host resolve the same stable reference.
|
|
20
|
+
- Preserve existing nonterminal built-in runs through one bounded migration.
|
|
21
|
+
- Remove old built-in file paths from active run bundles and queue records after migration.
|
|
22
|
+
- Apply the design to all built-ins, not only the monitor.
|
|
23
|
+
|
|
24
|
+
## Data model
|
|
25
|
+
|
|
26
|
+
A resolved workflow source is one of two forms:
|
|
27
|
+
|
|
28
|
+
```typescript
|
|
29
|
+
type WorkflowSource =
|
|
30
|
+
{ kind: "builtin"; id: string; revision: string } | { kind: "file"; path: string; hash: string };
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
`kind` is the discriminator. A built-in `id` is stable across installations. A built-in `revision` is an explicit catalog value that maintainers change when the definition changes. A file source keeps its absolute path and SHA-256 hash.
|
|
34
|
+
|
|
35
|
+
Run state and the run manifest will store `workflowSource`. Queue records will store a display name and one canonical source reference: `builtin:<id>` for built-ins and an absolute path for files. New code will stop writing `workflowPath` and `workflowHash` to run bundles.
|
|
36
|
+
|
|
37
|
+
This is a deliberate in-place change to the version 1 run-state and controller-store contracts. The migration is bounded to existing nonterminal built-in runs. It rewrites their path and hash fields to `workflowSource`, then updates their nonterminal queue rows to `builtin:<id>`. It does not keep a runtime fallback reader after migration.
|
|
38
|
+
|
|
39
|
+
## Architecture
|
|
40
|
+
|
|
41
|
+
The workflow layer will define a generic `BuiltinWorkflowCatalog`. The built-ins layer will create the catalog from imported workflow definitions. The Pi extension and standalone host will receive or import that catalog and pass it to the generic resolver. The workflow engine will receive a resolved `WorkflowSource`; it will not know how built-ins were registered.
|
|
42
|
+
|
|
43
|
+
The catalog will:
|
|
44
|
+
|
|
45
|
+
- validate stable built-in IDs;
|
|
46
|
+
- reject duplicate IDs and workflow names;
|
|
47
|
+
- compute a deterministic revision from the definition snapshot and package revision input;
|
|
48
|
+
- discover built-ins by name;
|
|
49
|
+
- resolve `builtin:<id>` without disk access;
|
|
50
|
+
- return the same imported definition for the lifetime of the process.
|
|
51
|
+
|
|
52
|
+
## Migration
|
|
53
|
+
|
|
54
|
+
Migration runs before a session or host claims resumable workflow work.
|
|
55
|
+
|
|
56
|
+
For each nonterminal run bundle:
|
|
57
|
+
|
|
58
|
+
1. Read the stored workflow path and hash.
|
|
59
|
+
2. Match only a catalog entry's registered legacy path pattern and workflow name.
|
|
60
|
+
3. Require the old hash to equal an explicitly registered legacy revision for that built-in.
|
|
61
|
+
4. Rewrite the manifest and state to `workflowSource: { kind: "builtin", id, revision }` with atomic file replacements.
|
|
62
|
+
5. Rewrite the matching nonterminal queue source reference to `builtin:<id>`.
|
|
63
|
+
6. Leave terminal bundles unchanged.
|
|
64
|
+
|
|
65
|
+
If identity or revision cannot be proved, leave the run unchanged and report a clear blocker. Do not guess from the filename alone.
|
|
66
|
+
|
|
67
|
+
## Scope and non-goals
|
|
68
|
+
|
|
69
|
+
This changes Pi Workflows only. It uses no Pi internals and changes no Pi session entry. It updates Pi Workflows run bundles and controller queue records as described above.
|
|
70
|
+
|
|
71
|
+
It does not add runtime compatibility readers, aliases, dual-write fields, or a permanent migration service. It does not hot-reload package-provided built-ins. A package update takes effect after Pi reload or restart.
|
|
72
|
+
|
|
73
|
+
## Acceptance criteria
|
|
74
|
+
|
|
75
|
+
- Starting `monitor` records `workflowSource.kind === "builtin"` and `id === "monitor"`.
|
|
76
|
+
- No new built-in run records an installation path as its identity.
|
|
77
|
+
- One process keeps its imported built-ins after files on disk change.
|
|
78
|
+
- Source-loaded Pi and the distribution-loaded host resolve the same built-in reference and revision.
|
|
79
|
+
- Project and global monitor overrides remain file sources and still reload.
|
|
80
|
+
- A changed built-in revision parks or refuses resume with a clear source-change error.
|
|
81
|
+
- Known existing nonterminal monitor runs migrate and resume.
|
|
82
|
+
- Unknown legacy paths or hashes are not migrated.
|
|
83
|
+
- No runtime fallback for old built-in paths remains after migration.
|
|
84
|
+
|
|
85
|
+
## Verification
|
|
86
|
+
|
|
87
|
+
Run:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
npm run check
|
|
91
|
+
npm run test:e2e
|
|
92
|
+
npx slophammer-ts@latest dry .
|
|
93
|
+
npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
|
|
94
|
+
npx -y @simpledoc/simpledoc check
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Also test a source-loaded Pi extension, a distribution-loaded standalone host, a project override, a changed built-in revision, and a copied legacy nonterminal run bundle with its matching queue row. After OnurPi updates its immutable pin, start a fresh Pi RPC process and run the built-in monitor through its first check.
|