@osolmaz/pi-workflows 0.12.1 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (209) hide show
  1. package/README.md +52 -28
  2. package/dist/builtins/autodoc.workflow.d.ts +4 -4
  3. package/dist/builtins/autoimplement.workflow.d.ts +369 -73
  4. package/dist/builtins/autoplan.workflow.d.ts +126 -13
  5. package/dist/builtins/autoplan.workflow.js +200 -29
  6. package/dist/builtins/autoplan.workflow.js.map +1 -1
  7. package/dist/builtins/catalog.js +6 -4
  8. package/dist/builtins/catalog.js.map +1 -1
  9. package/dist/builtins/index.d.ts +2 -1
  10. package/dist/builtins/index.js +1 -0
  11. package/dist/builtins/index.js.map +1 -1
  12. package/dist/builtins/monitor.workflow.d.ts +56 -21
  13. package/dist/builtins/monitor.workflow.js +497 -216
  14. package/dist/builtins/monitor.workflow.js.map +1 -1
  15. package/dist/builtins/plain-summary.workflow.d.ts +32 -0
  16. package/dist/builtins/plain-summary.workflow.js +134 -0
  17. package/dist/builtins/plain-summary.workflow.js.map +1 -0
  18. package/dist/builtins/plan-change.workflow.d.ts +170 -22
  19. package/dist/builtins/sanity-check.workflow.d.ts +45 -3
  20. package/dist/builtins/sanity-check.workflow.js +45 -7
  21. package/dist/builtins/sanity-check.workflow.js.map +1 -1
  22. package/dist/controllers/effects.d.ts +3 -2
  23. package/dist/controllers/effects.js +8 -1
  24. package/dist/controllers/effects.js.map +1 -1
  25. package/dist/controllers/index.d.ts +1 -1
  26. package/dist/controllers/index.js +1 -1
  27. package/dist/controllers/index.js.map +1 -1
  28. package/dist/controllers/manager.d.ts +1 -0
  29. package/dist/controllers/manager.js +41 -17
  30. package/dist/controllers/manager.js.map +1 -1
  31. package/dist/controllers/sqlite.d.ts +107 -85
  32. package/dist/controllers/sqlite.js +1611 -1486
  33. package/dist/controllers/sqlite.js.map +1 -1
  34. package/dist/controllers/store.d.ts +9 -8
  35. package/dist/controllers/store.js +1 -31
  36. package/dist/controllers/store.js.map +1 -1
  37. package/dist/controllers/types.d.ts +3 -0
  38. package/dist/controllers/workflow-engine-scheduler.d.ts +1 -1
  39. package/dist/controllers/workflow-engine-scheduler.js +3 -5
  40. package/dist/controllers/workflow-engine-scheduler.js.map +1 -1
  41. package/dist/controllers/workflows.d.ts +2 -2
  42. package/dist/controllers/workflows.js +11 -10
  43. package/dist/controllers/workflows.js.map +1 -1
  44. package/dist/extension/controller-host.js +2 -2
  45. package/dist/extension/controller-host.js.map +1 -1
  46. package/dist/extension/decision-channels.d.ts +2 -0
  47. package/dist/extension/decision-channels.js +311 -118
  48. package/dist/extension/decision-channels.js.map +1 -1
  49. package/dist/extension/executor.d.ts +9 -2
  50. package/dist/extension/executor.js +89 -2
  51. package/dist/extension/executor.js.map +1 -1
  52. package/dist/extension/herdr-viewer.d.ts +0 -1
  53. package/dist/extension/herdr-viewer.js +0 -4
  54. package/dist/extension/herdr-viewer.js.map +1 -1
  55. package/dist/extension/index.js +156 -103
  56. package/dist/extension/index.js.map +1 -1
  57. package/dist/extension/recorder.d.ts +3 -10
  58. package/dist/extension/recorder.js +13 -22
  59. package/dist/extension/recorder.js.map +1 -1
  60. package/dist/extension/step-message.d.ts +7 -2
  61. package/dist/extension/step-message.js +94 -2
  62. package/dist/extension/step-message.js.map +1 -1
  63. package/dist/extension/widget.js +6 -0
  64. package/dist/extension/widget.js.map +1 -1
  65. package/dist/host/rpc-executor.d.ts +3 -0
  66. package/dist/host/rpc-executor.js +2 -0
  67. package/dist/host/rpc-executor.js.map +1 -1
  68. package/dist/host/runner.d.ts +3 -6
  69. package/dist/host/runner.js +83 -32
  70. package/dist/host/runner.js.map +1 -1
  71. package/dist/render/graph-render.js +14 -8
  72. package/dist/render/graph-render.js.map +1 -1
  73. package/dist/state/database.d.ts +35 -0
  74. package/dist/state/database.js +287 -0
  75. package/dist/state/database.js.map +1 -0
  76. package/dist/state/index.d.ts +4 -0
  77. package/dist/state/index.js +5 -0
  78. package/dist/state/index.js.map +1 -0
  79. package/dist/state/json.d.ts +6 -0
  80. package/dist/state/json.js +38 -0
  81. package/dist/state/json.js.map +1 -0
  82. package/dist/state/mutation.d.ts +57 -0
  83. package/dist/state/mutation.js +255 -0
  84. package/dist/state/mutation.js.map +1 -0
  85. package/dist/state/schema.d.ts +6 -0
  86. package/dist/state/schema.js +473 -0
  87. package/dist/state/schema.js.map +1 -0
  88. package/dist/viewer/cli.d.ts +2 -2
  89. package/dist/viewer/cli.js +142 -122
  90. package/dist/viewer/cli.js.map +1 -1
  91. package/dist/viewer/render.d.ts +4 -4
  92. package/dist/viewer/render.js +2 -17
  93. package/dist/viewer/render.js.map +1 -1
  94. package/dist/viewer/session-reducer.d.ts +1 -1
  95. package/dist/viewer/session-reducer.js +1 -1
  96. package/dist/viewer/tui.d.ts +3 -3
  97. package/dist/viewer/tui.js +15 -13
  98. package/dist/viewer/tui.js.map +1 -1
  99. package/dist/viewer/watch.d.ts +2 -5
  100. package/dist/viewer/watch.js +13 -13
  101. package/dist/viewer/watch.js.map +1 -1
  102. package/dist/workflows/composition.js +12 -2
  103. package/dist/workflows/composition.js.map +1 -1
  104. package/dist/workflows/definition.d.ts +6 -2
  105. package/dist/workflows/definition.js +17 -0
  106. package/dist/workflows/definition.js.map +1 -1
  107. package/dist/workflows/engine.d.ts +6 -6
  108. package/dist/workflows/engine.js +183 -103
  109. package/dist/workflows/engine.js.map +1 -1
  110. package/dist/workflows/errors.d.ts +3 -3
  111. package/dist/workflows/errors.js +3 -3
  112. package/dist/workflows/human-decision.d.ts +29 -4
  113. package/dist/workflows/human-decision.js +547 -283
  114. package/dist/workflows/human-decision.js.map +1 -1
  115. package/dist/workflows/index.d.ts +4 -5
  116. package/dist/workflows/index.js +3 -4
  117. package/dist/workflows/index.js.map +1 -1
  118. package/dist/workflows/schema.js +17 -1
  119. package/dist/workflows/schema.js.map +1 -1
  120. package/dist/workflows/store.d.ts +83 -103
  121. package/dist/workflows/store.js +902 -951
  122. package/dist/workflows/store.js.map +1 -1
  123. package/dist/workflows/types.d.ts +55 -81
  124. package/docs/CONTROLLERS.md +10 -8
  125. package/docs/DEFERRED_TURNS.md +2 -2
  126. package/docs/DESIGN_PHILOSOPHY.md +4 -2
  127. package/docs/HUMAN_DECISIONS.md +13 -14
  128. package/docs/MONITOR.md +179 -231
  129. package/docs/SQLITE_STATE.md +219 -0
  130. package/docs/WORKFLOW_COMPOSITION.md +19 -19
  131. package/docs/WORKFLOW_STEP_MESSAGES.md +20 -18
  132. package/docs/WORKFLOW_UPDATES.md +10 -9
  133. package/docs/development.md +9 -9
  134. package/docs/live-replay-protocol.md +28 -37
  135. package/docs/plans/2026-08-16-workflow-updates-plan.md +1 -1
  136. package/docs/plans/2026-08-19-human-decision-gates-plan.md +1 -1
  137. package/docs/plans/2026-08-21-autoimplement-timeout-fallback-plan.md +1 -1
  138. package/docs/plans/2026-08-21-plan-change-approval-policy-plan.md +1 -1
  139. package/docs/plans/2026-08-21-sanity-check-plan.md +30 -16
  140. package/docs/plans/2026-08-22-goal-finishing-monitor-plan.md +200 -0
  141. package/docs/plans/2026-08-23-assistant-agent-completion-plan.md +701 -0
  142. package/docs/plans/2026-08-23-sqlite-state-plan.md +970 -0
  143. package/docs/plans/replayable-run-bundles-implementation-plan.md +2 -2
  144. package/docs/plans/session-event-replay-implementation-plan.md +1 -1
  145. package/docs/plans/tui-viewer-implementation-plan.md +1 -1
  146. package/docs/session-event-journal.md +63 -434
  147. package/docs/tui-viewer.md +13 -15
  148. package/docs/workflows.md +104 -50
  149. package/examples/workflows/plain-summary.workflow.ts +1 -0
  150. package/herdr-plugin.toml +1 -1
  151. package/package.json +1 -1
  152. package/plugins/herdr/viewer.mjs +1 -11
  153. package/skills/autoplan/SKILL.md +5 -4
  154. package/skills/monitor/SKILL.md +91 -163
  155. package/skills/pi-workflows/SKILL.md +7 -7
  156. package/skills/sanity-check/SKILL.md +1 -1
  157. package/src/builtins/autoplan.workflow.ts +290 -40
  158. package/src/builtins/catalog.ts +6 -4
  159. package/src/builtins/index.ts +7 -1
  160. package/src/builtins/monitor.workflow.ts +663 -249
  161. package/src/builtins/plain-summary.workflow.ts +185 -0
  162. package/src/builtins/sanity-check.workflow.ts +62 -7
  163. package/src/controllers/effects.ts +7 -0
  164. package/src/controllers/index.ts +0 -6
  165. package/src/controllers/manager.ts +72 -32
  166. package/src/controllers/sqlite.ts +2420 -1948
  167. package/src/controllers/store.ts +17 -38
  168. package/src/controllers/types.ts +3 -0
  169. package/src/controllers/workflow-engine-scheduler.ts +4 -8
  170. package/src/controllers/workflows.ts +45 -21
  171. package/src/extension/controller-host.ts +1 -2
  172. package/src/extension/decision-channels.ts +427 -144
  173. package/src/extension/executor.ts +105 -2
  174. package/src/extension/herdr-viewer.ts +0 -5
  175. package/src/extension/index.ts +180 -118
  176. package/src/extension/recorder.ts +13 -22
  177. package/src/extension/step-message.ts +106 -4
  178. package/src/extension/widget.ts +8 -0
  179. package/src/host/rpc-executor.ts +4 -0
  180. package/src/host/runner.ts +93 -41
  181. package/src/render/graph-render.ts +11 -4
  182. package/src/state/database.ts +375 -0
  183. package/src/state/index.ts +29 -0
  184. package/src/state/json.ts +42 -0
  185. package/src/state/mutation.ts +435 -0
  186. package/src/state/schema.ts +475 -0
  187. package/src/viewer/cli.ts +151 -144
  188. package/src/viewer/render.ts +12 -24
  189. package/src/viewer/session-reducer.ts +1 -1
  190. package/src/viewer/tui.ts +18 -17
  191. package/src/viewer/watch.ts +13 -17
  192. package/src/workflows/composition.ts +12 -3
  193. package/src/workflows/definition.ts +36 -3
  194. package/src/workflows/engine.ts +212 -116
  195. package/src/workflows/errors.ts +3 -3
  196. package/src/workflows/human-decision.ts +795 -340
  197. package/src/workflows/index.ts +15 -19
  198. package/src/workflows/schema.ts +19 -1
  199. package/src/workflows/store.ts +1295 -1163
  200. package/src/workflows/types.ts +61 -81
  201. package/dist/workflows/artifacts.d.ts +0 -40
  202. package/dist/workflows/artifacts.js +0 -155
  203. package/dist/workflows/artifacts.js.map +0 -1
  204. package/dist/workflows/migrate-sources.d.ts +0 -42
  205. package/dist/workflows/migrate-sources.js +0 -133
  206. package/dist/workflows/migrate-sources.js.map +0 -1
  207. package/docs/run-bundles.md +0 -481
  208. package/src/workflows/artifacts.ts +0 -188
  209. package/src/workflows/migrate-sources.ts +0 -178
@@ -1,12 +1,12 @@
1
1
  # Replayable run bundles — implementation plan
2
2
 
3
3
  Goal: make every workflow run bundle self-contained for replay, as specified
4
- in [run-bundles.md](../run-bundles.md). This is a hard cutover: one format,
4
+ in [run-bundles.md](../SQLITE_STATE.md). This is a hard cutover: one format,
5
5
  no readers for the previous layout, no compatibility code.
6
6
 
7
7
  ## Steps
8
8
 
9
- 1. **Spec** — rewrite `docs/run-bundles.md`: trace-first write discipline,
9
+ 1. **Spec** — rewrite `docs/SQLITE_STATE.md`: trace-first write discipline,
10
10
  `traceSeq` in `state.json`, content-addressed `artifacts/`, `session/`
11
11
  with binding + verbatim Pi entries, explicit `conversation` entry ranges
12
12
  on agent steps, event payload catalog. Review the data model with
@@ -343,7 +343,7 @@ resnapshot.
343
343
  ### Documentation and format replacement
344
344
 
345
345
  When code and tests pass, fold the target contract into
346
- `docs/run-bundles.md` and remove the target-status note from
346
+ `docs/SQLITE_STATE.md` and remove the target-status note from
347
347
  `docs/session-event-journal.md`. Update `docs/live-replay-protocol.md`,
348
348
  `docs/tui-viewer.md`, `docs/development.md`, and the README where users need new
349
349
  commands or controls.
@@ -30,7 +30,7 @@ way whether they come from the local filesystem or a WebSocket.
30
30
  1. **Scaffold** — Cargo crate (ratatui, crossterm, tokio, tokio-tungstenite,
31
31
  notify, serde, clap), rustfmt/clippy config, CI job alongside the npm
32
32
  checks.
33
- 2. **Bundle module** — serde types mirroring `docs/run-bundles.md`, bundle
33
+ 2. **Bundle module** — serde types mirroring `docs/SQLITE_STATE.md`, bundle
34
34
  reader (manifest-first, schema check, skip unknown), incremental NDJSON
35
35
  tailer tolerating torn lines, runs-directory watcher with polling
36
36
  fallback, artifact resolution with path containment.
@@ -1,470 +1,99 @@
1
- # Session event journal specification
2
-
3
- This document defines the target format for temporal Pi session history in a
4
- workflow run bundle. The journal records the order and timing of turns,
5
- messages, assistant output, and tool execution while a workflow runs.
6
-
7
- The format extends the authoritative bundle contract in
8
- [run-bundles.md](run-bundles.md).
9
-
10
- ## Bundle structure
11
-
12
- A session-bound run contains these files:
13
-
14
- ```text
15
- <run-id>/
16
- ├── manifest.json
17
- ├── workflow.json
18
- ├── state.json
19
- ├── trace.ndjson
20
- ├── session/
21
- │ ├── binding.json
22
- │ ├── entries.ndjson
23
- │ ├── events.ndjson
24
- │ └── capture.json
25
- └── artifacts/
26
- ```
27
-
28
- `events.ndjson` is an append-only journal. `capture.json` is an atomic status
29
- file that says whether the journal is complete.
30
-
31
- ## History ownership
32
-
33
- Each persisted file answers a different question.
34
-
35
- | File | Question | Authority |
36
- | ------------------------ | ---------------------------------------------------------- | ------------------------------------------------------- |
37
- | `trace.ndjson` | What did the workflow engine do? | Workflow transitions and outputs |
38
- | `session/entries.ndjson` | What settled in the Pi conversation? | Final Pi session entries |
39
- | `session/events.ndjson` | What happened over time while the conversation was active? | Timing for turns, messages, assistant output, and tools |
40
- | `session/capture.json` | Is the temporal history complete? | Capture status and final counts |
41
-
42
- A reader must not rebuild final conversation content from session events after
43
- a matching entry has settled. The Pi entry is the final content. Session events
44
- preserve how that content appeared and preserve partial work that never settled.
45
-
46
- The three NDJSON journals keep independent sequence spaces. A sequence number
47
- from one file cannot be compared with a sequence number from another file.
48
- Explicit ids provide cross-file linkage.
49
-
50
- ## Minimal journal
51
-
52
- A journal with one assistant text block can contain these lines:
53
-
54
- ```json
55
- {"seq":1,"at":"2026-07-30T10:00:00.010Z","nodeId":"review","attemptId":"a1","turnId":"t1","type":"turn_started","payload":{"turnIndex":0}}
56
- {"seq":2,"at":"2026-07-30T10:00:00.020Z","nodeId":"review","attemptId":"a1","turnId":"t1","messageId":"m1","type":"message_started","payload":{"role":"assistant"}}
57
- {"seq":3,"at":"2026-07-30T10:00:00.100Z","nodeId":"review","attemptId":"a1","turnId":"t1","messageId":"m1","type":"assistant_event","payload":{"type":"text_start","contentIndex":0}}
58
- {"seq":4,"at":"2026-07-30T10:00:00.140Z","nodeId":"review","attemptId":"a1","turnId":"t1","messageId":"m1","type":"assistant_event","payload":{"type":"text_delta","contentIndex":0,"delta":"Looks good."}}
59
- {"seq":5,"at":"2026-07-30T10:00:00.150Z","nodeId":"review","attemptId":"a1","turnId":"t1","messageId":"m1","type":"assistant_event","payload":{"type":"text_end","contentIndex":0,"content":"Looks good."}}
60
- {"seq":6,"at":"2026-07-30T10:00:00.170Z","nodeId":"review","attemptId":"a1","turnId":"t1","messageId":"m1","type":"message_finished","payload":{"role":"assistant","settled":true,"entryId":"e1"}}
61
- {"seq":7,"at":"2026-07-30T10:00:00.180Z","nodeId":"review","attemptId":"a1","turnId":"t1","type":"turn_finished","payload":{"turnIndex":0,"messageId":"m1","toolCallIds":[]}}
62
- ```
63
-
64
- The matching final Pi entry remains in `session/entries.ndjson`:
65
-
66
- ```json
67
- {
68
- "seq": 2,
69
- "at": "2026-07-30T10:00:00.165Z",
70
- "entry": {
71
- "type": "message",
72
- "id": "e1",
73
- "parentId": "e0",
74
- "timestamp": "2026-07-30T10:00:00.160Z",
75
- "message": { "role": "assistant", "content": [{ "type": "text", "text": "Looks good." }] }
76
- }
77
- }
78
- ```
79
-
80
- ## Event record
81
-
82
- Each line in `events.ndjson` has schema
83
- `pi-workflows.session-event.v1`:
84
-
85
- ```ts
86
- type WorkflowSessionEventRecord = {
87
- seq: number;
88
- at: string;
89
- nodeId: string;
90
- attemptId: string;
91
- turnId?: string;
92
- messageId?: string;
93
- toolCallId?: string;
94
- type:
95
- | "turn_started"
96
- | "turn_finished"
97
- | "message_started"
98
- | "assistant_event"
99
- | "message_finished"
100
- | "tool_execution_started"
101
- | "tool_execution_updated"
102
- | "tool_execution_finished";
103
- payload: Record<string, unknown>;
104
- };
105
- ```
106
-
107
- ### Fields
108
-
109
- | Field | Required | Type | Meaning |
110
- | ------------ | -------- | ---------------- | --------------------------------------------- |
111
- | `seq` | Yes | positive integer | Position in `events.ndjson`, starting at 1 |
112
- | `at` | Yes | ISO 8601 string | Time when the extension received the Pi event |
113
- | `nodeId` | Yes | string | Workflow node that owned the event |
114
- | `attemptId` | Yes | string | Workflow attempt that owned the event |
115
- | `turnId` | By event | string | Run-local turn id |
116
- | `messageId` | By event | string | Run-local message id |
117
- | `toolCallId` | By event | string | Pi tool-call id |
118
- | `type` | Yes | string | Normalized event type |
119
- | `payload` | Yes | object | Fields specific to the event type |
120
-
121
- `seq` must increase by exactly 1. Physical line order must match `seq` order.
122
- The writer must capture `at` before any asynchronous file work. Readers order
123
- events by `seq`. They use `at` only for playback timing.
124
-
125
- Every record has `nodeId` and `attemptId`. The recorder takes ownership from
126
- the active agent-step contract when the turn, message, or tool execution
127
- starts. That ownership remains fixed until the matching end event, even if the
128
- workflow moves to another node.
1
+ # Session event journal
129
2
 
130
- The ids under `turnId` and `messageId` are generated by the recorder. They are
131
- unique within one run and have no meaning outside that run. Readers must use
132
- them and `toolCallId` directly. Timing is never used to infer relationships.
133
-
134
- ## Event catalog
135
-
136
- The journal records normalized events from Pi's documented extension hooks.
137
- The payloads below are the complete version 1 contract. Unknown payload fields
138
- must be ignored.
3
+ Pi Workflows records the Pi conversation events associated with a workflow run. The journal supports live viewing, replay, and exact links from workflow steps to settled Pi entries.
139
4
 
140
- ### `turn_started`
5
+ All records live in the canonical [SQLite state](SQLITE_STATE.md) database.
141
6
 
142
- Source hook: `turn_start`.
7
+ ## Tables
143
8
 
144
- Required ids: `turnId`.
9
+ ### `session_segments`
145
10
 
146
- ```json
147
- { "turnIndex": 0 }
148
- ```
11
+ A segment is one capture owner tenure for one run. It stores:
149
12
 
150
- `turnIndex` is copied from Pi. The local `turnId` remains the journal identity.
13
+ - the run and optional node attempt
14
+ - the Pi session ID
15
+ - the content-addressed binding record
16
+ - recording, complete, or failed status
17
+ - accepted entry and event counts
18
+ - failure details when capture fails
19
+ - start and finish times
151
20
 
152
- ### `turn_finished`
21
+ A later owner writes another segment. It does not rewrite the earlier segment.
153
22
 
154
- Source hook: `turn_end`.
23
+ ### `session_entries`
155
24
 
156
- Required ids: `turnId`.
25
+ Each row stores one settled Pi session entry with a contiguous sequence, Pi entry ID, recorded time, and blob hash for the verbatim entry JSON.
157
26
 
158
- ```json
159
- { "turnIndex": 0, "messageId": "m1", "toolCallIds": ["call_1"] }
160
- ```
27
+ ### `session_events`
161
28
 
162
- The payload contains references instead of copying the final message and tool
163
- results. Final message content belongs in `entries.ndjson`, and final tool
164
- results belong to `tool_execution_finished` and the settled Pi entries.
29
+ Each row stores one normalized temporal event with a contiguous sequence and these references when applicable:
165
30
 
166
- ### `message_started`
31
+ - node and attempt ID
32
+ - turn ID
33
+ - message ID
34
+ - tool-call ID
35
+ - event payload blob
36
+ - receipt time
167
37
 
168
- Source hook: `message_start`.
38
+ ## Event types
169
39
 
170
- Required ids: `turnId`, `messageId`.
40
+ The current event types are:
171
41
 
172
- ```json
173
- { "role": "assistant" }
174
- ```
42
+ - `turn_started`
43
+ - `turn_finished`
44
+ - `message_started`
45
+ - `assistant_event`
46
+ - `message_finished`
47
+ - `tool_execution_started`
48
+ - `tool_execution_updated`
49
+ - `tool_execution_finished`
175
50
 
176
- `role` is the Pi message role observed at start.
51
+ Unknown future event types remain visible to generic readers.
177
52
 
178
- ### `assistant_event`
53
+ ## Write rules
179
54
 
180
- Source hook: `message_update`.
55
+ The recorder queues hot-path Pi events in memory and writes bounded batches. One transaction writes the complete batch, updates the segment count, increments the segment resource revision, and appends its audit event.
181
56
 
182
- Required ids: `turnId`, `messageId`.
57
+ The writer checks:
183
58
 
184
- The payload contains one normalized `AssistantMessageEvent`. The allowed shapes
185
- are:
59
+ - positive contiguous event sequence numbers
60
+ - required event envelope fields
61
+ - the 1 MiB per-event limit
62
+ - current segment state
63
+ - current owner authority when ownership is required
186
64
 
187
- ```ts
188
- type NormalizedAssistantEvent =
189
- | { type: "start" }
190
- | { type: "text_start"; contentIndex: number }
191
- | { type: "text_delta"; contentIndex: number; delta: string }
192
- | { type: "text_end"; contentIndex: number; content: string }
193
- | { type: "thinking_start"; contentIndex: number }
194
- | { type: "thinking_delta"; contentIndex: number; delta: string }
195
- | { type: "thinking_end"; contentIndex: number; content: string }
196
- | { type: "toolcall_start"; contentIndex: number }
197
- | { type: "toolcall_delta"; contentIndex: number; delta: string }
198
- | { type: "toolcall_end"; contentIndex: number; toolCall: unknown }
199
- | { type: "done"; reason: "stop" | "length" | "toolUse" }
200
- | { type: "error"; reason: "aborted" | "error" };
201
- ```
65
+ A failed batch writes a failed capture status. Failed and complete capture states are terminal. A later completion call cannot replace a failed capture.
202
66
 
203
- The recorder must discard Pi's cumulative `partial` field. It must also discard
204
- the cumulative `message` or `error` object on stream-terminal events. Persisting
205
- those snapshots once per chunk would make storage grow with every repeated
206
- prefix. Delta fields preserve the generated content without that duplication.
67
+ Workflow execution does not fail because temporal capture failed. The run and the capture report their states separately.
207
68
 
208
- The `content` on text and thinking end events and the final `toolCall` object
209
- are retained once. Readers may compare them with the folded deltas and report a
210
- mismatch. A mismatch does not replace the matching final Pi entry.
69
+ ## Settled entries
211
70
 
212
- The event writer must preserve every normalized assistant event. It may batch
213
- several records into one filesystem append, but it must not merge adjacent
214
- deltas or change their timestamps.
71
+ A `message_finished` event can refer to the Pi entry ID that settled the message. Replay first shows temporal deltas, then switches to the verbatim entry when that settled link is available.
215
72
 
216
- ### `message_finished`
73
+ Agent workflow steps also store their first and last Pi entry IDs. This makes the conversation slice for each step explicit without changing Pi session data.
217
74
 
218
- Source hook: `message_end`.
75
+ ## Read and replay rules
219
76
 
220
- Required ids: `turnId`, `messageId`.
77
+ Readers order events by `event_seq`, not by wall-clock time. The `recorded_at` value controls playback timing only.
221
78
 
222
- Settled message:
79
+ A capture is invalid when:
223
80
 
224
- ```json
225
- { "role": "assistant", "settled": true, "entryId": "e1" }
226
- ```
81
+ - sequences have a gap
82
+ - required references are missing
83
+ - row counts differ from the segment counters
84
+ - a terminal run still reports recording
85
+ - a settled event refers to an entry that does not exist
227
86
 
228
- Message without a recorded Pi entry:
87
+ The TypeScript and Rust viewers read the same committed rows. They do not repair capture state.
229
88
 
230
- ```json
231
- { "role": "assistant", "settled": false }
232
- ```
89
+ ## Privacy
233
90
 
234
- `entryId` is required when `settled` is true and forbidden when it is false.
235
- The recorder must obtain the id while it records the corresponding branch
236
- entry. Readers must not guess this link from message content or timestamps.
91
+ Session entries and events can contain prompts, generated text, thinking, tool arguments, tool results, paths, and command output. The workflow directory uses mode `0700`, and `state.sqlite` uses mode `0600`.
237
92
 
238
- ### `tool_execution_started`
93
+ Status output does not expose raw session payloads. Remote replay is loopback-only unless the operator uses an SSH tunnel.
239
94
 
240
- Source hook: `tool_execution_start`.
95
+ ## Pi API boundary
241
96
 
242
- Required ids: `turnId`, `messageId`, `toolCallId`.
97
+ The recorder uses Pi's documented `turn_start`, `turn_end`, `message_start`, `message_update`, `message_end`, `tool_execution_start`, `tool_execution_update`, and `tool_execution_end` hooks.
243
98
 
244
- ```json
245
- { "toolName": "read", "args": { "path": "README.md" } }
246
- ```
247
-
248
- The tool-call owner is fixed at this event. Later update and finish events with
249
- the same `toolCallId` keep that owner.
250
-
251
- ### `tool_execution_updated`
252
-
253
- Source hook: `tool_execution_update`.
254
-
255
- Required ids: `turnId`, `messageId`, `toolCallId`.
256
-
257
- ```json
258
- {}
259
- ```
260
-
261
- Version 1 records each update's occurrence and timestamp. Pi exposes
262
- `partialResult` as an arbitrary cumulative value. The recorder omits it to
263
- avoid repeatedly storing growing prefixes. The final result remains available
264
- on `tool_execution_finished` and in the Pi session entries.
265
-
266
- ### `tool_execution_finished`
267
-
268
- Source hook: `tool_execution_end`.
269
-
270
- Required ids: `turnId`, `messageId`, `toolCallId`.
271
-
272
- ```json
273
- { "toolName": "read", "isError": false, "result": { "content": [] } }
274
- ```
275
-
276
- `result` is the final public-hook result. Large string leaves use the bundle's
277
- existing artifact encoding. Readers must resolve those references through the
278
- same containment and size checks used for other bundle values.
279
-
280
- ## Capture status
281
-
282
- `session/capture.json` uses schema `pi-workflows.session-capture.v1`. The
283
- recorder creates it before the first session event and replaces it atomically.
284
-
285
- ```json
286
- {
287
- "schema": "pi-workflows.session-capture.v1",
288
- "eventSchema": "pi-workflows.session-event.v1",
289
- "status": "complete",
290
- "eventCount": 47,
291
- "entryCount": 6,
292
- "lastEventSeq": 47
293
- }
294
- ```
295
-
296
- | Field | Required | Type | Meaning |
297
- | -------------- | ----------- | -------------------- | -------------------------------------------- |
298
- | `schema` | Yes | string | Must equal `pi-workflows.session-capture.v1` |
299
- | `eventSchema` | Yes | string | Must equal `pi-workflows.session-event.v1` |
300
- | `status` | Yes | string | `recording`, `complete`, or `failed` |
301
- | `eventCount` | Yes | non-negative integer | Number of complete event records on disk |
302
- | `entryCount` | Yes | non-negative integer | Number of complete entry records on disk |
303
- | `lastEventSeq` | Yes | non-negative integer | Last durable event sequence, or 0 |
304
- | `failure` | Failed only | object | First capture failure |
305
-
306
- A failed status has this shape:
307
-
308
- ```json
309
- {
310
- "schema": "pi-workflows.session-capture.v1",
311
- "eventSchema": "pi-workflows.session-event.v1",
312
- "status": "failed",
313
- "eventCount": 31,
314
- "entryCount": 4,
315
- "lastEventSeq": 31,
316
- "failure": {
317
- "failedAt": "2026-07-30T10:00:01.900Z",
318
- "code": "event_write_failed",
319
- "message": "could not append session event batch"
320
- }
321
- }
322
- ```
323
-
324
- Status meanings:
325
-
326
- - `recording` means the run may append more events or entries. Its counts are
327
- lower bounds from the last atomic projection and may lag the files.
328
- - `complete` means all observed events were written, both session writers were
329
- drained, and the counts match the durable files.
330
- - `failed` means the temporal history may be incomplete. Workflow execution
331
- continues, and settled entries may still be usable.
332
-
333
- The first capture failure is retained. After an event append fails, the event
334
- writer must stop appending so a torn line remains at the file tail. The final
335
- status counts only complete lines. If `capture.json` is missing, invalid, still
336
- `recording` after a terminal workflow state, or inconsistent with the journal,
337
- the viewer must report unverified capture instead of treating the journal as
338
- empty or complete.
339
-
340
- ## Write lifecycle
341
-
342
- The recorder follows this order:
343
-
344
- 1. Bind the run to the Pi conversation.
345
- 2. Create `capture.json` with status `recording`.
346
- 3. Accept documented Pi events while an agent attempt owns the conversation.
347
- 4. Stamp correlation ids and sequence in each hook, then record the receipt time.
348
- 5. Queue normalized records in memory and return immediately from high-rate
349
- update hooks.
350
- 6. Append queued records in ordered batches on a dedicated writer.
351
- 7. At `message_end`, buffer that receipt and later events until a subsequent
352
- synchronized hook exposes the durable Pi entry, then attach its exact id
353
- without changing event order or receipt timestamps.
354
- 8. When the workflow requests terminal persistence during an active Pi turn,
355
- keep capture routed through that turn's final tool, message, and `turn_end`
356
- hooks.
357
- 9. Stop accepting events and drain the event and entry writers.
358
- 10. Write terminal `capture.json` atomically.
359
- 11. Allow the engine to append its terminal workflow event.
360
-
361
- This order keeps the bundle immutable once the terminal workflow event exists.
362
- Waiting for the active turn is bounded at 30 seconds. If the turn does not
363
- finish, capture stops as failed so it cannot hold workflow persistence
364
- indefinitely. A capture failure must never fail the workflow.
365
-
366
- The queue must have tested byte and record limits. Reaching either limit marks
367
- capture as failed and stops temporal capture. Silent event dropping is
368
- forbidden.
369
-
370
- ## Torn writes and validation
371
-
372
- Each event is one UTF-8 JSON object followed by `\n`. A reader may ignore an
373
- incomplete final line while capture is `recording`. Any malformed line before
374
- the tail is corruption.
375
-
376
- A reader validates these rules:
377
-
378
- - Required fields have the documented types.
379
- - `seq` starts at 1 and has no duplicates or gaps.
380
- - Correlation ids required by the event type are present.
381
- - Every finish event refers to a matching start event.
382
- - `message_finished.entryId` exists in `entries.ndjson` when `settled` is true.
383
- - `capture.json` counts match complete journal lines at terminal status.
384
- - No session file changes after terminal workflow state.
385
-
386
- Unknown event types and unknown payload fields are ignored. The reader still
387
- advances past their sequence numbers. An unsupported `eventSchema` makes the
388
- temporal journal unavailable. The reader must not guess another shape.
389
-
390
- ## Deterministic reduction
391
-
392
- Live display and replay must use the same reducer. The reducer processes events
393
- in `seq` order and keeps run-local state for turns, messages, content blocks,
394
- and tools.
395
-
396
- Text and thinking deltas append to their content blocks. Tool-call deltas use
397
- the same `messageId` and `contentIndex` addressing. Tool lifecycle events update the tool identified
398
- by `toolCallId`. A settled message switches to the matching verbatim Pi entry at
399
- `message_finished`. An unsealed message remains a partial reconstruction.
400
-
401
- Replay uses `at` to schedule events at 1x or another selected speed. Ordering
402
- always comes from `seq`. Seeking may use in-memory checkpoints and a timestamp
403
- index, but those are derived viewer data and do not belong in the run bundle.
404
-
405
- The TypeScript and Rust reducers must pass the same fixtures. Fixtures cover
406
- normal completion, thinking, tool calls, interleaved tool updates, aborts,
407
- missing final entries, unknown events, timestamp ties, and capture failure.
408
-
409
- ## Live transport
410
-
411
- The live replay protocol adds the journal and status to the existing run view:
412
-
413
- ```json
414
- {
415
- "session": {
416
- "binding": { "schema": "pi-workflows.session-binding.v1" },
417
- "entries": [],
418
- "events": [],
419
- "capture": { "schema": "pi-workflows.session-capture.v1" }
420
- }
421
- }
422
- ```
423
-
424
- The server tails `events.ndjson` like the existing trace and entry files.
425
- Journal growth uses an `append` patch at `/session/events`. Capture changes use
426
- a `replace` patch at `/session/capture`.
427
-
428
- The server batches pending append records into one patch for a short interval.
429
- It must preserve every event record and its sequence. This batching limits
430
- revision growth without changing the durable history. Reconnection always
431
- starts with a fresh snapshot, followed by later patches.
432
-
433
- ## Privacy and trust
434
-
435
- Session events can contain prompts, generated text, thinking, tool arguments,
436
- tool results, file paths, and command output. Bundle permissions remain `0700`
437
- for directories and `0600` for files. Remote viewing remains loopback-only and
438
- uses the existing SSH tunnel workflow.
439
-
440
- Readers must treat all strings and nested values as untrusted. Terminal output
441
- is sanitized before drawing. Artifact paths must stay inside the manifest's
442
- artifact directory.
443
-
444
- ## Versioning
445
-
446
- This change replaces the session-bound `pi-workflows.run-bundle.v1` contract in
447
- place. A session-bound bundle must contain both `events.ndjson` and
448
- `capture.json` after the implementation lands. Missing files are reported as
449
- an invalid or incomplete capture. There is no legacy read path.
450
-
451
- The new files use `pi-workflows.session-event.v1` and
452
- `pi-workflows.session-capture.v1`. Breaking either shape requires changing its
453
- schema identifier. New event types and optional payload fields may be added
454
- within version 1 because readers ignore unknown values.
455
-
456
- The network view grows within `pi-workflows.replay.v1`. Its existing unknown
457
- field rule makes `session.events` and `session.capture` additive protocol
458
- fields. No dual writer, migration, or fallback reader is part of this change.
459
-
460
- ## Contract impact
461
-
462
- The extension uses only Pi's documented `turn_start`, `turn_end`,
463
- `message_start`, `message_update`, `message_end`, `tool_execution_start`,
464
- `tool_execution_update`, and `tool_execution_end` hooks.
465
-
466
- Pi session state is unchanged. Normal Pi behavior still creates its own session
467
- entries, and the extension only reads those entries for bundling and linkage.
468
- Pi internals, provider protocols, and global Pi session files are unchanged.
469
- The only new persistent data is `session/events.ndjson` and
470
- `session/capture.json` inside the private workflow run bundle.
99
+ It does not change Pi session files, session entry schemas, or Pi internals.
@@ -16,18 +16,16 @@ cargo install pi-workflows
16
16
 
17
17
  ## Modes
18
18
 
19
- - `piw` browses the local runs directory (`PI_WORKFLOWS_RUNS_DIR` or
20
- `~/.pi/agent/workflows/runs`).
21
- - `piw <runId|runDir>` opens one run. A bare run id resolves inside the default
22
- runs directory.
23
- - `piw serve [--runs-dir <dir>] [--bind 127.0.0.1:9377]` exposes the runs
24
- directory over the [live replay protocol](live-replay-protocol.md). Only
19
+ - `piw` browses `~/.pi/agent/workflows/state.sqlite` through a read-only SQLite connection.
20
+ - `piw <runId>` opens one run from that database.
21
+ - `piw serve [--bind 127.0.0.1:9377]` exposes database-backed run views over the
22
+ [live replay protocol](live-replay-protocol.md). Only
25
23
  loopback addresses are accepted; use an SSH tunnel for remote viewing.
26
24
  - `piw --connect ws://…` reads from another `piw serve` process.
27
25
  - `piw --theme <name>` selects a theme for this invocation.
28
26
  - `piw --list-themes` prints the built-in theme names.
29
27
 
30
- Direct filesystem mode and connected mode use the same semantic run view.
28
+ Direct database mode and connected mode use the same semantic run view.
31
29
  The protocol is the network form of that view.
32
30
 
33
31
  ## Herdr
@@ -36,7 +34,7 @@ The npm package contains a native Herdr plugin. Link the installed package with
36
34
  `pi-workflows herdr setup`. A workflow running in Pi inside Herdr then shows
37
35
  `Ctrl+Shift+R piw` in its widget. When rows are hidden, the call to action shares
38
36
  the scroll-controls line. The shortcut and `/piw` command open
39
- the current bundle directly in a managed Herdr pane. The placement menu supports
37
+ the current run directly in a managed Herdr pane. The placement menu supports
40
38
  right, below, left, above, a new tab, and a new workspace.
41
39
 
42
40
  The integration resolves the calling pane at invocation time and uses returned
@@ -54,7 +52,7 @@ border to resize the bottom panel. PIW saves both sizes in its viewer config and
54
52
  clamps them when the terminal is smaller. A directly opened single run hides
55
53
  the browser.
56
54
 
57
- - **Run browser:** every bundle, newest first, with status, title, elapsed time,
55
+ - **Run browser:** every run, newest first, with status, title, elapsed time,
58
56
  and a `?` marker for a possibly interrupted run.
59
57
  - **Graph:** the complete workflow definition using the same layered layout as
60
58
  the TypeScript renderer. Full bordered cards are the default. Every card in
@@ -184,10 +182,10 @@ including nodes at the graph edges and graphs smaller than the viewport. `f`
184
182
  toggles follow. Keyboard or mouse panning turns it off; `f`, `0`, or returning
185
183
  to the latest position turns it back on.
186
184
 
187
- The conversation folds `session/events.ndjson` through the temporal cursor.
185
+ The conversation folds `session_events` rows through the temporal cursor.
188
186
  Unsealed messages stay visible as partial output. A settled `message_finished`
189
- with `entryId` switches that message to the matching verbatim record from
190
- `session/entries.ndjson`. Capture failures, sequence gaps, count mismatches,
187
+ with `entryId` switches that message to the matching verbatim `session_entries`
188
+ row. Capture failures, sequence gaps, count mismatches,
191
189
  and reconciliation diagnostics remain visible. Conversation auto-follow stays
192
190
  at the bottom until the user moves to an older message and returns with End.
193
191
 
@@ -198,9 +196,9 @@ listing and selected-run subscription after the server returns. Cached content
198
196
  stays visible but is labeled reconnecting or disconnected, never current.
199
197
  Revision gaps still force a fresh snapshot.
200
198
 
201
- Expanded remote prompt and output fields fetch artifact content on demand.
202
- The client caches bounded responses while the server enforces bundle path,
203
- symlink, and 4 MiB limits.
199
+ Expanded prompt and output fields come from content-addressed SQLite blobs.
200
+ The local reader uses query-only mode. Remote snapshots carry the same bounded
201
+ semantic view and do not expose a filesystem path.
204
202
 
205
203
  ## Interaction
206
204