@osolmaz/pi-workflows 0.13.2 → 0.13.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (134) hide show
  1. package/README.md +5 -3
  2. package/dist/builtins/autodoc.workflow.d.ts +5 -5
  3. package/dist/builtins/autoimplement.workflow.d.ts +77 -73
  4. package/dist/builtins/autoimplement.workflow.js +47 -8
  5. package/dist/builtins/autoimplement.workflow.js.map +1 -1
  6. package/dist/builtins/autoplan.workflow.d.ts +8 -8
  7. package/dist/builtins/autoplan.workflow.js +44 -35
  8. package/dist/builtins/autoplan.workflow.js.map +1 -1
  9. package/dist/builtins/catalog.js +2 -2
  10. package/dist/builtins/change-verification.workflow.d.ts +1 -1
  11. package/dist/builtins/plain-summary.workflow.d.ts +1 -1
  12. package/dist/builtins/plan-approval.workflow.d.ts +1 -1
  13. package/dist/builtins/plan-change.workflow.d.ts +29 -29
  14. package/dist/builtins/sanity-check.workflow.d.ts +3 -3
  15. package/dist/builtins/workspace-preparation.workflow.d.ts +1 -1
  16. package/dist/controllers/index.d.ts +2 -2
  17. package/dist/controllers/index.js.map +1 -1
  18. package/dist/controllers/sqlite.js +85 -23
  19. package/dist/controllers/sqlite.js.map +1 -1
  20. package/dist/controllers/types.d.ts +34 -0
  21. package/dist/controllers/workflow-engine-scheduler.d.ts +5 -1
  22. package/dist/controllers/workflow-engine-scheduler.js +83 -1
  23. package/dist/controllers/workflow-engine-scheduler.js.map +1 -1
  24. package/dist/controllers/workflows.d.ts +8 -1
  25. package/dist/controllers/workflows.js +36 -0
  26. package/dist/controllers/workflows.js.map +1 -1
  27. package/dist/extension/controller-host.d.ts +2 -1
  28. package/dist/extension/controller-host.js +1 -1
  29. package/dist/extension/controller-host.js.map +1 -1
  30. package/dist/extension/follow-up-coordinator.d.ts +27 -0
  31. package/dist/extension/follow-up-coordinator.js +131 -0
  32. package/dist/extension/follow-up-coordinator.js.map +1 -0
  33. package/dist/extension/index.d.ts +9 -0
  34. package/dist/extension/index.js +460 -75
  35. package/dist/extension/index.js.map +1 -1
  36. package/dist/extension/session-events.d.ts +2 -3
  37. package/dist/extension/session-events.js +11 -10
  38. package/dist/extension/session-events.js.map +1 -1
  39. package/dist/extension/widget.js +9 -0
  40. package/dist/extension/widget.js.map +1 -1
  41. package/dist/state/database.d.ts +2 -1
  42. package/dist/state/database.js +11 -13
  43. package/dist/state/database.js.map +1 -1
  44. package/dist/state/json.js +6 -1
  45. package/dist/state/json.js.map +1 -1
  46. package/dist/state/mutation.d.ts +1 -1
  47. package/dist/state/mutation.js.map +1 -1
  48. package/dist/state/prune.d.ts +18 -0
  49. package/dist/state/prune.js +373 -0
  50. package/dist/state/prune.js.map +1 -0
  51. package/dist/state/schema.d.ts +1 -1
  52. package/dist/state/schema.js +116 -18
  53. package/dist/state/schema.js.map +1 -1
  54. package/dist/viewer/cli.d.ts +3 -1
  55. package/dist/viewer/cli.js +55 -4
  56. package/dist/viewer/cli.js.map +1 -1
  57. package/dist/viewer/render.js +19 -1
  58. package/dist/viewer/render.js.map +1 -1
  59. package/dist/workflows/composition.d.ts +2 -0
  60. package/dist/workflows/composition.js +15 -0
  61. package/dist/workflows/composition.js.map +1 -1
  62. package/dist/workflows/definition.d.ts +6 -3
  63. package/dist/workflows/definition.js +3 -0
  64. package/dist/workflows/definition.js.map +1 -1
  65. package/dist/workflows/engine.d.ts +6 -1
  66. package/dist/workflows/engine.js +301 -35
  67. package/dist/workflows/engine.js.map +1 -1
  68. package/dist/workflows/human-decision.d.ts +1 -0
  69. package/dist/workflows/human-decision.js +25 -9
  70. package/dist/workflows/human-decision.js.map +1 -1
  71. package/dist/workflows/index.d.ts +3 -1
  72. package/dist/workflows/index.js +2 -0
  73. package/dist/workflows/index.js.map +1 -1
  74. package/dist/workflows/json-patch.d.ts +47 -0
  75. package/dist/workflows/json-patch.js +298 -0
  76. package/dist/workflows/json-patch.js.map +1 -0
  77. package/dist/workflows/schema.js +29 -1
  78. package/dist/workflows/schema.js.map +1 -1
  79. package/dist/workflows/settings.d.ts +128 -0
  80. package/dist/workflows/settings.js +199 -0
  81. package/dist/workflows/settings.js.map +1 -0
  82. package/dist/workflows/store.d.ts +75 -2
  83. package/dist/workflows/store.js +1349 -89
  84. package/dist/workflows/store.js.map +1 -1
  85. package/dist/workflows/tool-input.d.ts +22 -0
  86. package/dist/workflows/tool-input.js +43 -0
  87. package/dist/workflows/tool-input.js.map +1 -1
  88. package/dist/workflows/types.d.ts +40 -5
  89. package/docs/2026-08-25-workflow-follow-ups.md +132 -0
  90. package/docs/2026-08-25-workflow-settings.md +169 -0
  91. package/docs/DEFERRED_TURNS.md +6 -0
  92. package/docs/DESIGN_PHILOSOPHY.md +2 -0
  93. package/docs/SQLITE_STATE.md +31 -18
  94. package/docs/WORKFLOW_COMPOSITION.md +16 -1
  95. package/docs/WORKFLOW_STEP_MESSAGES.md +4 -2
  96. package/docs/plans/2026-08-25-live-workflow-settings-plan.md +532 -0
  97. package/docs/plans/2026-08-25-workflow-run-storage-plan.md +67 -0
  98. package/docs/tui-viewer.md +7 -6
  99. package/docs/workflows.md +22 -1
  100. package/examples/workflows/live-settings.workflow.ts +93 -0
  101. package/herdr-plugin.toml +1 -1
  102. package/package.json +1 -1
  103. package/skills/autoplan/SKILL.md +6 -6
  104. package/src/builtins/autoimplement.workflow.ts +56 -8
  105. package/src/builtins/autoplan.workflow.ts +45 -37
  106. package/src/builtins/catalog.ts +2 -2
  107. package/src/controllers/index.ts +6 -0
  108. package/src/controllers/sqlite.ts +129 -35
  109. package/src/controllers/types.ts +40 -0
  110. package/src/controllers/workflow-engine-scheduler.ts +103 -1
  111. package/src/controllers/workflows.ts +68 -0
  112. package/src/extension/controller-host.ts +6 -1
  113. package/src/extension/follow-up-coordinator.ts +151 -0
  114. package/src/extension/index.ts +538 -88
  115. package/src/extension/session-events.ts +15 -13
  116. package/src/extension/widget.ts +8 -0
  117. package/src/state/database.ts +12 -12
  118. package/src/state/json.ts +6 -1
  119. package/src/state/mutation.ts +4 -1
  120. package/src/state/prune.ts +502 -0
  121. package/src/state/schema.ts +116 -18
  122. package/src/viewer/cli.ts +52 -5
  123. package/src/viewer/render.ts +43 -1
  124. package/src/workflows/composition.ts +19 -0
  125. package/src/workflows/definition.ts +19 -5
  126. package/src/workflows/engine.ts +363 -35
  127. package/src/workflows/human-decision.ts +41 -11
  128. package/src/workflows/index.ts +43 -0
  129. package/src/workflows/json-patch.ts +375 -0
  130. package/src/workflows/schema.ts +31 -1
  131. package/src/workflows/settings.ts +430 -0
  132. package/src/workflows/store.ts +1977 -184
  133. package/src/workflows/tool-input.ts +58 -0
  134. package/src/workflows/types.ts +43 -4
@@ -0,0 +1,532 @@
1
+ ---
2
+ title: Change workflow settings during a run
3
+ author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
4
+ date: 2026-08-25
5
+ ---
6
+
7
+ # Change workflow settings during a run
8
+
9
+ A workflow run must be able to accept setting changes while it is active. The changes can affect future instructions, values, route choices, merge behavior, and other behavior that the workflow exposes. A workflow can also save follow-up prompts and send them as normal user messages after the run is officially complete.
10
+
11
+ Use one JSON settings value for each workflow scope. Apply setting changes with [JSON Patch](https://www.rfc-editor.org/rfc/rfc6902). Save every accepted patch in order. Keep follow-up prompts in a separate list because they are actions to perform later, not workflow settings.
12
+
13
+ This replaces the earlier idea of defining a custom command type for every possible setting change. JSON Patch gives all workflows one standard change format.
14
+
15
+ ## Goals
16
+
17
+ - Let a workflow declare the settings that users and other authorized actors can change.
18
+ - Let changes affect only future steps and future route choices.
19
+ - Keep running actions and completed work unchanged.
20
+ - Save changes in a clear order so restart and replay produce the same result.
21
+ - Keep the source of each change outside the patch data.
22
+ - Prevent a model from granting itself protected authority.
23
+ - Save several follow-up prompts and send them in order after completion.
24
+ - Keep the completed workflow terminal while follow-up prompts start normal agent turns.
25
+ - Keep the one-active-workflow-per-session rule.
26
+ - Use the existing Pi Workflows database and documented Pi extension APIs.
27
+
28
+ ## Non-goals
29
+
30
+ - Do not change workflow source, graph structure, immutable run input, completed outputs, or completed effects.
31
+ - Do not interrupt or restart a running action when settings change.
32
+ - Do not infer setting changes by parsing session history in the extension.
33
+ - Do not use progress updates or ordinary node outputs as another settings store.
34
+ - Do not put pause, resume, cancel, or follow-up delivery state inside the JSON settings value.
35
+ - Do not change Pi core or use private Pi APIs.
36
+ - Do not add an external queue, service, daemon, or database.
37
+ - Do not promise delivery after permanent loss of the process, database, machine, session, or provider.
38
+
39
+ ## Public authoring API
40
+
41
+ Add an optional `settings` declaration to `defineWorkflow`.
42
+
43
+ ```ts
44
+ import {
45
+ allowSettingsPath,
46
+ defineWorkflow,
47
+ settingsRoute,
48
+ workflowSettings,
49
+ } from "@osolmaz/pi-workflows";
50
+
51
+ type Settings = {
52
+ instructions: string[];
53
+ merge: "allow" | "forbid" | "ask";
54
+ variables: Record<string, unknown>;
55
+ };
56
+
57
+ export default defineWorkflow({
58
+ name: "example",
59
+
60
+ settings: workflowSettings<Settings>({
61
+ initial: {
62
+ instructions: [],
63
+ merge: "ask",
64
+ variables: {},
65
+ },
66
+
67
+ parse: parseSettings,
68
+
69
+ paths: [
70
+ allowSettingsPath("/instructions", {
71
+ read: ["session", "human"],
72
+ add: ["session", "human"],
73
+ remove: ["session", "human"],
74
+ replace: ["session", "human"],
75
+ }),
76
+ allowSettingsPath("/merge", {
77
+ read: ["session", "human"],
78
+ replace: ["human"],
79
+ }),
80
+ allowSettingsPath("/variables", {
81
+ read: ["session", "human"],
82
+ add: ["session", "human"],
83
+ remove: ["session", "human"],
84
+ replace: ["session", "human"],
85
+ }),
86
+ ],
87
+
88
+ validateChange: ({ before, after, actor }) => {
89
+ if (actor.type === "session" && before.merge !== "allow" && after.merge === "allow") {
90
+ throw new Error("A model cannot grant merge authority.");
91
+ }
92
+ },
93
+ }),
94
+
95
+ // nodes, routes, and exits
96
+ });
97
+ ```
98
+
99
+ The parser checks the complete settings value after every patch. `validateChange` handles rules that depend on both the old and new values.
100
+
101
+ A workflow with no `settings` declaration has no editable settings. A request to change its settings fails clearly.
102
+
103
+ ### Path rules
104
+
105
+ Path rules use JSON Pointer paths from [RFC 6901](https://www.rfc-editor.org/rfc/rfc6901). They deny changes by default.
106
+
107
+ A rule applies to its path and descendants on pointer-segment boundaries. The most specific matching rule wins. The workflow definition is invalid when two equally specific rules disagree.
108
+
109
+ Each JSON Patch operation needs these permissions:
110
+
111
+ | Operation | Required permission |
112
+ | --------- | --------------------------------------------------------------- |
113
+ | `test` | Read the path |
114
+ | `add` | Add at the destination |
115
+ | `remove` | Remove the source |
116
+ | `replace` | Replace the path |
117
+ | `copy` | Read the source and add or replace the destination |
118
+ | `move` | Read and remove the source, then add or replace the destination |
119
+
120
+ The engine checks the source and destination of `copy` and `move` separately. The `read` permission controls only JSON Patch `test`, `copy`, and `move` source access. It does not hide fields from workflow code. Every node in a settings scope receives the complete typed settings value.
121
+
122
+ ### Node context
123
+
124
+ Every node attempt receives one fixed settings value:
125
+
126
+ ```ts
127
+ type WorkflowNodeContext<TInput, TSettings> = {
128
+ input: TInput;
129
+ outputs: Record<string, unknown>;
130
+ results: Record<string, WorkflowNodeResult>;
131
+ state: WorkflowRunState;
132
+ settings: TSettings;
133
+ settingsScopeId?: string;
134
+ settingsChangeNumber?: number;
135
+ signal: AbortSignal;
136
+ };
137
+ ```
138
+
139
+ The engine deep-copies and freezes the settings before the node starts. A change accepted while the node runs does not alter that attempt. The next attempt or next step reads the latest settings.
140
+
141
+ The change number belongs to one settings scope in one run. It is not a workflow definition revision, package version, or Git revision. Use the existing SQLite resource revision for this number instead of adding another version system.
142
+
143
+ ## Changing settings
144
+
145
+ Add a `change-settings` action to the workflow tool.
146
+
147
+ ```json
148
+ {
149
+ "action": "change-settings",
150
+ "scopeId": "optional-scope-id",
151
+ "expectedChangeNumber": 12,
152
+ "patch": [
153
+ {
154
+ "op": "replace",
155
+ "path": "/merge",
156
+ "value": "forbid"
157
+ },
158
+ {
159
+ "op": "add",
160
+ "path": "/instructions/-",
161
+ "value": "Run the release checks before finishing."
162
+ }
163
+ ]
164
+ }
165
+ ```
166
+
167
+ The extension supplies the source and the stable request ID. The model cannot provide or replace those fields.
168
+
169
+ The engine performs one database transaction:
170
+
171
+ 1. Return the earlier result when the same request ID and same content were already accepted.
172
+ 2. Reject the request when the same ID has different content.
173
+ 3. Confirm that the run is running, paused, parked, or waiting at a checkpoint, and that the settings scope exists.
174
+ 4. Compare `expectedChangeNumber` when the caller supplied it.
175
+ 5. Parse and authorize every JSON Pointer used by the patch.
176
+ 6. Apply the patch to a copy of the current settings.
177
+ 7. Parse the complete result and run `validateChange`.
178
+ 8. Save the exact patch, old settings hash, new settings hash, source, time, and new change number.
179
+ 9. Update the current settings value and emit one run event.
180
+
181
+ When `expectedChangeNumber` is absent, the patch applies to the latest settings in database acceptance order. A JSON Patch `test` operation can protect one field without rejecting unrelated changes.
182
+
183
+ ### Limits
184
+
185
+ Use fixed safety limits:
186
+
187
+ - at most 256 operations in one patch;
188
+ - at most 64 KiB for one patch;
189
+ - at most 1 KiB for one JSON Pointer;
190
+ - at most 256 KiB for the resulting settings value.
191
+
192
+ Reject non-JSON values, unknown operation fields, invalid array indexes, invalid pointer escaping, unsafe object access, and invalid moves. A rejected patch must not change the saved settings.
193
+
194
+ ## Source and authority
195
+
196
+ Use the existing saved actor types:
197
+
198
+ - `session` for a model workflow-tool call in a Pi session;
199
+ - `human` for a direct interactive command or accepted human decision;
200
+ - `controller` for a controller request;
201
+ - `policy` only when existing saved policy code makes the change.
202
+
203
+ Save a separate source label, such as `workflow-tool`, `interactive-command`, or `controller-request`, when it helps inspection. The host supplies both fields. They never come from JSON Patch data.
204
+
205
+ A normal user message that asks the model to change settings still produces a `session` change through a model tool call. The workflow can allow safe session changes, such as forbidding merge, while requiring a verified `human` action to grant authority, spend money, use credentials, or widen repository scope.
206
+
207
+ Add a direct command for protected changes:
208
+
209
+ ```text
210
+ /workflow change-settings <json-patch>
211
+ ```
212
+
213
+ Add these exact controller methods:
214
+
215
+ ```ts
216
+ ctx.workflows.changeSettings({ requestKey, runId, scopeId, expectedChangeNumber, patch });
217
+ ctx.workflows.queueFollowUp({ requestKey, runId, prompt });
218
+ ctx.workflows.removeFollowUp({ requestKey, runId, followUpId });
219
+ ```
220
+
221
+ The model tool, direct command, and controller methods must call the same database operations. Do not add a new policy API until a real policy caller needs it.
222
+
223
+ ## Route changes
224
+
225
+ Add `settingsRoute()` for route choices that must observe the latest settings.
226
+
227
+ `settingsRoute()` is a pure compute step. It records the settings change number it used. Before the engine saves the selected route, it compares that number with the current settings scope.
228
+
229
+ If the number changed, the engine discards the unsaved result and runs the pure route step again. It never repeats an agent step, shell command, function action, or other external effect because settings changed.
230
+
231
+ Workflow authors place `settingsRoute()` after effectful work when later settings may change the next path.
232
+
233
+ ## Included workflows
234
+
235
+ The root workflow gets one settings scope when it declares settings. Each included workflow invocation that declares settings gets a new scope.
236
+
237
+ Each scope uses one row in `resources` with resource type `settings`. Its public scope ID is derived from the run ID, mount path, and saved invocation number. A repeated or re-entered included workflow gets a new invocation number and fresh settings. It does not reuse values from an earlier invocation.
238
+
239
+ Settings do not flow between parent and child automatically. A parent can initialize child settings through an explicit typed mapping when the child starts. A change targets the active local scope by default and can target another initialized scope by ID.
240
+
241
+ ## Saved data
242
+
243
+ Add these tables to the existing SQLite database:
244
+
245
+ - `workflow_settings` stores each root or included-workflow settings scope and its current value.
246
+ - `workflow_setting_changes` stores each accepted JSON Patch in order.
247
+ - `workflow_follow_ups` stores prompts that must run after completion.
248
+
249
+ Use one `resources` row of type `settings` for each settings scope. Its resource revision is the public change number. Use the existing content-addressed value store for settings, patches, prompts, and source details. Use existing leases for delivery claims.
250
+
251
+ Store these fields for each setting change:
252
+
253
+ - settings scope;
254
+ - stable request ID;
255
+ - change number;
256
+ - source type and source details;
257
+ - exact JSON Patch;
258
+ - old settings hash;
259
+ - new settings hash;
260
+ - acceptance time.
261
+
262
+ Add the settings scope, change number, and settings hash to each node attempt.
263
+
264
+ Do not store setting changes in `workflow_updates`. Updates keep their current progress and status meaning.
265
+
266
+ ## Follow-up prompts
267
+
268
+ Follow-up prompts use separate actions because they do not change workflow settings.
269
+
270
+ ```json
271
+ {
272
+ "action": "queue-follow-up",
273
+ "prompt": "Release version 0.5.0 and verify every published package."
274
+ }
275
+ ```
276
+
277
+ ```json
278
+ {
279
+ "action": "remove-follow-up",
280
+ "followUpId": "follow-up-id"
281
+ }
282
+ ```
283
+
284
+ Each accepted prompt gets a stable ID and order number. Several prompts are kept in acceptance order. Reusing the same request ID with the same content returns the earlier result. Reusing it with different content fails.
285
+
286
+ Use these states:
287
+
288
+ - `queued`: the workflow is running, paused, parked, or waiting at a checkpoint;
289
+ - `pending-presentation`: the workflow completed successfully and its final response has not settled;
290
+ - `ready`: the prompt can be sent;
291
+ - `sent`: the normal user message exists in the session;
292
+ - `removed`: an allowed actor removed it before delivery;
293
+ - `cancelled`: workflow failure, timeout, or user cancellation prevented delivery.
294
+
295
+ A delivery lease is temporary coordination state. It is not another follow-up state.
296
+
297
+ ### Run-state rules
298
+
299
+ | Run state | Setting changes | New follow-ups | Existing follow-ups |
300
+ | ----------------------------------------- | --------------- | -------------- | -------------------------------- |
301
+ | Running or paused | Accept | Accept | Keep queued |
302
+ | Parked queue entry with a nonterminal run | Accept | Accept | Keep queued |
303
+ | Waiting at a checkpoint | Accept | Accept | Keep queued |
304
+ | Continued from a checkpoint | Accept | Accept | Carry forward in order |
305
+ | Completed | Reject | Reject | Wait for presentation, then send |
306
+ | Failed, timed out, or cancelled | Reject | Reject | Cancel unsent items |
307
+
308
+ The checkpoint continuation is a new stored run, but it is the same logical workflow chain. The transaction that creates the continuation transfers the active settings scopes and queued follow-ups to the new run. Scope IDs, change numbers, saved changes, and current values stay unchanged. A crash cannot leave the new run without its settings or split its follow-up list.
309
+
310
+ - Pause and park keep queued prompts.
311
+ - Resume keeps their order.
312
+ - Successful completion moves them to `pending-presentation` in the same transaction that records the terminal run state.
313
+ - A run with no final presentation records that fact and makes them ready after terminal state.
314
+ - A final presentation must settle before the first prompt becomes ready.
315
+ - Workflow failure, timeout, and user cancellation cancel every unsent prompt with a saved reason.
316
+ - A terminal workflow accepts no new prompt, but a repeated request ID can still return its earlier result.
317
+ - A prompt can be removed only before it is sent. A `session` actor can remove only a prompt added by the same session tool source. A verified `human` actor can remove any unsent prompt for that run. Controllers can remove only prompts they added with the same controller resource.
318
+
319
+ ### Presentation and delivery
320
+
321
+ Save one durable presentation state for a completed run: `not-needed`, `pending`, `settled`, or `unavailable`. When the extension sends the hidden presentation prompt, include the run ID in its documented custom-message details. On `agent_settled`, save `settled` only after the active session branch contains that presentation message and its completed assistant child. On restart, use the same branch evidence to recover a missed database update. A timeout or a definite presentation failure saves `unavailable`; it does not claim that a presentation succeeded.
322
+
323
+ Add a follow-up coordinator to the Pi extension. It is separate from the existing interruption-turn coordinator because the two features solve different problems.
324
+
325
+ The coordinator sends one prompt at a time:
326
+
327
+ 1. Wait until the run is terminal and the saved presentation state is `not-needed`, `settled`, or `unavailable`.
328
+ 2. Wait until the target session is available, no workflow is active, and the prior agent turn has settled.
329
+ 3. Claim the first ready prompt with a lease.
330
+ 4. Search the current durable session branch for that follow-up ID.
331
+ 5. If the message already exists, save its session entry ID and sent time without sending it again.
332
+ 6. Otherwise, put the stable, nonsecret follow-up ID in the message text and call documented `pi.sendUserMessage`.
333
+ 7. Scan the durable session branch for the new user message, then save its session entry ID and sent time.
334
+ 8. Wait for that user turn to settle before considering the next prompt.
335
+
336
+ `pi.sendUserMessage` returns no message ID. Branch evidence is therefore required after both a new send and restart recovery. Do not expose session IDs or source details in the visible text.
337
+
338
+ If one follow-up starts another workflow, later prompts wait until that workflow ends. The completed original workflow remains terminal and never enters its start node again.
339
+
340
+ ## Restart and failure behavior
341
+
342
+ Setting changes are complete when their database transaction commits. Restart loads the current settings and verifies that the saved changes rebuild the same value.
343
+
344
+ Follow-up delivery handles these failures:
345
+
346
+ - A crash before the claim leaves the prompt ready.
347
+ - A crash after the claim lets another process continue after the lease expires.
348
+ - A crash before message append leaves no session message and permits a retry.
349
+ - A crash after message append but before the database update is resolved by finding the follow-up ID in the active session branch.
350
+ - A definite send failure releases the claim.
351
+ - An unavailable session leaves the prompt pending.
352
+
353
+ The current Pi API cannot guarantee that the model starts or finishes the follow-up turn exactly once. The implementation guarantees only the saved state, claim, normal-message evidence, and local delivery status that it can observe.
354
+
355
+ ## User and model surfaces
356
+
357
+ Extend workflow status, CLI output, the TUI, and the workflow card to show:
358
+
359
+ - current settings scope and change number;
360
+ - a bounded settings summary;
361
+ - accepted setting changes and their source type;
362
+ - the settings number used by each node and route;
363
+ - queued follow-up order and state;
364
+ - claim, sent message, removal, and cancellation status.
365
+
366
+ Keep large values in the value store. Show short summaries and hashes in normal views. Hide raw source IDs, session IDs, prompts, and other private content unless an explicit detailed view requires them.
367
+
368
+ Workflow step messages must show the active scope, current change number, a bounded settings summary, allowed paths, and exact tool calls. This tells the model how to handle later user messages without parsing workflow source.
369
+
370
+ ## Implementation steps
371
+
372
+ ### 1. Document and test JSON Patch
373
+
374
+ Add `src/workflows/json-patch.ts` and `test/json-patch.test.ts`. Implement all RFC 6902 operations over canonical JSON values. Add the limits and unsafe-object protections listed above.
375
+
376
+ ### 2. Add workflow settings types
377
+
378
+ Update:
379
+
380
+ - `src/workflows/types.ts`
381
+ - `src/workflows/definition.ts`
382
+ - `src/workflows/schema.ts`
383
+ - `src/workflows/catalog.ts`
384
+ - `src/workflows/store.ts`
385
+ - `src/workflows/index.ts`
386
+
387
+ Add `src/workflows/settings.ts`. Provide `workflowSettings()`, `allowSettingsPath()`, `settingsRoute()`, and its typed exhaustive edge helper.
388
+
389
+ ### 3. Add settings scopes for included workflows
390
+
391
+ Update `src/workflows/composition.ts`, workflow snapshots, and composition tests. Save a new scope for each included-workflow invocation and a new scope on every re-entry.
392
+
393
+ ### 4. Save settings and changes
394
+
395
+ Update:
396
+
397
+ - `src/state/schema.ts`
398
+ - `src/state/database.ts`
399
+ - `src/state/mutation.ts`
400
+ - `src/state/json.ts`
401
+ - `src/workflows/store.ts`
402
+
403
+ Add the three tables described above. Add atomic change, lookup, replay, and inspection methods.
404
+
405
+ ### 5. Bind settings to node attempts
406
+
407
+ Update `src/workflows/engine.ts`, `src/host/runner.ts`, and `src/host/rpc-executor.ts`. Save and expose one fixed settings value for every attempt.
408
+
409
+ ### 6. Add safe route changes
410
+
411
+ Update `src/workflows/graph.ts`, `src/workflows/engine.ts`, snapshots, and route tests. Retry only a pure `settingsRoute()` result when its change number becomes stale before the route is saved.
412
+
413
+ ### 7. Add tool and operator actions
414
+
415
+ Update:
416
+
417
+ - `src/workflows/tool-input.ts`
418
+ - `src/extension/workflow-tool.ts`
419
+ - `src/extension/index.ts`
420
+ - `src/extension/step-message.ts`
421
+ - `src/controllers/workflows.ts`
422
+ - `src/controllers/types.ts`
423
+
424
+ Add `change-settings`, `queue-follow-up`, and `remove-follow-up`. Keep source data outside model input. Use provider-compatible string action schemas.
425
+
426
+ ### 8. Add completion and delivery behavior
427
+
428
+ Update terminal handling in `src/workflows/engine.ts` and `src/workflows/store.ts`. Add `src/extension/follow-up-coordinator.ts` and connect it through `src/extension/index.ts`, `src/extension/session-events.ts`, and `src/extension/recorder.ts`.
429
+
430
+ Reuse lease and session-branch checks from `src/extension/deferred-turn-coordinator.ts`. Do not reuse interruption-turn rows or states.
431
+
432
+ ### 9. Add inspection views
433
+
434
+ Update:
435
+
436
+ - `src/viewer/cli.ts`
437
+ - `src/viewer/render.ts`
438
+ - `src/viewer/session-reducer.ts`
439
+ - `src/viewer/tui.ts`
440
+ - `src/render/graph-render.ts`
441
+ - `src/extension/widget.ts`
442
+ - `tui/src/state/reader.rs`
443
+ - `tui/src/state/types.rs`
444
+ - the affected Rust view and test files under `tui/`
445
+
446
+ Add bounded, private-by-default views for settings changes and follow-up prompts. Update the Rust schema check and reader in the same hard change so `piw` can open the new database.
447
+
448
+ ### 10. Add one example and one built-in use
449
+
450
+ Add `examples/workflows/live-settings.workflow.ts`. It must change future instructions, one variable, one route, and queue two follow-up prompts.
451
+
452
+ Update Autoimplement to expose future merge behavior and added instructions through the general settings API. A model can forbid merge when that reduces authority. It cannot grant missing merge authority. Keep release work out of Autoimplement-specific logic.
453
+
454
+ ### 11. Update documentation
455
+
456
+ Update:
457
+
458
+ - `docs/workflows.md`
459
+ - `docs/WORKFLOW_COMPOSITION.md`
460
+ - `docs/WORKFLOW_STEP_MESSAGES.md`
461
+ - `docs/SQLITE_STATE.md`
462
+ - `docs/DEFERRED_TURNS.md`
463
+ - `docs/DESIGN_PHILOSOPHY.md`
464
+ - `README.md`
465
+
466
+ Add `docs/2026-08-25-workflow-settings.md` and `docs/2026-08-25-workflow-follow-ups.md`. Keep this file as the dated implementation plan.
467
+
468
+ ## Tests
469
+
470
+ Add focused tests for:
471
+
472
+ - every RFC 6901 and RFC 6902 rule;
473
+ - malformed, oversized, unsafe, and unauthorized patches;
474
+ - complete settings parsing and cross-field rules;
475
+ - duplicate request IDs and conflicting reuse;
476
+ - expected change numbers and JSON Patch `test` operations;
477
+ - concurrent changes, cancellation, terminal state, and database rollback;
478
+ - fixed settings for agent, shell, function, compute, checkpoint, retry, pause, park, resume, timeout, and cancellation;
479
+ - route changes before selection, during selection, before save, and after save;
480
+ - root, nested, repeated, and re-entered workflow scopes;
481
+ - session, human, and controller sources, plus payload attempts to spoof actor data;
482
+ - changes and follow-ups while running, paused, parked, and waiting;
483
+ - checkpoint continuation with settings and follow-ups carried forward atomically;
484
+ - invalid initial settings, equally specific conflicting path rules, and a workflow with no settings;
485
+ - two concurrent setting writers on one scope and both directions of split `copy` and `move` permission;
486
+ - exact size and operation-count boundaries;
487
+ - step messages with scope ID, change number, allowed paths, and exact action shape;
488
+ - several queued prompts, removal authority, failure, timeout, cancellation, and presentation ordering;
489
+ - crashes before and after claim, send, session append, and database update;
490
+ - extension restart, lease expiry, session branches, and two delivery processes;
491
+ - a follow-up prompt that starts another workflow;
492
+ - CLI, TUI, graph, workflow card, redaction, and layout output;
493
+ - Autoimplement with no changes, merge forbidden later, and missing merge authority;
494
+ - restart rebuild mismatch that fails instead of silently changing saved state;
495
+ - the Rust `piw` reader opening the new schema and rejecting the old schema clearly;
496
+ - a real Pi run with model changes and several follow-up prompts.
497
+
498
+ Run:
499
+
500
+ ```bash
501
+ npm run check
502
+ npm run test:e2e
503
+ npx slophammer-ts@latest dry .
504
+ npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
505
+ git diff --check
506
+ npx -y @simpledoc/simpledoc check
507
+ ```
508
+
509
+ ## Rollout
510
+
511
+ This is an alpha hard change.
512
+
513
+ Keep existing `v1` names and the current database user version. Update the exact database shape and digest in place. Do not add a second schema, old-state reader, dual write, alias, fallback, feature flag, or migration shim.
514
+
515
+ Before upgrading, users must finish active work and back up or reset the old local database. The new version must leave incompatible state untouched and stop with a clear reset instruction. Increase the revision of every changed built-in workflow so an old active built-in run cannot resume with new behavior.
516
+
517
+ Do not publish a package, create a release, or update downstream pins without separate authorization.
518
+
519
+ ## Contract impact
520
+
521
+ - **Pi session state:** follow-up prompts are normal user messages whose text contains a short stable follow-up ID for restart checks. Final presentation messages include the run ID in documented custom-message details.
522
+ - **Other saved data:** add workflow settings, setting changes, follow-up prompts, node settings numbers, claims, and delivery status to the existing Pi Workflows SQLite database.
523
+ - **Pi internals:** unchanged.
524
+ - **Public Pi API:** use documented session lifecycle, branch reads, idle state, and `sendUserMessage` only.
525
+ - **Public Pi Workflows API:** add workflow settings declarations, settings-aware node context, safe route selection, and the three new actions.
526
+ - **External applications:** unchanged.
527
+
528
+ ## Limits
529
+
530
+ A workflow can change only settings and routes that its author exposes. It cannot rewrite code, history, completed work, or a running effect.
531
+
532
+ Follow-up delivery needs a later live Pi process and the target session. The current Pi API does not provide native exactly-once message delivery. If Pi later adds a durable user-message queue with caller IDs and queryable delivery results, only the delivery code should change. Workflow definitions, JSON Patch data, setting changes, and follow-up states should remain the same.
@@ -0,0 +1,67 @@
1
+ ---
2
+ title: Simplify workflow run storage
3
+ author: Onur Solmaz <2453968+osolmaz@users.noreply.github.com>
4
+ date: 2026-08-25
5
+ status: implemented
6
+ ---
7
+
8
+ # Simplify workflow run storage
9
+
10
+ ## Goal
11
+
12
+ Keep one SQLite database. Pi owns conversation history. Pi Workflows owns workflow facts. Store each
13
+ large prompt or output once, and build the public `pi-workflows.run-state.v1` view when a reader asks
14
+ for it.
15
+
16
+ This is a hard alpha cutover. The schema stays at version 1. An old database is rejected without any
17
+ change. There is no migration, second reader, fallback, alias, or `v2` schema.
18
+
19
+ ## Data model
20
+
21
+ `runs` keeps run identity, the definition reference, launch options, title, status, input, terminal
22
+ output or error, and timestamps. It does not keep source JSON, a human-decision receipt, carried-step
23
+ counts, or current-node fields.
24
+
25
+ `run_sources` keeps the immutable source identity for the root workflow and each mounted workflow.
26
+ The empty mount path identifies the root source. Mount names come from the definition snapshot.
27
+
28
+ `node_attempts` keeps attempt identity, node identity, status, one structured output, one error, and
29
+ times. It can keep one small execution receipt for action or assistant-message facts. It does not
30
+ keep input, contract, prompt, or general step-metadata blobs.
31
+
32
+ `run_steps` remains the ordered step list. A continuation refers to parent attempts. It stores an
33
+ output override only when the accepted human answer changes the carried checkpoint output. Readers
34
+ count these parent attempts to derive `carriedStepCount`.
35
+
36
+ `session_entries` remains the only durable copy of settled Pi entries. `attempt_entries` links an
37
+ attempt to its prompt, response, first, and last Pi entries. Interactive prompts and visible
38
+ assistant outputs are read from those entries. A structured submitted output or an action output
39
+ continues to use `node_attempts.output_hash`.
40
+
41
+ Current-node fields come from the one active attempt. `waitingOn` comes from the last step of a
42
+ waiting run. A continuation human-decision receipt comes from the decision and continuation rows.
43
+ Trace and session events keep order, IDs, status, and small timing facts. They do not copy prompts,
44
+ messages, tool arguments, tool results, or workflow outputs.
45
+
46
+ ## Cleanup
47
+
48
+ Add an explicit command:
49
+
50
+ ```text
51
+ pi-workflows state prune --before <timestamp> --dry-run
52
+ pi-workflows state prune --before <timestamp> --backup <absolute-path> --apply
53
+ ```
54
+
55
+ The command selects complete terminal run trees older than the cutoff. It refuses a tree with an
56
+ active queue, active lease, unsettled effect, controller reference, channel reference, or a step
57
+ reference from a run outside the tree. Apply mode requires and verifies a backup. It rechecks the
58
+ selection in an exclusive transaction, deletes the selected aggregates, removes unreferenced blobs,
59
+ checkpoints the WAL, vacuums the database, and runs SQLite integrity and foreign-key checks. Startup
60
+ never runs cleanup.
61
+
62
+ ## Verification
63
+
64
+ Focused tests cover repeated large outputs, repeated large interactive prompts, long delta-heavy
65
+ sessions, continuation step sharing, incompatible old state, and prune safety. The repository checks
66
+ run once after their code surface is final. Pi Reviewer then checks the pull request until no P0 or
67
+ P1 issue remains. CI must be green before the rebase merge and patch release.
@@ -73,12 +73,13 @@ the browser.
73
73
  - **Inspector:** Steps, Trace, Conversation, and Info tabs. Each tab is a
74
74
  bracketed symbol button with a full-label mouse target; the selected tab uses
75
75
  the accent surface. Steps can expand to full prompt, output, timestamps,
76
- error, and action receipt fields. Trace can
77
- show the selected attempt, the replay-visible prefix, or the full run, with
78
- expandable JSON payloads. Conversation shows live text, thinking, tool-call
79
- construction, and tool execution, then switches settled messages to the
80
- verbatim Pi entry. Raw records remain expandable. Info shows run metadata,
81
- final output, capture status, counts, and integrity diagnostics.
76
+ error, action data, and the settings scope and change number used by that
77
+ attempt. Trace can show the selected attempt, the replay-visible prefix, or
78
+ the full run, with expandable JSON payloads. Conversation shows live text,
79
+ thinking, tool-call construction, and tool execution, then switches settled
80
+ messages to the verbatim Pi entry. Raw records remain expandable. Info shows
81
+ run metadata, final output, settings scopes and change numbers, queued
82
+ follow-up states, capture status, counts, and integrity diagnostics.
82
83
  - **Timeline:** run status, elapsed time, replay track, playhead, position,
83
84
  playback controls, and speed. Every playback action uses the same bracketed
84
85
  symbol-button style as the inspector and theme actions. The controls accept
package/docs/workflows.md CHANGED
@@ -85,6 +85,9 @@ type WorkflowNodeContext = {
85
85
  outputs: Record<string, unknown>; // accepted output per finished node id
86
86
  results: Record<string, WorkflowNodeResult>; // full result records, including failures
87
87
  state: WorkflowRunState; // the live run state (read-only by convention)
88
+ settings?: unknown; // fixed typed settings for this attempt, when declared
89
+ settingsScopeId?: string;
90
+ settingsChangeNumber?: number;
88
91
  signal: AbortSignal; // aborted on node timeout or run cancellation
89
92
  };
90
93
  ```
@@ -98,6 +101,12 @@ Long-running compute, action, and checkpoint callbacks should observe
98
101
  steps). When the node times out or the run is cancelled, the engine stops
99
102
  waiting immediately, but only cooperative callbacks stop doing work.
100
103
 
104
+ A workflow can declare typed JSON settings with `workflowSettings()`. Each node
105
+ attempt receives one fixed copy. Authorized changes use RFC 6902 JSON Patch and
106
+ affect only later attempts. Use `settingsRoute()` for a pure route choice that
107
+ must retry when a newer settings change wins before route settlement. See
108
+ [Change workflow settings during a run](2026-08-25-workflow-settings.md).
109
+
101
110
  Function actions receive `WorkflowActionContext`, which adds
102
111
  `publishUpdate(update)`. Other callbacks keep the read-only node context.
103
112
 
@@ -424,6 +433,9 @@ The model sees one `workflow` tool. Its `action` field supports:
424
433
  - `status` for the active run or a supplied run ID.
425
434
  - `pause`, `resume`, and `cancel` for the active run.
426
435
  - `answer` with ordinary checkpoint input and an optional run ID. Protected `humanDecision()` gates reject this model-facing action.
436
+ - `change-settings` with an RFC 6902 patch, optional scope ID, and optional expected change number.
437
+ - `queue-follow-up` to save one ordered normal user prompt for after successful completion.
438
+ - `remove-follow-up` to remove an unsent prompt created by the same model source.
427
439
  - `update` for a non-completing update from the current agent attempt.
428
440
  - `submit` for the current workflow step contract.
429
441
 
@@ -432,7 +444,16 @@ workflow prompt then starts a new turn. This keeps the requesting turn outside
432
444
  the workflow's first attempt and prevents an early missing-submission reminder.
433
445
  The normal extension offers all actions. The headless RPC bridge offers only
434
446
  `update` and `submit`, so a workflow child cannot recursively control other
435
- runs.
447
+ runs. Direct `/workflow change-settings`, `queue-follow-up`, and
448
+ `remove-follow-up` commands use verified interactive provenance. Controller
449
+ code can use the matching `ctx.workflows` methods. All surfaces call the same
450
+ SQLite operations.
451
+
452
+ Follow-up prompts stay separate from workflow settings. Successful terminal
453
+ state is saved before delivery. A final presentation settles first. The Pi
454
+ extension then sends prompts in order as normal user messages without
455
+ reactivating the completed run. See [Continue normal work after a workflow
456
+ finishes](2026-08-25-workflow-follow-ups.md).
436
457
 
437
458
  ### Built-in plain summary
438
459