@osolmaz/pi-workflows 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +107 -26
- package/dist/builtins/monitor.workflow.d.ts +71 -0
- package/dist/builtins/monitor.workflow.js +234 -0
- package/dist/builtins/monitor.workflow.js.map +1 -0
- package/dist/controllers/conditions.d.ts +6 -0
- package/dist/controllers/conditions.js +68 -0
- package/dist/controllers/conditions.js.map +1 -0
- package/dist/controllers/definition.d.ts +6 -0
- package/dist/controllers/definition.js +45 -0
- package/dist/controllers/definition.js.map +1 -0
- package/dist/controllers/effects.d.ts +14 -0
- package/dist/controllers/effects.js +104 -0
- package/dist/controllers/effects.js.map +1 -0
- package/dist/controllers/errors.d.ts +12 -0
- package/dist/controllers/errors.js +25 -0
- package/dist/controllers/errors.js.map +1 -0
- package/dist/controllers/index.d.ts +13 -0
- package/dist/controllers/index.js +13 -0
- package/dist/controllers/index.js.map +1 -0
- package/dist/controllers/json.d.ts +5 -0
- package/dist/controllers/json.js +57 -0
- package/dist/controllers/json.js.map +1 -0
- package/dist/controllers/loader.d.ts +23 -0
- package/dist/controllers/loader.js +74 -0
- package/dist/controllers/loader.js.map +1 -0
- package/dist/controllers/manager.d.ts +58 -0
- package/dist/controllers/manager.js +399 -0
- package/dist/controllers/manager.js.map +1 -0
- package/dist/controllers/results.d.ts +5 -0
- package/dist/controllers/results.js +32 -0
- package/dist/controllers/results.js.map +1 -0
- package/dist/controllers/sqlite.d.ts +212 -0
- package/dist/controllers/sqlite.js +1009 -0
- package/dist/controllers/sqlite.js.map +1 -0
- package/dist/controllers/store.d.ts +112 -0
- package/dist/controllers/store.js +32 -0
- package/dist/controllers/store.js.map +1 -0
- package/dist/controllers/types.d.ts +159 -0
- package/dist/controllers/types.js +2 -0
- package/dist/controllers/types.js.map +1 -0
- package/dist/controllers/workflow-engine-scheduler.d.ts +25 -0
- package/dist/controllers/workflow-engine-scheduler.js +93 -0
- package/dist/controllers/workflow-engine-scheduler.js.map +1 -0
- package/dist/controllers/workflows.d.ts +27 -0
- package/dist/controllers/workflows.js +109 -0
- package/dist/controllers/workflows.js.map +1 -0
- package/dist/extension/controller-host.d.ts +47 -0
- package/dist/extension/controller-host.js +110 -0
- package/dist/extension/controller-host.js.map +1 -0
- package/dist/extension/executor.d.ts +14 -1
- package/dist/extension/executor.js +12 -2
- package/dist/extension/executor.js.map +1 -1
- package/dist/extension/index.d.ts +7 -0
- package/dist/extension/index.js +1118 -89
- package/dist/extension/index.js.map +1 -1
- package/dist/extension/recorder.d.ts +91 -0
- package/dist/extension/recorder.js +537 -0
- package/dist/extension/recorder.js.map +1 -0
- package/dist/extension/session-events.d.ts +134 -0
- package/dist/extension/session-events.js +60 -0
- package/dist/extension/session-events.js.map +1 -0
- package/dist/extension/widget.js +25 -24
- package/dist/extension/widget.js.map +1 -1
- package/dist/extension/workflow-tool.d.ts +28 -0
- package/dist/extension/workflow-tool.js +33 -0
- package/dist/extension/workflow-tool.js.map +1 -0
- package/dist/host/processes.d.ts +24 -0
- package/dist/host/processes.js +114 -0
- package/dist/host/processes.js.map +1 -0
- package/dist/host/rpc-bridge.d.ts +9 -0
- package/dist/host/rpc-bridge.js +39 -0
- package/dist/host/rpc-bridge.js.map +1 -0
- package/dist/host/rpc-executor.d.ts +38 -0
- package/dist/host/rpc-executor.js +254 -0
- package/dist/host/rpc-executor.js.map +1 -0
- package/dist/host/runner.d.ts +49 -0
- package/dist/host/runner.js +350 -0
- package/dist/host/runner.js.map +1 -0
- package/dist/render/canvas.d.ts +1 -1
- package/dist/render/canvas.js +5 -0
- package/dist/render/canvas.js.map +1 -1
- package/dist/render/graph-render.d.ts +5 -0
- package/dist/render/graph-render.js +211 -48
- package/dist/render/graph-render.js.map +1 -1
- package/dist/viewer/cli.d.ts +7 -3
- package/dist/viewer/cli.js +150 -19
- package/dist/viewer/cli.js.map +1 -1
- package/dist/viewer/render.js +19 -3
- package/dist/viewer/render.js.map +1 -1
- package/dist/viewer/session-reducer.d.ts +45 -0
- package/dist/viewer/session-reducer.js +266 -0
- package/dist/viewer/session-reducer.js.map +1 -0
- package/dist/workflows/artifacts.d.ts +40 -0
- package/dist/workflows/artifacts.js +155 -0
- package/dist/workflows/artifacts.js.map +1 -0
- package/dist/workflows/engine.d.ts +38 -0
- package/dist/workflows/engine.js +281 -20
- package/dist/workflows/engine.js.map +1 -1
- package/dist/workflows/errors.d.ts +23 -0
- package/dist/workflows/errors.js +38 -0
- package/dist/workflows/errors.js.map +1 -1
- package/dist/workflows/graph.js +0 -5
- package/dist/workflows/graph.js.map +1 -1
- package/dist/workflows/index.d.ts +3 -2
- package/dist/workflows/index.js +2 -1
- package/dist/workflows/index.js.map +1 -1
- package/dist/workflows/loader.d.ts +5 -3
- package/dist/workflows/loader.js +10 -1
- package/dist/workflows/loader.js.map +1 -1
- package/dist/workflows/schema.js +1 -1
- package/dist/workflows/schema.js.map +1 -1
- package/dist/workflows/store.d.ts +98 -10
- package/dist/workflows/store.js +921 -46
- package/dist/workflows/store.js.map +1 -1
- package/dist/workflows/types.d.ts +136 -3
- package/docs/CONTROLLERS.md +215 -0
- package/docs/development.md +51 -24
- package/docs/live-replay-protocol.md +155 -0
- package/docs/plans/2026-08-04-controller-runtime-plan.md +169 -0
- package/docs/plans/2026-08-05-always-on-workflows-plan.md +125 -0
- package/docs/plans/2026-08-10-agent-managed-monitor-workflows-plan.md +184 -0
- package/docs/plans/piw-viewer-experience-implementation-plan.md +674 -0
- package/docs/plans/replayable-run-bundles-implementation-plan.md +65 -0
- package/docs/plans/session-event-replay-implementation-plan.md +494 -0
- package/docs/plans/tui-viewer-implementation-plan.md +64 -0
- package/docs/run-bundles.md +359 -55
- package/docs/session-event-journal.md +470 -0
- package/docs/tui-viewer.md +218 -0
- package/docs/workflows.md +131 -11
- package/examples/controllers/pull-request.controller.ts +215 -0
- package/package.json +11 -2
- package/src/builtins/monitor.workflow.ts +278 -0
- package/src/controllers/conditions.ts +110 -0
- package/src/controllers/definition.ts +65 -0
- package/src/controllers/effects.ts +123 -0
- package/src/controllers/errors.ts +27 -0
- package/src/controllers/index.ts +90 -0
- package/src/controllers/json.ts +62 -0
- package/src/controllers/loader.ts +104 -0
- package/src/controllers/manager.ts +533 -0
- package/src/controllers/results.ts +46 -0
- package/src/controllers/sqlite.ts +1427 -0
- package/src/controllers/store.ts +160 -0
- package/src/controllers/types.ts +183 -0
- package/src/controllers/workflow-engine-scheduler.ts +145 -0
- package/src/controllers/workflows.ts +152 -0
- package/src/extension/controller-host.ts +163 -0
- package/src/extension/executor.ts +29 -2
- package/src/extension/index.ts +1315 -109
- package/src/extension/recorder.ts +662 -0
- package/src/extension/session-events.ts +119 -0
- package/src/extension/widget.ts +26 -24
- package/src/extension/workflow-tool.ts +59 -0
- package/src/host/processes.ts +119 -0
- package/src/host/rpc-bridge.ts +44 -0
- package/src/host/rpc-executor.ts +299 -0
- package/src/host/runner.ts +406 -0
- package/src/render/canvas.ts +19 -1
- package/src/render/graph-render.ts +277 -44
- package/src/viewer/cli.ts +167 -21
- package/src/viewer/render.ts +21 -3
- package/src/viewer/session-reducer.ts +347 -0
- package/src/workflows/artifacts.ts +188 -0
- package/src/workflows/engine.ts +365 -19
- package/src/workflows/errors.ts +45 -0
- package/src/workflows/graph.ts +0 -5
- package/src/workflows/index.ts +15 -0
- package/src/workflows/loader.ts +13 -3
- package/src/workflows/schema.ts +1 -1
- package/src/workflows/store.ts +1157 -48
- package/src/workflows/types.ts +151 -3
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# Replayable run bundles — implementation plan
|
|
2
|
+
|
|
3
|
+
Goal: make every workflow run bundle self-contained for replay, as specified
|
|
4
|
+
in [run-bundles.md](../run-bundles.md). This is a hard cutover: one format,
|
|
5
|
+
no readers for the previous layout, no compatibility code.
|
|
6
|
+
|
|
7
|
+
## Steps
|
|
8
|
+
|
|
9
|
+
1. **Spec** — rewrite `docs/run-bundles.md`: trace-first write discipline,
|
|
10
|
+
`traceSeq` in `state.json`, content-addressed `artifacts/`, `session/`
|
|
11
|
+
with binding + verbatim Pi entries, explicit `conversation` entry ranges
|
|
12
|
+
on agent steps, event payload catalog. Review the data model with
|
|
13
|
+
Schemator. _(done)_
|
|
14
|
+
2. **Types** — `ArtifactRef`/`ArtifactValue`, `ConversationRange`,
|
|
15
|
+
`WorkflowSessionBinding`, `WorkflowSessionEntryRecord`; `schema` +
|
|
16
|
+
`traceSeq` on `WorkflowRunState`; `promptText` renamed to `prompt`;
|
|
17
|
+
optional `session`/`artifacts` manifest paths. _(done)_
|
|
18
|
+
3. **Artifacts** — `src/workflows/artifacts.ts`: threshold-based string
|
|
19
|
+
externalization with `$escaped` collision handling, dedupe by content
|
|
20
|
+
hash, resolve/decode helpers. _(done)_
|
|
21
|
+
4. **Store** — trace append before projection writes, `traceSeq` stamping,
|
|
22
|
+
value encoding at the documented positions, session writers sharing the
|
|
23
|
+
trace sequence, 0700/0600 modes. _(done)_
|
|
24
|
+
5. **Engine** — outputs/receipts/conversation in terminal node events,
|
|
25
|
+
`input` in `run_started`, `finalOutput` in terminal run events, shared
|
|
26
|
+
store option. _(done)_
|
|
27
|
+
6. **Recorder** — `src/extension/recorder.ts` copies session entries into
|
|
28
|
+
the bundle via documented read APIs; executor mark/range hooks attach
|
|
29
|
+
`conversation` to accepted submissions; extension wires recording to
|
|
30
|
+
`message_end`/`agent_settled` and flushes before accepting a submission.
|
|
31
|
+
_(done)_
|
|
32
|
+
7. **Fixtures** — deterministic layout + render golden files under
|
|
33
|
+
`fixtures/layout/`, generated from the TypeScript reference and pinned by
|
|
34
|
+
a parity test, consumed by the Rust port. _(done)_
|
|
35
|
+
|
|
36
|
+
## Schemator review adjudication
|
|
37
|
+
|
|
38
|
+
The data model was challenged field-by-field with Schemator (Codex review
|
|
39
|
+
strategy). Outcomes:
|
|
40
|
+
|
|
41
|
+
- **Accepted** — `WorkflowRunState.currentNodeType` removed: written but
|
|
42
|
+
never read; every consumer derives the type of `currentNode` from the
|
|
43
|
+
definition snapshot.
|
|
44
|
+
- **Rejected** — `statusDetail` (live-UX label read by the widget and both
|
|
45
|
+
viewers), `paused` (orthogonal to `status`: a pause request holding at a
|
|
46
|
+
boundary while status stays `running`), `waitingOn` (well-specified:
|
|
47
|
+
checkpoint node id), `updatedAt` (cheap liveness signal for list views that
|
|
48
|
+
read only `state.json`), `durationMs` (engine-measured, not a timestamp
|
|
49
|
+
subtraction), `workflowPath` (provenance with a real re-run use case),
|
|
50
|
+
`piSessionFile` (documented provenance, explicitly never read for replay).
|
|
51
|
+
- **Rejected** — renaming trace/session `at` to `occurredAt`: `at` is the
|
|
52
|
+
event-envelope convention used throughout the trace format; the `…At`
|
|
53
|
+
suffix rule applies to document fields, and the deviation is deliberate.
|
|
54
|
+
- **Rejected** — `ArtifactRef.bytes` (lets viewers show sizes and verify
|
|
55
|
+
truncation without stat-ing every artifact) and
|
|
56
|
+
`WorkflowSessionBinding.boundAt` (binding.json stays self-describing
|
|
57
|
+
without consulting the trace).
|
|
58
|
+
|
|
59
|
+
## Non-goals
|
|
60
|
+
|
|
61
|
+
- Reading bundles written before this change.
|
|
62
|
+
- Normalizing Pi session entries (they are recorded verbatim; Pi owns the
|
|
63
|
+
shape).
|
|
64
|
+
- Capturing conversations for runs driven by non-conversation executors
|
|
65
|
+
(headless tests); `session/` is simply absent.
|
|
@@ -0,0 +1,494 @@
|
|
|
1
|
+
# Session event replay implementation plan
|
|
2
|
+
|
|
3
|
+
## Objective
|
|
4
|
+
|
|
5
|
+
Add live assistant streaming and deterministic temporal replay to PIW using the
|
|
6
|
+
journal defined in
|
|
7
|
+
[session-event-journal.md](../session-event-journal.md).
|
|
8
|
+
|
|
9
|
+
A completed implementation records the documented Pi event stream in each
|
|
10
|
+
session-bound run bundle, shows it while the run is active, and replays the same
|
|
11
|
+
history later. The workflow trace and final Pi entries keep their existing
|
|
12
|
+
roles.
|
|
13
|
+
|
|
14
|
+
## Boundaries
|
|
15
|
+
|
|
16
|
+
- Use only documented Pi extension hooks.
|
|
17
|
+
- Do not modify Pi source, Pi session state, global Pi session files, provider
|
|
18
|
+
requests, or provider response protocols.
|
|
19
|
+
- Keep `trace.ndjson`, `entries.ndjson`, and `events.ndjson` as separate
|
|
20
|
+
contracts.
|
|
21
|
+
- Store normalized semantic events. Never persist Pi's cumulative `partial`,
|
|
22
|
+
terminal `message`, or terminal `error` snapshots from
|
|
23
|
+
`AssistantMessageEvent`.
|
|
24
|
+
- Preserve every normalized assistant event and timestamp. Filesystem and
|
|
25
|
+
WebSocket batching must not merge event records.
|
|
26
|
+
- Keep capture failures separate from workflow failures.
|
|
27
|
+
- Keep `src/workflows` independent of Pi and both viewer layers.
|
|
28
|
+
- Preserve TypeScript and Rust reducer behavior through shared fixtures.
|
|
29
|
+
- Keep boxed node dimensions fixed while temporal events change their status or
|
|
30
|
+
metadata. Live updates must never reflow the graph.
|
|
31
|
+
- Replace the session-bound bundle contract in place. Do not add a legacy
|
|
32
|
+
reader, dual writer, migration, or fallback path.
|
|
33
|
+
- Keep bundle permissions, artifact containment, terminal sanitization, and
|
|
34
|
+
loopback-only remote viewing unchanged.
|
|
35
|
+
|
|
36
|
+
## Contract decisions
|
|
37
|
+
|
|
38
|
+
The durable history is split by authority:
|
|
39
|
+
|
|
40
|
+
- `trace.ndjson` records workflow transitions.
|
|
41
|
+
- `session/entries.ndjson` records final Pi session entries.
|
|
42
|
+
- `session/events.ndjson` records temporal session behavior.
|
|
43
|
+
- `session/capture.json` reports whether temporal capture completed.
|
|
44
|
+
|
|
45
|
+
The viewer orders temporal events by `seq`. The `at` timestamp controls replay
|
|
46
|
+
speed and never changes event order. Final Pi entries replace provisional
|
|
47
|
+
message reconstructions after explicit `entryId` linkage.
|
|
48
|
+
|
|
49
|
+
Version 1 records each `tool_execution_update` occurrence and timestamp but
|
|
50
|
+
omits Pi's arbitrary cumulative `partialResult`. Tool start arguments and the
|
|
51
|
+
final tool result are retained. This keeps tool timing without quadratic
|
|
52
|
+
storage.
|
|
53
|
+
|
|
54
|
+
## Schema review
|
|
55
|
+
|
|
56
|
+
Schemator accepted the session-event envelope without changes in one iteration.
|
|
57
|
+
A second review simplified `capture.json` over three iterations. The final
|
|
58
|
+
capture projection omits `runId` because the bundle supplies it. It also omits
|
|
59
|
+
start, update, and finish timestamps that readers can derive from the journal
|
|
60
|
+
and run state. Capture failures keep the durable `failedAt`, `code`, and
|
|
61
|
+
`message` fields. The retained event and entry counts let readers check the
|
|
62
|
+
append-only files without trusting a malformed tail.
|
|
63
|
+
|
|
64
|
+
## Implementation order
|
|
65
|
+
|
|
66
|
+
Work should land in the order below. Each stage has an exit criterion so a
|
|
67
|
+
partially completed branch remains reviewable.
|
|
68
|
+
|
|
69
|
+
### Persisted types
|
|
70
|
+
|
|
71
|
+
Add the persisted contracts to `src/workflows/types.ts`:
|
|
72
|
+
|
|
73
|
+
- `WorkflowSessionEventRecord`
|
|
74
|
+
- the event type union and payload types
|
|
75
|
+
- `WorkflowSessionCapture`
|
|
76
|
+
- capture status and failure types
|
|
77
|
+
|
|
78
|
+
Add schema constants and fixed paths in `src/workflows/store.ts`:
|
|
79
|
+
|
|
80
|
+
```text
|
|
81
|
+
session/events.ndjson
|
|
82
|
+
session/capture.json
|
|
83
|
+
pi-workflows.session-event.v1
|
|
84
|
+
pi-workflows.session-capture.v1
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Keep event payloads in camelCase. Validate required correlation ids when the
|
|
88
|
+
store receives a normalized record. The store must not import Pi event types.
|
|
89
|
+
|
|
90
|
+
Exit criterion: TypeScript tests can serialize valid persisted shapes and
|
|
91
|
+
reject invalid ones without loading the extension.
|
|
92
|
+
|
|
93
|
+
### Session event writer
|
|
94
|
+
|
|
95
|
+
Add a dedicated event append path to `WorkflowRunStore`. It must use its own
|
|
96
|
+
per-run sequence and append chain so high-rate event writes cannot queue ahead
|
|
97
|
+
of workflow-state persistence.
|
|
98
|
+
|
|
99
|
+
The store API should accept fully stamped ordered batches. It checks the next
|
|
100
|
+
expected `seq` inside the event append chain and writes one JSON line per
|
|
101
|
+
record. A batch uses one `appendFile` call while preserving separate records.
|
|
102
|
+
|
|
103
|
+
Add atomic `writeSessionCapture()` support. Create the recording projection
|
|
104
|
+
before the first event. At terminal capture, count complete event and entry
|
|
105
|
+
lines and write `complete` or `failed` with the durable counts.
|
|
106
|
+
|
|
107
|
+
After an append error, stop that run's event writer. Do not append after a torn
|
|
108
|
+
line. Preserve the first failure and let workflow persistence continue.
|
|
109
|
+
|
|
110
|
+
Exit criterion: store tests cover sequence assignment, concurrent batches,
|
|
111
|
+
torn writes, append failure, atomic capture replacement, final counts, and no
|
|
112
|
+
writes after terminal capture.
|
|
113
|
+
|
|
114
|
+
### Bounded recorder queue
|
|
115
|
+
|
|
116
|
+
Extend `SessionRecorder` with a memory queue in front of the store batch API.
|
|
117
|
+
The recorder assigns the next `seq`, captures `at`, and fixes correlation ids
|
|
118
|
+
before enqueueing each event. The `message_update` and `tool_execution_update`
|
|
119
|
+
paths then schedule a flush without awaiting filesystem work.
|
|
120
|
+
|
|
121
|
+
Use initial limits that are easy to test:
|
|
122
|
+
|
|
123
|
+
- Flush after 25 ms, 256 records, or 256 KiB, whichever comes first.
|
|
124
|
+
- Stop capture at 8,192 queued records or 16 MiB of queued JSON.
|
|
125
|
+
- Reject a normalized event larger than 1 MiB.
|
|
126
|
+
|
|
127
|
+
These are implementation safety limits. Measure the synthetic stress test
|
|
128
|
+
before changing them. Reaching a limit marks capture as failed and stops
|
|
129
|
+
temporal recording. Silent dropping is forbidden.
|
|
130
|
+
|
|
131
|
+
The writer keeps each event record intact. It may write many lines together,
|
|
132
|
+
but it must not combine adjacent text or thinking deltas.
|
|
133
|
+
|
|
134
|
+
`SessionRecorder.finish()` must wait for an active Pi turn through `turn_end`,
|
|
135
|
+
then stop accepting events, flush the queue, drain entry recording, write
|
|
136
|
+
terminal capture status, and return. `SessionRecorder.stop()` handles session
|
|
137
|
+
shutdown and marks an active turn as interrupted. The engine's existing
|
|
138
|
+
`onRunFinishing` hook must await `finish()` before it writes the terminal
|
|
139
|
+
workflow event.
|
|
140
|
+
|
|
141
|
+
Exit criterion: a 10,000-event synthetic stream keeps queue use bounded,
|
|
142
|
+
preserves all records below the limit, and does not block an unrelated workflow
|
|
143
|
+
snapshot behind the event append chain.
|
|
144
|
+
|
|
145
|
+
### Pi event normalization
|
|
146
|
+
|
|
147
|
+
Add a Pi-facing module under `src/extension`, separate from the pure store
|
|
148
|
+
contracts. It maps these documented hooks:
|
|
149
|
+
|
|
150
|
+
| Pi hook | Journal event |
|
|
151
|
+
| ----------------------- | ------------------------- |
|
|
152
|
+
| `turn_start` | `turn_started` |
|
|
153
|
+
| `turn_end` | `turn_finished` |
|
|
154
|
+
| `message_start` | `message_started` |
|
|
155
|
+
| `message_update` | `assistant_event` |
|
|
156
|
+
| `message_end` | `message_finished` |
|
|
157
|
+
| `tool_execution_start` | `tool_execution_started` |
|
|
158
|
+
| `tool_execution_update` | `tool_execution_updated` |
|
|
159
|
+
| `tool_execution_end` | `tool_execution_finished` |
|
|
160
|
+
|
|
161
|
+
The normalizer must exhaustively switch over the installed
|
|
162
|
+
`AssistantMessageEvent` union. Tests must fail when a new Pi event variant is
|
|
163
|
+
not handled.
|
|
164
|
+
|
|
165
|
+
For assistant events:
|
|
166
|
+
|
|
167
|
+
- Drop every `partial` field.
|
|
168
|
+
- Drop the cumulative `message` field from `done`.
|
|
169
|
+
- Drop the cumulative `error` object from `error`.
|
|
170
|
+
- Retain delta strings, block indexes, block-end content, final tool-call
|
|
171
|
+
objects, and stop reasons.
|
|
172
|
+
|
|
173
|
+
For tool updates, record the event with an empty payload. For tool finish,
|
|
174
|
+
encode the final public-hook result with the existing artifact rules.
|
|
175
|
+
|
|
176
|
+
Exit criterion: normalization tests cover every Pi event variant and assert
|
|
177
|
+
that serialized records contain none of the omitted cumulative fields.
|
|
178
|
+
|
|
179
|
+
### Ownership and correlation
|
|
180
|
+
|
|
181
|
+
Teach the recorder which agent attempt owns the conversation before the
|
|
182
|
+
executor delivers a prompt. Generate run-local turn and message ids as their
|
|
183
|
+
start hooks arrive.
|
|
184
|
+
|
|
185
|
+
Maintain maps for:
|
|
186
|
+
|
|
187
|
+
- Active attempt to turn.
|
|
188
|
+
- Turn to messages.
|
|
189
|
+
- Assistant message to generated tool calls.
|
|
190
|
+
- `toolCallId` to the owner captured at tool start.
|
|
191
|
+
|
|
192
|
+
Each end event uses the owner saved at its matching start. It must not read the
|
|
193
|
+
workflow's current node again. This prevents a late event from moving to the
|
|
194
|
+
next attempt.
|
|
195
|
+
|
|
196
|
+
Pi persists a message entry after `message_end` handlers return. Buffer that
|
|
197
|
+
receipt and later events until the next synchronized boundary hook or workflow
|
|
198
|
+
tool execution can record the branch entry. Then release the buffered records
|
|
199
|
+
in original hook order. Write `message_finished` with `settled: true` and
|
|
200
|
+
`entryId` only when the exact stable message identity matches a durable entry.
|
|
201
|
+
Use `settled: false` for an aborted or otherwise unrecorded message. Do not
|
|
202
|
+
match messages by text, timestamp, or position during replay.
|
|
203
|
+
|
|
204
|
+
Exit criterion: tests advance the workflow between start and finish hooks and
|
|
205
|
+
confirm that every late event retains its original `nodeId` and `attemptId`.
|
|
206
|
+
|
|
207
|
+
### Extension wiring and failure policy
|
|
208
|
+
|
|
209
|
+
Register the new handlers next to the existing session recorder handlers in
|
|
210
|
+
`src/extension/index.ts`. Keep event handler setup and teardown scoped to the
|
|
211
|
+
active workflow run.
|
|
212
|
+
|
|
213
|
+
High-rate update handlers enqueue synchronously. Boundary handlers may await a
|
|
214
|
+
flush when they need causal ordering or entry linkage. Capture errors are
|
|
215
|
+
caught, recorded in `capture.json`, and surfaced to the viewer. They must not
|
|
216
|
+
reject an agent turn, fail a tool, or change workflow status.
|
|
217
|
+
|
|
218
|
+
Session shutdown and workflow cancellation must stop the writer idempotently.
|
|
219
|
+
No timer, pending promise, or file handle may survive recorder shutdown.
|
|
220
|
+
|
|
221
|
+
Exit criterion: hook tests cover normal completion, abort, cancellation,
|
|
222
|
+
session shutdown, duplicate stop calls, write failure, and a workflow that
|
|
223
|
+
completes successfully after capture fails.
|
|
224
|
+
|
|
225
|
+
### Bundle readers
|
|
226
|
+
|
|
227
|
+
Extend both bundle readers with `session/events.ndjson` and
|
|
228
|
+
`session/capture.json`.
|
|
229
|
+
|
|
230
|
+
For TypeScript, update the standalone viewer's bundle model and loader. For
|
|
231
|
+
Rust, add serde types under `tui/src/bundle`, a third `NdjsonTailer`, and capture
|
|
232
|
+
status loading.
|
|
233
|
+
|
|
234
|
+
Readers must distinguish:
|
|
235
|
+
|
|
236
|
+
- Capture currently recording.
|
|
237
|
+
- Verified complete capture.
|
|
238
|
+
- Explicitly failed capture.
|
|
239
|
+
- Missing or invalid capture.
|
|
240
|
+
- Sequence gaps or count mismatches.
|
|
241
|
+
- A torn final line during recording.
|
|
242
|
+
|
|
243
|
+
Missing files in a session-bound bundle are an invalid or incomplete capture.
|
|
244
|
+
They are not treated as an old format with an alternate read path.
|
|
245
|
+
|
|
246
|
+
Exit criterion: TypeScript and Rust reader tests produce the same integrity
|
|
247
|
+
result for valid and invalid bundles, including failed capture and torn tails.
|
|
248
|
+
|
|
249
|
+
### Deterministic reducer
|
|
250
|
+
|
|
251
|
+
Implement a pure temporal reducer in TypeScript and Rust. Its input is the
|
|
252
|
+
settled entry list plus session events through a selected sequence. Its output
|
|
253
|
+
contains visible settled entries, active messages and content blocks, active or
|
|
254
|
+
finished tools, and integrity diagnostics.
|
|
255
|
+
|
|
256
|
+
The reducer follows these rules:
|
|
257
|
+
|
|
258
|
+
- Process by `seq` only.
|
|
259
|
+
- Append text and thinking deltas by `messageId` and `contentIndex`. Apply the
|
|
260
|
+
same addressing to tool-call deltas.
|
|
261
|
+
- Use block-end content to check the folded block.
|
|
262
|
+
- Switch to the verbatim Pi entry after a settled `message_finished`.
|
|
263
|
+
- Keep unsealed messages visible as partial output.
|
|
264
|
+
- Retain unknown event sequence positions while ignoring their payloads.
|
|
265
|
+
- Let settled entries win when a reconciliation check fails.
|
|
266
|
+
|
|
267
|
+
Create shared JSON fixtures under `fixtures/session-events/`. Generate expected
|
|
268
|
+
states with the TypeScript implementation and consume the same fixtures from
|
|
269
|
+
Rust. Include every event type and every documented failure mode.
|
|
270
|
+
|
|
271
|
+
Use an in-memory timestamp index and reducer checkpoints for seeking. Start
|
|
272
|
+
with one checkpoint every 256 events. Checkpoints are viewer cache data and
|
|
273
|
+
must never be written into the run bundle.
|
|
274
|
+
|
|
275
|
+
Exit criterion: all shared fixtures produce the same visible state and
|
|
276
|
+
integrity diagnostics in TypeScript and Rust at every recorded replay point.
|
|
277
|
+
|
|
278
|
+
### PIW live and replay views
|
|
279
|
+
|
|
280
|
+
Update the conversation pane to render reducer output. Live mode shows active
|
|
281
|
+
text, thinking, tool-call construction, and tool execution as events arrive.
|
|
282
|
+
Once a message settles, the pane renders the verbatim Pi entry.
|
|
283
|
+
|
|
284
|
+
Extend replay position with a temporal cursor while keeping workflow-step
|
|
285
|
+
selection intact. The replay clock schedules events from `at`, orders them by
|
|
286
|
+
`seq`, and applies the existing 1x, 2x, 5x, and 10x speed choices. Timestamp
|
|
287
|
+
ties remain deterministic because sequence order wins.
|
|
288
|
+
|
|
289
|
+
Follow mode stays attached to the newest temporal event. Manual scrolling or
|
|
290
|
+
seeking detaches it, and the existing live controls reattach it. Capture
|
|
291
|
+
failure or unverified capture must remain visible in the status area and Info
|
|
292
|
+
inspector.
|
|
293
|
+
|
|
294
|
+
Temporal state also fills the stable full node cards defined in
|
|
295
|
+
[piw-viewer-experience-implementation-plan.md](piw-viewer-experience-implementation-plan.md).
|
|
296
|
+
A node card reserves slots for its status symbol and label plus its exact id
|
|
297
|
+
and type. Separate slots hold start, branch, or terminal markers and every
|
|
298
|
+
branch label. Attempt metadata and timing remain visible beside the short detail
|
|
299
|
+
or outcome. Every node in the graph uses one canonical outer width
|
|
300
|
+
and height chosen before layout. Blank padding fills unused slots. Streaming
|
|
301
|
+
and timer ticks may change card contents. Replay and terminal settlement may do
|
|
302
|
+
the same without changing node bounds or edge routes.
|
|
303
|
+
|
|
304
|
+
The boxed card must show all of those fields without ellipses. Long immutable
|
|
305
|
+
text wraps during initial measurement, which increases the canonical size for
|
|
306
|
+
every node. Narrow terminals pan over the fixed graph. Unbounded prompts and
|
|
307
|
+
outputs remain in the inspector, along with error text and tool payloads.
|
|
308
|
+
|
|
309
|
+
Exit criterion: a manual run visibly streams text and thinking, shows tool
|
|
310
|
+
lifecycle changes, can seek into an unfinished message, and reaches the same
|
|
311
|
+
final conversation as `entries.ndjson`. Its node cards keep identical bounds at
|
|
312
|
+
every temporal cursor while their status and timing fields update.
|
|
313
|
+
|
|
314
|
+
### Live transport
|
|
315
|
+
|
|
316
|
+
Extend the run view in `docs/live-replay-protocol.md` and the Rust protocol
|
|
317
|
+
types:
|
|
318
|
+
|
|
319
|
+
```json
|
|
320
|
+
"session": {
|
|
321
|
+
"binding": {},
|
|
322
|
+
"entries": [],
|
|
323
|
+
"events": [],
|
|
324
|
+
"capture": {}
|
|
325
|
+
}
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
Tail event growth and send it through the existing `append` patch operation at
|
|
329
|
+
`/session/events`. Replace capture state at `/session/capture`.
|
|
330
|
+
|
|
331
|
+
Collect pending event records for up to 50 ms before producing one run patch.
|
|
332
|
+
Do not merge event records. The run revision advances once per patch batch, so
|
|
333
|
+
a token burst does not create one network revision per token.
|
|
334
|
+
|
|
335
|
+
Snapshots contain the complete event history and current capture status.
|
|
336
|
+
Reconnection, revision-gap recovery, and same-revision snapshot replacement use
|
|
337
|
+
the existing resubscription path.
|
|
338
|
+
|
|
339
|
+
Exit criterion: burst tests show bounded revision growth, exact event order,
|
|
340
|
+
and a byte-for-byte equivalent run view after snapshot plus patches or a fresh
|
|
341
|
+
resnapshot.
|
|
342
|
+
|
|
343
|
+
### Documentation and format replacement
|
|
344
|
+
|
|
345
|
+
When code and tests pass, fold the target contract into
|
|
346
|
+
`docs/run-bundles.md` and remove the target-status note from
|
|
347
|
+
`docs/session-event-journal.md`. Update `docs/live-replay-protocol.md`,
|
|
348
|
+
`docs/tui-viewer.md`, `docs/development.md`, and the README where users need new
|
|
349
|
+
commands or controls.
|
|
350
|
+
|
|
351
|
+
Keep `pi-workflows.run-bundle.v1` and `pi-workflows.replay.v1` as directed by
|
|
352
|
+
the repository's in-place replacement policy. Add the two new file schema ids.
|
|
353
|
+
Do not document or retain an old session layout.
|
|
354
|
+
|
|
355
|
+
Exit criterion: the documentation describes only the implemented contract and
|
|
356
|
+
all examples validate against real produced bundles.
|
|
357
|
+
|
|
358
|
+
## Tests
|
|
359
|
+
|
|
360
|
+
### Storage and normalization
|
|
361
|
+
|
|
362
|
+
- Every `AssistantMessageEvent` variant maps to the documented payload.
|
|
363
|
+
- No persisted record contains `partial`, terminal `message`, terminal `error`,
|
|
364
|
+
or tool `partialResult`.
|
|
365
|
+
- Stored bytes grow linearly for generated text of increasing length.
|
|
366
|
+
- Batch boundaries do not change event count, sequence, ids, or timestamps.
|
|
367
|
+
- Queue overflow and oversized events fail capture without failing the run.
|
|
368
|
+
- A failed append leaves at most one torn final line and prevents later event
|
|
369
|
+
appends.
|
|
370
|
+
- Terminal capture counts match valid lines on disk.
|
|
371
|
+
|
|
372
|
+
### Correlation and lifecycle
|
|
373
|
+
|
|
374
|
+
- Turn and message ids remain stable through late events, as do tool ownership
|
|
375
|
+
and workflow attempt ids.
|
|
376
|
+
- A settled message links to the exact recorded Pi entry.
|
|
377
|
+
- An aborted message remains unsealed and replayable.
|
|
378
|
+
- Recorder shutdown drains once and rejects later enqueue attempts.
|
|
379
|
+
- The terminal workflow event is physically later than terminal capture status.
|
|
380
|
+
- Bundle files remain unchanged after terminal workflow state.
|
|
381
|
+
|
|
382
|
+
### Reducer parity
|
|
383
|
+
|
|
384
|
+
- Text and thinking blocks fold correctly at each delta.
|
|
385
|
+
- Tool-call arguments fold before tool execution starts.
|
|
386
|
+
- Tool updates advance timing without copying cumulative output.
|
|
387
|
+
- Final entries replace provisional messages only at their explicit seal.
|
|
388
|
+
- Timestamp ties and backward timestamps preserve sequence order.
|
|
389
|
+
- Unknown events do not change known state.
|
|
390
|
+
- Reconciliation mismatches produce diagnostics while final entries remain
|
|
391
|
+
authoritative.
|
|
392
|
+
|
|
393
|
+
### Node-card stability
|
|
394
|
+
|
|
395
|
+
- Every boxed node uses the same graph-wide outer width and height.
|
|
396
|
+
- Node bounds and edge routes remain unchanged at every temporal cursor.
|
|
397
|
+
- Symbols for every status from queued through cancelled use their documented
|
|
398
|
+
slots and text labels.
|
|
399
|
+
- Start and terminal symbols remain visible beside runtime status data, as does
|
|
400
|
+
the branch count.
|
|
401
|
+
- Full node ids and types render without ellipses. Every branch label remains
|
|
402
|
+
visible. Attempt counts and timing fields remain visible, as do short details.
|
|
403
|
+
- Long card text expands the initial canonical dimensions and pads every shorter
|
|
404
|
+
card instead of causing later layout changes.
|
|
405
|
+
|
|
406
|
+
### Local and remote viewing
|
|
407
|
+
|
|
408
|
+
- Local tailing buffers a torn final event line.
|
|
409
|
+
- Append patches preserve every event under a high-rate burst.
|
|
410
|
+
- Revision gaps force a complete snapshot. Partial recovery is forbidden.
|
|
411
|
+
- Reconnect restores event and capture subscriptions.
|
|
412
|
+
- A fresh same-revision snapshot replaces stale cached events.
|
|
413
|
+
- Capture failure, missing capture, and interruption have distinct labels.
|
|
414
|
+
- Seeking never exposes events or entries later than the selected cursor.
|
|
415
|
+
|
|
416
|
+
### End-to-end coverage
|
|
417
|
+
|
|
418
|
+
Extend the mock OpenAI-compatible E2E server to emit text and thinking chunks
|
|
419
|
+
plus a tool-call stream. Run a real Pi process with the extension, then assert that:
|
|
420
|
+
|
|
421
|
+
- The bundle contains the expected temporal files.
|
|
422
|
+
- Capture finishes as `complete`.
|
|
423
|
+
- Normalized journal events match the streamed order.
|
|
424
|
+
- Finalized entries match the reducer's final state.
|
|
425
|
+
- The Rust viewer can replay an intermediate chunk.
|
|
426
|
+
- No files outside the temporary test directory change.
|
|
427
|
+
|
|
428
|
+
Add a second E2E failure case with an injected event-writer error. The workflow
|
|
429
|
+
must still finish, and PIW must show failed capture.
|
|
430
|
+
|
|
431
|
+
## Operational checks
|
|
432
|
+
|
|
433
|
+
Before calling the feature production-ready, measure a synthetic stream with at
|
|
434
|
+
least 100,000 small assistant events. Record total bytes, peak queued bytes,
|
|
435
|
+
append batch count, elapsed writer time, and workflow snapshot latency while the
|
|
436
|
+
stream runs.
|
|
437
|
+
|
|
438
|
+
This safety check does not select between competing designs. The design passes
|
|
439
|
+
when memory stays within the configured queue bound, stored size is
|
|
440
|
+
linear in delta bytes, no event is lost below the bound, and workflow snapshots
|
|
441
|
+
remain responsive. If the initial constants fail, change the smallest relevant
|
|
442
|
+
limit or batch size and rerun the same workload.
|
|
443
|
+
|
|
444
|
+
## Repository checks
|
|
445
|
+
|
|
446
|
+
Run focused TypeScript and Rust tests after each stage. Before finishing the
|
|
447
|
+
complete implementation, run:
|
|
448
|
+
|
|
449
|
+
```bash
|
|
450
|
+
npm run check
|
|
451
|
+
npm run test:e2e
|
|
452
|
+
npx slophammer-ts@latest dry .
|
|
453
|
+
npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
Also run the Rust checks directly while iterating on `tui/`:
|
|
457
|
+
|
|
458
|
+
```bash
|
|
459
|
+
cargo fmt --check --manifest-path tui/Cargo.toml
|
|
460
|
+
cargo clippy --manifest-path tui/Cargo.toml --all-targets -- -D warnings
|
|
461
|
+
cargo test --manifest-path tui/Cargo.toml
|
|
462
|
+
```
|
|
463
|
+
|
|
464
|
+
## Contract impact
|
|
465
|
+
|
|
466
|
+
Session state remains unchanged. The extension reads Pi's normal session
|
|
467
|
+
entries and public event payloads but does not append, edit, or remove Pi
|
|
468
|
+
entries.
|
|
469
|
+
|
|
470
|
+
The run bundle gains `session/events.ndjson` and `session/capture.json`. No
|
|
471
|
+
other persistent store changes. Pi internals and provider protocols remain
|
|
472
|
+
unchanged.
|
|
473
|
+
|
|
474
|
+
The public Pi API surface is limited to `turn_start`, `turn_end`,
|
|
475
|
+
`message_start`, `message_update`, `message_end`, `tool_execution_start`,
|
|
476
|
+
`tool_execution_update`, and `tool_execution_end`, plus the documented
|
|
477
|
+
read-only session manager getters already used by `SessionRecorder`.
|
|
478
|
+
|
|
479
|
+
## Completion criteria
|
|
480
|
+
|
|
481
|
+
The work is complete when:
|
|
482
|
+
|
|
483
|
+
1. Every session-bound run writes the specified journal and capture status.
|
|
484
|
+
2. High-rate hooks never await disk and cannot grow memory without a bound.
|
|
485
|
+
3. Capture failure is explicit and never changes workflow success or failure.
|
|
486
|
+
4. Local and remote PIW views show the same live temporal state.
|
|
487
|
+
5. Replay can seek to any event and deterministically reconstruct that point.
|
|
488
|
+
6. Final reducer state reconciles with the verbatim Pi entries.
|
|
489
|
+
7. TypeScript and Rust pass the same temporal fixtures.
|
|
490
|
+
8. Terminal bundles never change after the terminal workflow event.
|
|
491
|
+
9. Full boxed nodes show the documented ACPX-style fields and symbols while
|
|
492
|
+
keeping identical bounds across live updates and replay.
|
|
493
|
+
10. The full repository and E2E checks pass.
|
|
494
|
+
11. The implemented docs contain no legacy session format or fallback path.
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# Rust TUI viewer — implementation plan
|
|
2
|
+
|
|
3
|
+
Goal: `piw`, a Rust TUI in `tui/` for live viewing and replaying workflow
|
|
4
|
+
runs, per [tui-viewer.md](../tui-viewer.md) and
|
|
5
|
+
[live-replay-protocol.md](../live-replay-protocol.md).
|
|
6
|
+
|
|
7
|
+
## Architecture
|
|
8
|
+
|
|
9
|
+
One crate, library-first with a thin binary:
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
tui/
|
|
13
|
+
Cargo.toml # package "piw"
|
|
14
|
+
src/
|
|
15
|
+
main.rs # CLI (clap): browse / open / serve / connect
|
|
16
|
+
bundle/ # serde types, bundle reader, NDJSON tailer, runs watcher
|
|
17
|
+
source/ # RunSource: semantic run views + revisioned patches
|
|
18
|
+
protocol/ # message types, JSON Patch+ apply/diff, WS server+client
|
|
19
|
+
layout/ # port of src/render/graph.ts
|
|
20
|
+
render/ # port of canvas.ts + graph-render.ts (cell canvas, viewport)
|
|
21
|
+
ui/ # ratatui app: panes, transport, input, camera
|
|
22
|
+
tests/ # fixture parity, protocol round-trip, tailer tests
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
`RunSource` is the seam: the TUI consumes run views + patch streams the same
|
|
26
|
+
way whether they come from the local filesystem or a WebSocket.
|
|
27
|
+
|
|
28
|
+
## Steps
|
|
29
|
+
|
|
30
|
+
1. **Scaffold** — Cargo crate (ratatui, crossterm, tokio, tokio-tungstenite,
|
|
31
|
+
notify, serde, clap), rustfmt/clippy config, CI job alongside the npm
|
|
32
|
+
checks.
|
|
33
|
+
2. **Bundle module** — serde types mirroring `docs/run-bundles.md`, bundle
|
|
34
|
+
reader (manifest-first, schema check, skip unknown), incremental NDJSON
|
|
35
|
+
tailer tolerating torn lines, runs-directory watcher with polling
|
|
36
|
+
fallback, artifact resolution with path containment.
|
|
37
|
+
3. **Layout port** — `layout/` reproduces `layoutGraph` exactly; parity test
|
|
38
|
+
deserializes every `fixtures/layout/*.json` and compares ranks, edges,
|
|
39
|
+
segments, and node ranks.
|
|
40
|
+
4. **Render port** — cell canvas + boxed/line graph renderer; parity test
|
|
41
|
+
compares ANSI-stripped frames against fixture `frames`.
|
|
42
|
+
5. **Source + protocol** — `RunSource` builds run views, computes revisioned
|
|
43
|
+
JSON Patch+ diffs, serves them over WebSocket (`piw serve`); client mode
|
|
44
|
+
applies them. Round-trip test: filesystem view == connected view.
|
|
45
|
+
6. **TUI shell** — runs sidebar, graph pane with camera (follow/overview),
|
|
46
|
+
inspector tabs, status line; live mode follows patches.
|
|
47
|
+
7. **Replay transport** — seq-based scrubber, play/pause with speed, step
|
|
48
|
+
keys, detach-on-rewind, jump-to-live; per-position state reconstruction
|
|
49
|
+
from the event log.
|
|
50
|
+
8. **Conversation pane** — render recorded session entries, progressive
|
|
51
|
+
reveal by replay position, attempt-slice highlighting via `conversation`
|
|
52
|
+
ranges, artifact-backed content on demand.
|
|
53
|
+
9. **Polish** — mouse drag/wheel, zoom density levels, keybinding help,
|
|
54
|
+
terminal snapshot tests.
|
|
55
|
+
|
|
56
|
+
## Testing
|
|
57
|
+
|
|
58
|
+
- Golden parity against `fixtures/layout/` (regenerate with `npm run
|
|
59
|
+
fixtures`; update both implementations together).
|
|
60
|
+
- Tailer: torn lines, appends across reads, truncation detection.
|
|
61
|
+
- Protocol: snapshot+patch reconstruction equals direct view; revision gap
|
|
62
|
+
forces resnapshot.
|
|
63
|
+
- End-to-end: run a workflow via the engine test harness, watch the bundle
|
|
64
|
+
with the Rust source, assert the final view matches the bundle documents.
|