pi-worker-graph 0.1.0-dev.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (67) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +455 -0
  3. package/SECURITY.md +54 -0
  4. package/dist/config.d.ts +25 -0
  5. package/dist/config.d.ts.map +1 -0
  6. package/dist/config.js +181 -0
  7. package/dist/config.js.map +1 -0
  8. package/dist/context.d.ts +22 -0
  9. package/dist/context.d.ts.map +1 -0
  10. package/dist/context.js +81 -0
  11. package/dist/context.js.map +1 -0
  12. package/dist/coordination.d.ts +19 -0
  13. package/dist/coordination.d.ts.map +1 -0
  14. package/dist/coordination.js +267 -0
  15. package/dist/coordination.js.map +1 -0
  16. package/dist/execution-failure.d.ts +42 -0
  17. package/dist/execution-failure.d.ts.map +1 -0
  18. package/dist/execution-failure.js +90 -0
  19. package/dist/execution-failure.js.map +1 -0
  20. package/dist/extension.d.ts +8 -0
  21. package/dist/extension.d.ts.map +1 -0
  22. package/dist/extension.js +641 -0
  23. package/dist/extension.js.map +1 -0
  24. package/dist/graph.d.ts +44 -0
  25. package/dist/graph.d.ts.map +1 -0
  26. package/dist/graph.js +292 -0
  27. package/dist/graph.js.map +1 -0
  28. package/dist/index.d.ts +16 -0
  29. package/dist/index.d.ts.map +1 -0
  30. package/dist/index.js +8 -0
  31. package/dist/index.js.map +1 -0
  32. package/dist/json.d.ts +9 -0
  33. package/dist/json.d.ts.map +1 -0
  34. package/dist/json.js +66 -0
  35. package/dist/json.js.map +1 -0
  36. package/dist/orchestrator.d.ts +15 -0
  37. package/dist/orchestrator.d.ts.map +1 -0
  38. package/dist/orchestrator.js +473 -0
  39. package/dist/orchestrator.js.map +1 -0
  40. package/dist/output.d.ts +61 -0
  41. package/dist/output.d.ts.map +1 -0
  42. package/dist/output.js +248 -0
  43. package/dist/output.js.map +1 -0
  44. package/dist/pi-subprocess.d.ts +92 -0
  45. package/dist/pi-subprocess.d.ts.map +1 -0
  46. package/dist/pi-subprocess.js +897 -0
  47. package/dist/pi-subprocess.js.map +1 -0
  48. package/dist/run.d.ts +89 -0
  49. package/dist/run.d.ts.map +1 -0
  50. package/dist/run.js +562 -0
  51. package/dist/run.js.map +1 -0
  52. package/dist/store.d.ts +331 -0
  53. package/dist/store.d.ts.map +1 -0
  54. package/dist/store.js +1993 -0
  55. package/dist/store.js.map +1 -0
  56. package/dist/usage.d.ts +35 -0
  57. package/dist/usage.d.ts.map +1 -0
  58. package/dist/usage.js +88 -0
  59. package/dist/usage.js.map +1 -0
  60. package/docs/DECISIONS.md +221 -0
  61. package/docs/DESIGN.md +392 -0
  62. package/docs/NEXT.md +229 -0
  63. package/docs/PLAN.md +203 -0
  64. package/docs/worker-graph.example.json +12 -0
  65. package/extensions/index.ts +1 -0
  66. package/extensions/tsconfig.json +11 -0
  67. package/package.json +66 -0
package/docs/DESIGN.md ADDED
@@ -0,0 +1,392 @@
1
+ # Design
2
+
3
+ ## Purpose
4
+
5
+ `pi-worker-graph` lets one Pi orchestrator execute a dependency graph of writable
6
+ workers against one shared checkout. It minimizes context sharing: immutable edge
7
+ outputs carry planned dependencies, while a small journal carries unexpected
8
+ coordination facts.
9
+
10
+ The graph domain is implemented independently of Pi. Process execution,
11
+ persistence, tools, and user interface behavior are later integration layers.
12
+
13
+ ## Runtime roles
14
+
15
+ ### Orchestrator
16
+
17
+ The parent Pi session:
18
+
19
+ - understands the task and repository constraints;
20
+ - defines nodes and dependency edges;
21
+ - decides which ready nodes may safely overlap;
22
+ - receives graph progress and structured results;
23
+ - reviews the integrated checkout with read-only tools;
24
+ - commissions final validation as dependent tasks and weighs what they report,
25
+ because the mode suppresses the parent's own command execution;
26
+ - delegates repairs rather than editing while worker-graph mode is active.
27
+
28
+ ### Worker
29
+
30
+ A child Pi session:
31
+
32
+ - receives one bounded assignment;
33
+ - operates directly in the shared checkout;
34
+ - receives outputs only from its direct prerequisites;
35
+ - may publish run-scoped coordination events or messages;
36
+ - reports a structured result;
37
+ - cannot spawn another worker.
38
+
39
+ ## Graph domain
40
+
41
+ The public graph module accepts arbitrary payloads:
42
+
43
+ ```ts
44
+ interface GraphTask<TPayload = unknown> {
45
+ id: string;
46
+ needs?: readonly string[];
47
+ payload?: TPayload;
48
+ }
49
+
50
+ interface GraphRequest<TPayload = unknown> {
51
+ tasks: readonly GraphTask<TPayload>[];
52
+ concurrency?: number;
53
+ }
54
+ ```
55
+
56
+ Normalization trims IDs, deduplicates and sorts dependency lists, and preserves
57
+ task order and payload identity. Validation completes before execution:
58
+
59
+ - IDs are non-empty and unique after normalization.
60
+ - Every dependency names a node in the request.
61
+ - A node cannot depend on itself.
62
+ - The graph is acyclic.
63
+ - Concurrency, when provided, is a positive integer.
64
+
65
+ Executor-specific schemas add assignment text, working directory policy, expected
66
+ paths, acceptance criteria, profiles, and hard resource limits. The runtime must
67
+ apply graph-size and dependency-count limits before invoking the graph domain.
68
+
69
+ ## Node states and scheduling
70
+
71
+ Node states are:
72
+
73
+ ```text
74
+ pending -> running -> succeeded
75
+ | -> failed
76
+ | -> aborted
77
+ -> failed
78
+ -> aborted
79
+ -> blocked
80
+ ```
81
+
82
+ Terminal states do not transition. A task can fail or abort before reaching
83
+ `running` when startup fails or cancellation arrives before execution.
84
+
85
+ Scheduling uses deterministic topological frontiers:
86
+
87
+ 1. Find pending nodes whose direct prerequisites all succeeded.
88
+ 2. Start eligible nodes up to the configured concurrency limit.
89
+ 3. Persist each terminal result.
90
+ 4. Mark pending nodes blocked when any prerequisite failed, aborted, or was
91
+ blocked.
92
+ 5. Recompute the ready frontier.
93
+ 6. Finish when every node is succeeded, failed, aborted, or blocked.
94
+
95
+ A graph with no edges is an ordinary parallel batch. A linear graph is a chain.
96
+ The pure graph module returns every eligible node in a deterministic order; the
97
+ executor applies the configured concurrency limit when selecting work to start.
98
+ The graph module does not spawn workers.
99
+
100
+ ## Edge context
101
+
102
+ Each worker reports a compact result:
103
+
104
+ ```ts
105
+ interface NodeOutput {
106
+ schemaVersion: 1;
107
+ summary: string;
108
+ changedFiles: Array<{ path: string; description: string }>;
109
+ interfaces: string[];
110
+ decisions: string[];
111
+ validation: Array<{ command: string; result: string }>;
112
+ blockers: string[];
113
+ }
114
+ ```
115
+
116
+ The report schema is versioned independently of the enclosing filesystem record.
117
+ Reports require exactly these fields, use non-empty bounded strings, limit each
118
+ section's item count, and have a hard aggregate UTF-8 JSON size limit.
119
+ Changed-file paths use normalized repository-relative form; they are report data,
120
+ not authority to access a path. Untyped executor values are validated and copied
121
+ into immutable snapshots before publication. Malformed or oversized reports fail
122
+ the node. A valid report with one or more blockers also fails the node, but is
123
+ retained for parent review; its dependents do not run.
124
+
125
+ For a node, outputs from each task named directly in `needs` are serialized in
126
+ task-ID order into canonical named JSON blocks for the execution adapter to
127
+ prepend to its assignment. Worker-authored text is explicitly marked as
128
+ untrusted report data within those blocks. Full transcripts and undeclared report
129
+ fields are not propagated. The complete serialization, including block labels
130
+ and the warning, is measured in UTF-8 bytes against a hard limit. Overflow fails
131
+ the downstream node without truncation, and stays that way: truncating it would
132
+ run a dependent worker against an incomplete prerequisite contract, and a
133
+ retained artifact cannot repair that because artifacts never cross a dependency
134
+ edge (D19).
135
+
136
+ Persisted output records use their own envelope schema version, independently of
137
+ the worker-report schema version. Current envelopes include run and task identity,
138
+ attempt, completion time, status, bounded diagnostics, a reference to a
139
+ retained text artifact when there is one, and what the attempt spent.
140
+
141
+ Usage is recorded for every terminal status, aborted included: an attempt the
142
+ run stopped had still spent what it spent by then. A timeout and an abort are
143
+ decided by the runner, which discards whatever the executor eventually settles
144
+ with, so usage the executor reported is carried onto the runner's own outcome
145
+ rather than lost with it. Failures carry usage too, on
146
+ `TaskExecutionFailure`. An executor that does not account for its own spend
147
+ reports nothing rather than zeros, which would read as work that cost nothing.
148
+
149
+ Zero spend and unknown spend are different answers. An executor whose
150
+ telemetry was absent, or carried a field that was not a usable number, records
151
+ no usage rather than zeros: a complete-looking zero would read as work that
152
+ cost nothing. An absent optional field is still zero, because a provider that
153
+ bills no cache write reports none.
154
+
155
+ A run's total is derived from the outputs it published rather than stored
156
+ alongside them: derived is correct for a run that was interrupted, and needs no
157
+ summary kept in step. Reading it applies the same output-to-node-state
158
+ agreement every other reader requires, so a total is never assembled from
159
+ records the rest of the store refuses; only a task that published no output at
160
+ all is an expected absence. Such a task is reported as unaccounted when it ran,
161
+ and as not started when it did not — the first is a gap in the accounting, the
162
+ second is not.
163
+
164
+ Token counts come from the provider's telemetry and cost from the runtime's
165
+ pricing of those tokens, so cost is an estimate and tokens are the sturdier
166
+ number.
167
+
168
+ A retained text artifact is bounded long-form text that does not belong in the
169
+ structured report — a log, investigation notes, detailed review findings. A
170
+ worker publishes one deliberately, through an optional `artifact` field on the
171
+ final-report tool; it is never runtime spillover from a report that did not
172
+ fit. The runtime never parses it and never propagates it over a dependency
173
+ edge, so a report that leans on its artifact is an incomplete report. It is published under the same mutation lock as the output
174
+ that references it, and before that output, so an interruption can only strand
175
+ an artifact no output claims. An artifact is therefore readable only through
176
+ its reference, which names the byte length the artifact must still have. An
177
+ aborted task produced nothing to retain and may not publish one. Artifacts are
178
+ records with the same identity envelope as every other stored record, rather
179
+ than bare text files, so an artifact cannot be read as belonging to a run, task,
180
+ or attempt other than the one that wrote it. They are removed with their run.
181
+
182
+ ## Coordination journal
183
+
184
+ Journal events are for information discovered after scheduling:
185
+
186
+ ```ts
187
+ type EventKind =
188
+ | "decision"
189
+ | "interface"
190
+ | "risk"
191
+ | "conflict"
192
+ | "handoff"
193
+ | "progress";
194
+
195
+ interface RunEvent {
196
+ schemaVersion: 1;
197
+ kind: "run-event";
198
+ eventId: string;
199
+ runId: string;
200
+ taskId: string;
201
+ timestamp: string;
202
+ eventKind: EventKind;
203
+ message: string;
204
+ paths?: string[];
205
+ symbols?: string[];
206
+ recipients?: string[];
207
+ }
208
+ ```
209
+
210
+ Directed messages carry the same envelope with a sender and one recipient task.
211
+
212
+ Workers explicitly query relevant entries by recipient, path, symbol, or cursor.
213
+ The runtime does not inject the entire journal into every turn. Live steering is
214
+ outside the MVP.
215
+
216
+ Every record in a run shares one monotonic sequence, so an identifier is also a
217
+ position: a cursor names a point that no later record can precede, and a worker
218
+ polling with one cannot silently skip a record published between two reads. A
219
+ publisher holds the run mutation lock while checking the active owner and
220
+ linking its record, so ownership release cannot land between validation and
221
+ publication. Within the critical section, the record itself is exclusively
222
+ created under the first free identifier; an interrupted publisher leaves no
223
+ identifier behind.
224
+
225
+ The orchestrator and every worker of a run contend for that one lock, so the
226
+ parent has to tell a lock a live worker holds from one a worker was killed
227
+ while holding. It is not a judgement about elapsed time: the lock is claimed by
228
+ linking a record that is already complete on disk, so it never exists without
229
+ naming its holder, and contention is reported apart from an ownership conflict.
230
+ A parent mutation that meets a lock waits it out, and recovers it only when the
231
+ lock names a task of its own graph whose promise has already settled. Its own
232
+ task the graph runner knows to be finished; a sibling still running is waited
233
+ for rather than interrupted, so ordinary contention no longer costs a graph.
234
+ The runner recovers whatever remains before releasing ownership, once every
235
+ task has settled.
236
+
237
+ Coordination is bounded on both sides. A run retains a fixed maximum number of
238
+ records, and publishing past it fails explicitly instead of silently degrading
239
+ reads. One record is bounded as a whole when it is published, at half a page, so
240
+ field bounds cannot combine into a record larger than a page. A read is bounded
241
+ by a requested record count and by the serialized JSON array size, including its
242
+ brackets and separators, and reports a cursor for the remainder, so no record
243
+ can enlarge a worker's context beyond the page bound.
244
+
245
+ One journal holds both kinds, so a read passes over records it will never
246
+ return: the other kind, and another task's mail. Its cursor runs past those as
247
+ well as past what it delivered, so a polling worker pays for each record once
248
+ rather than re-reading the journal on every call. Only a record examined and
249
+ held back — the one that overflowed the page — stays ahead of the cursor. A
250
+ cursor consequently belongs to the query that produced it; reused under a
251
+ different filter or recipient it starts past records that filter would have
252
+ matched.
253
+
254
+ Publishing requires the run to have an active owner, but not the owner's
255
+ capability. Workers are the authors of coordination records and never hold the
256
+ capability that advances node state, so an unowned or finished run stays
257
+ immutable while worker writes remain unprivileged. Records are attributed to a
258
+ task rather than authenticated: every worker of a run shares this state root, as
259
+ they already share the checkout.
260
+
261
+ ## Persistence
262
+
263
+ The state root is resolved from runtime configuration and defaults beneath Pi's
264
+ agent data directory. It never defaults inside the target checkout. A bounded
265
+ run count is enforced through atomically created run reservations. Reaching the
266
+ limit fails before a worker starts and never deletes prior state implicitly.
267
+
268
+ Per-run layout:
269
+
270
+ ```text
271
+ <state-root>/runs/<run-id>/
272
+ run.json immutable graph and configuration
273
+ owner.json exclusive active lifecycle owner
274
+ nodes/<task-key>.json parent-owned current node state
275
+ outputs/<task-key>.json terminal structured output
276
+ artifacts/<task-key>.json bounded retained text for one task attempt
277
+ coordination/<id>.json immutable events and directed messages, in one
278
+ run-global sequence
279
+ sessions/<task-key>/... optional child session state
280
+ ```
281
+
282
+ Task IDs are validated or encoded before use as path components. Immutable
283
+ records are published by writing a same-directory temporary file with restrictive
284
+ permissions and atomically renaming it. Mutable parent-owned state is replaced
285
+ atomically. One extension instance permits only one graph lifecycle at a time.
286
+ Active graph execution claims an exclusive owner record before mutating node
287
+ state or outputs. Each mutation also holds a per-run filesystem lock through its
288
+ full asynchronous read/validate/commit sequence, so ownership cannot be
289
+ released and reassigned mid-mutation. Resumable or externally addressable runs
290
+ still require an API that acquires and verifies that ownership before advancing
291
+ an existing run.
292
+
293
+ ## Child process contract
294
+
295
+ Every worker receives explicit run and task identity plus the state directory,
296
+ and no other runtime state; in particular it never receives the run's ownership
297
+ capability. The child detects worker mode and registers coordination and
298
+ reporting tools, but not graph-spawning tools or orchestrator commands. Pi's
299
+ tool allowlist is strict over built-in and extension tools, so the adapter
300
+ allowlists the coordination tools on exactly the condition under which the child
301
+ registers them, and advertises them in the assignment only then. Secrets are not
302
+ added to child arguments, prompts, environment metadata persisted by the
303
+ runtime, or graph state.
304
+
305
+ The MVP transport is a one-shot Pi JSON-mode subprocess behind the execution
306
+ adapter. It runs without session persistence or discovered extensions, skills,
307
+ and prompt templates; only an explicit built-in tool allowlist and the child
308
+ report extension are active. Repository context files remain enabled as trusted
309
+ worker instructions. Task content is sent over stdin rather than argv.
310
+ The adapter owns process-group cancellation, forced cleanup, event-stream
311
+ framing bounds, and bounded exit classification. It projects capped progress
312
+ snapshots containing only task identity, phase, allowlisted tool names, and
313
+ numeric usage. Worker text and tool payloads are not forwarded to the parent.
314
+
315
+ Classification favours completed work over incidental process noise. A captured,
316
+ validated report outranks a provider error or a nonzero exit reported after it,
317
+ because the report is the task contract and the child is already finished by
318
+ then. Cancellation and a distrusted event stream still outrank a report, since
319
+ neither leaves it trustworthy. Bounds that protect the parent are framing bounds:
320
+ an unparseable line is skipped, never fatal, so a worker that legitimately emits
321
+ a large transcript is not failed after its edits have landed.
322
+
323
+ A report only counts when it was genuinely the worker's last action. Pi batches
324
+ the tool calls of one assistant message and honours a terminating result only
325
+ when the whole batch terminates, so the adapter reconstructs the batch the
326
+ report belonged to from the assistant `message_end` event and watches for tool
327
+ executions after it. A report that shared its batch, or that work outlived, is
328
+ rejected rather than reported as success.
329
+
330
+ The worker executable is identified positively rather than inferred. The adapter
331
+ resolves Pi's CLI entry point through this package's dependency on Pi and runs it
332
+ with the current JavaScript runtime, so there is no `PATH` search and no command
333
+ interpreter on any platform. `PI_CODING_AGENT` is not evidence of identity — Pi
334
+ exports it into every process it starts, including programs run by its own `bash`
335
+ tool — so it only corroborates the single-file-build case, where the running
336
+ executable is neither `node` nor `bun`. When neither route identifies Pi, the
337
+ adapter requires an explicit command instead of guessing through a shell.
338
+
339
+ ## Mode lifecycle
340
+
341
+ The extension provides:
342
+
343
+ ```text
344
+ /swarm on
345
+ /swarm status
346
+ /swarm off
347
+ ```
348
+
349
+ and an opt-in startup flag. The parent `worker_graph` tool is registered once but
350
+ removed from the active tool set until the mode is enabled. Worker children take
351
+ a mutually exclusive extension path and receive only their final-report tool.
352
+
353
+ Entering the mode snapshots the complete active-tool set, removes the built-in
354
+ `bash`, `edit`, and `write` tools, and activates `worker_graph` in their place.
355
+ The snapshot and enabled state are stored in a bounded custom Pi session entry,
356
+ restored on resume and tree navigation, and restored exactly when the mode ends.
357
+ Worker children never register this lifecycle.
358
+
359
+ ## Failure semantics
360
+
361
+ - Invalid graph: reject before spawning.
362
+ - Worker startup, provider, or tool failure: mark the node failed with bounded
363
+ diagnostics.
364
+ - Failed or aborted prerequisite: block dependents without running them.
365
+ - Abort: terminate running children and settle remaining nodes as aborted or
366
+ blocked.
367
+ - Journal failure: report it; never silently claim coordination succeeded.
368
+ - Worker-report overflow: reject the report so the worker can correct and
369
+ resubmit it; compact parent review projections mark every truncated field or
370
+ omitted report explicitly. Truncating a report is the runtime choosing which
371
+ structured fields to cut, so it does not do that (D19).
372
+ - Process crash: persisted terminal nodes remain terminal; running nodes become
373
+ interrupted and require an explicit retry or recovery decision.
374
+
375
+ `interrupted` is recovery metadata rather than a normal graph-domain terminal
376
+ state until retry semantics are defined.
377
+
378
+ ## Same-tree concurrency
379
+
380
+ The runtime does not claim to make simultaneous writes safe. It reduces risk
381
+ through:
382
+
383
+ - careful graph decomposition;
384
+ - optional advisory path and symbol claims;
385
+ - instructions to re-read files before editing;
386
+ - preference for exact edits over whole-file rewrites;
387
+ - avoiding broad formatter or generator operations during a parallel frontier;
388
+ - orchestrator review of the integrated checkout;
389
+ - deterministic repository checks before acceptance.
390
+
391
+ Cross-process file locking is not part of the initial design. It would introduce
392
+ hidden serialization without protecting semantic coupling across different files.
package/docs/NEXT.md ADDED
@@ -0,0 +1,229 @@
1
+ # Current development status
2
+
3
+ ## Implemented
4
+
5
+ The repository now has a small transport-independent core plus an initial Pi
6
+ adapter. Automated tests remain provider-free:
7
+
8
+ - normalized, fully validated DAGs with frozen graph structure;
9
+ - deterministic frontiers and guarded node transitions;
10
+ - failed and aborted dependency blocking;
11
+ - immutable versioned run manifests with opaque task storage keys;
12
+ - one bounded retained text artifact per task attempt, published deliberately
13
+ through an optional `worker_graph_report` field, revalidated at every
14
+ boundary it crosses, published before and vouched for by the output that
15
+ references it, kept off dependency edges, named by byte length in the
16
+ parent-facing result, and removed with its run;
17
+ - report and prerequisite-context overflow that stay fail-closed rather than
18
+ truncating into an artifact;
19
+ - per-attempt token and cost accounting persisted with the attempt, kept for a
20
+ failed, timed-out, or aborted attempt as well as a succeeded one, summed per
21
+ run through `/swarm usage` and `readRunUsage()`, and attributed per task in
22
+ the parent-facing result;
23
+ - accounting that keeps zero and unknown apart: absent or unusable telemetry
24
+ leaves an attempt unaccounted rather than free, a run total is only summed
25
+ from outputs that agree with node state, and a task that never ran is
26
+ reported apart from one that ran and recorded nothing;
27
+ - parent-owned node state and immutable terminal outputs;
28
+ - restrictive permissions and atomic filesystem publication;
29
+ - bounded UTF-8 JSON records with explicit read and identity errors;
30
+ - an injected asynchronous task-executor interface;
31
+ - a strict versioned structured worker-report contract;
32
+ - defensive report validation and immutable JSON snapshots;
33
+ - normalized repository-relative changed-file paths and bounded diagnostics;
34
+ - blocker-bearing report failure with retained parent-visible output;
35
+ - bounded concurrent DAG execution with deterministic frontier selection;
36
+ - durable terminal output before dependent activation;
37
+ - direct-prerequisite-only output propagation;
38
+ - deterministic task-ordered prerequisite report serialization;
39
+ - named JSON report blocks with explicit untrusted-data labeling;
40
+ - exact UTF-8 context accounting and fail-without-truncation overflow behavior;
41
+ - explicit provider/model/thinking/tool worker profiles with no model defaults;
42
+ - one-shot Pi JSON-mode subprocess workers in the target checkout;
43
+ - task delivery over stdin rather than child-process arguments;
44
+ - child isolation from discovered extensions, skills, prompt templates, and sessions;
45
+ - a child-only terminating structured final-report tool, with recoverable
46
+ rejection so a worker can correct and resubmit a report;
47
+ - enforcement that the report really was the worker's final action, covering
48
+ Pi's parallel tool batches;
49
+ - positive identification of Pi's executable, with no shell on any platform;
50
+ - event-stream framing bounds that skip unparseable lines instead of failing a
51
+ worker whose transcript is legitimately large;
52
+ - allowlisted adapter failure diagnostics, surfaced through one pre-run error
53
+ type and one execution failure type;
54
+ - process-group cancellation with forced termination fallback and post-exit
55
+ collection of orphaned grandchildren;
56
+ - sanitized executor failures, cancellation, and task timeouts;
57
+ - task, dependency, concurrency, payload, output, and context limits;
58
+ - bounded, redacted worker progress projection and aggregate usage accounting;
59
+ - a strict global `worker-graph.json` profile configuration;
60
+ - a default state root beneath Pi's agent directory, with checkout-local roots
61
+ rejected by the extension;
62
+ - one static, fully bounded `worker_graph` parent tool;
63
+ - explicit `/swarm on`, `/swarm status`, `/swarm off`, `/swarm usage`, and
64
+ `--swarm` activation;
65
+ - an optional configured orchestrator session model and thinking level, applied
66
+ on activation and restored on exit, refused unless the mode knows the
67
+ identifiers that restore it, the model is one Pi can find, and its provider
68
+ is authenticated, with a model the configuration stopped naming still put
69
+ back and a restore that cannot be performed reported rather than swallowed;
70
+ - session-persisted swarm mode with exact active-tool restoration and built-in
71
+ parent mutation tools suppressed while active, with branch-recorded state
72
+ governing session-tree navigation and activation refused for any tool set the
73
+ extension could not restore;
74
+ - compact, explicitly bounded worker-report projection for parent review and
75
+ focused follow-up graphs, delivered in a labeled block worker text cannot
76
+ close;
77
+ - a configurable retained-run cap enforced by atomically claimed capacity slots,
78
+ with deterministic arbitration between concurrent creators, no implicit
79
+ deletion, stranded slots reported for explicit removal, and inconsistent
80
+ run/slot state failing closed;
81
+ - explicit retention cleanup through `/swarm runs` and `/swarm delete`: a
82
+ listing of everything holding capacity, and deletion of a named run with its
83
+ slot, refused while an orchestrator holds the run and ordered so an
84
+ interruption can only strand a slot;
85
+ - immutable bounded run-scoped coordination events and directed inbox messages,
86
+ in one run-global journal, with cursor-based queries and child-only Pi tools;
87
+ - a run mutation lock that names its holder, so contention between the parent
88
+ and a publishing worker is reported apart from an ownership conflict, waited
89
+ out rather than resolved by force, and recovered only when the lock names a
90
+ task the graph runner knows has finished;
91
+ - coordination cursors that run past records a reader is never given, so
92
+ polling costs only the records published since the previous call;
93
+ - a verified npm artifact containing the compiled runtime, loaded from a clean
94
+ temporary install in offline Pi RPC mode;
95
+ - behavioral coverage using `node:test`, fakes, and temporary directories.
96
+
97
+ Normal parent sessions register the `/swarm` control command, but the
98
+ `worker_graph` tool is inactive until explicitly enabled. The report tool is
99
+ registered only inside explicitly marked worker children. Active graph
100
+ lifecycles claim an exclusive owner before mutating run state; resumable or
101
+ externally addressable runs are not implemented yet.
102
+
103
+ ## Verify the baseline
104
+
105
+ ```bash
106
+ npm install
107
+ npm run check
108
+ npm run build
109
+ cp docs/worker-graph.example.json ~/.pi/agent/worker-graph.json
110
+ pi -e .
111
+ ```
112
+
113
+ `npm run check` currently runs the adapter, configuration, context, extension,
114
+ graph, orchestrator, report, store, and runner suites. `npm run build` must run
115
+ before `pi -e .`, because `extensions/index.ts` re-exports the compiled entry
116
+ point from `dist/`. The configuration copy is required rather than optional:
117
+ the extension has no provider or model defaults, and `/swarm runs` and
118
+ `/swarm delete` resolve the state root through the same file, so a missing
119
+ `worker-graph.json` answers both subcommands with a configuration error.
120
+ Loading the package in Pi adds the `/swarm` control command, whose `runs` and
121
+ `delete` subcommands work whether or not the mode is enabled, but leaves the
122
+ parent tool set unchanged. The entry point registers
123
+ the worker report tool only when `PI_WORKER_GRAPH_ROLE=worker`, which the
124
+ parent sets on worker subprocesses and never on its own session. Active graph
125
+ lifecycles claim an exclusive owner record before mutating run state; resumable
126
+ or externally addressable runs are not implemented yet.
127
+
128
+ ## Next implementation slice
129
+
130
+ Prepare the vertical slice for a prerelease:
131
+
132
+ 1. Run an optional provider-backed smoke test with two independent workers and a
133
+ dependent validation node when provider-backed testing is desired; this is
134
+ intentionally skipped for the current local pass.
135
+ 2. Review, tag, and publish the npm prerelease through the maintainer-owned Git
136
+ and registry workflow.
137
+ 3. Type `/swarm runs` and `/swarm delete <run-id>` once in a live Pi session.
138
+ Their state-root resolution has now been exercised against a real agent
139
+ directory: a real `worker-graph.json` resolved the default
140
+ `~/.pi/agent/worker-graph` root, and a created run was listed with its slot
141
+ and then deleted, leaving `runs/slots` empty at mode `0700`. What remains
142
+ unverified is the command surface itself — argument parsing, the notified
143
+ text, and `ctx.cwd` as the working directory — because those need the
144
+ interactive session rather than a direct store call.
145
+ 4. Keep every automated path provider-free behind the existing fake subprocess
146
+ and injected orchestrator boundaries.
147
+ 5. Exercise the configured orchestrator model once in a live session. Its
148
+ activation, refusal, and restore paths are covered by fakes; what no test
149
+ can cover is Pi's own `setModel` against a real provider catalogue and real
150
+ authentication. Confirm specifically whether the shutdown restore lands:
151
+ `session_shutdown` awaits an asynchronous model change, and Pi does wait for
152
+ that handler on both quit paths: `interactive-mode.js` awaits
153
+ `runtimeHost.dispose()`, which awaits `emitSessionShutdownEvent`, which
154
+ awaits each handler in turn. It is not awaited on `emergencyTerminalExit`
155
+ or `uncaughtCrash`, where the terminal is already gone.
156
+
157
+ ## Deferred run-store work
158
+
159
+ Before a resumable or externally addressable run API is added, retain the
160
+ fail-closed ownership contract so two orchestrators cannot advance one run, and
161
+ the rule that a mutation lock is recovered only by a holder that can be shown
162
+ to have finished.
163
+
164
+ Retained text artifacts are complete end to end and their policy is settled in
165
+ D19: publication is deliberate, and neither overflow becomes truncation.
166
+
167
+ One question is left open deliberately. The orchestrator learns that an
168
+ artifact exists, and how large it is, but has no way to read it: the review
169
+ names `artifactBytes` and not a path, and no tool returns the text. A library
170
+ caller uses `readNodeArtifact()`. Giving the orchestrator the artifact's path
171
+ would make its own `read` tool sufficient, but that widens what the parent may
172
+ reach outside the checkout, so it is a decision rather than an addition.
173
+
174
+ ## Deferred worker instruction work
175
+
176
+ The worker concurrency contract is now sent in the worker prompt
177
+ (`src/pi-subprocess.ts`) and asserted by the adapter suite: stay inside the
178
+ assignment, prefer small exact edits, reconcile rather than restore a file to
179
+ the version first read, never run git restore/reset/checkout/stash/clean and
180
+ never commit, push, or branch, no repository-wide formatters or generators or
181
+ dependency updates without explicit ownership, re-read changed files before
182
+ reporting, and report an unclear semantic conflict as a blocker instead of
183
+ guessing. The orchestrator guidance in `src/orchestrator.ts` carries the
184
+ matching parent-side decomposition, overlap, serialization, and acceptance
185
+ rules.
186
+
187
+ What remains undecided is one report field. `NodeOutput` carries `summary`,
188
+ `changedFiles`, `interfaces`, `decisions`, `validation`, and `blockers`
189
+ (`src/output.ts`), so an observation about another worker's edits can only
190
+ reach the parent's terminal report as prose in `summary` or as a blocker.
191
+ Adding a field is a schema version change.
192
+
193
+ It is less pressing than it was. A worker that notices a concurrent change
194
+ already has a structured channel while it runs — a `conflict` coordination
195
+ event (`src/store.ts`), which the parent and other workers can read — and a
196
+ retained artifact for the long form of what it saw. Neither is the terminal
197
+ report, so the question stands; it is no longer a dead end.
198
+
199
+ ## Deferred budget work
200
+
201
+ Usage is now recorded but nothing acts on it. The runtime bounds tasks,
202
+ concurrency, payload, output, context, and per-task runtime; it has no token or
203
+ cost ceiling, so a graph can spend without limit as long as each worker stays
204
+ inside its timeout.
205
+
206
+ The next slice is a configured budget in `worker-graph.json`, checked in the
207
+ runner between frontiers and enforced through the existing abort path, which
208
+ already settles remaining nodes and returns a result with usage intact. Two
209
+ things to settle first: whether the budget counts tokens or cost — cost is the
210
+ provider's estimate, tokens are the sturdy number — and whether crossing it
211
+ aborts the graph or only refuses to open the next frontier.
212
+
213
+ ## Constraints to preserve
214
+
215
+ - Keep the implementation and exported API as small as possible.
216
+ - Validate the complete graph and runtime bounds before starting work.
217
+ - Do not add provider or model defaults.
218
+ - Do not add automatic Git operations or worktrees.
219
+ - Do not store runtime state in the target checkout by default.
220
+ - Do not propagate undeclared or unbounded context between tasks.
221
+ - Do not add recursive worker delegation.
222
+ - Use fake executors for automated integration tests.
223
+
224
+ ## Decisions still needed
225
+
226
+ Review barriers, retention cleanup, and worker-session resumption semantics
227
+ remain deferred until their runtime layers are implemented. Broader Pi
228
+ compatibility can be claimed only after testing versions beyond the current
229
+ 0.85.1 development pin.