@earendil-works/pi-durable 0.99.1 → 0.99.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (172) hide show
  1. package/README.md +435 -46
  2. package/dist/entries.d.ts +20 -2
  3. package/dist/entries.d.ts.map +1 -1
  4. package/dist/entries.js +14 -1
  5. package/dist/entries.js.map +1 -1
  6. package/dist/env/index.d.ts +13 -38
  7. package/dist/env/index.d.ts.map +1 -1
  8. package/dist/env/index.js +2 -0
  9. package/dist/env/index.js.map +1 -1
  10. package/dist/env/node.d.ts.map +1 -1
  11. package/dist/env/node.js +51 -58
  12. package/dist/env/node.js.map +1 -1
  13. package/dist/harness/compaction.d.ts +56 -0
  14. package/dist/harness/compaction.d.ts.map +1 -0
  15. package/dist/harness/compaction.js +385 -0
  16. package/dist/harness/compaction.js.map +1 -0
  17. package/dist/harness/config.d.ts +10 -1
  18. package/dist/harness/config.d.ts.map +1 -1
  19. package/dist/harness/config.js +6 -0
  20. package/dist/harness/config.js.map +1 -1
  21. package/dist/harness/context.d.ts +8 -0
  22. package/dist/harness/context.d.ts.map +1 -1
  23. package/dist/harness/context.js +32 -21
  24. package/dist/harness/context.js.map +1 -1
  25. package/dist/harness/events.d.ts +166 -0
  26. package/dist/harness/events.d.ts.map +1 -0
  27. package/dist/harness/events.js +319 -0
  28. package/dist/harness/events.js.map +1 -0
  29. package/dist/harness/generation.d.ts +37 -6
  30. package/dist/harness/generation.d.ts.map +1 -1
  31. package/dist/harness/generation.js +350 -72
  32. package/dist/harness/generation.js.map +1 -1
  33. package/dist/harness/harness.d.ts.map +1 -1
  34. package/dist/harness/harness.js +124 -3
  35. package/dist/harness/harness.js.map +1 -1
  36. package/dist/harness/inbox.d.ts +57 -0
  37. package/dist/harness/inbox.d.ts.map +1 -0
  38. package/dist/harness/inbox.js +99 -0
  39. package/dist/harness/inbox.js.map +1 -0
  40. package/dist/harness/json.d.ts +10 -0
  41. package/dist/harness/json.d.ts.map +1 -0
  42. package/dist/harness/json.js +33 -0
  43. package/dist/harness/json.js.map +1 -0
  44. package/dist/harness/live.d.ts +66 -7
  45. package/dist/harness/live.d.ts.map +1 -1
  46. package/dist/harness/live.js +71 -7
  47. package/dist/harness/live.js.map +1 -1
  48. package/dist/harness/output.d.ts +62 -0
  49. package/dist/harness/output.d.ts.map +1 -0
  50. package/dist/harness/output.js +261 -0
  51. package/dist/harness/output.js.map +1 -0
  52. package/dist/harness/prompt.d.ts +10 -5
  53. package/dist/harness/prompt.d.ts.map +1 -1
  54. package/dist/harness/prompt.js +66 -13
  55. package/dist/harness/prompt.js.map +1 -1
  56. package/dist/harness/registry.d.ts.map +1 -1
  57. package/dist/harness/registry.js +11 -2
  58. package/dist/harness/registry.js.map +1 -1
  59. package/dist/harness/scheduler.d.ts +37 -12
  60. package/dist/harness/scheduler.d.ts.map +1 -1
  61. package/dist/harness/scheduler.js +632 -72
  62. package/dist/harness/scheduler.js.map +1 -1
  63. package/dist/harness/submissions.d.ts +12 -7
  64. package/dist/harness/submissions.d.ts.map +1 -1
  65. package/dist/harness/submissions.js +72 -33
  66. package/dist/harness/submissions.js.map +1 -1
  67. package/dist/harness/tool.d.ts +36 -0
  68. package/dist/harness/tool.d.ts.map +1 -0
  69. package/dist/harness/tool.js +412 -0
  70. package/dist/harness/tool.js.map +1 -0
  71. package/dist/harness/types.d.ts +185 -11
  72. package/dist/harness/types.d.ts.map +1 -1
  73. package/dist/harness/types.js.map +1 -1
  74. package/dist/harness/usage.d.ts +18 -0
  75. package/dist/harness/usage.d.ts.map +1 -0
  76. package/dist/harness/usage.js +57 -0
  77. package/dist/harness/usage.js.map +1 -0
  78. package/dist/harness/view.d.ts +43 -0
  79. package/dist/harness/view.d.ts.map +1 -0
  80. package/dist/harness/view.js +194 -0
  81. package/dist/harness/view.js.map +1 -0
  82. package/dist/index.d.ts +10 -4
  83. package/dist/index.d.ts.map +1 -1
  84. package/dist/index.js +6 -1
  85. package/dist/index.js.map +1 -1
  86. package/dist/session/observation.d.ts +18 -11
  87. package/dist/session/observation.d.ts.map +1 -1
  88. package/dist/session/observation.js +24 -13
  89. package/dist/session/observation.js.map +1 -1
  90. package/dist/session/session.d.ts +10 -1
  91. package/dist/session/session.d.ts.map +1 -1
  92. package/dist/session/session.js +18 -4
  93. package/dist/session/session.js.map +1 -1
  94. package/dist/session/transaction.d.ts +10 -1
  95. package/dist/session/transaction.d.ts.map +1 -1
  96. package/dist/session/transaction.js +83 -27
  97. package/dist/session/transaction.js.map +1 -1
  98. package/dist/storage/jsonl/storage.d.ts +22 -2
  99. package/dist/storage/jsonl/storage.d.ts.map +1 -1
  100. package/dist/storage/jsonl/storage.js +1 -1
  101. package/dist/storage/jsonl/storage.js.map +1 -1
  102. package/dist/storage/memory.js +1 -1
  103. package/dist/storage/memory.js.map +1 -1
  104. package/dist/storage/sqlite/migrations.js +1 -1
  105. package/dist/storage/sqlite/migrations.js.map +1 -1
  106. package/dist/testing/storage-benchmark.d.ts.map +1 -1
  107. package/dist/testing/storage-benchmark.js +0 -1
  108. package/dist/testing/storage-benchmark.js.map +1 -1
  109. package/dist/testing/storage-conformance.d.ts.map +1 -1
  110. package/dist/testing/storage-conformance.js +34 -2
  111. package/dist/testing/storage-conformance.js.map +1 -1
  112. package/dist/tools/bash.d.ts +28 -0
  113. package/dist/tools/bash.d.ts.map +1 -0
  114. package/dist/tools/bash.js +69 -0
  115. package/dist/tools/bash.js.map +1 -0
  116. package/dist/tools/edit-diff.d.ts +89 -0
  117. package/dist/tools/edit-diff.d.ts.map +1 -0
  118. package/dist/tools/edit-diff.js +386 -0
  119. package/dist/tools/edit-diff.js.map +1 -0
  120. package/dist/tools/edit.d.ts +18 -0
  121. package/dist/tools/edit.d.ts.map +1 -0
  122. package/dist/tools/edit.js +116 -0
  123. package/dist/tools/edit.js.map +1 -0
  124. package/dist/tools/env.d.ts +5 -0
  125. package/dist/tools/env.d.ts.map +1 -0
  126. package/dist/tools/env.js +7 -0
  127. package/dist/tools/env.js.map +1 -0
  128. package/dist/tools/file-mutation-queue.d.ts +5 -0
  129. package/dist/tools/file-mutation-queue.d.ts.map +1 -0
  130. package/dist/tools/file-mutation-queue.js +46 -0
  131. package/dist/tools/file-mutation-queue.js.map +1 -0
  132. package/dist/tools/image.d.ts +2 -0
  133. package/dist/tools/image.d.ts.map +1 -0
  134. package/dist/tools/image.js +92 -0
  135. package/dist/tools/image.js.map +1 -0
  136. package/dist/tools/index.d.ts +5 -0
  137. package/dist/tools/index.d.ts.map +1 -0
  138. package/dist/tools/index.js +5 -0
  139. package/dist/tools/index.js.map +1 -0
  140. package/dist/tools/path-utils.d.ts +5 -0
  141. package/dist/tools/path-utils.d.ts.map +1 -0
  142. package/dist/tools/path-utils.js +26 -0
  143. package/dist/tools/path-utils.js.map +1 -0
  144. package/dist/tools/read.d.ts +17 -0
  145. package/dist/tools/read.d.ts.map +1 -0
  146. package/dist/tools/read.js +100 -0
  147. package/dist/tools/read.js.map +1 -0
  148. package/dist/tools/write.d.ts +10 -0
  149. package/dist/tools/write.d.ts.map +1 -0
  150. package/dist/tools/write.js +30 -0
  151. package/dist/tools/write.js.map +1 -0
  152. package/dist/{env/utils/truncate.d.ts → truncate.d.ts} +1 -17
  153. package/dist/truncate.d.ts.map +1 -0
  154. package/dist/truncate.js +146 -0
  155. package/dist/truncate.js.map +1 -0
  156. package/dist/types.d.ts +86 -19
  157. package/dist/types.d.ts.map +1 -1
  158. package/dist/types.js.map +1 -1
  159. package/docs/pico-v5-handoff.md +366 -86
  160. package/docs/pico-v5.md +1373 -283
  161. package/package.json +11 -3
  162. package/dist/env/utils/adaptive-publisher.d.ts +0 -24
  163. package/dist/env/utils/adaptive-publisher.d.ts.map +0 -1
  164. package/dist/env/utils/adaptive-publisher.js +0 -79
  165. package/dist/env/utils/adaptive-publisher.js.map +0 -1
  166. package/dist/env/utils/output-capture.d.ts +0 -31
  167. package/dist/env/utils/output-capture.d.ts.map +0 -1
  168. package/dist/env/utils/output-capture.js +0 -238
  169. package/dist/env/utils/output-capture.js.map +0 -1
  170. package/dist/env/utils/truncate.d.ts.map +0 -1
  171. package/dist/env/utils/truncate.js +0 -294
  172. package/dist/env/utils/truncate.js.map +0 -1
@@ -11,7 +11,7 @@ facades, membranes, document routing, view projection, events, or clone chains.
11
11
 
12
12
  - Obsolete `pico` and `pico4` prototypes were removed.
13
13
  - `pico3` remains.
14
- - Packages 1–15 are implemented in `packages/durable`; Package 10 was already satisfied by Chord's canonical structural diff implementation.
14
+ - Packages 1–20 are implemented in `packages/durable`; Package 10 was already satisfied by Chord's canonical structural diff implementation.
15
15
 
16
16
  ## 1. Records, cursors, and memory tables
17
17
 
@@ -441,80 +441,138 @@ awaits its own input submission rather than global idle.
441
441
  ## 16. First coding-agent tool turn
442
442
 
443
443
  Implement hook dispatch and `TaskRuntime.hooks`, deferred from Package 14
444
- (Session-wide and scoped to a conversation or its owned subtree) and wire the real generation → tool tasks → post-tools → generation
445
- chain. Implement offered-set checks, tool pinning from the phase snapshot,
446
- declaration and argument validation against both the offered declaration and
447
- the pinned implementation, hook composition, durable execution intent, stored
448
- replay policy, bounded output/details presentation documents, result entries, post-tools
449
- joining, controls, and `postTools`/`final` boundaries. The generation task now
450
- classifies tool calls and continues through the real built-in task chain;
451
- neither side uses a production fake successor.
452
-
453
- Implement preparation behavior for tool loadout additions/removals, same-name
454
- replacement ordering, complete baseline tool declarations, wrapped tools and
455
- failing wrappers, and hook memos.
444
+ (Session-wide and scoped to a conversation or its owned subtree, §7.2), and wire
445
+ the real generation → tool tasks → post-tools → generation chain (§8.3–8.5).
446
+ Neither side uses a production fake successor.
447
+
448
+ - Generation: tool declarations in preparation (§7.4: additions/removals,
449
+ same-name replacement, order rewrite, complete baseline after a head cut,
450
+ duplicate active names offered once, wrapped and failing wrappers,
451
+ `PromptInput.tools`); `toolExecution` pinned in `request`/`poll`; the tool
452
+ round commit (offered check against the committed context through `cutoff`,
453
+ unaffected by `beforeRequest`; `tool_unavailable`
454
+ results for calls not offered, parallel or `after`-chained sequential tool
455
+ tasks, post-tools, `pi.live.tools`, run handed to post-tools);
456
+ `beforeRequest`, `afterResponse`, and `onYield` continuations.
457
+ - `pi.tool` (§8.4): input `{ assistant, callId }`, resolve/validate/
458
+ `beforeTool`/validate/intent/execute/`afterTool`/result in one `call` handler,
459
+ recovery-only `execute` with the stored/current replay rule, bounded output and
460
+ details in the `pi.live.tools` slot, the terminal commit as final flush,
461
+ diagnostics (`api.diagnostic()`, result diagnostics, Harness truncation and
462
+ error codes, the `<harness>` block, `pi.tool-result` `data: { diagnostics }`),
463
+ and the abort handler.
464
+ - `pi.post-tools` (§8.5): `afterTools`, `addTools`, all-results `terminate`, the
465
+ `postTools`/`final` boundaries (no inbox placement until Package 17), and the
466
+ next generation. The `handoff` control moves to Package 17 with `reset()`,
467
+ which defines the headed entry both write.
468
+ - `pi.live.tools` and the checkpoint rule (§8.2); the scheduler cleanup hook for
469
+ `pi.post-tools` (ends the run) and `pi.tool` (marks the slot `done`).
470
+ - Surface: `ToolExecutionApi` without `conversation()`, which the `src` type
471
+ drops until Package 18 adds it with `ConversationHandle`; `api.env` and `HarnessOptions.env`,
472
+ `ToolRegistration.executionMode`, the `toolExecution` configuration field with
473
+ its getter/setter, `TaskRuntime.getTask()` and `entry()`, `HookApi`, and the
474
+ exported `GenerationTask`, `ToolTask`, and `PostToolsTask` tokens.
475
+ - `@earendil-works/pi-durable/tools`: copy `read`, `bash`, `edit`, and `write`
476
+ with their helpers from `packages/agent/src/harness/tools` and adapt them to
477
+ `ToolRegistration` (`api.env`, `api.output`, `api.details`). Image reading in
478
+ `read` is deferred; note it where the tool rejects or skips images. The env
479
+ shell streams raw output (`onOutput`) and spills past thresholds; `output()` is
480
+ the only place output is bounded, sanitized, and throttled. `prepareArguments`
481
+ restores the edit tool's argument repair.
456
482
 
457
483
  Acceptance: input → model tool call → registered local read/bash/edit operation →
458
484
  tool result → model answer → durable submission settlement. Run that path once
459
485
  normally and once interrupted/reopened. Test recovery from every tool and post-
460
- tools phase; offered-history enforcement; before/after hook rules; both stored/
461
- current replay-policy directions; default and overridden `outputLimits`; output
462
- content fallback; details replacement, last-details fallback, and coalesced
463
- commit settlement; drain-
464
- before-terminal ordering; abort/close with buffered output; invocation-bound
465
- handles and watches; unregistered active tools that are removed from the offered set, re-added after re-registration, and produce `tool_unavailable` results when called, without failing the request; a tool replaced mid-call
466
- finishing under its pinned implementation, including across phase boundaries;
467
- hooks surviving a task reload; duplicate active names offered once; order-only tool changes; atomic assistant/tool/post-tools commits;
468
- and all positional tool-history cases.
486
+ tools phase; offered-set enforcement, including a tool deactivated after
487
+ preparation still executing; before/after hook rules, hook scopes, and hook
488
+ memos; `onYield` continuations; both stored/current replay-policy directions;
489
+ default and overridden `outputLimits`; output content fallback; details
490
+ replacement, last-details fallback, and coalesced commit settlement; the terminal
491
+ commit as final flush; abort/close with buffered output; invocation-bound
492
+ watches; parallel and sequential rounds, including a per-tool sequential mode;
493
+ unregistered active tools that are removed from the offered set, re-added after
494
+ re-registration, and produce `tool_unavailable` results when called, without
495
+ failing the request; a tool replaced mid-call finishing under the implementation
496
+ it resolved; hooks surviving a task reload; order-only tool changes; atomic
497
+ assistant/tool/post-tools commits; `terminate` only when every result requests
498
+ it; `addTools`; tool fault/orphan leaving a synthesized context result;
499
+ diagnostic ordering, the `<harness>` block, and entry `data`; and all
500
+ positional tool-history cases. Also test the ported tools against
501
+ `NodeExecutionEnv`, and a wrapper supplying a different `api.env`.
502
+
503
+ Document in the durable README that the package root loads TypeBox through the
504
+ tool task's argument validation (pi-ai `validateToolArguments()`): about 23 MB of
505
+ peak RSS unbundled, about 4 MB in a tree-shaken bundle.
506
+
507
+ Tool output benchmark (`test/*.bench.ts`, memory, SQLite, and JSONL): drive the
508
+ real tool task, adaptive throttle, and `pi.live` commits with low (a line every
509
+ few seconds), normal (a compiler or test run), and high (continuous `cat` of a
510
+ large file) output rates, head and tail retention, one tool and several parallel
511
+ tools, over one long round. Report commits, operation bytes written, stored
512
+ size before and after the round's base, reclamation, heap and RSS, commit
513
+ latency, and reopen/replay time mid-round. Assert that head output commits as
514
+ Chord appends and that a sliding tail of non-repetitive output within 64 KiB
515
+ commits as trim plus append; report how often repetitive output falls back to a
516
+ full window set. The checkpoint rule stays "base whenever nothing runs" (§8.2) with
517
+ no delta-count bound; the benchmark decides whether that holds. Assign
518
+ `slot.output` as one string field so Chord can diff it; replacing the slot
519
+ object records a full set.
520
+
521
+ Throughput target: a plain `cat` of a 1 GiB file of unique lines through the
522
+ ported bash tool and the real Harness path (env capture and spill, `api.output`,
523
+ throttled `pi.live` commits, result entry) completes in about 0.4 s, like the
524
+ mini coding agent. Also drive `api.output()` directly with the same 1 GiB in
525
+ 64 KiB chunks: accepting a chunk must not cost work proportional to the
526
+ retained window.
469
527
 
470
528
  ## 17. Live UI and product state
471
529
 
472
- Complete submissions and inbox behavior: busy `steer`/`followUp`/`reject`,
473
- passive writes, withdrawal, ordered `postTools`/`final` selection, stale targets,
474
- self-head cuts, successor turns, queued reset/handoff, and every terminal cleanup.
530
+ Complete submissions and the inbox (§6): the `pi.inbox` document created by the
531
+ built-in setup; busy `steer`/`followUp`/`reject`; queued writes; admission to an
532
+ idle conversation with a non-empty inbox (queue, then a final boundary in the
533
+ same commit); withdrawal removing the item; `tx.placeSubmission()`; ordered
534
+ `postTools`/`final` selection by the new `steeringMode`/`followUpMode`
535
+ configuration fields (with getters/setters), writes before user items; stale
536
+ head writes; a reset at `postTools` ending the run with `reset`; successor runs;
537
+ and `onYield` applying only when the final boundary selected no user item and no
538
+ reset. Only an answer, `terminate`, or `handoff` applies the final boundary;
539
+ failure, a run task's abort, fault, and orphan leave the inbox alone.
475
540
  Successful inputs still require an answer; writes settle on placement and never
476
- start generation.
477
-
478
- Add `harness.blockedTasks()`, deferred from Package 14: the pending tasks this
479
- process cannot run and why (`missing_task`, `task_too_old`, `migration_failed`
480
- with the stored and registered versions and the migration error). It is derived
481
- from task records, the current registry snapshot, and scheduler memory, and is
482
- never persisted.
483
-
484
- Specify and implement the Harness activity view:
485
- the active conversations, notifications when a conversation becomes active or
486
- idle, all derived from committed turn-control and task
487
- state.
488
-
489
- Define any remaining built-in preference/presentation documents once with final
490
- schemas. Implement the structural `{ conversation, entries, docs }`
491
- `ConversationView`, `viewState()`, and `watch()`. Build the first revision lazily
492
- on the Session line. For each affected commit, derive and prepare one mounted
493
- operation batch from the complete candidate transaction before Storage
494
- admission; after success, only install prepared pointers/cursors and enqueue the
495
- exact immutable frame. Do not derive the mount through `subscribeCommits()`.
496
-
497
- Add the §9.4 notification adapter directly from uncoalesced committed
498
- publication, without another tracker or persistence authority. Wire TUI
499
- hydration to structural state/watch, print to its own `Submission`, and JSON/RPC
500
- to correlated commands plus ordered committed notifications. Transport
501
- backpressure and disconnect policy stay in the mode adapter.
541
+ start generation by themselves.
542
+
543
+ Define the `pi.reset` entry (§8.1), written by `Conversation.reset(handoff)`
544
+ through a write submission and by the post-tools `handoff` control.
545
+
546
+ Usage (§8.6): the `pi.usage` document created by the built-in setup, updated in
547
+ the same commit as every assistant entry (`appendAssistant()`) and every tool
548
+ result with `usage` (`ToolExecutionResult.usage`, `appendToolResult()`), and
549
+ `Harness.usage()` summing every conversation's document.
550
+
551
+ `ConversationView`, `viewState()`, and `watch()` (§9.3): one mount per
552
+ conversation, built lazily on the Session line, advanced from
553
+ `subscribeCommits()` publications, and dropped with its last observer.
554
+
555
+ Last: the experimental `watchEvents()` adapter (§9.4) with the snapshot event,
556
+ translated message and tool deltas, and overflow-to-snapshot.
557
+
558
+ Examples next to the existing ones in `test/examples/`: a print demo awaiting
559
+ its own `Submission`, a JSON demo with two modes, `--events` (adapter
560
+ events as JSON lines) and `--ops` (raw `ConversationView` frames), an inbox
561
+ demo, and a late-join demo attaching a view and an event stream mid-run. An
562
+ interactive TUI demo and coding-agent integration are later work.
502
563
 
503
564
  Table-test every submission transition, cross-type request-ID conflicts,
504
- interleaved queue selection, compact positional removal of large payloads,
505
- abort results, reopen waits, writes pending without a later boundary, busy reset
506
- placement, and orphan/fault cleanup. Test one view publication per Session
507
- commit; atomic entry/preview settlement; parent-linked active entries and heads;
508
- mounted create/recreate/retire; preparation rollback before Storage; empty-batch
509
- suppression and redundant nonempty revisions; contiguous delivery; stable public
510
- paths; O(1) immutable acquisition; structural sharing; serialized consumers;
511
- bounded reset behind an in-flight callback; durable retry/tool/collapse status;
512
- output truncation metadata; the documented placement of diagnostics in entries,
513
- terminal details, or bounded state; asynchronous consumer initialization; and
514
- absence of semantic projection. Verify watch overflow cannot erase a separately
515
- subscribed notification lifecycle, late clients hydrate structurally, and
516
- notifications expose committed throttled
517
- progress rather than raw provider frames.
565
+ interleaved queue selection under both queue modes, writes placed before user
566
+ items, stale head writes, a reset at each boundary, handoff, `onYield` with and
567
+ without queued items, compact positional removal of large payloads, withdrawal,
568
+ reopen waits, queued items surviving a failed run and drained by the next
569
+ submission, and orphan/fault cleanup. Test minimal Chord deltas for `pi.inbox`
570
+ and `pi.usage` changes. Test one view revision per touching commit; atomic
571
+ entry/document publication; fork-aware active entries and head cuts; mounted
572
+ create/retire; empty-batch suppression; stable paths; structural sharing; watch
573
+ overflow; and dropping the mount. Test the event translation of every
574
+ `MessageChange` and output change, the snapshot at attachment and after
575
+ overflow, and absence of raw provider frames.
518
576
 
519
577
  ## 18. Ownership and subagents
520
578
 
@@ -525,6 +583,18 @@ waits. Finish invocation-bound owned APIs used by tools and the foreground and
525
583
  background subagent provisioning patterns, including atomic task/conversation/
526
584
  registry creation and request-ID-safe submission recovery.
527
585
 
586
+ An abort cascade withdraws queued inputs in every owned conversation it reaches
587
+ (spec §5.4). There is no full-teardown primitive: `close()` stops everything
588
+ without outcomes, and a host cancels everything by aborting what `inspect()`
589
+ lists. No built-in subagent tool or supervisor task: two concise, product-style
590
+ examples, `test/examples/22-subagent-foreground.ts` (child owned by the tool task,
591
+ reported through `api.details({ conversationId })`, the UI attaching to the
592
+ child's events) and `test/examples/23-subagent-background.ts` (reworked in
593
+ Package 19: persistent subagents behind one `subagent` tool with spawn, send,
594
+ wait, stop, and status actions; a background anchor task owns each child, and a
595
+ background reporter task per message delivers it and posts the answer back as a
596
+ follow-up input, request-ID-safe across restarts), show the patterns.
597
+
528
598
  Test deep ownership trees, owner edges after terminal settlement, nested
529
599
  background boundaries, conversation abort/join with surviving passive writes and
530
600
  background tasks, cancellation of waiters without cancellation of work, and
@@ -533,24 +603,226 @@ current committed tail, empty source conversations, document fork policies,
533
603
  configuration overrides through `init`, foreground subagent cascade, and
534
604
  background supervisor recovery before and after submission admission.
535
605
 
536
- ## 19. Collapse and overflow
537
-
538
- Implement manual, threshold, and generation-overflow collapse; exchange-boundary
539
- range selection; summarization; retry policy; staleness checks; and headed
540
- summary entries. Wire generation's real overflow path directly to the collapse
541
- task, and complete `Conversation.collapse()` so it returns the admitted task ID.
542
-
543
- Generation preparation no longer rechecks the transcript before appending its
544
- system entries (§7.4, §12): only run tasks and turn boundaries write to a busy
545
- conversation. A collapse summary for a busy conversation must therefore be
546
- placed at a turn boundary or as a step of the run, never appended concurrently.
547
-
548
- Test model context before and after collapse, raw history preservation, provider
549
- failure, declined and stale work, manual/threshold/overflow admission, late-join
550
- presentation state, and reopen from every phase. Rerun generation overflow
551
- integration without a fake collapse kind.
552
-
553
- ## 20. Reload and final conformance
606
+ ## 19. Structured concurrency
607
+
608
+ Implement spec §5.5 and its consequences, replacing `after` and `pi.post-tools`.
609
+ No backward compatibility.
610
+
611
+ - Ownership: `TaskOptions.ownership` is required (`{ kind: "conversation" }` or
612
+ `{ kind: "task", taskId }`); `TaskRecord.owner?: TaskId`; child tasks live in
613
+ their owner's conversation; `background` only for conversation-owned tasks;
614
+ new task- or conversation-ownership requires a live owner (not `completing`
615
+ or `terminal`). Update every `createTask` call site, including tools'
616
+ `api.createTask`, examples, and tests. Storage: persist `owner` and the new
617
+ statuses (memory, JSONL, SQLite schema edited in place; no migrations while WIP), conformance cases. No owner
618
+ index: the scheduler derives edges from the live tasks it loads.
619
+ - States: real `waiting { checkpoint, on, policy }` and `completing { outcome }`
620
+ statuses in `TaskState` and every backend and status scan; `NextTaskState`
621
+ gains `waiting`; `runtime.outcomes()`; `inspect()` reports `completing`.
622
+ Remove `after` (records, options, scheduler dependency handling).
623
+ - Scheduler: a waiting task is never reserved until every task in `on` is
624
+ terminal (then reserved directly from `waiting` to `running` at its
625
+ checkpoint), unless an abort mark lets it reach its abort handler; `on` over
626
+ tasks it does not own requires `allSettled`; `on` rejects missing tasks, the
627
+ task itself, and its owner chain; failFast marks live owned siblings in `on` on a held or terminal
628
+ non-`completed` outcome, in reconcile. Terminal commits (task- and
629
+ scheduler-written) with live ordinary owned work become `completing`; a later
630
+ scheduler commit finalizes them when that work is gone, re-evaluated on every
631
+ commit and at open; scheduler-written holds defer their Harness cleanup
632
+ (`settleSchedulerOutcome`) to the final commit, task-written holds land their
633
+ other writes at hold; waiters and task-document retirement at the final
634
+ commit. Abort invocations start only when ordinary owned work is gone
635
+ (bottom-up, judged on committed records). Orphaning happens only where the
636
+ abort invocation would start, or directly in `abortTask()` when nothing owned
637
+ is live, so orphans never hold. Cascade (spec §5.4): only live owners cascade (abort mark or held
638
+ non-`completed` outcome); terminal owners never do; ordinary traversal and
639
+ idle follow task→task edges too. `Conversation.abort(context, { background:
640
+ true })` snapshot semantics.
641
+ - Built-ins: generation owns its tool tasks and waits `allSettled` in a new
642
+ `tools` phase that runs the old post-tools body (§8.5); sequential rounds
643
+ create one tool at a time from `pending`; the next generation is
644
+ conversation-owned; the generation abort handler appends `aborted` results for
645
+ unstarted calls. Remove `pi.post-tools`, `PostToolsTask`, `PostToolsHooks`
646
+ (`afterTools` moves to `GenerationHooks`). Tool tasks that own conversations
647
+ hold `completing` after their result entry. Events: `turn_end` when a
648
+ generation's outcome is committed (hold or terminal, whichever first).
649
+ - Docs: spec §5.5 and §12 footguns are written; update README (task ownership,
650
+ waiting, `completing`, abort order), CHANGELOG, and examples 12/22/23 plus a new
651
+ `test/examples/24-child-tasks.ts` (checkout with four payments: failFast,
652
+ `abortTask(checkout)`, crash and reopen, each printing outcomes).
653
+
654
+ Exhaustive tests (new `test/harness-structured.test.ts` plus updates): ownership
655
+ validation (missing ownership, cross-conversation child, background child,
656
+ owner completing/terminal); waiting with allSettled and failFast (failed,
657
+ aborted, faulted, orphaned children; held failures trigger failFast before the
658
+ failing child drains); `on` with already-terminal and non-owned tasks
659
+ (`allSettled` only); waiting on subsets in sequence; spawning without waiting;
660
+ `completing` for task- and scheduler-written outcomes, including work created
661
+ during the hold, finalization at open, `abortTask()` on a completing task, held
662
+ failure aborting below, waiters and document retirement only at the final
663
+ commit; bottom-up abort order across three levels (child terminal records before
664
+ the parent's abort handler starts), including an aborted waiting parent; `on`
665
+ validation (missing, self, owner chain, empty `on`); a waiting task whose
666
+ definition is missing when it may resume (blocked, then orphaned when aborted);
667
+ creating owned work in a finishing commit (task ownership rejects, a task in an
668
+ owned conversation holds); cascades through
669
+ task→task and task→conversation→task edges; background boundaries and the
670
+ `{ background: true }` abort; a terminal owner never cascading (interrogating a
671
+ finished or Esc'd subagent runs normally); crash/reopen at every point (before
672
+ and after create+wait, after a child fails before siblings are marked, after all
673
+ children terminal before the parent resumes, during holds and abort handlers);
674
+ generation tool rounds parallel and sequential with Esc before, during, and
675
+ after tools, unstarted sequential calls, faulted tool slots under generation
676
+ abort, faulted generation keeping run control until its tools drain, `turn_end`
677
+ before the successor's `turn_start` when a generation holds, and a tool holding
678
+ `completing` while its subagent's extension work runs. Rerun every existing suite; migrate the post-tools,
679
+ sequential-round, and dependency tests to the new shapes without weakening them.
680
+
681
+ ## 20. Compaction and overflow
682
+
683
+ Implement spec §8.7 and its consequences. Compaction never appends to a busy
684
+ conversation concurrently (§7.4 preparation does not recheck the transcript): a
685
+ blocking compaction appends while its generation holds the run and waits for it;
686
+ every other compaction places its summary through a write submission.
687
+
688
+ - Task: `pi.compaction` (`CompactionTask`, `CompactionHooks`, `CompactionInput`,
689
+ `CompactionCheckpoint`, `CompactionResult`, `CompactionReason`) with phases
690
+ `select`, `summarize`, `retry`, the abort handler, and the `beforeCompact`
691
+ hook. Register it with the built-ins. `CompactionEntry` token for
692
+ `pi.compaction` with `data: { reason }`.
693
+ - `ContextView.contributions`: each active entry's contribution after all
694
+ in-range edits (including edits on older in-range markers). Range selection is one pure function over those
695
+ contributions and `keepRecentTokens`, shared by the generation's checks and
696
+ `select`; the summarized messages are the prefix contributions ordered by
697
+ §2.1 rules 7–8. The generation's size estimate (§8.3) uses the newest
698
+ assistant appended after the head marker, not pi-ai's timestamp heuristic. Port the coding agent's serializer, summarization
699
+ system prompt, and structured prompt (one prompt, with a line about carrying an
700
+ earlier summary forward) into `src/harness/compaction.ts`; do not import from
701
+ coding-agent.
702
+ - Placement: direct append for a task-owned (blocking) compaction; for a
703
+ conversation-owned one, factor the admission body of `Submissions.submit()` into
704
+ a function over a `Tx` so the task's classifying commit admits the write
705
+ (request ID `compaction:<taskId>`) with the same idle/queued/stale/final-boundary
706
+ behavior. No new stale logic.
707
+ - Config: `CompactionPolicy`, `DEFAULT_COMPACTION_POLICY`,
708
+ `ConversationConfigState.compaction`, `getCompaction`/`setCompaction`.
709
+ - Live: `pi.live.compactions` (`CompactionStatus[]`, task ID order, key removed
710
+ when empty), added in the creating commit and removed in the outcome commit
711
+ (also at a `completing` hold);
712
+ `settleSchedulerOutcome` removes it for faulted/orphaned compactions.
713
+ - Generation: `compacted`/`overflow` checkpoint fields; threshold checks in
714
+ `prepare` (blocking child + wait; background conversation-owned task in the
715
+ commit moving to `request`); overflow classification before the retry branch;
716
+ failing an overflow generation whose compaction produced no entry.
717
+ - `Conversation.compact()` admits the manual task (conversation-owned, not
718
+ background) with its status in one commit and returns its ID.
719
+ - Usage: every classified summarization response adds its usage to
720
+ `pi.usage.models`.
721
+ - Events: `compaction_start`/`compaction_end`, snapshot `compactions`, batch
722
+ order per §9.4.
723
+ - Docs: README (compaction, policy, events), CHANGELOG, and a new product-style
724
+ example `test/examples/25-compaction.ts` (faux model: a long chat that crosses
725
+ the background threshold, the summary lands at a boundary, a manual `compact()`
726
+ while busy, and an overflow recovered by a blocking compaction, printing the
727
+ model context before and after).
728
+
729
+ Not in v1 (note only): a cache-friendly summary request that resends the
730
+ agent's exact last request plus one summarization user message with pi-ai
731
+ `toolChoice: "none"`, so the prefix including tool declarations is a cache hit.
732
+ It fits background compaction well below the window; overflow and a context
733
+ without room for the prompt and summary keep the serialized request.
734
+
735
+ Exhaustive tests (new `test/harness-compaction.test.ts`, faux provider, plus
736
+ updates):
737
+
738
+ - Range selection (pure): cut at a user entry, at an assistant entry mid-run
739
+ (one prompt followed by many tool rounds), never at a tool result or system
740
+ entry; a huge last tool result keeps its assistant; budget never reached and
741
+ only-the-marker prefix are nothing to compact; an existing marker is
742
+ summarized first and older in-range markers drop out; omitted and replaced
743
+ entries use their edited contribution; excluded error/aborted assistants are
744
+ not candidates; the §8.7 worked example.
745
+ - Summarization request: exact messages (system prompt, serialized
746
+ conversation, instructions line), tool-result truncation, no tools,
747
+ `cacheRetention: "none"`, no `deferred`, `maxTokens` from `reserveTokens` and
748
+ the model's max, pinned model/thinking/stream options surviving a config change
749
+ mid-summary.
750
+ - Summary entry: kind, wrapped text, `head` = first kept, `data.reason`; model
751
+ context after placement is `[summary, kept...]` followed by a full system
752
+ baseline on the next preparation; raw history unchanged and still scannable via
753
+ `entries()`; `pi.usage.models` includes every attempt, including failed,
754
+ retried, and stale ones; declines and hook-supplied summaries add none.
755
+ - Outcomes: nothing to compact (`{}`), `beforeCompact` decline (`{}`) and
756
+ supplied summary (no model call), first decision wins, hook throw reported and
757
+ ignored; `no_model`; retryable error then success; retries exhausted;
758
+ non-retryable error; `length` stop; tool call in the response; empty text.
759
+ - Manual: idle with empty inbox (placed at once, `{ submissionId }` whose submission is `done`);
760
+ idle with queued follow-ups after a failed run (final boundary places the
761
+ summary first, then starts the follow-up in the compacted context); busy
762
+ (queued, placed at the next `postTools` and the run continues in the compacted
763
+ context; placed at `final`); stale because a reset landed while summarizing;
764
+ stale because a later compaction cut further; an older summary with a later cut
765
+ placed after a newer one; two queued summaries in one boundary with the
766
+ second cutting before, at, and after the first (only "before" is stale); `Conversation.abort()` aborts a manual compaction and keeps an
767
+ already queued summary; `Submission.abort()` withdraws a queued summary;
768
+ `waitForIdle` waits for a manual compaction; `compact()` returns the ID
769
+ `waitForTask` settles with.
770
+ - Background threshold: starts only above the background threshold, only with
771
+ `backgroundTokens > 0`, only when no status is listed, not when disabled, not
772
+ without a cut, and not after a blocking compaction in the same generation; the
773
+ generation does not wait; summary placed at the next boundary; survives
774
+ `Conversation.abort()`; `abort({ background: true })` and `abortTask()` stop
775
+ it; idle waits ignore it.
776
+ - Blocking threshold: generation waits, nothing appended before the wait, then
777
+ re-prepares with a baseline and sends; request proceeds after nothing to
778
+ compact, decline, failure, fault, and direct `abortTask()` of the child; no
779
+ second compaction when still above the threshold; a background compaction in
780
+ flight ends stale; Esc during the blocking compaction aborts child then
781
+ generation (child terminal before the generation's abort handler) and settles
782
+ inputs `aborted`.
783
+ - Overflow: error entry appended and excluded from the retry request, attempt
784
+ unchanged, compaction then retry succeeds; second overflow fails
785
+ `model_error` with its error entry; overflow with compaction disabled, without
786
+ a cut, or after a threshold compaction fails; overflow whose compaction
787
+ declines/fails/aborts fails `model_error` with the overflow text; an overflow
788
+ error matching retryable patterns is still not retried; silent `stop`/`length`
789
+ overflow is an ordinary answer.
790
+ - Live status and events: status added in the creating commit with `blocking`,
791
+ `attempt`, and `retry` during backoff, removed with every outcome
792
+ (completed, failed, aborted, faulted, orphaned); several concurrent statuses;
793
+ `compaction_start`/`compaction_end` in batch order; snapshot `compactions` for a late joiner during summarize and retry;
794
+ view mount after a compaction head (kept entries stay mounted).
795
+ - Recovery: crash and reopen in `select` (hook reruns), `summarize` (request
796
+ resent once, no duplicate usage), `retry`, after placement, with the
797
+ generation waiting on a blocking compaction, with a summary queued, and with a
798
+ blocked compaction definition (orphaned on abort, status removed).
799
+ - Estimate and cut edge cases: a background summary queued mid-run and placed
800
+ at `postTools` makes the successor's estimate ignore the pre-compaction usage,
801
+ so no blocking compaction starts (also with a fixed Harness clock); an
802
+ assistant finishing while a summary is queued; raw order `assistant(call),
803
+ user, toolResult, assistant` never cuts at that user; edits carried by an
804
+ older in-range marker and a kept entry replacing a summarized one are reflected
805
+ in the summarized messages and the hook's `messages`; kept `pi.system` deltas
806
+ get omit edits in the next single baseline; compaction in a fork whose cut
807
+ falls on a parent entry (head, stale checks, view mount).
808
+ - Interactions: a summary queued before a reset in the same boundary (both
809
+ placed, reset wins) and after it (stale); a manual compaction's status blocks
810
+ a background start; a queued summary surviving a failed run is placed by the
811
+ next submission's final boundary before the new input; the summarization
812
+ request itself overflowing fails `model_error` and a waiting overflow
813
+ generation fails with the original overflow text; a retryable error after an
814
+ overflow compaction has the full retry budget; a background summary landing
815
+ during a retry backoff stays queued while a blocking compaction wins; an
816
+ application edit placed while summarizing is lost as §12 documents; a hook
817
+ that creates owned work and supplies a summary (placement and status removal
818
+ at hold, receipt after the child); `compact()` right after open enables
819
+ scheduling; a rejected classifying commit leaves no usage, submission,
820
+ summary, or outcome; idle admission that drains an older queued summary
821
+ together with the current one.
822
+ - Rerun every existing suite; generation, inbox, events, and view tests keep
823
+ passing unchanged except for the new optional fields.
824
+
825
+ ## 21. Reload and final conformance
554
826
 
555
827
  Complete any remaining §2.2 surface and lifecycle gates and the §7.5 in-process
556
828
  reload path through the registry: batch replacement while the Harness keeps
@@ -566,11 +838,10 @@ acquisition and an already-running callback that remains caller-owned across
566
838
  shutdown. Verify service withdrawal and client detach.
567
839
 
568
840
  Run the exhaustive public conformance matrix: stable persisted root identity;
569
- all root/create/lookup/fork/reset/collapse/abort/idle paths; every configuration
841
+ all root/create/lookup/fork/reset/compact/abort/idle paths; every configuration
570
842
  getter/setter and fork override; typed input/write submissions; task wait/abort;
571
843
  generic document access; registry changes before open, between open and resume,
572
- and while work runs; the activity view; structural watches; and
573
- blocked tasks surviving open. Compile-test every §2.2 and §3 owner/key/seed overload,
844
+ and while work runs; structural watches; and blocked tasks surviving open. Compile-test every §2.2 and §3 owner/key/seed overload,
574
845
  the normative usage sequences, and the Chord guide. The erased registry test must
575
846
  use a concrete narrowed-input task with multiple checkpoint phases and custom
576
847
  hooks. Verify that a Chord root-replacement delta remains distinct from a
@@ -579,3 +850,12 @@ Session-selected storage checkpoint.
579
850
  Run all package-specific tests and the repository check. Finish with a local
580
851
  coding-agent turn and a reopened interrupted turn through the public Harness,
581
852
  then stop for final review.
853
+
854
+ ## 22. Task graph view
855
+
856
+ A live, observable view of the Session's task graph for UIs and debugging, like
857
+ `ConversationView` for a conversation: every live task with its owner edge
858
+ (conversation or task), status (`running`, `waiting on …`, `completing`),
859
+ background flag, and owned conversations, published after each commit as a
860
+ replicated state or watch. `inspect()` already provides a one-off snapshot of
861
+ the live task records. Specify and build it after the final conformance package.