@earendil-works/pi-durable 0.99.1 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (203) hide show
  1. package/CHANGELOG.md +7 -0
  2. package/README.md +557 -46
  3. package/dist/entries.d.ts +20 -2
  4. package/dist/entries.d.ts.map +1 -1
  5. package/dist/entries.js +14 -1
  6. package/dist/entries.js.map +1 -1
  7. package/dist/env/index.d.ts +18 -38
  8. package/dist/env/index.d.ts.map +1 -1
  9. package/dist/env/index.js +2 -0
  10. package/dist/env/index.js.map +1 -1
  11. package/dist/env/node.d.ts +2 -0
  12. package/dist/env/node.d.ts.map +1 -1
  13. package/dist/env/node.js +53 -58
  14. package/dist/env/node.js.map +1 -1
  15. package/dist/harness/agent.d.ts +31 -0
  16. package/dist/harness/agent.d.ts.map +1 -0
  17. package/dist/harness/agent.js +209 -0
  18. package/dist/harness/agent.js.map +1 -0
  19. package/dist/harness/compaction.d.ts +56 -0
  20. package/dist/harness/compaction.d.ts.map +1 -0
  21. package/dist/harness/compaction.js +382 -0
  22. package/dist/harness/compaction.js.map +1 -0
  23. package/dist/harness/context.d.ts +8 -0
  24. package/dist/harness/context.d.ts.map +1 -1
  25. package/dist/harness/context.js +32 -21
  26. package/dist/harness/context.js.map +1 -1
  27. package/dist/harness/define.d.ts +18 -0
  28. package/dist/harness/define.d.ts.map +1 -0
  29. package/dist/harness/define.js +25 -0
  30. package/dist/harness/define.js.map +1 -0
  31. package/dist/harness/events.d.ts +166 -0
  32. package/dist/harness/events.d.ts.map +1 -0
  33. package/dist/harness/events.js +319 -0
  34. package/dist/harness/events.js.map +1 -0
  35. package/dist/harness/generation.d.ts +36 -7
  36. package/dist/harness/generation.d.ts.map +1 -1
  37. package/dist/harness/generation.js +356 -82
  38. package/dist/harness/generation.js.map +1 -1
  39. package/dist/harness/harness.d.ts.map +1 -1
  40. package/dist/harness/harness.js +141 -106
  41. package/dist/harness/harness.js.map +1 -1
  42. package/dist/harness/inbox.d.ts +59 -0
  43. package/dist/harness/inbox.d.ts.map +1 -0
  44. package/dist/harness/inbox.js +97 -0
  45. package/dist/harness/inbox.js.map +1 -0
  46. package/dist/harness/json.d.ts +10 -0
  47. package/dist/harness/json.d.ts.map +1 -0
  48. package/dist/harness/json.js +33 -0
  49. package/dist/harness/json.js.map +1 -0
  50. package/dist/harness/live.d.ts +66 -7
  51. package/dist/harness/live.d.ts.map +1 -1
  52. package/dist/harness/live.js +71 -7
  53. package/dist/harness/live.js.map +1 -1
  54. package/dist/harness/output.d.ts +62 -0
  55. package/dist/harness/output.d.ts.map +1 -0
  56. package/dist/harness/output.js +261 -0
  57. package/dist/harness/output.js.map +1 -0
  58. package/dist/harness/prompt.d.ts +9 -6
  59. package/dist/harness/prompt.d.ts.map +1 -1
  60. package/dist/harness/prompt.js +57 -14
  61. package/dist/harness/prompt.js.map +1 -1
  62. package/dist/harness/registry.d.ts +2 -3
  63. package/dist/harness/registry.d.ts.map +1 -1
  64. package/dist/harness/registry.js +69 -277
  65. package/dist/harness/registry.js.map +1 -1
  66. package/dist/harness/scheduler.d.ts +42 -12
  67. package/dist/harness/scheduler.d.ts.map +1 -1
  68. package/dist/harness/scheduler.js +649 -75
  69. package/dist/harness/scheduler.js.map +1 -1
  70. package/dist/harness/submissions.d.ts +14 -9
  71. package/dist/harness/submissions.d.ts.map +1 -1
  72. package/dist/harness/submissions.js +76 -34
  73. package/dist/harness/submissions.js.map +1 -1
  74. package/dist/harness/task-graph.d.ts +49 -0
  75. package/dist/harness/task-graph.d.ts.map +1 -0
  76. package/dist/harness/task-graph.js +163 -0
  77. package/dist/harness/task-graph.js.map +1 -0
  78. package/dist/harness/tool.d.ts +36 -0
  79. package/dist/harness/tool.d.ts.map +1 -0
  80. package/dist/harness/tool.js +415 -0
  81. package/dist/harness/tool.js.map +1 -0
  82. package/dist/harness/types.d.ts +330 -125
  83. package/dist/harness/types.d.ts.map +1 -1
  84. package/dist/harness/types.js.map +1 -1
  85. package/dist/harness/usage.d.ts +18 -0
  86. package/dist/harness/usage.d.ts.map +1 -0
  87. package/dist/harness/usage.js +57 -0
  88. package/dist/harness/usage.js.map +1 -0
  89. package/dist/harness/view.d.ts +43 -0
  90. package/dist/harness/view.d.ts.map +1 -0
  91. package/dist/harness/view.js +189 -0
  92. package/dist/harness/view.js.map +1 -0
  93. package/dist/index.d.ts +13 -5
  94. package/dist/index.d.ts.map +1 -1
  95. package/dist/index.js +8 -2
  96. package/dist/index.js.map +1 -1
  97. package/dist/session/observation.d.ts +18 -11
  98. package/dist/session/observation.d.ts.map +1 -1
  99. package/dist/session/observation.js +24 -13
  100. package/dist/session/observation.js.map +1 -1
  101. package/dist/session/session.d.ts +10 -1
  102. package/dist/session/session.d.ts.map +1 -1
  103. package/dist/session/session.js +23 -5
  104. package/dist/session/session.js.map +1 -1
  105. package/dist/session/transaction.d.ts +12 -3
  106. package/dist/session/transaction.d.ts.map +1 -1
  107. package/dist/session/transaction.js +90 -31
  108. package/dist/session/transaction.js.map +1 -1
  109. package/dist/storage/jsonl/storage.d.ts +22 -2
  110. package/dist/storage/jsonl/storage.d.ts.map +1 -1
  111. package/dist/storage/jsonl/storage.js +1 -1
  112. package/dist/storage/jsonl/storage.js.map +1 -1
  113. package/dist/storage/memory.js +1 -1
  114. package/dist/storage/memory.js.map +1 -1
  115. package/dist/storage/sqlite/database.d.ts +26 -19
  116. package/dist/storage/sqlite/database.d.ts.map +1 -1
  117. package/dist/storage/sqlite/database.js.map +1 -1
  118. package/dist/storage/sqlite/index.d.ts +1 -1
  119. package/dist/storage/sqlite/index.d.ts.map +1 -1
  120. package/dist/storage/sqlite/index.js.map +1 -1
  121. package/dist/storage/sqlite/migrations.js +7 -7
  122. package/dist/storage/sqlite/migrations.js.map +1 -1
  123. package/dist/storage/sqlite/node.d.ts +23 -7
  124. package/dist/storage/sqlite/node.d.ts.map +1 -1
  125. package/dist/storage/sqlite/node.js +129 -47
  126. package/dist/storage/sqlite/node.js.map +1 -1
  127. package/dist/storage/sqlite/storage.d.ts +12 -0
  128. package/dist/storage/sqlite/storage.d.ts.map +1 -1
  129. package/dist/storage/sqlite/storage.js +121 -129
  130. package/dist/storage/sqlite/storage.js.map +1 -1
  131. package/dist/testing/storage-benchmark.d.ts.map +1 -1
  132. package/dist/testing/storage-benchmark.js +0 -1
  133. package/dist/testing/storage-benchmark.js.map +1 -1
  134. package/dist/testing/storage-conformance.d.ts.map +1 -1
  135. package/dist/testing/storage-conformance.js +34 -2
  136. package/dist/testing/storage-conformance.js.map +1 -1
  137. package/dist/tools/bash.d.ts +28 -0
  138. package/dist/tools/bash.d.ts.map +1 -0
  139. package/dist/tools/bash.js +70 -0
  140. package/dist/tools/bash.js.map +1 -0
  141. package/dist/tools/edit-diff.d.ts +89 -0
  142. package/dist/tools/edit-diff.d.ts.map +1 -0
  143. package/dist/tools/edit-diff.js +386 -0
  144. package/dist/tools/edit-diff.js.map +1 -0
  145. package/dist/tools/edit.d.ts +18 -0
  146. package/dist/tools/edit.d.ts.map +1 -0
  147. package/dist/tools/edit.js +117 -0
  148. package/dist/tools/edit.js.map +1 -0
  149. package/dist/tools/env.d.ts +5 -0
  150. package/dist/tools/env.d.ts.map +1 -0
  151. package/dist/tools/env.js +7 -0
  152. package/dist/tools/env.js.map +1 -0
  153. package/dist/tools/file-mutation-queue.d.ts +9 -0
  154. package/dist/tools/file-mutation-queue.d.ts.map +1 -0
  155. package/dist/tools/file-mutation-queue.js +52 -0
  156. package/dist/tools/file-mutation-queue.js.map +1 -0
  157. package/dist/tools/image.d.ts +2 -0
  158. package/dist/tools/image.d.ts.map +1 -0
  159. package/dist/tools/image.js +92 -0
  160. package/dist/tools/image.js.map +1 -0
  161. package/dist/tools/index.d.ts +23 -0
  162. package/dist/tools/index.d.ts.map +1 -0
  163. package/dist/tools/index.js +15 -0
  164. package/dist/tools/index.js.map +1 -0
  165. package/dist/tools/path-utils.d.ts +5 -0
  166. package/dist/tools/path-utils.d.ts.map +1 -0
  167. package/dist/tools/path-utils.js +26 -0
  168. package/dist/tools/path-utils.js.map +1 -0
  169. package/dist/tools/read.d.ts +17 -0
  170. package/dist/tools/read.d.ts.map +1 -0
  171. package/dist/tools/read.js +101 -0
  172. package/dist/tools/read.js.map +1 -0
  173. package/dist/tools/write.d.ts +10 -0
  174. package/dist/tools/write.d.ts.map +1 -0
  175. package/dist/tools/write.js +31 -0
  176. package/dist/tools/write.js.map +1 -0
  177. package/dist/{env/utils/truncate.d.ts → truncate.d.ts} +1 -17
  178. package/dist/truncate.d.ts.map +1 -0
  179. package/dist/truncate.js +146 -0
  180. package/dist/truncate.js.map +1 -0
  181. package/dist/types.d.ts +97 -19
  182. package/dist/types.d.ts.map +1 -1
  183. package/dist/types.js.map +1 -1
  184. package/package.json +32 -14
  185. package/dist/env/utils/adaptive-publisher.d.ts +0 -24
  186. package/dist/env/utils/adaptive-publisher.d.ts.map +0 -1
  187. package/dist/env/utils/adaptive-publisher.js +0 -79
  188. package/dist/env/utils/adaptive-publisher.js.map +0 -1
  189. package/dist/env/utils/output-capture.d.ts +0 -31
  190. package/dist/env/utils/output-capture.d.ts.map +0 -1
  191. package/dist/env/utils/output-capture.js +0 -238
  192. package/dist/env/utils/output-capture.js.map +0 -1
  193. package/dist/env/utils/truncate.d.ts.map +0 -1
  194. package/dist/env/utils/truncate.js +0 -294
  195. package/dist/env/utils/truncate.js.map +0 -1
  196. package/dist/harness/config.d.ts +0 -20
  197. package/dist/harness/config.d.ts.map +0 -1
  198. package/dist/harness/config.js +0 -18
  199. package/dist/harness/config.js.map +0 -1
  200. package/docs/chord-delta-findings.md +0 -358
  201. package/docs/pico-v5-chord-usage.md +0 -373
  202. package/docs/pico-v5-handoff.md +0 -581
  203. package/docs/pico-v5.md +0 -3084
@@ -1,581 +0,0 @@
1
- # Pico5 implementation handoff
2
-
3
- `packages/durable/docs/pico-v5.md` is normative. Implement this list in order.
4
- After every package: run its tests, run `npm run check`, and stop for user review.
5
- Do not redesign later packages while implementing the current one.
6
-
7
- Pico3 is reference material only. Preserve useful behavior, not its capability
8
- facades, membranes, document routing, view projection, events, or clone chains.
9
-
10
- ## Status
11
-
12
- - Obsolete `pico` and `pico4` prototypes were removed.
13
- - `pico3` remains.
14
- - Packages 1–15 are implemented in `packages/durable`; Package 10 was already satisfied by Chord's canonical structural diff implementation.
15
-
16
- ## 1. Records, cursors, and memory tables
17
-
18
- Implement IDs, sequences, reserved root conversation ID `1`,
19
- `ConversationRecord`, `EntryRecord`, strict input/write `SubmissionRecord`
20
- values, live/terminal `TaskRecord` values, document records, storage writes, backend-opaque JSON cursors, and
21
- detached `MemoryStorage` tables.
22
- Reserve `Conversation` for the public conversation object, `Entry` for the typed
23
- entry definition, and `Task` for the typed executable definition returned by
24
- `defineTask()`.
25
-
26
- Test reserved root identity and immutable creation, mixed atomic commits,
27
- rollback, detached reads/writes, cursor boundaries,
28
- fork-aware entry scans through deep ancestor caps, head lookup,
29
- entry-to-commit lookup, full task replacement, and submission replacement/
30
- request-ID lookup.
31
-
32
- ## 2. Memory document records
33
-
34
- Add selected document base/delta writes, retirement, reincarnation,
35
- current/as-of membership, exact logical-address lookup, scoped scans, and
36
- materialized point-in-time reads. Storage keeps base/delta revisions private and
37
- returns a detached value plus its stored definition version. It applies Chord
38
- `Op[]` directly and receives no definition callbacks or unused candidate values.
39
-
40
- Test Session-, conversation-, and task-scoped documents, half-open lifetimes,
41
- create-plus-retire, retired historical membership, family queries, current-only
42
- reclamation, version boundaries, detached ownership, and no scans of unrelated
43
- document records.
44
-
45
- ## 3. SQLite backend
46
-
47
- Implement the complete storage contract with ordinary rows and indexed document
48
- records. Do not translate Chord operations into SQL JSON patches.
49
-
50
- Run the memory conformance suite after reopen. Test SQL transaction rollback,
51
- recent/ancient as-of reads, query plans, latest reclamation, WAL checkpointing,
52
- deleted-page reuse, and representative storage sizes.
53
-
54
- ## 4. JSONL publication
55
-
56
- Implement table writes in `main.jsonl`, one document sidecar per incarnation,
57
- one sidecar per live task, and one main marker per commit. Do not add a
58
- standalone-sidecar protocol.
59
-
60
- Copy, rather than import, the current `ExecutionEnv`, `FileSystem`, `Shell`, Node
61
- implementation, and their required utility files from `packages/agent/src/harness`
62
- into `packages/durable/src/env`. Copy only the environment-related slice, not
63
- agent skills, prompts, telemetry, or tool definitions. Extend the copied
64
- filesystem contract with exact-byte file truncation and file flushing. JSONL
65
- depends only on `FileSystem`, not the broader `ExecutionEnv`. Keep the portable
66
- environment and JSONL entry points free of Node built-ins; expose Node
67
- implementations only from `/env/node` and `/storage/jsonl/node`. Do not use the
68
- Pico3 implementation as source material.
69
-
70
- Refactor the current `MemoryStorage` state machinery into a two-phase prepared
71
- mutation: validation and detachment produce a candidate that can later be
72
- applied without failure. Build `MemoryStorage.commit()` on that pair, and reuse
73
- the same machinery for JSONL. JSONL must append every prepared sidecar record,
74
- append the main marker, and only then apply the prepared in-memory mutation.
75
- Serialization or preparation failure occurs before file I/O and does not poison
76
- the backend. Retained indexes/materializations remain detached from write
77
- arguments, and reads never expose backend-owned cached objects.
78
-
79
- JSONL creation has `fsync?: boolean`, defaulting to `false`. With `false`, append
80
- sidecars and then the marker without an explicit flush. With `true`, append all
81
- affected sidecars, flush each affected sidecar, and then append the main marker.
82
- Do not explicitly flush `main.jsonl` for ordinary publication. A main-only commit
83
- has no sidecars to flush. Any uncertain publication append or flush failure
84
- poisons the open backend and publishes no prepared in-memory mutation. Package 5
85
- adds the separate post-publication flush required to authorize reclamation.
86
-
87
- Fault-test torn/short sidecar writes, failures between sidecars, every marker
88
- boundary, unconfirmed tails, missing confirmed data, poisoned writes, exact-byte
89
- tail truncation, both fsync settings and their call ordering, detached retained
90
- state and reads, and browser-safe portable entry points. Run the complete storage
91
- conformance suite directly and after reopen.
92
-
93
- ## 5. JSONL reclamation
94
-
95
- Implement task-document retirement and current-only base reclamation using
96
- committed markers and descriptor invalidation. Remove a sidecar directly when no
97
- records remain; otherwise use temporary replacement and rename. With `fsync:
98
- true`, flush `main.jsonl` once before destructive reclamation so the authorizing
99
- marker cannot disappear while cleanup survives; if that flush fails, skip
100
- reclamation without failing the already-published commit. Flush a non-empty
101
- temporary replacement before rename. This is not publication flushing or main
102
- compaction.
103
-
104
- Crash-test every rewrite/rename boundary. Verify that rewindable history is
105
- never reclaimed and default no-fsync behavior matches the specification.
106
-
107
- ## 6–7. Tracker transaction core, definitions, and typed access
108
-
109
- **Prerequisite:** `@earendil-works/chord/delta` exports the canonical
110
- Astra-immutable-optimized `track`, `Tracker`, `Change`, and `Prepared`, and its
111
- draft placements reject values that are not strict JSON.
112
- Experimental variants under other Delta directories are not Pico APIs.
113
-
114
- Implement these packages as one milestone. Keep the implementation layers
115
- separate, but do not build a temporary untyped document-acquisition seam.
116
- Implement the generic `Tx` table surface these tests require: exact table reads,
117
- paginated conversation/entry/task scans, `ReadAfterWrite`, ID-creating writes,
118
- and full task replacement. Scans expose the Storage cursor and caller-selected
119
- limit; they never hide an unbounded full scan. Semantic
120
- conversation, entry, task, and scheduler behavior remains in Packages 13–17.
121
-
122
- Keep one Astra-immutable tracker per loaded document. Its trusted immutable
123
- `value` is the current shareable revision. `prepare()` emits detached
124
- self-contained operations and computes the next revision with the optimized
125
- immutable applier; operation placement payloads and that revision may share
126
- containers, and neither may be mutated. `adopt()` validates ownership and
127
- revision, then only pointer-swaps to the already-computed value.
128
-
129
- Implement scope-preserving singleton/family tokens and overloads for Session,
130
- conversation, and task owners. Only `tx.doc()` is get-or-create: singleton tokens
131
- supply `initial()`, while family calls always supply key and seed and use only the
132
- first seed when absent. Definitions are explicit typed arguments, not registered
133
- declarations; conflicting definitions claiming one persisted kind are
134
- unsupported caller misuse.
135
-
136
- On first `tx.doc()` access, memoize the acquisition promise by logical address
137
- before awaiting it, then call `tracker.beginChange()`. Repeated access returns the
138
- same overlay draft for the whole possibly async Session callback. The Session
139
- line permits only one open change per tracker. A callback that settles with an
140
- unresolved acquisition rejects: seal `Tx`, abort open changes, drain and abort
141
- the pending acquisition, and observe its failure. Callback failure aborts every
142
- change. Callback success prepares every change before Storage admission.
143
-
144
- Do not walk prepared operation payloads or selected bases for strict JSON; they
145
- are strict JSON by construction. Tracker branding and revision checks enforce
146
- ownership and staleness. Evaluate each staged document write exactly once and
147
- pass Storage only the selected base value or operation batch. Keep every previous immutable revision unchanged through Storage
148
- settlement. On success, adopt every prepared value by pointer swap and enqueue
149
- its immutable revision/operations publication before releasing the line. On
150
- Storage failure, abort prepared changes, poison the Session, and publish nothing.
151
- Preparation failures roll back normally; Package 8 adds checkpoint selection and
152
- its failure path.
153
-
154
- Initializer, migration, and replacement roots are copied into exclusive kernel
155
- ownership with a strict-JSON check before becoming trusted immutable revisions.
156
- Loaded and fork-copy roots come detached from Storage and are tracked without
157
- another copy. Chord copies and strict-JSON-checks every draft placement and
158
- throws at the offending assignment. Astra empty batches suppress
159
- ordinary writes, while replayable nonempty structural no-ops remain valid writes
160
- and publications. No runtime freezing or second operation-payload copy is
161
- required.
162
-
163
- Snapshot, source, and watch lookup never create and return `undefined` when
164
- absent. `snapshot()` returns the shareable immutable current revision; callers
165
- must copy before mutation. A read-only migration may cache its migrated immutable
166
- tracker together with the older stored-version marker, without writing; the next
167
- successful `tx.doc()` still writes the required current-version base.
168
- Transaction-staged creation or migration enters the shared cache only after its
169
- enclosing Storage commit succeeds. All cold loads run on the Session line; a
170
- loaded immutable revision may be read without copying.
171
-
172
- Test callback failure; escaped-draft revocation at callback settlement; concurrent duplicate
173
- acquisition; callback failure and success with a pending acquisition; late
174
- acquisition after sealing; concurrent initialization once; initial bases; family
175
- first-seed wins; scope/token mismatch; non-creating reads; shared immutable
176
- snapshots and stable prior revisions; empty-batch suppression and replayable
177
- redundant structural no-ops; multi-document preparation failure; uncertain
178
- Storage failure poisoning; old-revision stability through Storage settlement;
179
- pointer-swap and replacement adoption; operation/revision payload sharing under
180
- the trusted no-mutation contract; non-JSON initializer and draft-placement
181
- rejection; assignment copying and repeated-placement
182
- independence; authority and prepared-draft non-escape; terminal-task rejection;
183
- task-derived conversation identity; retirement; reincarnation-bound sources;
184
- and unload/reload. Include create-task-then-document,
185
- document-after-terminal rejection, and create-document-then-terminal settlement
186
- in one transaction; internal candidate validation must not trigger
187
- `ReadAfterWrite`.
188
-
189
- ## 8. Checkpoints and migration
190
-
191
- After tracker preparation, Session evaluates `checkpointWhen(value, ops, { deltasSinceBase })` exactly once
192
- for ordinary mutations and sends Storage only the selected base or delta.
193
- Implement required creation/version bases and lazy all-older-version migration
194
- on typed access; Harness open does not scan ordinary documents.
195
-
196
- Test read-only in-memory migration, `tx.doc()` migration rollback and coalescing
197
- with later edits, rewindable migration on current/historical read, the first
198
- successful `tx.doc()` version base even without a JSON change, newer-version
199
- rejection, migrated state/watch hydration without a write, subsequent operations
200
- against that migrated baseline, stored-version fork copying, unaccessed and unavailable-definition
201
- preservation, predicate failure rollback before Storage admission, and checkpoint
202
- starvation without backend heuristics.
203
-
204
- ## 9. Conversation document forks
205
-
206
- Using fixture conversations and entry-to-commit mappings, implement the `asOf`,
207
- `current`, and `initial` settings for singleton and family documents.
208
-
209
- Test opaque stored-version copying without definitions, retired membership, new
210
- child incarnations, later lazy migration, lazy `initial` creation, and exclusion
211
- of task- and Session-scoped documents.
212
-
213
- ## 10. Chord structural array operations
214
-
215
- **Chord-owned prerequisite/integration:** the canonical Astra-immutable operation
216
- generator must encode compact replayable array changes; Pico only verifies and
217
- consumes it.
218
-
219
- Improve the canonical generator so ordinary positional mutations encode
220
- scattered removals without carrying retained payloads. Callers must not write
221
- operations manually.
222
-
223
- Test front/tail/middle/scattered/all/no removal, retained 256 KiB and 1 MiB
224
- payloads, append plus removal, later nested/index writes, exact replay, unchanged
225
- previous immutable revisions, and equality between Astra's prepared candidate
226
- and immutable operation replay. One prepared document change remains one Session
227
- commit; no intermediate candidate is adopted or published.
228
-
229
- ## 11. Chord document state
230
-
231
- Use Chord's existing `ReplicatedStateSource` attachment contract internally, but
232
- expose one already-attached read-only `DocumentState` per acquisition. It must
233
- atomically hydrate in O(1) from the current immutable revision and publish later
234
- exact committed immutable value/operation frames without another tracker, value
235
- copy, or re-diff. Disposal unregisters that one state. Pico remains the sole
236
- document mutator.
237
-
238
- Test contiguous Chord delivery sequences, atomic hydrate/subscribe, a baseline
239
- that covers queued publication without duplicate application, exact value and
240
- operation reference sharing, retirement to `null`, incarnation-bound recreation,
241
- independent state disposal, migrated hydration, definition-free fork copies,
242
- tracker-cache unload, and trusted mutation footguns.
243
-
244
- ## 12. Document watches
245
-
246
- Implement non-creating `watchDoc` as an incarnation-bound `WatchHandle` that
247
- returns `undefined` when absent and atomically captures the current immutable
248
- revision in O(1) while registering for later exact committed frames. Before
249
- `start()`, its value remains the acquisition revision. After start, invoke one
250
- serialized asynchronous listener with each exact value and operation batch,
251
- advancing `watch.value` immediately before the callback. Preserve commit Context
252
- values without inheriting producer cancellation.
253
-
254
- Retain at most 100 pending frames, excluding the in-flight callback. Adding frame
255
- 101 replaces the complete undelivered suffix with one root replacement carrying
256
- the newest exact immutable revision and Context. Do not estimate serialized
257
- bytes, call `JSON.stringify()` for accounting, copy values, replay operations, or
258
- re-diff revisions.
259
-
260
- Test updates between acquisition/return/start; exact values and operation
261
- references; no callback overlap; listener-initiated commits; commits during an
262
- in-flight callback; overflow before start and behind an active callback;
263
- retirement folded into an overflow reset; replayable redundant structural
264
- commits; delivery Context value preservation; retained earlier revision
265
- stability; trusted mutation footguns; retirement before start and while active;
266
- recreation; idempotent stop; second-start rejection; cancellation during
267
- acquisition; cancellation/close during a callback without aborting or joining it;
268
- listener-error settlement; and invocation-owned cleanup in package 14.
269
-
270
- Packages 13–20 are vertical milestones. Each must leave one real public path
271
- working end to end; do not defer all integration to the last package.
272
-
273
- Cross-cutting constraints for these milestones:
274
-
275
- - Introduce each persisted built-in document kind once, with its final schema,
276
- history/fork policy, migration, checkpoint predicate, and public mount path.
277
- Record that concrete protocol in the normative specification when the kind
278
- lands. Do not add temporary built-in kinds or later replace a fake schema.
279
- - Keep all visible progress durable. Tests may use faux models and fake effects,
280
- but production code must use pi-ai's exported `Models` interface and the real
281
- task chain; do not add a Pico model adapter or production fake successor.
282
- - Prepare every loaded `ConversationView` revision from the complete candidate
283
- Session commit before Storage admission. Never rebuild a view by subscribing
284
- to already-committed table/document publications.
285
- - A milestone may leave later operations unimplemented, but it must not expose a
286
- temporary public facade. Extend the final §2.2 objects as later behavior lands.
287
-
288
- ## 13. Openable Harness
289
-
290
- Implement the first usable slice of the final §2.2 surface: `Harness.open/close`
291
- with a registry, `root(context, { init })`, `conversation(id)`,
292
- `createConversation({ ownership, init })`, and generic
293
- inherited Session document APIs (`Harness extends Session`). Implement the
294
- public `Entry` definition token and `Conversation` handles rather than adding an
295
- intermediate capability facade. `Conversation` provides `id`, conversation-bound
296
- `commit()`, fork-aware cursor `entries()`, `context()`, `fork(at, { ownership,
297
- init })`, and every configuration getter/setter.
298
-
299
- Implement the §7.1 registry core: `createRegistry()`, `tools.add`, `tools.wrap`
300
- composition, `batch`, keyed ordering with remembered positions, `Registration`
301
- `dispose`, immutable `snapshot()`, and list helpers. Hook,
302
- task, and system prompt registrations are stored and listed now; their dispatch
303
- lands in Packages 14–16.
304
-
305
- Bring conversation and entry semantics up to the normative contract: explicit
306
- ownership, fork-aware cursor pagination, head lookup, entry edits, and model
307
- context derivation. Context reduction includes newest-edit wins, positional PR
308
- #9548 `SystemMessage` replay, tool-result ordering, missing post-fork tool
309
- results, excluded stop reasons, and separate raw-history/model-context results.
310
-
311
- Define the final `ConversationConfig` document from §2.2. Raw document writes
312
- are trusted and unchecked. Root creation uses the internal reserved-ID
313
- bootstrap in one commit together with the default configuration and `init`; do
314
- not expose a temporary root-creation API.
315
-
316
- Open ignores live tasks in this milestone; Package 14 adds reconciliation.
317
-
318
- Acceptance: open persistent storage, obtain the root, mutate configuration and
319
- history through public handles, create and fork conversations, close, reopen,
320
- and verify stable root/conversation identity and state. Test lazy root creation
321
- with `init` run once; atomic create/fork with `init`; actual forks; deep ancestor
322
- caps; same-commit entry prefixes; cursor boundaries; self-head resolution;
323
- newest-edit wins; raw head-to-tail transcript versus model context; model-less
324
- and excluded assistant entries; replacements/omissions; multiple heads;
325
- positional system messages; tool-result ordering; missing post-fork results;
326
- every configuration getter/setter; `setActiveTools` duplicate and newly-added
327
- unregistered rejection with stale names allowed; default active tools from the
328
- registry; as-of configuration inheritance including unavailable names; registry
329
- add/dispose/batch rollback/nested-batch rejection/keyed order/wrap composition;
330
- and reopen.
331
-
332
- ## 14. Durable task runtime
333
-
334
- Implement `defineTask`, exhaustive phase maps, full checkpoint replacement,
335
- kind migration at reservation, runtime commits and memos, invocation lifetime
336
- gates, scheduler reservation, dependencies, terminal outcomes, typed waits,
337
- and joins. Complete the task-facing §2.2 methods: `resume`, `getTask`,
338
- `waitForTask`, `abortTask`, and the task-aware portion of idle waits. Task definitions come from the registry snapshot;
339
- add `RegistryReader.subscribe()` so registry changes wake the scheduler.
340
-
341
- Include the execution-critical abort core: durable direct-task marks,
342
- signal-and-join of an active run, fresh abort invocation, run-commit rejection
343
- after a mark, and close precedence. Deep owned-subtree cascading and background
344
- boundaries remain Package 18. Open now reconciles every surviving `running` task
345
- to `pending`. Missing, too-old, and unmigratable definitions leave the task pending and
346
- blocked (§5.4); registry changes wake the scheduler to reconsider them. Aborting
347
- a blocked task settles it as `orphaned` with the full cleanup; nothing else
348
- orphans a task. Implement per-phase registry snapshots, refresh at
349
- every normal phase boundary, and hand over when the task definition object changed
350
- and the new definition can reserve the task.
351
-
352
- Deferred to later packages; do not stub them in Package 14:
353
- - Orphaning writes the terminal `orphaned` record and retires task documents.
354
- Unanswered input submissions and clearing run control are added in Package
355
- 15, when submissions and run control exist.
356
- - `TaskRuntime.hooks` (the hook runner) is added in Package 16 with hook
357
- dispatch. Package 14 adds `runtime.registry` (the phase's registry snapshot)
358
- and `runtime.models`.
359
- - `TaskRuntime.conversation()` returns a `ConversationHandle`, whose `submit`
360
- needs Package 15 and whose `abort`/`waitForIdle` need ownership traversal; it
361
- is added with the invocation-bound owned APIs in Package 18.
362
- - Idle waits count live non-background tasks directly (Harness-wide or in the
363
- addressed conversation). Traversal through owned conversations and background
364
- boundaries replaces this in Package 18.
365
-
366
- Use a fake two-phase external effect to test the real runtime. Acceptance is an
367
- intent/effect/outcome task interrupted after intent, closed, reopened on the same
368
- storage, and safely resumed to a durable terminal receipt. Also test unchanged-
369
- checkpoint faulting, same-phase progress, cancellation precedence, thrown
370
- handlers, dependencies, result values and entry IDs, first-writer-wins memos,
371
- terminal checkpoint/memo removal, task-document retirement, close/reopen without
372
- abort marks or fabricated outcomes, no fresh phase/abort dispatch while closing,
373
- watch cleanup, blocked tasks unblocked
374
- by later registration, migration failure leaving the record unchanged, handover
375
- after a same-name task replacement, no handover to a missing or incompatible
376
- definition, no overlap between predecessor and successor
377
- invocations, abort of a blocked task settling as `orphaned` with full cleanup,
378
- run commits rejected after the mark, and crashes at every direct-task abort stage.
379
-
380
- Implemented design notes: every task transition is decided by one callback
381
- serialized on the Session line (reservation, abort marks, runtime commits, and the
382
- step before each phase). `Tx` has no task replacement; `runtime.commit()`
383
- callbacks return the typed next state. `suspend()`, `hold()`, `quiescent()`, and
384
- a public `markTask()` were dropped from §2.2.
385
-
386
- ## 15. First runnable no-tool chat turn
387
-
388
- Implement the smallest real input-to-answer vertical path. Extend Package 14's
389
- `orphaned` settlement to mark affected input submissions unanswered and clear
390
- matching run control in the same commit, through the Harness hook for
391
- scheduler-written outcomes (§5.4); the scheduler learns nothing about turns or
392
- task kinds. Implement the final `pi.live` document (§8.2) with `run` and
393
- `generation`, the built-in entry kinds (§8.1), and the `pi.generation` task
394
- (§8.3). The inbox document is defined in Package 17 with its first writer. Add
395
- input `Submission` admission, request-ID deduplication, reacquisition and
396
- waiting, idle placement, active-turn ownership, successful answer settlement,
397
- and terminal failure cleanup. Expose the final `SubmissionDraft` union. Idle
398
- write submissions append their entry and settle `done`. Until Package 17, every
399
- submission to a busy conversation rejects with `ConversationBusy` regardless of
400
- `whenBusy`, writing nothing. Busy steer/follow-up behavior, queued writes, and
401
- reset remain Package 17.
402
-
403
- Also implement the decided surface changes: typed entry tokens
404
- (`defineEntry<D extends JsonValue>`, token-first `tx.appendEntry()` and
405
- `tx.entry()` overloads); removal of `input` from `createConversation()`;
406
- built-in tasks pre-registered by `createRegistry()` (undisposable, not
407
- overridable, required by `Harness.open`); the `TaskRuntime` additions
408
- `snapshot`/`snapshotAsOf`, `context()`, `now()`, and `report()`; and the
409
- `streamOptions`/`retry` configuration fields with their getters and setters.
410
-
411
- Implement the §7.4 system prompt: rendering registered sections with `tag` and
412
- wrappers, and no-tool request preparation, including
413
- exact persisted rendered strings, positional system baselines/deltas with empty
414
- `content`, head-cut rebaselining with `ContextEdit` omissions, order-only
415
- two-entry rewrites, and section failure handling. No tools are offered in this
416
- package: preparation emits no tool declarations and passes `PromptInput.tools`
417
- as `[]`. Until Package 16, a `toolUse` response settles as the answer.
418
- Implement the ordinary generation phases needed for one response: preparation,
419
- request intent, durable throttled partials, attempts/retry classification,
420
- deferred handle polling/cancellation, assistant entry settlement, and input-
421
- submission completion.
422
-
423
- Call pi-ai only through its exported `Models` methods: `getModel()`,
424
- `streamSimple()`, `fetchDeferred()`, and `cancelDeferred()`. The test double must
425
- implement that same interface; production code gets no adapter. A missing configured model produces
426
- the durable `no_model` failure. All progress exposed to observers is committed
427
- state, never raw provider frames.
428
-
429
- Acceptance: `Harness.open → root → resume → submit(input) → Submission.wait →
430
- durable assistant answer → close/reopen`, using faux Models in tests. Exercise
431
- interruption and reopen before/after every implemented generation phase,
432
- aborted-partial conversion, deferred polling/cancellation, retryable and
433
- terminal model errors, no-visible-undurable updates, exact section order/value
434
- patches, complete post-head baselines, retained system deltas on both sides of a
435
- head marker, sections reading conversation documents through `input.read`,
436
- throwing sections, preparation
437
- rerun after a concurrent tail/config commit, fault and orphan turn cleanup,
438
- typed entry tokens, and durable submission settlement. This is also the first print-mode smoke path: print
439
- awaits its own input submission rather than global idle.
440
-
441
- ## 16. First coding-agent tool turn
442
-
443
- Implement hook dispatch and `TaskRuntime.hooks`, deferred from Package 14
444
- (Session-wide and scoped to a conversation or its owned subtree) and wire the real generation → tool tasks → post-tools → generation
445
- chain. Implement offered-set checks, tool pinning from the phase snapshot,
446
- declaration and argument validation against both the offered declaration and
447
- the pinned implementation, hook composition, durable execution intent, stored
448
- replay policy, bounded output/details presentation documents, result entries, post-tools
449
- joining, controls, and `postTools`/`final` boundaries. The generation task now
450
- classifies tool calls and continues through the real built-in task chain;
451
- neither side uses a production fake successor.
452
-
453
- Implement preparation behavior for tool loadout additions/removals, same-name
454
- replacement ordering, complete baseline tool declarations, wrapped tools and
455
- failing wrappers, and hook memos.
456
-
457
- Acceptance: input → model tool call → registered local read/bash/edit operation →
458
- tool result → model answer → durable submission settlement. Run that path once
459
- normally and once interrupted/reopened. Test recovery from every tool and post-
460
- tools phase; offered-history enforcement; before/after hook rules; both stored/
461
- current replay-policy directions; default and overridden `outputLimits`; output
462
- content fallback; details replacement, last-details fallback, and coalesced
463
- commit settlement; drain-
464
- before-terminal ordering; abort/close with buffered output; invocation-bound
465
- handles and watches; unregistered active tools that are removed from the offered set, re-added after re-registration, and produce `tool_unavailable` results when called, without failing the request; a tool replaced mid-call
466
- finishing under its pinned implementation, including across phase boundaries;
467
- hooks surviving a task reload; duplicate active names offered once; order-only tool changes; atomic assistant/tool/post-tools commits;
468
- and all positional tool-history cases.
469
-
470
- ## 17. Live UI and product state
471
-
472
- Complete submissions and inbox behavior: busy `steer`/`followUp`/`reject`,
473
- passive writes, withdrawal, ordered `postTools`/`final` selection, stale targets,
474
- self-head cuts, successor turns, queued reset/handoff, and every terminal cleanup.
475
- Successful inputs still require an answer; writes settle on placement and never
476
- start generation.
477
-
478
- Add `harness.blockedTasks()`, deferred from Package 14: the pending tasks this
479
- process cannot run and why (`missing_task`, `task_too_old`, `migration_failed`
480
- with the stored and registered versions and the migration error). It is derived
481
- from task records, the current registry snapshot, and scheduler memory, and is
482
- never persisted.
483
-
484
- Specify and implement the Harness activity view:
485
- the active conversations, notifications when a conversation becomes active or
486
- idle, all derived from committed turn-control and task
487
- state.
488
-
489
- Define any remaining built-in preference/presentation documents once with final
490
- schemas. Implement the structural `{ conversation, entries, docs }`
491
- `ConversationView`, `viewState()`, and `watch()`. Build the first revision lazily
492
- on the Session line. For each affected commit, derive and prepare one mounted
493
- operation batch from the complete candidate transaction before Storage
494
- admission; after success, only install prepared pointers/cursors and enqueue the
495
- exact immutable frame. Do not derive the mount through `subscribeCommits()`.
496
-
497
- Add the §9.4 notification adapter directly from uncoalesced committed
498
- publication, without another tracker or persistence authority. Wire TUI
499
- hydration to structural state/watch, print to its own `Submission`, and JSON/RPC
500
- to correlated commands plus ordered committed notifications. Transport
501
- backpressure and disconnect policy stay in the mode adapter.
502
-
503
- Table-test every submission transition, cross-type request-ID conflicts,
504
- interleaved queue selection, compact positional removal of large payloads,
505
- abort results, reopen waits, writes pending without a later boundary, busy reset
506
- placement, and orphan/fault cleanup. Test one view publication per Session
507
- commit; atomic entry/preview settlement; parent-linked active entries and heads;
508
- mounted create/recreate/retire; preparation rollback before Storage; empty-batch
509
- suppression and redundant nonempty revisions; contiguous delivery; stable public
510
- paths; O(1) immutable acquisition; structural sharing; serialized consumers;
511
- bounded reset behind an in-flight callback; durable retry/tool/collapse status;
512
- output truncation metadata; the documented placement of diagnostics in entries,
513
- terminal details, or bounded state; asynchronous consumer initialization; and
514
- absence of semantic projection. Verify watch overflow cannot erase a separately
515
- subscribed notification lifecycle, late clients hydrate structurally, and
516
- notifications expose committed throttled
517
- progress rather than raw provider frames.
518
-
519
- ## 18. Ownership and subagents
520
-
521
- Complete the remaining owned-conversation and abort semantics: durable foreground
522
- subtree cascades, signal/join/fresh-abort across ownership edges, background
523
- boundaries, ordinary and full traversal, conversation abort, and exact idle
524
- waits. Finish invocation-bound owned APIs used by tools and the foreground and
525
- background subagent provisioning patterns, including atomic task/conversation/
526
- registry creation and request-ID-safe submission recovery.
527
-
528
- Test deep ownership trees, owner edges after terminal settlement, nested
529
- background boundaries, conversation abort/join with surviving passive writes and
530
- background tasks, cancellation of waiters without cancellation of work, and
531
- atomic cancellation intent. Test default non-inheritance, inheritance from the
532
- current committed tail, empty source conversations, document fork policies,
533
- configuration overrides through `init`, foreground subagent cascade, and
534
- background supervisor recovery before and after submission admission.
535
-
536
- ## 19. Collapse and overflow
537
-
538
- Implement manual, threshold, and generation-overflow collapse; exchange-boundary
539
- range selection; summarization; retry policy; staleness checks; and headed
540
- summary entries. Wire generation's real overflow path directly to the collapse
541
- task, and complete `Conversation.collapse()` so it returns the admitted task ID.
542
-
543
- Generation preparation no longer rechecks the transcript before appending its
544
- system entries (§7.4, §12): only run tasks and turn boundaries write to a busy
545
- conversation. A collapse summary for a busy conversation must therefore be
546
- placed at a turn boundary or as a step of the run, never appended concurrently.
547
-
548
- Test model context before and after collapse, raw history preservation, provider
549
- failure, declined and stale work, manual/threshold/overflow admission, late-join
550
- presentation state, and reopen from every phase. Rerun generation overflow
551
- integration without a fake collapse kind.
552
-
553
- ## 20. Reload and final conformance
554
-
555
- Complete any remaining §2.2 surface and lifecycle gates and the §7.5 in-process
556
- reload path through the registry: batch replacement while the Harness keeps
557
- running, pinned in-flight work, and task handover at phase boundaries.
558
- Close/reopen is not required for reload.
559
-
560
- Test that close seals commit and mutation admission, lets already-admitted
561
- storage settlement finish despite caller cancellation, stops future state/watch
562
- delivery, joins task/tool/hook invocations outside the Session line, writes no
563
- abort or terminal outcome, starts no fresh abort invocation, and never runs old
564
- and new Harness generations concurrently. Include cancellation during watch
565
- acquisition and an already-running callback that remains caller-owned across
566
- shutdown. Verify service withdrawal and client detach.
567
-
568
- Run the exhaustive public conformance matrix: stable persisted root identity;
569
- all root/create/lookup/fork/reset/collapse/abort/idle paths; every configuration
570
- getter/setter and fork override; typed input/write submissions; task wait/abort;
571
- generic document access; registry changes before open, between open and resume,
572
- and while work runs; the activity view; structural watches; and
573
- blocked tasks surviving open. Compile-test every §2.2 and §3 owner/key/seed overload,
574
- the normative usage sequences, and the Chord guide. The erased registry test must
575
- use a concrete narrowed-input task with multiple checkpoint phases and custom
576
- hooks. Verify that a Chord root-replacement delta remains distinct from a
577
- Session-selected storage checkpoint.
578
-
579
- Run all package-specific tests and the repository check. Finish with a local
580
- coding-agent turn and a reopened interrupted turn through the public Harness,
581
- then stop for final review.