@earendil-works/pi-durable 0.99.2 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/README.md +196 -74
- package/dist/env/index.d.ts +5 -0
- package/dist/env/index.d.ts.map +1 -1
- package/dist/env/index.js.map +1 -1
- package/dist/env/node.d.ts +2 -0
- package/dist/env/node.d.ts.map +1 -1
- package/dist/env/node.js +2 -0
- package/dist/env/node.js.map +1 -1
- package/dist/harness/agent.d.ts +31 -0
- package/dist/harness/agent.d.ts.map +1 -0
- package/dist/harness/agent.js +209 -0
- package/dist/harness/agent.js.map +1 -0
- package/dist/harness/compaction.d.ts.map +1 -1
- package/dist/harness/compaction.js +8 -11
- package/dist/harness/compaction.js.map +1 -1
- package/dist/harness/define.d.ts +18 -0
- package/dist/harness/define.d.ts.map +1 -0
- package/dist/harness/define.js +25 -0
- package/dist/harness/define.js.map +1 -0
- package/dist/harness/events.d.ts +5 -5
- package/dist/harness/events.d.ts.map +1 -1
- package/dist/harness/events.js +6 -6
- package/dist/harness/events.js.map +1 -1
- package/dist/harness/generation.d.ts +2 -4
- package/dist/harness/generation.d.ts.map +1 -1
- package/dist/harness/generation.js +39 -43
- package/dist/harness/generation.js.map +1 -1
- package/dist/harness/harness.d.ts.map +1 -1
- package/dist/harness/harness.js +68 -154
- package/dist/harness/harness.js.map +1 -1
- package/dist/harness/inbox.d.ts +4 -2
- package/dist/harness/inbox.d.ts.map +1 -1
- package/dist/harness/inbox.js +3 -5
- package/dist/harness/inbox.js.map +1 -1
- package/dist/harness/prompt.d.ts +1 -3
- package/dist/harness/prompt.d.ts.map +1 -1
- package/dist/harness/prompt.js +1 -11
- package/dist/harness/prompt.js.map +1 -1
- package/dist/harness/registry.d.ts +2 -3
- package/dist/harness/registry.d.ts.map +1 -1
- package/dist/harness/registry.js +66 -283
- package/dist/harness/registry.js.map +1 -1
- package/dist/harness/scheduler.d.ts +7 -2
- package/dist/harness/scheduler.d.ts.map +1 -1
- package/dist/harness/scheduler.js +41 -27
- package/dist/harness/scheduler.js.map +1 -1
- package/dist/harness/submissions.d.ts +4 -4
- package/dist/harness/submissions.d.ts.map +1 -1
- package/dist/harness/submissions.js +7 -4
- package/dist/harness/submissions.js.map +1 -1
- package/dist/harness/task-graph.d.ts +49 -0
- package/dist/harness/task-graph.d.ts.map +1 -0
- package/dist/harness/task-graph.js +163 -0
- package/dist/harness/task-graph.js.map +1 -0
- package/dist/harness/tool.d.ts +1 -1
- package/dist/harness/tool.d.ts.map +1 -1
- package/dist/harness/tool.js +9 -6
- package/dist/harness/tool.js.map +1 -1
- package/dist/harness/types.d.ts +171 -140
- package/dist/harness/types.d.ts.map +1 -1
- package/dist/harness/types.js.map +1 -1
- package/dist/harness/view.d.ts +1 -1
- package/dist/harness/view.d.ts.map +1 -1
- package/dist/harness/view.js +2 -7
- package/dist/harness/view.js.map +1 -1
- package/dist/index.d.ts +4 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +2 -1
- package/dist/index.js.map +1 -1
- package/dist/session/session.d.ts.map +1 -1
- package/dist/session/session.js +5 -1
- package/dist/session/session.js.map +1 -1
- package/dist/session/transaction.d.ts +2 -2
- package/dist/session/transaction.d.ts.map +1 -1
- package/dist/session/transaction.js +7 -4
- package/dist/session/transaction.js.map +1 -1
- package/dist/storage/sqlite/database.d.ts +26 -19
- package/dist/storage/sqlite/database.d.ts.map +1 -1
- package/dist/storage/sqlite/database.js.map +1 -1
- package/dist/storage/sqlite/index.d.ts +1 -1
- package/dist/storage/sqlite/index.d.ts.map +1 -1
- package/dist/storage/sqlite/index.js.map +1 -1
- package/dist/storage/sqlite/migrations.js +6 -6
- package/dist/storage/sqlite/migrations.js.map +1 -1
- package/dist/storage/sqlite/node.d.ts +23 -7
- package/dist/storage/sqlite/node.d.ts.map +1 -1
- package/dist/storage/sqlite/node.js +129 -47
- package/dist/storage/sqlite/node.js.map +1 -1
- package/dist/storage/sqlite/storage.d.ts +12 -0
- package/dist/storage/sqlite/storage.d.ts.map +1 -1
- package/dist/storage/sqlite/storage.js +121 -129
- package/dist/storage/sqlite/storage.js.map +1 -1
- package/dist/tools/bash.d.ts +1 -1
- package/dist/tools/bash.d.ts.map +1 -1
- package/dist/tools/bash.js +3 -2
- package/dist/tools/bash.js.map +1 -1
- package/dist/tools/edit.d.ts +3 -3
- package/dist/tools/edit.d.ts.map +1 -1
- package/dist/tools/edit.js +5 -4
- package/dist/tools/edit.js.map +1 -1
- package/dist/tools/file-mutation-queue.d.ts +5 -1
- package/dist/tools/file-mutation-queue.d.ts.map +1 -1
- package/dist/tools/file-mutation-queue.js +39 -33
- package/dist/tools/file-mutation-queue.js.map +1 -1
- package/dist/tools/index.d.ts +18 -0
- package/dist/tools/index.d.ts.map +1 -1
- package/dist/tools/index.js +10 -0
- package/dist/tools/index.js.map +1 -1
- package/dist/tools/read.d.ts +3 -3
- package/dist/tools/read.d.ts.map +1 -1
- package/dist/tools/read.js +4 -3
- package/dist/tools/read.js.map +1 -1
- package/dist/tools/write.d.ts +1 -1
- package/dist/tools/write.d.ts.map +1 -1
- package/dist/tools/write.js +3 -2
- package/dist/tools/write.js.map +1 -1
- package/dist/types.d.ts +15 -4
- package/dist/types.d.ts.map +1 -1
- package/dist/types.js.map +1 -1
- package/package.json +25 -15
- package/dist/harness/config.d.ts +0 -29
- package/dist/harness/config.d.ts.map +0 -1
- package/dist/harness/config.js +0 -24
- package/dist/harness/config.js.map +0 -1
- package/docs/chord-delta-findings.md +0 -358
- package/docs/pico-v5-chord-usage.md +0 -373
- package/docs/pico-v5-handoff.md +0 -861
- package/docs/pico-v5.md +0 -4174
package/docs/pico-v5.md
DELETED
|
@@ -1,4174 +0,0 @@
|
|
|
1
|
-
# Pico5 specification
|
|
2
|
-
|
|
3
|
-
Pico5 is a durable, extensible agent harness. This document is normative.
|
|
4
|
-
Pico5 uses existing package types as follows:
|
|
5
|
-
|
|
6
|
-
```ts
|
|
7
|
-
import type {
|
|
8
|
-
AttachedReplicatedState,
|
|
9
|
-
Context,
|
|
10
|
-
Draft,
|
|
11
|
-
JsonRepresentation,
|
|
12
|
-
JsonValue,
|
|
13
|
-
ReplicatedState,
|
|
14
|
-
} from "@earendil-works/chord";
|
|
15
|
-
import type { Op } from "@earendil-works/chord/delta";
|
|
16
|
-
import type {
|
|
17
|
-
AssistantMessage,
|
|
18
|
-
CacheRetention,
|
|
19
|
-
DeferredHandle,
|
|
20
|
-
Message,
|
|
21
|
-
Models,
|
|
22
|
-
ModelThinkingLevel,
|
|
23
|
-
TextContent,
|
|
24
|
-
Tool,
|
|
25
|
-
ToolCall,
|
|
26
|
-
ToolReference,
|
|
27
|
-
ToolResultMessage,
|
|
28
|
-
Transport,
|
|
29
|
-
UserMessage,
|
|
30
|
-
} from "@earendil-works/pi-ai";
|
|
31
|
-
import type { ExecutionEnv } from "@earendil-works/pi-durable/env";
|
|
32
|
-
|
|
33
|
-
type JsonObject = { [key: string]: JsonValue };
|
|
34
|
-
type TaskOutcomeError = { message: string; detail?: JsonValue };
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
Pico5 targets the transcript `SystemMessage` contract from pi-ai PR
|
|
38
|
-
[#9548](https://github.com/earendil-works/pi/pull/9548). `Message` includes that
|
|
39
|
-
type once the PR lands.
|
|
40
|
-
|
|
41
|
-
The core rule is:
|
|
42
|
-
|
|
43
|
-
> A Session atomically commits immutable entries, full task records, and
|
|
44
|
-
> Chord-tracked documents. Only committed state is observable.
|
|
45
|
-
|
|
46
|
-
## 1. Terms and invariants
|
|
47
|
-
|
|
48
|
-
- A **Session** owns one mutation line, conversations, entries, tasks,
|
|
49
|
-
submissions, and documents.
|
|
50
|
-
- A **conversation** is a transcript scope. It may fork another conversation.
|
|
51
|
-
- An **entry** is an immutable transcript record.
|
|
52
|
-
- A **task** is a durable state machine attached to one conversation.
|
|
53
|
-
- A **document** is mutable JSON state represented by Chord operations and
|
|
54
|
-
occasional complete bases.
|
|
55
|
-
- A **definition** is a typed token describing one document or document family.
|
|
56
|
-
- A **source** exposes committed document changes to Chord without exposing a
|
|
57
|
-
mutable object.
|
|
58
|
-
- A **turn** is one assistant response and the tool calls it makes. A **run**
|
|
59
|
-
is the sequence of turns from an admitted input to the final answer; turn
|
|
60
|
-
boundaries (section 6) sit between its turns. A conversation is busy while a
|
|
61
|
-
run is active.
|
|
62
|
-
|
|
63
|
-
Required invariants:
|
|
64
|
-
|
|
65
|
-
1. One Session commit is atomic across all record and document writes.
|
|
66
|
-
2. A document update is published only after its storage commit succeeds.
|
|
67
|
-
3. All visible progress is durable. There is no volatile publication path.
|
|
68
|
-
4. External effects do not run inside the Session mutation transaction.
|
|
69
|
-
5. Entries and IDs are immutable and never reused after a committed write.
|
|
70
|
-
6. Document drafts are fully revoked when their transaction callback settles:
|
|
71
|
-
the Session synchronously prepares or aborts every open change at that point.
|
|
72
|
-
Values assigned into a draft are copied by value and must be strict JSON.
|
|
73
|
-
7. The mutation line remains held through storage settlement and committed-state
|
|
74
|
-
adoption. Commit/close observers run synchronously only to capture immutable
|
|
75
|
-
state; document-state and watch user callbacks run later, off the line.
|
|
76
|
-
8. An uncertain storage failure is fatal to the open Session. It publishes
|
|
77
|
-
nothing and must be reopened. Preparation and checkpoint failures occur before
|
|
78
|
-
storage admission and roll back normally.
|
|
79
|
-
|
|
80
|
-
## 2. Core records
|
|
81
|
-
|
|
82
|
-
The concrete JSON representations may add bookkeeping fields, but must preserve
|
|
83
|
-
these contracts.
|
|
84
|
-
|
|
85
|
-
```ts
|
|
86
|
-
declare const idBrand: unique symbol;
|
|
87
|
-
type Id<Kind extends string, Type = unknown> = number & {
|
|
88
|
-
readonly [idBrand]: {
|
|
89
|
-
readonly kind: Kind;
|
|
90
|
-
readonly type: Type;
|
|
91
|
-
};
|
|
92
|
-
};
|
|
93
|
-
type ConversationId = Id<"conversation">;
|
|
94
|
-
type EntryId = Id<"entry">;
|
|
95
|
-
type TaskId<Result = unknown> = Id<"task", Result>;
|
|
96
|
-
type SubmissionId = Id<"submission">;
|
|
97
|
-
type DocumentId = Id<"document">;
|
|
98
|
-
|
|
99
|
-
declare const seqBrand: unique symbol;
|
|
100
|
-
/** Strictly increases between commits; gaps are permitted. */
|
|
101
|
-
type Seq = number & { readonly [seqBrand]: "sequence" };
|
|
102
|
-
const ROOT_CONVERSATION_ID = 1 as ConversationId;
|
|
103
|
-
|
|
104
|
-
type ConversationOwnership =
|
|
105
|
-
| { readonly kind: "ownerless" }
|
|
106
|
-
| { readonly kind: "task"; readonly taskId: TaskId };
|
|
107
|
-
|
|
108
|
-
type ConversationRecord = {
|
|
109
|
-
readonly id: ConversationId;
|
|
110
|
-
readonly parent?: {
|
|
111
|
-
readonly conversationId: ConversationId;
|
|
112
|
-
readonly at: EntryId;
|
|
113
|
-
};
|
|
114
|
-
readonly owner?: {
|
|
115
|
-
readonly conversationId: ConversationId;
|
|
116
|
-
readonly taskId: TaskId;
|
|
117
|
-
};
|
|
118
|
-
};
|
|
119
|
-
```
|
|
120
|
-
|
|
121
|
-
The referenced pi-ai member is:
|
|
122
|
-
|
|
123
|
-
```ts
|
|
124
|
-
interface SystemMessage {
|
|
125
|
-
role: "system";
|
|
126
|
-
content: string | TextContent[];
|
|
127
|
-
sections?: Record<string, string | null>;
|
|
128
|
-
toolsAdded?: Tool[];
|
|
129
|
-
toolsRemoved?: ToolReference[];
|
|
130
|
-
timestamp: number;
|
|
131
|
-
}
|
|
132
|
-
```
|
|
133
|
-
|
|
134
|
-
In pi-ai, `content` is the base prompt on the leading message and additional
|
|
135
|
-
instruction text on later messages. `sections` is an ordered named patch: a
|
|
136
|
-
string adds or replaces a section, while `null` removes it. `toolsRemoved` is
|
|
137
|
-
applied before `toolsAdded` within one message. Replaying every system message
|
|
138
|
-
in transcript order yields the effective prompt and tool set.
|
|
139
|
-
|
|
140
|
-
Pico always writes `content: ""`. Every prompt part, including the preamble, is
|
|
141
|
-
a named section produced by the registry's system prompt slot (section 7.4).
|
|
142
|
-
|
|
143
|
-
```ts
|
|
144
|
-
type ContextEdit = {
|
|
145
|
-
readonly target: EntryId;
|
|
146
|
-
} & (
|
|
147
|
-
| { readonly action: "omit"; readonly messages?: never }
|
|
148
|
-
| { readonly action: "replace"; readonly messages: readonly Message[] }
|
|
149
|
-
);
|
|
150
|
-
|
|
151
|
-
type EntryRecord = {
|
|
152
|
-
readonly id: EntryId;
|
|
153
|
-
readonly conversationId: ConversationId;
|
|
154
|
-
readonly kind: string;
|
|
155
|
-
readonly model?: readonly Message[];
|
|
156
|
-
readonly data?: JsonValue;
|
|
157
|
-
readonly head?: EntryId;
|
|
158
|
-
readonly edits?: readonly ContextEdit[];
|
|
159
|
-
readonly byTaskId?: TaskId;
|
|
160
|
-
};
|
|
161
|
-
|
|
162
|
-
type EntryDraft = Omit<EntryRecord, "id" | "conversationId" | "byTaskId" | "head"> & {
|
|
163
|
-
readonly head?: EntryId | "self";
|
|
164
|
-
};
|
|
165
|
-
|
|
166
|
-
type SubmissionRecordBase = {
|
|
167
|
-
readonly id: SubmissionId;
|
|
168
|
-
readonly conversationId: ConversationId;
|
|
169
|
-
readonly requestId?: string;
|
|
170
|
-
};
|
|
171
|
-
|
|
172
|
-
type SubmissionRecord =
|
|
173
|
-
| (SubmissionRecordBase & {
|
|
174
|
-
readonly type: "input";
|
|
175
|
-
} & (
|
|
176
|
-
| {
|
|
177
|
-
readonly status: "queued";
|
|
178
|
-
readonly entry?: never;
|
|
179
|
-
readonly answer?: never;
|
|
180
|
-
readonly reason?: never;
|
|
181
|
-
readonly detail?: never;
|
|
182
|
-
}
|
|
183
|
-
| {
|
|
184
|
-
readonly status: "placed";
|
|
185
|
-
readonly entry: EntryId;
|
|
186
|
-
readonly answer?: never;
|
|
187
|
-
readonly reason?: never;
|
|
188
|
-
readonly detail?: never;
|
|
189
|
-
}
|
|
190
|
-
| {
|
|
191
|
-
readonly status: "done";
|
|
192
|
-
readonly entry: EntryId;
|
|
193
|
-
readonly answer: EntryId;
|
|
194
|
-
readonly reason?: never;
|
|
195
|
-
readonly detail?: never;
|
|
196
|
-
}
|
|
197
|
-
| {
|
|
198
|
-
readonly status: "unanswered";
|
|
199
|
-
readonly entry?: EntryId;
|
|
200
|
-
readonly answer?: never;
|
|
201
|
-
readonly reason: string;
|
|
202
|
-
readonly detail?: JsonValue;
|
|
203
|
-
}
|
|
204
|
-
))
|
|
205
|
-
| (SubmissionRecordBase & {
|
|
206
|
-
readonly type: "write";
|
|
207
|
-
} & (
|
|
208
|
-
| {
|
|
209
|
-
readonly status: "queued";
|
|
210
|
-
readonly entry?: never;
|
|
211
|
-
readonly answer?: never;
|
|
212
|
-
readonly reason?: never;
|
|
213
|
-
readonly detail?: never;
|
|
214
|
-
}
|
|
215
|
-
| {
|
|
216
|
-
readonly status: "done";
|
|
217
|
-
readonly entry: EntryId;
|
|
218
|
-
readonly answer?: never;
|
|
219
|
-
readonly reason?: never;
|
|
220
|
-
readonly detail?: never;
|
|
221
|
-
}
|
|
222
|
-
| {
|
|
223
|
-
readonly status: "unanswered";
|
|
224
|
-
readonly entry?: never;
|
|
225
|
-
readonly answer?: never;
|
|
226
|
-
readonly reason: string;
|
|
227
|
-
readonly detail?: JsonValue;
|
|
228
|
-
}
|
|
229
|
-
));
|
|
230
|
-
|
|
231
|
-
type SubmissionSettlement =
|
|
232
|
-
| { readonly status: "done"; readonly answer: EntryId }
|
|
233
|
-
| { readonly status: "unanswered"; readonly reason: string; readonly detail?: JsonValue };
|
|
234
|
-
|
|
235
|
-
type SubmissionCreate = SubmissionRecord extends infer Record
|
|
236
|
-
? Record extends SubmissionRecord
|
|
237
|
-
? Omit<Record, "id">
|
|
238
|
-
: never
|
|
239
|
-
: never;
|
|
240
|
-
```
|
|
241
|
-
|
|
242
|
-
ID brands are compile-time only. IDs and sequences remain ordinary numbers in
|
|
243
|
-
memory, JSON, JSONL, and SQLite. Code applies a brand only at a trusted creation
|
|
244
|
-
or decoding boundary. The distinct sequence brand prevents an entity ID from
|
|
245
|
-
being used as a document commit point.
|
|
246
|
-
|
|
247
|
-
Conversation history parenting and task ownership are separate:
|
|
248
|
-
|
|
249
|
-
- `parent` controls inherited entries and historical documents.
|
|
250
|
-
- `owner` records task attribution and connects scopes for subtree abort and idle traversal. It is not an access-control capability.
|
|
251
|
-
- Creation always supplies `ConversationOwnership` explicitly. The Session
|
|
252
|
-
derives the persisted owner's conversation from the named task; callers never
|
|
253
|
-
construct the persisted owner pair.
|
|
254
|
-
|
|
255
|
-
A conversation's owner remains recorded after the owning task becomes terminal.
|
|
256
|
-
|
|
257
|
-
### 2.1 Entries and context
|
|
258
|
-
|
|
259
|
-
Entry IDs are Session-global and ordered. `parent.at` is an entry in the parent
|
|
260
|
-
history visible to the child.
|
|
261
|
-
|
|
262
|
-
The active transcript is the raw entry range from the newest applicable `head`
|
|
263
|
-
through the tail. A head on an entry changes subsequent context; it does not
|
|
264
|
-
remove older entries from storage. Fork traversal is child entries followed by
|
|
265
|
-
parent entries through each `parent.at` cap.
|
|
266
|
-
|
|
267
|
-
Context derivation:
|
|
268
|
-
|
|
269
|
-
1. Find the newest visible entry `H` at or before the cutoff that has `head`.
|
|
270
|
-
2. Let `from = H.head`, or transcript start when `H` is absent.
|
|
271
|
-
3. Scan visible entries from `from` through the cutoff.
|
|
272
|
-
4. For each target, the newest edit in that range wins. `omit` contributes no
|
|
273
|
-
model messages; `replace` contributes its `messages` instead of the target's.
|
|
274
|
-
5. If `H` exists, context entries are `H` followed by non-head entries in the
|
|
275
|
-
range. Otherwise they are the range.
|
|
276
|
-
6. Keep every positional system message and its tool/section changes.
|
|
277
|
-
7. Move each assistant's tool results, found before the next assistant message,
|
|
278
|
-
directly after it in tool-call order, ahead of any intervening user or system
|
|
279
|
-
message. The provider protocol requires results to follow their call.
|
|
280
|
-
8. Synthesize an error result for every call without one, for example after a
|
|
281
|
-
fork cut or at an interrupted tail. Drop tool results with no preceding call.
|
|
282
|
-
9. Exclude model-less entries and assistant messages with `aborted`, `error`, or
|
|
283
|
-
`deferred` stop reasons from future provider requests.
|
|
284
|
-
|
|
285
|
-
Views carry raw active entries. UI reduction and model-context reduction are
|
|
286
|
-
separate consumers. Older stored history is available through the owning
|
|
287
|
-
`Conversation` object's cursor-based `entries()` scan.
|
|
288
|
-
|
|
289
|
-
### 2.2 Public Harness surface
|
|
290
|
-
|
|
291
|
-
This is the v1 host-facing API. Pico5 is not implemented yet, but implementations
|
|
292
|
-
must expose this shape rather than inventing a different facade during package
|
|
293
|
-
24.
|
|
294
|
-
|
|
295
|
-
```ts
|
|
296
|
-
type ModelRef = {
|
|
297
|
-
readonly provider: string;
|
|
298
|
-
readonly modelId: string;
|
|
299
|
-
};
|
|
300
|
-
|
|
301
|
-
type UserInput = UserMessage["content"];
|
|
302
|
-
|
|
303
|
-
type SubmissionDraft = {
|
|
304
|
-
readonly requestId?: string;
|
|
305
|
-
} & (
|
|
306
|
-
| {
|
|
307
|
-
readonly type: "input";
|
|
308
|
-
readonly content: UserInput;
|
|
309
|
-
readonly whenBusy?: "steer" | "followUp" | "reject";
|
|
310
|
-
readonly entry?: never;
|
|
311
|
-
}
|
|
312
|
-
| {
|
|
313
|
-
readonly type: "write";
|
|
314
|
-
readonly entry: EntryDraft;
|
|
315
|
-
readonly content?: never;
|
|
316
|
-
readonly whenBusy?: never;
|
|
317
|
-
}
|
|
318
|
-
);
|
|
319
|
-
|
|
320
|
-
type InputSubmissionDraft = Extract<SubmissionDraft, { readonly type: "input" }>;
|
|
321
|
-
|
|
322
|
-
/** Runs inside the creating commit, after the conversation and its configuration exist. */
|
|
323
|
-
type ConversationInit = (tx: Tx, conversationId: ConversationId) => void | Promise<void>;
|
|
324
|
-
|
|
325
|
-
type ConversationCreateOptions = {
|
|
326
|
-
readonly ownership: ConversationOwnership;
|
|
327
|
-
readonly init?: ConversationInit;
|
|
328
|
-
};
|
|
329
|
-
|
|
330
|
-
type AnyTask = {
|
|
331
|
-
readonly definition: {
|
|
332
|
-
readonly name: string;
|
|
333
|
-
readonly version: number;
|
|
334
|
-
readonly initial: unknown;
|
|
335
|
-
readonly phases: Readonly<Record<string, unknown>>;
|
|
336
|
-
readonly abort: unknown;
|
|
337
|
-
readonly migrate?: unknown;
|
|
338
|
-
readonly hooks?: object;
|
|
339
|
-
};
|
|
340
|
-
};
|
|
341
|
-
|
|
342
|
-
type HarnessOptions<Tool extends ToolRegistration = ToolRegistration> = {
|
|
343
|
-
readonly models: Models; // the pi-ai Models interface
|
|
344
|
-
readonly registry: RegistryReader<Tool>; // section 7.1
|
|
345
|
-
/** Default execution environment offered to tools (section 7.3). */
|
|
346
|
-
readonly env?: ExecutionEnv;
|
|
347
|
-
readonly now?: () => number;
|
|
348
|
-
readonly onReport?: (error: unknown) => void;
|
|
349
|
-
};
|
|
350
|
-
|
|
351
|
-
/** Curated pi-ai request options; absent fields use pi-ai defaults. */
|
|
352
|
-
type ConversationStreamOptions = {
|
|
353
|
-
transport?: Transport;
|
|
354
|
-
timeoutMs?: number;
|
|
355
|
-
/** Provider/SDK retries inside one request attempt. */
|
|
356
|
-
maxRetries?: number;
|
|
357
|
-
maxRetryDelayMs?: number;
|
|
358
|
-
headers?: Record<string, string>;
|
|
359
|
-
metadata?: JsonObject;
|
|
360
|
-
cacheRetention?: CacheRetention;
|
|
361
|
-
deferred?: boolean | { window?: "15m" | "1h" | "24h" };
|
|
362
|
-
};
|
|
363
|
-
|
|
364
|
-
/** Durable generation attempt retries; the JSON shape of pi-ai `RetryPolicy`. */
|
|
365
|
-
type ConversationRetryPolicy = {
|
|
366
|
-
enabled: boolean;
|
|
367
|
-
maxRetries: number;
|
|
368
|
-
baseDelayMs: number;
|
|
369
|
-
maxAgentDelayMs?: number;
|
|
370
|
-
};
|
|
371
|
-
|
|
372
|
-
/** Automatic compaction thresholds (section 8.7); manual compaction ignores `enabled`. */
|
|
373
|
-
type CompactionPolicy = {
|
|
374
|
-
/** Threshold and overflow compaction. */
|
|
375
|
-
enabled: boolean;
|
|
376
|
-
/** Room kept free for the answer: generation blocks to compact above `contextWindow - reserveTokens`. */
|
|
377
|
-
reserveTokens: number;
|
|
378
|
-
/** Approximate size of the recent context a summary keeps verbatim. */
|
|
379
|
-
keepRecentTokens: number;
|
|
380
|
-
/** Background compaction starts `backgroundTokens` below the blocking threshold; `0` disables it. */
|
|
381
|
-
backgroundTokens: number;
|
|
382
|
-
};
|
|
383
|
-
|
|
384
|
-
type ToolExecutionMode = "parallel" | "sequential";
|
|
385
|
-
|
|
386
|
-
/** How many queued items of one mode a boundary selects (section 6). */
|
|
387
|
-
type QueueMode = "all" | "one-at-a-time";
|
|
388
|
-
|
|
389
|
-
type ConversationConfigState = {
|
|
390
|
-
model?: ModelRef;
|
|
391
|
-
thinkingLevel: ModelThinkingLevel;
|
|
392
|
-
activeTools: string[];
|
|
393
|
-
streamOptions?: ConversationStreamOptions;
|
|
394
|
-
retry?: ConversationRetryPolicy;
|
|
395
|
-
/** Default `parallel`; see section 8.3. */
|
|
396
|
-
toolExecution?: ToolExecutionMode;
|
|
397
|
-
/** Default `one-at-a-time`; see section 6. */
|
|
398
|
-
steeringMode?: QueueMode;
|
|
399
|
-
/** Default `one-at-a-time`; see section 6. */
|
|
400
|
-
followUpMode?: QueueMode;
|
|
401
|
-
/** Default `DEFAULT_COMPACTION_POLICY`; see section 8.7. */
|
|
402
|
-
compaction?: CompactionPolicy;
|
|
403
|
-
};
|
|
404
|
-
|
|
405
|
-
/** Built-in rewindable configuration document; see below. */
|
|
406
|
-
declare const ConversationConfig: RewindableConversationDocToken<ConversationConfigState>;
|
|
407
|
-
|
|
408
|
-
/** Entry whose `data` has type `D`; `never` means the kind carries no data. */
|
|
409
|
-
type TypedEntry<D extends JsonValue> = Omit<EntryRecord, "data"> &
|
|
410
|
-
([D] extends [never] ? { readonly data?: never } : { readonly data: D });
|
|
411
|
-
|
|
412
|
-
type TypedEntryDraft<D extends JsonValue> = Omit<EntryDraft, "kind" | "data"> &
|
|
413
|
-
([D] extends [never] ? { readonly data?: never } : { readonly data: D });
|
|
414
|
-
|
|
415
|
-
interface Entry<D extends JsonValue = never> {
|
|
416
|
-
readonly kind: string;
|
|
417
|
-
is(entry: EntryRecord | undefined): entry is TypedEntry<D>;
|
|
418
|
-
}
|
|
419
|
-
|
|
420
|
-
function defineEntry<D extends JsonValue = never>(kind: string): Entry<D>;
|
|
421
|
-
|
|
422
|
-
type ContextView = {
|
|
423
|
-
readonly head: EntryRecord | undefined;
|
|
424
|
-
readonly entries: readonly EntryRecord[];
|
|
425
|
-
/** Per entry of `entries`, its model messages after edits and rule 9, before rules 7 and 8. */
|
|
426
|
-
readonly contributions: readonly (readonly Message[])[];
|
|
427
|
-
readonly messages: readonly Message[];
|
|
428
|
-
};
|
|
429
|
-
|
|
430
|
-
type SettledSubmissionRecord = SubmissionRecord & {
|
|
431
|
-
readonly status: "done" | "unanswered";
|
|
432
|
-
};
|
|
433
|
-
|
|
434
|
-
interface Submission {
|
|
435
|
-
readonly id: SubmissionId;
|
|
436
|
-
status(context: Context): Promise<SubmissionRecord>;
|
|
437
|
-
wait(context: Context): Promise<SettledSubmissionRecord>;
|
|
438
|
-
abort(context: Context): Promise<"aborted" | "already_placed" | "settled">;
|
|
439
|
-
}
|
|
440
|
-
|
|
441
|
-
type SettledTask<R> = TaskRecord<JsonValue, JsonValue, R> & {
|
|
442
|
-
readonly state: Extract<TaskState<JsonValue, R>, { status: "terminal" }>;
|
|
443
|
-
};
|
|
444
|
-
|
|
445
|
-
type TaskInspection = {
|
|
446
|
-
readonly record: TaskRecord<JsonValue, JsonValue, JsonValue>;
|
|
447
|
-
readonly state:
|
|
448
|
-
| { readonly kind: "running" }
|
|
449
|
-
| { readonly kind: "ready"; readonly migrates: boolean }
|
|
450
|
-
| { readonly kind: "waiting"; readonly on: readonly TaskId[] }
|
|
451
|
-
/** Outcome held until its ordinary owned work drains (section 5.5). */
|
|
452
|
-
| { readonly kind: "completing" }
|
|
453
|
-
| {
|
|
454
|
-
readonly kind: "blocked";
|
|
455
|
-
readonly reason: "missing_task" | "task_too_old" | "migration_failed";
|
|
456
|
-
readonly error?: unknown;
|
|
457
|
-
};
|
|
458
|
-
};
|
|
459
|
-
|
|
460
|
-
type HarnessInspection = {
|
|
461
|
-
readonly scheduling: "paused" | "running" | "closing";
|
|
462
|
-
readonly tasks: readonly TaskInspection[];
|
|
463
|
-
/** Queued and placed submissions in ID order. */
|
|
464
|
-
readonly submissions: readonly SubmissionRecord[];
|
|
465
|
-
readonly registry: readonly RegistryFailure[];
|
|
466
|
-
};
|
|
467
|
-
|
|
468
|
-
type ConversationWatch = WatchHandle<ConversationView>;
|
|
469
|
-
|
|
470
|
-
type HooksOf<K> = K extends Task<infer _I, infer _S, infer _R, infer H>
|
|
471
|
-
? H
|
|
472
|
-
: never;
|
|
473
|
-
|
|
474
|
-
interface Conversation {
|
|
475
|
-
readonly id: ConversationId;
|
|
476
|
-
submit(submission: SubmissionDraft, context: Context): Promise<Submission>;
|
|
477
|
-
|
|
478
|
-
getModel(context: Context): Promise<ModelRef | undefined>;
|
|
479
|
-
setModel(model: ModelRef | undefined, context: Context): Promise<void>;
|
|
480
|
-
getThinkingLevel(context: Context): Promise<ModelThinkingLevel>;
|
|
481
|
-
setThinkingLevel(level: ModelThinkingLevel, context: Context): Promise<void>;
|
|
482
|
-
getActiveTools(context: Context): Promise<readonly string[]>;
|
|
483
|
-
setActiveTools(names: readonly string[], context: Context): Promise<void>;
|
|
484
|
-
getStreamOptions(context: Context): Promise<ConversationStreamOptions>;
|
|
485
|
-
setStreamOptions(options: ConversationStreamOptions, context: Context): Promise<void>;
|
|
486
|
-
getRetryPolicy(context: Context): Promise<ConversationRetryPolicy>;
|
|
487
|
-
setRetryPolicy(policy: ConversationRetryPolicy | undefined, context: Context): Promise<void>;
|
|
488
|
-
getToolExecution(context: Context): Promise<ToolExecutionMode>;
|
|
489
|
-
setToolExecution(mode: ToolExecutionMode | undefined, context: Context): Promise<void>;
|
|
490
|
-
getSteeringMode(context: Context): Promise<QueueMode>;
|
|
491
|
-
setSteeringMode(mode: QueueMode | undefined, context: Context): Promise<void>;
|
|
492
|
-
getFollowUpMode(context: Context): Promise<QueueMode>;
|
|
493
|
-
setFollowUpMode(mode: QueueMode | undefined, context: Context): Promise<void>;
|
|
494
|
-
getCompaction(context: Context): Promise<CompactionPolicy>;
|
|
495
|
-
setCompaction(policy: CompactionPolicy | undefined, context: Context): Promise<void>;
|
|
496
|
-
|
|
497
|
-
commit<T>(
|
|
498
|
-
change: (tx: Tx) => T | Promise<T>,
|
|
499
|
-
context: Context,
|
|
500
|
-
): Promise<T>;
|
|
501
|
-
context(context: Context): Promise<ContextView>;
|
|
502
|
-
entries(
|
|
503
|
-
query: Omit<EntryQuery, "conversationId">,
|
|
504
|
-
limit: number,
|
|
505
|
-
cursor: Cursor | undefined,
|
|
506
|
-
context: Context,
|
|
507
|
-
): Promise<Page<EntryRecord, Cursor>>;
|
|
508
|
-
fork(
|
|
509
|
-
at: EntryId,
|
|
510
|
-
options: ConversationCreateOptions,
|
|
511
|
-
context: Context,
|
|
512
|
-
): Promise<Conversation>;
|
|
513
|
-
compact(instructions: string | undefined, context: Context): Promise<TaskId<CompactionResult>>;
|
|
514
|
-
reset(handoff: string | undefined, context: Context): Promise<void>;
|
|
515
|
-
/** `{ background: true }` also aborts background work under the conversation (section 5.4). */
|
|
516
|
-
abort(context: Context, options?: { readonly background?: boolean }): Promise<void>;
|
|
517
|
-
waitForIdle(context: Context): Promise<void>;
|
|
518
|
-
viewState(context: Context): Promise<AttachedReplicatedState<ConversationView>>;
|
|
519
|
-
watch(context: Context): Promise<ConversationWatch>;
|
|
520
|
-
}
|
|
521
|
-
|
|
522
|
-
interface Harness extends Session {
|
|
523
|
-
resume(): void;
|
|
524
|
-
|
|
525
|
-
root(
|
|
526
|
-
context: Context,
|
|
527
|
-
options?: { readonly init?: ConversationInit },
|
|
528
|
-
): Promise<Conversation>;
|
|
529
|
-
conversation(id: ConversationId, context: Context): Promise<Conversation | undefined>;
|
|
530
|
-
createConversation(options: ConversationCreateOptions, context: Context): Promise<Conversation>;
|
|
531
|
-
|
|
532
|
-
getTask<R>(id: TaskId<R>, context: Context): Promise<TaskRecord<JsonValue, JsonValue, R> | undefined>;
|
|
533
|
-
inspect(context: Context): Promise<HarnessInspection>;
|
|
534
|
-
submission(id: SubmissionId, context: Context): Promise<Submission | undefined>;
|
|
535
|
-
abortSubmission(
|
|
536
|
-
id: SubmissionId,
|
|
537
|
-
context: Context,
|
|
538
|
-
conversationId?: ConversationId,
|
|
539
|
-
): Promise<"aborted" | "already_placed" | "settled" | "not_found">;
|
|
540
|
-
abortTask(id: TaskId, context: Context): Promise<"marked" | "terminal">;
|
|
541
|
-
waitForTask<R>(id: TaskId<R>, context: Context): Promise<SettledTask<R>>;
|
|
542
|
-
waitForIdle(context: Context): Promise<void>;
|
|
543
|
-
/** Session total of every conversation's `pi.usage` (section 8.6). */
|
|
544
|
-
usage(context: Context): Promise<UsageState>;
|
|
545
|
-
}
|
|
546
|
-
|
|
547
|
-
declare const Harness: {
|
|
548
|
-
open<Tool extends ToolRegistration>(
|
|
549
|
-
storage: Storage,
|
|
550
|
-
options: HarnessOptions<Tool>,
|
|
551
|
-
context: Context,
|
|
552
|
-
): Promise<Harness>;
|
|
553
|
-
};
|
|
554
|
-
```
|
|
555
|
-
|
|
556
|
-
This intentionally retains the useful Pico3 host shape. It removes Pico3's
|
|
557
|
-
fixed `rewindable()`/`sticky()` accessors, namespace router, semantic view events,
|
|
558
|
-
and manual Chord view bridge. Typed Pico5 documents and the structural
|
|
559
|
-
conversation watch replace those surfaces. `submit()` durably admits either a
|
|
560
|
-
user input or passive entry write and returns one `Submission` that tracks its
|
|
561
|
-
settlement.
|
|
562
|
-
|
|
563
|
-
`Harness.open()` binds the Session to one application-owned registry (section
|
|
564
|
-
7.1). `createRegistry()` pre-registers the built-in task definitions (section 8);
|
|
565
|
-
they cannot be disposed or replaced, and open rejects a registry whose snapshot
|
|
566
|
-
lacks any of them or the built-in `pi` conversation setup.
|
|
567
|
-
Open changes surviving `running` tasks to `pending`, keeps `waiting` tasks
|
|
568
|
-
waiting, and re-evaluates finalization and `failFast` for `completing` and
|
|
569
|
-
`waiting` tasks (section 5.5). It never migrates or terminalizes a task because
|
|
570
|
-
its definition is missing or unmigratable. Such a task stays `pending` or `waiting` and is **blocked**: the
|
|
571
|
-
scheduler skips it and reconsiders it whenever the registry changes (section
|
|
572
|
-
5.4). Any built-in document touched by recovery migrates through its ordinary
|
|
573
|
-
typed access path. Open does not scan or migrate other documents. No handler
|
|
574
|
-
dispatches during open. Applications should register tasks before open so
|
|
575
|
-
recovered work can resume immediately; registration after open also unblocks it.
|
|
576
|
-
|
|
577
|
-
The root conversation always has reserved ID `ROOT_CONVERSATION_ID` (`1`).
|
|
578
|
-
`root(context, { init })` creates it lazily: when the root is absent, one commit
|
|
579
|
-
on the Session line creates the ownerless root, its default configuration, and
|
|
580
|
-
runs `init(tx, rootId)`. When the root already exists, `init` is ignored and no
|
|
581
|
-
write occurs. Reopen finds the same root by its reserved ID. A conversation with
|
|
582
|
-
no configured model produces a durable `no_model` generation failure.
|
|
583
|
-
|
|
584
|
-
`resume()` is idempotent while running and only enables scheduling. It does not
|
|
585
|
-
repeat open-time reconciliation, and it throws after close. Work that must happen
|
|
586
|
-
before any task runs, such as registration or seeding, happens before
|
|
587
|
-
`resume()`. Calls that ask for progress also enable scheduling, so they never
|
|
588
|
-
wait on a paused Harness: `Conversation.submit()`, `Conversation.compact()`, `Submission.wait()`,
|
|
589
|
-
`Harness.waitForTask()`, `Harness.waitForIdle()`, and
|
|
590
|
-
`Conversation.waitForIdle()`. Recovered work starts with them. A viewer that only
|
|
591
|
-
reads never enables scheduling.
|
|
592
|
-
|
|
593
|
-
`createConversation({ ownership, init })` and
|
|
594
|
-
`fork(at, { ownership, init })` commit atomically: the conversation, its
|
|
595
|
-
explicitly selected ownership, its configuration, and every write made by `init`.
|
|
596
|
-
A first input is an ordinary `submit()` afterward. A request ID makes a retried
|
|
597
|
-
`submit()` to the same conversation exactly-once; a host that must survive a
|
|
598
|
-
crash between the two calls first finds its conversation again, for example
|
|
599
|
-
through a key written by `init`. Host callers must choose ownerless or task
|
|
600
|
-
ownership; neither the Harness nor a conversation handle infers ownership from
|
|
601
|
-
call context. A new independent conversation receives the default configuration
|
|
602
|
-
before `init` runs. A fork receives the configuration visible at `at` through
|
|
603
|
-
the ordinary `asOf` fork policy; `init` may then override it with
|
|
604
|
-
`tx.doc(ConversationConfig, id)`. Other documents follow their own definitions
|
|
605
|
-
without special handling.
|
|
606
|
-
|
|
607
|
-
Every Harness commit that creates or forks a conversation runs the registry's
|
|
608
|
-
conversation setups (section 7.1) in the same commit, whether through the
|
|
609
|
-
conveniences or through raw `tx.createConversation()` and `tx.forkConversation()`,
|
|
610
|
-
for example inside a tool commit. The built-in `pi` setup runs first: an
|
|
611
|
-
independent conversation gets the default configuration and a fork keeps its
|
|
612
|
-
`asOf` copy, and both get empty `pi.live`, `pi.inbox`, and `pi.usage`.
|
|
613
|
-
Applications register setups for their own documents the same way. The
|
|
614
|
-
conveniences add only `init` and its checks, which run after every setup. A conversation created by a plain Session has no
|
|
615
|
-
built-in documents; its configuration reads as
|
|
616
|
-
`ConversationConfig.definition.initial()` until something writes it.
|
|
617
|
-
|
|
618
|
-
The built-in conversation configuration document is final at version 1:
|
|
619
|
-
|
|
620
|
-
| field | value |
|
|
621
|
-
|---|---|
|
|
622
|
-
| kind | `pi.conversation.config` |
|
|
623
|
-
| version | `1` (no migration) |
|
|
624
|
-
| scope/history/fork | conversation, `rewindable`, `asOf` |
|
|
625
|
-
| schema | `ConversationConfigState` |
|
|
626
|
-
| `initial()` | `{ thinkingLevel: "off", activeTools: [] }` |
|
|
627
|
-
| checkpoint | complete base on every change |
|
|
628
|
-
| view mount | `docs["pi.conversation.config"]` |
|
|
629
|
-
|
|
630
|
-
It has no prompt sections: prompt text is produced per request by the registry
|
|
631
|
-
(section 7.4). Any code may edit it with
|
|
632
|
-
`tx.doc(ConversationConfig, id)`, including in the same commit as a create or
|
|
633
|
-
fork. The getters return immutable committed values, falling back to
|
|
634
|
-
`initial()` when the document is absent; they never write. `getStreamOptions()`
|
|
635
|
-
returns `{}` and `getRetryPolicy()` returns the default policy
|
|
636
|
-
`{ enabled: true, maxRetries: 3, baseDelayMs: 2000, maxAgentDelayMs: 60000 }`
|
|
637
|
-
when the field is absent; `setRetryPolicy(undefined)` removes the field.
|
|
638
|
-
`getToolExecution()` returns `parallel` when the field is absent;
|
|
639
|
-
`setToolExecution(undefined)` removes it. `getSteeringMode()` and
|
|
640
|
-
`getFollowUpMode()` return `one-at-a-time` when their field is absent; their
|
|
641
|
-
setters remove the field for `undefined`. `getCompaction()` returns
|
|
642
|
-
`DEFAULT_COMPACTION_POLICY`, `{ enabled: true, reserveTokens: 16384,
|
|
643
|
-
keepRecentTokens: 20000, backgroundTokens: 32768 }`, when the field is absent;
|
|
644
|
-
`setCompaction(undefined)` removes it.
|
|
645
|
-
`streamOptions` are forwarded to every generation request of the conversation;
|
|
646
|
-
`streamOptions.maxRetries` are provider retries inside one request, while `retry`
|
|
647
|
-
governs durable generation attempts (section 8). Each setter performs
|
|
648
|
-
one ordinary Session commit against the document. A setter does not start
|
|
649
|
-
generation or append a system entry. Request preparation later compares the
|
|
650
|
-
desired configuration with transcript history and appends the required
|
|
651
|
-
positional system baseline or delta.
|
|
652
|
-
|
|
653
|
-
Default active tools for a root or independent conversation, including one
|
|
654
|
-
created through `tx.createConversation()` in a Harness commit, are the names of
|
|
655
|
-
every tool registered when its creation commit runs, in registry order.
|
|
656
|
-
`setActiveTools()` rejects the whole operation without a write when a name is
|
|
657
|
-
duplicated or when a name that was not already active in that conversation is
|
|
658
|
-
not registered. The create/fork conveniences check only one thing about `init`
|
|
659
|
-
writes: names that `init` newly activates must be registered. Names that were
|
|
660
|
-
already active are never revalidated, so stale unregistered names survive every
|
|
661
|
-
edit. All other raw document writes are trusted and unchecked, so other writers,
|
|
662
|
-
such as the generation's `addTools` handling, never fault because of registry movement. Readers tolerate what raw
|
|
663
|
-
writes can produce: request preparation offers the first occurrence of a
|
|
664
|
-
duplicated name, and the generation's `tools` phase appends only `addTools`
|
|
665
|
-
names not already active.
|
|
666
|
-
|
|
667
|
-
`init` runs after the conversation creation, which is a table write, so table
|
|
668
|
-
reads inside it throw `ReadAfterWrite` (section 4). Document access remains
|
|
669
|
-
available.
|
|
670
|
-
|
|
671
|
-
The configuration document records the desired loadout; the registry records
|
|
672
|
-
what this process can execute now. An active name may therefore be unregistered,
|
|
673
|
-
for example during extension reload, after restart before registration, or in a
|
|
674
|
-
fork opened by a process that lacks an extension. The document is never
|
|
675
|
-
rewritten because of registry movement, so a re-registered tool becomes available
|
|
676
|
-
again without host action. Hosts derive current availability as active names
|
|
677
|
-
that are currently registered.
|
|
678
|
-
|
|
679
|
-
A missing tool implementation never fails a request. Request preparation offers
|
|
680
|
-
only active names that are currently registered. If the replayed tool state still
|
|
681
|
-
offers an unregistered name, the appended system delta lists it in
|
|
682
|
-
`toolsRemoved`; when the name is registered again, a later delta adds its current
|
|
683
|
-
declaration. When the model calls a tool that its request did not offer, or whose
|
|
684
|
-
implementation is unregistered when its tool task runs, an error tool result with
|
|
685
|
-
an error diagnostic with code `tool_unavailable` (section 7.3) states that the tool is not available,
|
|
686
|
-
and the run continues so the model can react (section 8.3).
|
|
687
|
-
|
|
688
|
-
Configuration takes effect at turn boundaries. Generation preparation reads it
|
|
689
|
-
once per request and fixes that turn's prompt, offered tools, model, and request
|
|
690
|
-
options. A configuration change made while the turn runs, such as deactivating a
|
|
691
|
-
tool the model already called, applies at the next preparation; the current
|
|
692
|
-
turn's calls still execute. The retry policy is the exception: generation reads
|
|
693
|
-
it when it classifies an attempt's result, because it governs the next attempt
|
|
694
|
-
(section 8.3).
|
|
695
|
-
|
|
696
|
-
A `Conversation.commit()` is a Session commit bound to that conversation.
|
|
697
|
-
`tx.createTask()` defaults `TaskOptions.conversationId` to the bound conversation.
|
|
698
|
-
`Conversation.entries()` binds the query to that conversation and paginates its
|
|
699
|
-
fork-aware stored history; callers cannot substitute another conversation ID.
|
|
700
|
-
Generic Session-wide document operations remain available directly on `Harness`
|
|
701
|
-
because `Harness extends Session`.
|
|
702
|
-
|
|
703
|
-
`fork()` requires a concrete visible parent entry and explicit ownership, then
|
|
704
|
-
applies section 3.7.
|
|
705
|
-
`compact()` admits a manual compaction task in one commit and returns its ID,
|
|
706
|
-
not its future summary entry. The task is conversation-owned and not background,
|
|
707
|
-
so `Conversation.abort()` cancels it and idle waits include it. It does not take
|
|
708
|
-
run control, so it does not make the conversation busy: the conversation keeps
|
|
709
|
-
working while it summarizes, and its summary is placed through a write
|
|
710
|
-
submission, at once when idle, otherwise at the next boundary (section 8.7). `reset()` durably admits a write submission of a
|
|
711
|
-
`pi.reset` entry (section 8.1) with `head: "self"`, carrying the handoff text as
|
|
712
|
-
a user message when given, and then resolves; while busy, placement follows
|
|
713
|
-
section 6 and may occur later. Observe its placement through the conversation
|
|
714
|
-
watch. An idle wait does not guarantee placement of queued passive writes.
|
|
715
|
-
|
|
716
|
-
`abortTask()` commits `abortRequested`, then signals and joins the active run;
|
|
717
|
-
the scheduler then starts the abort invocation and marks the foreground-owned
|
|
718
|
-
subtree in its next reconcile commit (section 5.4). `Conversation.abort()` withdraws queued
|
|
719
|
-
input submissions, marks non-background tasks selected by ordinary ownership
|
|
720
|
-
traversal, signals them, and resolves only after that scope is ordinarily idle.
|
|
721
|
-
Passive writes and background subtrees survive. Conversation idle means no live
|
|
722
|
-
non-background task selected from that conversation. Harness idle applies the
|
|
723
|
-
same traversal from every ownerless conversation root. Waiting,
|
|
724
|
-
deadline-blocked, and `completing` work is still live and therefore not idle. Cancelling an idle
|
|
725
|
-
wait aborts only that waiter.
|
|
726
|
-
|
|
727
|
-
Conversation handles are stateless; compare them by `id`. Hosts discover
|
|
728
|
-
conversations through lookups and scans. There is no creation listener and no
|
|
729
|
-
activity view; `inspect()` and the idle waits report live work.
|
|
730
|
-
|
|
731
|
-
`submit()` returns after durable admission, not settlement. An input submission
|
|
732
|
-
creates a user message with the Harness clock's timestamp at placement, which is
|
|
733
|
-
admission when it is placed at once; `whenBusy` defaults to
|
|
734
|
-
`followUp`. A write submission uses the ordered passive path in section 6 and
|
|
735
|
-
never starts generation. `Submission.wait()` settles an input only after its
|
|
736
|
-
answer or terminal failure; it settles a write when the entry is placed or the
|
|
737
|
-
write becomes terminally unplaceable. Cancelling `wait()` only cancels that wait.
|
|
738
|
-
It does not withdraw the submission; `Submission.abort()` is the explicit queued
|
|
739
|
-
withdrawal operation. `Harness.submission()` reacquires a submission after
|
|
740
|
-
reopen; records remain queryable after settlement.
|
|
741
|
-
|
|
742
|
-
`abortTask()` durably requests cancellation and returns `marked` after the mark
|
|
743
|
-
is committed and any active run invocation has joined, or after a blocked task
|
|
744
|
-
without live ordinary owned work has been settled as `orphaned` (section 5.4).
|
|
745
|
-
The abort invocation starts once the task's ordinary owned work is no longer
|
|
746
|
-
live; a `completing` task is only marked, and its final record keeps its held
|
|
747
|
-
outcome with `abortRequested: true` (section 5.5). It does not await terminal
|
|
748
|
-
settlement. `waitForTask()` observes
|
|
749
|
-
the terminal receipt. Aborting an already terminal task returns `terminal`; an
|
|
750
|
-
unknown ID rejects. Explicit task abort includes a background task. Cancelling a
|
|
751
|
-
task or idle wait does not abort work.
|
|
752
|
-
|
|
753
|
-
`inspect()` returns live work at one point on the Session line, for recovery
|
|
754
|
-
decisions after open, viewers, and diagnostics. It writes nothing, does not
|
|
755
|
-
enable scheduling, and runs no task code. Each live task carries its state
|
|
756
|
-
under the current registry: `running` with an active invocation; `completing`
|
|
757
|
-
for a held outcome; `waiting` with the live tasks it still waits for, the live
|
|
758
|
-
part of its `on`, or, when abort-marked, its live ordinary owned work (section
|
|
759
|
-
5.5); `ready` when the next scheduling pass would reserve it, with `migrates`
|
|
760
|
-
when its definition is newer and has `migrate`; or `blocked` (section 5.4). A migration shows as failed only after the scheduler
|
|
761
|
-
tried it, or when the newer definition has no `migrate`; inspection never runs
|
|
762
|
-
one to find out. Blocked reasons are derived, never stored. Queued and placed
|
|
763
|
-
submissions and the registry's wrapper failures complete the view. Finished
|
|
764
|
-
tasks, settled submissions, transcripts, and documents use their own APIs.
|
|
765
|
-
|
|
766
|
-
`Conversation.viewState()` returns its current structural mount as a disposable
|
|
767
|
-
read-only Chord state. `Conversation.watch()` atomically captures that immutable
|
|
768
|
-
revision and registers for later exact complete-commit frames. Its `WatchHandle`
|
|
769
|
-
uses the same serialized asynchronous, bounded-buffer contract as `watchDoc()`
|
|
770
|
-
in section 9.2. Neither carries semantic events or owns a second persistence
|
|
771
|
-
authority.
|
|
772
|
-
|
|
773
|
-
`close()` seals mutation admission and
|
|
774
|
-
task reservation, signals invocations, and stops future watch deliveries. Outside
|
|
775
|
-
the Session line it lets already-admitted storage commits settle, joins
|
|
776
|
-
task/tool/hook invocations, then closes states and storage. Already-running watch
|
|
777
|
-
callbacks remain caller-owned and may finish independently. Close writes no task
|
|
778
|
-
outcome. Handles belong to that open Harness and must be reacquired after reopen.
|
|
779
|
-
|
|
780
|
-
## 3. Documents
|
|
781
|
-
|
|
782
|
-
### 3.1 Definitions
|
|
783
|
-
|
|
784
|
-
Scope directly determines document ownership and lifetime. Only conversation
|
|
785
|
-
documents declare history and fork behavior.
|
|
786
|
-
|
|
787
|
-
```ts
|
|
788
|
-
type LatestConversationSemantics = {
|
|
789
|
-
readonly scope: "conversation";
|
|
790
|
-
readonly history: "latest";
|
|
791
|
-
readonly fork: "current" | "initial";
|
|
792
|
-
};
|
|
793
|
-
|
|
794
|
-
type RewindableConversationSemantics = {
|
|
795
|
-
readonly scope: "conversation";
|
|
796
|
-
readonly history: "rewindable";
|
|
797
|
-
readonly fork: "asOf" | "current" | "initial";
|
|
798
|
-
};
|
|
799
|
-
|
|
800
|
-
type DocumentSemantics =
|
|
801
|
-
| { readonly scope: "session" }
|
|
802
|
-
| LatestConversationSemantics
|
|
803
|
-
| RewindableConversationSemantics
|
|
804
|
-
| { readonly scope: "task" };
|
|
805
|
-
|
|
806
|
-
type CheckpointInfo = {
|
|
807
|
-
/** Deltas already stored after the newest base, excluding this change. */
|
|
808
|
-
readonly deltasSinceBase: number;
|
|
809
|
-
};
|
|
810
|
-
|
|
811
|
-
type CommonDocDefinition<T extends JsonObject> = {
|
|
812
|
-
readonly kind: string;
|
|
813
|
-
readonly version: number;
|
|
814
|
-
initial(): T;
|
|
815
|
-
migrate?(value: JsonObject, fromVersion: number): T;
|
|
816
|
-
checkpointWhen?(value: Readonly<T>, ops: readonly Op[], info: CheckpointInfo): boolean;
|
|
817
|
-
};
|
|
818
|
-
|
|
819
|
-
type DocDefinition<T extends JsonObject> =
|
|
820
|
-
CommonDocDefinition<T> & DocumentSemantics;
|
|
821
|
-
|
|
822
|
-
type DocFamilyDefinition<T extends JsonObject, I extends JsonValue> =
|
|
823
|
-
Omit<CommonDocDefinition<T>, "initial"> & DocumentSemantics & {
|
|
824
|
-
readonly family: true;
|
|
825
|
-
initial(seed: I): T;
|
|
826
|
-
};
|
|
827
|
-
|
|
828
|
-
declare const docType: unique symbol;
|
|
829
|
-
interface DocToken<T extends JsonObject, D extends DocDefinition<T>> {
|
|
830
|
-
readonly definition: D;
|
|
831
|
-
readonly [docType]?: T;
|
|
832
|
-
}
|
|
833
|
-
interface DocFamilyToken<
|
|
834
|
-
T extends JsonObject,
|
|
835
|
-
I extends JsonValue,
|
|
836
|
-
D extends DocFamilyDefinition<T, I>,
|
|
837
|
-
> {
|
|
838
|
-
readonly definition: D;
|
|
839
|
-
readonly [docType]?: T;
|
|
840
|
-
}
|
|
841
|
-
|
|
842
|
-
type SessionDocToken<T extends JsonObject> = DocToken<
|
|
843
|
-
T,
|
|
844
|
-
CommonDocDefinition<T> & { readonly scope: "session" }
|
|
845
|
-
>;
|
|
846
|
-
type ConversationDocToken<T extends JsonObject> = DocToken<
|
|
847
|
-
T,
|
|
848
|
-
CommonDocDefinition<T> & (LatestConversationSemantics | RewindableConversationSemantics)
|
|
849
|
-
>;
|
|
850
|
-
type RewindableConversationDocToken<T extends JsonObject> = DocToken<
|
|
851
|
-
T,
|
|
852
|
-
CommonDocDefinition<T> & RewindableConversationSemantics
|
|
853
|
-
>;
|
|
854
|
-
type TaskDocToken<T extends JsonObject> = DocToken<
|
|
855
|
-
T,
|
|
856
|
-
CommonDocDefinition<T> & { readonly scope: "task" }
|
|
857
|
-
>;
|
|
858
|
-
|
|
859
|
-
type SessionDocFamilyToken<T extends JsonObject, I extends JsonValue> = DocFamilyToken<
|
|
860
|
-
T,
|
|
861
|
-
I,
|
|
862
|
-
DocFamilyDefinition<T, I> & { readonly scope: "session" }
|
|
863
|
-
>;
|
|
864
|
-
type ConversationDocFamilyToken<T extends JsonObject, I extends JsonValue> = DocFamilyToken<
|
|
865
|
-
T,
|
|
866
|
-
I,
|
|
867
|
-
DocFamilyDefinition<T, I> & (LatestConversationSemantics | RewindableConversationSemantics)
|
|
868
|
-
>;
|
|
869
|
-
type RewindableConversationDocFamilyToken<T extends JsonObject, I extends JsonValue> = DocFamilyToken<
|
|
870
|
-
T,
|
|
871
|
-
I,
|
|
872
|
-
DocFamilyDefinition<T, I> & RewindableConversationSemantics
|
|
873
|
-
>;
|
|
874
|
-
type TaskDocFamilyToken<T extends JsonObject, I extends JsonValue> = DocFamilyToken<
|
|
875
|
-
T,
|
|
876
|
-
I,
|
|
877
|
-
DocFamilyDefinition<T, I> & { readonly scope: "task" }
|
|
878
|
-
>;
|
|
879
|
-
|
|
880
|
-
function defineDoc<T extends JsonObject>(
|
|
881
|
-
definition: CommonDocDefinition<T> & { readonly scope: "session" },
|
|
882
|
-
): SessionDocToken<T>;
|
|
883
|
-
function defineDoc<T extends JsonObject>(
|
|
884
|
-
definition: CommonDocDefinition<T> & LatestConversationSemantics,
|
|
885
|
-
): ConversationDocToken<T>;
|
|
886
|
-
function defineDoc<T extends JsonObject>(
|
|
887
|
-
definition: CommonDocDefinition<T> & RewindableConversationSemantics,
|
|
888
|
-
): RewindableConversationDocToken<T>;
|
|
889
|
-
function defineDoc<T extends JsonObject>(
|
|
890
|
-
definition: CommonDocDefinition<T> & { readonly scope: "task" },
|
|
891
|
-
): TaskDocToken<T>;
|
|
892
|
-
|
|
893
|
-
function defineDocFamily<T extends JsonObject, I extends JsonValue>(
|
|
894
|
-
definition: Omit<CommonDocDefinition<T>, "initial"> & {
|
|
895
|
-
readonly family: true;
|
|
896
|
-
readonly scope: "session";
|
|
897
|
-
initial(seed: I): T;
|
|
898
|
-
},
|
|
899
|
-
): SessionDocFamilyToken<T, I>;
|
|
900
|
-
function defineDocFamily<T extends JsonObject, I extends JsonValue>(
|
|
901
|
-
definition: Omit<CommonDocDefinition<T>, "initial"> & LatestConversationSemantics & {
|
|
902
|
-
readonly family: true;
|
|
903
|
-
initial(seed: I): T;
|
|
904
|
-
},
|
|
905
|
-
): ConversationDocFamilyToken<T, I>;
|
|
906
|
-
function defineDocFamily<T extends JsonObject, I extends JsonValue>(
|
|
907
|
-
definition: Omit<CommonDocDefinition<T>, "initial"> & RewindableConversationSemantics & {
|
|
908
|
-
readonly family: true;
|
|
909
|
-
initial(seed: I): T;
|
|
910
|
-
},
|
|
911
|
-
): RewindableConversationDocFamilyToken<T, I>;
|
|
912
|
-
function defineDocFamily<T extends JsonObject, I extends JsonValue>(
|
|
913
|
-
definition: Omit<CommonDocDefinition<T>, "initial"> & {
|
|
914
|
-
readonly family: true;
|
|
915
|
-
readonly scope: "task";
|
|
916
|
-
initial(seed: I): T;
|
|
917
|
-
},
|
|
918
|
-
): TaskDocFamilyToken<T, I>;
|
|
919
|
-
```
|
|
920
|
-
|
|
921
|
-
Validation rules:
|
|
922
|
-
|
|
923
|
-
- Versions are positive integers.
|
|
924
|
-
- Typed access rejects when the token's scope or conversation history/fork policy
|
|
925
|
-
disagrees with the persisted incarnation. A migration cannot reinterpret those
|
|
926
|
-
lifetime semantics.
|
|
927
|
-
- Session documents are current-only and belong to the Session. Closing and
|
|
928
|
-
reopening the Session does not retire them.
|
|
929
|
-
- Conversation documents declare `history` and `fork`; `fork: "asOf"` requires
|
|
930
|
-
`history: "rewindable"`.
|
|
931
|
-
- Task documents are current-only, are never copied by a conversation fork, and
|
|
932
|
-
retire atomically when their task becomes terminal.
|
|
933
|
-
- `initial()` and `migrate()` return JSON objects.
|
|
934
|
-
|
|
935
|
-
`checkpointWhen()` only selects complete storage bases to bound replay. It does
|
|
936
|
-
not change scope, lifetime, history, or fork semantics.
|
|
937
|
-
|
|
938
|
-
Concrete built-in document grouping and semantics are declared when the built-in
|
|
939
|
-
definitions are implemented. The generic document mechanism does not special
|
|
940
|
-
case model, tool, inbox, or presentation state.
|
|
941
|
-
|
|
942
|
-
### 3.2 Records and lifetimes
|
|
943
|
-
|
|
944
|
-
A persisted document instance has one `DocumentRecord`:
|
|
945
|
-
|
|
946
|
-
```ts
|
|
947
|
-
type DocumentRecord = {
|
|
948
|
-
readonly id: DocumentId; // unique incarnation
|
|
949
|
-
readonly kind: string; // stable definition kind
|
|
950
|
-
readonly key?: string; // families only
|
|
951
|
-
readonly createdAt: Seq; // stamped by the committing storage
|
|
952
|
-
readonly retiredAt?: Seq;
|
|
953
|
-
} & (
|
|
954
|
-
| { readonly scope: { readonly kind: "session" } }
|
|
955
|
-
| ({ readonly scope: { readonly kind: "conversation"; readonly conversationId: ConversationId } } & (
|
|
956
|
-
| { readonly history: "latest"; readonly fork: "current" | "initial" }
|
|
957
|
-
| {
|
|
958
|
-
readonly history: "rewindable";
|
|
959
|
-
readonly fork: "asOf" | "current" | "initial";
|
|
960
|
-
}
|
|
961
|
-
))
|
|
962
|
-
| { readonly scope: { readonly kind: "task"; readonly taskId: TaskId } }
|
|
963
|
-
);
|
|
964
|
-
|
|
965
|
-
type DocumentCreate = DocumentRecord extends infer Record
|
|
966
|
-
? Record extends DocumentRecord
|
|
967
|
-
? Omit<Record, "createdAt" | "retiredAt">
|
|
968
|
-
: never
|
|
969
|
-
: never;
|
|
970
|
-
```
|
|
971
|
-
|
|
972
|
-
`id` is never reused. Retiring and recreating the same logical kind, scope, and
|
|
973
|
-
family key creates a new incarnation. Membership is the half-open interval
|
|
974
|
-
`createdAt <= at < retiredAt`; an unretired incarnation has no upper bound. A
|
|
975
|
-
creation retired in the same commit has an empty lifetime.
|
|
976
|
-
|
|
977
|
-
A singleton is identified logically by kind and scope. A family is identified
|
|
978
|
-
logically by kind, scope, and `key`. The record preserves scope and conversation
|
|
979
|
-
history/fork semantics so unavailable extension code does not make existing data
|
|
980
|
-
disappear. Definition versions belong to stored bases and deltas because one
|
|
981
|
-
incarnation may contain records written by multiple definition versions.
|
|
982
|
-
`DocumentCreate` is not another persisted record; it is the same scoped union
|
|
983
|
-
without storage-assigned lifetime fields.
|
|
984
|
-
|
|
985
|
-
### 3.3 Access and creation
|
|
986
|
-
|
|
987
|
-
There is no mutable `session.document()` API.
|
|
988
|
-
|
|
989
|
-
```ts
|
|
990
|
-
interface Session extends DocumentObserver {
|
|
991
|
-
commit<T>(change: (tx: Tx) => T | Promise<T>, context: Context): Promise<T>;
|
|
992
|
-
close(context: Context): Promise<void>;
|
|
993
|
-
subscribeCommits(listener: (publication: CommitPublication, context: Context) => void): () => void;
|
|
994
|
-
subscribeClose(listener: () => void): () => void;
|
|
995
|
-
|
|
996
|
-
snapshot<T extends JsonObject>(token: SessionDocToken<T>, context: Context): Promise<Readonly<T> | undefined>;
|
|
997
|
-
snapshot<T extends JsonObject>(token: ConversationDocToken<T>, conversationId: ConversationId, context: Context): Promise<Readonly<T> | undefined>;
|
|
998
|
-
snapshot<T extends JsonObject>(token: TaskDocToken<T>, taskId: TaskId, context: Context): Promise<Readonly<T> | undefined>;
|
|
999
|
-
snapshot<T extends JsonObject, I extends JsonValue>(token: SessionDocFamilyToken<T, I>, key: string, context: Context): Promise<Readonly<T> | undefined>;
|
|
1000
|
-
snapshot<T extends JsonObject, I extends JsonValue>(token: ConversationDocFamilyToken<T, I>, conversationId: ConversationId, key: string, context: Context): Promise<Readonly<T> | undefined>;
|
|
1001
|
-
snapshot<T extends JsonObject, I extends JsonValue>(token: TaskDocFamilyToken<T, I>, taskId: TaskId, key: string, context: Context): Promise<Readonly<T> | undefined>;
|
|
1002
|
-
|
|
1003
|
-
snapshotAsOf<T extends JsonObject>(token: RewindableConversationDocToken<T>, conversationId: ConversationId, at: EntryId, context: Context): Promise<Readonly<T> | undefined>;
|
|
1004
|
-
snapshotAsOf<T extends JsonObject, I extends JsonValue>(token: RewindableConversationDocFamilyToken<T, I>, conversationId: ConversationId, key: string, at: EntryId, context: Context): Promise<Readonly<T> | undefined>;
|
|
1005
|
-
|
|
1006
|
-
documentState<T extends JsonObject>(token: SessionDocToken<T>, context: Context): Promise<DocumentState<T> | undefined>;
|
|
1007
|
-
documentState<T extends JsonObject>(token: ConversationDocToken<T>, conversationId: ConversationId, context: Context): Promise<DocumentState<T> | undefined>;
|
|
1008
|
-
documentState<T extends JsonObject>(token: TaskDocToken<T>, taskId: TaskId, context: Context): Promise<DocumentState<T> | undefined>;
|
|
1009
|
-
documentState<T extends JsonObject, I extends JsonValue>(token: SessionDocFamilyToken<T, I>, key: string, context: Context): Promise<DocumentState<T> | undefined>;
|
|
1010
|
-
documentState<T extends JsonObject, I extends JsonValue>(token: ConversationDocFamilyToken<T, I>, conversationId: ConversationId, key: string, context: Context): Promise<DocumentState<T> | undefined>;
|
|
1011
|
-
documentState<T extends JsonObject, I extends JsonValue>(token: TaskDocFamilyToken<T, I>, taskId: TaskId, key: string, context: Context): Promise<DocumentState<T> | undefined>;
|
|
1012
|
-
}
|
|
1013
|
-
|
|
1014
|
-
interface Tx {
|
|
1015
|
-
conversation(id: ConversationId): Promise<ConversationRecord | undefined>;
|
|
1016
|
-
entry(id: EntryId): Promise<EntryRecord | undefined>;
|
|
1017
|
-
/** Undefined when the entry is absent or has another kind. */
|
|
1018
|
-
entry<D extends JsonValue>(token: Entry<D>, id: EntryId): Promise<TypedEntry<D> | undefined>;
|
|
1019
|
-
task(id: TaskId): Promise<TaskRecord<JsonValue, JsonValue, JsonValue> | undefined>;
|
|
1020
|
-
scanConversations(query: ConversationQuery, limit: number, cursor?: Cursor): Promise<Page<ConversationRecord, Cursor>>;
|
|
1021
|
-
scanEntries(query: EntryQuery, limit: number, cursor?: Cursor): Promise<Page<EntryRecord, Cursor>>;
|
|
1022
|
-
scanTasks(query: TaskQuery, limit: number, cursor?: Cursor): Promise<Page<TaskRecord<JsonValue, JsonValue, JsonValue>, Cursor>>;
|
|
1023
|
-
|
|
1024
|
-
createConversation(options: { readonly ownership: ConversationOwnership }): Promise<ConversationRecord>;
|
|
1025
|
-
forkConversation(
|
|
1026
|
-
parentConversationId: ConversationId,
|
|
1027
|
-
at: EntryId,
|
|
1028
|
-
options: { readonly ownership: ConversationOwnership },
|
|
1029
|
-
): Promise<ConversationRecord>;
|
|
1030
|
-
appendEntry(conversationId: ConversationId, value: EntryDraft): Promise<EntryRecord>;
|
|
1031
|
-
/** The token supplies `kind` and types `data`. */
|
|
1032
|
-
appendEntry<D extends JsonValue>(
|
|
1033
|
-
token: Entry<D>, conversationId: ConversationId, value: TypedEntryDraft<D>,
|
|
1034
|
-
): Promise<TypedEntry<D>>;
|
|
1035
|
-
createTask<I, S extends { phase: string }, R, H extends object>(
|
|
1036
|
-
task: Task<I, S, R, H>, input: I, options: TaskOptions,
|
|
1037
|
-
): Promise<TaskId<R>>;
|
|
1038
|
-
/** Settle a queued or placed submission; only a placed input can be answered. A settled one stays unchanged. */
|
|
1039
|
-
settleSubmission(id: SubmissionId, settlement: SubmissionSettlement): void;
|
|
1040
|
-
/** Newest visible entry of the conversation that carries a `head`. */
|
|
1041
|
-
latestHeadMarker(conversationId: ConversationId): Promise<(EntryRecord & { readonly head: EntryId }) | undefined>;
|
|
1042
|
-
/** Record a queued submission's placement at `entry`: an input becomes `placed`, a write `done` (section 6). The caller appends the entry and edits `pi.inbox` and `pi.live`. */
|
|
1043
|
-
placeSubmission(id: SubmissionId, entry: EntryId): void;
|
|
1044
|
-
|
|
1045
|
-
doc<T extends JsonObject>(token: SessionDocToken<T>): Promise<Draft<T>>;
|
|
1046
|
-
doc<T extends JsonObject>(token: ConversationDocToken<T>, conversationId: ConversationId): Promise<Draft<T>>;
|
|
1047
|
-
doc<T extends JsonObject>(token: TaskDocToken<T>, taskId: TaskId): Promise<Draft<T>>;
|
|
1048
|
-
doc<T extends JsonObject, I extends JsonValue>(token: SessionDocFamilyToken<T, I>, key: string, seed: I): Promise<Draft<T>>;
|
|
1049
|
-
doc<T extends JsonObject, I extends JsonValue>(token: ConversationDocFamilyToken<T, I>, conversationId: ConversationId, key: string, seed: I): Promise<Draft<T>>;
|
|
1050
|
-
doc<T extends JsonObject, I extends JsonValue>(token: TaskDocFamilyToken<T, I>, taskId: TaskId, key: string, seed: I): Promise<Draft<T>>;
|
|
1051
|
-
|
|
1052
|
-
retireDoc<T extends JsonObject>(token: SessionDocToken<T>): Promise<void>;
|
|
1053
|
-
retireDoc<T extends JsonObject>(token: ConversationDocToken<T>, conversationId: ConversationId): Promise<void>;
|
|
1054
|
-
retireDoc<T extends JsonObject>(token: TaskDocToken<T>, taskId: TaskId): Promise<void>;
|
|
1055
|
-
retireDoc<T extends JsonObject, I extends JsonValue>(token: SessionDocFamilyToken<T, I>, key: string): Promise<void>;
|
|
1056
|
-
retireDoc<T extends JsonObject, I extends JsonValue>(token: ConversationDocFamilyToken<T, I>, conversationId: ConversationId, key: string): Promise<void>;
|
|
1057
|
-
retireDoc<T extends JsonObject, I extends JsonValue>(token: TaskDocFamilyToken<T, I>, taskId: TaskId, key: string): Promise<void>;
|
|
1058
|
-
}
|
|
1059
|
-
```
|
|
1060
|
-
|
|
1061
|
-
ID-creating transaction methods are asynchronous because remote storage may
|
|
1062
|
-
allocate globally unique numeric IDs durably. Conversation creation always
|
|
1063
|
-
requires explicit ownership; no transaction wrapper injects the executing task.
|
|
1064
|
-
For task ownership, the caller supplies only a typed task ID. The Session derives
|
|
1065
|
-
the persisted owner conversation from the task's final candidate record.
|
|
1066
|
-
|
|
1067
|
-
An owner task may be committed or staged earlier in the same transaction. Before
|
|
1068
|
-
Storage admission, the Session rejects a missing, `completing`, terminal, or
|
|
1069
|
-
abort-marked owner of a new task or conversation, judged on the owner's final
|
|
1070
|
-
candidate record in that transaction. A task therefore cannot create owned work
|
|
1071
|
-
in the commit that finishes it. A child task whose explicit `conversationId`
|
|
1072
|
-
differs from its owner's conversation, and a child task with `background: true`,
|
|
1073
|
-
reject the same way. Existing owner edges remain valid when their owners finish
|
|
1074
|
-
afterward. Conversation
|
|
1075
|
-
creation returns an inert record, never an operational handle.
|
|
1076
|
-
`forkConversation()` additionally validates one visible entry and applies section
|
|
1077
|
-
3.7. Only public typed `tx.doc()` is get-or-create. Internal fork copying may create
|
|
1078
|
-
new incarnations directly from stored values without a definition. `tx.doc()`
|
|
1079
|
-
receives the definition token that supplies
|
|
1080
|
-
its static type, initializer, migration, and checkpoint policy. Scope-preserving
|
|
1081
|
-
token overloads require callers to supply only the concrete conversation/task ID
|
|
1082
|
-
and, for a family, its key and creation seed. Ordinary definitions are not
|
|
1083
|
-
registered and ordinary documents are not scanned at open.
|
|
1084
|
-
|
|
1085
|
-
- Existing instances are reconstructed and migrated lazily according to section
|
|
1086
|
-
3.6. A migration reached through `tx.doc()` is staged in its enclosing
|
|
1087
|
-
transaction.
|
|
1088
|
-
- A missing singleton calls the token's `initial()`. A missing family member calls
|
|
1089
|
-
`initial(seed)`. Creation stores an initial base.
|
|
1090
|
-
- The first acquisition of one logical address is memoized before awaiting. Later
|
|
1091
|
-
acquisitions in that transaction return the same draft; for a missing family,
|
|
1092
|
-
the first call's detached seed wins and later seeds are ignored.
|
|
1093
|
-
- `snapshot()`, `snapshotAsOf()`, `documentState()`, and `watchDoc()` never create
|
|
1094
|
-
or persist migration. They return `undefined` when the requested incarnation is
|
|
1095
|
-
absent and migrate a reconstructed value only in memory.
|
|
1096
|
-
- Task-scoped `tx.doc()` validates against the transaction's latest candidate task
|
|
1097
|
-
record, falling back to committed state. This internal validation is not a
|
|
1098
|
-
caller table read and does not trigger `ReadAfterWrite`. Its conversation is
|
|
1099
|
-
derived from the task record.
|
|
1100
|
-
- `retireDoc()` resolves the logical address without creating it. Retirement of
|
|
1101
|
-
an acquired draft persists its final content before retirement. A later
|
|
1102
|
-
`tx.doc()` at that address in the same transaction creates a new incarnation
|
|
1103
|
-
with a new draft and ID.
|
|
1104
|
-
- A terminal candidate rejects later task-document access. Terminal settlement
|
|
1105
|
-
retires both existing task documents and task documents created earlier in the
|
|
1106
|
-
same transaction.
|
|
1107
|
-
- `snapshot()` returns the current shareable immutable revision. Mutation of it
|
|
1108
|
-
or any retained descendant is unsupported; callers that need mutable ownership
|
|
1109
|
-
must copy it first.
|
|
1110
|
-
- `documentState()` returns a disposable read-only Chord state bound to one committed incarnation.
|
|
1111
|
-
- `tx.doc()` returns one revocable Astra overlay `Draft<T>` for the transaction.
|
|
1112
|
-
- Historical reads never create documents in the past.
|
|
1113
|
-
|
|
1114
|
-
`snapshotAsOf()` is available only for rewindable conversation documents. It
|
|
1115
|
-
validates that `at` is visible through the requested conversation's ancestry,
|
|
1116
|
-
selects the ancestor conversation that owns that entry, then finds the logical
|
|
1117
|
-
singleton/family incarnation whose creation/retirement interval contains the
|
|
1118
|
-
entry's commit. It never starts from today's incarnation and never creates an
|
|
1119
|
-
instance. It returns `undefined` when no such instance existed.
|
|
1120
|
-
|
|
1121
|
-
After `B` forks `A` at entry `E`, asking for `B`'s state at inherited `E` reads
|
|
1122
|
-
`A`'s historical instance; `B`'s copied incarnation was created later. If an
|
|
1123
|
-
instance was retired and recreated, historical lookup selects the incarnation
|
|
1124
|
-
alive at the target commit.
|
|
1125
|
-
|
|
1126
|
-
### 3.4 Mutation ownership
|
|
1127
|
-
|
|
1128
|
-
Pico uses Chord Delta's Astra-immutable transaction shape directly:
|
|
1129
|
-
|
|
1130
|
-
```ts
|
|
1131
|
-
interface Prepared<T extends object> {
|
|
1132
|
-
readonly base: T;
|
|
1133
|
-
readonly value: T;
|
|
1134
|
-
readonly ops: readonly Op[];
|
|
1135
|
-
readonly baseRevision: number;
|
|
1136
|
-
abort(): void;
|
|
1137
|
-
}
|
|
1138
|
-
interface Change<T extends object> {
|
|
1139
|
-
readonly state: Draft<T>;
|
|
1140
|
-
prepare(): Prepared<T>;
|
|
1141
|
-
abort(): void;
|
|
1142
|
-
}
|
|
1143
|
-
interface Tracker<T extends object> {
|
|
1144
|
-
readonly value: T;
|
|
1145
|
-
readonly revision: number;
|
|
1146
|
-
beginChange(): Change<T>;
|
|
1147
|
-
prepareReplace(value: T): Prepared<T>;
|
|
1148
|
-
adopt(prepared: Prepared<T>): void;
|
|
1149
|
-
}
|
|
1150
|
-
```
|
|
1151
|
-
|
|
1152
|
-
Each loaded document owns one tracker whose `value` is its current immutable
|
|
1153
|
-
revision. Immutability is a trusted ownership contract, not runtime freezing:
|
|
1154
|
-
a revision and its descendants must never be mutated after transfer to the
|
|
1155
|
-
tracker. `prepare()` revokes the draft, emits detached self-contained
|
|
1156
|
-
operations, and computes `value` with the optimized immutable applier before
|
|
1157
|
-
Storage admission. Unchanged subtrees are structurally shared with `base`.
|
|
1158
|
-
Operation placement payloads may also be shared with `value`; mutating either a
|
|
1159
|
-
prepared operation or an immutable revision is unsupported. Astra preserves an
|
|
1160
|
-
empty operation batch for changes it proves are no-ops, but Pico performs no
|
|
1161
|
-
additional whole-value equality pass for nonempty structural batches.
|
|
1162
|
-
|
|
1163
|
-
`abort()` is idempotent. `adopt()` accepts only a prepared result from that
|
|
1164
|
-
tracker at its current revision, then performs only a synchronous pointer swap
|
|
1165
|
-
to the already-computed immutable `value`. It performs no diffing, application,
|
|
1166
|
-
allocation, or callback. The Session line permits at most one open change per
|
|
1167
|
-
tracker and ensures no result becomes stale between preparation, Storage
|
|
1168
|
-
settlement, and adoption.
|
|
1169
|
-
|
|
1170
|
-
The first `tx.doc()` acquisition calls `tracker.beginChange()` and memoizes that
|
|
1171
|
-
change's draft for the rest of the possibly async Session callback.
|
|
1172
|
-
|
|
1173
|
-
Transaction behavior:
|
|
1174
|
-
|
|
1175
|
-
```text
|
|
1176
|
-
begin transaction
|
|
1177
|
-
acquire and memoize document changes by logical address
|
|
1178
|
-
mutate revocable Astra overlay drafts
|
|
1179
|
-
callback settles
|
|
1180
|
-
seal Tx; synchronously prepare or abort every open change, revoking every draft
|
|
1181
|
-
if any acquisition is pending: abort open changes, reject, then drain and abort it
|
|
1182
|
-
callback fails with no pending acquisition
|
|
1183
|
-
abort every open change; persist and publish nothing
|
|
1184
|
-
callback succeeds with no pending acquisition
|
|
1185
|
-
prepare every open change -> immutable next revision + self-contained Chord Op[]
|
|
1186
|
-
Session evaluates each required/ordinary document write exactly once
|
|
1187
|
-
Storage.commit persists the atomic batch while the Session line remains held
|
|
1188
|
-
storage succeeds
|
|
1189
|
-
adopt every prepared change by pointer swap and enqueue immutable revision/ops publication
|
|
1190
|
-
commit listeners, including conversation view mounts (section 9.3), capture it synchronously
|
|
1191
|
-
release the line; invoke listeners later
|
|
1192
|
-
storage fails
|
|
1193
|
-
abort every prepared change, poison Session, and publish nothing
|
|
1194
|
-
```
|
|
1195
|
-
|
|
1196
|
-
A pending acquisition that resolves after sealing never exposes a draft; its
|
|
1197
|
-
change is aborted and its promise rejects. The Session observes every such
|
|
1198
|
-
settlement before releasing the line. Initializer, migration, and replacement
|
|
1199
|
-
roots come from caller code: the Session copies each one into exclusive kernel
|
|
1200
|
-
ownership, rejecting any value that is not strict JSON, before it becomes an
|
|
1201
|
-
immutable tracker revision. Loaded and fork-copy roots are already detached
|
|
1202
|
-
strict JSON from Storage and enter the tracker without another copy. Chord's
|
|
1203
|
-
`track()` and `prepareReplace()` take ownership in O(1) without traversal, so
|
|
1204
|
-
every root they receive must come from one of these sources. Migration callbacks
|
|
1205
|
-
never receive a live tracker revision.
|
|
1206
|
-
|
|
1207
|
-
Preparation, validation, or checkpoint failure occurs before Storage admission
|
|
1208
|
-
and rolls back normally. The Session performs no
|
|
1209
|
-
strict-JSON walk of prepared operations or selected bases: roots are checked on
|
|
1210
|
-
entry and Chord checks every draft placement, so every revision, operation
|
|
1211
|
-
payload, and base is strict JSON by construction. Tracker branding and `baseRevision` enforce ownership and staleness; the
|
|
1212
|
-
Session never substitutes caller-created prepared values. The prepared immutable
|
|
1213
|
-
candidate itself becomes the adopted and published value. Storage receives that complete value only when the
|
|
1214
|
-
Session selects a base; otherwise it receives only the prepared operation batch.
|
|
1215
|
-
An existing current-version document with an empty batch writes and publishes
|
|
1216
|
-
nothing. A nonempty structural batch whose final value is deeply equal to its
|
|
1217
|
-
base remains a valid durable change and publication. Creation and required
|
|
1218
|
-
version transitions still write a base when their prepared batch is empty; an
|
|
1219
|
-
equal-value version base does not emit a watch update.
|
|
1220
|
-
|
|
1221
|
-
Values assigned into a draft are copied immediately by value. Repeated
|
|
1222
|
-
placements are independent. Chord checks each placement while copying it and
|
|
1223
|
-
throws at the offending assignment, before the draft changes, when the value is
|
|
1224
|
-
not strict JSON: `undefined` array elements or nested object values, non-finite
|
|
1225
|
-
numbers, functions, symbols, bigints, accessors, symbol keys, sparse arrays, or
|
|
1226
|
-
objects whose prototype is neither `Object.prototype` nor `null`. Assigning
|
|
1227
|
-
`undefined` directly to an object property deletes that property. Draft reads, draft writes, and all `Tx` operations
|
|
1228
|
-
reject after the callback settles. The prepared immutable value remains readable
|
|
1229
|
-
by Session-owned checkpoint and Storage preparation.
|
|
1230
|
-
|
|
1231
|
-
```ts
|
|
1232
|
-
let escaped: Draft<LiveState>;
|
|
1233
|
-
await session.commit(async tx => {
|
|
1234
|
-
escaped = await tx.doc(LiveDoc, conversationId);
|
|
1235
|
-
});
|
|
1236
|
-
escaped.message = message; // throws: the draft was revoked
|
|
1237
|
-
```
|
|
1238
|
-
|
|
1239
|
-
Fire-and-forget work that mutates a draft before the owner callback settles may
|
|
1240
|
-
silently enter that transaction and is unsupported.
|
|
1241
|
-
|
|
1242
|
-
### 3.5 Bases and checkpoints
|
|
1243
|
-
|
|
1244
|
-
Creation always stores a complete base.
|
|
1245
|
-
|
|
1246
|
-
For an ordinary later mutation, the definition alone decides whether the
|
|
1247
|
-
storage record is a base:
|
|
1248
|
-
|
|
1249
|
-
```ts
|
|
1250
|
-
const useBase = definition.checkpointWhen?.(candidateValue, ops, { deltasSinceBase }) ?? false;
|
|
1251
|
-
```
|
|
1252
|
-
|
|
1253
|
-
The Session evaluates this predicate exactly once after tracker preparation.
|
|
1254
|
-
`deltasSinceBase` counts the deltas already stored after the incarnation's newest
|
|
1255
|
-
base, excluding the change being evaluated. Storage reports it when it
|
|
1256
|
-
materializes the current value; the Session advances it after each adopted write
|
|
1257
|
-
and resets it after every base, so evaluation performs no Storage read.
|
|
1258
|
-
Creation and version transitions require bases and do not call it. The Session
|
|
1259
|
-
then gives Storage only the selected representation:
|
|
1260
|
-
|
|
1261
|
-
```text
|
|
1262
|
-
required or predicate true -> base with complete value
|
|
1263
|
-
otherwise -> delta with the prepared Chord operation batch
|
|
1264
|
-
```
|
|
1265
|
-
|
|
1266
|
-
A Chord root-replacement operation remains a delta unless the definition selected
|
|
1267
|
-
a checkpoint; it does not authorize reclamation. Predicate failure aborts the
|
|
1268
|
-
prepared transaction before Storage admission. Storage executes no definition
|
|
1269
|
-
code and never receives an unused complete candidate with a selected delta.
|
|
1270
|
-
|
|
1271
|
-
- For Session, task, and latest conversation documents, a committed base permits
|
|
1272
|
-
physical reclamation of older records.
|
|
1273
|
-
- For rewindable conversation documents, bases bound replay but never permit
|
|
1274
|
-
removal of addressable history.
|
|
1275
|
-
- A definition that never checkpoints may create an unbounded replay tail. That
|
|
1276
|
-
is a definition bug, not a backend heuristic.
|
|
1277
|
-
- Storage does not count encoded bytes, compare against `initial()`, or invent
|
|
1278
|
-
checkpoints.
|
|
1279
|
-
|
|
1280
|
-
A definition can bound replay directly:
|
|
1281
|
-
|
|
1282
|
-
```ts
|
|
1283
|
-
checkpointWhen: (_value, _ops, info) => info.deltasSinceBase >= 31
|
|
1284
|
-
```
|
|
1285
|
-
|
|
1286
|
-
A high-churn live document can checkpoint when it becomes empty:
|
|
1287
|
-
|
|
1288
|
-
```ts
|
|
1289
|
-
checkpointWhen: (value, _ops) =>
|
|
1290
|
-
value.message === undefined &&
|
|
1291
|
-
value.tools.length === 0
|
|
1292
|
-
```
|
|
1293
|
-
|
|
1294
|
-
### 3.6 Versions and migrations
|
|
1295
|
-
|
|
1296
|
-
One migration callback handles every supported older version.
|
|
1297
|
-
|
|
1298
|
-
```text
|
|
1299
|
-
stored == token -> use value
|
|
1300
|
-
stored < token -> call migrate(value, storedVersion)
|
|
1301
|
-
stored > token -> reject typed access
|
|
1302
|
-
no migrate -> reject older stored version
|
|
1303
|
-
```
|
|
1304
|
-
|
|
1305
|
-
`migrate()` is pure and returns a complete current-version value. Migration is
|
|
1306
|
-
access-driven: `tx.doc()`, `snapshot()`, `snapshotAsOf()`, `documentState()`, and
|
|
1307
|
-
`watchDoc()` reconstruct and migrate through the token supplied to that call.
|
|
1308
|
-
Harness open does not sweep ordinary documents.
|
|
1309
|
-
|
|
1310
|
-
- Read-only access migrates only in memory and never writes. It may cache a
|
|
1311
|
-
tracker over that migrated immutable revision together with the older stored
|
|
1312
|
-
version marker; the next successful `tx.doc()` access still writes the required
|
|
1313
|
-
current-version base. A cached tracker serves only tokens of the version its value was
|
|
1314
|
-
materialized for. Access with any other version, such as a token from reloaded
|
|
1315
|
-
extension code, drops it and reloads the stored value, so each token migrates
|
|
1316
|
-
from the stored version or rejects a newer one. Migration always starts from a detached stored value.
|
|
1317
|
-
`tx.doc()` stages migration in its enclosing transaction; callback failure
|
|
1318
|
-
persists nothing, and later draft edits coalesce into one final required base.
|
|
1319
|
-
- Rewindable history is not rewritten. Current and historical reconstructed
|
|
1320
|
-
values are migrated after replay.
|
|
1321
|
-
- The first `tx.doc()` transaction after any stored-version migration writes a
|
|
1322
|
-
required current-version base, even when the migrated JSON is deeply equal.
|
|
1323
|
-
- A fork obtains the selected stored value/version from Storage rather than a
|
|
1324
|
-
typed migrated tracker cache. The child copies that stored pair and migrates on
|
|
1325
|
-
later typed access.
|
|
1326
|
-
- Unaccessed documents and documents with unavailable definitions preserve their stored instances,
|
|
1327
|
-
versions, and bytes.
|
|
1328
|
-
|
|
1329
|
-
### 3.7 Forks
|
|
1330
|
-
|
|
1331
|
-
A conversation fork points to one concrete visible entry `E`.
|
|
1332
|
-
|
|
1333
|
-
The child transcript includes entries through `E`, even if the same commit also
|
|
1334
|
-
appended later entries. Document state at `E` is the final state of the commit
|
|
1335
|
-
containing `E`. Different document states require separate commits.
|
|
1336
|
-
|
|
1337
|
-
Each conversation document follows the history/fork policy persisted in its
|
|
1338
|
-
`DocumentRecord`:
|
|
1339
|
-
|
|
1340
|
-
| conversation setting | child value |
|
|
1341
|
-
|---|---|
|
|
1342
|
-
| `fork: "asOf"` | parent value at `E`'s commit |
|
|
1343
|
-
| `fork: "current"` | committed parent value selected when the fork commit runs |
|
|
1344
|
-
| `fork: "initial"` | no copied instance; initializer on first child access |
|
|
1345
|
-
|
|
1346
|
-
`current` and `asOf` copy logically present conversation singleton and family
|
|
1347
|
-
instances, preserving unknown definitions and their stored versions. Copied
|
|
1348
|
-
values become independent child instances with new IDs and initial bases.
|
|
1349
|
-
`initial` copies no instance; first access in the child creates it from the
|
|
1350
|
-
supplied definition. Task documents and tasks are never copied. Session documents
|
|
1351
|
-
remain shared and are not rewindable.
|
|
1352
|
-
|
|
1353
|
-
Fork copying reads committed stored values rather than typed tracker caches. A
|
|
1354
|
-
transaction that creates a fork therefore rejects if it also writes one of the
|
|
1355
|
-
parent's `fork: "current"` documents; commit the parent change first so the fork
|
|
1356
|
-
has one unambiguous stored source revision.
|
|
1357
|
-
|
|
1358
|
-
Forks stage backend-side `document.copy` commands carrying the child create
|
|
1359
|
-
record and an exact source incarnation/point. Storage materializes each source
|
|
1360
|
-
and persists its stored value/version as the child's independent initial base;
|
|
1361
|
-
remote Storage performs this server-side. Every copy reads committed pre-batch
|
|
1362
|
-
source state independent of write-array order. A selected source may not be
|
|
1363
|
-
created, changed, or retired in the same batch. Copying remains atomic with the
|
|
1364
|
-
conversation, overrides, registry writes, and other mutations.
|
|
1365
|
-
|
|
1366
|
-
An unaccessed copy retains only its descriptor in Session memory. Typed access
|
|
1367
|
-
inside the creating transaction lazily reads the detached source, migrates when
|
|
1368
|
-
required, and replaces the copy with one ordinary child create containing the
|
|
1369
|
-
final prepared value. Definition-free copies publish explicit `document.copy`
|
|
1370
|
-
metadata rather than a value. That metadata announces Storage-backed initial
|
|
1371
|
-
state and is never interpreted as a document value. Document states, watches,
|
|
1372
|
-
and mounted views hydrate by capturing their baseline and subscription
|
|
1373
|
-
atomically on the Session line: a later commit already present becomes the
|
|
1374
|
-
baseline, while one committed after registration is delivered. Publications
|
|
1375
|
-
and watches are convergence mechanisms, not audit streams. A mounted aggregate
|
|
1376
|
-
must acquire all of its document baselines and commit subscription in one
|
|
1377
|
-
Session-line operation so it never exposes a mixture from one commit.
|
|
1378
|
-
|
|
1379
|
-
## 4. Transactions and storage ownership
|
|
1380
|
-
|
|
1381
|
-
A Session commit callback may be asynchronous. It owns the Session mutation
|
|
1382
|
-
line through callback execution, preparation, storage settlement, committed
|
|
1383
|
-
baseline adoption, and publication enqueue. Commit observers run on the
|
|
1384
|
-
line; document-state and watch user callbacks run later.
|
|
1385
|
-
External model, process, tool, network, and human effects run outside it.
|
|
1386
|
-
`subscribeCommits()` observes complete immutable publications synchronously on
|
|
1387
|
-
the line after adoption. `subscribeClose()` observes close synchronously when it
|
|
1388
|
-
begins, after admission is sealed; the Harness stops watches and signals task
|
|
1389
|
-
invocations there. Both return idempotent disposers; their listeners must not
|
|
1390
|
-
throw, block, or call Session APIs. Document-state subscribers and watch
|
|
1391
|
-
listeners still run later, off the line.
|
|
1392
|
-
|
|
1393
|
-
```ts
|
|
1394
|
-
await session.commit(async tx => {
|
|
1395
|
-
const conversation = await tx.conversation(conversationId); // table read
|
|
1396
|
-
const live = await tx.doc(LiveDoc, conversationId);
|
|
1397
|
-
|
|
1398
|
-
await tx.appendEntry(conversationId, message); // first table write
|
|
1399
|
-
delete live.message; // document mutation remains valid
|
|
1400
|
-
await tx.createTask(Follow, {}, { ownership: { kind: "conversation" } }); // further table writes are fine
|
|
1401
|
-
}, context);
|
|
1402
|
-
```
|
|
1403
|
-
|
|
1404
|
-
Mutation admission occurs on the Session line before a commit callback starts. Closing seals mutation admission and task
|
|
1405
|
-
reservation. Already-admitted commits settle before storage closes. Once a
|
|
1406
|
-
commit is admitted, caller cancellation does not interrupt storage settlement or
|
|
1407
|
-
undo the commit. Cancelling a close wait does not reopen admission.
|
|
1408
|
-
|
|
1409
|
-
Table rules:
|
|
1410
|
-
|
|
1411
|
-
- Tables are conversations, entries, tasks, and submissions.
|
|
1412
|
-
- Table reads are allowed before the first table write.
|
|
1413
|
-
- Any table read after the first table write throws `ReadAfterWrite`.
|
|
1414
|
-
- Document access and read-your-writes remain available after table writes.
|
|
1415
|
-
- Creation methods return their created ID/record; callers do not read it back.
|
|
1416
|
-
|
|
1417
|
-
Storage ownership:
|
|
1418
|
-
|
|
1419
|
-
- Commit arguments are borrowed until `Storage.commit()` settles.
|
|
1420
|
-
- Anything retained after settlement is detached first.
|
|
1421
|
-
- Memory storage recursively copies retained JSON containers.
|
|
1422
|
-
- JSONL and SQLite detach through serialization and decoded indexes.
|
|
1423
|
-
- Every storage read returns a detached JSON value.
|
|
1424
|
-
- Immutable strings may be shared; mutable arrays and objects may not.
|
|
1425
|
-
|
|
1426
|
-
## 5. Tasks
|
|
1427
|
-
|
|
1428
|
-
### 5.1 Definition
|
|
1429
|
-
|
|
1430
|
-
```ts
|
|
1431
|
-
type TaskOutcome<R> =
|
|
1432
|
-
| { readonly status: "completed"; readonly result: R }
|
|
1433
|
-
| { readonly status: "failed"; readonly error: TaskOutcomeError; readonly result?: R }
|
|
1434
|
-
| { readonly status: "aborted"; readonly reason?: string; readonly result?: R }
|
|
1435
|
-
| { readonly status: "orphaned"; readonly reason: string }
|
|
1436
|
-
| { readonly status: "faulted"; readonly error: TaskOutcomeError };
|
|
1437
|
-
|
|
1438
|
-
type JoinPolicy = "failFast" | "allSettled";
|
|
1439
|
-
|
|
1440
|
-
type TaskState<S, R> =
|
|
1441
|
-
| { readonly status: "pending"; readonly checkpoint: S }
|
|
1442
|
-
| { readonly status: "running"; readonly checkpoint: S }
|
|
1443
|
-
/** Parked without an invocation until every task in `on` is terminal; then resumes at `checkpoint` (section 5.5). */
|
|
1444
|
-
| {
|
|
1445
|
-
readonly status: "waiting";
|
|
1446
|
-
readonly checkpoint: S;
|
|
1447
|
-
readonly on: readonly TaskId[];
|
|
1448
|
-
readonly policy: JoinPolicy;
|
|
1449
|
-
}
|
|
1450
|
-
/** Outcome decided; terminal once no ordinary owned work below is live (section 5.5). Runs no more code. */
|
|
1451
|
-
| { readonly status: "completing"; readonly outcome: TaskOutcome<R> }
|
|
1452
|
-
| { readonly status: "terminal"; readonly outcome: TaskOutcome<R> };
|
|
1453
|
-
|
|
1454
|
-
/** Who owns a task: its conversation (a top-level task) or another task of the same conversation. */
|
|
1455
|
-
type TaskOwnership = { readonly kind: "conversation" } | { readonly kind: "task"; readonly taskId: TaskId };
|
|
1456
|
-
|
|
1457
|
-
type TaskRecord<I, S, R> = {
|
|
1458
|
-
readonly id: TaskId<R>;
|
|
1459
|
-
readonly conversationId: ConversationId;
|
|
1460
|
-
readonly kind: string;
|
|
1461
|
-
readonly version: number;
|
|
1462
|
-
readonly input: I;
|
|
1463
|
-
/** Owning task; absent for a task its conversation owns. Immutable. */
|
|
1464
|
-
readonly owner?: TaskId;
|
|
1465
|
-
readonly background: boolean;
|
|
1466
|
-
readonly abortRequested: boolean;
|
|
1467
|
-
} & (
|
|
1468
|
-
| {
|
|
1469
|
-
readonly state: Extract<TaskState<S, R>, { status: "pending" | "running" | "waiting" }>;
|
|
1470
|
-
readonly memos?: Readonly<Record<string, JsonValue>>;
|
|
1471
|
-
}
|
|
1472
|
-
| {
|
|
1473
|
-
readonly state: Extract<TaskState<S, R>, { status: "completing" | "terminal" }>;
|
|
1474
|
-
readonly memos?: never;
|
|
1475
|
-
}
|
|
1476
|
-
);
|
|
1477
|
-
|
|
1478
|
-
type RunningTask<I, S, R> = TaskRecord<I, S, R> & {
|
|
1479
|
-
readonly state: Extract<TaskState<S, R>, { status: "running" }>;
|
|
1480
|
-
};
|
|
1481
|
-
|
|
1482
|
-
/**
|
|
1483
|
-
* State a task commits for itself: a replacement checkpoint, a wait, or its outcome. A returned `terminal` state
|
|
1484
|
-
* becomes `completing` while ordinary owned work below is live (section 5.5).
|
|
1485
|
-
*/
|
|
1486
|
-
type NextTaskState<S, R> = Extract<TaskState<S, R>, { status: "running" | "waiting" | "terminal" }>;
|
|
1487
|
-
|
|
1488
|
-
interface HookRunner<H extends object> {
|
|
1489
|
-
each<K extends keyof H>(name: K, invoke: (handler: NonNullable<H[K]>) => void | Promise<void>): Promise<void>;
|
|
1490
|
-
}
|
|
1491
|
-
|
|
1492
|
-
type PhaseHandler<I, P, S, R, H extends object> = (
|
|
1493
|
-
task: RunningTask<I, P, R>,
|
|
1494
|
-
runtime: TaskRuntime<I, S, R, H>,
|
|
1495
|
-
context: Context,
|
|
1496
|
-
) => Promise<void>;
|
|
1497
|
-
|
|
1498
|
-
interface TaskRuntime<I, S, R, H extends object> extends DocumentObserver, DocumentReader {
|
|
1499
|
-
readonly taskId: TaskId<R>;
|
|
1500
|
-
readonly conversationId: ConversationId;
|
|
1501
|
-
readonly signal: AbortSignal;
|
|
1502
|
-
/** Registry snapshot of the current phase; refreshed at every phase boundary. */
|
|
1503
|
-
readonly registry: RegistrySnapshot<ToolRegistration>;
|
|
1504
|
-
readonly models: Models;
|
|
1505
|
-
readonly hooks: HookRunner<H>;
|
|
1506
|
-
/** `HarnessOptions.env`; the tool task passes it to tools as `api.env`. */
|
|
1507
|
-
readonly env: ExecutionEnv | undefined;
|
|
1508
|
-
|
|
1509
|
-
commit(
|
|
1510
|
-
change: (
|
|
1511
|
-
tx: Tx,
|
|
1512
|
-
current: RunningTask<I, S, R>,
|
|
1513
|
-
) => NextTaskState<S, R> | undefined | Promise<NextTaskState<S, R> | undefined>,
|
|
1514
|
-
context: Context,
|
|
1515
|
-
): Promise<void>;
|
|
1516
|
-
|
|
1517
|
-
memo<T extends JsonValue>(name: string, context: Context): Promise<T | undefined>;
|
|
1518
|
-
memo<T extends JsonValue>(name: string, candidate: T, context: Context): Promise<T>;
|
|
1519
|
-
conversation(id: ConversationId, context: Context): Promise<ConversationHandle | undefined>;
|
|
1520
|
-
/** Committed task record. */
|
|
1521
|
-
getTask<T>(id: TaskId<T>, context: Context): Promise<TaskRecord<JsonValue, JsonValue, T> | undefined>;
|
|
1522
|
-
/** Terminal receipt; rejects when the invocation ends. */
|
|
1523
|
-
waitForTask<T>(id: TaskId<T>, context: Context): Promise<SettledTask<T>>;
|
|
1524
|
-
/** Outcomes of terminal tasks, in order; rejects when one is not terminal. Used after a wait (section 5.5). */
|
|
1525
|
-
outcomes<T>(ids: readonly TaskId<T>[], context: Context): Promise<TaskOutcome<T>[]>;
|
|
1526
|
-
/** Committed entry visible from the task's conversation. */
|
|
1527
|
-
entry(id: EntryId, context: Context): Promise<EntryRecord | undefined>;
|
|
1528
|
-
entry<D extends JsonValue>(token: Entry<D>, id: EntryId, context: Context): Promise<TypedEntry<D> | undefined>;
|
|
1529
|
-
/** Committed raw active transcript and model context, optionally cut off at `at`. */
|
|
1530
|
-
context(conversationId: ConversationId, context: Context, at?: EntryId): Promise<ContextView>;
|
|
1531
|
-
/** The Harness clock. */
|
|
1532
|
-
now(): number;
|
|
1533
|
-
/** Forward a non-fatal failure to `HarnessOptions.onReport`. */
|
|
1534
|
-
report(error: unknown): void;
|
|
1535
|
-
sleep(until: number, context: Context): Promise<void>;
|
|
1536
|
-
}
|
|
1537
|
-
|
|
1538
|
-
type TaskDefinition<I, S extends { phase: string }, R, H extends object> = {
|
|
1539
|
-
readonly name: string;
|
|
1540
|
-
readonly version: number;
|
|
1541
|
-
initial(input: I): S;
|
|
1542
|
-
readonly phases: {
|
|
1543
|
-
[P in S["phase"]]: PhaseHandler<I, Extract<S, { phase: P }>, S, R, H>;
|
|
1544
|
-
};
|
|
1545
|
-
abort(task: RunningTask<I, S, R>, runtime: TaskRuntime<I, S, R, H>, context: Context): Promise<void>;
|
|
1546
|
-
migrate?(input: JsonValue, checkpoint: JsonValue, fromVersion: number): {
|
|
1547
|
-
input: I;
|
|
1548
|
-
checkpoint: S;
|
|
1549
|
-
};
|
|
1550
|
-
readonly hooks?: H;
|
|
1551
|
-
};
|
|
1552
|
-
|
|
1553
|
-
interface Task<I, S extends { phase: string }, R, H extends object> {
|
|
1554
|
-
readonly definition: TaskDefinition<I, S, R, H>;
|
|
1555
|
-
}
|
|
1556
|
-
|
|
1557
|
-
type TaskOptions = {
|
|
1558
|
-
/** Required: a task always names its owner (section 5.5). */
|
|
1559
|
-
readonly ownership: TaskOwnership;
|
|
1560
|
-
/** Default: the owner task's conversation, or the transaction's bound conversation. */
|
|
1561
|
-
readonly conversationId?: ConversationId;
|
|
1562
|
-
/** Conversation-owned tasks only: excluded from conversation abort and idle, and from cascades. */
|
|
1563
|
-
readonly background?: boolean;
|
|
1564
|
-
};
|
|
1565
|
-
|
|
1566
|
-
function defineTask<I, S extends { phase: string }, R, H extends object = {}>(
|
|
1567
|
-
definition: TaskDefinition<I, S, R, H>,
|
|
1568
|
-
): Task<I, S, R, H>;
|
|
1569
|
-
```
|
|
1570
|
-
|
|
1571
|
-
`TaskId<R>` is the numeric task ID itself with an erased result-type brand. It
|
|
1572
|
-
replaces a separate task-reference wrapper: `createTask()` returns `TaskId<R>`,
|
|
1573
|
-
typed waits infer `R` from that ID, and result-agnostic operations accept
|
|
1574
|
-
`TaskId<unknown>`. A task's `conversationId` is immutable after creation; a
|
|
1575
|
-
replacement that moves an existing task to another conversation rejects before
|
|
1576
|
-
Storage admission so persisted conversation-owner edges cannot become stale.
|
|
1577
|
-
|
|
1578
|
-
The phase map is exhaustive and phase-narrowed. A handler may perform several
|
|
1579
|
-
commits around one effect, but each durable checkpoint is a full replacement.
|
|
1580
|
-
`TaskRuntime.commit()` rereads and gates the current durable task on the Session
|
|
1581
|
-
line before invoking its callback. It rejects when the invocation has ended, the
|
|
1582
|
-
Harness is closing, the task is terminal, or a run invocation's task carries an
|
|
1583
|
-
abort mark. Its `tx.createTask()` defaults to the task's conversation, and every
|
|
1584
|
-
entry it appends records the task as `byTaskId`. When the
|
|
1585
|
-
callback returns a state, the runtime replaces the task's state in the same
|
|
1586
|
-
commit, so the checkpoint or outcome is atomic with the callback's entries,
|
|
1587
|
-
documents, and child tasks and is type-checked against the task's checkpoint and
|
|
1588
|
-
result types. Returning nothing leaves the state unchanged. A terminal or
|
|
1589
|
-
completing state drops the memos. A `waiting` state ends the invocation even
|
|
1590
|
-
when its checkpoint is unchanged; it rejects when `on` names a missing task, the
|
|
1591
|
-
task itself, or a task on its owner chain, which could never finish first, and
|
|
1592
|
-
an empty `on` resumes at the next scheduling pass. `pending` is never returned;
|
|
1593
|
-
only reconciliation and handover write it.
|
|
1594
|
-
|
|
1595
|
-
`Tx` has no task replacement operation. A task changes only its own state,
|
|
1596
|
-
through its runtime. The scheduler owns reservation, reconciliation, handover,
|
|
1597
|
-
faults, and orphaning; `abortTask()` owns abort marks. Other code
|
|
1598
|
-
stops a task with `abortTask()` and reads its result with `waitForTask()`.
|
|
1599
|
-
|
|
1600
|
-
```ts
|
|
1601
|
-
// Intent, effect, outcome.
|
|
1602
|
-
prepare: async (task, runtime, context) => {
|
|
1603
|
-
await runtime.commit(() => ({ status: "running", checkpoint: { phase: "charge", key: newKey() } }), context);
|
|
1604
|
-
},
|
|
1605
|
-
charge: async (task, runtime, context) => {
|
|
1606
|
-
const receipt = await payments.charge(task.state.checkpoint.key); // idempotent by key
|
|
1607
|
-
await runtime.commit(async (tx, current) => {
|
|
1608
|
-
const entry = await tx.appendEntry(current.conversationId, receiptEntry(receipt));
|
|
1609
|
-
return { status: "terminal", outcome: { status: "completed", result: { entryId: entry.id } } };
|
|
1610
|
-
}, context);
|
|
1611
|
-
},
|
|
1612
|
-
// The abort handler decides the outcome; returning without one faults the task.
|
|
1613
|
-
abort: async (task, runtime, context) => {
|
|
1614
|
-
await payments.cancel(task.state.checkpoint);
|
|
1615
|
-
await runtime.commit(() => ({ status: "terminal", outcome: { status: "aborted", reason: "user" } }), context);
|
|
1616
|
-
},
|
|
1617
|
-
```
|
|
1618
|
-
`memo(name, candidate)` is one gated commit; `memo(name)` reads the committed
|
|
1619
|
-
record. `sleep(until)` compares against the Harness `now` clock and rejects when
|
|
1620
|
-
the invocation is signalled or its context is cancelled. Watches acquired through
|
|
1621
|
-
the runtime stop when the invocation ends. `snapshot()`/`snapshotAsOf()` read
|
|
1622
|
-
committed documents, for example to supply `PromptInput.read` (section 7.4).
|
|
1623
|
-
`getTask()` and `entry()` read committed records with one lookup each; `waitForTask()` waits for a terminal receipt,
|
|
1624
|
-
for example a child task created by a tool.
|
|
1625
|
-
`context()` captures its bounds on the Session line and derives the view from
|
|
1626
|
-
immutable entries off the line, like `Conversation.context()`. Like every runtime
|
|
1627
|
-
operation, these reject after the invocation ends.
|
|
1628
|
-
|
|
1629
|
-
Reservation durably changes `pending`, or `waiting` once it may resume, to
|
|
1630
|
-
`running`. One invocation runs phase
|
|
1631
|
-
handlers in sequence; checkpoint commits retain `running`. Before every phase,
|
|
1632
|
-
including the first, the scheduler runs one step callback on the Session line.
|
|
1633
|
-
It reads the committed task and synchronously applies the first matching rule
|
|
1634
|
-
below to the phase that just returned. It writes any fault or handover in the
|
|
1635
|
-
same commit, together with the Harness cleanup for a scheduler-written outcome
|
|
1636
|
-
(section 5.4), which may await document access inside that callback. It ends
|
|
1637
|
-
the invocation there when a rule stops it. A runtime commit the
|
|
1638
|
-
invocation queued earlier therefore either lands before the step and counts, or
|
|
1639
|
-
reaches the line after it and rejects. Before the first phase only rules 1–3
|
|
1640
|
-
apply:
|
|
1641
|
-
|
|
1642
|
-
1. Terminal, `completing`, or `waiting`: stop.
|
|
1643
|
-
2. Session closing: stop; preserve the checkpoint and any abort mark for reopen.
|
|
1644
|
-
3. Run mode with a durable abort mark: end and join the run invocation; a fresh
|
|
1645
|
-
abort invocation starts once the task's ordinary owned work is no longer
|
|
1646
|
-
live (section 5.5).
|
|
1647
|
-
4. Uncaught error: write terminal `faulted`.
|
|
1648
|
-
5. Checkpoint changed, including progress within the same phase: refresh the
|
|
1649
|
-
registry snapshot and either hand over (section 5.4) or invoke the phase
|
|
1650
|
-
handler in the same task invocation.
|
|
1651
|
-
6. Checkpoint unchanged: write terminal `faulted` because no durable progress
|
|
1652
|
-
was made.
|
|
1653
|
-
|
|
1654
|
-
An abort invocation runs its abort handler once. After it settles, a step
|
|
1655
|
-
applies rules 1, 2, and 4; a handler that returns without a terminal outcome
|
|
1656
|
-
faults the task. When Storage rejects a step's fault or handover write, the
|
|
1657
|
-
invocation still ends; the task stays `running` and the next reservation runs it
|
|
1658
|
-
again.
|
|
1659
|
-
|
|
1660
|
-
On open, running-task reconciliation changes surviving `running` tasks back to
|
|
1661
|
-
`pending`, preserving their checkpoint and abort mark; `waiting` tasks stay
|
|
1662
|
-
waiting, and `completing` and `waiting` tasks are re-evaluated (section 5.5). Task migration runs at
|
|
1663
|
-
reservation, atomically with `pending` or `waiting` -> `running`. One callback handles every
|
|
1664
|
-
supported older version. A task whose definition is missing, older than the
|
|
1665
|
-
stored version, or fails migration stays `pending` or `waiting` and blocked until a fitting
|
|
1666
|
-
definition is registered or the task is aborted (section 5.4).
|
|
1667
|
-
`close()` marks the runtime closing, seals admission and reservation, signals
|
|
1668
|
-
invocations, and stops watches. Outside the Session line it settles admitted
|
|
1669
|
-
commits and joins invocations before closing storage. Already-running watch callbacks remain
|
|
1670
|
-
caller-owned. Later runtime commits reject, and close writes no task outcome. Closing
|
|
1671
|
-
starts no fresh phase or abort invocation. It does not set abort marks,
|
|
1672
|
-
terminalize tasks, retire task documents, or publish document retirement. The
|
|
1673
|
-
hosting layer withdraws services and
|
|
1674
|
-
detaches clients; reconnecting to a reopened Session hydrates the last committed
|
|
1675
|
-
state and resumes recovery from its durable checkpoints.
|
|
1676
|
-
|
|
1677
|
-
### 5.2 Effect sandwich
|
|
1678
|
-
|
|
1679
|
-
```text
|
|
1680
|
-
commit intent phase
|
|
1681
|
-
perform external effect
|
|
1682
|
-
commit outcome or next phase
|
|
1683
|
-
```
|
|
1684
|
-
|
|
1685
|
-
Reopening in an intent phase means the effect may have happened. The phase
|
|
1686
|
-
handler retries safely, polls an external handle, or records interruption.
|
|
1687
|
-
Deferred providers are represented by a durable phase containing their handle
|
|
1688
|
-
and next poll time.
|
|
1689
|
-
|
|
1690
|
-
Runtime-owned memos are small first-writer-wins values stored in the live task
|
|
1691
|
-
envelope. Candidate insertion and reading the winner are one Session commit, so
|
|
1692
|
-
concurrent candidates return the same durable winner. Memos survive checkpoints
|
|
1693
|
-
and disappear in the terminal replacement. Bulk progress belongs in a document.
|
|
1694
|
-
|
|
1695
|
-
### 5.3 Terminal tasks
|
|
1696
|
-
|
|
1697
|
-
The terminal task record is the durable result receipt. Its result may directly
|
|
1698
|
-
contain a small value or reference an entry:
|
|
1699
|
-
|
|
1700
|
-
```ts
|
|
1701
|
-
{ status: "completed", result: { entryId: toolResultId } }
|
|
1702
|
-
```
|
|
1703
|
-
|
|
1704
|
-
A terminal transition atomically:
|
|
1705
|
-
|
|
1706
|
-
1. Writes the terminal task record.
|
|
1707
|
-
2. Appends any result entries.
|
|
1708
|
-
3. Retires all documents scoped to that task.
|
|
1709
|
-
4. Resolves any submissions settled by the task.
|
|
1710
|
-
|
|
1711
|
-
A task whose ordinary owned work is live splits this at a `completing` hold
|
|
1712
|
-
(section 5.5, rule 4): a task-written outcome's own writes land at hold, and
|
|
1713
|
-
items 1 and 3 and its waiters follow in the final commit.
|
|
1714
|
-
|
|
1715
|
-
The execution checkpoint and memos disappear from the terminal representation.
|
|
1716
|
-
Terminal records remain queryable for waits, waiters, inspection, and reopen.
|
|
1717
|
-
There are no free-standing task dependencies: ordering comes from a task waiting
|
|
1718
|
-
on other tasks (section 5.5). An abort mark lets pending work reach its abort handler,
|
|
1719
|
-
or its `orphaned` settlement when its definition is unavailable.
|
|
1720
|
-
|
|
1721
|
-
### 5.4 Scheduler, abort, and ownership
|
|
1722
|
-
|
|
1723
|
-
The scheduler serially reserves eligible tasks, then runs handlers off the
|
|
1724
|
-
Session line. One in-memory `TaskInvocation` contains mode, abort controller,
|
|
1725
|
-
and completion promise.
|
|
1726
|
-
|
|
1727
|
-
Abort protocol:
|
|
1728
|
-
|
|
1729
|
-
```text
|
|
1730
|
-
commit abortRequested
|
|
1731
|
-
signal and join active run invocation
|
|
1732
|
-
wait until ordinary owned work is no longer live (section 5.5)
|
|
1733
|
-
start a fresh abort invocation
|
|
1734
|
-
abort handler commits terminal outcome
|
|
1735
|
-
```
|
|
1736
|
-
|
|
1737
|
-
A run invocation may not commit after its durable abort mark appears. Every
|
|
1738
|
-
runtime operation rejects after its owning invocation ends, even while the
|
|
1739
|
-
Session remains open, and the invocation's signal and handler context abort when
|
|
1740
|
-
it ends for any reason. An invocation ends only after its last handler returned,
|
|
1741
|
-
so this cancels only detached leftovers, such as an unawaited `waitForTask()` or
|
|
1742
|
-
a fetch started with the handler's context; they could no longer write anything. Returning from one phase handler does not end an invocation
|
|
1743
|
-
that continues into another phase. Invocation mode is volatile and derived from
|
|
1744
|
-
the durable mark on reopen. Cancelling one caller's `Context` only cancels
|
|
1745
|
-
that call or wait; it does not durably abort shared work unless the invoked API
|
|
1746
|
-
commits an abort mark.
|
|
1747
|
-
|
|
1748
|
-
A task may create owned conversations and owned tasks (section 5.5).
|
|
1749
|
-
Conversations are durable scopes; tasks are the units of live work counted by
|
|
1750
|
-
idle and marked by abort. History parents are irrelevant to ownership traversal.
|
|
1751
|
-
|
|
1752
|
-
Ordinary traversal starts at an explicitly addressed conversation, visits its
|
|
1753
|
-
conversation-owned tasks, follows the tasks each non-background task owns, and
|
|
1754
|
-
follows the conversations each non-background task owns. It follows owner edges after the
|
|
1755
|
-
owner becomes terminal, but a background task is a boundary: ordinary traversal
|
|
1756
|
-
skips that task and its complete owned subtree.
|
|
1757
|
-
Direct conversation operations start inside that conversation regardless of its
|
|
1758
|
-
owner. Directly aborting a live background task includes that task and follows
|
|
1759
|
-
its ordinary owned subtree; nested background owners remain boundaries.
|
|
1760
|
-
`close()` stops every invocation, foreground and background, without writing an
|
|
1761
|
-
outcome, so the work resumes on reopen. A host that must durably cancel
|
|
1762
|
-
everything aborts each live task that `inspect()` lists.
|
|
1763
|
-
|
|
1764
|
-
`Conversation.abort()` withdraws queued inputs and marks live non-background
|
|
1765
|
-
tasks selected by ordinary traversal. With `{ background: true }` it crosses
|
|
1766
|
-
background boundaries: it marks every live task that traversal ignoring the
|
|
1767
|
-
background flag reaches when the operation is admitted and withdraws the queued
|
|
1768
|
-
inputs of every conversation it reaches. It then waits until every task it
|
|
1769
|
-
marked is terminal and the conversation is ordinarily idle; background work
|
|
1770
|
-
created afterwards is neither marked nor awaited. `Conversation.waitForIdle()` waits until
|
|
1771
|
-
that traversal contains no live non-background task. Harness idle performs the
|
|
1772
|
-
same traversal from every ownerless conversation root rather than globally
|
|
1773
|
-
counting tasks, so ordinary work below a background owner does not block it.
|
|
1774
|
-
Explicit `waitForTask()` waits for its referenced task regardless of the task's
|
|
1775
|
-
background flag.
|
|
1776
|
-
|
|
1777
|
-
An abort mark cascades idempotently to foreground-owned work. The durable
|
|
1778
|
-
cancellation intent is a live owner's own record: its abort mark, or a held
|
|
1779
|
-
`completing` outcome other than `completed` (section 5.5). A terminal owner never
|
|
1780
|
-
cascades: it became terminal only after its ordinary owned work drained, so its
|
|
1781
|
-
intent is already applied, and work later started in a conversation it owned,
|
|
1782
|
-
for example a user interrogating a finished subagent, runs normally and is
|
|
1783
|
-
reached by ordinary traversal from above. Queued inputs in its owned
|
|
1784
|
-
conversations stay queued, as after a failed run (section 6). The scheduler derives the marks in a
|
|
1785
|
-
later reconcile commit, and again at open, so a crash in between loses nothing;
|
|
1786
|
-
work already queued on the Session line may make one more commit first. Every
|
|
1787
|
-
live non-background task whose owner chain, walking up through owning tasks and
|
|
1788
|
-
owning conversations, reaches a cancelled live owner, without first crossing a
|
|
1789
|
-
background task that has no cancellation intent of its own, gets an abort mark, including work
|
|
1790
|
-
created there after the cascade. Deriving the marks in their own commit
|
|
1791
|
-
keeps them out of the owner's commit, which may already have written the tasks
|
|
1792
|
-
involved. Every conversation the cascade reaches is treated like
|
|
1793
|
-
`Conversation.abort()`: its queued input submissions become `unanswered` with
|
|
1794
|
-
reason `aborted` and leave its inbox, while queued writes stay. The cascade starts
|
|
1795
|
-
below the cancelled task, so that task's own conversation keeps its queue.
|
|
1796
|
-
Held outcomes `failed`, `faulted`, `orphaned`, and `aborted` record the same
|
|
1797
|
-
durable cancellation intent as an abort mark; `completed` does not. Conversation
|
|
1798
|
-
records and owner edges are never retired with the task. Active invocations are
|
|
1799
|
-
signalled after commit. A terminal receipt guarantees that the task's ordinary
|
|
1800
|
-
owned work drained; background work below it may continue.
|
|
1801
|
-
|
|
1802
|
-
Every run invocation start resolves the task's definition by `TaskRecord.kind`
|
|
1803
|
-
from the scheduler's current registry snapshot:
|
|
1804
|
-
|
|
1805
|
-
```text
|
|
1806
|
-
definition missing -> stay pending, blocked: missing_task
|
|
1807
|
-
stored version newer than it -> stay pending, blocked: task_too_old
|
|
1808
|
-
stored version older than it -> migrate(input, checkpoint, storedVersion)
|
|
1809
|
-
failure -> stay pending, blocked: migration_failed
|
|
1810
|
-
success -> commit migrated record + running atomically
|
|
1811
|
-
equal version -> commit running
|
|
1812
|
-
```
|
|
1813
|
-
|
|
1814
|
-
An abort invocation resolves the definition the same way. When the definition
|
|
1815
|
-
can take the task, its abort handler runs; otherwise the task is orphaned as
|
|
1816
|
-
described below.
|
|
1817
|
-
|
|
1818
|
-
A failed migration is reported once through `onReport` and retried only after
|
|
1819
|
-
the registry resolves a different definition object for the task's kind.
|
|
1820
|
-
|
|
1821
|
-
The Harness never terminalizes a task merely because registry code is missing or
|
|
1822
|
-
incompatible, neither at open nor later. A blocked task without an abort mark
|
|
1823
|
-
keeps its durable record unchanged, remains live, still blocks ordinary idle
|
|
1824
|
-
waits, and is reconsidered whenever the registry changes. The blocked reason is derived
|
|
1825
|
-
runtime state, not a persisted task status. Document migration remains
|
|
1826
|
-
access-driven.
|
|
1827
|
-
|
|
1828
|
-
Aborting a blocked task cannot run its abort handler, because that code is
|
|
1829
|
-
missing or cannot take the task. The Harness therefore settles it as terminal
|
|
1830
|
-
`orphaned` instead of `aborted`: `aborted` means the task's own abort handler
|
|
1831
|
-
ran and decided the outcome, while `orphaned` means no task code ran, so external
|
|
1832
|
-
effects the task started may remain uncleaned. The orphaned `reason` is the
|
|
1833
|
-
blocked reason (`missing_task`, `task_too_old`, or `migration_failed`). When
|
|
1834
|
-
`abortTask()` finds no active invocation, no live ordinary owned work, and a
|
|
1835
|
-
current snapshot that cannot take the task, the marking commit settles it
|
|
1836
|
-
directly; otherwise the scheduler settles it when it would reserve the abort
|
|
1837
|
-
invocation, which is after the owned work drained (section 5.5), so an orphaned
|
|
1838
|
-
outcome never holds. Only an abort (direct, by
|
|
1839
|
-
conversation, or by cascade) orphans a task; a missing definition alone never
|
|
1840
|
-
does. The orphaning commit performs the cleanup the task's code cannot: affected
|
|
1841
|
-
input submissions become unanswered with the reason, any matching active run
|
|
1842
|
-
control is cleared, and task-scoped documents retire. The terminal task record
|
|
1843
|
-
and unanswered submissions carry the reason; the only transcript entry written
|
|
1844
|
-
is the conversion of a committed generation partial (below).
|
|
1845
|
-
Faulting a run task performs the same control/submission cleanup with a
|
|
1846
|
-
`faulted` outcome; a fault while ordinary owned work is live holds, and its
|
|
1847
|
-
cleanup runs in the final commit (section 5.5, rule 4).
|
|
1848
|
-
|
|
1849
|
-
The scheduler knows nothing about runs or task kinds. The Harness, which owns
|
|
1850
|
-
submissions, run control, and the built-in tasks, gives it one hook that the
|
|
1851
|
-
scheduler calls in the commit that makes an outcome it wrote itself (`faulted`
|
|
1852
|
-
and `orphaned`) terminal, which is the final commit after a hold. The hook ignores tasks whose kind is not a built-in
|
|
1853
|
-
run, tool, or compaction kind, so it never creates `pi.live` elsewhere. It settles the run when
|
|
1854
|
-
`pi.live.run` names the task (section 8): a committed generation partial becomes
|
|
1855
|
-
an aborted `pi.assistant` entry, exactly as the generation abort handler converts
|
|
1856
|
-
it, so the transcript keeps what the model produced and `pi.usage` counts its
|
|
1857
|
-
spend (the scheduler's commit has no task scope, so the entry has no
|
|
1858
|
-
`byTaskId`); its inputs become `unanswered` with reason `faulted` (detail: the error
|
|
1859
|
-
message) or the blocked reason; and `run`, `generation`, and `tools` are
|
|
1860
|
-
removed. Faults come from task bugs or malformed provider data, such as a
|
|
1861
|
-
non-JSON value in a response, or a commit the Storage rejected without effect
|
|
1862
|
-
(`StorageRejected`). An uncertain storage failure poisons the Session and writes
|
|
1863
|
-
no outcome. For a `pi.tool` task it marks the task's tool slot `done`
|
|
1864
|
-
without an entry; the run continues, and context derivation synthesizes the
|
|
1865
|
-
missing result (section 2.1). For a `pi.compaction` task it removes the task's
|
|
1866
|
-
compaction status (section 8.7). Outcomes a task commits for itself do their own settlement.
|
|
1867
|
-
|
|
1868
|
-
At every normal phase boundary (section 5.1, rule 5), the step refreshes the
|
|
1869
|
-
invocation's registry snapshot. If the task definition resolved by name is a different object
|
|
1870
|
-
than the one the invocation started with and the new definition can reserve the
|
|
1871
|
-
task (same version, or a higher version with `migrate`), the invocation hands
|
|
1872
|
-
over: the step commits the task back to `pending` with its checkpoint, memos,
|
|
1873
|
-
and abort mark and ends the invocation, and the next reservation starts a fresh invocation under the new
|
|
1874
|
-
definition, applying the reservation rules above. When the definition is missing
|
|
1875
|
-
or cannot reserve the task, the invocation keeps running under its old
|
|
1876
|
-
definition, reports once through `onReport` per resolved definition, and
|
|
1877
|
-
reconsiders at its next boundary. A handler that
|
|
1878
|
-
never settles never hands over.
|
|
1879
|
-
|
|
1880
|
-
### 5.5 Structured concurrency
|
|
1881
|
-
|
|
1882
|
-
Tasks and conversations form one ownership tree. Every task names its owner at
|
|
1883
|
-
creation, `tx.createTask(task, input, { ownership })` with a required
|
|
1884
|
-
`TaskOwnership`: its conversation (`{ kind: "conversation" }`, a top-level task)
|
|
1885
|
-
or a task (`{ kind: "task", taskId }`, a child task). Every conversation is
|
|
1886
|
-
ownerless or owned by a task (section 2.2). A child task always lives in its
|
|
1887
|
-
owner's conversation; only owned conversations cross conversation boundaries.
|
|
1888
|
-
Owner edges are immutable.
|
|
1889
|
-
|
|
1890
|
-
The **ordinary owned work** of a task `T` is every task that ordinary traversal
|
|
1891
|
-
reaches below `T`: the tasks `T` owns, the tasks in the conversations `T` owns,
|
|
1892
|
-
and, transitively, the same for each of those that is not background. A
|
|
1893
|
-
background task and everything below it are excluded. A conversation-owned task
|
|
1894
|
-
may be background; a child task may not.
|
|
1895
|
-
|
|
1896
|
-
Two rules follow from the tree.
|
|
1897
|
-
|
|
1898
|
-
**Abort flows down** (section 5.4). A live owner's cancellation intent (its abort
|
|
1899
|
-
mark, or a held non-`completed` outcome) marks its ordinary owned work. Background
|
|
1900
|
-
tasks are boundaries unless aborted directly or by `Conversation.abort(context,
|
|
1901
|
-
{ background: true })`.
|
|
1902
|
-
|
|
1903
|
-
**Owned work does not outlive its owner's finish.** A task finishes only after
|
|
1904
|
-
its ordinary owned work drained:
|
|
1905
|
-
|
|
1906
|
-
1. A terminal state a task commits, or a terminal outcome the scheduler writes
|
|
1907
|
-
(`faulted`, `orphaned`), while the task's ordinary owned work is live is
|
|
1908
|
-
stored as `{ status: "completing", outcome }` instead. The scheduler writes
|
|
1909
|
-
the final `terminal` record in a later commit once no ordinary owned work is
|
|
1910
|
-
live, evaluated after every commit, including work created after the hold.
|
|
1911
|
-
Whether work is live is judged on the commit's candidate records, so work the
|
|
1912
|
-
finishing commit itself creates, for example a task in a conversation the
|
|
1913
|
-
task owns, holds it. At open it re-evaluates every `completing` task.
|
|
1914
|
-
2. A held outcome is final: no phase, runtime commit, or abort handler runs
|
|
1915
|
-
again, no definition is needed, the task is never reserved or migrated, and
|
|
1916
|
-
`abortTask()` on it only marks it and returns `marked`, which cancels the
|
|
1917
|
-
work below. The final record keeps the held outcome and the mark.
|
|
1918
|
-
3. A held non-`completed` outcome is cancellation intent, so the work below is
|
|
1919
|
-
aborted, drains, and then the task becomes terminal. A held `completed`
|
|
1920
|
-
outcome waits for the work to finish normally.
|
|
1921
|
-
4. Writes split at the hold. A task-written terminal commit lands all of its
|
|
1922
|
-
other writes, such as a tool's result entry and slot, at hold. Only the
|
|
1923
|
-
record's terminal state, task-document retirement, and task waiters are
|
|
1924
|
-
deferred to the final commit. A scheduler-written outcome writes only the
|
|
1925
|
-
record at hold; its Harness cleanup (section 5.4) runs in the final commit,
|
|
1926
|
-
so a faulted run task keeps `pi.live.run` until its tools drained.
|
|
1927
|
-
5. Waiters, idle waits, inspection, and ordinary traversal see a `completing`
|
|
1928
|
-
task as live until its final commit.
|
|
1929
|
-
|
|
1930
|
-
New owned work needs a live owner: `createTask` with task ownership and
|
|
1931
|
-
`createConversation` with task ownership reject an owner that is `completing`,
|
|
1932
|
-
`terminal`, or abort-marked in the commit's final candidate (section 3.3).
|
|
1933
|
-
Conversations stay usable after their owner finished: new runs
|
|
1934
|
-
started there, for example to interrogate a finished subagent, are ordinary work
|
|
1935
|
-
of that conversation, reached by traversal from above, and not part of any
|
|
1936
|
-
finish.
|
|
1937
|
-
|
|
1938
|
-
**Waiting.** A phase may commit `{ status: "waiting", checkpoint, on, policy }`.
|
|
1939
|
-
The task stops without an invocation and resumes at `checkpoint` once every task
|
|
1940
|
-
in `on` is terminal. `on` may name any tasks, including already terminal ones;
|
|
1941
|
-
tasks the waiting task does not own, whose `owner` is not the waiting task,
|
|
1942
|
-
require `policy: "allSettled"`. `on` may not name a missing task, the task
|
|
1943
|
-
itself, or a task on its owner chain (section 5.1). With `failFast`, the first
|
|
1944
|
-
task in `on` that holds or ends with a non-`completed` outcome gives every other
|
|
1945
|
-
live task in `on` an abort mark, in the scheduler's next reconcile commit. The waiting task itself is not marked:
|
|
1946
|
-
it resumes once all of `on` is terminal and reads their outcomes with
|
|
1947
|
-
`runtime.outcomes()`. A task may create children and keep running phases before
|
|
1948
|
-
it waits, and may wait on any subset of its children in sequence.
|
|
1949
|
-
|
|
1950
|
-
**Abort order is bottom-up.** An abort invocation of a task starts only once its
|
|
1951
|
-
ordinary owned work is no longer live, so an abort handler sees final outcomes
|
|
1952
|
-
below it. An abort mark on a waiting task lets it leave the wait early for its
|
|
1953
|
-
abort handler under this same rule; tasks in its `on` that it does not own are
|
|
1954
|
-
not awaited. Ordering is judged on committed records: code a child runs after
|
|
1955
|
-
its terminal commit is not ordered. An abort handler may not return `waiting`
|
|
1956
|
-
and cannot create owned children, because its task is abort-marked (section
|
|
1957
|
-
3.3): it compensates inline, or creates background conversation-owned tasks,
|
|
1958
|
-
which no cascade reaches, and waits for them with `runtime.waitForTask()`,
|
|
1959
|
-
which holds its invocation.
|
|
1960
|
-
|
|
1961
|
-
The cascade of the example `Checkout` task, which waits `failFast` on four
|
|
1962
|
-
`Payment` children: an expired card fails its payment; the three live payments
|
|
1963
|
-
get abort marks and refund in their own abort handlers; once all four are
|
|
1964
|
-
terminal, `Checkout` resumes, reads `[aborted, failed, aborted, aborted]`
|
|
1965
|
-
through `outcomes()`, and decides its own outcome. `abortTask(Checkout)` instead
|
|
1966
|
-
marks `Checkout`, the cascade marks the live payments, their abort handlers run
|
|
1967
|
-
first, then `Checkout.abort` runs and sees their final outcomes.
|
|
1968
|
-
|
|
1969
|
-
## 6. Submissions and inbox
|
|
1970
|
-
|
|
1971
|
-
Submission records back awaitable host objects. Admission is Harness-internal;
|
|
1972
|
-
boundaries place queued submissions with `tx.placeSubmission()`, and run tasks
|
|
1973
|
-
settle the inputs they answer with `tx.settleSubmission()`. Queued submissions
|
|
1974
|
-
wait in the built-in inbox document, an ordered list of tagged items:
|
|
1975
|
-
|
|
1976
|
-
```ts
|
|
1977
|
-
type InboxItem =
|
|
1978
|
-
| { readonly id: SubmissionId; readonly mode: "steer" | "followUp"; readonly content: UserInput }
|
|
1979
|
-
/** `entry` is the write's `EntryDraft`, stored as plain JSON. */
|
|
1980
|
-
| { readonly id: SubmissionId; readonly mode: "write"; readonly entry: JsonObject };
|
|
1981
|
-
|
|
1982
|
-
type InboxState = { items: InboxItem[] };
|
|
1983
|
-
```
|
|
1984
|
-
|
|
1985
|
-
| field | value |
|
|
1986
|
-
|---|---|
|
|
1987
|
-
| kind | `pi.inbox` |
|
|
1988
|
-
| version | `1` |
|
|
1989
|
-
| scope/history/fork | conversation, `latest`, `initial` |
|
|
1990
|
-
| `initial()` | `{ items: [] }` |
|
|
1991
|
-
| checkpoint | complete base whenever `items` is empty |
|
|
1992
|
-
| view mount | `docs["pi.inbox"]` |
|
|
1993
|
-
| created | with every Harness conversation (section 2.2) |
|
|
1994
|
-
|
|
1995
|
-
Items are in ID order. A queued input stores its content; its `pi.user` entry
|
|
1996
|
-
gets the Harness clock's timestamp at placement. Queued submissions belong to
|
|
1997
|
-
their conversation, so a fork starts with an empty inbox.
|
|
1998
|
-
|
|
1999
|
-
Run control lives in the built-in live document `pi.live` (section 8). Its
|
|
2000
|
-
optional `run` value names the task currently responsible for the run and its
|
|
2001
|
-
placed input-submission IDs. `run !== undefined` defines `busy`; get-or-create
|
|
2002
|
-
of the idle document does not. The value remains while generation, tools, and
|
|
2003
|
-
tools hand work to one another: `taskId` names the generation that settles the
|
|
2004
|
-
inputs, including while its tool round runs, then the next
|
|
2005
|
-
generation. Tool tasks never own the run; the current round's tool tasks are
|
|
2006
|
-
listed in `pi.live.tools`. The ID list is mutable state because a
|
|
2007
|
-
boundary adds placed steering inputs to an active run; every terminal path
|
|
2008
|
-
settles exactly the listed inputs.
|
|
2009
|
-
|
|
2010
|
-
Admission and terminal transitions:
|
|
2011
|
-
|
|
2012
|
-
| action | submission state | other writes |
|
|
2013
|
-
|---|---|---|
|
|
2014
|
-
| input, idle with empty inbox | `placed`, with user entry | create run and generation |
|
|
2015
|
-
| write, idle with empty inbox | `done`, with entry | append entry; no run |
|
|
2016
|
-
| input or write, busy or non-empty inbox | `queued` | append inbox item |
|
|
2017
|
-
| boundary places user item | `placed`, with entry | add ID to current or successor run |
|
|
2018
|
-
| boundary places write | `done`, with entry | append entry |
|
|
2019
|
-
| run answers | input `done`, with required answer entry | remove `run`, or hand it to a successor |
|
|
2020
|
-
| run fails or its task aborts | input `unanswered`, with reason | remove `run`; inbox unchanged |
|
|
2021
|
-
| withdraw queued item | `unanswered`, reason `aborted` | remove inbox item |
|
|
2022
|
-
| stale head write | `unanswered`, reason `stale` | remove inbox item |
|
|
2023
|
-
|
|
2024
|
-
`requestId` deduplicates within one conversation before any write; reusing one
|
|
2025
|
-
for the other submission type rejects. A busy input with `whenBusy: "reject"`
|
|
2026
|
-
writes no record and reports `ConversationBusy`. An idle input queues with mode
|
|
2027
|
-
`steer` when `whenBusy` is `steer` and `followUp` otherwise.
|
|
2028
|
-
|
|
2029
|
-
An idle conversation with a non-empty inbox, for example after a failed run,
|
|
2030
|
-
queues every new submission behind the waiting items and runs a final boundary
|
|
2031
|
-
in the same commit. Order is preserved: a follow-up queued before the failure is
|
|
2032
|
-
selected before the new input.
|
|
2033
|
-
|
|
2034
|
-
A `Submission` waits until `done` or `unanswered`; abort withdraws only a still-
|
|
2035
|
-
queued submission, reports `already_placed` for a placed input, and reports
|
|
2036
|
-
`settled` for any terminal submission. Conversation abort withdraws queued
|
|
2037
|
-
steer/follow-up submissions but keeps writes for later placement.
|
|
2038
|
-
|
|
2039
|
-
Boundary selection is deterministic by item ID, with the conversation's
|
|
2040
|
-
configured `steeringMode` and `followUpMode` (section 2.2): `one-at-a-time`
|
|
2041
|
-
selects the first item of that mode, `all` every item of that mode.
|
|
2042
|
-
|
|
2043
|
-
| boundary | write | steer | follow-up |
|
|
2044
|
-
|---|---|---|---|
|
|
2045
|
-
| `postTools` | all | first/all by mode | none |
|
|
2046
|
-
| `final` | all | first/all by mode | first/all by mode |
|
|
2047
|
-
|
|
2048
|
-
A boundary places its selected writes first, in ID order, and then its selected
|
|
2049
|
-
user items, in ID order. A user item queued before a reset or compaction summary
|
|
2050
|
-
therefore lands after it and runs in the new context. Queued user items never
|
|
2051
|
-
become stale because of a head.
|
|
2052
|
-
|
|
2053
|
-
A head write, queued or placed at once, whose target is older than the start of
|
|
2054
|
-
the active range, the newest head marker's `head`, is stale: placing it would bring back history that
|
|
2055
|
-
head cut. Heads placed earlier in the same boundary count: after a queued reset,
|
|
2056
|
-
a queued summary targeting an older entry is stale. A `head: "self"` write is
|
|
2057
|
-
never stale.
|
|
2058
|
-
|
|
2059
|
-
Compaction summaries are head writes (section 8.7), so this rule alone orders
|
|
2060
|
-
compactions by cut position: a summary cutting before the active range start is
|
|
2061
|
-
stale, and one cutting at or after it is placed, whenever it was selected.
|
|
2062
|
-
Example: background compaction B selects at tail 100 and cuts at 70. While it
|
|
2063
|
-
summarizes, a blocking compaction A cuts at 150 and appends its summary. B's
|
|
2064
|
-
summary then settles `stale`, because 70 is older than 150. Had A cut at 60
|
|
2065
|
-
instead, B would be placed after it: B summarized the context at its selection,
|
|
2066
|
-
the then-newest summary plus the entries before 70, which covers everything A's
|
|
2067
|
-
summary covers. That holds while the only heads are compaction summaries and
|
|
2068
|
-
resets, which make older cuts stale; an application edit placed while a
|
|
2069
|
-
compaction summarizes is lost, and an application head can be undone (section
|
|
2070
|
-
12).
|
|
2071
|
-
|
|
2072
|
-
A `postTools` boundary that selects a `head: "self"` write (a reset) behaves as
|
|
2073
|
-
`final`: it also selects follow-ups, and the current run ends with its inputs
|
|
2074
|
-
`unanswered` with reason `reset`, because its context was cut before an answer.
|
|
2075
|
-
A reset placed at a `final` boundary follows the answer, so the inputs are
|
|
2076
|
-
already `done`.
|
|
2077
|
-
|
|
2078
|
-
At ordinary `postTools`, generation continues even with no queued trigger;
|
|
2079
|
-
selected steer IDs join that continuation. A terminating or handoff tool round
|
|
2080
|
-
(section 8.5) uses final behavior instead. At `final`, the current run's placed
|
|
2081
|
-
input submissions settle first, unless an `onYield` continuation keeps them open
|
|
2082
|
-
(below); selected user IDs start one successor generation with those IDs as the
|
|
2083
|
-
new run's inputs. Writes never trigger generation by
|
|
2084
|
-
themselves. A final boundary without user triggers leaves the conversation idle,
|
|
2085
|
-
except for an `onYield` continuation (section 8.3), which applies only when the
|
|
2086
|
-
boundary selected no user item and no reset.
|
|
2087
|
-
|
|
2088
|
-
Only successful run ends apply the final boundary: an answer, `terminate`, or
|
|
2089
|
-
`handoff`. Failure, a run task's abort handler, fault, and orphan settle the
|
|
2090
|
-
run's inputs `unanswered` and leave the inbox alone (`Conversation.abort()`
|
|
2091
|
-
separately withdraws queued user items); the queued items stay visible in
|
|
2092
|
-
`pi.inbox` until the next submission's boundary or their withdrawal.
|
|
2093
|
-
|
|
2094
|
-
Selected and stale items are removed positionally while retained item order is
|
|
2095
|
-
preserved. Chord's Astra operation generator must express scattered removals
|
|
2096
|
-
without carrying retained values; IDs are not substituted for positional inbox
|
|
2097
|
-
semantics.
|
|
2098
|
-
|
|
2099
|
-
## 7. Registry, hooks, tools, and system prompt
|
|
2100
|
-
|
|
2101
|
-
### 7.1 Registry
|
|
2102
|
-
|
|
2103
|
-
Extension code reaches the Harness through one application-owned registry. The
|
|
2104
|
-
registry is process-local, may outlive a Harness, and is the only place tools,
|
|
2105
|
-
tool wrappers, hooks, tasks, and the system prompt are registered. Nothing
|
|
2106
|
-
in it is persisted; durable state stays in conversations, entries, tasks, and
|
|
2107
|
-
documents.
|
|
2108
|
-
|
|
2109
|
-
```ts
|
|
2110
|
-
type DocumentReader = Pick<Session, "snapshot" | "snapshotAsOf">;
|
|
2111
|
-
|
|
2112
|
-
interface Registration {
|
|
2113
|
-
/** Idempotent; removes exactly the registrations this token covers. */
|
|
2114
|
-
dispose(): void;
|
|
2115
|
-
}
|
|
2116
|
-
|
|
2117
|
-
type ToolWrapper<Tool extends ToolRegistration> = (tool: Tool) => Tool;
|
|
2118
|
-
|
|
2119
|
-
type HookScope = {
|
|
2120
|
-
readonly conversationId: ConversationId;
|
|
2121
|
-
/** Also match conversations owned, transitively, by tasks of this conversation. */
|
|
2122
|
-
readonly subtree?: boolean;
|
|
2123
|
-
};
|
|
2124
|
-
|
|
2125
|
-
type PromptInput<Tool extends ToolRegistration> = {
|
|
2126
|
-
readonly conversationId: ConversationId;
|
|
2127
|
-
/** Active and registered tools in configured order, as offered in this request. */
|
|
2128
|
-
readonly tools: readonly Tool[];
|
|
2129
|
-
/** Sections already in effect after replaying the active transcript. */
|
|
2130
|
-
readonly shown: Readonly<Record<string, string>>;
|
|
2131
|
-
readonly model?: ModelRef;
|
|
2132
|
-
readonly thinkingLevel: ModelThinkingLevel;
|
|
2133
|
-
/** Committed document reads. */
|
|
2134
|
-
readonly read: DocumentReader;
|
|
2135
|
-
};
|
|
2136
|
-
|
|
2137
|
-
type PromptSection<Tool extends ToolRegistration> = {
|
|
2138
|
-
readonly key: string;
|
|
2139
|
-
render(
|
|
2140
|
-
input: PromptInput<Tool>,
|
|
2141
|
-
context: Context,
|
|
2142
|
-
): string | undefined | Promise<string | undefined>;
|
|
2143
|
-
/** Default true: wrap the text as `<key>\n...\n</key>`. */
|
|
2144
|
-
readonly tag?: boolean;
|
|
2145
|
-
};
|
|
2146
|
-
|
|
2147
|
-
type PromptSectionWrapper<Tool extends ToolRegistration> = (
|
|
2148
|
-
section: PromptSection<Tool>,
|
|
2149
|
-
) => PromptSection<Tool>;
|
|
2150
|
-
|
|
2151
|
-
/**
|
|
2152
|
-
* Stages the documents every new conversation gets, inside the creating commit, after fork copies and before host
|
|
2153
|
-
* `init`. Table reads throw; a fork (`conversation.parent`) already holds its copied documents. A throw fails the
|
|
2154
|
-
* creation.
|
|
2155
|
-
*/
|
|
2156
|
-
type ConversationSetup = (
|
|
2157
|
-
tx: Tx,
|
|
2158
|
-
conversation: ConversationRecord,
|
|
2159
|
-
registry: RegistrySnapshot<ToolRegistration>,
|
|
2160
|
-
) => void | Promise<void>;
|
|
2161
|
-
|
|
2162
|
-
type RegistryFailure = {
|
|
2163
|
-
readonly kind: "tool" | "section";
|
|
2164
|
-
/** Tool name or section key. */
|
|
2165
|
-
readonly name: string;
|
|
2166
|
-
readonly error: unknown;
|
|
2167
|
-
};
|
|
2168
|
-
|
|
2169
|
-
interface RegistryReader<Tool extends ToolRegistration = ToolRegistration> {
|
|
2170
|
-
/** Immutable view of the whole current registry. */
|
|
2171
|
-
snapshot(): RegistrySnapshot<Tool>;
|
|
2172
|
-
/** Called synchronously after every publication; wakes the scheduler to reconsider blocked tasks. */
|
|
2173
|
-
subscribe(listener: () => void): () => void;
|
|
2174
|
-
}
|
|
2175
|
-
|
|
2176
|
-
interface RegistrySnapshot<Tool extends ToolRegistration> {
|
|
2177
|
-
/** Composed tools in registry order; a tool whose wrapper failed is absent. */
|
|
2178
|
-
tools(): readonly Tool[];
|
|
2179
|
-
tool(name: string): Tool | undefined;
|
|
2180
|
-
/** Base tool names in registry order, including tools whose wrappers fail. */
|
|
2181
|
-
toolNames(): readonly string[];
|
|
2182
|
-
task(name: string): AnyTask | undefined;
|
|
2183
|
-
/** Hooks registered for tasks with `task`'s name, in registry order. */
|
|
2184
|
-
hooks<K extends AnyTask>(task: K): readonly {
|
|
2185
|
-
readonly handlers: Partial<HooksOf<K>>;
|
|
2186
|
-
readonly scope?: HookScope;
|
|
2187
|
-
}[];
|
|
2188
|
-
/** Composed sections in registry order; a section whose wrapper failed is absent. */
|
|
2189
|
-
sections(): readonly PromptSection<Tool>[];
|
|
2190
|
-
/** Wrapper failures of this state; the Harness reports them where it uses them. */
|
|
2191
|
-
failures(): readonly RegistryFailure[];
|
|
2192
|
-
/** Conversation setups in registry order, the built-in `pi` setup first. */
|
|
2193
|
-
conversationSetups(): readonly { readonly key: string; readonly setup: ConversationSetup }[];
|
|
2194
|
-
}
|
|
2195
|
-
|
|
2196
|
-
interface Registry<Tool extends ToolRegistration = ToolRegistration> extends RegistryReader<Tool> {
|
|
2197
|
-
readonly tools: {
|
|
2198
|
-
add(tool: Tool): Registration;
|
|
2199
|
-
wrap(name: string, key: string, wrapper: ToolWrapper<Tool>): Registration;
|
|
2200
|
-
list(): readonly Tool[];
|
|
2201
|
-
};
|
|
2202
|
-
readonly hooks: {
|
|
2203
|
-
add<K extends AnyTask>(
|
|
2204
|
-
task: K,
|
|
2205
|
-
handlers: Partial<HooksOf<K>>,
|
|
2206
|
-
options?: { readonly scope?: HookScope; readonly key?: string },
|
|
2207
|
-
): Registration;
|
|
2208
|
-
};
|
|
2209
|
-
readonly tasks: {
|
|
2210
|
-
add(task: AnyTask): Registration;
|
|
2211
|
-
list(): readonly AnyTask[];
|
|
2212
|
-
};
|
|
2213
|
-
readonly conversations: {
|
|
2214
|
-
setup(key: string, setup: ConversationSetup): Registration;
|
|
2215
|
-
};
|
|
2216
|
-
readonly systemPrompt: {
|
|
2217
|
-
section(
|
|
2218
|
-
key: string,
|
|
2219
|
-
render: PromptSection<Tool>["render"],
|
|
2220
|
-
options?: { readonly tag?: boolean },
|
|
2221
|
-
): Registration;
|
|
2222
|
-
wrap(key: string, wrapperKey: string, wrapper: PromptSectionWrapper<Tool>): Registration;
|
|
2223
|
-
sections(): readonly PromptSection<Tool>[];
|
|
2224
|
-
};
|
|
2225
|
-
batch(register: () => void): Registration;
|
|
2226
|
-
}
|
|
2227
|
-
|
|
2228
|
-
function createRegistry<Tool extends ToolRegistration = ToolRegistration>(): Registry<Tool>;
|
|
2229
|
-
```
|
|
2230
|
-
|
|
2231
|
-
The `Tool` parameter lets applications attach metadata such as prompt snippets to
|
|
2232
|
-
their tools; the Harness only relies on `ToolRegistration`. Pi-ai declarations
|
|
2233
|
-
derived from a registered tool keep only pi-ai `Tool` fields (`toToolDeclaration`),
|
|
2234
|
-
so application metadata never enters the transcript.
|
|
2235
|
-
|
|
2236
|
-
Registration rules:
|
|
2237
|
-
|
|
2238
|
-
- `createRegistry()` starts with the built-in task definitions of section 8 and
|
|
2239
|
-
the built-in `pi` conversation setup registered first. They have no
|
|
2240
|
-
`Registration`, so they cannot be disposed, and the duplicate rule rejects
|
|
2241
|
-
another task with a built-in name or another `pi` setup. `tasks.list()`
|
|
2242
|
-
includes them. Hooks register against built-in tasks like any other task.
|
|
2243
|
-
Setup keys are unique and ordered like section keys.
|
|
2244
|
-
- Tool names, section keys, and task names are unique among published
|
|
2245
|
-
registrations. Tool wrapper keys are unique per tool name, section wrapper
|
|
2246
|
-
keys per section key, and hook keys per task name. Duplicates reject.
|
|
2247
|
-
Section keys match `^[a-z][a-z0-9_-]*$`.
|
|
2248
|
-
- Keyed slots are ordered by the position at which their key was first
|
|
2249
|
-
registered. The registry remembers that position forever, so re-registering an
|
|
2250
|
-
existing key, as a reload does, keeps its position; a new key appends. Hooks
|
|
2251
|
-
without a key append.
|
|
2252
|
-
- Registered objects are immutable and wrappers are pure: a wrapper returns a
|
|
2253
|
-
new object and never mutates its input.
|
|
2254
|
-
- Every call outside `batch()` publishes immediately. Publication is synchronous,
|
|
2255
|
-
invokes no extension callback, and notifies subscribers.
|
|
2256
|
-
|
|
2257
|
-
`batch(register)` stages every registration and disposal made while `register`
|
|
2258
|
-
runs and publishes once when it returns. It validates only the final staged
|
|
2259
|
-
state, so order inside the block does not matter: disposing an old tool and
|
|
2260
|
-
adding its replacement is a gapless reload. If `register` throws, returns a
|
|
2261
|
-
thenable, or the final state is invalid, nothing is published, staged disposals
|
|
2262
|
-
are rolled back, and `batch()` throws. Only registrations made synchronously
|
|
2263
|
-
inside `register` belong to the batch; a registration made later by a detached
|
|
2264
|
-
continuation is an ordinary immediate registration. Calling `batch()` inside a
|
|
2265
|
-
batch is a programming error and throws. The returned `Registration` covers every
|
|
2266
|
-
registration added in the batch that was not disposed inside it.
|
|
2267
|
-
|
|
2268
|
-
A snapshot is a plain immutable value; nothing is released. The scheduler takes
|
|
2269
|
-
one per phase-handler invocation and passes it through the runtime; handlers and
|
|
2270
|
-
hooks read that snapshot and never take their own. At every normal phase boundary
|
|
2271
|
-
the scheduler takes a fresh one (section 5.4). A tool task keeps the composed tool
|
|
2272
|
-
it pinned until execution settles. Different phases may observe different
|
|
2273
|
-
registry states; nothing requires one run to see a single registry state across
|
|
2274
|
-
its generation and tool tasks. Host operations, such as the create
|
|
2275
|
-
conveniences, take one snapshot inside their commit.
|
|
2276
|
-
|
|
2277
|
-
```ts
|
|
2278
|
-
// ToolTask is the built-in tool task's token; its hooks are listed in section 7.2.
|
|
2279
|
-
const registration = registry.batch(() => {
|
|
2280
|
-
registry.tools.add(grep);
|
|
2281
|
-
registry.hooks.add(ToolTask, { beforeTool: audit }, { key: "audit" });
|
|
2282
|
-
});
|
|
2283
|
-
// later, reload: publish the replacement at once
|
|
2284
|
-
registry.batch(() => {
|
|
2285
|
-
registration.dispose();
|
|
2286
|
-
registry.tools.add(grepV2);
|
|
2287
|
-
registry.hooks.add(ToolTask, { beforeTool: audit2 }, { key: "audit" });
|
|
2288
|
-
});
|
|
2289
|
-
```
|
|
2290
|
-
|
|
2291
|
-
The registry does not track which running work still uses a disposed
|
|
2292
|
-
registration. Work that already started keeps using the code it took, so an
|
|
2293
|
-
extension that frees resources right after `dispose()` can make a still-running
|
|
2294
|
-
call fail with an ordinary error result. Extensions that need graceful disposal
|
|
2295
|
-
manage their resources' lifetime themselves, for example by reference counting.
|
|
2296
|
-
|
|
2297
|
-
### 7.2 Hooks
|
|
2298
|
-
|
|
2299
|
-
A hook is a typed question asked by a task before it commits a decision. Hooks
|
|
2300
|
-
are declared by task definition and registered through `registry.hooks.add()`
|
|
2301
|
-
Session-wide or, with `scope`, for one conversation and optionally its owned
|
|
2302
|
-
subtree. Subtree matching follows conversation ownership, not history parents.
|
|
2303
|
-
Registration is typed by the task token, but dispatch matches the task's name,
|
|
2304
|
-
so hooks survive a reload of their task; keeping hook signatures compatible
|
|
2305
|
-
across task versions is the task author's responsibility. Hooks run in registry
|
|
2306
|
-
order off the line; a crash before the consuming commit may rerun them. Abort
|
|
2307
|
-
errors always propagate.
|
|
2308
|
-
|
|
2309
|
-
| hook | composition | ordinary throw |
|
|
2310
|
-
|---|---|---|
|
|
2311
|
-
| `beforeRequest` | replacement chain | report, continue |
|
|
2312
|
-
| `afterResponse` | all observers | report, continue |
|
|
2313
|
-
| `onYield` | first continuation wins | report, continue |
|
|
2314
|
-
| `beforeTool` | argument replacement chain; first block wins | block tool with error text |
|
|
2315
|
-
| `afterTool` | result replacement chain | report, continue |
|
|
2316
|
-
| `afterTools` | all observers | report, continue |
|
|
2317
|
-
| `beforeCompact` | first decision wins | report, continue |
|
|
2318
|
-
|
|
2319
|
-
Hooks use task memos for durable first-writer-wins decisions. There is no public
|
|
2320
|
-
semantic event channel; current UI status is document state.
|
|
2321
|
-
|
|
2322
|
-
```ts
|
|
2323
|
-
type HookResult<T> = T | undefined | Promise<T | undefined>;
|
|
2324
|
-
|
|
2325
|
-
/** What a hook may use: committed reads and the asking task's memos. */
|
|
2326
|
-
interface HookApi extends DocumentReader {
|
|
2327
|
-
readonly taskId: TaskId;
|
|
2328
|
-
readonly conversationId: ConversationId;
|
|
2329
|
-
memo<T extends JsonValue>(name: string, context: Context): Promise<T | undefined>;
|
|
2330
|
-
memo<T extends JsonValue>(name: string, candidate: T, context: Context): Promise<T>;
|
|
2331
|
-
}
|
|
2332
|
-
|
|
2333
|
-
interface GenerationHooks {
|
|
2334
|
-
/** Before every request attempt, including recovery; the result is used for that request only. */
|
|
2335
|
-
beforeRequest(
|
|
2336
|
-
request: { readonly messages: readonly Message[] },
|
|
2337
|
-
api: HookApi,
|
|
2338
|
-
context: Context,
|
|
2339
|
-
): HookResult<{ readonly messages: readonly Message[] }>;
|
|
2340
|
-
/** Every terminal provider message, before classification. */
|
|
2341
|
-
afterResponse(message: AssistantMessage, api: HookApi, context: Context): void | Promise<void>;
|
|
2342
|
-
/** A final answer; `continue` appends a user message and continues the run. */
|
|
2343
|
-
onYield(answer: AssistantMessage, api: HookApi, context: Context): HookResult<{ readonly continue: UserInput }>;
|
|
2344
|
-
/** After every tool of the round is terminal; `results` are the round's result entries in call order. */
|
|
2345
|
-
afterTools(assistant: EntryId, results: readonly EntryId[], api: HookApi, context: Context): void | Promise<void>;
|
|
2346
|
-
}
|
|
2347
|
-
|
|
2348
|
-
interface ToolHooks {
|
|
2349
|
-
/** Before intent; replaces the arguments or blocks the call with error text. */
|
|
2350
|
-
beforeTool(
|
|
2351
|
-
call: ToolCall,
|
|
2352
|
-
api: HookApi,
|
|
2353
|
-
context: Context,
|
|
2354
|
-
): HookResult<{ readonly arguments?: JsonObject; readonly block?: string }>;
|
|
2355
|
-
/** After execution, before the result entry; replaces the result. */
|
|
2356
|
-
afterTool(
|
|
2357
|
-
call: ToolCall,
|
|
2358
|
-
result: ToolExecutionResult,
|
|
2359
|
-
api: HookApi,
|
|
2360
|
-
context: Context,
|
|
2361
|
-
): HookResult<ToolExecutionResult>;
|
|
2362
|
-
}
|
|
2363
|
-
|
|
2364
|
-
interface CompactionHooks {
|
|
2365
|
-
/**
|
|
2366
|
-
* After range selection, before summarizing; the first decision wins. `entries` are the active entries the summary
|
|
2367
|
-
* replaces, the head marker first, and `messages` their model context, the summarizer's source; `firstKept` is the
|
|
2368
|
-
* first entry kept verbatim.
|
|
2369
|
-
*/
|
|
2370
|
-
beforeCompact(
|
|
2371
|
-
compaction: {
|
|
2372
|
-
readonly reason: CompactionReason;
|
|
2373
|
-
readonly entries: readonly EntryRecord[];
|
|
2374
|
-
readonly messages: readonly Message[];
|
|
2375
|
-
readonly firstKept: EntryId;
|
|
2376
|
-
readonly instructions?: string;
|
|
2377
|
-
},
|
|
2378
|
-
api: HookApi,
|
|
2379
|
-
context: Context,
|
|
2380
|
-
): HookResult<{ readonly decline: true } | { readonly summary: string }>;
|
|
2381
|
-
}
|
|
2382
|
-
```
|
|
2383
|
-
|
|
2384
|
-
`GenerationTask`, `ToolTask`, and `CompactionTask` are the exported built-in task
|
|
2385
|
-
tokens whose `H` parameters are these interfaces. `runtime.hooks.each(name,
|
|
2386
|
-
invoke)` calls `invoke` with every handler registered under `name` whose
|
|
2387
|
-
registration matches the task's conversation, in registry order of the phase
|
|
2388
|
-
snapshot. A registration matches when it has no scope, when its scope names the
|
|
2389
|
-
task's conversation, or, with `subtree`, when the task's conversation is owned
|
|
2390
|
-
transitively through tasks of the scope's conversation. Owner edges never change,
|
|
2391
|
-
so the Harness may cache each conversation's owner chain. An ordinary throw from
|
|
2392
|
-
`invoke` is reported through `onReport` and `each` continues with the next
|
|
2393
|
-
handler; once the invocation is signalled, the error propagates. A task that
|
|
2394
|
-
composes differently, such as `beforeTool` turning a throw into a block, or stops
|
|
2395
|
-
after a first decision, does so inside its own `invoke`. Hooks receive the task's
|
|
2396
|
-
runtime as their `HookApi`; hook memos and the task's own memos share one
|
|
2397
|
-
namespace, so hook authors prefix their memo names.
|
|
2398
|
-
|
|
2399
|
-
### 7.3 Tools
|
|
2400
|
-
|
|
2401
|
-
```ts
|
|
2402
|
-
type ToolControl = {
|
|
2403
|
-
readonly addTools?: readonly string[];
|
|
2404
|
-
readonly terminate?: true;
|
|
2405
|
-
readonly handoff?: string;
|
|
2406
|
-
};
|
|
2407
|
-
|
|
2408
|
-
/** Remark about a call for the model and the UI; never part of the tool's data. */
|
|
2409
|
-
type ToolDiagnostic = {
|
|
2410
|
-
readonly severity: "info" | "warn" | "error";
|
|
2411
|
-
readonly message: string;
|
|
2412
|
-
readonly code?: string;
|
|
2413
|
-
};
|
|
2414
|
-
|
|
2415
|
-
type ToolExecutionResult = {
|
|
2416
|
-
readonly content?: ToolResultMessage["content"];
|
|
2417
|
-
readonly isError?: boolean;
|
|
2418
|
-
readonly details?: JsonValue;
|
|
2419
|
-
readonly diagnostics?: readonly ToolDiagnostic[];
|
|
2420
|
-
/** Spend of the execution itself, such as a model call; stored on the result and in `pi.usage.tools`. */
|
|
2421
|
-
readonly usage?: Usage;
|
|
2422
|
-
readonly control?: ToolControl;
|
|
2423
|
-
};
|
|
2424
|
-
|
|
2425
|
-
interface ConversationHandle {
|
|
2426
|
-
readonly id: ConversationId;
|
|
2427
|
-
submit(submission: InputSubmissionDraft, context: Context): Promise<Submission>;
|
|
2428
|
-
abort(context: Context): Promise<void>;
|
|
2429
|
-
waitForIdle(context: Context): Promise<void>;
|
|
2430
|
-
}
|
|
2431
|
-
|
|
2432
|
-
interface ToolExecutionApi extends DocumentObserver, DocumentReader {
|
|
2433
|
-
readonly taskId: TaskId;
|
|
2434
|
-
readonly conversationId: ConversationId;
|
|
2435
|
-
readonly callId: string;
|
|
2436
|
-
/** `HarnessOptions.env` unless a wrapper supplies another environment. */
|
|
2437
|
-
readonly env: ExecutionEnv | undefined;
|
|
2438
|
-
output(chunk: string | Uint8Array): void;
|
|
2439
|
-
diagnostic(diagnostic: ToolDiagnostic): void;
|
|
2440
|
-
details(value: JsonValue, context: Context): Promise<void>;
|
|
2441
|
-
commit<T>(change: (tx: Tx) => T | Promise<T>, context: Context): Promise<T>;
|
|
2442
|
-
memo<T extends JsonValue>(name: string, context: Context): Promise<T | undefined>;
|
|
2443
|
-
memo<T extends JsonValue>(name: string, candidate: T, context: Context): Promise<T>;
|
|
2444
|
-
createTask<I, S extends { phase: string }, R, H extends object>(
|
|
2445
|
-
task: Task<I, S, R, H>,
|
|
2446
|
-
input: I,
|
|
2447
|
-
options: Omit<TaskOptions, "conversationId">,
|
|
2448
|
-
context: Context,
|
|
2449
|
-
): Promise<TaskId<R>>;
|
|
2450
|
-
getTask<R>(id: TaskId<R>, context: Context): Promise<TaskRecord<JsonValue, JsonValue, R> | undefined>;
|
|
2451
|
-
waitForTask<R>(id: TaskId<R>, context: Context): Promise<SettledTask<R>>;
|
|
2452
|
-
conversation(id: ConversationId, context: Context): Promise<ConversationHandle | undefined>;
|
|
2453
|
-
}
|
|
2454
|
-
|
|
2455
|
-
type ToolRegistration = Tool & {
|
|
2456
|
-
readonly replay?: "safe" | "unsafe";
|
|
2457
|
-
readonly executionMode?: ToolExecutionMode;
|
|
2458
|
-
/** Pure repair of commonly malformed arguments; runs before validation and must not mutate `args`. */
|
|
2459
|
-
prepareArguments?(args: JsonValue): JsonValue;
|
|
2460
|
-
readonly outputLimits?: {
|
|
2461
|
-
readonly maxBytes?: number;
|
|
2462
|
-
readonly maxLines?: number;
|
|
2463
|
-
readonly retain?: "head" | "tail";
|
|
2464
|
-
};
|
|
2465
|
-
execute(
|
|
2466
|
-
args: JsonValue,
|
|
2467
|
-
api: ToolExecutionApi,
|
|
2468
|
-
context: Context,
|
|
2469
|
-
): Promise<ToolExecutionResult>;
|
|
2470
|
-
};
|
|
2471
|
-
```
|
|
2472
|
-
|
|
2473
|
-
Omitted `replay` is `unsafe`. Omitted `outputLimits` are 50 KiB, 2,000 lines,
|
|
2474
|
-
and `retain: "head"`. Omitted `executionMode` follows the conversation's
|
|
2475
|
-
`toolExecution`; one `sequential` call makes its whole round sequential
|
|
2476
|
-
(section 8.3).
|
|
2477
|
-
|
|
2478
|
-
Tools reach files and processes only through `api.env`, never through an
|
|
2479
|
-
environment captured at registration, so the environment is chosen per call. The
|
|
2480
|
-
Harness supplies `HarnessOptions.env`. A wrapper may supply another, for example
|
|
2481
|
-
a per-conversation working directory or sandbox:
|
|
2482
|
-
|
|
2483
|
-
```ts
|
|
2484
|
-
registry.tools.wrap("bash", "workspace", (tool) => ({
|
|
2485
|
-
...tool,
|
|
2486
|
-
execute: async (args, api, context) => {
|
|
2487
|
-
const workspace = await api.snapshot(WorkspaceDoc, api.conversationId, context);
|
|
2488
|
-
return tool.execute(args, { ...api, env: environmentFor(workspace) }, context);
|
|
2489
|
-
},
|
|
2490
|
-
}));
|
|
2491
|
-
```
|
|
2492
|
-
|
|
2493
|
-
A tool rerun after recovery must get the same environment. When the state a
|
|
2494
|
-
wrapper reads can change while a call runs, the wrapper memoizes its choice with
|
|
2495
|
-
`api.memo()`. `api` is a plain object so wrappers can spread
|
|
2496
|
-
it. A tool that needs an environment and receives none throws, which produces an
|
|
2497
|
-
ordinary error result.
|
|
2498
|
-
|
|
2499
|
-
A running tool reports output and details to the UI, mirroring the two halves of
|
|
2500
|
-
its final result, plus diagnostics (below):
|
|
2501
|
-
|
|
2502
|
-
- `output(chunk)` appends running text output, like stdout. If `execute()` omits
|
|
2503
|
-
`content`, the final retained output becomes one text content item; no output
|
|
2504
|
-
becomes an empty content list. Retained output is an exact slice of whole lines
|
|
2505
|
-
of the stream (the first lines for `head`, the last for `tail`), trailing
|
|
2506
|
-
newline included; a single line longer than `maxBytes` is cut at the byte limit
|
|
2507
|
-
on a character boundary. Control characters other than tab and newline are
|
|
2508
|
-
removed from the retained text; accepting a chunk does no per-chunk sanitizing.
|
|
2509
|
-
- `details(value)` replaces the running details with a complete JSON value; it
|
|
2510
|
-
does not merge keys. If `execute()` omits `details`, the last value becomes the
|
|
2511
|
-
final `details`, so a renderer handles one details shape from the first
|
|
2512
|
-
update through the final result.
|
|
2513
|
-
|
|
2514
|
-
Neither is sent to the model while the tool runs. `output()` synchronously
|
|
2515
|
-
accepts UTF-8 output into that invocation-owned bounded buffer and throws after
|
|
2516
|
-
invocation end. Throttled commits publish the retained output, dropped
|
|
2517
|
-
byte/line counts, and the current details and diagnostics in its `pi.live.tools`
|
|
2518
|
-
slot. The throttle is adaptive, like the environment's shell output capture: the
|
|
2519
|
-
first change after an idle period commits at once; each commit then delays the
|
|
2520
|
-
next by at least 100 ms and by its written size at 100 KiB/s, so a large
|
|
2521
|
-
rewrite buys a proportionally longer pause. Changes made during the delay
|
|
2522
|
-
coalesce into the next commit. The throttle is Harness policy, not part of
|
|
2523
|
-
`outputLimits`, which only bounds what is retained. Explicit
|
|
2524
|
-
text in explicit result content is bounded by the same limits before transcript
|
|
2525
|
-
persistence; non-text content is retained as declared by its pi-ai type. When
|
|
2526
|
-
bounding drops text, the Harness adds a `warn` diagnostic with code `truncated`
|
|
2527
|
-
stating the dropped lines and bytes.
|
|
2528
|
-
|
|
2529
|
-
Diagnostics are a channel, not text. Remarks about a call, such as truncated
|
|
2530
|
-
output, a spill path, a corrected path, a file changed on disk, or a capped
|
|
2531
|
-
search, go through `api.diagnostic()` or `ToolExecutionResult.diagnostics`,
|
|
2532
|
-
never into the content the model reads as the tool's data. The Harness adds the
|
|
2533
|
-
ones it owns, and a tool adds only what it alone knows. Every diagnostic is
|
|
2534
|
-
model-visible; information only for UIs belongs in `details`. `diagnostic()`
|
|
2535
|
-
synchronously records the diagnostic in the tool's slot with the next throttled
|
|
2536
|
-
commit and throws after invocation end. At settlement the result's diagnostics
|
|
2537
|
-
are those recorded through `api`, then those in the result, then the Harness's.
|
|
2538
|
-
When there are any, the result content ends with one text item:
|
|
2539
|
-
|
|
2540
|
-
```text
|
|
2541
|
-
<harness>
|
|
2542
|
-
[warn] output truncated: 51,204 lines, 2,301,112 bytes dropped
|
|
2543
|
-
</harness>
|
|
2544
|
-
```
|
|
2545
|
-
|
|
2546
|
-
with one `[severity] message` line per diagnostic, and the `pi.tool-result`
|
|
2547
|
-
entry stores the structured list, possibly empty, as `data: { diagnostics }`
|
|
2548
|
-
(section 8.1). The
|
|
2549
|
-
stored message is exactly what the model saw, while UIs and code read the list.
|
|
2550
|
-
A `warn` diagnostic does not set `isError`.
|
|
2551
|
-
|
|
2552
|
-
`output()` never spills complete output to a file because spilling requires a
|
|
2553
|
-
filesystem, which may be remote or unavailable. A tool that must preserve
|
|
2554
|
-
complete output spills through the `ExecutionEnv` or `FileSystem` it was given,
|
|
2555
|
-
and reports the resulting path in a diagnostic, and in its details when a
|
|
2556
|
-
renderer needs it. The environment's shell streams raw output chunks and spills
|
|
2557
|
-
the complete output to a file once it crosses byte or line thresholds; it keeps
|
|
2558
|
-
no bounded view of its own, so `output()` is the one place output is bounded,
|
|
2559
|
-
sanitized, and throttled. The `bash` tool pipes those chunks into `output()`,
|
|
2560
|
-
reports the spill path as a diagnostic, and throws on a nonzero exit or timeout;
|
|
2561
|
-
the error result still carries the retained output and diagnostics.
|
|
2562
|
-
|
|
2563
|
-
The `details()` promise resolves after the corresponding or coalesced document
|
|
2564
|
-
commit. During normal settlement the tool task stops its throttle and awaits the
|
|
2565
|
-
commit in flight; the terminal commit, which appends the result entry, is the
|
|
2566
|
-
final flush and settles any `details()` promise still pending. Abort and close obey
|
|
2567
|
-
invocation and Session admission gates: uncommitted buffered updates may be
|
|
2568
|
-
discarded, while admitted commits settle. Cancellation, callback, tracker
|
|
2569
|
-
preparation, and checkpoint failures occur before Storage admission and do not
|
|
2570
|
-
poison the Session. An uncertain Storage failure follows the fatal Session rule.
|
|
2571
|
-
|
|
2572
|
-
Tools are registry entries (section 7.1) with name, description, JSON schema,
|
|
2573
|
-
replay policy, and execute function. `registry.tools.wrap(name, key, wrapper)`
|
|
2574
|
-
decorates a tool without replacing it. Each snapshot composes the current base
|
|
2575
|
-
tool with its wrappers in registry order; a wrapper never captures a base, so
|
|
2576
|
-
reloading the base keeps its wrappers. A wrapper that throws or returns a tool
|
|
2577
|
-
with a different name makes that tool absent from the snapshot (fail closed:
|
|
2578
|
-
not offered, and calls produce `tool_unavailable`) and is reported. A wrapper
|
|
2579
|
-
without a base contributes nothing. The composite supplies the declaration,
|
|
2580
|
-
argument validation, replay policy, and execution.
|
|
2581
|
-
|
|
2582
|
-
A tool call is accepted only if its request offered the tool. Generation checks
|
|
2583
|
-
this against the tool set replayed from the committed model context through the
|
|
2584
|
-
request's `cutoff`; `beforeRequest` replacements do not change it. It answers a
|
|
2585
|
-
call to a tool it did not offer with the `tool_unavailable` result described in
|
|
2586
|
-
section 2.2, without a tool task. A configuration change after preparation does
|
|
2587
|
-
not affect this check (section 2.2). The tool task resolves the composed tool
|
|
2588
|
-
once from its phase snapshot and uses that implementation until execution
|
|
2589
|
-
settles: one phase handler resolves, validates, runs `beforeTool`, records intent,
|
|
2590
|
-
executes, and commits the result, so no phase boundary separates resolution from
|
|
2591
|
-
settlement. An unregistered implementation produces `tool_unavailable`.
|
|
2592
|
-
The implementation's `prepareArguments`, if any, first repairs the call's
|
|
2593
|
-
arguments, for example `edits` sent as a JSON string; the stored call keeps what
|
|
2594
|
-
the model sent. Arguments are validated against the resolved implementation's
|
|
2595
|
-
schema before and after `beforeTool`; a failure, or a throwing repair, produces an
|
|
2596
|
-
`invalid_arguments` error result.
|
|
2597
|
-
|
|
2598
|
-
After hooks and validation, the tool task durably records the final arguments and
|
|
2599
|
-
resolved replay policy before execution. Recovery does not rerun `beforeTool`
|
|
2600
|
-
and passes the same stored arguments to `execute()`. A replay-safe tool may
|
|
2601
|
-
reconstruct a submission from those arguments when that transformation is pure.
|
|
2602
|
-
Random values, timestamps, mutable document/configuration reads, or other derived
|
|
2603
|
-
inputs that must remain stable are first captured in a durable memo, checkpoint,
|
|
2604
|
-
or task input. A background reporter task receives the final message in its own
|
|
2605
|
-
durable input so it can finish independently. Recovery does not let a
|
|
2606
|
-
changed registry declaration alter the stored replay policy.
|
|
2607
|
-
|
|
2608
|
-
A tool-acquired conversation handle accepts only input submissions; tools use
|
|
2609
|
-
ordinary transaction writes for passive entries.
|
|
2610
|
-
|
|
2611
|
-
A tool executes in a durable task. It may:
|
|
2612
|
-
|
|
2613
|
-
- publish bounded running output and details to its `pi.live.tools` slot;
|
|
2614
|
-
- commit memos;
|
|
2615
|
-
- create and wait for tasks;
|
|
2616
|
-
- atomically create or fork explicitly owned conversations through `commit()`;
|
|
2617
|
-
- observe documents for which it has a token/reference;
|
|
2618
|
-
- mutate authorized documents through `commit()`;
|
|
2619
|
-
- return bounded model content and separate diagnostic details.
|
|
2620
|
-
|
|
2621
|
-
Tool operations use the invoking task's admission and invocation-lifetime gates.
|
|
2622
|
-
Task creation defaults to that task's conversation. Trusted document and
|
|
2623
|
-
conversation access follows sections 3.3 and 9.2; there is no additional subtree
|
|
2624
|
-
authorization layer. Possession of a Session-global typed ID is sufficient in
|
|
2625
|
-
trusted code.
|
|
2626
|
-
|
|
2627
|
-
Tools have no conversation-creation convenience method. They create or fork a
|
|
2628
|
-
conversation inside `commit()`, state its ownership explicitly, atomically stage
|
|
2629
|
-
any related document/task writes, and receive only the inert record. After that
|
|
2630
|
-
commit settles, `conversation(id)` acquires an invocation-bound operational
|
|
2631
|
-
handle for submission, explicit abort, and idle waits without changing
|
|
2632
|
-
ownership. A commit callback must not call `conversation()` or use a previously
|
|
2633
|
-
acquired handle: nested Session operations and external effects are forbidden
|
|
2634
|
-
while the mutation line is held. A handle's operations reject after the
|
|
2635
|
-
invocation ends. A `Submission` returned by its `submit()` is invocation-bound in
|
|
2636
|
-
the same way; the admitted submission remains durable after those methods reject.
|
|
2637
|
-
Invocation-owned document watches stop when the invocation ends.
|
|
2638
|
-
|
|
2639
|
-
A foreground subagent conversation is explicitly owned by its tool task. One
|
|
2640
|
-
transaction creates or forks the child; the owner edge is its durable record, so
|
|
2641
|
-
a replay-safe rerun finds the child with `scanConversations({ ownerTaskId })`
|
|
2642
|
-
before creating one. After settlement the tool reacquires the child, submits
|
|
2643
|
-
with a request ID derived from its task ID, which a rerun keeps, and waits for
|
|
2644
|
-
that submission's result. Provider call IDs are not unique across a Session and
|
|
2645
|
-
must not serve as request IDs. Aborting or abnormally
|
|
2646
|
-
terminalizing the tool task cascades through the owned scope. The tool reports
|
|
2647
|
-
the child in its running details, for example `api.details({ conversationId })`,
|
|
2648
|
-
so a UI that sees the call can attach to the child's view or events.
|
|
2649
|
-
|
|
2650
|
-
A persistent background subagent outlives the parent's turns and can be
|
|
2651
|
-
messaged, steered, stopped, and listed later. One transaction, after
|
|
2652
|
-
deduplicating by the subagent's name in an application registry document,
|
|
2653
|
-
creates a background conversation-owned anchor task `A` in the parent, a child
|
|
2654
|
-
conversation `C` owned by `A`, and the registry mapping from the name to `C`.
|
|
2655
|
-
`A` completes at once; it holds `completing` while `C` has ordinary work
|
|
2656
|
-
(section 5.5) and then stays terminal. Its record keeps `background`, so the
|
|
2657
|
-
parent's ordinary abort and idle traversal stop at it, while
|
|
2658
|
-
`Conversation.abort(context, { background: true })` still reaches `C`'s work
|
|
2659
|
-
through it, and `Harness.waitForIdle()` does not wait for it. Later runs in `C`
|
|
2660
|
-
are ordinary work of `C`; a terminal owner never cascades.
|
|
2661
|
-
|
|
2662
|
-
Each message to a subagent is delivered by a background conversation-owned
|
|
2663
|
-
reporter task in the parent, created in the tool's transaction and keyed there
|
|
2664
|
-
by the tool task ID so a rerun of the call does not create another. The
|
|
2665
|
-
reporter submits to `C` with a request ID derived from its own task ID, waits
|
|
2666
|
-
for that submission's settlement, and posts the answer to the parent as an
|
|
2667
|
-
input with `whenBusy: "followUp"`, under another request ID derived from its
|
|
2668
|
-
task ID. A crash before either admission retries it; a crash after returns the
|
|
2669
|
-
existing `Submission`, so the child gets the message once and the parent gets
|
|
2670
|
-
the answer once. Submission admission therefore needs no private transaction
|
|
2671
|
-
shortcut and does not move onto `Tx`. A registry of names uses `fork:
|
|
2672
|
-
"initial"` so forks of the parent do not inherit it. A UI lists subagents from
|
|
2673
|
-
the registry; a child is working while its `pi.live.run` is set. Owner edges
|
|
2674
|
-
alone drive abort and idle traversal.
|
|
2675
|
-
|
|
2676
|
-
A tool result may request `addTools`, `terminate`, or `handoff`. The
|
|
2677
|
-
generation's `tools` phase (section 8.5) appends added tool names to the
|
|
2678
|
-
configured loadout; they take effect at the next preparation. The round
|
|
2679
|
-
terminates only when every result of the round requests `terminate`, as in the
|
|
2680
|
-
pi agent loop; the `tools` phase then uses a final boundary. Any
|
|
2681
|
-
`handoff` in the round, the last one in call order when several ask, ends the
|
|
2682
|
-
run the same way after appending a `pi.reset` entry with `head: "self"` and the
|
|
2683
|
-
handoff text as a user message (section 8.1), exactly what `reset(handoff)`
|
|
2684
|
-
writes.
|
|
2685
|
-
|
|
2686
|
-
A result's `usage` is stored on the tool-result message and added to the
|
|
2687
|
-
conversation's `pi.usage.tools[toolName]` in the result commit (section 8.6). A
|
|
2688
|
-
tool that runs an owned conversation must not report that conversation's spend
|
|
2689
|
-
again: the child's own `pi.usage` already counts it (section 12).
|
|
2690
|
-
|
|
2691
|
-
On reopen, a tool reruns only when both its stored intent policy and the current
|
|
2692
|
-
registered declaration say `safe`. A current `unsafe` declaration may veto a
|
|
2693
|
-
stored-safe replay; a current-safe declaration never upgrades stored unsafe. A
|
|
2694
|
-
tool with no current registration is treated as `unsafe`. Every other orphaned
|
|
2695
|
-
effect produces an interrupted result containing the
|
|
2696
|
-
durable partial output. Completed, failed, and aborted tool terminal outcomes
|
|
2697
|
-
retain their tool-result entry ID for the generation's `tools` phase.
|
|
2698
|
-
|
|
2699
|
-
Error results the Harness writes itself set `isError` and carry an `error`
|
|
2700
|
-
diagnostic with one of the codes `tool_unavailable`, `invalid_arguments`,
|
|
2701
|
-
`blocked`, `interrupted`, `aborted`, or `tool_error` (a throw from `execute()`)
|
|
2702
|
-
and the error text as its message. Their content is the durable partial output,
|
|
2703
|
-
if any, and `details` is the tool's last reported value, if any.
|
|
2704
|
-
|
|
2705
|
-
`@earendil-works/pi-durable/tools` provides `read`, `bash`, `edit`, and `write`
|
|
2706
|
-
factories, ported from the agent harness tools. They use only `api.env` and are
|
|
2707
|
-
registered like any other tool; nothing registers them automatically. `read`
|
|
2708
|
-
does not return images yet.
|
|
2709
|
-
|
|
2710
|
-
### 7.4 System prompt and dynamic tools
|
|
2711
|
-
|
|
2712
|
-
Pico has no durable prompt sections. The registry's system prompt slot produces
|
|
2713
|
-
the desired sections for each request; the transcript's `pi.system` entries are
|
|
2714
|
-
the only durable record of what the model saw. Pico stores prompt and tool
|
|
2715
|
-
changes directly as PR #9548 `SystemMessage` values at their transcript
|
|
2716
|
-
positions, always with empty `content`:
|
|
2717
|
-
|
|
2718
|
-
```ts
|
|
2719
|
-
type SystemEntry = EntryRecord & {
|
|
2720
|
-
readonly kind: "pi.system";
|
|
2721
|
-
readonly model: readonly [SystemMessage];
|
|
2722
|
-
};
|
|
2723
|
-
|
|
2724
|
-
const baseline: SystemMessage = {
|
|
2725
|
-
role: "system",
|
|
2726
|
-
content: "",
|
|
2727
|
-
sections: { preamble: renderedPreamble, cwd: renderedCwd },
|
|
2728
|
-
toolsAdded: allEffectiveTools,
|
|
2729
|
-
timestamp: now,
|
|
2730
|
-
};
|
|
2731
|
-
|
|
2732
|
-
const delta: SystemMessage = {
|
|
2733
|
-
role: "system",
|
|
2734
|
-
content: "",
|
|
2735
|
-
sections: { cwd: nextRenderedCwd, legacy: null },
|
|
2736
|
-
toolsRemoved: [{ name: "read" }],
|
|
2737
|
-
toolsAdded: [nextRead],
|
|
2738
|
-
timestamp: now,
|
|
2739
|
-
};
|
|
2740
|
-
```
|
|
2741
|
-
|
|
2742
|
-
Generation preparation takes these steps against its phase snapshot:
|
|
2743
|
-
|
|
2744
|
-
1. Read the committed configuration and replay the active transcript's system
|
|
2745
|
-
messages into the shown sections and offered tools.
|
|
2746
|
-
2. Compute the desired tools: active names that the snapshot resolves, in
|
|
2747
|
-
configured order, as composed by their wrappers.
|
|
2748
|
-
3. Render every registered section, in registry order, with `PromptInput`: the
|
|
2749
|
-
conversation, those tools, the shown sections, the configured model and
|
|
2750
|
-
thinking level, and a `DocumentReader` for committed documents (the task
|
|
2751
|
-
runtime). The results are the desired sections.
|
|
2752
|
-
4. Compare desired sections and tool declarations with the replayed state and
|
|
2753
|
-
append one positional `pi.system` entry when they differ.
|
|
2754
|
-
|
|
2755
|
-
Preparation does not recheck the transcript before appending: only the Harness
|
|
2756
|
-
writes to a busy conversation, through submissions, run tasks, boundaries, and a
|
|
2757
|
-
blocking compaction, which appends only while its generation waits for it
|
|
2758
|
-
(section 8.7), so the transcript it read is still current (section 12).
|
|
2759
|
-
|
|
2760
|
-
Model and thinking level are request options, not prompt state.
|
|
2761
|
-
|
|
2762
|
-
Registered sections are the only source of prompt text; there is no separate
|
|
2763
|
-
builder. `systemPrompt.section(key, render, { tag })` adds a section, and
|
|
2764
|
-
`systemPrompt.wrap(key, wrapperKey, wrapper)` decorates one, composed per
|
|
2765
|
-
snapshot like tool wrappers. Sections render in registry order: the position at
|
|
2766
|
-
which each key was first registered. A section whose `render` returns
|
|
2767
|
-
`undefined` is omitted, which is how a section varies by conversation. With
|
|
2768
|
-
`tag` omitted or true, text is wrapped as `<key>\n...\n</key>`. A section that
|
|
2769
|
-
throws keeps its shown text, if any, and is reported; the request is still sent.
|
|
2770
|
-
Errors thrown after the generation invocation is cancelled propagate; an abort
|
|
2771
|
-
error of the section's own, such as its fetch timing out, is an ordinary failure. With no registered sections, the desired section set is
|
|
2772
|
-
empty.
|
|
2773
|
-
|
|
2774
|
-
A minimal prompt is one untagged section:
|
|
2775
|
-
|
|
2776
|
-
```ts
|
|
2777
|
-
registry.systemPrompt.section("preamble", () => "You are a helpful assistant.", { tag: false });
|
|
2778
|
-
```
|
|
2779
|
-
|
|
2780
|
-
Sections read per-conversation data through `input.read`. For example, a
|
|
2781
|
-
coding agent keeps its own conversation document with the agent kind, preamble,
|
|
2782
|
-
and working directory; its preamble and cwd sections render from that document,
|
|
2783
|
-
and its AGENTS.md and skills sections return `undefined` for subagent
|
|
2784
|
-
conversations.
|
|
2785
|
-
Renderers must be deterministic for equal inputs: any change in rendered text,
|
|
2786
|
-
such as an embedded timestamp, appends a system delta and invalidates provider
|
|
2787
|
-
prompt caches.
|
|
2788
|
-
|
|
2789
|
-
Replay applies messages in transcript order. Non-empty `content` appends
|
|
2790
|
-
instructions. A section string adds or replaces that name without moving an
|
|
2791
|
-
existing section; `null` removes it, and a later re-addition appends it to the
|
|
2792
|
-
ordered section map. Within one message, tool removals happen before additions,
|
|
2793
|
-
so a same-name replacement gets the new declaration and position.
|
|
2794
|
-
|
|
2795
|
-
Preparation compares both values and order. If values can be patched without
|
|
2796
|
-
changing order, it emits the minimal patch. If shown and desired section order
|
|
2797
|
-
differ, one commit appends two `pi.system` entries: the first removes every
|
|
2798
|
-
shown section with `null`, and the second re-adds every desired section in
|
|
2799
|
-
desired order. This makes order-only changes and deletion/re-addition between
|
|
2800
|
-
requests replay exactly; merely restating equal values is insufficient.
|
|
2801
|
-
|
|
2802
|
-
A PR #9548 `SystemMessage` is always a patch, not a reset: it cannot remove
|
|
2803
|
-
previous `content` or restore section order merely by restating current values.
|
|
2804
|
-
Therefore, when a head removes the previous request-visible baseline, which is
|
|
2805
|
-
the case when the active context has a head marker and no `pi.system` entry
|
|
2806
|
-
was appended after that marker (has a higher ID), the new
|
|
2807
|
-
`pi.system` entry adds `ContextEdit` omissions for every earlier `pi.system`
|
|
2808
|
-
entry still retained after the cut. Its own message is then a complete baseline
|
|
2809
|
-
containing every desired section in order and every effective tool declaration.
|
|
2810
|
-
Model-context replay sees the new baseline instead of the omitted retained
|
|
2811
|
-
deltas. Preparation writes this baseline even when it restates the replayed
|
|
2812
|
-
values, so every later preparation finds a `pi.system` entry after the marker.
|
|
2813
|
-
Head rebaselining takes precedence over ordinary order/value patching.
|
|
2814
|
-
Without a head cut, an order mismatch uses the two-entry remove/re-add sequence
|
|
2815
|
-
above; only when order already matches does preparation emit the minimal changed
|
|
2816
|
-
values and `null` removals.
|
|
2817
|
-
|
|
2818
|
-
Tool changes are planned separately and ride on the last planned entry, or on
|
|
2819
|
-
one entry of their own when sections are unchanged. Declarations are compared
|
|
2820
|
-
with pi-ai `declarationsEqual()` and written with `toToolDeclaration()`, so
|
|
2821
|
-
application metadata never enters the transcript. A changed declaration is
|
|
2822
|
-
removed and re-added in the same message. Replay keeps retained tools in place
|
|
2823
|
-
and appends additions; when that would not yield the desired order, the message
|
|
2824
|
-
removes every offered tool and re-adds the desired tools in order. A head
|
|
2825
|
-
rebaseline adds every desired tool.
|
|
2826
|
-
|
|
2827
|
-
The rendered strings stored in historical `SystemMessage.sections` remain
|
|
2828
|
-
authoritative even if the current renderer changes. Pi-ai decides whether to
|
|
2829
|
-
send the messages positionally to a capable provider or fold them into one
|
|
2830
|
-
leading system message; Pico does not rewrite its stored transcript for provider
|
|
2831
|
-
compatibility.
|
|
2832
|
-
|
|
2833
|
-
### 7.5 Extension reload
|
|
2834
|
-
|
|
2835
|
-
Extension code is reloaded in process through the registry. The host publishes
|
|
2836
|
-
the new registrations and disposes the old ones in one `batch()`. The Harness
|
|
2837
|
-
keeps running; no close or reopen is required.
|
|
2838
|
-
|
|
2839
|
-
- New phase invocations and tool tasks use the new registrations immediately;
|
|
2840
|
-
running invocations see them at their next phase boundary.
|
|
2841
|
-
- Work already running keeps its snapshot until its phase handler settles, and a
|
|
2842
|
-
pinned tool keeps its implementation until its execution settles. Replacement
|
|
2843
|
-
never signals or interrupts it. Resources the old code uses are the
|
|
2844
|
-
extension's responsibility (section 7.1).
|
|
2845
|
-
- A task definition replaced by name hands over at its next normal phase boundary
|
|
2846
|
-
(section 5.4). A task definition must increase its version when the meaning
|
|
2847
|
-
of persisted input or checkpoint state changes and migrate supported older
|
|
2848
|
-
state; a failed migration blocks the task rather than terminalizing it.
|
|
2849
|
-
- Keyed registrations re-registered under the same key keep their position, so a
|
|
2850
|
-
reload does not reorder hooks, tools, or sections.
|
|
2851
|
-
- Document definitions are passed explicitly to typed access and need no
|
|
2852
|
-
registration; changed tokens take effect on their next access.
|
|
2853
|
-
|
|
2854
|
-
If old extension code ignores cancellation and never settles, new work already
|
|
2855
|
-
uses the replacement. Harness close still joins every invocation. Safe forced termination of arbitrary
|
|
2856
|
-
non-cooperative JavaScript requires worker/process isolation; that host
|
|
2857
|
-
terminates the process and reopens the Session from durable state. Old and new
|
|
2858
|
-
Harness instances must never own the same Session concurrently.
|
|
2859
|
-
|
|
2860
|
-
## 8. Built-in tasks
|
|
2861
|
-
|
|
2862
|
-
The initial implementation provides:
|
|
2863
|
-
|
|
2864
|
-
| kind | responsibility |
|
|
2865
|
-
|---|---|
|
|
2866
|
-
| `pi.generation` | prepare system/loadout, request or poll model, retry, classify response |
|
|
2867
|
-
| `pi.tool` | validate, hook, execute, persist output and details, append result |
|
|
2868
|
-
| `pi.compaction` | select a transcript range, summarize, place a headed summary |
|
|
2869
|
-
|
|
2870
|
-
Generation uses `HarnessOptions.models` without a Pico-specific model adapter. It
|
|
2871
|
-
resolves `models.getModel(ref.provider, ref.modelId)`, builds a pi-ai `Context`
|
|
2872
|
-
from the prepared prompt/messages/tools, and calls `models.streamSimple()` with
|
|
2873
|
-
the task invocation's abort signal and configured reasoning/options. Deferred
|
|
2874
|
-
continuation calls `models.fetchDeferred()` and `models.cancelDeferred()` with
|
|
2875
|
-
that same model and signal. Missing models and synchronous/streamed pi-ai errors
|
|
2876
|
-
are classified into the durable generation outcomes below.
|
|
2877
|
-
|
|
2878
|
-
Generation and tool progress are throttled durable document commits. A crash may
|
|
2879
|
-
lose only the uncommitted throttle window. Recovery converts committed partials
|
|
2880
|
-
to normal interrupted/aborted transcript entries, clears presentation state,
|
|
2881
|
-
and then retries or terminates according to the task phase. Retry deadlines,
|
|
2882
|
-
attempts, compaction, and tool progress are current document state for late
|
|
2883
|
-
joiners; completed-attempt usage/accounting is an entry or terminal detail.
|
|
2884
|
-
Truncation and spill paths are tool diagnostics (section 7.3).
|
|
2885
|
-
|
|
2886
|
-
Compaction changes model context by appending a summary entry with a head. It
|
|
2887
|
-
does not delete transcript history.
|
|
2888
|
-
|
|
2889
|
-
### 8.1 Built-in entries
|
|
2890
|
-
|
|
2891
|
-
Built-in entry kinds carry no `data`, except `pi.tool-result`, whose token is
|
|
2892
|
-
`Entry<{ diagnostics: ToolDiagnostic[] }>`; every tool result carries `data`,
|
|
2893
|
-
with an empty list when it has no diagnostics (section 7.3), and
|
|
2894
|
-
`pi.compaction`, whose token is `Entry<{ reason: CompactionReason }>`. Each kind
|
|
2895
|
-
is exported as an `Entry` token.
|
|
2896
|
-
|
|
2897
|
-
| kind | `model` | written by |
|
|
2898
|
-
|---|---|---|
|
|
2899
|
-
| `pi.user` | `[UserMessage]`, timestamp from the Harness clock at admission or placement | submissions, `onYield` continuations |
|
|
2900
|
-
| `pi.assistant` | `[AssistantMessage]` with any stop reason | generation |
|
|
2901
|
-
| `pi.system` | `[SystemMessage]` with `content: ""` (section 7.4) | generation preparation |
|
|
2902
|
-
| `pi.tool-result` | `[ToolResultMessage]` | tool tasks; generation for calls to tools its request did not offer |
|
|
2903
|
-
| `pi.reset` | absent, or `[UserMessage]` with the handoff text; always `head: "self"` | `reset()`, generation `tools` phase for `handoff` |
|
|
2904
|
-
| `pi.compaction` | `[UserMessage]` with the wrapped summary; `head` is the first kept entry | compaction tasks (section 8.7) |
|
|
2905
|
-
|
|
2906
|
-
Every generation response becomes a `pi.assistant` entry: answers, failed attempts
|
|
2907
|
-
with their error text and usage, and converted partials with stop reason
|
|
2908
|
-
`aborted`. Context derivation (section 2.1, rule 9) keeps failed and aborted
|
|
2909
|
-
messages out of later requests, so no separate usage or notice kind exists.
|
|
2910
|
-
|
|
2911
|
-
### 8.2 Live document
|
|
2912
|
-
|
|
2913
|
-
```ts
|
|
2914
|
-
type LiveState = {
|
|
2915
|
-
/** Run control (section 6); present exactly while the conversation is busy. */
|
|
2916
|
-
run?: { taskId: TaskId; inputs: SubmissionId[] };
|
|
2917
|
-
/** Presentation of the current generation attempt. */
|
|
2918
|
-
generation?: {
|
|
2919
|
-
attempt: number;
|
|
2920
|
-
/** Committed throttled partial of the in-flight response. */
|
|
2921
|
-
message?: JsonRepresentation<AssistantMessage>;
|
|
2922
|
-
/** Durable backoff before the next attempt. */
|
|
2923
|
-
retry?: { at: number; error: string };
|
|
2924
|
-
/** Provider-side deferred response being polled. */
|
|
2925
|
-
deferred?: { pollAt: number };
|
|
2926
|
-
};
|
|
2927
|
-
/** The current tool round in call order, from the tool-calling answer until the generation's `tools` phase ends it. */
|
|
2928
|
-
tools?: {
|
|
2929
|
-
callId: string;
|
|
2930
|
-
name: string;
|
|
2931
|
-
/**
|
|
2932
|
-
* Absent for a call not started yet (sequential round, `pending`) and for a call its request did not offer,
|
|
2933
|
-
* which starts `done` with the `entry` generation wrote.
|
|
2934
|
-
*/
|
|
2935
|
-
taskId?: TaskId;
|
|
2936
|
-
status: "pending" | "running" | "done";
|
|
2937
|
-
/** Retained running output and what the bounds dropped. */
|
|
2938
|
-
output?: string;
|
|
2939
|
-
droppedBytes?: number;
|
|
2940
|
-
droppedLines?: number;
|
|
2941
|
-
/** Last `details()` value. */
|
|
2942
|
-
details?: JsonValue;
|
|
2943
|
-
/** Diagnostics recorded through `api.diagnostic()`. */
|
|
2944
|
-
diagnostics?: ToolDiagnostic[];
|
|
2945
|
-
/** Result entry once done; absent when the tool task faulted or was orphaned. */
|
|
2946
|
-
entry?: EntryId;
|
|
2947
|
-
}[];
|
|
2948
|
-
/** Live compaction tasks in task ID order; absent when none (section 8.7). */
|
|
2949
|
-
compactions?: CompactionStatus[];
|
|
2950
|
-
};
|
|
2951
|
-
|
|
2952
|
-
type CompactionStatus = {
|
|
2953
|
-
taskId: TaskId<CompactionResult>;
|
|
2954
|
-
reason: CompactionReason;
|
|
2955
|
-
/** Whether a generation waits for it: a compaction the generation owns. */
|
|
2956
|
-
blocking: boolean;
|
|
2957
|
-
attempt: number;
|
|
2958
|
-
/** Durable backoff before the next summarization attempt. */
|
|
2959
|
-
retry?: { at: number; error: string };
|
|
2960
|
-
};
|
|
2961
|
-
```
|
|
2962
|
-
|
|
2963
|
-
| field | value |
|
|
2964
|
-
|---|---|
|
|
2965
|
-
| kind | `pi.live` |
|
|
2966
|
-
| version | `1` |
|
|
2967
|
-
| scope/history/fork | conversation, `latest`, `initial` |
|
|
2968
|
-
| `initial()` | `{}` |
|
|
2969
|
-
| checkpoint | complete base whenever nothing runs: `generation` absent and no `running` tool slot |
|
|
2970
|
-
| view mount | `docs["pi.live"]` |
|
|
2971
|
-
| created | with every Harness conversation (section 2.2) |
|
|
2972
|
-
|
|
2973
|
-
Nothing runs while idle, at a final boundary, in the commit where generation
|
|
2974
|
-
hands over to its tool round (every slot is still `pending` or `done`), between
|
|
2975
|
-
the tools of a sequential round, and whenever no tool of a parallel round is
|
|
2976
|
-
`running`. A slot holds output only while `running`, so every base is small, and
|
|
2977
|
-
the stored delta chain spans at most one generation, including its retries and
|
|
2978
|
-
deferred polls, or the overlapping execution of one round's tools. Retained head output grows by Chord string appends. A sliding tail
|
|
2979
|
-
usually becomes a front trim plus an append; when Chord's bounded overlap search
|
|
2980
|
-
finds no shared part, as with highly repetitive output or a window larger than
|
|
2981
|
-
its 64 KiB scan, the commit writes the retained window as one set. Either way
|
|
2982
|
-
each commit writes at most one window, and the throttle bounds the rate.
|
|
2983
|
-
|
|
2984
|
-
Generation creates `tools` with one `pending` slot per call in the commit that
|
|
2985
|
-
appends the tool-calling answer; a call its request did not offer starts `done`
|
|
2986
|
-
with its result entry. A tool task sets its slot `running` in its intent commit,
|
|
2987
|
-
publishes throttled output and details into it, and in its terminal commit sets
|
|
2988
|
-
`done` and `entry` and removes `output`, `droppedBytes`, `droppedLines`,
|
|
2989
|
-
`details`, and `diagnostics`, which the result entry now carries (section 7.3).
|
|
2990
|
-
The generation's `tools` phase removes `tools`. Slot updates apply only while a slot with the task's
|
|
2991
|
-
`taskId` exists; without one, the durable partial output, details, and
|
|
2992
|
-
diagnostics are empty.
|
|
2993
|
-
|
|
2994
|
-
A compaction's status is added in the commit that creates the task and removed
|
|
2995
|
-
in the commit that decides its outcome: the task's own outcome commit, even when
|
|
2996
|
-
that outcome holds as `completing` (section 5.5), or the scheduler's cleanup
|
|
2997
|
-
(section 5.4).
|
|
2998
|
-
It stays small: the summary text lives in the task's placement, never here. Partials are
|
|
2999
|
-
normalized to strict JSON before assignment. Every terminal path of a run task
|
|
3000
|
-
removes `run`, `generation`, and `tools` in the commit that settles the run's
|
|
3001
|
-
inputs. `tx.settleSubmission()` stages each input's new status
|
|
3002
|
-
and resolves the transaction's latest candidate submission record, falling back
|
|
3003
|
-
to committed state, during assembly, like task-document validation (section 3.3),
|
|
3004
|
-
so it is not a caller table read and works after the commit's first table write.
|
|
3005
|
-
|
|
3006
|
-
### 8.3 Generation
|
|
3007
|
-
|
|
3008
|
-
```ts
|
|
3009
|
-
type GenerationInput = {};
|
|
3010
|
-
type GenerationCheckpoint =
|
|
3011
|
-
| {
|
|
3012
|
-
phase: "prepare";
|
|
3013
|
-
attempt: number;
|
|
3014
|
-
/** The blocking compaction this generation waited for; it starts no other compaction. */
|
|
3015
|
-
compacted?: TaskId<CompactionResult>;
|
|
3016
|
-
/** Error text of the overflow that started `compacted`; checked once when `prepare` resumes. */
|
|
3017
|
-
overflow?: string;
|
|
3018
|
-
}
|
|
3019
|
-
| {
|
|
3020
|
-
phase: "request";
|
|
3021
|
-
attempt: number;
|
|
3022
|
-
compacted?: TaskId<CompactionResult>;
|
|
3023
|
-
model: ModelRef;
|
|
3024
|
-
thinkingLevel: ModelThinkingLevel;
|
|
3025
|
-
streamOptions: ConversationStreamOptions;
|
|
3026
|
-
toolExecution: ToolExecutionMode;
|
|
3027
|
-
/** Newest entry included in the request. */
|
|
3028
|
-
cutoff: EntryId;
|
|
3029
|
-
}
|
|
3030
|
-
| { phase: "retry"; attempt: number; compacted?: TaskId<CompactionResult>; until: number }
|
|
3031
|
-
| {
|
|
3032
|
-
phase: "poll";
|
|
3033
|
-
attempt: number;
|
|
3034
|
-
compacted?: TaskId<CompactionResult>;
|
|
3035
|
-
model: ModelRef;
|
|
3036
|
-
toolExecution: ToolExecutionMode;
|
|
3037
|
-
cutoff: EntryId;
|
|
3038
|
-
handle: DeferredHandle;
|
|
3039
|
-
pollAt: number;
|
|
3040
|
-
}
|
|
3041
|
-
| {
|
|
3042
|
-
/** Waiting on the round's tool tasks; `waiting` holds this checkpoint (section 8.5). */
|
|
3043
|
-
phase: "tools";
|
|
3044
|
-
assistant: EntryId;
|
|
3045
|
-
tools: TaskId<ToolTaskResult>[];
|
|
3046
|
-
/** Calls of a sequential round not started yet, in call order. */
|
|
3047
|
-
pending: string[];
|
|
3048
|
-
};
|
|
3049
|
-
type GenerationResult = { entryId: EntryId };
|
|
3050
|
-
```
|
|
3051
|
-
|
|
3052
|
-
`pi.generation` is version 1 and starts at `{ phase: "prepare", attempt: 1 }`.
|
|
3053
|
-
The run's inputs live in `pi.live.run`, not in the task input.
|
|
3054
|
-
|
|
3055
|
-
- `prepare` runs section 7.4 against the committed configuration. When no model
|
|
3056
|
-
is configured or `models.getModel()` does not know it, the task fails with
|
|
3057
|
-
`no_model`. When it resumes with `overflow` and its compaction did not
|
|
3058
|
-
complete with an `entryId`, the run fails with `model_error` and the overflow
|
|
3059
|
-
text as detail. Otherwise one commit appends the planned `pi.system` entries and
|
|
3060
|
-
moves to `request` with the new tail as `cutoff` and the configuration's model,
|
|
3061
|
-
thinking level, stream options, and tool execution mode. These stay fixed for
|
|
3062
|
-
this request attempt; a retry prepares again. `compacted` carries over to
|
|
3063
|
-
`request`, `retry`, `poll`, and the next `prepare`.
|
|
3064
|
-
- Before that commit, `prepare` checks the compaction thresholds (section 8.7)
|
|
3065
|
-
when the conversation's compaction policy is enabled, the model's
|
|
3066
|
-
`contextWindow` is positive, and `compacted` is absent. The estimate starts at
|
|
3067
|
-
the newest assistant message in the committed model context whose entry was
|
|
3068
|
-
appended after the head marker (all qualify without one) and whose usage is
|
|
3069
|
-
nonzero: pi-ai `calculateContextTokens()` of its usage, plus pi-ai
|
|
3070
|
-
`estimateMessageTokens()` of every context message after it and of the planned
|
|
3071
|
-
system messages. Its request included the marker, because a head is placed
|
|
3072
|
-
only while no request is in flight. Without such a message, every message is
|
|
3073
|
-
estimated. Both thresholds apply only when range selection finds a cut.
|
|
3074
|
-
- Above `contextWindow - reserveTokens`, the blocking threshold: instead of
|
|
3075
|
-
appending, one commit creates a compaction owned by the generation with reason
|
|
3076
|
-
`threshold`, adds its status, and commits `waiting` on it with `allSettled`
|
|
3077
|
-
and checkpoint `{ phase: "prepare", attempt, compacted }`. Whatever its
|
|
3078
|
-
outcome, `prepare` then runs again and sends the request.
|
|
3079
|
-
- Above `contextWindow - reserveTokens - backgroundTokens` with
|
|
3080
|
-
`backgroundTokens > 0` and no compaction status in `pi.live`: the commit that
|
|
3081
|
-
moves to `request` also creates a background compaction, owned by the
|
|
3082
|
-
conversation, with reason `threshold`, and adds its status. The generation
|
|
3083
|
-
does not wait for it. The retry policy is read when the
|
|
3084
|
-
attempt's result is classified, because it governs the next attempt (section
|
|
3085
|
-
2.2).
|
|
3086
|
-
- `request` and `poll` resolve the checkpoint's model through
|
|
3087
|
-
`models.getModel()`; an unknown model fails the task with `no_model`, like
|
|
3088
|
-
`prepare`. `request` converts a leftover partial (below) before it resolves
|
|
3089
|
-
the model.
|
|
3090
|
-
- `request` first converts a committed partial left in `pi.live` by an
|
|
3091
|
-
interrupted attempt into an aborted `pi.assistant` entry. It then streams the
|
|
3092
|
-
model context through `cutoff` with the invocation signal, the thinking level
|
|
3093
|
-
as `reasoning` (omitted for `off`), and the pinned `streamOptions`,
|
|
3094
|
-
committing throttled partials. Before streaming, the `beforeRequest` chain may
|
|
3095
|
-
replace the messages for this request only. Recovery resends the same
|
|
3096
|
-
committed messages with the same pinned model, thinking level, and stream
|
|
3097
|
-
options, and reruns `beforeRequest`.
|
|
3098
|
-
- `afterResponse` observes every terminal message, from `request` or `poll`,
|
|
3099
|
-
before classification; a still deferred result is not terminal.
|
|
3100
|
-
- Before classifying, the handler stops the partial throttle and awaits any
|
|
3101
|
-
partial commit in flight, so no stale partial lands after the outcome. The
|
|
3102
|
-
terminal message is classified in one commit that also clears the partial:
|
|
3103
|
-
- `stop`/`length`: before the commit, the `onYield` chain runs; the first
|
|
3104
|
-
`{ continue }` wins. The commit appends the answer and applies the final
|
|
3105
|
-
boundary (section 6). With a continuation and no selected user item or
|
|
3106
|
-
reset, it appends a `pi.user` entry with the continuation content, creates a
|
|
3107
|
-
successor generation, and hands it `pi.live.run`, keeping the inputs open.
|
|
3108
|
-
Otherwise it settles the run's inputs `done`, removes `run` and
|
|
3109
|
-
`generation`, and starts a successor run for the selected user items, if
|
|
3110
|
-
any; a dropped continuation is not retried. Both complete with
|
|
3111
|
-
`{ entryId }`.
|
|
3112
|
-
- `toolUse` with at least one tool call: the commit appends the assistant
|
|
3113
|
-
entry and starts the tool round described below, moving to `waiting` in the
|
|
3114
|
-
`tools` phase. A `toolUse` message without calls is classified like `stop`.
|
|
3115
|
-
- `error` that pi-ai `isContextOverflow()` recognizes, while the compaction
|
|
3116
|
-
policy is enabled, `compacted` is absent, and range selection finds a cut:
|
|
3117
|
-
the commit appends the error entry, removes `generation`, creates a
|
|
3118
|
-
compaction owned by the generation with reason `overflow`, adds its status,
|
|
3119
|
-
and commits `waiting` on it with `allSettled` and checkpoint `{ phase:
|
|
3120
|
-
"prepare", attempt, compacted, overflow }`; the recovery request does not
|
|
3121
|
-
count against the retry policy. Any other overflow,
|
|
3122
|
-
including a second one, is never retried and fails like the non-retryable
|
|
3123
|
-
errors below. A `stop` or `length` response is never classified as overflow;
|
|
3124
|
-
the next threshold check handles silent overflow.
|
|
3125
|
-
- `error` that `isRetryableAssistantError()` accepts while the conversation's
|
|
3126
|
-
retry policy allows another attempt (`enabled` and `attempt <= maxRetries`,
|
|
3127
|
-
so `maxRetries` counts retries after the first attempt): append the error entry and move to
|
|
3128
|
-
`retry` with `until = now + retryDelayMs(policy, attempt)`.
|
|
3129
|
-
- any other `error`, or `aborted` without an abort mark: append the error
|
|
3130
|
-
entry, settle the inputs `unanswered` with `model_error`, remove `run` and
|
|
3131
|
-
`generation`, and fail.
|
|
3132
|
-
- `deferred`: move to `poll` with `pollAt = now + (handle.pollAfterMs ?? 5000)`.
|
|
3133
|
-
- `retry` sleeps until `until`, then returns to `prepare` with the next attempt,
|
|
3134
|
-
so configuration changes made during the backoff apply.
|
|
3135
|
-
- `poll` sleeps until `pollAt` and calls `models.fetchDeferred()`. A still
|
|
3136
|
-
deferred result moves `pollAt` strictly later; any other result is classified
|
|
3137
|
-
as above.
|
|
3138
|
-
- The abort handler calls `models.cancelDeferred()` in `poll` when the model is
|
|
3139
|
-
known (a failure is reported), converts a committed partial, settles the inputs `unanswered` with
|
|
3140
|
-
`aborted`, removes `run` and `generation`, and ends `aborted`.
|
|
3141
|
-
|
|
3142
|
-
A tool round starts in the commit that appends the tool-calling answer:
|
|
3143
|
-
|
|
3144
|
-
1. The offered tools are replayed with pi-ai `getCurrentTools()` from the
|
|
3145
|
-
committed model context through `cutoff`: `request` already holds it, and
|
|
3146
|
-
`poll` derives it again. A call to a tool not offered gets its
|
|
3147
|
-
`tool_unavailable` result entry here, without a task.
|
|
3148
|
-
2. Every other call gets a `pi.tool` task owned by the generation, with input
|
|
3149
|
-
`{ assistant, callId }`. The round is sequential when the pinned
|
|
3150
|
-
`toolExecution` is `sequential` or any called tool's registration in the phase
|
|
3151
|
-
snapshot has `executionMode: "sequential"`: then only the first call gets its
|
|
3152
|
-
task now and the rest wait in `pending`. Otherwise every call gets its task at
|
|
3153
|
-
once and they run in parallel. The checkpoint's `tools` lists the tasks
|
|
3154
|
-
created so far and grows by one per started sequential call, while
|
|
3155
|
-
`pending` shrinks.
|
|
3156
|
-
3. The generation commits `waiting` on its tool tasks with `allSettled` and the
|
|
3157
|
-
`tools` checkpoint (section 8.5); `pi.live.run` stays with it.
|
|
3158
|
-
4. `pi.live.tools` receives the round's slots (section 8.2), and `generation` is
|
|
3159
|
-
removed.
|
|
3160
|
-
|
|
3161
|
-
Input submissions settle `unanswered` with one of these reasons: `no_model`,
|
|
3162
|
-
`model_error` (detail: provider error text), `aborted`, `faulted` (detail: error
|
|
3163
|
-
message), or an orphaning blocked reason (section 5.4). Fault and orphan
|
|
3164
|
-
settlement convert a committed partial into an aborted `pi.assistant` entry,
|
|
3165
|
-
like the abort handler.
|
|
3166
|
-
|
|
3167
|
-
### 8.4 Tool
|
|
3168
|
-
|
|
3169
|
-
```ts
|
|
3170
|
-
type ToolTaskInput = { assistant: EntryId; callId: string };
|
|
3171
|
-
type ToolTaskCheckpoint =
|
|
3172
|
-
| { phase: "call" }
|
|
3173
|
-
| { phase: "execute"; arguments: JsonObject; replay: "safe" | "unsafe" };
|
|
3174
|
-
type ToolTaskResult = { entryId: EntryId; control?: ToolControl };
|
|
3175
|
-
```
|
|
3176
|
-
|
|
3177
|
-
`pi.tool` is version 1 and starts at `{ phase: "call" }`. The input stays small
|
|
3178
|
-
because the terminal record keeps it; the call is read from the assistant entry.
|
|
3179
|
-
|
|
3180
|
-
- `call` reads the call with `runtime.entry()`, resolves the composed tool from
|
|
3181
|
-
its phase snapshot, validates the arguments with pi-ai
|
|
3182
|
-
`validateToolArguments()`, runs the `beforeTool` chain, and validates again
|
|
3183
|
-
(section 7.3). One commit then records intent: it moves to `execute` with the
|
|
3184
|
-
final arguments and the tool's replay policy and sets the slot `running`. The
|
|
3185
|
-
same handler executes the tool, runs the `afterTool` chain, and commits the
|
|
3186
|
-
result. An unregistered tool, invalid arguments, or a block commits the
|
|
3187
|
-
corresponding error result instead, without intent.
|
|
3188
|
-
- `execute` is reached only by recovery. When both the stored and the currently
|
|
3189
|
-
registered policy are `safe`, it executes again with the stored arguments,
|
|
3190
|
-
without `beforeTool`, and settles like `call`. Otherwise it commits an
|
|
3191
|
-
`interrupted` error result from the slot's durable partial output, details, and
|
|
3192
|
-
diagnostics, and ends `failed` with `{ entryId }`.
|
|
3193
|
-
- The result commit bounds the content, appends the diagnostics block (section
|
|
3194
|
-
7.3), appends one `pi.tool-result` entry with `model: [{ role: "toolResult",
|
|
3195
|
-
toolCallId, toolName, content, details, isError, timestamp }]` and
|
|
3196
|
-
`data: { diagnostics }`, marks the slot `done`, and
|
|
3197
|
-
completes with `{ entryId, control }`. An `isError` result still completes.
|
|
3198
|
-
- A throw from `execute()` becomes a `tool_error` result and the task ends
|
|
3199
|
-
`failed` with `{ entryId }`; once the invocation is signalled it propagates
|
|
3200
|
-
instead. Ending `failed`, like `aborted`, records cancellation intent (section
|
|
3201
|
-
5.4), so conversations the call owns, which nothing supervises any more, are
|
|
3202
|
-
aborted. A returned `isError` result still completes, and its call's owned work
|
|
3203
|
-
survives. Either way the run continues with the result entry. A tool task whose
|
|
3204
|
-
owned conversations still have ordinary work when it commits its result holds
|
|
3205
|
-
`completing` with the result entry already written (section 5.5); the
|
|
3206
|
-
generation resumes only when the tool task is terminal.
|
|
3207
|
-
- The abort handler commits an `aborted` error result from the slot's durable
|
|
3208
|
-
partial output, details, and diagnostics and ends `aborted` with `{ entryId }`.
|
|
3209
|
-
|
|
3210
|
-
### 8.5 Tool rounds
|
|
3211
|
-
|
|
3212
|
-
The generation resumes in its `tools` phase once every tool task it waits on is
|
|
3213
|
-
terminal. In a sequential round with calls left in `pending`, one commit creates
|
|
3214
|
-
the next call's owned tool task and waits on it again with the shorter `pending`.
|
|
3215
|
-
Otherwise it reads the tool records with `runtime.getTask()` and the round's
|
|
3216
|
-
result entries from the `pi.live.tools` slots, runs the `afterTools` observers,
|
|
3217
|
-
and then commits once:
|
|
3218
|
-
|
|
3219
|
-
- It appends every `addTools` name not already active to the configured
|
|
3220
|
-
loadout; the next preparation offers it.
|
|
3221
|
-
- When every result of the round requests `terminate`, or any requests
|
|
3222
|
-
`handoff`, it appends the handoff's `pi.reset` entry, if any, settles the
|
|
3223
|
-
run's inputs `done` with the tool-calling answer, removes `run` and `tools`,
|
|
3224
|
-
and applies the final boundary.
|
|
3225
|
-
- Otherwise it removes `tools` and applies the `postTools` boundary. When that
|
|
3226
|
-
boundary placed a reset, the run's inputs settle `unanswered` with `reset` and
|
|
3227
|
-
selected user items start a successor run (section 6). Otherwise selected
|
|
3228
|
-
steer IDs join `pi.live.run`, and it creates the next generation, owned by the
|
|
3229
|
-
conversation, and hands it the run.
|
|
3230
|
-
|
|
3231
|
-
It completes with `{ entryId }` of the tool-calling answer. Its abort handler
|
|
3232
|
-
runs only after its tool tasks are terminal (section 5.5). For every call still
|
|
3233
|
-
in `pending`, which never started, it appends an `aborted` error result; a
|
|
3234
|
-
started call whose task faulted or was orphaned keeps
|
|
3235
|
-
its `done` slot without an entry (section 8.2). It then settles the inputs
|
|
3236
|
-
`unanswered` with `aborted`, removes `run`, `generation`, and `tools`, and ends
|
|
3237
|
-
`aborted`. `abortTask()` on the generation therefore aborts its whole round.
|
|
3238
|
-
|
|
3239
|
-
### 8.6 Usage
|
|
3240
|
-
|
|
3241
|
-
```ts
|
|
3242
|
-
type UsageState = {
|
|
3243
|
-
/** Assistant entries and compaction summarization attempts, keyed `provider/modelId`. */
|
|
3244
|
-
models: Record<string, Usage>;
|
|
3245
|
-
/** Tool results, keyed by tool name; their usage has no model identity. */
|
|
3246
|
-
tools: Record<string, Usage>;
|
|
3247
|
-
};
|
|
3248
|
-
```
|
|
3249
|
-
|
|
3250
|
-
| field | value |
|
|
3251
|
-
|---|---|
|
|
3252
|
-
| kind | `pi.usage` |
|
|
3253
|
-
| version | `1` |
|
|
3254
|
-
| scope/history/fork | conversation, `latest`, `initial` |
|
|
3255
|
-
| `initial()` | `{ models: {}, tools: {} }` |
|
|
3256
|
-
| checkpoint | complete base on every change |
|
|
3257
|
-
| view mount | `docs["pi.usage"]` |
|
|
3258
|
-
| created | with every Harness conversation (section 2.2) |
|
|
3259
|
-
|
|
3260
|
-
`pi.usage` is the ledger of the conversation's own spend. Every built-in writer
|
|
3261
|
-
of a `pi.assistant` entry adds its message's `usage` to `models` under the
|
|
3262
|
-
message's own `provider/model`, and every writer of
|
|
3263
|
-
a `pi.tool-result` entry with `usage` adds it to `tools`, in the same commit.
|
|
3264
|
-
Every summarization attempt of a compaction task adds its response's `usage` to
|
|
3265
|
-
`models` in the commit that classifies it (section 8.7); that spend has no entry,
|
|
3266
|
-
whether the summary is placed, fails, or ends stale. Failed and aborted attempts
|
|
3267
|
-
count. A fork starts at zero, so no spend is counted
|
|
3268
|
-
twice. `Harness.usage()` sums every conversation's document into the Session
|
|
3269
|
-
total; other totals, such as an ownership subtree, are application sums over
|
|
3270
|
-
`scanConversations()`. Nothing stores a total across conversations.
|
|
3271
|
-
|
|
3272
|
-
### 8.7 Compaction
|
|
3273
|
-
|
|
3274
|
-
```ts
|
|
3275
|
-
type CompactionReason = "manual" | "threshold" | "overflow";
|
|
3276
|
-
type CompactionInput = { reason: CompactionReason; instructions?: string };
|
|
3277
|
-
/** The pinned summarization request. */
|
|
3278
|
-
type SummaryRequest = {
|
|
3279
|
-
attempt: number;
|
|
3280
|
-
model: ModelRef;
|
|
3281
|
-
thinkingLevel: ModelThinkingLevel;
|
|
3282
|
-
streamOptions: ConversationStreamOptions;
|
|
3283
|
-
maxTokens: number;
|
|
3284
|
-
/** Newest entry of the context the range was selected from. */
|
|
3285
|
-
tail: EntryId;
|
|
3286
|
-
/** First entry kept verbatim; the summary's `head`. */
|
|
3287
|
-
firstKept: EntryId;
|
|
3288
|
-
};
|
|
3289
|
-
type CompactionCheckpoint =
|
|
3290
|
-
| { phase: "select" }
|
|
3291
|
-
| ({ phase: "summarize" } & SummaryRequest)
|
|
3292
|
-
| ({ phase: "retry"; until: number } & SummaryRequest);
|
|
3293
|
-
/**
|
|
3294
|
-
* `entryId` of a blocking compaction's summary, or the `submissionId` of a conversation-owned compaction's summary
|
|
3295
|
-
* write; both absent when nothing was compacted.
|
|
3296
|
-
*/
|
|
3297
|
-
type CompactionResult = { entryId?: EntryId; submissionId?: SubmissionId };
|
|
3298
|
-
```
|
|
3299
|
-
|
|
3300
|
-
`pi.compaction` is version 1 and starts at `{ phase: "select" }`. Compaction
|
|
3301
|
-
replaces an old prefix of the model context with a summary entry whose `head` is
|
|
3302
|
-
the first kept entry (section 2.1). Raw history stays in storage. There are three
|
|
3303
|
-
ways to start one; the task is the same, only ownership and placement differ:
|
|
3304
|
-
|
|
3305
|
-
| started by | owner | background | generation waits | placement |
|
|
3306
|
-
|---|---|---|---|---|
|
|
3307
|
-
| `Conversation.compact()` | conversation | no | no | write submission |
|
|
3308
|
-
| generation above the background threshold | conversation | yes | no | write submission |
|
|
3309
|
-
| generation above the blocking threshold, or on overflow | the generation | no | yes | direct append |
|
|
3310
|
-
|
|
3311
|
-
Only a blocking compaction runs while the conversation's run waits for it. The
|
|
3312
|
-
others never take run control: the conversation keeps working while they
|
|
3313
|
-
summarize, and their summary is placed like any passive write. Compactions do
|
|
3314
|
-
not coordinate with each other; several may run at once, and section 6 decides
|
|
3315
|
-
between their summaries.
|
|
3316
|
-
|
|
3317
|
-
**Range selection** is a pure function of a `ContextView`, whose
|
|
3318
|
-
`contributions` give each active entry's model messages after every edit in the
|
|
3319
|
-
range (section 2.1, rules 4 and 9), including edits carried by older in-range
|
|
3320
|
-
markers, and `keepRecentTokens`.
|
|
3321
|
-
|
|
3322
|
-
1. Cut candidates are the non-marker entries whose contribution begins with a
|
|
3323
|
-
user or assistant message. Tool results and system entries are never
|
|
3324
|
-
candidates, so a kept assistant message keeps its tool results. A user entry
|
|
3325
|
-
is not a candidate either while a result for a call of the assistant before
|
|
3326
|
-
it follows it, before the next assistant (section 2.1, rule 7).
|
|
3327
|
-
2. Walk the non-marker entries from newest to oldest, adding pi-ai
|
|
3328
|
-
`estimateMessageTokens()` of each contribution. At the first entry where the
|
|
3329
|
-
sum reaches `keepRecentTokens`, the cut is the first candidate at or after
|
|
3330
|
-
that entry in transcript order, or the newest candidate when none follows.
|
|
3331
|
-
3. There is nothing to compact when the sum never reaches `keepRecentTokens`, or
|
|
3332
|
-
when no non-marker entry before the cut contributes a model message.
|
|
3333
|
-
|
|
3334
|
-
The summarized entries are the head marker, if any, followed by the non-marker
|
|
3335
|
-
entries before the cut. Their messages are their contributions, ordered as in
|
|
3336
|
-
section 2.1, rules 7 and 8, so an earlier summary is summarized again together
|
|
3337
|
-
with the history after it. Nothing is compacted while the context is smaller
|
|
3338
|
-
than `keepRecentTokens`, so with a window where `contextWindow - reserveTokens`
|
|
3339
|
-
is below it, overflow may come before any threshold compaction.
|
|
3340
|
-
|
|
3341
|
-
```text
|
|
3342
|
-
1 user 2 assistant (read) 3 tool result, 30k tokens 4 assistant 5 user 6 assistant
|
|
3343
|
-
keepRecentTokens 20k: the walk from 6 reaches 20k at 3; the first candidate at or after 3 is 4.
|
|
3344
|
-
The summary covers 1-3; the model context becomes [summary, 4, 5, 6].
|
|
3345
|
-
```
|
|
3346
|
-
|
|
3347
|
-
Phases:
|
|
3348
|
-
|
|
3349
|
-
- `select` reads the committed configuration and captures the committed
|
|
3350
|
-
context. When no model is configured or `models.getModel()` does not know it,
|
|
3351
|
-
the task fails with `no_model`. With nothing to compact it completes with
|
|
3352
|
-
`{}`. Otherwise the `beforeCompact` hooks (section 7.2) run with the
|
|
3353
|
-
summarized entries and messages; the first decision wins. `{ decline: true }` completes
|
|
3354
|
-
with `{}`, and `{ summary }` is placed as below in the same commit. Without a
|
|
3355
|
-
decision, one commit moves to `summarize` with attempt 1, the configured model,
|
|
3356
|
-
thinking level, and stream options, `maxTokens` = `min(floor(0.8 *
|
|
3357
|
-
reserveTokens), model.maxTokens)` (the model's value only when positive), the
|
|
3358
|
-
captured tail, and the cut as `firstKept`.
|
|
3359
|
-
- `summarize` derives the summarized messages again from the context at `tail`,
|
|
3360
|
-
which is immutable, and serializes them to text: `[User]: ...`,
|
|
3361
|
-
`[Assistant thinking]: ...`, `[Assistant]: ...`,
|
|
3362
|
-
`[Assistant tool calls]: name(key=json, ...)`, and `[Tool result]: ...`
|
|
3363
|
-
truncated to 2000 characters; system messages are omitted. The request has no
|
|
3364
|
-
tools and two messages: a system message with the built-in summarization
|
|
3365
|
-
system prompt, and a user message with the serialized text in
|
|
3366
|
-
`<conversation>` tags followed by the built-in summarization prompt, and
|
|
3367
|
-
`Additional focus: <instructions>` when the input has instructions. It uses the
|
|
3368
|
-
pinned thinking level as `reasoning`, and the pinned stream options without
|
|
3369
|
-
`deferred`, with `cacheRetention: "none"` and the pinned `maxTokens`.
|
|
3370
|
-
Recovery resends the same request. The response is classified in one commit
|
|
3371
|
-
that adds its usage to `pi.usage` (section 8.6):
|
|
3372
|
-
- `stop` with non-empty text and no tool call: the text is the summary, placed
|
|
3373
|
-
as below.
|
|
3374
|
-
- `error` that `isRetryableAssistantError()` accepts while the conversation's
|
|
3375
|
-
retry policy allows another attempt: move to `retry` with `until = now +
|
|
3376
|
-
retryDelayMs(policy, attempt)`, and set the status's `attempt` and `retry`.
|
|
3377
|
-
- anything else, including a `length` stop, which leaves the summary
|
|
3378
|
-
incomplete: fail with `model_error`.
|
|
3379
|
-
- `retry` sleeps until `until`, then returns to `summarize` with the next attempt
|
|
3380
|
-
and the same pinned request.
|
|
3381
|
-
|
|
3382
|
-
The built-in prompts are the coding agent's structured checkpoint format. The
|
|
3383
|
-
prompt also asks the model to carry forward an earlier summary at the start of
|
|
3384
|
-
the conversation.
|
|
3385
|
-
|
|
3386
|
-
**Placement.** The summary entry is
|
|
3387
|
-
|
|
3388
|
-
```ts
|
|
3389
|
-
{
|
|
3390
|
-
kind: "pi.compaction",
|
|
3391
|
-
head: firstKept,
|
|
3392
|
-
model: [{ role: "user", content: [{ type: "text", text: wrapped }], timestamp: now }],
|
|
3393
|
-
data: { reason },
|
|
3394
|
-
}
|
|
3395
|
-
```
|
|
3396
|
-
|
|
3397
|
-
where `wrapped` is `"The conversation history before this point was compacted
|
|
3398
|
-
into the following summary:\n\n<summary>\n"`, the summary, and
|
|
3399
|
-
`"\n</summary>"`. The commit that places it also removes the task's status and
|
|
3400
|
-
commits the task's `completed` outcome:
|
|
3401
|
-
|
|
3402
|
-
- A blocking compaction appends the entry directly and completes with
|
|
3403
|
-
`{ entryId }`. Its generation holds the run and waits, so nothing else writes
|
|
3404
|
-
the conversation.
|
|
3405
|
-
- A conversation-owned compaction admits the entry as a write submission with
|
|
3406
|
-
request ID `compaction:<taskId>`, following section 6 exactly: in an idle
|
|
3407
|
-
conversation with an empty inbox it is appended at once or settles `stale`;
|
|
3408
|
-
in an idle conversation with queued items it queues and a final boundary runs;
|
|
3409
|
-
in a busy one it queues for the next boundary. It completes with
|
|
3410
|
-
`{ submissionId }` and never waits for a queued placement;
|
|
3411
|
-
`Harness.submission()` observes the placement.
|
|
3412
|
-
|
|
3413
|
-
The next generation's preparation finds a head marker without a later
|
|
3414
|
-
`pi.system` entry and writes a complete system baseline (section 7.4).
|
|
3415
|
-
|
|
3416
|
-
**Cancellation.** The abort handler removes the task's status and ends
|
|
3417
|
-
`aborted`; it writes no entry. An attempt cut short by abort or a crash has no
|
|
3418
|
-
committed response, so its usage is not counted. A manual compaction is
|
|
3419
|
-
ordinary work of its conversation: `Conversation.abort()` aborts it. A
|
|
3420
|
-
background compaction survives it and stops only through `abortTask()` or
|
|
3421
|
-
`Conversation.abort(context, { background: true })`. A blocking compaction is
|
|
3422
|
-
aborted with its generation, before the generation's abort handler runs
|
|
3423
|
-
(section 5.5). A summary that is already queued is a write submission:
|
|
3424
|
-
conversation abort keeps it, and `Submission.abort()` withdraws it.
|
|
3425
|
-
|
|
3426
|
-
When a blocking compaction ends without an `entryId`, because it found nothing
|
|
3427
|
-
to compact, was declined, failed, faulted, or was aborted directly, a threshold
|
|
3428
|
-
generation sends its request anyway and an overflow generation fails with
|
|
3429
|
-
`model_error` (section 8.3).
|
|
3430
|
-
|
|
3431
|
-
|
|
3432
|
-
## 9. Document observation and Chord
|
|
3433
|
-
|
|
3434
|
-
### 9.1 Document state
|
|
3435
|
-
|
|
3436
|
-
Chord's existing replicated-state layer exposes this source-adoption contract:
|
|
3437
|
-
|
|
3438
|
-
```ts
|
|
3439
|
-
interface ReplicatedStateSourceFrame<T> {
|
|
3440
|
-
readonly cursor: number;
|
|
3441
|
-
readonly value: T;
|
|
3442
|
-
readonly ops: readonly Op[];
|
|
3443
|
-
readonly context: Context;
|
|
3444
|
-
}
|
|
3445
|
-
|
|
3446
|
-
interface ReplicatedStateSourceAttachment<T> {
|
|
3447
|
-
/** Fixed immutable snapshot captured at the atomic attachment boundary. */
|
|
3448
|
-
readonly snapshot: { readonly value: T; readonly cursor: number };
|
|
3449
|
-
/** Install the sole listener and synchronously drain every buffered frame. */
|
|
3450
|
-
activate(listener: (frame: ReplicatedStateSourceFrame<T>) => void): void;
|
|
3451
|
-
dispose(): void;
|
|
3452
|
-
}
|
|
3453
|
-
|
|
3454
|
-
interface ReplicatedStateSource<T> {
|
|
3455
|
-
/** Atomically capture a snapshot and begin buffering every later commit. */
|
|
3456
|
-
attach(): ReplicatedStateSourceAttachment<T>;
|
|
3457
|
-
}
|
|
3458
|
-
|
|
3459
|
-
interface ReplicatedStateSourceOptions {
|
|
3460
|
-
readonly onError?: (error: Error) => void;
|
|
3461
|
-
}
|
|
3462
|
-
|
|
3463
|
-
interface AttachedReplicatedState<T> extends ReplicatedState<T> {
|
|
3464
|
-
readonly value: T;
|
|
3465
|
-
dispose(): void;
|
|
3466
|
-
}
|
|
3467
|
-
|
|
3468
|
-
function replicatedState<T>(
|
|
3469
|
-
source: ReplicatedStateSource<T>,
|
|
3470
|
-
options?: ReplicatedStateSourceOptions,
|
|
3471
|
-
): AttachedReplicatedState<T>;
|
|
3472
|
-
|
|
3473
|
-
type DocumentState<T extends JsonObject> =
|
|
3474
|
-
AttachedReplicatedState<Readonly<T> | null>;
|
|
3475
|
-
|
|
3476
|
-
type WatchEnd =
|
|
3477
|
-
| { readonly reason: "stopped" | "cancelled" | "session_closed" | "retired" }
|
|
3478
|
-
| { readonly reason: "listener_error"; readonly error: Error };
|
|
3479
|
-
|
|
3480
|
-
interface WatchHandle<T> {
|
|
3481
|
-
/** Acquisition revision before start; latest delivered immutable revision afterward. */
|
|
3482
|
-
readonly value: T;
|
|
3483
|
-
/** Installs the sole serialized asynchronous listener. */
|
|
3484
|
-
start(listener: (value: T, ops: readonly Op[], context: Context) => Promise<void>): void;
|
|
3485
|
-
/** Idempotently stops future callbacks and returns this watch's terminal result. */
|
|
3486
|
-
stop(): Promise<WatchEnd>;
|
|
3487
|
-
/** Settles when the watch terminates; an already-running callback remains caller-owned. */
|
|
3488
|
-
readonly closed: Promise<WatchEnd>;
|
|
3489
|
-
}
|
|
3490
|
-
|
|
3491
|
-
type DocumentWatch<T extends JsonObject> = WatchHandle<Readonly<T> | null>;
|
|
3492
|
-
|
|
3493
|
-
interface DocumentObserver {
|
|
3494
|
-
watchDoc<T extends JsonObject>(token: SessionDocToken<T>, context: Context): Promise<DocumentWatch<T> | undefined>;
|
|
3495
|
-
watchDoc<T extends JsonObject>(token: ConversationDocToken<T>, conversationId: ConversationId, context: Context): Promise<DocumentWatch<T> | undefined>;
|
|
3496
|
-
watchDoc<T extends JsonObject>(token: TaskDocToken<T>, taskId: TaskId, context: Context): Promise<DocumentWatch<T> | undefined>;
|
|
3497
|
-
watchDoc<T extends JsonObject, I extends JsonValue>(token: SessionDocFamilyToken<T, I>, key: string, context: Context): Promise<DocumentWatch<T> | undefined>;
|
|
3498
|
-
watchDoc<T extends JsonObject, I extends JsonValue>(token: ConversationDocFamilyToken<T, I>, conversationId: ConversationId, key: string, context: Context): Promise<DocumentWatch<T> | undefined>;
|
|
3499
|
-
watchDoc<T extends JsonObject, I extends JsonValue>(token: TaskDocFamilyToken<T, I>, taskId: TaskId, key: string, context: Context): Promise<DocumentWatch<T> | undefined>;
|
|
3500
|
-
}
|
|
3501
|
-
```
|
|
3502
|
-
|
|
3503
|
-
`documentState()` attaches Pico's committed document stream to Chord before
|
|
3504
|
-
returning. The returned state is already hydrated with the exact shareable
|
|
3505
|
-
immutable revision, has no mutation methods, and publishes every later exact
|
|
3506
|
-
committed revision and operation batch without another tracker, value copy, or
|
|
3507
|
-
re-diff. Disposing it unregisters only that observation; Pico remains the sole
|
|
3508
|
-
document mutator.
|
|
3509
|
-
|
|
3510
|
-
State and watch acquisition never create; absent lookup returns `undefined`.
|
|
3511
|
-
Successful acquisition binds one concrete incarnation. Retirement publishes a
|
|
3512
|
-
JSON `null` root replacement and ends that incarnation's stream. If the state
|
|
3513
|
-
remains exposed, consumers see `null`, never stale state. A later recreation
|
|
3514
|
-
requires acquiring a new state or watch.
|
|
3515
|
-
|
|
3516
|
-
Each document state assigns its own in-memory contiguous Chord delivery sequence.
|
|
3517
|
-
Pico does not persist or expose that sequence through `WatchHandle`. States and watches retain immutable revisions independently of the loaded tracker
|
|
3518
|
-
cache. Reopen creates a new state lifetime and hydration.
|
|
3519
|
-
|
|
3520
|
-
Internally, state attachment uses Chord's synchronous atomic snapshot-and-register
|
|
3521
|
-
boundary: the snapshot includes every commit before attachment, and every later
|
|
3522
|
-
exact frame is buffered until activation. This prevents a snapshot from being
|
|
3523
|
-
paired with operations based on a newer unseen revision.
|
|
3524
|
-
|
|
3525
|
-
### 9.2 `watchDoc`
|
|
3526
|
-
|
|
3527
|
-
Tasks, hooks, and tools may observe any existing document for which their code
|
|
3528
|
-
has a token and owner/key. There is no additional subtree permission system inside trusted
|
|
3529
|
-
Session code.
|
|
3530
|
-
|
|
3531
|
-
On the Session line, `watchDoc()` resolves one existing concrete incarnation,
|
|
3532
|
-
captures its current immutable tracker revision in O(1), and registers for every
|
|
3533
|
-
later exact committed revision and operation batch. It returns `undefined` when
|
|
3534
|
-
absent. Before `start()`, `watch.value` remains the acquisition revision while
|
|
3535
|
-
later frames buffer.
|
|
3536
|
-
|
|
3537
|
-
```ts
|
|
3538
|
-
const watch = await api.watchDoc(JobOutputDoc, producerTaskId, context);
|
|
3539
|
-
if (watch === undefined) return;
|
|
3540
|
-
try {
|
|
3541
|
-
await initializeConsumer(watch.value, context);
|
|
3542
|
-
watch.start(async (value, ops, deliveryContext) => {
|
|
3543
|
-
await consume(value, ops, deliveryContext);
|
|
3544
|
-
});
|
|
3545
|
-
} catch (error) {
|
|
3546
|
-
await watch.stop();
|
|
3547
|
-
throw error;
|
|
3548
|
-
}
|
|
3549
|
-
```
|
|
3550
|
-
|
|
3551
|
-
`start()` installs the sole listener and schedules delivery; it never invokes
|
|
3552
|
-
user code inline. Each callback receives the exact immutable committed value and
|
|
3553
|
-
the exact operation batch that produced it. Immediately before invocation,
|
|
3554
|
-
`watch.value` advances to that value. The watch awaits the listener before
|
|
3555
|
-
starting its next callback, but Session commits never wait for callback
|
|
3556
|
-
settlement.
|
|
3557
|
-
|
|
3558
|
-
A watch retains at most 100 pending committed frames, excluding the frame already
|
|
3559
|
-
being delivered. Adding frame 101 replaces the complete undelivered suffix with
|
|
3560
|
-
one self-contained root replacement `[["r", newestValue]]`, using the newest
|
|
3561
|
-
exact immutable revision and that commit's Context. Later exact frames follow
|
|
3562
|
-
that replacement normally. Another overflow replaces the pending suffix again.
|
|
3563
|
-
No serialized-byte measurement, value copy, operation replay, or re-diff occurs.
|
|
3564
|
-
This is convergent observation, not an audit stream; consumers requiring every
|
|
3565
|
-
transition must persist those facts separately.
|
|
3566
|
-
|
|
3567
|
-
`watch.value` and every previously returned revision remain stable forever under
|
|
3568
|
-
the trusted immutability contract. Consumers must not mutate them or retained
|
|
3569
|
-
descendants. The selected commit Context's values are preserved without inheriting the
|
|
3570
|
-
producer's cancellation. A callback owns its own asynchronous work.
|
|
3571
|
-
|
|
3572
|
-
A watch remains bound to its original incarnation. Retirement queues the exact
|
|
3573
|
-
terminal `[["r", null]]` frame, or folds it into an overflow replacement with
|
|
3574
|
-
value `null`. After that callback settles, the watch closes as `retired` and never
|
|
3575
|
-
follows a replacement incarnation.
|
|
3576
|
-
|
|
3577
|
-
`stop()` is idempotent. It synchronously unregisters, discards pending frames,
|
|
3578
|
-
prevents another callback from starting, and returns the common terminal promise.
|
|
3579
|
-
An already-running callback is not aborted or joined and remains caller-owned.
|
|
3580
|
-
The acquisition `Context` governs watch lifetime. Cancellation during acquisition
|
|
3581
|
-
cleans up before rejecting; later cancellation and Session close stop future
|
|
3582
|
-
delivery similarly. Listener failure discards pending work and closes only that
|
|
3583
|
-
watch as `listener_error`. The first termination reason wins, and every late
|
|
3584
|
-
rejection is observed.
|
|
3585
|
-
|
|
3586
|
-
### 9.3 Conversation view
|
|
3587
|
-
|
|
3588
|
-
The public view is a fixed structural mount of the conversation's active
|
|
3589
|
-
transcript and its built-in documents:
|
|
3590
|
-
|
|
3591
|
-
```ts
|
|
3592
|
-
type ConversationView = {
|
|
3593
|
-
readonly conversation: ConversationRecord;
|
|
3594
|
-
/** Raw active entries, as `ContextView.entries` (section 2.1): the head marker, then the non-head entries from its head. */
|
|
3595
|
-
readonly entries: readonly EntryRecord[];
|
|
3596
|
-
/** `pi.conversation.config`, `pi.live`, `pi.inbox`, and `pi.usage`, keyed by kind; absent documents are absent. */
|
|
3597
|
-
readonly docs: Readonly<Record<string, JsonObject>>;
|
|
3598
|
-
};
|
|
3599
|
-
```
|
|
3600
|
-
|
|
3601
|
-
These kinds and their fields are public protocol. Third-party documents are
|
|
3602
|
-
exposed through their own Chord services, not mounted.
|
|
3603
|
-
|
|
3604
|
-
The Harness keeps at most one mount per conversation. The first `viewState()`,
|
|
3605
|
-
`watch()`, or event attachment builds its revision on the Session line from the
|
|
3606
|
-
committed active transcript and documents; the mount is dropped when its last
|
|
3607
|
-
observer detaches. A mount derives each next revision from the Session's
|
|
3608
|
-
`subscribeCommits()` publications, like `watchDoc()`: every publication is
|
|
3609
|
-
already durable, so the view shows only committed state. One publication that
|
|
3610
|
-
touches the conversation yields one Chord batch:
|
|
3611
|
-
|
|
3612
|
-
```text
|
|
3613
|
-
document op ["s", ["generation", "message"], value]
|
|
3614
|
-
-> view op ["s", ["docs", "pi.live", "generation", "message"], value]
|
|
3615
|
-
```
|
|
3616
|
-
|
|
3617
|
-
An appended entry without a head is a splice at the end of `entries`. An
|
|
3618
|
-
appended head marker `H` makes the entries `H` followed by the current non-head
|
|
3619
|
-
entries at or after `H.head`: the kept entries stay, and splices remove the
|
|
3620
|
-
others and insert `H` at the front. Every kept entry is already mounted because
|
|
3621
|
-
Harness writers never target a head before the current range: resets and
|
|
3622
|
-
handoffs head themselves, compaction summaries target a kept entry of the
|
|
3623
|
-
current range, and head writes that reach further back are stale (section 6). A raw head write that targets further back shows only the
|
|
3624
|
-
mounted entries (section 12).
|
|
3625
|
-
Document creation and retirement set and delete the `docs` key. Parent entries
|
|
3626
|
-
through `parent.at` are immutable, so a child's mount follows only its own
|
|
3627
|
-
conversation. A publication that touches nothing mounted creates no revision; a
|
|
3628
|
-
redundant nonempty batch remains a real publication. Mounted document revisions
|
|
3629
|
-
are structurally shared under the trusted immutability contract. The mount
|
|
3630
|
-
performs no semantic projection and owns no persistence.
|
|
3631
|
-
|
|
3632
|
-
`Conversation.viewState()` exposes the mount as a disposable read-only Chord
|
|
3633
|
-
state. `Conversation.watch()` exposes the same mount through the serialized
|
|
3634
|
-
exact-frame watch of section 9.2, with bounded pending frames and full-value
|
|
3635
|
-
overflow replacements.
|
|
3636
|
-
|
|
3637
|
-
### 9.4 Agent events (experimental)
|
|
3638
|
-
|
|
3639
|
-
`watchEvents(harness, conversationId, context): Promise<AgentEventStream>`,
|
|
3640
|
-
exported from the package root, is an experimental adapter that
|
|
3641
|
-
translates one conversation's committed publications into agent events shaped
|
|
3642
|
-
like the coding agent's `AgentSessionEvent`s. It owns no tracker or persistence,
|
|
3643
|
-
covers exactly one conversation (not its owned subtree), and emits an event only
|
|
3644
|
-
after the commit that makes it true. Its protocol may change without notice.
|
|
3645
|
-
|
|
3646
|
-
```ts
|
|
3647
|
-
type AgentEvent =
|
|
3648
|
-
| {
|
|
3649
|
-
type: "snapshot";
|
|
3650
|
-
entries: readonly EntryRecord[];
|
|
3651
|
-
run?: { inputs: readonly SubmissionId[] };
|
|
3652
|
-
/** Current generation attempt: its in-flight partial, retry backoff, or deferred poll. */
|
|
3653
|
-
generation?: { attempt: number; message?: AssistantMessage; retry?: { at: number; error: string }; deferred?: { pollAt: number } };
|
|
3654
|
-
tools: readonly ToolSlot[];
|
|
3655
|
-
/** `pi.live.compactions` (section 8.2). */
|
|
3656
|
-
compactions: readonly CompactionStatus[];
|
|
3657
|
-
inbox: readonly { id: SubmissionId; mode: InboxItem["mode"] }[];
|
|
3658
|
-
config: ConversationConfigState;
|
|
3659
|
-
usage: UsageState;
|
|
3660
|
-
}
|
|
3661
|
-
| { type: "run_start"; inputs: readonly SubmissionId[] }
|
|
3662
|
-
| { type: "run_end"; inputs: readonly SubmissionId[] }
|
|
3663
|
-
| { type: "turn_start" }
|
|
3664
|
-
| { type: "turn_end" }
|
|
3665
|
-
| { type: "message_start"; message: Message }
|
|
3666
|
-
| { type: "message_update"; usage: Usage; changes: readonly MessageChange[] }
|
|
3667
|
-
| { type: "message_end"; entry: EntryRecord }
|
|
3668
|
-
| { type: "tool_execution_start"; toolCallId: string; toolName: string; args: JsonObject }
|
|
3669
|
-
| {
|
|
3670
|
-
type: "tool_execution_update";
|
|
3671
|
-
toolCallId: string;
|
|
3672
|
-
toolName: string;
|
|
3673
|
-
/** A front trim and then an append of the retained window, or its replacement. */
|
|
3674
|
-
output?: { trimStart?: number; append?: string } | { set: string };
|
|
3675
|
-
details?: JsonValue;
|
|
3676
|
-
diagnostics?: readonly ToolDiagnostic[];
|
|
3677
|
-
}
|
|
3678
|
-
/** `entry` is absent when the tool task faulted or was orphaned. */
|
|
3679
|
-
| { type: "tool_execution_end"; toolCallId: string; toolName: string; entry?: EntryRecord }
|
|
3680
|
-
| { type: "inbox_update"; items: readonly { id: SubmissionId; mode: InboxItem["mode"] }[] }
|
|
3681
|
-
| { type: "submission"; record: SubmissionRecord }
|
|
3682
|
-
| { type: "auto_retry_start"; attempt: number; at: number; errorMessage: string }
|
|
3683
|
-
| { type: "auto_retry_end"; attempt: number }
|
|
3684
|
-
| { type: "deferred_poll"; pollAt: number }
|
|
3685
|
-
| { type: "entry_appended"; entry: EntryRecord }
|
|
3686
|
-
| { type: "config_changed"; config: ConversationConfigState }
|
|
3687
|
-
| { type: "usage_changed"; usage: UsageState }
|
|
3688
|
-
| { type: "task_failed"; taskId: TaskId; kind: string; message: string }
|
|
3689
|
-
| { type: "compaction_start"; taskId: TaskId; reason: CompactionReason; blocking: boolean }
|
|
3690
|
-
/** The task's receipt tells whether it produced a summary; the summary entry has its own events. */
|
|
3691
|
-
| { type: "compaction_end"; taskId: TaskId; reason: CompactionReason };
|
|
3692
|
-
|
|
3693
|
-
/** One change to the in-flight assistant message, relative to that message. */
|
|
3694
|
-
type MessageChange =
|
|
3695
|
-
| { type: "text_start" | "thinking_start" | "toolcall_start"; contentIndex: number; block: AssistantMessage["content"][number] }
|
|
3696
|
-
| { type: "text_delta" | "thinking_delta"; contentIndex: number; delta: string }
|
|
3697
|
-
| { type: "toolcall_delta"; contentIndex: number; path: readonly (string | number)[]; delta: string }
|
|
3698
|
-
| { type: "block"; contentIndex: number; block: AssistantMessage["content"][number] }
|
|
3699
|
-
| { type: "message"; message: AssistantMessage };
|
|
3700
|
-
|
|
3701
|
-
interface AgentEventStream {
|
|
3702
|
-
/** The `snapshot` event at attachment. */
|
|
3703
|
-
readonly snapshot: Extract<AgentEvent, { type: "snapshot" }>;
|
|
3704
|
-
start(listener: (events: readonly AgentEvent[], context: Context) => Promise<void>): void;
|
|
3705
|
-
stop(): Promise<WatchEnd>;
|
|
3706
|
-
readonly closed: Promise<WatchEnd>;
|
|
3707
|
-
}
|
|
3708
|
-
```
|
|
3709
|
-
|
|
3710
|
-
Every event is relative to the `snapshot` and the events before it, never to
|
|
3711
|
-
Pico's document layout. `snapshot` replaces everything a consumer holds; every
|
|
3712
|
-
other event applies on top of it. Attachment captures the snapshot and
|
|
3713
|
-
registers for later publications atomically on the Session line.
|
|
3714
|
-
|
|
3715
|
-
Events derive from committed changes:
|
|
3716
|
-
|
|
3717
|
-
- `run_start`/`run_end`: `pi.live.run` appears, is removed, or is replaced by a
|
|
3718
|
-
successor run whose first input differs. Steers joining the current run and
|
|
3719
|
-
handovers between its tasks are not run events. The inputs' outcomes are
|
|
3720
|
-
`submission` events.
|
|
3721
|
-
- `turn_start`: a `pi.generation` task is created. `turn_end`: a generation's
|
|
3722
|
-
outcome is committed, when it holds `completing` or becomes terminal,
|
|
3723
|
-
whichever comes first, so a successor created at hold starts after it.
|
|
3724
|
-
- `message_start`: the first committed partial of an attempt, or, for a message
|
|
3725
|
-
entry without a committed partial, the entry itself; `message_end` follows for
|
|
3726
|
-
every entry with `model` messages. Generation commits a partial only once it has
|
|
3727
|
-
content, and every built-in path that clears one appends its entry, so each
|
|
3728
|
-
started message ends with its entry. Entries without messages are
|
|
3729
|
-
`entry_appended`.
|
|
3730
|
-
- `message_update`: the Chord operations on the committed partial, translated:
|
|
3731
|
-
an insertion into `content` starts blocks, an append to a block's `text` or
|
|
3732
|
-
`thinking` is a delta, an append to a string inside a tool call's `arguments`
|
|
3733
|
-
is a `toolcall_delta` with the path relative to `arguments`, any other change
|
|
3734
|
-
inside a block sends that `block`, which already holds the batch's later
|
|
3735
|
-
changes to it, and any other change to `content` sends the whole `message`.
|
|
3736
|
-
Each update carries the partial's current `usage`, as the coding agent's JSON
|
|
3737
|
-
mode does; a change of only `usage` sends no changes. Only deltas travel, so throttled
|
|
3738
|
-
partials cause no write amplification on the wire.
|
|
3739
|
-
- `tool_execution_start`: a slot becomes `running`; `args` come from the tool
|
|
3740
|
-
task's intent checkpoint in the same commit. A slot that becomes `done`
|
|
3741
|
-
without running (a call not offered, invalid arguments, a block, a fault
|
|
3742
|
-
before intent) gets only `tool_execution_end`. `tool_execution_update`: output
|
|
3743
|
-
appends, front trims, and replacements of the slot's retained window, and the
|
|
3744
|
-
slot's current `details` and `diagnostics` when they change; removed ones, as
|
|
3745
|
-
when a safe replay restarts the tool, send `null` and `[]`.
|
|
3746
|
-
`tool_execution_end`: the slot becomes `done`, with its `pi.tool-result`
|
|
3747
|
-
entry, which carries the diagnostics, or without one after a fault or
|
|
3748
|
-
orphan. An unfinished slot that disappears because its run ended ends with the
|
|
3749
|
-
result entry appended in the same commit, as for the unstarted calls of an
|
|
3750
|
-
aborted round, directly before its `message_start`, or without an entry.
|
|
3751
|
-
- `inbox_update`, `config_changed`, `usage_changed`: the document changed; a
|
|
3752
|
-
retired one reads as its initial value, as in a snapshot.
|
|
3753
|
-
- `auto_retry_start`/`auto_retry_end`, `deferred_poll`: `pi.live.generation`
|
|
3754
|
-
gains or drops `retry`, or gains `deferred` or moves its `pollAt`.
|
|
3755
|
-
- `task_failed`: a task of the conversation settles `faulted` or `orphaned`.
|
|
3756
|
-
- `compaction_start`/`compaction_end`: a status appears in or disappears from
|
|
3757
|
-
`pi.live.compactions`. A retry backoff shows only in the snapshot's
|
|
3758
|
-
`compactions`. A queued summary is placed later with its own
|
|
3759
|
-
`message_start`/`message_end` and `submission` events.
|
|
3760
|
-
|
|
3761
|
-
One commit produces one batch, in this order: `tool_execution_start`,
|
|
3762
|
-
`message_start` of a first partial, `message_update`, `tool_execution_update`,
|
|
3763
|
-
and retry/deferred events; then entries in append order with their
|
|
3764
|
-
`message_start`/`message_end` or `entry_appended`, where a tool result's
|
|
3765
|
-
`tool_execution_end` directly precedes its `message_start`, as in the coding
|
|
3766
|
-
agent; then the `tool_execution_end` of tools ending without an entry;
|
|
3767
|
-
then `compaction_end`, `task_failed`, `turn_end`, `run_end`; then `submission`
|
|
3768
|
-
events in ID order; then `inbox_update`, `config_changed`, and `usage_changed`;
|
|
3769
|
-
and last `compaction_start`, `run_start`, and `turn_start`. A stream buffers at most 100 undelivered
|
|
3770
|
-
batches; adding another replaces every undelivered batch with one `snapshot` of
|
|
3771
|
-
the newest committed state. The stream therefore converges but does not promise
|
|
3772
|
-
every transition; `Submission.wait()` reports exact outcomes. A reconnecting
|
|
3773
|
-
consumer attaches again and starts from its `snapshot`; nothing is replayed.
|
|
3774
|
-
Transport backpressure and disconnect policy belong to the consumer.
|
|
3775
|
-
|
|
3776
|
-
Print mode awaits its own input `Submission` and prints its answer. A TUI
|
|
3777
|
-
renders `ConversationView`; events may drive transient animation.
|
|
3778
|
-
|
|
3779
|
-
## 10. Storage contract
|
|
3780
|
-
|
|
3781
|
-
Ordered scans are cursor-based. Exact identity lookups are keyed.
|
|
3782
|
-
|
|
3783
|
-
```ts
|
|
3784
|
-
type Page<T, C> = {
|
|
3785
|
-
readonly items: readonly T[];
|
|
3786
|
-
readonly next?: C;
|
|
3787
|
-
};
|
|
3788
|
-
|
|
3789
|
-
type Cursor = Readonly<Record<string, JsonValue>>;
|
|
3790
|
-
|
|
3791
|
-
type ConversationQuery = {
|
|
3792
|
-
readonly ownerConversationId?: ConversationId;
|
|
3793
|
-
readonly ownerTaskId?: TaskId;
|
|
3794
|
-
};
|
|
3795
|
-
|
|
3796
|
-
type EntryQuery = {
|
|
3797
|
-
readonly conversationId: ConversationId;
|
|
3798
|
-
readonly minEntryId?: EntryId; // inclusive
|
|
3799
|
-
readonly maxEntryId?: EntryId; // inclusive
|
|
3800
|
-
};
|
|
3801
|
-
|
|
3802
|
-
type TaskQuery = {
|
|
3803
|
-
readonly conversationId?: ConversationId;
|
|
3804
|
-
readonly kind?: string;
|
|
3805
|
-
readonly status?: "pending" | "running" | "waiting" | "completing" | "terminal";
|
|
3806
|
-
readonly abortRequested?: boolean;
|
|
3807
|
-
readonly background?: boolean;
|
|
3808
|
-
};
|
|
3809
|
-
|
|
3810
|
-
type SubmissionQuery = {
|
|
3811
|
-
readonly conversationId?: ConversationId;
|
|
3812
|
-
readonly status?: SubmissionRecord["status"];
|
|
3813
|
-
};
|
|
3814
|
-
|
|
3815
|
-
type DocumentPoint = Seq | "current";
|
|
3816
|
-
|
|
3817
|
-
type DocumentAddress = {
|
|
3818
|
-
readonly kind: string;
|
|
3819
|
-
readonly scope: DocumentRecord["scope"];
|
|
3820
|
-
readonly key?: string;
|
|
3821
|
-
};
|
|
3822
|
-
|
|
3823
|
-
type DocumentQuery = {
|
|
3824
|
-
readonly scope: DocumentRecord["scope"];
|
|
3825
|
-
readonly at: DocumentPoint;
|
|
3826
|
-
readonly kind?: string;
|
|
3827
|
-
};
|
|
3828
|
-
|
|
3829
|
-
type DocumentContent =
|
|
3830
|
-
| { readonly version: number; readonly kind: "base"; readonly value: JsonObject }
|
|
3831
|
-
| { readonly version: number; readonly kind: "delta"; readonly ops: readonly Op[] };
|
|
3832
|
-
|
|
3833
|
-
type StoredDocument = {
|
|
3834
|
-
readonly record: DocumentRecord;
|
|
3835
|
-
readonly version: number;
|
|
3836
|
-
readonly value: JsonObject;
|
|
3837
|
-
/** Deltas replayed after the selected base to materialize `value`. */
|
|
3838
|
-
readonly deltasSinceBase: number;
|
|
3839
|
-
};
|
|
3840
|
-
|
|
3841
|
-
type StorageWrite =
|
|
3842
|
-
| { readonly type: "conversation"; readonly value: ConversationRecord }
|
|
3843
|
-
| { readonly type: "entry"; readonly value: EntryRecord }
|
|
3844
|
-
| { readonly type: "task"; readonly value: TaskRecord<JsonValue, JsonValue, JsonValue> }
|
|
3845
|
-
| { readonly type: "submission"; readonly value: SubmissionRecord }
|
|
3846
|
-
| {
|
|
3847
|
-
readonly type: "document.create";
|
|
3848
|
-
readonly record: DocumentCreate;
|
|
3849
|
-
readonly content: Extract<DocumentContent, { kind: "base" }>;
|
|
3850
|
-
}
|
|
3851
|
-
| {
|
|
3852
|
-
readonly type: "document.copy";
|
|
3853
|
-
readonly record: DocumentCreate;
|
|
3854
|
-
readonly source: { readonly id: DocumentId; readonly at: DocumentPoint };
|
|
3855
|
-
}
|
|
3856
|
-
| {
|
|
3857
|
-
readonly type: "document.change";
|
|
3858
|
-
readonly id: DocumentId;
|
|
3859
|
-
readonly content: DocumentContent;
|
|
3860
|
-
}
|
|
3861
|
-
| { readonly type: "document.retire"; readonly id: DocumentId };
|
|
3862
|
-
|
|
3863
|
-
/**
|
|
3864
|
-
* Trusts the owning Session to supply semantically valid records, references,
|
|
3865
|
-
* ancestry, and transitions. Enforces atomicity, global ID ownership, immutable
|
|
3866
|
-
* conversation/entry creation, document record consistency, and detached
|
|
3867
|
-
* values; Session serializes commits. Sequences strictly increase but may have
|
|
3868
|
-
* gaps. Once commit() resolves, later reads through that Storage observe it.
|
|
3869
|
-
*/
|
|
3870
|
-
interface Storage {
|
|
3871
|
-
commit(writes: readonly StorageWrite[], context: Context): Promise<Seq>;
|
|
3872
|
-
/** Allocate from the one global numeric namespace; the generic brand is compile-time only. */
|
|
3873
|
-
mintId<I extends Id<string>>(): Promise<I>;
|
|
3874
|
-
|
|
3875
|
-
conversation(id: ConversationId, context: Context): Promise<ConversationRecord | undefined>;
|
|
3876
|
-
scanConversations(query: ConversationQuery, limit: number, cursor: Cursor | undefined, context: Context): Promise<Page<ConversationRecord, Cursor>>;
|
|
3877
|
-
|
|
3878
|
-
entry(id: EntryId, context: Context): Promise<{ readonly entry: EntryRecord; readonly commitSeq: Seq } | undefined>;
|
|
3879
|
-
entry(conversationId: ConversationId, id: EntryId, context: Context): Promise<{ readonly entry: EntryRecord; readonly commitSeq: Seq } | undefined>;
|
|
3880
|
-
findLatestHeadMarker(conversationId: ConversationId, atOrBeforeEntryId: EntryId | undefined, context: Context): Promise<(EntryRecord & { readonly head: EntryId }) | undefined>;
|
|
3881
|
-
scanEntries(query: EntryQuery, limit: number, cursor: Cursor | undefined, context: Context): Promise<Page<EntryRecord, Cursor>>;
|
|
3882
|
-
|
|
3883
|
-
task(id: TaskId, context: Context): Promise<TaskRecord<JsonValue, JsonValue, JsonValue> | undefined>;
|
|
3884
|
-
scanTasks(query: TaskQuery, limit: number, cursor: Cursor | undefined, context: Context): Promise<Page<TaskRecord<JsonValue, JsonValue, JsonValue>, Cursor>>;
|
|
3885
|
-
|
|
3886
|
-
submission(id: SubmissionId, context: Context): Promise<SubmissionRecord | undefined>;
|
|
3887
|
-
scanSubmissions(query: SubmissionQuery, limit: number, cursor: Cursor | undefined, context: Context): Promise<Page<SubmissionRecord, Cursor>>;
|
|
3888
|
-
submissionByRequest(conversationId: ConversationId, requestId: string, context: Context): Promise<SubmissionRecord | undefined>;
|
|
3889
|
-
|
|
3890
|
-
findDocument(address: DocumentAddress, at: DocumentPoint, context: Context): Promise<DocumentRecord | undefined>;
|
|
3891
|
-
document(id: DocumentId, at: DocumentPoint, context: Context): Promise<StoredDocument | undefined>;
|
|
3892
|
-
scanDocuments(query: DocumentQuery, limit: number, cursor: Cursor | undefined, context: Context): Promise<Page<DocumentRecord, Cursor>>;
|
|
3893
|
-
|
|
3894
|
-
close(context: Context): Promise<void>;
|
|
3895
|
-
}
|
|
3896
|
-
```
|
|
3897
|
-
|
|
3898
|
-
`StorageRejected` means a batch was rejected before any durable effect and is
|
|
3899
|
-
guaranteed not to have committed. Session rolls such a batch back normally;
|
|
3900
|
-
unknown failures after Storage admission remain fatal because their commit state
|
|
3901
|
-
is uncertain. Backends use `StorageRejected` for deterministic `document.copy`
|
|
3902
|
-
source, replay, and consistency failures only when rollback is guaranteed.
|
|
3903
|
-
|
|
3904
|
-
A `document.copy` reads committed pre-batch source state independent of command
|
|
3905
|
-
order. The source must be an alive conversation document at the selected point,
|
|
3906
|
-
and kind/key/history/fork must match the child create record. Storage persists
|
|
3907
|
-
one independent complete child base at the source's stored version. A batch may
|
|
3908
|
-
not create, change, or retire a selected source. Later source changes,
|
|
3909
|
-
reclamation, retirement, or backend reopen cannot affect the child.
|
|
3910
|
-
|
|
3911
|
-
Cursors are backend-owned JSON objects. Callers only round-trip them to the same
|
|
3912
|
-
scan on the same storage; cross-storage or cross-query use is unsupported. The
|
|
3913
|
-
Session owns the mutation line, so storage implementations do not add a second
|
|
3914
|
-
caller-facing commit mutex. Each backend still makes one admitted batch atomic.
|
|
3915
|
-
|
|
3916
|
-
`findLatestHeadMarker()` returns the newest visible entry carrying `head` at or
|
|
3917
|
-
below its optional inclusive cutoff. The returned entry is the marker; its
|
|
3918
|
-
`head` value is the actual lower bound for context. `scanEntries()` pages the
|
|
3919
|
-
inclusive ID range in newest-first order while applying every conversation
|
|
3920
|
-
ancestry cap. With no bounds it pages complete visible history. To read context
|
|
3921
|
-
through entry `E`, find the marker at or before `E`, then scan from
|
|
3922
|
-
`marker?.head` through `E`. For current context the upper bound is omitted.
|
|
3923
|
-
Conversation owner filters are indexed and conjunctive. They support ownership
|
|
3924
|
-
traversal without an all-conversation scan; application-maintained registries
|
|
3925
|
-
are not a substitute for these kernel indexes.
|
|
3926
|
-
|
|
3927
|
-
`entry(id)` combines exact global lookup with the commit sequence required by
|
|
3928
|
-
historical document reads. `entry(conversationId, id)` returns that pair only
|
|
3929
|
-
when the entry is visible through the requested conversation's ancestry. `limit`
|
|
3930
|
-
is always the maximum page size.
|
|
3931
|
-
`findDocument()` resolves one exact logical kind/scope/key address at current or
|
|
3932
|
-
historical membership. A missing key means the singleton, not every family
|
|
3933
|
-
member. `scanDocuments()` enumerates only the incarnations alive in one exact
|
|
3934
|
-
scope at its selected point and may restrict one family/singleton kind. It uses
|
|
3935
|
-
ascending incarnation IDs. There is no ordinary open-time all-document scan.
|
|
3936
|
-
Task queries support conversation, kind, live/terminal status, abort mark, and
|
|
3937
|
-
background status.
|
|
3938
|
-
|
|
3939
|
-
`document(id, at)` materializes one specific incarnation and never follows a
|
|
3940
|
-
replacement at the same logical address. Callers resolve an address with
|
|
3941
|
-
`findDocument()` when they do not already hold an incarnation ID. It selects the
|
|
3942
|
-
newest applicable base, applies its ordered Chord delta tail, and returns the detached materialized value, stored
|
|
3943
|
-
definition version, and number of replayed deltas after that base. Base/delta records are backend-private. The lookup never
|
|
3944
|
-
scans unrelated documents. An unknown ID returns `undefined`. At `"current"`, a
|
|
3945
|
-
retired incarnation returns `undefined`. A numeric lookup of a rewindable
|
|
3946
|
-
conversation incarnation returns `undefined` outside its half-open lifetime and
|
|
3947
|
-
reconstructs the selected value inside it. A numeric lookup of a known current-
|
|
3948
|
-
only incarnation rejects rather than depending on reclaimed content. Metadata
|
|
3949
|
-
membership remains queryable historically. A missing required base, a version
|
|
3950
|
-
change inside a delta tail, or an operation that cannot be applied inside an
|
|
3951
|
-
addressable lifetime is storage corruption, not absence.
|
|
3952
|
-
|
|
3953
|
-
One normalized batch contains at most one create/change content command per
|
|
3954
|
-
incarnation and may also retire that incarnation. Storage applies content before
|
|
3955
|
-
retirement independent of write-array order. Create plus retire stamps both
|
|
3956
|
-
lifetime bounds with the batch sequence. Retire plus create at one logical
|
|
3957
|
-
address makes the new incarnation current at that sequence. Deltas cannot cross
|
|
3958
|
-
a stored version boundary; a version transition must be a base.
|
|
3959
|
-
|
|
3960
|
-
The semantic conformance suite covers memory, SQLite, and JSONL.
|
|
3961
|
-
|
|
3962
|
-
## 11. Backends
|
|
3963
|
-
|
|
3964
|
-
### 11.1 Memory
|
|
3965
|
-
|
|
3966
|
-
Memory storage is the reference semantics. It copies retained write values and
|
|
3967
|
-
all read results. This deliberately simulates the ownership boundary naturally
|
|
3968
|
-
created by SQLite encoding/decoding and JSONL serialization; it is not defensive
|
|
3969
|
-
validation. It preserves rewindable records and reclaims latest records only
|
|
3970
|
-
after a committed base or retirement.
|
|
3971
|
-
|
|
3972
|
-
### 11.2 SQLite
|
|
3973
|
-
|
|
3974
|
-
One SQL transaction is one Session commit. SQLite stores:
|
|
3975
|
-
|
|
3976
|
-
- conversation, entry, task, and submission records;
|
|
3977
|
-
- document records;
|
|
3978
|
-
- indexed document bases/deltas by document and commit sequence.
|
|
3979
|
-
|
|
3980
|
-
Live task transitions replace one row. Terminal tasks remain as small records.
|
|
3981
|
-
Document reads use indexed base-plus-tail ranges. The first implementation stores
|
|
3982
|
-
Chord records directly; it does not translate generic operations to SQLite JSON
|
|
3983
|
-
functions.
|
|
3984
|
-
|
|
3985
|
-
Schema shape, WAL checkpoint cadence, and synchronous defaults are backend
|
|
3986
|
-
implementation choices validated by conformance, reopen, query-plan, and storage
|
|
3987
|
-
benchmarks.
|
|
3988
|
-
|
|
3989
|
-
### 11.3 JSONL
|
|
3990
|
-
|
|
3991
|
-
JSONL depends on the portable `FileSystem` capability, not the broader
|
|
3992
|
-
`ExecutionEnv`. It uses reclaimable sidecars without exposing them to the
|
|
3993
|
-
harness. Persistence alone does not provide the ownership boundary: any decoded
|
|
3994
|
-
indexes, materialized values, or caches retained in memory must be detached from
|
|
3995
|
-
commit arguments and must not be exposed directly by reads. A JSONL backend
|
|
3996
|
-
cannot simply add file appends around aliasing memory tables.
|
|
3997
|
-
|
|
3998
|
-
```text
|
|
3999
|
-
main.jsonl table writes, document records, and one marker per commit
|
|
4000
|
-
doc-<id>.jsonl one document incarnation
|
|
4001
|
-
task-<id>.jsonl live task replacements
|
|
4002
|
-
```
|
|
4003
|
-
|
|
4004
|
-
Publication protocol:
|
|
4005
|
-
|
|
4006
|
-
1. Append complete prepared records to every affected sidecar.
|
|
4007
|
-
2. Append one complete main marker listing those records.
|
|
4008
|
-
3. Publish in memory only after the marker write succeeds.
|
|
4009
|
-
|
|
4010
|
-
Every commit uses this protocol; there is no standalone-sidecar fast path.
|
|
4011
|
-
JSONL creation accepts an `fsync` option that defaults to `false`. Without
|
|
4012
|
-
`fsync`, it guarantees ordinary process-crash consistency, not survival of
|
|
4013
|
-
power, host, kernel, or filesystem failure. With `fsync: true`, the backend
|
|
4014
|
-
appends all affected sidecar records, flushes each affected sidecar, and only
|
|
4015
|
-
then appends the main marker. Ordinary publication does not explicitly flush
|
|
4016
|
-
`main.jsonl`; an acknowledged tail commit may therefore still disappear, but a
|
|
4017
|
-
marker that survives should not overtake its sidecar data. A main-only commit has
|
|
4018
|
-
no sidecars to flush. Before destructive reclamation with `fsync: true`, the
|
|
4019
|
-
backend flushes `main.jsonl` once so the authorizing marker cannot disappear
|
|
4020
|
-
while its replacement or removal survives. If that flush fails, the committed
|
|
4021
|
-
state remains published and reclamation is deferred. A non-empty temporary
|
|
4022
|
-
replacement is also flushed before rename.
|
|
4023
|
-
|
|
4024
|
-
Recovery:
|
|
4025
|
-
|
|
4026
|
-
- Remove torn final lines.
|
|
4027
|
-
- Ignore and remove unconfirmed sidecar tails.
|
|
4028
|
-
- Apply confirmed records only.
|
|
4029
|
-
- Missing required confirmed data is corruption and opening fails.
|
|
4030
|
-
- A later committed latest base or retirement may prove an earlier physical
|
|
4031
|
-
record unnecessary.
|
|
4032
|
-
- Any uncertain append failure poisons the open backend.
|
|
4033
|
-
|
|
4034
|
-
Reclamation starts only after the authorizing base/retirement commits. When no
|
|
4035
|
-
sidecar records remain, it removes the sidecar directly. Otherwise, it writes a
|
|
4036
|
-
temporary replacement, renames it, and invalidates cached file descriptors so
|
|
4037
|
-
future appends cannot target an unlinked inode. Flushing `main.jsonl` to
|
|
4038
|
-
authorize reclamation does not compact it. `main.jsonl` is not compacted in the
|
|
4039
|
-
initial implementation.
|
|
4040
|
-
|
|
4041
|
-
## 12. API footguns
|
|
4042
|
-
|
|
4043
|
-
These are contracts, not invitations to add defensive machinery:
|
|
4044
|
-
|
|
4045
|
-
- **Detached draft work:** draft reads, draft writes, and all `Tx` operations
|
|
4046
|
-
reject after the Session callback settles. Fire-and-forget work that runs
|
|
4047
|
-
before callback settlement can still mutate the active transaction and is
|
|
4048
|
-
unsupported.
|
|
4049
|
-
- **Read after write:** read every required table row before the first table
|
|
4050
|
-
write. Document drafts remain usable afterward; table reads do not.
|
|
4051
|
-
- **Writing to a busy conversation:** only the Harness appends to a conversation
|
|
4052
|
-
with an active run. Raw entries appended by `Harness.commit()` or a custom
|
|
4053
|
-
task while a generation prepares its request can misplace its system prompt entries;
|
|
4054
|
-
use a write submission.
|
|
4055
|
-
- **Queued items after a failed run:** failure and task abort leave the inbox alone.
|
|
4056
|
-
Queued follow-ups wait in `pi.inbox`, and their `wait()` does not settle, until
|
|
4057
|
-
the next submission's boundary places them or the host withdraws them. A
|
|
4058
|
-
compaction summary is such a submission: when a manual or background
|
|
4059
|
-
compaction finishes in that idle conversation, its final boundary places the
|
|
4060
|
-
waiting follow-ups and starts a run.
|
|
4061
|
-
- **Compaction spend:** every manual and background compaction pays for one
|
|
4062
|
-
summarization request per attempt, also when its summary ends stale because a
|
|
4063
|
-
later compaction cut further.
|
|
4064
|
-
- **Heads and edits during compaction:** a summary reflects the context when its
|
|
4065
|
-
compaction selected the range. An application edit placed while it
|
|
4066
|
-
summarizes, such as a write replacing entry 20, is lost when the summary cuts
|
|
4067
|
-
past its target: the summary still describes the old entry 20, and the edit's
|
|
4068
|
-
target leaves the range. An application head placed meanwhile, for example
|
|
4069
|
-
one cutting at 85 to forget entries 10-84, is undone by a summary cutting at 90:
|
|
4070
|
-
that summary still describes 10-84. Place such writes before compacting.
|
|
4071
|
-
- **Summary timestamps:** a queued summary's user message carries the time its
|
|
4072
|
-
compaction finished, not its placement. Code that judges usage staleness by
|
|
4073
|
-
message timestamps, such as pi-ai `estimateContextTokens()` over view
|
|
4074
|
-
messages, can misjudge; the Harness estimate uses entry order (section 8.3).
|
|
4075
|
-
- **Raw head writes into the past:** a head written directly with `tx.appendEntry()`
|
|
4076
|
-
that targets an entry before the conversation's active range changes model
|
|
4077
|
-
context, but a mounted view keeps only the entries it already holds until the
|
|
4078
|
-
mount is rebuilt. Use a write submission, whose stale check rejects it.
|
|
4079
|
-
- **Double-counted subagent spend:** a tool that runs an owned conversation must
|
|
4080
|
-
not report that conversation's usage in its result; the child's `pi.usage`
|
|
4081
|
-
already counts it, and subtree sums would count it twice.
|
|
4082
|
-
- **Owned work holds its owner:** a task that owns live ordinary work stays
|
|
4083
|
-
`completing` until that work ends (section 5.5). Foreground work an extension
|
|
4084
|
-
starts in a subagent's conversation holds the calling tool, and with it the
|
|
4085
|
-
run; interrogating a subagent while its tool is completing extends the hold.
|
|
4086
|
-
Esc or a host timeout ends it; work that should not hold its owner is created
|
|
4087
|
-
conversation-owned and `background`.
|
|
4088
|
-
- **Work created by hooks:** hooks run inside the asking task's invocation, so
|
|
4089
|
-
work they create owned by that task holds it. Work that must delay a task's
|
|
4090
|
-
finish has to be created before the task commits its outcome; a listener
|
|
4091
|
-
reacting to published events afterwards cannot extend it. A run task's owned
|
|
4092
|
-
work must not write the transcript after the run was released.
|
|
4093
|
-
- **Compensation in abort handlers:** an abort handler cannot create owned
|
|
4094
|
-
children, because its task is abort-marked. Compensate at the level
|
|
4095
|
-
that did the effect, in that task's own abort handler, or inline, or through
|
|
4096
|
-
background conversation-owned tasks awaited with `waitForTask()`; a
|
|
4097
|
-
non-background one would be aborted by any cascade that reaches its
|
|
4098
|
-
conversation.
|
|
4099
|
-
- **Hook memo names:** hooks share the asking task's memo namespace with the
|
|
4100
|
-
task and other hooks. Prefix memo names.
|
|
4101
|
-
- **Long transactions:** an async commit callback holds the Session mutation
|
|
4102
|
-
line. Never await models, tools, processes, network calls, humans, a nested
|
|
4103
|
-
Session commit, or a Session waiter inside it. Use methods on the current `Tx`.
|
|
4104
|
-
- **Explicit creation:** only typed `tx.doc()` creates an absent document. Snapshot,
|
|
4105
|
-
state, and watch lookup return `undefined` instead.
|
|
4106
|
-
- **Family initialization:** the first acquisition of an absent family address
|
|
4107
|
-
selects its seed. Existing instances and later calls ignore seeds; a seed is
|
|
4108
|
-
neither identity nor an update.
|
|
4109
|
-
- **Checkpoint starvation:** if `checkpointWhen()` never returns true, replay
|
|
4110
|
-
and current-only document storage can grow without bound while the document is
|
|
4111
|
-
live.
|
|
4112
|
-
- **Wrong fork setting:** `current`, `initial`, and `asOf` are product semantics,
|
|
4113
|
-
not optimizations. Changing one changes child conversation behavior.
|
|
4114
|
-
- **Schema stability:** document kinds and visible mount paths are
|
|
4115
|
-
persisted/public protocol. Value migration cannot rename a kind; a kind
|
|
4116
|
-
change requires explicit copy and retirement. Passing incompatible definition
|
|
4117
|
-
tokens that claim the same kind is unsupported caller misuse; Session does not
|
|
4118
|
-
maintain a document-definition registry to detect it.
|
|
4119
|
-
- **Trusted immutable revisions:** `snapshot()`, state values, watch values, and
|
|
4120
|
-
their descendants may share tracker-owned containers. Never mutate them; copy
|
|
4121
|
-
first when mutable ownership is required. Runtime freezing is not provided.
|
|
4122
|
-
- **Watch activation:** before `start()`, `watch.value` remains the immutable
|
|
4123
|
-
acquisition revision. Initialize the consumer from it first. After start, the
|
|
4124
|
-
property advances to each delivered immutable revision before its callback;
|
|
4125
|
-
every earlier reference remains stable.
|
|
4126
|
-
- **Buffered watches:** slow or unstarted watches retain up to 100 exact committed
|
|
4127
|
-
frames. Overflow replaces the undelivered suffix with one full-value root
|
|
4128
|
-
replacement, so intermediate committed states may be omitted. A consumer that
|
|
4129
|
-
must audit every transition must persist each fact in an immutable entry or
|
|
4130
|
-
journal and scan that history explicitly.
|
|
4131
|
-
- **Watch stop:** a listener may call and await `stop()`; it prevents future
|
|
4132
|
-
callbacks but neither aborts nor joins the callback already running.
|
|
4133
|
-
- **Durable progress cadence:** clients see only committed progress. A crash may
|
|
4134
|
-
lose the current uncommitted throttle window.
|
|
4135
|
-
- **Large terminal results:** terminal task records remain queryable. Put large
|
|
4136
|
-
results in entries or longer-lived documents and retain only their IDs in the
|
|
4137
|
-
outcome. Never reference a task-scoped document retired by that same outcome.
|
|
4138
|
-
- **Raw transcript:** view entries are not model context. Rendering edits,
|
|
4139
|
-
display-only entries, and model filtering require the appropriate reducer.
|
|
4140
|
-
- **Reserved `pi.` names:** task names, document kinds, and entry kinds starting
|
|
4141
|
-
with `pi.` belong to built-ins by convention. Nothing enforces it; reusing one
|
|
4142
|
-
collides with Harness behavior, such as `pi.system` entries being replayed as
|
|
4143
|
-
system messages.
|
|
4144
|
-
- **Early resource disposal:** disposing a `Registration` only stops new use.
|
|
4145
|
-
Freeing resources immediately can fail calls that are still running.
|
|
4146
|
-
- **Async batches:** `batch()` callbacks are synchronous. A callback that returns
|
|
4147
|
-
a promise publishes nothing and throws.
|
|
4148
|
-
- **Unstable prompt text:** a section renderer whose output changes without a
|
|
4149
|
-
real content change, for example by embedding the time, appends system deltas
|
|
4150
|
-
and defeats provider prompt caching.
|
|
4151
|
-
- **Durable stream options:** `streamOptions.headers` and `metadata` are stored
|
|
4152
|
-
in the conversation configuration history and copied into forks. Never put
|
|
4153
|
-
credentials there; `Models` resolves auth.
|
|
4154
|
-
- **Fatal storage errors:** after an uncertain storage failure the Session is
|
|
4155
|
-
poisoned. Do not catch the error and continue using it.
|
|
4156
|
-
- **JSONL durability:** default JSONL ordering handles ordinary process crashes;
|
|
4157
|
-
without durable mode it does not promise acknowledged commits survive power or
|
|
4158
|
-
host failure.
|
|
4159
|
-
|
|
4160
|
-
## 13. Non-goals
|
|
4161
|
-
|
|
4162
|
-
Pico5 initially has no:
|
|
4163
|
-
|
|
4164
|
-
- whole-Session DOM;
|
|
4165
|
-
- visible-undurable publication;
|
|
4166
|
-
- Session-kernel semantic event journal or independently maintained event state;
|
|
4167
|
-
- session-scoped rewindable documents;
|
|
4168
|
-
- automatic checkpoint heuristic;
|
|
4169
|
-
- automatic third-party view mounting;
|
|
4170
|
-
- CRDT/offline multi-writer merge;
|
|
4171
|
-
- SQL translation of arbitrary Chord operations;
|
|
4172
|
-
- JSONL global compaction or automatic corruption repair;
|
|
4173
|
-
- compatibility layer for removed Pico prototypes;
|
|
4174
|
-
- forced termination of non-cooperative extension code inside one process.
|