my-durable 0.0.0-stage → 0.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +7 -0
- package/LICENSE +21 -0
- package/README.md +328 -2
- package/ai_docs/durable_pi/announcement.md +860 -0
- package/ai_docs/durable_pi/changelog.md +51 -0
- package/ai_docs/durable_pi/chord-delta-findings.md +362 -0
- package/ai_docs/durable_pi/coding-agent-integration.md +70 -0
- package/ai_docs/durable_pi/pico-v5-chord-usage.md +391 -0
- package/ai_docs/durable_pi/pico-v5-handoff.md +921 -0
- package/ai_docs/durable_pi/readme.md +617 -0
- package/ai_docs/durable_pi/sources.md +22 -0
- package/ai_docs/durable_pi/spec.md +4696 -0
- package/ai_docs/durable_pi/vacation-planner.md +57 -0
- package/ai_docs/pi/chord.md +268 -0
- package/ai_docs/pi/cli-integration.md +110 -0
- package/ai_docs/pi/cli.md +325 -0
- package/ai_docs/pi/codemode.md +211 -0
- package/ai_docs/pi/compaction.md +467 -0
- package/ai_docs/pi/configuration.md +51 -0
- package/ai_docs/pi/containerization.md +187 -0
- package/ai_docs/pi/custom-provider.md +177 -0
- package/ai_docs/pi/environment-variables.md +102 -0
- package/ai_docs/pi/extensions.md +278 -0
- package/ai_docs/pi/how-pi-works.md +53 -0
- package/ai_docs/pi/index.md +43 -0
- package/ai_docs/pi/json.md +230 -0
- package/ai_docs/pi/keybindings.md +198 -0
- package/ai_docs/pi/llama-cpp.md +118 -0
- package/ai_docs/pi/mcp.md +274 -0
- package/ai_docs/pi/message-types.md +265 -0
- package/ai_docs/pi/models.md +209 -0
- package/ai_docs/pi/overview.md +166 -0
- package/ai_docs/pi/packages.md +133 -0
- package/ai_docs/pi/pi-agent-core.md +568 -0
- package/ai_docs/pi/pi-ai.md +1942 -0
- package/ai_docs/pi/prompt-templates.md +63 -0
- package/ai_docs/pi/providers.md +204 -0
- package/ai_docs/pi/quickstart.md +140 -0
- package/ai_docs/pi/readme.md +99 -0
- package/ai_docs/pi/rpc-commands.md +864 -0
- package/ai_docs/pi/rpc-extension-ui.md +204 -0
- package/ai_docs/pi/rpc.md +197 -0
- package/ai_docs/pi/sdk.md +151 -0
- package/ai_docs/pi/security.md +103 -0
- package/ai_docs/pi/session-format.md +297 -0
- package/ai_docs/pi/sessions.md +70 -0
- package/ai_docs/pi/settings.md +173 -0
- package/ai_docs/pi/shell-aliases.md +97 -0
- package/ai_docs/pi/skills.md +97 -0
- package/ai_docs/pi/slash-commands.md +64 -0
- package/ai_docs/pi/sources.md +57 -0
- package/ai_docs/pi/terminal-setup.md +223 -0
- package/ai_docs/pi/termux.md +121 -0
- package/ai_docs/pi/themes.md +140 -0
- package/ai_docs/pi/tmux.md +59 -0
- package/ai_docs/pi/toolkit.md +168 -0
- package/ai_docs/pi/tui.md +129 -0
- package/ai_docs/pi/usage.md +98 -0
- package/ai_docs/pi/virtual-models.md +118 -0
- package/ai_docs/pi/windows.md +71 -0
- package/apps/dpi-backend/.nvmrc +1 -0
- package/apps/dpi-backend/API.md +99 -0
- package/apps/dpi-backend/README.md +119 -0
- package/apps/dpi-backend/deploy/backend.env.example +9 -0
- package/apps/dpi-backend/deploy/dpi-backend.plist +26 -0
- package/apps/dpi-backend/deploy/dpi-backend.service +22 -0
- package/apps/dpi-backend/package.json +33 -0
- package/apps/dpi-backend/prompts/one_sentence.md +3 -0
- package/apps/dpi-backend/prompts/smart_ass_opus_fix.md +169 -0
- package/apps/dpi-backend/scripts/check-contract.mjs +151 -0
- package/apps/dpi-backend/scripts/migrate-playground.ts +66 -0
- package/apps/dpi-backend/scripts/record-nested-fixture.mts +154 -0
- package/apps/dpi-backend/scripts/token.ts +57 -0
- package/apps/dpi-backend/src/addons.ts +139 -0
- package/apps/dpi-backend/src/auth.ts +20 -0
- package/apps/dpi-backend/src/config.ts +54 -0
- package/apps/dpi-backend/src/contract-check.ts +118 -0
- package/apps/dpi-backend/src/errors.ts +50 -0
- package/apps/dpi-backend/src/extensions.ts +510 -0
- package/apps/dpi-backend/src/faux.ts +80 -0
- package/apps/dpi-backend/src/history.ts +165 -0
- package/apps/dpi-backend/src/idempotency.ts +20 -0
- package/apps/dpi-backend/src/jev.ts +43 -0
- package/apps/dpi-backend/src/jobs.ts +790 -0
- package/apps/dpi-backend/src/main.ts +76 -0
- package/apps/dpi-backend/src/model-catalog.ts +180 -0
- package/apps/dpi-backend/src/pairing-files.ts +142 -0
- package/apps/dpi-backend/src/pairing.ts +459 -0
- package/apps/dpi-backend/src/routes.ts +217 -0
- package/apps/dpi-backend/src/shared-state.ts +85 -0
- package/apps/dpi-backend/src/store.ts +1240 -0
- package/apps/dpi-backend/src/streams.ts +256 -0
- package/apps/dpi-backend/src/tunnel.ts +955 -0
- package/apps/dpi-backend/tsconfig.json +13 -0
- package/apps/dpi-playground/README.md +83 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/assets/index-CFGc7lHT.js +77 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/assets/index-DFdKpwm_.css +1 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/anthropic.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/deepseek.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/durable-pi.svg +14 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/gemini.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/grok.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/kimi.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/lab.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/meta.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/minimax.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/openai.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/qwen.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/xiaomi.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/icons/zai.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/dist/index.html +16 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/index.html +15 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/package.json +5 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/anthropic.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/deepseek.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/durable-pi.svg +14 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/gemini.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/grok.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/kimi.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/lab.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/meta.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/minimax.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/openai.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/qwen.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/xiaomi.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/public/icons/zai.png +0 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/App.vue +180 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/commands.ts +119 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/composables.ts +129 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/hub.ts +40 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/icon-path.ts +7 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/ids.ts +2 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/jobs.ts +80 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/link-policy.ts +16 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/markdown-policy.ts +115 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/markdown.ts +60 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/pair-close.ts +23 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/pair-handoff.ts +56 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/pair.ts +166 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/qrcode.ts +323 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/signature.ts +146 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/store.ts +373 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/client/stream.ts +183 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/AgentView.vue +342 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/Composer.vue +198 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/ContextBar.vue +254 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/DeviceCredentialsDialog.vue +40 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/DeviceIcon.vue +56 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/DirBrowser.vue +95 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/ExtensionsPanel.vue +96 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/HelpDialog.vue +69 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/Inspector.vue +425 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/JobCard.vue +221 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/MobileNav.vue +39 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/ModelPicker.vue +71 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/NetworkSidebar.vue +236 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/NewSessionDialog.vue +114 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/OfflinePanel.vue +128 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/PairDeviceWizard.vue +433 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/PromptDialog.vue +89 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/StateEditor.vue +179 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/TasksPanel.vue +280 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/Toast.vue +11 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/TokenManager.vue +175 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/TopBar.vue +114 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/TranscriptItem.vue +152 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/TreeRow.vue +195 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/components/TunnelStatusBar.vue +107 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/extensions.css +105 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/jobs.css +655 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/main.ts +22 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/network.css +675 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/src/style.css +3203 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/tsconfig.json +16 -0
- package/apps/dpi-playground/apps/dpi-playground-fe/vite.config.ts +14 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/package.json +5 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/audit.ts +406 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/client-ip.ts +72 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/credentials.ts +265 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/guard.ts +43 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/http-errors.ts +156 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/identity.ts +60 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/limiter.ts +165 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/main.ts +460 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/monitor.ts +320 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/mux.ts +277 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/pairing.ts +816 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/proxy.ts +128 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/rotation.ts +583 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/seed.ts +11 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/transport.ts +108 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/tunnel-token.ts +11 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/tunnel-transport.ts +560 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/src/tunnel.ts +873 -0
- package/apps/dpi-playground/apps/dpi-playground-hub/tsconfig.json +13 -0
- package/apps/dpi-playground/package.json +37 -0
- package/apps/dpi-sdk/README.md +74 -0
- package/apps/dpi-sdk/package.json +20 -0
- package/apps/dpi-sdk/scripts/net.ts +73 -0
- package/apps/dpi-sdk/src/api.ts +631 -0
- package/apps/dpi-sdk/src/app-state.ts +111 -0
- package/apps/dpi-sdk/src/client.ts +330 -0
- package/apps/dpi-sdk/src/errors.ts +16 -0
- package/apps/dpi-sdk/src/jobs.ts +99 -0
- package/apps/dpi-sdk/src/models.ts +472 -0
- package/apps/dpi-sdk/src/node/chunk.ts +488 -0
- package/apps/dpi-sdk/src/node/config.ts +175 -0
- package/apps/dpi-sdk/src/node/drivers.ts +402 -0
- package/apps/dpi-sdk/src/node/ops.ts +114 -0
- package/apps/dpi-sdk/src/node/private-dir.ts +48 -0
- package/apps/dpi-sdk/src/node/registry-migration.ts +549 -0
- package/apps/dpi-sdk/src/node/registry.ts +390 -0
- package/apps/dpi-sdk/src/node/sse-writer.ts +151 -0
- package/apps/dpi-sdk/src/protocol/backoff.ts +31 -0
- package/apps/dpi-sdk/src/protocol/limits.ts +199 -0
- package/apps/dpi-sdk/src/protocol/pairing.ts +311 -0
- package/apps/dpi-sdk/src/protocol/rest-routes.ts +105 -0
- package/apps/dpi-sdk/src/protocol/tunnel.ts +331 -0
- package/apps/dpi-sdk/src/sse.ts +117 -0
- package/apps/dpi-sdk/src/transcript.ts +291 -0
- package/apps/dpi-sdk/tsconfig.json +13 -0
- package/apps/dpi-sdk/w +20 -0
- package/apps/dpi-tui/README.md +54 -0
- package/apps/dpi-tui/bin/dpi-tui.mjs +5 -0
- package/apps/dpi-tui/package.json +20 -0
- package/apps/dpi-tui/src/app.ts +433 -0
- package/apps/dpi-tui/src/commands.ts +151 -0
- package/apps/dpi-tui/src/main.ts +122 -0
- package/apps/dpi-tui/src/pairing.ts +287 -0
- package/apps/dpi-tui/src/pickers.ts +238 -0
- package/apps/dpi-tui/src/render.ts +164 -0
- package/apps/dpi-tui/src/theme.ts +57 -0
- package/apps/dpi-tui/tsconfig.json +14 -0
- package/bin/device.mjs +5 -0
- package/bin/hub.mjs +5 -0
- package/bin/tui.mjs +5 -0
- package/deploy/Caddyfile +120 -0
- package/docs/verification.md +102 -0
- package/dpi-network.example.yaml +62 -0
- package/images/01_durable_agent_network.svg +185 -0
- package/images/02_pi_made_durable.svg +204 -0
- package/images/03_restart_mid_tool.svg +168 -0
- package/images/07_one_api_every_client.svg +123 -0
- package/images/08_device_session_agent.svg +123 -0
- package/images/09_compact_keep_history.svg +143 -0
- package/images/10_shared_app_state.svg +177 -0
- package/images/11_durable_jobs_wait.svg +141 -0
- package/images/12_tools_change_mid_run.svg +231 -0
- package/images/13_hook_blocks_tool.svg +122 -0
- package/images/14_fork_at_any_entry.svg +119 -0
- package/images/15_web_client.png +0 -0
- package/images/16_terminal_client.svg +43 -0
- package/images/17_devices_are_data.svg +182 -0
- package/images/18_should_every_agent_be_durable.svg +118 -0
- package/images/19_sqlite_on_every_device.svg +155 -0
- package/images/20_admission_and_effects.svg +134 -0
- package/justfile +33 -0
- package/package.json +41 -4
- package/scripts/verify.mjs +210 -0
|
@@ -0,0 +1,921 @@
|
|
|
1
|
+
> Source: [packages/durable/docs/pico-v5-handoff.md](https://github.com/earendil-works/pi/blob/b78e6a9085343ec0f308c3d377da72528b4cf7ee/packages/durable/docs/pico-v5-handoff.md)
|
|
2
|
+
> Retrieved with Firecrawl: 2026-10-05. Repository snapshot: `b78e6a9085343ec0f308c3d377da72528b4cf7ee`.
|
|
3
|
+
> Relative documentation links point to local copies where available, otherwise to the source snapshot.
|
|
4
|
+
|
|
5
|
+
# Pico5 implementation handoff
|
|
6
|
+
|
|
7
|
+
`packages/durable/docs/spec.md` is normative. Implement this list in order.
|
|
8
|
+
After every package: run its tests, run `npm run check`, and stop for user review.
|
|
9
|
+
Do not redesign later packages while implementing the current one.
|
|
10
|
+
|
|
11
|
+
Pico3 is reference material only. Preserve useful behavior, not its capability
|
|
12
|
+
facades, membranes, document routing, view projection, events, or clone chains.
|
|
13
|
+
|
|
14
|
+
## Status
|
|
15
|
+
|
|
16
|
+
- Obsolete `pico` and `pico4` prototypes were removed.
|
|
17
|
+
- `pico3` remains.
|
|
18
|
+
- Packages 1–23 are implemented in `packages/durable`; Package 10 was already satisfied by Chord's canonical structural diff implementation.
|
|
19
|
+
|
|
20
|
+
## 1. Records, cursors, and memory tables
|
|
21
|
+
|
|
22
|
+
Implement IDs, sequences, reserved root conversation ID `1`,
|
|
23
|
+
`ConversationRecord`, `EntryRecord`, strict input/write `SubmissionRecord`
|
|
24
|
+
values, live/terminal `TaskRecord` values, document records, storage writes, backend-opaque JSON cursors, and
|
|
25
|
+
detached `MemoryStorage` tables.
|
|
26
|
+
Reserve `Conversation` for the public conversation object, `Entry` for the typed
|
|
27
|
+
entry definition, and `Task` for the typed executable definition returned by
|
|
28
|
+
`defineTask()`.
|
|
29
|
+
|
|
30
|
+
Test reserved root identity and immutable creation, mixed atomic commits,
|
|
31
|
+
rollback, detached reads/writes, cursor boundaries,
|
|
32
|
+
fork-aware entry scans through deep ancestor caps, head lookup,
|
|
33
|
+
entry-to-commit lookup, full task replacement, and submission replacement/
|
|
34
|
+
request-ID lookup.
|
|
35
|
+
|
|
36
|
+
## 2. Memory document records
|
|
37
|
+
|
|
38
|
+
Add selected document base/delta writes, retirement, reincarnation,
|
|
39
|
+
current/as-of membership, exact logical-address lookup, scoped scans, and
|
|
40
|
+
materialized point-in-time reads. Storage keeps base/delta revisions private and
|
|
41
|
+
returns a detached value plus its stored definition version. It applies Chord
|
|
42
|
+
`Op[]` directly and receives no definition callbacks or unused candidate values.
|
|
43
|
+
|
|
44
|
+
Test Session-, conversation-, and task-scoped documents, half-open lifetimes,
|
|
45
|
+
create-plus-retire, retired historical membership, family queries, current-only
|
|
46
|
+
reclamation, version boundaries, detached ownership, and no scans of unrelated
|
|
47
|
+
document records.
|
|
48
|
+
|
|
49
|
+
## 3. SQLite backend
|
|
50
|
+
|
|
51
|
+
Implement the complete storage contract with ordinary rows and indexed document
|
|
52
|
+
records. Do not translate Chord operations into SQL JSON patches.
|
|
53
|
+
|
|
54
|
+
Run the memory conformance suite after reopen. Test SQL transaction rollback,
|
|
55
|
+
recent/ancient as-of reads, query plans, latest reclamation, WAL checkpointing,
|
|
56
|
+
deleted-page reuse, and representative storage sizes.
|
|
57
|
+
|
|
58
|
+
## 4. JSONL publication
|
|
59
|
+
|
|
60
|
+
Implement table writes in `main.jsonl`, one document sidecar per incarnation,
|
|
61
|
+
one sidecar per live task, and one main marker per commit. Do not add a
|
|
62
|
+
standalone-sidecar protocol.
|
|
63
|
+
|
|
64
|
+
Copy, rather than import, the current `ExecutionEnv`, `FileSystem`, `Shell`, Node
|
|
65
|
+
implementation, and their required utility files from `packages/agent/src/harness`
|
|
66
|
+
into `packages/durable/src/env`. Copy only the environment-related slice, not
|
|
67
|
+
agent skills, prompts, telemetry, or tool definitions. Extend the copied
|
|
68
|
+
filesystem contract with exact-byte file truncation and file flushing. JSONL
|
|
69
|
+
depends only on `FileSystem`, not the broader `ExecutionEnv`. Keep the portable
|
|
70
|
+
environment and JSONL entry points free of Node built-ins; expose Node
|
|
71
|
+
implementations only from `/env/node` and `/storage/jsonl/node`. Do not use the
|
|
72
|
+
Pico3 implementation as source material.
|
|
73
|
+
|
|
74
|
+
Refactor the current `MemoryStorage` state machinery into a two-phase prepared
|
|
75
|
+
mutation: validation and detachment produce a candidate that can later be
|
|
76
|
+
applied without failure. Build `MemoryStorage.commit()` on that pair, and reuse
|
|
77
|
+
the same machinery for JSONL. JSONL must append every prepared sidecar record,
|
|
78
|
+
append the main marker, and only then apply the prepared in-memory mutation.
|
|
79
|
+
Serialization or preparation failure occurs before file I/O and does not poison
|
|
80
|
+
the backend. Retained indexes/materializations remain detached from write
|
|
81
|
+
arguments, and reads never expose backend-owned cached objects.
|
|
82
|
+
|
|
83
|
+
JSONL creation has `fsync?: boolean`, defaulting to `false`. With `false`, append
|
|
84
|
+
sidecars and then the marker without an explicit flush. With `true`, append all
|
|
85
|
+
affected sidecars, flush each affected sidecar, and then append the main marker.
|
|
86
|
+
Do not explicitly flush `main.jsonl` for ordinary publication. A main-only commit
|
|
87
|
+
has no sidecars to flush. Any uncertain publication append or flush failure
|
|
88
|
+
poisons the open backend and publishes no prepared in-memory mutation. Package 5
|
|
89
|
+
adds the separate post-publication flush required to authorize reclamation.
|
|
90
|
+
|
|
91
|
+
Fault-test torn/short sidecar writes, failures between sidecars, every marker
|
|
92
|
+
boundary, unconfirmed tails, missing confirmed data, poisoned writes, exact-byte
|
|
93
|
+
tail truncation, both fsync settings and their call ordering, detached retained
|
|
94
|
+
state and reads, and browser-safe portable entry points. Run the complete storage
|
|
95
|
+
conformance suite directly and after reopen.
|
|
96
|
+
|
|
97
|
+
## 5. JSONL reclamation
|
|
98
|
+
|
|
99
|
+
Implement task-document retirement and current-only base reclamation using
|
|
100
|
+
committed markers and descriptor invalidation. Remove a sidecar directly when no
|
|
101
|
+
records remain; otherwise use temporary replacement and rename. With `fsync:
|
|
102
|
+
true`, flush `main.jsonl` once before destructive reclamation so the authorizing
|
|
103
|
+
marker cannot disappear while cleanup survives; if that flush fails, skip
|
|
104
|
+
reclamation without failing the already-published commit. Flush a non-empty
|
|
105
|
+
temporary replacement before rename. This is not publication flushing or main
|
|
106
|
+
compaction.
|
|
107
|
+
|
|
108
|
+
Crash-test every rewrite/rename boundary. Verify that rewindable history is
|
|
109
|
+
never reclaimed and default no-fsync behavior matches the specification.
|
|
110
|
+
|
|
111
|
+
## 6–7. Tracker transaction core, definitions, and typed access
|
|
112
|
+
|
|
113
|
+
**Prerequisite:** `@earendil-works/chord/delta` exports the canonical
|
|
114
|
+
Astra-immutable-optimized `track`, `Tracker`, `Change`, and `Prepared`, and its
|
|
115
|
+
draft placements reject values that are not strict JSON.
|
|
116
|
+
Experimental variants under other Delta directories are not Pico APIs.
|
|
117
|
+
|
|
118
|
+
Implement these packages as one milestone. Keep the implementation layers
|
|
119
|
+
separate, but do not build a temporary untyped document-acquisition seam.
|
|
120
|
+
Implement the generic `Tx` table surface these tests require: exact table reads,
|
|
121
|
+
paginated conversation/entry/task scans, `ReadAfterWrite`, ID-creating writes,
|
|
122
|
+
and full task replacement. Scans expose the Storage cursor and caller-selected
|
|
123
|
+
limit; they never hide an unbounded full scan. Semantic
|
|
124
|
+
conversation, entry, task, and scheduler behavior remains in Packages 13–17.
|
|
125
|
+
|
|
126
|
+
Keep one Astra-immutable tracker per loaded document. Its trusted immutable
|
|
127
|
+
`value` is the current shareable revision. `prepare()` emits detached
|
|
128
|
+
self-contained operations and computes the next revision with the optimized
|
|
129
|
+
immutable applier; operation placement payloads and that revision may share
|
|
130
|
+
containers, and neither may be mutated. `adopt()` validates ownership and
|
|
131
|
+
revision, then only pointer-swaps to the already-computed value.
|
|
132
|
+
|
|
133
|
+
Implement scope-preserving singleton/family tokens and overloads for Session,
|
|
134
|
+
conversation, and task owners. Only `tx.doc()` is get-or-create: singleton tokens
|
|
135
|
+
supply `initial()`, while family calls always supply key and seed and use only the
|
|
136
|
+
first seed when absent. Definitions are explicit typed arguments, not registered
|
|
137
|
+
declarations; conflicting definitions claiming one persisted kind are
|
|
138
|
+
unsupported caller misuse.
|
|
139
|
+
|
|
140
|
+
On first `tx.doc()` access, memoize the acquisition promise by logical address
|
|
141
|
+
before awaiting it, then call `tracker.beginChange()`. Repeated access returns the
|
|
142
|
+
same overlay draft for the whole possibly async Session callback. The Session
|
|
143
|
+
line permits only one open change per tracker. A callback that settles with an
|
|
144
|
+
unresolved acquisition rejects: seal `Tx`, abort open changes, drain and abort
|
|
145
|
+
the pending acquisition, and observe its failure. Callback failure aborts every
|
|
146
|
+
change. Callback success prepares every change before Storage admission.
|
|
147
|
+
|
|
148
|
+
Do not walk prepared operation payloads or selected bases for strict JSON; they
|
|
149
|
+
are strict JSON by construction. Tracker branding and revision checks enforce
|
|
150
|
+
ownership and staleness. Evaluate each staged document write exactly once and
|
|
151
|
+
pass Storage only the selected base value or operation batch. Keep every previous immutable revision unchanged through Storage
|
|
152
|
+
settlement. On success, adopt every prepared value by pointer swap and enqueue
|
|
153
|
+
its immutable revision/operations publication before releasing the line. On
|
|
154
|
+
Storage failure, abort prepared changes, poison the Session, and publish nothing.
|
|
155
|
+
Preparation failures roll back normally; Package 8 adds checkpoint selection and
|
|
156
|
+
its failure path.
|
|
157
|
+
|
|
158
|
+
Initializer, migration, and replacement roots are copied into exclusive kernel
|
|
159
|
+
ownership with a strict-JSON check before becoming trusted immutable revisions.
|
|
160
|
+
Loaded and fork-copy roots come detached from Storage and are tracked without
|
|
161
|
+
another copy. Chord copies and strict-JSON-checks every draft placement and
|
|
162
|
+
throws at the offending assignment. Astra empty batches suppress
|
|
163
|
+
ordinary writes, while replayable nonempty structural no-ops remain valid writes
|
|
164
|
+
and publications. No runtime freezing or second operation-payload copy is
|
|
165
|
+
required.
|
|
166
|
+
|
|
167
|
+
Snapshot, source, and watch lookup never create and return `undefined` when
|
|
168
|
+
absent. `snapshot()` returns the shareable immutable current revision; callers
|
|
169
|
+
must copy before mutation. A read-only migration may cache its migrated immutable
|
|
170
|
+
tracker together with the older stored-version marker, without writing; the next
|
|
171
|
+
successful `tx.doc()` still writes the required current-version base.
|
|
172
|
+
Transaction-staged creation or migration enters the shared cache only after its
|
|
173
|
+
enclosing Storage commit succeeds. All cold loads run on the Session line; a
|
|
174
|
+
loaded immutable revision may be read without copying.
|
|
175
|
+
|
|
176
|
+
Test callback failure; escaped-draft revocation at callback settlement; concurrent duplicate
|
|
177
|
+
acquisition; callback failure and success with a pending acquisition; late
|
|
178
|
+
acquisition after sealing; concurrent initialization once; initial bases; family
|
|
179
|
+
first-seed wins; scope/token mismatch; non-creating reads; shared immutable
|
|
180
|
+
snapshots and stable prior revisions; empty-batch suppression and replayable
|
|
181
|
+
redundant structural no-ops; multi-document preparation failure; uncertain
|
|
182
|
+
Storage failure poisoning; old-revision stability through Storage settlement;
|
|
183
|
+
pointer-swap and replacement adoption; operation/revision payload sharing under
|
|
184
|
+
the trusted no-mutation contract; non-JSON initializer and draft-placement
|
|
185
|
+
rejection; assignment copying and repeated-placement
|
|
186
|
+
independence; authority and prepared-draft non-escape; terminal-task rejection;
|
|
187
|
+
task-derived conversation identity; retirement; reincarnation-bound sources;
|
|
188
|
+
and unload/reload. Include create-task-then-document,
|
|
189
|
+
document-after-terminal rejection, and create-document-then-terminal settlement
|
|
190
|
+
in one transaction; internal candidate validation must not trigger
|
|
191
|
+
`ReadAfterWrite`.
|
|
192
|
+
|
|
193
|
+
## 8. Checkpoints and migration
|
|
194
|
+
|
|
195
|
+
After tracker preparation, Session evaluates `checkpointWhen(value, ops, { deltasSinceBase })` exactly once
|
|
196
|
+
for ordinary mutations and sends Storage only the selected base or delta.
|
|
197
|
+
Implement required creation/version bases and lazy all-older-version migration
|
|
198
|
+
on typed access; Harness open does not scan ordinary documents.
|
|
199
|
+
|
|
200
|
+
Test read-only in-memory migration, `tx.doc()` migration rollback and coalescing
|
|
201
|
+
with later edits, rewindable migration on current/historical read, the first
|
|
202
|
+
successful `tx.doc()` version base even without a JSON change, newer-version
|
|
203
|
+
rejection, migrated state/watch hydration without a write, subsequent operations
|
|
204
|
+
against that migrated baseline, stored-version fork copying, unaccessed and unavailable-definition
|
|
205
|
+
preservation, predicate failure rollback before Storage admission, and checkpoint
|
|
206
|
+
starvation without backend heuristics.
|
|
207
|
+
|
|
208
|
+
## 9. Conversation document forks
|
|
209
|
+
|
|
210
|
+
Using fixture conversations and entry-to-commit mappings, implement the `asOf`,
|
|
211
|
+
`current`, and `initial` settings for singleton and family documents.
|
|
212
|
+
|
|
213
|
+
Test opaque stored-version copying without definitions, retired membership, new
|
|
214
|
+
child incarnations, later lazy migration, lazy `initial` creation, and exclusion
|
|
215
|
+
of task- and Session-scoped documents.
|
|
216
|
+
|
|
217
|
+
## 10. Chord structural array operations
|
|
218
|
+
|
|
219
|
+
**Chord-owned prerequisite/integration:** the canonical Astra-immutable operation
|
|
220
|
+
generator must encode compact replayable array changes; Pico only verifies and
|
|
221
|
+
consumes it.
|
|
222
|
+
|
|
223
|
+
Improve the canonical generator so ordinary positional mutations encode
|
|
224
|
+
scattered removals without carrying retained payloads. Callers must not write
|
|
225
|
+
operations manually.
|
|
226
|
+
|
|
227
|
+
Test front/tail/middle/scattered/all/no removal, retained 256 KiB and 1 MiB
|
|
228
|
+
payloads, append plus removal, later nested/index writes, exact replay, unchanged
|
|
229
|
+
previous immutable revisions, and equality between Astra's prepared candidate
|
|
230
|
+
and immutable operation replay. One prepared document change remains one Session
|
|
231
|
+
commit; no intermediate candidate is adopted or published.
|
|
232
|
+
|
|
233
|
+
## 11. Chord document state
|
|
234
|
+
|
|
235
|
+
Use Chord's existing `ReplicatedStateSource` attachment contract internally, but
|
|
236
|
+
expose one already-attached read-only `DocumentState` per acquisition. It must
|
|
237
|
+
atomically hydrate in O(1) from the current immutable revision and publish later
|
|
238
|
+
exact committed immutable value/operation frames without another tracker, value
|
|
239
|
+
copy, or re-diff. Disposal unregisters that one state. Pico remains the sole
|
|
240
|
+
document mutator.
|
|
241
|
+
|
|
242
|
+
Test contiguous Chord delivery sequences, atomic hydrate/subscribe, a baseline
|
|
243
|
+
that covers queued publication without duplicate application, exact value and
|
|
244
|
+
operation reference sharing, retirement to `null`, incarnation-bound recreation,
|
|
245
|
+
independent state disposal, migrated hydration, definition-free fork copies,
|
|
246
|
+
tracker-cache unload, and trusted mutation footguns.
|
|
247
|
+
|
|
248
|
+
## 12. Document watches
|
|
249
|
+
|
|
250
|
+
Implement non-creating `watchDoc` as an incarnation-bound `WatchHandle` that
|
|
251
|
+
returns `undefined` when absent and atomically captures the current immutable
|
|
252
|
+
revision in O(1) while registering for later exact committed frames. Before
|
|
253
|
+
`start()`, its value remains the acquisition revision. After start, invoke one
|
|
254
|
+
serialized asynchronous listener with each exact value and operation batch,
|
|
255
|
+
advancing `watch.value` immediately before the callback. Preserve commit Context
|
|
256
|
+
values without inheriting producer cancellation.
|
|
257
|
+
|
|
258
|
+
Retain at most 100 pending frames, excluding the in-flight callback. Adding frame
|
|
259
|
+
101 replaces the complete undelivered suffix with one root replacement carrying
|
|
260
|
+
the newest exact immutable revision and Context. Do not estimate serialized
|
|
261
|
+
bytes, call `JSON.stringify()` for accounting, copy values, replay operations, or
|
|
262
|
+
re-diff revisions.
|
|
263
|
+
|
|
264
|
+
Test updates between acquisition/return/start; exact values and operation
|
|
265
|
+
references; no callback overlap; listener-initiated commits; commits during an
|
|
266
|
+
in-flight callback; overflow before start and behind an active callback;
|
|
267
|
+
retirement folded into an overflow reset; replayable redundant structural
|
|
268
|
+
commits; delivery Context value preservation; retained earlier revision
|
|
269
|
+
stability; trusted mutation footguns; retirement before start and while active;
|
|
270
|
+
recreation; idempotent stop; second-start rejection; cancellation during
|
|
271
|
+
acquisition; cancellation/close during a callback without aborting or joining it;
|
|
272
|
+
listener-error settlement; and invocation-owned cleanup in package 14.
|
|
273
|
+
|
|
274
|
+
Packages 13–20 are vertical milestones. Each must leave one real public path
|
|
275
|
+
working end to end; do not defer all integration to the last package.
|
|
276
|
+
|
|
277
|
+
Cross-cutting constraints for these milestones:
|
|
278
|
+
|
|
279
|
+
- Introduce each persisted built-in document kind once, with its final schema,
|
|
280
|
+
history/fork policy, migration, checkpoint predicate, and public mount path.
|
|
281
|
+
Record that concrete protocol in the normative specification when the kind
|
|
282
|
+
lands. Do not add temporary built-in kinds or later replace a fake schema.
|
|
283
|
+
- Keep all visible progress durable. Tests may use faux models and fake effects,
|
|
284
|
+
but production code must use pi-ai's exported `Models` interface and the real
|
|
285
|
+
task chain; do not add a Pico model adapter or production fake successor.
|
|
286
|
+
- Prepare every loaded `ConversationView` revision from the complete candidate
|
|
287
|
+
Session commit before Storage admission. Never rebuild a view by subscribing
|
|
288
|
+
to already-committed table/document publications.
|
|
289
|
+
- A milestone may leave later operations unimplemented, but it must not expose a
|
|
290
|
+
temporary public facade. Extend the final §2.2 objects as later behavior lands.
|
|
291
|
+
|
|
292
|
+
## 13. Openable Harness
|
|
293
|
+
|
|
294
|
+
Implement the first usable slice of the final §2.2 surface: `Harness.open/close`
|
|
295
|
+
with a registry, `root(context, { init })`, `conversation(id)`,
|
|
296
|
+
`createConversation({ ownership, init })`, and generic
|
|
297
|
+
inherited Session document APIs (`Harness extends Session`). Implement the
|
|
298
|
+
public `Entry` definition token and `Conversation` handles rather than adding an
|
|
299
|
+
intermediate capability facade. `Conversation` provides `id`, conversation-bound
|
|
300
|
+
`commit()`, fork-aware cursor `entries()`, `context()`, `fork(at, { ownership,
|
|
301
|
+
init })`, and every configuration getter/setter.
|
|
302
|
+
|
|
303
|
+
Implement the §7.1 registry core: `createRegistry()`, `tools.add`, `tools.wrap`
|
|
304
|
+
composition, `batch`, keyed ordering with remembered positions, `Registration`
|
|
305
|
+
`dispose`, immutable `snapshot()`, and list helpers. Hook,
|
|
306
|
+
task, and system prompt registrations are stored and listed now; their dispatch
|
|
307
|
+
lands in Packages 14–16.
|
|
308
|
+
|
|
309
|
+
Bring conversation and entry semantics up to the normative contract: explicit
|
|
310
|
+
ownership, fork-aware cursor pagination, head lookup, entry edits, and model
|
|
311
|
+
context derivation. Context reduction includes newest-edit wins, positional PR
|
|
312
|
+
#9548 `SystemMessage` replay, tool-result ordering, missing post-fork tool
|
|
313
|
+
results, excluded stop reasons, and separate raw-history/model-context results.
|
|
314
|
+
|
|
315
|
+
Define the final `ConversationConfig` document from §2.2. Raw document writes
|
|
316
|
+
are trusted and unchecked. Root creation uses the internal reserved-ID
|
|
317
|
+
bootstrap in one commit together with the default configuration and `init`; do
|
|
318
|
+
not expose a temporary root-creation API.
|
|
319
|
+
|
|
320
|
+
Open ignores live tasks in this milestone; Package 14 adds reconciliation.
|
|
321
|
+
|
|
322
|
+
Acceptance: open persistent storage, obtain the root, mutate configuration and
|
|
323
|
+
history through public handles, create and fork conversations, close, reopen,
|
|
324
|
+
and verify stable root/conversation identity and state. Test lazy root creation
|
|
325
|
+
with `init` run once; atomic create/fork with `init`; actual forks; deep ancestor
|
|
326
|
+
caps; same-commit entry prefixes; cursor boundaries; self-head resolution;
|
|
327
|
+
newest-edit wins; raw head-to-tail transcript versus model context; model-less
|
|
328
|
+
and excluded assistant entries; replacements/omissions; multiple heads;
|
|
329
|
+
positional system messages; tool-result ordering; missing post-fork results;
|
|
330
|
+
every configuration getter/setter; `setActiveTools` duplicate and newly-added
|
|
331
|
+
unregistered rejection with stale names allowed; default active tools from the
|
|
332
|
+
registry; as-of configuration inheritance including unavailable names; registry
|
|
333
|
+
add/dispose/batch rollback/nested-batch rejection/keyed order/wrap composition;
|
|
334
|
+
and reopen.
|
|
335
|
+
|
|
336
|
+
## 14. Durable task runtime
|
|
337
|
+
|
|
338
|
+
Implement `defineTask`, exhaustive phase maps, full checkpoint replacement,
|
|
339
|
+
kind migration at reservation, runtime commits and memos, invocation lifetime
|
|
340
|
+
gates, scheduler reservation, dependencies, terminal outcomes, typed waits,
|
|
341
|
+
and joins. Complete the task-facing §2.2 methods: `resume`, `getTask`,
|
|
342
|
+
`waitForTask`, `abortTask`, and the task-aware portion of idle waits. Task definitions come from the registry snapshot;
|
|
343
|
+
add `RegistryReader.subscribe()` so registry changes wake the scheduler.
|
|
344
|
+
|
|
345
|
+
Include the execution-critical abort core: durable direct-task marks,
|
|
346
|
+
signal-and-join of an active run, fresh abort invocation, run-commit rejection
|
|
347
|
+
after a mark, and close precedence. Deep owned-subtree cascading and background
|
|
348
|
+
boundaries remain Package 18. Open now reconciles every surviving `running` task
|
|
349
|
+
to `pending`. Missing, too-old, and unmigratable definitions leave the task pending and
|
|
350
|
+
blocked (§5.4); registry changes wake the scheduler to reconsider them. Aborting
|
|
351
|
+
a blocked task settles it as `orphaned` with the full cleanup; nothing else
|
|
352
|
+
orphans a task. Implement per-phase registry snapshots, refresh at
|
|
353
|
+
every normal phase boundary, and hand over when the task definition object changed
|
|
354
|
+
and the new definition can reserve the task.
|
|
355
|
+
|
|
356
|
+
Deferred to later packages; do not stub them in Package 14:
|
|
357
|
+
- Orphaning writes the terminal `orphaned` record and retires task documents.
|
|
358
|
+
Unanswered input submissions and clearing run control are added in Package
|
|
359
|
+
15, when submissions and run control exist.
|
|
360
|
+
- `TaskRuntime.hooks` (the hook runner) is added in Package 16 with hook
|
|
361
|
+
dispatch. Package 14 adds `runtime.registry` (the phase's registry snapshot)
|
|
362
|
+
and `runtime.models`.
|
|
363
|
+
- `TaskRuntime.conversation()` returns a `ConversationHandle`, whose `submit`
|
|
364
|
+
needs Package 15 and whose `abort`/`waitForIdle` need ownership traversal; it
|
|
365
|
+
is added with the invocation-bound owned APIs in Package 18.
|
|
366
|
+
- Idle waits count live non-background tasks directly (Harness-wide or in the
|
|
367
|
+
addressed conversation). Traversal through owned conversations and background
|
|
368
|
+
boundaries replaces this in Package 18.
|
|
369
|
+
|
|
370
|
+
Use a fake two-phase external effect to test the real runtime. Acceptance is an
|
|
371
|
+
intent/effect/outcome task interrupted after intent, closed, reopened on the same
|
|
372
|
+
storage, and safely resumed to a durable terminal receipt. Also test unchanged-
|
|
373
|
+
checkpoint faulting, same-phase progress, cancellation precedence, thrown
|
|
374
|
+
handlers, dependencies, result values and entry IDs, first-writer-wins memos,
|
|
375
|
+
terminal checkpoint/memo removal, task-document retirement, close/reopen without
|
|
376
|
+
abort marks or fabricated outcomes, no fresh phase/abort dispatch while closing,
|
|
377
|
+
watch cleanup, blocked tasks unblocked
|
|
378
|
+
by later registration, migration failure leaving the record unchanged, handover
|
|
379
|
+
after a same-name task replacement, no handover to a missing or incompatible
|
|
380
|
+
definition, no overlap between predecessor and successor
|
|
381
|
+
invocations, abort of a blocked task settling as `orphaned` with full cleanup,
|
|
382
|
+
run commits rejected after the mark, and crashes at every direct-task abort stage.
|
|
383
|
+
|
|
384
|
+
Implemented design notes: every task transition is decided by one callback
|
|
385
|
+
serialized on the Session line (reservation, abort marks, runtime commits, and the
|
|
386
|
+
step before each phase). `Tx` has no task replacement; `runtime.commit()`
|
|
387
|
+
callbacks return the typed next state. `suspend()`, `hold()`, `quiescent()`, and
|
|
388
|
+
a public `markTask()` were dropped from §2.2.
|
|
389
|
+
|
|
390
|
+
## 15. First runnable no-tool chat turn
|
|
391
|
+
|
|
392
|
+
Implement the smallest real input-to-answer vertical path. Extend Package 14's
|
|
393
|
+
`orphaned` settlement to mark affected input submissions unanswered and clear
|
|
394
|
+
matching run control in the same commit, through the Harness hook for
|
|
395
|
+
scheduler-written outcomes (§5.4); the scheduler learns nothing about turns or
|
|
396
|
+
task kinds. Implement the final `pi.live` document (§8.2) with `run` and
|
|
397
|
+
`generation`, the built-in entry kinds (§8.1), and the `pi.generation` task
|
|
398
|
+
(§8.3). The inbox document is defined in Package 17 with its first writer. Add
|
|
399
|
+
input `Submission` admission, request-ID deduplication, reacquisition and
|
|
400
|
+
waiting, idle placement, active-turn ownership, successful answer settlement,
|
|
401
|
+
and terminal failure cleanup. Expose the final `SubmissionDraft` union. Idle
|
|
402
|
+
write submissions append their entry and settle `done`. Until Package 17, every
|
|
403
|
+
submission to a busy conversation rejects with `ConversationBusy` regardless of
|
|
404
|
+
`whenBusy`, writing nothing. Busy steer/follow-up behavior, queued writes, and
|
|
405
|
+
reset remain Package 17.
|
|
406
|
+
|
|
407
|
+
Also implement the decided surface changes: typed entry tokens
|
|
408
|
+
(`defineEntry<D extends JsonValue>`, token-first `tx.appendEntry()` and
|
|
409
|
+
`tx.entry()` overloads); removal of `input` from `createConversation()`;
|
|
410
|
+
built-in tasks pre-registered by `createRegistry()` (undisposable, not
|
|
411
|
+
overridable, required by `Harness.open`); the `TaskRuntime` additions
|
|
412
|
+
`snapshot`/`snapshotAsOf`, `context()`, `now()`, and `report()`; and the
|
|
413
|
+
`streamOptions`/`retry` configuration fields with their getters and setters.
|
|
414
|
+
|
|
415
|
+
Implement the §7.4 system prompt: rendering registered sections with `tag` and
|
|
416
|
+
wrappers, and no-tool request preparation, including
|
|
417
|
+
exact persisted rendered strings, positional system baselines/deltas with empty
|
|
418
|
+
`content`, head-cut rebaselining with `ContextEdit` omissions, order-only
|
|
419
|
+
two-entry rewrites, and section failure handling. No tools are offered in this
|
|
420
|
+
package: preparation emits no tool declarations and passes `PromptInput.tools`
|
|
421
|
+
as `[]`. Until Package 16, a `toolUse` response settles as the answer.
|
|
422
|
+
Implement the ordinary generation phases needed for one response: preparation,
|
|
423
|
+
request intent, durable throttled partials, attempts/retry classification,
|
|
424
|
+
deferred handle polling/cancellation, assistant entry settlement, and input-
|
|
425
|
+
submission completion.
|
|
426
|
+
|
|
427
|
+
Call pi-ai only through its exported `Models` methods: `getModel()`,
|
|
428
|
+
`streamSimple()`, `fetchDeferred()`, and `cancelDeferred()`. The test double must
|
|
429
|
+
implement that same interface; production code gets no adapter. A missing configured model produces
|
|
430
|
+
the durable `no_model` failure. All progress exposed to observers is committed
|
|
431
|
+
state, never raw provider frames.
|
|
432
|
+
|
|
433
|
+
Acceptance: `Harness.open → root → resume → submit(input) → Submission.wait →
|
|
434
|
+
durable assistant answer → close/reopen`, using faux Models in tests. Exercise
|
|
435
|
+
interruption and reopen before/after every implemented generation phase,
|
|
436
|
+
aborted-partial conversion, deferred polling/cancellation, retryable and
|
|
437
|
+
terminal model errors, no-visible-undurable updates, exact section order/value
|
|
438
|
+
patches, complete post-head baselines, retained system deltas on both sides of a
|
|
439
|
+
head marker, sections reading conversation documents through `input.read`,
|
|
440
|
+
throwing sections, preparation
|
|
441
|
+
rerun after a concurrent tail/config commit, fault and orphan turn cleanup,
|
|
442
|
+
typed entry tokens, and durable submission settlement. This is also the first print-mode smoke path: print
|
|
443
|
+
awaits its own input submission rather than global idle.
|
|
444
|
+
|
|
445
|
+
## 16. First coding-agent tool turn
|
|
446
|
+
|
|
447
|
+
Implement hook dispatch and `TaskRuntime.hooks`, deferred from Package 14
|
|
448
|
+
(Session-wide and scoped to a conversation or its owned subtree, §7.2), and wire
|
|
449
|
+
the real generation → tool tasks → post-tools → generation chain (§8.3–8.5).
|
|
450
|
+
Neither side uses a production fake successor.
|
|
451
|
+
|
|
452
|
+
- Generation: tool declarations in preparation (§7.4: additions/removals,
|
|
453
|
+
same-name replacement, order rewrite, complete baseline after a head cut,
|
|
454
|
+
duplicate active names offered once, wrapped and failing wrappers,
|
|
455
|
+
`PromptInput.tools`); `toolExecution` pinned in `request`/`poll`; the tool
|
|
456
|
+
round commit (offered check against the committed context through `cutoff`,
|
|
457
|
+
unaffected by `beforeRequest`; `tool_unavailable`
|
|
458
|
+
results for calls not offered, parallel or `after`-chained sequential tool
|
|
459
|
+
tasks, post-tools, `pi.live.tools`, run handed to post-tools);
|
|
460
|
+
`beforeRequest`, `afterResponse`, and `onYield` continuations.
|
|
461
|
+
- `pi.tool` (§8.4): input `{ assistant, callId }`, resolve/validate/
|
|
462
|
+
`beforeTool`/validate/intent/execute/`afterTool`/result in one `call` handler,
|
|
463
|
+
recovery-only `execute` with the stored/current replay rule, bounded output and
|
|
464
|
+
details in the `pi.live.tools` slot, the terminal commit as final flush,
|
|
465
|
+
diagnostics (`api.diagnostic()`, result diagnostics, Harness truncation and
|
|
466
|
+
error codes, the `<harness>` block, `pi.tool-result` `data: { diagnostics }`),
|
|
467
|
+
and the abort handler.
|
|
468
|
+
- `pi.post-tools` (§8.5): `afterTools`, `addTools`, all-results `terminate`, the
|
|
469
|
+
`postTools`/`final` boundaries (no inbox placement until Package 17), and the
|
|
470
|
+
next generation. The `handoff` control moves to Package 17 with `reset()`,
|
|
471
|
+
which defines the headed entry both write.
|
|
472
|
+
- `pi.live.tools` and the checkpoint rule (§8.2); the scheduler cleanup hook for
|
|
473
|
+
`pi.post-tools` (ends the run) and `pi.tool` (marks the slot `done`).
|
|
474
|
+
- Surface: `ToolExecutionApi` without `conversation()`, which the `src` type
|
|
475
|
+
drops until Package 18 adds it with `ConversationHandle`; `api.env` and `HarnessOptions.env`,
|
|
476
|
+
`ToolRegistration.executionMode`, the `toolExecution` configuration field with
|
|
477
|
+
its getter/setter, `TaskRuntime.getTask()` and `entry()`, `HookApi`, and the
|
|
478
|
+
exported `GenerationTask`, `ToolTask`, and `PostToolsTask` tokens.
|
|
479
|
+
- `@earendil-works/pi-durable/tools`: copy `read`, `bash`, `edit`, and `write`
|
|
480
|
+
with their helpers from `packages/agent/src/harness/tools` and adapt them to
|
|
481
|
+
`ToolRegistration` (`api.env`, `api.output`, `api.details`). Image reading in
|
|
482
|
+
`read` is deferred; note it where the tool rejects or skips images. The env
|
|
483
|
+
shell streams raw output (`onOutput`) and spills past thresholds; `output()` is
|
|
484
|
+
the only place output is bounded, sanitized, and throttled. `prepareArguments`
|
|
485
|
+
restores the edit tool's argument repair.
|
|
486
|
+
|
|
487
|
+
Acceptance: input → model tool call → registered local read/bash/edit operation →
|
|
488
|
+
tool result → model answer → durable submission settlement. Run that path once
|
|
489
|
+
normally and once interrupted/reopened. Test recovery from every tool and post-
|
|
490
|
+
tools phase; offered-set enforcement, including a tool deactivated after
|
|
491
|
+
preparation still executing; before/after hook rules, hook scopes, and hook
|
|
492
|
+
memos; `onYield` continuations; both stored/current replay-policy directions;
|
|
493
|
+
default and overridden `outputLimits`; output content fallback; details
|
|
494
|
+
replacement, last-details fallback, and coalesced commit settlement; the terminal
|
|
495
|
+
commit as final flush; abort/close with buffered output; invocation-bound
|
|
496
|
+
watches; parallel and sequential rounds, including a per-tool sequential mode;
|
|
497
|
+
unregistered active tools that are removed from the offered set, re-added after
|
|
498
|
+
re-registration, and produce `tool_unavailable` results when called, without
|
|
499
|
+
failing the request; a tool replaced mid-call finishing under the implementation
|
|
500
|
+
it resolved; hooks surviving a task reload; order-only tool changes; atomic
|
|
501
|
+
assistant/tool/post-tools commits; `terminate` only when every result requests
|
|
502
|
+
it; `addTools`; tool fault/orphan leaving a synthesized context result;
|
|
503
|
+
diagnostic ordering, the `<harness>` block, and entry `data`; and all
|
|
504
|
+
positional tool-history cases. Also test the ported tools against
|
|
505
|
+
`NodeExecutionEnv`, and a wrapper supplying a different `api.env`.
|
|
506
|
+
|
|
507
|
+
Document in the durable README that the package root loads TypeBox through the
|
|
508
|
+
tool task's argument validation (pi-ai `validateToolArguments()`): about 23 MB of
|
|
509
|
+
peak RSS unbundled, about 4 MB in a tree-shaken bundle.
|
|
510
|
+
|
|
511
|
+
Tool output benchmark (`test/*.bench.ts`, memory, SQLite, and JSONL): drive the
|
|
512
|
+
real tool task, adaptive throttle, and `pi.live` commits with low (a line every
|
|
513
|
+
few seconds), normal (a compiler or test run), and high (continuous `cat` of a
|
|
514
|
+
large file) output rates, head and tail retention, one tool and several parallel
|
|
515
|
+
tools, over one long round. Report commits, operation bytes written, stored
|
|
516
|
+
size before and after the round's base, reclamation, heap and RSS, commit
|
|
517
|
+
latency, and reopen/replay time mid-round. Assert that head output commits as
|
|
518
|
+
Chord appends and that a sliding tail of non-repetitive output within 64 KiB
|
|
519
|
+
commits as trim plus append; report how often repetitive output falls back to a
|
|
520
|
+
full window set. The checkpoint rule stays "base whenever nothing runs" (§8.2) with
|
|
521
|
+
no delta-count bound; the benchmark decides whether that holds. Assign
|
|
522
|
+
`slot.output` as one string field so Chord can diff it; replacing the slot
|
|
523
|
+
object records a full set.
|
|
524
|
+
|
|
525
|
+
Throughput target: a plain `cat` of a 1 GiB file of unique lines through the
|
|
526
|
+
ported bash tool and the real Harness path (env capture and spill, `api.output`,
|
|
527
|
+
throttled `pi.live` commits, result entry) completes in about 0.4 s, like the
|
|
528
|
+
mini coding agent. Also drive `api.output()` directly with the same 1 GiB in
|
|
529
|
+
64 KiB chunks: accepting a chunk must not cost work proportional to the
|
|
530
|
+
retained window.
|
|
531
|
+
|
|
532
|
+
## 17. Live UI and product state
|
|
533
|
+
|
|
534
|
+
Complete submissions and the inbox (§6): the `pi.inbox` document created by the
|
|
535
|
+
built-in setup; busy `steer`/`followUp`/`reject`; queued writes; admission to an
|
|
536
|
+
idle conversation with a non-empty inbox (queue, then a final boundary in the
|
|
537
|
+
same commit); withdrawal removing the item; `tx.placeSubmission()`; ordered
|
|
538
|
+
`postTools`/`final` selection by the new `steeringMode`/`followUpMode`
|
|
539
|
+
configuration fields (with getters/setters), writes before user items; stale
|
|
540
|
+
head writes; a reset at `postTools` ending the run with `reset`; successor runs;
|
|
541
|
+
and `onYield` applying only when the final boundary selected no user item and no
|
|
542
|
+
reset. Only an answer, `terminate`, or `handoff` applies the final boundary;
|
|
543
|
+
failure, a run task's abort, fault, and orphan leave the inbox alone.
|
|
544
|
+
Successful inputs still require an answer; writes settle on placement and never
|
|
545
|
+
start generation by themselves.
|
|
546
|
+
|
|
547
|
+
Define the `pi.reset` entry (§8.1), written by `Conversation.reset(handoff)`
|
|
548
|
+
through a write submission and by the post-tools `handoff` control.
|
|
549
|
+
|
|
550
|
+
Usage (§8.6): the `pi.usage` document created by the built-in setup, updated in
|
|
551
|
+
the same commit as every assistant entry (`appendAssistant()`) and every tool
|
|
552
|
+
result with `usage` (`ToolExecutionResult.usage`, `appendToolResult()`), and
|
|
553
|
+
`Harness.usage()` summing every conversation's document.
|
|
554
|
+
|
|
555
|
+
`ConversationView`, `viewState()`, and `watch()` (§9.3): one mount per
|
|
556
|
+
conversation, built lazily on the Session line, advanced from
|
|
557
|
+
`subscribeCommits()` publications, and dropped with its last observer.
|
|
558
|
+
|
|
559
|
+
Last: the experimental `watchEvents()` adapter (§9.4) with the snapshot event,
|
|
560
|
+
translated message and tool deltas, and overflow-to-snapshot.
|
|
561
|
+
|
|
562
|
+
Examples next to the existing ones in `test/examples/`: a print demo awaiting
|
|
563
|
+
its own `Submission`, a JSON demo with two modes, `--events` (adapter
|
|
564
|
+
events as JSON lines) and `--ops` (raw `ConversationView` frames), an inbox
|
|
565
|
+
demo, and a late-join demo attaching a view and an event stream mid-run. An
|
|
566
|
+
interactive TUI demo and coding-agent integration are later work.
|
|
567
|
+
|
|
568
|
+
Table-test every submission transition, cross-type request-ID conflicts,
|
|
569
|
+
interleaved queue selection under both queue modes, writes placed before user
|
|
570
|
+
items, stale head writes, a reset at each boundary, handoff, `onYield` with and
|
|
571
|
+
without queued items, compact positional removal of large payloads, withdrawal,
|
|
572
|
+
reopen waits, queued items surviving a failed run and drained by the next
|
|
573
|
+
submission, and orphan/fault cleanup. Test minimal Chord deltas for `pi.inbox`
|
|
574
|
+
and `pi.usage` changes. Test one view revision per touching commit; atomic
|
|
575
|
+
entry/document publication; fork-aware active entries and head cuts; mounted
|
|
576
|
+
create/retire; empty-batch suppression; stable paths; structural sharing; watch
|
|
577
|
+
overflow; and dropping the mount. Test the event translation of every
|
|
578
|
+
`MessageChange` and output change, the snapshot at attachment and after
|
|
579
|
+
overflow, and absence of raw provider frames.
|
|
580
|
+
|
|
581
|
+
## 18. Ownership and subagents
|
|
582
|
+
|
|
583
|
+
Complete the remaining owned-conversation and abort semantics: durable foreground
|
|
584
|
+
subtree cascades, signal/join/fresh-abort across ownership edges, background
|
|
585
|
+
boundaries, ordinary and full traversal, conversation abort, and exact idle
|
|
586
|
+
waits. Finish invocation-bound owned APIs used by tools and the foreground and
|
|
587
|
+
background subagent provisioning patterns, including atomic task/conversation/
|
|
588
|
+
registry creation and request-ID-safe submission recovery.
|
|
589
|
+
|
|
590
|
+
An abort cascade withdraws queued inputs in every owned conversation it reaches
|
|
591
|
+
(spec §5.4). There is no full-teardown primitive: `close()` stops everything
|
|
592
|
+
without outcomes, and a host cancels everything by aborting what `inspect()`
|
|
593
|
+
lists. No built-in subagent tool or supervisor task: two concise, product-style
|
|
594
|
+
examples, `test/examples/22-subagent-foreground.ts` (child owned by the tool task,
|
|
595
|
+
reported through `api.details({ conversationId })`, the UI attaching to the
|
|
596
|
+
child's events) and `test/examples/23-subagent-background.ts` (reworked in
|
|
597
|
+
Package 19: persistent subagents behind one `subagent` tool with spawn, send,
|
|
598
|
+
wait, stop, and status actions; a background anchor task owns each child, and a
|
|
599
|
+
background reporter task per message delivers it and posts the answer back as a
|
|
600
|
+
follow-up input, request-ID-safe across restarts), show the patterns.
|
|
601
|
+
|
|
602
|
+
Test deep ownership trees, owner edges after terminal settlement, nested
|
|
603
|
+
background boundaries, conversation abort/join with surviving passive writes and
|
|
604
|
+
background tasks, cancellation of waiters without cancellation of work, and
|
|
605
|
+
atomic cancellation intent. Test default non-inheritance, inheritance from the
|
|
606
|
+
current committed tail, empty source conversations, document fork policies,
|
|
607
|
+
configuration overrides through `init`, foreground subagent cascade, and
|
|
608
|
+
background supervisor recovery before and after submission admission.
|
|
609
|
+
|
|
610
|
+
## 19. Structured concurrency
|
|
611
|
+
|
|
612
|
+
Implement spec §5.5 and its consequences, replacing `after` and `pi.post-tools`.
|
|
613
|
+
No backward compatibility.
|
|
614
|
+
|
|
615
|
+
- Ownership: `TaskOptions.ownership` is required (`{ kind: "conversation" }` or
|
|
616
|
+
`{ kind: "task", taskId }`); `TaskRecord.owner?: TaskId`; child tasks live in
|
|
617
|
+
their owner's conversation; `background` only for conversation-owned tasks;
|
|
618
|
+
new task- or conversation-ownership requires a live owner (not `completing`
|
|
619
|
+
or `terminal`). Update every `createTask` call site, including tools'
|
|
620
|
+
`api.createTask`, examples, and tests. Storage: persist `owner` and the new
|
|
621
|
+
statuses (memory, JSONL, SQLite schema edited in place; no migrations while WIP), conformance cases. No owner
|
|
622
|
+
index: the scheduler derives edges from the live tasks it loads.
|
|
623
|
+
- States: real `waiting { checkpoint, on, policy }` and `completing { outcome }`
|
|
624
|
+
statuses in `TaskState` and every backend and status scan; `NextTaskState`
|
|
625
|
+
gains `waiting`; `runtime.outcomes()`; `inspect()` reports `completing`.
|
|
626
|
+
Remove `after` (records, options, scheduler dependency handling).
|
|
627
|
+
- Scheduler: a waiting task is never reserved until every task in `on` is
|
|
628
|
+
terminal (then reserved directly from `waiting` to `running` at its
|
|
629
|
+
checkpoint), unless an abort mark lets it reach its abort handler; `on` over
|
|
630
|
+
tasks it does not own requires `allSettled`; `on` rejects missing tasks, the
|
|
631
|
+
task itself, and its owner chain; failFast marks live owned siblings in `on` on a held or terminal
|
|
632
|
+
non-`completed` outcome, in reconcile. Terminal commits (task- and
|
|
633
|
+
scheduler-written) with live ordinary owned work become `completing`; a later
|
|
634
|
+
scheduler commit finalizes them when that work is gone, re-evaluated on every
|
|
635
|
+
commit and at open; scheduler-written holds defer their Harness cleanup
|
|
636
|
+
(`settleSchedulerOutcome`) to the final commit, task-written holds land their
|
|
637
|
+
other writes at hold; waiters and task-document retirement at the final
|
|
638
|
+
commit. Abort invocations start only when ordinary owned work is gone
|
|
639
|
+
(bottom-up, judged on committed records). Orphaning happens only where the
|
|
640
|
+
abort invocation would start, or directly in `abortTask()` when nothing owned
|
|
641
|
+
is live, so orphans never hold. Cascade (spec §5.4): only live owners cascade (abort mark or held
|
|
642
|
+
non-`completed` outcome); terminal owners never do; ordinary traversal and
|
|
643
|
+
idle follow task→task edges too. `Conversation.abort(context, { background:
|
|
644
|
+
true })` snapshot semantics.
|
|
645
|
+
- Built-ins: generation owns its tool tasks and waits `allSettled` in a new
|
|
646
|
+
`tools` phase that runs the old post-tools body (§8.5); sequential rounds
|
|
647
|
+
create one tool at a time from `pending`; the next generation is
|
|
648
|
+
conversation-owned; the generation abort handler appends `aborted` results for
|
|
649
|
+
unstarted calls. Remove `pi.post-tools`, `PostToolsTask`, `PostToolsHooks`
|
|
650
|
+
(`afterTools` moves to `GenerationHooks`). Tool tasks that own conversations
|
|
651
|
+
hold `completing` after their result entry. Events: `turn_end` when a
|
|
652
|
+
generation's outcome is committed (hold or terminal, whichever first).
|
|
653
|
+
- Docs: spec §5.5 and §12 footguns are written; update README (task ownership,
|
|
654
|
+
waiting, `completing`, abort order), CHANGELOG, and examples 12/22/23 plus a new
|
|
655
|
+
`test/examples/24-child-tasks.ts` (checkout with four payments: failFast,
|
|
656
|
+
`abortTask(checkout)`, crash and reopen, each printing outcomes).
|
|
657
|
+
|
|
658
|
+
Exhaustive tests (new `test/harness-structured.test.ts` plus updates): ownership
|
|
659
|
+
validation (missing ownership, cross-conversation child, background child,
|
|
660
|
+
owner completing/terminal); waiting with allSettled and failFast (failed,
|
|
661
|
+
aborted, faulted, orphaned children; held failures trigger failFast before the
|
|
662
|
+
failing child drains); `on` with already-terminal and non-owned tasks
|
|
663
|
+
(`allSettled` only); waiting on subsets in sequence; spawning without waiting;
|
|
664
|
+
`completing` for task- and scheduler-written outcomes, including work created
|
|
665
|
+
during the hold, finalization at open, `abortTask()` on a completing task, held
|
|
666
|
+
failure aborting below, waiters and document retirement only at the final
|
|
667
|
+
commit; bottom-up abort order across three levels (child terminal records before
|
|
668
|
+
the parent's abort handler starts), including an aborted waiting parent; `on`
|
|
669
|
+
validation (missing, self, owner chain, empty `on`); a waiting task whose
|
|
670
|
+
definition is missing when it may resume (blocked, then orphaned when aborted);
|
|
671
|
+
creating owned work in a finishing commit (task ownership rejects, a task in an
|
|
672
|
+
owned conversation holds); cascades through
|
|
673
|
+
task→task and task→conversation→task edges; background boundaries and the
|
|
674
|
+
`{ background: true }` abort; a terminal owner never cascading (interrogating a
|
|
675
|
+
finished or Esc'd subagent runs normally); crash/reopen at every point (before
|
|
676
|
+
and after create+wait, after a child fails before siblings are marked, after all
|
|
677
|
+
children terminal before the parent resumes, during holds and abort handlers);
|
|
678
|
+
generation tool rounds parallel and sequential with Esc before, during, and
|
|
679
|
+
after tools, unstarted sequential calls, faulted tool slots under generation
|
|
680
|
+
abort, faulted generation keeping run control until its tools drain, `turn_end`
|
|
681
|
+
before the successor's `turn_start` when a generation holds, and a tool holding
|
|
682
|
+
`completing` while its subagent's extension work runs. Rerun every existing suite; migrate the post-tools,
|
|
683
|
+
sequential-round, and dependency tests to the new shapes without weakening them.
|
|
684
|
+
|
|
685
|
+
## 20. Compaction and overflow
|
|
686
|
+
|
|
687
|
+
Implement spec §8.7 and its consequences. Compaction never appends to a busy
|
|
688
|
+
conversation concurrently (§7.4 preparation does not recheck the transcript): a
|
|
689
|
+
blocking compaction appends while its generation holds the run and waits for it;
|
|
690
|
+
every other compaction places its summary through a write submission.
|
|
691
|
+
|
|
692
|
+
- Task: `pi.compaction` (`CompactionTask`, `CompactionHooks`, `CompactionInput`,
|
|
693
|
+
`CompactionCheckpoint`, `CompactionResult`, `CompactionReason`) with phases
|
|
694
|
+
`select`, `summarize`, `retry`, the abort handler, and the `beforeCompact`
|
|
695
|
+
hook. Register it with the built-ins. `CompactionEntry` token for
|
|
696
|
+
`pi.compaction` with `data: { reason }`.
|
|
697
|
+
- `ContextView.contributions`: each active entry's contribution after all
|
|
698
|
+
in-range edits (including edits on older in-range markers). Range selection is one pure function over those
|
|
699
|
+
contributions and `keepRecentTokens`, shared by the generation's checks and
|
|
700
|
+
`select`; the summarized messages are the prefix contributions ordered by
|
|
701
|
+
§2.1 rules 7–8. The generation's size estimate (§8.3) uses the newest
|
|
702
|
+
assistant appended after the head marker, not pi-ai's timestamp heuristic. Port the coding agent's serializer, summarization
|
|
703
|
+
system prompt, and structured prompt (one prompt, with a line about carrying an
|
|
704
|
+
earlier summary forward) into `src/harness/compaction.ts`; do not import from
|
|
705
|
+
coding-agent.
|
|
706
|
+
- Placement: direct append for a task-owned (blocking) compaction; for a
|
|
707
|
+
conversation-owned one, factor the admission body of `Submissions.submit()` into
|
|
708
|
+
a function over a `Tx` so the task's classifying commit admits the write
|
|
709
|
+
(request ID `compaction:<taskId>`) with the same idle/queued/stale/final-boundary
|
|
710
|
+
behavior. No new stale logic.
|
|
711
|
+
- Config: `CompactionPolicy`, `DEFAULT_COMPACTION_POLICY`,
|
|
712
|
+
`ConversationConfigState.compaction`, `getCompaction`/`setCompaction`.
|
|
713
|
+
- Live: `pi.live.compactions` (`CompactionStatus[]`, task ID order, key removed
|
|
714
|
+
when empty), added in the creating commit and removed in the outcome commit
|
|
715
|
+
(also at a `completing` hold);
|
|
716
|
+
`settleSchedulerOutcome` removes it for faulted/orphaned compactions.
|
|
717
|
+
- Generation: `compacted`/`overflow` checkpoint fields; threshold checks in
|
|
718
|
+
`prepare` (blocking child + wait; background conversation-owned task in the
|
|
719
|
+
commit moving to `request`); overflow classification before the retry branch;
|
|
720
|
+
failing an overflow generation whose compaction produced no entry.
|
|
721
|
+
- `Conversation.compact()` admits the manual task (conversation-owned, not
|
|
722
|
+
background) with its status in one commit and returns its ID.
|
|
723
|
+
- Usage: every classified summarization response adds its usage to
|
|
724
|
+
`pi.usage.models`.
|
|
725
|
+
- Events: `compaction_start`/`compaction_end`, snapshot `compactions`, batch
|
|
726
|
+
order per §9.4.
|
|
727
|
+
- Docs: README (compaction, policy, events), CHANGELOG, and a new product-style
|
|
728
|
+
example `test/examples/25-compaction.ts` (faux model: a long chat that crosses
|
|
729
|
+
the background threshold, the summary lands at a boundary, a manual `compact()`
|
|
730
|
+
while busy, and an overflow recovered by a blocking compaction, printing the
|
|
731
|
+
model context before and after).
|
|
732
|
+
|
|
733
|
+
Not in v1 (note only): a cache-friendly summary request that resends the
|
|
734
|
+
agent's exact last request plus one summarization user message with pi-ai
|
|
735
|
+
`toolChoice: "none"`, so the prefix including tool declarations is a cache hit.
|
|
736
|
+
It fits background compaction well below the window; overflow and a context
|
|
737
|
+
without room for the prompt and summary keep the serialized request.
|
|
738
|
+
|
|
739
|
+
Exhaustive tests (new `test/harness-compaction.test.ts`, faux provider, plus
|
|
740
|
+
updates):
|
|
741
|
+
|
|
742
|
+
- Range selection (pure): cut at a user entry, at an assistant entry mid-run
|
|
743
|
+
(one prompt followed by many tool rounds), never at a tool result or system
|
|
744
|
+
entry; a huge last tool result keeps its assistant; budget never reached and
|
|
745
|
+
only-the-marker prefix are nothing to compact; an existing marker is
|
|
746
|
+
summarized first and older in-range markers drop out; omitted and replaced
|
|
747
|
+
entries use their edited contribution; excluded error/aborted assistants are
|
|
748
|
+
not candidates; the §8.7 worked example.
|
|
749
|
+
- Summarization request: exact messages (system prompt, serialized
|
|
750
|
+
conversation, instructions line), tool-result truncation, no tools,
|
|
751
|
+
`cacheRetention: "none"`, no `deferred`, `maxTokens` from `reserveTokens` and
|
|
752
|
+
the model's max, pinned model/thinking/stream options surviving a config change
|
|
753
|
+
mid-summary.
|
|
754
|
+
- Summary entry: kind, wrapped text, `head` = first kept, `data.reason`; model
|
|
755
|
+
context after placement is `[summary, kept...]` followed by a full system
|
|
756
|
+
baseline on the next preparation; raw history unchanged and still scannable via
|
|
757
|
+
`entries()`; `pi.usage.models` includes every attempt, including failed,
|
|
758
|
+
retried, and stale ones; declines and hook-supplied summaries add none.
|
|
759
|
+
- Outcomes: nothing to compact (`{}`), `beforeCompact` decline (`{}`) and
|
|
760
|
+
supplied summary (no model call), first decision wins, hook throw reported and
|
|
761
|
+
ignored; `no_model`; retryable error then success; retries exhausted;
|
|
762
|
+
non-retryable error; `length` stop; tool call in the response; empty text.
|
|
763
|
+
- Manual: idle with empty inbox (placed at once, `{ submissionId }` whose submission is `done`);
|
|
764
|
+
idle with queued follow-ups after a failed run (final boundary places the
|
|
765
|
+
summary first, then starts the follow-up in the compacted context); busy
|
|
766
|
+
(queued, placed at the next `postTools` and the run continues in the compacted
|
|
767
|
+
context; placed at `final`); stale because a reset landed while summarizing;
|
|
768
|
+
stale because a later compaction cut further; an older summary with a later cut
|
|
769
|
+
placed after a newer one; two queued summaries in one boundary with the
|
|
770
|
+
second cutting before, at, and after the first (only "before" is stale); `Conversation.abort()` aborts a manual compaction and keeps an
|
|
771
|
+
already queued summary; `Submission.abort()` withdraws a queued summary;
|
|
772
|
+
`waitForIdle` waits for a manual compaction; `compact()` returns the ID
|
|
773
|
+
`waitForTask` settles with.
|
|
774
|
+
- Background threshold: starts only above the background threshold, only with
|
|
775
|
+
`backgroundTokens > 0`, only when no status is listed, not when disabled, not
|
|
776
|
+
without a cut, and not after a blocking compaction in the same generation; the
|
|
777
|
+
generation does not wait; summary placed at the next boundary; survives
|
|
778
|
+
`Conversation.abort()`; `abort({ background: true })` and `abortTask()` stop
|
|
779
|
+
it; idle waits ignore it.
|
|
780
|
+
- Blocking threshold: generation waits, nothing appended before the wait, then
|
|
781
|
+
re-prepares with a baseline and sends; request proceeds after nothing to
|
|
782
|
+
compact, decline, failure, fault, and direct `abortTask()` of the child; no
|
|
783
|
+
second compaction when still above the threshold; a background compaction in
|
|
784
|
+
flight ends stale; Esc during the blocking compaction aborts child then
|
|
785
|
+
generation (child terminal before the generation's abort handler) and settles
|
|
786
|
+
inputs `aborted`.
|
|
787
|
+
- Overflow: error entry appended and excluded from the retry request, attempt
|
|
788
|
+
unchanged, compaction then retry succeeds; second overflow fails
|
|
789
|
+
`model_error` with its error entry; overflow with compaction disabled, without
|
|
790
|
+
a cut, or after a threshold compaction fails; overflow whose compaction
|
|
791
|
+
declines/fails/aborts fails `model_error` with the overflow text; an overflow
|
|
792
|
+
error matching retryable patterns is still not retried; silent `stop`/`length`
|
|
793
|
+
overflow is an ordinary answer.
|
|
794
|
+
- Live status and events: status added in the creating commit with `blocking`,
|
|
795
|
+
`attempt`, and `retry` during backoff, removed with every outcome
|
|
796
|
+
(completed, failed, aborted, faulted, orphaned); several concurrent statuses;
|
|
797
|
+
`compaction_start`/`compaction_end` in batch order; snapshot `compactions` for a late joiner during summarize and retry;
|
|
798
|
+
view mount after a compaction head (kept entries stay mounted).
|
|
799
|
+
- Recovery: crash and reopen in `select` (hook reruns), `summarize` (request
|
|
800
|
+
resent once, no duplicate usage), `retry`, after placement, with the
|
|
801
|
+
generation waiting on a blocking compaction, with a summary queued, and with a
|
|
802
|
+
blocked compaction definition (orphaned on abort, status removed).
|
|
803
|
+
- Estimate and cut edge cases: a background summary queued mid-run and placed
|
|
804
|
+
at `postTools` makes the successor's estimate ignore the pre-compaction usage,
|
|
805
|
+
so no blocking compaction starts (also with a fixed Harness clock); an
|
|
806
|
+
assistant finishing while a summary is queued; raw order `assistant(call),
|
|
807
|
+
user, toolResult, assistant` never cuts at that user; edits carried by an
|
|
808
|
+
older in-range marker and a kept entry replacing a summarized one are reflected
|
|
809
|
+
in the summarized messages and the hook's `messages`; kept `pi.system` deltas
|
|
810
|
+
get omit edits in the next single baseline; compaction in a fork whose cut
|
|
811
|
+
falls on a parent entry (head, stale checks, view mount).
|
|
812
|
+
- Interactions: a summary queued before a reset in the same boundary (both
|
|
813
|
+
placed, reset wins) and after it (stale); a manual compaction's status blocks
|
|
814
|
+
a background start; a queued summary surviving a failed run is placed by the
|
|
815
|
+
next submission's final boundary before the new input; the summarization
|
|
816
|
+
request itself overflowing fails `model_error` and a waiting overflow
|
|
817
|
+
generation fails with the original overflow text; a retryable error after an
|
|
818
|
+
overflow compaction has the full retry budget; a background summary landing
|
|
819
|
+
during a retry backoff stays queued while a blocking compaction wins; an
|
|
820
|
+
application edit placed while summarizing is lost as §12 documents; a hook
|
|
821
|
+
that creates owned work and supplies a summary (placement and status removal
|
|
822
|
+
at hold, receipt after the child); `compact()` right after open enables
|
|
823
|
+
scheduling; a rejected classifying commit leaves no usage, submission,
|
|
824
|
+
summary, or outcome; idle admission that drains an older queued summary
|
|
825
|
+
together with the current one.
|
|
826
|
+
- Rerun every existing suite; generation, inbox, events, and view tests keep
|
|
827
|
+
passing unchanged except for the new optional fields.
|
|
828
|
+
|
|
829
|
+
## 21. Extensions and per-conversation agents
|
|
830
|
+
|
|
831
|
+
Implement the design agreed with Mario, now specified in `spec.md` (§2.2,
|
|
832
|
+
§5.1/§5.4 runtime surface, §6 queue modes, §7.1–§7.5, §8 where tasks read the
|
|
833
|
+
agent and settings, §9.3/§9.4 view mount and events, §12 footguns): extensions
|
|
834
|
+
installed by name replace keys, positions, `batch()`, wrappers-by-key, hook
|
|
835
|
+
scopes, and public conversation setups; the rewindable `pi.agent` document
|
|
836
|
+
(model, thinking level, extension and tool selection, `instructions`, `cwd`)
|
|
837
|
+
replaces `pi.conversation.config` and its getters and setters; run policies
|
|
838
|
+
become Harness-wide live `HarnessSettings`; the environment is built per call
|
|
839
|
+
by one host `env` function. The `Tool` generic stays. `CodingTools` bundles
|
|
840
|
+
read, write, edit, and bash; grep, find, and ls are not ported.
|
|
841
|
+
|
|
842
|
+
Implement in parts, each with tests: registry and extensions (install/replace,
|
|
843
|
+
uninstall, catalogue views, task-name collisions, wrapper failures); `pi.agent`,
|
|
844
|
+
`configure()`, and resolution (selection arrays and `add`/`remove`, same-name
|
|
845
|
+
replacement, wrappers, uninstalled names, `addTools`); settings resolution and
|
|
846
|
+
every reader (generation request options and retry, compaction thresholds, tool
|
|
847
|
+
execution, queue modes on the line); `env` for tools, `prepare`, and
|
|
848
|
+
`runtime.env()`; hooks resolved from the conversation's selected extensions per
|
|
849
|
+
phase; reload of an extension while work runs (running phases keep their
|
|
850
|
+
snapshot, pinned tools finish under old code, task definitions hand over at the
|
|
851
|
+
next phase boundary as in §5.4); view and events. Rewrite the examples,
|
|
852
|
+
including 22 and 23, to the new surface.
|
|
853
|
+
|
|
854
|
+
## 22. Lifecycle and final conformance
|
|
855
|
+
|
|
856
|
+
Package 21 of the previous plan was audited (three reviewers); these findings
|
|
857
|
+
remain and apply to the new surface.
|
|
858
|
+
|
|
859
|
+
Bugs:
|
|
860
|
+
- `runtime.now()` and `runtime.report()` stay usable after the invocation ends
|
|
861
|
+
(`scheduler.ts` runtime); §5.4 says every runtime operation rejects.
|
|
862
|
+
- A commit that only migrates a document through a newer token (version base, no
|
|
863
|
+
ops) is not published (`transaction.ts` `publishes()`), so observers of the
|
|
864
|
+
older shape keep a stale value until the next edit.
|
|
865
|
+
- A failed `Harness.open` closes with the caller's possibly cancelled context,
|
|
866
|
+
masking the original error; close without the caller's signal and rethrow.
|
|
867
|
+
- `ConversationWatch` alias missing; stale TODO at `types.ts` about
|
|
868
|
+
`subscribeCommits`; `resume()` doc comment omits progress calls.
|
|
869
|
+
|
|
870
|
+
Spec wording to settle with Mario first: close stops state and watch delivery at
|
|
871
|
+
seal (§2.2 says states close after the join); cancelling `close()` cancels only
|
|
872
|
+
that wait and a second `close()` awaits the same shutdown; add
|
|
873
|
+
`Conversation.abort()` to the progress calls; "old and new Harness generations"
|
|
874
|
+
means that after `await close()` no invocation code of the old Harness runs;
|
|
875
|
+
"service withdrawal and client detach" is proposed as a runnable Chord guide
|
|
876
|
+
test (facet host, in-process remote binding, host dispose before Harness close);
|
|
877
|
+
"normative usage sequences" are the spec's example code blocks, copied into one
|
|
878
|
+
compile-checked test.
|
|
879
|
+
|
|
880
|
+
Tests to add or strengthen: a committer cancelled while its commit is in
|
|
881
|
+
Storage still settles durably; a cancelled `close()` still completes shutdown;
|
|
882
|
+
a task handler, tool `execute`, and hook that ignore the signal keep `close()`
|
|
883
|
+
pending and Storage open until they return (the current tests pass with
|
|
884
|
+
`join()` deleted); a state receives no frame from a commit settling during
|
|
885
|
+
close; watch acquisition cancelled mid-line leaves no subscription; conversation
|
|
886
|
+
handle methods and `inspect()` (`scheduling: "closing"`) around close; each
|
|
887
|
+
progress call enables scheduling on a paused Harness and each read-only viewer
|
|
888
|
+
does not; registry change between open and resume; handover keeps memos;
|
|
889
|
+
compile tests for every §3.3/§9.1 overload (including `snapshotAsOf` rejecting a
|
|
890
|
+
non-rewindable token, token overloads of `tx.entry`/`appendEntry`), the
|
|
891
|
+
`SubmissionDraft` never-fields, `waitForTask<R>`, `compact()` result type, and
|
|
892
|
+
an erased extension/task with narrowed input, several phases, and custom hooks;
|
|
893
|
+
the Chord guide compiles (it uses a bare `Id` today) and runs.
|
|
894
|
+
|
|
895
|
+
Already covered, rerun only: root-replacement delta versus checkpoint, blocked
|
|
896
|
+
tasks surviving open, root identity across reopen, handover cases, a reopened
|
|
897
|
+
interrupted turn. Run all package tests and the repository check, finish with a
|
|
898
|
+
local coding-agent turn and a reopened interrupted turn through the public
|
|
899
|
+
Harness, then stop for final review.
|
|
900
|
+
|
|
901
|
+
Done. The spec wording is settled as proposed (§2.2 close and progress calls,
|
|
902
|
+
§5.1, §7.5, §9.1, §12). Close ends states and watches at the seal; a new Harness
|
|
903
|
+
may open the Storage once the old `close()` resolved. The lifecycle tests are in
|
|
904
|
+
`harness-lifecycle.test.ts`, the Chord guide runs as `chord-guide.test.ts`, and
|
|
905
|
+
the spec's usage examples compile in `spec-usage.test.ts`.
|
|
906
|
+
|
|
907
|
+
## 23. Task graph view
|
|
908
|
+
|
|
909
|
+
A live, observable view of the Session's task graph for UIs and debugging, like
|
|
910
|
+
`ConversationView` for a conversation: every live task with its owner edge
|
|
911
|
+
(conversation or task), status (`running`, `waiting on …`, `completing`),
|
|
912
|
+
background flag, and owned conversations, published after each commit as a
|
|
913
|
+
replicated state or watch. `inspect()` already provides a one-off snapshot of
|
|
914
|
+
the live task records. Specify and build it after the final conformance package.
|
|
915
|
+
|
|
916
|
+
Specified in `spec.md` §9.5: `Harness.taskGraph()` and `watchTaskGraph()` mount
|
|
917
|
+
every live task keyed by ID with its committed status (phase, wait, held outcome
|
|
918
|
+
status), owner edge, flags, and owned conversations. Derived states (`blocked`,
|
|
919
|
+
`ready`, the live part of `on`) stay in `inspect()`. Implemented in
|
|
920
|
+
`src/harness/task-graph.ts`, tested in `harness-task-graph.test.ts` and the
|
|
921
|
+
lifecycle tests, shown in example 24.
|