@zq-silk/yui 0.16.2 → 1.0.0-alpha

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (111) hide show
  1. package/README.md +6 -4
  2. package/dist/agent/agent.js +4 -9
  3. package/dist/agent/executionComponents.js +4 -4
  4. package/dist/agentRun/agentRun.js +8 -8
  5. package/dist/artifacts/managedGit.js +3 -56
  6. package/dist/brief/taskBrief.js +3 -3
  7. package/dist/cli/updateOrchestrator.js +2 -13
  8. package/dist/cli/updatePorts.js +36 -69
  9. package/dist/cli/upgradeCommand.js +4 -8
  10. package/dist/cli.js +4 -7
  11. package/dist/commands/controllerCommands.js +1 -1
  12. package/dist/commands/taskCommands.js +8 -1
  13. package/dist/commands/taskRoleRuntimeStatus.js +1 -1
  14. package/dist/context/runInputContract.js +1 -1
  15. package/dist/controller/agentRuntimeObserver.js +4 -4
  16. package/dist/controller/clientRuntime.js +19 -44
  17. package/dist/controller/controller.js +16 -4
  18. package/dist/controller/fileSchedulerStoreAdapter.js +6 -6
  19. package/dist/controller/globalInputDelivery.js +3 -1
  20. package/dist/controller/jobSupervisor.js +3 -3
  21. package/dist/controller/providerRetryDelivery.js +128 -118
  22. package/dist/controller/runtime.js +1 -1
  23. package/dist/controller/structuredProviderObservation.js +1 -1
  24. package/dist/coordination/workMailbox.js +4 -4
  25. package/dist/core/controllerIdentity.js +25 -0
  26. package/dist/core/controllerProcessIdentity.js +2 -2
  27. package/dist/core/controllerServer.js +4 -2
  28. package/dist/core/protocol.js +1 -1
  29. package/dist/domain/validation.js +6 -0
  30. package/dist/event/taskEvent.js +3 -3
  31. package/dist/execution/workItemExecution.js +2 -2
  32. package/dist/executor/agentExecutor.js +6 -6
  33. package/dist/executor/effectiveLaunch.js +3 -3
  34. package/dist/grant/capabilityGrant.js +2 -2
  35. package/dist/input/inputRequest.js +4 -4
  36. package/dist/integration/changeSet.js +3 -3
  37. package/dist/integration/integrationAttempt.js +3 -3
  38. package/dist/integration/integrationSourceApplication.js +1 -1
  39. package/dist/job/durableJob.js +1 -1
  40. package/dist/job/jobRunner.js +2 -2
  41. package/dist/message/message.js +8 -6
  42. package/dist/milestone/milestone.js +3 -3
  43. package/dist/profile/agentProfile.js +3 -3
  44. package/dist/release/runtimeRelease.js +9 -7
  45. package/dist/repository/project.js +3 -3
  46. package/dist/resources/liveReferences.js +1 -1
  47. package/dist/resources/sqliteResourceRegistry.js +2 -2
  48. package/dist/review/reviewRound.js +5 -5
  49. package/dist/role/role.js +7 -8
  50. package/dist/runtime/acpProtocol.js +2 -3
  51. package/dist/runtime/agentDriverObservation.js +1 -1
  52. package/dist/runtime/agentHost.js +3 -3
  53. package/dist/runtime/agentHostProtocol.js +2 -2
  54. package/dist/runtime/codexInteractiveHost.js +1 -1
  55. package/dist/runtime/launchBroker.js +1 -1
  56. package/dist/runtime/processExitObservation.js +1 -1
  57. package/dist/runtime/providerContinuationReconciliationService.js +84 -75
  58. package/dist/runtime/providerRuntimeIdentity.js +3 -3
  59. package/dist/runtime/runtimeCoherence.js +7 -3
  60. package/dist/runtime/runtimeObservation.js +3 -3
  61. package/dist/runtime/sessionOwnerIdentity.js +1 -1
  62. package/dist/runtime/taskRuntimeIsolation.js +3 -3
  63. package/dist/runtime/tmuxAdapters.js +3 -3
  64. package/dist/scheduler/taskWake.js +1 -1
  65. package/dist/storage/baselineSchema.js +606 -0
  66. package/dist/storage/homeLayout.js +5 -16
  67. package/dist/storage/recordValidation.js +16 -4
  68. package/dist/storage/sqliteSchema.js +106 -1741
  69. package/dist/storage/sqliteStore.js +16 -14
  70. package/dist/storage/storageSchema.js +8 -7
  71. package/dist/storage/storageVersions.js +23 -16
  72. package/dist/storage/taskStore.js +3 -29
  73. package/dist/storage/upgrade/upgradeOrchestrator.js +14 -102
  74. package/dist/task/task.js +15 -7
  75. package/dist/task/taskActivation.js +5 -4
  76. package/dist/telemetry/sqliteTelemetryStore.js +2 -2
  77. package/dist/verification/gateArtifact.js +5 -3
  78. package/dist/verification/verificationPlan.js +6 -7
  79. package/dist/workItem/workItem.js +6 -6
  80. package/dist/workspace/cleanupInspection.js +1 -9
  81. package/dist/worktree/managedWorkspace.js +3 -3
  82. package/docs/managed-turn-and-session-runtime.md +26 -14
  83. package/docs/managed-turn-and-session-runtime.zh-CN.md +18 -10
  84. package/docs/release-workflow.md +52 -300
  85. package/docs/release-workflow.zh-CN.md +41 -234
  86. package/docs/sqlite-control-plane-design.md +48 -289
  87. package/docs/sqlite-control-plane-design.zh-CN.md +37 -53
  88. package/docs/storage-baseline.md +132 -0
  89. package/docs/storage-baseline.zh-CN.md +106 -0
  90. package/docs/task-delivery.md +3 -4
  91. package/docs/task-delivery.zh-CN.md +3 -3
  92. package/docs/testing/verification-levels.md +40 -180
  93. package/docs/testing/verification-levels.zh-CN.md +27 -131
  94. package/i18n/README.zh-CN.md +5 -4
  95. package/package.json +1 -1
  96. package/dist/storage/migrations/agentFailureContext.js +0 -22
  97. package/dist/storage/migrations/agentRunContract.js +0 -159
  98. package/dist/storage/migrations/artifactsToGit.js +0 -338
  99. package/dist/storage/migrations/collapseWorktreeLayout.js +0 -963
  100. package/dist/storage/migrations/currentInputContract.js +0 -86
  101. package/dist/storage/migrations/currentRuntimeContract.js +0 -228
  102. package/dist/storage/migrations/historicalVerificationPlan.js +0 -35
  103. package/dist/storage/migrations/integrationContinuation.js +0 -105
  104. package/dist/storage/migrations/narrowAgentFailureContext.js +0 -65
  105. package/dist/storage/migrations/notificationOnlyWakes.js +0 -74
  106. package/dist/storage/migrations/removeRuntimeGeneration.js +0 -207
  107. package/dist/storage/migrations/submitIntent.js +0 -126
  108. package/dist/storage/migrations/unifyHomeLayout.js +0 -925
  109. package/dist/storage/migrations/verificationPlanV1.js +0 -162
  110. package/dist/storage/migrations/verificationPolicy.js +0 -74
  111. package/dist/storage/migrations/workItemHistory.js +0 -46
@@ -2,312 +2,71 @@
2
2
 
3
3
  # SQLite control-plane storage
4
4
 
5
- Yui has one authoritative product Store: `YUI_HOME/yui.db` in WAL mode. The
6
- highest contiguous, checksummed row in `schema_migrations` is the one Home
7
- storage version accepted by the running release.
5
+ Yui has one authoritative product Store: `YUI_HOME/yui.db`, in WAL mode.
6
+ `storage_schema` contains its single **major.minor** version and schema
7
+ checksum. The clean baseline is **1.0**, introduced by package 1.0.0-alpha.
8
8
 
9
9
  ## Authority
10
10
 
11
- - `yui.db` owns Tasks, WorkItems, AgentRuns, Messages, Decisions, results, Project
12
- Knowledge references, managed workspace records, runtime bindings, mailboxes,
13
- durable events, and configuration.
14
- - Provider Sessions, transcripts, processes, caches, telemetry, and runtime
15
- observations support execution and diagnosis; they do not replace durable
16
- Task facts.
17
- - Configuration and diagnostics outside the database do not define another
18
- storage version or permit rebuilding Task truth heuristically.
11
+ - SQLite owns Tasks, WorkItems, AgentRuns, Messages, Decisions, results,
12
+ Project Knowledge, workspace records, runtime bindings, mailboxes, events
13
+ and configuration.
14
+ - Provider Sessions, transcripts, processes, caches and telemetry support
15
+ execution and diagnosis; they do not replace durable Task truth.
16
+ - Record `schemaVersion` tags validate current envelopes. They are not
17
+ separately writable upgrade versions.
18
+ - `storage_migration_archive` preserves opaque original payloads and binary
19
+ audit evidence. It is not a compatibility reader, scheduler or cache.
19
20
 
20
21
  ## Admission
21
22
 
22
- Ordinary commands open a Home only when all of these are true:
23
+ Ordinary opens require the exact current format/version/checksum, the current
24
+ physical schema objects, and valid typed records. A missing database in a
25
+ non-empty Home or a missing format identity is not permission to initialize.
26
+ New Homes execute the final baseline DDL once, without replaying old DDL.
23
27
 
24
- 1. `yui.db` exists and its migration ledger is a valid immutable prefix.
25
- 2. The ledger head exactly matches the running CLI's current storage version.
26
- 3. Current record validation and reference integrity succeed.
27
-
28
- An older Home inside the CLI's supported range fails ordinary admission but is
29
- classified as upgradeable. `yui doctor` and `yui upgrade --dry-run` report the
30
- ordered path without changing the Home. Explicit `yui upgrade` is the only
31
- standalone mutation boundary: it quiesces the Controller, backs up `yui.db`,
32
- applies all missing migrations transactionally, and validates the current
33
- model. A newer, below-minimum, incomplete, or malformed Home fails closed.
34
- There is no runtime normalization, repair worker, file-Store fallback, dual
35
- read/write path, or second migration authority.
36
-
37
- Typed domain payloads use one current validator registry on writes, ordinary
38
- reads, bounded Context pages and full-Home diagnostics. Direct and worker-backed
39
- Stores do not have different validation strength. Invalid records are rejected
40
- without advancing the revision; an invalid stored record remains available for
41
- explicit diagnosis, never normalized into a valid-looking replacement.
42
-
43
- Storage 35 also has a migration-only `storage_migration_archive` table:
44
- `migration_version / family / record_key / payload / content`. It retains exact
45
- retired Project/gate payloads and binary logs without interpreting them in the
46
- current model. It is not scheduling state, a second cache or a runtime fallback.
47
- Task-scoped historical Integration retirement uses ordinary Task Events.
48
- Backups retain both current records and this audit archive.
28
+ Writes, ordinary reads, bounded Context pages and full-Home diagnostics share
29
+ the current domain validators. Invalid records fail without normalization;
30
+ failed writes do not advance the Home revision. An open writer rechecks its
31
+ captured schema identity before every mutation.
49
32
 
50
33
  ## Write and concurrency contract
51
34
 
52
- - Each mutation is one SQLite transaction.
53
- - WAL plus `synchronous=FULL` provides the durable commit boundary.
54
- - `home_meta.revision` is the Home-wide CAS/revision used by callers that need
55
- a frozen read/modify/write boundary.
56
- - Typed columns support indexed identity and status queries; the full validated
57
- record payload remains the durable domain representation.
58
- - Mailbox claim, exact AgentRun terminalization, active-pointer removal, result
59
- persistence, and downstream wake creation are transactionally coupled where
60
- they form one product fact.
61
- - Idempotency keys and unique constraints protect repeatable external-effect
62
- acknowledgements; they do not form a second workflow state machine.
35
+ - Each mutation is a SQLite transaction; WAL plus `synchronous=FULL` is the
36
+ durable commit boundary.
37
+ - `home_meta.revision` is the Home-wide CAS/revision, not a format version.
38
+ - Indexed typed columns support exact identity/status queries; full validated
39
+ payloads remain the durable domain representation.
40
+ - Mailbox claims, exact Run terminalization, active-pointer removal, result
41
+ persistence and downstream notifications commit together where they express
42
+ one product fact.
43
+ - Idempotency and unique constraints protect confirmed external effects,
44
+ without inventing a second planning protocol.
63
45
 
64
46
  ## AgentRun and Session boundary
65
47
 
66
- An AgentRun is an explicitly requested execution. It records associated visible
67
- inputs and the original result, not hidden reasoning or the full tool trace.
68
- A Provider Session can contain multiple Runs, ordinary native chat and
69
- notifications. Native chat and notifications do not automatically create Runs.
70
- Only an exactly correlated native terminal settles the Run; the Leader remains
71
- the authority for WorkItem and Task acceptance.
48
+ An AgentRun records an explicit execution request, visible input and original
49
+ result, not hidden reasoning. A native Session may contain several Runs and
50
+ ordinary conversations. Notifications alone do not create Runs; only exact
51
+ native evidence settles one. WorkItem and Task acceptance remain Agent-owned.
72
52
 
73
53
  ## Update behavior
74
54
 
75
- `yui update` stages an exact package and asks that staged binary to classify the
76
- Home as current, migration-ready, or blocked. It then stops the exact
77
- Controller, activates the same package, runs the staged release's complete
78
- migration chain when required, verifies the installed binary and current Home,
79
- and starts the replacement Controller.
55
+ Version APIs expose `"1.0"` strings, not floats. Default `upgrade` / `update`
56
+ supports only a known contiguous minor path within the same storage major.
57
+ The initial baseline has no such steps. Cross-major and old integer formats
58
+ require an independent explicit converter; they never enter a runtime fallback.
59
+ The updater preflights the exact staged package before activation, then rechecks
60
+ under its maintenance fence. Unknown ownership remains a blocker.
80
61
 
81
- Every persistent schema or payload change appends one immutable, contiguous
82
- storage migration. The CLI publishes both `storageVersion` and
83
- `minimumStorageVersion`; every valid Home in that inclusive range can upgrade
84
- directly to the current version without installing intermediate releases.
85
- The current version and migration floor are declared only in
86
- `src/storage/storageVersions.ts` and exposed by CLI identity. Homes below that
87
- floor are not migration inputs and remain untouched.
88
- The target binary's `upgrade --update-preflight` and `--update-apply` result
89
- shapes and parent-owned handover-lock proof remain backward compatible with
90
- every updater released from storage version 1 onward, so an old source CLI can
91
- still drive a much newer target's complete migration chain.
62
+ See [Storage baseline 1.0](./storage-baseline.md) for the independent old-v37
63
+ conversion, backup/rollback, artifact boundaries and cold startup procedure.
92
64
 
93
65
  ## Unified Home layout
94
66
 
95
- Every Yui self-managed directory lives under the single canonical `YUI_HOME`
96
- (default `~/.yui`; an explicit `YUI_HOME` is honoured verbatim). `YUI_HOME` is
97
- never inferred from the current working directory and never substituted with a
98
- username. `src/storage/homeLayout.ts` is the one authority that derives each
99
- managed root from Home:
100
-
101
- | Root | Path | Holds |
102
- |---|---|---|
103
- | Managed worktrees | `<home>/workspaces/tasks/<taskId>/<owner>/<projectDirectory>` | Actual Task/WorkItem/Review/Integration Git directories, addressed by the bound Project directory. Owners are `main`, `work-items/<id>`, `reviews/<id>`, `integrations/<id>` and `execution-lanes/<group>/<lane>`. |
104
- | Read-only context views | Within the same owner directory | Regenerable symlinks to read-only Project context only; writable entries are actual Git directories, not links. |
105
- | Global Role workspace | `<home>/workspaces/global` | Default cwd for Yui-auto-created Global Roles (the `yui setup` Operator/Leader and ad-hoc Global Roles added without an explicit `--workspace`). A plain cwd, not a managed Git workspace. |
106
- | Task provider runtimes | `<home>/runtime/task-runtimes` | Task provider data/cache/tmp; also the planning cwd at `…/planning/<taskId>`. |
107
- | Integration runtimes | `<home>/runtime/integration-runtimes` | The integration check's provider data/cache/tmp (a separate partition from Task runtimes). |
108
- | Update staging | `<home>/runtime/update-staging` | `yui update`'s side-by-side package install (an upgrade artifact). |
109
- | Release workflow scratch | `<home>/runtime/release-workflow` | The release workflow's smoke-install dir and verified publish-snapshot tarball (release artifacts). |
110
- | Storage backups | `<home>/backups` | Pre-upgrade DB backups (the fenced upgrade's rollback anchor). |
111
-
112
- Published migrations 1–23 remain unchanged, including Task artifacts in local
113
- Git (19), Integration continuation (20), force-archive evidence (21), and
114
- Controller-owned Host ingress (22), and unified message input control (23).
115
- The two offline layout steps are now 23→24 (`unify-home-layout`) and
116
- 24→25 (`collapse-worktree-layout`). Version 24's
117
- `workspaces/worktree` directory is an intermediate layout, not a second live
118
- root at version 25. A single upgrade applies the full pending chain.
119
-
120
- Stop this Home's writers and take a backup before upgrading. The layout steps
121
- copy and verify the registered Git trees, repair only the copies' links, and
122
- preserve old sources for manual recovery. Version 25 replaces registered
123
- Task-view symlinks with real writable directories; unrelated Task scratch is
124
- retained. Read-only context remains a view and can be promoted to a writable
125
- worktree when WorkItem scope expands. Do not delete the old sources until the
126
- new layout is verified; a failed upgrade requires manual residue cleanup and
127
- backup recovery, not automatic resume.
128
-
129
- Both runtime partitions (`runtime/task-runtimes`, `runtime/integration-runtimes`)
130
- are the ONLY Home subtrees a provider runtime root is allowed to overlap; a
131
- runtime root overlapping any other part of Home (the database, `workspaces/`,
132
- `projects/`) is still rejected by `assertTaskRuntimeIsolationPreflight`, so
133
- unifying the root does not weaken control-data or cross-owner isolation.
134
-
135
- `defaultWorkspace` is a user-facing cwd for external Project input only; it is
136
- **not** a second authority for internal managed paths, and is intentionally not
137
- an input to `homeLayout.ts`. A Yui-auto-created Global Role that carries no
138
- user-chosen cwd no longer falls back to it (or to `process.cwd()`): `yui setup`'s
139
- built-in Operator/Leader and `yui config role add` without `--workspace` now default to
140
- the Home-internal `managedGlobalRoleWorkspace(home)` (`<home>/workspaces/global`),
141
- and `setup` no longer fabricates an external Home-sibling `workspace/` — a
142
- `default-workspace` is persisted only if the user configured one. A user who
143
- *names* an external directory (explicit `--workspace`, or a configured
144
- `default-workspace`) keeps external-resource semantics; the outside-Home guard
145
- still applies to it. The "planning/global cwd" that criterion 1 places under Home
146
- is thus both the *disposable runtime cwd Yui materializes itself* — the Draft
147
- planning cwd (`planningRuntimeCwd`, under `runtime/task-runtimes/planning`) — and
148
- the auto-created Global Role cwd above; only an operator's *explicitly named*
149
- external directory stays outside by design.
150
-
151
- Only genuine short-path IPC socket ENDPOINTS remain outside Home, and only
152
- because a Unix-domain `sockaddr_un` path has a small fixed length budget that a
153
- deep Home path would exceed. Each is a single socket path, never a data/cache/tmp
154
- root:
155
-
156
- - the Controller socket (`/tmp/yui-<uid>/<homeId>.sock`),
157
- - the tmux server socket (`/tmp/tmux-<uid>` via the tmux namespace),
158
- - the Agent Host socket (`/tmp/yui-<uid>/agent-host/…sock`), and
159
- - the integration check's tmux socket dir (`/tmp/yi-<uid>-<digest>`), bound only
160
- into `TMUX_TMPDIR`.
161
-
162
- The integration check's ordinary runtime state is **not** an exception: its
163
- provider data, cache, and temp roots live in the Home partition above
164
- (`runtime/integration-runtimes`); `TMPDIR`/`TMP`/`TEMP` point there, and only
165
- `TMUX_TMPDIR` is redirected to the short `/tmp` socket dir.
166
-
167
- ## Migration 23 → 24: unify managed paths under Home
168
-
169
- Historically the managed worktrees lived under the out-of-Home
170
- `defaultWorkspace` (`<ws>/worktree`, `<ws>/tasks`) and the provider runtimes
171
- under a string-built Home sibling (`<home>.task-runtimes`). The one forward
172
- migration `unify-home-layout` (`src/storage/migrations/unifyHomeLayout.ts`)
173
- brings that content under Home and rewrites the persisted absolute pointers the
174
- runtime dereferences as live, without re-cloning Git content or renaming the
175
- path-independent Git refs. It runs as the migration's `migrateData` step inside
176
- the upgrade transaction, so schema and data advance atomically or roll back
177
- together.
178
-
179
- **Exactly one tree is physically relocated: the managed Git worktree tree.** It
180
- is the sole subtree that holds durable, non-regenerable content (committed **and**
181
- uncommitted work), so it alone is copied on disk. Everything else that "moves"
182
- moves only by pointer:
183
-
184
- - the per-Task symlink views (`<ws>/tasks`) are regenerable — the pointer is
185
- rewritten and `ensureWorkspaceView` rebuilds the links at the next launch;
186
- - the provider runtimes (`<home>.task-runtimes`) are disposable — the pointer is
187
- rewritten and the roots are recreated at the next launch.
188
-
189
- The worktree copy is **non-destructive and verified** (see *Recovery and
190
- rollback*): the source is copied (never renamed away), the replica's content
191
- digest is checked against the source, and only a verified replica is atomically
192
- published. The original worktree tree is **preserved** as the rollback anchor;
193
- removing it is a later, authorized, post-restart cleanup step, never part of this
194
- transaction.
195
-
196
- The pointer rewrite is **surgical, not a table sweep** — only records the runtime
197
- treats as live launch pointers are touched:
198
-
199
- - `managed_workspaces` — the authoritative registry (`path` column, payload
200
- `root`, every `entries[].path`). Every surviving row is live (dispositioned
201
- rows are deleted at cleanup).
202
- - active (`status='active'`) `turns` — **both** `run.effective.workspace` (the
203
- actual OS launch cwd source) and the `run.workspace` mirror, rewritten together
204
- because `validateRun` requires them to stay identical; a run's
205
- `.result.systemEvidence.workspaceSnapshot` is frozen Git evidence and is left
206
- byte-for-byte intact.
207
- - `role_session_sets` / `global_role_session_sets` — each live session's
208
- `effective.workspace` in the `sessions` map; terminal sessions in `history` are
209
- preserved.
210
- - `review_rounds` — the mirrored workspace (only while its `managed_workspaces`
211
- owner row still exists) and each OPEN execution lane; an orphaned mirror or a
212
- terminal lane is frozen evidence and is preserved.
213
- - `work_items` — each OPEN execution lane inside `executionGroups`; candidate
214
- snapshots (`work_item_candidates`) are frozen and preserved.
215
- - `task_roles.workspace` — the live launch cwd, including a Draft's planning Role
216
- under the old runtime sibling (never self-healed until activation).
217
- - `task_records.cwd` — self-heals on the next `prepareTaskWorkspace`, but is
218
- rewritten defensively to close the stale-read window.
219
-
220
- Everything else is preserved on purpose: `context_snapshots`, terminal `turns`
221
- (with their system evidence), terminal sessions, `work_item_candidates`, terminal
222
- execution lanes, terminal `durable_jobs`, `events`, and reports are frozen
223
- history. `resource_registry` is re-discovered from disk; `projects.path` is an
224
- external, user-owned checkout.
225
-
226
- The migration is applied **offline** and is **fail-closed and pre-checkable**.
227
- It is run by the standalone `yui upgrade` boundary AFTER the operator has stopped
228
- this Home's Controller, Agent Host, and any execution/Job writers; it does not
229
- orchestrate that shutdown, coordinate an online write-stop, or migrate a live
230
- Session. It keeps only the minimal preconditions it can implement directly:
231
-
232
- - It **refuses** if a queued or running `durable_jobs` step is bound to a tree
233
- about to relocate. A durable Job's runner is detached and could outlive an
234
- incompletely stopped Controller, so moving that tree would risk an in-flight
235
- silent move; this is the one residual runtime signal the offline migration
236
- still guards. Let the Job drain or cancel it, then re-run the upgrade.
237
- (`active_turns` is steady state, not an in-flight signal, and is deliberately
238
- not consulted.)
239
- - It **refuses** if a relocation target already exists at all — it is either a
240
- foreign directory or residue from a failed prior run, and the offline migration
241
- never adopts a pre-existing target. Confirm the source is intact, then move or
242
- remove the target and re-run the upgrade.
243
- - Every refusal is surfaced as a **collected, read-only pre-check**: `yui
244
- upgrade --dry-run` and the updater's `--update-preflight` run the same plan and
245
- the same blocking conditions execute would throw on, opening the DB read-only
246
- and reporting each independent blocker as `{reason, detail}` (blocked outcome)
247
- without mutating the Home — a genuine pre-check, not a best-effort guess.
248
- - A Home already in the unified layout (or a fresh Home with nothing to relocate)
249
- is a **no-op**.
250
-
251
- ### Recovery and rollback
252
-
253
- The migration keeps **no recovery manifest and no resumable state machine** — it
254
- is a one-time offline transform, not an interruptible online orchestration. The
255
- worktree relocation is **copy → digest-verify → atomic-publish → preserve-source**:
256
-
257
- 1. the relocation target must not already exist; a pre-existing target is refused
258
- up front (foreign directory or failed-run residue — never adopted);
259
- 2. the source is copied into a same-filesystem staging dir (`<to>.incoming`),
260
- never renamed away;
261
- 3. a content-addressed inventory digest of the replica is compared to the source
262
- — a mismatch deletes the staging copy and aborts (nothing published, source
263
- intact);
264
- 4. only a verified replica is `rename`d into the final target (atomic on one
265
- filesystem);
266
- 5. the original source tree is left in place as the rollback anchor.
267
-
268
- There is **no automatic idempotent recovery**. Because a pre-existing target is
269
- always refused, a run interrupted after a partial publish does not silently
270
- resume or adopt the partial tree on the next attempt: the operator inspects the
271
- preserved source, removes the incomplete target (and any `<to>.incoming`
272
- staging), and re-runs the upgrade from a clean state. The `--dry-run` /
273
- `--update-preflight` pre-check surfaces exactly this `target-conflict` before the
274
- apply transaction is entered, so the residue is reported, not discovered
275
- mid-migration.
276
-
277
- After the copy, the worktrees are reconnected. `git worktree repair` chases the
278
- absolute pointer files inside a worktree, so running it on a verbatim copy whose
279
- pointers still address the OLD source would rewrite the OLD source's `.git`
280
- files and corrupt the rollback anchor. The migration therefore **relinks first**:
281
- it deterministically repoints, in the NEW copy only, the two cross-reference
282
- pointer files (a linked worktree's `.git` stub and each
283
- `main/.git/worktrees/<name>/gitdir`) from OLD to NEW, and only THEN runs `git
284
- worktree repair` from each main clone at its new path as a belt-and-braces
285
- reconciliation now confined to the new tree. This keeps the preserved source a
286
- fully independent, working Git: its `.git` is byte-for-byte unchanged and it
287
- still resolves HEAD/index/status after the migration (verified empirically on a
288
- private disposable Home). **A repair failure is fatal** — it aborts the migration
289
- so the transaction rolls back rather than advancing the version over unrepaired
290
- worktrees.
291
-
292
- Because the data step runs inside the upgrade transaction, any throw rolls the
293
- schema back to its original version; the fenced upgrade orchestrator additionally takes a
294
- `database.backup()` and restores it on failure. Recovery from a failed run is
295
- **manual, not automatic**: because the source is never removed and the copy is
296
- digest-verified before publish, the preserved source is always intact, so the
297
- operator clears any partial target and re-runs the upgrade. No re-run can lose or
298
- corrupt the original content, but the tool does not itself resume an interrupted
299
- move set.
300
-
301
- **Old-source cleanup** is intentionally deferred and out of band: after a
302
- successful upgrade the old external `worktree`, `tasks`, and `<home>.task-runtimes`
303
- roots are left **in place** (not emptied) until an operator-authorized cleanup
304
- removes them. This keeps a full rollback anchor available across the first
305
- restart.
306
-
307
- **Rollback limits:** once the Controller restarts against the unified layout and
308
- begins writing new records under Home, restoring the pre-upgrade DB backup no
309
- longer matches the newly written on-disk state. Until that first post-upgrade
310
- write, the preserved old source plus the DB backup are a complete rollback pair;
311
- after it, the supported recovery is forward (the layout is already unified), not a
312
- downgrade to the split layout. Verify an upgrade only on a private, disposable
313
- Home before applying it to a shared environment.
67
+ Self-managed data stays under one canonical Home: Task worktrees under
68
+ `workspaces/tasks/<task>/<owner>/<project>`, Global scratch under
69
+ `workspaces/global`, runtime data under `runtime`, and backups under `backups`.
70
+ Explicit external Project inputs keep their external-resource semantics.
71
+ Only deliberately short IPC socket paths may live outside Home.
72
+ The converter does not relocate workspaces or rewrite their Git identities.
@@ -2,72 +2,56 @@
2
2
 
3
3
  # SQLite 控制面存储
4
4
 
5
- Yui 只有一个权威产品 Store:WAL 模式下的 `YUI_HOME/yui.db`。`schema_migrations`
6
- 中连续且带校验和的最高一行,就是当前发行版接受的那一个 Home 存储版本。
5
+ Yui 唯一的权威产品 Store 是 WAL 模式的 `YUI_HOME/yui.db`。
6
+ `storage_schema` 单行记录唯一的**主版本.小版本**与 Schema 摘要;
7
+ 1.0.0-alpha 软件包引入的纯净基线为 **1.0**。
7
8
 
8
9
  ## 权威
9
10
 
10
- - `yui.db` 拥有 Task、WorkItem、AgentRun、Message、Decision、结果、Project
11
- Knowledge 引用、受管工作区记录、运行时绑定、mailbox、持久事件和配置。
12
- - Provider Session、transcript、进程、缓存、telemetry 和运行时观察服务于执行
13
- 与诊断,不替代持久的 Task 事实。
14
- - 数据库之外的配置和诊断不定义另一个存储版本,也不允许启发式地重建 Task 真相。
11
+ SQLite 拥有 Task、WorkItem、AgentRun、Message、Decision、结果、Project
12
+ Knowledge、工作区、运行绑定、mailbox、事件及配置。Provider Session、
13
+ transcript、进程、缓存和 telemetry 服务于执行诊断,不替代持久 Task 真相。
14
+ 记录的 `schemaVersion` 仅校验当前格式,不构成独立升级轴。
15
+ `storage_migration_archive` 保留不透明原始负载和二进制审计证据,
16
+ 不是旧格式读取器、调度器或缓存。
15
17
 
16
18
  ## 准入
17
19
 
18
- 普通命令只有在同时满足以下条件时才打开 Home:
19
-
20
- 1. `yui.db` 存在,且其迁移账本是一个有效的不可变前缀。
21
- 2. 账本头恰好等于运行中 CLI 的当前存储版本。
22
- 3. 当前记录校验与引用完整性均通过。
23
-
24
- 落在 CLI 支持区间内的更旧 Home 无法通过普通准入,但被归类为可升级。
25
- `yui doctor` 和 `yui upgrade --dry-run` 会报告有序的升级路径而不改动 Home。
26
- 显式的 `yui upgrade` 是唯一的独立变更边界:它让 Controller 静止、备份
27
- `yui.db`、以事务方式套用所有缺失迁移,并校验当前模型。更新的、低于最低版本的、
28
- 不完整的或损坏的 Home 一律 fail closed。不存在运行时归一化、修复 worker、
29
- 文件 Store 回退、双读写路径或第二套迁移权威。
30
-
31
- 类型化领域记录在写入、普通读取、有界 Context 分页和全 Home 检查时,共用一份
32
- 现行校验注册表。直接 Store 与 worker-backed Store 不存在不同强度的校验。
33
- 无效写入不会推进 revision;已存在的无效记录只会报错并保留供明确诊断,
34
- 不会被自动规范化成看似有效的替代记录。
35
-
36
- 存储 35 新增仅由迁移写入的 `storage_migration_archive`:
37
- `migration_version / family / record_key / payload / content`。它保留退休的
38
- Project/gate 原始 payload 和二进制日志,不在当前模型中解释这些历史格式,
39
- 也不是调度状态、第二份缓存或运行时回退。Task 内的历史 Integration 使用普通
40
- Task 事件保存。数据库备份同时保留当前记录与这份审计存档。
20
+ 普通打开要求精确的当前格式、版本、摘要、物理结构与类型化记录。
21
+ 非空 Home 缺少数据库或版本身份时拒绝初始化。新 Home 一次创建最终 DDL,
22
+ 不重放旧迁移。写入、普通读取、Context 分页及全 Home 诊断共享当前校验器;
23
+ 无效记录只报错,不规范化修复;失败写入不推进 revision。
24
+ 已打开的写连接在每次修改前重查它捕获的 Schema 身份。
41
25
 
42
26
  ## 写入与并发合同
43
27
 
44
- - 每次修改是一个 SQLite 事务。
45
- - WAL 加 `synchronous=FULL` 提供持久提交边界。
46
- - `home_meta.revision` 是全 Home 范围的 CAS/revision,供需要冻结
47
- read/modify/write 边界的调用者使用。
48
- - 类型化列支持按身份和状态建索引查询;完整且经校验的记录负载仍是持久的领域表示。
49
- - mailbox 认领、精确的 AgentRun 终结、活动指针移除、结果持久化以及下游唤醒创建,
50
- 在它们构成同一条产品事实时以事务方式耦合。
51
- - 幂等键与唯一约束保护可重复的外部效果确认,不构成第二套工作流状态机。
28
+ 每次修改是 SQLite 事务,WAL 和 `synchronous=FULL` 保护持久提交。
29
+ `home_meta.revision` 是 Home 范围的 CAS/revision,不是格式版本。
30
+ 索引列支持精确查询,完整且经过校验的负载仍是领域表示。
31
+ mailbox 认领、精确 Run 终结、活动指针移除、结果持久化及下游通知,
32
+ 在构成同一产品事实时一并提交。幂等键和唯一约束保护外部效果,
33
+ 不另建规划协议。
52
34
 
53
35
  ## AgentRun 与 Session 边界
54
36
 
55
- AgentRun 是一次明确请求的执行。它记录相关的可见输入和原始结果,而不是隐藏的
56
- 推理过程或完整工具轨迹。一个 Provider Session 可以包含多个 Run、普通原生对话
57
- 和通知。原生对话和通知不会自动创建 Run。只有精确关联的原生终态才结算该 Run;
58
- WorkItem 与 Task 的验收权威仍归 Leader。
37
+ AgentRun 记录显式执行请求、可见输入和原始结果,不记录隐藏推理。
38
+ 一个原生 Session 可以包含多个 Run 和普通对话。通知本身不创建 Run,
39
+ 只有精确原生证据才能结算;WorkItem、Task 的验收仍由 Agent 判断。
59
40
 
60
41
  ## 更新行为
61
42
 
62
- `yui update` 暂存一个确切的包,并要求那个暂存二进制把 Home 判定为当前、可迁移
63
- 或受阻。随后它停止那个确切的 Controller、激活同一个包、在需要时运行暂存发行版
64
- 的完整迁移链、校验已安装二进制与当前 Home,再启动替换后的 Controller。
43
+ 版本 API 返回 `"1.0"` 字符串,不用浮点数。默认 `upgrade/update`
44
+ 只接受同一存储主版本内完整且连续的小版本路径;初始基线尚无升级步骤。
45
+ 跨主版本和旧整数格式只由独立显式转换器处理,运行包没有回退。
46
+ updater 先检查精确暂存包,再在维护锁内重查;未知所有权始终阻塞。
47
+
48
+ 独立旧 v37 转换、备份恢复、产物边界和冷启动流程见
49
+ [存储基线 1.0](./storage-baseline.zh-CN.md)。
50
+
51
+ ## Home 布局
65
52
 
66
- 每次持久 schema 或负载变更都追加一条不可变、连续的存储迁移。CLI 同时发布
67
- `storageVersion` 与 `minimumStorageVersion`;处在该闭区间内的每个有效 Home 都能
68
- 直接升级到当前版本,无需安装中间发行版。当前版本及最低支持版本统一由
69
- `src/storage/storageVersions.ts` 声明,并由 CLI 身份读取暴露。
70
- 低于该下限的 Home 不是迁移输入,保持原样不动。目标二进制的
71
- `upgrade --update-preflight` 与 `--update-apply` 结果形态,以及由父进程持有的
72
- 交接锁证明,对从存储版本 1 起发布的每个 updater 都保持向后兼容,因此一个旧的
73
- 源码 CLI 仍能驱动一个新得多的目标的完整迁移链。
53
+ 自管理数据位于规范 Home 内:Task worktree 在
54
+ `workspaces/tasks/<task>/<owner>/<project>`,Global scratch 在
55
+ `workspaces/global`,运行数据在 `runtime`,备份在 `backups`。
56
+ 显式外部 Project 保留外部资源语义;只有受限长度的 IPC socket 可位于 Home 外。
57
+ 一次性转换不搬迁工作区,也不改写 Git 身份。
@@ -0,0 +1,132 @@
1
+ # Storage baseline 1.0
2
+
3
+ Yui 1.0.0-alpha is the clean runtime baseline. Package version `1.0.0-alpha`, Home
4
+ storage version `1.0`, record envelope version `1`, and Controller protocol `1`
5
+ have different responsibilities. The later 1.0.0 package reuses the final
6
+ verified prerelease storage contract; it must not reset it again. Persistent
7
+ changes after the first alpha publication require explicit minor transitions.
8
+
9
+ ## Current runtime
10
+
11
+ - `storage_schema` contains one authoritative identity: format, major, minor,
12
+ schema checksum and creation time. JSON APIs expose storage versions as
13
+ canonical strings such as `"1.0"`, not floating-point numbers.
14
+ - New Homes execute the complete current DDL once. They do not replay the old
15
+ v1..v37 ledger. Ordinary reads validate the current identity, physical schema
16
+ and typed records; they never normalize data.
17
+ - Explicit `upgrade` / `update` can apply only a complete, contiguous **minor**
18
+ path within one major. The initial 1.0 baseline has no minor steps.
19
+ Cross-major, downgrade, unknown and old integer formats fail before activation.
20
+ An exact package selector is not permission to cross a storage major.
21
+ - Yui-owned record envelopes start at 1. Host control uses the distinct
22
+ `yui-agent-host-control/v1` identity, so it cannot accidentally adopt an old
23
+ `yui-agent-host/v1` producer. Context/Run/Host-event/Driver contracts already at
24
+ v1 remain there. External Provider protocols and package versions are unchanged.
25
+ - Business revisions, authority epochs, IDs, event sequences, native data and
26
+ immutable Context resources are not schema versions. Never reset them.
27
+ - `storage_migration_archive` is opaque audit evidence, not an executable
28
+ compatibility reader. Old numbers and original bytes in audit remain intact.
29
+ - Candidates have one current authority: the owning WorkItem's candidate array.
30
+ The baseline does not create the unused `work_item_candidates` or
31
+ `coordination_locks` tables, or the obsolete `idx_input_open` index.
32
+
33
+ The runtime tarball contains neither historical migration modules nor the
34
+ standalone converter. Source-level history and previous published packages
35
+ remain available for explicit diagnosis, not automatic runtime fallback.
36
+
37
+ ## One-time conversion from 0.16.2
38
+
39
+ The independent `yui-baseline-cutover` archive accepts only the exact frozen
40
+ 0.16.2 schema and its complete v37 ledger. Older Homes must first use 0.16.2.
41
+ Admission is based on this exact storage contract, not an installed package
42
+ version: an already-valid v37 Home does not need a cosmetic rewrite.
43
+ It never runs the old migration chain itself.
44
+
45
+ The unpublished v37 → 1.0 transition preserves Runs with missing optional Snapshot
46
+ references, including their original references and results. A retained record
47
+ need not be ready to execute: missing evidence must not block Home conversion.
48
+ The Leader/Operator can inspect and retire the affected Run; an explicit ordinary
49
+ retry creates a new Run and Snapshot from current authorized facts without
50
+ rewriting the old evidence. Exact Review/synthesis reuse still requires its
51
+ frozen evidence. Unknown formats, structural corruption and unsettled external
52
+ effects remain separate conversion blockers.
53
+
54
+ 1. Using 0.16.2, settle active Runs, Jobs, claimed notifications, in-flight
55
+ retries and unconfirmed effects. Preserve queued intent rather than marking
56
+ it completed. Stop managed Sessions and the Controller. `session stop --all`
57
+ refuses busy Sessions; it is not authority to force-stop or discard work.
58
+ 2. Stage the published 1.0.0-alpha package in a separate installation prefix, with
59
+ its native dependencies installed. Do not overwrite the global CLI or try to
60
+ run the new Controller against the old Home.
61
+ 3. Verify the converter archive checksum, unpack it, then use its entrypoint.
62
+ The examples below use **placeholders**, not a production Home:
63
+
64
+ ```sh
65
+ node /absolute/converter/cli.mjs \
66
+ --home /absolute/home --runtime /absolute/staged/package
67
+
68
+ node /absolute/converter/cli.mjs \
69
+ --home /absolute/home --runtime /absolute/staged/package \
70
+ --apply --backup-dir /absolute/new-backup-directory
71
+ ```
72
+
73
+ `--runtime` names the package directory containing `package.json`, `dist/` and
74
+ available dependencies, not its `bin/yui` entrypoint. The default invocation is
75
+ read-only. `--apply` requires a new backup directory outside Home, under an
76
+ existing canonical parent. Run from an external Operator shell, not a managed
77
+ Task or a Session that is itself being converted.
78
+
79
+ The tool refuses active durable execution, pending outbox operations, active
80
+ Session bindings, recorded live processes, observable processes referencing
81
+ this Home/database, pending native Inbox files and unfinished Controller
82
+ handover/discovery. Unknown identity for a recorded owner remains a blocker;
83
+ unrelated user processes are not adopted or terminated. Stop unmanaged writers
84
+ and external workspace editors too; a file copy is not a filesystem snapshot.
85
+ The maintenance fence and SQL write transaction protect the conversion.
86
+
87
+ Before mutation, the tool copies Home and creates a self-contained SQLite
88
+ backup with checksum. It converts only named Yui envelopes and active typed
89
+ verification plans, preserving each changed payload and the original ledger in
90
+ audit. User text, native payloads, frozen Context bytes, Git data, dirty files,
91
+ IDs, counters and Task outcomes are preserved.
92
+ Before dropping the two unused source tables, every original row is retained
93
+ under `baseline-v37/retired-table/<table>` in the audit archive. Their payloads
94
+ are opaque evidence, not active records to normalize. The receipt reports
95
+ `retiredRows` separately from changed current records.
96
+
97
+ Old `active-release.json` and `runtime-identity.json` are archived in
98
+ `retired-runtime/`, not relabelled as observations of the new runtime. The tool
99
+ also verifies and converts the typed isolation owner markers at the declared
100
+ runtime inventory paths, retaining their originals and recomputing only the
101
+ format-dependent fingerprint. Resource paths, namespace and port allocations
102
+ do not change. Unknown markers or links remain blockers. The tool
103
+ does not resume old Hosts, update the global installation, start a Controller,
104
+ or submit model input. After a successful conversion, use the staged 1.0.0-alpha CLI
105
+ to install the exact package and start only the new runtime. Start new managed
106
+ Sessions through the ordinary explicit lifecycle; history is not live authority.
107
+
108
+ ## Evidence and recovery
109
+
110
+ Success is `outcome: converted`, target `1.0`, with the exact backup path.
111
+ Repeating against a valid current Home returns `already-current` without
112
+ creating another backup or rewriting records.
113
+
114
+ - `backup/home/` preserves the Home tree; `backup/yui.db` is the consistent
115
+ standalone database snapshot; `receipt.json` identifies source, target,
116
+ checksum and completed conversion. `retired-runtime/` preserves old bindings.
117
+ - A validation failure rolls back the SQL transaction and restores runtime
118
+ bindings moved by that attempt. If restoration cannot be proven, the error
119
+ names the retained files; keep Home stopped.
120
+ - A crash or receipt-write failure is not proof that storage stayed old.
121
+ Inspect the actual format and backup before choosing recovery. There is no
122
+ automatic repair worker or speculative replay.
123
+ - To roll back, stop all new writers, preserve the failed/new Home separately,
124
+ and restore the old Home tree plus the standalone `yui.db` at the **same**
125
+ original Home path, without mixing in newer WAL/SHM files. Use only 0.16.2.
126
+ Review the converter-owned lock in the snapshot by exact process identity.
127
+ - After new business writes, restoring the old backup loses those writes.
128
+ Recovery then needs an explicit disposition; never restore automatically.
129
+
130
+ Real Home conversion and publication require separate user authorization.
131
+ Isolated fixture evidence does not claim that a particular production Home is
132
+ ready to convert.