@zq-silk/yui 0.16.2 → 1.0.0-alpha
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -4
- package/dist/agent/agent.js +4 -9
- package/dist/agent/executionComponents.js +4 -4
- package/dist/agentRun/agentRun.js +8 -8
- package/dist/artifacts/managedGit.js +3 -56
- package/dist/brief/taskBrief.js +3 -3
- package/dist/cli/updateOrchestrator.js +2 -13
- package/dist/cli/updatePorts.js +36 -69
- package/dist/cli/upgradeCommand.js +4 -8
- package/dist/cli.js +4 -7
- package/dist/commands/controllerCommands.js +1 -1
- package/dist/commands/taskCommands.js +8 -1
- package/dist/commands/taskRoleRuntimeStatus.js +1 -1
- package/dist/context/runInputContract.js +1 -1
- package/dist/controller/agentRuntimeObserver.js +4 -4
- package/dist/controller/clientRuntime.js +19 -44
- package/dist/controller/controller.js +16 -4
- package/dist/controller/fileSchedulerStoreAdapter.js +6 -6
- package/dist/controller/globalInputDelivery.js +3 -1
- package/dist/controller/jobSupervisor.js +3 -3
- package/dist/controller/providerRetryDelivery.js +128 -118
- package/dist/controller/runtime.js +1 -1
- package/dist/controller/structuredProviderObservation.js +1 -1
- package/dist/coordination/workMailbox.js +4 -4
- package/dist/core/controllerIdentity.js +25 -0
- package/dist/core/controllerProcessIdentity.js +2 -2
- package/dist/core/controllerServer.js +4 -2
- package/dist/core/protocol.js +1 -1
- package/dist/domain/validation.js +6 -0
- package/dist/event/taskEvent.js +3 -3
- package/dist/execution/workItemExecution.js +2 -2
- package/dist/executor/agentExecutor.js +6 -6
- package/dist/executor/effectiveLaunch.js +3 -3
- package/dist/grant/capabilityGrant.js +2 -2
- package/dist/input/inputRequest.js +4 -4
- package/dist/integration/changeSet.js +3 -3
- package/dist/integration/integrationAttempt.js +3 -3
- package/dist/integration/integrationSourceApplication.js +1 -1
- package/dist/job/durableJob.js +1 -1
- package/dist/job/jobRunner.js +2 -2
- package/dist/message/message.js +8 -6
- package/dist/milestone/milestone.js +3 -3
- package/dist/profile/agentProfile.js +3 -3
- package/dist/release/runtimeRelease.js +9 -7
- package/dist/repository/project.js +3 -3
- package/dist/resources/liveReferences.js +1 -1
- package/dist/resources/sqliteResourceRegistry.js +2 -2
- package/dist/review/reviewRound.js +5 -5
- package/dist/role/role.js +7 -8
- package/dist/runtime/acpProtocol.js +2 -3
- package/dist/runtime/agentDriverObservation.js +1 -1
- package/dist/runtime/agentHost.js +3 -3
- package/dist/runtime/agentHostProtocol.js +2 -2
- package/dist/runtime/codexInteractiveHost.js +1 -1
- package/dist/runtime/launchBroker.js +1 -1
- package/dist/runtime/processExitObservation.js +1 -1
- package/dist/runtime/providerContinuationReconciliationService.js +84 -75
- package/dist/runtime/providerRuntimeIdentity.js +3 -3
- package/dist/runtime/runtimeCoherence.js +7 -3
- package/dist/runtime/runtimeObservation.js +3 -3
- package/dist/runtime/sessionOwnerIdentity.js +1 -1
- package/dist/runtime/taskRuntimeIsolation.js +3 -3
- package/dist/runtime/tmuxAdapters.js +3 -3
- package/dist/scheduler/taskWake.js +1 -1
- package/dist/storage/baselineSchema.js +606 -0
- package/dist/storage/homeLayout.js +5 -16
- package/dist/storage/recordValidation.js +16 -4
- package/dist/storage/sqliteSchema.js +106 -1741
- package/dist/storage/sqliteStore.js +16 -14
- package/dist/storage/storageSchema.js +8 -7
- package/dist/storage/storageVersions.js +23 -16
- package/dist/storage/taskStore.js +3 -29
- package/dist/storage/upgrade/upgradeOrchestrator.js +14 -102
- package/dist/task/task.js +15 -7
- package/dist/task/taskActivation.js +5 -4
- package/dist/telemetry/sqliteTelemetryStore.js +2 -2
- package/dist/verification/gateArtifact.js +5 -3
- package/dist/verification/verificationPlan.js +6 -7
- package/dist/workItem/workItem.js +6 -6
- package/dist/workspace/cleanupInspection.js +1 -9
- package/dist/worktree/managedWorkspace.js +3 -3
- package/docs/managed-turn-and-session-runtime.md +26 -14
- package/docs/managed-turn-and-session-runtime.zh-CN.md +18 -10
- package/docs/release-workflow.md +52 -300
- package/docs/release-workflow.zh-CN.md +41 -234
- package/docs/sqlite-control-plane-design.md +48 -289
- package/docs/sqlite-control-plane-design.zh-CN.md +37 -53
- package/docs/storage-baseline.md +132 -0
- package/docs/storage-baseline.zh-CN.md +106 -0
- package/docs/task-delivery.md +3 -4
- package/docs/task-delivery.zh-CN.md +3 -3
- package/docs/testing/verification-levels.md +40 -180
- package/docs/testing/verification-levels.zh-CN.md +27 -131
- package/i18n/README.zh-CN.md +5 -4
- package/package.json +1 -1
- package/dist/storage/migrations/agentFailureContext.js +0 -22
- package/dist/storage/migrations/agentRunContract.js +0 -159
- package/dist/storage/migrations/artifactsToGit.js +0 -338
- package/dist/storage/migrations/collapseWorktreeLayout.js +0 -963
- package/dist/storage/migrations/currentInputContract.js +0 -86
- package/dist/storage/migrations/currentRuntimeContract.js +0 -228
- package/dist/storage/migrations/historicalVerificationPlan.js +0 -35
- package/dist/storage/migrations/integrationContinuation.js +0 -105
- package/dist/storage/migrations/narrowAgentFailureContext.js +0 -65
- package/dist/storage/migrations/notificationOnlyWakes.js +0 -74
- package/dist/storage/migrations/removeRuntimeGeneration.js +0 -207
- package/dist/storage/migrations/submitIntent.js +0 -126
- package/dist/storage/migrations/unifyHomeLayout.js +0 -925
- package/dist/storage/migrations/verificationPlanV1.js +0 -162
- package/dist/storage/migrations/verificationPolicy.js +0 -74
- package/dist/storage/migrations/workItemHistory.js +0 -46
|
@@ -2,312 +2,71 @@
|
|
|
2
2
|
|
|
3
3
|
# SQLite control-plane storage
|
|
4
4
|
|
|
5
|
-
Yui has one authoritative product Store: `YUI_HOME/yui.db
|
|
6
|
-
|
|
7
|
-
|
|
5
|
+
Yui has one authoritative product Store: `YUI_HOME/yui.db`, in WAL mode.
|
|
6
|
+
`storage_schema` contains its single **major.minor** version and schema
|
|
7
|
+
checksum. The clean baseline is **1.0**, introduced by package 1.0.0-alpha.
|
|
8
8
|
|
|
9
9
|
## Authority
|
|
10
10
|
|
|
11
|
-
-
|
|
12
|
-
Knowledge
|
|
13
|
-
|
|
14
|
-
- Provider Sessions, transcripts, processes, caches
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
11
|
+
- SQLite owns Tasks, WorkItems, AgentRuns, Messages, Decisions, results,
|
|
12
|
+
Project Knowledge, workspace records, runtime bindings, mailboxes, events
|
|
13
|
+
and configuration.
|
|
14
|
+
- Provider Sessions, transcripts, processes, caches and telemetry support
|
|
15
|
+
execution and diagnosis; they do not replace durable Task truth.
|
|
16
|
+
- Record `schemaVersion` tags validate current envelopes. They are not
|
|
17
|
+
separately writable upgrade versions.
|
|
18
|
+
- `storage_migration_archive` preserves opaque original payloads and binary
|
|
19
|
+
audit evidence. It is not a compatibility reader, scheduler or cache.
|
|
19
20
|
|
|
20
21
|
## Admission
|
|
21
22
|
|
|
22
|
-
Ordinary
|
|
23
|
+
Ordinary opens require the exact current format/version/checksum, the current
|
|
24
|
+
physical schema objects, and valid typed records. A missing database in a
|
|
25
|
+
non-empty Home or a missing format identity is not permission to initialize.
|
|
26
|
+
New Homes execute the final baseline DDL once, without replaying old DDL.
|
|
23
27
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
An older Home inside the CLI's supported range fails ordinary admission but is
|
|
29
|
-
classified as upgradeable. `yui doctor` and `yui upgrade --dry-run` report the
|
|
30
|
-
ordered path without changing the Home. Explicit `yui upgrade` is the only
|
|
31
|
-
standalone mutation boundary: it quiesces the Controller, backs up `yui.db`,
|
|
32
|
-
applies all missing migrations transactionally, and validates the current
|
|
33
|
-
model. A newer, below-minimum, incomplete, or malformed Home fails closed.
|
|
34
|
-
There is no runtime normalization, repair worker, file-Store fallback, dual
|
|
35
|
-
read/write path, or second migration authority.
|
|
36
|
-
|
|
37
|
-
Typed domain payloads use one current validator registry on writes, ordinary
|
|
38
|
-
reads, bounded Context pages and full-Home diagnostics. Direct and worker-backed
|
|
39
|
-
Stores do not have different validation strength. Invalid records are rejected
|
|
40
|
-
without advancing the revision; an invalid stored record remains available for
|
|
41
|
-
explicit diagnosis, never normalized into a valid-looking replacement.
|
|
42
|
-
|
|
43
|
-
Storage 35 also has a migration-only `storage_migration_archive` table:
|
|
44
|
-
`migration_version / family / record_key / payload / content`. It retains exact
|
|
45
|
-
retired Project/gate payloads and binary logs without interpreting them in the
|
|
46
|
-
current model. It is not scheduling state, a second cache or a runtime fallback.
|
|
47
|
-
Task-scoped historical Integration retirement uses ordinary Task Events.
|
|
48
|
-
Backups retain both current records and this audit archive.
|
|
28
|
+
Writes, ordinary reads, bounded Context pages and full-Home diagnostics share
|
|
29
|
+
the current domain validators. Invalid records fail without normalization;
|
|
30
|
+
failed writes do not advance the Home revision. An open writer rechecks its
|
|
31
|
+
captured schema identity before every mutation.
|
|
49
32
|
|
|
50
33
|
## Write and concurrency contract
|
|
51
34
|
|
|
52
|
-
- Each mutation is
|
|
53
|
-
|
|
54
|
-
- `home_meta.revision` is the Home-wide CAS/revision
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
acknowledgements; they do not form a second workflow state machine.
|
|
35
|
+
- Each mutation is a SQLite transaction; WAL plus `synchronous=FULL` is the
|
|
36
|
+
durable commit boundary.
|
|
37
|
+
- `home_meta.revision` is the Home-wide CAS/revision, not a format version.
|
|
38
|
+
- Indexed typed columns support exact identity/status queries; full validated
|
|
39
|
+
payloads remain the durable domain representation.
|
|
40
|
+
- Mailbox claims, exact Run terminalization, active-pointer removal, result
|
|
41
|
+
persistence and downstream notifications commit together where they express
|
|
42
|
+
one product fact.
|
|
43
|
+
- Idempotency and unique constraints protect confirmed external effects,
|
|
44
|
+
without inventing a second planning protocol.
|
|
63
45
|
|
|
64
46
|
## AgentRun and Session boundary
|
|
65
47
|
|
|
66
|
-
An AgentRun
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
Only an exactly correlated native terminal settles the Run; the Leader remains
|
|
71
|
-
the authority for WorkItem and Task acceptance.
|
|
48
|
+
An AgentRun records an explicit execution request, visible input and original
|
|
49
|
+
result, not hidden reasoning. A native Session may contain several Runs and
|
|
50
|
+
ordinary conversations. Notifications alone do not create Runs; only exact
|
|
51
|
+
native evidence settles one. WorkItem and Task acceptance remain Agent-owned.
|
|
72
52
|
|
|
73
53
|
## Update behavior
|
|
74
54
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
55
|
+
Version APIs expose `"1.0"` strings, not floats. Default `upgrade` / `update`
|
|
56
|
+
supports only a known contiguous minor path within the same storage major.
|
|
57
|
+
The initial baseline has no such steps. Cross-major and old integer formats
|
|
58
|
+
require an independent explicit converter; they never enter a runtime fallback.
|
|
59
|
+
The updater preflights the exact staged package before activation, then rechecks
|
|
60
|
+
under its maintenance fence. Unknown ownership remains a blocker.
|
|
80
61
|
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
`minimumStorageVersion`; every valid Home in that inclusive range can upgrade
|
|
84
|
-
directly to the current version without installing intermediate releases.
|
|
85
|
-
The current version and migration floor are declared only in
|
|
86
|
-
`src/storage/storageVersions.ts` and exposed by CLI identity. Homes below that
|
|
87
|
-
floor are not migration inputs and remain untouched.
|
|
88
|
-
The target binary's `upgrade --update-preflight` and `--update-apply` result
|
|
89
|
-
shapes and parent-owned handover-lock proof remain backward compatible with
|
|
90
|
-
every updater released from storage version 1 onward, so an old source CLI can
|
|
91
|
-
still drive a much newer target's complete migration chain.
|
|
62
|
+
See [Storage baseline 1.0](./storage-baseline.md) for the independent old-v37
|
|
63
|
+
conversion, backup/rollback, artifact boundaries and cold startup procedure.
|
|
92
64
|
|
|
93
65
|
## Unified Home layout
|
|
94
66
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
| Root | Path | Holds |
|
|
102
|
-
|---|---|---|
|
|
103
|
-
| Managed worktrees | `<home>/workspaces/tasks/<taskId>/<owner>/<projectDirectory>` | Actual Task/WorkItem/Review/Integration Git directories, addressed by the bound Project directory. Owners are `main`, `work-items/<id>`, `reviews/<id>`, `integrations/<id>` and `execution-lanes/<group>/<lane>`. |
|
|
104
|
-
| Read-only context views | Within the same owner directory | Regenerable symlinks to read-only Project context only; writable entries are actual Git directories, not links. |
|
|
105
|
-
| Global Role workspace | `<home>/workspaces/global` | Default cwd for Yui-auto-created Global Roles (the `yui setup` Operator/Leader and ad-hoc Global Roles added without an explicit `--workspace`). A plain cwd, not a managed Git workspace. |
|
|
106
|
-
| Task provider runtimes | `<home>/runtime/task-runtimes` | Task provider data/cache/tmp; also the planning cwd at `…/planning/<taskId>`. |
|
|
107
|
-
| Integration runtimes | `<home>/runtime/integration-runtimes` | The integration check's provider data/cache/tmp (a separate partition from Task runtimes). |
|
|
108
|
-
| Update staging | `<home>/runtime/update-staging` | `yui update`'s side-by-side package install (an upgrade artifact). |
|
|
109
|
-
| Release workflow scratch | `<home>/runtime/release-workflow` | The release workflow's smoke-install dir and verified publish-snapshot tarball (release artifacts). |
|
|
110
|
-
| Storage backups | `<home>/backups` | Pre-upgrade DB backups (the fenced upgrade's rollback anchor). |
|
|
111
|
-
|
|
112
|
-
Published migrations 1–23 remain unchanged, including Task artifacts in local
|
|
113
|
-
Git (19), Integration continuation (20), force-archive evidence (21), and
|
|
114
|
-
Controller-owned Host ingress (22), and unified message input control (23).
|
|
115
|
-
The two offline layout steps are now 23→24 (`unify-home-layout`) and
|
|
116
|
-
24→25 (`collapse-worktree-layout`). Version 24's
|
|
117
|
-
`workspaces/worktree` directory is an intermediate layout, not a second live
|
|
118
|
-
root at version 25. A single upgrade applies the full pending chain.
|
|
119
|
-
|
|
120
|
-
Stop this Home's writers and take a backup before upgrading. The layout steps
|
|
121
|
-
copy and verify the registered Git trees, repair only the copies' links, and
|
|
122
|
-
preserve old sources for manual recovery. Version 25 replaces registered
|
|
123
|
-
Task-view symlinks with real writable directories; unrelated Task scratch is
|
|
124
|
-
retained. Read-only context remains a view and can be promoted to a writable
|
|
125
|
-
worktree when WorkItem scope expands. Do not delete the old sources until the
|
|
126
|
-
new layout is verified; a failed upgrade requires manual residue cleanup and
|
|
127
|
-
backup recovery, not automatic resume.
|
|
128
|
-
|
|
129
|
-
Both runtime partitions (`runtime/task-runtimes`, `runtime/integration-runtimes`)
|
|
130
|
-
are the ONLY Home subtrees a provider runtime root is allowed to overlap; a
|
|
131
|
-
runtime root overlapping any other part of Home (the database, `workspaces/`,
|
|
132
|
-
`projects/`) is still rejected by `assertTaskRuntimeIsolationPreflight`, so
|
|
133
|
-
unifying the root does not weaken control-data or cross-owner isolation.
|
|
134
|
-
|
|
135
|
-
`defaultWorkspace` is a user-facing cwd for external Project input only; it is
|
|
136
|
-
**not** a second authority for internal managed paths, and is intentionally not
|
|
137
|
-
an input to `homeLayout.ts`. A Yui-auto-created Global Role that carries no
|
|
138
|
-
user-chosen cwd no longer falls back to it (or to `process.cwd()`): `yui setup`'s
|
|
139
|
-
built-in Operator/Leader and `yui config role add` without `--workspace` now default to
|
|
140
|
-
the Home-internal `managedGlobalRoleWorkspace(home)` (`<home>/workspaces/global`),
|
|
141
|
-
and `setup` no longer fabricates an external Home-sibling `workspace/` — a
|
|
142
|
-
`default-workspace` is persisted only if the user configured one. A user who
|
|
143
|
-
*names* an external directory (explicit `--workspace`, or a configured
|
|
144
|
-
`default-workspace`) keeps external-resource semantics; the outside-Home guard
|
|
145
|
-
still applies to it. The "planning/global cwd" that criterion 1 places under Home
|
|
146
|
-
is thus both the *disposable runtime cwd Yui materializes itself* — the Draft
|
|
147
|
-
planning cwd (`planningRuntimeCwd`, under `runtime/task-runtimes/planning`) — and
|
|
148
|
-
the auto-created Global Role cwd above; only an operator's *explicitly named*
|
|
149
|
-
external directory stays outside by design.
|
|
150
|
-
|
|
151
|
-
Only genuine short-path IPC socket ENDPOINTS remain outside Home, and only
|
|
152
|
-
because a Unix-domain `sockaddr_un` path has a small fixed length budget that a
|
|
153
|
-
deep Home path would exceed. Each is a single socket path, never a data/cache/tmp
|
|
154
|
-
root:
|
|
155
|
-
|
|
156
|
-
- the Controller socket (`/tmp/yui-<uid>/<homeId>.sock`),
|
|
157
|
-
- the tmux server socket (`/tmp/tmux-<uid>` via the tmux namespace),
|
|
158
|
-
- the Agent Host socket (`/tmp/yui-<uid>/agent-host/…sock`), and
|
|
159
|
-
- the integration check's tmux socket dir (`/tmp/yi-<uid>-<digest>`), bound only
|
|
160
|
-
into `TMUX_TMPDIR`.
|
|
161
|
-
|
|
162
|
-
The integration check's ordinary runtime state is **not** an exception: its
|
|
163
|
-
provider data, cache, and temp roots live in the Home partition above
|
|
164
|
-
(`runtime/integration-runtimes`); `TMPDIR`/`TMP`/`TEMP` point there, and only
|
|
165
|
-
`TMUX_TMPDIR` is redirected to the short `/tmp` socket dir.
|
|
166
|
-
|
|
167
|
-
## Migration 23 → 24: unify managed paths under Home
|
|
168
|
-
|
|
169
|
-
Historically the managed worktrees lived under the out-of-Home
|
|
170
|
-
`defaultWorkspace` (`<ws>/worktree`, `<ws>/tasks`) and the provider runtimes
|
|
171
|
-
under a string-built Home sibling (`<home>.task-runtimes`). The one forward
|
|
172
|
-
migration `unify-home-layout` (`src/storage/migrations/unifyHomeLayout.ts`)
|
|
173
|
-
brings that content under Home and rewrites the persisted absolute pointers the
|
|
174
|
-
runtime dereferences as live, without re-cloning Git content or renaming the
|
|
175
|
-
path-independent Git refs. It runs as the migration's `migrateData` step inside
|
|
176
|
-
the upgrade transaction, so schema and data advance atomically or roll back
|
|
177
|
-
together.
|
|
178
|
-
|
|
179
|
-
**Exactly one tree is physically relocated: the managed Git worktree tree.** It
|
|
180
|
-
is the sole subtree that holds durable, non-regenerable content (committed **and**
|
|
181
|
-
uncommitted work), so it alone is copied on disk. Everything else that "moves"
|
|
182
|
-
moves only by pointer:
|
|
183
|
-
|
|
184
|
-
- the per-Task symlink views (`<ws>/tasks`) are regenerable — the pointer is
|
|
185
|
-
rewritten and `ensureWorkspaceView` rebuilds the links at the next launch;
|
|
186
|
-
- the provider runtimes (`<home>.task-runtimes`) are disposable — the pointer is
|
|
187
|
-
rewritten and the roots are recreated at the next launch.
|
|
188
|
-
|
|
189
|
-
The worktree copy is **non-destructive and verified** (see *Recovery and
|
|
190
|
-
rollback*): the source is copied (never renamed away), the replica's content
|
|
191
|
-
digest is checked against the source, and only a verified replica is atomically
|
|
192
|
-
published. The original worktree tree is **preserved** as the rollback anchor;
|
|
193
|
-
removing it is a later, authorized, post-restart cleanup step, never part of this
|
|
194
|
-
transaction.
|
|
195
|
-
|
|
196
|
-
The pointer rewrite is **surgical, not a table sweep** — only records the runtime
|
|
197
|
-
treats as live launch pointers are touched:
|
|
198
|
-
|
|
199
|
-
- `managed_workspaces` — the authoritative registry (`path` column, payload
|
|
200
|
-
`root`, every `entries[].path`). Every surviving row is live (dispositioned
|
|
201
|
-
rows are deleted at cleanup).
|
|
202
|
-
- active (`status='active'`) `turns` — **both** `run.effective.workspace` (the
|
|
203
|
-
actual OS launch cwd source) and the `run.workspace` mirror, rewritten together
|
|
204
|
-
because `validateRun` requires them to stay identical; a run's
|
|
205
|
-
`.result.systemEvidence.workspaceSnapshot` is frozen Git evidence and is left
|
|
206
|
-
byte-for-byte intact.
|
|
207
|
-
- `role_session_sets` / `global_role_session_sets` — each live session's
|
|
208
|
-
`effective.workspace` in the `sessions` map; terminal sessions in `history` are
|
|
209
|
-
preserved.
|
|
210
|
-
- `review_rounds` — the mirrored workspace (only while its `managed_workspaces`
|
|
211
|
-
owner row still exists) and each OPEN execution lane; an orphaned mirror or a
|
|
212
|
-
terminal lane is frozen evidence and is preserved.
|
|
213
|
-
- `work_items` — each OPEN execution lane inside `executionGroups`; candidate
|
|
214
|
-
snapshots (`work_item_candidates`) are frozen and preserved.
|
|
215
|
-
- `task_roles.workspace` — the live launch cwd, including a Draft's planning Role
|
|
216
|
-
under the old runtime sibling (never self-healed until activation).
|
|
217
|
-
- `task_records.cwd` — self-heals on the next `prepareTaskWorkspace`, but is
|
|
218
|
-
rewritten defensively to close the stale-read window.
|
|
219
|
-
|
|
220
|
-
Everything else is preserved on purpose: `context_snapshots`, terminal `turns`
|
|
221
|
-
(with their system evidence), terminal sessions, `work_item_candidates`, terminal
|
|
222
|
-
execution lanes, terminal `durable_jobs`, `events`, and reports are frozen
|
|
223
|
-
history. `resource_registry` is re-discovered from disk; `projects.path` is an
|
|
224
|
-
external, user-owned checkout.
|
|
225
|
-
|
|
226
|
-
The migration is applied **offline** and is **fail-closed and pre-checkable**.
|
|
227
|
-
It is run by the standalone `yui upgrade` boundary AFTER the operator has stopped
|
|
228
|
-
this Home's Controller, Agent Host, and any execution/Job writers; it does not
|
|
229
|
-
orchestrate that shutdown, coordinate an online write-stop, or migrate a live
|
|
230
|
-
Session. It keeps only the minimal preconditions it can implement directly:
|
|
231
|
-
|
|
232
|
-
- It **refuses** if a queued or running `durable_jobs` step is bound to a tree
|
|
233
|
-
about to relocate. A durable Job's runner is detached and could outlive an
|
|
234
|
-
incompletely stopped Controller, so moving that tree would risk an in-flight
|
|
235
|
-
silent move; this is the one residual runtime signal the offline migration
|
|
236
|
-
still guards. Let the Job drain or cancel it, then re-run the upgrade.
|
|
237
|
-
(`active_turns` is steady state, not an in-flight signal, and is deliberately
|
|
238
|
-
not consulted.)
|
|
239
|
-
- It **refuses** if a relocation target already exists at all — it is either a
|
|
240
|
-
foreign directory or residue from a failed prior run, and the offline migration
|
|
241
|
-
never adopts a pre-existing target. Confirm the source is intact, then move or
|
|
242
|
-
remove the target and re-run the upgrade.
|
|
243
|
-
- Every refusal is surfaced as a **collected, read-only pre-check**: `yui
|
|
244
|
-
upgrade --dry-run` and the updater's `--update-preflight` run the same plan and
|
|
245
|
-
the same blocking conditions execute would throw on, opening the DB read-only
|
|
246
|
-
and reporting each independent blocker as `{reason, detail}` (blocked outcome)
|
|
247
|
-
without mutating the Home — a genuine pre-check, not a best-effort guess.
|
|
248
|
-
- A Home already in the unified layout (or a fresh Home with nothing to relocate)
|
|
249
|
-
is a **no-op**.
|
|
250
|
-
|
|
251
|
-
### Recovery and rollback
|
|
252
|
-
|
|
253
|
-
The migration keeps **no recovery manifest and no resumable state machine** — it
|
|
254
|
-
is a one-time offline transform, not an interruptible online orchestration. The
|
|
255
|
-
worktree relocation is **copy → digest-verify → atomic-publish → preserve-source**:
|
|
256
|
-
|
|
257
|
-
1. the relocation target must not already exist; a pre-existing target is refused
|
|
258
|
-
up front (foreign directory or failed-run residue — never adopted);
|
|
259
|
-
2. the source is copied into a same-filesystem staging dir (`<to>.incoming`),
|
|
260
|
-
never renamed away;
|
|
261
|
-
3. a content-addressed inventory digest of the replica is compared to the source
|
|
262
|
-
— a mismatch deletes the staging copy and aborts (nothing published, source
|
|
263
|
-
intact);
|
|
264
|
-
4. only a verified replica is `rename`d into the final target (atomic on one
|
|
265
|
-
filesystem);
|
|
266
|
-
5. the original source tree is left in place as the rollback anchor.
|
|
267
|
-
|
|
268
|
-
There is **no automatic idempotent recovery**. Because a pre-existing target is
|
|
269
|
-
always refused, a run interrupted after a partial publish does not silently
|
|
270
|
-
resume or adopt the partial tree on the next attempt: the operator inspects the
|
|
271
|
-
preserved source, removes the incomplete target (and any `<to>.incoming`
|
|
272
|
-
staging), and re-runs the upgrade from a clean state. The `--dry-run` /
|
|
273
|
-
`--update-preflight` pre-check surfaces exactly this `target-conflict` before the
|
|
274
|
-
apply transaction is entered, so the residue is reported, not discovered
|
|
275
|
-
mid-migration.
|
|
276
|
-
|
|
277
|
-
After the copy, the worktrees are reconnected. `git worktree repair` chases the
|
|
278
|
-
absolute pointer files inside a worktree, so running it on a verbatim copy whose
|
|
279
|
-
pointers still address the OLD source would rewrite the OLD source's `.git`
|
|
280
|
-
files and corrupt the rollback anchor. The migration therefore **relinks first**:
|
|
281
|
-
it deterministically repoints, in the NEW copy only, the two cross-reference
|
|
282
|
-
pointer files (a linked worktree's `.git` stub and each
|
|
283
|
-
`main/.git/worktrees/<name>/gitdir`) from OLD to NEW, and only THEN runs `git
|
|
284
|
-
worktree repair` from each main clone at its new path as a belt-and-braces
|
|
285
|
-
reconciliation now confined to the new tree. This keeps the preserved source a
|
|
286
|
-
fully independent, working Git: its `.git` is byte-for-byte unchanged and it
|
|
287
|
-
still resolves HEAD/index/status after the migration (verified empirically on a
|
|
288
|
-
private disposable Home). **A repair failure is fatal** — it aborts the migration
|
|
289
|
-
so the transaction rolls back rather than advancing the version over unrepaired
|
|
290
|
-
worktrees.
|
|
291
|
-
|
|
292
|
-
Because the data step runs inside the upgrade transaction, any throw rolls the
|
|
293
|
-
schema back to its original version; the fenced upgrade orchestrator additionally takes a
|
|
294
|
-
`database.backup()` and restores it on failure. Recovery from a failed run is
|
|
295
|
-
**manual, not automatic**: because the source is never removed and the copy is
|
|
296
|
-
digest-verified before publish, the preserved source is always intact, so the
|
|
297
|
-
operator clears any partial target and re-runs the upgrade. No re-run can lose or
|
|
298
|
-
corrupt the original content, but the tool does not itself resume an interrupted
|
|
299
|
-
move set.
|
|
300
|
-
|
|
301
|
-
**Old-source cleanup** is intentionally deferred and out of band: after a
|
|
302
|
-
successful upgrade the old external `worktree`, `tasks`, and `<home>.task-runtimes`
|
|
303
|
-
roots are left **in place** (not emptied) until an operator-authorized cleanup
|
|
304
|
-
removes them. This keeps a full rollback anchor available across the first
|
|
305
|
-
restart.
|
|
306
|
-
|
|
307
|
-
**Rollback limits:** once the Controller restarts against the unified layout and
|
|
308
|
-
begins writing new records under Home, restoring the pre-upgrade DB backup no
|
|
309
|
-
longer matches the newly written on-disk state. Until that first post-upgrade
|
|
310
|
-
write, the preserved old source plus the DB backup are a complete rollback pair;
|
|
311
|
-
after it, the supported recovery is forward (the layout is already unified), not a
|
|
312
|
-
downgrade to the split layout. Verify an upgrade only on a private, disposable
|
|
313
|
-
Home before applying it to a shared environment.
|
|
67
|
+
Self-managed data stays under one canonical Home: Task worktrees under
|
|
68
|
+
`workspaces/tasks/<task>/<owner>/<project>`, Global scratch under
|
|
69
|
+
`workspaces/global`, runtime data under `runtime`, and backups under `backups`.
|
|
70
|
+
Explicit external Project inputs keep their external-resource semantics.
|
|
71
|
+
Only deliberately short IPC socket paths may live outside Home.
|
|
72
|
+
The converter does not relocate workspaces or rewrite their Git identities.
|
|
@@ -2,72 +2,56 @@
|
|
|
2
2
|
|
|
3
3
|
# SQLite 控制面存储
|
|
4
4
|
|
|
5
|
-
Yui
|
|
6
|
-
|
|
5
|
+
Yui 唯一的权威产品 Store 是 WAL 模式的 `YUI_HOME/yui.db`。
|
|
6
|
+
`storage_schema` 单行记录唯一的**主版本.小版本**与 Schema 摘要;
|
|
7
|
+
1.0.0-alpha 软件包引入的纯净基线为 **1.0**。
|
|
7
8
|
|
|
8
9
|
## 权威
|
|
9
10
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
11
|
+
SQLite 拥有 Task、WorkItem、AgentRun、Message、Decision、结果、Project
|
|
12
|
+
Knowledge、工作区、运行绑定、mailbox、事件及配置。Provider Session、
|
|
13
|
+
transcript、进程、缓存和 telemetry 服务于执行诊断,不替代持久 Task 真相。
|
|
14
|
+
记录的 `schemaVersion` 仅校验当前格式,不构成独立升级轴。
|
|
15
|
+
`storage_migration_archive` 保留不透明原始负载和二进制审计证据,
|
|
16
|
+
不是旧格式读取器、调度器或缓存。
|
|
15
17
|
|
|
16
18
|
## 准入
|
|
17
19
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
落在 CLI 支持区间内的更旧 Home 无法通过普通准入,但被归类为可升级。
|
|
25
|
-
`yui doctor` 和 `yui upgrade --dry-run` 会报告有序的升级路径而不改动 Home。
|
|
26
|
-
显式的 `yui upgrade` 是唯一的独立变更边界:它让 Controller 静止、备份
|
|
27
|
-
`yui.db`、以事务方式套用所有缺失迁移,并校验当前模型。更新的、低于最低版本的、
|
|
28
|
-
不完整的或损坏的 Home 一律 fail closed。不存在运行时归一化、修复 worker、
|
|
29
|
-
文件 Store 回退、双读写路径或第二套迁移权威。
|
|
30
|
-
|
|
31
|
-
类型化领域记录在写入、普通读取、有界 Context 分页和全 Home 检查时,共用一份
|
|
32
|
-
现行校验注册表。直接 Store 与 worker-backed Store 不存在不同强度的校验。
|
|
33
|
-
无效写入不会推进 revision;已存在的无效记录只会报错并保留供明确诊断,
|
|
34
|
-
不会被自动规范化成看似有效的替代记录。
|
|
35
|
-
|
|
36
|
-
存储 35 新增仅由迁移写入的 `storage_migration_archive`:
|
|
37
|
-
`migration_version / family / record_key / payload / content`。它保留退休的
|
|
38
|
-
Project/gate 原始 payload 和二进制日志,不在当前模型中解释这些历史格式,
|
|
39
|
-
也不是调度状态、第二份缓存或运行时回退。Task 内的历史 Integration 使用普通
|
|
40
|
-
Task 事件保存。数据库备份同时保留当前记录与这份审计存档。
|
|
20
|
+
普通打开要求精确的当前格式、版本、摘要、物理结构与类型化记录。
|
|
21
|
+
非空 Home 缺少数据库或版本身份时拒绝初始化。新 Home 一次创建最终 DDL,
|
|
22
|
+
不重放旧迁移。写入、普通读取、Context 分页及全 Home 诊断共享当前校验器;
|
|
23
|
+
无效记录只报错,不规范化修复;失败写入不推进 revision。
|
|
24
|
+
已打开的写连接在每次修改前重查它捕获的 Schema 身份。
|
|
41
25
|
|
|
42
26
|
## 写入与并发合同
|
|
43
27
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
在它们构成同一条产品事实时以事务方式耦合。
|
|
51
|
-
- 幂等键与唯一约束保护可重复的外部效果确认,不构成第二套工作流状态机。
|
|
28
|
+
每次修改是 SQLite 事务,WAL 和 `synchronous=FULL` 保护持久提交。
|
|
29
|
+
`home_meta.revision` 是 Home 范围的 CAS/revision,不是格式版本。
|
|
30
|
+
索引列支持精确查询,完整且经过校验的负载仍是领域表示。
|
|
31
|
+
mailbox 认领、精确 Run 终结、活动指针移除、结果持久化及下游通知,
|
|
32
|
+
在构成同一产品事实时一并提交。幂等键和唯一约束保护外部效果,
|
|
33
|
+
不另建规划协议。
|
|
52
34
|
|
|
53
35
|
## AgentRun 与 Session 边界
|
|
54
36
|
|
|
55
|
-
AgentRun
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
WorkItem 与 Task 的验收权威仍归 Leader。
|
|
37
|
+
AgentRun 记录显式执行请求、可见输入和原始结果,不记录隐藏推理。
|
|
38
|
+
一个原生 Session 可以包含多个 Run 和普通对话。通知本身不创建 Run,
|
|
39
|
+
只有精确原生证据才能结算;WorkItem、Task 的验收仍由 Agent 判断。
|
|
59
40
|
|
|
60
41
|
## 更新行为
|
|
61
42
|
|
|
62
|
-
`
|
|
63
|
-
|
|
64
|
-
|
|
43
|
+
版本 API 返回 `"1.0"` 字符串,不用浮点数。默认 `upgrade/update`
|
|
44
|
+
只接受同一存储主版本内完整且连续的小版本路径;初始基线尚无升级步骤。
|
|
45
|
+
跨主版本和旧整数格式只由独立显式转换器处理,运行包没有回退。
|
|
46
|
+
updater 先检查精确暂存包,再在维护锁内重查;未知所有权始终阻塞。
|
|
47
|
+
|
|
48
|
+
独立旧 v37 转换、备份恢复、产物边界和冷启动流程见
|
|
49
|
+
[存储基线 1.0](./storage-baseline.zh-CN.md)。
|
|
50
|
+
|
|
51
|
+
## Home 布局
|
|
65
52
|
|
|
66
|
-
|
|
67
|
-
`
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
`upgrade --update-preflight` 与 `--update-apply` 结果形态,以及由父进程持有的
|
|
72
|
-
交接锁证明,对从存储版本 1 起发布的每个 updater 都保持向后兼容,因此一个旧的
|
|
73
|
-
源码 CLI 仍能驱动一个新得多的目标的完整迁移链。
|
|
53
|
+
自管理数据位于规范 Home 内:Task worktree 在
|
|
54
|
+
`workspaces/tasks/<task>/<owner>/<project>`,Global scratch 在
|
|
55
|
+
`workspaces/global`,运行数据在 `runtime`,备份在 `backups`。
|
|
56
|
+
显式外部 Project 保留外部资源语义;只有受限长度的 IPC socket 可位于 Home 外。
|
|
57
|
+
一次性转换不搬迁工作区,也不改写 Git 身份。
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
# Storage baseline 1.0
|
|
2
|
+
|
|
3
|
+
Yui 1.0.0-alpha is the clean runtime baseline. Package version `1.0.0-alpha`, Home
|
|
4
|
+
storage version `1.0`, record envelope version `1`, and Controller protocol `1`
|
|
5
|
+
have different responsibilities. The later 1.0.0 package reuses the final
|
|
6
|
+
verified prerelease storage contract; it must not reset it again. Persistent
|
|
7
|
+
changes after the first alpha publication require explicit minor transitions.
|
|
8
|
+
|
|
9
|
+
## Current runtime
|
|
10
|
+
|
|
11
|
+
- `storage_schema` contains one authoritative identity: format, major, minor,
|
|
12
|
+
schema checksum and creation time. JSON APIs expose storage versions as
|
|
13
|
+
canonical strings such as `"1.0"`, not floating-point numbers.
|
|
14
|
+
- New Homes execute the complete current DDL once. They do not replay the old
|
|
15
|
+
v1..v37 ledger. Ordinary reads validate the current identity, physical schema
|
|
16
|
+
and typed records; they never normalize data.
|
|
17
|
+
- Explicit `upgrade` / `update` can apply only a complete, contiguous **minor**
|
|
18
|
+
path within one major. The initial 1.0 baseline has no minor steps.
|
|
19
|
+
Cross-major, downgrade, unknown and old integer formats fail before activation.
|
|
20
|
+
An exact package selector is not permission to cross a storage major.
|
|
21
|
+
- Yui-owned record envelopes start at 1. Host control uses the distinct
|
|
22
|
+
`yui-agent-host-control/v1` identity, so it cannot accidentally adopt an old
|
|
23
|
+
`yui-agent-host/v1` producer. Context/Run/Host-event/Driver contracts already at
|
|
24
|
+
v1 remain there. External Provider protocols and package versions are unchanged.
|
|
25
|
+
- Business revisions, authority epochs, IDs, event sequences, native data and
|
|
26
|
+
immutable Context resources are not schema versions. Never reset them.
|
|
27
|
+
- `storage_migration_archive` is opaque audit evidence, not an executable
|
|
28
|
+
compatibility reader. Old numbers and original bytes in audit remain intact.
|
|
29
|
+
- Candidates have one current authority: the owning WorkItem's candidate array.
|
|
30
|
+
The baseline does not create the unused `work_item_candidates` or
|
|
31
|
+
`coordination_locks` tables, or the obsolete `idx_input_open` index.
|
|
32
|
+
|
|
33
|
+
The runtime tarball contains neither historical migration modules nor the
|
|
34
|
+
standalone converter. Source-level history and previous published packages
|
|
35
|
+
remain available for explicit diagnosis, not automatic runtime fallback.
|
|
36
|
+
|
|
37
|
+
## One-time conversion from 0.16.2
|
|
38
|
+
|
|
39
|
+
The independent `yui-baseline-cutover` archive accepts only the exact frozen
|
|
40
|
+
0.16.2 schema and its complete v37 ledger. Older Homes must first use 0.16.2.
|
|
41
|
+
Admission is based on this exact storage contract, not an installed package
|
|
42
|
+
version: an already-valid v37 Home does not need a cosmetic rewrite.
|
|
43
|
+
It never runs the old migration chain itself.
|
|
44
|
+
|
|
45
|
+
The unpublished v37 → 1.0 transition preserves Runs with missing optional Snapshot
|
|
46
|
+
references, including their original references and results. A retained record
|
|
47
|
+
need not be ready to execute: missing evidence must not block Home conversion.
|
|
48
|
+
The Leader/Operator can inspect and retire the affected Run; an explicit ordinary
|
|
49
|
+
retry creates a new Run and Snapshot from current authorized facts without
|
|
50
|
+
rewriting the old evidence. Exact Review/synthesis reuse still requires its
|
|
51
|
+
frozen evidence. Unknown formats, structural corruption and unsettled external
|
|
52
|
+
effects remain separate conversion blockers.
|
|
53
|
+
|
|
54
|
+
1. Using 0.16.2, settle active Runs, Jobs, claimed notifications, in-flight
|
|
55
|
+
retries and unconfirmed effects. Preserve queued intent rather than marking
|
|
56
|
+
it completed. Stop managed Sessions and the Controller. `session stop --all`
|
|
57
|
+
refuses busy Sessions; it is not authority to force-stop or discard work.
|
|
58
|
+
2. Stage the published 1.0.0-alpha package in a separate installation prefix, with
|
|
59
|
+
its native dependencies installed. Do not overwrite the global CLI or try to
|
|
60
|
+
run the new Controller against the old Home.
|
|
61
|
+
3. Verify the converter archive checksum, unpack it, then use its entrypoint.
|
|
62
|
+
The examples below use **placeholders**, not a production Home:
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
node /absolute/converter/cli.mjs \
|
|
66
|
+
--home /absolute/home --runtime /absolute/staged/package
|
|
67
|
+
|
|
68
|
+
node /absolute/converter/cli.mjs \
|
|
69
|
+
--home /absolute/home --runtime /absolute/staged/package \
|
|
70
|
+
--apply --backup-dir /absolute/new-backup-directory
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
`--runtime` names the package directory containing `package.json`, `dist/` and
|
|
74
|
+
available dependencies, not its `bin/yui` entrypoint. The default invocation is
|
|
75
|
+
read-only. `--apply` requires a new backup directory outside Home, under an
|
|
76
|
+
existing canonical parent. Run from an external Operator shell, not a managed
|
|
77
|
+
Task or a Session that is itself being converted.
|
|
78
|
+
|
|
79
|
+
The tool refuses active durable execution, pending outbox operations, active
|
|
80
|
+
Session bindings, recorded live processes, observable processes referencing
|
|
81
|
+
this Home/database, pending native Inbox files and unfinished Controller
|
|
82
|
+
handover/discovery. Unknown identity for a recorded owner remains a blocker;
|
|
83
|
+
unrelated user processes are not adopted or terminated. Stop unmanaged writers
|
|
84
|
+
and external workspace editors too; a file copy is not a filesystem snapshot.
|
|
85
|
+
The maintenance fence and SQL write transaction protect the conversion.
|
|
86
|
+
|
|
87
|
+
Before mutation, the tool copies Home and creates a self-contained SQLite
|
|
88
|
+
backup with checksum. It converts only named Yui envelopes and active typed
|
|
89
|
+
verification plans, preserving each changed payload and the original ledger in
|
|
90
|
+
audit. User text, native payloads, frozen Context bytes, Git data, dirty files,
|
|
91
|
+
IDs, counters and Task outcomes are preserved.
|
|
92
|
+
Before dropping the two unused source tables, every original row is retained
|
|
93
|
+
under `baseline-v37/retired-table/<table>` in the audit archive. Their payloads
|
|
94
|
+
are opaque evidence, not active records to normalize. The receipt reports
|
|
95
|
+
`retiredRows` separately from changed current records.
|
|
96
|
+
|
|
97
|
+
Old `active-release.json` and `runtime-identity.json` are archived in
|
|
98
|
+
`retired-runtime/`, not relabelled as observations of the new runtime. The tool
|
|
99
|
+
also verifies and converts the typed isolation owner markers at the declared
|
|
100
|
+
runtime inventory paths, retaining their originals and recomputing only the
|
|
101
|
+
format-dependent fingerprint. Resource paths, namespace and port allocations
|
|
102
|
+
do not change. Unknown markers or links remain blockers. The tool
|
|
103
|
+
does not resume old Hosts, update the global installation, start a Controller,
|
|
104
|
+
or submit model input. After a successful conversion, use the staged 1.0.0-alpha CLI
|
|
105
|
+
to install the exact package and start only the new runtime. Start new managed
|
|
106
|
+
Sessions through the ordinary explicit lifecycle; history is not live authority.
|
|
107
|
+
|
|
108
|
+
## Evidence and recovery
|
|
109
|
+
|
|
110
|
+
Success is `outcome: converted`, target `1.0`, with the exact backup path.
|
|
111
|
+
Repeating against a valid current Home returns `already-current` without
|
|
112
|
+
creating another backup or rewriting records.
|
|
113
|
+
|
|
114
|
+
- `backup/home/` preserves the Home tree; `backup/yui.db` is the consistent
|
|
115
|
+
standalone database snapshot; `receipt.json` identifies source, target,
|
|
116
|
+
checksum and completed conversion. `retired-runtime/` preserves old bindings.
|
|
117
|
+
- A validation failure rolls back the SQL transaction and restores runtime
|
|
118
|
+
bindings moved by that attempt. If restoration cannot be proven, the error
|
|
119
|
+
names the retained files; keep Home stopped.
|
|
120
|
+
- A crash or receipt-write failure is not proof that storage stayed old.
|
|
121
|
+
Inspect the actual format and backup before choosing recovery. There is no
|
|
122
|
+
automatic repair worker or speculative replay.
|
|
123
|
+
- To roll back, stop all new writers, preserve the failed/new Home separately,
|
|
124
|
+
and restore the old Home tree plus the standalone `yui.db` at the **same**
|
|
125
|
+
original Home path, without mixing in newer WAL/SHM files. Use only 0.16.2.
|
|
126
|
+
Review the converter-owned lock in the snapshot by exact process identity.
|
|
127
|
+
- After new business writes, restoring the old backup loses those writes.
|
|
128
|
+
Recovery then needs an explicit disposition; never restore automatically.
|
|
129
|
+
|
|
130
|
+
Real Home conversion and publication require separate user authorization.
|
|
131
|
+
Isolated fixture evidence does not claim that a particular production Home is
|
|
132
|
+
ready to convert.
|