@zq-silk/yui 0.6.3 → 0.6.4

package/README.md

@@ -881,94 +881,19 @@ atomic storage switch, purely to make that single switch recoverable — it is n
 a general backup/restore facility.)
 
 See [ARCHITECTURE.md](./ARCHITECTURE.md) for persistence and scheduling details.
-The reusable, user-driven acceptance plan is documented in
-[Operator routing and long-running Task E2E testing](./docs/testing/operator-routing-e2e-plan.md).
-
 ## Development
 
 ```sh
 npm ci
-npm run test:core
-```
-
-`npm run test:core` builds once and runs the same small, explicit core matrix as
-CI. During development, add the smallest affected test files for the behavior
-being changed.
-
-`npm test` (and `make test` / `make check`) remains the full **deterministic**
-diagnostic suite. Run it on demand for unusually cross-cutting changes or
-regression investigation; it is not a routine merge requirement. It never
-launches a real model or touches the global `yui` binary, a shared `YUI_HOME`,
-or a running production Session. It stays
-deterministic even when launched from inside a managed Yui Session, because it
-preloads `test/helpers/scrubSessionEnv.js` to strip every Yui-owned managed
-runtime value from the test process, including shared `YUI_HOME`, exact Leader
-action assertions, workspace projections, and Agent launch descriptors. Tests
-that touch Home/CLI/Controller/tmux explicitly supply a test-created isolated
-Home. The same preamble puts local refusal shims for bare `codex` and `claude`
-ahead of the caller's `PATH`; Session fixtures install observable Mock Agents
-inside their owned Home instead. Only a dedicated managed-identity child may
-opt out. The Provider E2E tier is exempt from the shims only after its explicit
-opt-in and mandatory isolation preflight path has been selected.
-
-### Test tiers
-
-Yui's tests are classified into five explicit, executable tiers so a reader
-never has to guess what a test actually exercised. Each tier declares whether it
-creates a Session, whether it calls a real model, and whether it stands up a
-disposable real runtime. Agent workflow for applying these tiers while developing
-Yui lives in [`.agents/skills/develop-yui/SKILL.md`](.agents/skills/develop-yui/SKILL.md); it is
-not part of the generic Leader, Worker, or Reviewer workflow:
-
-| Tier | Session | Real model | Disposable runtime | Preflight | Opt-in |
-| --- | --- | --- | --- | --- | --- |
-| Unit | no | no | no | no | — |
-| Isolated Integration | yes | no | yes | no | — |
-| Mock Agent Session | yes | no | yes | no | — |
-| Provider E2E | yes | **yes** | yes | **required** | `YUI_ALLOW_PROVIDER_E2E=1` |
-| Release E2E | **no** | **no** | yes | **required** | `YUI_ALLOW_RELEASE_E2E=1` |
-
-```sh
-make test-tier T=unit          # or: npm run test:tier -- unit
-npm run test:tier -- unit -- --test-name-pattern "test name"
-node scripts/run-test-tier.mjs list
+npm test
 ```
 
-The supported tier entrypoint always runs the canonical `npm run build` first.
-It therefore works on a fresh checkout and cannot mistake a present but stale
-`dist/cli.js` for current code. The raw `node --test dist/...` path remains an
-unsupported bypass of that freshness boundary.
-
-**Provider E2E is the only tier that calls a real model.** Release E2E, on its
-normal path, creates no Session and calls no model — it exercises
-binary/install/update/upgrade release flows against real npm/home/namespace
-resources. Both tiers are **privileged and fail-closed**: they live only in
-nested privileged manifests excluded from the default test glob, refuse to run
-without their opt-in env var, and execute through one wrapper that registers
-cleanup before observation and does not even evaluate the scenario module until
-the blocking isolation preflight (`assertIsolationReady`) passes. Active-Session
-observation is runner-owned and uses an all-scope Yui runtime inventory;
-scenario code cannot replace it or manufacture an empty result. The preflight
-requires an absolute
-checkout-local launcher; a run root proven **temporary and creator-bound owned by
-this run** — created via `createOwnedRunRoot` (mkdtemp + a random-token
-ownership receipt) and re-proven by that exact token, with a symlink run root
-refused and every path canonicalized so a symlink escape cannot pass a lexical
-check; the disposable `YUI_HOME`, workspace, isolated npm prefix, and unique
-runtime namespace all derived *inside that exact owned root* and **physically
-fenced** against symlink escape; and an **explicit** observation that zero
-production Sessions are active (missing evidence fails closed — it is never
-assumed empty). No bare `yui`, `make link` symlink, shared home, arbitrary or
-pre-existing foreign run root, symlinked path, or unproven Session state is
-tolerated. Real-runtime teardown scans and cleans only the creator-owned Home,
-uses Yui's exact process/pane/artifact identity fences, verifies the Home-derived
-tmux server is absent, and refuses environment overrides that redirect
-`YUI_HOME`. The reusable annotated-resource selector separately requires an
-exact non-empty creator token plus matching `ephemeral-test` marker; a missing
-token touches nothing and is a failed cleanup outcome. **Mock Agent Session
-transport success does not prove
-provider-native acceptance** — only the Provider E2E tier can record that. See
-[docs/testing/test-tiers.md](./docs/testing/test-tiers.md) for the full contract.
+The permanent suite is intentionally one seconds-scale core smoke. It checks
+CLI startup, a normal SQLite Task path, the supported migration graph, and the
+built-in Agent Drivers. Change-specific TDD fixtures and abnormal-data repros
+are temporary development evidence and are removed when the change is complete;
+they do not accumulate as permanent regression tests. See
+[the verification policy](./docs/testing/verification-levels.md).
 
 To make user terminals use this checkout, reversibly link the user-level `yui` command:
 

package/dist/storage/migration/productionRegistry.js

@@ -1303,24 +1303,68 @@ function requireAggregateV17Snapshot(snapshot) {
 /**
  * Runtime state is now projected exclusively from canonical
  * `runtime.observation` events. Offline upgrade inventory proves there are no
- * active Runs or live Sessions, so the historical lifecycle vocabulary needs
- * no dual-read adapter after this version-only boundary.
+ * active Runs or live Sessions. A retired Task has also explicitly abandoned
+ * its runtime, so its stored runtime inconsistencies no longer block the Home;
+ * the anomalous records themselves remain available as history. Non-retired
+ * Tasks still fail closed and receive the supported retirement command.
  */
 function migrateAggregateV18ToV19(snapshot) {
     requireAggregateV18Snapshot(snapshot);
+    const schemaManifest = {
+        ...snapshot.schemaManifest,
+        aggregateSchemaVersion: RUNTIME_OBSERVATION_AGGREGATE_TO_VERSION
+    };
+    if (snapshot.state === null)
+        return { schemaManifest, state: null };
+    const tasks = asObject(snapshot.state.tasks, "state tasks");
+    const nextTasks = {};
+    for (const [taskId, rawStoredTask] of Object.entries(tasks)) {
+        const storedTask = asObject(rawStoredTask, `Task aggregate ${taskId}`);
+        const rawTask = storedTask.task;
+        const task = rawTask !== null && typeof rawTask === "object" && !Array.isArray(rawTask)
+            ? rawTask
+            : undefined;
+        if (task?.status === "retired") {
+            nextTasks[taskId] = { ...storedTask };
+            continue;
+        }
+        requireResolvableActiveRunPointers(taskId, storedTask);
+        nextTasks[taskId] = { ...storedTask };
+    }
     return {
-        schemaManifest: {
-            ...snapshot.schemaManifest,
-            aggregateSchemaVersion: RUNTIME_OBSERVATION_AGGREGATE_TO_VERSION
-        },
-        state: snapshot.state === null
-            ? null
-            : {
-                ...snapshot.state,
-                schemaVersion: RUNTIME_OBSERVATION_AGGREGATE_TO_VERSION
-            }
+        schemaManifest,
+        state: {
+            ...snapshot.state,
+            schemaVersion: RUNTIME_OBSERVATION_AGGREGATE_TO_VERSION,
+            tasks: nextTasks
+        }
     };
 }
+function requireResolvableActiveRunPointers(taskId, storedTask) {
+    if (storedTask.activeRuns === undefined)
+        return;
+    const activeRuns = asObject(storedTask.activeRuns, `activeRunPointer map ${taskId}`);
+    if (Object.keys(activeRuns).length === 0)
+        return;
+    const agentRuns = asObject(storedTask.agentRuns, `agentRun map ${taskId}`);
+    for (const [pointer, rawActiveRun] of Object.entries(activeRuns)) {
+        const activeRun = asObject(rawActiveRun, `Active run ${taskId}/${pointer}`);
+        const runId = typeof activeRun.runId === "string" ? activeRun.runId.trim() : "";
+        if (runId.length === 0) {
+            throw new Error(`Active run pointer ${taskId}/${pointer} has an invalid runId. `
+                + taskRetirementUpgradeHint(taskId));
+        }
+        if (agentRuns[runId] === undefined) {
+            throw new Error(`Active run pointer ${taskId}/${pointer} references missing agent run ${runId}. `
+                + taskRetirementUpgradeHint(taskId));
+        }
+    }
+}
+function taskRetirementUpgradeHint(taskId) {
+    return `Retire Task ${taskId} with `
+        + `\`yui task retire ${taskId} --summary "abandon inconsistent runtime state"\`, `
+        + "then retry `yui update`.";
+}
 function requireAggregateV18Snapshot(snapshot) {
     if (snapshot.schemaManifest.aggregateSchemaVersion
         !== RUNTIME_OBSERVATION_AGGREGATE_FROM_VERSION) {

package/dist/storage/upgrade/sqliteRecordMigrationTarget.js

@@ -37,7 +37,7 @@ import { planMigration } from "../migration/planner.js";
 import { AmbiguousSwitchError } from "../migration/index.js";
 import { describeActiveRuntime, homeRuntimeIsActive, inspectHomeRuntime, inspectSourceVersionState, inspectSnapshotVersionState } from "./homeMigrationTarget.js";
 import { writeSwitchProgress } from "./switchProgress.js";
-import { COMMITTED_DATABASE_FILENAME, STAGED_DATABASE_FILENAME, computeDbFamilyChecksums, computeStateFamilyChecksums, populateSqliteFromState, readStateFromSqlite } from "./sqliteStateMigration.js";
+import { COMMITTED_DATABASE_FILENAME, STAGED_DATABASE_FILENAME, copySqlitePassthroughState, computeDbFamilyChecksums, computeStateFamilyChecksums, populateSqliteFromState, readStateFromSqlite } from "./sqliteStateMigration.js";
 /** Build the SQLite-backed record-migration target. */
 export function createSqliteRecordMigrationTarget(options) {
     const home = options.home;
@@ -92,6 +92,7 @@ export function createSqliteRecordMigrationTarget(options) {
                 updatedAt: now().toISOString()
             };
             populateSqliteFromState(home, snapshot.state ?? {}, STAGED_DATABASE_FILENAME);
+            copySqlitePassthroughState(home, COMMITTED_DATABASE_FILENAME, STAGED_DATABASE_FILENAME);
         },
         rebuildDerivedState(effects) {
             // The SQLite database is fully normalised by populateSqliteFromState;

package/dist/storage/upgrade/sqliteStateMigration.js

@@ -33,6 +33,90 @@ import { CURRENT_STORED_TASK_SCHEMA_VERSION } from "../taskStore.js";
 export const STAGED_DATABASE_FILENAME = "yui.db.staged";
 /** The committed database filename. */
 export const COMMITTED_DATABASE_FILENAME = "yui.db";
+/**
+ * Preserve SQLite-owned durable state that intentionally sits outside the
+ * state.json-shaped Task aggregate snapshot. Volatile coordination locks and
+ * derived projections are rebuilt or dropped at the offline boundary.
+ */
+export function copySqlitePassthroughState(home, sourceDatabaseFilename, targetDatabaseFilename) {
+    if (sourceDatabaseFilename === targetDatabaseFilename) {
+        throw new Error("SQLite passthrough copy requires distinct source and target databases.");
+    }
+    const source = new Database(join(home, sourceDatabaseFilename), { readonly: true });
+    const target = new Database(join(home, targetDatabaseFilename));
+    try {
+        target.pragma("foreign_keys = ON");
+        target.transaction(() => {
+            mergeGlobalSequences(source, target);
+            copyTableRows(source, target, "outbox");
+            copyMailboxSignals(source, target);
+            copyTableRows(source, target, "work_item_candidates");
+            copyTableRows(source, target, "review_findings");
+            copyTableRows(source, target, "telemetry");
+            if (sqliteTableExists(source, "telemetry_aggregate")) {
+                target.exec("DELETE FROM telemetry_aggregate");
+                copyTableRows(source, target, "telemetry_aggregate");
+            }
+            copyTableRows(source, target, "session_owners");
+            copyTableRows(source, target, "resource_registry");
+            copyTableRows(source, target, "gate_artifacts");
+            copyTableRows(source, target, "gate_artifact_logs");
+        })();
+    }
+    finally {
+        source.close();
+        target.close();
+    }
+}
+function mergeGlobalSequences(source, target) {
+    if (!sqliteTableExists(source, "global_sequences"))
+        return;
+    const rows = source.prepare("SELECT name, high_water FROM global_sequences").all();
+    const merge = target.prepare(`INSERT INTO global_sequences (name, high_water) VALUES (?, ?)
+     ON CONFLICT(name) DO UPDATE SET
+       high_water = MAX(global_sequences.high_water, excluded.high_water)`);
+    for (const row of rows)
+        merge.run(row.name, row.high_water);
+}
+function copyMailboxSignals(source, target) {
+    if (!sqliteTableExists(source, "mailbox_signals"))
+        return;
+    const rows = source.prepare(`SELECT m.target_key, s.sequence, s.reason, s.ref_type, s.ref_task_id,
+            s.ref_id, s.occurred_at, s.request_id
+       FROM mailbox_signals s
+       JOIN mailboxes m ON m.mailbox_id = s.mailbox_id
+      ORDER BY m.target_key, s.sequence`).iterate();
+    const findMailbox = target.prepare("SELECT mailbox_id FROM mailboxes WHERE target_key = ?");
+    const insert = target.prepare(`INSERT INTO mailbox_signals
+       (mailbox_id, sequence, reason, ref_type, ref_task_id, ref_id, occurred_at, request_id)
+     VALUES (?, ?, ?, ?, ?, ?, ?, ?)`);
+    for (const row of rows) {
+        const mailbox = findMailbox.get(row.target_key);
+        if (mailbox === undefined) {
+            throw new Error(`SQLite migration cannot preserve signals for missing mailbox ${row.target_key}.`);
+        }
+        insert.run(mailbox.mailbox_id, row.sequence, row.reason, row.ref_type, row.ref_task_id, row.ref_id, row.occurred_at, row.request_id);
+    }
+}
+function copyTableRows(source, target, table) {
+    if (!sqliteTableExists(source, table) || !sqliteTableExists(target, table))
+        return;
+    const columns = source.prepare(`PRAGMA table_info(${quoteSqliteIdentifier(table)})`).all().map((row) => row.name);
+    if (columns.length === 0)
+        return;
+    const quotedColumns = columns.map(quoteSqliteIdentifier);
+    const rows = source.prepare(`SELECT ${quotedColumns.join(", ")} FROM ${quoteSqliteIdentifier(table)}`).iterate();
+    const insert = target.prepare(`INSERT INTO ${quoteSqliteIdentifier(table)} (${quotedColumns.join(", ")}) `
+        + `VALUES (${columns.map(() => "?").join(", ")})`);
+    for (const row of rows)
+        insert.run(...columns.map((column) => row[column]));
+}
+function sqliteTableExists(db, table) {
+    return db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(table) !== undefined;
+}
+function quoteSqliteIdentifier(value) {
+    return `"${value.replaceAll("\"", "\"\"")}"`;
+}
 // ---------------------------------------------------------------------------
 // Canonical JSON and hashing
 // ---------------------------------------------------------------------------
@@ -157,6 +241,7 @@ function tasksOf(state) {
  * sidecar database file.
  */
 export function populateSqliteFromState(home, state, databaseFilename) {
+    const retiredActiveRuns = [];
     const store = new SqliteTaskStore(home, { databaseFilename, migration: true });
     try {
         store.transaction(() => {
@@ -234,6 +319,17 @@ export function populateSqliteFromState(home, state, databaseFilename) {
                 // Active-run pointers: the document stores { schemaVersion, runId }
                 // keyed by pointer; the store derives the pointer from the Run.
                 for (const [pointer, value] of Object.entries(stored.activeRuns)) {
+                    if (stored.task.status === "retired") {
+                        retiredActiveRuns.push({
+                            taskId,
+                            pointer,
+                            value,
+                            updatedAt: typeof stored.task.updatedAt === "string"
+                                ? stored.task.updatedAt
+                                : new Date().toISOString()
+                        });
+                        continue;
+                    }
                     const run = stored.agentRuns[value.runId];
                     if (run === undefined) {
                         throw new Error(`Active run pointer ${taskId}/${pointer} references missing agent run ${value.runId}.`);
@@ -291,6 +387,33 @@ export function populateSqliteFromState(home, state, databaseFilename) {
     finally {
         store.close();
     }
+    persistRetiredActiveRunPointers(home, databaseFilename, retiredActiveRuns);
+}
+/**
+ * A retired Task is an explicit isolation boundary. Its active-run rows are
+ * retained byte-for-byte at the logical record level even when their Run is
+ * missing; normal Tasks continue through the referentially strict store path.
+ */
+function persistRetiredActiveRunPointers(home, databaseFilename, pointers) {
+    if (pointers.length === 0)
+        return;
+    const db = new Database(join(home, databaseFilename));
+    try {
+        const insert = db.prepare(`INSERT INTO active_runs (task_id, pointer, run_id, payload, updated_at)
+       VALUES (?, ?, ?, ?, ?)`);
+        db.transaction(() => {
+            for (const entry of pointers) {
+                if (typeof entry.value.runId !== "string") {
+                    throw new Error(`Retired Task active run pointer ${entry.taskId}/${entry.pointer} `
+                        + "cannot be represented because runId is not a string.");
+                }
+                insert.run(entry.taskId, entry.pointer, entry.value.runId, JSON.stringify(entry.value), entry.updatedAt);
+            }
+        })();
+    }
+    finally {
+        db.close();
+    }
 }
 /**
  * Seed `global_sequences` from the numeric suffixes of existing task and

package/i18n/README.zh-CN.md

@@ -534,8 +534,6 @@ Agent 环境变量绑定只保存进程环境变量名，不保存 secret 值；
 Yui 面向一台机器上的一个受信任本地用户。它的 Web/API 仅支持 loopback，不包含远程或多用户 Web、分布式协调、backup/import/export、trash/restore、derived index、recovery journal、runtime lease、inactivity TTL、cooldown 或 recurring schedule。
 
 持久化和调度细节见 [ARCHITECTURE.md](../ARCHITECTURE.md)。
-可复用的用户视角验收方案见
-[Operator 路由与长期任务端到端测试方案](../docs/testing/operator-routing-e2e-plan.md)。
 
 ## 本地开发
 
@@ -546,6 +544,11 @@ npm test
 npm run lint
 ```
 
+`npm test` 只保留秒级核心 smoke：CLI 启动、正常 SQLite Task、受支持迁移和内置
+Agent Driver。针对当前修改编写的 TDD、异常数据和故障复现仅作为开发期证据，需求完成后
+删除，不累积为常驻回归测试。具体约束见
+[验证策略](../docs/testing/verification-levels.md)。
+
 如需让用户终端使用当前 checkout，可逆地接管用户级 `yui` 命令：
 
 ```sh

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@zq-silk/yui",
-  "version": "0.6.3",
+  "version": "0.6.4",
   "description": "Local control plane for long-running native agent CLI sessions backed by tmux.",
   "license": "MIT",
   "private": false,