driftseal 2.1.0 → 3.0.0

package/README.md

@@ -39,7 +39,8 @@ DriftSeal v2 is an outcome log rather than an intent-per-step log.
 
 ## Install
 
-DriftSeal requires Node.js 18 or newer.
+DriftSeal requires Node.js 22.13 or newer. Its derived outcome index uses the
+built-in `node:sqlite` module; no native database package is installed.
 
 ```sh
 npm install --global driftseal
@@ -155,12 +156,23 @@ has, `status`, `log`, and `lane` fall back to `main` with a warning; `begin`
 still refuses until you `lane switch main` or add the lane. `log --last N` can
 return more than N records when an open outcome sits on another lane.
 
-A derived lane index caches fold state in Git metadata (or beside a custom
-seal). Incremental rebuilds follow `indexedThrough` and `indexedLines`; a full
-rebuild happens when the log identity changes. Per-lane heads, reverse links,
-and WAL byte ranges are stored for a seek path that is not consumed yet. The
-index is reconstructable and is not committed with the log. Custom-home
-sidecars sit next to `events.jsonl`. When that directory is inside a Git
+A derived SQLite outcome index stores folded records in Git metadata (or beside
+a custom seal). It is a disposable read model; `events.jsonl` remains the only
+canonical history. Incremental sync follows `indexedThrough` and
+`indexedLines`. File identity makes unchanged hot reads constant-time, while a
+full checksum of the previously indexed WAL prefix validates every incremental
+catch-up. A source rewrite, incompatible schema, malformed indexed row, or
+SQLite read error triggers a full rebuild.
+
+`log --last N` uses a `(lane, reclaimed, ordinal)` SQLite index and reads only
+the selected outcome rows plus open outcomes from other lanes. Parked events
+are checked and folded as a targeted overlay using indexed committed event
+identities, so an open outcome does not force a committed-WAL scan. When a
+lock-free read sees a missing or stale database, DriftSeal folds the canonical
+WAL in memory instead of serving stale index data. Stored WAL byte ranges remain
+available for future projections but are not required for recent-log lookup.
+The database is reconstructable and is not committed with the log. Custom-home
+sidecars sit next to `events.jsonl`; when that directory is inside a Git
 worktree, they are listed in its `.gitignore`.
 
 ## Decisions and MADR
@@ -329,9 +341,9 @@ MADR tools, and the three migration tools. Resources are:
   DriftSeal; use `reclaim`, `unreclaim`, `lane`, and `absorb` instead of manual edits.
 - `.seal/madr/` stores numbered MADR documents.
 - `$DRIFTSEAL_HOME` replaces the `.seal` root.
-- The current lane and derived lane index live in Git metadata for a default
+- The current lane and derived SQLite outcome index live in Git metadata for a default
   repository seal, or beside a custom seal (`outcomes/.current-lane` and
-  `outcomes/.lane-index.json`). When the custom seal sits inside a Git
+  `outcomes/.outcome-index.sqlite`). When the custom seal sits inside a Git
   worktree, those sidecars are gitignored. They are reconstructable and are not
   part of the committed WAL.
 - Advisory hooks remind agents about lifecycle state but never broaden the

package/README.zh-CN.md

@@ -33,7 +33,8 @@ DriftSeal v2 从“按步骤记录 intent”改为“按交付记录 outcome”
 
 ## 安装
 
-DriftSeal 需要 Node.js 18 或更高版本。
+DriftSeal 需要 Node.js 22.13 或更高版本。派生 outcome index 使用 Node 内置的
+`node:sqlite`，不会安装 native database package。
 
 ```sh
 npm install --global driftseal
@@ -140,11 +141,19 @@ lane 不能改名或删除。`lane add` 打错的名字会一直出现在 `drift
 直到 `lane switch main` 或把该 lane 加回来。`log --last N` 在 open outcome 属于
 别的 lane 时，返回条数可以多于 N。
 
-派生的 lane index 缓存 fold 状态，放在 Git metadata（或自定义 seal 旁边）。
-增量重建跟随 `indexedThrough` 和 `indexedLines`；log 身份变化时全量重建。每条
-lane 的 head、反向链接和 WAL byte range 会写入 index，供尚未接入的 seek 路径使用。
-它可以重建，不会随 log 一起提交。自定义 home 下的 sidecar 放在 `events.jsonl`
-旁边；该目录在 Git worktree 内时，由目录里的 `.gitignore` 忽略。
+派生的 SQLite outcome index 放在 Git metadata（或自定义 seal 旁边）。它只是可以
+随时删除重建的 read model；`events.jsonl` 仍是唯一 canonical history。增量同步跟随
+`indexedThrough` 和 `indexedLines`。未变化的 hot read 用 file identity 做常量时间
+校验；增量追赶前会校验此前全部 WAL prefix 的 checksum。source 被改写、schema
+不兼容、indexed row 损坏或 SQLite read 失败时，DriftSeal 会全量重建。
+
+`log --last N` 使用 `(lane, reclaimed, ordinal)` SQLite index，只读取命中的 outcome
+row，并补上其他 lane 的 open outcome。parked event 会依据 index 中保存的 committed
+event identity 做定向 overlay，不再因此重扫 committed WAL。无锁读取遇到缺失或 stale
+database 时，会在内存中 fold canonical WAL，绝不会返回 stale index。保存的 WAL byte
+range 留给后续 projection 使用，但 recent-log lookup 不依赖它。database 可以重建，
+不会随 log 一起提交。自定义 home 下的 sidecar 放在 `events.jsonl` 旁边；该目录在 Git
+worktree 内时，由目录里的 `.gitignore` 忽略。
 
 ## Decision 与 MADR
 
@@ -301,8 +310,8 @@ resources 为：
   `absorb`，不要手改。
 - `.seal/madr/` 保存编号化 MADR。
 - `$DRIFTSEAL_HOME` 替换整个 `.seal` root。
-- 当前 lane 与派生 lane index 对默认 repo seal 存在 Git metadata 里，对自定义 seal
-  则放在旁边（`outcomes/.current-lane` 与 `outcomes/.lane-index.json`）。自定义
+- 当前 lane 与派生 SQLite outcome index 对默认 repo seal 存在 Git metadata 里，对自定义 seal
+  则放在旁边（`outcomes/.current-lane` 与 `outcomes/.outcome-index.sqlite`）。自定义
   seal 在 Git worktree 内时，这些 sidecar 会被 gitignore。它们可以重建，不是
   committed WAL 的一部分。
 - advisory hook 只提示 lifecycle 状态，不会扩大 repo 中 `AGENTS.md` 的政策边界。

package/benchmark/recent-log.js

@@ -0,0 +1,181 @@
+'use strict';
+
+const assert = require('node:assert/strict');
+const { execFileSync } = require('node:child_process');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { performance } = require('node:perf_hooks');
+const { openOutcomeIndex } = require('../lib/outcome-index-sqlite.js');
+
+const DRIFTSEAL = path.join(__dirname, '..', 'bin', 'driftseal.js');
+const quick = process.argv.includes('--quick');
+const large = process.argv.includes('--large');
+const unrelatedOutcomes = large ? 100_000 : quick ? 2_000 : 10_000;
+const laneOutcomes = large ? 1_000 : quick ? 20 : 100;
+const samples = quick ? 5 : 20;
+const root = fs.mkdtempSync(path.join(os.tmpdir(), 'driftseal-recent-log-benchmark-'));
+const home = path.join(root, 'seal');
+const outcomeDirectory = path.join(home, 'outcomes');
+const wal = path.join(outcomeDirectory, 'events.jsonl');
+const indexFile = path.join(outcomeDirectory, '.outcome-index.sqlite');
+const env = {
+  ...process.env,
+  DRIFTSEAL_HOME: home,
+  DRIFTSEAL_DECISION_HOME: path.join(home, 'madr'),
+};
+
+function runCli(args, envOverrides = {}) {
+  return execFileSync(process.execPath, [DRIFTSEAL, ...args], {
+    cwd: root,
+    encoding: 'utf8',
+    env: { ...env, ...envOverrides },
+    stdio: ['ignore', 'pipe', 'pipe'],
+    maxBuffer: 64 * 1024 * 1024,
+  });
+}
+
+function event(type, fields) {
+  return {
+    logVersion: 2,
+    schemaVersion: type === 'lane_add' || fields.lane ? 2 : 1,
+    type,
+    ...fields,
+  };
+}
+
+function outcomeEvents(id, outcome, lane, minute) {
+  const beganAt = new Date(Date.UTC(2026, 0, 1, 0, minute, 0)).toISOString();
+  const endedAt = new Date(Date.UTC(2026, 0, 1, 0, minute, 1)).toISOString();
+  return [
+    event('begin', {
+      id,
+      ts: beganAt,
+      outcome,
+      acceptance: [],
+      verify: null,
+      decisions: [],
+      ...(lane === 'main' ? {} : { lane }),
+    }),
+    event('end', {
+      id,
+      ts: endedAt,
+      status: 'abandoned',
+      note: `closed ${outcome}`,
+      verifyResult: null,
+      head: null,
+    }),
+  ];
+}
+
+function writeSyntheticWal() {
+  fs.mkdirSync(outcomeDirectory, { recursive: true });
+  const lines = [];
+  let sequence = 1;
+  for (let i = 0; i < unrelatedOutcomes; i++) {
+    const id = `2026-01-01-${String(sequence++).padStart(6, '0')}`;
+    lines.push(...outcomeEvents(id, `unrelated work ${i}`, 'main', i % 1440));
+  }
+  lines.push(
+    event('lane_add', {
+      id: 'lane:focus',
+      lane: 'focus',
+      description: 'Benchmark focus lane',
+      ts: '2026-01-02T00:00:00.000Z',
+    })
+  );
+  for (let i = 0; i < laneOutcomes; i++) {
+    const id = `2026-01-02-${String(sequence++).padStart(6, '0')}`;
+    lines.push(...outcomeEvents(id, `focus work ${i}`, 'focus', i));
+  }
+  fs.writeFileSync(wal, `${lines.map(JSON.stringify).join('\n')}\n`);
+  fs.writeFileSync(path.join(outcomeDirectory, '.current-lane'), 'focus\n');
+}
+
+function distribution(values) {
+  const sorted = [...values].sort((left, right) => left - right);
+  return {
+    p50: Number(sorted[Math.floor(sorted.length * 0.5)].toFixed(2)),
+    p95: Number(sorted[Math.min(sorted.length - 1, Math.floor(sorted.length * 0.95))].toFixed(2)),
+  };
+}
+
+function measure(action, count = samples) {
+  const durations = [];
+  let value;
+  for (let i = 0; i < count; i++) {
+    const started = performance.now();
+    value = action();
+    durations.push(performance.now() - started);
+  }
+  return { ...distribution(durations), value };
+}
+
+try {
+  writeSyntheticWal();
+  const cold = measure(() => runCli(['log', '--last', '3']), 1);
+  const hot = measure(() => runCli(['log', '--last', '3']));
+  const fullFold = measure(
+    () =>
+      runCli(['log', '--last', '3'], {
+        _DRIFTSEAL_TEST_DISABLE_OUTCOME_INDEX: '1',
+      }),
+    quick ? 1 : 3
+  );
+  assert.equal(hot.value, fullFold.value);
+  assert.match(hot.value, new RegExp(`focus work ${laneOutcomes - 1}`));
+  assert.doesNotMatch(hot.value, /unrelated work/);
+
+  const index = openOutcomeIndex(indexFile, { readOnly: true });
+  const direct = measure(() => index.queryRecent('focus', 3));
+  const queryPlan = index.explainRecent().map((row) => row.detail);
+  index.close();
+  assert.equal(direct.value.length, 3);
+  assert.ok(
+    queryPlan.some((detail) => /outcomes_lane_visible_ordinal/.test(detail)),
+    `expected recent query index in plan: ${queryPlan.join('; ')}`
+  );
+
+  const nextId = `2026-01-03-${String(unrelatedOutcomes + laneOutcomes + 1).padStart(6, '0')}`;
+  fs.appendFileSync(
+    wal,
+    `${outcomeEvents(nextId, 'incremental focus work', 'focus', 0)
+      .map(JSON.stringify)
+      .join('\n')}\n`
+  );
+  const incremental = measure(() => runCli(['log', '--last', '3']), 1);
+  assert.match(incremental.value, /incremental focus work/);
+
+  const startup = measure(() =>
+    execFileSync(process.execPath, ['-e', ''], {
+      cwd: root,
+      stdio: 'ignore',
+    })
+  );
+  const stats = {
+    mode: large ? 'large' : quick ? 'quick' : 'default',
+    node: process.version,
+    platform: `${process.platform}-${process.arch}`,
+    outcomes: {
+      unrelated: unrelatedOutcomes,
+      currentLane: laneOutcomes,
+    },
+    samples,
+    bytes: {
+      wal: fs.statSync(wal).size,
+      sqlite: fs.statSync(indexFile).size,
+    },
+    milliseconds: {
+      processStartup: { p50: startup.p50, p95: startup.p95 },
+      coldBuildAndRecent: { p50: cold.p50, p95: cold.p95 },
+      hotRecentCli: { p50: hot.p50, p95: hot.p95 },
+      hotRecentQuery: { p50: direct.p50, p95: direct.p95 },
+      fullFoldRecent: { p50: fullFold.p50, p95: fullFold.p95 },
+      incrementalAndRecent: { p50: incremental.p50, p95: incremental.p95 },
+    },
+    queryPlan,
+  };
+  process.stdout.write(`${JSON.stringify(stats, null, 2)}\n`);
+} finally {
+  fs.rmSync(root, { recursive: true, force: true });
+}