dsh-cortex-memory 0.1.0 → 0.2.0-rc.13

package/LICENSE

@@ -1,6 +1,6 @@
-MIT License
+MIT License
 
-Copyright (c) 2026 dsh-cortex-memory contributors
+Copyright (c) 2026 本鲸 / 凌寒
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
@@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+SOFTWARE.

package/README.md

@@ -1,154 +1,93 @@
-# Cortex Memory for DeepSeek Harness
+# dsh-cortex
 
-A layered, temporal, self-developing memory engine for DeepSeek Harness (DSH).
-It is designed for long-lived personal agents: memory that can be recalled,
-audited, consolidated and changed **without silently rewriting identity**.
+> 鏈哺鐨勬柊澶ц剳銆傚垎灞傘€佹椂搴忋€佸彲鑷垜宸╁浐鐨勮蹇嗗紩鎿庛€?> 鐘舵€侊細**鍙墦鍖呫€佸彲瀹夎**锛涙湰鏈哄紑鍙戝壇鏈粛鏈畨瑁咃紝涔熸病鏈夋帴 profile銆?> Windows 鐩存帴瀹夎/鍥炴粴瑙?[docs/install-windows.md](docs/install-windows.md)銆?
+`dsh-cortex` is a local-first memory engine for DeepSeek Harness. It is designed
+as a successor to the `dsh-shiori` + `dsh-shiori-companion` pair, not a patch on
+top of them.
 
-> Current public version: **0.1.0**
-> Package: `dsh-cortex-memory` (the npm name `dsh-cortex` is already taken by another project)
-> License: MIT
+## Why another memory plugin
 
-## Why
+The shiori stack is good at four things: a stable persona, a compact core,
+triggered spark cards and on-demand topics. It is weaker at temporal facts,
+episodic provenance, procedural learning, consolidation and calibrated
+forgetting. `dsh-cortex` keeps the good parts and adds the missing layers:
 
-Most memory plugins stop at "store and recall". Cortex adds four things that are
-hard to bolt on later:
-
-1. **Layered memory** — L0 identity, L1 working, L1.5 playbook, L2 episodic,
-   L3 semantic/temporal graph, L4 procedural, L5 reflection, archive.
-2. **Temporal truth** — event ledger, bitemporal facts, `strict / historical /
-   belief / associative` queries, DSR forgetting.
-3. **Gated growth** — consolidation, reflection, playbook candidates, policy
-   proposals; identity and core changes require approval and can be rolled back.
-4. **Guardian** — health audit, budgets, circuit breaker, backup/validate/restore,
-   privacy cascade, secret/injection scanning.
-
-## Install
-
-Requirements:
-
-- Node.js **>= 24** (uses the built-in `node:sqlite`)
-- DSH `>=0.2.1-rc.2 <0.4.0`
-
-### From a GitHub release tgz
-
-```powershell
-dsh plugin --profile desktop add "C:\path\to\dsh-cortex-memory-0.1.0.tgz"
-```
-
-### From npm (after publishing)
-
-```powershell
-dsh plugin --profile desktop add dsh-cortex-memory
-```
-
-The plugin owns `~/.dsh/cortex` and does not read or write another memory
-plugin's data.
-
-## Models
-
-Cortex works with **no model** by default:
-
-| Mode | What it uses | Key |
+| Layer | What it holds | Lifetime |
 |---|---|---|
-| `hash` | local lexical fallback | no |
-| `local` | OpenAI-compatible local service, default Ollama `bge-m3` | no |
-| `remote` | OpenAI-compatible `/embeddings` | **required** |
-
-For local semantic search:
-
-```powershell
-ollama pull bge-m3
-```
-
-Then in **Settings → Embedding / rerank models** choose `local`, keep
-`http://127.0.0.1:11434/v1`, model `bge-m3`, dimension `1024`.
-
-Remote mode refuses to save without an API key. Keys are stored in the local
-data directory and are never returned by the settings API.
-
-Optional rerank uses an OpenAI-compatible `/rerank` endpoint; if it is off or
-fails, fusion order is kept.
-
-## UI
-
-The client panel includes:
-
-- Self / working memory / search / timeline / graph
-- Emotion and life views
-- Guardian, evolution, audit, retention
-- **Playbook** with candidate review before activation
-- Settings: feature switches, embedding/rerank configuration, permission
-  overrides and rollback
-
-All settings changes that affect runtime take effect after a plugin reload or
-DSH restart.
-
-## CLI
-
-```powershell
-node bin/cortex.mjs --data C:\path\to\data health
-node bin/cortex.mjs --data C:\path\to\data search --query "米食"
-node bin/cortex.mjs --data C:\path\to\data export-md --dir C:\path\to\mirror
-```
-
-## Compatibility
-
-Tested in our environment:
-
-- Windows 11
-- DSH 0.2.2-rc.1
-- Node.js 26.13.0
-- Fresh `~/.dsh/cortex` data root
-- No native runtime dependencies
-
-Requirements:
-
-- **Node.js >= 24** (the engine uses `node:sqlite`)
-- DSH `>=0.2.1-rc.2 <0.4.0`
-- One profile should run only one `cortex` host entry
-
-It is **not** guaranteed to work on Node 20/22, older DSH releases, or a
-profile that already has another plugin using the same `cortex` id. This is a
-real compatibility boundary, not a marketing detail.
-
-## Testing
-
-```powershell
-npm test
-```
-
-The repository includes unit and integration tests for storage, retrieval,
-temporal facts, playbook, consolidation, guardian, API, client contract,
-installation packaging and edge cases. Public copy test result: **373 passing**.
-
-A package-level smoke test is also run from the packed tgz before release.
-
-
-## Architecture
-
-```text
-Host events -> capture -> encode -> event ledger + trace + facts + FTS
-                                      |
-prompt <- budgets <- tiered recall <- hybrid retrieval (FTS + graph + vector)
-                                      |
-                     reflection / consolidation / playbook / growth
-                                      |
-                     identity versions / policies / guardian / audit
-```
-
-See [docs/install-windows.md](docs/install-windows.md),
-[docs/models.md](docs/models.md) and
-[docs/compatibility.md](docs/compatibility.md).
-
-## Data and privacy
-
-- Default data root: `~/.dsh/cortex`
-- No telemetry and no network calls by default
-- `hash` mode is fully offline
-- `local` mode talks only to the configured local service
-- `remote` mode sends text to the configured embedding endpoint
-- Do not put API keys in prompts or issue reports
+| L0 identity | self-model, values, boundaries, stable traits | long, explicit promotion |
+| L1 working | current goals, focus, pinned context, recent turn buffer | minutes鈥揾ours |
+| L2 episodic | what happened, when, with whom, provenance, emotion | days鈥搚ears |
+| L3 semantic | entities, facts with validity windows, preferences | until invalidated |
+| L4 procedural | skills, workflows, preconditions, success/failure stats | until superseded |
+| L5 reflective | episode summaries, insights, narratives | derived, regenerable |
+| archive | cold but searchable, wake-word recall | until privacy deletion |
+
+## Engines
+
+- **event log** 鈥?append-only, hash-chained, `event_id` / `operation_id`; the
+  only source of truth.
+- **write pipeline** 鈥?scan secrets/injection 鈫?classify 鈫?dedupe 鈫?  conflict-resolve 鈫?link 鈫?persist; async jobs with idempotent operation IDs.
+- **temporal graph** 鈥?entities, facts and edges with `valid_from` /
+  `valid_to`; old facts are invalidated, never silently overwritten.
+- **hybrid retrieval** 鈥?FTS5 BM25 + optional vector + graph activation +
+  recency / importance / feedback, fused with RRF, diversified with a greedy
+  MMR-style marginal-gain pass; query filters (`kind:`, `tag:`, `after:`,
+  `importance:>`, 鈥?.
+- **DSR forgetting** 鈥?FSRS-inspired stability / difficulty / retrievability;
+  archive instead of delete, revive on successful recall.
+- **compression / retention** 鈥?archive only when a derived model covers the
+  trace, or when a reviewed retention plan says it is old/low-value; important
+  memories are never auto-archived.
+- **reflection** 鈥?episode segmentation, summaries, narrative threads,
+  duplicate/contradiction detection, optional LLM dream/summary.
+- **self model** 鈥?drives, mood, attention, appraisal, relationships,
+  initiatives, 24-category emotion/affect, identity versions.
+- **development** 鈥?maturity/stage from memory depth, feedback, calibration,
+  skills, narrative and relationships.
+- **evolution** 鈥?policy proposals 鈫?replay evaluation 鈫?explicit approval 鈫?  rollback; identity changes are separate and approval-only.
+- **guardian** 鈥?invariants, circuit breaker, budget, backup/validate/offline
+  restore, FTS repair, privacy cascade, secret/injection scanner.
+- **host runtime** 鈥?`session/event` turn capture + `user/message` /
+  `agent/pre-step` recall prefetch into the systemPrompt section.
+- **API/UI/CLI** 鈥?loopback-only HTTP API, sidebar panel, `bin/cortex.mjs`.
+
+## Status
+
+This is a source-only development tree. `npm test` uses `node:test` and the
+built-in `node:sqlite`; no native dependency and no network are required for
+the core suite. Profiles install a built tgz explicitly; this tree is not
+auto-linked into them.
+
+- **395** tests passing (unit + integration).
+- 21 model-facing tools.
+- **`cortex_as_of`**锛歴trict / historical / belief / associative銆?- **`cortex_playbook`**锛歀1.5 宸ヤ綔鍑嗗垯锛屽甫 helpful/harmful銆乿alidTo銆佽繃鏈熶笉娉ㄥ叆銆?- **`cortex_consolidate`**锛歀LM 鍐崇瓥娓呭崟 + 鏍￠獙 + `dream_runs` 鍙洖鏀捐褰曪紱鏃?LLM 鏃惰褰?skipped 骞堕浂鏀瑰姩銆?- **`cortex_search({tiered:true})`**锛歱erm/hot/cold 鍒嗗眰棰勭畻锛宑old 鍙粰绱㈠紩銆?- **寮傛娣峰悎妫€绱?*锛歚GET /search?async=true` / QA 璧?`searchAsync`锛涜繙绋?embedder 鍙储寮曪紝涓嶉樆濉炲啓銆?- 5k synthetic benchmark (needle, vector off): ~2.6-2.8k writes/s, p50 ~20-35ms; 100k migration ~2.3s. These are local numbers, not an official score.
+- Source/tests/docs 鈮?33k lines (includes `docs/book/`); source-only, not installed here.
+- Windows tgz: `dist/dsh-cortex-0.2.0-rc.11.tgz` (zero runtime deps; Node >= 24).
+
+## Design docs
+
+- [research notes](docs/research/README.md)
+- [architecture](docs/architecture.md)
+- [Anima: from memory engine to a growing individual](docs/anima.md)
+- [modules and invariants](docs/modules.md)
+- [operations / CLI / backup](docs/operations.md)
+- [security and privacy](docs/security.md)
+- [scope isolation and read-only sharing](docs/scopes.md)
+- [multimodal attachment metadata](docs/multimodal.md)
+- [benchmark](docs/benchmark.md)
+- [OpenAPI 3.1](docs/openapi.yaml)
+- [API client](src/api/client.js)
+- [UI guide](docs/ui.md)
+- [design book](docs/book/README.md)
+- [four-plugin comparison](docs/comparison-2026-11-11.md)
+- [Windows install and rollback](docs/install-windows.md)
+- [optional models](docs/models.md)
+- [shiori companion review](docs/companion-feature-review.md)
+- [ecosystem comparison 2026-11-11](docs/ecosystem-comparison-2026-11-11.md)
+- [roadmap](docs/roadmap.md)
 
 ## License
 
-MIT. See [LICENSE](LICENSE).
+MIT. External ideas are cited in the research notes; no third-party source
+code is copied into this package.
+

package/bench/ab-engine.mjs

@@ -0,0 +1,60 @@
+import { mkdtempSync, rmSync } from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { pathToFileURL } from 'node:url';
+import { generateSynthetic } from './synthetic.js';
+import { evaluateQueries, percentile } from './metrics.js';
+
+const engineRoot = process.argv[2];
+const task = process.argv[3] ?? 'paraphrase';
+const count = Number(process.argv[4] ?? 5000);
+const queryCount = Number(process.argv[5] ?? 100);
+
+if (!engineRoot) {
+  console.error('usage: node bench/ab-engine.mjs <engine-root> [needle|theme|paraphrase] [count] [queries]');
+  process.exit(2);
+}
+
+const { createCortex } = await import(pathToFileURL(path.join(engineRoot, 'src', 'service', 'cortex.js')).href);
+const dir = mkdtempSync(path.join(os.tmpdir(), 'cortex-ab-'));
+const cortex = createCortex({ config: { dataRoot: dir, dbFile: path.join(dir, 'cortex.sqlite'), contextBudget: 2200 } });
+try {
+  const dataset = generateSynthetic({ count, queries: queryCount, seed: 42, task });
+  const written = cortex.writeMany(dataset.traces.map((trace) => ({
+    text: trace.text,
+    title: trace.title,
+    tags: trace.tags,
+    importance: trace.importance,
+    explicit: true,
+    operationId: `ab:${trace.id}`,
+    now: Date.parse(trace.recordedAt),
+  })), { chunkSize: 250 });
+  const idMap = new Map();
+  for (let i = 0; i < written.length; i += 1) {
+    if (written[i].trace) idMap.set(dataset.traces[i].id, written[i].trace.id);
+  }
+  const results = [];
+  const latencies = [];
+  for (const query of dataset.queries) {
+    const start = performance.now();
+    const result = cortex.search(query.query, { limit: 10, budget: 2200 });
+    latencies.push(performance.now() - start);
+    results.push({
+      items: result.items,
+      relevant: query.relevant.map((id) => idMap.get(id)).filter(Boolean),
+    });
+  }
+  const metrics = evaluateQueries(results, { k: 10 });
+  console.log(JSON.stringify({
+    engine: engineRoot,
+    task,
+    count,
+    queries: dataset.queries.length,
+    p50Ms: Math.round((percentile(latencies, 50) ?? 0) * 100) / 100,
+    p95Ms: Math.round((percentile(latencies, 95) ?? 0) * 100) / 100,
+    ...metrics,
+  }, null, 2));
+} finally {
+  cortex.close();
+  rmSync(dir, { recursive: true, force: true });
+}

package/bench/ann-bench.mjs

@@ -0,0 +1,78 @@
+import { createAnnIndex } from '../src/vector/ann.js';
+
+function randomFactory(seed) {
+  let state = seed >>> 0;
+  return () => {
+    state = (state * 1664525 + 1013904223) >>> 0;
+    return state / 4294967296;
+  };
+}
+
+function makeVector(random, dim = 64) {
+  const value = Array.from({ length: dim }, () => random() * 2 - 1);
+  const norm = Math.sqrt(value.reduce((sum, item) => sum + item * item, 0)) || 1;
+  return value.map((item) => item / norm);
+}
+
+function cosine(a, b) {
+  let dot = 0;
+  for (let i = 0; i < a.length; i += 1) dot += a[i] * b[i];
+  return dot;
+}
+
+function percentile(values, p) {
+  const sorted = [...values].sort((a, b) => a - b);
+  if (!sorted.length) return null;
+  return sorted[Math.min(sorted.length - 1, Math.max(0, Math.ceil((p / 100) * sorted.length) - 1))];
+}
+
+const count = Number(process.argv[2] ?? 5000);
+const queries = Number(process.argv[3] ?? 100);
+const ef = Number(process.argv[4] ?? 120);
+const random = randomFactory(20261111);
+const items = Array.from({ length: count }, (_, index) => ({ id: `a${index}`, vector: makeVector(random) }));
+const queryVectors = Array.from({ length: queries }, () => makeVector(random));
+const ann = createAnnIndex({ M: 24, efConstruction: 200, efSearch: ef, seed: 20261111 });
+const buildStart = performance.now();
+ann.rebuild(items);
+const buildMs = performance.now() - buildStart;
+
+const exactTimes = [];
+const annTimes = [];
+const exactResults = [];
+const annResults = [];
+for (const query of queryVectors) {
+  const exactStart = performance.now();
+  const exact = items
+    .map((item) => ({ id: item.id, score: cosine(item.vector, query) }))
+    .sort((a, b) => b.score - a.score)
+    .slice(0, 10);
+  exactTimes.push(performance.now() - exactStart);
+  exactResults.push(exact);
+
+  const annStart = performance.now();
+  const approx = ann.search(query, 10, { ef }).slice(0, 10);
+  annTimes.push(performance.now() - annStart);
+  annResults.push(approx);
+}
+
+let hits = 0;
+let expectedTotal = 0;
+for (let index = 0; index < exactResults.length; index += 1) {
+  const expected = new Set(exactResults[index].map((item) => item.id));
+  for (const item of annResults[index]) if (expected.has(item.id)) hits += 1;
+  expectedTotal += expected.size;
+}
+
+console.log(JSON.stringify({
+  count,
+  queries,
+  dim: 64,
+  ef,
+  buildMs: Math.round(buildMs * 100) / 100,
+  recallAt10: expectedTotal ? Math.round((hits / expectedTotal) * 10000) / 10000 : null,
+  exactP50Ms: Math.round((percentile(exactTimes, 50) ?? 0) * 1000) / 1000,
+  annP50Ms: Math.round((percentile(annTimes, 50) ?? 0) * 1000) / 1000,
+  exactP95Ms: Math.round((percentile(exactTimes, 95) ?? 0) * 1000) / 1000,
+  annP95Ms: Math.round((percentile(annTimes, 95) ?? 0) * 1000) / 1000,
+}, null, 2));

package/bench/hnsw-bench.mjs

@@ -0,0 +1,79 @@
+import { createHnswIndex } from '../src/vector/hnsw.js';
+
+function randomFactory(seed) {
+  let state = seed >>> 0;
+  return () => {
+    state = (state * 1664525 + 1013904223) >>> 0;
+    return state / 4294967296;
+  };
+}
+
+function makeVector(random, dim = 64) {
+  const value = Array.from({ length: dim }, () => random() * 2 - 1);
+  const norm = Math.sqrt(value.reduce((sum, item) => sum + item * item, 0)) || 1;
+  return value.map((item) => item / norm);
+}
+
+function cosine(a, b) {
+  let dot = 0;
+  for (let i = 0; i < a.length; i += 1) dot += a[i] * b[i];
+  return dot;
+}
+
+function percentile(values, p) {
+  const sorted = [...values].sort((a, b) => a - b);
+  if (!sorted.length) return null;
+  return sorted[Math.min(sorted.length - 1, Math.max(0, Math.ceil((p / 100) * sorted.length) - 1))];
+}
+
+const count = Number(process.argv[2] ?? 5000);
+const queries = Number(process.argv[3] ?? 100);
+const ef = Number(process.argv[4] ?? 200);
+const random = randomFactory(20261111);
+const items = Array.from({ length: count }, (_, index) => ({ id: `h${index}`, vector: makeVector(random) }));
+const queryVectors = Array.from({ length: queries }, () => makeVector(random));
+const index = createHnswIndex({ M: 24, efConstruction: 200, efSearch: ef, seed: 20261111 });
+const buildStart = performance.now();
+index.rebuild(items);
+const buildMs = performance.now() - buildStart;
+
+const exactTimes = [];
+const hnswTimes = [];
+const exactResults = [];
+const hnswResults = [];
+for (const query of queryVectors) {
+  const exactStart = performance.now();
+  const exact = items
+    .map((item) => ({ id: item.id, score: cosine(item.vector, query) }))
+    .sort((a, b) => b.score - a.score)
+    .slice(0, 10);
+  exactTimes.push(performance.now() - exactStart);
+  exactResults.push(exact);
+
+  const hnswStart = performance.now();
+  const approx = index.search(query, 10, { ef }).slice(0, 10);
+  hnswTimes.push(performance.now() - hnswStart);
+  hnswResults.push(approx);
+}
+
+let hits = 0;
+let expectedTotal = 0;
+for (let index = 0; index < exactResults.length; index += 1) {
+  const expected = new Set(exactResults[index].map((item) => item.id));
+  for (const item of hnswResults[index]) if (expected.has(item.id)) hits += 1;
+  expectedTotal += expected.size;
+}
+
+console.log(JSON.stringify({
+  engine: 'hnsw',
+  count,
+  queries,
+  dim: 64,
+  ef,
+  buildMs: Math.round(buildMs * 100) / 100,
+  recallAt10: expectedTotal ? Math.round((hits / expectedTotal) * 10000) / 10000 : null,
+  exactP50Ms: Math.round((percentile(exactTimes, 50) ?? 0) * 1000) / 1000,
+  hnswP50Ms: Math.round((percentile(hnswTimes, 50) ?? 0) * 1000) / 1000,
+  exactP95Ms: Math.round((percentile(exactTimes, 95) ?? 0) * 1000) / 1000,
+  hnswP95Ms: Math.round((percentile(hnswTimes, 95) ?? 0) * 1000) / 1000,
+}, null, 2));

package/bench/metrics.js

@@ -4,6 +4,12 @@ export function recallAtK(ranked, relevant, k = 10) {
   return rel.size === 0 ? 0 : top.filter((id) => rel.has(id)).length / rel.size;
 }
 
+export function hitAtK(ranked, relevant, k = 10) {
+  const rel = new Set(relevant);
+  if (rel.size === 0) return 0;
+  return ranked.slice(0, Math.max(1, k)).some((id) => rel.has(id)) ? 1 : 0;
+}
+
 export function precisionAtK(ranked, relevant, k = 10) {
   const rel = new Set(relevant);
   const top = ranked.slice(0, Math.max(1, k));
@@ -43,6 +49,7 @@ export function evaluateQueries(results, { k = 10 } = {}) {
     const relevant = result.relevant ?? [];
     return {
       recall: recallAtK(ranked, relevant, k),
+      hit: hitAtK(ranked, relevant, k),
       precision: precisionAtK(ranked, relevant, k),
       mrr: reciprocalRank(ranked, relevant, k),
       ndcg: ndcgAtK(ranked, relevant, k),
@@ -52,6 +59,7 @@ export function evaluateQueries(results, { k = 10 } = {}) {
   return {
     n: rows.length,
     recallAtK: mean('recall'),
+    hitAtK: mean('hit'),
     precisionAtK: mean('precision'),
     mrrAtK: mean('mrr'),
     ndcgAtK: mean('ndcg'),

package/bench/migration-bench.mjs

@@ -0,0 +1,69 @@
+import { mkdtempSync, rmSync } from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { createCortex } from '../src/service/cortex.js';
+import { openDatabase, migrate, getMeta, closeDatabase, FTS_INDEX_VERSION } from '../src/store/database.js';
+import { createTraceRepository } from '../src/store/repositories/traces.js';
+import { ftsText } from '../src/memory/tokenizer.js';
+
+const count = Number(process.argv[2] ?? 5000);
+const dir = mkdtempSync(path.join(os.tmpdir(), `cortex-migrate-${count}-`));
+const dbFile = path.join(dir, 'cortex.sqlite');
+const totalStart = performance.now();
+const phases = {};
+let cortex;
+try {
+  cortex = createCortex({ config: { dataRoot: dir, dbFile, contextBudget: 2200 } });
+  console.error(`phase create done (${Math.round(performance.now() - totalStart)}ms)`);
+  const texts = [];
+  for (let i = 0; i < count; i += 1) {
+    texts.push({
+      text: i === 0 ? '白饭：升级后仍然应该能通过“米饭”的 soft 查询找到。' : `老库迁移样本 ${i}：普通记忆内容。`,
+      title: `迁移样本 #${i}`,
+      explicit: true,
+      operationId: `migrate:${count}:${i}`,
+      now: Date.UTC(2026, 0, 1) + i * 1000,
+    });
+  }
+  const writeStart = performance.now();
+  cortex.writeMany(texts, { chunkSize: 500 });
+  phases.writeMs = Math.round((performance.now() - writeStart) * 100) / 100;
+  console.error(`phase write done (${phases.writeMs}ms)`);
+  cortex.close();
+  cortex = null;
+
+  let opened = openDatabase({ dataRoot: dir, dbFile });
+  opened.db.exec('DELETE FROM traces_fts_index');
+  opened.db.prepare('DELETE FROM meta WHERE key = ?').run('fts_index_version');
+  const rows = opened.db.prepare('SELECT id, title, content, summary FROM traces').all();
+  const insert = opened.db.prepare('INSERT INTO traces_fts_index(trace_id, text) VALUES (?, ?)');
+  for (const row of rows) {
+    insert.run(row.id, ftsText(`${row.title ?? ''} ${row.content ?? ''} ${row.summary ?? ''}`));
+  }
+  closeDatabase(opened.db);
+  phases.oldIndexMs = Math.round((performance.now() - writeStart) * 100) / 100;
+  console.error(`phase old-index done (${phases.oldIndexMs}ms total from write)`);
+
+  const started = performance.now();
+  opened = openDatabase({ dataRoot: dir, dbFile });
+  const result = migrate(opened.db);
+  const elapsedMs = performance.now() - started;
+  phases.migrateMs = Math.round(elapsedMs * 100) / 100;
+  console.error(`phase migrate done (${phases.migrateMs}ms)`);
+  const softHits = createTraceRepository(opened.db).searchFts('米饭', { soft: true });
+  phases.totalMs = Math.round((performance.now() - totalStart) * 100) / 100;
+  console.log(JSON.stringify({
+    count,
+    ftsRebuilt: result.ftsRebuilt,
+    elapsedMs: Math.round(elapsedMs * 100) / 100,
+    phases,
+    ftsVersion: getMeta(opened.db, 'fts_index_version'),
+    expectedVersion: String(FTS_INDEX_VERSION),
+    softHits: softHits.length,
+    dbBytes: opened.db.prepare('PRAGMA page_count').get().page_count * opened.db.prepare('PRAGMA page_size').get().page_size,
+  }, null, 2));
+  closeDatabase(opened.db);
+} finally {
+  if (cortex) cortex.close();
+  rmSync(dir, { recursive: true, force: true });
+}

package/bench/run.js

@@ -11,11 +11,12 @@ export function runBenchmark({
   seed = 42,
   vector = process.env.CORTEX_BENCH_VECTOR === '1',
   limit = 10,
+  task = 'needle',
 } = {}) {
   const dir = mkdtempSync(path.join(os.tmpdir(), 'cortex-bench-'));
   const cortex = createCortex({ config: { dataRoot: dir, dbFile: path.join(dir, 'cortex.sqlite'), vectorEnabled: vector, contextBudget: 2200 } });
   try {
-    const dataset = generateSynthetic({ count, queries, seed });
+    const dataset = generateSynthetic({ count, queries, seed, task });
     const idMap = new Map();
     const writeStart = performance.now();
     const inputs = dataset.traces.map((trace) => ({
@@ -45,6 +46,7 @@ export function runBenchmark({
       generatedAt: dataset.generatedAt,
       seed,
       count,
+      task: dataset.task,
       queries: dataset.queries.length,
       vector,
       writeMs: Math.round(writeMs),
@@ -69,6 +71,7 @@ if (isMain) {
   const args = process.argv.slice(2);
   const count = Number(args.find((arg) => arg.startsWith('--count='))?.split('=')[1] ?? 10000);
   const queries = Number(args.find((arg) => arg.startsWith('--queries='))?.split('=')[1] ?? 100);
-  const result = runBenchmark({ count, queries, vector: args.includes('--vector') });
+  const task = args.find((arg) => arg.startsWith('--task='))?.split('=')[1] ?? 'needle';
+  const result = runBenchmark({ count, queries, task, vector: args.includes('--vector') });
   console.log(JSON.stringify(result, null, 2));
 }