claude-mem-lite 2.17.0 → 2.18.0

package/.claude-plugin/marketplace.json

@@ -10,7 +10,7 @@
   "plugins": [
     {
       "name": "claude-mem-lite",
-      "version": "2.17.0",
+      "version": "2.18.0",
       "source": "./",
       "description": "Lightweight persistent memory system for Claude Code — FTS5 search, episode batching, error-triggered recall"
     }

package/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-mem-lite",
-  "version": "2.17.0",
+  "version": "2.18.0",
   "description": "Lightweight persistent memory system for Claude Code — FTS5 search, episode batching, error-triggered recall",
   "author": {
     "name": "sdsrss"

package/README.md

@@ -53,7 +53,7 @@ The original sends **everything to the LLM and hopes it filters well**. claude-m
 ## Features
 
 - **Automatic capture** -- Hooks into Claude Code lifecycle (PostToolUse, SessionStart, Stop, UserPromptSubmit) to record observations without manual effort
-- **FTS5 search** -- BM25-ranked full-text search across observations, session summaries, and user prompts with importance weighting
+- **Hybrid search** -- FTS5 BM25 + TF-IDF vector cosine similarity, merged via Reciprocal Rank Fusion (RRF). FTS5 handles keyword matching; 512-dim TF-IDF vectors capture semantic similarity for recall beyond exact terms
 - **Timeline browsing** -- Navigate observations chronologically with anchor-based context windows
 - **Episode batching** -- Groups related file operations into coherent episodes before LLM encoding
 - **Error-triggered recall** -- Automatically searches memory when Bash errors occur, surfacing relevant past fixes
@@ -67,7 +67,10 @@ The original sends **everything to the LLM and hopes it filters well**. claude-m
 - **Read file tracking** -- Tracks files read during sessions for richer episode context
 - **Zero data loss** -- If LLM fails, observations are saved with degraded (inferred) metadata instead of being discarded
 - **Two-tier dedup** -- Jaccard similarity (5-minute window) + MinHash signatures (7-day cross-session window) prevent duplicates
-- **Synonym expansion** -- Abbreviations like `K8s`, `DB`, `auth` automatically expand to full forms in FTS5 search (48+ pairs)
+- **Synonym expansion** -- Abbreviations like `K8s`, `DB`, `auth` automatically expand to full forms in FTS5 search (100+ pairs including CJK↔EN cross-language mappings)
+- **CJK synonym extraction** -- Unsegmented Chinese text is scanned for known vocabulary words (数据库→database, 搜索→search, etc.) enabling cross-language memory recall
+- **Stop-word filtering** -- English stop words filtered from both TF-IDF vocabulary (reclaiming ~18% of vector dimensions) and FTS queries (preventing false negatives from noise terms like "how", "the", "does")
+- **Persisted vocabulary** -- TF-IDF vocabulary persisted to `vocab_state` table, preventing vector staleness when document frequencies shift. Vectors stay valid until explicit rebuild
 - **Pseudo-relevance feedback (PRF)** -- Top results seed expansion queries for broader recall
 - **Concept co-occurrence** -- Shared concepts across observations expand search to related topics
 - **Context-aware re-ranking** -- Active file overlap boosts relevance (exact match + directory-level half-weight)
@@ -88,6 +91,8 @@ The original sends **everything to the LLM and hopes it filters well**. claude-m
 - **Exponential recency decay** -- Type-differentiated half-lives (decisions: 90d, discoveries: 60d, bugfixes: 14d, changes: 7d) consistently applied in all ranking paths
 - **Prompt-time memory injection** -- UserPromptSubmit hook automatically searches and injects relevant past observations with recency and importance weighting
 - **Dual injection dedup** -- `user-prompt-search.js` and `handleUserPrompt` coordinate via temp file to prevent duplicate memory injection
+- **Result-dedup cooldown** -- User-prompt memory injection uses result-overlap detection (>80% ID overlap → skip) instead of time-based cooldown, allowing topic switches within seconds while preventing redundant injections
+- **OR query fallback** -- When AND-joined FTS5 queries return zero results, automatically relaxes to OR-joined queries for broader recall (applied in both user-prompt-search and hook-memory paths)
 - **Configurable LLM model** -- Switch between Haiku (fast/cheap) and Sonnet (deeper analysis) via `CLAUDE_MEM_MODEL` env var
 - **DB auto-recovery** -- Detects and cleans corrupted WAL/SHM files on startup; periodic WAL checkpoints prevent unbounded growth
 - **Schema auto-migration** -- Idempotent `ALTER TABLE` migrations run on every startup, safely adding new columns and indexes without data loss
@@ -202,7 +207,7 @@ rm -rf ~/claude-mem-lite/   # pre-v0.5 unhidden (if not auto-moved)
 | `mem_stats` | View statistics: counts, type distribution, top projects, daily activity. |
 | `mem_delete` | Delete observations by ID with preview/confirm workflow. FTS5 cleanup is automatic. |
 | `mem_compress` | Compress old low-value observations into weekly summaries to reduce noise. |
-| `mem_maintain` | Memory maintenance: scan for duplicates/stale/broken items, then execute cleanup/dedup operations. |
+| `mem_maintain` | Memory maintenance: scan for duplicates/stale/broken items, then execute cleanup/dedup/rebuild_vectors operations. |
 | `mem_registry` | Manage resource registry: search for skills/agents by need, list resources, view stats, import/remove tools, reindex. |
 
 ### Skill Commands (in Claude Code chat)
@@ -260,6 +265,16 @@ project, type, session_id, working_on, completed, unfinished,
 key_files, key_decisions, match_keywords, created_at_epoch
 ```
 
+**observation_vectors** -- TF-IDF vector embeddings for hybrid search
+```
+observation_id, vector (BLOB Float32Array), vocab_version, created_at_epoch
+```
+
+**vocab_state** -- Persisted TF-IDF vocabulary for stable vector indexing
+```
+term, term_index, idf, version, created_at_epoch
+```
+
 FTS5 indexes: `observations_fts` (title, subtitle, narrative, text, facts, concepts, lesson_learned), `session_summaries_fts`, `user_prompts_fts`
 
 ## How It Works
@@ -405,7 +420,9 @@ claude-mem-lite/
   hook-semaphore.mjs   # LLM concurrency control: file-based semaphore for background workers
   schema.mjs           # Database schema: single source of truth for tables, migrations, FTS5
   tool-schemas.mjs     # Shared Zod schemas for MCP tool validation
-  utils.mjs            # Shared utilities: FTS5 query building, BM25 weight constants, MinHash dedup, secret scrubbing
+  tfidf.mjs            # TF-IDF vector engine: tokenization, vocabulary building, vector computation, cosine similarity, RRF merge
+  tier.mjs             # Temporal tier system: activity-based time window classification
+  utils.mjs            # Shared utilities: FTS5 query building, BM25 weight constants, MinHash dedup, secret scrubbing, CJK synonym extraction
   # Resource registry
   registry.mjs         # Resource registry DB: schema, CRUD, FTS5, invocation tracking
   registry-retriever.mjs # FTS5 retrieval with synonym expansion and composite scoring

package/hook-llm.mjs

@@ -27,6 +27,11 @@ function buildFtsTextField(obs) {
   return { conceptsText, factsText, textField: [conceptsText, factsText, aliasesText, bigramText].filter(Boolean).join(' ') };
 }
 
+/**
+ * Save an observation to the database with three-tier dedup.
+ * @returns {number|null} The saved observation ID, or null if deduped.
+ *   Throws on DB error (callers should catch if needed).
+ */
 export function saveObservation(obs, projectOverride, sessionIdOverride, externalDb) {
   const db = externalDb || openDb();
   if (!db) return null;
@@ -41,7 +46,7 @@ export function saveObservation(obs, projectOverride, sessionIdOverride, externa
       VALUES (?, ?, ?, ?, ?, 'active')
     `).run(sessionId, sessionId, project, now.toISOString(), now.getTime());
 
-    // Three-tier dedup
+    // Three-tier dedup — returns null (not throw) for dedup hits
     // Tier 1 (fast): 5-min Jaccard on titles
     const fiveMinAgo = now.getTime() - DEDUP_WINDOW_MS;
     const recent = db.prepare(`
@@ -51,7 +56,7 @@ export function saveObservation(obs, projectOverride, sessionIdOverride, externa
     `).all(project, fiveMinAgo);
 
     if (obs.title && recent.some(r => jaccardSimilarity(r.title, obs.title) > 0.7)) {
-      return null;
+      return null; // dedup: Jaccard title match
     }
 
     // Tier 1.5: Extended title dedup for low-signal degraded titles
@@ -68,7 +73,7 @@ export function saveObservation(obs, projectOverride, sessionIdOverride, externa
         WHERE project = ? AND title = ? AND created_at_epoch > ? AND created_at_epoch <= ?
         LIMIT 1
       `).get(project, obs.title, sevenDaysAgo, fiveMinAgo);
-      if (exactDup) return null;
+      if (exactDup) return null; // dedup: exact title match
       // Phase 2: Jaccard similarity for near-duplicates (3-day window)
       const extRecent = db.prepare(`
         SELECT title FROM observations
@@ -76,7 +81,7 @@ export function saveObservation(obs, projectOverride, sessionIdOverride, externa
         ORDER BY created_at_epoch DESC LIMIT 60
       `).all(project, threeDaysAgo, fiveMinAgo);
       if (extRecent.some(r => jaccardSimilarity(r.title, obs.title) > 0.85)) {
-        return null;
+        return null; // dedup: low-signal Jaccard match
       }
     }
 
@@ -91,7 +96,7 @@ export function saveObservation(obs, projectOverride, sessionIdOverride, externa
       `).all(project, sevenDaysAgo);
 
       if (recentSigs.some(r => estimateJaccardFromMinHash(minhashSig, r.minhash_sig) > 0.8)) {
-        return null;
+        return null; // dedup: MinHash similarity match
       }
     }
 
@@ -117,6 +122,16 @@ export function saveObservation(obs, projectOverride, sessionIdOverride, externa
     );
     const savedId = Number(result.lastInsertRowid);
 
+    // Populate observation_files junction table (non-critical)
+    if (savedId && obs.files && obs.files.length > 0) {
+      try {
+        const insertFile = db.prepare('INSERT OR IGNORE INTO observation_files (obs_id, filename) VALUES (?, ?)');
+        for (const f of obs.files) {
+          if (typeof f === 'string' && f.length > 0) insertFile.run(savedId, f);
+        }
+      } catch (e) { debugCatch(e, 'saveObservation-obsFiles'); }
+    }
+
     // Write TF-IDF vector (non-critical)
     try {
       const vocab = getVocabulary(db);

package/hook-memory.mjs

@@ -1,7 +1,7 @@
 // claude-mem-lite — Semantic Memory Injection
 // Search past observations for relevant memories to inject as context at user-prompt time.
 
-import { sanitizeFtsQuery, debugCatch, OBS_BM25 } from './utils.mjs';
+import { sanitizeFtsQuery, relaxFtsQueryToOr, debugCatch, OBS_BM25 } from './utils.mjs';
 
 const MAX_MEMORY_INJECTIONS = 3;
 const MEMORY_LOOKBACK_MS = 60 * 86400000; // 60 days
@@ -44,13 +44,21 @@ export function searchRelevantMemories(db, userPrompt, project, excludeIds = [])
       ORDER BY ${OBS_BM25}
       LIMIT 10
     `);
-    const rows = selectStmt.all(ftsQuery, project, cutoff);
+    let rows = selectStmt.all(ftsQuery, project, cutoff);
+
+    // OR fallback when AND returns nothing
+    if (rows.length === 0) {
+      const orQuery = relaxFtsQueryToOr(ftsQuery);
+      if (orQuery) {
+        try { rows = selectStmt.all(orQuery, project, cutoff); } catch {}
+      }
+    }
 
     // Phase 2: Cross-project search for high-value decisions/discoveries
     // These are transferable insights (debugging patterns, architectural reasons, gotchas)
     let crossRows = [];
     try {
-      crossRows = db.prepare(`
+      const crossStmt = db.prepare(`
         SELECT o.id, o.type, o.title, o.importance, o.lesson_learned, o.project,
                ${OBS_BM25} as relevance
         FROM observations_fts
@@ -64,7 +72,14 @@ export function searchRelevantMemories(db, userPrompt, project, excludeIds = [])
           AND o.superseded_at IS NULL
         ORDER BY ${OBS_BM25}
         LIMIT 5
-      `).all(ftsQuery, project, cutoff);
+      `);
+      crossRows = crossStmt.all(ftsQuery, project, cutoff);
+      if (crossRows.length === 0) {
+        const orQuery = relaxFtsQueryToOr(ftsQuery);
+        if (orQuery) {
+          try { crossRows = crossStmt.all(orQuery, project, cutoff); } catch {}
+        }
+      }
     } catch (e) { debugCatch(e, 'crossProjectSearch'); }
 
     // Merge and score: same-project full weight, cross-project 0.7x
@@ -117,22 +132,20 @@ export function recallForFile(db, filePath, project) {
     const cutoff = Date.now() - FILE_RECALL_LOOKBACK_MS;
     // Escape SQL LIKE wildcards in filename to prevent injection
     const escaped = basename.replace(/%/g, '\\%').replace(/_/g, '\\_');
-    // Match both full paths (/path/to/file.mjs) and basename-only entries ("file.mjs")
-    // Two patterns avoid false positives: %/file.mjs"% won't match /webapp.mjs
-    const pathPattern = `%/${escaped}"%`;
-    const namePattern = `%"${escaped}"%`;
+    const likePattern = `%${escaped}`;
     const rows = db.prepare(`
-      SELECT id, type, title, importance, lesson_learned
-      FROM observations
-      WHERE project = ?
-        AND importance >= 2
-        AND COALESCE(compressed_into, 0) = 0
-        AND superseded_at IS NULL
-        AND created_at_epoch > ?
-        AND (files_modified LIKE ? ESCAPE '\\' OR files_modified LIKE ? ESCAPE '\\')
-      ORDER BY created_at_epoch DESC
+      SELECT DISTINCT o.id, o.type, o.title, o.importance, o.lesson_learned
+      FROM observations o
+      JOIN observation_files of2 ON of2.obs_id = o.id
+      WHERE o.project = ?
+        AND o.importance >= 2
+        AND COALESCE(o.compressed_into, 0) = 0
+        AND o.superseded_at IS NULL
+        AND o.created_at_epoch > ?
+        AND (of2.filename = ? OR of2.filename LIKE ? ESCAPE '\\')
+      ORDER BY o.created_at_epoch DESC
       LIMIT ?
-    `).all(project, cutoff, pathPattern, namePattern, MAX_FILE_RECALL);
+    `).all(project, cutoff, filePath, likePattern, MAX_FILE_RECALL);
     const now = Date.now();
     const updateStmt = db.prepare('UPDATE observations SET access_count = COALESCE(access_count, 0) + 1, last_accessed_at = ? WHERE id = ?');
     for (const r of rows) updateStmt.run(now, r.id);

package/hook.mjs

@@ -32,7 +32,7 @@ import { searchRelevantMemories, recallForFile } from './hook-memory.mjs';
 import { buildAndSaveHandoff, detectContinuationIntent, renderHandoffInjection, extractUnfinishedSummary } from './hook-handoff.mjs';
 import { checkForUpdate } from './hook-update.mjs';
 import { SKIP_TOOLS, SKIP_PREFIXES } from './skip-tools.mjs';
-import { buildVocabulary } from './tfidf.mjs';
+import { getVocabulary } from './tfidf.mjs';
 
 // Prevent recursive hooks from background claude -p calls
 // Background workers (llm-episode, llm-summary) are exempt — they're ours
@@ -719,8 +719,8 @@ async function handleSessionStart() {
     // CLAUDE.md: slim (summary + handoff state — observations already in stdout)
     updateClaudeMd([...summaryLines, ...handoffLines].join('\n'));
 
-    // Pre-build TF-IDF vocabulary cache for this session
-    try { buildVocabulary(db); } catch (e) { debugCatch(e, 'session-start-vocab'); }
+    // Pre-load TF-IDF vocabulary cache for this session (from DB, ~1ms)
+    try { getVocabulary(db); } catch (e) { debugCatch(e, 'session-start-vocab'); }
 
     // Auto-update check (24h throttle, 3s timeout, silent on failure)
     // Fire-and-forget: don't block SessionStart for up to 3s network timeout

package/install.mjs

@@ -206,6 +206,7 @@ async function install() {
     'registry.mjs', 'registry-scanner.mjs', 'registry-indexer.mjs',
     'registry-retriever.mjs', 'resource-discovery.mjs',
     'install-metadata.mjs', 'mem-cli.mjs', 'tier.mjs', 'tfidf.mjs',
+    'nlp.mjs', 'scoring-sql.mjs', 'stop-words.mjs',
   ];
 
   if (IS_DEV) {

package/mem-cli.mjs

@@ -5,7 +5,7 @@
 import { ensureDb, DB_PATH } from './schema.mjs';
 import { sanitizeFtsQuery, relaxFtsQueryToOr, truncate, typeIcon, inferProject, jaccardSimilarity, computeMinHash, scrubSecrets, cjkBigrams, OBS_BM25, TYPE_DECAY_CASE, getCurrentBranch } from './utils.mjs';
 import { TIER_CASE_SQL, tierSqlParams } from './tier.mjs';
-import { getVocabulary, computeVector } from './tfidf.mjs';
+import { getVocabulary, computeVector, vectorSearch, rrfMerge, VECTOR_SCAN_LIMIT } from './tfidf.mjs';
 import { basename, join } from 'path';
 import { readFileSync } from 'fs';
 
@@ -147,7 +147,7 @@ function searchFts(db, ftsQuery, { type, project, limit, dateFrom, dateTo, minIm
   const params = [...whereParams, ...orderParams, limit];
 
   // Scoring aligned with server.mjs: BM25 × type-decay × project_boost × importance × access_bonus
-  return db.prepare(`
+  const ftsRows = db.prepare(`
     SELECT o.id, o.type, o.title, o.subtitle, o.created_at, o.lesson_learned
     FROM observations_fts
     JOIN observations o ON observations_fts.rowid = o.id
@@ -159,6 +159,43 @@ function searchFts(db, ftsQuery, { type, project, limit, dateFrom, dateTo, minIm
       * (1.0 + 0.1 * LN(1 + COALESCE(o.access_count, 0)))
     LIMIT ?
   `).all(...params);
+
+  // Hybrid: vector search + RRF merge (best-effort)
+  try {
+    const vocab = getVocabulary(db);
+    if (vocab) {
+      const queryText = ftsQuery.replace(/['"()]/g, ' ');
+      const queryVec = computeVector(queryText, vocab);
+      if (queryVec) {
+        const vecResults = vectorSearch(db, queryVec, {
+          project: project || null,
+          vocabVersion: vocab.version,
+          limit: VECTOR_SCAN_LIMIT,
+        });
+        if (vecResults.length > 0 && ftsRows.length > 0) {
+          const rrfRanking = rrfMerge(ftsRows, vecResults);
+          const rowMap = new Map(ftsRows.map(r => [r.id, r]));
+          for (const vr of vecResults) {
+            if (!rowMap.has(vr.id)) {
+              const obs = db.prepare('SELECT id, type, title, subtitle, created_at, lesson_learned FROM observations WHERE id = ?').get(vr.id);
+              if (obs) rowMap.set(vr.id, obs);
+            }
+          }
+          return rrfRanking
+            .filter(rr => rowMap.has(rr.id))
+            .map(rr => rowMap.get(rr.id))
+            .slice(0, limit);
+        } else if (vecResults.length > 0 && ftsRows.length === 0) {
+          return vecResults
+            .map(vr => db.prepare('SELECT id, type, title, subtitle, created_at, lesson_learned FROM observations WHERE id = ?').get(vr.id))
+            .filter(Boolean)
+            .slice(0, limit);
+        }
+      }
+    }
+  } catch { /* vector search is best-effort */ }
+
+  return ftsRows;
 }
 
 function cmdRecent(db, args) {
@@ -203,15 +240,18 @@ function cmdRecall(db, args) {
   const filename = basename(file);
   const limit = parseInt(flags.limit, 10) || 10;
 
-  // Search both files_modified and files_read for the filename
+  // Search via observation_files junction table for indexed filename lookups
+  const escaped = filename.replace(/%/g, '\\%').replace(/_/g, '\\_');
+  const likePattern = `%${escaped}`;
   const rows = db.prepare(`
-    SELECT id, type, title, lesson_learned, created_at
-    FROM observations
-    WHERE COALESCE(compressed_into, 0) = 0
-      AND (files_modified LIKE ? OR files_read LIKE ?)
-    ORDER BY created_at_epoch DESC
+    SELECT DISTINCT o.id, o.type, o.title, o.lesson_learned, o.created_at
+    FROM observations o
+    JOIN observation_files of2 ON of2.obs_id = o.id
+    WHERE COALESCE(o.compressed_into, 0) = 0
+      AND (of2.filename = ? OR of2.filename LIKE ? ESCAPE '\\')
+    ORDER BY o.created_at_epoch DESC
     LIMIT ?
-  `).all(`%${filename}%`, `%${filename}%`, limit);
+  `).all(filename, likePattern, limit);
 
   if (rows.length === 0) {
     out(`[mem] No history for "${filename}"`);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-mem-lite",
-  "version": "2.17.0",
+  "version": "2.18.0",
   "description": "Lightweight persistent memory system for Claude Code",
   "type": "module",
   "engines": {

package/registry-retriever.mjs

@@ -2,6 +2,7 @@
 // Tier 2 of the 3-tier dispatch intelligence architecture
 
 import { debugCatch } from './utils.mjs';
+import { BASE_STOP_WORDS } from './stop-words.mjs';
 
 // ─── Domain Synonyms ─────────────────────────────────────────────────────────
 
@@ -227,16 +228,8 @@ export function buildEnhancedQuery(signals) {
  * @returns {string|null} FTS5 query string or null
  */
 const TEXT_QUERY_STOP_WORDS = new Set([
-  'the', 'a', 'an', 'is', 'are', 'was', 'were', 'be', 'been', 'being',
-  'have', 'has', 'had', 'do', 'does', 'did', 'will', 'would', 'could',
-  'should', 'may', 'might', 'can', 'shall', 'to', 'of', 'in', 'for',
-  'on', 'with', 'at', 'by', 'from', 'as', 'into', 'about', 'between',
-  'after', 'before', 'above', 'below', 'and', 'or', 'but', 'not', 'no',
-  'this', 'that', 'these', 'those', 'it', 'its', 'my', 'your', 'his',
-  'her', 'our', 'their', 'me', 'him', 'us', 'them', 'i', 'you', 'he',
-  'she', 'we', 'they', 'what', 'which', 'who', 'when', 'where', 'how',
-  'all', 'each', 'every', 'both', 'few', 'more', 'most', 'other', 'some',
-  'such', 'than', 'too', 'very', 'just', 'also', 'then', 'so', 'if',
+  ...BASE_STOP_WORDS,
+  // CJK stop words (particles, pronouns, common verbs)
   '的', '了', '是', '在', '我', '有', '和', '就', '不', '人', '都',
   '一', '一个', '上', '也', '这', '那', '你', '他', '她', '它', '们',
   '把', '让', '给', '用', '来', '去', '做', '说', '要', '会', '能',

package/schema.mjs

@@ -12,6 +12,9 @@ export const DB_DIR = process.env.CLAUDE_MEM_DIR || join(homedir(), '.claude-mem
 export const DB_PATH = join(DB_DIR, 'claude-mem-lite.db');
 export const REGISTRY_DB_PATH = join(DB_DIR, 'resource-registry.db');
 
+// Increment when schema changes (tables, columns, indexes, FTS, migrations)
+export const CURRENT_SCHEMA_VERSION = 18;
+
 const CORE_SCHEMA = `
   CREATE TABLE IF NOT EXISTS sdk_sessions (
     id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -116,6 +119,12 @@ const MIGRATIONS = [
  * The DB should have foreign_keys OFF before calling (enabled after dedup migration).
  */
 export function initSchema(db) {
+  // Fast path: skip all migrations if schema is already at current version
+  try {
+    const row = db.prepare('SELECT version FROM schema_version LIMIT 1').get();
+    if (row && row.version === CURRENT_SCHEMA_VERSION) return db;
+  } catch { /* table may not exist yet */ }
+
   // Create core tables
   db.exec(CORE_SCHEMA);
 
@@ -136,23 +145,21 @@ export function initSchema(db) {
       GROUP BY memory_session_id HAVING cnt > 1
     `).all();
 
-    if (dupes.length > 0) {
-      const dedup = db.transaction(() => {
-        for (const { memory_session_id } of dupes) {
-          const rows = db.prepare(`
-            SELECT s.id FROM sdk_sessions s
-            WHERE s.memory_session_id = ?
-            ORDER BY s.id ASC
-          `).all(memory_session_id);
-          for (let i = 1; i < rows.length; i++) {
-            db.prepare('DELETE FROM sdk_sessions WHERE id = ?').run(rows[i].id);
-          }
+    // Atomic: dedup + create unique index in one transaction
+    const dedupAndIndex = db.transaction(() => {
+      for (const { memory_session_id } of dupes) {
+        const rows = db.prepare(`
+          SELECT s.id FROM sdk_sessions s
+          WHERE s.memory_session_id = ?
+          ORDER BY s.id ASC
+        `).all(memory_session_id);
+        for (let i = 1; i < rows.length; i++) {
+          db.prepare('DELETE FROM sdk_sessions WHERE id = ?').run(rows[i].id);
         }
-      });
-      dedup();
-    }
-
-    db.exec(`CREATE UNIQUE INDEX IF NOT EXISTS idx_sess_memory_sid ON sdk_sessions(memory_session_id)`);
+      }
+      db.exec(`CREATE UNIQUE INDEX IF NOT EXISTS idx_sess_memory_sid ON sdk_sessions(memory_session_id)`);
+    });
+    dedupAndIndex();
   }
   db.pragma('foreign_keys = ON');
 
@@ -190,6 +197,45 @@ export function initSchema(db) {
     }
   } catch { /* non-critical */ }
 
+  // Observation files junction table for normalized file lookups (replaces LIKE scans on files_modified JSON)
+  db.exec(`
+    CREATE TABLE IF NOT EXISTS observation_files (
+      obs_id INTEGER NOT NULL REFERENCES observations(id) ON DELETE CASCADE,
+      filename TEXT NOT NULL,
+      UNIQUE(obs_id, filename)
+    )
+  `);
+  db.exec(`CREATE INDEX IF NOT EXISTS idx_obsfiles_filename ON observation_files(filename)`);
+
+  // Data migration: populate observation_files from existing observations.files_modified JSON
+  // Only runs once: when observation_files is empty but observations has rows with files_modified
+  try {
+    const obsFilesCount = db.prepare('SELECT COUNT(*) as c FROM observation_files').get().c;
+    if (obsFilesCount === 0) {
+      const obsWithFiles = db.prepare(
+        `SELECT id, files_modified FROM observations WHERE files_modified IS NOT NULL AND files_modified != '[]'`
+      ).all();
+      if (obsWithFiles.length > 0) {
+        const migrateFiles = db.transaction(() => {
+          const insertFile = db.prepare('INSERT OR IGNORE INTO observation_files (obs_id, filename) VALUES (?, ?)');
+          for (const row of obsWithFiles) {
+            try {
+              const files = JSON.parse(row.files_modified);
+              if (Array.isArray(files)) {
+                for (const f of files) {
+                  if (typeof f === 'string' && f.length > 0) {
+                    insertFile.run(row.id, f);
+                  }
+                }
+              }
+            } catch { /* skip malformed JSON */ }
+          }
+        });
+        migrateFiles();
+      }
+    }
+  } catch { /* non-critical — migration can retry on next open */ }
+
   // Observation vectors table for TF-IDF vector search
   db.exec(`
     CREATE TABLE IF NOT EXISTS observation_vectors (
@@ -201,6 +247,18 @@ export function initSchema(db) {
     )
   `);
 
+  // Persisted vocabulary for stable TF-IDF vector indexing
+  db.exec(`
+    CREATE TABLE IF NOT EXISTS vocab_state (
+      term TEXT NOT NULL,
+      term_index INTEGER NOT NULL,
+      idf REAL NOT NULL,
+      version TEXT NOT NULL,
+      created_at_epoch INTEGER NOT NULL
+    )
+  `);
+  db.exec('CREATE INDEX IF NOT EXISTS idx_vocab_state_version ON vocab_state(version)');
+
   // Project name normalization: migrate short names ("mem") to canonical form ("projects--mem")
   // Strategy: exact suffix match first, then substring match for package-name aliases
   // Idempotent: only runs when short-name records exist
@@ -242,6 +300,11 @@ export function initSchema(db) {
     }
   } catch { /* non-critical — normalization can retry on next open */ }
 
+  // Record schema version for fast-path on subsequent calls
+  db.exec('CREATE TABLE IF NOT EXISTS schema_version (version INTEGER NOT NULL)');
+  db.exec('DELETE FROM schema_version');
+  db.prepare('INSERT INTO schema_version (version) VALUES (?)').run(CURRENT_SCHEMA_VERSION);
+
   return db;
 }
 

package/scripts/user-prompt-search.js

@@ -5,70 +5,15 @@
 
 import { ensureDb } from '../schema.mjs';
 import { sanitizeFtsQuery, relaxFtsQueryToOr, truncate, typeIcon, inferProject, OBS_BM25, TYPE_DECAY_CASE } from '../utils.mjs';
-import { statSync, writeFileSync } from 'fs';
+import { writeFileSync, readFileSync } from 'fs';
+import { shouldSkip, detectIntent, shouldSkipByDedup, extractFiles, DEDUP_STALE_MS } from './prompt-search-utils.mjs';
 
 // ─── Constants ──────────────────────────────────────────────────────────────
 
-const COOLDOWN_FILE = `/tmp/.claude-mem-prompt-ctx-${inferProject()}`;
 const INJECTED_IDS_FILE = `/tmp/.claude-mem-injected-${inferProject()}`;
-const COOLDOWN_MS = 60_000;
 const MAX_RESULTS = 5;
 const LOOKBACK_MS = 60 * 86400000; // 60 days
 
-// ─── Skip Patterns ──────────────────────────────────────────────────────────
-
-const CONFIRM_RE = /^(y(es)?|no?|ok|done|go|sure|lgtm|thanks?|ty|继续|确认|好的|是的|对|嗯|行|可以|没问题)$/i;
-const SLASH_CMD_RE = /^\//;
-const PURE_OP_RE = /^(git\s+(commit|push|merge)|npm\s+(publish|deploy))\b/i;
-
-function shouldSkip(text) {
-  if (!text || text.length < 8) return true;
-  const trimmed = text.trim();
-  if (CONFIRM_RE.test(trimmed)) return true;
-  if (SLASH_CMD_RE.test(trimmed)) return true;
-  if (PURE_OP_RE.test(trimmed)) return true;
-  return false;
-}
-
-// ─── Cooldown ───────────────────────────────────────────────────────────────
-
-function checkCooldown() {
-  try {
-    const stat = statSync(COOLDOWN_FILE);
-    return (Date.now() - stat.mtimeMs) < COOLDOWN_MS;
-  } catch { return false; }
-}
-
-function touchCooldown() {
-  try { writeFileSync(COOLDOWN_FILE, String(Date.now())); } catch {}
-}
-
-// ─── Intent Detection ───────────────────────────────────────────────────────
-
-const INTENTS = [
-  // Error/debug intent
-  { pattern: /error|bug|crash|broken|fail|fix|报错|出错|错误|崩溃|修复/i, type: 'bugfix', limit: 3 },
-  // Decision/architecture intent (before recall — "为什么...之前" is a decision question, not recall)
-  { pattern: /why|decided|architecture|design|为什么|决定|架构|设计/i, type: 'decision', limit: 3 },
-  // Recall/history intent (catch-all temporal, lowest priority)
-  { pattern: /before|previously|last time|remember|之前|上次|以前|记得/i, type: null, limit: 5, useRecent: true },
-];
-
-function detectIntent(text) {
-  for (const intent of INTENTS) {
-    if (intent.pattern.test(text)) return intent;
-  }
-  return null;
-}
-
-// ─── File Path Detection ─────────────────────────────────────────────────────
-
-// Detect file paths in text
-function extractFiles(text) {
-  const matches = text.match(/[\w./-]+\.\w{1,10}/g) || [];
-  return matches.filter(m => m.includes('.') && !m.startsWith('http'));
-}
-
 // ─── DB Query Functions ─────────────────────────────────────────────────────
 
 function searchByFts(db, queryText, project, limit, typeFilter) {
@@ -124,20 +69,20 @@ function searchByFile(db, files, project, limit) {
     const basename = file.split('/').pop();
     if (!basename || basename.length < 2) continue;
     const escaped = basename.replace(/%/g, '\\%').replace(/_/g, '\\_');
-    const pathPattern = `%/${escaped}"%`;
-    const namePattern = `%"${escaped}"%`;
+    const likePattern = `%${escaped}`;
 
     const rows = db.prepare(`
-      SELECT id, type, title, lesson_learned
-      FROM observations
-      WHERE project = ?
-        AND importance >= 1
-        AND COALESCE(compressed_into, 0) = 0
-        AND created_at_epoch > ?
-        AND (files_modified LIKE ? ESCAPE '\\' OR files_read LIKE ? ESCAPE '\\')
-      ORDER BY created_at_epoch DESC
+      SELECT DISTINCT o.id, o.type, o.title, o.lesson_learned
+      FROM observations o
+      JOIN observation_files of2 ON of2.obs_id = o.id
+      WHERE o.project = ?
+        AND o.importance >= 1
+        AND COALESCE(o.compressed_into, 0) = 0
+        AND o.created_at_epoch > ?
+        AND (of2.filename = ? OR of2.filename LIKE ? ESCAPE '\\')
+      ORDER BY o.created_at_epoch DESC
       LIMIT ?
-    `).all(project, cutoff, pathPattern, namePattern, limit);
+    `).all(project, cutoff, file, likePattern, limit);
 
     results.push(...rows);
   }
@@ -226,9 +171,6 @@ async function main() {
   // Skip short/confirmation/slash-command/simple-op prompts
   if (shouldSkip(promptText)) return;
 
-  // Cooldown check — avoid flooding context on rapid prompts
-  if (checkCooldown()) return;
-
   let db;
   try {
     db = ensureDb();
@@ -264,14 +206,24 @@ async function main() {
       rows = rows.slice(0, MAX_RESULTS);
     }
 
+    const candidateIds = rows.map(r => r.id);
+    if (shouldSkipByDedup(candidateIds, INJECTED_IDS_FILE)) return;
+
     const output = formatResults(rows);
     if (output) {
       process.stdout.write(output + '\n');
-      touchCooldown();
-      // Write injected IDs for dedup with hook.mjs handleUserPrompt
+      // Write injected IDs for dedup with hook.mjs handleUserPrompt + self-dedup
       try {
-        const ids = rows.map(r => r.id);
-        writeFileSync(INJECTED_IDS_FILE, JSON.stringify({ ids, ts: Date.now() }));
+        let prevCount = 0;
+        try {
+          const prev = JSON.parse(readFileSync(INJECTED_IDS_FILE, 'utf8'));
+          if (prev.ts && Date.now() - prev.ts < DEDUP_STALE_MS) prevCount = prev.count || 0;
+        } catch {}
+        writeFileSync(INJECTED_IDS_FILE, JSON.stringify({
+          ids: candidateIds,
+          ts: Date.now(),
+          count: prevCount + 1,
+        }));
       } catch {}
     }
   } catch {

package/server-internals.mjs

@@ -2,6 +2,7 @@
 // Extracted from server.mjs for testability (server.mjs has top-level side effects)
 
 import { debugCatch, COMPRESSED_AUTO, COMPRESSED_PENDING_PURGE, OBS_BM25 } from './utils.mjs';
+import { BASE_STOP_WORDS } from './stop-words.mjs';
 
 // ─── Search Re-ranking Helpers ────────────────────────────────────────────
 
@@ -14,21 +15,15 @@ import { debugCatch, COMPRESSED_AUTO, COMPRESSED_PENDING_PURGE, OBS_BM25 } from
  */
 export function reRankWithContext(db, results, project) {
   if (!results || results.length === 0) return;
-  // Get recently active files (last 2 hours, same project)
+  // Get recently active files (last 2 hours, same project) via observation_files junction table
   const twoHoursAgo = Date.now() - 2 * 3600000;
-  const recentObs = db.prepare(`
-    SELECT files_modified FROM observations
-    WHERE project = ? AND created_at_epoch > ?
-    ORDER BY created_at_epoch DESC LIMIT 10
+  const recentFiles = db.prepare(`
+    SELECT DISTINCT of2.filename FROM observation_files of2
+    JOIN observations o ON o.id = of2.obs_id
+    WHERE o.project = ? AND o.created_at_epoch > ?
   `).all(project, twoHoursAgo);
 
-  const activeFiles = new Set();
-  for (const r of recentObs) {
-    try {
-      const files = JSON.parse(r.files_modified || '[]');
-      for (const f of files) activeFiles.add(f);
-    } catch (e) { debugCatch(e, 'reRankWithContext-parse'); }
-  }
+  const activeFiles = new Set(recentFiles.map(r => r.filename));
   if (activeFiles.size === 0) return;
 
   // Pre-compute active directories for directory-level matching
@@ -38,11 +33,25 @@ export function reRankWithContext(db, results, project) {
     if (lastSlash > 0) activeDirs.add(f.substring(0, lastSlash));
   }
 
-  for (const result of results) {
-    if (result.source !== 'obs' || !result.files_modified) continue;
-    let resultFiles;
-    try { resultFiles = JSON.parse(result.files_modified || '[]'); } catch (e) { debugCatch(e, 'reRankWithContext-resultFiles'); continue; }
-    if (resultFiles.length === 0) continue;
+  // Batch-fetch observation_files for all obs result IDs
+  const obsResults = results.filter(r => r.source === 'obs' && r.id);
+  if (obsResults.length === 0) return;
+  const obsIds = obsResults.map(r => r.id);
+  const placeholders = obsIds.map(() => '?').join(',');
+  const fileRows = db.prepare(
+    `SELECT obs_id, filename FROM observation_files WHERE obs_id IN (${placeholders})`
+  ).all(...obsIds);
+
+  // Build map: obs_id → [filenames]
+  const obsFileMap = new Map();
+  for (const row of fileRows) {
+    if (!obsFileMap.has(row.obs_id)) obsFileMap.set(row.obs_id, []);
+    obsFileMap.get(row.obs_id).push(row.filename);
+  }
+
+  for (const result of obsResults) {
+    const resultFiles = obsFileMap.get(result.id);
+    if (!resultFiles || resultFiles.length === 0) continue;
     const exactMatches = resultFiles.filter(f => activeFiles.has(f)).length;
     // Directory-level: same parent dir but different file (half weight)
     const dirMatches = resultFiles.filter(f => {
@@ -104,10 +113,8 @@ export function markSuperseded(db, results) {
 
 /** @type {Set<string>} Common words excluded from PRF term extraction */
 export const PRF_STOP_WORDS = new Set([
-  'the', 'and', 'for', 'with', 'from', 'that', 'this', 'was', 'were', 'been',
-  'have', 'has', 'had', 'are', 'but', 'not', 'all', 'can', 'into', 'when',
-  'which', 'their', 'will', 'would', 'could', 'should', 'also', 'than',
-  'then', 'its', 'use', 'used', 'using', 'some', 'new', 'added', 'updated',
+  ...BASE_STOP_WORDS,
+  'use', 'used', 'using', 'new', 'added', 'updated',
   'file', 'files', 'code', 'change', 'changed', 'changes',
 ]);
 

package/server.mjs

@@ -4,14 +4,14 @@
 
 import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
 import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
-import { jaccardSimilarity, truncate, typeIcon, sanitizeFtsQuery, relaxFtsQueryToOr, inferProject, computeMinHash, estimateJaccardFromMinHash, scrubSecrets, cjkBigrams, fmtDate, isoWeekKey, debugLog, debugCatch, COMPRESSED_PENDING_PURGE, OBS_BM25, SESS_BM25, TYPE_DECAY_CASE, getCurrentBranch } from './utils.mjs';
+import { jaccardSimilarity, truncate, typeIcon, sanitizeFtsQuery, relaxFtsQueryToOr, inferProject, computeMinHash, estimateJaccardFromMinHash, scrubSecrets, cjkBigrams, fmtDate, isoWeekKey, debugLog, debugCatch, COMPRESSED_PENDING_PURGE, OBS_BM25, SESS_BM25, TYPE_DECAY_CASE, getCurrentBranch, DEFAULT_DECAY_HALF_LIFE_MS } from './utils.mjs';
 import { ensureDb, DB_PATH, REGISTRY_DB_PATH } from './schema.mjs';
 import { reRankWithContext, markSuperseded, extractPRFTerms, expandQueryByConcepts, autoBoostIfNeeded, runIdleCleanup } from './server-internals.mjs';
 import { computeTier, TIER_CASE_SQL, tierSqlParams } from './tier.mjs';
 import { memSearchSchema, memTimelineSchema, memGetSchema, memDeleteSchema, memSaveSchema, memStatsSchema, memCompressSchema, memMaintainSchema, memRegistrySchema } from './tool-schemas.mjs';
 import { ensureRegistryDb, upsertResource } from './registry.mjs';
 import { searchResources } from './registry-retriever.mjs';
-import { getVocabulary, buildVocabulary, _resetVocabCache, computeVector, vectorSearch, rrfMerge } from './tfidf.mjs';
+import { getVocabulary, rebuildVocabulary, _resetVocabCache, computeVector, vectorSearch, rrfMerge } from './tfidf.mjs';
 import { createRequire } from 'module';
 
 const require = createRequire(import.meta.url);
@@ -102,7 +102,7 @@ function resolveProject(name) {
 //   Access bonus:  1 + 0.1 × ln(1 + access_count)
 
 // OBS_BM25, SESS_BM25, TYPE_DECAY_CASE imported from utils.mjs
-const RECENCY_HALF_LIFE_MS = 1209600000; // 14 days in milliseconds
+const RECENCY_HALF_LIFE_MS = DEFAULT_DECAY_HALF_LIFE_MS;
 
 // ─── MCP Server ─────────────────────────────────────────────────────────────
 
@@ -883,22 +883,28 @@ server.registerTool(
     const bigramText = cjkBigrams(safeTitle + ' ' + safeContent);
     const textField = bigramText ? safeContent + ' ' + bigramText : safeContent;
 
-    const result = db.prepare(`
-      INSERT INTO observations (memory_session_id, project, text, type, title, narrative, concepts, facts, files_read, files_modified, importance, minhash_sig, branch, created_at, created_at_epoch)
-      VALUES (?, ?, ?, ?, ?, ?, '', '', '[]', '[]', ?, ?, ?, ?, ?)
-    `).run(sessionId, project, textField, type, safeTitle, safeContent, args.importance ?? 1, minhashSig, getCurrentBranch(), now.toISOString(), now.getTime());
-
-    // Write TF-IDF vector
-    try {
-      const vocab = getVocabulary(db);
-      if (vocab) {
-        const vec = computeVector(safeTitle + ' ' + safeContent, vocab);
-        if (vec) {
-          db.prepare('INSERT OR REPLACE INTO observation_vectors (observation_id, vector, vocab_version, created_at_epoch) VALUES (?, ?, ?, ?)')
-            .run(Number(result.lastInsertRowid), Buffer.from(vec.buffer), vocab.version, Date.now());
+    // Atomic: insert observation + TF-IDF vector in one transaction
+    const saveTx = db.transaction(() => {
+      const result = db.prepare(`
+        INSERT INTO observations (memory_session_id, project, text, type, title, narrative, concepts, facts, files_read, files_modified, importance, minhash_sig, branch, created_at, created_at_epoch)
+        VALUES (?, ?, ?, ?, ?, ?, '', '', '[]', '[]', ?, ?, ?, ?, ?)
+      `).run(sessionId, project, textField, type, safeTitle, safeContent, args.importance ?? 1, minhashSig, getCurrentBranch(), now.toISOString(), now.getTime());
+
+      // Write TF-IDF vector
+      try {
+        const vocab = getVocabulary(db);
+        if (vocab) {
+          const vec = computeVector(safeTitle + ' ' + safeContent, vocab);
+          if (vec) {
+            db.prepare('INSERT OR REPLACE INTO observation_vectors (observation_id, vector, vocab_version, created_at_epoch) VALUES (?, ?, ?, ?)')
+              .run(Number(result.lastInsertRowid), Buffer.from(vec.buffer), vocab.version, Date.now());
+          }
         }
-      }
-    } catch (e) { debugCatch(e, 'mem_save-vector'); }
+      } catch (e) { debugCatch(e, 'mem_save-vector'); }
+
+      return result;
+    });
+    const result = saveTx();
 
     return { content: [{ type: 'text', text: `Saved as observation #${result.lastInsertRowid} [${type}] in project "${project}".` }] };
   })
@@ -1314,12 +1320,18 @@ server.registerTool(
           for (const group of args.merge_ids) {
             if (group.length < 2) continue;
             const [keepId, ...removeIds] = group;
-            for (const removeId of removeIds) mergeStmt.run(keepId, removeId);
-            totalMerged += removeIds.length;
+            for (const removeId of removeIds) {
+              const result = mergeStmt.run(keepId, removeId);
+              totalMerged += result.changes;
+            }
           }
           results.push(`Merged ${totalMerged} duplicate observations`);
         }
 
+        if (!ops.includes('dedup') && args.merge_ids) {
+          results.push('Warning: merge_ids provided but "dedup" not in operations — merge_ids ignored');
+        }
+
         if (ops.includes('purge_stale')) {
           // Delete observations previously marked as pending-purge by idle cleanup.
           // Requires user confirmation via /mem:update or /mem:mem.
@@ -1345,7 +1357,7 @@ server.registerTool(
       if (ops.includes('rebuild_vectors')) {
         try {
           _resetVocabCache();
-          const vocab = buildVocabulary(db);
+          const vocab = rebuildVocabulary(db);
           if (!vocab) {
             results.push('Vectors: no observations to build vocabulary from');
           } else {

package/utils.mjs

@@ -1,9 +1,16 @@
 // claude-mem-lite shared utilities
 // Used by server.mjs, hook.mjs, and tests
 
+
 import { basename, dirname } from 'path';
 import { execSync } from 'child_process';
 
+// ─── Re-exports from extracted modules ──────────────────────────────────────
+// Backward compatibility: all consumers import from utils.mjs
+
+export { DECAY_HALF_LIFE_BY_TYPE, DEFAULT_DECAY_HALF_LIFE_MS, OBS_BM25, SESS_BM25, TYPE_DECAY_CASE, OBS_FTS_COLUMNS } from './scoring-sql.mjs';
+export { cjkBigrams, extractCjkSynonymTokens, SYNONYM_MAP, expandToken, sanitizeFtsQuery, relaxFtsQueryToOr, FTS_STOP_WORDS, CJK_COMPOUNDS } from './nlp.mjs';
+
 // ─── Sentinel Values ────────────────────────────────────────────────────────
 
 /** compressed_into sentinel: auto-compressed without merge target */
@@ -11,45 +18,6 @@ export const COMPRESSED_AUTO = -1;
 /** compressed_into sentinel: pending user-confirmed purge (marked by idle cleanup) */
 export const COMPRESSED_PENDING_PURGE = -2;
 
-// ─── Type-Differentiated Recency Decay ──────────────────────────────────────
-
-/** Recency half-life per observation type (in milliseconds) */
-export const DECAY_HALF_LIFE_BY_TYPE = {
-  decision:  90 * 86400000,  // 90 days — architectural decisions persist
-  discovery: 60 * 86400000,  // 60 days — learned patterns last
-  feature:   30 * 86400000,  // 30 days — feature work is mid-range
-  bugfix:    14 * 86400000,  // 14 days — bugs are usually one-off
-  refactor:  14 * 86400000,  // 14 days — code cleanup
-  change:     7 * 86400000,  //  7 days — routine changes decay fast
-};
-export const DEFAULT_DECAY_HALF_LIFE_MS = 14 * 86400000;
-
-// ─── BM25 Weight Constants ──────────────────────────────────────────────────
-// Single source of truth for FTS5 BM25 weight expressions.
-// Column order must match ensureFTS() calls in schema.mjs.
-
-/** observations_fts BM25 weights: title=10, subtitle=5, narrative=5, text=3, facts=3, concepts=2, lesson_learned=8 */
-export const OBS_BM25 = 'bm25(observations_fts, 10, 5, 5, 3, 3, 2, 8)';
-
-/** session_summaries_fts BM25 weights: request=5, investigated=3, learned=3, completed=3, next_steps=2, notes=1, remaining_items=1 */
-export const SESS_BM25 = 'bm25(session_summaries_fts, 5, 3, 3, 3, 2, 1, 1)';
-
-/** FTS5 columns for observations (must match BM25 weight order) */
-export const OBS_FTS_COLUMNS = ['title', 'subtitle', 'narrative', 'text', 'facts', 'concepts', 'lesson_learned'];
-
-/** SQL CASE for type-differentiated recency decay half-lives (milliseconds) */
-export const TYPE_DECAY_CASE = `(
-  CASE o.type
-    WHEN 'decision'  THEN 7776000000.0
-    WHEN 'discovery' THEN 5184000000.0
-    WHEN 'feature'   THEN 2592000000.0
-    WHEN 'bugfix'    THEN 1209600000.0
-    WHEN 'refactor'  THEN 1209600000.0
-    WHEN 'change'    THEN  604800000.0
-    ELSE 1209600000.0
-  END
-)`;
-
 // ─── String Utilities ────────────────────────────────────────────────────────
 
 /**
@@ -229,223 +197,6 @@ export function typeIcon(type) {
   return icons[type] || '⚪';
 }
 
-// ─── FTS5 ────────────────────────────────────────────────────────────────────
-
-const FTS5_KEYWORDS = new Set(['AND', 'OR', 'NOT', 'NEAR']);
-
-// Synonym/abbreviation map: query abbreviation → expanded full forms
-// Bidirectional: both directions are registered so "K8s" finds "Kubernetes" and vice versa
-const SYNONYM_MAP = new Map();
-const SYNONYM_PAIRS = [
-  // Abbreviation ↔ full form
-  ['k8s', 'kubernetes'],
-  ['db', 'database'],
-  ['js', 'javascript'],
-  ['ts', 'typescript'],
-  ['py', 'python'],
-  ['ci', 'continuous integration'],
-  ['cd', 'continuous deployment'],
-  ['ws', 'websocket'],
-  ['auth', 'authentication'],
-  ['authn', 'authentication'],
-  ['authz', 'authorization'],
-  ['config', 'configuration'],
-  ['deps', 'dependencies'],
-  ['env', 'environment'],
-  ['infra', 'infrastructure'],
-  ['msg', 'message'],
-  ['pkg', 'package'],
-  ['repo', 'repository'],
-  ['req', 'request'],
-  ['res', 'response'],
-  ['ml', 'machine learning'],
-  ['ai', 'artificial intelligence'],
-  ['api', 'application programming interface'],
-  ['ui', 'user interface'],
-  ['ux', 'user experience'],
-  ['fe', 'frontend'],
-  ['be', 'backend'],
-  ['gql', 'graphql'],
-  ['tf', 'terraform'],
-  ['cdk', 'cloud development kit'],
-  ['iac', 'infrastructure as code'],
-  ['e2e', 'end to end'],
-  ['perf', 'performance'],
-  ['impl', 'implementation'],
-  ['fn', 'function'],
-  ['util', 'utility'],
-  ['utils', 'utilities'],
-  ['err', 'error'],
-  ['src', 'source'],
-  ['lib', 'library'],
-  ['dev', 'development'],
-  ['prod', 'production'],
-  ['async', 'asynchronous'],
-  ['sync', 'synchronous'],
-  // Semantic equivalents — precise synonyms only (overly broad bridges removed)
-  ['login', 'signin'],
-  ['bug', 'error'],
-  ['bug', 'defect'],
-  ['crash', 'panic'],
-  ['crash', 'segfault'],
-  ['slow', 'latency'],
-  ['remove', 'delete'],
-  ['setup', 'install'],
-  ['deploy', 'release'],
-  ['deploy', 'publish'],
-  ['refactor', 'restructure'],
-  ['test', 'spec'],
-  ['cache', 'caching'],
-  ['cache', 'memoize'],
-  ['optimize', 'optimization'],
-  ['fix', 'bugfix'],
-  ['fix', 'patch'],
-  ['debug', 'debugging'],
-  ['debug', 'troubleshoot'],
-  ['error', 'failure'],
-  ['migrate', 'migration'],
-  // ─── CJK ↔ EN cross-language synonyms ───
-  // Authentication & Authorization
-  ['认证', 'auth'], ['认证', 'authentication'], ['登录', 'login'], ['登录', 'auth'],
-  ['授权', 'authorization'], ['权限', 'permission'],
-  // Deployment & Operations
-  ['部署', 'deploy'], ['部署', 'deployment'], ['发布', 'release'], ['发布', 'publish'],
-  // Data & Storage
-  ['缓存', 'cache'], ['缓存', 'caching'],
-  ['数据库', 'database'], ['数据库', 'db'],
-  // Testing & Debugging
-  ['测试', 'test'], ['测试', 'testing'],
-  ['调试', 'debug'], ['调试', 'debugging'],
-  ['修复', 'fix'], ['修复', 'bugfix'],
-  // Code Quality
-  ['重构', 'refactor'], ['重构', 'refactoring'],
-  ['配置', 'config'], ['配置', 'configuration'],
-  // API & Networking
-  ['接口', 'api'], ['接口', 'endpoint'],
-  ['路由', 'route'], ['路由', 'routing'],
-  ['中间件', 'middleware'],
-  // UI & Components
-  ['组件', 'component'], ['模板', 'template'],
-  // Database Operations
-  ['迁移', 'migration'], ['迁移', 'migrate'],
-  ['索引', 'index'], ['查询', 'query'], ['查询', 'search'],
-  ['排序', 'sort'], ['分页', 'pagination'],
-  // Validation & Security
-  ['验证', 'validate'], ['验证', 'validation'],
-  ['加密', 'encrypt'], ['加密', 'encryption'],
-  ['会话', 'session'], ['令牌', 'token'],
-  // Patterns & Architecture
-  ['钩子', 'hook'], ['回调', 'callback'],
-  ['异步', 'async'], ['同步', 'sync'],
-  ['并发', 'concurrent'], ['线程', 'thread'],
-  // Performance
-  ['性能', 'performance'], ['性能', 'perf'],
-  ['内存', 'memory'], ['泄漏', 'leak'],
-  ['超时', 'timeout'], ['重试', 'retry'],
-  // Observability
-  ['日志', 'log'], ['日志', 'logging'],
-  ['监控', 'monitor'], ['告警', 'alert'],
-  // Build & Dependencies
-  ['依赖', 'dependency'], ['构建', 'build'], ['构建', 'compile'],
-  ['打包', 'bundle'], ['类型', 'type'], ['类型', 'typescript'],
-  // Errors
-  ['错误', 'error'], ['异常', 'exception'],
-  // Infrastructure
-  ['容器', 'container'], ['容器', 'docker'],
-  ['集群', 'cluster'], ['集群', 'kubernetes'],
-  ['网关', 'gateway'], ['负载', 'load balancing'],
-  ['队列', 'queue'], ['序列化', 'serialize'],
-];
-// Build bidirectional lookup (case-insensitive)
-for (const [abbr, full] of SYNONYM_PAIRS) {
-  const aLow = abbr.toLowerCase();
-  const fLow = full.toLowerCase();
-  if (!SYNONYM_MAP.has(aLow)) SYNONYM_MAP.set(aLow, new Set());
-  SYNONYM_MAP.get(aLow).add(fLow);
-  if (!SYNONYM_MAP.has(fLow)) SYNONYM_MAP.set(fLow, new Set());
-  SYNONYM_MAP.get(fLow).add(aLow);
-}
-
-// Format a term for FTS5: quote if it contains spaces, hyphens, or special chars
-function ftsToken(term) {
-  // Bare tokens are safe if purely alphanumeric or CJK characters
-  if (/^[a-zA-Z0-9\u4e00-\u9fff\u3400-\u4dbf]+$/.test(term)) return term;
-  return `"${term.replace(/"/g, '""')}"`;
-}
-
-function expandToken(token) {
-  const synonyms = SYNONYM_MAP.get(token.toLowerCase());
-  if (!synonyms || synonyms.size === 0) return ftsToken(token);
-  // FTS5 OR group: (original OR synonym1 OR "multi word synonym")
-  const parts = [ftsToken(token)];
-  for (const syn of synonyms) {
-    parts.push(ftsToken(syn));
-  }
-  return `(${parts.join(' OR ')})`;
-}
-
-/**
- * Sanitize and expand a user query into a valid FTS5 query string.
- * Strips special characters, expands synonyms, and joins with AND/space.
- * @param {string} query Raw user search query
- * @returns {string|null} FTS5-safe query or null if empty
- */
-export function sanitizeFtsQuery(query) {
-  if (!query) return null;
-  const cleaned = query
-    .replace(/[{}()[\]^~*:"\\]/g, ' ')
-    .replace(/(^|\s)-/g, '$1')
-    .trim();
-  if (!cleaned) return null;
-  const tokens = cleaned.split(/\s+/).filter(t =>
-    t && !/^-+$/.test(t) && !FTS5_KEYWORDS.has(t.toUpperCase()) && !/^NEAR\/\d+$/i.test(t)
-    // Skip single ASCII-letter tokens — too noisy for FTS5 (CJK single chars handled separately below)
-    && !(t.length === 1 && /^[a-zA-Z]$/.test(t))
-  );
-  if (tokens.length === 0) return null;
-  // Replace single CJK character tokens with bigrams for better phrase matching.
-  // Individual CJK chars ("系","统") are too noisy; bigrams ("系统") capture compound words.
-  const bigrams = cjkBigrams(cleaned);
-  const bigramSet = new Set(bigrams ? bigrams.split(' ').filter(Boolean) : []);
-  const hasBigrams = bigramSet.size > 0;
-  const finalTokens = [];
-  const seen = new Set();
-  const rawTokensSeen = new Set(); // track raw tokens to prevent bigram duplicates
-  for (const t of tokens) {
-    // Skip single CJK characters when we have bigrams — they're subsumed by bigram tokens
-    if (hasBigrams && /^[\u4e00-\u9fff\u3400-\u4dbf]$/.test(t)) continue;
-    const expanded = expandToken(t);
-    if (!seen.has(expanded)) { seen.add(expanded); rawTokensSeen.add(t); finalTokens.push(expanded); }
-  }
-  for (const bg of bigramSet) {
-    if (!seen.has(bg) && !rawTokensSeen.has(bg)) { seen.add(bg); finalTokens.push(bg); }
-  }
-  if (finalTokens.length === 0) return null;
-  // FTS5 requires explicit AND after parenthesized OR groups
-  const hasGroup = finalTokens.some(e => e.startsWith('('));
-  return finalTokens.join(hasGroup ? ' AND ' : ' ');
-}
-
-/**
- * Relax an AND-joined FTS5 query to OR-joined for fallback search.
- * Only useful when the original query has multiple tokens (single-token queries
- * are already as relaxed as possible).
- * @param {string} ftsQuery Original AND-joined FTS5 query from sanitizeFtsQuery
- * @returns {string|null} OR-joined query, or null if relaxation wouldn't help
- */
-export function relaxFtsQueryToOr(ftsQuery) {
-  if (!ftsQuery) return null;
-  // Replace AND joins with OR — handles both explicit " AND " and implicit space joins
-  const orQuery = ftsQuery.replace(/ AND /g, ' OR ');
-  // If no AND was present, tokens are space-joined (implicit AND); convert to OR
-  if (orQuery === ftsQuery && !ftsQuery.includes(' OR ')) {
-    const parts = ftsQuery.split(/\s+/);
-    if (parts.length < 2) return null; // single token — OR won't help
-    return parts.join(' OR ');
-  }
-  return orQuery !== ftsQuery ? orQuery : null;
-}
-
 // ─── Importance ──────────────────────────────────────────────────────────────
 
 /**
@@ -499,73 +250,6 @@ export function computeRuleImportance(episode) {
   return importance;
 }
 
-/**
- * Generate CJK bigrams from text for improved Chinese phrase matching in FTS5.
- * "修复了系统崩溃" → "修复 系统 统崩 崩溃"
- * @param {string} text Input text containing CJK characters
- * @returns {string} Space-separated bigrams
- */
-// Common CJK compound words (2-4 chars) — dictionary-first tokenization.
-// When a compound word is found, it's emitted as a whole token instead of being
-// split into overlapping bigrams. This dramatically reduces noise:
-// "数据库" → "数据库" (1 token) instead of "数据 据库" (2 noisy tokens)
-const CJK_COMPOUNDS = new Set([
-  // tech/programming
-  '数据库', '数据', '接口', '函数', '变量', '组件', '模块', '配置', '框架', '部署',
-  '测试', '调试', '编译', '打包', '构建', '缓存', '索引', '迁移', '回滚', '权限',
-  '认证', '授权', '加密', '解密', '序列', '并发', '异步', '同步', '线程', '进程',
-  '容器', '集群', '服务器', '中间件', '网关', '负载', '监控', '日志', '告警',
-  '前端', '后端', '全栈', '响应式', '路由', '状态', '渲染', '样式', '布局',
-  // actions
-  '修复', '重构', '优化', '升级', '安装', '卸载', '导入', '导出', '上传', '下载',
-  '提交', '推送', '合并', '发布', '上线', '回退', '审查', '审核', '评审',
-  // errors/issues
-  '报错', '崩溃', '泄露', '溢出', '死锁', '超时', '中断', '异常', '故障',
-  // architecture
-  '架构', '设计', '方案', '规划', '文档', '注释', '版本', '分支', '依赖',
-  '性能', '安全', '漏洞', '补丁',
-]);
-
-// Sort by length descending for greedy matching
-const CJK_SORTED = [...CJK_COMPOUNDS].sort((a, b) => b.length - a.length);
-
-/**
- * Generate search tokens from CJK text using dictionary-first tokenization.
- * Compound words are emitted whole; remaining chars use bigram fallback.
- * "修复了数据库崩溃" → "修复 数据库 崩溃" (3 clean tokens)
- * vs old bigram: "修复 复了 了数 数据 据库 库崩 崩溃" (7 noisy tokens)
- * @param {string} text Input text containing CJK characters
- * @returns {string} Space-separated tokens
- */
-export function cjkBigrams(text) {
-  if (!text) return '';
-  const runs = text.match(/[\u4e00-\u9fff\u3400-\u4dbf]{2,}/g) || [];
-  const tokens = [];
-  for (const run of runs) {
-    let i = 0;
-    while (i < run.length) {
-      let matched = false;
-      // Greedy dictionary match (longest first)
-      for (const word of CJK_SORTED) {
-        if (i + word.length <= run.length && run.slice(i, i + word.length) === word) {
-          tokens.push(word);
-          i += word.length;
-          matched = true;
-          break;
-        }
-      }
-      if (!matched) {
-        // Fallback: bigram for unknown compound
-        if (i + 1 < run.length) {
-          tokens.push(run[i] + run[i + 1]);
-        }
-        i++;
-      }
-    }
-  }
-  return [...new Set(tokens)].join(' ');
-}
-
 // ─── Project Inference ───────────────────────────────────────────────────────
 
 /**