claude-mem-lite 6.7.2 → 6.8.0

package/.claude-plugin/marketplace.json

@@ -9,7 +9,7 @@
   "plugins": [
     {
       "name": "claude-mem-lite",
-      "version": "6.7.2",
+      "version": "6.8.0",
       "source": "./",
       "homepage": "https://github.com/sdsrss/claude-mem-lite",
       "description": "Persistent long-term memory for Claude Code via MCP — captures coding decisions, bugfixes, and context across sessions. Hybrid FTS5 + TF-IDF search with episode batching. Single SQLite DB, no external services. A lighter, lower-cost alternative to claude-mem (episode batching + a smaller model; cost savings are an internal estimate, not a measured benchmark)."

package/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-mem-lite",
-  "version": "6.7.2",
+  "version": "6.8.0",
   "description": "Persistent long-term memory for Claude Code via MCP — captures coding decisions, bugfixes, and context across sessions. Hybrid FTS5 + TF-IDF search with episode batching. Single SQLite DB, no external services. A lighter, lower-cost alternative to claude-mem (episode batching + a smaller model; cost savings are an internal estimate, not a measured benchmark).",
   "author": {
     "name": "sdsrss"

package/README.md

@@ -236,6 +236,31 @@ rm -rf ~/claude-mem-lite/   # pre-v0.5 unhidden (if not auto-moved)
 ```
 
 <!-- normalize-per-project-note:start -->
+## Upgrading to 6.8.0
+
+**Two things change on upgrade. Neither needs an action from you, and neither is a schema
+change — an older build can still open the database.**
+
+*The first open backfills the file-lookup table, once.* Observations imported from a
+transcript before 6.7.2 carry their modified-files list but no row in the junction table the
+file-recall paths join, so asking about a file never found them. The repair was gated on that
+table being completely empty, which one ordinary `mem_save` falsifies forever. It now runs
+once per database, keyed on its own marker, and retries on a later open if it fails. A store
+that never ran `import-jsonl` matches no rows and pays nothing.
+
+Those rows become reachable through `recall` / `mem_recall` and the prompt-submit path. They
+do **not** become reachable through the pre-tool recall hook, which admits only
+`importance >= 2` and every imported row carries `1`.
+
+*`doctor --json` changes shape.* Checks that print a repair command now carry it in a
+`details` array — previously the human screen got the command and the JSON got only the
+diagnosis. And four checks (dev drift, managed files, and both hook-script branches) now
+report `"level": "fail"` where they used to report `"warn"`, with a new `"glyph": "warn"`
+recording that they still render as ⚠ rather than ✗. They always counted toward the issue
+total and the exit code; the level now says so. If you filter on `level === "fail"` you will
+see four findings you were missing. The `issues` count, the summary line and the exit code
+are unchanged.
+
 ## Upgrading to 6.1.0
 
 **One default changes, and only for the daily background pass.** Until 6.1.0 the unattended

package/README.zh-CN.md

@@ -199,6 +199,24 @@ rm -rf ~/claude-mem-lite/   # v0.5 前的非隐藏目录（如未自动迁移）
 ```
 
 <!-- normalize-per-project-note:start -->
+## 升级到 6.8.0
+
+**升级后有两处变化，都不需要你做什么，也都不是 schema 变更——旧版本仍能打开这个数据库。**
+
+*第一次开库会做一次性回填。* 6.7.2 之前从 transcript 导入的观察，带着「修改过的文件」列表，
+却没有在文件召回所 JOIN 的联结表里留下行，所以按文件去问永远找不到它们。原来的修复门是
+「这张表完全为空」——任何一次正常的 `mem_save` 都会让它永久为假。现在它按自己的标记每个库跑
+一次，失败则下次开库重试。从没跑过 `import-jsonl` 的库匹配不到任何行，代价为零。
+
+这些行会经 `recall` / `mem_recall` 和 UserPromptSubmit 这条路变得可达；**不会**经 pre-tool
+召回钩子变得可达——那条腿只收 `importance >= 2`，而导入的行一律是 `1`。
+
+*`doctor --json` 的输出形状变了。* 带修复命令的检查项现在把命令放在 `details` 数组里——此前
+人类界面拿到命令、JSON 界面只拿到诊断。另外四个检查项（dev drift、managed files、hook 脚本的
+两条）现在报 `"level": "fail"`，此前报 `"warn"`，并新增 `"glyph": "warn"` 记录它们在屏幕上
+仍渲染为 ⚠ 而非 ✗。它们一直都计入 issue 总数和退出码，现在 level 与之一致。如果你按
+`level === "fail"` 过滤，会多看到四条此前漏掉的发现。`issues` 计数、摘要行和退出码不变。
+
 ## 升级到 6.1.0
 
 **只有一个默认行为改变，且只影响每日后台任务。** 6.1.0 之前，无人值守的 `normalize` 会一次性

package/install.mjs

@@ -1715,8 +1715,26 @@ async function doctor() {
     checks.push({ level: 'fail', message: msg });
     if (!json) console.log(`  ✗ ${msg}`);
   };
+  // Detail / remedy lines. These were `--json`'s blind spot: `log()` was a no-op under
+  // --json and EVERY repair instruction doctor gives goes through it, so the machine face
+  // received the diagnosis and none of the treatment (R12 audit P2-3). 7 of its 16 call
+  // sites carry a runnable command; the other 9 are paths, notes and prose. They attach to
+  // the check they follow, not to a report-level bucket, because the stated purpose of
+  // --json is acting on an individual check.
+  //
+  // The count was first written here as "12 of 14" and both halves were wrong — pre-ship
+  // review recounted. 16 is also the count at the previous release, so no edit of this
+  // round moved it.
   const log = (msg) => {
-    if (!json) console.log(`  ${msg}`);
+    if (!json) {
+      console.log(`  ${msg}`);
+      return;
+    }
+    const last = checks[checks.length - 1];
+    // A detail before any check has nothing to attach to — same as today's drop, but the
+    // human face would show it, so this is the one line the two faces cannot share.
+    if (!last) return;
+    (last.details ??= []).push(msg.trim());
   };
 
   let issues = 0;
@@ -1730,6 +1748,28 @@ async function doctor() {
     warnings++;
     warn(msg);
   };
+  // The fourth cell of the matrix, and the one that had no home: findings rendered at ⚠
+  // severity that nevertheless REQUIRE action. Four checks used to spell it `warn(...)`
+  // followed by `issues++`, which made `issues` count rows reporting `level:'warn'` — so
+  // `checks.filter(c => c.level === 'fail')`, the exact use --json documents, under-reported
+  // by four (R12 audit P2-4). Severity and glyph are separate facts: `level` is what the
+  // counter and the exit code are derived from, `glyph` is how loud the screen is.
+  //
+  // The first draft of this comment justified the split by claiming that promoting these
+  // four to `fail()` would break `doctor-missing-files-severity` and
+  // `doctor-hook-script-manifest`. Pre-ship review measured it: rendering them as ✗ with
+  // `level:'fail'` and no glyph leaves both files 13/13 GREEN, because each asserts
+  // `level === 'warn' || level === 'fail'` — a disjunction over the only two values those
+  // checks can carry, so it cannot say NO about a severity change in either direction.
+  // The real reason is their stated INTENT (an unlinked-but-reachable module must not read
+  // as an error), which nothing enforced until this round's `doctor-failure-branches` pin.
+  // Self-counting like `dwarn`, so neither source scan in doctor-summary.test.mjs needs to
+  // reach inside it; the end-to-end counter check in doctor-json-face-parity.test.mjs does.
+  const issueWarn = (msg) => {
+    issues++;
+    checks.push({ level: 'fail', glyph: 'warn', message: msg });
+    if (!json) console.log(`  ⚠ ${msg}`);
+  };
 
   // Node version. The floor is read from package.json#engines rather than restated here —
   // see requiredNodeMajor. It is PRINTED on the ok line too, so the guard test has something
@@ -1992,8 +2032,13 @@ async function doctor() {
         `Disk footprint: DB ${mb(dbBytes)}MB, ${snaps.length} backup snapshot(s) ${mb(backupBytes)}MB (budget ${mb(backupBudgetBytes())}MB)`,
       );
     }
-  } catch {
-    /* footprint check is informational — never block doctor */
+  } catch (e) {
+    // "Informational" was the reason this was silent, and silence is the one thing it must
+    // not be: the import above is of a sibling module, so the run where it fails is a broken
+    // install — doctor's entire audience. The line vanished with no trace, which reads
+    // identically to a check that was never written (R12 audit P3-7). Every other "I could
+    // not look" in this file has its own sentence; this is the same shape as the four.
+    dwarn('Disk footprint: check failed — ' + e.message);
   }
 
   // Plugin/hook lifecycle state
@@ -2354,8 +2399,7 @@ async function doctor() {
             `${r.missingModuleCount} missing module: ${nameList(r.missingModuleFiles, r.missingModuleCount)}`,
           );
         }
-        warn(`Dev drift: ${parts.join('; ')} (${devRemedy})`);
-        issues++;
+        issueWarn(`Dev drift: ${parts.join('; ')} (${devRemedy})`);
       } else if (r.missingModuleCount > 0) {
         // Informational, NOT an issue: in a pure-symlink install every entry point resolves
         // to the repo, and Node resolves each module's imports against that REALPATH — so an
@@ -2391,12 +2435,11 @@ async function doctor() {
       // `claude-mem-lite update` is the observation editor (`update <id>`); the
       // self-updater is `self-update`. Naming the wrong one sent the user to a
       // usage error at the exact moment their install was incomplete.
-      warn(
+      issueWarn(
         `Managed files: ${r.missingCount} missing (${parts.join('; ')}) — a copy install resolves ` +
           `imports against the install dir, so these throw at hook time. Fix: claude-mem-lite self-update ` +
           `(or: node ${join(INSTALL_DIR, 'cli.mjs')} repair)`,
       );
-      issues++;
     }
     // Complete copy install: no message — drift is a dev-install concern.
   } catch (e) {
@@ -2423,12 +2466,11 @@ async function doctor() {
     if (skipScripts) {
       ok('Hook scripts: n/a (plugin-only install — hooks run from the plugin cache)');
     } else if (!h.present) {
-      warn(
+      issueWarn(
         `Hook scripts: ${join(INSTALL_DIR, 'scripts')} ` +
           `${h.dirSymlink ? 'is a dangling symlink' : 'is absent'} — all ${HOOK_SCRIPT_ENTRY_POINTS.size} hook ` +
           `commands name absolute paths under it, so no hook can fire. Fix: ${scriptRemedy}`,
       );
-      issues++;
     } else if (h.missingCount > 0) {
       const parts = [];
       if (h.missingEntryFiles.length > 0) {
@@ -2441,8 +2483,7 @@ async function doctor() {
           `${h.missingModuleFiles.length} imported helper (${h.missingModuleFiles.join(', ')}) — ERR_MODULE_NOT_FOUND at hook time`,
         );
       }
-      warn(`Hook scripts: ${h.missingCount} missing — ${parts.join('; ')}. Fix: ${scriptRemedy}`);
-      issues++;
+      issueWarn(`Hook scripts: ${h.missingCount} missing — ${parts.join('; ')}. Fix: ${scriptRemedy}`);
     } else {
       ok(
         `Hook scripts: ${HOOK_SCRIPT_FILES.length} present ` +
@@ -2608,7 +2649,12 @@ async function doctor() {
       } else {
         ok(`Plugin cache: ${versions.length} version(s) (${sizeStr})`);
       }
-    } catch {}
+    } catch (e) {
+      // Was empty to the point of carrying no comment. `existsSync` already said the path is
+      // there, so reaching here means it is unreadable or not a directory — a fact about the
+      // plugin install worth one line (R12 audit P3-7).
+      dwarn(`Plugin cache: could not read ${pluginCacheBase} — ${e.message}`);
+    }
   }
 
   if (json) {

package/lib/import-jsonl.mjs

@@ -36,6 +36,27 @@ const TOOL_TO_TYPE = {
   WebSearch: 'discovery',
 };
 
+/**
+ * The title an imported tool-use carries — and, byte for byte, the cross-run dedup key.
+ *
+ * Two call sites need this string: `importToolPair` stores it, and `tryImportToolPair`
+ * synthesizes it to ask whether a previous run already stored this row. They were separate
+ * expressions, and only one of them was SCRUBBED — so any transcript whose title holds
+ * something the scrubber rewrites (a bearer token in a `Bash` command, a path segment shaped
+ * like a key) never matched itself and re-imported on every run (R12 pre-ship review P3-4).
+ *
+ * This returns the RAW title and both sites scrub it through `scrubRecord` exactly once.
+ * The first cut scrubbed HERE, which left the storage side scrubbed twice (`scrubRecord`
+ * scrubs `title` again) against the preview's once — equal only while `scrubSecrets` is
+ * idempotent, which is an assumption about the pattern table, not a property anything
+ * enforces. Pre-ship review named the asymmetry; one scrub per side removes the assumption
+ * rather than betting on it.
+ */
+function importedObsTitle(toolUse) {
+  const detail = toolUse?.input?.command || toolEditPath(toolUse?.input) || '';
+  return `${toolUse?.name || 'unknown'}: ${String(detail).slice(0, 80)}`;
+}
+
 function dedupKey(parts) {
   return createHash('sha256').update(parts.join('\x1f')).digest('hex');
 }
@@ -141,7 +162,11 @@ function importToolPair(db, toolUse, toolResult, project) {
     (toolName === 'Edit' || toolName === 'Write' || toolName === 'NotebookEdit') && editedPath
       ? [editedPath]
       : [];
-  const filesRead = toolName === 'Read' && editedPath ? [editedPath] : [];
+  // `file_path`, not the shared `editedPath`: `toolEditPath` answers "which path did this
+  // tool WRITE", and falls back to `notebook_path` for NotebookEdit's sake. Routing the read
+  // column through it too gave a `Read` carrying only `notebook_path` a files_read entry —
+  // a shape no tool emits, and a behaviour change D#35 made without declaring it (P3-5).
+  const filesRead = toolName === 'Read' && toolUse.input?.file_path ? [toolUse.input.file_path] : [];
 
   // `narrative` carries the body and `text` is the derived search blob
   // (lib/observation-write.mjs rebuildObservationDerived). Writing the payload to `text`
@@ -151,18 +176,17 @@ function importToolPair(db, toolUse, toolResult, project) {
   // write directly.
   const body = `${inputJson}\n---\n${resultText}`;
   const safe = scrubRecord('observations', {
-    // This string IS the cross-run dedup key: tryImportToolPair synthesizes it
-    // byte-for-byte from the same expression to decide whether a row was already
-    // imported, so the two sites must be widened TOGETHER or every import
-    // duplicates. An earlier cut of this round kept the title on `file_path`
-    // alone to avoid the migration — and shipped `NotebookEdit: ` with an empty
-    // label for exactly the rows D#35 had just made reachable. Pre-ship review:
-    // "the surface the round unblocked renders a row that identifies nothing."
+    // This string IS the cross-run dedup key — `importedObsTitle` is the single home for
+    // it, because the two sites that must agree byte-for-byte did not. An earlier cut of
+    // this round kept the title on `file_path` alone to avoid the migration — and shipped
+    // `NotebookEdit: ` with an empty label for exactly the rows D#35 had just made
+    // reachable. Pre-ship review: "the surface the round unblocked renders a row that
+    // identifies nothing."
     //
     // The cost is real and bounded: NotebookEdit rows imported before v6.7.2
     // carry the old key, so the next import re-adds each of them ONCE and then
     // matches forever. Stated in the CHANGELOG rather than absorbed silently.
-    title: `${toolName}: ${(toolUse.input?.command || toolEditPath(toolUse.input) || '').slice(0, 80)}`,
+    title: importedObsTitle(toolUse),
     subtitle: '',
     text: body,
     narrative: body,
@@ -292,10 +316,10 @@ export async function importJsonl(db, path, { project }) {
     if (seenToolUseIds.has(callKey)) return false;
     seenToolUseIds.add(callKey);
 
-    // Cross-call dedup: synthesize the title the previous run would have
-    // written and check the seenObs set seeded from the DB.
-    const toolName = useEv.name || 'unknown';
-    const titlePreview = `${toolName}: ${(useEv.input?.command || toolEditPath(useEv.input) || '').slice(0, 80)}`;
+    // Cross-call dedup: synthesize the title the previous run would have written and check
+    // the seenObs set seeded from the DB. Same raw expression AND the same single scrub as
+    // the storage site, so the scrubber cannot rewrite one and not the other.
+    const titlePreview = scrubRecord('observations', { title: importedObsTitle(useEv) }).title;
     const ts = useEv.timestamp || new Date().toISOString();
     // Match the storage convention from importToolPair (memId-prefixed) so
     // the seenObs entries seeded from the DB can be matched on a re-run.

package/npm-shrinkwrap.json

@@ -1,12 +1,12 @@
 {
   "name": "claude-mem-lite",
-  "version": "6.7.2",
+  "version": "6.8.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "claude-mem-lite",
-      "version": "6.7.2",
+      "version": "6.8.0",
       "os": [
         "darwin",
         "linux",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-mem-lite",
-  "version": "6.7.2",
+  "version": "6.8.0",
   "description": "Persistent long-term memory for Claude Code via MCP — captures coding decisions, bugfixes, and context across sessions. Hybrid FTS5 + TF-IDF search with episode batching. Single SQLite DB, no external services. A lighter, lower-cost alternative to claude-mem (episode batching + a smaller model; cost savings are an internal estimate, not a measured benchmark).",
   "type": "module",
   "packageManager": "npm@10.9.2",

package/schema.mjs

@@ -831,42 +831,9 @@ export function initSchema(db) {
     }
   }
 
-  // Data migration: populate observation_files from existing observations.files_modified JSON
-  // Only runs once: when observation_files is empty but observations has rows with files_modified
-  try {
-    const obsFilesCount = db.prepare('SELECT COUNT(*) as c FROM observation_files').get().c;
-    if (obsFilesCount === 0) {
-      const obsWithFiles = db
-        .prepare(
-          `SELECT id, files_modified FROM observations WHERE files_modified IS NOT NULL AND files_modified != '[]'`,
-        )
-        .all();
-      if (obsWithFiles.length > 0) {
-        const migrateFiles = db.transaction(() => {
-          const insertFile = db.prepare(
-            'INSERT OR IGNORE INTO observation_files (obs_id, filename) VALUES (?, ?)',
-          );
-          for (const row of obsWithFiles) {
-            try {
-              const files = JSON.parse(row.files_modified);
-              if (Array.isArray(files)) {
-                for (const f of files) {
-                  if (typeof f === 'string' && f.length > 0) {
-                    insertFile.run(row.id, f);
-                  }
-                }
-              }
-            } catch {
-              /* skip malformed JSON */
-            }
-          }
-        });
-        migrateFiles();
-      }
-    }
-  } catch {
-    /* non-critical — migration can retry on next open */
-  }
+  // The files_modified -> observation_files backfill moved to runDeferredCleanups()
+  // (R12 pre-ship review P3-3). It used to live here gated on `COUNT(*) FROM
+  // observation_files === 0`, which is not the question: see DEFERRED_CLEANUPS.
 
   // observation_files orphan cleanup moved to runDeferredCleanups() (audit P1-5):
   // it now runs retryably outside the version fast-path. See DEFERRED_CLEANUPS.
@@ -1076,6 +1043,59 @@ const DEFERRED_CLEANUPS = [
     run: (db) =>
       db.prepare(`DELETE FROM observation_files WHERE obs_id NOT IN (SELECT id FROM observations)`).run(),
   },
+  {
+    // Backfill the junction from the files_modified JSON column. This ran inside initSchema
+    // for a long time, gated on `COUNT(*) FROM observation_files === 0` — which answers "has
+    // this store ever written an edge", not "has this backfill run". One real `mem_save`
+    // falsifies it forever, so every observation imported before v6.7.2 (the release that
+    // taught import-jsonl to write edges at all) stayed permanently unreachable by file, and
+    // re-importing could not repair them: cross-run dedup skips the row before the edge
+    // write. The marker here answers the question actually being asked, and an exception
+    // leaves it unset so the next open retries.
+    //
+    // Deliberately NOT a schema-version bump. A bump locks every older code home out of the
+    // database permanently (see lib/schema-skew.mjs), and on a plugin install that is
+    // reached routinely — far too much to charge for a derived table.
+    //
+    // One-shot by design: every live path that stores an observation routes its edges
+    // through the single junction writer (`insertObservationFiles`, lib/observation-write),
+    // so rows arriving after this pass need nothing from it. An earlier draft of this line
+    // said "the import and save paths both" — there are more than two entry points reaching
+    // that one writer (save, episode flush, insight promotion, restore, import), and the
+    // claim that matters is the single writer, not the count of callers.
+    // `NOT EXISTS` keeps the scan to the rows that are actually missing an edge, which is
+    // zero on a store that never imported.
+    name: 'backfill-observation-files',
+    run: (db) => {
+      const rows = db
+        .prepare(
+          `SELECT o.id, o.files_modified FROM observations o
+            WHERE o.files_modified IS NOT NULL AND o.files_modified != '[]'
+              AND NOT EXISTS (SELECT 1 FROM observation_files f WHERE f.obs_id = o.id)`,
+        )
+        .all();
+      if (rows.length === 0) return;
+      const insertFile = db.prepare(
+        'INSERT OR IGNORE INTO observation_files (obs_id, filename) VALUES (?, ?)',
+      );
+      db.transaction(() => {
+        for (const row of rows) {
+          let files;
+          try {
+            files = JSON.parse(row.files_modified);
+          } catch {
+            // One unparseable row must not cost every row after it its edges — the whole
+            // pass is marked done afterwards, so "skipped" here means "never backfilled".
+            continue;
+          }
+          if (!Array.isArray(files)) continue;
+          for (const f of files) {
+            if (typeof f === 'string' && f.length > 0) insertFile.run(row.id, f);
+          }
+        }
+      })();
+    },
+  },
   {
     // Project-name normalization: migrate short names ("mem") to canonical
     // ("projects--mem") by EXACT canonical-suffix match. Idempotent: only acts on