@modusensus/dsh-mneme 0.4.4 → 0.4.6

package/README.md

@@ -139,6 +139,25 @@ v0.3.0 起新增**记忆基因**层：从记忆里抽取**命名实体**、**带
 
 > 📖 详见 [实体结构化记忆设计](docs/ENTITIES.md) · [语义增强架构](docs/SEMANTIC.md) · [本地模型部署指南](docs/LOCAL_MODEL.md) · [从 v0.1 升级说明](docs/MIGRATION.md)
 
+### 记忆质量过滤 🧼（v0.4.6，默认开）
+
+写库前对每条记忆做**启发式质量打分**（纯函数，无 I/O、无共享状态）：元记忆词汇（谈论记忆系统本身）、自指类型标签、内容过短、重复度高、与近期记忆近似重复都会扣分（0-100）：
+
+- `score ≥ 60`：正常存储
+- `30 ≤ score < 60`：`quality_score` 落库，注入排序改为按 `importance × quality/100` 降权（degraded）
+- `score < 30`：归档并标记 `low_quality`——仍可显式搜索召回，只是**永不自动注入**
+
+`memoryQualityFilter.enabled` 可整体关闭，`archiveThreshold` / `degradeThreshold` / `minContentLength` 可调。
+
+### LLM 消耗审计 📊（v0.4.6，默认开）
+
+每次**后台 LLM 调用**（autoDream 整理 + 摘要、autoSummarize 压缩）都会写入 `llm_audit_logs` 表：`tokens` / `duration` / `status` / `source`（由哪个触发产生）。失败调用记为 `status=error`，绝不阻塞功能本体；`retentionDays`（默认 90）在启动时清理超期行。新增两个只读 API：
+
+- `GET /api/dsh-mneme/semantic/llm-audit?page=&pageSize=&source=` — 分页查询 + 按 source 过滤
+- `GET /api/dsh-mneme/semantic/llm-audit/stats?days=` — 近 N 天按 source 汇总预算（tokens / 次数 / 失败数）
+
+> 只读端点，与 list/search/semantic 一样在设置 `apiToken` 后仍保持开放。
+
 ## 🆕 最近版本亮点
 
 | 版本 | 亮点 |
@@ -163,6 +182,8 @@ v0.3.0 起新增**记忆基因**层：从记忆里抽取**命名实体**、**带
 | **v0.4.2** | ✅ 完成 | autoSummarize 自定义模型 | `summarizeProvider`/`summarizeModel` 配置项支持，可独立指定轻量模型（如 qwen3.6-plus）用于会话摘要，节省主模型 token；473 测试全绿 |
 | **v0.4.3** | ✅ 完成 | autoDream 大记忆量修复 | issue#9 B+A：`dreamMaxTokens` 上限 32768→131072 + `dreamReasoningEffort`/`sleepReasoningEffort` 思考开关（none 默认，主对话不受影响）；478 测试全绿 |
 | **v0.4.4** | ✅ 完成 | autoDream 决策覆盖修复 | issue#9 方案C：滑动窗口 `dreamMaxSnapshotSize`(默认200，updated_at 倒序截断) + 隐式 keep `dreamImplicitKeep`(默认true) + 覆盖率下限 `dreamMinExplicitCoverage`(默认50%) + 固定决策 schema；487 测试全绿 |
+| **v0.4.5** | ✅ 完成 | epistemic trust + recall eval | 记忆可信度分级 `trustEpistemicWeighting`（observation>inferred>subjective：检索排序优先高可信、注入标注 `[verified]`、dream merge/conflict 偏向高可信；opt-in 默认关）+ 检索评估 `evaluateRetrieval` 落库 `recall_evals`（`evalPersistTestResults` opt-in 默认关，生产检索始终走 `recall_runs` 无条件隔离）；518 测试全绿 |
+| **v0.4.6** | ✅ 完成 | 8 项修复（向量链路 + 注入/质量/审计） | 向量链路修复（embedSingle 适配 / `autoReindexOnBoot` 存量回填 / `vector_meta` 元数据）+ 注入语义召回 `hybridInject` + 同标题追加 `content_history` + 注入长度上限（单条 300 / 整块 1500）+ 记忆质量过滤 `memoryQualityFilter` + LLM 消耗审计 `llmAudit`（表 + 埋点 + 只读 API）；553 测试全绿 |
 | **v0.5.0+** | 🚀 远期 | 自进化记忆 | 兴趣漂移跟踪 + 跨 workspace 记忆共享（等 DSH 支持） |
 
 > 新能力一律做成**可开关的功能**（配置启用/关闭），默认保守开启、不破坏现有行为。`failure_memories` 表与 autoDream 决策引擎已为后续反思性成长铺好路。
@@ -264,6 +285,12 @@ dsh web
 | `entityExtractionMaxEntities` | `10` | 每次抽取实体数上限 |
 | `entityExtractionMaxAttrs` | `20` | 每实体属性数上限 |
 | `entitySearchEnabled` | `true` | `entity:` / `attr:` 前缀搜索开关 |
+| `trustEpistemicWeighting` | `false` | 记忆可信度加权（v0.4.5，opt-in 默认关）：记忆按来源分级 `observation`> `inferred` > `subjective`，开启后检索排序优先高可信记忆、注入对 observation 标注 `[verified]`、dream merge/conflict 偏向高可信一方；关闭时 `epistemic_status` 仅随保存落库、不参与行为 |
+| `evalPersistTestResults` | `false` | 检索评估落库（v0.4.5，opt-in 默认关）：开启后 `evaluateRetrieval` 把 precision/recall/mrr 快照写入 `recall_evals`；默认关时仅返回调用方不落库。生产 `searchMemories` 审计始终走 `recall_runs`，无条件不触碰 `recall_evals` |
+| `autoReindexOnBoot` | `true` | 存量记忆缺 embedding 时，向量已配置则启动后延迟后台按批次限速自动回填重建（设为 `false` 仅手动重建） |
+| `hybridInject` | `true` | 注入语义召回优先（v0.4.6，Bug4）：`injectCandidates` 带非空 query 时先走向量索引语义召回候选，规则筛选补足/去重；空 query / 无向量回退旧逻辑 |
+| `memoryQualityFilter` | `{enabled:true, archiveThreshold:30, degradeThreshold:60, minContentLength:10}` | 记忆质量过滤（v0.4.6，默认开）：写库前启发式打分 0-100，元记忆词汇/自指/过短/重复/近似重复扣分；≥60 正常存储，30-60 降权（注入排序按 importance×quality/100），<30 归档标记 `low_quality`（显式搜索仍可召回，永不自动注入） |
+| `llmAudit` | `{enabled:true, retentionDays:90}` | LLM 消耗审计（v0.4.6，默认开）：每次后台 LLM 调用（autoDream/autoSummarize）写 `llm_audit_logs`（tokens/duration/status/source）；失败记 error 不阻塞；只读 API `/api/dsh-mneme/semantic/llm-audit` + `/llm-audit/stats` |
 
 > 🔐 **API 安全**：DSH 无内置鉴权且默认仅监听 `127.0.0.1`。插件 API 默认开放（便于 Web 面板即装即用）。如需防护（如局域网暴露），在配置中设置 `apiToken`：写操作（画像/规则/命令）与密钥端点（`vector-config`、`vector-reindex`）需携带 `Authorization: Bearer <token>`（前端设置面板可填入同一 token），只读的 `list` / `search` / `semantic` 保持开放。`/api/dsh-mneme/vector-config` 返回的 `apiKey` 已掩码（`sk-***…`），存储仍保留明文供调用；前端回传空或掩码值表示"不改 key"。
 

package/lib/api.js

@@ -237,7 +237,9 @@ export function createApi(ctx, service, settings, commands, embedder, semantic =
         const task = viaIndex
           ? semantic.vectorIndex.rebuildIndex(embedder, { limit })
           : embedder.reindexMissing ? embedder.reindexMissing(limit) : Promise.resolve({ indexed: 0, skipped: 0, error: "vector-unavailable" });
-        task.then((result) => {
+        // Return the chain so awaiting callers (tests/health checks) observe the
+        // finished response rather than racing the async backfill.
+        return task.then((result) => {
           sendJson(res, 200, result);
         }).catch(() => {
           sendJson(res, 200, { indexed: 0, skipped: 0, error: "vector-failed" });
@@ -268,6 +270,42 @@ export function createApi(ctx, service, settings, commands, embedder, semantic =
     }
   });
 
+  // --- LLM audit trail (Bug8): paginated read + aggregate stats ---
+  // Read-only endpoints, so like list/search/semantic they stay open even when
+  // apiToken is set. The stats aggregate budget by source over the last N days.
+  register({
+    kind: "exact",
+    path: "/api/dsh-mneme/semantic/llm-audit",
+    handler(req, res) {
+      try {
+        const url = new URL(req.url, "http://localhost");
+        const page = Math.max(1, Number(url.searchParams.get("page") ?? 1) || 1);
+        const pageSize = Math.min(200, Math.max(1, Number(url.searchParams.get("pageSize") ?? 50) || 50));
+        const source = url.searchParams.get("source") ?? undefined;
+        const items = service.listLlmAudits?.({ limit: pageSize, offset: (page - 1) * pageSize, source }) ?? [];
+        const total = service.countLlmAudits?.({ source }) ?? items.length;
+        sendJson(res, 200, { items, total, page, pageSize });
+      } catch {
+        sendJson(res, 500, { error: "internal" });
+      }
+    }
+  });
+
+  register({
+    kind: "exact",
+    path: "/api/dsh-mneme/semantic/llm-audit/stats",
+    handler(req, res) {
+      try {
+        const url = new URL(req.url, "http://localhost");
+        const days = Math.max(1, Math.min(365, Number(url.searchParams.get("days") ?? 7) || 7));
+        const stats = service.getLlmAuditStats?.({ days }) ?? null;
+        sendJson(res, 200, stats ?? { error: "unavailable" });
+      } catch {
+        sendJson(res, 500, { error: "internal" });
+      }
+    }
+  });
+
   // --- health: mirror sync state (F-NEW-03 / v0.3.6) ---
   // Auth-gated; only returns a sanitized error code (never raw last_error which
   // may leak paths/token-like strings/internal hosts). On state read failure it
@@ -354,7 +392,7 @@ export function createApi(ctx, service, settings, commands, embedder, semantic =
   });
 
   return {
-    routes: 9,
+    routes: 11,
     dispose: () => {
       for (const dispose of disposers) dispose();
     }

package/lib/config.js

@@ -82,6 +82,15 @@ export const Config = z.object({
   vectorSearchThreshold: z.number().min(0).max(1).default(0.65),
   hybridSearchVectorWeight: z.number().min(0).max(1).default(0.6),
   hybridSearchKeywordWeight: z.number().min(0).max(1).default(0.4),
+  // Lazy auto-backfill of missing embeddings on boot (Bug2): when the vector
+  // API is configured and rows still lack an embedding, the index is rebuilt
+  // in the background after a short delay, rate-limited in batches. On by
+  // default; set false to keep the backfill manual only.
+  autoReindexOnBoot: z.boolean().default(true),
+  // Semantic-first injection (Bug4): when enabled, injectCandidates with a
+  // non-empty query recalls via the vector index first and falls back to the
+  // rule-based pick to fill/dedupe. Empty query / no vector → legacy behavior.
+  hybridInject: z.boolean().default(true),
 
   // --- semantic: rerank layer (v0.2) --------------------------------------
   // Opt-in by default (item ⑥): the local cross-encoder pulls in onnxruntime
@@ -167,4 +176,48 @@ export const Config = z.object({
     z.const("high"),
     z.const("none")
   ]).default("none"),
+
+  // --- epistemic trust: memory source credibility (v0.4.5) -----------------
+  // Distinguish memories by source: observation (measured / witnessed),
+  // subjective (opinion / guess) and inferred (derived from other evidence).
+  // Opt-in by default: when false (default) retrieval ranking, injection
+  // marking and dream merge/conflict keepSource are untouched and
+  // epistemic_status stays inert data (still written + inferred on save, just
+  // never used to influence behavior).
+  trustEpistemicWeighting: z.boolean().default(false),
+
+  // --- memory quality filter (Bug7) ------------------------------------------
+  // Heuristic gate on what deserves the injection/recall surface. When enabled,
+  // saveWithDedupe scores each new memory after dedupe and before write:
+  //   score >= degradeThreshold (60) → stored normally
+  //   archiveThreshold (30) <= score < 60 → quality_score persisted and the
+  //       injection sort re-ranks by importance * quality_score/100 (degraded)
+  //   score < 30 → archived + tagged low_quality (still explicitly searchable)
+  // Meta-memory markers, near-duplicates and repetitive filler lose points.
+  memoryQualityFilter: z.object({
+    enabled: z.boolean().default(true),
+    archiveThreshold: z.natural().min(1).max(100).default(30),
+    degradeThreshold: z.natural().min(1).max(100).default(60),
+    minContentLength: z.natural().min(1).max(1000).default(10)
+  }).default({}),
+
+  // --- LLM audit trail (Bug8) ------------------------------------------------
+  // Records every background LLM call (autoDream consolidation + summary,
+  // autoSummarize compression) into llm_audit_logs: tokens, duration, status
+  // and which trigger produced it. Failures are recorded as status=error and
+  // never block the feature. retentionDays bounds the table: older rows are
+  // purged on boot.
+  llmAudit: z.object({
+    enabled: z.boolean().default(true),
+    retentionDays: z.natural().min(1).max(3650).default(90)
+  }).default({}),
+
+  // --- recall evaluation: test-result storage (v0.4.6, 方案 B) --------------
+  // Separate retrieval evaluation snapshots from the production recall audit.
+  // When false (default) evaluateRetrieval still computes precision/recall/mrr
+  // and returns them to the caller, but writes nothing to recall_evals — the
+  // eval table only grows when the operator opts in. Production searchMemories
+  // audits to recall_runs and NEVER touches recall_evals, regardless of this
+  // flag (production isolation is unconditional).
+  evalPersistTestResults: z.boolean().default(false),
 });

package/lib/dream/decisions.js

@@ -1,5 +1,10 @@
 const ACTIONS = new Set(["keep", "merge", "archive", "conflict", "update", "create"]);
 
+// Epistemic trust (v0.4.5): when config.trustEpistemicWeighting is on, merge
+// keepSource and conflict winners prefer the higher-trust memory. Higher value
+// = preferred. observation (measured) > inferred (derived) > subjective (guess).
+const EPISTEMIC_PRIORITY = { observation: 3, inferred: 2, subjective: 1 };
+
 /**
  * Validate a dream decision list against a snapshot of eligible memories.
  * @param decisions - LLM-produced decision list.
@@ -252,12 +257,34 @@ function applyOne(d, service, snapshot, config = {}) {
   switch (d.action) {
     case "archive": return applyArchive(d, service, snapshot);
     case "merge": return applyMerge(d, service, snapshot, config);
-    case "conflict": return applyConflict(d, service, snapshot);
+    case "conflict": return applyConflict(d, service, snapshot, config);
     case "create": return applyCreate(d, service, config);
     default: return applyUpdate(d, service, snapshot, config);
   }
 }
 
+/**
+ * Highest-epistemic-priority UNARCHIVED id among `ids` (ties break toward
+ * `preferred`). Archived memories are never eligible keepers — promoting one
+ * would demote the real keepSource to a source and then hit the archived-keeper
+ * guard in applyMerge, silently skipping the whole merge. When `preferred`
+ * itself is archived (or missing), fall back to any unarchived candidate.
+ */
+function pickBestKeeper(ids, preferred, service) {
+  let best = null;
+  let bestP = -1;
+  for (const id of ids) {
+    const mem = service.getById(id);
+    if (!mem || mem.archived) continue; // archived/missing: ineligible keeper
+    const p = EPISTEMIC_PRIORITY[mem.epistemic_status] ?? 0;
+    if (p > bestP || (p === bestP && id === preferred)) {
+      bestP = p;
+      best = id;
+    }
+  }
+  return best ?? preferred;
+}
+
 /**
  * Mint a fresh memory (pattern discovery). No existing target, so no CAS guard.
  * Evidence ids ride in the content so a pattern stays traceable to its source
@@ -297,6 +324,13 @@ function applyArchive(d, service, snapshot) {
 }
 
 function applyMerge(d, service, snapshot, config = {}) {
+  // Epistemic trust (v0.4.5): when enabled, prefer an observation keeper over a
+  // subjective/inferred one. Mutating the decision keeps the receipt + committed
+  // record aligned with the actual keeper.
+  if (config.trustEpistemicWeighting === true) {
+    const best = pickBestKeeper(d.ids, d.keepSource, service);
+    if (best && best !== d.keepSource) d.keepSource = best;
+  }
   const sources = d.ids.filter((id) => id !== d.keepSource);
   // Idempotent replay: if every other source is already archived, this merge
   // already landed — skip so a replayed/concurrent decision never double-counts
@@ -334,7 +368,14 @@ function applyMerge(d, service, snapshot, config = {}) {
   };
 }
 
-function applyConflict(d, service, snapshot) {
+function applyConflict(d, service, snapshot, config = {}) {
+  // Epistemic trust (v0.4.5): when enabled, the observation side of a conflict
+  // is preferred as winner over a subjective/inferred one.
+  if (config.trustEpistemicWeighting === true) {
+    const pw = EPISTEMIC_PRIORITY[service.getById(d.winner)?.epistemic_status] ?? 0;
+    const pl = EPISTEMIC_PRIORITY[service.getById(d.loser)?.epistemic_status] ?? 0;
+    if (pl > pw) [d.winner, d.loser] = [d.loser, d.winner];
+  }
   const winner = service.getById(d.winner);
   const loser = service.getById(d.loser);
   if (!winner || !loser) return "skipped";

package/lib/dream.js

@@ -199,11 +199,13 @@ function buildRecordReceipts({ runId, committed, snapshot, policyEpoch }) {
  * accumulation covers both the real protocol ({type:"text-delta", index, text})
  * and looser test doubles ({type:"text-delta", text}); a terminal error/abort
  * surfaces as undefined. The caller decides how to treat an empty result.
+ * `onUsage` (optional, Bug8) receives any usage chunk for token accounting.
  */
-async function streamText(ctx, options) {
+async function streamText(ctx, options, onUsage) {
   let text = "";
   for await (const chunk of ctx.llm.stream(options)) {
     if (chunk.type === "text-delta" && typeof chunk.text === "string") text += chunk.text;
+    if (chunk.type === "usage" && typeof onUsage === "function") onUsage(chunk);
     if (chunk.type === "finish" && (chunk.reason?.kind === "error" || chunk.reason?.kind === "aborted")) {
       return undefined;
     }
@@ -211,6 +213,66 @@ async function streamText(ctx, options) {
   return text;
 }
 
+/**
+ * Bug8: wrap a background LLM call so its token/time/status are recorded in the
+ * llm_audit_logs table. Best-effort bookkeeping: a failure to WRITE the audit
+ * row is swallowed (never blocks the LLM call), while a failure of the call
+ * itself is captured as status='error' and re-thrown so the caller keeps its
+ * existing error path. `spec` carries the static metadata (trigger_source,
+ * operation_type, model_id, related_memory_ids); `body(reportUsage)` performs
+ * the actual stream consumption and is handed a usage reporter for the chunks.
+ */
+async function runAuditedLlm(ctx, service, config, spec, body) {
+  const audit = config?.llmAudit;
+  if (audit?.enabled === false || typeof service?.saveLlmAudit !== "function") return body(() => {});
+  const startedAt = Date.now();
+  const timestamp = new Date(startedAt).toISOString();
+  let inputTokens = 0;
+  let outputTokens = 0;
+  let status = "success";
+  let errorMessage = null;
+  let result;
+  try {
+    result = await body((usage) => {
+      if (!usage) return;
+      const i = usage.input_tokens ?? usage.inputTokens ?? usage.prompt_tokens ?? usage.promptTokens;
+      const o = usage.output_tokens ?? usage.outputTokens ?? usage.completion_tokens ?? usage.completionTokens;
+      if (Number.isFinite(i)) inputTokens = i;
+      if (Number.isFinite(o)) outputTokens = o;
+    });
+    if (result === undefined) {
+      // stream aborted/errored: the caller treats undefined as a failed run;
+      // record it as error here so the audit shows the truth.
+      status = "error";
+      errorMessage = errorMessage ?? "llm stream aborted or errored";
+    }
+    return result;
+  } catch (error) {
+    status = "error";
+    errorMessage = String(error?.message ?? error);
+    throw error;
+  } finally {
+    try {
+      service.saveLlmAudit({
+        timestamp,
+        trigger_source: spec.triggerSource,
+        operation_type: spec.operationType,
+        model_id: spec.modelId,
+        input_tokens: inputTokens,
+        output_tokens: outputTokens,
+        total_tokens: inputTokens + outputTokens,
+        cost_usd: 0,
+        duration_ms: Date.now() - startedAt,
+        status,
+        error_message: errorMessage,
+        related_memory_ids: spec.relatedMemoryIds ?? []
+      });
+    } catch (auditError) {
+      ctx.logger?.warn?.(`dsh-mneme: llm audit write failed: ${String(auditError)}`);
+    }
+  }
+}
+
 /**
  * Resolve the LLM route: agent default model (deployment) first, plugin config
  * (dreamProvider/dreamModel) as fallback. Falls through to undefined when no
@@ -487,7 +549,15 @@ export function createDreamScheduler({ onRun, thresholdCount = 10, thresholdChar
       : CONSOLIDATION_PROMPT;
     let decisionText;
     try {
-      decisionText = await streamText(ctx, {
+      // Bug8: the consolidation call is audited (tokens/time/status). A throw
+      // re-propagates to the catch below; an aborted stream returns undefined
+      // and is treated as a failed run after the check below.
+      decisionText = await runAuditedLlm(ctx, service, config, {
+        triggerSource: "autoDream",
+        operationType: "dream_consolidate",
+        modelId: `${route.provider}:${route.model}`,
+        relatedMemoryIds: [...snapshot.keys()]
+      }, (reportUsage) => streamText(ctx, {
         provider: route.provider,
         model: route.model,
         purpose: "compaction",
@@ -499,7 +569,7 @@ export function createDreamScheduler({ onRun, thresholdCount = 10, thresholdChar
           { role: "system", content: [{ type: "text", text: consolidationPrompt }] },
           { role: "user", content: [{ type: "text", text: listText }] }
         ]
-      });
+      }, reportUsage));
     } catch (error) {
       logger?.warn?.(`dsh-mneme dream: consolidation llm call failed: ${String(error)}`);
       return finish({ ok: false, error: "llm failed", summary: false });
@@ -637,7 +707,13 @@ export function createDreamScheduler({ onRun, thresholdCount = 10, thresholdChar
     // a failed run; summary:false marks a run that produced no summary.
     let summaryText;
     try {
-      summaryText = await streamText(ctx, {
+      // Bug8: the summary call is audited too (operation dream_summarize).
+      summaryText = await runAuditedLlm(ctx, service, config, {
+        triggerSource: "autoDream",
+        operationType: "dream_summarize",
+        modelId: `${route.provider}:${route.model}`,
+        relatedMemoryIds: []
+      }, (reportUsage) => streamText(ctx, {
         provider: route.provider,
         model: route.model,
         purpose: "compaction",
@@ -649,14 +725,18 @@ export function createDreamScheduler({ onRun, thresholdCount = 10, thresholdChar
           { role: "system", content: [{ type: "text", text: SUMMARY_PROMPT }] },
           { role: "user", content: [{ type: "text", text: service.all().filter((m) => !m.archived && m.type !== "summary").map((m) => `- ${m.title}: ${m.content}`).join("\n") }] }
         ]
-      });
+      }, reportUsage));
     } catch (error) {
       logger?.warn?.(`dsh-mneme dream: summary llm call failed: ${String(error)}`);
       return finish({ ok: false, error: "llm failed", applied, decisions: auditDecisions, outcome, frozen: frozenCount, summary: false });
     }
     let summaryStored = false;
     if (summaryText !== undefined && summaryText.trim()) {
-      service.saveWithDedupe({ type: "summary", title: "记忆库总览", content: summaryText.trim(), importance: 5, source: "dream" });
+      // Bug5 carve-out: the library overview is regenerated every run, so it
+      // must REPLACE the previous overview (not append — that would grow the
+      // summary unboundedly). `_overwrite` still archives the old overview into
+      // content_history before replacing it.
+      service.saveWithDedupe({ type: "summary", title: "记忆库总览", content: summaryText.trim(), importance: 5, source: "dream", _overwrite: true });
       summaryStored = true;
       // Re-embed the fresh summary so the index stays in sync with the store.
       if (semantic?.embedder && semantic?.vectorIndex) {

package/lib/embedding.js

@@ -4,6 +4,20 @@
 // proxy) and any provider exposing the standard embeddings API.
 const DEFAULT_TIMEOUT_MS = 15000;
 
+/** djb2 — stable, fast fingerprint for a provider/model string. Mirrors the
+ *  hash used by the local embedders so all backends share one fingerprint
+ *  format (model#hex) for vector_meta consistency checks. */
+function hashString(s) {
+  let h = 5381;
+  for (let i = 0; i < s.length; i++) h = ((h << 5) + h + s.charCodeAt(i)) >>> 0;
+  return h.toString(16);
+}
+
+/** Full provider+model fingerprint used for index-consistency checks. */
+function modelHashOf(model) {
+  return `${model}#${hashString(model)}`;
+}
+
 /** Normalize a configured baseUrl into the full embeddings endpoint URL. */
 function embeddingsUrl(baseUrl) {
   const base = String(baseUrl ?? "").trim().replace(/\/+$/, "");
@@ -50,8 +64,23 @@ export async function embedText({ baseUrl, apiKey, model }, text) {
  * Embedder bound to the current settings + store: on each write it re-embeds
  * the row's title+content and stores the vector. Failures are swallowed so a
  * flaky embedding endpoint never breaks memory writes.
+ *
+ * `vectorIndex` (optional) is the vector_meta fingerprint holder: after any
+ * successful embed the model that produced the vectors is recorded, so the
+ * index can detect drift and the auto-reindex backfill knows what to rebuild.
  */
-export function createEmbedder({ store, settings, logger }) {
+export function createEmbedder({ store, settings, logger, vectorIndex }) {
+  // Dimension of the most recent successful embed, exposed for fingerprinting.
+  let _dimension = 0;
+
+  /** Record the producing model fingerprint in vector_meta (best-effort). */
+  function markModel(cfg, dimension) {
+    if (!vectorIndex || typeof vectorIndex.markModel !== "function") return;
+    try {
+      vectorIndex.markModel(modelHashOf(cfg.model), dimension);
+    } catch { /* metadata write is best-effort */ }
+  }
+
   async function embedFor(id, title, content) {
     const cfg = settings.getVectorConfig();
     if (!cfg?.enabled || !cfg.baseUrl || !cfg.apiKey || !cfg.model) return;
@@ -59,6 +88,10 @@ export function createEmbedder({ store, settings, logger }) {
     const vector = await embedText(cfg, text);
     if (vector) {
       store.setEmbedding(id, vector);
+      _dimension = vector.length;
+      // Bug3: record which model produced the current vectors so the index can
+      // detect drift and skip a redundant backfill when nothing changed.
+      markModel(cfg, vector.length);
       logger?.info?.(`[dsh-mneme] embedded memory ${id} (dim=${vector.length})`);
     }
   }
@@ -74,7 +107,29 @@ export function createEmbedder({ store, settings, logger }) {
     async embed(query) {
       const cfg = settings.getVectorConfig();
       if (!cfg?.enabled || !cfg.baseUrl || !cfg.apiKey || !cfg.model) return null;
-      return embedText(cfg, query);
+      const vector = await embedText(cfg, query);
+      if (vector) _dimension = vector.length;
+      return vector;
+    },
+
+    // Bug1: single-text adapter. Local/ollama embedders expose embedSingle
+    // natively; the legacy OpenAI-compatible client only has embed. This
+    // adapter unifies the interface so vector-index rebuildIndex (which guards
+    // on `typeof embedder.embedSingle === "function"`) accepts this embedder.
+    async embedSingle(text) {
+      if (typeof this.embed === "function") return this.embed(text);
+      return null;
+    },
+
+    /** Model fingerprint (model#hex), or undefined when not configured. */
+    get modelHash() {
+      const cfg = settings.getVectorConfig();
+      return cfg?.enabled && cfg.model ? modelHashOf(cfg.model) : undefined;
+    },
+
+    /** Dimension of the last successful embed (0 when never embedded). */
+    get dimension() {
+      return _dimension || undefined;
     },
 
     /** Batch re-index rows still missing an embedding. */
@@ -88,9 +143,11 @@ export function createEmbedder({ store, settings, logger }) {
         const vector = await embedText(cfg, text);
         if (vector) {
           store.setEmbedding(row.id, vector);
+          _dimension = vector.length;
           indexed++;
         }
       }
+      if (indexed > 0) markModel(cfg, _dimension || undefined);
       return { indexed, skipped: rows.length - indexed };
     }
   };

package/lib/index.js

@@ -43,6 +43,15 @@ export const apply = (ctx, config) => {
   try {
     store.deleteOldFailures(new Date(Date.now() - 90 * 86400000).toISOString());
   } catch { /* non-fatal */ }
+  // Bug8: enforce llm_audit_logs retention on boot (config.llmAudit.retentionDays,
+  // default 90). Best-effort like the failure prune — the audit trail is
+  // bookkeeping and a failed purge must never block plugin boot.
+  try {
+    if (cfg.llmAudit?.enabled !== false) {
+      const retentionMs = Number.isInteger(cfg.llmAudit?.retentionDays) ? cfg.llmAudit.retentionDays : 90;
+      store.deleteOldLlmAudits(new Date(Date.now() - retentionMs * 86400000).toISOString());
+    }
+  } catch { /* non-fatal */ }
   const mirror = createMirror(memoryDir);
   const service = createService({ store, mirror, config: cfg, logger: ctx.logger });
 
@@ -99,7 +108,9 @@ export const apply = (ctx, config) => {
   let embedder = null;
   let reranker = null;
   if (cfg.embedProvider === "openai") {
-    embedder = createEmbedder({ store, settings, logger: ctx.logger });
+    // vectorIndex is passed so the legacy OpenAI embedder records the producing
+    // model fingerprint after each successful embed (Bug3).
+    embedder = createEmbedder({ store, settings, logger: ctx.logger, vectorIndex });
     service.setEmbedder(embedder);
     // legacy OpenAI embedder is immediately usable
     applyHumanEdits();
@@ -155,6 +166,62 @@ export const apply = (ctx, config) => {
     }
   }
 
+  // Bug2: lazy auto-backfill of missing embeddings on boot. When the vector API
+  // is configured and rows still lack an embedding (e.g. written before vector
+  // search was enabled) AND the vector_meta fingerprint is absent or stale, the
+  // index is rebuilt in the background after a short delay. Gated on
+  // cfg.autoReindexOnBoot; rate-limited in small batches so a large backlog
+  // never floods the provider. Failures degrade silently — search stays keyword.
+  function scheduleAutoReindex() {
+    if (cfg.autoReindexOnBoot === false) return;
+    const attempt = (tries) => {
+      try {
+        if (!embedder || typeof embedder.embedSingle !== "function") return;
+        if ("ready" in embedder && embedder.ready !== true) {
+          // Local/ollama embedders init asynchronously; give them a moment
+          // before giving up on this boot (next boot retries).
+          if (tries > 0) setTimeout(() => attempt(tries - 1), 2000);
+          return;
+        }
+        if (!store.needsEmbedding(1).length) return; // nothing to backfill
+        // Model fingerprint gate: vectors already produced by the same model
+        // mean there is no drift and no rebuild needed.
+        const current = embedder.modelHash;
+        if (current && vectorIndex.modelHash?.() === current) return;
+        const BATCH = 10;
+        const MAX_TOTAL = 500; // bound boot-time work
+        (async () => {
+          let indexed = 0;
+          for (let done = 0; done < MAX_TOTAL;) {
+            const rows = store.needsEmbedding(BATCH);
+            if (!rows.length) break;
+            for (const row of rows) {
+              try {
+                const text = [row.title, row.content].filter(Boolean).join("\n");
+                const vector = await embedder.embedSingle(text);
+                if (vector?.length) {
+                  store.setEmbedding(row.id, vector);
+                  indexed++;
+                }
+              } catch { /* skip the bad row */ }
+            }
+            done += rows.length;
+            // Rate limit: space out batches so the provider is not hammered.
+            if (store.needsEmbedding(1).length) await new Promise((r) => setTimeout(r, 200));
+          }
+          if (indexed > 0 && current) vectorIndex.markModel?.(current, embedder.dimension);
+          ctx.logger?.info?.(`[dsh-mneme] auto-reindex backfilled ${indexed} embeddings on boot`);
+        })().catch((error) => {
+          ctx.logger?.warn?.(`[dsh-mneme] auto-reindex failed: ${String(error)}`);
+        });
+      } catch (error) {
+        ctx.logger?.warn?.(`[dsh-mneme] auto-reindex failed: ${String(error)}`);
+      }
+    };
+    setTimeout(() => attempt(5), 5000);
+  }
+  scheduleAutoReindex();
+
   // Custom commands: register persisted commands into the DSH command registry
   // on boot; add/remove re-register live through the API.
   let commands = null;