@modusensus/dsh-mneme 0.4.1 → 0.4.3-beta.1

package/README.md

@@ -5,7 +5,7 @@
 [![npm version](https://img.shields.io/npm/v/@modusensus/dsh-mneme?color=blue&label=npm)](https://www.npmjs.com/package/@modusensus/dsh-mneme)
 [![license](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 [![Awesome](https://awesome-dsh-plugin.com/badge.svg)](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin)
-[![tests](https://img.shields.io/badge/tests-447%20passed-success)](https://github.com/modusensus/dsh-mneme)
+[![tests](https://img.shields.io/badge/tests-450%20passed-success)](https://github.com/modusensus/dsh-mneme)
 
 > 给 DeepSeek Harness 的跨会话记忆插件：让 Agent 记住你、记住项目、自动整理记忆。**Mneme**（Μνήμη）——希腊记忆女神 Mnemosyne 之名，掌管记忆与梦境，正如 autoDream 在后台巩固记忆。
 
@@ -54,6 +54,34 @@
 - **Fail-safe**：非法 LLM 输出（未知 id / 非法 action / 跨类型合并 / 越界 importance）拒绝整单，绝不破坏记忆库
 - **裁决审计**：每次运行写入 `dream_runs` 审计表（输入快照 sha256 digest + 完整输入快照 + 决策清单 + 逐 id 去向 + receipt），可离线回放；merge / conflict / update 幂等应用，重放/并发重复执行无累积副作用；update 记录 `_before` 快照
 
+#### dreamMaxTokens 调优指南
+
+默认 `4096` 已覆盖常规记忆库。当**记忆量大**（数万字符以上）时，决策清单与摘要可能超过默认预算，建议按规模调大：
+
+| 记忆库规模 | 建议 `dreamMaxTokens` |
+|-----------|----------------------|
+| 常规（<1 万字） | `4096`（默认） |
+| 中等（1 万-5 万字） | `65536` |
+| 大型（5 万字以上） | `131072`（上限） |
+
+> 若使用**思考型模型**（如 DeepSeek-R1 类），模型可能把全部预算花在 reasoning 上导致正文为空（日志出现 `no json array in llm output`）。此时把 `dreamReasoningEffort` 设为 `low` 可压制推理开销、把预算留给正文输出；sleep 侧对应 `sleepReasoningEffort`。默认 `none` 不传该字段，完全沿用模型自身默认，行为与旧版本一致。
+
+### Sleep Mode 系统级睡眠 💤（v0.4.0，opt-in）
+
+从 autoDream 的"被动阈值触发"升级为"主动定时维护 + 分层压缩"。系统空闲 `sleepIdleMinutes` 分钟自动执行深度维护，**默认关闭**（`sleepModeEnabled: false`），开启后行为：
+
+- **可中断**：AbortController 实现，用户恢复活动即中止当前周期（`noteWrite` 重置空闲计时 + 中断信号）
+- **串行安全**：睡眠周期走 `service.enqueue` 串行队列，与 autoDream 严格不重叠；`minRefTimeMs` 防止快照后被召回的记忆被误降级
+- **四阶段深度维护**：
+  1. `conflict_resolution`：全库冲突消解，strictness 三级可配（gentle 0.92 / normal 0.85 / aggressive 0.75）
+  2. `archival_demotion`：按 `last_accessed_at` 分层——30 天未召回压成摘要（原文进 `_full_content`，可无损恢复）、90 天完全归档
+  3. `pattern_discovery`：LLM 扫描近期记忆提炼规律，产出 `type=pattern` 记忆，evidence 强校验防伪造
+  4. `relation_completion`：检测孤立实体并补全隐含关系（共现 `related_to` / 项目 `part_of` / 技术 `depends_on`）
+- **Fail-safe**：每阶段独立 try/catch，LLM 故障只跳过对应阶段；无 LLM 路由时纯规则降级（demotion/relations）照常执行
+- **审计延续**：睡眠周期写入 `dream_runs`，`run_type='sleep'`，与 autoDream 共用审计表可追溯
+
+> 配置详见 `docs/SLEEP.md`；迁移说明见 `docs/MIGRATION.md`。
+
 ### Web 记忆面板
 
 官方设置面板 → 「记忆库设置」→「记忆」标签：按类型浏览、全文搜索；启用向量搜索后可用「语义」切换做向量召回。
@@ -115,6 +143,9 @@ v0.3.0 起新增**记忆基因**层：从记忆里抽取**命名实体**、**带
 
 | 版本 | 亮点 |
 |------|------|
+| **v0.4.2** | autoSummarize 自定义模型：`summarizeProvider`/`summarizeModel` 配置项，可独立指定轻量模型（如 qwen3.6-plus）用于会话摘要，节省主模型 token；473 测试全绿 |
+| **v0.4.0** | 系统级睡眠 Sleep Mode：空闲触发的四阶段深度维护（冲突消解 / 归档降级 / 模式发现 / 关系补全），可中断、串行安全、fail-safe，分层压缩释放冷记忆；471 测试全绿 |
+| **v0.3.9** | 修复第三方审计 4 项 FAIL：CAS 同事务原子化、Mirror 降级回执透传、逐 type 物理终态收敛、Generation 强整数校验与并发初始化稳定化 |
 | **v0.3.8** | audit peer 复验 6 项运行时阻断全部修复：desired generation 同事务原子递增（崩溃窗口不再静默跳过）、同步失败不静默、原子 generation 增量（多进程零丢失）、逐 type committed/failed/pending 回执、读取失败显式 unknown、generation 上界/负数 CHECK |
 | **v0.3.7** | 启动竞态修复：人工编辑 md 镜像后重启向量重建失败（回灌移入 init 就绪后 + scheduleEmbed 就绪门） |
 | **v0.3.6** | mirror 同步状态机：generation/applied_generation 债务建模、F-NEW-03 mirror 健康状态、持久 dirty + 启动 recoverMirror |
@@ -126,12 +157,13 @@ v0.3.0 起新增**记忆基因**层：从记忆里抽取**命名实体**、**带
 |------|------|------|------|
 | v0.2.x | ✅ 完成 | 语义增强 + 反思更新 | 本地 Embedding/Rerank/聚类、`failure_memories` 失败追踪 |
 | v0.3.0 | ✅ 完成 | 记忆基因 | entities/attrs/relations 三表 + 时间轴 + 实体搜索 |
-| v0.3.6–0.3.8 | ✅ 完成 | 镜像一致性 + 审计加固 | generation 同步状态机、audit peer 6 项运行时阻断修复、447 测试全绿 |
-| **v0.4.0** | ⏳ 规划 | 反思性成长 | 纠错双向回流（改记忆同时反思"为什么记错/召回错"）+ 规则演进（从 failure 提炼规律注入系统提示）+ 自适应参数 |
-| **v0.4.1** | ✅ 完成 | 系统级睡眠 | Sleep 调度器（空闲检测 + 最小间隔，无 cron）+ 三层深度整理（冲突消解 / 归档降级 / 模式发现）+ create 决策 + 访问 touch 免生成号，471 测试全绿 |
+| v0.3.6–0.3.8 | ✅ 完成 | 镜像一致性 + 审计加固 | generation 同步状态机、audit peer 6 项运行时阻断修复、450 测试全绿 |
+| v0.3.9 | ✅ 完成 | 审计加固 A/B/D/F | compareAndUpdate 同事务原子性、degraded 回执、逐 type 物理终态、整数 fail-closed、并发初始化稳定 |
+| **v0.4.0** | ✅ 完成 | 系统级睡眠 Sleep Mode | 空闲触发的四阶段深度维护（冲突消解 / 归档降级 / 模式发现 / 关系补全）、分层压缩、可中断串行 fail-safe；471 测试全绿 |
+| **v0.4.2** | ✅ 完成 | autoSummarize 自定义模型 | `summarizeProvider`/`summarizeModel` 配置项支持，可独立指定轻量模型（如 qwen3.6-plus）用于会话摘要，节省主模型 token；473 测试全绿 |
 | **v0.5.0+** | 🚀 远期 | 自进化记忆 | 兴趣漂移跟踪 + 跨 workspace 记忆共享（等 DSH 支持） |
 
-> 新能力一律做成**可开关的功能**（配置启用/关闭），默认保守开启、不破坏现有行为。`failure_memories` 表与 autoDream 决策引擎已为 v0.4.0 铺好路。
+> 新能力一律做成**可开关的功能**（配置启用/关闭），默认保守开启、不破坏现有行为。`failure_memories` 表与 autoDream 决策引擎已为后续反思性成长铺好路。
 
 ## 📦 安装
 
@@ -191,6 +223,7 @@ dsh web
 | `memoryDir` | `~/.dsh/memory` | 记忆存储目录（SQLite + Markdown） |
 | `autoInject` | `true` | 会话启动自动注入记忆 |
 | `autoSummarize` | `true` | 会话结束自动提炼摘要 |
+| `summarizeProvider` / `summarizeModel` | 空 | 摘要的 LLM 路由覆盖（空=使用当前会话模型）；推荐轻量模型节省主模型 token |
 | `maxInjectedItems` | `5` | 最多注入几条记忆 |
 | `importanceThreshold` | `3` | 注入的最低重要性（1-5） |
 | `autoDream` | `true` | 自动记忆整理开关 |
@@ -198,7 +231,8 @@ dsh web
 | `dreamThresholdChars` | `5000` | 触发整理的总字符阈值 |
 | `dreamDelayMs` | `2000` | 整理异步延迟（去抖） |
 | `dreamProvider` / `dreamModel` | 空 | dream 的 LLM 路由回退（默认用 agent 默认模型） |
-| `dreamMaxTokens` | `4096` | dream LLM 调用最大 token 数 |
+| `dreamMaxTokens` | `4096` | dream LLM 调用最大 token 数（上限 131072；大记忆量建议调大，见下方调优指南） |
+| `dreamReasoningEffort` | `none` | dream LLM 推理强度透传：`low` / `medium` / `high` / `none`（`none`=不传该字段，使用模型默认；思考型模型预算被推理耗尽导致正文为空时可设 `low`） |
 | `apiToken` | 空 | 可选 API 鉴权 token；设置后写操作与密钥接口要求 `Authorization: Bearer <apiToken>` |
 | `embedProvider` | `openai` | 语义后端：`openai`（默认，兼容 v0.1）/ `local`（ONNX 离线）/ `ollama` |
 | `localEmbedModel` | `Xenova/bge-small-zh-v1.5` | 本地 ONNX embedding 模型 |
@@ -270,7 +304,7 @@ src/
 lib/
 ├── client.js         # Web 面板（手写 ModuleLoader bundle）
 └── *.js              # src 的同步分发产物
-test/                 # 447 个 node:test 测试（含审计与三轴线压测不变量）
+test/                 # 450 个 node:test 测试（含审计与三轴线压测不变量）
 scripts/              # e2e-dsh.js 端到端演示 · stress-dsh.js 三轴线压测 · sync-lib.js 同步
 ```
 
@@ -279,7 +313,7 @@ scripts/              # e2e-dsh.js 端到端演示 · stress-dsh.js 三轴线压
 ```bash
 cd dsh-mneme
 npm install        # 安装 peer 依赖（以 devDependencies 形式，用于本地测试）
-npm test           # 运行 447 个测试
+npm test           # 运行 450 个测试
 npm run stress     # 三轴线压测：长会话检索 / 冲突仲裁 / 多 Agent 并发（离线 mock LLM）
 npm run sync       # 把 src/ 同步到 lib/（发布时由 prepack 钩子自动执行）
 ```

package/lib/api.js

@@ -354,7 +354,7 @@ export function createApi(ctx, service, settings, commands, embedder, semantic =
   });
 
   return {
-    routes: 7,
+    routes: 9,
     dispose: () => {
       for (const dispose of disposers) dispose();
     }

package/lib/config.js

@@ -4,6 +4,11 @@ export const Config = z.object({
   memoryDir: z.string().default("~/.dsh/memory"),
   autoInject: z.boolean().default(true),
   autoSummarize: z.boolean().default(true),
+  // Optional model override for summarization. When both are non-empty, they
+  // take priority over the session's current model. Empty = use the session's
+  // active provider/model (same as before).
+  summarizeProvider: z.string().default(""),
+  summarizeModel: z.string().default(""),
   maxInjectedItems: z.natural().min(1).max(20).default(5),
   importanceThreshold: z.natural().min(1).max(5).default(3),
   autoDream: z.boolean().default(true),
@@ -12,7 +17,18 @@ export const Config = z.object({
   dreamDelayMs: z.natural().min(0).max(60000).default(2000),
   dreamProvider: z.string(),
   dreamModel: z.string(),
-  dreamMaxTokens: z.natural().min(256).max(32768).default(4096),
+  dreamMaxTokens: z.natural().min(256).max(131072).default(4096),
+  // Pass-through reasoning effort for dream's LLM calls. 'none' (default)
+  // omits the field so the provider's own default applies; low/medium/high
+  // are forwarded verbatim. Useful to cap reasoning spend on thinking-type
+  // models that would otherwise drain the whole token budget and return an
+  // empty body ("no json array in llm output").
+  dreamReasoningEffort: z.union([
+    z.const("low"),
+    z.const("medium"),
+    z.const("high"),
+    z.const("none")
+  ]).default("none"),
   // Rule version for dream adjudication: when this bumps, older dream_runs
   // degrade to historical evidence (their receipts no longer drive live
   // decisions). Default 0 = no versioning in use yet.
@@ -92,31 +108,50 @@ export const Config = z.object({
   // Prefix/semantic search over entity names (used by recall).
   entitySearchEnabled: z.boolean().default(true),
 
-  // --- system-level sleep (v0.4.1) -----------------------------------------
-  // Opt-in: when false (default) the plugin never runs a sleep cycle, so the
-  // access-touch bookkeeping on recall/inject paths stays off too. Sleep is
-  // three phases: conflict resolution (reuses the dream conflict machinery),
-  // archival demotion (unrecalled memories tier down to summary then archive),
-  // and pattern discovery (LLM scans recent memories and mints type=pattern
-  // entries with evidence references).
-  sleepEnabled: z.boolean().default(false),
-  // A sleep run only fires when the store has been idle for this long and the
-  // last run is older than sleepMinIntervalHours. Idle detection replaces a
-  // cron-like schedule (DSH plugins have no resident crontab).
-  sleepIdleMinutes: z.natural().min(1).max(1440).default(30),
+  // --- sleep mode: idle-triggered deep maintenance (v0.4.0) ---------------
+  // Opt-in, off by default. Unlike autoDream (threshold-triggered, lightweight)
+  // sleep fires when the store has been quiet for sleepIdleMinutes and deep-
+  // maintains the whole library: conflict resolution, archival demotion,
+  // pattern discovery and entity relation completion. Abortable on user
+  // activity, audited into dream_runs (run_type='sleep'), and serialized with
+  // autoDream so the two never overlap.
+  sleepModeEnabled: z.boolean().default(false),
+  // Quiet window before a cycle fires (minutes).
+  sleepIdleMinutes: z.natural().min(1).max(60).default(5),
+  // Minimum gap between two sleep runs (hours) — a second idle window within
+  // this interval does not retrigger.
   sleepMinIntervalHours: z.natural().min(1).max(168).default(8),
-  // Unrecalled (COALESCE(last_accessed_at, updated_at, created_at)) beyond
-  // sleepArchiveDays → demote: full body moves to _full_content, content
-  // becomes a one-line summary. Beyond sleepDeepArchiveDays → archive.
-  sleepArchiveDays: z.natural().min(1).max(365).default(30),
-  sleepDeepArchiveDays: z.natural().min(1).max(3650).default(90),
-  // How many most-recent memories the pattern-discovery pass scans.
-  sleepPatternScanCount: z.natural().min(10).max(500).default(100),
-  // Max patterns minted per sleep run (mirrors decisions maxCreatePerRun).
-  sleepMaxPatterns: z.natural().min(1).max(20).default(5),
-  // Optional LLM route override; empty = fall back to agentDefaultModel then
-  // the dream route. Distinct from dreamProvider/dreamModel so the sleep pass
-  // can pin a cheaper model for its bulk summarization.
+  // Conflict adjudication strictness:
+  //   gentle    only high-confidence conflicts (threshold 0.92) are resolved
+  //   normal    standard dream-level (threshold 0.85)
+  //   aggressive low-confidence pairs are also adjudicated (threshold 0.75)
+  sleepConflictStrictness: z.union([
+    z.const("gentle"),
+    z.const("normal"),
+    z.const("aggressive")
+  ]).default("normal"),
+  // Archival demotion tiering (days since last access):
+  //   >= sleepArchiveDays  → shrink to summary, full body kept in _full_content
+  //   >= sleepCompressDays → archived outright (entity relations preserved)
+  sleepArchiveDays: z.natural().min(7).max(365).default(30),
+  sleepCompressDays: z.natural().min(7).max(365).default(90),
+  // Pattern discovery scan window (most recent memories to scan).
+  sleepPatternMinMemories: z.natural().min(10).max(1000).default(100),
+  // How far back pattern discovery considers entity attr changes (days).
+  sleepPatternLookbackDays: z.natural().min(1).max(90).default(30),
+  // Max pattern memories minted per run (0 = disabled).
+  sleepMaxPatternPerRun: z.natural().min(0).max(10).default(3),
+  // Optional LLM route override for sleep's bulk passes (empty = use dream
+  // route / agent default model).
   sleepProvider: z.string().default(""),
   sleepModel: z.string().default(""),
+  // Pass-through reasoning effort for sleep's LLM passes, same semantics as
+  // dreamReasoningEffort: 'none' (default) omits the field; low/medium/high
+  // are forwarded verbatim.
+  sleepReasoningEffort: z.union([
+    z.const("low"),
+    z.const("medium"),
+    z.const("high"),
+    z.const("none")
+  ]).default("none"),
 });

package/lib/dream/decisions.js

@@ -1,9 +1,5 @@
 const ACTIONS = new Set(["keep", "merge", "archive", "conflict", "update", "create"]);
 
-// create is used by sleep pattern discovery (v0.4.1). It fabricates a new
-// memory of any known type (default pattern) rather than touching existing ids.
-const CREATE_TYPES = new Set(["pattern", "preference", "project", "decision", "history", "summary"]);
-
 /**
  * Validate a dream decision list against a snapshot of eligible memories.
  * @param decisions - LLM-produced decision list.
@@ -13,7 +9,6 @@ const CREATE_TYPES = new Set(["pattern", "preference", "project", "decision", "h
 export function validateDecisions(decisions, snapshot, options = {}) {
   const errors = [];
   const maxUpdatePerRun = options.maxUpdatePerRun ?? 2;
-  const maxCreatePerRun = options.maxCreatePerRun ?? 5;
   const minAgeHours = options.minAgeHours ?? 24;
   if (!Array.isArray(decisions) || decisions.length === 0) {
     return { ok: false, errors: ["decision list must be a non-empty array"] };
@@ -25,9 +20,16 @@ export function validateDecisions(decisions, snapshot, options = {}) {
       errors.push(`${at}: invalid action ${JSON.stringify(d?.action)}`);
       continue;
     }
-    // create claims no existing id: it fabricates a new memory, so it runs its
-    // own field validation and skips the ids-required check + the claimed set.
-    if (d.action === "create") {
+    const ids = d.action === "conflict" ? [d.winner, d.loser] : (d.ids ?? []);
+    if (d.action === "conflict") {
+      if (!d.winner || !d.loser || d.winner === d.loser) {
+        errors.push(`${at}: conflict needs distinct winner and loser`);
+        continue;
+      }
+    } else if (d.action === "create") {
+      // Mint a fresh memory (sleep pattern discovery). Claims no existing id,
+      // so it skips the claiming loop below; evidence is optional provenance
+      // (already filtered to real ids by the caller) and is stored in content.
       if (typeof d.title !== "string" || !d.title.trim()) {
         errors.push(`${at}: create needs non-empty title`);
         continue;
@@ -38,28 +40,11 @@ export function validateDecisions(decisions, snapshot, options = {}) {
       }
       if (d.importance !== undefined && (!Number.isInteger(d.importance) || d.importance < 1 || d.importance > 5)) {
         errors.push(`${at}: create importance must be an integer 1-5 when provided`);
-        continue;
-      }
-      if (d.type !== undefined && (typeof d.type !== "string" || !CREATE_TYPES.has(d.type))) {
-        errors.push(`${at}: create type must be one of ${[...CREATE_TYPES].join(", ")}`);
-        continue;
       }
-      if (d.evidence !== undefined && !Array.isArray(d.evidence)) {
-        errors.push(`${at}: create evidence must be an array of memory ids`);
-        continue;
-      }
-      if (d.tags !== undefined && !Array.isArray(d.tags)) {
-        errors.push(`${at}: create tags must be an array`);
-        continue;
+      if (typeof d.type !== "string" || !d.type.trim()) {
+        errors.push(`${at}: create needs non-empty type`);
       }
       continue;
-    }
-    const ids = d.action === "conflict" ? [d.winner, d.loser] : (d.ids ?? []);
-    if (d.action === "conflict") {
-      if (!d.winner || !d.loser || d.winner === d.loser) {
-        errors.push(`${at}: conflict needs distinct winner and loser`);
-        continue;
-      }
     } else if (!Array.isArray(d.ids) || d.ids.length === 0) {
       errors.push(`${at}: ${d.action} needs non-empty ids`);
       continue;
@@ -129,9 +114,9 @@ export function validateDecisions(decisions, snapshot, options = {}) {
   if (updateCount > maxUpdatePerRun) {
     errors.push(`too many update decisions: ${updateCount} > ${maxUpdatePerRun}`);
   }
-  // Cap create churn: a pattern-discovery loop fabricating endless new memories
-  // would bloat the store, so a run can mint at most maxCreatePerRun.
+  // Cap pattern minting per run (sleepMaxPatternPerRun passes through here).
   const createCount = decisions.filter((d) => d.action === "create").length;
+  const maxCreatePerRun = options.maxCreatePerRun ?? 5;
   if (createCount > maxCreatePerRun) {
     errors.push(`too many create decisions: ${createCount} > ${maxCreatePerRun}`);
   }
@@ -242,46 +227,31 @@ function applyOne(d, service, snapshot, config = {}) {
     case "archive": return applyArchive(d, service, snapshot);
     case "merge": return applyMerge(d, service, snapshot, config);
     case "conflict": return applyConflict(d, service, snapshot);
-    case "create": return applyCreate(d, service, snapshot);
+    case "create": return applyCreate(d, service, config);
     default: return applyUpdate(d, service, snapshot, config);
   }
 }
 
 /**
- * Mint a new memory (sleep pattern discovery). saveWithDedupe dedupes by
- * (type, title) so a replayed create merges instead of duplicating — the
- * idempotency guard. Evidence ids are folded into tags as `ev:<id>` so a
- * pattern's provenance stays queryable after creation.
+ * Mint a fresh memory (pattern discovery). No existing target, so no CAS guard.
+ * Evidence ids ride in the content so a pattern stays traceable to its source
+ * memories. saveWithDedupe dedupes identical mints (idempotent replay-safe).
  */
-function applyCreate(d, service, snapshot) {
-  const evidence = Array.isArray(d.evidence) ? d.evidence : [];
-  const tags = [
-    ...(Array.isArray(d.tags) ? d.tags : []),
-    ...evidence.map((id) => `ev:${id}`)
-  ];
-  const result = service.saveWithDedupe({
-    type: d.type ?? "pattern",
-    title: d.title,
-    content: d.content,
-    importance: d.importance ?? 3,
-    tags,
-    source: "dream-create"
-  });
-  if (!result?.memory) return "skipped";
-  return {
-    applied: 1,
-    committed: {
-      action: "create",
-      id: result.memory.id,
-      type: d.type ?? "pattern",
-      title: d.title,
-      content: d.content,
-      importance: d.importance ?? 3,
-      evidence,
-      count_before: 0,
-      count_after: 1
-    }
-  };
+function applyCreate(d, service, config = {}) {
+  const title = String(d.title ?? "").trim();
+  const content = String(d.content ?? "").trim();
+  const importance = Number.isInteger(d.importance) ? d.importance : 3;
+  const type = typeof d.type === "string" ? d.type : "pattern";
+  const evidence = Array.isArray(d.evidence)
+    ? d.evidence.filter((id) => typeof id === "string")
+    : [];
+  const body = evidence.length > 0
+    ? `${content}\n\n[证据: ${evidence.join(", ")}]`
+    : content;
+  const created = service.saveWithDedupe({ type, title, content: body, importance });
+  const memory = created?.memory;
+  if (!memory) return "skipped"; // deduped/subsumed: nothing minted, clean no-op
+  return { applied: 1, committed: { action: "create", id: memory.id, type } };
 }
 
 function applyArchive(d, service, snapshot) {