@modusensus/dsh-mneme 0.4.1 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -7
- package/lib/api.js +1 -1
- package/lib/config.js +39 -24
- package/lib/dream/decisions.js +33 -63
- package/lib/{sleep.js → dream/sleep.js} +118 -29
- package/lib/index.js +9 -12
- package/lib/mirror.js +24 -12
- package/lib/service.js +115 -49
- package/lib/store.js +144 -75
- package/lib/summarize.js +6 -3
- package/package.json +3 -3
- package/scripts/e2e-dsh.js +4 -2
- package/src/api.js +1 -1
- package/src/config.js +39 -24
- package/src/dream/decisions.js +33 -63
- package/src/{sleep.js → dream/sleep.js} +118 -29
- package/src/index.js +9 -12
- package/src/mirror.js +24 -12
- package/src/service.js +115 -49
- package/src/store.js +144 -75
- package/src/summarize.js +6 -3
- package/test/mirror-generation.test.js +34 -1
- package/test/peer-blockers.test.js +42 -0
- package/test/sleep.test.js +297 -333
- package/test/summarize.test.js +35 -0
package/README.md
CHANGED
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
[](https://www.npmjs.com/package/@modusensus/dsh-mneme)
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
[](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin)
|
|
8
|
-
[](https://github.com/modusensus/dsh-mneme)
|
|
9
9
|
|
|
10
10
|
> 给 DeepSeek Harness 的跨会话记忆插件:让 Agent 记住你、记住项目、自动整理记忆。**Mneme**(Μνήμη)——希腊记忆女神 Mnemosyne 之名,掌管记忆与梦境,正如 autoDream 在后台巩固记忆。
|
|
11
11
|
|
|
@@ -54,6 +54,22 @@
|
|
|
54
54
|
- **Fail-safe**:非法 LLM 输出(未知 id / 非法 action / 跨类型合并 / 越界 importance)拒绝整单,绝不破坏记忆库
|
|
55
55
|
- **裁决审计**:每次运行写入 `dream_runs` 审计表(输入快照 sha256 digest + 完整输入快照 + 决策清单 + 逐 id 去向 + receipt),可离线回放;merge / conflict / update 幂等应用,重放/并发重复执行无累积副作用;update 记录 `_before` 快照
|
|
56
56
|
|
|
57
|
+
### Sleep Mode 系统级睡眠 💤(v0.4.0,opt-in)
|
|
58
|
+
|
|
59
|
+
从 autoDream 的"被动阈值触发"升级为"主动定时维护 + 分层压缩"。系统空闲 `sleepIdleMinutes` 分钟自动执行深度维护,**默认关闭**(`sleepModeEnabled: false`),开启后行为:
|
|
60
|
+
|
|
61
|
+
- **可中断**:AbortController 实现,用户恢复活动即中止当前周期(`noteWrite` 重置空闲计时 + 中断信号)
|
|
62
|
+
- **串行安全**:睡眠周期走 `service.enqueue` 串行队列,与 autoDream 严格不重叠;`minRefTimeMs` 防止快照后被召回的记忆被误降级
|
|
63
|
+
- **四阶段深度维护**:
|
|
64
|
+
1. `conflict_resolution`:全库冲突消解,strictness 三级可配(gentle 0.92 / normal 0.85 / aggressive 0.75)
|
|
65
|
+
2. `archival_demotion`:按 `last_accessed_at` 分层——30 天未召回压成摘要(原文进 `_full_content`,可无损恢复)、90 天完全归档
|
|
66
|
+
3. `pattern_discovery`:LLM 扫描近期记忆提炼规律,产出 `type=pattern` 记忆,evidence 强校验防伪造
|
|
67
|
+
4. `relation_completion`:检测孤立实体并补全隐含关系(共现 `related_to` / 项目 `part_of` / 技术 `depends_on`)
|
|
68
|
+
- **Fail-safe**:每阶段独立 try/catch,LLM 故障只跳过对应阶段;无 LLM 路由时纯规则降级(demotion/relations)照常执行
|
|
69
|
+
- **审计延续**:睡眠周期写入 `dream_runs`,`run_type='sleep'`,与 autoDream 共用审计表可追溯
|
|
70
|
+
|
|
71
|
+
> 配置详见 `docs/SLEEP.md`;迁移说明见 `docs/MIGRATION.md`。
|
|
72
|
+
|
|
57
73
|
### Web 记忆面板
|
|
58
74
|
|
|
59
75
|
官方设置面板 → 「记忆库设置」→「记忆」标签:按类型浏览、全文搜索;启用向量搜索后可用「语义」切换做向量召回。
|
|
@@ -115,6 +131,9 @@ v0.3.0 起新增**记忆基因**层:从记忆里抽取**命名实体**、**带
|
|
|
115
131
|
|
|
116
132
|
| 版本 | 亮点 |
|
|
117
133
|
|------|------|
|
|
134
|
+
| **v0.4.2** | autoSummarize 自定义模型:`summarizeProvider`/`summarizeModel` 配置项,可独立指定轻量模型(如 qwen3.6-plus)用于会话摘要,节省主模型 token;473 测试全绿 |
|
|
135
|
+
| **v0.4.0** | 系统级睡眠 Sleep Mode:空闲触发的四阶段深度维护(冲突消解 / 归档降级 / 模式发现 / 关系补全),可中断、串行安全、fail-safe,分层压缩释放冷记忆;471 测试全绿 |
|
|
136
|
+
| **v0.3.9** | 修复第三方审计 4 项 FAIL:CAS 同事务原子化、Mirror 降级回执透传、逐 type 物理终态收敛、Generation 强整数校验与并发初始化稳定化 |
|
|
118
137
|
| **v0.3.8** | audit peer 复验 6 项运行时阻断全部修复:desired generation 同事务原子递增(崩溃窗口不再静默跳过)、同步失败不静默、原子 generation 增量(多进程零丢失)、逐 type committed/failed/pending 回执、读取失败显式 unknown、generation 上界/负数 CHECK |
|
|
119
138
|
| **v0.3.7** | 启动竞态修复:人工编辑 md 镜像后重启向量重建失败(回灌移入 init 就绪后 + scheduleEmbed 就绪门) |
|
|
120
139
|
| **v0.3.6** | mirror 同步状态机:generation/applied_generation 债务建模、F-NEW-03 mirror 健康状态、持久 dirty + 启动 recoverMirror |
|
|
@@ -126,12 +145,13 @@ v0.3.0 起新增**记忆基因**层:从记忆里抽取**命名实体**、**带
|
|
|
126
145
|
|------|------|------|------|
|
|
127
146
|
| v0.2.x | ✅ 完成 | 语义增强 + 反思更新 | 本地 Embedding/Rerank/聚类、`failure_memories` 失败追踪 |
|
|
128
147
|
| v0.3.0 | ✅ 完成 | 记忆基因 | entities/attrs/relations 三表 + 时间轴 + 实体搜索 |
|
|
129
|
-
| v0.3.6–0.3.8 | ✅ 完成 | 镜像一致性 + 审计加固 | generation 同步状态机、audit peer 6 项运行时阻断修复、
|
|
130
|
-
|
|
|
131
|
-
| **v0.4.
|
|
148
|
+
| v0.3.6–0.3.8 | ✅ 完成 | 镜像一致性 + 审计加固 | generation 同步状态机、audit peer 6 项运行时阻断修复、450 测试全绿 |
|
|
149
|
+
| v0.3.9 | ✅ 完成 | 审计加固 A/B/D/F | compareAndUpdate 同事务原子性、degraded 回执、逐 type 物理终态、整数 fail-closed、并发初始化稳定 |
|
|
150
|
+
| **v0.4.0** | ✅ 完成 | 系统级睡眠 Sleep Mode | 空闲触发的四阶段深度维护(冲突消解 / 归档降级 / 模式发现 / 关系补全)、分层压缩、可中断串行 fail-safe;471 测试全绿 |
|
|
151
|
+
| **v0.4.2** | ✅ 完成 | autoSummarize 自定义模型 | `summarizeProvider`/`summarizeModel` 配置项支持,可独立指定轻量模型(如 qwen3.6-plus)用于会话摘要,节省主模型 token;473 测试全绿 |
|
|
132
152
|
| **v0.5.0+** | 🚀 远期 | 自进化记忆 | 兴趣漂移跟踪 + 跨 workspace 记忆共享(等 DSH 支持) |
|
|
133
153
|
|
|
134
|
-
> 新能力一律做成**可开关的功能**(配置启用/关闭),默认保守开启、不破坏现有行为。`failure_memories` 表与 autoDream
|
|
154
|
+
> 新能力一律做成**可开关的功能**(配置启用/关闭),默认保守开启、不破坏现有行为。`failure_memories` 表与 autoDream 决策引擎已为后续反思性成长铺好路。
|
|
135
155
|
|
|
136
156
|
## 📦 安装
|
|
137
157
|
|
|
@@ -191,6 +211,7 @@ dsh web
|
|
|
191
211
|
| `memoryDir` | `~/.dsh/memory` | 记忆存储目录(SQLite + Markdown) |
|
|
192
212
|
| `autoInject` | `true` | 会话启动自动注入记忆 |
|
|
193
213
|
| `autoSummarize` | `true` | 会话结束自动提炼摘要 |
|
|
214
|
+
| `summarizeProvider` / `summarizeModel` | 空 | 摘要的 LLM 路由覆盖(空=使用当前会话模型);推荐轻量模型节省主模型 token |
|
|
194
215
|
| `maxInjectedItems` | `5` | 最多注入几条记忆 |
|
|
195
216
|
| `importanceThreshold` | `3` | 注入的最低重要性(1-5) |
|
|
196
217
|
| `autoDream` | `true` | 自动记忆整理开关 |
|
|
@@ -270,7 +291,7 @@ src/
|
|
|
270
291
|
lib/
|
|
271
292
|
├── client.js # Web 面板(手写 ModuleLoader bundle)
|
|
272
293
|
└── *.js # src 的同步分发产物
|
|
273
|
-
test/ #
|
|
294
|
+
test/ # 450 个 node:test 测试(含审计与三轴线压测不变量)
|
|
274
295
|
scripts/ # e2e-dsh.js 端到端演示 · stress-dsh.js 三轴线压测 · sync-lib.js 同步
|
|
275
296
|
```
|
|
276
297
|
|
|
@@ -279,7 +300,7 @@ scripts/ # e2e-dsh.js 端到端演示 · stress-dsh.js 三轴线压
|
|
|
279
300
|
```bash
|
|
280
301
|
cd dsh-mneme
|
|
281
302
|
npm install # 安装 peer 依赖(以 devDependencies 形式,用于本地测试)
|
|
282
|
-
npm test # 运行
|
|
303
|
+
npm test # 运行 450 个测试
|
|
283
304
|
npm run stress # 三轴线压测:长会话检索 / 冲突仲裁 / 多 Agent 并发(离线 mock LLM)
|
|
284
305
|
npm run sync # 把 src/ 同步到 lib/(发布时由 prepack 钩子自动执行)
|
|
285
306
|
```
|
package/lib/api.js
CHANGED
package/lib/config.js
CHANGED
|
@@ -4,6 +4,11 @@ export const Config = z.object({
|
|
|
4
4
|
memoryDir: z.string().default("~/.dsh/memory"),
|
|
5
5
|
autoInject: z.boolean().default(true),
|
|
6
6
|
autoSummarize: z.boolean().default(true),
|
|
7
|
+
// Optional model override for summarization. When both are non-empty, they
|
|
8
|
+
// take priority over the session's current model. Empty = use the session's
|
|
9
|
+
// active provider/model (same as before).
|
|
10
|
+
summarizeProvider: z.string().default(""),
|
|
11
|
+
summarizeModel: z.string().default(""),
|
|
7
12
|
maxInjectedItems: z.natural().min(1).max(20).default(5),
|
|
8
13
|
importanceThreshold: z.natural().min(1).max(5).default(3),
|
|
9
14
|
autoDream: z.boolean().default(true),
|
|
@@ -92,31 +97,41 @@ export const Config = z.object({
|
|
|
92
97
|
// Prefix/semantic search over entity names (used by recall).
|
|
93
98
|
entitySearchEnabled: z.boolean().default(true),
|
|
94
99
|
|
|
95
|
-
// ---
|
|
96
|
-
// Opt-in
|
|
97
|
-
//
|
|
98
|
-
//
|
|
99
|
-
//
|
|
100
|
-
//
|
|
101
|
-
//
|
|
102
|
-
|
|
103
|
-
//
|
|
104
|
-
|
|
105
|
-
//
|
|
106
|
-
|
|
100
|
+
// --- sleep mode: idle-triggered deep maintenance (v0.4.0) ---------------
|
|
101
|
+
// Opt-in, off by default. Unlike autoDream (threshold-triggered, lightweight)
|
|
102
|
+
// sleep fires when the store has been quiet for sleepIdleMinutes and deep-
|
|
103
|
+
// maintains the whole library: conflict resolution, archival demotion,
|
|
104
|
+
// pattern discovery and entity relation completion. Abortable on user
|
|
105
|
+
// activity, audited into dream_runs (run_type='sleep'), and serialized with
|
|
106
|
+
// autoDream so the two never overlap.
|
|
107
|
+
sleepModeEnabled: z.boolean().default(false),
|
|
108
|
+
// Quiet window before a cycle fires (minutes).
|
|
109
|
+
sleepIdleMinutes: z.natural().min(1).max(60).default(5),
|
|
110
|
+
// Minimum gap between two sleep runs (hours) — a second idle window within
|
|
111
|
+
// this interval does not retrigger.
|
|
107
112
|
sleepMinIntervalHours: z.natural().min(1).max(168).default(8),
|
|
108
|
-
//
|
|
109
|
-
//
|
|
110
|
-
//
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
//
|
|
118
|
-
//
|
|
119
|
-
//
|
|
113
|
+
// Conflict adjudication strictness:
|
|
114
|
+
// gentle only high-confidence conflicts (threshold 0.92) are resolved
|
|
115
|
+
// normal standard dream-level (threshold 0.85)
|
|
116
|
+
// aggressive low-confidence pairs are also adjudicated (threshold 0.75)
|
|
117
|
+
sleepConflictStrictness: z.union([
|
|
118
|
+
z.const("gentle"),
|
|
119
|
+
z.const("normal"),
|
|
120
|
+
z.const("aggressive")
|
|
121
|
+
]).default("normal"),
|
|
122
|
+
// Archival demotion tiering (days since last access):
|
|
123
|
+
// >= sleepArchiveDays → shrink to summary, full body kept in _full_content
|
|
124
|
+
// >= sleepCompressDays → archived outright (entity relations preserved)
|
|
125
|
+
sleepArchiveDays: z.natural().min(7).max(365).default(30),
|
|
126
|
+
sleepCompressDays: z.natural().min(7).max(365).default(90),
|
|
127
|
+
// Pattern discovery scan window (most recent memories to scan).
|
|
128
|
+
sleepPatternMinMemories: z.natural().min(10).max(1000).default(100),
|
|
129
|
+
// How far back pattern discovery considers entity attr changes (days).
|
|
130
|
+
sleepPatternLookbackDays: z.natural().min(1).max(90).default(30),
|
|
131
|
+
// Max pattern memories minted per run (0 = disabled).
|
|
132
|
+
sleepMaxPatternPerRun: z.natural().min(0).max(10).default(3),
|
|
133
|
+
// Optional LLM route override for sleep's bulk passes (empty = use dream
|
|
134
|
+
// route / agent default model).
|
|
120
135
|
sleepProvider: z.string().default(""),
|
|
121
136
|
sleepModel: z.string().default(""),
|
|
122
137
|
});
|
package/lib/dream/decisions.js
CHANGED
|
@@ -1,9 +1,5 @@
|
|
|
1
1
|
const ACTIONS = new Set(["keep", "merge", "archive", "conflict", "update", "create"]);
|
|
2
2
|
|
|
3
|
-
// create is used by sleep pattern discovery (v0.4.1). It fabricates a new
|
|
4
|
-
// memory of any known type (default pattern) rather than touching existing ids.
|
|
5
|
-
const CREATE_TYPES = new Set(["pattern", "preference", "project", "decision", "history", "summary"]);
|
|
6
|
-
|
|
7
3
|
/**
|
|
8
4
|
* Validate a dream decision list against a snapshot of eligible memories.
|
|
9
5
|
* @param decisions - LLM-produced decision list.
|
|
@@ -13,7 +9,6 @@ const CREATE_TYPES = new Set(["pattern", "preference", "project", "decision", "h
|
|
|
13
9
|
export function validateDecisions(decisions, snapshot, options = {}) {
|
|
14
10
|
const errors = [];
|
|
15
11
|
const maxUpdatePerRun = options.maxUpdatePerRun ?? 2;
|
|
16
|
-
const maxCreatePerRun = options.maxCreatePerRun ?? 5;
|
|
17
12
|
const minAgeHours = options.minAgeHours ?? 24;
|
|
18
13
|
if (!Array.isArray(decisions) || decisions.length === 0) {
|
|
19
14
|
return { ok: false, errors: ["decision list must be a non-empty array"] };
|
|
@@ -25,9 +20,16 @@ export function validateDecisions(decisions, snapshot, options = {}) {
|
|
|
25
20
|
errors.push(`${at}: invalid action ${JSON.stringify(d?.action)}`);
|
|
26
21
|
continue;
|
|
27
22
|
}
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
23
|
+
const ids = d.action === "conflict" ? [d.winner, d.loser] : (d.ids ?? []);
|
|
24
|
+
if (d.action === "conflict") {
|
|
25
|
+
if (!d.winner || !d.loser || d.winner === d.loser) {
|
|
26
|
+
errors.push(`${at}: conflict needs distinct winner and loser`);
|
|
27
|
+
continue;
|
|
28
|
+
}
|
|
29
|
+
} else if (d.action === "create") {
|
|
30
|
+
// Mint a fresh memory (sleep pattern discovery). Claims no existing id,
|
|
31
|
+
// so it skips the claiming loop below; evidence is optional provenance
|
|
32
|
+
// (already filtered to real ids by the caller) and is stored in content.
|
|
31
33
|
if (typeof d.title !== "string" || !d.title.trim()) {
|
|
32
34
|
errors.push(`${at}: create needs non-empty title`);
|
|
33
35
|
continue;
|
|
@@ -38,28 +40,11 @@ export function validateDecisions(decisions, snapshot, options = {}) {
|
|
|
38
40
|
}
|
|
39
41
|
if (d.importance !== undefined && (!Number.isInteger(d.importance) || d.importance < 1 || d.importance > 5)) {
|
|
40
42
|
errors.push(`${at}: create importance must be an integer 1-5 when provided`);
|
|
41
|
-
continue;
|
|
42
|
-
}
|
|
43
|
-
if (d.type !== undefined && (typeof d.type !== "string" || !CREATE_TYPES.has(d.type))) {
|
|
44
|
-
errors.push(`${at}: create type must be one of ${[...CREATE_TYPES].join(", ")}`);
|
|
45
|
-
continue;
|
|
46
43
|
}
|
|
47
|
-
if (d.
|
|
48
|
-
errors.push(`${at}: create
|
|
49
|
-
continue;
|
|
50
|
-
}
|
|
51
|
-
if (d.tags !== undefined && !Array.isArray(d.tags)) {
|
|
52
|
-
errors.push(`${at}: create tags must be an array`);
|
|
53
|
-
continue;
|
|
44
|
+
if (typeof d.type !== "string" || !d.type.trim()) {
|
|
45
|
+
errors.push(`${at}: create needs non-empty type`);
|
|
54
46
|
}
|
|
55
47
|
continue;
|
|
56
|
-
}
|
|
57
|
-
const ids = d.action === "conflict" ? [d.winner, d.loser] : (d.ids ?? []);
|
|
58
|
-
if (d.action === "conflict") {
|
|
59
|
-
if (!d.winner || !d.loser || d.winner === d.loser) {
|
|
60
|
-
errors.push(`${at}: conflict needs distinct winner and loser`);
|
|
61
|
-
continue;
|
|
62
|
-
}
|
|
63
48
|
} else if (!Array.isArray(d.ids) || d.ids.length === 0) {
|
|
64
49
|
errors.push(`${at}: ${d.action} needs non-empty ids`);
|
|
65
50
|
continue;
|
|
@@ -129,9 +114,9 @@ export function validateDecisions(decisions, snapshot, options = {}) {
|
|
|
129
114
|
if (updateCount > maxUpdatePerRun) {
|
|
130
115
|
errors.push(`too many update decisions: ${updateCount} > ${maxUpdatePerRun}`);
|
|
131
116
|
}
|
|
132
|
-
// Cap
|
|
133
|
-
// would bloat the store, so a run can mint at most maxCreatePerRun.
|
|
117
|
+
// Cap pattern minting per run (sleepMaxPatternPerRun passes through here).
|
|
134
118
|
const createCount = decisions.filter((d) => d.action === "create").length;
|
|
119
|
+
const maxCreatePerRun = options.maxCreatePerRun ?? 5;
|
|
135
120
|
if (createCount > maxCreatePerRun) {
|
|
136
121
|
errors.push(`too many create decisions: ${createCount} > ${maxCreatePerRun}`);
|
|
137
122
|
}
|
|
@@ -242,46 +227,31 @@ function applyOne(d, service, snapshot, config = {}) {
|
|
|
242
227
|
case "archive": return applyArchive(d, service, snapshot);
|
|
243
228
|
case "merge": return applyMerge(d, service, snapshot, config);
|
|
244
229
|
case "conflict": return applyConflict(d, service, snapshot);
|
|
245
|
-
case "create": return applyCreate(d, service,
|
|
230
|
+
case "create": return applyCreate(d, service, config);
|
|
246
231
|
default: return applyUpdate(d, service, snapshot, config);
|
|
247
232
|
}
|
|
248
233
|
}
|
|
249
234
|
|
|
250
235
|
/**
|
|
251
|
-
* Mint a
|
|
252
|
-
*
|
|
253
|
-
*
|
|
254
|
-
* pattern's provenance stays queryable after creation.
|
|
236
|
+
* Mint a fresh memory (pattern discovery). No existing target, so no CAS guard.
|
|
237
|
+
* Evidence ids ride in the content so a pattern stays traceable to its source
|
|
238
|
+
* memories. saveWithDedupe dedupes identical mints (idempotent replay-safe).
|
|
255
239
|
*/
|
|
256
|
-
function applyCreate(d, service,
|
|
257
|
-
const
|
|
258
|
-
const
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
content
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
return {
|
|
272
|
-
applied: 1,
|
|
273
|
-
committed: {
|
|
274
|
-
action: "create",
|
|
275
|
-
id: result.memory.id,
|
|
276
|
-
type: d.type ?? "pattern",
|
|
277
|
-
title: d.title,
|
|
278
|
-
content: d.content,
|
|
279
|
-
importance: d.importance ?? 3,
|
|
280
|
-
evidence,
|
|
281
|
-
count_before: 0,
|
|
282
|
-
count_after: 1
|
|
283
|
-
}
|
|
284
|
-
};
|
|
240
|
+
function applyCreate(d, service, config = {}) {
|
|
241
|
+
const title = String(d.title ?? "").trim();
|
|
242
|
+
const content = String(d.content ?? "").trim();
|
|
243
|
+
const importance = Number.isInteger(d.importance) ? d.importance : 3;
|
|
244
|
+
const type = typeof d.type === "string" ? d.type : "pattern";
|
|
245
|
+
const evidence = Array.isArray(d.evidence)
|
|
246
|
+
? d.evidence.filter((id) => typeof id === "string")
|
|
247
|
+
: [];
|
|
248
|
+
const body = evidence.length > 0
|
|
249
|
+
? `${content}\n\n[证据: ${evidence.join(", ")}]`
|
|
250
|
+
: content;
|
|
251
|
+
const created = service.saveWithDedupe({ type, title, content: body, importance });
|
|
252
|
+
const memory = created?.memory;
|
|
253
|
+
if (!memory) return "skipped"; // deduped/subsumed: nothing minted, clean no-op
|
|
254
|
+
return { applied: 1, committed: { action: "create", id: memory.id, type } };
|
|
285
255
|
}
|
|
286
256
|
|
|
287
257
|
function applyArchive(d, service, snapshot) {
|
|
@@ -1,22 +1,30 @@
|
|
|
1
|
-
// System-level sleep (v0.4.
|
|
2
|
-
// the memory store.
|
|
1
|
+
// System-level sleep (v0.4.0): an idle-triggered, LLM-assisted deep pass over
|
|
2
|
+
// the memory store. Four independent, fail-safe phases:
|
|
3
3
|
// 1. conflict resolution — high-similarity same-type pairs are either parked
|
|
4
4
|
// for review (freeze mode) or adjudicated by the LLM (winner kept / loser
|
|
5
|
-
// archived), reusing the dream conflict machinery.
|
|
5
|
+
// archived), reusing the dream conflict machinery. Strictness-graded.
|
|
6
6
|
// 2. archival demotion — memories unreferenced past sleepArchiveDays shrink
|
|
7
7
|
// to a one-line summary with the full body moved to _full_content; past
|
|
8
|
-
//
|
|
8
|
+
// sleepCompressDays they are archived outright.
|
|
9
9
|
// 3. pattern discovery — the LLM scans the most recent memories and mints
|
|
10
10
|
// type=pattern entries carrying evidence id references.
|
|
11
|
+
// 4. relation completion — orphan entities (zero relations) get implied
|
|
12
|
+
// relations completed from memory co-occurrence.
|
|
11
13
|
// Each phase is wrapped so one failure never aborts the others, and a missing
|
|
12
|
-
// LLM route / semantic embedder only skips the phases that need it.
|
|
14
|
+
// LLM route / semantic embedder only skips the phases that need it. A run is
|
|
15
|
+
// abortable via an AbortController signal (user activity) — phases check the
|
|
16
|
+
// signal between batches so a running cycle yields promptly.
|
|
13
17
|
import { randomUUID, createHash } from "node:crypto";
|
|
14
|
-
import { validateDecisions, applyDecisions } from "./
|
|
15
|
-
import { findPotentialConflicts } from "./
|
|
16
|
-
import { buildReceipt } from "
|
|
18
|
+
import { validateDecisions, applyDecisions } from "./decisions.js";
|
|
19
|
+
import { findPotentialConflicts } from "./clustering.js";
|
|
20
|
+
import { buildReceipt } from "../dream.js";
|
|
17
21
|
|
|
18
22
|
const SUMMARY_MAX = 120;
|
|
19
|
-
|
|
23
|
+
// Conflict similarity threshold per strictness level (v0.4.0):
|
|
24
|
+
// gentle only high-confidence pairs (0.92) — first-time users
|
|
25
|
+
// normal standard dream-level (0.85) — default
|
|
26
|
+
// aggressive low-confidence pairs too (0.75) — bloated stores
|
|
27
|
+
const CONFLICT_THRESHOLDS = { gentle: 0.92, normal: 0.85, aggressive: 0.75 };
|
|
20
28
|
|
|
21
29
|
const CONFLICT_PROMPT = `你是记忆库冲突仲裁助手。下面是检测到的高相似度记忆对,可能内容矛盾或重复。
|
|
22
30
|
对每一对输出一个 decision 对象:
|
|
@@ -92,14 +100,17 @@ function makeSummary(m) {
|
|
|
92
100
|
* conflict_pending for human review (no LLM). Otherwise the LLM adjudicates:
|
|
93
101
|
* each pair → winner kept / loser archived. Returns a per-run summary.
|
|
94
102
|
*/
|
|
95
|
-
async function phaseConflicts(ctx, service, config, logger, runId, semantic = null) {
|
|
103
|
+
async function phaseConflicts(ctx, service, config, logger, runId, semantic = null, signal = null) {
|
|
96
104
|
const embedder = semantic?.embedder;
|
|
97
105
|
const vectorIndex = semantic?.vectorIndex;
|
|
98
106
|
if (!embedder || !vectorIndex || typeof embedder.embed !== "function") {
|
|
99
107
|
return { status: "skipped", reason: "no semantic embedder" };
|
|
100
108
|
}
|
|
109
|
+
const strictness = config.sleepConflictStrictness ?? "normal";
|
|
110
|
+
const threshold = CONFLICT_THRESHOLDS[strictness] ?? CONFLICT_THRESHOLDS.normal;
|
|
101
111
|
const memories = service.all().filter((m) => !m.archived && !m.forgotten && m.type !== "summary");
|
|
102
112
|
if (memories.length < 2) return { status: "skipped", reason: "too few memories" };
|
|
113
|
+
if (signal?.aborted) return { status: "aborted", reason: "user activity" };
|
|
103
114
|
|
|
104
115
|
// Backfill + collect vectors for every eligible memory (best effort).
|
|
105
116
|
const vectors = new Array(memories.length);
|
|
@@ -131,7 +142,7 @@ async function phaseConflicts(ctx, service, config, logger, runId, semantic = nu
|
|
|
131
142
|
const usableMemories = usable.map((i) => memories[i]);
|
|
132
143
|
const usableVectors = usable.map((i) => vectors[i]);
|
|
133
144
|
|
|
134
|
-
const pairs = findPotentialConflicts(usableMemories, usableVectors,
|
|
145
|
+
const pairs = findPotentialConflicts(usableMemories, usableVectors, threshold);
|
|
135
146
|
if (pairs.length === 0) return { status: "skipped", reason: "no conflicts found" };
|
|
136
147
|
|
|
137
148
|
// Dedupe: each memory participates in at most one pair, highest similarity
|
|
@@ -218,20 +229,21 @@ async function phaseConflicts(ctx, service, config, logger, runId, semantic = nu
|
|
|
218
229
|
* truncations, and the full body is preserved in _full_content so nothing is
|
|
219
230
|
* lost. Deterministic and cheap, so it runs even with no LLM route.
|
|
220
231
|
*/
|
|
221
|
-
function phaseDemotion(service, config, logger, runId) {
|
|
232
|
+
function phaseDemotion(service, config, logger, runId, signal = null) {
|
|
222
233
|
const archiveDays = config.sleepArchiveDays ?? 30;
|
|
223
|
-
const
|
|
234
|
+
const compressDays = config.sleepCompressDays ?? 90;
|
|
224
235
|
const archiveCut = Date.now() - archiveDays * 86400000;
|
|
225
|
-
const
|
|
236
|
+
const compressCut = Date.now() - compressDays * 86400000;
|
|
226
237
|
const demoted = [];
|
|
227
238
|
const archived = [];
|
|
228
239
|
for (const m of service.all()) {
|
|
240
|
+
if (signal?.aborted) break;
|
|
229
241
|
if (m.archived || m.forgotten) continue;
|
|
230
242
|
const ref = m.last_accessed_at ?? m.updated_at ?? m.created_at;
|
|
231
243
|
if (!ref) continue;
|
|
232
244
|
const t = new Date(ref).getTime();
|
|
233
245
|
if (Number.isNaN(t)) continue;
|
|
234
|
-
if (t <
|
|
246
|
+
if (t < compressCut) {
|
|
235
247
|
service.setArchived(m.id, true);
|
|
236
248
|
archived.push(m.id);
|
|
237
249
|
} else if (t < archiveCut) {
|
|
@@ -254,20 +266,21 @@ function phaseDemotion(service, config, logger, runId) {
|
|
|
254
266
|
* The empty snapshot is intentional: create claims no existing id, so the
|
|
255
267
|
* "every id claimed" invariant is trivially satisfied for pure-create lists.
|
|
256
268
|
*/
|
|
257
|
-
async function phasePatterns(ctx, service, config, logger, runId) {
|
|
269
|
+
async function phasePatterns(ctx, service, config, logger, runId, signal = null) {
|
|
258
270
|
const route = resolveSleepRoute(ctx, config, logger);
|
|
259
271
|
if (!route) return { status: "skipped", reason: "no llm route" };
|
|
260
|
-
const limit = config.
|
|
272
|
+
const limit = config.sleepPatternMinMemories ?? 100;
|
|
261
273
|
const memories = service
|
|
262
274
|
.list({ limit: 200, includeForgotten: false })
|
|
263
275
|
.filter((m) => !m.archived && m.type !== "summary" && m.type !== "pattern")
|
|
264
276
|
.sort((a, b) => (a.updated_at < b.updated_at ? 1 : -1))
|
|
265
277
|
.slice(0, limit);
|
|
266
278
|
if (memories.length === 0) return { status: "skipped", reason: "no memories to scan" };
|
|
279
|
+
if (signal?.aborted) return { status: "aborted", reason: "user activity" };
|
|
267
280
|
const listText = memories
|
|
268
281
|
.map((m) => `id=${m.id} | type=${m.type} | importance=${m.importance} | updated=${m.updated_at} | title=${m.title} | content=${m.content}`)
|
|
269
282
|
.join("\n");
|
|
270
|
-
const maxPatterns = config.
|
|
283
|
+
const maxPatterns = config.sleepMaxPatternPerRun ?? 3;
|
|
271
284
|
const text = await streamText(ctx, {
|
|
272
285
|
provider: route.provider,
|
|
273
286
|
model: route.model,
|
|
@@ -305,6 +318,61 @@ async function phasePatterns(ctx, service, config, logger, runId) {
|
|
|
305
318
|
};
|
|
306
319
|
}
|
|
307
320
|
|
|
321
|
+
/**
|
|
322
|
+
* Phase 4 — entity relation completion. Detects orphan entities (zero
|
|
323
|
+
* relations) and completes implied relations from memory co-occurrence:
|
|
324
|
+
* entities named in the same memory → related_to; container kinds
|
|
325
|
+
* (project/module) → part_of; tech-ish pairs → depends_on. Deterministic,
|
|
326
|
+
* no LLM — cheap, so it runs even without a route. saveRelation is
|
|
327
|
+
* bookkeeping (no write hook), so it never re-triggers the scheduler.
|
|
328
|
+
*/
|
|
329
|
+
function inferRelationType(a, b) {
|
|
330
|
+
if ((a.type === "project" || a.type === "module") && a.type !== b.type) return "part_of";
|
|
331
|
+
if ((b.type === "project" || b.type === "module") && b.type !== a.type) return "part_of";
|
|
332
|
+
if (/npm|plugin|api|sdk|lib|framework|package|deps?|build/i.test(`${a.name} ${b.name}`)) return "depends_on";
|
|
333
|
+
return "related_to";
|
|
334
|
+
}
|
|
335
|
+
|
|
336
|
+
function phaseRelations(service, config, logger, runId, signal = null) {
|
|
337
|
+
const entities = service.listEntities({ limit: 1000 }) ?? [];
|
|
338
|
+
if (entities.length < 2) return { status: "skipped", reason: "too few entities" };
|
|
339
|
+
const orphans = entities.filter((e) => (service.getRelations(e.id) ?? []).length === 0);
|
|
340
|
+
if (orphans.length === 0) return { status: "skipped", reason: "no orphan entities" };
|
|
341
|
+
const memories = service.all().filter((m) => !m.archived && !m.forgotten);
|
|
342
|
+
const seen = new Set();
|
|
343
|
+
const related = [];
|
|
344
|
+
const MAX_RELATIONS_PER_ORPHAN = 3;
|
|
345
|
+
for (const o of orphans) {
|
|
346
|
+
if (signal?.aborted) break;
|
|
347
|
+
let made = 0;
|
|
348
|
+
for (const m of memories) {
|
|
349
|
+
if (signal?.aborted || made >= MAX_RELATIONS_PER_ORPHAN) break;
|
|
350
|
+
const text = `${m.title ?? ""} ${m.content ?? ""}`;
|
|
351
|
+
if (!text.includes(o.name)) continue;
|
|
352
|
+
for (const other of entities) {
|
|
353
|
+
if (other.id === o.id || other.name === o.name) continue;
|
|
354
|
+
const key = [o.id, other.id].sort().join("|");
|
|
355
|
+
if (seen.has(key)) continue;
|
|
356
|
+
if (!text.includes(other.name)) continue;
|
|
357
|
+
const relationType = inferRelationType(o, other);
|
|
358
|
+
try {
|
|
359
|
+
service.saveRelation({ from_entity: o.id, to_entity: other.id, relation_type: relationType, memory_id: m.id, metadata: { source: "sleep_relation_completion" } });
|
|
360
|
+
seen.add(key);
|
|
361
|
+
related.push({ from: o.id, to: other.id, type: relationType });
|
|
362
|
+
made++;
|
|
363
|
+
} catch (error) {
|
|
364
|
+
logger?.warn?.(`dsh-mneme sleep: relation ${o.id}/${other.id} failed: ${String(error)}`);
|
|
365
|
+
}
|
|
366
|
+
}
|
|
367
|
+
}
|
|
368
|
+
}
|
|
369
|
+
return {
|
|
370
|
+
status: related.length > 0 ? "ok" : "noop",
|
|
371
|
+
orphanCount: orphans.length,
|
|
372
|
+
related
|
|
373
|
+
};
|
|
374
|
+
}
|
|
375
|
+
|
|
308
376
|
// ---------------------------------------------------------------- run
|
|
309
377
|
|
|
310
378
|
function deriveStatus(phases) {
|
|
@@ -323,10 +391,11 @@ function deriveStatus(phases) {
|
|
|
323
391
|
* run_type='sleep' audit row (same dream_runs table) so sleep activity is
|
|
324
392
|
* observable alongside consolidation runs.
|
|
325
393
|
*/
|
|
326
|
-
export async function runSleep(ctx, service, config, logger, semantic = null) {
|
|
394
|
+
export async function runSleep(ctx, service, config, logger, semantic = null, signal = null) {
|
|
327
395
|
const runId = randomUUID();
|
|
328
396
|
const phases = {};
|
|
329
397
|
const attempt = async (name, fn) => {
|
|
398
|
+
if (signal?.aborted) return; // user resumed activity — stop before next phase
|
|
330
399
|
try {
|
|
331
400
|
phases[name] = await fn();
|
|
332
401
|
} catch (error) {
|
|
@@ -334,9 +403,10 @@ export async function runSleep(ctx, service, config, logger, semantic = null) {
|
|
|
334
403
|
logger?.warn?.(`dsh-mneme sleep: ${name} phase failed: ${error?.message ?? error}`);
|
|
335
404
|
}
|
|
336
405
|
};
|
|
337
|
-
await attempt("conflicts", () => phaseConflicts(ctx, service, config, logger, runId, semantic));
|
|
338
|
-
await attempt("demotion", () => phaseDemotion(service, config, logger, runId));
|
|
339
|
-
await attempt("patterns", () => phasePatterns(ctx, service, config, logger, runId));
|
|
406
|
+
await attempt("conflicts", () => phaseConflicts(ctx, service, config, logger, runId, semantic, signal));
|
|
407
|
+
await attempt("demotion", () => phaseDemotion(service, config, logger, runId, signal));
|
|
408
|
+
await attempt("patterns", () => phasePatterns(ctx, service, config, logger, runId, signal));
|
|
409
|
+
await attempt("relations", () => phaseRelations(service, config, logger, runId, signal));
|
|
340
410
|
|
|
341
411
|
const status = deriveStatus(phases);
|
|
342
412
|
const route = resolveSleepRoute(ctx, config, logger);
|
|
@@ -398,11 +468,12 @@ export function createSleepScheduler({
|
|
|
398
468
|
let running = false;
|
|
399
469
|
let disposed = false;
|
|
400
470
|
let idleTimer = null;
|
|
471
|
+
let sleepAbort = null;
|
|
401
472
|
|
|
402
473
|
function armIdleTimer() {
|
|
403
474
|
if (disposed || idleTimer) return;
|
|
404
|
-
if (config.
|
|
405
|
-
const idleMs = (config.sleepIdleMinutes ??
|
|
475
|
+
if (config.sleepModeEnabled !== true) return;
|
|
476
|
+
const idleMs = (config.sleepIdleMinutes ?? 5) * 60000;
|
|
406
477
|
const delay = Math.max(0, idleMs - (now() - lastWriteAt)) + 1000;
|
|
407
478
|
idleTimer = setTimeoutFn(async () => {
|
|
408
479
|
idleTimer = null;
|
|
@@ -413,18 +484,29 @@ export function createSleepScheduler({
|
|
|
413
484
|
|
|
414
485
|
function shouldRun(at = now()) {
|
|
415
486
|
if (disposed || running) return false;
|
|
416
|
-
if (config.
|
|
417
|
-
if (at - lastWriteAt < (config.sleepIdleMinutes ??
|
|
418
|
-
|
|
487
|
+
if (config.sleepModeEnabled !== true) return false;
|
|
488
|
+
if (at - lastWriteAt < (config.sleepIdleMinutes ?? 5) * 60000) return false;
|
|
489
|
+
// lastRunAt === 0 means never ran — the min-interval check must not block
|
|
490
|
+
// the very first cycle (a real run stamps a nonzero timestamp).
|
|
491
|
+
if (lastRunAt > 0 && at - lastRunAt < (config.sleepMinIntervalHours ?? 8) * 3600000) return false;
|
|
419
492
|
return true;
|
|
420
493
|
}
|
|
421
494
|
|
|
422
495
|
/** Called on writes: resets the idle clock and re-arms the fire timer. The
|
|
423
496
|
* pending timer is cleared first — a stale timer armed against the old idle
|
|
424
497
|
* window would otherwise fire early, fail shouldRun, and leave nothing armed
|
|
425
|
-
* for the next window (a missed trigger until the next write).
|
|
498
|
+
* for the next window (a missed trigger until the next write).
|
|
499
|
+
*
|
|
500
|
+
* While a sleep run is executing (running=true) the in-flight AbortController
|
|
501
|
+
* is NOT aborted: the run's own writes (demoteToSummary / setArchived ride
|
|
502
|
+
* the normal write-hook path) would otherwise self-abort the cycle. External
|
|
503
|
+
* activity during the run still resets the idle clock here, so no new cycle
|
|
504
|
+
* fires until the store is quiet again. */
|
|
426
505
|
function noteWrite() {
|
|
427
506
|
lastWriteAt = now();
|
|
507
|
+
if (!running && sleepAbort) {
|
|
508
|
+
sleepAbort.abort(); // user resumed activity — interrupt an idle run
|
|
509
|
+
}
|
|
428
510
|
if (idleTimer) {
|
|
429
511
|
clearTimeoutFn(idleTimer);
|
|
430
512
|
idleTimer = null;
|
|
@@ -435,16 +517,19 @@ export function createSleepScheduler({
|
|
|
435
517
|
async function maybeSchedule() {
|
|
436
518
|
if (!shouldRun()) return false;
|
|
437
519
|
running = true;
|
|
520
|
+
const abort = new AbortController();
|
|
521
|
+
sleepAbort = abort;
|
|
438
522
|
try {
|
|
439
523
|
lastRunAt = now();
|
|
440
524
|
const result = await service.enqueue(() =>
|
|
441
|
-
onRun ? onRun() : Promise.resolve({ ok: true, skipped: true })
|
|
525
|
+
onRun ? onRun(abort.signal) : Promise.resolve({ ok: true, skipped: true })
|
|
442
526
|
);
|
|
443
527
|
return !!(result && result.ok);
|
|
444
528
|
} catch (error) {
|
|
445
529
|
logger?.warn?.(`dsh-mneme sleep: run failed: ${error?.message ?? error}`);
|
|
446
530
|
return false;
|
|
447
531
|
} finally {
|
|
532
|
+
sleepAbort = null;
|
|
448
533
|
running = false;
|
|
449
534
|
}
|
|
450
535
|
}
|
|
@@ -455,6 +540,10 @@ export function createSleepScheduler({
|
|
|
455
540
|
clearTimeoutFn(idleTimer);
|
|
456
541
|
idleTimer = null;
|
|
457
542
|
}
|
|
543
|
+
if (sleepAbort) {
|
|
544
|
+
sleepAbort.abort();
|
|
545
|
+
sleepAbort = null;
|
|
546
|
+
}
|
|
458
547
|
}
|
|
459
548
|
|
|
460
549
|
return { noteWrite, maybeSchedule, shouldRun, dispose };
|