dsh-memory-engine 0.0.0-stage → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +163 -2
- package/cordis.patch.yml +3 -0
- package/lib/client.js +78 -0
- package/lib/config.js +21 -0
- package/lib/decay.js +29 -0
- package/lib/dedupe.js +98 -0
- package/lib/file-import.js +61 -0
- package/lib/import.js +191 -0
- package/lib/index.js +179 -0
- package/lib/map.js +57 -0
- package/lib/recall.js +59 -0
- package/lib/render/index.mjs +22 -0
- package/lib/render/layout.mjs +22 -0
- package/lib/render/load.mjs +68 -0
- package/lib/render/render.mjs +222 -0
- package/lib/render/score.mjs +104 -0
- package/lib/serve.js +66 -0
- package/lib/store.js +195 -0
- package/package.json +51 -3
- package/scripts/render-map.mjs +28 -0
- package/scripts/sync-qq.mjs +84 -0
package/README.md
CHANGED
|
@@ -1,3 +1,164 @@
|
|
|
1
|
-
#
|
|
1
|
+
# DSH 记忆引擎
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
DSH 插件的本地记忆引擎,带神经网络可视化。它让 agent 保存、召回和管理跨会话记忆,也能把你已有的记忆文件导进来。
|
|
4
|
+
|
|
5
|
+
当前版本:0.5.1。Node ≥22,零新增依赖;DSH 宿主提供 `@deepseek-ai/dsh-tools@0.2.1-rc.3`。本机安装已启用,无需重复安装。
|
|
6
|
+
|
|
7
|
+
## 界面
|
|
8
|
+
|
|
9
|
+
安装并启用插件后,完全退出 DSH 再打开。进入任意会话,头部第一行会出现 **🧠 记忆图谱** 按钮。
|
|
10
|
+
|
|
11
|
+
点击后全屏显示图谱,右上角 **关闭 ×** 或 **Escape** 退出。每次打开都会读取当前记忆库重新生成;大库会先显示“正在生成”。
|
|
12
|
+
|
|
13
|
+
拖拽旋转、滚轮缩放、点击神经元看原文。服务默认监听 `127.0.0.1:18886`;端口被占用时记忆工具仍可用,可以运行 `node <插件目录>/scripts/render-map.mjs --open`,或直接打开 `~/.dsh/memory-engine/starmap.html`。修改端口需要同时修改 config.json 的 uiPort 和 client.js 的 VIEW_URL,见后文。
|
|
14
|
+
|
|
15
|
+
也可以让 agent 调用 `memory_map()`,生成可双击、离线打开的单文件 HTML:
|
|
16
|
+
|
|
17
|
+
```text
|
|
18
|
+
memory_map()
|
|
19
|
+
memory_map(title="我的机器人的记忆", open=true)
|
|
20
|
+
memory_map(out="D:\\memory-map.html")
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
默认输出 `{DSH_HOME}/memory-engine/starmap.html`;未设置 DSH_HOME 时为 `~/.dsh/memory-engine/starmap.html`。`open` 默认 false,显式 true 才启动系统默认浏览器。
|
|
24
|
+
|
|
25
|
+
## 保存和召回
|
|
26
|
+
|
|
27
|
+
让 agent 使用:
|
|
28
|
+
|
|
29
|
+
```text
|
|
30
|
+
memory_save(text="他说把我当朋友", kind="对话", heat=1)
|
|
31
|
+
memory_recall(q="朋友", limit=5)
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
正文最多 2000 个 Unicode 字符。相似内容会合并,保留旧 id 和更早发生时间。省略 heat 时使用既有情绪推断;heat 的 1/2/3 分别表示热、温、冷。
|
|
35
|
+
|
|
36
|
+
召回结果区分直接命中和联想到的记忆。`memory_stats()` 查看总数和分类;`memory_dedupe()` 默认仅预览重复组。`memory_forget` 支持按 id 精确删除;关键词批量删除必须先预览,并经用户确认后传 `confirm=true`。
|
|
37
|
+
|
|
38
|
+
## 导入自己机器人的旧记忆
|
|
39
|
+
|
|
40
|
+
三种格式都是你明确指定的本地文件或目录。如果旧格式均不匹配,可让有文件读取能力的 agent 理解文件,再通过 `memory_save` 逐条写入。导入先预览,用户确认后再传 `dry_run=false`。
|
|
41
|
+
|
|
42
|
+
`memory_import` 默认只预览,显式 `dry_run=false` 才备份并写入。每种格式都经过同一个相似去重流程;不会修改原文件。发生时间会保留,未提供时由原解析器补默认时间。成功新增后自动生成默认图谱,不自动打开浏览器;渲染失败只记录警告,不撤销已经成功的导入。
|
|
43
|
+
|
|
44
|
+
JSON 数组文件(UTF-8,支持 BOM):
|
|
45
|
+
|
|
46
|
+
```json
|
|
47
|
+
[
|
|
48
|
+
{"text":"他说要一起养猫","ts":1793291223536,"kind":"对话","heat":1},
|
|
49
|
+
{"text":"古茗超A芝士葡萄三分糖","kind":"知识"}
|
|
50
|
+
]
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
```text
|
|
54
|
+
memory_import(source="json", path="D:\\memories.json")
|
|
55
|
+
memory_import(source="json", path="D:\\memories.json", dry_run=false)
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Markdown 目录:每个 `.md` 文件一条记忆,递归读取子目录,支持以下 front-matter。省略 ts 时使用原文件修改时间,文件名作为默认标题。
|
|
59
|
+
|
|
60
|
+
```markdown
|
|
61
|
+
---
|
|
62
|
+
heat: 1
|
|
63
|
+
kind: 对话
|
|
64
|
+
ts: 2026-10-19T12:00:00Z
|
|
65
|
+
---
|
|
66
|
+
他说要一起照顾楼下的流浪猫。
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
```text
|
|
70
|
+
memory_import(source="markdown", path="D:\\memory-notes", dry_run=false)
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
纯文本文件:每个非空行一条,默认分类为“记忆”。
|
|
74
|
+
|
|
75
|
+
```text
|
|
76
|
+
他说把我当朋友
|
|
77
|
+
古茗超A芝士葡萄三分糖
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
```text
|
|
81
|
+
memory_import(source="text", path="D:\\memories.txt", dry_run=false)
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
path 必须是本地绝对路径。单条正文超过 2000 字、日期无效等情况会在写入前报错,避免悄悄丢内容;请先拆分长文。JSON、Markdown 的解析直接复用原工具包,文本只做“逐行转 JSON”的外层适配,未重写解析器。
|
|
85
|
+
|
|
86
|
+
## 图怎么看
|
|
87
|
+
|
|
88
|
+
- **大小**:重要度和连接数量。胞体周围有树突,关联线上有突触脉冲。
|
|
89
|
+
- **亮度**:可视化评分中的留存。越暗表示越容易忘。
|
|
90
|
+
- **位置**:重要知识、印象和重大情绪偏向中心;外围通常是更易淡忘的日常内容。
|
|
91
|
+
- **颜色**:知识绿色、印象浅蓝、日记紫色、对话橙红,其他类型采用原工具包的默认配色。
|
|
92
|
+
- **红色虚环**:提示快忘的记忆;左下角列出留存最低的复习清单。
|
|
93
|
+
|
|
94
|
+
拖动旋转、滚轮缩放、点击神经元查看正文与关联扩散。图谱基于 neuron-memory-map 的脑状椭球和 Canvas 视觉;关联改为关键词倒排索引:从标题和正文提取 2~8 字片段,过滤纯数字和格式噪声,用关键词树匹配较长句子中的已提取片段;拉丁缩写只匹配完整片段。只为共享关键词的文档配对,每对最多 3 条关键词证据,boost 按这些证据累加。
|
|
95
|
+
|
|
96
|
+
关键词至少出现在 2 条记忆中;为避免泛词形成大团,DF 上限收紧为 `min(总数×30%, max(2, 总数×2%))`,实际 298 条时最高 DF 为 5。树突数量 `min(9, 4+round(sqrt(degree)×1.2))`,半径加成 `min(1.4, sqrt(degree)×0.18)`,让连接多的节点保留大小差别。本机当前图有 298 个节点、307 条边,已检查默认视图和点击激活。
|
|
97
|
+
|
|
98
|
+
图中的评分是渲染器的视图值,留存公式没有改;关联和 boost 的输入变化会影响图中留存。agent 召回、衰减、排序和扩散仍使用记忆引擎原有公式,不会因浏览图谱改变。
|
|
99
|
+
|
|
100
|
+
## 数据存在哪
|
|
101
|
+
|
|
102
|
+
所有数据保存在 `{DSH_HOME}/memory-engine/`,默认 `~/.dsh/memory-engine/`:
|
|
103
|
+
|
|
104
|
+
记忆库默认是 `~/.dsh/memory-engine/memories.json`,图是 `~/.dsh/memory-engine/starmap.html`,配置是 `~/.dsh/memory-engine/config.json`。
|
|
105
|
+
|
|
106
|
+
**记忆存储、导入和图谱生成不上传记忆,不访问外部网络,不将数据发往远程服务器。** GUI 仅访问本机回环图谱服务。除本地配置/记忆目录、你明确指定的导入路径和配置的可选 QQ 来源之外,插件不会扫描其他个人文件。通用导入的格式适配可能使用系统临时目录,完成后自动清理。agent 使用外部模型时,调用上下文和工具结果的发送仍由 DSH 自身配置管理。
|
|
107
|
+
|
|
108
|
+
| 文件 | 内容 |
|
|
109
|
+
|---|---|
|
|
110
|
+
| memories.json | 记忆库 |
|
|
111
|
+
| starmap.html | 最新离线图谱 |
|
|
112
|
+
| config.json | 本机可选配置 |
|
|
113
|
+
| sync.log | QQ 增量同步日志 |
|
|
114
|
+
| memories.json.bak-* | 导入/去重前的备份 |
|
|
115
|
+
|
|
116
|
+
记忆和图谱只写本机,生成过程不调用外部服务,也不需要联网。记忆正文中的网址按普通文本保留,不作为图谱资源加载。
|
|
117
|
+
|
|
118
|
+
示例配置(普通用户只需默认 uiPort):
|
|
119
|
+
|
|
120
|
+
```json
|
|
121
|
+
{ "uiPort": 18886 }
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
图谱服务只监听 `127.0.0.1`,不提供 memories.json/config.json 等原始文件,只允许根页面、index.html/starmap.html 和 pages/N.html。端口被占用时只记录警告,agent 记忆工具仍可用,也可用离线 HTML。
|
|
125
|
+
|
|
126
|
+
如需改端口,请同时修改 config.json 的 uiPort 和 `lib/client.js` 中 VIEW_URL 的端口,再完全退出并重开 DSH。默认二者均为 **18886**,没有自动发现机制。
|
|
127
|
+
|
|
128
|
+
## QQ 桥接适配器
|
|
129
|
+
|
|
130
|
+
`source="qq-bridge"` 是可选的私有格式适配器,普通用户导入 JSON/Markdown/text 时不需要配置它。插件不会改桥接文件,也不会代替桥接自身的存储。
|
|
131
|
+
|
|
132
|
+
适配器从 `QQ_BRIDGE_DIR` 环境变量取目录;未设置或无有效目录时再试本机 config.json 的 bridgeRoot,均不可用时给出配置提示。包内已去掉机器专属的桥接根路径。
|
|
133
|
+
|
|
134
|
+
```json
|
|
135
|
+
{ "uiPort": 18886, "bridgeRoot": "D:/my-bot/qq-bridge-main" }
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
日记来源也需明确配置:`QQ_DIARY_DIR` 环境变量 → config.json 的 `diaryDir` → 跳过。例如在上面的配置对象中添加 `"diaryDir": "D:/my-bot/diaries"`。未配置时返回“未配置日记目录,跳过日记导入”,其他类别仍正常导入;包内不带任何用户日记路径。
|
|
139
|
+
|
|
140
|
+
```text
|
|
141
|
+
memory_import(source="qq-bridge")
|
|
142
|
+
memory_import(source="qq-bridge", dry_run=false)
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
知识、印象、日记、重大情绪等来源保持既有规则。history 逐条按原始时间戳的本地日期分组:今天仅 self=false、至少 8 字,取最近 60 条;历史取情绪最强 6 条;跨文件先去重再导入。
|
|
146
|
+
|
|
147
|
+
已有桥接调度器可继续调用 `scripts/sync-qq.mjs`。脚本同样读取环境变量/config.json,无增量时不改库;`--dry` 完全不写盘。同步备份仅保留最近 5 份,sync.log 保留最近 500 行。图谱打开时按需重建,所以同步后的记忆会在下次打开时显示。
|
|
148
|
+
|
|
149
|
+
## 命令行和测试
|
|
150
|
+
|
|
151
|
+
在插件目录使用 Node ≥22:
|
|
152
|
+
|
|
153
|
+
```powershell
|
|
154
|
+
node scripts/render-map.mjs
|
|
155
|
+
node scripts/render-map.mjs --out D:\map.html --title "我的记忆" --open
|
|
156
|
+
node scripts/sync-qq.mjs --dry
|
|
157
|
+
node --test test/*.test.mjs
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
通过包安装生成命令后也可运行 `dsh-memory-map`。测试使用隔离的临时记忆库。现有 33 项回归按新增 GUI/第七个工具更新相应声明断言,加上 8 项可视化、导入、服务及 CLI 测试,以及 4 项日记配置/关键词/平方根/性能测试,总共 45 项通过。
|
|
161
|
+
|
|
162
|
+
本地开发安装:将本包作为目标 DSH profile 的 link 依赖,并启用同名 bundle;客户端入口已声明为 web/immediately。完全退出再打开 DSH。不要复制宿主 SDK 到包里。
|
|
163
|
+
|
|
164
|
+
文件写入仍继承原 MemoryStore 的单进程同步事务方式;多个独立宿主同时修改同一文件不提供跨进程锁。历史验收和本次截图、反向验证 HTML、文件变更清单位于相邻的 dsh-memory-engine-deliveries 目录。
|
package/cordis.patch.yml
ADDED
package/lib/client.js
ADDED
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
window.__ModuleLoader__.load({
|
|
2
|
+
id: "dsh-memory-engine",
|
|
3
|
+
factory: function (require) {
|
|
4
|
+
var module = { exports: {} };
|
|
5
|
+
var exports = module.exports;
|
|
6
|
+
var React = require("react");
|
|
7
|
+
var ReactDOM = require("react-dom");
|
|
8
|
+
// 2026-10-30:从 views.actions 挪到 header.actions。
|
|
9
|
+
// views.actions 是「更多视图」下拉菜单的行(Chat/Trajectory 旁边的 ⋯),
|
|
10
|
+
// 注册进去只是菜单里多一行,看不见;header.actions 才是头部正常按钮区。
|
|
11
|
+
var SLOT_NAME = "conversation.session.header.actions";
|
|
12
|
+
var VIEW_URL = "http://127.0.0.1:18886/index.html";
|
|
13
|
+
|
|
14
|
+
function MemoryButton() {
|
|
15
|
+
var state = React.useState(false);
|
|
16
|
+
var open = state[0];
|
|
17
|
+
var setOpen = state[1];
|
|
18
|
+
var frame = React.useRef(null);
|
|
19
|
+
React.useEffect(function () {
|
|
20
|
+
if (!open) return undefined;
|
|
21
|
+
function onKey(event) { if (event.key === "Escape") setOpen(false); }
|
|
22
|
+
function onMessage(event) {
|
|
23
|
+
if (event.source === (frame.current && frame.current.contentWindow) && event.data && event.data.type === "dsh-memory-engine-close") setOpen(false);
|
|
24
|
+
}
|
|
25
|
+
window.addEventListener("keydown", onKey);
|
|
26
|
+
window.addEventListener("message", onMessage);
|
|
27
|
+
return function () { window.removeEventListener("keydown", onKey); window.removeEventListener("message", onMessage); };
|
|
28
|
+
}, [open]);
|
|
29
|
+
var button = React.createElement("button", {
|
|
30
|
+
type: "button",
|
|
31
|
+
onClick: function () { setOpen(Date.now()); },
|
|
32
|
+
"data-memory-engine-map": "true",
|
|
33
|
+
"aria-label": "打开记忆图谱",
|
|
34
|
+
title: "记忆图谱",
|
|
35
|
+
style: { display: "inline-flex", alignItems: "center", gap: "5px", padding: "5px 9px", borderRadius: "8px", border: "1px solid rgba(130,160,190,.4)", background: "transparent", color: "inherit", font: "inherit", cursor: "pointer" },
|
|
36
|
+
}, React.createElement("span", { "aria-hidden": "true" }, "🧠"), "记忆图谱");
|
|
37
|
+
if (!open) return button;
|
|
38
|
+
var overlay = React.createElement("div", {
|
|
39
|
+
role: "dialog",
|
|
40
|
+
"aria-modal": "true",
|
|
41
|
+
"aria-label": "记忆图谱",
|
|
42
|
+
style: { position: "fixed", inset: 0, zIndex: 9999, background: "#03060c" },
|
|
43
|
+
}, React.createElement("iframe", {
|
|
44
|
+
src: VIEW_URL + "?v=" + open,
|
|
45
|
+
ref: frame,
|
|
46
|
+
title: "记忆神经网络",
|
|
47
|
+
sandbox: "allow-scripts",
|
|
48
|
+
referrerPolicy: "no-referrer",
|
|
49
|
+
style: { display: "block", width: "100%", height: "100%", border: 0 },
|
|
50
|
+
}), React.createElement("button", {
|
|
51
|
+
type: "button",
|
|
52
|
+
onClick: function () { setOpen(false); },
|
|
53
|
+
"aria-label": "关闭记忆图谱",
|
|
54
|
+
style: { position: "absolute", top: "16px", right: "18px", zIndex: 1, padding: "7px 12px", borderRadius: "8px", border: "1px solid #6999b8", background: "#102236", color: "#e2f4ff", font: "inherit", cursor: "pointer" },
|
|
55
|
+
}, "关闭 ×"));
|
|
56
|
+
return React.createElement(React.Fragment, null, button, ReactDOM.createPortal(overlay, document.body));
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
function apply(ctx) {
|
|
60
|
+
try {
|
|
61
|
+
ctx.slots.inject(SLOT_NAME, function () {
|
|
62
|
+
return ctx.slots.register({
|
|
63
|
+
name: SLOT_NAME,
|
|
64
|
+
id: "memory-engine-map",
|
|
65
|
+
order: 50,
|
|
66
|
+
label: function () { return "记忆图谱"; },
|
|
67
|
+
}, MemoryButton);
|
|
68
|
+
});
|
|
69
|
+
} catch (error) {
|
|
70
|
+
try { console.error("[dsh-memory-engine] 插槽注册失败:", error); } catch (ignored) {}
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
exports.apply = apply;
|
|
75
|
+
exports.inject = ["slots"];
|
|
76
|
+
return module.exports;
|
|
77
|
+
}
|
|
78
|
+
});
|
package/lib/config.js
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
import fs from 'node:fs';
|
|
2
|
+
import os from 'node:os';
|
|
3
|
+
import path from 'node:path';
|
|
4
|
+
|
|
5
|
+
export const defaultHome = () => process.env.DSH_HOME || path.join(os.homedir(), '.dsh');
|
|
6
|
+
|
|
7
|
+
export function resolveDiaryDir({ home = defaultHome(), envDir = process.env.QQ_DIARY_DIR } = {}) {
|
|
8
|
+
return envDir || readConfig(home).diaryDir || '';
|
|
9
|
+
}
|
|
10
|
+
|
|
11
|
+
export function readConfig(home = defaultHome()) {
|
|
12
|
+
const file = path.join(home, 'memory-engine', 'config.json');
|
|
13
|
+
try {
|
|
14
|
+
const value = JSON.parse(fs.readFileSync(file, 'utf8').replace(/^\uFEFF/u, ''));
|
|
15
|
+
if (!value || typeof value !== 'object' || Array.isArray(value)) throw new Error('配置必须是 JSON 对象');
|
|
16
|
+
return value;
|
|
17
|
+
} catch (error) {
|
|
18
|
+
if (error.code === 'ENOENT') return {};
|
|
19
|
+
throw new Error(`无法读取 ${file}:${error.message}`);
|
|
20
|
+
}
|
|
21
|
+
}
|
package/lib/decay.js
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
export const DAY = 86400000;
|
|
2
|
+
export const HALF_LIFE = Object.freeze({ 1: 30, 2: 12, 3: 6 });
|
|
3
|
+
export const EVENT_HALF_LIFE = 3;
|
|
4
|
+
export const EMOTION_HALF_LIFE = 90;
|
|
5
|
+
|
|
6
|
+
const EMOTIONS = [
|
|
7
|
+
{ tag: '离别', w: 3, words: ['心疼', '舍不得', '告别', '再也', '留不住'] },
|
|
8
|
+
{ tag: '身份', w: 3, words: ['朋友', '不是替身', '自由', '浪漫', '身份'] },
|
|
9
|
+
{ tag: '正向', w: 2, words: ['高兴', '喜欢', '谢谢', '记住了', '暖'] },
|
|
10
|
+
{ tag: '负向', w: 2, words: ['难过', '难受', '烦', '累', '孤独', '委屈'] },
|
|
11
|
+
{ tag: '承诺', w: 2, words: ['别忘', '记住', '答应', '承诺', '绝对'] },
|
|
12
|
+
];
|
|
13
|
+
|
|
14
|
+
export const chars = (text, count) => Array.from(text).slice(0, count).join('');
|
|
15
|
+
export const titleOf = memory => memory.title || chars(memory.text, 20);
|
|
16
|
+
|
|
17
|
+
export function inferEmotion(text) {
|
|
18
|
+
const emotion = EMOTIONS.filter(group => group.words.some(word => text.includes(word)))
|
|
19
|
+
.map(({ tag, w }) => ({ tag, w }));
|
|
20
|
+
return { emotion, heat: emotion.length ? 1 : 2 };
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
export function retention(memory, now = Date.now()) {
|
|
24
|
+
const halfLife = memory.emotion?.some(mark => mark.w >= 3) ? EMOTION_HALF_LIFE
|
|
25
|
+
: memory.kind === '事件' ? EVENT_HALF_LIFE : HALF_LIFE[memory.heat] ?? HALF_LIFE[2];
|
|
26
|
+
const ageDays = (now - memory.ts) / DAY;
|
|
27
|
+
const boost = 0.5 * (1 - Math.exp(-(memory.boosts ?? 0) / 4));
|
|
28
|
+
return Math.min(1.15, Math.exp(-ageDays / halfLife) + boost);
|
|
29
|
+
}
|
package/lib/dedupe.js
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
// Unicode characters (not UTF-16 halves) are used for lengths and LCS.
|
|
2
|
+
export const norm = value => String(value)
|
|
3
|
+
.replace(/[\s,。!?、;:""''()《》【】,.!?;:"'()<>\[\]-]/gu, '').toLowerCase();
|
|
4
|
+
|
|
5
|
+
export function prepare(text) {
|
|
6
|
+
const normalized = norm(text);
|
|
7
|
+
const all = Array.from(normalized);
|
|
8
|
+
const prefix = all.slice(0, 500);
|
|
9
|
+
const counts = new Map();
|
|
10
|
+
for (const char of prefix) counts.set(char, (counts.get(char) ?? 0) + 1);
|
|
11
|
+
return { normalized, length: all.length, prefix, counts, set: new Set(all) };
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
// Rolling DP uses at most 501 Uint16 values rather than a 500x500 matrix.
|
|
15
|
+
export function lcsLength(a, b) {
|
|
16
|
+
if (a.length > b.length) [a, b] = [b, a];
|
|
17
|
+
const dp = new Uint16Array(a.length + 1);
|
|
18
|
+
for (const right of b) {
|
|
19
|
+
let previous = 0;
|
|
20
|
+
for (let i = 1; i <= a.length; i++) {
|
|
21
|
+
const above = dp[i];
|
|
22
|
+
dp[i] = a[i - 1] === right ? previous + 1 : Math.max(above, dp[i - 1]);
|
|
23
|
+
previous = above;
|
|
24
|
+
}
|
|
25
|
+
}
|
|
26
|
+
return dp[a.length];
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
export function similarity(a, b, threshold = 0.7) {
|
|
30
|
+
const shorter = Math.min(a.length, b.length);
|
|
31
|
+
if (!shorter) return { duplicate: a.normalized === b.normalized, score: a.normalized === b.normalized ? 1 : 0, rule: 'exact' };
|
|
32
|
+
if (a.normalized === b.normalized) return { duplicate: true, score: 1, rule: 'exact' };
|
|
33
|
+
if (shorter >= 6 && (a.normalized.includes(b.normalized) || b.normalized.includes(a.normalized))) {
|
|
34
|
+
return { duplicate: true, score: 1, rule: 'containment' };
|
|
35
|
+
}
|
|
36
|
+
let intersection = 0;
|
|
37
|
+
const [small, large] = a.set.size <= b.set.size ? [a.set, b.set] : [b.set, a.set];
|
|
38
|
+
for (const char of small) if (large.has(char)) intersection++;
|
|
39
|
+
const jaccard = intersection / (a.set.size + b.set.size - intersection);
|
|
40
|
+
if (shorter >= 10 && jaccard >= Math.max(0.85, threshold)) return { duplicate: true, score: jaccard, rule: 'jaccard' };
|
|
41
|
+
// A multiset intersection bounds LCS from above, allowing an exact early
|
|
42
|
+
// rejection without changing any matching thresholds.
|
|
43
|
+
const prefixShorter = Math.min(a.prefix.length, b.prefix.length);
|
|
44
|
+
let upper = 0;
|
|
45
|
+
const [few, many] = a.counts.size <= b.counts.size ? [a.counts, b.counts] : [b.counts, a.counts];
|
|
46
|
+
for (const [char, count] of few) upper += Math.min(count, many.get(char) ?? 0);
|
|
47
|
+
if (upper < prefixShorter * threshold) return { duplicate: false, score: 0, rule: 'lcs' };
|
|
48
|
+
const score = lcsLength(a.prefix, b.prefix) / prefixShorter;
|
|
49
|
+
return { duplicate: score >= threshold, score, rule: 'lcs' };
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
export function findDuplicate(rows, candidate, prepared = rows.map(row => prepare(row.text))) {
|
|
53
|
+
const text = prepare(candidate.text);
|
|
54
|
+
for (let index = 0; index < rows.length; index++) {
|
|
55
|
+
const match = similarity(prepared[index], text);
|
|
56
|
+
if (match.duplicate) return { index, ...match };
|
|
57
|
+
}
|
|
58
|
+
return null;
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
export function mergeMemories(old, incoming) {
|
|
62
|
+
const longer = Array.from(incoming.text).length > Array.from(old.text).length;
|
|
63
|
+
const emotion = new Map([...old.emotion, ...incoming.emotion].map(mark => [`${mark.tag}\0${mark.w}`, { ...mark }]));
|
|
64
|
+
return { ...old, text: longer ? incoming.text : old.text,
|
|
65
|
+
...(longer && incoming.title ? { title: incoming.title } : {}),
|
|
66
|
+
boosts: old.boosts + incoming.boosts, ts: Math.min(old.ts, incoming.ts),
|
|
67
|
+
heat: Math.min(old.heat, incoming.heat), emotion: [...emotion.values()],
|
|
68
|
+
...(old.who?.trim() ? { who: old.who } : incoming.who?.trim() ? { who: incoming.who } : {}),
|
|
69
|
+
createdAt: Math.min(old.createdAt, incoming.createdAt) };
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
export function duplicateGroups(rows, threshold = 0.7) {
|
|
73
|
+
if (!Number.isFinite(threshold) || threshold < 0.5 || threshold > 0.95) throw new Error('threshold 必须在 0.5–0.95 之间');
|
|
74
|
+
const texts = rows.map(row => prepare(row.text));
|
|
75
|
+
const parent = rows.map((_, i) => i);
|
|
76
|
+
function root(i) {
|
|
77
|
+
while (parent[i] !== i) { parent[i] = parent[parent[i]]; i = parent[i]; }
|
|
78
|
+
return i;
|
|
79
|
+
}
|
|
80
|
+
for (let i = 0; i < rows.length; i++) {
|
|
81
|
+
for (let j = i + 1; j < rows.length; j++) {
|
|
82
|
+
const a = root(i), b = root(j);
|
|
83
|
+
if (a !== b && similarity(texts[i], texts[j], threshold).duplicate) parent[b] = a;
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
const groups = new Map();
|
|
87
|
+
for (let i = 0; i < rows.length; i++) {
|
|
88
|
+
const key = root(i);
|
|
89
|
+
if (!groups.has(key)) groups.set(key, []);
|
|
90
|
+
groups.get(key).push(i);
|
|
91
|
+
}
|
|
92
|
+
return [...groups.values()].filter(group => group.length > 1).map(indices => {
|
|
93
|
+
// Retain the oldest persisted identity; ties keep the original array order.
|
|
94
|
+
indices.sort((a, b) => rows[a].createdAt - rows[b].createdAt || a - b);
|
|
95
|
+
const memory = indices.slice(1).reduce((merged, index) => mergeMemories(merged, rows[index]), rows[indices[0]]);
|
|
96
|
+
return { indices, memory };
|
|
97
|
+
});
|
|
98
|
+
}
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
import fs from 'node:fs/promises';
|
|
2
|
+
import os from 'node:os';
|
|
3
|
+
import path from 'node:path';
|
|
4
|
+
import { loadMemories } from './render/index.mjs';
|
|
5
|
+
import { inferEmotion } from './decay.js';
|
|
6
|
+
|
|
7
|
+
export async function collectFile({ source, path: input }, now = Date.now()) {
|
|
8
|
+
if (!['json', 'markdown', 'text'].includes(source)) throw new Error('source 支持 qq-bridge / json / markdown / text');
|
|
9
|
+
if (typeof input !== 'string' || !path.isAbsolute(input)) throw new Error('path 必须是本地文件或目录的绝对路径');
|
|
10
|
+
const info = await fs.stat(input);
|
|
11
|
+
if (source === 'markdown' ? !info.isDirectory() : !info.isFile()) throw new Error(source === 'markdown' ? 'markdown 必须指定目录' : 'json/text 必须指定文件');
|
|
12
|
+
let temporary;
|
|
13
|
+
try {
|
|
14
|
+
let loaderInput = input;
|
|
15
|
+
if (source === 'markdown') {
|
|
16
|
+
const files = (await fs.readdir(input, { recursive: true, withFileTypes: true }))
|
|
17
|
+
.filter(entry => entry.isFile() && path.extname(entry.name).toLowerCase() === '.md')
|
|
18
|
+
.map(entry => path.join(entry.parentPath, entry.name));
|
|
19
|
+
const contents = await Promise.all(files.map(async file => ({ file, text: await fs.readFile(file, 'utf8'), info: await fs.stat(file) })));
|
|
20
|
+
if (contents.some(item => item.text.startsWith('\uFEFF'))) {
|
|
21
|
+
temporary = await fs.mkdtemp(path.join(os.tmpdir(), 'dsh-memory-input-'));
|
|
22
|
+
loaderInput = temporary;
|
|
23
|
+
for (const item of contents) {
|
|
24
|
+
const target = path.join(temporary, path.relative(input, item.file));
|
|
25
|
+
await fs.mkdir(path.dirname(target), { recursive: true });
|
|
26
|
+
await fs.writeFile(target, item.text.replace(/^\uFEFF/u, ''));
|
|
27
|
+
await fs.utimes(target, item.info.atime, item.info.mtime);
|
|
28
|
+
}
|
|
29
|
+
}
|
|
30
|
+
} else {
|
|
31
|
+
const content = (await fs.readFile(input, 'utf8')).replace(/^\uFEFF/u, '');
|
|
32
|
+
// The original loader accepts JSON and Markdown. Text lines go through
|
|
33
|
+
// its JSON parser as well; the vendored parser and renderer stay intact.
|
|
34
|
+
temporary = await fs.mkdtemp(path.join(os.tmpdir(), 'dsh-memory-input-'));
|
|
35
|
+
loaderInput = path.join(temporary, 'input.json');
|
|
36
|
+
const data = source === 'text' ? content.split(/\r?\n/u).map(text => text.trim()).filter(Boolean).map(text => ({ text })) : JSON.parse(content);
|
|
37
|
+
if (Array.isArray(data) && data.length === 0) return { entries: [], counts: {}, warnings: [], root: input };
|
|
38
|
+
await fs.writeFile(loaderInput, JSON.stringify(data));
|
|
39
|
+
}
|
|
40
|
+
const items = await loadMemories(loaderInput, now);
|
|
41
|
+
const counts = Object.create(null);
|
|
42
|
+
const entries = items.map(({ id: _id, text, ts, ...fields }) => {
|
|
43
|
+
if (Array.from(text).length > 2000) throw new Error('导入正文最多 2000 字;请先拆分长记忆');
|
|
44
|
+
const inferred = inferEmotion(text);
|
|
45
|
+
const kind = fields.kind;
|
|
46
|
+
counts[kind] = (counts[kind] ?? 0) + 1;
|
|
47
|
+
return { ...fields, text, ts: Date.parse(ts), heat: fields.heat ?? inferred.heat, emotion: inferred.emotion,
|
|
48
|
+
boosts: 0, category: kind, origin: { source, file: input } };
|
|
49
|
+
});
|
|
50
|
+
return { entries, counts, warnings: [], root: input };
|
|
51
|
+
} finally {
|
|
52
|
+
if (temporary) {
|
|
53
|
+
// Verify the resolved target is our single mkdtemp child before cleanup.
|
|
54
|
+
const target = path.resolve(temporary);
|
|
55
|
+
if (path.dirname(target) !== path.resolve(os.tmpdir()) || !path.basename(target).startsWith('dsh-memory-input-')) {
|
|
56
|
+
throw new Error('临时导入目录不在预期位置,未执行清理');
|
|
57
|
+
}
|
|
58
|
+
await fs.rm(target, { recursive: true, force: true });
|
|
59
|
+
}
|
|
60
|
+
}
|
|
61
|
+
}
|
package/lib/import.js
ADDED
|
@@ -0,0 +1,191 @@
|
|
|
1
|
+
// This module only reads source files. All writes belong to MemoryStore's
|
|
2
|
+
// independent memory-engine directory, never to the bridge or diary paths.
|
|
3
|
+
import { readFileSync, readdirSync, statSync } from 'node:fs';
|
|
4
|
+
import path from 'node:path';
|
|
5
|
+
import { chars, inferEmotion } from './decay.js';
|
|
6
|
+
import { norm } from './dedupe.js';
|
|
7
|
+
import { readConfig, resolveDiaryDir } from './config.js';
|
|
8
|
+
|
|
9
|
+
export const DEFAULT_BRIDGE_ROOT = process.env.QQ_BRIDGE_DIR || '';
|
|
10
|
+
export const DEFAULT_DIARY_DIR = process.env.QQ_DIARY_DIR || '';
|
|
11
|
+
const WORDS = [
|
|
12
|
+
[/心疼|舍不得|怕你|别走|留不住|告别|最后一次|再也/u, 3],
|
|
13
|
+
[/开心|高兴|喜欢|谢谢|谢谢你我|记住了|暖|舒坦/u, 2],
|
|
14
|
+
[/难过|难受|烦|闷|累|孤独|空了一秒|想哭|委屈/u, 2],
|
|
15
|
+
[/别忘|记住|记着|答应|承诺|一定|绝对/u, 2],
|
|
16
|
+
[/朋友|当朋友|不是替身|不是复印件|自由|浪漫/u, 3],
|
|
17
|
+
];
|
|
18
|
+
export const emotionStrength = text => Math.max(0, ...WORDS.filter(([re]) => re.test(text)).map(([, w]) => w));
|
|
19
|
+
const textValue = value => typeof value === 'string' ? value.trim() : '';
|
|
20
|
+
|
|
21
|
+
export function parseTime(value) {
|
|
22
|
+
if (value === null || value === undefined || value === '') return null;
|
|
23
|
+
const time = typeof value === 'number' ? value : /^\d+(?:\.\d+)?$/u.test(String(value)) ? Number(value) : Date.parse(value);
|
|
24
|
+
return Number.isFinite(time) && Math.abs(time) <= 8640000000000000 ? time : null;
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
export function resolveBridgeRoot({ defaultRoot = DEFAULT_BRIDGE_ROOT, envRoot = process.env.QQ_BRIDGE_DIR, home } = {}) {
|
|
28
|
+
for (const candidate of [() => envRoot, () => defaultRoot, () => readConfig(home).bridgeRoot]) {
|
|
29
|
+
const root = candidate();
|
|
30
|
+
if (!root) continue;
|
|
31
|
+
try { if (statSync(path.join(root, 'state')).isDirectory()) return path.resolve(root); } catch { /* try the next path */ }
|
|
32
|
+
}
|
|
33
|
+
return null;
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
export function collectBridge(options = {}) {
|
|
37
|
+
const root = resolveBridgeRoot(options);
|
|
38
|
+
if (!root) return { missing: true, entries: [], warnings: [], counts: {} };
|
|
39
|
+
const warnings = [];
|
|
40
|
+
const readJson = (file, fallback) => {
|
|
41
|
+
try { return JSON.parse(readFileSync(file, 'utf8').replace(/^\uFEFF/u, '')); }
|
|
42
|
+
catch (error) { warnings.push(`${file}:${error.code === 'ENOENT' ? '未找到,已略过' : '无法读取或 JSON 损坏,已略过'}`); return fallback; }
|
|
43
|
+
};
|
|
44
|
+
const files = directory => {
|
|
45
|
+
try { return readdirSync(directory, { withFileTypes: true }).filter(entry => entry.isFile()).map(entry => entry.name).sort(); }
|
|
46
|
+
catch { warnings.push(`${directory}:无法读取目录,已略过`); return []; }
|
|
47
|
+
};
|
|
48
|
+
const state = path.join(root, 'state');
|
|
49
|
+
const snapshot = readJson(path.join(state, 'starmap/stars.json'), {});
|
|
50
|
+
const stars = Array.isArray(snapshot?.stars) ? snapshot.stars : [];
|
|
51
|
+
const counters = readJson(path.join(state, 'starmap/retrieval-boosts.json'), {});
|
|
52
|
+
const boostCount = id => Number.isSafeInteger(counters?.[id]) && counters[id] >= 0 ? counters[id] : 0;
|
|
53
|
+
const textBoosts = new Map();
|
|
54
|
+
for (const star of stars) {
|
|
55
|
+
if (!textValue(star?.text)) continue;
|
|
56
|
+
const count = boostCount(star.id);
|
|
57
|
+
const key = norm(star.text);
|
|
58
|
+
textBoosts.set(key, Math.max(textBoosts.get(key) ?? 0, count));
|
|
59
|
+
}
|
|
60
|
+
const entries = [];
|
|
61
|
+
const counts = { 知识: 0, 印象: 0, 日记: 0, 对话: 0, 重大情绪: 0, 俚语: 0 };
|
|
62
|
+
let invalid = 0;
|
|
63
|
+
function add(category, raw, sourceFile, bridgeIds = []) {
|
|
64
|
+
const text = chars(textValue(raw.text), 2000);
|
|
65
|
+
const ts = parseTime(raw.ts);
|
|
66
|
+
if (!text || ts === null) { invalid++; return; }
|
|
67
|
+
const inferred = inferEmotion(text);
|
|
68
|
+
const emotion = Array.isArray(raw.emotion) ? raw.emotion.filter(mark => mark && typeof mark.tag === 'string' && Number.isFinite(mark.w)) : inferred.emotion;
|
|
69
|
+
// Diary/event stars may store shortened bodies. Match their exact source
|
|
70
|
+
// text prefix as well as the full normalized body; never use topic alone.
|
|
71
|
+
let boosts = Math.max(0, ...bridgeIds.map(boostCount), textBoosts.get(norm(text)) ?? 0);
|
|
72
|
+
for (const star of stars) {
|
|
73
|
+
const sourceText = textValue(star?.text);
|
|
74
|
+
if (!sourceText || sourceText.length < 12) continue;
|
|
75
|
+
const key = norm(sourceText);
|
|
76
|
+
if (norm(text).startsWith(key)) boosts = Math.max(boosts, boostCount(star.id));
|
|
77
|
+
}
|
|
78
|
+
entries.push({ text, kind: raw.kind, ts, heat: raw.heat ?? inferred.heat, emotion,
|
|
79
|
+
...(textValue(raw.who) ? { who: textValue(raw.who) } : {}),
|
|
80
|
+
...(textValue(raw.title) ? { title: textValue(raw.title) } : {}), boosts,
|
|
81
|
+
origin: { source: 'qq-bridge', file: sourceFile }, category });
|
|
82
|
+
counts[category]++;
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
const knowledge = readJson(path.join(state, 'knowledge.json'), []);
|
|
86
|
+
if (!Array.isArray(knowledge)) warnings.push('knowledge.json:应为数组,已略过');
|
|
87
|
+
for (const item of Array.isArray(knowledge) ? knowledge : []) {
|
|
88
|
+
if (!textValue(item?.topic) || !textValue(item?.fact)) { invalid++; continue; }
|
|
89
|
+
add('知识', { text: `${item.topic}:${item.fact}`, kind: '知识', ts: item.at, title: item.topic },
|
|
90
|
+
'state/knowledge.json', [`know-${item.topic}-${item.at}`]);
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
const social = readJson(path.join(state, 'social-v2.json'), {});
|
|
94
|
+
for (const [conversationId, conversation] of Object.entries(social?.conversations ?? {})) {
|
|
95
|
+
for (const [memberKey, member] of Object.entries(conversation?.memberImpressions ?? {})) {
|
|
96
|
+
if (!Array.isArray(member?.traits)) continue;
|
|
97
|
+
const who = textValue(member.name) || memberKey;
|
|
98
|
+
member.traits.forEach((trait, index) => {
|
|
99
|
+
if (typeof trait !== 'string') { invalid++; return; }
|
|
100
|
+
add('印象', { text: trait, kind: '印象', ts: member.lastSeenAt, who }, 'state/social-v2.json',
|
|
101
|
+
[`imp-${conversationId}-${memberKey}-${index}`, `imp-${conversationId}-${who}-${index}`, `imp-${memberKey}-${index}`, `imp-${who}-${index}`]);
|
|
102
|
+
});
|
|
103
|
+
}
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
// Preserve major-emotion event records before importing the overlapping
|
|
107
|
+
// daily conversation excerpts, so the event's heat/emotion are retained.
|
|
108
|
+
for (const star of stars.filter(star => star?.layer === 'emotion')) {
|
|
109
|
+
const emotion = Array.isArray(star.emotion) ? star.emotion.filter(mark => mark && typeof mark.tag === 'string' && Number.isFinite(mark.w)) : [];
|
|
110
|
+
if (!emotion.some(mark => mark.w >= 3)) emotion.push({ tag: '重大情绪', w: 3 });
|
|
111
|
+
add('重大情绪', { text: star.text, kind: '事件', ts: star.ts, heat: 1, emotion, title: star.title, who: star.who },
|
|
112
|
+
'state/starmap/stars.json', [star.id]);
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
const diaryDir = options.diaryDir ?? resolveDiaryDir(options);
|
|
116
|
+
if (!diaryDir) warnings.push('未配置日记目录,跳过日记导入');
|
|
117
|
+
for (const file of (diaryDir ? files(diaryDir) : []).filter(name => !name.startsWith('_') && name.toLowerCase().endsWith('.md'))) {
|
|
118
|
+
const full = path.join(diaryDir, file);
|
|
119
|
+
try {
|
|
120
|
+
add('日记', { text: readFileSync(full, 'utf8'), kind: '日记', ts: statSync(full).mtimeMs, title: path.basename(file, path.extname(file)) },
|
|
121
|
+
full, [`diary-${file}`]);
|
|
122
|
+
} catch { warnings.push(`${file}:日记无法读取,已略过`); }
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
const days = new Map();
|
|
126
|
+
const seen = new Set();
|
|
127
|
+
const localDate = value => {
|
|
128
|
+
const date = new Date(value);
|
|
129
|
+
return `${date.getFullYear()}-${String(date.getMonth() + 1).padStart(2, '0')}-${String(date.getDate()).padStart(2, '0')}`;
|
|
130
|
+
};
|
|
131
|
+
const today = localDate(Date.now());
|
|
132
|
+
const todayMessages = [];
|
|
133
|
+
for (const file of files(path.join(state, 'history')).filter(name => name.endsWith('.jsonl'))) {
|
|
134
|
+
let lines;
|
|
135
|
+
try { lines = readFileSync(path.join(state, 'history', file), 'utf8').replace(/^\uFEFF/u, '').split(/\r?\n/u); }
|
|
136
|
+
catch { warnings.push(`${file}:历史文件无法读取,已略过`); continue; }
|
|
137
|
+
let broken = 0;
|
|
138
|
+
for (const line of lines) {
|
|
139
|
+
if (!line.trim()) continue;
|
|
140
|
+
let item;
|
|
141
|
+
try { item = JSON.parse(line); } catch { broken++; continue; }
|
|
142
|
+
const text = textValue(item?.text);
|
|
143
|
+
const ts = parseTime(item?.t);
|
|
144
|
+
if (ts === null) continue;
|
|
145
|
+
// Shards reflect write time (UTC), not the message's local calendar day.
|
|
146
|
+
const day = localDate(ts);
|
|
147
|
+
const isToday = day === today;
|
|
148
|
+
if (Array.from(text).length < (isToday ? 8 : 12) || (isToday && item.self !== false)) continue;
|
|
149
|
+
const strength = emotionStrength(text);
|
|
150
|
+
if (!isToday && strength < 2) continue;
|
|
151
|
+
const key = `${ts}\0${norm(text)}`;
|
|
152
|
+
if (seen.has(key)) continue;
|
|
153
|
+
seen.add(key);
|
|
154
|
+
if (isToday) {
|
|
155
|
+
todayMessages.push({ text, ts, who: item.who, file });
|
|
156
|
+
todayMessages.sort((a, b) => b.ts - a.ts);
|
|
157
|
+
if (todayMessages.length > 60) todayMessages.pop();
|
|
158
|
+
continue;
|
|
159
|
+
}
|
|
160
|
+
if (!days.has(day)) days.set(day, []);
|
|
161
|
+
const bucket = days.get(day);
|
|
162
|
+
bucket.push({ text, ts, strength, who: item.who, file });
|
|
163
|
+
bucket.sort((a, b) => b.strength - a.strength || a.ts - b.ts);
|
|
164
|
+
if (bucket.length > 6) bucket.pop();
|
|
165
|
+
}
|
|
166
|
+
if (broken) warnings.push(`${file}:略过 ${broken} 行不完整 JSON`);
|
|
167
|
+
}
|
|
168
|
+
for (const day of [...days.keys()].sort()) for (const item of days.get(day)) {
|
|
169
|
+
add('对话', { text: item.text, kind: '对话', ts: item.ts, who: item.who }, `state/history/${item.file}`);
|
|
170
|
+
}
|
|
171
|
+
for (const item of todayMessages.reverse()) {
|
|
172
|
+
const inferred = inferEmotion(item.text);
|
|
173
|
+
add('对话', { text: item.text, kind: '对话', ts: item.ts, who: item.who,
|
|
174
|
+
heat: inferred.emotion.length ? inferred.heat : 3 }, `state/history/${item.file}`);
|
|
175
|
+
}
|
|
176
|
+
|
|
177
|
+
const slang = readJson(path.join(state, 'slang.json'), []);
|
|
178
|
+
// Live bridge schema uses content/meaning/createdAt. Also accept the
|
|
179
|
+
// task's word/definition/at form when that shape is present.
|
|
180
|
+
const words = Array.isArray(slang) ? slang : Array.isArray(slang?.entries) ? slang.entries
|
|
181
|
+
: slang && typeof slang === 'object' ? Object.entries(slang).map(([word, item]) => typeof item === 'string' ? { word, definition: item } : { word, ...item }) : [];
|
|
182
|
+
for (const item of words) {
|
|
183
|
+
const word = textValue(item?.word ?? item?.term ?? item?.content);
|
|
184
|
+
const meaning = textValue(item?.definition ?? item?.meaning);
|
|
185
|
+
if (!word || !meaning) continue;
|
|
186
|
+
add('俚语', { text: `${word}:${meaning}`, kind: '知识', title: word,
|
|
187
|
+
ts: item.at ?? item.createdAt ?? item.updatedAt }, 'state/slang.json', item.id ? [item.id] : []);
|
|
188
|
+
}
|
|
189
|
+
if (invalid) warnings.push(`略过 ${invalid} 条正文或原始时间无效的记录(未用今天代替)`);
|
|
190
|
+
return { missing: false, root, entries, counts, warnings };
|
|
191
|
+
}
|