openclaw-memory-alibaba-mysql 0.1.9 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +37 -51
- package/categories.ts +26 -1
- package/config.ts +2 -2
- package/db.ts +115 -7
- package/index.ts +152 -24
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,33 +1,36 @@
|
|
|
1
|
-
#
|
|
1
|
+
# openclaw-memory-alibaba-mysql
|
|
2
2
|
|
|
3
|
-
OpenClaw 记忆插件,使用阿里云 RDS MySQL
|
|
3
|
+
OpenClaw 记忆插件,使用阿里云 RDS MySQL 做向量存储。支持用户记忆、全文对话记录、自进化记忆;可与 OpenClaw 的 before_agent_start / agent_end 配合,自动召回与自动落库。
|
|
4
4
|
|
|
5
|
-
##
|
|
5
|
+
## 功能概览
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
7
|
+
- **用户记忆**:从对话中自动抽取事实、偏好、决策(如「喜欢寿司」「打算用 Python」),支持按向量检索并注入到每轮上下文。
|
|
8
|
+
- **全文记忆**:按消息来源分为用户、助手、系统、工具等 6 类,每类按会话保留一份,每轮更新;只存真实对话内容,不存系统注入的上下文块。
|
|
9
|
+
- **自进化记忆**:从对话中抽取学习要点、错误、需求等,可选参与召回。
|
|
10
|
+
- **冲突与去重**:可选用 LLM 判断新记忆与已有记忆是否矛盾或重复,避免重复、矛盾条目堆积。
|
|
11
|
+
- **时间衰减**:召回时可对旧记忆降权,让近期信息更突出。
|
|
12
|
+
- **工具**:提供 `memory_recall`(按查询搜记忆)、`memory_store`(显式写入)、`memory_forget`(删除);若只依赖自动召回与自动抓取,可不给 agent 开放 recall/store,仅保留 forget 用于删除。
|
|
12
13
|
|
|
13
|
-
|
|
14
|
-
- **memory_store 工具**:可显式指定 `category`,默认 `user_memory_fact`。
|
|
15
|
-
- **召回**:`before_agent_start` 与 `memory_recall` 统一按向量检索用户记忆(及可选 self_improving),注入上下文。
|
|
14
|
+
## 记忆分类
|
|
16
15
|
|
|
17
|
-
|
|
16
|
+
| 类型 | 说明 |
|
|
17
|
+
|------|------|
|
|
18
|
+
| 用户事实 / 偏好 / 决策 | 从用户话里抽取的事实、喜好、决定,用于后续对话的上下文 |
|
|
19
|
+
| 全文·用户 / 助手 / 系统 / 工具 / 工具结果 / 其它 | 按消息角色分的完整对话记录,按会话维护、每轮更新 |
|
|
20
|
+
| 自进化(学习 / 错误 / 需求) | 从对话中抽取的可复用经验,可选参与召回 |
|
|
18
21
|
|
|
19
22
|
## 配置示例
|
|
20
23
|
|
|
21
|
-
###
|
|
24
|
+
### 最简配置
|
|
22
25
|
|
|
23
|
-
|
|
26
|
+
只接 MySQL 和向量服务,其余用默认(自动召回 + 自动抓取用户记忆,不开启全文与自进化):
|
|
24
27
|
|
|
25
28
|
```json
|
|
26
29
|
{
|
|
27
30
|
"plugins": {
|
|
28
|
-
"slots": { "memory": "
|
|
31
|
+
"slots": { "memory": "openclaw-memory-alibaba-mysql" },
|
|
29
32
|
"entries": {
|
|
30
|
-
"
|
|
33
|
+
"openclaw-memory-alibaba-mysql": {
|
|
31
34
|
"enabled": true,
|
|
32
35
|
"config": {
|
|
33
36
|
"mysql": {
|
|
@@ -50,16 +53,16 @@ OpenClaw 记忆插件,使用阿里云 RDS MySQL 向量存储。用户记忆细
|
|
|
50
53
|
}
|
|
51
54
|
```
|
|
52
55
|
|
|
53
|
-
###
|
|
56
|
+
### 功能全开
|
|
54
57
|
|
|
55
|
-
|
|
58
|
+
开启全文记忆、自进化、LLM 抽取与冲突处理、时间衰减等:
|
|
56
59
|
|
|
57
60
|
```json
|
|
58
61
|
{
|
|
59
62
|
"plugins": {
|
|
60
|
-
"slots": { "memory": "
|
|
63
|
+
"slots": { "memory": "openclaw-memory-alibaba-mysql" },
|
|
61
64
|
"entries": {
|
|
62
|
-
"
|
|
65
|
+
"openclaw-memory-alibaba-mysql": {
|
|
63
66
|
"enabled": true,
|
|
64
67
|
"config": {
|
|
65
68
|
"mysql": {
|
|
@@ -89,7 +92,7 @@ OpenClaw 记忆插件,使用阿里云 RDS MySQL 向量存储。用户记忆细
|
|
|
89
92
|
"memoryExtractionMethod": "llm",
|
|
90
93
|
"autoRecall": true,
|
|
91
94
|
"autoCapture": true,
|
|
92
|
-
"captureMaxChars":
|
|
95
|
+
"captureMaxChars": 5000,
|
|
93
96
|
"enableMemoryDecay": true,
|
|
94
97
|
"tableName": "openclaw_memories"
|
|
95
98
|
}
|
|
@@ -99,38 +102,21 @@ OpenClaw 记忆插件,使用阿里云 RDS MySQL 向量存储。用户记忆细
|
|
|
99
102
|
}
|
|
100
103
|
```
|
|
101
104
|
|
|
102
|
-
|
|
105
|
+
**常用配置说明**
|
|
103
106
|
|
|
104
|
-
-
|
|
105
|
-
- **
|
|
106
|
-
- **
|
|
107
|
-
- **
|
|
108
|
-
- **
|
|
109
|
-
|
|
110
|
-
|
|
107
|
+
- **mysql** / **embedding**:必填,用于连接数据库与生成向量。
|
|
108
|
+
- **llm**:使用「LLM 抽取」或「冲突/去重」时必填。
|
|
109
|
+
- **memoryExtractionMethod**:`"llm"`(默认)或 `"regex"`,控制如何从对话里抽取用户记忆与自改进记忆。
|
|
110
|
+
- **enableFullContextMemory**:是否按角色保存全文对话(每会话每类一份,每轮更新)。
|
|
111
|
+
- **enableSelfImprovingMemory**:是否启用自进化记忆的写入与召回。
|
|
112
|
+
- **memory_duplication_conflict_process**:是否用 LLM 做写入前的去重与矛盾判断。
|
|
113
|
+
- **captureMaxChars**:单条记忆与全文截断的最大字符数,建议 5000 以保留完整当轮对话。
|
|
114
|
+
- **enableMemoryDecay**:召回时是否对旧记忆做时间衰减(近期记忆权重更高)。
|
|
115
|
+
- **tableName**:存储表名,默认 `openclaw_memories`。
|
|
111
116
|
|
|
112
117
|
## 环境变量
|
|
113
118
|
|
|
114
|
-
|
|
115
|
-
- `DASHSCOPE_API_KEY`:DashScope 用于 embedding(若 apiKey 使用 `${DASHSCOPE_API_KEY}`)。
|
|
116
|
-
|
|
117
|
-
---
|
|
118
|
-
|
|
119
|
-
## 测试
|
|
120
|
-
|
|
121
|
-
```bash
|
|
122
|
-
export MYSQL_HOST=your-host MYSQL_PASSWORD=xxx DASHSCOPE_API_KEY=xxx
|
|
123
|
-
npx tsx test-agent-isolation.ts
|
|
124
|
-
npx tsx test-three-memory-types.ts
|
|
125
|
-
npx tsx test-full-context-memory.ts
|
|
126
|
-
```
|
|
127
|
-
|
|
128
|
-
---
|
|
129
|
-
|
|
130
|
-
## 文件说明
|
|
119
|
+
配置中可使用占位符引用环境变量,例如 `${MYSQL_PASSWORD}`、`${DASHSCOPE_API_KEY}`。常见需要准备的有:
|
|
131
120
|
|
|
132
|
-
-
|
|
133
|
-
-
|
|
134
|
-
- `db.ts`:MemoryDB 建表、store、search、softDelete(向量 COSINE)。
|
|
135
|
-
- `index.ts`:插件注册、memory_recall / memory_store / memory_forget、before_agent_start / agent_end。
|
|
136
|
-
- `prompts.ts`:用户记忆与 self_improving 的 LLM 抽取 Prompt。
|
|
121
|
+
- **MYSQL_HOST**, **MYSQL_USER**, **MYSQL_PASSWORD**:MySQL 连接信息。
|
|
122
|
+
- **DASHSCOPE_API_KEY**:DashScope API Key,用于 embedding(以及 LLM 若使用 DashScope)。
|
package/categories.ts
CHANGED
|
@@ -20,9 +20,32 @@ export type UserMemoryCategory =
|
|
|
20
20
|
| typeof USER_MEMORY_PREFERENCE
|
|
21
21
|
| typeof USER_MEMORY_DECISION;
|
|
22
22
|
|
|
23
|
-
/** Full context (conversation audit
|
|
23
|
+
/** Full context (conversation audit). Legacy single category. */
|
|
24
24
|
export const FULL_CONTEXT_MEMORY = "full_context_memory" as const;
|
|
25
25
|
|
|
26
|
+
/** Full context by source (user / assistant / system / tool / tool_result / others) */
|
|
27
|
+
export const FULL_CONTEXT_USER = "full_context_user" as const;
|
|
28
|
+
export const FULL_CONTEXT_ASSISTANT = "full_context_assistant" as const;
|
|
29
|
+
export const FULL_CONTEXT_SYSTEM = "full_context_system" as const;
|
|
30
|
+
export const FULL_CONTEXT_TOOL = "full_context_tool" as const;
|
|
31
|
+
export const FULL_CONTEXT_TOOL_RESULT = "full_context_tool_result" as const;
|
|
32
|
+
export const FULL_CONTEXT_OTHERS = "full_context_others" as const;
|
|
33
|
+
|
|
34
|
+
export const FULL_CONTEXT_SOURCE_CATEGORIES = [
|
|
35
|
+
FULL_CONTEXT_USER,
|
|
36
|
+
FULL_CONTEXT_ASSISTANT,
|
|
37
|
+
FULL_CONTEXT_SYSTEM,
|
|
38
|
+
FULL_CONTEXT_TOOL,
|
|
39
|
+
FULL_CONTEXT_TOOL_RESULT,
|
|
40
|
+
FULL_CONTEXT_OTHERS,
|
|
41
|
+
] as const;
|
|
42
|
+
|
|
43
|
+
export type FullContextSourceCategory = (typeof FULL_CONTEXT_SOURCE_CATEGORIES)[number];
|
|
44
|
+
|
|
45
|
+
export function isFullContextSourceCategory(cat: string): cat is FullContextSourceCategory {
|
|
46
|
+
return FULL_CONTEXT_SOURCE_CATEGORIES.includes(cat as FullContextSourceCategory);
|
|
47
|
+
}
|
|
48
|
+
|
|
26
49
|
/** Self-improving memory sub-categories */
|
|
27
50
|
export const SELF_IMPROVING_LEARNINGS = "self_improving_learnings" as const;
|
|
28
51
|
export const SELF_IMPROVING_ERRORS = "self_improving_errors" as const;
|
|
@@ -40,11 +63,13 @@ export type SelfImprovingCategory = (typeof SELF_IMPROVING_CATEGORIES)[number];
|
|
|
40
63
|
export type MemoryCategory =
|
|
41
64
|
| UserMemoryCategory
|
|
42
65
|
| typeof FULL_CONTEXT_MEMORY
|
|
66
|
+
| FullContextSourceCategory
|
|
43
67
|
| SelfImprovingCategory;
|
|
44
68
|
|
|
45
69
|
export const ALL_CATEGORIES: readonly MemoryCategory[] = [
|
|
46
70
|
...USER_MEMORY_CATEGORIES,
|
|
47
71
|
FULL_CONTEXT_MEMORY,
|
|
72
|
+
...FULL_CONTEXT_SOURCE_CATEGORIES,
|
|
48
73
|
...SELF_IMPROVING_CATEGORIES,
|
|
49
74
|
];
|
|
50
75
|
|
package/config.ts
CHANGED
|
@@ -38,7 +38,7 @@ export type MemoryConfig = {
|
|
|
38
38
|
enableFullContextMemory: boolean;
|
|
39
39
|
/** When false, self_improving_* is not written or recalled. Default false. */
|
|
40
40
|
enableSelfImprovingMemory: boolean;
|
|
41
|
-
/** How to extract user and self_improving memories in auto-capture: "
|
|
41
|
+
/** How to extract user and self_improving memories in auto-capture: "llm" (default) or "regex". Case-insensitive; invalid values fall back to "llm". When "llm", llm config is required. */
|
|
42
42
|
memoryExtractionMethod: "regex" | "llm";
|
|
43
43
|
autoRecall: boolean;
|
|
44
44
|
autoCapture: boolean;
|
|
@@ -210,7 +210,7 @@ export const memoryConfigSchema = {
|
|
|
210
210
|
? cfg.memoryExtractionMethod.trim().toLowerCase()
|
|
211
211
|
: "";
|
|
212
212
|
const memoryExtractionMethod: "regex" | "llm" =
|
|
213
|
-
rawMethod === "
|
|
213
|
+
rawMethod === "regex" ? "regex" : "llm";
|
|
214
214
|
const needsLlm = memory_duplication_conflict_process || memoryExtractionMethod === "llm";
|
|
215
215
|
if (needsLlm && (!cfg.llm || typeof cfg.llm !== "object")) {
|
|
216
216
|
throw new Error(
|
package/db.ts
CHANGED
|
@@ -23,6 +23,11 @@ export type MemorySearchResult = {
|
|
|
23
23
|
|
|
24
24
|
const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
|
|
25
25
|
|
|
26
|
+
/** Strip 4-byte UTF-8 (e.g. emojis) so text is safe for MySQL utf8 charset. Use as-is when table is utf8mb4. */
|
|
27
|
+
function stripFourByteUtf8(text: string): string {
|
|
28
|
+
return text.replace(/[\uD800-\uDBFF][\uDC00-\uDFFF]/g, "");
|
|
29
|
+
}
|
|
30
|
+
|
|
26
31
|
export class MemoryDB {
|
|
27
32
|
private pool: Pool | null = null;
|
|
28
33
|
private initPromise: Promise<void> | null = null;
|
|
@@ -73,10 +78,12 @@ export class MemoryDB {
|
|
|
73
78
|
category VARCHAR(64) DEFAULT '${USER_MEMORY_FACT}',
|
|
74
79
|
created_at BIGINT NOT NULL,
|
|
75
80
|
is_deleted TINYINT NOT NULL DEFAULT 0,
|
|
76
|
-
INDEX idx_agent_id (agent_id)
|
|
77
|
-
|
|
81
|
+
INDEX idx_agent_id (agent_id),
|
|
82
|
+
INDEX idx_agent_session_category (agent_id, session_id, category)
|
|
83
|
+
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
|
|
78
84
|
`);
|
|
79
85
|
await this.ensureIsDeletedColumn(conn);
|
|
86
|
+
await this.ensureSessionCategoryIndex(conn);
|
|
80
87
|
await this.tryCreateVectorIndex(conn);
|
|
81
88
|
} finally {
|
|
82
89
|
conn.release();
|
|
@@ -95,6 +102,18 @@ export class MemoryDB {
|
|
|
95
102
|
);
|
|
96
103
|
}
|
|
97
104
|
|
|
105
|
+
private async ensureSessionCategoryIndex(conn: mysql.PoolConnection): Promise<void> {
|
|
106
|
+
const [rows] = await conn.query(
|
|
107
|
+
`SELECT COUNT(1) AS cnt FROM information_schema.statistics
|
|
108
|
+
WHERE table_schema = DATABASE() AND table_name = ? AND index_name = 'idx_agent_session_category'`,
|
|
109
|
+
[this.tableName],
|
|
110
|
+
);
|
|
111
|
+
if (((rows as Array<{ cnt: number }>)[0]?.cnt ?? 0) > 0) return;
|
|
112
|
+
await conn.query(
|
|
113
|
+
`ALTER TABLE \`${this.tableName}\` ADD INDEX idx_agent_session_category (agent_id, session_id, category)`,
|
|
114
|
+
);
|
|
115
|
+
}
|
|
116
|
+
|
|
98
117
|
private async tryCreateVectorIndex(conn: mysql.PoolConnection): Promise<void> {
|
|
99
118
|
if (this.vectorIndexCreated) {
|
|
100
119
|
return;
|
|
@@ -123,18 +142,28 @@ export class MemoryDB {
|
|
|
123
142
|
|
|
124
143
|
async store(
|
|
125
144
|
agentId: string,
|
|
126
|
-
entry: {
|
|
145
|
+
entry: {
|
|
146
|
+
text: string;
|
|
147
|
+
vector: number[];
|
|
148
|
+
importance: number;
|
|
149
|
+
category: MemoryCategory;
|
|
150
|
+
userId?: string | null;
|
|
151
|
+
sessionId?: string | null;
|
|
152
|
+
},
|
|
127
153
|
): Promise<MemoryEntry> {
|
|
128
154
|
await this.ensureInitialized();
|
|
129
155
|
|
|
130
156
|
const id = randomUUID();
|
|
131
157
|
const createdAt = Date.now();
|
|
132
158
|
const vectorStr = JSON.stringify(entry.vector);
|
|
159
|
+
const userId = entry.userId ?? null;
|
|
160
|
+
const sessionId = entry.sessionId ?? null;
|
|
161
|
+
const textSafe = stripFourByteUtf8(entry.text);
|
|
133
162
|
|
|
134
163
|
await this.pool!.query(
|
|
135
|
-
`INSERT INTO \`${this.tableName}\` (id, agent_id, text, embedding, importance, category, created_at, is_deleted)
|
|
136
|
-
VALUES (?, ?, ?, VEC_FROMTEXT(?), ?, ?, ?, 0)`,
|
|
137
|
-
[id, agentId,
|
|
164
|
+
`INSERT INTO \`${this.tableName}\` (id, agent_id, user_id, session_id, text, embedding, importance, category, created_at, is_deleted)
|
|
165
|
+
VALUES (?, ?, ?, ?, ?, VEC_FROMTEXT(?), ?, ?, ?, 0)`,
|
|
166
|
+
[id, agentId, userId, sessionId, textSafe, vectorStr, entry.importance, entry.category, createdAt],
|
|
138
167
|
);
|
|
139
168
|
|
|
140
169
|
if (!this.vectorIndexCreated) {
|
|
@@ -149,13 +178,92 @@ export class MemoryDB {
|
|
|
149
178
|
return {
|
|
150
179
|
id,
|
|
151
180
|
agentId,
|
|
152
|
-
text:
|
|
181
|
+
text: textSafe,
|
|
153
182
|
importance: entry.importance,
|
|
154
183
|
category: entry.category,
|
|
155
184
|
createdAt,
|
|
156
185
|
};
|
|
157
186
|
}
|
|
158
187
|
|
|
188
|
+
/**
|
|
189
|
+
* Upsert for full-context by (agent_id, session_id, category): one row per session per category.
|
|
190
|
+
* If a row exists, UPDATE it; otherwise INSERT. Returns action and entry.
|
|
191
|
+
*/
|
|
192
|
+
async storeOrUpdateFullContext(
|
|
193
|
+
agentId: string,
|
|
194
|
+
sessionId: string | null,
|
|
195
|
+
entry: {
|
|
196
|
+
text: string;
|
|
197
|
+
vector: number[];
|
|
198
|
+
importance: number;
|
|
199
|
+
category: MemoryCategory;
|
|
200
|
+
userId?: string | null;
|
|
201
|
+
},
|
|
202
|
+
): Promise<{ action: "created" | "updated"; entry: MemoryEntry }> {
|
|
203
|
+
await this.ensureInitialized();
|
|
204
|
+
|
|
205
|
+
const textSafe = stripFourByteUtf8(entry.text);
|
|
206
|
+
const vectorStr = JSON.stringify(entry.vector);
|
|
207
|
+
const userId = entry.userId ?? null;
|
|
208
|
+
const createdAt = Date.now();
|
|
209
|
+
|
|
210
|
+
const [existingRows] = await this.pool!.query(
|
|
211
|
+
`SELECT id FROM \`${this.tableName}\`
|
|
212
|
+
WHERE agent_id = ? AND category = ? AND is_deleted = 0
|
|
213
|
+
AND ((? IS NULL AND session_id IS NULL) OR (session_id = ?))
|
|
214
|
+
LIMIT 1`,
|
|
215
|
+
[agentId, entry.category, sessionId, sessionId],
|
|
216
|
+
);
|
|
217
|
+
const existing = (existingRows as Array<{ id: string }>)[0];
|
|
218
|
+
|
|
219
|
+
if (existing) {
|
|
220
|
+
await this.pool!.query(
|
|
221
|
+
`UPDATE \`${this.tableName}\` SET text = ?, embedding = VEC_FROMTEXT(?), importance = ?, created_at = ?, user_id = ?
|
|
222
|
+
WHERE id = ? AND agent_id = ?`,
|
|
223
|
+
[textSafe, vectorStr, entry.importance, createdAt, userId, existing.id, agentId],
|
|
224
|
+
);
|
|
225
|
+
return {
|
|
226
|
+
action: "updated",
|
|
227
|
+
entry: {
|
|
228
|
+
id: existing.id,
|
|
229
|
+
agentId,
|
|
230
|
+
text: textSafe,
|
|
231
|
+
importance: entry.importance,
|
|
232
|
+
category: entry.category,
|
|
233
|
+
createdAt,
|
|
234
|
+
},
|
|
235
|
+
};
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
const id = randomUUID();
|
|
239
|
+
await this.pool!.query(
|
|
240
|
+
`INSERT INTO \`${this.tableName}\` (id, agent_id, user_id, session_id, text, embedding, importance, category, created_at, is_deleted)
|
|
241
|
+
VALUES (?, ?, ?, ?, ?, VEC_FROMTEXT(?), ?, ?, ?, 0)`,
|
|
242
|
+
[id, agentId, userId, sessionId, textSafe, vectorStr, entry.importance, entry.category, createdAt],
|
|
243
|
+
);
|
|
244
|
+
|
|
245
|
+
if (!this.vectorIndexCreated) {
|
|
246
|
+
const conn = await this.pool!.getConnection();
|
|
247
|
+
try {
|
|
248
|
+
await this.tryCreateVectorIndex(conn);
|
|
249
|
+
} finally {
|
|
250
|
+
conn.release();
|
|
251
|
+
}
|
|
252
|
+
}
|
|
253
|
+
|
|
254
|
+
return {
|
|
255
|
+
action: "created",
|
|
256
|
+
entry: {
|
|
257
|
+
id,
|
|
258
|
+
agentId,
|
|
259
|
+
text: textSafe,
|
|
260
|
+
importance: entry.importance,
|
|
261
|
+
category: entry.category,
|
|
262
|
+
createdAt,
|
|
263
|
+
},
|
|
264
|
+
};
|
|
265
|
+
}
|
|
266
|
+
|
|
159
267
|
/**
|
|
160
268
|
* Search memories by vector similarity.
|
|
161
269
|
* @param categories - When non-empty, only return rows with category IN (categories). Omit for no category filter.
|
package/index.ts
CHANGED
|
@@ -19,11 +19,18 @@ import {
|
|
|
19
19
|
SELF_IMPROVING_ERRORS,
|
|
20
20
|
SELF_IMPROVING_FEATURE_REQUESTS,
|
|
21
21
|
FULL_CONTEXT_MEMORY,
|
|
22
|
+
FULL_CONTEXT_USER,
|
|
23
|
+
FULL_CONTEXT_ASSISTANT,
|
|
24
|
+
FULL_CONTEXT_SYSTEM,
|
|
25
|
+
FULL_CONTEXT_TOOL,
|
|
26
|
+
FULL_CONTEXT_TOOL_RESULT,
|
|
27
|
+
FULL_CONTEXT_OTHERS,
|
|
22
28
|
type UserMemoryCategory,
|
|
23
29
|
type SelfImprovingCategory,
|
|
24
30
|
type MemoryCategory,
|
|
25
31
|
isUserMemoryCategory,
|
|
26
32
|
isSelfImprovingCategory,
|
|
33
|
+
isFullContextSourceCategory,
|
|
27
34
|
} from "./categories.js";
|
|
28
35
|
import {
|
|
29
36
|
DEFAULT_CAPTURE_MAX_CHARS,
|
|
@@ -146,9 +153,10 @@ function formatRelevantMemoriesContext(
|
|
|
146
153
|
}
|
|
147
154
|
|
|
148
155
|
function getThresholdForCategory(cfg: MemoryConfig, category: MemoryCategory): number {
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
156
|
+
if (isUserMemoryCategory(category) || isFullContextSourceCategory(category) || category === FULL_CONTEXT_MEMORY) {
|
|
157
|
+
return cfg.similarityThresholdUserMemory;
|
|
158
|
+
}
|
|
159
|
+
return cfg.similarityThresholdSelfImproving;
|
|
152
160
|
}
|
|
153
161
|
|
|
154
162
|
/** Apply time decay to recall results: effectiveScore = score * decay(createdAt). Returns new array sorted by effectiveScore desc. */
|
|
@@ -388,7 +396,7 @@ New memory text:
|
|
|
388
396
|
${newText}
|
|
389
397
|
"""
|
|
390
398
|
|
|
391
|
-
Existing similar memories (up to
|
|
399
|
+
Existing similar memories (up to 20):
|
|
392
400
|
${candidateList}
|
|
393
401
|
|
|
394
402
|
Rules:
|
|
@@ -468,15 +476,34 @@ function getTextPartsFromMessage(msg: Record<string, unknown>): string[] {
|
|
|
468
476
|
return parts;
|
|
469
477
|
}
|
|
470
478
|
|
|
471
|
-
/**
|
|
479
|
+
/** Lines by role for full-context (user / assistant / system / tool / tool_result / others). */
|
|
480
|
+
export type LinesByRole = {
|
|
481
|
+
user: string[];
|
|
482
|
+
assistant: string[];
|
|
483
|
+
system: string[];
|
|
484
|
+
tool: string[];
|
|
485
|
+
tool_result: string[];
|
|
486
|
+
others: string[];
|
|
487
|
+
};
|
|
488
|
+
|
|
489
|
+
/** From raw agent messages, collect user-only texts, full/conversation lines, and lines grouped by role. */
|
|
472
490
|
function parseMessagesForCapture(messages: unknown[]): {
|
|
473
491
|
userMessageTexts: string[];
|
|
474
492
|
allMessageLines: string[];
|
|
475
493
|
userAndAssistantLines: string[];
|
|
494
|
+
linesByRole: LinesByRole;
|
|
476
495
|
} {
|
|
477
496
|
const userMessageTexts: string[] = [];
|
|
478
497
|
const allMessageLines: string[] = [];
|
|
479
498
|
const userAndAssistantLines: string[] = [];
|
|
499
|
+
const linesByRole: LinesByRole = {
|
|
500
|
+
user: [],
|
|
501
|
+
assistant: [],
|
|
502
|
+
system: [],
|
|
503
|
+
tool: [],
|
|
504
|
+
tool_result: [],
|
|
505
|
+
others: [],
|
|
506
|
+
};
|
|
480
507
|
|
|
481
508
|
for (const msg of messages) {
|
|
482
509
|
if (!msg || typeof msg !== "object") continue;
|
|
@@ -489,9 +516,16 @@ function parseMessagesForCapture(messages: unknown[]): {
|
|
|
489
516
|
allMessageLines.push(line);
|
|
490
517
|
if (role !== "system") userAndAssistantLines.push(line);
|
|
491
518
|
if (role === "user") userMessageTexts.push(...parts);
|
|
519
|
+
|
|
520
|
+
if (role === "user") linesByRole.user.push(line);
|
|
521
|
+
else if (role === "assistant") linesByRole.assistant.push(line);
|
|
522
|
+
else if (role === "system") linesByRole.system.push(line);
|
|
523
|
+
else if (role === "tool") linesByRole.tool.push(line);
|
|
524
|
+
else if (role === "toolResult" || role === "tool_result") linesByRole.tool_result.push(line);
|
|
525
|
+
else linesByRole.others.push(line);
|
|
492
526
|
}
|
|
493
527
|
|
|
494
|
-
return { userMessageTexts, allMessageLines, userAndAssistantLines };
|
|
528
|
+
return { userMessageTexts, allMessageLines, userAndAssistantLines, linesByRole };
|
|
495
529
|
}
|
|
496
530
|
|
|
497
531
|
/** Truncate to max chars and append "..." if needed. */
|
|
@@ -514,12 +548,13 @@ function stripInjectedContextBlocks(text: string): string {
|
|
|
514
548
|
return out;
|
|
515
549
|
}
|
|
516
550
|
|
|
517
|
-
/** Build list of capture candidates: user (regex/LLM) + optional full-context + optional self-improving. */
|
|
551
|
+
/** Build list of capture candidates: user (regex/LLM) + optional full-context by source + optional self-improving. */
|
|
518
552
|
async function buildCaptureCandidates(
|
|
519
553
|
cfg: MemoryConfig,
|
|
520
554
|
userMessageTexts: string[],
|
|
521
555
|
allMessageLines: string[],
|
|
522
556
|
userAndAssistantLines: string[],
|
|
557
|
+
linesByRole: LinesByRole,
|
|
523
558
|
): Promise<CaptureCandidate[]> {
|
|
524
559
|
const candidates: CaptureCandidate[] = [];
|
|
525
560
|
|
|
@@ -542,13 +577,56 @@ async function buildCaptureCandidates(
|
|
|
542
577
|
}
|
|
543
578
|
}
|
|
544
579
|
|
|
545
|
-
// Full-context
|
|
546
|
-
if (cfg.enableFullContextMemory
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
580
|
+
// Full-context by source. Strip injected blocks from user/assistant so real conversation is stored (injected context can be huge and push out the actual question).
|
|
581
|
+
if (cfg.enableFullContextMemory) {
|
|
582
|
+
if (linesByRole.user.length > 0) {
|
|
583
|
+
const userText = linesByRole.user
|
|
584
|
+
.map((line) => stripInjectedContextBlocks(line))
|
|
585
|
+
.filter((s) => s.length > 0)
|
|
586
|
+
.join("\n");
|
|
587
|
+
if (userText.length > 0) {
|
|
588
|
+
candidates.push({
|
|
589
|
+
category: FULL_CONTEXT_USER,
|
|
590
|
+
text: truncateForCapture(userText, cfg.captureMaxChars),
|
|
591
|
+
});
|
|
592
|
+
}
|
|
593
|
+
}
|
|
594
|
+
if (linesByRole.assistant.length > 0) {
|
|
595
|
+
const assistantText = linesByRole.assistant
|
|
596
|
+
.map((line) => stripInjectedContextBlocks(line))
|
|
597
|
+
.filter((s) => s.length > 0)
|
|
598
|
+
.join("\n");
|
|
599
|
+
if (assistantText.length > 0) {
|
|
600
|
+
candidates.push({
|
|
601
|
+
category: FULL_CONTEXT_ASSISTANT,
|
|
602
|
+
text: truncateForCapture(assistantText, cfg.captureMaxChars),
|
|
603
|
+
});
|
|
604
|
+
}
|
|
605
|
+
}
|
|
606
|
+
if (linesByRole.system.length > 0) {
|
|
607
|
+
candidates.push({
|
|
608
|
+
category: FULL_CONTEXT_SYSTEM,
|
|
609
|
+
text: truncateForCapture(linesByRole.system.join("\n"), cfg.captureMaxChars),
|
|
610
|
+
});
|
|
611
|
+
}
|
|
612
|
+
if (linesByRole.tool.length > 0) {
|
|
613
|
+
candidates.push({
|
|
614
|
+
category: FULL_CONTEXT_TOOL,
|
|
615
|
+
text: truncateForCapture(linesByRole.tool.join("\n"), cfg.captureMaxChars),
|
|
616
|
+
});
|
|
617
|
+
}
|
|
618
|
+
if (linesByRole.tool_result.length > 0) {
|
|
619
|
+
candidates.push({
|
|
620
|
+
category: FULL_CONTEXT_TOOL_RESULT,
|
|
621
|
+
text: truncateForCapture(linesByRole.tool_result.join("\n"), cfg.captureMaxChars),
|
|
622
|
+
});
|
|
623
|
+
}
|
|
624
|
+
if (linesByRole.others.length > 0) {
|
|
625
|
+
candidates.push({
|
|
626
|
+
category: FULL_CONTEXT_OTHERS,
|
|
627
|
+
text: truncateForCapture(linesByRole.others.join("\n"), cfg.captureMaxChars),
|
|
628
|
+
});
|
|
629
|
+
}
|
|
552
630
|
}
|
|
553
631
|
|
|
554
632
|
// Self-improving: user + assistant only; strip injected blocks, then extract by regex or LLM
|
|
@@ -591,6 +669,7 @@ type StoreOneResult = { action: "created" | "updated"; entry: MemoryEntry };
|
|
|
591
669
|
function getDedupCategories(category: MemoryCategory): readonly MemoryCategory[] {
|
|
592
670
|
if (isUserMemoryCategory(category)) return USER_MEMORY_CATEGORIES;
|
|
593
671
|
if (category === FULL_CONTEXT_MEMORY) return [FULL_CONTEXT_MEMORY];
|
|
672
|
+
if (isFullContextSourceCategory(category)) return [category];
|
|
594
673
|
if (isSelfImprovingCategory(category)) return SELF_IMPROVING_CATEGORIES;
|
|
595
674
|
return [category];
|
|
596
675
|
}
|
|
@@ -602,29 +681,54 @@ async function storeOneCaptureItem(
|
|
|
602
681
|
cfg: MemoryConfig,
|
|
603
682
|
db: MemoryDB,
|
|
604
683
|
embeddings: Embeddings,
|
|
684
|
+
options?: { userId?: string | null; sessionId?: string | null },
|
|
605
685
|
): Promise<StoreOneResult> {
|
|
606
686
|
const importance = item.importance ?? DEFAULT_IMPORTANCE;
|
|
607
687
|
const vector = await embeddings.embed(item.text);
|
|
608
688
|
const threshold = getThresholdForCategory(cfg, item.category);
|
|
609
689
|
const dedupCategories = getDedupCategories(item.category);
|
|
690
|
+
const storePayload = {
|
|
691
|
+
text: item.text,
|
|
692
|
+
vector,
|
|
693
|
+
importance,
|
|
694
|
+
category: item.category,
|
|
695
|
+
userId: options?.userId ?? null,
|
|
696
|
+
sessionId: options?.sessionId ?? null,
|
|
697
|
+
};
|
|
698
|
+
|
|
699
|
+
// Full-context by source: upsert by (agent_id, session_id, category) so one row per session per category.
|
|
700
|
+
if (isFullContextSourceCategory(item.category)) {
|
|
701
|
+
const { action, entry } = await db.storeOrUpdateFullContext(agentId, options?.sessionId ?? null, {
|
|
702
|
+
text: storePayload.text,
|
|
703
|
+
vector: storePayload.vector,
|
|
704
|
+
importance: storePayload.importance,
|
|
705
|
+
category: storePayload.category,
|
|
706
|
+
userId: options?.userId ?? null,
|
|
707
|
+
});
|
|
708
|
+
return { action, entry };
|
|
709
|
+
}
|
|
610
710
|
|
|
611
711
|
if (!cfg.memory_duplication_conflict_process) {
|
|
612
712
|
const existing = await db.search(agentId, vector, 1, threshold, [...dedupCategories]);
|
|
613
713
|
if (existing.length > 0) await db.softDelete(agentId, existing[0].entry.id);
|
|
614
|
-
const entry = await db.store(agentId,
|
|
714
|
+
const entry = await db.store(agentId, storePayload);
|
|
615
715
|
return { action: existing.length > 0 ? "updated" : "created", entry };
|
|
616
716
|
}
|
|
617
717
|
|
|
618
|
-
|
|
619
|
-
|
|
718
|
+
// Lower recall bar for conflict/dedup for both user_memory_* and self_improving_*:
|
|
719
|
+
// contradictory or same-topic memories (e.g. "dislikes X" vs "loves X", or revised learnings) often have
|
|
720
|
+
// only moderate embedding similarity (~0.65–0.8); without this they may not enter the candidate list.
|
|
721
|
+
const recallMinScore = Math.max(0.5, threshold - 0.35);
|
|
722
|
+
const conflictCandidateLimit = 20;
|
|
723
|
+
const candidates = await db.search(agentId, vector, conflictCandidateLimit, recallMinScore, [...dedupCategories]);
|
|
620
724
|
if (candidates.length === 0) {
|
|
621
|
-
const entry = await db.store(agentId,
|
|
725
|
+
const entry = await db.store(agentId, storePayload);
|
|
622
726
|
return { action: "created", entry };
|
|
623
727
|
}
|
|
624
728
|
|
|
625
729
|
const decision = await decideInsertOrUpdate(cfg.llm!, item.text, candidates);
|
|
626
730
|
if (decision.action === "update") await db.softDelete(agentId, decision.memoryId);
|
|
627
|
-
const entry = await db.store(agentId,
|
|
731
|
+
const entry = await db.store(agentId, storePayload);
|
|
628
732
|
return { action: decision.action === "update" ? "updated" : "created", entry };
|
|
629
733
|
}
|
|
630
734
|
|
|
@@ -711,7 +815,16 @@ const memoryPlugin = {
|
|
|
711
815
|
|
|
712
816
|
const writableCategories: MemoryCategory[] = [
|
|
713
817
|
...USER_MEMORY_CATEGORIES,
|
|
714
|
-
...(cfg.enableFullContextMemory
|
|
818
|
+
...(cfg.enableFullContextMemory
|
|
819
|
+
? [
|
|
820
|
+
FULL_CONTEXT_USER,
|
|
821
|
+
FULL_CONTEXT_ASSISTANT,
|
|
822
|
+
FULL_CONTEXT_SYSTEM,
|
|
823
|
+
FULL_CONTEXT_TOOL,
|
|
824
|
+
FULL_CONTEXT_TOOL_RESULT,
|
|
825
|
+
FULL_CONTEXT_OTHERS,
|
|
826
|
+
]
|
|
827
|
+
: []),
|
|
715
828
|
...(cfg.enableSelfImprovingMemory ? SELF_IMPROVING_CATEGORIES : []),
|
|
716
829
|
];
|
|
717
830
|
api.registerTool(
|
|
@@ -741,7 +854,8 @@ const memoryPlugin = {
|
|
|
741
854
|
category?: MemoryCategory;
|
|
742
855
|
};
|
|
743
856
|
|
|
744
|
-
|
|
857
|
+
const isFullContext = category === FULL_CONTEXT_MEMORY || isFullContextSourceCategory(category);
|
|
858
|
+
if (isFullContext && !cfg.enableFullContextMemory) {
|
|
745
859
|
return {
|
|
746
860
|
content: [{ type: "text", text: "Full context memory is disabled. Enable enableFullContextMemory in config to use it." }],
|
|
747
861
|
details: { error: "full_context_memory_disabled" },
|
|
@@ -755,8 +869,10 @@ const memoryPlugin = {
|
|
|
755
869
|
}
|
|
756
870
|
|
|
757
871
|
const agentId = ctx.agentId ?? "default";
|
|
872
|
+
const userId = (ctx as { requesterSenderId?: string }).requesterSenderId ?? null;
|
|
873
|
+
const sessionId = (ctx as { sessionId?: string }).sessionId ?? null;
|
|
758
874
|
const item: CaptureCandidate = { category, text, importance };
|
|
759
|
-
const { action, entry } = await storeOneCaptureItem(agentId, item, cfg, db, embeddings);
|
|
875
|
+
const { action, entry } = await storeOneCaptureItem(agentId, item, cfg, db, embeddings, { userId, sessionId });
|
|
760
876
|
const preview = text.length > 100 ? text.slice(0, 100) + "..." : text;
|
|
761
877
|
return {
|
|
762
878
|
content: [{ type: "text", text: `${action === "updated" ? "Updated" : "Stored"}: "${preview}"` }],
|
|
@@ -881,20 +997,32 @@ const memoryPlugin = {
|
|
|
881
997
|
|
|
882
998
|
try {
|
|
883
999
|
const agentId = ctx.agentId ?? "default";
|
|
884
|
-
const {
|
|
1000
|
+
const userId = (ctx as { requesterSenderId?: string }).requesterSenderId ?? null;
|
|
1001
|
+
const sessionId = (ctx as { sessionId?: string }).sessionId ?? (event as { sessionId?: string }).sessionId ?? null;
|
|
1002
|
+
const { userMessageTexts, allMessageLines, userAndAssistantLines, linesByRole } = parseMessagesForCapture(
|
|
885
1003
|
event.messages,
|
|
886
1004
|
);
|
|
1005
|
+
api.logger.info(
|
|
1006
|
+
`openclaw-memory-alibaba-mysql: agent_end messages=${event.messages.length} user=${linesByRole.user.length} assistant=${linesByRole.assistant.length} system=${linesByRole.system.length} tool=${linesByRole.tool.length} tool_result=${linesByRole.tool_result.length} others=${linesByRole.others.length}`,
|
|
1007
|
+
);
|
|
1008
|
+
if (linesByRole.user.length === 0 && event.messages.length > 0) {
|
|
1009
|
+
const roles = (event.messages as Array<Record<string, unknown>>).map((m) => m.role ?? "?");
|
|
1010
|
+
api.logger.warn(
|
|
1011
|
+
`openclaw-memory-alibaba-mysql: no user lines parsed; message roles: ${roles.join(", ")}`,
|
|
1012
|
+
);
|
|
1013
|
+
}
|
|
887
1014
|
const toProcess = await buildCaptureCandidates(
|
|
888
1015
|
cfg,
|
|
889
1016
|
userMessageTexts,
|
|
890
1017
|
allMessageLines,
|
|
891
1018
|
userAndAssistantLines,
|
|
1019
|
+
linesByRole,
|
|
892
1020
|
);
|
|
893
1021
|
if (toProcess.length === 0) return;
|
|
894
1022
|
|
|
895
1023
|
let stored = 0;
|
|
896
1024
|
for (const item of toProcess) {
|
|
897
|
-
await storeOneCaptureItem(agentId, item, cfg, db, embeddings);
|
|
1025
|
+
await storeOneCaptureItem(agentId, item, cfg, db, embeddings, { userId, sessionId });
|
|
898
1026
|
stored++;
|
|
899
1027
|
}
|
|
900
1028
|
if (stored > 0) {
|