@amaster.ai/pi-memory-mem0 0.1.8 → 0.1.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +94 -35
- package/dist/dedup.d.ts +8 -2
- package/dist/dedup.d.ts.map +1 -1
- package/dist/dedup.js +92 -27
- package/dist/dedup.js.map +1 -1
- package/dist/index.d.ts +17 -3
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +247 -25
- package/dist/index.js.map +1 -1
- package/dist/prefetch.d.ts +21 -0
- package/dist/prefetch.d.ts.map +1 -0
- package/dist/prefetch.js +64 -0
- package/dist/prefetch.js.map +1 -0
- package/dist/privacy.d.ts +2 -1
- package/dist/privacy.d.ts.map +1 -1
- package/dist/privacy.js +3 -1
- package/dist/privacy.js.map +1 -1
- package/dist/provider.d.ts +28 -17
- package/dist/provider.d.ts.map +1 -1
- package/dist/provider.js +288 -50
- package/dist/provider.js.map +1 -1
- package/dist/tools.d.ts +27 -10
- package/dist/tools.d.ts.map +1 -1
- package/dist/tools.js +101 -89
- package/dist/tools.js.map +1 -1
- package/dist/types.d.ts +19 -5
- package/dist/types.d.ts.map +1 -1
- package/package.json +8 -3
package/README.md
CHANGED
|
@@ -2,22 +2,49 @@
|
|
|
2
2
|
|
|
3
3
|

|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Semantic memory extension powered by [Mem0](https://mem0.ai), with Platform, embedded, and self-hosted backends and three memory modes (hybrid, active, passive).
|
|
6
6
|
|
|
7
7
|
## How It Works
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
**Passive side**: after each conversation turn, the user + assistant messages are automatically sent to Mem0 for fact extraction and storage. When you send a prompt, a semantic search is prefetched in the background and relevant memories are recalled before the agent starts — **zero effort required**.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
**Active side**: the agent gets a `mem0_memory` tool (`search` / `add` / `get_all` / `delete`) so it can look up and manage memories on its own initiative — same idea as the official `@mem0/pi-agent-plugin`, but backed by this package's provider abstraction, so it works with Platform, embedded, **and** self-hosted Mem0.
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
Safety boundaries:
|
|
14
|
+
|
|
15
|
+
- **Recall channel**: recalled memories are injected as a custom message delivered on the user channel — never into the system prompt. Each entry is wrapped as `[UNTRUSTED MEMORY DATA]` (JSON-quoted), and entries matching prompt-injection patterns are replaced with `[BLOCKED UNTRUSTED MEMORY: ...]`.
|
|
16
|
+
- **Project namespacing**: memories are scoped to `userId:project:<cwd-hash>` by default, so one project's memories are not visible in another. Set `userIdScope: "exact"` when one identity must be shared across projects.
|
|
17
|
+
- **Credential redaction**: private keys, bearer tokens, and `api_key=…`-style values are redacted before anything is stored.
|
|
18
|
+
- **Platform mode disclosure**: in `platform` mode the captured turns leave your machine and are processed by Mem0 Cloud. Use `embedded` or `self-hosted` mode to keep memory traffic off third-party infrastructure.
|
|
19
|
+
|
|
20
|
+
## Modes
|
|
21
|
+
|
|
22
|
+
### Backend Modes
|
|
14
23
|
|
|
15
24
|
| Mode | Vector Store | Persistence | Dependencies | Use Case |
|
|
16
25
|
|------|-------------|-------------|--------------|----------|
|
|
17
26
|
| `platform` | Mem0 Cloud | Cloud-managed | `MEM0_API_KEY` | Quick start, multi-device sync |
|
|
18
|
-
| `
|
|
27
|
+
| `embedded` | Mem0 OSS vector store | Vector-store managed | LLM + Embedding API | Data privacy, no Mem0 Cloud |
|
|
28
|
+
| `self-hosted` | Mem0 OSS REST server | Server-managed | Server URL, optional API key | Shared infrastructure |
|
|
29
|
+
|
|
30
|
+
### Memory Modes
|
|
31
|
+
|
|
32
|
+
| `memoryMode` | Auto capture + recall | `mem0_memory` tool | Use Case |
|
|
33
|
+
|--------------|----------------------|--------------------|----------|
|
|
34
|
+
| `hybrid` (default) | ✅ | ✅ | Best recall: automatic context plus agent-driven lookup |
|
|
35
|
+
| `active` | ❌ | ✅ | No background traffic; the agent decides every read/write |
|
|
36
|
+
| `passive` | ✅ | ❌ | Zero tool surface; fully automatic memory |
|
|
37
|
+
|
|
38
|
+
```json
|
|
39
|
+
{
|
|
40
|
+
"pi-memory-mem0": {
|
|
41
|
+
"mode": "embedded",
|
|
42
|
+
"memoryMode": "hybrid"
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
```
|
|
19
46
|
|
|
20
|
-
## Architecture (
|
|
47
|
+
## Architecture (Embedded Mode)
|
|
21
48
|
|
|
22
49
|
```
|
|
23
50
|
User ←→ Agent ←→ Mem0 OSS Memory
|
|
@@ -33,10 +60,7 @@ User ←→ Agent ←→ Mem0 OSS Memory
|
|
|
33
60
|
|
|
34
61
|
## Quick Start
|
|
35
62
|
|
|
36
|
-
Store configuration in user/agent settings or in a trusted project's
|
|
37
|
-
`.pi/settings.json`. Project settings are ignored when trust is declined and do
|
|
38
|
-
not expand `${ENV_VAR}`; the environment-backed examples below therefore belong
|
|
39
|
-
in user or agent settings.
|
|
63
|
+
Store configuration in user/agent settings or in a trusted project's `.pi/settings.json`. Project settings are ignored when trust is declined and do not expand `${ENV_VAR}`; the environment-backed examples below therefore belong in user or agent settings.
|
|
40
64
|
|
|
41
65
|
### Platform Mode
|
|
42
66
|
|
|
@@ -50,14 +74,14 @@ in user or agent settings.
|
|
|
50
74
|
}
|
|
51
75
|
```
|
|
52
76
|
|
|
53
|
-
###
|
|
77
|
+
### Embedded Mode (Recommended)
|
|
54
78
|
|
|
55
79
|
Reuses API keys and base URLs from pi's configured model providers — **no extra environment variables needed**.
|
|
56
80
|
|
|
57
81
|
```json
|
|
58
82
|
{
|
|
59
83
|
"pi-memory-mem0": {
|
|
60
|
-
"mode": "
|
|
84
|
+
"mode": "embedded",
|
|
61
85
|
"userId": "${USER}"
|
|
62
86
|
}
|
|
63
87
|
}
|
|
@@ -65,6 +89,25 @@ Reuses API keys and base URLs from pi's configured model providers — **no extr
|
|
|
65
89
|
|
|
66
90
|
Defaults to OpenAI `text-embedding-3-small` (embedding) + `gpt-4.1-nano` (extraction). API keys and base URLs are automatically resolved from pi's model registry.
|
|
67
91
|
|
|
92
|
+
Existing settings with `mode: "open-source"` continue to load as `embedded`, but `open-source` is not part of the supported configuration interface. New settings should use `embedded`.
|
|
93
|
+
|
|
94
|
+
### Self-Hosted Mode
|
|
95
|
+
|
|
96
|
+
Calls the OSS REST server directly. The server uses `/memories` and `/search`, not the Mem0 Platform `/v1` paths.
|
|
97
|
+
|
|
98
|
+
```json
|
|
99
|
+
{
|
|
100
|
+
"pi-memory-mem0": {
|
|
101
|
+
"mode": "self-hosted",
|
|
102
|
+
"baseUrl": "${MEM0_BASE_URL}",
|
|
103
|
+
"apiKey": "${MEM0_API_KEY}",
|
|
104
|
+
"userId": "${PAPERCLIP_COMPANY_ID}",
|
|
105
|
+
"agentId": "${PAPERCLIP_AGENT_ID}",
|
|
106
|
+
"userIdScope": "exact"
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
```
|
|
110
|
+
|
|
68
111
|
### Custom Provider
|
|
69
112
|
|
|
70
113
|
When your model registry defines a custom provider with `api: "openai-completions"`, you can use it directly:
|
|
@@ -72,7 +115,7 @@ When your model registry defines a custom provider with `api: "openai-completion
|
|
|
72
115
|
```json
|
|
73
116
|
{
|
|
74
117
|
"pi-memory-mem0": {
|
|
75
|
-
"mode": "
|
|
118
|
+
"mode": "embedded",
|
|
76
119
|
"oss": {
|
|
77
120
|
"llm": {
|
|
78
121
|
"provider": "my-provider",
|
|
@@ -97,7 +140,7 @@ The extension automatically:
|
|
|
97
140
|
```json
|
|
98
141
|
{
|
|
99
142
|
"pi-memory-mem0": {
|
|
100
|
-
"mode": "
|
|
143
|
+
"mode": "embedded",
|
|
101
144
|
"userId": "${USER}",
|
|
102
145
|
"oss": {
|
|
103
146
|
"llm": {
|
|
@@ -121,7 +164,7 @@ For production workloads that need a dedicated vector database:
|
|
|
121
164
|
```json
|
|
122
165
|
{
|
|
123
166
|
"pi-memory-mem0": {
|
|
124
|
-
"mode": "
|
|
167
|
+
"mode": "embedded",
|
|
125
168
|
"oss": {
|
|
126
169
|
"vectorStore": {
|
|
127
170
|
"provider": "qdrant",
|
|
@@ -140,10 +183,14 @@ The configured vector store always owns persistence. To request an intentionally
|
|
|
140
183
|
|
|
141
184
|
| Field | Type | Default | Description |
|
|
142
185
|
|-------|------|---------|-------------|
|
|
143
|
-
| `mode` | `"platform"` \| `"
|
|
144
|
-
| `
|
|
145
|
-
| `
|
|
186
|
+
| `mode` | `"platform"` \| `"embedded"` \| `"self-hosted"` | `"platform"` | Operating mode |
|
|
187
|
+
| `memoryMode` | `"hybrid"` \| `"active"` \| `"passive"` | `"hybrid"` | Memory behavior: tool + automation, tool only, or automation only |
|
|
188
|
+
| `apiKey` | string | — | Platform or self-hosted API key. Supports `${MEM0_API_KEY}` |
|
|
189
|
+
| `baseUrl` | string | `https://api.mem0.ai` | Platform override; required for self-hosted mode |
|
|
190
|
+
| `requestTimeoutMs` | number | `30000` | Self-hosted request timeout |
|
|
146
191
|
| `userId` | string | `$USER` or `"default-user"` | Memory scoping identifier |
|
|
192
|
+
| `agentId` | string | — | Optional agent scope. Supports environment interpolation |
|
|
193
|
+
| `userIdScope` | `"project"` \| `"exact"` | `"project"` | Append the cwd hash or use `userId` verbatim |
|
|
147
194
|
| `topK` | number | `5` | Max recalled memories per turn |
|
|
148
195
|
| `useRegistryKeys` | boolean | `true` | Whether OSS mode resolves keys from pi registry |
|
|
149
196
|
| `oss.llm` | object | OpenAI gpt-4.1-nano | OSS extraction model |
|
|
@@ -153,14 +200,17 @@ The configured vector store always owns persistence. To request an intentionally
|
|
|
153
200
|
| `oss.historyDbPath` | string | `<home>/memories/mem0-history.db` | Shortcut for SQLite history DB path |
|
|
154
201
|
| `oss.disableHistory` | boolean | `false` | Disable mem0 operation history |
|
|
155
202
|
|
|
203
|
+
Embedded and self-hosted modes read and write the exact `userId` + `agentId` scope; Platform mode reads the user and agent entity scopes with OR because Mem0 Platform stores them as separate records. Existing memories with a null `agent_id` are not backfilled automatically.
|
|
204
|
+
|
|
156
205
|
## Data Storage
|
|
157
206
|
|
|
158
207
|
| Mode | Vector Data | History |
|
|
159
208
|
|------|-------------|---------|
|
|
160
209
|
| Platform | Mem0 Cloud | Cloud-managed |
|
|
161
|
-
|
|
|
162
|
-
|
|
|
163
|
-
|
|
|
210
|
+
| Embedded (default paths) | `<home>/memories/mem0-vectors.db` | `<home>/memories/mem0-history.db` |
|
|
211
|
+
| Embedded (`memory`, `dbPath: ":memory:"`) | Process-local SQLite; lost on restart | `<home>/memories/mem0-history.db` |
|
|
212
|
+
| Embedded (Qdrant) | Qdrant server | `<home>/memories/mem0-history.db` |
|
|
213
|
+
| Self-hosted | Remote server | Remote server |
|
|
164
214
|
|
|
165
215
|
The `home` directory is resolved via `resolveHome()` from `@amaster.ai/pi-shared/settings` (defaults to `~/.pi/agent`).
|
|
166
216
|
|
|
@@ -179,7 +229,7 @@ This happens transparently — just configure the provider name as it appears in
|
|
|
179
229
|
|
|
180
230
|
## Installation Notes
|
|
181
231
|
|
|
182
|
-
The default
|
|
232
|
+
The default Embedded configuration depends on `better-sqlite3` (native addon, transitive dependency of `mem0ai`) for both the vector store and history. This remains true for `dbPath: ":memory:"`: it changes where SQLite stores pages, not which vector-store implementation is used.
|
|
183
233
|
|
|
184
234
|
**For pi-agent users**: pi-agent's `package.json` includes `better-sqlite3` in `pnpm.onlyBuiltDependencies` — it compiles automatically during `pnpm install`. No extra steps needed.
|
|
185
235
|
|
|
@@ -197,11 +247,16 @@ If `better-sqlite3` fails to load (for example, because of a Node ABI mismatch),
|
|
|
197
247
|
|
|
198
248
|
## Tools
|
|
199
249
|
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
250
|
+
When `memoryMode` is `hybrid` or `active`, the agent gets one action-dispatched tool:
|
|
251
|
+
|
|
252
|
+
```
|
|
253
|
+
mem0_memory(action="search", query="...") # Semantic search over memories
|
|
254
|
+
mem0_memory(action="add", content="...") # Store a durable fact
|
|
255
|
+
mem0_memory(action="get_all") # List every stored memory
|
|
256
|
+
mem0_memory(action="delete", memory_id="...") # Remove a memory by id
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
Stored content is credential-redacted first; results are returned with the same `[UNTRUSTED MEMORY DATA]` wrapping as passive recall, including the memory ids needed for `delete`.
|
|
205
260
|
|
|
206
261
|
## Commands
|
|
207
262
|
|
|
@@ -209,31 +264,35 @@ If `better-sqlite3` fails to load (for example, because of a Node ABI mismatch),
|
|
|
209
264
|
/mem0 status # Show current status
|
|
210
265
|
/mem0 search <query> # Semantic search
|
|
211
266
|
/mem0 profile # List all memories
|
|
267
|
+
/mem0 add <text> # Store a memory manually
|
|
268
|
+
/mem0 dedup # Preview exact duplicates in the current scope
|
|
269
|
+
/mem0 dedup --apply # Confirm and remove exact duplicates
|
|
270
|
+
/mem0 delete <id> # Remove a memory by id
|
|
212
271
|
```
|
|
213
272
|
|
|
214
273
|
## Relationship with pi-memory
|
|
215
274
|
|
|
216
275
|
`pi-memory-mem0` and `pi-memory` run **independently in parallel** as separate extensions:
|
|
217
276
|
|
|
218
|
-
- `pi-memory`:
|
|
219
|
-
- `pi-memory-mem0`:
|
|
277
|
+
- `pi-memory`: Curated memory — the agent explicitly manages memories via the `memory_add` / `memory_replace` / `memory_remove` / `memory_read` tools, local `.md` files, hard char limits
|
|
278
|
+
- `pi-memory-mem0`: Semantic memory — automatic extraction/storage and semantic recall (passive), plus the `mem0_memory` tool for agent-driven semantic lookup (active); no capacity limits
|
|
220
279
|
|
|
221
|
-
They do not interfere with each other. `pi-memory-mem0`
|
|
222
|
-
text into the system prompt; the agent retrieves it through the search and
|
|
223
|
-
profile tools.
|
|
280
|
+
They do not interfere with each other, and their tool names do not collide. `pi-memory-mem0` injects recalled text as a custom user-channel message (never the system prompt); `pi-memory` injects its own context separately.
|
|
224
281
|
|
|
225
282
|
## Dedup API
|
|
226
283
|
|
|
227
|
-
The package exports a standalone
|
|
284
|
+
Normal Mem0 writes use inference, so Mem0's own duplicate and conflict handling remains the primary protection. The package also exports a standalone maintenance function for previewing or removing legacy exact duplicates:
|
|
228
285
|
|
|
229
286
|
```ts
|
|
230
287
|
import { dedupMemories } from "@amaster.ai/pi-memory-mem0/dedup";
|
|
231
288
|
|
|
232
289
|
const result = await dedupMemories({
|
|
233
290
|
userId: "my-user",
|
|
291
|
+
agentId: "my-agent",
|
|
234
292
|
config: { mode: "platform", apiKey: "..." },
|
|
293
|
+
dryRun: true,
|
|
235
294
|
});
|
|
236
|
-
// result: { total: 42,
|
|
295
|
+
// result: { total: 42, duplicatesFound: 3, duplicatesRemoved: 0, deleteFailures: 0 }
|
|
237
296
|
```
|
|
238
297
|
|
|
239
|
-
|
|
298
|
+
Dedup normalizes entries using Unicode NFC, case-insensitive comparison, and collapsed whitespace, then keeps the newest exact match. Platform user and agent entity scopes are processed independently; embedded and self-hosted modes use the exact `userId` + `agentId` scope. Platform and embedded scopes are limited to 10,000 memories; self-hosted dedup requests the server maximum of 1,000 and fails closed when that limit is reached, so it only applies to scopes proven to contain at most 999 memories. Invalid or repeated IDs, empty memory content, missing, invalid, or tied timestamps, incomplete pagination, inconsistent Platform counts, and cancellation all fail closed; individual deletion failures are reported, and no automatic background cleanup is installed.
|
package/dist/dedup.d.ts
CHANGED
|
@@ -4,20 +4,26 @@
|
|
|
4
4
|
* Normalizes content, identifies exact duplicates (case-insensitive, whitespace-collapsed),
|
|
5
5
|
* deletes the older entries, and keeps only the most recently updated.
|
|
6
6
|
*
|
|
7
|
-
*
|
|
7
|
+
* All modes use the provider interface. In embedded mode the configured
|
|
8
8
|
* vector store is the source of truth and its IDs remain stable across restarts.
|
|
9
9
|
*/
|
|
10
|
-
import { type ProviderResolver } from './provider.js';
|
|
10
|
+
import { type Mem0Provider, type ProviderResolver } from './provider.js';
|
|
11
11
|
import type { Mem0ExtensionConfig } from './types.js';
|
|
12
12
|
export interface DedupOptions {
|
|
13
13
|
userId: string;
|
|
14
|
+
agentId?: string;
|
|
14
15
|
config: Mem0ExtensionConfig;
|
|
15
16
|
resolveProvider?: ProviderResolver;
|
|
16
17
|
signal?: AbortSignal;
|
|
18
|
+
dryRun?: boolean;
|
|
19
|
+
approve?: (preview: DedupResult) => boolean | Promise<boolean>;
|
|
17
20
|
}
|
|
18
21
|
export interface DedupResult {
|
|
19
22
|
total: number;
|
|
23
|
+
duplicatesFound: number;
|
|
20
24
|
duplicatesRemoved: number;
|
|
25
|
+
deleteFailures: number;
|
|
21
26
|
}
|
|
22
27
|
export declare function dedupMemories(opts: DedupOptions): Promise<DedupResult>;
|
|
28
|
+
export declare function dedupProviderMemories(provider: Mem0Provider, opts: Pick<DedupOptions, 'userId' | 'agentId' | 'signal' | 'dryRun' | 'approve'>): Promise<DedupResult>;
|
|
23
29
|
//# sourceMappingURL=dedup.d.ts.map
|
package/dist/dedup.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"dedup.d.ts","sourceRoot":"","sources":["../src/dedup.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AACH,OAAO,EAAsB,KAAK,gBAAgB,EAAE,MAAM,eAAe,CAAC;
|
|
1
|
+
{"version":3,"file":"dedup.d.ts","sourceRoot":"","sources":["../src/dedup.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AACH,OAAO,EAAsB,KAAK,YAAY,EAAE,KAAK,gBAAgB,EAAE,MAAM,eAAe,CAAC;AAC7F,OAAO,KAAK,EAAE,mBAAmB,EAAc,MAAM,YAAY,CAAC;AAElE,MAAM,WAAW,YAAY;IAC3B,MAAM,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,mBAAmB,CAAC;IAC5B,eAAe,CAAC,EAAE,gBAAgB,CAAC;IACnC,MAAM,CAAC,EAAE,WAAW,CAAC;IACrB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,OAAO,CAAC,EAAE,CAAC,OAAO,EAAE,WAAW,KAAK,OAAO,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;CAChE;AAED,MAAM,WAAW,WAAW;IAC1B,KAAK,EAAE,MAAM,CAAC;IACd,eAAe,EAAE,MAAM,CAAC;IACxB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,cAAc,EAAE,MAAM,CAAC;CACxB;AAED,wBAAsB,aAAa,CAAC,IAAI,EAAE,YAAY,GAAG,OAAO,CAAC,WAAW,CAAC,CAQ5E;AAED,wBAAsB,qBAAqB,CACzC,QAAQ,EAAE,YAAY,EACtB,IAAI,EAAE,IAAI,CAAC,YAAY,EAAE,QAAQ,GAAG,SAAS,GAAG,QAAQ,GAAG,QAAQ,GAAG,SAAS,CAAC,GAC/E,OAAO,CAAC,WAAW,CAAC,CA6DtB"}
|
package/dist/dedup.js
CHANGED
|
@@ -4,61 +4,126 @@
|
|
|
4
4
|
* Normalizes content, identifies exact duplicates (case-insensitive, whitespace-collapsed),
|
|
5
5
|
* deletes the older entries, and keeps only the most recently updated.
|
|
6
6
|
*
|
|
7
|
-
*
|
|
7
|
+
* All modes use the provider interface. In embedded mode the configured
|
|
8
8
|
* vector store is the source of truth and its IDs remain stable across restarts.
|
|
9
9
|
*/
|
|
10
10
|
import { createMem0Provider } from './provider.js';
|
|
11
11
|
export async function dedupMemories(opts) {
|
|
12
|
-
const {
|
|
12
|
+
const { config, resolveProvider } = opts;
|
|
13
13
|
const provider = await createMem0Provider({
|
|
14
14
|
config,
|
|
15
15
|
...(resolveProvider ? { resolveProvider } : {}),
|
|
16
16
|
});
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
17
|
+
return dedupProviderMemories(provider, opts);
|
|
18
|
+
}
|
|
19
|
+
export async function dedupProviderMemories(provider, opts) {
|
|
20
|
+
const { userId, agentId, signal, dryRun = false, approve } = opts;
|
|
21
|
+
const scope = {
|
|
22
|
+
userId,
|
|
23
|
+
...(agentId ? { agentId } : {}),
|
|
24
|
+
...(signal ? { signal } : {}),
|
|
25
|
+
};
|
|
26
|
+
if (!provider.getDedupGroups) {
|
|
27
|
+
throw new Error('The configured Mem0 provider does not support safe deduplication.');
|
|
28
|
+
}
|
|
29
|
+
signal?.throwIfAborted();
|
|
30
|
+
const groups = await provider.getDedupGroups(scope);
|
|
31
|
+
signal?.throwIfAborted();
|
|
32
|
+
const memoryIds = new Set();
|
|
33
|
+
for (const item of groups.flat()) {
|
|
34
|
+
signal?.throwIfAborted();
|
|
35
|
+
const { id } = item;
|
|
36
|
+
if (!id || /\s/.test(id) || memoryIds.has(id)) {
|
|
37
|
+
throw new Error('Mem0 dedup received an invalid or duplicate memory ID.');
|
|
38
|
+
}
|
|
39
|
+
memoryIds.add(id);
|
|
40
|
+
if (!item.memory.trim()) {
|
|
41
|
+
throw new Error('Mem0 dedup received empty memory content.');
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
const total = groups.reduce((count, group) => count + group.length, 0);
|
|
45
|
+
const duplicateIds = new Set(groups.flatMap((group) => findDuplicateIds(group, signal)));
|
|
46
|
+
const preview = {
|
|
47
|
+
total,
|
|
48
|
+
duplicatesFound: duplicateIds.size,
|
|
49
|
+
duplicatesRemoved: 0,
|
|
50
|
+
deleteFailures: 0,
|
|
51
|
+
};
|
|
52
|
+
if (dryRun)
|
|
53
|
+
return preview;
|
|
54
|
+
if (approve) {
|
|
55
|
+
const approved = await approve(preview);
|
|
56
|
+
signal?.throwIfAborted();
|
|
57
|
+
if (!approved)
|
|
58
|
+
return preview;
|
|
20
59
|
}
|
|
21
|
-
const duplicateIds = findDuplicateIds(allMemories, signal);
|
|
22
60
|
let removed = 0;
|
|
61
|
+
let failures = 0;
|
|
23
62
|
for (const id of duplicateIds) {
|
|
24
|
-
|
|
25
|
-
break;
|
|
63
|
+
signal?.throwIfAborted();
|
|
26
64
|
try {
|
|
27
|
-
await provider.delete(id);
|
|
65
|
+
await (signal ? provider.delete(id, { signal }) : provider.delete(id));
|
|
66
|
+
signal?.throwIfAborted();
|
|
28
67
|
removed++;
|
|
29
68
|
}
|
|
30
69
|
catch {
|
|
31
|
-
|
|
70
|
+
signal?.throwIfAborted();
|
|
71
|
+
failures++;
|
|
32
72
|
}
|
|
33
73
|
}
|
|
34
|
-
return {
|
|
74
|
+
return {
|
|
75
|
+
total,
|
|
76
|
+
duplicatesFound: duplicateIds.size,
|
|
77
|
+
duplicatesRemoved: removed,
|
|
78
|
+
deleteFailures: failures,
|
|
79
|
+
};
|
|
35
80
|
}
|
|
36
81
|
/**
|
|
37
82
|
* Shared: find IDs of duplicate entries to remove (keeps the newest).
|
|
38
83
|
*/
|
|
39
84
|
function findDuplicateIds(allMemories, signal) {
|
|
40
|
-
const
|
|
41
|
-
const duplicateIds = [];
|
|
85
|
+
const groups = new Map();
|
|
42
86
|
for (const item of allMemories) {
|
|
43
|
-
|
|
44
|
-
break;
|
|
87
|
+
signal?.throwIfAborted();
|
|
45
88
|
const normalized = item.memory.normalize('NFC').replace(/\s+/g, ' ').trim().toLowerCase();
|
|
46
|
-
const
|
|
47
|
-
if (
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
89
|
+
const group = groups.get(normalized);
|
|
90
|
+
if (group)
|
|
91
|
+
group.push(item);
|
|
92
|
+
else
|
|
93
|
+
groups.set(normalized, [item]);
|
|
94
|
+
}
|
|
95
|
+
const duplicateIds = [];
|
|
96
|
+
for (const group of groups.values()) {
|
|
97
|
+
if (group.length < 2)
|
|
98
|
+
continue;
|
|
99
|
+
const [first, ...rest] = group;
|
|
100
|
+
if (!first)
|
|
101
|
+
continue;
|
|
102
|
+
let newest = first;
|
|
103
|
+
let newestTime = memoryTime(first);
|
|
104
|
+
const times = new Set([newestTime]);
|
|
105
|
+
for (const item of rest) {
|
|
106
|
+
signal?.throwIfAborted();
|
|
107
|
+
const time = memoryTime(item);
|
|
108
|
+
if (times.has(time)) {
|
|
109
|
+
throw new Error('Mem0 dedup cannot determine the newest duplicate.');
|
|
53
110
|
}
|
|
54
|
-
|
|
55
|
-
|
|
111
|
+
times.add(time);
|
|
112
|
+
if (time > newestTime) {
|
|
113
|
+
newest = item;
|
|
114
|
+
newestTime = time;
|
|
56
115
|
}
|
|
57
116
|
}
|
|
58
|
-
|
|
59
|
-
seen.set(normalized, item);
|
|
60
|
-
}
|
|
117
|
+
duplicateIds.push(...group.filter((item) => item !== newest).map((item) => item.id));
|
|
61
118
|
}
|
|
62
119
|
return duplicateIds;
|
|
63
120
|
}
|
|
121
|
+
function memoryTime(item) {
|
|
122
|
+
const timestamp = item.updated_at ?? item.created_at;
|
|
123
|
+
const time = timestamp ? new Date(timestamp).getTime() : Number.NaN;
|
|
124
|
+
if (!Number.isFinite(time)) {
|
|
125
|
+
throw new Error('Mem0 dedup cannot determine the newest duplicate.');
|
|
126
|
+
}
|
|
127
|
+
return time;
|
|
128
|
+
}
|
|
64
129
|
//# sourceMappingURL=dedup.js.map
|
package/dist/dedup.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"dedup.js","sourceRoot":"","sources":["../src/dedup.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AACH,OAAO,EAAE,kBAAkB,
|
|
1
|
+
{"version":3,"file":"dedup.js","sourceRoot":"","sources":["../src/dedup.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AACH,OAAO,EAAE,kBAAkB,EAA4C,MAAM,eAAe,CAAC;AAoB7F,MAAM,CAAC,KAAK,UAAU,aAAa,CAAC,IAAkB;IACpD,MAAM,EAAE,MAAM,EAAE,eAAe,EAAE,GAAG,IAAI,CAAC;IAEzC,MAAM,QAAQ,GAAG,MAAM,kBAAkB,CAAC;QACxC,MAAM;QACN,GAAG,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,eAAe,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAChD,CAAC,CAAC;IACH,OAAO,qBAAqB,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC;AAC/C,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,qBAAqB,CACzC,QAAsB,EACtB,IAAgF;IAEhF,MAAM,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,IAAI,CAAC;IAClE,MAAM,KAAK,GAAG;QACZ,MAAM;QACN,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAC/B,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAC9B,CAAC;IACF,IAAI,CAAC,QAAQ,CAAC,cAAc,EAAE,CAAC;QAC7B,MAAM,IAAI,KAAK,CAAC,mEAAmE,CAAC,CAAC;IACvF,CAAC;IACD,MAAM,EAAE,cAAc,EAAE,CAAC;IACzB,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,cAAc,CAAC,KAAK,CAAC,CAAC;IACpD,MAAM,EAAE,cAAc,EAAE,CAAC;IACzB,MAAM,SAAS,GAAG,IAAI,GAAG,EAAU,CAAC;IACpC,KAAK,MAAM,IAAI,IAAI,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;QACjC,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,MAAM,EAAE,EAAE,EAAE,GAAG,IAAI,CAAC;QACpB,IAAI,CAAC,EAAE,IAAI,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE,CAAC;YAC9C,MAAM,IAAI,KAAK,CAAC,wDAAwD,CAAC,CAAC;QAC5E,CAAC;QACD,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QAClB,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;YACxB,MAAM,IAAI,KAAK,CAAC,2CAA2C,CAAC,CAAC;QAC/D,CAAC;IACH,CAAC;IACD,MAAM,KAAK,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAEvE,MAAM,YAAY,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,gBAAgB,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC;IACzF,MAAM,OAAO,GAAG;QACd,KAAK;QACL,eAAe,EAAE,YAAY,CAAC,IAAI;QAClC,iBAAiB,EAAE,CAAC;QACpB,cAAc,EAAE,CAAC;KAClB,CAAC;IACF,IAAI,MAAM;QAAE,OAAO,OAAO,CAAC;IAC3B,IAAI,OAAO,EAAE,CAAC;QACZ,MAAM,QAAQ,GAAG,MAAM,OAAO,CAAC,OAAO,CAAC,CAAC;QACxC,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,IAAI,CAAC,QAAQ;YAAE,OAAO,OAAO,CAAC;IAChC,CAAC;IAED,IAAI,OAAO,GAAG,CAAC,CAAC;IAChB,IAAI,QAAQ,GAAG,CAAC,CAAC;IACjB,KAAK,MAAM,EAAE,IAAI,YAAY,EAAE,CAAC;QAC9B,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,IAAI,CAAC;YACH,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,EAAE,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;YACvE,MAAM,EAAE,cAAc,EAAE,CAAC;YACzB,OAAO,EAAE,CAAC;QACZ,CAAC;QAAC,MAAM,CAAC;YACP,MAAM,EAAE,cAAc,EAAE,CAAC;YACzB,QAAQ,EAAE,CAAC;QACb,CAAC;IACH,CAAC;IAED,OAAO;QACL,KAAK;QACL,eAAe,EAAE,YAAY,CAAC,IAAI;QAClC,iBAAiB,EAAE,OAAO;QAC1B,cAAc,EAAE,QAAQ;KACzB,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,SAAS,gBAAgB,CAAC,WAAyB,EAAE,MAAoB;IACvE,MAAM,MAAM,GAAG,IAAI,GAAG,EAAwB,CAAC;IAE/C,KAAK,MAAM,IAAI,IAAI,WAAW,EAAE,CAAC;QAC/B,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,MAAM,UAAU,GAAG,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;QAC1F,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,UAAU,CAAC,CAAC;QACrC,IAAI,KAAK;YAAE,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;;YACvB,MAAM,CAAC,GAAG,CAAC,UAAU,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC;IACtC,CAAC;IAED,MAAM,YAAY,GAAa,EAAE,CAAC;IAClC,KAAK,MAAM,KAAK,IAAI,MAAM,CAAC,MAAM,EAAE,EAAE,CAAC;QACpC,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC;YAAE,SAAS;QAC/B,MAAM,CAAC,KAAK,EAAE,GAAG,IAAI,CAAC,GAAG,KAAK,CAAC;QAC/B,IAAI,CAAC,KAAK;YAAE,SAAS;QACrB,IAAI,MAAM,GAAG,KAAK,CAAC;QACnB,IAAI,UAAU,GAAG,UAAU,CAAC,KAAK,CAAC,CAAC;QACnC,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC;QACpC,KAAK,MAAM,IAAI,IAAI,IAAI,EAAE,CAAC;YACxB,MAAM,EAAE,cAAc,EAAE,CAAC;YACzB,MAAM,IAAI,GAAG,UAAU,CAAC,IAAI,CAAC,CAAC;YAC9B,IAAI,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC;gBACpB,MAAM,IAAI,KAAK,CAAC,mDAAmD,CAAC,CAAC;YACvE,CAAC;YACD,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;YAChB,IAAI,IAAI,GAAG,UAAU,EAAE,CAAC;gBACtB,MAAM,GAAG,IAAI,CAAC;gBACd,UAAU,GAAG,IAAI,CAAC;YACpB,CAAC;QACH,CAAC;QACD,YAAY,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;IACvF,CAAC;IAED,OAAO,YAAY,CAAC;AACtB,CAAC;AAED,SAAS,UAAU,CAAC,IAAgB;IAClC,MAAM,SAAS,GAAG,IAAI,CAAC,UAAU,IAAI,IAAI,CAAC,UAAU,CAAC;IACrD,MAAM,IAAI,GAAG,SAAS,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC;IACpE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;QAC3B,MAAM,IAAI,KAAK,CAAC,mDAAmD,CAAC,CAAC;IACvE,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC"}
|
package/dist/index.d.ts
CHANGED
|
@@ -1,9 +1,23 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* pi-memory-mem0 —
|
|
2
|
+
* pi-memory-mem0 — Semantic memory extension powered by Mem0.
|
|
3
3
|
*
|
|
4
|
-
*
|
|
4
|
+
* Backend modes:
|
|
5
5
|
* - **platform**: Uses Mem0 Cloud API (needs MEM0_API_KEY)
|
|
6
|
-
* - **
|
|
6
|
+
* - **embedded**: Runs Mem0 OSS in-process
|
|
7
|
+
* - **self-hosted**: Calls a remote Mem0 OSS REST server
|
|
8
|
+
*
|
|
9
|
+
* Memory modes (config "memoryMode"):
|
|
10
|
+
* - **passive**: automatic capture + recall injection only
|
|
11
|
+
* - **active**: LLM-callable mem0_memory tool only
|
|
12
|
+
* - **hybrid** (default): both
|
|
13
|
+
*
|
|
14
|
+
* Passive side: after each conversation turn, user + assistant messages are
|
|
15
|
+
* sent to Mem0 for fact extraction and storage (credentials are redacted
|
|
16
|
+
* first). Recalled memories are injected as a custom message (delivered to
|
|
17
|
+
* the model on the user channel, never the system prompt) and wrapped as
|
|
18
|
+
* untrusted data. Active side: the mem0_memory tool lets the agent search,
|
|
19
|
+
* add, list, and delete memories on its own initiative, under the same
|
|
20
|
+
* redaction and untrusted-data boundaries.
|
|
7
21
|
*
|
|
8
22
|
* Configuration via settings.json key "pi-memory-mem0".
|
|
9
23
|
* Supports ${ENV_VAR:-fallback} in user and agent settings.
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AAGH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,iCAAiC,CAAC;AAmCpE,MAAM,CAAC,OAAO,UAAU,aAAa,CAAC,EAAE,EAAE,YAAY,GAAG,IAAI,CAgU5D"}
|