@amaster.ai/pi-memory-mem0 0.1.8 → 0.1.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,22 +2,49 @@
2
2
 
3
3
  ![pi-memory-mem0 preview](https://raw.githubusercontent.com/TGYD-helige/pi/master/packages/pi-memory-mem0/preview.png)
4
4
 
5
- Explicit semantic memory tools powered by [Mem0](https://mem0.ai), with support for both Platform (cloud) and Open-Source (local) modes.
5
+ Semantic memory extension powered by [Mem0](https://mem0.ai), with Platform, embedded, and self-hosted backends and three memory modes (hybrid, active, passive).
6
6
 
7
7
  ## How It Works
8
8
 
9
- Memories are saved and recalled through explicit tools. They are project-namespaced, and stored text is not injected into the system prompt.
9
+ **Passive side**: after each conversation turn, the user + assistant messages are automatically sent to Mem0 for fact extraction and storage. When you send a prompt, a semantic search is prefetched in the background and relevant memories are recalled before the agent starts — **zero effort required**.
10
10
 
11
- Use `mem0_save` and the search/profile tools when memory should be stored or retrieved.
11
+ **Active side**: the agent gets a `mem0_memory` tool (`search` / `add` / `get_all` / `delete`) so it can look up and manage memories on its own initiative — same idea as the official `@mem0/pi-agent-plugin`, but backed by this package's provider abstraction, so it works with Platform, embedded, **and** self-hosted Mem0.
12
12
 
13
- ## Two Modes
13
+ Safety boundaries:
14
+
15
+ - **Recall channel**: recalled memories are injected as a custom message delivered on the user channel — never into the system prompt. Each entry is wrapped as `[UNTRUSTED MEMORY DATA]` (JSON-quoted), and entries matching prompt-injection patterns are replaced with `[BLOCKED UNTRUSTED MEMORY: ...]`.
16
+ - **Project namespacing**: memories are scoped to `userId:project:<cwd-hash>` by default, so one project's memories are not visible in another. Set `userIdScope: "exact"` when one identity must be shared across projects.
17
+ - **Credential redaction**: private keys, bearer tokens, and `api_key=…`-style values are redacted before anything is stored.
18
+ - **Platform mode disclosure**: in `platform` mode the captured turns leave your machine and are processed by Mem0 Cloud. Use `embedded` or `self-hosted` mode to keep memory traffic off third-party infrastructure.
19
+
20
+ ## Modes
21
+
22
+ ### Backend Modes
14
23
 
15
24
  | Mode | Vector Store | Persistence | Dependencies | Use Case |
16
25
  |------|-------------|-------------|--------------|----------|
17
26
  | `platform` | Mem0 Cloud | Cloud-managed | `MEM0_API_KEY` | Quick start, multi-device sync |
18
- | `open-source` | Mem0 OSS vector store | Vector-store managed | LLM + Embedding API | Data privacy, no Mem0 Cloud |
27
+ | `embedded` | Mem0 OSS vector store | Vector-store managed | LLM + Embedding API | Data privacy, no Mem0 Cloud |
28
+ | `self-hosted` | Mem0 OSS REST server | Server-managed | Server URL, optional API key | Shared infrastructure |
29
+
30
+ ### Memory Modes
31
+
32
+ | `memoryMode` | Auto capture + recall | `mem0_memory` tool | Use Case |
33
+ |--------------|----------------------|--------------------|----------|
34
+ | `hybrid` (default) | ✅ | ✅ | Best recall: automatic context plus agent-driven lookup |
35
+ | `active` | ❌ | ✅ | No background traffic; the agent decides every read/write |
36
+ | `passive` | ✅ | ❌ | Zero tool surface; fully automatic memory |
37
+
38
+ ```json
39
+ {
40
+ "pi-memory-mem0": {
41
+ "mode": "embedded",
42
+ "memoryMode": "hybrid"
43
+ }
44
+ }
45
+ ```
19
46
 
20
- ## Architecture (Open-Source Mode)
47
+ ## Architecture (Embedded Mode)
21
48
 
22
49
  ```
23
50
  User ←→ Agent ←→ Mem0 OSS Memory
@@ -33,10 +60,7 @@ User ←→ Agent ←→ Mem0 OSS Memory
33
60
 
34
61
  ## Quick Start
35
62
 
36
- Store configuration in user/agent settings or in a trusted project's
37
- `.pi/settings.json`. Project settings are ignored when trust is declined and do
38
- not expand `${ENV_VAR}`; the environment-backed examples below therefore belong
39
- in user or agent settings.
63
+ Store configuration in user/agent settings or in a trusted project's `.pi/settings.json`. Project settings are ignored when trust is declined and do not expand `${ENV_VAR}`; the environment-backed examples below therefore belong in user or agent settings.
40
64
 
41
65
  ### Platform Mode
42
66
 
@@ -50,14 +74,14 @@ in user or agent settings.
50
74
  }
51
75
  ```
52
76
 
53
- ### Open-Source Mode (Recommended)
77
+ ### Embedded Mode (Recommended)
54
78
 
55
79
  Reuses API keys and base URLs from pi's configured model providers — **no extra environment variables needed**.
56
80
 
57
81
  ```json
58
82
  {
59
83
  "pi-memory-mem0": {
60
- "mode": "open-source",
84
+ "mode": "embedded",
61
85
  "userId": "${USER}"
62
86
  }
63
87
  }
@@ -65,6 +89,25 @@ Reuses API keys and base URLs from pi's configured model providers — **no extr
65
89
 
66
90
  Defaults to OpenAI `text-embedding-3-small` (embedding) + `gpt-4.1-nano` (extraction). API keys and base URLs are automatically resolved from pi's model registry.
67
91
 
92
+ Existing settings with `mode: "open-source"` continue to load as `embedded`, but `open-source` is not part of the supported configuration interface. New settings should use `embedded`.
93
+
94
+ ### Self-Hosted Mode
95
+
96
+ Calls the OSS REST server directly. The server uses `/memories` and `/search`, not the Mem0 Platform `/v1` paths.
97
+
98
+ ```json
99
+ {
100
+ "pi-memory-mem0": {
101
+ "mode": "self-hosted",
102
+ "baseUrl": "${MEM0_BASE_URL}",
103
+ "apiKey": "${MEM0_API_KEY}",
104
+ "userId": "${PAPERCLIP_COMPANY_ID}",
105
+ "agentId": "${PAPERCLIP_AGENT_ID}",
106
+ "userIdScope": "exact"
107
+ }
108
+ }
109
+ ```
110
+
68
111
  ### Custom Provider
69
112
 
70
113
  When your model registry defines a custom provider with `api: "openai-completions"`, you can use it directly:
@@ -72,7 +115,7 @@ When your model registry defines a custom provider with `api: "openai-completion
72
115
  ```json
73
116
  {
74
117
  "pi-memory-mem0": {
75
- "mode": "open-source",
118
+ "mode": "embedded",
76
119
  "oss": {
77
120
  "llm": {
78
121
  "provider": "my-provider",
@@ -97,7 +140,7 @@ The extension automatically:
97
140
  ```json
98
141
  {
99
142
  "pi-memory-mem0": {
100
- "mode": "open-source",
143
+ "mode": "embedded",
101
144
  "userId": "${USER}",
102
145
  "oss": {
103
146
  "llm": {
@@ -121,7 +164,7 @@ For production workloads that need a dedicated vector database:
121
164
  ```json
122
165
  {
123
166
  "pi-memory-mem0": {
124
- "mode": "open-source",
167
+ "mode": "embedded",
125
168
  "oss": {
126
169
  "vectorStore": {
127
170
  "provider": "qdrant",
@@ -140,10 +183,14 @@ The configured vector store always owns persistence. To request an intentionally
140
183
 
141
184
  | Field | Type | Default | Description |
142
185
  |-------|------|---------|-------------|
143
- | `mode` | `"platform"` \| `"open-source"` | `"platform"` | Operating mode |
144
- | `apiKey` | string | — | Required for platform mode. Supports `${MEM0_API_KEY}` |
145
- | `baseUrl` | string | `https://api.mem0.ai` | Custom platform endpoint |
186
+ | `mode` | `"platform"` \| `"embedded"` \| `"self-hosted"` | `"platform"` | Operating mode |
187
+ | `memoryMode` | `"hybrid"` \| `"active"` \| `"passive"` | `"hybrid"` | Memory behavior: tool + automation, tool only, or automation only |
188
+ | `apiKey` | string | — | Platform or self-hosted API key. Supports `${MEM0_API_KEY}` |
189
+ | `baseUrl` | string | `https://api.mem0.ai` | Platform override; required for self-hosted mode |
190
+ | `requestTimeoutMs` | number | `30000` | Self-hosted request timeout |
146
191
  | `userId` | string | `$USER` or `"default-user"` | Memory scoping identifier |
192
+ | `agentId` | string | — | Optional agent scope. Supports environment interpolation |
193
+ | `userIdScope` | `"project"` \| `"exact"` | `"project"` | Append the cwd hash or use `userId` verbatim |
147
194
  | `topK` | number | `5` | Max recalled memories per turn |
148
195
  | `useRegistryKeys` | boolean | `true` | Whether OSS mode resolves keys from pi registry |
149
196
  | `oss.llm` | object | OpenAI gpt-4.1-nano | OSS extraction model |
@@ -153,14 +200,17 @@ The configured vector store always owns persistence. To request an intentionally
153
200
  | `oss.historyDbPath` | string | `<home>/memories/mem0-history.db` | Shortcut for SQLite history DB path |
154
201
  | `oss.disableHistory` | boolean | `false` | Disable mem0 operation history |
155
202
 
203
+ Embedded and self-hosted modes read and write the exact `userId` + `agentId` scope; Platform mode reads the user and agent entity scopes with OR because Mem0 Platform stores them as separate records. Existing memories with a null `agent_id` are not backfilled automatically.
204
+
156
205
  ## Data Storage
157
206
 
158
207
  | Mode | Vector Data | History |
159
208
  |------|-------------|---------|
160
209
  | Platform | Mem0 Cloud | Cloud-managed |
161
- | Open-Source (default) | `<home>/memories/mem0-vectors.db` | `<home>/memories/mem0-history.db` |
162
- | Open-Source (`memory`, `dbPath: ":memory:"`) | Process-local SQLite; lost on restart | `<home>/memories/mem0-history.db` |
163
- | Open-Source (Qdrant) | Qdrant server | `<home>/memories/mem0-history.db` |
210
+ | Embedded (default paths) | `<home>/memories/mem0-vectors.db` | `<home>/memories/mem0-history.db` |
211
+ | Embedded (`memory`, `dbPath: ":memory:"`) | Process-local SQLite; lost on restart | `<home>/memories/mem0-history.db` |
212
+ | Embedded (Qdrant) | Qdrant server | `<home>/memories/mem0-history.db` |
213
+ | Self-hosted | Remote server | Remote server |
164
214
 
165
215
  The `home` directory is resolved via `resolveHome()` from `@amaster.ai/pi-shared/settings` (defaults to `~/.pi/agent`).
166
216
 
@@ -179,7 +229,7 @@ This happens transparently — just configure the provider name as it appears in
179
229
 
180
230
  ## Installation Notes
181
231
 
182
- The default Open-Source mode depends on `better-sqlite3` (native addon, transitive dependency of `mem0ai`) for both the vector store and history. This remains true for `dbPath: ":memory:"`: it changes where SQLite stores pages, not which vector-store implementation is used.
232
+ The default Embedded configuration depends on `better-sqlite3` (native addon, transitive dependency of `mem0ai`) for both the vector store and history. This remains true for `dbPath: ":memory:"`: it changes where SQLite stores pages, not which vector-store implementation is used.
183
233
 
184
234
  **For pi-agent users**: pi-agent's `package.json` includes `better-sqlite3` in `pnpm.onlyBuiltDependencies` — it compiles automatically during `pnpm install`. No extra steps needed.
185
235
 
@@ -197,11 +247,16 @@ If `better-sqlite3` fails to load (for example, because of a Node ABI mismatch),
197
247
 
198
248
  ## Tools
199
249
 
200
- | Tool | Description |
201
- |------|-------------|
202
- | `mem0_search` | Semantic search over long-term memories |
203
- | `mem0_profile` | List all stored memories |
204
- | `mem0_save` | Store a fact verbatim (bypasses LLM extraction) |
250
+ When `memoryMode` is `hybrid` or `active`, the agent gets one action-dispatched tool:
251
+
252
+ ```
253
+ mem0_memory(action="search", query="...") # Semantic search over memories
254
+ mem0_memory(action="add", content="...") # Store a durable fact
255
+ mem0_memory(action="get_all") # List every stored memory
256
+ mem0_memory(action="delete", memory_id="...") # Remove a memory by id
257
+ ```
258
+
259
+ Stored content is credential-redacted first; results are returned with the same `[UNTRUSTED MEMORY DATA]` wrapping as passive recall, including the memory ids needed for `delete`.
205
260
 
206
261
  ## Commands
207
262
 
@@ -209,31 +264,35 @@ If `better-sqlite3` fails to load (for example, because of a Node ABI mismatch),
209
264
  /mem0 status # Show current status
210
265
  /mem0 search <query> # Semantic search
211
266
  /mem0 profile # List all memories
267
+ /mem0 add <text> # Store a memory manually
268
+ /mem0 dedup # Preview exact duplicates in the current scope
269
+ /mem0 dedup --apply # Confirm and remove exact duplicates
270
+ /mem0 delete <id> # Remove a memory by id
212
271
  ```
213
272
 
214
273
  ## Relationship with pi-memory
215
274
 
216
275
  `pi-memory-mem0` and `pi-memory` run **independently in parallel** as separate extensions:
217
276
 
218
- - `pi-memory`: Active memory — agent explicitly manages via tools, local `.md` files, hard char limits
219
- - `pi-memory-mem0`: Explicit semantic memory tools backed by Mem0
277
+ - `pi-memory`: Curated memory — the agent explicitly manages memories via the `memory_add` / `memory_replace` / `memory_remove` / `memory_read` tools, local `.md` files, hard char limits
278
+ - `pi-memory-mem0`: Semantic memory — automatic extraction/storage and semantic recall (passive), plus the `mem0_memory` tool for agent-driven semantic lookup (active); no capacity limits
220
279
 
221
- They do not interfere with each other. `pi-memory-mem0` does not inject recalled
222
- text into the system prompt; the agent retrieves it through the search and
223
- profile tools.
280
+ They do not interfere with each other, and their tool names do not collide. `pi-memory-mem0` injects recalled text as a custom user-channel message (never the system prompt); `pi-memory` injects its own context separately.
224
281
 
225
282
  ## Dedup API
226
283
 
227
- The package exports a standalone deduplication function used by pi-memory's dreaming job:
284
+ Normal Mem0 writes use inference, so Mem0's own duplicate and conflict handling remains the primary protection. The package also exports a standalone maintenance function for previewing or removing legacy exact duplicates:
228
285
 
229
286
  ```ts
230
287
  import { dedupMemories } from "@amaster.ai/pi-memory-mem0/dedup";
231
288
 
232
289
  const result = await dedupMemories({
233
290
  userId: "my-user",
291
+ agentId: "my-agent",
234
292
  config: { mode: "platform", apiKey: "..." },
293
+ dryRun: true,
235
294
  });
236
- // result: { total: 42, duplicatesRemoved: 3 }
295
+ // result: { total: 42, duplicatesFound: 3, duplicatesRemoved: 0, deleteFailures: 0 }
237
296
  ```
238
297
 
239
- Normalizes entries (case-insensitive, whitespace-collapsed), identifies exact duplicates, and deletes the older ones through the configured provider. In OSS mode those deletes go directly to the vector store.
298
+ Dedup normalizes entries using Unicode NFC, case-insensitive comparison, and collapsed whitespace, then keeps the newest exact match. Platform user and agent entity scopes are processed independently; embedded and self-hosted modes use the exact `userId` + `agentId` scope. Platform and embedded scopes are limited to 10,000 memories; self-hosted dedup requests the server maximum of 1,000 and fails closed when that limit is reached, so it only applies to scopes proven to contain at most 999 memories. Invalid or repeated IDs, empty memory content, missing, invalid, or tied timestamps, incomplete pagination, inconsistent Platform counts, and cancellation all fail closed; individual deletion failures are reported, and no automatic background cleanup is installed.
package/dist/dedup.d.ts CHANGED
@@ -4,20 +4,26 @@
4
4
  * Normalizes content, identifies exact duplicates (case-insensitive, whitespace-collapsed),
5
5
  * deletes the older entries, and keeps only the most recently updated.
6
6
  *
7
- * Both Platform and OSS modes use the provider API. In OSS mode the configured
7
+ * All modes use the provider interface. In embedded mode the configured
8
8
  * vector store is the source of truth and its IDs remain stable across restarts.
9
9
  */
10
- import { type ProviderResolver } from './provider.js';
10
+ import { type Mem0Provider, type ProviderResolver } from './provider.js';
11
11
  import type { Mem0ExtensionConfig } from './types.js';
12
12
  export interface DedupOptions {
13
13
  userId: string;
14
+ agentId?: string;
14
15
  config: Mem0ExtensionConfig;
15
16
  resolveProvider?: ProviderResolver;
16
17
  signal?: AbortSignal;
18
+ dryRun?: boolean;
19
+ approve?: (preview: DedupResult) => boolean | Promise<boolean>;
17
20
  }
18
21
  export interface DedupResult {
19
22
  total: number;
23
+ duplicatesFound: number;
20
24
  duplicatesRemoved: number;
25
+ deleteFailures: number;
21
26
  }
22
27
  export declare function dedupMemories(opts: DedupOptions): Promise<DedupResult>;
28
+ export declare function dedupProviderMemories(provider: Mem0Provider, opts: Pick<DedupOptions, 'userId' | 'agentId' | 'signal' | 'dryRun' | 'approve'>): Promise<DedupResult>;
23
29
  //# sourceMappingURL=dedup.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"dedup.d.ts","sourceRoot":"","sources":["../src/dedup.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AACH,OAAO,EAAsB,KAAK,gBAAgB,EAAE,MAAM,eAAe,CAAC;AAC1E,OAAO,KAAK,EAAE,mBAAmB,EAAc,MAAM,YAAY,CAAC;AAElE,MAAM,WAAW,YAAY;IAC3B,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,mBAAmB,CAAC;IAC5B,eAAe,CAAC,EAAE,gBAAgB,CAAC;IACnC,MAAM,CAAC,EAAE,WAAW,CAAC;CACtB;AAED,MAAM,WAAW,WAAW;IAC1B,KAAK,EAAE,MAAM,CAAC;IACd,iBAAiB,EAAE,MAAM,CAAC;CAC3B;AAED,wBAAsB,aAAa,CAAC,IAAI,EAAE,YAAY,GAAG,OAAO,CAAC,WAAW,CAAC,CA2B5E"}
1
+ {"version":3,"file":"dedup.d.ts","sourceRoot":"","sources":["../src/dedup.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AACH,OAAO,EAAsB,KAAK,YAAY,EAAE,KAAK,gBAAgB,EAAE,MAAM,eAAe,CAAC;AAC7F,OAAO,KAAK,EAAE,mBAAmB,EAAc,MAAM,YAAY,CAAC;AAElE,MAAM,WAAW,YAAY;IAC3B,MAAM,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,mBAAmB,CAAC;IAC5B,eAAe,CAAC,EAAE,gBAAgB,CAAC;IACnC,MAAM,CAAC,EAAE,WAAW,CAAC;IACrB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,OAAO,CAAC,EAAE,CAAC,OAAO,EAAE,WAAW,KAAK,OAAO,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;CAChE;AAED,MAAM,WAAW,WAAW;IAC1B,KAAK,EAAE,MAAM,CAAC;IACd,eAAe,EAAE,MAAM,CAAC;IACxB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,cAAc,EAAE,MAAM,CAAC;CACxB;AAED,wBAAsB,aAAa,CAAC,IAAI,EAAE,YAAY,GAAG,OAAO,CAAC,WAAW,CAAC,CAQ5E;AAED,wBAAsB,qBAAqB,CACzC,QAAQ,EAAE,YAAY,EACtB,IAAI,EAAE,IAAI,CAAC,YAAY,EAAE,QAAQ,GAAG,SAAS,GAAG,QAAQ,GAAG,QAAQ,GAAG,SAAS,CAAC,GAC/E,OAAO,CAAC,WAAW,CAAC,CA6DtB"}
package/dist/dedup.js CHANGED
@@ -4,61 +4,126 @@
4
4
  * Normalizes content, identifies exact duplicates (case-insensitive, whitespace-collapsed),
5
5
  * deletes the older entries, and keeps only the most recently updated.
6
6
  *
7
- * Both Platform and OSS modes use the provider API. In OSS mode the configured
7
+ * All modes use the provider interface. In embedded mode the configured
8
8
  * vector store is the source of truth and its IDs remain stable across restarts.
9
9
  */
10
10
  import { createMem0Provider } from './provider.js';
11
11
  export async function dedupMemories(opts) {
12
- const { userId, config, resolveProvider, signal } = opts;
12
+ const { config, resolveProvider } = opts;
13
13
  const provider = await createMem0Provider({
14
14
  config,
15
15
  ...(resolveProvider ? { resolveProvider } : {}),
16
16
  });
17
- const allMemories = await provider.getAll({ userId });
18
- if (allMemories.length === 0) {
19
- return { total: 0, duplicatesRemoved: 0 };
17
+ return dedupProviderMemories(provider, opts);
18
+ }
19
+ export async function dedupProviderMemories(provider, opts) {
20
+ const { userId, agentId, signal, dryRun = false, approve } = opts;
21
+ const scope = {
22
+ userId,
23
+ ...(agentId ? { agentId } : {}),
24
+ ...(signal ? { signal } : {}),
25
+ };
26
+ if (!provider.getDedupGroups) {
27
+ throw new Error('The configured Mem0 provider does not support safe deduplication.');
28
+ }
29
+ signal?.throwIfAborted();
30
+ const groups = await provider.getDedupGroups(scope);
31
+ signal?.throwIfAborted();
32
+ const memoryIds = new Set();
33
+ for (const item of groups.flat()) {
34
+ signal?.throwIfAborted();
35
+ const { id } = item;
36
+ if (!id || /\s/.test(id) || memoryIds.has(id)) {
37
+ throw new Error('Mem0 dedup received an invalid or duplicate memory ID.');
38
+ }
39
+ memoryIds.add(id);
40
+ if (!item.memory.trim()) {
41
+ throw new Error('Mem0 dedup received empty memory content.');
42
+ }
43
+ }
44
+ const total = groups.reduce((count, group) => count + group.length, 0);
45
+ const duplicateIds = new Set(groups.flatMap((group) => findDuplicateIds(group, signal)));
46
+ const preview = {
47
+ total,
48
+ duplicatesFound: duplicateIds.size,
49
+ duplicatesRemoved: 0,
50
+ deleteFailures: 0,
51
+ };
52
+ if (dryRun)
53
+ return preview;
54
+ if (approve) {
55
+ const approved = await approve(preview);
56
+ signal?.throwIfAborted();
57
+ if (!approved)
58
+ return preview;
20
59
  }
21
- const duplicateIds = findDuplicateIds(allMemories, signal);
22
60
  let removed = 0;
61
+ let failures = 0;
23
62
  for (const id of duplicateIds) {
24
- if (signal?.aborted)
25
- break;
63
+ signal?.throwIfAborted();
26
64
  try {
27
- await provider.delete(id);
65
+ await (signal ? provider.delete(id, { signal }) : provider.delete(id));
66
+ signal?.throwIfAborted();
28
67
  removed++;
29
68
  }
30
69
  catch {
31
- // best-effort
70
+ signal?.throwIfAborted();
71
+ failures++;
32
72
  }
33
73
  }
34
- return { total: allMemories.length, duplicatesRemoved: removed };
74
+ return {
75
+ total,
76
+ duplicatesFound: duplicateIds.size,
77
+ duplicatesRemoved: removed,
78
+ deleteFailures: failures,
79
+ };
35
80
  }
36
81
  /**
37
82
  * Shared: find IDs of duplicate entries to remove (keeps the newest).
38
83
  */
39
84
  function findDuplicateIds(allMemories, signal) {
40
- const seen = new Map();
41
- const duplicateIds = [];
85
+ const groups = new Map();
42
86
  for (const item of allMemories) {
43
- if (signal?.aborted)
44
- break;
87
+ signal?.throwIfAborted();
45
88
  const normalized = item.memory.normalize('NFC').replace(/\s+/g, ' ').trim().toLowerCase();
46
- const existing = seen.get(normalized);
47
- if (existing) {
48
- const existingTime = existing.updated_at ? new Date(existing.updated_at).getTime() : 0;
49
- const currentTime = item.updated_at ? new Date(item.updated_at).getTime() : 0;
50
- if (currentTime > existingTime) {
51
- duplicateIds.push(existing.id);
52
- seen.set(normalized, item);
89
+ const group = groups.get(normalized);
90
+ if (group)
91
+ group.push(item);
92
+ else
93
+ groups.set(normalized, [item]);
94
+ }
95
+ const duplicateIds = [];
96
+ for (const group of groups.values()) {
97
+ if (group.length < 2)
98
+ continue;
99
+ const [first, ...rest] = group;
100
+ if (!first)
101
+ continue;
102
+ let newest = first;
103
+ let newestTime = memoryTime(first);
104
+ const times = new Set([newestTime]);
105
+ for (const item of rest) {
106
+ signal?.throwIfAborted();
107
+ const time = memoryTime(item);
108
+ if (times.has(time)) {
109
+ throw new Error('Mem0 dedup cannot determine the newest duplicate.');
53
110
  }
54
- else {
55
- duplicateIds.push(item.id);
111
+ times.add(time);
112
+ if (time > newestTime) {
113
+ newest = item;
114
+ newestTime = time;
56
115
  }
57
116
  }
58
- else {
59
- seen.set(normalized, item);
60
- }
117
+ duplicateIds.push(...group.filter((item) => item !== newest).map((item) => item.id));
61
118
  }
62
119
  return duplicateIds;
63
120
  }
121
+ function memoryTime(item) {
122
+ const timestamp = item.updated_at ?? item.created_at;
123
+ const time = timestamp ? new Date(timestamp).getTime() : Number.NaN;
124
+ if (!Number.isFinite(time)) {
125
+ throw new Error('Mem0 dedup cannot determine the newest duplicate.');
126
+ }
127
+ return time;
128
+ }
64
129
  //# sourceMappingURL=dedup.js.map
package/dist/dedup.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"dedup.js","sourceRoot":"","sources":["../src/dedup.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AACH,OAAO,EAAE,kBAAkB,EAAyB,MAAM,eAAe,CAAC;AAe1E,MAAM,CAAC,KAAK,UAAU,aAAa,CAAC,IAAkB;IACpD,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,eAAe,EAAE,MAAM,EAAE,GAAG,IAAI,CAAC;IAEzD,MAAM,QAAQ,GAAG,MAAM,kBAAkB,CAAC;QACxC,MAAM;QACN,GAAG,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,eAAe,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAChD,CAAC,CAAC;IACH,MAAM,WAAW,GAAG,MAAM,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;IAEtD,IAAI,WAAW,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC7B,OAAO,EAAE,KAAK,EAAE,CAAC,EAAE,iBAAiB,EAAE,CAAC,EAAE,CAAC;IAC5C,CAAC;IAED,MAAM,YAAY,GAAG,gBAAgB,CAAC,WAAW,EAAE,MAAM,CAAC,CAAC;IAE3D,IAAI,OAAO,GAAG,CAAC,CAAC;IAChB,KAAK,MAAM,EAAE,IAAI,YAAY,EAAE,CAAC;QAC9B,IAAI,MAAM,EAAE,OAAO;YAAE,MAAM;QAC3B,IAAI,CAAC;YACH,MAAM,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;YAC1B,OAAO,EAAE,CAAC;QACZ,CAAC;QAAC,MAAM,CAAC;YACP,cAAc;QAChB,CAAC;IACH,CAAC;IAED,OAAO,EAAE,KAAK,EAAE,WAAW,CAAC,MAAM,EAAE,iBAAiB,EAAE,OAAO,EAAE,CAAC;AACnE,CAAC;AAED;;GAEG;AACH,SAAS,gBAAgB,CAAC,WAAyB,EAAE,MAAoB;IACvE,MAAM,IAAI,GAAG,IAAI,GAAG,EAAsB,CAAC;IAC3C,MAAM,YAAY,GAAa,EAAE,CAAC;IAElC,KAAK,MAAM,IAAI,IAAI,WAAW,EAAE,CAAC;QAC/B,IAAI,MAAM,EAAE,OAAO;YAAE,MAAM;QAE3B,MAAM,UAAU,GAAG,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;QAC1F,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,CAAC;QAEtC,IAAI,QAAQ,EAAE,CAAC;YACb,MAAM,YAAY,GAAG,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YACvF,MAAM,WAAW,GAAG,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YAC9E,IAAI,WAAW,GAAG,YAAY,EAAE,CAAC;gBAC/B,YAAY,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;gBAC/B,IAAI,CAAC,GAAG,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;YAC7B,CAAC;iBAAM,CAAC;gBACN,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YAC7B,CAAC;QACH,CAAC;aAAM,CAAC;YACN,IAAI,CAAC,GAAG,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;QAC7B,CAAC;IACH,CAAC;IAED,OAAO,YAAY,CAAC;AACtB,CAAC"}
1
+ {"version":3,"file":"dedup.js","sourceRoot":"","sources":["../src/dedup.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AACH,OAAO,EAAE,kBAAkB,EAA4C,MAAM,eAAe,CAAC;AAoB7F,MAAM,CAAC,KAAK,UAAU,aAAa,CAAC,IAAkB;IACpD,MAAM,EAAE,MAAM,EAAE,eAAe,EAAE,GAAG,IAAI,CAAC;IAEzC,MAAM,QAAQ,GAAG,MAAM,kBAAkB,CAAC;QACxC,MAAM;QACN,GAAG,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,eAAe,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAChD,CAAC,CAAC;IACH,OAAO,qBAAqB,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC;AAC/C,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,qBAAqB,CACzC,QAAsB,EACtB,IAAgF;IAEhF,MAAM,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,GAAG,KAAK,EAAE,OAAO,EAAE,GAAG,IAAI,CAAC;IAClE,MAAM,KAAK,GAAG;QACZ,MAAM;QACN,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAC/B,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAC9B,CAAC;IACF,IAAI,CAAC,QAAQ,CAAC,cAAc,EAAE,CAAC;QAC7B,MAAM,IAAI,KAAK,CAAC,mEAAmE,CAAC,CAAC;IACvF,CAAC;IACD,MAAM,EAAE,cAAc,EAAE,CAAC;IACzB,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,cAAc,CAAC,KAAK,CAAC,CAAC;IACpD,MAAM,EAAE,cAAc,EAAE,CAAC;IACzB,MAAM,SAAS,GAAG,IAAI,GAAG,EAAU,CAAC;IACpC,KAAK,MAAM,IAAI,IAAI,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;QACjC,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,MAAM,EAAE,EAAE,EAAE,GAAG,IAAI,CAAC;QACpB,IAAI,CAAC,EAAE,IAAI,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE,CAAC;YAC9C,MAAM,IAAI,KAAK,CAAC,wDAAwD,CAAC,CAAC;QAC5E,CAAC;QACD,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QAClB,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;YACxB,MAAM,IAAI,KAAK,CAAC,2CAA2C,CAAC,CAAC;QAC/D,CAAC;IACH,CAAC;IACD,MAAM,KAAK,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAEvE,MAAM,YAAY,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,gBAAgB,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC;IACzF,MAAM,OAAO,GAAG;QACd,KAAK;QACL,eAAe,EAAE,YAAY,CAAC,IAAI;QAClC,iBAAiB,EAAE,CAAC;QACpB,cAAc,EAAE,CAAC;KAClB,CAAC;IACF,IAAI,MAAM;QAAE,OAAO,OAAO,CAAC;IAC3B,IAAI,OAAO,EAAE,CAAC;QACZ,MAAM,QAAQ,GAAG,MAAM,OAAO,CAAC,OAAO,CAAC,CAAC;QACxC,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,IAAI,CAAC,QAAQ;YAAE,OAAO,OAAO,CAAC;IAChC,CAAC;IAED,IAAI,OAAO,GAAG,CAAC,CAAC;IAChB,IAAI,QAAQ,GAAG,CAAC,CAAC;IACjB,KAAK,MAAM,EAAE,IAAI,YAAY,EAAE,CAAC;QAC9B,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,IAAI,CAAC;YACH,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,EAAE,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;YACvE,MAAM,EAAE,cAAc,EAAE,CAAC;YACzB,OAAO,EAAE,CAAC;QACZ,CAAC;QAAC,MAAM,CAAC;YACP,MAAM,EAAE,cAAc,EAAE,CAAC;YACzB,QAAQ,EAAE,CAAC;QACb,CAAC;IACH,CAAC;IAED,OAAO;QACL,KAAK;QACL,eAAe,EAAE,YAAY,CAAC,IAAI;QAClC,iBAAiB,EAAE,OAAO;QAC1B,cAAc,EAAE,QAAQ;KACzB,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,SAAS,gBAAgB,CAAC,WAAyB,EAAE,MAAoB;IACvE,MAAM,MAAM,GAAG,IAAI,GAAG,EAAwB,CAAC;IAE/C,KAAK,MAAM,IAAI,IAAI,WAAW,EAAE,CAAC;QAC/B,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,MAAM,UAAU,GAAG,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;QAC1F,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,UAAU,CAAC,CAAC;QACrC,IAAI,KAAK;YAAE,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;;YACvB,MAAM,CAAC,GAAG,CAAC,UAAU,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC;IACtC,CAAC;IAED,MAAM,YAAY,GAAa,EAAE,CAAC;IAClC,KAAK,MAAM,KAAK,IAAI,MAAM,CAAC,MAAM,EAAE,EAAE,CAAC;QACpC,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC;YAAE,SAAS;QAC/B,MAAM,CAAC,KAAK,EAAE,GAAG,IAAI,CAAC,GAAG,KAAK,CAAC;QAC/B,IAAI,CAAC,KAAK;YAAE,SAAS;QACrB,IAAI,MAAM,GAAG,KAAK,CAAC;QACnB,IAAI,UAAU,GAAG,UAAU,CAAC,KAAK,CAAC,CAAC;QACnC,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC;QACpC,KAAK,MAAM,IAAI,IAAI,IAAI,EAAE,CAAC;YACxB,MAAM,EAAE,cAAc,EAAE,CAAC;YACzB,MAAM,IAAI,GAAG,UAAU,CAAC,IAAI,CAAC,CAAC;YAC9B,IAAI,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC;gBACpB,MAAM,IAAI,KAAK,CAAC,mDAAmD,CAAC,CAAC;YACvE,CAAC;YACD,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;YAChB,IAAI,IAAI,GAAG,UAAU,EAAE,CAAC;gBACtB,MAAM,GAAG,IAAI,CAAC;gBACd,UAAU,GAAG,IAAI,CAAC;YACpB,CAAC;QACH,CAAC;QACD,YAAY,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;IACvF,CAAC;IAED,OAAO,YAAY,CAAC;AACtB,CAAC;AAED,SAAS,UAAU,CAAC,IAAgB;IAClC,MAAM,SAAS,GAAG,IAAI,CAAC,UAAU,IAAI,IAAI,CAAC,UAAU,CAAC;IACrD,MAAM,IAAI,GAAG,SAAS,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC;IACpE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;QAC3B,MAAM,IAAI,KAAK,CAAC,mDAAmD,CAAC,CAAC;IACvE,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC"}
package/dist/index.d.ts CHANGED
@@ -1,9 +1,23 @@
1
1
  /**
2
- * pi-memory-mem0 — Explicit semantic memory extension powered by Mem0.
2
+ * pi-memory-mem0 — Semantic memory extension powered by Mem0.
3
3
  *
4
- * Two modes:
4
+ * Backend modes:
5
5
  * - **platform**: Uses Mem0 Cloud API (needs MEM0_API_KEY)
6
- * - **open-source**: Runs locally with SQLite vector store (needs OPENAI_API_KEY or Ollama)
6
+ * - **embedded**: Runs Mem0 OSS in-process
7
+ * - **self-hosted**: Calls a remote Mem0 OSS REST server
8
+ *
9
+ * Memory modes (config "memoryMode"):
10
+ * - **passive**: automatic capture + recall injection only
11
+ * - **active**: LLM-callable mem0_memory tool only
12
+ * - **hybrid** (default): both
13
+ *
14
+ * Passive side: after each conversation turn, user + assistant messages are
15
+ * sent to Mem0 for fact extraction and storage (credentials are redacted
16
+ * first). Recalled memories are injected as a custom message (delivered to
17
+ * the model on the user channel, never the system prompt) and wrapped as
18
+ * untrusted data. Active side: the mem0_memory tool lets the agent search,
19
+ * add, list, and delete memories on its own initiative, under the same
20
+ * redaction and untrusted-data boundaries.
7
21
  *
8
22
  * Configuration via settings.json key "pi-memory-mem0".
9
23
  * Supports ${ENV_VAR:-fallback} in user and agent settings.
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AAGH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,iCAAiC,CAAC;AA2BpE,MAAM,CAAC,OAAO,UAAU,aAAa,CAAC,EAAE,EAAE,YAAY,GAAG,IAAI,CA6G5D"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;GAwBG;AAGH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,iCAAiC,CAAC;AAmCpE,MAAM,CAAC,OAAO,UAAU,aAAa,CAAC,EAAE,EAAE,YAAY,GAAG,IAAI,CAgU5D"}