memo-grafter 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (136) hide show
  1. package/.env.example +5 -3
  2. package/README.md +187 -182
  3. package/USER_GUIDE.md +932 -702
  4. package/dist/MemoGrafter.d.ts +1 -1
  5. package/dist/MemoGrafter.d.ts.map +1 -1
  6. package/dist/MemoGrafter.js +16 -5
  7. package/dist/MemoGrafter.js.map +1 -1
  8. package/dist/MemoGrafterAgent.d.ts +7 -1
  9. package/dist/MemoGrafterAgent.d.ts.map +1 -1
  10. package/dist/MemoGrafterAgent.js +45 -8
  11. package/dist/MemoGrafterAgent.js.map +1 -1
  12. package/dist/adapters/AnthropicAdapter.d.ts +9 -0
  13. package/dist/adapters/AnthropicAdapter.d.ts.map +1 -0
  14. package/dist/adapters/AnthropicAdapter.js +33 -0
  15. package/dist/adapters/AnthropicAdapter.js.map +1 -0
  16. package/dist/adapters/GeminiAdapter.d.ts +15 -0
  17. package/dist/adapters/GeminiAdapter.d.ts.map +1 -0
  18. package/dist/adapters/GeminiAdapter.js +48 -0
  19. package/dist/adapters/GeminiAdapter.js.map +1 -0
  20. package/dist/fleet/FleetStore.d.ts +1 -1
  21. package/dist/fleet/FleetStore.d.ts.map +1 -1
  22. package/dist/index.d.ts +5 -1
  23. package/dist/index.d.ts.map +1 -1
  24. package/dist/index.js +3 -0
  25. package/dist/index.js.map +1 -1
  26. package/dist/pipeline/GrafterPipeline.d.ts +1 -3
  27. package/dist/pipeline/GrafterPipeline.d.ts.map +1 -1
  28. package/dist/pipeline/GrafterPipeline.js +6 -19
  29. package/dist/pipeline/GrafterPipeline.js.map +1 -1
  30. package/dist/pipeline/IngestPipeline.d.ts +7 -3
  31. package/dist/pipeline/IngestPipeline.d.ts.map +1 -1
  32. package/dist/pipeline/IngestPipeline.js +31 -5
  33. package/dist/pipeline/IngestPipeline.js.map +1 -1
  34. package/dist/pipeline/RetrieverPipeline.d.ts +11 -0
  35. package/dist/pipeline/RetrieverPipeline.d.ts.map +1 -0
  36. package/dist/pipeline/RetrieverPipeline.js +73 -0
  37. package/dist/pipeline/RetrieverPipeline.js.map +1 -0
  38. package/dist/pipeline/SegmentProcessor.d.ts +3 -2
  39. package/dist/pipeline/SegmentProcessor.d.ts.map +1 -1
  40. package/dist/pipeline/SegmentProcessor.js +52 -40
  41. package/dist/pipeline/SegmentProcessor.js.map +1 -1
  42. package/dist/pipeline/TopicDriftDetector.d.ts +21 -3
  43. package/dist/pipeline/TopicDriftDetector.d.ts.map +1 -1
  44. package/dist/pipeline/TopicDriftDetector.js +143 -17
  45. package/dist/pipeline/TopicDriftDetector.js.map +1 -1
  46. package/dist/prompts/factRetrievalPrompt.d.ts +4 -0
  47. package/dist/prompts/factRetrievalPrompt.d.ts.map +1 -0
  48. package/dist/prompts/factRetrievalPrompt.js +25 -0
  49. package/dist/prompts/factRetrievalPrompt.js.map +1 -0
  50. package/dist/prompts/historyCompressionPrompt.d.ts +3 -0
  51. package/dist/prompts/historyCompressionPrompt.d.ts.map +1 -0
  52. package/dist/prompts/historyCompressionPrompt.js +4 -0
  53. package/dist/prompts/historyCompressionPrompt.js.map +1 -0
  54. package/dist/prompts/intentShiftPrompt.d.ts +3 -0
  55. package/dist/prompts/intentShiftPrompt.d.ts.map +1 -0
  56. package/dist/prompts/intentShiftPrompt.js +11 -0
  57. package/dist/prompts/intentShiftPrompt.js.map +1 -0
  58. package/dist/prompts/memoryInjectionPrompt.d.ts +4 -0
  59. package/dist/prompts/memoryInjectionPrompt.d.ts.map +1 -0
  60. package/dist/prompts/memoryInjectionPrompt.js +16 -0
  61. package/dist/prompts/memoryInjectionPrompt.js.map +1 -0
  62. package/dist/prompts/segmentExtractionPrompt.d.ts +3 -0
  63. package/dist/prompts/segmentExtractionPrompt.d.ts.map +1 -0
  64. package/dist/prompts/segmentExtractionPrompt.js +81 -0
  65. package/dist/prompts/segmentExtractionPrompt.js.map +1 -0
  66. package/dist/store/GraphStore.d.ts +16 -19
  67. package/dist/store/GraphStore.d.ts.map +1 -1
  68. package/dist/store/GraphStore.js +1 -576
  69. package/dist/store/GraphStore.js.map +1 -1
  70. package/dist/store/index.d.ts +3 -0
  71. package/dist/store/index.d.ts.map +1 -0
  72. package/dist/store/index.js +2 -0
  73. package/dist/store/index.js.map +1 -0
  74. package/dist/store/postgres-pgvector/GraphStore.d.ts +75 -0
  75. package/dist/store/postgres-pgvector/GraphStore.d.ts.map +1 -0
  76. package/dist/store/postgres-pgvector/GraphStore.js +782 -0
  77. package/dist/store/postgres-pgvector/GraphStore.js.map +1 -0
  78. package/dist/types.d.ts +69 -1
  79. package/dist/types.d.ts.map +1 -1
  80. package/dist/utils/drift/driftMarkers.d.ts +4 -0
  81. package/dist/utils/drift/driftMarkers.d.ts.map +1 -0
  82. package/dist/utils/drift/driftMarkers.js +31 -0
  83. package/dist/utils/drift/driftMarkers.js.map +1 -0
  84. package/dist/utils/drift/driftScore.d.ts +3 -0
  85. package/dist/utils/drift/driftScore.d.ts.map +1 -0
  86. package/dist/utils/drift/driftScore.js +11 -0
  87. package/dist/utils/drift/driftScore.js.map +1 -0
  88. package/dist/utils/drift/driftThreshold.d.ts +7 -0
  89. package/dist/utils/drift/driftThreshold.d.ts.map +1 -0
  90. package/dist/utils/drift/driftThreshold.js +21 -0
  91. package/dist/utils/drift/driftThreshold.js.map +1 -0
  92. package/dist/utils/drift/reentryMatch.d.ts +3 -0
  93. package/dist/utils/drift/reentryMatch.d.ts.map +1 -0
  94. package/dist/utils/drift/reentryMatch.js +10 -0
  95. package/dist/utils/drift/reentryMatch.js.map +1 -0
  96. package/dist/utils/extraction/segmentExtraction.d.ts +5 -0
  97. package/dist/utils/extraction/segmentExtraction.d.ts.map +1 -0
  98. package/dist/utils/extraction/segmentExtraction.js +82 -0
  99. package/dist/utils/extraction/segmentExtraction.js.map +1 -0
  100. package/dist/utils/reentry/reentryCues.d.ts +3 -0
  101. package/dist/utils/reentry/reentryCues.d.ts.map +1 -0
  102. package/dist/utils/reentry/reentryCues.js +14 -0
  103. package/dist/utils/reentry/reentryCues.js.map +1 -0
  104. package/dist/utils/reentry/reentryEdges.d.ts +16 -0
  105. package/dist/utils/reentry/reentryEdges.d.ts.map +1 -0
  106. package/dist/utils/reentry/reentryEdges.js +80 -0
  107. package/dist/utils/reentry/reentryEdges.js.map +1 -0
  108. package/dist/utils/reentry/reentrySimilarity.d.ts +5 -0
  109. package/dist/utils/reentry/reentrySimilarity.d.ts.map +1 -0
  110. package/dist/utils/reentry/reentrySimilarity.js +26 -0
  111. package/dist/utils/reentry/reentrySimilarity.js.map +1 -0
  112. package/dist/utils/reentry/reentryText.d.ts +6 -0
  113. package/dist/utils/reentry/reentryText.d.ts.map +1 -0
  114. package/dist/utils/reentry/reentryText.js +29 -0
  115. package/dist/utils/reentry/reentryText.js.map +1 -0
  116. package/dist/utils/reentry/types.d.ts +6 -0
  117. package/dist/utils/reentry/types.d.ts.map +1 -0
  118. package/dist/utils/reentry/types.js +2 -0
  119. package/dist/utils/reentry/types.js.map +1 -0
  120. package/dist/utils/text/normalizeText.d.ts +2 -0
  121. package/dist/utils/text/normalizeText.d.ts.map +1 -0
  122. package/dist/utils/text/normalizeText.js +5 -0
  123. package/dist/utils/text/normalizeText.js.map +1 -0
  124. package/dist/utils/text/terms.d.ts +3 -0
  125. package/dist/utils/text/terms.d.ts.map +1 -0
  126. package/dist/utils/text/terms.js +51 -0
  127. package/dist/utils/text/terms.js.map +1 -0
  128. package/dist/utils/text/tokenCount.d.ts +2 -0
  129. package/dist/utils/text/tokenCount.d.ts.map +1 -0
  130. package/dist/utils/text/tokenCount.js +4 -0
  131. package/dist/utils/text/tokenCount.js.map +1 -0
  132. package/dist/utils/vector/vectorLiteral.d.ts +3 -0
  133. package/dist/utils/vector/vectorLiteral.d.ts.map +1 -0
  134. package/dist/utils/vector/vectorLiteral.js +19 -0
  135. package/dist/utils/vector/vectorLiteral.js.map +1 -0
  136. package/package.json +84 -55
package/USER_GUIDE.md CHANGED
@@ -1,702 +1,932 @@
1
- # MemoGrafter User Guide
2
-
3
- ## Introduction
4
-
5
- MemoGrafter is an experimental Node.js and TypeScript framework for structured chatbot memory. It stores chatbot conversations as message buffers, topic segments, topic nodes, and graph edges. Later, it can inject relevant memory into a chatbot turn or copy selected memory into another chatbot/session.
6
-
7
- The project is intentionally focused. MemoGrafter is a chatbot memory framework, not an autonomous agent runtime. It does not run tools, schedule work, or decide goals for an agent. It helps a chatbot remember, retrieve, and transfer conversational context.
8
-
9
- The most important idea is memory grafting. A chatbot can build useful memory during one conversation, and another chatbot can absorb only the relevant parts.
10
-
11
- ## Requirements
12
-
13
- - Node.js 18 or newer.
14
- - TypeScript or modern JavaScript using ES modules.
15
- - PostgreSQL with the `pgvector` extension enabled.
16
- - An LLM adapter.
17
- - An embedding adapter.
18
- - An OpenAI API key only if using the included OpenAI adapters.
19
- - Redis only if enabling queue mode.
20
-
21
- MemoGrafter is server-side only. Do not run it in browser code.
22
-
23
- ## Installation
24
-
25
- Install from npm:
26
-
27
- ```bash
28
- npm install memo-grafter
29
- ```
30
-
31
- Install from a local clone before publishing or while developing:
32
-
33
- ```bash
34
- cd path/to/your-app
35
- npm install D:/cohort/projects/project-memoGrafter
36
- ```
37
-
38
- If you are working inside this repository, build it first:
39
-
40
- ```bash
41
- cd D:/cohort/projects/project-memoGrafter
42
- npm install
43
- npm run build
44
- ```
45
-
46
- ## Environment Setup
47
-
48
- Create a `.env` file in your app:
49
-
50
- ```bash
51
- DATABASE_URL=postgres://postgres:postgres@localhost:5432/memo_grafter
52
- OPENAI_API_KEY=sk-...
53
- REDIS_URL=redis://localhost:6379
54
- ```
55
-
56
- `DATABASE_URL` is required.
57
-
58
- `OPENAI_API_KEY` is required only when using `OpenAILLMAdapter` or `OpenAIEmbedAdapter`.
59
-
60
- `REDIS_URL` is optional and only needed when you pass `queue` config.
61
-
62
- Enable `pgvector` in PostgreSQL:
63
-
64
- ```sql
65
- CREATE EXTENSION IF NOT EXISTS vector;
66
- ```
67
-
68
- MemoGrafter creates its own tables during `initialize()`.
69
-
70
- Current v1 tables:
71
-
72
- - `mg_message_buffer`
73
- - `mg_segments`
74
- - `mg_topic_nodes`
75
- - `mg_topic_edges`
76
- - `mg_fleets`
77
- - `mg_fleet_agents`
78
-
79
- ## Quick Start
80
-
81
- Create `src/index.ts`:
82
-
83
- ```ts
84
- import "dotenv/config";
85
-
86
- import {
87
- MemoGrafterAgent,
88
- OpenAIEmbedAdapter,
89
- OpenAILLMAdapter,
90
- } from "memo-grafter";
91
-
92
- const agent = new MemoGrafterAgent({
93
- db: {
94
- connectionString: process.env.DATABASE_URL!,
95
- },
96
- llm: new OpenAILLMAdapter("gpt-4o"),
97
- embedder: new OpenAIEmbedAdapter("text-embedding-3-small"),
98
- });
99
-
100
- try {
101
- await agent.initialize();
102
-
103
- console.log(await agent.invoke("I am planning a Japan trip."));
104
- console.log(await agent.invoke("I like quiet towns and local cafes."));
105
- console.log(await agent.invoke("What should I remember while planning?"));
106
-
107
- const nodes = await agent.getActiveNodes();
108
- console.log(nodes.map((node) => ({ label: node.label, summary: node.summary })));
109
- } finally {
110
- await agent.close();
111
- }
112
- ```
113
-
114
- Run it:
115
-
116
- ```bash
117
- npx tsx --env-file=.env src/index.ts
118
- ```
119
-
120
- ## Core Concepts
121
-
122
- ### Messages
123
-
124
- A message is one user or assistant turn:
125
-
126
- ```ts
127
- export interface Message {
128
- role: "user" | "assistant";
129
- content: string;
130
- }
131
- ```
132
-
133
- `MemoGrafterAgent` keeps an in-memory history for the current session and stores messages in PostgreSQL during ingestion.
134
-
135
- ### Segments
136
-
137
- A segment is a range of messages that belong to the same topic. MemoGrafter uses drift detection to decide where topic boundaries are.
138
-
139
- Example:
140
-
141
- ```text
142
- messages 0-4 -> Japan travel planning
143
- messages 5-8 -> cover letter writing
144
- ```
145
-
146
- Segments are stored in `mg_segments`.
147
-
148
- ### Topic Nodes
149
-
150
- A topic node is the main unit of memory. It represents a segment as a label, summary, embedding, message range, and metadata.
151
-
152
- Important fields:
153
-
154
- - `id`: unique topic node ID.
155
- - `label`: short label.
156
- - `summary`: structured summary of the segment.
157
- - `embedding`: vector used for semantic search.
158
- - `messageRange`: source message range.
159
- - `topicOrder`: chronological order.
160
- - `driftScore`: topic-change score.
161
- - `agentColor`, `fleetId`, `agentId`: nullable fleet metadata.
162
-
163
- Topic nodes are stored in `mg_topic_nodes`.
164
-
165
- ### Graph Edges
166
-
167
- Edges connect related topic nodes. They can represent temporal, semantic, or grafted relationships.
168
-
169
- Edges are stored in `mg_topic_edges`.
170
-
171
- ### Grafting
172
-
173
- Grafting is the process of selecting memory nodes and turning them into useful context for another prompt or another chatbot.
174
-
175
- There are two common forms:
176
-
177
- - Preview memory with `graft()`.
178
- - Copy memory into another chatbot with `absorbFromAgent()` or `ingestGraftedNodes()`.
179
-
180
- ## Using MemoGrafterAgent
181
-
182
- `MemoGrafterAgent` is the easiest API to start with.
183
-
184
- ```ts
185
- const agent = new MemoGrafterAgent({
186
- db: {
187
- connectionString: process.env.DATABASE_URL!,
188
- },
189
- llm: new OpenAILLMAdapter("gpt-4o"),
190
- embedder: new OpenAIEmbedAdapter("text-embedding-3-small"),
191
- });
192
- ```
193
-
194
- Initialize it before use:
195
-
196
- ```ts
197
- await agent.initialize();
198
- ```
199
-
200
- Send user messages with `invoke()`:
201
-
202
- ```ts
203
- const answer = await agent.invoke("Help me plan a Kyoto itinerary.");
204
- ```
205
-
206
- Close resources when done:
207
-
208
- ```ts
209
- await agent.close();
210
- ```
211
-
212
- ### What `invoke()` Does
213
-
214
- On every call, `invoke()`:
215
-
216
- 1. Adds the user message to local history.
217
- 2. Loads existing topic nodes for the session.
218
- 3. Builds a memory injection prompt from those nodes.
219
- 4. Sends history plus memory prompt to the LLM.
220
- 5. Adds the assistant response to history.
221
- 6. Ingests the updated conversation into the memory graph.
222
-
223
- On the first turn there may be no memory to inject. Later turns can use memory created from earlier turns.
224
-
225
- ## Inspecting Memory
226
-
227
- Read active topic nodes:
228
-
229
- ```ts
230
- const nodes = await agent.getActiveNodes();
231
-
232
- for (const node of nodes) {
233
- console.log({
234
- id: node.id,
235
- label: node.label,
236
- summary: node.summary,
237
- messageRange: node.messageRange,
238
- topicOrder: node.topicOrder,
239
- driftScore: node.driftScore,
240
- });
241
- }
242
- ```
243
-
244
- Read active segments:
245
-
246
- ```ts
247
- const segments = await agent.getActiveSegments();
248
- console.log(segments);
249
- ```
250
-
251
- Read the in-memory chat history:
252
-
253
- ```ts
254
- const history = agent.getHistory();
255
- console.log(history);
256
- ```
257
-
258
- Read the session ID:
259
-
260
- ```ts
261
- console.log(agent.getSessionId());
262
- ```
263
-
264
- ## Grafting Memory
265
-
266
- Use `graft()` to preview what memory would be injected:
267
-
268
- ```ts
269
- const graft = await agent.graft();
270
-
271
- console.log(graft.systemPrompt);
272
- console.log(graft.nodes);
273
- console.log(graft.tokenCount);
274
- ```
275
-
276
- You can graft specific topic IDs:
277
-
278
- ```ts
279
- const nodes = await agent.getActiveNodes();
280
-
281
- const graft = await agent.graft([nodes[0]!.id]);
282
- ```
283
-
284
- `graft()` returns:
285
-
286
- - `systemPrompt`: memory context suitable for an LLM system prompt.
287
- - `nodes`: selected topic nodes.
288
- - `tokenCount`: estimated token count.
289
-
290
- ## Absorbing Memory Into Another Chatbot
291
-
292
- Use `absorbFromAgent()` to copy selected memory from one chatbot into another.
293
-
294
- ```ts
295
- const travelBot = new MemoGrafterAgent(config);
296
- const writingBot = new MemoGrafterAgent(config);
297
-
298
- await travelBot.initialize();
299
- await writingBot.initialize();
300
-
301
- await travelBot.invoke("I am planning a Japan trip.");
302
- await travelBot.invoke("I like quiet towns, bookstores, and local cafes.");
303
- await travelBot.invoke("My budget is around 2500 dollars.");
304
-
305
- const copiedNodes = await writingBot.absorbFromAgent(travelBot, {
306
- prompt: "Japan travel preferences",
307
- minSimilarity: 0.6,
308
- limit: 3,
309
- });
310
-
311
- console.log(`Copied ${copiedNodes.length} nodes.`);
312
-
313
- const response = await writingBot.invoke(
314
- "Suggest a reflective blog intro for my Japan trip."
315
- );
316
-
317
- console.log(response);
318
- ```
319
-
320
- ### Absorb By Semantic Prompt
321
-
322
- ```ts
323
- await targetAgent.absorbFromAgent(sourceAgent, {
324
- prompt: "Japan travel preferences",
325
- minSimilarity: 0.6,
326
- limit: 3,
327
- });
328
- ```
329
-
330
- Use this when you want MemoGrafter to find relevant memory by meaning.
331
-
332
- ### Absorb By Topic ID
333
-
334
- ```ts
335
- const sourceNodes = await sourceAgent.getActiveNodes();
336
-
337
- await targetAgent.absorbFromAgent(sourceAgent, {
338
- topicIds: [sourceNodes[0]!.id],
339
- });
340
- ```
341
-
342
- Use this when your UI lets a user choose memory nodes manually.
343
-
344
- ### Ingest Grafted Nodes Directly
345
-
346
- ```ts
347
- const graft = await sourceAgent.graft();
348
- await targetAgent.ingestGraftedNodes(graft.nodes);
349
- ```
350
-
351
- Use this when you want to inspect or filter a graft before copying it.
352
-
353
- ## Configuration
354
-
355
- Full shape:
356
-
357
- ```ts
358
- const agent = new MemoGrafterAgent({
359
- db: {
360
- connectionString: process.env.DATABASE_URL!,
361
- },
362
- llm,
363
- embedder,
364
- drift: {
365
- mode: "intent",
366
- windowSize: 5,
367
- threshold: 0.3,
368
- minSegmentMessages: 3,
369
- },
370
- graph: {
371
- topK: 5,
372
- hopDepth: 2,
373
- },
374
- inject: {
375
- bufferSize: 4,
376
- tokenBudget: 1500,
377
- },
378
- });
379
- ```
380
-
381
- ### `db`
382
-
383
- ```ts
384
- db: {
385
- connectionString: process.env.DATABASE_URL!,
386
- }
387
- ```
388
-
389
- PostgreSQL connection string.
390
-
391
- ### `llm`
392
-
393
- ```ts
394
- llm: new OpenAILLMAdapter("gpt-4o")
395
- ```
396
-
397
- Adapter used to generate assistant responses and summarize segments.
398
-
399
- ### `embedder`
400
-
401
- ```ts
402
- embedder: new OpenAIEmbedAdapter("text-embedding-3-small")
403
- ```
404
-
405
- Adapter used to create vectors for semantic search.
406
-
407
- ### `drift`
408
-
409
- ```ts
410
- drift: {
411
- mode: "intent",
412
- windowSize: 5,
413
- threshold: 0.3,
414
- minSegmentMessages: 3,
415
- }
416
- ```
417
-
418
- Controls topic boundary detection.
419
-
420
- - `mode`: `"intent"` or `"window"`.
421
- - `windowSize`: message window size for window mode.
422
- - `threshold`: drift sensitivity.
423
- - `minSegmentMessages`: minimum messages before a boundary.
424
-
425
- Use `"intent"` for most chatbot memory demos. In intent mode, user messages drive topic shifts.
426
-
427
- ### `graph`
428
-
429
- ```ts
430
- graph: {
431
- topK: 5,
432
- hopDepth: 2,
433
- }
434
- ```
435
-
436
- Controls graph retrieval and traversal.
437
-
438
- - `topK`: number of similar nodes to retrieve.
439
- - `hopDepth`: how far grafting walks graph neighbors.
440
-
441
- ### `inject`
442
-
443
- ```ts
444
- inject: {
445
- bufferSize: 4,
446
- tokenBudget: 1500,
447
- }
448
- ```
449
-
450
- Controls how much memory is inserted into the prompt.
451
-
452
- - `bufferSize`: nearby raw messages to include.
453
- - `tokenBudget`: approximate memory prompt budget.
454
-
455
- ## Queue Mode
456
-
457
- Without queue config, ingestion runs synchronously after `invoke()`.
458
-
459
- With queue config, MemoGrafter uses BullMQ and Redis:
460
-
461
- ```ts
462
- const agent = new MemoGrafterAgent({
463
- db: {
464
- connectionString: process.env.DATABASE_URL!,
465
- },
466
- llm: new OpenAILLMAdapter("gpt-4o"),
467
- embedder: new OpenAIEmbedAdapter("text-embedding-3-small"),
468
- queue: {
469
- redisUrl: process.env.REDIS_URL!,
470
- removeOnComplete: true,
471
- removeOnFail: true,
472
- },
473
- });
474
- ```
475
-
476
- Queue mode is useful when ingestion becomes too slow to run inline. Redis connection problems are logged as warnings and should not throw from normal chatbot invocation.
477
-
478
- ## Custom Adapters
479
-
480
- You can use any model provider if you implement the public adapter interfaces.
481
-
482
- Custom LLM adapter:
483
-
484
- ```ts
485
- import type { LLMAdapter, Message } from "memo-grafter";
486
-
487
- class MyLLMAdapter implements LLMAdapter {
488
- async complete(messages: Message[], system?: string): Promise<string> {
489
- // Call your model provider here.
490
- return "Assistant response";
491
- }
492
- }
493
- ```
494
-
495
- Custom embedding adapter:
496
-
497
- ```ts
498
- import type { EmbedAdapter } from "memo-grafter";
499
-
500
- class MyEmbedAdapter implements EmbedAdapter {
501
- async embed(text: string): Promise<number[]> {
502
- // Return an embedding vector from your provider here.
503
- return [];
504
- }
505
- }
506
- ```
507
-
508
- Use them normally:
509
-
510
- ```ts
511
- const agent = new MemoGrafterAgent({
512
- db: {
513
- connectionString: process.env.DATABASE_URL!,
514
- },
515
- llm: new MyLLMAdapter(),
516
- embedder: new MyEmbedAdapter(),
517
- });
518
- ```
519
-
520
- Your embedding vector dimension must match the vector dimension expected by the database schema.
521
-
522
- ## Fleet API
523
-
524
- Fleets let you group color-scoped worker chatbots and use a conductor to graft memory across workers.
525
-
526
- ```ts
527
- import {
528
- MemoGrafterFleet,
529
- OpenAIEmbedAdapter,
530
- OpenAILLMAdapter,
531
- } from "memo-grafter";
532
-
533
- const fleet = new MemoGrafterFleet(
534
- {
535
- db: {
536
- connectionString: process.env.DATABASE_URL!,
537
- },
538
- llm: new OpenAILLMAdapter("gpt-4o"),
539
- embedder: new OpenAIEmbedAdapter("text-embedding-3-small"),
540
- },
541
- {
542
- id: "support-fleet",
543
- name: "Support Fleet",
544
- }
545
- );
546
-
547
- await fleet.initialize();
548
-
549
- const conductor = fleet.createConductor();
550
- const billing = await fleet.createWorker({ color: "billing" });
551
- const technical = await fleet.createWorker({ color: "technical" });
552
-
553
- await billing.invoke("The customer needs help understanding invoice credits.");
554
- await conductor.graftColorIntoAgent("billing", technical);
555
-
556
- const answer = await technical.invoke(
557
- "Use any relevant billing context while helping with this technical issue."
558
- );
559
-
560
- console.log(answer);
561
-
562
- await fleet.close();
563
- ```
564
-
565
- The worker color `conductor` is reserved.
566
-
567
- Prompt-guided fleet grafting:
568
-
569
- ```ts
570
- await conductor.graftByPrompt("invoice credit policy", technical, {
571
- minSimilarity: 0.6,
572
- limit: 3,
573
- });
574
- ```
575
-
576
- ## Example Project
577
-
578
- This repository includes a runnable example:
579
-
580
- ```text
581
- examples/chatbot-memory-demo
582
- ```
583
-
584
- Run it:
585
-
586
- ```bash
587
- cd D:/cohort/projects/project-memoGrafter
588
- npm install
589
- npm run build
590
-
591
- cd examples/chatbot-memory-demo
592
- npm install
593
- cp .env.example .env
594
- npm run dev
595
- ```
596
-
597
- PowerShell:
598
-
599
- ```powershell
600
- Copy-Item .env.example .env
601
- ```
602
-
603
- The example creates a travel chatbot and a writing chatbot, transfers Japan travel memory, and asks the writing bot to use that transferred context.
604
-
605
- ## Troubleshooting
606
-
607
- ### `DATABASE_URL is not reachable`
608
-
609
- Check that PostgreSQL is running and the connection string is correct.
610
-
611
- Confirm `pgvector` is enabled:
612
-
613
- ```sql
614
- CREATE EXTENSION IF NOT EXISTS vector;
615
- ```
616
-
617
- ### No Topic Nodes Are Created
618
-
619
- Common causes:
620
-
621
- - The conversation is too short.
622
- - `minSegmentMessages` is too high for the demo.
623
- - The LLM adapter failed.
624
- - The embedding adapter failed.
625
- - Queue mode is enabled and background ingestion has not finished yet.
626
-
627
- For small demos, try:
628
-
629
- ```ts
630
- drift: {
631
- mode: "intent",
632
- threshold: 0.3,
633
- minSegmentMessages: 3,
634
- }
635
- ```
636
-
637
- ### Absorb Copies Zero Nodes
638
-
639
- Inspect the source memory:
640
-
641
- ```ts
642
- console.log(await sourceAgent.getActiveNodes());
643
- ```
644
-
645
- Then try a lower similarity threshold:
646
-
647
- ```ts
648
- await targetAgent.absorbFromAgent(sourceAgent, {
649
- prompt: "Japan travel preferences",
650
- minSimilarity: 0.3,
651
- limit: 3,
652
- });
653
- ```
654
-
655
- ### Redis Warnings
656
-
657
- Redis is only required when you pass `queue` config. If you do not need background ingestion, remove the `queue` section.
658
-
659
- ### Browser Runtime Error
660
-
661
- MemoGrafter is server-side only. Run it in Node.js.
662
-
663
- ## Production Notes
664
-
665
- MemoGrafter v0.1.0 is experimental. Treat it as a starting point for prototypes and evaluation, not a finished production memory platform.
666
-
667
- Practical notes:
668
-
669
- - Keep secrets in environment variables.
670
- - Use PostgreSQL with `pgvector` enabled.
671
- - Tune `tokenBudget` to control prompt size and cost.
672
- - Use queue mode if ingestion becomes slow.
673
- - Store your own user/session mapping outside MemoGrafter.
674
- - Call `close()` during graceful shutdown.
675
- - Do not expose database credentials or OpenAI keys to browser code.
676
- - Run your own evaluation before trusting memory transfer behavior in user-facing flows.
677
-
678
- ## Public API Overview
679
-
680
- Main exports:
681
-
682
- - `MemoGrafterAgent`
683
- - `MemoGrafter`
684
- - `MemoGrafterFleet`
685
- - `WorkerAgent`
686
- - `ConductorAgent`
687
- - `OpenAILLMAdapter`
688
- - `OpenAIEmbedAdapter`
689
- - public shared and fleet types
690
-
691
- Common `MemoGrafterAgent` methods:
692
-
693
- - `initialize()`: initialize storage.
694
- - `invoke(message)`: send a user message and receive an assistant response.
695
- - `getHistory()`: read local chat history.
696
- - `getSessionId()`: read the current session ID.
697
- - `getActiveNodes()`: inspect topic nodes.
698
- - `getActiveSegments()`: inspect topic segments.
699
- - `graft(topicIds?)`: preview memory injection.
700
- - `ingestGraftedNodes(nodes)`: copy provided nodes into this agent.
701
- - `absorbFromAgent(sourceAgent, options)`: select and copy memory from another agent.
702
- - `close()`: close database and queue resources.
1
+ # MemoGrafter User Guide
2
+
3
+ ## Introduction
4
+
5
+ MemoGrafter is an experimental Node.js and TypeScript framework for structured chatbot memory. It stores chatbot conversations as message buffers, topic segments, topic nodes, and graph edges. Later, it can inject relevant memory into a chatbot turn or copy selected memory into another chatbot/session.
6
+
7
+ The project is intentionally focused. MemoGrafter is a chatbot memory framework, not an autonomous agent runtime. It does not run tools, schedule work, or decide goals for an agent. It helps a chatbot remember, retrieve, and transfer conversational context.
8
+
9
+ The most important idea is memory grafting. A chatbot can build useful memory during one conversation, and another chatbot can absorb only the relevant parts.
10
+
11
+ ## Requirements
12
+
13
+ - Node.js 18 or newer.
14
+ - TypeScript or modern JavaScript using ES modules.
15
+ - PostgreSQL with the `pgvector` extension enabled for the built-in `PostgresGraphStore`.
16
+ - An LLM adapter.
17
+ - An embedding adapter.
18
+ - An OpenAI API key only if using the included OpenAI adapters.
19
+ - An Anthropic API key only if using the included Anthropic LLM adapter.
20
+ - A Gemini API key only if using the included Gemini adapters.
21
+ - Redis only if enabling queue mode.
22
+
23
+ MemoGrafter is server-side only. Do not run it in browser code.
24
+
25
+ ## Installation
26
+
27
+ Install from npm:
28
+
29
+ ```bash
30
+ npm install memo-grafter
31
+ ```
32
+
33
+ Install from a local clone before publishing or while developing:
34
+
35
+ ```bash
36
+ cd path/to/your-app
37
+ npm install D:/cohort/projects/project-memoGrafter
38
+ ```
39
+
40
+ If you are working inside this repository, build it first:
41
+
42
+ ```bash
43
+ cd D:/cohort/projects/project-memoGrafter
44
+ npm install
45
+ npm run build
46
+ ```
47
+
48
+ ## Environment Setup
49
+
50
+ Create a `.env` file in your app:
51
+
52
+ ```bash
53
+ DATABASE_URL=postgres://postgres:postgres@localhost:5432/memo_grafter
54
+ ANTHROPIC_API_KEY=sk-ant-...
55
+ GEMINI_API_KEY=...
56
+ OPENAI_API_KEY=sk-...
57
+ REDIS_URL=redis://localhost:6379
58
+ ```
59
+
60
+ `DATABASE_URL` is required when using the built-in PostgreSQL storage.
61
+
62
+ `OPENAI_API_KEY` is required only when using `OpenAILLMAdapter` or `OpenAIEmbedAdapter`.
63
+
64
+ `ANTHROPIC_API_KEY` is required only when using `AnthropicLLMAdapter`.
65
+
66
+ `GEMINI_API_KEY` is required only when using `GeminiLLMAdapter` or `GeminiEmbedAdapter`.
67
+
68
+ `REDIS_URL` is optional and only needed when you pass `queue` config.
69
+
70
+ Enable `pgvector` in PostgreSQL:
71
+
72
+ ```sql
73
+ CREATE EXTENSION IF NOT EXISTS vector;
74
+ ```
75
+
76
+ The built-in `PostgresGraphStore` creates its own tables during `initialize()`.
77
+
78
+ Current v1 tables:
79
+
80
+ - `mg_message_buffer`
81
+ - `mg_segments`
82
+ - `mg_topic_nodes`
83
+ - `mg_topic_edges`
84
+ - `mg_memory_nodes`
85
+ - `mg_memory_edges`
86
+ - `mg_fleets`
87
+ - `mg_fleet_agents`
88
+
89
+ ## Quick Start
90
+
91
+ Create `src/index.ts`:
92
+
93
+ ```ts
94
+ import "dotenv/config";
95
+
96
+ import {
97
+ MemoGrafterAgent,
98
+ OpenAIEmbedAdapter,
99
+ OpenAILLMAdapter,
100
+ } from "memo-grafter";
101
+
102
+ const agent = new MemoGrafterAgent({
103
+ db: {
104
+ connectionString: process.env.DATABASE_URL!,
105
+ },
106
+ llm: new OpenAILLMAdapter("gpt-4o"),
107
+ embedder: new OpenAIEmbedAdapter("text-embedding-3-small"),
108
+ });
109
+
110
+ try {
111
+ await agent.initialize();
112
+
113
+ console.log(await agent.invoke("I am planning a Japan trip."));
114
+ console.log(await agent.invoke("I like quiet towns and local cafes."));
115
+ console.log(await agent.invoke("What should I remember while planning?"));
116
+
117
+ const nodes = await agent.getActiveNodes();
118
+ console.log(nodes.map((node) => ({ label: node.label, summary: node.summary })));
119
+ } finally {
120
+ await agent.close();
121
+ }
122
+ ```
123
+
124
+ Run it:
125
+
126
+ ```bash
127
+ npx tsx --env-file=.env src/index.ts
128
+ ```
129
+
130
+ ## Core Concepts
131
+
132
+ ### Messages
133
+
134
+ A message is one user or assistant turn:
135
+
136
+ ```ts
137
+ export interface Message {
138
+ role: "user" | "assistant";
139
+ content: string;
140
+ }
141
+ ```
142
+
143
+ `MemoGrafterAgent` keeps an in-memory history for the current session and stores messages in PostgreSQL during ingestion.
144
+
145
+ ### Segments
146
+
147
+ A segment is a range of messages that belong to the same topic. MemoGrafter uses drift detection to decide where topic boundaries are.
148
+
149
+ Drift detection combines several signals:
150
+
151
+ - how far the current message is from the current topic embedding,
152
+ - whether the current message is a sharp pivot from the previous user message,
153
+ - whether the message contains structural phrases such as "by the way", "different topic", or "going back to",
154
+ - short-message dampening so filler like "okay" or "got it" is less likely to create false boundaries,
155
+ - optional LLM classification for ambiguous topic-shift scores.
156
+
157
+ Example:
158
+
159
+ ```text
160
+ messages 0-4 -> Japan travel planning
161
+ messages 5-8 -> cover letter writing
162
+ ```
163
+
164
+ Segments are stored in `mg_segments`.
165
+
166
+ ### Topic Nodes
167
+
168
+ A topic node is the main unit of memory. It represents a segment as a label, summary, embedding, message range, and metadata.
169
+
170
+ Important fields:
171
+
172
+ - `id`: unique topic node ID.
173
+ - `label`: short label.
174
+ - `summary`: structured summary of the segment.
175
+ - `embedding`: vector used for semantic search.
176
+ - `messageRange`: source message range.
177
+ - `topicOrder`: chronological order.
178
+ - `driftScore`: topic-change score.
179
+ - `agentColor`, `fleetId`, `agentId`: nullable fleet metadata.
180
+
181
+ Topic nodes are stored in `mg_topic_nodes`.
182
+
183
+ ### Memory Nodes
184
+
185
+ Memory nodes are typed atomic memories attached to topic nodes. They are the units used by targeted recall.
186
+
187
+ Important fields:
188
+
189
+ - `memoryType`: `"fact"`, `"insight"`, `"question"`, `"task"`, or `"reference"`.
190
+ - `subject`, `predicate`, `value`: the structured memory triple.
191
+ - `confidence`: confidence score from `0` to `1`.
192
+ - `topicNodeId`: parent topic node ID.
193
+ - `decayed`: whether the memory is stale.
194
+ - `supersededBy`: newer memory ID when this memory has been replaced.
195
+
196
+ Memory nodes are stored in `mg_memory_nodes`.
197
+
198
+ ### Graph Edges
199
+
200
+ Edges connect related topic nodes. They can represent temporal, semantic, grafted, or reentry relationships.
201
+
202
+ - `temporal`: one topic followed another in the conversation.
203
+ - `semantic`: two topics are similar by embedding search.
204
+ - `grafted`: a topic was copied from another session or chatbot.
205
+ - `reentry`: the conversation returned to an earlier topic after discussing something else.
206
+
207
+ Edges are stored in `mg_topic_edges`.
208
+
209
+ ### Grafting
210
+
211
+ Grafting is the process of selecting memory nodes and turning them into useful context for another prompt or another chatbot.
212
+
213
+ There are two common forms:
214
+
215
+ - Preview memory with `graft()`.
216
+ - Copy memory into another chatbot with `absorbFromAgent()` or `ingestGraftedNodes()`.
217
+
218
+ ## Using MemoGrafterAgent
219
+
220
+ `MemoGrafterAgent` is the easiest API to start with.
221
+
222
+ ```ts
223
+ const agent = new MemoGrafterAgent({
224
+ db: {
225
+ connectionString: process.env.DATABASE_URL!,
226
+ },
227
+ llm: new OpenAILLMAdapter("gpt-4o"),
228
+ embedder: new OpenAIEmbedAdapter("text-embedding-3-small"),
229
+ });
230
+ ```
231
+
232
+ Initialize it before use:
233
+
234
+ ```ts
235
+ await agent.initialize();
236
+ ```
237
+
238
+ Send user messages with `invoke()`:
239
+
240
+ ```ts
241
+ const answer = await agent.invoke("Help me plan a Kyoto itinerary.");
242
+ ```
243
+
244
+ Close resources when done:
245
+
246
+ ```ts
247
+ await agent.close();
248
+ ```
249
+
250
+ ### What `invoke()` Does
251
+
252
+ On every call, `invoke()`:
253
+
254
+ 1. Adds the user message to local history.
255
+ 2. Loads existing topic nodes for the session.
256
+ 3. Builds a memory injection prompt from those nodes.
257
+ 4. Sends history plus memory prompt to the LLM.
258
+ 5. Adds the assistant response to history.
259
+ 6. Ingests the updated conversation into the memory graph.
260
+
261
+ On the first turn there may be no memory to inject. Later turns can use memory created from earlier turns.
262
+
263
+ ### Targeted Recall
264
+
265
+ Use `recall()` when you want to retrieve structured memory by meaning without asking the LLM to produce an answer.
266
+
267
+ ```ts
268
+ const result = await agent.recall("deployment config", {
269
+ limit: 8,
270
+ minSimilarity: 0.55,
271
+ tokenBudget: 1000,
272
+ });
273
+
274
+ console.log(result.facts);
275
+ console.log(result.nodes);
276
+ console.log(result.systemPrompt);
277
+ console.log(result.tokenCount);
278
+ ```
279
+
280
+ `recall()` returns a `RetrievalResult`:
281
+
282
+ - `facts`: matching memory nodes with a `similarity` score.
283
+ - `nodes`: parent topic nodes for the included facts.
284
+ - `systemPrompt`: a formatted memory block that can be passed to an LLM if you choose.
285
+ - `tokenCount`: approximate token count for the included fact blocks.
286
+
287
+ Options:
288
+
289
+ - `limit`: max memory nodes to fetch before filtering. Defaults to `10`.
290
+ - `minSimilarity`: cosine similarity floor. Defaults to `0.6`.
291
+ - `tokenBudget`: max approximate tokens for included fact blocks. Defaults to `1200`.
292
+
293
+ `recall()` is side-effect free. It does not call `invoke()`, does not trigger a new LLM completion, does not mutate local history, and does not inject the result automatically. Your application decides whether to display the memories, add `result.systemPrompt` to a model call, or ignore the result.
294
+
295
+ If you call `recall()` immediately after `invoke()`, it only sees memory that has already been ingested into storage. In queue mode, wait for your background worker to finish before expecting newly created memories to appear.
296
+
297
+ ## Inspecting Memory
298
+
299
+ Read active topic nodes:
300
+
301
+ ```ts
302
+ const nodes = await agent.getActiveNodes();
303
+
304
+ for (const node of nodes) {
305
+ console.log({
306
+ id: node.id,
307
+ label: node.label,
308
+ summary: node.summary,
309
+ messageRange: node.messageRange,
310
+ topicOrder: node.topicOrder,
311
+ driftScore: node.driftScore,
312
+ });
313
+ }
314
+ ```
315
+
316
+ Read active segments:
317
+
318
+ ```ts
319
+ const segments = await agent.getActiveSegments();
320
+ console.log(segments);
321
+ ```
322
+
323
+ Read the in-memory chat history:
324
+
325
+ ```ts
326
+ const history = agent.getHistory();
327
+ console.log(history);
328
+ ```
329
+
330
+ Read the session ID:
331
+
332
+ ```ts
333
+ console.log(agent.getSessionId());
334
+ ```
335
+
336
+ ## Grafting Memory
337
+
338
+ Use `graft()` to preview what memory would be injected:
339
+
340
+ ```ts
341
+ const graft = await agent.graft();
342
+
343
+ console.log(graft.systemPrompt);
344
+ console.log(graft.nodes);
345
+ console.log(graft.tokenCount);
346
+ ```
347
+
348
+ You can graft specific topic IDs:
349
+
350
+ ```ts
351
+ const nodes = await agent.getActiveNodes();
352
+
353
+ const graft = await agent.graft([nodes[0]!.id]);
354
+ ```
355
+
356
+ `graft()` returns:
357
+
358
+ - `systemPrompt`: memory context suitable for an LLM system prompt.
359
+ - `nodes`: selected topic nodes.
360
+ - `tokenCount`: estimated token count.
361
+
362
+ ## Absorbing Memory Into Another Chatbot
363
+
364
+ Use `absorbFromAgent()` to copy selected memory from one chatbot into another.
365
+
366
+ ```ts
367
+ const travelBot = new MemoGrafterAgent(config);
368
+ const writingBot = new MemoGrafterAgent(config);
369
+
370
+ await travelBot.initialize();
371
+ await writingBot.initialize();
372
+
373
+ await travelBot.invoke("I am planning a Japan trip.");
374
+ await travelBot.invoke("I like quiet towns, bookstores, and local cafes.");
375
+ await travelBot.invoke("My budget is around 2500 dollars.");
376
+
377
+ const copiedNodes = await writingBot.absorbFromAgent(travelBot, {
378
+ prompt: "Japan travel preferences",
379
+ minSimilarity: 0.6,
380
+ limit: 3,
381
+ });
382
+
383
+ console.log(`Copied ${copiedNodes.length} nodes.`);
384
+
385
+ const response = await writingBot.invoke(
386
+ "Suggest a reflective blog intro for my Japan trip."
387
+ );
388
+
389
+ console.log(response);
390
+ ```
391
+
392
+ ### Absorb By Semantic Prompt
393
+
394
+ ```ts
395
+ await targetAgent.absorbFromAgent(sourceAgent, {
396
+ prompt: "Japan travel preferences",
397
+ minSimilarity: 0.6,
398
+ limit: 3,
399
+ });
400
+ ```
401
+
402
+ Use this when you want MemoGrafter to find relevant memory by meaning.
403
+
404
+ ### Absorb By Topic ID
405
+
406
+ ```ts
407
+ const sourceNodes = await sourceAgent.getActiveNodes();
408
+
409
+ await targetAgent.absorbFromAgent(sourceAgent, {
410
+ topicIds: [sourceNodes[0]!.id],
411
+ });
412
+ ```
413
+
414
+ Use this when your UI lets a user choose memory nodes manually.
415
+
416
+ ### Ingest Grafted Nodes Directly
417
+
418
+ ```ts
419
+ const graft = await sourceAgent.graft();
420
+ await targetAgent.ingestGraftedNodes(graft.nodes);
421
+ ```
422
+
423
+ Use this when you want to inspect or filter a graft before copying it.
424
+
425
+ ## Configuration
426
+
427
+ Full shape:
428
+
429
+ ```ts
430
+ const agent = new MemoGrafterAgent({
431
+ db: {
432
+ connectionString: process.env.DATABASE_URL!,
433
+ },
434
+ llm,
435
+ embedder,
436
+ drift: {
437
+ mode: "intent",
438
+ windowSize: 5,
439
+ driftSensitivity: "medium",
440
+ minSegmentMessages: 3,
441
+ llmAmbiguityDetection: false,
442
+ reentryDetection: true,
443
+ reentryThreshold: 0.85,
444
+ },
445
+ graph: {
446
+ topK: 5,
447
+ hopDepth: 2,
448
+ },
449
+ inject: {
450
+ bufferSize: 4,
451
+ tokenBudget: 1500,
452
+ },
453
+ });
454
+ ```
455
+
456
+ ### `db`
457
+
458
+ ```ts
459
+ db: {
460
+ connectionString: process.env.DATABASE_URL!,
461
+ }
462
+ ```
463
+
464
+ PostgreSQL connection string used by the built-in `PostgresGraphStore`.
465
+
466
+ MemoGrafter currently constructs `PostgresGraphStore` from this config internally. Advanced users can also import the storage contract directly:
467
+
468
+ ```ts
469
+ import {
470
+ PostgresGraphStore,
471
+ type GraphStore,
472
+ } from "memo-grafter";
473
+
474
+ const store: GraphStore = new PostgresGraphStore(process.env.DATABASE_URL!);
475
+ ```
476
+
477
+ `GraphStore` is the public storage interface. `PostgresGraphStore` is the default PostgreSQL and pgvector implementation.
478
+
479
+ Useful store inspection methods include:
480
+
481
+ - `getNodesBySession(sessionId)`: read topic nodes for a session.
482
+ - `getTopicNode(topicNodeId, sessionId?)`: read one topic node by ID.
483
+ - `getSegmentsBySession(sessionId)`: read topic segments for a session.
484
+ - `getEdgesByType(sessionId, type)`: inspect graph edges such as `"reentry"`, `"semantic"`, `"temporal"`, or `"grafted"`.
485
+
486
+ ### `llm`
487
+
488
+ ```ts
489
+ llm: new OpenAILLMAdapter("gpt-4o")
490
+ ```
491
+
492
+ Adapter used to generate assistant responses and summarize segments.
493
+
494
+ Anthropic models can be used with the included Anthropic adapter:
495
+
496
+ ```ts
497
+ llm: new AnthropicLLMAdapter("claude-sonnet-4-5")
498
+ ```
499
+
500
+ Gemini models can be used with the included Gemini adapter:
501
+
502
+ ```ts
503
+ llm: new GeminiLLMAdapter("gemini-2.5-flash")
504
+ ```
505
+
506
+ ### `embedder`
507
+
508
+ ```ts
509
+ embedder: new OpenAIEmbedAdapter("text-embedding-3-small")
510
+ ```
511
+
512
+ Adapter used to create vectors for semantic search.
513
+
514
+ Anthropic does not provide a native embedding API. Pair `AnthropicLLMAdapter` with `OpenAIEmbedAdapter` or another custom `EmbedAdapter`.
515
+
516
+ Gemini embeddings can be used with the included Gemini embedder:
517
+
518
+ ```ts
519
+ embedder: new GeminiEmbedAdapter("gemini-embedding-001")
520
+ ```
521
+
522
+ `GeminiEmbedAdapter` requests 1536-dimensional embeddings by default to match MemoGrafter's current `vector(1536)` database schema. If you change the schema dimension, pass the matching value as the second constructor argument.
523
+
524
+ ### `drift`
525
+
526
+ ```ts
527
+ drift: {
528
+ mode: "intent",
529
+ windowSize: 5,
530
+ driftSensitivity: "medium",
531
+ minSegmentMessages: 3,
532
+ llmAmbiguityDetection: false,
533
+ reentryDetection: true,
534
+ reentryThreshold: 0.85,
535
+ }
536
+ ```
537
+
538
+ Controls topic boundary detection.
539
+
540
+ - `mode`: `"intent"` or `"window"`.
541
+ - `windowSize`: message window size for window mode.
542
+ - `driftSensitivity`: preferred sensitivity preset, one of `"low"`, `"medium"`, or `"high"`.
543
+ - `threshold`: deprecated numeric threshold. It still works when `driftSensitivity` is not set, but MemoGrafter logs a one-time warning.
544
+ - `minSegmentMessages`: minimum messages before a boundary.
545
+ - `llmAmbiguityDetection`: optional LLM check for borderline topic shifts. Defaults to `false`.
546
+ - `reentryDetection`: whether to link later topic returns back to earlier topic nodes. Defaults to `true`.
547
+ - `reentryThreshold`: embedding similarity threshold for reentry detection. Defaults to `0.85`.
548
+
549
+ Use `"intent"` for most chatbot memory demos. In intent mode, user messages drive topic shifts.
550
+
551
+ Sensitivity presets resolve internally to numeric thresholds:
552
+
553
+ - `"low"`: `0.25`
554
+ - `"medium"`: `0.35`
555
+ - `"high"`: `0.50`
556
+
557
+ Use `"medium"` first. Boundaries are cut when a drift score exceeds the resolved threshold, so lower numeric thresholds split more readily and higher numeric thresholds require stronger evidence.
558
+
559
+ If both `driftSensitivity` and `threshold` are provided, `driftSensitivity` wins.
560
+
561
+ #### Reentry Detection
562
+
563
+ Reentry detection handles conversations that leave a topic and later return to it:
564
+
565
+ ```text
566
+ database choice -> authentication flow -> database connection pooling
567
+ ```
568
+
569
+ Without reentry detection, the later database discussion is just another topic node. With reentry detection, MemoGrafter creates a `reentry` edge from the later database node back to the earlier database node.
570
+
571
+ This helps graph traversal and memory injection recover earlier related context. A later question about connection pooling can still be connected to the original PostgreSQL/ACID discussion.
572
+
573
+ Reentry edges are written between the current rebuilt topic nodes. They do not point at deleted nodes from previous ingestion passes.
574
+
575
+ ### `graph`
576
+
577
+ ```ts
578
+ graph: {
579
+ topK: 5,
580
+ hopDepth: 2,
581
+ }
582
+ ```
583
+
584
+ Controls graph retrieval and traversal.
585
+
586
+ - `topK`: number of similar nodes to retrieve.
587
+ - `hopDepth`: how far grafting walks graph neighbors.
588
+
589
+ ### `inject`
590
+
591
+ ```ts
592
+ inject: {
593
+ bufferSize: 4,
594
+ tokenBudget: 1500,
595
+ }
596
+ ```
597
+
598
+ Controls how much memory is inserted into the prompt.
599
+
600
+ - `bufferSize`: nearby raw messages to include.
601
+ - `tokenBudget`: approximate memory prompt budget.
602
+
603
+ ## Queue Mode
604
+
605
+ Without queue config, ingestion runs synchronously after `invoke()`.
606
+
607
+ With queue config, MemoGrafter uses BullMQ and Redis:
608
+
609
+ ```ts
610
+ const agent = new MemoGrafterAgent({
611
+ db: {
612
+ connectionString: process.env.DATABASE_URL!,
613
+ },
614
+ llm: new OpenAILLMAdapter("gpt-4o"),
615
+ embedder: new OpenAIEmbedAdapter("text-embedding-3-small"),
616
+ queue: {
617
+ redisUrl: process.env.REDIS_URL!,
618
+ removeOnComplete: true,
619
+ removeOnFail: true,
620
+ },
621
+ });
622
+ ```
623
+
624
+ Queue mode is useful when ingestion becomes too slow to run inline. Redis connection problems are logged as warnings and should not throw from normal chatbot invocation.
625
+
626
+ ## Custom Adapters
627
+
628
+ You can use any model provider if you implement the public adapter interfaces.
629
+
630
+ Custom LLM adapter:
631
+
632
+ ```ts
633
+ import type { LLMAdapter, Message } from "memo-grafter";
634
+
635
+ class MyLLMAdapter implements LLMAdapter {
636
+ async complete(messages: Message[], system?: string): Promise<string> {
637
+ // Call your model provider here.
638
+ return "Assistant response";
639
+ }
640
+ }
641
+ ```
642
+
643
+ Custom embedding adapter:
644
+
645
+ ```ts
646
+ import type { EmbedAdapter } from "memo-grafter";
647
+
648
+ class MyEmbedAdapter implements EmbedAdapter {
649
+ async embed(text: string): Promise<number[]> {
650
+ // Return an embedding vector from your provider here.
651
+ return [];
652
+ }
653
+ }
654
+ ```
655
+
656
+ Use them normally:
657
+
658
+ ```ts
659
+ const agent = new MemoGrafterAgent({
660
+ db: {
661
+ connectionString: process.env.DATABASE_URL!,
662
+ },
663
+ llm: new MyLLMAdapter(),
664
+ embedder: new MyEmbedAdapter(),
665
+ });
666
+ ```
667
+
668
+ Your embedding vector dimension must match the vector dimension expected by the database schema.
669
+
670
+ ## Storage Backends
671
+
672
+ MemoGrafter exposes storage through the `GraphStore` interface. The built-in implementation is `PostgresGraphStore`, which stores relational data, graph edges, and vector search data in PostgreSQL with `pgvector`.
673
+
674
+ Most users do not need to instantiate a store directly. `MemoGrafter` and `MemoGrafterAgent` use `PostgresGraphStore` from the `db.connectionString` config:
675
+
676
+ ```ts
677
+ const agent = new MemoGrafterAgent({
678
+ db: {
679
+ connectionString: process.env.DATABASE_URL!,
680
+ },
681
+ llm,
682
+ embedder,
683
+ });
684
+ ```
685
+
686
+ If you are extending MemoGrafter, use `GraphStore` as the contract and keep concrete storage details behind an implementation:
687
+
688
+ ```ts
689
+ import type { GraphStore } from "memo-grafter";
690
+
691
+ class MyGraphStore implements GraphStore {
692
+ // Implement the full GraphStore contract.
693
+ }
694
+ ```
695
+
696
+ Future storage implementations can use the same interface without changing pipeline or fleet code. For example, a SQLite plus vector-database implementation would implement `GraphStore` while preserving the same behavior expected by ingestion, grafting, and fleet APIs.
697
+
698
+ ## Fleet API
699
+
700
+ Fleets let you group color-scoped worker chatbots and use a conductor to graft memory across workers.
701
+
702
+ ```ts
703
+ import {
704
+ MemoGrafterFleet,
705
+ OpenAIEmbedAdapter,
706
+ OpenAILLMAdapter,
707
+ } from "memo-grafter";
708
+
709
+ const fleet = new MemoGrafterFleet(
710
+ {
711
+ db: {
712
+ connectionString: process.env.DATABASE_URL!,
713
+ },
714
+ llm: new OpenAILLMAdapter("gpt-4o"),
715
+ embedder: new OpenAIEmbedAdapter("text-embedding-3-small"),
716
+ },
717
+ {
718
+ id: "support-fleet",
719
+ name: "Support Fleet",
720
+ }
721
+ );
722
+
723
+ await fleet.initialize();
724
+
725
+ const conductor = fleet.createConductor();
726
+ const billing = await fleet.createWorker({ color: "billing" });
727
+ const technical = await fleet.createWorker({ color: "technical" });
728
+
729
+ await billing.invoke("The customer needs help understanding invoice credits.");
730
+ await conductor.graftColorIntoAgent("billing", technical);
731
+
732
+ const answer = await technical.invoke(
733
+ "Use any relevant billing context while helping with this technical issue."
734
+ );
735
+
736
+ console.log(answer);
737
+
738
+ await fleet.close();
739
+ ```
740
+
741
+ The worker color `conductor` is reserved.
742
+
743
+ Prompt-guided fleet grafting:
744
+
745
+ ```ts
746
+ await conductor.graftByPrompt("invoice credit policy", technical, {
747
+ minSimilarity: 0.6,
748
+ limit: 3,
749
+ });
750
+ ```
751
+
752
+ ## Example Project
753
+
754
+ This repository includes a runnable example:
755
+
756
+ ```text
757
+ examples/chatbot-memory-demo
758
+ ```
759
+
760
+ Run it:
761
+
762
+ ```bash
763
+ cd D:/cohort/projects/project-memoGrafter
764
+ npm install
765
+ npm run build
766
+
767
+ cd examples/chatbot-memory-demo
768
+ npm install
769
+ cp .env.example .env
770
+ npm run dev
771
+ ```
772
+
773
+ PowerShell:
774
+
775
+ ```powershell
776
+ Copy-Item .env.example .env
777
+ ```
778
+
779
+ The example creates a travel chatbot and a writing chatbot, transfers Japan travel memory, and asks the writing bot to use that transferred context.
780
+
781
+ ## Troubleshooting
782
+
783
+ ### `DATABASE_URL is not reachable`
784
+
785
+ Check that PostgreSQL is running and the connection string is correct.
786
+
787
+ Confirm `pgvector` is enabled:
788
+
789
+ ```sql
790
+ CREATE EXTENSION IF NOT EXISTS vector;
791
+ ```
792
+
793
+ ### No Topic Nodes Are Created
794
+
795
+ Common causes:
796
+
797
+ - The conversation is too short.
798
+ - `minSegmentMessages` is too high for the demo.
799
+ - The LLM adapter failed.
800
+ - The embedding adapter failed.
801
+ - Queue mode is enabled and background ingestion has not finished yet.
802
+
803
+ For small demos, try:
804
+
805
+ ```ts
806
+ drift: {
807
+ mode: "intent",
808
+ driftSensitivity: "medium",
809
+ minSegmentMessages: 3,
810
+ }
811
+ ```
812
+
813
+ If segments are too coarse, try a lower resolved threshold:
814
+
815
+ ```ts
816
+ drift: {
817
+ mode: "intent",
818
+ driftSensitivity: "low",
819
+ minSegmentMessages: 2,
820
+ }
821
+ ```
822
+
823
+ If segments are too fragmented, try a higher resolved threshold:
824
+
825
+ ```ts
826
+ drift: {
827
+ mode: "intent",
828
+ driftSensitivity: "high",
829
+ minSegmentMessages: 4,
830
+ }
831
+ ```
832
+
833
+ ### Absorb Copies Zero Nodes
834
+
835
+ Inspect the source memory:
836
+
837
+ ```ts
838
+ console.log(await sourceAgent.getActiveNodes());
839
+ ```
840
+
841
+ Then try a lower similarity threshold:
842
+
843
+ ```ts
844
+ await targetAgent.absorbFromAgent(sourceAgent, {
845
+ prompt: "Japan travel preferences",
846
+ minSimilarity: 0.3,
847
+ limit: 3,
848
+ });
849
+ ```
850
+
851
+ ### Recall Returns Zero Facts
852
+
853
+ `recall()` searches atomic memory nodes, not raw chat messages. If it returns no facts:
854
+
855
+ - Make sure ingestion has completed.
856
+ - Confirm memory nodes exist for the session.
857
+ - Try a lower `minSimilarity`.
858
+ - Try a more specific query that uses the same vocabulary as the original conversation.
859
+
860
+ Example:
861
+
862
+ ```ts
863
+ const result = await agent.recall("Japan travel preferences", {
864
+ minSimilarity: 0.3,
865
+ limit: 5,
866
+ });
867
+ ```
868
+
869
+ ### Redis Warnings
870
+
871
+ Redis is only required when you pass `queue` config. If you do not need background ingestion, remove the `queue` section.
872
+
873
+ ### Browser Runtime Error
874
+
875
+ MemoGrafter is server-side only. Run it in Node.js.
876
+
877
+ ## Production Notes
878
+
879
+ MemoGrafter v0.1.0 is experimental. Treat it as a starting point for prototypes and evaluation, not a finished production memory platform.
880
+
881
+ Practical notes:
882
+
883
+ - Keep secrets in environment variables.
884
+ - Use PostgreSQL with `pgvector` enabled.
885
+ - Tune `tokenBudget` to control prompt size and cost.
886
+ - Use queue mode if ingestion becomes slow.
887
+ - Store your own user/session mapping outside MemoGrafter.
888
+ - Call `close()` during graceful shutdown.
889
+ - Do not expose database credentials or OpenAI keys to browser code.
890
+ - Run your own evaluation before trusting memory transfer behavior in user-facing flows.
891
+
892
+ ## Public API Overview
893
+
894
+ Main exports:
895
+
896
+ - `MemoGrafterAgent`
897
+ - `MemoGrafter`
898
+ - `MemoGrafterFleet`
899
+ - `WorkerAgent`
900
+ - `ConductorAgent`
901
+ - `AnthropicLLMAdapter`
902
+ - `GeminiLLMAdapter`
903
+ - `GeminiEmbedAdapter`
904
+ - `OpenAILLMAdapter`
905
+ - `OpenAIEmbedAdapter`
906
+ - `PostgresGraphStore`
907
+ - `GraphStore`
908
+ - `FleetAgentRecord`
909
+ - `RetrievalResult`
910
+ - `RetrieverConfig`
911
+ - public shared and fleet types
912
+
913
+ Useful `GraphStore` inspection methods:
914
+
915
+ - `getTopicNode(topicNodeId, sessionId?)`
916
+ - `getNodesBySession(sessionId)`
917
+ - `getSegmentsBySession(sessionId)`
918
+ - `getEdgesByType(sessionId, type)`
919
+
920
+ Common `MemoGrafterAgent` methods:
921
+
922
+ - `initialize()`: initialize storage.
923
+ - `invoke(message)`: send a user message and receive an assistant response.
924
+ - `getHistory()`: read local chat history.
925
+ - `getSessionId()`: read the current session ID.
926
+ - `getActiveNodes()`: inspect topic nodes.
927
+ - `getActiveSegments()`: inspect topic segments.
928
+ - `recall(query, options?)`: retrieve structured memory by semantic query.
929
+ - `graft(topicIds?)`: preview memory injection.
930
+ - `ingestGraftedNodes(nodes)`: copy provided nodes into this agent.
931
+ - `absorbFromAgent(sourceAgent, options)`: select and copy memory from another agent.
932
+ - `close()`: close database and queue resources.