@danielsimonjr/memoryjs 1.4.0 → 1.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,654 +1,710 @@
1
- # MemoryJS
2
-
3
- [![Version](https://img.shields.io/badge/version-1.2.0-blue.svg)](https://github.com/danielsimonjr/memoryjs)
4
- [![NPM](https://img.shields.io/npm/v/@danielsimonjr/memoryjs.svg)](https://www.npmjs.com/package/@danielsimonjr/memoryjs)
5
- [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
6
- [![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
7
-
8
- A **TypeScript knowledge graph library** for managing entities, relations, and observations with **advanced search capabilities**, **hierarchical organization**, and **multiple storage backends**.
9
-
10
- > **Core library** powering [@danielsimonjr/memory-mcp](https://www.npmjs.com/package/@danielsimonjr/memory-mcp). Provides **93 TypeScript files**, **~41K lines of code**, dual storage backends (JSONL/SQLite), sophisticated search algorithms (BM25, TF-IDF, fuzzy, semantic, hybrid), and a complete **Agent Memory System** for AI agents.
11
-
12
- ## Table of Contents
13
-
14
- - [Features](#features)
15
- - [Installation](#installation)
16
- - [Quick Start](#quick-start)
17
- - [Core Concepts](#core-concepts)
18
- - [Storage Options](#storage-options)
19
- - [Search Capabilities](#search-capabilities)
20
- - [Graph Algorithms](#graph-algorithms)
21
- - [Agent Memory System](#agent-memory-system)
22
- - [API Reference](#api-reference)
23
- - [Configuration](#configuration)
24
- - [Development](#development)
25
- - [Documentation](#documentation)
26
- - [License](#license)
27
-
28
- ## Features
29
-
30
- ### Core Capabilities
31
-
32
- - **Knowledge Graph Storage**: Entity-Relation-Observation model for structured data
33
- - **Dual Storage Backends**: JSONL (human-readable) or SQLite (FTS5, 3-10x faster)
34
- - **Full CRUD Operations**: Create, read, update, delete entities and relations
35
- - **Hierarchical Nesting**: Parent-child relationships for tree structures
36
- - **Timestamps**: Automatic createdAt and lastModified tracking
37
-
38
- ### Advanced Features
39
-
40
- | Category | Description |
41
- |----------|-------------|
42
- | **Search Algorithms** | Basic, TF-IDF ranked, BM25, Boolean (AND/OR/NOT), Fuzzy (Levenshtein), Semantic (embeddings), Hybrid |
43
- | **Graph Algorithms** | Shortest path (BFS), all paths, centrality metrics (degree, betweenness, PageRank), connected components |
44
- | **Hierarchical Nesting** | Parent-child relationships, ancestor/descendant traversal, subtree operations |
45
- | **Duplicate Detection** | Intelligent compression with similarity scoring |
46
- | **Tag Management** | Tags, aliases, bulk operations, importance scores (0-10) |
47
- | **Import/Export** | JSON, CSV, GraphML formats with Brotli compression |
48
- | **Analytics** | Graph statistics, validation, integrity checks |
49
-
50
- ### Module Statistics
51
-
52
- | Module | Files | Key Components |
53
- |--------|-------|----------------|
54
- | `agent/` | 19 | AgentMemoryManager, SessionManager, DecayEngine, WorkingMemoryManager |
55
- | `core/` | 12 | EntityManager, GraphStorage, SQLiteStorage, TransactionManager |
56
- | `search/` | 29 | SearchManager, BM25Search, HybridScorer, VectorStore, QueryPlanner |
57
- | `features/` | 9 | IOManager, ArchiveManager, CompressionManager, StreamingExporter |
58
- | `utils/` | 18 | BatchProcessor, CompressedCache, WorkerPoolManager, MemoryMonitor |
59
- | `types/` | 3 | Entity, Relation, AgentEntity, SessionEntity interfaces |
60
- | `workers/` | 2 | Levenshtein distance calculations |
61
-
62
- **Total:** 93 TypeScript files | ~41,000 lines of code | 657 exports | 91 classes | 216 interfaces
63
-
64
- ## Installation
65
-
66
- ```bash
67
- npm install @danielsimonjr/memoryjs
68
- ```
69
-
70
- ### Requirements
71
-
72
- - Node.js >= 18.0.0
73
- - TypeScript >= 5.0 (for development)
74
-
75
- ## Quick Start
76
-
77
- ### 1. Initialize Storage
78
-
79
- ```typescript
80
- import { ManagerContext } from '@danielsimonjr/memoryjs';
81
-
82
- // JSONL storage (default, human-readable)
83
- const ctx = new ManagerContext({
84
- storagePath: './memory.jsonl'
85
- });
86
-
87
- // Or SQLite storage (faster, FTS5 search)
88
- const ctx = new ManagerContext({
89
- storageType: 'sqlite',
90
- storagePath: './memory.db'
91
- });
92
- ```
93
-
94
- ### 2. Create Entities
95
-
96
- ```typescript
97
- await ctx.entityManager.createEntities([
98
- {
99
- name: 'TypeScript',
100
- entityType: 'language',
101
- observations: ['A typed superset of JavaScript'],
102
- tags: ['programming', 'frontend'],
103
- importance: 8
104
- },
105
- {
106
- name: 'Node.js',
107
- entityType: 'runtime',
108
- observations: ['JavaScript runtime built on V8'],
109
- tags: ['backend', 'server']
110
- }
111
- ]);
112
- ```
113
-
114
- ### 3. Create Relations
115
-
116
- ```typescript
117
- await ctx.relationManager.createRelations([
118
- { from: 'TypeScript', to: 'Node.js', relationType: 'runs_on' }
119
- ]);
120
- ```
121
-
122
- ### 4. Search
123
-
124
- ```typescript
125
- // Basic search
126
- const results = await ctx.searchManager.search('JavaScript');
127
-
128
- // Ranked search (TF-IDF scoring)
129
- const ranked = await ctx.searchManager.searchRanked('runtime environment', { limit: 10 });
130
-
131
- // Boolean search
132
- const filtered = await ctx.searchManager.booleanSearch('TypeScript AND runtime');
133
-
134
- // Fuzzy search (typo-tolerant)
135
- const fuzzy = await ctx.searchManager.fuzzySearch('Typscript', { threshold: 0.7 });
136
- ```
137
-
138
- ## Core Concepts
139
-
140
- ### Entities
141
-
142
- Primary nodes in the knowledge graph.
143
-
144
- ```typescript
145
- interface Entity {
146
- name: string; // Unique identifier
147
- entityType: string; // Classification (person, project, concept)
148
- observations: string[]; // Facts about the entity
149
- parentId?: string; // Parent entity for hierarchical nesting
150
- tags?: string[]; // Lowercase tags for categorization
151
- importance?: number; // 0-10 scale for prioritization
152
- createdAt?: string; // ISO 8601 timestamp
153
- lastModified?: string; // ISO 8601 timestamp
154
- }
155
- ```
156
-
157
- ### Relations
158
-
159
- Directed connections between entities.
160
-
161
- ```typescript
162
- interface Relation {
163
- from: string; // Source entity name
164
- to: string; // Target entity name
165
- relationType: string; // Relationship type (active voice)
166
- }
167
- ```
168
-
169
- ### Observations
170
-
171
- Discrete facts about entities. Each observation should be atomic and independently manageable. Use `addObservations()` to append new facts without overwriting existing ones.
172
-
173
- ### ManagerContext
174
-
175
- Central access point for all managers with lazy initialization:
176
-
177
- ```typescript
178
- ctx.entityManager // Entity CRUD + hierarchy
179
- ctx.relationManager // Relation management
180
- ctx.searchManager // All search operations
181
- ctx.tagManager // Tag aliases and bulk operations
182
- ctx.ioManager // Import/export/backup
183
- ctx.graphTraversal // Graph algorithms
184
- ctx.semanticSearch // Vector similarity search (optional)
185
- ```
186
-
187
- ## Storage Options
188
-
189
- ### Comparison
190
-
191
- | Feature | JSONL (Default) | SQLite (better-sqlite3) |
192
- |---------|-----------------|-------------------------|
193
- | Format | Human-readable text | Native binary database |
194
- | Transactions | Basic | Full ACID with WAL mode |
195
- | Full-Text Search | Basic | FTS5 with BM25 ranking |
196
- | Performance | Good | 3-10x faster |
197
- | Concurrency | Single-threaded | Thread-safe with async-mutex |
198
- | Best For | Small graphs, debugging | Large graphs (10k+ entities) |
199
-
200
- ### JSONL Storage
201
-
202
- ```typescript
203
- const ctx = new ManagerContext({
204
- storagePath: './memory.jsonl'
205
- });
206
- ```
207
-
208
- Features:
209
- - Human-readable line-delimited JSON
210
- - In-memory caching with write-through invalidation
211
- - Atomic writes via temp file + rename
212
- - Backward compatibility for legacy formats
213
-
214
- ### SQLite Storage
215
-
216
- ```typescript
217
- const ctx = new ManagerContext({
218
- storageType: 'sqlite',
219
- storagePath: './memory.db'
220
- });
221
- ```
222
-
223
- Features:
224
- - FTS5 full-text search with BM25 ranking
225
- - WAL mode for better concurrency
226
- - Referential integrity with ON DELETE CASCADE
227
- - ACID transactions
228
-
229
- ### Storage Files
230
-
231
- When using JSONL, related files are automatically created:
232
-
233
- ```
234
- /your/data/directory/
235
- ├── memory.jsonl # Main knowledge graph
236
- ├── memory-saved-searches.jsonl # Saved search queries
237
- ├── memory-tag-aliases.jsonl # Tag synonym mappings
238
- └── .backups/ # Timestamped backups
239
- ```
240
-
241
- ## Search Capabilities
242
-
243
- ### Search Methods
244
-
245
- | Method | Description | Use Case |
246
- |--------|-------------|----------|
247
- | `search()` | Basic substring matching | Simple queries |
248
- | `searchRanked()` | TF-IDF relevance scoring | Finding most relevant results |
249
- | `booleanSearch()` | AND/OR/NOT operators | Complex filtering |
250
- | `fuzzySearch()` | Levenshtein distance | Typo tolerance |
251
- | `hybridSearch()` | Semantic + lexical + symbolic | Multi-signal ranking |
252
-
253
- ### Basic Search
254
-
255
- ```typescript
256
- const results = await ctx.searchManager.search('TypeScript');
257
- ```
258
-
259
- ### Ranked Search (TF-IDF)
260
-
261
- ```typescript
262
- const ranked = await ctx.searchManager.searchRanked('JavaScript runtime', {
263
- limit: 10,
264
- minScore: 0.1
265
- });
266
- ```
267
-
268
- ### Boolean Search
269
-
270
- ```typescript
271
- // AND - both terms must match
272
- const results = await ctx.searchManager.booleanSearch('TypeScript AND runtime');
273
-
274
- // OR - either term matches
275
- const results = await ctx.searchManager.booleanSearch('frontend OR backend');
276
-
277
- // NOT - exclude term
278
- const results = await ctx.searchManager.booleanSearch('JavaScript NOT browser');
279
-
280
- // Parentheses for grouping
281
- const results = await ctx.searchManager.booleanSearch('(TypeScript OR JavaScript) AND server');
282
- ```
283
-
284
- ### Fuzzy Search
285
-
286
- ```typescript
287
- // Typo-tolerant search with threshold (0-1, higher = stricter)
288
- const results = await ctx.searchManager.fuzzySearch('Typscript', {
289
- threshold: 0.7
290
- });
291
- ```
292
-
293
- ### Hybrid Search
294
-
295
- Combines three signal layers for sophisticated ranking:
296
-
297
- ```typescript
298
- const results = await ctx.searchManager.hybridSearch('programming concepts', {
299
- weights: {
300
- semantic: 0.5, // Vector similarity (requires embeddings)
301
- lexical: 0.3, // TF-IDF text matching
302
- symbolic: 0.2 // Metadata (tags, importance, type)
303
- },
304
- filters: {
305
- entityTypes: ['concept'],
306
- minImportance: 5,
307
- tags: ['programming']
308
- }
309
- });
310
- ```
311
-
312
- ## Graph Algorithms
313
-
314
- ### Path Finding
315
-
316
- ```typescript
317
- // Shortest path between entities (BFS)
318
- const path = await ctx.graphTraversal.findShortestPath('A', 'Z');
319
- // Returns: ['A', 'B', 'C', 'Z']
320
-
321
- // All paths with max depth
322
- const paths = await ctx.graphTraversal.findAllPaths('A', 'Z', { maxDepth: 5 });
323
- // Returns: [['A', 'B', 'Z'], ['A', 'C', 'D', 'Z'], ...]
324
- ```
325
-
326
- ### Centrality Analysis
327
-
328
- ```typescript
329
- // Calculate importance metrics
330
- const centrality = await ctx.graphTraversal.getCentrality({
331
- algorithm: 'pagerank' // or 'degree', 'betweenness'
332
- });
333
- // Returns: Map<string, number> with entity scores
334
- ```
335
-
336
- ### Connected Components
337
-
338
- ```typescript
339
- // Find isolated subgraphs
340
- const components = await ctx.graphTraversal.getConnectedComponents();
341
- // Returns: [['A', 'B', 'C'], ['X', 'Y'], ...]
342
- ```
343
-
344
- ### Traversal
345
-
346
- ```typescript
347
- // Breadth-first traversal
348
- await ctx.graphTraversal.bfs('startNode', (node) => {
349
- console.log('Visited:', node.name);
350
- });
351
-
352
- // Depth-first traversal
353
- await ctx.graphTraversal.dfs('startNode', (node) => {
354
- console.log('Visited:', node.name);
355
- });
356
- ```
357
-
358
- ## Agent Memory System
359
-
360
- A complete memory system for AI agents with working memory, episodic memory, decay mechanisms, and multi-agent support.
361
-
362
- ### Key Components
363
-
364
- | Component | Description |
365
- |-----------|-------------|
366
- | **AgentMemoryManager** | Unified facade for all agent memory operations |
367
- | **SessionManager** | Session lifecycle management |
368
- | **WorkingMemoryManager** | Short-term memory with promotion to long-term |
369
- | **EpisodicMemoryManager** | Timeline-based episodic memory |
370
- | **DecayEngine** | Time-based memory importance decay |
371
- | **SalienceEngine** | Context-aware memory scoring |
372
- | **MultiAgentMemoryManager** | Shared memory with visibility controls |
373
- | **ConflictResolver** | Resolution strategies for concurrent updates |
374
-
375
- ### Quick Start
376
-
377
- ```typescript
378
- import { ManagerContext } from '@danielsimonjr/memoryjs';
379
-
380
- const ctx = new ManagerContext('./memory.jsonl');
381
- const agent = ctx.agentMemory();
382
-
383
- // Start a session
384
- const session = await agent.startSession({ agentId: 'my-agent' });
385
-
386
- // Add working memory
387
- await agent.addWorkingMemory({
388
- sessionId: session.name,
389
- content: 'User prefers dark mode',
390
- importance: 7
391
- });
392
-
393
- // Create episodic memory
394
- await agent.createEpisode('Completed onboarding flow', {
395
- sessionId: session.name,
396
- importance: 8
397
- });
398
-
399
- // Retrieve context for LLM prompt
400
- const context = await agent.retrieveForContext({
401
- maxTokens: 2000,
402
- includeEpisodic: true
403
- });
404
-
405
- // End session
406
- await agent.endSession(session.name);
407
- ```
408
-
409
- ### Memory Types
410
-
411
- ```typescript
412
- type MemoryType = 'working' | 'episodic' | 'semantic' | 'procedural';
413
- ```
414
-
415
- - **Working Memory**: Short-term, session-scoped memories that may be promoted
416
- - **Episodic Memory**: Timeline-based event memories with temporal ordering
417
- - **Semantic Memory**: Long-term factual knowledge
418
- - **Procedural Memory**: Learned behaviors and patterns
419
-
420
- ### Decay System
421
-
422
- Memories naturally decay over time unless reinforced:
423
-
424
- ```typescript
425
- // Configure decay behavior
426
- const agent = ctx.agentMemory({
427
- decay: {
428
- halfLifeHours: 168, // 1 week half-life
429
- minImportance: 0.1 // Never fully forget
430
- },
431
- enableAutoDecay: true
432
- });
433
-
434
- // Reinforce important memories
435
- await agent.confirmMemory('memory_name', 0.1); // Boost confidence
436
- await agent.promoteMemory('memory_name', 'episodic'); // Promote to long-term
437
- ```
438
-
439
- ### Multi-Agent Support
440
-
441
- ```typescript
442
- // Register agents
443
- agent.registerAgent('agent_1', {
444
- name: 'Research Agent',
445
- type: 'llm',
446
- trustLevel: 0.8,
447
- capabilities: ['read', 'write']
448
- });
449
-
450
- // Create memories with visibility controls
451
- await agent.addWorkingMemory({
452
- sessionId: session.name,
453
- content: 'Shared insight',
454
- visibility: 'shared', // 'private' | 'shared' | 'public'
455
- ownerAgentId: 'agent_1'
456
- });
457
-
458
- // Cross-agent search
459
- const results = await agent.searchCrossAgent('agent_2', 'query');
460
- ```
461
-
462
- ## API Reference
463
-
464
- ### EntityManager
465
-
466
- | Method | Description |
467
- |--------|-------------|
468
- | `createEntities(entities)` | Create multiple entities |
469
- | `deleteEntities(names)` | Delete entities by name |
470
- | `getEntityByName(name)` | Get single entity |
471
- | `addObservations(name, observations)` | Add observations to entity |
472
- | `deleteObservations(name, observations)` | Remove specific observations |
473
- | `addTags(name, tags)` | Add tags to entity |
474
- | `removeTags(name, tags)` | Remove tags from entity |
475
- | `setImportance(name, score)` | Set importance (0-10) |
476
- | `setEntityParent(name, parentName)` | Set/remove parent |
477
- | `getChildren(name)` | Get immediate children |
478
- | `getAncestors(name)` | Get ancestor chain |
479
- | `getDescendants(name)` | Get all descendants |
480
-
481
- ### RelationManager
482
-
483
- | Method | Description |
484
- |--------|-------------|
485
- | `createRelations(relations)` | Create multiple relations |
486
- | `getRelations(entityName)` | Get incoming/outgoing relations |
487
- | `deleteRelations(relations)` | Delete specific relations |
488
-
489
- ### SearchManager
490
-
491
- | Method | Description |
492
- |--------|-------------|
493
- | `search(query, options)` | Basic substring search |
494
- | `searchRanked(query, options)` | TF-IDF ranked search |
495
- | `booleanSearch(query, options)` | Boolean operators (AND/OR/NOT) |
496
- | `fuzzySearch(query, options)` | Levenshtein-based typo tolerance |
497
- | `hybridSearch(query, options)` | Multi-signal search |
498
- | `smartSearch(query, options)` | AI-assisted refinement |
499
-
500
- ### IOManager
501
-
502
- | Method | Description |
503
- |--------|-------------|
504
- | `exportGraph(format, options)` | Export to JSON/CSV/GraphML |
505
- | `importGraph(format, data, options)` | Import with merge strategies |
506
- | `createBackup(options)` | Create timestamped backup |
507
- | `restoreBackup(path)` | Restore from backup |
508
-
509
- ### GraphTraversal
510
-
511
- | Method | Description |
512
- |--------|-------------|
513
- | `findShortestPath(from, to)` | BFS shortest path |
514
- | `findAllPaths(from, to, options)` | All paths with max depth |
515
- | `getCentrality(options)` | Centrality metrics |
516
- | `getConnectedComponents()` | Find isolated subgraphs |
517
- | `bfs(start, visitor)` | Breadth-first traversal |
518
- | `dfs(start, visitor)` | Depth-first traversal |
519
-
520
- ## Configuration
521
-
522
- ### Environment Variables
523
-
524
- | Variable | Description | Default |
525
- |----------|-------------|---------|
526
- | `MEMORY_STORAGE_TYPE` | Storage backend: `jsonl` or `sqlite` | `jsonl` |
527
- | `EMBEDDING_PROVIDER` | Embedding provider: `openai`, `local`, or `none` | `none` |
528
- | `OPENAI_API_KEY` | OpenAI API key (required if provider is `openai`) | - |
529
-
530
- ## Development
531
-
532
- ### Prerequisites
533
-
534
- - Node.js 18+
535
- - npm 9+
536
- - TypeScript 5.0+
537
-
538
- ### Build Commands
539
-
540
- ```bash
541
- npm install # Install dependencies
542
- npm run build # Build TypeScript to dist/
543
- npm run build:watch # Watch mode compilation
544
- npm test # Run all tests
545
- npm run test:watch # Watch mode testing
546
- npm run test:coverage # Run with coverage report
547
- npm run typecheck # Type checking without emit
548
- ```
549
-
550
- ### Architecture
551
-
552
- ```
553
- ┌─────────────────────────────────────────────────────────────┐
554
- │ Layer 1: ManagerContext (Central Facade) │
555
- │ ┌───────────────────────────────────────────────────────┐ │
556
- │ │ Lazy-initialized access to all managers │ │
557
- │ └───────────────────────────────────────────────────────┘ │
558
- └──────────────────────────┬──────────────────────────────────┘
559
-
560
- ┌──────────────────────────┴──────────────────────────────────┐
561
- │ Layer 2: Specialized Managers │
562
- │ • EntityManager (CRUD + hierarchy + archive) │
563
- │ • RelationManager (relation CRUD) │
564
- │ • SearchManager (search + compression + analytics) │
565
- │ • IOManager (import + export + backup) │
566
- │ • TagManager (tag aliases) │
567
- │ • GraphTraversal (path finding, centrality) │
568
- │ • SemanticSearch (embeddings, similarity) │
569
- └──────────────────────────┬──────────────────────────────────┘
570
-
571
- ┌──────────────────────────┴──────────────────────────────────┐
572
- │ Layer 3: Storage Layer │
573
- │ GraphStorage (JSONL) or SQLiteStorage (better-sqlite3) │
574
- └─────────────────────────────────────────────────────────────┘
575
- ```
576
-
577
- ### Project Structure
578
-
579
- ```
580
- memoryjs/
581
- ├── src/ # Source (93 TypeScript files)
582
- ├── index.ts # Entry point
583
- ├── agent/ # Agent Memory System (19 files)
584
- ├── AgentMemoryManager.ts # Unified facade
585
- │ │ ├── SessionManager.ts # Session lifecycle
586
- │ ├── WorkingMemoryManager.ts # Working memory
587
- │ │ ├── EpisodicMemoryManager.ts # Episodic memory
588
- │ ├── DecayEngine.ts # Memory decay
589
- │ ├── SalienceEngine.ts # Context scoring
590
- │ ├── MultiAgentMemoryManager.ts # Multi-agent support
591
- │ ├── ConflictResolver.ts # Conflict resolution
592
- └── ...
593
- ├── core/ # Core managers (12 files)
594
- │ ├── ManagerContext.ts # Context holder (lazy init)
595
- │ ├── EntityManager.ts # Entity CRUD + hierarchy
596
- │ │ ├── RelationManager.ts # Relation CRUD
597
- │ ├── GraphStorage.ts # JSONL I/O + caching
598
- │ │ ├── SQLiteStorage.ts # SQLite with better-sqlite3
599
- │ ├── TransactionManager.ts # ACID transactions
600
- └── ...
601
- │ ├── search/ # Search implementations (29 files)
602
- │ │ ├── SearchManager.ts # Search orchestrator
603
- │ │ ├── BasicSearch.ts # Text matching
604
- │ │ ├── RankedSearch.ts # TF-IDF scoring
605
- │ │ ├── BooleanSearch.ts # AND/OR/NOT logic
606
- │ │ ├── FuzzySearch.ts # Typo tolerance
607
- │ │ ├── SemanticSearch.ts # Embedding-based
608
- │ │ ├── HybridSearchManager.ts # Multi-layer search
609
- │ └── ...
610
- │ ├── features/ # Advanced capabilities (9 files)
611
- │ │ ├── IOManager.ts # Import/export/backup
612
- │ │ ├── TagManager.ts # Tag aliases
613
- │ │ ├── ArchiveManager.ts # Entity archival
614
- │ │ ├── CompressionManager.ts # Duplicate detection
615
- │ │ └── ...
616
- │ ├── types/ # TypeScript definitions (3 files)
617
- │ ├── utils/ # Shared utilities (18 files)
618
- └── workers/ # Worker pool (2 files)
619
- ├── tests/ # Test suite (3600+ tests)
620
- │ ├── unit/ # Unit tests
621
- │ ├── integration/ # Integration tests
622
- └── performance/ # Benchmarks
623
- ├── docs/ # Documentation
624
- └── architecture/ # Architecture docs
625
- ├── tools/ # Development utilities
626
- │ ├── chunking-for-files/ # File splitting tool
627
- └── create-dependency-graph/ # Dependency analyzer
628
- └── README.md # This file
629
- ```
630
-
631
- ## Documentation
632
-
633
- Comprehensive architecture documentation in `docs/architecture/`:
634
-
635
- - [OVERVIEW.md](docs/architecture/OVERVIEW.md) - High-level project overview
636
- - [ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md) - Technical architecture and design
637
- - [COMPONENTS.md](docs/architecture/COMPONENTS.md) - Component breakdown
638
- - [DATAFLOW.md](docs/architecture/DATAFLOW.md) - Data flow patterns
639
- - [API.md](docs/architecture/API.md) - Complete API documentation
640
- - [DEPENDENCY_GRAPH.md](docs/architecture/DEPENDENCY_GRAPH.md) - Module dependencies
641
-
642
- ## License
643
-
644
- **MIT License** - see [LICENSE](LICENSE)
645
-
646
- ## Related
647
-
648
- - [@danielsimonjr/memory-mcp](https://github.com/danielsimonjr/memory-mcp) - MCP server built on this library
649
-
650
- ---
651
-
652
- **Repository:** https://github.com/danielsimonjr/memoryjs
653
- **NPM:** https://www.npmjs.com/package/@danielsimonjr/memoryjs
654
- **Issues:** https://github.com/danielsimonjr/memoryjs/issues
1
+ # MemoryJS
2
+
3
+ [![Version](https://img.shields.io/badge/version-1.8.0-blue.svg)](https://github.com/danielsimonjr/memoryjs)
4
+ [![NPM](https://img.shields.io/npm/v/@danielsimonjr/memoryjs.svg)](https://www.npmjs.com/package/@danielsimonjr/memoryjs)
5
+ [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
6
+ [![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
7
+
8
+ A **TypeScript knowledge graph library** for managing entities, relations, and observations with **advanced search capabilities**, **hierarchical organization**, and **multiple storage backends**.
9
+
10
+ > **Core library** powering [@danielsimonjr/memory-mcp](https://www.npmjs.com/package/@danielsimonjr/memory-mcp). Provides **113 TypeScript files**, **~50K lines of code**, dual storage backends (JSONL/SQLite), sophisticated search algorithms (BM25, TF-IDF, fuzzy, semantic, hybrid, temporal, LLM-planned), and a complete **Agent Memory System** for AI agents with role profiles, entropy filtering, recursive consolidation, collaborative synthesis, failure distillation, cognitive load analysis, and shared visibility hierarchies.
11
+
12
+ ## Table of Contents
13
+
14
+ - [Features](#features)
15
+ - [Installation](#installation)
16
+ - [Quick Start](#quick-start)
17
+ - [Core Concepts](#core-concepts)
18
+ - [Storage Options](#storage-options)
19
+ - [Search Capabilities](#search-capabilities)
20
+ - [Graph Algorithms](#graph-algorithms)
21
+ - [Agent Memory System](#agent-memory-system)
22
+ - [API Reference](#api-reference)
23
+ - [Configuration](#configuration)
24
+ - [Development](#development)
25
+ - [Documentation](#documentation)
26
+ - [License](#license)
27
+
28
+ ## Features
29
+
30
+ ### Core Capabilities
31
+
32
+ - **Knowledge Graph Storage**: Entity-Relation-Observation model for structured data
33
+ - **Dual Storage Backends**: JSONL (human-readable) or SQLite (FTS5, 3-10x faster)
34
+ - **Full CRUD Operations**: Create, read, update, delete entities and relations
35
+ - **Hierarchical Nesting**: Parent-child relationships for tree structures
36
+ - **Timestamps**: Automatic createdAt and lastModified tracking
37
+
38
+ ### Advanced Features
39
+
40
+ | Category | Description |
41
+ |----------|-------------|
42
+ | **Search Algorithms** | Basic, TF-IDF ranked, BM25, Boolean (AND/OR/NOT), Fuzzy (Levenshtein + N-gram pre-filter), Semantic (embeddings), Hybrid |
43
+ | **Graph Algorithms** | Shortest path (BFS), all paths, centrality metrics (degree, betweenness, PageRank), connected components |
44
+ | **Hierarchical Nesting** | Parent-child relationships, ancestor/descendant traversal, subtree operations |
45
+ | **Duplicate Detection** | Intelligent compression with similarity scoring |
46
+ | **Tag Management** | Tags, aliases, bulk operations, importance scores (0-10) |
47
+ | **Import/Export** | JSON, CSV, GraphML, GEXF, DOT, Markdown, Mermaid formats with Brotli compression |
48
+ | **Analytics** | Graph statistics, validation, integrity checks |
49
+ | **Temporal Queries** | Natural language time parsing ("last hour", "10 minutes ago") via `searchByTime()` and `ManagerContext.temporalSearch` |
50
+ | **Memory Distillation** | Post-retrieval policy filter (relevance + freshness + dedup) wired into `ContextWindowManager` |
51
+ | **Freshness Auditing** | `Entity.ttl` / `Entity.confidence`, `FreshnessManager` reports, TTL-aware decay and salience weighting |
52
+ | **N-gram Search** | Trigram index with Jaccard pre-filtering reduces Levenshtein candidate set in `FuzzySearch` |
53
+ | **LLM Query Planner** | Optional natural language → `StructuredQuery` decomposition via `LLMProvider`; `ManagerContext.queryNaturalLanguage()` |
54
+ | **Governance & Audit** | `AuditLog` (JSONL), `GovernanceManager` (transactions/rollback), `GovernancePolicy` (canCreate/canUpdate/canDelete) |
55
+
56
+ ### Module Statistics
57
+
58
+ | Module | Files | Key Components |
59
+ |--------|-------|----------------|
60
+ | `agent/` | 30 | AgentMemoryManager, SessionManager, DecayEngine, WorkingMemoryManager, ArtifactManager, DistillationPolicy, DistillationPipeline, RoleProfiles, EntropyFilter, ConsolidationScheduler, MemoryFormatter, CollaborativeSynthesis, FailureDistillation, CognitiveLoadAnalyzer, VisibilityResolver |
61
+ | `core/` | 13 | EntityManager, GraphStorage, SQLiteStorage, TransactionManager, RefIndex |
62
+ | `search/` | 34 | SearchManager, BM25Search, HybridScorer, VectorStore, QueryPlanner, TemporalQueryParser, TemporalSearch, NGramIndex, LLMQueryPlanner, LLMSearchExecutor |
63
+ | `features/` | 12 | IOManager, ArchiveManager, CompressionManager, StreamingExporter, FreshnessManager, AuditLog, GovernanceManager |
64
+ | `utils/` | 18 | BatchProcessor, CompressedCache, WorkerPoolManager, MemoryMonitor |
65
+ | `types/` | 4 | Entity, Relation, AgentEntity, SessionEntity, ArtifactEntity interfaces |
66
+ | `workers/` | 2 | Levenshtein distance calculations |
67
+
68
+ **Total:** 113 TypeScript files | ~50,000 lines of code | 720+ exports | 99+ classes | 240+ interfaces
69
+
70
+ ### New in 1.7.0
71
+
72
+ | Feature | Entry Point |
73
+ |---------|-------------|
74
+ | Role-Aware Memory Customization | `RoleProfileManager.apply(role)` — salience weights + budget splits |
75
+ | Entropy-Aware Filtering | `EntropyFilter` — Shannon entropy gate in `ConsolidationPipeline` |
76
+ | Recursive Memory Consolidation | `ConsolidationScheduler` — background dedup + merge to fixed point |
77
+ | Visual Salience Budget Allocation | `MemoryFormatter.formatWithSalienceBudget()` |
78
+ | Collaborative Memory Synthesis | `CollaborativeSynthesis.synthesize(entity, hopDepth)` |
79
+ | Failure-Driven Memory Distillation | `FailureDistillation.distill(failureEntity)` |
80
+ | Cognitive Load Metrics | `CognitiveLoadAnalyzer.analyze(memories)` → `CognitiveLoadReport` |
81
+ | Shared Memory Visibility Hierarchies | `VisibilityResolver.resolve(agentId, memories)` — 5-level model |
82
+
83
+ ### New in 1.6.0
84
+
85
+ | Feature | Entry Point |
86
+ |---------|-------------|
87
+ | Stable Index Dereferencing | `ctx.refIndex` — `register` / `resolve` / `deregister` |
88
+ | Artifact-Level Granularity | `ctx.agentMemory().artifactManager.createArtifact()` |
89
+ | Temporal Range Queries | `ctx.searchManager.searchByTime()` / `ctx.temporalSearch` |
90
+ | Memory Distillation Policy | `IDistillationPolicy` — wired into `ContextWindowManager` |
91
+ | Temporal Governance & Freshness | `ctx.freshnessManager` — TTL, confidence, staleness report |
92
+ | N-gram Hashing | Automatic — `FuzzySearch` now pre-filters via `NGramIndex` |
93
+ | LLM Query Planner | `ctx.queryNaturalLanguage(query, llmProvider?)` |
94
+ | Dynamic Memory Governance | `ctx.governanceManager` `withTransaction` / `GovernancePolicy` |
95
+
96
+ ## Installation
97
+
98
+ ```bash
99
+ npm install @danielsimonjr/memoryjs
100
+ ```
101
+
102
+ ### Requirements
103
+
104
+ - Node.js >= 18.0.0
105
+ - TypeScript >= 5.0 (for development)
106
+
107
+ ## Quick Start
108
+
109
+ ### 1. Initialize Storage
110
+
111
+ ```typescript
112
+ import { ManagerContext } from '@danielsimonjr/memoryjs';
113
+
114
+ // JSONL storage (default, human-readable)
115
+ const ctx = new ManagerContext('./memory.jsonl');
116
+
117
+ // Or SQLite storage (set MEMORY_STORAGE_TYPE=sqlite env var)
118
+ const ctx = new ManagerContext('./memory.db');
119
+ ```
120
+
121
+ ### 2. Create Entities
122
+
123
+ ```typescript
124
+ await ctx.entityManager.createEntities([
125
+ {
126
+ name: 'TypeScript',
127
+ entityType: 'language',
128
+ observations: ['A typed superset of JavaScript'],
129
+ tags: ['programming', 'frontend'],
130
+ importance: 8
131
+ },
132
+ {
133
+ name: 'Node.js',
134
+ entityType: 'runtime',
135
+ observations: ['JavaScript runtime built on V8'],
136
+ tags: ['backend', 'server']
137
+ }
138
+ ]);
139
+ ```
140
+
141
+ ### 3. Create Relations
142
+
143
+ ```typescript
144
+ await ctx.relationManager.createRelations([
145
+ { from: 'TypeScript', to: 'Node.js', relationType: 'runs_on' }
146
+ ]);
147
+ ```
148
+
149
+ ### 4. Search
150
+
151
+ ```typescript
152
+ // Basic search
153
+ const results = await ctx.searchManager.search('JavaScript');
154
+
155
+ // Ranked search (TF-IDF scoring)
156
+ const ranked = await ctx.searchManager.searchRanked('runtime environment', { limit: 10 });
157
+
158
+ // Boolean search
159
+ const filtered = await ctx.searchManager.booleanSearch('TypeScript AND runtime');
160
+
161
+ // Fuzzy search (typo-tolerant)
162
+ const fuzzy = await ctx.searchManager.fuzzySearch('Typscript', { threshold: 0.7 });
163
+ ```
164
+
165
+ ## Core Concepts
166
+
167
+ ### Entities
168
+
169
+ Primary nodes in the knowledge graph.
170
+
171
+ ```typescript
172
+ interface Entity {
173
+ name: string; // Unique identifier
174
+ entityType: string; // Classification (person, project, concept)
175
+ observations: string[]; // Facts about the entity
176
+ parentId?: string; // Parent entity for hierarchical nesting
177
+ tags?: string[]; // Lowercase tags for categorization
178
+ importance?: number; // 0-10 scale for prioritization
179
+ createdAt?: string; // ISO 8601 timestamp
180
+ lastModified?: string; // ISO 8601 timestamp
181
+ }
182
+ ```
183
+
184
+ ### Relations
185
+
186
+ Directed connections between entities.
187
+
188
+ ```typescript
189
+ interface Relation {
190
+ from: string; // Source entity name
191
+ to: string; // Target entity name
192
+ relationType: string; // Relationship type (active voice)
193
+ }
194
+ ```
195
+
196
+ ### Observations
197
+
198
+ Discrete facts about entities. Each observation should be atomic and independently manageable. Use `addObservations()` to append new facts without overwriting existing ones.
199
+
200
+ ### ManagerContext
201
+
202
+ Central access point for all managers with lazy initialization:
203
+
204
+ ```typescript
205
+ ctx.entityManager // Entity CRUD + hierarchy
206
+ ctx.relationManager // Relation management
207
+ ctx.searchManager // All search operations
208
+ ctx.tagManager // Tag aliases and bulk operations
209
+ ctx.ioManager // Import/export/backup
210
+ ctx.graphTraversal // Graph algorithms
211
+ ctx.archiveManager // Entity archival
212
+ ctx.analyticsManager // Graph statistics and validation
213
+ ctx.compressionManager // Duplicate detection, entity merging
214
+ ctx.semanticSearch // Vector similarity search (lazy, optional)
215
+ ctx.accessTracker // Memory access tracking (lazy)
216
+ ```
217
+
218
+ ## Storage Options
219
+
220
+ ### Comparison
221
+
222
+ | Feature | JSONL (Default) | SQLite (better-sqlite3) |
223
+ |---------|-----------------|-------------------------|
224
+ | Format | Human-readable text | Native binary database |
225
+ | Transactions | Basic | Full ACID with WAL mode |
226
+ | Full-Text Search | Basic | FTS5 with BM25 ranking |
227
+ | Performance | Good | 3-10x faster |
228
+ | Concurrency | Single-threaded | Thread-safe with async-mutex |
229
+ | Best For | Small graphs, debugging | Large graphs (10k+ entities) |
230
+
231
+ ### JSONL Storage
232
+
233
+ ```typescript
234
+ const ctx = new ManagerContext('./memory.jsonl');
235
+ ```
236
+
237
+ Features:
238
+ - Human-readable line-delimited JSON
239
+ - In-memory caching with write-through invalidation
240
+ - Atomic writes via temp file + rename
241
+ - Backward compatibility for legacy formats
242
+
243
+ ### SQLite Storage
244
+
245
+ ```typescript
246
+ // Set MEMORY_STORAGE_TYPE=sqlite environment variable
247
+ const ctx = new ManagerContext('./memory.db');
248
+ ```
249
+
250
+ Features:
251
+ - FTS5 full-text search with BM25 ranking
252
+ - WAL mode for better concurrency
253
+ - Referential integrity with ON DELETE CASCADE
254
+ - ACID transactions
255
+
256
+ ### Storage Files
257
+
258
+ When using JSONL, related files are automatically created:
259
+
260
+ ```
261
+ /your/data/directory/
262
+ ├── memory.jsonl # Main knowledge graph
263
+ ├── memory-saved-searches.jsonl # Saved search queries
264
+ ├── memory-tag-aliases.jsonl # Tag synonym mappings
265
+ └── .backups/ # Timestamped backups
266
+ ```
267
+
268
+ ## Search Capabilities
269
+
270
+ ### Search Methods
271
+
272
+ | Method | Description | Use Case |
273
+ |--------|-------------|----------|
274
+ | `search()` | Basic substring matching | Simple queries |
275
+ | `searchRanked()` | TF-IDF relevance scoring | Finding most relevant results |
276
+ | `booleanSearch()` | AND/OR/NOT operators | Complex filtering |
277
+ | `fuzzySearch()` | Levenshtein distance | Typo tolerance |
278
+ | `hybridSearch()` | Semantic + lexical + symbolic | Multi-signal ranking |
279
+
280
+ ### Basic Search
281
+
282
+ ```typescript
283
+ const results = await ctx.searchManager.search('TypeScript');
284
+ ```
285
+
286
+ ### Ranked Search (TF-IDF)
287
+
288
+ ```typescript
289
+ const ranked = await ctx.searchManager.searchRanked('JavaScript runtime', {
290
+ limit: 10,
291
+ minScore: 0.1
292
+ });
293
+ ```
294
+
295
+ ### Boolean Search
296
+
297
+ ```typescript
298
+ // AND - both terms must match
299
+ const results = await ctx.searchManager.booleanSearch('TypeScript AND runtime');
300
+
301
+ // OR - either term matches
302
+ const results = await ctx.searchManager.booleanSearch('frontend OR backend');
303
+
304
+ // NOT - exclude term
305
+ const results = await ctx.searchManager.booleanSearch('JavaScript NOT browser');
306
+
307
+ // Parentheses for grouping
308
+ const results = await ctx.searchManager.booleanSearch('(TypeScript OR JavaScript) AND server');
309
+ ```
310
+
311
+ ### Fuzzy Search
312
+
313
+ ```typescript
314
+ // Typo-tolerant search with threshold (0-1, higher = stricter)
315
+ const results = await ctx.searchManager.fuzzySearch('Typscript', {
316
+ threshold: 0.7
317
+ });
318
+ ```
319
+
320
+ ### Hybrid Search
321
+
322
+ Combines three signal layers for sophisticated ranking:
323
+
324
+ ```typescript
325
+ const results = await ctx.searchManager.hybridSearch('programming concepts', {
326
+ weights: {
327
+ semantic: 0.5, // Vector similarity (requires embeddings)
328
+ lexical: 0.3, // TF-IDF text matching
329
+ symbolic: 0.2 // Metadata (tags, importance, type)
330
+ },
331
+ filters: {
332
+ entityTypes: ['concept'],
333
+ minImportance: 5,
334
+ tags: ['programming']
335
+ }
336
+ });
337
+ ```
338
+
339
+ ## Graph Algorithms
340
+
341
+ ### Path Finding
342
+
343
+ ```typescript
344
+ // Shortest path between entities (BFS)
345
+ const path = await ctx.graphTraversal.findShortestPath('A', 'Z');
346
+ // Returns: ['A', 'B', 'C', 'Z']
347
+
348
+ // All paths with max depth
349
+ const paths = await ctx.graphTraversal.findAllPaths('A', 'Z', { maxDepth: 5 });
350
+ // Returns: [['A', 'B', 'Z'], ['A', 'C', 'D', 'Z'], ...]
351
+ ```
352
+
353
+ ### Centrality Analysis
354
+
355
+ ```typescript
356
+ // Calculate importance metrics
357
+ const centrality = await ctx.graphTraversal.getCentrality({
358
+ algorithm: 'pagerank' // or 'degree', 'betweenness'
359
+ });
360
+ // Returns: Map<string, number> with entity scores
361
+ ```
362
+
363
+ ### Connected Components
364
+
365
+ ```typescript
366
+ // Find isolated subgraphs
367
+ const components = await ctx.graphTraversal.getConnectedComponents();
368
+ // Returns: [['A', 'B', 'C'], ['X', 'Y'], ...]
369
+ ```
370
+
371
+ ### Traversal
372
+
373
+ ```typescript
374
+ // Breadth-first traversal
375
+ await ctx.graphTraversal.bfs('startNode', (node) => {
376
+ console.log('Visited:', node.name);
377
+ });
378
+
379
+ // Depth-first traversal
380
+ await ctx.graphTraversal.dfs('startNode', (node) => {
381
+ console.log('Visited:', node.name);
382
+ });
383
+ ```
384
+
385
+ ## Agent Memory System
386
+
387
+ A complete memory system for AI agents with working memory, episodic memory, decay mechanisms, and multi-agent support.
388
+
389
+ ### Key Components
390
+
391
+ | Component | Description |
392
+ |-----------|-------------|
393
+ | **AgentMemoryManager** | Unified facade for all agent memory operations |
394
+ | **SessionManager** | Session lifecycle management |
395
+ | **WorkingMemoryManager** | Short-term memory with promotion to long-term |
396
+ | **EpisodicMemoryManager** | Timeline-based episodic memory |
397
+ | **DecayEngine** | Time-based memory importance decay |
398
+ | **SalienceEngine** | Context-aware memory scoring |
399
+ | **MultiAgentMemoryManager** | Shared memory with visibility controls |
400
+ | **ConflictResolver** | Resolution strategies for concurrent updates |
401
+
402
+ ### Quick Start
403
+
404
+ ```typescript
405
+ import { ManagerContext } from '@danielsimonjr/memoryjs';
406
+
407
+ const ctx = new ManagerContext('./memory.jsonl');
408
+ const agent = ctx.agentMemory();
409
+
410
+ // Start a session
411
+ const session = await agent.startSession({ agentId: 'my-agent' });
412
+
413
+ // Add working memory
414
+ await agent.addWorkingMemory({
415
+ sessionId: session.name,
416
+ content: 'User prefers dark mode',
417
+ importance: 7
418
+ });
419
+
420
+ // Create episodic memory
421
+ await agent.createEpisode('Completed onboarding flow', {
422
+ sessionId: session.name,
423
+ importance: 8
424
+ });
425
+
426
+ // Retrieve context for LLM prompt
427
+ const context = await agent.retrieveForContext({
428
+ maxTokens: 2000,
429
+ includeEpisodic: true
430
+ });
431
+
432
+ // End session
433
+ await agent.endSession(session.name);
434
+ ```
435
+
436
+ ### Memory Types
437
+
438
+ ```typescript
439
+ type MemoryType = 'working' | 'episodic' | 'semantic' | 'procedural';
440
+ ```
441
+
442
+ - **Working Memory**: Short-term, session-scoped memories that may be promoted
443
+ - **Episodic Memory**: Timeline-based event memories with temporal ordering
444
+ - **Semantic Memory**: Long-term factual knowledge
445
+ - **Procedural Memory**: Learned behaviors and patterns
446
+
447
+ ### Decay System
448
+
449
+ Memories naturally decay over time unless reinforced:
450
+
451
+ ```typescript
452
+ // Configure decay behavior
453
+ const agent = ctx.agentMemory({
454
+ decay: {
455
+ halfLifeHours: 168, // 1 week half-life
456
+ minImportance: 0.1 // Never fully forget
457
+ },
458
+ enableAutoDecay: true
459
+ });
460
+
461
+ // Reinforce important memories
462
+ await agent.confirmMemory('memory_name', 0.1); // Boost confidence
463
+ await agent.promoteMemory('memory_name', 'episodic'); // Promote to long-term
464
+ ```
465
+
466
+ ### Multi-Agent Support
467
+
468
+ ```typescript
469
+ // Register agents
470
+ agent.registerAgent('agent_1', {
471
+ name: 'Research Agent',
472
+ type: 'llm',
473
+ trustLevel: 0.8,
474
+ capabilities: ['read', 'write']
475
+ });
476
+
477
+ // Create memories with visibility controls
478
+ await agent.addWorkingMemory({
479
+ sessionId: session.name,
480
+ content: 'Shared insight',
481
+ visibility: 'shared', // 'private' | 'shared' | 'public'
482
+ ownerAgentId: 'agent_1'
483
+ });
484
+
485
+ // Cross-agent search
486
+ const results = await agent.searchCrossAgent('agent_2', 'query');
487
+ ```
488
+
489
+ ## API Reference
490
+
491
+ ### EntityManager
492
+
493
+ | Method | Description |
494
+ |--------|-------------|
495
+ | `createEntities(entities)` | Create multiple entities |
496
+ | `deleteEntities(names)` | Delete entities by name |
497
+ | `getEntityByName(name)` | Get single entity |
498
+ | `addObservations(name, observations)` | Add observations to entity |
499
+ | `deleteObservations(name, observations)` | Remove specific observations |
500
+ | `addTags(name, tags)` | Add tags to entity |
501
+ | `removeTags(name, tags)` | Remove tags from entity |
502
+ | `setImportance(name, score)` | Set importance (0-10) |
503
+ | `setEntityParent(name, parentName)` | Set/remove parent |
504
+ | `getChildren(name)` | Get immediate children |
505
+ | `getAncestors(name)` | Get ancestor chain |
506
+ | `getDescendants(name)` | Get all descendants |
507
+
508
+ ### RelationManager
509
+
510
+ | Method | Description |
511
+ |--------|-------------|
512
+ | `createRelations(relations)` | Create multiple relations |
513
+ | `getRelations(entityName)` | Get incoming/outgoing relations |
514
+ | `deleteRelations(relations)` | Delete specific relations |
515
+
516
+ ### SearchManager
517
+
518
+ | Method | Description |
519
+ |--------|-------------|
520
+ | `search(query, options)` | Basic substring search |
521
+ | `searchRanked(query, options)` | TF-IDF ranked search |
522
+ | `booleanSearch(query, options)` | Boolean operators (AND/OR/NOT) |
523
+ | `fuzzySearch(query, options)` | Levenshtein-based typo tolerance |
524
+ | `hybridSearch(query, options)` | Multi-signal search |
525
+ | `autoSearch(query, limit?)` | Auto-select best search method |
526
+
527
+ ### IOManager
528
+
529
+ | Method | Description |
530
+ |--------|-------------|
531
+ | `exportGraph(format, options)` | Export to JSON/CSV/GraphML/GEXF/DOT/Markdown/Mermaid |
532
+ | `importGraph(format, data, options)` | Import with merge strategies |
533
+ | `createBackup(options)` | Create timestamped backup |
534
+ | `restoreBackup(path)` | Restore from backup |
535
+
536
+ ### GraphTraversal
537
+
538
+ | Method | Description |
539
+ |--------|-------------|
540
+ | `findShortestPath(from, to)` | BFS shortest path |
541
+ | `findAllPaths(from, to, options)` | All paths with max depth |
542
+ | `getCentrality(options)` | Centrality metrics |
543
+ | `getConnectedComponents()` | Find isolated subgraphs |
544
+ | `bfs(start, visitor)` | Breadth-first traversal |
545
+ | `dfs(start, visitor)` | Depth-first traversal |
546
+
547
+ ## Configuration
548
+
549
+ ### Environment Variables
550
+
551
+ | Variable | Description | Default |
552
+ |----------|-------------|---------|
553
+ | `MEMORY_STORAGE_TYPE` | Storage backend: `jsonl` or `sqlite` | `jsonl` |
554
+ | `MEMORY_EMBEDDING_PROVIDER` | Embedding provider: `openai`, `local`, or `none` | `none` |
555
+ | `MEMORY_OPENAI_API_KEY` | OpenAI API key (required if provider is `openai`) | - |
556
+
557
+ ## Development
558
+
559
+ ### Prerequisites
560
+
561
+ - Node.js 18+
562
+ - npm 9+
563
+ - TypeScript 5.0+
564
+
565
+ ### Build Commands
566
+
567
+ ```bash
568
+ npm install # Install dependencies
569
+ npm run build # Build TypeScript to dist/
570
+ npm run build:watch # Watch mode compilation
571
+ npm test # Run all tests
572
+ npm run test:watch # Watch mode testing
573
+ npm run test:coverage # Run with coverage report
574
+ npm run typecheck # Type checking without emit
575
+ ```
576
+
577
+ ### Architecture
578
+
579
+ ```
580
+ ┌─────────────────────────────────────────────────────────────┐
581
+ │ Layer 1: ManagerContext (Central Facade)
582
+ ┌───────────────────────────────────────────────────────┐ │
583
+ Lazy-initialized access to all managers │ │
584
+ └───────────────────────────────────────────────────────┘
585
+ └──────────────────────────┬──────────────────────────────────┘
586
+
587
+ ┌──────────────────────────┴──────────────────────────────────┐
588
+ Layer 2: Specialized Managers │
589
+ EntityManager (CRUD + hierarchy + archive) │
590
+ RelationManager (relation CRUD) │
591
+ SearchManager (search + compression + analytics) │
592
+ • IOManager (import + export + backup)
593
+ TagManager (tag aliases)
594
+ GraphTraversal (path finding, centrality)
595
+ SemanticSearch (embeddings, similarity) │
596
+ └──────────────────────────┬──────────────────────────────────┘
597
+
598
+ ┌──────────────────────────┴──────────────────────────────────┐
599
+ Layer 3: Storage Layer │
600
+ GraphStorage (JSONL) or SQLiteStorage (better-sqlite3)
601
+ └─────────────────────────────────────────────────────────────┘
602
+ ```
603
+
604
+ ### Project Structure
605
+
606
+ ```
607
+ memoryjs/
608
+ ├── src/ # Source (113 TypeScript files)
609
+ ├── index.ts # Entry point
610
+ │ ├── agent/ # Agent Memory System (30 files)
611
+ │ │ ├── AgentMemoryManager.ts # Unified facade
612
+ │ │ ├── SessionManager.ts # Session lifecycle
613
+ │ │ ├── WorkingMemoryManager.ts # Working memory
614
+ │ │ ├── EpisodicMemoryManager.ts # Episodic memory
615
+ │ │ ├── DecayEngine.ts # Memory decay (TTL-aware)
616
+ ├── SalienceEngine.ts # Context scoring (freshnessWeight)
617
+ ├── MultiAgentMemoryManager.ts # Multi-agent support
618
+ │ ├── ConflictResolver.ts # Conflict resolution
619
+ │ │ ├── ArtifactManager.ts # Artifact creation + stable refs
620
+ ├── DistillationPolicy.ts # Post-retrieval distillation policies
621
+ ├── DistillationPipeline.ts # Distillation pipeline executor
622
+ │ ├── RoleProfiles.ts # Role profiles + salience weight presets
623
+ │ │ ├── EntropyFilter.ts # Shannon entropy gate for consolidation
624
+ │ ├── ConsolidationScheduler.ts # Background recursive dedup+merge
625
+ │ │ ├── MemoryFormatter.ts # formatWithSalienceBudget()
626
+ ├── CollaborativeSynthesis.ts # Graph-neighbourhood multi-agent merge
627
+ │ ├── FailureDistillation.ts # Causal chain lesson extraction
628
+ │ │ ├── CognitiveLoadAnalyzer.ts # Token density + redundancy + diversity
629
+ │ │ ├── VisibilityResolver.ts # 5-level visibility + GroupMembership
630
+ │ │ └── ...
631
+ │ ├── core/ # Core managers (13 files)
632
+ │ │ ├── ManagerContext.ts # Context holder (lazy init)
633
+ │ │ ├── EntityManager.ts # Entity CRUD + hierarchy
634
+ │ │ ├── RelationManager.ts # Relation CRUD
635
+ │ │ ├── GraphStorage.ts # JSONL I/O + caching
636
+ │ │ ├── SQLiteStorage.ts # SQLite with better-sqlite3
637
+ │ │ ├── TransactionManager.ts # ACID transactions
638
+ │ │ ├── RefIndex.ts # Named refs for O(1) entity lookup
639
+ │ │ └── ...
640
+ │ ├── search/ # Search implementations (34 files)
641
+ │ │ ├── SearchManager.ts # Search orchestrator
642
+ │ │ ├── BasicSearch.ts # Text matching
643
+ │ │ ├── RankedSearch.ts # TF-IDF scoring
644
+ │ │ ├── BooleanSearch.ts # AND/OR/NOT logic
645
+ │ │ ├── FuzzySearch.ts # Typo tolerance (NGram pre-filtered)
646
+ │ │ ├── SemanticSearch.ts # Embedding-based
647
+ │ │ ├── HybridSearchManager.ts # Multi-layer search
648
+ │ │ ├── NGramIndex.ts # Trigram index + Jaccard pre-filter
649
+ │ │ ├── TemporalQueryParser.ts # Natural language time parsing
650
+ │ │ ├── TemporalSearch.ts # Time-range search executor
651
+ │ │ ├── LLMQueryPlanner.ts # NL → StructuredQuery decomposition
652
+ │ │ ├── LLMSearchExecutor.ts # LLM-planned search execution
653
+ │ │ └── ...
654
+ │ ├── features/ # Advanced capabilities (12 files)
655
+ │ │ ├── IOManager.ts # Import/export/backup
656
+ │ │ ├── TagManager.ts # Tag aliases
657
+ │ │ ├── ArchiveManager.ts # Entity archival
658
+ │ │ ├── CompressionManager.ts # Duplicate detection
659
+ │ │ ├── FreshnessManager.ts # TTL/confidence freshness reports
660
+ │ │ ├── AuditLog.ts # JSONL immutable audit trail
661
+ │ │ ├── GovernanceManager.ts # Transactions + policy enforcement
662
+ │ │ └── ...
663
+ │ ├── cli/ # CLI interface (6 files)
664
+ │ │ ├── index.ts # CLI entry point
665
+ │ │ ├── commands/ # Command implementations
666
+ │ │ ├── config.ts # Config file support
667
+ │ │ ├── formatters.ts # Output formatting
668
+ │ │ ├── interactive.ts # REPL mode
669
+ │ │ └── options.ts # CLI option parsing
670
+ │ ├── types/ # TypeScript definitions (4 files)
671
+ │ ├── utils/ # Shared utilities (18 files)
672
+ │ └── workers/ # Worker pool (2 files)
673
+ ├── tests/ # Test suite (4674 tests)
674
+ │ ├── unit/ # Unit tests
675
+ │ ├── integration/ # Integration tests
676
+ │ └── performance/ # Benchmarks
677
+ ├── docs/ # Documentation
678
+ │ └── architecture/ # Architecture docs
679
+ ├── tools/ # Development utilities
680
+ │ ├── chunking-for-files/ # File splitting tool
681
+ │ └── create-dependency-graph/ # Dependency analyzer
682
+ └── README.md # This file
683
+ ```
684
+
685
+ ## Documentation
686
+
687
+ Comprehensive architecture documentation in `docs/architecture/`:
688
+
689
+ - [OVERVIEW.md](docs/architecture/OVERVIEW.md) - High-level project overview
690
+ - [ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md) - Technical architecture and design
691
+ - [COMPONENTS.md](docs/architecture/COMPONENTS.md) - Component breakdown
692
+ - [DATAFLOW.md](docs/architecture/DATAFLOW.md) - Data flow patterns
693
+ - [API.md](docs/architecture/API.md) - Complete API documentation
694
+ - [DEPENDENCY_GRAPH.md](docs/architecture/DEPENDENCY_GRAPH.md) - Module dependencies
695
+ - [TEST_COVERAGE.md](docs/architecture/TEST_COVERAGE.md) - Test coverage analysis
696
+ - [AGENT_MEMORY.md](docs/architecture/AGENT_MEMORY.md) - Agent memory system design
697
+
698
+ ## License
699
+
700
+ **MIT License** - see [LICENSE](LICENSE)
701
+
702
+ ## Related
703
+
704
+ - [@danielsimonjr/memory-mcp](https://github.com/danielsimonjr/memory-mcp) - MCP server built on this library
705
+
706
+ ---
707
+
708
+ **Repository:** https://github.com/danielsimonjr/memoryjs
709
+ **NPM:** https://www.npmjs.com/package/@danielsimonjr/memoryjs
710
+ **Issues:** https://github.com/danielsimonjr/memoryjs/issues