@danielsimonjr/memoryjs 2.8.1 → 2.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,1048 +1,1048 @@
1
- # MemoryJS
2
-
3
- [![NPM](https://img.shields.io/npm/v/@danielsimonjr/memoryjs.svg)](https://www.npmjs.com/package/@danielsimonjr/memoryjs)
4
- [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
5
- [![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
6
-
7
- A TypeScript knowledge-graph library for AI agents and applications that need
8
- structured long-term memory. Entities, relations, and observations with
9
- multiple storage backends (JSONL, SQLite, PostgreSQL), advanced search (BM25, TF-IDF,
10
- fuzzy, semantic, hybrid, temporal, LLM-planned), graph algorithms, bitemporal
11
- versioning, RBAC, and a complete Agent Memory System (session lifecycle,
12
- working / episodic / semantic / procedural memory, decay, salience, consolidation,
13
- causal reasoning, world-model orchestration).
14
-
15
- > Powers [@danielsimonjr/memory-mcp](https://www.npmjs.com/package/@danielsimonjr/memory-mcp),
16
- > a Model Context Protocol server for Claude and other MCP clients. The
17
- > library and the MCP server share the same core; this package is published
18
- > independently so non-MCP applications can use it directly.
19
-
20
- For a runnable CLI: `npx -p @danielsimonjr/memoryjs memory --help`.
21
-
22
- ## Table of Contents
23
-
24
- - [Features](#features)
25
- - [Installation](#installation)
26
- - [Quick Start](#quick-start)
27
- - [Core Concepts](#core-concepts)
28
- - [Storage Options](#storage-options)
29
- - [Search Capabilities](#search-capabilities)
30
- - [Graph Algorithms](#graph-algorithms)
31
- - [Agent Memory System](#agent-memory-system)
32
- - [API Reference](#api-reference)
33
- - [Configuration](#configuration)
34
- - [Development](#development)
35
- - [Documentation](#documentation)
36
- - [License](#license)
37
-
38
- ## Features
39
-
40
- ### Knowledge graph
41
-
42
- | Capability | Entry point |
43
- |---|---|
44
- | Entity / relation / observation CRUD | `ctx.entityManager`, `ctx.relationManager`, `ctx.observationManager` |
45
- | Optimistic concurrency control on updates | `entityManager.updateEntity(name, updates, { expectedVersion })` |
46
- | Bitemporal versioning (entities + observations) | `invalidateEntity()` / `entityAsOf()` / `entityTimeline()` |
47
- | Temporal relations (validity windows) | `relationManager.invalidateRelation()` / `queryAsOf()` / `timeline()` |
48
- | Memory versioning with contradiction-driven supersession | `Entity.version` / `parentEntityName` / `rootEntityName` / `supersededBy` |
49
- | Project scoping | `Entity.projectId` + `MEMORY_DEFAULT_PROJECT_ID` |
50
- | Two-tier deletion (exact match → semantic fallback) | `ctx.semanticForget` |
51
- | Hierarchical nesting + traversal | `ctx.hierarchyManager` (ancestors / descendants / subtrees) |
52
- | Stable named references that survive entity renames | `ctx.refIndex.register()` / `resolve()` |
53
- | Multi-format import / export | `ctx.ioManager.exportGraph(format)` — JSON, CSV, GraphML, GEXF, DOT, Markdown, Mermaid |
54
- | W3C Linked Data export | `ctx.ioManager.exportGraph('turtle' \| 'rdf-xml' \| 'json-ld')` |
55
- | Conversation ingestion (format-agnostic) | `ctx.ioManager.ingest(input, options)` |
56
-
57
- ### Search & retrieval
58
-
59
- | Capability | Entry point |
60
- |---|---|
61
- | Auto-selecting search with method explanation | `ctx.searchManager.autoSearch(query)` |
62
- | TF-IDF + BM25 ranked search (incremental indexing) | `ctx.rankedSearch`, `BM25Search` |
63
- | Boolean (AND / OR / NOT) with AST parser | `ctx.searchManager.booleanSearch()` |
64
- | Fuzzy matching (Levenshtein, N-gram pre-filtered) | `ctx.searchManager.fuzzySearch()` |
65
- | Semantic search with pluggable embedding provider | `ctx.semanticSearch` (set `MEMORY_EMBEDDING_PROVIDER`) |
66
- | Hybrid (semantic + lexical + symbolic) | `new HybridSearchManager(ctx.storage, …).search(query)` |
67
- | Temporal range queries with natural-language parsing | `ctx.searchManager.searchByTime("last hour")` |
68
- | LLM-planned natural-language queries | `ctx.queryNaturalLanguage(query, llmProvider?)` |
69
- | Query diagnostics (`explainPlan`, index health) | `ctx.diagnostics` |
70
-
71
- ### Graph algorithms
72
-
73
- | Capability | Entry point |
74
- |---|---|
75
- | Shortest path + all paths | `ctx.graphTraversal.findShortestPath()` / `findAllPaths()` |
76
- | Centrality — degree, betweenness, PageRank, HITS | `ctx.graphTraversal.calculatePageRank()` / `calculateHITS()` |
77
- | Community detection (Louvain), clique enumeration | `ctx.graphTraversal.findCommunities()` / `findCliques()` |
78
- | Connected components | `ctx.graphTraversal.findConnectedComponents()` |
79
-
80
- ### Agent Memory System
81
-
82
- | Capability | Entry point |
83
- |---|---|
84
- | Sessions, working memory, episodic memory | `ctx.agentMemory()` — `startSession` / `addWorkingMemory` / `retrieveForContext` |
85
- | Turn-aware conversation memory with 4-tier dedup (exact / prefix / Jaccard / semantic) | `ctx.memoryEngine.addTurn()` / `getSessionTurns()` |
86
- | Decay, salience, freshness | `DecayEngine`, `SalienceEngine`, `ctx.freshnessManager` |
87
- | Role profiles — `researcher` / `planner` / `executor` / `reviewer` / `coordinator` | `MEMORY_AGENT_ROLE` or `RoleProfileManager.apply(role)` |
88
- | Memory consolidation, summarization, pattern detection | `ConsolidationPipeline`, `ConsolidationScheduler` |
89
- | Collaborative synthesis with conflict resolution | `CollaborativeSynthesis.synthesize()` / `resolveConflicts()` |
90
- | Failure-driven distillation, cognitive load analysis | `FailureDistillation`, `CognitiveLoadAnalyzer` |
91
- | Procedural memory (executable procedures with feedback refinement) | `ctx.procedureManager.addProcedure()` / `matchProcedure()` / `refineProcedure()` |
92
- | Active retrieval (iterative query rewriting) | `ctx.activeRetrieval.adaptiveRetrieve()` |
93
- | Causal reasoning — causes / effects / counterfactuals / cycle detection | `ctx.causalReasoner.findCauses()` / `findEffects()` / `counterfactual()` |
94
- | World-state orchestrator | `ctx.worldModelManager.getCurrentState()` / `predictOutcome()` |
95
- | Per-agent persistent journal | `AgentMemoryManager.writeDiary()` / `readDiary()` |
96
- | Prospective memory (intentions-to-act) | `ctx.prospectiveMemory.schedule()` / `fire()` / `cancel()` |
97
- | Failure memory (pre-task `applicability_hint` lookup) | `ctx.failureManager.record()` / `lookupForTask()` |
98
- | Plan memory (hierarchical goal trees) | `ctx.plan.createPlan()` / `pushSubGoal()` / `transitionNode()` |
99
- | Reflection memory (additive insights with content-hash dedup) | `ctx.reflectionManager.create()` / `getRelevantForSession()` |
100
- | Heuristic memory (condition → action guidelines) | `ctx.heuristicManager.add()` / `match()` / `reinforce()` |
101
- | Decision rationale (ADR-style records) | `ctx.decisionManager.propose()` / `accept()` / `supersede()` |
102
- | Project context (facts / conventions / commands / glossary) | `ctx.projectContextManager.upsert()` / `forContext()` |
103
- | Tool affordance (outcome-aware tool suggestions) | `ctx.toolAffordanceManager.recordOutcome()` / `suggestTool()` |
104
- | Trust hierarchy — `ground-truth` / `verified` / `inferred` / `unverified` | `MemorySource.trustLevel` + `inferTrustLevel()` |
105
-
106
- ### Storage & performance
107
-
108
- | Capability | Entry point |
109
- |---|---|
110
- | JSONL or SQLite backend (FTS5, BM25, WAL mode) | `MEMORY_STORAGE_TYPE=jsonl\|sqlite` |
111
- | Pluggable Memory Engine backend | `MEMORY_BACKEND=sqlite\|in-memory` |
112
- | Memory-mapped file loading for large stores | `MEMORY_USE_MMAP=true`, `MEMORY_MMAP_THRESHOLD_BYTES` |
113
- | Segment-sharded JSONL (FNV-routed N-way shards) | `MEMORY_STORAGE_SEGMENT_COUNT=1..1024` |
114
- | Columnar observation storage | `JsonlColumnStore` (env-gated) |
115
- | Tiered index — LRU hot / disk warm / Brotli cold | `LRUHotTier` → `DiskWarmTier` → `BrotliColdTier` via `TieredIndex` |
116
- | In-memory entity-cache compression | `ctx.compressedEntityCache`, `CompressedMap` |
117
- | Backup lifecycle (create / list / restore / delete) with symlink-attack guards | `ctx.ioManager` (delegates to `BackupManager`) |
118
- | Streaming export with Brotli compression | `ctx.streamingExporter` |
119
- | Entity archival to compressed storage | `ctx.archiveManager` |
120
- | Duplicate detection + entity merging | `ctx.compressionManager` |
121
-
122
- ### Governance, security, multi-agent
123
-
124
- | Capability | Entry point |
125
- |---|---|
126
- | Policy enforcement + transactional rollback | `ctx.governanceManager.withTransaction()` / `GovernancePolicy` |
127
- | Immutable JSONL audit trail | `ctx.auditLog` |
128
- | Strict-mode attribution enforcer | `CollaborationAuditEnforcer` (requires `agentId` on every mutation) |
129
- | RBAC — role / permission / matrix / middleware | `ctx.rbacMiddleware.checkPermission()` / `ctx.roleAssignmentStore` |
130
- | ABAC + row-level security + API-key scoping | `src/security/abac.ts`, `rls.ts`, `apiKeys.ts` |
131
- | PII redactor (email / SSN / CC / phone / IP) | `new PiiRedactor().redactGraph()` / `redactWithStats()` |
132
- | Visibility hierarchies (5-level: `private` / `team` / `org` / `shared` / `public`) | `VisibilityResolver.canAccess()` |
133
- | Time-window + role-gated visibility | `AgentEntity.visibleFrom` / `visibleUntil` / `allowedRoles[]` |
134
- | Path-traversal protection | `validateFilePath(path, baseDir?, confineToBase=true)` |
135
-
136
- ### Tooling
137
-
138
- | Capability | Entry point |
139
- |---|---|
140
- | Command-line interface | `npx -p @danielsimonjr/memoryjs memory --help` |
141
- | End-to-end smoke test against a fresh temp graph (~30 ops) | `memory smoke --keep --verbose` |
142
- | Diagnostic snapshot + graph health checks | `memory diag` / `memory health` |
143
- | Orphan + missing-parent + cycle detection with optional repair | `memory check [--apply]` |
144
- | Rebuild ranked + spell indexes | `memory reindex [--ranked\|--spell]` |
145
- | Inspect a single entity verbosely | `memory show <name>` |
146
- | Hierarchy tree (JSON or `--ascii`) | `memory tree [root]` |
147
- | Search-cache stats / clear | `memory cache stats \| clear` |
148
- | Interactive REPL | `memory interactive` |
149
-
150
-
151
- ## Installation
152
-
153
- ```bash
154
- npm install @danielsimonjr/memoryjs
155
- ```
156
-
157
- ### Requirements
158
-
159
- - Node.js >= 18.0.0
160
- - TypeScript >= 5.0 (for development)
161
-
162
- ## Quick Start
163
-
164
- ### 1. Initialize storage
165
-
166
- ```typescript
167
- import { ManagerContext } from '@danielsimonjr/memoryjs';
168
-
169
- // JSONL storage (default, human-readable)
170
- const ctx = new ManagerContext('./memory.jsonl');
171
-
172
- // Or SQLite (set MEMORY_STORAGE_TYPE=sqlite in the environment)
173
- const sqliteCtx = new ManagerContext('./memory.db');
174
- ```
175
-
176
- ### 2. Create entities
177
-
178
- ```typescript
179
- await ctx.entityManager.createEntities([
180
- {
181
- name: 'TypeScript',
182
- entityType: 'language',
183
- observations: ['A typed superset of JavaScript'],
184
- tags: ['programming', 'frontend'],
185
- importance: 8,
186
- },
187
- {
188
- name: 'Node.js',
189
- entityType: 'runtime',
190
- observations: ['JavaScript runtime built on V8'],
191
- tags: ['backend', 'server'],
192
- },
193
- ]);
194
- ```
195
-
196
- ### 3. Create relations
197
-
198
- ```typescript
199
- await ctx.relationManager.createRelations([
200
- { from: 'TypeScript', to: 'Node.js', relationType: 'runs_on' },
201
- ]);
202
- ```
203
-
204
- ### 4. Search
205
-
206
- ```typescript
207
- // Auto-select the best method
208
- const auto = await ctx.searchManager.autoSearch('JavaScript');
209
-
210
- // Substring + tag search
211
- const nodes = await ctx.searchManager.searchNodes('JavaScript');
212
-
213
- // Boolean (AND / OR / NOT) with an AST parser
214
- const both = await ctx.searchManager.booleanSearch('TypeScript AND runtime');
215
-
216
- // Fuzzy (typo-tolerant; N-gram pre-filtered)
217
- const fuzzy = await ctx.searchManager.fuzzySearch('Typscript', { threshold: 0.7 });
218
-
219
- // Ranked TF-IDF / BM25
220
- const ranked = await ctx.rankedSearch.searchNodesRanked('runtime environment',
221
- undefined, undefined, undefined, 10);
222
-
223
- // Active retrieval — iterative query rewriting until coverage threshold
224
- const adaptive = await ctx.activeRetrieval.adaptiveRetrieve({ query: 'memory leak' });
225
- console.log(adaptive.bestResults, adaptive.bestCoverage, adaptive.rounds);
226
- ```
227
-
228
- ### 5. Multi-agent collaboration
229
-
230
- ```typescript
231
- // Optimistic concurrency control — fail loudly on stale writes
232
- try {
233
- await ctx.entityManager.updateEntity('Alice',
234
- { importance: 9 },
235
- { expectedVersion: 3 });
236
- } catch (e) {
237
- if (e instanceof Error && e.name === 'VersionConflictError') {
238
- // Refetch + retry
239
- }
240
- }
241
-
242
- // Synthesize a view across agents + resolve conflicts
243
- const agent = ctx.agentMemory();
244
- const synth = await agent.collaborativeSynthesis.synthesize('Alice');
245
- const winners = agent.collaborativeSynthesis.resolveConflicts(synth, {
246
- strategy: 'highest_confidence',
247
- });
248
-
249
- // Enforce attribution on every mutation
250
- import { CollaborationAuditEnforcer, AuditLog } from '@danielsimonjr/memoryjs';
251
- const enforcer = new CollaborationAuditEnforcer(
252
- ctx.entityManager,
253
- new AuditLog('./audit.jsonl'),
254
- );
255
- await enforcer.createEntities(
256
- [{ name: 'X', entityType: 't', observations: ['fact'] }],
257
- 'agent-alice', // throws AttributionRequiredError if agentId is missing
258
- );
259
- ```
260
-
261
- ### 6. Bitemporal versioning
262
-
263
- ```typescript
264
- // Mark an entity as no longer valid at a specific time
265
- await ctx.entityManager.invalidateEntity('OldFact', '2025-12-31T00:00:00Z');
266
-
267
- // Time-travel query
268
- const past = await ctx.entityManager.entityAsOf('Alice', '2024-06-15T00:00:00Z');
269
-
270
- // Per-observation validity windows
271
- await ctx.observationManager.invalidateObservation(
272
- 'Alice', 'works at Acme', '2024-12-31T00:00:00Z',
273
- );
274
- const obsAtTime = await ctx.observationManager.observationsAsOf(
275
- 'Alice', '2024-06-15T00:00:00Z',
276
- );
277
- ```
278
-
279
- ### 7. Causal reasoning and world model
280
-
281
- ```typescript
282
- // Forward inference — "what does X cause?"
283
- const effects = await ctx.causalReasoner.findEffects('rain', ['flooding', 'erosion']);
284
-
285
- // Counterfactual — "what if we remove this edge?"
286
- const surviving = await ctx.causalReasoner.counterfactual({
287
- seed: 'rain', removeFrom: 'rain', removeTo: 'flooding', predict: 'flooding',
288
- });
289
-
290
- // World-state snapshot + diff
291
- const before = await ctx.worldModelManager.getCurrentState();
292
- // ... mutations ...
293
- const after = await ctx.worldModelManager.getCurrentState();
294
- const change = ctx.worldModelManager.detectStateChange(before, after);
295
- ```
296
-
297
- ### 8. RBAC and PII redaction
298
-
299
- ```typescript
300
- // Grant a role
301
- await ctx.roleAssignmentStore.assign({
302
- agentId: 'alice', role: 'writer', resourceType: 'entity',
303
- });
304
- ctx.rbacMiddleware.checkPermission('alice', 'write', 'entity'); // true
305
-
306
- // Redact PII from exports / logs
307
- import { PiiRedactor } from '@danielsimonjr/memoryjs';
308
- const redactor = new PiiRedactor();
309
- const cleanGraph = redactor.redactGraph(graph);
310
- const { text, stats } = redactor.redactWithStats(observation);
311
- ```
312
-
313
- ## Core Concepts
314
-
315
- ### Entity
316
-
317
- The primary node in the knowledge graph. Required fields are `name`, `entityType`,
318
- and `observations`; the rest are optional and unlock specific features:
319
-
320
- ```typescript
321
- interface Entity {
322
- name: string; // Unique identifier
323
- entityType: string; // Classification (person, project, concept, …)
324
- observations: string[]; // Atomic facts about the entity
325
- parentId?: string; // Parent entity for hierarchical nesting
326
- tags?: string[]; // Lowercase tags for categorisation
327
- importance?: number; // 0–10 scale for prioritisation
328
- createdAt?: string; // ISO 8601
329
- lastModified?: string; // ISO 8601
330
-
331
- // Freshness
332
- ttl?: number; // Seconds until stale
333
- confidence?: number; // [0, 1] belief strength
334
-
335
- // Project scoping + supersession
336
- projectId?: string; // Multi-project isolation
337
- version?: number; // Drives optimistic concurrency on updates
338
- parentEntityName?: string;
339
- rootEntityName?: string;
340
- isLatest?: boolean;
341
- supersededBy?: string;
342
-
343
- // Memory-engine deduplication
344
- contentHash?: string; // SHA-256 for O(1) exact-equality dedup
345
-
346
- // Bitemporal validity (orthogonal to supersession)
347
- validFrom?: string;
348
- validUntil?: string;
349
- observationMeta?: Array<{
350
- content: string;
351
- validFrom?: string;
352
- validUntil?: string;
353
- recordedAt?: string;
354
- }>;
355
- }
356
- ```
357
-
358
- ### Relation
359
-
360
- A directed edge between two entities:
361
-
362
- ```typescript
363
- interface Relation {
364
- from: string; // Source entity name
365
- to: string; // Target entity name
366
- relationType: string; // Relationship type (active voice — "depends_on", "wrote")
367
- }
368
- ```
369
-
370
- ### Observation
371
-
372
- A discrete atomic fact about an entity. Use `addObservations()` to append without
373
- overwriting; combined with the bitemporal axis and per-observation `validFrom` /
374
- `validUntil` you can preserve historical state instead of mutating in place.
375
-
376
- ### ManagerContext
377
-
378
- The central facade. Construct it with a storage-path string; every manager is
379
- exposed as a property with lazy initialisation:
380
-
381
- ```typescript
382
- const ctx = new ManagerContext('./memory.jsonl');
383
-
384
- // Core
385
- ctx.entityManager // Entity CRUD + hierarchy + temporal validity + OCC
386
- ctx.relationManager // Relation management + temporal invalidation
387
- ctx.observationManager // Observation CRUD + bitemporal axis
388
- ctx.hierarchyManager // Tree operations (parent / children / ancestors)
389
- ctx.searchManager // All search operations including `searchByTime`
390
- ctx.rankedSearch // TF-IDF / BM25 ranked search
391
- ctx.graphTraversal // BFS / DFS / shortest path / centrality / communities
392
- ctx.tagManager // Tag aliases + bulk operations
393
- ctx.refIndex // Stable name → entity O(1) lookup
394
-
395
- // Storage + I/O
396
- ctx.ioManager // Import / export (RDF/Turtle/JSON-LD) / backup / ingest
397
- ctx.archiveManager // Entity archival
398
- ctx.compressionManager // Duplicate detection, entity merging
399
- ctx.analyticsManager // Graph statistics + validation
400
- ctx.semanticForget // Two-tier deletion with audit
401
- ctx.governanceManager // Transactions + policy enforcement
402
- ctx.freshnessManager // TTL / confidence freshness reports
403
-
404
- // Search extensions
405
- ctx.semanticSearch // Vector similarity (lazy; needs embedding provider)
406
- ctx.temporalSearch // Natural-language time-range search
407
- ctx.activeRetrieval // Iterative query rewriting (no LLM required)
408
- ctx.queryNaturalLanguage // Convenience wrapper for NL → StructuredQuery
409
-
410
- // Memory engine + agent system
411
- ctx.memoryEngine // Turn-aware conversation memory (4-tier dedup)
412
- ctx.memoryBackend // Pluggable IMemoryBackend (in-memory / sqlite)
413
- ctx.agentMemory() // Agent Memory System facade
414
-
415
- // Memory intelligence
416
- ctx.memoryValidator // Consistency / contradictions / temporal-order checks
417
- ctx.trajectoryCompressor // Distill / abstract / merge redundant trajectories
418
-
419
- // Specialised memory managers
420
- ctx.procedureManager // Executable procedure memory + feedback refinement
421
- ctx.causalReasoner // findCauses / findEffects / counterfactual
422
- ctx.worldModelManager // State-snapshot orchestrator + diff
423
- ctx.heuristicManager // Condition → action guidelines
424
- ctx.decisionManager // ADR-style decision records
425
- ctx.projectContextManager
426
- ctx.toolAffordanceManager
427
- ctx.spellChecker // Vocabulary-driven spell suggestions
428
- ctx.exclusionManager // `do_not_remember` content-pattern rules
429
- ctx.observationDedupManager
430
-
431
- // Access control + audit
432
- ctx.roleAssignmentStore
433
- ctx.rbacMiddleware
434
- ```
435
-
436
- ## Storage Options
437
-
438
- ### Comparison
439
-
440
- | Feature | JSONL (Default) | SQLite (better-sqlite3) |
441
- |---------|-----------------|-------------------------|
442
- | Format | Human-readable text | Native binary database |
443
- | Transactions | Basic | Full ACID with WAL mode |
444
- | Full-Text Search | Basic | FTS5 with BM25 ranking |
445
- | Performance | Good | 3-10x faster |
446
- | Concurrency | Single-threaded | Thread-safe with async-mutex |
447
- | Best For | Small graphs, debugging | Large graphs (10k+ entities) |
448
-
449
- ### JSONL Storage
450
-
451
- ```typescript
452
- const ctx = new ManagerContext('./memory.jsonl');
453
- ```
454
-
455
- Features:
456
- - Human-readable line-delimited JSON
457
- - In-memory caching with write-through invalidation
458
- - Atomic writes via temp file + rename
459
- - Backward compatibility for legacy formats
460
-
461
- ### SQLite Storage
462
-
463
- ```typescript
464
- // Set MEMORY_STORAGE_TYPE=sqlite environment variable
465
- const ctx = new ManagerContext('./memory.db');
466
- ```
467
-
468
- Features:
469
- - FTS5 full-text search with BM25 ranking
470
- - WAL mode for better concurrency
471
- - Referential integrity with ON DELETE CASCADE
472
- - ACID transactions
473
-
474
- ### Storage Files
475
-
476
- When using JSONL, related files are automatically created:
477
-
478
- ```
479
- /your/data/directory/
480
- ├── memory.jsonl # Main knowledge graph
481
- ├── memory-saved-searches.jsonl # Saved search queries
482
- ├── memory-tag-aliases.jsonl # Tag synonym mappings
483
- └── .backups/ # Timestamped backups
484
- ```
485
-
486
- ## Search Capabilities
487
-
488
- ### Available methods
489
-
490
- | Method | Description | Use case |
491
- |---|---|---|
492
- | `searchManager.autoSearch(query)` | Auto-selects the best method + explains why | Default entry point |
493
- | `searchManager.searchNodes(query)` | Substring + tag matching | Simple queries |
494
- | `searchManager.booleanSearch(query)` | AND / OR / NOT with AST parser | Complex filters |
495
- | `searchManager.fuzzySearch(query)` | Levenshtein + N-gram pre-filter | Typo tolerance |
496
- | `searchManager.searchByTime(query)` | Natural-language time ranges (`chrono-node`) | "last hour", "since 2024-01-01" |
497
- | `rankedSearch.searchNodesRanked(query, ...)` | TF-IDF / BM25 ranking | Most-relevant ordering |
498
- | `semanticSearch.search(query)` | Vector similarity (requires embedding provider) | Semantic queries |
499
- | `new HybridSearchManager(...).search(query)` | Semantic + lexical + symbolic | Multi-signal ranking |
500
- | `activeRetrieval.adaptiveRetrieve({ query })` | Iterative query rewriting + coverage check | Adaptive refinement |
501
- | `queryNaturalLanguage(query, llmProvider?)` | LLM-planned NL decomposition (fallback: keyword) | Free-text queries |
502
- | `causalReasoner.findEffects(seed, candidates)` | Subgraph traversal over `causes`/`enables`/`prevents` | Causal inference |
503
-
504
- ### Auto-search
505
-
506
- ```typescript
507
- const auto = await ctx.searchManager.autoSearch('TypeScript runtime');
508
- console.log(auto.selectedMethod, auto.selectionReason);
509
- console.log(auto.results);
510
- ```
511
-
512
- ### Boolean search
513
-
514
- ```typescript
515
- // AND — both terms must match
516
- await ctx.searchManager.booleanSearch('TypeScript AND runtime');
517
-
518
- // OR — either term matches
519
- await ctx.searchManager.booleanSearch('frontend OR backend');
520
-
521
- // NOT — exclude term
522
- await ctx.searchManager.booleanSearch('JavaScript NOT browser');
523
-
524
- // Grouping
525
- await ctx.searchManager.booleanSearch('(TypeScript OR JavaScript) AND server');
526
- ```
527
-
528
- ### Fuzzy search
529
-
530
- ```typescript
531
- // Typo-tolerant; threshold 0–1 (higher = stricter)
532
- await ctx.searchManager.fuzzySearch('Typscript', { threshold: 0.7 });
533
- ```
534
-
535
- ### Hybrid search
536
-
537
- `HybridSearchManager` combines three signal layers. Instantiate it directly —
538
- it is exported from the package but not wired onto `ManagerContext` because the
539
- weighting + filter configuration is application-specific.
540
-
541
- ```typescript
542
- import { HybridSearchManager } from '@danielsimonjr/memoryjs';
543
-
544
- const hybrid = new HybridSearchManager(ctx.storage, ctx.rankedSearch);
545
- const results = await hybrid.search('programming concepts', {
546
- weights: {
547
- semantic: 0.5, // Vector similarity (requires embeddings)
548
- lexical: 0.3, // TF-IDF text matching
549
- symbolic: 0.2, // Metadata (tags, importance, type)
550
- },
551
- filters: {
552
- entityTypes: ['concept'],
553
- minImportance: 5,
554
- tags: ['programming'],
555
- },
556
- });
557
- ```
558
-
559
- ## Graph Algorithms
560
-
561
- ### Path finding
562
-
563
- ```typescript
564
- // Shortest path between entities (BFS)
565
- const result = await ctx.graphTraversal.findShortestPath('A', 'Z');
566
- console.log(result.path); // ['A', 'B', 'C', 'Z']
567
- console.log(result.distance); // 3
568
-
569
- // All paths with a depth bound
570
- const all = await ctx.graphTraversal.findAllPaths('A', 'Z', { maxDepth: 5 });
571
- // → [['A', 'B', 'Z'], ['A', 'C', 'D', 'Z'], …]
572
- ```
573
-
574
- ### Centrality
575
-
576
- Each centrality algorithm has its own method (no unified `getCentrality()` — the
577
- algorithms have different parameters and return shapes):
578
-
579
- ```typescript
580
- const degree = await ctx.graphTraversal.calculateDegreeCentrality();
581
- const between = await ctx.graphTraversal.calculateBetweennessCentrality();
582
- const pagerank = await ctx.graphTraversal.calculatePageRank();
583
- const hits = await ctx.graphTraversal.calculateHITS();
584
- // Each returns a Map<entityName, score>
585
- ```
586
-
587
- ### Connected components, communities, cliques
588
-
589
- ```typescript
590
- const components = await ctx.graphTraversal.findConnectedComponents();
591
- const communities = await ctx.graphTraversal.findCommunities(); // Louvain
592
- const cliques = await ctx.graphTraversal.findCliques(/* minSize */ 3);
593
- ```
594
-
595
- ### Traversal
596
-
597
- ```typescript
598
- // Returns a TraversalResult { visited, edges, depthMap }
599
- const bfs = ctx.graphTraversal.bfs('startNode', { maxDepth: 3 });
600
- const dfs = ctx.graphTraversal.dfs('startNode');
601
- ```
602
-
603
- ## Agent Memory System
604
-
605
- A memory system for AI agents with session lifecycle, working / episodic /
606
- semantic / procedural memory, decay, salience, multi-agent visibility, and
607
- specialised memory types (prospective, failure, plan, reflection, heuristic,
608
- decision rationale, project context, tool affordance).
609
-
610
- ### Key components
611
-
612
- | Component | Purpose |
613
- |---|---|
614
- | `AgentMemoryManager` | Unified facade for the system |
615
- | `SessionManager` | Session lifecycle (start / end / restore / checkpoint) |
616
- | `WorkingMemoryManager` | Short-term, session-scoped memories with promotion |
617
- | `EpisodicMemoryManager` | Timeline-based event memories |
618
- | `DecayEngine` | Time-based importance decay |
619
- | `SalienceEngine` | Context-aware memory scoring |
620
- | `MultiAgentMemoryManager` | Cross-agent memory with visibility controls |
621
- | `ConflictResolver` | Resolution strategies for concurrent updates |
622
-
623
- ### Quick start
624
-
625
- ```typescript
626
- import { ManagerContext } from '@danielsimonjr/memoryjs';
627
-
628
- const ctx = new ManagerContext('./memory.jsonl');
629
- const agent = ctx.agentMemory();
630
-
631
- // Start a session
632
- const session = await agent.startSession({ agentId: 'my-agent' });
633
-
634
- // Add working memory
635
- await agent.addWorkingMemory({
636
- sessionId: session.name,
637
- content: 'User prefers dark mode',
638
- importance: 7,
639
- });
640
-
641
- // Episodic event
642
- await agent.createEpisode('Completed onboarding flow', {
643
- sessionId: session.name,
644
- importance: 8,
645
- });
646
-
647
- // Retrieve context for an LLM prompt (token-budgeted)
648
- const context = await agent.retrieveForContext({
649
- maxTokens: 2000,
650
- includeEpisodic: true,
651
- });
652
-
653
- // End session
654
- await agent.endSession(session.name);
655
- ```
656
-
657
- ### Memory types
658
-
659
- ```typescript
660
- type MemoryType =
661
- | 'working' // Short-term, session-scoped; may be promoted
662
- | 'episodic' // Timeline-based event memories
663
- | 'semantic' // Long-term factual knowledge
664
- | 'procedural' // Executable procedures with feedback refinement
665
- | 'prospective' // Intentions-to-act at a future time / event / condition
666
- | 'failure' // Pre-task failure lookup keyed on applicability_hint
667
- | 'plan' // Hierarchical goal trees with sub-tasks + acceptance criteria
668
- | 'reflection' // Additive derived insights with content-hash dedup
669
- | 'heuristic'; // Condition → action guidelines
670
- ```
671
-
672
- Each specialised type has a dedicated manager on `ManagerContext` —
673
- `ctx.prospectiveMemory`, `ctx.failureManager`, `ctx.plan`, `ctx.reflectionManager`,
674
- `ctx.heuristicManager`, `ctx.decisionManager`, `ctx.projectContextManager`,
675
- `ctx.toolAffordanceManager`.
676
-
677
- **Trust hierarchy mixin**: every `MemorySource` may carry an optional
678
- categorical `trustLevel`: `'ground-truth' | 'verified' | 'inferred' | 'unverified'`.
679
- Used by the `'trust_level'` `ConflictStrategy` (with recency tiebreak) and
680
- backfilled from `method` + `reliability` via `inferTrustLevel(source)`.
681
-
682
- ### Decay
683
-
684
- Memories naturally decay over time unless reinforced:
685
-
686
- ```typescript
687
- const agent = ctx.agentMemory({
688
- decay: {
689
- halfLifeHours: 168, // One-week half-life
690
- minImportance: 0.1, // Never fully forget
691
- },
692
- enableAutoDecay: true,
693
- });
694
-
695
- // Reinforce a memory
696
- await agent.confirmMemory('memory_name', 0.1); // Boost confidence
697
- await agent.promoteMemory('memory_name', 'episodic'); // Promote to long-term
698
- ```
699
-
700
- ### Multi-agent
701
-
702
- ```typescript
703
- // Register an agent
704
- agent.registerAgent('agent_1', {
705
- name: 'Research Agent',
706
- type: 'llm',
707
- trustLevel: 0.8,
708
- capabilities: ['read', 'write'],
709
- });
710
-
711
- // Create with visibility (private / team / org / shared / public)
712
- await agent.addWorkingMemory({
713
- sessionId: session.name,
714
- content: 'Shared insight',
715
- visibility: 'shared',
716
- ownerAgentId: 'agent_1',
717
- });
718
-
719
- // Cross-agent search
720
- const results = await agent.searchCrossAgent('agent_2', 'query');
721
- ```
722
-
723
- ## API Reference
724
-
725
- Method tables below cover the most-used surfaces. For the full surface use
726
- your IDE's go-to-definition or read `dist/index.d.ts`.
727
-
728
- ### EntityManager
729
-
730
- | Method | Description |
731
- |---|---|
732
- | `createEntities(entities)` | Create multiple entities in one batch |
733
- | `deleteEntities(names)` | Delete entities by name |
734
- | `getEntity(name, options?)` | Get one entity (with optional access tracking) |
735
- | `updateEntity(name, updates, { expectedVersion? })` | Partial update; opt-in optimistic concurrency |
736
- | `addTags(name, tags)` / `removeTags(name, tags)` | Tag management |
737
- | `setImportance(name, score)` | Set importance (0–10) |
738
- | `getVersionChain(name)` / `getLatestVersion(name)` | Supersession chains |
739
- | `invalidateEntity(name, ended?)` | Set `validUntil` (bitemporal) |
740
- | `entityAsOf(name, asOf)` | Time-travel query |
741
- | `entityTimeline(name)` | Versions sorted by `validFrom` |
742
-
743
- ### RelationManager
744
-
745
- | Method | Description |
746
- |---|---|
747
- | `createRelations(relations)` | Create multiple relations |
748
- | `getRelations(entityName)` | Incoming + outgoing relations |
749
- | `deleteRelations(relations)` | Delete specific relations |
750
- | `invalidateRelation(from, type, to, ended?)` | Set `validUntil` on a relation |
751
- | `queryAsOf(entity, asOf, options?)` | Relations valid at time T |
752
- | `timeline(entity, options?)` | Chronological history |
753
-
754
- ### ObservationManager
755
-
756
- | Method | Description |
757
- |---|---|
758
- | `addObservations(adds, dedupOptions?)` | Add observations (optional dedup) |
759
- | `deleteObservations(deletions)` | Remove specific observations |
760
- | `getObservationsFor(entityName)` | Read observations (column-store-aware) |
761
- | `invalidateObservation(entity, content, ended?)` | Set per-observation `validUntil` |
762
- | `observationsAsOf(entity, asOf)` | Observations valid at time T |
763
-
764
- ### SearchManager (`ctx.searchManager`)
765
-
766
- | Method | Description |
767
- |---|---|
768
- | `searchNodes(query, options?)` | Substring + tag matching |
769
- | `booleanSearch(query, options?)` | AND / OR / NOT with AST |
770
- | `fuzzySearch(query, options?)` | Levenshtein + N-gram pre-filter |
771
- | `searchByTime(query, options?)` | Natural-language time ranges |
772
- | `autoSearch(query, limit?)` | Auto-select best method; returns `selectedMethod` + `selectionReason` |
773
- | `openNodes(names)` | Bulk-fetch entities by name |
774
-
775
- For ranked search use `ctx.rankedSearch.searchNodesRanked(query, tags?, min?, max?, limit?)`.
776
- For hybrid search, instantiate `HybridSearchManager` directly.
777
-
778
- ### IOManager (`ctx.ioManager`)
779
-
780
- | Method | Description |
781
- |---|---|
782
- | `exportGraph(graph, format)` | Export to `json` / `csv` / `graphml` / `gexf` / `dot` / `markdown` / `mermaid` / `turtle` / `rdf-xml` / `json-ld` |
783
- | `exportGraphWithCompression(graph, format, options?)` | Brotli-compressed export |
784
- | `importGraph(format, data, options?)` | Import with merge strategies (`replace` / `skip` / `merge` / `fail`) |
785
- | `ingest(input, options?)` | Conversation ingestion pipeline (format-agnostic) |
786
- | `splitTranscript(content, options?)` | Split multi-session transcripts |
787
- | `visualizeGraph(options?)` | Interactive HTML visualisation |
788
- | `createBackup(options?)` / `restoreFromBackup(path)` / `deleteBackup(path)` / `cleanOldBackups(keepCount?)` | Backup lifecycle |
789
-
790
- ### GraphTraversal (`ctx.graphTraversal`)
791
-
792
- | Method | Description |
793
- |---|---|
794
- | `findShortestPath(from, to)` | BFS shortest path |
795
- | `findAllPaths(from, to, options?)` | All paths with depth bound |
796
- | `findConnectedComponents()` | Isolated subgraphs |
797
- | `findCliques(minSize?)` | Maximal cliques |
798
- | `findCommunities()` | Louvain community detection |
799
- | `calculateDegreeCentrality()` / `calculateBetweennessCentrality()` / `calculatePageRank()` / `calculateHITS()` | Centrality (each returns `Map<entityName, score>`) |
800
- | `bfs(start, options?)` / `dfs(start, options?)` | Returns `TraversalResult` |
801
-
802
- ### CausalReasoner (`ctx.causalReasoner`)
803
-
804
- | Method | Description |
805
- |---|---|
806
- | `findEffects(cause, candidates, maxDepth?)` | Forward inference; sorted by causal-strength product |
807
- | `findCauses(effect, candidates, maxDepth?)` | Backward inference (symmetric inverse) |
808
- | `counterfactual({ seed, removeFrom, removeTo, predict })` | Pure: doesn't mutate the graph |
809
- | `detectCycles(seed, maxDepth?)` | Depth-bounded DFS over causal subgraph |
810
-
811
- ### ProcedureManager (`ctx.procedureManager`)
812
-
813
- | Method | Description |
814
- |---|---|
815
- | `addProcedure({ steps, ... })` | Persist a procedure; auto-generates id |
816
- | `getProcedure(id)` / `getStep(id, order)` / `getNextStep(id, order)` | Access |
817
- | `openSequencer(id)` | Stateful execution cursor with fallback support |
818
- | `matchProcedure(context, candidates, threshold?)` | Token-overlap match |
819
- | `refineProcedure(id, { succeeded, notes? })` | EWMA success-rate update |
820
-
821
- ### WorldModelManager (`ctx.worldModelManager`)
822
-
823
- | Method | Description |
824
- |---|---|
825
- | `getCurrentState()` | Snapshot from the live graph (capped at `maxSnapshotSize`) |
826
- | `validateFact(observation, entityName)` | Delegates to `MemoryValidator` if wired |
827
- | `predictOutcome(action, candidates)` | Delegates to `CausalReasoner.findEffects` |
828
- | `detectStateChange(before, after)` | Pure snapshot diff |
829
-
830
- ### ActiveRetrievalController (`ctx.activeRetrieval`)
831
-
832
- | Method | Description |
833
- |---|---|
834
- | `shouldRetrieve(context)` | Cost heuristic; rejects empty / over-budget |
835
- | `adaptiveRetrieve(context)` | Iterative rewrite + retrieve until coverage threshold |
836
-
837
- ### CollaborativeSynthesis (`ctx.agentMemory().collaborativeSynthesis`)
838
-
839
- | Method | Description |
840
- |---|---|
841
- | `synthesize(seedEntity, context?)` | BFS + salience scoring; surfaces multi-agent conflicts |
842
- | `resolveConflicts(result, policy)` | Pick winners per `most_recent` / `highest_confidence` / `highest_score` / `trusted_agent` |
843
-
844
- ### MemoryValidator (`ctx.memoryValidator`)
845
-
846
- | Method | Description |
847
- |---|---|
848
- | `validateConsistency(newObs, existing)` | Composite duplicate / semantic / low-confidence check |
849
- | `detectContradictions(entity)` | Delegates to `ContradictionDetector` |
850
- | `repairWithResolver(entity, competing, resolver, contradiction?, options?)` | Apply `ConflictResolver` strategies |
851
- | `validateTemporalOrder(observations)` | `[T=ISO]` ordering check |
852
- | `calculateReliability(entity)` | Confidence × confirmation × age penalty |
853
-
854
- ### RBAC (`ctx.rbacMiddleware`, `ctx.roleAssignmentStore`)
855
-
856
- | Method | Description |
857
- |---|---|
858
- | `rbacMiddleware.checkPermission(agentId, action, resourceType, resourceName?, now?)` | Returns `true` / `false`; falls back to `defaultRole` (default `reader`) |
859
- | `roleAssignmentStore.assign({ agentId, role, resourceType?, scope?, validFrom?, validUntil? })` | Grant a role |
860
- | `roleAssignmentStore.revoke(agentId, role, resourceType?)` | Remove a grant |
861
- | `roleAssignmentStore.listActive(agentId, now?)` | Active grants at a point in time |
862
-
863
- ### PiiRedactor
864
-
865
- | Method | Description |
866
- |---|---|
867
- | `redact(text)` | Apply patterns; returns redacted string |
868
- | `redactWithStats(text)` | Returns `{ text, stats: { totalRedactedBytes, countsByPattern } }` |
869
- | `redactGraph(graph)` | Apply to every observation in a graph-shaped object |
870
-
871
- ### Specialised memory managers
872
-
873
- | Manager | Path | Key methods |
874
- |---|---|---|
875
- | Prospective | `ctx.prospectiveMemory` | `schedule()` / `fire()` / `cancel()` / `expireDueIntentions()` / `getPending()` |
876
- | Failure | `ctx.failureManager` | `record()` / `lookupForTask()` / `markResolved()` / `getAll()` |
877
- | Plan | `ctx.plan` | `createPlan()` / `pushSubGoal()` / `transitionNode()` / `markPlanComplete()` / `getCurrentPath()` |
878
- | Reflection | `ctx.reflectionManager` | `create()` / `list()` / `getRelevantForSession()` / `archive()` |
879
- | Heuristic | `ctx.heuristicManager` | `add()` / `match()` / `reinforce()` / `recordContradiction()` / `detectConflicts()` |
880
- | Decision | `ctx.decisionManager` | `propose()` / `accept()` / `reject()` / `supersede()` / `findByContext()` / `getChain()` |
881
- | Project context | `ctx.projectContextManager` | `upsert()` / `appendFact()` / `appendCommand()` / `forContext()` |
882
- | Tool affordance | `ctx.toolAffordanceManager` | `recordOutcome()` / `suggestTool()` / `rollingStats()` |
883
- | Exclusion | `ctx.exclusionManager` | `add()` / `list()` / `remove()` / `check()` / `findMatchingMemories()` |
884
- | Observation dedup | `ctx.observationDedupManager` | `findDuplicateObservations()` / `findJaccardDuplicates()` |
885
- | Spell | `ctx.spellChecker` | `suggest()` / `rebuild()` / `vocabularySize()` |
886
-
887
- ## Configuration
888
-
889
- memoryjs is configured almost entirely through environment variables. The
890
- table below lists the most-used; the full reference (decay / salience /
891
- context-window / freshness / RBAC / mmap / segment / SQLite-pool / partial-index
892
- knobs) lives in [CLAUDE.md](CLAUDE.md#environment-variables).
893
-
894
- | Variable | Description | Default |
895
- |---|---|---|
896
- | `MEMORY_STORAGE_TYPE` | Storage backend: `jsonl` or `sqlite` | `jsonl` |
897
- | `MEMORY_FILE_PATH` | Override storage file path | (per `ManagerContext` ctor) |
898
- | `MEMORY_BACKEND` | Pluggable Memory-Engine backend: `sqlite` or `in-memory` | `sqlite` |
899
- | `MEMORY_EMBEDDING_PROVIDER` | Embedding provider: `openai`, `local`, or `none` | `local` |
900
- | `MEMORY_OPENAI_API_KEY` | OpenAI API key (required when provider is `openai`) | — |
901
- | `MEMORY_AUTO_INDEX_EMBEDDINGS` | Auto-build embedding index on entity create | `false` |
902
- | `MEMORY_AUTO_DECAY` | Enable background memory decay | `false` |
903
- | `MEMORY_DECAY_HALF_LIFE_HOURS` | Half-life for importance decay | `168` |
904
- | `MEMORY_GOVERNANCE_ENABLED` | Enable `GovernanceManager` policy enforcement | `false` |
905
- | `MEMORY_AUDIT_LOG_FILE` | Path for the audit JSONL trail | — |
906
- | `MEMORY_AGENT_ROLE` | Apply a built-in role profile (`researcher` / `planner` / `executor` / `reviewer` / `coordinator`) | — |
907
- | `MEMORY_VALIDATE_ON_STORE` | Run `MemoryValidator` before observation writes | `false` |
908
- | `MEMORY_AUDIT_ATTRIBUTION_REQUIRED` | `CollaborationAuditEnforcer` strict mode | `false` |
909
- | `MEMORY_RBAC_ENABLED` | Wire `RbacMiddleware` into `GovernancePolicy` | `false` |
910
- | `MEMORY_DEFAULT_VISIBILITY` | Default `AgentEntity.visibility` | `private` |
911
- | `MEMORY_USE_MMAP` | Use mmap for `GraphStorage.loadFromDisk` (strict `'true'` literal match) | `false` |
912
- | `MEMORY_MMAP_THRESHOLD_BYTES` | Minimum file size to trigger the mmap path; `0` = always | `104857600` |
913
- | `MEMORY_STORAGE_SEGMENT_COUNT` | FNV-routed JSONL shard count (1–1024) | `1` (off) |
914
- | `MEMORY_SQLITE_READ_POOL_SIZE` | Read-connection pool size for `SQLiteStorage` | `4` |
915
- | `MEMORY_SQLITE_AUTO_INDEX` | Enable `PartialIndexAdvisor`-driven automatic partial-index DDL | `false` |
916
- | `SKIP_BENCHMARKS` | Skip perf-benchmark tests | `false` |
917
- | `LOG_LEVEL` | `debug` / `info` / `warn` / `error` | (none) |
918
-
919
- ## Development
920
-
921
- ### Prerequisites
922
-
923
- - Node.js 18+
924
- - npm 9+
925
- - TypeScript 5.0+
926
-
927
- ### Common commands
928
-
929
- ```bash
930
- npm install # Install dependencies
931
- npm run build # Build TypeScript to dist/ (tsup; ESM + CJS dual output)
932
- npm run build:watch # Watch mode
933
- npm test # Run all tests
934
- npm run test:watch # Watch mode
935
- npm run test:coverage # Run with coverage report
936
- npm run test:ci # Excludes tests/performance/** (used by prepublishOnly)
937
- npm run typecheck # Type checking without emit
938
- npm run lint # ESLint (flat config; @typescript-eslint)
939
- npm run benchmark # Standalone synthetic benchmarks
940
- npm run bench # Vitest performance suite
941
- SKIP_BENCHMARKS=true npm test # Skip perf tests in the main suite
942
- ```
943
-
944
- ### Tooling
945
-
946
- ```bash
947
- npm run audit:plans # Detect plan-doc rot
948
- npx tsx tools/create-dependency-graph/create-dependency-graph.ts # Refresh DEPENDENCY_GRAPH.md
949
- npx tsx tools/chunking-for-files/chunking-for-files.ts split <file> # Split a large file
950
- npx tsx tools/chunking-for-files/chunking-for-files.ts merge <manifest> # Merge back
951
- npx tsx tools/migrate-from-jsonl-to-sqlite/... # JSONL → SQLite migration
952
- ```
953
-
954
- ### Architecture overview
955
-
956
- ```
957
- ┌──────────────────────────────────────────────────────────────────┐
958
- │ Layer 1: ManagerContext (Central Facade — lazy init) │
959
- └──────────────────────────┬───────────────────────────────────────┘
960
-
961
- ┌──────────────────────────┴───────────────────────────────────────┐
962
- │ Layer 2: Domain managers │
963
- │ Core: Entity / Relation / Observation / Hierarchy / Search / │
964
- │ GraphTraversal / Tags / RefIndex │
965
- │ I/O: IOManager / Archive / Compression / Analytics / Audit / │
966
- │ Governance / Freshness / SemanticForget │
967
- │ Search: Ranked / Semantic / Temporal / LLMQueryPlanner / │
968
- │ ActiveRetrievalController │
969
- │ Memory: MemoryEngine / MemoryBackend / ContextWindowManager / │
970
- │ AgentMemory(facade) │
971
- │ Intel: MemoryValidator / TrajectoryCompressor / │
972
- │ ExperienceExtractor / PatternDetector │
973
- │ Theory: ProcedureManager / CausalReasoner / WorldModelManager │
974
- │ Auth: RbacMiddleware / RoleAssignmentStore / AccessTracker │
975
- └──────────────────────────┬───────────────────────────────────────┘
976
-
977
- ┌──────────────────────────┴───────────────────────────────────────┐
978
- │ Layer 3: Storage │
979
- │ GraphStorage (JSONL) or SQLiteStorage (better-sqlite3, FTS5) │
980
- │ Pluggable IMemoryBackend: in-memory or sqlite-backed engine │
981
- └──────────────────────────────────────────────────────────────────┘
982
- ```
983
-
984
- ### Project layout
985
-
986
- ```
987
- src/
988
- ├── index.ts # Public entry point
989
- ├── agent/ # Agent Memory System (managers, decay, salience,
990
- │ # multi-agent, specialised memory types)
991
- ├── core/ # ManagerContext, entity / relation / observation
992
- │ # managers, storage backends, graph traversal
993
- ├── search/ # Ranked / boolean / fuzzy / semantic / hybrid /
994
- │ # temporal / LLM-planned search + indexes
995
- ├── features/ # IOManager, Archive, Compression, Audit,
996
- │ # Governance, Freshness, SemanticForget
997
- ├── cli/ # `memory` / `memoryjs` CLI binary
998
- ├── security/ # PII redactor, ABAC, RLS, API keys
999
- ├── types/ # Shared TypeScript definitions
1000
- ├── utils/ # Caching, schemas, compression adapters, logger
1001
- └── workers/ # Worker pool for CPU-intensive tasks
1002
-
1003
- tests/ # vitest test suite
1004
- ├── unit/ # Per-module unit tests
1005
- ├── integration/ # Cross-module workflows
1006
- ├── edge-cases/ # Boundary conditions
1007
- └── performance/ # Benchmarks (gated by SKIP_BENCHMARKS)
1008
-
1009
- tools/ # Dev utilities (chunker, dep-graph generator,
1010
- # migration, plan-doc audit, etc.)
1011
- ```
1012
-
1013
- See [`docs/architecture/DEPENDENCY_GRAPH.md`](docs/architecture/DEPENDENCY_GRAPH.md)
1014
- for the autogenerated dependency graph and module breakdown.
1015
-
1016
- ## Documentation
1017
-
1018
- Architecture documentation lives under `docs/architecture/`:
1019
-
1020
- - [OVERVIEW.md](docs/architecture/OVERVIEW.md) — high-level project overview
1021
- - [ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md) — technical design
1022
- - [COMPONENTS.md](docs/architecture/COMPONENTS.md) — module-by-module breakdown
1023
- - [DATAFLOW.md](docs/architecture/DATAFLOW.md) — data-flow patterns
1024
- - [API.md](docs/architecture/API.md) — full API reference
1025
- - [DEPENDENCY_GRAPH.md](docs/architecture/DEPENDENCY_GRAPH.md) — module dependencies (auto-generated)
1026
- - [AGENT_MEMORY.md](docs/architecture/AGENT_MEMORY.md) — Agent Memory System design
1027
- - [TEST_COVERAGE.md](docs/architecture/TEST_COVERAGE.md) — test coverage analysis
1028
-
1029
- Other:
1030
-
1031
- - [ARCHITECTURE_DECISIONS.md](docs/development/ARCHITECTURE_DECISIONS.md) — Architecture Decision Records
1032
- - [ROADMAP.md](docs/roadmap/ROADMAP.md) — feature roadmap
1033
- - [CLAUDE.md](CLAUDE.md) — full environment-variable reference + working notes
1034
- - [CHANGELOG.md](CHANGELOG.md) — version-by-version history
1035
-
1036
- ## License
1037
-
1038
- **MIT License** - see [LICENSE](LICENSE)
1039
-
1040
- ## Related
1041
-
1042
- - [@danielsimonjr/memory-mcp](https://github.com/danielsimonjr/memory-mcp) - MCP server built on this library
1043
-
1044
- ---
1045
-
1046
- **Repository:** https://github.com/danielsimonjr/memoryjs
1047
- **NPM:** https://www.npmjs.com/package/@danielsimonjr/memoryjs
1048
- **Issues:** https://github.com/danielsimonjr/memoryjs/issues
1
+ # MemoryJS
2
+
3
+ [![NPM](https://img.shields.io/npm/v/@danielsimonjr/memoryjs.svg)](https://www.npmjs.com/package/@danielsimonjr/memoryjs)
4
+ [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
5
+ [![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
6
+
7
+ A TypeScript knowledge-graph library for AI agents and applications that need
8
+ structured long-term memory. Entities, relations, and observations with
9
+ multiple storage backends (JSONL, SQLite, PostgreSQL), advanced search (BM25, TF-IDF,
10
+ fuzzy, semantic, hybrid, temporal, LLM-planned), graph algorithms, bitemporal
11
+ versioning, RBAC, and a complete Agent Memory System (session lifecycle,
12
+ working / episodic / semantic / procedural memory, decay, salience, consolidation,
13
+ causal reasoning, world-model orchestration).
14
+
15
+ > Powers [@danielsimonjr/memory-mcp](https://www.npmjs.com/package/@danielsimonjr/memory-mcp),
16
+ > a Model Context Protocol server for Claude and other MCP clients. The
17
+ > library and the MCP server share the same core; this package is published
18
+ > independently so non-MCP applications can use it directly.
19
+
20
+ For a runnable CLI: `npx -p @danielsimonjr/memoryjs memory --help`.
21
+
22
+ ## Table of Contents
23
+
24
+ - [Features](#features)
25
+ - [Installation](#installation)
26
+ - [Quick Start](#quick-start)
27
+ - [Core Concepts](#core-concepts)
28
+ - [Storage Options](#storage-options)
29
+ - [Search Capabilities](#search-capabilities)
30
+ - [Graph Algorithms](#graph-algorithms)
31
+ - [Agent Memory System](#agent-memory-system)
32
+ - [API Reference](#api-reference)
33
+ - [Configuration](#configuration)
34
+ - [Development](#development)
35
+ - [Documentation](#documentation)
36
+ - [License](#license)
37
+
38
+ ## Features
39
+
40
+ ### Knowledge graph
41
+
42
+ | Capability | Entry point |
43
+ |---|---|
44
+ | Entity / relation / observation CRUD | `ctx.entityManager`, `ctx.relationManager`, `ctx.observationManager` |
45
+ | Optimistic concurrency control on updates | `entityManager.updateEntity(name, updates, { expectedVersion })` |
46
+ | Bitemporal versioning (entities + observations) | `invalidateEntity()` / `entityAsOf()` / `entityTimeline()` |
47
+ | Temporal relations (validity windows) | `relationManager.invalidateRelation()` / `queryAsOf()` / `timeline()` |
48
+ | Memory versioning with contradiction-driven supersession | `Entity.version` / `parentEntityName` / `rootEntityName` / `supersededBy` |
49
+ | Project scoping | `Entity.projectId` + `MEMORY_DEFAULT_PROJECT_ID` |
50
+ | Two-tier deletion (exact match → semantic fallback) | `ctx.semanticForget` |
51
+ | Hierarchical nesting + traversal | `ctx.hierarchyManager` (ancestors / descendants / subtrees) |
52
+ | Stable named references that survive entity renames | `ctx.refIndex.register()` / `resolve()` |
53
+ | Multi-format import / export | `ctx.ioManager.exportGraph(format)` — JSON, CSV, GraphML, GEXF, DOT, Markdown, Mermaid |
54
+ | W3C Linked Data export | `ctx.ioManager.exportGraph('turtle' \| 'rdf-xml' \| 'json-ld')` |
55
+ | Conversation ingestion (format-agnostic) | `ctx.ioManager.ingest(input, options)` |
56
+
57
+ ### Search & retrieval
58
+
59
+ | Capability | Entry point |
60
+ |---|---|
61
+ | Auto-selecting search with method explanation | `ctx.searchManager.autoSearch(query)` |
62
+ | TF-IDF + BM25 ranked search (incremental indexing) | `ctx.rankedSearch`, `BM25Search` |
63
+ | Boolean (AND / OR / NOT) with AST parser | `ctx.searchManager.booleanSearch()` |
64
+ | Fuzzy matching (Levenshtein, N-gram pre-filtered) | `ctx.searchManager.fuzzySearch()` |
65
+ | Semantic search with pluggable embedding provider | `ctx.semanticSearch` (set `MEMORY_EMBEDDING_PROVIDER`) |
66
+ | Hybrid (semantic + lexical + symbolic) | `new HybridSearchManager(ctx.storage, …).search(query)` |
67
+ | Temporal range queries with natural-language parsing | `ctx.searchManager.searchByTime("last hour")` |
68
+ | LLM-planned natural-language queries | `ctx.queryNaturalLanguage(query, llmProvider?)` |
69
+ | Query diagnostics (`explainPlan`, index health) | `ctx.diagnostics` |
70
+
71
+ ### Graph algorithms
72
+
73
+ | Capability | Entry point |
74
+ |---|---|
75
+ | Shortest path + all paths | `ctx.graphTraversal.findShortestPath()` / `findAllPaths()` |
76
+ | Centrality — degree, betweenness, PageRank, HITS | `ctx.graphTraversal.calculatePageRank()` / `calculateHITS()` |
77
+ | Community detection (Louvain), clique enumeration | `ctx.graphTraversal.findCommunities()` / `findCliques()` |
78
+ | Connected components | `ctx.graphTraversal.findConnectedComponents()` |
79
+
80
+ ### Agent Memory System
81
+
82
+ | Capability | Entry point |
83
+ |---|---|
84
+ | Sessions, working memory, episodic memory | `ctx.agentMemory()` — `startSession` / `addWorkingMemory` / `retrieveForContext` |
85
+ | Turn-aware conversation memory with 4-tier dedup (exact / prefix / Jaccard / semantic) | `ctx.memoryEngine.addTurn()` / `getSessionTurns()` |
86
+ | Decay, salience, freshness | `DecayEngine`, `SalienceEngine`, `ctx.freshnessManager` |
87
+ | Role profiles — `researcher` / `planner` / `executor` / `reviewer` / `coordinator` | `MEMORY_AGENT_ROLE` or `RoleProfileManager.apply(role)` |
88
+ | Memory consolidation, summarization, pattern detection | `ConsolidationPipeline`, `ConsolidationScheduler` |
89
+ | Collaborative synthesis with conflict resolution | `CollaborativeSynthesis.synthesize()` / `resolveConflicts()` |
90
+ | Failure-driven distillation, cognitive load analysis | `FailureDistillation`, `CognitiveLoadAnalyzer` |
91
+ | Procedural memory (executable procedures with feedback refinement) | `ctx.procedureManager.addProcedure()` / `matchProcedure()` / `refineProcedure()` |
92
+ | Active retrieval (iterative query rewriting) | `ctx.activeRetrieval.adaptiveRetrieve()` |
93
+ | Causal reasoning — causes / effects / counterfactuals / cycle detection | `ctx.causalReasoner.findCauses()` / `findEffects()` / `counterfactual()` |
94
+ | World-state orchestrator | `ctx.worldModelManager.getCurrentState()` / `predictOutcome()` |
95
+ | Per-agent persistent journal | `AgentMemoryManager.writeDiary()` / `readDiary()` |
96
+ | Prospective memory (intentions-to-act) | `ctx.prospectiveMemory.schedule()` / `fire()` / `cancel()` |
97
+ | Failure memory (pre-task `applicability_hint` lookup) | `ctx.failureManager.record()` / `lookupForTask()` |
98
+ | Plan memory (hierarchical goal trees) | `ctx.plan.createPlan()` / `pushSubGoal()` / `transitionNode()` |
99
+ | Reflection memory (additive insights with content-hash dedup) | `ctx.reflectionManager.create()` / `getRelevantForSession()` |
100
+ | Heuristic memory (condition → action guidelines) | `ctx.heuristicManager.add()` / `match()` / `reinforce()` |
101
+ | Decision rationale (ADR-style records) | `ctx.decisionManager.propose()` / `accept()` / `supersede()` |
102
+ | Project context (facts / conventions / commands / glossary) | `ctx.projectContextManager.upsert()` / `forContext()` |
103
+ | Tool affordance (outcome-aware tool suggestions) | `ctx.toolAffordanceManager.recordOutcome()` / `suggestTool()` |
104
+ | Trust hierarchy — `ground-truth` / `verified` / `inferred` / `unverified` | `MemorySource.trustLevel` + `inferTrustLevel()` |
105
+
106
+ ### Storage & performance
107
+
108
+ | Capability | Entry point |
109
+ |---|---|
110
+ | JSONL or SQLite backend (FTS5, BM25, WAL mode) | `MEMORY_STORAGE_TYPE=jsonl\|sqlite` |
111
+ | Pluggable Memory Engine backend | `MEMORY_BACKEND=sqlite\|in-memory` |
112
+ | Memory-mapped file loading for large stores | `MEMORY_USE_MMAP=true`, `MEMORY_MMAP_THRESHOLD_BYTES` |
113
+ | Segment-sharded JSONL (FNV-routed N-way shards) | `MEMORY_STORAGE_SEGMENT_COUNT=1..1024` |
114
+ | Columnar observation storage | `JsonlColumnStore` (env-gated) |
115
+ | Tiered index — LRU hot / disk warm / Brotli cold | `LRUHotTier` → `DiskWarmTier` → `BrotliColdTier` via `TieredIndex` |
116
+ | In-memory entity-cache compression | `ctx.compressedEntityCache`, `CompressedMap` |
117
+ | Backup lifecycle (create / list / restore / delete) with symlink-attack guards | `ctx.ioManager` (delegates to `BackupManager`) |
118
+ | Streaming export with Brotli compression | `ctx.streamingExporter` |
119
+ | Entity archival to compressed storage | `ctx.archiveManager` |
120
+ | Duplicate detection + entity merging | `ctx.compressionManager` |
121
+
122
+ ### Governance, security, multi-agent
123
+
124
+ | Capability | Entry point |
125
+ |---|---|
126
+ | Policy enforcement + transactional rollback | `ctx.governanceManager.withTransaction()` / `GovernancePolicy` |
127
+ | Immutable JSONL audit trail | `ctx.auditLog` |
128
+ | Strict-mode attribution enforcer | `CollaborationAuditEnforcer` (requires `agentId` on every mutation) |
129
+ | RBAC — role / permission / matrix / middleware | `ctx.rbacMiddleware.checkPermission()` / `ctx.roleAssignmentStore` |
130
+ | ABAC + row-level security + API-key scoping | `src/security/abac.ts`, `rls.ts`, `apiKeys.ts` |
131
+ | PII redactor (email / SSN / CC / phone / IP) | `new PiiRedactor().redactGraph()` / `redactWithStats()` |
132
+ | Visibility hierarchies (5-level: `private` / `team` / `org` / `shared` / `public`) | `VisibilityResolver.canAccess()` |
133
+ | Time-window + role-gated visibility | `AgentEntity.visibleFrom` / `visibleUntil` / `allowedRoles[]` |
134
+ | Path-traversal protection | `validateFilePath(path, baseDir?, confineToBase=true)` |
135
+
136
+ ### Tooling
137
+
138
+ | Capability | Entry point |
139
+ |---|---|
140
+ | Command-line interface | `npx -p @danielsimonjr/memoryjs memory --help` |
141
+ | End-to-end smoke test against a fresh temp graph (~30 ops) | `memory smoke --keep --verbose` |
142
+ | Diagnostic snapshot + graph health checks | `memory diag` / `memory health` |
143
+ | Orphan + missing-parent + cycle detection with optional repair | `memory check [--apply]` |
144
+ | Rebuild ranked + spell indexes | `memory reindex [--ranked\|--spell]` |
145
+ | Inspect a single entity verbosely | `memory show <name>` |
146
+ | Hierarchy tree (JSON or `--ascii`) | `memory tree [root]` |
147
+ | Search-cache stats / clear | `memory cache stats \| clear` |
148
+ | Interactive REPL | `memory interactive` |
149
+
150
+
151
+ ## Installation
152
+
153
+ ```bash
154
+ npm install @danielsimonjr/memoryjs
155
+ ```
156
+
157
+ ### Requirements
158
+
159
+ - Node.js >= 18.0.0
160
+ - TypeScript >= 5.0 (for development)
161
+
162
+ ## Quick Start
163
+
164
+ ### 1. Initialize storage
165
+
166
+ ```typescript
167
+ import { ManagerContext } from '@danielsimonjr/memoryjs';
168
+
169
+ // JSONL storage (default, human-readable)
170
+ const ctx = new ManagerContext('./memory.jsonl');
171
+
172
+ // Or SQLite (set MEMORY_STORAGE_TYPE=sqlite in the environment)
173
+ const sqliteCtx = new ManagerContext('./memory.db');
174
+ ```
175
+
176
+ ### 2. Create entities
177
+
178
+ ```typescript
179
+ await ctx.entityManager.createEntities([
180
+ {
181
+ name: 'TypeScript',
182
+ entityType: 'language',
183
+ observations: ['A typed superset of JavaScript'],
184
+ tags: ['programming', 'frontend'],
185
+ importance: 8,
186
+ },
187
+ {
188
+ name: 'Node.js',
189
+ entityType: 'runtime',
190
+ observations: ['JavaScript runtime built on V8'],
191
+ tags: ['backend', 'server'],
192
+ },
193
+ ]);
194
+ ```
195
+
196
+ ### 3. Create relations
197
+
198
+ ```typescript
199
+ await ctx.relationManager.createRelations([
200
+ { from: 'TypeScript', to: 'Node.js', relationType: 'runs_on' },
201
+ ]);
202
+ ```
203
+
204
+ ### 4. Search
205
+
206
+ ```typescript
207
+ // Auto-select the best method
208
+ const auto = await ctx.searchManager.autoSearch('JavaScript');
209
+
210
+ // Substring + tag search
211
+ const nodes = await ctx.searchManager.searchNodes('JavaScript');
212
+
213
+ // Boolean (AND / OR / NOT) with an AST parser
214
+ const both = await ctx.searchManager.booleanSearch('TypeScript AND runtime');
215
+
216
+ // Fuzzy (typo-tolerant; N-gram pre-filtered)
217
+ const fuzzy = await ctx.searchManager.fuzzySearch('Typscript', { threshold: 0.7 });
218
+
219
+ // Ranked TF-IDF / BM25
220
+ const ranked = await ctx.rankedSearch.searchNodesRanked('runtime environment',
221
+ undefined, undefined, undefined, 10);
222
+
223
+ // Active retrieval — iterative query rewriting until coverage threshold
224
+ const adaptive = await ctx.activeRetrieval.adaptiveRetrieve({ query: 'memory leak' });
225
+ console.log(adaptive.bestResults, adaptive.bestCoverage, adaptive.rounds);
226
+ ```
227
+
228
+ ### 5. Multi-agent collaboration
229
+
230
+ ```typescript
231
+ // Optimistic concurrency control — fail loudly on stale writes
232
+ try {
233
+ await ctx.entityManager.updateEntity('Alice',
234
+ { importance: 9 },
235
+ { expectedVersion: 3 });
236
+ } catch (e) {
237
+ if (e instanceof Error && e.name === 'VersionConflictError') {
238
+ // Refetch + retry
239
+ }
240
+ }
241
+
242
+ // Synthesize a view across agents + resolve conflicts
243
+ const agent = ctx.agentMemory();
244
+ const synth = await agent.collaborativeSynthesis.synthesize('Alice');
245
+ const winners = agent.collaborativeSynthesis.resolveConflicts(synth, {
246
+ strategy: 'highest_confidence',
247
+ });
248
+
249
+ // Enforce attribution on every mutation
250
+ import { CollaborationAuditEnforcer, AuditLog } from '@danielsimonjr/memoryjs';
251
+ const enforcer = new CollaborationAuditEnforcer(
252
+ ctx.entityManager,
253
+ new AuditLog('./audit.jsonl'),
254
+ );
255
+ await enforcer.createEntities(
256
+ [{ name: 'X', entityType: 't', observations: ['fact'] }],
257
+ 'agent-alice', // throws AttributionRequiredError if agentId is missing
258
+ );
259
+ ```
260
+
261
+ ### 6. Bitemporal versioning
262
+
263
+ ```typescript
264
+ // Mark an entity as no longer valid at a specific time
265
+ await ctx.entityManager.invalidateEntity('OldFact', '2025-12-31T00:00:00Z');
266
+
267
+ // Time-travel query
268
+ const past = await ctx.entityManager.entityAsOf('Alice', '2024-06-15T00:00:00Z');
269
+
270
+ // Per-observation validity windows
271
+ await ctx.observationManager.invalidateObservation(
272
+ 'Alice', 'works at Acme', '2024-12-31T00:00:00Z',
273
+ );
274
+ const obsAtTime = await ctx.observationManager.observationsAsOf(
275
+ 'Alice', '2024-06-15T00:00:00Z',
276
+ );
277
+ ```
278
+
279
+ ### 7. Causal reasoning and world model
280
+
281
+ ```typescript
282
+ // Forward inference — "what does X cause?"
283
+ const effects = await ctx.causalReasoner.findEffects('rain', ['flooding', 'erosion']);
284
+
285
+ // Counterfactual — "what if we remove this edge?"
286
+ const surviving = await ctx.causalReasoner.counterfactual({
287
+ seed: 'rain', removeFrom: 'rain', removeTo: 'flooding', predict: 'flooding',
288
+ });
289
+
290
+ // World-state snapshot + diff
291
+ const before = await ctx.worldModelManager.getCurrentState();
292
+ // ... mutations ...
293
+ const after = await ctx.worldModelManager.getCurrentState();
294
+ const change = ctx.worldModelManager.detectStateChange(before, after);
295
+ ```
296
+
297
+ ### 8. RBAC and PII redaction
298
+
299
+ ```typescript
300
+ // Grant a role
301
+ await ctx.roleAssignmentStore.assign({
302
+ agentId: 'alice', role: 'writer', resourceType: 'entity',
303
+ });
304
+ ctx.rbacMiddleware.checkPermission('alice', 'write', 'entity'); // true
305
+
306
+ // Redact PII from exports / logs
307
+ import { PiiRedactor } from '@danielsimonjr/memoryjs';
308
+ const redactor = new PiiRedactor();
309
+ const cleanGraph = redactor.redactGraph(graph);
310
+ const { text, stats } = redactor.redactWithStats(observation);
311
+ ```
312
+
313
+ ## Core Concepts
314
+
315
+ ### Entity
316
+
317
+ The primary node in the knowledge graph. Required fields are `name`, `entityType`,
318
+ and `observations`; the rest are optional and unlock specific features:
319
+
320
+ ```typescript
321
+ interface Entity {
322
+ name: string; // Unique identifier
323
+ entityType: string; // Classification (person, project, concept, …)
324
+ observations: string[]; // Atomic facts about the entity
325
+ parentId?: string; // Parent entity for hierarchical nesting
326
+ tags?: string[]; // Lowercase tags for categorisation
327
+ importance?: number; // 0–10 scale for prioritisation
328
+ createdAt?: string; // ISO 8601
329
+ lastModified?: string; // ISO 8601
330
+
331
+ // Freshness
332
+ ttl?: number; // Seconds until stale
333
+ confidence?: number; // [0, 1] belief strength
334
+
335
+ // Project scoping + supersession
336
+ projectId?: string; // Multi-project isolation
337
+ version?: number; // Drives optimistic concurrency on updates
338
+ parentEntityName?: string;
339
+ rootEntityName?: string;
340
+ isLatest?: boolean;
341
+ supersededBy?: string;
342
+
343
+ // Memory-engine deduplication
344
+ contentHash?: string; // SHA-256 for O(1) exact-equality dedup
345
+
346
+ // Bitemporal validity (orthogonal to supersession)
347
+ validFrom?: string;
348
+ validUntil?: string;
349
+ observationMeta?: Array<{
350
+ content: string;
351
+ validFrom?: string;
352
+ validUntil?: string;
353
+ recordedAt?: string;
354
+ }>;
355
+ }
356
+ ```
357
+
358
+ ### Relation
359
+
360
+ A directed edge between two entities:
361
+
362
+ ```typescript
363
+ interface Relation {
364
+ from: string; // Source entity name
365
+ to: string; // Target entity name
366
+ relationType: string; // Relationship type (active voice — "depends_on", "wrote")
367
+ }
368
+ ```
369
+
370
+ ### Observation
371
+
372
+ A discrete atomic fact about an entity. Use `addObservations()` to append without
373
+ overwriting; combined with the bitemporal axis and per-observation `validFrom` /
374
+ `validUntil` you can preserve historical state instead of mutating in place.
375
+
376
+ ### ManagerContext
377
+
378
+ The central facade. Construct it with a storage-path string; every manager is
379
+ exposed as a property with lazy initialisation:
380
+
381
+ ```typescript
382
+ const ctx = new ManagerContext('./memory.jsonl');
383
+
384
+ // Core
385
+ ctx.entityManager // Entity CRUD + hierarchy + temporal validity + OCC
386
+ ctx.relationManager // Relation management + temporal invalidation
387
+ ctx.observationManager // Observation CRUD + bitemporal axis
388
+ ctx.hierarchyManager // Tree operations (parent / children / ancestors)
389
+ ctx.searchManager // All search operations including `searchByTime`
390
+ ctx.rankedSearch // TF-IDF / BM25 ranked search
391
+ ctx.graphTraversal // BFS / DFS / shortest path / centrality / communities
392
+ ctx.tagManager // Tag aliases + bulk operations
393
+ ctx.refIndex // Stable name → entity O(1) lookup
394
+
395
+ // Storage + I/O
396
+ ctx.ioManager // Import / export (RDF/Turtle/JSON-LD) / backup / ingest
397
+ ctx.archiveManager // Entity archival
398
+ ctx.compressionManager // Duplicate detection, entity merging
399
+ ctx.analyticsManager // Graph statistics + validation
400
+ ctx.semanticForget // Two-tier deletion with audit
401
+ ctx.governanceManager // Transactions + policy enforcement
402
+ ctx.freshnessManager // TTL / confidence freshness reports
403
+
404
+ // Search extensions
405
+ ctx.semanticSearch // Vector similarity (lazy; needs embedding provider)
406
+ ctx.temporalSearch // Natural-language time-range search
407
+ ctx.activeRetrieval // Iterative query rewriting (no LLM required)
408
+ ctx.queryNaturalLanguage // Convenience wrapper for NL → StructuredQuery
409
+
410
+ // Memory engine + agent system
411
+ ctx.memoryEngine // Turn-aware conversation memory (4-tier dedup)
412
+ ctx.memoryBackend // Pluggable IMemoryBackend (in-memory / sqlite)
413
+ ctx.agentMemory() // Agent Memory System facade
414
+
415
+ // Memory intelligence
416
+ ctx.memoryValidator // Consistency / contradictions / temporal-order checks
417
+ ctx.trajectoryCompressor // Distill / abstract / merge redundant trajectories
418
+
419
+ // Specialised memory managers
420
+ ctx.procedureManager // Executable procedure memory + feedback refinement
421
+ ctx.causalReasoner // findCauses / findEffects / counterfactual
422
+ ctx.worldModelManager // State-snapshot orchestrator + diff
423
+ ctx.heuristicManager // Condition → action guidelines
424
+ ctx.decisionManager // ADR-style decision records
425
+ ctx.projectContextManager
426
+ ctx.toolAffordanceManager
427
+ ctx.spellChecker // Vocabulary-driven spell suggestions
428
+ ctx.exclusionManager // `do_not_remember` content-pattern rules
429
+ ctx.observationDedupManager
430
+
431
+ // Access control + audit
432
+ ctx.roleAssignmentStore
433
+ ctx.rbacMiddleware
434
+ ```
435
+
436
+ ## Storage Options
437
+
438
+ ### Comparison
439
+
440
+ | Feature | JSONL (Default) | SQLite (better-sqlite3) |
441
+ |---------|-----------------|-------------------------|
442
+ | Format | Human-readable text | Native binary database |
443
+ | Transactions | Basic | Full ACID with WAL mode |
444
+ | Full-Text Search | Basic | FTS5 with BM25 ranking |
445
+ | Performance | Good | 3-10x faster |
446
+ | Concurrency | Single-threaded | Thread-safe with async-mutex |
447
+ | Best For | Small graphs, debugging | Large graphs (10k+ entities) |
448
+
449
+ ### JSONL Storage
450
+
451
+ ```typescript
452
+ const ctx = new ManagerContext('./memory.jsonl');
453
+ ```
454
+
455
+ Features:
456
+ - Human-readable line-delimited JSON
457
+ - In-memory caching with write-through invalidation
458
+ - Atomic writes via temp file + rename
459
+ - Backward compatibility for legacy formats
460
+
461
+ ### SQLite Storage
462
+
463
+ ```typescript
464
+ // Set MEMORY_STORAGE_TYPE=sqlite environment variable
465
+ const ctx = new ManagerContext('./memory.db');
466
+ ```
467
+
468
+ Features:
469
+ - FTS5 full-text search with BM25 ranking
470
+ - WAL mode for better concurrency
471
+ - Referential integrity with ON DELETE CASCADE
472
+ - ACID transactions
473
+
474
+ ### Storage Files
475
+
476
+ When using JSONL, related files are automatically created:
477
+
478
+ ```
479
+ /your/data/directory/
480
+ ├── memory.jsonl # Main knowledge graph
481
+ ├── memory-saved-searches.jsonl # Saved search queries
482
+ ├── memory-tag-aliases.jsonl # Tag synonym mappings
483
+ └── .backups/ # Timestamped backups
484
+ ```
485
+
486
+ ## Search Capabilities
487
+
488
+ ### Available methods
489
+
490
+ | Method | Description | Use case |
491
+ |---|---|---|
492
+ | `searchManager.autoSearch(query)` | Auto-selects the best method + explains why | Default entry point |
493
+ | `searchManager.searchNodes(query)` | Substring + tag matching | Simple queries |
494
+ | `searchManager.booleanSearch(query)` | AND / OR / NOT with AST parser | Complex filters |
495
+ | `searchManager.fuzzySearch(query)` | Levenshtein + N-gram pre-filter | Typo tolerance |
496
+ | `searchManager.searchByTime(query)` | Natural-language time ranges (`chrono-node`) | "last hour", "since 2024-01-01" |
497
+ | `rankedSearch.searchNodesRanked(query, ...)` | TF-IDF / BM25 ranking | Most-relevant ordering |
498
+ | `semanticSearch.search(query)` | Vector similarity (requires embedding provider) | Semantic queries |
499
+ | `new HybridSearchManager(...).search(query)` | Semantic + lexical + symbolic | Multi-signal ranking |
500
+ | `activeRetrieval.adaptiveRetrieve({ query })` | Iterative query rewriting + coverage check | Adaptive refinement |
501
+ | `queryNaturalLanguage(query, llmProvider?)` | LLM-planned NL decomposition (fallback: keyword) | Free-text queries |
502
+ | `causalReasoner.findEffects(seed, candidates)` | Subgraph traversal over `causes`/`enables`/`prevents` | Causal inference |
503
+
504
+ ### Auto-search
505
+
506
+ ```typescript
507
+ const auto = await ctx.searchManager.autoSearch('TypeScript runtime');
508
+ console.log(auto.selectedMethod, auto.selectionReason);
509
+ console.log(auto.results);
510
+ ```
511
+
512
+ ### Boolean search
513
+
514
+ ```typescript
515
+ // AND — both terms must match
516
+ await ctx.searchManager.booleanSearch('TypeScript AND runtime');
517
+
518
+ // OR — either term matches
519
+ await ctx.searchManager.booleanSearch('frontend OR backend');
520
+
521
+ // NOT — exclude term
522
+ await ctx.searchManager.booleanSearch('JavaScript NOT browser');
523
+
524
+ // Grouping
525
+ await ctx.searchManager.booleanSearch('(TypeScript OR JavaScript) AND server');
526
+ ```
527
+
528
+ ### Fuzzy search
529
+
530
+ ```typescript
531
+ // Typo-tolerant; threshold 0–1 (higher = stricter)
532
+ await ctx.searchManager.fuzzySearch('Typscript', { threshold: 0.7 });
533
+ ```
534
+
535
+ ### Hybrid search
536
+
537
+ `HybridSearchManager` combines three signal layers. Instantiate it directly —
538
+ it is exported from the package but not wired onto `ManagerContext` because the
539
+ weighting + filter configuration is application-specific.
540
+
541
+ ```typescript
542
+ import { HybridSearchManager } from '@danielsimonjr/memoryjs';
543
+
544
+ const hybrid = new HybridSearchManager(ctx.storage, ctx.rankedSearch);
545
+ const results = await hybrid.search('programming concepts', {
546
+ weights: {
547
+ semantic: 0.5, // Vector similarity (requires embeddings)
548
+ lexical: 0.3, // TF-IDF text matching
549
+ symbolic: 0.2, // Metadata (tags, importance, type)
550
+ },
551
+ filters: {
552
+ entityTypes: ['concept'],
553
+ minImportance: 5,
554
+ tags: ['programming'],
555
+ },
556
+ });
557
+ ```
558
+
559
+ ## Graph Algorithms
560
+
561
+ ### Path finding
562
+
563
+ ```typescript
564
+ // Shortest path between entities (BFS)
565
+ const result = await ctx.graphTraversal.findShortestPath('A', 'Z');
566
+ console.log(result.path); // ['A', 'B', 'C', 'Z']
567
+ console.log(result.distance); // 3
568
+
569
+ // All paths with a depth bound
570
+ const all = await ctx.graphTraversal.findAllPaths('A', 'Z', { maxDepth: 5 });
571
+ // → [['A', 'B', 'Z'], ['A', 'C', 'D', 'Z'], …]
572
+ ```
573
+
574
+ ### Centrality
575
+
576
+ Each centrality algorithm has its own method (no unified `getCentrality()` — the
577
+ algorithms have different parameters and return shapes):
578
+
579
+ ```typescript
580
+ const degree = await ctx.graphTraversal.calculateDegreeCentrality();
581
+ const between = await ctx.graphTraversal.calculateBetweennessCentrality();
582
+ const pagerank = await ctx.graphTraversal.calculatePageRank();
583
+ const hits = await ctx.graphTraversal.calculateHITS();
584
+ // Each returns a Map<entityName, score>
585
+ ```
586
+
587
+ ### Connected components, communities, cliques
588
+
589
+ ```typescript
590
+ const components = await ctx.graphTraversal.findConnectedComponents();
591
+ const communities = await ctx.graphTraversal.findCommunities(); // Louvain
592
+ const cliques = await ctx.graphTraversal.findCliques(/* minSize */ 3);
593
+ ```
594
+
595
+ ### Traversal
596
+
597
+ ```typescript
598
+ // Returns a TraversalResult { visited, edges, depthMap }
599
+ const bfs = ctx.graphTraversal.bfs('startNode', { maxDepth: 3 });
600
+ const dfs = ctx.graphTraversal.dfs('startNode');
601
+ ```
602
+
603
+ ## Agent Memory System
604
+
605
+ A memory system for AI agents with session lifecycle, working / episodic /
606
+ semantic / procedural memory, decay, salience, multi-agent visibility, and
607
+ specialised memory types (prospective, failure, plan, reflection, heuristic,
608
+ decision rationale, project context, tool affordance).
609
+
610
+ ### Key components
611
+
612
+ | Component | Purpose |
613
+ |---|---|
614
+ | `AgentMemoryManager` | Unified facade for the system |
615
+ | `SessionManager` | Session lifecycle (start / end / restore / checkpoint) |
616
+ | `WorkingMemoryManager` | Short-term, session-scoped memories with promotion |
617
+ | `EpisodicMemoryManager` | Timeline-based event memories |
618
+ | `DecayEngine` | Time-based importance decay |
619
+ | `SalienceEngine` | Context-aware memory scoring |
620
+ | `MultiAgentMemoryManager` | Cross-agent memory with visibility controls |
621
+ | `ConflictResolver` | Resolution strategies for concurrent updates |
622
+
623
+ ### Quick start
624
+
625
+ ```typescript
626
+ import { ManagerContext } from '@danielsimonjr/memoryjs';
627
+
628
+ const ctx = new ManagerContext('./memory.jsonl');
629
+ const agent = ctx.agentMemory();
630
+
631
+ // Start a session
632
+ const session = await agent.startSession({ agentId: 'my-agent' });
633
+
634
+ // Add working memory
635
+ await agent.addWorkingMemory({
636
+ sessionId: session.name,
637
+ content: 'User prefers dark mode',
638
+ importance: 7,
639
+ });
640
+
641
+ // Episodic event
642
+ await agent.createEpisode('Completed onboarding flow', {
643
+ sessionId: session.name,
644
+ importance: 8,
645
+ });
646
+
647
+ // Retrieve context for an LLM prompt (token-budgeted)
648
+ const context = await agent.retrieveForContext({
649
+ maxTokens: 2000,
650
+ includeEpisodic: true,
651
+ });
652
+
653
+ // End session
654
+ await agent.endSession(session.name);
655
+ ```
656
+
657
+ ### Memory types
658
+
659
+ ```typescript
660
+ type MemoryType =
661
+ | 'working' // Short-term, session-scoped; may be promoted
662
+ | 'episodic' // Timeline-based event memories
663
+ | 'semantic' // Long-term factual knowledge
664
+ | 'procedural' // Executable procedures with feedback refinement
665
+ | 'prospective' // Intentions-to-act at a future time / event / condition
666
+ | 'failure' // Pre-task failure lookup keyed on applicability_hint
667
+ | 'plan' // Hierarchical goal trees with sub-tasks + acceptance criteria
668
+ | 'reflection' // Additive derived insights with content-hash dedup
669
+ | 'heuristic'; // Condition → action guidelines
670
+ ```
671
+
672
+ Each specialised type has a dedicated manager on `ManagerContext` —
673
+ `ctx.prospectiveMemory`, `ctx.failureManager`, `ctx.plan`, `ctx.reflectionManager`,
674
+ `ctx.heuristicManager`, `ctx.decisionManager`, `ctx.projectContextManager`,
675
+ `ctx.toolAffordanceManager`.
676
+
677
+ **Trust hierarchy mixin**: every `MemorySource` may carry an optional
678
+ categorical `trustLevel`: `'ground-truth' | 'verified' | 'inferred' | 'unverified'`.
679
+ Used by the `'trust_level'` `ConflictStrategy` (with recency tiebreak) and
680
+ backfilled from `method` + `reliability` via `inferTrustLevel(source)`.
681
+
682
+ ### Decay
683
+
684
+ Memories naturally decay over time unless reinforced:
685
+
686
+ ```typescript
687
+ const agent = ctx.agentMemory({
688
+ decay: {
689
+ halfLifeHours: 168, // One-week half-life
690
+ minImportance: 0.1, // Never fully forget
691
+ },
692
+ enableAutoDecay: true,
693
+ });
694
+
695
+ // Reinforce a memory
696
+ await agent.confirmMemory('memory_name', 0.1); // Boost confidence
697
+ await agent.promoteMemory('memory_name', 'episodic'); // Promote to long-term
698
+ ```
699
+
700
+ ### Multi-agent
701
+
702
+ ```typescript
703
+ // Register an agent
704
+ agent.registerAgent('agent_1', {
705
+ name: 'Research Agent',
706
+ type: 'llm',
707
+ trustLevel: 0.8,
708
+ capabilities: ['read', 'write'],
709
+ });
710
+
711
+ // Create with visibility (private / team / org / shared / public)
712
+ await agent.addWorkingMemory({
713
+ sessionId: session.name,
714
+ content: 'Shared insight',
715
+ visibility: 'shared',
716
+ ownerAgentId: 'agent_1',
717
+ });
718
+
719
+ // Cross-agent search
720
+ const results = await agent.searchCrossAgent('agent_2', 'query');
721
+ ```
722
+
723
+ ## API Reference
724
+
725
+ Method tables below cover the most-used surfaces. For the full surface use
726
+ your IDE's go-to-definition or read `dist/index.d.ts`.
727
+
728
+ ### EntityManager
729
+
730
+ | Method | Description |
731
+ |---|---|
732
+ | `createEntities(entities)` | Create multiple entities in one batch |
733
+ | `deleteEntities(names)` | Delete entities by name |
734
+ | `getEntity(name, options?)` | Get one entity (with optional access tracking) |
735
+ | `updateEntity(name, updates, { expectedVersion? })` | Partial update; opt-in optimistic concurrency |
736
+ | `addTags(name, tags)` / `removeTags(name, tags)` | Tag management |
737
+ | `setImportance(name, score)` | Set importance (0–10) |
738
+ | `getVersionChain(name)` / `getLatestVersion(name)` | Supersession chains |
739
+ | `invalidateEntity(name, ended?)` | Set `validUntil` (bitemporal) |
740
+ | `entityAsOf(name, asOf)` | Time-travel query |
741
+ | `entityTimeline(name)` | Versions sorted by `validFrom` |
742
+
743
+ ### RelationManager
744
+
745
+ | Method | Description |
746
+ |---|---|
747
+ | `createRelations(relations)` | Create multiple relations |
748
+ | `getRelations(entityName)` | Incoming + outgoing relations |
749
+ | `deleteRelations(relations)` | Delete specific relations |
750
+ | `invalidateRelation(from, type, to, ended?)` | Set `validUntil` on a relation |
751
+ | `queryAsOf(entity, asOf, options?)` | Relations valid at time T |
752
+ | `timeline(entity, options?)` | Chronological history |
753
+
754
+ ### ObservationManager
755
+
756
+ | Method | Description |
757
+ |---|---|
758
+ | `addObservations(adds, dedupOptions?)` | Add observations (optional dedup) |
759
+ | `deleteObservations(deletions)` | Remove specific observations |
760
+ | `getObservationsFor(entityName)` | Read observations (column-store-aware) |
761
+ | `invalidateObservation(entity, content, ended?)` | Set per-observation `validUntil` |
762
+ | `observationsAsOf(entity, asOf)` | Observations valid at time T |
763
+
764
+ ### SearchManager (`ctx.searchManager`)
765
+
766
+ | Method | Description |
767
+ |---|---|
768
+ | `searchNodes(query, options?)` | Substring + tag matching |
769
+ | `booleanSearch(query, options?)` | AND / OR / NOT with AST |
770
+ | `fuzzySearch(query, options?)` | Levenshtein + N-gram pre-filter |
771
+ | `searchByTime(query, options?)` | Natural-language time ranges |
772
+ | `autoSearch(query, limit?)` | Auto-select best method; returns `selectedMethod` + `selectionReason` |
773
+ | `openNodes(names)` | Bulk-fetch entities by name |
774
+
775
+ For ranked search use `ctx.rankedSearch.searchNodesRanked(query, tags?, min?, max?, limit?)`.
776
+ For hybrid search, instantiate `HybridSearchManager` directly.
777
+
778
+ ### IOManager (`ctx.ioManager`)
779
+
780
+ | Method | Description |
781
+ |---|---|
782
+ | `exportGraph(graph, format)` | Export to `json` / `csv` / `graphml` / `gexf` / `dot` / `markdown` / `mermaid` / `turtle` / `rdf-xml` / `json-ld` |
783
+ | `exportGraphWithCompression(graph, format, options?)` | Brotli-compressed export |
784
+ | `importGraph(format, data, options?)` | Import with merge strategies (`replace` / `skip` / `merge` / `fail`) |
785
+ | `ingest(input, options?)` | Conversation ingestion pipeline (format-agnostic) |
786
+ | `splitTranscript(content, options?)` | Split multi-session transcripts |
787
+ | `visualizeGraph(options?)` | Interactive HTML visualisation |
788
+ | `createBackup(options?)` / `restoreFromBackup(path)` / `deleteBackup(path)` / `cleanOldBackups(keepCount?)` | Backup lifecycle |
789
+
790
+ ### GraphTraversal (`ctx.graphTraversal`)
791
+
792
+ | Method | Description |
793
+ |---|---|
794
+ | `findShortestPath(from, to)` | BFS shortest path |
795
+ | `findAllPaths(from, to, options?)` | All paths with depth bound |
796
+ | `findConnectedComponents()` | Isolated subgraphs |
797
+ | `findCliques(minSize?)` | Maximal cliques |
798
+ | `findCommunities()` | Louvain community detection |
799
+ | `calculateDegreeCentrality()` / `calculateBetweennessCentrality()` / `calculatePageRank()` / `calculateHITS()` | Centrality (each returns `Map<entityName, score>`) |
800
+ | `bfs(start, options?)` / `dfs(start, options?)` | Returns `TraversalResult` |
801
+
802
+ ### CausalReasoner (`ctx.causalReasoner`)
803
+
804
+ | Method | Description |
805
+ |---|---|
806
+ | `findEffects(cause, candidates, maxDepth?)` | Forward inference; sorted by causal-strength product |
807
+ | `findCauses(effect, candidates, maxDepth?)` | Backward inference (symmetric inverse) |
808
+ | `counterfactual({ seed, removeFrom, removeTo, predict })` | Pure: doesn't mutate the graph |
809
+ | `detectCycles(seed, maxDepth?)` | Depth-bounded DFS over causal subgraph |
810
+
811
+ ### ProcedureManager (`ctx.procedureManager`)
812
+
813
+ | Method | Description |
814
+ |---|---|
815
+ | `addProcedure({ steps, ... })` | Persist a procedure; auto-generates id |
816
+ | `getProcedure(id)` / `getStep(id, order)` / `getNextStep(id, order)` | Access |
817
+ | `openSequencer(id)` | Stateful execution cursor with fallback support |
818
+ | `matchProcedure(context, candidates, threshold?)` | Token-overlap match |
819
+ | `refineProcedure(id, { succeeded, notes? })` | EWMA success-rate update |
820
+
821
+ ### WorldModelManager (`ctx.worldModelManager`)
822
+
823
+ | Method | Description |
824
+ |---|---|
825
+ | `getCurrentState()` | Snapshot from the live graph (capped at `maxSnapshotSize`) |
826
+ | `validateFact(observation, entityName)` | Delegates to `MemoryValidator` if wired |
827
+ | `predictOutcome(action, candidates)` | Delegates to `CausalReasoner.findEffects` |
828
+ | `detectStateChange(before, after)` | Pure snapshot diff |
829
+
830
+ ### ActiveRetrievalController (`ctx.activeRetrieval`)
831
+
832
+ | Method | Description |
833
+ |---|---|
834
+ | `shouldRetrieve(context)` | Cost heuristic; rejects empty / over-budget |
835
+ | `adaptiveRetrieve(context)` | Iterative rewrite + retrieve until coverage threshold |
836
+
837
+ ### CollaborativeSynthesis (`ctx.agentMemory().collaborativeSynthesis`)
838
+
839
+ | Method | Description |
840
+ |---|---|
841
+ | `synthesize(seedEntity, context?)` | BFS + salience scoring; surfaces multi-agent conflicts |
842
+ | `resolveConflicts(result, policy)` | Pick winners per `most_recent` / `highest_confidence` / `highest_score` / `trusted_agent` |
843
+
844
+ ### MemoryValidator (`ctx.memoryValidator`)
845
+
846
+ | Method | Description |
847
+ |---|---|
848
+ | `validateConsistency(newObs, existing)` | Composite duplicate / semantic / low-confidence check |
849
+ | `detectContradictions(entity)` | Delegates to `ContradictionDetector` |
850
+ | `repairWithResolver(entity, competing, resolver, contradiction?, options?)` | Apply `ConflictResolver` strategies |
851
+ | `validateTemporalOrder(observations)` | `[T=ISO]` ordering check |
852
+ | `calculateReliability(entity)` | Confidence × confirmation × age penalty |
853
+
854
+ ### RBAC (`ctx.rbacMiddleware`, `ctx.roleAssignmentStore`)
855
+
856
+ | Method | Description |
857
+ |---|---|
858
+ | `rbacMiddleware.checkPermission(agentId, action, resourceType, resourceName?, now?)` | Returns `true` / `false`; falls back to `defaultRole` (default `reader`) |
859
+ | `roleAssignmentStore.assign({ agentId, role, resourceType?, scope?, validFrom?, validUntil? })` | Grant a role |
860
+ | `roleAssignmentStore.revoke(agentId, role, resourceType?)` | Remove a grant |
861
+ | `roleAssignmentStore.listActive(agentId, now?)` | Active grants at a point in time |
862
+
863
+ ### PiiRedactor
864
+
865
+ | Method | Description |
866
+ |---|---|
867
+ | `redact(text)` | Apply patterns; returns redacted string |
868
+ | `redactWithStats(text)` | Returns `{ text, stats: { totalRedactedBytes, countsByPattern } }` |
869
+ | `redactGraph(graph)` | Apply to every observation in a graph-shaped object |
870
+
871
+ ### Specialised memory managers
872
+
873
+ | Manager | Path | Key methods |
874
+ |---|---|---|
875
+ | Prospective | `ctx.prospectiveMemory` | `schedule()` / `fire()` / `cancel()` / `expireDueIntentions()` / `getPending()` |
876
+ | Failure | `ctx.failureManager` | `record()` / `lookupForTask()` / `markResolved()` / `getAll()` |
877
+ | Plan | `ctx.plan` | `createPlan()` / `pushSubGoal()` / `transitionNode()` / `markPlanComplete()` / `getCurrentPath()` |
878
+ | Reflection | `ctx.reflectionManager` | `create()` / `list()` / `getRelevantForSession()` / `archive()` |
879
+ | Heuristic | `ctx.heuristicManager` | `add()` / `match()` / `reinforce()` / `recordContradiction()` / `detectConflicts()` |
880
+ | Decision | `ctx.decisionManager` | `propose()` / `accept()` / `reject()` / `supersede()` / `findByContext()` / `getChain()` |
881
+ | Project context | `ctx.projectContextManager` | `upsert()` / `appendFact()` / `appendCommand()` / `forContext()` |
882
+ | Tool affordance | `ctx.toolAffordanceManager` | `recordOutcome()` / `suggestTool()` / `rollingStats()` |
883
+ | Exclusion | `ctx.exclusionManager` | `add()` / `list()` / `remove()` / `check()` / `findMatchingMemories()` |
884
+ | Observation dedup | `ctx.observationDedupManager` | `findDuplicateObservations()` / `findJaccardDuplicates()` |
885
+ | Spell | `ctx.spellChecker` | `suggest()` / `rebuild()` / `vocabularySize()` |
886
+
887
+ ## Configuration
888
+
889
+ memoryjs is configured almost entirely through environment variables. The
890
+ table below lists the most-used; the full reference (decay / salience /
891
+ context-window / freshness / RBAC / mmap / segment / SQLite-pool / partial-index
892
+ knobs) lives in [CLAUDE.md](CLAUDE.md#environment-variables).
893
+
894
+ | Variable | Description | Default |
895
+ |---|---|---|
896
+ | `MEMORY_STORAGE_TYPE` | Storage backend: `jsonl` or `sqlite` | `jsonl` |
897
+ | `MEMORY_FILE_PATH` | Override storage file path | (per `ManagerContext` ctor) |
898
+ | `MEMORY_BACKEND` | Pluggable Memory-Engine backend: `sqlite` or `in-memory` | `sqlite` |
899
+ | `MEMORY_EMBEDDING_PROVIDER` | Embedding provider: `openai`, `local`, or `none` | `local` |
900
+ | `MEMORY_OPENAI_API_KEY` | OpenAI API key (required when provider is `openai`) | — |
901
+ | `MEMORY_AUTO_INDEX_EMBEDDINGS` | Auto-build embedding index on entity create | `false` |
902
+ | `MEMORY_AUTO_DECAY` | Enable background memory decay | `false` |
903
+ | `MEMORY_DECAY_HALF_LIFE_HOURS` | Half-life for importance decay | `168` |
904
+ | `MEMORY_GOVERNANCE_ENABLED` | Enable `GovernanceManager` policy enforcement | `false` |
905
+ | `MEMORY_AUDIT_LOG_FILE` | Path for the audit JSONL trail | — |
906
+ | `MEMORY_AGENT_ROLE` | Apply a built-in role profile (`researcher` / `planner` / `executor` / `reviewer` / `coordinator`) | — |
907
+ | `MEMORY_VALIDATE_ON_STORE` | Run `MemoryValidator` before observation writes | `false` |
908
+ | `MEMORY_AUDIT_ATTRIBUTION_REQUIRED` | `CollaborationAuditEnforcer` strict mode | `false` |
909
+ | `MEMORY_RBAC_ENABLED` | Wire `RbacMiddleware` into `GovernancePolicy` | `false` |
910
+ | `MEMORY_DEFAULT_VISIBILITY` | Default `AgentEntity.visibility` | `private` |
911
+ | `MEMORY_USE_MMAP` | Use mmap for `GraphStorage.loadFromDisk` (strict `'true'` literal match) | `false` |
912
+ | `MEMORY_MMAP_THRESHOLD_BYTES` | Minimum file size to trigger the mmap path; `0` = always | `104857600` |
913
+ | `MEMORY_STORAGE_SEGMENT_COUNT` | FNV-routed JSONL shard count (1–1024) | `1` (off) |
914
+ | `MEMORY_SQLITE_READ_POOL_SIZE` | Read-connection pool size for `SQLiteStorage` | `4` |
915
+ | `MEMORY_SQLITE_AUTO_INDEX` | Enable `PartialIndexAdvisor`-driven automatic partial-index DDL | `false` |
916
+ | `SKIP_BENCHMARKS` | Skip perf-benchmark tests | `false` |
917
+ | `LOG_LEVEL` | `debug` / `info` / `warn` / `error` | (none) |
918
+
919
+ ## Development
920
+
921
+ ### Prerequisites
922
+
923
+ - Node.js 18+
924
+ - npm 9+
925
+ - TypeScript 5.0+
926
+
927
+ ### Common commands
928
+
929
+ ```bash
930
+ npm install # Install dependencies
931
+ npm run build # Build TypeScript to dist/ (tsup; ESM + CJS dual output)
932
+ npm run build:watch # Watch mode
933
+ npm test # Run all tests
934
+ npm run test:watch # Watch mode
935
+ npm run test:coverage # Run with coverage report
936
+ npm run test:ci # Excludes tests/performance/** (used by prepublishOnly)
937
+ npm run typecheck # Type checking without emit
938
+ npm run lint # ESLint (flat config; @typescript-eslint)
939
+ npm run benchmark # Standalone synthetic benchmarks
940
+ npm run bench # Vitest performance suite
941
+ SKIP_BENCHMARKS=true npm test # Skip perf tests in the main suite
942
+ ```
943
+
944
+ ### Tooling
945
+
946
+ ```bash
947
+ npm run audit:plans # Detect plan-doc rot
948
+ npx tsx tools/create-dependency-graph/create-dependency-graph.ts # Refresh DEPENDENCY_GRAPH.md
949
+ npx tsx tools/chunking-for-files/chunking-for-files.ts split <file> # Split a large file
950
+ npx tsx tools/chunking-for-files/chunking-for-files.ts merge <manifest> # Merge back
951
+ npx tsx tools/migrate-from-jsonl-to-sqlite/... # JSONL → SQLite migration
952
+ ```
953
+
954
+ ### Architecture overview
955
+
956
+ ```
957
+ ┌──────────────────────────────────────────────────────────────────┐
958
+ │ Layer 1: ManagerContext (Central Facade — lazy init) │
959
+ └──────────────────────────┬───────────────────────────────────────┘
960
+
961
+ ┌──────────────────────────┴───────────────────────────────────────┐
962
+ │ Layer 2: Domain managers │
963
+ │ Core: Entity / Relation / Observation / Hierarchy / Search / │
964
+ │ GraphTraversal / Tags / RefIndex │
965
+ │ I/O: IOManager / Archive / Compression / Analytics / Audit / │
966
+ │ Governance / Freshness / SemanticForget │
967
+ │ Search: Ranked / Semantic / Temporal / LLMQueryPlanner / │
968
+ │ ActiveRetrievalController │
969
+ │ Memory: MemoryEngine / MemoryBackend / ContextWindowManager / │
970
+ │ AgentMemory(facade) │
971
+ │ Intel: MemoryValidator / TrajectoryCompressor / │
972
+ │ ExperienceExtractor / PatternDetector │
973
+ │ Theory: ProcedureManager / CausalReasoner / WorldModelManager │
974
+ │ Auth: RbacMiddleware / RoleAssignmentStore / AccessTracker │
975
+ └──────────────────────────┬───────────────────────────────────────┘
976
+
977
+ ┌──────────────────────────┴───────────────────────────────────────┐
978
+ │ Layer 3: Storage │
979
+ │ GraphStorage (JSONL) or SQLiteStorage (better-sqlite3, FTS5) │
980
+ │ Pluggable IMemoryBackend: in-memory or sqlite-backed engine │
981
+ └──────────────────────────────────────────────────────────────────┘
982
+ ```
983
+
984
+ ### Project layout
985
+
986
+ ```
987
+ src/
988
+ ├── index.ts # Public entry point
989
+ ├── agent/ # Agent Memory System (managers, decay, salience,
990
+ │ # multi-agent, specialised memory types)
991
+ ├── core/ # ManagerContext, entity / relation / observation
992
+ │ # managers, storage backends, graph traversal
993
+ ├── search/ # Ranked / boolean / fuzzy / semantic / hybrid /
994
+ │ # temporal / LLM-planned search + indexes
995
+ ├── features/ # IOManager, Archive, Compression, Audit,
996
+ │ # Governance, Freshness, SemanticForget
997
+ ├── cli/ # `memory` / `memoryjs` CLI binary
998
+ ├── security/ # PII redactor, ABAC, RLS, API keys
999
+ ├── types/ # Shared TypeScript definitions
1000
+ ├── utils/ # Caching, schemas, compression adapters, logger
1001
+ └── workers/ # Worker pool for CPU-intensive tasks
1002
+
1003
+ tests/ # vitest test suite
1004
+ ├── unit/ # Per-module unit tests
1005
+ ├── integration/ # Cross-module workflows
1006
+ ├── edge-cases/ # Boundary conditions
1007
+ └── performance/ # Benchmarks (gated by SKIP_BENCHMARKS)
1008
+
1009
+ tools/ # Dev utilities (chunker, dep-graph generator,
1010
+ # migration, plan-doc audit, etc.)
1011
+ ```
1012
+
1013
+ See [`docs/architecture/DEPENDENCY_GRAPH.md`](docs/architecture/DEPENDENCY_GRAPH.md)
1014
+ for the autogenerated dependency graph and module breakdown.
1015
+
1016
+ ## Documentation
1017
+
1018
+ Architecture documentation lives under `docs/architecture/`:
1019
+
1020
+ - [OVERVIEW.md](docs/architecture/OVERVIEW.md) — high-level project overview
1021
+ - [ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md) — technical design
1022
+ - [COMPONENTS.md](docs/architecture/COMPONENTS.md) — module-by-module breakdown
1023
+ - [DATAFLOW.md](docs/architecture/DATAFLOW.md) — data-flow patterns
1024
+ - [API.md](docs/architecture/API.md) — full API reference
1025
+ - [DEPENDENCY_GRAPH.md](docs/architecture/DEPENDENCY_GRAPH.md) — module dependencies (auto-generated)
1026
+ - [AGENT_MEMORY.md](docs/architecture/AGENT_MEMORY.md) — Agent Memory System design
1027
+ - [TEST_COVERAGE.md](docs/architecture/TEST_COVERAGE.md) — test coverage analysis
1028
+
1029
+ Other:
1030
+
1031
+ - [ARCHITECTURE_DECISIONS.md](docs/development/ARCHITECTURE_DECISIONS.md) — Architecture Decision Records
1032
+ - [ROADMAP.md](docs/roadmap/ROADMAP.md) — feature roadmap
1033
+ - [CLAUDE.md](CLAUDE.md) — full environment-variable reference + working notes
1034
+ - [CHANGELOG.md](CHANGELOG.md) — version-by-version history
1035
+
1036
+ ## License
1037
+
1038
+ **MIT License** - see [LICENSE](LICENSE)
1039
+
1040
+ ## Related
1041
+
1042
+ - [@danielsimonjr/memory-mcp](https://github.com/danielsimonjr/memory-mcp) - MCP server built on this library
1043
+
1044
+ ---
1045
+
1046
+ **Repository:** https://github.com/danielsimonjr/memoryjs
1047
+ **NPM:** https://www.npmjs.com/package/@danielsimonjr/memoryjs
1048
+ **Issues:** https://github.com/danielsimonjr/memoryjs/issues