@levalicious/server-memory 0.0.14 → 0.0.15

package/README.md

@@ -1,6 +1,25 @@
 # Knowledge Graph Memory Server
 
-A basic implementation of persistent memory using a local knowledge graph. This lets Claude remember information about the user across chats.
+A persistent knowledge graph with binary storage, PageRank-based ranking, and Maximum Entropy Random Walk (MERW) exploration. Designed as an MCP server for use with LLM agents.
+
+## Storage Format
+
+The knowledge graph is stored in two binary files using a custom mmap-backed arena allocator:
+
+- **`<base>.graph`** — Entity records (72 bytes each), adjacency blocks, and node log
+- **`<base>.strings`** — Interned, refcounted string table
+
+This replaces the original JSONL format. The binary format supports O(1) entity lookup, POSIX file locking for concurrent access, and in-place mutation without rewriting the entire file.
+
+> [!NOTE]
+> **Migrating from JSONL:** If you have an existing `.json` knowledge graph, use the migration script:
+> ```sh
+> npx tsx scripts/migrate-jsonl.ts [path/to/memory.json]
+> ```
+> The original `.json` file is preserved. See also `scripts/verify-migration.ts` to validate the result. The `MEMORY_FILE_PATH` does not need to change.
+
+> [!NOTE]
+> **Automatic v1→v2 migration:** Graph files using the v1 format (64-byte entity records) are automatically migrated to v2 (72-byte records with MERW ψ field) on first open. The old file is preserved as `<name>.graph.v1`.
 
 ## Core Concepts
 
@@ -53,6 +72,27 @@ Example:
 }
 ```
 
+### Ranking
+
+Two ranking systems are maintained and updated after every graph mutation:
+
+- **PageRank (`pagerank`)** — Structural importance via Monte Carlo random walks on graph topology (Avrachenkov et al. Algorithm 4). Each mutation triggers a full sampling pass.
+- **LLM Rank (`llmrank`)** — Walker visit counts that track which nodes the LLM actually opens/searches. Primary sort for `llmrank` is walker visits, with PageRank as tiebreaker.
+
+### Maximum Entropy Random Walk (MERW)
+
+The `random_walk` tool uses MERW rather than a standard uniform random walk. MERW maximizes the global entropy rate by sampling uniformly among all paths in the graph, rather than locally maximizing entropy at each vertex.
+
+Transition probabilities are computed from the dominant eigenvector ψ of the (damped) adjacency matrix:
+
+```
+S_ij = (A_ij / λ) · (ψ_j / ψ_i)
+```
+
+The eigenvector is computed via sparse power iteration with teleportation damping (α=0.85), warm-started from the previously stored ψ values. After a small graph mutation, convergence typically requires only 2–5 iterations rather than a full cold start.
+
+**Practical effect:** Walks gravitate toward structurally rich regions of the graph rather than wandering down linear chains, making serendipitous exploration more productive.
+
 ## API
 
 ### Tools
@@ -110,51 +150,51 @@ Example:
   - Search for nodes using a regex pattern
   - Input: 
     - `query` (string): Regex pattern to search
-    - `sortBy` (string, optional): Sort field ("mtime", "obsMtime", or "name")
-    - `sortDir` (string, optional): Sort direction ("asc" or "desc")
-    - `entityCursor` (number, optional), `relationCursor` (number, optional)
+    - `sortBy` (string, optional): Sort field (`mtime`, `obsMtime`, `name`, `pagerank`, `llmrank`). Default: `llmrank`
+    - `sortDir` (string, optional): Sort direction (`asc` or `desc`)
+    - `direction` (string, optional): Edge direction filter (`forward`, `backward`, `any`). Default: `forward`
+    - `entityCursor`, `relationCursor` (number, optional): Pagination cursors
   - Searches across entity names, types, and observation content
   - Returns matching entities and their relations (paginated)
 
-- **open_nodes_filtered**
-  - Retrieve specific nodes by name with filtered relations
-  - Input: `names` (string[]), `entityCursor` (number, optional), `relationCursor` (number, optional)
-  - Returns:
-    - Requested entities
-    - Only relations where both endpoints are in the requested set
-  - Silently skips non-existent nodes (paginated)
-
 - **open_nodes**
   - Retrieve specific nodes by name
-  - Input: `names` (string[]), `entityCursor` (number, optional), `relationCursor` (number, optional)
-  - Returns:
-    - Requested entities
-    - Relations originating from requested entities
-  - Silently skips non-existent nodes (paginated)
+  - Input:
+    - `names` (string[]): Entity names to retrieve
+    - `direction` (string, optional): Edge direction filter (`forward`, `backward`, `any`). Default: `forward`
+    - `entityCursor`, `relationCursor` (number, optional): Pagination cursors
+  - Returns requested entities and relations originating from them (paginated)
+  - Silently skips non-existent nodes
 
 - **get_neighbors**
   - Get names of neighboring entities connected to a specific entity within a given depth
   - Input: 
     - `entityName` (string): The entity to find neighbors for
     - `depth` (number, default: 1): Maximum traversal depth
-    - `sortBy` (string, optional): Sort field ("mtime", "obsMtime", or "name")
-    - `sortDir` (string, optional): Sort direction ("asc" or "desc")
+    - `sortBy` (string, optional): Sort field (`mtime`, `obsMtime`, `name`, `pagerank`, `llmrank`). Default: `llmrank`
+    - `sortDir` (string, optional): Sort direction (`asc` or `desc`)
+    - `direction` (string, optional): Edge direction filter (`forward`, `backward`, `any`). Default: `forward`
     - `cursor` (number, optional): Pagination cursor
   - Returns neighbor names with timestamps (paginated)
   - Use `open_nodes` to get full entity data for neighbors
 
 - **find_path**
   - Find a path between two entities in the knowledge graph
-  - Input: `fromEntity` (string), `toEntity` (string), `maxDepth` (number, default: 5), `cursor` (number, optional)
+  - Input:
+    - `fromEntity` (string): Starting entity
+    - `toEntity` (string): Target entity
+    - `maxDepth` (number, default: 5): Maximum search depth
+    - `direction` (string, optional): Edge direction filter (`forward`, `backward`, `any`). Default: `forward`
+    - `cursor` (number, optional): Pagination cursor
   - Returns path between entities if one exists (paginated)
 
 - **get_entities_by_type**
   - Get all entities of a specific type
   - Input: 
     - `entityType` (string): Type to filter by
-    - `sortBy` (string, optional): Sort field ("mtime", "obsMtime", or "name")
-    - `sortDir` (string, optional): Sort direction ("asc" or "desc")
-    - `cursor` (number, optional)
+    - `sortBy` (string, optional): Sort field (`mtime`, `obsMtime`, `name`, `pagerank`, `llmrank`). Default: `llmrank`
+    - `sortDir` (string, optional): Sort direction (`asc` or `desc`)
+    - `cursor` (number, optional): Pagination cursor
   - Returns all entities matching the specified type (paginated)
 
 - **get_entity_types**
@@ -176,9 +216,9 @@ Example:
   - Get entities that have no relations (orphaned entities)
   - Input: 
     - `strict` (boolean, default: false): If true, returns entities not connected to 'Self' entity
-    - `sortBy` (string, optional): Sort field ("mtime", "obsMtime", or "name")
-    - `sortDir` (string, optional): Sort direction ("asc" or "desc")
-    - `cursor` (number, optional)
+    - `sortBy` (string, optional): Sort field (`mtime`, `obsMtime`, `name`, `pagerank`, `llmrank`). Default: `llmrank`
+    - `sortDir` (string, optional): Sort direction (`asc` or `desc`)
+    - `cursor` (number, optional): Pagination cursor
   - Returns entities with no connections (paginated)
 
 - **validate_graph**
@@ -195,13 +235,15 @@ Example:
   - Useful for interpreting `mtime`/`obsMtime` values from entities
 
 - **random_walk**
-  - Perform a random walk from a starting entity, following random relations
+  - Perform a MERW-weighted random walk from a starting entity
   - Input:
     - `start` (string): Name of the entity to start the walk from
     - `depth` (number, default: 3): Number of hops to take
     - `seed` (string, optional): Seed for reproducible walks
+    - `direction` (string, optional): Edge direction filter (`forward`, `backward`, `any`). Default: `forward`
+  - Neighbors are selected proportional to their MERW eigenvector component ψ
+  - Falls back to uniform sampling if ψ has not been computed
   - Returns the terminal entity name and the path taken
-  - Useful for serendipitous exploration of the knowledge graph
 
 - **sequentialthinking**
   - Record a thought in the knowledge graph
@@ -209,6 +251,15 @@ Example:
   - Creates a Thought entity and links it to the previous thought if provided
   - Returns the new thought's context ID for chaining
 
+- **kb_load**
+  - Load a plaintext document into the knowledge graph
+  - Input:
+    - `filePath` (string): Absolute path to a plaintext file (`.txt`, `.md`, `.tex`, source code, etc.)
+    - `title` (string, optional): Document title. Defaults to filename without extension
+    - `topK` (number, optional): Number of top TextRank sentences to highlight in the index. Default: 15
+  - Creates a doubly-linked chain of TextChunk entities, a Document entity, and a DocumentIndex with TextRank-selected entry points
+  - For PDFs, convert to text first (e.g., `pdftotext`)
+
 # Usage with Claude Desktop
 
 ### Setup

package/dist/scripts/delete-document.js

@@ -0,0 +1,91 @@
+#!/usr/bin/env node
+/**
+ * delete-document.ts — Remove a kb_load-style document and its TextChunk chain
+ * from the binary knowledge graph.
+ *
+ * Usage:
+ *   MEMORY_FILE_PATH=~/.local/share/memory/vscode.json npx tsx scripts/delete-document.ts <document-entity-name> [--live]
+ *
+ * Without --live, runs in dry-run mode: walks the chain, counts chunks, prints
+ * what would be deleted, but does not mutate anything.
+ *
+ * With --live, actually deletes the document entity, the index entity (if any),
+ * and every TextChunk in the chain.
+ */
+import { KnowledgeGraphManager } from '../server.js';
+const DOC_NAME = process.argv[2];
+const LIVE = process.argv.includes('--live');
+const BATCH_SIZE = 200;
+if (!DOC_NAME) {
+    console.error('Usage: npx tsx scripts/delete-document.ts <document-entity-name> [--live]');
+    process.exit(1);
+}
+const memoryFilePath = process.env.MEMORY_FILE_PATH ?? `${process.env.HOME}/.local/share/memory/vscode.json`;
+console.log(`Opening graph at: ${memoryFilePath}`);
+console.log(`Mode: ${LIVE ? '🔴 LIVE — will delete' : '🟢 DRY RUN — read only'}`);
+console.log();
+const mgr = new KnowledgeGraphManager(memoryFilePath);
+// ── Step 1: Open the document node, find starts_with target ──────────
+const docGraph = await mgr.openNodes([DOC_NAME], 'forward');
+const docEntity = docGraph.entities.find(e => e.name === DOC_NAME);
+if (!docEntity) {
+    console.error(`Entity "${DOC_NAME}" not found.`);
+    process.exit(1);
+}
+console.log(`Found document: "${DOC_NAME}" (type: ${docEntity.entityType})`);
+const startsWithRel = docGraph.relations.find(r => r.relationType === 'starts_with');
+if (!startsWithRel) {
+    console.error(`No "starts_with" relation found on "${DOC_NAME}". Is this a kb_load document?`);
+    process.exit(1);
+}
+const headChunkName = startsWithRel.to;
+console.log(`Head chunk: ${headChunkName}`);
+// ── Step 2: Walk the chain via "follows" relations ───────────────────
+const toDelete = [];
+let currentName = headChunkName;
+let visited = 0;
+while (currentName) {
+    toDelete.push(currentName);
+    visited++;
+    if (visited % 500 === 0) {
+        process.stdout.write(`  … walked ${visited} chunks\r`);
+    }
+    // Find the "follows" relation from this chunk
+    const chunkGraph = await mgr.openNodes([currentName], 'forward');
+    const followsRel = chunkGraph.relations.find(r => r.relationType === 'follows');
+    currentName = followsRel ? followsRel.to : '';
+}
+console.log(`\nChain walk complete: ${toDelete.length} TextChunks found.`);
+// ── Step 3: Check for an index entity ────────────────────────────────
+const indexName = `${DOC_NAME}__index`;
+const indexGraph = await mgr.openNodes([indexName], 'forward');
+const indexEntity = indexGraph.entities.find(e => e.name === indexName);
+const extraDeletes = [DOC_NAME];
+if (indexEntity) {
+    extraDeletes.push(indexName);
+    console.log(`Index entity found: "${indexName}"`);
+}
+else {
+    console.log(`No index entity "${indexName}" found (old-style import).`);
+}
+const totalDeletes = extraDeletes.length + toDelete.length;
+console.log(`\nTotal entities to delete: ${totalDeletes} (${extraDeletes.length} header + ${toDelete.length} chunks)`);
+if (!LIVE) {
+    console.log('\n✅ Dry run complete. Re-run with --live to actually delete.');
+    process.exit(0);
+}
+// ── Step 4: Delete in batches ────────────────────────────────────────
+console.log(`\nDeleting ${totalDeletes} entities in batches of ${BATCH_SIZE}...`);
+// Delete chunks first (the bulk), then the header entities
+let deleted = 0;
+for (let i = 0; i < toDelete.length; i += BATCH_SIZE) {
+    const batch = toDelete.slice(i, i + BATCH_SIZE);
+    await mgr.deleteEntities(batch);
+    deleted += batch.length;
+    process.stdout.write(`  Deleted ${deleted}/${toDelete.length} chunks\r`);
+}
+console.log(`\n  Chunks done.`);
+// Delete document + index
+await mgr.deleteEntities(extraDeletes);
+console.log(`  Deleted document header${indexEntity ? ' + index' : ''}.`);
+console.log(`\n🔴 Done. Removed ${totalDeletes} entities from the graph.`);

package/dist/server.js

@@ -8,6 +8,7 @@ import { fileURLToPath } from 'url';
 import { GraphFile, DIR_FORWARD, DIR_BACKWARD } from './src/graphfile.js';
 import { StringTable } from './src/stringtable.js';
 import { structuralSample } from './src/pagerank.js';
+import { computeMerwPsi } from './src/merw.js';
 import { validateExtension, loadDocument } from './src/kb_load.js';
 // Define memory file path using environment variable with fallback
 const defaultMemoryPath = path.join(path.dirname(fileURLToPath(import.meta.url)), 'memory.json');
@@ -127,54 +128,13 @@ function paginateItems(items, cursor = 0, maxChars = MAX_CHARS) {
     };
 }
 function paginateGraph(graph, entityCursor = 0, relationCursor = 0) {
-    // Build incrementally, measuring actual serialized size
-    const entityCount = graph.entities.length;
-    const relationCount = graph.relations.length;
-    // Start with empty result to measure base overhead
-    const emptyResult = {
-        entities: { items: [], nextCursor: null, totalCount: entityCount },
-        relations: { items: [], nextCursor: null, totalCount: relationCount }
-    };
-    let currentSize = JSON.stringify(emptyResult).length;
-    const resultEntities = [];
-    const resultRelations = [];
-    let entityIdx = entityCursor;
-    let relationIdx = relationCursor;
-    // Add entities until we hit the limit
-    while (entityIdx < graph.entities.length) {
-        const entity = graph.entities[entityIdx];
-        const entityJson = JSON.stringify(entity);
-        const addedChars = entityJson.length + (resultEntities.length > 0 ? 1 : 0);
-        if (currentSize + addedChars > MAX_CHARS) {
-            break;
-        }
-        resultEntities.push(entity);
-        currentSize += addedChars;
-        entityIdx++;
-    }
-    // Add relations with remaining space
-    while (relationIdx < graph.relations.length) {
-        const relation = graph.relations[relationIdx];
-        const relationJson = JSON.stringify(relation);
-        const addedChars = relationJson.length + (resultRelations.length > 0 ? 1 : 0);
-        if (currentSize + addedChars > MAX_CHARS) {
-            break;
-        }
-        resultRelations.push(relation);
-        currentSize += addedChars;
-        relationIdx++;
-    }
+    // Entities and relations have independent cursors, so paginate them
+    // independently — each gets the full budget.  The caller already has
+    // previously-returned pages and only needs the next page of whichever
+    // section it is advancing.
     return {
-        entities: {
-            items: resultEntities,
-            nextCursor: entityIdx < graph.entities.length ? entityIdx : null,
-            totalCount: entityCount
-        },
-        relations: {
-            items: resultRelations,
-            nextCursor: relationIdx < graph.relations.length ? relationIdx : null,
-            totalCount: relationCount
-        }
+        entities: paginateItems(graph.entities, entityCursor),
+        relations: paginateItems(graph.relations, relationCursor),
     };
 }
 // The KnowledgeGraphManager class contains all operations to interact with the knowledge graph
@@ -195,9 +155,10 @@ export class KnowledgeGraphManager {
         this.gf = new GraphFile(graphPath, this.st);
         this.nameIndex = new Map();
         this.rebuildNameIndex();
-        // Run initial structural sampling if graph is non-empty
+        // Run initial structural sampling and MERW if graph is non-empty
         if (this.nameIndex.size > 0) {
             structuralSample(this.gf, 1, 0.85);
+            computeMerwPsi(this.gf);
             this.gf.sync();
         }
     }
@@ -297,11 +258,12 @@ export class KnowledgeGraphManager {
             }
         });
     }
-    /** Re-run structural sampling (call after graph mutations) */
+    /** Re-run structural sampling and MERW eigenvector computation (call after graph mutations) */
     resample() {
         this.withWriteLock(() => {
             if (this.nameIndex.size > 0) {
                 structuralSample(this.gf, 1, 0.85);
+                computeMerwPsi(this.gf);
             }
         });
     }
@@ -827,7 +789,7 @@ export class KnowledgeGraphManager {
                 if (!offset)
                     break;
                 const edges = this.gf.getEdges(offset);
-                const validNeighbors = new Set();
+                const candidates = [];
                 for (const edge of edges) {
                     if (direction === 'forward' && edge.direction !== DIR_FORWARD)
                         continue;
@@ -835,14 +797,46 @@ export class KnowledgeGraphManager {
                         continue;
                     const targetRec = this.gf.readEntity(edge.targetOffset);
                     const neighborName = this.st.get(BigInt(targetRec.nameId));
-                    if (neighborName !== current)
-                        validNeighbors.add(neighborName);
+                    if (neighborName !== current && this.nameIndex.has(neighborName)) {
+                        candidates.push({ name: neighborName, psi: targetRec.psi });
+                    }
+                }
+                // Deduplicate: keep max psi per name (multiple edge types to same target)
+                const byName = new Map();
+                for (const c of candidates) {
+                    const existing = byName.get(c.name);
+                    if (existing === undefined || c.psi > existing) {
+                        byName.set(c.name, c.psi);
+                    }
                 }
-                const neighborArr = Array.from(validNeighbors).filter(n => this.nameIndex.has(n));
-                if (neighborArr.length === 0)
+                if (byName.size === 0)
                     break;
-                const idx = Math.floor(random() * neighborArr.length);
-                current = neighborArr[idx];
+                const neighborArr = Array.from(byName.entries());
+                // MERW-weighted sampling: probability proportional to ψ_j
+                // (The ψ_i denominator is constant for all neighbors and cancels in normalization)
+                let totalPsi = 0;
+                for (const [, psi] of neighborArr)
+                    totalPsi += psi;
+                let chosen;
+                if (totalPsi > 0) {
+                    // Weighted sampling by psi
+                    const r = random() * totalPsi;
+                    let cumulative = 0;
+                    chosen = neighborArr[neighborArr.length - 1][0]; // fallback
+                    for (const [name, psi] of neighborArr) {
+                        cumulative += psi;
+                        if (r <= cumulative) {
+                            chosen = name;
+                            break;
+                        }
+                    }
+                }
+                else {
+                    // psi not yet computed (all zero) — fall back to uniform
+                    const idx = Math.floor(random() * neighborArr.length);
+                    chosen = neighborArr[idx][0];
+                }
+                current = chosen;
                 pathNames.push(current);
             }
             return { entity: current, path: pathNames };
@@ -959,7 +953,7 @@ export function createServer(memoryFilePath) {
                 sizes: ["any"]
             }
         ],
-        version: "0.0.14",
+        version: "0.0.15",
     }, {
         capabilities: {
             tools: {},

package/dist/src/graphfile.js

@@ -4,6 +4,11 @@
  * All records live in a MemoryFile (graph.mem). Variable-length strings are
  * stored in a separate StringTable (strings.mem) and referenced by u32 ID.
  *
+ * Versioning:
+ *   MEMFILE_VERSION 1  = original 64-byte entity records (no psi field)
+ *   MEMFILE_VERSION 2  = 72-byte entity records with f64 psi for MERW
+ *   On open, version 1 files are migrated to version 2 automatically.
+ *
  * Graph file layout:
  *   [memfile header: 32 bytes]
  *   [graph header block: first allocation]
@@ -12,7 +17,7 @@
  *     u64 walker_total         total walker visits (global counter)
  *   [entity records, adj blocks, node log ...]
  *
- * EntityRecord: 64 bytes fixed
+ * EntityRecord: 72 bytes fixed (v2)
  *   u32  name_id         string table ID
  *   u32  type_id         string table ID
  *   u64  adj_offset      offset to AdjBlock (0 = no edges)
@@ -24,6 +29,7 @@
  *   u32  obs1_id         string table ID (0 = empty)
  *   u64  structural_visits  structural PageRank visit count
  *   u64  walker_visits      walker PageRank visit count
+ *   f64  psi             MERW dominant eigenvector component
  *
  * AdjBlock:
  *   u32  count
@@ -44,15 +50,21 @@
  *   u32  capacity
  *   u64  offsets[capacity]
  */
+import * as fs from 'fs';
 import { MemoryFile } from './memoryfile.js';
 // --- Constants ---
-export const ENTITY_RECORD_SIZE = 64;
+export const ENTITY_RECORD_SIZE = 72;
+const OLD_ENTITY_RECORD_SIZE = 64; // v1 layout without psi
 export const ADJ_ENTRY_SIZE = 24; // 8 + 4 + 4 + 8, naturally aligned
 const ADJ_HEADER_SIZE = 8; // count:u32 + capacity:u32
 const NODE_LOG_HEADER_SIZE = 8; // count:u32 + capacity:u32
 const GRAPH_HEADER_SIZE = 24; // node_log_offset:u64 + structural_total:u64 + walker_total:u64
 const INITIAL_ADJ_CAPACITY = 4;
 const INITIAL_LOG_CAPACITY = 256;
+// Graph-layer version stored in the memfile header version field
+const GRAPH_VERSION_V1 = 1; // original 64-byte entity records
+const GRAPH_VERSION_V2 = 2; // 72-byte entity records with f64 psi
+export const CURRENT_GRAPH_VERSION = GRAPH_VERSION_V2;
 // Direction flags
 export const DIR_FORWARD = 0n;
 export const DIR_BACKWARD = 1n;
@@ -72,7 +84,8 @@ const E_OBS1_ID = 40;
 // 4 bytes pad at 44
 const E_STRUCTURAL_VISITS = 48; // u64: 48..55, 8-aligned
 const E_WALKER_VISITS = 56; // u64: 56..63, 8-aligned
-// total = 64
+const E_PSI = 64; // f64: 64..71, 8-aligned (MERW eigenvector component)
+// total = 72
 // AdjEntry field offsets (within each entry)
 const AE_TARGET_DIR = 0;
 const AE_RELTYPE_ID = 8;
@@ -107,6 +120,7 @@ export function readEntityRecord(mf, offset) {
         obs1Id: buf.readUInt32LE(E_OBS1_ID),
         structuralVisits: buf.readBigUInt64LE(E_STRUCTURAL_VISITS),
         walkerVisits: buf.readBigUInt64LE(E_WALKER_VISITS),
+        psi: buf.readDoubleLE(E_PSI),
     };
 }
 export function writeEntityRecord(mf, rec) {
@@ -121,6 +135,7 @@ export function writeEntityRecord(mf, rec) {
     buf.writeUInt32LE(rec.obs1Id, E_OBS1_ID);
     buf.writeBigUInt64LE(rec.structuralVisits, E_STRUCTURAL_VISITS);
     buf.writeBigUInt64LE(rec.walkerVisits, E_WALKER_VISITS);
+    buf.writeDoubleLE(rec.psi, E_PSI);
     mf.write(rec.offset, buf);
 }
 export function readAdjBlock(mf, adjOffset) {
@@ -174,12 +189,98 @@ export class GraphFile {
         this.st = stringTable;
         const stats = this.mf.stats();
         if (stats.allocated <= 32n) {
+            // Fresh DB — initialize with current version
             this.graphHeaderOffset = this.initGraphHeader();
+            this.mf.setVersion(CURRENT_GRAPH_VERSION);
         }
         else {
-            // First allocation is graph header, at offset 40 (32 memfile header + 8 alloc_t header)
+            // Existing DB — check version and migrate if needed
             this.graphHeaderOffset = 40n;
+            const version = this.mf.getVersion();
+            if (version === GRAPH_VERSION_V1) {
+                this.migrateV1toV2(graphPath);
+            }
+            else if (version !== CURRENT_GRAPH_VERSION) {
+                throw new Error(`GraphFile: unknown version ${version} (expected ${GRAPH_VERSION_V1} or ${CURRENT_GRAPH_VERSION})`);
+            }
+        }
+    }
+    /**
+     * Migrate a v1 graph file (64-byte entity records) to v2 (72-byte with psi).
+     *
+     * Strategy: read all entities and edges from the current (v1) file into memory,
+     * close it, rename it to .v1, create a fresh file, write everything back with
+     * the new layout.
+     */
+    migrateV1toV2(graphPath) {
+        // 1. Read all entities and adjacency data from v1 layout
+        const offsets = this.getAllEntityOffsets();
+        const oldEntities = [];
+        for (const offset of offsets) {
+            // Read v1 entity (64 bytes) — the first 64 bytes match our struct minus psi
+            const buf = this.mf.read(offset, BigInt(OLD_ENTITY_RECORD_SIZE));
+            const rec = {
+                offset,
+                nameId: buf.readUInt32LE(E_NAME_ID),
+                typeId: buf.readUInt32LE(E_TYPE_ID),
+                adjOffset: buf.readBigUInt64LE(E_ADJ_OFFSET),
+                mtime: buf.readBigUInt64LE(E_MTIME),
+                obsMtime: buf.readBigUInt64LE(E_OBS_MTIME),
+                obsCount: buf.readUInt8(E_OBS_COUNT),
+                obs0Id: buf.readUInt32LE(E_OBS0_ID),
+                obs1Id: buf.readUInt32LE(E_OBS1_ID),
+                structuralVisits: buf.readBigUInt64LE(E_STRUCTURAL_VISITS),
+                walkerVisits: buf.readBigUInt64LE(E_WALKER_VISITS),
+                psi: 0,
+            };
+            const edges = rec.adjOffset !== 0n ? readAdjBlock(this.mf, rec.adjOffset).entries : [];
+            oldEntities.push({ rec, edges });
+        }
+        // 2. Read global counters
+        const structuralTotal = this.getStructuralTotal();
+        const walkerTotal = this.getWalkerTotal();
+        // 3. Close old file, rename to .v1 backup
+        this.mf.sync();
+        this.mf.close();
+        const backupPath = graphPath + '.v1';
+        fs.renameSync(graphPath, backupPath);
+        // 4. Create fresh v2 file
+        this.mf = new MemoryFile(graphPath, 65536);
+        this.mf.setVersion(CURRENT_GRAPH_VERSION);
+        this.graphHeaderOffset = this.initGraphHeader();
+        // 5. Write global counters
+        const ghBuf = Buffer.alloc(8);
+        ghBuf.writeBigUInt64LE(structuralTotal, 0);
+        this.mf.write(this.graphHeaderOffset + BigInt(GH_STRUCTURAL_TOTAL), ghBuf);
+        ghBuf.writeBigUInt64LE(walkerTotal, 0);
+        this.mf.write(this.graphHeaderOffset + BigInt(GH_WALKER_TOTAL), ghBuf);
+        // 6. Write all entities with new 72-byte layout, building offset remap
+        const offsetMap = new Map(); // old offset → new offset
+        for (const { rec } of oldEntities) {
+            const newOffset = this.mf.alloc(BigInt(ENTITY_RECORD_SIZE));
+            if (newOffset === 0n)
+                throw new Error('GraphFile migration: entity alloc failed');
+            offsetMap.set(rec.offset, newOffset);
+            const newRec = { ...rec, offset: newOffset, adjOffset: 0n, psi: 0 };
+            writeEntityRecord(this.mf, newRec);
+            this.nodeLogAppend(newOffset);
+        }
+        // 7. Rebuild adjacency blocks with remapped target offsets
+        for (const { rec, edges } of oldEntities) {
+            const newOffset = offsetMap.get(rec.offset);
+            for (const edge of edges) {
+                const newTarget = offsetMap.get(edge.targetOffset);
+                if (newTarget === undefined)
+                    continue; // skip dangling refs
+                this.addEdge(newOffset, {
+                    targetOffset: newTarget,
+                    direction: edge.direction,
+                    relTypeId: edge.relTypeId,
+                    mtime: edge.mtime,
+                });
+            }
         }
+        this.mf.sync();
     }
     initGraphHeader() {
         // Allocate graph header block
@@ -233,6 +334,7 @@ export class GraphFile {
             obs1Id: 0,
             structuralVisits: 0n,
             walkerVisits: 0n,
+            psi: 0,
         };
         writeEntityRecord(this.mf, rec);
         this.nodeLogAppend(offset);
@@ -534,6 +636,18 @@ export class GraphFile {
         const rec = this.readEntity(entityOffset);
         return Number(rec.walkerVisits) / Number(total);
     }
+    // --- MERW eigenvector ---
+    /** Write the psi (MERW eigenvector component) for an entity. */
+    setPsi(entityOffset, psi) {
+        const buf = Buffer.alloc(8);
+        buf.writeDoubleLE(psi, 0);
+        this.mf.write(entityOffset + BigInt(E_PSI), buf);
+    }
+    /** Read the psi (MERW eigenvector component) for an entity. */
+    getPsi(entityOffset) {
+        const buf = this.mf.read(entityOffset + BigInt(E_PSI), 8n);
+        return buf.readDoubleLE(0);
+    }
     // --- Lifecycle & Concurrency ---
     /** Acquire a shared (read) lock on the graph file. */
     lockShared() {

package/dist/src/memoryfile.js

@@ -115,6 +115,23 @@ export class MemoryFile {
         this.assertOpen();
         return native.stats(this.handle);
     }
+    /**
+     * Read the memfile version field (u32 at offset 4).
+     */
+    getVersion() {
+        this.assertOpen();
+        const buf = native.read(this.handle, 4n, 4n);
+        return buf.readUInt32LE(0);
+    }
+    /**
+     * Write the memfile version field (u32 at offset 4).
+     */
+    setVersion(version) {
+        this.assertOpen();
+        const buf = Buffer.alloc(4);
+        buf.writeUInt32LE(version, 0);
+        native.write(this.handle, 4n, buf);
+    }
     /**
      * Close the memory file. Syncs and unmaps.
      * The instance is unusable after this.

package/dist/src/merw.js

@@ -0,0 +1,160 @@
+/**
+ * Maximum Entropy Random Walk (MERW) — dominant eigenvector computation
+ * via power iteration on the graph's adjacency matrix.
+ *
+ * MERW transition probabilities:  S_ij = (A_ij / λ) * (ψ_j / ψ_i)
+ * Stationary distribution:        ρ_i  = ψ_i² / ‖ψ‖₂²
+ *
+ * We compute ψ (the dominant right eigenvector of A) using sparse power
+ * iteration directly on the GraphFile adjacency lists. No dense matrix
+ * is ever constructed.
+ *
+ * For directed graphs that may not be strongly connected, we add
+ * teleportation damping (like PageRank): at each step, follow an edge
+ * with probability `alpha`, or jump to a uniform random node with
+ * probability `(1 - alpha)`. This guarantees convergence to a unique
+ * positive eigenvector.
+ */
+import { DIR_FORWARD } from './graphfile.js';
+const DEFAULT_ALPHA = 0.85;
+const DEFAULT_MAX_ITER = 200;
+const DEFAULT_TOL = 1e-8;
+/**
+ * Compute the dominant eigenvector of the (damped) adjacency matrix
+ * via power iteration and write ψ_i into each entity record.
+ *
+ * Warm-starts from the ψ values already stored in the entity records.
+ * New nodes (psi === 0) are seeded with the mean of existing values.
+ * On a fresh graph (all zeros), falls back to uniform initialization.
+ *
+ * @param gf      GraphFile to operate on
+ * @param alpha   Damping factor (probability of following an edge). Default 0.85.
+ * @param maxIter Maximum iterations. Default 200.
+ * @param tol     Convergence tolerance (L2 norm of change). Default 1e-8.
+ * @returns       Number of iterations performed.
+ */
+export function computeMerwPsi(gf, alpha = DEFAULT_ALPHA, maxIter = DEFAULT_MAX_ITER, tol = DEFAULT_TOL) {
+    const offsets = gf.getAllEntityOffsets();
+    const n = offsets.length;
+    if (n === 0)
+        return 0;
+    // Build offset → index map for O(1) lookup
+    const indexMap = new Map();
+    for (let i = 0; i < n; i++) {
+        indexMap.set(offsets[i], i);
+    }
+    // Build sparse adjacency: for each node, list of forward neighbor indices
+    const adj = new Array(n);
+    for (let i = 0; i < n; i++) {
+        const edges = gf.getEdges(offsets[i]);
+        const neighbors = [];
+        for (const e of edges) {
+            if (e.direction !== DIR_FORWARD)
+                continue;
+            const j = indexMap.get(e.targetOffset);
+            if (j !== undefined)
+                neighbors.push(j);
+        }
+        adj[i] = neighbors;
+    }
+    // Warm-start: read existing ψ from entity records
+    let psi = new Float64Array(n);
+    let hasWarm = false;
+    let warmSum = 0;
+    let warmCount = 0;
+    for (let i = 0; i < n; i++) {
+        const val = gf.getPsi(offsets[i]);
+        psi[i] = val;
+        if (val > 0) {
+            hasWarm = true;
+            warmSum += val;
+            warmCount++;
+        }
+    }
+    if (hasWarm) {
+        // Seed new/zero nodes with the mean of existing nonzero values
+        const mean = warmSum / warmCount;
+        for (let i = 0; i < n; i++) {
+            if (psi[i] <= 0)
+                psi[i] = mean;
+        }
+    }
+    else {
+        // Cold start: uniform
+        const uniform = 1.0 / Math.sqrt(n);
+        psi.fill(uniform);
+    }
+    // Normalize initial vector to unit L2
+    let initNorm = 0;
+    for (let i = 0; i < n; i++)
+        initNorm += psi[i] * psi[i];
+    initNorm = Math.sqrt(initNorm);
+    if (initNorm > 0) {
+        for (let i = 0; i < n; i++)
+            psi[i] /= initNorm;
+    }
+    let psiNext = new Float64Array(n);
+    const teleport = (1.0 - alpha) / n;
+    let iter = 0;
+    for (iter = 0; iter < maxIter; iter++) {
+        // Matrix-vector multiply: psiNext = alpha * A * psi + (1-alpha)/n * sum(psi)
+        // Since ψ is normalized, sum(psi) components contribute uniformly.
+        // For the adjacency multiply, A_ij = 1 if edge i→j exists.
+        // Power iteration: psiNext_j = alpha * Σ_{i: i→j} psi_i  +  teleport * Σ_k psi_k
+        //
+        // We iterate over source nodes and scatter to targets.
+        psiNext.fill(0);
+        // Compute sum of psi for teleportation
+        let psiSum = 0;
+        for (let i = 0; i < n; i++)
+            psiSum += psi[i];
+        const teleportContrib = teleport * psiSum;
+        // Sparse multiply: scatter from sources to targets
+        for (let i = 0; i < n; i++) {
+            const neighbors = adj[i];
+            const val = alpha * psi[i];
+            for (const j of neighbors) {
+                psiNext[j] += val;
+            }
+        }
+        // Add teleportation
+        for (let i = 0; i < n; i++) {
+            psiNext[i] += teleportContrib;
+        }
+        // Normalize to unit L2
+        let norm = 0;
+        for (let i = 0; i < n; i++)
+            norm += psiNext[i] * psiNext[i];
+        norm = Math.sqrt(norm);
+        if (norm > 0) {
+            for (let i = 0; i < n; i++)
+                psiNext[i] /= norm;
+        }
+        // Check convergence: L2 norm of difference
+        let diff = 0;
+        for (let i = 0; i < n; i++) {
+            const d = psiNext[i] - psi[i];
+            diff += d * d;
+        }
+        diff = Math.sqrt(diff);
+        // Swap buffers
+        const tmp = psi;
+        psi = psiNext;
+        psiNext = tmp;
+        if (diff < tol) {
+            iter++;
+            break;
+        }
+    }
+    // Ensure all components are positive (Perron-Frobenius: dominant eigenvector is non-negative,
+    // but numerical noise can produce tiny negatives). Clamp to 0.
+    for (let i = 0; i < n; i++) {
+        if (psi[i] < 0)
+            psi[i] = 0;
+    }
+    // Write ψ_i into each entity record
+    for (let i = 0; i < n; i++) {
+        gf.setPsi(offsets[i], psi[i]);
+    }
+    return iter;
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@levalicious/server-memory",
-  "version": "0.0.14",
+  "version": "0.0.15",
   "description": "MCP server for enabling memory for Claude through a knowledge graph",
   "license": "MIT",
   "author": "Levalicious",