rag-memory-epf-mcp 3.3.5 → 3.4.0

package/README.md

@@ -13,7 +13,8 @@ A **project-local RAG memory** MCP server — knowledge graph + multilingual vec
 - **3-signal hybrid search** — vector similarity (bge-m3, 1024-dim) + FTS5 BM25 keyword matching + knowledge graph re-ranking, combined via Reciprocal Rank Fusion
 - **100+ languages** — Korean, Chinese, Japanese, Arabic, and more. Cross-lingual search works out of the box.
 - **Graph-aware scoring** — per-entity geometric decay (0.5^i) with hard cap prevents any single document from dominating results
-- **27 MCP tools** — entity/relation CRUD, document pipeline, multi-hop graph traversal, export/import, temporal queries
+- **30 MCP tools** — knowledge graph CRUD, document pipeline, hybrid search, multi-hop traversal, graph analytics (centrality / community detection / structure), export/import, temporal queries
+- **Codepoint-safe chunking** — chunk offsets are Unicode codepoints, language-neutral across SQL `substr`, Python slicing, and JS `[...str]` iteration. Korean/CJK/emoji documents stay aligned. Verified by a publish-time invariant test.
 - **SQLite optimized** — WAL mode, 32MB cache, 256MB mmap, FTS5 triggers, 7 indexes
 - **MCP SDK 1.27.1** — Tool Annotations (readOnly/destructive/idempotent), latest protocol 2025-11-25
 
@@ -35,7 +36,7 @@ A **project-local RAG memory** MCP server — knowledge graph + multilingual vec
 
 Place this `.mcp.json` in each project folder with its own `DB_FILE_PATH`. Each project maintains completely isolated memory.
 
-## Tools (27)
+## Tools (30)
 
 ### Knowledge Graph (7)
 | Tool | Description | Annotation |
@@ -80,6 +81,13 @@ Place this `.mcp.json` in each project folder with its own `DB_FILE_PATH`. Each
 | `runMigrations` | Apply pending migrations | idempotent |
 | `rollbackMigration` | Revert to a previous schema version | destructive |
 
+### Graph Analytics (3)
+| Tool | Description | Annotation |
+|------|------------|------------|
+| `getGraphMetrics` | Per-entity centrality (degree, betweenness, closeness, pagerank) | readOnly |
+| `detectCommunities` | Louvain community detection + modularity score | readOnly |
+| `analyzeGraphStructure` | Density, connected components, clustering coefficient | readOnly |
+
 ## Document Processing Pipeline
 
 ```
@@ -112,7 +120,7 @@ storeDocument(id, content, metadata)
 │  │  ├── chunks (sqlite-vec, 1024-dim)     │  │
 │  │  ├── entity_embeddings (sqlite-vec)    │  │
 │  │  ├── entities_fts + chunks_fts (FTS5)  │  │
-│  │  └── 7 migrations (auto-applied)       │  │
+│  │  └── 11 migrations (auto-applied)      │  │
 │  └────────────────────────────────────────┘  │
 │                                              │
 │  bge-m3 (ONNX, 100+ langs)                   │
@@ -128,6 +136,24 @@ storeDocument(id, content, metadata)
 
 ## Changelog
 
+### v3.3.6
+- **Publish-time invariant test** — `npm run verify:invariants` (wired as `prepublishOnly`) catches `chunkText` offset regressions before they ship. Tests ASCII / Korean / emoji-heavy / mixed CJK + supplementary plane / pure supplementary inputs against the codepoint-slice contract.
+- **`chunkText` extracted to `src/chunkText.ts`** — algorithm now testable in isolation. The class method is a thin wrapper. No user-facing API change.
+- **README accuracy** — tool count corrected to 30, migration count to 11, Graph Analytics tools surfaced.
+
+### v3.3.5
+- **Fix: chunk offsets stored as JS UTF-16 units instead of Unicode codepoints** — Korean/CJK/emoji documents had `start_pos`/`end_pos` that disagreed with SQL `substr` and Python slicing for any chunk crossing a supplementary character. `chunkText` now maintains parallel UTF-16 + codepoint cursors and reports codepoint offsets.
+- **Migration v11** — converts existing `chunk_metadata.start_pos`/`end_pos` from UTF-16 units to codepoints by re-locating each chunk via `indexOf` and counting codepoints.
+
+### v3.3.4
+- **Migration version 9 → 10 jump + `ALTER TABLE` idempotency guards** — some databases from early v3.x experiments (Ollama dimension swap) had recorded a migration at version 9, causing the new v9 migration to silently no-op. Bumped to version 10 and added `PRAGMA table_info` guards so the column-add is safe to re-run.
+
+### v3.3.3
+- **Separate token-space and char-space offsets in `chunk_metadata`** — added `start_token`/`end_token` columns. Existing `start_pos`/`end_pos` are reinterpreted as character offsets into `documents.content`. Backfill migration recomputes char offsets via `indexOf` with a per-document cursor; misses leave NULL so callers can re-chunk to repair.
+
+### v3.3.0
+- **Graph analytics (graphology)** — three new MCP tools: `getGraphMetrics` (degree / betweenness / closeness / pagerank), `detectCommunities` (Louvain + modularity), `analyzeGraphStructure` (density / components / clustering). 27 → 30 tools.
+
 ### v3.2.1
 - **Fix: `autoLinkEntities` silent failure** — was JOINing a non-existent `observations` table (observations are stored as JSON array column in `entities`). Changed to direct column select + `JSON.parse()`.
 
@@ -148,11 +174,11 @@ storeDocument(id, content, metadata)
 - **Multi-hop graph traversal** — `getNeighbors` tool with `WITH RECURSIVE` CTE, depth 1-5, cycle detection, bidirectional
 - **Embedding LRU cache** — 500-entry in-memory cache, skips redundant re-computation
 - **Configurable model** — `EMBEDDING_MODEL` env var to use alternative embedding models
-- 27 tools total
+- 27 tools total at this version (30 as of v3.3.0+)
 
 ### v1.8.0
 - **MCP SDK 1.27.1** — protocol 2025-11-25, security fix GHSA-345p-7cg4-v4c7 (CVSS 7.1)
-- **Tool Annotations** — all 27 tools annotated (readOnlyHint, destructiveHint, idempotentHint)
+- **Tool Annotations** — all tools annotated (readOnlyHint, destructiveHint, idempotentHint)
 - **SIGTERM graceful shutdown** — clean exit without ONNX mutex crash
 
 ### v1.7.0
@@ -188,8 +214,12 @@ git clone https://github.com/bripin123/rag-memory-epf-mcp.git
 cd rag-memory-epf-mcp
 npm install
 npm run build
+npm test                  # build + invariant verification
+npm run verify:invariants # standalone invariant test (assumes dist/ built)
 ```
 
+`npm publish` automatically runs `prepublishOnly` (`build` + `verify:invariants`); a chunk-offset regression blocks the publish at the source.
+
 ## License
 
 MIT License. See [LICENSE](LICENSE).

package/dist/index.js

@@ -21,6 +21,8 @@ import modularity from 'graphology-metrics/graph/modularity.js';
 import { getAllMCPTools, validateToolArgs, getSystemInfo } from './src/tools/tool-registry.js';
 // Import migration system
 import { MigrationManager } from './src/migrations/migration-manager.js';
+// Import chunk text algorithm (extracted for publish-time invariant testing)
+import { chunkText as splitTextIntoChunks } from './src/chunkText.js';
 import { migrations } from './src/migrations/migrations.js';
 import { createHash } from 'crypto';
 import { createRequire } from 'module';
@@ -45,34 +47,7 @@ const EMBEDDING_MODEL = process.env.EMBEDDING_MODEL || 'Xenova/bge-m3';
 // When a chunk is not at the document head/tail, any partial sequence at that
 // edge belongs to an adjacent chunk and must be removed so TextDecoder does
 // not emit U+FFFD. Pass trimHead/trimTail=false to preserve head/tail bytes.
-function trimIncompleteUtf8(bytes, trimHead, trimTail) {
-    let start = 0;
-    let end = bytes.length;
-    if (trimHead) {
-        while (start < end && (bytes[start] & 0xC0) === 0x80)
-            start++;
-    }
-    if (trimTail) {
-        let i = end - 1;
-        while (i >= start && (bytes[i] & 0xC0) === 0x80)
-            i--;
-        if (i >= start) {
-            const lead = bytes[i];
-            let needed = 1;
-            if ((lead & 0x80) === 0)
-                needed = 1;
-            else if ((lead & 0xE0) === 0xC0)
-                needed = 2;
-            else if ((lead & 0xF0) === 0xE0)
-                needed = 3;
-            else if ((lead & 0xF8) === 0xF0)
-                needed = 4;
-            if (end - i < needed)
-                end = i;
-        }
-    }
-    return bytes.subarray(start, end);
-}
+// (Implementation moved to src/chunkText.ts for testability.)
 function safeRowid(value) {
     const n = Number(value);
     if (!Number.isInteger(n) || n < 0) {
@@ -1148,60 +1123,17 @@ class RAGKnowledgeGraphManager {
     chunkText(text, maxTokens = 800, overlap = 160) {
         if (!this.encoding)
             throw new Error('Tokenizer not initialized');
-        const tokens = this.encoding.encode(text);
-        const chunks = [];
-        let utf16Cursor = 0;
-        let cpCursor = 0;
-        for (let i = 0; i < tokens.length; i += maxTokens - overlap) {
-            const chunkTokens = tokens.slice(i, i + maxTokens);
-            const decodedBytes = this.encoding.decode(chunkTokens);
-            const isFirst = i === 0;
-            const isLast = i + chunkTokens.length >= tokens.length;
-            const safeBytes = trimIncompleteUtf8(decodedBytes, !isFirst, !isLast);
-            const chunkText = new TextDecoder('utf-8').decode(safeBytes);
-            let startPos;
-            let endPos;
-            if (isFirst) {
-                startPos = 0;
-                endPos = [...chunkText].length;
-                utf16Cursor = 0;
-                cpCursor = 0;
-            }
-            else if (chunkText.length === 0) {
-                startPos = null;
-                endPos = null;
-            }
-            else {
-                const utfIdx = text.indexOf(chunkText, utf16Cursor);
-                if (utfIdx >= 0) {
-                    // Advance cpCursor by codepoints between the previous cursor and the
-                    // new chunk's start (handles overlap by anchoring at the previous
-                    // chunk's start, not its end).
-                    if (utfIdx > utf16Cursor) {
-                        cpCursor += [...text.slice(utf16Cursor, utfIdx)].length;
-                        utf16Cursor = utfIdx;
-                    }
-                    const cpLen = [...chunkText].length;
-                    startPos = cpCursor;
-                    endPos = cpCursor + cpLen;
-                }
-                else {
-                    startPos = null;
-                    endPos = null;
-                }
-            }
-            chunks.push({
-                id: '',
-                document_id: '',
-                chunk_index: chunks.length,
-                text: chunkText,
-                start_pos: startPos,
-                end_pos: endPos,
-                start_token: i,
-                end_token: i + chunkTokens.length
-            });
-        }
-        return chunks;
+        const segments = splitTextIntoChunks(text, this.encoding, maxTokens, overlap);
+        return segments.map((seg, idx) => ({
+            id: '',
+            document_id: '',
+            chunk_index: idx,
+            text: seg.text,
+            start_pos: seg.start_pos,
+            end_pos: seg.end_pos,
+            start_token: seg.start_token,
+            end_token: seg.end_token
+        }));
     }
     // Generate embeddings using sentence transformers
     // isQuery: true for search queries (adds instruction prefix), false for documents/entities
@@ -1238,6 +1170,46 @@ class RAGKnowledgeGraphManager {
         throw new Error('Embedding model not initialized. The server may still be loading the model — retry in a few seconds.');
     }
     // === NEW SEPARATE TOOLS ===
+    async syncDocumentFromFile(filePath, documentId, options = {}) {
+        if (!this.db)
+            throw new Error('Database not initialized');
+        // 1. Resolve content: raw file verbatim (default) or explicit override.
+        //    Content is read on the server and never routed through the model context.
+        const content = options.content !== undefined
+            ? options.content
+            : fsSync.readFileSync(filePath, 'utf-8');
+        const bytes = Buffer.byteLength(content, 'utf-8');
+        // 2. Metadata: default source=path, updated=today (YYYY-MM-DD); caller can override either.
+        const today = new Date().toISOString().slice(0, 10);
+        const metadata = { source: filePath, updated: today, ...(options.metadata || {}) };
+        console.error(`🔄 syncDocumentFromFile: ${documentId} <- ${filePath} (${bytes} bytes)`);
+        // 3. delete -> store -> chunk -> embed, reusing the existing pipeline methods.
+        await this.deleteDocuments(documentId);
+        await this.storeDocument(documentId, content, metadata);
+        const chunkResult = await this.chunkDocument(documentId, options.chunkParams || {});
+        const embedResult = await this.embedChunks(documentId);
+        // 4. Optional explicit entity links (in addition to auto term-matching in embedChunks).
+        let explicitlyLinked;
+        if (options.entityNames && options.entityNames.length > 0) {
+            const linkResult = await this.linkEntitiesToDocument(documentId, options.entityNames);
+            explicitlyLinked = linkResult.linkedEntities;
+        }
+        const linkedEntities = embedResult.linkedEntities ?? 0;
+        // 5. Terse summary only (no chunk text / content echo) to keep caller context flat.
+        const result = {
+            documentId,
+            bytes,
+            chunks: chunkResult.chunks.length,
+            embeddedChunks: embedResult.embeddedChunks,
+            linkedEntities,
+            ...(explicitlyLinked !== undefined ? { explicitlyLinked } : {}),
+        };
+        if (linkedEntities === 0 && explicitlyLinked === undefined) {
+            result.warning = 'linkedEntities=0: ensure the file content contains entity-name literals (e.g. a wiki anchor line "RAG entity: ...") so term-matching can link entities.';
+        }
+        console.error(`✅ syncDocumentFromFile done: ${documentId} (${result.chunks} chunks, ${result.embeddedChunks} embedded, ${linkedEntities} linked)`);
+        return result;
+    }
     async storeDocument(id, content, metadata = {}) {
         if (!this.db)
             throw new Error('Database not initialized');
@@ -2542,6 +2514,13 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
                 return { content: [{ type: "text", text: JSON.stringify(await ragKgManager.deleteDocuments(validatedArgs.documentIds), null, 2) }] };
             case "listDocuments":
                 return { content: [{ type: "text", text: JSON.stringify(await ragKgManager.listDocuments(validatedArgs.includeMetadata !== false), null, 2) }] };
+            case "syncDocumentFromFile":
+                return { content: [{ type: "text", text: JSON.stringify(await ragKgManager.syncDocumentFromFile(validatedArgs.path, validatedArgs.documentId, {
+                                metadata: validatedArgs.metadata,
+                                content: validatedArgs.content,
+                                entityNames: validatedArgs.entityNames,
+                                chunkParams: validatedArgs.chunkParams,
+                            }), null, 2) }] };
             // NEW: Entity embedding tools
             case "embedAllEntities":
                 return { content: [{ type: "text", text: JSON.stringify(await ragKgManager.embedAllEntities(), null, 2) }] };

package/dist/src/chunkText.d.ts

@@ -0,0 +1,10 @@
+import type { Tiktoken } from 'tiktoken';
+export interface ChunkSegment {
+    text: string;
+    start_pos: number | null;
+    end_pos: number | null;
+    start_token: number;
+    end_token: number;
+}
+export declare function trimIncompleteUtf8(bytes: Uint8Array, trimHead: boolean, trimTail: boolean): Uint8Array;
+export declare function chunkText(text: string, encoding: Tiktoken, maxTokens?: number, overlap?: number): ChunkSegment[];

package/dist/src/chunkText.js

@@ -0,0 +1,104 @@
+// Tokenize and chunk text using a BPE encoder while reporting both token-space
+// and char-space (Unicode codepoint) offsets back into the original string.
+//
+// BPE tokenizers (cl100k_base) split multi-byte UTF-8 sequences across tokens.
+// Slicing token arrays at arbitrary boundaries can leave incomplete UTF-8
+// prefix/suffix bytes, which TextDecoder replaces with U+FFFD (�). We trim the
+// incomplete sequences at chunk boundaries; overlap covers the removed bytes.
+//
+// Each chunk records both token-space offsets (start_token/end_token from the
+// BPE encoder loop) and char-space offsets (start_pos/end_pos into the original
+// text). Char offsets are Unicode codepoint counts — language-neutral, so SQL
+// substr, Python str slicing, and JS [...str] iteration all line up. JS's
+// native UTF-16 indexing differs for supplementary characters (emoji, rare CJK),
+// so the function maintains parallel UTF-16 and codepoint cursors and reports
+// codepoint offsets. On a coincidental indexOf miss the char offsets are NULL.
+//
+// Extracted to a standalone module so publish-time invariant tests can exercise
+// the algorithm directly without booting the full RAG-Memory stack.
+// trimIncompleteUtf8: strip incomplete UTF-8 sequences from the head/tail of a
+// byte buffer produced by decoding an arbitrary token slice. A multi-byte
+// codepoint that begins or ends on the cut edge belongs to an adjacent chunk
+// and must be removed so TextDecoder does not emit U+FFFD. Pass
+// trimHead/trimTail=false to preserve head/tail bytes (first/last chunks).
+export function trimIncompleteUtf8(bytes, trimHead, trimTail) {
+    let start = 0;
+    let end = bytes.length;
+    if (trimHead) {
+        while (start < end && (bytes[start] & 0xC0) === 0x80)
+            start++;
+    }
+    if (trimTail) {
+        let i = end - 1;
+        while (i >= start && (bytes[i] & 0xC0) === 0x80)
+            i--;
+        if (i >= start) {
+            const lead = bytes[i];
+            let needed = 1;
+            if ((lead & 0x80) === 0)
+                needed = 1;
+            else if ((lead & 0xE0) === 0xC0)
+                needed = 2;
+            else if ((lead & 0xF0) === 0xE0)
+                needed = 3;
+            else if ((lead & 0xF8) === 0xF0)
+                needed = 4;
+            if (end - i < needed)
+                end = i;
+        }
+    }
+    return bytes.subarray(start, end);
+}
+export function chunkText(text, encoding, maxTokens = 800, overlap = 160) {
+    const tokens = encoding.encode(text);
+    const segments = [];
+    let utf16Cursor = 0;
+    let cpCursor = 0;
+    for (let i = 0; i < tokens.length; i += maxTokens - overlap) {
+        const chunkTokens = tokens.slice(i, i + maxTokens);
+        const decodedBytes = encoding.decode(chunkTokens);
+        const isFirst = i === 0;
+        const isLast = i + chunkTokens.length >= tokens.length;
+        const safeBytes = trimIncompleteUtf8(decodedBytes, !isFirst, !isLast);
+        const chunkTextStr = new TextDecoder('utf-8').decode(safeBytes);
+        let startPos;
+        let endPos;
+        if (isFirst) {
+            startPos = 0;
+            endPos = [...chunkTextStr].length;
+            utf16Cursor = 0;
+            cpCursor = 0;
+        }
+        else if (chunkTextStr.length === 0) {
+            startPos = null;
+            endPos = null;
+        }
+        else {
+            const utfIdx = text.indexOf(chunkTextStr, utf16Cursor);
+            if (utfIdx >= 0) {
+                // Advance cpCursor by codepoints between the previous cursor and the
+                // new chunk's start (handles overlap by anchoring at the previous
+                // chunk's start, not its end).
+                if (utfIdx > utf16Cursor) {
+                    cpCursor += [...text.slice(utf16Cursor, utfIdx)].length;
+                    utf16Cursor = utfIdx;
+                }
+                const cpLen = [...chunkTextStr].length;
+                startPos = cpCursor;
+                endPos = cpCursor + cpLen;
+            }
+            else {
+                startPos = null;
+                endPos = null;
+            }
+        }
+        segments.push({
+            text: chunkTextStr,
+            start_pos: startPos,
+            end_pos: endPos,
+            start_token: i,
+            end_token: i + chunkTokens.length
+        });
+    }
+    return segments;
+}

package/dist/src/tools/rag-tools.d.ts

@@ -7,6 +7,7 @@ export declare const linkEntitiesToDocumentTool: ToolDefinition;
 export declare const getKnowledgeGraphStatsTool: ToolDefinition;
 export declare const deleteDocumentsTool: ToolDefinition;
 export declare const listDocumentsTool: ToolDefinition;
+export declare const syncDocumentFromFileTool: ToolDefinition;
 export declare const ragTools: {
     storeDocument: ToolDefinition;
     chunkDocument: ToolDefinition;
@@ -16,4 +17,5 @@ export declare const ragTools: {
     getKnowledgeGraphStats: ToolDefinition;
     deleteDocuments: ToolDefinition;
     listDocuments: ToolDefinition;
+    syncDocumentFromFile: ToolDefinition;
 };

package/dist/src/tools/rag-tools.js

@@ -516,6 +516,93 @@ export const listDocumentsTool = {
     schema: listDocumentsSchema,
     annotations: { readOnlyHint: true },
 };
+// === SYNC DOCUMENT FROM FILE TOOL ===
+const syncDocumentFromFileCapability = {
+    description: 'Read a file server-side and run the full document sync pipeline (delete, store, chunk, embed, link) in one call, returning only a terse summary',
+    parameters: {
+        type: 'object',
+        properties: {
+            path: {
+                type: 'string',
+                description: 'Absolute path of the file to sync. Content is read server-side and never routed through the model context.'
+            },
+            documentId: {
+                type: 'string',
+                description: 'RAG document ID to (re)create from the file'
+            },
+            metadata: {
+                type: 'object',
+                description: 'Metadata merged into the stored document (source defaults to path, updated defaults to today)',
+                additionalProperties: true,
+                optional: true
+            },
+            content: {
+                type: 'string',
+                description: 'Optional content override; if provided, this is stored instead of reading the file',
+                optional: true
+            },
+            entityNames: {
+                type: 'array',
+                description: 'Optional entities to explicitly link in addition to automatic term-matching',
+                items: { type: 'string' },
+                optional: true
+            },
+            chunkParams: {
+                type: 'object',
+                description: 'Optional chunking parameters { maxTokens, overlap }',
+                additionalProperties: true,
+                optional: true
+            }
+        },
+        required: ['path', 'documentId'],
+    },
+};
+const syncDocumentFromFileDescription = () => `<description>
+Read a file on the server and perform the entire document sync pipeline in a single call:
+deleteDocuments -> storeDocument -> chunkDocument -> embedChunks -> (optional) linkEntitiesToDocument.
+**The file content is read server-side and is NOT routed through the model context**, and only a
+terse summary is returned (no chunk text). This collapses the usual 5 tool calls per document into
+one and keeps the conversation context flat, which is the dominant cost of large /sync runs.
+</description>
+
+<importantNotes>
+- (!important!) **Content is read from \`path\` on the server** - it does not pass through the model context
+- (!important!) **Stores the raw file verbatim** by default (higher search fidelity); pass \`content\` to override
+- (!important!) **Idempotent** - the target documentId is fully re-synced (delete + recreate) each call
+- (!important!) **Returns terse counts only** - { documentId, bytes, chunks, embeddedChunks, linkedEntities }
+- (!important!) **Entity linking** happens automatically via term-matching during embedding; \`entityNames\` adds explicit links
+</importantNotes>
+
+<whenToUseThisTool>
+- During /sync to persist a Markdown/SSOT file into the RAG store without paying the context cost of reading it into the model
+- When re-syncing decision logs, current-focus, wiki pages, or any file-backed document
+- As a one-call replacement for the manual storeDocument + chunkDocument + embedChunks + linkEntitiesToDocument sequence
+</whenToUseThisTool>
+
+<bestPractices>
+- Use the document's canonical ID (e.g. \`current-focus\`, \`wiki-project-context\`)
+- Keep an entity-name literal (e.g. wiki anchor line "RAG entity: ...") in the file so term-matching can link entities; a warning is returned if linkedEntities is 0
+- Pass \`metadata.updated\` (YYYY-MM-DD) for change tracking
+</bestPractices>
+
+<examples>
+- Sync current-focus: {"path": "/abs/decisions/current-focus.md", "documentId": "current-focus", "metadata": {"updated": "2026-05-23"}}
+- With explicit links: {"path": "/abs/wiki/project-context.md", "documentId": "wiki-project-context", "entityNames": ["Ultimate AI Personal Assistant"]}
+</examples>`;
+const syncDocumentFromFileSchema = {
+    path: z.string().describe('Absolute path of the file to sync (read server-side)'),
+    documentId: z.string().describe('RAG document ID to (re)create from the file'),
+    metadata: z.record(z.any()).optional().describe('Metadata merged into the stored document'),
+    content: z.string().optional().describe('Optional content override; stored instead of reading the file'),
+    entityNames: z.array(z.string()).optional().describe('Optional entities to explicitly link'),
+    chunkParams: z.record(z.any()).optional().describe('Optional chunking parameters { maxTokens, overlap }'),
+};
+export const syncDocumentFromFileTool = {
+    capability: syncDocumentFromFileCapability,
+    description: syncDocumentFromFileDescription,
+    schema: syncDocumentFromFileSchema,
+    annotations: { idempotentHint: true },
+};
 // Export all RAG tools
 export const ragTools = {
     storeDocument: storeDocumentTool,
@@ -526,4 +613,5 @@ export const ragTools = {
     getKnowledgeGraphStats: getKnowledgeGraphStatsTool,
     deleteDocuments: deleteDocumentsTool,
     listDocuments: listDocumentsTool,
+    syncDocumentFromFile: syncDocumentFromFileTool,
 };

package/dist/src/tools/tool-registry.d.ts

@@ -20,6 +20,7 @@ export declare const allTools: {
     getKnowledgeGraphStats: ToolDefinition;
     deleteDocuments: ToolDefinition;
     listDocuments: ToolDefinition;
+    syncDocumentFromFile: ToolDefinition;
     createEntities: ToolDefinition;
     createRelations: ToolDefinition;
     addObservations: ToolDefinition;

package/package.json

@@ -1,7 +1,25 @@
 {
   "name": "rag-memory-epf-mcp",
-  "version": "3.3.5",
-  "description": "MCP server for project-local RAG memory with knowledge graph and multilingual vector search",
+  "version": "3.4.0",
+  "description": "Project-local RAG memory MCP server — knowledge graph + multilingual vector + FTS5 in a single SQLite file. Per-project isolation, 30 MCP tools, codepoint-safe chunking (Korean/CJK/emoji).",
+  "keywords": [
+    "mcp",
+    "model-context-protocol",
+    "rag",
+    "knowledge-graph",
+    "vector-search",
+    "fts5",
+    "sqlite",
+    "embeddings",
+    "bge-m3",
+    "multilingual",
+    "korean",
+    "claude-code",
+    "gemini-cli",
+    "codex-cli",
+    "memory",
+    "agent-memory"
+  ],
   "license": "MIT",
   "author": "bripin123",
   "homepage": "https://github.com/bripin123/rag-memory-epf-mcp",
@@ -21,7 +39,10 @@
   "scripts": {
     "build": "tsc && shx chmod +x dist/*.js",
     "prepare": "npm run build",
-    "watch": "tsc --watch"
+    "watch": "tsc --watch",
+    "verify:invariants": "node test/chunk-invariants.test.mjs",
+    "test": "npm run build && npm run verify:invariants",
+    "prepublishOnly": "npm run build && npm run verify:invariants"
   },
   "dependencies": {
     "@huggingface/transformers": "^3.5.1",