@lotargo/memory_plugin 1.3.0 → 1.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +107 -35
- package/mcp-server/config/config_manager.js +3 -3
- package/mcp-server/index.js +3 -9
- package/mcp-server/ingest/pipeline.js +8 -0
- package/mcp-server/memory.js +203 -192
- package/mcp-server/ml/model_manager.js +2 -2
- package/package.json +20 -5
- package/skills/using-memory/SKILL.md +25 -7
- package/mcp-server/admin/server.js +0 -228
package/README.md
CHANGED
|
@@ -29,7 +29,7 @@ Automatically remembers durable user facts, ingests complex document repositorie
|
|
|
29
29
|
|
|
30
30
|
Standard AI coding assistants lose context as soon as a chat session closes or a conversation is reset. You end up repeatedly re-explaining your preferences, architectural decisions, coding style, or project conventions.
|
|
31
31
|
|
|
32
|
-
`@lotargo/memory_plugin` gives your AI tools durable, 100% local long-term memory and document retrieval capabilities that persist across restarts and work seamlessly across all supported coding environments.
|
|
32
|
+
`@lotargo/memory_plugin` gives your AI tools durable, **persistent**, 100% local long-term memory and document retrieval capabilities that persist across restarts and work seamlessly across all supported coding environments. Any LLM-based coding agent (OpenCode, Claude Code, Codex, Antigravity) can query its own memory and hybrid knowledge base via the **Model Context Protocol (MCP)**.
|
|
33
33
|
|
|
34
34
|
> **Project Scope & Runtime Notes**:
|
|
35
35
|
> `@lotargo/memory_plugin` is designed primarily as a practical plugin to expand capabilities and streamline daily interaction with AI coding tools. Benchmark scores in this repository represent internal synthetic evaluation runs and are not intended as generalized RAG benchmarks.
|
|
@@ -58,7 +58,7 @@ Standard AI coding assistants lose context as soon as a chat session closes or a
|
|
|
58
58
|
Run the setup command to configure all detected AI environments automatically:
|
|
59
59
|
|
|
60
60
|
```bash
|
|
61
|
-
# Recommended: Global installation & setup (works across local CLI,
|
|
61
|
+
# Recommended: Global installation & setup (works across local CLI, Docker, and CI)
|
|
62
62
|
npm install -g @lotargo/memory_plugin && memory_plugin setup
|
|
63
63
|
|
|
64
64
|
# Or via npx
|
|
@@ -81,20 +81,23 @@ npm install -g @lotargo/memory_plugin && memory_plugin setup --claude
|
|
|
81
81
|
npm install -g @lotargo/memory_plugin && memory_plugin setup --codex
|
|
82
82
|
```
|
|
83
83
|
|
|
84
|
+
`setup` also accepts `--gemini` (alias for Antigravity) and `--local` (also registers the MCP server in the project-local `.agents/` directory for Antigravity). Without a specific flag, all detected environments are configured.
|
|
85
|
+
|
|
84
86
|
---
|
|
85
87
|
|
|
86
88
|
## Dual-Layer Architecture
|
|
87
89
|
|
|
88
90
|
1. **Layer 1: Notebook Store (Durable Facts)**
|
|
89
|
-
- **Tools**: `remember`, `recall`, `forget`
|
|
91
|
+
- **Tools**: `remember`, `recall`, `forget`, `update_fact`, `memory_info`
|
|
90
92
|
- **Scope**: User preferences, identity, project conventions, system rules.
|
|
91
93
|
- **Storage**: Human-readable Markdown format (`global` and per-project stores).
|
|
92
94
|
- **Performance**: Guaranteed 100% precision instant lookup without vector degradation or threshold filtering.
|
|
95
|
+
- **Project Scoping**: Project stores are bound to their canonical directory path (e.g. `f__projects_plugins_memory.md`), so identical project names in different folders never collide. Legacy basename stores are migrated automatically with a collision guard.
|
|
93
96
|
|
|
94
97
|
2. **Layer 2: RAG Knowledge Base (Technical Documents & Codebases)**
|
|
95
|
-
- **Tools**: `ingest_document`, `query_knowledge_base`, `manage_knowledge_base`
|
|
96
|
-
- **Capabilities**: Ingests raw text files, Markdown, HTML, and full code repositories.
|
|
97
|
-
- **Engine Components**: 3-tier hierarchy chunking (Big / Medium / Small), SQLite FTS5 BM25 search, ONNX dense vector embeddings (`multilingual-e5-small`), Reciprocal Rank Fusion (RRF / RSF), and GraphRAG Lite code symbol extraction.
|
|
98
|
+
- **Tools**: `ingest_document`, `query_knowledge_base`, `manage_knowledge_base`, `link_knowledge`
|
|
99
|
+
- **Capabilities**: Ingests raw text files, Markdown, HTML, Web URLs, and full code repositories.
|
|
100
|
+
- **Engine Components**: 3-tier hierarchy chunking (Big / Medium / Small), SQLite FTS5 BM25 search, ONNX dense vector embeddings (`multilingual-e5-small`), Reciprocal Rank Fusion (RRF / RSF), cross-encoder reranking (optional), and GraphRAG Lite code symbol extraction.
|
|
98
101
|
|
|
99
102
|
---
|
|
100
103
|
|
|
@@ -103,11 +106,14 @@ npm install -g @lotargo/memory_plugin && memory_plugin setup --codex
|
|
|
103
106
|
- **Zero Heavy Infrastructure**: No Docker, no Python server, no C++ compilation (`node-gyp`). Uses Node.js native SQLite database.
|
|
104
107
|
- **Bilingual & Multilingual Support**: State-of-the-art semantic precision across Russian, English, and technical code symbols.
|
|
105
108
|
- **3-Tier Hierarchy Chunking**: Document (Big) -> Section (Medium) -> Micro-Chunk (Small).
|
|
106
|
-
- **Hybrid RRF/RSF Fusion**: Combines SQLite FTS5 keyword precision with ONNX dense vector similarity.
|
|
109
|
+
- **Hybrid RRF/RSF Fusion**: Combines SQLite FTS5 keyword precision with ONNX dense vector similarity; lexical-only fallback when embeddings are disabled.
|
|
110
|
+
- **Semantic Search**: Cosine-similarity vector retrieval with multilingual ONNX embeddings (E5 / BGE model families).
|
|
111
|
+
- **Path-Based Project Memory**: Per-project stores keyed by canonical directory path, with automatic migration of legacy stores.
|
|
107
112
|
- **GraphRAG Lite**: Automatically links documents and extracted code symbols (classes, functions, types).
|
|
113
|
+
- **Memory-to-Knowledge Linking**: Associate notebook facts with specific documents or line ranges in the RAG base.
|
|
108
114
|
- **Content-Addressable Storage (CAS)**: Local S3-style compressed blob store for raw original documents.
|
|
109
115
|
- **Dual-Source Model Failover**: Automatic HuggingFace CDN model downloading with GitHub Repository Mirror fallback.
|
|
110
|
-
- **Interactive
|
|
116
|
+
- **Interactive TUI**: Terminal GUI (CLI menu) for runtime engine tuning, snapshot export/import, model cache management, and diagnostics.
|
|
111
117
|
|
|
112
118
|
---
|
|
113
119
|
|
|
@@ -119,17 +125,17 @@ npm install -g @lotargo/memory_plugin && memory_plugin setup --codex
|
|
|
119
125
|
| **OpenCode** | Native | Native plugin + MCP Server (`~/.config/opencode/opencode.json`) |
|
|
120
126
|
| **Claude Code** | Supported | MCP Server (`~/.claude.json`) |
|
|
121
127
|
| **Codex** | Supported | MCP Server (`~/.codex/config.toml`) |
|
|
122
|
-
| **Google Jules** | Experimental | MCP Server via global install (`npm install -g @lotargo/memory_plugin`)
|
|
128
|
+
| **Google Jules** | Experimental | MCP Server via global install + setup (`npm install -g @lotargo/memory_plugin && memory_plugin setup`) |
|
|
123
129
|
|
|
124
130
|
### Google Jules Integration (Experimental)
|
|
125
131
|
|
|
126
|
-
The plugin has been verified inside the **Google Jules** cloud workspace environment.
|
|
132
|
+
The plugin has been verified inside the **Google Jules** cloud workspace environment. This feature is **experimental**.
|
|
127
133
|
|
|
128
|
-
- **Setup Method**: Global pre-installation:
|
|
134
|
+
- **Setup Method**: Global pre-installation with auto-setup:
|
|
129
135
|
```bash
|
|
130
|
-
npm install -g @lotargo/memory_plugin
|
|
136
|
+
npm install -g @lotargo/memory_plugin && memory_plugin setup
|
|
131
137
|
```
|
|
132
|
-
- **Verification**: Google Jules automatically discovers the registered MCP server upon workspace initialization and seamlessly interacts with memory & RAG tools
|
|
138
|
+
- **Verification**: All current tools and capabilities have been verified inside the Google Jules cloud workspace. Google Jules automatically discovers the registered MCP server upon workspace initialization and seamlessly interacts with the full set of memory & RAG tools — `remember`, `recall`, `forget`, `update_fact`, `memory_info`, `link_knowledge`, `ingest_document`, `query_knowledge_base`, and `manage_knowledge_base` — including project-scoped memory, knowledge linking, and snapshot export/import.
|
|
133
139
|
- **Current Limitation**: All memory stores and vector indexes operate locally within the workspace environment. Cross-session cloud synchronization across different Jules runs is planned for upcoming releases.
|
|
134
140
|
|
|
135
141
|
---
|
|
@@ -138,11 +144,13 @@ The plugin has been verified inside the **Google Jules** cloud workspace environ
|
|
|
138
144
|
|
|
139
145
|
### 1. Memory Tools (Key-Value Notebook)
|
|
140
146
|
|
|
141
|
-
| Tool
|
|
142
|
-
|
|
|
143
|
-
| `remember`
|
|
144
|
-
| `recall`
|
|
145
|
-
| `forget`
|
|
147
|
+
| Tool | Scope / Target | Description |
|
|
148
|
+
| :-------------- | :-------------------------------------- | :---------------------------------------------------------------- |
|
|
149
|
+
| `remember` | `global` or `project` | Save an important durable fact or preference |
|
|
150
|
+
| `recall` | `project`, `global`, `all`, `list_projects` | Display saved facts; read another project's store via `project: '<path>'` |
|
|
151
|
+
| `forget` | Index ID, range, or query | Remove a saved fact (e.g. `"3-30"` ranges; `force` for protected) |
|
|
152
|
+
| `update_fact` | Index ID, metadata id, or text | Rewrite a fact while preserving its original date and links |
|
|
153
|
+
| `memory_info` | - | Show storage paths, fact counts, RAG stats, and package version |
|
|
146
154
|
|
|
147
155
|
### 2. Hybrid RAG Knowledge Base Tools
|
|
148
156
|
|
|
@@ -150,13 +158,50 @@ The plugin has been verified inside the **Google Jules** cloud workspace environ
|
|
|
150
158
|
| :---------------------- | :------------------------------ | :--------------------------------------------------------------------------- |
|
|
151
159
|
| `ingest_document` | Local files, Web URLs, Raw text | Ingest into 3-tier index with ONNX vector embeddings & symbol extraction |
|
|
152
160
|
| `query_knowledge_base` | Text / Code query | Perform hybrid RSF/RRF search (BM25 + Vector) to retrieve candidate sections |
|
|
153
|
-
| `manage_knowledge_base` | Actions / Documents |
|
|
161
|
+
| `manage_knowledge_base` | Actions / Documents | Stats, list, read, delete documents, or export/import snapshots |
|
|
162
|
+
| `link_knowledge` | Facts + Document ranges | Explicitly link a memory fact to a KB document or line range |
|
|
163
|
+
|
|
164
|
+
### 3. Native OpenCode Plugin
|
|
165
|
+
|
|
166
|
+
When installed as an OpenCode plugin, all MCP tools above plus `list-mcp-tools` and `mcp-reminder` are exposed. A chat hook (`experimental.chat.messages.transform`) automatically injects your saved memory into every conversation as a `<MEMORY>` block, so your agent starts each session already knowing your preferences and project context.
|
|
167
|
+
|
|
168
|
+
---
|
|
169
|
+
|
|
170
|
+
## GraphRAG Lite
|
|
171
|
+
|
|
172
|
+
The RAG engine includes a lightweight graph layer built on the same SQLite database. It combines code symbol extraction, hierarchy edges, and explicit memory-to-document links without requiring a separate graph store or an LLM at ingest time.
|
|
173
|
+
|
|
174
|
+
**Code Symbol Extraction** — during `ingest_document`, code symbols are extracted from the chunk content using fast regex heuristics (no language model needed):
|
|
175
|
+
|
|
176
|
+
- JavaScript / TypeScript: `function`, `class`, `interface`, `type`, `enum`, `const`, `let`, `var`
|
|
177
|
+
- Python: `def`, `class`
|
|
178
|
+
- Symbols shorter than 3 characters and reserved keywords (`const`, `let`, `var`, `function`, `class`, `import`, `export`, `from`, `return`, `if`, `for`, `while`, `def`, `self`) are filtered out.
|
|
179
|
+
|
|
180
|
+
**Graph Edges** — three built-in relation types are created automatically, and custom relation types are supported for explicit linking:
|
|
181
|
+
|
|
182
|
+
| Relation Type | Direction / Example |
|
|
183
|
+
| :------------------ | :----------------------------------------------------------- |
|
|
184
|
+
| `CONTAINS` | Document -> Section -> Micro-Chunk (3-tier hierarchy) |
|
|
185
|
+
| `DEFINES_SYMBOL` | Section -> `symbol:<name>` (extracted code symbol) |
|
|
186
|
+
| `LINKS_TO` (default) | Memory fact -> Document or line range (via `link_knowledge`) |
|
|
187
|
+
|
|
188
|
+
**Memory-to-Knowledge Linking** — the `link_knowledge` tool connects a notebook fact to a specific document or line range (`RULES_FOR`, `IMPLEMENTS`, `EXPLAINS`, `REFERENCES`, ...):
|
|
189
|
+
|
|
190
|
+
- `link` — create the link and its graph edge
|
|
191
|
+
- `list_links` — list all links, optionally filtered by fact key
|
|
192
|
+
- `get_doc_links` — list all links pointing to a given document
|
|
193
|
+
|
|
194
|
+
Linked facts are surfaced automatically in `recall` results as `🔗 [Linked Docs: ...]`, and `remember` accepts an optional `docId` to link immediately.
|
|
195
|
+
|
|
196
|
+
**Retrieval Integration** — `query_knowledge_base` augments each retrieved section with `defined_symbols`: the code symbols defined in that same section (a single-hop lookup along `DEFINES_SYMBOL` edges). Symbol extraction also improves BM25 scoring, since symbol names become searchable tokens.
|
|
197
|
+
|
|
198
|
+
**Lifecycle** — edges are rebuilt transactionally on re-ingest of the same document, and `manage_knowledge_base` delete operations clean up all graph edges and knowledge links owned by the document (including `GLOB`-matched section/micro-chunk suffixes).
|
|
154
199
|
|
|
155
200
|
---
|
|
156
201
|
|
|
157
|
-
## Interactive CLI
|
|
202
|
+
## Interactive TUI (CLI Menu)
|
|
158
203
|
|
|
159
|
-
Launch the interactive
|
|
204
|
+
Launch the interactive terminal UI to manage engine settings, inspect databases, tune retrieval parameters, and run diagnostics:
|
|
160
205
|
|
|
161
206
|
```bash
|
|
162
207
|
# From local repository folder:
|
|
@@ -170,23 +215,50 @@ memory_plugin cli
|
|
|
170
215
|
memory-cli
|
|
171
216
|
```
|
|
172
217
|
|
|
173
|
-
###
|
|
218
|
+
### TUI Menu Overview
|
|
174
219
|
|
|
175
220
|
The interactive menu exposes runtime parameters that `hybridQuery` honors, allowing search behavior modifications without restarting the MCP server. Use **Up / Down** arrows to navigate, **ENTER** to select, and **BACKSPACE** to go back.
|
|
176
221
|
|
|
177
|
-
| Block
|
|
178
|
-
|
|
|
179
|
-
| **Engine Settings**
|
|
180
|
-
|
|
|
181
|
-
|
|
|
182
|
-
|
|
|
183
|
-
|
|
|
184
|
-
|
|
|
185
|
-
|
|
|
186
|
-
|
|
|
187
|
-
|
|
|
222
|
+
| Block | Menu Item | Functionality |
|
|
223
|
+
| :-------------------------------------- | :------------------------------ | :----------------------------------------------------------------------------- |
|
|
224
|
+
| **Engine & Hybrid Search Settings** | Fusion Algorithm | Switch between `rsf`, `rrf`, `semantic_only`, `lexical_only`. |
|
|
225
|
+
| | RSF Alpha Balance | Weight of semantic over lexical in `rsf` fusion (`α ∈ [0,1]`). Default: `0.5`. |
|
|
226
|
+
| | Embedding Model | Select ONNX model (e.g. `Xenova/multilingual-e5-small`, custom HF models). |
|
|
227
|
+
| | Reranker Model | Enable Cross-Encoder reranking or disable for zero-latency fusion. |
|
|
228
|
+
| | Vector Batch Size | Ingestion vector batch size `[1 - 256]` (default `12`). |
|
|
229
|
+
| | GPU Attention Budget | GPU micro-batch attention budget `[1M - 16M]` (default `2.0M`, ~1.5 GB VRAM). |
|
|
230
|
+
| | CPU WASM Threads | ONNX WASM threads: `0` auto-detect or `1-16`. |
|
|
231
|
+
| | Execution Hardware | `cpu` or `webgpu` (experimental). |
|
|
232
|
+
| **Knowledge Base & Storage Management** | Notebook (Layer 1 Facts) | Browse and manage `global` and per-project `.md` fact stores. |
|
|
233
|
+
| | RAG Docs (Layer 2 Base) | List ingested documents, inspect chunk counts, and purge entries. |
|
|
234
|
+
| | Snapshot Export / Import | Export or restore the full RAG base + blob store as a JSON snapshot. |
|
|
235
|
+
| | Manage & Purge ML Model Cache | Inspect or purge downloaded ONNX model weights. |
|
|
236
|
+
| | Hard Reset | Purge RAG base, blob storage, and graph edges. |
|
|
237
|
+
| **Global Prompt & Integration** | Enable / Disable Global Prompt | Inject memory instructions into `~/.gemini/config/AGENTS.md`, `~/.codex/AGENTS.md`, `~/.claude/CLAUDE.md`. |
|
|
238
|
+
| **Diagnostics & System Actions** | Search Quality Benchmark | Execute in-process search evaluation across the benchmark query set. |
|
|
239
|
+
| | Verification Query | Run a test `hybridQuery` against the active index. |
|
|
240
|
+
| | Clear Benchmark Corpus Cache | Clear cached benchmark corpus. |
|
|
241
|
+
| | Reset Config to Factory Defaults| Restore default engine configuration. |
|
|
242
|
+
|
|
243
|
+
Settings persist to `<memory-dir>/config.json` and are immediately loaded by the MCP server.
|
|
244
|
+
|
|
245
|
+
---
|
|
246
|
+
|
|
247
|
+
## Configuration
|
|
248
|
+
|
|
249
|
+
The engine is configured through `<memory-dir>/config.json` (created with defaults on first run):
|
|
188
250
|
|
|
189
|
-
|
|
251
|
+
| Key | Default | Description |
|
|
252
|
+
| :-------------------- | :--------------------------------- | :-------------------------------------------------------------- |
|
|
253
|
+
| `fusionAlgorithm` | `rsf` | `rsf`, `rrf`, `semantic_only`, or `lexical_only` |
|
|
254
|
+
| `alpha` | `0.5` | Vector vs BM25 weight in RSF `[0.0 - 1.0]` |
|
|
255
|
+
| `embeddingModel` | `Xenova/multilingual-e5-small` | ONNX dense embedding model (E5 / BGE families supported) |
|
|
256
|
+
| `rerankerModel` | `none` | Cross-encoder reranker, or `Xenova/bge-reranker-base` |
|
|
257
|
+
| `rerankerEnabled` | `false` | Enable cross-encoder re-ranking |
|
|
258
|
+
| `batchSize` | `12` | Ingestion vector batch size `[1 - 256]` |
|
|
259
|
+
| `gpuAttentionBudget` | `2000000` | GPU micro-batch attention budget `[1M - 16M]` |
|
|
260
|
+
| `onnxThreads` | `0` | ONNX WASM threads: `0` auto-detect, or `1-16` |
|
|
261
|
+
| `executionDevice` | `cpu` | `cpu` or `webgpu` (experimental) |
|
|
190
262
|
|
|
191
263
|
---
|
|
192
264
|
|
|
@@ -241,7 +313,7 @@ Detailed technical documentation and architectural specifications are available
|
|
|
241
313
|
|
|
242
314
|
## Storage & Privacy
|
|
243
315
|
|
|
244
|
-
- **100% Local Storage**: All SQLite indexes, ONNX models, CAS blobs, and Markdown notebooks are stored locally
|
|
316
|
+
- **100% Local Storage**: All SQLite indexes, ONNX models, CAS blobs, and Markdown notebooks are stored locally in the memory directory. The location resolves to, in order of priority: `$MEMORY_DIR`, `$OPENCODE_CONFIG_DIR/memory`, the legacy `~/.config/opencode/memory` (on Windows: `%LOCALAPPDATA%\opencode\memory`), or `$XDG_CONFIG_HOME/opencode/memory`.
|
|
245
317
|
- **Dual-Source Failover Model Fetching**: Primary model weights are fetched from HuggingFace CDN with automatic failover to GitHub Repository Mirror.
|
|
246
318
|
- **Zero External Telemetry**: No third-party network calls are required after initial model setup.
|
|
247
319
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import fs from "fs";
|
|
2
2
|
import path from "path";
|
|
3
|
-
import { MEMORY_DIR,
|
|
3
|
+
import { MEMORY_DIR, ensureDirSync } from "../memory.js";
|
|
4
4
|
|
|
5
5
|
const CONFIG_FILE = path.join(MEMORY_DIR, "config.json");
|
|
6
6
|
|
|
@@ -23,7 +23,7 @@ export function getConfig() {
|
|
|
23
23
|
return cachedConfig;
|
|
24
24
|
}
|
|
25
25
|
|
|
26
|
-
|
|
26
|
+
ensureDirSync();
|
|
27
27
|
|
|
28
28
|
if (fs.existsSync(CONFIG_FILE)) {
|
|
29
29
|
try {
|
|
@@ -42,7 +42,7 @@ export function getConfig() {
|
|
|
42
42
|
}
|
|
43
43
|
|
|
44
44
|
export function saveConfig(newConfig) {
|
|
45
|
-
|
|
45
|
+
ensureDirSync();
|
|
46
46
|
cachedConfig = Object.freeze({ ...DEFAULT_CONFIG, ...newConfig });
|
|
47
47
|
try {
|
|
48
48
|
fs.writeFileSync(CONFIG_FILE, JSON.stringify(cachedConfig, null, 2), "utf-8");
|
package/mcp-server/index.js
CHANGED
|
@@ -39,13 +39,6 @@ if (cliArgs.includes("setup") || cliArgs.includes("install") || cliArgs.includes
|
|
|
39
39
|
process.exit(0);
|
|
40
40
|
}
|
|
41
41
|
|
|
42
|
-
if (cliArgs.includes("admin") || cliArgs.includes("--admin") || cliArgs.includes("-a")) {
|
|
43
|
-
const { startAdminServer } = await import("./admin/server.js");
|
|
44
|
-
await startAdminServer();
|
|
45
|
-
// Keep process running for web server
|
|
46
|
-
await new Promise(() => {});
|
|
47
|
-
}
|
|
48
|
-
|
|
49
42
|
if (cliArgs.includes("cli") || cliArgs.includes("config") || cliArgs.includes("--cli") || cliArgs.includes("-c")) {
|
|
50
43
|
const { runCli } = await import("./cli.js");
|
|
51
44
|
await runCli();
|
|
@@ -453,12 +446,13 @@ server.registerTool(
|
|
|
453
446
|
description:
|
|
454
447
|
"Ingest a document into the RAG knowledge base. " +
|
|
455
448
|
"Accepts local file paths, web URLs, or raw Markdown/text content. " +
|
|
449
|
+
"For type='file' the file is read from disk and indexed with a code-block wrapper. " +
|
|
456
450
|
"For type='url' the page is fetched and its content is indexed (not just the URL). " +
|
|
457
451
|
"Processes document through 3-tier hierarchy chunking (Big/Medium/Small), " +
|
|
458
452
|
"computes dense vectors, and extracts GraphRAG code symbols.",
|
|
459
453
|
inputSchema: z.object({
|
|
460
|
-
content: z.string().describe("Raw text content, file path, or web URL"),
|
|
461
|
-
type: z.enum(["text", "file", "url"]).nullish().transform((v) => v || "text").describe("Input content type: 'text', 'file', or 'url' (
|
|
454
|
+
content: z.string().describe("Raw text content, file path, or web URL. For type='file' this can be the file path (reads from disk) or the file content directly"),
|
|
455
|
+
type: z.enum(["text", "file", "url"]).nullish().transform((v) => v || "text").describe("Input content type: 'text' (raw content), 'file' (reads from disk, wraps in code block), or 'url' (fetches page content)"),
|
|
462
456
|
title: optStr().describe("Document title"),
|
|
463
457
|
path: optStr().describe("Original document file path"),
|
|
464
458
|
generateEmbeddings: defBool(true).describe("Compute dense vector embeddings"),
|
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { randomUUID } from "node:crypto";
|
|
2
|
+
import { readFile } from "node:fs/promises";
|
|
2
3
|
import { getDatabase, BLOBS_DIR } from "../db/database.js";
|
|
3
4
|
import { saveBlob, deleteBlob } from "../storage/blob_store.js";
|
|
4
5
|
import { normalizeContent, fetchUrlContent } from "./normalizer.js";
|
|
@@ -28,6 +29,13 @@ export async function ingestDocument({
|
|
|
28
29
|
effectiveType = "text";
|
|
29
30
|
effectiveTitle = title || fetched.title;
|
|
30
31
|
effectivePath = path || fetched.finalUrl || content;
|
|
32
|
+
} else if (type === "file") {
|
|
33
|
+
const filePath = effectivePath || content;
|
|
34
|
+
const needsRead = !content || content === filePath;
|
|
35
|
+
if (needsRead && filePath) {
|
|
36
|
+
content = await readFile(filePath, "utf-8");
|
|
37
|
+
effectivePath = filePath;
|
|
38
|
+
}
|
|
31
39
|
}
|
|
32
40
|
|
|
33
41
|
const { markdown, title: docTitle, metadata } = normalizeContent({ content, type: effectiveType, path: effectivePath, title: effectiveTitle });
|
package/mcp-server/memory.js
CHANGED
|
@@ -1,192 +1,203 @@
|
|
|
1
|
-
import { readFile, writeFile, mkdir, unlink, readdir } from "fs/promises";
|
|
2
|
-
import { existsSync } from "fs";
|
|
3
|
-
import { join, basename, resolve } from "path";
|
|
4
|
-
import { homedir } from "os";
|
|
5
|
-
|
|
6
|
-
function resolveMemoryDir() {
|
|
7
|
-
if (process.env.MEMORY_DIR) return process.env.MEMORY_DIR;
|
|
8
|
-
if (process.env.OPENCODE_CONFIG_DIR) return join(process.env.OPENCODE_CONFIG_DIR, "memory");
|
|
9
|
-
|
|
10
|
-
const legacyDir = join(homedir(), ".config", "opencode", "memory");
|
|
11
|
-
if (existsSync(legacyDir)) {
|
|
12
|
-
return legacyDir;
|
|
13
|
-
}
|
|
14
|
-
|
|
15
|
-
if (process.platform === "win32") {
|
|
16
|
-
const appData = process.env.LOCALAPPDATA || process.env.APPDATA || join(homedir(), "AppData", "Local");
|
|
17
|
-
return join(appData, "opencode", "memory");
|
|
18
|
-
}
|
|
19
|
-
|
|
20
|
-
const configHome = process.env.XDG_CONFIG_HOME || join(homedir(), ".config");
|
|
21
|
-
return join(configHome, "opencode", "memory");
|
|
22
|
-
}
|
|
23
|
-
|
|
24
|
-
export const MEMORY_DIR = resolveMemoryDir();
|
|
25
|
-
export const GLOBAL_KEY = "global";
|
|
26
|
-
|
|
27
|
-
export async function ensureDir() {
|
|
28
|
-
if (!existsSync(MEMORY_DIR)) await mkdir(MEMORY_DIR, { recursive: true });
|
|
29
|
-
const storageDir = join(MEMORY_DIR, "storage");
|
|
30
|
-
const blobsDir = join(storageDir, "blobs");
|
|
31
|
-
const modelsDir = join(storageDir, "models");
|
|
32
|
-
const exportsDir = join(MEMORY_DIR, "exports");
|
|
33
|
-
if (!existsSync(blobsDir)) await mkdir(blobsDir, { recursive: true });
|
|
34
|
-
if (!existsSync(modelsDir)) await mkdir(modelsDir, { recursive: true });
|
|
35
|
-
if (!existsSync(exportsDir)) await mkdir(exportsDir, { recursive: true });
|
|
36
|
-
}
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
}
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
}
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
}
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
const
|
|
190
|
-
const
|
|
191
|
-
|
|
192
|
-
|
|
1
|
+
import { readFile, writeFile, mkdir, unlink, readdir } from "fs/promises";
|
|
2
|
+
import { existsSync, mkdirSync } from "fs";
|
|
3
|
+
import { join, basename, resolve } from "path";
|
|
4
|
+
import { homedir } from "os";
|
|
5
|
+
|
|
6
|
+
function resolveMemoryDir() {
|
|
7
|
+
if (process.env.MEMORY_DIR) return process.env.MEMORY_DIR;
|
|
8
|
+
if (process.env.OPENCODE_CONFIG_DIR) return join(process.env.OPENCODE_CONFIG_DIR, "memory");
|
|
9
|
+
|
|
10
|
+
const legacyDir = join(homedir(), ".config", "opencode", "memory");
|
|
11
|
+
if (existsSync(legacyDir)) {
|
|
12
|
+
return legacyDir;
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
if (process.platform === "win32") {
|
|
16
|
+
const appData = process.env.LOCALAPPDATA || process.env.APPDATA || join(homedir(), "AppData", "Local");
|
|
17
|
+
return join(appData, "opencode", "memory");
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
const configHome = process.env.XDG_CONFIG_HOME || join(homedir(), ".config");
|
|
21
|
+
return join(configHome, "opencode", "memory");
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
export const MEMORY_DIR = resolveMemoryDir();
|
|
25
|
+
export const GLOBAL_KEY = "global";
|
|
26
|
+
|
|
27
|
+
export async function ensureDir() {
|
|
28
|
+
if (!existsSync(MEMORY_DIR)) await mkdir(MEMORY_DIR, { recursive: true });
|
|
29
|
+
const storageDir = join(MEMORY_DIR, "storage");
|
|
30
|
+
const blobsDir = join(storageDir, "blobs");
|
|
31
|
+
const modelsDir = join(storageDir, "models");
|
|
32
|
+
const exportsDir = join(MEMORY_DIR, "exports");
|
|
33
|
+
if (!existsSync(blobsDir)) await mkdir(blobsDir, { recursive: true });
|
|
34
|
+
if (!existsSync(modelsDir)) await mkdir(modelsDir, { recursive: true });
|
|
35
|
+
if (!existsSync(exportsDir)) await mkdir(exportsDir, { recursive: true });
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
export function ensureDirSync() {
|
|
39
|
+
if (!existsSync(MEMORY_DIR)) mkdirSync(MEMORY_DIR, { recursive: true });
|
|
40
|
+
const storageDir = join(MEMORY_DIR, "storage");
|
|
41
|
+
const blobsDir = join(storageDir, "blobs");
|
|
42
|
+
const modelsDir = join(storageDir, "models");
|
|
43
|
+
const exportsDir = join(MEMORY_DIR, "exports");
|
|
44
|
+
if (!existsSync(blobsDir)) mkdirSync(blobsDir, { recursive: true });
|
|
45
|
+
if (!existsSync(modelsDir)) mkdirSync(modelsDir, { recursive: true });
|
|
46
|
+
if (!existsSync(exportsDir)) mkdirSync(exportsDir, { recursive: true });
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
// Canonical absolute path key: forward slashes, lowercase drive letter on win32.
|
|
50
|
+
export function canonicalPath(dir) {
|
|
51
|
+
let p = resolve(dir || process.cwd());
|
|
52
|
+
if (process.platform === "win32") {
|
|
53
|
+
p = p.replace(/\\/g, "/").replace(/^([a-zA-Z]):/, (_, d) => `${d.toLowerCase()}:`);
|
|
54
|
+
}
|
|
55
|
+
return p;
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
// Project store key = full directory path. This removes basename collisions and
|
|
59
|
+
// binds each store to the real project location.
|
|
60
|
+
export function projectKey(worktree, directory) {
|
|
61
|
+
return canonicalPath(worktree || directory);
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
// Display label for a project (basename of the resolved directory).
|
|
65
|
+
export function projectName(worktree, directory) {
|
|
66
|
+
const dir = worktree || directory || process.cwd();
|
|
67
|
+
return dir ? basename(resolve(dir)) : "default";
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
export function scopeKey(scope, worktree, directory) {
|
|
71
|
+
return scope === "global" ? GLOBAL_KEY : projectKey(worktree, directory);
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
function slugify(key) {
|
|
75
|
+
return key.replace(/[^a-zA-Z0-9_-]/g, "_");
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
function memoryPath(key) {
|
|
79
|
+
return join(MEMORY_DIR, `${slugify(key)}.md`);
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
export function memoryFileName(key) {
|
|
83
|
+
return basename(memoryPath(key));
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
export function storeFilePath(key) {
|
|
87
|
+
return memoryPath(key);
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
function parseMeta(content) {
|
|
91
|
+
const m = content.match(/<!-- path: (.+?) -->/);
|
|
92
|
+
return { path: m ? m[1].trim() : null };
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
function isSimpleKey(key) {
|
|
96
|
+
return /^[a-zA-Z0-9_-]+$/.test(key);
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
// Lazy migration: when reading a project path store that doesn't exist yet but a
|
|
100
|
+
// legacy <basename>.md store (without path binding) does, claim it under the path.
|
|
101
|
+
async function maybeMigrateLegacy(key) {
|
|
102
|
+
if (key === GLOBAL_KEY || isSimpleKey(key)) return null;
|
|
103
|
+
const legacyBasename = basename(key);
|
|
104
|
+
if (!legacyBasename) return null;
|
|
105
|
+
const legacyFp = join(MEMORY_DIR, `${legacyBasename}.md`);
|
|
106
|
+
if (slugify(key) === legacyBasename || !existsSync(legacyFp)) return null;
|
|
107
|
+
const content = await readFile(legacyFp, "utf-8");
|
|
108
|
+
if (parseMeta(content).path) return null; // already bound to another project
|
|
109
|
+
// Collision guard: a different path with the same basename is already bound,
|
|
110
|
+
// so this legacy store is ambiguous and must not be silently claimed.
|
|
111
|
+
const files = await readdir(MEMORY_DIR).catch(() => []);
|
|
112
|
+
for (const f of files) {
|
|
113
|
+
if (!f.endsWith(".md") || f === `${legacyBasename}.md` || f === `${GLOBAL_KEY}.md`) continue;
|
|
114
|
+
try {
|
|
115
|
+
const other = parseMeta(await readFile(join(MEMORY_DIR, f), "utf-8")).path;
|
|
116
|
+
if (other && basename(other) === legacyBasename) return null;
|
|
117
|
+
} catch (e) {}
|
|
118
|
+
}
|
|
119
|
+
const facts = content.split("\n").filter((l) => l.startsWith("- ["));
|
|
120
|
+
await writeMemory(key, facts);
|
|
121
|
+
try {
|
|
122
|
+
await unlink(legacyFp);
|
|
123
|
+
} catch (e) {}
|
|
124
|
+
return facts;
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
export async function readMemory(key) {
|
|
128
|
+
const fp = memoryPath(key);
|
|
129
|
+
if (existsSync(fp)) {
|
|
130
|
+
const content = await readFile(fp, "utf-8");
|
|
131
|
+
return content.split("\n").filter((l) => l.startsWith("- ["));
|
|
132
|
+
}
|
|
133
|
+
const migrated = await maybeMigrateLegacy(key);
|
|
134
|
+
return migrated || [];
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
export async function readMemoryRaw(key) {
|
|
138
|
+
return (await readMemory(key)).map((e) => e.slice(2));
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
export async function writeMemory(key, entries) {
|
|
142
|
+
const lines = [];
|
|
143
|
+
if (key === GLOBAL_KEY) {
|
|
144
|
+
lines.push("# Global Memory", "");
|
|
145
|
+
} else {
|
|
146
|
+
lines.push(`# Memory: ${basename(key) || key}`, "");
|
|
147
|
+
if (!isSimpleKey(key)) {
|
|
148
|
+
lines.push(`<!-- path: ${key} -->`, "");
|
|
149
|
+
}
|
|
150
|
+
}
|
|
151
|
+
const content = lines.join("\n") + "\n" + (entries.length ? entries.join("\n") + "\n" : "");
|
|
152
|
+
await writeFile(memoryPath(key), content);
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
export async function listProjectStores() {
|
|
156
|
+
const stores = [];
|
|
157
|
+
const files = await readdir(MEMORY_DIR).catch(() => []);
|
|
158
|
+
for (const f of files) {
|
|
159
|
+
if (!f.endsWith(".md") || f === `${GLOBAL_KEY}.md`) continue;
|
|
160
|
+
const fp = join(MEMORY_DIR, f);
|
|
161
|
+
let content = "";
|
|
162
|
+
try {
|
|
163
|
+
content = await readFile(fp, "utf-8");
|
|
164
|
+
} catch (e) {
|
|
165
|
+
continue;
|
|
166
|
+
}
|
|
167
|
+
const facts = content.split("\n").filter((l) => l.startsWith("- ["));
|
|
168
|
+
const meta = parseMeta(content);
|
|
169
|
+
const key = meta.path || f.slice(0, -3);
|
|
170
|
+
stores.push({
|
|
171
|
+
key,
|
|
172
|
+
path: meta.path,
|
|
173
|
+
basename: basename(meta.path || key) || key,
|
|
174
|
+
file: f,
|
|
175
|
+
count: facts.length,
|
|
176
|
+
legacy: !meta.path,
|
|
177
|
+
});
|
|
178
|
+
}
|
|
179
|
+
stores.sort((a, b) => a.basename.localeCompare(b.basename));
|
|
180
|
+
return stores;
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
// Bind an unbound legacy store (e.g. "comfy-meta-viewer") to a directory path.
|
|
184
|
+
export async function migrateLegacyStore(legacyKey, targetDir) {
|
|
185
|
+
const legacyFp = join(MEMORY_DIR, `${legacyKey.replace(/[^a-zA-Z0-9_-]/g, "_")}.md`);
|
|
186
|
+
if (!existsSync(legacyFp)) return { ok: false, reason: "not_found", key: legacyKey };
|
|
187
|
+
const content = await readFile(legacyFp, "utf-8");
|
|
188
|
+
if (parseMeta(content).path) return { ok: false, reason: "already_bound", key: legacyKey };
|
|
189
|
+
const targetKey = projectKey(targetDir, null);
|
|
190
|
+
const facts = content.split("\n").filter((l) => l.startsWith("- ["));
|
|
191
|
+
await writeMemory(targetKey, facts);
|
|
192
|
+
try {
|
|
193
|
+
await unlink(legacyFp);
|
|
194
|
+
} catch (e) {}
|
|
195
|
+
return { ok: true, key: targetKey, file: memoryPath(targetKey), facts: facts.length };
|
|
196
|
+
}
|
|
197
|
+
|
|
198
|
+
export function today() {
|
|
199
|
+
const d = new Date();
|
|
200
|
+
const date = `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, "0")}-${String(d.getDate()).padStart(2, "0")}`;
|
|
201
|
+
const time = `${String(d.getHours()).padStart(2, "0")}:${String(d.getMinutes()).padStart(2, "0")}`;
|
|
202
|
+
return `${date} ${time}`;
|
|
203
|
+
}
|
|
@@ -388,10 +388,10 @@ export async function getReranker(modelName = "Xenova/bge-reranker-base", progre
|
|
|
388
388
|
return rerankerInstance;
|
|
389
389
|
}
|
|
390
390
|
|
|
391
|
-
|
|
391
|
+
const cacheDir = ensureValidModelDirectory();
|
|
392
392
|
|
|
393
393
|
const { pipeline, env } = await import("@huggingface/transformers");
|
|
394
|
-
env.cacheDir =
|
|
394
|
+
env.cacheDir = cacheDir;
|
|
395
395
|
env.allowLocalModels = true;
|
|
396
396
|
env.allowRemoteModels = true;
|
|
397
397
|
env.remoteHost = "https://huggingface.co";
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@lotargo/memory_plugin",
|
|
3
|
-
"version": "1.3.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "1.3.2",
|
|
4
|
+
"description": "100% local hybrid RAG memory for AI coding agents (OpenCode, Claude Code, Codex, Antigravity). MCP server + plugin: persistent user facts, document ingestion, vector + SQLite FTS5 retrieval across sessions.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "opencode-plugin/index.js",
|
|
7
7
|
"scripts": {
|
|
@@ -33,14 +33,29 @@
|
|
|
33
33
|
"skills"
|
|
34
34
|
],
|
|
35
35
|
"keywords": [
|
|
36
|
+
"memory",
|
|
37
|
+
"persistent-memory",
|
|
38
|
+
"ai-memory",
|
|
39
|
+
"ai-agent",
|
|
40
|
+
"ai-agents",
|
|
41
|
+
"rag",
|
|
42
|
+
"knowledge-base",
|
|
43
|
+
"llm",
|
|
44
|
+
"mcp",
|
|
45
|
+
"mcp-server",
|
|
46
|
+
"model-context-protocol",
|
|
47
|
+
"mcp-tools",
|
|
36
48
|
"opencode",
|
|
37
49
|
"claude-code",
|
|
38
50
|
"codex",
|
|
39
51
|
"antigravity",
|
|
40
52
|
"plugin",
|
|
41
|
-
"
|
|
42
|
-
"
|
|
43
|
-
"
|
|
53
|
+
"vector-search",
|
|
54
|
+
"hybrid-search",
|
|
55
|
+
"semantic-search",
|
|
56
|
+
"embeddings",
|
|
57
|
+
"sqlite",
|
|
58
|
+
"fts5",
|
|
44
59
|
"context"
|
|
45
60
|
],
|
|
46
61
|
"author": "Lotargo",
|
|
@@ -1,14 +1,15 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: using-memory
|
|
3
|
-
description: Comprehensive guide for using the Memory
|
|
3
|
+
description: Comprehensive guide for using the Memory, Hybrid RAG Knowledge Engine & MCP Helper tools (remember, recall, forget, update_fact, memory_info, link_knowledge, ingest_document, query_knowledge_base, manage_knowledge_base, list-mcp-tools, mcp-reminder). Trigger proactively whenever user preferences, project conventions, technology stack choices, or architecture decisions are introduced, or when querying ingested documentation, indexing files/repos, managing persistent knowledge, or looking up available MCP tool integrations.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Using Memory
|
|
6
|
+
# Using Memory, Hybrid RAG Knowledge Engine & MCP Helper Tools
|
|
7
7
|
|
|
8
|
-
You have access to a persistent dual-layer memory engine supercharged with an **Agent-Driven Knowledge Graph
|
|
8
|
+
You have access to a persistent dual-layer memory engine supercharged with an **Agent-Driven Knowledge Graph** and general MCP integration helpers:
|
|
9
9
|
1. **Layer 1: Notebook Store (Key-Value Facts)**: Stores high-signal personal preferences, project conventions, and durable rules in clean Markdown.
|
|
10
10
|
2. **Layer 2: RAG Knowledge Base**: Indexes documentation, repositories, and technical guides for hybrid semantic retrieval.
|
|
11
11
|
3. **Layer 3: Agent-Driven Knowledge Graph**: Connects Notebook facts (Layer 1) to specific Knowledge Base documents, sections, and **exact line ranges** (Layer 2).
|
|
12
|
+
4. **Integration Layer (General MCP Helpers)**: Quickly discovers connected MCP servers and identifies appropriate tools for specific tasks.
|
|
12
13
|
|
|
13
14
|
---
|
|
14
15
|
|
|
@@ -28,7 +29,9 @@ You have access to a persistent dual-layer memory engine supercharged with an **
|
|
|
28
29
|
| User asks to index a documentation URL, file, or repository | `ingest_document` | `content` or `source_path`, `title`, `metadata` |
|
|
29
30
|
| User asks a complex question about indexed docs or code | `query_knowledge_base` | `query`, `limit`, `generateEmbeddings` |
|
|
30
31
|
| Read full raw content of an ambiguous/abstract document | `manage_knowledge_base` | `action: "read_document"`, `docId` |
|
|
31
|
-
| User asks to view database stats, list indexed docs, or export snapshots | `manage_knowledge_base` | `action` ("stats", "list", "read_document", "delete", "export_snapshot") |
|
|
32
|
+
| User asks to view database stats, list indexed docs, or export snapshots | `manage_knowledge_base` | `action` ("stats", "list", "read_document", "delete", "export_snapshot", "import_snapshot") |
|
|
33
|
+
| Discover available MCP servers and their specific purposes | `list-mcp-tools` | — |
|
|
34
|
+
| Ask which MCP tool / server is suitable for a specific task | `mcp-reminder` | `task` (string, e.g., "db migration") |
|
|
32
35
|
|
|
33
36
|
---
|
|
34
37
|
|
|
@@ -113,7 +116,7 @@ Use this tool when adding technical documentation, API specs, architectural docu
|
|
|
113
116
|
|
|
114
117
|
### Hybrid Retrieval (`query_knowledge_base`)
|
|
115
118
|
Use this tool BEFORE answering deep architectural or technical questions when indexed documents exist.
|
|
116
|
-
- Performs **Hybrid RRF Fusion** combining SQLite FTS5 BM25 keyword matching with dense ONNX vector semantic search.
|
|
119
|
+
- Performs **Hybrid RRF/RSF Fusion** combining SQLite FTS5 BM25 keyword matching with dense ONNX vector semantic search.
|
|
117
120
|
- Returns candidate sections with breadcrumb paths and defined code symbols (classes, functions, types).
|
|
118
121
|
|
|
119
122
|
#### Query Formulation Rules (CRITICAL for retrieval quality)
|
|
@@ -154,13 +157,28 @@ In such cases, use the **Full Raw Document Reading** mechanism:
|
|
|
154
157
|
- Use `action: "list"` to see all ingested documents.
|
|
155
158
|
- Use `action: "read_document"` with `docId` to read the complete raw text content of any document.
|
|
156
159
|
- Use `action: "delete"` with `docId` to remove an outdated document and purge its CAS blob.
|
|
160
|
+
- Use `action: "export_snapshot"` with `snapshotPath` to export a JSON backup of the RAG base.
|
|
161
|
+
- Use `action: "import_snapshot"` with `snapshotPath` to import and merge a JSON backup into the current database.
|
|
157
162
|
|
|
158
163
|
---
|
|
159
164
|
|
|
160
|
-
## 4.
|
|
165
|
+
## 4. General MCP Helpers (`list-mcp-tools`, `mcp-reminder`)
|
|
166
|
+
|
|
167
|
+
### Discovering Connected MCP Servers (`list-mcp-tools`)
|
|
168
|
+
When working in multi-server environments (e.g., OpenCode, Claude Code), you might have several auxiliary servers installed (for database, UI design, browser automation, etc.).
|
|
169
|
+
- Use `list-mcp-tools` to immediately view all registered servers and their descriptions. This avoids guessing what other capabilities are available in the current workspace.
|
|
170
|
+
|
|
171
|
+
### Contextual Tool Reminders (`mcp-reminder`)
|
|
172
|
+
- If you are unsure which tool/server is best suited for the task at hand (e.g., how to do browser testing, or run a database migration), run `mcp-reminder(task: "your current task definition")`.
|
|
173
|
+
- It analyzes your task and suggests appropriate servers (like `playwright` for testing, `supabase` for DB, or `stitch` for UI design).
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
## 5. Core Directives for AI Agents
|
|
161
178
|
|
|
162
179
|
1. **Read Memories First (MANDATORY)**: At the very start of any session or conversation, your VERY FIRST STEP MUST BE to execute `recall` to load all saved facts, user context, and project guidelines BEFORE performing any other task or code analysis.
|
|
163
180
|
2. **Be Proactive**: When the user mentions a durable preference, personal fact, or constraint, save it immediately using `remember`. Do not wait for explicit user commands.
|
|
164
|
-
3. **Check Knowledge Base First**: If a
|
|
181
|
+
3. **Check Knowledge Base First**: If a query is related to specialized documentation, APIs, or project architectures, call `query_knowledge_base` using concept-dense search phrases.
|
|
165
182
|
4. **Inspect Ambiguous Docs Directly**: If querying produces low relevance scores on abstractly-named documents, call `manage_knowledge_base(action: "read_document")` to inspect the full text directly.
|
|
166
183
|
5. **Keep Memory Clean**: If a preference changes, call `update_fact` to edit it in place, or `remember` with `supersedes` to keep a version trail. Use `keep: true` for facts that must survive an accidental `forget`, and give ephemeral facts a `ttl` so stale ones surface as `[EXPIRED]`.
|
|
184
|
+
6. **Leverage MCP Servers**: Proactively list available tools using `list-mcp-tools` and query `mcp-reminder` if unsure of which platform tool can help you automate tasks.
|
|
@@ -1,228 +0,0 @@
|
|
|
1
|
-
import { createServer } from "node:http";
|
|
2
|
-
import { readFileSync, existsSync, statSync } from "node:fs";
|
|
3
|
-
import { join, dirname } from "node:path";
|
|
4
|
-
import { fileURLToPath } from "node:url";
|
|
5
|
-
import { getDatabase, DB_PATH, BLOBS_DIR } from "../db/database.js";
|
|
6
|
-
import { ingestDocument, deleteDocument } from "../ingest/pipeline.js";
|
|
7
|
-
import { hybridQuery } from "../retrieval/retriever.js";
|
|
8
|
-
import { exportSnapshot, importSnapshot } from "./snapshot.js";
|
|
9
|
-
import { readBlob } from "../storage/blob_store.js";
|
|
10
|
-
|
|
11
|
-
const __filename = fileURLToPath(import.meta.url);
|
|
12
|
-
const __dirname = dirname(__filename);
|
|
13
|
-
|
|
14
|
-
export function findAvailablePort(startPort = 8765, maxPort = 8785) {
|
|
15
|
-
return new Promise((resolve, reject) => {
|
|
16
|
-
let port = startPort;
|
|
17
|
-
const tryPort = () => {
|
|
18
|
-
if (port > maxPort) {
|
|
19
|
-
return reject(new Error(`No free port found between ${startPort} and ${maxPort}`));
|
|
20
|
-
}
|
|
21
|
-
const server = createServer();
|
|
22
|
-
server.listen(port, () => {
|
|
23
|
-
server.close(() => resolve(port));
|
|
24
|
-
});
|
|
25
|
-
server.on("error", () => {
|
|
26
|
-
port++;
|
|
27
|
-
tryPort();
|
|
28
|
-
});
|
|
29
|
-
};
|
|
30
|
-
tryPort();
|
|
31
|
-
});
|
|
32
|
-
}
|
|
33
|
-
|
|
34
|
-
function parseJsonBody(req) {
|
|
35
|
-
return new Promise((resolve, reject) => {
|
|
36
|
-
let body = "";
|
|
37
|
-
req.on("data", (chunk) => {
|
|
38
|
-
body += chunk.toString();
|
|
39
|
-
});
|
|
40
|
-
req.on("end", () => {
|
|
41
|
-
try {
|
|
42
|
-
resolve(body ? JSON.parse(body) : {});
|
|
43
|
-
} catch (err) {
|
|
44
|
-
reject(err);
|
|
45
|
-
}
|
|
46
|
-
});
|
|
47
|
-
req.on("error", reject);
|
|
48
|
-
});
|
|
49
|
-
}
|
|
50
|
-
|
|
51
|
-
export async function startAdminServer({ port = null, customDb = null, customBlobDir = BLOBS_DIR } = {}) {
|
|
52
|
-
const db = customDb || getDatabase();
|
|
53
|
-
const selectedPort = port || (await findAvailablePort());
|
|
54
|
-
const htmlPath = join(__dirname, "index.html");
|
|
55
|
-
|
|
56
|
-
const server = createServer(async (req, res) => {
|
|
57
|
-
const url = new URL(req.url, `http://${req.headers.host || "localhost"}`);
|
|
58
|
-
const pathname = url.pathname;
|
|
59
|
-
|
|
60
|
-
// Helper for CORS and JSON response
|
|
61
|
-
const sendJson = (data, status = 200) => {
|
|
62
|
-
res.writeHead(status, {
|
|
63
|
-
"Content-Type": "application/json",
|
|
64
|
-
"Access-Control-Allow-Origin": "*",
|
|
65
|
-
"Access-Control-Allow-Methods": "GET, POST, DELETE, OPTIONS",
|
|
66
|
-
"Access-Control-Allow-Headers": "Content-Type",
|
|
67
|
-
});
|
|
68
|
-
res.end(JSON.stringify(data));
|
|
69
|
-
};
|
|
70
|
-
|
|
71
|
-
if (req.method === "OPTIONS") {
|
|
72
|
-
res.writeHead(204, {
|
|
73
|
-
"Access-Control-Allow-Origin": "*",
|
|
74
|
-
"Access-Control-Allow-Methods": "GET, POST, DELETE, OPTIONS",
|
|
75
|
-
"Access-Control-Allow-Headers": "Content-Type",
|
|
76
|
-
});
|
|
77
|
-
return res.end();
|
|
78
|
-
}
|
|
79
|
-
|
|
80
|
-
try {
|
|
81
|
-
// 1. Static HTML SPA
|
|
82
|
-
if (pathname === "/" || pathname === "/index.html") {
|
|
83
|
-
if (!existsSync(htmlPath)) {
|
|
84
|
-
res.writeHead(404, { "Content-Type": "text/plain" });
|
|
85
|
-
return res.end("index.html not found");
|
|
86
|
-
}
|
|
87
|
-
const html = readFileSync(htmlPath, "utf-8");
|
|
88
|
-
res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" });
|
|
89
|
-
return res.end(html);
|
|
90
|
-
}
|
|
91
|
-
|
|
92
|
-
// 2. API: Stats
|
|
93
|
-
if (pathname === "/api/stats" && req.method === "GET") {
|
|
94
|
-
const docCount = db.prepare("SELECT COUNT(*) as cnt FROM documents").get().cnt;
|
|
95
|
-
const secCount = db.prepare("SELECT COUNT(*) as cnt FROM sections").get().cnt;
|
|
96
|
-
const chunkCount = db.prepare("SELECT COUNT(*) as cnt FROM micro_chunks").get().cnt;
|
|
97
|
-
const edgeCount = db.prepare("SELECT COUNT(*) as cnt FROM graph_edges").get().cnt;
|
|
98
|
-
let dbSize = 0;
|
|
99
|
-
if (existsSync(DB_PATH)) {
|
|
100
|
-
try {
|
|
101
|
-
dbSize = statSync(DB_PATH).size;
|
|
102
|
-
} catch {}
|
|
103
|
-
}
|
|
104
|
-
return sendJson({
|
|
105
|
-
documents: docCount,
|
|
106
|
-
sections: secCount,
|
|
107
|
-
micro_chunks: chunkCount,
|
|
108
|
-
graph_edges: edgeCount,
|
|
109
|
-
db_size_bytes: dbSize,
|
|
110
|
-
});
|
|
111
|
-
}
|
|
112
|
-
|
|
113
|
-
// 3. API: Documents List
|
|
114
|
-
if (pathname === "/api/documents" && req.method === "GET") {
|
|
115
|
-
const docs = db.prepare("SELECT * FROM documents ORDER BY updated_at DESC").all();
|
|
116
|
-
return sendJson(docs);
|
|
117
|
-
}
|
|
118
|
-
|
|
119
|
-
// 4. API: Document Detail
|
|
120
|
-
if (pathname.startsWith("/api/documents/") && req.method === "GET") {
|
|
121
|
-
const docId = pathname.replace("/api/documents/", "");
|
|
122
|
-
const doc = db.prepare("SELECT * FROM documents WHERE id = ?").get(docId);
|
|
123
|
-
if (!doc) return sendJson({ error: "Document not found" }, 404);
|
|
124
|
-
|
|
125
|
-
const sections = db.prepare("SELECT * FROM sections WHERE doc_id = ?").all(docId);
|
|
126
|
-
const microChunks = db.prepare("SELECT id, section_id, token_count FROM micro_chunks WHERE doc_id = ?").all(docId);
|
|
127
|
-
const edges = db.prepare("SELECT * FROM graph_edges WHERE source_id = ? OR target_id = ?").all(docId, docId);
|
|
128
|
-
|
|
129
|
-
let blobContent = null;
|
|
130
|
-
if (doc.blob_hash) {
|
|
131
|
-
try {
|
|
132
|
-
blobContent = await readBlob(doc.blob_hash, customBlobDir);
|
|
133
|
-
} catch {}
|
|
134
|
-
}
|
|
135
|
-
|
|
136
|
-
return sendJson({ doc, sections, microChunks, edges, blobContent });
|
|
137
|
-
}
|
|
138
|
-
|
|
139
|
-
// 5. API: Delete Document
|
|
140
|
-
if (pathname.startsWith("/api/documents/") && req.method === "DELETE") {
|
|
141
|
-
const docId = pathname.replace("/api/documents/", "");
|
|
142
|
-
const result = await deleteDocument(docId, db, customBlobDir);
|
|
143
|
-
return sendJson(result);
|
|
144
|
-
}
|
|
145
|
-
|
|
146
|
-
// 6. API: Ingest Document
|
|
147
|
-
if (pathname === "/api/ingest" && req.method === "POST") {
|
|
148
|
-
const body = await parseJsonBody(req);
|
|
149
|
-
const result = await ingestDocument({
|
|
150
|
-
content: body.content,
|
|
151
|
-
type: body.type || "text",
|
|
152
|
-
path: body.path || null,
|
|
153
|
-
title: body.title || null,
|
|
154
|
-
generateEmbeddings: body.generateEmbeddings !== false,
|
|
155
|
-
customDb: db,
|
|
156
|
-
customBlobDir,
|
|
157
|
-
});
|
|
158
|
-
return sendJson(result, 201);
|
|
159
|
-
}
|
|
160
|
-
|
|
161
|
-
// 7. API: Query Knowledge Base
|
|
162
|
-
if (pathname === "/api/query" && req.method === "POST") {
|
|
163
|
-
const body = await parseJsonBody(req);
|
|
164
|
-
const results = await hybridQuery({
|
|
165
|
-
query: body.query,
|
|
166
|
-
limit: body.limit || 5,
|
|
167
|
-
generateEmbeddings: body.generateEmbeddings !== false,
|
|
168
|
-
customDb: db,
|
|
169
|
-
});
|
|
170
|
-
return sendJson({ results });
|
|
171
|
-
}
|
|
172
|
-
|
|
173
|
-
// 8. API: Graph Visualizer Data
|
|
174
|
-
if (pathname === "/api/graph" && req.method === "GET") {
|
|
175
|
-
const docs = db.prepare("SELECT id, title, path FROM documents").all();
|
|
176
|
-
const edges = db.prepare("SELECT * FROM graph_edges").all();
|
|
177
|
-
|
|
178
|
-
const nodes = docs.map((d) => ({
|
|
179
|
-
id: d.id,
|
|
180
|
-
label: d.title || d.path || d.id,
|
|
181
|
-
type: "DOCUMENT",
|
|
182
|
-
}));
|
|
183
|
-
|
|
184
|
-
// Add code symbol nodes
|
|
185
|
-
const symbolEdges = edges.filter((e) => e.relation_type === "DEFINES_SYMBOL");
|
|
186
|
-
for (const se of symbolEdges) {
|
|
187
|
-
if (!nodes.some((n) => n.id === se.target_id)) {
|
|
188
|
-
nodes.push({
|
|
189
|
-
id: se.target_id,
|
|
190
|
-
label: se.target_id,
|
|
191
|
-
type: "CODE_SYMBOL",
|
|
192
|
-
});
|
|
193
|
-
}
|
|
194
|
-
}
|
|
195
|
-
|
|
196
|
-
return sendJson({ nodes, edges });
|
|
197
|
-
}
|
|
198
|
-
|
|
199
|
-
// 9. API: Export Snapshot
|
|
200
|
-
if (pathname === "/api/snapshot/export" && (req.method === "GET" || req.method === "POST")) {
|
|
201
|
-
const snapshot = await exportSnapshot({ customDb: db, customBlobDir });
|
|
202
|
-
return sendJson(snapshot);
|
|
203
|
-
}
|
|
204
|
-
|
|
205
|
-
// 10. API: Import Snapshot
|
|
206
|
-
if (pathname === "/api/snapshot/import" && req.method === "POST") {
|
|
207
|
-
const body = await parseJsonBody(req);
|
|
208
|
-
const result = await importSnapshot({ customDb: db, customBlobDir, snapshotPathOrData: body });
|
|
209
|
-
return sendJson(result);
|
|
210
|
-
}
|
|
211
|
-
|
|
212
|
-
// 404 Fallback
|
|
213
|
-
res.writeHead(404, { "Content-Type": "text/plain" });
|
|
214
|
-
res.end("Not Found");
|
|
215
|
-
} catch (err) {
|
|
216
|
-
console.error("Admin server error:", err);
|
|
217
|
-
sendJson({ error: err.message }, 500);
|
|
218
|
-
}
|
|
219
|
-
});
|
|
220
|
-
|
|
221
|
-
return new Promise((resolve) => {
|
|
222
|
-
server.listen(selectedPort, () => {
|
|
223
|
-
const url = `http://localhost:${selectedPort}`;
|
|
224
|
-
console.log(`🚀 memory-agent Web Admin Dashboard running at ${url}`);
|
|
225
|
-
resolve({ server, port: selectedPort, url });
|
|
226
|
-
});
|
|
227
|
-
});
|
|
228
|
-
}
|