@lotargo/memory_plugin 1.6.5 → 1.6.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/CHANGELOG.md +34 -0
  2. package/README.md +576 -443
  3. package/mcp-server/benchmarks/fetch_real_corpus.js +351 -0
  4. package/mcp-server/benchmarks/gpu_profile_benchmark.js +170 -0
  5. package/mcp-server/benchmarks/policy_dominance_test.js +221 -0
  6. package/mcp-server/benchmarks/quality_evaluator.js +598 -0
  7. package/mcp-server/benchmarks/raw_corpus_data.js +613 -0
  8. package/mcp-server/benchmarks/run_benchmarks.js +366 -0
  9. package/mcp-server/benchmarks/stress_ingestion.js +195 -0
  10. package/mcp-server/benchmarks/table_code_retrieval.js +453 -0
  11. package/mcp-server/benchmarks/test_dual_layer.js +141 -0
  12. package/mcp-server/cli/direct_commands.js +39 -0
  13. package/mcp-server/cli.js +16 -5
  14. package/mcp-server/cli_boot.js +4 -1
  15. package/mcp-server/client_cli.js +73 -0
  16. package/mcp-server/client_paths.js +44 -0
  17. package/mcp-server/client_registration.js +38 -0
  18. package/mcp-server/codex_config.js +86 -8
  19. package/mcp-server/db/database.js +14 -21
  20. package/mcp-server/db/migrations.js +66 -77
  21. package/mcp-server/db/rag_blob_transport.js +143 -0
  22. package/mcp-server/db/rag_sync.js +284 -0
  23. package/mcp-server/db/sync_queue.js +219 -307
  24. package/mcp-server/dev_link.js +142 -0
  25. package/mcp-server/fact_format.js +44 -12
  26. package/mcp-server/index.js +17 -7
  27. package/mcp-server/ingest/exporter.js +44 -38
  28. package/mcp-server/ingest/pipeline.js +260 -248
  29. package/mcp-server/persona_migration.js +39 -0
  30. package/mcp-server/prompt_manager.js +162 -55
  31. package/mcp-server/rag_scope.js +83 -0
  32. package/mcp-server/retrieval/retriever.js +99 -64
  33. package/mcp-server/setup.js +150 -100
  34. package/mcp-server/storage/blob_store.js +53 -1
  35. package/mcp-server/tools/core/knowledge_read_core.js +163 -0
  36. package/mcp-server/tools/core/memory_core.js +24 -4
  37. package/mcp-server/tools/core/memory_routing.js +10 -0
  38. package/mcp-server/tools/core/note_core.js +53 -0
  39. package/mcp-server/tools/core/rag_query_core.js +169 -0
  40. package/mcp-server/tools/index.js +11 -9
  41. package/mcp-server/tools/memory_tools.js +4 -1
  42. package/mcp-server/tools/note_tools.js +35 -0
  43. package/mcp-server/tools/rag_tools.js +211 -364
  44. package/mcp-server/uninstall.js +627 -0
  45. package/opencode-plugin/index.js +80 -12
  46. package/opencode-plugin/main.js +136 -0
  47. package/package.json +17 -34
  48. package/skills/using-memory/SKILL.md +28 -19
package/README.md CHANGED
@@ -1,453 +1,586 @@
1
- <div align="center">
2
-
3
- <img src="./assets/hero.jpg" alt="@lotargo/memory_plugin" width="800" style="max-width: 100%; border-radius: 12px; margin-bottom: 16px;">
4
-
5
- <br>
6
-
7
- <img src="./assets/title.svg" alt="@lotargo/memory_plugin" width="520" style="max-width: 100%; margin-bottom: 12px;">
8
-
9
- <br>
10
-
11
- [![npm version](https://img.shields.io/npm/v/@lotargo/memory_plugin)](https://www.npmjs.com/package/@lotargo/memory_plugin)
12
- [![npm downloads](https://img.shields.io/npm/dt/@lotargo/memory_plugin)](https://www.npmjs.com/package/@lotargo/memory_plugin)
13
- [![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
14
- [![node version](https://img.shields.io/badge/node-%3E%3D22.5.0-brightgreen)](https://nodejs.org)
15
- [![mcp](https://img.shields.io/badge/MCP-Supported-8A2BE2)](https://modelcontextprotocol.io)
16
- [![storage](https://img.shields.io/badge/Storage-Local%20%2B%20Cloud%20Sync-success)](#storage--privacy)
17
-
18
- <br>
19
-
20
- **Zero-Docker Local Hybrid RAG Engine & Long-Term Memory for AI Coding Agents**
21
-
22
- Automatically remembers durable user facts, ingests complex document repositories, and performs high-precision hybrid retrieval across sessions, platforms, and devices.
23
-
24
- </div>
25
-
26
- ---
27
-
28
- ## Overview
29
-
30
- Standard AI coding assistants lose context as soon as a chat session closes or a conversation is reset. You end up repeatedly re-explaining your preferences, architectural decisions, coding style, or project conventions.
31
-
32
- `@lotargo/memory_plugin` gives your AI tools durable, **persistent**, local-first long-term memory and document retrieval capabilities that persist across restarts and work seamlessly across all supported coding environments. Any LLM-based coding agent (OpenCode, Claude Code, Codex, Antigravity / Gemini CLI) can query its own memory and hybrid knowledge base via the **Model Context Protocol (MCP)**.
33
-
34
- > **Project Scope & Runtime Notes**:
35
- > `@lotargo/memory_plugin` is designed primarily as a practical plugin to expand capabilities and streamline daily interaction with AI coding tools. Benchmark scores in this repository represent internal synthetic evaluation runs and are not intended as generalized RAG benchmarks.
36
- >
37
- > **Hardware Acceleration**: GPU execution mode is an experimental feature and may vary in stability across different operating systems or models. For optimal stability and consistent runtime performance, using standard CPU mode with `multilingual-e5-small` or `multilingual-e5-base` is recommended.
38
-
39
- ### Practical Use Cases
40
-
41
- - **Architectural Decisions**: _"In this project, we use Fastify instead of Express and strict schema validation via Zod."_
42
- - **Coding Conventions**: _"Place all helper utilities inside `src/utils/` and cover new functions with Vitest tests."_
43
- - **Environment Constraints**: _"Our target deployment environment is Node.js 20 on AWS Lambda."_
44
- - **User Profile & Tone**: _"My name is Alex. I prefer concise, direct answers without conversational filler."_
45
-
46
- ---
47
-
48
- ## Quick Start
49
-
50
- ### Minimum System Requirements
51
-
52
- - **Node.js**: `22.5.0` or higher (required by the built-in `node:sqlite` module)
53
- - **Package Manager**: `npm` / `npx` (included with Node.js)
54
- - **Supported Environment**: OpenCode, Antigravity / Gemini CLI, Claude Code, Codex, or Google Jules
55
-
56
- ### Installation & Auto-Setup
57
-
58
- Run the setup command to configure all detected AI environments automatically:
59
-
60
- ```bash
61
- # Recommended: Global installation & auto-setup across all environments
62
- npm install -g @lotargo/memory_plugin && memory_plugin setup
63
-
64
- # Or via npx
65
- npx @lotargo/memory_plugin setup
66
- ```
67
-
68
- To target a specific environment:
69
-
70
- ```bash
71
- # Antigravity / Gemini CLI
72
- memory_plugin setup --antigravity
73
-
74
- # OpenCode
75
- memory_plugin setup --opencode
76
-
77
- # Claude Code
78
- memory_plugin setup --claude
79
-
80
- # Codex
1
+ <div align="center">
2
+
3
+ <img src="https://raw.githubusercontent.com/Lotargo/memory_pugin/main/assets/hero.jpg" alt="@lotargo/memory_plugin" width="800" style="max-width: 100%; border-radius: 12px; margin-bottom: 16px;">
4
+
5
+ <br>
6
+
7
+ <img src="https://raw.githubusercontent.com/Lotargo/memory_pugin/main/assets/title.svg" alt="@lotargo/memory_plugin" width="520" style="max-width: 100%; margin-bottom: 12px;">
8
+
9
+ <br>
10
+
11
+ [![npm version](https://img.shields.io/npm/v/@lotargo/memory_plugin)](https://www.npmjs.com/package/@lotargo/memory_plugin)
12
+ [![npm downloads](https://img.shields.io/npm/dt/@lotargo/memory_plugin)](https://www.npmjs.com/package/@lotargo/memory_plugin)
13
+ [![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
14
+ [![node version](https://img.shields.io/badge/node-%3E%3D22.5.0-brightgreen)](https://nodejs.org)
15
+ [![mcp](https://img.shields.io/badge/MCP-Supported-8A2BE2)](https://modelcontextprotocol.io)
16
+ [![storage](https://img.shields.io/badge/Storage-Local%20%2B%20Cloud%20Sync-success)](#storage-privacy-and-security)
17
+
18
+ <br>
19
+
20
+ **Local-first long-term memory, cold episodic notes, and hybrid RAG for AI coding agents**
21
+
22
+ One memory system for OpenCode, Codex, Claude Code, Gemini CLI, Antigravity, Google Jules, and other MCP clients.
23
+
24
+ </div>
25
+
26
+ ---
27
+
28
+ ## Why This Project Exists
29
+
30
+ AI coding assistants forget user preferences, architectural decisions, investigations, and project context when a session ends. They also tend to mix very different kinds of information into one oversized prompt.
31
+
32
+ `@lotargo/memory_plugin` separates persistent knowledge into the right storage class:
33
+
34
+ | What you want to preserve | Tool | Storage behavior |
35
+ | :--- | :--- | :--- |
36
+ | Concise facts, preferences, constraints, conventions, and persona settings | `remember` | **Hot Notebook memory**; available during session initialization |
37
+ | Detailed decisions, research, investigations, experiments, and handoffs | `remember_note` | **Cold RAG Memory Note**; searchable but not injected into every session |
38
+ | Files, URLs, documentation, reports, specifications, and code | `ingest_document` | **Curated external knowledge** in the RAG index |
39
+
40
+ The same engine adds Git-based project isolation, semantic search, full raw-source expansion, explicit fact-to-document links, optional Turso synchronization, and native OpenCode auto-injection.
41
+
42
+ ### Highlights
43
+
44
+ - Human-readable Markdown Notebook facts with stable IDs, TTL, protection, tags, superseding, and explicit `fact` / `directive` semantics.
45
+ - Agent-authored long-form RAG Memory Notes for cold or episodic context.
46
+ - Hybrid SQLite FTS5 BM25 + local ONNX vector retrieval with RSF/RRF fusion.
47
+ - Compact semantic TOC discovery through `resultMode: "index"`, followed by deliberate full-source expansion.
48
+ - PDF, DOCX, XLSX/XLS/CSV, Markdown, text, HTML/URL, and source-code ingestion.
49
+ - Three-tier document hierarchy, retrieval-policy expansion for tables/code, and GraphRAG Lite symbol extraction.
50
+ - Git-identity project scopes that follow a repository across directories, machines, and operating systems.
51
+ - Active persona overlays shared across OpenCode, Codex, Claude Code, Gemini CLI, and Antigravity.
52
+ - Local-only, cloud-only, and bidirectional hybrid-sync modes, including portable raw RAG blobs and deletion tombstones.
53
+ - No Docker, external vector database, hosted embedding API, or telemetry.
54
+
55
+ > This is a practical agent-memory system, not a claim of generalized benchmark superiority. Repository benchmark results describe the included evaluation corpus and configuration.
56
+
57
+ ---
58
+
59
+ ## Quick Start
60
+
61
+ ### Requirements
62
+
63
+ - Node.js `22.5.0` or newer; the project uses the built-in `node:sqlite` module.
64
+ - npm/npx.
65
+ - OpenCode, Codex, Claude Code, Gemini CLI, Antigravity, Google Jules, or another MCP-capable client.
66
+
67
+ CPU execution with `Xenova/multilingual-e5-small` is the recommended stable default. WebGPU execution is experimental.
68
+
69
+ ### Install and Configure
70
+
71
+ Configure every supported client location:
72
+
73
+ ```bash
74
+ npm install -g @lotargo/memory_plugin
75
+ memory_plugin setup
76
+ ```
77
+
78
+ Or run setup without a permanent global installation:
79
+
80
+ ```bash
81
+ npx @lotargo/memory_plugin setup
82
+ ```
83
+
84
+ Target one client when needed:
85
+
86
+ ```bash
87
+ memory_plugin setup --opencode
81
88
  memory_plugin setup --codex
89
+ memory_plugin setup --claude
90
+ memory_plugin setup --antigravity
91
+ memory_plugin setup --gemini # Gemini CLI (~/.gemini/settings.json)
92
+ ```
93
+
94
+ Use `--local` with Antigravity setup to create the workspace-local `.agents/mcp_config.json` even when `.agents/` does not yet exist.
95
+
96
+ Claude Code, Gemini CLI, and Codex setup/uninstall use their native MCP lifecycle commands when available. An ownership-checked config edit is retained as a compatibility fallback for missing, older, or non-functional client CLIs. Antigravity remains a separate integration because it uses a different config layout.
97
+
98
+ Setup also installs the bundled `using-memory` skill and managed memory instructions for the selected clients. Existing unrelated configuration is preserved.
99
+
100
+ ### Uninstall
101
+
102
+ Remove the plugin from one or all clients without deleting Notebook/RAG data:
103
+
104
+ ```bash
105
+ memory_plugin uninstall --dry-run # preview
106
+ memory_plugin uninstall # remove all clients, keep data
107
+ memory_plugin uninstall --purge --yes # also delete local data (MEMORY_DIR, prompt state)
108
+ memory_plugin uninstall --opencode --purge-cache # explicitly remove this plugin's OpenCode cache
109
+ memory_plugin uninstall --opencode --claude # only selected clients
110
+ npx @lotargo/memory_plugin uninstall --dry-run
111
+ memory_plugin setup --uninstall --purge # alias
82
112
  ```
83
113
 
84
- Codex is configured with a direct executable chain (`node` `mcp-server/boot.js`),
85
- not `npx`. This avoids Windows stdio handshake failures caused by `.cmd` launchers.
86
- Running setup again safely migrates legacy `npx` entries and preserves unrelated
87
- sections in `~/.codex/config.toml`.
114
+ What `uninstall` removes by default (without `--purge`):
88
115
 
89
- `memory_plugin setup --codex` updates only the Codex MCP registration, Codex prompt,
90
- and the Codex-compatible skill locations (`~/.codex/skills` and the shared
91
- `~/.agents/skills`). It does not modify Claude Code, OpenCode, or Antigravity files.
116
+ - `~/.config/opencode/opencode.json` plugin entry (including `file://` dev link)
117
+ - `~/.claude.json` `mcpServers.memory-agent`
118
+ - `~/.gemini/settings.json` Gemini CLI `mcpServers.memory-agent`
119
+ - `~/.gemini/config/mcp_config.json` and `.agents/mcp_config.json` — Antigravity `mcpServers.memory-agent`
120
+ - `~/.codex/config.toml` — `[mcp_servers.memory-agent]` (only if owned by this plugin)
121
+ - Managed prompt blocks from `~/.codex/AGENTS.md`, `~/.claude/CLAUDE.md`, `~/.gemini/GEMINI.md`, and `~/.gemini/config/AGENTS.md`
122
+ - `using-memory` skill from each client's `skills/` directory
92
123
 
93
- To verify registration, the Node runtime, MCP initialization, tool discovery, and
94
- real `memory_info` / `recall(scope="all")` calls:
124
+ Existing user content outside the managed prompt/persona markers is preserved. Foreign `memory-agent` registrations, modified/non-owned `using-memory` skills, unrelated file plugins, and other packages in the `@lotargo` OpenCode cache namespace are left untouched.
125
+
126
+ Normal uninstall keeps OpenCode's package cache, matching the host lifecycle. `--purge-cache` removes only exact cache directories owned by this package; unrelated packages, including other packages in the `@lotargo` namespace, remain untouched.
127
+
128
+ With `--purge` it also deletes `MEMORY_DIR` (`~/.config/opencode/memory` by default) and the memory-agent prompt state. Purge resolves and validates every target before changing client configuration, rejects filesystem roots, home/workspace/config roots and broad top-level paths, follows symlinks for validation, and prints the exact targets before interactive confirmation. The npm package itself is removed separately with `npm uninstall -g @lotargo/memory_plugin`. Restart clients after uninstall.
129
+
130
+ On Linux/macOS, `XDG_CONFIG_HOME` and `XDG_CACHE_HOME` are respected for OpenCode configuration, prompt state, and package-cache cleanup. `OPENCODE_CONFIG_DIR` and `MEMORY_DIR` remain explicit overrides on every platform.
131
+
132
+ ### Verify Codex
133
+
134
+ Codex uses a direct executable chain (`node` -> `mcp-server/boot.js`) instead of an `npx`/`.cmd` launcher, avoiding Windows stdio handshake failures. Setup safely migrates legacy registrations in `~/.codex/config.toml`.
95
135
 
96
136
  ```bash
97
137
  memory_plugin doctor --codex
98
138
  ```
99
139
 
100
- The doctor checks the server process itself. If it passes but a fresh Codex Desktop
101
- task still does not expose the tools, the remaining fault is in Desktop tool
102
- exposure rather than the memory MCP launcher.
103
-
104
- `setup` also accepts `--gemini` (alias for Antigravity) and `--local` (registers the MCP server in the project-local `.agents/` directory). Without a specific flag, all detected environments are configured.
105
-
106
- ### Headless & Cloud Setup (CI / Docker / Cloud Workspaces)
107
-
108
- For headless environments (e.g., Google Jules, VPS, CI/CD pipelines), configure auth and sync mode in a single non-interactive command:
109
-
110
- ```bash
111
- # Authenticate via Turso API token and set hybrid sync mode
112
- memory_plugin setup --api-key <TURSO_API_TOKEN> --mode hybrid-sync
113
-
114
- # Or set sync mode if already authorized
115
- memory_plugin setup --mode only-cloud
116
- ```
117
-
118
- ---
119
-
120
- ## Multi-Layer Architecture
121
-
122
- 1. **Layer 1: Notebook Store (Durable Facts)**
123
- - **Tools**: `remember`, `recall`, `get_fact`, `update_fact`, `forget`, `memory_info`
124
- - **Scope**: User preferences, identity, project conventions, system rules.
125
- - **Storage**: Human-readable Markdown format (`global` and per-project stores).
126
- - **Fact Schema**: Every fact is formatted as `**Title** — body` with inline metadata badges (`[id]`, `[ttl]`, `[keep]`, `[tags]`, `[supersedes]`, `[inject]`).
127
- - **Project Identity**: Project stores are bound to a **Git-based project identity** — the normalized remote URL (`git:github.com/owner/repo`) or `git:local:<repo basename>` — never to a directory path. Memories follow the repository across machines, OSes, and subdirectories. Legacy path/basename stores can be linked and merged via `link_project_memory`.
128
-
129
- 2. **Layer 2: RAG Knowledge Base (Technical Documents & Codebases)**
130
- - **Tools**: `ingest_document`, `query_knowledge_base`, `batch_query_knowledge_base`, `manage_knowledge_base`, `reindex_knowledge_base`
131
- - **Capabilities**: Ingests raw text files, Markdown, HTML, Web URLs, office documents (PDF, DOCX, XLSX, CSV), and codebases.
132
- - **Curation Model**: Stores only project-relevant sources likely to be reused—for example important web findings and current library/framework documentation that may be newer than model training. Ingest the useful document or excerpt, then link it to the project memory it supports; do not archive everything the agent encounters.
133
- - **Scope Isolation**: New documents default to the current linked Git project. `scope: "all"` searches global sources plus the current project's sources; outside a Git repository it searches global sources only. Use `scope: "global"` only for genuinely cross-project material.
134
- - **Stable Shared Sources**: Ingesting the same path or URL in another scope reuses one document and adds a scope association. Re-ingesting updated content preserves the document ID and its fact links; removing one scope does not delete a document still used by another scope.
135
- - **Engine Components**: 3-tier hierarchy chunking (Big / Medium / Small), SQLite FTS5 BM25 search, ONNX dense vector embeddings (`multilingual-e5-small`), Reciprocal Rank Fusion (RRF / RSF), cross-encoder reranking (optional), and GraphRAG Lite code symbol extraction.
136
-
137
- 3. **Layer 3: Agent-Driven Knowledge Graph**
138
- - **Tools**: `link_knowledge` (plus `docId`, `startLine`, `endLine` in `remember`)
139
- - **Capabilities**: Connects Layer 1 notebook facts directly to Layer 2 documents, sections, or exact line ranges with semantic edge relations (`RULES_FOR`, `IMPLEMENTS`, `EXPLAINS`, `REFERENCES`).
140
- - **Surfacing**: Linked facts automatically highlight target documents and line ranges in `recall` output (`🔗 [Linked Docs: ...]`).
141
-
142
- ---
143
-
144
- ## Cloud Synchronization & Database Modes (Turso / LibSQL)
145
-
146
- The plugin provides local-first SQLite persistence with optional cloud database synchronization powered by Turso (LibSQL):
147
-
148
- ### 3 Storage Sync Modes
149
-
150
- | Mode | Command / Flag | Description |
151
- | :--- | :------------- | :---------- |
152
- | **`only-local`** (default) | `--mode only-local` | 100% local SQLite database. No external network traffic for data storage. |
153
- | **`only-cloud`** | `--mode only-cloud` | Direct LibSQL connection to a remote Turso database instance. |
154
- | **`hybrid-sync`** | `--mode hybrid-sync` | Local SQLite performance with background asynchronous synchronization to Turso Cloud, featuring automatic reverse-sync and conflict resolution. |
155
-
156
- ### Cloud Failover & Circuit Breaker
157
-
158
- When operating in cloud modes (`only-cloud` or `hybrid-sync`), the database engine incorporates a built-in **Circuit Breaker**:
159
- - If the primary cloud database endpoint is unreachable or encounters network failure, queries fail over to the secondary cloud endpoint configured in `failoverUrl`. Failover is disabled when `failoverUrl` is empty (the default).
160
- - In `hybrid-sync` the local SQLite copy keeps serving reads regardless; in `only-cloud` no local database is opened, so an outage with no `failoverUrl` surfaces as an error.
161
- - Prevents agent blocking or crash loops during internet outages or cloud service degradation.
162
-
163
- ### Secure Credential Storage
164
-
165
- Cloud authentication tokens are never written to `config.json`. They are stored in `auth_secrets.enc`, encrypted with **AES-256-GCM** using a key derived via **PBKDF2-HMAC-SHA256 (600,000 iterations)** from a stable machine fingerprint (OS machine ID + platform + architecture). The file is written with owner-only permissions (`0600`).
166
-
167
- > **Note:** this is not an OS keychain (DPAPI / Keychain / Secret Service). The fingerprint components are readable by other processes running as the same user, so the encryption protects against file exfiltration and casual inspection, not against a compromised local account. Secrets are bound to the machine — copying `auth_secrets.enc` to another computer will not decrypt.
168
- >
169
- > **Exception:** the headless fallback via `MEMORY_DIR/.env` (`TURSO_DB_URL`, `TURSO_DB_TOKEN`, `TURSO_API_TOKEN`) stores credentials in **plain text** by design, for Docker/CI deployments.
170
-
171
- ---
172
-
173
- ## Available Tools
174
-
175
- The package currently exposes **17 unique tool names** across its two integration surfaces:
176
-
177
- | Integration surface | Tool count | Composition |
178
- | :------------------ | ---------: | :---------- |
179
- | **MCP server** (Codex, Claude Code, Antigravity / Gemini CLI and other MCP clients) | **15** | 6 Notebook + 4 identity/graph + 5 RAG tools |
180
- | **Native OpenCode plugin** | **17** | 15 shared memory/identity/RAG tools + 2 OpenCode-only helpers |
181
-
182
- The two surfaces are intentionally counted separately. The OpenCode plugin exposes
183
- all 15 shared tools, including `batch_query_knowledge_base`, and adds the native
184
- `list-mcp-tools` and `mcp-reminder` helpers.
185
-
186
- ### MCP Server Tools (15)
187
-
188
- #### 1. Memory Notebook Tools (Layer 1)
189
-
190
- | Tool | Scope / Target | Key Parameters | Description |
191
- | :--- | :------------- | :------------- | :---------- |
192
- | `remember` | `project` / `global` | `fact`, `title`, `scope`, `docId`, `startLine`, `endLine`, `relationType`, `ttl`, `keep`, `tags`, `supersedes` | Save a durable fact or preference. Supports optional title, document linking, TTL, keep protection, tags, and version superseding. |
193
- | `recall` | `all`, `project`, `global`, `list_projects` | `scope`, `project`, `query`, `tags`, `since`, `until`, `mode`, `offset`, `limit`, `includeSuperseded` | Display saved facts with metadata badges and linked docs. Superseded facts are hidden by default; opt into history with `includeSuperseded: true`. Filtered results retain stable storage indices for safe `forget` operations. |
194
- | `get_fact` | `all`, `project`, `global` | `id`, `scope` | Retrieve full text, raw line, and metadata of a single fact by its metadata ID (e.g. `"8f3a2c"`). |
195
- | `update_fact` | `project` / `global` | `id`, `newText`, `title`, `scope` | Rewrite a fact (and optionally its `**Title**`) while preserving its original creation date, metadata, and knowledge links. |
196
- | `forget` | `project` / `global` | `id` / `range` / `query`, `scope`, `force` | Remove a fact by index number, ID, range (e.g. `"3-30"`), or query. Requires `force: true` for protected (`[KEEP]`) facts. |
197
- | `memory_info` | - | - | Show storage paths, fact counts, RAG statistics, git identity bindings, and package version. |
198
-
199
- #### 2. Project Identity Tools
200
-
201
- | Tool | Key Parameters | Description |
202
- | :--- | :------------- | :---------- |
203
- | `link_project_memory` | `directory`, `remote` | Link a directory path to a Git-based project identity key, register aliases, and migrate legacy path/basename stores with deduplication. |
204
- | `unlink_project_memory` | `directory`, `purge` | Remove a path alias binding for a directory. Optionally purge the project identity entry if `purge: true`. |
205
- | `relink_project_memory` | `directory`, `remote` | Switch a project's primary identity to a new remote URL and merge all stored facts into the target store with fact-text deduplication. |
206
-
207
- #### 3. RAG Knowledge Base & Graph Tools (Layers 2 & 3)
208
-
209
- | Tool | Key Parameters | Description |
210
- | :--- | :------------- | :---------- |
211
- | `ingest_document` | `content`, `type`, `title`, `path`, `scope`, `generateEmbeddings` | Ingest local files, URLs, or raw text into the 3-tier index. Defaults to the current linked Git project; use `scope: "global"` for intentionally shared knowledge. |
212
- | `query_knowledge_base` | `query`, `scope`, `limit`, `instruction`, `generateEmbeddings` | Perform hybrid search (RSF/RRF BM25 + dense vectors). `all` means global plus the current project, while `project` and `global` restrict retrieval explicitly. |
213
- | `batch_query_knowledge_base` | `queries` (array), `scope`, `limit`, `instruction`, `generateEmbeddings` | Execute multiple scoped queries in one call. All embeddings are computed in one ONNX pass, making this the preferred API for comparisons and multi-part research. |
214
- | `manage_knowledge_base` | `action`, `scope`, `docId`, `snapshotPath` | Inspect, list, read, or unlink documents within the selected scope, or export/import complete snapshots. Delete defaults to the current project (global outside Git); broader removal requires explicit `global` or `all`. |
215
- | `reindex_knowledge_base` | `model`, `dimension` | Re-embed all stored vectors with the active (or specified) embedding model and vector dimension. Use after switching the embedding model or vector dimension so previously indexed documents remain retrievable. Preserves documents, FTS index, graph edges, and fact links. |
216
- | `link_knowledge` | `action`, `factText`, `docId`, `scope`, `startLine`, `endLine`, `relationType` | Create, list, or retrieve semantic graph links connecting Notebook facts to Knowledge Base documents, sections, or line ranges. Actions: `link`, `list_links`, `get_doc_links`. |
217
-
218
- ### Native OpenCode Plugin Tools (17)
219
-
220
- | Group | Count | Tools |
221
- | :---- | ----: | :---- |
222
- | **Memory Notebook** | 6 | `remember`, `recall`, `get_fact`, `forget`, `update_fact`, `memory_info` |
223
- | **Project Identity & Knowledge Graph** | 4 | `link_knowledge`, `link_project_memory`, `unlink_project_memory`, `relink_project_memory` |
224
- | **RAG Knowledge Base** | 5 | `ingest_document`, `query_knowledge_base`, `batch_query_knowledge_base`, `manage_knowledge_base`, `reindex_knowledge_base` |
225
- | **OpenCode-only helpers** | 2 | `list-mcp-tools`, `mcp-reminder` |
226
-
227
- The OpenCode-only helpers have the following purpose:
228
-
229
- | Tool | Key Parameters | Description |
230
- | :--- | :------------- | :---------- |
231
- | `list-mcp-tools` | - | Discover connected MCP servers and their intended use cases. |
232
- | `mcp-reminder` | `task` | Recommend the appropriate MCP tool or server for a developer task. |
233
-
234
- ---
235
-
236
- ## CLI Command Reference
237
-
238
- The plugin provides both direct non-interactive CLI commands and an interactive terminal UI (TUI):
239
-
240
- ```bash
241
- # Executable commands (available globally or via npx)
242
- memory_plugin <command> [options]
243
- # or
244
- memory-cli <command> [options]
245
- ```
246
-
247
- Both binaries accept the same commands. `memory_plugin` with **no** command starts the MCP server on stdio; `memory-cli` with no command opens the interactive TUI. Use `--help` on either for the full usage text.
248
-
249
- > **Secrets:** prefer the environment variables `TURSO_API_TOKEN`, `TURSO_DB_URL` and `TURSO_DB_TOKEN` over `--api-key` / `--db-token` flags — arguments passed on the command line are visible in the process list and shell history. Without a flag or env var, `login` prompts for the token on stdin with echo disabled.
250
-
251
- ### Direct Non-Interactive Commands
252
-
253
- | Command | Options / Flags | Description |
254
- | :------ | :-------------- | :---------- |
255
- | **`setup`** | `--antigravity`, `--opencode`, `--claude`, `--codex`, `--local`, `--api-key <TOKEN>`, `--mode <MODE>` | Configures MCP server registrations across detected environments and sets initial cloud auth/sync mode. |
256
- | **`link`** | `--dir <path>`, `--remote <url>` | Links a directory to a Git project identity or remote URL. |
257
- | **`unlink`** | `--dir <path>`, `--purge` | Unlinks a directory path alias. `--purge` removes the identity record. |
258
- | **`relink`** | `--remote <url>`, `--dir <path>` | Relinks project identity to a new remote URL and merges facts. |
259
- | **`identity`** | `--dir <path>` | Inspects Git project identity key, primary remote, name, and toplevel path for a directory. |
260
- | **`migrate_titles`** | `--key <key>` | Auto-generates `**Title**` prefixes for legacy facts without titles. |
261
- | **`enable-prompt`** | - | Injects memory agent instructions into client agent files (`AGENTS.md`, `CLAUDE.md`). |
262
- | **`disable-prompt`** | - | Removes memory agent instructions from client agent files. |
263
- | **`doctor`** | `--codex` | Validates Codex config, direct Node runtime, MCP initialize/tools/list, and live `memory_info` + `recall` calls. |
264
- | **`login`** | `--api-token`, `--from-env`, `--db-url <URL> --db-token`, `$TURSO_API_TOKEN`, `$TURSO_DB_TOKEN` | Authenticates with Turso Cloud via API token, direct DB token, or environment variables. Token values are read from the environment or a hidden stdin prompt. |
265
- | **`logout`** | `--api-key` | Signs out of Turso Cloud or removes stored API key while retaining DB session. |
266
- | **`auth-status`** | - | Displays authentication source, endpoint URL, username, organization, database, and sync mode. |
267
-
268
- ### Interactive TUI (CLI Menu)
269
-
270
- Launch the interactive terminal UI to manage engine settings, tune retrieval algorithms, inspect databases, and run diagnostics:
271
-
272
- ```bash
273
- memory_plugin cli
274
- # or
275
- memory-cli
276
- ```
277
-
278
- #### TUI Menu Navigation
279
-
280
- Use **Up / Down** arrows to navigate, **ENTER** to select, and **BACKSPACE** to go back.
281
-
282
- - **Engine & Hybrid Search Settings**: Switch fusion algorithms (`rsf`, `rrf`, `semantic_only`, `lexical_only`), adjust RSF $\alpha$ balance, select ONNX embedding models, set a fixed embedding vector dimension, toggle Cross-Encoder rerankers, configure GPU attention budget, and set WASM threads.
283
- - **Knowledge Base & Storage Management**: Browse Layer 1 facts, manage Layer 2 RAG docs, re-embed all vectors after switching model/dimension (`[REINDEX]`), export/import JSON snapshots, purge model cache, or perform a hard reset.
284
- - **Global Prompt & Integration**: Toggle memory instruction sync across client configurations (`~/.gemini/config/AGENTS.md`, `~/.codex/AGENTS.md`, `~/.claude/CLAUDE.md`).
285
- - **Diagnostics & System Actions**: Execute a hybrid search verification query, run the graph & notebook linking check, and reset config to factory defaults.
286
-
287
- ---
288
-
289
- ## Agent Skill & System Prompt Integration
290
-
291
- ### Built-in Agent Skill (`using-memory`)
292
-
293
- The plugin bundles a pre-configured Agent Skill located at [`skills/using-memory/SKILL.md`](./skills/using-memory/SKILL.md). When installed in supported environments (such as Antigravity / Gemini CLI or OpenCode), coding agents automatically read this skill to:
294
- 1. **Initialize Sessions**: Invoke full-body `recall(scope: "all")` first, loading global memory plus only the current Git project's memory (or global-only outside Git).
295
- 2. **Register Git Projects**: Inspect `memory_info` after recall and automatically call `link_project_memory` when the current Git identity reports `Registry: unlinked`.
296
- 3. **Proactively Save Context**: Automatically call `remember` whenever you share durable facts, tech stack choices, or coding guidelines.
297
- 4. **Curate and Use RAG Deliberately**: Preserve important web findings and current technical documentation only when they are likely to matter again. Store project-specific sources in the current project scope, reserve global scope for reusable cross-project material, query with concept-dense searches, and avoid low-value bulk ingestion.
298
- 5. **Architect Knowledge Graphs**: Use `link_knowledge` or `remember(docId, startLine, endLine)` to connect concise project memories to their supporting technical sources.
299
-
300
- ### Global System Prompt Synchronization
301
-
302
- Run `memory_plugin enable-prompt` to automatically inject memory agent guidance into your global LLM configuration files:
303
- - `~/.gemini/config/AGENTS.md` (Antigravity / Gemini CLI)
304
- - `~/.codex/AGENTS.md` (Codex)
305
- - `~/.claude/CLAUDE.md` (Claude Code)
306
-
307
- This ensures your coding assistant is always aware of its memory capabilities even in fresh workspace sessions.
308
-
309
- ---
310
-
311
- ## GraphRAG Lite
312
-
313
- The RAG engine includes a lightweight graph layer built on SQLite. It combines code symbol extraction, hierarchy edges, and explicit memory-to-document links without requiring external graph databases or LLM calls at ingest time.
314
-
315
- ### Multilingual Code Symbol Parsing
316
-
317
- During `ingest_document`, code symbols are extracted from code blocks using fast regex heuristics across the following language families:
318
-
319
- - **JavaScript / TypeScript**: `function`, `class`, `interface`, `type`, `enum`, `const`, `let`, `var`
320
- - **Python**: `def`, `class`
321
- - **Go**: `struct`, `interface`, `func` (including methods with receivers)
322
- - **Rust**: `struct`, `enum`, `trait`, `fn` (including async/pub)
323
- - **C++**: `class`, `struct`, `namespace`, functions and methods
324
- - **Java & Kotlin**: `class`, `interface`, `record`, `enum`, `fun`, synchronized methods
325
- - **C#**: `class`, `interface`, `struct`, `record`, methods (properties without a parameter list are not captured)
326
- - **PHP**: `class`, `interface`, `trait`, functions
327
- - **Ruby**: `module`, `class`, methods
328
-
329
- ### Graph Edges
330
-
331
- | Relation Type | Direction / Example |
332
- | :--- | :--- |
333
- | `CONTAINS` | Document -> Section -> Micro-Chunk (3-tier hierarchy) |
334
- | `DEFINES_SYMBOL` | Section -> `symbol:<name>` (extracted code symbol) |
335
- | `LINKS_TO` (default) | Memory fact -> Document or line range (via `link_knowledge`) |
336
-
337
- ---
338
-
339
- ## Supported Platforms
340
-
341
- | Platform | Status | Configuration Mechanism |
342
- | :--- | :--- | :--- |
343
- | **Antigravity / Gemini CLI** | Supported | MCP Server (`~/.gemini/config/mcp_config.json` & `.agents/mcp_config.json`) |
344
- | **OpenCode** | Native | Native plugin with 16 tools (`~/.config/opencode/opencode.json`) |
345
- | **Claude Code** | Supported | MCP Server (`~/.claude.json`) |
346
- | **Codex** | Supported | MCP Server (`~/.codex/config.toml`) |
347
- | **Google Jules** | Supported | MCP Server via global install + setup (`memory_plugin setup`) |
348
-
349
- ---
350
-
351
- ## Configuration
352
-
353
- The engine is configured through `<memory-dir>/config.json` (created with defaults on first run):
354
-
355
- | Key | Default | Description |
356
- | :--- | :--- | :--- |
357
- | `mode` | `only-local` | Storage sync mode: `only-local`, `only-cloud`, or `hybrid-sync` |
358
- | `fusionAlgorithm` | `rsf` | `rsf`, `rrf`, `semantic_only`, or `lexical_only` |
359
- | `alpha` | `0.5` | Vector vs BM25 weight in RSF `[0.0 - 1.0]` |
360
- | `embeddingModel` | `Xenova/multilingual-e5-small` | ONNX dense embedding model (E5 / BGE families supported) |
361
- | `vectorDimension` | `0` | Fixed embedding vector dimension; `0` = auto-detect from the model |
362
- | `rerankerModel` | `none` | Cross-encoder reranker model (e.g. `Xenova/bge-reranker-base`) |
363
- | `rerankerEnabled` | `false` | Enable cross-encoder re-ranking |
364
- | `batchSize` | `12` | Ingestion vector batch size `[1 - 256]` |
365
- | `gpuAttentionBudget` | `2000000` | GPU micro-batch attention budget `[1M - 16M]` |
366
- | `onnxThreads` | `0` | ONNX WASM threads: `0` auto-detect, or `1-16` |
367
- | `executionDevice` | `cpu` | `cpu` or `webgpu` (experimental) |
368
- | `vectorScanLimit` | `50000` | Max micro-chunks scanned per vector query (`0` = unlimited) |
369
- | `policyExpansion` | `true` | Expand table_summary/code_signature policy chunks for better recall (slight MRR trade-off). Disable for pure micro_chunk precision. |
370
- | `conflictStrategy` | `merge` | Hybrid-sync conflict resolution: `merge`, `cloud-wins`, or `local-wins` |
371
- | `tursoUrl` | `""` | Primary Turso endpoint URL (set by `login`) |
372
- | `failoverUrl` | `""` | Secondary cloud endpoint for the circuit breaker; empty = failover disabled |
373
- | `authorized` | `false` | Set to `true` once a cloud login completed |
374
- | `username` | `""` | Account username from the Turso profile |
375
- | `ingestAllowedPaths` | `[]` | Extra directories `ingest_document(type: "file")` may read from |
376
- | `ingestAllowAnyPath` | `false` | Escape hatch: allow reading **any** path from disk (unsafe) |
377
-
378
- > `ingest_document(type: "file")` reads only from the current working directory and the plugin data directory by default. This prevents a prompt-injected agent from pulling `~/.ssh/id_rsa` or `.env` into the knowledge base (and, in `hybrid-sync`, into the cloud). Widen it deliberately via `ingestAllowedPaths`.
379
-
380
- ---
381
-
382
- ## Testing & Benchmarking
383
-
384
- To run the automated test suite and benchmarks locally, from the repository root:
385
-
386
- ```bash
387
- # Unit + integration + cloud suites (18 files) — fast and fully offline
388
- npm test
389
-
390
- # End-to-end smoke test with REAL ONNX embeddings run before a release
391
- npm run smoke
392
-
393
- # Search quality & ingestion benchmarks
394
- npm run benchmark
395
- ```
396
-
397
- ### Two testing modes, and why both exist
398
-
399
- `npm test` runs every suite with `generateEmbeddings: false`. That keeps it fast
400
- and offline (no model download, no network), but it means the **dense-vector half
401
- of the engine is never exercised** retrieval falls back to BM25-only.
402
-
403
- `npm run smoke` covers exactly that blind spot: it ingests a document with real
404
- ONNX vectors and asserts that hybrid retrieval returns a non-zero cosine
405
- similarity, plus that a Russian query still reaches an English document
406
- (something BM25 cannot do). It also walks the full user journey — remember →
407
- recall → ingest → query → link → update → get → forget — and checks the ingest
408
- path guard.
409
-
410
- This split is not academic: a regression in `v1.5.3+` disabled vector search
411
- entirely (`node:sqlite` returns BLOBs as `Uint8Array`, and a `Buffer.isBuffer()`
412
- guard discarded every stored vector) while all offline suites stayed green. The
413
- smoke test exists so that class of failure cannot ship unnoticed again.
414
-
415
- The smoke test reuses the model weights already cached in your data directory, so
416
- it does not re-download them. Point `MEMORY_MODEL_CACHE` at a cache directory to
417
- override the lookup; without any cache the weights are fetched once.
418
-
419
- For complete methodology details and search quality evaluation metrics, see [`docs/BENCHMARKS.md`](./docs/BENCHMARKS.md).
420
-
421
- ### Empirical Search Quality Results
422
-
423
- Evaluated across a 32-document technical corpus (21 queries) using Mean Reciprocal Rank (MRR@5), Recall@5, and Normalized Discounted Cumulative Gain (NDCG@5):
424
-
425
- | Retrieval Strategy | MRR@5 | Recall@5 | NDCG@5 |
426
- | :---------------------------- | :--------: | :---------: | :--------: |
427
- | BM25 Lexical Search Only | 0.6706 | 76.19% | 0.6934 |
428
- | Dense ONNX Vector Only | 0.8135 | 100.00% | 0.8612 |
429
- | Hybrid RRF ($k=60$) | 0.8810 | 95.24% | 0.8997 |
430
- | **Hybrid RSF ($\alpha=0.5$)** | **0.9286** | **100.00%** | **0.9473** |
431
-
432
- ---
433
-
434
- ## Storage & Privacy
435
-
436
- - **Local-First Storage**: All SQLite indexes, ONNX models, CAS blobs, and Markdown notebooks are stored locally in the memory directory. Resolution order: `$MEMORY_DIR` → `$OPENCODE_CONFIG_DIR/memory` → the legacy `~/.config/opencode/memory` directory when it already exists (this takes precedence on Windows too) → `%LOCALAPPDATA%\opencode\memory`.
437
- - **Model Weights**: ONNX weights are downloaded on first use from `https://huggingface.co` and cached locally; later runs are fully offline.
438
- - **Zero Telemetry**: No third-party analytics or telemetry calls are made. The only outbound traffic is the one-off model download, plus Turso requests in the cloud sync modes.
439
-
440
- ### Known Dependency Advisories
441
-
442
- `npm audit` reports three high-severity advisories with no upstream fix available. Both affected packages are only reachable through explicit user action:
443
-
444
- | Package | Advisory | Reachability | Mitigation |
445
- | :--- | :--- | :--- | :--- |
446
- | `xlsx` (SheetJS) | Prototype Pollution ([GHSA-4r6h-8v6p-xvw6](https://github.com/advisories/GHSA-4r6h-8v6p-xvw6)), ReDoS ([GHSA-5pgg-2g8v-p4x9](https://github.com/advisories/GHSA-5pgg-2g8v-p4x9)) | Only when you ingest a `.xlsx` / `.xls` / `.csv` file | Do not ingest untrusted spreadsheets. A migration to `exceljs` is under consideration. |
447
- | `sharp` `<0.35.0` (via `@huggingface/transformers`) | libvips CVE-2026-33327 / 33328 / 35590 / 35591 ([GHSA-f88m-g3jw-g9cj](https://github.com/advisories/GHSA-f88m-g3jw-g9cj)) | Image decoding path only; this plugin runs text embedding models exclusively and never feeds images to `sharp` | Not reachable in normal use; will clear once `@huggingface/transformers` bumps `sharp`. |
448
-
449
- ---
450
-
451
- ## License
452
-
453
- [MIT](./LICENSE)
140
+ The doctor validates the configured Node runtime, MCP initialization, tool discovery, and real `memory_info` and `recall(scope: "all")` calls.
141
+
142
+ ### Headless / CI Setup
143
+
144
+ ```bash
145
+ # Authenticate with a Turso account token and enable hybrid sync
146
+ memory_plugin setup --api-key <TURSO_API_TOKEN> --mode hybrid-sync
147
+
148
+ # Or change mode when credentials already exist
149
+ memory_plugin setup --mode only-cloud
150
+ ```
151
+
152
+ Prefer `TURSO_API_TOKEN`, `TURSO_DB_URL`, and `TURSO_DB_TOKEN` environment variables over command-line secrets because shell arguments may appear in process lists and history.
153
+
154
+ ### Local Repository Development
155
+
156
+ ```bash
157
+ npm install
158
+ npm run dev:link
159
+ ```
160
+
161
+ `dev:link` performs an npm global link for the `memory_plugin`, `memory-agent`, and `memory-cli` binaries; rewrites only this plugin's OpenCode entry to an absolute `file://` URL for `opencode-plugin/main.js`; creates `opencode.json.memory-dev-backup` on first use; synchronizes managed prompts; and copies the current skill to all client skill locations.
162
+
163
+ After code changes, restart OpenCode to reload the module. Codex, Claude Code, Gemini CLI, and Antigravity load prompt and skill files at session start, so open a new task/session after synchronization. Publishing to npm is not required for local testing.
164
+
165
+ ---
166
+
167
+ ## Memory Architecture
168
+
169
+ ### 1. Hot Notebook Memory
170
+
171
+ Notebook memory stores concise, high-signal context in Markdown:
172
+
173
+ ```text
174
+ - [2026-08-22 10:00] **API Convention** Use Fastify and Zod for new services <!-- id:a1b2c3, keep:1, tags:arch, kind:fact -->
175
+ ```
176
+
177
+ Supported metadata includes:
178
+
179
+ - `id`: stable short identifier used by `get_fact`, `update_fact`, and `forget`.
180
+ - `kind`: `fact` for descriptive context or `directive` for active personalization/working instructions.
181
+ - `ttl`: `90d`, `2w`, `24h`, `12m`, or a bare day count. Expired entries are retained and marked `[EXPIRED]`.
182
+ - `keep`: protects an entry from ordinary deletion.
183
+ - `tags`: recall filters and legacy classification metadata.
184
+ - `supersedes` / `supersededBy`: preserves version history while excluding obsolete facts from active recall.
185
+
186
+ `recall(scope: "all")` returns global facts plus only the current Git-linked project's facts. Full bodies are the default and should be used for session initialization; `mode: "headers"` is only for compact inventories.
187
+
188
+ ### 2. Cold RAG Memory Notes
189
+
190
+ Use `remember_note` when the reusable value is in the detailed record itself:
191
+
192
+ ```text
193
+ remember_note(
194
+ title: "Authentication Investigation",
195
+ content: "Detailed symptoms, experiments, rejected explanations, and final cause...",
196
+ kind: "research",
197
+ tags: "auth,incident",
198
+ scope: "project"
199
+ )
200
+ ```
201
+
202
+ Supported note kinds are `decision`, `research`, `context`, `handoff`, and `note`. Notes are represented as virtual RAG documents with stable `docId` and content-addressed `blobHash`. They are searchable with the same engine as external sources but are not injected into every session.
203
+
204
+ Recommended discovery flow:
205
+
206
+ ```text
207
+ query_knowledge_base(query: "authentication token decryption investigation", resultMode: "index")
208
+ -> inspect compact candidates and stable doc_id values
209
+ manage_knowledge_base(action: "read_document", docId: "selected-id")
210
+ -> expand the complete raw note only when needed
211
+ ```
212
+
213
+ Use `resultMode: "snippet"` when retrieved passages are immediately useful. Use `resultMode: "index"` when first identifying the correct source; index mode intentionally omits bodies and disables large policy expansion.
214
+
215
+ ### 3. Curated External Knowledge
216
+
217
+ `ingest_document` accepts:
218
+
219
+ - Raw text or Markdown (`type: "text"`).
220
+ - Local files (`type: "file"`), including PDF, DOCX, XLSX, XLS, CSV, text, Markdown, and source code.
221
+ - Web pages (`type: "url"`), which are fetched and normalized instead of indexing the URL string.
222
+
223
+ RAG is a curated library, not an automatic archive. Ingest reliable sources likely to matter again, particularly current documentation or project specifications. Project scope is the default; use global scope only for intentionally reusable cross-project knowledge.
224
+
225
+ ### Hot + Cold Linking
226
+
227
+ When a decision needs both quick orientation and detailed history:
228
+
229
+ 1. Save the concise conclusion with `remember`.
230
+ 2. Save the rationale or investigation with `remember_note`.
231
+ 3. Connect them with `link_knowledge`, using the note's returned `docId`.
232
+
233
+ This keeps startup context small while preserving the complete reasoning trail without duplicating the note body into Notebook memory.
234
+
235
+ ---
236
+
237
+ ## Persona and Agent Personalization
238
+
239
+ Notebook entries have explicit semantics:
240
+
241
+ ```text
242
+ kind: "fact" # descriptive context
243
+ kind: "directive" # active user-approved personality or working configuration
244
+ ```
245
+
246
+ Use `kind: "directive"` for personality, behavior, tone, communication style, preferences, or working conventions the agent should actively apply. Explicit `kind` is authoritative; persuasive wording alone does not turn a fact into an instruction.
247
+
248
+ ### OpenCode
249
+
250
+ The native plugin performs complete session initialization automatically:
251
+
252
+ - Global and current-project descriptive entries are injected into `<MEMORY_FACTS>`.
253
+ - Active global directives are separated into `<PERSONAL_AGENT_OVERLAY>`.
254
+ - Directives are promoted through OpenCode's system-prompt transform.
255
+ - Agents are instructed not to perform a redundant startup `recall`; manual or filtered recall remains available.
256
+
257
+ ### Codex, Claude Code, Gemini CLI, and Antigravity
258
+
259
+ These clients receive plugin-owned instruction and persona blocks in:
260
+
261
+ - `~/.codex/AGENTS.md`
262
+ - `~/.claude/CLAUDE.md`
263
+ - `~/.gemini/GEMINI.md`
264
+ - `~/.gemini/config/AGENTS.md`
265
+
266
+ The global Notebook is the source of truth. Managed prompt blocks are generated views and update automatically after global directive changes, relevant cloud pulls, setup, or `dev:link`.
267
+
268
+ Manual synchronization:
269
+
270
+ ```bash
271
+ memory-cli sync-persona
272
+ npm run persona:sync # from the repository
273
+ ```
274
+
275
+ Legacy entries using `persona`, `behavior`, `speech`, `style`, `tone`, `preference(s)`, `instruction(s)`, `directive`, or `inject:1` metadata remain compatible. Permanently classify them as explicit directives with the idempotent migration:
276
+
277
+ ```bash
278
+ memory-cli migrate-persona --dry-run
279
+ memory-cli migrate-persona
280
+ npm run persona:migrate # from the repository
281
+ ```
282
+
283
+ Higher-priority platform and safety instructions remain authoritative.
284
+
285
+ ---
286
+
287
+ ## Project Identity and Scope Isolation
288
+
289
+ Project memory is Git-first:
290
+
291
+ - Repositories with a remote use `git:<normalized-host-and-path>`, for example `git:github.com/owner/repo`.
292
+ - Repositories without a remote use `git:local:<repository-name>`.
293
+ - Every subdirectory of the same repository resolves to the same identity.
294
+ - Outside Git, project memory is not created; global memory remains available.
295
+
296
+ The SQLite identity registry stores remote, path, and basename aliases. It supports moving a repository between directories or operating systems without changing its logical memory identity.
297
+
298
+ | Tool | Purpose |
299
+ | :--- | :--- |
300
+ | `link_project_memory` | Register the current Git identity and merge compatible legacy path/basename facts and RAG scope data. |
301
+ | `unlink_project_memory` | Remove a path alias; optionally purge the identity record. |
302
+ | `relink_project_memory` | Move/merge facts and RAG scope data to a new normalized remote identity. |
303
+
304
+ For both Notebook and RAG retrieval, `all` means **global + current project**, never all known projects. Unrelated project memories and documents are isolated.
305
+
306
+ ---
307
+
308
+ ## Retrieval and Knowledge Graph
309
+
310
+ ### Hybrid Retrieval
311
+
312
+ The local retrieval pipeline combines:
313
+
314
+ - SQLite FTS5 BM25 lexical search.
315
+ - Local ONNX dense embeddings (`Xenova/multilingual-e5-small` by default).
316
+ - RSF (default), RRF, semantic-only, or lexical-only ranking.
317
+ - Optional cross-encoder reranking.
318
+ - Batched query embeddings through `batch_query_knowledge_base`.
319
+ - Optional fixed vector dimensions and an experimental WebGPU execution mode.
320
+
321
+ Queries should be short, concept-dense phrases. For multi-part research or comparisons, use `batch_query_knowledge_base`; all query embeddings are computed in one ONNX pass.
322
+
323
+ ### Three-Tier Chunking and Policy Expansion
324
+
325
+ Each document is partitioned into three retrieval levels: section-level big chunks, medium blocks, and micro chunks. Tables receive compact summaries and code blocks receive signature chunks. With `policyExpansion: true` (default), matching summaries/signatures expand to their full source blocks for content-rich retrieval. Set the configuration to `false` when pure micro-chunk precision is preferred.
326
+
327
+ Re-ingesting an updated path/URL preserves its stable document ID and knowledge links while rebuilding chunks, vectors, policies, and structural edges. Ingesting the same source in another scope adds a scope association without duplicating the document.
328
+
329
+ ### GraphRAG Lite
330
+
331
+ The SQLite graph layer requires no external graph database or ingestion-time LLM:
332
+
333
+ | Relation | Meaning |
334
+ | :--- | :--- |
335
+ | `CONTAINS` | Document -> Section -> Micro Chunk graph hierarchy |
336
+ | `DEFINES_SYMBOL` | A document section defines an extracted code symbol |
337
+ | `LINKS_TO` and custom relations | A Notebook fact points to a document, note, section, or line range |
338
+
339
+ Code-symbol extraction covers JavaScript/TypeScript, Python, Go, Rust, C++, Java/Kotlin, C#, PHP, and Ruby patterns.
340
+
341
+ ---
342
+
343
+ ## Cloud Synchronization
344
+
345
+ Cloud support uses Turso / LibSQL and is optional.
346
+
347
+ | Mode | Behavior |
348
+ | :--- | :--- |
349
+ | `only-local` (default) | Markdown notebooks, SQLite index, CAS blobs, and models remain local. |
350
+ | `only-cloud` | Notebook and database operations use Turso directly; raw RAG blobs are materialized into a verified local cache when read. |
351
+ | `hybrid-sync` | Local-first reads/writes with background push, reverse synchronization, and conflict resolution. |
352
+
353
+ Hybrid synchronization covers Notebook stores and complete RAG state: documents, scopes, sections, chunks, vectors, retrieval policies, graph edges, fact links, compressed raw CAS blobs, and deletion tombstones. Raw notes/documents can therefore be expanded on another device rather than returning metadata without source content.
354
+
355
+ Notebook conflict strategies:
356
+
357
+ - `merge` (default): union fact lines with local order first and deduplication.
358
+ - `cloud-wins`.
359
+ - `local-wins`.
360
+
361
+ Cloud operations retry with timeouts and can switch to `failoverUrl` after repeated primary failures. In `hybrid-sync`, local SQLite continues serving reads during an outage. In `only-cloud`, an unavailable primary with no failover surfaces as an error.
362
+
363
+ ### Authentication
364
+
365
+ ```bash
366
+ memory-cli login
367
+ memory-cli login --api-token # hidden prompt if value omitted
368
+ memory-cli login --from-env
369
+ memory-cli login --db-url <URL> # token from prompt or TURSO_DB_TOKEN
370
+ memory-cli auth-status
371
+ memory-cli logout
372
+ ```
373
+
374
+ Stored tokens live in `auth_secrets.enc`, not `config.json`. They are encrypted with AES-256-GCM using PBKDF2-HMAC-SHA256 (600,000 iterations) over a stable machine fingerprint and written with owner-only permissions where supported.
375
+
376
+ This is not an OS keychain. It protects against casual inspection/file-only exfiltration, not a compromised local user account. Encrypted secrets are machine-bound. The headless `.env` fallback stores credentials in plaintext by design.
377
+
378
+ ---
379
+
380
+ ## Tool Reference
381
+
382
+ The MCP server exposes **16 tools**. The native OpenCode plugin exposes the same 16 plus two OpenCode-specific helpers, for **18 total**.
383
+
384
+ ### Notebook and Cold Memory
385
+
386
+ | Tool | Important parameters | Purpose |
387
+ | :--- | :--- | :--- |
388
+ | `remember` | `fact`, `title`, `kind`, `scope`, `directory`, `ttl`, `keep`, `tags`, `supersedes`, optional link fields | Save a concise hot fact or directive. |
389
+ | `recall` | `scope`, `directory`, `query`, `tags`, `since`, `until`, `mode`, `offset`, `limit`, `includeSuperseded` | Load/filter Notebook facts and linked-document references. |
390
+ | `get_fact` | `id`, `scope`, `directory` | Read one fact and all metadata by stable ID. |
391
+ | `update_fact` | `id`, `newText`, `title`, `kind`, `scope`, `directory` | Update/reclassify a fact while preserving date, metadata, and links. |
392
+ | `forget` | `query`, `scope`, `directory`, `force` | Delete by index, range, ID, or text; `force` overrides `[KEEP]`. |
393
+ | `memory_info` | `directory` | Show version, storage paths/counts, Git identity/registry state, and RAG statistics. |
394
+ | `remember_note` | `title`, `content`, `kind`, `tags`, `scope`, `directory`, `generateEmbeddings` | Save a detailed cold/episodic note into RAG. |
395
+
396
+ ### Identity and Knowledge Graph
397
+
398
+ | Tool | Important parameters | Purpose |
399
+ | :--- | :--- | :--- |
400
+ | `link_project_memory` | `directory`, `remote` | Register Git identity and migrate compatible legacy data. |
401
+ | `unlink_project_memory` | `directory`, `purge` | Remove an alias or purge its registry identity. |
402
+ | `relink_project_memory` | `directory`, `remote` | Move/merge memory into a new Git remote identity. |
403
+ | `link_knowledge` | `action`, `factText`, `docId`, `scope`, `startLine`, `endLine`, `relationType` | Link facts to documents/notes or inspect graph links. |
404
+
405
+ ### RAG Knowledge Base
406
+
407
+ | Tool | Important parameters | Purpose |
408
+ | :--- | :--- | :--- |
409
+ | `ingest_document` | `content`, `type`, `title`, `path`, `scope`, `directory`, `generateEmbeddings` | Ingest raw text, a local file, or a URL. |
410
+ | `query_knowledge_base` | `query`, `scope`, `limit`, `instruction`, `resultMode`, `generateEmbeddings`, `directory` | Run one hybrid query in snippet or compact index mode. |
411
+ | `batch_query_knowledge_base` | `queries`, `scope`, `limit`, `instruction`, `resultMode`, `generateEmbeddings`, `directory` | Run several queries with one embedding batch. |
412
+ | `manage_knowledge_base` | `action`, `scope`, `docId`, `snapshotPath`, `directory` | Stats, list, full raw read, scoped delete/unlink, snapshot export/import. |
413
+ | `reindex_knowledge_base` | `model`, `dimension` | Rebuild vectors after changing model/dimension while preserving source and graph data. |
414
+
415
+ ### OpenCode-Only Helpers
416
+
417
+ | Tool | Purpose |
418
+ | :--- | :--- |
419
+ | `list-mcp-tools` | Show connected MCP servers and their intended roles. |
420
+ | `mcp-reminder` | Suggest a connected MCP/tool family for a described task. |
421
+
422
+ ---
423
+
424
+ ## CLI Reference
425
+
426
+ `memory_plugin` and `memory-agent` are MCP stdio entry points. `memory_plugin setup` performs client installation, while `memory_plugin cli` or `memory-cli` opens the interactive control panel. Direct administration commands should use `memory-cli`.
427
+
428
+ | Command | Purpose |
429
+ | :--- | :--- |
430
+ | `memory_plugin setup [client flags] [--mode <mode>]` | Configure clients, skills, prompts, and optional cloud mode/auth. |
431
+ | `memory_plugin doctor --codex` | Validate Codex configuration and live MCP behavior. |
432
+ | `memory-cli` | Open the interactive TUI. |
433
+ | `memory-cli login ...` / `logout` / `auth-status` | Manage Turso authentication. |
434
+ | `memory-cli link --dir <path> [--remote <url>]` | Link a Git project identity. |
435
+ | `memory-cli unlink --dir <path> [--purge]` | Remove an alias or registry identity. |
436
+ | `memory-cli relink --dir <path> --remote <url>` | Move/merge into a new remote identity. |
437
+ | `memory-cli identity --dir <path>` | Inspect resolved Git identity. |
438
+ | `memory-cli migrate_titles [--key <key>]` | Add titles to legacy Notebook entries. |
439
+ | `memory-cli enable-prompt` / `disable-prompt` | Add/remove only plugin-owned memory instruction blocks. |
440
+ | `memory-cli sync-persona` | Regenerate managed persona blocks from global directives. |
441
+ | `memory-cli migrate-persona [--dry-run]` | Convert legacy persona metadata to explicit `kind:directive`. |
442
+ | `memory-cli dev-link` | Link the installed binaries/OpenCode plugin to the working repository. |
443
+ | `memory-cli uninstall [--purge] [--purge-cache] [--dry-run] [--yes] [client flags]` | Remove plugin, MCP entries, prompts and skills; `--purge` deletes local data, while `--purge-cache` explicitly removes only this plugin's OpenCode cache. |
444
+
445
+ The TUI provides retrieval configuration, model management, Notebook/RAG browsing, reindexing, snapshots, cloud settings, prompt integration, diagnostics, and reset actions. Use Up/Down, Enter, and Backspace to navigate.
446
+
447
+ ---
448
+
449
+ ## Client Integration
450
+
451
+ | Client | Integration | Session initialization | Tool count |
452
+ | :--- | :--- | :--- | ---: |
453
+ | OpenCode | Native plugin in `~/.config/opencode/opencode.json` | Full memory auto-injection + system persona transform | 18 |
454
+ | Codex | MCP server in `~/.codex/config.toml` | Managed prompt requires full `recall(scope: "all")` | 16 |
455
+ | Claude Code | MCP server in `~/.claude.json` | Managed prompt requires full `recall(scope: "all")` | 16 |
456
+ | Gemini CLI | MCP server in `~/.gemini/settings.json` | Managed `~/.gemini/GEMINI.md` prompt requires full `recall(scope: "all")` | 16 |
457
+ | Antigravity | MCP server in `~/.gemini/config/mcp_config.json` and optional `.agents/mcp_config.json` | Managed prompt requires full `recall(scope: "all")` | 16 |
458
+ | Google Jules / generic MCP | MCP stdio server | Client instructions should initialize with full recall | 16 |
459
+
460
+ The bundled [`using-memory` skill](./skills/using-memory/SKILL.md) teaches agents to:
461
+
462
+ 1. Avoid duplicate recall when OpenCode already auto-injected memory.
463
+ 2. Perform full unfiltered recall first in clients without auto-injection.
464
+ 3. Apply `kind:directive` entries as active configuration.
465
+ 4. Register unlinked Git identities with `link_project_memory`.
466
+ 5. Route concise facts, long internal notes, and external sources to the correct store.
467
+ 6. Use semantic index discovery before expanding a full note/document.
468
+ 7. Save high-signal knowledge proactively and avoid transient noise.
469
+
470
+ ---
471
+
472
+ ## Configuration
473
+
474
+ Configuration is stored in `<memory-dir>/config.json`.
475
+
476
+ | Key | Default | Meaning |
477
+ | :--- | :--- | :--- |
478
+ | `mode` | `only-local` | `only-local`, `only-cloud`, or `hybrid-sync` |
479
+ | `conflictStrategy` | `merge` | Notebook conflict policy: `merge`, `cloud-wins`, `local-wins` |
480
+ | `fusionAlgorithm` | `rsf` | `rsf`, `rrf`, `semantic_only`, or `lexical_only` |
481
+ | `alpha` | `0.5` | Dense-vector weight for RSF |
482
+ | `embeddingModel` | `Xenova/multilingual-e5-small` | Local Hugging Face/ONNX embedding model |
483
+ | `vectorDimension` | `0` | Fixed vector size; `0` auto-detects model output |
484
+ | `vectorScanLimit` | `50000` | Maximum vector candidates; `0` is unlimited |
485
+ | `rerankerModel` | `none` | Optional cross-encoder model |
486
+ | `rerankerEnabled` | `false` | Enable cross-encoder reranking |
487
+ | `batchSize` | `12` | Ingestion embedding batch size |
488
+ | `policyExpansion` | `true` | Expand matched table summaries/code signatures |
489
+ | `executionDevice` | `cpu` | `cpu` or experimental `webgpu` |
490
+ | `gpuAttentionBudget` | `2000000` | Experimental GPU micro-batch budget |
491
+ | `onnxThreads` | `0` | WASM thread count; `0` auto-detects |
492
+ | `tursoUrl` | `""` | Primary LibSQL endpoint populated by login |
493
+ | `failoverUrl` | `""` | Optional secondary cloud endpoint |
494
+ | `authorized` | `false` | Whether cloud authorization completed |
495
+ | `username` | `""` | Authenticated Turso username |
496
+ | `ingestAllowedPaths` | `[]` | Additional directories allowed for local-file ingestion |
497
+ | `ingestAllowAnyPath` | `false` | Unsafe escape hatch allowing arbitrary file reads |
498
+
499
+ `ingest_document(type: "file")` is restricted to the current working directory, the plugin data directory, and explicitly allowed paths. This prevents a prompt-injected agent from silently indexing unrelated secrets such as SSH keys or `.env` files.
500
+
501
+ ---
502
+
503
+ ## Storage, Privacy, and Security
504
+
505
+ The data-directory resolution order is:
506
+
507
+ 1. `MEMORY_DIR`.
508
+ 2. `$OPENCODE_CONFIG_DIR/memory`.
509
+ 3. Existing legacy `~/.config/opencode/memory`.
510
+ 4. `%LOCALAPPDATA%/opencode/memory` on Windows.
511
+ 5. `$XDG_CONFIG_HOME/opencode/memory` or `~/.config/opencode/memory` elsewhere.
512
+
513
+ Important paths inside it:
514
+
515
+ ```text
516
+ global.md global Notebook facts/directives
517
+ git_<identity>.md per-project Notebook facts
518
+ config.json non-secret configuration
519
+ auth_secrets.enc encrypted cloud credentials
520
+ storage/memory.sqlite RAG, graph, identity registry, sync state
521
+ storage/blobs/ content-addressed compressed raw sources
522
+ storage/models/ cached ONNX models
523
+ exports/ snapshots/exports
524
+ ```
525
+
526
+ - No telemetry or analytics are sent.
527
+ - Model weights download from Hugging Face on first use and remain cached afterward.
528
+ - Network access is otherwise limited to explicit URL ingestion and configured Turso cloud modes.
529
+ - Snapshot path validation and local ingestion allowlists restrict arbitrary filesystem access.
530
+ - SQLite uses foreign keys, migrations, transactions, and a busy timeout for concurrent access.
531
+
532
+ ### Dependency Advisories
533
+
534
+ At the time of this README update, `npm audit` reports three high-severity findings with no npm-available fix:
535
+
536
+ | Package | Exposure in this project | Mitigation |
537
+ | :--- | :--- | :--- |
538
+ | `xlsx` | Spreadsheet parsing when the user explicitly ingests XLSX/XLS/CSV | Do not ingest untrusted spreadsheets. Advisories: [prototype pollution](https://github.com/advisories/GHSA-4r6h-8v6p-xvw6), [ReDoS](https://github.com/advisories/GHSA-5pgg-2g8v-p4x9). |
539
+ | `sharp` via `@huggingface/transformers` | Transformers dependency includes image decoding, while this project supplies text to embedding/reranking pipelines | Normal text-memory use does not exercise the image path. Advisory: [libvips inherited vulnerabilities](https://github.com/advisories/GHSA-f88m-g3jw-g9cj). |
540
+
541
+ ---
542
+
543
+ ## Testing and Benchmarks
544
+
545
+ ```bash
546
+ npm test # 25 unit, integration, and simulated-cloud suites
547
+ npm run smoke # real ONNX vectors and end-to-end memory journey
548
+ npm run test:rag # retrieval quality evaluation
549
+ npm run benchmark # full search benchmark report
550
+ npm run benchmark:table-code
551
+ ```
552
+
553
+ The fast suites use `generateEmbeddings: false` in retrieval paths for deterministic offline coverage. `npm run smoke` covers the dense-vector path with real cached/downloaded model weights and checks multilingual semantic retrieval. Both modes are needed: lexical-only tests cannot catch a broken vector serialization or ONNX execution path.
554
+
555
+ The unified suites cover fact formatting, typed directives, persona migration/synchronization, client prompt safety, Codex launcher compatibility, Git identity isolation, RAG scopes, policy expansion, RAG Memory Notes, semantic index output, raw blob portability, reverse sync, tombstones, snapshots, MCP contracts, and cloud authentication workflows.
556
+
557
+ See [`docs/BENCHMARKS.md`](./docs/BENCHMARKS.md) for methodology and detailed reports.
558
+
559
+ ### Included Search Evaluation
560
+
561
+ The stored 32-document / 21-query technical corpus produced:
562
+
563
+ | Strategy | MRR@5 | Recall@5 | NDCG@5 |
564
+ | :--- | :---: | :---: | :---: |
565
+ | BM25 lexical only | 0.6706 | 76.19% | 0.6934 |
566
+ | Dense ONNX only | 0.8135 | 100.00% | 0.8612 |
567
+ | Hybrid RRF (`k=60`) | 0.8810 | 95.24% | 0.8997 |
568
+ | **Hybrid RSF (`alpha=0.5`)** | **0.9286** | **100.00%** | **0.9473** |
569
+
570
+ ---
571
+
572
+ ## Troubleshooting
573
+
574
+ - **`No such built-in module: node:sqlite`**: install Node.js `22.5.0` or newer.
575
+ - **Codex tools are missing**: run `memory_plugin setup --codex`, then `memory_plugin doctor --codex`, and open a new Codex task.
576
+ - **OpenCode still runs old code**: restart OpenCode. For repository development, confirm `npm run dev:link` points its plugin entry to `opencode-plugin/main.js`.
577
+ - **Persona changes are not visible**: run `memory-cli sync-persona`, then start a new CLI session/task. Use `memory-cli migrate-persona --dry-run` for legacy entries.
578
+ - **Project recall is empty**: call `memory_info`; if a Git identity is `Registry: unlinked`, run `link_project_memory` or `memory-cli link --dir <repo>`.
579
+ - **A raw note/document exists only in cloud**: `manage_knowledge_base(action: "read_document")` automatically materializes and verifies its CAS blob locally when cloud credentials are available.
580
+ - **Embedding model changed**: run `reindex_knowledge_base` or use the TUI `[REINDEX]` action.
581
+
582
+ ---
583
+
584
+ ## License
585
+
586
+ [MIT](./LICENSE)