@lotargo/memory_plugin 1.6.7 → 1.6.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +154 -129
- package/README.md +611 -586
- package/mcp-server/admin/auth.js +675 -645
- package/mcp-server/ingest/normalizer.js +1 -1
- package/package.json +33 -17
- package/mcp-server/benchmarks/fetch_real_corpus.js +0 -351
- package/mcp-server/benchmarks/gpu_profile_benchmark.js +0 -170
- package/mcp-server/benchmarks/policy_dominance_test.js +0 -221
- package/mcp-server/benchmarks/quality_evaluator.js +0 -598
- package/mcp-server/benchmarks/raw_corpus_data.js +0 -613
- package/mcp-server/benchmarks/run_benchmarks.js +0 -366
- package/mcp-server/benchmarks/stress_ingestion.js +0 -195
- package/mcp-server/benchmarks/table_code_retrieval.js +0 -453
- package/mcp-server/benchmarks/test_dual_layer.js +0 -141
package/README.md
CHANGED
|
@@ -1,586 +1,611 @@
|
|
|
1
|
-
<div align="center">
|
|
2
|
-
|
|
3
|
-
<img src="
|
|
4
|
-
|
|
5
|
-
<br>
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
[](https://www.npmjs.com/package/@lotargo/memory_plugin)
|
|
8
|
+
[](https://www.npmjs.com/package/@lotargo/memory_plugin)
|
|
9
|
+
[](./LICENSE)
|
|
10
|
+
[](https://nodejs.org)
|
|
11
|
+
[](https://modelcontextprotocol.io)
|
|
12
|
+
[](#storage-privacy-and-security)
|
|
13
|
+
|
|
14
|
+
**Local-first long-term memory, hybrid RAG, and agent personalization for AI coding agents**
|
|
15
|
+
|
|
16
|
+
One memory system for OpenCode, Codex, Claude Code, Gemini CLI, Antigravity, Google Jules, and other MCP clients.
|
|
17
|
+
|
|
18
|
+
</div>
|
|
19
|
+
|
|
20
|
+
# @lotargo/memory_plugin — Local-First Memory, Hybrid RAG & Agent Personalization
|
|
21
|
+
|
|
22
|
+
<p align="center">
|
|
23
|
+
<a href="#quick-start">Quick Start</a> ·
|
|
24
|
+
<a href="#memory-architecture">Memory Architecture</a> ·
|
|
25
|
+
<a href="#persona-and-agent-personalization">Persona</a> ·
|
|
26
|
+
<a href="#retrieval-and-knowledge-graph">RAG & Retrieval</a> ·
|
|
27
|
+
<a href="#cloud-synchronization">Cloud Sync</a> ·
|
|
28
|
+
<a href="#client-integration">Clients</a> ·
|
|
29
|
+
<a href="#storage-privacy-and-security">Security</a>
|
|
30
|
+
</p>
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Why This Project Exists
|
|
35
|
+
|
|
36
|
+
AI coding assistants forget user preferences, architectural decisions, investigations, and project context when a session ends. They also tend to mix very different kinds of information into one oversized prompt.
|
|
37
|
+
|
|
38
|
+
`@lotargo/memory_plugin` separates persistent knowledge into the right storage class:
|
|
39
|
+
|
|
40
|
+
| What you want to preserve | Tool | Storage behavior |
|
|
41
|
+
| :--- | :--- | :--- |
|
|
42
|
+
| Concise facts, preferences, constraints, conventions, and persona settings | `remember` | **Hot Notebook memory**; available during session initialization |
|
|
43
|
+
| Detailed decisions, research, investigations, experiments, and handoffs | `remember_note` | **Cold RAG Memory Note**; searchable but not injected into every session |
|
|
44
|
+
| Files, URLs, documentation, reports, specifications, and code | `ingest_document` | **Curated external knowledge** in the RAG index |
|
|
45
|
+
|
|
46
|
+
The same engine adds Git-based project isolation, semantic search, full raw-source expansion, explicit fact-to-document links, optional Turso synchronization, and native OpenCode auto-injection.
|
|
47
|
+
|
|
48
|
+
### Architecture at a Glance
|
|
49
|
+
|
|
50
|
+
<img src="./assets/02_project_evolution.png" alt="Evolution of memory_plugin from notebook memory through RAG, persona and cross-client agent state" width="100%">
|
|
51
|
+
|
|
52
|
+
### Highlights
|
|
53
|
+
|
|
54
|
+
- Human-readable Markdown Notebook facts with stable IDs, TTL, protection, tags, superseding, and explicit `fact` / `directive` semantics.
|
|
55
|
+
- Agent-authored long-form RAG Memory Notes for cold or episodic context.
|
|
56
|
+
- Hybrid SQLite FTS5 BM25 + local ONNX vector retrieval with RSF/RRF fusion.
|
|
57
|
+
- Compact semantic TOC discovery through `resultMode: "index"`, followed by deliberate full-source expansion.
|
|
58
|
+
- PDF, DOCX, XLSX/XLS/CSV, Markdown, text, HTML/URL, and source-code ingestion.
|
|
59
|
+
- Three-tier document hierarchy, retrieval-policy expansion for tables/code, and GraphRAG Lite symbol extraction.
|
|
60
|
+
- Git-identity project scopes that follow a repository across directories, machines, and operating systems.
|
|
61
|
+
- Active persona overlays shared across OpenCode, Codex, Claude Code, Gemini CLI, and Antigravity.
|
|
62
|
+
- Local-only, cloud-only, and bidirectional hybrid-sync modes, including portable raw RAG blobs and deletion tombstones.
|
|
63
|
+
- No Docker, external vector database, hosted embedding API, or telemetry.
|
|
64
|
+
|
|
65
|
+
> This is a practical agent-memory system, not a claim of generalized benchmark superiority. Repository benchmark results describe the included evaluation corpus and configuration.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## Quick Start
|
|
70
|
+
|
|
71
|
+
### Requirements
|
|
72
|
+
|
|
73
|
+
- Node.js `22.5.0` or newer; the project uses the built-in `node:sqlite` module.
|
|
74
|
+
- npm/npx.
|
|
75
|
+
- OpenCode, Codex, Claude Code, Gemini CLI, Antigravity, Google Jules, or another MCP-capable client.
|
|
76
|
+
|
|
77
|
+
CPU execution with `Xenova/multilingual-e5-small` is the recommended stable default. WebGPU execution is experimental.
|
|
78
|
+
|
|
79
|
+
### Install / Uninstall All Clients
|
|
80
|
+
|
|
81
|
+
The normal installation path is one command. It configures every supported client directly, so there is no separate package-only installation step:
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
npx -y @lotargo/memory_plugin setup
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
This registers memory_plugin in OpenCode, Codex, Claude Code, Gemini CLI, and Antigravity, installs the bundled `using-memory` skill, and adds managed memory instructions where supported. Existing unrelated configuration is preserved.
|
|
88
|
+
|
|
89
|
+
Remove the integration from all clients while keeping Notebook/RAG data:
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
npx -y @lotargo/memory_plugin uninstall
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Remove the integrations and delete local memory data:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
npx -y @lotargo/memory_plugin uninstall --purge --yes
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Preview the uninstall without changing anything:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
npx -y @lotargo/memory_plugin uninstall --dry-run
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
What `uninstall` removes by default (without `--purge`):
|
|
108
|
+
|
|
109
|
+
- `~/.config/opencode/opencode.json` - plugin entry, including `file://` dev links.
|
|
110
|
+
- `~/.claude.json` - `mcpServers.memory-agent`.
|
|
111
|
+
- `~/.gemini/settings.json` - Gemini CLI `mcpServers.memory-agent`.
|
|
112
|
+
- `~/.gemini/config/mcp_config.json` and `.agents/mcp_config.json` - Antigravity `mcpServers.memory-agent`.
|
|
113
|
+
- `~/.codex/config.toml` - `[mcp_servers.memory-agent]` only when owned by this plugin.
|
|
114
|
+
- Managed prompt blocks from Codex, Claude Code, Gemini CLI, and Antigravity instruction files.
|
|
115
|
+
- The bundled `using-memory` skill from each client's managed `skills/` directory.
|
|
116
|
+
|
|
117
|
+
Existing user content outside plugin-owned markers is preserved. Foreign `memory-agent` registrations, modified/non-owned skills, unrelated file plugins, and other packages in the `@lotargo` OpenCode cache namespace are left untouched.
|
|
118
|
+
|
|
119
|
+
Normal uninstall keeps OpenCode's package cache. `--purge-cache` removes only exact cache directories owned by this package. With `--purge`, the plugin also deletes `MEMORY_DIR` and its prompt state after resolving and validating every target, rejecting dangerous roots and broad parent paths, and displaying the targets before confirmation.
|
|
120
|
+
|
|
121
|
+
On Linux/macOS, `XDG_CONFIG_HOME` and `XDG_CACHE_HOME` are respected. `OPENCODE_CONFIG_DIR` and `MEMORY_DIR` remain explicit overrides on every platform.
|
|
122
|
+
|
|
123
|
+
### Install One Client
|
|
124
|
+
|
|
125
|
+
Use the same one-shot setup command with a client flag when you only want one integration:
|
|
126
|
+
|
|
127
|
+
```bash
|
|
128
|
+
npx -y @lotargo/memory_plugin setup --opencode
|
|
129
|
+
npx -y @lotargo/memory_plugin setup --codex
|
|
130
|
+
npx -y @lotargo/memory_plugin setup --claude
|
|
131
|
+
npx -y @lotargo/memory_plugin setup --antigravity
|
|
132
|
+
npx -y @lotargo/memory_plugin setup --gemini
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Use `--local` with Antigravity to create the workspace-local `.agents/mcp_config.json` even when `.agents/` does not yet exist:
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
npx -y @lotargo/memory_plugin setup --antigravity --local
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
The same client flags can be used for targeted removal, for example:
|
|
142
|
+
|
|
143
|
+
```bash
|
|
144
|
+
npx -y @lotargo/memory_plugin uninstall --opencode
|
|
145
|
+
npx -y @lotargo/memory_plugin uninstall --codex
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Claude Code, Gemini CLI, and Codex setup/uninstall use their native MCP lifecycle commands when available. An ownership-checked config edit is retained as a compatibility fallback for missing, older, or non-functional client CLIs. Antigravity remains a separate integration because it uses a different config layout.
|
|
149
|
+
|
|
150
|
+
### Verify Codex
|
|
151
|
+
|
|
152
|
+
Codex uses a direct executable chain (`node` -> `mcp-server/boot.js`) instead of an `npx`/`.cmd` launcher, avoiding Windows stdio handshake failures. Setup safely migrates legacy registrations in `~/.codex/config.toml`.
|
|
153
|
+
|
|
154
|
+
```bash
|
|
155
|
+
npx -y @lotargo/memory_plugin doctor --codex
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
The doctor validates the configured Node runtime, MCP initialization, tool discovery, and real `memory_info` and `recall(scope: "all")` calls.
|
|
159
|
+
|
|
160
|
+
### Headless / CI Setup
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
# Authenticate with a Turso account token and enable hybrid sync
|
|
164
|
+
npx -y @lotargo/memory_plugin setup --api-key <TURSO_API_TOKEN> --mode hybrid-sync
|
|
165
|
+
|
|
166
|
+
# Or change mode when credentials already exist
|
|
167
|
+
npx -y @lotargo/memory_plugin setup --mode only-cloud
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
Prefer `TURSO_API_TOKEN`, `TURSO_DB_URL`, and `TURSO_DB_TOKEN` environment variables over command-line secrets because shell arguments may appear in process lists and history.
|
|
171
|
+
|
|
172
|
+
### Local Repository Development
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
npm install
|
|
176
|
+
npm run dev:link
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
`dev:link` performs an npm global link for the `memory_plugin`, `memory-agent`, and `memory-cli` binaries; rewrites only this plugin's OpenCode entry to an absolute `file://` URL for `opencode-plugin/main.js`; creates `opencode.json.memory-dev-backup` on first use; synchronizes managed prompts; and copies the current skill to all client skill locations.
|
|
180
|
+
|
|
181
|
+
After code changes, restart OpenCode to reload the module. Codex, Claude Code, Gemini CLI, and Antigravity load prompt and skill files at session start, so open a new task/session after synchronization. Publishing to npm is not required for local testing.
|
|
182
|
+
|
|
183
|
+
---
|
|
184
|
+
|
|
185
|
+
## Memory Architecture
|
|
186
|
+
|
|
187
|
+
<img src="./assets/03_hot_memory_vs_cold_rag.png" alt="Hot persistent memory versus cold RAG retrieval architecture in memory_plugin" width="100%">
|
|
188
|
+
|
|
189
|
+
The architecture deliberately separates **small, always-useful context** from **large, on-demand knowledge**. This keeps session initialization useful without turning persistent memory into an ever-growing prompt.
|
|
190
|
+
|
|
191
|
+
### 1. Hot Notebook Memory
|
|
192
|
+
|
|
193
|
+
Notebook memory stores concise, high-signal context in Markdown:
|
|
194
|
+
|
|
195
|
+
```text
|
|
196
|
+
- [2026-08-22 10:00] **API Convention** — Use Fastify and Zod for new services <!-- id:a1b2c3, keep:1, tags:arch, kind:fact -->
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
Supported metadata includes:
|
|
200
|
+
|
|
201
|
+
- `id`: stable short identifier used by `get_fact`, `update_fact`, and `forget`.
|
|
202
|
+
- `kind`: `fact` for descriptive context or `directive` for active personalization/working instructions.
|
|
203
|
+
- `ttl`: `90d`, `2w`, `24h`, `12m`, or a bare day count. Expired entries are retained and marked `[EXPIRED]`.
|
|
204
|
+
- `keep`: protects an entry from ordinary deletion.
|
|
205
|
+
- `tags`: recall filters and legacy classification metadata.
|
|
206
|
+
- `supersedes` / `supersededBy`: preserves version history while excluding obsolete facts from active recall.
|
|
207
|
+
|
|
208
|
+
`recall(scope: "all")` returns global facts plus only the current Git-linked project's facts. Full bodies are the default and should be used for session initialization; `mode: "headers"` is only for compact inventories.
|
|
209
|
+
|
|
210
|
+
### 2. Cold RAG Memory Notes
|
|
211
|
+
|
|
212
|
+
Use `remember_note` when the reusable value is in the detailed record itself:
|
|
213
|
+
|
|
214
|
+
```text
|
|
215
|
+
remember_note(
|
|
216
|
+
title: "Authentication Investigation",
|
|
217
|
+
content: "Detailed symptoms, experiments, rejected explanations, and final cause...",
|
|
218
|
+
kind: "research",
|
|
219
|
+
tags: "auth,incident",
|
|
220
|
+
scope: "project"
|
|
221
|
+
)
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Supported note kinds are `decision`, `research`, `context`, `handoff`, and `note`. Notes are represented as virtual RAG documents with stable `docId` and content-addressed `blobHash`. They are searchable with the same engine as external sources but are not injected into every session.
|
|
225
|
+
|
|
226
|
+
Recommended discovery flow:
|
|
227
|
+
|
|
228
|
+
```text
|
|
229
|
+
query_knowledge_base(query: "authentication token decryption investigation", resultMode: "index")
|
|
230
|
+
-> inspect compact candidates and stable doc_id values
|
|
231
|
+
manage_knowledge_base(action: "read_document", docId: "selected-id")
|
|
232
|
+
-> expand the complete raw note only when needed
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Use `resultMode: "snippet"` when retrieved passages are immediately useful. Use `resultMode: "index"` when first identifying the correct source; index mode intentionally omits bodies and disables large policy expansion.
|
|
236
|
+
|
|
237
|
+
### 3. Curated External Knowledge
|
|
238
|
+
|
|
239
|
+
`ingest_document` accepts:
|
|
240
|
+
|
|
241
|
+
- Raw text or Markdown (`type: "text"`).
|
|
242
|
+
- Local files (`type: "file"`), including PDF, DOCX, XLSX, XLS, CSV, text, Markdown, and source code.
|
|
243
|
+
- Web pages (`type: "url"`), which are fetched and normalized instead of indexing the URL string.
|
|
244
|
+
|
|
245
|
+
RAG is a curated library, not an automatic archive. Ingest reliable sources likely to matter again, particularly current documentation or project specifications. Project scope is the default; use global scope only for intentionally reusable cross-project knowledge.
|
|
246
|
+
|
|
247
|
+
### Hot + Cold Linking
|
|
248
|
+
|
|
249
|
+
When a decision needs both quick orientation and detailed history:
|
|
250
|
+
|
|
251
|
+
1. Save the concise conclusion with `remember`.
|
|
252
|
+
2. Save the rationale or investigation with `remember_note`.
|
|
253
|
+
3. Connect them with `link_knowledge`, using the note's returned `docId`.
|
|
254
|
+
|
|
255
|
+
This keeps startup context small while preserving the complete reasoning trail without duplicating the note body into Notebook memory.
|
|
256
|
+
|
|
257
|
+
---
|
|
258
|
+
|
|
259
|
+
## Persona and Agent Personalization
|
|
260
|
+
|
|
261
|
+
<img src="./assets/04_fact_vs_directive.png" alt="Fact versus directive semantics in memory_plugin" width="100%">
|
|
262
|
+
|
|
263
|
+
Notebook entries have explicit semantics:
|
|
264
|
+
|
|
265
|
+
```text
|
|
266
|
+
kind: "fact" # descriptive context — what the agent knows
|
|
267
|
+
kind: "directive" # active configuration — how the agent should behave
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
Use `kind: "directive"` for personality, behavior, tone, communication style, preferences, or working conventions the agent should actively apply. Explicit `kind` is authoritative; persuasive wording alone does not turn a fact into an instruction.
|
|
271
|
+
|
|
272
|
+
<img src="./assets/05_persona_as_runtime_state.png" alt="The same model with neutral, coding-focused and personalized runtime state" width="100%">
|
|
273
|
+
|
|
274
|
+
### OpenCode
|
|
275
|
+
|
|
276
|
+
The native plugin performs complete session initialization automatically:
|
|
277
|
+
|
|
278
|
+
- Global and current-project descriptive entries are injected into `<MEMORY_FACTS>`.
|
|
279
|
+
- Active global directives are separated into `<PERSONAL_AGENT_OVERLAY>`.
|
|
280
|
+
- Directives are promoted through OpenCode's system-prompt transform.
|
|
281
|
+
- Agents are instructed not to perform a redundant startup `recall`; manual or filtered recall remains available.
|
|
282
|
+
|
|
283
|
+
### Codex, Claude Code, Gemini CLI, and Antigravity
|
|
284
|
+
|
|
285
|
+
These clients receive plugin-owned instruction and persona blocks in:
|
|
286
|
+
|
|
287
|
+
- `~/.codex/AGENTS.md`
|
|
288
|
+
- `~/.claude/CLAUDE.md`
|
|
289
|
+
- `~/.gemini/GEMINI.md`
|
|
290
|
+
- `~/.gemini/config/AGENTS.md`
|
|
291
|
+
|
|
292
|
+
The global Notebook is the source of truth. Managed prompt blocks are generated views and update automatically after global directive changes, relevant cloud pulls, setup, or `dev:link`.
|
|
293
|
+
|
|
294
|
+
Manual synchronization:
|
|
295
|
+
|
|
296
|
+
```bash
|
|
297
|
+
memory-cli sync-persona
|
|
298
|
+
npm run persona:sync # from the repository
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
Legacy entries using `persona`, `behavior`, `speech`, `style`, `tone`, `preference(s)`, `instruction(s)`, `directive`, or `inject:1` metadata remain compatible. Permanently classify them as explicit directives with the idempotent migration:
|
|
302
|
+
|
|
303
|
+
```bash
|
|
304
|
+
memory-cli migrate-persona --dry-run
|
|
305
|
+
memory-cli migrate-persona
|
|
306
|
+
npm run persona:migrate # from the repository
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
Higher-priority platform and safety instructions remain authoritative.
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
313
|
+
## Project Identity and Scope Isolation
|
|
314
|
+
|
|
315
|
+
Project memory is Git-first:
|
|
316
|
+
|
|
317
|
+
- Repositories with a remote use `git:<normalized-host-and-path>`, for example `git:github.com/owner/repo`.
|
|
318
|
+
- Repositories without a remote use `git:local:<repository-name>`.
|
|
319
|
+
- Every subdirectory of the same repository resolves to the same identity.
|
|
320
|
+
- Outside Git, project memory is not created; global memory remains available.
|
|
321
|
+
|
|
322
|
+
The SQLite identity registry stores remote, path, and basename aliases. It supports moving a repository between directories or operating systems without changing its logical memory identity.
|
|
323
|
+
|
|
324
|
+
| Tool | Purpose |
|
|
325
|
+
| :--- | :--- |
|
|
326
|
+
| `link_project_memory` | Register the current Git identity and merge compatible legacy path/basename facts and RAG scope data. |
|
|
327
|
+
| `unlink_project_memory` | Remove a path alias; optionally purge the identity record. |
|
|
328
|
+
| `relink_project_memory` | Move/merge facts and RAG scope data to a new normalized remote identity. |
|
|
329
|
+
|
|
330
|
+
For both Notebook and RAG retrieval, `all` means **global + current project**, never all known projects. Unrelated project memories and documents are isolated.
|
|
331
|
+
|
|
332
|
+
---
|
|
333
|
+
|
|
334
|
+
## Retrieval and Knowledge Graph
|
|
335
|
+
|
|
336
|
+
### Hybrid Retrieval
|
|
337
|
+
|
|
338
|
+
The local retrieval pipeline combines:
|
|
339
|
+
|
|
340
|
+
- SQLite FTS5 BM25 lexical search.
|
|
341
|
+
- Local ONNX dense embeddings (`Xenova/multilingual-e5-small` by default).
|
|
342
|
+
- RSF (default), RRF, semantic-only, or lexical-only ranking.
|
|
343
|
+
- Optional cross-encoder reranking.
|
|
344
|
+
- Batched query embeddings through `batch_query_knowledge_base`.
|
|
345
|
+
- Optional fixed vector dimensions and an experimental WebGPU execution mode.
|
|
346
|
+
|
|
347
|
+
Queries should be short, concept-dense phrases. For multi-part research or comparisons, use `batch_query_knowledge_base`; all query embeddings are computed in one ONNX pass.
|
|
348
|
+
|
|
349
|
+
### Three-Tier Chunking and Policy Expansion
|
|
350
|
+
|
|
351
|
+
Each document is partitioned into three retrieval levels: section-level big chunks, medium blocks, and micro chunks. Tables receive compact summaries and code blocks receive signature chunks. With `policyExpansion: true` (default), matching summaries/signatures expand to their full source blocks for content-rich retrieval. Set the configuration to `false` when pure micro-chunk precision is preferred.
|
|
352
|
+
|
|
353
|
+
Re-ingesting an updated path/URL preserves its stable document ID and knowledge links while rebuilding chunks, vectors, policies, and structural edges. Ingesting the same source in another scope adds a scope association without duplicating the document.
|
|
354
|
+
|
|
355
|
+
### GraphRAG Lite
|
|
356
|
+
|
|
357
|
+
The SQLite graph layer requires no external graph database or ingestion-time LLM:
|
|
358
|
+
|
|
359
|
+
| Relation | Meaning |
|
|
360
|
+
| :--- | :--- |
|
|
361
|
+
| `CONTAINS` | Document -> Section -> Micro Chunk graph hierarchy |
|
|
362
|
+
| `DEFINES_SYMBOL` | A document section defines an extracted code symbol |
|
|
363
|
+
| `LINKS_TO` and custom relations | A Notebook fact points to a document, note, section, or line range |
|
|
364
|
+
|
|
365
|
+
Code-symbol extraction covers JavaScript/TypeScript, Python, Go, Rust, C++, Java/Kotlin, C#, PHP, and Ruby patterns.
|
|
366
|
+
|
|
367
|
+
---
|
|
368
|
+
|
|
369
|
+
## Cloud Synchronization
|
|
370
|
+
|
|
371
|
+
Cloud support uses Turso / LibSQL and is optional.
|
|
372
|
+
|
|
373
|
+
| Mode | Behavior |
|
|
374
|
+
| :--- | :--- |
|
|
375
|
+
| `only-local` (default) | Markdown notebooks, SQLite index, CAS blobs, and models remain local. |
|
|
376
|
+
| `only-cloud` | Notebook and database operations use Turso directly; raw RAG blobs are materialized into a verified local cache when read. |
|
|
377
|
+
| `hybrid-sync` | Local-first reads/writes with background push, reverse synchronization, and conflict resolution. |
|
|
378
|
+
|
|
379
|
+
Hybrid synchronization covers Notebook stores and complete RAG state: documents, scopes, sections, chunks, vectors, retrieval policies, graph edges, fact links, compressed raw CAS blobs, and deletion tombstones. Raw notes/documents can therefore be expanded on another device rather than returning metadata without source content.
|
|
380
|
+
|
|
381
|
+
Notebook conflict strategies:
|
|
382
|
+
|
|
383
|
+
- `merge` (default): union fact lines with local order first and deduplication.
|
|
384
|
+
- `cloud-wins`.
|
|
385
|
+
- `local-wins`.
|
|
386
|
+
|
|
387
|
+
Cloud operations retry with timeouts and can switch to `failoverUrl` after repeated primary failures. In `hybrid-sync`, local SQLite continues serving reads during an outage. In `only-cloud`, an unavailable primary with no failover surfaces as an error.
|
|
388
|
+
|
|
389
|
+
### Authentication
|
|
390
|
+
|
|
391
|
+
```bash
|
|
392
|
+
memory-cli login
|
|
393
|
+
memory-cli login --api-token # hidden prompt if value omitted
|
|
394
|
+
memory-cli login --from-env
|
|
395
|
+
memory-cli login --db-url <URL> # token from prompt or TURSO_DB_TOKEN
|
|
396
|
+
memory-cli auth-status
|
|
397
|
+
memory-cli logout
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
Stored tokens live in `auth_secrets.enc`, not `config.json`. They are encrypted with AES-256-GCM using PBKDF2-HMAC-SHA256 (600,000 iterations) over a stable machine fingerprint and written with owner-only permissions where supported.
|
|
401
|
+
|
|
402
|
+
This is not an OS keychain. It protects against casual inspection/file-only exfiltration, not a compromised local user account. Encrypted secrets are machine-bound. The headless `.env` fallback stores credentials in plaintext by design.
|
|
403
|
+
|
|
404
|
+
---
|
|
405
|
+
|
|
406
|
+
## Tool Reference
|
|
407
|
+
|
|
408
|
+
The MCP server exposes **16 tools**. The native OpenCode plugin exposes the same 16 plus two OpenCode-specific helpers, for **18 total**.
|
|
409
|
+
|
|
410
|
+
### Notebook and Cold Memory
|
|
411
|
+
|
|
412
|
+
| Tool | Important parameters | Purpose |
|
|
413
|
+
| :--- | :--- | :--- |
|
|
414
|
+
| `remember` | `fact`, `title`, `kind`, `scope`, `directory`, `ttl`, `keep`, `tags`, `supersedes`, optional link fields | Save a concise hot fact or directive. |
|
|
415
|
+
| `recall` | `scope`, `directory`, `query`, `tags`, `since`, `until`, `mode`, `offset`, `limit`, `includeSuperseded` | Load/filter Notebook facts and linked-document references. |
|
|
416
|
+
| `get_fact` | `id`, `scope`, `directory` | Read one fact and all metadata by stable ID. |
|
|
417
|
+
| `update_fact` | `id`, `newText`, `title`, `kind`, `scope`, `directory` | Update/reclassify a fact while preserving date, metadata, and links. |
|
|
418
|
+
| `forget` | `query`, `scope`, `directory`, `force` | Delete by index, range, ID, or text; `force` overrides `[KEEP]`. |
|
|
419
|
+
| `memory_info` | `directory` | Show version, storage paths/counts, Git identity/registry state, and RAG statistics. |
|
|
420
|
+
| `remember_note` | `title`, `content`, `kind`, `tags`, `scope`, `directory`, `generateEmbeddings` | Save a detailed cold/episodic note into RAG. |
|
|
421
|
+
|
|
422
|
+
### Identity and Knowledge Graph
|
|
423
|
+
|
|
424
|
+
| Tool | Important parameters | Purpose |
|
|
425
|
+
| :--- | :--- | :--- |
|
|
426
|
+
| `link_project_memory` | `directory`, `remote` | Register Git identity and migrate compatible legacy data. |
|
|
427
|
+
| `unlink_project_memory` | `directory`, `purge` | Remove an alias or purge its registry identity. |
|
|
428
|
+
| `relink_project_memory` | `directory`, `remote` | Move/merge memory into a new normalized remote identity. |
|
|
429
|
+
| `link_knowledge` | `action`, `factText`, `docId`, `scope`, `startLine`, `endLine`, `relationType` | Link facts to documents/notes or inspect graph links. |
|
|
430
|
+
|
|
431
|
+
### RAG Knowledge Base
|
|
432
|
+
|
|
433
|
+
| Tool | Important parameters | Purpose |
|
|
434
|
+
| :--- | :--- | :--- |
|
|
435
|
+
| `ingest_document` | `content`, `type`, `title`, `path`, `scope`, `directory`, `generateEmbeddings` | Ingest raw text, a local file, or a URL. |
|
|
436
|
+
| `query_knowledge_base` | `query`, `scope`, `limit`, `instruction`, `resultMode`, `generateEmbeddings`, `directory` | Run one hybrid query in snippet or compact index mode. |
|
|
437
|
+
| `batch_query_knowledge_base` | `queries`, `scope`, `limit`, `instruction`, `resultMode`, `generateEmbeddings`, `directory` | Run several queries with one embedding batch. |
|
|
438
|
+
| `manage_knowledge_base` | `action`, `scope`, `docId`, `snapshotPath`, `directory` | Stats, list, full raw read, scoped delete/unlink, snapshot export/import. |
|
|
439
|
+
| `reindex_knowledge_base` | `model`, `dimension` | Rebuild vectors after changing model/dimension while preserving source and graph data. |
|
|
440
|
+
|
|
441
|
+
### OpenCode-Only Helpers
|
|
442
|
+
|
|
443
|
+
| Tool | Purpose |
|
|
444
|
+
| :--- | :--- |
|
|
445
|
+
| `list-mcp-tools` | Show connected MCP servers and their intended roles. |
|
|
446
|
+
| `mcp-reminder` | Suggest a connected MCP/tool family for a described task. |
|
|
447
|
+
|
|
448
|
+
---
|
|
449
|
+
|
|
450
|
+
## CLI Reference
|
|
451
|
+
|
|
452
|
+
`memory_plugin` and `memory-agent` are MCP stdio entry points. The Quick Start uses `npx -y @lotargo/memory_plugin ...`, so a separate global npm installation is not required. If the package is installed globally for development or administration, `memory_plugin setup` performs client installation, while `memory_plugin cli` or `memory-cli` opens the interactive control panel. Direct administration commands should use `memory-cli`.
|
|
453
|
+
|
|
454
|
+
| Command | Purpose |
|
|
455
|
+
| :--- | :--- |
|
|
456
|
+
| `memory_plugin setup [client flags] [--mode <mode>]` | Configure clients, skills, prompts, and optional cloud mode/auth. |
|
|
457
|
+
| `memory_plugin doctor --codex` | Validate Codex configuration and live MCP behavior. |
|
|
458
|
+
| `memory-cli` | Open the interactive TUI. |
|
|
459
|
+
| `memory-cli login ...` / `logout` / `auth-status` | Manage Turso authentication. |
|
|
460
|
+
| `memory-cli link --dir <path> [--remote <url>]` | Link a Git project identity. |
|
|
461
|
+
| `memory-cli unlink --dir <path> [--purge]` | Remove an alias or registry identity. |
|
|
462
|
+
| `memory-cli relink --dir <path> --remote <url>` | Move/merge into a new remote identity. |
|
|
463
|
+
| `memory-cli identity --dir <path>` | Inspect resolved Git identity. |
|
|
464
|
+
| `memory-cli migrate_titles [--key <key>]` | Add titles to legacy Notebook entries. |
|
|
465
|
+
| `memory-cli enable-prompt` / `disable-prompt` | Add/remove only plugin-owned memory instruction blocks. |
|
|
466
|
+
| `memory-cli sync-persona` | Regenerate managed persona blocks from global directives. |
|
|
467
|
+
| `memory-cli migrate-persona [--dry-run]` | Convert legacy persona metadata to explicit `kind:directive`. |
|
|
468
|
+
| `memory-cli dev-link` | Link the installed binaries/OpenCode plugin to the working repository. |
|
|
469
|
+
| `memory-cli uninstall [--purge] [--purge-cache] [--dry-run] [--yes] [client flags]` | Remove plugin, MCP entries, prompts and skills; `--purge` deletes local data, while `--purge-cache` explicitly removes only this plugin's OpenCode cache. |
|
|
470
|
+
|
|
471
|
+
The TUI provides retrieval configuration, model management, Notebook/RAG browsing, reindexing, snapshots, cloud settings, prompt integration, diagnostics, and reset actions. Use Up/Down, Enter, and Backspace to navigate.
|
|
472
|
+
|
|
473
|
+
---
|
|
474
|
+
|
|
475
|
+
## Client Integration
|
|
476
|
+
|
|
477
|
+
| Client | Integration | Session initialization | Tool count |
|
|
478
|
+
| :--- | :--- | :--- | ---: |
|
|
479
|
+
| OpenCode | Native plugin in `~/.config/opencode/opencode.json` | Full memory auto-injection + system persona transform | 18 |
|
|
480
|
+
| Codex | MCP server in `~/.codex/config.toml` | Managed prompt requires full `recall(scope: "all")` | 16 |
|
|
481
|
+
| Claude Code | MCP server in `~/.claude.json` | Managed prompt requires full `recall(scope: "all")` | 16 |
|
|
482
|
+
| Gemini CLI | MCP server in `~/.gemini/settings.json` | Managed `~/.gemini/GEMINI.md` prompt requires full `recall(scope: "all")` | 16 |
|
|
483
|
+
| Antigravity | MCP server in `~/.gemini/config/mcp_config.json` and optional `.agents/mcp_config.json` | Managed prompt requires full `recall(scope: "all")` | 16 |
|
|
484
|
+
| Google Jules / generic MCP | MCP stdio server | Client instructions should initialize with full recall | 16 |
|
|
485
|
+
|
|
486
|
+
The bundled [`using-memory` skill](./skills/using-memory/SKILL.md) teaches agents to:
|
|
487
|
+
|
|
488
|
+
1. Avoid duplicate recall when OpenCode already auto-injected memory.
|
|
489
|
+
2. Perform full unfiltered recall first in clients without auto-injection.
|
|
490
|
+
3. Apply `kind:directive` entries as active configuration.
|
|
491
|
+
4. Register unlinked Git identities with `link_project_memory`.
|
|
492
|
+
5. Route concise facts, long internal notes, and external sources to the correct store.
|
|
493
|
+
6. Use semantic index discovery before expanding a full note/document.
|
|
494
|
+
7. Save high-signal knowledge proactively and avoid transient noise.
|
|
495
|
+
|
|
496
|
+
---
|
|
497
|
+
|
|
498
|
+
## Configuration
|
|
499
|
+
|
|
500
|
+
Configuration is stored in `<memory-dir>/config.json`.
|
|
501
|
+
|
|
502
|
+
| Key | Default | Meaning |
|
|
503
|
+
| :--- | :--- | :--- |
|
|
504
|
+
| `mode` | `only-local` | `only-local`, `only-cloud`, or `hybrid-sync` |
|
|
505
|
+
| `conflictStrategy` | `merge` | Notebook conflict policy: `merge`, `cloud-wins`, `local-wins` |
|
|
506
|
+
| `fusionAlgorithm` | `rsf` | `rsf`, `rrf`, `semantic_only`, or `lexical_only` |
|
|
507
|
+
| `alpha` | `0.5` | Dense-vector weight for RSF |
|
|
508
|
+
| `embeddingModel` | `Xenova/multilingual-e5-small` | Local Hugging Face/ONNX embedding model |
|
|
509
|
+
| `vectorDimension` | `0` | Fixed vector size; `0` auto-detects model output |
|
|
510
|
+
| `vectorScanLimit` | `50000` | Maximum vector candidates; `0` is unlimited |
|
|
511
|
+
| `rerankerModel` | `none` | Optional cross-encoder model |
|
|
512
|
+
| `rerankerEnabled` | `false` | Enable cross-encoder reranking |
|
|
513
|
+
| `batchSize` | `12` | Ingestion embedding batch size |
|
|
514
|
+
| `policyExpansion` | `true` | Expand matched table summaries/code signatures |
|
|
515
|
+
| `executionDevice` | `cpu` | `cpu` or experimental `webgpu` |
|
|
516
|
+
| `gpuAttentionBudget` | `2000000` | Experimental GPU micro-batch budget |
|
|
517
|
+
| `onnxThreads` | `0` | WASM thread count; `0` auto-detects |
|
|
518
|
+
| `tursoUrl` | `""` | Primary LibSQL endpoint populated by login |
|
|
519
|
+
| `failoverUrl` | `""` | Optional secondary cloud endpoint |
|
|
520
|
+
| `authorized` | `false` | Whether cloud authorization completed |
|
|
521
|
+
| `username` | `""` | Authenticated Turso username |
|
|
522
|
+
| `ingestAllowedPaths` | `[]` | Additional directories allowed for local-file ingestion |
|
|
523
|
+
| `ingestAllowAnyPath` | `false` | Unsafe escape hatch allowing arbitrary file reads |
|
|
524
|
+
|
|
525
|
+
`ingest_document(type: "file")` is restricted to the current working directory, the plugin data directory, and explicitly allowed paths. This prevents a prompt-injected agent from silently indexing unrelated secrets such as SSH keys or `.env` files.
|
|
526
|
+
|
|
527
|
+
---
|
|
528
|
+
|
|
529
|
+
## Storage, Privacy, and Security
|
|
530
|
+
|
|
531
|
+
The data-directory resolution order is:
|
|
532
|
+
|
|
533
|
+
1. `MEMORY_DIR`.
|
|
534
|
+
2. `$OPENCODE_CONFIG_DIR/memory`.
|
|
535
|
+
3. Existing legacy `~/.config/opencode/memory`.
|
|
536
|
+
4. `%LOCALAPPDATA%/opencode/memory` on Windows.
|
|
537
|
+
5. `$XDG_CONFIG_HOME/opencode/memory` or `~/.config/opencode/memory` elsewhere.
|
|
538
|
+
|
|
539
|
+
Important paths inside it:
|
|
540
|
+
|
|
541
|
+
```text
|
|
542
|
+
global.md global Notebook facts/directives
|
|
543
|
+
git_<identity>.md per-project Notebook facts
|
|
544
|
+
config.json non-secret configuration
|
|
545
|
+
auth_secrets.enc encrypted cloud credentials
|
|
546
|
+
storage/memory.sqlite RAG, graph, identity registry, sync state
|
|
547
|
+
storage/blobs/ content-addressed compressed raw sources
|
|
548
|
+
storage/models/ cached ONNX models
|
|
549
|
+
exports/ snapshots/exports
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
- No telemetry or analytics are sent.
|
|
553
|
+
- Model weights download from Hugging Face on first use and remain cached afterward.
|
|
554
|
+
- Network access is otherwise limited to explicit URL ingestion and configured Turso cloud modes.
|
|
555
|
+
- Snapshot path validation and local ingestion allowlists restrict arbitrary filesystem access.
|
|
556
|
+
- SQLite uses foreign keys, migrations, transactions, and a busy timeout for concurrent access.
|
|
557
|
+
|
|
558
|
+
### Dependency Advisories
|
|
559
|
+
|
|
560
|
+
Spreadsheet ingestion uses SheetJS CE `0.20.3` from the official SheetJS CDN rather than the stale `xlsx@0.18.5` package in the public npm registry. This version is outside the affected ranges for the known [prototype pollution](https://github.com/advisories/GHSA-4r6h-8v6p-xvw6) and [ReDoS](https://github.com/advisories/GHSA-5pgg-2g8v-p4x9) advisories.
|
|
561
|
+
|
|
562
|
+
`npm audit` may still report the high-severity `sharp` / libvips advisory inherited through `@huggingface/transformers`. The project uses Transformers only for text feature extraction and explicitly sets `env.sharp = false`; it does not pass images through the Transformers image-decoding path. The upstream dependency currently constrains `sharp` below the patched `0.35.x` line, so this warning remains transitive until Transformers updates its dependency. Advisory: [GHSA-f88m-g3jw-g9cj](https://github.com/advisories/GHSA-f88m-g3jw-g9cj).
|
|
563
|
+
|
|
564
|
+
---
|
|
565
|
+
|
|
566
|
+
## Repository Testing and Benchmarks
|
|
567
|
+
|
|
568
|
+
> These commands are intended for a **source checkout of the repository**. Test suites and benchmark harnesses are intentionally excluded from the published npm tarball.
|
|
569
|
+
|
|
570
|
+
```bash
|
|
571
|
+
npm test # unified unit, integration, and simulated-cloud suites
|
|
572
|
+
npm run smoke # real ONNX vectors and end-to-end memory journey
|
|
573
|
+
npm run test:rag # retrieval quality evaluation
|
|
574
|
+
npm run benchmark # full search benchmark report
|
|
575
|
+
npm run benchmark:table-code
|
|
576
|
+
```
|
|
577
|
+
|
|
578
|
+
The fast suites use `generateEmbeddings: false` in retrieval paths for deterministic offline coverage. `npm run smoke` covers the dense-vector path with real cached/downloaded model weights and checks multilingual semantic retrieval. Both modes are needed: lexical-only tests cannot catch a broken vector serialization or ONNX execution path.
|
|
579
|
+
|
|
580
|
+
The unified suites cover fact formatting, typed directives, persona migration/synchronization, client prompt safety, Codex launcher compatibility, Git identity isolation, RAG scopes, policy expansion, RAG Memory Notes, semantic index output, raw blob portability, reverse sync, tombstones, snapshots, MCP contracts, spreadsheet parsing, and cloud authentication workflows.
|
|
581
|
+
|
|
582
|
+
See [`docs/BENCHMARKS.md`](./docs/BENCHMARKS.md) for methodology and detailed reports.
|
|
583
|
+
|
|
584
|
+
### Included Search Evaluation
|
|
585
|
+
|
|
586
|
+
The stored 32-document / 21-query technical corpus produced:
|
|
587
|
+
|
|
588
|
+
| Strategy | MRR@5 | Recall@5 | NDCG@5 |
|
|
589
|
+
| :--- | :---: | :---: | :---: |
|
|
590
|
+
| BM25 lexical only | 0.6706 | 76.19% | 0.6934 |
|
|
591
|
+
| Dense ONNX only | 0.8135 | 100.00% | 0.8612 |
|
|
592
|
+
| Hybrid RRF (`k=60`) | 0.8810 | 95.24% | 0.8997 |
|
|
593
|
+
| **Hybrid RSF (`alpha=0.5`)** | **0.9286** | **100.00%** | **0.9473** |
|
|
594
|
+
|
|
595
|
+
---
|
|
596
|
+
|
|
597
|
+
## Troubleshooting
|
|
598
|
+
|
|
599
|
+
- **`No such built-in module: node:sqlite`**: install Node.js `22.5.0` or newer.
|
|
600
|
+
- **Codex tools are missing**: run `npx -y @lotargo/memory_plugin setup --codex`, then `npx -y @lotargo/memory_plugin doctor --codex`, and open a new Codex task.
|
|
601
|
+
- **OpenCode still runs old code**: restart OpenCode. For repository development, confirm `npm run dev:link` points its plugin entry to `opencode-plugin/main.js`.
|
|
602
|
+
- **Persona changes are not visible**: run `memory-cli sync-persona`, then start a new CLI session/task. Use `memory-cli migrate-persona --dry-run` for legacy entries.
|
|
603
|
+
- **Project recall is empty**: call `memory_info`; if a Git identity is `Registry: unlinked`, run `link_project_memory` or `memory-cli link --dir <repo>`.
|
|
604
|
+
- **A raw note/document exists only in cloud**: `manage_knowledge_base(action: "read_document")` automatically materializes and verifies its CAS blob locally when cloud credentials are available.
|
|
605
|
+
- **Embedding model changed**: run `reindex_knowledge_base` or use the TUI `[REINDEX]` action.
|
|
606
|
+
|
|
607
|
+
---
|
|
608
|
+
|
|
609
|
+
## License
|
|
610
|
+
|
|
611
|
+
[MIT](./LICENSE)
|