opencode-memory-pro 1.3.8 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +120 -1
- package/dist/index.js +40 -20
- package/dist/llm.js +29 -0
- package/dist/store.js +64 -8
- package/dist/tools/memory.js +37 -0
- package/opencode-memory-pro.example.json +169 -0
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -40,7 +40,7 @@ Published on npm — install directly (requires OpenCode ≥ 1.x and Node.js ≥
|
|
|
40
40
|
opencode plugin opencode-memory-pro
|
|
41
41
|
```
|
|
42
42
|
|
|
43
|
-
The latest release is **v1.
|
|
43
|
+
The latest release is **v1.4.0** on [npm](https://www.npmjs.com/package/opencode-memory-pro); source and releases are on [GitHub](https://github.com/tman204-50/opencode-memory-pro).
|
|
44
44
|
|
|
45
45
|
Remove the old plugin pin at the same time:
|
|
46
46
|
|
|
@@ -48,6 +48,87 @@ Remove the old plugin pin at the same time:
|
|
|
48
48
|
opencode plugin lancedb-opencode-pro -g # removes pin (if installed)
|
|
49
49
|
```
|
|
50
50
|
|
|
51
|
+
### Getting started
|
|
52
|
+
|
|
53
|
+
**1. Install and restart OpenCode** — done above. That's it for a baseline
|
|
54
|
+
setup: the plugin works with **zero configuration**.
|
|
55
|
+
|
|
56
|
+
**2. What you get out of the box, and what needs config:**
|
|
57
|
+
|
|
58
|
+
| Capability | Out of the box | Needs config to enhance |
|
|
59
|
+
|---|---|---|
|
|
60
|
+
| Recall | Works — falls back to pure BM25 if no embedder is reachable | **Embedding model** → semantic/hybrid vector search |
|
|
61
|
+
| Capture (session → memories) | Works — offline heuristic keyword capture | **LLM summary model** → LLM-quality extraction + abstractive digests |
|
|
62
|
+
| Digests (`memory_summarize` / `memory_expire`) | Extractive offline digests | Same LLM summary model → abstractive digests |
|
|
63
|
+
|
|
64
|
+
> **Nothing below is required** — every enhancement has an offline fallback.
|
|
65
|
+
> But configuring an embedding model makes recall dramatically better
|
|
66
|
+
> (semantic similarity instead of keyword-only), and configuring an LLM
|
|
67
|
+
> summary model makes captured memories higher quality and digests far more
|
|
68
|
+
> useful.
|
|
69
|
+
|
|
70
|
+
**3. (Optional) configure an embedding model.**
|
|
71
|
+
|
|
72
|
+
The plugin stores memories in a vector store; the embedding model decides how
|
|
73
|
+
well recall can find semantically related memories. Two options:
|
|
74
|
+
|
|
75
|
+
- **Local (no API key, no cost):** default — `ollama` +
|
|
76
|
+
`nomic-embed-text` at `http://127.0.0.1:11434`. Requires Ollama running.
|
|
77
|
+
- **OpenAI-compatible (hosted):** e.g. OpenAI, OpenRouter, or any endpoint
|
|
78
|
+
that serves the `/embeddings` API. Set `embedding.provider` to `"openai"`,
|
|
79
|
+
the model, the base URL, and an API key:
|
|
80
|
+
|
|
81
|
+
```json
|
|
82
|
+
{
|
|
83
|
+
"embedding": {
|
|
84
|
+
"provider": "openai",
|
|
85
|
+
"model": "openai/text-embedding-3-small",
|
|
86
|
+
"baseUrl": "https://openrouter.ai/api/v1",
|
|
87
|
+
"apiKey": "sk-..."
|
|
88
|
+
}
|
|
89
|
+
}
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
If the embedder is unreachable, recall falls back to pure BM25 over the FTS
|
|
93
|
+
index and capture still works — the plugin is offline-tolerant by design.
|
|
94
|
+
|
|
95
|
+
**4. (Optional) configure an LLM summary model** (for LLM-quality
|
|
96
|
+
capture/digests).
|
|
97
|
+
|
|
98
|
+
With `capture.mode: "llm"`, on session idle the plugin sends the session
|
|
99
|
+
buffer to an LLM (via an ephemeral OpenCode SDK session) which returns
|
|
100
|
+
structured memories, and digests become LLM-written abstractive summaries.
|
|
101
|
+
The LLM is addressed by **OpenCode provider + model IDs** — OpenCode owns
|
|
102
|
+
routing, auth, and base URLs, so no API key or baseUrl lives in the plugin
|
|
103
|
+
config. The provider must be resolvable in your `opencode.json`:
|
|
104
|
+
|
|
105
|
+
```json
|
|
106
|
+
{
|
|
107
|
+
"capture": {
|
|
108
|
+
"mode": "llm",
|
|
109
|
+
"llm": { "provider": "openrouter", "model": "z-ai/glm-5.3-flash" }
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
On any LLM failure, capture **falls back to heuristics** and records an
|
|
115
|
+
`llm-fallback` capture event — the plugin never breaks because the LLM is
|
|
116
|
+
unavailable.
|
|
117
|
+
|
|
118
|
+
**5. (Optional) start from the full annotated example** — the package includes
|
|
119
|
+
`opencode-memory-pro.example.json` with every option documented in-file. Copy
|
|
120
|
+
it to `~/.config/opencode/opencode-memory-pro.json` and edit:
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
cp node_modules/opencode-memory-pro/opencode-memory-pro.example.json ~/.config/opencode/opencode-memory-pro.json
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
**Healthy installs are never silent:** at startup the plugin logs a warning
|
|
127
|
+
if it detects missing pieces (e.g. `capture.mode: "llm"` without a resolvable
|
|
128
|
+
provider, or an OpenAI embedder without a key), and `memory_stats` reports the
|
|
129
|
+
same as `degradedFlags`, plus `llmHealth` — so you can always tell what's
|
|
130
|
+
running at full strength vs. degraded.
|
|
131
|
+
|
|
51
132
|
## Configuration
|
|
52
133
|
|
|
53
134
|
The sidecar file `opencode-memory-pro.json` is resolved from (first match
|
|
@@ -408,6 +489,44 @@ so your memories and graph carry over untouched.
|
|
|
408
489
|
|
|
409
490
|
## Changelog
|
|
410
491
|
|
|
492
|
+
### v1.4.0 (2026-09-06)
|
|
493
|
+
|
|
494
|
+
Dedup correctness overhaul — the write-time duplicate check compared against
|
|
495
|
+
the wrong score type, and the resulting flags were a one-way ratchet:
|
|
496
|
+
|
|
497
|
+
- **Write-time dedup now compares a raw cosine similarity**: the capture path
|
|
498
|
+
went through the hybrid `search()` API, whose RRF score is algebraically
|
|
499
|
+
`>= 1.0` for `limit: 1` (and up to `1.4` with importance) — so every capture
|
|
500
|
+
in a non-empty scope compared `>= 1.0` against `dedup.writeThreshold`
|
|
501
|
+
(clamped to `[0,1]`) and got falsely flagged as a potential duplicate.
|
|
502
|
+
`storeCapturedMemory` now calls `findSimilarVectors` (the same raw cosine
|
|
503
|
+
primitive consolidation measures) and compares that to the threshold.
|
|
504
|
+
Consequence: recall scores can no longer exceed 100%, and
|
|
505
|
+
`dedup.enabled`'s write-time detection actually detects.
|
|
506
|
+
- **False duplicate flags now self-correct**: `isPotentialDuplicate` was a
|
|
507
|
+
one-way ratchet — consolidation never cleared it, so `memory_stats`
|
|
508
|
+
`flaggedCount` only grew (153 flagged / 0 merged observed on a live store).
|
|
509
|
+
`consolidateDuplicates` now revalidates flags against the real cosine
|
|
510
|
+
threshold and clears (`isPotentialDuplicate`/`duplicateOf` removed) any
|
|
511
|
+
flagged row whose closest found neighbor never reaches the merge bar.
|
|
512
|
+
Returns `clearedFlags` so tools can report the correction.
|
|
513
|
+
- **Auto-consolidation cooldown is per-scope**: the shared
|
|
514
|
+
`lastConsolidateAt` timestamp meant the first scope to consolidate blocked
|
|
515
|
+
all other scopes for 30 minutes. Cooldowns are now tracked per scope
|
|
516
|
+
(same for the retention sweep, which had the identical flaw).
|
|
517
|
+
- **Scope cache staleness bound**: the per-process version counter can't see
|
|
518
|
+
writes from another opencode process sharing the same `dbPath`, so process A
|
|
519
|
+
could serve stale records indefinitely. Cache entries now reload after a
|
|
520
|
+
60s age bound even when the local version is unchanged (configurable via
|
|
521
|
+
`cache.staleAfterMs`; 0 restores pure version gating).
|
|
522
|
+
- **Consistent truncation warnings**: `deleteByIdForce`'s 100k-row fallback
|
|
523
|
+
scan and `pruneScope`'s 100k-row read now log a warning when the cap is hit,
|
|
524
|
+
matching `getCachedScopes`.
|
|
525
|
+
- **Tests**: three new integration tests — the dedup write-check primitive
|
|
526
|
+
returns cosine in `[0,1]` (plus a guard that the old RRF path still scores
|
|
527
|
+
`>= 1.0`), consolidation clears false flags, and the scope cache reloads
|
|
528
|
+
after the age bound when a second process writes behind its back.
|
|
529
|
+
|
|
411
530
|
### v1.3.8 (2026-09-06)
|
|
412
531
|
|
|
413
532
|
Fixes `memory_forget(force=true)` being unable to permanently delete a memory
|
package/dist/index.js
CHANGED
|
@@ -11,7 +11,7 @@ import { requestLLMCapture, isOwnSession } from "./llm.js";
|
|
|
11
11
|
import { createMemoryTools, createFeedbackTools, createEpisodicTools } from "./tools/index.js";
|
|
12
12
|
import { sweepExpiredMemories } from "./tools/memory.js";
|
|
13
13
|
import { createGraphStore } from "./graph.js";
|
|
14
|
-
const PLUGIN_VERSION = "1.
|
|
14
|
+
const PLUGIN_VERSION = "1.4.0";
|
|
15
15
|
const SCHEMA_VERSION = 1;
|
|
16
16
|
// Event-driven dedup: run consolidateDuplicates on session.idle (throttled to
|
|
17
17
|
// this interval so chatty sessions aren't re-scanning the store every turn)
|
|
@@ -138,6 +138,23 @@ const plugin = async (input) => {
|
|
|
138
138
|
if (!state.startupLogged) {
|
|
139
139
|
state.startupLogged = true;
|
|
140
140
|
log("info", `Plugin v${PLUGIN_VERSION} initialized`);
|
|
141
|
+
// STARTUP_DEGRADED (1.3.9): one proactive warning when the
|
|
142
|
+
// install can't reach full features, so fresh users know what
|
|
143
|
+
// to configure instead of discovering degraded mode later.
|
|
144
|
+
const missing = [];
|
|
145
|
+
const emb = state.config.embedding ?? {};
|
|
146
|
+
if (state.config.capture?.mode === "llm" && (!state.config.capture?.llm?.provider || !state.config.capture?.llm?.model)) {
|
|
147
|
+
missing.push("capture.llm.{provider,model} (LLM capture will fall back to heuristics)");
|
|
148
|
+
}
|
|
149
|
+
if (emb.provider === "openai" && !emb.apiKey) {
|
|
150
|
+
missing.push("embedding.apiKey (recall will fall back to BM25-only)");
|
|
151
|
+
}
|
|
152
|
+
if (emb.provider !== "openai" && !(emb.baseUrl ?? "")) {
|
|
153
|
+
missing.push("embedding.baseUrl (defaults to http://127.0.0.1:11434)");
|
|
154
|
+
}
|
|
155
|
+
if (missing.length > 0) {
|
|
156
|
+
log("warn", `Memory plugin running degraded: missing ${missing.join(", ")}. See README "Quick start" for full-feature setup.`);
|
|
157
|
+
}
|
|
141
158
|
}
|
|
142
159
|
},
|
|
143
160
|
event: async ({ event }) => {
|
|
@@ -580,12 +597,16 @@ async function createRuntimeState(input) {
|
|
|
580
597
|
activeEpisodes: new Map(),
|
|
581
598
|
sessionErrors: new Map(),
|
|
582
599
|
lastRecall: null,
|
|
600
|
+
// PER_SCOPE_COOLDOWN (1.4.0): cooldowns are per-scope (Map keyed by scope)
|
|
601
|
+
// instead of a single shared timestamp — one shared value meant the
|
|
602
|
+
// first scope to consolidate/sweep blocked every other scope for the
|
|
603
|
+
// whole 30-minute cooldown, even scopes that had never run.
|
|
583
604
|
consolidationInProgress: new Map(),
|
|
584
|
-
lastConsolidateAt:
|
|
605
|
+
lastConsolidateAt: new Map(),
|
|
585
606
|
// MEMORY_RETENTION (1.0): digest-then-hide expiry sweep state — same
|
|
586
607
|
// throttle pattern as consolidation (cooldown-gated, one per scope).
|
|
587
608
|
sweepInProgress: new Map(),
|
|
588
|
-
lastSweepAt:
|
|
609
|
+
lastSweepAt: new Map(),
|
|
589
610
|
ensureInitialized: async () => {
|
|
590
611
|
if (state.initialized)
|
|
591
612
|
return;
|
|
@@ -783,22 +804,19 @@ async function storeCapturedMemory(state, opts) {
|
|
|
783
804
|
}
|
|
784
805
|
let isPotentialDuplicate = false;
|
|
785
806
|
let duplicateOf = null;
|
|
807
|
+
// DEDUP_COSINE_CHECK (1.4.0): the write-time dedup check used to go
|
|
808
|
+
// through the hybrid search() API, whose RRF score is algebraically >= 1.0
|
|
809
|
+
// for limit:1 (rrfScore = 1/(rrfK+1) * (rrfK+1) == 1.0, then multiplied by
|
|
810
|
+
// an importance factor in [1, 1.4]) — so every capture with any same-dim
|
|
811
|
+
// record in the scope compared >= 1.0 against writeThreshold (clamped
|
|
812
|
+
// [0,1]) and got falsely flagged as a duplicate. Now it uses
|
|
813
|
+
// findSimilarVectors, which returns a raw cosine similarity in [0,1], the
|
|
814
|
+
// same primitive consolidateDuplicates measures against.
|
|
786
815
|
if (state.config.dedup.enabled) {
|
|
787
|
-
const similar = await state.store.
|
|
788
|
-
query: opts.text,
|
|
789
|
-
queryVector: vector,
|
|
790
|
-
scopes: [opts.scope],
|
|
791
|
-
limit: 1,
|
|
792
|
-
vectorWeight: 1.0,
|
|
793
|
-
bm25Weight: 0.0,
|
|
794
|
-
minScore: 0.0,
|
|
795
|
-
rrfK: 60,
|
|
796
|
-
recencyBoost: false,
|
|
797
|
-
globalDiscountFactor: 1.0,
|
|
798
|
-
});
|
|
816
|
+
const similar = await state.store.findSimilarVectors(vector, opts.scope, 1);
|
|
799
817
|
if (similar.length > 0 && similar[0].score >= state.config.dedup.writeThreshold) {
|
|
800
818
|
isPotentialDuplicate = true;
|
|
801
|
-
duplicateOf = similar[0].
|
|
819
|
+
duplicateOf = similar[0].id;
|
|
802
820
|
}
|
|
803
821
|
}
|
|
804
822
|
const memoryId = generateId();
|
|
@@ -849,11 +867,12 @@ async function maybeConsolidateDuplicates(state, scope, force = false) {
|
|
|
849
867
|
if (state.consolidationInProgress.get(scope))
|
|
850
868
|
return;
|
|
851
869
|
if (!force) {
|
|
852
|
-
const
|
|
870
|
+
const last = state.lastConsolidateAt.get(scope) ?? 0;
|
|
871
|
+
const elapsed = Date.now() - last;
|
|
853
872
|
if (elapsed < CONSOLIDATE_COOLDOWN_MS)
|
|
854
873
|
return;
|
|
855
874
|
}
|
|
856
|
-
state.lastConsolidateAt
|
|
875
|
+
state.lastConsolidateAt.set(scope, Date.now());
|
|
857
876
|
state.consolidationInProgress.set(scope, true);
|
|
858
877
|
state.store
|
|
859
878
|
.consolidateDuplicates(scope, state.config.dedup.consolidateThreshold, state.config.dedup.candidateLimit)
|
|
@@ -871,11 +890,12 @@ async function maybeSweepExpiredMemories(state, scope, force = false) {
|
|
|
871
890
|
if (state.sweepInProgress.get(scope))
|
|
872
891
|
return;
|
|
873
892
|
if (!force) {
|
|
874
|
-
const
|
|
893
|
+
const last = state.lastSweepAt.get(scope) ?? 0;
|
|
894
|
+
const elapsed = Date.now() - last;
|
|
875
895
|
if (elapsed < CONSOLIDATE_COOLDOWN_MS)
|
|
876
896
|
return;
|
|
877
897
|
}
|
|
878
|
-
state.lastSweepAt
|
|
898
|
+
state.lastSweepAt.set(scope, Date.now());
|
|
879
899
|
state.sweepInProgress.set(scope, true);
|
|
880
900
|
sweepExpiredMemories(state, { scope })
|
|
881
901
|
.then((result) => {
|
package/dist/llm.js
CHANGED
|
@@ -46,6 +46,30 @@ const OWN_SESSION_IDS = new Set();
|
|
|
46
46
|
export function isOwnSession(sessionID) {
|
|
47
47
|
return typeof sessionID === "string" && OWN_SESSION_IDS.has(sessionID);
|
|
48
48
|
}
|
|
49
|
+
// LLM_HEALTH (1.3.9): module-level runtime health for the capture/summary LLM,
|
|
50
|
+
// mirroring the embedder pattern so memory_stats can report both sides.
|
|
51
|
+
const globalLlmHealth = {
|
|
52
|
+
status: "never-called", // never-called | healthy | error
|
|
53
|
+
lastError: null,
|
|
54
|
+
lastSuccess: null,
|
|
55
|
+
errorCount: 0,
|
|
56
|
+
lastConfig: null,
|
|
57
|
+
};
|
|
58
|
+
export function getLlmHealth() {
|
|
59
|
+
return { ...globalLlmHealth };
|
|
60
|
+
}
|
|
61
|
+
export function setLlmHealth(patch) {
|
|
62
|
+
Object.assign(globalLlmHealth, patch);
|
|
63
|
+
}
|
|
64
|
+
export function resetLlmHealth() {
|
|
65
|
+
Object.assign(globalLlmHealth, {
|
|
66
|
+
status: "never-called",
|
|
67
|
+
lastError: null,
|
|
68
|
+
lastSuccess: null,
|
|
69
|
+
errorCount: 0,
|
|
70
|
+
lastConfig: null,
|
|
71
|
+
});
|
|
72
|
+
}
|
|
49
73
|
/**
|
|
50
74
|
* Tolerant JSON parse of the extraction model's reply. Accepts a bare array
|
|
51
75
|
* or an object wrapping an array under "memories"/"items"; strips markdown
|
|
@@ -169,6 +193,7 @@ export async function requestLLMDigest(client, llmConfig, texts, targetChars, gr
|
|
|
169
193
|
async function runEphemeralPrompt(client, llmConfig, system, userText, title) {
|
|
170
194
|
let sessionId = null;
|
|
171
195
|
try {
|
|
196
|
+
globalLlmHealth.lastConfig = { provider: llmConfig?.provider ?? null, model: llmConfig?.model ?? null };
|
|
172
197
|
const created = await client.session.create({
|
|
173
198
|
body: { title: `opencode-memory-pro ${title}` },
|
|
174
199
|
});
|
|
@@ -176,6 +201,7 @@ async function runEphemeralPrompt(client, llmConfig, system, userText, title) {
|
|
|
176
201
|
sessionId = createdPayload?.id;
|
|
177
202
|
if (!sessionId) {
|
|
178
203
|
log("warn", `[llm] ${title}: session.create did not return an id (got ${JSON.stringify(createdPayload)?.slice(0, 200)})`);
|
|
204
|
+
setLlmHealth({ status: "error", lastError: "session.create returned no id", lastSuccess: globalLlmHealth.lastSuccess, errorCount: globalLlmHealth.errorCount + 1 });
|
|
179
205
|
return null;
|
|
180
206
|
}
|
|
181
207
|
OWN_SESSION_IDS.add(sessionId);
|
|
@@ -191,12 +217,15 @@ async function runEphemeralPrompt(client, llmConfig, system, userText, title) {
|
|
|
191
217
|
const text = extractAssistantText(response);
|
|
192
218
|
if (!text) {
|
|
193
219
|
log("warn", `[llm] ${title}: session.prompt succeeded but returned no text parts (provider=${llmConfig.provider}, model=${llmConfig.model})`);
|
|
220
|
+
setLlmHealth({ status: "error", lastError: "session.prompt returned no text parts", lastSuccess: globalLlmHealth.lastSuccess, errorCount: globalLlmHealth.errorCount + 1 });
|
|
194
221
|
return null;
|
|
195
222
|
}
|
|
223
|
+
setLlmHealth({ status: "healthy", lastError: null, lastSuccess: Date.now(), errorCount: 0 });
|
|
196
224
|
return text;
|
|
197
225
|
}
|
|
198
226
|
catch (error) {
|
|
199
227
|
log("warn", `[llm] ${title}: ${error instanceof Error ? error.message : String(error)} (provider=${llmConfig.provider}, model=${llmConfig.model})`);
|
|
228
|
+
setLlmHealth({ status: "error", lastError: error instanceof Error ? error.message : String(error), lastSuccess: globalLlmHealth.lastSuccess, errorCount: globalLlmHealth.errorCount + 1 });
|
|
200
229
|
return null;
|
|
201
230
|
}
|
|
202
231
|
finally {
|
package/dist/store.js
CHANGED
|
@@ -10,6 +10,13 @@ const DEFAULT_CACHE_CONFIG = {
|
|
|
10
10
|
maxScopes: 10,
|
|
11
11
|
maxRecordsPerScope: 1000,
|
|
12
12
|
enabled: true,
|
|
13
|
+
// SCOPE_CACHE_STALENESS (1.4.0): the version counter only sees THIS
|
|
14
|
+
// process's writes, so when two opencode processes share one dbPath the
|
|
15
|
+
// scope cache could serve stale records forever. A modest age-based
|
|
16
|
+
// staleness bound forces a reload after staleAfterMs even when the local
|
|
17
|
+
// version is unchanged, bounding cross-process staleness without a schema
|
|
18
|
+
// change. 0 disables the age check (pure version gating, pre-1.4.0).
|
|
19
|
+
staleAfterMs: 60 * 1000,
|
|
13
20
|
};
|
|
14
21
|
// ANN_TUNABLES (1.3.0): nprobes controls IVF recall-vs-latency on filtered
|
|
15
22
|
// vector searches; the consolidation query batch controls how many ANN
|
|
@@ -642,6 +649,9 @@ export class MemoryStore {
|
|
|
642
649
|
}
|
|
643
650
|
const table = this.requireTable();
|
|
644
651
|
const rows = await table.query().limit(100000).toArray();
|
|
652
|
+
if (rows.length === 100000) {
|
|
653
|
+
log("warn", "[store] deleteByIdForce fallback scan hit the 100000-row cap; the target may not be found if it lives beyond the cap");
|
|
654
|
+
}
|
|
645
655
|
const match = rows.find((row) => this.matchesId(row.id, id));
|
|
646
656
|
if (!match)
|
|
647
657
|
return false;
|
|
@@ -711,6 +721,9 @@ export class MemoryStore {
|
|
|
711
721
|
}
|
|
712
722
|
async pruneScope(scope, maxEntries) {
|
|
713
723
|
const rows = await this.list(scope, 100000);
|
|
724
|
+
if (rows.length === 100000) {
|
|
725
|
+
log("warn", `[store] pruneScope scanned up to the 100000-row cap for scope=${scope}; entries older than the newest 100k are not candidates for pruning`);
|
|
726
|
+
}
|
|
714
727
|
if (rows.length <= maxEntries)
|
|
715
728
|
return 0;
|
|
716
729
|
const flagged = rows.filter((r) => {
|
|
@@ -748,7 +761,7 @@ export class MemoryStore {
|
|
|
748
761
|
let rows = await this.readByScopesIncludingMerged([scope]);
|
|
749
762
|
rows = rows.filter((r) => r.status === undefined || r.status === null || r.status === "" || r.status === "active");
|
|
750
763
|
if (rows.length === 0) {
|
|
751
|
-
return { mergedPairs: 0, updatedRecords: 0, skippedRecords: 0 };
|
|
764
|
+
return { mergedPairs: 0, updatedRecords: 0, skippedRecords: 0, clearedFlags: 0 };
|
|
752
765
|
}
|
|
753
766
|
const BATCH_SIZE = 100;
|
|
754
767
|
const FALLBACK_THRESHOLD = 500;
|
|
@@ -763,12 +776,23 @@ export class MemoryStore {
|
|
|
763
776
|
row,
|
|
764
777
|
norm: this.scopeCache.get(scope)?.norms.get(row.id) ?? vecNorm(row.vector),
|
|
765
778
|
}));
|
|
779
|
+
// DEDUP_FLAG_REVALIDATION (1.4.0): the write-time dedup check used to
|
|
780
|
+
// flag nearly every capture (RRF score >= 1.0 vs writeThreshold in
|
|
781
|
+
// [0,1]), and the flag was a one-way ratchet — nothing ever cleared it,
|
|
782
|
+
// so flaggedCount only grew. Consolidation is where a real cosine
|
|
783
|
+
// comparison happens, so flagged rows whose closest found neighbor
|
|
784
|
+
// stays below the consolidate threshold get the flag cleared; rows
|
|
785
|
+
// that DO have a near-duplicate keep it.
|
|
786
|
+
const metaById = new Map(rowsWithNorms.map(({ row }) => [row.id, parseMetadata(row.metadataJson)]));
|
|
787
|
+
const flaggedIds = new Set([...metaById].filter(([, meta]) => meta.isPotentialDuplicate === true).map(([id]) => id));
|
|
788
|
+
const bestSimByFlagged = new Map();
|
|
789
|
+
const mergedIds = new Set();
|
|
790
|
+
let clearedFlags = 0;
|
|
766
791
|
log("debug", `[consolidate] scope=${scope} rows=${rows.length} threshold=${threshold} candidateLimit=${candidateLimit} batchSize=${BATCH_SIZE} fallbackThreshold=${FALLBACK_THRESHOLD}`);
|
|
767
792
|
const processWithANN = async () => {
|
|
768
793
|
let localMerged = 0;
|
|
769
794
|
let localUpdated = 0;
|
|
770
795
|
let localSkipped = 0;
|
|
771
|
-
const mergedIds = new Set();
|
|
772
796
|
const totalChunks = Math.ceil(rowsWithNorms.length / BATCH_SIZE);
|
|
773
797
|
for (let chunkIdx = 0; chunkIdx < totalChunks; chunkIdx++) {
|
|
774
798
|
const chunkStart = chunkIdx * BATCH_SIZE;
|
|
@@ -805,6 +829,12 @@ export class MemoryStore {
|
|
|
805
829
|
if (mergedIds.has(b.row.id))
|
|
806
830
|
continue;
|
|
807
831
|
const sim = storeFastCosine(a.row.vector, b.row.vector, a.norm, b.norm);
|
|
832
|
+
if (flaggedIds.has(a.row.id)) {
|
|
833
|
+
bestSimByFlagged.set(a.row.id, Math.max(bestSimByFlagged.get(a.row.id) ?? -1, sim));
|
|
834
|
+
}
|
|
835
|
+
if (flaggedIds.has(b.row.id)) {
|
|
836
|
+
bestSimByFlagged.set(b.row.id, Math.max(bestSimByFlagged.get(b.row.id) ?? -1, sim));
|
|
837
|
+
}
|
|
808
838
|
if (sim < threshold)
|
|
809
839
|
continue;
|
|
810
840
|
const aMeta = parseMetadata(a.row.metadataJson);
|
|
@@ -879,7 +909,6 @@ export class MemoryStore {
|
|
|
879
909
|
let localMerged = 0;
|
|
880
910
|
let localUpdated = 0;
|
|
881
911
|
let localSkipped = 0;
|
|
882
|
-
const mergedIds = new Set();
|
|
883
912
|
for (let i = 0; i < rowsWithNorms.length; i += 1) {
|
|
884
913
|
const a = rowsWithNorms[i];
|
|
885
914
|
if (mergedIds.has(a.row.id))
|
|
@@ -889,6 +918,12 @@ export class MemoryStore {
|
|
|
889
918
|
if (mergedIds.has(b.row.id))
|
|
890
919
|
continue;
|
|
891
920
|
const sim = storeFastCosine(a.row.vector, b.row.vector, a.norm, b.norm);
|
|
921
|
+
if (flaggedIds.has(a.row.id)) {
|
|
922
|
+
bestSimByFlagged.set(a.row.id, Math.max(bestSimByFlagged.get(a.row.id) ?? -1, sim));
|
|
923
|
+
}
|
|
924
|
+
if (flaggedIds.has(b.row.id)) {
|
|
925
|
+
bestSimByFlagged.set(b.row.id, Math.max(bestSimByFlagged.get(b.row.id) ?? -1, sim));
|
|
926
|
+
}
|
|
892
927
|
if (sim < threshold)
|
|
893
928
|
continue;
|
|
894
929
|
const aMeta = parseMetadata(a.row.metadataJson);
|
|
@@ -955,14 +990,33 @@ export class MemoryStore {
|
|
|
955
990
|
}
|
|
956
991
|
else {
|
|
957
992
|
log("warn", `[consolidate] Skipping fallback for large scope (${rows.length} >= ${FALLBACK_THRESHOLD})`);
|
|
958
|
-
return { mergedPairs: 0, updatedRecords: 0, skippedRecords: 0 };
|
|
993
|
+
return { mergedPairs: 0, updatedRecords: 0, skippedRecords: 0, clearedFlags: 0 };
|
|
959
994
|
}
|
|
960
995
|
}
|
|
961
|
-
|
|
996
|
+
// DEDUP_FLAG_REVALIDATION (1.4.0): clear false duplicate flags. Rows
|
|
997
|
+
// whose best found neighbor never reached the merge threshold were
|
|
998
|
+
// flagged by the pre-1.4.0 RRF write-check (or carry a flag made stale
|
|
999
|
+
// by later edits); unsetting isPotentialDuplicate lets flaggedCount
|
|
1000
|
+
// self-correct instead of ratcheting up forever.
|
|
1001
|
+
for (const [id, bestSim] of bestSimByFlagged) {
|
|
1002
|
+
if (mergedIds.has(id) || bestSim >= threshold)
|
|
1003
|
+
continue;
|
|
1004
|
+
const meta = metaById.get(id);
|
|
1005
|
+
if (!meta || meta.isPotentialDuplicate !== true)
|
|
1006
|
+
continue;
|
|
1007
|
+
delete meta.isPotentialDuplicate;
|
|
1008
|
+
delete meta.duplicateOf;
|
|
1009
|
+
await this.requireTable().update({
|
|
1010
|
+
where: `id = '${escapeSql(id)}'`,
|
|
1011
|
+
values: { metadataJson: JSON.stringify(meta) },
|
|
1012
|
+
});
|
|
1013
|
+
clearedFlags += 1;
|
|
1014
|
+
}
|
|
1015
|
+
if (mergedPairs > 0 || clearedFlags > 0) {
|
|
962
1016
|
this.invalidateScope(scope);
|
|
963
1017
|
}
|
|
964
1018
|
await this.maybeOptimizeAll(false);
|
|
965
|
-
return { mergedPairs, updatedRecords, skippedRecords };
|
|
1019
|
+
return { mergedPairs, updatedRecords, skippedRecords, clearedFlags };
|
|
966
1020
|
}
|
|
967
1021
|
// ANN_CONSOLIDATION (1.1.7): previously this did
|
|
968
1022
|
// query().where(scope).limit(limit).toArray() — which returns the FIRST N
|
|
@@ -1647,7 +1701,9 @@ export class MemoryStore {
|
|
|
1647
1701
|
for (const scope of scopes) {
|
|
1648
1702
|
const currentVersion = this.scopeVersions.get(scope) ?? 0;
|
|
1649
1703
|
let entry = this.scopeCache.get(scope);
|
|
1650
|
-
|
|
1704
|
+
const maxAgeMs = Number.isFinite(this.cacheConfig.staleAfterMs) ? this.cacheConfig.staleAfterMs : 0;
|
|
1705
|
+
const staleByAge = maxAgeMs > 0 && (entry ? Date.now() - (entry.loadedAt ?? entry.lastAccessTimestamp) > maxAgeMs : false);
|
|
1706
|
+
if (!entry || entry.version !== currentVersion || staleByAge) {
|
|
1651
1707
|
if (entry) {
|
|
1652
1708
|
this.cacheStats.evictions++;
|
|
1653
1709
|
}
|
|
@@ -1667,7 +1723,7 @@ export class MemoryStore {
|
|
|
1667
1723
|
for (const record of sortedRecords) {
|
|
1668
1724
|
norms.set(record.id, vecNorm(record.vector));
|
|
1669
1725
|
}
|
|
1670
|
-
entry = { records: sortedRecords, tokenized, idf, norms, lastAccessTimestamp: Date.now(), version: currentVersion };
|
|
1726
|
+
entry = { records: sortedRecords, tokenized, idf, norms, loadedAt: Date.now(), lastAccessTimestamp: Date.now(), version: currentVersion };
|
|
1671
1727
|
this.scopeCache.set(scope, entry);
|
|
1672
1728
|
this.cacheStats.misses++;
|
|
1673
1729
|
this.enforceMaxScopes();
|
package/dist/tools/memory.js
CHANGED
|
@@ -4,10 +4,36 @@ import { generateId } from "../utils.js";
|
|
|
4
4
|
import { getEmbedderHealth } from "../embedder.js";
|
|
5
5
|
import { extractiveDigest, retentionCandidates } from "../store.js";
|
|
6
6
|
import { requestLLMDigest } from "../llm.js";
|
|
7
|
+
import { getLlmHealth } from "../llm.js";
|
|
7
8
|
import { log } from "../logger.js";
|
|
8
9
|
function unavailableMessage(provider) {
|
|
9
10
|
return `Memory store unavailable (${provider} embedding may be offline). Will retry automatically.`;
|
|
10
11
|
}
|
|
12
|
+
// DEGRADED_FLAGS (1.3.9): report what the user is missing for full features.
|
|
13
|
+
// Returns a list of human-readable strings, empty when running at full strength.
|
|
14
|
+
function computeDegradedFlags(state, embedderHealth, graphStats) {
|
|
15
|
+
const flags = [];
|
|
16
|
+
const emb = state.config?.embedding ?? {};
|
|
17
|
+
if (state.config?.capture?.mode === "llm") {
|
|
18
|
+
const cap = state.config.capture;
|
|
19
|
+
if (!cap?.llm?.provider || !cap?.llm?.model) {
|
|
20
|
+
flags.push("llm-capture-unconfigured: capture.mode=llm but capture.llm.provider/model is missing — capture will fall back to heuristics");
|
|
21
|
+
}
|
|
22
|
+
}
|
|
23
|
+
if (emb.provider === "openai" && !emb.apiKey) {
|
|
24
|
+
flags.push("embedding-api-key-missing: embedding.provider=openai but no apiKey (or OPENCODE_MEMORY_PRO_OPENAI_API_KEY) is set — recall will fall back to BM25-only");
|
|
25
|
+
}
|
|
26
|
+
if (emb.provider !== "openai" && !(emb.baseUrl ?? "")) {
|
|
27
|
+
flags.push("embedding-baseurl-missing: embedding.provider=ollama but no baseUrl (defaults to http://127.0.0.1:11434) — recall will fall back to BM25-only");
|
|
28
|
+
}
|
|
29
|
+
if (graphStats && graphStats.enabled === false && state.config?.graph?.enabled) {
|
|
30
|
+
flags.push("graph-disabled: graph.enabled=true but the graph store did not initialize (check graph.dbPath)");
|
|
31
|
+
}
|
|
32
|
+
if (state.config?.capture?.llm?.provider && state.config?.capture?.llm?.model && getLlmHealth().status === "error") {
|
|
33
|
+
flags.push("llm-unhealthy: last LLM capture/digest call failed — falling back to heuristics/extractive digests");
|
|
34
|
+
}
|
|
35
|
+
return flags;
|
|
36
|
+
}
|
|
11
37
|
// LLM_CAPTURE (1.1): mode-aware digest builder shared by memory_summarize
|
|
12
38
|
// and the retention sweep. capture.mode === "llm" → abstractive LLM digest
|
|
13
39
|
// via an ephemeral SDK session (falls back to the offline extractive digest
|
|
@@ -246,6 +272,7 @@ export function createMemoryTools(state) {
|
|
|
246
272
|
const incompatibleVectors = await state.store.countIncompatibleVectors(buildScopeFilter(scope, state.config.includeGlobalScope), await state.embedder.dim());
|
|
247
273
|
const health = state.store.getIndexHealth();
|
|
248
274
|
const embedderHealth = getEmbedderHealth();
|
|
275
|
+
const llmHealth = getLlmHealth();
|
|
249
276
|
const searchMode = embedderHealth.fallbackActive ? "bm25-only" : state.config.retrieval.mode;
|
|
250
277
|
const eventTtl = state.config.retention
|
|
251
278
|
? await state.store.getEventTtlStatus()
|
|
@@ -276,9 +303,19 @@ export function createMemoryTools(state) {
|
|
|
276
303
|
embeddingModel: state.config.embedding.model,
|
|
277
304
|
searchMode,
|
|
278
305
|
embedderHealth,
|
|
306
|
+
capture: {
|
|
307
|
+
mode: state.config.capture?.mode ?? "heuristics",
|
|
308
|
+
llm: {
|
|
309
|
+
provider: state.config.capture?.llm?.provider ?? null,
|
|
310
|
+
model: state.config.capture?.llm?.model ?? null,
|
|
311
|
+
configured: Boolean(state.config.capture?.llm?.provider && state.config.capture?.llm?.model),
|
|
312
|
+
},
|
|
313
|
+
llmHealth,
|
|
314
|
+
},
|
|
279
315
|
eventTtl,
|
|
280
316
|
graph: graphStats,
|
|
281
317
|
memoryRetention,
|
|
318
|
+
degradedFlags: computeDegradedFlags(state, embedderHealth, graphStats),
|
|
282
319
|
}, null, 2);
|
|
283
320
|
},
|
|
284
321
|
}),
|
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
{
|
|
2
|
+
"_comment": "opencode-memory-pro example configuration. Copy this file to ~/.config/opencode/opencode-memory-pro.json (or ~/.opencode/opencode-memory-pro.json) and edit. Every key is optional and matches the built-in defaults; delete anything you don't need. The plugin works with zero configuration — these options only tune/enhance it.",
|
|
3
|
+
"provider": "opencode-memory-pro",
|
|
4
|
+
"dbPath": "~/.opencode/memory/lancedb",
|
|
5
|
+
"embedding": {
|
|
6
|
+
"_comment": "Vector search / hybrid recall. Default: local Ollama, no key needed. For OpenAI-compatible endpoints (OpenAI, OpenRouter, etc.), set provider to \"openai\", add baseUrl + model + apiKey. If no embedder is reachable, recall falls back to pure BM25.",
|
|
7
|
+
"provider": "ollama",
|
|
8
|
+
"model": "nomic-embed-text",
|
|
9
|
+
"baseUrl": "http://127.0.0.1:11434",
|
|
10
|
+
"timeoutMs": 6000,
|
|
11
|
+
"retry": {
|
|
12
|
+
"enabled": true,
|
|
13
|
+
"maxAttempts": 3,
|
|
14
|
+
"initialDelayMs": 1000,
|
|
15
|
+
"backoffMultiplier": 2
|
|
16
|
+
}
|
|
17
|
+
},
|
|
18
|
+
"retrieval": {
|
|
19
|
+
"mode": "hybrid",
|
|
20
|
+
"vectorWeight": 0.7,
|
|
21
|
+
"bm25Weight": 0.3,
|
|
22
|
+
"minScore": 0.2,
|
|
23
|
+
"rrfK": 60,
|
|
24
|
+
"recencyBoost": true,
|
|
25
|
+
"recencyHalfLifeHours": 72,
|
|
26
|
+
"importanceWeight": 0.4,
|
|
27
|
+
"feedbackWeight": 0.3
|
|
28
|
+
},
|
|
29
|
+
"injection": {
|
|
30
|
+
"mode": "fixed",
|
|
31
|
+
"maxMemories": 3,
|
|
32
|
+
"minMemories": 1,
|
|
33
|
+
"budgetTokens": 4096,
|
|
34
|
+
"maxCharsPerMemory": 1200,
|
|
35
|
+
"summarization": "none",
|
|
36
|
+
"summaryTargetChars": 300,
|
|
37
|
+
"scoreDropTolerance": 0.15,
|
|
38
|
+
"injectionFloor": 0.2,
|
|
39
|
+
"codeSummarization": {
|
|
40
|
+
"enabled": true,
|
|
41
|
+
"pureCodeThreshold": 500,
|
|
42
|
+
"maxCodeLines": 15,
|
|
43
|
+
"codeTruncationMode": "smart",
|
|
44
|
+
"preserveComments": true,
|
|
45
|
+
"preserveImports": false
|
|
46
|
+
},
|
|
47
|
+
"taskTypeProfiles": {
|
|
48
|
+
"coding": {
|
|
49
|
+
"maxMemories": 4,
|
|
50
|
+
"budgetTokens": 5120,
|
|
51
|
+
"summaryTargetChars": 400,
|
|
52
|
+
"categoryWeights": {
|
|
53
|
+
"decision": 1.5,
|
|
54
|
+
"entity": 1.2,
|
|
55
|
+
"fact": 1,
|
|
56
|
+
"preference": 0.8,
|
|
57
|
+
"other": 0.5
|
|
58
|
+
}
|
|
59
|
+
},
|
|
60
|
+
"documentation": {
|
|
61
|
+
"maxMemories": 3,
|
|
62
|
+
"budgetTokens": 3072,
|
|
63
|
+
"summaryTargetChars": 500,
|
|
64
|
+
"categoryWeights": {
|
|
65
|
+
"decision": 1.4,
|
|
66
|
+
"fact": 1.3,
|
|
67
|
+
"entity": 1.2,
|
|
68
|
+
"preference": 0.8,
|
|
69
|
+
"other": 0.5
|
|
70
|
+
}
|
|
71
|
+
},
|
|
72
|
+
"review": {
|
|
73
|
+
"maxMemories": 3,
|
|
74
|
+
"budgetTokens": 4096,
|
|
75
|
+
"summaryTargetChars": 300,
|
|
76
|
+
"categoryWeights": {
|
|
77
|
+
"preference": 1.4,
|
|
78
|
+
"decision": 1.2,
|
|
79
|
+
"entity": 1,
|
|
80
|
+
"fact": 0.9,
|
|
81
|
+
"other": 0.5
|
|
82
|
+
}
|
|
83
|
+
},
|
|
84
|
+
"release": {
|
|
85
|
+
"maxMemories": 4,
|
|
86
|
+
"budgetTokens": 6144,
|
|
87
|
+
"summaryTargetChars": 350,
|
|
88
|
+
"categoryWeights": {
|
|
89
|
+
"decision": 1.5,
|
|
90
|
+
"entity": 1.3,
|
|
91
|
+
"fact": 1.2,
|
|
92
|
+
"preference": 0.8,
|
|
93
|
+
"other": 0.5
|
|
94
|
+
}
|
|
95
|
+
},
|
|
96
|
+
"general": {
|
|
97
|
+
"maxMemories": 3,
|
|
98
|
+
"budgetTokens": 4096,
|
|
99
|
+
"summaryTargetChars": 300,
|
|
100
|
+
"categoryWeights": {
|
|
101
|
+
"decision": 1.3,
|
|
102
|
+
"fact": 1,
|
|
103
|
+
"entity": 1,
|
|
104
|
+
"preference": 0.9,
|
|
105
|
+
"other": 0.5
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
},
|
|
110
|
+
"dedup": {
|
|
111
|
+
"enabled": true,
|
|
112
|
+
"writeThreshold": 0.92,
|
|
113
|
+
"consolidateThreshold": 0.95,
|
|
114
|
+
"candidateLimit": 50
|
|
115
|
+
},
|
|
116
|
+
"graph": {
|
|
117
|
+
"_comment": "Offline entity graph: co-occurrence + typed relations (uses/depends_on/...), powers [graph+X%] recall boosts and BFS expansion. No LLM needed.",
|
|
118
|
+
"enabled": true,
|
|
119
|
+
"dbPath": "~/.opencode/memory/graph.db",
|
|
120
|
+
"boostLambda": 0.3,
|
|
121
|
+
"maxEntitiesPerMemory": 20,
|
|
122
|
+
"maxEdgeProvenance": 20,
|
|
123
|
+
"typedEdges": true,
|
|
124
|
+
"expansionEnabled": true,
|
|
125
|
+
"maxHops": 2,
|
|
126
|
+
"expansionLimit": 5,
|
|
127
|
+
"expansionLambda": 0.3
|
|
128
|
+
},
|
|
129
|
+
"summarize": {
|
|
130
|
+
"enabled": true,
|
|
131
|
+
"minAgeDays": 30,
|
|
132
|
+
"minGroupSize": 3,
|
|
133
|
+
"targetChars": 500,
|
|
134
|
+
"replace": false
|
|
135
|
+
},
|
|
136
|
+
"capture": {
|
|
137
|
+
"_comment": "\"heuristics\" (default) = offline keyword capture, no LLM needed. \"llm\" = LLM-quality extraction + abstractive digests, addressed via OpenCode provider/model IDs — the provider must be resolvable in your opencode.json (e.g. \"openrouter\"). Falls back to heuristics on any failure.",
|
|
138
|
+
"mode": "heuristics",
|
|
139
|
+
"llm": {
|
|
140
|
+
"provider": "openrouter",
|
|
141
|
+
"model": "z-ai/glm-5.3-flash"
|
|
142
|
+
}
|
|
143
|
+
},
|
|
144
|
+
"scoping": "global",
|
|
145
|
+
"includeGlobalScope": true,
|
|
146
|
+
"globalDetectionThreshold": 2,
|
|
147
|
+
"globalDiscountFactor": 0.7,
|
|
148
|
+
"unusedDaysThreshold": 30,
|
|
149
|
+
"minCaptureChars": 80,
|
|
150
|
+
"maxEntriesPerScope": 3000,
|
|
151
|
+
"retention": {
|
|
152
|
+
"effectivenessEventsDays": 90,
|
|
153
|
+
"memory": {
|
|
154
|
+
"enabled": true,
|
|
155
|
+
"unusedDays": 60,
|
|
156
|
+
"minAgeDays": 180,
|
|
157
|
+
"minGroupSize": 2,
|
|
158
|
+
"targetChars": 500,
|
|
159
|
+
"minImportance": 0.3,
|
|
160
|
+
"protectedCategories": [
|
|
161
|
+
"digest"
|
|
162
|
+
]
|
|
163
|
+
}
|
|
164
|
+
},
|
|
165
|
+
"logging": {
|
|
166
|
+
"level": "info",
|
|
167
|
+
"file": null
|
|
168
|
+
}
|
|
169
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "opencode-memory-pro",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.4.0",
|
|
4
4
|
"description": "LanceDB-backed long-term memory provider for OpenCode — standalone fork of lancedb-opencode-pro with entity graph, lifecycle, and retention",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -15,7 +15,8 @@
|
|
|
15
15
|
"files": [
|
|
16
16
|
"dist",
|
|
17
17
|
"README.md",
|
|
18
|
-
"LICENSE"
|
|
18
|
+
"LICENSE",
|
|
19
|
+
"opencode-memory-pro.example.json"
|
|
19
20
|
],
|
|
20
21
|
"keywords": [
|
|
21
22
|
"opencode",
|