plugmem 0.4.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +527 -196
  2. package/index.d.ts +231 -4
  3. package/index.js +3 -1
  4. package/package.json +8 -8
package/README.md CHANGED
@@ -1,20 +1,26 @@
1
- # plugmem-napi
1
+ # plugmem
2
2
 
3
3
  > ⚠️ Experimental. plugmem is mostly an AI-built experiment — written with
4
4
  > the help of a small local model (Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf) and various
5
5
  > Claude models, in roughly equal measure. Expect non-professional design
6
6
  > choices, rough edges, broken behavior, or mistakes. Use it at your own risk.
7
7
 
8
- `plugmem-napi` is the **native Node.js addon** for the plugmem
9
- [temporal-memory engine](https://docs.rs/plugmem-core/latest) it embeds
10
- [`plugmem-host`](https://docs.rs/plugmem-host/latest) **in the Node process**
11
- (real mmap, file locking, cross-process MVCC — the whole engine, unchanged) and
12
- exposes it to **JavaScript / TypeScript** as a `Plugmem` class. It is published
13
- to npm as **`plugmem`**.
8
+ An embeddable bitemporal memory database for local-first applications and
9
+ agents, embedded in your Node process. It stores short facts and answers a
10
+ query with ranked facts and edges plus an optional bounded rendered block.
14
11
 
15
- Because it is native (not WebAssembly), there is no whole-file-in-RAM copy and no
16
- 4 GiB ceiling: the OS pages the snapshot in and out exactly as it does for the
17
- Rust library. It loads in **Node, and any N-API host (Deno, Bun)**.
12
+ File-backed on disk, no server, no daemon. It links into your process the way
13
+ SQLite does: the engine is [`plugmem-host`](https://docs.rs/plugmem-host/latest)
14
+ compiled to a native addon through [napi-rs](https://napi.rs), so there is no
15
+ WebAssembly copy of the file in RAM and no 4 GiB ceiling. Runs on Node, Deno and
16
+ Bun.
17
+
18
+ **Contents:** [Install](#install) · [Quick start](#quick-start) ·
19
+ [What it stores](#what-it-stores) · [Two clocks](#two-clocks) ·
20
+ [How recall works](#how-recall-works) · [API](#api) · [Errors](#errors) ·
21
+ [Configuration](#configuration-and-embeddings) ·
22
+ [Async](#async-and-the-event-loop) · [Many memories](#many-memories-in-one-directory) ·
23
+ [What it is not for](#what-it-is-not-for)
18
24
 
19
25
  ## Install
20
26
 
@@ -22,115 +28,324 @@ Rust library. It loads in **Node, and any N-API host (Deno, Bun)**.
22
28
  $ npm install plugmem
23
29
  ```
24
30
 
25
- `npm i plugmem` pulls the meta package, which through `optionalDependencies`
26
- installs only the prebuilt binary for your platform one of
27
- `plugmem-{linux-x64-gnu, linux-arm64-gnu, darwin-x64, darwin-arm64,
28
- win32-x64-msvc, win32-arm64-msvc}`. No toolchain, no build step.
31
+ That pulls a meta package which, through `optionalDependencies`, installs only
32
+ the prebuilt binary for your platform: one of `plugmem-{linux-x64-gnu,
33
+ linux-arm64-gnu, darwin-x64, darwin-arm64, win32-x64-msvc, win32-arm64-msvc}`.
34
+ No toolchain, no build step.
35
+
36
+ ## Quick start
37
+
38
+ ```typescript
39
+ import { Plugmem } from "plugmem";
40
+
41
+ const db = await Plugmem.open("agent.plugmem");
42
+
43
+ await db.remember({ text: "the user prefers tokio", entity: "user", tags: ["pref"] });
44
+ await db.remember({ text: "the release ships on friday", entity: "release" });
45
+
46
+ const res = await db.recall({ query: "which runtime?", k: 5 });
47
+ console.log(res.rendered); // paste this into the prompt
48
+ // - [f0] user: the user prefers tokio (2026-08; active)
49
+
50
+ db.close();
51
+ ```
52
+
53
+ `Plugmem.open` is a static method, not a constructor, because opening replays a
54
+ journal and maps a snapshot — work proportional to the file — and a JavaScript
55
+ constructor has no way to hand that to a worker thread. Everything is typed:
56
+ `index.d.ts` is generated from the Rust, so a TypeScript host gets real
57
+ autocomplete on arguments and results.
58
+
59
+ ## What it stores
60
+
61
+ One **fact** is one statement. It carries:
62
+
63
+ | Field | Meaning |
64
+ |---|---|
65
+ | `text` | the statement itself, and what lexical search indexes |
66
+ | `entity` | the subject, by name — created on first mention, shared across facts |
67
+ | `tags` | filters, not ranking: a query asking for a tag requires it |
68
+ | `metadata` | an opaque `Record<string,string>`. The engine stores and returns it and never looks inside — use it for a URI to the real payload elsewhere, a mime type, an external key |
69
+ | `vector` | an optional embedding; supply your own, or let a configured embedder produce it |
70
+ | `validFrom` | when the statement became true |
71
+
72
+ Entities are joined by **typed edges**: `link({ src: "ann", rel: "hires", dst: "bob" })`.
73
+ An edge can name the fact it follows from (`provenance`), so a later reader can
74
+ answer "why does the memory think this" instead of trusting a bare relationship.
75
+
76
+ Facts are never rewritten in place. `revise` closes the old one and chains a
77
+ successor; `forget` tombstones a fact and the next `maintain` erases it from
78
+ disk. `unlink` closes an edge the same way `revise` closes a fact.
29
79
 
30
- ## Which door is this?
80
+ ## Two clocks
31
81
 
32
- plugmem is **embedded-first, like SQLite**. Pick the door for your language:
82
+ This is the part worth reading, because it is the one thing that behaves
83
+ differently from every other store.
33
84
 
34
- | You are… | Use | Why |
35
- |---|---|---|
36
- | **writing JavaScript / TypeScript for Node** | **`plugmem-napi`** (this, npm `plugmem`) | The engine *in your Node process*, native speed, typed for TS. |
37
- | **writing Rust** | [`plugmem-host`](https://docs.rs/plugmem-host/latest) | The engine in your process, like linking SQLite. |
38
- | **an agent, or another language** (Python, Go…) | [`plugmem-mcp`](https://docs.rs/plugmem-mcp/latest) | A long-lived stdio JSON-RPC sidecar; language-independent. |
39
- | a person at a **terminal / script** | [`plugmem-cli`](https://docs.rs/plugmem-cli/latest) | The human door. |
85
+ Every fact carries **two** timestamps, not one:
40
86
 
41
- So: **Node/TS napi; Rust host; an agent or other language → MCP; a human →
42
- the CLI.**
87
+ - **`validFrom` / `validTo`** when the statement was true.
88
+ - **`recordedAt`** — when the memory learned it. Set by the engine, never by you.
43
89
 
44
- ## Usage (TypeScript)
90
+ They are different questions, and one timestamp cannot hold both.
45
91
 
46
- Every argument and result is typed napi generates `index.d.ts`, so a TS host
47
- gets full autocomplete and checking. A memory is opened with the static
48
- `Plugmem.open`, not `new`: opening replays a journal and maps a snapshot, and a
49
- JavaScript constructor has no way to hand that to a worker thread.
92
+ Unlike the Rust library, this binding reads the system clock on every call — you
93
+ never pass `now` so `recordedAt` is always the moment of the write:
50
94
 
51
95
  ```typescript
52
- import { Plugmem } from "plugmem";
96
+ await db.remember({ text: "lives in Moscow", entity: "kim" });
97
+ const between = Date.now();
98
+ await db.revise(0, { text: "lives in Berlin", entity: "kim" });
53
99
 
54
- const db = await Plugmem.open("agent.plugmem"); // or { readOnly: true }
100
+ (await db.recall({ entities: ["kim"] })).rendered;
101
+ // - [f1] kim: lives in Berlin (2026-08; active)
55
102
 
56
- const out = await db.remember({
57
- text: "prefers tokio",
58
- entity: "user",
59
- tags: ["pref"],
60
- links: [{ rel: "works_at", entity: "acme" }],
61
- metadata: { source: "chat", uri: "s3://bucket/note.txt" }, // opaque; a pointer
62
- });
63
- out.id; // number
64
- out.similar; // Similar[] — the engine surfaces conflicts, you decide
103
+ (await db.recall({ entities: ["kim"], asOf: between })).rendered;
104
+ // - [f0] kim: lives in Moscow (2026-08 → 2026-08; closed)
105
+ ```
65
106
 
66
- const res = await db.recall({ query: "runtime?", k: 5 });
67
- res.rendered; // the prompt-ready block
68
- res.facts; // RecalledFact[] — { id, score, entity, recordedAt, … }
107
+ `revise` closed the first fact's interval rather than deleting it, which is why
108
+ the second query has something to answer with.
69
109
 
70
- const card = db.get(out.id);
71
- card.metadata; // Record<string,string> keys sorted, {} when none
110
+ `asOf` moves **both** clocks: a fact answers only if it was valid at that
111
+ instant *and* had already been recorded by then. The second half is the one
112
+ people trip over — an `asOf` earlier than a fact's `recordedAt` sees nothing,
113
+ because the memory genuinely knew nothing then. Answering with today's knowledge
114
+ would be the wrong answer to "what did I hold".
72
115
 
73
- await db.revise(out.id, { text: "prefers async-std" });
74
- await db.link({ src: "user", rel: "works_at", dst: "acme" });
75
- db.unlink({ src: "user", rel: "works_at", dst: "acme" }); // closes the current edge
116
+ `validFrom` is the other half: a statement that became true before you heard of
117
+ it. Recording today that someone moved a week ago closes the previous interval a
118
+ week ago rather than now, so a query as of three days back finds **neither** —
119
+ the old fact had stopped being true, and the new one was not yet known. That is
120
+ not a hole in the model; it is the honest answer for that instant, and it is
121
+ what a single timestamp cannot express.
76
122
 
77
- await db.checkpoint(); // async (see below)
78
- db.close(); // release the file + lock explicitly
123
+ Two more queries over the same axes:
124
+
125
+ ```typescript
126
+ await db.recall({ range: [from, to] }); // what did I record in this window
127
+ await db.recall({ query: "kim", closed: true }); // include closed revisions
79
128
  ```
80
129
 
81
- ## Configuration & embeddings
130
+ Use `revise` when something changed and `forget` only when a fact was simply
131
+ wrong: `forget` destroys the "what was true then" answer, `revise` keeps it.
132
+
133
+ Edges are temporal too, so `asOf` walks the graph as it stood then — through
134
+ relationships that have since been unlinked.
135
+
136
+ ## How recall works
137
+
138
+ Not a vector lookup. Four sources run and are fused by
139
+ [reciprocal-rank fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf)
140
+ with a recency boost; tags filter and are not a source:
82
141
 
83
- The constructor resolves settings **exactly like the CLI and MCP server**: an
84
- explicit `config` path wins, else `$PLUGMEM_CONFIG`, else the platform config
85
- directory, else all defaults. The database path is resolved as an explicit
86
- constructor path, then `$PLUGMEM_DB`, then `[database].path`, then the platform
87
- data directory. See the [full settings reference](https://github.com/m62624/plugmem/blob/main/crates/plugmem-host/SETTINGS.md)
88
- for all fields and OS-specific paths.
142
+ | Source | What it finds |
143
+ |---|---|
144
+ | **Lexical** [BM25](https://en.wikipedia.org/wiki/Okapi_BM25) over a Unicode ([UAX #29](https://unicode.org/reports/tr29/)) tokenizer | exact terms, keyword overlap |
145
+ | **Semantic** — int8-quantized cosine, a flat scan below a threshold and an [HNSW](https://arxiv.org/abs/1603.09320) graph above | meaning, nearest neighbours |
146
+ | **Graph** typed edges walked from the query's anchor entities | relational knowledge |
147
+ | **Temporal** range scans over the `recordedAt` index, plus the validity test | "what was true then", time windows |
148
+
149
+ The sources compose. A query with no `query` string still answers from tags,
150
+ entities and time. **Without an embedder the system is complete** — the other
151
+ three sources need no model, no network and no API key.
152
+
153
+ The result carries both a `rendered` block, selected greedily under a token
154
+ budget and ready to paste, and the structured `facts`/`edges` behind it:
89
155
 
90
156
  ```typescript
91
- const db = await Plugmem.open(undefined, { config: "./plugmem.toml" });
157
+ const res = await db.recall({
158
+ query: "release plans",
159
+ entities: ["ann"], // graph anchors
160
+ tags: ["work"], // filter: a fact must carry all of these
161
+ k: 10, // cap the number of facts
162
+ tokenBudget: 400, // cap the size of the block — your context budget
163
+ graphDepth: 3, // how far to walk from the anchors (default 2)
164
+ });
165
+
166
+ res.rendered; // string, prompt-ready
167
+ res.facts; // { id, score, entity, recordedAt, validFrom, validTo, sources }[]
168
+ res.edges; // { src, rel, dst, provenance }[] — what the graph walked
169
+ res.truncated; // true if selection stopped at k or the budget with more left
92
170
  ```
93
171
 
94
- ```toml
95
- # plugmem.toml
96
- [database]
97
- path = "/path/to/memory.plugmem" # optional example
172
+ `remember` returns the new fact's id **plus any live facts it may duplicate or
173
+ contradict**. The engine never merges or deletes on its own — it surfaces the
174
+ tension and your code decides:
98
175
 
99
- [engine]
100
- dim = 768 # embedding size (0 = vectors off)
176
+ ```typescript
177
+ const out = await db.remember({ text: "the user prefers async-std", entity: "user" });
178
+ for (const s of out.similar) {
179
+ // s.id, s.score, s.reason: "LexicalOverlap" | "VectorCosine"
180
+ // → revise it, forget it, or keep both. Your call.
181
+ }
182
+ ```
101
183
 
102
- [embedder] # optional — omit for lexical/tag/graph/time only
103
- kind = "ollama" # or openai / lmstudio / vllm / llamacpp
104
- url = "http://localhost:11434/v1/embeddings"
105
- model = "nomic-embed-text"
184
+ ## API
185
+
186
+ Every method wraps the identically-named verb of the Rust `Database`; this layer
187
+ only moves arguments and results across the boundary.
188
+
189
+ **Writing** — all return promises:
190
+
191
+ | Method | Does |
192
+ |---|---|
193
+ | `remember(args)` | store a fact; resolves with its id and similar facts |
194
+ | `rememberMany(args[])` | store a batch: one embedding round-trip, one journal sync |
195
+ | `revise(id, args)` | close a fact and record its successor |
196
+ | `forget(id)` | tombstone a fact; resolves with whether it was live |
197
+ | `link(args)` | upsert a typed edge, optionally with `provenance` |
198
+ | `unlink(args)` | close the current edge; resolves with whether one was open |
199
+
200
+ **Reading** — synchronous ones touch mapped memory and return in microseconds:
201
+
202
+ | Method | Does |
203
+ |---|---|
204
+ | `recall(args?)` | ranked, fused, token-budgeted result (async) |
205
+ | `get(id)` | one fact's full card, or `null` (sync) |
206
+ | `tagsOf(id)` | that fact's tags (sync) |
207
+ | `stats()` | engine size counters (sync) |
208
+ | `path()` | the file this handle resolved to (sync) |
209
+ | `export()` | every open fact as one array (async, unbounded — see below) |
210
+ | `exportPage(cursor?)` | the same data in bounded pages of 128 (async) |
211
+ | `exportEdges(onBatch)` | every current edge, streamed in batches (async) |
212
+ | `configWarnings()` | anything in `config.toml` nothing claimed (sync) |
213
+
214
+ **Upkeep** — all async, all on a worker thread:
215
+
216
+ | Method | Does |
217
+ |---|---|
218
+ | `maintain(mode?)` | `"auto"` (default), `"compact"`, `"reindex-text"`, `"optimize-vectors"`, `"full"`. No mode ever drops a revision or an edge version |
219
+ | `checkpoint()` | flush the journal into a fresh snapshot |
220
+ | `verify()` | full content-integrity sweep; rejects on the first inconsistency |
221
+ | `scrub(options?)` | start a resumable byte-level check of the snapshot |
222
+ | `recover(src, dst, options?)` | module function: salvage a damaged file into a clean copy |
223
+
224
+ **Read-only handles** (`{ readOnly: true }`) observe another process's writer
225
+ over a published snapshot. The read verbs answer, the write verbs throw, and two
226
+ more appear: `generation()` (the pinned snapshot number) and `refresh()` (adopt
227
+ the writer's latest checkpoint, returning whether a newer one existed).
228
+
229
+ `close()` releases the file and its lock; every verb afterwards throws, and
230
+ calling it twice is a no-op.
231
+
232
+ ### Bringing your own embedding
233
+
234
+ `remember`, `revise`, `rememberMany` and `recall` all take an optional `vector`
235
+ whose length must equal the configured `dim`. Given one, it **replaces** the
236
+ embedder for that call — nothing is sent to the provider:
237
+
238
+ ```typescript
239
+ const own = await myEmbedder(text);
240
+ await db.remember({ text, vector: own });
241
+ const res = await db.recall({ query: text, vector: own });
242
+ ```
243
+
244
+ Use it for vectors you already have, for a model that is not an OpenAI-shaped
245
+ HTTP endpoint, or for a deterministic test with no network.
246
+
247
+ ### Backing up: facts are only half of it
248
+
249
+ `export`/`exportPage` dump facts. An **edge** is a statement between two
250
+ entities — `kim -works_on-> plugmem` — and belongs to no single fact, so a dump
251
+ of facts alone loses the graph. `exportEdges` is the other half.
252
+
253
+ It streams: the walk runs on a worker and hands your callback one batch at a
254
+ time, so memory stays flat whether the graph has ten edges or ten million. When
255
+ a callback is slower than the walk, the *worker* waits — never the event loop.
256
+
257
+ ```javascript
258
+ const edges = [];
259
+ const count = await db.exportEdges((batch) => edges.push(...batch));
260
+ ```
261
+
262
+ `count` is `2` here, and `edges` is complete the moment the promise resolves —
263
+ no extra tick needed:
264
+
265
+ ```json
266
+ [
267
+ { "src": "kim", "rel": "works_on", "dst": "plugmem", "provenance": 0 },
268
+ { "src": "kim", "rel": "reports_to", "dst": "ann" }
269
+ ]
106
270
  ```
107
271
 
108
- With an `[embedder]`, a text-only `remember`/`recall` **auto-embeds** the
109
- provider's HTTP call runs outside the engine lock. Without one, there is no
110
- embedder and vector recall is skipped (lexical, tag, graph and time recall still
111
- answer). The optional `dim` open option sets the embedding size when there is no
112
- config; if the config configured an embedder, its dimension governs and `dim`
113
- must agree. A `{ readOnly: true }` handle cannot auto-embed inside the engine —
114
- embedding into a zero-copy mapping is what read-only exists to avoid — so this
115
- binding embeds the query itself before the read, exactly as the CLI and the MCP
116
- server do. A text `recall` therefore reaches the vector source in both modes.
117
-
118
- ## The verbs
119
-
120
- Every method here is the identically-named `plugmem-host` `Database` verb; the
121
- engine logic is entirely the host's.
122
-
123
- **Writer** (default): `remember`, `rememberMany`, `recall`, `revise(id, args)`,
124
- `forget(id)`, `link`, `unlink`, `get(id)`, `tagsOf(id)`, `stats`, `export`,
125
- `exportPage(cursor?)`, `verify`, and the async maintenance verbs below.
126
- **Read-only** (`{ readOnly: true }`, observing another process's writer):
127
- `recall`, `get`, `tagsOf`, `stats`, `export`, `exportPage(cursor?)`, `verify`,
128
- plus `generation()` (the pinned snapshot generation) and `refresh()` (advance
129
- to the writer's latest checkpoint); the write verbs throw.
130
- **Both**: `path()` the file the handle resolved to, which is the only way to
131
- learn it when the constructor was given no path.
132
-
133
- ### Errors
272
+ `provenance` is the fact the edge follows from, when it was recorded with one.
273
+ It is **absent** rather than zero when there is none, so it can never be
274
+ mistaken for fact `0` as the second edge above shows.
275
+
276
+ ### Checking a file has not rotted
277
+
278
+ `verify()` and `scrub()` ask different questions, and neither replaces the other:
279
+
280
+ - **`verify()`** does the *content* agree with itself? Text is valid UTF-8,
281
+ each vector belongs to its fact, both directions of every edge match.
282
+ - **`scrub()`** — are the *bytes* the ones that were written? It recomputes each
283
+ section's checksum and the whole-file hash. This is what catches a flipped bit
284
+ that the structure happily accepts.
285
+
286
+ A scrub is paced by you rather than run in one go, so it stays affordable on a
287
+ live database the model ZFS uses. Each step checks up to a budget's worth of
288
+ bytes and returns:
289
+
290
+ ```javascript
291
+ const scrub = await db.scrub(); // default budget: 1 MiB per step
292
+ let step;
293
+ while ((step = await scrub.next()) !== null) {
294
+ // step.doneBytes of step.totalBytes progress through the snapshot file
295
+ }
296
+ ```
297
+
298
+ `next()` returns a promise because a step reads from disk, not because hashing
299
+ is slow: over a memory-mapped file the bytes are paged in as they are read, so a
300
+ step is I/O of whatever length your storage takes. On the JS thread that would
301
+ freeze the process.
302
+
303
+ Two things to know. **Holding the object holds a lock** on the snapshot
304
+ generation it is scanning, so run it to completion or `close()` it. And it is
305
+ one-shot: after it returns `null`, or throws, `active()` is `false` and you ask
306
+ the database for another.
307
+
308
+ ```javascript
309
+ const partial = await db.scrub({ budget: 16 * 1024 });
310
+ await partial.next(); // { doneBytes: 16384, totalBytes: <the file's size> }
311
+ partial.close(); // released; further next() calls return null
312
+ ```
313
+
314
+ Damage rejects with `PLUGMEM_ENGINE` naming what failed its checksum.
315
+
316
+ ### Repairing a damaged file
317
+
318
+ `recover` is a module function, not a method: it works on **paths**, and takes
319
+ the source's exclusive lock, so close your handle first.
320
+
321
+ ```javascript
322
+ import { recover } from "plugmem";
323
+
324
+ const report = await recover("memory.plugmem", "repaired.plugmem");
325
+ // { kept: 1, droppedText: 0, droppedVector: 0, droppedMetadata: 0 }
326
+ ```
327
+
328
+ **The source is never written.** It stays exactly as it was, as evidence; this
329
+ produces a repaired copy beside it, and swapping them is your decision. `dst`
330
+ must therefore be a different path — passing the same one throws.
331
+
332
+ The three `dropped` counts are the damage: each is a fact the source could not
333
+ produce intact. All zero means the image was content-clean and this was a
334
+ compaction. Memory stays proportional to the record count rather than the file,
335
+ so a database far larger than RAM can be recovered.
336
+
337
+ It handles **content** damage — the kind `verify()` reports. A snapshot whose
338
+ container will not parse at all is not salvageable here; that is what a backup
339
+ is for.
340
+
341
+ ### The one thing the Rust library has and this does not
342
+
343
+ `import`. The JSONL dump format is defined by
344
+ [`plugmem-cli`](https://docs.rs/plugmem-cli/latest), not by the engine — there is
345
+ no `import` verb to mirror. A Node program holding records already has
346
+ `rememberMany` and `link`, which is what an importer is made of.
347
+
348
+ ## Errors
134
349
 
135
350
  Every failure plugmem itself decides carries a stable `code`, so a program
136
351
  branches on it instead of on wording:
@@ -149,153 +364,269 @@ try {
149
364
  `PLUGMEM_INVALID_NAME` from an argument that was refused; `PLUGMEM_CLOSED`,
150
365
  `PLUGMEM_READ_ONLY`, `PLUGMEM_WRITER_ONLY` and `PLUGMEM_BUSY` from calling a
151
366
  verb the handle cannot serve; `PLUGMEM_ENGINE` from the engine itself, carrying
152
- the host's own message.
367
+ its own message.
153
368
 
154
369
  The code is there whether the verb threw or the promise rejected — the two are
155
370
  the same contract, so nothing has to be handled twice.
156
371
 
157
- An argument that shapes an answer is refused rather than dropped: `range` must
158
- be exactly `[from, to]`, and `range`, `asOf` and `validFrom` must each be a
159
- finite, non-negative instant. Silently ignoring one produced an answer computed
160
- without it indistinguishable from a correct one.
372
+ An argument that would shape an answer is refused rather than dropped: `range`
373
+ must be exactly `[from, to]`, and `range`, `asOf` and `validFrom` must each be a
374
+ finite, non-negative instant. Silently ignoring one produces an answer computed
375
+ without it, indistinguishable from a correct one.
161
376
 
162
- ### What the host has and this does not
377
+ ## Configuration and embeddings
163
378
 
164
- The list above is the whole supported `Plugmem` surface. The only host
165
- operation intentionally kept out of this boundary is path-level recovery:
379
+ Settings resolve from an explicit `config` path, then `$PLUGMEM_CONFIG`, then
380
+ the platform config directory, then defaults. The database path resolves from an
381
+ explicit argument, then `$PLUGMEM_DB`, then `[database].path`, then the platform
382
+ data directory.
166
383
 
167
- | host verb | boundary note |
168
- |---|---|
169
- | `recover` | salvaging a damaged file is a path-level operation on the disk the process is running on — [`plugmem-cli recover`](https://docs.rs/plugmem-cli/latest)'s job, like `import` and `scrub`. |
170
- | `remember_many` | Exposed as async `rememberMany(items)`. It writes a batch with one embedding round-trip and resolves with outcomes in input order. |
171
- | `export_each` | Exposed as pull-based `exportPage(cursor?)`. Each Promise returns at most 128 facts and releases the native read lock before JS processes them; this gives Node backpressure without a cross-thread callback. |
172
- | `tags_of` | Exposed as synchronous `tagsOf(id)`, returning one fact's tags or an empty array. |
384
+ ```typescript
385
+ const db = await Plugmem.open(undefined, { config: "./plugmem.toml" });
386
+ ```
387
+
388
+ ```toml
389
+ # plugmem.toml
390
+ [database]
391
+ path = "/path/to/memory.plugmem"
173
392
 
174
- **No `import` verb** either — bulk-loading a `backup.jsonl` reads a file on disk,
175
- which is the CLI's job. A Node host can use `rememberMany` for bounded batches
176
- when it already owns the input records.
393
+ [engine]
394
+ dim = 768 # embedding size (0 = vectors off)
177
395
 
178
- ## Many memories in one directory (optional)
396
+ [recall] # optional every key has a tuned default
397
+ w_vec = 2.0 # trust meaning over keywords in this memory
398
+ half_life_days = 30 # and treat anything older than a month as stale
179
399
 
180
- **Default: one memory, one file.** `Plugmem.open(path)` and nothing here applies.
400
+ [embedder] # optional omit for lexical/tag/graph/time only
401
+ kind = "ollama" # or openai / lmstudio / vllm / llamacpp
402
+ url = "http://localhost:11434/v1/embeddings"
403
+ model = "nomic-embed-text"
181
404
 
182
- A process that serves many independent memories — one per conversation, per
183
- tenant, per project can point at a directory and address them by name:
405
+ [maintenance]
406
+ fsync = "each_op" # or "on_snapshot": faster, loses the journal tail on an OS crash
407
+ ```
184
408
 
185
- ```ts
186
- import { Workspace, type DbEntry } from "plugmem";
409
+ `[engine]` is what a database is *built* with; changing one of those on an
410
+ existing file is refused. `[recall]` and `[index]` are the opposite — reopening
411
+ with different weights is how you change the ranking, so tune them freely. All
412
+ of them are in the [full settings reference](https://github.com/m62624/plugmem/blob/main/crates/plugmem-host/SETTINGS.md).
187
413
 
188
- const ws = new Workspace("/srv/memories", { maxOpen: 16, idleTimeoutMs: 60_000 });
414
+ ### When a key is misspelled
189
415
 
190
- // `open` hands back the same `Plugmem` class, so a named memory has exactly the
191
- // verbs a path-opened one has. A first write to an unused name creates it.
192
- await (await ws.open("chat-42")).remember({ text: "prefers tokio" });
416
+ Unknown keys and sections do not stop anything, but they are not swallowed
417
+ either a misspelled `w_vec` changes no behaviour, and silence would leave you
418
+ believing you had tuned something. **Read them once after opening**, because a
419
+ native addon has nowhere sensible to print:
193
420
 
194
- // Do not know the name? Ask what each memory is for. Owners are searchable too,
195
- // even though an owner is a graph edge rather than text.
196
- await ws.describe("chat-42", { description: "release planning", owner: "ann" });
197
- const hits: DbEntry[] = await ws.find("release planning"); // [{ db: "chat-42", … }]
421
+ ```javascript
422
+ const db = await Plugmem.open("agent.plugmem", { config: "./plugmem.toml" });
423
+ for (const warning of db.configWarnings()) console.warn(warning);
424
+ // unknown setting [recall].w_vector did you mean `w_vec`?
198
425
  ```
199
426
 
200
- A name is `[a-z0-9][a-z0-9_-]*` and **cannot represent a path**, so it resolves
201
- to exactly one file inside the directory traversal is not filtered out, it is
202
- unconstructible. `open(name, false)` refuses a name that does not exist yet,
203
- which is what a read should do so a typo is diagnosed rather than answered with
204
- an empty result.
427
+ With an `[embedder]`, a text-only `remember`/`recall` embeds automatically, and
428
+ the provider's HTTP call happens outside the engine lock. The `dim` open option
429
+ sets the embedding size when there is no config; if the config built an
430
+ embedder, its dimension governs and `dim` must agree.
431
+
432
+ A read-only handle cannot embed inside the engine — writing into a zero-copy
433
+ mapping is exactly what read-only exists to avoid — so this binding embeds the
434
+ query itself before the read. A text `recall` reaches the vector source in both
435
+ modes.
436
+
437
+ The [full settings reference](https://github.com/m62624/plugmem/blob/main/crates/plugmem-host/SETTINGS.md)
438
+ lists every field and the OS-specific paths.
439
+
440
+ ## Async and the event loop
205
441
 
206
- The pool bounds how many stay open; `closeIdle()` releases the rest. Call it on
207
- a timer: an open memory holds the file's **exclusive lock**, so a long-running
208
- process that never let go would make its memories unreachable from anything else
209
- on the machine. That is what the idle timeout is for — liveness, not memory.
442
+ Node runs all JavaScript on one thread, so a native call that waits on an
443
+ embedder's HTTP round trip or an fsync would freeze every timer, socket and
444
+ callback in the process. Anything that can do that runs on a libuv worker and
445
+ returns a promise instead.
210
446
 
211
- Two things to know before building on it. Memories are **independent**: nothing
212
- searches across them and no entity links between them, so a fact filed in the
213
- wrong one is unreachable from the other rather than merely misplaced. And **who
214
- may reach which memory is not this package's responsibility** — the name comes
215
- from your code, so put the policy there.
447
+ Promises: `Plugmem.open`, `remember`, `rememberMany`, `revise`, `recall`,
448
+ `forget`, `link`, `unlink`, `export`, `exportPage`, `verify`, `maintain`,
449
+ `checkpoint`, and every `Workspace` verb except `closeIdle`, `openCount` and
450
+ `close`.
216
451
 
217
- Every `Workspace` verb that touches a database returns a promise — `open`,
218
- `list`, `entries`, `find`, `describe`, `archive`, `reindex`, `verify` because
219
- the registry is itself a plugmem memory and a named memory is a real file being
220
- opened. `closeIdle()` and `openCount()` stay synchronous.
452
+ Synchronous: `path`, `get`, `stats`, `tagsOf`, `generation`, `refresh`, `close`.
453
+ These touch mapped memory and return in microseconds, where a promise would be
454
+ pure ceremony.
221
455
 
222
- `reindex()` and `verify()` return promises: they open and read every memory in
223
- the directory, which is not work for the main thread.
456
+ Arguments are still checked on your thread: a refused one **throws** at the call
457
+ site rather than rejecting later, so a mistake in your code and a failure in the
458
+ engine never arrive the same way.
224
459
 
225
- ## Async and concurrency
460
+ ### Two costs a promise does not hide
226
461
 
227
- Operations with unbounded storage or batch work use napi-rs `AsyncTask`: they
228
- return a **`Promise`** and run on Node's **libuv** worker pool, keeping the event
229
- loop available for application code. This includes `rememberMany`,
230
- `exportPage`, `maintain`, `checkpoint`, `reindex` and `verify`.
462
+ **The worker pool is shared, and it has four threads by default.** libuv runs
463
+ `fs`, `dns.lookup`, `zlib` and `crypto.pbkdf2` on the same pool this addon uses.
464
+ The event loop stays free either way, but four concurrent plugmem tasks fill the
465
+ default pool and everything else queues behind them. Measured on one machine, a
466
+ 4 MiB `fs.readFile` in the same process:
231
467
 
232
- For a bounded export, call `exportPage()` once, process its `facts`, then pass
233
- `nextCursor` to the next call until it is absent:
468
+ | | `fs.readFile` |
469
+ |---|---|
470
+ | idle pool | 1.7 ms |
471
+ | 4 plugmem tasks in flight | 29 577 ms |
472
+ | the same, `UV_THREADPOOL_SIZE=8` | 1.4 ms |
473
+
474
+ plugmem's tasks are unusually long — `maintain('full')` is minutes on a large
475
+ memory — so raise `UV_THREADPOOL_SIZE` if the process does anything else with
476
+ libuv while maintenance runs.
477
+
478
+ The pool is also the ceiling on **concurrent embedding**. With an `[embedder]`
479
+ configured, each `remember`/`recall` occupies one worker for its HTTP round
480
+ trip, so at the default four, four is as parallel as it gets. Against a mock
481
+ provider with a fixed 100 ms latency, 16 concurrent recalls took 404 ms on the
482
+ default pool and 101 ms at `UV_THREADPOOL_SIZE=16` — the same 16 requests, four
483
+ waves or one. If a process issues many concurrent recalls against a remote
484
+ provider, size the pool for that, not for the CPU.
485
+
486
+ **`export()` builds its whole result on your thread.** The scan is on a worker,
487
+ but every fact becomes a JavaScript object during the promise's resolution, and
488
+ that part is main-thread work by definition. On 100 000 facts it holds the
489
+ thread for about 244 ms of the call's 289 ms. `exportPage()` over the same
490
+ memory holds it for **0 ms**, in 128-fact pages:
234
491
 
235
492
  ```ts
236
493
  let cursor: number | undefined;
237
494
  do {
238
495
  const page = await db.exportPage(cursor);
239
- for (const fact of page.facts) {
240
- await destination.write(fact);
241
- }
496
+ for (const fact of page.facts) await destination.write(fact);
242
497
  cursor = page.nextCursor;
243
498
  } while (cursor !== undefined);
244
499
  ```
245
500
 
246
- The Promise is the completion boundary for that page: no callback remains
247
- queued, and no database lock is held while the loop body runs. Each page has at
248
- most 128 facts. A read-only handle pages one immutable checkpoint; when paging a
249
- writer for a snapshot-style backup, do not mutate it between calls.
501
+ Each promise owns exactly one page and resolves only after its native scan
502
+ completed; no database lock is held while your loop body runs. A writer may
503
+ change between pages, so do not mutate it during a snapshot-style dump a
504
+ read-only handle pages one immutable checkpoint and is stable.
250
505
 
251
- `rememberMany(items)` performs one batch embedding pass and one journal sync,
252
- then resolves with outcomes in input order. A maintenance call may also return
253
- a no-op report when there is nothing to purge, reindex or optimize.
506
+ ### Concurrency
254
507
 
255
- `maintain(mode?)` takes `"auto"` (the default), `"compact"`, `"reindex-text"`,
256
- `"optimize-vectors"` or `"full"`. No mode ever drops a fact revision or an edge
257
- version; the heavier ones buy bytes and index freshness. `"full"` is the only
258
- one that repacks the edge arenas, which a relink-heavy workload fragments.
508
+ A `Plugmem` handle is safe to use from anywhere in your process. Reads run
509
+ concurrently; writes serialize behind the engine's lock for the microseconds
510
+ they take. A second *process* opening the same file for writing is refused with
511
+ `PLUGMEM_LOCKED` rather than corrupting it, while any number of read-only
512
+ handles map the same file at once — a writer and its readers coexist across
513
+ processes, sharing the OS page cache.
259
514
 
260
- ### What is async, and why
515
+ ## Many memories in one directory
261
516
 
262
- `remember`, `revise`, `recall`, `rememberMany`, `exportPage`, `maintain` and
263
- `checkpoint` return promises. Everything else — `get`, `stats`, `tagsOf`,
264
- `forget`, `link`, `unlink`, `verify`, `export`, `path`, `generation`,
265
- `refresh` — is synchronous.
517
+ **Default: one logical memory backed by a local database layout.** `Plugmem.open(path)` and nothing here
518
+ applies.
266
519
 
267
- The line is drawn at blocking work. Node runs all JavaScript on **one** thread,
268
- so a native call that waits on an embedder's HTTP round trip or on an fsync
269
- freezes every timer, socket and callback in the process for as long as it takes.
270
- The verbs above can do exactly that — with an `[embedder]` configured,
271
- `remember` and a text `recall` each cost a request to the provider — so they run
272
- on a libuv worker and hand JavaScript a promise. The rest touch only mapped
273
- memory and return in microseconds, where a promise would be pure ceremony.
520
+ The problem this solves: a process serving many conversations, tenants or
521
+ projects wants each to have its **own** memory nothing from one surfacing in
522
+ another without managing a pile of file paths by hand. Give a name, get a
523
+ memory:
274
524
 
275
- Arguments are still checked on your thread: a refused one **throws** at the call
276
- site rather than rejecting later, so a mistake in your code and a failure in the
277
- engine never arrive the same way.
525
+ ```ts
526
+ import { Workspace, type DbEntry } from "plugmem";
278
527
 
279
- ### Bringing your own embedding
528
+ const ws = new Workspace("/srv/memories");
280
529
 
281
- `remember`, `revise`, `rememberMany` and `recall` all take an optional
282
- `vector` a precomputed embedding whose length must equal the configured `dim`:
530
+ // The same `Plugmem` class comes back, so a named memory has exactly the verbs
531
+ // a path-opened one has. A first write to an unused name creates it.
532
+ const chat = await ws.open("chat-42");
533
+ await chat.remember({ text: "prefers tokio" });
283
534
 
284
- ```typescript
285
- const own = await myEmbedder(text); // your model, your pipeline
286
- await db.remember({ text, vector: own }); // nothing is sent to `[embedder]`
287
- const res = await db.recall({ query: text, vector: own });
535
+ // Another name is another memory. They cannot see each other.
536
+ const other = await ws.open("chat-99");
537
+ (await other.recall({ query: "tokio" })).facts.length; // 0
538
+ ```
539
+
540
+ Memories are **independent by design**: nothing searches across them and no
541
+ entity links between them. A fact filed under the wrong name is not merely
542
+ misplaced, it is unreachable from the other memory.
543
+
544
+ If you do not know the name, ask what each memory is for. Descriptions are
545
+ searchable, and so are owners, even though an owner is stored as a graph edge
546
+ rather than as text:
547
+
548
+ ```ts
549
+ await ws.describe("chat-42", { description: "release planning", owner: "ann" });
550
+
551
+ const hits: DbEntry[] = await ws.find("release planning"); // → [{ db: "chat-42", … }]
552
+ const byOwner: DbEntry[] = await ws.find("ann"); // → the same memory
288
553
  ```
289
554
 
290
- Given one, it **replaces** the embedder for that call the engine embeds only
291
- when the field is absent. Use it for vectors you already have, for a model that
292
- is not an OpenAI-shaped HTTP endpoint, or for a deterministic test with no
293
- network. The CLI (`--vector`) and the MCP tools (`vector`) take the same thing.
555
+ A name is `[a-z0-9][a-z0-9_-]*` and **cannot express a path**, so it resolves to
556
+ exactly one named database inside the directory traversal is not filtered out, it is
557
+ unconstructible. `ws.open(name, false)` refuses a name that does not exist yet,
558
+ which is what a read should do so a typo is diagnosed rather than answered with
559
+ an empty result.
560
+
561
+ **Who may reach which memory is not this package's job.** The name comes from
562
+ your code, so the policy belongs there.
563
+
564
+ ### Running it
565
+
566
+ | Method | Does |
567
+ |---|---|
568
+ | `open(name, create?)` | open (default: create if missing) and hand back a `Plugmem` |
569
+ | `list()` | every memory in the directory, from the filesystem — including undescribed ones |
570
+ | `entries()` | every described memory, from the registry |
571
+ | `find(query, k?)` | memories whose description or owner best matches |
572
+ | `describe(name, args)` | record what a memory is for; revises rather than duplicating |
573
+ | `archive(name)` | label it archived, keeping its description. Nothing is moved or deleted |
574
+ | `reindex()` | rebuild the registry from the memories' own descriptions |
575
+ | `verify()` | report disagreements between registry and directory; repairs nothing |
576
+ | `closeIdle()` | close memories unused past the idle timeout (sync) |
577
+ | `openCount()` | how many are open right now (sync) |
578
+ | `close()` | close every pooled memory and the registry |
579
+
580
+ `closeIdle()` matters more than it looks. An open memory holds its file's
581
+ exclusive lock, so a long-running process that never lets go makes its memories
582
+ unreachable from anything else on the machine. Call it on a timer — that is what
583
+ the idle timeout is for, liveness rather than memory. The pool bounds how many
584
+ stay open at once (`maxOpen`, default 16, least-recently-used closed to make
585
+ room):
586
+
587
+ ```ts
588
+ const ws = new Workspace("/srv/memories", { maxOpen: 16, idleTimeoutMs: 60_000 });
589
+ setInterval(() => ws.closeIdle(), 30_000);
590
+ ```
591
+
592
+ A `Plugmem` handed out by `open()` is **not** closed by `ws.close()`: it is its
593
+ own handle holding its own lock until you close it or it is garbage collected.
594
+
595
+ `verify()` reports and never repairs, because a workspace is a directory a
596
+ person can edit, and guessing at their intent is how a consistency check loses
597
+ data.
598
+
599
+ ## What it is not for
600
+
601
+ plugmem is for local-first application and agent memory: one process, one local database,
602
+ no service to operate. Its design centre is around 100 000 active facts on one machine, and
603
+ the benchmarks track 1M-operation profiles to show how the same engine behaves
604
+ under heavier local load.
605
+
606
+ It is **not** a vector database and not built for multi-million vector
607
+ workloads, cluster sharding, multi-tenant serving or managed nearest-neighbour
608
+ search. For those, use a dedicated system — [Qdrant](https://qdrant.tech),
609
+ [Milvus](https://milvus.io), [Weaviate](https://weaviate.io),
610
+ [Pinecone](https://www.pinecone.io) or
611
+ [pgvector](https://github.com/pgvector/pgvector).
612
+
613
+ ## Other ways in
614
+
615
+ The same engine ships four ways. This package is the Node one.
616
+
617
+ | You are | Use |
618
+ |---|---|
619
+ | writing JavaScript / TypeScript for Node | **this package** |
620
+ | writing Rust | [`plugmem-host`](https://docs.rs/plugmem-host/latest) — the engine in your process |
621
+ | an agent, or another language | [`plugmem-mcp`](https://docs.rs/plugmem-mcp/latest) — a stdio JSON-RPC sidecar |
622
+ | a person at a terminal | [`plugmem-cli`](https://docs.rs/plugmem-cli/latest) |
294
623
 
295
- `close()` releases the file and its lock; every verb afterwards throws, and it is
296
- idempotent (the handle is also released on garbage collection, but `close()`
297
- makes the moment explicit e.g. before reopening the same file read-only).
624
+ **Working with an LLM agent?** There is a companion
625
+ [skill](https://github.com/m62624/plugmem/blob/main/skill/SKILL.md) describing
626
+ the remember/recall loop, the contradiction workflow and the verbs. This package
627
+ ships it: `skill()` returns the text and `skillVersion()` the version it was
628
+ written against.
298
629
 
299
630
  ## License
300
631
 
301
- MIT.
632
+ MIT. Source: <https://github.com/m62624/plugmem>
package/index.d.ts CHANGED
@@ -86,6 +86,28 @@ export interface RecallArgs {
86
86
  k?: number
87
87
  /** Include closed revisions (default false). */
88
88
  closed?: boolean
89
+ /**
90
+ * Token budget of the `rendered` block (default 512). That block is what
91
+ * goes into a prompt, so this is the knob deciding how much of the
92
+ * context window a recall may spend.
93
+ */
94
+ tokenBudget?: number
95
+ /**
96
+ * HNSW beam width for the vector source (default: the configured
97
+ * `hnsw_ef_search`). Higher is more accurate and slower; ignored while
98
+ * the engine is still in the flat regime, below `flat_to_hnsw`.
99
+ */
100
+ ef?: number
101
+ /**
102
+ * How many edges the graph source may follow from an anchor entity
103
+ * (default: the configured `graph_depth`). `0` asks for the anchors' own
104
+ * facts and no neighbours.
105
+ *
106
+ * Per call for the same reason `k` and `tokenBudget` are: how wide a net
107
+ * to cast belongs to the question. "What is known around this person"
108
+ * wants more hops than "what is this person's stated preference".
109
+ */
110
+ graphDepth?: number
89
111
  /**
90
112
  * A precomputed embedding. Its length must equal the configured `dim`.
91
113
  *
@@ -104,6 +126,59 @@ export interface LinkArgs {
104
126
  rel: string
105
127
  /** Destination entity name. */
106
128
  dst: string
129
+ /**
130
+ * The fact this edge follows from, recorded on the edge and returned by
131
+ * graph recall — the answer to "why is this edge here". Ignored by
132
+ * `unlink`, which closes an edge rather than opening one.
133
+ */
134
+ provenance?: number
135
+ }
136
+ /** Options for [`recover`]. */
137
+ export interface RecoverOptions {
138
+ /**
139
+ * Embedding dimension, as `OpenOptions.dim`. It must match what the source
140
+ * database stores, so a memory with vectors needs this (or a `config`
141
+ * naming the embedder) or the salvage cannot open it.
142
+ */
143
+ dim?: number
144
+ /** Path to a `config.toml`; the standard discovery applies when omitted. */
145
+ config?: string
146
+ }
147
+ /**
148
+ * Salvages a content-corrupt memory: reads `src`, drops the facts that fail
149
+ * the content checks, and writes a clean, compacted image to `dst`.
150
+ *
151
+ * **`src` is never written.** The damaged file stays exactly as it was, as
152
+ * evidence — this produces a repaired copy beside it, and swapping them is
153
+ * your decision, not this function's. `dst` must therefore be a different
154
+ * path.
155
+ *
156
+ * **It handles *content* damage**, the kind `verify` reports: text that is not
157
+ * valid UTF-8, a vector slot that disagrees with its fact, metadata that will
158
+ * not decode. *Structural* damage — a snapshot whose container will not parse
159
+ * at all — is not salvageable here and rejects; that is what a backup is for.
160
+ *
161
+ * Memory stays proportional to the record count rather than the image: the
162
+ * two large pools are streamed through temp files, so a database far bigger
163
+ * than RAM can be recovered as long as its graph fits.
164
+ *
165
+ * **Async**: it opens, sweeps and rewrites a whole database.
166
+ *
167
+ * @throws `PLUGMEM_LOCKED` if either path is open elsewhere; `PLUGMEM_ENGINE`
168
+ * if `dst` resolves to the same file as `src`, or the source will not parse;
169
+ * `PLUGMEM_OPEN` for an IO failure on either path.
170
+ */
171
+ export declare function recover(src: string, dst: string, options?: RecoverOptions | undefined | null): Promise<RecoverReport>
172
+ /** Options for `Plugmem#scrub`. */
173
+ export interface ScrubOptions {
174
+ /**
175
+ * Bytes to hash per step, at most. Default: the engine's own (1 MiB).
176
+ *
177
+ * This is the knob trading responsiveness for throughput: smaller steps
178
+ * return to JavaScript more often, larger ones spend less time crossing
179
+ * back and forth. It changes no answer, only the grain.
180
+ */
181
+ budget?: number
107
182
  }
108
183
  /** One similar / potentially-conflicting live fact surfaced by `remember`. */
109
184
  export interface Similar {
@@ -226,6 +301,12 @@ export interface FactSnapshot {
226
301
  }
227
302
  /** One exported fact — the id-free, import-ready shape. */
228
303
  export interface ExportedFact {
304
+ /**
305
+ * The fact's id in the database it came from. Informational: an import
306
+ * assigns fresh ids. Present because edges name their provenance fact by
307
+ * id, so a dump carrying edges needs something for them to point at.
308
+ */
309
+ id: number
229
310
  /** The fact text. */
230
311
  text: string
231
312
  /** Subject entity name, if any. */
@@ -239,6 +320,52 @@ export interface ExportedFact {
239
320
  /** Validity start (unix ms; preserved on import). */
240
321
  validFrom: number
241
322
  }
323
+ /**
324
+ * One exported edge — the shape `exportEdges` streams, and the same fields the
325
+ * CLI's JSONL dump writes for an edge.
326
+ *
327
+ * Edges are not part of a fact's dump: a fact names its tags and metadata, but
328
+ * an edge is a statement *between* two entities and outlives any single fact.
329
+ * That is why a complete backup is the two streams together.
330
+ */
331
+ export interface ExportedEdge {
332
+ /** Source entity name. */
333
+ src: string
334
+ /** The relation, verbatim. */
335
+ rel: string
336
+ /** Destination entity name. */
337
+ dst: string
338
+ /**
339
+ * The fact this edge follows from, if it was recorded with one. Absent
340
+ * rather than a sentinel, so "no provenance" cannot be mistaken for fact 0.
341
+ */
342
+ provenance?: number
343
+ }
344
+ /**
345
+ * What a `recover` salvaged, and what it had to leave behind.
346
+ *
347
+ * The three `dropped` counts are the damage: each is a fact the source could
348
+ * not produce intact, so a non-zero total means the recovered copy is smaller
349
+ * than the original claimed to be. Zeroes across the board mean the image was
350
+ * content-clean and this was a compaction.
351
+ */
352
+ export interface RecoverReport {
353
+ /** Facts written to the destination — the survivors. */
354
+ kept: number
355
+ /** Dropped: the stored text was not valid UTF-8. */
356
+ droppedText: number
357
+ /** Dropped: the vector slot was out of range or disagreed with the fact. */
358
+ droppedVector: number
359
+ /** Dropped: the metadata blob did not decode to a well-formed map. */
360
+ droppedMetadata: number
361
+ }
362
+ /** How far a `Scrub` has got. */
363
+ export interface ScrubProgress {
364
+ /** Bytes checksummed so far; equals `totalBytes` on the last step. */
365
+ doneBytes: number
366
+ /** The generation file's length — the total to checksum. */
367
+ totalBytes: number
368
+ }
242
369
  /** One bounded page returned by `exportPage`. */
243
370
  export interface ExportPage {
244
371
  /** Open facts in fact-id order; never longer than the native page bound. */
@@ -474,6 +601,18 @@ export declare class Plugmem {
474
601
  * snapshot (`PLUGMEM_NEEDS_CHECKPOINT`), or on an IO error.
475
602
  */
476
603
  static open(path?: string | undefined | null, options?: OpenOptions | undefined | null): Promise<Plugmem>
604
+ /**
605
+ * What `config.toml` said that nothing claimed — a misspelled key, a
606
+ * misspelled section — one human-readable line each, empty when the file
607
+ * was clean.
608
+ *
609
+ * It is a value rather than a printed warning because a native addon has
610
+ * nowhere sensible to print: stderr belongs to the host application, which
611
+ * may be a server that logs as JSON. **Read it once after opening and log
612
+ * it your own way** — ignoring it puts you back where a typo silently
613
+ * changes nothing.
614
+ */
615
+ configWarnings(): Array<string>
477
616
  /**
478
617
  * The file this memory is open on.
479
618
  *
@@ -549,10 +688,14 @@ export declare class Plugmem {
549
688
  /**
550
689
  * Every currently-open fact, as one array (id-free, import-ready).
551
690
  *
552
- * **Async, but still unbounded**: the scan is off the JS thread, yet the
553
- * whole memory is materialized into a single array before it resolves, so
554
- * the peak memory is the whole export. `exportPage` is the same data in
555
- * bounded pages prefer it for anything but a small memory or a script.
691
+ * **Async, but still unbounded, in two ways.** The scan runs on a worker,
692
+ * yet the whole memory is materialized into a single array before it
693
+ * resolves, so peak memory is the whole export. And `resolve` runs on the
694
+ * JS thread by definition, so building one JavaScript object per fact
695
+ * stalls it: measured at ~244 ms of a 289 ms call over 100 000 facts,
696
+ * against 0 ms for the same data through `exportPage`. A promise hides
697
+ * neither cost. Prefer `exportPage` for anything but a small memory or a
698
+ * script.
556
699
  */
557
700
  export(): Promise<ExportedFact[]>
558
701
  /**
@@ -566,6 +709,50 @@ export declare class Plugmem {
566
709
  * mutate it during a snapshot-style backup; a read-only handle is stable.
567
710
  */
568
711
  exportPage(cursor?: number | undefined | null): Promise<ExportPage>
712
+ /**
713
+ * Streams every current edge to `onBatch`, in batches, and resolves with
714
+ * the number streamed.
715
+ *
716
+ * **The other half of a backup.** `export`/`exportPage` dump facts; an edge
717
+ * is a statement between two entities and belongs to no single fact, so a
718
+ * dump of facts alone silently loses the graph.
719
+ *
720
+ * **Async and bounded.** The walk runs on a libuv worker; batches reach
721
+ * `onBatch` on the JS thread through a threadsafe function with a queue of
722
+ * four. When the callback is slower than the walk, the *worker* waits — the
723
+ * event loop never does, and peak memory is four batches whatever the
724
+ * database's size.
725
+ *
726
+ * **When the promise resolves, every batch has been delivered.** Worth
727
+ * saying because a threadsafe-function call only queues work for the JS
728
+ * thread, so the natural implementation resolves early and hands you a
729
+ * half-filled accumulator on some runs and a full one on others. Each
730
+ * batch is acknowledged from the JS thread and the worker waits for the
731
+ * receipts, so reading your results straight after the `await` is correct.
732
+ *
733
+ * `onBatch` throwing is not caught here: it surfaces as an uncaught error,
734
+ * as it would from any callback Node invokes on your behalf.
735
+ */
736
+ exportEdges(onBatch: (edges: ExportedEdge[]) => void): Promise<number>
737
+ /**
738
+ * Starts a resumable byte-level check of the current published snapshot,
739
+ * and resolves with the [`Scrub`](crate::scrub::Scrub) that paces it.
740
+ *
741
+ * The counterpart to `verify`, not a replacement: `verify` asks whether
742
+ * the *content* agrees with itself, this asks whether the *bytes* are the
743
+ * ones that were written. Only the second catches a flipped bit that the
744
+ * structure accepts.
745
+ *
746
+ * Available on a writer as well as a read-only handle: a scrub reads the
747
+ * published generation from disk and has no opinion about which handle
748
+ * asked. It does need something published — checkpoint once first.
749
+ *
750
+ * **The returned object holds a shared lock** on the generation it scans
751
+ * for as long as it lives, so run it to completion or `close()` it.
752
+ *
753
+ * @throws `PLUGMEM_NEEDS_CHECKPOINT` when nothing has been published yet.
754
+ */
755
+ scrub(options?: ScrubOptions | undefined | null): Promise<Scrub>
569
756
  /** One fact's tags, or an empty array for an unknown or tombstoned id. */
570
757
  tagsOf(id: number): Array<string>
571
758
  /**
@@ -610,6 +797,46 @@ export declare class Plugmem {
610
797
  */
611
798
  close(): void
612
799
  }
800
+ /**
801
+ * A resumable byte-level check of one snapshot generation.
802
+ *
803
+ * **Holding this object holds a lock.** It pins the generation it is scanning
804
+ * with a shared lock for its whole life, so the writer's garbage collection
805
+ * cannot reclaim that generation under it. Finish the scan, or `close()` it —
806
+ * do not park one indefinitely.
807
+ *
808
+ * One-shot: once it has returned `null`, or thrown, it is done. Ask the
809
+ * database for another to scan again.
810
+ */
811
+ export declare class Scrub {
812
+ /**
813
+ * Hashes the next slice and resolves with the progress so far, or `null`
814
+ * when the scan is complete.
815
+ *
816
+ * **Async**: see the module note — a step is disk I/O, not arithmetic.
817
+ *
818
+ * Concurrent calls are serialized rather than refused: the cursor has one
819
+ * position, so two overlapping steps would be two different slices of the
820
+ * same scan, which is what a caller pacing it from two places asked for.
821
+ *
822
+ * @throws on the first mismatch, naming what failed its checksum. The
823
+ * scrub is finished at that point; a further call resolves `null`.
824
+ */
825
+ next(): Promise<ScrubProgress | null>
826
+ /**
827
+ * Releases the pinned generation now, abandoning an unfinished scan.
828
+ *
829
+ * Idempotent, and the same bargain as `Plugmem#close`: waiting for the
830
+ * garbage collector to do it means the writer cannot reclaim that
831
+ * generation until it happens.
832
+ */
833
+ close(): void
834
+ /**
835
+ * Whether this scrub still has work — `false` once it finished, failed or
836
+ * was closed. Cheap: it inspects the handle, not the file.
837
+ */
838
+ active(): boolean
839
+ }
613
840
  /**
614
841
  * A directory of named memories — the napi mirror of
615
842
  * [`plugmem_host::Workspace`].
package/index.js CHANGED
@@ -310,9 +310,11 @@ if (!nativeBinding) {
310
310
  throw new Error(`Failed to load native binding`)
311
311
  }
312
312
 
313
- const { Plugmem, MaintainMode, Workspace, version, about, settingsHelp, skill, skillFull, skillVersion } = nativeBinding
313
+ const { Plugmem, recover, Scrub, MaintainMode, Workspace, version, about, settingsHelp, skill, skillFull, skillVersion } = nativeBinding
314
314
 
315
315
  module.exports.Plugmem = Plugmem
316
+ module.exports.recover = recover
317
+ module.exports.Scrub = Scrub
316
318
  module.exports.MaintainMode = MaintainMode
317
319
  module.exports.Workspace = Workspace
318
320
  module.exports.version = version
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "plugmem",
3
- "version": "0.4.0",
4
- "description": "Native Node.js addon for plugmem: an embedded long-term memory engine for LLM agents (remember / recall / revise / forget over one local file).",
3
+ "version": "0.6.0",
4
+ "description": "Native Node.js addon for plugmem: an embedded bitemporal memory and retrieval engine for local-first applications and agents (remember / recall / revise / forget over one local database).",
5
5
  "repository": {
6
6
  "type": "git",
7
7
  "url": "git+https://github.com/m62624/plugmem.git"
@@ -41,11 +41,11 @@
41
41
  "typecheck": "tsc -p tsconfig.json"
42
42
  },
43
43
  "optionalDependencies": {
44
- "plugmem-linux-x64-gnu": "0.4.0",
45
- "plugmem-linux-arm64-gnu": "0.4.0",
46
- "plugmem-darwin-x64": "0.4.0",
47
- "plugmem-darwin-arm64": "0.4.0",
48
- "plugmem-win32-x64-msvc": "0.4.0",
49
- "plugmem-win32-arm64-msvc": "0.4.0"
44
+ "plugmem-linux-x64-gnu": "0.6.0",
45
+ "plugmem-linux-arm64-gnu": "0.6.0",
46
+ "plugmem-darwin-x64": "0.6.0",
47
+ "plugmem-darwin-arm64": "0.6.0",
48
+ "plugmem-win32-x64-msvc": "0.6.0",
49
+ "plugmem-win32-arm64-msvc": "0.6.0"
50
50
  }
51
51
  }