@alineo-labs/memory 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,394 @@
1
+ # `@alineo-labs/memory`
2
+
3
+ A provider-agnostic memory layer for alineo agents: working memory, semantic recall, and
4
+ episodic recall over the existing ledger — scoped by a durable `resourceId` that survives past
5
+ any one sandbox session.
6
+
7
+ This package owns memory **concepts, scoping, and pipeline** (injection, compaction,
8
+ lifecycle hooks). It does not own a storage backend itself — bring your own
9
+ `IWorkingMemoryProvider` / `ISemanticMemoryProvider`, whether that's the in-memory reference
10
+ implementations shipped here, the file-based `@alineo-labs/sqlite-memory` backend, or the
11
+ shared `@alineo-labs/postgres-memory` backend.
12
+
13
+ ## Why `resourceId`, not `sandboxId`
14
+
15
+ A `sandboxId` identifies one sandbox _session_. A `resourceId` is a durable identity (a user,
16
+ an account, a project) expected to outlive any number of sandbox sessions. Working and
17
+ semantic memory are keyed by `resourceId` because their entire point is surviving past the
18
+ session they were learned in.
19
+
20
+ `resourceId` (and `parentSandboxId`, for forked sandboxes) ride along in the ledger's existing
21
+ `sandbox_created` event payload — the same mechanism `SandboxDetails.runId` already used —
22
+ so there's no ledger schema change. Set it when creating a sandbox:
23
+
24
+ ```ts
25
+ const sb = await client.sandbox({
26
+ image: "node:22",
27
+ resources: { cpu: "500m", memory: "512Mi" },
28
+ resourceId: "user-42", // ties this session's memory to a durable resource
29
+ });
30
+ ```
31
+
32
+ `resume()`, `restoreSnapshot()`, and `sb.fork()` all inherit the originating session's
33
+ `resourceId` automatically, the same way they already inherit `runId`.
34
+
35
+ ```ts
36
+ import type { ResourceRef } from "@alineo-labs/memory";
37
+
38
+ const ref: ResourceRef = { resourceId: "user-42" }; // teamId?: string for shared/team scoping
39
+ ```
40
+
41
+ ## Three capabilities, three independent slots
42
+
43
+ Memory is split by capability, not unified into one adapter — real backends support genuinely
44
+ different capability subsets, so a monolithic interface would force fake implementations of
45
+ capabilities a backend doesn't have.
46
+
47
+ ### Working memory — required
48
+
49
+ Structured per-resource key/value facts. Every `Memory` instance needs at least this.
50
+
51
+ ```ts
52
+ import { Memory, InMemoryWorkingMemoryProvider } from "@alineo-labs/memory";
53
+
54
+ const memory = new Memory({ workingMemory: new InMemoryWorkingMemoryProvider() });
55
+
56
+ await memory.workingMemory.set(ref, "preferredLanguage", "TypeScript");
57
+ await memory.workingMemory.get(ref, "preferredLanguage"); // "TypeScript"
58
+ await memory.workingMemory.list(ref); // { preferredLanguage: "TypeScript" }
59
+ await memory.workingMemory.delete(ref, "preferredLanguage");
60
+ ```
61
+
62
+ ### Semantic memory — optional
63
+
64
+ Vector recall over remembered facts. Omitting it is a first-class, typed state: calling
65
+ `remember()`/`recall()` on a `Memory` with no semantic provider throws `MemoryCapabilityError`.
66
+
67
+ ```ts
68
+ import {
69
+ Memory,
70
+ InMemoryWorkingMemoryProvider,
71
+ InMemorySemanticMemoryProvider,
72
+ } from "@alineo-labs/memory";
73
+
74
+ const memory = new Memory({
75
+ workingMemory: new InMemoryWorkingMemoryProvider(),
76
+ semantic: new InMemorySemanticMemoryProvider(myEmbeddingProvider),
77
+ });
78
+
79
+ await memory.remember(ref, { content: "prefers dark mode" });
80
+ const facts = await memory.recall(ref, "UI preferences", { topK: 5 });
81
+ ```
82
+
83
+ `EmbeddingProvider` is a minimal `{ id, embed(texts) }` shape — pass any embedding model you
84
+ like. `@alineo-labs/model-providers` ships `createNvidiaEmbeddingProvider()`, matching this
85
+ shape structurally with no dependency on this package at all.
86
+
87
+ #### Verified memory
88
+
89
+ Every fact returned by `recall()`/`listAll()` carries a computed `verified` flag: `true` when
90
+ `remember()` was called with a `sourceRef` pointing at a real ledger entry, `false` for a
91
+ free-form fact. This is computed, never caller-set — passing `verified: true` on a free-form
92
+ fact's input is silently ignored:
93
+
94
+ ```ts
95
+ await memory.remember(ref, {
96
+ content: "user confirmed the refund",
97
+ sourceRef: { sandboxId: sb.sandboxId, entryIndex: 42 }, // ties it to a real ledger entry
98
+ });
99
+
100
+ const [fact] = await memory.recall(ref, "refund");
101
+ fact.verified; // true — traceable back to that ledger entry
102
+ ```
103
+
104
+ A fact worth trusting more than a hallucinated summary is one you can point at real execution
105
+ history — no other memory framework can do this because it requires an execution ledger to
106
+ point _at_ in the first place.
107
+
108
+ ### Episodic memory — a function, not a provider
109
+
110
+ A read-shaped view over the sandbox ledger, reshaped by `resourceId`. No new storage.
111
+
112
+ ```ts
113
+ import { episodicRecall } from "@alineo-labs/memory";
114
+
115
+ const entries = await episodicRecall(adapter, ref, { limit: 100 });
116
+
117
+ // Include ancestor sessions reached via sb.fork()'s parentSandboxId chain:
118
+ const withHistory = await episodicRecall(adapter, ref, { branch: "lineage" });
119
+ ```
120
+
121
+ By default, sessions are resolved by matching `SandboxDetails.resourceId` (or, for ledger data
122
+ written before that field existed, by matching the ledger's `name` against `resourceId`).
123
+ Supply `resolveSessions` to use a different convention entirely.
124
+
125
+ #### Branch-true episodic memory
126
+
127
+ `episodicRecall({branch: "lineage"})` flattens fork ancestry into one merged stream — good
128
+ enough for "what led up to this session," but it can't tell you "what happened on a _sibling_
129
+ branch forked from the same point." `episodicTree()` returns the actual fork tree instead:
130
+
131
+ ```ts
132
+ import { episodicTree } from "@alineo-labs/memory";
133
+
134
+ const [root] = await episodicTree(adapter, ref);
135
+ root.entries; // this session's own ledger entries
136
+ root.children; // sessions forked from it — each with its own .entries and .children
137
+ ```
138
+
139
+ Every resolved session's ancestor chain is pulled in automatically (same as `lineage`), so a
140
+ tree with a resolved-but-orphaned node never happens — its ancestors are added as parents even
141
+ if they weren't in the original resolved set.
142
+
143
+ ## Real backends
144
+
145
+ | Package | Working memory | Semantic memory | Notes |
146
+ | ------------------------------------ | ------------------------------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
147
+ | `@alineo-labs/memory` (this package) | `InMemoryWorkingMemoryProvider` | `InMemorySemanticMemoryProvider` | Process-local, non-durable — reference implementations only. |
148
+ | `@alineo-labs/sqlite-memory` | `SQLiteWorkingMemoryProvider` | `SQLiteSemanticMemoryProvider` | File-based via `bun:sqlite`, zero external services, survives restarts. Ranks recall with `sqlite-vec`'s native `vec0` index when the extension loads (verified on win32/x64); falls back to an in-JS cosine scan otherwise — check `.hasVectorIndex`. |
149
+ | `@alineo-labs/postgres-memory` | `PostgresWorkingMemoryProvider` | `PostgresSemanticMemoryProvider` | Shared, multi-process backend. Row-level security isolates `teamId`-scoped rows. Ranks recall with a `pgvector` HNSW index when the extension can be installed; falls back to an in-JS cosine scan otherwise — check `.hasVectorIndex`. |
150
+
151
+ ```ts
152
+ import { Memory } from "@alineo-labs/memory";
153
+ import {
154
+ SQLiteWorkingMemoryProvider,
155
+ SQLiteSemanticMemoryProvider,
156
+ } from "@alineo-labs/sqlite-memory";
157
+ import { createNvidiaEmbeddingProvider } from "@alineo-labs/model-providers";
158
+
159
+ const memory = new Memory({
160
+ workingMemory: new SQLiteWorkingMemoryProvider("./alineo-memory.db"),
161
+ semantic: new SQLiteSemanticMemoryProvider("./alineo-memory.db", createNvidiaEmbeddingProvider()),
162
+ });
163
+ ```
164
+
165
+ Neither backend package has been run against a live database in this repo's own test suite
166
+ (no Postgres instance, and the SQLite one is unit-tested with `bun:sqlite` directly) — they
167
+ ship type-checked and, for SQLite, tested against a real file; treat the Postgres one as
168
+ reviewed-but-unverified until it's run against an actual server.
169
+
170
+ ## Compaction
171
+
172
+ A `remember()`'d fact stays verbatim forever unless pruned. `compactSemanticMemory()` (or
173
+ `Memory.compactSemanticMemory()`) drops old/excess facts, for any provider implementing the
174
+ optional `IPrunableSemanticMemoryProvider` capability (`listAll`/`forget` — all three semantic
175
+ providers above support it):
176
+
177
+ ```ts
178
+ await memory.compactSemanticMemory(ref, { maxFacts: 500, maxAgeMs: 30 * 24 * 60 * 60 * 1000 });
179
+ ```
180
+
181
+ Age-based removal runs first; the count cap is then applied to whatever's left. Throws if the
182
+ configured provider doesn't support pruning.
183
+
184
+ Pass `summarize` to consolidate instead of just deleting — it receives the facts about to be
185
+ removed (oldest first) and returns replacement contents, written _before_ the originals are
186
+ dropped:
187
+
188
+ ```ts
189
+ await memory.compactSemanticMemory(ref, {
190
+ maxFacts: 500,
191
+ summarize: async (facts) => [
192
+ await myLlmCall(`Summarize into one fact: ${facts.map((f) => f.content).join("; ")}`),
193
+ ],
194
+ });
195
+ ```
196
+
197
+ `summarize` is caller-supplied — this package never depends on a concrete model, the same
198
+ principle as `EmbeddingProvider`.
199
+
200
+ ### Automatic compaction
201
+
202
+ Configure `autoCompact` on `Memory` to run a compaction check after every `remember()`, instead
203
+ of remembering to call `compactSemanticMemory()` yourself on a schedule:
204
+
205
+ ```ts
206
+ const memory = new Memory({
207
+ workingMemory: new InMemoryWorkingMemoryProvider(),
208
+ semantic: new InMemorySemanticMemoryProvider(embeddings),
209
+ autoCompact: { maxFacts: 500, checkEvery: 10 }, // check once per 10 remember() calls
210
+ });
211
+ ```
212
+
213
+ Silently skipped if the configured provider doesn't support pruning; a failed compaction
214
+ check fails the `remember()` call it rode along with (not silently swallowed) — raise
215
+ `checkEvery`, or call `compactSemanticMemory()` on a separate schedule instead, to decouple
216
+ the two.
217
+
218
+ ## Context injection
219
+
220
+ `buildContextSnippet(memory, ref, opts)` assembles a plain-text block from a resource's working
221
+ memory and (if a query is given and a semantic provider is configured) its most relevant
222
+ semantic memories — ready to prepend to a prompt:
223
+
224
+ ```ts
225
+ import { buildContextSnippet } from "@alineo-labs/memory";
226
+
227
+ const context = await buildContextSnippet(memory, ref, { query: userMessage, topK: 5 });
228
+ for await (const chunk of agent.prompt(context ? `${context}\n\n${userMessage}` : userMessage)) {
229
+ process.stdout.write(chunk);
230
+ }
231
+ ```
232
+
233
+ This builds the string only — `alineo`'s Pi bridge has no hook today for prepending to a
234
+ session's system prompt automatically, so injecting it into an actual conversation is still a
235
+ call the surrounding app makes.
236
+
237
+ ## Structured working memory
238
+
239
+ `IWorkingMemoryProvider` is raw, untyped key/value. `SchemaWorkingMemory<T>` wraps it with a
240
+ validated, typed profile stored under one key — useful for the common "agent maintains a
241
+ structured user profile" shape:
242
+
243
+ ```ts
244
+ import { z } from "zod";
245
+ import { SchemaWorkingMemory } from "@alineo-labs/memory";
246
+
247
+ const ProfileSchema = z.object({
248
+ name: z.string().optional(),
249
+ preferredLanguage: z.string().optional(),
250
+ });
251
+
252
+ const profile = new SchemaWorkingMemory(workingMemoryProvider, ProfileSchema);
253
+ await profile.update(ref, { preferredLanguage: "TypeScript" }); // merges + validates
254
+ await profile.get(ref); // { preferredLanguage: "TypeScript" }
255
+ ```
256
+
257
+ `SchemaValidator<T>` is a minimal `{ parse(data): T }` shape — any Zod schema satisfies it
258
+ directly; this package never depends on a concrete schema library.
259
+
260
+ ## Agent-callable memory tools
261
+
262
+ `createMemoryTools(memory, ref)` returns a set of tool definitions (name, description, JSON
263
+ Schema parameters, executor) in the shape most agent tool-calling conventions expect — so a
264
+ _model_, not just the surrounding application code, can decide to persist or retrieve a fact
265
+ mid-conversation:
266
+
267
+ ```ts
268
+ import { createMemoryTools } from "@alineo-labs/memory";
269
+
270
+ const tools = createMemoryTools(memory, ref);
271
+ // [{ name: "set_working_memory", ... }, { name: "get_working_memory", ... },
272
+ // { name: "remember_fact", ... }, { name: "recall_facts", ... } — last two only if
273
+ // memory.hasSemanticMemory ]
274
+ ```
275
+
276
+ `alineo`'s Pi bridge doesn't yet expose a way to register caller-defined tools into a running
277
+ Pi session, so wiring these into an actual live agent conversation is left to the caller today
278
+ — this ships the tool _definitions_, ready to adapt into whatever surface ends up supporting
279
+ them.
280
+
281
+ ## Forkable memory
282
+
283
+ Sandboxes already fork copy-on-write via `sb.fork()`; `Memory.fork()` gives memory the same
284
+ property — an independent snapshot copy the child can mutate without ever touching the
285
+ parent's:
286
+
287
+ ```ts
288
+ const {
289
+ ref: childRef,
290
+ workingKeysCopied,
291
+ semanticFactsCopied,
292
+ } = await memory.fork(parentRef, "child-resource-id");
293
+
294
+ await memory.workingMemory.set(childRef, "note", "only visible to the child now");
295
+ await memory.workingMemory.get(parentRef, "note"); // undefined — the parent is untouched
296
+ ```
297
+
298
+ Working memory is always copied in full. Semantic memory is copied only if the configured
299
+ provider supports the pruning capability (`listAll` is what makes enumerating "everything to
300
+ copy" possible) — `semanticFactsCopied` is `0`, not an error, otherwise.
301
+
302
+ `Alineo.spawn()` calls this automatically when the parent agent has `.memory` configured — a
303
+ spawned child (a sandbox-level fork under the hood) gets its own independent memory copy with
304
+ no extra wiring.
305
+
306
+ ## Team access control
307
+
308
+ `ResourceRef.teamId` isolates data _structurally_ in every backend (it's part of the storage
309
+ key), but only `@alineo-labs/postgres-memory` enforces it as an actual access-control boundary
310
+ via row-level security — a caller for the in-memory or SQLite backends who deliberately passes
311
+ the "wrong" `teamId` can still read it. `withTeamAccessControl()` /
312
+ `withTeamAccessControlSemantic()` close that gap for any backend:
313
+
314
+ ```ts
315
+ import { withTeamAccessControl, withTeamAccessControlSemantic } from "@alineo-labs/memory";
316
+
317
+ const checker = { canAccess: (teamId: string) => currentUser.teamIds.includes(teamId) };
318
+
319
+ const memory = new Memory({
320
+ workingMemory: withTeamAccessControl(new SQLiteWorkingMemoryProvider("./mem.db"), checker),
321
+ semantic: withTeamAccessControlSemantic(mySemanticProvider, checker),
322
+ });
323
+ ```
324
+
325
+ Every call on a `ResourceRef` carrying a `teamId` is checked against `checker.canAccess()`
326
+ first, throwing `MemoryAccessDeniedError` before the wrapped provider is ever touched. A `ref`
327
+ with no `teamId` always passes through untouched. Works alongside Postgres's own RLS for
328
+ defense-in-depth — it isn't an either/or.
329
+
330
+ ## Sandbox lifecycle binding
331
+
332
+ `createMemoryLifecycleHooks(memory, ref)` returns a `SandboxHooks` object (composable via
333
+ `@alineo-labs/core`'s `composeHooks()`) that records the most recently active `sandboxId` and
334
+ the most recent checkpoint's metadata into working memory — a durable answer to "what was the
335
+ last checkpoint for this resource" without re-deriving it from the ledger every time. It does
336
+ not restore sandbox state itself; that's `@alineo-labs/core`'s job.
337
+
338
+ ```ts
339
+ import { composeHooks } from "@alineo-labs/core";
340
+ import { createMemoryLifecycleHooks } from "@alineo-labs/memory";
341
+
342
+ const sb = await client.sandbox({
343
+ image: "node:22",
344
+ resources: { cpu: "500m", memory: "512Mi" },
345
+ resourceId: ref.resourceId,
346
+ hooks: composeHooks([createMemoryLifecycleHooks(memory, ref)]),
347
+ });
348
+ ```
349
+
350
+ ## Agent wiring
351
+
352
+ `alineo`'s `Alineo` class accepts an optional `memory` on `load()`/`resume()`/`attach()`, and
353
+ `spawn()` carries it over to the child automatically:
354
+
355
+ ```ts
356
+ const agent = await Alineo.load(spec, { adapter, memory });
357
+ await agent.memory?.remember(agent.resourceRef, { content: "user prefers concise answers" });
358
+ ```
359
+
360
+ `agent.resourceRef` defaults `resourceId` to the agent's own `name` — the same convention
361
+ `episodicRecall()`'s default resolver expects, so no extra wiring is needed to make episodic
362
+ recall work for an agent's own sandbox sessions.
363
+
364
+ `Alineo.spawn()` also calls `memory.fork()` automatically (see "Forkable memory" above) when
365
+ the parent has `.memory` configured — a spawned child gets its own independent memory copy,
366
+ seeded from the parent, with no extra call needed.
367
+
368
+ ## What's intentionally out of scope here
369
+
370
+ - **Pi tool-call integration** — `createMemoryTools()` produces framework-agnostic tool
371
+ definitions; actually registering them into a running Pi session needs a change to
372
+ `alineo`'s Pi bridge that hasn't been made.
373
+ - **Automatic prompt injection** — `buildContextSnippet()` builds the string; nothing calls it
374
+ automatically at session start. That's still an explicit call the surrounding app makes.
375
+ - **A ledger-native branch/lane concept** — `episodicTree()` reconstructs the fork tree from
376
+ `parentSandboxId` at read time; alineo's ledger itself still has no first-class branch
377
+ concept the way e.g. Pi's own session storage does. If the ledger ever grows one,
378
+ `episodicTree()` would become a thinner read over it instead of doing the reconstruction
379
+ itself.
380
+ - **`withTeamAccessControl()` is app-layer, not a security boundary of its own** — it's a
381
+ correct-by-construction gate in front of whatever backend, but the `TeamAccessChecker` you
382
+ supply is doing all the real work; a buggy or bypassed checker is still a hole. It doesn't
383
+ replace real infrastructure-level isolation for a genuinely adversarial multi-tenant setup.
384
+ - Restoring sandbox filesystem/process state from a checkpoint — `createMemoryLifecycleHooks`
385
+ only records checkpoint _metadata_ into working memory; the actual restore is
386
+ `@alineo-labs/core`'s `resume()`.
387
+
388
+ ## Examples & cookbook
389
+
390
+ - [`examples/memory-basics`](https://github.com/DrejT/alineo/tree/main/examples/memory-basics)
391
+ — every capability on this page, demonstrated standalone. No OpenSandbox, no API key.
392
+ - [`cookbooks/persistent-agent-memory`](https://github.com/DrejT/alineo/tree/main/cookbooks/persistent-agent-memory)
393
+ — the real end-to-end scenario: a Pi agent whose memory survives across separate sandbox
394
+ sessions entirely, using `@alineo-labs/sqlite-memory`.