limbic 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -0
- package/README.md +507 -0
- package/dist/index.cjs +2021 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +891 -0
- package/dist/index.d.ts +891 -0
- package/dist/index.js +1939 -0
- package/dist/index.js.map +1 -0
- package/package.json +83 -0
package/README.md
ADDED
|
@@ -0,0 +1,507 @@
|
|
|
1
|
+
# limbic
|
|
2
|
+
|
|
3
|
+
[](https://github.com/Redrum624/limbic/releases)
|
|
4
|
+
[](https://github.com/Redrum624/limbic/releases/latest)
|
|
5
|
+

|
|
6
|
+

|
|
7
|
+

|
|
8
|
+

|
|
9
|
+
|
|
10
|
+
> **limbic** — *the brain circuitry where emotion and memory meet.*
|
|
11
|
+
|
|
12
|
+
**A portable memory engine for LLM agents.** limbic gives an agent a memory that
|
|
13
|
+
behaves like one: importance-weighted scoring, per-category forgetting curves,
|
|
14
|
+
emotional salience, retrieval that refuses to return five paraphrases of the
|
|
15
|
+
same fact, and LLM extraction of new memories from conversation. It is a
|
|
16
|
+
TypeScript port of the memory subsystem of a private production engine by the
|
|
17
|
+
same author ("the origin engine"), and the port is proved rather than claimed:
|
|
18
|
+
cross-language golden fixtures pin the scoring and the diversity selection, and
|
|
19
|
+
the suite re-verifies them on every run.
|
|
20
|
+
|
|
21
|
+
Every TypeScript memory option today is either a cloud-service client, a
|
|
22
|
+
framework store with no memory model (namespace/key JSON with get/put/search),
|
|
23
|
+
or an engine whose defaults phone home to a hosted embedding API. limbic is the
|
|
24
|
+
other thing: **local-first, LLM-agnostic, zero required runtime dependencies.**
|
|
25
|
+
The only network call it can make is to an Ollama host *you* configure; the only
|
|
26
|
+
LLM it can call is the `complete` function *you* inject. There is no telemetry
|
|
27
|
+
and no default cloud provider.
|
|
28
|
+
|
|
29
|
+
## Why limbic
|
|
30
|
+
|
|
31
|
+
- **Zero required runtime dependencies.** `better-sqlite3`, `node-llama-cpp` and
|
|
32
|
+
`@huggingface/transformers` are optional peers behind dynamic `import()`, each
|
|
33
|
+
absence reported with an install hint, never a crash at module load.
|
|
34
|
+
- **A real memory model.** Four-channel scoring (recency, importance, relevance,
|
|
35
|
+
emotion), half-life decay per category with importance floors, and GIST
|
|
36
|
+
diversity selection (arXiv:2405.18754) — not a similarity top-`k` with extra
|
|
37
|
+
steps.
|
|
38
|
+
- **Parity is measured, not asserted.** All 22 of divsel's golden diversity
|
|
39
|
+
cases pass with *exact* numeric agreement; the origin engine's scoring fixture
|
|
40
|
+
passes at its own absolute `1e-6`; four deliberate rule mutations are graded
|
|
41
|
+
in-repo and reproduce the reference's published failure counts. See
|
|
42
|
+
[Parity](#parity).
|
|
43
|
+
- **Nothing at the edges is fatal.** An embedder that is down costs the cosine
|
|
44
|
+
channel, not the write and not the turn. Diversity that throws degrades to
|
|
45
|
+
top-`k` by score.
|
|
46
|
+
- **Lifecycle-complete.** `limbic.close()` releases whatever the engine holds —
|
|
47
|
+
the SQLite handle, a loaded GGUF model, an ONNX session — and is idempotent.
|
|
48
|
+
- **Dual ESM/CJS**, types for both, strict TypeScript with
|
|
49
|
+
`noUncheckedIndexedAccess`, Node ≥ 20.
|
|
50
|
+
|
|
51
|
+
## Install
|
|
52
|
+
|
|
53
|
+
**Not yet on npm.** `GET https://registry.npmjs.org/limbic` returned HTTP 404 on
|
|
54
|
+
2026-09-02 — the name is unclaimed and no publish has run. Until then, install
|
|
55
|
+
from a checkout:
|
|
56
|
+
|
|
57
|
+
```sh
|
|
58
|
+
git clone https://github.com/Redrum624/limbic.git
|
|
59
|
+
cd limbic
|
|
60
|
+
npm ci # installs dev deps; the prepare script builds dist/ for you
|
|
61
|
+
npm test # 311 passing
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Consume it from another project in any of the usual ways — the `prepare` script
|
|
65
|
+
means all three produce a built `dist/`:
|
|
66
|
+
|
|
67
|
+
```sh
|
|
68
|
+
npm i /path/to/limbic # path dependency (dist/ built by npm ci above)
|
|
69
|
+
npm pack /path/to/limbic && npm i limbic-0.1.0.tgz # tarball
|
|
70
|
+
npm i github:Redrum624/limbic # git dependency; npm runs prepare in a temp clone
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Once published: `npm i limbic`. The optional peers, each unlocking one feature:
|
|
74
|
+
|
|
75
|
+
```sh
|
|
76
|
+
npm i better-sqlite3 # + SQLite persistence (SqliteStore)
|
|
77
|
+
npm i node-llama-cpp # + in-process GGUF embeddings
|
|
78
|
+
npm i @huggingface/transformers # + transformers.js embeddings
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
## Quickstart
|
|
82
|
+
|
|
83
|
+
### Pure in-memory — no dependencies at all
|
|
84
|
+
|
|
85
|
+
```ts
|
|
86
|
+
import { createLimbic } from "limbic";
|
|
87
|
+
|
|
88
|
+
const limbic = createLimbic();
|
|
89
|
+
|
|
90
|
+
await limbic.remember("User's name is Ada", { category: "personal_fact", importance: 0.9 });
|
|
91
|
+
await limbic.remember("User is allergic to shellfish", { category: "health", importance: 0.95 });
|
|
92
|
+
|
|
93
|
+
const hits = await limbic.retrieve("what should I avoid cooking?", 3);
|
|
94
|
+
// [{ memory: { content: "User is allergic to shellfish", ... }, score: 0.5825 }, ...]
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
That runs with no model, no network and nothing on disk, and the `0.5825` is
|
|
98
|
+
what the snippet actually returns (verified against the built package,
|
|
99
|
+
2026-09-02): with no embedder the cosine channel is **MISSING** — which is not
|
|
100
|
+
the same as a similarity of `0`, and is handled as such throughout — so the
|
|
101
|
+
score is `0.25 × recency + 0.35 × importance` with a relevance and emotion
|
|
102
|
+
channel of zero.
|
|
103
|
+
|
|
104
|
+
### With an embedder
|
|
105
|
+
|
|
106
|
+
```ts
|
|
107
|
+
import { OllamaEmbedder, createLimbic } from "limbic";
|
|
108
|
+
|
|
109
|
+
const limbic = createLimbic({
|
|
110
|
+
embedder: new OllamaEmbedder({ host: "http://127.0.0.1:11434", model: "nomic-embed-text" }),
|
|
111
|
+
});
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
`OllamaEmbedder` POSTs `{ model, input }` to `{host}/api/embed` and reads
|
|
115
|
+
`json.embeddings`; the legacy `/api/embeddings` is deliberately unsupported.
|
|
116
|
+
Prefer the IPv4 literal over `localhost`: on a dual-stack host where Ollama
|
|
117
|
+
binds IPv4 only, Node's resolver can hand back `::1`. Or fully in-process, no
|
|
118
|
+
server at all:
|
|
119
|
+
|
|
120
|
+
```ts
|
|
121
|
+
import { TransformersEmbedder, createLimbic } from "limbic";
|
|
122
|
+
|
|
123
|
+
const limbic = createLimbic({ embedder: new TransformersEmbedder() });
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
```ts
|
|
127
|
+
import { NodeLlamaCppEmbedder, SqliteStore, createLimbic } from "limbic";
|
|
128
|
+
|
|
129
|
+
const limbic = createLimbic({
|
|
130
|
+
store: await SqliteStore.open("./memories.db"),
|
|
131
|
+
embedder: new NodeLlamaCppEmbedder({ modelPath: "/models/nomic-embed-text-v1.5.Q4_K_M.gguf" }),
|
|
132
|
+
});
|
|
133
|
+
// ...
|
|
134
|
+
await limbic.close(); // releases the SQLite handle and the loaded model
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Full scripts: [`examples/ollama-companion.ts`](examples/ollama-companion.ts) and
|
|
138
|
+
[`examples/node-llama-cpp.ts`](examples/node-llama-cpp.ts) (run with
|
|
139
|
+
`npx tsx examples/<name>.ts` — `tsx` is fetched by npx, it is not a
|
|
140
|
+
devDependency).
|
|
141
|
+
|
|
142
|
+
### Extraction needs a `complete`
|
|
143
|
+
|
|
144
|
+
```ts
|
|
145
|
+
const limbic = createLimbic({
|
|
146
|
+
// Your LLM call. limbic ships no provider SDK and never will.
|
|
147
|
+
complete: async (prompt) => callYourModel(prompt),
|
|
148
|
+
});
|
|
149
|
+
const extracted = await limbic.extract(conversation); // throws without a complete
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
The extractor's output is untrusted model text: it is parsed defensively and
|
|
153
|
+
`importance`/`confidence` are clamped, but review what you `remember()` from it.
|
|
154
|
+
A user turn can steer what the model extracts — that is inherent to LLM
|
|
155
|
+
extraction and documented in `src/extraction.ts`.
|
|
156
|
+
|
|
157
|
+
## API
|
|
158
|
+
|
|
159
|
+
### `createLimbic(options?)`
|
|
160
|
+
|
|
161
|
+
| Option | Default | Meaning |
|
|
162
|
+
|---|---|---|
|
|
163
|
+
| `store` | `new MemStore()` | Any `MemoryStore`. `SqliteStore` is built in behind an optional peer. |
|
|
164
|
+
| `embedder` | none | Any `Embedder`. Absent means keyword-only scoring and no vectors stored. |
|
|
165
|
+
| `complete` | none | Any `CompleteFn`. Absent means `extract()` **throws** rather than silently returning `[]`. |
|
|
166
|
+
| `weights` | `DEFAULT_WEIGHTS` | `{ recency: 0.25, importance: 0.35, relevance: 0.25, emotion: 0.15 }` |
|
|
167
|
+
| `lambda` | `0.5` | GIST's diversity weight. divsel's own default is `1.0` — see the bench. |
|
|
168
|
+
| `pool` | `50` | How many scored rows to diversify over. |
|
|
169
|
+
|
|
170
|
+
Returns a `Limbic` handle: `remember` / `extract` / `retrieve` / `decayPass` /
|
|
171
|
+
`close` / `store`.
|
|
172
|
+
|
|
173
|
+
### Exports, exhaustively
|
|
174
|
+
|
|
175
|
+
Everything `import { ... } from "limbic"` can name, grouped as `src/index.ts`
|
|
176
|
+
groups them. Types are marked *(type)*.
|
|
177
|
+
|
|
178
|
+
**Engine** (`src/index.ts`)
|
|
179
|
+
|
|
180
|
+
| Name | What it is |
|
|
181
|
+
|---|---|
|
|
182
|
+
| `createLimbic` | Build an engine from the options above. |
|
|
183
|
+
| `Limbic` *(type)* | The handle: `remember`, `extract`, `retrieve`, `decayPass`, `close`, `store`. |
|
|
184
|
+
| `LimbicOptions` *(type)* | What `createLimbic` accepts. |
|
|
185
|
+
| `FADE_THRESHOLD` | `0.05` — below this strength `decayPass` deletes the row. |
|
|
186
|
+
|
|
187
|
+
**Core types** (`src/types.ts`)
|
|
188
|
+
|
|
189
|
+
| Name | What it is |
|
|
190
|
+
|---|---|
|
|
191
|
+
| `Memory` *(type)* | The stored record: content, category, importance, keywords, timestamps, optional embedding and emotion. |
|
|
192
|
+
| `ExtractedMemory` *(type)* | What the extractor emits before anything is saved. |
|
|
193
|
+
| `MemoryCategory` *(type)* | The category union (`personal_fact`, `health`, …) — open to unknown strings. |
|
|
194
|
+
| `MemoryEmotion` *(type)* | `{ label, intensity }` — the pair the emotion channel scores. |
|
|
195
|
+
| `Embedder` *(type)* | `{ model, embed(texts) => Float32Array[] }` — the seam every embedder fills. |
|
|
196
|
+
| `CompleteFn` *(type)* | `(prompt, opts?) => Promise<string>` — your LLM. |
|
|
197
|
+
| `ScoreWeights` *(type)* | The four channel weights. |
|
|
198
|
+
| `DEFAULT_WEIGHTS` | `{ recency: 0.25, importance: 0.35, relevance: 0.25, emotion: 0.15 }`. |
|
|
199
|
+
| `EMBED_BLEND` | `0.3` — the cosine share of the final score when both vectors exist. |
|
|
200
|
+
|
|
201
|
+
**Stores** (`src/store.ts`, `src/stores/sqlite.ts`)
|
|
202
|
+
|
|
203
|
+
| Name | What it is |
|
|
204
|
+
|---|---|
|
|
205
|
+
| `MemoryStore` *(type)* | The store seam: `save`, `get`, `all`, `search`, `delete`, `updateAccess`, `count`. |
|
|
206
|
+
| `MemStore` | The zero-dependency in-memory default. Unbounded until you schedule `decayPass`. |
|
|
207
|
+
| `SqliteStore` | SQLite persistence behind the `better-sqlite3` peer; `open()`, `close()`, streaming `decayCandidates()`. |
|
|
208
|
+
| `MISSING_SQLITE_PEER` | The install-hint message thrown when the peer is absent. |
|
|
209
|
+
| `DEFAULT_ALL_LIMIT` | `200` — `all()`'s default row cap. |
|
|
210
|
+
| `DecayCandidate` *(type)* | The scalar slice of a row `decayPass` reads (no embedding materialised). |
|
|
211
|
+
|
|
212
|
+
**Scoring** (`src/internal/scoring.ts`)
|
|
213
|
+
|
|
214
|
+
| Name | What it is |
|
|
215
|
+
|---|---|
|
|
216
|
+
| `scoreMemory` | `(memory, query: ScoreQuery, now, weights?) => number` — the one-number score. Takes `now` explicitly; never reads the wall clock. |
|
|
217
|
+
| `scoreMemoryDetailed` | Same, returning the per-channel `ScoreBreakdown`. |
|
|
218
|
+
| `ScoreQuery` *(type)* | `{ keywords, embedding?, targetEmotion? }` — not a bare string. |
|
|
219
|
+
| `ScoreBreakdown` *(type)* | The four channels, the base, the blend, the final. |
|
|
220
|
+
| `RECENCY_HALF_LIFE_DAYS` | `7` — recency halves weekly. |
|
|
221
|
+
| `EMOTION_HIGH_THRESHOLD` | `0.7` — intensity at or above it earns the full `+0.5` emotion bonus. |
|
|
222
|
+
| `EMOTION_MEDIUM_THRESHOLD` | `0.4` — at or above it, `+0.3`; below, `intensity × 0.3`. |
|
|
223
|
+
|
|
224
|
+
**Decay** (`src/decay.ts`)
|
|
225
|
+
|
|
226
|
+
| Name | What it is |
|
|
227
|
+
|---|---|
|
|
228
|
+
| `calculateDecay` | `(DecayArgs) => number` — strength after decay, rounded half-to-even at 3 dp (Python's `round`). |
|
|
229
|
+
| `DecayArgs` *(type)* | `{ originalStrength, daysSinceCreation, daysSinceAccess, importance, category, accessCount }`. |
|
|
230
|
+
| `CATEGORY_HALF_LIFE_DAYS` | Per-category half-lives: `relationship` 365 d down to `emotion`/`work` 30 d. |
|
|
231
|
+
| `DEFAULT_HALF_LIFE_DAYS` | `60` — for a category not in the table. |
|
|
232
|
+
| `IMPORTANCE_DECAY_FACTOR` | Threshold/factor pairs stretching or shrinking the half-life by importance. |
|
|
233
|
+
| `ACCESS_REINFORCEMENT_DAYS` | `5` — days of half-life added per **recorded** access (see [Decay](#decay)). |
|
|
234
|
+
| `STRENGTH_FLOOR_HIGH` | `0.3` — a memory at importance ≥ 0.8 never falls below this. |
|
|
235
|
+
| `STRENGTH_FLOOR_MEDIUM` | `0.1` — the floor at importance ≥ 0.6. |
|
|
236
|
+
|
|
237
|
+
**Retrieval** (`src/retrieve.ts`)
|
|
238
|
+
|
|
239
|
+
| Name | What it is |
|
|
240
|
+
|---|---|
|
|
241
|
+
| `retrieve` | Score the pool, then diversify it. May return fewer than `k` — see below. |
|
|
242
|
+
| `scorePool` | Score every candidate row; sorted by score. |
|
|
243
|
+
| `diversify` | Run GIST over an already-sorted pool; membership changes, order never does. |
|
|
244
|
+
| `RetrieveOptions` *(type)* | Per-call overrides: `pool`, `lambda`, `weights`, `embedder`, `targetEmotion`, `now`, `diversify`. |
|
|
245
|
+
| `ScoredMemory` *(type)* | `{ memory, score }`. |
|
|
246
|
+
| `DEFAULT_LAMBDA` | `0.5`. |
|
|
247
|
+
| `DEFAULT_POOL` | `50`. |
|
|
248
|
+
|
|
249
|
+
**Diversity** (`src/diversity.ts`)
|
|
250
|
+
|
|
251
|
+
| Name | What it is |
|
|
252
|
+
|---|---|
|
|
253
|
+
| `gistSelect` | GIST selection, ids in, ids out. |
|
|
254
|
+
| `gistSelectFull` | The full result: indices plus `f`, `g`, `div`, `threshold`, `stage`, `dMax`. |
|
|
255
|
+
| `GistSelectOptions` *(type)* | `{ metric?, utility?, exhaustiveThresholds?, diameter?, diameterSweeps? }`. |
|
|
256
|
+
| `GistResult` *(type)* | What `gistSelectFull` returns. |
|
|
257
|
+
| `Metric` / `UtilityKind` / `Utilities` / `DiameterMode` / `Stage` *(types)* | The contract's enums: both metrics, `linear` / `coverage` / `facility_location`, exact and approximate diameter. |
|
|
258
|
+
| `DiversityError` / `DiversityErrorCode` *(type)* | Typed failures; `retrieve` catches them and degrades to top-`k`. |
|
|
259
|
+
| `F32_EPSILON` | `1.1920928955078125e-7` — the `f32` machine epsilon the kernel is pinned to. |
|
|
260
|
+
|
|
261
|
+
**Extraction** (`src/extraction.ts`)
|
|
262
|
+
|
|
263
|
+
| Name | What it is |
|
|
264
|
+
|---|---|
|
|
265
|
+
| `extractFromConversation` | `(complete, conversation) => ExtractedMemory[]` — window, prompt, parse, gate. |
|
|
266
|
+
| `buildExtractionPrompt` | The prompt for a conversation, or `null` when it is too short to bother. |
|
|
267
|
+
| `formatConversation` | The fenced, delimited transcript block spliced into the prompt. |
|
|
268
|
+
| `parseExtractionResponse` | Defensive JSON parse of model output; clamps `importance` to `[0, 1]`. |
|
|
269
|
+
| `passesSaveGate` | `importance >= 0.4 && confidence >= 0.6`. |
|
|
270
|
+
| `categoryFor` | Extraction type → `MemoryCategory`; unknown types map to `general`, not dropped. |
|
|
271
|
+
| `EXTRACTION_PROMPT` | The default prompt. Yours to replace. |
|
|
272
|
+
| `EXTRACTION_TO_CATEGORY` | The type→category table behind `categoryFor`. |
|
|
273
|
+
| `KNOWN_EXTRACTION_TYPES` | The set the origin engine recognises. |
|
|
274
|
+
| `CONVERSATION_WINDOW` | `10` — turns considered. |
|
|
275
|
+
| `MIN_CONVERSATION_CHARS` | `50` — below this, no extraction call at all. |
|
|
276
|
+
| `MIN_IMPORTANCE` / `MIN_CONFIDENCE` | `0.4` / `0.6` — the save gate's two halves. |
|
|
277
|
+
| `ChatTurn` *(type)* | `{ role, content }`. |
|
|
278
|
+
|
|
279
|
+
**Embedders** (`src/embedders/`)
|
|
280
|
+
|
|
281
|
+
| Name | What it is |
|
|
282
|
+
|---|---|
|
|
283
|
+
| `OllamaEmbedder` / `OllamaEmbedderOptions` *(type)* | POST `/api/embed` with a bounded timeout; host validated at construction. |
|
|
284
|
+
| `DEFAULT_OLLAMA_HOST` | `http://127.0.0.1:11434`. |
|
|
285
|
+
| `NodeLlamaCppEmbedder` / `NodeLlamaCppEmbedderOptions` *(type)* | In-process GGUF embeddings; `dispose()` frees the model and context. |
|
|
286
|
+
| `TransformersEmbedder` / `TransformersEmbedderOptions` *(type)* | transformers.js embeddings; `dispose()` releases the ONNX session. |
|
|
287
|
+
| `EmbedderUnavailableError` | Thrown when a peer is missing or a host is unreachable — with the install hint. |
|
|
288
|
+
| `isEmbedderUnavailable` | Type guard for the above. |
|
|
289
|
+
|
|
290
|
+
### Scoring
|
|
291
|
+
|
|
292
|
+
```
|
|
293
|
+
recency = 0.5 ^ (daysSinceLastAccess / 7)
|
|
294
|
+
base = clamp(0.25*recency + 0.35*importance + 0.25*relevance + 0.15*emotion, 0, 1)
|
|
295
|
+
final = base (no comparable vector)
|
|
296
|
+
final = clamp(0.70*base + 0.30*max(0, cosine), 0, 1) (both vectors present)
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
`cosine == null` means **MISSING**, never `0.0` — a port that substituted `0.0`
|
|
300
|
+
would fail the scoring fixture.
|
|
301
|
+
|
|
302
|
+
### Decay
|
|
303
|
+
|
|
304
|
+
`calculateDecay` applies per-category half-lives, an importance factor that
|
|
305
|
+
stretches or shrinks the half-life, and floors so that a memory at
|
|
306
|
+
importance ≥ 0.8 never falls below 0.3. `decayPass()` walks the store —
|
|
307
|
+
streaming scalar slices when the store supports it, so no embedding BLOBs are
|
|
308
|
+
materialised — deletes anything below `FADE_THRESHOLD = 0.05`, and reports
|
|
309
|
+
`{ decayed, faded }`. Schedule it yourself; nothing in the engine runs it for
|
|
310
|
+
you, and `MemStore` grows until something does.
|
|
311
|
+
|
|
312
|
+
**Reinforcement is the caller's contract.** Each *recorded* access adds
|
|
313
|
+
`ACCESS_REINFORCEMENT_DAYS = 5` days of half-life — but nothing in the engine
|
|
314
|
+
records one. `retrieve()` reads rows; it does not touch `accessCount` or
|
|
315
|
+
`lastAccessed`. A hit reinforces a memory only if you call
|
|
316
|
+
`store.updateAccess(id)` for the hits you actually use. Skip that and every
|
|
317
|
+
memory decays as if it were never read.
|
|
318
|
+
|
|
319
|
+
### Retrieval and diversity
|
|
320
|
+
|
|
321
|
+
`retrieve` scores the pool, then hands the rows that carry a vector to GIST
|
|
322
|
+
(arXiv:2405.18754v3), maximising `g(S) + lambda * div(S)`. Three properties are
|
|
323
|
+
load-bearing, and each has a test that fails if it breaks:
|
|
324
|
+
|
|
325
|
+
1. **Diversity changes membership, never order.** The pool is sorted by score,
|
|
326
|
+
so a position in it *is* its rank; the result is assembled by index.
|
|
327
|
+
2. **A memory with no embedding is selectable but edge-free.** It can take a
|
|
328
|
+
slot GIST left open and cannot lower the spread.
|
|
329
|
+
3. **The fill never re-admits what the selector rejected.** So `retrieve` can
|
|
330
|
+
return **fewer than `k`** rows. That short return is a signal, not a bug: it
|
|
331
|
+
means `lambda` is too high for this corpus.
|
|
332
|
+
|
|
333
|
+
## Parity
|
|
334
|
+
|
|
335
|
+
limbic does not claim to behave "like" its references; it reproduces their
|
|
336
|
+
committed fixtures, and the suite re-hashes both fixtures on every run
|
|
337
|
+
(`test/fixtures.hash.test.ts`) so a silent edit fails loudly. Full provenance,
|
|
338
|
+
hashes and the one documented metadata exception:
|
|
339
|
+
[`test/fixtures/PROVENANCE.md`](test/fixtures/PROVENANCE.md).
|
|
340
|
+
|
|
341
|
+
### Diversity — divsel's `golden-selection.json`
|
|
342
|
+
|
|
343
|
+
| | |
|
|
344
|
+
|---|---|
|
|
345
|
+
| Fixture | 22 cases, schema 1, generator `divsel 0.1.0`, copied byte-for-byte (`sha256 73713cd2…`) |
|
|
346
|
+
| Rules | divsel `docs/CONFORMANCE.md`, `sha256 829bc087…` (commit `02c546f` on divsel's current `main`) |
|
|
347
|
+
| Cases passing | **22 of 22 — none skipped**, case 20 (approximate diameter, the only optional case) included |
|
|
348
|
+
| Scope | both metrics, all three utilities (`linear`, `coverage`, `facility_location`), exhaustive thresholds, approximate diameter |
|
|
349
|
+
| Numeric agreement | **exact** — worst tolerance-budget share across all 22 cases × 5 numeric fields is **0** |
|
|
350
|
+
|
|
351
|
+
That last row is not a rounding claim. divsel computes every distance in `f32`
|
|
352
|
+
with a fixed 16-accumulator reduction order, and `src/internal/gist.ts`
|
|
353
|
+
reproduces that kernel with `Math.fround` rather than computing in `f64`, so the
|
|
354
|
+
two agree bit-for-bit instead of merely within `1e-6`. The suite asserts it,
|
|
355
|
+
gated at 0.1% of each field's tolerance budget.
|
|
356
|
+
|
|
357
|
+
The tolerance rules are the current ones: divsel's earlier blanket bound was
|
|
358
|
+
measured (by divsel) to fail correct ports 69 times by up to 8.1×, and was
|
|
359
|
+
replaced by a per-primitive `tol(x)`, with `expected_f` bounded as
|
|
360
|
+
`tol(expected_g) + lam*tol(expected_div)` because `f = g + lam*div` is derived.
|
|
361
|
+
|
|
362
|
+
And the reader is not vacuous — that is measured *in this repo*, not quoted.
|
|
363
|
+
`test/diversity.golden.test.ts` builds a driver clone proved field-identical to
|
|
364
|
+
`gistSelectFull` on all 22 cases, injects four deliberate rule mutations, and
|
|
365
|
+
asserts the failure counts CONFORMANCE.md publishes:
|
|
366
|
+
|
|
367
|
+
| Mutation | limbic measures | CONFORMANCE.md says |
|
|
368
|
+
|---|---|---|
|
|
369
|
+
| strict `>` in the sweep fold (rule 2) | 15 of 22 | 15 |
|
|
370
|
+
| argmax ties to the highest index (rule 1) | 9 of 22 | nine |
|
|
371
|
+
| `lambda` doubled | 20 of 22 | 20 |
|
|
372
|
+
| `div(\|S\| ≤ 1) = 0` instead of `d_max` (rule 4) | case 7 alone | case 7 |
|
|
373
|
+
|
|
374
|
+
> **`lambda` defaults to `0.5` in limbic**, matching the origin engine's
|
|
375
|
+
> default. **divsel's own default is `1.0`.** The bench below shows the
|
|
376
|
+
> crossover sitting between them on a clustered corpus, so limbic's default is
|
|
377
|
+
> the conservative one: it diversifies less, not more.
|
|
378
|
+
|
|
379
|
+
### Scoring — the origin engine's `golden-scoring.json`
|
|
380
|
+
|
|
381
|
+
Numeric content copied untouched from the origin engine (two metadata strings
|
|
382
|
+
neutralised for publication — documented, with hashes, in
|
|
383
|
+
[`PROVENANCE.md`](test/fixtures/PROVENANCE.md)); `sha256 89755bf6…` pins the
|
|
384
|
+
published bytes. Clock pinned to the fixture's own `now = 2026-08-26T12:00:00`,
|
|
385
|
+
asserted at the fixture's own **absolute** `1e-6` on `expected_base_score`,
|
|
386
|
+
`expected_final_score` and both ranking arrays. `cosine == null` means
|
|
387
|
+
**MISSING**, never `0.0` — a port that substituted `0.0` would fail the
|
|
388
|
+
`portuguese` row.
|
|
389
|
+
|
|
390
|
+
### Deliberate deviations from the origin engine
|
|
391
|
+
|
|
392
|
+
| limbic | The origin engine | Why |
|
|
393
|
+
|---|---|---|
|
|
394
|
+
| `id: string` | integer autoincrement | limbic must support non-SQL stores and caller-supplied ids. |
|
|
395
|
+
| `extractionType: string` | closed enum | Unknown types map to `general` and are kept; the origin engine drops the row. |
|
|
396
|
+
| Prompt placeholder substituted literally | `str.format(...)` | The origin engine's formatting call raises on its own prompt's JSON example and the exception is swallowed, so its LLM extraction path returns `[]` today. limbic replaces the one `{conversation}` token and leaves the example intact. |
|
|
397
|
+
| `extract()` never writes | writes through when configured to | `remember()` is the only writer. The save gate travels with the data as `passesSaveGate`. |
|
|
398
|
+
| `Memory.emotion?: { label, intensity }` | read from the source conversation | The caller supplies the scored pair; `Memory.feeling` is the extractor's free-text tone and is not what is scored. |
|
|
399
|
+
| `scoreMemory(m, q, now)` | reads the wall clock | An explicit clock is what makes the fixture evaluable. |
|
|
400
|
+
|
|
401
|
+
## Bench
|
|
402
|
+
|
|
403
|
+
`npm run bench` — 60 memories, 12 planted topics × 5 near-paraphrases each,
|
|
404
|
+
dim 48, fixed seed, `k = 8`, clusters counted as connected components at
|
|
405
|
+
`cosine > 0.92`.
|
|
406
|
+
|
|
407
|
+
| strategy | redundancy ↓ | clusters hit | coverage ↑ | facts / 1k prompt chars ↑ |
|
|
408
|
+
|---|---|---|---|---|
|
|
409
|
+
| naive top-`k` | 0.9794 | 2 / 12 | 0.167 | 3.086 |
|
|
410
|
+
| `gistSelect` λ=0 | 0.9794 | 2 / 12 | 0.167 | 3.086 |
|
|
411
|
+
| `gistSelect` λ=0.5 *(default)* | 0.9794 | 2 / 12 | 0.167 | 3.086 |
|
|
412
|
+
| `gistSelect` λ=1 | **0.1387** | **8 / 12** | **0.667** | **12.346** |
|
|
413
|
+
| `gistSelect` λ=2 | 0.1387 | 8 / 12 | 0.667 | 12.346 |
|
|
414
|
+
| `gistSelect` λ=4 | 0.1387 | 8 / 12 | 0.667 | 12.346 |
|
|
415
|
+
| `gistSelect` λ=8 | 0.1387 | 8 / 12 | 0.667 | 12.346 |
|
|
416
|
+
|
|
417
|
+
Above the crossover the same eight slots and the same 648 prompt characters
|
|
418
|
+
carry **four times the distinct facts**, and redundancy falls from "every pick
|
|
419
|
+
has a near-twin" to "almost none does". Cost: a mean **≈3.7 ms** per selection
|
|
420
|
+
at this size against **≈80 ns** for a slice (`npm run bench`, 2026-09-02, one
|
|
421
|
+
machine — expect drift) — GIST runs `2 + |D|` greedy passes, 32 thresholds at
|
|
422
|
+
`eps = 0.1`.
|
|
423
|
+
|
|
424
|
+
**Read the top three rows as the honest half.** At `λ ≤ 0.5` GIST returns
|
|
425
|
+
exactly top-`k` here, and that is *correct*: below the crossover the score given
|
|
426
|
+
up to reach another cluster outweighs the diversity gained. Whether `0.5` is
|
|
427
|
+
right for your corpus depends on how steep your score gradient is across topics.
|
|
428
|
+
Sweep it. And the corpus is **synthetic and n = 1**, built with a cluster
|
|
429
|
+
structure chosen to make redundancy visible — treat the table as a mechanism
|
|
430
|
+
demo, not a general benchmark.
|
|
431
|
+
|
|
432
|
+
## Architecture
|
|
433
|
+
|
|
434
|
+
```
|
|
435
|
+
src/
|
|
436
|
+
index.ts createLimbic — wires the pieces, owns ids and close()
|
|
437
|
+
types.ts Memory, ExtractedMemory, the seams (Embedder, CompleteFn)
|
|
438
|
+
store.ts MemoryStore seam + MemStore (zero-dep default)
|
|
439
|
+
stores/sqlite.ts SqliteStore, behind the better-sqlite3 peer
|
|
440
|
+
retrieve.ts score the pool -> diversify -> fill, preserving spread
|
|
441
|
+
decay.ts calculateDecay — half-lives, importance factor, floors
|
|
442
|
+
extraction.ts prompt, window, defensive parse, save gate
|
|
443
|
+
diversity.ts the public GIST surface + typed errors
|
|
444
|
+
embedders/ ollama, node-llama-cpp, transformers, shared errors
|
|
445
|
+
internal/ scoring, the f32 GIST kernel, vec ops, store shared bits
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
Everything in `src/internal/` is implementation; the curated re-exports in
|
|
449
|
+
`src/index.ts` are the API. One import direction, no cycles: `index` → pipeline
|
|
450
|
+
modules → `internal`.
|
|
451
|
+
|
|
452
|
+
## Development
|
|
453
|
+
|
|
454
|
+
```sh
|
|
455
|
+
npm ci # install + build dist/ (prepare)
|
|
456
|
+
npm test # vitest — 311 passed | 5 skipped, no network
|
|
457
|
+
npm run typecheck # tsc --noEmit
|
|
458
|
+
npm run build # tsup — ESM + CJS + d.ts/d.cts
|
|
459
|
+
npm run bench # the redundancy bench above
|
|
460
|
+
|
|
461
|
+
# Live Ollama integration suite — opt-in, off by default and in CI:
|
|
462
|
+
LIMBIC_LIVE=1 npx vitest run test/embedders.ollama.live.test.ts
|
|
463
|
+
# Optionally LIMBIC_OLLAMA_HOST=http://127.0.0.1:11434 to point it elsewhere.
|
|
464
|
+
```
|
|
465
|
+
|
|
466
|
+
The default suite makes **zero network calls**; the 5 skips are the live suite
|
|
467
|
+
declining to run without `LIMBIC_LIVE=1`. Golden fixtures are pinned by sha256
|
|
468
|
+
and never hand-edited — see [`CONTRIBUTING.md`](CONTRIBUTING.md).
|
|
469
|
+
|
|
470
|
+
## Documentation
|
|
471
|
+
|
|
472
|
+
| File | What is in it |
|
|
473
|
+
|---|---|
|
|
474
|
+
| [`CHANGELOG.md`](CHANGELOG.md) | What shipped in 0.1.0, and the pre-publication hardening after it |
|
|
475
|
+
| [`CONTRIBUTING.md`](CONTRIBUTING.md) | Dev setup, the golden-fixture rule, PR expectations |
|
|
476
|
+
| [`test/fixtures/PROVENANCE.md`](test/fixtures/PROVENANCE.md) | Where both fixtures come from, their hashes, and the rules they are read under |
|
|
477
|
+
| [`examples/`](examples) | Two runnable end-to-end scripts |
|
|
478
|
+
| [`LICENSE`](LICENSE) | Apache-2.0, in full |
|
|
479
|
+
|
|
480
|
+
## Credits
|
|
481
|
+
|
|
482
|
+
- **[divsel](https://github.com/Redrum624/divsel)** — the Rust GIST reference
|
|
483
|
+
implementation by the same author. Its golden fixture and CONFORMANCE rules
|
|
484
|
+
are the contract `gistSelect` reproduces.
|
|
485
|
+
- **GIST** — the diversity-selection algorithm,
|
|
486
|
+
[arXiv:2405.18754](https://arxiv.org/abs/2405.18754).
|
|
487
|
+
- **The origin engine** — a private production engine by the same author; the
|
|
488
|
+
source of the scoring, decay and extraction semantics and of the scoring
|
|
489
|
+
fixture. It stays unnamed here.
|
|
490
|
+
|
|
491
|
+
The limbic system is the brain circuitry where emotion and memory meet. That is
|
|
492
|
+
exactly the scope of this library.
|
|
493
|
+
|
|
494
|
+
## Downloads
|
|
495
|
+
|
|
496
|
+

|
|
497
|
+
|
|
498
|
+
<sub>The curve builds from publish day — the GitHub API keeps no earlier download history.</sub>
|
|
499
|
+
|
|
500
|
+
## License
|
|
501
|
+
|
|
502
|
+
Apache-2.0 © 2026 Redrum624.
|
|
503
|
+
|
|
504
|
+
---
|
|
505
|
+
|
|
506
|
+
**limbic** — *remembers what matters, forgets what doesn't, and never tells you
|
|
507
|
+
the same thing five times.*
|